[PATCH 0/4] Fixes for useradd-staticids.bbclass

[PATCH 0/4] Fixes for useradd-staticids.bbclass

[Peter Kjellerstedt]

The main change here is to fix the regression reported in
https://bugzilla.yoctoproject.org/show_bug.cgi?id=9777 which resulted
in setting USERADD_ERROR_DYNAMIC having no effect if the user or group
was not listed at all in a passwd or groups file.

In the ticket discussion there was also a suggestion to allow the
error to be turned into a warning. I have implemented this as well,
whereby one can now set USERADD_ERROR_DYNAMIC to either "error" or
"warn" to decide which behavior one wants. (It is of course still
possible to set it to "1" for backwards compatibility.)

The latter change requires some updates to the documentation, which is
provided in the last commit. This is intended for the yocto-docs
repository.

//Peter

The following changes since commit 4799e1e89fd9b0f8b606e3c502fe89f32d45fe76:

  gtk+3: Add patch for --disable-opengl (2016-06-17 11:17:21 +0100)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib pkj/useradd-staticids-regression
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=pkj/useradd-staticids-regression

Peter Kjellerstedt (4):
  documentation.conf: Add information about USERADD varaibles
  useradd-staticids.bbclass: Restore failure on missing UIDs/GIDs
  useradd-staticids.bbclass: Allow missing UIDs/GIDs to generate
    warnings
  Update the documentation of USERADD_ERROR_DYNAMIC

 documentation/ref-manual/ref-variables.xml | 19 +++++++++++--------
 meta/classes/useradd-staticids.bbclass     | 22 ++++++++++++++++------
 meta/conf/documentation.conf               |  4 ++++
 3 files changed, 31 insertions(+), 14 deletions(-)

-- 
2.9.0

[PATCH 1/4] documentation.conf: Add information about USERADD varaibles

[Peter Kjellerstedt]

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
---
 meta/conf/documentation.conf | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/meta/conf/documentation.conf b/meta/conf/documentation.conf
index c0025f5..38639d5 100644
--- a/meta/conf/documentation.conf
+++ b/meta/conf/documentation.conf
@@ -452,8 +452,12 @@ UBOOT_SUFFIX[doc] = "Points to the generated U-Boot extension."
 UBOOT_TARGET[doc] = "Specifies the target used for building U-Boot."
 USE_DEVFS[doc] = "When building images, specifies to populate or not /dev. This variable defaults to '1' (leave directory empty, surely because devtmpfs do the job). Set it to '0' to use makedevs (or consider using a custom file with IMAGE_DEVICE_TABLES)."
 USER_CLASSES[doc] = "List of additional classes to use when building images that enable extra features."
+USERADD_ERROR_DYNAMIC[doc] = "Forces the OpenEmbedded build system to produce an error if the user identification (uid) and group identification (gid) values are not defined in files/passwd and files/group files."
+USERADD_GID_TABLES[doc] = "Specifies a password file to use for obtaining static group identification (gid) values when the OpenEmbedded build system adds a group to the system during package installation."
 USERADD_PACKAGES[doc] = "When a recipe inherits the useradd class, this variable specifies the individual packages within the recipe that require users and/or groups to be added."
 USERADD_PARAM[doc] = "When a recipe inherits the useradd class, this variable specifies for a package what parameters should be passed to the useradd command if you wish to add a user to the system when the package is installed."
+USERADD_UID_TABLES[doc] = "Specifies a password file to use for obtaining static user identification (uid) values when the OpenEmbedded build system adds a user to the system during package installation."
+USERADDEXTENSION[doc] = "When set to 'useradd-staticids', causes the OpenEmbedded build system to base all user and group additions on a static passwd and group files found in BBPATH."
 
 #W
 
-- 
2.9.0

[PATCH 2/4] useradd-staticids.bbclass: Restore failure on missing UIDs/GIDs

[Peter Kjellerstedt]

A regression was introduced with commit 3149319a whereby setting
USERADD_ERROR_DYNAMIC no longer resulted in an error for users and
groups that were missing numeric UIDs and GIDs but were not mentioned
at all in any passwd or groups file.

[YOCTO #9777]

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
---
 meta/classes/useradd-staticids.bbclass | 19 +++++++++++++------
 1 file changed, 13 insertions(+), 6 deletions(-)

diff --git a/meta/classes/useradd-staticids.bbclass b/meta/classes/useradd-staticids.bbclass
index d963f28..90835cf 100644
--- a/meta/classes/useradd-staticids.bbclass
+++ b/meta/classes/useradd-staticids.bbclass
@@ -50,6 +50,11 @@ def update_useradd_static_config(d):
 
         return id_table
 
+    def handle_missing_id(id, type, pkg):
+        if d.getVar('USERADD_ERROR_DYNAMIC', True) == '1':
+            #bb.error("Skipping recipe %s, package %s which adds %sname %s does not have a static ID defined." % (d.getVar('PN', True),  pkg, type, id))
+            raise bb.build.FuncFailed("%s - %s: %sname %s does not have a static ID defined." % (d.getVar('PN', True), pkg, type, id))
+
     # We parse and rewrite the useradd components
     def rewrite_useradd(params):
         # The following comes from --help on useradd from shadow
@@ -116,6 +121,8 @@ def update_useradd_static_config(d):
                 users = merge_files(get_passwd_list(d), 7)
 
             if uaargs.LOGIN not in users:
+                if not uaargs.uid or not uaargs.uid.isdigit() or not uaargs.gid:
+                    handle_missing_id(uaargs.LOGIN, 'user', pkg)
                 continue
 
             field = users[uaargs.LOGIN]
@@ -165,9 +172,8 @@ def update_useradd_static_config(d):
             uaargs.shell = field[6] or uaargs.shell
 
             # Should be an error if a specific option is set...
-            if d.getVar('USERADD_ERROR_DYNAMIC', True) == '1' and not ((uaargs.uid and uaargs.uid.isdigit()) and uaargs.gid):
-                #bb.error("Skipping recipe %s, package %s which adds username %s does not have a static uid defined." % (d.getVar('PN', True),  pkg, uaargs.LOGIN))
-                raise bb.build.FuncFailed("%s - %s: Username %s does not have a static uid defined." % (d.getVar('PN', True), pkg, uaargs.LOGIN))
+            if not uaargs.uid or not uaargs.uid.isdigit() or not uaargs.gid:
+                 handle_missing_id(uaargs.LOGIN, 'user', pkg)
 
             # Reconstruct the args...
             newparam  = ['', ' --defaults'][uaargs.defaults]
@@ -248,6 +254,8 @@ def update_useradd_static_config(d):
                 groups = merge_files(get_group_list(d), 4)
 
             if gaargs.GROUP not in groups:
+                if not gaargs.gid or not gaargs.gid.isdigit():
+                    handle_missing_id(gaargs.GROUP, 'group', pkg)
                 continue
 
             field = groups[gaargs.GROUP]
@@ -257,9 +265,8 @@ def update_useradd_static_config(d):
                     bb.warn("%s: Changing groupname %s's gid from (%s) to (%s), verify configuration files!" % (d.getVar('PN', True), gaargs.GROUP, gaargs.gid, field[2]))
                 gaargs.gid = field[2]
 
-            if d.getVar('USERADD_ERROR_DYNAMIC', True) == '1' and not (gaargs.gid and gaargs.gid.isdigit()):
-                #bb.error("Skipping recipe %s, package %s which adds groupname %s does not have a static gid defined." % (d.getVar('PN', True),  pkg, gaargs.GROUP))
-                raise bb.build.FuncFailed("%s - %s: Groupname %s does not have a static gid defined." % (d.getVar('PN', True), pkg, gaargs.GROUP))
+            if not gaargs.gid or not gaargs.gid.isdigit():
+                handle_missing_id(gaargs.GROUP, 'group', pkg)
 
             # Reconstruct the args...
             newparam  = ['', ' --force'][gaargs.force]
-- 
2.9.0

[PATCH 3/4] useradd-staticids.bbclass: Allow missing UIDs/GIDs to generate warnings

[Peter Kjellerstedt]

Previously when USERADD_ERROR_DYNAMIC was set to "1", an exception was
raised if no numeric UID/GID could be determined for a user/group. Now
it is possible to set it to either "error", which results in the old
behavior, or "warn" in which case a warning is issued instead.

For backwards compatibility reasons, it is still possible to set
USERADD_ERROR_DYNAMIC to "1" and get an exception in case of failure.

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
---
 meta/classes/useradd-staticids.bbclass | 5 ++++-
 meta/conf/documentation.conf           | 2 +-
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/meta/classes/useradd-staticids.bbclass b/meta/classes/useradd-staticids.bbclass
index 90835cf..b6e498c 100644
--- a/meta/classes/useradd-staticids.bbclass
+++ b/meta/classes/useradd-staticids.bbclass
@@ -51,9 +51,12 @@ def update_useradd_static_config(d):
         return id_table
 
     def handle_missing_id(id, type, pkg):
-        if d.getVar('USERADD_ERROR_DYNAMIC', True) == '1':
+        # For backwards compatibility we accept "1" in addition to "error"
+        if d.getVar('USERADD_ERROR_DYNAMIC', True) == 'error' or d.getVar('USERADD_ERROR_DYNAMIC', True) == '1':
             #bb.error("Skipping recipe %s, package %s which adds %sname %s does not have a static ID defined." % (d.getVar('PN', True),  pkg, type, id))
             raise bb.build.FuncFailed("%s - %s: %sname %s does not have a static ID defined." % (d.getVar('PN', True), pkg, type, id))
+        elif d.getVar('USERADD_ERROR_DYNAMIC', True) == 'warn':
+            bb.warn("%s - %s: %sname %s does not have a static ID defined." % (d.getVar('PN', True), pkg, type, id))
 
     # We parse and rewrite the useradd components
     def rewrite_useradd(params):
diff --git a/meta/conf/documentation.conf b/meta/conf/documentation.conf
index 38639d5..fa904c0 100644
--- a/meta/conf/documentation.conf
+++ b/meta/conf/documentation.conf
@@ -452,7 +452,7 @@ UBOOT_SUFFIX[doc] = "Points to the generated U-Boot extension."
 UBOOT_TARGET[doc] = "Specifies the target used for building U-Boot."
 USE_DEVFS[doc] = "When building images, specifies to populate or not /dev. This variable defaults to '1' (leave directory empty, surely because devtmpfs do the job). Set it to '0' to use makedevs (or consider using a custom file with IMAGE_DEVICE_TABLES)."
 USER_CLASSES[doc] = "List of additional classes to use when building images that enable extra features."
-USERADD_ERROR_DYNAMIC[doc] = "Forces the OpenEmbedded build system to produce an error if the user identification (uid) and group identification (gid) values are not defined in files/passwd and files/group files."
+USERADD_ERROR_DYNAMIC[doc] = "If set to 'error', forces the OpenEmbedded build system to produce an error if the user identification (uid) and group identification (gid) values are not defined in files/passwd and files/group files. If set to 'warn', a warning will be issued instead."
 USERADD_GID_TABLES[doc] = "Specifies a password file to use for obtaining static group identification (gid) values when the OpenEmbedded build system adds a group to the system during package installation."
 USERADD_PACKAGES[doc] = "When a recipe inherits the useradd class, this variable specifies the individual packages within the recipe that require users and/or groups to be added."
 USERADD_PARAM[doc] = "When a recipe inherits the useradd class, this variable specifies for a package what parameters should be passed to the useradd command if you wish to add a user to the system when the package is installed."
-- 
2.9.0

[PATCH 4/4] Update the documentation of USERADD_ERROR_DYNAMIC

[Peter Kjellerstedt]

Document that USERADD_ERROR_DYNAMIC can now take "error" or "warn" as
values, to either cause an exception or a warning in case no numeric
UID/GID can be determined for a user or group.

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
---
 documentation/ref-manual/ref-variables.xml | 19 +++++++++++--------
 1 file changed, 11 insertions(+), 8 deletions(-)

diff --git a/documentation/ref-manual/ref-variables.xml b/documentation/ref-manual/ref-variables.xml
index 0252757..f825f6c 100644
--- a/documentation/ref-manual/ref-variables.xml
+++ b/documentation/ref-manual/ref-variables.xml
@@ -14381,16 +14381,19 @@ recipes-graphics/xorg-font/font-alias_1.0.3.bb:PR = "${INC_PR}.3"
 
         <glossentry id='var-USERADD_ERROR_DYNAMIC'><glossterm>USERADD_ERROR_DYNAMIC</glossterm>
             <info>
-               USERADD_ERROR_DYNAMIC[doc] = "Forces the OpenEmbedded build system to produce an error if the user identification (uid) and group identification (gid) values are not defined in files/passwd and files/group files."
+               USERADD_ERROR_DYNAMIC[doc] = "If set to 'error', forces the OpenEmbedded build system to produce an error if the user identification (uid) and group identification (gid) values are not defined in files/passwd and files/group files. If set to 'warn', a warning will be issued instead."
             </info>
             <glossdef>
                 <para role="glossdeffirst">
 <!--                <para role="glossdeffirst"><imagedata fileref="figures/define-generic.png" /> -->
-                    Forces the OpenEmbedded build system to produce an error
-                    if the user identification (<filename>uid</filename>) and
-                    group identification (<filename>gid</filename>) values
-                    are not defined in <filename>files/passwd</filename>
-                    and <filename>files/group</filename> files.
+
+                    If set to "error", forces the OpenEmbedded build system to
+                    produce an error if the user identification
+                    (<filename>uid</filename>) and group identification
+                    (<filename>gid</filename>) values are not defined
+                    in <filename>files/passwd</filename>
+                    and <filename>files/group</filename> files. If set to
+                    "warn", a warning will be issued instead.
                 </para>
 
                 <para>
@@ -14406,7 +14409,7 @@ recipes-graphics/xorg-font/font-alias_1.0.3.bb:PR = "${INC_PR}.3"
                     your <filename>local.conf</filename> file as
                     follows:
                     <literallayout class='monospaced'>
-     USERADD_ERROR_DYNAMIC = "1"
+     USERADD_ERROR_DYNAMIC = "error"
                     </literallayout>
                     Overriding the default behavior implies you are going to
                     also take steps to set static <filename>uid</filename> and
@@ -14562,7 +14565,7 @@ recipes-graphics/xorg-font/font-alias_1.0.3.bb:PR = "${INC_PR}.3"
 
         <glossentry id='var-USERADDEXTENSION'><glossterm>USERADDEXTENSION</glossterm>
             <info>
-               USERADDEXTENSION[doc] = "When set to "useradd-staticids", causes the OpenEmbedded build system to base all user and group additions on a static passwd and group files found in BBPATH."
+               USERADDEXTENSION[doc] = "When set to 'useradd-staticids', causes the OpenEmbedded build system to base all user and group additions on a static passwd and group files found in BBPATH."
             </info>
             <glossdef>
                 <para role="glossdeffirst">
-- 
2.9.0

Re: [PATCH 1/4] documentation.conf: Add information about USERADD varaibles

[Mark Hatle]

On 6/17/16 8:41 AM, Peter Kjellerstedt wrote:
> Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
> ---
>  meta/conf/documentation.conf | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/meta/conf/documentation.conf b/meta/conf/documentation.conf
> index c0025f5..38639d5 100644
> --- a/meta/conf/documentation.conf
> +++ b/meta/conf/documentation.conf
> @@ -452,8 +452,12 @@ UBOOT_SUFFIX[doc] = "Points to the generated U-Boot extension."
>  UBOOT_TARGET[doc] = "Specifies the target used for building U-Boot."
>  USE_DEVFS[doc] = "When building images, specifies to populate or not /dev. This variable defaults to '1' (leave directory empty, surely because devtmpfs do the job). Set it to '0' to use makedevs (or consider using a custom file with IMAGE_DEVICE_TABLES)."
>  USER_CLASSES[doc] = "List of additional classes to use when building images that enable extra features."
> +USERADD_ERROR_DYNAMIC[doc] = "Forces the OpenEmbedded build system to produce an error if the user identification (uid) and group identification (gid) values are not defined in files/passwd and files/group files."

I think the description is incorrect.  Specifically:

...values are not defined in files/passwd and files/group files.

should be

...values are not defined in the USERADD_UID_TABLES and USERADD_GID_TABLES.

> +USERADD_GID_TABLES[doc] = "Specifies a password file to use for obtaining static group identification (gid) values when the OpenEmbedded build system adds a group to the system during package installation."
>  USERADD_PACKAGES[doc] = "When a recipe inherits the useradd class, this variable specifies the individual packages within the recipe that require users and/or groups to be added."
>  USERADD_PARAM[doc] = "When a recipe inherits the useradd class, this variable specifies for a package what parameters should be passed to the useradd command if you wish to add a user to the system when the package is installed."
> +USERADD_UID_TABLES[doc] = "Specifies a password file to use for obtaining static user identification (uid) values when the OpenEmbedded build system adds a user to the system during package installation."
> +USERADDEXTENSION[doc] = "When set to 'useradd-staticids', causes the OpenEmbedded build system to base all user and group additions on a static passwd and group files found in BBPATH."

again, here I think we need to clarify "static passwd and group" to refer to the
USERADD_UID_TABLES and USERADD_GID_TABLES or someone might not understand that
is what is needed.

>  
>  #W
>  
> 

Re: [PATCH 1/4] documentation.conf: Add information about USERADD varaibles

[Peter Kjellerstedt]

> -----Original Message-----
> From: openembedded-core-bounces@lists.openembedded.org
> [mailto:openembedded-core-bounces@lists.openembedded.org] On Behalf Of
> Mark Hatle
> Sent: den 17 juni 2016 15:55
> To: openembedded-core@lists.openembedded.org
> Subject: Re: [OE-core] [PATCH 1/4] documentation.conf: Add information
> about USERADD varaibles
> 
> On 6/17/16 8:41 AM, Peter Kjellerstedt wrote:
> > Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
> > ---
> >  meta/conf/documentation.conf | 4 ++++
> >  1 file changed, 4 insertions(+)
> >
> > diff --git a/meta/conf/documentation.conf
> b/meta/conf/documentation.conf
> > index c0025f5..38639d5 100644
> > --- a/meta/conf/documentation.conf
> > +++ b/meta/conf/documentation.conf
> > @@ -452,8 +452,12 @@ UBOOT_SUFFIX[doc] = "Points to the generated U-
> Boot extension."
> >  UBOOT_TARGET[doc] = "Specifies the target used for building U-Boot."
> >  USE_DEVFS[doc] = "When building images, specifies to populate or not
> /dev. This variable defaults to '1' (leave directory empty, surely
> because devtmpfs do the job). Set it to '0' to use makedevs (or
> consider using a custom file with IMAGE_DEVICE_TABLES)."
> >  USER_CLASSES[doc] = "List of additional classes to use when building
> images that enable extra features."
> > +USERADD_ERROR_DYNAMIC[doc] = "Forces the OpenEmbedded build system
> to produce an error if the user identification (uid) and group
> identification (gid) values are not defined in files/passwd and
> files/group files."
> 
> I think the description is incorrect.  Specifically:
> 
> ...values are not defined in files/passwd and files/group files.
> 
> should be
> 
> ...values are not defined in the USERADD_UID_TABLES and
> USERADD_GID_TABLES.
> 
> > +USERADD_GID_TABLES[doc] = "Specifies a password file to use for
> obtaining static group identification (gid) values when the
> OpenEmbedded build system adds a group to the system during package
> installation."
> >  USERADD_PACKAGES[doc] = "When a recipe inherits the useradd class,
> this variable specifies the individual packages within the recipe that
> require users and/or groups to be added."
> >  USERADD_PARAM[doc] = "When a recipe inherits the useradd class, this
> variable specifies for a package what parameters should be passed to
> the useradd command if you wish to add a user to the system when the
> package is installed."
> > +USERADD_UID_TABLES[doc] = "Specifies a password file to use for
> obtaining static user identification (uid) values when the OpenEmbedded
> build system adds a user to the system during package installation."
> > +USERADDEXTENSION[doc] = "When set to 'useradd-staticids', causes the
> OpenEmbedded build system to base all user and group additions on a
> static passwd and group files found in BBPATH."
> 
> again, here I think we need to clarify "static passwd and group" to
> refer to the
> USERADD_UID_TABLES and USERADD_GID_TABLES or someone might not
> understand that
> is what is needed.
> 
> >
> >  #W
> >
> >


Ok, I will send an updated set of changes.

//Peter