[PATCH V6 0/3] Add a mechanism for image level user/group configuration

[PATCH V6 0/3] Add a mechanism for image level user/group configuration

[Qi.Chen]

From: Chen Qi <Qi.Chen@windriver.com>

Changes from V5:
1. Change the configuration format of EXTRA_USERS_PARAMS

The following changes since commit a63229917a5708de2d161aba0d67168ce0da6365:

  meta-yocto-bsp: update reference board SRCREVs (2013-07-10 09:45:51 +0100)

are available in the git repository at:

  git://git.pokylinux.org/poky-contrib ChenQi/extrausers
  http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=ChenQi/extrausers

Chen Qi (3):
  useradd_base.bbclass: add a new bbclass
  useradd.bbclass: code refactor
  extrausers.bbclass: add a new bbclass

 meta/classes/extrausers.bbclass   |   61 ++++++++++
 meta/classes/useradd.bbclass      |   99 ++--------------
 meta/classes/useradd_base.bbclass |  230 +++++++++++++++++++++++++++++++++++++
 3 files changed, 299 insertions(+), 91 deletions(-)
 create mode 100644 meta/classes/extrausers.bbclass
 create mode 100644 meta/classes/useradd_base.bbclass

-- 
1.7.9.5

[PATCH V6 1/3] useradd_base.bbclass: add a new bbclass

[Qi.Chen]

From: Chen Qi <Qi.Chen@windriver.com>

This class is mainly a collection of basic functions for user/group
settings.

This class is intended to be inherited by useradd.bbclass and the
extrausers.bbclass.

[YOCTO #4074]

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
 meta/classes/useradd_base.bbclass |  230 +++++++++++++++++++++++++++++++++++++
 1 file changed, 230 insertions(+)
 create mode 100644 meta/classes/useradd_base.bbclass

diff --git a/meta/classes/useradd_base.bbclass b/meta/classes/useradd_base.bbclass
new file mode 100644
index 0000000..7aafe29
--- /dev/null
+++ b/meta/classes/useradd_base.bbclass
@@ -0,0 +1,230 @@
+# This bbclass provides basic functionality for user/group settings.
+# This bbclass is intended to be inherited by useradd.bbclass and
+# extrausers.bbclass.
+
+# The following functions basically have similar logic.
+# *) Perform necessary checks before invoking the actual command
+# *) Invoke the actual command, make retries if necessary
+# *) Error out if an error occurs.
+
+# Note that before invoking these functions, make sure the global variable
+# PSEUDO is set up correctly.
+
+perform_groupadd () {
+	local rootdir="$1"
+	local opts="$2"
+	local retries="$3"
+	bbnote "Performing groupadd with [$opts] and $retries times of retry"
+	local groupname=`echo "$opts" | awk '{ print $NF }'`
+	local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
+	if test "x$group_exists" = "x"; then
+		local count=0
+		while true; do
+			eval $PSEUDO groupadd $opts || true
+			group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
+			if test "x$group_exists" = "x"; then
+				bbwarn "groupadd command did not succeed. Retrying..."
+				sleep 1
+			else
+				break
+			fi
+			count=`expr $count + 1`
+			if test $count = $retries; then
+				bbfatal "Tried running groupadd command $retries times without scucess, giving up"
+			fi
+		done
+	else
+		bbwarn "group $groupname already exists, not re-creating it"
+	fi
+}
+
+perform_useradd () {
+	local rootdir="$1"
+	local opts="$2"
+	local retries="$3"
+	bbnote "Performing useradd with [$opts] and $retries times of retry"
+	local username=`echo "$opts" | awk '{ print $NF }'`
+	local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
+	if test "x$user_exists" = "x"; then
+	       local count=0
+	       while true; do
+		       eval $PSEUDO useradd $opts || true
+		       user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
+		       if test "x$user_exists" = "x"; then
+			       bbwarn "useradd command did not succeed. Retrying..."
+			       sleep 1
+		       else
+			       break
+		       fi
+		       count=`expr $count + 1`
+		       if test $count = $retries; then
+				bbfatal "Tried running useradd command $retries times without scucess, giving up"
+		       fi
+	       done
+	else
+		bbwarn "user $username already exists, not re-creating it"
+	fi
+}
+
+perform_groupmems () {
+	local rootdir="$1"
+	local opts="$2"
+	local retries="$3"
+	bbnote "Performing groupmems with [$opts] and $retries times of retry"
+	local groupname=`echo "$opts" | awk '{ for (i = 1; i < NF; i++) if ($i == "-g" || $i == "--group") print $(i+1) }'`
+	local username=`echo "$opts" | awk '{ for (i = 1; i < NF; i++) if ($i == "-a" || $i == "--add") print $(i+1) }'`
+	bbnote "Running groupmems command with group $groupname and user $username"
+	# groupmems fails if /etc/gshadow does not exist
+	local gshadow=""
+	if [ -f $rootdir${sysconfdir}/gshadow ]; then
+		gshadow="yes"
+	else
+		gshadow="no"
+		touch $rootdir${sysconfdir}/gshadow
+	fi
+	local mem_exists="`grep "^$groupname:[^:]*:[^:]*:\([^,]*,\)*$username\(,[^,]*\)*" $rootdir/etc/group || true`"
+	if test "x$mem_exists" = "x"; then
+		local count=0
+		while true; do
+			eval $PSEUDO groupmems $opts || true
+			mem_exists="`grep "^$groupname:[^:]*:[^:]*:\([^,]*,\)*$username\(,[^,]*\)*" $rootdir/etc/group || true`"
+			if test "x$mem_exists" = "x"; then
+				bbwarn "groupmems command did not succeed. Retrying..."
+				sleep 1
+			else
+				break
+			fi
+			count=`expr $count + 1`
+			if test $count = $retries; then
+				if test "x$gshadow" = "xno"; then
+					rm -f $rootdir${sysconfdir}/gshadow
+					rm -f $rootdir${sysconfdir}/gshadow-
+				fi
+				bbfatal "Tried running groupmems command $retries times without scucess, giving up"
+			fi
+		done
+	else
+		bbwarn "group $groupname already contains $username, not re-adding it"
+	fi
+	if test "x$gshadow" = "xno"; then
+		rm -f $rootdir${sysconfdir}/gshadow
+		rm -f $rootdir${sysconfdir}/gshadow-
+	fi
+}
+
+perform_groupdel () {
+	local rootdir="$1"
+	local opts="$2"
+	local retries="$3"
+	bbnote "Performing groupdel with [$opts] and $retries times of retry"
+	local groupname=`echo "$opts" | awk '{ print $NF }'`
+	local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
+	if test "x$group_exists" != "x"; then
+		local count=0
+		while true; do
+			eval $PSEUDO groupdel $opts || true
+			group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
+			if test "x$group_exists" != "x"; then
+				bbwarn "groupdel command did not succeed. Retrying..."
+				sleep 1
+			else
+				break
+			fi
+			count=`expr $count + 1`
+			if test $count = $retries; then
+				bbfatal "Tried running groupdel command $retries times without scucess, giving up"
+			fi
+		done
+	else
+		bbwarn "group $groupname doesn't exist, not removing it"
+	fi
+}
+
+perform_userdel () {
+	local rootdir="$1"
+	local opts="$2"
+	local retries="$3"
+	bbnote "Performing userdel with [$opts] and $retries times of retry"
+	local username=`echo "$opts" | awk '{ print $NF }'`
+	local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
+	if test "x$user_exists" != "x"; then
+	       local count=0
+	       while true; do
+		       eval $PSEUDO userdel $opts || true
+		       user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
+		       if test "x$user_exists" != "x"; then
+			       bbwarn "userdel command did not succeed. Retrying..."
+			       sleep 1
+		       else
+			       break
+		       fi
+		       count=`expr $count + 1`
+		       if test $count = $retries; then
+				bbfatal "Tried running userdel command $retries times without scucess, giving up"
+		       fi
+	       done
+	else
+		bbwarn "user $username doesn't exist, not removing it"
+	fi
+}
+
+perform_groupmod () {
+	# Other than the return value of groupmod, there's no simple way to judge whether the command
+	# succeeds, so we disable -e option temporarily
+	set +e
+	local rootdir="$1"
+	local opts="$2"
+	local retries="$3"
+	bbnote "Performing groupmod with [$opts] and $retries times of retry"
+	local groupname=`echo "$opts" | awk '{ print $NF }'`
+	local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
+	if test "x$group_exists" != "x"; then
+		local count=0
+		while true; do
+			eval $PSEUDO groupmod $opts
+			if test $? != 0; then
+				bbwarn "groupmod command did not succeed. Retrying..."
+				sleep 1
+			else
+				break
+			fi
+			count=`expr $count + 1`
+			if test $count = $retries; then
+				bbfatal "Tried running groupmod command $retries times without scucess, giving up"
+			fi
+		done
+	else
+		bbwarn "group $groupname doesn't exist, unable to modify it"
+	fi
+	set -e
+}
+
+perform_usermod () {
+	# Same reason with groupmod, temporarily disable -e option
+	set +e
+	local rootdir="$1"
+	local opts="$2"
+	local retries="$3"
+	bbnote "Performing usermod with [$opts] and $retries times of retry"
+	local username=`echo "$opts" | awk '{ print $NF }'`
+	local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
+	if test "x$user_exists" != "x"; then
+	       local count=0
+	       while true; do
+		       eval $PSEUDO usermod $opts
+		       if test $? != 0; then
+			       bbwarn "usermod command did not succeed. Retrying..."
+			       sleep 1
+		       else
+			       break
+		       fi
+		       count=`expr $count + 1`
+		       if test $count = $retries; then
+				bbfatal "Tried running usermod command $retries times without scucess, giving up"
+		       fi
+	       done
+	else
+		bbwarn "user $username doesn't exist, unable to modify it"
+	fi
+	set -e
+}
-- 
1.7.9.5

[PATCH V6 2/3] useradd.bbclass: code refactor

[Qi.Chen]

From: Chen Qi <Qi.Chen@windriver.com>

The basic functions have moved to useradd_base.bbclass. So this class
only needs to inherit useradd_base.bbclass and use the functions defined
there. The reason is to avoid code duplication with extrausers.bbclass.

[YOCTO #4074]

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
 meta/classes/useradd.bbclass |   99 ++++--------------------------------------
 1 file changed, 8 insertions(+), 91 deletions(-)

diff --git a/meta/classes/useradd.bbclass b/meta/classes/useradd.bbclass
index 3fe011d..9b3ac25 100644
--- a/meta/classes/useradd.bbclass
+++ b/meta/classes/useradd.bbclass
@@ -1,3 +1,5 @@
+inherit useradd_base
+
 # base-passwd-cross provides the default passwd and group files in the
 # target sysroot, and shadow -native and -sysroot provide the utilities
 # and support files needed to add and modify user and group accounts
@@ -44,30 +46,7 @@ if test "x$GROUPADD_PARAM" != "x"; then
 	opts=`echo "$GROUPADD_PARAM" | cut -d ';' -f 1`
 	remaining=`echo "$GROUPADD_PARAM" | cut -d ';' -f 2-`
 	while test "x$opts" != "x"; do
-		groupname=`echo "$opts" | awk '{ print $NF }'`
-		group_exists=`grep "^$groupname:" $SYSROOT/etc/group || true`
-		if test "x$group_exists" = "x"; then
-			count=1
-			while true; do
-				eval $PSEUDO groupadd $OPT $opts || true
-				group_exists=`grep "^$groupname:" $SYSROOT/etc/group || true`
-				if test "x$group_exists" = "x"; then
-					# File locking issues can require us to retry the command
-					echo "WARNING: groupadd command did not succeed. Retrying..."
-					sleep 1
-				else
-					break
-				fi
-				count=`expr $count + 1`
-				if test $count = 11; then
-					echo "ERROR: tried running groupadd command 10 times without success, giving up"
-					exit 1
-				fi
-			done		
-		else
-			echo "Note: group $groupname already exists, not re-creating it"
-		fi
-
+		perform_groupadd "$SYSROOT" "$OPT $opts" 10
 		if test "x$opts" = "x$remaining"; then
 			break
 		fi
@@ -83,32 +62,7 @@ if test "x$USERADD_PARAM" != "x"; then
 	opts=`echo "$USERADD_PARAM" | cut -d ';' -f 1`
 	remaining=`echo "$USERADD_PARAM" | cut -d ';' -f 2-`
 	while test "x$opts" != "x"; do
-		# useradd does not have a -f option, so we have to check if the
-		# username already exists manually
-		username=`echo "$opts" | awk '{ print $NF }'`
-		user_exists=`grep "^$username:" $SYSROOT/etc/passwd || true`
-		if test "x$user_exists" = "x"; then
-			count=1
-			while true; do
-				eval $PSEUDO useradd $OPT $opts || true
-				user_exists=`grep "^$username:" $SYSROOT/etc/passwd || true`
-				if test "x$user_exists" = "x"; then
-					# File locking issues can require us to retry the command
-					echo "WARNING: useradd command did not succeed. Retrying..."
-					sleep 1
-				else
-					break
-				fi
-				count=`expr $count + 1`
-				if test $count = 11; then
-					echo "ERROR: tried running useradd command 10 times without success, giving up"
-					exit 1
-				fi
-			done
-		else
-			echo "Note: username $username already exists, not re-creating it"
-		fi
-
+		perform_useradd "$SYSROOT" "$OPT $opts" 10
 		if test "x$opts" = "x$remaining"; then
 			break
 		fi
@@ -119,58 +73,18 @@ fi
 
 if test "x$GROUPMEMS_PARAM" != "x"; then
 	echo "Running groupmems commands..."
-	# groupmems fails if /etc/gshadow does not exist
-	if [ -f $SYSROOT${sysconfdir}/gshadow ]; then
-		gshadow="yes"
-	else
-		gshadow="no"
-		touch $SYSROOT${sysconfdir}/gshadow
-	fi
 	# Invoke multiple instances of groupmems for parameter lists
 	# separated by ';'
 	opts=`echo "$GROUPMEMS_PARAM" | cut -d ';' -f 1`
 	remaining=`echo "$GROUPMEMS_PARAM" | cut -d ';' -f 2-`
 	while test "x$opts" != "x"; do
-		groupname=`echo "$opts" | awk '{ for (i = 1; i < NF; i++) if ($i == "-g" || $i == "--group") print $(i+1) }'`
-		username=`echo "$opts" | awk '{ for (i = 1; i < NF; i++) if ($i == "-a" || $i == "--add") print $(i+1) }'`
-		echo "$groupname $username"
-		mem_exists=`grep "^$groupname:[^:]*:[^:]*:\([^,]*,\)*$username\(,[^,]*\)*" $SYSROOT/etc/group || true`
-		if test "x$mem_exists" = "x"; then
-			count=1
-			while true; do
-				eval $PSEUDO groupmems $OPT $opts || true
-				mem_exists=`grep "^$groupname:[^:]*:[^:]*:\([^,]*,\)*$username\(,[^,]*\)*" $SYSROOT/etc/group || true`
-				if test "x$mem_exists" = "x"; then
-					# File locking issues can require us to retry the command
-					echo "WARNING: groupmems command did not succeed. Retrying..."
-					sleep 1
-				else
-					break
-				fi
-				count=`expr $count + 1`
-				if test $count = 11; then
-					echo "ERROR: tried running groupmems command 10 times without success, giving up"
-					if test "x$gshadow" = "xno"; then
-						rm -f $SYSROOT${sysconfdir}/gshadow
-						rm -f $SYSROOT${sysconfdir}/gshadow-
-					fi
-					exit 1
-				fi
-			done
-		else
-			echo "Note: group $groupname already contains $username, not re-adding it"
-		fi
-
+		perform_groupmems "$SYSROOT" "$OPT $opts" 10
 		if test "x$opts" = "x$remaining"; then
 			break
 		fi
 		opts=`echo "$remaining" | cut -d ';' -f 1`
 		remaining=`echo "$remaining" | cut -d ';' -f 2-`
 	done
-	if test "x$gshadow" = "xno"; then
-		rm -f $SYSROOT${sysconfdir}/gshadow
-		rm -f $SYSROOT${sysconfdir}/gshadow-
-	fi
 fi
 }
 
@@ -254,6 +168,9 @@ fakeroot python populate_packages_prepend () {
         preinst = d.getVar('pkg_preinst_%s' % pkg, True) or d.getVar('pkg_preinst', True)
         if not preinst:
             preinst = '#!/bin/sh\n'
+        preinst += 'perform_groupadd () {\n%s}\n' % d.getVar('perform_groupadd', True)
+        preinst += 'perform_useradd () {\n%s}\n' % d.getVar('perform_useradd', True)
+        preinst += 'perform_groupmems () {\n%s}\n' % d.getVar('perform_groupmems', True)
         preinst += d.getVar('useradd_preinst', True)
         d.setVar('pkg_preinst_%s' % pkg, preinst)
 
-- 
1.7.9.5

[PATCH V6 3/3] extrausers.bbclass: add a new bbclass

[Qi.Chen]

From: Chen Qi <Qi.Chen@windriver.com>

This class is dedicated to image level user/group configuration.
It inherits useradd_base.bbclass.

Users need to inherit this class in their layers or local.conf to
make the setting of EXTRA_USERS_PARAMS effective.

For detailed configuration format of EXTRA_USERS_PARAMS, please
refer to local.conf.sample.extended.

[YOCTO #4074]

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
 meta/classes/extrausers.bbclass |   61 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 61 insertions(+)
 create mode 100644 meta/classes/extrausers.bbclass

diff --git a/meta/classes/extrausers.bbclass b/meta/classes/extrausers.bbclass
new file mode 100644
index 0000000..8670a2a
--- /dev/null
+++ b/meta/classes/extrausers.bbclass
@@ -0,0 +1,61 @@
+# This bbclass is mainly used for image level user/group configuration.
+# Inherit this class if you want to make EXTRA_USERS_PARAMS effective.
+
+# Below is an example showing how to use this functionality.
+# INHERIT += "extrausers"
+# EXTRA_USERS_PARAMS = "\
+# useradd -p '' tester; \
+# groupadd developers; \
+# userdel nobody; \
+# groupdel -g video; \
+# groupmod -g 1020 developers; \
+# usermod -s /bin/sh tester; \
+# "
+
+
+inherit useradd_base
+
+IMAGE_INSTALL_append = " ${@['', 'base-passwd shadow'][bool(d.getVar('EXTRA_USERS_PARAMS', True))]}"
+
+# Image level user / group settings
+ROOTFS_POSTPROCESS_COMMAND_append = " set_user_group;"
+
+# Image level user / group settings
+set_user_group () {
+	user_group_settings="${EXTRA_USERS_PARAMS}"
+	export PSEUDO="${FAKEROOTENV} ${STAGING_DIR_NATIVE}${bindir}/pseudo"
+	setting=`echo $user_group_settings | cut -d ';' -f1`
+	remaining=`echo $user_group_settings | cut -d ';' -f2-`
+	while test "x$setting" != "x"; do
+		cmd=`echo $setting | cut -d ' ' -f1`
+		opts=`echo $setting | cut -d ' ' -f2-`
+		# Different from useradd.bbclass, there's no file locking issue here, as
+		# this setting is actually a serial process. So we only retry once.
+		case $cmd in
+			useradd)
+				perform_useradd "${IMAGE_ROOTFS}" "-R ${IMAGE_ROOTFS} $opts" 1
+				;;
+			groupadd)
+				perform_groupadd "${IMAGE_ROOTFS}" "-R ${IMAGE_ROOTFS} $opts" 1
+				;;
+			userdel)
+				perform_userdel "${IMAGE_ROOTFS}" "-R ${IMAGE_ROOTFS} $opts" 1
+				;;
+			groupdel)
+				perform_groupdel "${IMAGE_ROOTFS}" "-R ${IMAGE_ROOTFS} $opts" 1
+				;;
+			usermod)
+				perform_usermod "${IMAGE_ROOTFS}" "-R ${IMAGE_ROOTFS} $opts" 1
+				;;
+			groupmod)
+				perform_groupmod "${IMAGE_ROOTFS}" "-R ${IMAGE_ROOTFS} $opts" 1
+				;;
+			*)
+				bbfatal "Invalid command in EXTRA_USERS_PARAMS: $cmd"
+				;;
+		esac
+		# iterate to the next setting
+		setting=`echo $remaining | cut -d ';' -f1`
+		remaining=`echo $remaining | cut -d ';' -f2-`
+	done
+}
-- 
1.7.9.5

Re: [PATCH V6 2/3] useradd.bbclass: code refactor

[Otavio Salvador]

On Wed, Jul 17, 2013 at 3:08 AM,  <Qi.Chen@windriver.com> wrote:
> From: Chen Qi <Qi.Chen@windriver.com>
>
> The basic functions have moved to useradd_base.bbclass. So this class
> only needs to inherit useradd_base.bbclass and use the functions defined
> there. The reason is to avoid code duplication with extrausers.bbclass.
>
> [YOCTO #4074]
>
> Signed-off-by: Chen Qi <Qi.Chen@windriver.com>

I think this patch (2) should be squashed onto patch 1; so it easy git
bisect in case we're trying to figure out a regression.

--
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://projetos.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750