[PATCH 00/20] udev-cache related changes

[PATCH 00/20] udev-cache related changes

[Ben Shelton]

This patchset contains changes to the udev-cache scripts to reduce size on
disk, to improve error reporting, and to fix bugs, and a few fixes and tweaks
to udev overall.

The following changes since commit 5a09acef73b5cfc59bbcd3e93603c99a8c078bd8:

  local.conf.sample.extended: fix example for EXTRA_USERS_PARAMS (2014-08-04 17:38:17 +0100)

are available in the git repository at:

  git://github.com/ni/openembedded-core dev/bshelton/udev_cache
  https://github.com/ni/openembedded-core/tree/dev/bshelton/udev_cache

Ben Shelton (2):
  busybox: tar: enable CONFIG_FEATURE_TAR_LONG_OPTIONS
  udev-cache: fix udev-cache changes to work with busybox tar

Richard Tollerton (18):
  udev-cache: Compress the cache
  udev-cache: choose a more descriptive cache filename
  udev-cache: Honor VERBOSE for bootup message
  udev-cache: Don't ignore errors from cache extract
  udev-cache: Remove superfluous subshell on extract
  udev-cache: Update cache tarball atomically
  udev-cache: Create cache asynchronously
  udev-cache.default: documentation update
  udev-cache: parametrize sysconf file paths
  udev-cache: parametrize tar options
  udev-cache: fix timestamp errors on systems lacking an RTC
  udev-cache: Avoid caching udev.cache or non-devfs filesystems
  udev-cache: refactor; improve verbosity and error handling
  udev-cache: don't attempt to extract cache if it doesn't exist
  udev-cache: invalidate on rules.d changes
  udev: Make build MACHINE-specific
  udev: Disable keymap support on machines lacking keyboard support
  udev: don't halt if devtmpfs can't be mounted

 meta/recipes-core/busybox/busybox/defconfig    |  2 +-
 meta/recipes-core/udev/udev.inc                |  7 ++++++
 meta/recipes-core/udev/udev/init               | 30 +++++++++++++++-----------
 meta/recipes-core/udev/udev/udev-cache         | 21 ++++++++++++++----
 meta/recipes-core/udev/udev/udev-cache.default | 18 ++++++++++++++--
 5 files changed, 58 insertions(+), 20 deletions(-)

-- 
2.0.4

[PATCH 01/20] udev-cache: Compress the cache

[Ben Shelton]

From: Richard Tollerton <rich.tollerton@ni.com>

$DEVCACHE is observed to be 100k uncompressed; compressing it reduces
its size to ~5k.

Natinst-Rally-ID: TA44427
Acked-by: Gratian Crisan <gratian.crisan@ni.com>
Natinst-ReviewBoard-ID: 58620
Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>
---
 meta/recipes-core/udev/udev/init               | 2 +-
 meta/recipes-core/udev/udev/udev-cache         | 2 +-
 meta/recipes-core/udev/udev/udev-cache.default | 2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/meta/recipes-core/udev/udev/init b/meta/recipes-core/udev/udev/init
index f2c84d5..1e69861 100644
--- a/meta/recipes-core/udev/udev/init
+++ b/meta/recipes-core/udev/udev/init
@@ -69,7 +69,7 @@ case "$1" in
 		    readfiles /etc/udev/cache.data
 		    OLDDATA="$READDATA"
 		    if [ "$OLDDATA" = "$NEWDATA" ]; then
-                            (cd /; tar xf $DEVCACHE > /dev/null 2>&1)
+                            (cd /; tar xzf $DEVCACHE > /dev/null 2>&1)
                             not_first_boot=1
                             [ "$VERBOSE" != "no" ] && echo "udev: using cache file $DEVCACHE"
                             [ -e /dev/shm/udev.cache ] && rm -f /dev/shm/udev.cache
diff --git a/meta/recipes-core/udev/udev/udev-cache b/meta/recipes-core/udev/udev/udev-cache
index db5a513..3769062 100644
--- a/meta/recipes-core/udev/udev/udev-cache
+++ b/meta/recipes-core/udev/udev/udev-cache
@@ -25,7 +25,7 @@ fi
 
 if [ "$DEVCACHE" != "" -a -e /dev/shm/udev.cache ]; then
 	echo "Populating dev cache"
-	(cd /; tar cf "$DEVCACHE" dev)
+	(cd /; tar czf "$DEVCACHE" dev)
 	mv /dev/shm/udev.cache /etc/udev/cache.data
 fi
 
diff --git a/meta/recipes-core/udev/udev/udev-cache.default b/meta/recipes-core/udev/udev/udev-cache.default
index 2093336..909ec87 100644
--- a/meta/recipes-core/udev/udev/udev-cache.default
+++ b/meta/recipes-core/udev/udev/udev-cache.default
@@ -1,5 +1,5 @@
 # Default for /etc/init.d/udev
 
 # Comment this out to disable device cache
-DEVCACHE="/etc/dev.tar"
+DEVCACHE="/etc/dev.tar.gz"
 PROBE_PLATFORM_BUS="yes"
-- 
2.0.4

[PATCH 02/20] udev-cache: choose a more descriptive cache filename

[Ben Shelton]

From: Richard Tollerton <rich.tollerton@ni.com>

"/etc/dev.tar.gz" doesn't adequately imply that udev-cache
maintains it. Instead, call it "/etc/udev-cache.tar.gz".

Natinst-Rally-ID: TA44427
Acked-by: Gratian Crisan <gratian.crisan@ni.com>
Natinst-ReviewBoard-ID: 58620
Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>
---
 meta/recipes-core/udev/udev/udev-cache.default | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-core/udev/udev/udev-cache.default b/meta/recipes-core/udev/udev/udev-cache.default
index 909ec87..a3b7326 100644
--- a/meta/recipes-core/udev/udev/udev-cache.default
+++ b/meta/recipes-core/udev/udev/udev-cache.default
@@ -1,5 +1,5 @@
 # Default for /etc/init.d/udev
 
 # Comment this out to disable device cache
-DEVCACHE="/etc/dev.tar.gz"
+DEVCACHE="/etc/udev-cache.tar.gz"
 PROBE_PLATFORM_BUS="yes"
-- 
2.0.4

[PATCH 03/20] udev-cache: Honor VERBOSE for bootup message

[Ben Shelton]

From: Richard Tollerton <rich.tollerton@ni.com>

Natinst-Rally-ID: TA44427
Acked-by: Gratian Crisan <gratian.crisan@ni.com>
Natinst-ReviewBoard-ID: 58620
Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>
---
 meta/recipes-core/udev/udev/udev-cache | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-core/udev/udev/udev-cache b/meta/recipes-core/udev/udev/udev-cache
index 3769062..154d2ad 100644
--- a/meta/recipes-core/udev/udev/udev-cache
+++ b/meta/recipes-core/udev/udev/udev-cache
@@ -24,7 +24,7 @@ if [ "$ROOTFS_READ_ONLY" = "yes" ]; then
 fi
 
 if [ "$DEVCACHE" != "" -a -e /dev/shm/udev.cache ]; then
-	echo "Populating dev cache"
+	[ "${VERBOSE}" != "no" ] && echo "Populating dev cache"
 	(cd /; tar czf "$DEVCACHE" dev)
 	mv /dev/shm/udev.cache /etc/udev/cache.data
 fi
-- 
2.0.4

[PATCH 04/20] udev-cache: Don't ignore errors from cache extract

[Ben Shelton]

From: Richard Tollerton <rich.tollerton@ni.com>

Currently, error messages generated during tarball extract are
suppressed. This seems ham-handed, particularly since under normal
operation, the only expected error ought to be the attempted extraction
of pipes, which typically only means /dev/log.

So stop the redirections and add an --exclude=log to suppress the error
message when attempting to extract /dev/log.

This requires CONFIG_FEATURE_TAR_LONG_OPTIONS if using busybox tar.

Natinst-Rally-ID: TA44427
Acked-by: Gratian Crisan <gratian.crisan@ni.com>
Natinst-ReviewBoard-ID: 58620
Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>
---
 meta/recipes-core/udev/udev/init | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-core/udev/udev/init b/meta/recipes-core/udev/udev/init
index 1e69861..a96309d 100644
--- a/meta/recipes-core/udev/udev/init
+++ b/meta/recipes-core/udev/udev/init
@@ -69,7 +69,7 @@ case "$1" in
 		    readfiles /etc/udev/cache.data
 		    OLDDATA="$READDATA"
 		    if [ "$OLDDATA" = "$NEWDATA" ]; then
-                            (cd /; tar xzf $DEVCACHE > /dev/null 2>&1)
+                            (cd /; tar xzf $DEVCACHE --exclude=log)
                             not_first_boot=1
                             [ "$VERBOSE" != "no" ] && echo "udev: using cache file $DEVCACHE"
                             [ -e /dev/shm/udev.cache ] && rm -f /dev/shm/udev.cache
-- 
2.0.4

[PATCH 05/20] busybox: tar: enable CONFIG_FEATURE_TAR_LONG_OPTIONS

[Ben Shelton]

This config flag is needed to support the --exclude option, which is
needed for the commit 'udev-cache: Don't ignore errors from cache
extract'.

Enabling this option adds 352 bytes to the size of busybox on i386.

Signed-off-by: Ben Shelton <ben.shelton@ni.com>
---
 meta/recipes-core/busybox/busybox/defconfig | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-core/busybox/busybox/defconfig b/meta/recipes-core/busybox/busybox/defconfig
index 0107231..d2a67c2 100644
--- a/meta/recipes-core/busybox/busybox/defconfig
+++ b/meta/recipes-core/busybox/busybox/defconfig
@@ -156,7 +156,7 @@ CONFIG_FEATURE_TAR_FROM=y
 CONFIG_FEATURE_TAR_OLDGNU_COMPATIBILITY=y
 # CONFIG_FEATURE_TAR_OLDSUN_COMPATIBILITY is not set
 CONFIG_FEATURE_TAR_GNU_EXTENSIONS=y
-# CONFIG_FEATURE_TAR_LONG_OPTIONS is not set
+CONFIG_FEATURE_TAR_LONG_OPTIONS=y
 # CONFIG_FEATURE_TAR_TO_COMMAND is not set
 # CONFIG_FEATURE_TAR_UNAME_GNAME is not set
 # CONFIG_FEATURE_TAR_NOPRESERVE_TIME is not set
-- 
2.0.4

[PATCH 06/20] udev-cache: Remove superfluous subshell on extract

[Ben Shelton]

From: Richard Tollerton <rich.tollerton@ni.com>

tar -C exists in both GNU and busybox tar, so use that instead of
(cd /; tar ...). This allows the subshell to be removed.

Natinst-Rally-ID: TA44427
Acked-by: Gratian Crisan <gratian.crisan@ni.com>
Natinst-ReviewBoard-ID: 58620
Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>
---
 meta/recipes-core/udev/udev/init       | 2 +-
 meta/recipes-core/udev/udev/udev-cache | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-core/udev/udev/init b/meta/recipes-core/udev/udev/init
index a96309d..bcb9040 100644
--- a/meta/recipes-core/udev/udev/init
+++ b/meta/recipes-core/udev/udev/init
@@ -69,7 +69,7 @@ case "$1" in
 		    readfiles /etc/udev/cache.data
 		    OLDDATA="$READDATA"
 		    if [ "$OLDDATA" = "$NEWDATA" ]; then
-                            (cd /; tar xzf $DEVCACHE --exclude=log)
+                            tar xzf $DEVCACHE -C / --exclude=log
                             not_first_boot=1
                             [ "$VERBOSE" != "no" ] && echo "udev: using cache file $DEVCACHE"
                             [ -e /dev/shm/udev.cache ] && rm -f /dev/shm/udev.cache
diff --git a/meta/recipes-core/udev/udev/udev-cache b/meta/recipes-core/udev/udev/udev-cache
index 154d2ad..11c5f0e 100644
--- a/meta/recipes-core/udev/udev/udev-cache
+++ b/meta/recipes-core/udev/udev/udev-cache
@@ -25,7 +25,7 @@ fi
 
 if [ "$DEVCACHE" != "" -a -e /dev/shm/udev.cache ]; then
 	[ "${VERBOSE}" != "no" ] && echo "Populating dev cache"
-	(cd /; tar czf "$DEVCACHE" dev)
+	tar czf "$DEVCACHE" dev -C / --exclude=log
 	mv /dev/shm/udev.cache /etc/udev/cache.data
 fi
 
-- 
2.0.4

[PATCH 07/20] udev-cache: Update cache tarball atomically

[Ben Shelton]

From: Richard Tollerton <rich.tollerton@ni.com>

Overwriting the tarball in-place could cause a partial write, if the
system stops at an inopportune time. This is mitigated by first writing to a
temporary file, then moving that file on top of the final location.

Natinst-Rally-ID: TA44427
Acked-by: Gratian Crisan <gratian.crisan@ni.com>
Natinst-ReviewBoard-ID: 58620
Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>
---
 meta/recipes-core/udev/udev/udev-cache | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-core/udev/udev/udev-cache b/meta/recipes-core/udev/udev/udev-cache
index 11c5f0e..679adb4 100644
--- a/meta/recipes-core/udev/udev/udev-cache
+++ b/meta/recipes-core/udev/udev/udev-cache
@@ -25,7 +25,8 @@ fi
 
 if [ "$DEVCACHE" != "" -a -e /dev/shm/udev.cache ]; then
 	[ "${VERBOSE}" != "no" ] && echo "Populating dev cache"
-	tar czf "$DEVCACHE" dev -C / --exclude=log
+	tar czf "${DEVCACHE}.tmp" dev -C / --exclude=log
+	mv -f "${DEVCACHE}.tmp" "$DEVCACHE"
 	mv /dev/shm/udev.cache /etc/udev/cache.data
 fi
 
-- 
2.0.4

[PATCH 08/20] udev-cache: Create cache asynchronously

[Ben Shelton]

From: Richard Tollerton <rich.tollerton@ni.com>

Don't hold up the boot while the cache tarball is created.

Natinst-Rally-ID: TA44427
Acked-by: Gratian Crisan <gratian.crisan@ni.com>
Natinst-ReviewBoard-ID: 58620
Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>
---
 meta/recipes-core/udev/udev/udev-cache | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/meta/recipes-core/udev/udev/udev-cache b/meta/recipes-core/udev/udev/udev-cache
index 679adb4..e730608 100644
--- a/meta/recipes-core/udev/udev/udev-cache
+++ b/meta/recipes-core/udev/udev/udev-cache
@@ -25,9 +25,11 @@ fi
 
 if [ "$DEVCACHE" != "" -a -e /dev/shm/udev.cache ]; then
 	[ "${VERBOSE}" != "no" ] && echo "Populating dev cache"
-	tar czf "${DEVCACHE}.tmp" dev -C / --exclude=log
-	mv -f "${DEVCACHE}.tmp" "$DEVCACHE"
-	mv /dev/shm/udev.cache /etc/udev/cache.data
+	(
+		tar czf "${DEVCACHE}.tmp" dev -C / --exclude=log
+		mv -f "${DEVCACHE}.tmp" "$DEVCACHE"
+		mv /dev/shm/udev.cache /etc/udev/cache.data
+	) &
 fi
 
 exit 0
-- 
2.0.4

[PATCH 09/20] udev-cache.default: documentation update

[Ben Shelton]

From: Richard Tollerton <rich.tollerton@ni.com>

Natinst-Rally-ID: TA44427
Acked-by: Gratian Crisan <gratian.crisan@ni.com>
Natinst-ReviewBoard-ID: 58620
Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>
---
 meta/recipes-core/udev/udev/udev-cache.default | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-core/udev/udev/udev-cache.default b/meta/recipes-core/udev/udev/udev-cache.default
index a3b7326..656c2a4 100644
--- a/meta/recipes-core/udev/udev/udev-cache.default
+++ b/meta/recipes-core/udev/udev/udev-cache.default
@@ -1,4 +1,4 @@
-# Default for /etc/init.d/udev
+# Default for /etc/init.d/udev, /etc/init.d/udev-cache
 
 # Comment this out to disable device cache
 DEVCACHE="/etc/udev-cache.tar.gz"
-- 
2.0.4

[PATCH 10/20] udev-cache: parametrize sysconf file paths

[Ben Shelton]

From: Richard Tollerton <rich.tollerton@ni.com>

The udev-cache facility uses files that represent system states, to
ensure that the cache tarball is valid to apply. These paths were
hardcoded in several places; collect them into DEVCACHE_SYSCONF and
DEVCACHE_CURRENT_SYSCONF in the defaults file.

Natinst-Rally-ID: TA44427
Acked-by: Gratian Crisan <gratian.crisan@ni.com>
Natinst-ReviewBoard-ID: 58620
Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>
---
 meta/recipes-core/udev/udev/init               | 8 ++++----
 meta/recipes-core/udev/udev/udev-cache         | 4 ++--
 meta/recipes-core/udev/udev/udev-cache.default | 4 ++++
 3 files changed, 10 insertions(+), 6 deletions(-)

diff --git a/meta/recipes-core/udev/udev/init b/meta/recipes-core/udev/udev/init
index bcb9040..5db029a 100644
--- a/meta/recipes-core/udev/udev/init
+++ b/meta/recipes-core/udev/udev/init
@@ -66,13 +66,13 @@ case "$1" in
             if [ -e $DEVCACHE ]; then
 		    readfiles $CMP_FILE_LIST
 		    NEWDATA="$READDATA"
-		    readfiles /etc/udev/cache.data
+		    readfiles "$DEVCACHE_SYSCONF"
 		    OLDDATA="$READDATA"
 		    if [ "$OLDDATA" = "$NEWDATA" ]; then
                             tar xzf $DEVCACHE -C / --exclude=log
                             not_first_boot=1
                             [ "$VERBOSE" != "no" ] && echo "udev: using cache file $DEVCACHE"
-                            [ -e /dev/shm/udev.cache ] && rm -f /dev/shm/udev.cache
+                            [ -e "$DEVCACHE_CURRENT_SYSCONF" ] && rm -f "$DEVCACHE_CURRENT_SYSCONF"
                     else
 			    # Output detailed reason why the cached /dev is not used
 			    if [ "$VERBOSE" != "no" ]; then
@@ -81,14 +81,14 @@ case "$1" in
 				    echo "udev: olddata: $OLDDATA"
 				    echo "udev: newdata: $NEWDATA"
 			    fi
-			    echo "$NEWDATA" > /dev/shm/udev.cache
+			    echo "$NEWDATA" > "$DEVCACHE_CURRENT_SYSCONF"
                     fi
 	    else
 		    if [ "$ROOTFS_READ_ONLY" != "yes" ]; then
 			    # If rootfs is not read-only, it's possible that a new udev cache would be generated;
 			    # otherwise, we do not bother to read files.
 			    readfiles $CMP_FILE_LIST
-			    echo "$READDATA" > /dev/shm/udev.cache
+			    echo "$READDATA" > "$DEVCACHE_CURRENT_SYSCONF"
 		    fi
             fi
     fi
diff --git a/meta/recipes-core/udev/udev/udev-cache b/meta/recipes-core/udev/udev/udev-cache
index e730608..7406e07 100644
--- a/meta/recipes-core/udev/udev/udev-cache
+++ b/meta/recipes-core/udev/udev/udev-cache
@@ -23,12 +23,12 @@ if [ "$ROOTFS_READ_ONLY" = "yes" ]; then
     exit 0
 fi
 
-if [ "$DEVCACHE" != "" -a -e /dev/shm/udev.cache ]; then
+if [ "$DEVCACHE" != "" -a -e "$DEVCACHE_CURRENT_SYSCONF" ]; then
 	[ "${VERBOSE}" != "no" ] && echo "Populating dev cache"
 	(
 		tar czf "${DEVCACHE}.tmp" dev -C / --exclude=log
 		mv -f "${DEVCACHE}.tmp" "$DEVCACHE"
-		mv /dev/shm/udev.cache /etc/udev/cache.data
+		mv "$DEVCACHE_CURRENT_SYSCONF" "$DEVCACHE_SYSCONF"
 	) &
 fi
 
diff --git a/meta/recipes-core/udev/udev/udev-cache.default b/meta/recipes-core/udev/udev/udev-cache.default
index 656c2a4..7f39a68 100644
--- a/meta/recipes-core/udev/udev/udev-cache.default
+++ b/meta/recipes-core/udev/udev/udev-cache.default
@@ -2,4 +2,8 @@
 
 # Comment this out to disable device cache
 DEVCACHE="/etc/udev-cache.tar.gz"
+
+DEVCACHE_SYSCONF="/etc/udev/cache.data"
+DEVCACHE_CURRENT_SYSCONF="/dev/shm/udev.cache"
+
 PROBE_PLATFORM_BUS="yes"
-- 
2.0.4

[PATCH 11/20] udev-cache: parametrize tar options

[Ben Shelton]

From: Richard Tollerton <rich.tollerton@ni.com>

The options to pass to `tar` for cache create/extract are likely to
change due to system configuration circumstances (e.g. using busybox tar
instead of GNU tar). So move the more detailed options into
the defaults file to allow for easier overrides.

Natinst-Rally-ID: TA44427
Acked-by: Gratian Crisan <gratian.crisan@ni.com>
Natinst-ReviewBoard-ID: 58620
Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>
---
 meta/recipes-core/udev/udev/init               | 2 +-
 meta/recipes-core/udev/udev/udev-cache         | 2 +-
 meta/recipes-core/udev/udev/udev-cache.default | 4 ++++
 3 files changed, 6 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-core/udev/udev/init b/meta/recipes-core/udev/udev/init
index 5db029a..dc39d95 100644
--- a/meta/recipes-core/udev/udev/init
+++ b/meta/recipes-core/udev/udev/init
@@ -69,7 +69,7 @@ case "$1" in
 		    readfiles "$DEVCACHE_SYSCONF"
 		    OLDDATA="$READDATA"
 		    if [ "$OLDDATA" = "$NEWDATA" ]; then
-                            tar xzf $DEVCACHE -C / --exclude=log
+                            tar xzf $DEVCACHE -C / $DEVCACHE_EXTRACT_OPTS
                             not_first_boot=1
                             [ "$VERBOSE" != "no" ] && echo "udev: using cache file $DEVCACHE"
                             [ -e "$DEVCACHE_CURRENT_SYSCONF" ] && rm -f "$DEVCACHE_CURRENT_SYSCONF"
diff --git a/meta/recipes-core/udev/udev/udev-cache b/meta/recipes-core/udev/udev/udev-cache
index 7406e07..29700e0 100644
--- a/meta/recipes-core/udev/udev/udev-cache
+++ b/meta/recipes-core/udev/udev/udev-cache
@@ -26,7 +26,7 @@ fi
 if [ "$DEVCACHE" != "" -a -e "$DEVCACHE_CURRENT_SYSCONF" ]; then
 	[ "${VERBOSE}" != "no" ] && echo "Populating dev cache"
 	(
-		tar czf "${DEVCACHE}.tmp" dev -C / --exclude=log
+		tar czf "${DEVCACHE}.tmp" dev -C / $DEVCACHE_CREATE_OPTS
 		mv -f "${DEVCACHE}.tmp" "$DEVCACHE"
 		mv "$DEVCACHE_CURRENT_SYSCONF" "$DEVCACHE_SYSCONF"
 	) &
diff --git a/meta/recipes-core/udev/udev/udev-cache.default b/meta/recipes-core/udev/udev/udev-cache.default
index 7f39a68..30d7faa 100644
--- a/meta/recipes-core/udev/udev/udev-cache.default
+++ b/meta/recipes-core/udev/udev/udev-cache.default
@@ -6,4 +6,8 @@ DEVCACHE="/etc/udev-cache.tar.gz"
 DEVCACHE_SYSCONF="/etc/udev/cache.data"
 DEVCACHE_CURRENT_SYSCONF="/dev/shm/udev.cache"
 
+# - Avoid /dev/log because it's a pipe.
+DEVCACHE_EXTRACT_OPTS="--exclude=log"
+DEVCACHE_CREATE_OPTS="--exclude=log"
+
 PROBE_PLATFORM_BUS="yes"
-- 
2.0.4

[PATCH 12/20] udev-cache: fix timestamp errors on systems lacking an RTC

[Ben Shelton]

From: Richard Tollerton <rich.tollerton@ni.com>

If the system lacks an RTC, the time will typically start from the
epoch, and may get set to the "correct" time later (if at all).
In this circumstance, the timestamps in the cache tarball will always be
in the future. gnutar complains bitterly about this.

To work around this, use `tar -m` to strip mtimes during extract.

Natinst-Rally-ID: TA44427
Acked-by: Gratian Crisan <gratian.crisan@ni.com>
Natinst-ReviewBoard-ID: 58620
Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>
---
 meta/recipes-core/udev/udev/udev-cache.default | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-core/udev/udev/udev-cache.default b/meta/recipes-core/udev/udev/udev-cache.default
index 30d7faa..554142a 100644
--- a/meta/recipes-core/udev/udev/udev-cache.default
+++ b/meta/recipes-core/udev/udev/udev-cache.default
@@ -7,7 +7,8 @@ DEVCACHE_SYSCONF="/etc/udev/cache.data"
 DEVCACHE_CURRENT_SYSCONF="/dev/shm/udev.cache"
 
 # - Avoid /dev/log because it's a pipe.
-DEVCACHE_EXTRACT_OPTS="--exclude=log"
+# - Don't restore mtimes. Avoids errors on systems lacking an RTC.
+DEVCACHE_EXTRACT_OPTS="--exclude=log -m"
 DEVCACHE_CREATE_OPTS="--exclude=log"
 
 PROBE_PLATFORM_BUS="yes"
-- 
2.0.4

[PATCH 13/20] udev-cache: Avoid caching udev.cache or non-devfs filesystems

[Ben Shelton]

From: Richard Tollerton <rich.tollerton@ni.com>

If $DEVCACHE_CURRENT_SYSCONF gets stored in the cache tarball, and
extracted, then udev-cache will needlessly rebuild the cache on every
future boot. The direct fix to this is to explicitly exclude it.

Investigating this also uncovered that we're also archiving everything
under other filesystems, including /dev/cgroup, /dev/shm, etc. This
shouldn't be happening. The fix is to use `tar --one-file-system`, but
this is GNU-specific.

Natinst-Rally-ID: TA44427
Acked-by: Gratian Crisan <gratian.crisan@ni.com>
Natinst-ReviewBoard-ID: 58620
Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>
---
 meta/recipes-core/udev/udev/udev-cache.default | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-core/udev/udev/udev-cache.default b/meta/recipes-core/udev/udev/udev-cache.default
index 554142a..c71826e 100644
--- a/meta/recipes-core/udev/udev/udev-cache.default
+++ b/meta/recipes-core/udev/udev/udev-cache.default
@@ -8,7 +8,8 @@ DEVCACHE_CURRENT_SYSCONF="/dev/shm/udev.cache"
 
 # - Avoid /dev/log because it's a pipe.
 # - Don't restore mtimes. Avoids errors on systems lacking an RTC.
-DEVCACHE_EXTRACT_OPTS="--exclude=log -m"
-DEVCACHE_CREATE_OPTS="--exclude=log"
+# - (gnutar specific) Stay on the same fs.
+DEVCACHE_EXTRACT_OPTS="--exclude=log -m --exclude=udev.cache --one-file-system"
+DEVCACHE_CREATE_OPTS="--exclude=log --exclude=udev.cache --one-file-system"
 
 PROBE_PLATFORM_BUS="yes"
-- 
2.0.4

[PATCH 14/20] udev-cache: refactor; improve verbosity and error handling

[Ben Shelton]

From: Richard Tollerton <rich.tollerton@ni.com>

Most of /etc/init.d/udev-cache is in a conditional block which can be
replaced by a `[ ... ] || exit 0` to reduce nesting. This eliminates
a `rm -f` which was dead code anyway.

This also provides the opportunity to emit a few more messages if
VERBOSE is set, which may help with future debugging.

Finally, make installing the new $DEVCACHE_SYSCONF contingent on
successfully creating the cache tarball.

Natinst-Rally-ID: TA44427
Acked-by: Gratian Crisan <gratian.crisan@ni.com>
Natinst-ReviewBoard-ID: 58620
Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>
---
 meta/recipes-core/udev/udev/udev-cache | 16 +++++++++-------
 1 file changed, 9 insertions(+), 7 deletions(-)

diff --git a/meta/recipes-core/udev/udev/udev-cache b/meta/recipes-core/udev/udev/udev-cache
index 29700e0..7eea5fb 100644
--- a/meta/recipes-core/udev/udev/udev-cache
+++ b/meta/recipes-core/udev/udev/udev-cache
@@ -23,13 +23,15 @@ if [ "$ROOTFS_READ_ONLY" = "yes" ]; then
     exit 0
 fi
 
-if [ "$DEVCACHE" != "" -a -e "$DEVCACHE_CURRENT_SYSCONF" ]; then
-	[ "${VERBOSE}" != "no" ] && echo "Populating dev cache"
-	(
-		tar czf "${DEVCACHE}.tmp" dev -C / $DEVCACHE_CREATE_OPTS
-		mv -f "${DEVCACHE}.tmp" "$DEVCACHE"
+[ "$DEVCACHE" != "" ] || exit 0
+[ "${VERBOSE}" == "no" ] || echo -n "$0: checking for ${DEVCACHE_CURRENT_SYSCONF}... "
+[ -e "$DEVCACHE_CURRENT_SYSCONF" ] || { echo "not found"; exit 0; }
+[ "${VERBOSE}" == "no" ] || echo "found; building cache."
+
+(
+	tar czf "${DEVCACHE}.tmp" dev -C / $DEVCACHE_CREATE_OPTS && \
+		mv -f "${DEVCACHE}.tmp" "$DEVCACHE" && \
 		mv "$DEVCACHE_CURRENT_SYSCONF" "$DEVCACHE_SYSCONF"
-	) &
-fi
+) &
 
 exit 0
-- 
2.0.4

[PATCH 15/20] udev-cache: don't attempt to extract cache if it doesn't exist

[Ben Shelton]

From: Richard Tollerton <rich.tollerton@ni.com>

If you try to invalidate the udev cache solely by deleting the tarball,
while forgetting to also remove the config file ($DEVCACHE_SYSCONF),
then tar will fail.

Fortunately the cache will get regenerated on this boot because
$DEVCACHE_CURRENT_SYSCONF still exists, but to avoid the error, we
should check to see if the tarball actually exists before we try to
extract it.

Natinst-Rally-ID: TA44427
Acked-by: Gratian Crisan <gratian.crisan@ni.com>
Natinst-ReviewBoard-ID: 58620
Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>
---
 meta/recipes-core/udev/udev/init | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-core/udev/udev/init b/meta/recipes-core/udev/udev/init
index dc39d95..e4a196f 100644
--- a/meta/recipes-core/udev/udev/init
+++ b/meta/recipes-core/udev/udev/init
@@ -68,7 +68,7 @@ case "$1" in
 		    NEWDATA="$READDATA"
 		    readfiles "$DEVCACHE_SYSCONF"
 		    OLDDATA="$READDATA"
-		    if [ "$OLDDATA" = "$NEWDATA" ]; then
+		    if [ "$OLDDATA" = "$NEWDATA" -a -f "$DEVCACHE" ]; then
                             tar xzf $DEVCACHE -C / $DEVCACHE_EXTRACT_OPTS
                             not_first_boot=1
                             [ "$VERBOSE" != "no" ] && echo "udev: using cache file $DEVCACHE"
-- 
2.0.4

[PATCH 16/20] udev-cache: invalidate on rules.d changes

[Ben Shelton]

From: Richard Tollerton <rich.tollerton@ni.com>

Presently, $DEVCACHE_CURRENT_SYSCONF doesn't change if rules are
modified, which may cause the cache to preserve an old configuration.
To fix, include `ls -l` output for all rules, which ought to be
reasonably fast to run on boot.

Natinst-Rally-ID: TA44427
Acked-by: Gratian Crisan <gratian.crisan@ni.com>
Natinst-ReviewBoard-ID: 58620
Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>
---
 meta/recipes-core/udev/udev/init | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/meta/recipes-core/udev/udev/init b/meta/recipes-core/udev/udev/init
index e4a196f..3a52e98 100644
--- a/meta/recipes-core/udev/udev/init
+++ b/meta/recipes-core/udev/udev/init
@@ -63,9 +63,10 @@ case "$1" in
     # A list of files which are used as a criteria to judge whether the udev cache could be reused.
     CMP_FILE_LIST="/proc/version /proc/cmdline /proc/devices /proc/atags"
     if [ "$DEVCACHE" != "" ]; then
+	    RULELIST=`ls -l /lib/udev/rules.d/* /etc/udev/rules.d/* 2>/dev/null`
             if [ -e $DEVCACHE ]; then
 		    readfiles $CMP_FILE_LIST
-		    NEWDATA="$READDATA"
+		    NEWDATA="$READDATA$RULELIST"
 		    readfiles "$DEVCACHE_SYSCONF"
 		    OLDDATA="$READDATA"
 		    if [ "$OLDDATA" = "$NEWDATA" -a -f "$DEVCACHE" ]; then
@@ -77,7 +78,7 @@ case "$1" in
 			    # Output detailed reason why the cached /dev is not used
 			    if [ "$VERBOSE" != "no" ]; then
 				    echo "udev: udev cache not used"
-				    echo "udev: we use $CMP_FILE_LIST as criteria to judge whether the cache /dev could be resued"
+				    echo "udev: we use $CMP_FILE_LIST and ls -l [/lib][etc]/udev/rules.d/* as criteria to judge whether the cached /dev could be reused"
 				    echo "udev: olddata: $OLDDATA"
 				    echo "udev: newdata: $NEWDATA"
 			    fi
@@ -88,7 +89,7 @@ case "$1" in
 			    # If rootfs is not read-only, it's possible that a new udev cache would be generated;
 			    # otherwise, we do not bother to read files.
 			    readfiles $CMP_FILE_LIST
-			    echo "$READDATA" > "$DEVCACHE_CURRENT_SYSCONF"
+			    echo "$READDATA$RULELIST" > "$DEVCACHE_CURRENT_SYSCONF"
 		    fi
             fi
     fi
-- 
2.0.4

[PATCH 17/20] udev-cache: fix udev-cache changes to work with busybox tar

[Ben Shelton]

busybox tar does not support the --one-file-system option that we use to
keep tar from archiving unnecessary files that live under /dev but are
not mounted under the devtmpfs.  Instead, when GNU tar is not present,
use find to pass tar a list of the files that need to be cached.

Additionally, busybox tar does not complain when timestamps are in the
future, and the OE busybox defconfig does not have the
TAR_NOPRESERVE_TIME option enabled, so don't use the '-m' option with
busybox tar.

Signed-off-by: Ben Shelton <ben.shelton@ni.com>
---
 meta/recipes-core/udev/udev/init               | 14 +++++++++-----
 meta/recipes-core/udev/udev/udev-cache         | 12 ++++++++++--
 meta/recipes-core/udev/udev/udev-cache.default |  4 ++++
 3 files changed, 23 insertions(+), 7 deletions(-)

diff --git a/meta/recipes-core/udev/udev/init b/meta/recipes-core/udev/udev/init
index 3a52e98..6caf7d5 100644
--- a/meta/recipes-core/udev/udev/init
+++ b/meta/recipes-core/udev/udev/init
@@ -70,11 +70,15 @@ case "$1" in
 		    readfiles "$DEVCACHE_SYSCONF"
 		    OLDDATA="$READDATA"
 		    if [ "$OLDDATA" = "$NEWDATA" -a -f "$DEVCACHE" ]; then
-                            tar xzf $DEVCACHE -C / $DEVCACHE_EXTRACT_OPTS
-                            not_first_boot=1
-                            [ "$VERBOSE" != "no" ] && echo "udev: using cache file $DEVCACHE"
-                            [ -e "$DEVCACHE_CURRENT_SYSCONF" ] && rm -f "$DEVCACHE_CURRENT_SYSCONF"
-                    else
+			    if [ ! `readlink -f /bin/tar` = "/bin/tar.tar" ]; then
+				    tar xzf $DEVCACHE -C / $BUSYBOX_DEVCACHE_EXTRACT_OPTS
+			    else
+				    tar xzf $DEVCACHE -C / $DEVCACHE_EXTRACT_OPTS
+			    fi
+			    not_first_boot=1
+			    [ "$VERBOSE" != "no" ] && echo "udev: using cache file $DEVCACHE"
+			    [ -e "$DEVCACHE_CURRENT_SYSCONF" ] && rm -f "$DEVCACHE_CURRENT_SYSCONF"
+		    else
 			    # Output detailed reason why the cached /dev is not used
 			    if [ "$VERBOSE" != "no" ]; then
 				    echo "udev: udev cache not used"
diff --git a/meta/recipes-core/udev/udev/udev-cache b/meta/recipes-core/udev/udev/udev-cache
index 7eea5fb..43bc67a 100644
--- a/meta/recipes-core/udev/udev/udev-cache
+++ b/meta/recipes-core/udev/udev/udev-cache
@@ -28,10 +28,18 @@ fi
 [ -e "$DEVCACHE_CURRENT_SYSCONF" ] || { echo "not found"; exit 0; }
 [ "${VERBOSE}" == "no" ] || echo "found; building cache."
 
-(
+if [ ! `readlink -f /bin/tar` = "/bin/tar.tar" ]; then
+	(
+	find /dev \( -type b -o -type c -o -type l \) -xdev | cut -c 2- | xargs tar czf "${DEVCACHE}.tmp" -C / $BUSYBOX_DEVCACHE_CREATE_OPTS && \
+		mv -f "${DEVCACHE}.tmp" "$DEVCACHE" && \
+		mv "$DEVCACHE_CURRENT_SYSCONF" "$DEVCACHE_SYSCONF"
+	) &
+else
+	(
 	tar czf "${DEVCACHE}.tmp" dev -C / $DEVCACHE_CREATE_OPTS && \
 		mv -f "${DEVCACHE}.tmp" "$DEVCACHE" && \
 		mv "$DEVCACHE_CURRENT_SYSCONF" "$DEVCACHE_SYSCONF"
-) &
+	) &
+fi
 
 exit 0
diff --git a/meta/recipes-core/udev/udev/udev-cache.default b/meta/recipes-core/udev/udev/udev-cache.default
index c71826e..d2c1e5c 100644
--- a/meta/recipes-core/udev/udev/udev-cache.default
+++ b/meta/recipes-core/udev/udev/udev-cache.default
@@ -12,4 +12,8 @@ DEVCACHE_CURRENT_SYSCONF="/dev/shm/udev.cache"
 DEVCACHE_EXTRACT_OPTS="--exclude=log -m --exclude=udev.cache --one-file-system"
 DEVCACHE_CREATE_OPTS="--exclude=log --exclude=udev.cache --one-file-system"
 
+# Busybox tar does not support --one-file-system option
+BUSYBOX_DEVCACHE_EXTRACT_OPTS="--exclude=log --exclude=udev.cache"
+BUSYBOX_DEVCACHE_CREATE_OPTS="--exclude=log --exclude=udev.cache"
+
 PROBE_PLATFORM_BUS="yes"
-- 
2.0.4

[PATCH 18/20] udev: Make build MACHINE-specific

[Ben Shelton]

From: Richard Tollerton <rich.tollerton@ni.com>

For space reasons, udev needs to be configured differently depending on
MACHINE_FEATURES. So set PACKAGE_ARCH accordingly.

Natinst-Rally-ID: TA44427
Acked-by: Gratian Crisan <gratian.crisan@ni.com>
Natinst-ReviewBoard-ID: 58621
Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>
---
 meta/recipes-core/udev/udev.inc | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/meta/recipes-core/udev/udev.inc b/meta/recipes-core/udev/udev.inc
index 11204aa..8394064 100644
--- a/meta/recipes-core/udev/udev.inc
+++ b/meta/recipes-core/udev/udev.inc
@@ -15,6 +15,12 @@ LDFLAGS += "-lrt"
 DEPENDS = "acl glib-2.0 libusb usbutils pciutils gperf-native libxslt-native util-linux"
 RPROVIDES_${PN} = "hotplug"
 
+#
+# packages which content depend on MACHINE_FEATURES/COMBINED_FEATURES need to be MACHINE_ARCH
+#
+PACKAGE_ARCH = "${MACHINE_ARCH}"
+
+
 SRC_URI = "${KERNELORG_MIRROR}/linux/utils/kernel/hotplug/udev-${PV}.tar.gz \
            file://0001-Fixing-keyboard_force_release.sh-shell-script-path.patch \
            file://avoid-mouse-autosuspend.patch \
-- 
2.0.4

[PATCH 19/20] udev: Disable keymap support on machines lacking keyboard support

[Ben Shelton]

From: Richard Tollerton <rich.tollerton@ni.com>

/lib/udev/keymaps/ is 244k. On headless machines, this is wasted space.

Natinst-Rally-ID: TA44427
Acked-by: Gratian Crisan <gratian.crisan@ni.com>
Natinst-ReviewBoard-ID: 58621
Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>
---
 meta/recipes-core/udev/udev.inc | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/recipes-core/udev/udev.inc b/meta/recipes-core/udev/udev.inc
index 8394064..2d2150c 100644
--- a/meta/recipes-core/udev/udev.inc
+++ b/meta/recipes-core/udev/udev.inc
@@ -51,6 +51,7 @@ EXTRA_OECONF = "--disable-introspection \
                 --with-rootlibdir=${base_libdir} \
                 --with-rootprefix= \
                 --without-systemdsystemunitdir \
+                ${@base_contains('MACHINE_FEATURES', 'keyboard', '', '--disable-keymap', d)} \
                "
 
 PACKAGES =+ "udev-cache"
-- 
2.0.4

[PATCH 20/20] udev: don't halt if devtmpfs can't be mounted

[Ben Shelton]

From: Richard Tollerton <rich.tollerton@ni.com>

Halting if udev cannot start is problematic for a couple reasons.
- Compared to a reboot loop, halting is much more difficult to recover
  from in remote deployments.
- If the rootfs has a prepopulated /dev (which happens somewhat often),
  the system may be able to boot up just fine, or at least fine enough
  for the administrator to log in and fix things.

Instead of halting, just exit, and let the administrator deal with the
potential reboot loop.

Natinst-Rally-ID: TA44427
Acked-by: Gratian Crisan <gratian.crisan@ni.com>
Acked-by: Scot Salmon <scot.salmon@ni.com>
Natinst-ReviewBoard-ID: 58623
Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>
---
 meta/recipes-core/udev/udev/init | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/meta/recipes-core/udev/udev/init b/meta/recipes-core/udev/udev/init
index 6caf7d5..f6a8391 100644
--- a/meta/recipes-core/udev/udev/init
+++ b/meta/recipes-core/udev/udev/init
@@ -45,8 +45,7 @@ case "$1" in
     if ! grep -q devtmpfs /proc/filesystems
     then
         echo "Missing devtmpfs, which is required for udev to run";
-        echo "Halting..."
-        halt
+	exit 1
     fi
     # mount the devtmpfs on /dev, if not already done
     LANG=C awk '$2 == "/dev" && ($3 == "devtmpfs") { exit 1 }' /proc/mounts && {
-- 
2.0.4

Re: [PATCH 01/20] udev-cache: Compress the cache

[Otavio Salvador]

On Mon, Aug 4, 2014 at 3:40 PM, Ben Shelton <ben.shelton@ni.com> wrote:
> From: Richard Tollerton <rich.tollerton@ni.com>
>
> $DEVCACHE is observed to be 100k uncompressed; compressing it reduces
> its size to ~5k.
>
> Natinst-Rally-ID: TA44427
> Acked-by: Gratian Crisan <gratian.crisan@ni.com>
> Natinst-ReviewBoard-ID: 58620
> Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>

Acked-by: Otavio Salvador <otavio@ossystems.com.br>

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: [PATCH 02/20] udev-cache: choose a more descriptive cache filename

[Otavio Salvador]

On Mon, Aug 4, 2014 at 3:40 PM, Ben Shelton <ben.shelton@ni.com> wrote:
> From: Richard Tollerton <rich.tollerton@ni.com>
>
> "/etc/dev.tar.gz" doesn't adequately imply that udev-cache
> maintains it. Instead, call it "/etc/udev-cache.tar.gz".
>
> Natinst-Rally-ID: TA44427
> Acked-by: Gratian Crisan <gratian.crisan@ni.com>
> Natinst-ReviewBoard-ID: 58620
> Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>
> ---


Acked-by: Otavio Salvador <otavio@ossystems.com.br>

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: [PATCH 03/20] udev-cache: Honor VERBOSE for bootup message

[Otavio Salvador]

On Mon, Aug 4, 2014 at 3:40 PM, Ben Shelton <ben.shelton@ni.com> wrote:
> From: Richard Tollerton <rich.tollerton@ni.com>
>
> Natinst-Rally-ID: TA44427
> Acked-by: Gratian Crisan <gratian.crisan@ni.com>
> Natinst-ReviewBoard-ID: 58620
> Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>
> ---


Acked-by: Otavio Salvador <otavio@ossystems.com.br>

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: [PATCH 04/20] udev-cache: Don't ignore errors from cache extract

[Otavio Salvador]

On Mon, Aug 4, 2014 at 3:40 PM, Ben Shelton <ben.shelton@ni.com> wrote:
> From: Richard Tollerton <rich.tollerton@ni.com>
>
> Currently, error messages generated during tarball extract are
> suppressed. This seems ham-handed, particularly since under normal
> operation, the only expected error ought to be the attempted extraction
> of pipes, which typically only means /dev/log.
>
> So stop the redirections and add an --exclude=log to suppress the error
> message when attempting to extract /dev/log.
>
> This requires CONFIG_FEATURE_TAR_LONG_OPTIONS if using busybox tar.
>
> Natinst-Rally-ID: TA44427
> Acked-by: Gratian Crisan <gratian.crisan@ni.com>
> Natinst-ReviewBoard-ID: 58620
> Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>
> ---


Acked-by: Otavio Salvador <otavio@ossystems.com.br>

I didn't test this at runtime so my only question is did you check if
we provide this option by default in busybox?

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: [PATCH 05/20] busybox: tar: enable CONFIG_FEATURE_TAR_LONG_OPTIONS

[Otavio Salvador]

On Mon, Aug 4, 2014 at 3:40 PM, Ben Shelton <ben.shelton@ni.com> wrote:
> This config flag is needed to support the --exclude option, which is
> needed for the commit 'udev-cache: Don't ignore errors from cache
> extract'.
>
> Enabling this option adds 352 bytes to the size of busybox on i386.
>
> Signed-off-by: Ben Shelton <ben.shelton@ni.com>


Acked-by: Otavio Salvador <otavio@ossystems.com.br>

You answered my question. Please invert the order of this patch to
come before 04/20 or it may cause runtime error if someone builds an
image on this on previous patch.

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: [PATCH 06/20] udev-cache: Remove superfluous subshell on extract

[Otavio Salvador]

On Mon, Aug 4, 2014 at 3:40 PM, Ben Shelton <ben.shelton@ni.com> wrote:
> From: Richard Tollerton <rich.tollerton@ni.com>
>
> tar -C exists in both GNU and busybox tar, so use that instead of
> (cd /; tar ...). This allows the subshell to be removed.
>
> Natinst-Rally-ID: TA44427
> Acked-by: Gratian Crisan <gratian.crisan@ni.com>
> Natinst-ReviewBoard-ID: 58620
> Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>
> ---


Acked-by: Otavio Salvador <otavio@ossystems.com.br>

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: [PATCH 07/20] udev-cache: Update cache tarball atomically

[Otavio Salvador]

On Mon, Aug 4, 2014 at 3:40 PM, Ben Shelton <ben.shelton@ni.com> wrote:
> From: Richard Tollerton <rich.tollerton@ni.com>
>
> Overwriting the tarball in-place could cause a partial write, if the
> system stops at an inopportune time. This is mitigated by first writing to a
> temporary file, then moving that file on top of the final location.
>
> Natinst-Rally-ID: TA44427
> Acked-by: Gratian Crisan <gratian.crisan@ni.com>
> Natinst-ReviewBoard-ID: 58620
> Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>


Acked-by: Otavio Salvador <otavio@ossystems.com.br>

As an improvement, this could be written in tmpfs so it avoid
duplicated writting. For NAND devices without much free space this
might matter.

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: [PATCH 20/20] udev: don't halt if devtmpfs can't be mounted

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 1858 bytes --]

On Mon, Aug 04, 2014 at 01:41:12PM -0500, Ben Shelton wrote:
> From: Richard Tollerton <rich.tollerton@ni.com>
> 
> Halting if udev cannot start is problematic for a couple reasons.
> - Compared to a reboot loop, halting is much more difficult to recover
>   from in remote deployments.
> - If the rootfs has a prepopulated /dev (which happens somewhat often),
>   the system may be able to boot up just fine, or at least fine enough
>   for the administrator to log in and fix things.
> 
> Instead of halting, just exit, and let the administrator deal with the
> potential reboot loop.
> 
> Natinst-Rally-ID: TA44427
> Acked-by: Gratian Crisan <gratian.crisan@ni.com>
> Acked-by: Scot Salmon <scot.salmon@ni.com>
> Natinst-ReviewBoard-ID: 58623
> Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>
> ---
>  meta/recipes-core/udev/udev/init | 3 +--
>  1 file changed, 1 insertion(+), 2 deletions(-)
> 
> diff --git a/meta/recipes-core/udev/udev/init b/meta/recipes-core/udev/udev/init
> index 6caf7d5..f6a8391 100644
> --- a/meta/recipes-core/udev/udev/init
> +++ b/meta/recipes-core/udev/udev/init
> @@ -45,8 +45,7 @@ case "$1" in
>      if ! grep -q devtmpfs /proc/filesystems
>      then
>          echo "Missing devtmpfs, which is required for udev to run";
> -        echo "Halting..."
> -        halt
> +	exit 1

Please use spaces for indentation like previous lines.

>      fi
>      # mount the devtmpfs on /dev, if not already done
>      LANG=C awk '$2 == "/dev" && ($3 == "devtmpfs") { exit 1 }' /proc/mounts && {
> -- 
> 2.0.4
> 
> -- 
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 188 bytes --]

Re: [PATCH 08/20] udev-cache: Create cache asynchronously

[Otavio Salvador]

On Mon, Aug 4, 2014 at 3:41 PM, Ben Shelton <ben.shelton@ni.com> wrote:
> From: Richard Tollerton <rich.tollerton@ni.com>
>
> Don't hold up the boot while the cache tarball is created.
>
> Natinst-Rally-ID: TA44427
> Acked-by: Gratian Crisan <gratian.crisan@ni.com>
> Natinst-ReviewBoard-ID: 58620
> Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>

I am not sure about this one. I see the value you are adding here but
I worry how often something can be connected during this process and
change the contents along the way. Did you see something as that
during your tests?

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: [PATCH 19/20] udev: Disable keymap support on machines lacking keyboard support

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 1495 bytes --]

On Mon, Aug 04, 2014 at 01:41:11PM -0500, Ben Shelton wrote:
> From: Richard Tollerton <rich.tollerton@ni.com>
> 
> /lib/udev/keymaps/ is 244k. On headless machines, this is wasted space.
> 
> Natinst-Rally-ID: TA44427
> Acked-by: Gratian Crisan <gratian.crisan@ni.com>
> Natinst-ReviewBoard-ID: 58621
> Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>
> ---
>  meta/recipes-core/udev/udev.inc | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/meta/recipes-core/udev/udev.inc b/meta/recipes-core/udev/udev.inc
> index 8394064..2d2150c 100644
> --- a/meta/recipes-core/udev/udev.inc
> +++ b/meta/recipes-core/udev/udev.inc
> @@ -51,6 +51,7 @@ EXTRA_OECONF = "--disable-introspection \
>                  --with-rootlibdir=${base_libdir} \
>                  --with-rootprefix= \
>                  --without-systemdsystemunitdir \
> +                ${@base_contains('MACHINE_FEATURES', 'keyboard', '', '--disable-keymap', d)} \

udev isn't MACHINE_ARCH, so you cannot use MACHINE_FEATURES there.

But you can add PACKAGECONFIG and let DISTRO decide if all supported
MACHINEs are headless, it can disable it.

>                 "
>  
>  PACKAGES =+ "udev-cache"
> -- 
> 2.0.4
> 
> -- 
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 188 bytes --]

Re: [PATCH 09/20] udev-cache.default: documentation update

[Otavio Salvador]

On Mon, Aug 4, 2014 at 3:41 PM, Ben Shelton <ben.shelton@ni.com> wrote:
> From: Richard Tollerton <rich.tollerton@ni.com>
>
> Natinst-Rally-ID: TA44427
> Acked-by: Gratian Crisan <gratian.crisan@ni.com>
> Natinst-ReviewBoard-ID: 58620
> Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>
> ---


Acked-by: Otavio Salvador <otavio@ossystems.com.br>

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: [PATCH 18/20] udev: Make build MACHINE-specific

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 1756 bytes --]

On Mon, Aug 04, 2014 at 01:41:10PM -0500, Ben Shelton wrote:
> From: Richard Tollerton <rich.tollerton@ni.com>
> 
> For space reasons, udev needs to be configured differently depending on
> MACHINE_FEATURES. So set PACKAGE_ARCH accordingly.
> 
> Natinst-Rally-ID: TA44427
> Acked-by: Gratian Crisan <gratian.crisan@ni.com>
> Natinst-ReviewBoard-ID: 58621
> Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>
> ---
>  meta/recipes-core/udev/udev.inc | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/meta/recipes-core/udev/udev.inc b/meta/recipes-core/udev/udev.inc
> index 11204aa..8394064 100644
> --- a/meta/recipes-core/udev/udev.inc
> +++ b/meta/recipes-core/udev/udev.inc
> @@ -15,6 +15,12 @@ LDFLAGS += "-lrt"
>  DEPENDS = "acl glib-2.0 libusb usbutils pciutils gperf-native libxslt-native util-linux"
>  RPROVIDES_${PN} = "hotplug"
>  
> +#
> +# packages which content depend on MACHINE_FEATURES/COMBINED_FEATURES need to be MACHINE_ARCH
> +#
> +PACKAGE_ARCH = "${MACHINE_ARCH}"

A lot of recipes are depending on udev and aren't MACHINE_ARCH (this
would make them effectively MACHINE_ARCH).

e.g. bluez4, ofono, clutter, mesa, pulseaudio, ... and everything
depending on them as well.

>  SRC_URI = "${KERNELORG_MIRROR}/linux/utils/kernel/hotplug/udev-${PV}.tar.gz \
>             file://0001-Fixing-keyboard_force_release.sh-shell-script-path.patch \
>             file://avoid-mouse-autosuspend.patch \
> -- 
> 2.0.4
> 
> -- 
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 188 bytes --]

Re: [PATCH 10/20] udev-cache: parametrize sysconf file paths

[Otavio Salvador]

On Mon, Aug 4, 2014 at 3:41 PM, Ben Shelton <ben.shelton@ni.com> wrote:
> From: Richard Tollerton <rich.tollerton@ni.com>
>
> The udev-cache facility uses files that represent system states, to
> ensure that the cache tarball is valid to apply. These paths were
> hardcoded in several places; collect them into DEVCACHE_SYSCONF and
> DEVCACHE_CURRENT_SYSCONF in the defaults file.
>
> Natinst-Rally-ID: TA44427
> Acked-by: Gratian Crisan <gratian.crisan@ni.com>
> Natinst-ReviewBoard-ID: 58620
> Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>
> ---
...
> --- a/meta/recipes-core/udev/udev/udev-cache
> +++ b/meta/recipes-core/udev/udev/udev-cache
> @@ -23,12 +23,12 @@ if [ "$ROOTFS_READ_ONLY" = "yes" ]; then
>      exit 0
>  fi
>
> -if [ "$DEVCACHE" != "" -a -e /dev/shm/udev.cache ]; then
> +if [ "$DEVCACHE" != "" -a -e "$DEVCACHE_CURRENT_SYSCONF" ]; then
>         [ "${VERBOSE}" != "no" ] && echo "Populating dev cache"
>         (
>                 tar czf "${DEVCACHE}.tmp" dev -C / --exclude=log
>                 mv -f "${DEVCACHE}.tmp" "$DEVCACHE"
> -               mv /dev/shm/udev.cache /etc/udev/cache.data
> +               mv "$DEVCACHE_CURRENT_SYSCONF" "$DEVCACHE_SYSCONF"
>         ) &
>  fi
>
> diff --git a/meta/recipes-core/udev/udev/udev-cache.default b/meta/recipes-core/udev/udev/udev-cache.default
> index 656c2a4..7f39a68 100644
> --- a/meta/recipes-core/udev/udev/udev-cache.default
> +++ b/meta/recipes-core/udev/udev/udev-cache.default
> @@ -2,4 +2,8 @@
>
>  # Comment this out to disable device cache
>  DEVCACHE="/etc/udev-cache.tar.gz"
> +
> +DEVCACHE_SYSCONF="/etc/udev/cache.data"
> +DEVCACHE_CURRENT_SYSCONF="/dev/shm/udev.cache"
> +
>  PROBE_PLATFORM_BUS="yes"
> --

Most users won't need to change this so I think this should have a
default value in the script and don't be added in the default.

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: [PATCH 11/20] udev-cache: parametrize tar options

[Otavio Salvador]

On Mon, Aug 4, 2014 at 3:41 PM, Ben Shelton <ben.shelton@ni.com> wrote:
> From: Richard Tollerton <rich.tollerton@ni.com>
>
> The options to pass to `tar` for cache create/extract are likely to
> change due to system configuration circumstances (e.g. using busybox tar
> instead of GNU tar). So move the more detailed options into
> the defaults file to allow for easier overrides.
>
> Natinst-Rally-ID: TA44427
> Acked-by: Gratian Crisan <gratian.crisan@ni.com>
> Natinst-ReviewBoard-ID: 58620
> Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>

Same comment as 10/20.

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: [PATCH 08/20] udev-cache: Create cache asynchronously

[Richard Tollerton]

Otavio Salvador <otavio@ossystems.com.br> writes:

> I am not sure about this one. I see the value you are adding here but
> I worry how often something can be connected during this process and
> change the contents along the way. Did you see something as that
> during your tests?

No, but point taken. I didn't really test "physical" hotplug during
bootup -- and in particular, for this to be a persisting issue, the
added/removed device must be removed/added during this boot -- but I
suppose that is totally possible.

This reflects an underlying race condition, starting at the computation
of NEWDATA/OLDDATA in the udev initscript, and ending in the tarball
creation in udev-cache. I suppose that we can mitigate this by spinning
in udev-cache until the devcache stops changing...? Cheesy, untested,
dirty RFC follows:

diff --git a/meta/recipes-core/udev/udev/udev-cache b/meta/recipes-core/udev/udev/udev-cache
index 9ad3b4d..3b744c5 100644
--- a/meta/recipes-core/udev/udev/udev-cache
+++ b/meta/recipes-core/udev/udev/udev-cache
@@ -28,6 +28,13 @@ fi
 [ -e "$DEVCACHE_CURRENT_SYSCONF" ] || exit 0
 [ "${VERBOSE}" == "no" ] || echo "found; building cache."
 
+# assume readfiles and CMP_FILE_LIST are copied from udev initscript
+readfiles "$DEVCACHE_SYSCONF"
+DATA2="$READDATA"
+
+while true; do
+	DATA1="$DATA2"
+
 	if [ ! `readlink -f /bin/tar` = "/bin/tar.tar" ]; then
 		(
 			find /dev \( -type b -o -type c -o -type l \) -xdev | cut -c 2- | xargs tar czf "${DEVCACHE}.tmp" -C / $BUSYBOX_DEVCACHE_CREATE_OPTS && \
@@ -42,4 +49,13 @@ else
 		) &
 	fi
 
+	readfiles $CMP_FILE_LIST
+	DATA2="$READDATA$RULELIST"
+	if [ "$DATA1" = "$DATA2" ]; then
+		break
+	else
+		echo "$DATA2" > "$DEVCACHE_CURRENT_SYSCONF"
+	fi
+done
+
 exit 0


> -- 
> Otavio Salvador                             O.S. Systems
> http://www.ossystems.com.br        http://code.ossystems.com.br
> Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750
> -- 

-- 
Richard Tollerton <rich.tollerton@ni.com>

Re: [PATCH 08/20] udev-cache: Create cache asynchronously

[Otavio Salvador]

On Mon, Aug 4, 2014 at 6:18 PM, Richard Tollerton <rich.tollerton@ni.com> wrote:
> Otavio Salvador <otavio@ossystems.com.br> writes:
>
>> I am not sure about this one. I see the value you are adding here but
>> I worry how often something can be connected during this process and
>> change the contents along the way. Did you see something as that
>> during your tests?
>
> No, but point taken. I didn't really test "physical" hotplug during
> bootup -- and in particular, for this to be a persisting issue, the
> added/removed device must be removed/added during this boot -- but I
> suppose that is totally possible.
>
> This reflects an underlying race condition, starting at the computation
> of NEWDATA/OLDDATA in the udev initscript, and ending in the tarball
> creation in udev-cache. I suppose that we can mitigate this by spinning
> in udev-cache until the devcache stops changing...? Cheesy, untested,
> dirty RFC follows:

You need to check the list of files taken by 'tar' and compare. This
mitigates it but does not really solves the issue.

Maybe we could "copy" the nodes to a tmp area and use this as a
snapshot? The copy should be fast and makes it mostly "atomic".

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: [PATCH 08/20] udev-cache: Create cache asynchronously

[Richard Tollerton]

Otavio Salvador <otavio@ossystems.com.br> writes:

> On Mon, Aug 4, 2014 at 6:18 PM, Richard Tollerton <rich.tollerton@ni.com> wrote:
>> Otavio Salvador <otavio@ossystems.com.br> writes:
>>
>>> I am not sure about this one. I see the value you are adding here but
>>> I worry how often something can be connected during this process and
>>> change the contents along the way. Did you see something as that
>>> during your tests?
>>
>> No, but point taken. I didn't really test "physical" hotplug during
>> bootup -- and in particular, for this to be a persisting issue, the
>> added/removed device must be removed/added during this boot -- but I
>> suppose that is totally possible.
>>
>> This reflects an underlying race condition, starting at the computation
>> of NEWDATA/OLDDATA in the udev initscript, and ending in the tarball
>> creation in udev-cache. I suppose that we can mitigate this by spinning
>> in udev-cache until the devcache stops changing...? Cheesy, untested,
>> dirty RFC follows:
>
> You need to check the list of files taken by 'tar' and compare. This
> mitigates it but does not really solves the issue.

Hmm, I'm confused. If we relied entirely on /dev contents, how would we
detect (and correct for) if a device was removed between the execution
of udev and of udev-cache?

> Maybe we could "copy" the nodes to a tmp area and use this as a
> snapshot? The copy should be fast and makes it mostly "atomic".

I agree that the copy of the contents of /dev needs to be as fast as
possible -- your about using tmpfs as a staging area is well taken. But
beyond that, it seems to me that the root issue you articulated is about
ensuring adequate synchronization of udev.cache and the tarball;
rereading CMP_FILE_LIST in udev-cache thus seems unavoidable.

> -- 
> Otavio Salvador                             O.S. Systems
> http://www.ossystems.com.br        http://code.ossystems.com.br
> Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

-- 
Richard Tollerton <rich.tollerton@ni.com>

Re: [PATCH 10/20] udev-cache: parametrize sysconf file paths

[Ben Shelton]

On 08/04, Otavio Salvador wrote:
> On Mon, Aug 4, 2014 at 3:41 PM, Ben Shelton <ben.shelton@ni.com> wrote:
> > From: Richard Tollerton <rich.tollerton@ni.com>
> >
> > The udev-cache facility uses files that represent system states, to
> > ensure that the cache tarball is valid to apply. These paths were
> > hardcoded in several places; collect them into DEVCACHE_SYSCONF and
> > DEVCACHE_CURRENT_SYSCONF in the defaults file.
> >
> > Natinst-Rally-ID: TA44427
> > Acked-by: Gratian Crisan <gratian.crisan@ni.com>
> > Natinst-ReviewBoard-ID: 58620
> > Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>
> > ---
> ...
> > --- a/meta/recipes-core/udev/udev/udev-cache
> > +++ b/meta/recipes-core/udev/udev/udev-cache
> > @@ -23,12 +23,12 @@ if [ "$ROOTFS_READ_ONLY" = "yes" ]; then
> >      exit 0
> >  fi
> >
> > -if [ "$DEVCACHE" != "" -a -e /dev/shm/udev.cache ]; then
> > +if [ "$DEVCACHE" != "" -a -e "$DEVCACHE_CURRENT_SYSCONF" ]; then
> >         [ "${VERBOSE}" != "no" ] && echo "Populating dev cache"
> >         (
> >                 tar czf "${DEVCACHE}.tmp" dev -C / --exclude=log
> >                 mv -f "${DEVCACHE}.tmp" "$DEVCACHE"
> > -               mv /dev/shm/udev.cache /etc/udev/cache.data
> > +               mv "$DEVCACHE_CURRENT_SYSCONF" "$DEVCACHE_SYSCONF"
> >         ) &
> >  fi
> >
> > diff --git a/meta/recipes-core/udev/udev/udev-cache.default b/meta/recipes-core/udev/udev/udev-cache.default
> > index 656c2a4..7f39a68 100644
> > --- a/meta/recipes-core/udev/udev/udev-cache.default
> > +++ b/meta/recipes-core/udev/udev/udev-cache.default
> > @@ -2,4 +2,8 @@
> >
> >  # Comment this out to disable device cache
> >  DEVCACHE="/etc/udev-cache.tar.gz"
> > +
> > +DEVCACHE_SYSCONF="/etc/udev/cache.data"
> > +DEVCACHE_CURRENT_SYSCONF="/dev/shm/udev.cache"
> > +
> >  PROBE_PLATFORM_BUS="yes"
> > --
> 
> Most users won't need to change this so I think this should have a
> default value in the script and don't be added in the default.
> 

My concern is that the uses of both DEVCACHE_SYSCONF and
DEVCACHE_CURRENT_SYSCONF are split across two script files: udev/init
and udev/udev-cache.  I don't like replicating the same default setting
in both, but if that's preferable to adding it in the default, that's
OK.

Ben


> -- 
> Otavio Salvador                             O.S. Systems
> http://www.ossystems.com.br        http://code.ossystems.com.br
> Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: [PATCH 11/20] udev-cache: parametrize tar options

[Ben Shelton]

On 08/04, Otavio Salvador wrote:
> On Mon, Aug 4, 2014 at 3:41 PM, Ben Shelton <ben.shelton@ni.com> wrote:
> > From: Richard Tollerton <rich.tollerton@ni.com>
> >
> > The options to pass to `tar` for cache create/extract are likely to
> > change due to system configuration circumstances (e.g. using busybox tar
> > instead of GNU tar). So move the more detailed options into
> > the defaults file to allow for easier overrides.
> >
> > Natinst-Rally-ID: TA44427
> > Acked-by: Gratian Crisan <gratian.crisan@ni.com>
> > Natinst-ReviewBoard-ID: 58620
> > Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>
> 
> Same comment as 10/20.
> 

In this one, I'm OK with setting the default in the scripts themselves,
since each environment variable is only used in one of the scripts.

Ben

> -- 
> Otavio Salvador                             O.S. Systems
> http://www.ossystems.com.br        http://code.ossystems.com.br
> Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: [PATCH 08/20] udev-cache: Create cache asynchronously

[Otavio Salvador]

On Mon, Aug 4, 2014 at 6:37 PM, Richard Tollerton <rich.tollerton@ni.com> wrote:
> Otavio Salvador <otavio@ossystems.com.br> writes:
>
>> On Mon, Aug 4, 2014 at 6:18 PM, Richard Tollerton <rich.tollerton@ni.com> wrote:
>>> Otavio Salvador <otavio@ossystems.com.br> writes:
>>>
>>>> I am not sure about this one. I see the value you are adding here but
>>>> I worry how often something can be connected during this process and
>>>> change the contents along the way. Did you see something as that
>>>> during your tests?
>>>
>>> No, but point taken. I didn't really test "physical" hotplug during
>>> bootup -- and in particular, for this to be a persisting issue, the
>>> added/removed device must be removed/added during this boot -- but I
>>> suppose that is totally possible.
>>>
>>> This reflects an underlying race condition, starting at the computation
>>> of NEWDATA/OLDDATA in the udev initscript, and ending in the tarball
>>> creation in udev-cache. I suppose that we can mitigate this by spinning
>>> in udev-cache until the devcache stops changing...? Cheesy, untested,
>>> dirty RFC follows:
>>
>> You need to check the list of files taken by 'tar' and compare. This
>> mitigates it but does not really solves the issue.
>
> Hmm, I'm confused. If we relied entirely on /dev contents, how would we
> detect (and correct for) if a device was removed between the execution
> of udev and of udev-cache?
>
>> Maybe we could "copy" the nodes to a tmp area and use this as a
>> snapshot? The copy should be fast and makes it mostly "atomic".
>
> I agree that the copy of the contents of /dev needs to be as fast as
> possible -- your about using tmpfs as a staging area is well taken. But
> beyond that, it seems to me that the root issue you articulated is about
> ensuring adequate synchronization of udev.cache and the tarball;
> rereading CMP_FILE_LIST in udev-cache thus seems unavoidable.

Yes, that's why I think we need:

 - copy contents to a staging area
 - generate the tarball

This way we avoid the following steps:

 - generate the list of contents
 - generate the tarball
 - generate tarball's contents
 - compare, loop if does not match.

Makes sense?

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: [PATCH 10/20] udev-cache: parametrize sysconf file paths

[Otavio Salvador]

On Mon, Aug 4, 2014 at 6:38 PM, Ben Shelton <ben.shelton@ni.com> wrote:
> On 08/04, Otavio Salvador wrote:
>> On Mon, Aug 4, 2014 at 3:41 PM, Ben Shelton <ben.shelton@ni.com> wrote:
>> > From: Richard Tollerton <rich.tollerton@ni.com>
>> >
>> > The udev-cache facility uses files that represent system states, to
>> > ensure that the cache tarball is valid to apply. These paths were
>> > hardcoded in several places; collect them into DEVCACHE_SYSCONF and
>> > DEVCACHE_CURRENT_SYSCONF in the defaults file.
>> >
>> > Natinst-Rally-ID: TA44427
>> > Acked-by: Gratian Crisan <gratian.crisan@ni.com>
>> > Natinst-ReviewBoard-ID: 58620
>> > Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>
>> > ---
>> ...
>> > --- a/meta/recipes-core/udev/udev/udev-cache
>> > +++ b/meta/recipes-core/udev/udev/udev-cache
>> > @@ -23,12 +23,12 @@ if [ "$ROOTFS_READ_ONLY" = "yes" ]; then
>> >      exit 0
>> >  fi
>> >
>> > -if [ "$DEVCACHE" != "" -a -e /dev/shm/udev.cache ]; then
>> > +if [ "$DEVCACHE" != "" -a -e "$DEVCACHE_CURRENT_SYSCONF" ]; then
>> >         [ "${VERBOSE}" != "no" ] && echo "Populating dev cache"
>> >         (
>> >                 tar czf "${DEVCACHE}.tmp" dev -C / --exclude=log
>> >                 mv -f "${DEVCACHE}.tmp" "$DEVCACHE"
>> > -               mv /dev/shm/udev.cache /etc/udev/cache.data
>> > +               mv "$DEVCACHE_CURRENT_SYSCONF" "$DEVCACHE_SYSCONF"
>> >         ) &
>> >  fi
>> >
>> > diff --git a/meta/recipes-core/udev/udev/udev-cache.default b/meta/recipes-core/udev/udev/udev-cache.default
>> > index 656c2a4..7f39a68 100644
>> > --- a/meta/recipes-core/udev/udev/udev-cache.default
>> > +++ b/meta/recipes-core/udev/udev/udev-cache.default
>> > @@ -2,4 +2,8 @@
>> >
>> >  # Comment this out to disable device cache
>> >  DEVCACHE="/etc/udev-cache.tar.gz"
>> > +
>> > +DEVCACHE_SYSCONF="/etc/udev/cache.data"
>> > +DEVCACHE_CURRENT_SYSCONF="/dev/shm/udev.cache"
>> > +
>> >  PROBE_PLATFORM_BUS="yes"
>> > --
>>
>> Most users won't need to change this so I think this should have a
>> default value in the script and don't be added in the default.
>>
>
> My concern is that the uses of both DEVCACHE_SYSCONF and
> DEVCACHE_CURRENT_SYSCONF are split across two script files: udev/init
> and udev/udev-cache.  I don't like replicating the same default setting
> in both, but if that's preferable to adding it in the default, that's
> OK.

I think this also provides a backward compatibility for users
providing their own default file. Otherwise you'll break those.

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: [PATCH 10/20] udev-cache: parametrize sysconf file paths

[Ben Shelton]

On 08/04, Otavio Salvador wrote:
> On Mon, Aug 4, 2014 at 6:38 PM, Ben Shelton <ben.shelton@ni.com> wrote:
> > On 08/04, Otavio Salvador wrote:
> >> On Mon, Aug 4, 2014 at 3:41 PM, Ben Shelton <ben.shelton@ni.com> wrote:
> >> > From: Richard Tollerton <rich.tollerton@ni.com>
> >> >
> >> > The udev-cache facility uses files that represent system states, to
> >> > ensure that the cache tarball is valid to apply. These paths were
> >> > hardcoded in several places; collect them into DEVCACHE_SYSCONF and
> >> > DEVCACHE_CURRENT_SYSCONF in the defaults file.
> >> >
> >> > Natinst-Rally-ID: TA44427
> >> > Acked-by: Gratian Crisan <gratian.crisan@ni.com>
> >> > Natinst-ReviewBoard-ID: 58620
> >> > Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>
> >> > ---
> >> ...
> >> > --- a/meta/recipes-core/udev/udev/udev-cache
> >> > +++ b/meta/recipes-core/udev/udev/udev-cache
> >> > @@ -23,12 +23,12 @@ if [ "$ROOTFS_READ_ONLY" = "yes" ]; then
> >> >      exit 0
> >> >  fi
> >> >
> >> > -if [ "$DEVCACHE" != "" -a -e /dev/shm/udev.cache ]; then
> >> > +if [ "$DEVCACHE" != "" -a -e "$DEVCACHE_CURRENT_SYSCONF" ]; then
> >> >         [ "${VERBOSE}" != "no" ] && echo "Populating dev cache"
> >> >         (
> >> >                 tar czf "${DEVCACHE}.tmp" dev -C / --exclude=log
> >> >                 mv -f "${DEVCACHE}.tmp" "$DEVCACHE"
> >> > -               mv /dev/shm/udev.cache /etc/udev/cache.data
> >> > +               mv "$DEVCACHE_CURRENT_SYSCONF" "$DEVCACHE_SYSCONF"
> >> >         ) &
> >> >  fi
> >> >
> >> > diff --git a/meta/recipes-core/udev/udev/udev-cache.default b/meta/recipes-core/udev/udev/udev-cache.default
> >> > index 656c2a4..7f39a68 100644
> >> > --- a/meta/recipes-core/udev/udev/udev-cache.default
> >> > +++ b/meta/recipes-core/udev/udev/udev-cache.default
> >> > @@ -2,4 +2,8 @@
> >> >
> >> >  # Comment this out to disable device cache
> >> >  DEVCACHE="/etc/udev-cache.tar.gz"
> >> > +
> >> > +DEVCACHE_SYSCONF="/etc/udev/cache.data"
> >> > +DEVCACHE_CURRENT_SYSCONF="/dev/shm/udev.cache"
> >> > +
> >> >  PROBE_PLATFORM_BUS="yes"
> >> > --
> >>
> >> Most users won't need to change this so I think this should have a
> >> default value in the script and don't be added in the default.
> >>
> >
> > My concern is that the uses of both DEVCACHE_SYSCONF and
> > DEVCACHE_CURRENT_SYSCONF are split across two script files: udev/init
> > and udev/udev-cache.  I don't like replicating the same default setting
> > in both, but if that's preferable to adding it in the default, that's
> > OK.
> 
> I think this also provides a backward compatibility for users
> providing their own default file. Otherwise you'll break those.

Fair point.  I'll make the change.

Thanks,
Ben

> 
> -- 
> Otavio Salvador                             O.S. Systems
> http://www.ossystems.com.br        http://code.ossystems.com.br
> Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: [PATCH 08/20] udev-cache: Create cache asynchronously

[Richard Tollerton]

Otavio Salvador <otavio@ossystems.com.br> writes:

> On Mon, Aug 4, 2014 at 6:37 PM, Richard Tollerton <rich.tollerton@ni.com> wrote:
>> Otavio Salvador <otavio@ossystems.com.br> writes:
>>
>>> On Mon, Aug 4, 2014 at 6:18 PM, Richard Tollerton <rich.tollerton@ni.com> wrote:
>>>> Otavio Salvador <otavio@ossystems.com.br> writes:
>>>>
>>>>> I am not sure about this one. I see the value you are adding here but
>>>>> I worry how often something can be connected during this process and
>>>>> change the contents along the way. Did you see something as that
>>>>> during your tests?
>>>>
>>>> No, but point taken. I didn't really test "physical" hotplug during
>>>> bootup -- and in particular, for this to be a persisting issue, the
>>>> added/removed device must be removed/added during this boot -- but I
>>>> suppose that is totally possible.
>>>>
>>>> This reflects an underlying race condition, starting at the computation
>>>> of NEWDATA/OLDDATA in the udev initscript, and ending in the tarball
>>>> creation in udev-cache. I suppose that we can mitigate this by spinning
>>>> in udev-cache until the devcache stops changing...? Cheesy, untested,
>>>> dirty RFC follows:
>>>
>>> You need to check the list of files taken by 'tar' and compare. This
>>> mitigates it but does not really solves the issue.
>>
>> Hmm, I'm confused. If we relied entirely on /dev contents, how would we
>> detect (and correct for) if a device was removed between the execution
>> of udev and of udev-cache?
>>
>>> Maybe we could "copy" the nodes to a tmp area and use this as a
>>> snapshot? The copy should be fast and makes it mostly "atomic".
>>
>> I agree that the copy of the contents of /dev needs to be as fast as
>> possible -- your about using tmpfs as a staging area is well taken. But
>> beyond that, it seems to me that the root issue you articulated is about
>> ensuring adequate synchronization of udev.cache and the tarball;
>> rereading CMP_FILE_LIST in udev-cache thus seems unavoidable.
>
> Yes, that's why I think we need:
>
>  - copy contents to a staging area
>  - generate the tarball
>
> This way we avoid the following steps:
>
>  - generate the list of contents
>  - generate the tarball
>  - generate tarball's contents
>  - compare, loop if does not match.
>
> Makes sense?

Yes, with one friendly amendment: "copy contents" means copying
$CMP_FILE_LIST in addition to /dev. Then we rebuild cache.data from
that.

> -- 
> Otavio Salvador                             O.S. Systems
> http://www.ossystems.com.br        http://code.ossystems.com.br
> Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

-- 
Richard Tollerton <rich.tollerton@ni.com>

Re: [PATCH 01/20] udev-cache: Compress the cache

[Khem Raj]

On 14-08-04 13:40:53, Ben Shelton wrote:
> From: Richard Tollerton <rich.tollerton@ni.com>
> 
> $DEVCACHE is observed to be 100k uncompressed; compressing it reduces
> its size to ~5k.
> 
> Natinst-Rally-ID: TA44427
> Acked-by: Gratian Crisan <gratian.crisan@ni.com>
> Natinst-ReviewBoard-ID: 58620
> Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>
> ---
>  meta/recipes-core/udev/udev/init               | 2 +-
>  meta/recipes-core/udev/udev/udev-cache         | 2 +-
>  meta/recipes-core/udev/udev/udev-cache.default | 2 +-
>  3 files changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/meta/recipes-core/udev/udev/init b/meta/recipes-core/udev/udev/init
> index f2c84d5..1e69861 100644
> --- a/meta/recipes-core/udev/udev/init
> +++ b/meta/recipes-core/udev/udev/init
> @@ -69,7 +69,7 @@ case "$1" in
>  		    readfiles /etc/udev/cache.data
>  		    OLDDATA="$READDATA"
>  		    if [ "$OLDDATA" = "$NEWDATA" ]; then
> -                            (cd /; tar xf $DEVCACHE > /dev/null 2>&1)
> +                            (cd /; tar xzf $DEVCACHE > /dev/null 2>&1)

wouldnt cf still handle gz and xz or any other compression ?

Re: [PATCH 01/20] udev-cache: Compress the cache

[Ben Shelton]

On 08/04, Khem Raj wrote:
> On 14-08-04 13:40:53, Ben Shelton wrote:
> > From: Richard Tollerton <rich.tollerton@ni.com>
> > 
> > $DEVCACHE is observed to be 100k uncompressed; compressing it reduces
> > its size to ~5k.
> > 
> > Natinst-Rally-ID: TA44427
> > Acked-by: Gratian Crisan <gratian.crisan@ni.com>
> > Natinst-ReviewBoard-ID: 58620
> > Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>
> > ---
> >  meta/recipes-core/udev/udev/init               | 2 +-
> >  meta/recipes-core/udev/udev/udev-cache         | 2 +-
> >  meta/recipes-core/udev/udev/udev-cache.default | 2 +-
> >  3 files changed, 3 insertions(+), 3 deletions(-)
> > 
> > diff --git a/meta/recipes-core/udev/udev/init b/meta/recipes-core/udev/udev/init
> > index f2c84d5..1e69861 100644
> > --- a/meta/recipes-core/udev/udev/init
> > +++ b/meta/recipes-core/udev/udev/init
> > @@ -69,7 +69,7 @@ case "$1" in
> >  		    readfiles /etc/udev/cache.data
> >  		    OLDDATA="$READDATA"
> >  		    if [ "$OLDDATA" = "$NEWDATA" ]; then
> > -                            (cd /; tar xf $DEVCACHE > /dev/null 2>&1)
> > +                            (cd /; tar xzf $DEVCACHE > /dev/null 2>&1)
> 
> wouldnt cf still handle gz and xz or any other compression ?

Do you mean xf?  If so, then yes, I think it would.

What would be the advantage of doing it that way?  Just one fewer
change?

Thanks,
Ben

Re: [PATCH 01/20] udev-cache: Compress the cache

[Khem Raj]

[-- Attachment #1: Type: text/plain, Size: 1655 bytes --]

On Aug 4, 2014 4:45 PM, "Ben Shelton" <ben.shelton@ni.com> wrote:
>
> On 08/04, Khem Raj wrote:
> > On 14-08-04 13:40:53, Ben Shelton wrote:
> > > From: Richard Tollerton <rich.tollerton@ni.com>
> > >
> > > $DEVCACHE is observed to be 100k uncompressed; compressing it reduces
> > > its size to ~5k.
> > >
> > > Natinst-Rally-ID: TA44427
> > > Acked-by: Gratian Crisan <gratian.crisan@ni.com>
> > > Natinst-ReviewBoard-ID: 58620
> > > Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>
> > > ---
> > >  meta/recipes-core/udev/udev/init               | 2 +-
> > >  meta/recipes-core/udev/udev/udev-cache         | 2 +-
> > >  meta/recipes-core/udev/udev/udev-cache.default | 2 +-
> > >  3 files changed, 3 insertions(+), 3 deletions(-)
> > >
> > > diff --git a/meta/recipes-core/udev/udev/init
b/meta/recipes-core/udev/udev/init
> > > index f2c84d5..1e69861 100644
> > > --- a/meta/recipes-core/udev/udev/init
> > > +++ b/meta/recipes-core/udev/udev/init
> > > @@ -69,7 +69,7 @@ case "$1" in
> > >                 readfiles /etc/udev/cache.data
> > >                 OLDDATA="$READDATA"
> > >                 if [ "$OLDDATA" = "$NEWDATA" ]; then
> > > -                            (cd /; tar xf $DEVCACHE > /dev/null 2>&1)
> > > +                            (cd /; tar xzf $DEVCACHE > /dev/null
2>&1)
> >
> > wouldnt cf still handle gz and xz or any other compression ?
>
> Do you mean xf?  If so, then yes, I think it would.
>
> What would be the advantage of doing it that way?  Just one fewer
> change?

yes and tomorrow you might want to use different compression this wont
change
>
> Thanks,
> Ben
>

[-- Attachment #2: Type: text/html, Size: 2460 bytes --]

Re: [PATCH 01/20] udev-cache: Compress the cache

[Randy MacLeod]

On 14-08-04 02:40 PM, Ben Shelton wrote:
> From: Richard Tollerton <rich.tollerton@ni.com>
>
> $DEVCACHE is observed to be 100k uncompressed; compressing it reduces
> its size to ~5k.
>
> Natinst-Rally-ID: TA44427
> Acked-by: Gratian Crisan <gratian.crisan@ni.com>
> Natinst-ReviewBoard-ID: 58620


This is a good series of patches; thanks for sending them.

It would be nice if you could remove your company's ID tags
from the commit log. For example, you create and use could use
a YP bugzilla ID ( https://bugzilla.yoctoproject.org/ ) and
then match that up in your internal process.

Thanks,
../Randy


> Signed-off-by: Richard Tollerton <rich.tollerton@ni.com>
> ---
>   meta/recipes-core/udev/udev/init               | 2 +-
>   meta/recipes-core/udev/udev/udev-cache         | 2 +-
>   meta/recipes-core/udev/udev/udev-cache.default | 2 +-
>   3 files changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/meta/recipes-core/udev/udev/init b/meta/recipes-core/udev/udev/init
> index f2c84d5..1e69861 100644
> --- a/meta/recipes-core/udev/udev/init
> +++ b/meta/recipes-core/udev/udev/init
> @@ -69,7 +69,7 @@ case "$1" in
>   		    readfiles /etc/udev/cache.data
>   		    OLDDATA="$READDATA"
>   		    if [ "$OLDDATA" = "$NEWDATA" ]; then
> -                            (cd /; tar xf $DEVCACHE > /dev/null 2>&1)
> +                            (cd /; tar xzf $DEVCACHE > /dev/null 2>&1)
>                               not_first_boot=1
>                               [ "$VERBOSE" != "no" ] && echo "udev: using cache file $DEVCACHE"
>                               [ -e /dev/shm/udev.cache ] && rm -f /dev/shm/udev.cache
> diff --git a/meta/recipes-core/udev/udev/udev-cache b/meta/recipes-core/udev/udev/udev-cache
> index db5a513..3769062 100644
> --- a/meta/recipes-core/udev/udev/udev-cache
> +++ b/meta/recipes-core/udev/udev/udev-cache
> @@ -25,7 +25,7 @@ fi
>
>   if [ "$DEVCACHE" != "" -a -e /dev/shm/udev.cache ]; then
>   	echo "Populating dev cache"
> -	(cd /; tar cf "$DEVCACHE" dev)
> +	(cd /; tar czf "$DEVCACHE" dev)
>   	mv /dev/shm/udev.cache /etc/udev/cache.data
>   fi
>
> diff --git a/meta/recipes-core/udev/udev/udev-cache.default b/meta/recipes-core/udev/udev/udev-cache.default
> index 2093336..909ec87 100644
> --- a/meta/recipes-core/udev/udev/udev-cache.default
> +++ b/meta/recipes-core/udev/udev/udev-cache.default
> @@ -1,5 +1,5 @@
>   # Default for /etc/init.d/udev
>
>   # Comment this out to disable device cache
> -DEVCACHE="/etc/dev.tar"
> +DEVCACHE="/etc/dev.tar.gz"
>   PROBE_PLATFORM_BUS="yes"
>


-- 
# Randy MacLeod. SMTS, Linux, Wind River
Direct: 613.963.1350

Re: [PATCH 01/20] udev-cache: Compress the cache

[Otavio Salvador]

On Tue, Aug 5, 2014 at 4:00 PM, Randy MacLeod
<randy.macleod@windriver.com> wrote:
> On 14-08-04 02:40 PM, Ben Shelton wrote:
>>
>> From: Richard Tollerton <rich.tollerton@ni.com>
>>
>> $DEVCACHE is observed to be 100k uncompressed; compressing it reduces
>> its size to ~5k.
>>
>> Natinst-Rally-ID: TA44427
>> Acked-by: Gratian Crisan <gratian.crisan@ni.com>
>> Natinst-ReviewBoard-ID: 58620
>
>
>
> This is a good series of patches; thanks for sending them.
>
> It would be nice if you could remove your company's ID tags
> from the commit log. For example, you create and use could use
> a YP bugzilla ID ( https://bugzilla.yoctoproject.org/ ) and
> then match that up in your internal process.

I don't think we should ask for this. I saw several of us to include
Id tags in past commits (I also did) and I think it is fine. We
shouldn't ask an extra step for people to be able to track it (open a
YP bugzilla ID issue for example).

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: [PATCH 01/20] udev-cache: Compress the cache

[Randy MacLeod]

On 14-08-05 04:09 PM, Otavio Salvador wrote:
> On Tue, Aug 5, 2014 at 4:00 PM, Randy MacLeod
> <randy.macleod@windriver.com> wrote:
>> On 14-08-04 02:40 PM, Ben Shelton wrote:
>>>
>>> From: Richard Tollerton <rich.tollerton@ni.com>
>>>
>>> $DEVCACHE is observed to be 100k uncompressed; compressing it reduces
>>> its size to ~5k.
>>>
>>> Natinst-Rally-ID: TA44427
>>> Acked-by: Gratian Crisan <gratian.crisan@ni.com>
>>> Natinst-ReviewBoard-ID: 58620
>>
>>
>>
>> This is a good series of patches; thanks for sending them.
>>
>> It would be nice if you could remove your company's ID tags
>> from the commit log. For example, you create and use could use
>> a YP bugzilla ID ( https://bugzilla.yoctoproject.org/ ) and
>> then match that up in your internal process.
>
> I don't think we should ask for this. I saw several of us to include
> Id tags in past commits (I also did) and I think it is fine. We
> shouldn't ask an extra step for people to be able to track it (open a
> YP bugzilla ID issue for example).
>

Wind River patches used to include a "CQID" tag but we've changed
our process to avoid needing such internal tags. If National
Instruments can do so as well, that'd be best.

I did check my oe-core email list history and this seems like the
first patch from NI that has the tags included so I thought
I'd reply and see if we can get the tags dropped.

Ben, what's the story?

-- 
# Randy MacLeod. SMTS, Linux, Wind River
Direct: 613.963.1350

Re: [PATCH 01/20] udev-cache: Compress the cache

[Ben Shelton]

On 08/05, Randy MacLeod wrote:
> On 14-08-05 04:09 PM, Otavio Salvador wrote:
> >On Tue, Aug 5, 2014 at 4:00 PM, Randy MacLeod
> ><randy.macleod@windriver.com> wrote:
> >>On 14-08-04 02:40 PM, Ben Shelton wrote:
> >>>
> >>>From: Richard Tollerton <rich.tollerton@ni.com>
> >>>
> >>>$DEVCACHE is observed to be 100k uncompressed; compressing it reduces
> >>>its size to ~5k.
> >>>
> >>>Natinst-Rally-ID: TA44427
> >>>Acked-by: Gratian Crisan <gratian.crisan@ni.com>
> >>>Natinst-ReviewBoard-ID: 58620
> >>
> >>
> >>
> >>This is a good series of patches; thanks for sending them.
> >>
> >>It would be nice if you could remove your company's ID tags
> >>from the commit log. For example, you create and use could use
> >>a YP bugzilla ID ( https://bugzilla.yoctoproject.org/ ) and
> >>then match that up in your internal process.
> >
> >I don't think we should ask for this. I saw several of us to include
> >Id tags in past commits (I also did) and I think it is fine. We
> >shouldn't ask an extra step for people to be able to track it (open a
> >YP bugzilla ID issue for example).
> >
> 
> Wind River patches used to include a "CQID" tag but we've changed
> our process to avoid needing such internal tags. If National
> Instruments can do so as well, that'd be best.
> 
> I did check my oe-core email list history and this seems like the
> first patch from NI that has the tags included so I thought
> I'd reply and see if we can get the tags dropped.
> 
> Ben, what's the story?

In the past, I've stripped the tags before posting the patches.  Looks
like I forgot this time -- I'll remove the tags when I submit v2 of the
patch set.

Thanks,
Ben

> 
> -- 
> # Randy MacLeod. SMTS, Linux, Wind River
> Direct: 613.963.1350

Patch message guidelines and internal/corporate fields

[Richard Tollerton]

Randy MacLeod <randy.macleod@windriver.com> writes:

> Wind River patches used to include a "CQID" tag but we've changed
> our process to avoid needing such internal tags. If National
> Instruments can do so as well, that'd be best.
>
> I did check my oe-core email list history and this seems like the
> first patch from NI that has the tags included so I thought
> I'd reply and see if we can get the tags dropped.

IIRC, we pinged Phil a few months ago on this topic, and he thought
internal tags were OK. I think Wind River might have been cited as an
example.

I see that
http://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines hasn't
been updated with this requirement. Should it be? Will that require TSC
approval?

> -- 
> # Randy MacLeod. SMTS, Linux, Wind River
> Direct: 613.963.1350
> -- 

-- 
Richard Tollerton <rich.tollerton@ni.com>

Re: Patch message guidelines and internal/corporate fields

[Richard Purdie]

On Fri, 2014-08-22 at 10:24 -0500, Richard Tollerton wrote:
> Randy MacLeod <randy.macleod@windriver.com> writes:
> 
> > Wind River patches used to include a "CQID" tag but we've changed
> > our process to avoid needing such internal tags. If National
> > Instruments can do so as well, that'd be best.
> >
> > I did check my oe-core email list history and this seems like the
> > first patch from NI that has the tags included so I thought
> > I'd reply and see if we can get the tags dropped.
> 
> IIRC, we pinged Phil a few months ago on this topic, and he thought
> internal tags were OK. I think Wind River might have been cited as an
> example.
> 
> I see that
> http://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines hasn't
> been updated with this requirement. Should it be? Will that require TSC
> approval?

I don't have a strong preference on this to be honest. I can imagine
cases where its useful to have some kind of tracking of issues into the
final commits. 

Obviously it would be better if everyone can understand what the numbers
mean, equally, it seems pointless to force people to strip them when
they might be useful to people contributing to the project.

If they are used as a substitute for a good commit message, that
wouldn't be acceptable. Also, if they were taking over the commit
messages, that would able be unacceptable. So if we can keep them to a
small part of the commit, I'm prepared to let them pass but it can't be
at the expense of good commit messages.

I don't believe we need a TSC decision, that would only be needed if
there were strong disagreements we were unable to resolve and I don't
think we're quite there yet :) I'll let others comment though and see
where we're at.

Cheers,

Richard

Re: Patch message guidelines and internal/corporate fields

[Paul Eggleton]

On Saturday 23 August 2014 09:40:44 Richard Purdie wrote:
> On Fri, 2014-08-22 at 10:24 -0500, Richard Tollerton wrote:
> > Randy MacLeod <randy.macleod@windriver.com> writes:
> > > Wind River patches used to include a "CQID" tag but we've changed
> > > our process to avoid needing such internal tags. If National
> > > Instruments can do so as well, that'd be best.
> > > 
> > > I did check my oe-core email list history and this seems like the
> > > first patch from NI that has the tags included so I thought
> > > I'd reply and see if we can get the tags dropped.
> > 
> > IIRC, we pinged Phil a few months ago on this topic, and he thought
> > internal tags were OK. I think Wind River might have been cited as an
> > example.
> > 
> > I see that
> > http://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines hasn't
> > been updated with this requirement. Should it be? Will that require TSC
> > approval?
> 
> I don't have a strong preference on this to be honest. I can imagine
> cases where its useful to have some kind of tracking of issues into the
> final commits.
> 
> Obviously it would be better if everyone can understand what the numbers
> mean, equally, it seems pointless to force people to strip them when
> they might be useful to people contributing to the project.
> 
> If they are used as a substitute for a good commit message, that
> wouldn't be acceptable. Also, if they were taking over the commit
> messages, that would able be unacceptable. So if we can keep them to a
> small part of the commit, I'm prepared to let them pass but it can't be
> at the expense of good commit messages.
> 
> I don't believe we need a TSC decision, that would only be needed if
> there were strong disagreements we were unable to resolve and I don't
> think we're quite there yet :) I'll let others comment though and see
> where we're at.

FWIW I agree with all of the above - if they make contributors' lives easier 
then I don't see a good reason to disallow their usage given we're only 
talking about one line at the end of a proper commit message.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: Patch message guidelines and internal/corporate fields

[Khem Raj]

On 14-08-24 16:33:39, Paul Eggleton wrote:
> On Saturday 23 August 2014 09:40:44 Richard Purdie wrote:
> > On Fri, 2014-08-22 at 10:24 -0500, Richard Tollerton wrote:
> > > Randy MacLeod <randy.macleod@windriver.com> writes:
> > > > Wind River patches used to include a "CQID" tag but we've changed
> > > > our process to avoid needing such internal tags. If National
> > > > Instruments can do so as well, that'd be best.
> > > > 
> > > > I did check my oe-core email list history and this seems like the
> > > > first patch from NI that has the tags included so I thought
> > > > I'd reply and see if we can get the tags dropped.
> > > 
> > > IIRC, we pinged Phil a few months ago on this topic, and he thought
> > > internal tags were OK. I think Wind River might have been cited as an
> > > example.
> > > 
> > > I see that
> > > http://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines hasn't
> > > been updated with this requirement. Should it be? Will that require TSC
> > > approval?
> > 
> > I don't have a strong preference on this to be honest. I can imagine
> > cases where its useful to have some kind of tracking of issues into the
> > final commits.
> > 
> > Obviously it would be better if everyone can understand what the numbers
> > mean, equally, it seems pointless to force people to strip them when
> > they might be useful to people contributing to the project.
> > 
> > If they are used as a substitute for a good commit message, that
> > wouldn't be acceptable. Also, if they were taking over the commit
> > messages, that would able be unacceptable. So if we can keep them to a
> > small part of the commit, I'm prepared to let them pass but it can't be
> > at the expense of good commit messages.
> > 
> > I don't believe we need a TSC decision, that would only be needed if
> > there were strong disagreements we were unable to resolve and I don't
> > think we're quite there yet :) I'll let others comment though and see
> > where we're at.
> 
> FWIW I agree with all of the above - if they make contributors' lives easier 
> then I don't see a good reason to disallow their usage given we're only 
> talking about one line at the end of a proper commit message.

I would rather think that if the information is not accessible to community
members, its not a useful to add these numbers. Secondly it could be fixing
someone else's bug in a different bug tracking system with different
tracking number and if he she does a backport and adds his tracking
number as well to commit. Now think of what will happen if people start
to do it for Linux kernel. So IMHO its best that companies track it in
their respective issue tracking systems by adding upstream commit urls or SHA ids
whatever is convinient to support any automation tools they might
have(if any) That way we solve the problem for everyone. On
these grounds I would not recommend adding it in commit messages. We let Yocto
bugzilla entries in commits because that system is accessible to everyone using OE.

Thanks

-Khem