[Buildroot] [PATCH 01/14 v4] system: with no init system, only allow custom skeleton

[Buildroot] [PATCH 01/14 v4] system: with no init system, only allow custom skeleton

[Yann E. MORIN]

When there is no init system (i.e. a custom one), our bundled default
skeleton is most probably un-fit for that (non-)init system.

This will be the case when we introduce per init system skeletons. The
systemd skeleton will be unfit except for a systemd init; the sysv
skeleton will be unfit except for a sysv-like init system.

In case they are using no init system (really, a custom one), the user
should be responsible for providing their own, custom skeleton as well.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
 system/Config.in | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/system/Config.in b/system/Config.in
index 828df4217e..ed539dcbe0 100644
--- a/system/Config.in
+++ b/system/Config.in
@@ -3,11 +3,17 @@ menu "System configuration"
 choice
 	prompt "Root FS skeleton"
 
+# When not using an init system (i.e. a custom one), our default skeleton
+# is most probably unfit; so, allow only the custom skeleton in that case.
 config BR2_ROOTFS_SKELETON_DEFAULT
 	bool "default target skeleton"
+	depends on !BR2_INIT_NONE
 	help
 	  Use default target skeleton
 
+comment "default target skeleton needs an init system"
+	depends on BR2_INIT_NONE
+
 config BR2_ROOTFS_SKELETON_CUSTOM
 	bool "custom target skeleton"
 	help
-- 
2.11.0

[Buildroot] [PATCH 02/14 v4] package/skeleton: drop dependency on host-mkpasswd

[Yann E. MORIN]

Setting the root pasword is done in a target-finalize hook, so we do not
need to enforce a dependency from the skeleton onto host-mkpasswd.

Dropping that dependency will simplify making skeleton a virtual
package (in up-coming changes).

Instead, it is now selected as any other package. As such, it is
guaranteed to be built before target-finalize.

This however introduces a slight change in behaviour: previously,
host-mkpasswd would only be built if we needed to hash the root password
from its plain-text value. Now, host-mkpasswd is always built as soon as
the root password is non-empty, even if already pre-hashed.

Since host-mkpasswd is a really tiny weeny package bundled in Buildroot,
with only two C files, built as a single unit with a single gcc call,
the overhead is really minimal. Compared to the simplifications this
will allow in the skeleton packages (plural: common, sysv, systemd,
custom) to come, this overhead is acceptable.

Yet another simplification, even if small, to ease providing multiple
skeletons.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>

---
Changes v3 -> v4:
  - add blurb that is is selected, so built before target-finalize
    (Arnout)
---
 package/skeleton/skeleton.mk | 1 -
 system/Config.in             | 1 +
 2 files changed, 1 insertion(+), 1 deletion(-)

diff --git a/package/skeleton/skeleton.mk b/package/skeleton/skeleton.mk
index 4a66f6ceca..4ec1ecbb51 100644
--- a/package/skeleton/skeleton.mk
+++ b/package/skeleton/skeleton.mk
@@ -121,7 +121,6 @@ SKELETON_ROOT_PASSWORD =
 else ifneq ($(filter $$1$$% $$5$$% $$6$$%,$(SKELETON_TARGET_GENERIC_ROOT_PASSWD)),)
 SKELETON_ROOT_PASSWORD = '$(SKELETON_TARGET_GENERIC_ROOT_PASSWD)'
 else
-SKELETON_DEPENDENCIES += host-mkpasswd
 # This variable will only be evaluated in the finalize stage, so we can
 # be sure that host-mkpasswd will have already been built by that time.
 SKELETON_ROOT_PASSWORD = "`$(MKPASSWD) -m "$(SKELETON_TARGET_GENERIC_PASSWD_METHOD)" "$(SKELETON_TARGET_GENERIC_ROOT_PASSWD)"`"
diff --git a/system/Config.in b/system/Config.in
index ed539dcbe0..5cb921f108 100644
--- a/system/Config.in
+++ b/system/Config.in
@@ -224,6 +224,7 @@ config BR2_ROOTFS_MERGED_USR
 config BR2_TARGET_ENABLE_ROOT_LOGIN
 	bool "Enable root login with password"
 	default y
+	select BR2_PACKAGE_HOST_MKPASSWD if BR2_TARGET_GENERIC_ROOT_PASSWD != ""
 	help
 	  Allow root to log in with a password.
 
-- 
2.11.0

[Buildroot] [PATCH 03/14 v4] package/skeleton: select it rather than default to y

[Yann E. MORIN]

Our current skeleton is tailored to sysv-like init systems; it is not
fit for systemd-based systems. So, in upcoming changes, we'll add
another skeleton for systemd.

This means we can no longer have the current skeleton default to 'y', or
it would be enabled also for systemd, which would be incorrect.

So, we remove the default to 'y' but have it selected by the default
skeleton choice.

However, we do not yet have a way to directly build (really, install)
the custom skeleton, it is built (really, installed) as a dependency of
the default skeleton. So we must also forcibly select the default
skeleton when using a custom one.

Until we have the means to do only one or the other; i.e. when we have a
virtual skeleton.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
 package/skeleton/Config.in | 1 -
 system/Config.in           | 2 ++
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/package/skeleton/Config.in b/package/skeleton/Config.in
index d25147bd92..b22ac66b35 100644
--- a/package/skeleton/Config.in
+++ b/package/skeleton/Config.in
@@ -1,5 +1,4 @@
 config BR2_PACKAGE_SKELETON
 	bool
-	default y
 	help
 	  The basic skeleton for your rootfs.
diff --git a/system/Config.in b/system/Config.in
index 5cb921f108..9b42fbbbac 100644
--- a/system/Config.in
+++ b/system/Config.in
@@ -8,6 +8,7 @@ choice
 config BR2_ROOTFS_SKELETON_DEFAULT
 	bool "default target skeleton"
 	depends on !BR2_INIT_NONE
+	select BR2_PACKAGE_SKELETON
 	help
 	  Use default target skeleton
 
@@ -16,6 +17,7 @@ comment "default target skeleton needs an init system"
 
 config BR2_ROOTFS_SKELETON_CUSTOM
 	bool "custom target skeleton"
+	select BR2_PACKAGE_SKELETON
 	help
 	  Use custom target skeleton.
 
-- 
2.11.0

[Buildroot] [PATCH 04/14 v4] package/skeleton: split out into skeleton-custom

[Yann E. MORIN]

For the custom skeleton, we practicaly do nothing, except ensure it
contains the basic, required directories, and that those are properly
setup wrt. merged /usr.

Furthermore, our current skeleton is not fit for systemd, and we'll
have to split things out into various skeletons.

So, off-load the custom skeleton into its own package.

Thus, the existing skeleton package is now limited to:

  - when using our default skeleton, install and tweak it properly;

  - when using a custom skeleton, do nothing except for depending on
    the skeleton-custom package.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>

---
Note: this calls for making skeleton a virtual package. This will come
in follwup patches, after skeleton has lost all its code to the various
skeleton packages.

---
Changes v3 -> v4:
  - lib{32.64} are also tweaked in the skeleton  (Arnout)
---
 package/Config.in                          |   1 +
 package/skeleton-custom/Config.in          |   3 +
 package/skeleton-custom/skeleton-custom.mk | 105 +++++++++++++++++++++++++++++
 package/skeleton/skeleton.mk               |  51 +-------------
 system/Config.in                           |   2 +-
 5 files changed, 112 insertions(+), 50 deletions(-)
 create mode 100644 package/skeleton-custom/Config.in
 create mode 100644 package/skeleton-custom/skeleton-custom.mk

diff --git a/package/Config.in b/package/Config.in
index 484c75327a..5e6e53d42b 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -2,6 +2,7 @@ menu "Target packages"
 
 	source "package/busybox/Config.in"
 	source "package/skeleton/Config.in"
+	source "package/skeleton-custom/Config.in"
 
 menu "Audio and video applications"
 	source "package/alsa-utils/Config.in"
diff --git a/package/skeleton-custom/Config.in b/package/skeleton-custom/Config.in
new file mode 100644
index 0000000000..b12bd8f73c
--- /dev/null
+++ b/package/skeleton-custom/Config.in
@@ -0,0 +1,3 @@
+config BR2_PACKAGE_SKELETON_CUSTOM
+	bool
+	select BR2_PACKAGE_SKELETON
diff --git a/package/skeleton-custom/skeleton-custom.mk b/package/skeleton-custom/skeleton-custom.mk
new file mode 100644
index 0000000000..ed7b7ac7c5
--- /dev/null
+++ b/package/skeleton-custom/skeleton-custom.mk
@@ -0,0 +1,105 @@
+################################################################################
+#
+# skeleton-custom
+#
+################################################################################
+
+SKELETON_CUSTOM_ADD_TOOLCHAIN_DEPENDENCY = NO
+SKELETON_CUSTOM_ADD_SKELETON_DEPENDENCY = NO
+
+SKELETON_CUSTOM_INSTALL_STAGING = YES
+
+SKELETON_CUSTOM_PATH = $(call qstrip,$(BR2_ROOTFS_SKELETON_CUSTOM_PATH))
+
+ifeq ($(BR2_PACKAGE_SKELETON_CUSTOM)$(BR_BUILDING),yy)
+ifeq ($(SKELETON_CUSTOM_PATH),)
+$(error No path specified for the custom skeleton)
+endif
+endif
+
+# Extract the inode numbers for all of those directories. In case any is
+# a symlink, we want to get the inode of the pointed-to directory, so we
+# append '/.' to be sure we get the target directory. Since the symlinks
+# can be anyway (/bin -> /usr/bin or /usr/bin -> /bin), we do that for
+# all of them.
+#
+SKELETON_CUSTOM_LIB_INODE = $(shell stat -c '%i' $(SKELETON_CUSTOM_PATH)/lib/. 2>/dev/null)
+SKELETON_CUSTOM_BIN_INODE = $(shell stat -c '%i' $(SKELETON_CUSTOM_PATH)/bin/. 2>/dev/null)
+SKELETON_CUSTOM_SBIN_INODE = $(shell stat -c '%i' $(SKELETON_CUSTOM_PATH)/sbin/. 2>/dev/null)
+SKELETON_CUSTOM_USR_LIB_INODE = $(shell stat -c '%i' $(SKELETON_CUSTOM_PATH)/usr/lib/. 2>/dev/null)
+SKELETON_CUSTOM_USR_BIN_INODE = $(shell stat -c '%i' $(SKELETON_CUSTOM_PATH)/usr/bin/. 2>/dev/null)
+SKELETON_CUSTOM_USR_SBIN_INODE = $(shell stat -c '%i' $(SKELETON_CUSTOM_PATH)/usr/sbin/. 2>/dev/null)
+
+# Ensure that the custom skeleton has /lib, /bin and /sbin, and their
+# /usr counterparts
+ifeq ($(SKELETON_CUSTOM_LIB_INODE),)
+SKELETON_CUSTOM_MISSING_DIRS += /lib
+endif
+ifeq ($(SKELETON_CUSTOM_USR_LIB_INODE),)
+SKELETON_CUSTOM_MISSING_DIRS += /usr/lib
+endif
+ifeq ($(SKELETON_CUSTOM_BIN_INODE),)
+SKELETON_CUSTOM_MISSING_DIRS += /bin
+endif
+ifeq ($(SKELETON_CUSTOM_USR_BIN_INODE),)
+SKELETON_CUSTOM_MISSING_DIRS += /usr/bin
+endif
+ifeq ($(SKELETON_CUSTOM_SBIN_INODE),)
+SKELETON_CUSTOM_MISSING_DIRS += /sbin
+endif
+ifeq ($(SKELETON_CUSTOM_USR_SBIN_INODE),)
+SKELETON_CUSTOM_MISSING_DIRS += /usr/sbin
+endif
+
+# For a merged /usr, ensure that /lib, /bin and /sbin and their /usr
+# counterparts are appropriately setup as symlinks ones to the others.
+ifeq ($(BR2_ROOTFS_MERGED_USR),y)
+
+ifneq ($(SKELETON_CUSTOM_LIB_INODE),$(SKELETON_CUSTOM_USR_LIB_INODE))
+SKELETON_CUSTOM_NOT_MERGED_USR_DIRS += /lib
+endif
+ifneq ($(SKELETON_CUSTOM_BIN_INODE),$(SKELETON_CUSTOM_USR_BIN_INODE))
+SKELETON_CUSTOM_NOT_MERGED_USR_DIRS += /bin
+endif
+ifneq ($(SKELETON_CUSTOM_SBIN_INODE),$(SKELETON_CUSTOM_USR_SBIN_INODE))
+SKELETON_CUSTOM_NOT_MERGED_USR_DIRS += /sbin
+endif
+
+endif # merged /usr
+
+ifeq ($(BR2_PACKAGE_SKELETON_CUSTOM)$(BR_BUILDING),yy)
+ifneq ($(SKELETON_CUSTOM_MISSING_DIRS),)
+$(error The custom skeleton in $(SKELETON_CUSTOM_PATH) is \
+	missing those directories or symlinks: \
+	$(SKELETON_CUSTOM_MISSING_DIRS))
+endif
+ifneq ($(SKELETON_CUSTOM_NOT_MERGED_USR_DIRS),)
+$(error The custom skeleton in $(SKELETON_PATH) is not \
+       using a merged /usr for the following directories: \
+       $(SKELETON_CUSTOM_NOT_MERGED_USR_DIRS))
+endif
+endif
+
+# Provided by the 'skeleton' package:
+# - SKELETON_RSYNC
+# - SKELETON_LIB_SYMLINK
+
+# The target-dir-warning file and the lib{32,64} symlinks are the only
+# things we customise in the custom skeleton.
+define SKELETON_CUSTOM_INSTALL_TARGET_CMDS
+	$(call SYSTEM_RSYNC,$(SKELETON_CUSTOM_PATH),$(TARGET_DIR))
+	$(call SYSTEM_LIB_SYMLINK,$(TARGET_DIR))
+	$(INSTALL) -m 0644 support/misc/target-dir-warning.txt \
+		$(TARGET_DIR_WARNING_FILE)
+endef
+
+# For the staging dir, we don't really care what we install, but we
+# need the /lib and /usr/lib appropriately setup. Since we ensure,
+# above, that they are correct in the skeleton, we can simply copy the
+# skeleton to staging.
+define SKELETON_CUSTOM_INSTALL_STAGING_CMDS
+	$(call SYSTEM_RSYNC,$(SKELETON_CUSTOM_PATH),$(STAGING_DIR))
+	$(call SYSTEM_LIB_SYMLINK,$(STAGING_DIR))
+endef
+
+$(eval $(generic-package))
diff --git a/package/skeleton/skeleton.mk b/package/skeleton/skeleton.mk
index 4ec1ecbb51..717bd65ada 100644
--- a/package/skeleton/skeleton.mk
+++ b/package/skeleton/skeleton.mk
@@ -14,57 +14,14 @@ SKELETON_ADD_SKELETON_DEPENDENCY = NO
 # The skeleton also handles the merged /usr case in the sysroot
 SKELETON_INSTALL_STAGING = YES
 
-ifeq ($(BR2_ROOTFS_SKELETON_CUSTOM),y)
+ifeq ($(BR2_PACKAGE_SKELETON_CUSTOM),y)
 
-SKELETON_PATH = $(call qstrip,$(BR2_ROOTFS_SKELETON_CUSTOM_PATH))
-
-ifeq ($(BR_BUILDING),y)
-ifeq ($(SKELETON_PATH),)
-$(error No path specified for the custom skeleton)
-endif
-endif
-
-ifeq ($(BR2_ROOTFS_MERGED_USR),y)
-
-# Ensure the user has prepared a merged /usr.
-#
-# Extract the inode numbers for all of those directories. In case any is
-# a symlink, we want to get the inode of the pointed-to directory, so we
-# append '/.' to be sure we get the target directory. Since the symlinks
-# can be anyway (/bin -> /usr/bin or /usr/bin -> /bin), we do that for
-# all of them.
-#
-SKELETON_LIB_INODE = $(shell stat -c '%i' $(SKELETON_PATH)/lib/.)
-SKELETON_BIN_INODE = $(shell stat -c '%i' $(SKELETON_PATH)/bin/.)
-SKELETON_SBIN_INODE = $(shell stat -c '%i' $(SKELETON_PATH)/sbin/.)
-SKELETON_USR_LIB_INODE = $(shell stat -c '%i' $(SKELETON_PATH)/usr/lib/.)
-SKELETON_USR_BIN_INODE = $(shell stat -c '%i' $(SKELETON_PATH)/usr/bin/.)
-SKELETON_USR_SBIN_INODE = $(shell stat -c '%i' $(SKELETON_PATH)/usr/sbin/.)
-
-ifneq ($(SKELETON_LIB_INODE),$(SKELETON_USR_LIB_INODE))
-SKELETON_CUSTOM_NOT_MERGED_USR += /lib
-endif
-ifneq ($(SKELETON_BIN_INODE),$(SKELETON_USR_BIN_INODE))
-SKELETON_CUSTOM_NOT_MERGED_USR += /bin
-endif
-ifneq ($(SKELETON_SBIN_INODE),$(SKELETON_USR_SBIN_INODE))
-SKELETON_CUSTOM_NOT_MERGED_USR += /sbin
-endif
-
-ifneq ($(SKELETON_CUSTOM_NOT_MERGED_USR),)
-$(error The custom skeleton in $(SKELETON_PATH) is not \
-	using a merged /usr for the following directories: \
-	$(SKELETON_CUSTOM_NOT_MERGED_USR))
-endif
-
-endif # merged /usr
+SKELETON_DEPENDENCIES = skeleton-custom
 
 else # ! custom skeleton
 
 SKELETON_PATH = system/skeleton
 
-endif # ! custom skeleton
-
 define SKELETON_INSTALL_TARGET_CMDS
 	$(call SYSTEM_RSYNC,$(SKELETON_PATH),$(TARGET_DIR))
 	$(call SYSTEM_USR_SYMLINKS_OR_DIRS,$(TARGET_DIR))
@@ -87,10 +44,6 @@ define SKELETON_INSTALL_STAGING_CMDS
 	$(call SYSTEM_LIB_SYMLINK,$(STAGING_DIR))
 endef
 
-# The TARGET_FINALIZE_HOOKS must be sourced only if the users choose to use the
-# default skeleton.
-ifeq ($(BR2_ROOTFS_SKELETON_DEFAULT),y)
-
 SKELETON_TARGET_GENERIC_HOSTNAME = $(call qstrip,$(BR2_TARGET_GENERIC_HOSTNAME))
 SKELETON_TARGET_GENERIC_ISSUE = $(call qstrip,$(BR2_TARGET_GENERIC_ISSUE))
 SKELETON_TARGET_GENERIC_ROOT_PASSWD = $(call qstrip,$(BR2_TARGET_GENERIC_ROOT_PASSWD))
diff --git a/system/Config.in b/system/Config.in
index 9b42fbbbac..5f8770d6c9 100644
--- a/system/Config.in
+++ b/system/Config.in
@@ -17,7 +17,7 @@ comment "default target skeleton needs an init system"
 
 config BR2_ROOTFS_SKELETON_CUSTOM
 	bool "custom target skeleton"
-	select BR2_PACKAGE_SKELETON
+	select BR2_PACKAGE_SKELETON_CUSTOM
 	help
 	  Use custom target skeleton.
 
-- 
2.11.0

[Buildroot] [PATCH 05/14 v4] package/skeleton: split out into skeleton-common

[Yann E. MORIN]

Move all the handling of the default skeleton into a new package,
skeleton-common.

We don;t name it skeleton-default, because it will be further split
later, into a skeleton for sysv and another for systemd, with some parts
still common between the two. So just name it skeleton-common right now;
this will save us a rename later.

We keep in the skeleton package all definitions of macros and variables
that can be used by other skeleton packages and/or init systems, and
just offload the actual feeding of the skeleton to the new package.

Note: it would be technically sound to move the skeleton files together
within a sub-directory of the skeleton-common package. However, we refer
the user to those fioles, from various locations (manual, packages).It
will indeed be easier for the user to find those files in
system/skeleton/ rather than in package/skeleton-common/skeleton/

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>

---
Changes v3 -> v4:
  - nothing, except add a blurb why skeleton files were not moved
---
 package/Config.in                          |  1 +
 package/skeleton-common/Config.in          |  3 +
 package/skeleton-common/skeleton-common.mk | 96 ++++++++++++++++++++++++++++++
 package/skeleton/skeleton.mk               | 87 +--------------------------
 system/Config.in                           |  2 +-
 5 files changed, 102 insertions(+), 87 deletions(-)
 create mode 100644 package/skeleton-common/Config.in
 create mode 100644 package/skeleton-common/skeleton-common.mk

diff --git a/package/Config.in b/package/Config.in
index 5e6e53d42b..8a67e86d7e 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -2,6 +2,7 @@ menu "Target packages"
 
 	source "package/busybox/Config.in"
 	source "package/skeleton/Config.in"
+	source "package/skeleton-common/Config.in"
 	source "package/skeleton-custom/Config.in"
 
 menu "Audio and video applications"
diff --git a/package/skeleton-common/Config.in b/package/skeleton-common/Config.in
new file mode 100644
index 0000000000..6c094f6466
--- /dev/null
+++ b/package/skeleton-common/Config.in
@@ -0,0 +1,3 @@
+config BR2_PACKAGE_SKELETON_COMMON
+	bool
+	select BR2_PACKAGE_SKELETON
diff --git a/package/skeleton-common/skeleton-common.mk b/package/skeleton-common/skeleton-common.mk
new file mode 100644
index 0000000000..788a17dd61
--- /dev/null
+++ b/package/skeleton-common/skeleton-common.mk
@@ -0,0 +1,96 @@
+################################################################################
+#
+# skeleton-common
+#
+################################################################################
+
+# The skeleton can't depend on the toolchain, since all packages depends on the
+# skeleton and the toolchain is a target package, as is skeleton.
+# Hence, skeleton would depends on the toolchain and the toolchain would depend
+# on skeleton.
+SKELETON_COMMON_ADD_TOOLCHAIN_DEPENDENCY = NO
+SKELETON_COMMON_ADD_SKELETON_DEPENDENCY = NO
+
+# The skeleton also handles the merged /usr case in the sysroot
+SKELETON_COMMON_INSTALL_STAGING = YES
+
+SKELETON_COMMON_PATH = system/skeleton
+
+define SKELETON_COMMON_INSTALL_TARGET_CMDS
+	$(call SYSTEM_RSYNC,$(SKELETON_COMMON_PATH),$(TARGET_DIR))
+	$(call SYSTEM_USR_SYMLINKS_OR_DIRS,$(TARGET_DIR))
+	$(call SYSTEM_LIB_SYMLINK,$(TARGET_DIR))
+	$(INSTALL) -m 0644 support/misc/target-dir-warning.txt \
+		$(TARGET_DIR_WARNING_FILE)
+endef
+
+# For the staging dir, we don't really care about /bin and /sbin.
+# But for consistency with the target dir, and to simplify the code,
+# we still handle them for the merged or non-merged /usr cases.
+# Since the toolchain is not yet available, the staging is not yet
+# populated, so we need to create the directories in /usr
+define SKELETON_COMMON_INSTALL_STAGING_CMDS
+	$(INSTALL) -d -m 0755 $(STAGING_DIR)/usr/lib
+	$(INSTALL) -d -m 0755 $(STAGING_DIR)/usr/bin
+	$(INSTALL) -d -m 0755 $(STAGING_DIR)/usr/sbin
+	$(INSTALL) -d -m 0755 $(STAGING_DIR)/usr/include
+	$(call SYSTEM_USR_SYMLINKS_OR_DIRS,$(STAGING_DIR))
+	$(call SYSTEM_LIB_SYMLINK,$(STAGING_DIR))
+endef
+
+SKELETON_COMMON_HOSTNAME = $(call qstrip,$(BR2_TARGET_GENERIC_HOSTNAME))
+SKELETON_COMMON_ISSUE = $(call qstrip,$(BR2_TARGET_GENERIC_ISSUE))
+SKELETON_COMMON_ROOT_PASSWD = $(call qstrip,$(BR2_TARGET_GENERIC_ROOT_PASSWD))
+SKELETON_COMMON_PASSWD_METHOD = $(call qstrip,$(BR2_TARGET_GENERIC_PASSWD_METHOD))
+SKELETON_COMMON_BIN_SH = $(call qstrip,$(BR2_SYSTEM_BIN_SH))
+
+ifneq ($(SKELETON_COMMON_HOSTNAME),)
+define SKELETON_COMMON_SET_HOSTNAME
+	mkdir -p $(TARGET_DIR)/etc
+	echo "$(SKELETON_COMMON_HOSTNAME)" > $(TARGET_DIR)/etc/hostname
+	$(SED) '$$a \127.0.1.1\t$(SKELETON_COMMON_HOSTNAME)' \
+		-e '/^127.0.1.1/d' $(TARGET_DIR)/etc/hosts
+endef
+SKELETON_COMMON_TARGET_FINALIZE_HOOKS += SKELETON_COMMON_SET_HOSTNAME
+endif
+
+ifneq ($(SKELETON_COMMON_ISSUE),)
+define SKELETON_COMMON_SET_ISSUE
+	mkdir -p $(TARGET_DIR)/etc
+	echo "$(SKELETON_COMMON_ISSUE)" > $(TARGET_DIR)/etc/issue
+endef
+SKELETON_COMMON_TARGET_FINALIZE_HOOKS += SKELETON_COMMON_SET_ISSUE
+endif
+
+ifeq ($(BR2_TARGET_ENABLE_ROOT_LOGIN),y)
+ifeq ($(SKELETON_COMMON_ROOT_PASSWD),)
+SKELETON_ROOT_PASSWORD =
+else ifneq ($(filter $$1$$% $$5$$% $$6$$%,$(SKELETON_COMMON_ROOT_PASSWD)),)
+SKELETON_ROOT_PASSWORD = '$(SKELETON_COMMON_ROOT_PASSWD)'
+else
+# This variable will only be evaluated in the finalize stage, so we can
+# be sure that host-mkpasswd will have already been built by that time.
+SKELETON_ROOT_PASSWORD = "`$(MKPASSWD) -m "$(SKELETON_COMMON_PASSWD_METHOD)" "$(SKELETON_COMMON_ROOT_PASSWD)"`"
+endif
+else # !BR2_TARGET_ENABLE_ROOT_LOGIN
+SKELETON_ROOT_PASSWORD = "*"
+endif
+define SKELETON_COMMON_SET_ROOT_PASSWD
+	$(SED) s,^root:[^:]*:,root:$(SKELETON_ROOT_PASSWORD):, $(TARGET_DIR)/etc/shadow
+endef
+SKELETON_COMMON_TARGET_FINALIZE_HOOKS += SKELETON_COMMON_SET_ROOT_PASSWD
+
+ifeq ($(BR2_SYSTEM_BIN_SH_NONE),y)
+define SKELETON_COMMON_BIN_SH
+	rm -f $(TARGET_DIR)/bin/sh
+endef
+else
+ifneq ($(SKELETON_COMMON_BIN_SH),)
+define SKELETON_COMMON_BIN_SH
+	ln -sf $(SKELETON_COMMON_BIN_SH) $(TARGET_DIR)/bin/sh
+endef
+endif
+endif
+SKELETON_COMMON_TARGET_FINALIZE_HOOKS += SKELETON_COMMON_BIN_SH
+
+$(eval $(generic-package))
diff --git a/package/skeleton/skeleton.mk b/package/skeleton/skeleton.mk
index 717bd65ada..1d3e4fbe3c 100644
--- a/package/skeleton/skeleton.mk
+++ b/package/skeleton/skeleton.mk
@@ -11,95 +11,10 @@
 SKELETON_ADD_TOOLCHAIN_DEPENDENCY = NO
 SKELETON_ADD_SKELETON_DEPENDENCY = NO
 
-# The skeleton also handles the merged /usr case in the sysroot
-SKELETON_INSTALL_STAGING = YES
-
 ifeq ($(BR2_PACKAGE_SKELETON_CUSTOM),y)
-
 SKELETON_DEPENDENCIES = skeleton-custom
-
-else # ! custom skeleton
-
-SKELETON_PATH = system/skeleton
-
-define SKELETON_INSTALL_TARGET_CMDS
-	$(call SYSTEM_RSYNC,$(SKELETON_PATH),$(TARGET_DIR))
-	$(call SYSTEM_USR_SYMLINKS_OR_DIRS,$(TARGET_DIR))
-	$(call SYSTEM_LIB_SYMLINK,$(TARGET_DIR))
-	$(INSTALL) -m 0644 support/misc/target-dir-warning.txt \
-		$(TARGET_DIR_WARNING_FILE)
-endef
-
-# For the staging dir, we don't really care about /bin and /sbin.
-# But for consistency with the target dir, and to simplify the code,
-# we still handle them for the merged or non-merged /usr cases.
-# Since the toolchain is not yet available, the staging is not yet
-# populated, so we need to create the directories in /usr
-define SKELETON_INSTALL_STAGING_CMDS
-	$(INSTALL) -d -m 0755 $(STAGING_DIR)/usr/lib
-	$(INSTALL) -d -m 0755 $(STAGING_DIR)/usr/bin
-	$(INSTALL) -d -m 0755 $(STAGING_DIR)/usr/sbin
-	$(INSTALL) -d -m 0755 $(STAGING_DIR)/usr/include
-	$(call SYSTEM_USR_SYMLINKS_OR_DIRS,$(STAGING_DIR))
-	$(call SYSTEM_LIB_SYMLINK,$(STAGING_DIR))
-endef
-
-SKELETON_TARGET_GENERIC_HOSTNAME = $(call qstrip,$(BR2_TARGET_GENERIC_HOSTNAME))
-SKELETON_TARGET_GENERIC_ISSUE = $(call qstrip,$(BR2_TARGET_GENERIC_ISSUE))
-SKELETON_TARGET_GENERIC_ROOT_PASSWD = $(call qstrip,$(BR2_TARGET_GENERIC_ROOT_PASSWD))
-SKELETON_TARGET_GENERIC_PASSWD_METHOD = $(call qstrip,$(BR2_TARGET_GENERIC_PASSWD_METHOD))
-SKELETON_TARGET_GENERIC_BIN_SH = $(call qstrip,$(BR2_SYSTEM_BIN_SH))
-
-ifneq ($(SKELETON_TARGET_GENERIC_HOSTNAME),)
-define SKELETON_SET_HOSTNAME
-	mkdir -p $(TARGET_DIR)/etc
-	echo "$(SKELETON_TARGET_GENERIC_HOSTNAME)" > $(TARGET_DIR)/etc/hostname
-	$(SED) '$$a \127.0.1.1\t$(SKELETON_TARGET_GENERIC_HOSTNAME)' \
-		-e '/^127.0.1.1/d' $(TARGET_DIR)/etc/hosts
-endef
-TARGET_FINALIZE_HOOKS += SKELETON_SET_HOSTNAME
-endif
-
-ifneq ($(SKELETON_TARGET_GENERIC_ISSUE),)
-define SKELETON_SET_ISSUE
-	mkdir -p $(TARGET_DIR)/etc
-	echo "$(SKELETON_TARGET_GENERIC_ISSUE)" > $(TARGET_DIR)/etc/issue
-endef
-TARGET_FINALIZE_HOOKS += SKELETON_SET_ISSUE
-endif
-
-ifeq ($(BR2_TARGET_ENABLE_ROOT_LOGIN),y)
-ifeq ($(SKELETON_TARGET_GENERIC_ROOT_PASSWD),)
-SKELETON_ROOT_PASSWORD =
-else ifneq ($(filter $$1$$% $$5$$% $$6$$%,$(SKELETON_TARGET_GENERIC_ROOT_PASSWD)),)
-SKELETON_ROOT_PASSWORD = '$(SKELETON_TARGET_GENERIC_ROOT_PASSWD)'
-else
-# This variable will only be evaluated in the finalize stage, so we can
-# be sure that host-mkpasswd will have already been built by that time.
-SKELETON_ROOT_PASSWORD = "`$(MKPASSWD) -m "$(SKELETON_TARGET_GENERIC_PASSWD_METHOD)" "$(SKELETON_TARGET_GENERIC_ROOT_PASSWD)"`"
-endif
-else # !BR2_TARGET_ENABLE_ROOT_LOGIN
-SKELETON_ROOT_PASSWORD = "*"
-endif
-
-define SKELETON_SET_ROOT_PASSWD
-	$(SED) s,^root:[^:]*:,root:$(SKELETON_ROOT_PASSWORD):, $(TARGET_DIR)/etc/shadow
-endef
-TARGET_FINALIZE_HOOKS += SKELETON_SET_ROOT_PASSWD
-
-ifeq ($(BR2_SYSTEM_BIN_SH_NONE),y)
-define SKELETON_BIN_SH
-	rm -f $(TARGET_DIR)/bin/sh
-endef
 else
-ifneq ($(SKELETON_TARGET_GENERIC_BIN_SH),)
-define SKELETON_BIN_SH
-	ln -sf $(SKELETON_TARGET_GENERIC_BIN_SH) $(TARGET_DIR)/bin/sh
-endef
-endif
+SKELETON_DEPENDENCIES = skeleton-common
 endif
-TARGET_FINALIZE_HOOKS += SKELETON_BIN_SH
-
-endif # BR2_ROOTFS_SKELETON_DEFAULT
 
 $(eval $(generic-package))
diff --git a/system/Config.in b/system/Config.in
index 5f8770d6c9..c27b013784 100644
--- a/system/Config.in
+++ b/system/Config.in
@@ -8,7 +8,7 @@ choice
 config BR2_ROOTFS_SKELETON_DEFAULT
 	bool "default target skeleton"
 	depends on !BR2_INIT_NONE
-	select BR2_PACKAGE_SKELETON
+	select BR2_PACKAGE_SKELETON_COMMON
 	help
 	  Use default target skeleton
 
-- 
2.11.0

[Buildroot] [PATCH 06/14 v4] package/skeleton: make it a virtual package

[Yann E. MORIN]

We now have two packages that can act as a skeleton, skeleton-common,
also known as our default skeleton, and skeleton-custom.

This means that the skeleton package can be a standard virtual package
now.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>

---
Changes v3 -> v4:
  - drop now-legacy paragraph from the commit log  (Arnout)
---
 package/skeleton-common/Config.in          | 5 ++++-
 package/skeleton-common/skeleton-common.mk | 2 ++
 package/skeleton-custom/Config.in          | 5 ++++-
 package/skeleton-custom/skeleton-custom.mk | 2 ++
 package/skeleton/Config.in                 | 9 +++++++--
 package/skeleton/skeleton.mk               | 8 +-------
 system/Config.in                           | 4 ++++
 7 files changed, 24 insertions(+), 11 deletions(-)

diff --git a/package/skeleton-common/Config.in b/package/skeleton-common/Config.in
index 6c094f6466..5675e873cf 100644
--- a/package/skeleton-common/Config.in
+++ b/package/skeleton-common/Config.in
@@ -1,3 +1,6 @@
 config BR2_PACKAGE_SKELETON_COMMON
 	bool
-	select BR2_PACKAGE_SKELETON
+	select BR2_PACKAGE_HAS_SKELETON
+
+config BR2_PACKAGE_PROVIDES_SKELETON
+	default "skeleton-common" if BR2_PACKAGE_SKELETON_COMMON
diff --git a/package/skeleton-common/skeleton-common.mk b/package/skeleton-common/skeleton-common.mk
index 788a17dd61..ee44eaf810 100644
--- a/package/skeleton-common/skeleton-common.mk
+++ b/package/skeleton-common/skeleton-common.mk
@@ -11,6 +11,8 @@
 SKELETON_COMMON_ADD_TOOLCHAIN_DEPENDENCY = NO
 SKELETON_COMMON_ADD_SKELETON_DEPENDENCY = NO
 
+SKELETON_COMMON_PROVIDES = skeleton
+
 # The skeleton also handles the merged /usr case in the sysroot
 SKELETON_COMMON_INSTALL_STAGING = YES
 
diff --git a/package/skeleton-custom/Config.in b/package/skeleton-custom/Config.in
index b12bd8f73c..601c3b247e 100644
--- a/package/skeleton-custom/Config.in
+++ b/package/skeleton-custom/Config.in
@@ -1,3 +1,6 @@
 config BR2_PACKAGE_SKELETON_CUSTOM
 	bool
-	select BR2_PACKAGE_SKELETON
+	select BR2_PACKAGE_HAS_SKELETON
+
+config BR2_PACKAGE_PROVIDES_SKELETON
+	default "skeleton-custom" if BR2_PACKAGE_SKELETON_CUSTOM
diff --git a/package/skeleton-custom/skeleton-custom.mk b/package/skeleton-custom/skeleton-custom.mk
index ed7b7ac7c5..3d0243de4d 100644
--- a/package/skeleton-custom/skeleton-custom.mk
+++ b/package/skeleton-custom/skeleton-custom.mk
@@ -7,6 +7,8 @@
 SKELETON_CUSTOM_ADD_TOOLCHAIN_DEPENDENCY = NO
 SKELETON_CUSTOM_ADD_SKELETON_DEPENDENCY = NO
 
+SKELETON_CUSTOM_PROVIDES = skeleton
+
 SKELETON_CUSTOM_INSTALL_STAGING = YES
 
 SKELETON_CUSTOM_PATH = $(call qstrip,$(BR2_ROOTFS_SKELETON_CUSTOM_PATH))
diff --git a/package/skeleton/Config.in b/package/skeleton/Config.in
index b22ac66b35..efaa1e135f 100644
--- a/package/skeleton/Config.in
+++ b/package/skeleton/Config.in
@@ -1,4 +1,9 @@
 config BR2_PACKAGE_SKELETON
 	bool
-	help
-	  The basic skeleton for your rootfs.
+	default y
+
+config BR2_PACKAGE_HAS_SKELETON
+	bool
+
+config BR2_PACKAGE_PROVIDES_SKELETON
+	string
diff --git a/package/skeleton/skeleton.mk b/package/skeleton/skeleton.mk
index 1d3e4fbe3c..d380f41649 100644
--- a/package/skeleton/skeleton.mk
+++ b/package/skeleton/skeleton.mk
@@ -11,10 +11,4 @@
 SKELETON_ADD_TOOLCHAIN_DEPENDENCY = NO
 SKELETON_ADD_SKELETON_DEPENDENCY = NO
 
-ifeq ($(BR2_PACKAGE_SKELETON_CUSTOM),y)
-SKELETON_DEPENDENCIES = skeleton-custom
-else
-SKELETON_DEPENDENCIES = skeleton-common
-endif
-
-$(eval $(generic-package))
+$(eval $(virtual-package))
diff --git a/system/Config.in b/system/Config.in
index c27b013784..75b6a8edac 100644
--- a/system/Config.in
+++ b/system/Config.in
@@ -1,5 +1,9 @@
 menu "System configuration"
 
+# Note: usually, it is not possible to select a provider of a virtual
+# package. But here we have an exception: there are only two providers
+# and they only get selected each by separate entries in this choice.
+# So this is a safe situation.
 choice
 	prompt "Root FS skeleton"
 
-- 
2.11.0

[Buildroot] [PATCH 07/14 v4] package/skeleton-common: simplify staging install

[Yann E. MORIN]

We don't really care what's going in staging, as long as it is properly
setup for merged/non-merged /usr, especially for the lib/ directory.

So we can just copy the skeleton as-is.

This simplify the maintenance, should we ever need to tweak the layout:
we'd just have to do it once in the skeleton directory to have it
propagated to both target and staging.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
 package/skeleton-common/skeleton-common.mk | 14 +++++---------
 1 file changed, 5 insertions(+), 9 deletions(-)

diff --git a/package/skeleton-common/skeleton-common.mk b/package/skeleton-common/skeleton-common.mk
index ee44eaf810..475c07e32d 100644
--- a/package/skeleton-common/skeleton-common.mk
+++ b/package/skeleton-common/skeleton-common.mk
@@ -26,18 +26,14 @@ define SKELETON_COMMON_INSTALL_TARGET_CMDS
 		$(TARGET_DIR_WARNING_FILE)
 endef
 
-# For the staging dir, we don't really care about /bin and /sbin.
-# But for consistency with the target dir, and to simplify the code,
-# we still handle them for the merged or non-merged /usr cases.
-# Since the toolchain is not yet available, the staging is not yet
-# populated, so we need to create the directories in /usr
+# We don't care much about what goes in staging, as long as it is
+# correctly setup for merged/non-merged /usr. The simplest is to
+# fill it in with the content of the skeleton.
 define SKELETON_COMMON_INSTALL_STAGING_CMDS
-	$(INSTALL) -d -m 0755 $(STAGING_DIR)/usr/lib
-	$(INSTALL) -d -m 0755 $(STAGING_DIR)/usr/bin
-	$(INSTALL) -d -m 0755 $(STAGING_DIR)/usr/sbin
-	$(INSTALL) -d -m 0755 $(STAGING_DIR)/usr/include
+	$(call SYSTEM_RSYNC,$(SKELETON_COMMON_PATH),$(STAGING_DIR))
 	$(call SYSTEM_USR_SYMLINKS_OR_DIRS,$(STAGING_DIR))
 	$(call SYSTEM_LIB_SYMLINK,$(STAGING_DIR))
+	$(INSTALL) -d -m 0755 $(STAGING_DIR)/usr/include
 endef
 
 SKELETON_COMMON_HOSTNAME = $(call qstrip,$(BR2_TARGET_GENERIC_HOSTNAME))
-- 
2.11.0

[Buildroot] [PATCH 08/14 v4] package/skeleton: introduce sysv- and systemd-specific skeletons

[Yann E. MORIN]

Currently, we use the same skeleton for sysv-like init systems and
systemd, even though systemd has some peculiarities that makes out
default skeleton unfit.

So, we'll need to provide different skeletons (really, only part of it)
for sysv-like and systemd.

Introduce two new skeleton packages, aptly named skeleton-sysv and
skeleton-systemd, one for each, that are both providers of the skeleton
package, in lieue of the skeleton-common, which is now a simple
dependency of both the two new skeletons.

Those packages are empty for now. In followup changes:
  - sysv-specific stuff will be moved out of skeleton-common and into
    skeleton-sysv;
  - systemd-specific stuff will be added to skeleton-systemd.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
 package/Config.in                            |  2 ++
 package/skeleton-common/Config.in            |  4 ----
 package/skeleton-common/skeleton-common.mk   |  2 --
 package/skeleton-systemd/Config.in           |  7 +++++++
 package/skeleton-systemd/skeleton-systemd.mk | 18 ++++++++++++++++++
 package/skeleton-sysv/Config.in              |  7 +++++++
 package/skeleton-sysv/skeleton-sysv.mk       | 18 ++++++++++++++++++
 system/Config.in                             | 10 ++++++----
 8 files changed, 58 insertions(+), 10 deletions(-)
 create mode 100644 package/skeleton-systemd/Config.in
 create mode 100644 package/skeleton-systemd/skeleton-systemd.mk
 create mode 100644 package/skeleton-sysv/Config.in
 create mode 100644 package/skeleton-sysv/skeleton-sysv.mk

diff --git a/package/Config.in b/package/Config.in
index 8a67e86d7e..390477072d 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -4,6 +4,8 @@ menu "Target packages"
 	source "package/skeleton/Config.in"
 	source "package/skeleton-common/Config.in"
 	source "package/skeleton-custom/Config.in"
+	source "package/skeleton-systemd/Config.in"
+	source "package/skeleton-sysv/Config.in"
 
 menu "Audio and video applications"
 	source "package/alsa-utils/Config.in"
diff --git a/package/skeleton-common/Config.in b/package/skeleton-common/Config.in
index 5675e873cf..9abed4292a 100644
--- a/package/skeleton-common/Config.in
+++ b/package/skeleton-common/Config.in
@@ -1,6 +1,2 @@
 config BR2_PACKAGE_SKELETON_COMMON
 	bool
-	select BR2_PACKAGE_HAS_SKELETON
-
-config BR2_PACKAGE_PROVIDES_SKELETON
-	default "skeleton-common" if BR2_PACKAGE_SKELETON_COMMON
diff --git a/package/skeleton-common/skeleton-common.mk b/package/skeleton-common/skeleton-common.mk
index 475c07e32d..57cee48931 100644
--- a/package/skeleton-common/skeleton-common.mk
+++ b/package/skeleton-common/skeleton-common.mk
@@ -11,8 +11,6 @@
 SKELETON_COMMON_ADD_TOOLCHAIN_DEPENDENCY = NO
 SKELETON_COMMON_ADD_SKELETON_DEPENDENCY = NO
 
-SKELETON_COMMON_PROVIDES = skeleton
-
 # The skeleton also handles the merged /usr case in the sysroot
 SKELETON_COMMON_INSTALL_STAGING = YES
 
diff --git a/package/skeleton-systemd/Config.in b/package/skeleton-systemd/Config.in
new file mode 100644
index 0000000000..c507264598
--- /dev/null
+++ b/package/skeleton-systemd/Config.in
@@ -0,0 +1,7 @@
+config BR2_PACKAGE_SKELETON_SYSTEMD
+	bool
+	select BR2_PACKAGE_HAS_SKELETON
+	select BR2_PACKAGE_SKELETON_COMMON
+
+config BR2_PACKAGE_PROVIDES_SKELETON
+	default "skeleton-systemd" if BR2_PACKAGE_SKELETON_SYSTEMD
diff --git a/package/skeleton-systemd/skeleton-systemd.mk b/package/skeleton-systemd/skeleton-systemd.mk
new file mode 100644
index 0000000000..cec6359007
--- /dev/null
+++ b/package/skeleton-systemd/skeleton-systemd.mk
@@ -0,0 +1,18 @@
+################################################################################
+#
+# skeleton-systemd
+#
+################################################################################
+
+# The skeleton can't depend on the toolchain, since all packages depends on the
+# skeleton and the toolchain is a target package, as is skeleton.
+# Hence, skeleton would depends on the toolchain and the toolchain would depend
+# on skeleton.
+SKELETON_SYSTEMD_ADD_TOOLCHAIN_DEPENDENCY = NO
+SKELETON_SYSTEMD_ADD_SKELETON_DEPENDENCY = NO
+
+SKELETON_SYSTEMD_DEPENDENCIES = skeleton-common
+
+SKELETON_SYSTEMD_PROVIDES = skeleton
+
+$(eval $(generic-package))
diff --git a/package/skeleton-sysv/Config.in b/package/skeleton-sysv/Config.in
new file mode 100644
index 0000000000..2f6dbd9673
--- /dev/null
+++ b/package/skeleton-sysv/Config.in
@@ -0,0 +1,7 @@
+config BR2_PACKAGE_SKELETON_SYSV
+	bool
+	select BR2_PACKAGE_HAS_SKELETON
+	select BR2_PACKAGE_SKELETON_COMMON
+
+config BR2_PACKAGE_PROVIDES_SKELETON
+	default "skeleton-sysv" if BR2_PACKAGE_SKELETON_SYSV
diff --git a/package/skeleton-sysv/skeleton-sysv.mk b/package/skeleton-sysv/skeleton-sysv.mk
new file mode 100644
index 0000000000..b0c2b6bac1
--- /dev/null
+++ b/package/skeleton-sysv/skeleton-sysv.mk
@@ -0,0 +1,18 @@
+################################################################################
+#
+# skeleton-sysv
+#
+################################################################################
+
+# The skeleton can't depend on the toolchain, since all packages depends on the
+# skeleton and the toolchain is a target package, as is skeleton.
+# Hence, skeleton would depends on the toolchain and the toolchain would depend
+# on skeleton.
+SKELETON_SYSV_ADD_TOOLCHAIN_DEPENDENCY = NO
+SKELETON_SYSV_ADD_SKELETON_DEPENDENCY = NO
+
+SKELETON_SYSV_DEPENDENCIES = skeleton-common
+
+SKELETON_SYSV_PROVIDES = skeleton
+
+$(eval $(generic-package))
diff --git a/system/Config.in b/system/Config.in
index 75b6a8edac..65139c0bfd 100644
--- a/system/Config.in
+++ b/system/Config.in
@@ -1,9 +1,9 @@
 menu "System configuration"
 
 # Note: usually, it is not possible to select a provider of a virtual
-# package. But here we have an exception: there are only two providers
-# and they only get selected each by separate entries in this choice.
-# So this is a safe situation.
+# package. But here we have an exception: there are only three providers
+# and they only get selected by separate entries in this choice and
+# under different, exclusive conditions. So this is a safe situation.
 choice
 	prompt "Root FS skeleton"
 
@@ -12,7 +12,9 @@ choice
 config BR2_ROOTFS_SKELETON_DEFAULT
 	bool "default target skeleton"
 	depends on !BR2_INIT_NONE
-	select BR2_PACKAGE_SKELETON_COMMON
+	select BR2_PACKAGE_SKELETON_SYSV if BR2_INIT_SYSV
+	select BR2_PACKAGE_SKELETON_SYSV if BR2_INIT_BUSYBOX
+	select BR2_PACKAGE_SKELETON_SYSTEMD if BR2_INIT_SYSTEMD
 	help
 	  Use default target skeleton
 
-- 
2.11.0

[Buildroot] [PATCH 00/14 v4] Hello All!

[Yann E. MORIN]

Hello All!

This series is a proposal to fix our handling of systemd as an init
system.

Our default skeleton is not well suited for systemd:

  - we have /var/log a symlink to /tmp/log, but the way systemd starts
    hides the journals:
    - start systemd-journald, stores journals in /tmp/log/ (because of
      the redirection above)
    - mounts a tmpfs over /tmp, thus hidding the journals.

  - on a read-only rootfs, systemd expects /var to be read-write, which
    we do not provide. Read-only rootfs will be handled in a separate
    series.

All of this sounds trivial, but fixing it is definitely not.

The overall idea is that we need different skeletons, one for each type
of init systems: sysv, systemd, others (aka custom).

The skeleton package becomes a virtual package, that is provided by one
of four implementations:

  - skeleton-sysv or skeleton-systemd, as their names imply, for either
    SysV- or systemd-based systems, and which depend on skeleton-common;

  - skeleton-common, that provides base files and directories comon to
    all skeletons (except custom, below); skeleton-common is also a
    provider for the skeleton virtual package, for use with no-init (aka
    custom init) systems;

  - skeleton-custom, for user-provided custom skeleton, and of which we
    expect it is fully prepared; skeleton-custom does not depend on
    skeleton-common.

Eventually, we add support for runing systemd on a read-only rootfs.
This requires that we play tricks with /var and the systemd-tmpfiles
factory feautre.

A series of runtime tests have also been added, running on the QEMU
vexpress:

    init system         rootfs      DHCP?
    -------------------------------------
    busybox             ext4        no
    busybox             ext4        eth0
    busybox             squashfs    no
    busybox             squashfs    eth0

  * systemd-ifupdown    ext4        eth0
  * systemd-ifupdown    squashfs    eth0
 ** systemd-networkd    ext4        eth0
 ** systemd-networkd    squashfs    eth0
*** systemd-full        ext4        eth0
*** systemd-full        squashfs    eth0

  + no-init             squashfs    no

  * systemd-networkd: only what gets selected automatically when
    systemd is used as init system, with all other options unset;

 ** systemd-ifupdown: like systemd-networkd, but with networkd
    disabled, thus using ifupdown instead;

*** systemd-full: all systemd options enabled, implies networkd;

  + a shell is launched as init

Those new tests are not exhaustive, especially the systemd one; instead,
they just test basic features: what init is being used, and did we get
an IP adress for eth0? More tests can be added later on.

Changes v3 -> v4:
  - fix the read-only support  (Arnout)
  - add reviewed-by tags from Arnout. Thanks a lot!
  - enhance some commit logs after those reviews
  - simplify no-init skeleton-common  (Arnout)

Changes v2 -> v3:
  - too much to note, mostly rewriten from scratch

Changes v1 -> v2:
  - enhance the commit logs


Regards,
Yann E. MORIN.


The following changes since commit de46cc9be0873fbd60b19552c5687ad142aa0c99

  test-pkg: use merge_config.sh to merge the fragments (2017-07-25 23:07:03 +0200)


are available in the git repository at:

  git://git.buildroot.org/~ymorin/git/buildroot.git

for you to fetch changes up to bcdaf713d6a72b04818c74b1810db0beac85972b

  support/testing: add runtime testing for read-only systemd (2017-07-25 23:11:14 +0200)


----------------------------------------------------------------
Yann E. MORIN (14):
      system: with no init system, only allow custom skeleton
      package/skeleton: drop dependency on host-mkpasswd
      package/skeleton: select it rather than default to y
      package/skeleton: split out into skeleton-custom
      package/skeleton: split out into skeleton-common
      package/skeleton: make it a virtual package
      package/skeleton-common: simplify staging install
      package/skeleton: introduce sysv- and systemd-specific skeletons
      system: separate sysv and systemd parts of the skeleton
      system: no-init systems may use our default, common skeleton
      support/testing: add runtime testing for init systems
      fs: add pre- and post-command hooks
      system: make systemd work on a read-only rootfs
      support/testing: add runtime testing for read-only systemd

 fs/common.mk                                       |   4 +
 package/Config.in                                  |   4 +
 package/pkg-generic.mk                             |   4 +
 package/skeleton-common/Config.in                  |  17 +++
 package/skeleton-common/skeleton-common.mk         |  92 +++++++++++++
 package/skeleton-custom/Config.in                  |   6 +
 package/skeleton-custom/skeleton-custom.mk         | 107 +++++++++++++++
 package/skeleton-systemd/Config.in                 |   7 +
 package/skeleton-systemd/skeleton-systemd.mk       |  71 ++++++++++
 package/skeleton-sysv/Config.in                    |   7 +
 package/skeleton-sysv/skeleton-sysv.mk             |  22 +++
 {system => package/skeleton-sysv}/skeleton/dev/log |   0
 .../skeleton-sysv}/skeleton/dev/pts/.empty         |   0
 .../skeleton-sysv}/skeleton/dev/shm/.empty         |   0
 .../skeleton-sysv}/skeleton/etc/fstab              |   0
 .../skeleton-sysv}/skeleton/var/cache              |   0
 .../skeleton-sysv}/skeleton/var/lib/misc           |   0
 .../skeleton-sysv}/skeleton/var/lock               |   0
 {system => package/skeleton-sysv}/skeleton/var/log |   0
 {system => package/skeleton-sysv}/skeleton/var/run |   0
 .../skeleton-sysv}/skeleton/var/spool              |   0
 {system => package/skeleton-sysv}/skeleton/var/tmp |   0
 package/skeleton/Config.in                         |   8 +-
 package/skeleton/skeleton.mk                       | 141 +------------------
 .../testing/tests/init/__init__.py                 |   0
 support/testing/tests/init/base.py                 |  47 +++++++
 .../testing/tests/init/systemd-factory/var/foo/bar |   1 +
 support/testing/tests/init/test_busybox.py         |  67 +++++++++
 support/testing/tests/init/test_none.py            |  32 +++++
 support/testing/tests/init/test_systemd.py         | 152 +++++++++++++++++++++
 system/Config.in                                   |  13 +-
 system/skeleton/dev/{pts => }/.empty               |   0
 32 files changed, 659 insertions(+), 143 deletions(-)
 create mode 100644 package/skeleton-common/Config.in
 create mode 100644 package/skeleton-common/skeleton-common.mk
 create mode 100644 package/skeleton-custom/Config.in
 create mode 100644 package/skeleton-custom/skeleton-custom.mk
 create mode 100644 package/skeleton-systemd/Config.in
 create mode 100644 package/skeleton-systemd/skeleton-systemd.mk
 create mode 100644 package/skeleton-sysv/Config.in
 create mode 100644 package/skeleton-sysv/skeleton-sysv.mk
 rename {system => package/skeleton-sysv}/skeleton/dev/log (100%)
 copy {system => package/skeleton-sysv}/skeleton/dev/pts/.empty (100%)
 rename {system => package/skeleton-sysv}/skeleton/dev/shm/.empty (100%)
 rename {system => package/skeleton-sysv}/skeleton/etc/fstab (100%)
 rename {system => package/skeleton-sysv}/skeleton/var/cache (100%)
 rename {system => package/skeleton-sysv}/skeleton/var/lib/misc (100%)
 rename {system => package/skeleton-sysv}/skeleton/var/lock (100%)
 rename {system => package/skeleton-sysv}/skeleton/var/log (100%)
 rename {system => package/skeleton-sysv}/skeleton/var/run (100%)
 rename {system => package/skeleton-sysv}/skeleton/var/spool (100%)
 rename {system => package/skeleton-sysv}/skeleton/var/tmp (100%)
 copy system/skeleton/dev/pts/.empty => support/testing/tests/init/__init__.py (100%)
 create mode 100644 support/testing/tests/init/base.py
 create mode 100644 support/testing/tests/init/systemd-factory/var/foo/bar
 create mode 100644 support/testing/tests/init/test_busybox.py
 create mode 100644 support/testing/tests/init/test_none.py
 create mode 100644 support/testing/tests/init/test_systemd.py
 rename system/skeleton/dev/{pts => }/.empty (100%)

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 09/14 v4] system: separate sysv and systemd parts of the skeleton

[Yann E. MORIN]

For systemd, we create a simple /etc/fstab with only an entry for /, as
systemd otherwise automatically mounts what it needs where it needs it.

systemd does not like that the content of /var be symlinks to /tmp,
especially journald that starts before /tmp is mounted, and thus the
journal files are hidden from view, which causes quite a bit of fuss...

Instead, move the current /var to a sysv-only skeleton.

systemd at install time will create the /var content it needs, so we
just create an empty /var for systemd.

systemd would create /home and /srv at runtime if they are missing, but
it is better to create them right now, to simplify supporting systemd on
a RO filesystem in the (near) future.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>

---
Changes v3 -> v4:
  - add blurb about /etc/fstab  (Arnout)
  - blurb about /home and /srv moved from here to the commit log itself
---
 package/skeleton-systemd/skeleton-systemd.mk              | 7 +++++++
 package/skeleton-sysv/skeleton-sysv.mk                    | 4 ++++
 {system => package/skeleton-sysv}/skeleton/dev/log        | 0
 {system => package/skeleton-sysv}/skeleton/dev/pts/.empty | 0
 {system => package/skeleton-sysv}/skeleton/dev/shm/.empty | 0
 {system => package/skeleton-sysv}/skeleton/etc/fstab      | 0
 {system => package/skeleton-sysv}/skeleton/var/cache      | 0
 {system => package/skeleton-sysv}/skeleton/var/lib/misc   | 0
 {system => package/skeleton-sysv}/skeleton/var/lock       | 0
 {system => package/skeleton-sysv}/skeleton/var/log        | 0
 {system => package/skeleton-sysv}/skeleton/var/run        | 0
 {system => package/skeleton-sysv}/skeleton/var/spool      | 0
 {system => package/skeleton-sysv}/skeleton/var/tmp        | 0
 system/skeleton/dev/.empty                                | 0
 14 files changed, 11 insertions(+)
 rename {system => package/skeleton-sysv}/skeleton/dev/log (100%)
 rename {system => package/skeleton-sysv}/skeleton/dev/pts/.empty (100%)
 rename {system => package/skeleton-sysv}/skeleton/dev/shm/.empty (100%)
 rename {system => package/skeleton-sysv}/skeleton/etc/fstab (100%)
 rename {system => package/skeleton-sysv}/skeleton/var/cache (100%)
 rename {system => package/skeleton-sysv}/skeleton/var/lib/misc (100%)
 rename {system => package/skeleton-sysv}/skeleton/var/lock (100%)
 rename {system => package/skeleton-sysv}/skeleton/var/log (100%)
 rename {system => package/skeleton-sysv}/skeleton/var/run (100%)
 rename {system => package/skeleton-sysv}/skeleton/var/spool (100%)
 rename {system => package/skeleton-sysv}/skeleton/var/tmp (100%)
 create mode 100644 system/skeleton/dev/.empty

diff --git a/package/skeleton-systemd/skeleton-systemd.mk b/package/skeleton-systemd/skeleton-systemd.mk
index cec6359007..384715e1c9 100644
--- a/package/skeleton-systemd/skeleton-systemd.mk
+++ b/package/skeleton-systemd/skeleton-systemd.mk
@@ -15,4 +15,11 @@ SKELETON_SYSTEMD_DEPENDENCIES = skeleton-common
 
 SKELETON_SYSTEMD_PROVIDES = skeleton
 
+define SKELETON_SYSTEMD_INSTALL_TARGET_CMDS
+	mkdir -p $(TARGET_DIR)/home
+	mkdir -p $(TARGET_DIR)/srv
+	mkdir -p $(TARGET_DIR)/var
+	echo "/dev/root / auto rw 0 1" >$(TARGET_DIR)/etc/fstab
+endef
+
 $(eval $(generic-package))
diff --git a/package/skeleton-sysv/skeleton-sysv.mk b/package/skeleton-sysv/skeleton-sysv.mk
index b0c2b6bac1..b5fa396ac2 100644
--- a/package/skeleton-sysv/skeleton-sysv.mk
+++ b/package/skeleton-sysv/skeleton-sysv.mk
@@ -15,4 +15,8 @@ SKELETON_SYSV_DEPENDENCIES = skeleton-common
 
 SKELETON_SYSV_PROVIDES = skeleton
 
+define SKELETON_SYSV_INSTALL_TARGET_CMDS
+	$(call SYSTEM_RSYNC,$(SKELETON_SYSV_PKGDIR)/skeleton,$(TARGET_DIR))
+endef
+
 $(eval $(generic-package))
diff --git a/system/skeleton/dev/log b/package/skeleton-sysv/skeleton/dev/log
similarity index 100%
rename from system/skeleton/dev/log
rename to package/skeleton-sysv/skeleton/dev/log
diff --git a/system/skeleton/dev/pts/.empty b/package/skeleton-sysv/skeleton/dev/pts/.empty
similarity index 100%
rename from system/skeleton/dev/pts/.empty
rename to package/skeleton-sysv/skeleton/dev/pts/.empty
diff --git a/system/skeleton/dev/shm/.empty b/package/skeleton-sysv/skeleton/dev/shm/.empty
similarity index 100%
rename from system/skeleton/dev/shm/.empty
rename to package/skeleton-sysv/skeleton/dev/shm/.empty
diff --git a/system/skeleton/etc/fstab b/package/skeleton-sysv/skeleton/etc/fstab
similarity index 100%
rename from system/skeleton/etc/fstab
rename to package/skeleton-sysv/skeleton/etc/fstab
diff --git a/system/skeleton/var/cache b/package/skeleton-sysv/skeleton/var/cache
similarity index 100%
rename from system/skeleton/var/cache
rename to package/skeleton-sysv/skeleton/var/cache
diff --git a/system/skeleton/var/lib/misc b/package/skeleton-sysv/skeleton/var/lib/misc
similarity index 100%
rename from system/skeleton/var/lib/misc
rename to package/skeleton-sysv/skeleton/var/lib/misc
diff --git a/system/skeleton/var/lock b/package/skeleton-sysv/skeleton/var/lock
similarity index 100%
rename from system/skeleton/var/lock
rename to package/skeleton-sysv/skeleton/var/lock
diff --git a/system/skeleton/var/log b/package/skeleton-sysv/skeleton/var/log
similarity index 100%
rename from system/skeleton/var/log
rename to package/skeleton-sysv/skeleton/var/log
diff --git a/system/skeleton/var/run b/package/skeleton-sysv/skeleton/var/run
similarity index 100%
rename from system/skeleton/var/run
rename to package/skeleton-sysv/skeleton/var/run
diff --git a/system/skeleton/var/spool b/package/skeleton-sysv/skeleton/var/spool
similarity index 100%
rename from system/skeleton/var/spool
rename to package/skeleton-sysv/skeleton/var/spool
diff --git a/system/skeleton/var/tmp b/package/skeleton-sysv/skeleton/var/tmp
similarity index 100%
rename from system/skeleton/var/tmp
rename to package/skeleton-sysv/skeleton/var/tmp
diff --git a/system/skeleton/dev/.empty b/system/skeleton/dev/.empty
new file mode 100644
index 0000000000..e69de29bb2
-- 
2.11.0

[Buildroot] [PATCH 10/14 v4] system: no-init systems may use our default, common skeleton

[Yann E. MORIN]

Requiring that no-init system indeed provide their own custom skeletons
is a bit too-high a burden, especially for quick one-off tests when
testing our packages.

Allow no-init systems to use our default skeleton, but only use the
common part, for the specific sysv or systemd ones are unfit.

Reported-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>

---
Changes v3 -> v4:
  - select BR2_PACKAGE_HAS_SKELETON from BR2_PACKAGE_SKELETON_COMMON_ONLY,
    drop now-unneeded comment  (Arnout)
---
 package/skeleton-common/Config.in | 15 +++++++++++++++
 system/Config.in                  |  5 +----
 2 files changed, 16 insertions(+), 4 deletions(-)

diff --git a/package/skeleton-common/Config.in b/package/skeleton-common/Config.in
index 9abed4292a..fc9d818e37 100644
--- a/package/skeleton-common/Config.in
+++ b/package/skeleton-common/Config.in
@@ -1,2 +1,17 @@
 config BR2_PACKAGE_SKELETON_COMMON
 	bool
+
+if BR2_PACKAGE_SKELETON_COMMON
+
+config BR2_PACKAGE_SKELETON_COMMON_ONLY
+	bool
+	default y
+	depends on !BR2_PACKAGE_SKELETON_SYSV
+	depends on !BR2_PACKAGE_SKELETON_SYSTEMD
+	depends on !BR2_PACKAGE_SKELETON_CUSTOM
+	select BR2_PACKAGE_HAS_SKELETON
+
+config BR2_PACKAGE_PROVIDES_SKELETON
+	default "skeleton-common" if BR2_PACKAGE_SKELETON_COMMON_ONLY
+
+endif
diff --git a/system/Config.in b/system/Config.in
index 65139c0bfd..4d417a93b3 100644
--- a/system/Config.in
+++ b/system/Config.in
@@ -11,16 +11,13 @@ choice
 # is most probably unfit; so, allow only the custom skeleton in that case.
 config BR2_ROOTFS_SKELETON_DEFAULT
 	bool "default target skeleton"
-	depends on !BR2_INIT_NONE
 	select BR2_PACKAGE_SKELETON_SYSV if BR2_INIT_SYSV
 	select BR2_PACKAGE_SKELETON_SYSV if BR2_INIT_BUSYBOX
 	select BR2_PACKAGE_SKELETON_SYSTEMD if BR2_INIT_SYSTEMD
+	select BR2_PACKAGE_SKELETON_COMMON if BR2_INIT_NONE
 	help
 	  Use default target skeleton
 
-comment "default target skeleton needs an init system"
-	depends on BR2_INIT_NONE
-
 config BR2_ROOTFS_SKELETON_CUSTOM
 	bool "custom target skeleton"
 	select BR2_PACKAGE_SKELETON_CUSTOM
-- 
2.11.0

[Buildroot] [PATCH 11/14 v4] support/testing: add runtime testing for init systems

[Yann E. MORIN]

The "builtin" kernel does not boot a systemd-based system, so
we resort to building the same one as currently used by our
qemu_arm_vexpress_defconfig.

We test the 8 following combinations:

  - busybox, read-only, without network
  - busybox, read-only, with network
  - busybox, read-write, without network
  - busybox, read-write, with network

  - basic systemd, read-write, network w/ ifupdown
  - basic systemd, read-write, network w/ networkd
  - full systemd, read-write, network w/ networkd

  - no init system, read-only, without network

The tests just verify what the /sbin/init binary is, and that we were
able to grab an IP address. More tests can be added later, for example
to check each systemd features (journal, tmpfiles...)

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>

---
Changes v3 -> v4:
  - drop the systemd read-only tests for now, we do not support it yet.
---
 support/testing/tests/init/__init__.py     |  0
 support/testing/tests/init/base.py         | 47 +++++++++++++++++
 support/testing/tests/init/test_busybox.py | 67 ++++++++++++++++++++++++
 support/testing/tests/init/test_none.py    | 32 ++++++++++++
 support/testing/tests/init/test_systemd.py | 82 ++++++++++++++++++++++++++++++
 5 files changed, 228 insertions(+)
 create mode 100644 support/testing/tests/init/__init__.py
 create mode 100644 support/testing/tests/init/base.py
 create mode 100644 support/testing/tests/init/test_busybox.py
 create mode 100644 support/testing/tests/init/test_none.py
 create mode 100644 support/testing/tests/init/test_systemd.py

diff --git a/support/testing/tests/init/__init__.py b/support/testing/tests/init/__init__.py
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/support/testing/tests/init/base.py b/support/testing/tests/init/base.py
new file mode 100644
index 0000000000..a261d7dd46
--- /dev/null
+++ b/support/testing/tests/init/base.py
@@ -0,0 +1,47 @@
+import os
+import subprocess
+import infra.basetest
+
+class InitSystemBase(infra.basetest.BRTest):
+
+    def startEmulator(self, fs_type, kernel=None, dtb=None, init=None):
+        img = os.path.join(self.builddir, "images", "rootfs.{}".format(fs_type))
+        subprocess.call(["truncate", "-s", "%1M", img])
+
+        options = ["-drive",
+                   "file={},if=sd,format=raw".format(img),
+                   "-M", "vexpress-a9"]
+
+        if kernel is None:
+            kernel = "builtin"
+        else:
+            kernel = os.path.join(self.builddir, "images", kernel)
+            options.extend(["-dtb", os.path.join(self.builddir, "images",
+                                             "{}.dtb".format(dtb))])
+
+        kernel_cmdline = ["root=/dev/mmcblk0",
+                          "rootfstype={}".format(fs_type),
+                          "rootwait",
+                          "ro",
+                          "console=ttyAMA0"]
+
+        if not init is None:
+            kernel_cmdline.extend(["init={}".format(init)])
+
+        self.emulator.boot(arch="armv7",
+                           kernel=kernel,
+                           kernel_cmdline=kernel_cmdline,
+                           options=options)
+
+        if init is None:
+            self.emulator.login()
+
+    def checkInit(self, path):
+        cmd = "cmp /proc/1/exe {}".format(path)
+        _, exit_code = self.emulator.run(cmd)
+        self.assertEqual(exit_code, 0)
+
+    def checkNetwork(self, interface, exitCode=0):
+        cmd = "ip addr show {} |grep inet".format(interface)
+        _, exit_code = self.emulator.run(cmd)
+        self.assertEqual(exit_code, exitCode)
diff --git a/support/testing/tests/init/test_busybox.py b/support/testing/tests/init/test_busybox.py
new file mode 100644
index 0000000000..c3e425bf5d
--- /dev/null
+++ b/support/testing/tests/init/test_busybox.py
@@ -0,0 +1,67 @@
+import infra.basetest
+from tests.init.base import InitSystemBase as InitSystemBase
+
+class InitSystemBusyboxBase(InitSystemBase):
+    config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
+        """
+        # BR2_TARGET_ROOTFS_TAR is not set
+        """
+
+    def checkInit(self):
+        super(InitSystemBusyboxBase, self).checkInit("/bin/busybox")
+
+
+#-------------------------------------------------------------------------------
+class TestInitSystemBusyboxRo(InitSystemBusyboxBase):
+    config = InitSystemBusyboxBase.config + \
+        """
+        # BR2_TARGET_GENERIC_REMOUNT_ROOTFS_RW is not set
+        BR2_TARGET_ROOTFS_SQUASHFS=y
+        """
+
+    def test_run(self):
+        self.startEmulator("squashfs")
+        self.checkInit()
+        self.checkNetwork("eth0", 1)
+
+
+#-------------------------------------------------------------------------------
+class TestInitSystemBusyboxRw(InitSystemBusyboxBase):
+    config = InitSystemBusyboxBase.config + \
+        """
+        BR2_TARGET_ROOTFS_EXT2=y
+        """
+
+    def test_run(self):
+        self.startEmulator("ext2")
+        self.checkInit()
+        self.checkNetwork("eth0", 1)
+
+
+#-------------------------------------------------------------------------------
+class TestInitSystemBusyboxRoNet(InitSystemBusyboxBase):
+    config = InitSystemBusyboxBase.config + \
+        """
+        BR2_SYSTEM_DHCP="eth0"
+        # BR2_TARGET_GENERIC_REMOUNT_ROOTFS_RW is not set
+        BR2_TARGET_ROOTFS_SQUASHFS=y
+        """
+
+    def test_run(self):
+        self.startEmulator("squashfs")
+        self.checkInit()
+        self.checkNetwork("eth0")
+
+
+#-------------------------------------------------------------------------------
+class TestInitSystemBusyboxRwNet(InitSystemBusyboxBase):
+    config = InitSystemBusyboxBase.config + \
+        """
+        BR2_SYSTEM_DHCP="eth0"
+        BR2_TARGET_ROOTFS_EXT2=y
+        """
+
+    def test_run(self):
+        self.startEmulator("ext2")
+        self.checkInit()
+        self.checkNetwork("eth0")
diff --git a/support/testing/tests/init/test_none.py b/support/testing/tests/init/test_none.py
new file mode 100644
index 0000000000..f55db5d3e0
--- /dev/null
+++ b/support/testing/tests/init/test_none.py
@@ -0,0 +1,32 @@
+import pexpect
+
+import infra.basetest
+from tests.init.base import InitSystemBase as InitSystemBase
+
+class TestInitSystemNone(InitSystemBase):
+    config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
+        """
+        BR2_INIT_NONE=y
+        # BR2_TARGET_ROOTFS_TAR is not set
+        BR2_TARGET_ROOTFS_SQUASHFS=y
+        """
+
+    def test_run(self):
+        self.startEmulator(fs_type="squashfs", init="/bin/sh")
+        index = self.emulator.qemu.expect(["/bin/sh: can't access tty; job control turned off", pexpect.TIMEOUT], timeout=60)
+        if index != 0:
+            self.emulator.logfile.write("==> System does not boot")
+            raise SystemError("System does not boot")
+        index = self.emulator.qemu.expect(["#", pexpect.TIMEOUT], timeout=60)
+        if index != 0:
+            self.emulator.logfile.write("==> System does not boot")
+            raise SystemError("System does not boot")
+
+        out, exit_code = self.emulator.run("sh -c 'echo $PPID'")
+        self.assertEqual(exit_code, 0)
+        self.assertEqual(out[0], "1")
+
+        _, exit_code = self.emulator.run("mount -t proc none /proc")
+        self.assertEqual(exit_code, 0)
+
+        self.checkInit("/bin/sh")
diff --git a/support/testing/tests/init/test_systemd.py b/support/testing/tests/init/test_systemd.py
new file mode 100644
index 0000000000..7a80aa1145
--- /dev/null
+++ b/support/testing/tests/init/test_systemd.py
@@ -0,0 +1,82 @@
+import infra.basetest
+from tests.init.base import InitSystemBase as InitSystemBase
+
+class InitSystemSystemdBase(InitSystemBase):
+    config = \
+        """
+        BR2_arm=y
+        BR2_TOOLCHAIN_EXTERNAL=y
+        BR2_INIT_SYSTEMD=y
+        BR2_TARGET_GENERIC_GETTY_PORT="ttyAMA0"
+        BR2_LINUX_KERNEL=y
+        BR2_LINUX_KERNEL_CUSTOM_VERSION=y
+        BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE="4.11.3"
+        BR2_LINUX_KERNEL_DEFCONFIG="vexpress"
+        BR2_LINUX_KERNEL_DTS_SUPPORT=y
+        BR2_LINUX_KERNEL_INTREE_DTS_NAME="vexpress-v2p-ca9"
+        # BR2_TARGET_ROOTFS_TAR is not set
+        """
+
+    def checkInit(self):
+        super(InitSystemSystemdBase, self).checkInit("/lib/systemd/systemd")
+
+
+#-------------------------------------------------------------------------------
+class TestInitSystemSystemdRwNetworkd(InitSystemSystemdBase):
+    config = InitSystemSystemdBase.config + \
+        """
+        BR2_SYSTEM_DHCP="eth0"
+        BR2_TARGET_ROOTFS_EXT2=y
+        """
+
+    def test_run(self):
+        self.startEmulator("ext2", "zImage", "vexpress-v2p-ca9")
+        self.checkInit()
+        self.checkNetwork("eth0")
+
+
+#-------------------------------------------------------------------------------
+class TestInitSystemSystemdRwIfupdown(InitSystemSystemdBase):
+    config = InitSystemSystemdBase.config + \
+        """
+        BR2_SYSTEM_DHCP="eth0"
+        # BR2_PACKAGE_SYSTEMD_NETWORKD is not set
+        # BR2_TARGET_GENERIC_REMOUNT_ROOTFS_RW is not set
+        BR2_TARGET_ROOTFS_EXT2=y
+        """
+
+    def test_run(self):
+        self.startEmulator("ext2", "zImage", "vexpress-v2p-ca9")
+        self.checkInit()
+        self.checkNetwork("eth0")
+
+
+#-------------------------------------------------------------------------------
+class TestInitSystemSystemdRwFull(InitSystemSystemdBase):
+    config = InitSystemSystemdBase.config + \
+        """
+        BR2_SYSTEM_DHCP="eth0"
+        BR2_PACKAGE_SYSTEMD_JOURNAL_GATEWAY=y
+        BR2_PACKAGE_SYSTEMD_BACKLIGHT=y
+        BR2_PACKAGE_SYSTEMD_BINFMT=y
+        BR2_PACKAGE_SYSTEMD_COREDUMP=y
+        BR2_PACKAGE_SYSTEMD_FIRSTBOOT=y
+        BR2_PACKAGE_SYSTEMD_HIBERNATE=y
+        BR2_PACKAGE_SYSTEMD_IMPORTD=y
+        BR2_PACKAGE_SYSTEMD_LOCALED=y
+        BR2_PACKAGE_SYSTEMD_LOGIND=y
+        BR2_PACKAGE_SYSTEMD_MACHINED=y
+        BR2_PACKAGE_SYSTEMD_POLKIT=y
+        BR2_PACKAGE_SYSTEMD_QUOTACHECK=y
+        BR2_PACKAGE_SYSTEMD_RANDOMSEED=y
+        BR2_PACKAGE_SYSTEMD_RFKILL=y
+        BR2_PACKAGE_SYSTEMD_SMACK_SUPPORT=y
+        BR2_PACKAGE_SYSTEMD_SYSUSERS=y
+        BR2_PACKAGE_SYSTEMD_VCONSOLE=y
+        BR2_TARGET_ROOTFS_EXT2=y
+        """
+
+    def test_run(self):
+        self.startEmulator("ext2", "zImage", "vexpress-v2p-ca9")
+        self.checkInit()
+        self.checkNetwork("eth0")
-- 
2.11.0

[Buildroot] [PATCH 12/14 v4] fs: add pre- and post-command hooks

[Yann E. MORIN]

In some cases, the directory structure we want in the filesystem is not
exactly what we have in target/

For example, when systemd is used on a read-only rootfs, /var must be a
tmpfs. However, we may have packages that install stuff in there, and
set important rights (via the permission-table). So, at build time, we
need /var to be a symlink to the remanent location (/usr/share/factory)
while at runtime we need /var to be a directory.

One option would have seen to have /var as a real directory even during
build time, and in a target-finalize hook, move everything out of there
and into the "factory" location. However, that's not possible because
it's too early: some packages may want to set ownership and/or acces
rights on directories or files in /var, and this is only done in the
fakeroot script, which is called only later during the assembling of the
filesystem images.

Also, there would have been no way to undo the tweak (i.e. we need to
restore the /var symlink so that subsequent builds continue to work) if
it were done as a target-finalize hook.

The only solution is to allow packages to register pre- and post-hooks
that are called right before and right after the rootfs commands are
executed, and inside in the fakeroot script.

We can however not re-use the BR2_ROOTFS_POST_FAKEROOT_SCRIPT feature
either because it is done before the filesystem command, but there is
nothing that is done after. Also, we don't want to add to, and modify a
user-supplied variable.

So, we introduce two new variables that packages can set to add the
commands they need to run to tweak the filesystem right at the last
moment.

Those hooks are not documented on-purpose; they are probably going to
only ever be used by systemd.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Romain Naour <romain.naour@gmail.com>

---
Changes v2 -> v3:
  - add space between >> and $$   (Romain)
---
 fs/common.mk           | 4 ++++
 package/pkg-generic.mk | 4 ++++
 2 files changed, 8 insertions(+)

diff --git a/fs/common.mk b/fs/common.mk
index 14e0a6b868..9a7758ff49 100644
--- a/fs/common.mk
+++ b/fs/common.mk
@@ -95,10 +95,14 @@ endif
 	$$(foreach s,$$(call qstrip,$$(BR2_ROOTFS_POST_FAKEROOT_SCRIPT)),\
 		echo "echo '$$(TERM_BOLD)>>>   Executing fakeroot script $$(s)$$(TERM_RESET)'" >> $$(FAKEROOT_SCRIPT); \
 		echo $$(s) $$(TARGET_DIR) $$(BR2_ROOTFS_POST_SCRIPT_ARGS) >> $$(FAKEROOT_SCRIPT)$$(sep))
+	$$(foreach hook,$$(ROOTFS_PRE_CMD_HOOKS),\
+		$$(call PRINTF,$$($$(hook))) >> $$(FAKEROOT_SCRIPT))
 ifeq ($$(BR2_REPRODUCIBLE),y)
 	echo "find $$(TARGET_DIR) -print0 | xargs -0 -r touch -hd @$$(SOURCE_DATE_EPOCH)" >> $$(FAKEROOT_SCRIPT)
 endif
 	$$(call PRINTF,$$(ROOTFS_$(2)_CMD)) >> $$(FAKEROOT_SCRIPT)
+	$$(foreach hook,$$(ROOTFS_POST_CMD_HOOKS),\
+		$$(call PRINTF,$$($$(hook))) >> $$(FAKEROOT_SCRIPT))
 	chmod a+x $$(FAKEROOT_SCRIPT)
 	PATH=$$(BR_PATH) $$(HOST_DIR)/bin/fakeroot -- $$(FAKEROOT_SCRIPT)
 	$$(INSTALL) -m 0644 support/misc/target-dir-warning.txt $$(TARGET_DIR_WARNING_FILE)
diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
index 130fd67560..90bd8517f9 100644
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -612,6 +612,8 @@ $(2)_POST_INSTALL_IMAGES_HOOKS  ?=
 $(2)_PRE_LEGAL_INFO_HOOKS       ?=
 $(2)_POST_LEGAL_INFO_HOOKS      ?=
 $(2)_TARGET_FINALIZE_HOOKS      ?=
+$(2)_ROOTFS_PRE_CMD_HOOKS       ?=
+$(2)_ROOTFS_POST_CMD_HOOKS      ?=
 
 # human-friendly targets and target sequencing
 $(1):			$(1)-install
@@ -931,6 +933,8 @@ ifneq ($$($(2)_USERS),)
 PACKAGES_USERS += $$($(2)_USERS)$$(sep)
 endif
 TARGET_FINALIZE_HOOKS += $$($(2)_TARGET_FINALIZE_HOOKS)
+ROOTFS_PRE_CMD_HOOKS += $$($(2)_ROOTFS_PRE_CMD_HOOKS)
+ROOTFS_POST_CMD_HOOKS += $$($(2)_ROOTFS_POST_CMD_HOOKS)
 
 ifeq ($$($(2)_SITE_METHOD),svn)
 DL_TOOLS_DEPENDENCIES += svn
-- 
2.11.0

[Buildroot] [PATCH 13/14 v4] system: make systemd work on a read-only rootfs

[Yann E. MORIN]

When the rootfs is readonly, systemd will expect /var to be writable.
Because we do not really have a R/W filesystem to mount on /var, we make
it a tmpfs [*], and use the systemd-tmpfiles feature to populate it with
"factory" defaults.

We obtain those factory defaults by redirecting /var to that location at
build time, using a symlink /var -> /usr/share/factory which is the
location in which systemd-tmpfiles will look for when instructed to
"recursively copy" a directory.

With a line like:

    C /var/something - - - -

it will look for /usr/share/factory/something and copy it (recursively
if it is a directory) to /var/something, but only if it does not already
exist there.

We also mark this copy with the exclamation mark, as it is only safe to
copy on boot, not when changing targets.

To be noted: the real format for such lines are:

    C /var/something - - - - /from/where/to/copy/something

But if the source is not given, then it is implicitly taken from
/usr/share/factory (which in our case is as-good a location as whatever
else, so we use it, and thus we need not specify the source of the
copy).

Note that we treat symlinks a little bit specially, by creating symlinks
to the factory defaults rather than copying them.

Finally, /var at build time is a symlink, but at runtime, it must be a
directory (so we can mount the tmpfs over there). We can't change that
as a target-finalize hook, because:

  - some packages may want to set ownership and/or access rights on
    files or directories in /var, and that only happens while assembling
    the filesystem images; changing /var from a symlink to a (then
    empty) directory would break this;

  - /var would be a directory on sub-sequent builds (until the next
    "make clean").

Instead, we use the newly-introduce pre- and post-rootfs command hooks,
to turn /var into a directory before assembling the image, and back to a
symlink after assembling the image.

[*] People who want the factory-defaults only on first boot will have
    to tweak the fstab to mount something else than a tmpfs on /var.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Romain Naour <romain.naour@gmail.com>

---
Changes v2 -> v3;
  - fix typoes in commit log  (Romain)
---
 package/skeleton-systemd/skeleton-systemd.mk | 50 ++++++++++++++++++++++++++--
 system/Config.in                             |  1 -
 2 files changed, 48 insertions(+), 3 deletions(-)

diff --git a/package/skeleton-systemd/skeleton-systemd.mk b/package/skeleton-systemd/skeleton-systemd.mk
index 384715e1c9..860ebea2cd 100644
--- a/package/skeleton-systemd/skeleton-systemd.mk
+++ b/package/skeleton-systemd/skeleton-systemd.mk
@@ -15,11 +15,57 @@ SKELETON_SYSTEMD_DEPENDENCIES = skeleton-common
 
 SKELETON_SYSTEMD_PROVIDES = skeleton
 
+ifeq ($(BR2_TARGET_GENERIC_REMOUNT_ROOTFS_RW),y)
+
+define SKELETON_SYSTEMD_ROOT_RO_OR_RW
+	echo "/dev/root / auto rw 0 1" >$(TARGET_DIR)/etc/fstab
+	mkdir -p $(TARGET_DIR)/var
+endef
+
+else
+
+# On a R/O rootfs, /var is a tmpfs filesystem. So, at build time, we
+# redirect /var to the "factory settings" location. Just before the
+# filesystem gets created, the /var symlink will be replaced with
+# a real (but empty) directory, and the "factory files" will be copied
+# back there by the tmpfiles.d mechanism.
+define SKELETON_SYSTEMD_ROOT_RO_OR_RW
+	mkdir -p $(TARGET_DIR)/etc/systemd/tmpfiles.d
+	mkdir -p $(TARGET_DIR)/usr/share/factory/var
+	ln -s usr/share/factory/var $(TARGET_DIR)/var
+	echo "/dev/root / auto ro 0 1" >$(TARGET_DIR)/etc/fstab
+	echo "tmpfs /var tmpfs mode=1777 0 0" >>$(TARGET_DIR)/etc/fstab
+endef
+
+define SKELETON_SYSTEMD_PRE_ROOTFS_VAR
+	rm -f $(TARGET_DIR)/var
+	mkdir $(TARGET_DIR)/var
+	for i in $(TARGET_DIR)/usr/share/factory/var/*; do \
+		j="$${i#$(TARGET_DIR)/usr/share/factory}"; \
+		if [ -L "$${i}" ]; then \
+			printf "L+! %s - - - - %s\n" \
+				"$${j}" "../usr/share/factory/$${j}" \
+			|| exit 1; \
+		else \
+			printf "C! %s - - - -\n" "$${j}" \
+			|| exit 1; \
+		fi; \
+	done >$(TARGET_DIR)/etc/tmpfiles.d/var-factory.conf
+endef
+SKELETON_SYSTEMD_ROOTFS_PRE_CMD_HOOKS += SKELETON_SYSTEMD_PRE_ROOTFS_VAR
+
+define SKELETON_SYSTEMD_POST_ROOTFS_VAR
+	rm -rf $(TARGET_DIR)/var
+	ln -s usr/share/factory/var $(TARGET_DIR)/var
+endef
+SKELETON_SYSTEMD_ROOTFS_POST_CMD_HOOKS += SKELETON_SYSTEMD_POST_ROOTFS_VAR
+
+endif
+
 define SKELETON_SYSTEMD_INSTALL_TARGET_CMDS
 	mkdir -p $(TARGET_DIR)/home
 	mkdir -p $(TARGET_DIR)/srv
-	mkdir -p $(TARGET_DIR)/var
-	echo "/dev/root / auto rw 0 1" >$(TARGET_DIR)/etc/fstab
+	$(SKELETON_SYSTEMD_ROOT_RO_OR_RW)
 endef
 
 $(eval $(generic-package))
diff --git a/system/Config.in b/system/Config.in
index 4d417a93b3..0235897735 100644
--- a/system/Config.in
+++ b/system/Config.in
@@ -138,7 +138,6 @@ config BR2_INIT_SYSTEMD
 	depends on BR2_TOOLCHAIN_HEADERS_AT_LEAST_3_10
 	select BR2_ROOTFS_MERGED_USR
 	select BR2_PACKAGE_SYSTEMD
-	select BR2_TARGET_GENERIC_REMOUNT_ROOTFS_RW if BR2_ROOTFS_SKELETON_DEFAULT
 
 comment "systemd needs a glibc toolchain, headers >= 3.10"
 	depends on !(BR2_TOOLCHAIN_USES_GLIBC \
-- 
2.11.0

[Buildroot] [PATCH 14/14 v4] support/testing: add runtime testing for read-only systemd

[Yann E. MORIN]

We add the 3 following combinations:

  - basic systemd, read-only, network w/ ifupdown
  - basic systemd, read-only, network w/ networkd
  - full systemd, read-only, network w/ networkd

The tests just verify what the /sbin/init binary is, and that we were
able to grab an IP address. More tests can be added later, for example
to check each systemd features (journal, tmpfiles...)

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 .../testing/tests/init/systemd-factory/var/foo/bar |  1 +
 support/testing/tests/init/test_systemd.py         | 70 ++++++++++++++++++++++
 2 files changed, 71 insertions(+)
 create mode 100644 support/testing/tests/init/systemd-factory/var/foo/bar

diff --git a/support/testing/tests/init/systemd-factory/var/foo/bar b/support/testing/tests/init/systemd-factory/var/foo/bar
new file mode 100644
index 0000000000..323fae03f4
--- /dev/null
+++ b/support/testing/tests/init/systemd-factory/var/foo/bar
@@ -0,0 +1 @@
+foobar
diff --git a/support/testing/tests/init/test_systemd.py b/support/testing/tests/init/test_systemd.py
index 7a80aa1145..b1b6517373 100644
--- a/support/testing/tests/init/test_systemd.py
+++ b/support/testing/tests/init/test_systemd.py
@@ -22,6 +22,28 @@ class InitSystemSystemdBase(InitSystemBase):
 
 
 #-------------------------------------------------------------------------------
+class TestInitSystemSystemdRoNetworkd(InitSystemSystemdBase):
+    config = InitSystemSystemdBase.config + \
+        """
+        BR2_SYSTEM_DHCP="eth0"
+        # BR2_TARGET_GENERIC_REMOUNT_ROOTFS_RW is not set
+        BR2_ROOTFS_OVERLAY="{}"
+        BR2_TARGET_ROOTFS_SQUASHFS=y
+        """.format(infra.filepath("tests/init/systemd-factory"))
+
+    def test_run(self):
+        self.startEmulator("squashfs", "zImage", "vexpress-v2p-ca9")
+        self.checkInit()
+        self.checkNetwork("eth0")
+
+        # This one must be executed on the target, to check that
+        # the factory feature works as expected
+        out, exit_code = self.emulator.run("cat /var/foo/bar")
+        self.assertEqual(exit_code, 0)
+        self.assertEqual(out[0], "foobar")
+
+
+#-------------------------------------------------------------------------------
 class TestInitSystemSystemdRwNetworkd(InitSystemSystemdBase):
     config = InitSystemSystemdBase.config + \
         """
@@ -36,6 +58,22 @@ class TestInitSystemSystemdRwNetworkd(InitSystemSystemdBase):
 
 
 #-------------------------------------------------------------------------------
+class TestInitSystemSystemdRoIfupdown(InitSystemSystemdBase):
+    config = InitSystemSystemdBase.config + \
+        """
+        BR2_SYSTEM_DHCP="eth0"
+        # BR2_PACKAGE_SYSTEMD_NETWORKD is not set
+        # BR2_TARGET_GENERIC_REMOUNT_ROOTFS_RW is not set
+        BR2_TARGET_ROOTFS_SQUASHFS=y
+        """
+
+    def test_run(self):
+        self.startEmulator("squashfs", "zImage", "vexpress-v2p-ca9")
+        self.checkInit()
+        self.checkNetwork("eth0")
+
+
+#-------------------------------------------------------------------------------
 class TestInitSystemSystemdRwIfupdown(InitSystemSystemdBase):
     config = InitSystemSystemdBase.config + \
         """
@@ -52,6 +90,38 @@ class TestInitSystemSystemdRwIfupdown(InitSystemSystemdBase):
 
 
 #-------------------------------------------------------------------------------
+class TestInitSystemSystemdRoFull(InitSystemSystemdBase):
+    config = InitSystemSystemdBase.config + \
+        """
+        BR2_SYSTEM_DHCP="eth0"
+        # BR2_TARGET_GENERIC_REMOUNT_ROOTFS_RW is not set
+        BR2_PACKAGE_SYSTEMD_JOURNAL_GATEWAY=y
+        BR2_PACKAGE_SYSTEMD_BACKLIGHT=y
+        BR2_PACKAGE_SYSTEMD_BINFMT=y
+        BR2_PACKAGE_SYSTEMD_COREDUMP=y
+        BR2_PACKAGE_SYSTEMD_FIRSTBOOT=y
+        BR2_PACKAGE_SYSTEMD_HIBERNATE=y
+        BR2_PACKAGE_SYSTEMD_IMPORTD=y
+        BR2_PACKAGE_SYSTEMD_LOCALED=y
+        BR2_PACKAGE_SYSTEMD_LOGIND=y
+        BR2_PACKAGE_SYSTEMD_MACHINED=y
+        BR2_PACKAGE_SYSTEMD_POLKIT=y
+        BR2_PACKAGE_SYSTEMD_QUOTACHECK=y
+        BR2_PACKAGE_SYSTEMD_RANDOMSEED=y
+        BR2_PACKAGE_SYSTEMD_RFKILL=y
+        BR2_PACKAGE_SYSTEMD_SMACK_SUPPORT=y
+        BR2_PACKAGE_SYSTEMD_SYSUSERS=y
+        BR2_PACKAGE_SYSTEMD_VCONSOLE=y
+        BR2_TARGET_ROOTFS_SQUASHFS=y
+        """
+
+    def test_run(self):
+        self.startEmulator("squashfs", "zImage", "vexpress-v2p-ca9")
+        self.checkInit()
+        self.checkNetwork("eth0")
+
+
+#-------------------------------------------------------------------------------
 class TestInitSystemSystemdRwFull(InitSystemSystemdBase):
     config = InitSystemSystemdBase.config + \
         """
-- 
2.11.0

[Buildroot] [PATCH 04/14 v4] package/skeleton: split out into skeleton-custom

[Arnout Vandecappelle]

 Hi Yann,

On 25-07-17 23:14, Yann E. MORIN wrote:
> For the custom skeleton, we practicaly do nothing, except ensure it
                              do practically

> contains the basic, required directories, and that those are properly
> setup wrt. merged /usr.
[snip]

 I don't like this: you are doing *a whole lot* more than just moving stuff from
skeleton to skeleton-custom (and renaming the variables):

> +SKELETON_CUSTOM_LIB_INODE = $(shell stat -c '%i' $(SKELETON_CUSTOM_PATH)/lib/. 2>/dev/null)
> -SKELETON_LIB_INODE = $(shell stat -c '%i' $(SKELETON_PATH)/lib/.)

> +# Ensure that the custom skeleton has /lib, /bin and /sbin, and their
> +# /usr counterparts
> +ifeq ($(SKELETON_CUSTOM_LIB_INODE),)
> +SKELETON_CUSTOM_MISSING_DIRS += /lib
> +endif

> +# For a merged /usr, ensure that /lib, /bin and /sbin and their /usr
> +# counterparts are appropriately setup as symlinks ones to the others.

> +ifneq ($(SKELETON_CUSTOM_LIB_INODE),$(SKELETON_CUSTOM_USR_LIB_INODE))
> +SKELETON_CUSTOM_NOT_MERGED_USR_DIRS += /lib
> +endif
> -ifneq ($(SKELETON_LIB_INODE),$(SKELETON_USR_LIB_INODE))
> -SKELETON_CUSTOM_NOT_MERGED_USR += /lib
> -endif

> +ifeq ($(BR2_PACKAGE_SKELETON_CUSTOM)$(BR_BUILDING),yy)
> +ifneq ($(SKELETON_CUSTOM_MISSING_DIRS),)
> +$(error The custom skeleton in $(SKELETON_CUSTOM_PATH) is \
> +	missing those directories or symlinks: \
> +	$(SKELETON_CUSTOM_MISSING_DIRS))
> +endif

 Now, I understand that it's tricky to rebase things correctly, and splitting
this up is anyway mainly relevant for review, so I've just updated the commit
message to reflect all these changes.

> +$(error The custom skeleton in $(SKELETON_PATH) is not \
> +       using a merged /usr for the following directories: \
> +       $(SKELETON_CUSTOM_NOT_MERGED_USR_DIRS))

 Spaces should be tab.

> +# Provided by the 'skeleton' package:
> +# - SKELETON_RSYNC
> +# - SKELETON_LIB_SYMLINK

 As mentioned in an earlier review, this is in system now and it's quite clear
that SYSTEM_RSYNC comes from system.mk, so I've removed this.


 I've fixed all that and was going to apply, but then I noticed that patch 1
breaks our tests, so I refrained from pushing it. You can fetch it from
https://gitlab.com/arnout/buildroot (master branch; will be rebased away in a
couple of days).

 Regards,
 Arnout

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] [PATCH 01/14 v4] system: with no init system, only allow custom skeleton

[Arnout Vandecappelle]

On 25-07-17 23:14, Yann E. MORIN wrote:
> When there is no init system (i.e. a custom one), our bundled default
> skeleton is most probably un-fit for that (non-)init system.
> 
> This will be the case when we introduce per init system skeletons. The
> systemd skeleton will be unfit except for a systemd init; the sysv
> skeleton will be unfit except for a sysv-like init system.
> 
> In case they are using no init system (really, a custom one), the user
> should be responsible for providing their own, custom skeleton as well.
> 
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>

 There is actually a problem with this one. A lot of our tests in
support/testing, as well as the test-pkg script and the autobuilders, will build
with BR2_INIT_NONE=y but without specifying a custom skeleton. That makes this
series essentially non-bisectable.

 Thomas, Peter, how bad a problem do you think this is?

 Yann, how bad would it be to rebase the series without this patch? At first
sight, up to patch 7 should be independent of it...

 Regards,
 Arnout

> ---
>  system/Config.in | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/system/Config.in b/system/Config.in
> index 828df4217e..ed539dcbe0 100644
> --- a/system/Config.in
> +++ b/system/Config.in
> @@ -3,11 +3,17 @@ menu "System configuration"
>  choice
>  	prompt "Root FS skeleton"
>  
> +# When not using an init system (i.e. a custom one), our default skeleton
> +# is most probably unfit; so, allow only the custom skeleton in that case.
>  config BR2_ROOTFS_SKELETON_DEFAULT
>  	bool "default target skeleton"
> +	depends on !BR2_INIT_NONE
>  	help
>  	  Use default target skeleton
>  
> +comment "default target skeleton needs an init system"
> +	depends on BR2_INIT_NONE
> +
>  config BR2_ROOTFS_SKELETON_CUSTOM
>  	bool "custom target skeleton"
>  	help
> 

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] [PATCH 04/14 v4] package/skeleton: split out into skeleton-custom

[Arnout Vandecappelle]

On 26-07-17 02:16, Arnout Vandecappelle wrote:
>  I've fixed all that and was going to apply, but then I noticed that patch 1
> breaks our tests, so I refrained from pushing it. You can fetch it from
> https://gitlab.com/arnout/buildroot (master branch; will be rebased away in a
> couple of days).

 So I pushed this commit with the runtime testing patch 11 on top of to gitlab
to see what would happen, and the systemd test fails. It seems the getty on
ttyAMA0 can't be started. Is that to be expected, due to the /var/log->/tmp symlink?

 You can inspect the output on [1], but it's not saying much.

 Regards,
 Arnout

[1]
https://gitlab.com/arnout/buildroot/-/jobs/24489865/artifacts/raw/test-output/TestInitSystemSystemdRwFull-run.log

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] [PATCH 04/14 v4] package/skeleton: split out into skeleton-custom

[Ricardo Martincoski]

Hello,

On Wed, Jul 26, 2017 at 08:49 AM, Arnout Vandecappelle wrote:

> On 26-07-17 02:16, Arnout Vandecappelle wrote:
>>  I've fixed all that and was going to apply, but then I noticed that patch 1
>> breaks our tests, so I refrained from pushing it. You can fetch it from
>> https://gitlab.com/arnout/buildroot (master branch; will be rebased away in a
>> couple of days).
> 
>  So I pushed this commit with the runtime testing patch 11 on top of to gitlab
> to see what would happen, and the systemd test fails. It seems the getty on
> ttyAMA0 can't be started. Is that to be expected, due to the /var/log->/tmp symlink?
> 
>  You can inspect the output on [1], but it's not saying much.

It seems a timeout while waiting for the login prompt.

> [1]
> https://gitlab.com/arnout/buildroot/-/jobs/24489865/artifacts/raw/test-output/TestInitSystemSystemdRwFull-run.log

This test you triggered ran on the runner e11ae361.

I borrowed your commit (e8e38e9e) and did some tests.

I got lucky and the test I triggered ran on the shared runner 4e4528ca and
passed [2].
Then I retriggered it to run on my private runner, while I was also running
a resource intensive task (actually I was running the same test in the same
commit locally with the default -j 0) and the test in the runner failed [3]
(while the local test passed).

[2] https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/25090126
[3] https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/25101360


TestIPythonPy3 for instance seems to be have tidy timeout for math_floor_test.
Its first execution [4] ran in the (free, as in free beer!) elastic runner from
gitlab and failed. The second execution [5] ran in my private runner (a docker
in my personal computer) and passed.

[4] https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/25090131
[5] https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/25099593


Concerning the test infra (I say that including gitlab) we should choose a path
to follow:
1) Use private runners
2) Adapt timeouts when a test sporadically fails and keep using the gitlab
runners ([6] shows that we already use them)

[6] https://gitlab.com/buildroot.org/buildroot/-/jobs/24184768

I would go with option 2. We are not doing CI for now, so sporadic failures are
not a big concern IMO.

Regards,
Ricardo

[Buildroot] [PATCH 04/14 v4] package/skeleton: split out into skeleton-custom

[Thomas Petazzoni]

Hello,

On Fri, 28 Jul 2017 00:02:34 -0300, Ricardo Martincoski wrote:

> This test you triggered ran on the runner e11ae361.
> 
> I borrowed your commit (e8e38e9e) and did some tests.
> 
> I got lucky and the test I triggered ran on the shared runner 4e4528ca and
> passed [2].
> Then I retriggered it to run on my private runner, while I was also running
> a resource intensive task (actually I was running the same test in the same
> commit locally with the default -j 0) and the test in the runner failed [3]
> (while the local test passed).
> 
> [2] https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/25090126
> [3] https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/25101360
> 
> 
> TestIPythonPy3 for instance seems to be have tidy timeout for math_floor_test.
> Its first execution [4] ran in the (free, as in free beer!) elastic runner from
> gitlab and failed. The second execution [5] ran in my private runner (a docker
> in my personal computer) and passed.
> 
> [4] https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/25090131
> [5] https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/25099593
> 
> Concerning the test infra (I say that including gitlab) we should choose a path
> to follow:
> 1) Use private runners

So it is possible to "contribute" as a private runner for your own CI
projects hosted on the public Gitlab ?

> 2) Adapt timeouts when a test sporadically fails and keep using the gitlab
> runners ([6] shows that we already use them)
> 
> [6] https://gitlab.com/buildroot.org/buildroot/-/jobs/24184768
> 
> I would go with option 2. We are not doing CI for now, so sporadic failures are
> not a big concern IMO.

The idea of using Gitlab is (amongst other things) to leverage the
public runners. That being said, sporadic failures are really not nice,
because they always cause confusion.

Why don't we significantly increase the timeout, like to several
minutes? In most cases, the test will be successful, and the expected
output will arrive in a few seconds. It's only when there is a real
problem in the test that we will hit the timeout, but in this case,
having the test take a few more minutes is not a big deal.

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

[Buildroot] [PATCH 04/14 v4] package/skeleton: split out into skeleton-custom

[Arnout Vandecappelle]

On 28-07-17 08:18, Thomas Petazzoni wrote:
> Hello,
> 
> On Fri, 28 Jul 2017 00:02:34 -0300, Ricardo Martincoski wrote:
[snip]
>> Concerning the test infra (I say that including gitlab) we should choose a path
>> to follow:
>> 1) Use private runners
> 
> So it is possible to "contribute" as a private runner for your own CI
> projects hosted on the public Gitlab ?

 Absolutely! It is in fact more or less intended to be used that way, i.e. as a
company you let gitlab.com handle your repository, but you provide your own
machines to run tests.

 That is certainly one way to go. It would also allow us to do some additional
things that are currently impossible, e.g. generating the buildroot-base docker
image.


>> 2) Adapt timeouts when a test sporadically fails and keep using the gitlab
>> runners ([6] shows that we already use them)
>>
>> [6] https://gitlab.com/buildroot.org/buildroot/-/jobs/24184768
>>
>> I would go with option 2. We are not doing CI for now, so sporadic failures are
>> not a big concern IMO.
> 
> The idea of using Gitlab is (amongst other things) to leverage the
> public runners. That being said, sporadic failures are really not nice,
> because they always cause confusion.

 Agreed.


> Why don't we significantly increase the timeout, like to several
> minutes? In most cases, the test will be successful, and the expected
> output will arrive in a few seconds. It's only when there is a real
> problem in the test that we will hit the timeout, but in this case,
> having the test take a few more minutes is not a big deal.

 That's actually a very good idea.

 However, I do wonder if that will really help. Looking at the log of [1] again,
it really looks like for 15 seconds, the *only* thing going on was starting
getty. That makes me think that there must be something else going on than just
a timeout (increasing the timeout was my first thought as well).

 As it stands, things are rather hard to debug. Perhaps we should do something
like store the modified rootfs as an artefact so it can be downloaded and you
can extract the journal from it?

 Regards,
 Arnout


[1]
https://gitlab.com/arnout/buildroot/-/jobs/24489865/artifacts/raw/test-output/TestInitSystemSystemdRwFull-run.log


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] [PATCH 02/14 v4] package/skeleton: drop dependency on host-mkpasswd

[Thomas Petazzoni]

Hello,

On Tue, 25 Jul 2017 23:14:23 +0200, Yann E. MORIN wrote:
> Setting the root pasword is done in a target-finalize hook, so we do not
> need to enforce a dependency from the skeleton onto host-mkpasswd.
> 
> Dropping that dependency will simplify making skeleton a virtual
> package (in up-coming changes).
> 
> Instead, it is now selected as any other package. As such, it is
> guaranteed to be built before target-finalize.
> 
> This however introduces a slight change in behaviour: previously,
> host-mkpasswd would only be built if we needed to hash the root password
> from its plain-text value. Now, host-mkpasswd is always built as soon as
> the root password is non-empty, even if already pre-hashed.
> 
> Since host-mkpasswd is a really tiny weeny package bundled in Buildroot,
> with only two C files, built as a single unit with a single gcc call,
> the overhead is really minimal. Compared to the simplifications this
> will allow in the skeleton packages (plural: common, sysv, systemd,
> custom) to come, this overhead is acceptable.
> 
> Yet another simplification, even if small, to ease providing multiple
> skeletons.
> 
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
> 
> ---
> Changes v3 -> v4:
>   - add blurb that is is selected, so built before target-finalize
>     (Arnout)
> ---
>  package/skeleton/skeleton.mk | 1 -
>  system/Config.in             | 1 +
>  2 files changed, 1 insertion(+), 1 deletion(-)

Applied to master, thanks.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH 03/14 v4] package/skeleton: select it rather than default to y

[Thomas Petazzoni]

Hello,

On Tue, 25 Jul 2017 23:14:24 +0200, Yann E. MORIN wrote:
> Our current skeleton is tailored to sysv-like init systems; it is not
> fit for systemd-based systems. So, in upcoming changes, we'll add
> another skeleton for systemd.
> 
> This means we can no longer have the current skeleton default to 'y', or
> it would be enabled also for systemd, which would be incorrect.
> 
> So, we remove the default to 'y' but have it selected by the default
> skeleton choice.
> 
> However, we do not yet have a way to directly build (really, install)
> the custom skeleton, it is built (really, installed) as a dependency of
> the default skeleton. So we must also forcibly select the default
> skeleton when using a custom one.
> 
> Until we have the means to do only one or the other; i.e. when we have a
> virtual skeleton.
> 
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
> ---
>  package/skeleton/Config.in | 1 -
>  system/Config.in           | 2 ++
>  2 files changed, 2 insertions(+), 1 deletion(-)

Applied to master, thanks.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com