[Buildroot] [RFC 00/11] common init infrastructure

[Buildroot] [RFC 00/11] common init infrastructure

[Alex Suykov]

This series is intended to clean up buildroot init system somewhat,
removing the need for extensive sysv and systemd handling code
in every package that installs init bits, and allowing alternative
init systems to be implemented with reasonable amount of effort.

Overview of the changes:

* drop per-package sysv and systemd files, introduce common format
  used to generate files for both of them, use a script to do
  the conversion from common format to sysv or systemd.

* remove sysv and systemd installation code from package .mk files,
  let pkg-generic handle it.

* remove sysv-specific files from skeleton, and sysv-specific hacks
  from system/system.mk. Use a dedicated script to write inittab,
  handling global conditionals (root rw/ro, serial getty and so on)

* remove user definitions from package .mk files, and mkdirs from
  per-package sysv initscript, use per-package .users and .files
  instead in a way similar to .hash files.

Expected benefits:

* (almost) all packages get both sysv and systemd files at the same
  time and without the need to support them separately.

* init-related changes can be introduced without patching
  all 70+ individual packages that install startup files.

* new init systems can be introduced with only minor effort,
  including lightweight ones with controlled (foreground) daemons.

This series includes the infrastructure changes, and a couple of
package patches to show how it looks like. Complete set of package
patches uploaded here: https://github.com/arsv/br/


This series is for reviewing, not for merging.

-- 
2.0.3

[Buildroot] [RFC 01/11] common service startup files

[Alex Suykov]

This patch introduces support infrastructure for per-project .run files.

Any .run files in package directory are picked up by pkg-generic code,
fed to support/init/install-run script which converts them into either
sysv initscripts or systemd unit files and installs those into predefined
locations. The package itself provides only the .run files, no code
in package .mk file is necessary.

Packages that install some of their services conditionally may set
$(PKG)_INIT = list of run files to install. As a side effect, this
removes the need to handle S99foo S99bar vs foo.service bar.service
with the package .mk file.

Packages that need to substititue some values within their service
files may set $(PKG)_INIT = KEY:value KEY:value ..., which will be
handled by install-run. The substitutions will be applied to sysv
and systemd files.

In case particular init system needs hand-crafted startup files
or cannot handle certain services at all, install-run may be told
so using special lines in the .run file.

This patch does not remove existing BR2_INIT_{SYSV,SYSTEMD} hooks.
Each package should either provide .run files or define hooks, but not both.
---
 package/Makefile.in          |  10 +
 package/pkg-generic.mk       |  23 +++
 support/init/install-run     | 457 +++++++++++++++++++++++++++++++++++++++++++
 support/init/install-tmpconf |   7 +
 4 files changed, 497 insertions(+)
 create mode 100755 support/init/install-run
 create mode 100755 support/init/install-tmpconf

diff --git a/package/Makefile.in b/package/Makefile.in
index 803b162..599fe7f 100644
--- a/package/Makefile.in
+++ b/package/Makefile.in
@@ -408,6 +408,16 @@ ifeq ($(BR2_COMPILER_PARANOID_UNSAFE_PATH),y)
 export BR_COMPILER_PARANOID_UNSAFE_PATH=enabled
 endif
 
+ifdef BR2_INIT_SYSV
+BR2_INIT = initscripts
+else ifdef BR2_INIT_BUSYBOX
+BR2_INIT = initscripts
+else ifdef BR2_INIT_SYSTEMD
+BR2_INIT = systemd
+else
+BR2_INIT = none
+endif
+
 include package/pkg-download.mk
 include package/pkg-autotools.mk
 include package/pkg-cmake.mk
diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
index c1b379b..fe7de8a 100644
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -762,10 +762,33 @@ ifneq ($$(call suitable-extractor,$$($(2)_SOURCE)),$$(XZCAT))
 DL_TOOLS_DEPENDENCIES += $$(firstword $$(call suitable-extractor,$$($(2)_SOURCE)))
 endif
 
+# Init, users and files are for target image only
+ifeq ($4,target)
+
+$(2)_INIT ?= $$(notdir $$(wildcard package/$1/*.run))
+$(2)_POST_INSTALL_TARGET_HOOKS += $(2)_INSTALL_INIT
+
+# This will expand to (nothing) if none of the files are present
+# in the package directory.
+ifndef $(2)_INSTALL_INIT
+define $(2)_INSTALL_INIT
+	$$(call ifargs,support/init/install-run $$(BR2_INIT) \
+		$$($$(PKG)_INIT_SUBST),$$(addprefix package/$1/,$$($$(PKG)_INIT)))
+	$$(call ifargs,support/init/install-tmpconf $$(BR2_INIT),$$(wildcard package/$1/*.tmp.conf))
+endef
+endif
+
+endif # $4 == target
+
 endif # $(2)_KCONFIG_VAR
 endef # inner-generic-package
 
 ################################################################################
+define ifargs
+	$(if $2,$1 $2)
+endef
+
+################################################################################
 # generic-package -- the target generator macro for generic packages
 ################################################################################
 
diff --git a/support/init/install-run b/support/init/install-run
new file mode 100755
index 0000000..24b79c7
--- /dev/null
+++ b/support/init/install-run
@@ -0,0 +1,457 @@
+#!/usr/bin/env python
+
+# Usage:
+#
+#   install-init [-] init-system [VAR1:val1 VAR2:val2 ...] file1.run file2.run ...
+#
+# init-system is one the values from inits array below.
+#
+# The script will write relevant configuration files to output/target/etc/...,
+# so it should be called from the top Buildroot directory.
+# Use - to write all files to the current directory (useful for debugging)
+
+# Bundling all init systems into a single script, vs having install-init-(system)
+# scripts for each system, may look strange, but it turns out most of the code
+# below is common for all init systems.
+
+from sys import stderr, argv
+from re import match, sub
+from os.path import dirname, basename
+from os import makedirs, fdopen, fstat, fchmod
+from os import O_RDONLY, O_WRONLY, O_CREAT, O_TRUNC
+from os import open as osopen
+from os import read as osread
+from os import write as oswrite
+
+inits = ['initscripts', 'systemd']
+output = None
+nopath = False
+
+class Run:
+    def __init__(self, name, path):
+        self.path = path    # full .run file name, for error reporting
+        self.name = name    # service name = stem(basename(path))
+
+        # This is a bit more complicated than it should be
+        # to allow reporting incorrect global keys while still
+        # accepting unadorned commands for pre/post groups.
+
+        for k in ['description', 'user', 'group', 'umask',
+                  'pidfile', 'priority',
+                  'after', 'requires', 'wantedby',
+                  'busname', 'conflicts', 'killwith', 'restart']:
+            setattr(self, k, None)
+
+        # sequence:
+        #   pre-exec-post
+        #   pre-start-stop-post
+        #   pre  (for run-once scripts)
+        self.pre = []
+        self.post = []
+        self.acts = ['exec', 'start', 'stop', 'reload']
+        for k in self.acts:
+            setattr(self, k, None)
+
+        for k in inits:
+            setattr(self, k, None)
+
+        # variable substitutions
+        # (default values from .run until update() is called)
+        self.subs = {}
+
+    def has(self, key):
+        return hasattr(self, key)
+
+    def get(self, key):
+        return getattr(self, key)
+
+    def set(self, key, val):
+        setattr(self, key, val)
+
+    def update(self, values):
+        self.subs.update(values)
+
+    def isempty(self):
+        return (not self.exec and not self.start and not self.pre and not self.post)
+
+    def substcmds(self):
+        for k in ['pre', 'post']:
+            attr = getattr(self, k)
+            for i, s in enumerate(attr):
+                attr[i] = self.substvars(s)
+        for k in ['exec', 'start', 'stop', 'reload']:
+            attr = getattr(self, k)
+            if not attr is None:
+                setattr(self, k, self.substvars(attr))
+
+    def substvars(self, value):
+        def var(m):
+            k = m.group(1)
+            if k in self.subs:
+                return self.subs[k]
+            else:
+                return m.group(0)
+
+        ret = sub(r'\$([A-Za-z]+)', var, value)
+        return ret
+
+
+def die(message, *args):
+    stderr.write((message+"\n") % tuple(args))
+    exit(-1)
+
+
+def openread(filename):
+    try:
+        fh = open(filename)
+    except OSError:
+        die("Can't open %s" % filename)
+    return fh
+
+def parse(name, path):
+    run = Run(name, path)
+    fh = openread(path)
+    inscript = False    # after the first empty line
+    pastpre = False     # after exec/start/stop
+    firstline = True
+
+    for line in fh:
+        line = line.strip()
+
+        # Use ## for description, but only if it is the first line
+        if firstline and match('^## ', line):
+            run.description = line[3:]
+        else:
+            firstline = False
+
+        # Skip comments in non-command section, but pass them through
+        # when inscript
+        if not inscript and match(r'^#', line):
+            continue
+
+        # Empty line marks the start of script section
+        if not inscript and match(r'^\s*$', line):
+            inscript = True
+            continue
+
+        m = match(r'^[A-Z_0-9]+=(.*)', line)
+        if m and not inscript:
+            run.subs[m.group(1)] = m.group(2)
+            continue
+
+        m = match(r'^([A-Za-z]+)(\s+(.*))?', line)
+        if m:
+            key, val = m.group(1), m.group(3)
+        else:
+            key, val = None, None
+
+        if not inscript and not key:
+            die("%s: bad line", path)
+
+        if (not inscript and key) or (inscript and key in run.acts):
+            if run.get(key):
+                die("%s: cannot re-assign %s", path, key)
+            elif run.has(key):
+                run.set(key, val if m.group(2) else True)
+            else:
+                die("%s: unknown key %s", path, key)
+            if key in run.acts:
+                pastpre = True
+        elif inscript:
+            if pastpre:
+                run.post.append(line)
+            else:
+                run.pre.append(line)
+
+    if not run.priority:
+        if run.exec:
+            run.priority = '50'
+        elif run.start:
+            run.priority = '40'
+        else:
+            run.priority = '10'
+
+    return run
+
+def openwrite(path, mode):
+    global nopath
+
+    if nopath:
+        path = basename(path)
+    else:
+        try:
+            makedirs(dirname(path))
+        except OSError:
+            pass
+
+    fd = osopen(path, O_WRONLY | O_CREAT | O_TRUNC)
+    fchmod(fd, mode)
+    return fd
+
+def writeto(path, mode):
+    global output
+    fd = openwrite(path, mode)
+    output = fdopen(fd, 'w')
+
+# write
+def w(fmt, *subst):
+    global output
+    output.write((fmt+"\n") % subst)
+
+# write newline
+def wnl(): w("")
+
+# write lines
+def wl(fmt, array):
+    for l in array:
+        w(fmt, l)
+
+# write-tab, write-tab-tab
+def wt(fmt, *subst): w("\t" + fmt, *subst)
+def wtt(fmt, *subst): w("\t\t" + fmt, *subst)
+
+# ------------------------------------------------------------------------------
+
+# return either cmd or su cmd; the caller naturally handles shell commands
+def maybesu(r, cmd):
+    if r.user:
+        return "su - %s -c '%s'" % (r.user, cmd)
+    else:
+        return cmd
+
+# ------------------------------------------------------------------------------
+
+def initscripts(r):
+    writeto('output/target/etc/init.d/S%02d%s' % (int(r.priority), r.name), 0o755)
+
+    (start, stop) = initscripts_ssd(r)
+
+    w("#!/bin/sh\n")
+    if r.description:
+        w("# %s\n", r.description)
+
+    for k, v in r.subs.items():
+        w("%s=%s", k, v)
+    if r.subs:
+        wnl()
+
+    if r.umask:
+        w("umask %s", r.umask)
+
+    w("case \"$1\" in")
+    w(" start)")
+    for c in r.pre:
+        wtt("%s", maybesu(r, c))
+    if r.start:
+        wtt("%s", maybesu(r.start))
+    elif start:
+        wtt("%s", start)
+    elif not r.pre and not r.post:
+        die("%s: cannot write initscript without any commands to run", r.path)
+    wtt(";;")
+
+    wt("stop)")
+    if r.stop and not r.stop is True:
+        wtt("%s", r.stop)
+    elif r.start or r.exec:
+        wtt("%s", stop)
+    for c in r.post:
+        wtt("%s", maybesu(c))
+    wtt(";;")
+
+    if r.reload:
+        wt("reload)")
+        wtt("%s", r.reload)
+        wtt(";;")
+
+    wt("restart)")
+    wtt("$0 stop")
+    wtt("$0 start")
+    wtt(";;")
+
+    w("esac")
+
+# generate start-stop-daemon commands from r.exec
+# run -> (start, stop)
+def initscripts_ssd(r):
+    if not r.exec:
+        return None, None
+
+    if r.pidfile:
+        pidfile = r.pidfile
+    else:
+        pidfile = '/var/run/%s.pid' % r.name
+
+    ssdopts = ['-S']
+    sskopts = ['-K']
+    if r.user:
+        ssdopts.append('-c')
+        ssdopts.append(r.user)
+    if r.group:
+        ssdopts.append('-g')
+        ssdopts.append(r.group)
+    if not r.pidfile and not r.start:
+        ssdopts.append('-m')
+    if pidfile:
+        ssdopts.append('-p')
+        ssdopts.append(pidfile)
+        sskopts.append('-p')
+        sskopts.append(pidfile)
+    ssdopts = (" ".join(ssdopts))
+    sskopts = (" ".join(sskopts))
+
+    cmd = sub(r'\s.*', '', r.exec)
+    arg = sub(r'^\S+', ' --', r.exec) if cmd != r.exec else ''
+
+    start = "start-stop-daemon %s -b %s%s" % (ssdopts, cmd, arg)
+    stop = "start-stop-daemon %s" % sskopts
+
+    return (start, stop)
+
+# ------------------------------------------------------------------------------
+
+def systemd(r):
+    writeto("output/target/usr/lib/systemd/system/%s.service" % r.name, 0o644)
+    r.substcmds()
+
+    if r.after:
+        after = r.after
+    elif r.priority and r.priority >= '90':
+        after = "syslog.service network.service"
+    elif r.priority and r.priority >= '50':
+        after = "syslog.service"
+    else:
+        after = None
+
+    w("[Unit]")
+    if r.description:
+        w("Description=%s", r.description)
+    if after:
+        w("After=%s", after)
+    if r.requires:
+        w("Requires=%s", r.requires)
+    if r.conflicts:
+        w("Conflicts=%s", r.conflicts)
+    wnl()
+
+    w("[Service]")
+    if r.user:
+        w("User=%s", r.user)
+    if r.group:
+        w("Group=%s", r.group)
+    if r.pidfile and not r.exec:
+        w("PIDFile=%s", r.pidfile)
+    if r.start or r.exec:
+        for c in r.pre:
+            w("ExecStartPre=%s", c)
+
+    if r.exec:
+        w("ExecStart=%s", r.exec)
+    elif r.start:
+        w("Type=forking")
+        w("ExecStart=%s", r.start)
+    elif r.pre:
+        w("Type=oneshot")
+        wl("ExecStart=%s", r.pre)
+
+    if r.stop and not r.exec:
+        w("ExecStop=%s", r.stop)
+
+    for c in r.post:
+        w("ExecStartPost=%s", c)
+    if r.reload:
+        w("ExecReload=%s", r.reload)
+    if r.restart:
+        w("Restart=%s", r.restart)
+    elif r.exec:
+        w("Restart=always")
+    wnl()
+
+    w("[Install]")
+    if r.wantedby:
+        w("WantedBy=%s", r.wantedby)
+    else:
+        w("WantedBy=multi-user.target")
+
+# ------------------------------------------------------------------------------
+
+# Usage:
+#     bypass systemd foo.service
+#     bypass initscripts foo.init foo.line
+#     bypass inittab -
+
+bypasstable = {
+    'initscripts': [
+        ('*:*.init',    'output/target/etc/init.d/S$1$2'),
+        ('*.init',      'output/target/etc/init.d/S50$1'),
+        ('*.conf',      'output/target/etc/config.d/*'),
+        ('S*',          'output/target/etc/init.d/*') ],
+    'systemd': [
+        ('*.service',   'output/target/usr/lib/systemd/system/*.service'),
+        ('*.tmp.conf',  'output/target/usr/lib/tmpfiles.d/*.conf') ]
+}
+
+def bypass(run, init, arg):
+    if arg == '-': return
+    rules = bypasstable[init]
+    for f in arg.split():
+        copied = False
+        for r in rules:
+            m = match(r[0].replace('*','(.*)'), basename(f))
+            if not m: continue
+            copyfile(sub(r'.*:', '', f),
+                    r[1].replace('*', m.group(1))
+                        .replace('$1', m.group(1))
+                        .replace('$2', m.group(2)))
+            copied = True
+            break
+        if not copied:
+            die('no destination for %s', f)
+
+def copyfile(src, dst):
+    f = osopen(src, O_RDONLY)
+    s = fstat(f)
+    o = openwrite(dst, s.st_mode)
+    buf = osread(f, s.st_size)
+    oswrite(o, buf)
+
+# ------------------------------------------------------------------------------
+
+if len(argv) > 1 and argv[1] == '-':
+    nopath = True
+    del argv[1]
+
+if len(argv) < 2:
+    die("Usage: install-init [-] init-system file.run file.run ...")
+
+init = argv[1]
+
+if not init:
+    die("Bad call")
+
+if init in inits:
+    writer = globals()[init]
+else:
+    die("Unknown init system %s", init)
+
+subst = []
+files = []
+
+for a in argv[2:]:
+    eq = a.find(':')
+    if eq >= 0:
+        subst.append((a[0:eq], a[eq+1:]))
+    elif a.endswith('.run'):
+        files.append((basename(a)[:-4], a))
+    else:
+        die("Bad argument: %s", a)
+
+for f in files:
+    run = parse(f[0], f[1])
+    run.update(subst)
+    if getattr(run, init):
+        bypass(run, init, getattr(run, init))
+    if r.isempty():
+        continue
+    else:
+        writer(run)
diff --git a/support/init/install-tmpconf b/support/init/install-tmpconf
new file mode 100755
index 0000000..080dcd7
--- /dev/null
+++ b/support/init/install-tmpconf
@@ -0,0 +1,7 @@
+#!/bin/sh
+
+test "$1" = 'systemd' && shift || exit
+mkdir -p output/target/etc/tmpfiles.d/
+for i in "$@"; do
+	cp $i output/target/etc/tmpfiles.d/${i/.tmp.conf/.conf}
+done
-- 
2.0.3

[Buildroot] [RFC 01/11] common service startup files

[Arnout Vandecappelle]

On 21/03/15 19:27, Alex Suykov wrote:
> This patch introduces support infrastructure for per-project .run files.
> 
> Any .run files in package directory are picked up by pkg-generic code,
> fed to support/init/install-run script which converts them into either
> sysv initscripts or systemd unit files and installs those into predefined
> locations. The package itself provides only the .run files, no code
> in package .mk file is necessary.
> 
> Packages that install some of their services conditionally may set
> $(PKG)_INIT = list of run files to install. As a side effect, this
> removes the need to handle S99foo S99bar vs foo.service bar.service
> with the package .mk file.
> 
> Packages that need to substititue some values within their service

 substitute

> files may set $(PKG)_INIT = KEY:value KEY:value ..., which will be
> handled by install-run. The substitutions will be applied to sysv
> and systemd files.

 I don't see why this is needed, and it probably makes the install-run script
more complicated.

> 
> In case particular init system needs hand-crafted startup files
> or cannot handle certain services at all, install-run may be told
> so using special lines in the .run file.
> 
> This patch does not remove existing BR2_INIT_{SYSV,SYSTEMD} hooks.

 Good!

> Each package should either provide .run files or define hooks, but not both.
> ---

 Missing SoB, but perhaps that's intentional because it's not ready for inclusion?

>  package/Makefile.in          |  10 +
>  package/pkg-generic.mk       |  23 +++
>  support/init/install-run     | 457 +++++++++++++++++++++++++++++++++++++++++++
>  support/init/install-tmpconf |   7 +
>  4 files changed, 497 insertions(+)
>  create mode 100755 support/init/install-run
>  create mode 100755 support/init/install-tmpconf
> 
> diff --git a/package/Makefile.in b/package/Makefile.in
> index 803b162..599fe7f 100644
> --- a/package/Makefile.in
> +++ b/package/Makefile.in
> @@ -408,6 +408,16 @@ ifeq ($(BR2_COMPILER_PARANOID_UNSAFE_PATH),y)
>  export BR_COMPILER_PARANOID_UNSAFE_PATH=enabled
>  endif
>  
> +ifdef BR2_INIT_SYSV
> +BR2_INIT = initscripts
> +else ifdef BR2_INIT_BUSYBOX
> +BR2_INIT = initscripts
> +else ifdef BR2_INIT_SYSTEMD
> +BR2_INIT = systemd
> +else
> +BR2_INIT = none
> +endif

 It's nicer to do this kind of switch in system/Config.in:

config BR2_INIT
	string
	default "initscripts" if BR2_INIT_SYSV || BR2_INIT_BUSYBOX
	default "systemd" if BR2_INIT_SYSTEMD
	default "none"

> +
>  include package/pkg-download.mk
>  include package/pkg-autotools.mk
>  include package/pkg-cmake.mk
> diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
> index c1b379b..fe7de8a 100644
> --- a/package/pkg-generic.mk
> +++ b/package/pkg-generic.mk
> @@ -762,10 +762,33 @@ ifneq ($$(call suitable-extractor,$$($(2)_SOURCE)),$$(XZCAT))
>  DL_TOOLS_DEPENDENCIES += $$(firstword $$(call suitable-extractor,$$($(2)_SOURCE)))
>  endif
>  
> +# Init, users and files are for target image only
> +ifeq ($4,target)
> +
> +$(2)_INIT ?= $$(notdir $$(wildcard package/$1/*.run))

 package/$1/ is not correct, because it could be outside of the package
directory (e.g. external) or in a subdirectory. Use $(pkgdir) instead (note:
only one $!).

> +$(2)_POST_INSTALL_TARGET_HOOKS += $(2)_INSTALL_INIT
> +
> +# This will expand to (nothing) if none of the files are present
> +# in the package directory.
> +ifndef $(2)_INSTALL_INIT

 There should never be any reason for a package to override _INSTALL_INIT, they
can just avoid adding *.run files or setting _INIT.

 Therefore, I'd also move this to the %/.stamp_target_installed rule. That makes
it easier to understand because of less $$. Note that that is also the place
where the current init handling is done. With that done, there is also no need
to keep the definition of _INIT in the ifeq($4,target) condition.

> +define $(2)_INSTALL_INIT
> +	$$(call ifargs,support/init/install-run $$(BR2_INIT) \
> +		$$($$(PKG)_INIT_SUBST),$$(addprefix package/$1/,$$($$(PKG)_INIT)))

 The line is split at an inconvenient place here.

 I wouldn't do this $(notdir) and $(addprefix) magic. It's OK for the package
.mk file to be required to provide the full path to the .run file.

> +	$$(call ifargs,support/init/install-tmpconf $$(BR2_INIT),$$(wildcard package/$1/*.tmp.conf))

 This tmpdir support should probably be in a separate patch.

 BTW, shouldn't tmpdirs be handled for sysv as well? Typically, the init script
will create the tmpdir...

> +endef
> +endif
> +
> +endif # $4 == target
> +
>  endif # $(2)_KCONFIG_VAR
>  endef # inner-generic-package
>  
>  ################################################################################
> +define ifargs
> +	$(if $2,$1 $2)
> +endef

 This should go to pkg-utils instead.

> +
> +################################################################################
>  # generic-package -- the target generator macro for generic packages
>  ################################################################################
>  
> diff --git a/support/init/install-run b/support/init/install-run
> new file mode 100755
> index 0000000..24b79c7
> --- /dev/null
> +++ b/support/init/install-run

 I've spent about an hour trying to understand this script, and I'm still not
fully there... So this probably falls in the "too complicated" category. I still
have a bunch of stylistic remarks below, but here are some overall thoughts.

 - The script should do something as simple as possible. The variable
substitution is an example of something that is (probably) adding complexity,
and it can be done in a different way, e.g. running a sed script afterwards.

 - The script should only handle cases that are actually used. So start with
something extremely simple that covers a few interesting cases, and leave the
others to use the old initscripts system.

 - There should be a lot more comments to explain what is done and why.

 - You should start with the documentation of what it's supposed to do. That
makes it easier for us to understand what it's doing (and to say which things it
shouldn't do :-)

 - Avoid the getattr() etc. magic. It's easier to understand what's going on if
things are in a simple and explicit dict.

 - Use existing infrastructure where possible. For instance, you could change
the .run format into .ini format and use configparser, or you could use the
shlex module for parsing.

 - Don't make names too short, they should always be descriptive. 'w' is not a
good name for a function. If typing is a problem for you, get an editor that
supports completion.

 - You should take care of the exit conditions (and document them). If there is
something wrong with a .run file or with the script itself, it should exit
fatally and the build should be stopped. But if the .run file contains something
that is not supported for this init system, this should be non-fatal.


> @@ -0,0 +1,457 @@
> +#!/usr/bin/env python
> +
> +# Usage:
> +#
> +#   install-init [-] init-system [VAR1:val1 VAR2:val2 ...] file1.run file2.run ...
> +#
> +# init-system is one the values from inits array below.
> +#
> +# The script will write relevant configuration files to output/target/etc/...,
> +# so it should be called from the top Buildroot directory.
> +# Use - to write all files to the current directory (useful for debugging)
> +
> +# Bundling all init systems into a single script, vs having install-init-(system)
> +# scripts for each system, may look strange, but it turns out most of the code
> +# below is common for all init systems.
> +
> +from sys import stderr, argv

 We don't do that. Just

import sys

   ... sys.argv ...


> +from re import match, sub
> +from os.path import dirname, basename
> +from os import makedirs, fdopen, fstat, fchmod
> +from os import O_RDONLY, O_WRONLY, O_CREAT, O_TRUNC
> +from os import open as osopen
> +from os import read as osread
> +from os import write as oswrite
> +
> +inits = ['initscripts', 'systemd']
> +output = None

 Don't put this in a global variable. A member of Run would be a good place.

> +nopath = False
> +
> +class Run:
> +    def __init__(self, name, path):
> +        self.path = path    # full .run file name, for error reporting
> +        self.name = name    # service name = stem(basename(path))

 This splitting into path and name should be handled here, in the constructor,
and not in the caller.

> +
> +        # This is a bit more complicated than it should be
> +        # to allow reporting incorrect global keys while still
> +        # accepting unadorned commands for pre/post groups.
> +
> +        for k in ['description', 'user', 'group', 'umask',
> +                  'pidfile', 'priority',
> +                  'after', 'requires', 'wantedby',
> +                  'busname', 'conflicts', 'killwith', 'restart']:
> +            setattr(self, k, None)

 This will also be simpler if you just use a dict.

> +
> +        # sequence:
> +        #   pre-exec-post
> +        #   pre-start-stop-post
> +        #   pre  (for run-once scripts)
> +        self.pre = []
> +        self.post = []
> +        self.acts = ['exec', 'start', 'stop', 'reload']
> +        for k in self.acts:
> +            setattr(self, k, None)
> +
> +        for k in inits:
> +            setattr(self, k, None)
> +
> +        # variable substitutions
> +        # (default values from .run until update() is called)
> +        self.subs = {}
> +
> +    def has(self, key):
> +        return hasattr(self, key)
> +
> +    def get(self, key):
> +        return getattr(self, key)
> +
> +    def set(self, key, val):
> +        setattr(self, key, val)
> +
> +    def update(self, values):
> +        self.subs.update(values)
> +
> +    def isempty(self):
> +        return (not self.exec and not self.start and not self.pre and not self.post)

 exec is a reserved word in python2.

> +
> +    def substcmds(self):
> +        for k in ['pre', 'post']:
> +            attr = getattr(self, k)
> +            for i, s in enumerate(attr):
> +                attr[i] = self.substvars(s)
> +        for k in ['exec', 'start', 'stop', 'reload']:
> +            attr = getattr(self, k)
> +            if not attr is None:
> +                setattr(self, k, self.substvars(attr))
> +
> +    def substvars(self, value):
> +        def var(m):
> +            k = m.group(1)
> +            if k in self.subs:
> +                return self.subs[k]
> +            else:
> +                return m.group(0)
> +
> +        ret = sub(r'\$([A-Za-z]+)', var, value)
> +        return ret
> +
> +
> +def die(message, *args):
> +    stderr.write((message+"\n") % tuple(args))
> +    exit(-1)
> +
> +
> +def openread(filename):
> +    try:
> +        fh = open(filename)
> +    except OSError:
> +        die("Can't open %s" % filename)

 This wrapper doesn't make a lot of sense. Such failures should anyway not
occur, because the script is always called with .run files. So it's OK to dump
an exception stack on the user.

> +    return fh
> +
> +def parse(name, path):
> +    run = Run(name, path)
> +    fh = openread(path)
> +    inscript = False    # after the first empty line
> +    pastpre = False     # after exec/start/stop
> +    firstline = True
> +
> +    for line in fh:
> +        line = line.strip()
> +
> +        # Use ## for description, but only if it is the first line
> +        if firstline and match('^## ', line):
> +            run.description = line[3:]

 Why this ## hack? "description = This is program foo" is much more descriptive.

> +        else:
> +            firstline = False
> +
> +        # Skip comments in non-command section, but pass them through
> +        # when inscript
> +        if not inscript and match(r'^#', line):
> +            continue
> +
> +        # Empty line marks the start of script section
> +        if not inscript and match(r'^\s*$', line):
> +            inscript = True
> +            continue
> +
> +        m = match(r'^[A-Z_0-9]+=(.*)', line)
> +        if m and not inscript:
> +            run.subs[m.group(1)] = m.group(2)
> +            continue
> +
> +        m = match(r'^([A-Za-z]+)(\s+(.*))?', line)
> +        if m:
> +            key, val = m.group(1), m.group(3)
> +        else:
> +            key, val = None, None
> +
> +        if not inscript and not key:
> +            die("%s: bad line", path)
> +
> +        if (not inscript and key) or (inscript and key in run.acts):
> +            if run.get(key):
> +                die("%s: cannot re-assign %s", path, key)
> +            elif run.has(key):
> +                run.set(key, val if m.group(2) else True)
> +            else:
> +                die("%s: unknown key %s", path, key)
> +            if key in run.acts:
> +                pastpre = True
> +        elif inscript:
> +            if pastpre:
> +                run.post.append(line)
> +            else:
> +                run.pre.append(line)
> +
> +    if not run.priority:
> +        if run.exec:
> +            run.priority = '50'
> +        elif run.start:
> +            run.priority = '40'
> +        else:
> +            run.priority = '10'

 The reasoning behind these static priorities should be explained somewhere.

> +
> +    return run
> +
> +def openwrite(path, mode):
> +    global nopath
> +
> +    if nopath:
> +        path = basename(path)
> +    else:
> +        try:
> +            makedirs(dirname(path))
> +        except OSError:
> +            pass
> +
> +    fd = osopen(path, O_WRONLY | O_CREAT | O_TRUNC)
> +    fchmod(fd, mode)
> +    return fd

 This whole umask thing is useless IMHO. You also don't provide an example for it.

 If someone is so paranoid that they think an init script should be
non-readable, they should give it a BR2_ROOTFS_DEVICE_TABLE entry.

 And anyway, use chmod instead of fchmod.

 So, if the umask is gone, you can just do open(path, 'w')

> +
> +def writeto(path, mode):
> +    global output
> +    fd = openwrite(path, mode)
> +    output = fdopen(fd, 'w')
> +
> +# write
> +def w(fmt, *subst):
> +    global output
> +    output.write((fmt+"\n") % subst)

 Making a function for this is really not worth it.

> +
> +# write newline
> +def wnl(): w("")
> +
> +# write lines
> +def wl(fmt, array):
> +    for l in array:
> +        w(fmt, l)
> +
> +# write-tab, write-tab-tab
> +def wt(fmt, *subst): w("\t" + fmt, *subst)
> +def wtt(fmt, *subst): w("\t\t" + fmt, *subst)
> +
> +# ------------------------------------------------------------------------------
> +
> +# return either cmd or su cmd; the caller naturally handles shell commands
> +def maybesu(r, cmd):

 Should be a method of Run.

> +    if r.user:
> +        return "su - %s -c '%s'" % (r.user, cmd)
> +    else:
> +        return cmd
> +
> +# ------------------------------------------------------------------------------
> +
> +def initscripts(r):
> +    writeto('output/target/etc/init.d/S%02d%s' % (int(r.priority), r.name), 0o755)

 Ah yes here you need to set the exec bit of course.

 output/ is wrong, it could be modified by O=. Instead, you should pass
TARGET_DIR in the environment or on the commandline and use that.

> +
> +    (start, stop) = initscripts_ssd(r)
> +
> +    w("#!/bin/sh\n")
> +    if r.description:
> +        w("# %s\n", r.description)
> +
> +    for k, v in r.subs.items():
> +        w("%s=%s", k, v)
> +    if r.subs:
> +        wnl()
> +
> +    if r.umask:
> +        w("umask %s", r.umask)
> +
> +    w("case \"$1\" in")
> +    w(" start)")
> +    for c in r.pre:
> +        wtt("%s", maybesu(r, c))
> +    if r.start:
> +        wtt("%s", maybesu(r.start))
> +    elif start:
> +        wtt("%s", start)
> +    elif not r.pre and not r.post:
> +        die("%s: cannot write initscript without any commands to run", r.path)
> +    wtt(";;")
> +
> +    wt("stop)")
> +    if r.stop and not r.stop is True:
> +        wtt("%s", r.stop)
> +    elif r.start or r.exec:
> +        wtt("%s", stop)
> +    for c in r.post:
> +        wtt("%s", maybesu(c))
> +    wtt(";;")
> +
> +    if r.reload:
> +        wt("reload)")
> +        wtt("%s", r.reload)
> +        wtt(";;")
> +
> +    wt("restart)")
> +    wtt("$0 stop")
> +    wtt("$0 start")
> +    wtt(";;")
> +
> +    w("esac")

 A multiline template with %-formatting would be easier to understand. So:

completestart = map(maybesu,r.start).join('\n\t\t')
completestart += ...
completestop = ...

output.write("""
#! /bin/sh
%s

case "$1" in
	start)
		%s;;
	stop)
		%s;;
	restart|reload)
		$0 stop
		$0 start
		;;
esac
""" % (substitions, completestart, completestop)



> +
> +# generate start-stop-daemon commands from r.exec
> +# run -> (start, stop)
> +def initscripts_ssd(r):
> +    if not r.exec:
> +        return None, None
> +
> +    if r.pidfile:
> +        pidfile = r.pidfile
> +    else:
> +        pidfile = '/var/run/%s.pid' % r.name
> +
> +    ssdopts = ['-S']
> +    sskopts = ['-K']

 ssdopts and sskopts are visually too similar to make it easy to distinguish.
What about startopts and stopopts?

> +    if r.user:
> +        ssdopts.append('-c')
> +        ssdopts.append(r.user)
> +    if r.group:
> +        ssdopts.append('-g')
> +        ssdopts.append(r.group)
> +    if not r.pidfile and not r.start:
> +        ssdopts.append('-m')
> +    if pidfile:
> +        ssdopts.append('-p')
> +        ssdopts.append(pidfile)
> +        sskopts.append('-p')
> +        sskopts.append(pidfile)
> +    ssdopts = (" ".join(ssdopts))

 It's ugly if a variable is first a list and then a string. But you don't need
to do it here, just do the join below where it's used to construct start/stop.

> +    sskopts = (" ".join(sskopts))
> +
> +    cmd = sub(r'\s.*', '', r.exec)
> +    arg = sub(r'^\S+', ' --', r.exec) if cmd != r.exec else ''

 	cmd = re.sub(r'\s.*', ' -- ', 1)
	start = "start-stop-daemon %s -b %s" % (" ".join(ssdopts), cmd)

> +
> +    start = "start-stop-daemon %s -b %s%s" % (ssdopts, cmd, arg)
> +    stop = "start-stop-daemon %s" % sskopts

 I prefer an explicit /sbin/start-stop-daemon - when manually starting/stopping,
/sbin is not always in the path.


> +
> +    return (start, stop)
> +
> +# ------------------------------------------------------------------------------
> +
> +def systemd(r):
> +    writeto("output/target/usr/lib/systemd/system/%s.service" % r.name, 0o644)
> +    r.substcmds()
> +
> +    if r.after:
> +        after = r.after
> +    elif r.priority and r.priority >= '90':
> +        after = "syslog.service network.service"
> +    elif r.priority and r.priority >= '50':

 If that is the meaning of the priorities, perhaps it's better to give them
names rather than numbers.

> +        after = "syslog.service"
> +    else:
> +        after = None
> +
> +    w("[Unit]")
> +    if r.description:
> +        w("Description=%s", r.description)
> +    if after:
> +        w("After=%s", after)
> +    if r.requires:
> +        w("Requires=%s", r.requires)
> +    if r.conflicts:
> +        w("Conflicts=%s", r.conflicts)
> +    wnl()
> +
> +    w("[Service]")
> +    if r.user:
> +        w("User=%s", r.user)
> +    if r.group:
> +        w("Group=%s", r.group)
> +    if r.pidfile and not r.exec:
> +        w("PIDFile=%s", r.pidfile)
> +    if r.start or r.exec:
> +        for c in r.pre:
> +            w("ExecStartPre=%s", c)
> +
> +    if r.exec:
> +        w("ExecStart=%s", r.exec)
> +    elif r.start:
> +        w("Type=forking")
> +        w("ExecStart=%s", r.start)
> +    elif r.pre:
> +        w("Type=oneshot")
> +        wl("ExecStart=%s", r.pre)
> +
> +    if r.stop and not r.exec:
> +        w("ExecStop=%s", r.stop)
> +
> +    for c in r.post:
> +        w("ExecStartPost=%s", c)
> +    if r.reload:
> +        w("ExecReload=%s", r.reload)
> +    if r.restart:
> +        w("Restart=%s", r.restart)
> +    elif r.exec:
> +        w("Restart=always")
> +    wnl()
> +
> +    w("[Install]")
> +    if r.wantedby:
> +        w("WantedBy=%s", r.wantedby)
> +    else:
> +        w("WantedBy=multi-user.target")
> +
> +# ------------------------------------------------------------------------------
> +
> +# Usage:
> +#     bypass systemd foo.service
> +#     bypass initscripts foo.init foo.line
> +#     bypass inittab -

 This bypass syntax is hard to understand IMHO. Can you find something
simpler/clearer and more verbose? If you would switch to ini format, it could be
something like

[initscripts]
cp = package/foo/S10start
     /etc/init.d/S10start

[systemd]
cp-ln = package/foo/foo.service
        /usr/lib/systemd/system/foo.service
        /etc/systemd/system/multi-user.target.wants

> +
> +bypasstable = {
> +    'initscripts': [
> +        ('*:*.init',    'output/target/etc/init.d/S$1$2'),
> +        ('*.init',      'output/target/etc/init.d/S50$1'),
> +        ('*.conf',      'output/target/etc/config.d/*'),
> +        ('S*',          'output/target/etc/init.d/*') ],
> +    'systemd': [
> +        ('*.service',   'output/target/usr/lib/systemd/system/*.service'),
> +        ('*.tmp.conf',  'output/target/usr/lib/tmpfiles.d/*.conf') ]
> +}
> +
> +def bypass(run, init, arg):
> +    if arg == '-': return
> +    rules = bypasstable[init]
> +    for f in arg.split():
> +        copied = False
> +        for r in rules:
> +            m = match(r[0].replace('*','(.*)'), basename(f))
> +            if not m: continue
> +            copyfile(sub(r'.*:', '', f),
> +                    r[1].replace('*', m.group(1))
> +                        .replace('$1', m.group(1))
> +                        .replace('$2', m.group(2)))
> +            copied = True
> +            break
> +        if not copied:
> +            die('no destination for %s', f)
> +
> +def copyfile(src, dst):

 Use shutil.copyfile instead of inventing your own.

> +    f = osopen(src, O_RDONLY)
> +    s = fstat(f)
> +    o = openwrite(dst, s.st_mode)
> +    buf = osread(f, s.st_size)
> +    oswrite(o, buf)
> +
> +# ------------------------------------------------------------------------------
> +
> +if len(argv) > 1 and argv[1] == '-':
> +    nopath = True
> +    del argv[1]
> +
> +if len(argv) < 2:
> +    die("Usage: install-init [-] init-system file.run file.run ...")
> +
> +init = argv[1]
> +
> +if not init:
> +    die("Bad call")
> +
> +if init in inits:
> +    writer = globals()[init]
> +else:
> +    die("Unknown init system %s", init)
> +
> +subst = []
> +files = []
> +
> +for a in argv[2:]:
> +    eq = a.find(':')
> +    if eq >= 0:
> +        subst.append((a[0:eq], a[eq+1:]))
> +    elif a.endswith('.run'):
> +        files.append((basename(a)[:-4], a))
> +    else:
> +        die("Bad argument: %s", a)
> +
> +for f in files:
> +    run = parse(f[0], f[1])
> +    run.update(subst)
> +    if getattr(run, init):
> +        bypass(run, init, getattr(run, init))
> +    if r.isempty():
> +        continue
> +    else:
> +        writer(run)
> diff --git a/support/init/install-tmpconf b/support/init/install-tmpconf
> new file mode 100755
> index 0000000..080dcd7
> --- /dev/null
> +++ b/support/init/install-tmpconf
> @@ -0,0 +1,7 @@
> +#!/bin/sh
> +
> +test "$1" = 'systemd' && shift || exit
> +mkdir -p output/target/etc/tmpfiles.d/
> +for i in "$@"; do
> +	cp $i output/target/etc/tmpfiles.d/${i/.tmp.conf/.conf}
> +done

 You also didn't give an example of this, but I have the feeling it would make
more sense to put this in the .run script as well. For sysv, the tmpfile should
be created in the init script; for systemd, it's done by tmpfiles.


 Regards,
 Arnout

> 


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [RFC 01/11] common service startup files

[Alex Suykov]

Sun, Mar 22, 2015 at 12:57:50AM +0100, Arnout Vandecappelle wrote:

Thanks for extensive review, a lot of good points which I skip in
this reply because they should go directly into the code.

> Missing SoB, but perhaps that's intentional because it's not ready for inclusion?
Well, partially, and partially I just forgot -s when re-splitting commits.

> > files may set $(PKG)_INIT = KEY:value KEY:value ..., which will be
> > handled by install-run. The substitutions will be applied to sysv
> > and systemd files.
> 
>  I don't see why this is needed, and it probably makes the install-run script
> more complicated.
> 
BR2_PACKAGE_GPSD_DEVICES, and keeping it a variable in the resulting initscript.
Or maybe in /etc/default.
If sed substitutiions are ok, this can be done much simplier of course:

    sed s!DEVICES!/dev/ttyS1! package/gpsd/gpsd.run > build/gpsd-.../gpsd.run
    support/init/install-run build/gpsd-.../gpsd.run

> BTW, shouldn't tmpdirs be handled for sysv as well? Typically, the init script
> will create the tmpdir...
I saw this as systemd-specific.
Looking at the packages, yes, makes sense to have some kind of generic tmpdir
variable.

>  - The script should only handle cases that are actually used. So start with
> something extremely simple that covers a few interesting cases, and leave the
> others to use the old initscripts system.
Why I was trying handle as many packages as possible: I'm pretty sure something
like 80% of them will work immediately with only the basic support,
but it would be really bad to find a show-stopper among the remaining 20%.
Something that would require changing the way the whole thing works.

Perhaps it's ok to drop bad cases for now, leave them as initscripts-only
and expect the users do something about the unsupported cases themselves.

>  - Use existing infrastructure where possible. For instance, you could change
> the .run format into .ini format and use configparser, or you could use the
> shlex module for parsing.
Regarding ini files, I was thinking about using a kind of extended/simplified
.service format. Systemd being systemd, it would cover all cases, and it's got
some documentation already.

The problem is that there are pretty long shell scripts that do not fit well
within .ini files. Examples: sunxi-mali, ti-gfx, am33-cm3 and maybe some others.
That's why I tried to keep .run files looking somewhat like shell scripts.

Well, that and I don't like systemd documentation all that much.

> This bypass syntax is hard to understand IMHO. Can you find something
> simpler/clearer and more verbose? If you would switch to ini format, it could be
> something like
> 
> [initscripts]
> cp = package/foo/S10start
>      /etc/init.d/S10start
> 
> [systemd]
> cp-ln = package/foo/foo.service
>         /usr/lib/systemd/system/foo.service
>         /etc/systemd/system/multi-user.target.wants

The primary goal of the whole changeset is to separate package-specific
and init-system-specific things, moving the latter out of packages,
so that a change in systemd directory layout (for instance) would not require
patching all the packages.

This is why I try to specify startup ordering (priority 10 or multi-user.target
in this case) indirectly, and why I try to avoid explicit paths.
I see this as a part of init config format, not a part of service configuration.

The package does not need to know where to put those and how to name them.
All it needs is to have its process started, at some point.

[Buildroot] [RFC 01/11] common service startup files

[Arnout Vandecappelle]

On 22/03/15 13:39, Alex Suykov wrote:
>> >  - The script should only handle cases that are actually used. So start with
>> > something extremely simple that covers a few interesting cases, and leave the
>> > others to use the old initscripts system.
> Why I was trying handle as many packages as possible: I'm pretty sure something
> like 80% of them will work immediately with only the basic support,
> but it would be really bad to find a show-stopper among the remaining 20%.
> Something that would require changing the way the whole thing works.
> 
> Perhaps it's ok to drop bad cases for now, leave them as initscripts-only
> and expect the users do something about the unsupported cases themselves.

 I think it's important that there is always a fallback with custom stuff. And
for me, it's perfectly OK if the infrastructure handles only 50% of the cases
and the other 50% have to be custom.


 Regards,
 Arnout


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [RFC 01/11] common service startup files

[Arnout Vandecappelle]

On 22/03/15 13:39, Alex Suykov wrote:
> The problem is that there are pretty long shell scripts that do not fit well
> within .ini files. Examples: sunxi-mali, ti-gfx, am33-cm3 and maybe some others.
> That's why I tried to keep .run files looking somewhat like shell scripts.

 Why would it not fit in an ini file? You just have to add some indentation:


startscript =
  echo 1 > /sys/devices/...
  bla bla bla
  and we're done



 Regards,
 Arnout

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [RFC 01/11] common service startup files

[Yann E. MORIN]

Alex, All,

On 2015-03-21 20:27 +0200, Alex Suykov spake thusly:
> This patch introduces support infrastructure for per-project .run files.
> 
> Any .run files in package directory are picked up by pkg-generic code,
> fed to support/init/install-run script which converts them into either
> sysv initscripts or systemd unit files and installs those into predefined
> locations. The package itself provides only the .run files, no code
> in package .mk file is necessary.
> 
> Packages that install some of their services conditionally may set
> $(PKG)_INIT = list of run files to install. As a side effect, this
> removes the need to handle S99foo S99bar vs foo.service bar.service
> with the package .mk file.

I think this shoule be always set in the .mk, even if not conditional,
and that pkg-generic only handles what's been explicitly declared by the
package.

> Packages that need to substititue some values within their service
> files may set $(PKG)_INIT = KEY:value KEY:value ..., which will be
> handled by install-run. The substitutions will be applied to sysv
> and systemd files.

What we previously discussed was something like the following (for
sysv-init):

    /etc/init.d/S??foo  <- startup script for package 'foo'

    /etc/defaults/foo   <- contains default configuration of package
                           'foo', as installed by Buildroot.

    /etc/config/foo     <- optional file, contains the user-supplied
                           override configuration for package 'foo'

Then, the "startup script" would read those files, starting with the one
in /etc/defaults/ and then, if present, the one in /etc/config/ . This
would allow /etc to be read-only, and /etc/config to be a read-write
mountpoint.

Those files would be provided by the packages, like is done for now.

And similarly for systemd, except a lot of packages are now providing
systemd unit files, and there is no reason we would provide our own
instead, since that would probably bit-rot in the long term, so we
probably would prefer to use whatever upstream provides.

So, what we really care about are sysv initscripts, since they are in a
very bad shape:
  - upstream-provided scripts are too ugly, and often tied to a
    particular distro;
  - our own scripts are mot very clean either, for some have been around
    for a looong time.

> In case particular init system needs hand-crafted startup files
> or cannot handle certain services at all, install-run may be told
> so using special lines in the .run file.

I'm afraid this would be abused in the long run, becasue .run syntax is
too specific to Buildroot.

> This patch does not remove existing BR2_INIT_{SYSV,SYSTEMD} hooks.
> Each package should either provide .run files or define hooks, but not both.

Right.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [RFC 01/11] common service startup files

[Arnout Vandecappelle]

On 22/03/15 12:45, Yann E. MORIN wrote:
> Alex, All,
> 
> On 2015-03-21 20:27 +0200, Alex Suykov spake thusly:
>> This patch introduces support infrastructure for per-project .run files.
>>
>> Any .run files in package directory are picked up by pkg-generic code,
>> fed to support/init/install-run script which converts them into either
>> sysv initscripts or systemd unit files and installs those into predefined
>> locations. The package itself provides only the .run files, no code
>> in package .mk file is necessary.
>>
>> Packages that install some of their services conditionally may set
>> $(PKG)_INIT = list of run files to install. As a side effect, this
>> removes the need to handle S99foo S99bar vs foo.service bar.service
>> with the package .mk file.
> 
> I think this shoule be always set in the .mk, even if not conditional,
> and that pkg-generic only handles what's been explicitly declared by the
> package.

 Well, that's not how we do it for .hash and .patch...

 I would rather remove the option of setting _INIT from the package.mk file, if
the goal is to simplify things.


>> Packages that need to substititue some values within their service
>> files may set $(PKG)_INIT = KEY:value KEY:value ..., which will be
>> handled by install-run. The substitutions will be applied to sysv
>> and systemd files.
> 
> What we previously discussed was something like the following (for
> sysv-init):

 We discussed it but I didn't agree :-)

 I want to keep things simple, and any customisation should go through fs-overlay.

> 
>     /etc/init.d/S??foo  <- startup script for package 'foo'
> 
>     /etc/defaults/foo   <- contains default configuration of package
>                            'foo', as installed by Buildroot.
> 
>     /etc/config/foo     <- optional file, contains the user-supplied
>                            override configuration for package 'foo'
> 
> Then, the "startup script" would read those files, starting with the one
> in /etc/defaults/ and then, if present, the one in /etc/config/ . This
> would allow /etc to be read-only, and /etc/config to be a read-write
> mountpoint.

 Sounds nice, but you'd still need something like uci to manage /etc/config, and
then it probably wouldn't even work with uci...

> 
> Those files would be provided by the packages, like is done for now.
> 
> And similarly for systemd, except a lot of packages are now providing
> systemd unit files, and there is no reason we would provide our own
> instead, since that would probably bit-rot in the long term, so we
> probably would prefer to use whatever upstream provides.

 I totally agree with this: if a package provides a systemd unit file, it should
be used and we shouldn't touch it.

 However, I believe that a package will normally just install the unit file and
not add the symlink to actually start it, so that's something we still have to
take care of.

> 
> So, what we really care about are sysv initscripts, since they are in a
> very bad shape:
>   - upstream-provided scripts are too ugly, and often tied to a
>     particular distro;
>   - our own scripts are mot very clean either, for some have been around
>     for a looong time.

 Agreed.

> 
>> In case particular init system needs hand-crafted startup files
>> or cannot handle certain services at all, install-run may be told
>> so using special lines in the .run file.
> 
> I'm afraid this would be abused in the long run, becasue .run syntax is
> too specific to Buildroot.

 I'm not sure what you mean with abuse. For downstream packages, we don't really
care if they're using custom scripts or .run, and for patches to buildroot we'll
make sure that they use .run if possible.


 Regards,
 Arnout


> 
>> This patch does not remove existing BR2_INIT_{SYSV,SYSTEMD} hooks.
>> Each package should either provide .run files or define hooks, but not both.
> 
> Right.
> 
> Regards,
> Yann E. MORIN.
> 


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [RFC 02/11] per-package .users and .files lists

[Alex Suykov]

Following .run and .hash files, move package-specific users and
file permissions from .mk strings into standalone files, to be
picked up by pkg-generic code.
No additional code in package .mk is necessary.

This change does not simplify things much at present, but it
removes inter-package dependency via make variables.
Without this patch, reading all enabled package/*/*.mk files from
a single Makefile was necessary to build user and permission lists.
With this patch, the data is stored outside of make, possibly
allowing one make invocation per package .mk file in the future.

To keep installed per-package .users and .files, a new directory
is introduced: output/parts.
The directory is meant for stuff that is installed during build stage
but does not belong in the target/ directory, at least not without
some processing.
---
 Makefile                   |  1 +
 fs/common.mk               | 21 +++++++++++++--------
 package/pkg-generic.mk     |  2 ++
 support/init/install-parts |  5 +++++
 4 files changed, 21 insertions(+), 8 deletions(-)
 create mode 100755 support/init/install-parts

diff --git a/Makefile b/Makefile
index 33a89b1..98f67d3 100644
--- a/Makefile
+++ b/Makefile
@@ -156,6 +156,7 @@ endif
 BR_GRAPH_OUT := $(or $(BR2_GRAPH_OUT),pdf)
 
 BUILD_DIR := $(BASE_DIR)/build
+PARTS_DIR := $(BASE_DIR)/parts
 BINARIES_DIR := $(BASE_DIR)/images
 TARGET_DIR := $(BASE_DIR)/target
 # initial definition so that 'make clean' works for most users, even without
diff --git a/fs/common.mk b/fs/common.mk
index 5d07f00..e60c82f 100644
--- a/fs/common.mk
+++ b/fs/common.mk
@@ -28,12 +28,14 @@
 # BR2_TARGET_ROOTFS_$(FSTYPE)_LZMA exist and are enabled, then the
 # macro will automatically generate a compressed filesystem image.
 
-FAKEROOT_SCRIPT = $(BUILD_DIR)/_fakeroot.fs
-FULL_DEVICE_TABLE = $(BUILD_DIR)/_device_table.txt
+FAKEROOT_SCRIPT = $(PARTS_DIR)/fakeroot.fs
+FULL_DEVICE_TABLE = $(PARTS_DIR)/device_table.txt
 ROOTFS_DEVICE_TABLES = $(call qstrip,$(BR2_ROOTFS_DEVICE_TABLE) \
 	$(BR2_ROOTFS_STATIC_DEVICE_TABLE))
-USERS_TABLE = $(BUILD_DIR)/_users_table.txt
+USERS_TABLE = $(PARTS_DIR)/users_table.txt
 ROOTFS_USERS_TABLES = $(call qstrip,$(BR2_ROOTFS_USERS_TABLES))
+PARTS_LIST_USERS = $(wildcard $(PARTS_DIR)/users/*.users)
+PARTS_LIST_FILES = $(wildcard $(PARTS_DIR)/files/*.files)
 
 # Since this function will be called from within an $(eval ...)
 # all variable references except the arguments must be $$-quoted.
@@ -41,7 +43,7 @@ define ROOTFS_TARGET_INTERNAL
 
 # extra deps
 ROOTFS_$(2)_DEPENDENCIES += host-fakeroot host-makedevs \
-	$$(if $$(PACKAGES_USERS),host-mkpasswd)
+	$$(if $$(PARTS_LIST_USERS),host-mkpasswd)
 
 ifeq ($$(BR2_TARGET_ROOTFS_$(2)_GZIP),y)
 ROOTFS_$(2)_COMPRESS_EXT = .gz
@@ -77,16 +79,19 @@ $$(BINARIES_DIR)/rootfs.$(1): target-finalize $$(ROOTFS_$(2)_DEPENDENCIES)
 	echo "chown -h -R 0:0 $$(TARGET_DIR)" >> $$(FAKEROOT_SCRIPT)
 ifneq ($$(ROOTFS_DEVICE_TABLES),)
 	cat $$(ROOTFS_DEVICE_TABLES) > $$(FULL_DEVICE_TABLE)
-ifeq ($$(BR2_ROOTFS_DEVICE_CREATION_STATIC),y)
-	printf '$$(subst $$(sep),\n,$$(PACKAGES_DEVICES_TABLE))' >> $$(FULL_DEVICE_TABLE)
+ifdef PARTS_LIST_FILES
+	cat $$(PARTS_LIST_FILES) \
+		$$(if $$(BR2_ROOTFS_DEVICE_CREATION_STATIC),,| grep -v ^/dev) \
+			>> $$(FULL_DEVICE_TABLE)
 endif
-	printf '$$(subst $$(sep),\n,$$(PACKAGES_PERMISSIONS_TABLE))' >> $$(FULL_DEVICE_TABLE)
 	echo "$$(HOST_DIR)/usr/bin/makedevs -d $$(FULL_DEVICE_TABLE) $$(TARGET_DIR)" >> $$(FAKEROOT_SCRIPT)
 endif
 ifneq ($$(ROOTFS_USERS_TABLES),)
 	cat $$(ROOTFS_USERS_TABLES) >> $$(USERS_TABLE)
 endif
-	printf '$$(subst $$(sep),\n,$$(PACKAGES_USERS))' >> $$(USERS_TABLE)
+ifdef PARTS_LIST_USERS
+	cat $$(PARTS_LIST_USERS) >> $$(USERS_TABLE)
+endif
 	PATH=$$(BR_PATH) $$(TOPDIR)/support/scripts/mkusers $$(USERS_TABLE) $$(TARGET_DIR) >> $$(FAKEROOT_SCRIPT)
 	echo "$$(ROOTFS_$(2)_CMD)" >> $$(FAKEROOT_SCRIPT)
 	chmod a+x $$(FAKEROOT_SCRIPT)
diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
index fe7de8a..009b116 100644
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -774,6 +774,8 @@ ifndef $(2)_INSTALL_INIT
 define $(2)_INSTALL_INIT
 	$$(call ifargs,support/init/install-run $$(BR2_INIT) \
 		$$($$(PKG)_INIT_SUBST),$$(addprefix package/$1/,$$($$(PKG)_INIT)))
+	$$(call ifargs,support/init/install-parts users,$$(wildcard package/$1/*.users))
+	$$(call ifargs,support/init/install-parts files,$$(wildcard package/$1/*.files))
 	$$(call ifargs,support/init/install-tmpconf $$(BR2_INIT),$$(wildcard package/$1/*.tmp.conf))
 endef
 endif
diff --git a/support/init/install-parts b/support/init/install-parts
new file mode 100755
index 0000000..f5305fd
--- /dev/null
+++ b/support/init/install-parts
@@ -0,0 +1,5 @@
+#!/bin/sh
+
+part="$1"; shift; test -z "$part" && exit 1
+mkdir -p output/parts/$part/
+cp "$@" output/parts/$part/
-- 
2.0.3

[Buildroot] [RFC 02/11] per-package .users and .files lists

[Arnout Vandecappelle]

On 21/03/15 19:27, Alex Suykov wrote:
> This change does not simplify things much at present, but it
> removes inter-package dependency via make variables.
> Without this patch, reading all enabled package/*/*.mk files from
> a single Makefile was necessary to build user and permission lists.

 But we're not planning to change that... Running a separate make per package
sounds like a terribly bad idea...

> With this patch, the data is stored outside of make, possibly
> allowing one make invocation per package .mk file in the future.

 So, is there any reason left to move USERS and PERMS to a file?

 It has one big disadvantage: it spreads out the package logic over different
files. We currently only have a separate file for the hashes, and that was
probably a bad idea to begin with (as witnessed by the many version bumps that
forget to update the hash file).


 Regards,
 Arnout
-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [RFC 02/11] per-package .users and .files lists

[Alex Suykov]

Sun, Mar 22, 2015 at 03:35:09PM +0100, Arnout Vandecappelle wrote:

> > With this patch, the data is stored outside of make, possibly
> > allowing one make invocation per package .mk file in the future.
> 
>  So, is there any reason left to move USERS and PERMS to a file?

No, none at present, so I'm dropping this idea completely.

> It has one big disadvantage: it spreads out the package logic over different
> files. We currently only have a separate file for the hashes, and that was
> probably a bad idea to begin with (as witnessed by the many version bumps that
> forget to update the hash file).

Ok, that was for the large part to match current .hash handling.
I wonder if it makes sense to move hashes into .mk files then, Arch-style.

[Buildroot] [RFC 03/11] init/finalize script

[Alex Suykov]

The patch moves the code that handles non-package parts of init
configuration out of packages and out of skeleton, gathering it
in a dedicated script.

For sysv init, the script writes /etc/inittab, making filesystems
are mounted, udev is handled somehow, syslog is started and network
is brought up. For systemd, the script handles target symlinks
and aliases.

The idea is that options the configure the system as a whole should
be handled outside of individual packages.

So instead of letting eudev to install itself as the udev handler,
or maybe letting busybox install mdev, it is decided later and
outside of either eudev or busybox.

As a side effect, this patch makes sure sysv-specific files are not
installed at all if systemd is chosen for the init.
---
 package/busybox/busybox.mk            |   8 --
 package/busybox/mdev.conf             |  35 ------
 package/eudev/S10udev                 |  50 --------
 package/eudev/eudev.mk                |   4 -
 support/init/finalize                 | 207 ++++++++++++++++++++++++++++++++++
 support/init/init.d/S10mdev           |  21 ++++
 support/init/init.d/S10udev           |  50 ++++++++
 support/init/init.d/rcK               |  26 +++++
 support/init/init.d/rcS               |  26 +++++
 support/init/mdev.conf                |  35 ++++++
 support/init/sysv.fini                |   7 ++
 support/init/sysv.init                |  14 +++
 system/skeleton/etc/init.d/S20urandom |  54 ---------
 system/skeleton/etc/init.d/S40network |  28 -----
 system/skeleton/etc/init.d/rcK        |  27 -----
 system/skeleton/etc/init.d/rcS        |  27 -----
 system/skeleton/etc/inittab           |  35 ------
 system/system.mk                      |  32 +-----
 18 files changed, 390 insertions(+), 296 deletions(-)
 delete mode 100644 package/busybox/mdev.conf
 delete mode 100755 package/eudev/S10udev
 create mode 100755 support/init/finalize
 create mode 100755 support/init/init.d/S10mdev
 create mode 100755 support/init/init.d/S10udev
 create mode 100755 support/init/init.d/rcK
 create mode 100755 support/init/init.d/rcS
 create mode 100644 support/init/mdev.conf
 create mode 100644 support/init/sysv.fini
 create mode 100644 support/init/sysv.init
 delete mode 100755 system/skeleton/etc/init.d/S20urandom
 delete mode 100755 system/skeleton/etc/init.d/S40network
 delete mode 100755 system/skeleton/etc/init.d/rcK
 delete mode 100755 system/skeleton/etc/init.d/rcS
 delete mode 100644 system/skeleton/etc/inittab

diff --git a/package/busybox/busybox.mk b/package/busybox/busybox.mk
index a3ac7e7..90f7293 100644
--- a/package/busybox/busybox.mk
+++ b/package/busybox/busybox.mk
@@ -56,14 +56,6 @@ endef
 
 # If mdev will be used for device creation enable it and copy S10mdev to /etc/init.d
 ifeq ($(BR2_ROOTFS_DEVICE_CREATION_DYNAMIC_MDEV),y)
-define BUSYBOX_INSTALL_MDEV_SCRIPT
-	$(INSTALL) -D -m 0755 package/busybox/S10mdev \
-		$(TARGET_DIR)/etc/init.d/S10mdev
-endef
-define BUSYBOX_INSTALL_MDEV_CONF
-	$(INSTALL) -D -m 0644 package/busybox/mdev.conf \
-		$(TARGET_DIR)/etc/mdev.conf
-endef
 define BUSYBOX_SET_MDEV
 	$(call KCONFIG_ENABLE_OPT,CONFIG_MDEV,$(BUSYBOX_BUILD_CONFIG))
 	$(call KCONFIG_ENABLE_OPT,CONFIG_FEATURE_MDEV_CONF,$(BUSYBOX_BUILD_CONFIG))
diff --git a/package/busybox/mdev.conf b/package/busybox/mdev.conf
deleted file mode 100644
index 247c0ed..0000000
--- a/package/busybox/mdev.conf
+++ /dev/null
@@ -1,35 +0,0 @@
-# null may already exist; therefore ownership has to be changed with command
-null		root:root 666 @chmod 666 $MDEV
-zero		root:root 666
-full		root:root 666
-random		root:root 444
-urandom		root:root 444
-hwrandom	root:root 444
-grsec		root:root 660
-
-kmem		root:root 640
-mem		root:root 640
-port		root:root 640
-# console may already exist; therefore ownership has to be changed with command
-console		root:tty 600 @chmod 600 $MDEV
-ptmx		root:tty 666
-pty.*		root:tty 660
-
-# Typical devices
-tty		root:tty 666
-tty[0-9]*	root:tty 660
-vcsa*[0-9]*	root:tty 660
-ttyS[0-9]*	root:root 660
-
-# alsa sound devices
-pcm.*		root:audio 660 =snd/
-control.*	root:audio 660 =snd/
-midi.*		root:audio 660 =snd/
-seq		root:audio 660 =snd/
-timer		root:audio 660 =snd/
-
-# input stuff
-event[0-9]+	root:root 640 =input/
-mice		root:root 640 =input/
-mouse[0-9]	root:root 640 =input/
-ts[0-9]		root:root 600 =input/
diff --git a/package/eudev/S10udev b/package/eudev/S10udev
deleted file mode 100755
index 8382bec..0000000
--- a/package/eudev/S10udev
+++ /dev/null
@@ -1,50 +0,0 @@
-#!/bin/sh
-#
-# udev	This is a minimal non-LSB version of a UDEV startup script.  It
-#	was derived by stripping down the udev-058 LSB version for use
-#	with buildroot on embedded hardware using Linux 2.6.34+ kernels.
-#
-#	You may need to customize this for your system's resource limits
-#	(including startup time!) and administration.  For example, if
-#	your early userspace has a custom initramfs or initrd you might
-#	need /dev much earlier; or without hotpluggable busses (like USB,
-#	PCMCIA, MMC/SD, and so on) your /dev might be static after boot.
-#
-#	This script assumes your system boots right into the eventual root
-#	filesystem, and that init runs this udev script before any programs
-#	needing more device nodes than the bare-bones set -- /dev/console,
-#	/dev/zero, /dev/null -- that's needed to boot and run this script.
-#
-
-# Check for missing binaries
-UDEV_BIN=/sbin/udevd
-test -x $UDEV_BIN || exit 5
-
-# Check for config file and read it
-UDEV_CONFIG=/etc/udev/udev.conf
-test -r $UDEV_CONFIG || exit 6
-. $UDEV_CONFIG
-
-case "$1" in
-    start)
-        printf "Populating ${udev_root:-/dev} using udev: "
-        printf '\000\000\000\000' > /proc/sys/kernel/hotplug
-        $UDEV_BIN -d || (echo "FAIL" && exit 1)
-        udevadm trigger --type=subsystems --action=add
-        udevadm trigger --type=devices --action=add
-        udevadm settle --timeout=30 || echo "udevadm settle failed"
-        echo "done"
-        ;;
-    stop)
-        # Stop execution of events
-        udevadm control --stop-exec-queue
-        killall udevd
-        ;;
-    *)
-        echo "Usage: $0 {start|stop}"
-        exit 1
-        ;;
-esac
-
-
-exit 0
diff --git a/package/eudev/eudev.mk b/package/eudev/eudev.mk
index f5159a2..dab519e 100644
--- a/package/eudev/eudev.mk
+++ b/package/eudev/eudev.mk
@@ -40,10 +40,6 @@ else
 EUDEV_CONF_OPTS += --disable-gudev
 endif
 
-define EUDEV_INSTALL_INIT_SYSV
-	$(INSTALL) -m 0755 package/eudev/S10udev $(TARGET_DIR)/etc/init.d/S10udev
-endef
-
 # Required by default rules for input devices
 define EUDEV_USERS
 	- - input -1 * - - - Input device group
diff --git a/support/init/finalize b/support/init/finalize
new file mode 100755
index 0000000..fada2be
--- /dev/null
+++ b/support/init/finalize
@@ -0,0 +1,207 @@
+#!/usr/bin/env python
+
+from re import match, sub
+from sys import stderr, argv
+from os import listdir, lstat, symlink, unlink
+from os.path import islink, isdir, dirname
+from os import mkdir, makedirs, fstat, fchmod
+from os import O_RDONLY, O_WRONLY, O_CREAT, O_TRUNC
+from os import open as osopen
+from os import read as osread
+from os import write as oswrite
+
+def die(message, *args):
+    stderr.write((message+"\n") % tuple(args))
+    exit(-1)
+
+class Br2:
+    def __init__(self, dotconfig):
+        with open(dotconfig) as f:
+            for l in f:
+                m = match(r'^BR2_(.*)=(.*)', l)
+                if not m: continue
+                setattr(self, m.group(1), self.cfgval(m.group(2)))
+    def cfgval(self, x):
+        if x == 'y':
+            return True
+        elif x == 'n':
+            return False
+        else:
+            return sub(r'^"(.*)"', r'\1', x)
+
+    def __getattr__(self, k):
+        return self.__dict__[k] if k in self.__dict__ else None
+
+def grepval(filename, key):
+    patt = key + '='
+    plen = len(patt)
+    with open(filename) as f:
+        for l in f:
+            if l.startswith(patt):
+                return l[plen:]
+    return None
+
+def lnsf(target, link):
+    try:
+        if islink(link):
+            unlink(link)
+    except OSError:
+        pass
+
+    symlink(target, link)
+
+def writeto(path):
+    global output
+    output = open(path, 'w')
+
+def pipefile(path):
+    global output
+    with open(path) as f:
+        for l in f:
+            output.write(l)
+
+def copyfile(src, dst):
+    f = osopen(src, O_RDONLY)
+    s = fstat(f)
+    if not isdir(dirname(dst)): makedirs(dirname(dst))
+    o = osopen(dst, O_WRONLY | O_CREAT | O_TRUNC)
+    fchmod(o, s.st_mode)
+    buf = osread(f, s.st_size)
+    oswrite(o, buf)
+
+# write
+def w(fmt, *subst):
+    global output
+    output.write((fmt+"\n") % subst)
+
+# write newline
+def wnl(): w("")
+
+# ------------------------------------------------------------------------------
+
+# Common code for sysv-style systems
+
+def sysv_mount():
+    global BR2
+    w('# Startup the system')
+    w('null::sysinit:/bin/mount -t proc proc /proc')
+    if BR2.TARGET_GENERIC_REMOUNT_ROOTFS_RW:
+        w('null::sysinit:/bin/mount -o remount,rw /')
+    w('null::sysinit:/bin/mkdir -p /dev/pts')
+    w('null::sysinit:/bin/mkdir -p /dev/shm')
+    w('null::sysinit:/bin/mount -a')
+
+def sysv_net():
+    w('null::sysinit:/bin/hostname -F /etc/hostname')
+    w('null::sysinit:/sbin/ifup -a')
+
+def sysv_syslog():
+    global BR2
+    if BR2.PACKAGE_BUSYBOX or BR2.PACKAGE_RSYSLOG:
+        wnl()
+        w("# Start syslog before any daemons")
+        w("null::respawn:/usr/sbin/syslogd -n")
+    if BR2.PACKAGE_BUSYBOX:
+        w("null::respawn:/usr/sbin/klogd -n")
+
+def sysv_getty():
+    global BR2
+    if not BR2.TARGET_GENERIC_GETTY:
+        return
+    port = BR2.TARGET_GENERIC_GETTY_PORT
+    term = BR2.TARGET_GENERIC_GETTY_TERM
+    opts = BR2.TARGET_GENERIC_GETTY_OPTIONS
+    if port:
+        wnl()
+        w("# Start serial console")
+        w("%s::respawn:/sbin/getty -L %s %s %s", port, opts, port, term)
+
+# pri:name:rlvls:act:cmd -> pri
+def sysv_prio(line):
+    return line.split(':')[0]
+
+# pri:name:rlvls:act:cmd -> name:rlvls:act:cmd
+def sysv_line(line):
+    return ":".join(line.split(':')[1:])
+
+# read everything from path/*.line files into a single array
+def readlines(path):
+    lines = [ ]
+    if not isdir(path):
+        return lines
+    for de in listdir(path):
+        if not de.endswith('.line'): continue
+        with open(path + '/' + de) as f:
+            for l in f:
+                if l: lines.append(l)
+    return lines
+
+# ------------------------------------------------------------------------------
+
+# Initscripts need their static inittab written, and rc* files installed.
+# The only decisions to be made here are about system-wide services
+# with no .run files (syslog and such)
+def initscripts():
+    copyfile('support/init/init.d/rcS', 'output/target/etc/init.d/rcS')
+    copyfile('support/init/init.d/rcK', 'output/target/etc/init.d/rcK')
+
+    if BR2.ROOTFS_DEVICE_CREATION_DYNAMIC_EUDEV:
+        copyfile('support/init/init.d/S10udev', 'output/target/etc/init.d/S10udev')
+    elif BR2.ROOTFS_DEVICE_CREATION_DYNAMIC_MDEV:
+        copyfile('support/init/init.d/S10mdev', 'output/target/etc/init.d/S10udev')
+
+    # The stuff in sysv.init and sysv.fini should not be static actually.
+    # For now it just the current BR setup, but that's probably not right.
+    # At least swap and network depend heavily on target configuration.
+    writeto('output/target/etc/inittab')
+    pipefile('support/init/sysv.init')
+    wnl()
+
+    sysv_mount()
+    sysv_net()
+    sysv_syslog()
+    sysv_getty()
+    wnl()
+
+    w("# Run rc scripts on startup and shutdown")
+    w("null::sysinit:/etc/init.d/rcS")
+    w("null::shutdown:/etc/init.d/rcK")
+    wnl()
+
+    pipefile('support/init/sysv.fini')
+
+# ------------------------------------------------------------------------------
+
+# Installed service files must be linked to relevant *.wants directories.
+# Other than that, there is nothing to do with systemd configuration.
+def systemd():
+    sd = "output/target/usr/lib/systemd/system"
+    for de in listdir(sd):
+        if not de.endswith(".service"): continue
+        path = sd+'/'+de
+        wantedby = grepval(path, 'WantedBy')
+        aliases = grepval(path, 'Alias')
+        if wantedby:
+            for t in wantedby.split():
+                lnsf("../%s" % de, "%s/%s.wants/%s" % (sd, t, de))
+        if aliases:
+            for a in aliases.split():
+                lnsf(de, "%s.service" % a)
+
+# ------------------------------------------------------------------------------
+
+output = None
+inits = [ 'initscripts', 'systemd' ]
+BR2 = Br2('.config')
+
+if len(argv) > 1 and argv[1] == '-':
+    nopath = True
+    del argv[1]
+
+if len(argv) < 2:
+    die("Usage: finalize-init [-] init-system")
+
+if argv[1] in inits:
+    globals()[argv[1]]()
+else:
+    die("Unknown init system %s", argv[1])
diff --git a/support/init/init.d/S10mdev b/support/init/init.d/S10mdev
new file mode 100755
index 0000000..d386d42
--- /dev/null
+++ b/support/init/init.d/S10mdev
@@ -0,0 +1,21 @@
+#!/bin/sh
+#
+# Start mdev....
+#
+
+case "$1" in
+  start)
+	echo "Starting mdev..."
+	echo /sbin/mdev >/proc/sys/kernel/hotplug
+	/sbin/mdev -s
+	;;
+  stop)
+	;;
+  restart|reload)
+	;;
+  *)
+	echo "Usage: $0 {start|stop|restart}"
+	exit 1
+esac
+
+exit $?
diff --git a/support/init/init.d/S10udev b/support/init/init.d/S10udev
new file mode 100755
index 0000000..8382bec
--- /dev/null
+++ b/support/init/init.d/S10udev
@@ -0,0 +1,50 @@
+#!/bin/sh
+#
+# udev	This is a minimal non-LSB version of a UDEV startup script.  It
+#	was derived by stripping down the udev-058 LSB version for use
+#	with buildroot on embedded hardware using Linux 2.6.34+ kernels.
+#
+#	You may need to customize this for your system's resource limits
+#	(including startup time!) and administration.  For example, if
+#	your early userspace has a custom initramfs or initrd you might
+#	need /dev much earlier; or without hotpluggable busses (like USB,
+#	PCMCIA, MMC/SD, and so on) your /dev might be static after boot.
+#
+#	This script assumes your system boots right into the eventual root
+#	filesystem, and that init runs this udev script before any programs
+#	needing more device nodes than the bare-bones set -- /dev/console,
+#	/dev/zero, /dev/null -- that's needed to boot and run this script.
+#
+
+# Check for missing binaries
+UDEV_BIN=/sbin/udevd
+test -x $UDEV_BIN || exit 5
+
+# Check for config file and read it
+UDEV_CONFIG=/etc/udev/udev.conf
+test -r $UDEV_CONFIG || exit 6
+. $UDEV_CONFIG
+
+case "$1" in
+    start)
+        printf "Populating ${udev_root:-/dev} using udev: "
+        printf '\000\000\000\000' > /proc/sys/kernel/hotplug
+        $UDEV_BIN -d || (echo "FAIL" && exit 1)
+        udevadm trigger --type=subsystems --action=add
+        udevadm trigger --type=devices --action=add
+        udevadm settle --timeout=30 || echo "udevadm settle failed"
+        echo "done"
+        ;;
+    stop)
+        # Stop execution of events
+        udevadm control --stop-exec-queue
+        killall udevd
+        ;;
+    *)
+        echo "Usage: $0 {start|stop}"
+        exit 1
+        ;;
+esac
+
+
+exit 0
diff --git a/support/init/init.d/rcK b/support/init/init.d/rcK
new file mode 100755
index 0000000..b1fb8f3
--- /dev/null
+++ b/support/init/init.d/rcK
@@ -0,0 +1,26 @@
+#!/bin/sh
+
+
+# Stop all init scripts in /etc/init.d
+# executing them in reversed numerical order.
+#
+for i in $(ls -r /etc/init.d/S??*) ;do
+
+     # Ignore dangling symlinks (if any).
+     [ ! -f "$i" ] && continue
+
+     case "$i" in
+	*.sh)
+	    # Source shell script for speed.
+	    (
+		trap - INT QUIT TSTP
+		set stop
+		. $i
+	    )
+	    ;;
+	*)
+	    # No sh extension, so fork subprocess.
+	    $i stop
+	    ;;
+    esac
+done
diff --git a/support/init/init.d/rcS b/support/init/init.d/rcS
new file mode 100755
index 0000000..bd11d5a
--- /dev/null
+++ b/support/init/init.d/rcS
@@ -0,0 +1,26 @@
+#!/bin/sh
+
+
+# Start all init scripts in /etc/init.d
+# executing them in numerical order.
+#
+for i in /etc/init.d/S??* ;do
+
+     # Ignore dangling symlinks (if any).
+     [ ! -f "$i" ] && continue
+
+     case "$i" in
+	*.sh)
+	    # Source shell script for speed.
+	    (
+		trap - INT QUIT TSTP
+		set start
+		. $i
+	    )
+	    ;;
+	*)
+	    # No sh extension, so fork subprocess.
+	    $i start
+	    ;;
+    esac
+done
diff --git a/support/init/mdev.conf b/support/init/mdev.conf
new file mode 100644
index 0000000..247c0ed
--- /dev/null
+++ b/support/init/mdev.conf
@@ -0,0 +1,35 @@
+# null may already exist; therefore ownership has to be changed with command
+null		root:root 666 @chmod 666 $MDEV
+zero		root:root 666
+full		root:root 666
+random		root:root 444
+urandom		root:root 444
+hwrandom	root:root 444
+grsec		root:root 660
+
+kmem		root:root 640
+mem		root:root 640
+port		root:root 640
+# console may already exist; therefore ownership has to be changed with command
+console		root:tty 600 @chmod 600 $MDEV
+ptmx		root:tty 666
+pty.*		root:tty 660
+
+# Typical devices
+tty		root:tty 666
+tty[0-9]*	root:tty 660
+vcsa*[0-9]*	root:tty 660
+ttyS[0-9]*	root:root 660
+
+# alsa sound devices
+pcm.*		root:audio 660 =snd/
+control.*	root:audio 660 =snd/
+midi.*		root:audio 660 =snd/
+seq		root:audio 660 =snd/
+timer		root:audio 660 =snd/
+
+# input stuff
+event[0-9]+	root:root 640 =input/
+mice		root:root 640 =input/
+mouse[0-9]	root:root 640 =input/
+ts[0-9]		root:root 600 =input/
diff --git a/support/init/sysv.fini b/support/init/sysv.fini
new file mode 100644
index 0000000..e68f97e
--- /dev/null
+++ b/support/init/sysv.fini
@@ -0,0 +1,7 @@
+# Stuff to do for the 3-finger salute
+null::ctrlaltdel:/sbin/reboot
+
+# Stuff to do before rebooting
+null::shutdown:/sbin/ifdown -a
+null::shutdown:/sbin/swapoff -a
+null::shutdown:/bin/umount -a -r
diff --git a/support/init/sysv.init b/support/init/sysv.init
new file mode 100644
index 0000000..b56e3c3
--- /dev/null
+++ b/support/init/sysv.init
@@ -0,0 +1,14 @@
+# /etc/inittab
+#
+# Copyright (C) 2001 Erik Andersen <andersen@codepoet.org>
+#
+# Note: BusyBox init doesn't support runlevels.  The runlevels field is
+# completely ignored by BusyBox init. If you want runlevels, use
+# sysvinit.
+#
+# Format for each entry: <id>:<runlevels>:<action>:<process>
+#
+# id        == tty to run on, or empty for /dev/console
+# runlevels == ignored
+# action    == one of sysinit, respawn, askfirst, wait, and once
+# process   == program to run
diff --git a/system/skeleton/etc/init.d/S20urandom b/system/skeleton/etc/init.d/S20urandom
deleted file mode 100755
index f73cea5..0000000
--- a/system/skeleton/etc/init.d/S20urandom
+++ /dev/null
@@ -1,54 +0,0 @@
-#! /bin/sh
-#
-# urandom	This script saves the random seed between reboots.
-#		It is called from the boot, halt and reboot scripts.
-#
-# Version:	@(#)urandom  1.33  22-Jun-1998  miquels at cistron.nl
-#
-
-[ -c /dev/urandom ] || exit 0
-#. /etc/default/rcS
-
-case "$1" in
-	start|"")
-		if [ "$VERBOSE" != no ]
-		then
-			echo -n "Initializing random number generator... "
-		fi
-		# Load and then save 512 bytes,
-		# which is the size of the entropy pool
-		if [ -f /etc/random-seed ]
-		then
-			cat /etc/random-seed >/dev/urandom
-		fi
-		# check for read only file system
-		if ! touch /etc/random-seed 2>/dev/null
-		then
-			echo "read-only file system detected...done"
-			exit
-		fi
-		rm -f /etc/random-seed
-		umask 077
-		dd if=/dev/urandom of=/etc/random-seed count=1 \
-			>/dev/null 2>&1 || echo "urandom start: failed."
-		umask 022
-		[ "$VERBOSE" != no ] && echo "done."
-		;;
-	stop)
-		if ! touch /etc/random-seed 2>/dev/null
-                then
-                        exit
-                fi
-		# Carry a random seed from shut-down to start-up;
-		# see documentation in linux/drivers/char/random.c
-		[ "$VERBOSE" != no ] && echo -n "Saving random seed... "
-		umask 077
-		dd if=/dev/urandom of=/etc/random-seed count=1 \
-			>/dev/null 2>&1 || echo "urandom stop: failed."
-		[ "$VERBOSE" != no ] && echo "done."
-		;;
-	*)
-		echo "Usage: urandom {start|stop}" >&2
-		exit 1
-		;;
-esac
diff --git a/system/skeleton/etc/init.d/S40network b/system/skeleton/etc/init.d/S40network
deleted file mode 100755
index bfdd491..0000000
--- a/system/skeleton/etc/init.d/S40network
+++ /dev/null
@@ -1,28 +0,0 @@
-#!/bin/sh
-#
-# Start the network....
-#
-
-# Debian ifupdown needs the /run/network lock directory
-mkdir -p /run/network
-
-case "$1" in
-  start)
- 	echo "Starting network..."
-	/sbin/ifup -a
-	;;
-  stop)
-	echo -n "Stopping network..."
-	/sbin/ifdown -a
-	;;
-  restart|reload)
-	"$0" stop
-	"$0" start
-	;;
-  *)
-	echo "Usage: $0 {start|stop|restart}"
-	exit 1
-esac
-
-exit $?
-
diff --git a/system/skeleton/etc/init.d/rcK b/system/skeleton/etc/init.d/rcK
deleted file mode 100755
index 59e9c54..0000000
--- a/system/skeleton/etc/init.d/rcK
+++ /dev/null
@@ -1,27 +0,0 @@
-#!/bin/sh
-
-
-# Stop all init scripts in /etc/init.d
-# executing them in reversed numerical order.
-#
-for i in $(ls -r /etc/init.d/S??*) ;do
-
-     # Ignore dangling symlinks (if any).
-     [ ! -f "$i" ] && continue
-
-     case "$i" in
-	*.sh)
-	    # Source shell script for speed.
-	    (
-		trap - INT QUIT TSTP
-		set stop
-		. $i
-	    )
-	    ;;
-	*)
-	    # No sh extension, so fork subprocess.
-	    $i stop
-	    ;;
-    esac
-done
-
diff --git a/system/skeleton/etc/init.d/rcS b/system/skeleton/etc/init.d/rcS
deleted file mode 100755
index de41153..0000000
--- a/system/skeleton/etc/init.d/rcS
+++ /dev/null
@@ -1,27 +0,0 @@
-#!/bin/sh
-
-
-# Start all init scripts in /etc/init.d
-# executing them in numerical order.
-#
-for i in /etc/init.d/S??* ;do
-
-     # Ignore dangling symlinks (if any).
-     [ ! -f "$i" ] && continue
-
-     case "$i" in
-	*.sh)
-	    # Source shell script for speed.
-	    (
-		trap - INT QUIT TSTP
-		set start
-		. $i
-	    )
-	    ;;
-	*)
-	    # No sh extension, so fork subprocess.
-	    $i start
-	    ;;
-    esac
-done
-
diff --git a/system/skeleton/etc/inittab b/system/skeleton/etc/inittab
deleted file mode 100644
index b1892c1..0000000
--- a/system/skeleton/etc/inittab
+++ /dev/null
@@ -1,35 +0,0 @@
-# /etc/inittab
-#
-# Copyright (C) 2001 Erik Andersen <andersen@codepoet.org>
-#
-# Note: BusyBox init doesn't support runlevels.  The runlevels field is
-# completely ignored by BusyBox init. If you want runlevels, use
-# sysvinit.
-#
-# Format for each entry: <id>:<runlevels>:<action>:<process>
-#
-# id        == tty to run on, or empty for /dev/console
-# runlevels == ignored
-# action    == one of sysinit, respawn, askfirst, wait, and once
-# process   == program to run
-
-# Startup the system
-null::sysinit:/bin/mount -t proc proc /proc
-null::sysinit:/bin/mount -o remount,rw /
-null::sysinit:/bin/mkdir -p /dev/pts
-null::sysinit:/bin/mkdir -p /dev/shm
-null::sysinit:/bin/mount -a
-null::sysinit:/bin/hostname -F /etc/hostname
-# now run any rc scripts
-::sysinit:/etc/init.d/rcS
-
-# Put a getty on the serial port
-#ttyS0::respawn:/sbin/getty -L ttyS0 115200 vt100 # GENERIC_SERIAL
-
-# Stuff to do for the 3-finger salute
-::ctrlaltdel:/sbin/reboot
-
-# Stuff to do before rebooting
-::shutdown:/etc/init.d/rcK
-::shutdown:/sbin/swapoff -a
-::shutdown:/bin/umount -a -r
diff --git a/system/system.mk b/system/system.mk
index 4a1eb4a..c5ce1d3 100644
--- a/system/system.mk
+++ b/system/system.mk
@@ -87,35 +87,11 @@ endef
 endif
 TARGET_FINALIZE_HOOKS += SYSTEM_BIN_SH
 
-ifeq ($(BR2_TARGET_GENERIC_GETTY),y)
-ifeq ($(BR2_PACKAGE_SYSVINIT),y)
-# In sysvinit inittab, the "id" must not be longer than 4 bytes, so we
-# skip the "tty" part and keep only the remaining.
-define SYSTEM_GETTY
-	$(SED) '/# GENERIC_SERIAL$$/s~^.*#~$(shell echo $(TARGET_GENERIC_GETTY_PORT) | tail -c+4)::respawn:/sbin/getty -L $(TARGET_GENERIC_GETTY_OPTIONS) $(TARGET_GENERIC_GETTY_PORT) $(TARGET_GENERIC_GETTY_BAUDRATE) $(TARGET_GENERIC_GETTY_TERM) #~' \
-		$(TARGET_DIR)/etc/inittab
-endef
-else
-# Add getty to busybox inittab
-define SYSTEM_GETTY
-	$(SED) '/# GENERIC_SERIAL$$/s~^.*#~$(TARGET_GENERIC_GETTY_PORT)::respawn:/sbin/getty -L $(TARGET_GENERIC_GETTY_OPTIONS) $(TARGET_GENERIC_GETTY_PORT) $(TARGET_GENERIC_GETTY_BAUDRATE) $(TARGET_GENERIC_GETTY_TERM) #~' \
-		$(TARGET_DIR)/etc/inittab
-endef
-endif
-TARGET_FINALIZE_HOOKS += SYSTEM_GETTY
-endif
-
-ifeq ($(BR2_TARGET_GENERIC_REMOUNT_ROOTFS_RW),y)
-# Find commented line, if any, and remove leading '#'s
-define SYSTEM_REMOUNT_RW
-	$(SED) '/^#.*-o remount,rw \/$$/s~^#\+~~' $(TARGET_DIR)/etc/inittab
-endef
-else
-# Find uncommented line, if any, and add a leading '#'
-define SYSTEM_REMOUNT_RW
-	$(SED) '/^[^#].*-o remount,rw \/$$/s~^~#~' $(TARGET_DIR)/etc/inittab
+ifneq ($(BR2_INIT),)
+define FINALIZE_INIT
+	support/init/finalize $(BR2_INIT)
 endef
+TARGET_FINALIZE_HOOKS += FINALIZE_INIT
 endif
-TARGET_FINALIZE_HOOKS += SYSTEM_REMOUNT_RW
 
 endif # BR2_ROOTFS_SKELETON_DEFAULT
-- 
2.0.3

[Buildroot] [RFC 04/11] help entries for Init system config menu

[Alex Suykov]

Primary focus is on (auto-)respawning and runtime control.
Background/foreground distinction is not mentioned,
since it is not likely to mean much for users making a choice here.
---
 system/Config.in | 58 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 58 insertions(+)

diff --git a/system/Config.in b/system/Config.in
index 9973cc2..c0ee345 100644
--- a/system/Config.in
+++ b/system/Config.in
@@ -75,15 +75,46 @@ config BR2_TARGET_GENERIC_PASSWD_METHOD
 choice
 	prompt "Init system"
 	default BR2_INIT_BUSYBOX
+	help
+	  Upon bootup, kernel spawns init and init is expected
+	  to spawn anything else.
+
+	  Buildroot can pre-configure the target filesystem
+	  so that everything necessary to start the system
+	  is done and all installed daemons are started.
+
+	  There are several possible configurations providing
+	  varying degrees of control over the spawned processes.
 
 config BR2_INIT_BUSYBOX
 	bool "BusyBox"
 	select BR2_PACKAGE_BUSYBOX
+	help
+	  Minimalistic init implementation from busybox
+	  with systemV-style init scripts.
+
+	  No runlevel support.
+	  Daemons can be stopped or restarted at runtime.
+	  Daemons are started in background using scripts
+	  from /etc/init.d, and controlled using pid files.
+	  Failed processes are not respawned.
 
 config BR2_INIT_SYSV
 	bool "systemV"
 	select BR2_PACKAGE_BUSYBOX_SHOW_OTHERS # sysvinit
 	select BR2_PACKAGE_SYSVINIT
+	help
+	  System V init, probably the best-known init
+	  implementation in Linux, with simplified SysV-style
+	  initscripts.
+
+	  Supports runlevels, but this particular configuration
+	  does not use them.
+
+	  Daemons can be stopped or restarted at runtime.
+	  Daemons are started in background using scripts
+	  from /etc/init.d, and controlled using pid files.
+	  Failed processes are not respawned.
 
 config BR2_INIT_SYSTEMD
 	bool "systemd"
@@ -98,6 +129,27 @@ config BR2_INIT_SYSTEMD
 	depends on !BR2_STATIC_LIBS
 	depends on BR2_TOOLCHAIN_HEADERS_AT_LEAST_3_7
 	select BR2_PACKAGE_SYSTEMD
+	help
+	  systemd is more that just an init implementation.
+	  It is a composite package handling most aspects
+	  of system initialization and daemon supervision.
+
+	  All daemons will be started in foreground mode,
+	  configured to be respawed on failure.
+
+	  Systemd uses arbitrary-named "targets" instead
+	  of systemV-style runlevels, and allows stopping and
+	  restarting daemons at runtime using systemctl command.
+
+	  Due its pervasiveness and extensive feature range,
+	  choosing systemd shapes the rest of the system.
+	  A systemd-based buildroot will differ little
+	  from any major systemd-based Linux distribution
+	  in pretty much any aspects of the boot process,
+	  initialization, runtime configuration and process
+	  supervision.
+
+	  Beware of its large size and numerous dependencies.
 
 comment 'systemd needs an (e)glibc toolchain, headers >= 3.7'
 	depends on !(BR2_TOOLCHAIN_USES_GLIBC \
@@ -105,6 +157,12 @@ comment 'systemd needs an (e)glibc toolchain, headers >= 3.7'
 
 config BR2_INIT_NONE
 	bool "None"
+	help
+	  Do not install any kind of init system.
+
+	  Make sure your initrd and/or root fs skeleton provide
+	  some executable for kernel to start, otherwise
+	  the system will panic at boot.
 
 endchoice
 
-- 
2.0.3

[Buildroot] [RFC 05/11] bare bb init configuration

[Alex Suykov]

This patch introduces a new init system: bb init without
system V initscripts.

Busybox init is quite capable of running foreground daemons,
with fast startup and auto-restarts but without the overhead
of systemd. It lacks telinit and cannot start or stop daemons
at runtime, but not every systems needs runtime control.

With the infrastructure mostly in place, implementing this
is straightforward. Per-package initlines are installed in
output/parts/ and init/finalize script picks them up to form
a complete inittab.

Whenever a daemon needs some pre-start initialization and/or
can not be started directly from inittab for other reasons,
a shell script is created in /etc/rc/.

To keep things organized, initialization scripts (scripts that
do not start a long-running foreground process) are put in
/etc/rc/sys/ while anything directly in /etc/rc/ is expected
to be used for :respawn: lines.
---
 package/Makefile.in      |  2 ++
 support/init/finalize    | 44 ++++++++++++++++++++++++-
 support/init/install-run | 83 ++++++++++++++++++++++++++++++++++++++++++++++--
 support/init/rc.sys/mdev |  4 +++
 support/init/rc.sys/udev |  8 +++++
 system/Config.in         | 16 ++++++++--
 6 files changed, 152 insertions(+), 5 deletions(-)
 create mode 100755 support/init/rc.sys/mdev
 create mode 100755 support/init/rc.sys/udev

diff --git a/package/Makefile.in b/package/Makefile.in
index 599fe7f..df3b3b7 100644
--- a/package/Makefile.in
+++ b/package/Makefile.in
@@ -412,6 +412,8 @@ ifdef BR2_INIT_SYSV
 BR2_INIT = initscripts
 else ifdef BR2_INIT_BUSYBOX
 BR2_INIT = initscripts
+else ifdef BR2_INIT_BUSYBOX_BARE
+BR2_INIT = inittab
 else ifdef BR2_INIT_SYSTEMD
 BR2_INIT = systemd
 else
diff --git a/support/init/finalize b/support/init/finalize
index fada2be..ba4f462 100755
--- a/support/init/finalize
+++ b/support/init/finalize
@@ -172,6 +172,48 @@ def initscripts():
 
 # ------------------------------------------------------------------------------
 
+# Inittab is almost the same as initscripts, except all services
+# are listed directly instead of running rcS and rcK
+def inittab():
+    writeto('output/target/etc/inittab')
+    pipefile('support/init/sysv.init')
+    wnl()
+
+    sysv_mount()
+    sysv_net()
+    # These are started *before* anything else, in particular before
+    # any system initialization, to catch possible messages and
+    # to give a chance to peek at the system state if something hangs.
+    sysv_syslog()
+    sysv_getty()
+    inittab_udev()
+
+    # This follows initscript idea of priority, with mixed
+    # w-type and s-type entries. Makes little sense actually,
+    # but the code is a bit shorter this way, and install-init
+    # already kinda splits w/s-type with its default prio settings.
+    lines = readlines('output/parts/inittab/')
+    lines = map(sysv_line, sorted(lines, key=sysv_prio))
+    if lines:
+        wnl()
+        for l in lines: w('%s', l.rstrip())
+
+    wnl()
+    pipefile('support/init/sysv.fini')
+
+def inittab_udev():
+    if BR2.ROOTFS_DEVICE_CREATION_DYNAMIC_EUDEV:
+        copyfile('support/init/rc.sys/udev', 'output/target/etc/rc/sys/udev')
+        w('null:2345:respawn:/sbin/udevd')
+        w('null:2345:wait:/etc/rc/sys/udev')
+    elif BR2.ROOTFS_DEVICE_CREATION_DYNAMIC_MDEV:
+        # mdev is not a deamon
+        copyfile('support/init/rc.sys/mdev', 'output/target/etc/rc/sys/mdev')
+        copyfile('support/init/mdev.conf', 'output/target/etc/mdev.conf')
+        w('null:2345:wait:/etc/rc/sys/mdev')
+
+# ------------------------------------------------------------------------------
+
 # Installed service files must be linked to relevant *.wants directories.
 # Other than that, there is nothing to do with systemd configuration.
 def systemd():
@@ -191,7 +233,7 @@ def systemd():
 # ------------------------------------------------------------------------------
 
 output = None
-inits = [ 'initscripts', 'systemd' ]
+inits = [ 'inittab', 'initscripts', 'systemd' ]
 BR2 = Br2('.config')
 
 if len(argv) > 1 and argv[1] == '-':
diff --git a/support/init/install-run b/support/init/install-run
index 24b79c7..5a06e63 100755
--- a/support/init/install-run
+++ b/support/init/install-run
@@ -23,7 +23,7 @@ from os import open as osopen
 from os import read as osread
 from os import write as oswrite
 
-inits = ['initscripts', 'systemd']
+inits = ['inittab', 'initscripts', 'systemd']
 output = None
 nopath = False
 
@@ -37,7 +37,7 @@ class Run:
         # accepting unadorned commands for pre/post groups.
 
         for k in ['description', 'user', 'group', 'umask',
-                  'pidfile', 'priority',
+                  'pidfile', 'priority', 'short',
                   'after', 'requires', 'wantedby',
                   'busname', 'conflicts', 'killwith', 'restart']:
             setattr(self, k, None)
@@ -219,6 +219,81 @@ def maybesu(r, cmd):
     else:
         return cmd
 
+# return either cmd, sh cmd or su cmd; the caller can *not* handle
+# shell commands, so anything unusual must be escaped with sh -c
+def maybesush(r, cmd):
+    # everything below assumes the caller uses execvp even with no sh
+    # which is ok for all init systemd that use su
+    shneeded = match(r'^.*[<>&|$"()]', cmd)
+
+    if r.user:
+        if shneeded:
+            return "su - %s -c '%s'" % (r.user, cmd)
+        else:
+            return "su - %s %s" % (r.user, cmd)
+    else:
+        if shneeded:
+            return "sh -c '%s'" % (r.user, cmd)
+        else:
+            return cmd
+
+# ------------------------------------------------------------------------------
+
+def inittab(r):
+    short = r.short if r.short else r.name
+    rlvls = '345' # BR does not use runlevels
+    writeto('output/parts/inittab/%s.line' % r.name, 0o644)
+    script = None
+    r.substcmds()
+
+    if r.umask:
+        r.pre.insert(0, "umask %s" % r.umask)
+
+    if r.exec:
+        action, cmd = 'respawn', r.exec
+    elif r.start:
+        action, cmd = 'wait', r.start
+    elif len(r.pre) == 1:
+        action, cmd = 'wait', r.pre[0]
+    else:
+        action, cmd = 'wait', None
+    if ((r.exec or r.start) and r.pre) or (len(r.pre) > 1):
+        sdir = 'rc' if r.start or r.exec else 'rc/sys'
+        script = '/etc/%s/%s' % (sdir, r.name)
+        cmd = script
+
+    if not script:
+        cmd = maybesush(r, cmd)
+
+    # let tty be null for all anything intalled from .run files
+    w("%02d:%s:%s:%s:%s", int(r.priority), 'null', rlvls, action, cmd)
+
+    if not script:
+        return
+
+    writeto('output/target%s' % script, 0o755)
+    w("#!/bin/sh")
+    if r.description:
+        w("# %s", r.description)
+    wnl
+
+    for c in r.pre:
+        # this may result in profoundly ugly code, but luckily
+        # the only service that uses it atm has exactly one line pre
+        w("%s", maybesu(r, c))
+
+    if r.exec:
+        w("exec %s", maybesu(r, r.exec))
+    elif r.start:
+        w("%s", maybesu(r, r.start))
+
+    # TODO: handle pre-post scripts
+    # This should result in *two* lines, not one, with the second
+    # line being probably :06: with relevant setup in finalize-init
+    # to make sysvinit switch to those runlevels on reboot/poweroff
+    # XXX: would that be a natural solution for sysv?
+    # XXX: bb init does not support runlevels
+
 # ------------------------------------------------------------------------------
 
 def initscripts(r):
@@ -381,6 +456,10 @@ def systemd(r):
 #     bypass inittab -
 
 bypasstable = {
+    'inittab': [
+        ('*.line',      'output/parts/inittab/*'),
+        ('sys:*',       'output/target/etc/rc/sys/*'),
+        ('*',           'output/target/etc/rc/*') ],
     'initscripts': [
         ('*:*.init',    'output/target/etc/init.d/S$1$2'),
         ('*.init',      'output/target/etc/init.d/S50$1'),
diff --git a/support/init/rc.sys/mdev b/support/init/rc.sys/mdev
new file mode 100755
index 0000000..dd66441
--- /dev/null
+++ b/support/init/rc.sys/mdev
@@ -0,0 +1,4 @@
+#!/bin/sh
+
+echo /sbin/mdev >/proc/sys/kernel/hotplug
+exec /sbin/mdev -s
diff --git a/support/init/rc.sys/udev b/support/init/rc.sys/udev
new file mode 100755
index 0000000..d5e40ea
--- /dev/null
+++ b/support/init/rc.sys/udev
@@ -0,0 +1,8 @@
+#!/bin/sh
+
+# udevd must be running by this point!
+
+printf '\000\000\000\000' > /proc/sys/kernel/hotplug
+udevadm trigger --type=subsystems --action=add
+udevadm trigger --type=devices --action=add
+udevadm settle --timeout=30
diff --git a/system/Config.in b/system/Config.in
index c0ee345..d25ad85 100644
--- a/system/Config.in
+++ b/system/Config.in
@@ -86,8 +86,20 @@ choice
 	  There are several possible configurations providing
 	  varying degrees of control over the spawned processes.
 
+config BR2_INIT_BUSYBOX_BARE
+	bool "BusyBox / inittab"
+	select BR2_PACKAGE_BUSYBOX
+	help
+	  Minimalistic init implementation from busybox.
+	  Daemons are written directly to /etc/inittab.
+
+	  No runlevel support.
+	  No runtime control over processes.
+	  Daemons are started in foreground mode,
+	  failed processes are respawned.
+
 config BR2_INIT_BUSYBOX
-	bool "BusyBox"
+	bool "BusyBox / initscripts"
 	select BR2_PACKAGE_BUSYBOX
 	help
 	  Minimalistic init implementation from busybox
@@ -100,7 +112,7 @@ config BR2_INIT_BUSYBOX
 	  Failed processes are not respawned.
 
 config BR2_INIT_SYSV
-	bool "systemV"
+	bool "sysvinit / initscripts"
 	select BR2_PACKAGE_BUSYBOX_SHOW_OTHERS # sysvinit
 	select BR2_PACKAGE_SYSVINIT
 	help
-- 
2.0.3

[Buildroot] [RFC 06/11] ptp: new init infrastructure

[Alex Suykov]

Typical package: a simple foreground daemon.
---
 package/ptpd/S65ptpd  | 29 -----------------------------
 package/ptpd/ptpd.mk  |  5 -----
 package/ptpd/ptpd.run |  1 +
 3 files changed, 1 insertion(+), 34 deletions(-)
 delete mode 100755 package/ptpd/S65ptpd
 create mode 100644 package/ptpd/ptpd.run

diff --git a/package/ptpd/S65ptpd b/package/ptpd/S65ptpd
deleted file mode 100755
index 4206c5e..0000000
--- a/package/ptpd/S65ptpd
+++ /dev/null
@@ -1,29 +0,0 @@
-#!/bin/sh
-#
-# Start ptpd
-#
-
-case "$1" in
-  start)
-	echo -n "Starting ptpd: "
-	start-stop-daemon -S -q -x /usr/sbin/ptpd -- -S
-	if [ $? != 0 ]; then
-		echo "FAILED"
-		exit 1
-	else
-		echo "OK"
-	fi
-	;;
-  stop)
-	echo -n "Stopping ptpd: "
-	start-stop-daemon -K -q -x /usr/sbin/ptpd
-	echo "OK"
-	;;
-  restart|reload)
-	;;
-  *)
-	echo "Usage: $0 {start|stop|restart}"
-	exit 1
-esac
-
-exit $?
diff --git a/package/ptpd/ptpd.mk b/package/ptpd/ptpd.mk
index ac9ab98..b08a5db 100644
--- a/package/ptpd/ptpd.mk
+++ b/package/ptpd/ptpd.mk
@@ -17,9 +17,4 @@ define PTPD_INSTALL_TARGET_CMDS
 	$(INSTALL) -m 755 -D $(@D)/src/ptpd $(TARGET_DIR)/usr/sbin/ptpd
 endef
 
-define PTPD_INSTALL_INIT_SYSV
-	$(INSTALL) -m 755 -D package/ptpd/S65ptpd \
-		$(TARGET_DIR)/etc/init.d/S65ptpd
-endef
-
 $(eval $(generic-package))
diff --git a/package/ptpd/ptpd.run b/package/ptpd/ptpd.run
new file mode 100644
index 0000000..bf391c1
--- /dev/null
+++ b/package/ptpd/ptpd.run
@@ -0,0 +1 @@
+exec /usr/sbin/ptpd -c -S
-- 
2.0.3

[Buildroot] [RFC 07/11] upmpcli: new init infrastructure

[Alex Suykov]

Typical package: simple foreground daemon and its effective user entry.
---
 package/upmpdcli/S99upmpdcli    | 40 ----------------------------------------
 package/upmpdcli/upmpdcli.mk    |  9 ---------
 package/upmpdcli/upmpdcli.run   |  1 +
 package/upmpdcli/upmpdcli.users |  1 +
 4 files changed, 2 insertions(+), 49 deletions(-)
 delete mode 100644 package/upmpdcli/S99upmpdcli
 create mode 100644 package/upmpdcli/upmpdcli.run
 create mode 100644 package/upmpdcli/upmpdcli.users

diff --git a/package/upmpdcli/S99upmpdcli b/package/upmpdcli/S99upmpdcli
deleted file mode 100644
index 07b10a6..0000000
--- a/package/upmpdcli/S99upmpdcli
+++ /dev/null
@@ -1,40 +0,0 @@
-#!/bin/sh
-
-NAME=upmpdcli
-DAEMON=/usr/bin/$NAME
-CONFFILE=/etc/$NAME.conf
-PIDFILE=/var/run/$NAME.pid
-DAEMON_ARGS="-D -c $CONFFILE"
-
-# Sanity checks
-test -f $DAEMON || exit 0
-
-start() {
-        echo -n "Starting $NAME: "
-        start-stop-daemon --start --quiet --background --exec $DAEMON \
-                -- $DAEMON_ARGS \
-                && echo "OK" || echo "FAIL"
-}
-
-stop() {
-        echo -n "Stopping $NAME: "
-        start-stop-daemon --stop --quiet --pidfile $PIDFILE \
-                && echo "OK" || echo "FAIL"
-}
-
-case "$1" in
-        start)
-                start
-                ;;
-        stop)
-                stop
-                ;;
-        restart)
-                stop
-                sleep 1
-                start
-                ;;
-        *)
-                echo "Usage: $0 {start|stop|restart}"
-                exit 1
-esac
diff --git a/package/upmpdcli/upmpdcli.mk b/package/upmpdcli/upmpdcli.mk
index 0be4d91..ab9e447 100644
--- a/package/upmpdcli/upmpdcli.mk
+++ b/package/upmpdcli/upmpdcli.mk
@@ -10,15 +10,6 @@ UPMPDCLI_LICENSE = GPLv2+
 UPMPDCLI_LICENSE_FILES = COPYING
 UPMPDCLI_DEPENDENCIES = libmpdclient libupnpp
 
-# Upmpdcli only runs if user upmpdcli exists
-define UPMPDCLI_USERS
-	upmpdcli -1 upmpdcli -1 * - - - Upmpdcli MPD UPnP Renderer Front-End
-endef
-
-define UPMPDCLI_INSTALL_INIT_SYSV
-	$(INSTALL) -D -m 0755 package/upmpdcli/S99upmpdcli $(TARGET_DIR)/etc/init.d/S99upmpdcli
-endef
-
 define UPMPDCLI_INSTALL_CONF_FILE
 	$(INSTALL) -D -m 0755 $(@D)/src/upmpdcli.conf $(TARGET_DIR)/etc/upmpdcli.conf
 endef
diff --git a/package/upmpdcli/upmpdcli.run b/package/upmpdcli/upmpdcli.run
new file mode 100644
index 0000000..54bd8ec
--- /dev/null
+++ b/package/upmpdcli/upmpdcli.run
@@ -0,0 +1 @@
+exec /usr/bin/upmpdcli
diff --git a/package/upmpdcli/upmpdcli.users b/package/upmpdcli/upmpdcli.users
new file mode 100644
index 0000000..c0adbd8
--- /dev/null
+++ b/package/upmpdcli/upmpdcli.users
@@ -0,0 +1 @@
+upmpdcli -1 upmpdcli -1 * - - - Upmpdcli MPD UPnP Renderer Front-End
-- 
2.0.3

[Buildroot] [RFC 08/11] acpid: new init infrastructure

[Alex Suykov]

Systemd has built-in support for acpi events and does not need acpid.
---
 package/acpid/S02acpid  | 22 ----------------------
 package/acpid/acpid.mk  |  5 -----
 package/acpid/acpid.run |  3 +++
 3 files changed, 3 insertions(+), 27 deletions(-)
 delete mode 100755 package/acpid/S02acpid
 create mode 100644 package/acpid/acpid.run

diff --git a/package/acpid/S02acpid b/package/acpid/S02acpid
deleted file mode 100755
index 0840305..0000000
--- a/package/acpid/S02acpid
+++ /dev/null
@@ -1,22 +0,0 @@
-#!/bin/sh
-
-case "$1" in
-	start)
-		echo -n "Starting acpid: "
-		start-stop-daemon -S -q -m -b -p /var/run/acpid.pid --exec /usr/sbin/acpid -- -n
-		[ $? = 0 ] && echo "OK" || echo "FAIL"
-		;;
-	stop)
-		echo -n "Stopping acpid: "
-		start-stop-daemon -K -q -p /var/run/acpid.pid
-		[ $? = 0 ] && echo "OK" || echo "FAIL"
-		;;
-	restart)
-		"$0" stop
-		sleep 1
-		"$0" start
-		;;
-	*)
-		echo "Usage: $0 {start|stop|restart}"
-		;;
-esac
diff --git a/package/acpid/acpid.mk b/package/acpid/acpid.mk
index 7a6a478..61fc6c7 100644
--- a/package/acpid/acpid.mk
+++ b/package/acpid/acpid.mk
@@ -10,11 +10,6 @@ ACPID_SITE = http://downloads.sourceforge.net/project/acpid2
 ACPID_LICENSE = GPLv2+
 ACPID_LICENSE_FILES = COPYING
 
-define ACPID_INSTALL_INIT_SYSV
-	$(INSTALL) -D -m 0755 package/acpid/S02acpid \
-		$(TARGET_DIR)/etc/init.d/S02acpid
-endef
-
 define ACPID_SET_EVENTS
 	mkdir -p $(TARGET_DIR)/etc/acpi/events
 	printf "event=button[ /]power\naction=/sbin/poweroff\n" \
diff --git a/package/acpid/acpid.run b/package/acpid/acpid.run
new file mode 100644
index 0000000..79c6216
--- /dev/null
+++ b/package/acpid/acpid.run
@@ -0,0 +1,3 @@
+## ACPI daemon
+systemd -
+exec /usr/sbin/acpid -f -n
-- 
2.0.3

[Buildroot] [RFC 09/11] am33x-cm3: new init infrastructure

[Alex Suykov]

Non-daemon entry.
---
 package/am33x-cm3/S93-am335x-pm-firmware-load | 6 ------
 package/am33x-cm3/am33x-cm3.mk                | 5 -----
 package/am33x-cm3/am33x-cm3.run               | 6 ++++++
 3 files changed, 6 insertions(+), 11 deletions(-)
 delete mode 100755 package/am33x-cm3/S93-am335x-pm-firmware-load
 create mode 100644 package/am33x-cm3/am33x-cm3.run

diff --git a/package/am33x-cm3/S93-am335x-pm-firmware-load b/package/am33x-cm3/S93-am335x-pm-firmware-load
deleted file mode 100755
index 56c17b3..0000000
--- a/package/am33x-cm3/S93-am335x-pm-firmware-load
+++ /dev/null
@@ -1,6 +0,0 @@
-#!/bin/sh
-
-# Load the PM CM3 firmware
-echo 1 > /sys/devices/ocp.2/44d00000.wkup_m3/firmware/am335x-pm-firmware.bin/loading
-cat /lib/firmware/am335x-pm-firmware.bin > /sys/devices/ocp.2/44d00000.wkup_m3/firmware/am335x-pm-firmware.bin/data
-echo 0 > /sys/devices/ocp.2/44d00000.wkup_m3/firmware/am335x-pm-firmware.bin/loading
diff --git a/package/am33x-cm3/am33x-cm3.mk b/package/am33x-cm3/am33x-cm3.mk
index 01cb9a0..b00f6bb 100644
--- a/package/am33x-cm3/am33x-cm3.mk
+++ b/package/am33x-cm3/am33x-cm3.mk
@@ -23,9 +23,4 @@ define AM33X_CM3_INSTALL_TARGET_CMDS
 		$(TARGET_DIR)/lib/firmware/am335x-pm-firmware.bin
 endef
 
-define AM33X_CM3_INSTALL_INIT_SYSV
-	$(INSTALL) -m 0755 -D package/am33x-cm3/S93-am335x-pm-firmware-load \
-		$(TARGET_DIR)/etc/init.d/S93-am335x-pm-firmware-load
-endef
-
 $(eval $(generic-package))
diff --git a/package/am33x-cm3/am33x-cm3.run b/package/am33x-cm3/am33x-cm3.run
new file mode 100644
index 0000000..68629a4
--- /dev/null
+++ b/package/am33x-cm3/am33x-cm3.run
@@ -0,0 +1,5 @@
+## Load the PM CM3 firmware
+
+echo 1 > /sys/devices/ocp.2/44d00000.wkup_m3/firmware/am335x-pm-firmware.bin/loading
+cat /lib/firmware/am335x-pm-firmware.bin > /sys/devices/ocp.2/44d00000.wkup_m3/firmware/am335x-pm-firmware.bin/data
+echo 0 > /sys/devices/ocp.2/44d00000.wkup_m3/firmware/am335x-pm-firmware.bin/loading
-- 
2.0.3

[Buildroot] [RFC 10/11] postgresql: new init infrastructure

[Alex Suykov]

One of the most complicated entries. Users, files, and pre-start
initialization that must be done with a specific user id.
---
 package/postgresql/S50postgresql      | 44 -----------------------------------
 package/postgresql/postgresql.files   |  1 +
 package/postgresql/postgresql.mk      | 17 --------------
 package/postgresql/postgresql.run     |  6 +++++
 package/postgresql/postgresql.service | 24 -------------------
 package/postgresql/postgresql.users   |  1 +
 6 files changed, 8 insertions(+), 85 deletions(-)
 delete mode 100644 package/postgresql/S50postgresql
 create mode 100644 package/postgresql/postgresql.files
 create mode 100644 package/postgresql/postgresql.run
 delete mode 100644 package/postgresql/postgresql.service
 create mode 100644 package/postgresql/postgresql.users

diff --git a/package/postgresql/S50postgresql b/package/postgresql/S50postgresql
deleted file mode 100644
index 86a8d83..0000000
--- a/package/postgresql/S50postgresql
+++ /dev/null
@@ -1,44 +0,0 @@
-#!/bin/sh
-#
-# start postgresql
-#
-
-umask 077
-
-if [ ! -f /var/lib/pgsql/PG_VERSION ]; then
-	echo "Initializing postgresql data base..."
-	su - postgres -c '/usr/bin/pg_ctl initdb -D /var/lib/pgsql'
-	echo "done"
-fi
-
-start() {
-	echo -n "Starting postgresql: "
-	su - postgres -c '/usr/bin/pg_ctl start -D /var/lib/pgsql -l logfile'
-	echo "OK"
-}
-stop() {
-	echo -n "Stopping postgresql: "
-	su - postgres -c '/usr/bin/pg_ctl stop -D /var/lib/pgsql -m fast'
-	echo "OK"
-}
-restart() {
-	stop
-	start
-}
-
-case "$1" in
-	start)
-		start
-		;;
-	stop)
-		stop
-		;;
-	restart|reload)
-		restart
-		;;
-	*)
-		echo "Usage: $0 {start|stop|restart}"
-		exit 1
-esac
-
-exit $?
diff --git a/package/postgresql/postgresql.files b/package/postgresql/postgresql.files
new file mode 100644
index 0000000..81a3a25
--- /dev/null
+++ b/package/postgresql/postgresql.files
@@ -0,0 +1 @@
+/var/lib/pgsql		d	0700	postgresql	postgresql	-	-	-	-	-
diff --git a/package/postgresql/postgresql.mk b/package/postgresql/postgresql.mk
index bc382ce..1fbe1cf 100644
--- a/package/postgresql/postgresql.mk
+++ b/package/postgresql/postgresql.mk
@@ -60,10 +60,6 @@ ifeq ($(BR2_PACKAGE_OPENSSL),y)
 	POSTGRESQL_CONF_OPTS += --with-openssl
 endif
 
-define POSTGRESQL_USERS
-	postgres -1 postgres -1 * /var/lib/pgsql /bin/sh - PostgreSQL Server
-endef
-
 define POSTGRESQL_INSTALL_TARGET_FIXUP
 	$(INSTALL) -dm 0700 $(TARGET_DIR)/var/lib/pgsql
 	$(RM) -rf $(TARGET_DIR)/usr/lib/postgresql/pgxs
@@ -78,17 +74,4 @@ endef
 
 POSTGRESQL_POST_INSTALL_STAGING_HOOKS += POSTGRESQL_INSTALL_CUSTOM_PG_CONFIG
 
-define POSTGRESQL_INSTALL_INIT_SYSV
-	$(INSTALL) -m 0755 -D package/postgresql/S50postgresql \
-		$(TARGET_DIR)/etc/init.d/S50postgresql
-endef
-
-define POSTGRESQL_INSTALL_INIT_SYSTEMD
-	$(INSTALL) -D -m 644 package/postgresql/postgresql.service \
-		$(TARGET_DIR)/usr/lib/systemd/system/postgresql.service
-	mkdir -p $(TARGET_DIR)/etc/systemd/system/multi-user.target.wants
-	ln -fs ../../../../usr/lib/systemd/system/postgresql.service \
-		$(TARGET_DIR)/etc/systemd/system/multi-user.target.wants/postgresql.service
-endef
-
 $(eval $(autotools-package))
diff --git a/package/postgresql/postgresql.run b/package/postgresql/postgresql.run
new file mode 100644
index 0000000..c7fcf36
--- /dev/null
+++ b/package/postgresql/postgresql.run
@@ -0,0 +1,6 @@
+## PostgreSQL database server
+user postgres
+group postgres
+
+test -f /var/lib/pgsql/PG_VERSION || /usr/bin/initdb -D /var/lib/pgsql
+exec /usr/bin/postmaster -D /var/lib/pgsql
diff --git a/package/postgresql/postgresql.service b/package/postgresql/postgresql.service
deleted file mode 100644
index 4a96258..0000000
--- a/package/postgresql/postgresql.service
+++ /dev/null
@@ -1,24 +0,0 @@
-[Unit]
-Description=PostgreSQL database server
-After=network.target
-
-[Service]
-Type=forking
-
-# start timeout disabled because initdb may run a little
-# longer (eg. 5 minutes on RaspberryPi)
-TimeoutStartSec=0
-
-User=postgres
-Group=postgres
-
-SyslogIdentifier=postgres
-PIDFile=/var/lib/pgsql/postmaster.pid
-
-ExecStartPre=/bin/sh -c "if [ ! -f /var/lib/pgsql/PG_VERSION ]; then /usr/bin/pg_ctl initdb -D /var/lib/pgsql; fi"
-ExecStart=/usr/bin/pg_ctl start -D /var/lib/pgsql -w -l /var/lib/pgsql/logfile
-ExecReload=/usr/bin/pg_ctl reload -D /var/lib/pgsql
-ExecStop=/usr/bin/pg_ctl stop -D /var/lib/pgsql -m fast
-
-[Install]
-WantedBy=multi-user.target
diff --git a/package/postgresql/postgresql.users b/package/postgresql/postgresql.users
new file mode 100644
index 0000000..9724144
--- /dev/null
+++ b/package/postgresql/postgresql.users
@@ -0,0 +1 @@
+postgres -1 postgres -1 * /var/lib/pgsql /bin/sh - PostgreSQL Server
-- 
2.0.3

[Buildroot] [RFC 11/11] openvpn: new init infrastructure

[Alex Suykov]

The only case so far that install-run can not handle.
Openvpn initscript spawns several processes, one for each
configured tunnel.

Initscripts is the only system that allows this, primarily
because it does not need to track spawned processes.
---
 package/openvpn/S60openvpn   | 103 -------------------------------------------
 package/openvpn/openvpn.init | 103 +++++++++++++++++++++++++++++++++++++++++++
 package/openvpn/openvpn.mk   |   5 ---
 package/openvpn/openvpn.run  |   1 +
 4 files changed, 104 insertions(+), 108 deletions(-)
 delete mode 100755 package/openvpn/S60openvpn
 create mode 100755 package/openvpn/openvpn.init
 create mode 100644 package/openvpn/openvpn.run

diff --git a/package/openvpn/S60openvpn b/package/openvpn/S60openvpn
deleted file mode 100755
index 94bdc60..0000000
--- a/package/openvpn/S60openvpn
+++ /dev/null
@@ -1,103 +0,0 @@
-#!/bin/sh -e
-#
-# Original version by Robert Leslie
-# <rob@mars.org>, edited by iwj and cs
-# Modified for openvpn by Alberto Gonzalez Iniesta <agi@agi.as>
-# Modified for restarting / starting / stopping single tunnels by Richard Mueller <mueller@teamix.net>
-
-test $DEBIAN_SCRIPT_DEBUG && set -v -x
-
-DAEMON=/usr/sbin/openvpn
-CONFIG_DIR=/etc/openvpn
-test -x $DAEMON || exit 0
-test -d $CONFIG_DIR || exit 0
-
-start_vpn () {
-    $DAEMON --daemon --writepid /var/run/openvpn.$NAME.pid \
-            --config $CONFIG_DIR/$NAME.conf --cd $CONFIG_DIR || echo -n " FAILED->"
-    echo -n " $NAME"
-}
-stop_vpn () {
-   kill `cat $PIDFILE` || true
-  rm $PIDFILE
-}
-
-case "$1" in
-start)
-  echo -n "Starting openvpn:"
-
-  if test -z $2 ; then
-    for CONFIG in `cd $CONFIG_DIR; ls *.conf 2> /dev/null`; do
-      NAME=${CONFIG%%.conf}
-      start_vpn
-    done
-  else
-    if test -e $CONFIG_DIR/$2.conf ; then
-      NAME=$2
-      start_vpn
-    else
-      echo -n " No such VPN: $2"
-    fi
-  fi
-  echo "."
-
-  ;;
-stop)
-  echo -n "Stopping openvpn:"
-
-  if test -z $2 ; then
-    for PIDFILE in `ls /var/run/openvpn.*.pid 2> /dev/null`; do
-      NAME=`echo $PIDFILE | cut -c18-`
-      NAME=${NAME%%.pid}
-      stop_vpn
-      echo -n " $NAME"
-    done
-  else
-    if test -e /var/run/openvpn.$2.pid ; then
-      PIDFILE=`ls /var/run/openvpn.$2.pid 2> /dev/null`
-      NAME=`echo $PIDFILE | cut -c18-`
-      NAME=${NAME%%.pid}
-      stop_vpn
-      echo -n " $NAME"
-    else
-      echo -n " No such VPN: $2"
-    fi
-  fi
-  echo "."
-  ;;
-# We only 'reload' for running VPNs. New ones will only start with 'start' or 'restart'.
-reload|force-reload)
-  echo -n "Reloading openvpn:"
-  for PIDFILE in `ls /var/run/openvpn.*.pid 2> /dev/null`; do
-    NAME=`echo $PIDFILE | cut -c18-`
-    NAME=${NAME%%.pid}
-# If openvpn if running under a different user than root we'll need to restart
-    if egrep '^( |\t)*user' $CONFIG_DIR/$NAME.conf > /dev/null 2>&1 ; then
-      stop_vpn
-      sleep 1
-      start_vpn
-      echo -n "(restarted)"
-    else
-      kill -HUP `cat $PIDFILE` || true
-#    start-stop-daemon --stop --signal HUP --quiet --oknodo \
-#      --exec $DAEMON --pidfile $PIDFILE
-    echo -n " $NAME"
-    fi
-  done
-  echo "."
-  ;;
-
-restart)
-  $0 stop $2
-  sleep 1
-  $0 start $2
-  ;;
-*)
-  echo "Usage: $0 {start|stop|reload|restart|force-reload}" >&2
-  exit 1
-  ;;
-esac
-
-exit 0
-
-# vim:set ai et sts=2 sw=2 tw=0:
diff --git a/package/openvpn/openvpn.init b/package/openvpn/openvpn.init
new file mode 100755
index 0000000..94bdc60
--- /dev/null
+++ b/package/openvpn/openvpn.init
@@ -0,0 +1,103 @@
+#!/bin/sh -e
+#
+# Original version by Robert Leslie
+# <rob@mars.org>, edited by iwj and cs
+# Modified for openvpn by Alberto Gonzalez Iniesta <agi@agi.as>
+# Modified for restarting / starting / stopping single tunnels by Richard Mueller <mueller@teamix.net>
+
+test $DEBIAN_SCRIPT_DEBUG && set -v -x
+
+DAEMON=/usr/sbin/openvpn
+CONFIG_DIR=/etc/openvpn
+test -x $DAEMON || exit 0
+test -d $CONFIG_DIR || exit 0
+
+start_vpn () {
+    $DAEMON --daemon --writepid /var/run/openvpn.$NAME.pid \
+            --config $CONFIG_DIR/$NAME.conf --cd $CONFIG_DIR || echo -n " FAILED->"
+    echo -n " $NAME"
+}
+stop_vpn () {
+   kill `cat $PIDFILE` || true
+  rm $PIDFILE
+}
+
+case "$1" in
+start)
+  echo -n "Starting openvpn:"
+
+  if test -z $2 ; then
+    for CONFIG in `cd $CONFIG_DIR; ls *.conf 2> /dev/null`; do
+      NAME=${CONFIG%%.conf}
+      start_vpn
+    done
+  else
+    if test -e $CONFIG_DIR/$2.conf ; then
+      NAME=$2
+      start_vpn
+    else
+      echo -n " No such VPN: $2"
+    fi
+  fi
+  echo "."
+
+  ;;
+stop)
+  echo -n "Stopping openvpn:"
+
+  if test -z $2 ; then
+    for PIDFILE in `ls /var/run/openvpn.*.pid 2> /dev/null`; do
+      NAME=`echo $PIDFILE | cut -c18-`
+      NAME=${NAME%%.pid}
+      stop_vpn
+      echo -n " $NAME"
+    done
+  else
+    if test -e /var/run/openvpn.$2.pid ; then
+      PIDFILE=`ls /var/run/openvpn.$2.pid 2> /dev/null`
+      NAME=`echo $PIDFILE | cut -c18-`
+      NAME=${NAME%%.pid}
+      stop_vpn
+      echo -n " $NAME"
+    else
+      echo -n " No such VPN: $2"
+    fi
+  fi
+  echo "."
+  ;;
+# We only 'reload' for running VPNs. New ones will only start with 'start' or 'restart'.
+reload|force-reload)
+  echo -n "Reloading openvpn:"
+  for PIDFILE in `ls /var/run/openvpn.*.pid 2> /dev/null`; do
+    NAME=`echo $PIDFILE | cut -c18-`
+    NAME=${NAME%%.pid}
+# If openvpn if running under a different user than root we'll need to restart
+    if egrep '^( |\t)*user' $CONFIG_DIR/$NAME.conf > /dev/null 2>&1 ; then
+      stop_vpn
+      sleep 1
+      start_vpn
+      echo -n "(restarted)"
+    else
+      kill -HUP `cat $PIDFILE` || true
+#    start-stop-daemon --stop --signal HUP --quiet --oknodo \
+#      --exec $DAEMON --pidfile $PIDFILE
+    echo -n " $NAME"
+    fi
+  done
+  echo "."
+  ;;
+
+restart)
+  $0 stop $2
+  sleep 1
+  $0 start $2
+  ;;
+*)
+  echo "Usage: $0 {start|stop|reload|restart|force-reload}" >&2
+  exit 1
+  ;;
+esac
+
+exit 0
+
+# vim:set ai et sts=2 sw=2 tw=0:
diff --git a/package/openvpn/openvpn.mk b/package/openvpn/openvpn.mk
index 3854b23..6937d50 100644
--- a/package/openvpn/openvpn.mk
+++ b/package/openvpn/openvpn.mk
@@ -52,9 +52,4 @@ define OPENVPN_INSTALL_TARGET_CMDS
 		$(TARGET_DIR)/usr/sbin/openvpn
 endef
 
-define OPENVPN_INSTALL_INIT_SYSV
-	$(INSTALL) -m 755 -D package/openvpn/S60openvpn \
-		$(TARGET_DIR)/etc/init.d/S60openvpn
-endef
-
 $(eval $(autotools-package))
diff --git a/package/openvpn/openvpn.run b/package/openvpn/openvpn.run
new file mode 100644
index 0000000..20ce79c
--- /dev/null
+++ b/package/openvpn/openvpn.run
@@ -0,0 +1 @@
+initscripts 60:openvpn.init
-- 
2.0.3

[Buildroot] [RFC 00/11] common init infrastructure

[Arnout Vandecappelle]

On 21/03/15 19:26, Alex Suykov wrote:
> This series is intended to clean up buildroot init system somewhat,
> removing the need for extensive sysv and systemd handling code
> in every package that installs init bits, and allowing alternative
> init systems to be implemented with reasonable amount of effort.

 Wow, impressive. I hadn't imagined it would be possible to get this far.

 I have a couple of overall remarks before I dive into the individual patches.

- Although we have a dependency on python, we don't currently use python for any
of the core functionality (i.e., just building). Personally I like python a lot,
but now you already have to know make and shell pretty well in order to touch
the buildroot core. Adding python to the mix is making the threshold even higher.

- The generation script looks a bit complicated, while on the other hand the
.run files are really terse. I'd prefer a bit more verbosity in the .run files,
especially if that can make the generation script a bit simpler.

- The changes for .users and .files are pretty much independent from this series
so they could be put in a separate series. And I'm anyway not convinced that
they're worthwhile.


 Regards,
 Arnout


> Overview of the changes:
> 
> * drop per-package sysv and systemd files, introduce common format
>   used to generate files for both of them, use a script to do
>   the conversion from common format to sysv or systemd.
> 
> * remove sysv and systemd installation code from package .mk files,
>   let pkg-generic handle it.
> 
> * remove sysv-specific files from skeleton, and sysv-specific hacks
>   from system/system.mk. Use a dedicated script to write inittab,
>   handling global conditionals (root rw/ro, serial getty and so on)
> 
> * remove user definitions from package .mk files, and mkdirs from
>   per-package sysv initscript, use per-package .users and .files
>   instead in a way similar to .hash files.
> 
> Expected benefits:
> 
> * (almost) all packages get both sysv and systemd files at the same
>   time and without the need to support them separately.
> 
> * init-related changes can be introduced without patching
>   all 70+ individual packages that install startup files.
> 
> * new init systems can be introduced with only minor effort,
>   including lightweight ones with controlled (foreground) daemons.
> 
> This series includes the infrastructure changes, and a couple of
> package patches to show how it looks like. Complete set of package
> patches uploaded here: https://github.com/arsv/br/
> 
> 
> This series is for reviewing, not for merging.
> 


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [RFC 00/11] common init infrastructure

[Alex Suykov]

Sat, Mar 21, 2015 at 09:41:22PM +0100, Arnout Vandecappelle wrote:

> - Although we have a dependency on python, we don't currently use python for any
> of the core functionality (i.e., just building). Personally I like python a lot,
> but now you already have to know make and shell pretty well in order to touch
> the buildroot core. Adding python to the mix is making the threshold even higher.
Ah, I see. I decided to go with python specifically to avoid extra dependencies,
after looking at the language stats in support/scripts.
Now indeed python scripts there are not build scripts.

> - The generation script looks a bit complicated, while on the other hand the
> .run files are really terse. I'd prefer a bit more verbosity in the .run files,
> especially if that can make the generation script a bit simpler.
Makes sense. And it may help to drop python dependency at some point.
I need to think on this and maybe try things to see what could work better.

Initially the format was somewhat simplier, with prefixed lines.
Definitely easier to parse, but there were some quirks with that approach.

[Buildroot] [RFC 00/11] common init infrastructure

[Yann E. MORIN]

Alex, All,

On 2015-03-21 20:26 +0200, Alex Suykov spake thusly:
> This series is intended to clean up buildroot init system somewhat,
> removing the need for extensive sysv and systemd handling code
> in every package that installs init bits, and allowing alternative
> init systems to be implemented with reasonable amount of effort.
> 
> Overview of the changes:
> 
> * drop per-package sysv and systemd files, introduce common format
>   used to generate files for both of them, use a script to do
>   the conversion from common format to sysv or systemd.

I have not delve too deep in the implementation (because, Python! ;-) ),
but from the examples, it seems it would still be possible to start more
than one "daemon" for a service, like is done by samba, right?

> * remove sysv and systemd installation code from package .mk files,
>   let pkg-generic handle it.
> 
> * remove sysv-specific files from skeleton, and sysv-specific hacks
>   from system/system.mk. Use a dedicated script to write inittab,
>   handling global conditionals (root rw/ro, serial getty and so on)
> 
> * remove user definitions from package .mk files, and mkdirs from
>   per-package sysv initscript, use per-package .users and .files
>   instead in a way similar to .hash files.

How do you handle conditional creation of users/groups? Let's take a
supposed package that can install a server and/or a client (like a SQL
related package, for example). The server (sqld) needs a specific
user+group, but the client (sql-cli) does not. Currently, this is easily
done by emclosing the user defintion between a 'if server' block.

> Expected benefits:
> 
> * (almost) all packages get both sysv and systemd files at the same
>   time and without the need to support them separately.
> 
> * init-related changes can be introduced without patching
>   all 70+ individual packages that install startup files.
> 
> * new init systems can be introduced with only minor effort,
>   including lightweight ones with controlled (foreground) daemons.
> 
> This series includes the infrastructure changes, and a couple of
> package patches to show how it looks like. Complete set of package
> patches uploaded here: https://github.com/arsv/br/

It would be much nicer if you were to push your branch there, rather
than a set of patches. Having a branch would mean we could fetch from
it.

> This series is for reviewing, not for merging.

One thing that is wrong throughout the series, is that you hard-code the
path to "outoput/" and this breaks out-of-tree builds [0]. You should use
$(BASE_DIR) instead.

[0] http://buildroot.net/downloads/manual/manual.html#_building_out_of_tree

Also, I think this series is changing a lot of different things at the
same time. For example, there is infra changes to handle:
  - init script,
  - users/groups,
  - skeleton /etc/ stuff

This series should be split in simpler sub-series, or at least into much
simpler patches. For example, patch 4/11 that adds help entries to init
systems could probably sent first (after a bit of rewording to remove
references to the new init infra), because it is a valuable addition,
and it does not really depend on the new infra.

Otherwise, I have a feeling this is pushing things a bit too far. I'm
usually very happy with "abstraction" and I more than once have pushed
for such changes. However, in this area, I'm a bit sceptic as to whether
we should introduce our own "initscript" format (the '.run' files).

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [RFC 00/11] common init infrastructure

[Alex Suykov]

Sun, Mar 22, 2015 at 12:28:07PM +0100, Yann E. MORIN wrote:

> How do you handle conditional creation of users/groups? Let's take a
> supposed package that can install a server and/or a client (like a SQL
> related package, for example). The server (sqld) needs a specific
> user+group, but the client (sql-cli) does not. Currently, this is easily
> done by emclosing the user defintion between a 'if server' block.
I think it makes sense to install users unconditionally.
Mostly because not installing them is a minor optimization of the resulting
rootfs image and I see no good way to handle it.

If necessary, perhaps listing .users to be included in the project .mk
file may be an option, the same way it is done with .run files.

> Otherwise, I have a feeling this is pushing things a bit too far. I'm
> usually very happy with "abstraction" and I more than once have pushed
> for such changes. However, in this area, I'm a bit sceptic as to whether
> we should introduce our own "initscript" format (the '.run' files).
That's a valid point, but I think buildroot already uses its own
initscript format, namely its sysv(-style) initscripts.

They are supplied with buildroot, they are non-standard, and they are
nearly useless outside of buildroot.

So I see it a choice between different formats for buildroot-specific
service description, and the one currently in use does not look like
a good data storage format at all.

[Buildroot] [RFC 00/11] common init infrastructure

[Arnout Vandecappelle]

On 22/03/15 14:23, Alex Suykov wrote:
> Sun, Mar 22, 2015 at 12:28:07PM +0100, Yann E. MORIN wrote:
[snip]
>> Otherwise, I have a feeling this is pushing things a bit too far. I'm
>> usually very happy with "abstraction" and I more than once have pushed
>> for such changes. However, in this area, I'm a bit sceptic as to whether
>> we should introduce our own "initscript" format (the '.run' files).
> That's a valid point, but I think buildroot already uses its own
> initscript format, namely its sysv(-style) initscripts.
> 
> They are supplied with buildroot, they are non-standard, and they are
> nearly useless outside of buildroot.
> 
> So I see it a choice between different formats for buildroot-specific
> service description, and the one currently in use does not look like
> a good data storage format at all.

 The point is that the shell scripts provided by buildroot are just shell, so
"anyone" can understand and modify them. While the .run format is something new
that nobody knows, and even if it is documented it will be non-trivial to find
the documentation.

 That's also why I say that the .run format should be verbose, to make it easy
to read and modify.


 Regards,
 Arnout


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F