RFC: poky-tiny: init procedure

RFC: poky-tiny: init procedure

[Darren Hart]

For those of you using poky-tiny or are interested in building very very
small systems, I would appreciate your thoughts here.

Currently poky-tiny images will boot and run /bin/sh, which results in
error messages to the console about being unable to open the tty and job
control being disabled (this is a common problem, if not commonly
understood).

The shell must be session leader to open the tty, and the tty must not
be /dev/console (it should be a vt or a physical tty like ttyS0), the
tty is required for job control (handling signals, etc.).

The Problem:
poky-tiny should boot to a shell without error messages and have
job-control.

The Vision:
The goals of poky-tiny are to be an initial starting point from which to
build a distribution that does what you want, and NOTHING more.

The Proposal:
o Do not include the standard Busybox init
o Do provide a basic /init script that can be easily modified
o Do provide a basic shell
  o Without init, this requires setsid and cttyhack to easily find the
    right device
o Do not provide inittab functionality

I can accomplish this by updating busybox to include:
CONFIG_SETSID=y
CONFIG_CTTYHACK=y

And providing a basic init script as follows:

#!/bin/sh
mount none -t proc /proc
mount none -t sysfs /sys
mkdir /dev/pts
mount none -t devpts /dev/pts
ifup lo

# Become session leader and try to find a real tty (e.g. ttyS0)
setsid cttyhack sh

This results in a system that boots with the virtual filesystems
mounted, the local network interface up, and a shell with job control
running. Nothing else.

Potential Problems:
Should the script be /init (for initramfs images) or /sbin/init for
images on block devices. Note that the default method of running
poky-tiny is with an initramfs as it doesn't support any block devices
by default.

Adding services like dropbear to your image will not get started on boot
as their init scripts will not be run. I'm OK with documenting this
behavior as part of creating your own distribution based on tiny.

If the shell crashes, or you exit, the kernel panics (because init returns).

Thoughts/Comments/Criticism welcome.

Thanks!

-- 
Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel

Re: RFC: poky-tiny: init procedure

[Tim Bird]

On 6/13/2012 5:33 PM, Darren Hart wrote:
> For those of you using poky-tiny or are interested in building very very
> small systems, I would appreciate your thoughts here.
>
> Currently poky-tiny images will boot and run /bin/sh, which results in
> error messages to the console about being unable to open the tty and job
> control being disabled (this is a common problem, if not commonly
> understood).
>
> The shell must be session leader to open the tty, and the tty must not
> be /dev/console (it should be a vt or a physical tty like ttyS0), the
> tty is required for job control (handling signals, etc.).
>
> The Problem:
> poky-tiny should boot to a shell without error messages and have
> job-control.
>
> The Vision:
> The goals of poky-tiny are to be an initial starting point from which to
> build a distribution that does what you want, and NOTHING more.
>
> The Proposal:
> o Do not include the standard Busybox init
> o Do provide a basic /init script that can be easily modified
> o Do provide a basic shell
>    o Without init, this requires setsid and cttyhack to easily find the
>      right device
> o Do not provide inittab functionality
>
> I can accomplish this by updating busybox to include:
> CONFIG_SETSID=y
> CONFIG_CTTYHACK=y
>
> And providing a basic init script as follows:
>
> #!/bin/sh
> mount none -t proc /proc
> mount none -t sysfs /sys
> mkdir /dev/pts
> mount none -t devpts /dev/pts
> ifup lo
>
> # Become session leader and try to find a real tty (e.g. ttyS0)
> setsid cttyhack sh
>
> This results in a system that boots with the virtual filesystems
> mounted, the local network interface up, and a shell with job control
> running. Nothing else.
>
> Potential Problems:
> Should the script be /init (for initramfs images) or /sbin/init for
> images on block devices. Note that the default method of running
> poky-tiny is with an initramfs as it doesn't support any block devices
> by default.
>
> Adding services like dropbear to your image will not get started on boot
> as their init scripts will not be run. I'm OK with documenting this
> behavior as part of creating your own distribution based on tiny.
>
> If the shell crashes, or you exit, the kernel panics (because init returns).
>
> Thoughts/Comments/Criticism welcome.
>

This sounds like a great approach.  What I've found at Sony is that it's 
nice to
have a template for turning on individual services, that users can 
decide to enable or not.
You don't want to include everything under the sun, but it's nice to 
have a few basic
things spelled out (but disabled) so that users can easily enable them.

How about adding something like:
if [ -f /etc/rc.local ] ; then
  exec /etc/rc.local
fi

You could have this be as is, or the lines could be commented out.

Inside /etc/rc.local you have a bunch of commented-out lines for turning on
services like telnetd, dropbear, mounting debugfs, starting networking 
(ifup eth0),
etc.  If the user wants any of these individual items, they uncomment the
appropriate lines in /etc/rc.local.

IMHO, you DON'T want to create some fancy directory-based system like 
sysVinit
that automatically handles anything that could possibly be included.  But
having a couple of common cases that the user can easily enable is 
pretty handy.

Alternatively, you could have these commented-out lines in the init 
script itself.
This saves you an additional file in the rootfs, as well as the fork to exec
/etc/rc.local.  (I suppose you could also 'source' this, to avoid the fork.)

Just some ideas.
  -- Tim

Re: RFC: poky-tiny: init procedure

[Darren Hart]

On 06/13/2012 06:09 PM, Tim Bird wrote:
> On 6/13/2012 5:33 PM, Darren Hart wrote:
>> For those of you using poky-tiny or are interested in building very very
>> small systems, I would appreciate your thoughts here.
>>
>> Currently poky-tiny images will boot and run /bin/sh, which results in
>> error messages to the console about being unable to open the tty and job
>> control being disabled (this is a common problem, if not commonly
>> understood).
>>
>> The shell must be session leader to open the tty, and the tty must not
>> be /dev/console (it should be a vt or a physical tty like ttyS0), the
>> tty is required for job control (handling signals, etc.).
>>
>> The Problem:
>> poky-tiny should boot to a shell without error messages and have
>> job-control.
>>
>> The Vision:
>> The goals of poky-tiny are to be an initial starting point from which to
>> build a distribution that does what you want, and NOTHING more.
>>
>> The Proposal:
>> o Do not include the standard Busybox init
>> o Do provide a basic /init script that can be easily modified
>> o Do provide a basic shell
>>    o Without init, this requires setsid and cttyhack to easily find the
>>      right device
>> o Do not provide inittab functionality
>>
>> I can accomplish this by updating busybox to include:
>> CONFIG_SETSID=y
>> CONFIG_CTTYHACK=y
>>
>> And providing a basic init script as follows:
>>
>> #!/bin/sh
>> mount none -t proc /proc
>> mount none -t sysfs /sys
>> mkdir /dev/pts
>> mount none -t devpts /dev/pts
>> ifup lo
>>
>> # Become session leader and try to find a real tty (e.g. ttyS0)
>> setsid cttyhack sh
>>
>> This results in a system that boots with the virtual filesystems
>> mounted, the local network interface up, and a shell with job control
>> running. Nothing else.
>>
>> Potential Problems:
>> Should the script be /init (for initramfs images) or /sbin/init for
>> images on block devices. Note that the default method of running
>> poky-tiny is with an initramfs as it doesn't support any block devices
>> by default.
>>
>> Adding services like dropbear to your image will not get started on boot
>> as their init scripts will not be run. I'm OK with documenting this
>> behavior as part of creating your own distribution based on tiny.
>>
>> If the shell crashes, or you exit, the kernel panics (because init returns).
>>
>> Thoughts/Comments/Criticism welcome.
>>
> 
> This sounds like a great approach.  What I've found at Sony is that it's 
> nice to
> have a template for turning on individual services, that users can 
> decide to enable or not.
> You don't want to include everything under the sun, but it's nice to 
> have a few basic
> things spelled out (but disabled) so that users can easily enable them.
> 
> How about adding something like:
> if [ -f /etc/rc.local ] ; then
>   exec /etc/rc.local
> fi
> 
> You could have this be as is, or the lines could be commented out.
> 
> Inside /etc/rc.local you have a bunch of commented-out lines for turning on
> services like telnetd, dropbear, mounting debugfs, starting networking 
> (ifup eth0),
> etc.  If the user wants any of these individual items, they uncomment the
> appropriate lines in /etc/rc.local.
> 
> IMHO, you DON'T want to create some fancy directory-based system like 
> sysVinit
> that automatically handles anything that could possibly be included.  But
> having a couple of common cases that the user can easily enable is 
> pretty handy.
> 
> Alternatively, you could have these commented-out lines in the init 
> script itself.
> This saves you an additional file in the rootfs, as well as the fork to exec
> /etc/rc.local.  (I suppose you could also 'source' this, to avoid the fork.)
> 
> Just some ideas.

I like the rc.local idea as that makes it easily extendable without even
having to modify the original distro definition.

Excellent suggestions, thank you!

-- 
Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel

Re: RFC: poky-tiny: init procedure

[Darren Hart]

On 06/13/2012 06:20 PM, Darren Hart wrote:
> 
> 
> On 06/13/2012 06:09 PM, Tim Bird wrote:
>> On 6/13/2012 5:33 PM, Darren Hart wrote:
>>> For those of you using poky-tiny or are interested in building very very
>>> small systems, I would appreciate your thoughts here.
>>>
>>> Currently poky-tiny images will boot and run /bin/sh, which results in
>>> error messages to the console about being unable to open the tty and job
>>> control being disabled (this is a common problem, if not commonly
>>> understood).
>>>
>>> The shell must be session leader to open the tty, and the tty must not
>>> be /dev/console (it should be a vt or a physical tty like ttyS0), the
>>> tty is required for job control (handling signals, etc.).
>>>
>>> The Problem:
>>> poky-tiny should boot to a shell without error messages and have
>>> job-control.
>>>
>>> The Vision:
>>> The goals of poky-tiny are to be an initial starting point from which to
>>> build a distribution that does what you want, and NOTHING more.
>>>
>>> The Proposal:
>>> o Do not include the standard Busybox init
>>> o Do provide a basic /init script that can be easily modified
>>> o Do provide a basic shell
>>>    o Without init, this requires setsid and cttyhack to easily find the
>>>      right device
>>> o Do not provide inittab functionality
>>>
>>> I can accomplish this by updating busybox to include:
>>> CONFIG_SETSID=y
>>> CONFIG_CTTYHACK=y
>>>
>>> And providing a basic init script as follows:
>>>
>>> #!/bin/sh
>>> mount none -t proc /proc
>>> mount none -t sysfs /sys
>>> mkdir /dev/pts
>>> mount none -t devpts /dev/pts
>>> ifup lo
>>>
>>> # Become session leader and try to find a real tty (e.g. ttyS0)
>>> setsid cttyhack sh
>>>
>>> This results in a system that boots with the virtual filesystems
>>> mounted, the local network interface up, and a shell with job control
>>> running. Nothing else.
>>>
>>> Potential Problems:
>>> Should the script be /init (for initramfs images) or /sbin/init for
>>> images on block devices. Note that the default method of running
>>> poky-tiny is with an initramfs as it doesn't support any block devices
>>> by default.
>>>
>>> Adding services like dropbear to your image will not get started on boot
>>> as their init scripts will not be run. I'm OK with documenting this
>>> behavior as part of creating your own distribution based on tiny.
>>>
>>> If the shell crashes, or you exit, the kernel panics (because init returns).
>>>
>>> Thoughts/Comments/Criticism welcome.
>>>
>>
>> This sounds like a great approach.  What I've found at Sony is that it's 
>> nice to
>> have a template for turning on individual services, that users can 
>> decide to enable or not.
>> You don't want to include everything under the sun, but it's nice to 
>> have a few basic
>> things spelled out (but disabled) so that users can easily enable them.
>>
>> How about adding something like:
>> if [ -f /etc/rc.local ] ; then
>>   exec /etc/rc.local
>> fi
>>
>> You could have this be as is, or the lines could be commented out.
>>
>> Inside /etc/rc.local you have a bunch of commented-out lines for turning on
>> services like telnetd, dropbear, mounting debugfs, starting networking 
>> (ifup eth0),
>> etc.  If the user wants any of these individual items, they uncomment the
>> appropriate lines in /etc/rc.local.
>>
>> IMHO, you DON'T want to create some fancy directory-based system like 
>> sysVinit
>> that automatically handles anything that could possibly be included.  But
>> having a couple of common cases that the user can easily enable is 
>> pretty handy.
>>
>> Alternatively, you could have these commented-out lines in the init 
>> script itself.
>> This saves you an additional file in the rootfs, as well as the fork to exec
>> /etc/rc.local.  (I suppose you could also 'source' this, to avoid the fork.)
>>
>> Just some ideas.
> 
> I like the rc.local idea as that makes it easily extendable without even
> having to modify the original distro definition.
> 
> Excellent suggestions, thank you!
> 

Updated init with Tim's recommendations and a loop to avoid the kernel
panic on shell exit:

#!/bin/sh

# Mount the Linux kernel virtual filesystems
mount none -t proc /proc
mount none -t sysfs /sys
mkdir /dev/pts
mount none -t devpts /dev/pts

ifup lo

# Allow for distro or local customizations
if [ -f /etc/rc.local ] ; then
        source /etc/rc.local
fi

# Become session leader and try to find a real tty (e.g. ttyS0)
while true; do
        setsid cttyhack sh
        echo "Console sh exited with $?, respawning..."
        sleep 1
done


I believe this is a reasonable set of things to expect to be working on
ANY system built with the Yocto Project (currently poky-tiny is defined
in meta-yocto).

-- 
Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel

Re: RFC: poky-tiny: init procedure

[Tomas Frydrych]

Hi Darren,

On 14/06/12 01:33, Darren Hart wrote:
> o Do not include the standard Busybox init
...
> o Do not provide inittab functionality

I am not entirely clear what you are hoping to gain by creating a home
grown init solution?

A system that runs nothing but a shell is really not useful for anything
all, everyone using it will be adding some sort of services, so the
question of how the extending works (or does not work), needs to be in
the forefront of the design. My main reservation is that you are
suggesting to break one of the basic premisses behind the whole
ecosystem, namely that if I add a package that provides a service to an
image, I get that service running; 'fix by documentation' is never a fix.

So back to my original question, what are the expected benefits to the
Poky users of not using initd in such minimal systems, and do you have
any numbers to show it is worth it? Maybe the numbers are compelling,
but considering that currently Poky does not even support systemd,
adding yet another, home grown, init system seems like a step in the
wrong direction (perhaps sorting out the systemd mess is an opportunity
to deal with the init sequence in a more generic way).

(I hear what you are saying about a system that only includes packages
you want and nothing else, but this is orthogonal to the system size; I
for one want to be able to create a midsized system that includes only
packages that I want and nothing else. :) )

Tomas

Re: RFC: poky-tiny: init procedure

[Darren Hart]

On 06/14/2012 12:11 AM, Tomas Frydrych wrote:
> Hi Darren,
> 
> On 14/06/12 01:33, Darren Hart wrote:
>> o Do not include the standard Busybox init
> ...
>> o Do not provide inittab functionality
> 
> I am not entirely clear what you are hoping to gain by creating a home
> grown init solution?

Calling it an init "solution" is a bit of a stretch. It's really the
absence of an init system. It's what is there in place of an init
system. Just enough to get the system up.

The gain is really to allow people to do whatever they want here,
without having to undo a major piece like init first. Clean slate.

> 
> A system that runs nothing but a shell is really not useful for anything
> all, everyone using it will be adding some sort of services, so the
> question of how the extending works (or does not work), needs to be in
> the forefront of the design.

Agreed. I thought Tim's proposal of an rc.local script for customization
was a good answer to that question. Anything beyond this and people
should really just use the traditional init system.

> My main reservation is that you are
> suggesting to break one of the basic premisses behind the whole
> ecosystem, namely that if I add a package that provides a service to an
> image, I get that service running; 'fix by documentation' is never a fix.

This is a valid point. However, for something like poky-tiny that is
meant as a building block, and not an ultimate solution, documenting the
process of extending it seems to be a very reasonable thing to me (in
fact it's really required).

> So back to my original question, what are the expected benefits to the
> Poky users of not using initd in such minimal systems, and do you have
> any numbers to show it is worth it?

I don't have numbers for init specifically, and those would be good to
generate.

However, right now poky-tiny just executes /bin/sh, has no job control,
and panics if the shell exits. This solution improves the kick-the-tires
experience with poky-tiny, without pulling in all of init, and provides
a better defined launch point for custom development.

> Maybe the numbers are compelling,
> but considering that currently Poky does not even support systemd,
> adding yet another, home grown, init system seems like a step in the
> wrong direction

Again, we're talking about 14 lines of shell, so "init system" is
overstating it significantly.

> (perhaps sorting out the systemd mess is an opportunity
> to deal with the init sequence in a more generic way).

Agreed, that would be nice, but probably orthogonal to this.

> 
> (I hear what you are saying about a system that only includes packages
> you want and nothing else, but this is orthogonal to the system size;

If so, then it's an ample intersection where they cross :-)

> I
> for one want to be able to create a midsized system that includes only
> packages that I want and nothing else. :) )

I would think the default poky and core-image-minimal would serve that
goal very well.

Thank you for the comments.

-- 
Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel

Re: RFC: poky-tiny: init procedure

[Tomas Frydrych]

Hi Darren,

On 14/06/12 22:09, Darren Hart wrote:
> This solution improves the kick-the-tires
> experience with poky-tiny, without pulling in all of init, 

I think you really should quantify what 'all of init' means, without
this you are addressing a problem that is merely perceived. Just a quick
look shows that the sysvinit package is about 120KB unpacked, for that
you get a solution that is tested, robust, and supported across Yocto.


> Again, we're talking about 14 lines of shell, so "init system" is
> overstating it significantly.

Sure, but every new wheel starts exactly this way, and we are already
discussing how to make it do things that sysvinit does for you :-)


Tomas

Re: RFC: poky-tiny: init procedure

[Thomas Petazzoni]

Le Fri, 15 Jun 2012 07:31:45 +0100,
Tomas Frydrych <tf+lists.yocto@r-finger.com> a écrit :

> Hi Darren,
> 
> On 14/06/12 22:09, Darren Hart wrote:
> > This solution improves the kick-the-tires
> > experience with poky-tiny, without pulling in all of init, 
> 
> I think you really should quantify what 'all of init' means, without
> this you are addressing a problem that is merely perceived. Just a
> quick look shows that the sysvinit package is about 120KB unpacked,
> for that you get a solution that is tested, robust, and supported
> across Yocto.

FWIW, Busybox comes with its own implementation of a very simple and
basic init, which only takes a few KB of space inside the Busybox
binary. That's what we use by default in Buildroot, and it works just
fine. It's a really good trade-off between no init at all, and a
full-blown sysvinit system.

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

Re: RFC: poky-tiny: init procedure

[Tim Bird]

On 06/14/2012 11:31 PM, Tomas Frydrych wrote:
> Hi Darren,
> 
> On 14/06/12 22:09, Darren Hart wrote:
>> This solution improves the kick-the-tires
>> experience with poky-tiny, without pulling in all of init, 
> 
> I think you really should quantify what 'all of init' means, without
> this you are addressing a problem that is merely perceived. Just a quick
> look shows that the sysvinit package is about 120KB unpacked, for that
> you get a solution that is tested, robust, and supported across Yocto.

120KB (of anything) is too much to add to a system that is targeted at
being a minimal starting point.  The mechanism added here to support
local customization is about 100 bytes (including the comment).

IMHO, the whole notion of starting with a big system and
subtracting what you don't want in order to create a minimal
system is the wrong approach.  This doesn't scale (down),
because Linux and distros just keep getting bigger, and
the work required to do this subtraction effort
to continue to hit arbitrarily small footprints is
getting harder and harder over time.

A better approach is to start as small as absolutely
possible, and build up from there.  For really small
system, it is less effort to add what you want to an
essentially empty system, than to go the other direction.

With regard to sysVinit - it's a great system for end users,
because it means that packages can slot their startup and
shutdown scripts into the filesystem automatically, and
it relieves the burden of managing this from the user.
The scripts have to be written to deal with myriad
interactions with other components, and they are
somewhat bloated because of this. However, for deeply
embedded, no such automation or bloat is desired.  For
super-constrained systems, you want to recognize and
control 100% of what's running on the system.

Just to give you context, I'm striving for a system that
runs in 1MB RAM.

> 
>> Again, we're talking about 14 lines of shell, so "init system" is
>> overstating it significantly.
> 
> Sure, but every new wheel starts exactly this way, and we are already
> discussing how to make it do things that sysvinit does for you :-)

I'm not.  If this approach doesn't work for you, then feel free to
use sysvinit, or busybox init, or some other system that is more
heavy-weight than this.  There are plenty of other choices.

I realize you said that with a smiley, so I'm not trying to be
dismissive or harsh, but I do think that if you want the features
of some other init system, then the right answer is to use them,
and not complain about this one.
 -- Tim

=============================
Tim Bird
Architecture Group Chair, CE Workgroup of the Linux Foundation
Senior Staff Engineer, Sony Network Entertainment
=============================

Re: RFC: poky-tiny: init procedure

[Tim Bird]

On 06/14/2012 12:11 AM, Tomas Frydrych wrote:
> A system that runs nothing but a shell is really not useful for anything
> all, everyone using it will be adding some sort of services, so the
> question of how the extending works (or does not work), needs to be in
> the forefront of the design. My main reservation is that you are
> suggesting to break one of the basic premisses behind the whole
> ecosystem, namely that if I add a package that provides a service to an
> image, I get that service running; 'fix by documentation' is never a fix.

I didn't address this in my other response, and I
think it's a valid observation.  I agree that
everyone will want to add something to the init,
and that 'fix by documentation' is, in general, a
poor solution.

I don't have any good solution here (one that scales,
or is likely to be adopted widely.)  You don't really
need documentation, if you have the few lines of shell
script that are necessary to start a service in minimal
mode.  The standard init scripts handle lots of interactions
with other services and runtime conditions.  This is why
they are bloated (and the abstractions in them are
almost incomprehensible to read.)

At Sony we've tried various things - like automatically
stripping init scripts of uncalled branches, removing
conditionals that don't apply, etc.  We've also looked
at automatically converting init scripts to C code, to
speed up execution.  You still end up with gobs of code
that doesn't apply to your situation getting executed
at boot time, an overly-complicated test scenario, and a
certain fragility in the overall init system.

One of our early efforts, getting busybox
to not fork commands in init scripts that it had as
builtins, proved to be a significant feature that is still
valuable today to improve boot time.

What we have settled on as a generic solution is basically
what I described in response to Darren's initial post.
You have a list of the short shell snippets required to start
commonly-used services in /etc/rc.local.  These
are commented out by default, but a developer or product
team can uncomment them to enable the associated services.
This keeps everything in one place and provides developers
the commands needed to start things.  Customization to
handle interactions or special cases can proceed from
there.  It's not ideal, in that a developer may have to
re-create or rediscover some configuration, setting, or
start sequence to handle a more complicated usage scenario.
But in order to hit our targets for size or boot time, it
has been helpful to do things this way.  The developer is
forced to manually enable each service, and the start
sequence, commands and command parameters are obvious
from the script.  (This is rarely true of init scripts).

If anyone has good ideas or other experience for
the init sequence for highly customized,
resource-constrained products, I'd like to hear about
them.
 -- Tim

=============================
Tim Bird
Architecture Group Chair, CE Workgroup of the Linux Foundation
Senior Staff Engineer, Sony Network Entertainment
=============================

Re: RFC: poky-tiny: init procedure

[Tomas Frydrych]

On 15/06/12 20:49, Tim Bird wrote:
> On 06/14/2012 11:31 PM, Tomas Frydrych wrote:
> IMHO, the whole notion of starting with a big system and
> subtracting what you don't want in order to create a minimal
> system is the wrong approach.

At no point in this discussion was such an approach advocated by anyone.


> I realize you said that with a smiley, so I'm not trying to be
> dismissive or harsh, but I do think that if you want the features
> of some other init system, then the right answer is to use them,
> and not complain about this one.

I am not complaining, I have asked what the specific, quantifiable,
objective of creating the home grown init 'procedure' is, IMHO that is
an entirely reasonable question to an RFC, the answer to which should
inform the design.

If we are talking about targeting 1MB of RAM then it is unavoidable that
the system has to be hand crafted to a fair degree. It is not what I'd
consider the typical Poky use case, but within those constrains, I'd
probably be looking at eliminating shell scripts altogether in my
customization of whatever Poky offers.

Tomas

Re: RFC: poky-tiny: init procedure

[Darren Hart]

On 06/15/2012 12:05 AM, Thomas Petazzoni wrote:
> Le Fri, 15 Jun 2012 07:31:45 +0100,
> Tomas Frydrych <tf+lists.yocto@r-finger.com> a écrit :
> 
>> Hi Darren,
>>
>> On 14/06/12 22:09, Darren Hart wrote:
>>> This solution improves the kick-the-tires
>>> experience with poky-tiny, without pulling in all of init, 
>>
>> I think you really should quantify what 'all of init' means, without
>> this you are addressing a problem that is merely perceived. Just a
>> quick look shows that the sysvinit package is about 120KB unpacked,
>> for that you get a solution that is tested, robust, and supported
>> across Yocto.
> 
> FWIW, Busybox comes with its own implementation of a very simple and
> basic init, which only takes a few KB of space inside the Busybox
> binary. That's what we use by default in Buildroot, and it works just
> fine. It's a really good trade-off between no init at all, and a
> full-blown sysvinit system.
> 

Good point on this. I had a look. Enabling the basics here, including
the ability to give the controlling console to commands starting with a
dash in inittab results in a 5664 byte delta (compared with 2560 bytes
for enabling setsid and cttyhack). Note that the help in busybox
suggests the cttyhack may be more reliable than the init support for
handing over the controlling terminal.

So the difference between the two options is about 3k, but enabling
setsid and cttyhack may enable others to things besides what I am
looking to do.

I have sent a patch to oe-core to enable SETSID and CTTYHACK in the
default defconfig.

-- 
Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel

Re: RFC: poky-tiny: init procedure

[Darren Hart]

On 06/15/2012 02:26 PM, Tomas Frydrych wrote:
> 
> On 15/06/12 20:49, Tim Bird wrote:
>> On 06/14/2012 11:31 PM, Tomas Frydrych wrote:
>> IMHO, the whole notion of starting with a big system and
>> subtracting what you don't want in order to create a minimal
>> system is the wrong approach.
> 
> At no point in this discussion was such an approach advocated by anyone.

Well, that is the logical alternative. If we suggest using sysvinit now
because it's the standard, then soon we'll say that for systemd, which
means I'll have to include cgroups in my kernel, etc etc. I dont think
Tim's comment was wrong there. Of course "big system" is subjective, to
me that's anything over 4 MB of storage and 8MB of RAM, for Tim, that's
1 MB of RAM. And for the purposes of poky-tiny, anything that adds
functionality, complexity, storage or memory size, or runtime needs to
be considered for the chopping block.

> 
> 
>> I realize you said that with a smiley, so I'm not trying to be
>> dismissive or harsh, but I do think that if you want the features
>> of some other init system, then the right answer is to use them,
>> and not complain about this one.
> 
> I am not complaining, I have asked what the specific, quantifiable,
> objective of creating the home grown init 'procedure' is, IMHO that is
> an entirely reasonable question to an RFC, the answer to which should
> inform the design.

Agreed, a very valid question and one I am still working through the
numbers for. We save ~3k in busybox size by using SETSID and CTTYHACK
rather than the busybox INIT. This is less than 1% of the busybox
binary, not a huge savings. I'll look at the rest, but it's more
difficult to quantify as it is dependent on the number of sysvinit
scripts used - and of course the more that are used, the more likely the
image should be using sysvinit anyway. So... I'll look into this some more.

> 
> If we are talking about targeting 1MB of RAM then it is unavoidable that
> the system has to be hand crafted to a fair degree. It is not what I'd
> consider the typical Poky use case, but within those constrains, I'd
> probably be looking at eliminating shell scripts altogether in my
> customization of whatever Poky offers.

poky-tiny is looking to define a new typical use-case that differs from
the poky use case (poky and poky-tiny the DISTROs of course). Perhaps
you just meant Poky the project, in which case, I think the same point
applies, we're trying to extend the typical use cases to include very
small systems.

Your point on eliminating shell scripts is a good one though. My
approach here has been to make it as minimal as is practical while still
having a somewhat usable system "out of the box". To me this means a
shell is available, networking is supported, etc etc (see the poky
distro definition for more specifics).

Your main point is of course still valid: What do I get by dropping init
and is it worth being different for that? I'm still looking.

-- 
Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel

Re: RFC: poky-tiny: init procedure

[Darren Hart]

On 06/15/2012 02:10 PM, Tim Bird wrote:
> On 06/14/2012 12:11 AM, Tomas Frydrych wrote:
>> A system that runs nothing but a shell is really not useful for anything
>> all, everyone using it will be adding some sort of services, so the
>> question of how the extending works (or does not work), needs to be in
>> the forefront of the design. My main reservation is that you are
>> suggesting to break one of the basic premisses behind the whole
>> ecosystem, namely that if I add a package that provides a service to an
>> image, I get that service running; 'fix by documentation' is never a fix.
> 
> I didn't address this in my other response, and I
> think it's a valid observation.  I agree that
> everyone will want to add something to the init,
> and that 'fix by documentation' is, in general, a
> poor solution.
> 
> I don't have any good solution here (one that scales,
> or is likely to be adopted widely.)  You don't really
> need documentation, if you have the few lines of shell
> script that are necessary to start a service in minimal
> mode.  The standard init scripts handle lots of interactions
> with other services and runtime conditions.  This is why
> they are bloated (and the abstractions in them are
> almost incomprehensible to read.)
> 
> At Sony we've tried various things - like automatically
> stripping init scripts of uncalled branches, removing
> conditionals that don't apply, etc.  We've also looked
> at automatically converting init scripts to C code, to
> speed up execution.  You still end up with gobs of code
> that doesn't apply to your situation getting executed
> at boot time, an overly-complicated test scenario, and a
> certain fragility in the overall init system.
> 
> One of our early efforts, getting busybox
> to not fork commands in init scripts that it had as
> builtins, proved to be a significant feature that is still
> valuable today to improve boot time.
> 
> What we have settled on as a generic solution is basically
> what I described in response to Darren's initial post.
> You have a list of the short shell snippets required to start
> commonly-used services in /etc/rc.local.  These
> are commented out by default, but a developer or product
> team can uncomment them to enable the associated services.
> This keeps everything in one place and provides developers
> the commands needed to start things.  Customization to
> handle interactions or special cases can proceed from
> there.  It's not ideal, in that a developer may have to
> re-create or rediscover some configuration, setting, or
> start sequence to handle a more complicated usage scenario.
> But in order to hit our targets for size or boot time, it
> has been helpful to do things this way.  The developer is
> forced to manually enable each service, and the start
> sequence, commands and command parameters are obvious
> from the script.  (This is rarely true of init scripts).
> 

Does this mean you just don't worry about respawning services? If
the.... I dunno... ambient-light-sensor-daemon crashes, is the user
expected to just reboot the device?

> If anyone has good ideas or other experience for
> the init sequence for highly customized,
> resource-constrained products, I'd like to hear about
> them.

So Systemd does great things for deterministic boot with clearly defined
dependencies between services which also allows for as much parallelism
as possible (maybe that doesn't matter for your devices). I haven't
looked at how big systemd is what how much it would bloat my kernel to
support it. Have you?

As I understand it, it brings in things like udev and basically starts
to make my system look like a desktop/server. systemd-tiny ftw!

-- 
Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel

Re: RFC: poky-tiny: init procedure

[Tomas Frydrych]

Hi Darren,

On 16/06/12 00:15, Darren Hart wrote:
> I dont think
> Tim's comment was wrong there. Of course "big system" is subjective, to
> me that's anything over 4 MB of storage and 8MB of RAM, for Tim, that's
> 1 MB of RAM.

Indeed, I was thinking of something like the OpenRisc board
(http://opencores.org/shop,item,9), which is a 1MB flash / 32MB RAM,
which I now realize is a huge system by both your and Tim standards :)


> Your point on eliminating shell scripts is a good one though.

In my experience scripts tend show fairly visibly on poky bootchart,
that's all.

I was thinking that for a small system, where I know exactly what I
want, the rc.local file could probably be fairly easily replaced with a
custom config-less spawner written in C. If done with a view to permit
easy modification of the .c source file, it would not be any more hassle
customizing than tweaking a shell script. But you would loose the
ability to tweak config over ssh, which is a significant trade off for
development / debugging, so I think the rc.local is a better default
approach (and alternatives can always be dropped in using the
ALTERNATIVES mechanism anyway).

I think the busybox sysvinit is probably worth experimenting with to get
some idea what size of a system it would be a viable alternative for the
default, and document that somewhere.

Tomas

Re: RFC: poky-tiny: init procedure

[Richard Purdie]

On Fri, 2012-06-15 at 14:10 -0700, Tim Bird wrote:
> One of our early efforts, getting busybox
> to not fork commands in init scripts that it had as
> builtins, proved to be a significant feature that is still
> valuable today to improve boot time.

I'd add that one thing we did with Poky's init system a while ago was
went through and for every fork we found, tried to find ways to avoid
it. Certainly, a few years ago, we were down to about four busybox/shell
util (grep/sed/cat) forks in the init scripts through to an X desktop.

I've not looked at this recently but we certainly used to be pretty good
in that area.

We also source init scripts, not execute them where they have the
extension .sh, again for performance (less exec overhead). This all
stayed pretty much compatible with sysvinit and is what we use to this
day. I appreciate we do need to improve, there is plenty of room for it
and look at things like systemd.

Cheers,

Richard