[Buildroot] issues without busybox

[Buildroot] issues without busybox

[Gilles]

Folks,

Buildroot is a fantastic effort for small footprint devices. However, in my case - where I'd rather have something a bit more SystemV like with MMU, no busybox, a lot of things are broken.

Just naming a few I recently fixed to get past some issues:

- init network script S40network which tries to use ifup/ifdown only provided in ifupdown package not included.
- S45connman sript (if used) which relies on start-stop-daemon only provided with dpkg.

I understand that dependencies quickly add up to a nightmare and I for one would agree that it should be left up to the user of buildroot to figure them out.

What is the buildroot community feeling about this subject? I understand that the main focus is on busybox, is there a group of people interested in the non-busybox approach? I would be willing to contribute, after all, I have to do this work right now to get me up and going. This might benefit other people.

Cheers,
Gilles
.

[Buildroot] issues without busybox

[Thomas Petazzoni]

Dear Gilles,

On Mon, 21 Jul 2014 12:57:50 -0700, Gilles wrote:

> Buildroot is a fantastic effort for small footprint devices. However,
> in my case - where I'd rather have something a bit more SystemV like
> with MMU, no busybox, a lot of things are broken.
> 
> Just naming a few I recently fixed to get past some issues:
> 
> - init network script S40network which tries to use ifup/ifdown only
> provided in ifupdown package not included.
> - S45connman sript (if used) which relies on start-stop-daemon only
> provided with dpkg.
> 
> I understand that dependencies quickly add up to a nightmare and I
> for one would agree that it should be left up to the user of
> buildroot to figure them out.
> 
> What is the buildroot community feeling about this subject? I
> understand that the main focus is on busybox, is there a group of
> people interested in the non-busybox approach? I would be willing to
> contribute, after all, I have to do this work right now to get me up
> and going. This might benefit other people.

Thanks a lot for reporting those issues. Patches are definitely welcome
to fix this.

I believe one direction we should potentially investigate is to have
one common skeleton for the base stuff, and then separate additional
skeletons for busybox init, sysv init and systemd init.

Regarding the specific issues you're raising here, I'm not exactly sure
how to solve them:

 * For the network, we could make sysvinit depend on ifupdown, but this
   sounds a bit strong. Then it would mean that we should make the init
   script installation conditional. Or maybe installed just by ifupdown
   on one side, and busybox on the other side?

 * Regarding start-stop-daemon, I believe all (most?) our init scripts
   rely on start-stop-daemon. So I'm not sure how to handle that...

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] issues without busybox

[Gustavo Zacarias]

On 07/21/2014 05:15 PM, Thomas Petazzoni wrote:
> I believe one direction we should potentially investigate is to have
> one common skeleton for the base stuff, and then separate additional
> skeletons for busybox init, sysv init and systemd init.

Hi.
I think i've already mentioned i'm planning on a proposal to revamp the
init system/initscript options.
The idea would be to make them consistent and document how a proper
initscript should be made, and get them all in line with this.
Also interesting would be to make them configurable, in many cases
daemons have options and don't use a configuration file, so let's make
one i say. Actually let's make two :) A default file for read-only
filesystem, and some another that overrides the default, good for RO
filesystems which have a separate partition for that.
We could make the start/stop order configurable too via a file similar
to the device table, if this file lives in the target filesystem it
could be nicer too - but well that depends on how far we'd like to go.
The idea would be to use as much pure shell as possible for this to keep
necessary dependencies to a minimum.
Haven't thought out much of the systemd option yet, i need to dig
somewhat deeper into it, or it could be handled separately since it's
quite different from the usual inits.

> Regarding the specific issues you're raising here, I'm not exactly sure
> how to solve them:
> 
>  * For the network, we could make sysvinit depend on ifupdown, but this
>    sounds a bit strong. Then it would mean that we should make the init
>    script installation conditional. Or maybe installed just by ifupdown
>    on one side, and busybox on the other side?

We can make the different BR2_INIT_* options select what's appropiate,
if someone wants to "roll their own" they can select None.

>  * Regarding start-stop-daemon, I believe all (most?) our init scripts
>    rely on start-stop-daemon. So I'm not sure how to handle that...

We can throw a compatibility function/alias/script for different scenarios.
But i think getting what we want to do on a clean sheet would be best,
and then work on the patches.
Regards.

[Buildroot] issues without busybox

[Gilles]

On Jul 21, 2014, at 13:28 , Gustavo Zacarias <gustavo@zacarias.com.ar> wrote:

> On 07/21/2014 05:15 PM, Thomas Petazzoni wrote:
>> I believe one direction we should potentially investigate is to have
>> one common skeleton for the base stuff, and then separate additional
>> skeletons for busybox init, sysv init and systemd init.
> 
> Hi.
> I think i've already mentioned i'm planning on a proposal to revamp the
> init system/initscript options.
> The idea would be to make them consistent and document how a proper
> initscript should be made, and get them all in line with this.
> Also interesting would be to make them configurable, in many cases
> daemons have options and don't use a configuration file, so let's make
> one i say.


I don't know, there are already SO MANY alternatives to SystemV init with
event driven init like boot scripts, initNG, launchd (my favorite), SMF, runit, syscan, upstart, einit...
It's almost a full time job to stay on top of it all as it is.


> Actually let's make two :) A default file for read-only
> filesystem, and some another that overrides the default, good for RO
> filesystems which have a separate partition for that.
> We could make the start/stop order configurable too via a file similar
> to the device table, if this file lives in the target filesystem it
> could be nicer too - but well that depends on how far we'd like to go.
> The idea would be to use as much pure shell as possible for this to keep
> necessary dependencies to a minimum.
> Haven't thought out much of the systemd option yet, i need to dig
> somewhat deeper into it, or it could be handled separately since it's
> quite different from the usual inits.
> 
>> Regarding the specific issues you're raising here, I'm not exactly sure
>> how to solve them:
>> 
>> * For the network, we could make sysvinit depend on ifupdown, but this
>>   sounds a bit strong. Then it would mean that we should make the init
>>   script installation conditional. Or maybe installed just by ifupdown
>>   on one side, and busybox on the other side?
> 
> We can make the different BR2_INIT_* options select what's appropiate,
> if someone wants to "roll their own" they can select None.
> 
>> * Regarding start-stop-daemon, I believe all (most?) our init scripts
>>   rely on start-stop-daemon. So I'm not sure how to handle that...
> 
> We can throw a compatibility function/alias/script for different scenarios.
> But i think getting what we want to do on a clean sheet would be best,
> and then work on the patches.
> Regards.

[Buildroot] issues without busybox

[Gustavo Zacarias]

On 07/21/2014 06:35 PM, Gilles wrote:
> I don't know, there are already SO MANY alternatives to SystemV init with
> event driven init like boot scripts, initNG, launchd (my favorite), SMF, runit, syscan, upstart, einit...
> It's almost a full time job to stay on top of it all as it is.

Hi.
I'm not proposing making an init system itself, just bridging the
available options in buildroot in a consistent way.
These things could be done in separate steps:

1) Make initscript consistent.
I've been tweaking some a bit so as to use the return value and say if
it indeed started correctly instead of a blind echo.

2) Make them configurable.
Use a simple method such as sourcing /etc/default/$NAME for the default
variable assignments and /etc/config/$NAME to override for example.
Buildroot can ship /etc/default/$NAME which can be in a RO partition and
the user can (or not) have a RW partition mounted in /etc/config where
he can customize the initscript behaviour.
Easy to handle without any unification filesystem and easy to do a
factory reset.

3) Work around missing utilities like start-stop-daemon and such by
wrapping up such things in helper scripts that will call the appropiate
utility.

Something like that :)

Regards.

[Buildroot] issues without busybox

[Eric Le Bihan]

Gustavo, All,
On Mon, Jul 21, 2014 at 05:28:24PM -0300, Gustavo Zacarias wrote:
> On 07/21/2014 05:15 PM, Thomas Petazzoni wrote:
> > I believe one direction we should potentially investigate is to have
> > one common skeleton for the base stuff, and then separate additional
> > skeletons for busybox init, sysv init and systemd init.
>
> Hi.
> I think i've already mentioned i'm planning on a proposal to revamp the
> init system/initscript options.
> The idea would be to make them consistent and document how a proper
> initscript should be made, and get them all in line with this.
> Also interesting would be to make them configurable, in many cases
> daemons have options and don't use a configuration file, so let's make
> one i say. Actually let's make two :) A default file for read-only
> filesystem, and some another that overrides the default, good for RO
> filesystems which have a separate partition for that.

Would this look like what is done in package/dropbear/S50dropbear?

  test -r /etc/default/dropbear && . /etc/default/dropbear

The file /etc/default/dropbear defines $DROPBEAR_ARGS, which is used on the
command line of the program.

Would the location of this alternative configuration file be configurable from
"menuconfig"?

> We could make the start/stop order configurable too via a file similar
> to the device table, if this file lives in the target filesystem it
> could be nicer too - but well that depends on how far we'd like to go.
> The idea would be to use as much pure shell as possible for this to keep
> necessary dependencies to a minimum.
> Haven't thought out much of the systemd option yet, i need to dig
> somewhat deeper into it, or it could be handled separately since it's
> quite different from the usual inits.

Currently, only 8 packages install systemd unit files. The pattern is always
the same:

  define <pkg>_INSTALL_INIT_SYSTEMD
  	$(INSTALL) -D -m 644 package/<pkg>/<pkg>.service $(TARGET_DIR)/etc/systemd/system/<pkg>.service
  	mkdir -p $(TARGET_DIR)/etc/systemd/system/multi-user.target.wants
  	ln -fs ../<pkg>.service $(TARGET_DIR)/etc/systemd/system/multi-user.target.wants/<pkg>.service
  endef

I think about simplifying this by adding a script, also in the tradition of
makedevs and mkusers, which would be executed when generating the rootfs. This
script would install all the unit files declared like this:

  define <pkg>_SYSTEMD_UNITS
  	<unit file> <target unit>
  endef

For example, if <pkg> installs a service which is to be triggered when the
system reaches multi-user.target:

  define <pkg>_SYSTEMD_UNITS
  	package/<pkg>/<pkg>.service multi-user.target.wants
  endef

Could the same be done for SysV/Busybox init scripts?

  define <pkg>_SYSV_INIT_SCRIPTS
  	<init script> <default start/stop order>
  endef

For example, to install a SysV init script for Dropbear, we would declare:

  define DROPBEAR_SYSV_INIT_SCRIPTS
  	package/dropbear/dropbear 50
  endef

A script would install all init scripts and create the associated symbolic
links. The default start/stop order could be overriden by a custom
configuration file.


Best regards,
ELB

[Buildroot] issues without busybox

[Gustavo Zacarias]

On 07/22/2014 09:17 AM, Eric Le Bihan wrote:

>> Hi.
>> I think i've already mentioned i'm planning on a proposal to revamp the
>> init system/initscript options.
>> The idea would be to make them consistent and document how a proper
>> initscript should be made, and get them all in line with this.
>> Also interesting would be to make them configurable, in many cases
>> daemons have options and don't use a configuration file, so let's make
>> one i say. Actually let's make two :) A default file for read-only
>> filesystem, and some another that overrides the default, good for RO
>> filesystems which have a separate partition for that.
> 
> Would this look like what is done in package/dropbear/S50dropbear?
> 
>   test -r /etc/default/dropbear && . /etc/default/dropbear
> 
> The file /etc/default/dropbear defines $DROPBEAR_ARGS, which is used on the
> command line of the program.
> 
> Would the location of this alternative configuration file be configurable from
> "menuconfig"?

Yes, that's the spirit of it.
In it's simplest form:

NAME=dropbear
[ -f /etc/default/$NAME ] && source /etc/default/$NAME
[ -f /etc/config/$NAME ] && source /etc/config/$NAME

If default config exists parse it, then if override config exists also
parse it.
If some config is required we can either check for set minimum values or
not be so smart and just check that one was parsed:

[ -f /etc/default/$NAME ] && source /etc/default/$NAME && configured=1

And then check for the value of configured before attempting to start
anything to avoid unnecessary noise.

>> We could make the start/stop order configurable too via a file similar
>> to the device table, if this file lives in the target filesystem it
>> could be nicer too - but well that depends on how far we'd like to go.
>> The idea would be to use as much pure shell as possible for this to keep
>> necessary dependencies to a minimum.
>> Haven't thought out much of the systemd option yet, i need to dig
>> somewhat deeper into it, or it could be handled separately since it's
>> quite different from the usual inits.
> 
> Currently, only 8 packages install systemd unit files. The pattern is always
> the same:
> 
>   define <pkg>_INSTALL_INIT_SYSTEMD
>   	$(INSTALL) -D -m 644 package/<pkg>/<pkg>.service $(TARGET_DIR)/etc/systemd/system/<pkg>.service
>   	mkdir -p $(TARGET_DIR)/etc/systemd/system/multi-user.target.wants
>   	ln -fs ../<pkg>.service $(TARGET_DIR)/etc/systemd/system/multi-user.target.wants/<pkg>.service
>   endef
> 
> I think about simplifying this by adding a script, also in the tradition of
> makedevs and mkusers, which would be executed when generating the rootfs. This
> script would install all the unit files declared like this:
> 
>   define <pkg>_SYSTEMD_UNITS
>   	<unit file> <target unit>
>   endef
> 
> For example, if <pkg> installs a service which is to be triggered when the
> system reaches multi-user.target:
> 
>   define <pkg>_SYSTEMD_UNITS
>   	package/<pkg>/<pkg>.service multi-user.target.wants
>   endef

That's great.

> Could the same be done for SysV/Busybox init scripts?
> 
>   define <pkg>_SYSV_INIT_SCRIPTS
>   	<init script> <default start/stop order>
>   endef
> 
> For example, to install a SysV init script for Dropbear, we would declare:
> 
>   define DROPBEAR_SYSV_INIT_SCRIPTS
>   	package/dropbear/dropbear 50
>   endef
> 
> A script would install all init scripts and create the associated symbolic
> links. The default start/stop order could be overriden by a custom
> configuration file.

For SysV-style i think start and stop should be separate fields.
Most of the time you use reversed order for start vs stop, but for some
services you might not want to stop at all, for example some firewall
rules script.
Also using something similar to system/device_table.txt for initscripts
allows for some neat customization for users, they just need to point to
their own customized version to enable/disable services and change
order, a file that would live in board/ customizations easily.
I think the same could be done for systemd.
Alternatively we could just make it part of the config files and tweak
the rcK/rcS scripts to deal with this, something like:

/etc/default/dropbear:
DROPBEAR_START=YES
DROPBEAR_START_ORDER=20
DROPBEAR_STOP_ORDER=50

And then just override via /etc/config/dropbear if you don't want it:
DROPBEAR_START=NO

The user could even overstep on it from a post-build script, or from the
package file:

DROPBEAR_DEFAULT_CONFIG_FILES = dropbear

And just conditionally check if it exists in the skeleton to avoid
overwriting it, if not copy it.

Voila.
Regards.

[Buildroot] issues without busybox

[Peter Korsgaard]

>>>>> "Gilles" == Gilles  <gilles@whospot.com> writes:

 > Folks,

 > Buildroot is a fantastic effort for small footprint devices. However,
 > in my case - where I'd rather have something a bit more SystemV like
 > with MMU, no busybox, a lot of things are broken.

 > Just naming a few I recently fixed to get past some issues:

 > - init network script S40network which tries to use ifup/ifdown only provided in ifupdown package not included.
 > - S45connman sript (if used) which relies on start-stop-daemon only provided with dpkg.

I don't quite understand why you want to get completely rid of
busybox? It is not like it is very big or anything.

As you have noticed, buildroot today implicitly requires busybox in the
rootfs. You can replace various components provided by busybox
(sysvinit, coreutils, bash, ..), but you still need busybox for
ifup/ifdown and start-stop-daemon (and probably others) - Is that a real
problem?

-- 
Bye, Peter Korsgaard