[Buildroot] Udev on a live CD system does not appear to populate /dev

[Buildroot] Udev on a live CD system does not appear to populate /dev

[Antony Vennard]

Hi All,

I'm currently looking at using udev for a live cd environment built with
buildroot, and I'm having trouble getting udev to pick up the machine's
devices.

I have some questions and some details that will (hopefully) help
identify my problem

Firstly, my kernel is configured to use devtmpfs:

681:CONFIG_DEVTMPFS=y
682:CONFIG_DEVTMPFS_MOUNT=y

so as I understand it, /dev should be mounted as a tmpfs by the kernel
and populated with /dev/null, /dev/console etc. Yet, my
buildroot/output/target contains a dev folder with subdirectories
created by buildroot.

So, I tried both creating the entries with target/generic_device_table
for tty0, tty1, console, null. This works, and udev reports it is
scanning for devices and done, yet /dev never picks up e.g. /dev/sda, or
/dev/disk/..., or so on. I have, in this configuration, copied
/usr/lib/udev/rules.d/* to buildroot/output/target/etc/udev/rules.d to
no avail; as far as I can see whatever rules I supply, even from my
Fedora or Ubuntu distributions, do not seem to affect udev.

Wondering if this creation method was wrong, I have then removed all of
the entries in /dev so that the kernel can mount its own devtmpfs. When
booting, /dev/null and /dev/tty1 are reported as missing. I imagine
there is nothing in /dev at this stage

Various possibly related points:
 * I thought udev required udevd, yet I can't see that either running or
available on the ISO, nor can I find a configuration option to create it.
 * I have D-Bus built for the system, as I read udev required it.
 * I can see sda1 in /sys/block/sda/sda1; likewise for other attached
storage devices.

Am I doing anything very wrong? I get the feeling I must be missing
something for this to not work, yet I'm not sure what.

Can anyone offer any advice? Happy to provide config snippets etc.

Using buildroot 2012.08.

Thanks,

Antony


-- 
________________

Macrium? Software is a trading name of Paramount Software UK Ltd, 
Registered in England and Wales Number 02973414
Registered Office: Kilburn House, Manchester Science Park, Lloyd St. North, 
Manchester M15 6SE United Kingdom

The information contained in this e-mail is confidential, privileged, or 
otherwise protected from disclosure. It is intended only for the use of the 
authorized individual as indicated in the e-mail. Any unauthorized 
disclosure, copying, distribution or taking of any action based on the 
contents of this material is strictly prohibited.

If you have received this e-mail in error, please delete it immediately.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 895 bytes
Desc: OpenPGP digital signature
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20121106/29afbc10/attachment.asc>

[Buildroot] Udev on a live CD system does not appear to populate /dev

[Stefan Fröberg]

6.11.2012 16:02, Antony Vennard kirjoitti:
> Hi All,
>
> I'm currently looking at using udev for a live cd environment built with
> buildroot, and I'm having trouble getting udev to pick up the machine's
> devices.
>
> I have some questions and some details that will (hopefully) help
> identify my problem
>
> Firstly, my kernel is configured to use devtmpfs:
>
> 681:CONFIG_DEVTMPFS=y
> 682:CONFIG_DEVTMPFS_MOUNT=y
>
> so as I understand it, /dev should be mounted as a tmpfs by the kernel
> and populated with /dev/null, /dev/console etc. Yet, my
> buildroot/output/target contains a dev folder with subdirectories
> created by buildroot.
>
> So, I tried both creating the entries with target/generic_device_table
> for tty0, tty1, console, null. This works, and udev reports it is
> scanning for devices and done, yet /dev never picks up e.g. /dev/sda, or
> /dev/disk/..., or so on. I have, in this configuration, copied
> /usr/lib/udev/rules.d/* to buildroot/output/target/etc/udev/rules.d to
> no avail; as far as I can see whatever rules I supply, even from my
> Fedora or Ubuntu distributions, do not seem to affect udev.
>
> Wondering if this creation method was wrong, I have then removed all of
> the entries in /dev so that the kernel can mount its own devtmpfs. When
> booting, /dev/null and /dev/tty1 are reported as missing. I imagine
> there is nothing in /dev at this stage
Im not sure but I tought that device nodes /dev/null and /dev/console
(and some others)
have to always be created manually
(either with buildroot device tables, by doing the mknod thing or just
by copying those
nodes from host system to output/target/dev or fs/skeleton/dev)
to be of any use.

At least that's what I have done always, created those /dev/null and
/dev/cosole and also
/dev/ram0 etc.. myself into output/target/dev and then make my initramfs
from that directory and
 let the hotplug program (either busybox's own mdev or udev) do the rest.

Here's what I have minimally in my own buildroot's fs/skeleton/dev
folder (which is copied at the end of the buildroot
process into output/target/dev )

crw-------.  1 root root 5, 1 11.1.2012 console
lrwxrwxrwx.  1 root root   10 19.3.2012 log -> ../tmp/log
brw-rw----.  1 root root 7, 0 20.8. 00:58 loop0
crw-------.  1 root root 1, 1 11.1.2012 mem
crw-r--r--.  1 root root 1, 3 12.1.2012 null
drwxr-xr-x.  2 root root 1024 12.1.2012 pts
brw-r-----.  1 root root 5, 0 12.1.2012 ram0
crw-r--r--.  1 root root 5, 0 12.1.2012 tty
crw-------.  1 root root 4, 0 11.1.2012 tty0
crw-------.  1 root root 4, 1 11.1.2012 tty1
crw-------.  1 root root 1, 5 11.1.2012 zero

Now, some of those might not be strictly necessary and should be created
by mdev/udev.
But it's good to have them just in case.

Besides some pre-made device nodes, make sure that you have S10udev in your
output/target/etc/init.d.
That is the one responsible of starting udev.

You can also copy S10udev to fs/skeleton/etc/init.d to make sure that
it's included in
output/target/etc/init.d everytime you make complete new rebuild
(make clean && make)


Im presuming you *are* using initramfs in your early device initialization ?
(if not then you absolutely should. Live CD without initramfs is useless)

Best regards
Stefan
> Various possibly related points:
>  * I thought udev required udevd, yet I can't see that either running or
> available on the ISO, nor can I find a configuration option to create it.
>  * I have D-Bus built for the system, as I read udev required it.
>  * I can see sda1 in /sys/block/sda/sda1; likewise for other attached
> storage devices.
>
> Am I doing anything very wrong? I get the feeling I must be missing
> something for this to not work, yet I'm not sure what.
>
> Can anyone offer any advice? Happy to provide config snippets etc.
>
> Using buildroot 2012.08.
>
> Thanks,
>
> Antony
>
>
>
>
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20121106/3d142432/attachment.html>

[Buildroot] Udev on a live CD system does not appear to populate /dev

[Arnout Vandecappelle]

On 06/11/12 15:02, Antony Vennard wrote:
> Hi All,
>
> I'm currently looking at using udev for a live cd environment built with
> buildroot, and I'm having trouble getting udev to pick up the machine's
> devices.
>
> I have some questions and some details that will (hopefully) help
> identify my problem

  It would help if you can do 'make savedefconfig' and post the
resulting defconfig file.

>
> Firstly, my kernel is configured to use devtmpfs:
>
> 681:CONFIG_DEVTMPFS=y
> 682:CONFIG_DEVTMPFS_MOUNT=y
>
> so as I understand it, /dev should be mounted as a tmpfs by the kernel
> and populated with /dev/null, /dev/console etc.

  Since you're making a live CD, you'll probably boot with an
initrd/initramfs.  The kernel then doesn't mount /dev automatically,
it has to be done by init script in /init (which is called instead of
/sbin/init for an initramfs).  Buildroot creates this script for you
when you build a cpio or initramfs filesystem (i.e. the things that are
typically used as initrd image).  If you're trying to use an ext2 as
an initrd (which is a bad idea), you have to copy fs/cpio/init to the
target yourself.

> Yet, my
> buildroot/output/target contains a dev folder with subdirectories
> created by buildroot.

  Yes, that's a bit silly.  It doesn't really matter, because the devtmpfs
gets mounted over it and those files are no longer visible.

  If you have an initrd/initramfs, /dev/console should ideally be there,
otherwise you get the message 'Failed to open an initial console'.  That
message is harmless though because the /init script mounts /dev, which
implicitly creates /dev/console, and when /sbin/init is execed it can use
this /dev/console.

> So, I tried both creating the entries with target/generic_device_table
> for tty0, tty1, console, null. This works,

  Yep, this sounds like you're missing a /init.

> and udev reports it is scanning for devices and done,
> yet /dev never picks up e.g. /dev/sda, or
> /dev/disk/..., or so on. I have, in this configuration, copied
> /usr/lib/udev/rules.d/* to buildroot/output/target/etc/udev/rules.d to
> no avail; as far as I can see whatever rules I supply, even from my
> Fedora or Ubuntu distributions, do not seem to affect udev.

  Note that udev doesn't create anything in /dev - it relies on the kernel
to create the device nodes. udev only does renaming, chmodding, chowning
and making symlinks.

>
> Wondering if this creation method was wrong, I have then removed all of
> the entries in /dev so that the kernel can mount its own devtmpfs. When
> booting, /dev/null and /dev/tty1 are reported as missing. I imagine
> there is nothing in /dev at this stage

  This is weird.  If there's no /dev/console, how can init report that
/dev/tty1 is missing?

[snip]

  Regards,
  Arnout
-- 
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 +32-16-286540
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] Udev on a live CD system does not appear to populate /dev

[Arnout Vandecappelle]

On 06/11/12 19:37, Stefan Fr?berg wrote:
> Here's what I have minimally in my own buildroot's fs/skeleton/dev folder (which is copied at the end of the buildroot
> process into output/target/dev )

  Err... That can only work if you have sudo in your post-build script - which
we consider bad practice.  We have the device_table to deal with device nodes.
And all of that shouldn't be necessary if you use devtmpfs (or udev or mdev,
which implies devtmpfs).  If you use static device nodes, then the default
device_table_dev creates all of those.

  If that doesn't work for you, then we should try to fix it in buildroot.

  Regards,
  Arnout
-- 
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 +32-16-286540
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] Udev on a live CD system does not appear to populate /dev

[Stefan Fröberg]

7.11.2012 1:46, Arnout Vandecappelle kirjoitti:
> On 06/11/12 19:37, Stefan Fr?berg wrote:
>> Here's what I have minimally in my own buildroot's fs/skeleton/dev
>> folder (which is copied at the end of the buildroot
>> process into output/target/dev )
>
>  Err... That can only work if you have sudo in your post-build script
> - which
> we consider bad practice.  We have the device_table to deal with
> device nodes.
> And all of that shouldn't be necessary if you use devtmpfs (or udev or
> mdev,
> which implies devtmpfs).  If you use static device nodes, then the
> default
> device_table_dev creates all of those.
>
Nope, no sudo in my live Cd.
Just few pre-made device nodes in fs/skeleton/dev and my special made
init stuff in fs/skeleton/etc/init.d

Well ok, I confess ... I *did* modify /etc/init.d/rcS a little bit for
my needs and also added unionfs mounting to S15mount
and some few other little things

So far I have been very happy with my own personal Live CD Linux distro.
It has 873 MB worth of various userspace utilities in 700 MB CD-R disc.
Feat which would have not been posible without initramfs (XZ
compressed), unionfs and squashfs.

The lowest RAM amount I managed to start it was 192 MB.
With Xorg, Fluxbox and Firefox :-)

Without buildroot and my lil tweakings I could have never made it.

Best regards
Stefan


>  If that doesn't work for you, then we should try to fix it in buildroot.
>
>  Regards,
>  Arnout

[Buildroot] Udev on a live CD system does not appear to populate /dev

[Antony Vennard]

Hi All,

I'm not using initramfs - I'm using the ISO 9660 target which as far as I
can see produces an initrd, a kernel and an ext2 raw image which it mounts
as a ramdisk. If I select the initramfs options, the cpio filesystem is
generated prior to the iso9660 one, but the initramfs afterwards! Which
means, as far as I can see, that initramfs would not be placed on the disk
- unless buildroot manipulates the iso filesystem post creation? I could of
course mkisofs myself, but I was trying to avoid that - although if this
fixes udev, I'll do it.

I've nothing /dev related in my fs/skeleton, and only the default values in
device_table.txt. Since I'm using udev, I did expect it would populate
everything it needed in /dev - although unless I add entries for null,
console and a tty into device_table.txt, I end up with a distribution that
loops forever saying "could not find /dev/tty1" (if I've configured tty1 as
the location for getty for example). I can't mitigate this by re-writing
the inittab to launch getty "once" as opposed to respawn, since buildroot
rewrites that - and askfirst:/bin/sh works, except that the respawn error
messages are written all over /dev/console, making it unusable...!

By the sounds of things this is wrong, but I'm currently attempting to
preconfigure some devices in device_table.txt - e.g. /dev/null,
/dev/urandom et al pseudo devices, and I'm borrowing Ubuntu's rules.d to
see if these are any more effective than Fedora's. Leaving device_table.txt
as its default causes the tty1 missing loop. I am not expecting this to
work - I cannot see how udev rules differ greatly between distributions -
just eliminating it as a possible cause.

Any further ideas as to what I can try? Is mdev a better choice?

Thanks for your help,

Antony

-- 
________________

Macrium(tm) Software is a trading name of Paramount Software UK Ltd, 
Registered in England and Wales Number 02973414
Registered Office: Kilburn House, Manchester Science Park, Lloyd St. North, 
Manchester M15 6SE United Kingdom

The information contained in this e-mail is confidential, privileged, or 
otherwise protected from disclosure. It is intended only for the use of the 
authorized individual as indicated in the e-mail. Any unauthorized 
disclosure, copying, distribution or taking of any action based on the 
contents of this material is strictly prohibited.

If you have received this e-mail in error, please delete it immediately.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20121108/af2cc0e5/attachment.html>

[Buildroot] Udev on a live CD system does not appear to populate /dev

[Stefan Fröberg]

Hi Antony

Just one guestion: What bootloader are you using for your Live CD ?
(I use grub)

8.11.2012 11:56, Antony Vennard kirjoitti:
> Hi All,
>
> I'm not using initramfs - I'm using the ISO 9660 target which as far
> as I can see produces an initrd, a kernel and an ext2 raw image which
> it mounts as a ramdisk. If I select the initramfs options, the cpio
> filesystem is generated prior to the iso9660 one, *but the initramfs
> afterwards!* Which means, as far as I can see, that initramfs would
> not be placed on the disk - unless buildroot manipulates the iso
> filesystem post creation? I could of course mkisofs myself, but I was
> trying to avoid that - although if this fixes udev, I'll do it.
>

Keep in mind that the buildroot way of making initramfs (that is if you
select intramfs from menuconfig) is
to embed your initramfs directly into kernel.

There are three disadvantages in here:

1 ) It will make your kernel fatter
2 ) It will makes things more complicated than necessary
You might have two initramfs settings: One that is embedded directly
into your kernel and
another that you have made by hand by yourself. And it's very possible
that they could have conflicting stuff between them
You can have several initramfs files (at least with grub bootloader),
the last ones overlaying the first ones, in order.
3 ) It will makes less flexibe.
 
All kernels since 2.6 have initramfs inside them, it's just that by
default that environment is empty.

Embedding initramfs directly to kernel might be good for those users who
don't use any bootloader or are using a bootloader which can't
be feeded external, separate initramfs file(s) (like grub) and want to
have minimal system always available (as piggybacked inside kernel).


> I've nothing /dev related in my fs/skeleton, and only the default
> values in device_table.txt. Since I'm using udev, I did expect it
> would populate everything it needed in /dev - although unless I add
> entries for null, console and a tty into device_table.txt, I end up
> with a distribution that loops forever saying "could not find
> /dev/tty1" (if I've configured tty1 as the location for getty for
> example). I can't mitigate this by re-writing the inittab to launch
> getty "once" as opposed to respawn, since buildroot rewrites that -
> and askfirst:/bin/sh works, except that the respawn error messages are
> written all over /dev/console, making it unusable...!
>
> By the sounds of things this is wrong, but I'm currently attempting to
> preconfigure some devices in device_table.txt - e.g. /dev/null,
> /dev/urandom et al pseudo devices, and I'm borrowing Ubuntu's rules.d
> to see if these are any more effective than Fedora's. Leaving
> device_table.txt as its default causes the tty1 missing loop. I am not
> expecting this to work - I cannot see how udev rules differ greatly
> between distributions - just eliminating it as a possible cause.
>
> Any further ideas as to what I can try? Is mdev a better choice?
>
Well mdev is at least easier to configure than udev.

The only thing you must make sure in your init process is that /proc and
/sys are mounted as early as possible (/etc/inittab is good place for that)
and that echo /sbin/mdev > /proc/sys/kernel/hotplug   is also somewhere
in /etc/init.d/rcS (maybe at the start of that file)

And check that you have etc/mdev.conf and etc/init.d/S10mdev (witch is
called by /etc/init.d/rcS)

This is what I would do (and have done with my live CD)
Forget the buildroot way of making initramfs for now and see if you can
construct live CD by hand:

1 ) make clean

2 ) make sure you have all the needed init stuff in  fs/skeleton/etc
directory (inittab, init.d/rcS, init.d/S* files etc.....)
     also make sure that sbin/init is a symlink to /bin/busybox
(I have also symlink of init --> /bin/busybox in top of fs/skeleton but
I not sure if that is stricly needed ...
I think kernel should be looking from /sbin/init if no other init found)

3 )  do make menuconfig as you would normally but remember to unselect
cpio, initramfs and iso9660 from menuconfig
       remember to select grub from under Bootloader section if you
don't already have it
       don't select too much of stuff even if you have massive amounts 
of RAM in your test systems.
       reason being that you are making initramfs later here from your
output/target directory and if you have massive
      amount's of stuff in there then it will take forever to boot. (I
have managed to do live CD with massive amounts of stuff on it and
     booting fast with only 192 MB RAM but that needed some major hacking)

4 ) do    make linux-menuconfig  and check that under General setup the
setting Initramfs source files is empty
     also check the compression support here that you want to use for
your external initramfs (let's suppose its' the default GZ for now)

5 ) do make

6 ) after buildroot has finished make sure that everything is okay in
your output/target (check especially boot/ etc/ and /dev directory)
     also make sure you have /boot/grub/menu.lst or /boot/grub/grub.conf
(the other is usually just symlink to other) and check it's settings
    (If you are not familiar with grub then I can help with it's
settings, especially important is the line where you tell grub the external
   initramfs file)
     remove the symlink (iso9660 format does not support symlinks)

7 ) Create directory where you want to put your kernel & bootloader
(a.ka. the whole contents of /boot dir) and external initramfs file
     that is generated next:   
    
      mkdir /livecd && cp -r output/target/boot /livecd

8 ) from output/target dir   do    find .  -not \( -path "./boot" -prune
\) -print0 | cpio --null -ov --format=newc | gzip -9 > /livecd/initramfs.gz
(we don't need boot directory inside initramfs. it needs to stay outside
of it. that why it was copied previously)

9 ) make iso9660 normally from the contents of your /livecd
mkdir /livecd_output && cd /livecd && mkisofs --joliet -R -b
boot/grub/stage2_eltorito -no-emul-boot -boot-load-size 4
-boot-info-table -iso-level 4 -hide-rr-moved -c boot.catalog -o
/livecd_output/your_livecd.iso .

(remember that . at the end)

10) burn the iso to CD or even better use qemu or VirtualBox to boot
your iso without wasting CD disc.

If this manual way worked then next try buildroot way.

Best regards
Stefan

> Thanks for your help,
>
> Antony
>
> ________________
>
> Macrium^(TM) Software is a trading name of Paramount Software UK Ltd,
> Registered in England and Wales Number 02973414
> Registered Office: Kilburn House, Manchester Science Park, Lloyd St.
> North, Manchester M15 6SE United Kingdom
>
> The information contained in this e-mail is confidential, privileged,
> or otherwise protected from disclosure. It is intended only for the
> use of the authorized individual as indicated in the e-mail. Any
> unauthorized disclosure, copying, distribution or taking of any action
> based on the contents of this material is strictly prohibited.
>
> If you have received this e-mail in error, please delete it immediately.
>
>
>
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20121108/d5fa34a3/attachment.html>

[Buildroot] Udev on a live CD system does not appear to populate /dev

[Arnout Vandecappelle]

On 11/08/12 10:56, Antony Vennard wrote:
> I'm not using initramfs - I'm using the ISO 9660 target which as far as I can see produces an initrd, a kernel and an
> ext2 raw image which it mounts as a ramdisk.

  Right!  I completely forgot that!  The iso9660 target is currently broken
(and has been for quite some time already).  For the time being, you have to
manually copy fs/cpio/init to the target (either copy it to $1/init in the
post-build script, or copy it to skeleton/init and do a clean build).

  Or, you could modify fs/iso9660/iso9660.mk and Config.in to use a cpio
archive as rootfs instead of an ext2.  If you do that and it works, please
post a patch for inclusion in buildroot, thanks!

  Regards,
  Arnout
-- 
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 +32-16-286540
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] Udev on a live CD system does not appear to populate /dev

[Antony Vennard]

Hi Stefan,

Thanks for that. I have followed your instructions for making a live cd
manually with mkisofs - actually this is helpful since a later job is to
add UEFI booting to the disk, which is easier this way (although I am
going to look at modifying buildroot, if I can).

I am using grub, and have modified my grub command line to use:

title LiveCD
kernel /vmlinuz vga=0x317
initrd /initramfs.bz2

The kernel has bz2, xz and gzip initramfs support built in - so all good
there. However, the kernel complains there's no root parameter. What
should I specify for this?

Thanks,

Antony

On 08/11/2012 12:35, Stefan Fr?berg wrote:
> Hi Antony
> 
> Just one guestion: What bootloader are you using for your Live CD ?
> (I use grub)
> 
> 8.11.2012 11:56, Antony Vennard kirjoitti:
>> Hi All,
>>
>> I'm not using initramfs - I'm using the ISO 9660 target which as far
>> as I can see produces an initrd, a kernel and an ext2 raw image which
>> it mounts as a ramdisk. If I select the initramfs options, the cpio
>> filesystem is generated prior to the iso9660 one, *but the initramfs
>> afterwards!* Which means, as far as I can see, that initramfs would
>> not be placed on the disk - unless buildroot manipulates the iso
>> filesystem post creation? I could of course mkisofs myself, but I was
>> trying to avoid that - although if this fixes udev, I'll do it.
>>
> 
> Keep in mind that the buildroot way of making initramfs (that is if you
> select intramfs from menuconfig) is
> to embed your initramfs directly into kernel.
> 
> There are three disadvantages in here:
> 
> 1 ) It will make your kernel fatter
> 2 ) It will makes things more complicated than necessary
> You might have two initramfs settings: One that is embedded directly
> into your kernel and
> another that you have made by hand by yourself. And it's very possible
> that they could have conflicting stuff between them
> You can have several initramfs files (at least with grub bootloader),
> the last ones overlaying the first ones, in order.
> 3 ) It will makes less flexibe.
>  
> All kernels since 2.6 have initramfs inside them, it's just that by
> default that environment is empty.
> 
> Embedding initramfs directly to kernel might be good for those users who
> don't use any bootloader or are using a bootloader which can't
> be feeded external, separate initramfs file(s) (like grub) and want to
> have minimal system always available (as piggybacked inside kernel).
> 
> 
>> I've nothing /dev related in my fs/skeleton, and only the default
>> values in device_table.txt. Since I'm using udev, I did expect it
>> would populate everything it needed in /dev - although unless I add
>> entries for null, console and a tty into device_table.txt, I end up
>> with a distribution that loops forever saying "could not find
>> /dev/tty1" (if I've configured tty1 as the location for getty for
>> example). I can't mitigate this by re-writing the inittab to launch
>> getty "once" as opposed to respawn, since buildroot rewrites that -
>> and askfirst:/bin/sh works, except that the respawn error messages are
>> written all over /dev/console, making it unusable...!
>>
>> By the sounds of things this is wrong, but I'm currently attempting to
>> preconfigure some devices in device_table.txt - e.g. /dev/null,
>> /dev/urandom et al pseudo devices, and I'm borrowing Ubuntu's rules.d
>> to see if these are any more effective than Fedora's. Leaving
>> device_table.txt as its default causes the tty1 missing loop. I am not
>> expecting this to work - I cannot see how udev rules differ greatly
>> between distributions - just eliminating it as a possible cause.
>>
>> Any further ideas as to what I can try? Is mdev a better choice?
>>
> Well mdev is at least easier to configure than udev.
> 
> The only thing you must make sure in your init process is that /proc and
> /sys are mounted as early as possible (/etc/inittab is good place for that)
> and that echo /sbin/mdev > /proc/sys/kernel/hotplug   is also somewhere
> in /etc/init.d/rcS (maybe at the start of that file)
> 
> And check that you have etc/mdev.conf and etc/init.d/S10mdev (witch is
> called by /etc/init.d/rcS)
> 
> This is what I would do (and have done with my live CD)
> Forget the buildroot way of making initramfs for now and see if you can
> construct live CD by hand:
> 
> 1 ) make clean
> 
> 2 ) make sure you have all the needed init stuff in  fs/skeleton/etc
> directory (inittab, init.d/rcS, init.d/S* files etc.....)
>      also make sure that sbin/init is a symlink to /bin/busybox
> (I have also symlink of init --> /bin/busybox in top of fs/skeleton but
> I not sure if that is stricly needed ...
> I think kernel should be looking from /sbin/init if no other init found)
> 
> 3 )  do make menuconfig as you would normally but remember to unselect
> cpio, initramfs and iso9660 from menuconfig
>        remember to select grub from under Bootloader section if you
> don't already have it
>        don't select too much of stuff even if you have massive amounts 
> of RAM in your test systems.
>        reason being that you are making initramfs later here from your
> output/target directory and if you have massive
>       amount's of stuff in there then it will take forever to boot. (I
> have managed to do live CD with massive amounts of stuff on it and
>      booting fast with only 192 MB RAM but that needed some major hacking)
> 
> 4 ) do    make linux-menuconfig  and check that under General setup the
> setting Initramfs source files is empty
>      also check the compression support here that you want to use for
> your external initramfs (let's suppose its' the default GZ for now)
> 
> 5 ) do make
> 
> 6 ) after buildroot has finished make sure that everything is okay in
> your output/target (check especially boot/ etc/ and /dev directory)
>      also make sure you have /boot/grub/menu.lst or /boot/grub/grub.conf
> (the other is usually just symlink to other) and check it's settings
>     (If you are not familiar with grub then I can help with it's
> settings, especially important is the line where you tell grub the external
>    initramfs file)
>      remove the symlink (iso9660 format does not support symlinks)
> 
> 7 ) Create directory where you want to put your kernel & bootloader
> (a.ka. the whole contents of /boot dir) and external initramfs file
>      that is generated next:   
>     
>       mkdir /livecd && cp -r output/target/boot /livecd
> 
> 8 ) from output/target dir   do    find .  -not \( -path "./boot" -prune
> \) -print0 | cpio --null -ov --format=newc | gzip -9 > /livecd/initramfs.gz
> (we don't need boot directory inside initramfs. it needs to stay outside
> of it. that why it was copied previously)
> 
> 9 ) make iso9660 normally from the contents of your /livecd
> mkdir /livecd_output && cd /livecd && mkisofs --joliet -R -b
> boot/grub/stage2_eltorito -no-emul-boot -boot-load-size 4
> -boot-info-table -iso-level 4 -hide-rr-moved -c boot.catalog -o
> /livecd_output/your_livecd.iso .
> 
> (remember that . at the end)
> 
> 10) burn the iso to CD or even better use qemu or VirtualBox to boot
> your iso without wasting CD disc.
> 
> If this manual way worked then next try buildroot way.
> 
> Best regards
> Stefan
> 
>> Thanks for your help,
>>
>> Antony
>>
>> ________________
>>
>> Macrium? Software is a trading name of Paramount Software UK Ltd,
>> Registered in England and Wales Number 02973414
>> Registered Office: Kilburn House, Manchester Science Park, Lloyd St.
>> North, Manchester M15 6SE United Kingdom
>>
>> The information contained in this e-mail is confidential, privileged,
>> or otherwise protected from disclosure. It is intended only for the
>> use of the authorized individual as indicated in the e-mail. Any
>> unauthorized disclosure, copying, distribution or taking of any action
>> based on the contents of this material is strictly prohibited.
>>
>> If you have received this e-mail in error, please delete it immediately.
>>
>>
>>
>> _______________________________________________
>> buildroot mailing list
>> buildroot at busybox.net
>> http://lists.busybox.net/mailman/listinfo/buildroot
> 

-- 
Antony Vennard
Software Engineer

Macrium Software
Kilburn House,
Manchester Science Park
Lloyd Street North
Manchester M15 6SE
UK

Tel: +44(0)161 226 1128

Web       http://www.macrium.com/
Blog      http://blog.macrium.com/
Twitter   http://twitter.com/MacriumReflect
Facebook  http://www.facebook.com/Macrium


-- 
________________

Macrium? Software is a trading name of Paramount Software UK Ltd, 
Registered in England and Wales Number 02973414
Registered Office: Kilburn House, Manchester Science Park, Lloyd St. North, 
Manchester M15 6SE United Kingdom

The information contained in this e-mail is confidential, privileged, or 
otherwise protected from disclosure. It is intended only for the use of the 
authorized individual as indicated in the e-mail. Any unauthorized 
disclosure, copying, distribution or taking of any action based on the 
contents of this material is strictly prohibited.

If you have received this e-mail in error, please delete it immediately.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 895 bytes
Desc: OpenPGP digital signature
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20121108/5730efff/attachment.asc>

[Buildroot] Udev on a live CD system does not appear to populate /dev

[Stefan Fröberg]

8.11.2012 18:02, Antony Vennard kirjoitti:
> Hi Stefan,
>
> Thanks for that. I have followed your instructions for making a live cd
> manually with mkisofs - actually this is helpful since a later job is to
> add UEFI booting to the disk, which is easier this way (although I am
> going to look at modifying buildroot, if I can).
>
> I am using grub, and have modified my grub command line to use:
>
> title LiveCD
> kernel /vmlinuz vga=0x317
> initrd /initramfs.bz2
>
> The kernel has bz2, xz and gzip initramfs support built in - so all good
> there. However, the kernel complains there's no root parameter. What
> should I specify for this?
root=/dev/ram0

So that would be:
kernel /vmlinuz vga=0x317 root=/dev/ram0

If you want to you can give it ramdisk_size parameter (from
https://www.kernel.org/doc/Documentation/kernel-parameters.txt)
but that should not be nowadays necessary.

Just make sure that you have enough ram memory for total amount of the
size (uncompressed) of your initramfs  + some ram needed to actually run
your applications.

and make sure that /dev/ram0 is in fs/skeleton/dev (or output/target/dev
if you hollowed my instructions) and
regenerate initramfs

If you hit the limit of your CD disc (or RAM) but want to still add some
more stuff into it then
me know. I have solved it, but like I said, it's hackis and dirty method :-)

Best regards
Stefan

> Thanks,
>
> Antony
>
> On 08/11/2012 12:35, Stefan Fr?berg wrote:
>> Hi Antony
>>
>> Just one guestion: What bootloader are you using for your Live CD ?
>> (I use grub)
>>
>> 8.11.2012 11:56, Antony Vennard kirjoitti:
>>> Hi All,
>>>
>>> I'm not using initramfs - I'm using the ISO 9660 target which as far
>>> as I can see produces an initrd, a kernel and an ext2 raw image which
>>> it mounts as a ramdisk. If I select the initramfs options, the cpio
>>> filesystem is generated prior to the iso9660 one, *but the initramfs
>>> afterwards!* Which means, as far as I can see, that initramfs would
>>> not be placed on the disk - unless buildroot manipulates the iso
>>> filesystem post creation? I could of course mkisofs myself, but I was
>>> trying to avoid that - although if this fixes udev, I'll do it.
>>>
>> Keep in mind that the buildroot way of making initramfs (that is if you
>> select intramfs from menuconfig) is
>> to embed your initramfs directly into kernel.
>>
>> There are three disadvantages in here:
>>
>> 1 ) It will make your kernel fatter
>> 2 ) It will makes things more complicated than necessary
>> You might have two initramfs settings: One that is embedded directly
>> into your kernel and
>> another that you have made by hand by yourself. And it's very possible
>> that they could have conflicting stuff between them
>> You can have several initramfs files (at least with grub bootloader),
>> the last ones overlaying the first ones, in order.
>> 3 ) It will makes less flexibe.
>>  
>> All kernels since 2.6 have initramfs inside them, it's just that by
>> default that environment is empty.
>>
>> Embedding initramfs directly to kernel might be good for those users who
>> don't use any bootloader or are using a bootloader which can't
>> be feeded external, separate initramfs file(s) (like grub) and want to
>> have minimal system always available (as piggybacked inside kernel).
>>
>>
>>> I've nothing /dev related in my fs/skeleton, and only the default
>>> values in device_table.txt. Since I'm using udev, I did expect it
>>> would populate everything it needed in /dev - although unless I add
>>> entries for null, console and a tty into device_table.txt, I end up
>>> with a distribution that loops forever saying "could not find
>>> /dev/tty1" (if I've configured tty1 as the location for getty for
>>> example). I can't mitigate this by re-writing the inittab to launch
>>> getty "once" as opposed to respawn, since buildroot rewrites that -
>>> and askfirst:/bin/sh works, except that the respawn error messages are
>>> written all over /dev/console, making it unusable...!
>>>
>>> By the sounds of things this is wrong, but I'm currently attempting to
>>> preconfigure some devices in device_table.txt - e.g. /dev/null,
>>> /dev/urandom et al pseudo devices, and I'm borrowing Ubuntu's rules.d
>>> to see if these are any more effective than Fedora's. Leaving
>>> device_table.txt as its default causes the tty1 missing loop. I am not
>>> expecting this to work - I cannot see how udev rules differ greatly
>>> between distributions - just eliminating it as a possible cause.
>>>
>>> Any further ideas as to what I can try? Is mdev a better choice?
>>>
>> Well mdev is at least easier to configure than udev.
>>
>> The only thing you must make sure in your init process is that /proc and
>> /sys are mounted as early as possible (/etc/inittab is good place for that)
>> and that echo /sbin/mdev > /proc/sys/kernel/hotplug   is also somewhere
>> in /etc/init.d/rcS (maybe at the start of that file)
>>
>> And check that you have etc/mdev.conf and etc/init.d/S10mdev (witch is
>> called by /etc/init.d/rcS)
>>
>> This is what I would do (and have done with my live CD)
>> Forget the buildroot way of making initramfs for now and see if you can
>> construct live CD by hand:
>>
>> 1 ) make clean
>>
>> 2 ) make sure you have all the needed init stuff in  fs/skeleton/etc
>> directory (inittab, init.d/rcS, init.d/S* files etc.....)
>>      also make sure that sbin/init is a symlink to /bin/busybox
>> (I have also symlink of init --> /bin/busybox in top of fs/skeleton but
>> I not sure if that is stricly needed ...
>> I think kernel should be looking from /sbin/init if no other init found)
>>
>> 3 )  do make menuconfig as you would normally but remember to unselect
>> cpio, initramfs and iso9660 from menuconfig
>>        remember to select grub from under Bootloader section if you
>> don't already have it
>>        don't select too much of stuff even if you have massive amounts 
>> of RAM in your test systems.
>>        reason being that you are making initramfs later here from your
>> output/target directory and if you have massive
>>       amount's of stuff in there then it will take forever to boot. (I
>> have managed to do live CD with massive amounts of stuff on it and
>>      booting fast with only 192 MB RAM but that needed some major hacking)
>>
>> 4 ) do    make linux-menuconfig  and check that under General setup the
>> setting Initramfs source files is empty
>>      also check the compression support here that you want to use for
>> your external initramfs (let's suppose its' the default GZ for now)
>>
>> 5 ) do make
>>
>> 6 ) after buildroot has finished make sure that everything is okay in
>> your output/target (check especially boot/ etc/ and /dev directory)
>>      also make sure you have /boot/grub/menu.lst or /boot/grub/grub.conf
>> (the other is usually just symlink to other) and check it's settings
>>     (If you are not familiar with grub then I can help with it's
>> settings, especially important is the line where you tell grub the external
>>    initramfs file)
>>      remove the symlink (iso9660 format does not support symlinks)
>>
>> 7 ) Create directory where you want to put your kernel & bootloader
>> (a.ka. the whole contents of /boot dir) and external initramfs file
>>      that is generated next:   
>>     
>>       mkdir /livecd && cp -r output/target/boot /livecd
>>
>> 8 ) from output/target dir   do    find .  -not \( -path "./boot" -prune
>> \) -print0 | cpio --null -ov --format=newc | gzip -9 > /livecd/initramfs.gz
>> (we don't need boot directory inside initramfs. it needs to stay outside
>> of it. that why it was copied previously)
>>
>> 9 ) make iso9660 normally from the contents of your /livecd
>> mkdir /livecd_output && cd /livecd && mkisofs --joliet -R -b
>> boot/grub/stage2_eltorito -no-emul-boot -boot-load-size 4
>> -boot-info-table -iso-level 4 -hide-rr-moved -c boot.catalog -o
>> /livecd_output/your_livecd.iso .
>>
>> (remember that . at the end)
>>
>> 10) burn the iso to CD or even better use qemu or VirtualBox to boot
>> your iso without wasting CD disc.
>>
>> If this manual way worked then next try buildroot way.
>>
>> Best regards
>> Stefan
>>
>>> Thanks for your help,
>>>
>>> Antony
>>>
>>> ________________
>>>
>>> Macrium? Software is a trading name of Paramount Software UK Ltd,
>>> Registered in England and Wales Number 02973414
>>> Registered Office: Kilburn House, Manchester Science Park, Lloyd St.
>>> North, Manchester M15 6SE United Kingdom
>>>
>>> The information contained in this e-mail is confidential, privileged,
>>> or otherwise protected from disclosure. It is intended only for the
>>> use of the authorized individual as indicated in the e-mail. Any
>>> unauthorized disclosure, copying, distribution or taking of any action
>>> based on the contents of this material is strictly prohibited.
>>>
>>> If you have received this e-mail in error, please delete it immediately.
>>>
>>>
>>>
>>> _______________________________________________
>>> buildroot mailing list
>>> buildroot at busybox.net
>>> http://lists.busybox.net/mailman/listinfo/buildroot

[Buildroot] Udev on a live CD system does not appear to populate /dev

[Stefan Fröberg]

I forgot to mention one more thing:

If you ever want to use the maximum compression method supported by
modern kernels (XZ compression)
then change that initramfs generation command to this:

find .  -not \( -path "./boot" -prune \) -print0 | cpio --null -ov --format=newc
 | xz --check=crc32 > /livecd/initramfs.xz

Without that --check=crc32 parameter you won't be able to boot
XZ-compressed initramfs.

Regards
Stefan

8.11.2012 18:02, Antony Vennard kirjoitti:
> Hi Stefan,
>
> Thanks for that. I have followed your instructions for making a live cd
> manually with mkisofs - actually this is helpful since a later job is to
> add UEFI booting to the disk, which is easier this way (although I am
> going to look at modifying buildroot, if I can).
>
> I am using grub, and have modified my grub command line to use:
>
> title LiveCD
> kernel /vmlinuz vga=0x317
> initrd /initramfs.bz2
>
> The kernel has bz2, xz and gzip initramfs support built in - so all good
> there. However, the kernel complains there's no root parameter. What
> should I specify for this?
>
> Thanks,
>
> Antony
>
> On 08/11/2012 12:35, Stefan Fr?berg wrote:
>> Hi Antony
>>
>> Just one guestion: What bootloader are you using for your Live CD ?
>> (I use grub)
>>
>> 8.11.2012 11:56, Antony Vennard kirjoitti:
>>> Hi All,
>>>
>>> I'm not using initramfs - I'm using the ISO 9660 target which as far
>>> as I can see produces an initrd, a kernel and an ext2 raw image which
>>> it mounts as a ramdisk. If I select the initramfs options, the cpio
>>> filesystem is generated prior to the iso9660 one, *but the initramfs
>>> afterwards!* Which means, as far as I can see, that initramfs would
>>> not be placed on the disk - unless buildroot manipulates the iso
>>> filesystem post creation? I could of course mkisofs myself, but I was
>>> trying to avoid that - although if this fixes udev, I'll do it.
>>>
>> Keep in mind that the buildroot way of making initramfs (that is if you
>> select intramfs from menuconfig) is
>> to embed your initramfs directly into kernel.
>>
>> There are three disadvantages in here:
>>
>> 1 ) It will make your kernel fatter
>> 2 ) It will makes things more complicated than necessary
>> You might have two initramfs settings: One that is embedded directly
>> into your kernel and
>> another that you have made by hand by yourself. And it's very possible
>> that they could have conflicting stuff between them
>> You can have several initramfs files (at least with grub bootloader),
>> the last ones overlaying the first ones, in order.
>> 3 ) It will makes less flexibe.
>>  
>> All kernels since 2.6 have initramfs inside them, it's just that by
>> default that environment is empty.
>>
>> Embedding initramfs directly to kernel might be good for those users who
>> don't use any bootloader or are using a bootloader which can't
>> be feeded external, separate initramfs file(s) (like grub) and want to
>> have minimal system always available (as piggybacked inside kernel).
>>
>>
>>> I've nothing /dev related in my fs/skeleton, and only the default
>>> values in device_table.txt. Since I'm using udev, I did expect it
>>> would populate everything it needed in /dev - although unless I add
>>> entries for null, console and a tty into device_table.txt, I end up
>>> with a distribution that loops forever saying "could not find
>>> /dev/tty1" (if I've configured tty1 as the location for getty for
>>> example). I can't mitigate this by re-writing the inittab to launch
>>> getty "once" as opposed to respawn, since buildroot rewrites that -
>>> and askfirst:/bin/sh works, except that the respawn error messages are
>>> written all over /dev/console, making it unusable...!
>>>
>>> By the sounds of things this is wrong, but I'm currently attempting to
>>> preconfigure some devices in device_table.txt - e.g. /dev/null,
>>> /dev/urandom et al pseudo devices, and I'm borrowing Ubuntu's rules.d
>>> to see if these are any more effective than Fedora's. Leaving
>>> device_table.txt as its default causes the tty1 missing loop. I am not
>>> expecting this to work - I cannot see how udev rules differ greatly
>>> between distributions - just eliminating it as a possible cause.
>>>
>>> Any further ideas as to what I can try? Is mdev a better choice?
>>>
>> Well mdev is at least easier to configure than udev.
>>
>> The only thing you must make sure in your init process is that /proc and
>> /sys are mounted as early as possible (/etc/inittab is good place for that)
>> and that echo /sbin/mdev > /proc/sys/kernel/hotplug   is also somewhere
>> in /etc/init.d/rcS (maybe at the start of that file)
>>
>> And check that you have etc/mdev.conf and etc/init.d/S10mdev (witch is
>> called by /etc/init.d/rcS)
>>
>> This is what I would do (and have done with my live CD)
>> Forget the buildroot way of making initramfs for now and see if you can
>> construct live CD by hand:
>>
>> 1 ) make clean
>>
>> 2 ) make sure you have all the needed init stuff in  fs/skeleton/etc
>> directory (inittab, init.d/rcS, init.d/S* files etc.....)
>>      also make sure that sbin/init is a symlink to /bin/busybox
>> (I have also symlink of init --> /bin/busybox in top of fs/skeleton but
>> I not sure if that is stricly needed ...
>> I think kernel should be looking from /sbin/init if no other init found)
>>
>> 3 )  do make menuconfig as you would normally but remember to unselect
>> cpio, initramfs and iso9660 from menuconfig
>>        remember to select grub from under Bootloader section if you
>> don't already have it
>>        don't select too much of stuff even if you have massive amounts 
>> of RAM in your test systems.
>>        reason being that you are making initramfs later here from your
>> output/target directory and if you have massive
>>       amount's of stuff in there then it will take forever to boot. (I
>> have managed to do live CD with massive amounts of stuff on it and
>>      booting fast with only 192 MB RAM but that needed some major hacking)
>>
>> 4 ) do    make linux-menuconfig  and check that under General setup the
>> setting Initramfs source files is empty
>>      also check the compression support here that you want to use for
>> your external initramfs (let's suppose its' the default GZ for now)
>>
>> 5 ) do make
>>
>> 6 ) after buildroot has finished make sure that everything is okay in
>> your output/target (check especially boot/ etc/ and /dev directory)
>>      also make sure you have /boot/grub/menu.lst or /boot/grub/grub.conf
>> (the other is usually just symlink to other) and check it's settings
>>     (If you are not familiar with grub then I can help with it's
>> settings, especially important is the line where you tell grub the external
>>    initramfs file)
>>      remove the symlink (iso9660 format does not support symlinks)
>>
>> 7 ) Create directory where you want to put your kernel & bootloader
>> (a.ka. the whole contents of /boot dir) and external initramfs file
>>      that is generated next:   
>>     
>>       mkdir /livecd && cp -r output/target/boot /livecd
>>
>> 8 ) from output/target dir   do    find .  -not \( -path "./boot" -prune
>> \) -print0 | cpio --null -ov --format=newc | gzip -9 > /livecd/initramfs.gz
>> (we don't need boot directory inside initramfs. it needs to stay outside
>> of it. that why it was copied previously)
>>
>> 9 ) make iso9660 normally from the contents of your /livecd
>> mkdir /livecd_output && cd /livecd && mkisofs --joliet -R -b
>> boot/grub/stage2_eltorito -no-emul-boot -boot-load-size 4
>> -boot-info-table -iso-level 4 -hide-rr-moved -c boot.catalog -o
>> /livecd_output/your_livecd.iso .
>>
>> (remember that . at the end)
>>
>> 10) burn the iso to CD or even better use qemu or VirtualBox to boot
>> your iso without wasting CD disc.
>>
>> If this manual way worked then next try buildroot way.
>>
>> Best regards
>> Stefan
>>
>>> Thanks for your help,
>>>
>>> Antony
>>>
>>> ________________
>>>
>>> Macrium? Software is a trading name of Paramount Software UK Ltd,
>>> Registered in England and Wales Number 02973414
>>> Registered Office: Kilburn House, Manchester Science Park, Lloyd St.
>>> North, Manchester M15 6SE United Kingdom
>>>
>>> The information contained in this e-mail is confidential, privileged,
>>> or otherwise protected from disclosure. It is intended only for the
>>> use of the authorized individual as indicated in the e-mail. Any
>>> unauthorized disclosure, copying, distribution or taking of any action
>>> based on the contents of this material is strictly prohibited.
>>>
>>> If you have received this e-mail in error, please delete it immediately.
>>>
>>>
>>>
>>> _______________________________________________
>>> buildroot mailing list
>>> buildroot at busybox.net
>>> http://lists.busybox.net/mailman/listinfo/buildroot

[Buildroot] Udev on a live CD system does not appear to populate /dev

[Arnout Vandecappelle]

On 11/07/12 18:56, Stefan Fr?berg wrote:
> Nope, no sudo in my live Cd.
> Just few pre-made device nodes in fs/skeleton/dev

  So, when buildroot copies these to output/target/dev you get:
cp: cannot create special file `...': Operation not permitted
unless you run the build as root.

  Regards,
  Arnout
-- 
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 +32-16-286540
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] Udev on a live CD system does not appear to populate /dev

[Arnout Vandecappelle]

On 11/08/12 17:02, Antony Vennard wrote:
> I am using grub, and have modified my grub command line to use:
>
> title LiveCD
> kernel /vmlinuz vga=0x317
> initrd /initramfs.bz2
>
> The kernel has bz2, xz and gzip initramfs support built in - so all good
> there. However, the kernel complains there's no root parameter. What
> should I specify for this?

  If you build a cpio rootfs, you shouldn't need any additional command line
parameters because the kernel never gets to mounting the root filesystem.  It
just runs the /init in the initramfs.

  Using a cpio rootfs should also solve your problems with the device nodes,
because we provide a /init that mounts devtmpfs explicitly.

  Regards,
  Arnout

-- 
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 +32-16-286540
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] Udev on a live CD system does not appear to populate /dev

[Antony Vennard]

On 08/11/2012 21:17, Arnout Vandecappelle wrote:
>  If you build a cpio rootfs, you shouldn't need any additional command line
> parameters because the kernel never gets to mounting the root
> filesystem.  It
> just runs the /init in the initramfs.
> 
>  Using a cpio rootfs should also solve your problems with the device nodes,
> because we provide a /init that mounts devtmpfs explicitly.
> 
>  Regards,
>  Arnout
> 

Yep, I'd come to the conclusion I needed an /init to exist, constructing
the root file system. Manually creating the CD as a test, the kernel
panics (root file system not found) if init doesn't exist. I used the
one from fs/cpio/init as you suggested.

The ram_disk_size parameter does appear to be entirely optional.

I'm now on to the next problem - that script exec's /sbin/init which is
really /bin/busybox... which is exiting straight away or crashing, as
the kernel is now panicking about an exiting init :(

I'm going to have an experiment with this, see if I can understand what
the issue is.

I have also grabbed a copy of buildroot git. Once I've got this working,
I'll have a go at modifying buildroot to build iso9660 targets via this
mechanism. I'm no expert in this kind of thing, but it'll be a good
learning experience.

I may also potentially be looking to integrate EFI boot into our live CD
at some stage - if this is of interest I can have a go at adding this to
buildroot also.

Thanks for your help,

Antony


-- 
________________

Macrium? Software is a trading name of Paramount Software UK Ltd, 
Registered in England and Wales Number 02973414
Registered Office: Kilburn House, Manchester Science Park, Lloyd St. North, 
Manchester M15 6SE United Kingdom

The information contained in this e-mail is confidential, privileged, or 
otherwise protected from disclosure. It is intended only for the use of the 
authorized individual as indicated in the e-mail. Any unauthorized 
disclosure, copying, distribution or taking of any action based on the 
contents of this material is strictly prohibited.

If you have received this e-mail in error, please delete it immediately.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 895 bytes
Desc: OpenPGP digital signature
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20121109/742b79b0/attachment.asc>

[Buildroot] Udev on a live CD system does not appear to populate /dev

[Arnout Vandecappelle]

On 11/09/12 13:20, Antony Vennard wrote:
> I may also potentially be looking to integrate EFI boot into our live CD
> at some stage - if this is of interest I can have a go at adding this to
> buildroot also.

  I guess that will mean you'll need grub2.  A posted grub2 patches a while
ago, if you could get them a spin and give feedback, that would be nice.  Or
I can repost them if you like.

  Regards,
  Arnout
-- 
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 +32-16-286540
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] Udev on a live CD system does not appear to populate /dev

[Antony Vennard]

I'm not entirely sure - I believe grub legacy can do efi; Fedora 17
certainly uses uses a variant of grub 0.97 for its EFI bootloader by
default, although they're moving to grub2 apparently.

Don't worry, I'll find the patches in git when ready - I can see them in
the git log.

Thanks,

Antony

On 09/11/2012 13:01, Arnout Vandecappelle wrote:
> On 11/09/12 13:20, Antony Vennard wrote:
>> I may also potentially be looking to integrate EFI boot into our live CD
>> at some stage - if this is of interest I can have a go at adding this to
>> buildroot also.
> 
>  I guess that will mean you'll need grub2.  A posted grub2 patches a while
> ago, if you could get them a spin and give feedback, that would be
> nice.  Or
> I can repost them if you like.
> 
>  Regards,
>  Arnout


-- 
________________

Macrium? Software is a trading name of Paramount Software UK Ltd, 
Registered in England and Wales Number 02973414
Registered Office: Kilburn House, Manchester Science Park, Lloyd St. North, 
Manchester M15 6SE United Kingdom

The information contained in this e-mail is confidential, privileged, or 
otherwise protected from disclosure. It is intended only for the use of the 
authorized individual as indicated in the e-mail. Any unauthorized 
disclosure, copying, distribution or taking of any action based on the 
contents of this material is strictly prohibited.

If you have received this e-mail in error, please delete it immediately.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 895 bytes
Desc: OpenPGP digital signature
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20121109/e9be69a7/attachment.asc>

[Buildroot] Udev on a live CD system does not appear to populate /dev

[Stefan Fröberg]

9.11.2012 14:20, Antony Vennard kirjoitti:
> On 08/11/2012 21:17, Arnout Vandecappelle wrote:
>>  If you build a cpio rootfs, you shouldn't need any additional command line
>> parameters because the kernel never gets to mounting the root
>> filesystem.  It
>> just runs the /init in the initramfs.
>>
>>  Using a cpio rootfs should also solve your problems with the device nodes,
>> because we provide a /init that mounts devtmpfs explicitly.
>>
>>  Regards,
>>  Arnout
>>
> Yep, I'd come to the conclusion I needed an /init to exist, constructing
> the root file system. Manually creating the CD as a test, the kernel
> panics (root file system not found) if init doesn't exist. I used the
> one from fs/cpio/init as you suggested.
>
> The ram_disk_size parameter does appear to be entirely optional.
>
> I'm now on to the next problem - that script exec's /sbin/init which is
> really /bin/busybox... which is exiting straight away or crashing, as
> the kernel is now panicking about an exiting init :(
>
> I'm going to have an experiment with this, see if I can understand what
> the issue is.
Well, at least now it find's your root device (aka dynamic ramdisk aka
/dev/ram0).
That's good.

Yes, it will throw kernel panic because init is never supposed to exit.
So stuff inside your /etc/ or /etc/init.d needs some fine tuning.

Do you now also have symlink of init --> /bin/busybox in your initramfs
root also ?

If you wan't you can feed the grub kernel loading line a  init=/bin/sh  
(or /bin/ash or whatever your shell is)
That should just start your shell (skip everything else) in live cd so
that you can look inside the running system what
works and what really happened
(aka if your /dev has all the nodes, are all the necessary stuff there,
etc...)

Also, if that init=/bin/sh does not work (it should always work) then
it's clear that somethings is wrong with your busybox.

After kernel has loaded it seeks for either /init or /sbin/init then it
looks for /etc/inittab file.
In /etc/inittab it calls /etc/init.d/rcS which is responsible of
starting all your other services (mdev,network,urandom etc..)
inside /etc/init.d directory


I have included here my own live CD init stuff (only the bare
essentials, most of the default buildroot stuff remaining intact).
It's a combination of buildroot, gentoo and some of my own stuff (with
all my CD-limit breaking hacks removed).
It's not perfect but it works.

Just remember that if you are not using /bin/login then just replace that

tty1::respawn:-/bin/login

line with your own shell (/bin/sh, /bin/ash etc...)

Regards
Stefan
> I have also grabbed a copy of buildroot git. Once I've got this working,
> I'll have a go at modifying buildroot to build iso9660 targets via this
> mechanism. I'm no expert in this kind of thing, but it'll be a good
> learning experience.
>
> I may also potentially be looking to integrate EFI boot into our live CD
> at some stage - if this is of interest I can have a go at adding this to
> buildroot also.
>
> Thanks for your help,
>
> Antony
>
>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: copy_of_my_etc_init_stuff.tar.bz2
Type: application/octet-stream
Size: 11689 bytes
Desc: not available
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20121109/c3b3b3a0/attachment-0001.obj>

[Buildroot] Udev on a live CD system does not appear to populate /dev

[Arnout Vandecappelle]

On 11/09/12 14:09, Antony Vennard wrote:
> I'm not entirely sure - I believe grub legacy can do efi; Fedora 17
> certainly uses uses a variant of grub 0.97 for its EFI bootloader by
> default, although they're moving to grub2 apparently.
>
> Don't worry, I'll find the patches in git when ready - I can see them in
> the git log.

  They're not committed yet, so you have to look on the list instead.

  Regards,
  Arnout

-- 
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 +32-16-286540
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] Udev on a live CD system does not appear to populate /dev

[Arnout Vandecappelle]

On 08/11/12 17:29, Stefan Fr?berg wrote:
> If you ever want to use the maximum compression method supported by
> modern kernels (XZ compression)
> then change that initramfs generation command to this:
>
> find .  -not \( -path "./boot" -prune \) -print0 | cpio --null -ov --format=newc
>   | xz --check=crc32>  /livecd/initramfs.xz
>
> Without that --check=crc32 parameter you won't be able to boot
> XZ-compressed initramfs.

  It would be good to have this in buildroot.  Care to add a config option
BR2_TARGET_ROOTFS_CPIO_XZ?  Using host-xz, of course, since not all systems
have xz installed.


  Regards,
  Arnout
-- 
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 +32-16-286540
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] Udev on a live CD system does not appear to populate /dev

[Stefan Fröberg]

13.11.2012 8:53, Arnout Vandecappelle kirjoitti:
> On 08/11/12 17:29, Stefan Fr?berg wrote:
>> If you ever want to use the maximum compression method supported by
>> modern kernels (XZ compression)
>> then change that initramfs generation command to this:
>>
>> find .  -not \( -path "./boot" -prune \) -print0 | cpio --null -ov
>> --format=newc
>>   | xz --check=crc32>  /livecd/initramfs.xz
>>
>> Without that --check=crc32 parameter you won't be able to boot
>> XZ-compressed initramfs.
>
>  It would be good to have this in buildroot.  Care to add a config option
> BR2_TARGET_ROOTFS_CPIO_XZ?  Using host-xz, of course, since not all
> systems
> have xz installed.
>

Sure, no problem.

So far I have only done initramfs my own manual way but I will take this
weekend a look how buildroot does its
own initramfs.

Regards
Stefan

>
>  Regards,
>  Arnout

[Buildroot] Udev on a live CD system does not appear to populate /dev

[Stefan Fröberg]

Arnout, has there been some changes to grub 0.97 (aka legacy version) or
the way grub is compiled
since buildroot 2012.05 ?

Today I copied my old, 2012.05 buildroot .config to latest git-version
and tried to build live cd the buildroot way.

Building went mostly smoothly with just few errors in using custom
busybox config (that i have always been using),
it's login applet and the linux-pam.   And dbus was complaining about
missing /var/lib/ directory in target).

These were not a big deal to fix.

But the real problem now is that grub refuses to load. It just shows
Loading stage2 .... and hangs there.

If I copy my old grub stuff  (namely, the stage2_eltorito file) from
2012.05 build to inside the boot/grub/ of the
new buildroot generated livecd (of course, first mounted throught
loopback and copied to some other dir for modifying)
and then build the livecd from that manually, it will work and show grub
menu normally.

I did a quick *md5sum* and *ls* and there indeed is a difference with
the git-version and 2012.05 version of
stage2_eltorito file,  git-version of stage2_eltorito being very large
(258 KB vs. working 103 KB)

Even tought I have not changed anything in my working 2012.05 buildroot
.config when I copied it
to newer git-version.
(same gcc version, same optimizations, same binutils etc...)

Regards
Stefan

13.11.2012 8:53, Arnout Vandecappelle kirjoitti:
> On 08/11/12 17:29, Stefan Fr?berg wrote:
>> If you ever want to use the maximum compression method supported by
>> modern kernels (XZ compression)
>> then change that initramfs generation command to this:
>>
>> find .  -not \( -path "./boot" -prune \) -print0 | cpio --null -ov
>> --format=newc
>>   | xz --check=crc32>  /livecd/initramfs.xz
>>
>> Without that --check=crc32 parameter you won't be able to boot
>> XZ-compressed initramfs.
>
>  It would be good to have this in buildroot.  Care to add a config option
> BR2_TARGET_ROOTFS_CPIO_XZ?  Using host-xz, of course, since not all
> systems
> have xz installed.
>
>
>  Regards,
>  Arnout

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20121118/443c2576/attachment.html>

[Buildroot] Udev on a live CD system does not appear to populate /dev

[Arnout Vandecappelle]

On 17/11/12 23:54, Stefan Fr?berg wrote:
[snip]
> But the real problem now is that grub refuses to load. It just shows Loading stage2 .... and hangs there.
>
> If I copy my old grub stuff  (namely, the stage2_eltorito file) from 2012.05 build to inside the boot/grub/ of the
> new buildroot generated livecd (of course, first mounted throught loopback and copied to some other dir for modifying)
> and then build the livecd from that manually, it will work and show grub menu normally.
>
> I did a quick *md5sum* and *ls* and there indeed is a difference with the git-version and 2012.05 version of
> stage2_eltorito file,  git-version of stage2_eltorito being very large (258 KB vs. working 103 KB)
>
  Is this on a x86_64 target? Then a difference is that we removed
-fPIC -DPIC from CFLAGS.  Can you try if that makes a difference?

  Regards,
  Arnout
-- 
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 +32-16-286540
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] Udev on a live CD system does not appear to populate /dev

[Stefan Fröberg]

18.11.2012 22:02, Arnout Vandecappelle kirjoitti:
> On 17/11/12 23:54, Stefan Fr?berg wrote:
> [snip]
>> But the real problem now is that grub refuses to load. It just shows
>> Loading stage2 .... and hangs there.
>>
>> If I copy my old grub stuff  (namely, the stage2_eltorito file) from
>> 2012.05 build to inside the boot/grub/ of the
>> new buildroot generated livecd (of course, first mounted throught
>> loopback and copied to some other dir for modifying)
>> and then build the livecd from that manually, it will work and show
>> grub menu normally.
>>
>> I did a quick *md5sum* and *ls* and there indeed is a difference with
>> the git-version and 2012.05 version of
>> stage2_eltorito file,  git-version of stage2_eltorito being very
>> large (258 KB vs. working 103 KB)
>>
>  Is this on a x86_64 target? Then a difference is that we removed
> -fPIC -DPIC from CFLAGS.  Can you try if that makes a difference?
>

No, just building i586 target inside 32-bit Fedora 16 host.

Ok, Im now going to rebuild both 2012.05 and latest git again and
see what happens.

Regards
Stefan

>  Regards,
>  Arnout

[Buildroot] Udev on a live CD system does not appear to populate /dev

[Stefan Fröberg]

18.11.2012 22:11, Stefan Fr?berg kirjoitti:
> 18.11.2012 22:02, Arnout Vandecappelle kirjoitti:
>> On 17/11/12 23:54, Stefan Fr?berg wrote:
>> [snip]
>>> But the real problem now is that grub refuses to load. It just shows
>>> Loading stage2 .... and hangs there.
>>>
>>> If I copy my old grub stuff  (namely, the stage2_eltorito file) from
>>> 2012.05 build to inside the boot/grub/ of the
>>> new buildroot generated livecd (of course, first mounted throught
>>> loopback and copied to some other dir for modifying)
>>> and then build the livecd from that manually, it will work and show
>>> grub menu normally.
>>>
>>> I did a quick *md5sum* and *ls* and there indeed is a difference with
>>> the git-version and 2012.05 version of
>>> stage2_eltorito file,  git-version of stage2_eltorito being very
>>> large (258 KB vs. working 103 KB)
>>>
>>  Is this on a x86_64 target? Then a difference is that we removed
>> -fPIC -DPIC from CFLAGS.  Can you try if that makes a difference?
>>
> No, just building i586 target inside 32-bit Fedora 16 host.
>
> Ok, Im now going to rebuild both 2012.05 and latest git again and
> see what happens.
>
> Regards
> Stefan
>
>>  Regards,
>>  Arnout
>

I have now investigated this matter three days and found out the following:

1 )
That size difference (258 KB vs. working 103 KB) was  because in
buildroot grub I had enabled every network NIC available.

But the following was shocking to me:
The working 103 KB stage2_eltorito was not from buildroot at all in the
first place but from Gentoo!
I vaguely remember copying it to buildroot system from my Gentoo setup
several months ago.
Because I found out that the livecd build process did not work.

But this also means that even 2012.05 buildroot grub did not work for me.

2 )

I built my own version of grub with my own uClibc-based distro and found
out that grub is very nit-pick about CFLAGS and LDFLAGS it is feed.
It doesn't like optimiziation very much.
So after unsetting both flags and applying four patches from Gentoo I
managed to run my very own made grub. Yay!! :-)

3 )

Naturally I then tried this CFLAGS and LDFLAGS unsetting in
boot/grub/grub.mk.
Also used only default buildroot configuration, with newest git.
The damn thing still didn't show grub menu! :-(

And now Im beginning to suspect that buildroot grub is very broken as it
is now and needs some serious patching.
The patches it uses (grub_0.97-35.diff.gz) are from old debian tree from
2008!!

I will now  grab the latest debian patches (grub_0.97-66.diff.gz), use
only -Os for CFLAGS and nothing for LDFLAGS and see how far it will take me
this time.

Regards
Stefan