[Xenomai-help] Xenomai Live CD

[Xenomai-help] Xenomai Live CD

[Theo Veenker]

Hi all,

I've managed to create a Xenomai Live CD based on the Ubuntu Dapper
desktop install CD. Many webpages tell you how to do it (for instance
https://help.ubuntu.com/community/LiveCDCustomization/6.06) but the
important thing is to have squashfs and unionfs support in the kernel.
The following is what I did.

I hope someone can pick it up from here as I have very little time to
do it.

TODO:
   - Remove Ubuntu Installer stuff from CD
   - Remove unnecessary packages from the squashfs file system.
   - Change isolinux/isolinux.cfg etc.


Here it goes (I think/hope I haven't forgotten any steps):


Retrieve kernel 2.6.17[.14] sources and unpack into $KERNEL_DIR.
Retrieve xenomai 2.3.0 sources and unpack into $XENOMAI_DIR.
Retrieve squashfs3.2-r2 (http://squashfs.sourceforge.net/) into $SQUASHFS_DIR.
Also compile and installl the mksquashfs tool.
Retrieve unionfs-1.3 (http://www.filesystems.org/project-unionfs.html)
into $UNIONFS_DIR.

Apply xenomai patches:
   cd $XENOMAI_DIR
   ./scripts/prepare-kernel.sh --linux=$KERNEL_DIR --arch=i386
	--adeos=ksrc/arch/i386/patches/adeos-ipipe-2.6.17-i386-1.5-02.patch

Apply squashfs patches:
   cd $KERNEL_DIR
   patch -p1 < $SQUASHFS_DIR/kernel-patches/linux-2.6.17/squashfs3.2-patch

Apply unionfs patches:
   cd $UNIONFS_DIR
   ./patch-kernel.sh $KERNEL_DIR


Configure kernel:
   CONFIG_BLK_DEV_LOOP=m
   CONFIG_BLK_DEV_RAM=y
   CONFIG_BLK_DEV_RAM_COUNT=16
   CONFIG_BLK_DEV_RAM_SIZE=4096
   CONFIG_BLK_DEV_INITRD=y
   CONFIG_SQUASHFS=m
   CONFIG_UNION_FS=m

Compile and install kernel. On my particular system this yields (v20 being
my local version number):
   /boot/vmlinuz-2.6.17.14-xenomai-v20
   /lib/modules/2.6.17.14-xenomai-v20/

Compile and install xenomai. On my system this yields:
   /usr/xenomai-2.3.0-v20/
   /usr/rt (symlink to xenomai-2.3.0-v20)

Make sure this kernel boots OK. Also check realtime behaviour.



Retrieve ubuntu-6.06.1-desktop-i386.iso (http://www.ubuntu.com/) and
let $ISOIMG refer to it.

Set some convenience symbols:
   XENOMAI_VER=xenomai-2.3.0-v20
   export KERNEL_VER=2.6.17.14-xenomai-v20

Loop-mount the ISO image and the squashfs file system contained in it.
   mkdir -p mnt/iso mnt/squashfs
   mount -o loop $ISOIMG mnt/iso
   mount -t squashfs -o loop mnt/iso/casper/filesystem.squashfs mnt/squashfs

Copy contents of the ISO image and the squashfs file system.
   mkdir iso squashfs
   rsync --exclude=/casper/filesystem.squashfs -a mnt/iso/ iso
   cp -a mnt/squashfs/* squashfs

Remove unwanted stuff from ISO image.
   rm -rf iso/programs

Unmount the squashfs file system and the ISO image.
   umount mnt/squashfs mnt/iso
   rm -rf mnt

Modify squashfs contents; install custom kernel and modules.
   cd squashfs
   rm -f boot/vmlinuz* boot/initrd* boot/System* boot/config* boot/abi*
   rm -f initrd.img vmlinuz
   rm -rf lib/modules/*
   rm -rf usr/$XENOMAI_VER
   rm -f usr/rt
   cp /boot/*$KERNEL_VER boot
   cp -a /lib/modules/$KERNEL_VER lib/modules
   cp -a /usr/$XENOMAI_VER usr
   ln -s $XENOMAI_VER usr/rt
   ln -s boot/vmlinuz-$KERNEL_VER vmlinuz
   ln -s boot/initrd.img-$KERNEL_VER initrd.img
   cd ..

Build initrd.gz
   chroot squashfs
   mkinitramfs -o /initrd.gz $KERNEL_VER
   exit

Put custom kernel and initrd in iso/casper/.
   mv squashfs/initrd.gz squashfs/boot/initrd.img-$KERNEL_VER
   cp squashfs/boot/vmlinuz-$KERNEL_VER iso/casper/vmlinuz
   cp squashfs/boot/initrd.img-$KERNEL_VER iso/casper/initrd.gz

Create new iso/casper/filesystem.manifest* files.
   rm iso/casper/filesystem.*
   chroot squashfs dpkg-query -W --showformat='${Package} ${Version}\n' > 
iso/casper/filesystem.manifest
   cp iso/casper/filesystem.manifest iso/casper/filesystem.manifest-desktop
   sed -i -e '/ubiquity/d' iso/casper/filesystem.manifest-desktop

Create squashfs file system file in iso/casper/.
   mksquashfs squashfs iso/casper/filesystem.squashfs
   chmod 444 iso/casper/filesystem.squashfs

Recreate md5sum.txt.
   cd iso
   rm -f md5sum.txt
   find . -type f -exec md5sum {} \; > ../md5sum.txt
   cd ..
   mv md5sum.txt iso

Create ISO image.
   cd iso
   mkisofs -V "Xenomai Ubuntu 6.06 CD" \
         -o xenomai-ubuntu-6.06.1-desktop-i386.iso \
         -b isolinux/isolinux.bin -c isolinux/boot.cat \
         -no-emul-boot -boot-load-size 4 -boot-info-table \
         -l -R -r .
   cd ..

Burn the ISO image onto a CD and boot from it (choose first option from menu).
Once everything is up, open a terminal and in /usr/rt/bin run sudo ./latency

Good luck!

Theo

Re: [Xenomai-help] Xenomai Live CD

[Paul]

Hi Theo

On Saturday 07 April 2007 22:18, Theo Veenker wrote:
> I've managed to create a Xenomai Live CD based on the Ubuntu Dapper
> desktop install CD. Many webpages tell you how to do it (for instance
> https://help.ubuntu.com/community/LiveCDCustomization/6.06) but the
> important thing is to have squashfs and unionfs support in the kernel.
> The following is what I did.

> TODO:
>    - Remove Ubuntu Installer stuff from CD
>    - Remove unnecessary packages from the squashfs file system.
>    - Change isolinux/isolinux.cfg etc.

Having built a number of custom LiveCDs, it is much easier to start from 
scratch rather than trying to remove stuff from an existing build. Stripping 
down a build often leaves cruft behind...

> Here it goes (I think/hope I haven't forgotten any steps):

> Apply xenomai patches:
> Apply squashfs patches:
> Apply unionfs patches:

> Compile and install kernel. On my particular system this yields (v20 being
> my local version number):
>    /boot/vmlinuz-2.6.17.14-xenomai-v20
>    /lib/modules/2.6.17.14-xenomai-v20/

If my experiences with Debian/etch is anything to go by, the grub-update 
scripts will see "xeno" and tag the vmlinuz as a Xen virtual machine image. 
Probably not an issue for a LiveCD, but would be if it were to be used as an 
installation target..

> Compile and install xenomai. On my system this yields:
>    /usr/xenomai-2.3.0-v20/

Hrmm... This breaks the package management philosophy behind most 
distributions - However, there is a Debian rule set that negates that 
problem..

<snipped CLI>
> Retrieve ubuntu-6.06.1-desktop-i386.iso (http://www.ubuntu.com/)
> Loop-mount the ISO image and the squashfs file system contained in it.
> Copy contents of the ISO image and the squashfs file system.
> Remove unwanted stuff from ISO image.
> Unmount the squashfs file system and the ISO image.
> Modify squashfs contents; install custom kernel and modules.
> Build initrd.gz
> Put custom kernel and initrd in iso/casper/.
> Create new iso/casper/filesystem.manifest* files.
> Create squashfs file system file in iso/casper/.
> Recreate md5sum.txt.
> Create ISO image.
>
> Burn the ISO image onto a CD and boot from it (choose first option from
> menu). Once everything is up, open a terminal and in /usr/rt/bin run sudo
> ./latency

http://www.morphix.org has a set of tools that greatly simplifies the task of 
building LiveCDs - mmaker requires a simple XML file listing the main 
components to include on the ISO, and isomaker does the grunt behind the 
bootable parts. Having deb packages for both the kernel and Xenomai is an 
advantage if/when anyone wants to do an install or upgrade.

When unionfs is shown to be stable on a 2.6.20 kernel, it might be worth while 
setting up an autobuild system in conjunction with the Morphix project..
[Side note: I already do nightly Sarge based Morphix modules, so slipping in a 
Xenomai version once in a while is possible].


Regards, Paul.

Re: [Xenomai-help] Xenomai Live CD

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 672 bytes --]

Paul wrote:
> ...
> http://www.morphix.org has a set of tools that greatly simplifies the task of 
> building LiveCDs - mmaker requires a simple XML file listing the main 
> components to include on the ISO, and isomaker does the grunt behind the 
> bootable parts. Having deb packages for both the kernel and Xenomai is an 
> advantage if/when anyone wants to do an install or upgrade.

Morphix looks fairly promising on first sight. Maybe you guys would like
 to stick heads together, creating a Morphix derivative for Xenomai
testing, benchmarking, etc.? For sure, you will get support regarding
useful tests scenarios or reasonable .configs here.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 249 bytes --]

Re: [Xenomai-help] Xenomai Live CD

[Theo Veenker]

Paul wrote:
> Hi Theo
> 
> On Saturday 07 April 2007 22:18, Theo Veenker wrote:
>> I've managed to create a Xenomai Live CD based on the Ubuntu Dapper
>> desktop install CD. Many webpages tell you how to do it (for instance
>> https://help.ubuntu.com/community/LiveCDCustomization/6.06) but the
>> important thing is to have squashfs and unionfs support in the kernel.
>> The following is what I did.
> 
>> TODO:
>>    - Remove Ubuntu Installer stuff from CD
>>    - Remove unnecessary packages from the squashfs file system.
>>    - Change isolinux/isolinux.cfg etc.
> 
> Having built a number of custom LiveCDs, it is much easier to start from 
> scratch rather than trying to remove stuff from an existing build. Stripping 
> down a build often leaves cruft behind...

So you are saying I went through all this ff-ing trouble for nothing?
You have all the knowledge and experience but did't bother putting it
to use for a Xenomai live CD? If it is not too much work maybe you could
put something together. I'm sure many people, including me, would be
interested.

> 
>> Here it goes (I think/hope I haven't forgotten any steps):
> 
>> Apply xenomai patches:
>> Apply squashfs patches:
>> Apply unionfs patches:
> 
>> Compile and install kernel. On my particular system this yields (v20 being
>> my local version number):
>>    /boot/vmlinuz-2.6.17.14-xenomai-v20
>>    /lib/modules/2.6.17.14-xenomai-v20/
> 
> If my experiences with Debian/etch is anything to go by, the grub-update 
> scripts will see "xeno" and tag the vmlinuz as a Xen virtual machine image. 
> Probably not an issue for a LiveCD, but would be if it were to be used as an 
> installation target..

Not on my systems. Anyway I didn't say how to do it, I said what I did.


> 
> http://www.morphix.org has a set of tools that greatly simplifies the task of 
> building LiveCDs - mmaker requires a simple XML file listing the main 
> components to include on the ISO, and isomaker does the grunt behind the 
> bootable parts. Having deb packages for both the kernel and Xenomai is an 
> advantage if/when anyone wants to do an install or upgrade.
> 
> When unionfs is shown to be stable on a 2.6.20 kernel, it might be worth while 
> setting up an autobuild system in conjunction with the Morphix project..
> [Side note: I already do nightly Sarge based Morphix modules, so slipping in a 
> Xenomai version once in a while is possible].

Would be nice. I'm very interested in having a (small) live cd which I can
customize and give to my clients.

Regards,
Theo

Re: [Xenomai-help] Xenomai Live CD

[Paul]

Hi Theo

On Sunday 08 April 2007 09:07, Theo Veenker wrote:
> So you are saying I went through all this ff-ing trouble for nothing?

Not at all... And my apologies if my comments came across as being negative.

> You have all the knowledge and experience but did't bother putting it
> to use for a Xenomai live CD? If it is not too much work maybe you could
> put something together. I'm sure many people, including me, would be
> interested.

The bulk of the work (for Morphix builds) is in the patching & packaging of 
the kernel - Once that is done, it boils down to deciding what applications 
to put on a CD.. As to the question of "why I didn't do it" - To be honest, 
didn't think about it or realise there was a need for one.

> > [Side note: I already do nightly Sarge based Morphix modules,
> > so slipping in a Xenomai version once in a while is possible].
>
> Would be nice. I'm very interested in having a (small) live cd which I can
> customize and give to my clients.

I've posted an RFC to the Morphix list suggesting the next kernel uses the 
Xenomai patches, it won't be a major problem if the answer is no - I can 
always add the unionfs & squashfs patches to the kernel debs currently found 
on http://zathras.tuxcnc.org/xenomai
 You have shown that there is interest in a LiveCD, so perhaps we could work 
together on one - I have the build system set up and running, you know what 
packages you need on a CD..


Regards, Paul.

Re: [Xenomai-help] Xenomai Live CD

[Theo Veenker]

Paul wrote:
> Hi Theo
> 
> On Sunday 08 April 2007 09:07, Theo Veenker wrote:
> 
>>So you are saying I went through all this ff-ing trouble for nothing?
> 
> 
> Not at all... And my apologies if my comments came across as being negative.
> 
> 
>>You have all the knowledge and experience but did't bother putting it
>>to use for a Xenomai live CD? If it is not too much work maybe you could
>>put something together. I'm sure many people, including me, would be
>>interested.
> 
> 
> The bulk of the work (for Morphix builds) is in the patching & packaging of 
> the kernel - Once that is done, it boils down to deciding what applications 
> to put on a CD.. As to the question of "why I didn't do it" - To be honest, 
> didn't think about it or realise there was a need for one.
> 
> 
>>>[Side note: I already do nightly Sarge based Morphix modules,
>>>so slipping in a Xenomai version once in a while is possible].
>>
>>Would be nice. I'm very interested in having a (small) live cd which I can
>>customize and give to my clients.
> 
> 
> I've posted an RFC to the Morphix list suggesting the next kernel uses the 
> Xenomai patches, it won't be a major problem if the answer is no - I can 
> always add the unionfs & squashfs patches to the kernel debs currently found 
> on http://zathras.tuxcnc.org/xenomai
>  You have shown that there is interest in a LiveCD, so perhaps we could work 
> together on one - I have the build system set up and running, you know what 
> packages you need on a CD..

I don't know what you have in mind but I think there a two types of use.

1) A CD that can be used to quicly check the real-time capabilities of a PC.
    Preferably with multiple images to select in boot menu: with/without SMI
    work-around, UP/SMP, etc. if possible.

2) A CD with a live system (with gnome or kde) on it. Of course it should
    be possible to recreate the CD and add desired (real-time) apps to it.
    In my case I like to be able to give our non-technical students/researchers
    a CD which allows them to play with the same (real-time) linux apps as
    they see in the lab.

Regards,
Theo

Re: [Xenomai-help] Xenomai Live CD

[Paul]

Hi Theo

On Sunday 08 April 2007 09:07, Theo Veenker wrote:
> > [Side note: I already do nightly Sarge based Morphix modules, so slipping
> > in a Xenomai version once in a while is possible].

> Would be nice. I'm very interested in having a (small) live cd which I can
> customize and give to my clients.

Finally got round to setting up an Etch based build system for the Morphix 
project[1] so am now in a position to look again at a minimal live CD. An 
assortment of kernel packages were compiled overnight for various CPU 
flavours (586, 686, k7, & k8) and the initial base ISO boots without any 
problem under qemu.

 Aside from the core Xenomai tools and libraries, what are the main packages 
you require - We have a blank canvas here, so just about anything can be 
added.


Regards, Paul.


[1] http://www.morphix.org

Re: [Xenomai-help] Xenomai Live CD

[Laurent COIGNOT]

[-- Attachment #1: Type: text/plain, Size: 213 bytes --]

>> Would be nice. I'm very interested in having a (small) live cd which I
can customize and give to my clients.

I would be interested in this issue in order to practice easily with
students ...

Regards

Laurent

[-- Attachment #2: Type: text/html, Size: 254 bytes --]

Re: [Xenomai-help] Xenomai Live CD

[Roland Tollenaar]

Hi,

I missed most of this thread, but I have a live Xenomai version I am
very busy using. In fact I have it booting and running from USB as well.

Relevant?

RR

Laurent COIGNOT wrote:
>  >> Would be nice. I'm very interested in having a (small) live cd which 
> I can customize and give to my clients.
> 
> I would be interested in this issue in order to practice easily with 
> students ...
> 
> Regards
> 
> Laurent
> 
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Xenomai-help mailing list
> Xenomai-help@domain.hid
> https://mail.gna.org/listinfo/xenomai-help

Re: [Xenomai-help] Xenomai Live CD

[Theo Veenker]

Paul wrote:
> Hi Theo
> 
> On Sunday 08 April 2007 09:07, Theo Veenker wrote:
>>> [Side note: I already do nightly Sarge based Morphix modules, so slipping
>>> in a Xenomai version once in a while is possible].
> 
>> Would be nice. I'm very interested in having a (small) live cd which I can
>> customize and give to my clients.
> 
> Finally got round to setting up an Etch based build system for the Morphix 
> project[1] so am now in a position to look again at a minimal live CD. An 
> assortment of kernel packages were compiled overnight for various CPU 
> flavours (586, 686, k7, & k8) and the initial base ISO boots without any 
> problem under qemu.
> 
>  Aside from the core Xenomai tools and libraries, what are the main packages 
> you require - We have a blank canvas here, so just about anything can be 
> added.

In my case I would need X-Windows (gnome wm) + ALSA, my user application +
kernel module (which of course needs to be recompiled for the particular
kernel). My preference would be to have something I can change and build
myself. Even if it is just an ISO image that I have unpack, add my own
stuff to it and turn it into an ISO again. As long as it's clear where
to put what I'm happy with anything that works.

Regards,
Theo

Re: [Xenomai-help] Xenomai Live CD

[Paul]

On Monday 21 May 2007 10:46, Theo Veenker wrote:
> Paul wrote:
> > Hi Theo
> >
> > On Sunday 08 April 2007 09:07, Theo Veenker wrote:
> >>> [Side note: I already do nightly Sarge based Morphix modules, so
> >>> slipping in a Xenomai version once in a while is possible].
> >>
> >> Would be nice. I'm very interested in having a (small) live cd which I
> >> can customize and give to my clients.
> >
> > Finally got round to setting up an Etch based build system for the
> > Morphix project[1] so am now in a position to look again at a minimal
> > live CD. An assortment of kernel packages were compiled overnight for
> > various CPU flavours (586, 686, k7, & k8) and the initial base ISO boots
> > without any problem under qemu.
> >
> >  Aside from the core Xenomai tools and libraries, what are the main
> > packages you require - We have a blank canvas here, so just about
> > anything can be added.
>
> In my case I would need X-Windows (gnome wm) + ALSA, my user application +
> kernel module (which of course needs to be recompiled for the particular
> kernel). My preference would be to have something I can change and build
> myself. Even if it is just an ISO image that I have unpack, add my own
> stuff to it and turn it into an ISO again. As long as it's clear where
> to put what I'm happy with anything that works.

This is where Morphix is well suited - A main module with the window manager, 
sound support, and general purpose utilities. A secondary mini module (or 
just a tarball) would hold just the custom application & kernel module.. Once 
the main components have been built, should you need to change your custom 
application, only the mini-module/tarball needs to be remastered before 
running mkisofs.

A bare Gnome module (complete with Xorg) weighs in at around 272M, and the 
base module containing a Xenomai patched kernel & libraries adds another 47M. 
After I've had a chance to run further tests, I'll put them on a server along 
with the XML templates and kernel packages.


Regards, Paul.

Re: [Xenomai-help] Xenomai Live CD

[Paul]

[a few days later]... One minimal Live CD based on Morphix is now available. 
It uses a 2.6.20.11 kernel patched with Xenomai trunk (r2456) and compiled 
for a generic 586..

 At a little under 46Meg, it includes a minimal Linux install along with the 
Xenomai test utilities - Can be used as a basis for a full blown live CD.

Available from http://zathras.tuxcnc.org/xenomai-live.iso


Regards, Paul.

Re: [Xenomai-help] Xenomai Live CD

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 688 bytes --]

Paul wrote:
> [a few days later]... One minimal Live CD based on Morphix is now available. 

Cool!

> It uses a 2.6.20.11 kernel patched with Xenomai trunk (r2456) and compiled 
> for a generic 586..

Nice, the bleeding edge. :)

> 
>  At a little under 46Meg, it includes a minimal Linux install along with the 
> Xenomai test utilities - Can be used as a basis for a full blown live CD.

Your internet link is sloooow (60KByte/s at best), guess we should
mirror the final version on a faster server then.

> 
> Available from http://zathras.tuxcnc.org/xenomai-live.iso

Currently downloading, will then give it a try under some VM, expect
feedback later.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 250 bytes --]

Re: [Xenomai-help] Xenomai Live CD

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 1768 bytes --]

Jan Kiszka wrote:
> Paul wrote:
>> [a few days later]... One minimal Live CD based on Morphix is now available. 
> 
> Cool!
> 
>> It uses a 2.6.20.11 kernel patched with Xenomai trunk (r2456) and compiled 
>> for a generic 586..
> 
> Nice, the bleeding edge. :)
> 
>>  At a little under 46Meg, it includes a minimal Linux install along with the 
>> Xenomai test utilities - Can be used as a basis for a full blown live CD.
> 
> Your internet link is sloooow (60KByte/s at best), guess we should
> mirror the final version on a faster server then.
> 
>> Available from http://zathras.tuxcnc.org/xenomai-live.iso
> 
> Currently downloading, will then give it a try under some VM, expect
> feedback later.

 o less: command not found -- mhm, I personally like this tool a lot.

 o killall: command not found -- mhmmmm.

 o Kernel is configured with SMP and comes with the I-pipe tracer. Hope,
   no one does benchmarks :). Having separate kernels would be nice on
   the long term (e.g. a debugging kernel with all stuff on).

 o The SMI work-around is off -- same as above, might be a benchmark
   killer. That makes me wonder if we shouldn't export this knob as a
   module parameter for xeno_nucleus...

 o Benchmarks: please include the cache calibrator as non-RT load.

 o No dhcpcd, thus only manual network setup. I would fire up dhcp by
   default.

 o No ssh(d), thus no remote login.

[o What about stuffing RTnet on the CD as well? Default off, the user
   may tweak the setup manually then once the system is running. We had
   such a request recently on the RTnet list.]

 o We need the Xenomai logo somewhere. :)

I would surely find more, but I better stop playing for now.

Thanks for your effort!
Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 250 bytes --]