How can I create a truly minimal distribution that runs entirely from RAM?

How can I create a truly minimal distribution that runs entirely from RAM?

[p32]

[-- Attachment #1: Type: text/plain, Size: 1904 bytes --]

Hello everyone,

I am currently using this ( https://source.codeaurora.org/external/imx/imx-manifest/tree/imx-5.4.3-2.0.0.xml?h=imx-linux-zeus ) Yocto-based build setup provided by NXP to create a custom Linux distribution for one of the i.MX boards. My custom image is based on the core-image-minimal recipe and works fine, i.e., runs on the platform as expected. However, I have to following two issues:

Although core-image-minimal is documented as "A small image just capable of allowing a device to boot", I can tell from the running system that it contains a huge number of components that I think are not be strictly necessary to boot the device. For instance, the boot log contains entries about an FPGA manager framework, Bluetooth, Ethernet, KVM, USB, and a lot of i.MX-specific modules such as for DMA or power management. For evaluation purposes, I want to get rid of all of these and end up with a *truly* minimal Linux system that is able to boot, schedule its tasks, and to communicate via UART. How can I achieve this without losing the i.MX support, i.e., the generation of a bootloader and suitable device tree files?

Furthermore, I would like the minimal system to run entirely from RAM. More specifically: After being started from the SD card, U-Boot should start the Linux distribution via initramfs. I am able to generate some kind of initramfs binary using the following changes:
# local.conf
INITRAMFS_IMAGE = "recipe-name"
INITRAMFS_IMAGE_BUNDLE = "1"
# recipe-name.bb
IMAGE_FSTYPES = "${INITRAMFS_FSTYPES}"
PACKAGE_INSTALL = "${IMAGE_INSTALL}"

However, this does not affect the generated U-Boot, which means that U-Boot still tries to boot from an SD card partition. What is the "right way" to make use of the Image-initramfs-board.bin or the image-board.cpio.gz files that Yocto creates in this case?

Any help yould be greatly appreciated.

Kind regards!

[-- Attachment #2: Type: text/html, Size: 2238 bytes --]

Re: [yocto] How can I create a truly minimal distribution that runs entirely from RAM?

[Zoran]

BBB example:
https://www.yoctoproject.org/pipermail/yocto/2018-July/041696.html

The line:
DISTRO_FEATURES_append = " ram"

Should be:
DISTRO_FEATURES_append = " nfs"

BSP Traces for BBB (YOCTO Warrior):
https://github.com/ZoranStojsavljevic/bbb-yocto/blob/master/bbb-releases/bbb-warrior/target-bbb-platform-traces.txt

And the ash script in the U-Boot supporting above written:
https://github.com/ZoranStojsavljevic/bbb-yocto/blob/master/custom/u-boot.ash#L3

Line 3 to 10.

Please, pay attention to line 1 as well!

Zee
_______

On Thu, Mar 11, 2021 at 12:32 AM p32 via lists.yoctoproject.org
<p32=tuta.io@lists.yoctoproject.org> wrote:
>
> Hello everyone,
>
> I am currently using this Yocto-based build setup provided by NXP to create a custom Linux distribution for one of the i.MX boards. My custom image is based on the core-image-minimal recipe and works fine, i.e., runs on the platform as expected. However, I have to following two issues:
>
> Although core-image-minimal is documented as "A small image just capable of allowing a device to boot", I can tell from the running system that it contains a huge number of components that I think are not be strictly necessary to boot the device. For instance, the boot log contains entries about an FPGA manager framework, Bluetooth, Ethernet, KVM, USB, and a lot of i.MX-specific modules such as for DMA or power management. For evaluation purposes, I want to get rid of all of these and end up with a truly minimal Linux system that is able to boot, schedule its tasks, and to communicate via UART. How can I achieve this without losing the i.MX support, i.e., the generation of a bootloader and suitable device tree files?
>
> Furthermore, I would like the minimal system to run entirely from RAM. More specifically: After being started from the SD card, U-Boot should start the Linux distribution via initramfs. I am able to generate some kind of initramfs binary using the following changes:
> # local.conf
> INITRAMFS_IMAGE = "recipe-name"
> INITRAMFS_IMAGE_BUNDLE = "1"
> # recipe-name.bb
> IMAGE_FSTYPES = "${INITRAMFS_FSTYPES}"
> PACKAGE_INSTALL = "${IMAGE_INSTALL}"
>
> However, this does not affect the generated U-Boot, which means that U-Boot still tries to boot from an SD card partition. What is the "right way" to make use of the Image-initramfs-board.bin or the image-board.cpio.gz files that Yocto creates in this case?
>
> Any help yould be greatly appreciated.
>
> Kind regards!
> 
>

Re: How can I create a truly minimal distribution that runs entirely from RAM?

[p32]

[-- Attachment #1: Type: text/plain, Size: 681 bytes --]

Thank you very much for your help on the second issue! I was unaware of the fact that another mkimage call is necessary. After taking a look at the the references you provided, I was able to boot the system from an initramfs.

However, my current approach requires two manual steps after running Yocto: I need to call mkimage on the cpio.xz file and to extend/configure the U-Boot environment in the running system. Is there a way to automate this?

More specifically, is it possible to...
* have Yocto generate an initramfs.cpio.xz.uboot file instead of just an initramfs.cpio.xz file and to
* modify the default environment that Yocto will compile into the U-Boot binary?

[-- Attachment #2: Type: text/html, Size: 730 bytes --]

Re: [yocto] How can I create a truly minimal distribution that runs entirely from RAM?

[Zoran]

> 1. have Yocto generate an initramfs.cpio.xz.uboot file
> instead of just an initramfs.cpio.xz file and to

I assume this is not too hard to achieve. Somewhere in some bitbake
config file this should be added, but either me do not know that.

So, we'll both wait for this info, maybe some new variable should be
defined for such cases as initramfs, for YOCTO build system to
generate.

For example, adding INITRAMFS_CONF = "1" into local.conf (initially
this variable should be set to INITRAMFS_CONF ??= "0") in some YOCTO
defconfig file?!

> 2. modify the default environment that Yocto will
> compile into the U-Boot binary?

This, I believe, is achievable by the following steps:

      1. Taking/cloning last U-Boot from denx git;
      2. Modifying the ./include/configs/<board.h> file, introducing
the following:

          #ifdef CONFIG_SUPPORT_INITRAMFS_BOOT
          #define INITRAMFS_ENV \
                  <place here modified by you initramfs ash script>
          #else
          #define INITRAMFS_ENV ""
          #endif

        3. Compile U-boot, place it on SDcard and test, to see if you
are able to make it work after rebooting the system;
        4. tar again U-Boot source code with these changes, and upload
it on your server;
        5. Change the U-boot recipe to be downloaded from your server!

Another approach I do not know (maybe YOCTO people do know a better
approach from inside the YOCTO build system).

Hope this helps.

Zoran
_______


On Fri, Mar 12, 2021 at 10:49 PM p32 via lists.yoctoproject.org
<p32=tuta.io@lists.yoctoproject.org> wrote:
>
> Thank you very much for your help on the second issue! I was unaware of the fact that another mkimage call is necessary. After taking a look at the the references you provided, I was able to boot the system from an initramfs.
>
> However, my current approach requires two manual steps after running Yocto: I need to call mkimage on the cpio.xz file and to extend/configure the U-Boot environment in the running system. Is there a way to automate this?
>
> More specifically, is it possible to...
>
> have Yocto generate an initramfs.cpio.xz.uboot file instead of just an initramfs.cpio.xz file and to
> modify the default environment that Yocto will compile into the U-Boot binary?
>
>
> 
>

Re: How can I create a truly minimal distribution that runs entirely from RAM?

[p32]

[-- Attachment #1: Type: text/plain, Size: 1194 bytes --]

Thank you very much. I figured out that you can have Yocto create a suitable U-Boot wrapper as follows (from the image recipe):
IMAGE_FSTYPES = "cpio.xz.u-boot"

Now there is only one last issue that I wasn't able to solve yet: I would like Yocto to not only generate this U-Boot file but also add it to the boot partition of a wic.gz image. I tried to extend the image recipe as follows:
IMAGE_FSTYPES = "cpio.xz.u-boot wic.gz"
IMAGE_BOOT_FILES_append += "${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.cpio.xz.u-boot"

Unfortunately, there is a dependency issue here. BitBake schedules the do_image_wic task before the do_image_cpio task, which creates the u-boot file. I tried to fix it as follows (from the image recipe):
do_image_wic[depends] = "my-image-recipe:do_image_cpio"

While the dependency graph acknowledges this dependency, BitBake does not seem to care about it. Whatever I try, do_image_wic is always executed first and, obviously, does not succeed. I think the problem is comparable to the unsolved one outlined here:
https://stackoverflow.com/questions/58954170/yocto-creating-a-dependency-for-wic-to-cpio-gz-image

How can I instruct Yocto to execute do_image_cpio first?

[-- Attachment #2: Type: text/html, Size: 1283 bytes --]

Re: [yocto] How can I create a truly minimal distribution that runs entirely from RAM?

[Zoran]

> How can I instruct Yocto to execute do_image_cpio first?

YOCTO people are entitled to answer that question. Aren't ya, INTEL folks???

Actually, iNTEL (IOTG) is responsible for that (since YOCTO support is
90% from INTEL), and I assume INTEL YOCTO people are entitled
answering that///

Thank you,
Zee (Zoran)
_______

On Mon, Mar 15, 2021 at 12:17 AM p32 via lists.yoctoproject.org
<p32=tuta.io@lists.yoctoproject.org> wrote:
>
> Thank you very much. I figured out that you can have Yocto create a suitable U-Boot wrapper as follows (from the image recipe):
> IMAGE_FSTYPES = "cpio.xz.u-boot"
>
> Now there is only one last issue that I wasn't able to solve yet: I would like Yocto to not only generate this U-Boot file but also add it to the boot partition of a wic.gz image. I tried to extend the image recipe as follows:
> IMAGE_FSTYPES = "cpio.xz.u-boot wic.gz"
> IMAGE_BOOT_FILES_append += "${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.cpio.xz.u-boot"
>
> Unfortunately, there is a dependency issue here. BitBake schedules the do_image_wic task before the do_image_cpio task, which creates the u-boot file. I tried to fix it as follows (from the image recipe):
> do_image_wic[depends] = "my-image-recipe:do_image_cpio"
>
> While the dependency graph acknowledges this dependency, BitBake does not seem to care about it. Whatever I try, do_image_wic is always executed first and, obviously, does not succeed. I think the problem is comparable to the unsolved one outlined here:
> https://stackoverflow.com/questions/58954170/yocto-creating-a-dependency-for-wic-to-cpio-gz-image
>
> How can I instruct Yocto to execute do_image_cpio first?
>
> 
>

Re: [yocto] How can I create a truly minimal distribution that runs entirely from RAM?

[Nicolas Dechesne]

hey Zoran,


On Mon, Mar 15, 2021 at 5:01 PM Zoran <zoran.stojsavljevic@gmail.com> wrote:
>
> > How can I instruct Yocto to execute do_image_cpio first?
>
> YOCTO people are entitled to answer that question. Aren't ya, INTEL folks???
>
> Actually, iNTEL (IOTG) is responsible for that (since YOCTO support is
> 90% from INTEL), and I assume INTEL YOCTO people are entitled
> answering that///

Please be aware that this is a community mailing list for the Yocto
Project, and we expect everyone to be as pleasant as possible. I don't
think the tone of your last email is appropriate for our mailing
lists. Please refrain yourself from such claims, or we will need to
use moderation. Let's make sure our discussions are about technical
content and in case of any doubt, please refer to our CoC, at
https://www.yoctoproject.org/community/code-of-conduct/.

>
>
> Thank you,
> Zee (Zoran)
> _______
>
> On Mon, Mar 15, 2021 at 12:17 AM p32 via lists.yoctoproject.org
> <p32=tuta.io@lists.yoctoproject.org> wrote:
> >
> > Thank you very much. I figured out that you can have Yocto create a suitable U-Boot wrapper as follows (from the image recipe):
> > IMAGE_FSTYPES = "cpio.xz.u-boot"
> >
> > Now there is only one last issue that I wasn't able to solve yet: I would like Yocto to not only generate this U-Boot file but also add it to the boot partition of a wic.gz image. I tried to extend the image recipe as follows:
> > IMAGE_FSTYPES = "cpio.xz.u-boot wic.gz"
> > IMAGE_BOOT_FILES_append += "${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.cpio.xz.u-boot"
> >
> > Unfortunately, there is a dependency issue here. BitBake schedules the do_image_wic task before the do_image_cpio task, which creates the u-boot file. I tried to fix it as follows (from the image recipe):
> > do_image_wic[depends] = "my-image-recipe:do_image_cpio"
> >
> > While the dependency graph acknowledges this dependency, BitBake does not seem to care about it. Whatever I try, do_image_wic is always executed first and, obviously, does not succeed. I think the problem is comparable to the unsolved one outlined here:
> > https://stackoverflow.com/questions/58954170/yocto-creating-a-dependency-for-wic-to-cpio-gz-image
> >
> > How can I instruct Yocto to execute do_image_cpio first?
> >
> >
> >
>
> 
>

Re: [yocto] How can I create a truly minimal distribution that runs entirely from RAM?

[Zoran]

I can tell you... Ross!

Please, focus on The Problem. Could you, please???

Since, I, an independent entity, am trying to help the people!

OK? Since I do not care about political IOTG development, rather than
on overall (NOT INTEL) resolution of the problem?!

Do you understand the core of the problem (since I am the only one
trying to help here)???

Or do I need to explain it more to (you and INTEL) the ground???

Did you contribute to the solution of the problem??? Or you need salt
and pepper from me?

Do you get it???

Thank you for understanding,
Zoran Stojsavljevic
_______

Re: [yocto] How can I create a truly minimal distribution that runs entirely from RAM?

[Zoran]

> Please be aware that this is a community mailing list for the Yocto
> Project, and we expect everyone to be as pleasant as possible
> I don't think the tone of your last email is appropriate for our
> mailing lists.

Unfortunately, Nicolas, you are barking under the wrong tree.

Please, go to Ross Burton, bark there, and align the (?) views (nothing against
your attack against me).

Thank you,
Zoran Stojsavljevic
_______

On Mon, Mar 15, 2021 at 6:21 PM Nicolas Dechesne
<nicolas.dechesne@linaro.org> wrote:
>
> hey Zoran,
>
>
> On Mon, Mar 15, 2021 at 5:01 PM Zoran <zoran.stojsavljevic@gmail.com> wrote:
> >
> > > How can I instruct Yocto to execute do_image_cpio first?
> >
> > YOCTO people are entitled to answer that question. Aren't ya, INTEL folks???
> >
> > Actually, iNTEL (IOTG) is responsible for that (since YOCTO support is
> > 90% from INTEL), and I assume INTEL YOCTO people are entitled
> > answering that///
>
> Please be aware that this is a community mailing list for the Yocto
> Project, and we expect everyone to be as pleasant as possible. I don't
> think the tone of your last email is appropriate for our mailing
> lists. Please refrain yourself from such claims, or we will need to
> use moderation. Let's make sure our discussions are about technical
> content and in case of any doubt, please refer to our CoC, at
> https://www.yoctoproject.org/community/code-of-conduct/.
>
> >
> >
> > Thank you,
> > Zee (Zoran)
> > _______
> >
> > On Mon, Mar 15, 2021 at 12:17 AM p32 via lists.yoctoproject.org
> > <p32=tuta.io@lists.yoctoproject.org> wrote:
> > >
> > > Thank you very much. I figured out that you can have Yocto create a suitable U-Boot wrapper as follows (from the image recipe):
> > > IMAGE_FSTYPES = "cpio.xz.u-boot"
> > >
> > > Now there is only one last issue that I wasn't able to solve yet: I would like Yocto to not only generate this U-Boot file but also add it to the boot partition of a wic.gz image. I tried to extend the image recipe as follows:
> > > IMAGE_FSTYPES = "cpio.xz.u-boot wic.gz"
> > > IMAGE_BOOT_FILES_append += "${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.cpio.xz.u-boot"
> > >
> > > Unfortunately, there is a dependency issue here. BitBake schedules the do_image_wic task before the do_image_cpio task, which creates the u-boot file. I tried to fix it as follows (from the image recipe):
> > > do_image_wic[depends] = "my-image-recipe:do_image_cpio"
> > >
> > > While the dependency graph acknowledges this dependency, BitBake does not seem to care about it. Whatever I try, do_image_wic is always executed first and, obviously, does not succeed. I think the problem is comparable to the unsolved one outlined here:
> > > https://stackoverflow.com/questions/58954170/yocto-creating-a-dependency-for-wic-to-cpio-gz-image
> > >
> > > How can I instruct Yocto to execute do_image_cpio first?
> > >
> > >
> > >
> >
> > 
> >

Re: [yocto] How can I create a truly minimal distribution that runs entirely from RAM?

[Richard Purdie]

On Mon, 2021-03-15 at 17:01 +0100, Zoran wrote:
> > How can I instruct Yocto to execute do_image_cpio first?
> 
> YOCTO people are entitled to answer that question. Aren't ya, INTEL folks???
> 
> Actually, iNTEL (IOTG) is responsible for that (since YOCTO support is
> 90% from INTEL), and I assume INTEL YOCTO people are entitled
> answering that///

For the record and for the alleviation of any doubt, Intel was one of 
a group of founders of the project and continues to be a valued 
contributor but has never been 90% of the project. Intel has reduced
its involvement more recently both in Yocto Project and in other open source
efforts as it's focus has changed over time. Many of the people who
did work for Intel and contributed to the project now work for different
organisations but still contribute, Ross and myself included.

I'd *strongly* suggest sticking to technical discussion rather than
directing comments at any specific company in future. Your comments are
confusing and off putting for people.

Cheers,

Richard

Re: [yocto] How can I create a truly minimal distribution that runs entirely from RAM?

[Joshua Watt]

[-- Attachment #1: Type: text/plain, Size: 2120 bytes --]


On 3/14/21 6:16 PM, p32 via lists.yoctoproject.org wrote:
> Thank you very much. I figured out that you can have Yocto create a 
> suitable U-Boot wrapper as follows (from the image recipe):
> IMAGE_FSTYPES = "cpio.xz.u-boot"
>
> Now there is only one last issue that I wasn't able to solve yet: I 
> would like Yocto to not only generate this U-Boot file but also add it 
> to the boot partition of a wic.gz image. I tried to extend the image 
> recipe as follows:
> IMAGE_FSTYPES = "cpio.xz.u-boot wic.gz"
> IMAGE_BOOT_FILES_append += 
> "${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.cpio.xz.u-boot"
>
> Unfortunately, there is a dependency issue here. BitBake schedules the 
> do_image_wic task before the do_image_cpio task, which creates the 
> u-boot file. I tried to fix it as follows (from the image recipe):
> do_image_wic[depends] = "my-image-recipe:do_image_cpio"
>
> While the dependency graph acknowledges this dependency, BitBake does 
> not seem to care about it. Whatever I try, do_image_wic is always 
> executed first and, obviously, does not succeed. I think the problem 
> is comparable to the unsolved one outlined here:
> https://stackoverflow.com/questions/58954170/yocto-creating-a-dependency-for-wic-to-cpio-gz-image
>
> How can I instruct Yocto to execute do_image_cpio first?

You might try splitting it up with two images; one that makes your CPIO 
and one that makes the wic image. Since the wic image (presumably?) 
doesn't even have the rootfs specified, it probably doesn't even matter 
what the image is (e.g. you could use core-image-minimal as a test, or 
try to make some even slimmer empty image).

I think by doing that you could do something like:

  IMAGE_BOOT_FILES += "my-cpio-image.cpio.xz.u-boot"

  IMAGE_FILE_DEPENDS += "my-cpio-image"

Then:

  bitbake core-image-minimal

Would sort of do what you are asking

I think they key is that you need a "root file system image" that is 
*not* your CPIO because wic expects to be tied to one that it can write 
to the .wic image, even if thats not actually what you are doing.

>
> 
>

[-- Attachment #2: Type: text/html, Size: 3232 bytes --]