Architecture of .wic approach; image of images

Architecture of .wic approach; image of images

[Leon Woestenberg]

[-- Attachment #1: Type: text/plain, Size: 658 bytes --]

Hello all,

What is the intended approach of building images using WIC?

Most image types (except .wic) build a (root or initial ram) filesystem
from *packages*, then create the respective archive or filesystem from it.

The .wic image type however, builds an image from *images* rather than
packages.

It therefore seems incorrect we can just add IMAGE_FSTYPES += "wic",
because the corresponding .wks might depend on multiple other images than
just ${PN}.

Should we, or do we have a recipe example that builds only a .wic image,
based on one or more other existing images? (and thus not populate a
filesystem itself)

Regards,

Leon.

[-- Attachment #2: Type: text/html, Size: 864 bytes --]

Re: [yocto] Architecture of .wic approach; image of images

[Maciej Pijanowski]

[-- Attachment #1: Type: text/plain, Size: 1891 bytes --]


On 28.11.2019 09:37, Leon Woestenberg wrote:
> Hello all,
>
> What is the intended approach of building images using WIC?
>
> Most image types (except .wic) build a (root or initial ram) 
> filesystem from *packages*, then create the respective archive or 
> filesystem from it.
I think the rootfs is *always* assembled in the workdir (no matter which 
IMAGE_FSTYPES you choose.
>
> The .wic image type however, builds an image from *images* rather than 
> packages.
I'm not sure if this is correct. WIC is just one of the IMAGE_FSTYPES to 
choose from.
It does a little bit more then the tar.gz FSTYPE. The tar.gz just 
compresses the content
of rootfs. The WIC FSTYPE creates a disk image and installs the rootfs 
on a given
partition (depends on the WKS file chosen). The rootfs source is the same in
those both cases, no matter which FSTYPES you choose. It is made up of 
packages.
>
> It therefore seems incorrect we can just add IMAGE_FSTYPES += "wic", 
> because the corresponding .wks might depend on multiple other images 
> than just ${PN}.
>
> Should we, or do we have a recipe example that builds only a .wic 
> image, based on one or more other existing images? (and thus not 
> populate a filesystem itself)
I have not seen such case yet. IIUC, rootfs in the workdir is always 
populated when the image is built.
>
> Regards,
>
> Leon.
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
>
> View/Reply Online (#47445): https://lists.yoctoproject.org/g/yocto/message/47445
> Mute This Topic: https://lists.yoctoproject.org/mt/63986097/3616795
> Group Owner: yocto+owner@lists.yoctoproject.org
> Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub  [maciej.pijanowski@3mdeb.com]
> -=-=-=-=-=-=-=-=-=-=-=-

-- 
Maciej Pijanowski
Embedded Systems Engineer
https://3mdeb.com | @3mdeb_com


[-- Attachment #2: Type: text/html, Size: 4034 bytes --]

Re: [yocto] Architecture of .wic approach; image of images

[Josef Holzmayr]

On Thu, Nov 28, 2019 at 09:37:12AM +0100, Leon Woestenberg wrote:
> Hello all,
> 
> What is the intended approach of building images using WIC?
> 
> Most image types (except .wic) build a (root or initial ram) filesystem
> from *packages*, then create the respective archive or filesystem from it.
> 
> The .wic image type however, builds an image from *images* rather than
> packages.

Thats *almost* correct. It builds an image from a *artifacts*.

> It therefore seems incorrect we can just add IMAGE_FSTYPES += "wic",
> because the corresponding .wks might depend on multiple other images than
> just ${PN}.

If it does, then your dependency architecture is probably broken. You
have to understand that building an image might produce artifacts that
are not actually included in the rootfs: many systems expect the kernel
to be external to it, and even most do so for u-boot. Hence, the mindset
is as follows: the image recipe biulds everything that you need. It does
not care about the actual for those things shall be poured into.

> Should we, or do we have a recipe example that builds only a .wic image,
> based on one or more other existing images? (and thus not populate a
> filesystem itself)

See above. The image recipe produces all the needed artififacts, that a
*seomthing* later in the pipeline needs. This *something* then takes
those artifacts and pours them into a binary form you can use. Its maybe
bad naming that the configuration of that binary output form is
IMAGE_FSTYPE, which suggests a relation to the image. But de factor,
mentally replace IMAGE_FSTYPE with BINARY_RESULT_FORMAT, and then it
should become clear that this is actually two seperate stages of the
build pipeline.

Greetz

> 
> Regards,
> 
> Leon.

> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> 
> View/Reply Online (#47445): https://lists.yoctoproject.org/g/yocto/message/47445
> Mute This Topic: https://lists.yoctoproject.org/mt/63986097/3618269
> Group Owner: yocto+owner@lists.yoctoproject.org
> Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub  [holzmayr@rsi-elektrotechnik.de]
> -=-=-=-=-=-=-=-=-=-=-=-


-- 
———————————————
Josef Holzmayr
Software Developer Embedded Systems

Tel: +49 8444 9204-48
Fax: +49 8444 9204-50

R-S-I Elektrotechnik GmbH & Co. KG
Woelkestrasse 11
D-85301 Schweitenkirchen
www.rsi-elektrotechnik.de
———————————————
Amtsgericht Ingolstadt – GmbH: HRB 191328 – KG: HRA 170393
Geschäftsführer: Dr.-Ing. Michael Sorg, Dipl.-Ing. Franz Sorg
Ust-IdNr: DE 128592548 

_____________________________________________________________
Amtsgericht Ingolstadt - GmbH: HRB 191328 - KG: HRA 170363
Geschäftsführer: Dr.-Ing. Michael Sorg, Dipl.-Ing. Franz Sorg
USt-IdNr.: DE 128592548

Re: [yocto] Architecture of .wic approach; image of images

[Leon Woestenberg]

Let me then first describe the issue I am having, which I think
results from my misunderstanding then.

I would like to package a UEFI bootloader (GRUB), bzImage and
initramfs.cpio.gz into an FAT partition using WIC.

1) Should I adapt my existing my-initramfs recipe as follows:

IMAGE_FSTYPES += "wic"
WKS_FILE = "my-disk-image.wks"

my-disk-image.wks:
part /boot --source bootimg-efi-initramfs
--sourceparams="loader=grub-efi,initrd=my-initramfs.cpio.gz" --ondisk
sda --label msdos --active --align 1024 --use-uuid
bootloader --ptable gpt --timeout=1

2) Or should I create a new recipe that builds the WIC and depends on
the initramfs?

Remember, I have no rootfs other than the (non-bundled) initramfs.
Depending on the answers, I will come up with the issues I see.

Re: [yocto] Architecture of .wic approach; image of images

[Josef Holzmayr]

On Thu, Nov 28, 2019 at 12:11:52PM +0100, Leon Woestenberg wrote:
> Let me then first describe the issue I am having, which I think
> results from my misunderstanding then.
> 
> I would like to package a UEFI bootloader (GRUB), bzImage and
> initramfs.cpio.gz into an FAT partition using WIC.
> 
> 1) Should I adapt my existing my-initramfs recipe as follows:
> 
> IMAGE_FSTYPES += "wic"
> WKS_FILE = "my-disk-image.wks"

I think I'm starting to see the problem. Do you set those things in your
image recipe? If thats the case, this is where the misunderstandings
happen. An image file is meant to define what goes *into* the image, not
how its gonna look like for a specific hardware incarnation. Think about
core-image-minimal. If it included things like that, how could we use it
as a "hello world"-style image for about every platform?

So, those are things that are hardware, and therefore MACHINE specific.
And thats where they should end up, in your machine config. Thats the
way things are meant:
- the image recipe defines the contents.
- the machine configuration defines the "shape" (binary, partitions,
  layout, whatever) that your specific platform can use.

Greetz

> 
> my-disk-image.wks:
> part /boot --source bootimg-efi-initramfs
> --sourceparams="loader=grub-efi,initrd=my-initramfs.cpio.gz" --ondisk
> sda --label msdos --active --align 1024 --use-uuid
> bootloader --ptable gpt --timeout=1
> 
> 2) Or should I create a new recipe that builds the WIC and depends on
> the initramfs?
> 
> Remember, I have no rootfs other than the (non-bundled) initramfs.
> Depending on the answers, I will come up with the issues I see.

-- 
———————————————
Josef Holzmayr
Software Developer Embedded Systems

Tel: +49 8444 9204-48
Fax: +49 8444 9204-50

R-S-I Elektrotechnik GmbH & Co. KG
Woelkestrasse 11
D-85301 Schweitenkirchen
www.rsi-elektrotechnik.de
———————————————
Amtsgericht Ingolstadt – GmbH: HRB 191328 – KG: HRA 170393
Geschäftsführer: Dr.-Ing. Michael Sorg, Dipl.-Ing. Franz Sorg
Ust-IdNr: DE 128592548 

_____________________________________________________________
Amtsgericht Ingolstadt - GmbH: HRB 191328 - KG: HRA 170363
Geschäftsführer: Dr.-Ing. Michael Sorg, Dipl.-Ing. Franz Sorg
USt-IdNr.: DE 128592548

Re: [yocto] Architecture of .wic approach; image of images

[Leon Woestenberg]

On Thu, Nov 28, 2019 at 12:41 PM Josef Holzmayr
<holzmayr@rsi-elektrotechnik.de> wrote:
> On Thu, Nov 28, 2019 at 12:11:52PM +0100, Leon Woestenberg wrote:
> > Let me then first describe the issue I am having, which I think
> > results from my misunderstanding then.
> >
> > I would like to package a UEFI bootloader (GRUB), bzImage and
> > initramfs.cpio.gz into an FAT partition using WIC.
> >
> > 1) Should I adapt my existing my-initramfs recipe as follows:
> >
> > IMAGE_FSTYPES += "wic"
> > WKS_FILE = "my-disk-image.wks"
>
> I think I'm starting to see the problem. Do you set those things in your
> image recipe?

IMAGE_FSTYPES += "wic" goes in the image recipe, yes.

WKS_FILE is there as well for this image, because I need to create
multiple images with different WKS's.

> If it included things like that, how could we use it
> as a "hello world"-style image for about every platform?
>
I want to create several images. The one I described is a recovery image.
I also want an application image which has a different WKS.

However, this is not the real issue. I first would like to understand
if I need to follow 1) or 2).

Regards,

Leon.

Re: [yocto] Architecture of .wic approach; image of images

[Josef Holzmayr]

On Thu, Nov 28, 2019 at 12:53:54PM +0100, Leon Woestenberg wrote:
> On Thu, Nov 28, 2019 at 12:41 PM Josef Holzmayr
> <holzmayr@rsi-elektrotechnik.de> wrote:
> > On Thu, Nov 28, 2019 at 12:11:52PM +0100, Leon Woestenberg wrote:
> > > Let me then first describe the issue I am having, which I think
> > > results from my misunderstanding then.
> > >
> > > I would like to package a UEFI bootloader (GRUB), bzImage and
> > > initramfs.cpio.gz into an FAT partition using WIC.
> > >
> > > 1) Should I adapt my existing my-initramfs recipe as follows:
> > >
> > > IMAGE_FSTYPES += "wic"
> > > WKS_FILE = "my-disk-image.wks"
> >
> > I think I'm starting to see the problem. Do you set those things in your
> > image recipe?
> 
> IMAGE_FSTYPES += "wic" goes in the image recipe, yes.
> 
> WKS_FILE is there as well for this image, because I need to create
> multiple images with different WKS's.
> 
> > If it included things like that, how could we use it
> > as a "hello world"-style image for about every platform?
> >
> I want to create several images. The one I described is a recovery image.
> I also want an application image which has a different WKS.
> 
> However, this is not the real issue. I first would like to understand
> if I need to follow 1) or 2).

Au contraire :) It is the real issue because neither 1) nor 2) is the
correct way, and I pretty much explained why that is.

What you actually describing are two combinations of image+machine or
image+distro, depending on the exact situation.

Greetz

-- 
———————————————
Josef Holzmayr
Software Developer Embedded Systems

Tel: +49 8444 9204-48
Fax: +49 8444 9204-50

R-S-I Elektrotechnik GmbH & Co. KG
Woelkestrasse 11
D-85301 Schweitenkirchen
www.rsi-elektrotechnik.de
———————————————
Amtsgericht Ingolstadt – GmbH: HRB 191328 – KG: HRA 170393
Geschäftsführer: Dr.-Ing. Michael Sorg, Dipl.-Ing. Franz Sorg
Ust-IdNr: DE 128592548 

_____________________________________________________________
Amtsgericht Ingolstadt - GmbH: HRB 191328 - KG: HRA 170363
Geschäftsführer: Dr.-Ing. Michael Sorg, Dipl.-Ing. Franz Sorg
USt-IdNr.: DE 128592548

Re: [yocto] Architecture of .wic approach; image of images

[Leon Woestenberg]

> Au contraire :) It is the real issue because neither 1) nor 2) is the
> correct way, and I pretty much explained why that is.


So let's then consider the case that the WKS is machine specific. I am
fine with that.

- Does Yocto support building multiple images from one configuration?
- (How) can I build multiple WIC images with different WKSs?

I am not asking anything strange.
I just cannot find an example of a grub + initramfs + kernel packages
into FAT through WIC.
Or documentation on how to implement this correctly.

Regards,

Leon.

Re: [yocto] Architecture of .wic approach; image of images

[Josef Holzmayr]

On Thu, Nov 28, 2019 at 01:32:56PM +0100, Leon Woestenberg wrote:
> > Au contraire :) It is the real issue because neither 1) nor 2) is the
> > correct way, and I pretty much explained why that is.
> 
> 
> So let's then consider the case that the WKS is machine specific. I am
> fine with that.
> 
> - Does Yocto support building multiple images from one configuration?

Of course it does. If you have set DISTRO and MACHINE, you can 
bitbake core-image-minimal
bitbake core-image-x11
bitbake ... whatever image.

Thats exactly the point of the MACHINE / DISTRO / IMAGE seperation.

> - (How) can I build multiple WIC images with different WKSs?

That on the other hand is a good question. The documentation is not
specific on the number of files you can set in WKS_FILE, so i guess at
the moment it is limited to one. So one could investigate if it is worth
augmenting that, or just harness the multiconfig features we have those
days. Their syntax is a little more complicated, but should totally
serve this purpose.

Greetz.

> 
> I am not asking anything strange.
> I just cannot find an example of a grub + initramfs + kernel packages
> into FAT through WIC.
> Or documentation on how to implement this correctly.
> 
> Regards,
> 
> Leon.

-- 
———————————————
Josef Holzmayr
Software Developer Embedded Systems

Tel: +49 8444 9204-48
Fax: +49 8444 9204-50

R-S-I Elektrotechnik GmbH & Co. KG
Woelkestrasse 11
D-85301 Schweitenkirchen
www.rsi-elektrotechnik.de
———————————————
Amtsgericht Ingolstadt – GmbH: HRB 191328 – KG: HRA 170393
Geschäftsführer: Dr.-Ing. Michael Sorg, Dipl.-Ing. Franz Sorg
Ust-IdNr: DE 128592548 

_____________________________________________________________
Amtsgericht Ingolstadt - GmbH: HRB 191328 - KG: HRA 170363
Geschäftsführer: Dr.-Ing. Michael Sorg, Dipl.-Ing. Franz Sorg
USt-IdNr.: DE 128592548

Re: [yocto] Architecture of .wic approach; image of images

[Leon Woestenberg]

On Thu, Nov 28, 2019 at 1:52 PM Josef Holzmayr
<holzmayr@rsi-elektrotechnik.de> wrote:
>
> On Thu, Nov 28, 2019 at 01:32:56PM +0100, Leon Woestenberg wrote:
> > - Does Yocto support building multiple images from one configuration?
>
> Of course it does. If you have set DISTRO and MACHINE, you can
> bitbake core-image-minimal
> bitbake core-image-x11
> bitbake ... whatever image.
>
> Thats exactly the point of the MACHINE / DISTRO / IMAGE seperation.
>
Yes, up to there everything is clean and nice.
And I understand I can build multiple images. I have used this feature
since 2006.

My point is, that we do not support images of images correctly. Or, I
do not understand how it works.

If I have an initramfs image, my next step is to build a disk image
that has the initramfs.cpio.gz, but without generating a new rootfs.

How would I do that?

Re: [yocto] Architecture of .wic approach; image of images

[Leon Woestenberg]

[-- Attachment #1: Type: text/plain, Size: 1233 bytes --]

On Thu, Nov 28, 2019 at 10:01 AM Josef Holzmayr <
holzmayr@rsi-elektrotechnik.de> wrote:

> On Thu, Nov 28, 2019 at 09:37:12AM +0100, Leon Woestenberg wrote:
>
> > The .wic image type however, builds an image from *images* rather than
> > packages.
>
> Thats *almost* correct. It builds an image from a *artifacts*.
>

My finding is: ...only if *artifacts* (also) includes a rootfs.

The image class dependency on rootfs seems very hardcoded. I cannot create
a recipe that works without *creating* a rootfs.
(even though I have no rootfs other than the initramfs image which is
already there, as an *artifact*).

inherit image

IMAGE_FSTYPES = "wic"
DEPENDS += "my-initramfs"

IMAGE_FEATURES = ""
IMAGE_INSTALL = ""

# This image does not generate its own root filesystem
# I have tried to exclude it, but it's hardcoded even in buildhistory.
#do_rootfs[noexec] = "1"
#do_image[noexec] = "1"
#do_image_wic[noexec] = "1"
#do_rootfs_wicenv[noexec] = "1"


WKS_FILE = "my-disk-image.wks"
my-disk-image.wks:
part /boot --source bootimg-efi
--sourceparams="loader=grub-efi,initrd=my-initramfs.cpio.gz" --ondisk
sda --label msdos --active --align 1024 --use-uuid
bootloader --ptable gpt --timeout=1

[-- Attachment #2: Type: text/html, Size: 1884 bytes --]