best practices regarding using WKS_FILES to reference multiple .wks files?

best practices regarding using WKS_FILES to reference multiple .wks files?

[Robert P. J. Day]

  quick observation -- variable glossary makes no reference to
WKS_FILES variable (or a few other wic-related variables as i think i
mentioned once, but that's not why we're here).

  simple practice for wic is to define a single .wks file which
defines all partitioning for the target. however, what i'm running
into in my current project is that that single .wks file defines a
couple dozen partitions, many of which are maintained by a number of
different developers.

  unsurprisingly, since all this content is in a single file, and
those developers are regularly tweaking their partition attributes,
it's a regular occurrence to have commits step on each other when
merging to master.

  is it reasonable (or even doable) to define a collection of
reasonably independent .wks files that, collectively, define the final
partitioning, and use WKS_FILES to pull them all in, thus avoiding all
those merge conflicts?

  is this approach documented anywhere? is it considered good style?

rday

Re: [docs] best practices regarding using WKS_FILES to reference multiple .wks files?

[Quentin Schulz]

Hi Robert,

On Sun, Oct 03, 2021 at 10:53:54AM -0400, Robert P. J. Day wrote:
> 
>   quick observation -- variable glossary makes no reference to
> WKS_FILES variable (or a few other wic-related variables as i think i
> mentioned once, but that's not why we're here).
> 
>   simple practice for wic is to define a single .wks file which
> defines all partitioning for the target. however, what i'm running
> into in my current project is that that single .wks file defines a
> couple dozen partitions, many of which are maintained by a number of
> different developers.
> 
>   unsurprisingly, since all this content is in a single file, and
> those developers are regularly tweaking their partition attributes,
> it's a regular occurrence to have commits step on each other when
> merging to master.
> 
>   is it reasonable (or even doable) to define a collection of
> reasonably independent .wks files that, collectively, define the final
> partitioning, and use WKS_FILES to pull them all in, thus avoiding all
> those merge conflicts?
> 

No. WKS_FILES is not doing what you think it is.

WKS_FILES is a list of candidates for the .wks file that will be used by
wic. First to be found (left to right) will be used.
c.f. https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/classes/image_types_wic.bbclass#n57,
https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/classes/image_types_wic.bbclass#n39

What I found reading the code is that an include statement should work
in the wks file, c.f. https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/scripts/lib/wic/ksparser.py#n282

I don't know if they are supposed to be in the same directory, if the
path after the include directive is absolute, relative and if relative,
to which directory.

If that works, I think this is something that should be documented :)

I think there was a request on IRC some weeks/months ago to support
creating multiple kind of images with different wks files. That is also
something that makes sense to me :)

Cheers,
Quentin

Re: [docs] best practices regarding using WKS_FILES to reference multiple .wks files?

[Robert P. J. Day]

On Mon, 4 Oct 2021, Quentin Schulz wrote:

> Hi Robert,
>
> On Sun, Oct 03, 2021 at 10:53:54AM -0400, Robert P. J. Day wrote:
> >
> >   quick observation -- variable glossary makes no reference to
> > WKS_FILES variable (or a few other wic-related variables as i think i
> > mentioned once, but that's not why we're here).
> >
> >   simple practice for wic is to define a single .wks file which
> > defines all partitioning for the target. however, what i'm running
> > into in my current project is that that single .wks file defines a
> > couple dozen partitions, many of which are maintained by a number of
> > different developers.
> >
> >   unsurprisingly, since all this content is in a single file, and
> > those developers are regularly tweaking their partition attributes,
> > it's a regular occurrence to have commits step on each other when
> > merging to master.
> >
> >   is it reasonable (or even doable) to define a collection of
> > reasonably independent .wks files that, collectively, define the final
> > partitioning, and use WKS_FILES to pull them all in, thus avoiding all
> > those merge conflicts?
> >
>
> No. WKS_FILES is not doing what you think it is.

  story of my life. :-)

> WKS_FILES is a list of candidates for the .wks file that will be used by
> wic. First to be found (left to right) will be used.
> c.f. https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/classes/image_types_wic.bbclass#n57,
> https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/classes/image_types_wic.bbclass#n39
>
> What I found reading the code is that an include statement should work
> in the wks file, c.f. https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/scripts/lib/wic/ksparser.py#n282
>
> I don't know if they are supposed to be in the same directory, if
> the path after the include directive is absolute, relative and if
> relative, to which directory.
>
> If that works, I think this is something that should be documented
> :)

  yup, that could very well be what i'm after, having the same effect.

rday

Re: [docs] best practices regarding using WKS_FILES to reference multiple .wks files?

[Michael Opdenacker]

Quentin, Robert,

Thank you for the research and for the details about WKS_FILES

On 10/4/21 11:34 AM, Robert P. J. Day wrote:
> On Mon, 4 Oct 2021, Quentin Schulz wrote:
>
>> Hi Robert,
>>
>> On Sun, Oct 03, 2021 at 10:53:54AM -0400, Robert P. J. Day wrote:
>>>   quick observation -- variable glossary makes no reference to
>>> WKS_FILES variable (or a few other wic-related variables as i think i
>>> mentioned once, but that's not why we're here).
>>>
>>>   simple practice for wic is to define a single .wks file which
>>> defines all partitioning for the target. however, what i'm running
>>> into in my current project is that that single .wks file defines a
>>> couple dozen partitions, many of which are maintained by a number of
>>> different developers.
>>>
>>>   unsurprisingly, since all this content is in a single file, and
>>> those developers are regularly tweaking their partition attributes,
>>> it's a regular occurrence to have commits step on each other when
>>> merging to master.
>>>
>>>   is it reasonable (or even doable) to define a collection of
>>> reasonably independent .wks files that, collectively, define the final
>>> partitioning, and use WKS_FILES to pull them all in, thus avoiding all
>>> those merge conflicts?
>>>
>> No. WKS_FILES is not doing what you think it is.
>   story of my life. :-)
>
>> WKS_FILES is a list of candidates for the .wks file that will be used by
>> wic. First to be found (left to right) will be used.
>> c.f. https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/classes/image_types_wic.bbclass#n57,
>> https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/classes/image_types_wic.bbclass#n39
>>
>> What I found reading the code is that an include statement should work
>> in the wks file, c.f. https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/scripts/lib/wic/ksparser.py#n282
>>
>> I don't know if they are supposed to be in the same directory, if
>> the path after the include directive is absolute, relative and if
>> relative, to which directory.
>>
>> If that works, I think this is something that should be documented
>> :)
>   yup, that could very well be what i'm after, having the same effect.


Did you manage to try this mechanism? If you have details about how it
works, I'll be happy to add documentation for setting.
I created a bug in the meantime, so I cannot forget:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=14590

Thanks in advance
Michael.

-- 
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com