wic appending ".1" to a file name on multiple images build

wic appending ".1" to a file name on multiple images build

[Alan Martinovic]

Hi,
I'm creating two images with the single bitbake command and
on inherits the other (production, and a dev which "require production").

Building them at the same time causes an issue:

Traceback (most recent call lastd):
  File "/home/alan/workspace/senic-os-dev/oe/scripts/lib/wic/filemap.py",
line 120, in _open_image_file
     self._f_image = open(self._image_path, 'rb')
  FileNotFoundError: [Errno 2] No such file or directory:
'/home/alan/workspace/senic-os-dev/build/tmp-glibc/deploy/images/senic-hub/u-boot-sunxi-with-spl.bin.1'


The cause of the issue seems to be that `wic create`,
even though it gets the correct file path through the
wks file ("u-boot-sunxi-with-spl.bin")
somewhere in the execution of wic create that
fiile gets renamed (to "u-boot-sunxi-with-spl.bin.1")
and results in an above issue.

This only happens when the images are built in parallel,
running each separately gives no errors.

Am open for suggestions on how to approach this. :)


Be Well,
Alan

Re: wic appending ".1" to a file name on multiple images build

[Alexander Kanavin]

2018-06-01 13:38 GMT+03:00 Alan Martinovic <alan.martinovic@senic.com>:
> Building them at the same time causes an issue:
>
> Traceback (most recent call lastd):
>   File "/home/alan/workspace/senic-os-dev/oe/scripts/lib/wic/filemap.py",
> line 120, in _open_image_file
>      self._f_image = open(self._image_path, 'rb')
>   FileNotFoundError: [Errno 2] No such file or directory:
> '/home/alan/workspace/senic-os-dev/build/tmp-glibc/deploy/images/senic-hub/u-boot-sunxi-with-spl.bin.1'
>
>
> The cause of the issue seems to be that `wic create`,
> even though it gets the correct file path through the
> wks file ("u-boot-sunxi-with-spl.bin")
> somewhere in the execution of wic create that
> fiile gets renamed (to "u-boot-sunxi-with-spl.bin.1")
> and results in an above issue.
>
> This only happens when the images are built in parallel,
> running each separately gives no errors.
>
> Am open for suggestions on how to approach this. :)

By applying code archaeology skills obviously, which are a must-have
for anyone involved in Yocto :)

Specifically here: where and how is self._image_path set? What are the
places in code that call into the path setter? Where do they get the
image name from? Does any of them manipulate the name and/or rename
the file and why?
Should the name include the image name to avoid such clashes and
races? And so on. :)

Alex

Re: wic appending ".1" to a file name on multiple images build

[Alan Martinovic]

Yeah, a deep dive it is then. :)

Am just trying to print from the code using the Python debugger (as
used in the source file).
    self._log.debug("Print this")

but it's not printing anything even when running bitbake with -DDDvvv

On Fri, Jun 1, 2018 at 12:52 PM, Alexander Kanavin
<alex.kanavin@gmail.com> wrote:
> 2018-06-01 13:38 GMT+03:00 Alan Martinovic <alan.martinovic@senic.com>:
>> Building them at the same time causes an issue:
>>
>> Traceback (most recent call lastd):
>>   File "/home/alan/workspace/senic-os-dev/oe/scripts/lib/wic/filemap.py",
>> line 120, in _open_image_file
>>      self._f_image = open(self._image_path, 'rb')
>>   FileNotFoundError: [Errno 2] No such file or directory:
>> '/home/alan/workspace/senic-os-dev/build/tmp-glibc/deploy/images/senic-hub/u-boot-sunxi-with-spl.bin.1'
>>
>>
>> The cause of the issue seems to be that `wic create`,
>> even though it gets the correct file path through the
>> wks file ("u-boot-sunxi-with-spl.bin")
>> somewhere in the execution of wic create that
>> fiile gets renamed (to "u-boot-sunxi-with-spl.bin.1")
>> and results in an above issue.
>>
>> This only happens when the images are built in parallel,
>> running each separately gives no errors.
>>
>> Am open for suggestions on how to approach this. :)
>
> By applying code archaeology skills obviously, which are a must-have
> for anyone involved in Yocto :)
>
> Specifically here: where and how is self._image_path set? What are the
> places in code that call into the path setter? Where do they get the
> image name from? Does any of them manipulate the name and/or rename
> the file and why?
> Should the name include the image name to avoid such clashes and
> races? And so on. :)
>
> Alex

Re: wic appending ".1" to a file name on multiple images build

[Alexander Kanavin]

It probably goes to some log file?

Alex

> On 1 Jun 2018, at 14.56, Alan Martinovic <alan.martinovic@senic.com> wrote:
> 
> Yeah, a deep dive it is then. :)
> 
> Am just trying to print from the code using the Python debugger (as
> used in the source file).
>    self._log.debug("Print this")
> 
> but it's not printing anything even when running bitbake with -DDDvvv
> 
> On Fri, Jun 1, 2018 at 12:52 PM, Alexander Kanavin
> <alex.kanavin@gmail.com> wrote:
>> 2018-06-01 13:38 GMT+03:00 Alan Martinovic <alan.martinovic@senic.com>:
>>> Building them at the same time causes an issue:
>>> 
>>> Traceback (most recent call lastd):
>>>  File "/home/alan/workspace/senic-os-dev/oe/scripts/lib/wic/filemap.py",
>>> line 120, in _open_image_file
>>>     self._f_image = open(self._image_path, 'rb')
>>>  FileNotFoundError: [Errno 2] No such file or directory:
>>> '/home/alan/workspace/senic-os-dev/build/tmp-glibc/deploy/images/senic-hub/u-boot-sunxi-with-spl.bin.1'
>>> 
>>> 
>>> The cause of the issue seems to be that `wic create`,
>>> even though it gets the correct file path through the
>>> wks file ("u-boot-sunxi-with-spl.bin")
>>> somewhere in the execution of wic create that
>>> fiile gets renamed (to "u-boot-sunxi-with-spl.bin.1")
>>> and results in an above issue.
>>> 
>>> This only happens when the images are built in parallel,
>>> running each separately gives no errors.
>>> 
>>> Am open for suggestions on how to approach this. :)
>> 
>> By applying code archaeology skills obviously, which are a must-have
>> for anyone involved in Yocto :)
>> 
>> Specifically here: where and how is self._image_path set? What are the
>> places in code that call into the path setter? Where do they get the
>> image name from? Does any of them manipulate the name and/or rename
>> the file and why?
>> Should the name include the image name to avoid such clashes and
>> races? And so on. :)
>> 
>> Alex

Re: wic appending ".1" to a file name on multiple images build

[Anuj Mittal]

On 06/01/2018 07:56 PM, Alan Martinovic wrote:
> Yeah, a deep dive it is then. :)
> 
> Am just trying to print from the code using the Python debugger (as
> used in the source file).
>     self._log.debug("Print this")
> 
> but it's not printing anything even when running bitbake with -DDDvvv

wic create with -D should help you get started.