* [PATCH] rm_work.bbclass: re-enable recursive do_rm_work_all
@ 2017-03-14 15:29 Patrick Ohly
2017-03-16 22:32 ` Burton, Ross
0 siblings, 1 reply; 3+ messages in thread
From: Patrick Ohly @ 2017-03-14 15:29 UTC (permalink / raw)
To: openembedded-core
When rewriting the do_rm_work injection, do_rm_work_all had been
removed because it seemed obsolete, as do_build now always triggers
do_rm_work.
However, do_build does not get triggered for all recipes and thus
do_rm_work was not called for recipes that got built only
partially. For example, zlib depends indirectly on
zlib-native:do_populate_sysroot. Because of that dependency,
zlib-native got compiled, but do_rm_work was never called for it.
Re-introducing do_rm_work_all fixes that by making do_build depend on
do_rm_work_all, which then recursively depends on do_rm_work of all
dependencies. This has the unintended side-effect that do_rm_work then
also triggers additional work (like do_populate_lic) that normally
doesn't need to be done for a build. This seems like the lesser evil,
compared to an incomplete cleanup because it mostly enables the
lighter tasks after do_populate_sysroot.
The real solution would be to have two kinds of relationships: a weak
ordering relationship ("if A and B are enabled, A must run before B,
but B can also run without A") and hard dependencies ("B cannot run
unless A has run before").
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
---
meta/classes/rm_work.bbclass | 11 ++++++++---
1 file changed, 8 insertions(+), 3 deletions(-)
diff --git a/meta/classes/rm_work.bbclass b/meta/classes/rm_work.bbclass
index 054c937..2490a69 100644
--- a/meta/classes/rm_work.bbclass
+++ b/meta/classes/rm_work.bbclass
@@ -102,6 +102,12 @@ do_rm_work () {
rm -f $i
done
}
+do_rm_work_all () {
+ :
+}
+do_rm_work_all[recrdeptask] = "do_rm_work"
+do_rm_work_all[noexec] = "1"
+addtask rm_work_all after before do_build
do_populate_sdk[postfuncs] += "rm_work_populatesdk"
rm_work_populatesdk () {
@@ -138,9 +144,8 @@ python inject_rm_work() {
# do_build inherits additional runtime dependencies on
# other recipes and thus will typically run much later than completion of
# work in the recipe itself.
- deps = bb.build.preceedtask('do_build', True, d)
- if 'do_build' in deps:
- deps.remove('do_build')
+ deps = set(bb.build.preceedtask('do_build', True, d))
+ deps.difference_update(('do_build', 'do_rm_work_all'))
# In practice, addtask() here merely updates the dependencies.
bb.build.addtask('do_rm_work', 'do_build', ' '.join(deps), d)
}
base-commit: 477805b913a6c4b4b630e42f08cd9e59f1e4e254
--
git-series 0.9.1
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [PATCH] rm_work.bbclass: re-enable recursive do_rm_work_all
2017-03-14 15:29 [PATCH] rm_work.bbclass: re-enable recursive do_rm_work_all Patrick Ohly
@ 2017-03-16 22:32 ` Burton, Ross
2017-03-17 7:20 ` Patrick Ohly
0 siblings, 1 reply; 3+ messages in thread
From: Burton, Ross @ 2017-03-16 22:32 UTC (permalink / raw)
To: Patrick Ohly; +Cc: OE-core
[-- Attachment #1: Type: text/plain, Size: 3386 bytes --]
On 14 March 2017 at 15:29, Patrick Ohly <patrick.ohly@intel.com> wrote:
> When rewriting the do_rm_work injection, do_rm_work_all had been
> removed because it seemed obsolete, as do_build now always triggers
> do_rm_work.
>
> However, do_build does not get triggered for all recipes and thus
> do_rm_work was not called for recipes that got built only
> partially. For example, zlib depends indirectly on
> zlib-native:do_populate_sysroot. Because of that dependency,
> zlib-native got compiled, but do_rm_work was never called for it.
>
> Re-introducing do_rm_work_all fixes that by making do_build depend on
> do_rm_work_all, which then recursively depends on do_rm_work of all
> dependencies. This has the unintended side-effect that do_rm_work then
> also triggers additional work (like do_populate_lic) that normally
> doesn't need to be done for a build. This seems like the lesser evil,
> compared to an incomplete cleanup because it mostly enables the
> lighter tasks after do_populate_sysroot.
>
> The real solution would be to have two kinds of relationships: a weak
> ordering relationship ("if A and B are enabled, A must run before B,
> but B can also run without A") and hard dependencies ("B cannot run
> unless A has run before").
>
If I do a build with rm_work enabled I tend to get this error at rootfs
time:
ERROR: core-image-sato-1.0-r0 do_image_wic: Error executing a python
function in exec_python_func() autogenerated:
The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
0001:
*** 0002:extend_recipe_sysroot(d)
0003:
File: '/home/ross/Yocto/poky/meta/classes/staging.bbclass', lineno: 623,
function: extend_recipe_sysroot
0619: dest = newmanifest[l]
0620: if l.endswith("/"):
0621: staging_copydir(l, targetdir, dest,
seendirs)
0622: continue
*** 0623: staging_copyfile(l, targetdir, dest,
postinsts, seendirs)
0624:
0625: for f in fixme:
0626: if f == '':
0627: staging_processfixme(fixme[f], recipesysroot,
recipesysroot, recipesysrootnative, d)
File: '/home/ross/Yocto/poky/meta/classes/staging.bbclass', lineno: 269,
function: staging_copyfile
0265: os.symlink(linkto, dest)
0266: #bb.warn(c)
0267: else:
0268: try:
*** 0269: os.link(c, dest)
0270: except OSError as err:
0271: if err.errno == errno.EXDEV:
0272: bb.utils.copyfile(c, dest)
0273: else:
Exception: FileExistsError: [Errno 17] File exists:
'/data/poky-master/tmp/sysroots-components/corei7-64/glibc/usr/lib/crt1.o'
->
'/data/poky-master/tmp/work/intel_corei7_64-poky-linux/core-image-sato/1.0-r0/recipe-sysroot/usr/lib/crt1.o'
ERROR: core-image-sato-1.0-r0 do_image_wic: Function failed:
extend_recipe_sysroot
ERROR: Logfile of failure stored in:
/data/poky-master/tmp/work/intel_corei7_64-poky-linux/core-image-sato/1.0-r0/temp/log.do_image_wic.21552
ERROR: Task
(/home/ross/Yocto/poky/meta/recipes-sato/images/core-image-sato.bb:do_image_wic)
failed with exit code '1'
Reverting this patch makes it go away.
Ross
[-- Attachment #2: Type: text/html, Size: 5245 bytes --]
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [PATCH] rm_work.bbclass: re-enable recursive do_rm_work_all
2017-03-16 22:32 ` Burton, Ross
@ 2017-03-17 7:20 ` Patrick Ohly
0 siblings, 0 replies; 3+ messages in thread
From: Patrick Ohly @ 2017-03-17 7:20 UTC (permalink / raw)
To: Burton, Ross; +Cc: OE-core
On Thu, 2017-03-16 at 22:32 +0000, Burton, Ross wrote:
> If I do a build with rm_work enabled I tend to get this error at
> rootfs time:
>
>
> ERROR: core-image-sato-1.0-r0 do_image_wic: Error executing a python
> function in exec_python_func() autogenerated:
....
> Exception: FileExistsError: [Errno 17] File exists:
> '/data/poky-master/tmp/sysroots-components/corei7-64/glibc/usr/lib/crt1.o' -> '/data/poky-master/tmp/work/intel_corei7_64-poky-linux/core-image-sato/1.0-r0/recipe-sysroot/usr/lib/crt1.o'
>
>
> ERROR: core-image-sato-1.0-r0 do_image_wic: Function failed:
> extend_recipe_sysroot
> ERROR: Logfile of failure stored
> in: /data/poky-master/tmp/work/intel_corei7_64-poky-linux/core-image-sato/1.0-r0/temp/log.do_image_wic.21552
> ERROR: Task
> (/home/ross/Yocto/poky/meta/recipes-sato/images/core-image-sato.bb:do_image_wic) failed with exit code '1'
>
>
> Reverting this patch makes it go away.
I think I have seen that before, but not necessarily (not sure anymore)
in the context of rm_work. If I remember correctly, these were really
two different crt1.o files, one from glibc, the other from
glibc-initial.
I couldn't reproduce it at that time. I'll check whether I can trigger
it now.
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2017-03-17 7:20 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-03-14 15:29 [PATCH] rm_work.bbclass: re-enable recursive do_rm_work_all Patrick Ohly
2017-03-16 22:32 ` Burton, Ross
2017-03-17 7:20 ` Patrick Ohly
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.