All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH] rm_work.bbclass: re-enable recursive do_rm_work_all
@ 2017-03-14 15:29 Patrick Ohly
  2017-03-16 22:32 ` Burton, Ross
  0 siblings, 1 reply; 3+ messages in thread
From: Patrick Ohly @ 2017-03-14 15:29 UTC (permalink / raw)
  To: openembedded-core

When rewriting the do_rm_work injection, do_rm_work_all had been
removed because it seemed obsolete, as do_build now always triggers
do_rm_work.

However, do_build does not get triggered for all recipes and thus
do_rm_work was not called for recipes that got built only
partially. For example, zlib depends indirectly on
zlib-native:do_populate_sysroot.  Because of that dependency,
zlib-native got compiled, but do_rm_work was never called for it.

Re-introducing do_rm_work_all fixes that by making do_build depend on
do_rm_work_all, which then recursively depends on do_rm_work of all
dependencies. This has the unintended side-effect that do_rm_work then
also triggers additional work (like do_populate_lic) that normally
doesn't need to be done for a build. This seems like the lesser evil,
compared to an incomplete cleanup because it mostly enables the
lighter tasks after do_populate_sysroot.

The real solution would be to have two kinds of relationships: a weak
ordering relationship ("if A and B are enabled, A must run before B,
but B can also run without A") and hard dependencies ("B cannot run
unless A has run before").

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
---
 meta/classes/rm_work.bbclass | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/meta/classes/rm_work.bbclass b/meta/classes/rm_work.bbclass
index 054c937..2490a69 100644
--- a/meta/classes/rm_work.bbclass
+++ b/meta/classes/rm_work.bbclass
@@ -102,6 +102,12 @@ do_rm_work () {
         rm -f $i
     done
 }
+do_rm_work_all () {
+    :
+}
+do_rm_work_all[recrdeptask] = "do_rm_work"
+do_rm_work_all[noexec] = "1"
+addtask rm_work_all after before do_build
 
 do_populate_sdk[postfuncs] += "rm_work_populatesdk"
 rm_work_populatesdk () {
@@ -138,9 +144,8 @@ python inject_rm_work() {
         # do_build inherits additional runtime dependencies on
         # other recipes and thus will typically run much later than completion of
         # work in the recipe itself.
-        deps = bb.build.preceedtask('do_build', True, d)
-        if 'do_build' in deps:
-            deps.remove('do_build')
+        deps = set(bb.build.preceedtask('do_build', True, d))
+        deps.difference_update(('do_build', 'do_rm_work_all'))
         # In practice, addtask() here merely updates the dependencies.
         bb.build.addtask('do_rm_work', 'do_build', ' '.join(deps), d)
 }

base-commit: 477805b913a6c4b4b630e42f08cd9e59f1e4e254
-- 
git-series 0.9.1


^ permalink raw reply related	[flat|nested] 3+ messages in thread

* Re: [PATCH] rm_work.bbclass: re-enable recursive do_rm_work_all
  2017-03-14 15:29 [PATCH] rm_work.bbclass: re-enable recursive do_rm_work_all Patrick Ohly
@ 2017-03-16 22:32 ` Burton, Ross
  2017-03-17  7:20   ` Patrick Ohly
  0 siblings, 1 reply; 3+ messages in thread
From: Burton, Ross @ 2017-03-16 22:32 UTC (permalink / raw)
  To: Patrick Ohly; +Cc: OE-core

[-- Attachment #1: Type: text/plain, Size: 3386 bytes --]

On 14 March 2017 at 15:29, Patrick Ohly <patrick.ohly@intel.com> wrote:

> When rewriting the do_rm_work injection, do_rm_work_all had been
> removed because it seemed obsolete, as do_build now always triggers
> do_rm_work.
>
> However, do_build does not get triggered for all recipes and thus
> do_rm_work was not called for recipes that got built only
> partially. For example, zlib depends indirectly on
> zlib-native:do_populate_sysroot.  Because of that dependency,
> zlib-native got compiled, but do_rm_work was never called for it.
>
> Re-introducing do_rm_work_all fixes that by making do_build depend on
> do_rm_work_all, which then recursively depends on do_rm_work of all
> dependencies. This has the unintended side-effect that do_rm_work then
> also triggers additional work (like do_populate_lic) that normally
> doesn't need to be done for a build. This seems like the lesser evil,
> compared to an incomplete cleanup because it mostly enables the
> lighter tasks after do_populate_sysroot.
>
> The real solution would be to have two kinds of relationships: a weak
> ordering relationship ("if A and B are enabled, A must run before B,
> but B can also run without A") and hard dependencies ("B cannot run
> unless A has run before").
>

If I do a build with rm_work enabled I tend to get this error at rootfs
time:

ERROR: core-image-sato-1.0-r0 do_image_wic: Error executing a python
function in exec_python_func() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:extend_recipe_sysroot(d)
     0003:
File: '/home/ross/Yocto/poky/meta/classes/staging.bbclass', lineno: 623,
function: extend_recipe_sysroot
     0619:                    dest = newmanifest[l]
     0620:                    if l.endswith("/"):
     0621:                        staging_copydir(l, targetdir, dest,
seendirs)
     0622:                        continue
 *** 0623:                    staging_copyfile(l, targetdir, dest,
postinsts, seendirs)
     0624:
     0625:    for f in fixme:
     0626:        if f == '':
     0627:            staging_processfixme(fixme[f], recipesysroot,
recipesysroot, recipesysrootnative, d)
File: '/home/ross/Yocto/poky/meta/classes/staging.bbclass', lineno: 269,
function: staging_copyfile
     0265:        os.symlink(linkto, dest)
     0266:        #bb.warn(c)
     0267:    else:
     0268:        try:
 *** 0269:            os.link(c, dest)
     0270:        except OSError as err:
     0271:            if err.errno == errno.EXDEV:
     0272:                bb.utils.copyfile(c, dest)
     0273:            else:
Exception: FileExistsError: [Errno 17] File exists:
'/data/poky-master/tmp/sysroots-components/corei7-64/glibc/usr/lib/crt1.o'
->
'/data/poky-master/tmp/work/intel_corei7_64-poky-linux/core-image-sato/1.0-r0/recipe-sysroot/usr/lib/crt1.o'

ERROR: core-image-sato-1.0-r0 do_image_wic: Function failed:
extend_recipe_sysroot
ERROR: Logfile of failure stored in:
/data/poky-master/tmp/work/intel_corei7_64-poky-linux/core-image-sato/1.0-r0/temp/log.do_image_wic.21552
ERROR: Task
(/home/ross/Yocto/poky/meta/recipes-sato/images/core-image-sato.bb:do_image_wic)
failed with exit code '1'

Reverting this patch makes it go away.

Ross

[-- Attachment #2: Type: text/html, Size: 5245 bytes --]

^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: [PATCH] rm_work.bbclass: re-enable recursive do_rm_work_all
  2017-03-16 22:32 ` Burton, Ross
@ 2017-03-17  7:20   ` Patrick Ohly
  0 siblings, 0 replies; 3+ messages in thread
From: Patrick Ohly @ 2017-03-17  7:20 UTC (permalink / raw)
  To: Burton, Ross; +Cc: OE-core

On Thu, 2017-03-16 at 22:32 +0000, Burton, Ross wrote:
> If I do a build with rm_work enabled I tend to get this error at
> rootfs time:
> 
> 
> ERROR: core-image-sato-1.0-r0 do_image_wic: Error executing a python
> function in exec_python_func() autogenerated:
....
> Exception: FileExistsError: [Errno 17] File exists:
> '/data/poky-master/tmp/sysroots-components/corei7-64/glibc/usr/lib/crt1.o' -> '/data/poky-master/tmp/work/intel_corei7_64-poky-linux/core-image-sato/1.0-r0/recipe-sysroot/usr/lib/crt1.o'
> 
> 
> ERROR: core-image-sato-1.0-r0 do_image_wic: Function failed:
> extend_recipe_sysroot
> ERROR: Logfile of failure stored
> in: /data/poky-master/tmp/work/intel_corei7_64-poky-linux/core-image-sato/1.0-r0/temp/log.do_image_wic.21552
> ERROR: Task
> (/home/ross/Yocto/poky/meta/recipes-sato/images/core-image-sato.bb:do_image_wic) failed with exit code '1'
> 
> 
> Reverting this patch makes it go away.

I think I have seen that before, but not necessarily (not sure anymore)
in the context of rm_work. If I remember correctly, these were really
two different crt1.o files, one from glibc, the other from
glibc-initial.

I couldn't reproduce it at that time. I'll check whether I can trigger
it now.

-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.





^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2017-03-17  7:20 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-03-14 15:29 [PATCH] rm_work.bbclass: re-enable recursive do_rm_work_all Patrick Ohly
2017-03-16 22:32 ` Burton, Ross
2017-03-17  7:20   ` Patrick Ohly

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.