From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qk0-f177.google.com (mail-qk0-f177.google.com [209.85.220.177]) by mail.openembedded.org (Postfix) with ESMTP id 4FEEA77935 for ; Thu, 16 Mar 2017 22:32:36 +0000 (UTC) Received: by mail-qk0-f177.google.com with SMTP id 1so51915861qkl.3 for ; Thu, 16 Mar 2017 15:32:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=F5ybEh4cJVfE2MG58RF/FrGahIGKnIH8GcVgVE0V6kM=; b=JkLSh2/CGUHaep43Ovuz3NtT8kPeCUhgW8KX+no7iOZ2zPQyfY53UPHV+SYpKqqd4Y M7X+rEN+z9IjHt0XXc+lI11dqGChD/uhauuB3Atgus1oVYEr1iQfJ4quV/jKq0vec9Tw QBbTe8mcBjgUZkdKkD+cpK07N0wObj1tgIAwVJYnWRq0TqtBPMgvrM1KahbH46Xh8tCs xloA+ZZcos7cKBbM2sF5v/IozFSQFyP0Qp6fZTrbC4cwGLQUICb+mSgxzJ4sZGPq2imM Lwz2HeGSARn8Dnkj/byo68VbI3VlLI7ocXV7ZlrvETK/HHeb+4DjH/OvoNODBVvU109S fmYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=F5ybEh4cJVfE2MG58RF/FrGahIGKnIH8GcVgVE0V6kM=; b=fGi4G9AP9iAwbU+yjcAqvt5hcTiVIYQyK0HxeoBKC7igTeVKH6UGmNzRSAkjZparCN uodtdcaXNQ57vSqH+ILmn2H/i+fC3F6cnEO4uvvsZM2WmgafwxZayEG731RyFJPurJ6d w3XQBcBRS5GGDLKI4o5Y4KRRS0ZA6yREFuiS60U8roao0v17lwtswY1cH6OG3Y7nVYMf madmCYg5gFPIkkrVt1eWb+n83j9Tu4tbiZy7jKonNwftHUKv3flp/O9omNuXJ4tUzfBZ 9IVyLw+0Un/A7hfBd8xjGfltbDHStZmMuReu5fkCezscirpWKOyZht9Vj+ppmEdguqsq p4rA== X-Gm-Message-State: AFeK/H1cOG2Wyb5PYuiZ1muKFyGb0LR4ZlnrCnk8hzWvgQNBUdZDH60yAqWngRPIMAqStpXqjFc8hiR+o+2kH0hd X-Received: by 10.55.175.195 with SMTP id y186mr10212905qke.26.1489703557858; Thu, 16 Mar 2017 15:32:37 -0700 (PDT) MIME-Version: 1.0 Received: by 10.140.22.166 with HTTP; Thu, 16 Mar 2017 15:32:17 -0700 (PDT) In-Reply-To: References: From: "Burton, Ross" Date: Thu, 16 Mar 2017 22:32:17 +0000 Message-ID: To: Patrick Ohly Cc: OE-core Subject: Re: [PATCH] rm_work.bbclass: re-enable recursive do_rm_work_all X-BeenThere: openembedded-core@lists.openembedded.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Patches and discussions about the oe-core layer List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Mar 2017 22:32:37 -0000 Content-Type: multipart/alternative; boundary=94eb2c06093c63734d054ae0a31b --94eb2c06093c63734d054ae0a31b Content-Type: text/plain; charset=UTF-8 On 14 March 2017 at 15:29, Patrick Ohly wrote: > When rewriting the do_rm_work injection, do_rm_work_all had been > removed because it seemed obsolete, as do_build now always triggers > do_rm_work. > > However, do_build does not get triggered for all recipes and thus > do_rm_work was not called for recipes that got built only > partially. For example, zlib depends indirectly on > zlib-native:do_populate_sysroot. Because of that dependency, > zlib-native got compiled, but do_rm_work was never called for it. > > Re-introducing do_rm_work_all fixes that by making do_build depend on > do_rm_work_all, which then recursively depends on do_rm_work of all > dependencies. This has the unintended side-effect that do_rm_work then > also triggers additional work (like do_populate_lic) that normally > doesn't need to be done for a build. This seems like the lesser evil, > compared to an incomplete cleanup because it mostly enables the > lighter tasks after do_populate_sysroot. > > The real solution would be to have two kinds of relationships: a weak > ordering relationship ("if A and B are enabled, A must run before B, > but B can also run without A") and hard dependencies ("B cannot run > unless A has run before"). > If I do a build with rm_work enabled I tend to get this error at rootfs time: ERROR: core-image-sato-1.0-r0 do_image_wic: Error executing a python function in exec_python_func() autogenerated: The stack trace of python calls that resulted in this exception/failure was: File: 'exec_python_func() autogenerated', lineno: 2, function: 0001: *** 0002:extend_recipe_sysroot(d) 0003: File: '/home/ross/Yocto/poky/meta/classes/staging.bbclass', lineno: 623, function: extend_recipe_sysroot 0619: dest = newmanifest[l] 0620: if l.endswith("/"): 0621: staging_copydir(l, targetdir, dest, seendirs) 0622: continue *** 0623: staging_copyfile(l, targetdir, dest, postinsts, seendirs) 0624: 0625: for f in fixme: 0626: if f == '': 0627: staging_processfixme(fixme[f], recipesysroot, recipesysroot, recipesysrootnative, d) File: '/home/ross/Yocto/poky/meta/classes/staging.bbclass', lineno: 269, function: staging_copyfile 0265: os.symlink(linkto, dest) 0266: #bb.warn(c) 0267: else: 0268: try: *** 0269: os.link(c, dest) 0270: except OSError as err: 0271: if err.errno == errno.EXDEV: 0272: bb.utils.copyfile(c, dest) 0273: else: Exception: FileExistsError: [Errno 17] File exists: '/data/poky-master/tmp/sysroots-components/corei7-64/glibc/usr/lib/crt1.o' -> '/data/poky-master/tmp/work/intel_corei7_64-poky-linux/core-image-sato/1.0-r0/recipe-sysroot/usr/lib/crt1.o' ERROR: core-image-sato-1.0-r0 do_image_wic: Function failed: extend_recipe_sysroot ERROR: Logfile of failure stored in: /data/poky-master/tmp/work/intel_corei7_64-poky-linux/core-image-sato/1.0-r0/temp/log.do_image_wic.21552 ERROR: Task (/home/ross/Yocto/poky/meta/recipes-sato/images/core-image-sato.bb:do_image_wic) failed with exit code '1' Reverting this patch makes it go away. Ross --94eb2c06093c63734d054ae0a31b Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

= On 14 March 2017 at 15:29, Patrick Ohly <patrick.ohly@intel.com&g= t; wrote:
When rewriting the do_rm_work inje= ction, do_rm_work_all had been
removed because it seemed obsolete, as do_build now always triggers
do_rm_work.

However, do_build does not get triggered for all recipes and thus
do_rm_work was not called for recipes that got built only
partially. For example, zlib depends indirectly on
zlib-native:do_populate_sysroot.=C2=A0 Because of that dependency,
zlib-native got compiled, but do_rm_work was never called for it.

Re-introducing do_rm_work_all fixes that by making do_build depend on
do_rm_work_all, which then recursively depends on do_rm_work of all
dependencies. This has the unintended side-effect that do_rm_work then
also triggers additional work (like do_populate_lic) that normally
doesn't need to be done for a build. This seems like the lesser evil, compared to an incomplete cleanup because it mostly enables the
lighter tasks after do_populate_sysroot.

The real solution would be to have two kinds of relationships: a weak
ordering relationship ("if A and B are enabled, A must run before B, but B can also run without A") and hard dependencies ("B cannot r= un
unless A has run before").

If I do a b= uild with rm_work enabled I tend to get this error at rootfs time:

ERROR: core-image-sato-1.0-r0 do_image_wic: Error executing a = python function in exec_python_func() autogenerated:

The stack trace of python ca= lls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function= : <module>
=C2=A0 =C2=A0 =C2=A00001:<= /div>
=C2=A0*** 0002:extend_recipe_sysroot(d)
=C2=A0 =C2=A0 =C2=A00003:
File: '/home/ross/Yocto/poky/meta/classes/staging.bbclass&#= 39;, lineno: 623, function: extend_recipe_sysroot
=C2=A0 =C2=A0 =C2=A00619: =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0dest =3D newmanifest[l]
=C2=A0 =C2=A0 =C2=A00620: =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0if l.endswith("/"):
=C2=A0 =C2=A0 =C2=A00621: =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0staging_copydir(l, targ= etdir, dest, seendirs)
=C2=A0 =C2=A0 =C2=A0= 0622: =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0continue
=C2=A0*** 0623: =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0staging_c= opyfile(l, targetdir, dest, postinsts, seendirs)
=C2=A0 =C2=A0 =C2=A00624:
=C2=A0 =C2= =A0 =C2=A00625: =C2=A0 =C2=A0for f in fixme:
=C2=A0 =C2=A0 =C2=A00626: =C2=A0 =C2=A0 =C2=A0 =C2=A0if f =3D=3D '= 9;:
=C2=A0 =C2=A0 =C2=A00627: =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0staging_processfixme(fixme[f], recipesysroot, r= ecipesysroot, recipesysrootnative, d)
File:= '/home/ross/Yocto/poky/meta/classes/staging.bbclass', lineno: 269,= function: staging_copyfile
=C2=A0 =C2=A0 = =C2=A00265: =C2=A0 =C2=A0 =C2=A0 =C2=A0os.symlink(linkto, dest)
=C2=A0 =C2=A0 =C2=A00266: =C2=A0 =C2=A0 =C2=A0 =C2=A0#= bb.warn(c)
=C2=A0 =C2=A0 =C2=A00267: =C2=A0= =C2=A0else:
=C2=A0 =C2=A0 =C2=A00268: =C2= =A0 =C2=A0 =C2=A0 =C2=A0try:
=C2=A0*** 0269= : =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0os.link(c, dest)
=C2=A0 =C2=A0 =C2=A00270: =C2=A0 =C2=A0 =C2=A0 =C2=A0excep= t OSError as err:
=C2=A0 =C2=A0 =C2=A00271:= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if err.errno =3D=3D errno.EXDEV:<= /div>
=C2=A0 =C2=A0 =C2=A00272: =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0bb.utils.copyfile(c, dest)
=C2=A0 =C2=A0 =C2=A00273: =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0else:
Exception: FileExistsEr= ror: [Errno 17] File exists: '/data/poky-master/tmp/sysroots-components= /corei7-64/glibc/usr/lib/crt1.o' -> '/data/poky-master/tmp/work/= intel_corei7_64-poky-linux/core-image-sato/1.0-r0/recipe-sysroot/usr/lib/cr= t1.o'

ERROR: core-image-sato-1.0-r0 do_image_wic: Function failed: extend_rec= ipe_sysroot
ERROR: Logfile of failure store= d in: /data/poky-master/tmp/work/intel_corei7_64-poky-linux/core-image-sato= /1.0-r0/temp/log.do_image_wic.21552
ERROR: = Task (/home/ross/Yocto/poky/meta/recipes-sato/images/core-image-sato.bb:do_= image_wic) failed with exit code '1'

Reverting this patch makes it go awa= y.

Ros= s
--94eb2c06093c63734d054ae0a31b--