Are all changes necessary for this to work already in master? Yesterday I've noticed that rm_work for some components which are early in the dependency (like qtbase) are executed relatively late (together with do_package_qa). So I've tried very naive way to find out if the rm_work tasks are executed sooner or not just by comparing Task IDs in build of the same image built from scratch (without sstate) with Dizzy, Morty and current master. First I've stripped unnecessary prefix and names of proprietary components (in case someone wants me to share these lists): grep "^NOTE: Running task .*, do_rm_work)$" log.build |sed 's#/jenkins/mjansa/build-[^/]*/##; s#meta-lg-webos/[^:]*:#private-component:#g; s#^NOTE: Running task ##g' > rm_work.tasks.dizzy with slightly different regexp for morty and master: grep "^NOTE: Running task .*:do_rm_work)$" ld.gold/log.m16p |sed 's#/jenkins/mjansa/build-[^/]*/##; s#meta-lg-webos/[^:]*:#private-component:#g; s#^NOTE: Running task ##g' > rm_work.tasks.morty and then I did even more naive thing to compare average task id of rm-work jobs with following results: for i in rm_work.tasks.*; do echo $i; export COUNT=0 SUM=0; for TASK in `cat $i | cut -f 1 -d\ `; do COUNT=`expr $COUNT + 1`; SUM=`expr $SUM + $TASK`; done; echo "AVG = `expr $SUM / $C OUNT`; COUNT = $COUNT"; done rm_work.tasks.dizzy AVG = 6429; COUNT = 764 rm_work.tasks.master AVG = 7570; COUNT = 891 rm_work.tasks.master.qemux86 AVG = 5527; COUNT = 665 rm_work.tasks.morty AVG = 6689; COUNT = 786 rm_work.tasks.morty.gold AVG = 6764; COUNT = 786 rm_work.tasks.morty.gold is the same build as in rm_work.tasks.morty just with ld-is-gold added to DISTRO_FEATUREs (as I was testing build time to compare ld.bfd and ld.gold in our images). rm_work.tasks.master.qemux86 is the same build as rm_work.tasks.master but for qemux86, all other builds are for some ARM board we use Then few interesting steps: gcc-cross looks good (not available for dizzy build which is using external toolchain) $ grep gcc-cross_ rm_work.tasks.* rm_work.tasks.master:510 of 14470 (oe-core/meta/recipes-devtools/gcc/gcc-cross_6.3.bb:do_rm_work) rm_work.tasks.master.qemux86:515 of 10296 (oe-core/meta/recipes-devtools/gcc/gcc-cross_6.3.bb:do_rm_work) rm_work.tasks.morty:2592 of 12021 (oe-core/meta/recipes-devtools/gcc/gcc-cross_6.2.bb:do_rm_work) rm_work.tasks.morty.gold:2734 of 12021 (oe-core/meta/recipes-devtools/gcc/gcc-cross_6.2.bb:do_rm_work) qtdeclarative-native got rm_work a bit later, whcih might be caused only by the increased number of tasks thanks to RSS $ grep native.*qtdeclarative rm_work.tasks.* rm_work.tasks.dizzy:2101 of 11766 (ID: 11128, virtual:native:meta-qt5/recipes-qt/qt5/qtdeclarative_git.bb, do_rm_work) rm_work.tasks.master:2614 of 14470 (virtual:native:meta-qt5/recipes-qt/qt5/qtdeclarative_git.bb:do_rm_work) rm_work.tasks.master.qemux86:2521 of 10296 (virtual:native:meta-qt5/recipes-qt/qt5/qtdeclarative_git.bb:do_rm_work) rm_work.tasks.morty:1513 of 12021 (virtual:native:meta-qt5/recipes-qt/qt5/qtdeclarative_git.bb:do_rm_work) rm_work.tasks.morty.gold:1514 of 12021 (virtual:native:meta-qt5/recipes-qt/qt5/qtdeclarative_git.bb:do_rm_work) and here is the target qtdeclarative which trigered this whole naive analysis: $ grep qtdeclarative rm_work.tasks.* | grep -v native rm_work.tasks.dizzy:4952 of 11766 (ID: 6670, meta-qt5/recipes-qt/qt5/ qtdeclarative_git.bb, do_rm_work) rm_work.tasks.master:4317 of 14470 (meta-qt5/recipes-qt/qt5/qtdeclarative_git.bb:do_rm_work) rm_work.tasks.master.qemux86:10142 of 10296 (meta-qt5/recipes-qt/qt5/qtdeclarative_git.bb:do_rm_work) rm_work.tasks.morty:6753 of 12021 (meta-qt5/recipes-qt/qt5/qtdeclarative_git.bb:do_rm_work) rm_work.tasks.morty.gold:6883 of 12021 (meta-qt5/recipes-qt/qt5/qtdeclarative_git.bb:do_rm_work) If we dismiss the strange case in rm_work.tasks.master.qemux86 then it seems to perform at least as good as old completion BB_SCHEDULER. But I wanted to ask if there is something else we can do or you were planing to do, because IIRC you shared some longer analysis of what could be improved here and I'm not sure if you managed to implement it all. It feels to me that rm_work has high priority, but still it's "blocked" by e.g. do_package_qa which gets executed late and then immediately followed by rm_work. in ideal case I would really like to have a switch which will force rm_work to take absolute priority over other tasks, it doesn't take very long to delete the files in tmpfs and would allow me to do tmpfs builds on builders with smaller RAM. The "state of bitbake world" builds are performed in 74G tmpfs (for whole tmpdir-glibc) and yesterday's builds started to fail again (when it happens to run chromium and chromium-wayland at the same time) - the manual solution for this I'm using for last couple years is to build in "steps" which force to run rm_work for all included components, so e.g. bitbake gcc-cross-arm && bitbake small-image && bitbake chromium && bitbake chromium-wayland && bitbake big-image && bitbake world will keep the tmpfs usage peaks much lower than running just bitbake world On Fri, Jan 6, 2017 at 10:55 AM, Patrick Ohly wrote: > Having do_rm_work depend on do_build had one major disadvantage: > do_build depends on the do_build of other recipes, to ensure that > runtime dependencies also get built. The effect is that when work on a > recipe is complete and it could get cleaned up, do_rm_work still > doesn't run because it waits for those other recipes, thus leading to > more temporary disk space usage than really needed. > > The right solution is to inject do_rm_work before do_build and after > all tasks of the recipe. Achieving that depends on the new bitbake > support for prioritizing anonymous functions to ensure that > rm_work.bbclass gets to see a full set of existing tasks when adding > its own one. This is relevant, for example, for do_analyseimage in > meta-security-isafw's isafw.bbclass. > > In addition, the new "rm_work" scheduler is used by default. It > prioritizes finishing recipes over continuing with the more > important recipes (with "importance" determined by the number of > reverse-dependencies). > > Benchmarking (see "rm_work + pybootchart enhancements" on the OE-core > mailing list) showed that builds with the modified rm_work.bbclass > were both faster (albeit not by much) and required considerably less > disk space (14230MiB instead of 18740MiB for core-image-sato). > Interestingly enough, builds with rm_work.bbclass were also faster > than those without. > > Signed-off-by: Patrick Ohly > --- > meta/classes/rm_work.bbclass | 31 ++++++++++++++++++------------- > 1 file changed, 18 insertions(+), 13 deletions(-) > > diff --git a/meta/classes/rm_work.bbclass b/meta/classes/rm_work.bbclass > index 3516c7e..1205104 100644 > --- a/meta/classes/rm_work.bbclass > +++ b/meta/classes/rm_work.bbclass > @@ -11,16 +11,13 @@ > # RM_WORK_EXCLUDE += "icu-native icu busybox" > # > > -# Use the completion scheduler by default when rm_work is active > +# Use the dedicated rm_work scheduler by default when rm_work is active > # to try and reduce disk usage > -BB_SCHEDULER ?= "completion" > +BB_SCHEDULER ?= "rm_work" > > # Run the rm_work task in the idle scheduling class > BB_TASK_IONICE_LEVEL_task-rm_work = "3.0" > > -RMWORK_ORIG_TASK := "${BB_DEFAULT_TASK}" > -BB_DEFAULT_TASK = "rm_work_all" > - > do_rm_work () { > # If the recipe name is in the RM_WORK_EXCLUDE, skip the recipe. > for p in ${RM_WORK_EXCLUDE}; do > @@ -97,13 +94,6 @@ do_rm_work () { > rm -f $i > done > } > -addtask rm_work after do_${RMWORK_ORIG_TASK} > - > -do_rm_work_all () { > - : > -} > -do_rm_work_all[recrdeptask] = "do_rm_work" > -addtask rm_work_all after do_rm_work > > do_populate_sdk[postfuncs] += "rm_work_populatesdk" > rm_work_populatesdk () { > @@ -117,7 +107,7 @@ rm_work_rootfs () { > } > rm_work_rootfs[cleandirs] = "${WORKDIR}/rootfs" > > -python () { > +python __anonymous_rm_work() { > if bb.data.inherits_class('kernel', d): > d.appendVar("RM_WORK_EXCLUDE", ' ' + d.getVar("PN")) > # If the recipe name is in the RM_WORK_EXCLUDE, skip the recipe. > @@ -126,4 +116,19 @@ python () { > if pn in excludes: > d.delVarFlag('rm_work_rootfs', 'cleandirs') > d.delVarFlag('rm_work_populatesdk', 'cleandirs') > + else: > + # Inject do_rm_work into the tasks of the current recipe such > that do_build > + # depends on it and that it runs after all other tasks that block > do_build, > + # i.e. after all work on the current recipe is done. The reason > for taking > + # this approach instead of making do_rm_work depend on do_build > is that > + # do_build inherits additional runtime dependencies on > + # other recipes and thus will typically run much later than > completion of > + # work in the recipe itself. > + deps = bb.build.preceedtask('do_build', True, d) > + if 'do_build' in deps: > + deps.remove('do_build') > + bb.build.addtask('do_rm_work', 'do_build', ' '.join(deps), d) > } > +# Higher priority than the normal 100, and thus we run after other > +# classes like package_rpm.bbclass which also add custom tasks. > +__anonymous_rm_work[__anonprio] = "1000" > -- > 2.1.4 > > -- > _______________________________________________ > Openembedded-core mailing list > Openembedded-core@lists.openembedded.org > http://lists.openembedded.org/mailman/listinfo/openembedded-core >