Re: [PATCH 1/1] rm_work.bbclass: reserve 'image' folder when remove WORKDIR

["Xu, Dongxiao" <dongxiao.xu@intel.com>]

Richard Purdie wrote:
> On Wed, 2011-01-26 at 14:30 +0800, poky-bounces@yoctoproject.org
> wrote: 
>> From: Dongxiao Xu <dongxiao.xu@intel.com>
>> 
>> After the implementation of per machine sysroot, do_populate_sysroot
>> and do_package need to be re-run if we build two machines of one
>> architecture (in case that sstate prebuilt result could not be reused
>> for certain recipes). But if removing all the WORKDIR contents after
>> the first macine build, the second machine's do_populate_sysroot and
>> do_package will fail due to the missing of "image" directory.
>> 
>> Thus reserve 'image' folder when doing rm_work.
> 
> Whilst I understand the intent, this is just putting wallpaper over
> cracks, sooner or later the crack will reappear, possibly worse than
> before.  
> 
> For example if you run "bitbake xxx -c install -f" for a target xxx
> which has had rm_work run against it, the result will be a nasty
> error in Poky.  
> 
> OE's rm_work.bbclass is a little cleverer than ours and has this:
> 
> addhandler rmwork_stampfixing_eventhandler python
>     rmwork_stampfixing_eventhandler() { if bb.event.getName(e) ==
>         "StampUpdate": for (fn, task) in e.targets:
>             if task == 'rm_work_all':
>                 continue
>             stamp = "%s.do_rm_work" % e.stampPrefix[fn]
>             if os.path.exists(stamp):
>                 dir = "%s.*" % e.stampPrefix[fn]
>                 bb.note("Removing stamps: " + dir)
>                 os.system('rm -f '+ dir) }
> 
> which is intended to fix my specific example above. It won't fix the
> new corner cases we've introduced with machine specific sysroots
> though. So what do we do to fix this?  
> 
> I'm actually tempted to tie together rm_work, sstate and the
> scenequeue. 
> When rm_work wipes out the workdir, it also migrates the stamps into
> _setscene stamps as if the package had been installed by sstate. We'd
> then remove the unpack, fetch, patch and so on stamps just leaving
> the setscene varients. This would mean the above wouldn't be needed
> and most things should "just work".    
> 
> So to be specific, we'd take stamps like:
> 
> xxx-1.4.7-r3.do_compile
> xxx-1.4.7-r3.do_configure
> xxx-1.4.7-r3.do_fetch
> xxx-1.4.7-r3.do_generate_toolchain_file
> xxx-1.4.7-r3.do_install
> xxx-1.4.7-r3.do_package.emenlow
> xxx-1.4.7-r3.do_package_write
> xxx-1.4.7-r3.do_package_write_ipk
> xxx-1.4.7-r3.do_package_write_rpm
> xxx-1.4.7-r3.do_patch
> xxx-1.4.7-r3.do_populate_sysroot.emenlow
> xxx-1.4.7-r3.do_setscene
> xxx-1.4.7-r3.do_unpack
> 
> and end up with:
> 
> xxx-1.4.7-r3.do_package_setscene.emenlow
> xxx-1.4.7-r3.do_package_write_ipk_setscene
> xxx-1.4.7-r3.do_package_write_rpm_setscene
> xxx-1.4.7-r3.do_populate_sysroot_setscene.emenlow
> 
> Does that sound reasonable?
> 
> Then both codepaths use the same underlying code and everything
> should be fine apart from some hardcoded assumptions about stamp
> filenames which we'll probably just have to live with for now. The
> function will need to handle stamps including the taskhash too, e.g.:
> 
> xxx-1.4.7-r3.do_unpack.HASHHERE

Hi Richard, 

I think this way is workable to track down the several rm_work issues.

Could you help to clarify what are "hardcoded assumptions about stamp filenames"?

Also you mentioned we need to handle the HASH stamp, do you mean we need to reserve those HASH stamps and they should not be removed while rm_work?

I just wrote a patch that implement the above senario and now is doing test for it.

Please help to review it to see if I understand your point correctly.

http://git.pokylinux.org/cgit/cgit.cgi/poky-contrib/log/?h=dxu4/distro

There are two patches in the branch, the first one is a fix patch that add "${MACHINE}" tag into stamps of do_populate_sysroot_setscene and do_package_setscene, and the second one is to handle rm_work issue.

Thanks,
Dongxiao

> 
> Cheers,
> 
> Richard