Re: Master stability update

["Tian, Kevin" <kevin.tian@intel.com>]

> From: Richard Purdie [mailto:richard.purdie@linuxfoundation.org]
> Sent: Wednesday, January 26, 2011 6:01 PM
> 
> On Wed, 2011-01-26 at 11:52 +0800, Xu, Dongxiao wrote:
> > Tian, Kevin wrote:
> > >> From: Tian, Kevin
> > >> Sent: Wednesday, January 26, 2011 10:52 AM
> > >>
> > >>> From: Xu, Dongxiao
> > >>> Sent: Wednesday, January 26, 2011 10:46 AM
> > >>>
> > >>> Richard Purdie wrote:
> > >>>> We merged the features into master. I just want to highlight the
> > >>>> status of the following issues:
> > >>>>
> > >>>> * pseudo problem causing failure of meta-toolchain and corruption
> > >>>> of certain file permissions - fixed in master
> > >>>>
> > >>>> * the -live image issues some people and the autobuilder were
> > >>>> seeing - fixed in master
> > >>>>
> > >>>> * the perf issue for linux-yocto-stable - fixed in master
> > >>>>
> > >>>> The following are know issues with master:
> > >>>>
> > >>>> * Using rm_work and switching machines to machines of the same
> > >>>> "multimachine" architecture breaks.
> > >>>
> > >>> After sysroot is per machine, we may need to reserve the "image"
> > >>> folder
> > >> even
> > >>> when rm_work, since for the second machine build,
> > >>> do_populate_sysroot and do_package will be re-run to populate files
> > >>> into a different sysroot folder.
> > >>>
> > >>>>
> > >>>> * When switching machines of the same "multimachine" architecture
> > >>>> (e.g. emenlow to atom-pc), some sstate packages are changing
> > >>>> checksums when at first glance they shouldn't (e.g. perl
> > >>>> do_install). A quick scan of my sstate directory:
> > >>>
> > >>> I also have a look at my local build sstate directories, there are
> > >>> 17 packages whose prebuilt result could not shared betweeen atom-pc
> > >>> and emenlow. Some
> > >>> of them may be not a problem since there are ${MACHINE} variables
> > >>> used in do_install(), some others are strange why do_install will
> > >>> have two different signatures between two different machines.
> > >>>
> > >>> I will spend some time investigating it.
> > >>>
> > >>
> > >> Note that checksum changes may not come from do_install itself, which
> > >> including the hash from other tasks that do_install depends on. To
> > >> capture the latter possibility, you can simply use "bitbake -S
> > >> poky-image-minimal" which only generates .siginfo for all the tasks
> > >> w/o actually running them. This way you can compare the difference
> > >> between emenlow and atom-pc easily to see what actually change.
> > >>
> > >
> > > After a quick check, I guess this is not a problem. for example
> > > perl.do_configure changes checksum due to $MACHINE in dependency
> > > variables as Dongxiao said.
> > > Once perl is not reusable, at least ~10 recieps (libzypp, libtool,
> > > rpm, ...) are not reusable too because perl is in their task
> > > dependency chain.
> > >
> > > I didn't check all the differences yet, but overall feeling is that
> > > this should be OK as Dongxiao mentioned there're several obvious
> > > MACHINE related recipes. I'll leave to Dongxiao for final
> > > confirmation. :-)
> > >
> > > Thanks
> > > Kevin
> >
> > Thanks Kevin for the suggestion which can quickly track down the issue.
> >
> > Within my poky-image-sdk and meta-toolchain-sdk build, there are totally 17
> recipes in core2-poky-linux folder whose prebuilt result could not be shared
> between two machine of one architecture.
> >
> > They are: connman, kexec-tools, libzypp, sat-solver, rpm, perl,
> matchbox-keyboard, matchbox-panel-2, netbase, pulseaudio, screenshot,
> task-poky-sdk, tslib, xf86-input-evdev, xf86-input-keyboard, xf86-input-mouse,
> zypper.
> >
> > I used bitbake-diffsig tool to dump the sigdata, and results are:
> >
> > Connman: its do_configure depends on hal's do_populate_sysroot, while hal
> is a recipe of machine specific.
> > Pulseaudio: its do_configure depends on hal's do_populate_sysroot, while hal
> is a recipe of machine specific.
> >
> > Kexec-tools: its do_configure depends on linux-yocto-stable's
> do_populate_sysroot, which is a machine specific recipe.
> >
> > Perl: ${MACHINE} in do_configure.
> > Rpm: its do_configure depends on perl's do_populate_sysroot.
> > Sat-solver: its do_configure depends on rpm's do_populate_sysroot.
> > Libzypp: its do_configure depends on sat-solver's do_populate_sysroot.
> > Zypper: its do_configure depends on libzypp's do_populate_sysroot
> >
> > Matchbox-panel-2: its do_configure contains "${MACHINE_FEATURES}"
> > Matchbox-keyboard: its do_configure depends on matchbox-panel-2's
> do_populate_sysroot.
> > Screenshot: its do_configure depends on matchbox-panel-2's
> do_populate_sysroot
> > Task-poky-sdk: depends on matchbox-panel-2
> >
> > Netbase: ${MACHINE} in do_install.
> > Tslib: ${MACHINE} in do_install.
> >
> > Xf86-input-evdev, xf86-input-keyboard, and xf86-input-mouse: xorg-xserver
> dependency differences, which emenlow has its own xorg-xserver layer.
> >
> 
> So for each of these we need to decide if rebuilding is a valid or
> invalid thing to do.
> 
> The one that jumps out at me is perl. Why does it need MACHINE in
> do_configure and is it really MACHINE dependent? If you look at the
> do_configure code, its a rather strange check against the "native"
> machine which we don't use/support. I'd argue that machine is error
> prone, problematic and replaced by our nativesdk code so I'm in favour
> of removing that check.
> 
> Some of the other items above are trickier. In some cases we should
> perhaps be marking them as machine specific, e.g. mb-panel-2 ? and maybe
> also whitelisting some dependencies if we're sure those items don't
> change if the dependency changes.
> 
> The nice thing is now that we can make something like mb-panel-2 machine
> specific and not have the whole world collapse as it would have done
> previously so we can start to benefit from this change! :)
> 

Yes, this at least proves that overall now we're in a good state with recent changes. :-)

Your suggestions are good, and we'll continue look into them to reduce the machine
specific bits as possible.

Thanks
Kevin