* [PATCH 0/1] state.bbclass: check before open the manifest @ 2013-08-29 13:13 Robert Yang 2013-08-29 13:13 ` [PATCH 1/1] sstate.bbclass: " Robert Yang 0 siblings, 1 reply; 8+ messages in thread From: Robert Yang @ 2013-08-29 13:13 UTC (permalink / raw) To: openembedded-core The following changes since commit 5745e45b18e5099e94b4d5a73bc97dc6d4cdc91f: buildtools-tarball: Add python-pkgutil (2013-08-29 00:23:20 +0100) are available in the git repository at: git://git.pokylinux.org/poky-contrib robert/sstate http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/sstate Robert Yang (1): sstate.bbclass: check before open the manifest meta/classes/sstate.bbclass | 3 +++ 1 file changed, 3 insertions(+) -- 1.7.10.4 ^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH 1/1] sstate.bbclass: check before open the manifest 2013-08-29 13:13 [PATCH 0/1] state.bbclass: check before open the manifest Robert Yang @ 2013-08-29 13:13 ` Robert Yang 2013-08-29 17:00 ` Richard Purdie 0 siblings, 1 reply; 8+ messages in thread From: Robert Yang @ 2013-08-29 13:13 UTC (permalink / raw) To: openembedded-core The python stack trace would be printed if we: $ bitbake make (the make-3.82 will be built) // Edit make.inc $ bitbake make-3.81 [snip] *** 0004: mfile = open(manifest) 0005: entries = mfile.readlines() 0006: mfile.close() 0007: 0008: for entry in entries: Exception: IOError: [Errno 2] No such file or directory: xxx [snip] This because the make-3.81 and make-3.82 are being built at the same time, the manifest may have been removed by make-3.82, but make-3.81 still opens it, so the error happens. Check before open the manifest would fix the problem. [YOCTO #5067] Signed-off-by: Robert Yang <liezhi.yang@windriver.com> --- meta/classes/sstate.bbclass | 3 +++ 1 file changed, 3 insertions(+) diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass index c86f393..6e71086 100644 --- a/meta/classes/sstate.bbclass +++ b/meta/classes/sstate.bbclass @@ -302,6 +302,9 @@ def sstate_clean_cachefiles(d): def sstate_clean_manifest(manifest, d): import oe.path + if not os.path.exists(manifest): + return True + mfile = open(manifest) entries = mfile.readlines() mfile.close() -- 1.7.10.4 ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH 1/1] sstate.bbclass: check before open the manifest 2013-08-29 13:13 ` [PATCH 1/1] sstate.bbclass: " Robert Yang @ 2013-08-29 17:00 ` Richard Purdie 2013-08-30 1:24 ` Robert Yang 0 siblings, 1 reply; 8+ messages in thread From: Richard Purdie @ 2013-08-29 17:00 UTC (permalink / raw) To: Robert Yang; +Cc: openembedded-core On Thu, 2013-08-29 at 09:13 -0400, Robert Yang wrote: > The python stack trace would be printed if we: > > $ bitbake make (the make-3.82 will be built) > // Edit make.inc > $ bitbake make-3.81 This is at best hiding the real problem. Why is bitbake building make-3.82 when you asked for make-3.81? I tried this here and also see the warning: ERROR: Multiple .bb files are due to be built which each provide make (/media/build1/poky/meta/recipes-devtools/make/make_3.81.bb /media/build1/poky/meta/recipes-devtools/make/make_3.82.bb). This usually means one provides something the other doesn't and should. so bitbake is basically telling you there is a problem already. So there are two issues: a) Why is make 3.82 being built? b) If multiple identical PNs are being built we probably should hard error out since its not supported in the slightest. The sstate race you mention is the least of the problems :(. Cheers, Richard ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 1/1] sstate.bbclass: check before open the manifest 2013-08-29 17:00 ` Richard Purdie @ 2013-08-30 1:24 ` Robert Yang 2013-08-30 15:29 ` Richard Purdie 0 siblings, 1 reply; 8+ messages in thread From: Robert Yang @ 2013-08-30 1:24 UTC (permalink / raw) To: Richard Purdie; +Cc: openembedded-core On 08/30/2013 01:00 AM, Richard Purdie wrote: > On Thu, 2013-08-29 at 09:13 -0400, Robert Yang wrote: >> The python stack trace would be printed if we: >> >> $ bitbake make (the make-3.82 will be built) >> // Edit make.inc >> $ bitbake make-3.81 > > This is at best hiding the real problem. Why is bitbake building > make-3.82 when you asked for make-3.81? > > I tried this here and also see the warning: > > ERROR: Multiple .bb files are due to be built which each provide make > (/media/build1/poky/meta/recipes-devtools/make/make_3.81.bb /media/build1/poky/meta/recipes-devtools/make/make_3.82.bb). > This usually means one provides something the other doesn't and should. > > so bitbake is basically telling you there is a problem already. So there > are two issues: > What I thought was that let the user know the normal error, but we can do more to fix it as you pointed out. > a) Why is make 3.82 being built? Ah, yes, that's problem, I will do more investigation. > b) If multiple identical PNs are being built we probably should hard > error out since its not supported in the slightest. The sstate race you > mention is the least of the problems :(. > OK, I will try to fix such a case: bitbake make-3.81 make-3.82 // Robert > Cheers, > > Richard > > > ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 1/1] sstate.bbclass: check before open the manifest 2013-08-30 1:24 ` Robert Yang @ 2013-08-30 15:29 ` Richard Purdie 2013-08-31 10:51 ` Robert Yang 0 siblings, 1 reply; 8+ messages in thread From: Richard Purdie @ 2013-08-30 15:29 UTC (permalink / raw) To: Robert Yang; +Cc: openembedded-core On Fri, 2013-08-30 at 09:24 +0800, Robert Yang wrote: > > On 08/30/2013 01:00 AM, Richard Purdie wrote: > > On Thu, 2013-08-29 at 09:13 -0400, Robert Yang wrote: > >> The python stack trace would be printed if we: > >> > >> $ bitbake make (the make-3.82 will be built) > >> // Edit make.inc > >> $ bitbake make-3.81 > > > > This is at best hiding the real problem. Why is bitbake building > > make-3.82 when you asked for make-3.81? > > > > I tried this here and also see the warning: > > > > ERROR: Multiple .bb files are due to be built which each provide make > > (/media/build1/poky/meta/recipes-devtools/make/make_3.81.bb /media/build1/poky/meta/recipes-devtools/make/make_3.82.bb). > > This usually means one provides something the other doesn't and should. > > > > so bitbake is basically telling you there is a problem already. So there > > are two issues: > > > > What I thought was that let the user know the normal error, but we can do > more to fix it as you pointed out. > > > a) Why is make 3.82 being built? > > Ah, yes, that's problem, I will do more investigation. > > > b) If multiple identical PNs are being built we probably should hard > > error out since its not supported in the slightest. The sstate race you > > mention is the least of the problems :(. > > > > OK, I will try to fix such a case: > > bitbake make-3.81 make-3.82 I think this should just give an error. Cheers, Richard ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 1/1] sstate.bbclass: check before open the manifest 2013-08-30 15:29 ` Richard Purdie @ 2013-08-31 10:51 ` Robert Yang 2013-08-31 22:07 ` Otavio Salvador 0 siblings, 1 reply; 8+ messages in thread From: Robert Yang @ 2013-08-31 10:51 UTC (permalink / raw) To: Richard Purdie; +Cc: openembedded-core Hi RP, Thanks, I've made a draft patch for fixing why make-3.82 is being built, the problem is that when we run "bitbake make-3.81": - For 'make' itself, it will build make_3.81.bb - But for make-dev/dbg, it will build make_3.82.bb since there is no PREFERRED_VERSION and it will use the highest version. This draft patch sets the PREFERRED_VERSION for the pkg and will fix the problem, I will send it to bitbake-devel later. diff --git a/bitbake/lib/bb/taskdata.py b/bitbake/lib/bb/taskdata.py index 58fe199..fc8ee6d 100644 --- a/bitbake/lib/bb/taskdata.py +++ b/bitbake/lib/bb/taskdata.py @@ -429,6 +429,22 @@ class TaskData: return all_p = dataCache.providers[item] + logger.warn("pppreferred: %s" % dataCache.preferred) + + # Check whether item has the one of following formats and set the + # PREFERRED_VERSION if it does: + # - <pn>-<pv> + # - <pn>-<pv>-<pr> (when no PE) + # - <pn>_<pe>-<pv>-<pr> (when PE) + for fn in all_p: + pn = dataCache.pkg_fn[fn] + pe = dataCache.pkg_pepvpr[fn][0] + pv = dataCache.pkg_pepvpr[fn][1] + pr = dataCache.pkg_pepvpr[fn][2] + if not pe and (item == "%s-%s" % (pn, pv) or item == "%s-%s-%s" % (pn, pv, pr)): + cfgData.setVar("PREFERRED_VERSION_" + pn, pv) + elif pe and item == "%s_%s-%s-%s" % (pn, pe, pv, pr): + cfgData.setVar("PREFERRED_VERSION_" + pn, pv) eligible, foundUnique = bb.providers.filterProviders(all_p, item, cfgData, dataCache) eligible = [p for p in eligible if not self.getfn_id(p) in self.failed_fnids] // Robert On 08/30/2013 11:29 PM, Richard Purdie wrote: > On Fri, 2013-08-30 at 09:24 +0800, Robert Yang wrote: >> >> On 08/30/2013 01:00 AM, Richard Purdie wrote: >>> On Thu, 2013-08-29 at 09:13 -0400, Robert Yang wrote: >>>> The python stack trace would be printed if we: >>>> >>>> $ bitbake make (the make-3.82 will be built) >>>> // Edit make.inc >>>> $ bitbake make-3.81 >>> >>> This is at best hiding the real problem. Why is bitbake building >>> make-3.82 when you asked for make-3.81? >>> >>> I tried this here and also see the warning: >>> >>> ERROR: Multiple .bb files are due to be built which each provide make >>> (/media/build1/poky/meta/recipes-devtools/make/make_3.81.bb /media/build1/poky/meta/recipes-devtools/make/make_3.82.bb). >>> This usually means one provides something the other doesn't and should. >>> >>> so bitbake is basically telling you there is a problem already. So there >>> are two issues: >>> >> >> What I thought was that let the user know the normal error, but we can do >> more to fix it as you pointed out. >> >>> a) Why is make 3.82 being built? >> >> Ah, yes, that's problem, I will do more investigation. >> >>> b) If multiple identical PNs are being built we probably should hard >>> error out since its not supported in the slightest. The sstate race you >>> mention is the least of the problems :(. >>> >> >> OK, I will try to fix such a case: >> >> bitbake make-3.81 make-3.82 > > I think this should just give an error. > > Cheers, > > Richard > > > ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH 1/1] sstate.bbclass: check before open the manifest 2013-08-31 10:51 ` Robert Yang @ 2013-08-31 22:07 ` Otavio Salvador 2013-09-02 1:59 ` Robert Yang 0 siblings, 1 reply; 8+ messages in thread From: Otavio Salvador @ 2013-08-31 22:07 UTC (permalink / raw) To: Robert Yang; +Cc: Patches and discussions about the oe-core layer On Sat, Aug 31, 2013 at 7:51 AM, Robert Yang <liezhi.yang@windriver.com> wrote: > > Hi RP, > > Thanks, I've made a draft patch for fixing why make-3.82 is being built, > the problem is that when we run "bitbake make-3.81": > - For 'make' itself, it will build make_3.81.bb > - But for make-dev/dbg, it will build make_3.82.bb since there is no > PREFERRED_VERSION and it will use the highest version. > > This draft patch sets the PREFERRED_VERSION for the pkg and will fix the > problem, I will send it to bitbake-devel later. Nice discovery :-) This does seem to address a core issue. > diff --git a/bitbake/lib/bb/taskdata.py b/bitbake/lib/bb/taskdata.py > index 58fe199..fc8ee6d 100644 > --- a/bitbake/lib/bb/taskdata.py > +++ b/bitbake/lib/bb/taskdata.py > @@ -429,6 +429,22 @@ class TaskData: > return > > all_p = dataCache.providers[item] > + logger.warn("pppreferred: %s" % dataCache.preferred) > + > + # Check whether item has the one of following formats and set the > + # PREFERRED_VERSION if it does: > + # - <pn>-<pv> > + # - <pn>-<pv>-<pr> (when no PE) > + # - <pn>_<pe>-<pv>-<pr> (when PE) > + for fn in all_p: > + pn = dataCache.pkg_fn[fn] > + pe = dataCache.pkg_pepvpr[fn][0] > + pv = dataCache.pkg_pepvpr[fn][1] > + pr = dataCache.pkg_pepvpr[fn][2] > + if not pe and (item == "%s-%s" % (pn, pv) or item == "%s-%s-%s" > % (pn, pv, pr)): > + cfgData.setVar("PREFERRED_VERSION_" + pn, pv) > + elif pe and item == "%s_%s-%s-%s" % (pn, pe, pv, pr): > + cfgData.setVar("PREFERRED_VERSION_" + pn, pv) I understand the problem here but maybe this should be checked in bitbake and avoid iterate in all packages? -- Otavio Salvador O.S. Systems http://www.ossystems.com.br http://code.ossystems.com.br Mobile: +55 (53) 9981-7854 Mobile: +1 (347) 903-9750 ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 1/1] sstate.bbclass: check before open the manifest 2013-08-31 22:07 ` Otavio Salvador @ 2013-09-02 1:59 ` Robert Yang 0 siblings, 0 replies; 8+ messages in thread From: Robert Yang @ 2013-09-02 1:59 UTC (permalink / raw) To: Otavio Salvador; +Cc: Patches and discussions about the oe-core layer On 09/01/2013 06:07 AM, Otavio Salvador wrote: > On Sat, Aug 31, 2013 at 7:51 AM, Robert Yang <liezhi.yang@windriver.com> wrote: >> >> Hi RP, >> >> Thanks, I've made a draft patch for fixing why make-3.82 is being built, >> the problem is that when we run "bitbake make-3.81": >> - For 'make' itself, it will build make_3.81.bb >> - But for make-dev/dbg, it will build make_3.82.bb since there is no >> PREFERRED_VERSION and it will use the highest version. >> >> This draft patch sets the PREFERRED_VERSION for the pkg and will fix the >> problem, I will send it to bitbake-devel later. > > Nice discovery :-) This does seem to address a core issue. > >> diff --git a/bitbake/lib/bb/taskdata.py b/bitbake/lib/bb/taskdata.py >> index 58fe199..fc8ee6d 100644 >> --- a/bitbake/lib/bb/taskdata.py >> +++ b/bitbake/lib/bb/taskdata.py >> @@ -429,6 +429,22 @@ class TaskData: >> return >> >> all_p = dataCache.providers[item] >> + logger.warn("pppreferred: %s" % dataCache.preferred) >> + >> + # Check whether item has the one of following formats and set the >> + # PREFERRED_VERSION if it does: >> + # - <pn>-<pv> >> + # - <pn>-<pv>-<pr> (when no PE) >> + # - <pn>_<pe>-<pv>-<pr> (when PE) >> + for fn in all_p: >> + pn = dataCache.pkg_fn[fn] >> + pe = dataCache.pkg_pepvpr[fn][0] >> + pv = dataCache.pkg_pepvpr[fn][1] >> + pr = dataCache.pkg_pepvpr[fn][2] >> + if not pe and (item == "%s-%s" % (pn, pv) or item == "%s-%s-%s" >> % (pn, pv, pr)): >> + cfgData.setVar("PREFERRED_VERSION_" + pn, pv) >> + elif pe and item == "%s_%s-%s-%s" % (pn, pe, pv, pr): >> + cfgData.setVar("PREFERRED_VERSION_" + pn, pv) > > I understand the problem here but maybe this should be checked in > bitbake and avoid iterate in all packages? > Yes, you are right, it should be sent to bitbake-devel mailing list, and I'm trying to avoid iterating in all packages. // Robert ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2013-09-02 2:00 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2013-08-29 13:13 [PATCH 0/1] state.bbclass: check before open the manifest Robert Yang 2013-08-29 13:13 ` [PATCH 1/1] sstate.bbclass: " Robert Yang 2013-08-29 17:00 ` Richard Purdie 2013-08-30 1:24 ` Robert Yang 2013-08-30 15:29 ` Richard Purdie 2013-08-31 10:51 ` Robert Yang 2013-08-31 22:07 ` Otavio Salvador 2013-09-02 1:59 ` Robert Yang
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.