* [PATCH 0/1 V3] sstate.bbclass: remove previous version's stamp
@ 2014-01-20 11:42 Robert Yang
2014-01-20 11:42 ` [PATCH 1/1] " Robert Yang
0 siblings, 1 reply; 9+ messages in thread
From: Robert Yang @ 2014-01-20 11:42 UTC (permalink / raw)
To: openembedded-core
* V3: (Fix RP's comments)
- Avoid matching multiple tasks
- Fix the removing for BB_SIGNATURE_HANDLER = "noop".
* V2: (Fix RP's comments)
- Use STAMPCLEAN instead of STAMP
- Respect to the extrainf
// Robert
The following changes since commit b908caca1af1d4150eafd3d3001ea19599657270:
poky-tiny.conf: Add PACKAGECONFIG to disable opkg-utils python dependencies (2014-01-19 17:18:04 +0000)
are available in the git repository at:
git://git.pokylinux.org/poky-contrib rbt/clean_stamp
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=rbt/clean_stamp
Robert Yang (1):
sstate.bbclass: remove previous version's stamp
meta/classes/sstate.bbclass | 31 ++++++++++++++++++++++---------
1 file changed, 22 insertions(+), 9 deletions(-)
--
1.8.3.1
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH 1/1] sstate.bbclass: remove previous version's stamp
2014-01-20 11:42 [PATCH 0/1 V3] sstate.bbclass: remove previous version's stamp Robert Yang
@ 2014-01-20 11:42 ` Robert Yang
0 siblings, 0 replies; 9+ messages in thread
From: Robert Yang @ 2014-01-20 11:42 UTC (permalink / raw)
To: openembedded-core
There is a potential problem if we don't remove the previous version's
stamp, for example:
The depend chain is:
libtool-native -> autoconf-native -> m4-native
We have two m4-native: 1.4.9 and 1.4.7
1) Clean all of them to make a fresh build so that we can reproduce the
problem
$ bitbake m4-native autoconf-native libtool-native -ccleansstate
2) Build libtool-native so that the m4-native_1.4.17 will be built
$ bitbake libtool-native
3) Set PREFERRED_VERSION_m4-native = "1.4.9" and build again
$ bitbake libtool-native
4) Set PREFERRED_VERSION_m4-native = "1.4.17" and build again
$ bitbake libtool-native -ccleansstate && bitbake libtool-native
Then the build will fail:
[snip]
| m4: unrecognized option '--gnu'
| Try `m4 --help' for more information.
| autom4te: m4 failed with exit status: 1
[snip]
The is because when we change m4-native to 1.4.17 and build
libtool-native again:
5) libtool-native depends on autoconf-native, and autoconf-native's
version isn't change, so it can remove the current stamp and mirror
the sstate (the one depends on m4-native_1.4.9) from the SSTATE_DIR
correctly.
6) The mirrored autoconf-native depends on m4-native_1.4.17's
do_populate_sysroot, and the stamp is already there (which is made
by step 2), so it would do nothing, but this is incorrect, since
the one that really in the sysroot is m4-native_1.4.9, then the
error happens.
Remove previous version's stamp in sstate_clean() will fix the problem.
[YOCTO #5422]
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
meta/classes/sstate.bbclass | 31 ++++++++++++++++++++++---------
1 file changed, 22 insertions(+), 9 deletions(-)
diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
index 35c3f85..23d7de6 100644
--- a/meta/classes/sstate.bbclass
+++ b/meta/classes/sstate.bbclass
@@ -331,11 +331,17 @@ def sstate_clean_manifest(manifest, d):
def sstate_clean(ss, d):
import oe.path
+ import glob
d2 = d.createCopy()
+ stamp_clean = d.getVar("STAMPCLEAN", True)
extrainf = d.getVarFlag("do_" + ss['task'], 'stamp-extra-info', True)
if extrainf:
d2.setVar("SSTATE_MANMACH", extrainf)
+ wildcard_stfile = "%s.do_%s*.%s" % (stamp_clean, ss['task'], extrainf)
+ else:
+ wildcard_stfile = "%s.do_%s*" % (stamp_clean, ss['task'])
+
manifest = d2.expand("${SSTATE_MANFILEPREFIX}.%s" % ss['name'])
if os.path.exists(manifest):
@@ -350,15 +356,22 @@ def sstate_clean(ss, d):
for lock in locks:
bb.utils.unlockfile(lock)
- stfile = d.getVar("STAMP", True) + ".do_" + ss['task']
- oe.path.remove(stfile)
- oe.path.remove(stfile + "_setscene")
- if extrainf:
- oe.path.remove(stfile + ".*" + extrainf)
- oe.path.remove(stfile + "_setscene" + ".*" + extrainf)
- else:
- oe.path.remove(stfile + ".*")
- oe.path.remove(stfile + "_setscene" + ".*")
+ # Remove the current and previous stamps, but keep the sigdata.
+ #
+ # The glob() matches do_task* which may match multiple tasks, for
+ # example: do_package and do_package_write_ipk, so we need to
+ # exactly match *.do_task.* and *.do_task_setscene.*
+ rm_stamp = '.do_%s.' % ss['task']
+ rm_setscene = '.do_%s_setscene.' % ss['task']
+ # For BB_SIGNATURE_HANDLER = "noop"
+ rm_nohash = ".do_%s" % ss['task']
+ for stfile in glob.glob(wildcard_stfile):
+ # Keep the sigdata
+ if ".sigdata." in stfile:
+ continue
+ if rm_stamp in stfile or rm_setscene in stfile or \
+ stfile.endswith(rm_nohash):
+ oe.path.remove(stfile)
CLEANFUNCS += "sstate_cleanall"
--
1.8.3.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH 1/1] sstate.bbclass: remove previous version's stamp
2014-01-17 12:17 ` Richard Purdie
@ 2014-01-20 9:28 ` Robert Yang
0 siblings, 0 replies; 9+ messages in thread
From: Robert Yang @ 2014-01-20 9:28 UTC (permalink / raw)
To: Richard Purdie; +Cc: openembedded-core
On 01/17/2014 08:17 PM, Richard Purdie wrote:
> On Fri, 2014-01-17 at 19:01 +0800, Robert Yang wrote:
>>
>> On 01/17/2014 06:36 PM, Richard Purdie wrote:
>>> On Fri, 2014-01-17 at 14:43 +0800, Robert Yang wrote:
>>>> + # Keep the sigdata
>>>> + if not re.match(re_sigdata, stfile):
>>>> + oe.path.remove(stfile)
>>>
>>> I'm not sure its worth the overhead of using regexps here. Can we not do
>>> something simple like:
>>>
>>> if ".sigdata." not in stfile:
>>>
>>
>> Sounds better, I've updated the PULL:
>>
>> git://git.pokylinux.org/poky-contrib rbt/clean_stamp
>>
>> and here is the patch:
>>
>> diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
>> index 35c3f85..d4e0dcb 100644
>> --- a/meta/classes/sstate.bbclass
>> +++ b/meta/classes/sstate.bbclass
>> @@ -331,11 +331,17 @@ def sstate_clean_manifest(manifest, d):
>>
>> def sstate_clean(ss, d):
>> import oe.path
>> + import glob
>>
>> d2 = d.createCopy()
>> + stamp_clean = d.getVar("STAMPCLEAN", True)
>> extrainf = d.getVarFlag("do_" + ss['task'], 'stamp-extra-info', True)
>> if extrainf:
>> d2.setVar("SSTATE_MANMACH", extrainf)
>> + wildcard_stfile = "%s.do_%s*.%s" % (stamp_clean, ss['task'], extrainf)
>> + else:
>> + wildcard_stfile = "%s.do_%s*" % (stamp_clean, ss['task'])
>> +
>> manifest = d2.expand("${SSTATE_MANFILEPREFIX}.%s" % ss['name'])
>>
>> if os.path.exists(manifest):
>> @@ -350,15 +356,11 @@ def sstate_clean(ss, d):
>> for lock in locks:
>> bb.utils.unlockfile(lock)
>>
>> - stfile = d.getVar("STAMP", True) + ".do_" + ss['task']
>> - oe.path.remove(stfile)
>> - oe.path.remove(stfile + "_setscene")
>> - if extrainf:
>> - oe.path.remove(stfile + ".*" + extrainf)
>> - oe.path.remove(stfile + "_setscene" + ".*" + extrainf)
>> - else:
>> - oe.path.remove(stfile + ".*")
>> - oe.path.remove(stfile + "_setscene" + ".*")
>> + # Remove the current and previous stamps, but keep the sigdata
>> + for stfile in glob.glob(wildcard_stfile):
>> + # Keep the sigdata
>> + if ".sigdata." not in stfile:
>> + oe.path.remove(stfile)
>
> Sorry Robert, there is one more issue here. You're doing "do_%s*" but
> this can match multiple tasks, e.g. do_package* would also wipe out
> do_package_write_ipk, do_package_write_rpm etc.
>
Ah, it's my fault, the V3 is incoming.
// Robert
> The easiest way to handle this is probably in the for loop, making sure
> that do_<task>. or do_<task>_setscene. are in the filename.
>
> It might be worth putting some comments in the code about these
> subtleties.
>
> Cheers,
>
> Richard
>
>
>
>
>
>
>
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH 1/1] sstate.bbclass: remove previous version's stamp
2014-01-17 11:01 ` Robert Yang
@ 2014-01-17 12:17 ` Richard Purdie
2014-01-20 9:28 ` Robert Yang
0 siblings, 1 reply; 9+ messages in thread
From: Richard Purdie @ 2014-01-17 12:17 UTC (permalink / raw)
To: Robert Yang; +Cc: openembedded-core
On Fri, 2014-01-17 at 19:01 +0800, Robert Yang wrote:
>
> On 01/17/2014 06:36 PM, Richard Purdie wrote:
> > On Fri, 2014-01-17 at 14:43 +0800, Robert Yang wrote:
> >> + # Keep the sigdata
> >> + if not re.match(re_sigdata, stfile):
> >> + oe.path.remove(stfile)
> >
> > I'm not sure its worth the overhead of using regexps here. Can we not do
> > something simple like:
> >
> > if ".sigdata." not in stfile:
> >
>
> Sounds better, I've updated the PULL:
>
> git://git.pokylinux.org/poky-contrib rbt/clean_stamp
>
> and here is the patch:
>
> diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
> index 35c3f85..d4e0dcb 100644
> --- a/meta/classes/sstate.bbclass
> +++ b/meta/classes/sstate.bbclass
> @@ -331,11 +331,17 @@ def sstate_clean_manifest(manifest, d):
>
> def sstate_clean(ss, d):
> import oe.path
> + import glob
>
> d2 = d.createCopy()
> + stamp_clean = d.getVar("STAMPCLEAN", True)
> extrainf = d.getVarFlag("do_" + ss['task'], 'stamp-extra-info', True)
> if extrainf:
> d2.setVar("SSTATE_MANMACH", extrainf)
> + wildcard_stfile = "%s.do_%s*.%s" % (stamp_clean, ss['task'], extrainf)
> + else:
> + wildcard_stfile = "%s.do_%s*" % (stamp_clean, ss['task'])
> +
> manifest = d2.expand("${SSTATE_MANFILEPREFIX}.%s" % ss['name'])
>
> if os.path.exists(manifest):
> @@ -350,15 +356,11 @@ def sstate_clean(ss, d):
> for lock in locks:
> bb.utils.unlockfile(lock)
>
> - stfile = d.getVar("STAMP", True) + ".do_" + ss['task']
> - oe.path.remove(stfile)
> - oe.path.remove(stfile + "_setscene")
> - if extrainf:
> - oe.path.remove(stfile + ".*" + extrainf)
> - oe.path.remove(stfile + "_setscene" + ".*" + extrainf)
> - else:
> - oe.path.remove(stfile + ".*")
> - oe.path.remove(stfile + "_setscene" + ".*")
> + # Remove the current and previous stamps, but keep the sigdata
> + for stfile in glob.glob(wildcard_stfile):
> + # Keep the sigdata
> + if ".sigdata." not in stfile:
> + oe.path.remove(stfile)
Sorry Robert, there is one more issue here. You're doing "do_%s*" but
this can match multiple tasks, e.g. do_package* would also wipe out
do_package_write_ipk, do_package_write_rpm etc.
The easiest way to handle this is probably in the for loop, making sure
that do_<task>. or do_<task>_setscene. are in the filename.
It might be worth putting some comments in the code about these
subtleties.
Cheers,
Richard
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH 1/1] sstate.bbclass: remove previous version's stamp
2014-01-17 10:36 ` Richard Purdie
@ 2014-01-17 11:01 ` Robert Yang
2014-01-17 12:17 ` Richard Purdie
0 siblings, 1 reply; 9+ messages in thread
From: Robert Yang @ 2014-01-17 11:01 UTC (permalink / raw)
To: Richard Purdie; +Cc: openembedded-core
On 01/17/2014 06:36 PM, Richard Purdie wrote:
> On Fri, 2014-01-17 at 14:43 +0800, Robert Yang wrote:
>> + # Keep the sigdata
>> + if not re.match(re_sigdata, stfile):
>> + oe.path.remove(stfile)
>
> I'm not sure its worth the overhead of using regexps here. Can we not do
> something simple like:
>
> if ".sigdata." not in stfile:
>
Sounds better, I've updated the PULL:
git://git.pokylinux.org/poky-contrib rbt/clean_stamp
and here is the patch:
diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
index 35c3f85..d4e0dcb 100644
--- a/meta/classes/sstate.bbclass
+++ b/meta/classes/sstate.bbclass
@@ -331,11 +331,17 @@ def sstate_clean_manifest(manifest, d):
def sstate_clean(ss, d):
import oe.path
+ import glob
d2 = d.createCopy()
+ stamp_clean = d.getVar("STAMPCLEAN", True)
extrainf = d.getVarFlag("do_" + ss['task'], 'stamp-extra-info', True)
if extrainf:
d2.setVar("SSTATE_MANMACH", extrainf)
+ wildcard_stfile = "%s.do_%s*.%s" % (stamp_clean, ss['task'], extrainf)
+ else:
+ wildcard_stfile = "%s.do_%s*" % (stamp_clean, ss['task'])
+
manifest = d2.expand("${SSTATE_MANFILEPREFIX}.%s" % ss['name'])
if os.path.exists(manifest):
@@ -350,15 +356,11 @@ def sstate_clean(ss, d):
for lock in locks:
bb.utils.unlockfile(lock)
- stfile = d.getVar("STAMP", True) + ".do_" + ss['task']
- oe.path.remove(stfile)
- oe.path.remove(stfile + "_setscene")
- if extrainf:
- oe.path.remove(stfile + ".*" + extrainf)
- oe.path.remove(stfile + "_setscene" + ".*" + extrainf)
- else:
- oe.path.remove(stfile + ".*")
- oe.path.remove(stfile + "_setscene" + ".*")
+ # Remove the current and previous stamps, but keep the sigdata
+ for stfile in glob.glob(wildcard_stfile):
+ # Keep the sigdata
+ if ".sigdata." not in stfile:
+ oe.path.remove(stfile)
CLEANFUNCS += "sstate_cleanall"
// Robert
> Cheers,
>
> Richard
>
>
>
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH 1/1] sstate.bbclass: remove previous version's stamp
2014-01-17 6:43 ` [PATCH 1/1] " Robert Yang
@ 2014-01-17 10:36 ` Richard Purdie
2014-01-17 11:01 ` Robert Yang
0 siblings, 1 reply; 9+ messages in thread
From: Richard Purdie @ 2014-01-17 10:36 UTC (permalink / raw)
To: Robert Yang; +Cc: openembedded-core
On Fri, 2014-01-17 at 14:43 +0800, Robert Yang wrote:
> There is a potential problem if we don't remove the previous version's
> stamp, for example:
>
> The depend chain is:
> libtool-native -> autoconf-native -> m4-native
> We have two m4-native: 1.4.9 and 1.4.7
>
> 1) Clean all of them to make a fresh build so that we can reproduce the
> problem
> $ bitbake m4-native autoconf-native libtool-native -ccleansstate
>
> 2) Build libtool-native so that the m4-native_1.4.17 will be built
> $ bitbake libtool-native
>
> 3) Set PREFERRED_VERSION_m4-native = "1.4.9" and build again
> $ bitbake libtool-native
>
> 4) Set PREFERRED_VERSION_m4-native = "1.4.17" and build again
> $ bitbake libtool-native -ccleansstate && bitbake libtool-native
>
> Then the build will fail:
> [snip]
> | m4: unrecognized option '--gnu'
> | Try `m4 --help' for more information.
> | autom4te: m4 failed with exit status: 1
> [snip]
>
> The is because when we change m4-native to 1.4.17 and build
> libtool-native again:
> 5) libtool-native depends on autoconf-native, and autoconf-native's
> version isn't change, so it can remove the current stamp and mirror
> the sstate (the one depends on m4-native_1.4.9) from the SSTATE_DIR
> correctly.
>
> 6) The mirrored autoconf-native depends on m4-native_1.4.17's
> do_populate_sysroot, and the stamp is already there (which is made
> by step 2), so it would do nothing, but this is incorrect, since
> the one that really in the sysroot is m4-native_1.4.9, then the
> error happens.
>
> Remove previous version's stamp in sstate_clean() will fix the problem.
>
> [YOCTO #5422]
>
> Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
> ---
> meta/classes/sstate.bbclass | 22 +++++++++++++---------
> 1 file changed, 13 insertions(+), 9 deletions(-)
>
> diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
> index 35c3f85..bad8b78 100644
> --- a/meta/classes/sstate.bbclass
> +++ b/meta/classes/sstate.bbclass
> @@ -331,11 +331,18 @@ def sstate_clean_manifest(manifest, d):
>
> def sstate_clean(ss, d):
> import oe.path
> + import glob
> + import re
>
> d2 = d.createCopy()
> + stamp_clean = d.getVar("STAMPCLEAN", True)
> extrainf = d.getVarFlag("do_" + ss['task'], 'stamp-extra-info', True)
> if extrainf:
> d2.setVar("SSTATE_MANMACH", extrainf)
> + wildcard_stfile = "%s.do_%s*.%s" % (stamp_clean, ss['task'], extrainf)
> + else:
> + wildcard_stfile = "%s.do_%s*" % (stamp_clean, ss['task'])
> +
> manifest = d2.expand("${SSTATE_MANFILEPREFIX}.%s" % ss['name'])
>
> if os.path.exists(manifest):
> @@ -350,15 +357,12 @@ def sstate_clean(ss, d):
> for lock in locks:
> bb.utils.unlockfile(lock)
>
> - stfile = d.getVar("STAMP", True) + ".do_" + ss['task']
> - oe.path.remove(stfile)
> - oe.path.remove(stfile + "_setscene")
> - if extrainf:
> - oe.path.remove(stfile + ".*" + extrainf)
> - oe.path.remove(stfile + "_setscene" + ".*" + extrainf)
> - else:
> - oe.path.remove(stfile + ".*")
> - oe.path.remove(stfile + "_setscene" + ".*")
> + # Remove the current and previous stamps, but keep the sigdata
> + re_sigdata = ".*\.do_%s\.sigdata\." % ss['task']
> + for stfile in glob.glob(wildcard_stfile):
> + # Keep the sigdata
> + if not re.match(re_sigdata, stfile):
> + oe.path.remove(stfile)
I'm not sure its worth the overhead of using regexps here. Can we not do
something simple like:
if ".sigdata." not in stfile:
Cheers,
Richard
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH 1/1] sstate.bbclass: remove previous version's stamp
2014-01-17 6:43 [PATCH 0/1 V2] " Robert Yang
@ 2014-01-17 6:43 ` Robert Yang
2014-01-17 10:36 ` Richard Purdie
0 siblings, 1 reply; 9+ messages in thread
From: Robert Yang @ 2014-01-17 6:43 UTC (permalink / raw)
To: openembedded-core
There is a potential problem if we don't remove the previous version's
stamp, for example:
The depend chain is:
libtool-native -> autoconf-native -> m4-native
We have two m4-native: 1.4.9 and 1.4.7
1) Clean all of them to make a fresh build so that we can reproduce the
problem
$ bitbake m4-native autoconf-native libtool-native -ccleansstate
2) Build libtool-native so that the m4-native_1.4.17 will be built
$ bitbake libtool-native
3) Set PREFERRED_VERSION_m4-native = "1.4.9" and build again
$ bitbake libtool-native
4) Set PREFERRED_VERSION_m4-native = "1.4.17" and build again
$ bitbake libtool-native -ccleansstate && bitbake libtool-native
Then the build will fail:
[snip]
| m4: unrecognized option '--gnu'
| Try `m4 --help' for more information.
| autom4te: m4 failed with exit status: 1
[snip]
The is because when we change m4-native to 1.4.17 and build
libtool-native again:
5) libtool-native depends on autoconf-native, and autoconf-native's
version isn't change, so it can remove the current stamp and mirror
the sstate (the one depends on m4-native_1.4.9) from the SSTATE_DIR
correctly.
6) The mirrored autoconf-native depends on m4-native_1.4.17's
do_populate_sysroot, and the stamp is already there (which is made
by step 2), so it would do nothing, but this is incorrect, since
the one that really in the sysroot is m4-native_1.4.9, then the
error happens.
Remove previous version's stamp in sstate_clean() will fix the problem.
[YOCTO #5422]
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
meta/classes/sstate.bbclass | 22 +++++++++++++---------
1 file changed, 13 insertions(+), 9 deletions(-)
diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
index 35c3f85..bad8b78 100644
--- a/meta/classes/sstate.bbclass
+++ b/meta/classes/sstate.bbclass
@@ -331,11 +331,18 @@ def sstate_clean_manifest(manifest, d):
def sstate_clean(ss, d):
import oe.path
+ import glob
+ import re
d2 = d.createCopy()
+ stamp_clean = d.getVar("STAMPCLEAN", True)
extrainf = d.getVarFlag("do_" + ss['task'], 'stamp-extra-info', True)
if extrainf:
d2.setVar("SSTATE_MANMACH", extrainf)
+ wildcard_stfile = "%s.do_%s*.%s" % (stamp_clean, ss['task'], extrainf)
+ else:
+ wildcard_stfile = "%s.do_%s*" % (stamp_clean, ss['task'])
+
manifest = d2.expand("${SSTATE_MANFILEPREFIX}.%s" % ss['name'])
if os.path.exists(manifest):
@@ -350,15 +357,12 @@ def sstate_clean(ss, d):
for lock in locks:
bb.utils.unlockfile(lock)
- stfile = d.getVar("STAMP", True) + ".do_" + ss['task']
- oe.path.remove(stfile)
- oe.path.remove(stfile + "_setscene")
- if extrainf:
- oe.path.remove(stfile + ".*" + extrainf)
- oe.path.remove(stfile + "_setscene" + ".*" + extrainf)
- else:
- oe.path.remove(stfile + ".*")
- oe.path.remove(stfile + "_setscene" + ".*")
+ # Remove the current and previous stamps, but keep the sigdata
+ re_sigdata = ".*\.do_%s\.sigdata\." % ss['task']
+ for stfile in glob.glob(wildcard_stfile):
+ # Keep the sigdata
+ if not re.match(re_sigdata, stfile):
+ oe.path.remove(stfile)
CLEANFUNCS += "sstate_cleanall"
--
1.8.3.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH 1/1] sstate.bbclass: remove previous version's stamp
2014-01-15 7:09 ` [PATCH 1/1] " Robert Yang
@ 2014-01-15 9:40 ` Richard Purdie
0 siblings, 0 replies; 9+ messages in thread
From: Richard Purdie @ 2014-01-15 9:40 UTC (permalink / raw)
To: Robert Yang; +Cc: openembedded-core
On Wed, 2014-01-15 at 15:09 +0800, Robert Yang wrote:
> There is a potential problem if we don't remove the previous version's
> stamp, for example:
>
> The depend chain is:
> libtool-native -> autoconf-native -> m4-native
> We have two m4-native: 1.4.9 and 1.4.7
>
> 1) Clean all of them to make a fresh build so that we can reproduce the
> problem
> $ bitbake m4-native autoconf-native libtool-native -ccleansstate
>
> 2) Build libtool-native so that the m4-native_1.4.17 will be built
> $ bitbake libtool-native
>
> 3) Set PREFERRED_VERSION_m4-native = "1.4.9" and build again
> $ bitbake libtool-native
>
> 4) Set PREFERRED_VERSION_m4-native = "1.4.17" and build again
> $ bitbake libtool-native -ccleansstate && bitbake libtool-native
>
> Then the build will fail:
> [snip]
> | m4: unrecognized option '--gnu'
> | Try `m4 --help' for more information.
> | autom4te: m4 failed with exit status: 1
> [snip]
>
> The is because when we change m4-native to 1.4.17 and build
> libtool-native again:
> 5) libtool-native depends on autoconf-native, and autoconf-native's
> version isn't change, so it can remove the current stamp and mirror
> the sstate (the one depends on m4-native_1.4.9) from the SSTATE_DIR
> correctly.
>
> 6) The mirrored autoconf-native depends on m4-native_1.4.17's
> do_populate_sysroot, and the stamp is already there (which is made
> by step 2), so it would do nothing, but this is incorrect, since
> the one that really in the sysroot is m4-native_1.4.9, then the
> error happens.
>
> Remove previous version's stamp in sstate_clean() will fix the problem.
>
> [YOCTO #5422]
>
> Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
> ---
> meta/classes/sstate.bbclass | 19 ++++++++++---------
> 1 file changed, 10 insertions(+), 9 deletions(-)
>
> diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
> index 35c3f85..8802d79 100644
> --- a/meta/classes/sstate.bbclass
> +++ b/meta/classes/sstate.bbclass
> @@ -331,6 +331,8 @@ def sstate_clean_manifest(manifest, d):
>
> def sstate_clean(ss, d):
> import oe.path
> + import glob
> + import re
>
> d2 = d.createCopy()
> extrainf = d.getVarFlag("do_" + ss['task'], 'stamp-extra-info', True)
> @@ -350,15 +352,14 @@ def sstate_clean(ss, d):
> for lock in locks:
> bb.utils.unlockfile(lock)
>
> - stfile = d.getVar("STAMP", True) + ".do_" + ss['task']
> - oe.path.remove(stfile)
> - oe.path.remove(stfile + "_setscene")
> - if extrainf:
> - oe.path.remove(stfile + ".*" + extrainf)
> - oe.path.remove(stfile + "_setscene" + ".*" + extrainf)
> - else:
> - oe.path.remove(stfile + ".*")
> - oe.path.remove(stfile + "_setscene" + ".*")
> + stamp_dir = os.path.dirname(d.getVar("STAMP", True))
> + # Remove the current and previous stamps, but keep the sigdata
> + wildcard_stfile = "%s/*.do_%s*" % (stamp_dir, ss['task'])
> + re_stfile_sigdata = "%s/.*\.do_%s\.sigdata\.*" % (stamp_dir, ss['task'])
> + for stfile in glob.glob(wildcard_stfile):
> + # Keep the sigdata
> + if not re.match(re_stfile_sigdata, stfile):
> + oe.path.remove(stfile)
>
You can't do this, the extrainf data *is* important since for tasks
which are machine specific, you must not trample on another machine's
stamp files.
Please use STAMPCLEAN here instead of STAMP, setup an appropriate glob
or two glob expressions based on that, search for it, then iterate the
result and only delete anything that doesn't have sigdata in it.
Cheers,
Richard
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH 1/1] sstate.bbclass: remove previous version's stamp
2014-01-15 7:09 [PATCH 0/1] " Robert Yang
@ 2014-01-15 7:09 ` Robert Yang
2014-01-15 9:40 ` Richard Purdie
0 siblings, 1 reply; 9+ messages in thread
From: Robert Yang @ 2014-01-15 7:09 UTC (permalink / raw)
To: openembedded-core
There is a potential problem if we don't remove the previous version's
stamp, for example:
The depend chain is:
libtool-native -> autoconf-native -> m4-native
We have two m4-native: 1.4.9 and 1.4.7
1) Clean all of them to make a fresh build so that we can reproduce the
problem
$ bitbake m4-native autoconf-native libtool-native -ccleansstate
2) Build libtool-native so that the m4-native_1.4.17 will be built
$ bitbake libtool-native
3) Set PREFERRED_VERSION_m4-native = "1.4.9" and build again
$ bitbake libtool-native
4) Set PREFERRED_VERSION_m4-native = "1.4.17" and build again
$ bitbake libtool-native -ccleansstate && bitbake libtool-native
Then the build will fail:
[snip]
| m4: unrecognized option '--gnu'
| Try `m4 --help' for more information.
| autom4te: m4 failed with exit status: 1
[snip]
The is because when we change m4-native to 1.4.17 and build
libtool-native again:
5) libtool-native depends on autoconf-native, and autoconf-native's
version isn't change, so it can remove the current stamp and mirror
the sstate (the one depends on m4-native_1.4.9) from the SSTATE_DIR
correctly.
6) The mirrored autoconf-native depends on m4-native_1.4.17's
do_populate_sysroot, and the stamp is already there (which is made
by step 2), so it would do nothing, but this is incorrect, since
the one that really in the sysroot is m4-native_1.4.9, then the
error happens.
Remove previous version's stamp in sstate_clean() will fix the problem.
[YOCTO #5422]
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
meta/classes/sstate.bbclass | 19 ++++++++++---------
1 file changed, 10 insertions(+), 9 deletions(-)
diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
index 35c3f85..8802d79 100644
--- a/meta/classes/sstate.bbclass
+++ b/meta/classes/sstate.bbclass
@@ -331,6 +331,8 @@ def sstate_clean_manifest(manifest, d):
def sstate_clean(ss, d):
import oe.path
+ import glob
+ import re
d2 = d.createCopy()
extrainf = d.getVarFlag("do_" + ss['task'], 'stamp-extra-info', True)
@@ -350,15 +352,14 @@ def sstate_clean(ss, d):
for lock in locks:
bb.utils.unlockfile(lock)
- stfile = d.getVar("STAMP", True) + ".do_" + ss['task']
- oe.path.remove(stfile)
- oe.path.remove(stfile + "_setscene")
- if extrainf:
- oe.path.remove(stfile + ".*" + extrainf)
- oe.path.remove(stfile + "_setscene" + ".*" + extrainf)
- else:
- oe.path.remove(stfile + ".*")
- oe.path.remove(stfile + "_setscene" + ".*")
+ stamp_dir = os.path.dirname(d.getVar("STAMP", True))
+ # Remove the current and previous stamps, but keep the sigdata
+ wildcard_stfile = "%s/*.do_%s*" % (stamp_dir, ss['task'])
+ re_stfile_sigdata = "%s/.*\.do_%s\.sigdata\.*" % (stamp_dir, ss['task'])
+ for stfile in glob.glob(wildcard_stfile):
+ # Keep the sigdata
+ if not re.match(re_stfile_sigdata, stfile):
+ oe.path.remove(stfile)
CLEANFUNCS += "sstate_cleanall"
--
1.8.3.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
end of thread, other threads:[~2014-01-20 11:42 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-01-20 11:42 [PATCH 0/1 V3] sstate.bbclass: remove previous version's stamp Robert Yang
2014-01-20 11:42 ` [PATCH 1/1] " Robert Yang
-- strict thread matches above, loose matches on Subject: below --
2014-01-17 6:43 [PATCH 0/1 V2] " Robert Yang
2014-01-17 6:43 ` [PATCH 1/1] " Robert Yang
2014-01-17 10:36 ` Richard Purdie
2014-01-17 11:01 ` Robert Yang
2014-01-17 12:17 ` Richard Purdie
2014-01-20 9:28 ` Robert Yang
2014-01-15 7:09 [PATCH 0/1] " Robert Yang
2014-01-15 7:09 ` [PATCH 1/1] " Robert Yang
2014-01-15 9:40 ` Richard Purdie
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.