* [PATCH 01/10][auh] upgradehelper.py: Only run with one recipe or all.
2015-07-29 20:50 [PATCH 00/10][auh] Add support for buildhistory and minor fixes Aníbal Limón
@ 2015-07-29 20:50 ` Aníbal Limón
2015-07-31 13:48 ` Paul Eggleton
2015-07-29 20:50 ` [PATCH 02/10][auh] upgradehelper.py: Validate if upgrade is needed Aníbal Limón
` (8 subsequent siblings)
9 siblings, 1 reply; 18+ messages in thread
From: Aníbal Limón @ 2015-07-29 20:50 UTC (permalink / raw)
To: yocto; +Cc: paul.eggleton
Remove ability to run with several recipes because isn't make sense.
Supported modes are,
$ ./upgradehelper.py all
$ ./upgradehelper.py recipe --to_version VERSION
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
upgradehelper.py | 34 ++++++++++------------------------
1 file changed, 10 insertions(+), 24 deletions(-)
diff --git a/upgradehelper.py b/upgradehelper.py
index d6ac7fd..63b30e4 100755
--- a/upgradehelper.py
+++ b/upgradehelper.py
@@ -65,7 +65,7 @@ def parse_cmdline():
parser = argparse.ArgumentParser(description='Package Upgrade Helper',
formatter_class=argparse.RawTextHelpFormatter,
epilog=help_text)
- parser.add_argument("recipe", nargs="+", help="recipe to be upgraded")
+ parser.add_argument("recipe", help="recipe to be upgraded")
parser.add_argument("-t", "--to_version",
help="version to upgrade the recipe to")
@@ -507,14 +507,6 @@ class Updater(object):
self.statistics.update(self.pn, self.new_ver, self.maintainer, error)
- if self.interactive and attempted_pkgs < total_pkgs:
- I(" %s: Proceed to next recipe? (Y/n)" % self.pn)
- answer = sys.stdin.readline().strip().upper()
-
- if answer != 'Y' and answer != '':
- I("Aborted by user!")
- exit(0)
-
if (attempted_pkgs > 1):
I("%s" % self.statistics.pkg_stats())
if self.send_email:
@@ -700,24 +692,18 @@ if __name__ == "__main__":
settings, maintainer_override = parse_config_file(args.config_file)
- if len(args.recipe) == 1 and args.recipe[0] == "all":
+ if args.recipe == "all":
updater = UniverseUpdater()
updater.run()
- elif len(args.recipe) >= 1:
- if len(args.recipe) == 1:
- if not args.to_version:
- E(" For upgrade only one recipe you must specify --to_version\n")
- exit(1)
-
- if not args.maintainer and args.send_emails:
- E(" For upgrade only one recipe and send email you must specify --maintainer\n")
- exit(1)
+ else:
+ if not args.to_version:
+ E(" For upgrade only one recipe you must specify --to_version\n")
+ exit(1)
- pkg_list = [(args.recipe[0], args.to_version, args.maintainer)]
- else:
- pkg_list = []
- for pkg in args.recipe:
- pkg_list.append((pkg, None, None))
+ if not args.maintainer and args.send_emails:
+ E(" For upgrade only one recipe and send email you must specify --maintainer\n")
+ exit(1)
+ pkg_list = [(args.recipe, args.to_version, args.maintainer)]
updater = Updater(args.auto_mode, args.send_emails, args.skip_compilation)
updater.run(pkg_list)
--
1.9.1
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH 02/10][auh] upgradehelper.py: Validate if upgrade is needed
2015-07-29 20:50 [PATCH 00/10][auh] Add support for buildhistory and minor fixes Aníbal Limón
2015-07-29 20:50 ` [PATCH 01/10][auh] upgradehelper.py: Only run with one recipe or all Aníbal Limón
@ 2015-07-29 20:50 ` Aníbal Limón
2015-07-29 20:50 ` [PATCH 03/10][auh] upgradehelper.py: Only send emails when patch file exist Aníbal Limón
` (7 subsequent siblings)
9 siblings, 0 replies; 18+ messages in thread
From: Aníbal Limón @ 2015-07-29 20:50 UTC (permalink / raw)
To: yocto; +Cc: paul.eggleton
When run upgradehelper only with one recipe upgrade needed
validation isn't did.
[YOCTO #7947]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
upgradehelper.py | 24 +++++++++++++++++++++---
1 file changed, 21 insertions(+), 3 deletions(-)
diff --git a/upgradehelper.py b/upgradehelper.py
index 63b30e4..f10d974 100755
--- a/upgradehelper.py
+++ b/upgradehelper.py
@@ -207,6 +207,9 @@ class Updater(object):
def _detect_repo(self):
self._get_env()
+ if self.env['PV'] == self.new_ver:
+ raise UpgradeNotNeededError
+
if self.git is not None:
return
@@ -496,9 +499,24 @@ class Updater(object):
step()
I(" %s: Upgrade SUCCESSFUL! Please test!" % self.pn)
- except Error as e:
- E(" %s: %s" % (self.pn, e.message))
- E(" %s: Upgrade FAILED! Logs and/or file diffs are available in %s" % (self.pn, self.workdir))
+ except Exception as e:
+ if isinstance(e, UpgradeNotNeededError):
+ I(" %s: %s" % (self.pn, e.message))
+ elif isinstance(e, UnsupportedProtocolError):
+ I(" %s: %s" % (self.pn, e.message))
+ else:
+ if not isinstance(e, Error):
+ import traceback
+ msg = "Failed(unknown error)\n" + traceback.format_exc()
+ e = Error(message=msg)
+ error = e
+
+ E(" %s: %s" % (self.pn, e.message))
+
+ if os.listdir(self.workdir):
+ E(" %s: Upgrade FAILED! Logs and/or file diffs are available in %s"
+ % (self.pn, self.workdir))
+
error = e
self._commit_changes()
--
1.9.1
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH 03/10][auh] upgradehelper.py: Only send emails when patch file exist
2015-07-29 20:50 [PATCH 00/10][auh] Add support for buildhistory and minor fixes Aníbal Limón
2015-07-29 20:50 ` [PATCH 01/10][auh] upgradehelper.py: Only run with one recipe or all Aníbal Limón
2015-07-29 20:50 ` [PATCH 02/10][auh] upgradehelper.py: Validate if upgrade is needed Aníbal Limón
@ 2015-07-29 20:50 ` Aníbal Limón
2015-07-29 20:50 ` [PATCH 04/10][auh] bitbake.py: Improve performance on env() call Aníbal Limón
` (6 subsequent siblings)
9 siblings, 0 replies; 18+ messages in thread
From: Aníbal Limón @ 2015-07-29 20:50 UTC (permalink / raw)
To: yocto; +Cc: paul.eggleton
Remove old decision of send email based on error because only
makes sense to send email when patch file was created.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
upgradehelper.py | 10 ++--------
1 file changed, 2 insertions(+), 8 deletions(-)
diff --git a/upgradehelper.py b/upgradehelper.py
index f10d974..13fc7dd 100755
--- a/upgradehelper.py
+++ b/upgradehelper.py
@@ -358,7 +358,8 @@ class Updater(object):
self.git.reset_hard(1)
self.git.clean_untracked()
- if self.send_email:
+ # only send email to Maintainer when patch file exist
+ if self.send_email and self.patch_file:
mail_header = \
"Hello,\n\nYou are receiving this email because you are the maintainer\n" \
"of *%s* recipe and this is to let you know that the automatic attempt\n" \
@@ -383,13 +384,6 @@ class Updater(object):
"Any problem please contact Anibal Limon <anibal.limon@intel.com>.\n\n" \
"Regards,\nThe Upgrade Helper"
- # only send email to Maintainer when is an error that can handle
- if err and not isinstance(err, MaintainerError):
- D( "%s: Don't send email to maintainer because the error was " \
- "%s and the information isn't useful, please review it." \
- % (self.pn, type(err).__name__))
- return
-
if self.maintainer in maintainer_override:
to_addr = maintainer_override[self.maintainer]
else:
--
1.9.1
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH 04/10][auh] bitbake.py: Improve performance on env() call
2015-07-29 20:50 [PATCH 00/10][auh] Add support for buildhistory and minor fixes Aníbal Limón
` (2 preceding siblings ...)
2015-07-29 20:50 ` [PATCH 03/10][auh] upgradehelper.py: Only send emails when patch file exist Aníbal Limón
@ 2015-07-29 20:50 ` Aníbal Limón
2015-07-29 20:50 ` [PATCH 05/10][auh] upgradehelper.py: Add own step for loading environment Aníbal Limón
` (5 subsequent siblings)
9 siblings, 0 replies; 18+ messages in thread
From: Aníbal Limón @ 2015-07-29 20:50 UTC (permalink / raw)
To: yocto; +Cc: paul.eggleton
When AUH load recipe enviroment use bitbake -e call and parses the
output using regex VAR=VALUE.
For improve performance discard all the comments on cmd output adding
a output_filter using grep.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
bitbake.py | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/bitbake.py b/bitbake.py
index 3631896..54389fb 100644
--- a/bitbake.py
+++ b/bitbake.py
@@ -42,7 +42,7 @@ class Bitbake(object):
self.log_dir = None
super(Bitbake, self).__init__()
- def _cmd(self, recipe, options=None, env_var=None):
+ def _cmd(self, recipe, options=None, env_var=None, output_filter=None):
cmd = ""
if env_var is not None:
cmd += env_var + " "
@@ -52,6 +52,9 @@ class Bitbake(object):
cmd += recipe
+ if output_filter is not None:
+ cmd += ' | grep ' + output_filter
+
os.chdir(self.build_dir)
try:
@@ -74,7 +77,7 @@ class Bitbake(object):
return os.path.join(self.log_dir, "bitbake_log.txt")
def env(self, recipe):
- return self._cmd(recipe, "-e")
+ return self._cmd(recipe, "-e", output_filter="-v \"^#\"")
def fetch(self, recipe):
return self._cmd(recipe, "-c fetch")
--
1.9.1
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH 05/10][auh] upgradehelper.py: Add own step for loading environment
2015-07-29 20:50 [PATCH 00/10][auh] Add support for buildhistory and minor fixes Aníbal Limón
` (3 preceding siblings ...)
2015-07-29 20:50 ` [PATCH 04/10][auh] bitbake.py: Improve performance on env() call Aníbal Limón
@ 2015-07-29 20:50 ` Aníbal Limón
2015-07-31 13:48 ` Paul Eggleton
2015-07-29 20:50 ` [PATCH 06/10][auh] upgradehelper.py: UniverseUpdate don't abort if recipe checkpkg fails Aníbal Limón
` (4 subsequent siblings)
9 siblings, 1 reply; 18+ messages in thread
From: Aníbal Limón @ 2015-07-29 20:50 UTC (permalink / raw)
To: yocto; +Cc: paul.eggleton
Instead of load environment in detecting git repository use own
function making easy to debug.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
upgradehelper.py | 11 ++++++-----
1 file changed, 6 insertions(+), 5 deletions(-)
diff --git a/upgradehelper.py b/upgradehelper.py
index 13fc7dd..9119cb0 100755
--- a/upgradehelper.py
+++ b/upgradehelper.py
@@ -147,6 +147,7 @@ class Updater(object):
self.upgrade_steps = [
(self._create_workdir, None),
+ (self._get_env, "Loading environment ..."),
(self._detect_repo, "Detecting git repository location ..."),
(self._clean_repo, "Cleaning git repository of temporary branch ..."),
(self._detect_recipe_type, None),
@@ -205,18 +206,16 @@ class Updater(object):
os.remove(os.path.join(self.workdir, f))
def _detect_repo(self):
- self._get_env()
-
- if self.env['PV'] == self.new_ver:
+ if self.env['PKGV'] == self.new_ver:
raise UpgradeNotNeededError
- if self.git is not None:
+ # UniverseUpdater use git poky respository
+ if isinstance(self.git, UniverseUpdater):
return
self.git = Git(self.recipe_dir)
stdout = self.git.status()
-
if stdout != "":
if self.interactive:
W(" %s: git repository has uncommited work which will be dropped! Proceed? (y/N)" % self.pn)
@@ -527,6 +526,8 @@ class Updater(object):
class UniverseUpdater(Updater):
def __init__(self):
Updater.__init__(self, True, True)
+
+ # XXX: assume that the poky directory is the first entry in the PATH
self.git = Git(os.path.dirname(os.getenv('PATH', False).split(':')[0]))
# read history file
--
1.9.1
^ permalink raw reply related [flat|nested] 18+ messages in thread* Re: [PATCH 05/10][auh] upgradehelper.py: Add own step for loading environment
2015-07-29 20:50 ` [PATCH 05/10][auh] upgradehelper.py: Add own step for loading environment Aníbal Limón
@ 2015-07-31 13:48 ` Paul Eggleton
0 siblings, 0 replies; 18+ messages in thread
From: Paul Eggleton @ 2015-07-31 13:48 UTC (permalink / raw)
To: Aníbal Limón; +Cc: yocto
On Wednesday 29 July 2015 15:50:48 Aníbal Limón wrote:
> Instead of load environment in detecting git repository use own
> function making easy to debug.
The changes here don't seem to entirely match up with the commit message.
> Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
> ---
> upgradehelper.py | 11 ++++++-----
> 1 file changed, 6 insertions(+), 5 deletions(-)
>
> diff --git a/upgradehelper.py b/upgradehelper.py
> index 13fc7dd..9119cb0 100755
> --- a/upgradehelper.py
> +++ b/upgradehelper.py
> @@ -147,6 +147,7 @@ class Updater(object):
>
> self.upgrade_steps = [
> (self._create_workdir, None),
> + (self._get_env, "Loading environment ..."),
> (self._detect_repo, "Detecting git repository location ..."),
> (self._clean_repo, "Cleaning git repository of temporary branch
> ..."), (self._detect_recipe_type, None),
> @@ -205,18 +206,16 @@ class Updater(object):
> os.remove(os.path.join(self.workdir, f))
>
> def _detect_repo(self):
> - self._get_env()
> -
> - if self.env['PV'] == self.new_ver:
> + if self.env['PKGV'] == self.new_ver:
> raise UpgradeNotNeededError
>
> - if self.git is not None:
> + # UniverseUpdater use git poky respository
> + if isinstance(self.git, UniverseUpdater):
> return
>
> self.git = Git(self.recipe_dir)
>
> stdout = self.git.status()
> -
> if stdout != "":
> if self.interactive:
> W(" %s: git repository has uncommited work which will be
> dropped! Proceed? (y/N)" % self.pn) @@ -527,6 +526,8 @@ class
> Updater(object):
> class UniverseUpdater(Updater):
> def __init__(self):
> Updater.__init__(self, True, True)
> +
> + # XXX: assume that the poky directory is the first entry in the
> PATH self.git = Git(os.path.dirname(os.getenv('PATH',
> False).split(':')[0]))
>
> # read history file
--
Paul Eggleton
Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 18+ messages in thread
* [PATCH 06/10][auh] upgradehelper.py: UniverseUpdate don't abort if recipe checkpkg fails.
2015-07-29 20:50 [PATCH 00/10][auh] Add support for buildhistory and minor fixes Aníbal Limón
` (4 preceding siblings ...)
2015-07-29 20:50 ` [PATCH 05/10][auh] upgradehelper.py: Add own step for loading environment Aníbal Limón
@ 2015-07-29 20:50 ` Aníbal Limón
2015-07-29 20:50 ` [PATCH 07/10][auh] bitbake.py: Rename bitbake_log.txt to bitbake_error_log Aníbal Limón
` (3 subsequent siblings)
9 siblings, 0 replies; 18+ messages in thread
From: Aníbal Limón @ 2015-07-29 20:50 UTC (permalink / raw)
To: yocto; +Cc: paul.eggleton
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
upgradehelper.py | 5 +----
1 file changed, 1 insertion(+), 4 deletions(-)
diff --git a/upgradehelper.py b/upgradehelper.py
index 9119cb0..7bc8eed 100755
--- a/upgradehelper.py
+++ b/upgradehelper.py
@@ -285,10 +285,7 @@ class Updater(object):
C(" \"distrodata.bbclass\" not inherited. Consider adding "
"the following to your local.conf:\n\n"
"INHERIT =+ \"distrodata\"\n")
- else:
- C(line)
-
- exit(1)
+ exit(1)
def _parse_checkpkg_file(self, file_path):
import csv
--
1.9.1
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH 07/10][auh] bitbake.py: Rename bitbake_log.txt to bitbake_error_log
2015-07-29 20:50 [PATCH 00/10][auh] Add support for buildhistory and minor fixes Aníbal Limón
` (5 preceding siblings ...)
2015-07-29 20:50 ` [PATCH 06/10][auh] upgradehelper.py: UniverseUpdate don't abort if recipe checkpkg fails Aníbal Limón
@ 2015-07-29 20:50 ` Aníbal Limón
2015-07-29 20:50 ` [PATCH 08/10][auh] upgradehelper.py: Remove unused references to Buildhistory class Aníbal Limón
` (2 subsequent siblings)
9 siblings, 0 replies; 18+ messages in thread
From: Aníbal Limón @ 2015-07-29 20:50 UTC (permalink / raw)
To: yocto; +Cc: paul.eggleton
Bitbake log only is written when error appears, rename it to be
consistent with information into the log.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
bitbake.py | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/bitbake.py b/bitbake.py
index 54389fb..e23dc28 100644
--- a/bitbake.py
+++ b/bitbake.py
@@ -36,6 +36,8 @@ for path in os.environ["PATH"].split(':'):
sys.path.insert(0, os.path.join(path, "../lib"))
import bb
+BITBAKE_ERROR_LOG = 'bitbake_error_log.txt'
+
class Bitbake(object):
def __init__(self, build_dir):
self.build_dir = build_dir
@@ -63,7 +65,7 @@ class Bitbake(object):
D("%s returned:\n%s" % (cmd, e.__str__()))
if self.log_dir is not None and os.path.exists(self.log_dir):
- with open(os.path.join(self.log_dir, "bitbake_log.txt"), "w+") as log:
+ with open(os.path.join(self.log_dir, BITBAKE_ERROR_LOG), "w+") as log:
log.write(e.stdout)
raise Error("\'" + cmd + "\' failed", e.stdout, e.stderr)
@@ -74,7 +76,7 @@ class Bitbake(object):
self.log_dir = dir
def get_stdout_log(self):
- return os.path.join(self.log_dir, "bitbake_log.txt")
+ return os.path.join(self.log_dir, BITBAKE_ERROR_LOG)
def env(self, recipe):
return self._cmd(recipe, "-e", output_filter="-v \"^#\"")
--
1.9.1
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH 08/10][auh] upgradehelper.py: Remove unused references to Buildhistory class
2015-07-29 20:50 [PATCH 00/10][auh] Add support for buildhistory and minor fixes Aníbal Limón
` (6 preceding siblings ...)
2015-07-29 20:50 ` [PATCH 07/10][auh] bitbake.py: Rename bitbake_log.txt to bitbake_error_log Aníbal Limón
@ 2015-07-29 20:50 ` Aníbal Limón
2015-07-29 20:50 ` [PATCH 09/10][auh] upgradehelper.py: Move upstream versioning code to UniverseUpdater Aníbal Limón
2015-07-29 20:50 ` [PATCH 10/10][auh] upgradehelper: Add support for generate buildhistory recipe diff's Aníbal Limón
9 siblings, 0 replies; 18+ messages in thread
From: Aníbal Limón @ 2015-07-29 20:50 UTC (permalink / raw)
To: yocto; +Cc: paul.eggleton
Buildhistory class isn't implemented so removed related code.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
upgradehelper.py | 11 +----------
1 file changed, 1 insertion(+), 10 deletions(-)
diff --git a/upgradehelper.py b/upgradehelper.py
index 7bc8eed..a31d041 100755
--- a/upgradehelper.py
+++ b/upgradehelper.py
@@ -43,7 +43,7 @@ from datetime import date
import shutil
from errors import *
from git import Git
-from bitbake import Bitbake, BuildHistory
+from bitbake import Bitbake
from emailhandler import Email
from statistics import Statistics
from recipe import Recipe
@@ -135,7 +135,6 @@ class Updater(object):
os.mkdir(self.uh_work_dir)
self.bb = Bitbake(get_build_dir())
- self.buildhistory = BuildHistory(get_build_dir())
self.git = None
self.author_email = settings.get('from', 'uh@not.set')
self.author = "Upgrade Helper <%s>" % self.author_email
@@ -266,14 +265,6 @@ class Updater(object):
I(" %s: compiling for %s ..." % (self.pn, machine))
self.recipe.compile(machine)
- def _review(self):
- # Check build_history
- if not self.skip_compilation:
- I(" %s: Checking buildhistory ..." % self.pn)
- self.buildhistory.set_work_dir(self.workdir)
- if self.buildhistory.diff(len(self.machines)):
- I(" %s: Wrote buildhistory-diff output ..." % self.pn)
-
def _check_upstream_versions(self, packages=[("universe", None, None)]):
I(" Fetching upstream version(s) ...")
--
1.9.1
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH 09/10][auh] upgradehelper.py: Move upstream versioning code to UniverseUpdater
2015-07-29 20:50 [PATCH 00/10][auh] Add support for buildhistory and minor fixes Aníbal Limón
` (7 preceding siblings ...)
2015-07-29 20:50 ` [PATCH 08/10][auh] upgradehelper.py: Remove unused references to Buildhistory class Aníbal Limón
@ 2015-07-29 20:50 ` Aníbal Limón
2015-07-29 20:50 ` [PATCH 10/10][auh] upgradehelper: Add support for generate buildhistory recipe diff's Aníbal Limón
9 siblings, 0 replies; 18+ messages in thread
From: Aníbal Limón @ 2015-07-29 20:50 UTC (permalink / raw)
To: yocto; +Cc: paul.eggleton
Reorder code related to upstream versioning detection no makes sense
to have in Updater because only in UniverseUpdater is used.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
upgradehelper.py | 161 ++++++++++++++++++++++++++-----------------------------
1 file changed, 77 insertions(+), 84 deletions(-)
diff --git a/upgradehelper.py b/upgradehelper.py
index a31d041..4d8685c 100755
--- a/upgradehelper.py
+++ b/upgradehelper.py
@@ -265,63 +265,12 @@ class Updater(object):
I(" %s: compiling for %s ..." % (self.pn, machine))
self.recipe.compile(machine)
- def _check_upstream_versions(self, packages=[("universe", None, None)]):
- I(" Fetching upstream version(s) ...")
-
- try:
- self.bb.checkpkg(" ".join([p[0] for p in packages]))
- except Error as e:
- for line in e.stdout.split('\n'):
- if line.find("ERROR: Task do_checkpkg does not exist") == 0:
- C(" \"distrodata.bbclass\" not inherited. Consider adding "
- "the following to your local.conf:\n\n"
- "INHERIT =+ \"distrodata\"\n")
- exit(1)
-
- def _parse_checkpkg_file(self, file_path):
- import csv
-
- pkgs_list = []
-
- with open(file_path, "r") as f:
- reader = csv.reader(f, delimiter='\t')
- for row in reader:
- if reader.line_num == 1: # skip header line
- continue
-
- pn = row[0]
- cur_ver = row[1]
- next_ver = row[2]
- status = row[11]
- maintainer = row[14]
- no_upgrade_reason = row[15]
-
- if status == 'UPDATE' and not no_upgrade_reason:
- pkgs_list.append((pn, next_ver, maintainer))
- else:
- if no_upgrade_reason:
- D(" Skip package %s (status = %s, current version = %s," \
- " next version = %s, no upgrade reason = %s)" %
- (pn, status, cur_ver, next_ver, no_upgrade_reason))
- else:
- D(" Skip package %s (status = %s, current version = %s," \
- " next version = %s)" %
- (pn, status, cur_ver, next_ver))
- return pkgs_list
-
def _get_packages_to_upgrade(self, packages=None):
if packages is None:
- return []
-
- if len(packages) == 1:
- # if user specified the version to upgrade to, just return the
- # tuple intact
- if packages[0][1] is not None:
- return packages
-
- self._check_upstream_versions(packages)
-
- return self._parse_checkpkg_file(get_build_dir() + "/tmp/log/checkpkg.csv")
+ I( "Nothing to upgrade")
+ exit(0)
+ else:
+ return packages
# this function will be called at the end of each recipe upgrade
def pkg_upgrade_handler(self, err):
@@ -530,6 +479,75 @@ class UniverseUpdater(Updater):
line.split(',')[3],
line.split(',')[4]]
+ def _update_master(self):
+ I(" Drop all uncommited changes (including untracked) ...")
+ self.git.reset_hard()
+ self.git.clean_untracked()
+
+ self.git.checkout_branch("master")
+ try:
+ self.git.delete_branch("upgrades")
+ except Error:
+ pass
+ I(" Sync master ...")
+ self.git.pull()
+ self.git.create_branch("upgrades")
+
+ def _prepare(self):
+ if settings.get("clean_sstate", "no") == "yes" and \
+ os.path.exists(os.path.join(get_build_dir(), "sstate-cache")):
+ I(" Removing sstate directory ...")
+ shutil.rmtree(os.path.join(get_build_dir(), "sstate-cache"))
+ if settings.get("clean_tmp", "no") == "yes" and \
+ os.path.exists(os.path.join(get_build_dir(), "tmp")):
+ I(" Removing tmp directory ...")
+ shutil.rmtree(os.path.join(get_build_dir(), "tmp"))
+
+
+ def _check_upstream_versions(self, packages=[("universe", None, None)]):
+ I(" Fetching upstream version(s) ...")
+
+ try:
+ self.bb.checkpkg(" ".join([p[0] for p in packages]))
+ except Error as e:
+ for line in e.stdout.split('\n'):
+ if line.find("ERROR: Task do_checkpkg does not exist") == 0:
+ C(" \"distrodata.bbclass\" not inherited. Consider adding "
+ "the following to your local.conf:\n\n"
+ "INHERIT =+ \"distrodata\"\n")
+ exit(1)
+
+ def _parse_checkpkg_file(self, file_path):
+ import csv
+
+ pkgs_list = []
+
+ with open(file_path, "r") as f:
+ reader = csv.reader(f, delimiter='\t')
+ for row in reader:
+ if reader.line_num == 1: # skip header line
+ continue
+
+ pn = row[0]
+ cur_ver = row[1]
+ next_ver = row[2]
+ status = row[11]
+ maintainer = row[14]
+ no_upgrade_reason = row[15]
+
+ if status == 'UPDATE' and not no_upgrade_reason:
+ pkgs_list.append((pn, next_ver, maintainer))
+ else:
+ if no_upgrade_reason:
+ D(" Skip package %s (status = %s, current version = %s," \
+ " next version = %s, no upgrade reason = %s)" %
+ (pn, status, cur_ver, next_ver, no_upgrade_reason))
+ else:
+ D(" Skip package %s (status = %s, current version = %s," \
+ " next version = %s)" %
+ (pn, status, cur_ver, next_ver))
+ return pkgs_list
+
# checks if maintainer is in whitelist and that the recipe itself is not
# blacklisted: python, gcc, etc. Also, check the history if the recipe
# hasn't already been tried
@@ -580,30 +598,6 @@ class UniverseUpdater(Updater):
return True
- def update_master(self):
- I(" Drop all uncommited changes (including untracked) ...")
- self.git.reset_hard()
- self.git.clean_untracked()
-
- self.git.checkout_branch("master")
- try:
- self.git.delete_branch("upgrades")
- except Error:
- pass
- I(" Sync master ...")
- self.git.pull()
- self.git.create_branch("upgrades")
-
- def prepare(self):
- if settings.get("clean_sstate", "no") == "yes" and \
- os.path.exists(os.path.join(get_build_dir(), "sstate-cache")):
- I(" Removing sstate directory ...")
- shutil.rmtree(os.path.join(get_build_dir(), "sstate-cache"))
- if settings.get("clean_tmp", "no") == "yes" and \
- os.path.exists(os.path.join(get_build_dir(), "tmp")):
- I(" Removing tmp directory ...")
- shutil.rmtree(os.path.join(get_build_dir(), "tmp"))
-
def _get_packages_to_upgrade(self, packages=None):
last_date_checked = None
last_master_commit = None
@@ -649,7 +643,7 @@ class UniverseUpdater(Updater):
return pkgs_list
- def update_history(self, pn, new_ver, maintainer, upgrade_status):
+ def _update_history(self, pn, new_ver, maintainer, upgrade_status):
with open(self.history_file + ".tmp", "w+") as tmp_file:
if os.path.exists(self.history_file):
with open(self.history_file) as history:
@@ -661,15 +655,14 @@ class UniverseUpdater(Updater):
upgrade_status + "\n")
os.rename(self.history_file + ".tmp", self.history_file)
- # overriding the base method
def pkg_upgrade_handler(self, err):
super(UniverseUpdater, self).pkg_upgrade_handler(err)
- self.update_history(self.pn, self.new_ver, self.maintainer,
+ self._update_history(self.pn, self.new_ver, self.maintainer,
self._get_status_msg(err))
def run(self):
- self.update_master()
- self.prepare()
+ self._update_master()
+ self._prepare()
super(UniverseUpdater, self).run()
def close_child_processes(signal_id, frame):
--
1.9.1
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH 10/10][auh] upgradehelper: Add support for generate buildhistory recipe diff's
2015-07-29 20:50 [PATCH 00/10][auh] Add support for buildhistory and minor fixes Aníbal Limón
` (8 preceding siblings ...)
2015-07-29 20:50 ` [PATCH 09/10][auh] upgradehelper.py: Move upstream versioning code to UniverseUpdater Aníbal Limón
@ 2015-07-29 20:50 ` Aníbal Limón
2015-07-31 15:29 ` Paul Eggleton
9 siblings, 1 reply; 18+ messages in thread
From: Aníbal Limón @ 2015-07-29 20:50 UTC (permalink / raw)
To: yocto; +Cc: paul.eggleton
Now AUH generates buildhistory diff when recipe was upgraded
successfully.
For enable this feature set into local.conf.
INHERIT += "buildhistory"
BUILDHISTORY_COMMIT = "1"
Summary of the changes,
- bitbake.py: Enable environment generation without recipe,
removes Buildhistory class.
- buildhistory.py: Add buildhistory class that generates initial
buildhistory revision and keeps track of build revisions for
generate diff when build is successful.
- upgradehelper.py: Add buildhistory steps for upgrade, add
support for detect when buildhistory is enabled loading the
environment without recipe.
[YOCTO #7175]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
bitbake.py | 41 ++------------------
buildhistory.py | 75 ++++++++++++++++++++++++++++++++++++
errors.py | 6 +++
upgradehelper.py | 115 ++++++++++++++++++++++++++++++++++++++++---------------
4 files changed, 170 insertions(+), 67 deletions(-)
create mode 100644 buildhistory.py
diff --git a/bitbake.py b/bitbake.py
index e23dc28..a1587ce 100644
--- a/bitbake.py
+++ b/bitbake.py
@@ -44,7 +44,7 @@ class Bitbake(object):
self.log_dir = None
super(Bitbake, self).__init__()
- def _cmd(self, recipe, options=None, env_var=None, output_filter=None):
+ def _cmd(self, recipe=None, options=None, env_var=None, output_filter=None):
cmd = ""
if env_var is not None:
cmd += env_var + " "
@@ -52,7 +52,8 @@ class Bitbake(object):
if options is not None:
cmd += options + " "
- cmd += recipe
+ if recipe is not None:
+ cmd += recipe
if output_filter is not None:
cmd += ' | grep ' + output_filter
@@ -78,7 +79,7 @@ class Bitbake(object):
def get_stdout_log(self):
return os.path.join(self.log_dir, BITBAKE_ERROR_LOG)
- def env(self, recipe):
+ def env(self, recipe=None):
return self._cmd(recipe, "-e", output_filter="-v \"^#\"")
def fetch(self, recipe):
@@ -104,37 +105,3 @@ class Bitbake(object):
def dependency_graph(self, package_list):
return self._cmd(package_list, "-g")
-
-class BuildHistory(object):
- def __init__(self, build_dir):
- self.build_dir = build_dir
- self.work_dir = None
-
- def set_work_dir(self, work_dir):
- self.work_dir = work_dir
-
- # Return True if buildhistory-diff gives output
- def diff(self, revision_steps):
- os.chdir(self.build_dir)
- cmd = "buildhistory-diff HEAD~" + str(revision_steps)
-
- try:
- stdout, stderr = bb.process.run(cmd)
- # Write diff output to log file if there is any
-
- if stdout and os.path.exists(self.work_dir):
- with open(os.path.join(self.work_dir, "buildhistory.txt"), "w+") as log:
- log.write(stdout)
- return True
- except bb.process.ExecutionError as e:
- for line in e.stdout.split('\n'):
- if line.find("Buildhistory directory \"buildhistory/\" does not exist") == 0:
- C(" \"buildhistory.bbclass\" not inherited. Consider adding "
- "the following to your local.conf:\n\n"
- "INHERIT =+ \"buildhistory\"\n"
- "BUILDHISTORY_COMMIT = \"1\"\n\n"
- "Do not remove any other inherited class in the process (e.g. distrodata)\n")
- exit(1)
-
- return False
-
diff --git a/buildhistory.py b/buildhistory.py
new file mode 100644
index 0000000..1732f23
--- /dev/null
+++ b/buildhistory.py
@@ -0,0 +1,75 @@
+#!/usr/bin/env python
+# vim: set ts=4 sw=4 et:
+#
+# Copyright (c) 2015 Intel Corporation
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License
+# as published by the Free Software Foundation; either version 2
+# of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+#
+
+import os
+import logging as log
+from logging import info as I
+from logging import debug as D
+from logging import error as E
+from logging import critical as C
+import sys
+from errors import *
+
+from bitbake import *
+from git import Git
+
+os.environ['BB_ENV_EXTRAWHITE'] = os.environ['BB_ENV_EXTRAWHITE'] + \
+ " BUILDHISTORY_DIR"
+
+class BuildHistory(object):
+ def __init__(self, bb, pn, workdir):
+ self.bb = bb
+ self.pn = pn
+ self.workdir = workdir
+ self.revs = []
+
+ self.buildhistory_dir = os.path.join(self.workdir, 'buildhistory')
+ if not os.path.exists(self.buildhistory_dir):
+ os.mkdir(self.buildhistory_dir)
+
+ self.git = Git(self.buildhistory_dir)
+
+ os.environ["BUILDHISTORY_DIR"] = self.buildhistory_dir
+
+ def init(self, machines):
+ self.bb.cleanall(self.pn)
+ for machine in machines:
+ self.bb.complete(self.pn, machine)
+ self.revs.append(self.git.last_commit("master"))
+
+ def add(self):
+ self.revs.append(self.git.last_commit("master"))
+
+ def diff(self):
+ rev_initial = self.revs[0]
+ rev_final = self.revs[-1]
+
+ cmd = "buildhistory-diff -a -p %s %s %s" % (self.buildhistory_dir,
+ rev_initial, rev_final)
+
+ try:
+ stdout, stderr = bb.process.run(cmd)
+
+ if stdout and os.path.exists(self.workdir):
+ with open(os.path.join(self.workdir, "buildhistory-diff.txt"),
+ "w+") as log:
+ log.write(stdout)
+ except bb.process.ExecutionError as e:
+ W( "%s: Buildhistory checking fails\n%s" % (self.pn, e.stdout))
diff --git a/errors.py b/errors.py
index 7194944..1504fa5 100644
--- a/errors.py
+++ b/errors.py
@@ -85,3 +85,9 @@ class UpgradeNotNeededError(Error):
def __str__(self):
return "Failed(up to date)"
+class EmptyEnvError(Error):
+ def __init__(self, stdout):
+ super(EmptyEnvError, self).__init__("Empty environment returned", stdout)
+
+ def __str__(self):
+ return "Failed(get_env)"
diff --git a/upgradehelper.py b/upgradehelper.py
index 4d8685c..a8bc5ec 100755
--- a/upgradehelper.py
+++ b/upgradehelper.py
@@ -44,6 +44,7 @@ import shutil
from errors import *
from git import Git
from bitbake import Bitbake
+from buildhistory import BuildHistory
from emailhandler import Email
from statistics import Statistics
from recipe import Recipe
@@ -145,23 +146,40 @@ class Updater(object):
self.machines = settings.get('machines', 'qemux86 qemux86-64 qemuarm qemumips qemuppc').split()
self.upgrade_steps = [
+ (self._load_env, "Loading environment ..."),
(self._create_workdir, None),
- (self._get_env, "Loading environment ..."),
(self._detect_repo, "Detecting git repository location ..."),
(self._clean_repo, "Cleaning git repository of temporary branch ..."),
(self._detect_recipe_type, None),
+ (self._buildhistory_init, None),
(self._unpack_original, "Fetch & unpack original version ..."),
(self._rename, "Renaming recipes, reset PR (if exists) ..."),
(self._cleanall, "Clean all ..."),
(self._fetch, "Fetch new version (old checksums) ..."),
- (self._compile, None)
+ (self._compile, None),
+ (self._buildhistory_diff, None)
]
+ try:
+ self.base_env = self._get_env()
+ except EmptyEnvError as e:
+ import traceback
+ E( " %s\n%s" % (e.message, traceback.format_exc()))
+ E( " Bitbake output:\n%s" % (e.stdout))
+ exit(1)
+ self.buildhistory_enabled = self._buildhistory_is_enabled()
+
self.email_handler = Email(settings)
self.statistics = Statistics()
- def _get_env(self):
- stdout = self.bb.env(self.pn)
+ def _get_status_msg(self, err):
+ if err:
+ return str(err)
+ else:
+ return "Succeeded"
+
+ def _get_env(self, pn=None):
+ stdout = self.bb.env(pn)
assignment = re.compile("^([^ \t=]*)=(.*)")
bb_env = dict()
@@ -173,38 +191,46 @@ class Updater(object):
bb_env[m.group(1)] = m.group(2).strip("\"")
- self.env = bb_env
- self.recipe_dir = os.path.dirname(self.env['FILE'])
+ if not bb_env:
+ raise EmptyEnvError(stdout)
- def _detect_recipe_type(self):
- if self.env['SRC_URI'].find("ftp://") != -1 or \
- self.env['SRC_URI'].find("http://") != -1 or \
- self.env['SRC_URI'].find("https://") != -1:
- recipe = Recipe
- elif self.env['SRC_URI'].find("git://") != -1:
- recipe = GitRecipe
- else:
- raise UnsupportedProtocolError
+ return bb_env
- self.recipe = recipe(self.env, self.new_ver, self.interactive, self.workdir,
- self.recipe_dir, self.bb, self.git)
+ def _buildhistory_is_enabled(self):
+ enabled = False
- def _get_status_msg(self, err):
- if err:
- return str(err)
- else:
- return "Succeeded"
+ if 'buildhistory' in self.base_env['INHERIT']:
+ if not 'BUILDHISTORY_COMMIT' in self.base_env:
+ E(" Buildhistory was enabled but need"\
+ " BUILDHISTORY_COMMIT=1 please set.")
+ exit(1)
+
+ if not self.base_env['BUILDHISTORY_COMMIT'] == '1':
+ E(" Buildhistory was enabled but need"\
+ " BUILDHISTORY_COMMIT=1 please set.")
+ exit(1)
+
+ if self.skip_compilation:
+ W(" Buildhistory disabled because user" \
+ " skip compilation!")
+ else:
+ enabled = True
+
+ return enabled
+
+ def _load_env(self):
+ self.env = self._get_env(self.pn)
def _create_workdir(self):
self.workdir = os.path.join(self.uh_work_dir, self.pn)
- if not os.path.exists(self.workdir):
- os.mkdir(self.workdir)
- else:
- for f in os.listdir(self.workdir):
- os.remove(os.path.join(self.workdir, f))
+ if os.path.exists(self.workdir):
+ shutil.rmtree(self.workdir)
+ os.mkdir(self.workdir)
def _detect_repo(self):
+ self.recipe_dir = os.path.dirname(self.env['FILE'])
+
if self.env['PKGV'] == self.new_ver:
raise UpgradeNotNeededError
@@ -227,7 +253,7 @@ class Updater(object):
self.git.reset_hard()
self.git.clean_untracked()
- self._get_env()
+ self.env = self._get_env(self.pn)
def _clean_repo(self):
try:
@@ -239,14 +265,34 @@ class Updater(object):
except:
pass
+ def _detect_recipe_type(self):
+ if self.env['SRC_URI'].find("ftp://") != -1 or \
+ self.env['SRC_URI'].find("http://") != -1 or \
+ self.env['SRC_URI'].find("https://") != -1:
+ recipe = Recipe
+ elif self.env['SRC_URI'].find("git://") != -1:
+ recipe = GitRecipe
+ else:
+ raise UnsupportedProtocolError
+
+ self.recipe = recipe(self.env, self.new_ver, self.interactive, self.workdir,
+ self.recipe_dir, self.bb, self.git)
+
+ def _buildhistory_init(self):
+ if self.buildhistory_enabled == False:
+ return
+
+ self.buildhistory = BuildHistory(self.bb, self.pn, self.workdir)
+ I(" %s: Initial buildhistory for %s ..." % (self.pn, self.machines))
+ self.buildhistory.init(self.machines)
+
def _unpack_original(self):
self.recipe.unpack()
def _rename(self):
self.recipe.rename()
- # fetch new environment
- self._get_env()
+ self.env = self._get_env(self.pn)
self.recipe.update_env(self.env)
@@ -264,6 +310,15 @@ class Updater(object):
for machine in self.machines:
I(" %s: compiling for %s ..." % (self.pn, machine))
self.recipe.compile(machine)
+ if self.buildhistory is not None:
+ self.buildhistory.add()
+
+ def _buildhistory_diff(self):
+ if self.buildhistory_enabled == False:
+ return
+
+ I(" %s: Checking buildhistory ..." % self.pn)
+ self.buildhistory.diff()
def _get_packages_to_upgrade(self, packages=None):
if packages is None:
--
1.9.1
^ permalink raw reply related [flat|nested] 18+ messages in thread* Re: [PATCH 10/10][auh] upgradehelper: Add support for generate buildhistory recipe diff's
2015-07-29 20:50 ` [PATCH 10/10][auh] upgradehelper: Add support for generate buildhistory recipe diff's Aníbal Limón
@ 2015-07-31 15:29 ` Paul Eggleton
2015-07-31 15:39 ` Aníbal Limón
0 siblings, 1 reply; 18+ messages in thread
From: Paul Eggleton @ 2015-07-31 15:29 UTC (permalink / raw)
To: Aníbal Limón; +Cc: yocto
Hi Aníbal,
On Wednesday 29 July 2015 15:50:53 Aníbal Limón wrote:
> Now AUH generates buildhistory diff when recipe was upgraded
> successfully.
>
> For enable this feature set into local.conf.
>
> INHERIT += "buildhistory"
> BUILDHISTORY_COMMIT = "1"
>
> Summary of the changes,
>
> - bitbake.py: Enable environment generation without recipe,
> removes Buildhistory class.
> - buildhistory.py: Add buildhistory class that generates initial
> buildhistory revision and keeps track of build revisions for
> generate diff when build is successful.
> - upgradehelper.py: Add buildhistory steps for upgrade, add
> support for detect when buildhistory is enabled loading the
> environment without recipe.
So this looks OK, but I can't tell if it ensures you get the difference between
successful builds rather than the difference to the last build - we need to
ensure that a failed build of a recipe in between isn't considered as part of
the comparison. In practice I guess this would mean storing the buildhistory
revision of the last successful build for each recipe.
Cheers,
Paul
--
Paul Eggleton
Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 10/10][auh] upgradehelper: Add support for generate buildhistory recipe diff's
2015-07-31 15:29 ` Paul Eggleton
@ 2015-07-31 15:39 ` Aníbal Limón
2015-07-31 16:10 ` Paul Eggleton
0 siblings, 1 reply; 18+ messages in thread
From: Aníbal Limón @ 2015-07-31 15:39 UTC (permalink / raw)
To: Paul Eggleton; +Cc: yocto
Hi Paul,
On 31/07/15 10:29, Paul Eggleton wrote:
> Hi Aníbal,
>
> On Wednesday 29 July 2015 15:50:53 Aníbal Limón wrote:
>> Now AUH generates buildhistory diff when recipe was upgraded
>> successfully.
>>
>> For enable this feature set into local.conf.
>>
>> INHERIT += "buildhistory"
>> BUILDHISTORY_COMMIT = "1"
>>
>> Summary of the changes,
>>
>> - bitbake.py: Enable environment generation without recipe,
>> removes Buildhistory class.
>> - buildhistory.py: Add buildhistory class that generates initial
>> buildhistory revision and keeps track of build revisions for
>> generate diff when build is successful.
>> - upgradehelper.py: Add buildhistory steps for upgrade, add
>> support for detect when buildhistory is enabled loading the
>> environment without recipe.
> So this looks OK, but I can't tell if it ensures you get the difference between
> successful builds rather than the difference to the last build - we need to
> ensure that a failed build of a recipe in between isn't considered as part of
> the comparison. In practice I guess this would mean storing the buildhistory
> revision of the last successful build for each recipe.
I only made a diff if the build success in all machines this guarantee
that only
stores good information.
alimon
>
> Cheers,
> Paul
>
^ permalink raw reply [flat|nested] 18+ messages in thread* Re: [PATCH 10/10][auh] upgradehelper: Add support for generate buildhistory recipe diff's
2015-07-31 15:39 ` Aníbal Limón
@ 2015-07-31 16:10 ` Paul Eggleton
2015-07-31 16:19 ` Aníbal Limón
0 siblings, 1 reply; 18+ messages in thread
From: Paul Eggleton @ 2015-07-31 16:10 UTC (permalink / raw)
To: Aníbal Limón; +Cc: yocto
On Friday 31 July 2015 10:39:37 Aníbal Limón wrote:
> Hi Paul,
>
> On 31/07/15 10:29, Paul Eggleton wrote:
> > Hi Aníbal,
> >
> > On Wednesday 29 July 2015 15:50:53 Aníbal Limón wrote:
> >> Now AUH generates buildhistory diff when recipe was upgraded
> >> successfully.
> >>
> >> For enable this feature set into local.conf.
> >>
> >> INHERIT += "buildhistory"
> >> BUILDHISTORY_COMMIT = "1"
> >>
> >> Summary of the changes,
> >>
> >> - bitbake.py: Enable environment generation without recipe,
> >>
> >> removes Buildhistory class.
> >>
> >> - buildhistory.py: Add buildhistory class that generates initial
> >>
> >> buildhistory revision and keeps track of build revisions for
> >> generate diff when build is successful.
> >>
> >> - upgradehelper.py: Add buildhistory steps for upgrade, add
> >>
> >> support for detect when buildhistory is enabled loading the
> >> environment without recipe.
> >
> > So this looks OK, but I can't tell if it ensures you get the difference
> > between successful builds rather than the difference to the last build -
> > we need to ensure that a failed build of a recipe in between isn't
> > considered as part of the comparison. In practice I guess this would mean
> > storing the buildhistory revision of the last successful build for each
> > recipe.
>
> I only made a diff if the build success in all machines this guarantee
> that only stores good information.
OK, I think I'm following now.
One other thing - I'm not particularly keen on the BB_ENV_EXTRAWHITE thing -
can you perhaps query the build system for the path it's using instead?
CHeers,
Paul
Cheers,
Paul
--
Paul Eggleton
Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 10/10][auh] upgradehelper: Add support for generate buildhistory recipe diff's
2015-07-31 16:10 ` Paul Eggleton
@ 2015-07-31 16:19 ` Aníbal Limón
2015-07-31 16:21 ` Paul Eggleton
0 siblings, 1 reply; 18+ messages in thread
From: Aníbal Limón @ 2015-07-31 16:19 UTC (permalink / raw)
To: Paul Eggleton; +Cc: yocto
On 31/07/15 11:10, Paul Eggleton wrote:
> On Friday 31 July 2015 10:39:37 Aníbal Limón wrote:
>> Hi Paul,
>>
>> On 31/07/15 10:29, Paul Eggleton wrote:
>>> Hi Aníbal,
>>>
>>> On Wednesday 29 July 2015 15:50:53 Aníbal Limón wrote:
>>>> Now AUH generates buildhistory diff when recipe was upgraded
>>>> successfully.
>>>>
>>>> For enable this feature set into local.conf.
>>>>
>>>> INHERIT += "buildhistory"
>>>> BUILDHISTORY_COMMIT = "1"
>>>>
>>>> Summary of the changes,
>>>>
>>>> - bitbake.py: Enable environment generation without recipe,
>>>>
>>>> removes Buildhistory class.
>>>>
>>>> - buildhistory.py: Add buildhistory class that generates initial
>>>>
>>>> buildhistory revision and keeps track of build revisions for
>>>> generate diff when build is successful.
>>>>
>>>> - upgradehelper.py: Add buildhistory steps for upgrade, add
>>>>
>>>> support for detect when buildhistory is enabled loading the
>>>> environment without recipe.
>>> So this looks OK, but I can't tell if it ensures you get the difference
>>> between successful builds rather than the difference to the last build -
>>> we need to ensure that a failed build of a recipe in between isn't
>>> considered as part of the comparison. In practice I guess this would mean
>>> storing the buildhistory revision of the last successful build for each
>>> recipe.
>> I only made a diff if the build success in all machines this guarantee
>> that only stores good information.
> OK, I think I'm following now.
>
> One other thing - I'm not particularly keen on the BB_ENV_EXTRAWHITE thing -
> can you perhaps query the build system for the path it's using instead?
I modified this env var to pass BUILDHISTORY_DIR variable per recipe
this enables
to save buildhistory into AUH workdir
(build/upgrade-helper/work/recipe/buildhistory).
alimon
>
> CHeers,
> Paul
>
> Cheers,
> Paul
>
^ permalink raw reply [flat|nested] 18+ messages in thread* Re: [PATCH 10/10][auh] upgradehelper: Add support for generate buildhistory recipe diff's
2015-07-31 16:19 ` Aníbal Limón
@ 2015-07-31 16:21 ` Paul Eggleton
0 siblings, 0 replies; 18+ messages in thread
From: Paul Eggleton @ 2015-07-31 16:21 UTC (permalink / raw)
To: Aníbal Limón; +Cc: yocto
On Friday 31 July 2015 11:19:04 Aníbal Limón wrote:
> On 31/07/15 11:10, Paul Eggleton wrote:
> > On Friday 31 July 2015 10:39:37 Aníbal Limón wrote:
> >> Hi Paul,
> >>
> >> On 31/07/15 10:29, Paul Eggleton wrote:
> >>> Hi Aníbal,
> >>>
> >>> On Wednesday 29 July 2015 15:50:53 Aníbal Limón wrote:
> >>>> Now AUH generates buildhistory diff when recipe was upgraded
> >>>> successfully.
> >>>>
> >>>> For enable this feature set into local.conf.
> >>>>
> >>>> INHERIT += "buildhistory"
> >>>> BUILDHISTORY_COMMIT = "1"
> >>>>
> >>>> Summary of the changes,
> >>>>
> >>>> - bitbake.py: Enable environment generation without recipe,
> >>>>
> >>>> removes Buildhistory class.
> >>>>
> >>>> - buildhistory.py: Add buildhistory class that generates initial
> >>>>
> >>>> buildhistory revision and keeps track of build revisions
> >>>> for
> >>>> generate diff when build is successful.
> >>>>
> >>>> - upgradehelper.py: Add buildhistory steps for upgrade, add
> >>>>
> >>>> support for detect when buildhistory is enabled loading the
> >>>> environment without recipe.
> >>>
> >>> So this looks OK, but I can't tell if it ensures you get the difference
> >>> between successful builds rather than the difference to the last build -
> >>> we need to ensure that a failed build of a recipe in between isn't
> >>> considered as part of the comparison. In practice I guess this would
> >>> mean
> >>> storing the buildhistory revision of the last successful build for each
> >>> recipe.
> >>
> >> I only made a diff if the build success in all machines this guarantee
> >> that only stores good information.
> >
> > OK, I think I'm following now.
> >
> > One other thing - I'm not particularly keen on the BB_ENV_EXTRAWHITE thing
> > - can you perhaps query the build system for the path it's using instead?
> I modified this env var to pass BUILDHISTORY_DIR variable per recipe
> this enables
> to save buildhistory into AUH workdir
> (build/upgrade-helper/work/recipe/buildhistory).
Yes, I understand that, I just think using BB_ENV_EXTRAWHITE to pass this in
is a horrible hack and I'd rather not do it that way.
Cheers,
Paul
--
Paul Eggleton
Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 18+ messages in thread