[[AUH] PATCHv2 00/13] Improvements and small fixes

[[AUH] PATCHv2 00/13] Improvements and small fixes

[Aníbal Limón]

This changeset fixes comments made by Paul Eggleton and adds 3 new commits
the last ones related to send email improvements.

The changes can be found at,

http://git.yoctoproject.org/cgit/cgit.cgi/auto-upgrade-helper/log/?h=devel

Aníbal Limón (13):
  upgradehelper.py: Add step for build gcc-runtime at init.
  upgradehelper.py: Fix bug when buildhistory isn't enabled
  recipe.py: Improvements and refactor of fetch method.
  buildhistory.py: Don't modify BB_ENV_EXTRAWHITE before init.
  upgradehelper.py: Add sanity test for ensure that git is configured.
  buildhistory.py: Add missing warning import. git.py: Fix last_commit
    method.
  upgradehelper.py: Improve work directory structure
  upgradehelper.py: Add support for do recipe upgrades based on builddep
  upgradehelper.py: Add support for preserve statistics into work
    directory.
  requirements.txt: Add file for now only with GitPython
  upgradehelper.py: Only send email to Maintainer when upgrade succeed.
  upgradehelper.py: Fix indent due to previous commit.
  upgradehelper.py: Add support for preserve emails.

 buildhistory.py  |   8 +-
 git.py           |   2 +-
 recipe.py        |  58 ++++++-----
 requirements.txt |   1 +
 upgradehelper.py | 297 ++++++++++++++++++++++++++++++++++++-------------------
 5 files changed, 237 insertions(+), 129 deletions(-)
 create mode 100644 requirements.txt

-- 
2.1.4

[[AUH] PATCHv2 01/13] upgradehelper.py: Add step for build gcc-runtime at init.

[Aníbal Limón]

When try to upgrade a recipe the first recipe in the list
takes too much time because gcc-runtime build is needed.

So add previous step to build gcc-runtime for every machine.

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
 upgradehelper.py | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/upgradehelper.py b/upgradehelper.py
index b5a8abd..bb19b65 100755
--- a/upgradehelper.py
+++ b/upgradehelper.py
@@ -467,6 +467,11 @@ class Updater(object):
             W("No recipes attempted, not sending status mail!")
 
     def run(self, package_list=None):
+        I(" Building gcc runtimes ...")
+        for machine in self.machines:
+            I("  building gcc runtime for %s" % machine)
+            self.bb.complete("gcc-runtime", machine)
+
         pkgs_to_upgrade = self._get_packages_to_upgrade(package_list)
 
         total_pkgs = len(pkgs_to_upgrade)
-- 
2.1.4

[[AUH] PATCHv2 02/13] upgradehelper.py: Fix bug when buildhistory isn't enabled

[Aníbal Limón]

Don't use buildhistory object if buildhistory_enabled is False
because cause undefined variable error.

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
 upgradehelper.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/upgradehelper.py b/upgradehelper.py
index bb19b65..9b321f6 100755
--- a/upgradehelper.py
+++ b/upgradehelper.py
@@ -310,7 +310,7 @@ class Updater(object):
         for machine in self.machines:
             I(" %s: compiling for %s ..." % (self.pn, machine))
             self.recipe.compile(machine)
-            if self.buildhistory is not None:
+            if self.buildhistory_enabled == True:
                 self.buildhistory.add()
 
     def _buildhistory_diff(self):
-- 
2.1.4

[[AUH] PATCHv2 03/13] recipe.py: Improvements and refactor of fetch method.

[Aníbal Limón]

Make more clear fetch method changing recursive manner to
loop and split logic into try fetch and suffix change.

Now when fails only retry changing recipe suffix when recipe
isn't git based because does not make sense suffixes in SCM's.

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
 recipe.py | 58 +++++++++++++++++++++++++++++++++-------------------------
 1 file changed, 33 insertions(+), 25 deletions(-)

diff --git a/recipe.py b/recipe.py
index 6fd8f24..64e6e14 100644
--- a/recipe.py
+++ b/recipe.py
@@ -55,7 +55,6 @@ class Recipe(object):
             "tar.gz", "tgz", "zip", "tar.bz2", "tar.xz", "tar.lz4", "bz2",
             "lz4", "orig.tar.gz", "src.tar.gz", "src.rpm", "src.tgz",
             "svnr\d+.tar.bz2", "stable.tar.gz", "src.rpm"]
-        self.suffix_index = 0
         self.old_env = None
 
         self.commit_msg = self.env['PN'] + ": upgrade to " + self.new_ver + "\n\n"
@@ -540,33 +539,42 @@ class Recipe(object):
         self.bb.unpack(self.env['PN'])
 
     def fetch(self):
-        try:
-            self.bb.fetch(self.env['PN'])
-        except Error as e:
-            machine, failed_recipes = self._get_failed_recipes(e.stdout)
-            if not self.env['PN'] in failed_recipes:
-                raise Error("unknown error occured during fetch")
-
-            fetch_log = failed_recipes[self.env['PN']][1]
-            if self.suffix_index < len(self.suffixes) and self._is_uri_failure(fetch_log):
-                I(" Trying new SRC_URI suffix: %s ..." % self.suffixes[self.suffix_index])
-                self._change_source_suffix(self.suffixes[self.suffix_index])
-                self.suffix_index += 1
-                self.fetch()
-            
-            if not self._is_uri_failure(fetch_log):
-                if not self.checksums_changed:
+        from gitrecipe import GitRecipe
+
+        def _try_fetch():
+            try:
+                self.bb.fetch(self.env['PN'])
+                return
+            except Error as e:
+                machine, failed_recipes = self._get_failed_recipes(e.stdout)
+                if not self.env['PN'] in failed_recipes:
+                    raise Error("Unknown error occured during fetch",
+                            stdout = e.stdout, stderr = e.stderr)
+
+                fetch_log = failed_recipes[self.env['PN']][1]
+
+                if not self._is_uri_failure(fetch_log) and not \
+                        self.checksums_changed:
                     self._change_recipe_checksums(fetch_log)
-                    return
-                else:
-                    raise FetchError()
+                    self.checksums_changed = True
+                    return True
 
-                if self.recipes_renamed and not self.checksums_changed:
-                    raise Error("fetch succeeded without changing checksums")
+                return False
+
+        succeed = _try_fetch()
+        if not succeed and not isinstance(self, GitRecipe):
+            for sfx in self.suffixes:
+                I(" Trying new SRC_URI suffix: %s ..." % sfx)
+                self._change_source_suffix(sfx)
+
+                succeed = _try_fetch()
+                if succeed:
+                    break
 
-            elif self.suffix_index == len(self.suffixes):
-                # Every suffix tried without success
-                raise FetchError()                
+        if not succeed:
+            raise Error("Can't built a valid SRC_URI")
+        elif self.recipes_renamed and not self.checksums_changed:
+            raise Error("Fetch succeeded without changing checksums")
 
     def cleanall(self):
         self.bb.cleanall(self.env['PN'])
-- 
2.1.4

[[AUH] PATCHv2 04/13] buildhistory.py: Don't modify BB_ENV_EXTRAWHITE before init.

[Aníbal Limón]

Move code for add BUILDHISTORY_DIR to __init__ method because
it fails when import in early step.

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
 buildhistory.py | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/buildhistory.py b/buildhistory.py
index 1732f23..59566c8 100644
--- a/buildhistory.py
+++ b/buildhistory.py
@@ -30,9 +30,6 @@ from errors import *
 from bitbake import *
 from git import Git
 
-os.environ['BB_ENV_EXTRAWHITE'] = os.environ['BB_ENV_EXTRAWHITE'] + \
-                                    " BUILDHISTORY_DIR"
-
 class BuildHistory(object):
     def __init__(self, bb, pn, workdir):
         self.bb = bb
@@ -46,6 +43,8 @@ class BuildHistory(object):
 
         self.git = Git(self.buildhistory_dir)
 
+        os.environ['BB_ENV_EXTRAWHITE'] = os.environ['BB_ENV_EXTRAWHITE'] + \
+                                    " BUILDHISTORY_DIR"
         os.environ["BUILDHISTORY_DIR"] = self.buildhistory_dir
 
     def init(self, machines):
-- 
2.1.4

[[AUH] PATCHv2 05/13] upgradehelper.py: Add sanity test for ensure that git is configured.

[Aníbal Limón]

[YOCTO #8390]

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
 upgradehelper.py | 16 +++++++++++-----
 1 file changed, 11 insertions(+), 5 deletions(-)

diff --git a/upgradehelper.py b/upgradehelper.py
index 9b321f6..195d8c9 100755
--- a/upgradehelper.py
+++ b/upgradehelper.py
@@ -28,6 +28,7 @@
 
 import argparse
 import os
+import subprocess
 import logging as log
 from logging import debug as D
 from logging import info as I
@@ -734,17 +735,22 @@ if __name__ == "__main__":
     global settings
     global maintainer_override
 
+    if not os.getenv('BUILDDIR', False):
+        E(" You must source oe-init-build-env before running this script!\n")
+        exit(1)
+
+    devnull = open(os.devnull, 'wb')
+    if subprocess.call(["git", "config", "user.name"], stdout=devnull,stderr=devnull) or \
+        subprocess.call(["git", "config", "user.email"], stdout=devnull, stderr=devnull):
+        E(" Git isn't configured please configure user name and email\n")
+        exit(1)
+
     signal.signal(signal.SIGINT, close_child_processes)
 
     debug_levels = [log.CRITICAL, log.ERROR, log.WARNING, log.INFO, log.DEBUG]
     args = parse_cmdline()
     log.basicConfig(format='%(levelname)s:%(message)s',
                     level=debug_levels[args.debug_level - 1])
-
-    if not os.getenv('BUILDDIR', False):
-        E(" You must source oe-init-build-env before running this script!\n")
-        exit(1)
-
     settings, maintainer_override = parse_config_file(args.config_file)
 
     if args.recipe == "all":
-- 
2.1.4

[[AUH] PATCHv2 06/13] buildhistory.py: Add missing warning import. git.py: Fix last_commit method.

[Aníbal Limón]

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
 buildhistory.py | 3 ++-
 git.py          | 2 +-
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/buildhistory.py b/buildhistory.py
index 59566c8..842efc3 100644
--- a/buildhistory.py
+++ b/buildhistory.py
@@ -20,8 +20,9 @@
 
 import os
 import logging as log
-from logging import info as I
 from logging import debug as D
+from logging import info as I
+from logging import warning as W
 from logging import error as E
 from logging import critical as C
 import sys
diff --git a/git.py b/git.py
index 3eebac3..9661364 100644
--- a/git.py
+++ b/git.py
@@ -88,7 +88,7 @@ class Git(object):
         return self._cmd("clean -fd")
 
     def last_commit(self, branch_name):
-        return self._cmd("log --pretty=format:\"%H\" -1" + branch_name)
+        return self._cmd("log --pretty=format:\"%H\" -1 " + branch_name)
 
     def ls_remote(self, repo_url=None, options=None, refs=None):
         cmd = "ls-remote"
-- 
2.1.4

[[AUH] PATCHv2 07/13] upgradehelper.py: Improve work directory structure

[Aníbal Limón]

Add work directory with datetime suffix to preserve recipe upgrades.

Create tree folders all, succeed and failed,
	all	- Contains all the recipes work
	succeed	- Contains a symlinks to the recipes that successful
		  upgrade
	failed	- Contains a symlinks to the recipes that fail
		  upgrade

This helps to navigate over patches that have a successful upgrade.

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
 upgradehelper.py | 24 +++++++++++++++++-------
 1 file changed, 17 insertions(+), 7 deletions(-)

diff --git a/upgradehelper.py b/upgradehelper.py
index 195d8c9..ae8fc48 100755
--- a/upgradehelper.py
+++ b/upgradehelper.py
@@ -132,9 +132,16 @@ class Updater(object):
         if not os.path.exists(self.uh_dir):
             os.mkdir(self.uh_dir)
 
-        self.uh_work_dir = os.path.join(self.uh_dir, "work")
-        if not os.path.exists(self.uh_work_dir):
-            os.mkdir(self.uh_work_dir)
+        self.uh_work_dir = os.path.join(self.uh_dir, "work-%s" % \
+                datetime.now().strftime("%Y%m%d%H%M"))
+        os.mkdir(self.uh_work_dir)
+
+        self.uh_recipes_all_dir = os.path.join(self.uh_work_dir, "all")
+        os.mkdir(self.uh_recipes_all_dir)
+        self.uh_recipes_succeed_dir = os.path.join(self.uh_work_dir, "succeed")
+        os.mkdir(self.uh_recipes_succeed_dir)
+        self.uh_recipes_failed_dir = os.path.join(self.uh_work_dir, "failed")
+        os.mkdir(self.uh_recipes_failed_dir)
 
         self.bb = Bitbake(get_build_dir())
         self.git = None
@@ -223,10 +230,7 @@ class Updater(object):
         self.env = self._get_env(self.pn)
 
     def _create_workdir(self):
-        self.workdir = os.path.join(self.uh_work_dir, self.pn)
-
-        if os.path.exists(self.workdir):
-            shutil.rmtree(self.workdir)
+        self.workdir = os.path.join(self.uh_recipes_all_dir, self.pn)
         os.mkdir(self.workdir)
 
     def _detect_repo(self):
@@ -490,6 +494,9 @@ class Updater(object):
                         I(" %s: %s" % (self.pn, msg))
                     step()
 
+                os.symlink(self.workdir, os.path.join( \
+                    self.uh_recipes_succeed_dir, self.pn))
+
                 I(" %s: Upgrade SUCCESSFUL! Please test!" % self.pn)
             except Exception as e:
                 if isinstance(e, UpgradeNotNeededError):
@@ -511,6 +518,9 @@ class Updater(object):
 
                 error = e
 
+                os.symlink(self.workdir, os.path.join( \
+                    self.uh_recipes_failed_dir, self.pn))
+
             self._commit_changes()
 
             self.pkg_upgrade_handler(error)
-- 
2.1.4

[[AUH] PATCHv2 08/13] upgradehelper.py: Add support for do recipe upgrades based on builddep

[Aníbal Limón]

Now upgradehelper runs recipe upgrades based on build dependencies, the
dependency information is taken of bitbake dependency graph.

So if recipe a depends in build time on recipe b and c these recipes
b and c are first tried to upgrade then a.

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
 upgradehelper.py | 114 +++++++++++++++++++++++++++++++++++++++++--------------
 1 file changed, 86 insertions(+), 28 deletions(-)

diff --git a/upgradehelper.py b/upgradehelper.py
index ae8fc48..5bb066a 100755
--- a/upgradehelper.py
+++ b/upgradehelper.py
@@ -91,7 +91,6 @@ def parse_cmdline():
 def get_build_dir():
     return os.getenv('BUILDDIR')
 
-
 def parse_config_file(config_file):
     settings = dict()
     maintainer_override = dict()
@@ -435,26 +434,6 @@ class Updater(object):
             I(" %s: %s" % (self.pn, e.stdout))
             raise e
 
-    def _order_list(self, package_list):
-        try:
-            self.bb.dependency_graph(' '.join(p[0] for p in package_list))
-        except Error as e:
-            multiple_providers = False
-            for l in e.stdout.split('\n'):
-                if l.find("ERROR: Multiple .bb files are due to be built which each provide") == 0:
-                    multiple_providers = True
-
-            if not multiple_providers:
-                raise e
-
-        dep_file = os.path.join(get_build_dir(), "pn-buildlist")
-        ordered_list = []
-        with open(dep_file) as deps:
-            for d in deps:
-                ordered_list.extend(p for p in package_list if p[0] == d.strip())
-
-        return ordered_list
-
     def send_status_mail(self):
         if "status_recipients" not in settings:
             E("Could not send status email, no recipients set!")
@@ -471,16 +450,100 @@ class Updater(object):
         else:
             W("No recipes attempted, not sending status mail!")
 
+
+    def _order_pkgs_to_upgrade(self, pkgs_to_upgrade):
+        def _get_pn_dep_dic(pn_list, dependency_file): 
+            import re
+
+            pn_dep_dic = {}
+
+            with open(dependency_file) as dep:
+                data = dep.read()
+                dep.close()
+
+                for line in data.split('\n'):
+                    m = re.search('^"(.*)" -> "(.*)"$', line)
+                    if not m:
+                        continue
+
+                    pn = m.group(1)
+                    pn_dep = m.group(2)
+                    if pn == pn_dep:
+                        continue
+
+                    if pn in pn_list:
+                        if pn_dep in pn_list:
+                            if pn in pn_dep_dic.keys():
+                                pn_dep_dic[pn].append(pn_dep)
+                            else:
+                                pn_dep_dic[pn] = [pn_dep]
+                        elif not pn in pn_dep_dic.keys():
+                            pn_dep_dic[pn] = []
+
+            return pn_dep_dic
+
+        def _dep_resolve(graph, node, resolved, seen):
+            seen.append(node)
+
+            for edge in graph[node]:
+                if edge not in resolved:
+                    if edge in seen:
+                        raise RuntimeError("Packages %s and %s have " \
+                                "a circular dependency." \
+                                % (node, edge))
+                    _dep_resolve(graph, edge, resolved, seen)
+
+            resolved.append(node)
+
+
+        pn_list = []
+        for pn, new_ver, maintainer in pkgs_to_upgrade:
+            pn_list.append(pn)
+
+        try:
+           self.bb.dependency_graph(' '.join(pn_list))
+        except Error as e:
+            multiple_providers = False
+            for l in e.stdout.split('\n'):
+                if l.find("ERROR: Multiple .bb files are due to be built which each provide") == 0:
+                    multiple_providers = True
+            if not multiple_providers:
+                raise e
+
+        dependency_file = os.path.join(get_build_dir(), "pn-depends.dot")
+
+        pkgs_to_upgrade_ordered = []
+        pn_list_ordered = []
+
+        pn_dep_dic = _get_pn_dep_dic(pn_list, dependency_file)
+        if pn_dep_dic:
+            root = "__root_node__"
+            pn_dep_dic[root] = pn_dep_dic.keys()
+            _dep_resolve(pn_dep_dic, root, pn_list_ordered, [])
+            pn_list_ordered.remove(root)
+
+        for pn_ordered in pn_list_ordered:
+            for pn, new_ver, maintainer in pkgs_to_upgrade:
+                if pn == pn_ordered: 
+                    pkgs_to_upgrade_ordered.append([pn, new_ver, maintainer])
+
+        return pkgs_to_upgrade_ordered
+
     def run(self, package_list=None):
         I(" Building gcc runtimes ...")
         for machine in self.machines:
             I("  building gcc runtime for %s" % machine)
             self.bb.complete("gcc-runtime", machine)
 
-        pkgs_to_upgrade = self._get_packages_to_upgrade(package_list)
-
+        pkgs_to_upgrade = self._order_pkgs_to_upgrade(
+                self._get_packages_to_upgrade(package_list))
         total_pkgs = len(pkgs_to_upgrade)
 
+        I(" ########### The list of recipes to be upgraded #############")
+        for p, v, m in pkgs_to_upgrade:
+            I(" %s, %s, %s" % (p, v, m))
+        I(" ############################################################")
+
         attempted_pkgs = 0
         for self.pn, self.new_ver, self.maintainer in pkgs_to_upgrade:
             error = None
@@ -708,11 +771,6 @@ class UniverseUpdater(Updater):
             last_check.write(current_date + "," + cur_master_commit + "," +
                              last_checkpkg_file)
 
-        I(" ########### The list of recipes to be upgraded ############")
-        for p, v, m in pkgs_list:
-            I(" %s, %s, %s" % (p, v, m))
-        I(" ############################################################")
-
         return pkgs_list
 
     def _update_history(self, pn, new_ver, maintainer, upgrade_status):
-- 
2.1.4

[[AUH] PATCHv2 09/13] upgradehelper.py: Add support for preserve statistics into work directory.

[Aníbal Limón]

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
 upgradehelper.py | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/upgradehelper.py b/upgradehelper.py
index 5bb066a..a3ca373 100755
--- a/upgradehelper.py
+++ b/upgradehelper.py
@@ -591,7 +591,16 @@ class Updater(object):
             self.statistics.update(self.pn, self.new_ver, self.maintainer, error)
 
         if (attempted_pkgs > 1):
-            I("%s" % self.statistics.pkg_stats())
+            statistics_summary = self.statistics.pkg_stats() + \
+                    self.statistics.maintainer_stats()
+
+            statistics_file = os.path.join(self.uh_work_dir,
+                    "statistics_summary")
+            with open(statistics_file, "w+") as f:
+                f.write(statistics_summary)
+
+            I("%s" % statistics_summary)
+
             if self.send_email:
                 self.send_status_mail()
 
-- 
2.1.4

[[AUH] PATCHv2 10/13] requirements.txt: Add file for now only with GitPython

[Aníbal Limón]

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
 requirements.txt | 1 +
 1 file changed, 1 insertion(+)
 create mode 100644 requirements.txt

diff --git a/requirements.txt b/requirements.txt
new file mode 100644
index 0000000..64b1ada
--- /dev/null
+++ b/requirements.txt
@@ -0,0 +1 @@
+GitPython
-- 
2.1.4

[[AUH] PATCHv2 11/13] upgradehelper.py: Only send email to Maintainer when upgrade succeed.

[Aníbal Limón]

Change the policy for send emails to only send when upgrade succeed
to avoid waste time of the Maintainers when upgrade process fails
because the failure patch generated isn't useful.

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
 upgradehelper.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/upgradehelper.py b/upgradehelper.py
index a3ca373..c2bf506 100755
--- a/upgradehelper.py
+++ b/upgradehelper.py
@@ -353,8 +353,6 @@ class Updater(object):
             self.git.reset_hard(1)
             self.git.clean_untracked()
 
-        # only send email to Maintainer when patch file exist
-        if self.send_email and self.patch_file:
             mail_header = \
                 "Hello,\n\nYou are receiving this email because you are the maintainer\n" \
                 "of *%s* recipe and this is to let you know that the automatic attempt\n" \
@@ -414,6 +412,8 @@ class Updater(object):
                 if os.path.isfile(attachment_fullpath):
                     attachments.append(attachment_fullpath)
 
+        # Only send email to Maintainer when recipe upgrade succeed.
+        if self.send_email and not err:
             self.email_handler.send_email(to_addr, subject, msg_body, attachments, cc_addr=cc_addr)
 
     def _commit_changes(self):
-- 
2.1.4

[[AUH] PATCHv2 12/13] upgradehelper.py: Fix indent due to previous commit.

[Aníbal Limón]

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
 upgradehelper.py | 109 +++++++++++++++++++++++++++----------------------------
 1 file changed, 53 insertions(+), 56 deletions(-)

diff --git a/upgradehelper.py b/upgradehelper.py
index c2bf506..f7ce425 100755
--- a/upgradehelper.py
+++ b/upgradehelper.py
@@ -353,64 +353,61 @@ class Updater(object):
             self.git.reset_hard(1)
             self.git.clean_untracked()
 
-            mail_header = \
-                "Hello,\n\nYou are receiving this email because you are the maintainer\n" \
-                "of *%s* recipe and this is to let you know that the automatic attempt\n" \
-                "to upgrade the recipe to *%s* has %s.\n\n"
-
-            license_change_info = \
-                "*LICENSE CHANGED* please review the %s file and update the LICENSE\n" \
-                "variable in the recipe if is needed.\n\n"
-
-            next_steps_info = \
-                "The recipe has been successfully compiled for machines %s.\n\n" \
-                "Next steps:\n" \
-                "    - apply the patch: git am %s\n" \
-                "    - check that required patches have not been removed from the recipe\n" \
-                "    - compile an image that contains the package\n" \
-                "    - perform some basic sanity tests\n" \
-                "    - amend the patch and sign it off: git commit -s --reset-author --amend\n" \
-                "    - send it to the list\n\n" \
-
-            mail_footer = \
-                "Attached are the patch, license diff (if change) and bitbake log.\n" \
-                "Any problem please contact Anibal Limon <anibal.limon@intel.com>.\n\n" \
-                "Regards,\nThe Upgrade Helper"
-
-            if self.maintainer in maintainer_override:
-                to_addr = maintainer_override[self.maintainer]
-            else:
-                to_addr = self.maintainer
-
-            cc_addr = None
-            if "status_recipients" in settings:
-                cc_addr = settings["status_recipients"].split()
-
-            subject = "[AUH] " + self.pn + ": upgrading to " + self.new_ver
-            if err is None:
-                subject += " SUCCEEDED"
-            else:
-                subject += " FAILED"
-
-            msg_body = mail_header % (self.pn, self.new_ver,
-                    self._get_status_msg(err))
-
-            license_diff_fn = self.recipe.get_license_diff_file_name()
-            if license_diff_fn:
-                msg_body += license_change_info % license_diff_fn
-
-            if err is None:
-                msg_body += next_steps_info % (', '.join(self.machines),
-                        os.path.basename(self.patch_file))
+        mail_header = \
+            "Hello,\n\nYou are receiving this email because you are the maintainer\n" \
+            "of *%s* recipe and this is to let you know that the automatic attempt\n" \
+            "to upgrade the recipe to *%s* has %s.\n\n"
+
+        license_change_info = \
+            "*LICENSE CHANGED* please review the %s file and update the LICENSE\n" \
+            "variable in the recipe if is needed.\n\n"
+
+        next_steps_info = \
+            "The recipe has been successfully compiled for machines %s.\n\n" \
+            "Next steps:\n" \
+            "    - apply the patch: git am %s\n" \
+            "    - check that required patches have not been removed from the recipe\n" \
+            "    - compile an image that contains the package\n" \
+            "    - perform some basic sanity tests\n" \
+            "    - amend the patch and sign it off: git commit -s --reset-author --amend\n" \
+            "    - send it to the list\n\n" \
+
+        mail_footer = \
+            "Attached are the patch, license diff (if change) and bitbake log.\n" \
+            "Any problem please contact Anibal Limon <anibal.limon@intel.com>.\n\n" \
+            "Regards,\nThe Upgrade Helper"
+
+        if self.maintainer in maintainer_override:
+            to_addr = maintainer_override[self.maintainer]
+        else:
+            to_addr = self.maintainer
 
-            msg_body += mail_footer
+        cc_addr = None
+        if "status_recipients" in settings:
+            cc_addr = settings["status_recipients"].split()
 
-            # Add possible attachments to email
-            attachments = []
-            for attachment in os.listdir(self.workdir):
-                attachment_fullpath = os.path.join(self.workdir, attachment)
-                if os.path.isfile(attachment_fullpath):
-                    attachments.append(attachment_fullpath)
+        subject = "[AUH] " + self.pn + ": upgrading to " + self.new_ver
+        if not err:
+            subject += " SUCCEEDED"
+        else:
+            subject += " FAILED"
+        msg_body = mail_header % (self.pn, self.new_ver,
+                self._get_status_msg(err))
+        license_diff_fn = self.recipe.get_license_diff_file_name()
+        if license_diff_fn:
+            msg_body += license_change_info % license_diff_fn
+        if not err:
+            msg_body += next_steps_info % (', '.join(self.machines),
+                    os.path.basename(self.patch_file))
+
+        msg_body += mail_footer
+
+        # Add possible attachments to email
+        attachments = []
+        for attachment in os.listdir(self.workdir):
+            attachment_fullpath = os.path.join(self.workdir, attachment)
+            if os.path.isfile(attachment_fullpath):
+                attachments.append(attachment_fullpath)
 
         # Only send email to Maintainer when recipe upgrade succeed.
         if self.send_email and not err:
-- 
2.1.4

[[AUH] PATCHv2 13/13] upgradehelper.py: Add support for preserve emails.

[Aníbal Limón]

This support is useful for review purposes after upgrade process
and these emails are saved into recipe work directory.

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
---
 upgradehelper.py | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/upgradehelper.py b/upgradehelper.py
index f7ce425..5eb9f5a 100755
--- a/upgradehelper.py
+++ b/upgradehelper.py
@@ -413,6 +413,20 @@ class Updater(object):
         if self.send_email and not err:
             self.email_handler.send_email(to_addr, subject, msg_body, attachments, cc_addr=cc_addr)
 
+        # Preserve email for review purposes.
+        email_file = os.path.join(self.uh_work_dir,
+                    "email_summary")
+        with open(email_file, "w+") as f:
+            f.write("To: %s\n" % to_addr)
+            if isinstance(cc_addr, list):
+                f.write("To: %s\n" % ' '.join(to_addr))
+            else:
+                f.write("Cc: %s\n" % to_addr)
+
+            f.write("Subject: %s\n" % subject)
+            f.write("Attachments: %s\n" % ' '.join(attachments))
+            f.write("\n%s\n" % msg_body)
+
     def _commit_changes(self):
         try:
             self.patch_file = None
-- 
2.1.4