[PATCH 00/14] Toaster patchset

[PATCH 00/14] Toaster patchset

[Alex DAMIAN]

From: Alexandru DAMIAN <alexandru.damian@intel.com>

Hello,

This is a toaster patchset that fixes various issues in the interface,
and in build process.

Can you please pull ?

Thanks,
Alex

The following changes since commit 4b07195405b124ff39703e1ddc226dee420fed22:

  toaster: Fix the orderkey to match the column (2015-03-16 17:43:51 +0000)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib adamian/20150317-submission-bb
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=adamian/20150317-submission-bb

Alexandru DAMIAN (7):
  toasterui: improve info in the toaster_ui.log
  toaster: targets page performance improvement
  toastergui: fix machines page search
  toastergui: builds and projects outcome filter
  toaster: do not throw exception on misconfigured data
  toasterui: fix error message parameter
  toastergui: recipe and layer identification

Belen Barros Pena (1):
  toaster: display bitbake output after process start

Dave Lerner (1):
  toaster: leave cancel enabled for release change

David Reyna (3):
  toaster: Incorrect breadcrumb behaviour in the project page
  toaster: display machine and project values in input fields
  toaster: insure IMAGE_INSTALL_append values have a space prefix

Michael Wood (2):
  toaster: Add cancel build to latest build section
  toaster: tasks trigger the select related tables before ordering

 lib/bb/ui/buildinfohelper.py                       | 31 ++++++++++++----
 lib/bb/ui/toasterui.py                             |  8 +++++
 lib/toaster/bldcontrol/localhostbecontroller.py    | 14 +++++---
 lib/toaster/orm/models.py                          | 14 ++++----
 lib/toaster/toastergui/static/js/libtoaster.js     | 28 +++++++++++++++
 .../toastergui/templates/baseprojectpage.html      |  6 ++++
 .../toastergui/templates/managed_mrb_section.html  | 42 +++++++++-------------
 lib/toaster/toastergui/templates/mrb_section.html  | 39 --------------------
 lib/toaster/toastergui/templates/project.html      |  6 ++--
 lib/toaster/toastergui/templates/projectconf.html  | 18 ++++++----
 lib/toaster/toastergui/templates/projects.html     |  2 +-
 lib/toaster/toastergui/views.py                    | 35 ++++++++++++------
 12 files changed, 138 insertions(+), 105 deletions(-)

-- 
1.9.1

[PATCH 01/14] toasterui: improve info in the toaster_ui.log

[Alex DAMIAN]

From: Alexandru DAMIAN <alexandru.damian@intel.com>

We improve logging and signalling of errors in the
toaster_ui.log to facilitate debugging on remote systems.

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
---
 lib/bb/ui/buildinfohelper.py | 12 ++++++++----
 lib/bb/ui/toasterui.py       |  8 ++++++++
 2 files changed, 16 insertions(+), 4 deletions(-)

diff --git a/lib/bb/ui/buildinfohelper.py b/lib/bb/ui/buildinfohelper.py
index cf8c444..b511abe 100644
--- a/lib/bb/ui/buildinfohelper.py
+++ b/lib/bb/ui/buildinfohelper.py
@@ -33,6 +33,7 @@ from toaster.orm.models import Task_Dependency, Package_Dependency
 from toaster.orm.models import Recipe_Dependency
 from bb.msg import BBLogFormatter as format
 from django.db import models
+from pprint import pformat
 import logging
 
 from django.db import transaction, connection
@@ -282,7 +283,6 @@ class ORMWrapper(object):
             # we might have a race condition here, as the project layers may change between the build trigger and the actual build execution
             # but we can only match on the layer name, so the worst thing can happen is a mis-identification of the layer, not a total failure
 
-            from pprint import pformat
             # note that this is different
             buildrequest = BuildRequest.objects.get(pk = br_id)
             for brl in buildrequest.brlayer_set.all():
@@ -467,7 +467,7 @@ class ORMWrapper(object):
             Package_Dependency.objects.bulk_create(packagedeps_objs)
 
         if (len(errormsg) > 0):
-            raise Exception(errormsg)
+            logger.warn("buildinfohelper: target_package_info could not identify recipes: \n%s" % errormsg)
 
     def save_target_image_file_information(self, target_obj, file_name, file_size):
         target_image_file = Target_Image_File.objects.create( target = target_obj,
@@ -885,9 +885,13 @@ class BuildInfoHelper(object):
             assert localfilepath.startswith("/")
 
             recipe_information = self._get_recipe_information_from_taskfile(taskfile)
-            recipe_object = Recipe.objects.get(layer_version = recipe_information['layer_version'],
+            try:
+                recipe_object = Recipe.objects.get(layer_version = recipe_information['layer_version'],
                             file_path__endswith = recipe_information['file_path'],
                             name = recipename)
+            except Recipe.DoesNotExist:
+                logger.error("Could not find recipe for recipe_information %s name %s" % (pformat(recipe_information), name))
+                raise
 
             task_information = {}
             task_information['build'] = self.internal_state['build']
@@ -1127,7 +1131,7 @@ class BuildInfoHelper(object):
         Task_Dependency.objects.bulk_create(taskdeps_objects)
 
         if (len(errormsg) > 0):
-            raise Exception(errormsg)
+            logger.warn("buildinfohelper: dependency info not identify recipes: \n%s" % errormsg)
 
 
     def store_build_package_information(self, event):
diff --git a/lib/bb/ui/toasterui.py b/lib/bb/ui/toasterui.py
index df9f362..db119ce 100644
--- a/lib/bb/ui/toasterui.py
+++ b/lib/bb/ui/toasterui.py
@@ -327,9 +327,17 @@ def main(server, eventHandler, params ):
         except Exception as e:
             # print errors to log
             import traceback
+            from pprint import pformat
             exception_data = traceback.format_exc()
             logger.error("%s\n%s" % (e, exception_data))
 
+            exc_type, exc_value, tb = sys.exc_info()
+            if tb is not None:
+                curr = tb
+                while curr is not None:
+                    logger.warn("Error data dump %s\n%s\n" % (traceback.format_tb(curr,1), pformat(curr.tb_frame.f_locals)))
+                    curr = curr.tb_next
+
             # save them to database, if possible; if it fails, we already logged to console.
             try:
                 buildinfohelper.store_log_exception("%s\n%s" % (str(e), exception_data))
-- 
1.9.1

[PATCH 02/14] toaster: targets page performance improvement

[Alex DAMIAN]

From: Alexandru DAMIAN <alexandru.damian@intel.com>

Yet another performance improvement, this time by forcing
two-step evaluation instead of using subqueries to select data.

This avoid using full-table lookups on un-indexed temporary
tables.

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
---
 lib/toaster/toastergui/views.py | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/lib/toaster/toastergui/views.py b/lib/toaster/toastergui/views.py
index fe016cc..439c068 100755
--- a/lib/toaster/toastergui/views.py
+++ b/lib/toaster/toastergui/views.py
@@ -2767,7 +2767,10 @@ if toastermain.settings.MANAGED:
         queryset_with_search = _get_queryset(Recipe, queryset_all, None, search_term, ordering_string, '-name')
 
         # get unique values for 'name', and select the maximum ID for each entry (the max id is the newest one)
-        queryset_with_search_maxids = queryset_with_search.values('name').distinct().annotate(max_id=Max('id')).values_list('max_id')
+
+        # force evaluation of the query here; to process the MAX/GROUP BY, a temporary table is used, on which indexing is very slow
+        # by forcing the evaluation here we also prime the caches
+        queryset_with_search_maxids = map(lambda i: i[0], list(queryset_with_search.values('name').distinct().annotate(max_id=Max('id')).values_list('max_id')))
 
         queryset_with_search = queryset_with_search.filter(id__in=queryset_with_search_maxids).select_related('layer_version', 'layer_version__layer', 'layer_version__up_branch', 'layer_source')
 
-- 
1.9.1

[PATCH 03/14] toaster: display bitbake output after process start

[Alex DAMIAN]

From: Belen Barros Pena <belen.barros.pena@intel.com>

If the bitbake server exists after the initial process start,
the only output displayed shows proper server start, with
no debug info.

This patch also brings the server log into the error message,
so the user has better info about what's going on.

Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
---
 lib/toaster/bldcontrol/localhostbecontroller.py | 14 +++++++++-----
 lib/toaster/orm/models.py                       | 12 +++++++-----
 2 files changed, 16 insertions(+), 10 deletions(-)

diff --git a/lib/toaster/bldcontrol/localhostbecontroller.py b/lib/toaster/bldcontrol/localhostbecontroller.py
index 374991b..1bd5143 100644
--- a/lib/toaster/bldcontrol/localhostbecontroller.py
+++ b/lib/toaster/bldcontrol/localhostbecontroller.py
@@ -82,6 +82,8 @@ class LocalhostBEController(BuildEnvironmentController):
         assert self.pokydirname and os.path.exists(self.pokydirname)
         self._createdirpath(self.be.builddir)
         self._shellcmd("bash -c \"source %s/oe-init-build-env %s\"" % (self.pokydirname, self.be.builddir))
+        # delete the templateconf.cfg; it may come from an unsupported layer configuration
+        os.remove(os.path.join(self.be.builddir, "conf/templateconf.cfg"))
 
 
     def writeConfFile(self, file_name, variable_list = None, raw = None):
@@ -115,9 +117,10 @@ class LocalhostBEController(BuildEnvironmentController):
                 f.seek(0, 2)    # jump to the end
                 toaster_ui_log_filelength = f.tell()
 
-        cmd = "bash -c \"source %s/oe-init-build-env %s && bitbake --read conf/toaster-pre.conf --postread conf/toaster.conf --server-only -t xmlrpc -B 0.0.0.0:0 >toaster_server.log && DATABASE_URL=%s BBSERVER=0.0.0.0:-1 daemon -d -i -D %s -o toaster_ui.log -- %s --observe-only -u toasterui &\"" % (self.pokydirname, self.be.builddir,
+        cmd = "bash -c \"source %s/oe-init-build-env %s 2>&1 >toaster_server.log && bitbake --read conf/toaster-pre.conf --postread conf/toaster.conf --server-only -t xmlrpc -B 0.0.0.0:0 2>&1 >toaster_server.log && DATABASE_URL=%s BBSERVER=0.0.0.0:-1 daemon -d -i -D %s -o toaster_ui.log -- %s --observe-only -u toasterui &\"" % (self.pokydirname, self.be.builddir,
                 self.dburl, self.be.builddir, own_bitbake)
         port = "-1"
+        logger.debug("localhostbecontroller: starting builder \n%s\n" % cmd)
         cmdoutput = self._shellcmd(cmd)
         for i in cmdoutput.split("\n"):
             if i.startswith("Bitbake server address"):
@@ -136,7 +139,7 @@ class LocalhostBEController(BuildEnvironmentController):
 
         retries = 0
         started = False
-        while not started and retries < 30:
+        while not started and retries < 10:
             started = _toaster_ui_started(toaster_ui_log_filepath, toaster_ui_log_filelength)
             import time
             logger.debug("localhostbecontroller: Waiting bitbake server to start")
@@ -144,7 +147,8 @@ class LocalhostBEController(BuildEnvironmentController):
             retries += 1
 
         if not started:
-            raise BuildSetupException("localhostbecontroller: Bitbake server did not start in 15 seconds, aborting (Error: '%s')" % (cmdoutput))
+            toaster_server_log = open(os.path.join(self.be.builddir, "toaster_server.log"), "r").read()
+            raise BuildSetupException("localhostbecontroller: Bitbake server did not start in 5 seconds, aborting (Error: '%s' '%s')" % (cmdoutput, toaster_server_log))
 
         logger.debug("localhostbecontroller: Started bitbake server")
 
@@ -180,7 +184,7 @@ class LocalhostBEController(BuildEnvironmentController):
         base = components[-2] if components[-1] == "git" else components[-1]
 
         if branch != "HEAD":
-            return os.path.join(self.be.sourcedir, "_%s_%s.toaster_cloned" % (base, branch))
+            return "_%s_%s.toaster_cloned" % (base, branch)
 
 
         # word of attention; this is a localhost-specific issue; only on the localhost we expect to have "HEAD" releases
@@ -239,7 +243,7 @@ class LocalhostBEController(BuildEnvironmentController):
 
         # 3. checkout the repositories
         for giturl, commit in gitrepos.keys():
-            localdirname = self.getGitCloneDirectory(giturl, commit)
+            localdirname = os.path.join(self.be.sourcedir, self.getGitCloneDirectory(giturl, commit))
             logger.debug("localhostbecontroller: giturl %s:%s checking out in current directory %s" % (giturl, commit, localdirname))
 
             # make sure our directory is a git repository
diff --git a/lib/toaster/orm/models.py b/lib/toaster/orm/models.py
index 2132f87..eea98b5 100644
--- a/lib/toaster/orm/models.py
+++ b/lib/toaster/orm/models.py
@@ -52,7 +52,7 @@ class ToasterSetting(models.Model):
     value = models.CharField(max_length=255)
 
     def __unicode__(self):
-        return "Setting %s" % self.name
+        return "Setting %s = " % (self.name, self.value)
 
 class ProjectManager(models.Manager):
     def create_project(self, name, release):
@@ -100,7 +100,7 @@ class Project(models.Model):
     objects     = ProjectManager()
 
     def __unicode__(self):
-        return "%s (%s, %s)" % (self.name, self.release, self.bitbake_version)
+        return "%s (Release %s, BBV %s)" % (self.name, self.release, self.bitbake_version)
 
     def get_current_machine_name(self):
         try:
@@ -896,7 +896,7 @@ class BitbakeVersion(models.Model):
     dirpath = models.CharField(max_length=255)
 
     def __unicode__(self):
-        return "%s (%s)" % (self.name, self.branch)
+        return "%s (Branch: %s)" % (self.name, self.branch)
 
 
 class Release(models.Model):
@@ -1055,8 +1055,10 @@ class Layer_Version(models.Model):
 
 #        raise Exception(candidate_layer_versions)
 
-        release_priorities = map(lambda x: (x.layer_source_id, x.priority), project.release.releaselayersourcepriority_set.all().order_by("-priority"))
+        release_priorities = {}
 
+        for ls_id, prio in map(lambda x: (x.layer_source_id, x.priority), project.release.releaselayersourcepriority_set.all().order_by("-priority")):
+            release_priorities[ls_id] = prio
 
         def _get_ls_priority(ls):
             # if there is no layer source, we have minus infinite priority, as we don't want this layer selected
@@ -1081,7 +1083,7 @@ class Layer_Version(models.Model):
         raise Exception("Cannot determine the vcs_reference for layer version %s" % vars(self))
 
     def __unicode__(self):
-        return  str(self.layer) + "(%s,%s)" % (self.get_vcs_reference(), self.build.project if self.build is not None else "None")
+        return "%d %s (VCS %s, Project %s)" % (self.pk, str(self.layer), self.get_vcs_reference(), self.build.project if self.build is not None else "No project")
 
     class Meta:
         unique_together = ("layer_source", "up_id")
-- 
1.9.1

[PATCH 04/14] toastergui: fix machines page search

[Alex DAMIAN]

From: Alexandru DAMIAN <alexandru.damian@intel.com>

This patch fixes the machine page search.

[YOCTO #7407]

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
---
 lib/toaster/toastergui/views.py | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/lib/toaster/toastergui/views.py b/lib/toaster/toastergui/views.py
index 439c068..fa34e56 100755
--- a/lib/toaster/toastergui/views.py
+++ b/lib/toaster/toastergui/views.py
@@ -2867,15 +2867,17 @@ if toastermain.settings.MANAGED:
         # for that object type. copypasta for all needed table searches
         (filter_string, search_term, ordering_string) = _search_tuple(request, Machine)
 
-        queryset_all = Machine.objects.all()
-        queryset_all = queryset_all.prefetch_related('layer_version')
-
         prj = Project.objects.get(pk = request.session['project_id'])
         compatible_layers = prj.compatible_layerversions()
 
+        queryset_all = Machine.objects.filter(layer_version__in=compatible_layers)
+        queryset_all = _get_queryset(Machine, queryset_all, None, search_term, ordering_string, 'name')
+
+        queryset_all = queryset_all.prefetch_related('layer_version')
+
+
         # Make sure we only show machines / layers which are compatible
         # with the current project
-        queryset_all = queryset_all.filter(layer_version__in=compatible_layers)
 
         project_layers = ProjectLayer.objects.filter(project_id=request.session['project_id']).values_list('layercommit',flat=True)
 
-- 
1.9.1

[PATCH 05/14] toastergui: builds and projects outcome filter

[Alex DAMIAN]

From: Alexandru DAMIAN <alexandru.damian@intel.com>

This patch fixes the Outcome filter in the all builds and
all projects pages in Build mode.

[YOCTO #7450]

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
---
 lib/toaster/toastergui/views.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/toaster/toastergui/views.py b/lib/toaster/toastergui/views.py
index fa34e56..e87c9f2 100755
--- a/lib/toaster/toastergui/views.py
+++ b/lib/toaster/toastergui/views.py
@@ -1924,8 +1924,8 @@ if toastermain.settings.MANAGED:
                      'filter' : {'class' : 'outcome',
                                  'label': 'Show:',
                                  'options' : [
-                                             ('Successful builds', 'state:' + str(BuildRequest.REQ_COMPLETED), queryset_all.filter(state=str(BuildRequest.REQ_COMPLETED)).count()),  # this is the field search expression
-                                             ('Failed builds', 'state:'+ str(BuildRequest.REQ_FAILED), queryset_all.filter(state=str(BuildRequest.REQ_FAILED)).count()),
+                                             ('Successful builds', 'build__outcome:' + str(Build.SUCCEEDED), queryset_all.filter(build__outcome = Build.SUCCEEDED).count()),  # this is the field search expression
+                                             ('Failed builds', 'build__outcome:NOT'+ str(Build.SUCCEEDED), queryset_all.exclude(build__outcome = Build.SUCCEEDED).count()),
                                              ]
                                 }
                     },
-- 
1.9.1

[PATCH 06/14] toaster: do not throw exception on misconfigured data

[Alex DAMIAN]

From: Alexandru DAMIAN <alexandru.damian@intel.com>

We do not want to throw an exception while handling exceptions
that may print information, so replace the exception throwing
with returning data.

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
---
 lib/toaster/orm/models.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/toaster/orm/models.py b/lib/toaster/orm/models.py
index eea98b5..625eb1e 100644
--- a/lib/toaster/orm/models.py
+++ b/lib/toaster/orm/models.py
@@ -1080,7 +1080,7 @@ class Layer_Version(models.Model):
             return self.branch
         if self.up_branch is not None:
             return self.up_branch.name
-        raise Exception("Cannot determine the vcs_reference for layer version %s" % vars(self))
+        return ("Cannot determine the vcs_reference for layer version %s" % vars(self))
 
     def __unicode__(self):
         return "%d %s (VCS %s, Project %s)" % (self.pk, str(self.layer), self.get_vcs_reference(), self.build.project if self.build is not None else "No project")
-- 
1.9.1

[PATCH 07/14] toasterui: fix error message parameter

[Alex DAMIAN]

From: Alexandru DAMIAN <alexandru.damian@intel.com>

Fix the error message parameter.

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
---
 lib/bb/ui/buildinfohelper.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/bb/ui/buildinfohelper.py b/lib/bb/ui/buildinfohelper.py
index b511abe..06a4490 100644
--- a/lib/bb/ui/buildinfohelper.py
+++ b/lib/bb/ui/buildinfohelper.py
@@ -890,7 +890,7 @@ class BuildInfoHelper(object):
                             file_path__endswith = recipe_information['file_path'],
                             name = recipename)
             except Recipe.DoesNotExist:
-                logger.error("Could not find recipe for recipe_information %s name %s" % (pformat(recipe_information), name))
+                logger.error("Could not find recipe for recipe_information %s name %s" % (pformat(recipe_information), recipename))
                 raise
 
             task_information = {}
-- 
1.9.1

[PATCH 08/14] toastergui: recipe and layer identification

[Alex DAMIAN]

From: Alexandru DAMIAN <alexandru.damian@intel.com>

This patch fixes the recipe and layer identification by path
when Toaster uses relative paths.

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
---
 lib/bb/ui/buildinfohelper.py | 19 ++++++++++++++++---
 1 file changed, 16 insertions(+), 3 deletions(-)

diff --git a/lib/bb/ui/buildinfohelper.py b/lib/bb/ui/buildinfohelper.py
index 06a4490..64bd94e 100644
--- a/lib/bb/ui/buildinfohelper.py
+++ b/lib/bb/ui/buildinfohelper.py
@@ -287,9 +287,13 @@ class ORMWrapper(object):
             buildrequest = BuildRequest.objects.get(pk = br_id)
             for brl in buildrequest.brlayer_set.all():
                 localdirname = os.path.join(bc.getGitCloneDirectory(brl.giturl, brl.commit), brl.dirpath)
-                logger.warn("Matched %s to BRlayer %s" % (pformat(layer_information["local_path"]), localdirname))
+                # we get a relative path, unless running in HEAD mode where the path is absolute
+                if not localdirname.startswith("/"):
+                    localdirname = os.path.join(bc.be.sourcedir, localdirname)
+                #logger.debug(1, "Localdirname %s lcal_path %s" % (localdirname, layer_information['local_path']))
                 if localdirname.startswith(layer_information['local_path']):
                     # we matched the BRLayer, but we need the layer_version that generated this BR; reverse of the Project.schedule_build()
+                    #logger.debug(1, "Matched %s to BRlayer %s" % (pformat(layer_information["local_path"]), localdirname))
                     for pl in buildrequest.project.projectlayer_set.filter(layercommit__layer__name = brl.name):
                         if pl.layercommit.layer.vcs_url == brl.giturl :
                             layer = pl.layercommit.layer
@@ -297,7 +301,7 @@ class ORMWrapper(object):
                             layer.save()
                             return layer
 
-            raise Exception("Unidentified layer %s" % pformat(layer_information))
+            raise NotExisting("Unidentified layer %s" % pformat(layer_information))
 
 
     def save_target_file_information(self, build_obj, target_obj, filedata):
@@ -703,6 +707,9 @@ class BuildInfoHelper(object):
             # Heuristics: we match the path to where the layers have been checked out
             for brl in sorted(BuildRequest.objects.get(pk = br_id).brlayer_set.all(), reverse = True, key = _slkey_managed):
                 localdirname = os.path.join(bc.getGitCloneDirectory(brl.giturl, brl.commit), brl.dirpath)
+                # we get a relative path, unless running in HEAD mode where the path is absolute
+                if not localdirname.startswith("/"):
+                    localdirname = os.path.join(bc.be.sourcedir, localdirname)
                 if path.startswith(localdirname):
                     #logger.warn("-- managed: matched path %s with layer %s " % (path, localdirname))
                     # we matched the BRLayer, but we need the layer_version that generated this br
@@ -775,7 +782,10 @@ class BuildInfoHelper(object):
         layerinfos = BuildInfoHelper._get_data_from_event(event)
         self.internal_state['lvs'] = {}
         for layer in layerinfos:
-            self.internal_state['lvs'][self.orm_wrapper.get_update_layer_object(layerinfos[layer], self.brbe)] = layerinfos[layer]['version']
+            try:
+                self.internal_state['lvs'][self.orm_wrapper.get_update_layer_object(layerinfos[layer], self.brbe)] = layerinfos[layer]['version']
+            except NotExisting as nee:
+                logger.warn("buildinfohelper: cannot identify layer exception:%s " % nee)
 
 
     def store_started_build(self, event):
@@ -886,6 +896,9 @@ class BuildInfoHelper(object):
 
             recipe_information = self._get_recipe_information_from_taskfile(taskfile)
             try:
+                if recipe_information['file_path'].startswith(recipe_information['layer_version'].layer.local_path):
+                    recipe_information['file_path'] = recipe_information['file_path'][len(recipe_information['layer_version'].layer.local_path):].lstrip("/")
+
                 recipe_object = Recipe.objects.get(layer_version = recipe_information['layer_version'],
                             file_path__endswith = recipe_information['file_path'],
                             name = recipename)
-- 
1.9.1

[PATCH 09/14] toaster: leave cancel enabled for release change

[Alex DAMIAN]

From: Dave Lerner <dave.lerner@windriver.com>

When you click the 'change' icon to edit the project release, the
"cancel" button, should always be enabled so that the user can
return to the 'not editable' state at any time.

[YOCTO #7151]

Signed-off-by: Dave Lerner <dave.lerner@windriver.com>
---
 lib/toaster/toastergui/templates/project.html | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/toaster/toastergui/templates/project.html b/lib/toaster/toastergui/templates/project.html
index badc8a6..35147c0 100644
--- a/lib/toaster/toastergui/templates/project.html
+++ b/lib/toaster/toastergui/templates/project.html
@@ -412,7 +412,7 @@ vim: expandtab tabstop=2
           <option ng-repeat="r in releases" value="{[r.id]}" ng-selected="r.id == project.release.id">{[r.description]}</option>
         </select>
         <input type="submit" class="btn" style="margin-left:5px;" value="Save" ng-disabled="project.release.id == projectVersion"/>
-        <input type="reset"  class="btn btn-link" value="Cancel" ng-click="toggle('#change-project-version')" ng-disabled="project.release.id == projectVersion"/>
+        <input type="reset"  class="btn btn-link" value="Cancel" ng-click="toggle('#change-project-version')"/>
 
       </form>
     </div>
-- 
1.9.1

[PATCH 10/14] toaster: Incorrect breadcrumb behaviour in the project page

[Alex DAMIAN]

From: David Reyna <David.Reyna@windriver.com>

The last item in the breadcrumb for the project page should not be a link.

[YOCTO #7157]

Signed-off-by: David Reyna <David.Reyna@windriver.com>
---
 lib/toaster/toastergui/templates/baseprojectpage.html | 6 ++++++
 lib/toaster/toastergui/views.py                       | 1 +
 2 files changed, 7 insertions(+)

diff --git a/lib/toaster/toastergui/templates/baseprojectpage.html b/lib/toaster/toastergui/templates/baseprojectpage.html
index e140925..0807f4c 100644
--- a/lib/toaster/toastergui/templates/baseprojectpage.html
+++ b/lib/toaster/toastergui/templates/baseprojectpage.html
@@ -11,10 +11,16 @@
             <li><a href="{% url 'all-builds' %}">All builds</a></li>
             {% block parentbreadcrumb %}
         {% if project %}
+          {% if project_html %}
+            <li>
+                {{project.name}}
+            </li>
+          {% else %}
             <li>
                 <a href="{%url 'project' project.id %}"><span id="project_name">{{project.name}}</span>
                 </a>
             </li>
+          {% endif %}
         {% endif %}
             {% endblock %}
             {% block localbreadcrumb %}{% endblock %}
diff --git a/lib/toaster/toastergui/views.py b/lib/toaster/toastergui/views.py
index e87c9f2..661023e 100755
--- a/lib/toaster/toastergui/views.py
+++ b/lib/toaster/toastergui/views.py
@@ -2142,6 +2142,7 @@ if toastermain.settings.MANAGED:
             "targets" : map(lambda x: {"target" : x.target, "task" : x.task, "pk": x.pk}, prj.projecttarget_set.all()),
             "freqtargets": freqtargets,
             "releases": map(lambda x: {"id": x.pk, "name": x.name, "description":x.description}, Release.objects.all()),
+            "project_html": 1,
         }
         try:
             context["machine"] = {"name": prj.projectvariable_set.get(name="MACHINE").value}
-- 
1.9.1

[PATCH 11/14] toaster: Add cancel build to latest build section

[Alex DAMIAN]

From: Michael Wood <michael.g.wood@intel.com>

Add this functionality to the common managed_mrb_section
Make sure we are using the correct version of this template in the
projects template and remove now redundant code.

[YOCTO #7351]

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
---
 lib/toaster/toastergui/static/js/libtoaster.js     | 28 +++++++++++++++
 .../toastergui/templates/managed_mrb_section.html  | 42 +++++++++-------------
 lib/toaster/toastergui/templates/mrb_section.html  | 39 --------------------
 lib/toaster/toastergui/templates/projects.html     |  2 +-
 lib/toaster/toastergui/views.py                    | 11 ++++--
 5 files changed, 53 insertions(+), 69 deletions(-)

diff --git a/lib/toaster/toastergui/static/js/libtoaster.js b/lib/toaster/toastergui/static/js/libtoaster.js
index 3832066..fcf82ac 100644
--- a/lib/toaster/toastergui/static/js/libtoaster.js
+++ b/lib/toaster/toastergui/static/js/libtoaster.js
@@ -93,6 +93,33 @@ var libtoaster = (function (){
     } });
   }
 
+  /* cancelABuild:
+   * url: xhr_projectbuild
+   * builds_ids: space separated list of build request ids
+   * onsuccess: callback for successful execution
+   * onfail: callback for failed execution
+   */
+  function _cancelABuild(url, build_ids, onsuccess, onfail){
+    $.ajax( {
+        type: "POST",
+        url: url,
+        data: { 'buildCancel': build_ids },
+        headers: { 'X-CSRFToken' : $.cookie('csrftoken')},
+        success: function (_data) {
+          if (_data.error !== "ok") {
+            console.warn(_data.error);
+          } else {
+            if (onsuccess !== undefined) onsuccess(_data);
+          }
+        },
+        error: function (_data) {
+          console.warn("Call failed");
+          console.warn(_data);
+          if (onfail) onfail(data);
+        }
+    });
+  }
+
   /* Get a project's configuration info */
   function _getProjectInfo(url, projectId, onsuccess, onfail){
     $.ajax({
@@ -197,6 +224,7 @@ var libtoaster = (function (){
   return {
     reload_params : reload_params,
     startABuild : _startABuild,
+    cancelABuild : _cancelABuild,
     makeTypeahead : _makeTypeahead,
     getProjectInfo: _getProjectInfo,
     getLayerDepsForProject : _getLayerDepsForProject,
diff --git a/lib/toaster/toastergui/templates/managed_mrb_section.html b/lib/toaster/toastergui/templates/managed_mrb_section.html
index 51610e4..a6d4ac6 100644
--- a/lib/toaster/toastergui/templates/managed_mrb_section.html
+++ b/lib/toaster/toastergui/templates/managed_mrb_section.html
@@ -3,6 +3,7 @@
 {% load humanize %}
 
 {%if mru|length > 0%}
+{# Template provides the latest builds section requires mru in the context which can be added from _managed_get_latest_builds #}
   <div class="page-header top-air">
       <h1>
           Latest builds
@@ -118,6 +119,7 @@
             <div class="span4 lead">Build queued
               <i title="This build will start as soon as a build server is available" class="icon-question-sign get-help get-help-blue heading-help" data-toggle="tooltip"></i>
             </div>
+            <button class="btn btn-info pull-right cancel-build-btn" data-build-id="{{buildrequest.id}}" data-request-url="{% url 'xhr_projectbuild' buildrequest.project.id %}" >Cancel</button>
 
          {% elif buildrequest.state == buildrequest.REQ_CREATED %}
 
@@ -155,37 +157,25 @@
 
 <script>
 
-/* ensure csrf cookie exists {% csrf_token %} */
-function _makeXHRBuildCall(url, data, onsuccess, onfail) {
-    $.ajax( {
-        type: "POST",
-        url: url,
-        data: data,
-        headers: { 'X-CSRFToken' : $.cookie('csrftoken')},
-        success: function (_data) {
-            if (_data.error != "ok") {
-                alert(_data.error);
-            } else {
-                if (onsuccess != undefined) onsuccess(_data);
-            }
-        },
-        error: function (_data) {
-            alert("Call failed");
-            console.log(_data);
-            if (onfail) onfail(data);
-        } });
+function scheduleBuild(url, projectName, projectUrl, buildlist) {
+  console.log("scheduleBuild");
+  libtoaster.startABuild(url, null, buildlist.join(" "), function(){
+    window.location.reload();
+  }, null);
 }
 
+$(document).ready(function(){
 
-function scheduleBuild(url, projectName, projectUrl, buildlist) {
-  console.log("scheduleBuild");
-  _makeXHRBuildCall(url, {targets: buildlist.join(" ")}, function (_data) {
+  $(".cancel-build-btn").click(function (){
+    var url = $(this).data('request-url');
+    var buildIds = $(this).data('build-id');
+    var btn = $(this);
 
-      $('#latest-builds').prepend("<div class=\"alert alert-info project-name\"><span class=\"label label-info\"><a href=\""+projectUrl+"\">"+projectName+"</a></span><div class=\"row-fluid\">" +
-  "<div class=\"span5 lead\">" + buildlist.join(" ") +
-  "</div><div class=\"span4 lead\">Build queued <i title=\"This build will start as soon as a build server is available\" class=\"icon-question-sign get-help get-help-blue heading-help\"></i></div></div></div>");
+    libtoaster.cancelABuild(url, buildIds, function(){
+      btn.parents(".alert").fadeOut();
+    }, null);
   });
-}
+});
 
 </script>
 
diff --git a/lib/toaster/toastergui/templates/mrb_section.html b/lib/toaster/toastergui/templates/mrb_section.html
index 432955a..c7bddf0 100644
--- a/lib/toaster/toastergui/templates/mrb_section.html
+++ b/lib/toaster/toastergui/templates/mrb_section.html
@@ -50,9 +50,6 @@
               <span class="lead{%if not MANAGED or not build.project%} pull-right{%endif%}">
                 Build time: <a href="{% url 'buildtime' build.pk %}">{{ build.timespent|sectohms }}</a>
               </span>
-          {% if MANAGED and build.project %}
-              <a class="btn {%if build.outcome == build.SUCCEEDED%}btn-success{%elif build.outcome == build.FAILED%}btn-danger{%else%}btn-info{%endif%} pull-right" onclick="scheduleBuild({% url 'xhr_projectbuild' build.project.id as bpi%}{{bpi|json}}, {{build.project.name|json}}, {{build.get_sorted_target_list|mapselect:'target'|json}})">Run again</a>
-          {% endif %}
             </div>
     {%endif%}
     {%if build.outcome == build.IN_PROGRESS %}
@@ -68,41 +65,5 @@
 
   {% endfor %}
   </div>
-
-<script>
-
-function _makeXHRBuildCall(url, data, onsuccess, onfail) {
-    $.ajax( {
-        type: "POST",
-        url: url,
-        data: data,
-        headers: { 'X-CSRFToken' : $.cookie('csrftoken')},
-        success: function (_data) {
-            if (_data.error != "ok") {
-                console.warn(_data.error);
-            } else {
-                if (onsuccess != undefined) onsuccess(_data);
-            }
-        },
-        error: function (_data) {
-            console.warn("Call failed");
-            console.warn(_data);
-            if (onfail) onfail(data);
-        } });
-}
-
-
-function scheduleBuild(url, projectName, buildlist) {
-  console.warn("scheduleBuild");
-  _makeXHRBuildCall(url, {targets: buildlist.join(" ")}, function (_data) {
-
-      $('#latest-builds').prepend('<div class="alert alert-info" style="padding-top:0px">' + '<span class="label label-info" style="font-weight: normal; margin-bottom: 5px; margin-left:-15px; padding-top:5px;">'+projectName+'</span><div class="row-fluid">' +
-  '<div class="span4 lead">' + buildlist.join(" ") +
-  '</div><div class="span4 lead pull-right">Build queued. Your build will start shortly.</div></div></div>');
-  });
-}
-
-</script>
-
 {%endif%}
 
diff --git a/lib/toaster/toastergui/templates/projects.html b/lib/toaster/toastergui/templates/projects.html
index 88d5bd3..88ee4bc 100644
--- a/lib/toaster/toastergui/templates/projects.html
+++ b/lib/toaster/toastergui/templates/projects.html
@@ -7,7 +7,7 @@
 {% block pagecontent %}
 
 
-  {% include "mrb_section.html" %}
+  {% include "managed_mrb_section.html" %}
 
 
   <div class="page-header top-air">
diff --git a/lib/toaster/toastergui/views.py b/lib/toaster/toastergui/views.py
index 661023e..26c8760 100755
--- a/lib/toaster/toastergui/views.py
+++ b/lib/toaster/toastergui/views.py
@@ -1871,8 +1871,7 @@ if toastermain.settings.MANAGED:
 
         # build view-specific information; this is rendered specifically in the builds page, at the top of the page (i.e. Recent builds)
         # most recent build is like projects' most recent builds, but across all projects
-        build_mru = BuildRequest.objects.all()
-        build_mru = list(build_mru.filter(Q(state__lt=BuildRequest.REQ_COMPLETED) or Q(state=BuildRequest.REQ_DELETED)).order_by("-pk")) + list(build_mru.filter(state__in=[BuildRequest.REQ_COMPLETED, BuildRequest.REQ_FAILED]).order_by("-pk")[:3])
+        build_mru = _managed_get_latest_builds()
 
         fstypes_map = {};
         for build_request in build_info:
@@ -3130,6 +3129,12 @@ if toastermain.settings.MANAGED:
             }
             return render(request, "unavailable_artifact.html", context)
 
+    # This returns the mru object that is needed for the
+    # managed_mrb_section.html template
+    def _managed_get_latest_builds():
+        build_mru = BuildRequest.objects.all()
+        build_mru = list(build_mru.filter(Q(state__lt=BuildRequest.REQ_COMPLETED) or Q(state=BuildRequest.REQ_DELETED)).order_by("-pk")) + list(build_mru.filter(state__in=[BuildRequest.REQ_COMPLETED, BuildRequest.REQ_FAILED]).order_by("-pk")[:3])
+        return build_mru
 
 
     def projects(request):
@@ -3153,7 +3158,7 @@ if toastermain.settings.MANAGED:
         project_info = _build_page_range(Paginator(queryset, pagesize), request.GET.get('page', 1))
 
         # build view-specific information; this is rendered specifically in the builds page, at the top of the page (i.e. Recent builds)
-        build_mru = Build.objects.order_by("-started_on")[:3]
+        build_mru = _managed_get_latest_builds()
 
         # translate the project's build target strings
         fstypes_map = {};
-- 
1.9.1

[PATCH 12/14] toaster: tasks trigger the select related tables before ordering

[Alex DAMIAN]

From: Michael Wood <michael.g.wood@intel.com>

In two of the cases the data is changed from a QuerySet to a sorted list
so we aren't able to use the QuerySet api at this point, moving it
before the data type is changed fixes this.

[YOCTO #7404]

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
---
 lib/toaster/toastergui/views.py | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/lib/toaster/toastergui/views.py b/lib/toaster/toastergui/views.py
index 26c8760..ea47d96 100755
--- a/lib/toaster/toastergui/views.py
+++ b/lib/toaster/toastergui/views.py
@@ -877,7 +877,10 @@ def tasks_common(request, build_id, variant, task_anchor):
         return _redirect_parameters( variant, request.GET, mandatory_parameters, build_id = build_id)
     (filter_string, search_term, ordering_string) = _search_tuple(request, Task)
     queryset_all = Task.objects.filter(build=build_id).exclude(order__isnull=True).exclude(outcome=Task.OUTCOME_NA)
+    queryset_all = queryset_all.select_related("recipe", "build")
+
     queryset_with_search = _get_queryset(Task, queryset_all, None , search_term, ordering_string, 'order')
+
     if ordering_string.startswith('outcome'):
         queryset = _get_queryset(Task, queryset_all, filter_string, search_term, 'order:+', 'order')
         queryset = sorted(queryset, key=lambda ur: (ur.outcome_text), reverse=ordering_string.endswith('-'))
@@ -887,7 +890,6 @@ def tasks_common(request, build_id, variant, task_anchor):
     else:
         queryset = _get_queryset(Task, queryset_all, filter_string, search_term, ordering_string, 'order')
 
-    queryset = queryset.select_related("recipe", "build")
 
     # compute the anchor's page
     if anchor:
-- 
1.9.1

[PATCH 13/14] toaster: display machine and project values in input fields

[Alex DAMIAN]

From: David Reyna <David.Reyna@windriver.com>

Explicitly preset the machine and project values in their respective
input fields.

[YOCTO #7170]

Signed-off-by: David Reyna <David.Reyna@windriver.com>
---
 lib/toaster/toastergui/templates/project.html | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/toaster/toastergui/templates/project.html b/lib/toaster/toastergui/templates/project.html
index 35147c0..0f538b9 100644
--- a/lib/toaster/toastergui/templates/project.html
+++ b/lib/toaster/toastergui/templates/project.html
@@ -355,7 +355,7 @@ vim: expandtab tabstop=2
               You cannot really compare the builds for the new machine with the previous ones.
         </div>
         <form ng-submit="editProjectSettings('#select-machine')" class="input-append">
-              <input type="text" id="machine" autocomplete="off" ng-model="machineName" typeahead="m.name for m in getAutocompleteSuggestions('machines', $viewValue)" typeahead-template-url="suggestion_details" />
+              <input type="text" id="machine" autocomplete="off" ng-model="machineName" value="{[machine.name]}" typeahead="m.name for m in getAutocompleteSuggestions('machines', $viewValue)" typeahead-template-url="suggestion_details" />
               <input type="submit" id="apply-change-machine" class="btn" type="button" ng-disabled="machineName == machine.name || machineName.length == 0" value="Save"></input>
               <input type="reset" id="cancel-machine" class="btn btn-link" ng-click="toggle('#select-machine')" value="Cancel"></input>
               {% csrf_token %}
@@ -391,7 +391,7 @@ vim: expandtab tabstop=2
     </p>
     <div id="change-project-name" style="display:none;">
       <form ng-submit="editProjectSettings('#change-project-name')" class="input-append">
-        <input type="text" class="input-xlarge" id="type-project-name" ng-model="projectName">
+        <input type="text" class="input-xlarge" id="type-project-name" ng-model="projectName" value="{[project.name]}">
         <input type="submit" class="btn" value="Save" ng-disabled="project.name == projectName"/>
         <input type="reset" class="btn btn-link" value="Cancel" ng-click="toggle('#change-project-name')">
      </form>
-- 
1.9.1

[PATCH 14/14] toaster: insure IMAGE_INSTALL_append values have a space prefix

[Alex DAMIAN]

From: David Reyna <David.Reyna@windriver.com>

When the IMAGE_INSTALL_append value is not empty, it must have a space prefix
in order for the "append" operator to concatenate the values correctly, so we
enforce it in this variable's submit javascript.

[YOCTO #7394]

Signed-off-by: David Reyna <David.Reyna@windriver.com>
---
 lib/toaster/toastergui/templates/projectconf.html | 18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)

diff --git a/lib/toaster/toastergui/templates/projectconf.html b/lib/toaster/toastergui/templates/projectconf.html
index ffdf760..d8ead81 100644
--- a/lib/toaster/toastergui/templates/projectconf.html
+++ b/lib/toaster/toastergui/templates/projectconf.html
@@ -468,7 +468,7 @@
             // re-assert the event handlers
         }
 
-	/* ensure cookie exists {% csrf_token %} */
+        /* ensure cookie exists {% csrf_token %} */
         function postEditAjaxRequest(reqdata) {
             var ajax = $.ajax({
                     type:"POST",
@@ -621,10 +621,13 @@
             // change IMAGE_INSTALL_append variable
             $('#change-image_install-icon').click(function() {
                 // preset the edit value
-                var current_val = $("span#image_install").text();
+                var current_val = $("span#image_install").text().trim();
                 if (current_val == "Not set") {
                     current_val="";
                     $("#apply-change-image_install").attr("disabled","disabled");
+                } else {
+                    // insure these non-empty values have single space prefix
+                    current_val=" " + current_val;
                 }
                 $("input#new-image_install").val(current_val);
 
@@ -643,7 +646,7 @@
             });
 
             $("#new-image_install").on('input', function(){
-                if ($(this).val().length == 0) {
+                if ($(this).val().trim().length == 0) {
                     $("#apply-change-image_install").attr("disabled","disabled");
                 }
                 else {
@@ -652,13 +655,14 @@
             });
 
             $('#apply-change-image_install').click(function(){
-                var name = $('#new-image_install').val();
-                postEditAjaxRequest({"configvarChange" : 'IMAGE_INSTALL_append:'+name});
-                $('#image_install').text(name);
+                // insure these non-empty values have single space prefix
+                var value = " " + $('#new-image_install').val().trim();
+                postEditAjaxRequest({"configvarChange" : 'IMAGE_INSTALL_append:'+value});
+                $('#image_install').text(value);
                 $('#image_install').removeClass('muted');
                 $("#change-image_install-form").slideUp(function () {
                     $('#image_install, #change-image_install-icon').show();
-                    if (name.length > -1) {
+                    if (value.length > -1) {
                         $('#delete-image_install-icon').show();
                         setDeleteTooltip($('#delete-image_install-icon'));
                     }
-- 
1.9.1