* [PATCH 0/5] toaster: cummulative 26062017 patch
@ 2017-06-27 20:44 David Reyna
2017-06-27 20:44 ` [PATCH 1/5] toaster: address Django-1.10 API deprecations David Reyna
` (4 more replies)
0 siblings, 5 replies; 6+ messages in thread
From: David Reyna @ 2017-06-27 20:44 UTC (permalink / raw)
To: bitbake-devel
From: David Reyna <David.Reyna@windriver.com>
*** NOTE for MERGE: New Files: ***
create mode 100644 bitbake/lib/toaster/orm/migrations/0016_clone_progress.py
create mode 100644 bitbake/lib/toaster/orm/migrations/0017_distro_clone.py
create mode 100644 bitbake/lib/toaster/toastergui/templates/distro_btn.html
I have been advised that: "If the patch creates any NEW files, the integration
scripts that pull it into bitbake will fail. So, if new files are created as
part of this patch set, you need to explicitly point that out in the body of
the email for the patch set"
- David
=================
The following changes since commit b8358bc61435ec996ecd6931fb6d6d41bb4f710b:
bitbake: bitbake-selftest: add bb.tests.event to bitbake-selftest (2017-06-23 14:14:17 +0100)
are available in the git repository at:
git://git.yoctoproject.org/poky-contrib dreyna/submit/dreyna/toaster/cummulative_062017_patch
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=dreyna/submit/dreyna/toaster/cummulative_062017_patch
David Reyna (5):
toaster: address Django-1.10 API deprecations
toaster: git clone progress bar
toaster: Add distro selection support
toaster: large package set breaks sqlite query
toaster: test 'commit' first in get_vcs_reference
bin/toaster | 10 ++-
lib/toaster/bldcollector/urls.py | 10 ++-
lib/toaster/bldcontrol/localhostbecontroller.py | 12 +++
.../management/commands/checksettings.py | 6 +-
.../bldcontrol/management/commands/runbuilds.py | 6 +-
lib/toaster/orm/management/commands/lsupdates.py | 27 ++++++-
lib/toaster/orm/migrations/0016_clone_progress.py | 24 ++++++
lib/toaster/orm/migrations/0017_distro_clone.py | 25 ++++++
lib/toaster/orm/models.py | 50 +++++++++++-
lib/toaster/toastergui/api.py | 13 +++-
lib/toaster/toastergui/static/js/mrbsection.js | 15 ++++
lib/toaster/toastergui/static/js/projectpage.js | 73 +++++++++++++++++
lib/toaster/toastergui/tables.py | 91 ++++++++++++++++++++++
lib/toaster/toastergui/templates/base.html | 1 +
.../toastergui/templates/baseprojectpage.html | 1 +
lib/toaster/toastergui/templates/distro_btn.html | 20 +++++
lib/toaster/toastergui/templates/mrb_section.html | 29 ++++++-
lib/toaster/toastergui/templates/project.html | 16 ++++
lib/toaster/toastergui/typeaheads.py | 30 +++++++
lib/toaster/toastergui/urls.py | 69 +++++++++-------
lib/toaster/toastergui/views.py | 13 +++-
lib/toaster/toastergui/widgets.py | 4 +
.../toastermain/management/commands/buildslist.py | 6 +-
lib/toaster/toastermain/settings.py | 70 ++++++++++-------
lib/toaster/toastermain/urls.py | 8 +-
toaster-requirements.txt | 2 +-
26 files changed, 539 insertions(+), 92 deletions(-)
create mode 100644 bitbake/lib/toaster/orm/migrations/0016_clone_progress.py
create mode 100644 bitbake/lib/toaster/orm/migrations/0017_distro_clone.py
create mode 100644 bitbake/lib/toaster/toastergui/templates/distro_btn.html
--
1.9.1
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH 1/5] toaster: address Django-1.10 API deprecations
2017-06-27 20:44 [PATCH 0/5] toaster: cummulative 26062017 patch David Reyna
@ 2017-06-27 20:44 ` David Reyna
2017-06-27 20:44 ` [PATCH 2/5] toaster: git clone progress bar David Reyna
` (3 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: David Reyna @ 2017-06-27 20:44 UTC (permalink / raw)
To: bitbake-devel
From: David Reyna <David.Reyna@windriver.com>
There are four main API deprecations in Django-1.10:
(a) String view arguments to url() must be replaced by
the explicit class reference
(b) New TEMPLATES stucture in settings.py consolidates
TEMPLATE_DIRS, TEMPLATE_CONTEXT_PROCESSORS,
TEMPLATE_LOADERS, TEMPLATE_STRING_IF_INVALID, and
TEMPLATE_DEBUG
(c) patterns() wrapper in url() is removed, with
urlpatterns now a simple list
(d) NoArgsCommand in commands() must be replace by
BaseCommand, and handle_noargs() changed to
handle()
Also, the Django version checker must be updated to accept
two digit sub-version numbers (e.g. "1.8" < "1.10")
[YOCTO #11684]
Signed-off-by: David Reyna <David.Reyna@windriver.com>
---
bin/toaster | 10 +++-
lib/toaster/bldcollector/urls.py | 10 ++--
.../management/commands/checksettings.py | 6 +-
.../bldcontrol/management/commands/runbuilds.py | 6 +-
lib/toaster/orm/management/commands/lsupdates.py | 8 +--
lib/toaster/toastergui/urls.py | 61 +++++++++----------
.../toastermain/management/commands/buildslist.py | 6 +-
lib/toaster/toastermain/settings.py | 70 +++++++++++++---------
lib/toaster/toastermain/urls.py | 8 ++-
toaster-requirements.txt | 2 +-
10 files changed, 105 insertions(+), 82 deletions(-)
diff --git a/bin/toaster b/bin/toaster
index 61a4a0f..c2e33fe 100755
--- a/bin/toaster
+++ b/bin/toaster
@@ -116,8 +116,14 @@ verify_prereq() {
# Verify Django version
reqfile=$(python3 -c "import os; print(os.path.realpath('$BBBASEDIR/toaster-requirements.txt'))")
exp='s/Django\([><=]\+\)\([^,]\+\),\([><=]\+\)\(.\+\)/'
- exp=$exp'import sys,django;version=django.get_version().split(".");'
- exp=$exp'sys.exit(not (version \1 "\2".split(".") and version \3 "\4".split(".")))/p'
+ # expand version parts to 2 digits to support 1.10.x > 1.8
+ # (note:helper functions hard to insert in-line)
+ exp=$exp'import sys,django;'
+ exp=$exp'version=["%02d" % int(n) for n in django.get_version().split(".")];'
+ exp=$exp'vmin=["%02d" % int(n) for n in "\2".split(".")];'
+ exp=$exp'vmax=["%02d" % int(n) for n in "\4".split(".")];'
+ exp=$exp'sys.exit(not (version \1 vmin and version \3 vmax))'
+ exp=$exp'/p'
if ! sed -n "$exp" $reqfile | python3 - ; then
req=`grep ^Django $reqfile`
echo "This program needs $req"
diff --git a/lib/toaster/bldcollector/urls.py b/lib/toaster/bldcollector/urls.py
index 64722f2..076afdb 100644
--- a/lib/toaster/bldcollector/urls.py
+++ b/lib/toaster/bldcollector/urls.py
@@ -1,7 +1,7 @@
#
# BitBake Toaster Implementation
#
-# Copyright (C) 2014 Intel Corporation
+# Copyright (C) 2014-2017 Intel Corporation
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License version 2 as
@@ -19,7 +19,9 @@
from django.conf.urls import patterns, include, url
-urlpatterns = patterns('bldcollector.views',
+import bldcollector.views
+
+urlpatterns = [
# landing point for pushing a bitbake_eventlog.json file to this toaster instace
- url(r'^eventfile$', 'eventfile', name='eventfile'),
- )
+ url(r'^eventfile$', bldcollector.views.eventfile, name='eventfile'),
+]
diff --git a/lib/toaster/bldcontrol/management/commands/checksettings.py b/lib/toaster/bldcontrol/management/commands/checksettings.py
index 2ed994f..d11166e 100644
--- a/lib/toaster/bldcontrol/management/commands/checksettings.py
+++ b/lib/toaster/bldcontrol/management/commands/checksettings.py
@@ -1,4 +1,4 @@
-from django.core.management.base import NoArgsCommand, CommandError
+from django.core.management.base import BaseCommand, CommandError
from django.db import transaction
from django.core.management import call_command
@@ -18,7 +18,7 @@ def DN(path):
return os.path.dirname(path)
-class Command(NoArgsCommand):
+class Command(BaseCommand):
args = ""
help = "Verifies that the configured settings are valid and usable, or prompts the user to fix the settings."
@@ -152,7 +152,7 @@ class Command(NoArgsCommand):
- def handle_noargs(self, **options):
+ def handle(self, **options):
retval = 0
retval += self._verify_build_environment()
retval += self._verify_default_settings()
diff --git a/lib/toaster/bldcontrol/management/commands/runbuilds.py b/lib/toaster/bldcontrol/management/commands/runbuilds.py
index df11f9d..8eacb5e 100644
--- a/lib/toaster/bldcontrol/management/commands/runbuilds.py
+++ b/lib/toaster/bldcontrol/management/commands/runbuilds.py
@@ -1,4 +1,4 @@
-from django.core.management.base import NoArgsCommand
+from django.core.management.base import BaseCommand
from django.db import transaction
from django.db.models import Q
@@ -16,7 +16,7 @@ import os
logger = logging.getLogger("toaster")
-class Command(NoArgsCommand):
+class Command(BaseCommand):
args = ""
help = "Schedules and executes build requests as possible. "\
"Does not return (interrupt with Ctrl-C)"
@@ -168,7 +168,7 @@ class Command(NoArgsCommand):
except Exception as e:
logger.warn("runbuilds: schedule exception %s" % str(e))
- def handle_noargs(self, **options):
+ def handle(self, **options):
pidfile_path = os.path.join(os.environ.get("BUILDDIR", "."),
".runbuilds.pid")
diff --git a/lib/toaster/orm/management/commands/lsupdates.py b/lib/toaster/orm/management/commands/lsupdates.py
index 68c6c42..90f07c9 100644
--- a/lib/toaster/orm/management/commands/lsupdates.py
+++ b/lib/toaster/orm/management/commands/lsupdates.py
@@ -4,7 +4,7 @@
#
# BitBake Toaster Implementation
#
-# Copyright (C) 2016 Intel Corporation
+# Copyright (C) 2016-2017 Intel Corporation
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License version 2 as
@@ -19,7 +19,7 @@
# with this program; if not, write to the Free Software Foundation, Inc.,
# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-from django.core.management.base import NoArgsCommand
+from django.core.management.base import BaseCommand
from orm.models import LayerSource, Layer, Release, Layer_Version
from orm.models import LayerVersionDependency, Machine, Recipe
@@ -56,7 +56,7 @@ class Spinner(threading.Thread):
self.signal = False
-class Command(NoArgsCommand):
+class Command(BaseCommand):
args = ""
help = "Updates locally cached information from a layerindex server"
@@ -307,5 +307,5 @@ class Command(NoArgsCommand):
os.system('setterm -cursor on')
- def handle_noargs(self, **options):
+ def handle(self, **options):
self.update()
diff --git a/lib/toaster/toastergui/urls.py b/lib/toaster/toastergui/urls.py
index d92f190..1ad79a4 100644
--- a/lib/toaster/toastergui/urls.py
+++ b/lib/toaster/toastergui/urls.py
@@ -1,7 +1,7 @@
#
# BitBake Toaster Implementation
#
-# Copyright (C) 2013 Intel Corporation
+# Copyright (C) 2013-2017 Intel Corporation
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License version 2 as
@@ -25,49 +25,50 @@ from toastergui import buildtables
from toastergui import typeaheads
from toastergui import api
from toastergui import widgets
+from toastergui import views
-urlpatterns = patterns('toastergui.views',
+urlpatterns = [
# landing page
- url(r'^landing/$', 'landing', name='landing'),
+ url(r'^landing/$', views.landing, name='landing'),
url(r'^builds/$',
tables.AllBuildsTable.as_view(template_name="builds-toastertable.html"),
name='all-builds'),
# build info navigation
- url(r'^build/(?P<build_id>\d+)$', 'builddashboard', name="builddashboard"),
+ url(r'^build/(?P<build_id>\d+)$', views.builddashboard, name="builddashboard"),
url(r'^build/(?P<build_id>\d+)/tasks/$',
buildtables.BuildTasksTable.as_view(
template_name="buildinfo-toastertable.html"),
name='tasks'),
- url(r'^build/(?P<build_id>\d+)/task/(?P<task_id>\d+)$', 'task', name='task'),
+ url(r'^build/(?P<build_id>\d+)/task/(?P<task_id>\d+)$', views.task, name='task'),
url(r'^build/(?P<build_id>\d+)/recipes/$',
buildtables.BuiltRecipesTable.as_view(
template_name="buildinfo-toastertable.html"),
name='recipes'),
- url(r'^build/(?P<build_id>\d+)/recipe/(?P<recipe_id>\d+)/active_tab/(?P<active_tab>\d{1})$', 'recipe', name='recipe'),
+ url(r'^build/(?P<build_id>\d+)/recipe/(?P<recipe_id>\d+)/active_tab/(?P<active_tab>\d{1})$', views.recipe, name='recipe'),
- url(r'^build/(?P<build_id>\d+)/recipe/(?P<recipe_id>\d+)$', 'recipe', name='recipe'),
- url(r'^build/(?P<build_id>\d+)/recipe_packages/(?P<recipe_id>\d+)$', 'recipe_packages', name='recipe_packages'),
+ url(r'^build/(?P<build_id>\d+)/recipe/(?P<recipe_id>\d+)$', views.recipe, name='recipe'),
+ url(r'^build/(?P<build_id>\d+)/recipe_packages/(?P<recipe_id>\d+)$', views.recipe_packages, name='recipe_packages'),
url(r'^build/(?P<build_id>\d+)/packages/$',
buildtables.BuiltPackagesTable.as_view(
template_name="buildinfo-toastertable.html"),
name='packages'),
- url(r'^build/(?P<build_id>\d+)/package/(?P<package_id>\d+)$', 'package_built_detail',
+ url(r'^build/(?P<build_id>\d+)/package/(?P<package_id>\d+)$', views.package_built_detail,
name='package_built_detail'),
url(r'^build/(?P<build_id>\d+)/package_built_dependencies/(?P<package_id>\d+)$',
- 'package_built_dependencies', name='package_built_dependencies'),
+ views.package_built_dependencies, name='package_built_dependencies'),
url(r'^build/(?P<build_id>\d+)/package_included_detail/(?P<target_id>\d+)/(?P<package_id>\d+)$',
- 'package_included_detail', name='package_included_detail'),
+ views.package_included_detail, name='package_included_detail'),
url(r'^build/(?P<build_id>\d+)/package_included_dependencies/(?P<target_id>\d+)/(?P<package_id>\d+)$',
- 'package_included_dependencies', name='package_included_dependencies'),
+ views.package_included_dependencies, name='package_included_dependencies'),
url(r'^build/(?P<build_id>\d+)/package_included_reverse_dependencies/(?P<target_id>\d+)/(?P<package_id>\d+)$',
- 'package_included_reverse_dependencies', name='package_included_reverse_dependencies'),
+ views.package_included_reverse_dependencies, name='package_included_reverse_dependencies'),
url(r'^build/(?P<build_id>\d+)/target/(?P<target_id>\d+)$',
buildtables.InstalledPackagesTable.as_view(
@@ -75,11 +76,11 @@ urlpatterns = patterns('toastergui.views',
name='target'),
- url(r'^dentries/build/(?P<build_id>\d+)/target/(?P<target_id>\d+)$', 'xhr_dirinfo', name='dirinfo_ajax'),
- url(r'^build/(?P<build_id>\d+)/target/(?P<target_id>\d+)/dirinfo$', 'dirinfo', name='dirinfo'),
- url(r'^build/(?P<build_id>\d+)/target/(?P<target_id>\d+)/dirinfo_filepath/_(?P<file_path>(?:/[^/\n]+)*)$', 'dirinfo', name='dirinfo_filepath'),
- url(r'^build/(?P<build_id>\d+)/configuration$', 'configuration', name='configuration'),
- url(r'^build/(?P<build_id>\d+)/configvars$', 'configvars', name='configvars'),
+ url(r'^dentries/build/(?P<build_id>\d+)/target/(?P<target_id>\d+)$', views.xhr_dirinfo, name='dirinfo_ajax'),
+ url(r'^build/(?P<build_id>\d+)/target/(?P<target_id>\d+)/dirinfo$', views.dirinfo, name='dirinfo'),
+ url(r'^build/(?P<build_id>\d+)/target/(?P<target_id>\d+)/dirinfo_filepath/_(?P<file_path>(?:/[^/\n]+)*)$', views.dirinfo, name='dirinfo_filepath'),
+ url(r'^build/(?P<build_id>\d+)/configuration$', views.configuration, name='configuration'),
+ url(r'^build/(?P<build_id>\d+)/configvars$', views.configvars, name='configvars'),
url(r'^build/(?P<build_id>\d+)/buildtime$',
buildtables.BuildTimeTable.as_view(
template_name="buildinfo-toastertable.html"),
@@ -97,26 +98,26 @@ urlpatterns = patterns('toastergui.views',
# image information dir
url(r'^build/(?P<build_id>\d+)/target/(?P<target_id>\d+)/packagefile/(?P<packagefile_id>\d+)$',
- 'image_information_dir', name='image_information_dir'),
+ views.image_information_dir, name='image_information_dir'),
# build download artifact
- url(r'^build/(?P<build_id>\d+)/artifact/(?P<artifact_type>\w+)/id/(?P<artifact_id>\w+)', 'build_artifact', name="build_artifact"),
+ url(r'^build/(?P<build_id>\d+)/artifact/(?P<artifact_type>\w+)/id/(?P<artifact_id>\w+)', views.build_artifact, name="build_artifact"),
# project URLs
- url(r'^newproject/$', 'newproject', name='newproject'),
+ url(r'^newproject/$', views.newproject, name='newproject'),
url(r'^projects/$',
tables.ProjectsTable.as_view(template_name="projects-toastertable.html"),
name='all-projects'),
- url(r'^project/(?P<pid>\d+)/$', 'project', name='project'),
- url(r'^project/(?P<pid>\d+)/configuration$', 'projectconf', name='projectconf'),
+ url(r'^project/(?P<pid>\d+)/$', views.project, name='project'),
+ url(r'^project/(?P<pid>\d+)/configuration$', views.projectconf, name='projectconf'),
url(r'^project/(?P<pid>\d+)/builds/$',
tables.ProjectBuildsTable.as_view(template_name="projectbuilds-toastertable.html"),
name='projectbuilds'),
# the import layer is a project-specific functionality;
- url(r'^project/(?P<pid>\d+)/importlayer$', 'importlayer', name='importlayer'),
+ url(r'^project/(?P<pid>\d+)/importlayer$', views.importlayer, name='importlayer'),
# the table pages that have been converted to ToasterTable widget
url(r'^project/(?P<pid>\d+)/machines/$',
@@ -142,7 +143,7 @@ urlpatterns = patterns('toastergui.views',
name="projectlayers"),
url(r'^project/(?P<pid>\d+)/layer/(?P<layerid>\d+)$',
- 'layerdetails', name='layerdetails'),
+ views.layerdetails, name='layerdetails'),
url(r'^project/(?P<pid>\d+)/layer/(?P<layerid>\d+)/recipes/$',
tables.LayerRecipesTable.as_view(template_name="generic-toastertable-page.html"),
@@ -166,7 +167,7 @@ urlpatterns = patterns('toastergui.views',
name="customrecipe"),
url(r'^project/(?P<pid>\d+)/customrecipe/(?P<recipe_id>\d+)/download$',
- 'customrecipe_download',
+ views.customrecipe_download,
name="customrecipedownload"),
url(r'^project/(?P<pid>\d+)/recipe/(?P<recipe_id>\d+)$',
@@ -186,9 +187,9 @@ urlpatterns = patterns('toastergui.views',
typeaheads.GitRevisionTypeAhead.as_view(),
name='xhr_gitrevtypeahead'),
- url(r'^xhr_testreleasechange/(?P<pid>\d+)$', 'xhr_testreleasechange',
+ url(r'^xhr_testreleasechange/(?P<pid>\d+)$', views.xhr_testreleasechange,
name='xhr_testreleasechange'),
- url(r'^xhr_configvaredit/(?P<pid>\d+)$', 'xhr_configvaredit',
+ url(r'^xhr_configvaredit/(?P<pid>\d+)$', views.xhr_configvaredit,
name='xhr_configvaredit'),
url(r'^xhr_layer/(?P<pid>\d+)/(?P<layerversion_id>\d+)$',
@@ -200,7 +201,7 @@ urlpatterns = patterns('toastergui.views',
name='xhr_layer'),
# JS Unit tests
- url(r'^js-unit-tests/$', 'jsunittests', name='js-unit-tests'),
+ url(r'^js-unit-tests/$', views.jsunittests, name='js-unit-tests'),
# image customisation functionality
url(r'^xhr_customrecipe/(?P<recipe_id>\d+)'
@@ -237,4 +238,4 @@ urlpatterns = patterns('toastergui.views',
# default redirection
url(r'^$', RedirectView.as_view(url='landing', permanent=True)),
-)
+]
diff --git a/lib/toaster/toastermain/management/commands/buildslist.py b/lib/toaster/toastermain/management/commands/buildslist.py
index 8dfef0a..70b5812 100644
--- a/lib/toaster/toastermain/management/commands/buildslist.py
+++ b/lib/toaster/toastermain/management/commands/buildslist.py
@@ -1,13 +1,13 @@
-from django.core.management.base import NoArgsCommand, CommandError
+from django.core.management.base import BaseCommand, CommandError
from orm.models import Build
import os
-class Command(NoArgsCommand):
+class Command(BaseCommand):
args = ""
help = "Lists current builds"
- def handle_noargs(self,**options):
+ def handle(self,**options):
for b in Build.objects.all():
print("%d: %s %s %s" % (b.pk, b.machine, b.distro, ",".join([x.target for x in b.target_set.all()])))
diff --git a/lib/toaster/toastermain/settings.py b/lib/toaster/toastermain/settings.py
index 1fd649c..54ab31f 100644
--- a/lib/toaster/toastermain/settings.py
+++ b/lib/toaster/toastermain/settings.py
@@ -24,7 +24,6 @@
import os
DEBUG = True
-TEMPLATE_DEBUG = DEBUG
# Set to True to see the SQL queries in console
SQL_DEBUG = False
@@ -161,12 +160,47 @@ STATICFILES_FINDERS = (
# Make this unique, and don't share it with anybody.
SECRET_KEY = 'NOT_SUITABLE_FOR_HOSTED_DEPLOYMENT'
-# List of callables that know how to import templates from various sources.
-TEMPLATE_LOADERS = (
- 'django.template.loaders.filesystem.Loader',
- 'django.template.loaders.app_directories.Loader',
-# 'django.template.loaders.eggs.Loader',
-)
+class InvalidString(str):
+ def __mod__(self, other):
+ from django.template.base import TemplateSyntaxError
+ raise TemplateSyntaxError(
+ "Undefined variable or unknown value for: \"%s\"" % other)
+
+TEMPLATES = [
+ {
+ 'BACKEND': 'django.template.backends.django.DjangoTemplates',
+ 'DIRS': [
+ # Put strings here, like "/home/html/django_templates" or "C:/www/django/templates".
+ # Always use forward slashes, even on Windows.
+ # Don't forget to use absolute paths, not relative paths.
+ ],
+ 'OPTIONS': {
+ 'context_processors': [
+ # Insert your TEMPLATE_CONTEXT_PROCESSORS here or use this
+ # list if you haven't customized them:
+ 'django.contrib.auth.context_processors.auth',
+ 'django.template.context_processors.debug',
+ 'django.template.context_processors.i18n',
+ 'django.template.context_processors.media',
+ 'django.template.context_processors.static',
+ 'django.template.context_processors.tz',
+ 'django.contrib.messages.context_processors.messages',
+ # Custom
+ 'django.core.context_processors.request',
+ 'toastergui.views.managedcontextprocessor',
+
+ ],
+ 'loaders': [
+ # List of callables that know how to import templates from various sources.
+ 'django.template.loaders.filesystem.Loader',
+ 'django.template.loaders.app_directories.Loader',
+ #'django.template.loaders.eggs.Loader',
+ ],
+ 'string_if_invalid': InvalidString("%s"),
+ 'debug': DEBUG,
+ },
+ },
+]
MIDDLEWARE_CLASSES = (
'django.middleware.common.CommonMiddleware',
@@ -203,22 +237,6 @@ ROOT_URLCONF = 'toastermain.urls'
# Python dotted path to the WSGI application used by Django's runserver.
WSGI_APPLICATION = 'toastermain.wsgi.application'
-TEMPLATE_DIRS = (
- # Put strings here, like "/home/html/django_templates" or "C:/www/django/templates".
- # Always use forward slashes, even on Windows.
- # Don't forget to use absolute paths, not relative paths.
-)
-
-TEMPLATE_CONTEXT_PROCESSORS = ('django.contrib.auth.context_processors.auth',
- 'django.core.context_processors.debug',
- 'django.core.context_processors.i18n',
- 'django.core.context_processors.media',
- 'django.core.context_processors.static',
- 'django.core.context_processors.tz',
- 'django.contrib.messages.context_processors.messages',
- "django.core.context_processors.request",
- 'toastergui.views.managedcontextprocessor',
- )
INSTALLED_APPS = (
'django.contrib.auth',
@@ -348,10 +366,4 @@ connection_created.connect(activate_synchronous_off)
#
-class InvalidString(str):
- def __mod__(self, other):
- from django.template.base import TemplateSyntaxError
- raise TemplateSyntaxError(
- "Undefined variable or unknown value for: \"%s\"" % other)
-TEMPLATE_STRING_IF_INVALID = InvalidString("%s")
diff --git a/lib/toaster/toastermain/urls.py b/lib/toaster/toastermain/urls.py
index 1f8599e..bb32559 100644
--- a/lib/toaster/toastermain/urls.py
+++ b/lib/toaster/toastermain/urls.py
@@ -23,6 +23,8 @@ from django.conf.urls import patterns, include, url
from django.views.generic import RedirectView
from django.views.decorators.cache import never_cache
+import bldcollector.views
+
import logging
logger = logging.getLogger("toaster")
@@ -31,7 +33,7 @@ logger = logging.getLogger("toaster")
from django.contrib import admin
admin.autodiscover()
-urlpatterns = patterns('',
+urlpatterns = [
# Examples:
# url(r'^toaster/', include('toaster.foo.urls')),
@@ -42,11 +44,11 @@ urlpatterns = patterns('',
# This is here to maintain backward compatibility and will be deprecated
# in the future.
- url(r'^orm/eventfile$', 'bldcollector.views.eventfile'),
+ url(r'^orm/eventfile$', bldcollector.views.eventfile),
# if no application is selected, we have the magic toastergui app here
url(r'^$', never_cache(RedirectView.as_view(url='/toastergui/', permanent=True))),
-)
+]
import toastermain.settings
diff --git a/toaster-requirements.txt b/toaster-requirements.txt
index e61c8e2..3f47650 100644
--- a/toaster-requirements.txt
+++ b/toaster-requirements.txt
@@ -1,3 +1,3 @@
-Django>1.8,<1.9
+Django>1.8,<1.9.9
beautifulsoup4>=4.4.0
pytz
--
1.9.1
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH 2/5] toaster: git clone progress bar
2017-06-27 20:44 [PATCH 0/5] toaster: cummulative 26062017 patch David Reyna
2017-06-27 20:44 ` [PATCH 1/5] toaster: address Django-1.10 API deprecations David Reyna
@ 2017-06-27 20:44 ` David Reyna
2017-06-27 20:44 ` [PATCH 3/5] toaster: Add distro selection support David Reyna
` (2 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: David Reyna @ 2017-06-27 20:44 UTC (permalink / raw)
To: bitbake-devel
From: David Reyna <David.Reyna@windriver.com>
If a project has a lot of additional layers, the build may
appear to hang while those layers are checked out.
This patch adds a clone progress bar that is visible before
the parsing progress appears.
[YOCTO #9916]
Signed-off-by: David Reyna <David.Reyna@windriver.com>
---
lib/toaster/bldcontrol/localhostbecontroller.py | 12 ++++++++++
lib/toaster/orm/migrations/0016_clone_progress.py | 24 +++++++++++++++++++
lib/toaster/orm/models.py | 15 ++++++++++++
lib/toaster/toastergui/static/js/mrbsection.js | 15 ++++++++++++
lib/toaster/toastergui/templates/mrb_section.html | 29 ++++++++++++++++++++++-
lib/toaster/toastergui/widgets.py | 4 ++++
6 files changed, 98 insertions(+), 1 deletion(-)
create mode 100644 bitbake/lib/toaster/orm/migrations/0016_clone_progress.py
diff --git a/lib/toaster/bldcontrol/localhostbecontroller.py b/lib/toaster/bldcontrol/localhostbecontroller.py
index 62e62fe..2916246 100644
--- a/lib/toaster/bldcontrol/localhostbecontroller.py
+++ b/lib/toaster/bldcontrol/localhostbecontroller.py
@@ -85,6 +85,11 @@ class LocalhostBEController(BuildEnvironmentController):
return local_checkout_path
+ def setCloneStatus(self,bitbake,status,total,current):
+ bitbake.req.build.repos_cloned=current
+ bitbake.req.build.repos_to_clone=total
+ bitbake.req.build.save()
+
def setLayers(self, bitbake, layers, targets):
""" a word of attention: by convention, the first layer for any build will be poky! """
@@ -147,7 +152,13 @@ class LocalhostBEController(BuildEnvironmentController):
logger.info("Using pre-checked out source for layer %s", cached_layers)
# 3. checkout the repositories
+ clone_count=0
+ clone_total=len(gitrepos.keys())
+ self.setCloneStatus(bitbake,'Started',clone_total,clone_count)
for giturl, commit in gitrepos.keys():
+ self.setCloneStatus(bitbake,'progress',clone_total,clone_count)
+ clone_count += 1
+
localdirname = os.path.join(self.be.sourcedir, self.getGitCloneDirectory(giturl, commit))
logger.debug("localhostbecontroller: giturl %s:%s checking out in current directory %s" % (giturl, commit, localdirname))
@@ -198,6 +209,7 @@ class LocalhostBEController(BuildEnvironmentController):
if name != "bitbake":
layerlist.append(localdirpath.rstrip("/"))
+ self.setCloneStatus(bitbake,'complete',clone_total,clone_count)
logger.debug("localhostbecontroller: current layer list %s " % pformat(layerlist))
# 5. create custom layer and add custom recipes to it
diff --git a/lib/toaster/orm/migrations/0016_clone_progress.py b/lib/toaster/orm/migrations/0016_clone_progress.py
new file mode 100644
index 0000000..852b878
--- /dev/null
+++ b/lib/toaster/orm/migrations/0016_clone_progress.py
@@ -0,0 +1,24 @@
+# -*- coding: utf-8 -*-
+from __future__ import unicode_literals
+
+from django.db import migrations, models
+
+class Migration(migrations.Migration):
+
+ dependencies = [
+ ('orm', '0015_layer_local_source_dir'),
+ ]
+
+ operations = [
+ migrations.AddField(
+ model_name='build',
+ name='repos_cloned',
+ field=models.IntegerField(default=0),
+ ),
+ migrations.AddField(
+ model_name='build',
+ name='repos_to_clone',
+ field=models.IntegerField(default=1),
+ ),
+ ]
+
diff --git a/lib/toaster/orm/models.py b/lib/toaster/orm/models.py
index a49f9a4..13bd117 100644
--- a/lib/toaster/orm/models.py
+++ b/lib/toaster/orm/models.py
@@ -437,6 +437,12 @@ class Build(models.Model):
# number of recipes parsed so far for this build
recipes_parsed = models.IntegerField(default=0)
+ # number of repos to clone for this build
+ repos_to_clone = models.IntegerField(default=1)
+
+ # number of repos cloned so far for this build
+ repos_cloned = models.IntegerField(default=0)
+
@staticmethod
def get_recent(project=None):
"""
@@ -667,6 +673,13 @@ class Build(models.Model):
else:
return False
+ def is_cloning(self):
+ """
+ True if the build is still cloning repos
+ """
+ return self.outcome == Build.IN_PROGRESS and \
+ self.repos_cloned < self.repos_to_clone
+
def is_parsing(self):
"""
True if the build is still parsing recipes
@@ -698,6 +711,8 @@ class Build(models.Model):
return 'Cancelling';
elif self.is_queued():
return 'Queued'
+ elif self.is_cloning():
+ return 'Cloning'
elif self.is_parsing():
return 'Parsing'
elif self.is_starting():
diff --git a/lib/toaster/toastergui/static/js/mrbsection.js b/lib/toaster/toastergui/static/js/mrbsection.js
index 73d0935..c0c5fa9 100644
--- a/lib/toaster/toastergui/static/js/mrbsection.js
+++ b/lib/toaster/toastergui/static/js/mrbsection.js
@@ -61,6 +61,12 @@ function mrbSectionInit(ctx){
return (cached.recipes_parsed_percentage !== build.recipes_parsed_percentage);
}
+ // returns true if the number of repos cloned/to clone changed
+ function cloneProgressChanged(build) {
+ var cached = getCached(build);
+ return (cached.repos_cloned_percentage !== build.repos_cloned_percentage);
+ }
+
function refreshMostRecentBuilds(){
libtoaster.getMostRecentBuilds(
libtoaster.ctx.mostRecentBuildsUrl,
@@ -100,6 +106,15 @@ function mrbSectionInit(ctx){
container.html(html);
}
+ else if (cloneProgressChanged(build)) {
+ // update the clone progress text
+ selector = '#repos-cloned-percentage-' + build.id;
+ $(selector).html(build.repos_cloned_percentage);
+
+ // update the recipe progress bar
+ selector = '#repos-cloned-percentage-bar-' + build.id;
+ $(selector).width(build.repos_cloned_percentage + '%');
+ }
else if (tasksProgressChanged(build)) {
// update the task progress text
selector = '#build-pc-done-' + build.id;
diff --git a/lib/toaster/toastergui/templates/mrb_section.html b/lib/toaster/toastergui/templates/mrb_section.html
index b761ffe..c5b9fe9 100644
--- a/lib/toaster/toastergui/templates/mrb_section.html
+++ b/lib/toaster/toastergui/templates/mrb_section.html
@@ -64,7 +64,9 @@
</div>
<div data-build-state="<%:state%>">
- <%if state == 'Parsing'%>
+ <%if state == 'Cloning'%>
+ <%include tmpl='#cloning-repos-build-template'/%>
+ <%else state == 'Parsing'%>
<%include tmpl='#parsing-recipes-build-template'/%>
<%else state == 'Queued'%>
<%include tmpl='#queued-build-template'/%>
@@ -98,6 +100,31 @@
</div>
</script>
+<!-- cloning repos build -->
+<script id="cloning-repos-build-template" type="text/x-jsrender">
+ <!-- progress bar and parse completion percentage -->
+ <div data-role="build-status" class="col-md-4 col-md-offset-1 progress-info">
+ <!-- progress bar -->
+ <div class="progress">
+ <div id="repos-cloned-percentage-bar-<%:id%>"
+ style="width: <%:repos_cloned_percentage%>%;"
+ class="progress-bar">
+ </div>
+ </div>
+ </div>
+
+ <div class="col-md-4 progress-info">
+ <!-- parse completion percentage -->
+ <span class="glyphicon glyphicon-question-sign get-help get-help-blue"
+ title="Toaster is cloning the repos required for your build">
+ </span>
+
+ Cloning <span id="repos-cloned-percentage-<%:id%>"><%:repos_cloned_percentage%></span>% complete
+
+ <%include tmpl='#cancel-template'/%>
+ </div>
+</script>
+
<!-- parsing recipes build -->
<script id="parsing-recipes-build-template" type="text/x-jsrender">
<!-- progress bar and parse completion percentage -->
diff --git a/lib/toaster/toastergui/widgets.py b/lib/toaster/toastergui/widgets.py
index 6b7b981..67c1ff9 100644
--- a/lib/toaster/toastergui/widgets.py
+++ b/lib/toaster/toastergui/widgets.py
@@ -511,6 +511,10 @@ class MostRecentBuildsView(View):
int((build_obj.recipes_parsed /
build_obj.recipes_to_parse) * 100)
+ build['repos_cloned_percentage'] = \
+ int((build_obj.repos_cloned /
+ build_obj.repos_to_clone) * 100)
+
tasks_complete_percentage = 0
if build_obj.outcome in (Build.SUCCEEDED, Build.FAILED):
tasks_complete_percentage = 100
--
1.9.1
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH 3/5] toaster: Add distro selection support
2017-06-27 20:44 [PATCH 0/5] toaster: cummulative 26062017 patch David Reyna
2017-06-27 20:44 ` [PATCH 1/5] toaster: address Django-1.10 API deprecations David Reyna
2017-06-27 20:44 ` [PATCH 2/5] toaster: git clone progress bar David Reyna
@ 2017-06-27 20:44 ` David Reyna
2017-06-27 20:44 ` [PATCH 4/5] toaster: large package set breaks sqlite query David Reyna
2017-06-27 20:44 ` [PATCH 5/5] toaster: test 'commit' first in get_vcs_reference David Reyna
4 siblings, 0 replies; 6+ messages in thread
From: David Reyna @ 2017-06-27 20:44 UTC (permalink / raw)
To: bitbake-devel
From: David Reyna <David.Reyna@windriver.com>
Add the ability to select a distro in the project page,
based on values from the Layer Index. Add a distro selection
page with the add layer feature, based on the add machine
page.
[YOCTO #10632]
Signed-off-by: David Reyna <David.Reyna@windriver.com>
---
lib/toaster/orm/management/commands/lsupdates.py | 19 +++++
lib/toaster/orm/migrations/0017_distro_clone.py | 25 ++++++
lib/toaster/orm/models.py | 31 ++++++++
lib/toaster/toastergui/api.py | 13 +++-
lib/toaster/toastergui/static/js/projectpage.js | 73 +++++++++++++++++
lib/toaster/toastergui/tables.py | 91 ++++++++++++++++++++++
lib/toaster/toastergui/templates/base.html | 1 +
.../toastergui/templates/baseprojectpage.html | 1 +
lib/toaster/toastergui/templates/distro_btn.html | 20 +++++
lib/toaster/toastergui/templates/project.html | 16 ++++
lib/toaster/toastergui/typeaheads.py | 30 +++++++
lib/toaster/toastergui/urls.py | 8 ++
12 files changed, 325 insertions(+), 3 deletions(-)
create mode 100644 bitbake/lib/toaster/orm/migrations/0017_distro_clone.py
create mode 100644 bitbake/lib/toaster/toastergui/templates/distro_btn.html
diff --git a/lib/toaster/orm/management/commands/lsupdates.py b/lib/toaster/orm/management/commands/lsupdates.py
index 90f07c9..0b0d4ff 100644
--- a/lib/toaster/orm/management/commands/lsupdates.py
+++ b/lib/toaster/orm/management/commands/lsupdates.py
@@ -23,6 +23,7 @@ from django.core.management.base import BaseCommand
from orm.models import LayerSource, Layer, Release, Layer_Version
from orm.models import LayerVersionDependency, Machine, Recipe
+from orm.models import Distro
import os
import sys
@@ -249,6 +250,24 @@ class Command(BaseCommand):
depends_on=lvd)
self.mini_progress("Layer version dependencies", i, total)
+ # update Distros
+ logger.info("Fetching distro information")
+ distros_info = _get_json_response(
+ apilinks['distros'] + "?filter=layerbranch__branch__name:%s" %
+ "OR".join(whitelist_branch_names))
+
+ total = len(distros_info)
+ for i, di in enumerate(distros_info):
+ distro, created = Distro.objects.get_or_create(
+ name=di['name'],
+ layer_version=Layer_Version.objects.get(
+ pk=li_layer_branch_id_to_toaster_lv_id[di['layerbranch']]))
+ distro.up_date = di['updated']
+ distro.name = di['name']
+ distro.description = di['description']
+ distro.save()
+ self.mini_progress("distros", i, total)
+
# update machines
logger.info("Fetching machine information")
machines_info = _get_json_response(
diff --git a/lib/toaster/orm/migrations/0017_distro_clone.py b/lib/toaster/orm/migrations/0017_distro_clone.py
new file mode 100644
index 0000000..d3c5901
--- /dev/null
+++ b/lib/toaster/orm/migrations/0017_distro_clone.py
@@ -0,0 +1,25 @@
+# -*- coding: utf-8 -*-
+from __future__ import unicode_literals
+
+from django.db import migrations, models
+
+class Migration(migrations.Migration):
+
+ dependencies = [
+ ('orm', '0016_clone_progress'),
+ ]
+
+ operations = [
+ migrations.CreateModel(
+ name='Distro',
+ fields=[
+ ('id', models.AutoField(verbose_name='ID', serialize=False, auto_created=True, primary_key=True)),
+ ('up_id', models.IntegerField(default=None, null=True)),
+ ('up_date', models.DateTimeField(default=None, null=True)),
+ ('name', models.CharField(max_length=255)),
+ ('description', models.CharField(max_length=255)),
+ ('layer_version', models.ForeignKey(to='orm.Layer_Version')),
+ ],
+ ),
+ ]
+
diff --git a/lib/toaster/orm/models.py b/lib/toaster/orm/models.py
index 13bd117..5c14727 100644
--- a/lib/toaster/orm/models.py
+++ b/lib/toaster/orm/models.py
@@ -321,6 +321,22 @@ class Project(models.Model):
return queryset
+ def get_available_distros(self):
+ """ Returns QuerySet of all Distros which are provided by the
+ Layers currently added to the Project """
+ queryset = Distro.objects.filter(
+ layer_version__in=self.get_project_layer_versions())
+
+ return queryset
+
+ def get_all_compatible_distros(self):
+ """ Returns QuerySet of all the compatible Wind River distros available to the
+ project including ones from Layers not currently added """
+ queryset = Distro.objects.filter(
+ layer_version__in=self.get_all_compatible_layer_versions())
+
+ return queryset
+
def get_available_recipes(self):
""" Returns QuerySet of all the recipes that are provided by layers
added to this project """
@@ -1795,6 +1811,21 @@ def signal_runbuilds():
except FileNotFoundError:
logger.info("Stopping existing runbuilds: no current process found")
+class Distro(models.Model):
+ search_allowed_fields = ["name", "description", "layer_version__layer__name"]
+ up_date = models.DateTimeField(null = True, default = None)
+
+ layer_version = models.ForeignKey('Layer_Version')
+ name = models.CharField(max_length=255)
+ description = models.CharField(max_length=255)
+
+ def get_vcs_distro_file_link_url(self):
+ path = self.name+'.conf'
+ return self.layer_version.get_vcs_file_link_url(path)
+
+ def __unicode__(self):
+ return "Distro " + self.name + "(" + self.description + ")"
+
django.db.models.signals.post_save.connect(invalidate_cache)
django.db.models.signals.post_delete.connect(invalidate_cache)
django.db.models.signals.m2m_changed.connect(invalidate_cache)
diff --git a/lib/toaster/toastergui/api.py b/lib/toaster/toastergui/api.py
index 5b03550..88d6aa7 100644
--- a/lib/toaster/toastergui/api.py
+++ b/lib/toaster/toastergui/api.py
@@ -874,6 +874,12 @@ class XhrProject(View):
machinevar.value = request.POST['machineName']
machinevar.save()
+ # Distro name change
+ if 'distroName' in request.POST:
+ distrovar = prj.projectvariable_set.get(name="DISTRO")
+ distrovar.value = request.POST['distroName']
+ distrovar.save()
+
return JsonResponse({"error": "ok"})
def get(self, request, *args, **kwargs):
@@ -960,10 +966,11 @@ class XhrProject(View):
except ProjectVariable.DoesNotExist:
data["machine"] = None
try:
- data["distro"] = project.projectvariable_set.get(
- name="DISTRO").value
+ data["distro"] = {"name":
+ project.projectvariable_set.get(
+ name="DISTRO").value}
except ProjectVariable.DoesNotExist:
- data["distro"] = "-- not set yet"
+ data["distro"] = None
data['error'] = "ok"
diff --git a/lib/toaster/toastergui/static/js/projectpage.js b/lib/toaster/toastergui/static/js/projectpage.js
index 21adf81..506471e 100644
--- a/lib/toaster/toastergui/static/js/projectpage.js
+++ b/lib/toaster/toastergui/static/js/projectpage.js
@@ -15,6 +15,13 @@ function projectPageInit(ctx) {
var machineInputForm = $("#machine-input-form");
var invalidMachineNameHelp = $("#invalid-machine-name-help");
+ var distroChangeInput = $("#distro-change-input");
+ var distroChangeBtn = $("#distro-change-btn");
+ var distroForm = $("#select-distro-form");
+ var distroChangeFormToggle = $("#change-distro-toggle");
+ var distroNameTitle = $("#project-distro-name");
+ var distroChangeCancel = $("#cancel-distro-change");
+
var freqBuildBtn = $("#freq-build-btn");
var freqBuildList = $("#freq-build-list");
@@ -26,6 +33,7 @@ function projectPageInit(ctx) {
var currentLayerAddSelection;
var currentMachineAddSelection = "";
+ var currentDistroAddSelection = "";
var urlParams = libtoaster.parseUrlParams();
@@ -45,6 +53,17 @@ function projectPageInit(ctx) {
updateMachineName(prjInfo.machine.name);
}
+ /* If we're receiving a distro set from the url and it's different from
+ * our current distro then activate set machine sequence.
+ */
+ if (urlParams.hasOwnProperty('setDistro') &&
+ urlParams.setDistro !== prjInfo.distro.name){
+ distroChangeInput.val(urlParams.setDistro);
+ distroChangeBtn.click();
+ } else {
+ updateDistroName(prjInfo.distro.name);
+ }
+
/* Now we're really ready show the page */
$("#project-page").show();
@@ -278,6 +297,60 @@ function projectPageInit(ctx) {
});
+ /* Change distro functionality */
+
+ distroChangeFormToggle.click(function(){
+ distroForm.slideDown();
+ distroNameTitle.hide();
+ $(this).hide();
+ });
+
+ distroChangeCancel.click(function(){
+ distroForm.slideUp(function(){
+ distroNameTitle.show();
+ distroChangeFormToggle.show();
+ });
+ });
+
+ function updateDistroName(distroName){
+ distroChangeInput.val(distroName);
+ distroNameTitle.text(distroName);
+ }
+
+ libtoaster.makeTypeahead(distroChangeInput,
+ libtoaster.ctx.distrosTypeAheadUrl,
+ { }, function(item){
+ currentDistroAddSelection = item.name;
+ distroChangeBtn.removeAttr("disabled");
+ });
+
+ distroChangeBtn.click(function(e){
+ e.preventDefault();
+ /* We accept any value regardless of typeahead selection or not */
+ if (distroChangeInput.val().length === 0)
+ return;
+
+ currentDistroAddSelection = distroChangeInput.val();
+
+ libtoaster.editCurrentProject(
+ { distroName : currentDistroAddSelection },
+ function(){
+ /* Success machine changed */
+ updateDistroName(currentDistroAddSelection);
+ distroChangeCancel.click();
+
+ /* Show the alert message */
+ var message = $('<span>You have changed the distro to: <strong><span id="notify-machine-name"></span></strong></span>');
+ message.find("#notify-machine-name").text(currentDistroAddSelection);
+ libtoaster.showChangeNotification(message);
+ },
+ function(){
+ /* Failed machine changed */
+ console.warn("Failed to change distro");
+ });
+ });
+
+
/* Change release functionality */
function updateProjectRelease(release){
releaseTitle.text(release.description);
diff --git a/lib/toaster/toastergui/tables.py b/lib/toaster/toastergui/tables.py
index e2d23c1..dca2fa2 100644
--- a/lib/toaster/toastergui/tables.py
+++ b/lib/toaster/toastergui/tables.py
@@ -23,6 +23,7 @@ from toastergui.widgets import ToasterTable
from orm.models import Recipe, ProjectLayer, Layer_Version, Machine, Project
from orm.models import CustomImageRecipe, Package, Target, Build, LogMessage, Task
from orm.models import CustomImagePackage, Package_DependencyManager
+from orm.models import Distro
from django.db.models import Q, Max, Sum, Count, When, Case, Value, IntegerField
from django.conf.urls import url
from django.core.urlresolvers import reverse, resolve
@@ -1536,3 +1537,93 @@ class ProjectBuildsTable(BuildsTable):
context['build_in_progress_none_completed'] = False
return context
+
+
+class DistrosTable(ToasterTable):
+ """Table of Distros in Toaster"""
+
+ def __init__(self, *args, **kwargs):
+ super(DistrosTable, self).__init__(*args, **kwargs)
+ self.empty_state = "Toaster has no distro information for this project. Sadly, distro information cannot be obtained from builds, so this page will remain empty."
+ self.title = "Compatible Distros"
+ self.default_orderby = "name"
+
+ def get_context_data(self, **kwargs):
+ context = super(DistrosTable, self).get_context_data(**kwargs)
+ context['project'] = Project.objects.get(pk=kwargs['pid'])
+ return context
+
+ def setup_filters(self, *args, **kwargs):
+ project = Project.objects.get(pk=kwargs['pid'])
+
+ in_current_project_filter = TableFilter(
+ "in_current_project",
+ "Filter by project Distros"
+ )
+
+ in_project_action = TableFilterActionToggle(
+ "in_project",
+ "Distro provided by layers added to this project",
+ ProjectFilters.in_project(self.project_layers)
+ )
+
+ not_in_project_action = TableFilterActionToggle(
+ "not_in_project",
+ "Distros provided by layers not added to this project",
+ ProjectFilters.not_in_project(self.project_layers)
+ )
+
+ in_current_project_filter.add_action(in_project_action)
+ in_current_project_filter.add_action(not_in_project_action)
+ self.add_filter(in_current_project_filter)
+
+ def setup_queryset(self, *args, **kwargs):
+ prj = Project.objects.get(pk = kwargs['pid'])
+ self.queryset = prj.get_all_compatible_distros()
+ self.queryset = self.queryset.order_by(self.default_orderby)
+
+ self.static_context_extra['current_layers'] = \
+ self.project_layers = \
+ prj.get_project_layer_versions(pk=True)
+
+ def setup_columns(self, *args, **kwargs):
+
+ self.add_column(title="Distro",
+ hideable=False,
+ orderable=True,
+ field_name="name")
+
+ self.add_column(title="Description",
+ field_name="description")
+
+ layer_link_template = '''
+ <a href="{% url 'layerdetails' extra.pid data.layer_version.id %}">
+ {{data.layer_version.layer.name}}</a>
+ '''
+
+ self.add_column(title="Layer",
+ static_data_name="layer_version__layer__name",
+ static_data_template=layer_link_template,
+ orderable=True)
+
+ self.add_column(title="Git revision",
+ help_text="The Git branch, tag or commit. For the layers from the OpenEmbedded layer source, the revision is always the branch compatible with the Yocto Project version you selected for this project",
+ hidden=True,
+ field_name="layer_version__get_vcs_reference")
+
+ wrtemplate_file_template = '''<code>conf/machine/{{data.name}}.conf</code>
+ <a href="{{data.get_vcs_machine_file_link_url}}" target="_blank"><span class="glyphicon glyphicon-new-window"></i></a>'''
+
+ self.add_column(title="Distro file",
+ hidden=True,
+ static_data_name="templatefile",
+ static_data_template=wrtemplate_file_template)
+
+
+ self.add_column(title="Select",
+ help_text="Sets the selected distro to the project",
+ hideable=False,
+ filter_name="in_current_project",
+ static_data_name="add-del-layers",
+ static_data_template='{% include "distro_btn.html" %}')
+
diff --git a/lib/toaster/toastergui/templates/base.html b/lib/toaster/toastergui/templates/base.html
index 11c6f91..0fbe17b 100644
--- a/lib/toaster/toastergui/templates/base.html
+++ b/lib/toaster/toastergui/templates/base.html
@@ -49,6 +49,7 @@
recipesTypeAheadUrl: {% url 'xhr_recipestypeahead' project.id as paturl%}{{paturl|json}},
layersTypeAheadUrl: {% url 'xhr_layerstypeahead' project.id as paturl%}{{paturl|json}},
machinesTypeAheadUrl: {% url 'xhr_machinestypeahead' project.id as paturl%}{{paturl|json}},
+ distrosTypeAheadUrl: {% url 'xhr_distrostypeahead' project.id as paturl%}{{paturl|json}},
projectBuildsUrl: {% url 'projectbuilds' project.id as pburl %}{{pburl|json}},
xhrCustomRecipeUrl : "{% url 'xhr_customrecipe' %}",
projectId : {{project.id}},
diff --git a/lib/toaster/toastergui/templates/baseprojectpage.html b/lib/toaster/toastergui/templates/baseprojectpage.html
index 8427d25..f2bb2eb 100644
--- a/lib/toaster/toastergui/templates/baseprojectpage.html
+++ b/lib/toaster/toastergui/templates/baseprojectpage.html
@@ -32,6 +32,7 @@ $(document).ready(function(){
<li><a href="{% url 'projectsoftwarerecipes' project.id %}">Software recipes</a></li>
<li><a href="{% url 'projectmachines' project.id %}">Machines</a></li>
<li><a href="{% url 'projectlayers' project.id %}">Layers</a></li>
+ <li><a href="{% url 'projectdistros' project.id %}">Distros</a></li>
<li class="nav-header">Extra configuration</li>
<li><a href="{% url 'projectconf' project.id %}">BitBake variables</a></li>
diff --git a/lib/toaster/toastergui/templates/distro_btn.html b/lib/toaster/toastergui/templates/distro_btn.html
new file mode 100644
index 0000000..fac7947
--- /dev/null
+++ b/lib/toaster/toastergui/templates/distro_btn.html
@@ -0,0 +1,20 @@
+<a href="{% url 'project' extra.pid %}?setDistro={{data.name}}" class="btn btn-default btn-block layer-exists-{{data.layer_version.id}}"
+ {% if data.layer_version.pk not in extra.current_layers %}
+ style="display:none;"
+ {% endif %}>
+ Set distro</a>
+<a class="btn btn-default btn-block layerbtn layer-add-{{data.layer_version.id}}" data-layer='{
+ "id": {{data.layer_version.id}},
+ "name": "{{data.layer_version.layer.name}}",
+ "xhrLayerUrl": "{% url "xhr_layer" extra.pid data.pk %}",
+ "layerdetailurl": "{%url 'layerdetails' extra.pid data.layer_version.id %}"
+ }' data-directive="add"
+ {% if data.layer_version.pk in extra.current_layers %}
+ style="display:none;"
+ {% endif %}
+>
+ <span class="glyphicon glyphicon-plus"></span>
+ Add layer
+ <span class="glyphicon glyphicon-question-sign get-help" title="To select this distro, you must first add the {{data.layer_version.layer.name}} layer to your project"></i>
+</a>
+
diff --git a/lib/toaster/toastergui/templates/project.html b/lib/toaster/toastergui/templates/project.html
index ab7e665..11603d1 100644
--- a/lib/toaster/toastergui/templates/project.html
+++ b/lib/toaster/toastergui/templates/project.html
@@ -77,6 +77,22 @@
</form>
</div>
+ <div class="well well-transparent" id="distro-section">
+ <h3>Distro</h3>
+
+ <p class="lead"><span id="project-distro-name"></span> <span class="glyphicon glyphicon-edit" id="change-distro-toggle"></span></p>
+
+ <form id="select-distro-form" style="display:none;" class="form-inline">
+ <span class="help-block">Distro suggestions come from the Layer Index</a></span>
+ <div class="form-group">
+ <input class="form-control" id="distro-change-input" autocomplete="off" value="" data-provide="typeahead" data-minlength="1" data-autocomplete="off" type="text">
+ </div>
+ <button id="distro-change-btn" class="btn btn-default" type="button">Save</button>
+ <a href="#" id="cancel-distro-change" class="btn btn-link">Cancel</a>
+ <p class="form-link"><a href="{% url 'projectdistros' project.id %}">View compatible distros</a></p>
+ </form>
+ </div>
+
<div class="well well-transparent">
<h3>Most built recipes</h3>
diff --git a/lib/toaster/toastergui/typeaheads.py b/lib/toaster/toastergui/typeaheads.py
index 58c650f..5aa0f8d 100644
--- a/lib/toaster/toastergui/typeaheads.py
+++ b/lib/toaster/toastergui/typeaheads.py
@@ -100,6 +100,36 @@ class MachinesTypeAhead(ToasterTypeAhead):
return results
+class DistrosTypeAhead(ToasterTypeAhead):
+ """ Typeahead for all the distros available in the current project's
+ configuration """
+ def __init__(self):
+ super(DistrosTypeAhead, self).__init__()
+
+ def apply_search(self, search_term, prj, request):
+ distros = prj.get_available_distros()
+ distros = distros.order_by("name")
+
+ primary_results = distros.filter(name__istartswith=search_term)
+ secondary_results = distros.filter(name__icontains=search_term).exclude(pk__in=primary_results)
+ tertiary_results = distros.filter(layer_version__layer__name__icontains=search_term).exclude(pk__in=primary_results).exclude(pk__in=secondary_results)
+
+ results = []
+
+ for distro in list(primary_results) + list(secondary_results) + list(tertiary_results):
+
+ detail = "[ %s ]" % (distro.layer_version.layer.name)
+ needed_fields = {
+ 'id' : distro.pk,
+ 'name' : distro.name,
+ 'detail' : detail,
+ }
+
+ results.append(needed_fields)
+
+ return results
+
+
class RecipesTypeAhead(ToasterTypeAhead):
""" Typeahead for all the recipes available in the current project's
configuration """
diff --git a/lib/toaster/toastergui/urls.py b/lib/toaster/toastergui/urls.py
index 1ad79a4..6aebc3f 100644
--- a/lib/toaster/toastergui/urls.py
+++ b/lib/toaster/toastergui/urls.py
@@ -158,6 +158,11 @@ urlpatterns = [
name=tables.LayerMachinesTable.__name__.lower()),
+ url(r'^project/(?P<pid>\d+)/distros/$',
+ tables.DistrosTable.as_view(template_name="generic-toastertable-page.html"),
+ name="projectdistros"),
+
+
url(r'^project/(?P<pid>\d+)/customrecipe/(?P<custrecipeid>\d+)/selectpackages/$',
tables.SelectPackagesTable.as_view(), name="recipeselectpackages"),
@@ -187,6 +192,9 @@ urlpatterns = [
typeaheads.GitRevisionTypeAhead.as_view(),
name='xhr_gitrevtypeahead'),
+ url(r'^xhr_typeahead/(?P<pid>\d+)/distros$',
+ typeaheads.DistrosTypeAhead.as_view(), name='xhr_distrostypeahead'),
+
url(r'^xhr_testreleasechange/(?P<pid>\d+)$', views.xhr_testreleasechange,
name='xhr_testreleasechange'),
url(r'^xhr_configvaredit/(?P<pid>\d+)$', views.xhr_configvaredit,
--
1.9.1
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH 4/5] toaster: large package set breaks sqlite query
2017-06-27 20:44 [PATCH 0/5] toaster: cummulative 26062017 patch David Reyna
` (2 preceding siblings ...)
2017-06-27 20:44 ` [PATCH 3/5] toaster: Add distro selection support David Reyna
@ 2017-06-27 20:44 ` David Reyna
2017-06-27 20:44 ` [PATCH 5/5] toaster: test 'commit' first in get_vcs_reference David Reyna
4 siblings, 0 replies; 6+ messages in thread
From: David Reyna @ 2017-06-27 20:44 UTC (permalink / raw)
To: bitbake-devel
From: David Reyna <David.Reyna@windriver.com>
If you build a project with a large package set, you will get a crash
in "views.py" when the dashboard attempts to fetch the package set to
calculate the package count and size. This is a sqlite limitation, and
it fails with as few as 1220 packages.
[YOCTO #11717]
Signed-off-by: David Reyna <David.Reyna@windriver.com>
---
lib/toaster/toastergui/views.py | 13 +++++++++----
1 file changed, 9 insertions(+), 4 deletions(-)
diff --git a/lib/toaster/toastergui/views.py b/lib/toaster/toastergui/views.py
index 75c5911..5720b9d 100755
--- a/lib/toaster/toastergui/views.py
+++ b/lib/toaster/toastergui/views.py
@@ -457,10 +457,15 @@ def builddashboard( request, build_id ):
npkg = 0
pkgsz = 0
package = None
- for package in Package.objects.filter(id__in = [x.package_id for x in t.target_installed_package_set.all()]):
- pkgsz = pkgsz + package.size
- if package.installed_name:
- npkg = npkg + 1
+ # Chunk the query to avoid "too many SQL variables" error
+ package_set = t.target_installed_package_set.all()
+ package_set_len = len(package_set)
+ for ps_start in range(0,package_set_len,500):
+ ps_stop = min(ps_start+500,package_set_len)
+ for package in Package.objects.filter(id__in = [x.package_id for x in package_set[ps_start:ps_stop]]):
+ pkgsz = pkgsz + package.size
+ if package.installed_name:
+ npkg = npkg + 1
elem['npkg'] = npkg
elem['pkgsz'] = pkgsz
ti = Target_Image_File.objects.filter(target_id = t.id)
--
1.9.1
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH 5/5] toaster: test 'commit' first in get_vcs_reference
2017-06-27 20:44 [PATCH 0/5] toaster: cummulative 26062017 patch David Reyna
` (3 preceding siblings ...)
2017-06-27 20:44 ` [PATCH 4/5] toaster: large package set breaks sqlite query David Reyna
@ 2017-06-27 20:44 ` David Reyna
4 siblings, 0 replies; 6+ messages in thread
From: David Reyna @ 2017-06-27 20:44 UTC (permalink / raw)
To: bitbake-devel
From: David Reyna <David.Reyna@windriver.com>
The 'commit' value should be tested and used first when resolving the ref
for a layer, since that is an explicit override in each layer index
'layeritem' record. If should take precedence over more-global 'branch'
and 'release' values, instead of being last.
[YOCTO #11515]
Signed-off-by: David Reyna <David.Reyna@windriver.com>
---
lib/toaster/orm/models.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/lib/toaster/orm/models.py b/lib/toaster/orm/models.py
index 5c14727..630e4a0 100644
--- a/lib/toaster/orm/models.py
+++ b/lib/toaster/orm/models.py
@@ -1516,12 +1516,12 @@ class Layer_Version(models.Model):
return self._handle_url_path(self.layer.vcs_web_tree_base_url, '')
def get_vcs_reference(self):
+ if self.commit is not None and len(self.commit) > 0:
+ return self.commit
if self.branch is not None and len(self.branch) > 0:
return self.branch
if self.release is not None:
return self.release.name
- if self.commit is not None and len(self.commit) > 0:
- return self.commit
return 'N/A'
def get_detailspage_url(self, project_id=None):
--
1.9.1
^ permalink raw reply related [flat|nested] 6+ messages in thread
end of thread, other threads:[~2017-06-27 20:53 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-06-27 20:44 [PATCH 0/5] toaster: cummulative 26062017 patch David Reyna
2017-06-27 20:44 ` [PATCH 1/5] toaster: address Django-1.10 API deprecations David Reyna
2017-06-27 20:44 ` [PATCH 2/5] toaster: git clone progress bar David Reyna
2017-06-27 20:44 ` [PATCH 3/5] toaster: Add distro selection support David Reyna
2017-06-27 20:44 ` [PATCH 4/5] toaster: large package set breaks sqlite query David Reyna
2017-06-27 20:44 ` [PATCH 5/5] toaster: test 'commit' first in get_vcs_reference David Reyna
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.