* [PATCH 01/18] toaster: Upgrade to Django 1.8.6 and remove South
2015-12-10 3:56 [PATCH 00/18] toaster: upgrade to Django 1.8 brian avery
@ 2015-12-10 3:56 ` brian avery
2015-12-10 3:56 ` [PATCH 03/18] toaster: Fix references to app paths brian avery
` (15 subsequent siblings)
16 siblings, 0 replies; 19+ messages in thread
From: brian avery @ 2015-12-10 3:56 UTC (permalink / raw)
To: bitbake-devel
From: Elliot Smith <elliot.smith@intel.com>
Upgrade Django to long-term support version.
Django now provides its own migration framework, so remove
requirement for South.
[YOCTO #8364]
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
---
bin/toaster | 8 ++------
lib/toaster/toastermain/settings.py | 1 -
toaster-requirements.txt | 3 +--
3 files changed, 3 insertions(+), 9 deletions(-)
diff --git a/bin/toaster b/bin/toaster
index 6d1ec05..528e17e 100755
--- a/bin/toaster
+++ b/bin/toaster
@@ -176,15 +176,11 @@ notify_chldexit() {
verify_prereq() {
# Verify prerequisites
- if ! echo "import django; print (1,) == django.VERSION[0:1] and django.VERSION[1:2][0] in (6,)" | python 2>/dev/null | grep True >/dev/null; then
- printf "This program needs Django 1.6. Please install with\n\npip install django==1.6\n"
+ if ! echo "import django; print (1,) == django.VERSION[0:1] and django.VERSION[1:2][0] in (8,)" | python 2>/dev/null | grep True >/dev/null; then
+ printf "This program needs Django 1.8. Please install with\n\npip install django<=1.8.7\n"
return 2
fi
- if ! echo "import south; print reduce(lambda x, y: 2 if x==2 else 0 if x == 0 else y, map(lambda x: 1+cmp(x[1]-x[0],0), zip([0,8,4], map(int,south.__version__.split(\".\"))))) > 0" | python 2>/dev/null | grep True >/dev/null; then
- printf "This program needs South 0.8.4. Please install with\n\npip install south==0.8.4\n"
- return 2
- fi
return 0
}
diff --git a/lib/toaster/toastermain/settings.py b/lib/toaster/toastermain/settings.py
index de686b2..7c559bc 100644
--- a/lib/toaster/toastermain/settings.py
+++ b/lib/toaster/toastermain/settings.py
@@ -276,7 +276,6 @@ INSTALLED_APPS = (
'django.contrib.humanize',
'bldcollector',
'toastermain',
- 'south',
)
diff --git a/toaster-requirements.txt b/toaster-requirements.txt
index 1d7d21b..5d34494 100644
--- a/toaster-requirements.txt
+++ b/toaster-requirements.txt
@@ -1,5 +1,4 @@
-Django==1.6
-South==0.8.4
+Django==1.8.6
argparse==1.2.1
wsgiref==0.1.2
beautifulsoup4>=4.4.0
--
1.9.1
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH 03/18] toaster: Fix references to app paths
2015-12-10 3:56 [PATCH 00/18] toaster: upgrade to Django 1.8 brian avery
2015-12-10 3:56 ` [PATCH 01/18] toaster: Upgrade to Django 1.8.6 and remove South brian avery
@ 2015-12-10 3:56 ` brian avery
2015-12-10 3:56 ` [PATCH 04/18] toaster: Create default project with get_or_create* method brian avery
` (14 subsequent siblings)
16 siblings, 0 replies; 19+ messages in thread
From: brian avery @ 2015-12-10 3:56 UTC (permalink / raw)
To: bitbake-devel
From: Ed Bartosh <ed.bartosh@linux.intel.com>
The buildinfohelper runs outside of Django, but still needs
access to our Django app classes.
Previously, the imports referenced the toaster.* app, which worked
fine. But in Django 1.8, this causes an error about the same
module being loaded multiple times from different paths.
Change the paths to our Django modules so they don't cause
this error to be thrown. We can do this as we've added our
application libraries to sys.path in the buildinfohelper anyway.
[YOCTO #8364]
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
---
lib/bb/ui/buildinfohelper.py | 16 ++++++++--------
1 file changed, 8 insertions(+), 8 deletions(-)
diff --git a/lib/bb/ui/buildinfohelper.py b/lib/bb/ui/buildinfohelper.py
index 1d41bba..c099551 100644
--- a/lib/bb/ui/buildinfohelper.py
+++ b/lib/bb/ui/buildinfohelper.py
@@ -33,14 +33,14 @@ def _configure_toaster():
sys.path.append(os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(__file__))), 'toaster'))
_configure_toaster()
-from toaster.orm.models import Build, Task, Recipe, Layer_Version, Layer, Target, LogMessage, HelpText
-from toaster.orm.models import Target_Image_File, BuildArtifact
-from toaster.orm.models import Variable, VariableHistory
-from toaster.orm.models import Package, Package_File, Target_Installed_Package, Target_File
-from toaster.orm.models import Task_Dependency, Package_Dependency
-from toaster.orm.models import Recipe_Dependency
-
-from toaster.orm.models import Project
+from orm.models import Build, Task, Recipe, Layer_Version, Layer, Target, LogMessage, HelpText
+from orm.models import Target_Image_File, BuildArtifact
+from orm.models import Variable, VariableHistory
+from orm.models import Package, Package_File, Target_Installed_Package, Target_File
+from orm.models import Task_Dependency, Package_Dependency
+from orm.models import Recipe_Dependency
+
+from orm.models import Project
from bldcontrol.models import BuildEnvironment, BuildRequest
from bb.msg import BBLogFormatter as formatter
--
1.9.1
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH 04/18] toaster: Create default project with get_or_create* method
2015-12-10 3:56 [PATCH 00/18] toaster: upgrade to Django 1.8 brian avery
2015-12-10 3:56 ` [PATCH 01/18] toaster: Upgrade to Django 1.8.6 and remove South brian avery
2015-12-10 3:56 ` [PATCH 03/18] toaster: Fix references to app paths brian avery
@ 2015-12-10 3:56 ` brian avery
2015-12-10 3:56 ` [PATCH 05/18] toaster: Start Django machinery for database access brian avery
` (13 subsequent siblings)
16 siblings, 0 replies; 19+ messages in thread
From: brian avery @ 2015-12-10 3:56 UTC (permalink / raw)
To: bitbake-devel
From: Elliot Smith <elliot.smith@intel.com>
Rather than maintain data as part of the migrations (as was
done for the default project previously), create the default
(cli builds) project on demand as a by-product of getting
it from the database.
[YOCTO #8364]
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
---
lib/bb/ui/buildinfohelper.py | 9 +++++++--
lib/toaster/orm/models.py | 23 +++++++++++++++--------
lib/toaster/toastergui/views.py | 2 +-
3 files changed, 23 insertions(+), 11 deletions(-)
diff --git a/lib/bb/ui/buildinfohelper.py b/lib/bb/ui/buildinfohelper.py
index c099551..87e03e7 100644
--- a/lib/bb/ui/buildinfohelper.py
+++ b/lib/bb/ui/buildinfohelper.py
@@ -146,7 +146,7 @@ class ORMWrapper(object):
prj = Project.objects.get(pk = project_id)
else: # this build was triggered by a legacy system, or command line interactive mode
- prj = Project.objects.get_default_project()
+ prj = Project.objects.get_or_create_default_project()
logger.debug(1, "buildinfohelper: project is not specified, defaulting to %s" % prj)
@@ -308,6 +308,11 @@ class ORMWrapper(object):
# then we are wholly responsible for the data
# and therefore we return the 'real' recipe rather than the build
# history copy of the recipe.
+ if recipe_information['layer_version'].build is not None and \
+ recipe_information['layer_version'].build.project == \
+ Project.objects.get_or_create_default_project():
+ return recipe
+
if built_recipe is None:
return recipe
@@ -345,7 +350,7 @@ class ORMWrapper(object):
# If we're doing a command line build then associate this new layer with the
# project to avoid it 'contaminating' toaster data
project = None
- if build_obj.project == Project.objects.get_default_project():
+ if build_obj.project == Project.objects.get_or_create_default_project():
project = build_obj.project
layer_version_object, _ = Layer_Version.objects.get_or_create(
diff --git a/lib/toaster/orm/models.py b/lib/toaster/orm/models.py
index 0174233..4a868e7 100644
--- a/lib/toaster/orm/models.py
+++ b/lib/toaster/orm/models.py
@@ -91,18 +91,25 @@ class ProjectManager(models.Manager):
return prj
- def create(self, *args, **kwargs):
- raise Exception("Invalid call to Project.objects.create. Use Project.objects.create_project() to create a project")
-
# return single object with is_default = True
- def get_default_project(self):
+ def get_or_create_default_project(self):
projects = super(ProjectManager, self).filter(is_default = True)
+
if len(projects) > 1:
- raise Exception("Inconsistent project data: multiple " +
- "default projects (i.e. with is_default=True)")
+ raise Exception('Inconsistent project data: multiple ' +
+ 'default projects (i.e. with is_default=True)')
elif len(projects) < 1:
- raise Exception("Inconsistent project data: no default project found")
- return projects[0]
+ options = {
+ 'name': 'Command line builds',
+ 'short_description': 'Project for builds started outside Toaster',
+ 'is_default': True
+ }
+ project = Project.objects.create(**options)
+ project.save()
+
+ return project
+ else:
+ return projects[0]
class Project(models.Model):
search_allowed_fields = ['name', 'short_description', 'release__name', 'release__branch_name']
diff --git a/lib/toaster/toastergui/views.py b/lib/toaster/toastergui/views.py
index 243bb09..16f27b8 100755
--- a/lib/toaster/toastergui/views.py
+++ b/lib/toaster/toastergui/views.py
@@ -73,7 +73,7 @@ class MimeTypeFinder(object):
def landing(request):
# in build mode, we redirect to the command-line builds page
# if there are any builds for the default (cli builds) project
- default_project = Project.objects.get_default_project()
+ default_project = Project.objects.get_or_create_default_project()
default_project_builds = Build.objects.filter(project = default_project)
# we only redirect to projects page if there is a user-generated project
--
1.9.1
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH 05/18] toaster: Start Django machinery for database access
2015-12-10 3:56 [PATCH 00/18] toaster: upgrade to Django 1.8 brian avery
` (2 preceding siblings ...)
2015-12-10 3:56 ` [PATCH 04/18] toaster: Create default project with get_or_create* method brian avery
@ 2015-12-10 3:56 ` brian avery
2015-12-10 3:56 ` [PATCH 06/18] toaster: rename get_query_set -> get_queryset brian avery
` (12 subsequent siblings)
16 siblings, 0 replies; 19+ messages in thread
From: brian avery @ 2015-12-10 3:56 UTC (permalink / raw)
To: bitbake-devel
From: Elliot Smith <elliot.smith@intel.com>
buildinfohelper, with the new import paths for our Django models
and Django 1.8, was not getting an active connection to the database.
In buildinfohelper, call django.setup() explicitly to make sure
that the database connection is ready and models can be queried
and saved.
[YOCTO #8364]
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
---
lib/bb/ui/buildinfohelper.py | 3 +++
1 file changed, 3 insertions(+)
diff --git a/lib/bb/ui/buildinfohelper.py b/lib/bb/ui/buildinfohelper.py
index 87e03e7..04ce23a 100644
--- a/lib/bb/ui/buildinfohelper.py
+++ b/lib/bb/ui/buildinfohelper.py
@@ -24,6 +24,7 @@ import os
os.environ["DJANGO_SETTINGS_MODULE"] = "toaster.toastermain.settings"
+import django
from django.utils import timezone
@@ -33,6 +34,8 @@ def _configure_toaster():
sys.path.append(os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(__file__))), 'toaster'))
_configure_toaster()
+django.setup()
+
from orm.models import Build, Task, Recipe, Layer_Version, Layer, Target, LogMessage, HelpText
from orm.models import Target_Image_File, BuildArtifact
from orm.models import Variable, VariableHistory
--
1.9.1
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH 00/18] toaster: upgrade to Django 1.8
@ 2015-12-10 3:56 brian avery
2015-12-10 3:56 ` [PATCH 01/18] toaster: Upgrade to Django 1.8.6 and remove South brian avery
` (16 more replies)
0 siblings, 17 replies; 19+ messages in thread
From: brian avery @ 2015-12-10 3:56 UTC (permalink / raw)
To: bitbake-devel
This rather large patchset upgrades toaster to work with Django 1.8.
It was tested with Django 1.8.6 and 1.8.7.
yay for LTS!
-b
The following changes since commit 2cb81fb08395ec9cd68b02bc48391c9610eac9d9:
toasterui: process SetBRBE event (2015-12-09 19:45:13 -0800)
are available in the git repository at:
git://git.yoctoproject.org/poky-contrib bavery/submit/ed/toaster/django_upgrade-8364
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=bavery/submit/ed/toaster/django_upgrade-8364
Ed Bartosh (9):
toaster: Remove South migrations
toaster: Fix references to app paths
toaster: rename get_query_set -> get_queryset
toaster: use OneToOneField instead of ForeignKey
toaster: fixed format strings
toaster: work around 'database is locked' error
toaster: removed extra calls of migrate
toaster: monkey patch Queryset
toaster: rework checking of Django version
Elliot Smith (9):
toaster: Upgrade to Django 1.8.6 and remove South
toaster: Create default project with get_or_create* method
toaster: Start Django machinery for database access
toaster: Update API used to make runbuilds methods run in transactions
toaster: Prevent deprecation warnings for RedirectView
toaster: Update deprecated manage.py command
toaster: Check Django version against toaster-requirements.txt
toaster: Remove compatible_layerversions() method
toaster: Amend regex for MySQL database URLs
bin/toaster | 44 +-
lib/bb/ui/buildinfohelper.py | 26 +-
lib/toaster/bldcollector/urls.py | 1 -
lib/toaster/bldcontrol/bbcontroller.py | 4 +-
lib/toaster/bldcontrol/localhostbecontroller.py | 13 +-
.../bldcontrol/management/commands/runbuilds.py | 6 +-
lib/toaster/bldcontrol/migrations/0001_initial.py | 265 +++---
...onment_sourcedir__add_field_buildenvironment.py | 106 ---
.../0003_auto__add_field_brlayer_dirpath.py | 99 ---
.../bldcontrol/migrations/0004_loadinitialdata.py | 104 ---
.../migrations/0005_auto__add_brerror.py | 112 ---
.../migrations/0006_auto__add_brbitbake.py | 128 ---
...st_environment__chg_field_buildrequest_build.py | 145 ----
.../bldcontrol/migrations/0008_brarchive.py | 138 ---
.../0009_auto__add_field_brlayer_layer_version.py | 180 ----
lib/toaster/bldcontrol/models.py | 2 +-
lib/toaster/bldcontrol/tests.py | 6 +-
lib/toaster/orm/migrations/0001_initial.py | 928 ++++++++++++---------
.../0002_auto__add_field_build_timespent.py | 180 ----
lib/toaster/orm/migrations/0003_timespent.py | 182 ----
.../0004_auto__add_field_package_installed_name.py | 181 ----
...e__add_target_file__add_field_variablehistor.py | 281 -------
...ge_size__add_field_target_license_manifest_p.py | 235 ------
.../orm/migrations/0007_auto__add_helptext.py | 214 -----
...story_operation__chg_field_recipe_descriptio.py | 225 -----
..._add_projectlayer__add_projecttarget__add_pr.py | 286 -------
...anch__add_field_project_short_description__a.py | 257 ------
.../0011_auto__add_field_projectlayer_dirpath.py | 242 ------
...layer_optional__add_field_projecttarget_task.py | 252 ------
...erversiondependency__add_unique_layerversion.py | 710 ----------------
...mmary__chg_field_layer_summary__chg_field_re.py | 336 --------
...web_url__add_field_layer_vcs_web_tree_base_u.py | 336 --------
...lptext__chg_field_release_branch__add_index_.py | 359 --------
...faultlayer__add_releaselayersourcepriority__.py | 396 ---------
.../0018_auto__add_field_layer_version_project.py | 331 --------
.../orm/migrations/0019_auto__add_buildartifact.py | 342 --------
...ion_local_path__add_field_recipe_pathflags__.py | 361 --------
...ect__chg_field_project_bitbake_version__chg_.py | 371 --------
...k__add_field_layer_version_local_path__del_f.py | 343 --------
...ings_no__del_field_build_errors_no__del_fiel.py | 353 --------
.../0024_auto__add_field_recipe_is_image.py | 338 --------
.../0025_auto__add_field_project_is_default.py | 346 --------
.../orm/migrations/0026_set_default_project.py | 374 ---------
...e__add_unique_customimagerecipe_name_project.py | 375 ---------
.../0028_auto__chg_field_logmessage_message.py | 345 --------
lib/toaster/orm/models.py | 108 ++-
lib/toaster/toastergui/urls.py | 2 +-
lib/toaster/toastergui/views.py | 24 +-
lib/toaster/toastermain/settings.py | 4 +-
lib/toaster/toastermain/urls.py | 2 +-
toaster-requirements.txt | 3 +-
51 files changed, 779 insertions(+), 10222 deletions(-)
delete mode 100644 bitbake/lib/toaster/bldcontrol/migrations/0002_auto__add_field_buildenvironment_sourcedir__add_field_buildenvironment.py
delete mode 100644 bitbake/lib/toaster/bldcontrol/migrations/0003_auto__add_field_brlayer_dirpath.py
delete mode 100644 bitbake/lib/toaster/bldcontrol/migrations/0004_loadinitialdata.py
delete mode 100644 bitbake/lib/toaster/bldcontrol/migrations/0005_auto__add_brerror.py
delete mode 100644 bitbake/lib/toaster/bldcontrol/migrations/0006_auto__add_brbitbake.py
delete mode 100644 bitbake/lib/toaster/bldcontrol/migrations/0007_auto__add_field_buildrequest_environment__chg_field_buildrequest_build.py
delete mode 100644 bitbake/lib/toaster/bldcontrol/migrations/0008_brarchive.py
delete mode 100644 bitbake/lib/toaster/bldcontrol/migrations/0009_auto__add_field_brlayer_layer_version.py
delete mode 100644 bitbake/lib/toaster/orm/migrations/0002_auto__add_field_build_timespent.py
delete mode 100644 bitbake/lib/toaster/orm/migrations/0003_timespent.py
delete mode 100644 bitbake/lib/toaster/orm/migrations/0004_auto__add_field_package_installed_name.py
delete mode 100644 bitbake/lib/toaster/orm/migrations/0005_auto__add_target_image_file__add_target_file__add_field_variablehistor.py
delete mode 100644 bitbake/lib/toaster/orm/migrations/0006_auto__add_field_target_image_size__add_field_target_license_manifest_p.py
delete mode 100644 bitbake/lib/toaster/orm/migrations/0007_auto__add_helptext.py
delete mode 100644 bitbake/lib/toaster/orm/migrations/0008_auto__chg_field_variablehistory_operation__chg_field_recipe_descriptio.py
delete mode 100644 bitbake/lib/toaster/orm/migrations/0009_auto__add_projectvariable__add_projectlayer__add_projecttarget__add_pr.py
delete mode 100644 bitbake/lib/toaster/orm/migrations/0010_auto__add_field_project_branch__add_field_project_short_description__a.py
delete mode 100644 bitbake/lib/toaster/orm/migrations/0011_auto__add_field_projectlayer_dirpath.py
delete mode 100644 bitbake/lib/toaster/orm/migrations/0012_auto__add_field_projectlayer_optional__add_field_projecttarget_task.py
delete mode 100644 bitbake/lib/toaster/orm/migrations/0013_auto__add_release__add_layerversiondependency__add_unique_layerversion.py
delete mode 100644 bitbake/lib/toaster/orm/migrations/0014_auto__chg_field_package_summary__chg_field_layer_summary__chg_field_re.py
delete mode 100644 bitbake/lib/toaster/orm/migrations/0015_auto__add_field_layer_vcs_web_url__add_field_layer_vcs_web_tree_base_u.py
delete mode 100644 bitbake/lib/toaster/orm/migrations/0016_auto__add_field_release_helptext__chg_field_release_branch__add_index_.py
delete mode 100644 bitbake/lib/toaster/orm/migrations/0017_auto__del_toastersettingdefaultlayer__add_releaselayersourcepriority__.py
delete mode 100644 bitbake/lib/toaster/orm/migrations/0018_auto__add_field_layer_version_project.py
delete mode 100644 bitbake/lib/toaster/orm/migrations/0019_auto__add_buildartifact.py
delete mode 100644 bitbake/lib/toaster/orm/migrations/0020_auto__add_field_layer_version_local_path__add_field_recipe_pathflags__.py
delete mode 100644 bitbake/lib/toaster/orm/migrations/0021_auto__chg_field_build_project__chg_field_project_bitbake_version__chg_.py
delete mode 100644 bitbake/lib/toaster/orm/migrations/0022_auto__add_field_target_task__add_field_layer_version_local_path__del_f.py
delete mode 100644 bitbake/lib/toaster/orm/migrations/0023_auto__del_field_build_warnings_no__del_field_build_errors_no__del_fiel.py
delete mode 100644 bitbake/lib/toaster/orm/migrations/0024_auto__add_field_recipe_is_image.py
delete mode 100644 bitbake/lib/toaster/orm/migrations/0025_auto__add_field_project_is_default.py
delete mode 100644 bitbake/lib/toaster/orm/migrations/0026_set_default_project.py
delete mode 100644 bitbake/lib/toaster/orm/migrations/0027_auto__add_customimagerecipe__add_unique_customimagerecipe_name_project.py
delete mode 100644 bitbake/lib/toaster/orm/migrations/0028_auto__chg_field_logmessage_message.py
--
1.9.1
^ permalink raw reply [flat|nested] 19+ messages in thread
* [PATCH 06/18] toaster: rename get_query_set -> get_queryset
2015-12-10 3:56 [PATCH 00/18] toaster: upgrade to Django 1.8 brian avery
` (3 preceding siblings ...)
2015-12-10 3:56 ` [PATCH 05/18] toaster: Start Django machinery for database access brian avery
@ 2015-12-10 3:56 ` brian avery
2015-12-10 3:56 ` [PATCH 07/18] toaster: Update API used to make runbuilds methods run in transactions brian avery
` (11 subsequent siblings)
16 siblings, 0 replies; 19+ messages in thread
From: brian avery @ 2015-12-10 3:56 UTC (permalink / raw)
To: bitbake-devel
From: Ed Bartosh <ed.bartosh@linux.intel.com>
Renamed get_query_set to get_queryset as get_queryset causes
deprecation warning in Django 1.7 and is deprecated in Django 1.8
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
---
lib/toaster/orm/models.py | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/lib/toaster/orm/models.py b/lib/toaster/orm/models.py
index 4a868e7..b25910a 100644
--- a/lib/toaster/orm/models.py
+++ b/lib/toaster/orm/models.py
@@ -598,8 +598,8 @@ class Package(models.Model):
class Package_DependencyManager(models.Manager):
use_for_related_fields = True
- def get_query_set(self):
- return super(Package_DependencyManager, self).get_query_set().exclude(package_id = F('depends_on__id'))
+ def get_queryset(self):
+ return super(Package_DependencyManager, self).get_queryset().exclude(package_id = F('depends_on__id'))
class Package_Dependency(models.Model):
TYPE_RDEPENDS = 0
@@ -699,8 +699,8 @@ class Recipe(models.Model):
class Recipe_DependencyManager(models.Manager):
use_for_related_fields = True
- def get_query_set(self):
- return super(Recipe_DependencyManager, self).get_query_set().exclude(recipe_id = F('depends_on__id'))
+ def get_queryset(self):
+ return super(Recipe_DependencyManager, self).get_queryset().exclude(recipe_id = F('depends_on__id'))
class Recipe_Dependency(models.Model):
TYPE_DEPENDS = 0
--
1.9.1
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH 07/18] toaster: Update API used to make runbuilds methods run in transactions
2015-12-10 3:56 [PATCH 00/18] toaster: upgrade to Django 1.8 brian avery
` (4 preceding siblings ...)
2015-12-10 3:56 ` [PATCH 06/18] toaster: rename get_query_set -> get_queryset brian avery
@ 2015-12-10 3:56 ` brian avery
2015-12-10 3:56 ` [PATCH 08/18] toaster: Prevent deprecation warnings for RedirectView brian avery
` (10 subsequent siblings)
16 siblings, 0 replies; 19+ messages in thread
From: brian avery @ 2015-12-10 3:56 UTC (permalink / raw)
To: bitbake-devel
From: Elliot Smith <elliot.smith@intel.com>
runbuilds previously had its methods decorated with
transaction.commit_on_success, which is now deprecated. transaction.atomic
is an (almost) drop-in replacement for this, so use this instead.
[YOCTO #8364]
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
---
lib/toaster/bldcontrol/management/commands/runbuilds.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/lib/toaster/bldcontrol/management/commands/runbuilds.py b/lib/toaster/bldcontrol/management/commands/runbuilds.py
index 48dc618..d40dedb 100644
--- a/lib/toaster/bldcontrol/management/commands/runbuilds.py
+++ b/lib/toaster/bldcontrol/management/commands/runbuilds.py
@@ -14,14 +14,14 @@ class Command(NoArgsCommand):
help = "Schedules and executes build requests as possible. Does not return (interrupt with Ctrl-C)"
- @transaction.commit_on_success
+ @transaction.atomic
def _selectBuildEnvironment(self):
bec = getBuildEnvironmentController(lock = BuildEnvironment.LOCK_FREE)
bec.be.lock = BuildEnvironment.LOCK_LOCK
bec.be.save()
return bec
- @transaction.commit_on_success
+ @transaction.atomic
def _selectBuildRequest(self):
br = BuildRequest.objects.filter(state = BuildRequest.REQ_QUEUED).order_by('pk')[0]
br.state = BuildRequest.REQ_INPROGRESS
--
1.9.1
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH 08/18] toaster: Prevent deprecation warnings for RedirectView
2015-12-10 3:56 [PATCH 00/18] toaster: upgrade to Django 1.8 brian avery
` (5 preceding siblings ...)
2015-12-10 3:56 ` [PATCH 07/18] toaster: Update API used to make runbuilds methods run in transactions brian avery
@ 2015-12-10 3:56 ` brian avery
2015-12-10 3:56 ` [PATCH 09/18] toaster: Update deprecated manage.py command brian avery
` (9 subsequent siblings)
16 siblings, 0 replies; 19+ messages in thread
From: brian avery @ 2015-12-10 3:56 UTC (permalink / raw)
To: bitbake-devel
From: Elliot Smith <elliot.smith@intel.com>
The API for RedirectView is due to change in Django 1.9, which
means that Toaster generates deprecation warnings.
Set the "permanent" flag when constructing RedirectView instances
to prevent this warning.
[YOCTO #8364]
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
---
lib/toaster/bldcollector/urls.py | 1 -
lib/toaster/toastergui/urls.py | 2 +-
lib/toaster/toastermain/urls.py | 2 +-
3 files changed, 2 insertions(+), 3 deletions(-)
diff --git a/lib/toaster/bldcollector/urls.py b/lib/toaster/bldcollector/urls.py
index 144387b..64722f2 100644
--- a/lib/toaster/bldcollector/urls.py
+++ b/lib/toaster/bldcollector/urls.py
@@ -18,7 +18,6 @@
from django.conf.urls import patterns, include, url
-from django.views.generic import RedirectView
urlpatterns = patterns('bldcollector.views',
# landing point for pushing a bitbake_eventlog.json file to this toaster instace
diff --git a/lib/toaster/toastergui/urls.py b/lib/toaster/toastergui/urls.py
index a1adbb7..2bf2d99 100644
--- a/lib/toaster/toastergui/urls.py
+++ b/lib/toaster/toastergui/urls.py
@@ -168,5 +168,5 @@ urlpatterns = patterns('toastergui.views',
name='xhr_customrecipe'),
# default redirection
- url(r'^$', RedirectView.as_view( url= 'landing')),
+ url(r'^$', RedirectView.as_view(url='landing', permanent=True)),
)
diff --git a/lib/toaster/toastermain/urls.py b/lib/toaster/toastermain/urls.py
index 5ce5a92..534679d 100644
--- a/lib/toaster/toastermain/urls.py
+++ b/lib/toaster/toastermain/urls.py
@@ -45,7 +45,7 @@ urlpatterns = patterns('',
url(r'^orm/eventfile$', 'bldcollector.views.eventfile'),
# if no application is selected, we have the magic toastergui app here
- url(r'^$', never_cache(RedirectView.as_view(url='/toastergui/'))),
+ url(r'^$', never_cache(RedirectView.as_view(url='/toastergui/', permanent=True))),
)
import toastermain.settings
--
1.9.1
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH 09/18] toaster: Update deprecated manage.py command
2015-12-10 3:56 [PATCH 00/18] toaster: upgrade to Django 1.8 brian avery
` (6 preceding siblings ...)
2015-12-10 3:56 ` [PATCH 08/18] toaster: Prevent deprecation warnings for RedirectView brian avery
@ 2015-12-10 3:56 ` brian avery
2015-12-10 3:56 ` [PATCH 10/18] toaster: Check Django version against toaster-requirements.txt brian avery
` (8 subsequent siblings)
16 siblings, 0 replies; 19+ messages in thread
From: brian avery @ 2015-12-10 3:56 UTC (permalink / raw)
To: bitbake-devel
From: Elliot Smith <elliot.smith@intel.com>
syncdb is deprecated in favour of migrate in Django 1.7:
https://docs.djangoproject.com/en/1.8/releases/1.7/#schema-migrations
Update to the "migrate" command in Toaster's start script.
[YOCTO #8364]
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
---
bin/toaster | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/bin/toaster b/bin/toaster
index 528e17e..e3c7867 100755
--- a/bin/toaster
+++ b/bin/toaster
@@ -56,7 +56,7 @@ webserverStartAll()
retval=0
# you can always add a superuser later via
# python bitbake/lib/toaster/manage.py python manage.py createsuperuser --username=<ME>
- python $BBBASEDIR/lib/toaster/manage.py syncdb --noinput || retval=1
+ python $BBBASEDIR/lib/toaster/manage.py migrate --noinput || retval=1
python $BBBASEDIR/lib/toaster/manage.py migrate orm || retval=2
--
1.9.1
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH 10/18] toaster: Check Django version against toaster-requirements.txt
2015-12-10 3:56 [PATCH 00/18] toaster: upgrade to Django 1.8 brian avery
` (7 preceding siblings ...)
2015-12-10 3:56 ` [PATCH 09/18] toaster: Update deprecated manage.py command brian avery
@ 2015-12-10 3:56 ` brian avery
2015-12-10 3:56 ` [PATCH 11/18] toaster: Remove compatible_layerversions() method brian avery
` (7 subsequent siblings)
16 siblings, 0 replies; 19+ messages in thread
From: brian avery @ 2015-12-10 3:56 UTC (permalink / raw)
To: bitbake-devel
From: Elliot Smith <elliot.smith@intel.com>
Instead of hard-coding the required Django version in the start
script, look it up from the toaster-requirements.txt file.
[YOCTO #8364]
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
---
bin/toaster | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/bin/toaster b/bin/toaster
index e3c7867..0862a35 100755
--- a/bin/toaster
+++ b/bin/toaster
@@ -175,9 +175,15 @@ notify_chldexit() {
verify_prereq() {
# Verify prerequisites
+ regex="([0-9])\.([0-9])"
+ for line in `cat $BBBASEDIR/toaster-requirements.txt | grep Django` ; do
+ [[ `echo $line` =~ $regex ]]
+ required_major_version="${BASH_REMATCH[1]}"
+ required_minor_version="${BASH_REMATCH[2]}"
+ done
- if ! echo "import django; print (1,) == django.VERSION[0:1] and django.VERSION[1:2][0] in (8,)" | python 2>/dev/null | grep True >/dev/null; then
- printf "This program needs Django 1.8. Please install with\n\npip install django<=1.8.7\n"
+ if ! echo "import django; print ($required_major_version,) == django.VERSION[0:1] and django.VERSION[1:2][0] in ($required_minor_version,)" | python 2>/dev/null | grep True >/dev/null; then
+ printf "This program needs Django $required_major_version.$required_minor_version. Please install with\n\npip install -r $BBBASEDIR/toaster-requirements.txt\n"
return 2
fi
--
1.9.1
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH 11/18] toaster: Remove compatible_layerversions() method
2015-12-10 3:56 [PATCH 00/18] toaster: upgrade to Django 1.8 brian avery
` (8 preceding siblings ...)
2015-12-10 3:56 ` [PATCH 10/18] toaster: Check Django version against toaster-requirements.txt brian avery
@ 2015-12-10 3:56 ` brian avery
2015-12-10 3:56 ` [PATCH 12/18] toaster: Amend regex for MySQL database URLs brian avery
` (6 subsequent siblings)
16 siblings, 0 replies; 19+ messages in thread
From: brian avery @ 2015-12-10 3:56 UTC (permalink / raw)
To: bitbake-devel
From: Elliot Smith <elliot.smith@intel.com>
Remove compatible_layerversions() and replace calls to it with
calls to get_all_compatible_layer_versions().
The sorting done in compatible_layerversions() is not relevant,
and the code can be simplified by using get_all_compatible_layer_versions().
[YOCTO #8364]
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
---
lib/toaster/orm/models.py | 21 +++------------------
lib/toaster/toastergui/views.py | 22 ++++++++++++++--------
2 files changed, 17 insertions(+), 26 deletions(-)
diff --git a/lib/toaster/orm/models.py b/lib/toaster/orm/models.py
index b25910a..f07feda 100644
--- a/lib/toaster/orm/models.py
+++ b/lib/toaster/orm/models.py
@@ -195,23 +195,6 @@ class Project(models.Model):
except (Variable.DoesNotExist,IndexError):
return( "not_found" )
- # returns a queryset of compatible layers for a project
- def compatible_layerversions(self, release = None, layer_name = None):
- logger.warning("This function is deprecated")
- if release == None:
- release = self.release
- # layers on the same branch or layers specifically set for this project
- queryset = Layer_Version.objects.filter(((Q(up_branch__name = release.branch_name) & Q(project = None)) | Q(project = self)) & Q(build__isnull=True))
-
- if layer_name is not None:
- # we select only a layer name
- queryset = queryset.filter(layer__name = layer_name)
-
- # order by layer version priority
- queryset = queryset.filter(Q(layer_source=None) | Q(layer_source__releaselayersourcepriority__release = release)).select_related('layer_source', 'layer', 'up_branch', "layer_source__releaselayersourcepriority__priority").order_by("-layer_source__releaselayersourcepriority__priority")
-
- return queryset
-
def get_all_compatible_layer_versions(self):
""" Returns Queryset of all Layer_Versions which are compatible with
this project"""
@@ -1184,7 +1167,9 @@ class Layer_Version(models.Model):
return self._handle_url_path(self.layer.vcs_web_tree_base_url, '')
def get_equivalents_wpriority(self, project):
- return project.compatible_layerversions(layer_name = self.layer.name)
+ layer_versions = project.get_all_compatible_layer_versions()
+ filtered = layer_versions.filter(layer__name = self.layer.name)
+ return filtered.order_by("-layer_source__releaselayersourcepriority__priority")
def get_vcs_reference(self):
if self.branch is not None and len(self.branch) > 0:
diff --git a/lib/toaster/toastergui/views.py b/lib/toaster/toastergui/views.py
index 16f27b8..bd334b9 100755
--- a/lib/toaster/toastergui/views.py
+++ b/lib/toaster/toastergui/views.py
@@ -2264,16 +2264,17 @@ if True:
prj.bitbake_version = prj.release.bitbake_version
prj.save()
# we need to change the layers
- for i in prj.projectlayer_set.all():
+ for project in prj.projectlayer_set.all():
# find and add a similarly-named layer on the new branch
try:
- lv = prj.compatible_layerversions(layer_name = i.layercommit.layer.name)[0]
- ProjectLayer.objects.get_or_create(project = prj, layercommit = lv)
+ layer_versions = prj.get_all_compatible_layer_versions()
+ layer_versions = layer_versions.filter(layer__name = project.layercommit.layer.name)
+ ProjectLayer.objects.get_or_create(project = prj, layercommit = layer_versions.first())
except IndexError:
pass
finally:
# get rid of the old entry
- i.delete()
+ project.delete()
if 'machineName' in request.POST:
machinevar = prj.projectvariable_set.get(name="MACHINE")
@@ -2383,12 +2384,17 @@ if True:
retval = []
- for i in prj.projectlayer_set.all():
- lv = prj.compatible_layerversions(release = Release.objects.get(pk=new_release_id)).filter(layer__name = i.layercommit.layer.name)
+ for project in prj.projectlayer_set.all():
+ release = Release.objects.get(pk = new_release_id)
+
+ layer_versions = prj.get_all_compatible_layer_versions()
+ layer_versions = layer_versions.filter(release = release)
+ layer_versions = layer_versions.filter(layer__name = project.layercommit.layer.name)
+
# there is no layer_version with the new release id,
# and the same name
- if lv.count() < 1:
- retval.append(i)
+ if layer_versions.count() < 1:
+ retval.append(project)
return response({"error":"ok",
"rows" : map( _lv_to_dict(prj),
--
1.9.1
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH 12/18] toaster: Amend regex for MySQL database URLs
2015-12-10 3:56 [PATCH 00/18] toaster: upgrade to Django 1.8 brian avery
` (9 preceding siblings ...)
2015-12-10 3:56 ` [PATCH 11/18] toaster: Remove compatible_layerversions() method brian avery
@ 2015-12-10 3:56 ` brian avery
2015-12-10 3:56 ` [PATCH 13/18] toaster: use OneToOneField instead of ForeignKey brian avery
` (5 subsequent siblings)
16 siblings, 0 replies; 19+ messages in thread
From: brian avery @ 2015-12-10 3:56 UTC (permalink / raw)
To: bitbake-devel
From: Elliot Smith <elliot.smith@intel.com>
The database URL detection doesn't admit a MySQL URL without a port.
As this is a common case (e.g. you would set the HOST to a mysql.sock
path if pointing at a local MySQL server, with no port setting),
amend the regex so it will correctly recognise paths, as well
as HOST:PORT URLs.
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
---
lib/toaster/toastermain/settings.py | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/lib/toaster/toastermain/settings.py b/lib/toaster/toastermain/settings.py
index 7c559bc..b6bdcd8 100644
--- a/lib/toaster/toastermain/settings.py
+++ b/lib/toaster/toastermain/settings.py
@@ -64,6 +64,7 @@ if 'sqlite' in DATABASES['default']['ENGINE']:
if 'DATABASE_URL' in os.environ:
dburl = os.environ['DATABASE_URL']
+
if dburl.startswith('sqlite3://'):
result = re.match('sqlite3://(.*)', dburl)
if result is None:
@@ -78,7 +79,7 @@ if 'DATABASE_URL' in os.environ:
}
elif dburl.startswith('mysql://'):
# URL must be in this form: mysql://user:pass@host:port/name
- result = re.match(r"mysql://([^:]*):([^@]*)@([^:]*):(\d+)/([^/]*)", dburl)
+ result = re.match(r"mysql://([^:]*):([^@]*)@([^:]*):(\d*)/([^/]*)", dburl)
if result is None:
raise Exception("ERROR: Could not read mysql database url: %s" % dburl)
DATABASES['default'] = {
--
1.9.1
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH 13/18] toaster: use OneToOneField instead of ForeignKey
2015-12-10 3:56 [PATCH 00/18] toaster: upgrade to Django 1.8 brian avery
` (10 preceding siblings ...)
2015-12-10 3:56 ` [PATCH 12/18] toaster: Amend regex for MySQL database URLs brian avery
@ 2015-12-10 3:56 ` brian avery
2015-12-10 3:56 ` [PATCH 14/18] toaster: fixed format strings brian avery
` (4 subsequent siblings)
16 siblings, 0 replies; 19+ messages in thread
From: brian avery @ 2015-12-10 3:56 UTC (permalink / raw)
To: bitbake-devel
From: Ed Bartosh <ed.bartosh@linux.intel.com>
Used OneToOneField to reference BuildRequest in BRBitbake model.
Fixed django warning:
WARNINGS: Setting unique=True on a ForeignKey has the same effect
as using a OneToOneField.
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
---
lib/toaster/bldcontrol/bbcontroller.py | 4 ++--
lib/toaster/bldcontrol/localhostbecontroller.py | 13 ++++++-------
lib/toaster/bldcontrol/management/commands/runbuilds.py | 2 +-
lib/toaster/bldcontrol/models.py | 2 +-
lib/toaster/bldcontrol/tests.py | 6 +++---
5 files changed, 13 insertions(+), 14 deletions(-)
diff --git a/lib/toaster/bldcontrol/bbcontroller.py b/lib/toaster/bldcontrol/bbcontroller.py
index 781ff73..1387bda 100644
--- a/lib/toaster/bldcontrol/bbcontroller.py
+++ b/lib/toaster/bldcontrol/bbcontroller.py
@@ -141,10 +141,10 @@ class BuildEnvironmentController(object):
raise Exception("FIXME: Must override in order to actually start the BB server")
- def setLayers(self, bbs, ls):
+ def setLayers(self, bitbake, ls):
""" Checks-out bitbake executor and layers from git repositories.
Sets the layer variables in the config file, after validating local layer paths.
- The bitbakes must be a 1-length list of BRBitbake
+ bitbake must be a single BRBitbake instance
The layer paths must be in a list of BRLayer object
a word of attention: by convention, the first layer for any build will be poky!
diff --git a/lib/toaster/bldcontrol/localhostbecontroller.py b/lib/toaster/bldcontrol/localhostbecontroller.py
index 00228e9..4f6f15c 100644
--- a/lib/toaster/bldcontrol/localhostbecontroller.py
+++ b/lib/toaster/bldcontrol/localhostbecontroller.py
@@ -115,18 +115,17 @@ class LocalhostBEController(BuildEnvironmentController):
return local_checkout_path
- def setLayers(self, bitbakes, layers, targets):
+ def setLayers(self, bitbake, layers, targets):
""" a word of attention: by convention, the first layer for any build will be poky! """
assert self.be.sourcedir is not None
- assert len(bitbakes) == 1
# set layers in the layersource
# 1. get a list of repos with branches, and map dirpaths for each layer
gitrepos = {}
- gitrepos[(bitbakes[0].giturl, bitbakes[0].commit)] = []
- gitrepos[(bitbakes[0].giturl, bitbakes[0].commit)].append( ("bitbake", bitbakes[0].dirpath) )
+ gitrepos[(bitbake.giturl, bitbake.commit)] = []
+ gitrepos[(bitbake.giturl, bitbake.commit)].append( ("bitbake", bitbake.dirpath) )
for layer in layers:
# we don't process local URLs
@@ -198,7 +197,7 @@ class LocalhostBEController(BuildEnvironmentController):
# make sure we have a working bitbake
if not os.path.exists(os.path.join(self.pokydirname, 'bitbake')):
logger.debug("localhostbecontroller: checking bitbake into the poky dirname %s " % self.pokydirname)
- self._shellcmd("git clone -b \"%s\" \"%s\" \"%s\" " % (bitbakes[0].commit, bitbakes[0].giturl, os.path.join(self.pokydirname, 'bitbake')))
+ self._shellcmd("git clone -b \"%s\" \"%s\" \"%s\" " % (bitbake.commit, bitbake.giturl, os.path.join(self.pokydirname, 'bitbake')))
# verify our repositories
for name, dirpath in gitrepos[(giturl, commit)]:
@@ -224,7 +223,7 @@ class LocalhostBEController(BuildEnvironmentController):
for target in targets:
try:
customrecipe = CustomImageRecipe.objects.get(name=target.target,
- project=bitbakes[0].req.project)
+ project=bitbake.req.project)
except CustomImageRecipe.DoesNotExist:
continue # not a custom recipe, skip
@@ -278,7 +277,7 @@ class LocalhostBEController(BuildEnvironmentController):
def triggerBuild(self, bitbake, layers, variables, targets):
- # set up the buid environment with the needed layers
+ # set up the build environment with the needed layers
self.setLayers(bitbake, layers, targets)
# get the bb server running with the build req id and build env id
diff --git a/lib/toaster/bldcontrol/management/commands/runbuilds.py b/lib/toaster/bldcontrol/management/commands/runbuilds.py
index d40dedb..edf71a7 100644
--- a/lib/toaster/bldcontrol/management/commands/runbuilds.py
+++ b/lib/toaster/bldcontrol/management/commands/runbuilds.py
@@ -57,7 +57,7 @@ class Command(NoArgsCommand):
br.save()
# this triggers an async build
- bec.triggerBuild(br.brbitbake_set.all(), br.brlayer_set.all(), br.brvariable_set.all(), br.brtarget_set.all())
+ bec.triggerBuild(br.brbitbake, br.brlayer_set.all(), br.brvariable_set.all(), br.brtarget_set.all())
except Exception as e:
logger.error("runbuilds: Error launching build %s" % e)
diff --git a/lib/toaster/bldcontrol/models.py b/lib/toaster/bldcontrol/models.py
index ab41105..a3a49ce 100644
--- a/lib/toaster/bldcontrol/models.py
+++ b/lib/toaster/bldcontrol/models.py
@@ -106,7 +106,7 @@ class BRLayer(models.Model):
layer_version = models.ForeignKey(Layer_Version, null=True)
class BRBitbake(models.Model):
- req = models.ForeignKey(BuildRequest, unique = True) # only one bitbake for a request
+ req = models.OneToOneField(BuildRequest) # only one bitbake for a request
giturl = models.CharField(max_length =254)
commit = models.CharField(max_length = 254)
dirpath = models.CharField(max_length = 254)
diff --git a/lib/toaster/bldcontrol/tests.py b/lib/toaster/bldcontrol/tests.py
index f54cf7f..141b42a 100644
--- a/lib/toaster/bldcontrol/tests.py
+++ b/lib/toaster/bldcontrol/tests.py
@@ -18,7 +18,7 @@ import subprocess
import os
# standard poky data hardcoded for testing
-BITBAKE_LAYERS = [type('bitbake_info', (object,), { "giturl": "git://git.yoctoproject.org/poky.git", "dirpath": "", "commit": "HEAD"})]
+BITBAKE_LAYER = type('bitbake_info', (object,), { "giturl": "git://git.yoctoproject.org/poky.git", "dirpath": "", "commit": "HEAD"})
POKY_LAYERS = [
type('poky_info', (object,), { "name": "meta", "giturl": "git://git.yoctoproject.org/poky.git", "dirpath": "meta", "commit": "HEAD"}),
type('poky_info', (object,), { "name": "meta-yocto", "giturl": "git://git.yoctoproject.org/poky.git", "dirpath": "meta-yocto", "commit": "HEAD"}),
@@ -53,7 +53,7 @@ class BEControllerTests(object):
bc = self._getBEController(obe)
try:
# setting layers, skip any layer info
- bc.setLayers(BITBAKE_LAYERS, POKY_LAYERS)
+ bc.setLayers(BITBAKE_LAYER, POKY_LAYERS)
except NotImplementedException, e:
print "Test skipped due to command not implemented yet"
return True
@@ -80,7 +80,7 @@ class BEControllerTests(object):
layerSet = False
try:
# setting layers, skip any layer info
- layerSet = bc.setLayers(BITBAKE_LAYERS, POKY_LAYERS)
+ layerSet = bc.setLayers(BITBAKE_LAYER, POKY_LAYERS)
except NotImplementedException:
print "Test skipped due to command not implemented yet"
return True
--
1.9.1
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH 14/18] toaster: fixed format strings
2015-12-10 3:56 [PATCH 00/18] toaster: upgrade to Django 1.8 brian avery
` (11 preceding siblings ...)
2015-12-10 3:56 ` [PATCH 13/18] toaster: use OneToOneField instead of ForeignKey brian avery
@ 2015-12-10 3:56 ` brian avery
2015-12-10 3:56 ` [PATCH 15/18] toaster: work around 'database is locked' error brian avery
` (3 subsequent siblings)
16 siblings, 0 replies; 19+ messages in thread
From: brian avery @ 2015-12-10 3:56 UTC (permalink / raw)
To: bitbake-devel
From: Ed Bartosh <ed.bartosh@linux.intel.com>
Printing id as a number breaks toaster if object is not saved
yet as id/pk is None in this case.
Changed format spec to print id as a string in __str__
and __unicode__ methods of Build, Task and LayerVersion models.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
---
lib/toaster/orm/models.py | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/lib/toaster/orm/models.py b/lib/toaster/orm/models.py
index f07feda..3c51c6f 100644
--- a/lib/toaster/orm/models.py
+++ b/lib/toaster/orm/models.py
@@ -370,7 +370,7 @@ class Build(models.Model):
return self.get_outcome_text()
def __str__(self):
- return "%d %s %s" % (self.id, self.project, ",".join([t.target for t in self.target_set.all()]))
+ return "%s %s %s" % (self.id, self.project, ",".join([t.target for t in self.target_set.all()]))
# an Artifact is anything that results from a Build, and may be of interest to the user, and is not stored elsewhere
@@ -552,7 +552,7 @@ class Task(models.Model):
sstate_text = property(get_sstate_text)
def __unicode__(self):
- return "%d(%d) %s:%s" % (self.pk, self.build.pk, self.recipe.name, self.task_name)
+ return "%s(%s) %s:%s" % (self.pk, self.build.pk, self.recipe.name, self.task_name)
class Meta:
ordering = ('order', 'recipe' ,)
@@ -1203,7 +1203,7 @@ class Layer_Version(models.Model):
return sorted(result, key=lambda x: x.layer.name)
def __unicode__(self):
- return "%d %s (VCS %s, Project %s)" % (self.pk, str(self.layer), self.get_vcs_reference(), self.build.project if self.build is not None else "No project")
+ return "%s %s (VCS %s, Project %s)" % (self.pk, str(self.layer), self.get_vcs_reference(), self.build.project if self.build is not None else "No project")
class Meta:
unique_together = ("layer_source", "up_id")
--
1.9.1
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH 15/18] toaster: work around 'database is locked' error
2015-12-10 3:56 [PATCH 00/18] toaster: upgrade to Django 1.8 brian avery
` (12 preceding siblings ...)
2015-12-10 3:56 ` [PATCH 14/18] toaster: fixed format strings brian avery
@ 2015-12-10 3:56 ` brian avery
2015-12-10 3:56 ` [PATCH 16/18] toaster: removed extra calls of migrate brian avery
` (2 subsequent siblings)
16 siblings, 0 replies; 19+ messages in thread
From: brian avery @ 2015-12-10 3:56 UTC (permalink / raw)
To: bitbake-devel
From: Ed Bartosh <ed.bartosh@linux.intel.com>
When sqlite can not cope with a stream of 'insert' queries it throws
'database is locked' exception.
Wrapping model.save in transaction.atomic context and repeating the call
should solve this issue.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
---
lib/toaster/orm/models.py | 20 +++++++++++++++++++-
1 file changed, 19 insertions(+), 1 deletion(-)
diff --git a/lib/toaster/orm/models.py b/lib/toaster/orm/models.py
index 3c51c6f..0d598ac 100644
--- a/lib/toaster/orm/models.py
+++ b/lib/toaster/orm/models.py
@@ -29,10 +29,28 @@ from django.core import validators
from django.conf import settings
import django.db.models.signals
-
import logging
logger = logging.getLogger("toaster")
+if 'sqlite' in settings.DATABASES['default']['ENGINE']:
+ from django.db import transaction, OperationalError
+ from time import sleep
+
+ _base_save = models.Model.save
+ def save(self, *args, **kwargs):
+ while True:
+ try:
+ with transaction.atomic():
+ return _base_save(self, *args, **kwargs)
+ except OperationalError as err:
+ if 'database is locked' in str(err):
+ logger.warning("%s, model: %s, args: %s, kwargs: %s",
+ err, self.__class__, args, kwargs)
+ sleep(0.5)
+ continue
+ raise
+
+ models.Model.save = save
class GitURLValidator(validators.URLValidator):
import re
--
1.9.1
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH 16/18] toaster: removed extra calls of migrate
2015-12-10 3:56 [PATCH 00/18] toaster: upgrade to Django 1.8 brian avery
` (13 preceding siblings ...)
2015-12-10 3:56 ` [PATCH 15/18] toaster: work around 'database is locked' error brian avery
@ 2015-12-10 3:56 ` brian avery
2015-12-10 3:56 ` [PATCH 17/18] toaster: monkey patch Queryset brian avery
2015-12-10 3:56 ` [PATCH 18/18] toaster: rework checking of Django version brian avery
16 siblings, 0 replies; 19+ messages in thread
From: brian avery @ 2015-12-10 3:56 UTC (permalink / raw)
To: bitbake-devel
From: Ed Bartosh <ed.bartosh@linux.intel.com>
One manage.py migrate should be enough to apply migrations for
all applications. Extra calls just slow toaster script down.
Removed calls of migrate for orm and bldcontrol apps.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
---
bin/toaster | 20 +-------------------
1 file changed, 1 insertion(+), 19 deletions(-)
diff --git a/bin/toaster b/bin/toaster
index 0862a35..7645c56 100755
--- a/bin/toaster
+++ b/bin/toaster
@@ -58,26 +58,8 @@ webserverStartAll()
# python bitbake/lib/toaster/manage.py python manage.py createsuperuser --username=<ME>
python $BBBASEDIR/lib/toaster/manage.py migrate --noinput || retval=1
- python $BBBASEDIR/lib/toaster/manage.py migrate orm || retval=2
-
- if [ $retval -eq 1 ]; then
- echo "Failed db sync, aborting system start" 1>&2
- return $retval
- fi
-
- python $BBBASEDIR/lib/toaster/manage.py migrate orm || retval=1
-
- if [ $retval -eq 1 ]; then
- printf "\nError on orm migration, rolling back...\n"
- python $BBBASEDIR/lib/toaster/manage.py migrate orm 0001_initial --fake
- return $retval
- fi
-
- python $BBBASEDIR/lib/toaster/manage.py migrate bldcontrol || retval=1
-
if [ $retval -eq 1 ]; then
- printf "\nError on bldcontrol migration, rolling back...\n"
- python $BBBASEDIR/lib/toaster/manage.py migrate bldcontrol 0001_initial --fake
+ echo "Failed migrations, aborting system start" 1>&2
return $retval
fi
--
1.9.1
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH 17/18] toaster: monkey patch Queryset
2015-12-10 3:56 [PATCH 00/18] toaster: upgrade to Django 1.8 brian avery
` (14 preceding siblings ...)
2015-12-10 3:56 ` [PATCH 16/18] toaster: removed extra calls of migrate brian avery
@ 2015-12-10 3:56 ` brian avery
2015-12-10 3:56 ` [PATCH 18/18] toaster: rework checking of Django version brian avery
16 siblings, 0 replies; 19+ messages in thread
From: brian avery @ 2015-12-10 3:56 UTC (permalink / raw)
To: bitbake-devel
From: Ed Bartosh <ed.bartosh@linux.intel.com>
Fixed 'database is locked' issue by monkey patching django QuerySet
methods.
The actual patching places were found by bisecting Django codebase.
This commit should be removed after Django is fixed if it's fixed
at all.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
---
lib/toaster/orm/models.py | 30 ++++++++++++++++++++++++++++++
1 file changed, 30 insertions(+)
diff --git a/lib/toaster/orm/models.py b/lib/toaster/orm/models.py
index 0d598ac..6e87c54 100644
--- a/lib/toaster/orm/models.py
+++ b/lib/toaster/orm/models.py
@@ -52,6 +52,36 @@ if 'sqlite' in settings.DATABASES['default']['ENGINE']:
models.Model.save = save
+ # HACK: Monkey patch Django to fix 'database is locked' issue
+
+ from django.db.models.query import QuerySet
+ _base_insert = QuerySet._insert
+ def _insert(self, *args, **kwargs):
+ with transaction.atomic(using=self.db, savepoint=False):
+ return _base_insert(self, *args, **kwargs)
+ QuerySet._insert = _insert
+
+ from django.utils import six
+ def _create_object_from_params(self, lookup, params):
+ """
+ Tries to create an object using passed params.
+ Used by get_or_create and update_or_create
+ """
+ try:
+ obj = self.create(**params)
+ return obj, True
+ except IntegrityError:
+ exc_info = sys.exc_info()
+ try:
+ return self.get(**lookup), False
+ except self.model.DoesNotExist:
+ pass
+ six.reraise(*exc_info)
+
+ QuerySet._create_object_from_params = _create_object_from_params
+
+ # end of HACK
+
class GitURLValidator(validators.URLValidator):
import re
regex = re.compile(
--
1.9.1
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH 18/18] toaster: rework checking of Django version
2015-12-10 3:56 [PATCH 00/18] toaster: upgrade to Django 1.8 brian avery
` (15 preceding siblings ...)
2015-12-10 3:56 ` [PATCH 17/18] toaster: monkey patch Queryset brian avery
@ 2015-12-10 3:56 ` brian avery
16 siblings, 0 replies; 19+ messages in thread
From: brian avery @ 2015-12-10 3:56 UTC (permalink / raw)
To: bitbake-devel
From: Ed Bartosh <ed.bartosh@linux.intel.com>
Checked django version considering information from
toaster-requirements.txt, e.g. if requirements file contains
line "Django>1.8,<1.9" toaster should be able to check that
requirement correctly.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
---
bin/toaster | 24 ++++++++++--------------
toaster-requirements.txt | 2 +-
2 files changed, 11 insertions(+), 15 deletions(-)
diff --git a/bin/toaster b/bin/toaster
index 7645c56..1c1e029 100755
--- a/bin/toaster
+++ b/bin/toaster
@@ -156,16 +156,15 @@ notify_chldexit() {
verify_prereq() {
- # Verify prerequisites
- regex="([0-9])\.([0-9])"
- for line in `cat $BBBASEDIR/toaster-requirements.txt | grep Django` ; do
- [[ `echo $line` =~ $regex ]]
- required_major_version="${BASH_REMATCH[1]}"
- required_minor_version="${BASH_REMATCH[2]}"
- done
-
- if ! echo "import django; print ($required_major_version,) == django.VERSION[0:1] and django.VERSION[1:2][0] in ($required_minor_version,)" | python 2>/dev/null | grep True >/dev/null; then
- printf "This program needs Django $required_major_version.$required_minor_version. Please install with\n\npip install -r $BBBASEDIR/toaster-requirements.txt\n"
+ # Verify Django version
+ reqfile=$(python -c "import os; print os.path.realpath('$BBBASEDIR/toaster-requirements.txt')")
+ exp='s/Django\([><=]\+\)\([^,]\+\),\([><=]\+\)\(.\+\)/'
+ exp=$exp'import sys,django;version=django.get_version().split(".");'
+ exp=$exp'sys.exit(not (version \1 "\2".split(".") and version \3 "\4".split(".")))/p'
+ if ! sed -n "$exp" $reqfile | python - ; then
+ req=`grep ^Django $reqfile`
+ echo "This program needs $req"
+ echo "Please install with pip install -r $reqfile"
return 2
fi
@@ -265,10 +264,7 @@ if [ "$1" = 'restart-bitbake' ] ; then
return $rc
fi
-if ! verify_prereq; then
- echo "Error: Could not verify that the needed dependencies are installed. Please use virtualenv and pip to install dependencies listed in toaster-requirements.txt" 1>&2
- return 1
-fi
+verify_prereq || return 1
# We make sure we're running in the current shell and in a good environment
if [ -z "$BUILDDIR" ] || ! which bitbake >/dev/null 2>&1 ; then
diff --git a/toaster-requirements.txt b/toaster-requirements.txt
index 5d34494..0e8c742 100644
--- a/toaster-requirements.txt
+++ b/toaster-requirements.txt
@@ -1,4 +1,4 @@
-Django==1.8.6
+Django>1.8,<1.9
argparse==1.2.1
wsgiref==0.1.2
beautifulsoup4>=4.4.0
--
1.9.1
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH 14/18] toaster: fixed format strings
2015-12-08 15:31 [PATCH 00/18] Upgrade to Django 1.8 Ed Bartosh
@ 2015-12-08 15:32 ` Ed Bartosh
0 siblings, 0 replies; 19+ messages in thread
From: Ed Bartosh @ 2015-12-08 15:32 UTC (permalink / raw)
To: toaster
Printing id as a number breaks toaster if object is not saved
yet as id/pk is None in this case.
Changed format spec to print id as a string in __str__
and __unicode__ methods of Build, Task and LayerVersion models.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
bitbake/lib/toaster/orm/models.py | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/bitbake/lib/toaster/orm/models.py b/bitbake/lib/toaster/orm/models.py
index 5b8a45d..7b99a1f 100644
--- a/bitbake/lib/toaster/orm/models.py
+++ b/bitbake/lib/toaster/orm/models.py
@@ -370,7 +370,7 @@ class Build(models.Model):
return self.get_outcome_text()
def __str__(self):
- return "%d %s %s" % (self.id, self.project, ",".join([t.target for t in self.target_set.all()]))
+ return "%s %s %s" % (self.id, self.project, ",".join([t.target for t in self.target_set.all()]))
# an Artifact is anything that results from a Build, and may be of interest to the user, and is not stored elsewhere
@@ -552,7 +552,7 @@ class Task(models.Model):
sstate_text = property(get_sstate_text)
def __unicode__(self):
- return "%d(%d) %s:%s" % (self.pk, self.build.pk, self.recipe.name, self.task_name)
+ return "%s(%s) %s:%s" % (self.pk, self.build.pk, self.recipe.name, self.task_name)
class Meta:
ordering = ('order', 'recipe' ,)
@@ -1202,7 +1202,7 @@ class Layer_Version(models.Model):
return sorted(result, key=lambda x: x.layer.name)
def __unicode__(self):
- return "%d %s (VCS %s, Project %s)" % (self.pk, str(self.layer), self.get_vcs_reference(), self.build.project if self.build is not None else "No project")
+ return "%s %s (VCS %s, Project %s)" % (self.pk, str(self.layer), self.get_vcs_reference(), self.build.project if self.build is not None else "No project")
class Meta:
unique_together = ("layer_source", "up_id")
--
2.1.4
^ permalink raw reply related [flat|nested] 19+ messages in thread
end of thread, other threads:[~2015-12-10 3:53 UTC | newest]
Thread overview: 19+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-12-10 3:56 [PATCH 00/18] toaster: upgrade to Django 1.8 brian avery
2015-12-10 3:56 ` [PATCH 01/18] toaster: Upgrade to Django 1.8.6 and remove South brian avery
2015-12-10 3:56 ` [PATCH 03/18] toaster: Fix references to app paths brian avery
2015-12-10 3:56 ` [PATCH 04/18] toaster: Create default project with get_or_create* method brian avery
2015-12-10 3:56 ` [PATCH 05/18] toaster: Start Django machinery for database access brian avery
2015-12-10 3:56 ` [PATCH 06/18] toaster: rename get_query_set -> get_queryset brian avery
2015-12-10 3:56 ` [PATCH 07/18] toaster: Update API used to make runbuilds methods run in transactions brian avery
2015-12-10 3:56 ` [PATCH 08/18] toaster: Prevent deprecation warnings for RedirectView brian avery
2015-12-10 3:56 ` [PATCH 09/18] toaster: Update deprecated manage.py command brian avery
2015-12-10 3:56 ` [PATCH 10/18] toaster: Check Django version against toaster-requirements.txt brian avery
2015-12-10 3:56 ` [PATCH 11/18] toaster: Remove compatible_layerversions() method brian avery
2015-12-10 3:56 ` [PATCH 12/18] toaster: Amend regex for MySQL database URLs brian avery
2015-12-10 3:56 ` [PATCH 13/18] toaster: use OneToOneField instead of ForeignKey brian avery
2015-12-10 3:56 ` [PATCH 14/18] toaster: fixed format strings brian avery
2015-12-10 3:56 ` [PATCH 15/18] toaster: work around 'database is locked' error brian avery
2015-12-10 3:56 ` [PATCH 16/18] toaster: removed extra calls of migrate brian avery
2015-12-10 3:56 ` [PATCH 17/18] toaster: monkey patch Queryset brian avery
2015-12-10 3:56 ` [PATCH 18/18] toaster: rework checking of Django version brian avery
-- strict thread matches above, loose matches on Subject: below --
2015-12-08 15:31 [PATCH 00/18] Upgrade to Django 1.8 Ed Bartosh
2015-12-08 15:32 ` [PATCH 14/18] toaster: fixed format strings Ed Bartosh
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.