Fixes for Toaster broken on master (xmlrpc)

Fixes for Toaster broken on master (xmlrpc)

[Reyna, David]

Hi all,

* The recent changes to bitbake broke several parts of Toaster.

  "[bitbake-devel] [PATCH 5/6] server: Rework the server API so	process and xmlrpc servers coexist"

I have 2/3 fixed with the below patch. 

The final piece is for the ability to cancel builds, which broke when they deleted xmlrpc.py" and replaced it with "xmlrpcclient.py". I am working on that now.

* There are other oddities resulting from this bb change. For example for command line builds I sometimes get this (see Bug 11833).

  [build]$ bitbake core-image-minimal
  Gave up; Too many tries: timeout while attempting to communicate with bitbake server
  [build]$

Also, I cannot get "bitbake -m" nor "bitbake --status-only" to work:

  [build]$ bitbake -m --remote-server=0.0.0.0:51957 
  [build]$ bitbake --status-only --remote-server=127.0.0.1:51957
  Traceback (most recent call last):
  File "/opt/dreyna/toaster_master/poky/bitbake/bin/bitbake", line 48, in <module>
    cookerdata.CookerConfiguration()))
  File "/opt/dreyna/toaster_master/poky/bitbake/lib/bb/main.py", line 370, in bitbake_main
    server_connection, ui_module = setup_bitbake(configParams, configuration)
  File "/opt/dreyna/toaster_master/poky/bitbake/lib/bb/main.py", line 423, in setup_bitbake
    configParams.observe_only, configParams.xmlrpctoken)
  File "/opt/dreyna/toaster_master/poky/bitbake/lib/bb/server/xmlrpcclient.py", line 151, in connectXMLRPC
    raise e
  File "/opt/dreyna/toaster_master/poky/bitbake/lib/bb/server/xmlrpcclient.py", line 147, in connectXMLRPC
    connection = BitBakeXMLRPCServerConnection(host, port, (ip, 0), observer_only, featureset)
  File "/opt/dreyna/toaster_master/poky/bitbake/lib/bb/server/xmlrpcclient.py", line 89, in __init__
    self.events = uievent.BBUIEventQueue(self.connection, self.clientinfo)
  File "/opt/dreyna/toaster_master/poky/bitbake/lib/bb/ui/uievent.py", line 72, in __init__
    raise Exception(errmsg)
  Exception: Could not register UI event handler. Error: Cooker is busy: running, host 127.0.0.1, port  55744
  [build]$ 

- David

-----Original Message-----
From: toaster-bounces@yoctoproject.org [mailto:toaster-bounces@yoctoproject.org] On Behalf Of David Reyna
Sent: Thursday, July 27, 2017 7:09 PM
To: toaster@yoctoproject.org
Subject: [Toaster] [PATCH] toaster: move to new bitbake xmlrpc default

From: David Reyna <David.Reyna@windriver.com>

The bitbake option "-t SERVERTYPE" was deprecated and can be
removed since the desired XMLRPC listener now the default. Also,
the bitbake server port cannot be "-1" anymore and must be
explicitly passed. There is also a race condition for the bblock
file to not only be created but to actually be populated, so
a delay test loop is required (usually only one pass).

[YOCTO #11851]

Signed-off-by: David Reyna <David.Reyna@windriver.com>
---
 .../lib/toaster/bldcontrol/localhostbecontroller.py   | 19 +++++++++++++++----
 1 file changed, 15 insertions(+), 4 deletions(-)

diff --git a/bitbake/lib/toaster/bldcontrol/localhostbecontroller.py b/bitbake/lib/toaster/bldcontrol/localhostbecontroller.py
index 2916246..6142f7e 100644
--- a/bitbake/lib/toaster/bldcontrol/localhostbecontroller.py
+++ b/bitbake/lib/toaster/bldcontrol/localhostbecontroller.py
@@ -24,6 +24,7 @@ import os
 import sys
 import re
 import shutil
+import time
 from django.db import transaction
 from django.db.models import Q
 from bldcontrol.models import BuildEnvironment, BRLayer, BRVariable, BRTarget, BRBitbake
@@ -331,12 +332,22 @@ class LocalhostBEController(BuildEnvironmentController):
         bitbake = os.path.join(self.pokydirname, 'bitbake', 'bin', 'bitbake')
         toasterlayers = os.path.join(builddir,"conf/toaster-bblayers.conf")
         self._shellcmd('bash -c \"source %s %s; BITBAKE_UI="knotty" %s --read %s --read %s '
-                       '--server-only -t xmlrpc -B 0.0.0.0:0\"' % (oe_init,
+                       '--server-only -B 0.0.0.0:0\"' % (oe_init,
                        builddir, bitbake, confpath, toasterlayers), self.be.sourcedir)
 
         # read port number from bitbake.lock
         self.be.bbport = ""
         bblock = os.path.join(builddir, 'bitbake.lock')
+        # allow 10 seconds for bb lock file to appear but also be populated
+        for lock_check in range(10):
+            if not os.path.exists(bblock):
+                logger.debug("localhostbecontroller: waiting for bblock file to appear")
+                time.sleep(1)
+                continue
+            if 10 < os.stat(bblock).st_size:
+                break
+            logger.debug("localhostbecontroller: waiting for bblock content to appear")
+            time.sleep(1)
         with open(bblock) as fplock:
             for line in fplock:
                 if ":" in line:
@@ -365,10 +376,10 @@ class LocalhostBEController(BuildEnvironmentController):
         log = os.path.join(builddir, 'toaster_ui.log')
         local_bitbake = os.path.join(os.path.dirname(os.getenv('BBBASEDIR')),
                                      'bitbake')
-        self._shellcmd(['bash -c \"(TOASTER_BRBE="%s" BBSERVER="0.0.0.0:-1" '
+        self._shellcmd(['bash -c \"(TOASTER_BRBE="%s" BBSERVER="0.0.0.0:%s" '
                         '%s %s -u toasterui --token="" >>%s 2>&1;'
-                        'BITBAKE_UI="knotty" BBSERVER=0.0.0.0:-1 %s -m)&\"' \
-                        % (brbe, local_bitbake, bbtargets, log, bitbake)],
+                        'BITBAKE_UI="knotty" BBSERVER=0.0.0.0:%s %s -m)&\"' \
+                        % (brbe, self.be.bbport, local_bitbake, bbtargets, log, self.be.bbport, bitbake)],
                         builddir, nowait=True)
 
         logger.debug('localhostbecontroller: Build launched, exiting. '
-- 
1.9.1

-- 
_______________________________________________
toaster mailing list
toaster@yoctoproject.org
https://lists.yoctoproject.org/listinfo/toaster