* [PATCH 00/10 v2] Hob2 related bitbake changes
@ 2011-12-15 7:14 Dongxiao Xu
2011-12-15 7:14 ` [PATCH 01/10] command.py: Modify needcache value for certain functions Dongxiao Xu
` (10 more replies)
0 siblings, 11 replies; 12+ messages in thread
From: Dongxiao Xu @ 2011-12-15 7:14 UTC (permalink / raw)
To: bitbake-devel
Hi Richard,
This pull request is the second version that introduces several hob2 related bitbake changes.
Please help to review and pull.
Changes from v1:
Add "resolve" option in generateTargetsTree() instead of adding a new generateTargetsTreePro function. Thanks for Josh's suggestions.
Thanks,
Dongxiao
The following changes since commit 71b53a3f0766ca464560a1f6a449f9424fbdf7ae:
Matthew McClintock (1):
siggen.py: If both sigs have a variable in it's whitelist then don't say it's changed
are available in the git repository at:
git://git.pokylinux.org/poky-contrib dxu4/hob2-bitbake-change-v2
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=dxu4/hob2-bitbake-change-v2
Dongxiao Xu (10):
command.py: Modify needcache value for certain functions
cache: Use configuration's hash value to validate cache
cooker: user bb.configuration.data to inject events
command.py: add initCooker API
command.py: add parseConfigurationFiles API
command.py: add resolve option for generateTargetsTree API
event.py: Add a new event PackageInfo
xmlrpc: Change BitbakeServerInfo init function
cooker: remove command import in cooker.py
bitbake: add a new option "--server-only"
bin/bitbake | 23 +++++++++++++------
lib/bb/cache.py | 32 ++++++++++-----------------
lib/bb/command.py | 31 +++++++++++++++++++++++---
lib/bb/cooker.py | 55 ++++++++++++++++++++++++++++++++++++++--------
lib/bb/data_smart.py | 17 ++++++++++++++
lib/bb/event.py | 9 +++++++
lib/bb/server/xmlrpc.py | 8 +++---
7 files changed, 130 insertions(+), 45 deletions(-)
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH 01/10] command.py: Modify needcache value for certain functions
2011-12-15 7:14 [PATCH 00/10 v2] Hob2 related bitbake changes Dongxiao Xu
@ 2011-12-15 7:14 ` Dongxiao Xu
2011-12-15 7:14 ` [PATCH 02/10] cache: Use configuration's hash value to validate cache Dongxiao Xu
` (9 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Dongxiao Xu @ 2011-12-15 7:14 UTC (permalink / raw)
To: bitbake-devel
for findConfigFiels() and findFilesMatchingInDir() functions, they
don't need to parse all the bb files, thus setting the needcache
value to be False.
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
---
lib/bb/command.py | 4 ++--
1 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/lib/bb/command.py b/lib/bb/command.py
index 83907f6..5dec6a9 100644
--- a/lib/bb/command.py
+++ b/lib/bb/command.py
@@ -241,7 +241,7 @@ class CommandsAsync:
command.cooker.findConfigFiles(varname)
command.finishAsyncCommand()
- findConfigFiles.needcache = True
+ findConfigFiles.needcache = False
def findFilesMatchingInDir(self, command, params):
"""
@@ -253,7 +253,7 @@ class CommandsAsync:
command.cooker.findFilesMatchingInDir(pattern, directory)
command.finishAsyncCommand()
- findFilesMatchingInDir.needcache = True
+ findFilesMatchingInDir.needcache = False
def findConfigFilePath(self, command, params):
"""
--
1.7.0.4
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH 02/10] cache: Use configuration's hash value to validate cache
2011-12-15 7:14 [PATCH 00/10 v2] Hob2 related bitbake changes Dongxiao Xu
2011-12-15 7:14 ` [PATCH 01/10] command.py: Modify needcache value for certain functions Dongxiao Xu
@ 2011-12-15 7:14 ` Dongxiao Xu
2011-12-15 7:14 ` [PATCH 03/10] cooker: user bb.configuration.data to inject events Dongxiao Xu
` (8 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Dongxiao Xu @ 2011-12-15 7:14 UTC (permalink / raw)
To: bitbake-devel
Previously we use the file time stamp to judge if a cache is valid.
Here this commit introduce a new method, which calculates the total
hash value for a certain configuration's key/value paris, and tag
it into cache filename, for example, bb_cache.dat.xxxyyyzzz.
This mechanism also ensures the cache's correctness if user
dynamically setting variables from some frontend GUI, like HOB.
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
---
lib/bb/cache.py | 32 ++++++++++++--------------------
lib/bb/cooker.py | 4 +++-
lib/bb/data_smart.py | 17 +++++++++++++++++
3 files changed, 32 insertions(+), 21 deletions(-)
diff --git a/lib/bb/cache.py b/lib/bb/cache.py
index 6b7fa6f..955b6df 100644
--- a/lib/bb/cache.py
+++ b/lib/bb/cache.py
@@ -42,10 +42,10 @@ except ImportError:
logger.info("Importing cPickle failed. "
"Falling back to a very slow implementation.")
-__cache_version__ = "142"
+__cache_version__ = "143"
-def getCacheFile(path, filename):
- return os.path.join(path, filename)
+def getCacheFile(path, filename, data_hash):
+ return os.path.join(path, filename + "." + data_hash)
# RecipeInfoCommon defines common data retrieving methods
# from meta data for caches. CoreRecipeInfo as well as other
@@ -254,7 +254,7 @@ class Cache(object):
BitBake Cache implementation
"""
- def __init__(self, data, caches_array):
+ def __init__(self, data, data_hash, caches_array):
# Pass caches_array information into Cache Constructor
# It will be used in later for deciding whether we
# need extra cache file dump/load support
@@ -266,6 +266,7 @@ class Cache(object):
self.data = None
self.data_fn = None
self.cacheclean = True
+ self.data_hash = data_hash
if self.cachedir in [None, '']:
self.has_cache = False
@@ -274,26 +275,17 @@ class Cache(object):
return
self.has_cache = True
- self.cachefile = getCacheFile(self.cachedir, "bb_cache.dat")
+ self.cachefile = getCacheFile(self.cachedir, "bb_cache.dat", self.data_hash)
logger.debug(1, "Using cache in '%s'", self.cachedir)
bb.utils.mkdirhier(self.cachedir)
- # If any of configuration.data's dependencies are newer than the
- # cache there isn't even any point in loading it...
- newest_mtime = 0
- deps = data.getVar("__base_depends")
-
- old_mtimes = [old_mtime for _, old_mtime in deps]
- old_mtimes.append(newest_mtime)
- newest_mtime = max(old_mtimes)
-
cache_ok = True
if self.caches_array:
for cache_class in self.caches_array:
if type(cache_class) is type and issubclass(cache_class, RecipeInfoCommon):
- cachefile = getCacheFile(self.cachedir, cache_class.cachefile)
- cache_ok = cache_ok and (bb.parse.cached_mtime_noerror(cachefile) >= newest_mtime)
+ cachefile = getCacheFile(self.cachedir, cache_class.cachefile, self.data_hash)
+ cache_ok = cache_ok and os.path.exists(cachefile)
cache_class.init_cacheData(self)
if cache_ok:
self.load_cachefile()
@@ -327,7 +319,7 @@ class Cache(object):
# Calculate the correct cachesize of all those cache files
for cache_class in self.caches_array:
if type(cache_class) is type and issubclass(cache_class, RecipeInfoCommon):
- cachefile = getCacheFile(self.cachedir, cache_class.cachefile)
+ cachefile = getCacheFile(self.cachedir, cache_class.cachefile, self.data_hash)
with open(cachefile, "rb") as cachefile:
cachesize += os.fstat(cachefile.fileno()).st_size
@@ -335,7 +327,7 @@ class Cache(object):
for cache_class in self.caches_array:
if type(cache_class) is type and issubclass(cache_class, RecipeInfoCommon):
- cachefile = getCacheFile(self.cachedir, cache_class.cachefile)
+ cachefile = getCacheFile(self.cachedir, cache_class.cachefile, self.data_hash)
with open(cachefile, "rb") as cachefile:
pickled = pickle.Unpickler(cachefile)
while cachefile:
@@ -588,7 +580,7 @@ class Cache(object):
for cache_class in self.caches_array:
if type(cache_class) is type and issubclass(cache_class, RecipeInfoCommon):
cache_class_name = cache_class.__name__
- cachefile = getCacheFile(self.cachedir, cache_class.cachefile)
+ cachefile = getCacheFile(self.cachedir, cache_class.cachefile, self.data_hash)
file_dict[cache_class_name] = open(cachefile, "wb")
pickler_dict[cache_class_name] = pickle.Pickler(file_dict[cache_class_name], pickle.HIGHEST_PROTOCOL)
@@ -693,7 +685,7 @@ def init(cooker):
Files causing parsing errors are evicted from the cache.
"""
- return Cache(cooker.configuration.data)
+ return Cache(cooker.configuration.data, cooker.configuration.data_hash)
class CacheData(object):
diff --git a/lib/bb/cooker.py b/lib/bb/cooker.py
index cbe0d71..2c02e28 100644
--- a/lib/bb/cooker.py
+++ b/lib/bb/cooker.py
@@ -832,6 +832,7 @@ class BBCooker:
bb.parse.init_parser(data)
bb.event.fire(bb.event.ConfigParsed(), data)
self.configuration.data = data
+ self.configuration.data_hash = data.get_hash()
def handleCollections( self, collections ):
"""Handle collections"""
@@ -1399,6 +1400,7 @@ class CookerParser(object):
self.filelist = filelist
self.cooker = cooker
self.cfgdata = cooker.configuration.data
+ self.cfghash = cooker.configuration.data_hash
# Accounting statistics
self.parsed = 0
@@ -1414,7 +1416,7 @@ class CookerParser(object):
self.num_processes = int(self.cfgdata.getVar("BB_NUMBER_PARSE_THREADS", True) or
multiprocessing.cpu_count())
- self.bb_cache = bb.cache.Cache(self.cfgdata, cooker.caches_array)
+ self.bb_cache = bb.cache.Cache(self.cfgdata, self.cfghash, cooker.caches_array)
self.fromcache = []
self.willparse = []
for filename in self.filelist:
diff --git a/lib/bb/data_smart.py b/lib/bb/data_smart.py
index ea13478..06d8d0b 100644
--- a/lib/bb/data_smart.py
+++ b/lib/bb/data_smart.py
@@ -31,6 +31,7 @@ BitBake build tools.
import copy, re
from collections import MutableMapping
import logging
+import hashlib
import bb, bb.codeparser
from bb import utils
from bb.COW import COWDictBase
@@ -459,3 +460,19 @@ class DataSmart(MutableMapping):
def __delitem__(self, var):
self.delVar(var)
+
+ def get_hash(self):
+ data = ""
+ keys = iter(self)
+ for key in keys:
+ if key == "TIME":
+ continue
+ if key == "__depends":
+ deps = list(self.getVar(key, False))
+ deps.sort()
+ value = [deps[i][0] for i in range(len(deps))]
+ else:
+ value = self.getVar(key, False) or ""
+ data = data + key + str(value)
+
+ return hashlib.md5(data).hexdigest()
--
1.7.0.4
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH 03/10] cooker: user bb.configuration.data to inject events
2011-12-15 7:14 [PATCH 00/10 v2] Hob2 related bitbake changes Dongxiao Xu
2011-12-15 7:14 ` [PATCH 01/10] command.py: Modify needcache value for certain functions Dongxiao Xu
2011-12-15 7:14 ` [PATCH 02/10] cache: Use configuration's hash value to validate cache Dongxiao Xu
@ 2011-12-15 7:14 ` Dongxiao Xu
2011-12-15 7:14 ` [PATCH 04/10] command.py: add initCooker API Dongxiao Xu
` (7 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Dongxiao Xu @ 2011-12-15 7:14 UTC (permalink / raw)
To: bitbake-devel
For buildTargets function, we use bb.configuration.data as parameter
to inject events, since in hob environment, some variables are
modified dynamically and bb.configuration.event_data may out of date.
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
---
lib/bb/cooker.py | 4 ++--
1 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/lib/bb/cooker.py b/lib/bb/cooker.py
index 2c02e28..6ddb38e 100644
--- a/lib/bb/cooker.py
+++ b/lib/bb/cooker.py
@@ -1085,7 +1085,7 @@ class BBCooker:
return False
if not retval:
- bb.event.fire(bb.event.BuildCompleted(buildname, targets, failures), self.configuration.event_data)
+ bb.event.fire(bb.event.BuildCompleted(buildname, targets, failures), self.configuration.data)
self.command.finishAsyncCommand()
return False
if retval is True:
@@ -1095,7 +1095,7 @@ class BBCooker:
self.buildSetVars()
buildname = self.configuration.data.getVar("BUILDNAME")
- bb.event.fire(bb.event.BuildStarted(buildname, targets), self.configuration.event_data)
+ bb.event.fire(bb.event.BuildStarted(buildname, targets), self.configuration.data)
localdata = data.createCopy(self.configuration.data)
bb.data.update_data(localdata)
--
1.7.0.4
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH 04/10] command.py: add initCooker API
2011-12-15 7:14 [PATCH 00/10 v2] Hob2 related bitbake changes Dongxiao Xu
` (2 preceding siblings ...)
2011-12-15 7:14 ` [PATCH 03/10] cooker: user bb.configuration.data to inject events Dongxiao Xu
@ 2011-12-15 7:14 ` Dongxiao Xu
2011-12-15 7:14 ` [PATCH 05/10] command.py: add parseConfigurationFiles API Dongxiao Xu
` (6 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Dongxiao Xu @ 2011-12-15 7:14 UTC (permalink / raw)
To: bitbake-devel
initCooker is to set the cooker to the initial state with nothing
parsed.
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
---
lib/bb/command.py | 6 ++++++
lib/bb/cooker.py | 13 +++++++++++++
2 files changed, 19 insertions(+), 0 deletions(-)
diff --git a/lib/bb/command.py b/lib/bb/command.py
index 5dec6a9..701b286 100644
--- a/lib/bb/command.py
+++ b/lib/bb/command.py
@@ -157,6 +157,12 @@ class CommandsSync:
value = params[1]
command.cooker.configuration.data.setVar(varname, value)
+ def initCooker(self, command, params):
+ """
+ Init the cooker to initial state with nothing parsed
+ """
+ command.cooker.initialize()
+
def resetCooker(self, command, params):
"""
Reset the cooker to its initial state, thus forcing a reparse for
diff --git a/lib/bb/cooker.py b/lib/bb/cooker.py
index 6ddb38e..666242f 100644
--- a/lib/bb/cooker.py
+++ b/lib/bb/cooker.py
@@ -167,6 +167,15 @@ class BBCooker:
self.parser = None
+ def initConfigurationData(self):
+ self.configuration.data = bb.data.init()
+
+ if not self.server_registration_cb:
+ bb.data.setVar("BB_WORKERCONTEXT", "1", self.configuration.data)
+
+ filtered_keys = bb.utils.approved_variables()
+ bb.data.inheritFromOS(self.configuration.data, self.savedenv, filtered_keys)
+
def loadConfigurationData(self):
self.configuration.data = bb.data.init()
@@ -1301,6 +1310,10 @@ class BBCooker:
def reparseFiles(self):
return
+ def initialize(self):
+ self.state = state.initial
+ self.initConfigurationData()
+
def reset(self):
self.state = state.initial
self.loadConfigurationData()
--
1.7.0.4
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH 05/10] command.py: add parseConfigurationFiles API
2011-12-15 7:14 [PATCH 00/10 v2] Hob2 related bitbake changes Dongxiao Xu
` (3 preceding siblings ...)
2011-12-15 7:14 ` [PATCH 04/10] command.py: add initCooker API Dongxiao Xu
@ 2011-12-15 7:14 ` Dongxiao Xu
2011-12-15 7:14 ` [PATCH 06/10] command.py: add resolve option for generateTargetsTree API Dongxiao Xu
` (5 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Dongxiao Xu @ 2011-12-15 7:14 UTC (permalink / raw)
To: bitbake-devel
The parseConfigurationFiles API calls the related function in
cooker.py to parse config files.
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
---
lib/bb/command.py | 10 ++++++++++
1 files changed, 10 insertions(+), 0 deletions(-)
diff --git a/lib/bb/command.py b/lib/bb/command.py
index 701b286..6b4a598 100644
--- a/lib/bb/command.py
+++ b/lib/bb/command.py
@@ -326,3 +326,13 @@ class CommandsAsync:
else:
command.finishAsyncCommand()
compareRevisions.needcache = True
+
+ def parseConfigurationFiles(self, command, params):
+ """
+ Parse the configuration files
+ """
+ prefiles = params[0]
+ postfiles = params[1]
+ command.cooker.parseConfigurationFiles(prefiles, postfiles)
+ command.finishAsyncCommand()
+ parseConfigurationFiles.needcache = False
--
1.7.0.4
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH 06/10] command.py: add resolve option for generateTargetsTree API
2011-12-15 7:14 [PATCH 00/10 v2] Hob2 related bitbake changes Dongxiao Xu
` (4 preceding siblings ...)
2011-12-15 7:14 ` [PATCH 05/10] command.py: add parseConfigurationFiles API Dongxiao Xu
@ 2011-12-15 7:14 ` Dongxiao Xu
2011-12-15 7:14 ` [PATCH 07/10] event.py: Add a new event PackageInfo Dongxiao Xu
` (4 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Dongxiao Xu @ 2011-12-15 7:14 UTC (permalink / raw)
To: bitbake-devel
Currently we have generateTargetsTree API, which is used to get
dependency information. However in that tree, there will be
"virtual/xxx" in depends fields. Therefore we add the resolve option
to replace it with its real providers.
Besides, for packages that provided by multiple recipes, we will find
their preverred provider.
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
---
lib/bb/command.py | 11 +++++++++--
lib/bb/cooker.py | 30 +++++++++++++++++++++++++-----
2 files changed, 34 insertions(+), 7 deletions(-)
diff --git a/lib/bb/command.py b/lib/bb/command.py
index 6b4a598..e52041d 100644
--- a/lib/bb/command.py
+++ b/lib/bb/command.py
@@ -227,14 +227,21 @@ class CommandsAsync:
included in the package list.
If pkg_list provided use that list (plus any extras brought in by
klass) rather than generating a tree for all packages.
+
+ Add a new option "resolve" to indicate if we need to resolve the
+ replacement for "virtual/xxx" like pn.
"""
klass = params[0]
- if len(params) > 1:
+ resolve = False
+ if len(params) > 2:
+ pkg_list = params[1]
+ resolve = params[2]
+ elif len(params) > 1:
pkg_list = params[1]
else:
pkg_list = []
- command.cooker.generateTargetsTree(klass, pkg_list)
+ command.cooker.generateTargetsTree(klass, pkg_list, resolve)
command.finishAsyncCommand()
generateTargetsTree.needcache = True
diff --git a/lib/bb/cooker.py b/lib/bb/cooker.py
index 666242f..4401a66 100644
--- a/lib/bb/cooker.py
+++ b/lib/bb/cooker.py
@@ -436,7 +436,7 @@ class BBCooker:
return depend_tree
- def generatePkgDepTreeData(self, pkgs_to_build, task):
+ def generatePkgDepTreeData(self, pkgs_to_build, task, resolve=False):
"""
Create a dependency tree of pkgs_to_build, returning the data.
"""
@@ -462,6 +462,7 @@ class BBCooker:
summary = self.status.summary[fn]
lic = self.status.license[fn]
section = self.status.section[fn]
+ rdepends = self.status.rundeps[fn]
if pn not in depend_tree["pn"]:
depend_tree["pn"][pn] = {}
depend_tree["pn"][pn]["filename"] = fn
@@ -469,6 +470,7 @@ class BBCooker:
depend_tree["pn"][pn]["summary"] = summary
depend_tree["pn"][pn]["license"] = lic
depend_tree["pn"][pn]["section"] = section
+ depend_tree["pn"][pn]["packages"] = rdepends.keys()
if fnid not in seen_fnids:
seen_fnids.append(fnid)
@@ -476,13 +478,24 @@ class BBCooker:
depend_tree["depends"][pn] = []
for dep in taskdata.depids[fnid]:
- depend_tree["depends"][pn].append(taskdata.build_names_index[dep])
+ if resolve:
+ item = taskdata.build_names_index[dep]
+ pn_provider = ""
+ targetid = taskdata.getbuild_id(item)
+ if targetid in taskdata.build_targets:
+ fnid = taskdata.build_targets[targetid][0]
+ fn_provider = taskdata.fn_index[fnid]
+ pn_provider = self.status.pkg_fn[fn_provider]
+ else:
+ pn_provider = item
+ depend_tree["depends"][pn].append(pn_provider)
+ else:
+ depend_tree["depends"][pn].append(taskdata.build_names_index[dep])
depend_tree["rdepends-pn"][pn] = []
for rdep in taskdata.rdepids[fnid]:
depend_tree["rdepends-pn"][pn].append(taskdata.run_names_index[rdep])
- rdepends = self.status.rundeps[fn]
for package in rdepends:
depend_tree["rdepends-pkg"][package] = []
for rdepend in rdepends[package]:
@@ -491,6 +504,13 @@ class BBCooker:
for package in packages:
if package not in depend_tree["packages"]:
+ if resolve:
+ targetid = taskdata.getrun_id(package)
+ if targetid in taskdata.run_targets:
+ fnid = taskdata.run_targets[targetid][0]
+ fn = taskdata.fn_index[fnid]
+ pn = self.status.pkg_fn[fn]
+ version = "%s:%s-%s" % self.status.pkg_pepvpr[fn]
depend_tree["packages"][package] = {}
depend_tree["packages"][package]["pn"] = pn
depend_tree["packages"][package]["filename"] = fn
@@ -728,7 +748,7 @@ class BBCooker:
return pkg_list
- def generateTargetsTree(self, klass=None, pkgs=[]):
+ def generateTargetsTree(self, klass=None, pkgs=[], resolve=False):
"""
Generate a dependency tree of buildable targets
Generate an event with the result
@@ -743,7 +763,7 @@ class BBCooker:
pkgs = pkgs + extra_pkgs
# generate a dependency tree for all our packages
- tree = self.generatePkgDepTreeData(pkgs, 'build')
+ tree = self.generatePkgDepTreeData(pkgs, 'build', resolve)
bb.event.fire(bb.event.TargetsTreeGenerated(tree), self.configuration.data)
def buildWorldTargetList(self):
--
1.7.0.4
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH 07/10] event.py: Add a new event PackageInfo
2011-12-15 7:14 [PATCH 00/10 v2] Hob2 related bitbake changes Dongxiao Xu
` (5 preceding siblings ...)
2011-12-15 7:14 ` [PATCH 06/10] command.py: add resolve option for generateTargetsTree API Dongxiao Xu
@ 2011-12-15 7:14 ` Dongxiao Xu
2011-12-15 7:14 ` [PATCH 08/10] xmlrpc: Change BitbakeServerInfo init function Dongxiao Xu
` (3 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Dongxiao Xu @ 2011-12-15 7:14 UTC (permalink / raw)
To: bitbake-devel
This event is to pass package information.
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
---
lib/bb/event.py | 9 +++++++++
1 files changed, 9 insertions(+), 0 deletions(-)
diff --git a/lib/bb/event.py b/lib/bb/event.py
index 088cd7c..67543e7 100644
--- a/lib/bb/event.py
+++ b/lib/bb/event.py
@@ -459,3 +459,12 @@ class LogHandler(logging.Handler):
def filter(self, record):
record.taskpid = worker_pid
return True
+
+class PackageInfo(Event):
+ """
+ Package information for GUI
+ """
+ def __init__(self, recipe, pkginfolist):
+ Event.__init__(self)
+ self._recipe = recipe
+ self._pkginfolist = pkginfolist
--
1.7.0.4
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH 08/10] xmlrpc: Change BitbakeServerInfo init function
2011-12-15 7:14 [PATCH 00/10 v2] Hob2 related bitbake changes Dongxiao Xu
` (6 preceding siblings ...)
2011-12-15 7:14 ` [PATCH 07/10] event.py: Add a new event PackageInfo Dongxiao Xu
@ 2011-12-15 7:14 ` Dongxiao Xu
2011-12-15 7:15 ` [PATCH 09/10] cooker: remove command import in cooker.py Dongxiao Xu
` (2 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Dongxiao Xu @ 2011-12-15 7:14 UTC (permalink / raw)
To: bitbake-devel
Pass host and port to BitbakeServerInfo class instead of the "server"
instance. With this change, GUI can connect with server individually
by host address and port.
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
---
lib/bb/server/xmlrpc.py | 8 ++++----
1 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/lib/bb/server/xmlrpc.py b/lib/bb/server/xmlrpc.py
index a7ac969..b5980c6 100644
--- a/lib/bb/server/xmlrpc.py
+++ b/lib/bb/server/xmlrpc.py
@@ -242,9 +242,9 @@ class BitBakeXMLRPCServer(SimpleXMLRPCServer):
return
class BitbakeServerInfo():
- def __init__(self, server):
- self.host = server.host
- self.port = server.port
+ def __init__(self, host, port):
+ self.host = host
+ self.port = port
class BitBakeServerConnection():
def __init__(self, serverinfo):
@@ -278,7 +278,7 @@ class BitBakeServer(object):
return self.server.register_idle_function
def saveConnectionDetails(self):
- self.serverinfo = BitbakeServerInfo(self.server)
+ self.serverinfo = BitbakeServerInfo(self.server.host, self.server.port)
def detach(self, cooker_logfile):
daemonize.createDaemon(self.server.serve_forever, cooker_logfile)
--
1.7.0.4
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH 09/10] cooker: remove command import in cooker.py
2011-12-15 7:14 [PATCH 00/10 v2] Hob2 related bitbake changes Dongxiao Xu
` (7 preceding siblings ...)
2011-12-15 7:14 ` [PATCH 08/10] xmlrpc: Change BitbakeServerInfo init function Dongxiao Xu
@ 2011-12-15 7:15 ` Dongxiao Xu
2011-12-15 7:15 ` [PATCH 10/10] bitbake: add a new option "--server-only" Dongxiao Xu
2011-12-19 11:00 ` [PATCH 00/10 v2] Hob2 related bitbake changes Xu, Dongxiao
10 siblings, 0 replies; 12+ messages in thread
From: Dongxiao Xu @ 2011-12-15 7:15 UTC (permalink / raw)
To: bitbake-devel
There is no direct use of command in cooker.py, and it is using
bb.command instead. Remove command in the import list.
This fixes a problem of embedded import between command.py and
cooker.py.
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
---
lib/bb/cooker.py | 4 ++--
1 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/lib/bb/cooker.py b/lib/bb/cooker.py
index 4401a66..54bf880 100644
--- a/lib/bb/cooker.py
+++ b/lib/bb/cooker.py
@@ -34,8 +34,8 @@ from cStringIO import StringIO
from contextlib import closing
from functools import wraps
from collections import defaultdict
-import bb, bb.exceptions
-from bb import utils, data, parse, event, cache, providers, taskdata, command, runqueue
+import bb, bb.exceptions, bb.command
+from bb import utils, data, parse, event, cache, providers, taskdata, runqueue
logger = logging.getLogger("BitBake")
collectlog = logging.getLogger("BitBake.Collection")
--
1.7.0.4
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH 10/10] bitbake: add a new option "--server-only"
2011-12-15 7:14 [PATCH 00/10 v2] Hob2 related bitbake changes Dongxiao Xu
` (8 preceding siblings ...)
2011-12-15 7:15 ` [PATCH 09/10] cooker: remove command import in cooker.py Dongxiao Xu
@ 2011-12-15 7:15 ` Dongxiao Xu
2011-12-19 11:00 ` [PATCH 00/10 v2] Hob2 related bitbake changes Xu, Dongxiao
10 siblings, 0 replies; 12+ messages in thread
From: Dongxiao Xu @ 2011-12-15 7:15 UTC (permalink / raw)
To: bitbake-devel
Create a new option "--server-only" for bitbake command, which allows
bitbake runs as a server, and let frontend connect the server itself.
"--server-only" should work with "-t xmlrpc", or bitbake will exit.
bitbake --server-only -t xmlrpc will print out the server address and
port information.
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
---
bin/bitbake | 23 ++++++++++++++++-------
1 files changed, 16 insertions(+), 7 deletions(-)
diff --git a/bin/bitbake b/bin/bitbake
index 9eb558b..c2e6822 100755
--- a/bin/bitbake
+++ b/bin/bitbake
@@ -165,6 +165,9 @@ Default BBFILES are the .bb files in the current directory.""")
parser.add_option("", "--revisions-changed", help = "Set the exit code depending on whether upstream floating revisions have changed or not",
action = "store_true", dest = "revisions_changed", default = False)
+ parser.add_option("", "--server-only", help = "Run bitbake without UI, the frontend can connect with bitbake server itself",
+ action = "store_true", dest = "server_only", default = False)
+
options, args = parser.parse_args(sys.argv)
configuration = BBConfiguration(options)
@@ -186,6 +189,9 @@ Default BBFILES are the .bb files in the current directory.""")
sys.exit("FATAL: Invalid server type '%s' specified.\n"
"Valid interfaces: xmlrpc, process [default], none." % servertype)
+ if configuration.server_only and configuration.servertype != "xmlrpc":
+ sys.exit("FATAL: If '--server-only' is defined, we must set the servertype as 'xmlrpc'.\n")
+
# Save a logfile for cooker into the current working directory. When the
# server is daemonized this logfile will be truncated.
cooker_logfile = os.path.join(os.getcwd(), "cooker.log")
@@ -222,14 +228,17 @@ Default BBFILES are the .bb files in the current directory.""")
logger.removeHandler(handler)
- # Setup a connection to the server (cooker)
- server_connection = server.establishConnection()
+ if not configuration.server_only:
+ # Setup a connection to the server (cooker)
+ server_connection = server.establishConnection()
- try:
- return server.launchUI(ui_main, server_connection.connection, server_connection.events)
- finally:
- bb.event.ui_queue = []
- server_connection.terminate()
+ try:
+ return server.launchUI(ui_main, server_connection.connection, server_connection.events)
+ finally:
+ bb.event.ui_queue = []
+ server_connection.terminate()
+ else:
+ print("server address: %s, server port: %s" % (server.serverinfo.host, server.serverinfo.port))
return 1
--
1.7.0.4
^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [PATCH 00/10 v2] Hob2 related bitbake changes
2011-12-15 7:14 [PATCH 00/10 v2] Hob2 related bitbake changes Dongxiao Xu
` (9 preceding siblings ...)
2011-12-15 7:15 ` [PATCH 10/10] bitbake: add a new option "--server-only" Dongxiao Xu
@ 2011-12-19 11:00 ` Xu, Dongxiao
10 siblings, 0 replies; 12+ messages in thread
From: Xu, Dongxiao @ 2011-12-19 11:00 UTC (permalink / raw)
To: Xu, Dongxiao, bitbake-devel
Hi Richard,
Could you hold this patchset for a moment? I have some new version for it and will send out soon.
Sorry for inconvenience.
Thanks,
Dongxiao
> -----Original Message-----
> From: bitbake-devel-bounces@lists.openembedded.org
> [mailto:bitbake-devel-bounces@lists.openembedded.org] On Behalf Of
> Dongxiao Xu
> Sent: Thursday, December 15, 2011 3:15 PM
> To: bitbake-devel@lists.openembedded.org
> Subject: [bitbake-devel] [PATCH 00/10 v2] Hob2 related bitbake changes
>
> Hi Richard,
>
> This pull request is the second version that introduces several hob2 related
> bitbake changes.
> Please help to review and pull.
>
> Changes from v1:
> Add "resolve" option in generateTargetsTree() instead of adding a new
> generateTargetsTreePro function. Thanks for Josh's suggestions.
>
> Thanks,
> Dongxiao
>
> The following changes since commit
> 71b53a3f0766ca464560a1f6a449f9424fbdf7ae:
> Matthew McClintock (1):
> siggen.py: If both sigs have a variable in it's whitelist then don't say it's
> changed
>
> are available in the git repository at:
>
> git://git.pokylinux.org/poky-contrib dxu4/hob2-bitbake-change-v2
>
> http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=dxu4/hob2-bitbake-chang
> e-v2
>
> Dongxiao Xu (10):
> command.py: Modify needcache value for certain functions
> cache: Use configuration's hash value to validate cache
> cooker: user bb.configuration.data to inject events
> command.py: add initCooker API
> command.py: add parseConfigurationFiles API
> command.py: add resolve option for generateTargetsTree API
> event.py: Add a new event PackageInfo
> xmlrpc: Change BitbakeServerInfo init function
> cooker: remove command import in cooker.py
> bitbake: add a new option "--server-only"
>
> bin/bitbake | 23 +++++++++++++------
> lib/bb/cache.py | 32 ++++++++++-----------------
> lib/bb/command.py | 31 +++++++++++++++++++++++---
> lib/bb/cooker.py | 55
> ++++++++++++++++++++++++++++++++++++++--------
> lib/bb/data_smart.py | 17 ++++++++++++++
> lib/bb/event.py | 9 +++++++
> lib/bb/server/xmlrpc.py | 8 +++---
> 7 files changed, 130 insertions(+), 45 deletions(-)
>
>
> _______________________________________________
> bitbake-devel mailing list
> bitbake-devel@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/bitbake-devel
^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2011-12-19 11:08 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-12-15 7:14 [PATCH 00/10 v2] Hob2 related bitbake changes Dongxiao Xu
2011-12-15 7:14 ` [PATCH 01/10] command.py: Modify needcache value for certain functions Dongxiao Xu
2011-12-15 7:14 ` [PATCH 02/10] cache: Use configuration's hash value to validate cache Dongxiao Xu
2011-12-15 7:14 ` [PATCH 03/10] cooker: user bb.configuration.data to inject events Dongxiao Xu
2011-12-15 7:14 ` [PATCH 04/10] command.py: add initCooker API Dongxiao Xu
2011-12-15 7:14 ` [PATCH 05/10] command.py: add parseConfigurationFiles API Dongxiao Xu
2011-12-15 7:14 ` [PATCH 06/10] command.py: add resolve option for generateTargetsTree API Dongxiao Xu
2011-12-15 7:14 ` [PATCH 07/10] event.py: Add a new event PackageInfo Dongxiao Xu
2011-12-15 7:14 ` [PATCH 08/10] xmlrpc: Change BitbakeServerInfo init function Dongxiao Xu
2011-12-15 7:15 ` [PATCH 09/10] cooker: remove command import in cooker.py Dongxiao Xu
2011-12-15 7:15 ` [PATCH 10/10] bitbake: add a new option "--server-only" Dongxiao Xu
2011-12-19 11:00 ` [PATCH 00/10 v2] Hob2 related bitbake changes Xu, Dongxiao
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.