* [PATCH 00/12] devtool improvements
@ 2017-12-05 1:41 Paul Eggleton
2017-12-05 1:41 ` [PATCH 01/12] devtool: refactor return for _extract_source() Paul Eggleton
` (13 more replies)
0 siblings, 14 replies; 16+ messages in thread
From: Paul Eggleton @ 2017-12-05 1:41 UTC (permalink / raw)
To: openembedded-core
A set of improvements for devtool, mostly aimed at improving the
handling of recipes that fetch multiple source trees (i.e. have multiple
tarballs or repositories in SRC_URI), with a couple of other fixes
thrown in.
The following changes since commit a7cd9d1183be603777fc9c8c448281fe01224f7b:
lib/oe/utils: remove param_bool() (2017-12-02 11:24:38 +0000)
are available in the git repository at:
git://git.openembedded.org/openembedded-core-contrib paule/devtool32-oe
http://cgit.openembedded.org/openembedded-core-contrib/log/?h=paule/devtool32-oe
Paul Eggleton (12):
devtool: refactor return for _extract_source()
devtool: reset: delete bbappend file if _check_preserve() doesn't
devtool: finish: fix erroneously creating bbappend for relative paths
devtool: deploy-target: don't specify ssh/scp port unless user does
devtool: extract: fix usage with kern-tools-native
devtool: modify: pick up commits from previously created source tree
lib/oe/recipeutils: allow patch_recipe_file() to be re-called
devtool: support extracting multiple source trees
devtool: fix handling of linux-yocto after multisrc changes
oe-selftest: devtool: fix kernel test for multisrc changes
oe-selftest: devtool: add test for multiple source trees
lib/oe/recipeutils: add .txz extension to archive list
.../devtool/devtool-test-multisrc-subdir_1.0.bb | 13 +
.../devtool-test-multisrc/example-files.tar.gz | Bin 0 -> 236 bytes
.../devtool/devtool-test-multisrc/example.patch | 7 +
.../devtool-test-multisrc/mypackage-1.0.tar.gz | Bin 0 -> 903 bytes
.../devtool/devtool-test-multisrc_1.0.bb | 11 +
meta/classes/devtool-source.bbclass | 63 ++-
meta/classes/externalsrc.bbclass | 2 +-
meta/classes/kernel-yocto.bbclass | 7 +-
meta/lib/oe/recipeutils.py | 15 +-
meta/lib/oe/utils.py | 24 ++
meta/lib/oeqa/selftest/cases/devtool.py | 124 +++++-
scripts/devtool | 61 ++-
scripts/lib/devtool/__init__.py | 14 +
scripts/lib/devtool/deploy.py | 12 +-
scripts/lib/devtool/standard.py | 426 ++++++++++++++-------
scripts/lib/devtool/upgrade.py | 4 +-
16 files changed, 601 insertions(+), 182 deletions(-)
create mode 100644 meta-selftest/recipes-test/devtool/devtool-test-multisrc-subdir_1.0.bb
create mode 100644 meta-selftest/recipes-test/devtool/devtool-test-multisrc/example-files.tar.gz
create mode 100644 meta-selftest/recipes-test/devtool/devtool-test-multisrc/example.patch
create mode 100644 meta-selftest/recipes-test/devtool/devtool-test-multisrc/mypackage-1.0.tar.gz
create mode 100644 meta-selftest/recipes-test/devtool/devtool-test-multisrc_1.0.bb
--
2.9.5
^ permalink raw reply [flat|nested] 16+ messages in thread
* [PATCH 01/12] devtool: refactor return for _extract_source()
2017-12-05 1:41 [PATCH 00/12] devtool improvements Paul Eggleton
@ 2017-12-05 1:41 ` Paul Eggleton
2017-12-05 1:41 ` [PATCH 02/12] devtool: reset: delete bbappend file if _check_preserve() doesn't Paul Eggleton
` (12 subsequent siblings)
13 siblings, 0 replies; 16+ messages in thread
From: Paul Eggleton @ 2017-12-05 1:41 UTC (permalink / raw)
To: openembedded-core
Use a namedtuple to return information to the caller, since I've been
expanding that information we should avoid having to change all of the
calling code each time.
Additionally, it turned out that a bunch of the callers were checking
for None being returned in the initial_rev value, but that's no longer
possible, so tidy up the calling code.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
scripts/lib/devtool/standard.py | 29 ++++++++++-------------------
scripts/lib/devtool/upgrade.py | 4 ++--
2 files changed, 12 insertions(+), 21 deletions(-)
diff --git a/scripts/lib/devtool/standard.py b/scripts/lib/devtool/standard.py
index 26187a0..b1669b5 100644
--- a/scripts/lib/devtool/standard.py
+++ b/scripts/lib/devtool/standard.py
@@ -29,7 +29,7 @@ import scriptutils
import errno
import glob
import filecmp
-from collections import OrderedDict
+from collections import OrderedDict, namedtuple
from devtool import exec_build_env_command, setup_tinfoil, check_workspace_recipe, use_external_build, setup_git_repo, recipe_to_append, get_bbclassextend_targets, update_unlockedsigs, check_prerelease_version, check_git_repo_dirty, check_git_repo_op, DevtoolError
from devtool import parse_recipe
@@ -437,15 +437,11 @@ def extract(args, config, basepath, workspace):
return 1
srctree = os.path.abspath(args.srctree)
- initial_rev, _ = _extract_source(srctree, args.keep_temp, args.branch, False, config, basepath, workspace, args.fixed_setup, rd, tinfoil, no_overrides=args.no_overrides)
+ _extract_source(srctree, args.keep_temp, args.branch, False, config, basepath, workspace, args.fixed_setup, rd, tinfoil, no_overrides=args.no_overrides)
logger.info('Source tree extracted to %s' % srctree)
-
- if initial_rev:
- return 0
- else:
- return 1
finally:
tinfoil.shutdown()
+ return 0
def sync(args, config, basepath, workspace):
"""Entry point for the devtool 'sync' subcommand"""
@@ -461,17 +457,15 @@ def sync(args, config, basepath, workspace):
return 1
srctree = os.path.abspath(args.srctree)
- initial_rev, _ = _extract_source(srctree, args.keep_temp, args.branch, True, config, basepath, workspace, args.fixed_setup, rd, tinfoil, no_overrides=True)
+ _extract_source(srctree, args.keep_temp, args.branch, True, config, basepath, workspace, args.fixed_setup, rd, tinfoil, no_overrides=True)
logger.info('Source tree %s synchronized' % srctree)
-
- if initial_rev:
- return 0
- else:
- return 1
finally:
tinfoil.shutdown()
+ return 0
+ExtractSourceResult = namedtuple('ExtractSourceResult', ['initial_rev', 'srcsubdir'])
+
def _extract_source(srctree, keep_temp, devbranch, sync, config, basepath, workspace, fixed_setup, d, tinfoil, no_overrides=False):
"""Extract sources of a recipe"""
import oe.recipeutils
@@ -512,8 +506,6 @@ def _extract_source(srctree, keep_temp, devbranch, sync, config, basepath, works
if extra_overrides:
logger.info('SRC_URI contains some conditional appends/prepends - will create branches to represent these')
- initial_rev = None
-
appendexisted = False
recipefile = d.getVar('FILE')
appendfile = recipe_to_append(recipefile, config)
@@ -659,7 +651,7 @@ def _extract_source(srctree, keep_temp, devbranch, sync, config, basepath, works
logger.info('Preserving temporary directory %s' % tempdir)
else:
shutil.rmtree(tempdir)
- return initial_rev, srcsubdir_rel
+ return ExtractSourceResult(initial_rev, srcsubdir_rel)
def _add_md5(config, recipename, filename):
"""Record checksum of a file (or recursively for a directory) to the md5-file of the workspace"""
@@ -759,9 +751,8 @@ def modify(args, config, basepath, workspace):
commits = []
check_commits = False
if not args.no_extract:
- initial_rev, _ = _extract_source(srctree, args.keep_temp, args.branch, False, config, basepath, workspace, args.fixed_setup, rd, tinfoil, no_overrides=args.no_overrides)
- if not initial_rev:
- return 1
+ ret = _extract_source(srctree, args.keep_temp, args.branch, False, config, basepath, workspace, args.fixed_setup, rd, tinfoil, no_overrides=args.no_overrides)
+ initial_rev = ret.initial_rev
logger.info('Source tree extracted to %s' % srctree)
# Get list of commits since this revision
(stdout, _) = bb.process.run('git rev-list --reverse %s..HEAD' % initial_rev, cwd=srctree)
diff --git a/scripts/lib/devtool/upgrade.py b/scripts/lib/devtool/upgrade.py
index f6141bf..7afdd80 100644
--- a/scripts/lib/devtool/upgrade.py
+++ b/scripts/lib/devtool/upgrade.py
@@ -474,12 +474,12 @@ def upgrade(args, config, basepath, workspace):
rf = None
try:
logger.info('Extracting current version source...')
- rev1, srcsubdir1 = standard._extract_source(srctree, False, 'devtool-orig', False, config, basepath, workspace, args.fixed_setup, rd, tinfoil, no_overrides=args.no_overrides)
+ current_ret = standard._extract_source(srctree, False, 'devtool-orig', False, config, basepath, workspace, args.fixed_setup, rd, tinfoil, no_overrides=args.no_overrides)
logger.info('Extracting upgraded version source...')
rev2, md5, sha256, srcbranch, srcsubdir2 = _extract_new_source(args.version, srctree, args.no_patch,
args.srcrev, args.srcbranch, args.branch, args.keep_temp,
tinfoil, rd)
- rf, copied = _create_new_recipe(args.version, md5, sha256, args.srcrev, srcbranch, srcsubdir1, srcsubdir2, config.workspace_path, tinfoil, rd)
+ rf, copied = _create_new_recipe(args.version, md5, sha256, args.srcrev, srcbranch, current_ret.srcsubdir, srcsubdir2, config.workspace_path, tinfoil, rd)
except bb.process.CmdError as e:
_upgrade_error(e, rf, srctree)
except DevtoolError as e:
--
2.9.5
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH 02/12] devtool: reset: delete bbappend file if _check_preserve() doesn't
2017-12-05 1:41 [PATCH 00/12] devtool improvements Paul Eggleton
2017-12-05 1:41 ` [PATCH 01/12] devtool: refactor return for _extract_source() Paul Eggleton
@ 2017-12-05 1:41 ` Paul Eggleton
2017-12-05 1:41 ` [PATCH 03/12] devtool: finish: fix erroneously creating bbappend for relative paths Paul Eggleton
` (11 subsequent siblings)
13 siblings, 0 replies; 16+ messages in thread
From: Paul Eggleton @ 2017-12-05 1:41 UTC (permalink / raw)
To: openembedded-core
If the .devtool_md5 file doesn't contain a reference to the bbappend
file (e.g. because devtool was interrupted before it could write that
out) then _check_preserve() won't delete it, so we need to delete it
separately because otherwise the recipe won't actually be reset.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
scripts/lib/devtool/standard.py | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/scripts/lib/devtool/standard.py b/scripts/lib/devtool/standard.py
index b1669b5..f59c6d6 100644
--- a/scripts/lib/devtool/standard.py
+++ b/scripts/lib/devtool/standard.py
@@ -1781,6 +1781,13 @@ def _reset(recipes, no_clean, config, basepath, workspace):
for pn in recipes:
_check_preserve(config, pn)
+ appendfile = workspace[pn]['bbappend']
+ if os.path.exists(appendfile):
+ # This shouldn't happen, but is possible if devtool errored out prior to
+ # writing the md5 file. We need to delete this here or the recipe won't
+ # actually be reset
+ os.remove(appendfile)
+
preservepath = os.path.join(config.workspace_path, 'attic', pn, pn)
def preservedir(origdir):
if os.path.exists(origdir):
--
2.9.5
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH 03/12] devtool: finish: fix erroneously creating bbappend for relative paths
2017-12-05 1:41 [PATCH 00/12] devtool improvements Paul Eggleton
2017-12-05 1:41 ` [PATCH 01/12] devtool: refactor return for _extract_source() Paul Eggleton
2017-12-05 1:41 ` [PATCH 02/12] devtool: reset: delete bbappend file if _check_preserve() doesn't Paul Eggleton
@ 2017-12-05 1:41 ` Paul Eggleton
2017-12-05 1:41 ` [PATCH 04/12] devtool: deploy-target: don't specify ssh/scp port unless user does Paul Eggleton
` (10 subsequent siblings)
13 siblings, 0 replies; 16+ messages in thread
From: Paul Eggleton @ 2017-12-05 1:41 UTC (permalink / raw)
To: openembedded-core
After OE-Core rev 5e3fe00a0233d563781849a44f53885b4e924a9c we call
os.path.abspath() on the original layer path, but we later compare that
to the destination layer path. If that layer path isn't absolute but is
effectively the same path, it should be writing to the original recipe
but because we weren't making it absolute we were writing a bbappend
instead. Call os.path.abspath() on the destination path as well to avoid
that.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
scripts/lib/devtool/standard.py | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git a/scripts/lib/devtool/standard.py b/scripts/lib/devtool/standard.py
index f59c6d6..cdd2346 100644
--- a/scripts/lib/devtool/standard.py
+++ b/scripts/lib/devtool/standard.py
@@ -1848,9 +1848,7 @@ def _get_layer(layername, d):
layerdir = layers.get('meta', None)
else:
layerdir = layers.get(layername, None)
- if layerdir:
- layerdir = os.path.abspath(layerdir)
- return layerdir or layername
+ return os.path.abspath(layerdir or layername)
def finish(args, config, basepath, workspace):
"""Entry point for the devtool 'finish' subcommand"""
--
2.9.5
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH 04/12] devtool: deploy-target: don't specify ssh/scp port unless user does
2017-12-05 1:41 [PATCH 00/12] devtool improvements Paul Eggleton
` (2 preceding siblings ...)
2017-12-05 1:41 ` [PATCH 03/12] devtool: finish: fix erroneously creating bbappend for relative paths Paul Eggleton
@ 2017-12-05 1:41 ` Paul Eggleton
2017-12-05 1:41 ` [PATCH 05/12] devtool: extract: fix usage with kern-tools-native Paul Eggleton
` (9 subsequent siblings)
13 siblings, 0 replies; 16+ messages in thread
From: Paul Eggleton @ 2017-12-05 1:41 UTC (permalink / raw)
To: openembedded-core
If the user doesn't specify a port then we should avoid specifying one
on the ssh/scp command line in case the user has configured one for the
host they are connecting to, which was being overridden unnecessarily.
Fixes [YOCTO #12381].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
scripts/lib/devtool/deploy.py | 12 ++++--------
1 file changed, 4 insertions(+), 8 deletions(-)
diff --git a/scripts/lib/devtool/deploy.py b/scripts/lib/devtool/deploy.py
index 9cc4927..52e261d 100644
--- a/scripts/lib/devtool/deploy.py
+++ b/scripts/lib/devtool/deploy.py
@@ -213,9 +213,7 @@ def deploy(args, config, basepath, workspace):
scp_port = ''
ssh_port = ''
- if not args.port:
- raise DevtoolError("If you specify -P/--port then you must provide the port to be used to connect to the target")
- else:
+ if args.port:
scp_port = "-P %s" % args.port
ssh_port = "-p %s" % args.port
@@ -280,9 +278,7 @@ def undeploy(args, config, basepath, workspace):
scp_port = ''
ssh_port = ''
- if not args.port:
- raise DevtoolError("If you specify -P/--port then you must provide the port to be used to connect to the target")
- else:
+ if args.port:
scp_port = "-P %s" % args.port
ssh_port = "-p %s" % args.port
@@ -328,7 +324,7 @@ def register_commands(subparsers, context):
parser_deploy.add_argument('-n', '--dry-run', help='List files to be deployed only', action='store_true')
parser_deploy.add_argument('-p', '--no-preserve', help='Do not preserve existing files', action='store_true')
parser_deploy.add_argument('--no-check-space', help='Do not check for available space before deploying', action='store_true')
- parser_deploy.add_argument('-P', '--port', default='22', help='Port to use for connection to the target')
+ parser_deploy.add_argument('-P', '--port', help='Specify port to use for connection to the target')
strip_opts = parser_deploy.add_mutually_exclusive_group(required=False)
strip_opts.add_argument('-S', '--strip',
@@ -350,5 +346,5 @@ def register_commands(subparsers, context):
parser_undeploy.add_argument('-s', '--show-status', help='Show progress/status output', action='store_true')
parser_undeploy.add_argument('-a', '--all', help='Undeploy all recipes deployed on the target', action='store_true')
parser_undeploy.add_argument('-n', '--dry-run', help='List files to be undeployed only', action='store_true')
- parser_undeploy.add_argument('-P', '--port', default='22', help='Port to use for connection to the target')
+ parser_undeploy.add_argument('-P', '--port', help='Specify port to use for connection to the target')
parser_undeploy.set_defaults(func=undeploy)
--
2.9.5
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH 05/12] devtool: extract: fix usage with kern-tools-native
2017-12-05 1:41 [PATCH 00/12] devtool improvements Paul Eggleton
` (3 preceding siblings ...)
2017-12-05 1:41 ` [PATCH 04/12] devtool: deploy-target: don't specify ssh/scp port unless user does Paul Eggleton
@ 2017-12-05 1:41 ` Paul Eggleton
2017-12-05 1:41 ` [PATCH 06/12] devtool: modify: pick up commits from previously created source tree Paul Eggleton
` (8 subsequent siblings)
13 siblings, 0 replies; 16+ messages in thread
From: Paul Eggleton @ 2017-12-05 1:41 UTC (permalink / raw)
To: openembedded-core
The kern-tools-native recipe as it currently stands is unusual in that
it fetches source from a repository but sets S = "${WORKDIR}" which
causes some problems. First you get a failure because we're calling "git
commit" unconditionally even if there are no local files, and there
aren't any in this case which means the commit fails.
After that's fixed, we hit another problem where "recipe-sysroot-native"
subdirectory appears in the extracted source tree. We don't want that so
exclude it from copying.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
meta/classes/devtool-source.bbclass | 6 ++++++
scripts/lib/devtool/standard.py | 6 +++---
2 files changed, 9 insertions(+), 3 deletions(-)
diff --git a/meta/classes/devtool-source.bbclass b/meta/classes/devtool-source.bbclass
index 56882a4..540a602 100644
--- a/meta/classes/devtool-source.bbclass
+++ b/meta/classes/devtool-source.bbclass
@@ -118,7 +118,13 @@ python devtool_post_unpack() {
os.path.basename(fname) not in recipe_patches]
srcsubdir = d.getVar('DEVTOOL_PATCH_SRCDIR')
# Move source files to S
+ excludevars = ['RECIPE_SYSROOT', 'RECIPE_SYSROOT_NATIVE']
+ excludepaths = []
+ for excludevar in excludevars:
+ excludepaths.append(os.path.relpath(d.getVar(excludevar), workdir) + os.sep)
for path in src_files:
+ if path.startswith(tuple(excludepaths)):
+ continue
_move_file(os.path.join(workdir, path),
os.path.join(srcsubdir, path))
elif os.path.dirname(srcsubdir) != workdir:
diff --git a/scripts/lib/devtool/standard.py b/scripts/lib/devtool/standard.py
index cdd2346..a6656e4 100644
--- a/scripts/lib/devtool/standard.py
+++ b/scripts/lib/devtool/standard.py
@@ -634,9 +634,9 @@ def _extract_source(srctree, keep_temp, devbranch, sync, config, basepath, works
addfiles.append(os.path.join(relpth, fn))
if addfiles:
bb.process.run('git add %s' % ' '.join(addfiles), cwd=srctree)
- useroptions = []
- oe.patch.GitApplyTree.gitCommandUserOptions(useroptions, d=d)
- bb.process.run('git %s commit -a -m "Committing local file symlinks\n\n%s"' % (' '.join(useroptions), oe.patch.GitApplyTree.ignore_commit_prefix), cwd=srctree)
+ useroptions = []
+ oe.patch.GitApplyTree.gitCommandUserOptions(useroptions, d=d)
+ bb.process.run('git %s commit -a -m "Committing local file symlinks\n\n%s"' % (' '.join(useroptions), oe.patch.GitApplyTree.ignore_commit_prefix), cwd=srctree)
if is_kernel_yocto:
logger.info('Copying kernel config to srctree')
--
2.9.5
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH 06/12] devtool: modify: pick up commits from previously created source tree
2017-12-05 1:41 [PATCH 00/12] devtool improvements Paul Eggleton
` (4 preceding siblings ...)
2017-12-05 1:41 ` [PATCH 05/12] devtool: extract: fix usage with kern-tools-native Paul Eggleton
@ 2017-12-05 1:41 ` Paul Eggleton
2017-12-05 1:41 ` [PATCH 07/12] lib/oe/recipeutils: allow patch_recipe_file() to be re-called Paul Eggleton
` (7 subsequent siblings)
13 siblings, 0 replies; 16+ messages in thread
From: Paul Eggleton @ 2017-12-05 1:41 UTC (permalink / raw)
To: openembedded-core
If you use devtool modify, then devtool reset, keep the source tree and
then devtool modify on the same recipe with the -n option to re-use the
existing source tree, we should pick up the commit hashes properly from
the source tree so that later on devtool finish has these to compare to
the commits in the tree at that time. We also need to be careful the
second time around that we only get the original commits rather than the
current HEAD which may be the result of user changes (hence using
"devtool-patched", the tag that was placed at the original HEAD).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
scripts/lib/devtool/standard.py | 10 +++++++---
1 file changed, 7 insertions(+), 3 deletions(-)
diff --git a/scripts/lib/devtool/standard.py b/scripts/lib/devtool/standard.py
index a6656e4..ae48406 100644
--- a/scripts/lib/devtool/standard.py
+++ b/scripts/lib/devtool/standard.py
@@ -750,13 +750,11 @@ def modify(args, config, basepath, workspace):
initial_rev = None
commits = []
check_commits = False
+ torev = 'HEAD'
if not args.no_extract:
ret = _extract_source(srctree, args.keep_temp, args.branch, False, config, basepath, workspace, args.fixed_setup, rd, tinfoil, no_overrides=args.no_overrides)
initial_rev = ret.initial_rev
logger.info('Source tree extracted to %s' % srctree)
- # Get list of commits since this revision
- (stdout, _) = bb.process.run('git rev-list --reverse %s..HEAD' % initial_rev, cwd=srctree)
- commits = stdout.split()
check_commits = True
else:
if os.path.exists(os.path.join(srctree, '.git')):
@@ -767,6 +765,7 @@ def modify(args, config, basepath, workspace):
stdout = ''
if stdout:
check_commits = True
+ torev = 'devtool-patched'
for line in stdout.splitlines():
if line.startswith('*'):
(stdout, _) = bb.process.run('git rev-parse devtool-base', cwd=srctree)
@@ -776,6 +775,11 @@ def modify(args, config, basepath, workspace):
(stdout, _) = bb.process.run('git rev-parse HEAD', cwd=srctree)
initial_rev = stdout.rstrip()
+ if initial_rev:
+ # Get list of commits since this revision
+ (stdout, _) = bb.process.run('git rev-list --reverse %s..%s' % (initial_rev, torev), cwd=srctree)
+ commits = stdout.split()
+
branch_patches = {}
if check_commits:
# Check if there are override branches
--
2.9.5
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH 07/12] lib/oe/recipeutils: allow patch_recipe_file() to be re-called
2017-12-05 1:41 [PATCH 00/12] devtool improvements Paul Eggleton
` (5 preceding siblings ...)
2017-12-05 1:41 ` [PATCH 06/12] devtool: modify: pick up commits from previously created source tree Paul Eggleton
@ 2017-12-05 1:41 ` Paul Eggleton
2017-12-05 1:41 ` [PATCH 08/12] devtool: support extracting multiple source trees Paul Eggleton
` (6 subsequent siblings)
13 siblings, 0 replies; 16+ messages in thread
From: Paul Eggleton @ 2017-12-05 1:41 UTC (permalink / raw)
To: openembedded-core
If patch_recipe_file() is called with output redirection on the same
file twice in succession, then we don't want to wipe out the changes the
first call made so we need to be reading in the redirected file if it
exists instead of the original one.
This is important to enable devtool finish to work with multiple source
trees within the same recipe.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
meta/lib/oe/recipeutils.py | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)
diff --git a/meta/lib/oe/recipeutils.py b/meta/lib/oe/recipeutils.py
index 4e0859e..4928727 100644
--- a/meta/lib/oe/recipeutils.py
+++ b/meta/lib/oe/recipeutils.py
@@ -258,13 +258,18 @@ def patch_recipe_file(fn, values, patch=False, relpath='', redirect_output=None)
since this cannot handle all situations.
"""
- with open(fn, 'r') as f:
+ read_fn = fn
+ if redirect_output:
+ redirect_fn = os.path.join(redirect_output, os.path.basename(fn))
+ if os.path.exists(redirect_fn):
+ read_fn = redirect_fn
+ with open(read_fn, 'r') as f:
fromlines = f.readlines()
_, tolines = patch_recipe_lines(fromlines, values)
if redirect_output:
- with open(os.path.join(redirect_output, os.path.basename(fn)), 'w') as f:
+ with open(redirect_fn, 'w') as f:
f.writelines(tolines)
return None
elif patch:
--
2.9.5
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH 08/12] devtool: support extracting multiple source trees
2017-12-05 1:41 [PATCH 00/12] devtool improvements Paul Eggleton
` (6 preceding siblings ...)
2017-12-05 1:41 ` [PATCH 07/12] lib/oe/recipeutils: allow patch_recipe_file() to be re-called Paul Eggleton
@ 2017-12-05 1:41 ` Paul Eggleton
2017-12-05 1:41 ` [PATCH 09/12] devtool: fix handling of linux-yocto after multisrc changes Paul Eggleton
` (5 subsequent siblings)
13 siblings, 0 replies; 16+ messages in thread
From: Paul Eggleton @ 2017-12-05 1:41 UTC (permalink / raw)
To: openembedded-core
If you have multiple source trees being extracted to the work directory
within a recipe (e.g. you have two tarballs referred to in SRC_URI) and
one isn't being extracted into the other, then devtool failed to extract
all the sources because it only took the source tree that S pointed
into. To fix this, we need to take a look at the work directory after
do_unpack and see if there are any additional subdirectories; if so we
need to put the main source tree in a subdirectory and put the
additional subdirectories next to it. We also ensure that symlinks from
the work directory get created at the end of do_unpack to point to
these (so that references in the recipe continue to work).
In addition to multiple source trees at the work directory level, this
patch also handles multiple nested git trees (where you have multiple
git URLs in SRC_URI with one or more inside another). These caused a
different problem, where changes in sub-repos are not fully captured at
the top level - we need to handle each repo separately.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
meta/classes/devtool-source.bbclass | 57 ++++-
meta/lib/oe/recipeutils.py | 4 +-
meta/lib/oe/utils.py | 24 +++
meta/lib/oeqa/selftest/cases/devtool.py | 2 +-
scripts/devtool | 61 ++++--
scripts/lib/devtool/__init__.py | 14 ++
scripts/lib/devtool/standard.py | 370 +++++++++++++++++++++++---------
7 files changed, 396 insertions(+), 136 deletions(-)
diff --git a/meta/classes/devtool-source.bbclass b/meta/classes/devtool-source.bbclass
index 540a602..30c9bdb 100644
--- a/meta/classes/devtool-source.bbclass
+++ b/meta/classes/devtool-source.bbclass
@@ -69,7 +69,7 @@ python devtool_post_unpack() {
import shutil
sys.path.insert(0, os.path.join(d.getVar('COREBASE'), 'scripts', 'lib'))
import scriptutils
- from devtool import setup_git_repo
+ from devtool import setup_git_repo, find_git_repos
tempdir = d.getVar('DEVTOOL_TEMPDIR')
workdir = d.getVar('WORKDIR')
@@ -95,8 +95,26 @@ python devtool_post_unpack() {
oe.recipeutils.get_recipe_patches(d)]
local_files = oe.recipeutils.get_recipe_local_files(d)
- # Ignore local files with subdir={BP}
+ excludeitems = recipe_patches + list(local_files.keys())
+ pthvars = ['RECIPE_SYSROOT', 'RECIPE_SYSROOT_NATIVE', 'S']
+ for pthvar in pthvars:
+ relp = os.path.relpath(d.getVar(pthvar), d.getVar('WORKDIR'))
+ if not relp.startswith('..'):
+ excludeitems.append(relp.split(os.sep)[0])
+ extradirs = []
srcabspath = os.path.abspath(srcsubdir)
+ if srcabspath != os.path.abspath(workdir):
+ for pth in os.listdir(workdir):
+ if pth in excludeitems:
+ continue
+ wpth = os.path.join(workdir, pth)
+ if os.path.isdir(wpth) and os.listdir(wpth):
+ extradirs.append(wpth)
+
+ repos = find_git_repos(srcabspath)
+ extradirs.extend(repos)
+
+ # Ignore local files with subdir={BP}
local_files = [fname for fname in local_files if
os.path.exists(os.path.join(workdir, fname)) and
(srcabspath == workdir or not
@@ -145,11 +163,20 @@ python devtool_post_unpack() {
(stdout, _) = bb.process.run('git rev-parse HEAD', cwd=srcsubdir)
initial_rev = stdout.rstrip()
+
+ initial_revs = {}
+ for extradir in extradirs:
+ setup_git_repo(extradir, d.getVar('PV'), devbranch, d=d)
+ (stdout, _) = bb.process.run('git rev-parse HEAD', cwd=extradir)
+ initial_revs[extradir] = stdout.rstrip()
+
with open(os.path.join(tempdir, 'initial_rev'), 'w') as f:
f.write(initial_rev)
with open(os.path.join(tempdir, 'srcsubdir'), 'w') as f:
- f.write(srcsubdir)
+ f.write('%s\n' % srcsubdir)
+ for extradir in extradirs:
+ f.write('%s=%s\n' % (extradir, initial_revs[extradir]))
}
python devtool_pre_patch() {
@@ -160,18 +187,25 @@ python devtool_pre_patch() {
python devtool_post_patch() {
import shutil
tempdir = d.getVar('DEVTOOL_TEMPDIR')
+
+ srcdirs = []
with open(os.path.join(tempdir, 'srcsubdir'), 'r') as f:
- srcsubdir = f.read()
+ for line in f:
+ line = line.rstrip()
+ if line:
+ srcdirs.append(line.split('=')[0])
+ srcsubdir = srcdirs[0]
+
with open(os.path.join(tempdir, 'initial_rev'), 'r') as f:
initial_rev = f.read()
- def rm_patches():
- patches_dir = os.path.join(srcsubdir, 'patches')
+ def rm_patches(pth):
+ patches_dir = os.path.join(pth, 'patches')
if os.path.exists(patches_dir):
shutil.rmtree(patches_dir)
# Restore any "patches" directory that was actually part of the source tree
try:
- bb.process.run('git checkout -- patches', cwd=srcsubdir)
+ bb.process.run('git checkout -- patches', cwd=pth)
except bb.process.ExecutionError:
pass
@@ -194,7 +228,7 @@ python devtool_post_patch() {
localdata = bb.data.createCopy(d)
localdata.setVar('OVERRIDES', ':'.join(no_overrides))
bb.build.exec_func('do_patch', localdata)
- rm_patches()
+ rm_patches(srcsubdir)
# Now we need to reconcile the dev branch with the no-overrides one
# (otherwise we'd likely be left with identical commits that have different hashes)
bb.process.run('git checkout %s' % devbranch, cwd=srcsubdir)
@@ -212,12 +246,15 @@ python devtool_post_patch() {
# Run do_patch function with the override applied
localdata.appendVar('OVERRIDES', ':%s' % override)
bb.build.exec_func('do_patch', localdata)
- rm_patches()
+ rm_patches(srcsubdir)
# Now we need to reconcile the new branch with the no-overrides one
# (otherwise we'd likely be left with identical commits that have different hashes)
bb.process.run('git rebase devtool-no-overrides', cwd=srcsubdir)
bb.process.run('git checkout %s' % devbranch, cwd=srcsubdir)
- bb.process.run('git tag -f devtool-patched', cwd=srcsubdir)
+ for srcdir in srcdirs:
+ bb.process.run('git tag -f devtool-patched', cwd=srcdir)
+ if srcdir != srcsubdir:
+ rm_patches(srcdir)
}
python devtool_post_configure() {
diff --git a/meta/lib/oe/recipeutils.py b/meta/lib/oe/recipeutils.py
index 4928727..f4a40c2 100644
--- a/meta/lib/oe/recipeutils.py
+++ b/meta/lib/oe/recipeutils.py
@@ -445,10 +445,10 @@ def get_recipe_patches(d):
"""Get a list of the patches included in SRC_URI within a recipe."""
import oe.patch
patches = oe.patch.src_patches(d, expand=False)
- patchfiles = []
+ patchfiles = OrderedDict()
for patch in patches:
_, _, local, _, _, parm = bb.fetch.decodeurl(patch)
- patchfiles.append(local)
+ patchfiles[local] = parm
return patchfiles
diff --git a/meta/lib/oe/utils.py b/meta/lib/oe/utils.py
index 1897c5f..9670dd7 100644
--- a/meta/lib/oe/utils.py
+++ b/meta/lib/oe/utils.py
@@ -368,3 +368,27 @@ class ImageQAFailed(bb.build.FuncFailed):
msg = msg + ' (%s)' % self.description
return msg
+
+def is_path_under(path1, possible_parent):
+ """
+ Return True if a path is underneath another, False otherwise.
+ Multiple paths to test can be specified (as a list or tuple) in
+ which case all specified test paths must be under the parent to
+ return True.
+ """
+ def abs_path_trailing(pth):
+ pth_abs = os.path.abspath(pth)
+ if not pth_abs.endswith(os.sep):
+ pth_abs += os.sep
+ return pth_abs
+
+ possible_parent_abs = abs_path_trailing(possible_parent)
+ if isinstance(path1, str):
+ testpaths = [path1]
+ else:
+ testpaths = path1
+ for path in testpaths:
+ path_abs = abs_path_trailing(path)
+ if not path_abs.startswith(possible_parent_abs):
+ return False
+ return True
diff --git a/meta/lib/oeqa/selftest/cases/devtool.py b/meta/lib/oeqa/selftest/cases/devtool.py
index 43280cd..b3063de 100644
--- a/meta/lib/oeqa/selftest/cases/devtool.py
+++ b/meta/lib/oeqa/selftest/cases/devtool.py
@@ -7,7 +7,7 @@ import fnmatch
import oeqa.utils.ftools as ftools
from oeqa.selftest.case import OESelftestTestCase
-from oeqa.utils.commands import runCmd, bitbake, get_bb_var, create_temp_layer
+from oeqa.utils.commands import runCmd, bitbake, get_bb_var, create_temp_layer, CommandError
from oeqa.utils.commands import get_bb_vars, runqemu, get_test_layer
from oeqa.core.decorator.oeid import OETestID
diff --git a/scripts/devtool b/scripts/devtool
index a651d8f..bccbb92 100755
--- a/scripts/devtool
+++ b/scripts/devtool
@@ -25,6 +25,8 @@ import re
import configparser
import subprocess
import logging
+import json
+from collections import OrderedDict
basepath = ''
workspace = {}
@@ -109,31 +111,58 @@ def read_workspace():
if not context.fixed_setup:
_enable_workspace_layer(config.workspace_path, config, basepath)
+ def process_inline_json(strvalue, in_value):
+ if strvalue.count('{') == strvalue.count('}') and strvalue.count('[') == strvalue.count(']'):
+ pnvalues[in_value] = json.loads(strvalue, object_pairs_hook=OrderedDict)
+ return True
+ return False
+
logger.debug('Reading workspace in %s' % config.workspace_path)
externalsrc_re = re.compile(r'^EXTERNALSRC(_pn-([^ =]+))? *= *"([^"]*)"$')
for fn in glob.glob(os.path.join(config.workspace_path, 'appends', '*.bbappend')):
with open(fn, 'r') as f:
pnvalues = {}
+ in_value = None
+ strvalue = ''
for line in f:
- res = externalsrc_re.match(line.rstrip())
- if res:
- pn = res.group(2) or os.path.splitext(os.path.basename(fn))[0].split('_')[0]
- # Find the recipe file within the workspace, if any
- bbfile = os.path.basename(fn).replace('.bbappend', '.bb').replace('%', '*')
- recipefile = glob.glob(os.path.join(config.workspace_path,
- 'recipes',
- pn,
- bbfile))
- if recipefile:
- recipefile = recipefile[0]
- pnvalues['srctree'] = res.group(3)
- pnvalues['bbappend'] = fn
- pnvalues['recipefile'] = recipefile
- elif line.startswith('# srctreebase: '):
- pnvalues['srctreebase'] = line.split(':', 1)[1].strip()
+ if in_value:
+ if not line.startswith('#'):
+ logger.error('Syntax error in %s - invalid in-line json' % fn)
+ sys.exit(1)
+ strvalue += line[1:]
+ if process_inline_json(strvalue, in_value):
+ in_value = None
+ else:
+ res = externalsrc_re.match(line.rstrip())
+ if res:
+ pn = res.group(2) or os.path.splitext(os.path.basename(fn))[0].split('_')[0]
+ # Find the recipe file within the workspace, if any
+ bbfile = os.path.basename(fn).replace('.bbappend', '.bb').replace('%', '*')
+ recipefile = glob.glob(os.path.join(config.workspace_path,
+ 'recipes',
+ pn,
+ bbfile))
+ if recipefile:
+ recipefile = recipefile[0]
+ pnvalues['srctree'] = res.group(3)
+ pnvalues['bbappend'] = fn
+ pnvalues['recipefile'] = recipefile
+ elif line.startswith('# srctreebase: '):
+ pnvalues['srctreebase'] = line.split(':', 1)[1].strip()
+ elif line.startswith('# srctreetop: '):
+ pnvalues['srctreetop'] = line.split(':', 1)[1].strip()
+ elif line.startswith('# extradirs: '):
+ strvalue = line.split(':', 1)[1].strip()
+ in_value = 'extradirs'
+ if process_inline_json(strvalue, in_value):
+ in_value = None
if pnvalues:
if not pnvalues.get('srctreebase', None):
pnvalues['srctreebase'] = pnvalues['srctree']
+ if not 'srctreetop' in pnvalues:
+ pnvalues['srctreetop'] = pnvalues['srctreebase']
+ if not 'extradirs' in pnvalues:
+ pnvalues['extradirs'] = []
logger.debug('Found recipe %s' % pnvalues)
workspace[pn] = pnvalues
diff --git a/scripts/lib/devtool/__init__.py b/scripts/lib/devtool/__init__.py
index 07d774d..5cdf07a 100644
--- a/scripts/lib/devtool/__init__.py
+++ b/scripts/lib/devtool/__init__.py
@@ -223,6 +223,20 @@ def setup_git_repo(repodir, version, devbranch, basetag='devtool-base', d=None):
bb.process.run('git checkout -b %s' % devbranch, cwd=repodir)
bb.process.run('git tag -f %s' % basetag, cwd=repodir)
+def find_git_repos(pth, toplevel=False):
+ """
+ Find git repositories under a path
+ """
+ repos = []
+ if toplevel and os.path.isdir(os.path.join(pth, '.git')):
+ repos.append(pth)
+ for root, dirs, _ in os.walk(pth):
+ for dfn in dirs:
+ dfp = os.path.join(root, dfn)
+ if os.path.isdir(os.path.join(dfp, '.git')) and dfp not in repos:
+ repos.append(dfp)
+ return repos
+
def recipe_to_append(recipefile, config, wildcard=False):
"""
Convert a recipe file to a bbappend file path within the workspace.
diff --git a/scripts/lib/devtool/standard.py b/scripts/lib/devtool/standard.py
index ae48406..cadd038 100644
--- a/scripts/lib/devtool/standard.py
+++ b/scripts/lib/devtool/standard.py
@@ -29,8 +29,9 @@ import scriptutils
import errno
import glob
import filecmp
+import json
from collections import OrderedDict, namedtuple
-from devtool import exec_build_env_command, setup_tinfoil, check_workspace_recipe, use_external_build, setup_git_repo, recipe_to_append, get_bbclassextend_targets, update_unlockedsigs, check_prerelease_version, check_git_repo_dirty, check_git_repo_op, DevtoolError
+from devtool import exec_build_env_command, setup_tinfoil, check_workspace_recipe, use_external_build, setup_git_repo, recipe_to_append, get_bbclassextend_targets, update_unlockedsigs, check_prerelease_version, check_git_repo_dirty, check_git_repo_op, find_git_repos, DevtoolError
from devtool import parse_recipe
logger = logging.getLogger('devtool')
@@ -464,7 +465,7 @@ def sync(args, config, basepath, workspace):
return 0
-ExtractSourceResult = namedtuple('ExtractSourceResult', ['initial_rev', 'srcsubdir'])
+ExtractSourceResult = namedtuple('ExtractSourceResult', ['initial_rev', 'srcsubdir', 'extradirs'])
def _extract_source(srctree, keep_temp, devbranch, sync, config, basepath, workspace, fixed_setup, d, tinfoil, no_overrides=False):
"""Extract sources of a recipe"""
@@ -576,15 +577,27 @@ def _extract_source(srctree, keep_temp, devbranch, sync, config, basepath, works
if not res:
raise DevtoolError('Extracting source for %s failed' % pn)
+ tempworkdir = os.path.join(tempdir, 'workdir')
+ extra_revs = OrderedDict()
+ extradirs = []
try:
with open(os.path.join(tempdir, 'initial_rev'), 'r') as f:
initial_rev = f.read()
with open(os.path.join(tempdir, 'srcsubdir'), 'r') as f:
- srcsubdir = f.read()
+ srcsubdir = f.readline().rstrip()
+ for line in f:
+ line = line.rstrip()
+ if '=' in line:
+ linesplit = line.rstrip().split('=', 1)
+ extradir = linesplit[0]
+ extradirs.append(extradir)
+ extra_revs[os.path.relpath(extradir, tempworkdir)] = linesplit[1]
+
except FileNotFoundError as e:
raise DevtoolError('Something went wrong with source extraction - the devtool-source class was not active or did not function correctly:\n%s' % str(e))
- srcsubdir_rel = os.path.relpath(srcsubdir, os.path.join(tempdir, 'workdir'))
+
+ srcsubdir_rel = os.path.relpath(srcsubdir, tempworkdir)
tempdir_localdir = os.path.join(tempdir, 'oe-local-files')
srctree_localdir = os.path.join(srctree, 'oe-local-files')
@@ -613,6 +626,19 @@ def _extract_source(srctree, keep_temp, devbranch, sync, config, basepath, works
logger.info('Adding local source files to srctree...')
shutil.move(tempdir_localdir, srcsubdir)
+ if extra_revs:
+ logger.warn('This recipe fetches more than one source tree. Each source tree will be tracked in a separate git repository.')
+ if not oe.utils.is_path_under(extradirs, srcsubdir):
+ # There's more than one source directory at the top level, we're going to need to create the parent
+ os.mkdir(srctree)
+ for extradir in extradirs:
+ shutil.move(extradir, srctree)
+ # Write a file to record which one is the main src in case we need to re-modify the same tree later
+ with open(os.path.join(srctree, '.devtool'), 'w') as f:
+ data = OrderedDict()
+ data['srcsubdir'] = os.path.basename(srcsubdir)
+ data['extradirs'] = extra_revs
+ json.dump(data, f, indent=' ')
shutil.move(srcsubdir, srctree)
if os.path.abspath(d.getVar('S')) == os.path.abspath(d.getVar('WORKDIR')):
@@ -640,7 +666,7 @@ def _extract_source(srctree, keep_temp, devbranch, sync, config, basepath, works
if is_kernel_yocto:
logger.info('Copying kernel config to srctree')
- shutil.copy2(os.path.join(tempdir, '.config'), srctree)
+ shutil.copy2(os.path.join(tempdir, '.config'), os.path.join(srctree, srcsubdir_rel))
finally:
if appendbackup:
@@ -651,7 +677,7 @@ def _extract_source(srctree, keep_temp, devbranch, sync, config, basepath, works
logger.info('Preserving temporary directory %s' % tempdir)
else:
shutil.rmtree(tempdir)
- return ExtractSourceResult(initial_rev, srcsubdir_rel)
+ return ExtractSourceResult(initial_rev, srcsubdir_rel, extra_revs)
def _add_md5(config, recipename, filename):
"""Record checksum of a file (or recursively for a directory) to the md5-file of the workspace"""
@@ -704,6 +730,27 @@ def _check_preserve(config, recipename):
tf.write(line)
os.rename(newfile, origfile)
+def _get_initial_rev(srctree):
+ """
+ Given a source tree, get the initial revision and determine if it was prepared by devtool
+ """
+ initial_rev = None
+ try:
+ (stdout, _) = bb.process.run('git branch --contains devtool-base', cwd=srctree)
+ except bb.process.ExecutionError:
+ stdout = ''
+ if stdout:
+ was_devtool = True
+ for line in stdout.splitlines():
+ if line.startswith('*'):
+ (stdout, _) = bb.process.run('git rev-parse devtool-base', cwd=srctree)
+ initial_rev = stdout.rstrip()
+ if not initial_rev:
+ # Otherwise, just grab the head revision
+ (stdout, _) = bb.process.run('git rev-parse HEAD', cwd=srctree)
+ initial_rev = stdout.rstrip()
+ return initial_rev, was_devtool
+
def modify(args, config, basepath, workspace):
"""Entry point for the devtool 'modify' subcommand"""
import bb
@@ -751,29 +798,40 @@ def modify(args, config, basepath, workspace):
commits = []
check_commits = False
torev = 'HEAD'
+ extradirs = OrderedDict()
+ srctreetop = None
+ extra_revs = {}
if not args.no_extract:
ret = _extract_source(srctree, args.keep_temp, args.branch, False, config, basepath, workspace, args.fixed_setup, rd, tinfoil, no_overrides=args.no_overrides)
initial_rev = ret.initial_rev
+ extradirs = ret.extradirs
+ if extradirs and not oe.utils.is_path_under(extradirs.keys(), ret.srcsubdir):
+ # Multiple source trees, we need to be inside the main one
+ srctreetop = srctree
+ srctree = os.path.join(srctree, ret.srcsubdir)
logger.info('Source tree extracted to %s' % srctree)
check_commits = True
else:
+ devtoolfile = os.path.join(srctree, '.devtool')
+ if os.path.exists(devtoolfile):
+ with open(devtoolfile, 'r') as f:
+ cfg = json.load(f, object_pairs_hook=OrderedDict)
+ srcsubdir = cfg.get('srcsubdir', '')
+ if srcsubdir:
+ srctreetop = srctree
+ srctree = os.path.abspath(os.path.join(srctree, srcsubdir))
+ extradirs.update(cfg.get('extradirs', {}))
+ else:
+ # Find any extra source trees
+ extradirpaths = find_git_repos(srctree)
+ for extradirpath in extradirpaths:
+ extradirs[os.path.relpath(extradirpath, srctree)], _ = _get_initial_rev(extradirpath)
+
if os.path.exists(os.path.join(srctree, '.git')):
# Check if it's a tree previously extracted by us
- try:
- (stdout, _) = bb.process.run('git branch --contains devtool-base', cwd=srctree)
- except bb.process.ExecutionError:
- stdout = ''
- if stdout:
- check_commits = True
+ initial_rev, check_commits = _get_initial_rev(srctree)
+ if check_commits:
torev = 'devtool-patched'
- for line in stdout.splitlines():
- if line.startswith('*'):
- (stdout, _) = bb.process.run('git rev-parse devtool-base', cwd=srctree)
- initial_rev = stdout.rstrip()
- if not initial_rev:
- # Otherwise, just grab the head revision
- (stdout, _) = bb.process.run('git rev-parse HEAD', cwd=srctree)
- initial_rev = stdout.rstrip()
if initial_rev:
# Get list of commits since this revision
@@ -807,6 +865,23 @@ def modify(args, config, basepath, workspace):
# Need to grab this here in case the source is within a subdirectory
srctreebase = srctree
+ if extradirs:
+ extradirentries = []
+ for extradir, extrarev in extradirs.items():
+ extradirentry = OrderedDict()
+ if srctreetop:
+ # Several source trees at the top level
+ extradirentry['path'] = os.path.join(srctreetop, extradir)
+ else:
+ # All of the extra source trees are under the main one, so chop off
+ # the top level subdirectory
+ extradirentry['path'] = os.path.join(srctree, os.sep.join(extradir.split(os.sep)[1:]))
+ extradirentry['initial_rev'] = extrarev
+ # Get list of commits since this revision
+ (stdout, _) = bb.process.run('git rev-list --reverse %s..HEAD' % extrarev, cwd=extradirentry['path'])
+ extradirentry['commits'] = stdout.split()
+ extradirentries.append(extradirentry)
+
# Check that recipe isn't using a shared workdir
s = os.path.abspath(rd.getVar('S'))
workdir = os.path.abspath(rd.getVar('WORKDIR'))
@@ -842,6 +917,23 @@ def modify(args, config, basepath, workspace):
' cp ${B}/.config ${S}/.config.baseline\n'
' ln -sfT ${B}/.config ${S}/.config.new\n'
'}\n')
+
+ if extradirs:
+ f.write('\n# srctreetop: %s\n' % srctreetop)
+ extradir_json = json.dumps(extradirentries, indent=' ')
+ prefix = '# extradirs:'
+ for line in extradir_json.splitlines():
+ f.write('%s %s\n' % (prefix, line))
+ prefix = '#'
+ f.write('\ndo_unpack[postfuncs] += "devtool_symlink_srctrees"\n')
+ f.write('devtool_symlink_srctrees () {\n')
+ # We could have done a loop in the function itself, but given there's
+ # usually only going to be a small number, there's not much point
+ for extradiritem in extradirentries:
+ if not oe.utils.is_path_under(extradiritem['path'], srctree):
+ f.write(' ln -sf %s ${WORKDIR}/\n' % extradiritem['path'])
+ f.write('}\n')
+
if initial_rev:
f.write('\n# initial_rev: %s\n' % initial_rev)
for commit in commits:
@@ -1056,7 +1148,7 @@ def rename(args, config, basepath, workspace):
return 0
-def _get_patchset_revs(srctree, recipe_path, initial_rev=None):
+def _get_patchset_revs(recipename, workspace, srctree, initial_rev=None):
"""Get initial and update rev of a recipe. These are the start point of the
whole patchset and start point for the patches to be re-generated/updated.
"""
@@ -1067,10 +1159,15 @@ def _get_patchset_revs(srctree, recipe_path, initial_rev=None):
cwd=srctree)
branchname = stdout.rstrip()
+ append = workspace[recipename]['bbappend']
+ if not os.path.exists(append):
+ raise DevtoolError('unable to find workspace bbappend for recipe %s' %
+ recipename)
+
# Parse initial rev from recipe if not specified
commits = []
patches = []
- with open(recipe_path, 'r') as f:
+ with open(append, 'r') as f:
for line in f:
if line.startswith('# initial_rev:'):
if not initial_rev:
@@ -1080,6 +1177,15 @@ def _get_patchset_revs(srctree, recipe_path, initial_rev=None):
elif line.startswith('# patches_%s:' % branchname):
patches = line.split(':')[-1].strip().split(',')
+ if srctree != workspace[recipename]['srctree']:
+ # Extra dir
+ for extradirentry in workspace[recipename]['extradirs']:
+ if srctree == extradirentry['path']:
+ commits = extradirentry['commits']
+ break
+ else:
+ raise Exception('Failed to find extradir entry for %s' % srctree)
+
update_rev = initial_rev
changed_revs = None
if initial_rev:
@@ -1160,7 +1266,7 @@ def _remove_source_files(append, files, destpath, no_report_remove=False, dry_ru
raise
-def _export_patches(srctree, rd, start_rev, destdir, changed_revs=None):
+def _export_patches(srctree, rd, start_rev, destdir, changed_revs=None, origsrcdir=None):
"""Export patches from srctree to given location.
Returns three-tuple of dicts:
1. updated - patches that already exist in SRCURI
@@ -1175,9 +1281,21 @@ def _export_patches(srctree, rd, start_rev, destdir, changed_revs=None):
added = OrderedDict()
seqpatch_re = re.compile('^([0-9]{4}-)?(.+)')
+ patches = oe.recipeutils.get_recipe_patches(rd)
+ if origsrcdir:
+ if not origsrcdir.endswith(os.sep):
+ origsrcdir = origsrcdir + os.sep
+ s = rd.getVar('S')
+ for patch, parms in list(patches.items()):
+ patchdir = os.path.abspath(os.path.join(s, parms.get('patchdir', '.')))
+ if not patchdir.endswith(os.sep):
+ patchdir = patchdir + os.sep
+ if not patchdir.startswith(origsrcdir):
+ del patches[patch]
+
existing_patches = dict((os.path.basename(path), path) for path in
- oe.recipeutils.get_recipe_patches(rd))
- logger.debug('Existing patches: %s' % existing_patches)
+ patches)
+ logger.debug('Existing patches (for %s): %s' % (srctree, existing_patches))
# Generate patches from Git, exclude local files directory
patch_pathspec = _git_exclude_path(srctree, 'oe-local-files')
@@ -1370,6 +1488,11 @@ def _update_recipe_srcrev(recipename, workspace, srctree, rd, appendlayerdir, wi
recipedir = os.path.basename(recipefile)
logger.info('Updating SRCREV in recipe %s%s' % (recipedir, dry_run_suffix))
+ # Work out what the extracted path would have been (so we can use it
+ # to filter patches when there are extra source directories)
+ s = rd.getVar('S')
+ origsrcdir = os.path.abspath(os.path.join(s, os.path.relpath(srctree, workspace[recipename]['srctree'])))
+
# Get HEAD revision
try:
stdout, _ = bb.process.run('git rev-parse HEAD', cwd=srctree)
@@ -1398,7 +1521,8 @@ def _update_recipe_srcrev(recipename, workspace, srctree, rd, appendlayerdir, wi
patches_dir = tempfile.mkdtemp(dir=tempdir)
old_srcrev = rd.getVar('SRCREV') or ''
upd_p, new_p, del_p = _export_patches(srctree, rd, old_srcrev,
- patches_dir)
+ patches_dir,
+ origsrcdir=origsrcdir)
logger.debug('Patches: update %s, new %s, delete %s' % (dict(upd_p), dict(new_p), dict(del_p)))
# Remove deleted local files and "overlapping" patches
@@ -1444,6 +1568,10 @@ def _update_recipe_srcrev(recipename, workspace, srctree, rd, appendlayerdir, wi
if update_srcuri:
patchfields['SRC_URI'] = ' '.join(srcuri)
ret = oe.recipeutils.patch_recipe(rd, recipefile, patchfields, redirect_output=dry_run_outdir)
+ # Set the values into the datastore for the benefit of the next
+ # call (if extradirs are set)
+ for var, value in patchfields.items():
+ rd.setVar(var, value)
finally:
shutil.rmtree(tempdir)
if not 'git://' in orig_src_uri:
@@ -1461,12 +1589,8 @@ def _update_recipe_patch(recipename, workspace, srctree, rd, appendlayerdir, wil
recipefile = rd.getVar('FILE')
recipedir = os.path.dirname(recipefile)
- append = workspace[recipename]['bbappend']
- if not os.path.exists(append):
- raise DevtoolError('unable to find workspace bbappend for recipe %s' %
- recipename)
- initial_rev, update_rev, changed_revs, filter_patches = _get_patchset_revs(srctree, append, initial_rev)
+ initial_rev, update_rev, changed_revs, filter_patches = _get_patchset_revs(recipename, workspace, srctree, initial_rev)
if not initial_rev:
raise DevtoolError('Unable to find initial revision - please specify '
'it with --initial-rev')
@@ -1476,6 +1600,17 @@ def _update_recipe_patch(recipename, workspace, srctree, rd, appendlayerdir, wil
if not dl_dir.endswith('/'):
dl_dir += '/'
+ # Work out what the extracted path would have been (so we can use it
+ # to filter patches when there are extra source directories)
+ s = rd.getVar('S')
+ origsrcdir = os.path.abspath(os.path.join(s, os.path.relpath(srctree, workspace[recipename]['srctree'])))
+ # Determine patchdir for any new patches
+ patchdir = os.path.relpath(origsrcdir, s)
+ if patchdir != '.':
+ patchsuffix = ';patchdir=%s' % patchdir
+ else:
+ patchsuffix = ''
+
dry_run_suffix = ' (dry-run)' if dry_run_outdir else ''
tempdir = tempfile.mkdtemp(prefix='devtool')
@@ -1494,14 +1629,16 @@ def _update_recipe_patch(recipename, workspace, srctree, rd, appendlayerdir, wil
# Get all patches from source tree and check if any should be removed
all_patches_dir = tempfile.mkdtemp(dir=tempdir)
_, _, del_p = _export_patches(srctree, rd, initial_rev,
- all_patches_dir)
+ all_patches_dir,
+ origsrcdir=origsrcdir)
# Remove deleted local files and patches
remove_files = list(del_f.values()) + list(del_p.values())
# Get updated patches from source tree
patches_dir = tempfile.mkdtemp(dir=tempdir)
upd_p, new_p, _ = _export_patches(srctree, rd, update_rev,
- patches_dir, changed_revs)
+ patches_dir, changed_revs,
+ origsrcdir=origsrcdir)
logger.debug('Pre-filtering: update: %s, new: %s' % (dict(upd_p), dict(new_p)))
if filter_patches:
new_p = {}
@@ -1531,7 +1668,7 @@ def _update_recipe_patch(recipename, workspace, srctree, rd, appendlayerdir, wil
removevalues=removevalues,
redirect_output=dry_run_outdir)
else:
- logger.info('No patches or local source files needed updating')
+ return False, None, []
else:
# Update existing files
files_dir = _determine_files_dir(rd)
@@ -1553,7 +1690,7 @@ def _update_recipe_patch(recipename, workspace, srctree, rd, appendlayerdir, wil
# replace the entry in SRC_URI with our local version
logger.info('Replacing remote patch %s with updated local version' % basepath)
path = os.path.join(files_dir, basepath)
- _replace_srcuri_entry(srcuri, basepath, 'file://%s' % basepath)
+ _replace_srcuri_entry(srcuri, basepath, 'file://%s%s' % (basepath, patchsuffix))
updaterecipe = True
else:
logger.info('Updating patch %s%s' % (basepath, dry_run_suffix))
@@ -1575,7 +1712,7 @@ def _update_recipe_patch(recipename, workspace, srctree, rd, appendlayerdir, wil
os.path.join(files_dir, basepath),
dry_run_outdir=dry_run_outdir,
base_outdir=recipedir)
- srcuri.append('file://%s' % basepath)
+ srcuri.append('file://%s%s' % (basepath, patchsuffix))
updaterecipe = True
# Update recipe, if needed
if _remove_file_entries(srcuri, remove_files)[0]:
@@ -1586,9 +1723,11 @@ def _update_recipe_patch(recipename, workspace, srctree, rd, appendlayerdir, wil
ret = oe.recipeutils.patch_recipe(rd, recipefile,
{'SRC_URI': ' '.join(srcuri)},
redirect_output=dry_run_outdir)
+ # Set the value into the datastore for the benefit of the next
+ # call (if extradirs are set)
+ rd.setVar('SRC_URI', ' '.join(srcuri))
elif not updatefiles:
# Neither patches nor recipe were updated
- logger.info('No patches or files need updating')
return False, None, []
finally:
shutil.rmtree(tempdir)
@@ -1619,70 +1758,82 @@ def _guess_recipe_update_mode(srctree, rdata):
return 'patch'
def _update_recipe(recipename, workspace, rd, mode, appendlayerdir, wildcard_version, no_remove, initial_rev, no_report_remove=False, dry_run_outdir=None, no_overrides=False):
- srctree = workspace[recipename]['srctree']
- if mode == 'auto':
- mode = _guess_recipe_update_mode(srctree, rd)
+ main_srctree = workspace[recipename]['srctree']
- override_branches = []
- mainbranch = None
- startbranch = None
- if not no_overrides:
- stdout, _ = bb.process.run('git branch', cwd=srctree)
- other_branches = []
- for line in stdout.splitlines():
- branchname = line[2:]
- if line.startswith('* '):
- startbranch = branchname
- if branchname.startswith(override_branch_prefix):
- override_branches.append(branchname)
- else:
- other_branches.append(branchname)
+ appendfile = workspace[recipename]['bbappend']
+ srctrees = OrderedDict([(main_srctree, initial_rev)])
+ for extradirentry in workspace[recipename]['extradirs']:
+ srctrees[extradirentry['path']] = extradirentry['initial_rev']
+
+ logger.debug('Considering source trees: %s' % srctrees)
+ if mode == 'auto':
+ mode = _guess_recipe_update_mode(main_srctree, rd)
+
+ crd = bb.data.createCopy(rd)
+ for srctree, src_initial_rev in srctrees.items():
+ override_branches = []
+ mainbranch = None
+ startbranch = None
+ if not no_overrides:
+ stdout, _ = bb.process.run('git branch', cwd=srctree)
+ other_branches = []
+ for line in stdout.splitlines():
+ branchname = line[2:]
+ if line.startswith('* '):
+ startbranch = branchname
+ if branchname.startswith(override_branch_prefix):
+ override_branches.append(branchname)
+ else:
+ other_branches.append(branchname)
+
+ if override_branches:
+ logger.debug('_update_recipe: override branches: %s' % override_branches)
+ logger.debug('_update_recipe: other branches: %s' % other_branches)
+ if startbranch.startswith(override_branch_prefix):
+ if len(other_branches) == 1:
+ mainbranch = other_branches[1]
+ else:
+ raise DevtoolError('Unable to determine main branch - please check out the main branch in source tree first')
+ else:
+ mainbranch = startbranch
+
+ checkedout = None
+ anyupdated = False
+ appendfile = None
+ allremoved = []
if override_branches:
- logger.debug('_update_recipe: override branches: %s' % override_branches)
- logger.debug('_update_recipe: other branches: %s' % other_branches)
- if startbranch.startswith(override_branch_prefix):
- if len(other_branches) == 1:
- mainbranch = other_branches[1]
+ logger.info('Handling main branch (%s)...' % mainbranch)
+ if startbranch != mainbranch:
+ bb.process.run('git checkout %s' % mainbranch, cwd=srctree)
+ checkedout = mainbranch
+ try:
+ branchlist = [mainbranch] + override_branches
+ for branch in branchlist:
+ if branch != mainbranch:
+ logger.info('Handling branch %s...' % branch)
+ override = branch[len(override_branch_prefix):]
+ crd.setVar('OVERRIDES', '%s:%s' % rd.getVar('OVERRIDES') + override)
+ bb.process.run('git checkout %s' % branch, cwd=srctree)
+ checkedout = branch
+
+ if mode == 'srcrev':
+ updated, appendf, removed = _update_recipe_srcrev(recipename, workspace, srctree, crd, appendlayerdir, wildcard_version, no_remove, no_report_remove, dry_run_outdir)
+ elif mode == 'patch':
+ updated, appendf, removed = _update_recipe_patch(recipename, workspace, srctree, crd, appendlayerdir, wildcard_version, no_remove, no_report_remove, src_initial_rev, dry_run_outdir)
else:
- raise DevtoolError('Unable to determine main branch - please check out the main branch in source tree first')
- else:
- mainbranch = startbranch
+ raise DevtoolError('update_recipe: invalid mode %s' % mode)
+ if updated:
+ anyupdated = True
+ if appendf:
+ appendfile = appendf
+ allremoved.extend(removed)
+ finally:
+ if startbranch and checkedout != startbranch:
+ bb.process.run('git checkout %s' % startbranch, cwd=srctree)
- checkedout = None
- anyupdated = False
- appendfile = None
- allremoved = []
- if override_branches:
- logger.info('Handling main branch (%s)...' % mainbranch)
- if startbranch != mainbranch:
- bb.process.run('git checkout %s' % mainbranch, cwd=srctree)
- checkedout = mainbranch
- try:
- branchlist = [mainbranch] + override_branches
- for branch in branchlist:
- crd = bb.data.createCopy(rd)
- if branch != mainbranch:
- logger.info('Handling branch %s...' % branch)
- override = branch[len(override_branch_prefix):]
- crd.appendVar('OVERRIDES', ':%s' % override)
- bb.process.run('git checkout %s' % branch, cwd=srctree)
- checkedout = branch
-
- if mode == 'srcrev':
- updated, appendf, removed = _update_recipe_srcrev(recipename, workspace, srctree, crd, appendlayerdir, wildcard_version, no_remove, no_report_remove, dry_run_outdir)
- elif mode == 'patch':
- updated, appendf, removed = _update_recipe_patch(recipename, workspace, srctree, crd, appendlayerdir, wildcard_version, no_remove, no_report_remove, initial_rev, dry_run_outdir)
- else:
- raise DevtoolError('update_recipe: invalid mode %s' % mode)
- if updated:
- anyupdated = True
- if appendf:
- appendfile = appendf
- allremoved.extend(removed)
- finally:
- if startbranch and checkedout != startbranch:
- bb.process.run('git checkout %s' % startbranch, cwd=srctree)
+ if not anyupdated:
+ logger.info('No patches or files need updating')
return anyupdated, appendfile, allremoved
@@ -1808,16 +1959,16 @@ def _reset(recipes, no_clean, config, basepath, workspace):
# We don't automatically create this dir next to appends, but the user can
preservedir(os.path.join(config.workspace_path, 'appends', pn))
- srctreebase = workspace[pn]['srctreebase']
- if os.path.isdir(srctreebase):
- if os.listdir(srctreebase):
+ srctreetop = workspace[pn]['srctreetop']
+ if os.path.isdir(srctreetop):
+ if os.listdir(srctreetop):
# We don't want to risk wiping out any work in progress
logger.info('Leaving source tree %s as-is; if you no '
'longer need it then please delete it manually'
- % srctreebase)
+ % srctreetop)
else:
# This is unlikely, but if it's empty we can just remove it
- os.rmdir(srctreebase)
+ os.rmdir(srctreetop)
clean_preferred_provider(pn, config.workspace_path)
@@ -1868,12 +2019,17 @@ def finish(args, config, basepath, workspace):
srctree = workspace[args.recipename]['srctree']
check_git_repo_op(srctree, [corebasedir])
- dirty = check_git_repo_dirty(srctree)
- if dirty:
- if args.force:
- logger.warning('Source tree is not clean, continuing as requested by -f/--force')
- else:
- raise DevtoolError('Source tree is not clean:\n\n%s\nEnsure you have committed your changes or use -f/--force if you are sure there\'s nothing that needs to be committed' % dirty)
+ srctrees = [srctree]
+ for extradirentry in workspace[args.recipename]['extradirs']:
+ srctrees.append(extradirentry['path'])
+ for pth in srctrees:
+ dirty = check_git_repo_dirty(pth)
+ if dirty:
+ if args.force:
+ logger.warning('Source tree %s is not clean, continuing as requested by -f/--force' % pth)
+ else:
+ raise DevtoolError('Source tree %s is not clean:\n\n%s\nEnsure you have committed your changes or use -f/--force if you are sure there\'s nothing that needs to be committed' % (pth, dirty))
+
no_clean = False
tinfoil = setup_tinfoil(basepath=basepath, tracking=True)
--
2.9.5
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH 09/12] devtool: fix handling of linux-yocto after multisrc changes
2017-12-05 1:41 [PATCH 00/12] devtool improvements Paul Eggleton
` (7 preceding siblings ...)
2017-12-05 1:41 ` [PATCH 08/12] devtool: support extracting multiple source trees Paul Eggleton
@ 2017-12-05 1:41 ` Paul Eggleton
2017-12-05 1:41 ` [PATCH 10/12] oe-selftest: devtool: fix kernel test for " Paul Eggleton
` (4 subsequent siblings)
13 siblings, 0 replies; 16+ messages in thread
From: Paul Eggleton @ 2017-12-05 1:41 UTC (permalink / raw)
To: openembedded-core
devtool now handles multiple source trees for any recipe that includes
them in SRC_URI, extracting them all side-by-side so that you can make
changes in any of them. As a result, when running devtool modify on a
linux-yocto kernel recipe under the source path you will get a "source"
subdirectory containing the kernel source and a "kernel-meta"
subdirectory next to it containing the kernel metadata. (Previously you
just got the source tree and the kernel metadata remained in the work
directory). We create a symlink automatically at do_unpack from the work
directory so that it can still be found there, however
kernel_feature_dirs() expects to find the kernel-meta repository and
we also now need to make externalsrc remove that so that it doesn't
unpack and overwrite the one we've already extracted. Change
kernel_feature_dirs() so that if there are no kmeta entries in SRC_URI,
it will fall back to a directory named ${KMETA} if it happens to be
present in the work directory, ignoring how it got there.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
meta/classes/externalsrc.bbclass | 2 +-
meta/classes/kernel-yocto.bbclass | 7 ++++++-
scripts/lib/devtool/standard.py | 4 +++-
3 files changed, 10 insertions(+), 3 deletions(-)
diff --git a/meta/classes/externalsrc.bbclass b/meta/classes/externalsrc.bbclass
index 65dd13d..1ce53ee 100644
--- a/meta/classes/externalsrc.bbclass
+++ b/meta/classes/externalsrc.bbclass
@@ -65,7 +65,7 @@ python () {
url_data = fetch.ud[url]
parm = url_data.parm
if (url_data.type == 'file' or
- 'type' in parm and parm['type'] == 'kmeta'):
+ (parm.get('type', None) == 'kmeta' and d.getVar('EXTERNALSRC_KMETA') != "1")):
local_srcuri.append(url)
d.setVar('SRC_URI', ' '.join(local_srcuri))
diff --git a/meta/classes/kernel-yocto.bbclass b/meta/classes/kernel-yocto.bbclass
index 1d44795..18a7113 100644
--- a/meta/classes/kernel-yocto.bbclass
+++ b/meta/classes/kernel-yocto.bbclass
@@ -42,7 +42,12 @@ def find_kernel_feature_dirs(d):
destdir = parm["destsuffix"]
if type == "kmeta":
feature_dirs.append(destdir)
-
+ if not feature_dirs:
+ # If the kernel-meta directory already exists (e.g from externalsrc)
+ # with EXTERNALSRC_KMETA = "1" then use it
+ kmetadir = d.getVar('KMETA')
+ if os.path.isdir(os.path.join(d.getVar('WORKDIR'), kmetadir)):
+ feature_dirs.append(kmetadir)
return feature_dirs
# find the master/machine source branch. In the same way that the fetcher proceses
diff --git a/scripts/lib/devtool/standard.py b/scripts/lib/devtool/standard.py
index cadd038..fe33c7e 100644
--- a/scripts/lib/devtool/standard.py
+++ b/scripts/lib/devtool/standard.py
@@ -909,7 +909,9 @@ def modify(args, config, basepath, workspace):
if bb.data.inherits_class('kernel', rd):
f.write('SRCTREECOVEREDTASKS = "do_validate_branches do_kernel_checkout '
- 'do_fetch do_unpack do_kernel_configme do_kernel_configcheck"\n')
+ 'do_fetch do_kernel_configme do_kernel_configcheck"\n')
+ if bb.data.inherits_class('kernel-yocto', rd):
+ f.write('EXTERNALSRC_KMETA = "1"\n')
f.write('\ndo_patch() {\n'
' :\n'
'}\n')
--
2.9.5
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH 10/12] oe-selftest: devtool: fix kernel test for multisrc changes
2017-12-05 1:41 [PATCH 00/12] devtool improvements Paul Eggleton
` (8 preceding siblings ...)
2017-12-05 1:41 ` [PATCH 09/12] devtool: fix handling of linux-yocto after multisrc changes Paul Eggleton
@ 2017-12-05 1:41 ` Paul Eggleton
2017-12-05 1:41 ` [PATCH 11/12] oe-selftest: devtool: add test for multiple source trees Paul Eggleton
` (3 subsequent siblings)
13 siblings, 0 replies; 16+ messages in thread
From: Paul Eggleton @ 2017-12-05 1:41 UTC (permalink / raw)
To: openembedded-core
Fix the oe-selftest test to understand that the source will be
found in a "source" subdirectory if the kernel inherits linux-yocto
since there are multiple trees extracted.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
meta/lib/oeqa/selftest/cases/devtool.py | 13 +++++++++----
1 file changed, 9 insertions(+), 4 deletions(-)
diff --git a/meta/lib/oeqa/selftest/cases/devtool.py b/meta/lib/oeqa/selftest/cases/devtool.py
index b3063de..96072be 100644
--- a/meta/lib/oeqa/selftest/cases/devtool.py
+++ b/meta/lib/oeqa/selftest/cases/devtool.py
@@ -1676,9 +1676,14 @@ class DevtoolTests(DevtoolBase):
bitbake('%s -c clean' % kernel_provider)
#Step 4.1
runCmd('devtool modify virtual/kernel -x %s' % tempdir)
- self.assertExists(os.path.join(tempdir, 'Makefile'), 'Extracted source could not be found')
+ # If it's linux-yocto, we'll have two "source" trees (one the actual source,
+ # the other the kernel metadata)
+ tempsrc = os.path.join(tempdir, 'source')
+ if not os.path.isdir(tempsrc):
+ tempsrc = tempdir
+ self.assertExists(os.path.join(tempsrc, 'Makefile'), 'Extracted source could not be found')
#Step 4.2
- configfile = os.path.join(tempdir,'.config')
+ configfile = os.path.join(tempsrc, '.config')
diff = runCmd('diff %s %s' % (tmpconfig, configfile))
self.assertEqual(0,diff.status,'Kernel .config file is not the same using bitbake and devtool')
#Step 4.3
@@ -1689,12 +1694,12 @@ class DevtoolTests(DevtoolBase):
self.assertExists(kernelfile, 'Kernel was not build correctly')
#Modify the kernel source
- modfile = os.path.join(tempdir,'arch/x86/boot/header.S')
+ modfile = os.path.join(tempsrc, 'arch/x86/boot/header.S')
modstring = "Use a boot loader. Devtool testing."
modapplied = runCmd("sed -i 's/Use a boot loader./%s/' %s" % (modstring, modfile))
self.assertEqual(0,modapplied.status,'Modification to %s on kernel source failed' % modfile)
#Modify the configuration
- codeconfigfile = os.path.join(tempdir,'.config.new')
+ codeconfigfile = os.path.join(tempsrc, '.config.new')
modconfopt = "CONFIG_SG_POOL=n"
modconf = runCmd("sed -i 's/CONFIG_SG_POOL=y/%s/' %s" % (modconfopt, codeconfigfile))
self.assertEqual(0,modconf.status,'Modification to %s failed' % codeconfigfile)
--
2.9.5
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH 11/12] oe-selftest: devtool: add test for multiple source trees
2017-12-05 1:41 [PATCH 00/12] devtool improvements Paul Eggleton
` (9 preceding siblings ...)
2017-12-05 1:41 ` [PATCH 10/12] oe-selftest: devtool: fix kernel test for " Paul Eggleton
@ 2017-12-05 1:41 ` Paul Eggleton
2017-12-05 1:41 ` [PATCH 12/12] lib/oe/recipeutils: add .txz extension to archive list Paul Eggleton
` (2 subsequent siblings)
13 siblings, 0 replies; 16+ messages in thread
From: Paul Eggleton @ 2017-12-05 1:41 UTC (permalink / raw)
To: openembedded-core
Add two synthetic tests for devtool modify + devtool finish: first with
multiple source trees side-by-side, and second to test with one as a
subdirectory of the main source tree. These also test devtool finish's
recently added dry-run option and that detects and errors on uncommitted
changes without being forced.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
.../devtool/devtool-test-multisrc-subdir_1.0.bb | 13 +++
.../devtool-test-multisrc/example-files.tar.gz | Bin 0 -> 236 bytes
.../devtool/devtool-test-multisrc/example.patch | 7 ++
.../devtool-test-multisrc/mypackage-1.0.tar.gz | Bin 0 -> 903 bytes
.../devtool/devtool-test-multisrc_1.0.bb | 11 +++
meta/lib/oeqa/selftest/cases/devtool.py | 109 +++++++++++++++++++++
6 files changed, 140 insertions(+)
create mode 100644 meta-selftest/recipes-test/devtool/devtool-test-multisrc-subdir_1.0.bb
create mode 100644 meta-selftest/recipes-test/devtool/devtool-test-multisrc/example-files.tar.gz
create mode 100644 meta-selftest/recipes-test/devtool/devtool-test-multisrc/example.patch
create mode 100644 meta-selftest/recipes-test/devtool/devtool-test-multisrc/mypackage-1.0.tar.gz
create mode 100644 meta-selftest/recipes-test/devtool/devtool-test-multisrc_1.0.bb
diff --git a/meta-selftest/recipes-test/devtool/devtool-test-multisrc-subdir_1.0.bb b/meta-selftest/recipes-test/devtool/devtool-test-multisrc-subdir_1.0.bb
new file mode 100644
index 0000000..0430382
--- /dev/null
+++ b/meta-selftest/recipes-test/devtool/devtool-test-multisrc-subdir_1.0.bb
@@ -0,0 +1,13 @@
+SUMMARY = "Recipe to test multiple source trees"
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://settings-daemon.c;beginline=1;endline=26;md5=8d77ba1c7a55df48d8d06c5f3d86b49d"
+
+SRC_URI = "git://git.yoctoproject.org/xsettings-daemon;name=xsettings-daemon \
+ git://git.yoctoproject.org/libfakekey;name=libfakekey;destsuffix=git/libfakekey \
+"
+
+SRCREV_xsettings-daemon = "b2e5da502f8c5ff75e9e6da771372ef8e40fd9a2"
+SRCREV_libfakekey = "7ad885912efb2131e80914e964d5e635b0d07b40"
+
+S = "${WORKDIR}/git"
+
diff --git a/meta-selftest/recipes-test/devtool/devtool-test-multisrc/example-files.tar.gz b/meta-selftest/recipes-test/devtool/devtool-test-multisrc/example-files.tar.gz
new file mode 100644
index 0000000000000000000000000000000000000000..fdab4b7367cda3613468c7813f47acaaac74368c
GIT binary patch
literal 236
zcmV<I02BWoiwFR^niN_91MSt{3W6{c25_%^ioHOzKOT4OV|2NTMS+><gdV>c2AY--
zQ;ELc&A<)LfwRxqo4J&^-5t-lt@qI)xTiI>tI)*z>r*6@<Rk=1(h(O@`A>|NX=kXZ
zov9d$O4H<@&du%rC_9+Hv$a#fYqSGS)8zg9Rr)f2CH-;6gLg9F&*y)zj+x@>PPSs*
zU1059h+`kZc<b(9Y2*Bh;5BCRFMI$0L_YGL5c0o<tQf|A>waKe<NV9uHRkhAz5I9Y
mKS%!8kWIb)TQ>y&000000000000000U%3O3K98mVC;$MDWO;W0
literal 0
HcmV?d00001
diff --git a/meta-selftest/recipes-test/devtool/devtool-test-multisrc/example.patch b/meta-selftest/recipes-test/devtool/devtool-test-multisrc/example.patch
new file mode 100644
index 0000000..2e48bf1
--- /dev/null
+++ b/meta-selftest/recipes-test/devtool/devtool-test-multisrc/example.patch
@@ -0,0 +1,7 @@
+diff -udNr example-files/test2 example-files2/test2
+--- example-files/test2 2017-11-22 10:29:11.766891385 +1300
++++ example-files2/test2 2017-11-22 10:30:08.920094705 +1300
+@@ -1 +1,3 @@
+ Another example file
++
++More content added here
diff --git a/meta-selftest/recipes-test/devtool/devtool-test-multisrc/mypackage-1.0.tar.gz b/meta-selftest/recipes-test/devtool/devtool-test-multisrc/mypackage-1.0.tar.gz
new file mode 100644
index 0000000000000000000000000000000000000000..b513b283e9adece54a2495ad5f9eaf8eb9eebdd0
GIT binary patch
literal 903
zcmV;219<!&iwFP}s}x!Q1MOE!Z`(K!_F2DTkkf9Vb!^8Epf|>%V`62=kmP1_lqK5Y
zt|bbh+-&~+&M0=XF3{$ZwkXOE0R#?bzIla1x_qkB?4NWm{}}cM*S&9o0b(|r(04eS
z4F3N;qc^-64JO0UWIP=82E*ZOHtxaX+k|t8qiNFypjW5I!`Jj)+y6;XUi5z+t{+5j
z*I$X`4Si69r_=G5`ac>BZ$9h)&3JOtgTWid@J0OM{a?$bRK_S>fih6YMsA;A-=tM5
za~F1vl%RKz6=}1VF0>lb>IrJu7&Oq^HdU3X_K-rR>nCtfwJngx=w17mHWIaSNR82%
zN|6w9ogGVAwP{NZcj_Pw{MHr{uHuu$)$g5`T&4$bR0Xto-h)Tg7W&viBTd_=j7VHi
zRdzV$6vcV_ph|UeLFSFx7p(yYX^#ezpio^Xb*^^wm0f1_aeGjva3NR3y*;)lG$hPq
zMOK)}wQj)30|tf^3PbPGeTlDAqUdWXYI`b^A<1K*%gbD_O3v=sROnQ8M!Ckabw2-=
zSxXYq-|53aKT>v?u5v|5nfH#9;8wcTA7z(KmmyTT#bBRfpeEEWtvzj-BE{&p@>DbQ
zixqZAc21}v|9B#*R$&8b-E<B=PqdE#EIGvCBDrS~2NA<M3O@*sdvL{KTwl3xFOp@r
zNdPq>79<Z4E`S9O@TUko7x-W6h{rL65jbMC_60|w2<HCA!vO@hXc&YE_yR`}vL+#r
z#}k*}F|n<9G+*M9-3nhM4=y+hkp#rLKw1WC7A0c7@mU1xO|%YUjsbYc8;D>Lp&P#9
zLDEOJC<Fch7Z5L*?{~f&w!s9VE<~D(k+@qXunc{VqwtnvZ0y$O&%Q9PxzEJP1&^)R
z9q+6{q`~w^?-bzul6Nxn$nbBTh%lf`=3$UTxO6emDEVZ2FJkTji$qK{Swtc7J5*CN
z2|E@v4EQq*)f+C_f-1D$#QY^1k24?1VluvHV&D1EqsZb1;PU=2ah}su?q_f5<CXg#
zUjUQy{XZU02Xz0>M)v;yF1UUNbeB-$yZad4T|0IX{5rTi{}E^2iodmwSI_@+`04yl
zMs)s1Q{1-a|2u#IzIH#-V^98@mNgE5)Bl4@kH&%?p7;_q_%nch8}#jLu&}VOu&}VO
du&}VOu&}VOu&}VOu(0?E@E>2ucu@c-008QizRdst
literal 0
HcmV?d00001
diff --git a/meta-selftest/recipes-test/devtool/devtool-test-multisrc_1.0.bb b/meta-selftest/recipes-test/devtool/devtool-test-multisrc_1.0.bb
new file mode 100644
index 0000000..2b81f83
--- /dev/null
+++ b/meta-selftest/recipes-test/devtool/devtool-test-multisrc_1.0.bb
@@ -0,0 +1,11 @@
+SUMMARY = "Recipe to test multiple source trees"
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
+
+SRC_URI = "file://mypackage-${PV}.tar.gz \
+ file://example-files.tar.gz \
+ file://example.patch;patchdir=../example-files \
+"
+
+S = "${WORKDIR}/mypackage-${PV}"
+
diff --git a/meta/lib/oeqa/selftest/cases/devtool.py b/meta/lib/oeqa/selftest/cases/devtool.py
index 96072be..b051b7c 100644
--- a/meta/lib/oeqa/selftest/cases/devtool.py
+++ b/meta/lib/oeqa/selftest/cases/devtool.py
@@ -1568,6 +1568,115 @@ class DevtoolTests(DevtoolBase):
if files:
self.fail('Unexpected file(s) copied next to bbappend: %s' % ', '.join(files))
+ def test_devtool_finish_modify_multisrc(self):
+ # Check preconditions
+ self.assertTrue(not os.path.exists(self.workspacedir), 'This test cannot be run with a workspace directory under the build directory')
+ # Try modifying a recipe
+ self.track_for_cleanup(self.workspacedir)
+ recipe = 'devtool-test-multisrc'
+ recipefile = get_bb_var('FILE', recipe)
+ recipedir = os.path.dirname(recipefile)
+ result = runCmd('git status --porcelain .', cwd=recipedir)
+ if result.output.strip():
+ self.fail('Recipe directory for %s contains uncommitted changes' % recipe)
+ tempdir = tempfile.mkdtemp(prefix='devtoolqa')
+ self.track_for_cleanup(tempdir)
+ self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
+ result = runCmd('devtool modify %s %s' % (recipe, tempdir))
+ mainsrc = os.path.join(tempdir, 'mypackage-1.0')
+ extrasrc = os.path.join(tempdir, 'example-files')
+ self.assertExists(os.path.join(mainsrc, 'mainfile.c'), 'Extracted main source tree could not be found')
+ self.assertExists(os.path.join(extrasrc, 'test2'), 'Extracted extra source tree could not be found')
+ # Try dry-run finishing without changes
+ result = runCmd('devtool finish %s meta-selftest -N' % recipe)
+ self.assertIn('No patches or files need updating', result.output)
+ # Make some changes to both the main and extra source trees
+ with open(os.path.join(mainsrc, 'mainfile.c'), 'a') as f:
+ f.write('\n/* Additional comment */\n')
+ result = runCmd('git commit -a -m "Add a comment"', cwd=mainsrc)
+ with open(os.path.join(extrasrc, 'test2'), 'a') as f:
+ f.write('\nAnother line\n')
+ result = runCmd('git commit -a -m "Add another line"', cwd=extrasrc)
+ # Try finishing now with dry-run to see if it reports what we expect
+ result = runCmd('devtool finish %s meta-selftest -N' % recipe)
+ self.assertNotIn('No patches or files need updating', result.output)
+ self.assertIn('+ file://0001-Add-a-comment.patch \\', result.output)
+ self.assertIn('+ file://0001-Add-another-line.patch;patchdir=../example-files \\', result.output)
+ self.assertNotIn('Removing', result.output)
+ # Now really finish
+ self.add_command_to_tearDown('rm -rf %s ; cd %s ; git checkout %s' % (recipedir, os.path.dirname(recipedir), recipedir))
+ result = runCmd('devtool finish %s meta-selftest' % recipe)
+ expected_status = [(' M', '.*/%s$' % os.path.basename(recipefile)),
+ ('??', '.*/.*-Add-a-comment.patch$'),
+ ('??', '.*/.*-Add-another-line.patch$')]
+ self._check_repo_status(recipedir, expected_status)
+
+ def test_devtool_finish_modify_multisrc_subdir(self):
+ # Check preconditions
+ self.assertTrue(not os.path.exists(self.workspacedir), 'This test cannot be run with a workspace directory under the build directory')
+ # Try modifying a recipe
+ self.track_for_cleanup(self.workspacedir)
+ recipe = 'devtool-test-multisrc-subdir'
+ recipefile = get_bb_var('FILE', recipe)
+ recipedir = os.path.dirname(recipefile)
+ result = runCmd('git status --porcelain .', cwd=recipedir)
+ if result.output.strip():
+ self.fail('Recipe directory for %s contains uncommitted changes' % recipe)
+ tempdir = tempfile.mkdtemp(prefix='devtoolqa')
+ self.track_for_cleanup(tempdir)
+ self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
+ result = runCmd('devtool modify %s %s' % (recipe, tempdir))
+ mainsrc = tempdir
+ extrasrc = os.path.join(tempdir, 'libfakekey')
+ self.assertExists(os.path.join(mainsrc, 'xsettings-client.c'), 'Extracted main source tree could not be found')
+ self.assertExists(os.path.join(extrasrc, 'configure.ac'), 'Extracted extra source tree could not be found')
+ # Try dry-run finishing without changes
+ result = runCmd('devtool finish %s meta-selftest -N' % recipe)
+ self.assertIn('No patches or files need updating', result.output)
+ # Make some changes to both the main and extra source trees
+ with open(os.path.join(mainsrc, 'xsettings-client.c'), 'a') as f:
+ f.write('\n/* Additional comment */\n')
+ result = runCmd('git commit -a -m "Add a comment"', cwd=mainsrc)
+ with open(os.path.join(extrasrc, 'README'), 'a') as f:
+ f.write('\nAnother line\n')
+ result = runCmd('git commit -a -m "Add another line to README"', cwd=extrasrc)
+ # Try finishing now with dry-run to see if it reports what we expect
+ # However, the first time it will fail because git treats sub-repositories
+ # the same as submodules, so the parent will appear to have uncommitted changes
+ failed = False
+ try:
+ result = runCmd('devtool finish %s meta-selftest -N' % recipe, assert_error=False)
+ except CommandError as err:
+ if err.retcode != 1:
+ self.fail('Unexpected failure from devtool finish:\n%s' % err.output)
+ lines = []
+ collecting = False
+ for line in err.output.splitlines():
+ if collecting:
+ if line:
+ lines.append(line)
+ elif line.startswith('ERROR: Source tree %s is not clean' % mainsrc):
+ collecting = True
+ lines = lines[:-1]
+ self.assertEqual(lines, [' M libfakekey'], 'Unexpected modifications to main source tree')
+ if not collecting:
+ self.fail('devtool finish did not report uncommitted changes as expected:\n%s' % err.output)
+ failed = True
+ if not failed:
+ self.fail('devtool finish was expected to fail but did not:\n%s' % result.output)
+ result = runCmd('devtool finish %s meta-selftest -N -f' % recipe)
+ self.assertNotIn('No patches or files need updating', result.output)
+ self.assertIn('+ file://0001-Add-a-comment.patch \\', result.output)
+ self.assertIn('+ file://0001-Add-another-line-to-README.patch;patchdir=libfakekey \\', result.output)
+ self.assertNotIn('Removing', result.output)
+ self.add_command_to_tearDown('rm -rf %s ; cd %s ; git checkout %s' % (recipedir, os.path.dirname(recipedir), recipedir))
+ result = runCmd('devtool finish %s -f meta-selftest' % recipe)
+ expected_status = [(' M', '.*/%s$' % os.path.basename(recipefile)),
+ ('??', '.*/%s/$' % recipe)]
+ self._check_repo_status(recipedir, expected_status)
+ self.assertExists(os.path.join(os.path.dirname(recipefile), recipe, '0001-Add-a-comment.patch'))
+ self.assertExists(os.path.join(os.path.dirname(recipefile), recipe, '0001-Add-another-line-to-README.patch'))
+
@OETestID(1626)
def test_devtool_rename(self):
# Check preconditions
--
2.9.5
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH 12/12] lib/oe/recipeutils: add .txz extension to archive list
2017-12-05 1:41 [PATCH 00/12] devtool improvements Paul Eggleton
` (10 preceding siblings ...)
2017-12-05 1:41 ` [PATCH 11/12] oe-selftest: devtool: add test for multiple source trees Paul Eggleton
@ 2017-12-05 1:41 ` Paul Eggleton
2017-12-05 2:07 ` ✗ patchtest: failure for devtool improvements Patchwork
2017-12-13 14:53 ` [PATCH 00/12] " Alexander Kanavin
13 siblings, 0 replies; 16+ messages in thread
From: Paul Eggleton @ 2017-12-05 1:41 UTC (permalink / raw)
To: openembedded-core
Prompted by bitbake commit 2ba8a6b25ccc12e7b543e8450121e5311c7a701d,
add .txz to the list of archives used within get_recipe_local_files()
here as well.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
meta/lib/oe/recipeutils.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/meta/lib/oe/recipeutils.py b/meta/lib/oe/recipeutils.py
index f4a40c2..ac3b027 100644
--- a/meta/lib/oe/recipeutils.py
+++ b/meta/lib/oe/recipeutils.py
@@ -413,7 +413,7 @@ def get_recipe_local_files(d, patches=False, archives=False):
# fetcher) though note that this only encompasses actual container formats
# i.e. that can contain multiple files as opposed to those that only
# contain a compressed stream (i.e. .tar.gz as opposed to just .gz)
- archive_exts = ['.tar', '.tgz', '.tar.gz', '.tar.Z', '.tbz', '.tbz2', '.tar.bz2', '.tar.xz', '.tar.lz', '.zip', '.jar', '.rpm', '.srpm', '.deb', '.ipk', '.tar.7z', '.7z']
+ archive_exts = ['.tar', '.tgz', '.tar.gz', '.tar.Z', '.tbz', '.tbz2', '.tar.bz2', '.txz', '.tar.xz', '.tar.lz', '.zip', '.jar', '.rpm', '.srpm', '.deb', '.ipk', '.tar.7z', '.7z']
ret = {}
for uri in uris:
if fetch.ud[uri].type == 'file':
--
2.9.5
^ permalink raw reply related [flat|nested] 16+ messages in thread
* ✗ patchtest: failure for devtool improvements
2017-12-05 1:41 [PATCH 00/12] devtool improvements Paul Eggleton
` (11 preceding siblings ...)
2017-12-05 1:41 ` [PATCH 12/12] lib/oe/recipeutils: add .txz extension to archive list Paul Eggleton
@ 2017-12-05 2:07 ` Patchwork
2017-12-13 14:53 ` [PATCH 00/12] " Alexander Kanavin
13 siblings, 0 replies; 16+ messages in thread
From: Patchwork @ 2017-12-05 2:07 UTC (permalink / raw)
To: Paul Eggleton; +Cc: openembedded-core
== Series Details ==
Series: devtool improvements
Revision: 1
URL : https://patchwork.openembedded.org/series/10071/
State : failure
== Summary ==
Thank you for submitting this patch series to OpenEmbedded Core. This is
an automated response. Several tests have been executed on the proposed
series by patchtest resulting in the following failures:
* Issue A patch file has been added, but does not have a Signed-off-by tag [test_signed_off_by_presence]
Suggested fix Sign off the added patch file (meta-selftest/recipes-test/devtool/devtool-test-multisrc/example.patch)
* Issue Added patch file is missing Upstream-Status in the header [test_upstream_status_presence_format]
Suggested fix Add Upstream-Status: <Valid status> to the header of meta-selftest/recipes-test/devtool/devtool-test-multisrc/example.patch
Standard format Upstream-Status: <Valid status>
Valid status Pending, Accepted, Backport, Denied, Inappropriate [reason], Submitted [where]
If you believe any of these test results are incorrect, please reply to the
mailing list (openembedded-core@lists.openembedded.org) raising your concerns.
Otherwise we would appreciate you correcting the issues and submitting a new
version of the patchset if applicable. Please ensure you add/increment the
version number when sending the new version (i.e. [PATCH] -> [PATCH v2] ->
[PATCH v3] -> ...).
---
Guidelines: https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines
Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
Test suite: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH 00/12] devtool improvements
2017-12-05 1:41 [PATCH 00/12] devtool improvements Paul Eggleton
` (12 preceding siblings ...)
2017-12-05 2:07 ` ✗ patchtest: failure for devtool improvements Patchwork
@ 2017-12-13 14:53 ` Alexander Kanavin
2017-12-13 17:46 ` Paul Eggleton
13 siblings, 1 reply; 16+ messages in thread
From: Alexander Kanavin @ 2017-12-13 14:53 UTC (permalink / raw)
To: Paul Eggleton, openembedded-core, Burton, Ross, Richard Purdie
On 12/05/2017 03:41 AM, Paul Eggleton wrote:
> A set of improvements for devtool, mostly aimed at improving the
> handling of recipes that fetch multiple source trees (i.e. have multiple
> tarballs or repositories in SRC_URI), with a couple of other fixes
> thrown in.
Unfortunately, this badly breaks 'devtool upgrade':
======
ak@kanavin-desktop:~/development/poky/build$ devtool upgrade openssl -V
1.0.2n
.....
WARNING: This recipe fetches more than one source tree. Each source tree
will be tracked in a separate git repository.
NOTE: Extracting upgraded version source...
ERROR: Execution of 'git checkout devtool-base -b devtool-1.0.2n' failed
with exit code 128:
fatal: 'devtool-base' is not a commit and a branch 'devtool-1.0.2n'
cannot be created from it
=====
I would strongly request that this patchset not be merged into master
until the issue is resolved.
Alex
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH 00/12] devtool improvements
2017-12-13 14:53 ` [PATCH 00/12] " Alexander Kanavin
@ 2017-12-13 17:46 ` Paul Eggleton
0 siblings, 0 replies; 16+ messages in thread
From: Paul Eggleton @ 2017-12-13 17:46 UTC (permalink / raw)
To: Alexander Kanavin; +Cc: openembedded-core
Hi Alex,
On Thursday, 14 December 2017 3:53:22 AM NZDT Alexander Kanavin wrote:
> On 12/05/2017 03:41 AM, Paul Eggleton wrote:
> > A set of improvements for devtool, mostly aimed at improving the
> > handling of recipes that fetch multiple source trees (i.e. have multiple
> > tarballs or repositories in SRC_URI), with a couple of other fixes
> > thrown in.
>
> Unfortunately, this badly breaks 'devtool upgrade':
>
> ======
> ak@kanavin-desktop:~/development/poky/build$ devtool upgrade openssl -V
> 1.0.2n
> .....
> WARNING: This recipe fetches more than one source tree. Each source tree
> will be tracked in a separate git repository.
> NOTE: Extracting upgraded version source...
> ERROR: Execution of 'git checkout devtool-base -b devtool-1.0.2n' failed
> with exit code 128:
> fatal: 'devtool-base' is not a commit and a branch 'devtool-1.0.2n'
> cannot be created from it
> =====
>
> I would strongly request that this patchset not be merged into master
> until the issue is resolved.
That's interesting - the devtool upgrade tests apparently passed so clearly we
have a hole in our tests. I will look into it.
Cheers,
Paul
--
Paul Eggleton
Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 16+ messages in thread
end of thread, other threads:[~2017-12-13 17:46 UTC | newest]
Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-12-05 1:41 [PATCH 00/12] devtool improvements Paul Eggleton
2017-12-05 1:41 ` [PATCH 01/12] devtool: refactor return for _extract_source() Paul Eggleton
2017-12-05 1:41 ` [PATCH 02/12] devtool: reset: delete bbappend file if _check_preserve() doesn't Paul Eggleton
2017-12-05 1:41 ` [PATCH 03/12] devtool: finish: fix erroneously creating bbappend for relative paths Paul Eggleton
2017-12-05 1:41 ` [PATCH 04/12] devtool: deploy-target: don't specify ssh/scp port unless user does Paul Eggleton
2017-12-05 1:41 ` [PATCH 05/12] devtool: extract: fix usage with kern-tools-native Paul Eggleton
2017-12-05 1:41 ` [PATCH 06/12] devtool: modify: pick up commits from previously created source tree Paul Eggleton
2017-12-05 1:41 ` [PATCH 07/12] lib/oe/recipeutils: allow patch_recipe_file() to be re-called Paul Eggleton
2017-12-05 1:41 ` [PATCH 08/12] devtool: support extracting multiple source trees Paul Eggleton
2017-12-05 1:41 ` [PATCH 09/12] devtool: fix handling of linux-yocto after multisrc changes Paul Eggleton
2017-12-05 1:41 ` [PATCH 10/12] oe-selftest: devtool: fix kernel test for " Paul Eggleton
2017-12-05 1:41 ` [PATCH 11/12] oe-selftest: devtool: add test for multiple source trees Paul Eggleton
2017-12-05 1:41 ` [PATCH 12/12] lib/oe/recipeutils: add .txz extension to archive list Paul Eggleton
2017-12-05 2:07 ` ✗ patchtest: failure for devtool improvements Patchwork
2017-12-13 14:53 ` [PATCH 00/12] " Alexander Kanavin
2017-12-13 17:46 ` Paul Eggleton
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.