[master][PATCH v4 0/3] Devtool: provide easy means of

[master][PATCH v4 0/3] Devtool: provide easy means of

[Sai Hari Chandana Kalluri]

This patch series provides support for the user to run menuconfig command in the
devtool flow. This would allow the user to modify the current configurations and
generate a config fragment to update the recipe using devtool finish. Devtool
menuconfig command will work on all packages that contain menuconfig as a task.

1. The implementation checks if devtool menuconfig command is called for a valid
package.
2. It checks for oe-local-files dir within source and creates one if
needed, this directory is needed to store the final generated config fragment so
that devtool finish can update the recipe.
3. Menuconfig command is called for users to make necessary changes. After
saving the changes, diffconfig command is run to generate the fragment.

Currently, when the user runs devtool modify command, it checks out the entire
source tree which is a bit of an over head in time and space. This patch series
also provides a way to create a copy(hard links) of the kernel source, if
present, from work-shared to workspace to be more efficient .

Also, if the kernel source is not present in the staging kernel dir and the user
fetches the source tree in workspace using devtool modify, then this patch
series creates a copy of source from workspace to work-shared. This is
necessary for packages that may use the kernel source.

[YOCTO #10416]

---                                                                                                               
changelog v2
1. Add a check to verify the set kernel version by user and kernel version                                        
present in work-shared before creating hard links to workspace.
2. Add a check to verify the set kernel version by user and kernel version                                        
present in workspace do not match, when placing a copy of kernel-source
from workspace to work-shared.

changelog v3
1. Use oe.path.copyhardlinktree() to create copy of source from work-shared to
devtool workspace.
2. Split all the changes into appropriate patches. Move all
menuconfig command specific changes under devtool menuconfig command patch
(3/3).
 
changelog v4
1. create a new function that implements the creation of symbolic links for
oe-local-files to workspace within _extract_sourcei(). This avoids code duplication
and allows the function to be reused within devtool modify when copying source from
work-shared to devtool workspace.
2. use check_workspace_recipe() within menuconfig.py

Sai Hari Chandana Kalluri (3):
  devtool/standard.py: Update devtool modify to copy source from
    work-shared if its already downloaded
  devtool/standard.py: Create a copy of kernel source within work-shared
    if not present
  devtool: provide support for devtool menuconfig command

 scripts/lib/devtool/menuconfig.py |  80 +++++++++++++++++++++
 scripts/lib/devtool/standard.py   | 141 +++++++++++++++++++++++++++++++-------
 2 files changed, 197 insertions(+), 24 deletions(-)
 create mode 100644 scripts/lib/devtool/menuconfig.py

-- 
2.7.4

[master][PATCH v4 1/3] devtool/standard.py: Update devtool modify to copy source from work-shared if its already downloaded

[Sai Hari Chandana Kalluri]

In the regular devtool modify flow, the kernel source is fetched by
running do_fetch task. This is an overhead in time and space.

This patch updates modify command to check if the kernel source is
already downloaded. If so, then instead of calling do_fetch, copy the
source from work-shared to devtool workspace by creating hard links
else run the usual devtool modify flow and call do_fetch task.

[YOCTO #10416]

Signed-off-by: Sai Hari Chandana Kalluri <chandana.kalluri@xilinx.com>
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
---
 scripts/lib/devtool/standard.py | 121 ++++++++++++++++++++++++++++++++--------
 1 file changed, 98 insertions(+), 23 deletions(-)

diff --git a/scripts/lib/devtool/standard.py b/scripts/lib/devtool/standard.py
index 0a1e329..33193f8 100644
--- a/scripts/lib/devtool/standard.py
+++ b/scripts/lib/devtool/standard.py
@@ -471,6 +471,30 @@ def sync(args, config, basepath, workspace):
     finally:
         tinfoil.shutdown()
 
+def symblink_oelocal_files_srctree(rd,srctree):
+   if os.path.abspath(rd.getVar('S')) == os.path.abspath(rd.getVar('WORKDIR')):
+   # If recipe extracts to ${WORKDIR}, symlink the files into the srctree
+   # (otherwise the recipe won't build as expected)
+         local_files_dir = os.path.join(srctree, 'oe-local-files')
+         addfiles = []
+         for root, _, files in os.walk(local_files_dir):
+             relpth = os.path.relpath(root, local_files_dir)
+             if relpth != '.':
+                   bb.utils.mkdirhier(os.path.join(srctree, relpth))
+             for fn in files:
+                 if fn == '.gitignore':
+                     continue
+                 destpth = os.path.join(srctree, relpth, fn)
+                 if os.path.exists(destpth):
+                     os.unlink(destpth)
+                 os.symlink('oe-local-files/%s' % fn, destpth)
+                 addfiles.append(os.path.join(relpth, fn))
+         if addfiles:
+            bb.process.run('git add %s' % ' '.join(addfiles), cwd=srctree)
+         useroptions = []
+         oe.patch.GitApplyTree.gitCommandUserOptions(useroptions, d=d)
+         bb.process.run('git %s commit -a -m "Committing local file symlinks\n\n%s"' % (' '.join(useroptions), oe.patch.GitApplyTree.ignore_commit_prefix), cwd=srctree)
+
 
 def _extract_source(srctree, keep_temp, devbranch, sync, config, basepath, workspace, fixed_setup, d, tinfoil, no_overrides=False):
     """Extract sources of a recipe"""
@@ -627,29 +651,7 @@ def _extract_source(srctree, keep_temp, devbranch, sync, config, basepath, works
                 shutil.move(tempdir_localdir, srcsubdir)
 
             shutil.move(srcsubdir, srctree)
-
-            if os.path.abspath(d.getVar('S')) == os.path.abspath(d.getVar('WORKDIR')):
-                # If recipe extracts to ${WORKDIR}, symlink the files into the srctree
-                # (otherwise the recipe won't build as expected)
-                local_files_dir = os.path.join(srctree, 'oe-local-files')
-                addfiles = []
-                for root, _, files in os.walk(local_files_dir):
-                    relpth = os.path.relpath(root, local_files_dir)
-                    if relpth != '.':
-                        bb.utils.mkdirhier(os.path.join(srctree, relpth))
-                    for fn in files:
-                        if fn == '.gitignore':
-                            continue
-                        destpth = os.path.join(srctree, relpth, fn)
-                        if os.path.exists(destpth):
-                            os.unlink(destpth)
-                        os.symlink('oe-local-files/%s' % fn, destpth)
-                        addfiles.append(os.path.join(relpth, fn))
-                if addfiles:
-                    bb.process.run('git add %s' % ' '.join(addfiles), cwd=srctree)
-                useroptions = []
-                oe.patch.GitApplyTree.gitCommandUserOptions(useroptions, d=d)
-                bb.process.run('git %s commit -a -m "Committing local file symlinks\n\n%s"' % (' '.join(useroptions), oe.patch.GitApplyTree.ignore_commit_prefix), cwd=srctree)
+            symblink_oelocal_files_srctree(d,srctree)
 
         if is_kernel_yocto:
             logger.info('Copying kernel config to srctree')
@@ -717,11 +719,31 @@ def _check_preserve(config, recipename):
                     tf.write(line)
     os.rename(newfile, origfile)
 
+def get_staging_kver(srcdir):
+     #Kernel version from work-shared
+     kerver = []
+     staging_kerVer=""
+     if os.path.exists(srcdir) and os.listdir(srcdir):
+            with open(os.path.join(srcdir,"Makefile")) as f:
+                 version = [next(f) for x in range(5)][1:4]
+                 for word in version:
+                       kerver.append(word.split('= ')[1].split('\n')[0])
+                 staging_kerVer = ".".join(kerver)
+     return staging_kerVer
+
+def get_staging_kbranch(srcdir):
+      staging_kbranch = ""
+      if os.path.exists(srcdir) and os.listdir(srcdir):
+          (branch, _) = bb.process.run('git branch | grep \* | cut -d \' \' -f2', cwd=srcdir)
+          staging_kbranch = "".join(branch.split('\n')[0])
+      return staging_kbranch
+
 def modify(args, config, basepath, workspace):
     """Entry point for the devtool 'modify' subcommand"""
     import bb
     import oe.recipeutils
     import oe.patch
+    import oe.path
 
     if args.recipename in workspace:
         raise DevtoolError("recipe %s is already in your workspace" %
@@ -763,6 +785,59 @@ def modify(args, config, basepath, workspace):
         initial_rev = None
         commits = []
         check_commits = False
+
+        if bb.data.inherits_class('kernel-yocto', rd):
+               #Current set kernel version
+               kernelVersion = rd.getVar('LINUX_VERSION')
+               srcdir = rd.getVar('STAGING_KERNEL_DIR')
+               kbranch = rd.getVar('KBRANCH')
+
+               staging_kerVer = get_staging_kver(srcdir)
+               staging_kbranch = get_staging_kbranch(srcdir)
+               if (os.path.exists(srcdir) and os.listdir(srcdir)) and (kernelVersion in staging_kerVer and staging_kbranch == kbranch):
+                  oe.path.copyhardlinktree(srcdir,srctree)
+                  workdir = rd.getVar('WORKDIR')
+                  srcsubdir = rd.getVar('S')
+                  localfilesdir = os.path.join(srctree,'oe-local-files') 
+                  # Move local source files into separate subdir
+                  recipe_patches = [os.path.basename(patch) for patch in oe.recipeutils.get_recipe_patches(rd)]
+                  local_files = oe.recipeutils.get_recipe_local_files(rd)
+
+                  for key in local_files.copy():
+                        if key.endswith('scc'):
+                              sccfile = open(local_files[key], 'r')
+                              for l in sccfile:
+                                  line = l.split()
+                                  if line and line[0] in ('kconf', 'patch'):
+                                        cfg = os.path.join(os.path.dirname(local_files[key]), line[-1])
+                                        if not cfg in local_files.values():
+                                            local_files[line[-1]] = cfg
+                                            shutil.copy2(cfg, workdir)
+                              sccfile.close()
+
+                  # Ignore local files with subdir={BP}
+                  srcabspath = os.path.abspath(srcsubdir)
+                  local_files = [fname for fname in local_files if os.path.exists(os.path.join(workdir, fname)) and  (srcabspath == workdir or not  os.path.join(workdir, fname).startswith(srcabspath + os.sep))]
+                  if local_files:
+                       for fname in local_files:
+                              _move_file(os.path.join(workdir, fname), os.path.join(srctree, 'oe-local-files', fname))
+                       with open(os.path.join(srctree, 'oe-local-files', '.gitignore'), 'w') as f:
+                              f.write('# Ignore local files, by default. Remove this file ''if you want to commit the directory to Git\n*\n')
+
+                  symblink_oelocal_files_srctree(rd,srctree)
+
+                  task = 'do_configure'
+                  res = tinfoil.build_targets(pn, task, handle_events=True)
+
+                  # Copy .config to workspace 
+                  kconfpath=rd.getVar('B')
+                  logger.info('Copying kernel config to workspace')
+                  shutil.copy2(os.path.join(kconfpath, '.config'),srctree)
+
+                  # Set this to true, we still need to get initial_rev
+                  # by parsing the git repo
+                  args.no_extract = True
+
         if not args.no_extract:
             initial_rev, _ = _extract_source(srctree, args.keep_temp, args.branch, False, config, basepath, workspace, args.fixed_setup, rd, tinfoil, no_overrides=args.no_overrides)
             if not initial_rev:
-- 
2.7.4

[master][PATCH v4 2/3] devtool/standard.py: Create a copy of kernel source within work-shared if not present

[Sai Hari Chandana Kalluri]

If kernel source is not already downloaded i.e staging kernel dir is
empty, place a copy of the source when the user runs devtool modify
linux-yocto.  This way the kernel source is available for other packages
that use it.

[YOCTO #10416]

Signed-off-by: Sai Hari Chandana Kalluri <chandana.kalluri@xilinx.com>
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
---
 scripts/lib/devtool/standard.py | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

diff --git a/scripts/lib/devtool/standard.py b/scripts/lib/devtool/standard.py
index 33193f8..9267d6e 100644
--- a/scripts/lib/devtool/standard.py
+++ b/scripts/lib/devtool/standard.py
@@ -500,6 +500,7 @@ def _extract_source(srctree, keep_temp, devbranch, sync, config, basepath, works
     """Extract sources of a recipe"""
     import oe.recipeutils
     import oe.patch
+    import oe.path
 
     pn = d.getVar('PN')
 
@@ -596,7 +597,7 @@ def _extract_source(srctree, keep_temp, devbranch, sync, config, basepath, works
         with open(preservestampfile, 'w') as f:
             f.write(d.getVar('STAMP'))
         try:
-            if bb.data.inherits_class('kernel-yocto', d):
+            if is_kernel_yocto:
                 # We need to generate the kernel config
                 task = 'do_configure'
             else:
@@ -623,6 +624,18 @@ def _extract_source(srctree, keep_temp, devbranch, sync, config, basepath, works
             raise DevtoolError('Something went wrong with source extraction - the devtool-source class was not active or did not function correctly:\n%s' % str(e))
         srcsubdir_rel = os.path.relpath(srcsubdir, os.path.join(tempdir, 'workdir'))
 
+        # Check if work-shared is empty, if yes 
+        # find source and copy to work-shared
+        if is_kernel_yocto:
+              workshareddir = d.getVar('STAGING_KERNEL_DIR')
+              staging_kerVer = get_staging_kver(workshareddir) 
+              kernelVersion = d.getVar('LINUX_VERSION')
+              if (os.path.exists(workshareddir) and (not os.listdir(workshareddir) or kernelVersion!=staging_kerVer)): 
+                   shutil.rmtree(workshareddir)
+                   oe.path.copyhardlinktree(srcsubdir,workshareddir)
+              elif (not os.path.exists(workshareddir)):
+                   oe.path.copyhardlinktree(srcsubdir,workshareddir)
+
         tempdir_localdir = os.path.join(tempdir, 'oe-local-files')
         srctree_localdir = os.path.join(srctree, 'oe-local-files')
 
-- 
2.7.4

[master][PATCH v4 3/3] devtool: provide support for devtool menuconfig command

[Sai Hari Chandana Kalluri]

All packages that support the menuconfig task will be able to run
devtool menuconfig command. This would allow the user to modify the
current configure options and create a config fragment which can be
added to a recipe using devtool finish.

1. The patch checks if devtool menuconfig command is called for a valid
package.
2. It checks for oe-local-files dir within source and creates one if
needed, this directory is needed to store the final generated config
fragment so that devtool finish can update the recipe.
3. Menuconfig command is called for users to make necessary changes.
After saving the changes, diffconfig command is run to generate the
fragment.

Syntax:
	devtool menuconfig <package name>
	 Ex: devtool menuconfig linux-yocto

The config fragment is saved as devtool-fragment.cfg within
oe-local-files dir.

	Ex:
<workspace_path>/sources/linux-yocto/oe-local-files/devtool-fragment.cfg

Run devtool finish to update the recipe by appending the config fragment
to SRC_URI and place a copy of the fragment within the layer where the
recipe resides.
	Ex: devtool finish linux-yocto meta

[YOCTO #10416]

Signed-off-by: Sai Hari Chandana Kalluri <chandana.kalluri@xilinx.com>
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
---
 scripts/lib/devtool/menuconfig.py | 80 +++++++++++++++++++++++++++++++++++++++
 scripts/lib/devtool/standard.py   |  5 +++
 2 files changed, 85 insertions(+)
 create mode 100644 scripts/lib/devtool/menuconfig.py

diff --git a/scripts/lib/devtool/menuconfig.py b/scripts/lib/devtool/menuconfig.py
new file mode 100644
index 0000000..0ebcf74
--- /dev/null
+++ b/scripts/lib/devtool/menuconfig.py
@@ -0,0 +1,80 @@
+# OpenEmbedded Development tool - menuconfig command plugin
+#
+# Copyright (C) 2018 Xilinx
+# Written by: Chandana Kalluri <ckalluri@xilinx.com>
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+"""Devtool menuconfig plugin"""
+
+import os
+import bb
+import logging
+import argparse
+import re
+import glob
+from devtool import setup_tinfoil, parse_recipe, DevtoolError, standard, exec_build_env_command
+from devtool import check_workspace_recipe
+
+logger = logging.getLogger('devtool')
+
+def menuconfig(args, config, basepath, workspace):
+    """Entry point for the devtool 'menuconfig' subcommand"""
+
+    rd = "" 
+    kconfigpath = ""
+    pn_src = ""
+    localfilesdir = ""
+    workspace_dir = ""
+    tinfoil = setup_tinfoil(basepath=basepath)
+    try:
+      rd = parse_recipe(config, tinfoil, args.component, appends=True, filter_workspace=False)
+      if not rd:
+         return 1
+
+      check_workspace_recipe(workspace, args.component)
+      pn =  rd.getVar('PN', True)
+
+      if not rd.getVarFlag('do_menuconfig','task'):
+         raise DevtoolError("This recipe does not support menuconfig option")
+
+      workspace_dir = os.path.join(basepath,'workspace/sources')
+      kconfigpath = rd.getVar('B')
+      pn_src = os.path.join(workspace_dir,pn)
+
+      #add check to see if oe_local_files exists or not
+      localfilesdir = os.path.join(pn_src,'oe-local-files') 
+      if not os.path.exists(localfilesdir):
+          bb.utils.mkdirhier(localfilesdir)
+          #Add gitignore to ensure source tree is clean
+          gitignorefile = os.path.join(localfilesdir,'.gitignore')
+          with open(gitignorefile, 'w') as f:
+                  f.write('# Ignore local files, by default. Remove this file if you want to commit the directory to Git\n')
+                  f.write('*\n')
+
+    finally:
+      tinfoil.shutdown()
+
+    logger.info('Launching menuconfig')
+    exec_build_env_command(config.init_path, basepath, 'bitbake -c menuconfig %s' % pn, watch=True) 
+    fragment = os.path.join(localfilesdir, 'devtool-fragment.cfg')
+    res = standard._create_kconfig_diff(pn_src,rd,fragment)
+
+    return 0
+
+def register_commands(subparsers, context):
+    """register devtool subcommands from this plugin"""
+    parser_menuconfig = subparsers.add_parser('menuconfig',help='Alter build-time configuration for a recipe', description='Launches the make menuconfig command (for recipes where do_menuconfig is available), allowing users to make changes to the build-time configuration. Creates a config fragment corresponding to changes made.', group='advanced') 
+    parser_menuconfig.add_argument('component', help='compenent to alter config')
+    parser_menuconfig.set_defaults(func=menuconfig,fixed_setup=context.fixed_setup)
diff --git a/scripts/lib/devtool/standard.py b/scripts/lib/devtool/standard.py
index 9267d6e..6ef7255 100644
--- a/scripts/lib/devtool/standard.py
+++ b/scripts/lib/devtool/standard.py
@@ -942,6 +942,11 @@ def modify(args, config, basepath, workspace):
                         '    cp ${B}/.config ${S}/.config.baseline\n'
                         '    ln -sfT ${B}/.config ${S}/.config.new\n'
                         '}\n')
+            if rd.getVarFlag('do_menuconfig','task'):
+                    f.write('\ndo_configure_append() {\n'
+                    '    cp ${B}/.config ${S}/.config.baseline\n'
+                    '    ln -sfT ${B}/.config ${S}/.config.new\n'
+                    '}\n')
             if initial_rev:
                 f.write('\n# initial_rev: %s\n' % initial_rev)
                 for commit in commits:
-- 
2.7.4

Re: [master][PATCH v4 0/3] Devtool: provide easy means of

[Paul Eggleton]

Hi Chandana

My apologies for the delay in reviewing this - other work got in the way of OE-Core contributions but that should be dealt with now.

On Thursday, 18 April 2019 10:42:56 AM NZST Sai Hari Chandana Kalluri wrote:
> This patch series provides support for the user to run menuconfig command in the
> devtool flow. This would allow the user to modify the current configurations and
> generate a config fragment to update the recipe using devtool finish. Devtool
> menuconfig command will work on all packages that contain menuconfig as a task.
> 
> 1. The implementation checks if devtool menuconfig command is called for a valid
> package.
> 2. It checks for oe-local-files dir within source and creates one if
> needed, this directory is needed to store the final generated config fragment so
> that devtool finish can update the recipe.
> 3. Menuconfig command is called for users to make necessary changes. After
> saving the changes, diffconfig command is run to generate the fragment.
> 
> Currently, when the user runs devtool modify command, it checks out the entire
> source tree which is a bit of an over head in time and space. This patch series
> also provides a way to create a copy(hard links) of the kernel source, if
> present, from work-shared to workspace to be more efficient .
> 
> Also, if the kernel source is not present in the staging kernel dir and the user
> fetches the source tree in workspace using devtool modify, then this patch
> series creates a copy of source from workspace to work-shared. This is
> necessary for packages that may use the kernel source.

Looking over the patches again, we're close but the following need to be corrected:

1) The devtool.DevtoolUpdateTests.test_devtool_update_recipe_local_files test in oe-selftest fails:

---------------- snip ----------------
INFO: Adding local source files to srctree...
Traceback (most recent call last):
  File "/data/poky/scripts/devtool", line 334, in <module>
    ret = main()
  File "/data/poky/scripts/devtool", line 321, in main
    ret = args.func(args, config, basepath, workspace)
  File "/data/poky/scripts/lib/devtool/standard.py", line 845, in modify
    initial_rev, _ = _extract_source(srctree, args.keep_temp, args.branch, False, config, basepath, workspace, args.fixed_setup, rd, tinfoil, no_overrides=args.no_overrides)
  File "/data/poky/scripts/lib/devtool/standard.py", line 657, in _extract_source
    symblink_oelocal_files_srctree(d,srctree)
  File "/data/poky/scripts/lib/devtool/standard.py", line 485, in symblink_oelocal_files_srctree
    oe.patch.GitApplyTree.gitCommandUserOptions(useroptions, d=d)
NameError: name 'oe' is not defined
---------------- snip ----------------

Probably just a missing import.

2) Please change "symblink" to "symlink"

3) Indentation is very inconsistent in the added code. Please use four spaces everywhere.

I would also like to see oe-selftest tests covering this functionality, but I won't require them to ack these patches (especially given the delays on my part).

Thanks
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: [master][PATCH v4 0/3] Devtool: provide easy means of

[Paul Eggleton]

Hi Chandana

Having added quite a bit to devtool relating to the kernel, we really ought to extend our oe-selftest tests to cover the new functionality. Would you be able to take care of that?

Thanks
Paul

On Wednesday, 5 June 2019 5:07:55 PM NZDT Paul Eggleton wrote:
> Hi Chandana
> 
> My apologies for the delay in reviewing this - other work got in the way of OE-Core contributions but that should be dealt with now.
> 
> On Thursday, 18 April 2019 10:42:56 AM NZST Sai Hari Chandana Kalluri wrote:
> > This patch series provides support for the user to run menuconfig command in the
> > devtool flow. This would allow the user to modify the current configurations and
> > generate a config fragment to update the recipe using devtool finish. Devtool
> > menuconfig command will work on all packages that contain menuconfig as a task.
> > 
> > 1. The implementation checks if devtool menuconfig command is called for a valid
> > package.
> > 2. It checks for oe-local-files dir within source and creates one if
> > needed, this directory is needed to store the final generated config fragment so
> > that devtool finish can update the recipe.
> > 3. Menuconfig command is called for users to make necessary changes. After
> > saving the changes, diffconfig command is run to generate the fragment.
> > 
> > Currently, when the user runs devtool modify command, it checks out the entire
> > source tree which is a bit of an over head in time and space. This patch series
> > also provides a way to create a copy(hard links) of the kernel source, if
> > present, from work-shared to workspace to be more efficient .
> > 
> > Also, if the kernel source is not present in the staging kernel dir and the user
> > fetches the source tree in workspace using devtool modify, then this patch
> > series creates a copy of source from workspace to work-shared. This is
> > necessary for packages that may use the kernel source.
> 
> Looking over the patches again, we're close but the following need to be corrected:
> 
> 1) The devtool.DevtoolUpdateTests.test_devtool_update_recipe_local_files test in oe-selftest fails:
> 
> ---------------- snip ----------------
> INFO: Adding local source files to srctree...
> Traceback (most recent call last):
>   File "/data/poky/scripts/devtool", line 334, in <module>
>     ret = main()
>   File "/data/poky/scripts/devtool", line 321, in main
>     ret = args.func(args, config, basepath, workspace)
>   File "/data/poky/scripts/lib/devtool/standard.py", line 845, in modify
>     initial_rev, _ = _extract_source(srctree, args.keep_temp, args.branch, False, config, basepath, workspace, args.fixed_setup, rd, tinfoil, no_overrides=args.no_overrides)
>   File "/data/poky/scripts/lib/devtool/standard.py", line 657, in _extract_source
>     symblink_oelocal_files_srctree(d,srctree)
>   File "/data/poky/scripts/lib/devtool/standard.py", line 485, in symblink_oelocal_files_srctree
>     oe.patch.GitApplyTree.gitCommandUserOptions(useroptions, d=d)
> NameError: name 'oe' is not defined
> ---------------- snip ----------------
> 
> Probably just a missing import.
> 
> 2) Please change "symblink" to "symlink"
> 
> 3) Indentation is very inconsistent in the added code. Please use four spaces everywhere.
> 
> I would also like to see oe-selftest tests covering this functionality, but I won't require them to ack these patches (especially given the delays on my part).
> 
> Thanks
> Paul
> 
> 


-- 

Paul Eggleton
Intel System Software Products

Re: [master][PATCH v4 0/3] Devtool: provide easy means of

[Chandana Kalluri]

Hello Paul

Yes, I am in process of sending oeself test patches for devtool reset --remove-source command as commented In the other patch. I will also add the oeselftest for devtool kernel and send them out. 

Thanks,
Chandana

> -----Original Message-----
> From: Paul Eggleton <paul.eggleton@linux.intel.com>
> Sent: Tuesday, October 29, 2019 3:52 PM
> To: Chandana Kalluri <ckalluri@xilinx.com>
> Cc: openembedded-core@lists.openembedded.org
> Subject: Re: [OE-core] [OE-Core][master][PATCH v4 0/3] Devtool: provide easy
> means of
> 
> Hi Chandana
> 
> Having added quite a bit to devtool relating to the kernel, we really ought to
> extend our oe-selftest tests to cover the new functionality. Would you be able
> to take care of that?
> 
> Thanks
> Paul
> 
> On Wednesday, 5 June 2019 5:07:55 PM NZDT Paul Eggleton wrote:
> > Hi Chandana
> >
> > My apologies for the delay in reviewing this - other work got in the way of
> OE-Core contributions but that should be dealt with now.
> >
> > On Thursday, 18 April 2019 10:42:56 AM NZST Sai Hari Chandana Kalluri
> wrote:
> > > This patch series provides support for the user to run menuconfig
> > > command in the devtool flow. This would allow the user to modify the
> > > current configurations and generate a config fragment to update the
> > > recipe using devtool finish. Devtool menuconfig command will work on all
> packages that contain menuconfig as a task.
> > >
> > > 1. The implementation checks if devtool menuconfig command is called
> > > for a valid package.
> > > 2. It checks for oe-local-files dir within source and creates one if
> > > needed, this directory is needed to store the final generated config
> > > fragment so that devtool finish can update the recipe.
> > > 3. Menuconfig command is called for users to make necessary changes.
> > > After saving the changes, diffconfig command is run to generate the
> fragment.
> > >
> > > Currently, when the user runs devtool modify command, it checks out
> > > the entire source tree which is a bit of an over head in time and
> > > space. This patch series also provides a way to create a copy(hard
> > > links) of the kernel source, if present, from work-shared to workspace to be
> more efficient .
> > >
> > > Also, if the kernel source is not present in the staging kernel dir
> > > and the user fetches the source tree in workspace using devtool
> > > modify, then this patch series creates a copy of source from
> > > workspace to work-shared. This is necessary for packages that may use the
> kernel source.
> >
> > Looking over the patches again, we're close but the following need to be
> corrected:
> >
> > 1) The devtool.DevtoolUpdateTests.test_devtool_update_recipe_local_files
> test in oe-selftest fails:
> >
> > ---------------- snip ----------------
> > INFO: Adding local source files to srctree...
> > Traceback (most recent call last):
> >   File "/data/poky/scripts/devtool", line 334, in <module>
> >     ret = main()
> >   File "/data/poky/scripts/devtool", line 321, in main
> >     ret = args.func(args, config, basepath, workspace)
> >   File "/data/poky/scripts/lib/devtool/standard.py", line 845, in modify
> >     initial_rev, _ = _extract_source(srctree, args.keep_temp, args.branch,
> False, config, basepath, workspace, args.fixed_setup, rd, tinfoil,
> no_overrides=args.no_overrides)
> >   File "/data/poky/scripts/lib/devtool/standard.py", line 657, in
> _extract_source
> >     symblink_oelocal_files_srctree(d,srctree)
> >   File "/data/poky/scripts/lib/devtool/standard.py", line 485, in
> symblink_oelocal_files_srctree
> >     oe.patch.GitApplyTree.gitCommandUserOptions(useroptions, d=d)
> > NameError: name 'oe' is not defined
> > ---------------- snip ----------------
> >
> > Probably just a missing import.
> >
> > 2) Please change "symblink" to "symlink"
> >
> > 3) Indentation is very inconsistent in the added code. Please use four spaces
> everywhere.
> >
> > I would also like to see oe-selftest tests covering this functionality, but I
> won't require them to ack these patches (especially given the delays on my
> part).
> >
> > Thanks
> > Paul
> >
> >
> 
> 
> --
> 
> Paul Eggleton
> Intel System Software Products
> 

Re: [master][PATCH v4 0/3] Devtool: provide easy means of

[Paul Eggleton]

Hi Chandana,

Thanks! FYI the reason I bring this up is that it appears there's a problem with linux-yocto and SRCREV_machine values:

  https://bugzilla.yoctoproject.org/show_bug.cgi?id=13614

Would you be able to look into that as well?

Thanks
Paul

On Wednesday, 30 October 2019 12:03:35 PM NZDT Chandana Kalluri wrote:
> Hello Paul
> 
> Yes, I am in process of sending oeself test patches for devtool reset --remove-source command as commented In the other patch. I will also add the oeselftest for devtool kernel and send them out. 
> 
> Thanks,
> Chandana
> 
> > -----Original Message-----
> > From: Paul Eggleton <paul.eggleton@linux.intel.com>
> > Sent: Tuesday, October 29, 2019 3:52 PM
> > To: Chandana Kalluri <ckalluri@xilinx.com>
> > Cc: openembedded-core@lists.openembedded.org
> > Subject: Re: [OE-core] [OE-Core][master][PATCH v4 0/3] Devtool: provide easy
> > means of
> > 
> > Hi Chandana
> > 
> > Having added quite a bit to devtool relating to the kernel, we really ought to
> > extend our oe-selftest tests to cover the new functionality. Would you be able
> > to take care of that?
> > 
> > Thanks
> > Paul
> > 
> > On Wednesday, 5 June 2019 5:07:55 PM NZDT Paul Eggleton wrote:
> > > Hi Chandana
> > >
> > > My apologies for the delay in reviewing this - other work got in the way of
> > OE-Core contributions but that should be dealt with now.
> > >
> > > On Thursday, 18 April 2019 10:42:56 AM NZST Sai Hari Chandana Kalluri
> > wrote:
> > > > This patch series provides support for the user to run menuconfig
> > > > command in the devtool flow. This would allow the user to modify the
> > > > current configurations and generate a config fragment to update the
> > > > recipe using devtool finish. Devtool menuconfig command will work on all
> > packages that contain menuconfig as a task.
> > > >
> > > > 1. The implementation checks if devtool menuconfig command is called
> > > > for a valid package.
> > > > 2. It checks for oe-local-files dir within source and creates one if
> > > > needed, this directory is needed to store the final generated config
> > > > fragment so that devtool finish can update the recipe.
> > > > 3. Menuconfig command is called for users to make necessary changes.
> > > > After saving the changes, diffconfig command is run to generate the
> > fragment.
> > > >
> > > > Currently, when the user runs devtool modify command, it checks out
> > > > the entire source tree which is a bit of an over head in time and
> > > > space. This patch series also provides a way to create a copy(hard
> > > > links) of the kernel source, if present, from work-shared to workspace to be
> > more efficient .
> > > >
> > > > Also, if the kernel source is not present in the staging kernel dir
> > > > and the user fetches the source tree in workspace using devtool
> > > > modify, then this patch series creates a copy of source from
> > > > workspace to work-shared. This is necessary for packages that may use the
> > kernel source.
> > >
> > > Looking over the patches again, we're close but the following need to be
> > corrected:
> > >
> > > 1) The devtool.DevtoolUpdateTests.test_devtool_update_recipe_local_files
> > test in oe-selftest fails:
> > >
> > > ---------------- snip ----------------
> > > INFO: Adding local source files to srctree...
> > > Traceback (most recent call last):
> > >   File "/data/poky/scripts/devtool", line 334, in <module>
> > >     ret = main()
> > >   File "/data/poky/scripts/devtool", line 321, in main
> > >     ret = args.func(args, config, basepath, workspace)
> > >   File "/data/poky/scripts/lib/devtool/standard.py", line 845, in modify
> > >     initial_rev, _ = _extract_source(srctree, args.keep_temp, args.branch,
> > False, config, basepath, workspace, args.fixed_setup, rd, tinfoil,
> > no_overrides=args.no_overrides)
> > >   File "/data/poky/scripts/lib/devtool/standard.py", line 657, in
> > _extract_source
> > >     symblink_oelocal_files_srctree(d,srctree)
> > >   File "/data/poky/scripts/lib/devtool/standard.py", line 485, in
> > symblink_oelocal_files_srctree
> > >     oe.patch.GitApplyTree.gitCommandUserOptions(useroptions, d=d)
> > > NameError: name 'oe' is not defined
> > > ---------------- snip ----------------
> > >
> > > Probably just a missing import.
> > >
> > > 2) Please change "symblink" to "symlink"
> > >
> > > 3) Indentation is very inconsistent in the added code. Please use four spaces
> > everywhere.
> > >
> > > I would also like to see oe-selftest tests covering this functionality, but I
> > won't require them to ack these patches (especially given the delays on my
> > part).
> > >
> > > Thanks
> > > Paul
> > >
> > >
> > 
> > 
> > --
> > 
> > Paul Eggleton
> > Intel System Software Products
> > 
> 
> 


-- 

Paul Eggleton
Intel System Software Products