[PATCH 00/12] Include binaries in test cases and update runexported.py

[PATCH 00/12] Include binaries in test cases and update runexported.py

[Costin Constantin]

The set of patches addresses Yocto binararies inclusion related features and
updates to the runexported.py test launcher script for machines where poky
environment is not available.
As a result runtime test cases can be written to include native and target
specific binaries, both packed as .rpm files or pre-unpacked for the situation
where the remote target has no package manager. runexported.py includes new
functionality.

For all the above, enhancements are described in:
[YOCTO #7850], [YOCTO #8478], [YOCTO #8479], [YOCTO #8481], [YOCTO #8530]
[YOCTO #8534], [YOCTO #8535], [YOCTO #8536]

The following changes since commit f1f3716776078d68bd9e3734bca881a486dc2ea3:

  meta: more removals of redunant FILES_${PN}-dbg (2015-12-16 12:12:18 +0000)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib ciorga/CI2-YOCTO7850
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ciorga/CI2-YOCTO7850

Costin Constantin (4):
  oeqa/utils/testexport.py: add functionality for exporting binaries
  classes/testimage.bbclass: add support for binaries export
  utils/decorators.py: add TestNeedsBin() decorator
  buildtools-with-tc.bb: extend buildtools-tarball to include test cases

Lucian Musat (8):
  oeqa/runexported: Add option to run arbitrary tests.
  oeqa/runexported: Fix a problem with ssh_target_log symlink.
  oeqa/testimage: Added support for folder names in TEST_SUITES.
  oeqa/runexported: Added support for folder names in TEST_SUITES.
  oeqa/runexported: The runner supports tests from other layers.
  oeqa/runexported: Add parameter support for machine arch.
  oeqa/testimage: Added export features.
  oeqa/runtime: Copy all __init__.py files from all layers.

 meta/classes/testimage.bbclass               | 257 ++++++++++++++++++++++++--
 meta/lib/oeqa/runexported.py                 |  57 +++++-
 meta/lib/oeqa/utils/decorators.py            |  48 +++++
 meta/lib/oeqa/utils/testexport.py            | 263 +++++++++++++++++++++++++++
 meta/recipes-core/meta/buildtools-with-tc.bb |  37 ++++
 5 files changed, 641 insertions(+), 21 deletions(-)
 create mode 100644 meta/lib/oeqa/utils/testexport.py
 create mode 100644 meta/recipes-core/meta/buildtools-with-tc.bb

-- 
2.5.0

[PATCH 01/12] oeqa/utils/testexport.py: add functionality for exporting binaries

[Costin Constantin]

This new file is encapsulating functionality for both
running tests with binaries support via TestNeedsBin() decorator
and exporting these binaries via testimage.bbclass file.
Addresses [YOCTO #7850], [YOCTO #8478], [YOCTO #8481],
[YOCTO #8536], [YOCTO #8694].

Signed-off-by: Costin Constantin <costin.c.constantin@intel.com>
---
 meta/lib/oeqa/utils/testexport.py | 263 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 263 insertions(+)
 create mode 100644 meta/lib/oeqa/utils/testexport.py

diff --git a/meta/lib/oeqa/utils/testexport.py b/meta/lib/oeqa/utils/testexport.py
new file mode 100644
index 0000000..92b460d
--- /dev/null
+++ b/meta/lib/oeqa/utils/testexport.py
@@ -0,0 +1,263 @@
+# Copyright (C) 2015 Intel Corporation
+#
+# Released under the MIT license (see COPYING.MIT)
+
+# Provides functions to help with exporting binaries obtained from built targets
+
+import os, re, glob as g, shutil as sh,sys
+from time import sleep
+from commands import runCmd
+from difflib import SequenceMatcher as SM
+
+try:
+    import bb
+except ImportError:
+    class my_log():
+        def __init__(self):
+            pass
+        def plain(self, msg):
+            if msg:
+                print msg
+        def warn(self, msg):
+            if msg:
+                print "WARNING: " + msg
+        def fatal(self, msg):
+            if msg:
+                print "FATAL:" + msg
+                sys.exit(1)
+    bb = my_log()    
+
+
+def determine_if_poky_env():
+    """
+    used to determine if we are inside the poky env or not. Usefull for remote machine where poky is not present
+    """
+    check_env = True if ("/scripts" and "/bitbake/bin") in os.getenv("PATH") else False
+    return check_env
+
+
+def get_dest_folder(tune_features, folder_list):
+    """
+    Function to determine what rpm deploy dir to choose for a given architecture based on TUNE_FEATURES
+    """
+    features_list = tune_features.split(" ")
+    features_list.reverse()
+    features_list = "_".join(features_list)
+    match_rate = 0
+    best_match = None
+    for folder in folder_list:
+        curr_match_rate = SM(None, folder, features_list).ratio()
+        if curr_match_rate > match_rate:
+            match_rate = curr_match_rate
+            best_match = folder
+    return best_match
+
+
+def process_binaries(d, params):
+    param_list = params.split("_")
+    export_env = d.getVar("TEST_EXPORT_ONLY")
+
+    def extract_binary(pth_to_pkg, dest_pth=None):
+        cpio_command = runCmd("which cpio")
+        rpm2cpio_command = runCmd("ls /usr/bin/rpm2cpio")
+        if (cpio_command.status != 0) and (rpm2cpio_command.status != 0):
+            bb.fatal("Either \"rpm2cpio\" or \"cpio\" tools are not available on your system."
+                    "All binaries extraction processes will not be available, crashing all related tests."
+                    "Please install them according to your OS recommendations") # will exit here
+        if dest_pth:
+            os.chdir(dest_pth)
+        else:
+            os.chdir("%s" % os.sep)# this is for native package
+        extract_bin_command = runCmd("%s %s | %s -idm" % (rpm2cpio_command.output, pth_to_pkg, cpio_command.output)) # semi-hardcoded because of a bug on poky's rpm2cpio
+        return extract_bin_command
+
+    if determine_if_poky_env(): # machine with poky environment
+        exportpath = d.getVar("TEST_EXPORT_DIR", True) if export_env else d.getVar("DEPLOY_DIR", True)
+        rpm_deploy_dir = d.getVar("DEPLOY_DIR_RPM", True)
+        arch = get_dest_folder(d.getVar("TUNE_FEATURES", True), os.listdir(rpm_deploy_dir))
+        arch_rpm_dir = os.path.join(rpm_deploy_dir, arch)
+        extracted_bin_dir = os.path.join(exportpath,"binaries", arch, "extracted_binaries")
+        packaged_bin_dir = os.path.join(exportpath,"binaries", arch, "packaged_binaries")
+        # creating necessary directory structure in case testing is done in poky env.
+        if export_env == "0":
+            if not os.path.exists(extracted_bin_dir): bb.utils.mkdirhier(extracted_bin_dir)
+            if not os.path.exists(packaged_bin_dir): bb.utils.mkdirhier(packaged_bin_dir)
+
+        if param_list[3] == "native":
+            if export_env == "1": #this is a native package and we only need to copy it. no need for extraction
+                native_rpm_dir = os.path.join(rpm_deploy_dir, get_dest_folder("{} nativesdk".format(d.getVar("BUILD_SYS")), os.listdir(rpm_deploy_dir)))
+                native_rpm_file_list = [item for item in os.listdir(native_rpm_dir) if re.search("nativesdk-" + param_list[0] + "-([0-9]+\.*)", item)]
+                if not native_rpm_file_list:
+                    bb.warn("Couldn't find any version of {} native package. Related tests will most probably fail.".format(param_list[0]))
+                    return ""
+                for item in native_rpm_file_list:# will copy all versions of package. Used version will be selected on remote machine
+                    bb.plain("Copying native package file: %s" % item)
+                    sh.copy(os.path.join(rpm_deploy_dir, native_rpm_dir, item), os.path.join(d.getVar("TEST_EXPORT_DIR", True), "binaries", "native"))
+            else: # nothing to do here; running tests under bitbake, so we asume native binaries are in sysroots dir.
+                if param_list[1] or param_list[4]:
+                    bb.warn("Native binary %s %s%s. Running tests under bitbake environment. Version can't be checked except when the test itself does it"
+                            " and binary can't be removed."%(param_list[0],"has assigned ver. " + param_list[1] if param_list[1] else "",
+                            ", is marked for removal" if param_list[4] else ""))
+        else:# the package is target aka DUT intended and it is either required to be delivered in an extracted form or in a packaged version
+            target_rpm_file_list = [item for item in os.listdir(arch_rpm_dir) if re.search(param_list[0] + "-([0-9]+\.*)", item)]
+            if not target_rpm_file_list:
+                bb.warn("Couldn't find any version of target package %s. Please ensure it was built. "
+                        "Related tests will probably fail." % param_list[0])
+                return ""
+            if param_list[2] == "rpm": # binary should be deployed as rpm; (other, .deb, .ipk? ;  in the near future)
+                for item in target_rpm_file_list: # copying all related rpm packages. "Intuition" reasons, someone may need other versions too. Deciding later on version
+                    bb.plain("Copying target specific packaged file: %s" % item)
+                    sh.copy(os.path.join(arch_rpm_dir, item), packaged_bin_dir)
+                    return "copied"
+            else: # it is required to extract the binary
+                if param_list[1]: # the package is versioned
+                    for item in target_rpm_file_list:
+                        if re.match(".*-{}-.*\.rpm".format(param_list[1]), item):
+                            destination = os.path.join(extracted_bin_dir,param_list[0], param_list[1])
+                            bb.utils.mkdirhier(destination)
+                            extract_binary(os.path.join(arch_rpm_dir, item), destination)
+                            break
+                    else:
+                        bb.warn("Couldn't find the desired version %s for target binary %s. Related test cases will probably fail." % (param_list[1], param_list[0]))
+                        return ""
+                    return "extracted"
+                else: # no version provided, just extract one binary
+                    destination = os.path.join(extracted_bin_dir,param_list[0],
+                                               re.search(".*-([0-9]+\.[0-9]+)-.*rpm", target_rpm_file_list[0]).group(1))
+                    bb.utils.mkdirhier(destination)
+                    extract_binary(os.path.join(arch_rpm_dir, target_rpm_file_list[0]), destination)
+                    return "extracted"
+    else: # remote machine
+        binaries_path = os.getenv("bin_dir")# in order to know where the binaries are, bin_dir is set as env. variable
+        if param_list[3] == "native": #need to extract the native pkg here
+            native_rpm_dir = os.path.join(binaries_path, "native")
+            native_rpm_file_list = os.listdir(native_rpm_dir)
+            for item in native_rpm_file_list:
+                if param_list[1] and re.match("nativesdk-{}-{}-.*\.rpm".format(param_list[0], param_list[1]), item): # native package has version
+                    extract_binary(os.path.join(native_rpm_dir, item))
+                    break
+                else:# just copy any related native binary
+                    found_version = re.match("nativesdk-{}-([0-9]+\.[0-9]+)-".format(param_list[0]), item).group(1)
+                    if found_version:
+                        extract_binary(os.path.join(native_rpm_dir, item))
+            else:
+                bb.warn("Couldn't find native package %s%s. Related test cases will be influenced." %
+                        (param_list[0], " with version " + param_list[1] if param_list[1] else ""))
+                return
+
+        else: # this is for target device
+            if param_list[2] == "rpm":
+                return "No need to extract, this is an .rpm file"
+            arch = get_dest_folder(d.getVar("TUNE_FEATURES", True), os.listdir(binaries_path))
+            extracted_bin_path = os.path.join(binaries_path, arch, "extracted_binaries")
+            extracted_bin_list = [item for item in os.listdir(extracted_bin_path)]
+            packaged_bin_path = os.path.join(binaries_path, arch, "packaged_binaries")
+            packaged_bin_file_list = os.listdir(packaged_bin_path)
+            # see if the package is already in the extracted ones; maybe it was deployed when exported the env.
+            if os.path.exists(os.path.join(extracted_bin_path, param_list[0], param_list[1] if param_list[1] else "")):
+                return "binary %s is already extracted" % param_list[0]
+            else: # we need to search for it in the packaged binaries directory. It may have been shipped after export
+                for item in packaged_bin_file_list:
+                    if param_list[1]:
+                        if re.match("%s-%s.*rpm" % (param_list[0], param_list[1]), item): # package with version
+                            if not os.path.exists(os.path.join(extracted_bin_path, param_list[0],param_list[1])):
+                                os.makedirs(os.path.join(extracted_bin_path, param_list[0], param_list[1]))
+                                extract_binary(os.path.join(packaged_bin_path, item), os.path.join(extracted_bin_path, param_list[0],param_list[1]))
+                                bb.plain("Using {} for {}".format(os.path.join(packaged_bin_path, item), param_list[0]))
+                            break
+                    else:
+                        if re.match("%s-.*rpm" % param_list[0], item):
+                            found_version = re.match(".*-([0-9]+\.[0-9]+)-", item).group(1)
+                            if not os.path.exists(os.path.join(extracted_bin_path, param_list[0], found_version)):
+                                os.makedirs(os.path.join(extracted_bin_path, param_list[0], found_version))
+                                bb.plain("Used ver. %s for %s" % (found_version, param_list[0]))
+                                extract_binary(os.path.join(packaged_bin_path, item), os.path.join(extracted_bin_path, param_list[0], found_version))
+                            break
+                else:
+                    bb.warn("Couldn't find target package %s%s. Please ensure it is available "
+                            "in either of these directories: extracted_binaries or packaged_binaries. "
+                            "Related tests will probably fail." % (param_list[0], " with version " + param_list[1] if param_list[1] else ""))
+                    return
+                return "Binary %s extracted successfully." % param_list[0]
+
+
+def files_to_copy(base_dir):
+    """
+    Produces a list of files relative to the base dir path sent as param
+    :return: the list of relative path files
+    """
+    files_list = []
+    dir_list = [base_dir]
+    count = 1
+    dir_count = 1
+    while (dir_count == 1 or dir_count != count):
+        count = dir_count
+        for dir in dir_list:
+            for item in os.listdir(dir):
+                if os.path.isdir(os.path.join(dir, item)) and os.path.join(dir, item) not in dir_list:
+                   dir_list.append(os.path.join(dir, item))
+                   dir_count = len(dir_list)
+                elif os.path.join(dir, item) not in files_list and os.path.isfile(os.path.join(dir, item)):
+                   files_list.append(os.path.join(dir, item))
+    return files_list
+
+
+def send_bin_to_DUT(d,params):
+    from oeqa.oetest import oeRuntimeTest
+    param_list = params.split("_")
+    cleanup_list = list()
+    bins_dir = os.path.join(d.getVar("TEST_EXPORT_DIR", True), "binaries") if determine_if_poky_env() \
+                    else os.getenv("bin_dir")
+    arch = get_dest_folder(d.getVar("TUNE_FEATURES", True), os.listdir(bins_dir))
+    arch_rpms_dir = os.path.join(bins_dir, arch, "packaged_binaries")
+    extracted_bin_dir = os.path.join(bins_dir, arch, "extracted_binaries", param_list[0])
+
+    def send_extracted_binary():
+        bin_local_dir = os.path.join(extracted_bin_dir, param_list[1] if param_list[1] else os.listdir(extracted_bin_dir)[0])
+        for item in files_to_copy(bin_local_dir):
+            split_path = item.split(bin_local_dir)[1]
+            path_on_DUT = split_path if split_path[0] is "/" else "/" + split_path # create the path as on DUT; eg. /usr/bin/bin_file
+            (status, output) = oeRuntimeTest.tc.target.copy_to(item, path_on_DUT)
+            if status != 0:
+                bb.warn("Failed to copy %s binary file %s on the remote target: %s" %
+                        (param_list[0], "ver. " + param_list[1] if param_list[1] else "", d.getVar("MACHINE")))
+                return
+            if param_list[4] == "rm":
+                cleanup_list.append(path_on_DUT)
+        return cleanup_list
+
+    def send_rpm(remote_path): # if it is not required to have an extracted binary, but to send an .rpm file
+        rpm_to_send = ""
+        for item in os.listdir(arch_rpms_dir):
+            if param_list[1] and re.match("%s-%s-.*rpm"%(param_list[0], param_list[1]), item):
+                rpm_to_send = item
+                break
+            elif re.match("%s-[0-9]+\.[0-9]+-.*rpm" % param_list[0], item):
+                rpm_to_send = item
+                break
+        else:
+            bb.warn("No rpm package found for %s %s in .rpm files dir %s. Skipping deployment." %
+                    (param_list[0], "ver. " + param_list[1] if param_list[1] else "", rpms_file_dir) )
+            return
+        (status, output) = oeRuntimeTest.tc.target.copy_to(os.path.join(arch_rpms_dir, rpm_to_send), remote_path)
+        if status != 0:
+                bb.warn("Failed to copy %s on the remote target: %s" %(param_list[0], d.getVar("MACHINE")))
+                return
+        if param_list[4] == "rm":
+            cleanup_list.append(os.path.join(remote_path, rpm_to_send))
+            return cleanup_list
+
+    if param_list[2] == "rpm": # send an .rpm file
+        return send_rpm("/home/root") # rpms will be sent on home dir of remote machine
+    else:
+        return send_extracted_binary()
+
+
+def rm_bin(removal_list): # need to know both if the binary is sent archived and the path where it is sent if archived
+    from oeqa.oetest import oeRuntimeTest
+    for item in removal_list:
+        (status,output) = oeRuntimeTest.tc.target.run("rm " + item)
+        if status != 0:
+            bb.warn("Failed to remove: %s. Please ensure connection with the target device is up and running and "
+                     "you have the needed rights." % item)
+
-- 
2.5.0

[PATCH 02/12] classes/testimage.bbclass: add support for binaries export

[Costin Constantin]

Some test cases require support for target and native binaries.
This enhancement is designed to help exporting testing environment
for machines where poky/bitbake environment is not available.
At the same time, tarball files are created that encapsulate
native, target specific and runner related files separatelly.
This helps deployment of separate functionality files in
their own tarballs.

Partial fix for [YOCTO #7850].

Signed-off-by: Costin Constantin <costin.c.constantin@intel.com>
---
 meta/classes/testimage.bbclass | 126 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 126 insertions(+)

diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index e833db4..bcf113f 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -263,7 +263,133 @@ def exportTests(d,tc):
         for f in files:
             shutil.copy2(os.path.join(root, f), os.path.join(exportpath, "oeqa/runtime/files"))
 
+    #integrating binaries too
+    # creting needed directory structure
+    arch = te.get_dest_folder(d.getVar("TUNE_FEATURES", True), os.listdir(d.getVar("DEPLOY_DIR_RPM", True)))
+    bb.utils.mkdirhier(os.path.join(exportpath,"tar_files"))
+    bb.utils.mkdirhier(os.path.join(exportpath,"binaries", arch, "packaged_binaries"))
+    bb.utils.mkdirhier(os.path.join(exportpath,"binaries", "native"))
+    bb.utils.mkdirhier(os.path.join(exportpath,"binaries", arch, "extracted_binaries"))
+    with open(os.path.join(exportpath,"binaries",arch, "mapping.cfg"), "a") as mapping_file:
+        #prepare the needed info for the mapping file
+        mapping_params = dict()
+        mapping_params[d.getVar("MACHINE",True)] = (d.getVar("TARGET_SYS",True), d.getVar("TUNE_FEATURES",True))
+        mapping_file.write(mapping_params.__str__())
+
+    populate_binaries(d)
+
+    # create the "runner" tar file, but before that erase them if created in a previous run
+    [runner_tar, bin_tar, native_tar] = [os.path.join(exportpath,"tar_files", item) for item in ("runner_files.tar", "{}_binaries.tar".format(arch), "native_binaries.tar")]
+    for item in os.listdir(exportpath): # create the runner tarball
+        if item not in ('binaries','tar_files'):
+            create_tar(runner_tar, os.path.join(exportpath,item), item.replace(exportpath, ""))
+    #create the binaries tar file
+    for item in os.listdir(os.path.join(exportpath, "binaries")):
+        if re.search("native", item): # creating native binaries tarfile
+            create_tar(os.path.join(exportpath,"tar_files", os.path.basename(native_tar)), os.path.join(exportpath,"binaries", "native"), os.path.join("binaries", item.replace(exportpath, "")))
+        else: # creating target binaries tar file
+            create_tar(os.path.join(exportpath,"tar_files",os.path.basename(bin_tar)), os.path.join(exportpath,"binaries", arch), os.path.join("binaries", item.replace(exportpath, "")))
     bb.plain("Exported tests to: %s" % exportpath)
+    for item in os.listdir(exportpath):
+        if item == "tar_files":
+            bb.plain("Exported tarballs to: {}".format(exportpath + "/tar_files"))
+
+def create_tar(tar_fl,file_to_add, a_name=None):
+    import tarfile as tar
+    with tar.open(tar_fl,"a") as tar_file:
+        tar_file.add(file_to_add, arcname=a_name) 
+
+def obtain_binaries(d, param):
+    """
+    this func. will extract binaries based on their version (if required)
+    """
+    import oeqa.utils.testexport as te
+    te.process_binaries(d, param) # for native packages, this is not extracting binaries, but only copies them
+                                  # for a local machine with poky on it the user should build native binaries
+
+def read_test_files(file_pth):
+    """
+    The only way to read what binaries are to be included in the tarball file(s) is to search for each
+    test file (.py) and see if the decorator TestNeedsBin is present. If yes, just extract the information
+    from there. The lib/oeqa/runtime dir is taken as default to read files from. The result is a list 
+    containing needed information like binary name, binary, version etc. in a string of values separated 
+    by "_". This aproach was taken because it helps when TestNeedsBin contains multiple binaries and some
+    are with identical name and different version or when same binary is needed both in extracted and 
+    packaged mode. A dictionary would have eliminated the duplicate same key (in our case binary name) 
+    fields.
+    """
+    import re
+    bin_ver = list()
+    f_to_list = list()
+    #the actual reading part of the file:
+    with open(file_pth, 'r') as fl:
+        f_to_list = list(fl)
+    def process_found_tuple(list_of_tuples):
+        for params in eval("{}".format(list_of_tuples)): # in our case, value is a list of values
+            ver = ""
+            packaged = ""
+            t = ""
+            for item in eval("{}".format(params)):
+                if re.match("[0-9]+\.",item):
+                    ver = item
+                elif item == "rpm":
+                    packaged = "rpm"
+                elif item == "native":
+                    t = item
+            bin_ver.append("{}_{}_{}_{}".format(params[0],ver,packaged,t))
+
+    def search_for_tuples_of_params(i):
+        read_tuple = ""
+        ind = i
+        if re.search('\(\(', f_to_list[ind]) and not f_to_list[ind].startswith("#"): # the line is sure to be a valid one, not a comment
+            read_tuple += f_to_list[ind].strip()
+            while True:
+                if re.search('\)\)', read_tuple):
+                    read_tuple = read_tuple.replace("@TestNeedsBin","")#read_tuple[read_tuple.find('((')+1:read_tuple.find('))')+1]
+                    break
+                else:
+                    ind += 1
+                    read_tuple += f_to_list[ind].strip()
+            process_found_tuple(read_tuple)
+            return ind
+        return
+
+    # parsing all found lists
+    for index,line in enumerate(f_to_list): 
+        line=line.strip()
+        if re.search("@TestNeedsBin", line) and not line.startswith("#") and not (re.search('\(\(', line) or re.search('\)\)', line)):
+            packaged = ""
+            version = ""
+            t = ""
+            for index, value in enumerate(eval(line.replace("@TestNeedsBin", ""))):
+                if index == 0:
+                    bin_name = value
+                elif index == 1 and re.match("[0-9]+\.",value):
+                    version = value
+                elif value == "rpm":
+                    packaged = value
+                elif value == "native":
+                    t = value
+            bin_ver.append("{}_{}_{}_{}".format(bin_name,version,packaged,t))
+        elif re.search("@TestNeedsBin", line) and not line.startswith("#") and (re.search('\(\(', line) or re.search('\)\)', line)):
+            search_for_tuples_of_params(index)
+
+    return bin_ver
+
+def populate_binaries(d):
+    default_pth_to_files = os.path.join(d.getVar("COREBASE",True), "meta/lib/oeqa/runtime")
+    pth_to_file = list()
+    pth_to_file.append(default_pth_to_files)
+    bin_with_version = list()
+    bbpath = d.getVar("BBPATH", True).split(':')
+    testlist = get_tests_list(d)
+    for pth in bbpath:
+        for item in testlist:
+            test_file_pth = os.path.join(pth, "lib", item.replace(".", os.sep) + ".py")
+            if os.path.exists(test_file_pth):
+                bin_with_version.extend(read_test_files(test_file_pth))
+    for item in set(bin_with_version):
+        obtain_binaries(d,item)
 
 
 def testimage_main(d):
-- 
2.5.0

[PATCH 03/12] utils/decorators.py: add TestNeedsBin() decorator

[Costin Constantin]

TestNeedsBin() is used in all tests where binaries are required.
It offers functionality to send binaries to DUTs
both in unpacked (raw)  and in an .rpm form.
It is able to handle native binaries, intended for the machine
launching tests on DUTs.
It also offers functionality for removing DUTs
related binaries after finishing the test.

Signed-off-by: Costin Constantin <costin.c.constantin@intel.com>
---
 meta/lib/oeqa/utils/decorators.py | 48 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)

diff --git a/meta/lib/oeqa/utils/decorators.py b/meta/lib/oeqa/utils/decorators.py
index 0d79223..93a728e 100644
--- a/meta/lib/oeqa/utils/decorators.py
+++ b/meta/lib/oeqa/utils/decorators.py
@@ -12,7 +12,9 @@ import sys
 import unittest
 import threading
 import signal
+import re
 from functools import wraps
+import testexport as te
 
 #get the "result" object from one of the upper frames provided that one of these upper frames is a unittest.case frame
 class getResults(object):
@@ -260,3 +262,49 @@ def timeout_handler(seconds):
         else:
             return fn
     return decorator
+
+class TestNeedsBin(object):
+    """
+    This decorator provides binary support for test cases
+    """
+    def __init__(self, *args):
+        self.params_list = list()
+        self. clean_list = list()
+        if args:# these are tuples of values
+            for param_tuple in args:
+                bn,bv,t, p,rm_b = ("", "", "", "", "")
+                for index,value in enumerate(param_tuple):
+                    if index == 0:
+                        bn = value
+                    elif index == 1 and re.match("[0-9]+\.",value):
+                        bv = value
+                    elif value == "rpm":
+                        p = value
+                    elif value == "native":
+                        t = value
+                    elif value == "rm":
+                        rm_b = value
+                self.params_list.append("%s_%s_%s_%s_%s" % (bn, bv, p, t, rm_b))
+
+    def deploy_binary(self, params):
+        from oeqa.oetest import oeRuntimeTest
+        p = params.split("_")
+        if p[3] == "native":
+            te.process_binaries(oeRuntimeTest.tc.d, params)
+        else:
+            status = te.process_binaries(oeRuntimeTest.tc.d, params)
+            if status:
+                bin_to_rm = te.send_bin_to_DUT(oeRuntimeTest.tc.d, params)
+                if bin_to_rm:
+                    self.clean_list.extend(bin_to_rm)
+                    
+    def __call__(self, func):
+        def wrapped_f(*args):
+            for item in set(self.params_list):
+                self.deploy_binary(item)
+            func(*args)
+            te.rm_bin(self.clean_list) # used to remove sent binaries 
+            return
+        wrapped_f.__name__ = func.__name__
+        return wrapped_f
+
-- 
2.5.0

[PATCH 04/12] oeqa/runexported: Add option to run arbitrary tests.

[Costin Constantin]

From: Lucian Musat <george.l.musat@intel.com>

You can now overwrite the default TEST_SUITES from the
json file and can choose the tests you want to run.
Also you can display the list of tests.

Signed-off-by: Lucian Musat <george.l.musat@intel.com>
---
 meta/lib/oeqa/runexported.py | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/meta/lib/oeqa/runexported.py b/meta/lib/oeqa/runexported.py
index dba0d7a..d273d2f 100755
--- a/meta/lib/oeqa/runexported.py
+++ b/meta/lib/oeqa/runexported.py
@@ -86,6 +86,8 @@ def main():
             specified in the json if that directory actually exists or it will error out.")
     parser.add_argument("-l", "--log-dir", dest="log_dir", help="This sets the path for TEST_LOG_DIR. If not specified \
             the current dir is used. This is used for usually creating a ssh log file and a scp test file.")
+    parser.add_argument("--list-tests", dest="list_tests", help="This lists the current TEST_SUITES that will be run.", action='store_true')
+    parser.add_argument("-r","--run-tests", dest="run_tests", help="Overwrite TEST_SUITES from the json file with custom list of tests.", nargs = '*')
     parser.add_argument("json", help="The json file exported by the build system", default="testdata.json", nargs='?')
 
     args = parser.parse_args()
@@ -129,8 +131,15 @@ def main():
         if key != "d" and key != "target" and key != "host_dumper":
             setattr(tc, key, loaded[key])
 
-    target.exportStart()
-    runTests(tc)
+    if args.run_tests:
+        tc.testslist = args.run_tests
+
+    if args.list_tests:
+        for test in tc.testslist:
+            print test
+    else:
+        target.exportStart()
+        runTests(tc)
 
     return 0
 
-- 
2.5.0

[PATCH 05/12] oeqa/runexported: Fix a problem with ssh_target_log symlink.

[Costin Constantin]

From: Lucian Musat <george.l.musat@intel.com>

Signed-off-by: Lucian Musat <george.l.musat@intel.com>
---
 meta/lib/oeqa/runexported.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/runexported.py b/meta/lib/oeqa/runexported.py
index d273d2f..f147089 100755
--- a/meta/lib/oeqa/runexported.py
+++ b/meta/lib/oeqa/runexported.py
@@ -49,7 +49,7 @@ class FakeTarget(object):
     def exportStart(self):
         self.sshlog = os.path.join(self.testdir, "ssh_target_log.%s" % self.datetime)
         sshloglink = os.path.join(self.testdir, "ssh_target_log")
-        if os.path.exists(sshloglink):
+        if os.path.lexists(sshloglink):
             os.remove(sshloglink)
         os.symlink(self.sshlog, sshloglink)
         print("SSH log file: %s" %  self.sshlog)
-- 
2.5.0

[PATCH 06/12] oeqa/testimage: Added support for folder names in TEST_SUITES.

[Costin Constantin]

From: Lucian Musat <george.l.musat@intel.com>

You can add oeqa.runtime.<foldername> and it will run all the
tests found in it. Basically extends the functionality of feature
[YOCTO #7834]

Signed-off-by: Lucian Musat <george.l.musat@intel.com>
---
 meta/classes/testimage.bbclass | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index bcf113f..40442b7 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -140,7 +140,16 @@ def get_tests_list(d, type="runtime"):
     for testname in testsuites:
         if testname != "auto":
             if testname.startswith("oeqa."):
-                testslist.append(testname)
+                for p in bbpath:
+                    test_location = os.path.join(p, 'lib', os.sep.join(testname.split('.')))
+                    if os.path.isfile(test_location+".py") or \
+                        (os.path.isfile(test_location.rsplit(os.sep, 1)[0]+".py")) or \
+                        (os.path.isfile(test_location.rsplit(os.sep, 2)[0]+".py")):
+                        testslist.append(testname)
+                    elif os.path.isdir(test_location):
+                        for files in os.listdir(test_location):
+                            if (files.endswith(".py")) and not files.startswith("_"):
+                                testslist.append(testname+'.'+files.split('.')[0])
                 continue
             found = False
             for p in bbpath:
-- 
2.5.0

[PATCH 07/12] oeqa/runexported: Added support for folder names in TEST_SUITES.

[Costin Constantin]

From: Lucian Musat <george.l.musat@intel.com>

It can accept parameters like oeqa.runtime.<foldername> and it
will run all test files from that folder.

Signed-off-by: Lucian Musat <george.l.musat@intel.com>
---
 meta/lib/oeqa/runexported.py | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/meta/lib/oeqa/runexported.py b/meta/lib/oeqa/runexported.py
index f147089..c3ce79a 100755
--- a/meta/lib/oeqa/runexported.py
+++ b/meta/lib/oeqa/runexported.py
@@ -138,6 +138,12 @@ def main():
         for test in tc.testslist:
             print test
     else:
+        for index, test in enumerate(tc.testslist):
+            if os.path.isdir(os.sep.join(test.split('.'))):
+                del tc.testslist[index]
+                for files in os.listdir(os.sep.join(test.split('.'))):
+                    if (files.endswith(".py")) and not files.startswith("_"):
+                        tc.testslist.insert(index, test+'.'+files.split('.')[0])
         target.exportStart()
         runTests(tc)
 
-- 
2.5.0

[PATCH 08/12] oeqa/runexported: The runner supports tests from other layers.

[Costin Constantin]

From: Lucian Musat <george.l.musat@intel.com>

It looks in "extralayers" folder if it can't find the test
that needs to be run.

Signed-off-by: Lucian Musat <george.l.musat@intel.com>
---
 meta/lib/oeqa/runexported.py | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/meta/lib/oeqa/runexported.py b/meta/lib/oeqa/runexported.py
index c3ce79a..b8d621c 100755
--- a/meta/lib/oeqa/runexported.py
+++ b/meta/lib/oeqa/runexported.py
@@ -92,6 +92,12 @@ def main():
 
     args = parser.parse_args()
 
+    if os.path.isdir("extralayers"):
+        extrapaths = [x[0] for x in os.walk('extralayers')]
+        for path in extrapaths:
+            if os.path.isdir(os.path.join(path,"oeqa")) and (os.path.join(path,"oeqa") not in sys.path):
+                sys.path.append(os.path.abspath(path))
+
     with open(args.json, "r") as f:
         loaded = json.load(f)
 
@@ -144,6 +150,14 @@ def main():
                 for files in os.listdir(os.sep.join(test.split('.'))):
                     if (files.endswith(".py")) and not files.startswith("_"):
                         tc.testslist.insert(index, test+'.'+files.split('.')[0])
+            elif not os.path.isfile(os.path.join(os.sep.join(test.split('.')), '.py')):
+                for testpath in sys.path:
+                    directory = os.path.join(testpath, os.sep.join(test.split('.')))
+                    if os.path.isdir(directory):
+                        del tc.testslist[index]
+                        for files in os.listdir(directory):
+                            if (files.endswith(".py")) and not files.startswith("_"):
+                                tc.testslist.insert(index, test+'.'+files.split('.')[0])
         target.exportStart()
         runTests(tc)
 
-- 
2.5.0

[PATCH 09/12] oeqa/runexported: Add parameter support for machine arch.

[Costin Constantin]

From: Lucian Musat <george.l.musat@intel.com>

Added support to change machine architecture on the fly and
support for future TestNeedsBin decorator for tests that need
binaries.

Signed-off-by: Lucian Musat <george.l.musat@intel.com>
Signed-off-by: Costin Constantin <costin.c.constantin@intel.com>
---
 meta/lib/oeqa/runexported.py | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/meta/lib/oeqa/runexported.py b/meta/lib/oeqa/runexported.py
index b8d621c..3a6bad7 100755
--- a/meta/lib/oeqa/runexported.py
+++ b/meta/lib/oeqa/runexported.py
@@ -22,6 +22,9 @@ import sys
 import os
 import time
 import argparse
+import re
+
+from oeqa.utils.testsexport import get_dest_folder
 
 try:
     import simplejson as json
@@ -88,10 +91,13 @@ def main():
             the current dir is used. This is used for usually creating a ssh log file and a scp test file.")
     parser.add_argument("--list-tests", dest="list_tests", help="This lists the current TEST_SUITES that will be run.", action='store_true')
     parser.add_argument("-r","--run-tests", dest="run_tests", help="Overwrite TEST_SUITES from the json file with custom list of tests.", nargs = '*')
+    parser.add_argument("-m","--machine", dest="machine", help="Overwrite MACHINE from the json file ")
     parser.add_argument("json", help="The json file exported by the build system", default="testdata.json", nargs='?')
 
     args = parser.parse_args()
 
+    os.environ["bin_dir"] = os.path.join(os.getcwd(), "binaries") # exporting bin_dir path
+
     if os.path.isdir("extralayers"):
         extrapaths = [x[0] for x in os.walk('extralayers')]
         for path in extrapaths:
@@ -137,6 +143,22 @@ def main():
         if key != "d" and key != "target" and key != "host_dumper":
             setattr(tc, key, loaded[key])
 
+    if args.machine:
+        d['MACHINE'] = args.machine
+        bin_dir = os.getenv("bin_dir")
+        found = False
+        if os.path.exists(bin_dir):
+            for item in os.listdir(bin_dir):
+                if not re.search("native", item) and os.path.exists(os.path.join(bin_dir, item, "mapping.cfg")):
+                    with open(os.path.join(bin_dir, item, "mapping.cfg"), "r") as mapping_file:
+                        mapping = eval(mapping_file.read())
+                        if args.machine in mapping.keys():
+                            d['TARGET_SYS'] = mapping[args.machine][0]
+                            d['TUNE_FEATURES'] = mapping[args.machine][1]
+                            found = True
+        if not found:
+            print("WARNING: Cannot find binaries for provided machine %s." % args.machine)
+
     if args.run_tests:
         tc.testslist = args.run_tests
 
-- 
2.5.0

[PATCH 10/12] oeqa/testimage: Added export features.

[Costin Constantin]

From: Lucian Musat <george.l.musat@intel.com>

It is now possible to export tests from all the layers that are
added in bblayers even when grouped in folders. They are exported
in another folder called "extralayers" and nicely grouped per layer
so the test files won't mingle.
If a layer contains a conf/test folder then export that as well.

Signed-off-by: Lucian Musat <george.l.musat@intel.com>
---
 meta/classes/testimage.bbclass | 113 ++++++++++++++++++++++++++++++++++-------
 1 file changed, 96 insertions(+), 17 deletions(-)

diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index 40442b7..88aa941 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -161,6 +161,11 @@ def get_tests_list(d, type="runtime"):
                     testslist.append("oeqa." + type + "." + testname)
                     found = True
                     break
+                elif os.path.exists(os.path.join(p, 'lib', 'oeqa', type, testname.split(".")[0]))\
+                    and os.path.isdir(os.path.join(p, 'lib', 'oeqa', type, testname.split(".")[0])):
+                    testslist.append("oeqa." + type + "." + testname)
+                    found = True
+                    break
             if not found:
                 bb.fatal('Test %s specified in TEST_SUITES could not be found in lib/oeqa/runtime under BBPATH' % testname)
 
@@ -182,6 +187,24 @@ def get_tests_list(d, type="runtime"):
 
     return testslist
 
+def get_extra_layers(d):
+    default_layers = ['meta-yocto','meta-yocto-bsp']
+    extra_layers = []
+
+    layer_list = [var for var in d.getVar("BBLAYERS", True).split(" ") if var]
+
+    for layer in layer_list:
+        if (os.path.basename(os.path.normpath(layer)) not in default_layers)\
+        and os.path.exists(os.path.join(layer,"lib","oeqa")):
+            extra_layers.append(layer)
+    return extra_layers
+
+def get_layer(fullpath):
+    try:
+        layer = os.path.basename(fullpath.split(os.path.join("lib","oeqa"))[0].rstrip(os.sep))
+    except IndexError:
+        layer = None
+    return layer
 
 def exportTests(d,tc):
     import json
@@ -237,6 +260,13 @@ def exportTests(d,tc):
     #   - __init__.py files
     bb.utils.mkdirhier(os.path.join(exportpath, "oeqa/runtime/files"))
     bb.utils.mkdirhier(os.path.join(exportpath, "oeqa/utils"))
+    if len(get_extra_layers(d)) > 1:
+        bb.utils.mkdirhier(os.path.join(exportpath, "extralayers"))
+        for layer in get_extra_layers(d):
+            if os.path.basename(os.path.normpath(layer)) != "meta":
+                bb.utils.mkdirhier(os.path.join(exportpath, "extralayers/%s/oeqa/runtime/files" % os.path.basename(os.path.normpath(layer))))
+                bb.utils.mkdirhier(os.path.join(exportpath, "extralayers/%s/oeqa/utils" % os.path.basename(os.path.normpath(layer))))
+
     # copy test modules, this should cover tests in other layers too
     bbpath = d.getVar("BBPATH", True).split(':')
     for t in tc.testslist:
@@ -244,6 +274,18 @@ def exportTests(d,tc):
         if re.search("\w+\.\w+\.test_\S+", t):
             t = '.'.join(t.split('.')[:3])
         mod = pkgutil.get_loader(t)
+        if (str(os.path.join("meta","lib","oeqa")) not in mod.filename):
+            if get_layer(mod.filename):
+                layerpath = os.path.join(exportpath, "extralayers", get_layer(mod.filename))
+            else:
+                layerpath = exportpath
+        else:
+            layerpath = exportpath
+
+        if os.path.isdir(mod.filename):
+            isfolder = True
+        else:
+            shutil.copy2(mod.filename, os.path.join(layerpath, "oeqa/runtime"))
         # More depth than usual?
         if (t.count('.') > 2):
             for p in bbpath:
@@ -253,24 +295,61 @@ def exportTests(d,tc):
                     target_folder = os.path.join(exportpath, "oeqa", "runtime", os.path.basename(foldername))
                     if not os.path.exists(target_folder):
                         shutil.copytree(foldername, target_folder)
-        if not isfolder:
-            shutil.copy2(mod.filename, os.path.join(exportpath, "oeqa/runtime"))
+        if isfolder:
+            target_folder = os.path.join(layerpath, "oeqa", "runtime", os.path.basename(mod.filename))
+            if not os.path.exists(target_folder):
+                shutil.copytree(mod.filename, target_folder)
+
+         if not isfolder:
+            shutil.copy2(mod.filename, os.path.join(layerpath, "oeqa/runtime"))
     # copy __init__.py files
-    oeqadir = pkgutil.get_loader("oeqa").filename
-    shutil.copy2(os.path.join(oeqadir, "__init__.py"), os.path.join(exportpath, "oeqa"))
-    shutil.copy2(os.path.join(oeqadir, "runtime/__init__.py"), os.path.join(exportpath, "oeqa/runtime"))
-    # copy oeqa/oetest.py and oeqa/runexported.py
-    shutil.copy2(os.path.join(oeqadir, "oetest.py"), os.path.join(exportpath, "oeqa"))
-    shutil.copy2(os.path.join(oeqadir, "runexported.py"), exportpath)
-    # copy oeqa/utils/*.py
-    for root, dirs, files in os.walk(os.path.join(oeqadir, "utils")):
-        for f in files:
-            if f.endswith(".py"):
-                shutil.copy2(os.path.join(root, f), os.path.join(exportpath, "oeqa/utils"))
-    # copy oeqa/runtime/files/*
-    for root, dirs, files in os.walk(os.path.join(oeqadir, "runtime/files")):
-        for f in files:
-            shutil.copy2(os.path.join(root, f), os.path.join(exportpath, "oeqa/runtime/files"))
+    for _oeqadir in get_extra_layers(d):
+        oeqadir = os.path.join(_oeqadir,"lib","oeqa")
+        # copy oeqa/oetest.py and oeqa/runexported.py
+        if os.path.basename(_oeqadir) == "meta":
+            layerpath = exportpath
+            # Make sure we always copy the minimum required files from meta
+            shutil.copy2(os.path.join(oeqadir, "oetest.py"), os.path.join(layerpath, "oeqa"))
+            shutil.copy2(os.path.join(oeqadir, "__init__.py"), os.path.join(layerpath, "oeqa"))
+            shutil.copy2(os.path.join(oeqadir, "runtime", "__init__.py"), os.path.join(layerpath, "oeqa", "runtime"))
+            shutil.copy2(os.path.join(oeqadir, "runexported.py"), layerpath)
+            for root, dirs, files in os.walk(os.path.join(oeqadir, "utils")):
+                for f in files:
+                    if f.endswith(".py"):
+                        shutil.copy2(os.path.join(root, f), os.path.join(layerpath, "oeqa/utils"))
+        else:
+            if oeqadir in mod.filename:
+                layerpath = os.path.join(exportpath, "extralayers", get_layer(mod.filename))
+
+        try:
+            if oeqadir in mod.filename:
+                shutil.copy2(os.path.join(oeqadir, "__init__.py"), os.path.join(layerpath, "oeqa"))
+        except IOError:
+            pass
+        try:
+            if oeqadir in mod.filename:
+                shutil.copy2(os.path.join(oeqadir, "runtime/__init__.py"), os.path.join(layerpath, "oeqa/runtime"))
+        except IOError:
+            pass
+        # copy oeqa/utils/*.py
+        for root, dirs, files in os.walk(os.path.join(oeqadir, "utils")):
+            for f in files:
+                if f.endswith(".py") and (oeqadir in mod.filename):
+                    shutil.copy2(os.path.join(root, f), os.path.join(layerpath, "oeqa/utils"))
+        # copy oeqa/runtime/files/*
+        for root, dirs, files in os.walk(os.path.join(oeqadir, "runtime/files")):
+            for f in files:
+                if oeqadir in mod.filename:
+                    shutil.copy2(os.path.join(root, f), os.path.join(layerpath, "oeqa/runtime/files"))
+        # copy conf/test folder
+        if os.path.isdir(os.path.join(_oeqadir, "conf", "test")):
+            if os.path.join("meta","lib","oeqa") in oeqadir:
+                bb.utils.mkdirhier(os.path.join(exportpath, "conf/test"))
+            else:
+                bb.utils.mkdirhier(os.path.join(exportpath, "extralayers/%s/conf/test" % os.path.basename(os.path.normpath(layer))))
+            for root, dirs, files in os.walk(os.path.join(_oeqadir, "conf", "test")):
+                for f in files:
+                    shutil.copy2(os.path.join(root, f), os.path.join(layerpath, "conf/test"))
 
     #integrating binaries too
     # creting needed directory structure
-- 
2.5.0

[PATCH 11/12] oeqa/runtime: Copy all __init__.py files from all layers.

[Costin Constantin]

From: Lucian Musat <george.l.musat@intel.com>

Copy all the __init__.py files from all the extra layers
also, specified in bblayers.conf, in accordance to
TEST_SUITES variable.

Signed-off-by: Lucian Musat <george.l.musat@intel.com>
---
 meta/classes/testimage.bbclass | 67 +++++++++++++++++++++++-------------------
 meta/lib/oeqa/runexported.py   |  2 +-
 2 files changed, 38 insertions(+), 31 deletions(-)

diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index 88aa941..83a11ee 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -211,6 +211,7 @@ def exportTests(d,tc):
     import shutil
     import pkgutil
     import re
+    import oeqa.utils.testexport as te
 
     exportpath = d.getVar("TEST_EXPORT_DIR", True)
 
@@ -300,37 +301,37 @@ def exportTests(d,tc):
             if not os.path.exists(target_folder):
                 shutil.copytree(mod.filename, target_folder)
 
-         if not isfolder:
+        if not isfolder:
             shutil.copy2(mod.filename, os.path.join(layerpath, "oeqa/runtime"))
-    # copy __init__.py files
-    for _oeqadir in get_extra_layers(d):
-        oeqadir = os.path.join(_oeqadir,"lib","oeqa")
-        # copy oeqa/oetest.py and oeqa/runexported.py
-        if os.path.basename(_oeqadir) == "meta":
-            layerpath = exportpath
-            # Make sure we always copy the minimum required files from meta
-            shutil.copy2(os.path.join(oeqadir, "oetest.py"), os.path.join(layerpath, "oeqa"))
-            shutil.copy2(os.path.join(oeqadir, "__init__.py"), os.path.join(layerpath, "oeqa"))
-            shutil.copy2(os.path.join(oeqadir, "runtime", "__init__.py"), os.path.join(layerpath, "oeqa", "runtime"))
-            shutil.copy2(os.path.join(oeqadir, "runexported.py"), layerpath)
-            for root, dirs, files in os.walk(os.path.join(oeqadir, "utils")):
-                for f in files:
-                    if f.endswith(".py"):
-                        shutil.copy2(os.path.join(root, f), os.path.join(layerpath, "oeqa/utils"))
-        else:
-            if oeqadir in mod.filename:
-                layerpath = os.path.join(exportpath, "extralayers", get_layer(mod.filename))
-
-        try:
-            if oeqadir in mod.filename:
+        # copy __init__.py files
+        for _oeqadir in get_extra_layers(d):
+            oeqadir = os.path.join(_oeqadir,"lib","oeqa")
+            # copy oeqa/oetest.py and oeqa/runexported.py
+            if os.path.basename(_oeqadir) == "meta":
+                layerpath = exportpath
+                # Make sure we always copy the minimum required files from meta
+                shutil.copy2(os.path.join(oeqadir, "oetest.py"), os.path.join(layerpath, "oeqa"))
                 shutil.copy2(os.path.join(oeqadir, "__init__.py"), os.path.join(layerpath, "oeqa"))
-        except IOError:
-            pass
-        try:
-            if oeqadir in mod.filename:
-                shutil.copy2(os.path.join(oeqadir, "runtime/__init__.py"), os.path.join(layerpath, "oeqa/runtime"))
-        except IOError:
-            pass
+                shutil.copy2(os.path.join(oeqadir, "runtime", "__init__.py"), os.path.join(layerpath, "oeqa", "runtime"))
+                shutil.copy2(os.path.join(oeqadir, "runexported.py"), layerpath)
+                for root, dirs, files in os.walk(os.path.join(oeqadir, "utils")):
+                    for f in files:
+                        if f.endswith(".py"):
+                            shutil.copy2(os.path.join(root, f), os.path.join(layerpath, "oeqa/utils"))
+            else:
+                if oeqadir in mod.filename:
+                    layerpath = os.path.join(exportpath, "extralayers", get_layer(mod.filename))
+
+            try:
+                if oeqadir in mod.filename:
+                    shutil.copy2(os.path.join(oeqadir, "__init__.py"), os.path.join(layerpath, "oeqa"))
+            except IOError:
+                pass
+            try:
+                if oeqadir in mod.filename:
+                    shutil.copy2(os.path.join(oeqadir, "runtime/__init__.py"), os.path.join(layerpath, "oeqa/runtime"))
+            except IOError:
+                pass
         # copy oeqa/utils/*.py
         for root, dirs, files in os.walk(os.path.join(oeqadir, "utils")):
             for f in files:
@@ -347,13 +348,19 @@ def exportTests(d,tc):
                 bb.utils.mkdirhier(os.path.join(exportpath, "conf/test"))
             else:
                 bb.utils.mkdirhier(os.path.join(exportpath, "extralayers/%s/conf/test" % os.path.basename(os.path.normpath(layer))))
+                layerpath = os.path.join(exportpath, "extralayers", os.path.basename(os.path.normpath(layer)))
             for root, dirs, files in os.walk(os.path.join(_oeqadir, "conf", "test")):
                 for f in files:
                     shutil.copy2(os.path.join(root, f), os.path.join(layerpath, "conf/test"))
 
     #integrating binaries too
     # creting needed directory structure
-    arch = te.get_dest_folder(d.getVar("TUNE_FEATURES", True), os.listdir(d.getVar("DEPLOY_DIR_RPM", True)))
+    try:
+        arch = te.get_dest_folder(d.getVar("TUNE_FEATURES", True), os.listdir(d.getVar("DEPLOY_DIR_RPM", True)))
+    except OSError:
+        # Make a default folder
+        bb.utils.mkdirhier(os.path.join(d.getVar("DEPLOY_DIR_RPM", True), "i586"))
+        arch = "i586"
     bb.utils.mkdirhier(os.path.join(exportpath,"tar_files"))
     bb.utils.mkdirhier(os.path.join(exportpath,"binaries", arch, "packaged_binaries"))
     bb.utils.mkdirhier(os.path.join(exportpath,"binaries", "native"))
diff --git a/meta/lib/oeqa/runexported.py b/meta/lib/oeqa/runexported.py
index 3a6bad7..2ce4922 100755
--- a/meta/lib/oeqa/runexported.py
+++ b/meta/lib/oeqa/runexported.py
@@ -24,7 +24,7 @@ import time
 import argparse
 import re
 
-from oeqa.utils.testsexport import get_dest_folder
+from oeqa.utils.testexport import get_dest_folder
 
 try:
     import simplejson as json
-- 
2.5.0

[PATCH 12/12] buildtools-with-tc.bb: extend buildtools-tarball to include test cases

[Costin Constantin]

recipes-core/meta/buildtools-with-tc.bb is a new recipe that uses
all buildtools-tarball has to offer and adds test cases support.

Signed-off-by: Costin Constantin <costin.c.constantin@intel.com>
---
 meta/recipes-core/meta/buildtools-with-tc.bb | 37 ++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)
 create mode 100644 meta/recipes-core/meta/buildtools-with-tc.bb

diff --git a/meta/recipes-core/meta/buildtools-with-tc.bb b/meta/recipes-core/meta/buildtools-with-tc.bb
new file mode 100644
index 0000000..84a57ee
--- /dev/null
+++ b/meta/recipes-core/meta/buildtools-with-tc.bb
@@ -0,0 +1,37 @@
+DESCRIPTION = "This recipe is used to extend the functionality of buildtools-tarball \
+               by adding the test harness plus binaries that might be required for DUTs.\
+              "
+
+require ${COREBASE}/meta/recipes-core/meta/buildtools-tarball.bb
+inherit testimage
+
+SDK_TITLE += "with test cases and binaries"
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://${COREBASE}/LICENSE;md5=4d92cd373abda3937c2bc47fbc49d690 \
+                    file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
+
+TEST_EXPORT_ONLY = "1"
+TEST_TARGET = "simpleremote"
+IMAGE_NO_MANIFEST = "1"
+
+addtask testimage before do_populate_sdk
+
+tar_sdk_prepend () {
+  mkdir ${SDK_OUTPUT}${SDKPATH}/exported_tests 2>/dev/null
+  for i in $(ls ${TEST_EXPORT_DIR})
+  do
+    if [ ${i} != "tar_files" ]
+    then
+      cp -r ${TEST_EXPORT_DIR}/${i} ${SDK_OUTPUT}${SDKPATH}/exported_tests
+    fi
+  done
+  sed -i 's/\"pkgmanifest\": \"\"/\"pkgmanifest\": \"\\ndropbear\\n\"/' ${SDK_OUTPUT}${SDKPATH}/exported_tests/testdata.json
+  cp ${SDK_OUTPUT}${SDKPATH}/exported_tests/testdata.json ${TEST_EXPORT_DIR}
+}
+
+create_shar_append () {
+  message='echo "To run exported tests, go into exported_tests directory and run ./runexported.py"'
+  sed -i "/exit\ 0/i \
+${message}" ${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.sh
+}
+
-- 
2.5.0