All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Adrian Freihofer" <adrian.freihofer@gmail.com>
To: openembedded-core@lists.openembedded.org
Cc: Adrian Freihofer <adrian.freihofer@siemens.com>
Subject: [PATCH 2/3] testimage: support additional reports for ptests
Date: Thu,  3 Jun 2021 19:21:48 +0200	[thread overview]
Message-ID: <20210603172149.100248-2-adrian.freihofer@siemens.com> (raw)
In-Reply-To: <20210603172149.100248-1-adrian.freihofer@siemens.com>

This adds a new optional feature to the ptest run-time test. Most unit
test frameworks such as googletest generate JUnit like XML reports which
can be processed e.g. by GitLab CI or Jenkins.

Example: A run-ptest script executes a googletest based unit test:
  /usr/bin/my-unittest --gtest_output="xml:/tmp/ptest-xml/"

The new variable TESTIMAGE_PTEST_REPORT_DIR allows to configure
bitbake -c testimage to fetch the reports from the target device and
store them into a subfolder of TEST_LOG_DIR. It's possible to fetch
report files from different locations on the target device to different
subfolders on the host.
---
 meta/classes/testimage.bbclass       |  5 +++++
 meta/lib/oeqa/runtime/cases/ptest.py | 24 ++++++++++++++++++++++++
 2 files changed, 29 insertions(+)

diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index 43de9d4d76..d01892136f 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -47,6 +47,11 @@ TESTIMAGE_AUTO ??= "0"
 # TESTIMAGE_BOOT_PATTERNS[search_login_succeeded] = "webserver@[a-zA-Z0-9\-]+:~#"
 # The accepted flags are the following: search_reached_prompt, send_login_user, search_login_succeeded, search_cmd_finished.
 # They are prefixed with either search/send, to differentiate if the pattern is meant to be sent or searched to/from the target terminal
+# TESTIMAGE_PTEST_REPORT_DIR might be used to fetch additional reports (e.g. JUnit like xml files) generated by ptests from the target device.
+# A ; separate list of remote_path:host_path is expected. The host_path is optional. It defaults to "reports".
+# For example if some ptests (such as ptest-example.bb) create additional reports in /tmp/ptest-xml/ the following line in the image recipe
+# configures the ptest imagetest to fetch the xml reports into a "xml-reports" subfolder of TEST_LOG_DIR:
+# TESTIMAGE_PTEST_REPORT_DIR = "/tmp/ptest-xml/*.xml:xml-reports"
 
 TEST_LOG_DIR ?= "${WORKDIR}/testimage"
 
diff --git a/meta/lib/oeqa/runtime/cases/ptest.py b/meta/lib/oeqa/runtime/cases/ptest.py
index 0800f3c27f..7b3560a4b0 100644
--- a/meta/lib/oeqa/runtime/cases/ptest.py
+++ b/meta/lib/oeqa/runtime/cases/ptest.py
@@ -110,3 +110,27 @@ class PtestRunnerTest(OERuntimeTestCase):
         if failmsg:
             self.logger.warning("There were failing ptests.")
             self.fail(failmsg)
+
+        # Fetch log files e.g. JUnit like xml files from the target device
+        ptest_report_dir = self.td.get('TESTIMAGE_PTEST_REPORT_DIR', '')
+        if ptest_report_dir:
+            for test_log_dir_ptest in ptest_report_dir.split(';'):
+                src_tgt = test_log_dir_ptest.split(':')
+                if len(src_tgt) == 1:
+                    tgt_dir_abs = os.path.join(ptest_log_dir, "reports")
+                elif len(src_tgt) == 2:
+                    tgt_dir_abs = os.path.join(ptest_log_dir, src_tgt[1])
+                else:
+                    self.logger.error("Invalid TESTIMAGE_PTEST_REPORT_DIR setting")
+                self.copy_logs(src_tgt[0], tgt_dir_abs)
+
+    def copy_logs(self, remoteSrc, localDst):
+        self.logger.debug("Fetching from target: %s to %s" % (remoteSrc, localDst))
+        if os.path.exists(localDst):
+            from shutil import rmtree
+            rmtree(localDst)
+        os.makedirs(localDst)
+        try:
+            self.target.copyFrom(remoteSrc, localDst)
+        except AssertionError:
+            pass
-- 
2.31.1


  reply	other threads:[~2021-06-03 17:22 UTC|newest]

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-06-03 17:21 [PATCH 1/3] meta-skeleton: add a ptest example Adrian Freihofer
2021-06-03 17:21 ` Adrian Freihofer [this message]
2021-06-03 17:43   ` [OE-core] [PATCH 2/3] testimage: support additional reports for ptests Alexander Kanavin
2021-06-03 17:21 ` [PATCH 3/3] runtime_test.py: add new testimage ptest test case Adrian Freihofer
2021-06-03 17:46 ` [OE-core] [PATCH 1/3] meta-skeleton: add a ptest example Alexander Kanavin
2021-06-03 19:53   ` Adrian Freihofer
2021-06-03 20:00     ` Alexander Kanavin
2021-06-05  8:38       ` Adrian Freihofer
2021-06-05 19:47         ` Alexander Kanavin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210603172149.100248-2-adrian.freihofer@siemens.com \
    --to=adrian.freihofer@gmail.com \
    --cc=adrian.freihofer@siemens.com \
    --cc=openembedded-core@lists.openembedded.org \
    --subject='Re: [PATCH 2/3] testimage: support additional reports for ptests' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.