Re: [OE-core] [PATCH 2/3] testimage: support additional reports for ptests

["Alexander Kanavin" <alex.kanavin@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 3961 bytes --]

Why does this need to be specifically in ptest.py? It only copies some
files over from the target; you can write a separate runtime testcase that
does the same.

Alex

On Thu, 3 Jun 2021 at 19:22, Adrian Freihofer <adrian.freihofer@gmail.com>
wrote:

> This adds a new optional feature to the ptest run-time test. Most unit
> test frameworks such as googletest generate JUnit like XML reports which
> can be processed e.g. by GitLab CI or Jenkins.
>
> Example: A run-ptest script executes a googletest based unit test:
>   /usr/bin/my-unittest --gtest_output="xml:/tmp/ptest-xml/"
>
> The new variable TESTIMAGE_PTEST_REPORT_DIR allows to configure
> bitbake -c testimage to fetch the reports from the target device and
> store them into a subfolder of TEST_LOG_DIR. It's possible to fetch
> report files from different locations on the target device to different
> subfolders on the host.
> ---
>  meta/classes/testimage.bbclass       |  5 +++++
>  meta/lib/oeqa/runtime/cases/ptest.py | 24 ++++++++++++++++++++++++
>  2 files changed, 29 insertions(+)
>
> diff --git a/meta/classes/testimage.bbclass
> b/meta/classes/testimage.bbclass
> index 43de9d4d76..d01892136f 100644
> --- a/meta/classes/testimage.bbclass
> +++ b/meta/classes/testimage.bbclass
> @@ -47,6 +47,11 @@ TESTIMAGE_AUTO ??= "0"
>  # TESTIMAGE_BOOT_PATTERNS[search_login_succeeded] = "webserver@
> [a-zA-Z0-9\-]+:~#"
>  # The accepted flags are the following: search_reached_prompt,
> send_login_user, search_login_succeeded, search_cmd_finished.
>  # They are prefixed with either search/send, to differentiate if the
> pattern is meant to be sent or searched to/from the target terminal
> +# TESTIMAGE_PTEST_REPORT_DIR might be used to fetch additional reports
> (e.g. JUnit like xml files) generated by ptests from the target device.
> +# A ; separate list of remote_path:host_path is expected. The host_path
> is optional. It defaults to "reports".
> +# For example if some ptests (such as ptest-example.bb) create
> additional reports in /tmp/ptest-xml/ the following line in the image recipe
> +# configures the ptest imagetest to fetch the xml reports into a
> "xml-reports" subfolder of TEST_LOG_DIR:
> +# TESTIMAGE_PTEST_REPORT_DIR = "/tmp/ptest-xml/*.xml:xml-reports"
>
>  TEST_LOG_DIR ?= "${WORKDIR}/testimage"
>
> diff --git a/meta/lib/oeqa/runtime/cases/ptest.py
> b/meta/lib/oeqa/runtime/cases/ptest.py
> index 0800f3c27f..7b3560a4b0 100644
> --- a/meta/lib/oeqa/runtime/cases/ptest.py
> +++ b/meta/lib/oeqa/runtime/cases/ptest.py
> @@ -110,3 +110,27 @@ class PtestRunnerTest(OERuntimeTestCase):
>          if failmsg:
>              self.logger.warning("There were failing ptests.")
>              self.fail(failmsg)
> +
> +        # Fetch log files e.g. JUnit like xml files from the target device
> +        ptest_report_dir = self.td.get('TESTIMAGE_PTEST_REPORT_DIR', '')
> +        if ptest_report_dir:
> +            for test_log_dir_ptest in ptest_report_dir.split(';'):
> +                src_tgt = test_log_dir_ptest.split(':')
> +                if len(src_tgt) == 1:
> +                    tgt_dir_abs = os.path.join(ptest_log_dir, "reports")
> +                elif len(src_tgt) == 2:
> +                    tgt_dir_abs = os.path.join(ptest_log_dir, src_tgt[1])
> +                else:
> +                    self.logger.error("Invalid TESTIMAGE_PTEST_REPORT_DIR
> setting")
> +                self.copy_logs(src_tgt[0], tgt_dir_abs)
> +
> +    def copy_logs(self, remoteSrc, localDst):
> +        self.logger.debug("Fetching from target: %s to %s" % (remoteSrc,
> localDst))
> +        if os.path.exists(localDst):
> +            from shutil import rmtree
> +            rmtree(localDst)
> +        os.makedirs(localDst)
> +        try:
> +            self.target.copyFrom(remoteSrc, localDst)
> +        except AssertionError:
> +            pass
> --
> 2.31.1
>
>
> 
>
>

[-- Attachment #2: Type: text/html, Size: 4767 bytes --]