All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Alexander Kanavin" <alex.kanavin@gmail.com>
To: Adrian Freihofer <adrian.freihofer@gmail.com>
Cc: OE-core <openembedded-core@lists.openembedded.org>,
	 Adrian Freihofer <adrian.freihofer@siemens.com>
Subject: Re: [OE-core] [PATCH 2/3] testimage: support additional reports for ptests
Date: Thu, 3 Jun 2021 19:43:37 +0200	[thread overview]
Message-ID: <CANNYZj_CueS0nTOwqkh6f57mqSEgh9w=o5VAcHjvBgFv7Q4Z0w@mail.gmail.com> (raw)
In-Reply-To: <20210603172149.100248-2-adrian.freihofer@siemens.com>

[-- Attachment #1: Type: text/plain, Size: 3961 bytes --]

Why does this need to be specifically in ptest.py? It only copies some
files over from the target; you can write a separate runtime testcase that
does the same.

Alex

On Thu, 3 Jun 2021 at 19:22, Adrian Freihofer <adrian.freihofer@gmail.com>
wrote:

> This adds a new optional feature to the ptest run-time test. Most unit
> test frameworks such as googletest generate JUnit like XML reports which
> can be processed e.g. by GitLab CI or Jenkins.
>
> Example: A run-ptest script executes a googletest based unit test:
>   /usr/bin/my-unittest --gtest_output="xml:/tmp/ptest-xml/"
>
> The new variable TESTIMAGE_PTEST_REPORT_DIR allows to configure
> bitbake -c testimage to fetch the reports from the target device and
> store them into a subfolder of TEST_LOG_DIR. It's possible to fetch
> report files from different locations on the target device to different
> subfolders on the host.
> ---
>  meta/classes/testimage.bbclass       |  5 +++++
>  meta/lib/oeqa/runtime/cases/ptest.py | 24 ++++++++++++++++++++++++
>  2 files changed, 29 insertions(+)
>
> diff --git a/meta/classes/testimage.bbclass
> b/meta/classes/testimage.bbclass
> index 43de9d4d76..d01892136f 100644
> --- a/meta/classes/testimage.bbclass
> +++ b/meta/classes/testimage.bbclass
> @@ -47,6 +47,11 @@ TESTIMAGE_AUTO ??= "0"
>  # TESTIMAGE_BOOT_PATTERNS[search_login_succeeded] = "webserver@
> [a-zA-Z0-9\-]+:~#"
>  # The accepted flags are the following: search_reached_prompt,
> send_login_user, search_login_succeeded, search_cmd_finished.
>  # They are prefixed with either search/send, to differentiate if the
> pattern is meant to be sent or searched to/from the target terminal
> +# TESTIMAGE_PTEST_REPORT_DIR might be used to fetch additional reports
> (e.g. JUnit like xml files) generated by ptests from the target device.
> +# A ; separate list of remote_path:host_path is expected. The host_path
> is optional. It defaults to "reports".
> +# For example if some ptests (such as ptest-example.bb) create
> additional reports in /tmp/ptest-xml/ the following line in the image recipe
> +# configures the ptest imagetest to fetch the xml reports into a
> "xml-reports" subfolder of TEST_LOG_DIR:
> +# TESTIMAGE_PTEST_REPORT_DIR = "/tmp/ptest-xml/*.xml:xml-reports"
>
>  TEST_LOG_DIR ?= "${WORKDIR}/testimage"
>
> diff --git a/meta/lib/oeqa/runtime/cases/ptest.py
> b/meta/lib/oeqa/runtime/cases/ptest.py
> index 0800f3c27f..7b3560a4b0 100644
> --- a/meta/lib/oeqa/runtime/cases/ptest.py
> +++ b/meta/lib/oeqa/runtime/cases/ptest.py
> @@ -110,3 +110,27 @@ class PtestRunnerTest(OERuntimeTestCase):
>          if failmsg:
>              self.logger.warning("There were failing ptests.")
>              self.fail(failmsg)
> +
> +        # Fetch log files e.g. JUnit like xml files from the target device
> +        ptest_report_dir = self.td.get('TESTIMAGE_PTEST_REPORT_DIR', '')
> +        if ptest_report_dir:
> +            for test_log_dir_ptest in ptest_report_dir.split(';'):
> +                src_tgt = test_log_dir_ptest.split(':')
> +                if len(src_tgt) == 1:
> +                    tgt_dir_abs = os.path.join(ptest_log_dir, "reports")
> +                elif len(src_tgt) == 2:
> +                    tgt_dir_abs = os.path.join(ptest_log_dir, src_tgt[1])
> +                else:
> +                    self.logger.error("Invalid TESTIMAGE_PTEST_REPORT_DIR
> setting")
> +                self.copy_logs(src_tgt[0], tgt_dir_abs)
> +
> +    def copy_logs(self, remoteSrc, localDst):
> +        self.logger.debug("Fetching from target: %s to %s" % (remoteSrc,
> localDst))
> +        if os.path.exists(localDst):
> +            from shutil import rmtree
> +            rmtree(localDst)
> +        os.makedirs(localDst)
> +        try:
> +            self.target.copyFrom(remoteSrc, localDst)
> +        except AssertionError:
> +            pass
> --
> 2.31.1
>
>
> 
>
>

[-- Attachment #2: Type: text/html, Size: 4767 bytes --]

  reply	other threads:[~2021-06-03 17:43 UTC|newest]

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-06-03 17:21 [PATCH 1/3] meta-skeleton: add a ptest example Adrian Freihofer
2021-06-03 17:21 ` [PATCH 2/3] testimage: support additional reports for ptests Adrian Freihofer
2021-06-03 17:43   ` Alexander Kanavin [this message]
2021-06-03 17:21 ` [PATCH 3/3] runtime_test.py: add new testimage ptest test case Adrian Freihofer
2021-06-03 17:46 ` [OE-core] [PATCH 1/3] meta-skeleton: add a ptest example Alexander Kanavin
2021-06-03 19:53   ` Adrian Freihofer
2021-06-03 20:00     ` Alexander Kanavin
2021-06-05  8:38       ` Adrian Freihofer
2021-06-05 19:47         ` Alexander Kanavin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CANNYZj_CueS0nTOwqkh6f57mqSEgh9w=o5VAcHjvBgFv7Q4Z0w@mail.gmail.com' \
    --to=alex.kanavin@gmail.com \
    --cc=adrian.freihofer@gmail.com \
    --cc=adrian.freihofer@siemens.com \
    --cc=openembedded-core@lists.openembedded.org \
    --subject='Re: [OE-core] [PATCH 2/3] testimage: support additional reports for ptests' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.