Linux-kselftest Archive on lore.kernel.org
 help / color / Atom feed
From: shuah <shuah@kernel.org>
To: Alan Maguire <alan.maguire@oracle.com>,
	brendanhiggins@google.com, frowand.list@gmail.com,
	gregkh@linuxfoundation.org, linux-kselftest@vger.kernel.org
Cc: corbet@lwn.net, linux-kernel@vger.kernel.org,
	kunit-dev@googlegroups.com, linux-doc@vger.kernel.org,
	shuah <shuah@kernel.org>
Subject: Re: [PATCH v8 kunit-next 0/4] kunit: add debugfs representation to show results
Date: Thu, 26 Mar 2020 14:14:46 -0600
Message-ID: <2333d91d-2575-87ba-d8bb-60023cc19990@kernel.org> (raw)
In-Reply-To: <1585232710-322-1-git-send-email-alan.maguire@oracle.com>

On 3/26/20 8:25 AM, Alan Maguire wrote:
> When kunit tests are run on native (i.e. non-UML) environments, the results
> of test execution are often intermixed with dmesg output.  This patch
> series attempts to solve this by providing a debugfs representation
> of the results of the last test run, available as
> 
> /sys/kernel/debug/kunit/<testsuite>/results
> 
> Changes since v7:
> 
> - renamed KUNIT_INDENT[2] to KUNIT_SUBTEST_INDENT, KUNIT_SUBSUBTEST_INDENT
>    and added more description to their definitions to clarify why they
>    are defined as they are (Shuah)
> - defined KUNIT_SUBSUBTEST_INDENT directly as 8 spaces to avoid
>    checkpatch error (Shuah)
> 
> Changes since v6:
> 
> - fixed regexp parsing in kunit_parser.py to ensure test results are read
>    successfully with 4-space indentation (Brendan, patch 3)
> 
> Changes since v5:
> 
> - replaced undefined behaviour use of snprintf(buf, ..., buf) in
>    kunit_log() with a function to append string to existing log
>    (Frank, patch 1)
> - added clarification on log size limitations to documentation
>    (Frank, patch 4)
> 
> Changes since v4:
> 
> - added suite-level log expectations to kunit log test (Brendan, patch 2)
> - added log expectations (of it being NULL) for case where
>    CONFIG_KUNIT_DEBUGFS=n to kunit log test (patch 2)
> - added patch 3 which replaces subtest tab indentation with 4 space
>    indentation as per TAP 14 spec (Frank, patch 3)
> 
> Changes since v3:
> 
> - added CONFIG_KUNIT_DEBUGFS to support conditional compilation of debugfs
>    representation, including string logging (Frank, patch 1)
> - removed unneeded NULL check for test_case in
>    kunit_suite_for_each_test_case() (Frank, patch 1)
> - added kunit log test to verify logging multiple strings works
>    (Frank, patch 2)
> - rephrased description of results file (Frank, patch 3)
> 
> Changes since v2:
> 
> - updated kunit_status2str() to kunit_status_to_string() and made it
>    static inline in include/kunit/test.h (Brendan)
> - added log string to struct kunit_suite and kunit_case, with log
>    pointer in struct kunit pointing at the case log.  This allows us
>    to collect kunit_[err|info|warning]() messages at the same time
>    as we printk() them.  This solves for the most part the sharing
>    of log messages between test execution and debugfs since we
>    just print the suite log (which contains the test suite preamble)
>    and the individual test logs.  The only exception is the suite-level
>    status, which we cannot store in the suite log as it would mean
>    we'd print the suite and its status prior to the suite's results.
>    (Brendan, patch 1)
> - dropped debugfs-based kunit run patch for now so as not to cause
>    problems with tests currently under development (Brendan)
> - fixed doc issues with code block (Brendan, patch 3)
> 
> Changes since v1:
>   - trimmed unneeded include files in lib/kunit/debugfs.c (Greg)
>   - renamed global debugfs functions to be prefixed with kunit_ (Greg)
>   - removed error checking for debugfs operations (Greg)
> 
> Alan Maguire (4):
>    kunit: add debugfs /sys/kernel/debug/kunit/<suite>/results display
>    kunit: add log test
>    kunit: subtests should be indented 4 spaces according to TAP
>    kunit: update documentation to describe debugfs representation
> 
>   Documentation/dev-tools/kunit/usage.rst |  14 +++
>   include/kunit/test.h                    |  63 ++++++++++++--
>   lib/kunit/Kconfig                       |   8 ++
>   lib/kunit/Makefile                      |   4 +
>   lib/kunit/assert.c                      |  79 ++++++++---------
>   lib/kunit/debugfs.c                     | 116 +++++++++++++++++++++++++
>   lib/kunit/debugfs.h                     |  30 +++++++
>   lib/kunit/kunit-test.c                  |  44 +++++++++-
>   lib/kunit/test.c                        | 148 +++++++++++++++++++++++++-------
>   tools/testing/kunit/kunit_parser.py     |  10 +--
>   10 files changed, 430 insertions(+), 86 deletions(-)
>   create mode 100644 lib/kunit/debugfs.c
>   create mode 100644 lib/kunit/debugfs.h
> 

Thanks. Applied to linux-kselftest kunit for 5.7-rc1.

thanks,
-- Shuah

      parent reply index

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-03-26 14:25 Alan Maguire
2020-03-26 14:25 ` [PATCH v8 kunit-next 1/4] kunit: add debugfs /sys/kernel/debug/kunit/<suite>/results display Alan Maguire
2020-03-26 14:25 ` [PATCH v8 kunit-next 2/4] kunit: add log test Alan Maguire
2020-03-26 14:25 ` [PATCH v8 kunit-next 3/4] kunit: subtests should be indented 4 spaces according to TAP Alan Maguire
2020-03-26 14:25 ` [PATCH v8 kunit-next 4/4] kunit: update documentation to describe debugfs representation Alan Maguire
2020-03-26 20:14 ` shuah [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=2333d91d-2575-87ba-d8bb-60023cc19990@kernel.org \
    --to=shuah@kernel.org \
    --cc=alan.maguire@oracle.com \
    --cc=brendanhiggins@google.com \
    --cc=corbet@lwn.net \
    --cc=frowand.list@gmail.com \
    --cc=gregkh@linuxfoundation.org \
    --cc=kunit-dev@googlegroups.com \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-kselftest@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-kselftest Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-kselftest/0 linux-kselftest/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-kselftest linux-kselftest/ https://lore.kernel.org/linux-kselftest \
		linux-kselftest@vger.kernel.org
	public-inbox-index linux-kselftest

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-kselftest


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git