linux-nvdimm.lists.01.org archive mirror
 help / color / mirror / Atom feed
From: Brendan Higgins <brendanhiggins-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
To: gregkh-hQyY1W1yCW8ekmWlsbkhG0B+6BGkLq7r@public.gmane.org,
	keescook-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org,
	mcgrof-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org,
	shuah-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org
Cc: brakmo-b10kYP2dOMg@public.gmane.org,
	jdike-OPE4K8JWMJJBDgjK7y7TUQ@public.gmane.org,
	dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org,
	linux-kselftest-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
	frowand.list-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org,
	robh-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org,
	linux-nvdimm-hn68Rpc1hR1g9hUCZPvPmw@public.gmane.org,
	richard-/L3Ra7n9ekc@public.gmane.org,
	knut.omang-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org,
	kieran.bingham-ryLnwIuWjnjg/C1BVhZhaw@public.gmane.org,
	Felix Guo
	<felixguoxiuping-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>,
	joel-U3u1mxZcP9KHXe+LvDLADg@public.gmane.org,
	khilman-rdvid1DuHRBWk0Htik3J/w@public.gmane.org,
	Brendan Higgins
	<brendanhiggins-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>,
	Tim.Bird-7U/KSKJipcs@public.gmane.org,
	linux-um-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org,
	rostedt-nx8X9YLhiw1AfugRpC6u6w@public.gmane.org,
	julia.lawall-L2FTfq7BK8M@public.gmane.org,
	kunit-dev-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org,
	linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
	daniel-/w4YWyX8dFk@public.gmane.org,
	mpe-Gsx/Oe8HsFggBc27wqDAHg@public.gmane.org,
	joe-6d6DIl74uiNBDgjK7y7TUQ@public.gmane.org
Subject: [RFC v3 12/19] kunit: add KUnit wrapper script and simple output parser
Date: Wed, 28 Nov 2018 11:36:29 -0800	[thread overview]
Message-ID: <20181128193636.254378-13-brendanhiggins@google.com> (raw)
In-Reply-To: <20181128193636.254378-1-brendanhiggins-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>

The KUnit wrapper script interfaces with the two modules
(kunit_config.py and kunit_kernel.py) and provides a command line
interface for running KUnit tests. This interface allows the caller to
specify options like test timeouts. The script handles configuring,
building and running the kernel and tests.

The output parser (kunit_parser.py) simply strips out all the output
from the kernel that is outputted as part of it's initialization
sequence. This ensures that only the output from KUnit is displayed
on the screen.

A full version of the output is written to test.log, or can be seen by
passing --raw_output to the wrapper script.

Signed-off-by: Felix Guo <felixguoxiuping-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Signed-off-by: Brendan Higgins <brendanhiggins-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
---
 tools/testing/kunit/kunit.py        | 40 +++++++++++++++++++++++++++++
 tools/testing/kunit/kunit_kernel.py |  3 +--
 tools/testing/kunit/kunit_parser.py | 24 +++++++++++++++++
 3 files changed, 65 insertions(+), 2 deletions(-)
 create mode 100755 tools/testing/kunit/kunit.py
 create mode 100644 tools/testing/kunit/kunit_parser.py

diff --git a/tools/testing/kunit/kunit.py b/tools/testing/kunit/kunit.py
new file mode 100755
index 0000000000000..1356be404996b
--- /dev/null
+++ b/tools/testing/kunit/kunit.py
@@ -0,0 +1,40 @@
+#!/usr/bin/python3
+# SPDX-License-Identifier: GPL-2.0
+
+# A thin wrapper on top of the KUnit Kernel
+
+import argparse
+import sys
+import os
+
+import kunit_config
+import kunit_kernel
+import kunit_parser
+
+parser = argparse.ArgumentParser(description='Runs KUnit tests.')
+
+parser.add_argument('--raw_output', help='don\'t format output from kernel',
+		    action='store_true')
+
+parser.add_argument('--timeout', help='maximum number of seconds to allow for '
+		    'all tests to run. This does not include time taken to '
+		    'build the tests.', type=int, default=300,
+		    metavar='timeout')
+
+cli_args = parser.parse_args()
+linux = kunit_kernel.LinuxSourceTree()
+
+success = linux.build_reconfig()
+if not success:
+	quit()
+
+print('Building KUnit Kernel ...')
+success = linux.build_um_kernel()
+if not success:
+	quit()
+
+print('Starting KUnit Kernel ...')
+if cli_args.raw_output:
+	kunit_parser.raw_output(linux.run_kernel(timeout=cli_args.timeout))
+else:
+	kunit_parser.parse_run_tests(linux.run_kernel(timeout=cli_args.timeout))
diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py
index bba7ea7ca1869..623f25b16f6c8 100644
--- a/tools/testing/kunit/kunit_kernel.py
+++ b/tools/testing/kunit/kunit_kernel.py
@@ -116,8 +116,7 @@ class LinuxSourceTree(object):
 			return False
 		return True
 
-	def run_kernel(self, args=[]):
-		timeout = None
+	def run_kernel(self, args=[], timeout=None):
 		args.extend(['mem=256M'])
 		process = self._ops.linux_bin(args, timeout)
 		with open('test.log', 'w') as f:
diff --git a/tools/testing/kunit/kunit_parser.py b/tools/testing/kunit/kunit_parser.py
new file mode 100644
index 0000000000000..1dff3adb73bd3
--- /dev/null
+++ b/tools/testing/kunit/kunit_parser.py
@@ -0,0 +1,24 @@
+# SPDX-License-Identifier: GPL-2.0
+
+import re
+
+kunit_start_re = re.compile('console .* enabled')
+kunit_end_re = re.compile('List of all partitions:')
+
+def isolate_kunit_output(kernel_output):
+	started = False
+	for line in kernel_output:
+		if kunit_start_re.match(line):
+			started = True
+		elif kunit_end_re.match(line):
+			break
+		elif started:
+			yield line
+
+def raw_output(kernel_output):
+	for line in kernel_output:
+		print(line)
+
+def parse_run_tests(kernel_output):
+	for output in isolate_kunit_output(kernel_output):
+		print(output)
-- 
2.20.0.rc0.387.gc7a69e6b6c-goog

  parent reply	other threads:[~2018-11-28 19:36 UTC|newest]

Thread overview: 118+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-11-28 19:36 [RFC v3 00/19] kunit: introduce KUnit, the Linux kernel unit testing framework Brendan Higgins
2018-11-28 19:36 ` [RFC v3 01/19] kunit: test: add KUnit test runner core Brendan Higgins
2018-11-30  3:14   ` Luis Chamberlain
2018-12-01  1:51     ` Brendan Higgins
2018-12-01  2:57       ` Luis Chamberlain
2018-12-05 13:15     ` Anton Ivanov
2018-12-05 14:45       ` Arnd Bergmann
2018-12-05 14:49         ` Anton Ivanov
2018-11-30  3:28   ` Luis Chamberlain
     [not found]     ` <20181130032802.GG18410-dAjH6bxAqesAS62YNPtMr3dQhYtBYE6JAL8bYrjMMd8@public.gmane.org>
2018-12-01  2:08       ` Brendan Higgins
2018-12-01  3:10         ` Luis Chamberlain
     [not found]           ` <20181201031049.GL28501-dAjH6bxAqesAS62YNPtMr3dQhYtBYE6JAL8bYrjMMd8@public.gmane.org>
2018-12-03 22:47             ` Brendan Higgins
2018-12-01  3:02   ` Luis Chamberlain
2018-11-28 19:36 ` [RFC v3 10/19] kunit: test: add test managed resource tests Brendan Higgins
     [not found] ` <20181128193636.254378-1-brendanhiggins-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
2018-11-28 19:36   ` [RFC v3 02/19] kunit: test: add test resource management API Brendan Higgins
2018-11-28 19:36   ` [RFC v3 03/19] kunit: test: add string_stream a std::stream like string builder Brendan Higgins
2018-11-30  3:29     ` Luis Chamberlain
2018-12-01  2:14       ` Brendan Higgins
2018-12-01  3:12         ` Luis Chamberlain
2018-12-03 10:55       ` Petr Mladek
2018-12-04  0:35         ` Brendan Higgins
2018-11-28 19:36   ` [RFC v3 04/19] kunit: test: add test_stream a std::stream like logger Brendan Higgins
2018-11-28 19:36   ` [RFC v3 05/19] kunit: test: add the concept of expectations Brendan Higgins
2018-11-28 19:36   ` [RFC v3 06/19] arch: um: enable running kunit from User Mode Linux Brendan Higgins
2018-11-28 21:26     ` Rob Herring
2018-11-30  3:37       ` Luis Chamberlain
2018-11-30 14:05         ` Rob Herring
2018-11-30 18:22           ` Luis Chamberlain
     [not found]             ` <20181130182203.GS18410-dAjH6bxAqesAS62YNPtMr3dQhYtBYE6JAL8bYrjMMd8@public.gmane.org>
2018-12-03 23:22               ` Brendan Higgins
2018-11-30  3:30     ` Luis Chamberlain
2018-11-28 19:36   ` [RFC v3 07/19] kunit: test: add initial tests Brendan Higgins
2018-11-30  3:40     ` Luis Chamberlain
2018-12-03 23:26       ` Brendan Higgins
2018-12-03 23:43         ` Luis Chamberlain
2018-11-28 19:36   ` [RFC v3 08/19] arch: um: add shim to trap to allow installing a fault catcher for tests Brendan Higgins
2018-11-30  3:34     ` Luis Chamberlain
     [not found]       ` <20181130033429.GK18410-dAjH6bxAqesAS62YNPtMr3dQhYtBYE6JAL8bYrjMMd8@public.gmane.org>
2018-12-03 23:34         ` Brendan Higgins
     [not found]           ` <CAFd5g45+MAVaSW8HN9x57Y8Um=TV1Oa=-K8yExPBS-4KjLyciQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2018-12-03 23:46             ` Luis Chamberlain
     [not found]               ` <20181203234628.GR28501-dAjH6bxAqesAS62YNPtMr3dQhYtBYE6JAL8bYrjMMd8@public.gmane.org>
2018-12-04  0:44                 ` Brendan Higgins
2018-11-30  3:41     ` Luis Chamberlain
2018-12-03 23:37       ` Brendan Higgins
2018-11-28 19:36   ` [RFC v3 09/19] kunit: test: add the concept of assertions Brendan Higgins
2018-11-28 19:36   ` [RFC v3 11/19] kunit: add Python libraries for handing KUnit config and kernel Brendan Higgins
2018-11-29 13:54     ` Kieran Bingham
     [not found]       ` <841cf4ae-501b-05ae-5863-a51010709b67-ryLnwIuWjnjg/C1BVhZhaw@public.gmane.org>
2018-12-03 23:48         ` Brendan Higgins
2018-12-04 20:47           ` Luis Chamberlain
2018-12-06 12:32             ` Kieran Bingham
2018-12-06 15:37               ` Matthew Wilcox
2018-12-07 11:30                 ` Kieran Bingham
2018-12-11 14:09                 ` Petr Mladek
2018-12-11 14:41                   ` Steven Rostedt
2018-12-11 17:01                     ` Anton Ivanov
2019-02-09  0:40                       ` Brendan Higgins
2018-12-07  1:05               ` Luis Chamberlain
2018-12-07 18:35               ` Kent Overstreet
2018-11-30  3:44     ` Luis Chamberlain
2018-12-03 23:50       ` Brendan Higgins
2018-12-04 20:48         ` Luis Chamberlain
2018-11-28 19:36   ` Brendan Higgins [this message]
2018-11-28 19:36   ` [RFC v3 13/19] kunit: improve output from python wrapper Brendan Higgins
2018-11-28 19:36   ` [RFC v3 14/19] Documentation: kunit: add documentation for KUnit Brendan Higgins
2018-11-29 13:56     ` Kieran Bingham
2018-11-30  3:45       ` Luis Chamberlain
     [not found]         ` <20181130034525.GP18410-dAjH6bxAqesAS62YNPtMr3dQhYtBYE6JAL8bYrjMMd8@public.gmane.org>
2018-12-03 23:53           ` Brendan Higgins
2018-12-06 12:16             ` Kieran Bingham
2019-02-09  0:56               ` Brendan Higgins
2019-02-11 12:16                 ` Kieran Bingham
2019-02-12 22:10                   ` Brendan Higgins
2019-02-13 21:55                     ` Kieran Bingham
2019-02-14  0:17                       ` Brendan Higgins
2019-02-14 17:26                         ` Luis Chamberlain
2019-02-14 22:07                           ` Brendan Higgins
2018-11-28 19:36   ` [RFC v3 15/19] MAINTAINERS: add entry for KUnit the unit testing framework Brendan Higgins
2018-11-28 19:36   ` [RFC v3 17/19] of: unittest: migrate tests to run on KUnit Brendan Higgins
2018-11-28 20:56     ` Rob Herring
2018-11-30  0:39       ` Randy Dunlap
     [not found]         ` <18814973-8f0a-4647-a097-fcc3dc0b3cd3-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
2018-12-04  0:13           ` Brendan Higgins
2018-12-04 13:40             ` Rob Herring
     [not found]               ` <CAL_JsqL_PivQbrJFEusdKAy-2EQtKL3OHbmyYSK9bzuTOQegqA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2018-12-05 23:42                 ` Brendan Higgins
2018-12-07  0:41                   ` Rob Herring
2018-12-04  0:08       ` Brendan Higgins
2019-02-13  1:44       ` Brendan Higgins
2019-02-14 20:10         ` Rob Herring
2019-02-14 21:52           ` Brendan Higgins
2019-02-18 22:56         ` Frank Rowand
2019-02-28  0:29           ` Brendan Higgins
2018-12-04 10:56     ` Frank Rowand
2018-11-28 19:36   ` [RFC v3 18/19] of: unittest: split out a couple of test cases from unittest Brendan Higgins
2018-12-04 10:58     ` Frank Rowand
2018-12-05 23:54       ` Brendan Higgins
2019-02-14 23:57         ` Frank Rowand
2019-02-15  0:56           ` Brendan Higgins
2019-02-15  2:05             ` Frank Rowand
2019-02-15 10:56               ` Brendan Higgins
2019-02-18 22:25                 ` Frank Rowand
2019-02-20 20:44                   ` Frank Rowand
2019-02-20 20:47                     ` Frank Rowand
2019-02-28  3:52                     ` Brendan Higgins
2019-03-22  0:22                       ` Frank Rowand
2019-03-22  1:30                         ` Brendan Higgins
2019-03-22  1:47                           ` Frank Rowand
2019-03-25 22:15                             ` Brendan Higgins
2019-09-20 16:57                           ` Rob Herring
2019-09-21 23:57                             ` Frank Rowand
2019-03-22  1:34                         ` Frank Rowand
2019-03-25 22:18                           ` Brendan Higgins
2018-11-28 19:36   ` [RFC v3 19/19] of: unittest: split up some super large test cases Brendan Higgins
2018-11-28 19:36 ` [RFC v3 16/19] arch: um: make UML unflatten device tree when testing Brendan Higgins
2018-11-28 21:16   ` Rob Herring
     [not found]     ` <CAL_JsqK5cG=QzMBFSZ31_-3ujnxqxv=jj3XYajbRLT7yWYZbfw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2018-12-04  0:00       ` Brendan Higgins
2018-11-30  3:46   ` Luis Chamberlain
2018-12-04  0:02     ` Brendan Higgins
2018-12-04 10:52 ` [RFC v3 00/19] kunit: introduce KUnit, the Linux kernel unit testing framework Frank Rowand
2018-12-04 11:40 ` Frank Rowand
2018-12-04 13:49   ` Rob Herring
2018-12-05 23:10     ` Brendan Higgins
2019-03-22  0:27       ` Frank Rowand
2019-03-25 22:04         ` Brendan Higgins

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20181128193636.254378-13-brendanhiggins@google.com \
    --to=brendanhiggins-hpiqsd4aklfqt0dzr+alfa@public.gmane.org \
    --cc=Tim.Bird-7U/KSKJipcs@public.gmane.org \
    --cc=brakmo-b10kYP2dOMg@public.gmane.org \
    --cc=daniel-/w4YWyX8dFk@public.gmane.org \
    --cc=dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org \
    --cc=felixguoxiuping-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org \
    --cc=frowand.list-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org \
    --cc=gregkh-hQyY1W1yCW8ekmWlsbkhG0B+6BGkLq7r@public.gmane.org \
    --cc=jdike-OPE4K8JWMJJBDgjK7y7TUQ@public.gmane.org \
    --cc=joe-6d6DIl74uiNBDgjK7y7TUQ@public.gmane.org \
    --cc=joel-U3u1mxZcP9KHXe+LvDLADg@public.gmane.org \
    --cc=julia.lawall-L2FTfq7BK8M@public.gmane.org \
    --cc=keescook-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org \
    --cc=khilman-rdvid1DuHRBWk0Htik3J/w@public.gmane.org \
    --cc=kieran.bingham-ryLnwIuWjnjg/C1BVhZhaw@public.gmane.org \
    --cc=knut.omang-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org \
    --cc=kunit-dev-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org \
    --cc=linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    --cc=linux-kselftest-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    --cc=linux-nvdimm-hn68Rpc1hR1g9hUCZPvPmw@public.gmane.org \
    --cc=linux-um-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org \
    --cc=mcgrof-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org \
    --cc=mpe-Gsx/Oe8HsFggBc27wqDAHg@public.gmane.org \
    --cc=richard-/L3Ra7n9ekc@public.gmane.org \
    --cc=robh-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org \
    --cc=rostedt-nx8X9YLhiw1AfugRpC6u6w@public.gmane.org \
    --cc=shuah-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).