All of lore.kernel.org
 help / color / mirror / Atom feed
* Kunit fixes update for Linux 5.10-rc3
@ 2020-11-05 16:55 Shuah Khan
  2020-11-05 20:02 ` Linus Torvalds
  0 siblings, 1 reply; 6+ messages in thread
From: Shuah Khan @ 2020-11-05 16:55 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: Shuah Khan, Brendan Higgins, linux-kernel, linux-kselftest

[-- Attachment #1: Type: text/plain, Size: 2327 bytes --]

Hi Linus,

Please pull the following Kunit fixes update for Linux 5.10-rc3

This Kunit update for Linux 5.10-rc3 consists of several kunit_tool
and documentation fixes.

diff is attached.

thanks,
-- Shuah

----------------------------------------------------------------
The following changes since commit 3650b228f83adda7e5ee532e2b90429c03f7b9ec:

   Linux 5.10-rc1 (2020-10-25 15:14:11 -0700)

are available in the Git repository at:

   git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest 
tags/linux-kselftest-kunit-fixes-5.10-rc3

for you to fetch changes up to 0d0d245104a42e593adcf11396017a6420c08ba8:

   kunit: tools: fix kunit_tool tests for parsing test plans (2020-10-26 
13:25:40 -0600)

----------------------------------------------------------------
linux-kselftest-kunit-fixes-5.10-rc3

This Kunit update for Linux 5.10-rc3 consists of several kunit_tool
and documentation fixes.

----------------------------------------------------------------
Andy Shevchenko (1):
       kunit: Don't fail test suites if one of them is empty

Brendan Higgins (1):
       kunit: tools: fix kunit_tool tests for parsing test plans

David Gow (1):
       kunit: Fix kunit.py --raw_output option

Mauro Carvalho Chehab (1):
       kunit: test: fix remaining kernel-doc warnings

SeongJae Park (1):
       Documentation: kunit: Update Kconfig parts for KUNIT's module support

  Documentation/dev-tools/kunit/start.rst            |   2 +-
  Documentation/dev-tools/kunit/usage.rst            |   5 ++++
  include/kunit/test.h                               |  16 +++++------
  tools/testing/kunit/kunit_parser.py                |   3 +-
  tools/testing/kunit/kunit_tool_test.py             |  32 
++++++++++++++++-----
  .../kunit/test_data/test_config_printk_time.log    | Bin 1584 -> 1605 
bytes
  .../test_data/test_interrupted_tap_output.log      | Bin 1982 -> 2003 
bytes
  .../test_data/test_kernel_panic_interrupt.log      | Bin 1321 -> 1342 
bytes
  .../kunit/test_data/test_multiple_prefixes.log     | Bin 1832 -> 1861 
bytes
  .../kunit/test_data/test_pound_no_prefix.log       | Bin 1193 -> 1200 
bytes
  tools/testing/kunit/test_data/test_pound_sign.log  | Bin 1656 -> 1676 
bytes
  11 files changed, 40 insertions(+), 18 deletions(-)

----------------------------------------------------------------

[-- Attachment #2: linux-kselftest-kunit-fixes-5.10-rc3.diff --]
[-- Type: text/x-patch, Size: 8868 bytes --]

diff --git a/Documentation/dev-tools/kunit/start.rst b/Documentation/dev-tools/kunit/start.rst
index d23385e3e159..454f307813ea 100644
--- a/Documentation/dev-tools/kunit/start.rst
+++ b/Documentation/dev-tools/kunit/start.rst
@@ -197,7 +197,7 @@ Now add the following to ``drivers/misc/Kconfig``:
 
 	config MISC_EXAMPLE_TEST
 		bool "Test for my example"
-		depends on MISC_EXAMPLE && KUNIT
+		depends on MISC_EXAMPLE && KUNIT=y
 
 and the following to ``drivers/misc/Makefile``:
 
diff --git a/Documentation/dev-tools/kunit/usage.rst b/Documentation/dev-tools/kunit/usage.rst
index 961d3ea3ca19..62142a47488c 100644
--- a/Documentation/dev-tools/kunit/usage.rst
+++ b/Documentation/dev-tools/kunit/usage.rst
@@ -561,6 +561,11 @@ Once the kernel is built and installed, a simple
 
 ...will run the tests.
 
+.. note::
+   Note that you should make sure your test depends on ``KUNIT=y`` in Kconfig
+   if the test does not support module build.  Otherwise, it will trigger
+   compile errors if ``CONFIG_KUNIT`` is ``m``.
+
 Writing new tests for other architectures
 -----------------------------------------
 
diff --git a/include/kunit/test.h b/include/kunit/test.h
index 9197da792336..db1b0ae666c4 100644
--- a/include/kunit/test.h
+++ b/include/kunit/test.h
@@ -252,13 +252,14 @@ static inline int kunit_run_all_tests(void)
 }
 #endif /* IS_BUILTIN(CONFIG_KUNIT) */
 
+#ifdef MODULE
 /**
- * kunit_test_suites() - used to register one or more &struct kunit_suite
- *			 with KUnit.
+ * kunit_test_suites_for_module() - used to register one or more
+ *			 &struct kunit_suite with KUnit.
  *
- * @suites_list...: a statically allocated list of &struct kunit_suite.
+ * @__suites: a statically allocated list of &struct kunit_suite.
  *
- * Registers @suites_list with the test framework. See &struct kunit_suite for
+ * Registers @__suites with the test framework. See &struct kunit_suite for
  * more information.
  *
  * If a test suite is built-in, module_init() gets translated into
@@ -267,7 +268,6 @@ static inline int kunit_run_all_tests(void)
  * module_{init|exit} functions for the builtin case when registering
  * suites via kunit_test_suites() below.
  */
-#ifdef MODULE
 #define kunit_test_suites_for_module(__suites)				\
 	static int __init kunit_test_suites_init(void)			\
 	{								\
@@ -294,7 +294,7 @@ static inline int kunit_run_all_tests(void)
  * kunit_test_suites() - used to register one or more &struct kunit_suite
  *			 with KUnit.
  *
- * @suites: a statically allocated list of &struct kunit_suite.
+ * @__suites: a statically allocated list of &struct kunit_suite.
  *
  * Registers @suites with the test framework. See &struct kunit_suite for
  * more information.
@@ -308,10 +308,10 @@ static inline int kunit_run_all_tests(void)
  * module.
  *
  */
-#define kunit_test_suites(...)						\
+#define kunit_test_suites(__suites...)						\
 	__kunit_test_suites(__UNIQUE_ID(array),				\
 			    __UNIQUE_ID(suites),			\
-			    __VA_ARGS__)
+			    ##__suites)
 
 #define kunit_test_suite(suite)	kunit_test_suites(&suite)
 
diff --git a/tools/testing/kunit/kunit_parser.py b/tools/testing/kunit/kunit_parser.py
index 8019e3dd4c32..84a1af2581f5 100644
--- a/tools/testing/kunit/kunit_parser.py
+++ b/tools/testing/kunit/kunit_parser.py
@@ -66,7 +66,6 @@ def isolate_kunit_output(kernel_output):
 def raw_output(kernel_output):
 	for line in kernel_output:
 		print(line)
-		yield line
 
 DIVIDER = '=' * 60
 
@@ -242,7 +241,7 @@ def parse_test_suite(lines: List[str], expected_suite_index: int) -> TestSuite:
 		return None
 	test_suite.name = name
 	expected_test_case_num = parse_subtest_plan(lines)
-	if not expected_test_case_num:
+	if expected_test_case_num is None:
 		return None
 	while expected_test_case_num > 0:
 		test_case = parse_test_case(lines)
diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py
index 99c3c5671ea4..0b60855fb819 100755
--- a/tools/testing/kunit/kunit_tool_test.py
+++ b/tools/testing/kunit/kunit_tool_test.py
@@ -179,7 +179,7 @@ class KUnitParserTest(unittest.TestCase):
 		print_mock = mock.patch('builtins.print').start()
 		result = kunit_parser.parse_run_tests(
 			kunit_parser.isolate_kunit_output(file.readlines()))
-		print_mock.assert_any_call(StrContains("no kunit output detected"))
+		print_mock.assert_any_call(StrContains('no tests run!'))
 		print_mock.stop()
 		file.close()
 
@@ -198,39 +198,57 @@ class KUnitParserTest(unittest.TestCase):
 			'test_data/test_config_printk_time.log')
 		with open(prefix_log) as file:
 			result = kunit_parser.parse_run_tests(file.readlines())
-		self.assertEqual('kunit-resource-test', result.suites[0].name)
+			self.assertEqual(
+				kunit_parser.TestStatus.SUCCESS,
+				result.status)
+			self.assertEqual('kunit-resource-test', result.suites[0].name)
 
 	def test_ignores_multiple_prefixes(self):
 		prefix_log = get_absolute_path(
 			'test_data/test_multiple_prefixes.log')
 		with open(prefix_log) as file:
 			result = kunit_parser.parse_run_tests(file.readlines())
-		self.assertEqual('kunit-resource-test', result.suites[0].name)
+			self.assertEqual(
+				kunit_parser.TestStatus.SUCCESS,
+				result.status)
+			self.assertEqual('kunit-resource-test', result.suites[0].name)
 
 	def test_prefix_mixed_kernel_output(self):
 		mixed_prefix_log = get_absolute_path(
 			'test_data/test_interrupted_tap_output.log')
 		with open(mixed_prefix_log) as file:
 			result = kunit_parser.parse_run_tests(file.readlines())
-		self.assertEqual('kunit-resource-test', result.suites[0].name)
+			self.assertEqual(
+				kunit_parser.TestStatus.SUCCESS,
+				result.status)
+			self.assertEqual('kunit-resource-test', result.suites[0].name)
 
 	def test_prefix_poundsign(self):
 		pound_log = get_absolute_path('test_data/test_pound_sign.log')
 		with open(pound_log) as file:
 			result = kunit_parser.parse_run_tests(file.readlines())
-		self.assertEqual('kunit-resource-test', result.suites[0].name)
+			self.assertEqual(
+				kunit_parser.TestStatus.SUCCESS,
+				result.status)
+			self.assertEqual('kunit-resource-test', result.suites[0].name)
 
 	def test_kernel_panic_end(self):
 		panic_log = get_absolute_path('test_data/test_kernel_panic_interrupt.log')
 		with open(panic_log) as file:
 			result = kunit_parser.parse_run_tests(file.readlines())
-		self.assertEqual('kunit-resource-test', result.suites[0].name)
+			self.assertEqual(
+				kunit_parser.TestStatus.TEST_CRASHED,
+				result.status)
+			self.assertEqual('kunit-resource-test', result.suites[0].name)
 
 	def test_pound_no_prefix(self):
 		pound_log = get_absolute_path('test_data/test_pound_no_prefix.log')
 		with open(pound_log) as file:
 			result = kunit_parser.parse_run_tests(file.readlines())
-		self.assertEqual('kunit-resource-test', result.suites[0].name)
+			self.assertEqual(
+				kunit_parser.TestStatus.SUCCESS,
+				result.status)
+			self.assertEqual('kunit-resource-test', result.suites[0].name)
 
 class KUnitJsonTest(unittest.TestCase):
 
diff --git a/tools/testing/kunit/test_data/test_config_printk_time.log b/tools/testing/kunit/test_data/test_config_printk_time.log
index c02ca773946d..6bdb57f76eac 100644
Binary files a/tools/testing/kunit/test_data/test_config_printk_time.log and b/tools/testing/kunit/test_data/test_config_printk_time.log differ
diff --git a/tools/testing/kunit/test_data/test_interrupted_tap_output.log b/tools/testing/kunit/test_data/test_interrupted_tap_output.log
index 5c73fb3a1c6f..1fb677728abe 100644
Binary files a/tools/testing/kunit/test_data/test_interrupted_tap_output.log and b/tools/testing/kunit/test_data/test_interrupted_tap_output.log differ
diff --git a/tools/testing/kunit/test_data/test_kernel_panic_interrupt.log b/tools/testing/kunit/test_data/test_kernel_panic_interrupt.log
index c045eee75f27..a014ffe9725e 100644
Binary files a/tools/testing/kunit/test_data/test_kernel_panic_interrupt.log and b/tools/testing/kunit/test_data/test_kernel_panic_interrupt.log differ
diff --git a/tools/testing/kunit/test_data/test_multiple_prefixes.log b/tools/testing/kunit/test_data/test_multiple_prefixes.log
index bc48407dcc36..0ad78481a0b4 100644
Binary files a/tools/testing/kunit/test_data/test_multiple_prefixes.log and b/tools/testing/kunit/test_data/test_multiple_prefixes.log differ
diff --git a/tools/testing/kunit/test_data/test_pound_no_prefix.log b/tools/testing/kunit/test_data/test_pound_no_prefix.log
index 2ceb360be7d5..dc4cf09a96d0 100644
Binary files a/tools/testing/kunit/test_data/test_pound_no_prefix.log and b/tools/testing/kunit/test_data/test_pound_no_prefix.log differ
diff --git a/tools/testing/kunit/test_data/test_pound_sign.log b/tools/testing/kunit/test_data/test_pound_sign.log
index 28ffa5ba03bf..3f358e3a7ba0 100644
Binary files a/tools/testing/kunit/test_data/test_pound_sign.log and b/tools/testing/kunit/test_data/test_pound_sign.log differ

^ permalink raw reply related	[flat|nested] 6+ messages in thread
* Re: Kunit fixes update for Linux 5.10-rc3
  2020-11-05 16:55 Kunit fixes update for Linux 5.10-rc3 Shuah Khan
@ 2020-11-05 20:02 ` Linus Torvalds
  2020-11-05 20:25   ` Linus Torvalds
  2020-11-05 22:34   ` Shuah Khan
  0 siblings, 2 replies; 6+ messages in thread
From: Linus Torvalds @ 2020-11-05 20:02 UTC (permalink / raw)
  To: Shuah Khan
  Cc: Brendan Higgins, Linux Kernel Mailing List,
	open list:KERNEL SELFTEST FRAMEWORK

On Thu, Nov 5, 2020 at 8:55 AM Shuah Khan <skhan@linuxfoundation.org> wrote:
>
>   .../kunit/test_data/test_config_printk_time.log    | Bin 1584 -> 1605 bytes
>   .../test_data/test_interrupted_tap_output.log      | Bin 1982 -> 2003 bytes
>   .../test_data/test_kernel_panic_interrupt.log      | Bin 1321 -> 1342 bytes
>   .../kunit/test_data/test_multiple_prefixes.log     | Bin 1832 -> 1861

This diffstat is a bit annoying.

And the reason is that we have

    [torvalds@ryzen linux]$ cat tools/testing/kunit/.gitattributes
    test_data/* binary

iow, you're telling git that everything in that test_data directory is
binary blobs. Which isn't actually true.

This isn't new, but it was just more noticeable this time around.

That binary marker comes from commit afc63da64f1e ("kunit:
kunit_parser: make parser more robust") and I'm not really seeing the
reason for it.

                 Linus

^ permalink raw reply	[flat|nested] 6+ messages in thread
* Re: Kunit fixes update for Linux 5.10-rc3
  2020-11-05 20:02 ` Linus Torvalds
@ 2020-11-05 20:25   ` Linus Torvalds
  2020-11-05 22:01     ` Shuah Khan
  2020-11-05 22:34   ` Shuah Khan
  1 sibling, 1 reply; 6+ messages in thread
From: Linus Torvalds @ 2020-11-05 20:25 UTC (permalink / raw)
  To: Shuah Khan
  Cc: Brendan Higgins, Linux Kernel Mailing List,
	open list:KERNEL SELFTEST FRAMEWORK

On Thu, Nov 5, 2020 at 12:02 PM Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> On Thu, Nov 5, 2020 at 8:55 AM Shuah Khan <skhan@linuxfoundation.org> wrote:
> >
> >   .../kunit/test_data/test_config_printk_time.log    | Bin 1584 -> 1605 bytes
> >   .../test_data/test_interrupted_tap_output.log      | Bin 1982 -> 2003 bytes
> >   .../test_data/test_kernel_panic_interrupt.log      | Bin 1321 -> 1342 bytes
> >   .../kunit/test_data/test_multiple_prefixes.log     | Bin 1832 -> 1861
>
> This diffstat is a bit annoying.

Oh, and another note: because you don't have the "[GIT PULL}" marker
on the subject line, the pr-tracker-bot doesn't react to the pull
requests, and you didn't get notified that it's been pulled.

Just FYI - I personally don't care, as your pull requests do show up
in _my_ queue, because you have "git" and "pull" in the body of the
message. But the automation doesn't see them.

                 Linus

^ permalink raw reply	[flat|nested] 6+ messages in thread
* Re: Kunit fixes update for Linux 5.10-rc3
  2020-11-05 20:25   ` Linus Torvalds
@ 2020-11-05 22:01     ` Shuah Khan
  2020-11-05 22:54       ` Linus Torvalds
  0 siblings, 1 reply; 6+ messages in thread
From: Shuah Khan @ 2020-11-05 22:01 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: Brendan Higgins, Linux Kernel Mailing List,
	open list:KERNEL SELFTEST FRAMEWORK, Shuah Khan

On 11/5/20 1:25 PM, Linus Torvalds wrote:
> On Thu, Nov 5, 2020 at 12:02 PM Linus Torvalds
> <torvalds@linux-foundation.org> wrote:
>>
>> On Thu, Nov 5, 2020 at 8:55 AM Shuah Khan <skhan@linuxfoundation.org> wrote:
>>>
>>>    .../kunit/test_data/test_config_printk_time.log    | Bin 1584 -> 1605 bytes
>>>    .../test_data/test_interrupted_tap_output.log      | Bin 1982 -> 2003 bytes
>>>    .../test_data/test_kernel_panic_interrupt.log      | Bin 1321 -> 1342 bytes
>>>    .../kunit/test_data/test_multiple_prefixes.log     | Bin 1832 -> 1861
>>
>> This diffstat is a bit annoying.
> 
> Oh, and another note: because you don't have the "[GIT PULL}" marker
> on the subject line, the pr-tracker-bot doesn't react to the pull
> requests, and you didn't get notified that it's been pulled.
> 
> Just FYI - I personally don't care, as your pull requests do show up
> in _my_ queue, because you have "git" and "pull" in the body of the
> message. But the automation doesn't see them.
> 
Oops. I don't usually miss adding [GIT PULL] - will pay more attention.

I am sorry about that. My bad on my part. Can I chalk it up
to this weird week?

thanks,
-- Shuah



^ permalink raw reply	[flat|nested] 6+ messages in thread
* Re: Kunit fixes update for Linux 5.10-rc3
  2020-11-05 20:02 ` Linus Torvalds
  2020-11-05 20:25   ` Linus Torvalds
@ 2020-11-05 22:34   ` Shuah Khan
  1 sibling, 0 replies; 6+ messages in thread
From: Shuah Khan @ 2020-11-05 22:34 UTC (permalink / raw)
  To: Linus Torvalds, Brendan Higgins
  Cc: Linux Kernel Mailing List, open list:KERNEL SELFTEST FRAMEWORK

On 11/5/20 1:02 PM, Linus Torvalds wrote:
> On Thu, Nov 5, 2020 at 8:55 AM Shuah Khan <skhan@linuxfoundation.org> wrote:
>>
>>    .../kunit/test_data/test_config_printk_time.log    | Bin 1584 -> 1605 bytes
>>    .../test_data/test_interrupted_tap_output.log      | Bin 1982 -> 2003 bytes
>>    .../test_data/test_kernel_panic_interrupt.log      | Bin 1321 -> 1342 bytes
>>    .../kunit/test_data/test_multiple_prefixes.log     | Bin 1832 -> 1861
> 
> This diffstat is a bit annoying.
> 
> And the reason is that we have
> 
>      [torvalds@ryzen linux]$ cat tools/testing/kunit/.gitattributes
>      test_data/* binary
> 
> iow, you're telling git that everything in that test_data directory is
> binary blobs. Which isn't actually true.
> 
> This isn't new, but it was just more noticeable this time around.
> 
> That binary marker comes from commit afc63da64f1e ("kunit:
> kunit_parser: make parser more robust") and I'm not really seeing the
> reason for it.
> 

You are right. It appears there is no need to add these binary blobs.
Brendan and I discussed this and he will send a fix to remove the blobs
and redo the test output. In other words fix commit afc63da64f1e.

Sorry for not catching this earlier.

thanks,
-- Shuah


^ permalink raw reply	[flat|nested] 6+ messages in thread
* Re: Kunit fixes update for Linux 5.10-rc3
  2020-11-05 22:01     ` Shuah Khan
@ 2020-11-05 22:54       ` Linus Torvalds
  0 siblings, 0 replies; 6+ messages in thread
From: Linus Torvalds @ 2020-11-05 22:54 UTC (permalink / raw)
  To: Shuah Khan
  Cc: Brendan Higgins, Linux Kernel Mailing List,
	open list:KERNEL SELFTEST FRAMEWORK

On Thu, Nov 5, 2020 at 2:01 PM Shuah Khan <skhan@linuxfoundation.org> wrote:
>
> Oops. I don't usually miss adding [GIT PULL] - will pay more attention.

No problem. As mentioned, my own workflow doesn't actually care as
long as the body of the mail has that "please pull" and "git"
mentioned.

And I wouldn't have noticed the pr-tracker-bot reply missing either,
if it wasn't for the fact that I replied to the pull request for other
reasons..

So no worries, just a note on how that "[GIT PULL]" marker might be
helpful for _you_ to see "oh, Linus pulled it, I can forget about it".

            Linus

^ permalink raw reply	[flat|nested] 6+ messages in thread
end of thread, other threads:[~2020-11-05 22:54 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-11-05 16:55 Kunit fixes update for Linux 5.10-rc3 Shuah Khan
2020-11-05 20:02 ` Linus Torvalds
2020-11-05 20:25   ` Linus Torvalds
2020-11-05 22:01     ` Shuah Khan
2020-11-05 22:54       ` Linus Torvalds
2020-11-05 22:34   ` Shuah Khan

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.