Linux-kselftest Archive on lore.kernel.org
 help / color / Atom feed
* [PATCH v1] kunit: tools: fix kunit_tool tests for parsing test plans
@ 2020-10-21 20:39 Brendan Higgins
  2020-10-22  2:07 ` David Gow
  0 siblings, 1 reply; 2+ messages in thread
From: Brendan Higgins @ 2020-10-21 20:39 UTC (permalink / raw)
  To: shuah, davidgow; +Cc: linux-kselftest, kunit-dev, linux-kernel, Brendan Higgins

Some tests logs for kunit_tool tests are missing their test plans
causing their tests to fail; fix this by adding the test plans.

Fixes: 45dcbb6f5ef7 ("kunit: test: add test plan to KUnit TAP format")
Signed-off-by: Brendan Higgins <brendanhiggins@google.com>
---
 tools/testing/kunit/kunit_tool_test.py        |  32 ++++++++++++++----
 .../test_data/test_config_printk_time.log     | Bin 1584 -> 1605 bytes
 .../test_data/test_interrupted_tap_output.log | Bin 1982 -> 2003 bytes
 .../test_data/test_kernel_panic_interrupt.log | Bin 1321 -> 1342 bytes
 .../test_data/test_multiple_prefixes.log      | Bin 1832 -> 1861 bytes
 .../kunit/test_data/test_pound_no_prefix.log  | Bin 1193 -> 1200 bytes
 .../kunit/test_data/test_pound_sign.log       | Bin 1656 -> 1676 bytes
 7 files changed, 25 insertions(+), 7 deletions(-)

diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py
index 99c3c5671ea48..0b60855fb8198 100755
--- a/tools/testing/kunit/kunit_tool_test.py
+++ b/tools/testing/kunit/kunit_tool_test.py
@@ -179,7 +179,7 @@ class KUnitParserTest(unittest.TestCase):
 		print_mock = mock.patch('builtins.print').start()
 		result = kunit_parser.parse_run_tests(
 			kunit_parser.isolate_kunit_output(file.readlines()))
-		print_mock.assert_any_call(StrContains("no kunit output detected"))
+		print_mock.assert_any_call(StrContains('no tests run!'))
 		print_mock.stop()
 		file.close()
 
@@ -198,39 +198,57 @@ class KUnitParserTest(unittest.TestCase):
 			'test_data/test_config_printk_time.log')
 		with open(prefix_log) as file:
 			result = kunit_parser.parse_run_tests(file.readlines())
-		self.assertEqual('kunit-resource-test', result.suites[0].name)
+			self.assertEqual(
+				kunit_parser.TestStatus.SUCCESS,
+				result.status)
+			self.assertEqual('kunit-resource-test', result.suites[0].name)
 
 	def test_ignores_multiple_prefixes(self):
 		prefix_log = get_absolute_path(
 			'test_data/test_multiple_prefixes.log')
 		with open(prefix_log) as file:
 			result = kunit_parser.parse_run_tests(file.readlines())
-		self.assertEqual('kunit-resource-test', result.suites[0].name)
+			self.assertEqual(
+				kunit_parser.TestStatus.SUCCESS,
+				result.status)
+			self.assertEqual('kunit-resource-test', result.suites[0].name)
 
 	def test_prefix_mixed_kernel_output(self):
 		mixed_prefix_log = get_absolute_path(
 			'test_data/test_interrupted_tap_output.log')
 		with open(mixed_prefix_log) as file:
 			result = kunit_parser.parse_run_tests(file.readlines())
-		self.assertEqual('kunit-resource-test', result.suites[0].name)
+			self.assertEqual(
+				kunit_parser.TestStatus.SUCCESS,
+				result.status)
+			self.assertEqual('kunit-resource-test', result.suites[0].name)
 
 	def test_prefix_poundsign(self):
 		pound_log = get_absolute_path('test_data/test_pound_sign.log')
 		with open(pound_log) as file:
 			result = kunit_parser.parse_run_tests(file.readlines())
-		self.assertEqual('kunit-resource-test', result.suites[0].name)
+			self.assertEqual(
+				kunit_parser.TestStatus.SUCCESS,
+				result.status)
+			self.assertEqual('kunit-resource-test', result.suites[0].name)
 
 	def test_kernel_panic_end(self):
 		panic_log = get_absolute_path('test_data/test_kernel_panic_interrupt.log')
 		with open(panic_log) as file:
 			result = kunit_parser.parse_run_tests(file.readlines())
-		self.assertEqual('kunit-resource-test', result.suites[0].name)
+			self.assertEqual(
+				kunit_parser.TestStatus.TEST_CRASHED,
+				result.status)
+			self.assertEqual('kunit-resource-test', result.suites[0].name)
 
 	def test_pound_no_prefix(self):
 		pound_log = get_absolute_path('test_data/test_pound_no_prefix.log')
 		with open(pound_log) as file:
 			result = kunit_parser.parse_run_tests(file.readlines())
-		self.assertEqual('kunit-resource-test', result.suites[0].name)
+			self.assertEqual(
+				kunit_parser.TestStatus.SUCCESS,
+				result.status)
+			self.assertEqual('kunit-resource-test', result.suites[0].name)
 
 class KUnitJsonTest(unittest.TestCase):
 
diff --git a/tools/testing/kunit/test_data/test_config_printk_time.log b/tools/testing/kunit/test_data/test_config_printk_time.log
index c02ca773946d641291e27d44d73174cc16a17d9d..6bdb57f76eacef0396e68942cb3fa983b6992bab 100644
GIT binary patch
delta 25
hcmdnMbChSob{0cDJ>$uC%rcYXnAkR+OlM`}0sw1)2Xg=b

delta 10
RcmX@gvw>&A_Kl}2Spgdh1kV5f

diff --git a/tools/testing/kunit/test_data/test_interrupted_tap_output.log b/tools/testing/kunit/test_data/test_interrupted_tap_output.log
index 5c73fb3a1c6fd13a9b163a48d02eb33b0315a375..1fb677728abeb0fe6aa5edb3a0387c05906b9815 100644
GIT binary patch
delta 21
dcmdnTf0=*6b{0cDJ>$t2n3Xo3{L0421prz=2g3jW

delta 17
Zcmcc2zmI>y_Q@YucqXy1ZM^%H4FE=v2d@AC

diff --git a/tools/testing/kunit/test_data/test_kernel_panic_interrupt.log b/tools/testing/kunit/test_data/test_kernel_panic_interrupt.log
index c045eee75f27fefaabf3ba073d9e282721c19a67..a014ffe9725e3c4e81697f742cee2652a41b2108 100644
GIT binary patch
delta 25
hcmZ3<wU2AUb{0cDJ>$uC%rcYXnAkR+OkiQ;0sv{Y2U!3B

delta 10
RcmdnTwUTSX_Kl~DSO6Mc1hoJF

diff --git a/tools/testing/kunit/test_data/test_multiple_prefixes.log b/tools/testing/kunit/test_data/test_multiple_prefixes.log
index bc48407dcc36c44665c7d2ac620e42e7caf98481..0ad78481a0b450bf463ca7aaab0ae739d4e43018 100644
GIT binary patch
delta 16
YcmZ3%ca(3!^~t;}vKwy%urYE005`P-oB#j-

delta 15
XcmX@gw}NlN^~vX$gf_D>+Oq)wG*bm+

diff --git a/tools/testing/kunit/test_data/test_pound_no_prefix.log b/tools/testing/kunit/test_data/test_pound_no_prefix.log
index 2ceb360be7d52cbee4b0a5a426456605ded7466e..dc4cf09a96d077bbc0b5fbcb312251742e390379 100644
GIT binary patch
delta 17
YcmZ3<xq)*+J&U29p7F+79Tr9|05Q}Ar~m)}

delta 10
RcmdnMxsr22{l-=;762DW1K$7u

diff --git a/tools/testing/kunit/test_data/test_pound_sign.log b/tools/testing/kunit/test_data/test_pound_sign.log
index 28ffa5ba03bfa81ea02ea9d38e7de7acf3dd9e5d..3f358e3a7ba0d118c1dc15e5f637fb8ffb5aa388 100644
GIT binary patch
delta 19
bcmeyt)5AMqJBy*7p7G>PCZ&xh*RuiuM}Y?y

delta 14
WcmeC-{lPO~`{cDuLL2X{X9WN;2?l!r


base-commit: 7cf726a59435301046250c42131554d9ccc566b8
-- 
2.29.0.rc1.297.gfa9743e501-goog


^ permalink raw reply	[flat|nested] 2+ messages in thread

* Re: [PATCH v1] kunit: tools: fix kunit_tool tests for parsing test plans
  2020-10-21 20:39 [PATCH v1] kunit: tools: fix kunit_tool tests for parsing test plans Brendan Higgins
@ 2020-10-22  2:07 ` David Gow
  0 siblings, 0 replies; 2+ messages in thread
From: David Gow @ 2020-10-22  2:07 UTC (permalink / raw)
  To: Brendan Higgins
  Cc: Shuah Khan, open list:KERNEL SELFTEST FRAMEWORK,
	KUnit Development, Linux Kernel Mailing List

On Thu, Oct 22, 2020 at 4:39 AM Brendan Higgins
<brendanhiggins@google.com> wrote:
>
> Some tests logs for kunit_tool tests are missing their test plans
> causing their tests to fail; fix this by adding the test plans.
>
> Fixes: 45dcbb6f5ef7 ("kunit: test: add test plan to KUnit TAP format")
> Signed-off-by: Brendan Higgins <brendanhiggins@google.com>

Excellent: this fixes all of the failing tests in kunit_tool_test.py for me.

Reviewed-by: David Gow <davidgow@google.com>

-- David

> ---
>  tools/testing/kunit/kunit_tool_test.py        |  32 ++++++++++++++----
>  .../test_data/test_config_printk_time.log     | Bin 1584 -> 1605 bytes
>  .../test_data/test_interrupted_tap_output.log | Bin 1982 -> 2003 bytes
>  .../test_data/test_kernel_panic_interrupt.log | Bin 1321 -> 1342 bytes
>  .../test_data/test_multiple_prefixes.log      | Bin 1832 -> 1861 bytes
>  .../kunit/test_data/test_pound_no_prefix.log  | Bin 1193 -> 1200 bytes
>  .../kunit/test_data/test_pound_sign.log       | Bin 1656 -> 1676 bytes
>  7 files changed, 25 insertions(+), 7 deletions(-)
>
> diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py
> index 99c3c5671ea48..0b60855fb8198 100755
> --- a/tools/testing/kunit/kunit_tool_test.py
> +++ b/tools/testing/kunit/kunit_tool_test.py
> @@ -179,7 +179,7 @@ class KUnitParserTest(unittest.TestCase):
>                 print_mock = mock.patch('builtins.print').start()
>                 result = kunit_parser.parse_run_tests(
>                         kunit_parser.isolate_kunit_output(file.readlines()))
> -               print_mock.assert_any_call(StrContains("no kunit output detected"))
> +               print_mock.assert_any_call(StrContains('no tests run!'))
>                 print_mock.stop()
>                 file.close()
>
> @@ -198,39 +198,57 @@ class KUnitParserTest(unittest.TestCase):
>                         'test_data/test_config_printk_time.log')
>                 with open(prefix_log) as file:
>                         result = kunit_parser.parse_run_tests(file.readlines())
> -               self.assertEqual('kunit-resource-test', result.suites[0].name)
> +                       self.assertEqual(
> +                               kunit_parser.TestStatus.SUCCESS,
> +                               result.status)
> +                       self.assertEqual('kunit-resource-test', result.suites[0].name)
>
>         def test_ignores_multiple_prefixes(self):
>                 prefix_log = get_absolute_path(
>                         'test_data/test_multiple_prefixes.log')
>                 with open(prefix_log) as file:
>                         result = kunit_parser.parse_run_tests(file.readlines())
> -               self.assertEqual('kunit-resource-test', result.suites[0].name)
> +                       self.assertEqual(
> +                               kunit_parser.TestStatus.SUCCESS,
> +                               result.status)
> +                       self.assertEqual('kunit-resource-test', result.suites[0].name)
>
>         def test_prefix_mixed_kernel_output(self):
>                 mixed_prefix_log = get_absolute_path(
>                         'test_data/test_interrupted_tap_output.log')
>                 with open(mixed_prefix_log) as file:
>                         result = kunit_parser.parse_run_tests(file.readlines())
> -               self.assertEqual('kunit-resource-test', result.suites[0].name)
> +                       self.assertEqual(
> +                               kunit_parser.TestStatus.SUCCESS,
> +                               result.status)
> +                       self.assertEqual('kunit-resource-test', result.suites[0].name)
>
>         def test_prefix_poundsign(self):
>                 pound_log = get_absolute_path('test_data/test_pound_sign.log')
>                 with open(pound_log) as file:
>                         result = kunit_parser.parse_run_tests(file.readlines())
> -               self.assertEqual('kunit-resource-test', result.suites[0].name)
> +                       self.assertEqual(
> +                               kunit_parser.TestStatus.SUCCESS,
> +                               result.status)
> +                       self.assertEqual('kunit-resource-test', result.suites[0].name)
>
>         def test_kernel_panic_end(self):
>                 panic_log = get_absolute_path('test_data/test_kernel_panic_interrupt.log')
>                 with open(panic_log) as file:
>                         result = kunit_parser.parse_run_tests(file.readlines())
> -               self.assertEqual('kunit-resource-test', result.suites[0].name)
> +                       self.assertEqual(
> +                               kunit_parser.TestStatus.TEST_CRASHED,
> +                               result.status)
> +                       self.assertEqual('kunit-resource-test', result.suites[0].name)
>
>         def test_pound_no_prefix(self):
>                 pound_log = get_absolute_path('test_data/test_pound_no_prefix.log')
>                 with open(pound_log) as file:
>                         result = kunit_parser.parse_run_tests(file.readlines())
> -               self.assertEqual('kunit-resource-test', result.suites[0].name)
> +                       self.assertEqual(
> +                               kunit_parser.TestStatus.SUCCESS,
> +                               result.status)
> +                       self.assertEqual('kunit-resource-test', result.suites[0].name)
>
>  class KUnitJsonTest(unittest.TestCase):
>
> diff --git a/tools/testing/kunit/test_data/test_config_printk_time.log b/tools/testing/kunit/test_data/test_config_printk_time.log
> index c02ca773946d641291e27d44d73174cc16a17d9d..6bdb57f76eacef0396e68942cb3fa983b6992bab 100644
> GIT binary patch
> delta 25
> hcmdnMbChSob{0cDJ>$uC%rcYXnAkR+OlM`}0sw1)2Xg=b
>
> delta 10
> RcmX@gvw>&A_Kl}2Spgdh1kV5f
>
> diff --git a/tools/testing/kunit/test_data/test_interrupted_tap_output.log b/tools/testing/kunit/test_data/test_interrupted_tap_output.log
> index 5c73fb3a1c6fd13a9b163a48d02eb33b0315a375..1fb677728abeb0fe6aa5edb3a0387c05906b9815 100644
> GIT binary patch
> delta 21
> dcmdnTf0=*6b{0cDJ>$t2n3Xo3{L0421prz=2g3jW
>
> delta 17
> Zcmcc2zmI>y_Q@YucqXy1ZM^%H4FE=v2d@AC
>
> diff --git a/tools/testing/kunit/test_data/test_kernel_panic_interrupt.log b/tools/testing/kunit/test_data/test_kernel_panic_interrupt.log
> index c045eee75f27fefaabf3ba073d9e282721c19a67..a014ffe9725e3c4e81697f742cee2652a41b2108 100644
> GIT binary patch
> delta 25
> hcmZ3<wU2AUb{0cDJ>$uC%rcYXnAkR+OkiQ;0sv{Y2U!3B
>
> delta 10
> RcmdnTwUTSX_Kl~DSO6Mc1hoJF
>
> diff --git a/tools/testing/kunit/test_data/test_multiple_prefixes.log b/tools/testing/kunit/test_data/test_multiple_prefixes.log
> index bc48407dcc36c44665c7d2ac620e42e7caf98481..0ad78481a0b450bf463ca7aaab0ae739d4e43018 100644
> GIT binary patch
> delta 16
> YcmZ3%ca(3!^~t;}vKwy%urYE005`P-oB#j-
>
> delta 15
> XcmX@gw}NlN^~vX$gf_D>+Oq)wG*bm+
>
> diff --git a/tools/testing/kunit/test_data/test_pound_no_prefix.log b/tools/testing/kunit/test_data/test_pound_no_prefix.log
> index 2ceb360be7d52cbee4b0a5a426456605ded7466e..dc4cf09a96d077bbc0b5fbcb312251742e390379 100644
> GIT binary patch
> delta 17
> YcmZ3<xq)*+J&U29p7F+79Tr9|05Q}Ar~m)}
>
> delta 10
> RcmdnMxsr22{l-=;762DW1K$7u
>
> diff --git a/tools/testing/kunit/test_data/test_pound_sign.log b/tools/testing/kunit/test_data/test_pound_sign.log
> index 28ffa5ba03bfa81ea02ea9d38e7de7acf3dd9e5d..3f358e3a7ba0d118c1dc15e5f637fb8ffb5aa388 100644
> GIT binary patch
> delta 19
> bcmeyt)5AMqJBy*7p7G>PCZ&xh*RuiuM}Y?y
>
> delta 14
> WcmeC-{lPO~`{cDuLL2X{X9WN;2?l!r
>
>
> base-commit: 7cf726a59435301046250c42131554d9ccc566b8
> --
> 2.29.0.rc1.297.gfa9743e501-goog
>

^ permalink raw reply	[flat|nested] 2+ messages in thread

end of thread, back to index

Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-10-21 20:39 [PATCH v1] kunit: tools: fix kunit_tool tests for parsing test plans Brendan Higgins
2020-10-22  2:07 ` David Gow

Linux-kselftest Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-kselftest/0 linux-kselftest/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-kselftest linux-kselftest/ https://lore.kernel.org/linux-kselftest \
		linux-kselftest@vger.kernel.org
	public-inbox-index linux-kselftest

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-kselftest


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git