* [PATCH 0/2] kselftest: fix rtctest timeout
@ 2019-05-23 22:42 Alexandre Belloni
2019-05-23 22:42 ` [PATCH 1/2] selftests/harness: Allow test to configure timeout Alexandre Belloni
` (2 more replies)
0 siblings, 3 replies; 7+ messages in thread
From: Alexandre Belloni @ 2019-05-23 22:42 UTC (permalink / raw)
To: Shuah Khan
Cc: Kees Cook, Jeffrin Thalakkottoor, linux-kselftest, linux-kernel,
linux-rtc, Alexandre Belloni
Hi,
Commit a745f7af3cbd ("selftests/harness: Add 30 second timeout per
test") wrongly assumed that no individual test would run for more than
30 seconds and this silently broke rtctest.
Please consider the following patches as fixes for v5.2 to avoid having
any non working release.
Thanks,
Alexandre Belloni (2):
selftests/harness: Allow test to configure timeout
selftests: rtc: rtctest: specify timeouts
tools/testing/selftests/kselftest_harness.h | 17 ++++++++++++-----
tools/testing/selftests/rtc/rtctest.c | 6 +++---
2 files changed, 15 insertions(+), 8 deletions(-)
--
2.21.0
^ permalink raw reply [flat|nested] 7+ messages in thread* [PATCH 1/2] selftests/harness: Allow test to configure timeout 2019-05-23 22:42 [PATCH 0/2] kselftest: fix rtctest timeout Alexandre Belloni @ 2019-05-23 22:42 ` Alexandre Belloni 2019-05-23 22:49 ` Kees Cook 2019-05-23 22:42 ` [PATCH 2/2] selftests: rtc: rtctest: specify timeouts Alexandre Belloni 2019-05-23 22:57 ` [PATCH 0/2] kselftest: fix rtctest timeout shuah 2 siblings, 1 reply; 7+ messages in thread From: Alexandre Belloni @ 2019-05-23 22:42 UTC (permalink / raw) To: Shuah Khan Cc: Kees Cook, Jeffrin Thalakkottoor, linux-kselftest, linux-kernel, linux-rtc, Alexandre Belloni Commit a745f7af3cbd ("selftests/harness: Add 30 second timeout per test") adds an hardcoded 30s timeout to all tests. Unfortunately, rtctest has two tests taking up to 60s. Allow for individual tests to define their own timeout. Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> --- tools/testing/selftests/kselftest_harness.h | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/tools/testing/selftests/kselftest_harness.h b/tools/testing/selftests/kselftest_harness.h index 941d9391377f..2067c6b0e8a1 100644 --- a/tools/testing/selftests/kselftest_harness.h +++ b/tools/testing/selftests/kselftest_harness.h @@ -62,6 +62,7 @@ #include <sys/wait.h> #include <unistd.h> +#define TEST_TIMEOUT_DEFAULT 30 /* Utilities exposed to the test definitions */ #ifndef TH_LOG_STREAM @@ -169,7 +170,8 @@ static void test_name(struct __test_metadata *_metadata); \ static struct __test_metadata _##test_name##_object = \ { .name = "global." #test_name, \ - .fn = &test_name, .termsig = _signal }; \ + .fn = &test_name, .termsig = _signal, \ + .timeout = TEST_TIMEOUT_DEFAULT, }; \ static void __attribute__((constructor)) _register_##test_name(void) \ { \ __register_test(&_##test_name##_object); \ @@ -280,12 +282,15 @@ */ /* TODO(wad) register fixtures on dedicated test lists. */ #define TEST_F(fixture_name, test_name) \ - __TEST_F_IMPL(fixture_name, test_name, -1) + __TEST_F_IMPL(fixture_name, test_name, -1, TEST_TIMEOUT_DEFAULT) #define TEST_F_SIGNAL(fixture_name, test_name, signal) \ - __TEST_F_IMPL(fixture_name, test_name, signal) + __TEST_F_IMPL(fixture_name, test_name, signal, TEST_TIMEOUT_DEFAULT) -#define __TEST_F_IMPL(fixture_name, test_name, signal) \ +#define TEST_F_TIMEOUT(fixture_name, test_name, timeout) \ + __TEST_F_IMPL(fixture_name, test_name, -1, timeout) + +#define __TEST_F_IMPL(fixture_name, test_name, signal, tmout) \ static void fixture_name##_##test_name( \ struct __test_metadata *_metadata, \ FIXTURE_DATA(fixture_name) *self); \ @@ -307,6 +312,7 @@ .name = #fixture_name "." #test_name, \ .fn = &wrapper_##fixture_name##_##test_name, \ .termsig = signal, \ + .timeout = tmout, \ }; \ static void __attribute__((constructor)) \ _register_##fixture_name##_##test_name(void) \ @@ -632,6 +638,7 @@ struct __test_metadata { int termsig; int passed; int trigger; /* extra handler after the evaluation */ + int timeout; __u8 step; bool no_print; /* manual trigger when TH_LOG_STREAM is not available */ struct __test_metadata *prev, *next; @@ -696,7 +703,7 @@ void __run_test(struct __test_metadata *t) t->passed = 1; t->trigger = 0; printf("[ RUN ] %s\n", t->name); - alarm(30); + alarm(t->timeout); child_pid = fork(); if (child_pid < 0) { printf("ERROR SPAWNING TEST CHILD\n"); -- 2.21.0 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH 1/2] selftests/harness: Allow test to configure timeout 2019-05-23 22:42 ` [PATCH 1/2] selftests/harness: Allow test to configure timeout Alexandre Belloni @ 2019-05-23 22:49 ` Kees Cook 0 siblings, 0 replies; 7+ messages in thread From: Kees Cook @ 2019-05-23 22:49 UTC (permalink / raw) To: Alexandre Belloni Cc: Shuah Khan, Jeffrin Thalakkottoor, linux-kselftest, linux-kernel, linux-rtc On Fri, May 24, 2019 at 12:42:22AM +0200, Alexandre Belloni wrote: > Commit a745f7af3cbd ("selftests/harness: Add 30 second timeout per test") > adds an hardcoded 30s timeout to all tests. Unfortunately, rtctest has two > tests taking up to 60s. Allow for individual tests to define their own > timeout. > > Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Ah yes. Very nice; thanks! Reviewed-by: Kees Cook <keescook@chromium.org> > --- > tools/testing/selftests/kselftest_harness.h | 17 ++++++++++++----- > 1 file changed, 12 insertions(+), 5 deletions(-) > > diff --git a/tools/testing/selftests/kselftest_harness.h b/tools/testing/selftests/kselftest_harness.h > index 941d9391377f..2067c6b0e8a1 100644 > --- a/tools/testing/selftests/kselftest_harness.h > +++ b/tools/testing/selftests/kselftest_harness.h > @@ -62,6 +62,7 @@ > #include <sys/wait.h> > #include <unistd.h> > > +#define TEST_TIMEOUT_DEFAULT 30 > > /* Utilities exposed to the test definitions */ > #ifndef TH_LOG_STREAM > @@ -169,7 +170,8 @@ > static void test_name(struct __test_metadata *_metadata); \ > static struct __test_metadata _##test_name##_object = \ > { .name = "global." #test_name, \ > - .fn = &test_name, .termsig = _signal }; \ > + .fn = &test_name, .termsig = _signal, \ > + .timeout = TEST_TIMEOUT_DEFAULT, }; \ > static void __attribute__((constructor)) _register_##test_name(void) \ > { \ > __register_test(&_##test_name##_object); \ > @@ -280,12 +282,15 @@ > */ > /* TODO(wad) register fixtures on dedicated test lists. */ > #define TEST_F(fixture_name, test_name) \ > - __TEST_F_IMPL(fixture_name, test_name, -1) > + __TEST_F_IMPL(fixture_name, test_name, -1, TEST_TIMEOUT_DEFAULT) > > #define TEST_F_SIGNAL(fixture_name, test_name, signal) \ > - __TEST_F_IMPL(fixture_name, test_name, signal) > + __TEST_F_IMPL(fixture_name, test_name, signal, TEST_TIMEOUT_DEFAULT) > > -#define __TEST_F_IMPL(fixture_name, test_name, signal) \ > +#define TEST_F_TIMEOUT(fixture_name, test_name, timeout) \ > + __TEST_F_IMPL(fixture_name, test_name, -1, timeout) > + > +#define __TEST_F_IMPL(fixture_name, test_name, signal, tmout) \ > static void fixture_name##_##test_name( \ > struct __test_metadata *_metadata, \ > FIXTURE_DATA(fixture_name) *self); \ > @@ -307,6 +312,7 @@ > .name = #fixture_name "." #test_name, \ > .fn = &wrapper_##fixture_name##_##test_name, \ > .termsig = signal, \ > + .timeout = tmout, \ > }; \ > static void __attribute__((constructor)) \ > _register_##fixture_name##_##test_name(void) \ > @@ -632,6 +638,7 @@ struct __test_metadata { > int termsig; > int passed; > int trigger; /* extra handler after the evaluation */ > + int timeout; > __u8 step; > bool no_print; /* manual trigger when TH_LOG_STREAM is not available */ > struct __test_metadata *prev, *next; > @@ -696,7 +703,7 @@ void __run_test(struct __test_metadata *t) > t->passed = 1; > t->trigger = 0; > printf("[ RUN ] %s\n", t->name); > - alarm(30); > + alarm(t->timeout); > child_pid = fork(); > if (child_pid < 0) { > printf("ERROR SPAWNING TEST CHILD\n"); > -- > 2.21.0 > -- Kees Cook ^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH 2/2] selftests: rtc: rtctest: specify timeouts 2019-05-23 22:42 [PATCH 0/2] kselftest: fix rtctest timeout Alexandre Belloni 2019-05-23 22:42 ` [PATCH 1/2] selftests/harness: Allow test to configure timeout Alexandre Belloni @ 2019-05-23 22:42 ` Alexandre Belloni 2019-05-23 22:50 ` Kees Cook 2019-05-23 22:57 ` [PATCH 0/2] kselftest: fix rtctest timeout shuah 2 siblings, 1 reply; 7+ messages in thread From: Alexandre Belloni @ 2019-05-23 22:42 UTC (permalink / raw) To: Shuah Khan Cc: Kees Cook, Jeffrin Thalakkottoor, linux-kselftest, linux-kernel, linux-rtc, Alexandre Belloni uie_read is a commonly failing test that will block forever on buggy rtc drivers. Shorten its timeout so it fails earlier. Also increase the timeout for the two alarm test on a minute boundary. Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> --- tools/testing/selftests/rtc/rtctest.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/tools/testing/selftests/rtc/rtctest.c b/tools/testing/selftests/rtc/rtctest.c index b2065536d407..66af608fb4c6 100644 --- a/tools/testing/selftests/rtc/rtctest.c +++ b/tools/testing/selftests/rtc/rtctest.c @@ -49,7 +49,7 @@ TEST_F(rtc, date_read) { rtc_tm.tm_hour, rtc_tm.tm_min, rtc_tm.tm_sec); } -TEST_F(rtc, uie_read) { +TEST_F_TIMEOUT(rtc, uie_read, NUM_UIE + 2) { int i, rc, irq = 0; unsigned long data; @@ -211,7 +211,7 @@ TEST_F(rtc, alarm_wkalm_set) { ASSERT_EQ(new, secs); } -TEST_F(rtc, alarm_alm_set_minute) { +TEST_F_TIMEOUT(rtc, alarm_alm_set_minute, 65) { struct timeval tv = { .tv_sec = 62 }; unsigned long data; struct rtc_time tm; @@ -264,7 +264,7 @@ TEST_F(rtc, alarm_alm_set_minute) { ASSERT_EQ(new, secs); } -TEST_F(rtc, alarm_wkalm_set_minute) { +TEST_F_TIMEOUT(rtc, alarm_wkalm_set_minute, 65) { struct timeval tv = { .tv_sec = 62 }; struct rtc_wkalrm alarm = { 0 }; struct rtc_time tm; -- 2.21.0 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH 2/2] selftests: rtc: rtctest: specify timeouts 2019-05-23 22:42 ` [PATCH 2/2] selftests: rtc: rtctest: specify timeouts Alexandre Belloni @ 2019-05-23 22:50 ` Kees Cook 0 siblings, 0 replies; 7+ messages in thread From: Kees Cook @ 2019-05-23 22:50 UTC (permalink / raw) To: Alexandre Belloni Cc: Shuah Khan, Jeffrin Thalakkottoor, linux-kselftest, linux-kernel, linux-rtc On Fri, May 24, 2019 at 12:42:23AM +0200, Alexandre Belloni wrote: > uie_read is a commonly failing test that will block forever on buggy rtc > drivers. Shorten its timeout so it fails earlier. Also increase the timeout > for the two alarm test on a minute boundary. > > Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Reviewed-by: Kees Cook <keescook@chromium.org> > --- > tools/testing/selftests/rtc/rtctest.c | 6 +++--- > 1 file changed, 3 insertions(+), 3 deletions(-) > > diff --git a/tools/testing/selftests/rtc/rtctest.c b/tools/testing/selftests/rtc/rtctest.c > index b2065536d407..66af608fb4c6 100644 > --- a/tools/testing/selftests/rtc/rtctest.c > +++ b/tools/testing/selftests/rtc/rtctest.c > @@ -49,7 +49,7 @@ TEST_F(rtc, date_read) { > rtc_tm.tm_hour, rtc_tm.tm_min, rtc_tm.tm_sec); > } > > -TEST_F(rtc, uie_read) { > +TEST_F_TIMEOUT(rtc, uie_read, NUM_UIE + 2) { > int i, rc, irq = 0; > unsigned long data; > > @@ -211,7 +211,7 @@ TEST_F(rtc, alarm_wkalm_set) { > ASSERT_EQ(new, secs); > } > > -TEST_F(rtc, alarm_alm_set_minute) { > +TEST_F_TIMEOUT(rtc, alarm_alm_set_minute, 65) { > struct timeval tv = { .tv_sec = 62 }; > unsigned long data; > struct rtc_time tm; > @@ -264,7 +264,7 @@ TEST_F(rtc, alarm_alm_set_minute) { > ASSERT_EQ(new, secs); > } > > -TEST_F(rtc, alarm_wkalm_set_minute) { > +TEST_F_TIMEOUT(rtc, alarm_wkalm_set_minute, 65) { > struct timeval tv = { .tv_sec = 62 }; > struct rtc_wkalrm alarm = { 0 }; > struct rtc_time tm; > -- > 2.21.0 > -- Kees Cook ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH 0/2] kselftest: fix rtctest timeout 2019-05-23 22:42 [PATCH 0/2] kselftest: fix rtctest timeout Alexandre Belloni 2019-05-23 22:42 ` [PATCH 1/2] selftests/harness: Allow test to configure timeout Alexandre Belloni 2019-05-23 22:42 ` [PATCH 2/2] selftests: rtc: rtctest: specify timeouts Alexandre Belloni @ 2019-05-23 22:57 ` shuah 2019-05-23 23:32 ` shuah 2 siblings, 1 reply; 7+ messages in thread From: shuah @ 2019-05-23 22:57 UTC (permalink / raw) To: Alexandre Belloni Cc: Kees Cook, Jeffrin Thalakkottoor, linux-kselftest, linux-kernel, linux-rtc, shuah On 5/23/19 4:42 PM, Alexandre Belloni wrote: > Hi, > > Commit a745f7af3cbd ("selftests/harness: Add 30 second timeout per > test") wrongly assumed that no individual test would run for more than > 30 seconds and this silently broke rtctest. > > Please consider the following patches as fixes for v5.2 to avoid having > any non working release. > > Thanks, > > Alexandre Belloni (2): > selftests/harness: Allow test to configure timeout > selftests: rtc: rtctest: specify timeouts > > tools/testing/selftests/kselftest_harness.h | 17 ++++++++++++----- > tools/testing/selftests/rtc/rtctest.c | 6 +++--- > 2 files changed, 15 insertions(+), 8 deletions(-) > Thanks for fixing them quickly. I will pull these in. I have one more fix from Kees already queued up. Jeffrin! Would you like to test these to see if they work for you and send Tested-by tag. I don't see 1/2 in my Inbox. I have Kees's reply to it. Odd. thanks, -- Shuah ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH 0/2] kselftest: fix rtctest timeout 2019-05-23 22:57 ` [PATCH 0/2] kselftest: fix rtctest timeout shuah @ 2019-05-23 23:32 ` shuah 0 siblings, 0 replies; 7+ messages in thread From: shuah @ 2019-05-23 23:32 UTC (permalink / raw) To: Alexandre Belloni Cc: Kees Cook, Jeffrin Thalakkottoor, linux-kselftest, linux-kernel, linux-rtc, shuah On 5/23/19 4:57 PM, shuah wrote: > On 5/23/19 4:42 PM, Alexandre Belloni wrote: >> Hi, >> >> Commit a745f7af3cbd ("selftests/harness: Add 30 second timeout per >> test") wrongly assumed that no individual test would run for more than >> 30 seconds and this silently broke rtctest. >> >> Please consider the following patches as fixes for v5.2 to avoid having >> any non working release. >> >> Thanks, >> >> Alexandre Belloni (2): >> selftests/harness: Allow test to configure timeout >> selftests: rtc: rtctest: specify timeouts >> >> tools/testing/selftests/kselftest_harness.h | 17 ++++++++++++----- >> tools/testing/selftests/rtc/rtctest.c | 6 +++--- >> 2 files changed, 15 insertions(+), 8 deletions(-) >> > > Thanks for fixing them quickly. > > I will pull these in. I have one more fix from Kees already queued > up. > > Jeffrin! Would you like to test these to see if they work for you > and send Tested-by tag. > > I don't see 1/2 in my Inbox. I have Kees's reply to it. Odd. > It showed up late. I have 1/2 in my Inbox now. Must have taken a scenic route. :) thanks, -- Shuah ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2019-05-23 23:32 UTC | newest] Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-05-23 22:42 [PATCH 0/2] kselftest: fix rtctest timeout Alexandre Belloni 2019-05-23 22:42 ` [PATCH 1/2] selftests/harness: Allow test to configure timeout Alexandre Belloni 2019-05-23 22:49 ` Kees Cook 2019-05-23 22:42 ` [PATCH 2/2] selftests: rtc: rtctest: specify timeouts Alexandre Belloni 2019-05-23 22:50 ` Kees Cook 2019-05-23 22:57 ` [PATCH 0/2] kselftest: fix rtctest timeout shuah 2019-05-23 23:32 ` shuah
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).