linux-modules.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH] testsuite: Add facility to skip tests.
@ 2020-05-16 12:54 Marius Bakke
  2020-07-08 18:16 ` Lucas De Marchi
  0 siblings, 1 reply; 6+ messages in thread
From: Marius Bakke @ 2020-05-16 12:54 UTC (permalink / raw)
  To: linux-modules

The Makefile helpfully warns that some tests will fail when
--sysconfdir != /etc, but there are no provisions to easily disable
those.  This commit provides an escape hatch.
---
 testsuite/testsuite.c | 9 +++++++++
 testsuite/testsuite.h | 1 +
 2 files changed, 10 insertions(+)

diff --git a/testsuite/testsuite.c b/testsuite/testsuite.c
index e46f3d8..ff41057 100644
--- a/testsuite/testsuite.c
+++ b/testsuite/testsuite.c
@@ -37,6 +37,7 @@
 #include "testsuite.h"
 
 static const char *ANSI_HIGHLIGHT_GREEN_ON = "\x1B[1;32m";
+static const char *ANSI_HIGHLIGHT_YELLOW_ON = "\x1B[1;33m";
 static const char *ANSI_HIGHLIGHT_RED_ON =  "\x1B[1;31m";
 static const char *ANSI_HIGHLIGHT_OFF = "\x1B[0m";
 
@@ -948,6 +949,14 @@ static inline int test_run_parent(const struct test *t, int fdout[2],
 	int err;
 	bool matchout, match_modules;
 
+	if (t->skip == true) {
+		LOG("%sSKIPPED%s: %s\n",
+			ANSI_HIGHLIGHT_YELLOW_ON, ANSI_HIGHLIGHT_OFF,
+			t->name);
+		err = EXIT_SUCCESS;
+		goto exit;
+	}
+
 	/* Close write-fds */
 	if (t->output.out != NULL)
 		close(fdout[1]);
diff --git a/testsuite/testsuite.h b/testsuite/testsuite.h
index 7ed96bf..8029c64 100644
--- a/testsuite/testsuite.h
+++ b/testsuite/testsuite.h
@@ -109,6 +109,7 @@ struct test {
 	const struct keyval *env_vars;
 	bool need_spawn;
 	bool expected_fail;
+	bool skip;
 	bool print_outputs;
 } __attribute__((aligned(8)));
 
-- 
2.26.2


^ permalink raw reply related	[flat|nested] 6+ messages in thread

* Re: [PATCH] testsuite: Add facility to skip tests.
  2020-05-16 12:54 [PATCH] testsuite: Add facility to skip tests Marius Bakke
@ 2020-07-08 18:16 ` Lucas De Marchi
  2020-08-01 15:58   ` Marius Bakke
  0 siblings, 1 reply; 6+ messages in thread
From: Lucas De Marchi @ 2020-07-08 18:16 UTC (permalink / raw)
  To: Marius Bakke; +Cc: linux-modules

On Sat, May 16, 2020 at 6:04 AM Marius Bakke <marius@devup.no> wrote:
>
> The Makefile helpfully warns that some tests will fail when
> --sysconfdir != /etc, but there are no provisions to easily disable
> those.  This commit provides an escape hatch.

nice... but are we missing a patch to actually make it skip?

> ---
>  testsuite/testsuite.c | 9 +++++++++
>  testsuite/testsuite.h | 1 +
>  2 files changed, 10 insertions(+)
>
> diff --git a/testsuite/testsuite.c b/testsuite/testsuite.c
> index e46f3d8..ff41057 100644
> --- a/testsuite/testsuite.c
> +++ b/testsuite/testsuite.c
> @@ -37,6 +37,7 @@
>  #include "testsuite.h"
>
>  static const char *ANSI_HIGHLIGHT_GREEN_ON = "\x1B[1;32m";
> +static const char *ANSI_HIGHLIGHT_YELLOW_ON = "\x1B[1;33m";
>  static const char *ANSI_HIGHLIGHT_RED_ON =  "\x1B[1;31m";
>  static const char *ANSI_HIGHLIGHT_OFF = "\x1B[0m";
>
> @@ -948,6 +949,14 @@ static inline int test_run_parent(const struct test *t, int fdout[2],
>         int err;
>         bool matchout, match_modules;
>
> +       if (t->skip == true) {

only if (t->skip)  would be less verbose and preferred I think.

thanks
Lucas de Marchi

> +               LOG("%sSKIPPED%s: %s\n",
> +                       ANSI_HIGHLIGHT_YELLOW_ON, ANSI_HIGHLIGHT_OFF,
> +                       t->name);
> +               err = EXIT_SUCCESS;
> +               goto exit;
> +       }
> +
>         /* Close write-fds */
>         if (t->output.out != NULL)
>                 close(fdout[1]);
> diff --git a/testsuite/testsuite.h b/testsuite/testsuite.h
> index 7ed96bf..8029c64 100644
> --- a/testsuite/testsuite.h
> +++ b/testsuite/testsuite.h
> @@ -109,6 +109,7 @@ struct test {
>         const struct keyval *env_vars;
>         bool need_spawn;
>         bool expected_fail;
> +       bool skip;
>         bool print_outputs;
>  } __attribute__((aligned(8)));
>
> --
> 2.26.2
>


-- 
Lucas De Marchi

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH] testsuite: Add facility to skip tests.
  2020-07-08 18:16 ` Lucas De Marchi
@ 2020-08-01 15:58   ` Marius Bakke
  2020-08-01 16:02     ` [PATCH v2 1/2] " Marius Bakke
  2020-08-01 16:02     ` [PATCH v2 2/2] testsuite: Automatically skip tests that fail when sysconfdir != /etc Marius Bakke
  0 siblings, 2 replies; 6+ messages in thread
From: Marius Bakke @ 2020-08-01 15:58 UTC (permalink / raw)
  To: Lucas De Marchi; +Cc: linux-modules

[-- Attachment #1: Type: text/plain, Size: 1195 bytes --]

Lucas De Marchi <lucas.de.marchi@gmail.com> writes:

> On Sat, May 16, 2020 at 6:04 AM Marius Bakke <marius@devup.no> wrote:
>>
>> The Makefile helpfully warns that some tests will fail when
>> --sysconfdir != /etc, but there are no provisions to easily disable
>> those.  This commit provides an escape hatch.
>
> nice... but are we missing a patch to actually make it skip?

Uh...  That makes sense.  (I maintained a list down-stream, derp.)

>> @@ -948,6 +949,14 @@ static inline int test_run_parent(const struct test *t, int fdout[2],
>>         int err;
>>         bool matchout, match_modules;
>>
>> +       if (t->skip == true) {
>
> only if (t->skip)  would be less verbose and preferred I think.

Fixed in v2, thanks!

Marius Bakke (2):
  testsuite: Add facility to skip tests.
  testsuite: Automatically skip tests that fail when sysconfdir != /etc.

 Makefile.am                | 12 +++++-------
 configure.ac               |  2 ++
 testsuite/test-blacklist.c |  3 +++
 testsuite/test-depmod.c    | 12 ++++++++++++
 testsuite/test-modprobe.c  |  6 ++++++
 testsuite/testsuite.c      |  9 +++++++++
 testsuite/testsuite.h      |  1 +
 7 files changed, 38 insertions(+), 7 deletions(-)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 483 bytes --]

^ permalink raw reply	[flat|nested] 6+ messages in thread

* [PATCH v2 1/2] testsuite: Add facility to skip tests.
  2020-08-01 15:58   ` Marius Bakke
@ 2020-08-01 16:02     ` Marius Bakke
  2021-01-08  3:47       ` Lucas De Marchi
  2020-08-01 16:02     ` [PATCH v2 2/2] testsuite: Automatically skip tests that fail when sysconfdir != /etc Marius Bakke
  1 sibling, 1 reply; 6+ messages in thread
From: Marius Bakke @ 2020-08-01 16:02 UTC (permalink / raw)
  To: linux-modules; +Cc: Lucas De Marchi

The Makefile helpfully warns that some tests will fail when
--sysconfdir != /etc, but there are no provisions to easily disable
those.  This commit provides an escape hatch.
---
 testsuite/testsuite.c | 9 +++++++++
 testsuite/testsuite.h | 1 +
 2 files changed, 10 insertions(+)

diff --git a/testsuite/testsuite.c b/testsuite/testsuite.c
index e46f3d8..05df553 100644
--- a/testsuite/testsuite.c
+++ b/testsuite/testsuite.c
@@ -37,6 +37,7 @@
 #include "testsuite.h"
 
 static const char *ANSI_HIGHLIGHT_GREEN_ON = "\x1B[1;32m";
+static const char *ANSI_HIGHLIGHT_YELLOW_ON = "\x1B[1;33m";
 static const char *ANSI_HIGHLIGHT_RED_ON =  "\x1B[1;31m";
 static const char *ANSI_HIGHLIGHT_OFF = "\x1B[0m";
 
@@ -948,6 +949,14 @@ static inline int test_run_parent(const struct test *t, int fdout[2],
 	int err;
 	bool matchout, match_modules;
 
+	if (t->skip) {
+		LOG("%sSKIPPED%s: %s\n",
+			ANSI_HIGHLIGHT_YELLOW_ON, ANSI_HIGHLIGHT_OFF,
+			t->name);
+		err = EXIT_SUCCESS;
+		goto exit;
+	}
+
 	/* Close write-fds */
 	if (t->output.out != NULL)
 		close(fdout[1]);
diff --git a/testsuite/testsuite.h b/testsuite/testsuite.h
index 7ed96bf..8029c64 100644
--- a/testsuite/testsuite.h
+++ b/testsuite/testsuite.h
@@ -109,6 +109,7 @@ struct test {
 	const struct keyval *env_vars;
 	bool need_spawn;
 	bool expected_fail;
+	bool skip;
 	bool print_outputs;
 } __attribute__((aligned(8)));
 
-- 
2.28.0


^ permalink raw reply related	[flat|nested] 6+ messages in thread

* [PATCH v2 2/2] testsuite: Automatically skip tests that fail when sysconfdir != /etc.
  2020-08-01 15:58   ` Marius Bakke
  2020-08-01 16:02     ` [PATCH v2 1/2] " Marius Bakke
@ 2020-08-01 16:02     ` Marius Bakke
  1 sibling, 0 replies; 6+ messages in thread
From: Marius Bakke @ 2020-08-01 16:02 UTC (permalink / raw)
  To: linux-modules; +Cc: Lucas De Marchi

---
 Makefile.am                | 12 +++++-------
 configure.ac               |  2 ++
 testsuite/test-blacklist.c |  3 +++
 testsuite/test-depmod.c    | 12 ++++++++++++
 testsuite/test-modprobe.c  |  6 ++++++
 5 files changed, 28 insertions(+), 7 deletions(-)

diff --git a/Makefile.am b/Makefile.am
index 8eadb99..cfebc37 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -265,13 +265,7 @@ TESTSUITE_OVERRIDE_LIBS = \
 TESTSUITE_OVERRIDE_LIBS_LDFLAGS = \
 	avoid-version -module -shared -export-dynamic -rpath /nowhere -ldl
 
-check-sysconfdir:
-	$(AM_V_at)if test "$(sysconfdir)" != "/etc" -a "$(sysconfdir)" != "/etc/"; then \
-		echo "warning: Some tests will fail without --sysconfdir=/etc" >&2; \
-	fi
-.PHONY: check-sysconfdir
-
-check-am: rootfs check-sysconfdir
+check-am: rootfs
 
 
 EXTRA_DIST += \
@@ -326,6 +320,10 @@ TESTSUITE_LDADD = \
 	testsuite/libtestsuite.la libkmod/libkmod-internal.la \
 	shared/libshared.la
 
+if KMOD_SYSCONFDIR_NOT_ETC
+TESTSUITE_CPPFLAGS += -DKMOD_SYSCONFDIR_NOT_ETC
+endif
+
 check_LTLIBRARIES += testsuite/libtestsuite.la
 testsuite_libtestsuite_la_SOURCES = \
 	testsuite/testsuite.c testsuite/testsuite.h
diff --git a/configure.ac b/configure.ac
index 4a65d6b..2115b03 100644
--- a/configure.ac
+++ b/configure.ac
@@ -213,6 +213,8 @@ GTK_DOC_CHECK([1.14],[--flavour no-tmpl-flat])
 ], [
 AM_CONDITIONAL([ENABLE_GTK_DOC], false)])
 
+# Some tests are skipped when sysconfdir != /etc.
+AM_CONDITIONAL([KMOD_SYSCONFDIR_NOT_ETC], [test "x$sysconfdir" != "x/etc"])
 
 #####################################################################
 # Default CFLAGS and LDFLAGS
diff --git a/testsuite/test-blacklist.c b/testsuite/test-blacklist.c
index 969567d..d03eedb 100644
--- a/testsuite/test-blacklist.c
+++ b/testsuite/test-blacklist.c
@@ -95,6 +95,9 @@ fail_lookup:
 }
 
 DEFINE_TEST(blacklist_1,
+#if defined(KMOD_SYSCONFDIR_NOT_ETC)
+        .skip = true,
+#endif
 	.description = "check if modules are correctly blacklisted",
 	.config = {
 		[TC_ROOTFS] = TESTSUITE_ROOTFS "test-blacklist/",
diff --git a/testsuite/test-depmod.c b/testsuite/test-depmod.c
index 47dafb4..261559c 100644
--- a/testsuite/test-depmod.c
+++ b/testsuite/test-depmod.c
@@ -42,6 +42,9 @@ static noreturn int depmod_modules_order_for_compressed(const struct test *t)
 }
 
 DEFINE_TEST(depmod_modules_order_for_compressed,
+#if defined(KMOD_SYSCONFDIR_NOT_ETC)
+        .skip = true,
+#endif
 	.description = "check if depmod let aliases in right order when using compressed modules",
 	.config = {
 		[TC_UNAME_R] = MODULES_ORDER_UNAME,
@@ -121,6 +124,9 @@ static noreturn int depmod_detect_loop(const struct test *t)
 	exit(EXIT_FAILURE);
 }
 DEFINE_TEST(depmod_detect_loop,
+#if defined(KMOD_SYSCONFDIR_NOT_ETC)
+        .skip = true,
+#endif
 	.description = "check if depmod detects module loops correctly",
 	.config = {
 		[TC_UNAME_R] = "4.4.4",
@@ -144,6 +150,9 @@ static noreturn int depmod_search_order_external_first(const struct test *t)
 	exit(EXIT_FAILURE);
 }
 DEFINE_TEST(depmod_search_order_external_first,
+#if defined(KMOD_SYSCONFDIR_NOT_ETC)
+        .skip = true,
+#endif
 	.description = "check if depmod honor external keyword with higher priority",
 	.config = {
 		[TC_UNAME_R] = "4.4.4",
@@ -196,6 +205,9 @@ static noreturn int depmod_search_order_override(const struct test *t)
 	exit(EXIT_FAILURE);
 }
 DEFINE_TEST(depmod_search_order_override,
+#if defined(KMOD_SYSCONFDIR_NOT_ETC)
+        .skip = true,
+#endif
 	.description = "check if depmod honor override keyword",
 	.config = {
 		[TC_UNAME_R] = "4.4.4",
diff --git a/testsuite/test-modprobe.c b/testsuite/test-modprobe.c
index f908d56..f6bed8b 100644
--- a/testsuite/test-modprobe.c
+++ b/testsuite/test-modprobe.c
@@ -83,6 +83,9 @@ static noreturn int modprobe_show_alias_to_none(const struct test *t)
 	exit(EXIT_FAILURE);
 }
 DEFINE_TEST(modprobe_show_alias_to_none,
+#if defined(KMOD_SYSCONFDIR_NOT_ETC)
+        .skip = true,
+#endif
 	.description = "check if modprobe --show-depends doesn't explode with an alias to nothing",
 	.config = {
 		[TC_UNAME_R] = "4.4.4",
@@ -172,6 +175,9 @@ static noreturn int modprobe_softdep_loop(const struct test *t)
 	exit(EXIT_FAILURE);
 }
 DEFINE_TEST(modprobe_softdep_loop,
+#if defined(KMOD_SYSCONFDIR_NOT_ETC)
+        .skip = true,
+#endif
 	.description = "check if modprobe breaks softdep loop",
 	.config = {
 		[TC_UNAME_R] = "4.4.4",
-- 
2.28.0


^ permalink raw reply related	[flat|nested] 6+ messages in thread

* Re: [PATCH v2 1/2] testsuite: Add facility to skip tests.
  2020-08-01 16:02     ` [PATCH v2 1/2] " Marius Bakke
@ 2021-01-08  3:47       ` Lucas De Marchi
  0 siblings, 0 replies; 6+ messages in thread
From: Lucas De Marchi @ 2021-01-08  3:47 UTC (permalink / raw)
  To: Marius Bakke; +Cc: linux-modules

Sorry for the delay. This is now applied.

Lucas De Marchi

On Sat, Aug 1, 2020 at 9:02 AM Marius Bakke <marius@devup.no> wrote:
>
> The Makefile helpfully warns that some tests will fail when
> --sysconfdir != /etc, but there are no provisions to easily disable
> those.  This commit provides an escape hatch.
> ---
>  testsuite/testsuite.c | 9 +++++++++
>  testsuite/testsuite.h | 1 +
>  2 files changed, 10 insertions(+)
>
> diff --git a/testsuite/testsuite.c b/testsuite/testsuite.c
> index e46f3d8..05df553 100644
> --- a/testsuite/testsuite.c
> +++ b/testsuite/testsuite.c
> @@ -37,6 +37,7 @@
>  #include "testsuite.h"
>
>  static const char *ANSI_HIGHLIGHT_GREEN_ON = "\x1B[1;32m";
> +static const char *ANSI_HIGHLIGHT_YELLOW_ON = "\x1B[1;33m";
>  static const char *ANSI_HIGHLIGHT_RED_ON =  "\x1B[1;31m";
>  static const char *ANSI_HIGHLIGHT_OFF = "\x1B[0m";
>
> @@ -948,6 +949,14 @@ static inline int test_run_parent(const struct test *t, int fdout[2],
>         int err;
>         bool matchout, match_modules;
>
> +       if (t->skip) {
> +               LOG("%sSKIPPED%s: %s\n",
> +                       ANSI_HIGHLIGHT_YELLOW_ON, ANSI_HIGHLIGHT_OFF,
> +                       t->name);
> +               err = EXIT_SUCCESS;
> +               goto exit;
> +       }
> +
>         /* Close write-fds */
>         if (t->output.out != NULL)
>                 close(fdout[1]);
> diff --git a/testsuite/testsuite.h b/testsuite/testsuite.h
> index 7ed96bf..8029c64 100644
> --- a/testsuite/testsuite.h
> +++ b/testsuite/testsuite.h
> @@ -109,6 +109,7 @@ struct test {
>         const struct keyval *env_vars;
>         bool need_spawn;
>         bool expected_fail;
> +       bool skip;
>         bool print_outputs;
>  } __attribute__((aligned(8)));
>
> --
> 2.28.0
>

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2021-01-08  3:48 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-05-16 12:54 [PATCH] testsuite: Add facility to skip tests Marius Bakke
2020-07-08 18:16 ` Lucas De Marchi
2020-08-01 15:58   ` Marius Bakke
2020-08-01 16:02     ` [PATCH v2 1/2] " Marius Bakke
2021-01-08  3:47       ` Lucas De Marchi
2020-08-01 16:02     ` [PATCH v2 2/2] testsuite: Automatically skip tests that fail when sysconfdir != /etc Marius Bakke

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).