[PATCH 0/6] prepare for release

[PATCH 0/6] prepare for release

[Lucas De Marchi]

I already applied the first patch since I was thinking about just
quickly testing in a pristine environment before releasing.
However I went ahead and 1) added support for more than one distro
using mkosi and 2) debugged why the testsuite was failing for me.

After this I think we are safe to do a new release.


Lucas De Marchi (6):
  build: add mkosi hooks
  build: use tool from configure
  testsuite: generalize mkosi support for other distros
  testsuite: add Fedora's mkosi configuration
  testsuite: add missing error handling
  testsuite: explain why overriding function may fail

 Makefile.am                  | 10 ++++++++++
 configure.ac                 |  1 +
 testsuite/init_module.c      |  9 +++++++--
 testsuite/mkosi/.gitignore   |  3 +++
 testsuite/mkosi/mkosi.arch   | 25 +++++++++++++++++++++++++
 testsuite/mkosi/mkosi.build  | 38 ++++++++++++++++++++++++++++++++++++++
 testsuite/mkosi/mkosi.fedora | 26 ++++++++++++++++++++++++++
 testsuite/testsuite.h        |  2 ++
 8 files changed, 112 insertions(+), 2 deletions(-)
 create mode 100644 testsuite/mkosi/.gitignore
 create mode 100644 testsuite/mkosi/mkosi.arch
 create mode 100755 testsuite/mkosi/mkosi.build
 create mode 100644 testsuite/mkosi/mkosi.fedora

-- 
2.14.3

[PATCH 1/6] build: add mkosi hooks

[Lucas De Marchi]

Right now there's support for building on Archlinux only.
---
 Makefile.am                   |  8 ++++++++
 testsuite/mkosi/.gitignore    |  4 ++++
 testsuite/mkosi/mkosi.arch    | 22 ++++++++++++++++++++++
 testsuite/mkosi/mkosi.build   | 23 +++++++++++++++++++++++
 testsuite/mkosi/mkosi.default |  1 +
 5 files changed, 58 insertions(+)
 create mode 100644 testsuite/mkosi/.gitignore
 create mode 100644 testsuite/mkosi/mkosi.arch
 create mode 100755 testsuite/mkosi/mkosi.build
 create mode 120000 testsuite/mkosi/mkosi.default

diff --git a/Makefile.am b/Makefile.am
index 2a1eb50..b7ee6b1 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -498,3 +498,11 @@ tar: kmod-$(VERSION).tar.xz kmod-$(VERSION).tar.sign
 
 tar-sync: kmod-$(VERSION).tar.xz kmod-$(VERSION).tar.sign
 	kup put kmod-$(VERSION).tar.xz  kmod-$(VERSION).tar.sign /pub/linux/utils/kernel/kmod/
+
+# ------------------------------------------------------------------------------
+# mkosi
+# ------------------------------------------------------------------------------
+
+mkosi:
+	-mkdir $(top_srcdir)/testsuite/mkosi/mkosi.cache
+	mkosi -C $(top_srcdir)/testsuite/mkosi --build-sources ../../ -fi
diff --git a/testsuite/mkosi/.gitignore b/testsuite/mkosi/.gitignore
new file mode 100644
index 0000000..024d05b
--- /dev/null
+++ b/testsuite/mkosi/.gitignore
@@ -0,0 +1,4 @@
+/image.raw*
+/.mkosi-*
+/mkosi.cache
+/rootfs
diff --git a/testsuite/mkosi/mkosi.arch b/testsuite/mkosi/mkosi.arch
new file mode 100644
index 0000000..308dcdb
--- /dev/null
+++ b/testsuite/mkosi/mkosi.arch
@@ -0,0 +1,22 @@
+[Distribution]
+Distribution=arch
+Release=(rolling)
+
+[Packages]
+Packages = valgrind
+BuildPackages =
+	automake
+	gcc
+	git
+	make
+	pkg-config
+	python2
+	python2-future
+	autoconf
+	gtk-doc
+	docbook-xml
+	docbook-xsl
+	linux-headers
+
+[Partitions]
+RootSize = 4G
diff --git a/testsuite/mkosi/mkosi.build b/testsuite/mkosi/mkosi.build
new file mode 100755
index 0000000..a6b325f
--- /dev/null
+++ b/testsuite/mkosi/mkosi.build
@@ -0,0 +1,23 @@
+#!/bin/bash -ex
+
+if [ -f configure ]; then
+    make distclean
+fi
+
+rm -rf build
+mkdir build
+cd build
+
+kdirs=(/usr/lib/modules/*/build/Makefile)
+if [[ ! -f ${kdirs[0]} ]]; then
+  printf '==> Unable to find kernel headers to build modules for tests\n' >&2
+  return 1
+fi
+
+kdir=${kdirs[0]%/Makefile}
+IFS=/ read _ _ _ kver _ <<<"$kdir"
+
+../autogen.sh c
+make -j
+make check KDIR="$kdir" KVER="$kver"
+make install
diff --git a/testsuite/mkosi/mkosi.default b/testsuite/mkosi/mkosi.default
new file mode 120000
index 0000000..695b71a
--- /dev/null
+++ b/testsuite/mkosi/mkosi.default
@@ -0,0 +1 @@
+mkosi.arch
\ No newline at end of file
-- 
2.14.3

[PATCH 2/6] build: use tool from configure

[Lucas De Marchi]

This way we make sure the tool will be the one we actually configured
before going through sudo.
---
 Makefile.am  | 4 ++--
 configure.ac | 1 +
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/Makefile.am b/Makefile.am
index b7ee6b1..3ea4274 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -504,5 +504,5 @@ tar-sync: kmod-$(VERSION).tar.xz kmod-$(VERSION).tar.sign
 # ------------------------------------------------------------------------------
 
 mkosi:
-	-mkdir $(top_srcdir)/testsuite/mkosi/mkosi.cache
-	mkosi -C $(top_srcdir)/testsuite/mkosi --build-sources ../../ -fi
+	-$(MKDIR_P) $(top_srcdir)/testsuite/mkosi/mkosi.cache
+	$(MKOSI) -C $(top_srcdir)/testsuite/mkosi --build-sources ../../ -fi
diff --git a/configure.ac b/configure.ac
index ca94bf8..932e87d 100644
--- a/configure.ac
+++ b/configure.ac
@@ -31,6 +31,7 @@ AC_PROG_MKDIR_P
 AC_PROG_LN_S
 PKG_PROG_PKG_CONFIG
 AC_PATH_PROG([XSLTPROC], [xsltproc])
+AC_PATH_PROG([MKOSI], [mkosi])
 
 AC_PROG_CC_C99
 
-- 
2.14.3

[PATCH 3/6] testsuite: generalize mkosi support for other distros

[Lucas De Marchi]

Instead of using the mkosi.default symlink, use an env var passed from
the build system. We would need to pass the --default switch nonetheless
or change the symlink, making the git tree dirty.

Also, search for installed kernel headers in a way that's compatible
with more distros. On Fedora, for example, the
/usr/lib/modules/<kver>/build symlink is only available if there's a
kernel installed. We don't care about a kernel installed since we don't
need to boot it on a real machine: the only thing we need is the
kernel-devel package.
---
 Makefile.am                   |  4 +++-
 testsuite/mkosi/.gitignore    |  3 +--
 testsuite/mkosi/mkosi.arch    |  3 +++
 testsuite/mkosi/mkosi.build   | 29 ++++++++++++++++++++++-------
 testsuite/mkosi/mkosi.default |  1 -
 5 files changed, 29 insertions(+), 11 deletions(-)
 delete mode 120000 testsuite/mkosi/mkosi.default

diff --git a/Makefile.am b/Makefile.am
index 3ea4274..7b01201 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -503,6 +503,8 @@ tar-sync: kmod-$(VERSION).tar.xz kmod-$(VERSION).tar.sign
 # mkosi
 # ------------------------------------------------------------------------------
 
+DISTRO ?= "arch"
+
 mkosi:
 	-$(MKDIR_P) $(top_srcdir)/testsuite/mkosi/mkosi.cache
-	$(MKOSI) -C $(top_srcdir)/testsuite/mkosi --build-sources ../../ -fi
+	$(MKOSI) -C $(top_srcdir)/testsuite/mkosi --build-sources ../../ --default mkosi.${DISTRO} -fi
diff --git a/testsuite/mkosi/.gitignore b/testsuite/mkosi/.gitignore
index 024d05b..0e0981a 100644
--- a/testsuite/mkosi/.gitignore
+++ b/testsuite/mkosi/.gitignore
@@ -1,4 +1,3 @@
-/image.raw*
+/*-image.raw*
 /.mkosi-*
 /mkosi.cache
-/rootfs
diff --git a/testsuite/mkosi/mkosi.arch b/testsuite/mkosi/mkosi.arch
index 308dcdb..5ac0a45 100644
--- a/testsuite/mkosi/mkosi.arch
+++ b/testsuite/mkosi/mkosi.arch
@@ -2,6 +2,9 @@
 Distribution=arch
 Release=(rolling)
 
+[Output]
+Output = arch-image.raw
+
 [Packages]
 Packages = valgrind
 BuildPackages =
diff --git a/testsuite/mkosi/mkosi.build b/testsuite/mkosi/mkosi.build
index a6b325f..53fc797 100755
--- a/testsuite/mkosi/mkosi.build
+++ b/testsuite/mkosi/mkosi.build
@@ -1,5 +1,26 @@
 #!/bin/bash -ex
 
+function find_kdir() {
+    local kdirs=(/usr/lib/modules/*/build/Makefile /usr/src/kernels/*/Makefile)
+    local kdir=""
+
+    for f in "${kdirs[@]}"; do
+        if [ -f "$f" ]; then
+            kdir=$f
+            break
+        fi
+    done
+
+    if [ -z "$kdir" ]; then
+        printf '==> Unable to find kernel headers to build modules for tests\n' >&2
+        exit 1
+    fi
+
+    kdir=${kdir%/Makefile}
+
+    echo $kdir
+}
+
 if [ -f configure ]; then
     make distclean
 fi
@@ -8,13 +29,7 @@ rm -rf build
 mkdir build
 cd build
 
-kdirs=(/usr/lib/modules/*/build/Makefile)
-if [[ ! -f ${kdirs[0]} ]]; then
-  printf '==> Unable to find kernel headers to build modules for tests\n' >&2
-  return 1
-fi
-
-kdir=${kdirs[0]%/Makefile}
+kdir=$(find_kdir)
 IFS=/ read _ _ _ kver _ <<<"$kdir"
 
 ../autogen.sh c
diff --git a/testsuite/mkosi/mkosi.default b/testsuite/mkosi/mkosi.default
deleted file mode 120000
index 695b71a..0000000
--- a/testsuite/mkosi/mkosi.default
+++ /dev/null
@@ -1 +0,0 @@
-mkosi.arch
\ No newline at end of file
-- 
2.14.3

[PATCH 4/6] testsuite: add Fedora's mkosi configuration

[Lucas De Marchi]

To use the Fedora configuration rather than the default, one should
use:

	# make DISTRO=fedora mkosi

While at it also reduce the root partition size for Arch, since it
doesn't need that much.
---
 testsuite/mkosi/mkosi.arch   |  2 +-
 testsuite/mkosi/mkosi.fedora | 26 ++++++++++++++++++++++++++
 2 files changed, 27 insertions(+), 1 deletion(-)
 create mode 100644 testsuite/mkosi/mkosi.fedora

diff --git a/testsuite/mkosi/mkosi.arch b/testsuite/mkosi/mkosi.arch
index 5ac0a45..324dfb0 100644
--- a/testsuite/mkosi/mkosi.arch
+++ b/testsuite/mkosi/mkosi.arch
@@ -22,4 +22,4 @@ BuildPackages =
 	linux-headers
 
 [Partitions]
-RootSize = 4G
+RootSize = 2G
diff --git a/testsuite/mkosi/mkosi.fedora b/testsuite/mkosi/mkosi.fedora
new file mode 100644
index 0000000..7be7dd9
--- /dev/null
+++ b/testsuite/mkosi/mkosi.fedora
@@ -0,0 +1,26 @@
+[Distribution]
+Distribution=fedora
+Release=27
+
+[Output]
+Output = fedora-image.raw
+
+[Packages]
+Packages = valgrind
+BuildPackages =
+	autoconf
+	automake
+	gcc
+	git
+	gtk-doc
+	kernel-devel
+	libtool
+	libxslt
+	make
+	pkgconf-pkg-config
+	xml-common
+	xz-devel
+	zlib-devel
+
+[Partitions]
+RootSize = 2G
-- 
2.14.3

[PATCH 5/6] testsuite: add missing error handling

[Lucas De Marchi]

If we passed an invalid argument to a test it would segfault rather than
returning an error code.
---
 testsuite/testsuite.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/testsuite/testsuite.h b/testsuite/testsuite.h
index bb0eb50..2b31483 100644
--- a/testsuite/testsuite.h
+++ b/testsuite/testsuite.h
@@ -151,6 +151,8 @@ int test_run(const struct test *t);
 		arg = test_init(__start_kmod_tests, __stop_kmod_tests, argc, argv);		\
 		if (arg == 0)									\
 			return 0;								\
+		if (arg < 0)									\
+			return EXIT_FAILURE;							\
 												\
 		if (arg < argc) {								\
 			t = test_find(__start_kmod_tests, __stop_kmod_tests, argv[arg]);	\
-- 
2.14.3

[PATCH 6/6] testsuite: explain why overriding function may fail

[Lucas De Marchi]

On my computer `testsuite/test-modprobe modprobe_install_cmd_loop` was
failing because when it forks off the shell the child process ends up
calling syscall() which are are supposed to wrap. Here's the backtrace:

	#0  0x00007ffff6fdb66b in raise () from /lib64/libc.so.6
	#1  0x00007ffff6fdd381 in abort () from /lib64/libc.so.6
	#2  0x00007ffff77bac97 in syscall (__sysno=<optimized out>)
	    at testsuite/init_module.c:362
	#3  0x00007fffef92d4e7 in hashmap_base_new.lto_priv () from /lib64/libnss_systemd.so.2
	#4  0x00007fffef953f50 in sd_bus_open_system () from /lib64/libnss_systemd.so.2
	#5  0x00007fffef943123 in _nss_systemd_getpwuid_r () from /lib64/libnss_systemd.so.2
	#6  0x00007ffff707eea5 in getpwuid_r@@GLIBC_2.2.5 () from /lib64/libc.so.6
	#7  0x00007ffff707e608 in getpwuid () from /lib64/libc.so.6
	#8  0x00005555555859e1 in get_current_user_info.part ()
	#9  0x00005555555a375a in initialize_shell_variables ()
	#10 0x0000555555580fde in shell_initialize ()
	#11 0x00005555555846ff in main ()<Paste>

The reason it fails on my system and not on, for e.g., a new one set up with
mkosi is that the call to getpwuid() depends on the contents
/etc/nsswitch.conf. The systemd module calls syscall() to implement gettid()
which we can't forward due to being a variadic function.

No fix is provided here, but at least it's explained why this happens.
---
 testsuite/init_module.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/testsuite/init_module.c b/testsuite/init_module.c
index b7d220b..199186b 100644
--- a/testsuite/init_module.c
+++ b/testsuite/init_module.c
@@ -356,9 +356,14 @@ TS_EXPORT long int syscall(long int __sysno, ...)
 	}
 
 	/*
-	 * FIXME: no way to call the libc function - let's hope there are no
-	 * other users.
+	 * FIXME: no way to call the libc function due since this is a
+	 * variadic argument function and we don't have a vsyscall() variant
+	 * this may fail if a library or process is trying to call syscall()
+	 * directly, for example to implement gettid().
 	 */
+	fprintf(stderr, "FIXME FIXME FIXME: could not wrap call to syscall(%ld), this should not happen\n",
+		__sysno);
+
 	abort();
 }
 
-- 
2.14.3

Re: [PATCH 0/6] prepare for release

[Lucas De Marchi]

On Wed, Jan 3, 2018 at 2:29 PM, Lucas De Marchi
<lucas.demarchi@intel.com> wrote:
> I already applied the first patch since I was thinking about just
> quickly testing in a pristine environment before releasing.
> However I went ahead and 1) added support for more than one distro
> using mkosi and 2) debugged why the testsuite was failing for me.
>
> After this I think we are safe to do a new release.
>
>
> Lucas De Marchi (6):
>   build: add mkosi hooks
>   build: use tool from configure
>   testsuite: generalize mkosi support for other distros
>   testsuite: add Fedora's mkosi configuration
>   testsuite: add missing error handling
>   testsuite: explain why overriding function may fail

All patches applied.

Lucas De Marchi

>
>  Makefile.am                  | 10 ++++++++++
>  configure.ac                 |  1 +
>  testsuite/init_module.c      |  9 +++++++--
>  testsuite/mkosi/.gitignore   |  3 +++
>  testsuite/mkosi/mkosi.arch   | 25 +++++++++++++++++++++++++
>  testsuite/mkosi/mkosi.build  | 38 ++++++++++++++++++++++++++++++++++++++
>  testsuite/mkosi/mkosi.fedora | 26 ++++++++++++++++++++++++++
>  testsuite/testsuite.h        |  2 ++
>  8 files changed, 112 insertions(+), 2 deletions(-)
>  create mode 100644 testsuite/mkosi/.gitignore
>  create mode 100644 testsuite/mkosi/mkosi.arch
>  create mode 100755 testsuite/mkosi/mkosi.build
>  create mode 100644 testsuite/mkosi/mkosi.fedora
>
> --
> 2.14.3
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-modules" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Lucas De Marchi