[PATCH v06 01/36] Add scripts/headers_compile_test.sh: compile test for exported uapi headers

[PATCH v06 01/36] Add scripts/headers_compile_test.sh: compile test for exported uapi headers

[Mikko Rapeli]

Users of kernel uapi header files would be happier if they did not
contain kernel specific parts and would contain #include statements
for all other header files that they depend on, and in general would
compile also in user space. If Linux kernal uapi headers compile in
user space, then they can also be used to do ABI and API compatibility
checks against different kernel versions. ABI checks can also be used
by Linux distribution build systems like yocto/bitbake to avoid
recompiling dependencies if changes are ABI compatible.

For each header file exported to userspace, this script tries to
compile it together with minimal header files from GCC and libc, and
reports results.

In the test execution environment, libc and GCC header file locations
are parsed from gcc compiler configuration.

Some gcc and kernel headers depend on libc headers which are made available
by copying from the compiler default location to a temporary location and
removing possibly existing kernel headers from this directory. This is a bit
of a hack but seems to work in multiple environments so test can be
executed without installing headers to /usr/include and compiling a full
Linux distribution on top.

Tested natively on:

Debian unstable, i586-linux-gnu and gcc 4.9.2
Raspbian Wheezy, arm-linux-gnueabihf and gcc 4.6.3
Ubuntu 12.04 LTS, x86_64-linux-gnu and gcc 4.6.3

Tested cross compilation using standard CROSS_COMPILE=/path/to/gcc with:

arm-linux-gnueabihf-gcc (crosstool-NG linaro-1.13.1-4.9-2014.09 - Linaro GCC 4.9-2014.09) 4.9.2 20140904 (prerelease)

Execute the script in the directory where kernel header files are installed.
For example with kernel tree from v4.13-rc3-216-g0a23ea65ce9f on x86 i586:

$ make headers_install
$ cd usr/include
$ ../../scripts/headers_compile_test.sh
...
Kernel header compile test statistics:

42 files failed the kernel header compile test.
821 files passed the kernel header compile test.

Once all these errors have been fixed, this test could be added to
'make headers_check'.

In addition to plain compile test, there is also a glibc header
compatibility test, which compiles all kernel uapi headers against a
non-conflicting set of all GNU libc headers. The test covers
variants where kernel header is included before libc header
and libc header before kernel header.

Tested in Debian unstable with libc6 version 2.24-14.

Example execution with output:

$ make headers_install
$ cd usr/include
$ ../../scripts/headers_compile_test.sh -lk
Testing that /home/mcfrisk/src/linux-2.6/usr/headers_compile_test_include.qd5RuE/libc_headers.h compiles
cc -Wall -c -nostdinc -I /usr/lib/gcc/i686-linux-gnu/6/include -I /usr/lib/gcc/i686-linux-gnu/6/include-fixed -I . -I /home/mcfrisk/src/linux-2.6/usr/headers_compile_test_include.qd5RuE -I /home/mcfrisk/src/linux-2.6/usr/headers_compile_test_include.qd5RuE/i686-linux-gnu -o /dev/null /home/mcfrisk/src/linux-2.6/usr/include/drm/i810_drm.h
PASSED: drm/i810_drm.h
cc -Wall -c -nostdinc -I /usr/lib/gcc/i686-linux-gnu/6/include -I /usr/lib/gcc/i686-linux-gnu/6/include-fixed -I . -I /home/mcfrisk/src/linux-2.6/usr/headers_compile_test_include.qd5RuE -I /home/mcfrisk/src/linux-2.6/usr/headers_compile_test_include.qd5RuE/i686-linux-gnu -o /dev/null /home/mcfrisk/src/linux-2.6/usr/include/drm/i810_drm.h_libc_before_kernel.h
...
In file included from /home/mcfrisk/src/linux-2.6/usr/include/asm/sigcontext.h_l
ibc_before_kernel.h:2:0:
./asm/sigcontext.h:291:8: error: redefinition of ‘struct sigcontext’
 struct sigcontext {
        ^~~~~~~~~~
In file included from /home/mcfrisk/src/linux-2.6/usr/headers_compile_test_inclu
de.qd5RuE/signal.h:306:0,
                 from /home/mcfrisk/src/linux-2.6/usr/headers_compile_test_inclu
de.qd5RuE/sys/param.h:28,
                 from /home/mcfrisk/src/linux-2.6/usr/headers_compile_test_inclu
de.qd5RuE/resolv.h:62,
                 from /home/mcfrisk/src/linux-2.6/usr/headers_compile_test_inclu
de.qd5RuE/libc_headers.h:103,
                 from /home/mcfrisk/src/linux-2.6/usr/include/asm/sigcontext.h_l
ibc_before_kernel.h:1:
/home/mcfrisk/src/linux-2.6/usr/headers_compile_test_include.qd5RuE/bits/sigcont
ext.h:93:8: note: originally defined here
 struct sigcontext
        ^~~~~~~~~~
FAILED libc before kernel test: asm/sigcontext.h
...
Kernel header compile test statistics:

42 files failed the kernel header compile test.
821 files passed the kernel header compile test.

libc and kernel header compatibility test statistics:
73 files failed libc before kernel include test.
790 files passed libc before kernel include test.

Since I've been stubborn enough to work with this script and fixes to the
test failures for a few years, I added my self to MAINTAINERS.

Signed-off-by: Mikko Rapeli <mikko.rapeli@iki.fi>
Cc: Alexander Stein <alexander.stein@systec-electronic.com>
Cc: Gabriel Laskar <gabriel@lse.epita.fr>
Cc: Dmitry V. Levin <ldv@altlinux.org>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: netdev@vger.kernel.org
Cc: linux-api@vger.kernel.org
---
 MAINTAINERS                     |   5 +
 scripts/headers_compile_test.sh | 507 ++++++++++++++++++++++++++++++++++++++++
 2 files changed, 512 insertions(+)
 create mode 100755 scripts/headers_compile_test.sh

diff --git a/MAINTAINERS b/MAINTAINERS
index 567343b8ffaa..f1d6d50932cf 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -3527,6 +3527,11 @@ S:	Maintained
 F:	mm/memcontrol.c
 F:	mm/swap_cgroup.c
 
+COMPILE TEST FOR HEADER FILES EXPORTED TO USERSPACE
+M:	Mikko Rapeli <mikko.rapeli@iki.fi>
+S:	Maintained
+F:	scripts/headers_compile_test.sh
+
 CORETEMP HARDWARE MONITORING DRIVER
 M:	Fenghua Yu <fenghua.yu@intel.com>
 L:	linux-hwmon@vger.kernel.org
diff --git a/scripts/headers_compile_test.sh b/scripts/headers_compile_test.sh
new file mode 100755
index 000000000000..99de50d32b7b
--- /dev/null
+++ b/scripts/headers_compile_test.sh
@@ -0,0 +1,507 @@
+#!/bin/bash
+
+help() {
+	cat << EOF_HELP
+Userspace compile test for exported kernel headers.
+
+    Copyright (C) 2015 Mikko Rapeli <mikko.rapeli@iki.fi>
+
+    This program is free software; you can redistribute it and/or
+    modify it under the terms of the GNU General Public License
+    as published by the Free Software Foundation; version 2
+    of the License.
+
+Execute in root directory of exported kernel headers in Linux kernel source
+tree. Sets up gcc and libc headers without existing kernel headers to
+a temporary environment and tries to compile all exported header files
+from current directory against them. Return value is zero if all tests pass,
+non-zero if something goes wrong during execution or if any file failed
+the compile tests.
+
+Supported arguments:
+
+    -h|--help          print help
+    -k|--keep          don't cleanup temporary header files and directories
+    -lk|--libc-kernel  test for conflicts between kernel and libc headers
+                       when libc headers are included before kernel headers
+    -kl|--kernel-libc  test for conflicts between kernel and libc headers
+                       when kernel headers are included before libc headers
+    -l|--libc          test both -lk and -kl
+    -v|--verbose       print more verbose output
+
+Example in Linux kernel source tree:
+
+    \$ make headers_install
+    \$ cd usr/include
+    \$ $( readlink -f "$0" )
+
+EOF_HELP
+}
+
+# bash due to arithmetics and pipefail
+set -euo pipefail
+
+KEEP=0
+HELP=0
+LIBC_TEST=0
+LIBC_KERNEL_TEST=0
+KERNEL_LIBC_TEST=0
+
+# command line arguments
+for p in "$@"; do
+	case "$p" in
+		-k|--keep)
+			KEEP=1
+		;;
+		-l|--libc)
+			LIBC_TEST=1
+			LIBC_KERNEL_TEST=1
+			KERNEL_LIBC_TEST=1
+		;;
+		-lk|--libc-kernel)
+			LIBC_TEST=1
+			LIBC_KERNEL_TEST=1
+		;;
+		-kl|--kernel-libc)
+			LIBC_TEST=1
+			KERNEL_LIBC_TEST=1
+		;;
+		-h|--help)
+			HELP=1
+		;;
+		-v|--verbose)
+			set -x
+		;;
+		*)
+			help
+			echo "Unknown argument: $p"
+			exit 1
+		;;
+	esac
+done
+
+if [ "$HELP" != "0" ]; then help; exit 0; fi
+
+# sanity test
+if [ ! -d ./linux ]; then
+	echo Sanity check error: ./linux directory not found
+	echo Should be called in usr/include after \'make headers_install\'.
+	echo Returns number of failed files, 0 if none.
+	exit 1
+fi
+
+# Support CC variable for compiler and ccache, and cross compiling.
+# CC is used without quotes to support CC="ccache gcc".
+set +u
+if [ "$CC"foobar == "foobar" ]; then
+	CC=cc
+fi
+
+if [ "$CROSS_COMPILE"foobar != "foobar" ]; then
+	# Using gcc name since some cross compiler tool chains don't provide
+	# the cc symlink. Using eval to expand ~ to $HOME.
+	CC="$( eval echo "$CROSS_COMPILE"gcc )"
+fi
+set -u
+
+# Kernel headers refer to some gcc and libc headers so make them available.
+set +u
+if [ "$ARCH_TRIPLET"foobar == "foobar" ]; then
+	# Taking triplet from gcc/cpp
+	ARCH_TRIPLET="$( $CC -v -x c -E - < /dev/null 2>&1 | \
+			grep Target | sed -e 's/Target: //' )"
+fi
+
+if [ "$LIBC"foobar == "foobar" ]; then
+	# trying to grep libc includes from gcc/cpp defaults
+	_TEMP="$( $CC -v -x c -E - < /dev/null 2>&1 | \
+		sed -n -e '/^#include <...> search starts here:$/,/^End of search list.$/{//!p}' | \
+		sed -e 's/^\ \//\//g' | \
+		grep '/usr/include' )"
+
+	# sanity check and prepare LIBC dirs
+	for d in $_TEMP; do
+		if [ ! -d "$d" ]; then
+			echo "$d not a directory"
+			exit 1
+		fi
+		LIBC="$LIBC $d"
+	done
+fi
+set -u
+
+# Copy libc include files to temp directory for the tests.
+COMPILE_TEST_INC="$( readlink -f \
+		"$( mktemp -d ../headers_compile_test_include.XXXXXX )" )"
+
+# cleanup if keep not set
+if [ "$KEEP" = "0" ]; then
+	trap 'rm -rf "$COMPILE_TEST_INC"' EXIT
+else
+	trap 'printf \
+"Temporary directory not cleaned up! Remove manually:\n${COMPILE_TEST_INC}\n"' \
+		EXIT
+fi
+
+for d in $LIBC; do
+	# check if last part of dir is the arch triplet, cross compile paths
+	# can have it also elsewhere so just the last one counts.
+	if ! ( echo "$d" | egrep "$ARCH_TRIPLET$" > /dev/null ); then
+		# hopefully just main libc dir, e.g. /usr/include,
+		# follow symlinks from e.g. /usr/include/bits
+		cp -aL "$d"/* "$COMPILE_TEST_INC"/
+	elif ( echo "$d" | egrep "$ARCH_TRIPLET$" > /dev/null ); then
+		# hopefully the arch specific dir, e.g. /usr/include/x86_64-linux-gnu
+		cp -ar "$d"/* "$COMPILE_TEST_INC/"
+	else
+		echo "$d unexpected, bailing out"
+		exit 1
+	fi
+done
+
+# A single header with all non-conflicting libc headers to test kernel
+# headers against libc headers for conflicts.
+if [ "$LIBC_TEST" != 0 ]; then
+	# List taken from Debian unstable libc6 version 2.21-9.
+	# Some glibc headers conflict with each other so they
+	# are filtered out. Not perfect but better than nothing.
+        #
+	# $ for f in $( egrep "\.h$" /var/lib/dpkg/info/libc6-dev\:i386.list | sed -e 's|/usr/include/||'| sort | grep -v arpa | grep -v linux-gnu | grep -v rpcsvc | grep -v regexp.h | grep -v rpc | grep -v scsi | grep -v talkd ); do echo "#include <$f>"; done > libc_headers.h
+
+	cat > "$COMPILE_TEST_INC/libc_headers.h" << EOF_LIBC_HEADERS
+#include <aio.h>
+#include <aliases.h>
+#include <alloca.h>
+#include <argp.h>
+#include <argz.h>
+#include <ar.h>
+#include <assert.h>
+#include <byteswap.h>
+#include <complex.h>
+#include <cpio.h>
+#include <crypt.h>
+#include <ctype.h>
+#include <dirent.h>
+#include <dlfcn.h>
+#include <elf.h>
+#include <endian.h>
+#include <envz.h>
+#include <err.h>
+#include <errno.h>
+#include <error.h>
+#include <execinfo.h>
+#include <fcntl.h>
+#include <features.h>
+#include <fenv.h>
+#include <fmtmsg.h>
+#include <fnmatch.h>
+#include <fstab.h>
+#include <fts.h>
+#include <ftw.h>
+#include <_G_config.h>
+#include <gconv.h>
+#include <getopt.h>
+#include <glob.h>
+#include <gnu-versions.h>
+#include <grp.h>
+#include <gshadow.h>
+#include <iconv.h>
+#include <ifaddrs.h>
+#include <inttypes.h>
+#include <langinfo.h>
+#include <lastlog.h>
+#include <libgen.h>
+#include <libintl.h>
+#include <libio.h>
+#include <limits.h>
+#include <link.h>
+#include <locale.h>
+#include <malloc.h>
+#include <math.h>
+#include <mcheck.h>
+#include <memory.h>
+#include <mntent.h>
+#include <monetary.h>
+#include <mqueue.h>
+#include <netash/ash.h>
+#include <netatalk/at.h>
+#include <netax25/ax25.h>
+#include <netdb.h>
+#include <neteconet/ec.h>
+#include <net/ethernet.h>
+#include <net/if_arp.h>
+#include <net/if.h>
+#include <net/if_packet.h>
+#include <net/if_ppp.h>
+#include <net/if_shaper.h>
+#include <net/if_slip.h>
+#include <netinet/ether.h>
+#include <netinet/icmp6.h>
+#include <netinet/if_ether.h>
+#include <netinet/if_fddi.h>
+#include <netinet/if_tr.h>
+#include <netinet/igmp.h>
+#include <netinet/in.h>
+#include <netinet/in_systm.h>
+#include <netinet/ip6.h>
+#include <netinet/ip.h>
+#include <netinet/ip_icmp.h>
+#include <netinet/tcp.h>
+#include <netinet/udp.h>
+#include <netipx/ipx.h>
+#include <netiucv/iucv.h>
+#include <netpacket/packet.h>
+#include <net/ppp-comp.h>
+#include <net/ppp_defs.h>
+#include <netrom/netrom.h>
+#include <netrose/rose.h>
+#include <net/route.h>
+#include <nfs/nfs.h>
+#include <nl_types.h>
+#include <nss.h>
+#include <obstack.h>
+#include <paths.h>
+#include <poll.h>
+#include <printf.h>
+#include <protocols/routed.h>
+#include <protocols/rwhod.h>
+#include <protocols/timed.h>
+#include <pthread.h>
+#include <pty.h>
+#include <pwd.h>
+#include <re_comp.h>
+#include <regex.h>
+#include <resolv.h>
+#include <sched.h>
+#include <search.h>
+#include <semaphore.h>
+#include <setjmp.h>
+#include <sgtty.h>
+#include <shadow.h>
+#include <signal.h>
+#include <spawn.h>
+#include <stab.h>
+#include <stdc-predef.h>
+#include <stdint.h>
+#include <stdio_ext.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <strings.h>
+#include <stropts.h>
+#include <syscall.h>
+#include <sysexits.h>
+#include <syslog.h>
+#include <tar.h>
+#include <termio.h>
+#include <termios.h>
+#include <tgmath.h>
+#include <thread_db.h>
+#include <time.h>
+#include <ttyent.h>
+#include <uchar.h>
+#include <ucontext.h>
+#include <ulimit.h>
+#include <unistd.h>
+#include <ustat.h>
+#include <utime.h>
+#include <utmp.h>
+#include <utmpx.h>
+#include <values.h>
+#include <wait.h>
+#include <wchar.h>
+#include <wctype.h>
+#include <wordexp.h>
+#include <xlocale.h>
+EOF_LIBC_HEADERS
+
+fi # LIBC_TEST
+
+# Simulate libc headers without kernel headers by removing
+# all known kernel header dirs from the copied libc ones.
+# This seems to magically work.
+_KERNEL_DIRS="$( find . -type d | grep -v '^\.$' )"
+( cd "$COMPILE_TEST_INC" && rm -rf $_KERNEL_DIRS )
+
+# GCC headers
+set +u
+if [ "$GCC_INC"foobar == "foobar" ]; then
+	# Take from $CC default system include paths, filter out
+	# /usr/local/include and /usr/include stuff first, then try to match
+	# for gcc.
+	_TEMP="$( $CC -v -x c -E - < /dev/null 2>&1 | \
+		sed -n -e '/^#include <...> search starts here:$/,/^End of search list.$/{//!p}' | \
+		sed -e 's/^\ \//\//g' | \
+		egrep -v '/usr/local/include' | \
+		egrep -v '/usr/include' | \
+		grep gcc | \
+		xargs )"
+
+	# merge and prepare for use with $CC
+	for d in $_TEMP; do
+		# sanity test
+		if [ ! -d "$d" ]; then
+			echo "$d: is not a directory"
+			exit 1
+		fi
+		GCC_INC="$GCC_INC -I $d"
+	done
+fi
+set -u
+
+# sanity check: test that plain libc headers compile
+if [ "$LIBC_TEST" != 0 ]; then
+	echo "Testing that $COMPILE_TEST_INC/libc_headers.h compiles"
+	$CC -Wall -c -nostdinc $GCC_INC -I . \
+		-I "$COMPILE_TEST_INC" \
+		-I "$COMPILE_TEST_INC/$ARCH_TRIPLET" \
+		-o /dev/null \
+		"$COMPILE_TEST_INC/libc_headers.h"
+fi
+
+# Summary counters:
+_FAILED=0
+_PASSED=0
+_LIBC_FAILED=0
+_LIBC_PASSED=0
+_LIBC_BEFORE_KERNEL_FAILED=0
+_LIBC_BEFORE_KERNEL_PASSED=0
+_KERNEL_BEFORE_LIBC_FAILED=0
+_KERNEL_BEFORE_LIBC_PASSED=0
+
+# For each header file, try to compile it using the headers we prepared.
+for f in $( find . -name "*\.h" -printf "%P\n" ); do
+	_FAIL=0
+	_FAIL_LIBC=0
+	_FAIL_LIBC_BEFORE_KERNEL=0
+	_FAIL_KERNEL_BEFORE_LIBC=0
+
+	# compile test, CC not quoted to support ccache
+	echo $CC -Wall -c -nostdinc $GCC_INC -I . -I "$COMPILE_TEST_INC" -I "$COMPILE_TEST_INC/$ARCH_TRIPLET" -o /dev/null "$PWD/$f"
+	$CC -Wall -c -nostdinc $GCC_INC -I . -I "$COMPILE_TEST_INC" -I "$COMPILE_TEST_INC/$ARCH_TRIPLET" -o /dev/null "$PWD/$f" \
+		|| _FAIL=1
+
+	# report errors
+	if [ "$_FAIL" -gt 0 ]; then
+		echo "FAILED: $f"
+		_FAILED="$(( _FAILED + 1 ))"
+	else
+		echo "PASSED: $f"
+		_PASSED="$(( _PASSED + 1))"
+	fi
+
+	# libc header conflict tests
+	if [ "$LIBC_TEST" != 0 ]; then
+		_LIBC_BEFORE_KERNEL="$PWD/$f"_libc_before_kernel.h
+		_KERNEL_BEFORE_LIBC="$PWD/$f"_kernel_before_libc.h
+
+		# libc header included before kernel header
+		if [ "$LIBC_KERNEL_TEST" != 0 ]; then
+			cat > "$_LIBC_BEFORE_KERNEL" << EOF_LIBC_BEFORE_KERNEL
+#include <libc_headers.h>
+#include <$f>
+EOF_LIBC_BEFORE_KERNEL
+			echo \
+				$CC -Wall -c -nostdinc $GCC_INC \
+				-I . -I "$COMPILE_TEST_INC" \
+				-I "$COMPILE_TEST_INC/$ARCH_TRIPLET" \
+				-o /dev/null "$_LIBC_BEFORE_KERNEL"
+			$CC -Wall -c -nostdinc $GCC_INC \
+				-I . -I "$COMPILE_TEST_INC" \
+				-I "$COMPILE_TEST_INC/$ARCH_TRIPLET" \
+				-o /dev/null "$_LIBC_BEFORE_KERNEL" \
+				|| _FAIL_LIBC_BEFORE_KERNEL=1
+
+			# report errors
+			if [ "$_FAIL_LIBC_BEFORE_KERNEL" -gt 0 ]; then
+				echo "FAILED libc before kernel test: $f"
+				_LIBC_BEFORE_KERNEL_FAILED="$(( _LIBC_BEFORE_KERNEL_FAILED + 1 ))"
+			else
+				echo "PASSED libc before kernel test: $f"
+				_LIBC_BEFORE_KERNEL_PASSED="$(( _LIBC_BEFORE_KERNEL_PASSED + 1))"
+			fi
+		fi
+
+		# kernel header included before libc
+		if [ "$KERNEL_LIBC_TEST" != 0 ]; then
+			cat > "$_KERNEL_BEFORE_LIBC" << EOF_KERNEL_BEFORE_LIBC
+#include <$f>
+#include <libc_headers.h>
+EOF_KERNEL_BEFORE_LIBC
+			echo \
+				$CC -Wall -c -nostdinc $GCC_INC \
+				-I . -I "$COMPILE_TEST_INC" \
+				-I "$COMPILE_TEST_INC/$ARCH_TRIPLET" \
+				-o /dev/null "$_KERNEL_BEFORE_LIBC"
+			$CC -Wall -c -nostdinc $GCC_INC \
+				-I . -I "$COMPILE_TEST_INC" \
+				-I "$COMPILE_TEST_INC/$ARCH_TRIPLET" \
+				-o /dev/null "$_KERNEL_BEFORE_LIBC" \
+				|| _FAIL_KERNEL_BEFORE_LIBC=1
+
+			# report errors
+			if [ "$_FAIL_KERNEL_BEFORE_LIBC" -gt 0 ]; then
+				echo "FAILED kernel before libc test: $f"
+				_KERNEL_BEFORE_LIBC_FAILED="$(( _KERNEL_BEFORE_LIBC_FAILED + 1 ))"
+			else
+				echo "PASSED kernel before libc test: $f"
+				_KERNEL_BEFORE_LIBC_PASSED="$(( _KERNEL_BEFORE_LIBC_PASSED + 1))"
+			fi
+		fi
+
+		# libc summary
+		if [ "$_FAIL_LIBC_BEFORE_KERNEL" -gt 0 -o "$_FAIL_KERNEL_BEFORE_LIBC" -gt 0 ]; then
+			_LIBC_FAILED="$(( _LIBC_FAILED + 1))"
+		else
+			_LIBC_PASSED="$(( _LIBC_PASSED + 1))"
+		fi
+
+		if [ "$KEEP" = "0" ]; then
+			rm -f "$_LIBC_BEFORE_KERNEL" "$_KERNEL_BEFORE_LIBC"
+		fi
+	fi # LIBC_TEST
+done
+
+cat << EOF_STATS
+
+Kernel header compile test statistics:
+
+$_FAILED files failed the kernel header compile test.
+$_PASSED files passed the kernel header compile test.
+
+EOF_STATS
+
+if [ "$LIBC_TEST" != 0 ]; then
+	cat << EOF_LIBC_STATS
+libc and kernel header compatibility test statistics:
+EOF_LIBC_STATS
+
+if [ "$LIBC_KERNEL_TEST" != 0 ] && [ "$KERNEL_LIBC_TEST" != 0 ]; then
+	cat << EOF_LIBC_COMBINED
+$_LIBC_FAILED files failed the libc compatibility test.
+$_LIBC_PASSED files passed the libc compatibility test.
+EOF_LIBC_COMBINED
+fi
+
+if [ "$LIBC_KERNEL_TEST" != 0 ]; then
+	cat << EOF_LIBC_KERNEL
+$_LIBC_BEFORE_KERNEL_FAILED files failed libc before kernel include test.
+$_LIBC_BEFORE_KERNEL_PASSED files passed libc before kernel include test.
+EOF_LIBC_KERNEL
+fi
+
+if [ "$KERNEL_LIBC_TEST" != 0 ]; then
+	cat << EOF_KERNEL_LIBC
+$_KERNEL_BEFORE_LIBC_FAILED files failed kernel before libc include test.
+$_KERNEL_BEFORE_LIBC_PASSED files passed kernel before libc include test.
+EOF_KERNEL_LIBC
+fi
+
+fi # LIBC_TEST
+
+# return value, summary of all failures.
+if [ "$(( $_FAILED + $_LIBC_BEFORE_KERNEL_FAILED + $_KERNEL_BEFORE_LIBC_FAILED ))" != 0 ]; then
+	exit 1
+else
+	exit 0
+fi
-- 
2.13.3

[PATCH v06 15/36] uapi linux/socket.h: include sys/socket.h in user space

[Mikko Rapeli]

This libc header has sockaddr definition in user space.

Fixes user space compilation errors like these from kernel headers including
only linux/socket.h:

error: field ‘ifru_addr’ has incomplete type
struct sockaddr ifru_addr;
error: field ‘_sockaddr’ has incomplete type
struct sockaddr  _sockaddr;
error: invalid application of ‘sizeof’ to incomplete type ‘struct sockaddr’

With this following uapi headers now compile in user space:

rdma/rdma_user_rxe.h
linux/vm_sockets.h
linux/ncp_fs.h
linux/nfc.h
linux/phonet.h

Signed-off-by: Mikko Rapeli <mikko.rapeli@iki.fi>
Cc: netdev@vger.kernel.org
Cc: Dmitry V. Levin <ldv@altlinux.org>
---
 include/uapi/linux/socket.h | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/include/uapi/linux/socket.h b/include/uapi/linux/socket.h
index 76ab0c68561e..8a81197cc08b 100644
--- a/include/uapi/linux/socket.h
+++ b/include/uapi/linux/socket.h
@@ -1,6 +1,10 @@
 #ifndef _UAPI_LINUX_SOCKET_H
 #define _UAPI_LINUX_SOCKET_H
 
+#ifndef __KERNEL__
+#include <sys/socket.h>
+#endif
+
 /*
  * Desired design of maximum size and alignment (see RFC2553)
  */
-- 
2.13.3

[PATCH v06 18/36] uapi linux/errqueue.h: include linux/time.h in user space

[Mikko Rapeli]

linux/time.h conflicts with user space header time.h. Try to be compatible
with both.

Fixes userspace compilation error:

error: array type has incomplete element type
 struct timespec ts[3];

Signed-off-by: Mikko Rapeli <mikko.rapeli@iki.fi>
Cc: Willem de Bruijn <willemb@google.com>
Cc: Soheil Hassas Yeganeh <soheil@google.com>
Cc: netdev@vger.kernel.org
---
 include/uapi/linux/errqueue.h | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/include/uapi/linux/errqueue.h b/include/uapi/linux/errqueue.h
index 07bdce1f444a..b310b2c6d94f 100644
--- a/include/uapi/linux/errqueue.h
+++ b/include/uapi/linux/errqueue.h
@@ -3,6 +3,12 @@
 
 #include <linux/types.h>
 
+#ifdef __KERNEL__
+#include <linux/time.h>
+#else
+#include <time.h>
+#endif /* __KERNEL__ */
+
 struct sock_extended_err {
 	__u32	ee_errno;	
 	__u8	ee_origin;
-- 
2.13.3

[PATCH v06 20/36] uapi linux/scc.h: include linux/sockios.h

[Mikko Rapeli]

Fixes userspace compilation error:

error: ‘SIOCDEVPRIVATE’ undeclared here (not in a function)
  SIOCSCCRESERVED = SIOCDEVPRIVATE

Signed-off-by: Mikko Rapeli <mikko.rapeli@iki.fi>
Cc: netdev@vger.kernel.org
Cc: linux-hams@vger.kernel.org
Cc: Joerg Reuter <jreuter@yaina.de>
---
 include/uapi/linux/scc.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/include/uapi/linux/scc.h b/include/uapi/linux/scc.h
index 72b6b8153374..2992b88ca4ce 100644
--- a/include/uapi/linux/scc.h
+++ b/include/uapi/linux/scc.h
@@ -3,6 +3,7 @@
 #ifndef _UAPI_SCC_H
 #define _UAPI_SCC_H
 
+#include <linux/sockios.h>
 
 /* selection of hardware types */
 
-- 
2.13.3

[PATCH v06 35/36] uapi linux/tls.h: don't include <net/tcp.h> in user space

[Mikko Rapeli]

It is not needed and not part of uapi headers, but causes
user space compilation error:

fatal error: net/tcp.h: No such file or directory
 #include <net/tcp.h>
                     ^

Signed-off-by: Mikko Rapeli <mikko.rapeli@iki.fi>
Cc: Dave Watson <davejwatson@fb.com>
Cc: Ilya Lesokhin <ilyal@mellanox.com>
Cc: Aviad Yehezkel <aviadye@mellanox.com>
---
 include/uapi/linux/tls.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/include/uapi/linux/tls.h b/include/uapi/linux/tls.h
index cc1d21db35d8..d87c698623f2 100644
--- a/include/uapi/linux/tls.h
+++ b/include/uapi/linux/tls.h
@@ -37,7 +37,9 @@
 #include <asm/byteorder.h>
 #include <linux/socket.h>
 #include <linux/tcp.h>
+#ifdef __KERNEL__
 #include <net/tcp.h>
+#endif
 
 /* TLS socket options */
 #define TLS_TX			1	/* Set transmit parameters */
-- 
2.13.3

Re: [PATCH v06 18/36] uapi linux/errqueue.h: include linux/time.h in user space

[Willem de Bruijn]

On Sun, Aug 6, 2017 at 12:44 PM, Mikko Rapeli <mikko.rapeli@iki.fi> wrote:
> linux/time.h conflicts with user space header time.h. Try to be compatible
> with both.
>
> Fixes userspace compilation error:
>
> error: array type has incomplete element type
>  struct timespec ts[3];
>
> Signed-off-by: Mikko Rapeli <mikko.rapeli@iki.fi>
> Cc: Willem de Bruijn <willemb@google.com>
> Cc: Soheil Hassas Yeganeh <soheil@google.com>
> Cc: netdev@vger.kernel.org
> ---
>  include/uapi/linux/errqueue.h | 6 ++++++
>  1 file changed, 6 insertions(+)
>
> diff --git a/include/uapi/linux/errqueue.h b/include/uapi/linux/errqueue.h
> index 07bdce1f444a..b310b2c6d94f 100644
> --- a/include/uapi/linux/errqueue.h
> +++ b/include/uapi/linux/errqueue.h
> @@ -3,6 +3,12 @@
>
>  #include <linux/types.h>
>
> +#ifdef __KERNEL__
> +#include <linux/time.h>
> +#else
> +#include <time.h>
> +#endif /* __KERNEL__ */

This will break applications that include <linux/time.h> manually.

I previously sent a patch to use libc-compat to make compilation succeed
when both are included in the case where <linux/time.h> is included after
<time.h>.

  https://lkml.org/lkml/2016/9/12/872

The inverse will require changes to the libc header to avoid redefining
symbols already defined by <linux/time.h>

The second patch in that 2-patch set included <linux/time.h>
unconditionally after the fix. This broke builds that also included
<time.h> in the wrong order. I did not resubmit the first patch as a
stand-alone, as it is not sufficient to avoid breakage.

Re: [PATCH v06 18/36] uapi linux/errqueue.h: include linux/time.h in user space

[Willem de Bruijn]

On Sun, Aug 6, 2017 at 4:23 PM, Willem de Bruijn
<willemdebruijn.kernel@gmail.com> wrote:
> On Sun, Aug 6, 2017 at 12:44 PM, Mikko Rapeli <mikko.rapeli@iki.fi> wrote:
>> linux/time.h conflicts with user space header time.h. Try to be compatible
>> with both.
>>
>> Fixes userspace compilation error:
>>
>> error: array type has incomplete element type
>>  struct timespec ts[3];
>>
>> Signed-off-by: Mikko Rapeli <mikko.rapeli@iki.fi>
>> Cc: Willem de Bruijn <willemb@google.com>
>> Cc: Soheil Hassas Yeganeh <soheil@google.com>
>> Cc: netdev@vger.kernel.org
>> ---
>>  include/uapi/linux/errqueue.h | 6 ++++++
>>  1 file changed, 6 insertions(+)
>>
>> diff --git a/include/uapi/linux/errqueue.h b/include/uapi/linux/errqueue.h
>> index 07bdce1f444a..b310b2c6d94f 100644
>> --- a/include/uapi/linux/errqueue.h
>> +++ b/include/uapi/linux/errqueue.h
>> @@ -3,6 +3,12 @@
>>
>>  #include <linux/types.h>
>>
>> +#ifdef __KERNEL__
>> +#include <linux/time.h>
>> +#else
>> +#include <time.h>
>> +#endif /* __KERNEL__ */
>
> This will break applications that include <linux/time.h> manually.

Also, the patch title reads "include <linux/time.h> in user space",
but it includes <time.h> in that environment.

Re: [PATCH v06 18/36] uapi linux/errqueue.h: include linux/time.h in user space

[Mikko Rapeli]

On Sun, Aug 06, 2017 at 04:23:16PM -0400, Willem de Bruijn wrote:
> On Sun, Aug 6, 2017 at 12:44 PM, Mikko Rapeli <mikko.rapeli-X3B1VOXEql0@public.gmane.org> wrote:
> > linux/time.h conflicts with user space header time.h. Try to be compatible
> > with both.
> >
> > Fixes userspace compilation error:
> >
> > error: array type has incomplete element type
> >  struct timespec ts[3];
> >
> > Signed-off-by: Mikko Rapeli <mikko.rapeli-X3B1VOXEql0@public.gmane.org>
> > Cc: Willem de Bruijn <willemb-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
> > Cc: Soheil Hassas Yeganeh <soheil-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
> > Cc: netdev-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> > ---
> >  include/uapi/linux/errqueue.h | 6 ++++++
> >  1 file changed, 6 insertions(+)
> >
> > diff --git a/include/uapi/linux/errqueue.h b/include/uapi/linux/errqueue.h
> > index 07bdce1f444a..b310b2c6d94f 100644
> > --- a/include/uapi/linux/errqueue.h
> > +++ b/include/uapi/linux/errqueue.h
> > @@ -3,6 +3,12 @@
> >
> >  #include <linux/types.h>
> >
> > +#ifdef __KERNEL__
> > +#include <linux/time.h>
> > +#else
> > +#include <time.h>
> > +#endif /* __KERNEL__ */
> 
> This will break applications that include <linux/time.h> manually.
> I previously sent a patch to use libc-compat to make compilation succeed
> when both are included in the case where <linux/time.h> is included after
> <time.h>.
> 
>   https://lkml.org/lkml/2016/9/12/872
> 
> The inverse will require changes to the libc header to avoid redefining
> symbols already defined by <linux/time.h>
> 
> The second patch in that 2-patch set included <linux/time.h>
> unconditionally after the fix. This broke builds that also included
> <time.h> in the wrong order. I did not resubmit the first patch as a
> stand-alone, as it is not sufficient to avoid breakage.

I wasn't aware of your change, but I was about to send this to fix the
case when glibc <time.h> is included before <linux/time.h>:

https://github.com/mcfrisk/linux/commit/f3952a27b8a21c6478d26e6246055383483f6a66

but you also ran into problems where <linux/time.h> is included before
<time.h> which need fixes in libc header side.

So how to proceed with these?

I don't like leaving a few dozen non-compiling header files into uapi.

-Mikko

Re: [PATCH v06 18/36] uapi linux/errqueue.h: include linux/time.h in user space

[Mikko Rapeli]

On Sun, Aug 06, 2017 at 04:26:50PM -0400, Willem de Bruijn wrote:
> On Sun, Aug 6, 2017 at 4:23 PM, Willem de Bruijn
> <willemdebruijn.kernel@gmail.com> wrote:
> > On Sun, Aug 6, 2017 at 12:44 PM, Mikko Rapeli <mikko.rapeli@iki.fi> wrote:
> >> linux/time.h conflicts with user space header time.h. Try to be compatible
> >> with both.
> >>
> >> Fixes userspace compilation error:
> >>
> >> error: array type has incomplete element type
> >>  struct timespec ts[3];
> >>
> >> Signed-off-by: Mikko Rapeli <mikko.rapeli@iki.fi>
> >> Cc: Willem de Bruijn <willemb@google.com>
> >> Cc: Soheil Hassas Yeganeh <soheil@google.com>
> >> Cc: netdev@vger.kernel.org
> >> ---
> >>  include/uapi/linux/errqueue.h | 6 ++++++
> >>  1 file changed, 6 insertions(+)
> >>
> >> diff --git a/include/uapi/linux/errqueue.h b/include/uapi/linux/errqueue.h
> >> index 07bdce1f444a..b310b2c6d94f 100644
> >> --- a/include/uapi/linux/errqueue.h
> >> +++ b/include/uapi/linux/errqueue.h
> >> @@ -3,6 +3,12 @@
> >>
> >>  #include <linux/types.h>
> >>
> >> +#ifdef __KERNEL__
> >> +#include <linux/time.h>
> >> +#else
> >> +#include <time.h>
> >> +#endif /* __KERNEL__ */
> >
> > This will break applications that include <linux/time.h> manually.
> 
> Also, the patch title reads "include <linux/time.h> in user space",
> but it includes <time.h> in that environment.

Oops, missed while squashing some commits. Will fix. Thanks!

-Mikko

Re: [PATCH v06 18/36] uapi linux/errqueue.h: include linux/time.h in user space

[Willem de Bruijn]

>> > +#ifdef __KERNEL__
>> > +#include <linux/time.h>
>> > +#else
>> > +#include <time.h>
>> > +#endif /* __KERNEL__ */
>>
>> This will break applications that include <linux/time.h> manually.
>> I previously sent a patch to use libc-compat to make compilation succeed
>> when both are included in the case where <linux/time.h> is included after
>> <time.h>.
>>
>>   https://lkml.org/lkml/2016/9/12/872
>>
>> The inverse will require changes to the libc header to avoid redefining
>> symbols already defined by <linux/time.h>
>>
>> The second patch in that 2-patch set included <linux/time.h>
>> unconditionally after the fix. This broke builds that also included
>> <time.h> in the wrong order. I did not resubmit the first patch as a
>> stand-alone, as it is not sufficient to avoid breakage.
>
> I wasn't aware of your change, but I was about to send this to fix the
> case when glibc <time.h> is included before <linux/time.h>:
>
> https://github.com/mcfrisk/linux/commit/f3952a27b8a21c6478d26e6246055383483f6a66

There are a few differences between the two. Including <time.h> does not
unconditionally define all the symbols. Some are conditional on additional
state, such as __timespec_defined.

> but you also ran into problems where <linux/time.h> is included before
> <time.h> which need fixes in libc header side.
>
> So how to proceed with these?

The libc-compat change is a good fix that can be submitted on its own.

> I don't like leaving a few dozen non-compiling header files into uapi.

I agree, but I do not see a simple solution.

Unless libc has the analogous change, including either <time.h> or
<linux/time.h> in userspace can unfortunately cause breakage.

The added include if __KERNEL__ is defined should be safe, though.

Re: [PATCH v06 18/36] uapi linux/errqueue.h: include linux/time.h in user space

[Mikko Rapeli]

On Sun, Aug 06, 2017 at 05:24:20PM -0400, Willem de Bruijn wrote:
> >> > +#ifdef __KERNEL__
> >> > +#include <linux/time.h>
> >> > +#else
> >> > +#include <time.h>
> >> > +#endif /* __KERNEL__ */
> >>
> >> This will break applications that include <linux/time.h> manually.
> >> I previously sent a patch to use libc-compat to make compilation succeed
> >> when both are included in the case where <linux/time.h> is included after
> >> <time.h>.
> >>
> >>   https://lkml.org/lkml/2016/9/12/872
> >>
> >> The inverse will require changes to the libc header to avoid redefining
> >> symbols already defined by <linux/time.h>
> >>
> >> The second patch in that 2-patch set included <linux/time.h>
> >> unconditionally after the fix. This broke builds that also included
> >> <time.h> in the wrong order. I did not resubmit the first patch as a
> >> stand-alone, as it is not sufficient to avoid breakage.
> >
> > I wasn't aware of your change, but I was about to send this to fix the
> > case when glibc <time.h> is included before <linux/time.h>:
> >
> > https://github.com/mcfrisk/linux/commit/f3952a27b8a21c6478d26e6246055383483f6a66
> 
> There are a few differences between the two. Including <time.h> does not
> unconditionally define all the symbols. Some are conditional on additional
> state, such as __timespec_defined.

Yep, your patch seems better for libc-compat.h. Could you send it again?

> > but you also ran into problems where <linux/time.h> is included before
> > <time.h> which need fixes in libc header side.
> >
> > So how to proceed with these?
> 
> The libc-compat change is a good fix that can be submitted on its own.

Yes, please do so.

> > I don't like leaving a few dozen non-compiling header files into uapi.
> 
> I agree, but I do not see a simple solution.
> 
> Unless libc has the analogous change, including either <time.h> or
> <linux/time.h> in userspace can unfortunately cause breakage.
> 
> The added include if __KERNEL__ is defined should be safe, though.

Yes, for the kernel side, but your libc-compat change would nice for
userspace, where something will break for sure, but providing source
API compatibility is sometimes impossible.

To summarize, this change from me, and your libc-compat.c for time.h, or?

-Mikko

Re: [PATCH v06 18/36] uapi linux/errqueue.h: include linux/time.h in user space

[Willem de Bruijn]

On Sun, Aug 6, 2017 at 5:33 PM, Mikko Rapeli <mikko.rapeli@iki.fi> wrote:
> On Sun, Aug 06, 2017 at 05:24:20PM -0400, Willem de Bruijn wrote:
>> >> > +#ifdef __KERNEL__
>> >> > +#include <linux/time.h>
>> >> > +#else
>> >> > +#include <time.h>
>> >> > +#endif /* __KERNEL__ */
>> >>
>> >> This will break applications that include <linux/time.h> manually.
>> >> I previously sent a patch to use libc-compat to make compilation succeed
>> >> when both are included in the case where <linux/time.h> is included after
>> >> <time.h>.
>> >>
>> >>   https://lkml.org/lkml/2016/9/12/872
>> >>
>> >> The inverse will require changes to the libc header to avoid redefining
>> >> symbols already defined by <linux/time.h>
>> >>
>> >> The second patch in that 2-patch set included <linux/time.h>
>> >> unconditionally after the fix. This broke builds that also included
>> >> <time.h> in the wrong order. I did not resubmit the first patch as a
>> >> stand-alone, as it is not sufficient to avoid breakage.
>> >
>> > I wasn't aware of your change, but I was about to send this to fix the
>> > case when glibc <time.h> is included before <linux/time.h>:
>> >
>> > https://github.com/mcfrisk/linux/commit/f3952a27b8a21c6478d26e6246055383483f6a66
>>
>> There are a few differences between the two. Including <time.h> does not
>> unconditionally define all the symbols. Some are conditional on additional
>> state, such as __timespec_defined.
>
> Yep, your patch seems better for libc-compat.h. Could you send it again?

Okay. Or feel free to include it in the patchset if that helps resolve
dependencies.

>> > I don't like leaving a few dozen non-compiling header files into uapi.
>>
>> I agree, but I do not see a simple solution.
>>
>> Unless libc has the analogous change, including either <time.h> or
>> <linux/time.h> in userspace can unfortunately cause breakage.
>>
>> The added include if __KERNEL__ is defined should be safe, though.
>
> Yes, for the kernel side, but your libc-compat change would nice for
> userspace, where something will break for sure, but providing source
> API compatibility is sometimes impossible.
>
> To summarize, this change from me, and your libc-compat.c for time.h, or?

I'm still afraid that this patch as is will break builds that include
<linux/time.h> first.

Re: [PATCH v06 18/36] uapi linux/errqueue.h: include linux/time.h in user space

[Mikko Rapeli]

On Sun, Aug 06, 2017 at 05:42:13PM -0400, Willem de Bruijn wrote:
> On Sun, Aug 6, 2017 at 5:33 PM, Mikko Rapeli <mikko.rapeli-X3B1VOXEql0@public.gmane.org> wrote:
> > On Sun, Aug 06, 2017 at 05:24:20PM -0400, Willem de Bruijn wrote:
> >> >> > +#ifdef __KERNEL__
> >> >> > +#include <linux/time.h>
> >> >> > +#else
> >> >> > +#include <time.h>
> >> >> > +#endif /* __KERNEL__ */
> >> >>
> >> >> This will break applications that include <linux/time.h> manually.
> >> >> I previously sent a patch to use libc-compat to make compilation succeed
> >> >> when both are included in the case where <linux/time.h> is included after
> >> >> <time.h>.
> >> >>
> >> >>   https://lkml.org/lkml/2016/9/12/872
> >> >>
> >> >> The inverse will require changes to the libc header to avoid redefining
> >> >> symbols already defined by <linux/time.h>
> >> >>
> >> >> The second patch in that 2-patch set included <linux/time.h>
> >> >> unconditionally after the fix. This broke builds that also included
> >> >> <time.h> in the wrong order. I did not resubmit the first patch as a
> >> >> stand-alone, as it is not sufficient to avoid breakage.
> >> >
> >> > I wasn't aware of your change, but I was about to send this to fix the
> >> > case when glibc <time.h> is included before <linux/time.h>:
> >> >
> >> > https://github.com/mcfrisk/linux/commit/f3952a27b8a21c6478d26e6246055383483f6a66
> >>
> >> There are a few differences between the two. Including <time.h> does not
> >> unconditionally define all the symbols. Some are conditional on additional
> >> state, such as __timespec_defined.
> >
> > Yep, your patch seems better for libc-compat.h. Could you send it again?
> 
> Okay. Or feel free to include it in the patchset if that helps resolve
> dependencies.

If you don't have the time, I will send tomorrow a new version of this
patch which fixes the commit topic and before that your libc-compat.h change
so both could be applied together.

Feel free to be faster :)

> >> > I don't like leaving a few dozen non-compiling header files into uapi.
> >>
> >> I agree, but I do not see a simple solution.
> >>
> >> Unless libc has the analogous change, including either <time.h> or
> >> <linux/time.h> in userspace can unfortunately cause breakage.
> >>
> >> The added include if __KERNEL__ is defined should be safe, though.
> >
> > Yes, for the kernel side, but your libc-compat change would nice for
> > userspace, where something will break for sure, but providing source
> > API compatibility is sometimes impossible.
> >
> > To summarize, this change from me, and your libc-compat.c for time.h, or?
> 
> I'm still afraid that this patch as is will break builds that include
> <linux/time.h> first.

I agree, but I also want uapi headers to cleanly compile. I know this might
break stuff on userspace side which rely on these broken header file
dependencies, but if the fix to just re-order include
statements I'm fine with it, also when the complaints hit my inbox.

If I had the CPU time, memory and disk space, I'd do a full yocto distro
build to see how badly userspace could break but I don't at home.

-Mikko

Re: [PATCH v06 18/36] uapi linux/errqueue.h: include linux/time.h in user space

[Willem de Bruijn]

On Sun, Aug 6, 2017 at 5:52 PM, Mikko Rapeli <mikko.rapeli-X3B1VOXEql0@public.gmane.org> wrote:
> On Sun, Aug 06, 2017 at 05:42:13PM -0400, Willem de Bruijn wrote:
>> On Sun, Aug 6, 2017 at 5:33 PM, Mikko Rapeli <mikko.rapeli-X3B1VOXEql0@public.gmane.org> wrote:
>> > On Sun, Aug 06, 2017 at 05:24:20PM -0400, Willem de Bruijn wrote:
>> >> >> > +#ifdef __KERNEL__
>> >> >> > +#include <linux/time.h>
>> >> >> > +#else
>> >> >> > +#include <time.h>
>> >> >> > +#endif /* __KERNEL__ */
>> >> >>
>> >> >> This will break applications that include <linux/time.h> manually.
>> >> >> I previously sent a patch to use libc-compat to make compilation succeed
>> >> >> when both are included in the case where <linux/time.h> is included after
>> >> >> <time.h>.
>> >> >>
>> >> >>   https://lkml.org/lkml/2016/9/12/872
>> >> >>
>> >> >> The inverse will require changes to the libc header to avoid redefining
>> >> >> symbols already defined by <linux/time.h>
>> >> >>
>> >> >> The second patch in that 2-patch set included <linux/time.h>
>> >> >> unconditionally after the fix. This broke builds that also included
>> >> >> <time.h> in the wrong order. I did not resubmit the first patch as a
>> >> >> stand-alone, as it is not sufficient to avoid breakage.
>> >> >
>> >> > I wasn't aware of your change, but I was about to send this to fix the
>> >> > case when glibc <time.h> is included before <linux/time.h>:
>> >> >
>> >> > https://github.com/mcfrisk/linux/commit/f3952a27b8a21c6478d26e6246055383483f6a66
>> >>
>> >> There are a few differences between the two. Including <time.h> does not
>> >> unconditionally define all the symbols. Some are conditional on additional
>> >> state, such as __timespec_defined.
>> >
>> > Yep, your patch seems better for libc-compat.h. Could you send it again?
>>
>> Okay. Or feel free to include it in the patchset if that helps resolve
>> dependencies.
>
> If you don't have the time, I will send tomorrow a new version of this
> patch which fixes the commit topic and before that your libc-compat.h change
> so both could be applied together.

Please do. Thanks!

Re: [PATCH v06 15/36] uapi linux/socket.h: include sys/socket.h in user space

[Dmitry V. Levin]

[-- Attachment #1: Type: text/plain, Size: 1420 bytes --]

On Sun, Aug 06, 2017 at 06:44:06PM +0200, Mikko Rapeli wrote:
> This libc header has sockaddr definition in user space.
> 
> Fixes user space compilation errors like these from kernel headers including
> only linux/socket.h:
> 
> error: field ‘ifru_addr’ has incomplete type
> struct sockaddr ifru_addr;
> error: field ‘_sockaddr’ has incomplete type
> struct sockaddr  _sockaddr;
> error: invalid application of ‘sizeof’ to incomplete type ‘struct sockaddr’
> 
> With this following uapi headers now compile in user space:
> 
> rdma/rdma_user_rxe.h
> linux/vm_sockets.h
> linux/ncp_fs.h
> linux/nfc.h
> linux/phonet.h
> 
> Signed-off-by: Mikko Rapeli <mikko.rapeli-X3B1VOXEql0@public.gmane.org>
> Cc: netdev-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> Cc: Dmitry V. Levin <ldv-u2l5PoMzF/Vg9hUCZPvPmw@public.gmane.org>
> ---
>  include/uapi/linux/socket.h | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/include/uapi/linux/socket.h b/include/uapi/linux/socket.h
> index 76ab0c68561e..8a81197cc08b 100644
> --- a/include/uapi/linux/socket.h
> +++ b/include/uapi/linux/socket.h
> @@ -1,6 +1,10 @@
>  #ifndef _UAPI_LINUX_SOCKET_H
>  #define _UAPI_LINUX_SOCKET_H
>  
> +#ifndef __KERNEL__
> +#include <sys/socket.h>
> +#endif

This is scary because of infamous libc vs uapi interoperability issues.
Couldn't we fix affected headers instead?


-- 
ldv

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: [PATCH v06 35/36] uapi linux/tls.h: don't include <net/tcp.h> in user space

[Dmitry V. Levin]

[-- Attachment #1: Type: text/plain, Size: 939 bytes --]

On Sun, Aug 06, 2017 at 06:44:26PM +0200, Mikko Rapeli wrote:
> It is not needed and not part of uapi headers, but causes
> user space compilation error:
> 
> fatal error: net/tcp.h: No such file or directory
>  #include <net/tcp.h>
>                      ^
> 
> Signed-off-by: Mikko Rapeli <mikko.rapeli@iki.fi>
> Cc: Dave Watson <davejwatson@fb.com>
> Cc: Ilya Lesokhin <ilyal@mellanox.com>
> Cc: Aviad Yehezkel <aviadye@mellanox.com>
> ---
>  include/uapi/linux/tls.h | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/include/uapi/linux/tls.h b/include/uapi/linux/tls.h
> index cc1d21db35d8..d87c698623f2 100644
> --- a/include/uapi/linux/tls.h
> +++ b/include/uapi/linux/tls.h
> @@ -37,7 +37,9 @@
>  #include <asm/byteorder.h>
>  #include <linux/socket.h>
>  #include <linux/tcp.h>
> +#ifdef __KERNEL__
>  #include <net/tcp.h>
> +#endif

Let's move it to include/net/tls.h instead.


-- 
ldv

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: [PATCH v06 15/36] uapi linux/socket.h: include sys/socket.h in user space

[Arnd Bergmann]

On Wed, Aug 9, 2017 at 1:00 AM, Dmitry V. Levin <ldv-u2l5PoMzF/Vg9hUCZPvPmw@public.gmane.org> wrote:
> On Sun, Aug 06, 2017 at 06:44:06PM +0200, Mikko Rapeli wrote:
>> This libc header has sockaddr definition in user space.
>>
>> Fixes user space compilation errors like these from kernel headers including
>> only linux/socket.h:
>>
>> error: field ‘ifru_addr’ has incomplete type
>> struct sockaddr ifru_addr;
>> error: field ‘_sockaddr’ has incomplete type
>> struct sockaddr  _sockaddr;
>> error: invalid application of ‘sizeof’ to incomplete type ‘struct sockaddr’
>>
>> With this following uapi headers now compile in user space:
>>
>> rdma/rdma_user_rxe.h
>> linux/vm_sockets.h
>> linux/ncp_fs.h
>> linux/nfc.h
>> linux/phonet.h
>>

>>
>> +#ifndef __KERNEL__
>> +#include <sys/socket.h>
>> +#endif
>
> This is scary because of infamous libc vs uapi interoperability issues.
> Couldn't we fix affected headers instead?

Yes, I think that would be better. Maybe we can introduce a
'struct __kernel_sockaddr' in include/uapi/linux/socket.h to
go along with __kernel_sa_family_t and __kernel_sockaddr_storage,
and then use those types in the other kernel headers?

       Arnd

Re: [PATCH v06 35/36] uapi linux/tls.h: don't include <net/tcp.h> in user space

[Dmitry V. Levin]

[-- Attachment #1: Type: text/plain, Size: 1173 bytes --]

On Wed, Aug 09, 2017 at 02:25:54AM +0300, Dmitry V. Levin wrote:
> On Sun, Aug 06, 2017 at 06:44:26PM +0200, Mikko Rapeli wrote:
> > It is not needed and not part of uapi headers, but causes
> > user space compilation error:
> > 
> > fatal error: net/tcp.h: No such file or directory
> >  #include <net/tcp.h>
> >                      ^
> > 
> > Signed-off-by: Mikko Rapeli <mikko.rapeli@iki.fi>
> > Cc: Dave Watson <davejwatson@fb.com>
> > Cc: Ilya Lesokhin <ilyal@mellanox.com>
> > Cc: Aviad Yehezkel <aviadye@mellanox.com>
> > ---
> >  include/uapi/linux/tls.h | 2 ++
> >  1 file changed, 2 insertions(+)
> > 
> > diff --git a/include/uapi/linux/tls.h b/include/uapi/linux/tls.h
> > index cc1d21db35d8..d87c698623f2 100644
> > --- a/include/uapi/linux/tls.h
> > +++ b/include/uapi/linux/tls.h
> > @@ -37,7 +37,9 @@
> >  #include <asm/byteorder.h>
> >  #include <linux/socket.h>
> >  #include <linux/tcp.h>
> > +#ifdef __KERNEL__
> >  #include <net/tcp.h>
> > +#endif
> 
> Let's move it to include/net/tls.h instead.

So everybody ignored this and *new* uapi header was released
in a totally unusable form along with v4.13.


-- 
ldv

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: [PATCH v06 35/36] uapi linux/tls.h: don't include <net/tcp.h> in user space

[Mikko Rapeli]

On Mon, Sep 04, 2017 at 07:15:11PM +0300, Dmitry V. Levin wrote:
> On Wed, Aug 09, 2017 at 02:25:54AM +0300, Dmitry V. Levin wrote:
> > On Sun, Aug 06, 2017 at 06:44:26PM +0200, Mikko Rapeli wrote:
> > > It is not needed and not part of uapi headers, but causes
> > > user space compilation error:
> > > 
> > > fatal error: net/tcp.h: No such file or directory
> > >  #include <net/tcp.h>
> > >                      ^
> > > 
> > > Signed-off-by: Mikko Rapeli <mikko.rapeli-X3B1VOXEql0@public.gmane.org>
> > > Cc: Dave Watson <davejwatson-b10kYP2dOMg@public.gmane.org>
> > > Cc: Ilya Lesokhin <ilyal-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> > > Cc: Aviad Yehezkel <aviadye-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> > > ---
> > >  include/uapi/linux/tls.h | 2 ++
> > >  1 file changed, 2 insertions(+)
> > > 
> > > diff --git a/include/uapi/linux/tls.h b/include/uapi/linux/tls.h
> > > index cc1d21db35d8..d87c698623f2 100644
> > > --- a/include/uapi/linux/tls.h
> > > +++ b/include/uapi/linux/tls.h
> > > @@ -37,7 +37,9 @@
> > >  #include <asm/byteorder.h>
> > >  #include <linux/socket.h>
> > >  #include <linux/tcp.h>
> > > +#ifdef __KERNEL__
> > >  #include <net/tcp.h>
> > > +#endif
> > 
> > Let's move it to include/net/tls.h instead.
> 
> So everybody ignored this and *new* uapi header was released
> in a totally unusable form along with v4.13.

Should issues like these be filed into bugzilla instead?

Maybe emails are easier to ignore than bugzilla tickets.

-Mikko