[PATCH v4 02/79] scripts/headers_compile_test.sh: compile test script for exported headers

From: Mikko Rapeli <mikko.rapeli@iki.fi>
To: linux-kernel@vger.kernel.org
Cc: mikko.rapeli@iki.fi
Subject: [PATCH v4 02/79] scripts/headers_compile_test.sh: compile test script for exported headers
Date: Thu, 15 Oct 2015 07:55:40 +0200	[thread overview]
Message-ID: <1444888618-4506-3-git-send-email-mikko.rapeli@iki.fi> (raw)
In-Reply-To: <1444888618-4506-1-git-send-email-mikko.rapeli@iki.fi>

Users of kernel header files would be happier if they did not contain
kernel specific parts and would contain #include statements for all
other header files that they depend on, and in general would compile.

For each header file exported to userspace, this script creates
a simple .c file which just includes the header file. Then it
tries to compile it together with minimal header files from GCC
and libc, and reports results.

Default libc and GCC header file locations are parsed from compiler
configuration.

Kernel headers depend on GCC headers so their path is included in
the test compiler command line.

Some gcc and kernel headers depend on libc headers which are made available
by copying from the compiler default location to a temporary location and
removing possibly existing kernel headers from this directory. This is a bit
of a hack but seems to work in multiple environments.

Tested natively on:

Debian unstable, i586-linux-gnu and gcc 4.9.2
Raspbian Wheezy, arm-linux-gnueabihf and gcc 4.6.3
Ubuntu 12.04 LTS, x86_64-linux-gnu and gcc 4.6.3

Tested cross compilation using standard CROSS_COMPILE=/path/to/gcc with:

arm-linux-gnueabihf-gcc (crosstool-NG linaro-1.13.1-4.9-2014.09 - Linaro GCC 4.9-2014.09) 4.9.2 20140904 (prerelease)

Execute the script in the directory where kernel header files are installed.
For example:

$ make headers_install
$ cd usr/include
$ ../../scripts/headers_compile_test.sh

Example statistics from 3.18.4 kernel:

130 files failed the compile test.
609 files passed the compile test.

Example error types from 3.18.4 kernel:

$ ../../scripts/headers_compile_test.sh 2>&1 | \
grep error: | sed -e 's/.*error://g' | sort | uniq -c | sort -rn

    352  unknown type name ‘uint32_t’
    134  unknown type name ‘uint64_t’
    103  unknown type name ‘size_t’
     43  unknown type name ‘__kernel_ulong_t’
     38  unknown type name ‘uint8_t’
     24  unknown type name ‘int32_t’
     22  field ‘addr’ has incomplete type
     18  field ‘tstamp’ has incomplete type
     16  unknown type name ‘__kernel_time_t’
     16  field ‘in’ has incomplete type
     16  field ‘in6’ has incomplete type
     14  unknown type name ‘__be16’
     13  ‘IFNAMSIZ’ undeclared here (not in a function)
     12  unknown type name ‘uint16_t’
      9  field ‘ifru_netmask’ has incomplete type
      9  field ‘ifru_hwaddr’ has incomplete type
      9  field ‘ifru_dstaddr’ has incomplete type
      9  field ‘ifru_broadaddr’ has incomplete type
      9  field ‘ifru_addr’ has incomplete type
      8  unknown type name ‘__kernel_pid_t’
      7  unknown type name ‘u_short’
      7  unknown type name ‘pid_t’
      6  invalid application of ‘sizeof’ to incomplete type ‘struct timespec’
      6  field ‘src’ has incomplete type
      6  field ‘audio_tstamp’ has incomplete type
      6  array type has incomplete element type
      5  unknown type name ‘__kernel_long_t’
      5  requested alignment is not an integer constant
      5  field ‘smsk’ has incomplete type
      4  unknown type name ‘__kernel_uid32_t’
      4  unknown type name ‘__kernel_gid32_t’
      4  ‘ETH_ALEN’ undeclared here (not in a function)
      3  unknown type name ‘int64_t’
      3  unknown type name ‘caddr_t’
      3  ‘IPSET_ERR_TYPE_SPECIFIC’ undeclared here (not in a function)
      3  field ‘trigger_tstamp’ has incomplete type
      3  field ‘src_addr’ has incomplete type
      3  field ‘sin_addr’ has incomplete type
      3  field ‘laddr’ has incomplete type
      3  field ‘id’ has incomplete type
      3  field ‘dmsk’ has incomplete type
      3  field ‘bssid’ has incomplete type
      3  expected specifier-qualifier-list before ‘uint64_t’
      2  unknown type name ‘u_long’
      2  unknown type name ‘stack_t’
      2  unknown type name ‘sigset_t’
      2  unknown type name ‘sa_family_t’
      2  unknown type name ‘__kernel_mode_t’
      2  unknown type name ‘__kernel_key_t’
      2  unknown type name ‘elf_gregset_t’
      2  unknown type name ‘bool’
      2  ‘uint64_t’ undeclared here (not in a function)
      2  ‘true’ undeclared (first use in this function)
      2  ‘NAME_MAX’ undeclared here (not in a function)
      2  ‘__kernel_mode_t’ undeclared here (not in a function)
      2  invalid application of ‘sizeof’ to incomplete type ‘struct sockaddr’
      2  field ‘uc_mcontext’ has incomplete type
      2  field ‘tmsk’ has incomplete type
      2  field ‘tgt’ has incomplete type
      2  field ‘shm_perm’ has incomplete type
      2  field ‘sem_perm’ has incomplete type
      2  field ‘raddr’ has incomplete type
      2  field ‘msg_perm’ has incomplete type
      2  field ‘grp’ has incomplete type
      2  field ‘dst’ has incomplete type
      2  field ‘dst_addr’ has incomplete type
      2  field ‘arp_pa’ has incomplete type
      2  field ‘arp_netmask’ has incomplete type
      2  field ‘arp_ha’ has incomplete type
      2  ‘false’ undeclared (first use in this function)
      1  xen/interface/xen.h: No such file or directory
      1  via_drmclient.h: No such file or directory
      1  unknown type name ‘wait_queue_head_t’
      1  unknown type name ‘snd_seq_client_type_t’
      1  unknown type name ‘int16_t’
      1  unknown type name ‘ino_t’
      1  unknown type name ‘elf_greg_t’
      1  unknown type name ‘elf_fpxregset_t’
      1  unknown type name ‘elf_fpregset_t’
      1  unknown type name ‘__be32’
      1  ‘SIOCDEVPRIVATE’ undeclared here (not in a function)
      1  ‘sa_family_t’ undeclared here (not in a function)
      1  ‘NULL’ undeclared (first use in this function)
      1  ‘MSG_FIN’ undeclared here (not in a function)
      1  ‘MAX_IPOPTLEN’ undeclared here (not in a function)
      1  ‘MAX_ADDR_LEN’ undeclared here (not in a function)
      1  ‘IFHWADDRLEN’ undeclared here (not in a function)
      1  field ‘vmask’ has incomplete type
      1  field ‘vifc_rmt_addr’ has incomplete type
      1  field ‘vifc_lcl_addr’ has incomplete type
      1  field ‘vaddr’ has incomplete type
      1  field ‘uc_chain’ has incomplete type
      1  field ‘tgt_ip’ has incomplete type
      1  field ‘tcp’ has incomplete type
      1  field ‘sw_reserved’ has incomplete type
      1  field ‘sspp_addr’ has incomplete type
      1  field ‘ssp_addr’ has incomplete type
      1  field ‘src_mask’ has incomplete type
      1  field ‘src_ip’ has incomplete type
      1  field ‘spt_address’ has incomplete type
      1  field ‘spp_address’ has incomplete type
      1  field ‘spinfo_address’ has incomplete type
      1  field ‘spc_aaddr’ has incomplete type
      1  field ‘sas_obs_rto_ipaddr’ has incomplete type
      1  field ‘saddr’ has incomplete type
      1  field ‘rtmsg_src’ has incomplete type
      1  field ‘rtmsg_gateway’ has incomplete type
      1  field ‘rtmsg_dst’ has incomplete type
      1  field ‘rt_genmask’ has incomplete type
      1  field ‘rt_gateway’ has incomplete type
      1  field ‘rt_dst’ has incomplete type
      1  field ‘real’ has incomplete type
      1  field ‘prefix’ has incomplete type
      1  field ‘obj_list’ has incomplete type
      1  field ‘mfcc_origin’ has incomplete type
      1  field ‘mfcc_mcastgrp’ has incomplete type
      1  field ‘mf6cc_origin’ has incomplete type
      1  field ‘mf6cc_mcastgrp’ has incomplete type
      1  field ‘mask’ has incomplete type
      1  field ‘ival2’ has incomplete type
      1  field ‘ival1’ has incomplete type
      1  field ‘iph’ has incomplete type
      1  field ‘ip’ has incomplete type
      1  field ‘im_src’ has incomplete type
      1  field ‘im_dst’ has incomplete type
      1  field ‘im6_src’ has incomplete type
      1  field ‘im6_dst’ has incomplete type
      1  field ‘gw’ has incomplete type
      1  field ‘expected’ has incomplete type
      1  field ‘dst_mask’ has incomplete type
      1  field ‘dest_addr’ has incomplete type
      1  field ‘daddr’ has incomplete type
      1  field ‘ap_addr’ has incomplete type
      1  field ‘a6’ has incomplete type
      1  field ‘a4’ has incomplete type
      1  expected specifier-qualifier-list before ‘DECLARE_BITMAP’
      1  expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘snd_seq_client_type_t’
      1  #error "patchkey.h included directly"
      1  ‘DLM_RESNAME_MAXLEN’ undeclared here (not in a function)

Signed-off-by: Mikko Rapeli <mikko.rapeli@iki.fi>
---
 scripts/headers_compile_test.sh | 143 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 143 insertions(+)
 create mode 100755 scripts/headers_compile_test.sh

diff --git a/scripts/headers_compile_test.sh b/scripts/headers_compile_test.sh
new file mode 100755
index 0000000..9a73945
--- /dev/null
+++ b/scripts/headers_compile_test.sh
@@ -0,0 +1,143 @@
+#!/bin/bash
+# bash due to arithmetics and pipefail
+set -euo pipefail
+
+# Debugging
+#set -x
+
+echo Simple compile test for header files meant for userspace.
+
+# sanity test
+if [ ! -d ./linux ]; then
+	echo Sanity check error: ./linux directory not found
+	echo Should be called in usr/include after \'make headers_install\'.
+	echo Returns number of failed files, 0 if none.
+	exit 1
+fi
+
+# Support CC variable for compiler and ccache, and cross compiling.
+# CC is used without quotes to support CC="ccache gcc".
+set +u
+if [ "$CC"foobar == "foobar" ]; then
+	CC=cc
+fi
+
+if [ "$CROSS_COMPILE"foobar != "foobar" ]; then
+	# Using gcc name since some cross compiler tool chains don't provide
+	# the cc symlink
+	CC="$CROSS_COMPILE"gcc
+fi
+set -u
+
+# Kernel headers refer to some gcc and libc headers so make them available.
+set +u
+if [ "$ARCH_TRIPLET"foobar == "foobar" ]; then
+	# Taking triplet from gcc/cpp
+	ARCH_TRIPLET="$( $CC -v -x c -E - < /dev/null 2>&1 | \
+			grep Target | sed -e 's/Target: //' )"
+fi
+
+if [ "$LIBC"foobar == "foobar" ]; then
+	# trying to grep libc includes from gcc/cpp defaults
+	_TEMP="$( $CC -v -x c -E - < /dev/null 2>&1 | \
+		sed -n -e '/^#include <...> search starts here:$/,/^End of search list.$/{//!p}' | \
+		sed -e 's/^\ \//\//g' | \
+		grep '/usr/include' )"
+
+	# sanity check and prepare LIBC dirs
+	for d in $_TEMP; do
+		if [ ! -d "$d" ]; then
+			echo "$d not a directory"
+			exit 1
+		fi
+		LIBC="$LIBC $d"
+	done
+fi
+set -u
+
+# Copy libc include files to temp directory for the tests.
+
+COMPILE_TEST_INC=../headers_compile_test_include
+rm -rf "$COMPILE_TEST_INC"
+mkdir -p "$COMPILE_TEST_INC"
+
+for d in $LIBC; do
+	# check if last part of dir is the arch triplet, cross compile paths
+	# can have it also elsewhere so just the last one counts.
+	if !( echo "$d" | egrep "$ARCH_TRIPLET$" > /dev/null ); then
+		# hopefully just main libc dir, e.g. /usr/include
+		cp -a "$d"/* "$COMPILE_TEST_INC"/
+	elif ( echo "$d" | egrep "$ARCH_TRIPLET$" > /dev/null ); then
+		# hopefully the arch specific dir, e.g. /usr/include/x86_64-linux-gnu
+		cp -ar "$d"/* "$COMPILE_TEST_INC/"
+	else
+		echo "$d unexpected, bailing out"
+		exit 1
+	fi
+done
+
+# Simulate libc headers without kernel headers by removing
+# all known kernel header dirs from the copied libc ones.
+# This seems to magically work.
+_KERNEL_DIRS="$( find . -type d | grep -v '^\.$' )"
+( cd "$COMPILE_TEST_INC" && rm -rf $_KERNEL_DIRS )
+
+# GCC headers
+set +u
+if [ "$GCC_INC"foobar == "foobar" ]; then
+	# Take from $CC default system include paths, filter out
+	# /usr/local/include and /usr/include stuff first, then try to match
+	# for gcc.
+	_TEMP="$( $CC -v -x c -E - < /dev/null 2>&1 | \
+		sed -n -e '/^#include <...> search starts here:$/,/^End of search list.$/{//!p}' | \
+		sed -e 's/^\ \//\//g' | \
+		egrep -v '/usr/local/include' | \
+		egrep -v '/usr/include' | \
+		grep gcc | \
+		xargs )"
+
+	# merge and prepare for use with $CC
+	for d in $_TEMP; do
+		# sanity test
+		if [ ! -d "$d" ]; then
+			echo "$d: is not a directory"
+			exit 1
+		fi
+		GCC_INC="$GCC_INC -I $d"
+	done
+fi
+set -u
+
+# For each header file, create a .c which includes the header file
+# and try to compile it using only current directory for searching other
+# included header files.
+_FAILED=0
+_PASSED=0
+for f in $( find . -name "*\.h" | xargs ); do
+	_FAIL=0
+	CFILE="$( echo "$( dirname "$f" )"/"$( basename "$f" .h )".c )"
+
+	# create .c file
+	echo "#include <"$( echo "$f" | sed -e 's|^.\/||' )">" \
+		> "$(dirname "$f")"/"$(basename "$f" .h)".c
+
+	# compile test, CC not quoted to support ccache
+	echo $CC -Wall -c -nostdinc $GCC_INC -I . -I "$COMPILE_TEST_INC" -I "$COMPILE_TEST_INC/$ARCH_TRIPLET" "$CFILE"
+	$CC -Wall -c -nostdinc $GCC_INC -I . -I "$COMPILE_TEST_INC" -I "$COMPILE_TEST_INC/$ARCH_TRIPLET" "$CFILE" \
+		|| _FAIL=1
+
+	# report errors
+	if [ "$_FAIL" -gt 0 ]; then
+		echo "FAILED: $f"
+		_FAILED="$(( $_FAILED + 1 ))"
+	else
+		echo "PASSED: $f"
+		_PASSED="$(( $_PASSED + 1))"
+	fi
+done
+
+echo Statistics:
+echo "$_FAILED files failed the compile test."
+echo "$_PASSED files passed the compile test."
+
+exit "$_FAILED"
-- 
2.5.0

