[PATCH 0/9] tools/nolibc: fix build issues on aarch64 after unistd cleanup

[PATCH 0/9] tools/nolibc: fix build issues on aarch64 after unistd cleanup

[Willy Tarreau]

Hi Paul,

as per the recent discussion with Mark, I've updated the nolibc header to
reflect latest upstream which is needed to build on arm64, and I performed
the few cleanups that Mark rightfully suggested.

The following patches were taken from the upstream code and this time I
carefully copied the original commit IDs in hope not to miss such fixes
anymore in the future.

I've build-tested these on x86_64, i586, arm(v5 & v7), arm64, mips and
mipsel, using compilers ranging from gcc 3.4 to gcc 9.3 so I think we're
good for these archs now.

Just let me know if you prefer a pull request, as I can do that as well.

Thanks!
Willy


Willy Tarreau (9):
  tools/nolibc: the definition dup() was missing
  tools/nolibc: make dup2() rely on dup3() when available
  tools/nolibc: make getpgrp() fall back to getpgid(0)
  tools/nolibc: implement fork() based on clone()
  tools/nolibc: implement poll() based on ppoll()
  tools/nolibc: get timeval, timespec and timezone from linux/time.h
  tools/nolibc: remove incorrect definitions of __ARCH_WANT_*
  tools/nolibc: emit a detailed error when missing alternate syscall
    number definitions
  tools/nolibc: fix position of -lgcc in the documented example

 tools/include/nolibc/nolibc.h | 153 +++++++++++++++++++++++++---------
 1 file changed, 115 insertions(+), 38 deletions(-)

-- 
2.28.0

[PATCH 1/9] tools/nolibc: the definition dup() was missing

[Willy Tarreau]

dup() was mistakenly forgotten while sys_dup() was defined.
[This is nolibc's upstream commit 47cc42a79c92]

Signed-off-by: Willy Tarreau <w@1wt.eu>
---
 tools/include/nolibc/nolibc.h | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/tools/include/nolibc/nolibc.h b/tools/include/nolibc/nolibc.h
index e61d36cd4e50..3115c6467d10 100644
--- a/tools/include/nolibc/nolibc.h
+++ b/tools/include/nolibc/nolibc.h
@@ -1852,6 +1852,18 @@ int close(int fd)
 	return ret;
 }
 
+static __attribute__((unused))
+int dup(int fd)
+{
+	int ret = sys_dup(fd);
+
+	if (ret < 0) {
+		SET_ERRNO(-ret);
+		ret = -1;
+	}
+	return ret;
+}
+
 static __attribute__((unused))
 int dup2(int old, int new)
 {
-- 
2.28.0

[PATCH 2/9] tools/nolibc: make dup2() rely on dup3() when available

[Willy Tarreau]

A recent boot failure on 5.4-rc3 on arm64 revealed that sys_dup2()
is not available and that only sys_dup3() is implemented, thus let's
detect this and fall back to sys_dup3() when available.
[This is nolibc's upstream commit fd5272ec2c66]

Signed-off-by: Willy Tarreau <w@1wt.eu>
---
 tools/include/nolibc/nolibc.h | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)

diff --git a/tools/include/nolibc/nolibc.h b/tools/include/nolibc/nolibc.h
index 3115c6467d10..5fda4d844054 100644
--- a/tools/include/nolibc/nolibc.h
+++ b/tools/include/nolibc/nolibc.h
@@ -1502,10 +1502,22 @@ int sys_dup(int fd)
 	return my_syscall1(__NR_dup, fd);
 }
 
+#ifdef __NR_dup3
+static __attribute__((unused))
+int sys_dup3(int old, int new, int flags)
+{
+	return my_syscall3(__NR_dup3, old, new, flags);
+}
+#endif
+
 static __attribute__((unused))
 int sys_dup2(int old, int new)
 {
+#ifdef __NR_dup3
+	return my_syscall3(__NR_dup3, old, new, 0);
+#else
 	return my_syscall2(__NR_dup2, old, new);
+#endif
 }
 
 static __attribute__((unused))
@@ -1876,6 +1888,20 @@ int dup2(int old, int new)
 	return ret;
 }
 
+#ifdef __NR_dup3
+static __attribute__((unused))
+int dup3(int old, int new, int flags)
+{
+	int ret = sys_dup3(old, new, flags);
+
+	if (ret < 0) {
+		SET_ERRNO(-ret);
+		ret = -1;
+	}
+	return ret;
+}
+#endif
+
 static __attribute__((unused))
 int execve(const char *filename, char *const argv[], char *const envp[])
 {
-- 
2.28.0

[PATCH 3/9] tools/nolibc: make getpgrp() fall back to getpgid(0)

[Willy Tarreau]

getpgrp() is not implemented on arm64, getpgid(0) must be used
instead.
[This is nolibc's upstream commit 2379f25073f9]

Signed-off-by: Willy Tarreau <w@1wt.eu>
---
 tools/include/nolibc/nolibc.h | 20 +++++++++++++++++++-
 1 file changed, 19 insertions(+), 1 deletion(-)

diff --git a/tools/include/nolibc/nolibc.h b/tools/include/nolibc/nolibc.h
index 5fda4d844054..9209da89044a 100644
--- a/tools/include/nolibc/nolibc.h
+++ b/tools/include/nolibc/nolibc.h
@@ -1544,10 +1544,16 @@ int sys_getdents64(int fd, struct linux_dirent64 *dirp, int count)
 	return my_syscall3(__NR_getdents64, fd, dirp, count);
 }
 
+static __attribute__((unused))
+pid_t sys_getpgid(pid_t pid)
+{
+	return my_syscall1(__NR_getpgid, pid);
+}
+
 static __attribute__((unused))
 pid_t sys_getpgrp(void)
 {
-	return my_syscall0(__NR_getpgrp);
+	return sys_getpgid(0);
 }
 
 static __attribute__((unused))
@@ -1950,6 +1956,18 @@ int getdents64(int fd, struct linux_dirent64 *dirp, int count)
 	return ret;
 }
 
+static __attribute__((unused))
+pid_t getpgid(pid_t pid)
+{
+	pid_t ret = sys_getpgid(pid);
+
+	if (ret < 0) {
+		SET_ERRNO(-ret);
+		ret = -1;
+	}
+	return ret;
+}
+
 static __attribute__((unused))
 pid_t getpgrp(void)
 {
-- 
2.28.0

[PATCH 4/9] tools/nolibc: implement fork() based on clone()

[Willy Tarreau]

Some archs such as arm64 do not have fork() and have to use clone()
instead so let's make fork() always use clone() when available. This
requires to include signal.h to get the definition of SIGCHLD.
[This is nolibc's upstream commit d2dc42fd6149]

Signed-off-by: Willy Tarreau <w@1wt.eu>
---
 tools/include/nolibc/nolibc.h | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/tools/include/nolibc/nolibc.h b/tools/include/nolibc/nolibc.h
index 9209da89044a..fdd5524e0e54 100644
--- a/tools/include/nolibc/nolibc.h
+++ b/tools/include/nolibc/nolibc.h
@@ -271,6 +271,8 @@ struct stat {
 #define WEXITSTATUS(status)   (((status) & 0xff00) >> 8)
 #define WIFEXITED(status)     (((status) & 0x7f) == 0)
 
+/* for SIGCHLD */
+#include <asm/signal.h>
 
 /* Below comes the architecture-specific code. For each architecture, we have
  * the syscall declarations and the _start code definition. This is the only
@@ -1529,7 +1531,15 @@ int sys_execve(const char *filename, char *const argv[], char *const envp[])
 static __attribute__((unused))
 pid_t sys_fork(void)
 {
+#ifdef __NR_clone
+	/* note: some archs only have clone() and not fork(). Different archs
+	 * have a different API, but most archs have the flags on first arg and
+	 * will not use the rest with no other flag.
+	 */
+	return my_syscall5(__NR_clone, SIGCHLD, 0, 0, 0, 0);
+#else
 	return my_syscall0(__NR_fork);
+#endif
 }
 
 static __attribute__((unused))
-- 
2.28.0

[PATCH 5/9] tools/nolibc: implement poll() based on ppoll()

[Willy Tarreau]

Some architectures like arm64 do not implement poll() and have to
use ppoll() instead.
[This is nolibc's upstream commit 800f75c13ede]

Signed-off-by: Willy Tarreau <w@1wt.eu>
---
 tools/include/nolibc/nolibc.h | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/tools/include/nolibc/nolibc.h b/tools/include/nolibc/nolibc.h
index fdd5524e0e54..833693faf53c 100644
--- a/tools/include/nolibc/nolibc.h
+++ b/tools/include/nolibc/nolibc.h
@@ -1652,7 +1652,17 @@ int sys_pivot_root(const char *new, const char *old)
 static __attribute__((unused))
 int sys_poll(struct pollfd *fds, int nfds, int timeout)
 {
+#if defined(__NR_ppoll)
+	struct timespec t;
+
+	if (timeout >= 0) {
+		t.tv_sec  = timeout / 1000;
+		t.tv_nsec = (timeout % 1000) * 1000000;
+	}
+	return my_syscall4(__NR_ppoll, fds, nfds, (timeout >= 0) ? &t : NULL, NULL);
+#else
 	return my_syscall3(__NR_poll, fds, nfds, timeout);
+#endif
 }
 
 static __attribute__((unused))
-- 
2.28.0

[PATCH 6/9] tools/nolibc: get timeval, timespec and timezone from linux/time.h

[Willy Tarreau]

These ones conflict with linux/time.h when building, so better take
them from there directly.
[This is nolibc's upstream commit dc45f5426b0c]

Signed-off-by: Willy Tarreau <w@1wt.eu>
---
 tools/include/nolibc/nolibc.h | 19 +------------------
 1 file changed, 1 insertion(+), 18 deletions(-)

diff --git a/tools/include/nolibc/nolibc.h b/tools/include/nolibc/nolibc.h
index 833693faf53c..611d9d15899d 100644
--- a/tools/include/nolibc/nolibc.h
+++ b/tools/include/nolibc/nolibc.h
@@ -94,6 +94,7 @@
 #include <asm/errno.h>
 #include <linux/fs.h>
 #include <linux/loop.h>
+#include <linux/time.h>
 
 #define NOLIBC
 
@@ -152,24 +153,6 @@ struct pollfd {
 	short int revents;
 };
 
-/* for select() */
-struct timeval {
-	long    tv_sec;
-	long    tv_usec;
-};
-
-/* for pselect() */
-struct timespec {
-	long    tv_sec;
-	long    tv_nsec;
-};
-
-/* for gettimeofday() */
-struct timezone {
-	int tz_minuteswest;
-	int tz_dsttime;
-};
-
 /* for getdents64() */
 struct linux_dirent64 {
 	uint64_t       d_ino;
-- 
2.28.0

[PATCH 7/9] tools/nolibc: remove incorrect definitions of __ARCH_WANT_*

[Willy Tarreau]

These definitions were added with support for aarch64 when some
syscall definitions were missing (__NR_dup2, __NR_fork,
__NR_getpgrp, etc), but these were actually wrong because these
syscalls do not exist on this platform, and definiting these
resulted in exposing invalid definitions.

The missing syscalls were since implemented based on the newer ones
(__NR_dup3,  __NR_clone, __NR_getpgid) so these incorrect defintions
are definitely not needed anymore.

Thanks to Mark Rutland for spotting this incorrect analysis and
explaining why it was wrong.

[This is nolibc's upstream commit 00b1b0d9b2a4]

Reported-by: Mark Rutland <mark.rutland@arm.com>
Link: https://lore.kernel.org/lkml/20210119153147.GA5083@paulmck-ThinkPad-P72
Cc: "Paul E. McKenney" <paulmck@kernel.org>
Cc: valentin.schneider@arm.com
Signed-off-by: Willy Tarreau <w@1wt.eu>
---
 tools/include/nolibc/nolibc.h | 8 --------
 1 file changed, 8 deletions(-)

diff --git a/tools/include/nolibc/nolibc.h b/tools/include/nolibc/nolibc.h
index 611d9d15899d..475d956ed1d6 100644
--- a/tools/include/nolibc/nolibc.h
+++ b/tools/include/nolibc/nolibc.h
@@ -81,14 +81,6 @@
  *
  */
 
-/* Some archs (at least aarch64) don't expose the regular syscalls anymore by
- * default, either because they have an "_at" replacement, or because there are
- * more modern alternatives. For now we'd rather still use them.
- */
-#define __ARCH_WANT_SYSCALL_NO_AT
-#define __ARCH_WANT_SYSCALL_NO_FLAGS
-#define __ARCH_WANT_SYSCALL_DEPRECATED
-
 #include <asm/unistd.h>
 #include <asm/ioctls.h>
 #include <asm/errno.h>
-- 
2.28.0

[PATCH 8/9] tools/nolibc: emit a detailed error when missing alternate syscall number definitions

[Willy Tarreau]

Some syscalls can be implemented from different __NR_* variants. For
example, sys_dup2() can be implemented based on __NR_dup3 or __NR_dup2.
In this case it's useful to produce an error message when neither are
detected, mentioning the supported alternatives. In case of trouble with
includes, it will help the user search for the right one (e.g __NR_dup3)
instead of just the fallback (__NR_dup2) which might simply not exist on
the platform.
[This is nolibc's upstream commit a21080d2ba41]

Suggested-by: Mark Rutland <mark.rutland@arm.com>
Link: https://lore.kernel.org/lkml/20210120145447.GC77728@C02TD0UTHF1T.local/
Signed-off-by: Willy Tarreau <w@1wt.eu>
---
 tools/include/nolibc/nolibc.h | 52 ++++++++++++++++++++++++++---------
 1 file changed, 39 insertions(+), 13 deletions(-)

diff --git a/tools/include/nolibc/nolibc.h b/tools/include/nolibc/nolibc.h
index 475d956ed1d6..618acad6c932 100644
--- a/tools/include/nolibc/nolibc.h
+++ b/tools/include/nolibc/nolibc.h
@@ -1446,8 +1446,10 @@ int sys_chmod(const char *path, mode_t mode)
 {
 #ifdef __NR_fchmodat
 	return my_syscall4(__NR_fchmodat, AT_FDCWD, path, mode, 0);
-#else
+#elif defined(__NR_chmod)
 	return my_syscall2(__NR_chmod, path, mode);
+#else
+#error Neither __NR_fchmodat nor __NR_chmod defined, cannot implement sys_chmod()
 #endif
 }
 
@@ -1456,8 +1458,10 @@ int sys_chown(const char *path, uid_t owner, gid_t group)
 {
 #ifdef __NR_fchownat
 	return my_syscall5(__NR_fchownat, AT_FDCWD, path, owner, group, 0);
-#else
+#elif defined(__NR_chown)
 	return my_syscall3(__NR_chown, path, owner, group);
+#else
+#error Neither __NR_fchownat nor __NR_chown defined, cannot implement sys_chown()
 #endif
 }
 
@@ -1492,8 +1496,10 @@ int sys_dup2(int old, int new)
 {
 #ifdef __NR_dup3
 	return my_syscall3(__NR_dup3, old, new, 0);
-#else
+#elif defined(__NR_dup2)
 	return my_syscall2(__NR_dup2, old, new);
+#else
+#error Neither __NR_dup3 nor __NR_dup2 defined, cannot implement sys_dup2()
 #endif
 }
 
@@ -1512,8 +1518,10 @@ pid_t sys_fork(void)
 	 * will not use the rest with no other flag.
 	 */
 	return my_syscall5(__NR_clone, SIGCHLD, 0, 0, 0, 0);
-#else
+#elif defined(__NR_fork)
 	return my_syscall0(__NR_fork);
+#else
+#error Neither __NR_clone nor __NR_fork defined, cannot implement sys_fork()
 #endif
 }
 
@@ -1570,8 +1578,10 @@ int sys_link(const char *old, const char *new)
 {
 #ifdef __NR_linkat
 	return my_syscall5(__NR_linkat, AT_FDCWD, old, AT_FDCWD, new, 0);
-#else
+#elif defined(__NR_link)
 	return my_syscall2(__NR_link, old, new);
+#else
+#error Neither __NR_linkat nor __NR_link defined, cannot implement sys_link()
 #endif
 }
 
@@ -1586,8 +1596,10 @@ int sys_mkdir(const char *path, mode_t mode)
 {
 #ifdef __NR_mkdirat
 	return my_syscall3(__NR_mkdirat, AT_FDCWD, path, mode);
-#else
+#elif defined(__NR_mkdir)
 	return my_syscall2(__NR_mkdir, path, mode);
+#else
+#error Neither __NR_mkdirat nor __NR_mkdir defined, cannot implement sys_mkdir()
 #endif
 }
 
@@ -1596,8 +1608,10 @@ long sys_mknod(const char *path, mode_t mode, dev_t dev)
 {
 #ifdef __NR_mknodat
 	return my_syscall4(__NR_mknodat, AT_FDCWD, path, mode, dev);
-#else
+#elif defined(__NR_mknod)
 	return my_syscall3(__NR_mknod, path, mode, dev);
+#else
+#error Neither __NR_mknodat nor __NR_mknod defined, cannot implement sys_mknod()
 #endif
 }
 
@@ -1613,8 +1627,10 @@ int sys_open(const char *path, int flags, mode_t mode)
 {
 #ifdef __NR_openat
 	return my_syscall4(__NR_openat, AT_FDCWD, path, flags, mode);
-#else
+#elif defined(__NR_open)
 	return my_syscall3(__NR_open, path, flags, mode);
+#else
+#error Neither __NR_openat nor __NR_open defined, cannot implement sys_open()
 #endif
 }
 
@@ -1635,8 +1651,10 @@ int sys_poll(struct pollfd *fds, int nfds, int timeout)
 		t.tv_nsec = (timeout % 1000) * 1000000;
 	}
 	return my_syscall4(__NR_ppoll, fds, nfds, (timeout >= 0) ? &t : NULL, NULL);
-#else
+#elif defined(__NR_poll)
 	return my_syscall3(__NR_poll, fds, nfds, timeout);
+#else
+#error Neither __NR_ppoll nor __NR_poll defined, cannot implement sys_poll()
 #endif
 }
 
@@ -1676,11 +1694,13 @@ int sys_select(int nfds, fd_set *rfds, fd_set *wfds, fd_set *efds, struct timeva
 		t.tv_nsec = timeout->tv_usec * 1000;
 	}
 	return my_syscall6(__NR_pselect6, nfds, rfds, wfds, efds, timeout ? &t : NULL, NULL);
-#else
+#elif defined(__NR__newselect) || defined(__NR_select)
 #ifndef __NR__newselect
 #define __NR__newselect __NR_select
 #endif
 	return my_syscall5(__NR__newselect, nfds, rfds, wfds, efds, timeout);
+#else
+#error None of __NR_select, __NR_pselect6, nor __NR__newselect defined, cannot implement sys_select()
 #endif
 }
 
@@ -1705,8 +1725,10 @@ int sys_stat(const char *path, struct stat *buf)
 #ifdef __NR_newfstatat
 	/* only solution for arm64 */
 	ret = my_syscall4(__NR_newfstatat, AT_FDCWD, path, &stat, 0);
-#else
+#elif defined(__NR_stat)
 	ret = my_syscall2(__NR_stat, path, &stat);
+#else
+#error Neither __NR_newfstatat nor __NR_stat defined, cannot implement sys_stat()
 #endif
 	buf->st_dev     = stat.st_dev;
 	buf->st_ino     = stat.st_ino;
@@ -1730,8 +1752,10 @@ int sys_symlink(const char *old, const char *new)
 {
 #ifdef __NR_symlinkat
 	return my_syscall3(__NR_symlinkat, old, AT_FDCWD, new);
-#else
+#elif defined(__NR_symlink)
 	return my_syscall2(__NR_symlink, old, new);
+#else
+#error Neither __NR_symlinkat nor __NR_symlink defined, cannot implement sys_symlink()
 #endif
 }
 
@@ -1752,8 +1776,10 @@ int sys_unlink(const char *path)
 {
 #ifdef __NR_unlinkat
 	return my_syscall3(__NR_unlinkat, AT_FDCWD, path, 0);
-#else
+#elif defined(__NR_unlink)
 	return my_syscall1(__NR_unlink, path);
+#else
+#error Neither __NR_unlinkat nor __NR_unlink defined, cannot implement sys_unlink()
 #endif
 }
 
-- 
2.28.0

[PATCH 9/9] tools/nolibc: fix position of -lgcc in the documented example

[Willy Tarreau]

The documentation header in the file provides a command line example
to explain how to build, but it places -lgcc before the source,
which usually fails with libgcc.a (e.g. on ARM when uidiv is
needed). Let's fix this before it leaks into makefiles.
[This is nolibc's upstream commit b5e282089223]

Signed-off-by: Willy Tarreau <w@1wt.eu>
---
 tools/include/nolibc/nolibc.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/include/nolibc/nolibc.h b/tools/include/nolibc/nolibc.h
index 618acad6c932..8b7a9830dd22 100644
--- a/tools/include/nolibc/nolibc.h
+++ b/tools/include/nolibc/nolibc.h
@@ -71,7 +71,7 @@
  *
  * A simple static executable may be built this way :
  *      $ gcc -fno-asynchronous-unwind-tables -fno-ident -s -Os -nostdlib \
- *            -static -include nolibc.h -lgcc -o hello hello.c
+ *            -static -include nolibc.h -o hello hello.c -lgcc
  *
  * A very useful calling convention table may be found here :
  *      http://man7.org/linux/man-pages/man2/syscall.2.html
-- 
2.28.0

[PATCH 10/9] tools/rcutorture: fix position of -lgcc in mkinitrd.sh

[Willy Tarreau]

I placed -lgcc poorly in the build options, resulting in possible build
failures that are typically encountered on ARM when uidiv is required;
-lgcc must be placed after the source files.

Signed-off-by: Willy Tarreau <w@1wt.eu>
---

Sorry for this last one, I figured after my last documentation fix that
the incorrect command line did indeed slip into one command line that is
in mkinitrd.sh. It would break ARM builds if a divide is required by the
init code.

 tools/testing/selftests/rcutorture/bin/mkinitrd.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/testing/selftests/rcutorture/bin/mkinitrd.sh b/tools/testing/selftests/rcutorture/bin/mkinitrd.sh
index 38e424d..70d62fd 100755
--- a/tools/testing/selftests/rcutorture/bin/mkinitrd.sh
+++ b/tools/testing/selftests/rcutorture/bin/mkinitrd.sh
@@ -70,7 +70,7 @@ if echo -e "#if __x86_64__||__i386__||__i486__||__i586__||__i686__" \
 	# architecture supported by nolibc
         ${CROSS_COMPILE}gcc -fno-asynchronous-unwind-tables -fno-ident \
 		-nostdlib -include ../../../../include/nolibc/nolibc.h \
-		-lgcc -s -static -Os -o init init.c
+		-s -static -Os -o init init.c -lgcc
 else
 	${CROSS_COMPILE}gcc -s -static -Os -o init init.c
 fi
-- 
2.9.0

Re: [PATCH 0/9] tools/nolibc: fix build issues on aarch64 after unistd cleanup

[Valentin Schneider]

On 21/01/21 08:20, Willy Tarreau wrote:
> Hi Paul,
>
> as per the recent discussion with Mark, I've updated the nolibc header to
> reflect latest upstream which is needed to build on arm64, and I performed
> the few cleanups that Mark rightfully suggested.
>
> The following patches were taken from the upstream code and this time I
> carefully copied the original commit IDs in hope not to miss such fixes
> anymore in the future.
>
> I've build-tested these on x86_64, i586, arm(v5 & v7), arm64, mips and
> mipsel, using compilers ranging from gcc 3.4 to gcc 9.3 so I think we're
> good for these archs now.
>
> Just let me know if you prefer a pull request, as I can do that as well.
>

This lets me run the following invocation without a hitch:

tools/testing/selftests/rcutorture/bin/kvm.sh --allcpus --duration 10 --configs "4*SRCU-P" --trust-make

where before I would get some errors building the initrd due to missing
__NR_foo.

Tested-by: Valentin Schneider <valentin.schneider@arm.com>

Thanks for having a look at this! And kuddos to Mark for nudging the
right people.

> Thanks!
> Willy
>
>
> Willy Tarreau (9):
>   tools/nolibc: the definition dup() was missing
>   tools/nolibc: make dup2() rely on dup3() when available
>   tools/nolibc: make getpgrp() fall back to getpgid(0)
>   tools/nolibc: implement fork() based on clone()
>   tools/nolibc: implement poll() based on ppoll()
>   tools/nolibc: get timeval, timespec and timezone from linux/time.h
>   tools/nolibc: remove incorrect definitions of __ARCH_WANT_*
>   tools/nolibc: emit a detailed error when missing alternate syscall
>     number definitions
>   tools/nolibc: fix position of -lgcc in the documented example
>
>  tools/include/nolibc/nolibc.h | 153 +++++++++++++++++++++++++---------
>  1 file changed, 115 insertions(+), 38 deletions(-)
>
> --
> 2.28.0

Re: [PATCH 0/9] tools/nolibc: fix build issues on aarch64 after unistd cleanup

[Willy Tarreau]

On Thu, Jan 21, 2021 at 11:05:48AM +0000, Valentin Schneider wrote:
> This lets me run the following invocation without a hitch:
> 
> tools/testing/selftests/rcutorture/bin/kvm.sh --allcpus --duration 10 --configs "4*SRCU-P" --trust-make
> 
> where before I would get some errors building the initrd due to missing
> __NR_foo.
> 
> Tested-by: Valentin Schneider <valentin.schneider@arm.com>

Thank you Valentin! Much appreciated!

Willy

Re: [PATCH 0/9] tools/nolibc: fix build issues on aarch64 after unistd cleanup

[Mark Rutland]

Hi Willy, Paul,

On Thu, Jan 21, 2021 at 08:20:22AM +0100, Willy Tarreau wrote:
> Hi Paul,
> 
> as per the recent discussion with Mark, I've updated the nolibc header to
> reflect latest upstream which is needed to build on arm64, and I performed
> the few cleanups that Mark rightfully suggested.
> 
> The following patches were taken from the upstream code and this time I
> carefully copied the original commit IDs in hope not to miss such fixes
> anymore in the future.
> 
> I've build-tested these on x86_64, i586, arm(v5 & v7), arm64, mips and
> mipsel, using compilers ranging from gcc 3.4 to gcc 9.3 so I think we're
> good for these archs now.

I've given this a spin atop v5.11-rc4, building natively on arm64 on a
Debian 10.7 system, and with the whole series applied I'm able to run
the rcutorture kvm.sh script without issue (the CONFIG warnings are
unrelated to nolibc):

| [mark@gravadlaks:~/src/linux]% ./tools/testing/selftests/rcutorture/bin/kvm.sh --cpus 255 --configs "TREE03"  --kmake-arg "CROSS_COMPILE=aarch64-linux-gnu- ARCH=arm64" --duration 1
| Creating a statically linked C-language initrd
| Done creating a statically linked C-language initrd
| Results directory: /home/mark/src/linux/tools/testing/selftests/rcutorture/res/2021.01.21-10.53.24
| ./tools/testing/selftests/rcutorture/bin/kvm.sh --cpus 255 --configs TREE03 --kmake-arg CROSS_COMPILE=aarch64-linux-gnu- ARCH=arm64 --duration 1
| ----Start batch 1: Thu 21 Jan 10:53:24 GMT 2021
| TREE03 16: Starting build. Thu 21 Jan 10:53:24 GMT 2021
| TREE03 16: Waiting for build to complete. Thu 21 Jan 10:53:24 GMT 2021
| TREE03 16: Build complete. Thu 21 Jan 10:56:25 GMT 2021
| ---- TREE03 16: Kernel present. Thu 21 Jan 10:56:25 GMT 2021
| ---- Starting kernels. Thu 21 Jan 10:56:25 GMT 2021
| ---- All kernel runs complete. Thu 21 Jan 10:57:35 GMT 2021
| ---- TREE03 16: Build/run results:
|  --- Thu 21 Jan 10:53:24 GMT 2021: Starting build
| :CONFIG_HYPERVISOR_GUEST=y: improperly set
| :CONFIG_KVM_GUEST=y: improperly set
|  --- Thu 21 Jan 10:56:25 GMT 2021: Starting kernel
| QEMU 3.1.0 monitor - type 'help' for more information
| (qemu) Monitoring qemu job at pid 258620
| Grace period for qemu job at pid 258620
| 
| 
|  --- Thu 21 Jan 10:53:24 GMT 2021 Test summary:
| Results directory: /home/mark/src/linux/tools/testing/selftests/rcutorture/res/2021.01.21-10.53.24
| ./tools/testing/selftests/rcutorture/bin/kvm.sh --cpus 255 --configs TREE03 --kmake-arg CROSS_COMPILE=aarch64-linux-gnu- ARCH=arm64 --duration 1
| TREE03 ------- 475 GPs (7.91667/s) [rcu: g5737 f0x0 total-gps=1713]
| :CONFIG_HYPERVISOR_GUEST=y: improperly set
| :CONFIG_KVM_GUEST=y: improperly set

So FWIW:

Tested-by: Mark Rutland <mark.rutland@arm.com> [arm64]

It would be great if this could be applied soon so that it's possible to
use the rcutorture scripts without applying local hacks.

Willy, thanks for sorting this out, especially so quickly!

Thanks,
Mark.

Re: [PATCH 0/9] tools/nolibc: fix build issues on aarch64 after unistd cleanup

[Mark Rutland]

On Thu, Jan 21, 2021 at 11:05:48AM +0000, Valentin Schneider wrote:
> Thanks for having a look at this! And kuddos to Mark for nudging the
> right people.

Likewise to Paul for poking Willy!

Thanks all,
Mark.

Re: [PATCH 0/9] tools/nolibc: fix build issues on aarch64 after unistd cleanup

[Willy Tarreau]

On Thu, Jan 21, 2021 at 11:11:17AM +0000, Mark Rutland wrote:
> So FWIW:
> 
> Tested-by: Mark Rutland <mark.rutland@arm.com> [arm64]

Perfect, thanks! Paul, may I let you copy-paste the tested-by yourself ?
If you prefer I'm fine with resending a series to you, I just don't want
to needlessly spam you :-)

> It would be great if this could be applied soon so that it's possible to
> use the rcutorture scripts without applying local hacks.

Makes sense. I was wondering, should we mark them for stable ? I don't
know if anyone relies on these tests to validate stable kernels in
fact.

> Willy, thanks for sorting this out, especially so quickly!

You're welcome, and thanks to you for the detailed report and explanations.

Willy

Re: [PATCH 0/9] tools/nolibc: fix build issues on aarch64 after unistd cleanup

[Paul E. McKenney]

On Thu, Jan 21, 2021 at 03:18:09PM +0100, Willy Tarreau wrote:
> On Thu, Jan 21, 2021 at 11:11:17AM +0000, Mark Rutland wrote:
> > So FWIW:
> > 
> > Tested-by: Mark Rutland <mark.rutland@arm.com> [arm64]

Thank you all!

> Perfect, thanks! Paul, may I let you copy-paste the tested-by yourself ?
> If you prefer I'm fine with resending a series to you, I just don't want
> to needlessly spam you :-)

Done, with Valentin's and Mark's Tested-by.

> > It would be great if this could be applied soon so that it's possible to
> > use the rcutorture scripts without applying local hacks.
> 
> Makes sense. I was wondering, should we mark them for stable ? I don't
> know if anyone relies on these tests to validate stable kernels in
> fact.

I added Fixes tags that should make this happen, and they are now visible
at -rcu branch "dev".  Could you please check them for me?

c261145 tools/nolibc: Add the definition for dup()
79f220e tools/nolibc: Make dup2() rely on dup3() when available
c0c7c10 tools/nolibc: Make getpgrp() fall back to getpgid(0)
be60ca4 tools/nolibc: Implement fork() based on clone()
5b1c827 tools/nolibc: Implement poll() based on ppoll()
70ca7ae tools/nolibc: Get timeval, timespec and timezone from linux/time.h
f65d711 tools/nolibc: Remove incorrect definitions of __ARCH_WANT_*
35635d7 tools/nolibc: Emit detailed error for missing alternate syscall number definitions
3c6ce7a tools/nolibc: Fix position of -lgcc in the documented example
26cec81 tools/rcutorture: Fix position of -lgcc in mkinitrd.sh

> > Willy, thanks for sorting this out, especially so quickly!
> 
> You're welcome, and thanks to you for the detailed report and explanations.

Again, thank you all!

On getting this upstream quickly, if all goes well I expect to include
this in my pull request for the upcoming merge window.

							Thanx, Paul

Re: [PATCH 0/9] tools/nolibc: fix build issues on aarch64 after unistd cleanup

[Willy Tarreau]

On Thu, Jan 21, 2021 at 11:54:32AM -0800, Paul E. McKenney wrote:
> > > It would be great if this could be applied soon so that it's possible to
> > > use the rcutorture scripts without applying local hacks.
> > 
> > Makes sense. I was wondering, should we mark them for stable ? I don't
> > know if anyone relies on these tests to validate stable kernels in
> > fact.
> 
> I added Fixes tags that should make this happen, and they are now visible
> at -rcu branch "dev".  Could you please check them for me?

I've just rerun all previous tests from my history and all of them are
OK. Please note however that I only did the manual build test, not through
the whole kvm.sh script, but a diff shows that the involved files are byte
for byte identical to those that Valentin and Mark tested, so for me that's
OK as well.

By the way, thank you for having completed the commit messages, I hope you
didn't spend too much time on this.

Cheers,
Willy

Re: [PATCH 0/9] tools/nolibc: fix build issues on aarch64 after unistd cleanup

[Paul E. McKenney]

On Thu, Jan 21, 2021 at 09:17:14PM +0100, Willy Tarreau wrote:
> On Thu, Jan 21, 2021 at 11:54:32AM -0800, Paul E. McKenney wrote:
> > > > It would be great if this could be applied soon so that it's possible to
> > > > use the rcutorture scripts without applying local hacks.
> > > 
> > > Makes sense. I was wondering, should we mark them for stable ? I don't
> > > know if anyone relies on these tests to validate stable kernels in
> > > fact.
> > 
> > I added Fixes tags that should make this happen, and they are now visible
> > at -rcu branch "dev".  Could you please check them for me?
> 
> I've just rerun all previous tests from my history and all of them are
> OK. Please note however that I only did the manual build test, not through
> the whole kvm.sh script, but a diff shows that the involved files are byte
> for byte identical to those that Valentin and Mark tested, so for me that's
> OK as well.

Byte-for-byte identical works for me, thank you!

> By the way, thank you for having completed the commit messages, I hope you
> didn't spend too much time on this.

Not a problem!  I definitely had the easy end of this job.  ;-)

							Thanx, Paul

Re: [PATCH 0/9] tools/nolibc: fix build issues on aarch64 after unistd cleanup

[Paul E. McKenney]

On Thu, Jan 21, 2021 at 11:11:17AM +0000, Mark Rutland wrote:

[ . . . ]

> I've given this a spin atop v5.11-rc4, building natively on arm64 on a
> Debian 10.7 system, and with the whole series applied I'm able to run
> the rcutorture kvm.sh script without issue (the CONFIG warnings are
> unrelated to nolibc):
> 
> | [mark@gravadlaks:~/src/linux]% ./tools/testing/selftests/rcutorture/bin/kvm.sh --cpus 255 --configs "TREE03"  --kmake-arg "CROSS_COMPILE=aarch64-linux-gnu- ARCH=arm64" --duration 1
> | Creating a statically linked C-language initrd
> | Done creating a statically linked C-language initrd
> | Results directory: /home/mark/src/linux/tools/testing/selftests/rcutorture/res/2021.01.21-10.53.24
> | ./tools/testing/selftests/rcutorture/bin/kvm.sh --cpus 255 --configs TREE03 --kmake-arg CROSS_COMPILE=aarch64-linux-gnu- ARCH=arm64 --duration 1
> | ----Start batch 1: Thu 21 Jan 10:53:24 GMT 2021
> | TREE03 16: Starting build. Thu 21 Jan 10:53:24 GMT 2021
> | TREE03 16: Waiting for build to complete. Thu 21 Jan 10:53:24 GMT 2021
> | TREE03 16: Build complete. Thu 21 Jan 10:56:25 GMT 2021
> | ---- TREE03 16: Kernel present. Thu 21 Jan 10:56:25 GMT 2021
> | ---- Starting kernels. Thu 21 Jan 10:56:25 GMT 2021
> | ---- All kernel runs complete. Thu 21 Jan 10:57:35 GMT 2021
> | ---- TREE03 16: Build/run results:
> |  --- Thu 21 Jan 10:53:24 GMT 2021: Starting build
> | :CONFIG_HYPERVISOR_GUEST=y: improperly set
> | :CONFIG_KVM_GUEST=y: improperly set

These two (apparently harmless) error messages are due to these lines
in CFcommon:

CONFIG_HYPERVISOR_GUEST=y
CONFIG_KVM_GUEST=y

It looks like CONFIG_HYPERVISOR_GUEST is specific to x86, while KVM_GUEST
is specific to x86, powerpc, and mips.  (It appears that arm64 doesn't
need anything here?)  Given this variety, I need to make rcutorture
know very early on what arch it is building for.  My current approach of
looking at the vmlinux file won't work because I need to get the config
right before building the kernel.

One approach would be to look at the initrd/init file, but doing this
reliably would mean removing the ability for users to supply their own
initrd file trees.  Another approach would be to look at the current
environment, perhaps using "uname -m", which will work until someone
wants to cross-build.  Yet another approach would be to parse the target
line from the output of "${CROSS_COMPILE}gcc -v".

Left to myself, I would parse the output of "${CROSS_COMPILE}gcc -v".

Thoughts?

							Thanx, Paul

> |  --- Thu 21 Jan 10:56:25 GMT 2021: Starting kernel
> | QEMU 3.1.0 monitor - type 'help' for more information
> | (qemu) Monitoring qemu job at pid 258620
> | Grace period for qemu job at pid 258620
> | 
> | 
> |  --- Thu 21 Jan 10:53:24 GMT 2021 Test summary:
> | Results directory: /home/mark/src/linux/tools/testing/selftests/rcutorture/res/2021.01.21-10.53.24
> | ./tools/testing/selftests/rcutorture/bin/kvm.sh --cpus 255 --configs TREE03 --kmake-arg CROSS_COMPILE=aarch64-linux-gnu- ARCH=arm64 --duration 1
> | TREE03 ------- 475 GPs (7.91667/s) [rcu: g5737 f0x0 total-gps=1713]
> | :CONFIG_HYPERVISOR_GUEST=y: improperly set
> | :CONFIG_KVM_GUEST=y: improperly set
> 
> So FWIW:
> 
> Tested-by: Mark Rutland <mark.rutland@arm.com> [arm64]
> 
> It would be great if this could be applied soon so that it's possible to
> use the rcutorture scripts without applying local hacks.
> 
> Willy, thanks for sorting this out, especially so quickly!
> 
> Thanks,
> Mark.

Re: [PATCH 0/9] tools/nolibc: fix build issues on aarch64 after unistd cleanup

[Mark Rutland]

On Fri, Jan 22, 2021 at 04:03:26AM -0800, Paul E. McKenney wrote:
> On Thu, Jan 21, 2021 at 11:11:17AM +0000, Mark Rutland wrote:
> 
> [ . . . ]
> 
> > I've given this a spin atop v5.11-rc4, building natively on arm64 on a
> > Debian 10.7 system, and with the whole series applied I'm able to run
> > the rcutorture kvm.sh script without issue (the CONFIG warnings are
> > unrelated to nolibc):
> > 
> > | [mark@gravadlaks:~/src/linux]% ./tools/testing/selftests/rcutorture/bin/kvm.sh --cpus 255 --configs "TREE03"  --kmake-arg "CROSS_COMPILE=aarch64-linux-gnu- ARCH=arm64" --duration 1
> > | Creating a statically linked C-language initrd
> > | Done creating a statically linked C-language initrd
> > | Results directory: /home/mark/src/linux/tools/testing/selftests/rcutorture/res/2021.01.21-10.53.24
> > | ./tools/testing/selftests/rcutorture/bin/kvm.sh --cpus 255 --configs TREE03 --kmake-arg CROSS_COMPILE=aarch64-linux-gnu- ARCH=arm64 --duration 1
> > | ----Start batch 1: Thu 21 Jan 10:53:24 GMT 2021
> > | TREE03 16: Starting build. Thu 21 Jan 10:53:24 GMT 2021
> > | TREE03 16: Waiting for build to complete. Thu 21 Jan 10:53:24 GMT 2021
> > | TREE03 16: Build complete. Thu 21 Jan 10:56:25 GMT 2021
> > | ---- TREE03 16: Kernel present. Thu 21 Jan 10:56:25 GMT 2021
> > | ---- Starting kernels. Thu 21 Jan 10:56:25 GMT 2021
> > | ---- All kernel runs complete. Thu 21 Jan 10:57:35 GMT 2021
> > | ---- TREE03 16: Build/run results:
> > |  --- Thu 21 Jan 10:53:24 GMT 2021: Starting build
> > | :CONFIG_HYPERVISOR_GUEST=y: improperly set
> > | :CONFIG_KVM_GUEST=y: improperly set
> 
> These two (apparently harmless) error messages are due to these lines
> in CFcommon:
> 
> CONFIG_HYPERVISOR_GUEST=y
> CONFIG_KVM_GUEST=y

Yup; I had figured this out, but since this wasn't getting in the way of
actually running the torture tests I had assumed we could deal with that
at some indefinite point in the future, or simply ignore it. ;)

> It looks like CONFIG_HYPERVISOR_GUEST is specific to x86, while KVM_GUEST
> is specific to x86, powerpc, and mips.  (It appears that arm64 doesn't
> need anything here?) 

Yup, we don't need any special options -- arm64 doesn't stricly need any
guest enlightenment to run under a hypervisor, so we never had a need
for KVM_GUEST or HYPERVISOR_GUEST. We have all the common
paravritualized drivers (e.g. virtio) in defconfig too, so that should
all work out of the box.

> Given this variety, I need to make rcutorture know very early on what
> arch it is building for.  My current approach of looking at the
> vmlinux file won't work because I need to get the config right before
> building the kernel.
> 
> One approach would be to look at the initrd/init file, but doing this
> reliably would mean removing the ability for users to supply their own
> initrd file trees.  Another approach would be to look at the current
> environment, perhaps using "uname -m", which will work until someone
> wants to cross-build.  Yet another approach would be to parse the target
> line from the output of "${CROSS_COMPILE}gcc -v".
> 
> Left to myself, I would parse the output of "${CROSS_COMPILE}gcc -v".

Heh, I hadn't considered parsing the target line from that output, and I
guess that works for native builds where "${CROSS_COMPILE}" = "". Neat
trick!

That sounds sensible to me!

Thanks,
Mark.

Re: [PATCH 0/9] tools/nolibc: fix build issues on aarch64 after unistd cleanup

[Paul E. McKenney]

On Fri, Jan 22, 2021 at 12:25:42PM +0000, Mark Rutland wrote:
> On Fri, Jan 22, 2021 at 04:03:26AM -0800, Paul E. McKenney wrote:
> > On Thu, Jan 21, 2021 at 11:11:17AM +0000, Mark Rutland wrote:
> > 
> > [ . . . ]
> > 
> > > I've given this a spin atop v5.11-rc4, building natively on arm64 on a
> > > Debian 10.7 system, and with the whole series applied I'm able to run
> > > the rcutorture kvm.sh script without issue (the CONFIG warnings are
> > > unrelated to nolibc):
> > > 
> > > | [mark@gravadlaks:~/src/linux]% ./tools/testing/selftests/rcutorture/bin/kvm.sh --cpus 255 --configs "TREE03"  --kmake-arg "CROSS_COMPILE=aarch64-linux-gnu- ARCH=arm64" --duration 1
> > > | Creating a statically linked C-language initrd
> > > | Done creating a statically linked C-language initrd
> > > | Results directory: /home/mark/src/linux/tools/testing/selftests/rcutorture/res/2021.01.21-10.53.24
> > > | ./tools/testing/selftests/rcutorture/bin/kvm.sh --cpus 255 --configs TREE03 --kmake-arg CROSS_COMPILE=aarch64-linux-gnu- ARCH=arm64 --duration 1
> > > | ----Start batch 1: Thu 21 Jan 10:53:24 GMT 2021
> > > | TREE03 16: Starting build. Thu 21 Jan 10:53:24 GMT 2021
> > > | TREE03 16: Waiting for build to complete. Thu 21 Jan 10:53:24 GMT 2021
> > > | TREE03 16: Build complete. Thu 21 Jan 10:56:25 GMT 2021
> > > | ---- TREE03 16: Kernel present. Thu 21 Jan 10:56:25 GMT 2021
> > > | ---- Starting kernels. Thu 21 Jan 10:56:25 GMT 2021
> > > | ---- All kernel runs complete. Thu 21 Jan 10:57:35 GMT 2021
> > > | ---- TREE03 16: Build/run results:
> > > |  --- Thu 21 Jan 10:53:24 GMT 2021: Starting build
> > > | :CONFIG_HYPERVISOR_GUEST=y: improperly set
> > > | :CONFIG_KVM_GUEST=y: improperly set
> > 
> > These two (apparently harmless) error messages are due to these lines
> > in CFcommon:
> > 
> > CONFIG_HYPERVISOR_GUEST=y
> > CONFIG_KVM_GUEST=y
> 
> Yup; I had figured this out, but since this wasn't getting in the way of
> actually running the torture tests I had assumed we could deal with that
> at some indefinite point in the future, or simply ignore it. ;)
> 
> > It looks like CONFIG_HYPERVISOR_GUEST is specific to x86, while KVM_GUEST
> > is specific to x86, powerpc, and mips.  (It appears that arm64 doesn't
> > need anything here?) 
> 
> Yup, we don't need any special options -- arm64 doesn't stricly need any
> guest enlightenment to run under a hypervisor, so we never had a need
> for KVM_GUEST or HYPERVISOR_GUEST. We have all the common
> paravritualized drivers (e.g. virtio) in defconfig too, so that should
> all work out of the box.
> 
> > Given this variety, I need to make rcutorture know very early on what
> > arch it is building for.  My current approach of looking at the
> > vmlinux file won't work because I need to get the config right before
> > building the kernel.
> > 
> > One approach would be to look at the initrd/init file, but doing this
> > reliably would mean removing the ability for users to supply their own
> > initrd file trees.  Another approach would be to look at the current
> > environment, perhaps using "uname -m", which will work until someone
> > wants to cross-build.  Yet another approach would be to parse the target
> > line from the output of "${CROSS_COMPILE}gcc -v".
> > 
> > Left to myself, I would parse the output of "${CROSS_COMPILE}gcc -v".
> 
> Heh, I hadn't considered parsing the target line from that output, and I
> guess that works for native builds where "${CROSS_COMPILE}" = "". Neat
> trick!

Credit to Willy Tarreau.  Me, I just realized that he needed to do
-something- to create rcutorture's initrd.  ;-)

> That sounds sensible to me!

Let me see what I can do.

My thought is to add optional CFcommon.<arch> files, and pull them in
when there is a match.  But I will give it more thought.

							Thanx, Paul