[PATCH 0/8] nolibc: small updates required for the self-tests

[PATCH 0/8] nolibc: small updates required for the self-tests

[Willy Tarreau]

Hello Paul,

while developing the nolibc self-tests, I naturally faced a few
limitations ranging from missing POLL* flags to missing strcmp() or
getenv(), as well as the ability for printf() to print "(null)" on
"%s" instead of crashing.

This series adds a makefile and a headers installation target that
creates a sysroot made of kernel headers and nolibc headers, which
significantly ease building programs.

I already have some test code with roughly 75 tests, but I find it
misplaced in the nolibc dir, I need to move it to testing/selftests/
before sending it to you.

This series is intended to be an add-on to what you already have in your
dev branch. I don't intend to send you much more stuff on top of this,
but I expect to see an update from Ammar's patch set (CCed). With all
this I think we'll have a good basis to easily add new tests.

Thanks!
Willy

---
Willy Tarreau (8):
  tools/nolibc/stdio: make printf(%s) accept NULL
  tools/nolibc/stdlib: add a simple getenv() implementation
  tools/nolibc/stdio: add support for '%p' to vfprintf()
  tools/nolibc/string: add strcmp() and strncmp()
  tools/nolibc/sys: add syscall definition for getppid()
  tools/nolibc/types: add poll() and waitpid() flag definitions
  tools/nolibc: add a makefile to install headers
  tools/nolibc: add the nolibc subdir to the common Makefile

 tools/Makefile                |  4 ++++
 tools/include/nolibc/Makefile | 43 +++++++++++++++++++++++++++++++++++
 tools/include/nolibc/stdio.h  | 32 +++++++++++++++++---------
 tools/include/nolibc/stdlib.h | 23 +++++++++++++++++++
 tools/include/nolibc/string.h | 23 +++++++++++++++++++
 tools/include/nolibc/sys.h    | 17 ++++++++++++++
 tools/include/nolibc/types.h  | 10 ++++++++
 7 files changed, 141 insertions(+), 11 deletions(-)
 create mode 100644 tools/include/nolibc/Makefile

-- 
2.35.1

[PATCH 1/8] tools/nolibc/stdio: make printf(%s) accept NULL

[Willy Tarreau]

It's often convenient to support this, especially in test programs where
a NULL may correspond to an allocation error or a non-existing value.
Let's make printf("%s") support being passed a NULL. In this case it
prints "(null)" like glibc's printf().

Signed-off-by: Willy Tarreau <w@1wt.eu>
---
 tools/include/nolibc/stdio.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/tools/include/nolibc/stdio.h b/tools/include/nolibc/stdio.h
index cb4d3ab3a565..559ebe052a75 100644
--- a/tools/include/nolibc/stdio.h
+++ b/tools/include/nolibc/stdio.h
@@ -220,6 +220,8 @@ int vfprintf(FILE *stream, const char *fmt, va_list args)
 			}
 			else if (c == 's') {
 				outstr = va_arg(args, char *);
+				if (!outstr)
+					outstr="(null)";
 			}
 			else if (c == '%') {
 				/* queue it verbatim */
-- 
2.35.1

[PATCH 2/8] tools/nolibc/stdlib: add a simple getenv() implementation

[Willy Tarreau]

This implementation relies on an extern definition of the environ
variable, that the caller must declare and initialize from envp.

Signed-off-by: Willy Tarreau <w@1wt.eu>
---
 tools/include/nolibc/stdlib.h | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

diff --git a/tools/include/nolibc/stdlib.h b/tools/include/nolibc/stdlib.h
index 733105c574ee..aca8616335e3 100644
--- a/tools/include/nolibc/stdlib.h
+++ b/tools/include/nolibc/stdlib.h
@@ -60,6 +60,29 @@ int atoi(const char *s)
 	return atol(s);
 }
 
+/* Tries to find the environment variable named <name> in the environment array
+ * pointed to by global variable "environ" which must be declared as a char **,
+ * and must be terminated by a NULL (it is recommended to set this variable to
+ * the "envp" argument of main()). If the requested environment variable exists
+ * its value is returned otherwise NULL is returned.
+ */
+static __attribute__((unused))
+char *getenv(const char *name)
+{
+	extern char **environ;
+	int idx, i;
+
+	if (environ) {
+		for (idx = 0; environ[idx]; idx++) {
+			for (i = 0; name[i] && name[i] == environ[idx][i];)
+				i++;
+			if (!name[i] && environ[idx][i] == '=')
+				return &environ[idx][i+1];
+		}
+	}
+	return NULL;
+}
+
 /* Converts the unsigned long integer <in> to its hex representation into
  * buffer <buffer>, which must be long enough to store the number and the
  * trailing zero (17 bytes for "ffffffffffffffff" or 9 for "ffffffff"). The
-- 
2.35.1

[PATCH 3/8] tools/nolibc/stdio: add support for '%p' to vfprintf()

[Willy Tarreau]

%p remains quite useful in test code, and the code path can easily be
merged with the existing "%x" thus only adds ~50 bytes, thus let's
add it.

Signed-off-by: Willy Tarreau <w@1wt.eu>
---
 tools/include/nolibc/stdio.h | 30 +++++++++++++++++++-----------
 1 file changed, 19 insertions(+), 11 deletions(-)

diff --git a/tools/include/nolibc/stdio.h b/tools/include/nolibc/stdio.h
index 559ebe052a75..15dedf8d0902 100644
--- a/tools/include/nolibc/stdio.h
+++ b/tools/include/nolibc/stdio.h
@@ -163,7 +163,7 @@ char *fgets(char *s, int size, FILE *stream)
 
 
 /* minimal vfprintf(). It supports the following formats:
- *  - %[l*]{d,u,c,x}
+ *  - %[l*]{d,u,c,x,p}
  *  - %s
  *  - unknown modifiers are ignored.
  */
@@ -184,8 +184,12 @@ int vfprintf(FILE *stream, const char *fmt, va_list args)
 		if (escape) {
 			/* we're in an escape sequence, ofs == 1 */
 			escape = 0;
-			if (c == 'c' || c == 'd' || c == 'u' || c == 'x') {
-				if (lpref) {
+			if (c == 'c' || c == 'd' || c == 'u' || c == 'x' || c == 'p') {
+				char *out = tmpbuf;
+
+				if (c == 'p')
+					v = va_arg(args, unsigned long);
+				else if (lpref) {
 					if (lpref > 1)
 						v = va_arg(args, unsigned long long);
 					else
@@ -202,18 +206,22 @@ int vfprintf(FILE *stream, const char *fmt, va_list args)
 				}
 
 				switch (c) {
+				case 'c':
+					out[0] = v;
+					out[1] = 0;
+					break;
 				case 'd':
-					i64toa_r(v, tmpbuf);
+					i64toa_r(v, out);
 					break;
 				case 'u':
-					u64toa_r(v, tmpbuf);
+					u64toa_r(v, out);
 					break;
-				case 'x':
-					u64toh_r(v, tmpbuf);
-					break;
-				default: /* 'c' */
-					tmpbuf[0] = v;
-					tmpbuf[1] = 0;
+				case 'p':
+					*(out++) = '0';
+					*(out++) = 'x';
+					/* fall through */
+				default: /* 'x' and 'p' above */
+					u64toh_r(v, out);
 					break;
 				}
 				outstr = tmpbuf;
-- 
2.35.1

[PATCH 4/8] tools/nolibc/string: add strcmp() and strncmp()

[Willy Tarreau]

We need these functions all the time, including when checking environment
variables and parsing command-line arguments. These implementations were
optimized to show optimal code size on a wide range of compilers (22 bytes
return included for strcmp(), 33 for strncmp()).

Signed-off-by: Willy Tarreau <w@1wt.eu>
---
 tools/include/nolibc/string.h | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

diff --git a/tools/include/nolibc/string.h b/tools/include/nolibc/string.h
index 4554b6fcb400..0d5e870c7c0b 100644
--- a/tools/include/nolibc/string.h
+++ b/tools/include/nolibc/string.h
@@ -102,6 +102,17 @@ char *strchr(const char *s, int c)
 	return NULL;
 }
 
+static __attribute__((unused))
+int strcmp(const char *a, const char *b)
+{
+	unsigned int c;
+	int diff;
+
+	while (!(diff = (unsigned char)*a++ - (c = (unsigned char)*b++)) && c)
+		;
+	return diff;
+}
+
 static __attribute__((unused))
 char *strcpy(char *dst, const char *src)
 {
@@ -184,6 +195,18 @@ char *strncat(char *dst, const char *src, size_t size)
 	return orig;
 }
 
+static __attribute__((unused))
+int strncmp(const char *a, const char *b, size_t size)
+{
+	unsigned int c;
+	int diff = 0;
+
+	while (size-- &&
+	       !(diff = (unsigned char)*a++ - (c = (unsigned char)*b++)) && c)
+		;
+
+	return diff;
+}
 
 static __attribute__((unused))
 char *strncpy(char *dst, const char *src, size_t size)
-- 
2.35.1

[PATCH 5/8] tools/nolibc/sys: add syscall definition for getppid()

[Willy Tarreau]

This is essentially for completeness as it's not the most often used
in regtests.

Signed-off-by: Willy Tarreau <w@1wt.eu>
---
 tools/include/nolibc/sys.h | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/tools/include/nolibc/sys.h b/tools/include/nolibc/sys.h
index 28437863c63f..4d4308d5d111 100644
--- a/tools/include/nolibc/sys.h
+++ b/tools/include/nolibc/sys.h
@@ -464,6 +464,23 @@ pid_t getpid(void)
 }
 
 
+/*
+ * pid_t getppid(void);
+ */
+
+static __attribute__((unused))
+pid_t sys_getppid(void)
+{
+	return my_syscall0(__NR_getppid);
+}
+
+static __attribute__((unused))
+pid_t getppid(void)
+{
+	return sys_getppid();
+}
+
+
 /*
  * pid_t gettid(void);
  */
-- 
2.35.1

[PATCH 6/8] tools/nolibc/types: add poll() and waitpid() flag definitions

[Willy Tarreau]

- POLLIN etc were missing, so poll() could only be used with timeouts.
- WNOHANG was not defined and is convenient to check if a child is still
  running

Signed-off-by: Willy Tarreau <w@1wt.eu>
---
 tools/include/nolibc/types.h | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/tools/include/nolibc/types.h b/tools/include/nolibc/types.h
index e4026e740b56..357e60ad38a8 100644
--- a/tools/include/nolibc/types.h
+++ b/tools/include/nolibc/types.h
@@ -82,6 +82,9 @@
 #define WEXITSTATUS(status) (((status) & 0xff00) >> 8)
 #define WIFEXITED(status)   (((status) & 0x7f) == 0)
 
+/* waitpid() flags */
+#define WNOHANG      1
+
 /* standard exit() codes */
 #define EXIT_SUCCESS 0
 #define EXIT_FAILURE 1
@@ -122,6 +125,13 @@ typedef struct {
 	} while (0)
 
 /* for poll() */
+#define POLLIN          0x0001
+#define POLLPRI         0x0002
+#define POLLOUT         0x0004
+#define POLLERR         0x0008
+#define POLLHUP         0x0010
+#define POLLNVAL        0x0020
+
 struct pollfd {
 	int fd;
 	short int events;
-- 
2.35.1

[PATCH 7/8] tools/nolibc: add a makefile to install headers

[Willy Tarreau]

This provides a target "headers_standalone" which installs the nolibc's
arch-specific headers with "arch.h" taken from the current arch (or a
concatenation of both i386 and x86_64 for arch=x86), then installs
kernel headers. This creates a convenient sysroot which is directly
usable by a bare-metal compiler to create any executable.

Signed-off-by: Willy Tarreau <w@1wt.eu>
---
 tools/include/nolibc/Makefile | 43 +++++++++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)
 create mode 100644 tools/include/nolibc/Makefile

diff --git a/tools/include/nolibc/Makefile b/tools/include/nolibc/Makefile
new file mode 100644
index 000000000000..7d4d61b08637
--- /dev/null
+++ b/tools/include/nolibc/Makefile
@@ -0,0 +1,43 @@
+# SPDX-License-Identifier: GPL-2.0
+# Makefile for nolibc installation and tests
+include ../../scripts/Makefile.include
+
+# we're in ".../tools/include/nolibc"
+ifeq ($(srctree),)
+srctree := $(patsubst %/tools/include/,%,$(dir $(CURDIR)))
+endif
+
+nolibc_arch := $(patsubst arm64,aarch64,$(ARCH))
+arch_file := arch-$(nolibc_arch).h
+all_files := ctype.h errno.h nolibc.h signal.h std.h stdio.h stdlib.h string.h \
+             sys.h time.h types.h unistd.h
+
+# install all headers needed to support a bare-metal compiler
+all:
+
+# Note: when ARCH is "x86" we concatenate both x86_64 and i386
+headers:
+	$(Q)mkdir -p $(OUTPUT)sysroot
+	$(Q)mkdir -p $(OUTPUT)sysroot/include
+	$(Q)cp $(all_files) $(OUTPUT)sysroot/include/
+	$(Q)if [ "$(ARCH)" = "x86" ]; then      \
+		sed -e                          \
+		  's,^#ifndef _NOLIBC_ARCH_X86_64_H,#if !defined(_NOLIBC_ARCH_X86_64_H) \&\& defined(__x86_64__),' \
+		  arch-x86_64.h;                \
+		sed -e                          \
+		  's,^#ifndef _NOLIBC_ARCH_I386_H,#if !defined(_NOLIBC_ARCH_I386_H) \&\& !defined(__x86_64__),' \
+		  arch-i386.h;                  \
+	elif [ -e "$(arch_file)" ]; then        \
+		cat $(arch_file);               \
+	else                                    \
+		echo "Fatal: architecture $(ARCH) not yet supported by nolibc." >&2; \
+		exit 1;                         \
+	fi > $(OUTPUT)sysroot/include/arch.h
+
+headers_standalone: headers
+	$(Q)$(MAKE) -C $(srctree) headers
+	$(Q)$(MAKE) -C $(srctree) headers_install INSTALL_HDR_PATH=$(OUTPUT)/sysroot
+
+clean:
+	$(call QUIET_CLEAN, nolibc) rm -rf "$(OUTPUT)sysroot"
+
-- 
2.35.1

[PATCH 8/8] tools/nolibc: add the nolibc subdir to the common Makefile

[Willy Tarreau]

The Makefile in tools/ is used to forward options to the makefiles
in the various subdirs. Let's add nolibc there so that it becomes
possible to make tools/nolibc_headers_standalone from the main tree
to simply create a completely usable sysroot.

Signed-off-by: Willy Tarreau <w@1wt.eu>
---
 tools/Makefile | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/tools/Makefile b/tools/Makefile
index db2f7b8ebed5..724134f0e56c 100644
--- a/tools/Makefile
+++ b/tools/Makefile
@@ -24,6 +24,7 @@ help:
 	@echo '  intel-speed-select     - Intel Speed Select tool'
 	@echo '  kvm_stat               - top-like utility for displaying kvm statistics'
 	@echo '  leds                   - LEDs  tools'
+	@echo '  nolibc                 - nolibc headers testing and installation'
 	@echo '  objtool                - an ELF object analysis tool'
 	@echo '  pci                    - PCI tools'
 	@echo '  perf                   - Linux performance measurement and analysis tool'
@@ -74,6 +75,9 @@ bpf/%: FORCE
 libapi: FORCE
 	$(call descend,lib/api)
 
+nolibc_%: FORCE
+	$(call descend,include/nolibc,$(patsubst nolibc_%,%,$@))
+
 # The perf build does not follow the descend function setup,
 # invoking it via it's own make rule.
 PERF_O   = $(if $(O),$(O)/tools/perf,)
-- 
2.35.1

Re: [PATCH 0/8] nolibc: small updates required for the self-tests

[Paul E. McKenney]

On Mon, Mar 21, 2022 at 06:33:06PM +0100, Willy Tarreau wrote:
> Hello Paul,
> 
> while developing the nolibc self-tests, I naturally faced a few
> limitations ranging from missing POLL* flags to missing strcmp() or
> getenv(), as well as the ability for printf() to print "(null)" on
> "%s" instead of crashing.
> 
> This series adds a makefile and a headers installation target that
> creates a sysroot made of kernel headers and nolibc headers, which
> significantly ease building programs.
> 
> I already have some test code with roughly 75 tests, but I find it
> misplaced in the nolibc dir, I need to move it to testing/selftests/
> before sending it to you.
> 
> This series is intended to be an add-on to what you already have in your
> dev branch. I don't intend to send you much more stuff on top of this,
> but I expect to see an update from Ammar's patch set (CCed). With all
> this I think we'll have a good basis to easily add new tests.

I have pulled this series in for review and testing, thank you!

It won't go into -next until v5.18-rc1 comes out, in about two weeks,
but it will at least be in -rcu.

I removed an ostensibly extraneous blank line from the end of
tools/include/nolibc/Makefile.  So please let me know if that blank line
is actually necessary.

							Thanx, Paul

> Thanks!
> Willy
> 
> ---
> Willy Tarreau (8):
>   tools/nolibc/stdio: make printf(%s) accept NULL
>   tools/nolibc/stdlib: add a simple getenv() implementation
>   tools/nolibc/stdio: add support for '%p' to vfprintf()
>   tools/nolibc/string: add strcmp() and strncmp()
>   tools/nolibc/sys: add syscall definition for getppid()
>   tools/nolibc/types: add poll() and waitpid() flag definitions
>   tools/nolibc: add a makefile to install headers
>   tools/nolibc: add the nolibc subdir to the common Makefile
> 
>  tools/Makefile                |  4 ++++
>  tools/include/nolibc/Makefile | 43 +++++++++++++++++++++++++++++++++++
>  tools/include/nolibc/stdio.h  | 32 +++++++++++++++++---------
>  tools/include/nolibc/stdlib.h | 23 +++++++++++++++++++
>  tools/include/nolibc/string.h | 23 +++++++++++++++++++
>  tools/include/nolibc/sys.h    | 17 ++++++++++++++
>  tools/include/nolibc/types.h  | 10 ++++++++
>  7 files changed, 141 insertions(+), 11 deletions(-)
>  create mode 100644 tools/include/nolibc/Makefile
> 
> -- 
> 2.35.1
> 

Re: [PATCH 0/8] nolibc: small updates required for the self-tests

[Willy Tarreau]

On Mon, Mar 21, 2022 at 01:29:08PM -0700, Paul E. McKenney wrote:
> On Mon, Mar 21, 2022 at 06:33:06PM +0100, Willy Tarreau wrote:
> > Hello Paul,
> > 
> > while developing the nolibc self-tests, I naturally faced a few
> > limitations ranging from missing POLL* flags to missing strcmp() or
> > getenv(), as well as the ability for printf() to print "(null)" on
> > "%s" instead of crashing.
> > 
> > This series adds a makefile and a headers installation target that
> > creates a sysroot made of kernel headers and nolibc headers, which
> > significantly ease building programs.
> > 
> > I already have some test code with roughly 75 tests, but I find it
> > misplaced in the nolibc dir, I need to move it to testing/selftests/
> > before sending it to you.
> > 
> > This series is intended to be an add-on to what you already have in your
> > dev branch. I don't intend to send you much more stuff on top of this,
> > but I expect to see an update from Ammar's patch set (CCed). With all
> > this I think we'll have a good basis to easily add new tests.
> 
> I have pulled this series in for review and testing, thank you!
> 
> It won't go into -next until v5.18-rc1 comes out, in about two weeks,
> but it will at least be in -rcu.

Perfect, thank you!

> I removed an ostensibly extraneous blank line from the end of
> tools/include/nolibc/Makefile.  So please let me know if that blank line
> is actually necessary.

Oh I'm very sorry, that the type of thing I'm very careful about so it
has definitely escaped my checks.

Thanks,
Willy

Re: [PATCH 0/8] nolibc: small updates required for the self-tests

[Paul E. McKenney]

On Mon, Mar 21, 2022 at 09:33:19PM +0100, Willy Tarreau wrote:
> On Mon, Mar 21, 2022 at 01:29:08PM -0700, Paul E. McKenney wrote:
> > On Mon, Mar 21, 2022 at 06:33:06PM +0100, Willy Tarreau wrote:
> > > Hello Paul,
> > > 
> > > while developing the nolibc self-tests, I naturally faced a few
> > > limitations ranging from missing POLL* flags to missing strcmp() or
> > > getenv(), as well as the ability for printf() to print "(null)" on
> > > "%s" instead of crashing.
> > > 
> > > This series adds a makefile and a headers installation target that
> > > creates a sysroot made of kernel headers and nolibc headers, which
> > > significantly ease building programs.
> > > 
> > > I already have some test code with roughly 75 tests, but I find it
> > > misplaced in the nolibc dir, I need to move it to testing/selftests/
> > > before sending it to you.
> > > 
> > > This series is intended to be an add-on to what you already have in your
> > > dev branch. I don't intend to send you much more stuff on top of this,
> > > but I expect to see an update from Ammar's patch set (CCed). With all
> > > this I think we'll have a good basis to easily add new tests.
> > 
> > I have pulled this series in for review and testing, thank you!
> > 
> > It won't go into -next until v5.18-rc1 comes out, in about two weeks,
> > but it will at least be in -rcu.
> 
> Perfect, thank you!
> 
> > I removed an ostensibly extraneous blank line from the end of
> > tools/include/nolibc/Makefile.  So please let me know if that blank line
> > is actually necessary.
> 
> Oh I'm very sorry, that the type of thing I'm very careful about so it
> has definitely escaped my checks.

"To err is human!"  ;-)

							Thanx, Paul