[PATCH v1 00/17] selftests/nolibc: allow run with minimal kernel config

[PATCH v1 00/17] selftests/nolibc: allow run with minimal kernel config

[Zhangjin Wu]

Hi, Willy

This patchset mainly allows speed up the nolibc test with a minimal
kernel config.

As the nolibc supported architectures become more and more, the 'run'
test with DEFCONFIG may cost several hours, which is not friendly to
develop testing and even for release testing, so, smaller kernel configs
may be required, and firstly, we should let nolibc-test work with less
kernel config options, this patchset aims to this goal.

This patchset mainly remove the dependency from procfs, tmpfs, net and
memfd_create, many failures have been fixed up.

When CONFIG_TMPFS and CONFIG_SHMEM are disabled, kernel will provide a
ramfs based tmpfs (mm/shmem.c), it will be used as a choice to fix up
some failures and also allow skip less tests.

Besides, it also adds musl support, improves glibc support and fixes up
a kernel cmdline passing use case.

This is based on the dev.2023.06.14a branch of linux-rcu [1], all of the
supported architectures are tested (with local minimal configs, [5]
pasted the one for i386) without failures:

           arch/board | result
          ------------|------------
      arm/vexpress-a9 | 138 test(s) passed, 1 skipped, 0 failed. See all results in /labs/linux-lab/logging/nolibc/arm-vexpress-a9-nolibc-test.log
         aarch64/virt | 138 test(s) passed, 1 skipped, 0 failed. See all results in /labs/linux-lab/logging/nolibc/aarch64-virt-nolibc-test.log
          ppc/g3beige | not supported
              i386/pc | 136 test(s) passed, 3 skipped, 0 failed. See all results in /labs/linux-lab/logging/nolibc/i386-pc-nolibc-test.log
            x86_64/pc | 138 test(s) passed, 1 skipped, 0 failed. See all results in /labs/linux-lab/logging/nolibc/x86_64-pc-nolibc-test.log
         mipsel/malta | 138 test(s) passed, 1 skipped, 0 failed. See all results in /labs/linux-lab/logging/nolibc/mipsel-malta-nolibc-test.log
     loongarch64/virt | 138 test(s) passed, 1 skipped, 0 failed. See all results in /labs/linux-lab/logging/nolibc/loongarch64-virt-nolibc-test.log
         riscv64/virt | 136 test(s) passed, 3 skipped, 0 failed. See all results in /labs/linux-lab/logging/nolibc/riscv64-virt-nolibc-test.log
         riscv32/virt | no test log found
s390x/s390-ccw-virtio | 138 test(s) passed, 1 skipped, 0 failed. See all results in /labs/linux-lab/logging/nolibc/s390x-s390-ccw-virtio-nolibc-test.log

Notes:
  * The skipped ones are -fstackprotector, chmod_self and chown_self

    The -fstackprotector skip is due to gcc version.
    chmod_self and chmod_self skips are due to procfs not enabled

  * ppc/g3beige support is added locally, but not added in this patchset

    will send ppc support as a new patchset, it depends on v2 test
    report patchset [3] and the v5 rv32 support, require changes on
    Makefile

  * riscv32/virt support is still in review, see v5 rv32 support [4]

This patchset doesn't depends on any of my other nolibc patch series,
but the new rmdir() routine added in this patchset may be requird to
apply the __sysret() from our v4 syscall helper series [2] after that
series being merged, currently, we use the old method to let it compile
without any dependency.

Here explains all of the patches:

* selftests/nolibc: stat_fault: silence NULL argument warning with glibc
  selftests/nolibc: gettid: restore for glibc and musl
  selftests/nolibc: add _LARGEFILE64_SOURCE for musl

  The above 3 patches adds musl compile support and improve glibc support.

  It is able to build and run nolibc-test with musl libc now, but there
  are some failures/skips due to the musl its own issues/requirements:

    $ sudo ./nolibc-test  | grep -E 'FAIL|SKIP'
    8 sbrk = 1 ENOMEM                                               [FAIL]
    9 brk = -1 ENOMEM                                               [FAIL]
    46 limit_int_fast16_min = -2147483648                           [FAIL]
    47 limit_int_fast16_max = 2147483647                            [FAIL]
    49 limit_int_fast32_min = -2147483648                           [FAIL]
    50 limit_int_fast32_max = 2147483647                            [FAIL]
    0 -fstackprotector not supported                                [SKIPPED]

  musl disabled sbrk and brk for some conflicts with its malloc and the
  fast version of int types are defined in 32bit, which differs from nolibc
  and glibc. musl reserved the sbrk(0) to allow get current brk, we
  added a test for this in the v4 __sysret() helper series [2].

* selftests/nolibc: fix up kernel parameters support

  kernel cmdline allows pass two types of parameters, one is without
  '=', another is with '=', the first one is passed as init arguments,
  the sencond one is passed as init environment variables.

  Our nolibc-test prefer arguments to environment variables, this not
  work when users add such parameters in the kernel cmdline:

    noapic NOLIBC_TEST=syscall

  So, this patch will verify the setting from arguments at first, if it
  is no valid, will try the environment variables instead.

* selftests/nolibc: stat_timestamps: remove procfs dependency

  Use '/' instead of /proc/self, or we can add a 'has_proc' condition
  for this test case, but it is not that necessary to skip the whole
  stat_timestamps tests for such a subtest binding to /proc/self.

  Welcome suggestion from Thomas.

* tools/nolibc: add rmdir() support
  selftests/nolibc: add a new rmdir() test case

  rmdir() routine and test case are added for the coming requirement.

  Note, if the __sysret() patchset [2] is applied before us, this patch
  should be rebased on it and apply the __sysret() helper.

* selftests/nolibc: fix up failures when there is no procfs

  call rmdir() to remove /proc completely to rework the checking of
  /proc, before, the existing of /proc not means the procfs is really
  mounted.

* selftests/nolibc: rename proc variable to has_proc
  selftests/nolibc: rename euid0 variable to is_root

  align with the has_gettid, has_xxx variables.

* selftests/nolibc: prepare tmpfs and hugetlbfs
  selftests/nolibc: rename chmod_net to chmod_good
  selftests/nolibc: link_cross: support tmpfs
  selftests/nolibc: rename chroot_exe to chroot_file

  use file from /tmp instead of file from /proc when there is no procfs
  this avoid skipping the chmod_net, link_cross, chroot_exe tests

* selftests/nolibc: vfprintf: silence memfd_create() warning
  selftests/nolibc: vfprintf: skip if neither tmpfs nor hugetlbfs
  selftests/nolibc: vfprintf: support tmpfs and hugetlbfs

  memfd_create from kernel >= v6.2 forcely warn on missing
  MFD_NOEXEC_SEAL flag, the first one silence it with such flag, for
  older kernels, use 0 flag as before.

  since memfd_create() depends on TMPFS or HUGETLBFS, the second one
  skip the whole vfprintf instead of simply fail if memfd_create() not
  work.

  the 3rd one futher try the ramfs based tmpfs even when memfd_create()
  not work.

At last, let's simply discuss about the configs, I have prepared minimal
configs for all of the nolibc supported architectures but not sure where
should we put them, what about tools/testing/selftests/nolibc/configs ?

Thanks!

Best regards,
Zhangjin
---

[1]: https://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git/
[2]: https://lore.kernel.org/linux-riscv/cover.1687187451.git.falcon@tinylab.org/
[3]: https://lore.kernel.org/lkml/cover.1687156559.git.falcon@tinylab.org/
[4]: https://lore.kernel.org/linux-riscv/cover.1687176996.git.falcon@tinylab.org/
[5]: https://pastebin.com/5jq0Vxbz 

Zhangjin Wu (17):
  selftests/nolibc: stat_fault: silence NULL argument warning with glibc
  selftests/nolibc: gettid: restore for glibc and musl
  selftests/nolibc: add _LARGEFILE64_SOURCE for musl
  selftests/nolibc: fix up kernel parameters support
  selftests/nolibc: stat_timestamps: remove procfs dependency
  tools/nolibc: add rmdir() support
  selftests/nolibc: add a new rmdir() test case
  selftests/nolibc: fix up failures when there is no procfs
  selftests/nolibc: rename proc variable to has_proc
  selftests/nolibc: rename euid0 variable to is_root
  selftests/nolibc: prepare tmpfs and hugetlbfs
  selftests/nolibc: rename chmod_net to chmod_good
  selftests/nolibc: link_cross: support tmpfs
  selftests/nolibc: rename chroot_exe to chroot_file
  selftests/nolibc: vfprintf: silence memfd_create() warning
  selftests/nolibc: vfprintf: skip if neither tmpfs nor hugetlbfs
  selftests/nolibc: vfprintf: support tmpfs and hugetlbfs

 tools/include/nolibc/sys.h                   |  28 ++++
 tools/testing/selftests/nolibc/nolibc-test.c | 132 +++++++++++++++----
 2 files changed, 138 insertions(+), 22 deletions(-)

-- 
2.25.1

[PATCH v1 01/17] selftests/nolibc: stat_fault: silence NULL argument warning with glibc

[Zhangjin Wu]

Use another invalid address (void *)1 instead of NULL to silence this
compile warning with glibc:

    $ make libc-test
      CC      libc-test
    nolibc-test.c: In function ‘run_syscall’:
    nolibc-test.c:622:49: warning: null argument where non-null required (argument 1) [-Wnonnull]
      622 |   CASE_TEST(stat_fault);        EXPECT_SYSER(1, stat(NULL, &stat_buf), -1, EFAULT); break;
          |                                                 ^~~~
    nolibc-test.c:304:79: note: in definition of macro ‘EXPECT_SYSER2’
      304 |  do { if (!cond) pad_spc(llen, 64, "[SKIPPED]\n"); else ret += expect_syserr2(expr, expret, experr1, experr2, llen); } while (0)
          |                                                                               ^~~~
    nolibc-test.c:622:33: note: in expansion of macro ‘EXPECT_SYSER’
      622 |   CASE_TEST(stat_fault);        EXPECT_SYSER(1, stat(NULL, &stat_buf), -1, EFAULT); break;

Signed-off-by: Zhangjin Wu <falcon@tinylab.org>
---
 tools/testing/selftests/nolibc/nolibc-test.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/testing/selftests/nolibc/nolibc-test.c b/tools/testing/selftests/nolibc/nolibc-test.c
index 486334981e60..99afec93dfae 100644
--- a/tools/testing/selftests/nolibc/nolibc-test.c
+++ b/tools/testing/selftests/nolibc/nolibc-test.c
@@ -619,7 +619,7 @@ int run_syscall(int min, int max)
 		CASE_TEST(select_stdout);     EXPECT_SYSNE(1, ({ fd_set fds; FD_ZERO(&fds); FD_SET(1, &fds); select(2, NULL, &fds, NULL, NULL); }), -1); break;
 		CASE_TEST(select_fault);      EXPECT_SYSER(1, select(1, (void *)1, NULL, NULL, 0), -1, EFAULT); break;
 		CASE_TEST(stat_blah);         EXPECT_SYSER(1, stat("/proc/self/blah", &stat_buf), -1, ENOENT); break;
-		CASE_TEST(stat_fault);        EXPECT_SYSER(1, stat(NULL, &stat_buf), -1, EFAULT); break;
+		CASE_TEST(stat_fault);        EXPECT_SYSER(1, stat((void *)1, &stat_buf), -1, EFAULT); break;
 		CASE_TEST(stat_timestamps);   EXPECT_SYSZR(1, test_stat_timestamps()); break;
 		CASE_TEST(symlink_root);      EXPECT_SYSER(1, symlink("/", "/"), -1, EEXIST); break;
 		CASE_TEST(unlink_root);       EXPECT_SYSER(1, unlink("/"), -1, EISDIR); break;
-- 
2.25.1

[PATCH v1 02/17] selftests/nolibc: gettid: restore for glibc and musl

[Zhangjin Wu]

As the gettid manpage [1] shows, glibc 2.30 has gettid support, so,
let's enable the test for glibc >= 2.30.

gettid works on musl too.

[1]: https://man7.org/linux/man-pages/man2/gettid.2.html

Signed-off-by: Zhangjin Wu <falcon@tinylab.org>
---
 tools/testing/selftests/nolibc/nolibc-test.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/tools/testing/selftests/nolibc/nolibc-test.c b/tools/testing/selftests/nolibc/nolibc-test.c
index 99afec93dfae..739c9daa91b6 100644
--- a/tools/testing/selftests/nolibc/nolibc-test.c
+++ b/tools/testing/selftests/nolibc/nolibc-test.c
@@ -548,6 +548,7 @@ int run_syscall(int min, int max)
 	int tmp;
 	int ret = 0;
 	void *p1, *p2;
+	int has_gettid = 1;
 
 	/* <proc> indicates whether or not /proc is mounted */
 	proc = stat("/proc", &stat_buf) == 0;
@@ -555,6 +556,11 @@ int run_syscall(int min, int max)
 	/* this will be used to skip certain tests that can't be run unprivileged */
 	euid0 = geteuid() == 0;
 
+	/* from 2.30, glibc provides gettid() */
+#if defined(__GLIBC_MINOR__) && defined(__GLIBC__)
+	has_gettid = __GLIBC__ > 2 || (__GLIBC__ == 2 && __GLIBC_MINOR__ >= 30);
+#endif
+
 	for (test = min; test >= 0 && test <= max; test++) {
 		int llen = 0; /* line length */
 
@@ -564,9 +570,7 @@ int run_syscall(int min, int max)
 		switch (test + __LINE__ + 1) {
 		CASE_TEST(getpid);            EXPECT_SYSNE(1, getpid(), -1); break;
 		CASE_TEST(getppid);           EXPECT_SYSNE(1, getppid(), -1); break;
-#ifdef NOLIBC
-		CASE_TEST(gettid);            EXPECT_SYSNE(1, gettid(), -1); break;
-#endif
+		CASE_TEST(gettid);            EXPECT_SYSNE(has_gettid, gettid(), -1); break;
 		CASE_TEST(getpgid_self);      EXPECT_SYSNE(1, getpgid(0), -1); break;
 		CASE_TEST(getpgid_bad);       EXPECT_SYSER(1, getpgid(-1), -1, ESRCH); break;
 		CASE_TEST(kill_0);            EXPECT_SYSZR(1, kill(getpid(), 0)); break;
-- 
2.25.1

[PATCH v1 03/17] selftests/nolibc: add _LARGEFILE64_SOURCE for musl

[Zhangjin Wu]

_GNU_SOURCE Implies _LARGEFILE64_SOURCE in glibc, but in musl, the
default configuration doesn't enable _LARGEFILE64_SOURCE.

From include/dirent.h of musl, getdents64 is provided as getdents when
_LARGEFILE64_SOURCE is defined.

    #if defined(_LARGEFILE64_SOURCE)
    ...
    #define getdents64 getdents
    #endif

Let's define _LARGEFILE64_SOURCE to fix up this compile error:

    tools/testing/selftests/nolibc/nolibc-test.c: In function ‘test_getdents64’:
    tools/testing/selftests/nolibc/nolibc-test.c:453:8: warning: implicit declaration of function ‘getdents64’; did you mean ‘getdents’? [-Wimplicit-function-declaration]
      453 |  ret = getdents64(fd, (void *)buffer, sizeof(buffer));
          |        ^~~~~~~~~~
          |        getdents
    /usr/bin/ld: /tmp/ccKILm5u.o: in function `test_getdents64':
    nolibc-test.c:(.text+0xe3e): undefined reference to `getdents64'
    collect2: error: ld returned 1 exit status

Signed-off-by: Zhangjin Wu <falcon@tinylab.org>
---
 tools/testing/selftests/nolibc/nolibc-test.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tools/testing/selftests/nolibc/nolibc-test.c b/tools/testing/selftests/nolibc/nolibc-test.c
index 739c9daa91b6..d43116553288 100644
--- a/tools/testing/selftests/nolibc/nolibc-test.c
+++ b/tools/testing/selftests/nolibc/nolibc-test.c
@@ -1,6 +1,7 @@
 /* SPDX-License-Identifier: GPL-2.0 */
 
 #define _GNU_SOURCE
+#define _LARGEFILE64_SOURCE
 
 /* libc-specific include files
  * The program may be built in 3 ways:
-- 
2.25.1

[PATCH v1 04/17] selftests/nolibc: fix up kernel parameters support

[Zhangjin Wu]

kernel parameters allow pass two types of strings, one type is like
'noapic', another type is like 'panic=5', the first type is passed as
arguments of the init program, the second type is passed as environment
variables of the init program.

when users pass kernel parameters like this:

    noapic NOLIBC_TEST=syscall

our nolibc-test program will ignore the NOLIBC_TEST environment
variable.

Let's verify the first type (from arguments), if it is invalid, get the
test setting from NOLIBC_TEST environment variable instead.

Signed-off-by: Zhangjin Wu <falcon@tinylab.org>
---
 tools/testing/selftests/nolibc/nolibc-test.c | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/nolibc/nolibc-test.c b/tools/testing/selftests/nolibc/nolibc-test.c
index d43116553288..ebec948ec808 100644
--- a/tools/testing/selftests/nolibc/nolibc-test.c
+++ b/tools/testing/selftests/nolibc/nolibc-test.c
@@ -951,6 +951,7 @@ int main(int argc, char **argv, char **envp)
 	int ret = 0;
 	int err;
 	int idx;
+	int len, valid = 0;
 	char *test;
 
 	environ = envp;
@@ -969,7 +970,19 @@ int main(int argc, char **argv, char **envp)
 	 *    syscall:5-15[:.*],stdlib:8-10
 	 */
 	test = argv[1];
-	if (!test)
+
+	/* verify the test setting from argv[1] */
+	if (test) {
+		for (idx = 0; test_names[idx].name; idx++) {
+			len = strlen(test_names[idx].name);
+			if (strncmp(test, test_names[idx].name, len) == 0) {
+				valid = 1;
+				break;
+			}
+		}
+	}
+
+	if (!valid)
 		test = getenv("NOLIBC_TEST");
 
 	if (test) {
-- 
2.25.1

[PATCH v1 05/17] selftests/nolibc: stat_timestamps: remove procfs dependency

[Zhangjin Wu]

Since it is not really necessary to use /proc/self here, instead of
adding a condition check, we use the always existing '/' path instead of
/proc/self, this eventually let it work without procfs.

Signed-off-by: Zhangjin Wu <falcon@tinylab.org>
---
 tools/testing/selftests/nolibc/nolibc-test.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/testing/selftests/nolibc/nolibc-test.c b/tools/testing/selftests/nolibc/nolibc-test.c
index ebec948ec808..2ef44176f7a9 100644
--- a/tools/testing/selftests/nolibc/nolibc-test.c
+++ b/tools/testing/selftests/nolibc/nolibc-test.c
@@ -520,7 +520,7 @@ static int test_stat_timestamps(void)
 	if (sizeof(st.st_atim.tv_sec) != sizeof(st.st_atime))
 		return 1;
 
-	if (stat("/proc/self/", &st))
+	if (stat("/", &st))
 		return 1;
 
 	if (st.st_atim.tv_sec != st.st_atime || st.st_atim.tv_nsec > 1000000000)
-- 
2.25.1

[PATCH v1 06/17] tools/nolibc: add rmdir() support

[Zhangjin Wu]

A reverse operation of mkdir is meaningful, add rmdir() here.

This is required by nolibc-test to remove /proc if CONFIG_PROC_FS not
enabled.

Signed-off-by: Zhangjin Wu <falcon@tinylab.org>
---
 tools/include/nolibc/sys.h | 28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)

diff --git a/tools/include/nolibc/sys.h b/tools/include/nolibc/sys.h
index 856249a11890..8ddfd9185da6 100644
--- a/tools/include/nolibc/sys.h
+++ b/tools/include/nolibc/sys.h
@@ -716,6 +716,34 @@ int mkdir(const char *path, mode_t mode)
 	return ret;
 }
 
+/*
+ * int rmdir(const char *path);
+ */
+
+static __attribute__((unused))
+int sys_rmdir(const char *path)
+{
+#ifdef __NR_rmdir
+	return my_syscall1(__NR_rmdir, path);
+#elif defined(__NR_unlinkat)
+	return my_syscall3(__NR_unlinkat, AT_FDCWD, path, AT_REMOVEDIR);
+#else
+	return -ENOSYS;
+#endif
+}
+
+static __attribute__((unused))
+int rmdir(const char *path)
+{
+	int ret = sys_rmdir(path);
+
+	if (ret < 0) {
+		SET_ERRNO(-ret);
+		ret = -1;
+	}
+	return ret;
+}
+
 
 /*
  * int mknod(const char *path, mode_t mode, dev_t dev);
-- 
2.25.1

[PATCH v1 07/17] selftests/nolibc: add a new rmdir() test case

[Zhangjin Wu]

A new rmdir_blah test case is added to remove a non-existing /blah,
which expects failure with ENOENT errno.

Signed-off-by: Zhangjin Wu <falcon@tinylab.org>
---
 tools/testing/selftests/nolibc/nolibc-test.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tools/testing/selftests/nolibc/nolibc-test.c b/tools/testing/selftests/nolibc/nolibc-test.c
index 2ef44176f7a9..9673c338d42e 100644
--- a/tools/testing/selftests/nolibc/nolibc-test.c
+++ b/tools/testing/selftests/nolibc/nolibc-test.c
@@ -612,6 +612,7 @@ int run_syscall(int min, int max)
 		CASE_TEST(lseek_m1);          EXPECT_SYSER(1, lseek(-1, 0, SEEK_SET), -1, EBADF); break;
 		CASE_TEST(lseek_0);           EXPECT_SYSER(1, lseek(0, 0, SEEK_SET), -1, ESPIPE); break;
 		CASE_TEST(mkdir_root);        EXPECT_SYSER(1, mkdir("/", 0755), -1, EEXIST); break;
+		CASE_TEST(rmdir_blah);        EXPECT_SYSER(1, rmdir("/blah"), -1, ENOENT); break;
 		CASE_TEST(open_tty);          EXPECT_SYSNE(1, tmp = open("/dev/null", 0), -1); if (tmp != -1) close(tmp); break;
 		CASE_TEST(open_blah);         EXPECT_SYSER(1, tmp = open("/proc/self/blah", 0), -1, ENOENT); if (tmp != -1) close(tmp); break;
 		CASE_TEST(poll_null);         EXPECT_SYSZR(1, poll(NULL, 0, 0)); break;
-- 
2.25.1

[PATCH v1 08/17] selftests/nolibc: fix up failures when there is no procfs

[Zhangjin Wu]

When there is no procfs support, the /proc is not mountable, but the
/proc directory has been created in the prepare() stage and therefore,
the checking of /proc in the run_syscall() stage will be always true and
at last will fail all of the procfs dependent test cases.

To solve this issue, one method is checking /proc/self instead of /proc,
another one is removing the /proc directory completely (when there is
really no procfs support), we apply the second method to avoid mislead
the users.

Signed-off-by: Zhangjin Wu <falcon@tinylab.org>
---
 tools/testing/selftests/nolibc/nolibc-test.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/tools/testing/selftests/nolibc/nolibc-test.c b/tools/testing/selftests/nolibc/nolibc-test.c
index 9673c338d42e..e5a218ef8a2d 100644
--- a/tools/testing/selftests/nolibc/nolibc-test.c
+++ b/tools/testing/selftests/nolibc/nolibc-test.c
@@ -928,8 +928,11 @@ int prepare(void)
 
 	/* try to mount /proc if not mounted. Silently fail otherwise */
 	if (stat("/proc/.", &stat_buf) == 0 || mkdir("/proc", 0755) == 0) {
-		if (stat("/proc/self", &stat_buf) != 0)
-			mount("/proc", "/proc", "proc", 0, 0);
+		if (stat("/proc/self", &stat_buf) != 0) {
+			/* If not mountable, remove /proc completely to avoid misuse */
+			if (mount("none", "/proc", "proc", 0, 0) != 0)
+				rmdir("/proc");
+		}
 	}
 
 	return 0;
-- 
2.25.1

[PATCH v1 09/17] selftests/nolibc: rename proc variable to has_proc

[Zhangjin Wu]

More conditional variables will be added, let's rename them with more
meaningful 'has_' or 'is_' prefix.

This converts proc to has_proc to reflect whether or not proc/ is
mounted.

Signed-off-by: Zhangjin Wu <falcon@tinylab.org>
---
 tools/testing/selftests/nolibc/nolibc-test.c | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/tools/testing/selftests/nolibc/nolibc-test.c b/tools/testing/selftests/nolibc/nolibc-test.c
index e5a218ef8a2d..929d43b95acc 100644
--- a/tools/testing/selftests/nolibc/nolibc-test.c
+++ b/tools/testing/selftests/nolibc/nolibc-test.c
@@ -544,15 +544,15 @@ int run_syscall(int min, int max)
 	struct timezone tz;
 	struct stat stat_buf;
 	int euid0;
-	int proc;
+	int has_proc;
 	int test;
 	int tmp;
 	int ret = 0;
 	void *p1, *p2;
 	int has_gettid = 1;
 
-	/* <proc> indicates whether or not /proc is mounted */
-	proc = stat("/proc", &stat_buf) == 0;
+	/* <has_proc> indicates whether or not /proc is mounted */
+	has_proc = stat("/proc", &stat_buf) == 0;
 
 	/* this will be used to skip certain tests that can't be run unprivileged */
 	euid0 = geteuid() == 0;
@@ -582,12 +582,12 @@ int run_syscall(int min, int max)
 		CASE_TEST(chdir_root);        EXPECT_SYSZR(1, chdir("/")); break;
 		CASE_TEST(chdir_dot);         EXPECT_SYSZR(1, chdir(".")); break;
 		CASE_TEST(chdir_blah);        EXPECT_SYSER(1, chdir("/blah"), -1, ENOENT); break;
-		CASE_TEST(chmod_net);         EXPECT_SYSZR(proc, chmod("/proc/self/net", 0555)); break;
-		CASE_TEST(chmod_self);        EXPECT_SYSER(proc, chmod("/proc/self", 0555), -1, EPERM); break;
-		CASE_TEST(chown_self);        EXPECT_SYSER(proc, chown("/proc/self", 0, 0), -1, EPERM); break;
+		CASE_TEST(chmod_net);         EXPECT_SYSZR(has_proc, chmod("/proc/self/net", 0555)); break;
+		CASE_TEST(chmod_self);        EXPECT_SYSER(has_proc, chmod("/proc/self", 0555), -1, EPERM); break;
+		CASE_TEST(chown_self);        EXPECT_SYSER(has_proc, chown("/proc/self", 0, 0), -1, EPERM); break;
 		CASE_TEST(chroot_root);       EXPECT_SYSZR(euid0, chroot("/")); break;
 		CASE_TEST(chroot_blah);       EXPECT_SYSER(1, chroot("/proc/self/blah"), -1, ENOENT); break;
-		CASE_TEST(chroot_exe);        EXPECT_SYSER(proc, chroot("/proc/self/exe"), -1, ENOTDIR); break;
+		CASE_TEST(chroot_exe);        EXPECT_SYSER(has_proc, chroot("/proc/self/exe"), -1, ENOTDIR); break;
 		CASE_TEST(close_m1);          EXPECT_SYSER(1, close(-1), -1, EBADF); break;
 		CASE_TEST(close_dup);         EXPECT_SYSZR(1, close(dup(0))); break;
 		CASE_TEST(dup_0);             tmp = dup(0);  EXPECT_SYSNE(1, tmp, -1); close(tmp); break;
@@ -608,7 +608,7 @@ int run_syscall(int min, int max)
 		CASE_TEST(link_root1);        EXPECT_SYSER(1, link("/", "/"), -1, EEXIST); break;
 		CASE_TEST(link_blah);         EXPECT_SYSER(1, link("/proc/self/blah", "/blah"), -1, ENOENT); break;
 		CASE_TEST(link_dir);          EXPECT_SYSER(euid0, link("/", "/blah"), -1, EPERM); break;
-		CASE_TEST(link_cross);        EXPECT_SYSER(proc, link("/proc/self/net", "/blah"), -1, EXDEV); break;
+		CASE_TEST(link_cross);        EXPECT_SYSER(has_proc, link("/proc/self/net", "/blah"), -1, EXDEV); break;
 		CASE_TEST(lseek_m1);          EXPECT_SYSER(1, lseek(-1, 0, SEEK_SET), -1, EBADF); break;
 		CASE_TEST(lseek_0);           EXPECT_SYSER(1, lseek(0, 0, SEEK_SET), -1, ESPIPE); break;
 		CASE_TEST(mkdir_root);        EXPECT_SYSER(1, mkdir("/", 0755), -1, EEXIST); break;
-- 
2.25.1

[PATCH v1 10/17] selftests/nolibc: rename euid0 variable to is_root

[Zhangjin Wu]

More conditional variables will be added, let's rename them with more
meaningful 'has_' or 'is_' prefix.

euid0 is converted to is_root to indicate whether or not running as
root.

Signed-off-by: Zhangjin Wu <falcon@tinylab.org>
---
 tools/testing/selftests/nolibc/nolibc-test.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/tools/testing/selftests/nolibc/nolibc-test.c b/tools/testing/selftests/nolibc/nolibc-test.c
index 929d43b95acc..69f59a395746 100644
--- a/tools/testing/selftests/nolibc/nolibc-test.c
+++ b/tools/testing/selftests/nolibc/nolibc-test.c
@@ -543,7 +543,7 @@ int run_syscall(int min, int max)
 	struct timeval tv;
 	struct timezone tz;
 	struct stat stat_buf;
-	int euid0;
+	int is_root;
 	int has_proc;
 	int test;
 	int tmp;
@@ -555,7 +555,7 @@ int run_syscall(int min, int max)
 	has_proc = stat("/proc", &stat_buf) == 0;
 
 	/* this will be used to skip certain tests that can't be run unprivileged */
-	euid0 = geteuid() == 0;
+	is_root = geteuid() == 0;
 
 	/* from 2.30, glibc provides gettid() */
 #if defined(__GLIBC_MINOR__) && defined(__GLIBC__)
@@ -585,7 +585,7 @@ int run_syscall(int min, int max)
 		CASE_TEST(chmod_net);         EXPECT_SYSZR(has_proc, chmod("/proc/self/net", 0555)); break;
 		CASE_TEST(chmod_self);        EXPECT_SYSER(has_proc, chmod("/proc/self", 0555), -1, EPERM); break;
 		CASE_TEST(chown_self);        EXPECT_SYSER(has_proc, chown("/proc/self", 0, 0), -1, EPERM); break;
-		CASE_TEST(chroot_root);       EXPECT_SYSZR(euid0, chroot("/")); break;
+		CASE_TEST(chroot_root);       EXPECT_SYSZR(is_root, chroot("/")); break;
 		CASE_TEST(chroot_blah);       EXPECT_SYSER(1, chroot("/proc/self/blah"), -1, ENOENT); break;
 		CASE_TEST(chroot_exe);        EXPECT_SYSER(has_proc, chroot("/proc/self/exe"), -1, ENOTDIR); break;
 		CASE_TEST(close_m1);          EXPECT_SYSER(1, close(-1), -1, EBADF); break;
@@ -607,7 +607,7 @@ int run_syscall(int min, int max)
 		CASE_TEST(ioctl_tiocinq);     EXPECT_SYSZR(1, ioctl(0, TIOCINQ, &tmp)); break;
 		CASE_TEST(link_root1);        EXPECT_SYSER(1, link("/", "/"), -1, EEXIST); break;
 		CASE_TEST(link_blah);         EXPECT_SYSER(1, link("/proc/self/blah", "/blah"), -1, ENOENT); break;
-		CASE_TEST(link_dir);          EXPECT_SYSER(euid0, link("/", "/blah"), -1, EPERM); break;
+		CASE_TEST(link_dir);          EXPECT_SYSER(is_root, link("/", "/blah"), -1, EPERM); break;
 		CASE_TEST(link_cross);        EXPECT_SYSER(has_proc, link("/proc/self/net", "/blah"), -1, EXDEV); break;
 		CASE_TEST(lseek_m1);          EXPECT_SYSER(1, lseek(-1, 0, SEEK_SET), -1, EBADF); break;
 		CASE_TEST(lseek_0);           EXPECT_SYSER(1, lseek(0, 0, SEEK_SET), -1, ESPIPE); break;
-- 
2.25.1

[PATCH v1 11/17] selftests/nolibc: prepare tmpfs and hugetlbfs

[Zhangjin Wu]

Let's make the /tmp and /hugetlb directories and mount tmpfs and
hugetlbfs respectively, they will be used to check the existing of tmpfs
and hugetlbfs.

We will also use tmpfs to let some test cases run without procfs.

Signed-off-by: Zhangjin Wu <falcon@tinylab.org>
---
 tools/testing/selftests/nolibc/nolibc-test.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/tools/testing/selftests/nolibc/nolibc-test.c b/tools/testing/selftests/nolibc/nolibc-test.c
index 69f59a395746..8b587961e46a 100644
--- a/tools/testing/selftests/nolibc/nolibc-test.c
+++ b/tools/testing/selftests/nolibc/nolibc-test.c
@@ -935,6 +935,18 @@ int prepare(void)
 		}
 	}
 
+	/* try to mount /tmp if not mounted. Silently fail otherwise */
+	if (stat("/tmp/.", &stat_buf) == 0 || mkdir("/tmp", 0755) == 0) {
+		if (mount("none", "/tmp", "tmpfs", 0, 0) != 0)
+			rmdir("/tmp");
+	}
+
+	/* try to mount /hugetlb if not mounted. Silently fail otherwise */
+	if (stat("/hugetlb/.", &stat_buf) == 0 || mkdir("/hugetlb", 0755) == 0) {
+		if (mount("none", "/hugetlb", "hugetlbfs", 0, 0) != 0)
+			rmdir("/hugetlb");
+	}
+
 	return 0;
 }
 
-- 
2.25.1

[PATCH v1 12/17] selftests/nolibc: rename chmod_net to chmod_good

[Zhangjin Wu]

When CONFIG_NET is not enabled, there would be no /proc/self/net, let's
use /tmp/blah in such case and rename chmod_net to chmod_good.

This allows to test chmod_good with tmpfs even when procfs is not there.

Signed-off-by: Zhangjin Wu <falcon@tinylab.org>
---
 tools/testing/selftests/nolibc/nolibc-test.c | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/nolibc/nolibc-test.c b/tools/testing/selftests/nolibc/nolibc-test.c
index 8b587961e46a..eca0070151b6 100644
--- a/tools/testing/selftests/nolibc/nolibc-test.c
+++ b/tools/testing/selftests/nolibc/nolibc-test.c
@@ -550,10 +550,22 @@ int run_syscall(int min, int max)
 	int ret = 0;
 	void *p1, *p2;
 	int has_gettid = 1;
+	int has_tmpdir = 0;
+	char *tmpdir = NULL;
 
 	/* <has_proc> indicates whether or not /proc is mounted */
 	has_proc = stat("/proc", &stat_buf) == 0;
 
+	/* <has_tmpdir> indicates whether or not /tmp/blah is there */
+	if (stat("/proc/self/net", &stat_buf) == 0) {
+		tmpdir = "/proc/self/net";
+		has_tmpdir = 1;
+	} else if (stat("/tmp/.", &stat_buf) == 0) {
+		tmpdir = "/tmp/blah";
+		if (mkdir(tmpdir, 0755) == 0)
+			has_tmpdir = 1;
+	}
+
 	/* this will be used to skip certain tests that can't be run unprivileged */
 	is_root = geteuid() == 0;
 
@@ -582,7 +594,7 @@ int run_syscall(int min, int max)
 		CASE_TEST(chdir_root);        EXPECT_SYSZR(1, chdir("/")); break;
 		CASE_TEST(chdir_dot);         EXPECT_SYSZR(1, chdir(".")); break;
 		CASE_TEST(chdir_blah);        EXPECT_SYSER(1, chdir("/blah"), -1, ENOENT); break;
-		CASE_TEST(chmod_net);         EXPECT_SYSZR(has_proc, chmod("/proc/self/net", 0555)); break;
+		CASE_TEST(chmod_good);        EXPECT_SYSZR(has_tmpdir, chmod(tmpdir, 0555)); break;
 		CASE_TEST(chmod_self);        EXPECT_SYSER(has_proc, chmod("/proc/self", 0555), -1, EPERM); break;
 		CASE_TEST(chown_self);        EXPECT_SYSER(has_proc, chown("/proc/self", 0, 0), -1, EPERM); break;
 		CASE_TEST(chroot_root);       EXPECT_SYSZR(is_root, chroot("/")); break;
-- 
2.25.1

[PATCH v1 13/17] selftests/nolibc: link_cross: support tmpfs

[Zhangjin Wu]

When CONFIG_NET is not enabled, there would be no /proc/self/net, use
/tmp/blah in such case.

This allows to test link_cross with tmpfs even when procfs is not there.

Signed-off-by: Zhangjin Wu <falcon@tinylab.org>
---
 tools/testing/selftests/nolibc/nolibc-test.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/testing/selftests/nolibc/nolibc-test.c b/tools/testing/selftests/nolibc/nolibc-test.c
index eca0070151b6..c900564219e8 100644
--- a/tools/testing/selftests/nolibc/nolibc-test.c
+++ b/tools/testing/selftests/nolibc/nolibc-test.c
@@ -620,7 +620,7 @@ int run_syscall(int min, int max)
 		CASE_TEST(link_root1);        EXPECT_SYSER(1, link("/", "/"), -1, EEXIST); break;
 		CASE_TEST(link_blah);         EXPECT_SYSER(1, link("/proc/self/blah", "/blah"), -1, ENOENT); break;
 		CASE_TEST(link_dir);          EXPECT_SYSER(is_root, link("/", "/blah"), -1, EPERM); break;
-		CASE_TEST(link_cross);        EXPECT_SYSER(has_proc, link("/proc/self/net", "/blah"), -1, EXDEV); break;
+		CASE_TEST(link_cross);        EXPECT_SYSER(has_tmpdir, link(tmpdir, "/blah"), -1, EXDEV); break;
 		CASE_TEST(lseek_m1);          EXPECT_SYSER(1, lseek(-1, 0, SEEK_SET), -1, EBADF); break;
 		CASE_TEST(lseek_0);           EXPECT_SYSER(1, lseek(0, 0, SEEK_SET), -1, ESPIPE); break;
 		CASE_TEST(mkdir_root);        EXPECT_SYSER(1, mkdir("/", 0755), -1, EEXIST); break;
-- 
2.25.1

[PATCH v1 14/17] selftests/nolibc: rename chroot_exe to chroot_file

[Zhangjin Wu]

When there is no procfs, let's use tmpfs and create a tmp file for
chroot_exe test.

Since chroot_exe is mainly testing the not directory case (ENOTDIR), so,
rename it to chroot_file may be better.

Signed-off-by: Zhangjin Wu <falcon@tinylab.org>
---
 tools/testing/selftests/nolibc/nolibc-test.c | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/nolibc/nolibc-test.c b/tools/testing/selftests/nolibc/nolibc-test.c
index c900564219e8..19e4ef5ce578 100644
--- a/tools/testing/selftests/nolibc/nolibc-test.c
+++ b/tools/testing/selftests/nolibc/nolibc-test.c
@@ -547,11 +547,14 @@ int run_syscall(int min, int max)
 	int has_proc;
 	int test;
 	int tmp;
+	int fd;
 	int ret = 0;
 	void *p1, *p2;
 	int has_gettid = 1;
 	int has_tmpdir = 0;
+	int has_tmpfile = 0;
 	char *tmpdir = NULL;
+	char *tmpfile = NULL;
 
 	/* <has_proc> indicates whether or not /proc is mounted */
 	has_proc = stat("/proc", &stat_buf) == 0;
@@ -560,10 +563,20 @@ int run_syscall(int min, int max)
 	if (stat("/proc/self/net", &stat_buf) == 0) {
 		tmpdir = "/proc/self/net";
 		has_tmpdir = 1;
+
+		tmpfile = "/proc/self/exe";
+		has_tmpfile = 1;
 	} else if (stat("/tmp/.", &stat_buf) == 0) {
 		tmpdir = "/tmp/blah";
 		if (mkdir(tmpdir, 0755) == 0)
 			has_tmpdir = 1;
+
+		tmpfile = "/tmp/dummy";
+		fd = open(tmpfile, O_CREAT);
+		if (fd != -1) {
+			has_tmpfile = 1;
+			close(fd);
+		}
 	}
 
 	/* this will be used to skip certain tests that can't be run unprivileged */
@@ -599,7 +612,7 @@ int run_syscall(int min, int max)
 		CASE_TEST(chown_self);        EXPECT_SYSER(has_proc, chown("/proc/self", 0, 0), -1, EPERM); break;
 		CASE_TEST(chroot_root);       EXPECT_SYSZR(is_root, chroot("/")); break;
 		CASE_TEST(chroot_blah);       EXPECT_SYSER(1, chroot("/proc/self/blah"), -1, ENOENT); break;
-		CASE_TEST(chroot_exe);        EXPECT_SYSER(has_proc, chroot("/proc/self/exe"), -1, ENOTDIR); break;
+		CASE_TEST(chroot_file);       EXPECT_SYSER(has_tmpfile, chroot(tmpfile), -1, ENOTDIR); break;
 		CASE_TEST(close_m1);          EXPECT_SYSER(1, close(-1), -1, EBADF); break;
 		CASE_TEST(close_dup);         EXPECT_SYSZR(1, close(dup(0))); break;
 		CASE_TEST(dup_0);             tmp = dup(0);  EXPECT_SYSNE(1, tmp, -1); close(tmp); break;
-- 
2.25.1

[PATCH v1 15/17] selftests/nolibc: vfprintf: silence memfd_create() warning

[Zhangjin Wu]

pass MFD_NOEXEC_SEAL flag to memfd_create() to silence this kernel
warning inserted in the middle of the whole test result:

    memfd_create() without MFD_EXEC nor MFD_NOEXEC_SEAL, pid=1

The mixed test result looks this:

    Running test 'vfprintf'
    0 emptymemfd_create() without MFD_EXEC nor MFD_NOEXEC_SEAL, pid=1 'init'
     "" = ""                                                  [OK]

From v6.2, MFD_NOEXEC_SEAL must be passed for the non-executable memfd.

Since MFD_NOEXEC_SEAL is a whole new flag, to avoid adding ugly #ifdef
macros, let's use magic number here directly.

Signed-off-by: Zhangjin Wu <falcon@tinylab.org>
---
 tools/testing/selftests/nolibc/nolibc-test.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/nolibc/nolibc-test.c b/tools/testing/selftests/nolibc/nolibc-test.c
index 19e4ef5ce578..8b1ce9911c5c 100644
--- a/tools/testing/selftests/nolibc/nolibc-test.c
+++ b/tools/testing/selftests/nolibc/nolibc-test.c
@@ -774,7 +774,17 @@ static int expect_vfprintf(int llen, size_t c, const char *expected, const char
 	FILE *memfile;
 	va_list args;
 
-	fd = memfd_create("vfprintf", 0);
+	/* silence warning for kernel >= v6.2:
+	 *
+	 *   "memfd_create() without MFD_EXEC nor MFD_NOEXEC_SEAL, pid=<pid>"
+	 *
+	 * try MFD_NOEXEC_SEAL (0x0008U) flag for kernels >= v6.2, error means
+	 * the kernel is too old and require old flags
+	 */
+	fd = memfd_create("vfprintf", 0x0008U);
+	if (fd == -1)
+		fd = memfd_create("vfprintf", 0);
+
 	if (fd == -1) {
 		pad_spc(llen, 64, "[FAIL]\n");
 		return 1;
-- 
2.25.1

[PATCH v1 16/17] selftests/nolibc: vfprintf: skip if neither tmpfs nor hugetlbfs

[Zhangjin Wu]

As fs/Kconfig shows, MEMFD_CREATE depends on TMPFS or HUGETLBFS:

    config MEMFD_CREATE
    	def_bool TMPFS || HUGETLBFS

Let's skip vfprintf test if they are not there.

The /tmp and /hugetlb directories have been created to mount tmpfs and
hugetlbfs respectively, if they are not enabled in kernel configuration,
neither /tmp nor /hugetlb will be created.

Signed-off-by: Zhangjin Wu <falcon@tinylab.org>
---
 tools/testing/selftests/nolibc/nolibc-test.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/tools/testing/selftests/nolibc/nolibc-test.c b/tools/testing/selftests/nolibc/nolibc-test.c
index 8b1ce9911c5c..85fa64746cde 100644
--- a/tools/testing/selftests/nolibc/nolibc-test.c
+++ b/tools/testing/selftests/nolibc/nolibc-test.c
@@ -769,11 +769,22 @@ int run_stdlib(int min, int max)
 
 static int expect_vfprintf(int llen, size_t c, const char *expected, const char *fmt, ...)
 {
+	struct stat stat_buf;
 	int ret, fd, w, r;
+	int tmpfs = 0, hugetlbfs = 0;
 	char buf[100];
 	FILE *memfile;
 	va_list args;
 
+	/* memfd_create depends on tmpfs or hugetlbfs */
+	tmpfs = stat("/tmp/.", &stat_buf) == 0;
+	hugetlbfs = stat("/hugetlb/.", &stat_buf) == 0;
+
+	if (!tmpfs && !hugetlbfs) {
+		pad_spc(llen, 64, "[SKIPPED]\n");
+		return 0;
+	}
+
 	/* silence warning for kernel >= v6.2:
 	 *
 	 *   "memfd_create() without MFD_EXEC nor MFD_NOEXEC_SEAL, pid=<pid>"
-- 
2.25.1

[PATCH v1 17/17] selftests/nolibc: vfprintf: support tmpfs and hugetlbfs

[Zhangjin Wu]

When CONFIG_TMPFS not enabled, kernel will use the ramfs based tmpfs
instead, but memfd_create doesn't work with such tmpfs, for this type of
tmpfs, let's use it instead of memfd_create.

At the same time, let's support hugetlbfs too if there is one.

Signed-off-by: Zhangjin Wu <falcon@tinylab.org>
---
 tools/testing/selftests/nolibc/nolibc-test.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/tools/testing/selftests/nolibc/nolibc-test.c b/tools/testing/selftests/nolibc/nolibc-test.c
index 85fa64746cde..bff72365a158 100644
--- a/tools/testing/selftests/nolibc/nolibc-test.c
+++ b/tools/testing/selftests/nolibc/nolibc-test.c
@@ -796,6 +796,14 @@ static int expect_vfprintf(int llen, size_t c, const char *expected, const char
 	if (fd == -1)
 		fd = memfd_create("vfprintf", 0);
 
+	/* memfd_create not work with ramfs based tmpfs, try tmpfs and hugetlbfs in order instead */
+	if (fd == -1) {
+		if (tmpfs)
+			fd = open("/tmp/vfprintf", O_CREAT | O_TRUNC | O_RDWR);
+		else if (hugetlbfs)
+			fd = open("/hugetlb/vfprintf", O_CREAT | O_TRUNC | O_RDWR);
+	}
+
 	if (fd == -1) {
 		pad_spc(llen, 64, "[FAIL]\n");
 		return 1;
-- 
2.25.1

Re: [PATCH v1 00/17] selftests/nolibc: allow run with minimal kernel config

[Thomas Weißschuh]

Hi Zhangjin,

some general comments for the whole series.

On 2023-06-21 20:52:30+0800, Zhangjin Wu wrote:
> Hi, Willy
> 
> This patchset mainly allows speed up the nolibc test with a minimal
> kernel config.
> 
> As the nolibc supported architectures become more and more, the 'run'
> test with DEFCONFIG may cost several hours, which is not friendly to
> develop testing and even for release testing, so, smaller kernel configs
> may be required, and firstly, we should let nolibc-test work with less
> kernel config options, this patchset aims to this goal.
> 
> This patchset mainly remove the dependency from procfs, tmpfs, net and
> memfd_create, many failures have been fixed up.
> 
> When CONFIG_TMPFS and CONFIG_SHMEM are disabled, kernel will provide a
> ramfs based tmpfs (mm/shmem.c), it will be used as a choice to fix up
> some failures and also allow skip less tests.

Did you look into how much this duplicates from the kernels already
existing "tinyconfig" and "kvm_guest.config" functionality?

And it would be interesting how much impact the enablement of procfs,
tmpfs, net and memfd_create has in constrast to the minimal
configuration.
It seems unfortunate to me to complicate the testsuite to handle such
uncommon scenarios.

> Besides, it also adds musl support, improves glibc support and fixes up
> a kernel cmdline passing use case.
> 
> This is based on the dev.2023.06.14a branch of linux-rcu [1], all of the
> supported architectures are tested (with local minimal configs, [5]
> pasted the one for i386) without failures:
> 
>            arch/board | result
>           ------------|------------
>       arm/vexpress-a9 | 138 test(s) passed, 1 skipped, 0 failed. See all results in /labs/linux-lab/logging/nolibc/arm-vexpress-a9-nolibc-test.log
>          aarch64/virt | 138 test(s) passed, 1 skipped, 0 failed. See all results in /labs/linux-lab/logging/nolibc/aarch64-virt-nolibc-test.log
>           ppc/g3beige | not supported
>               i386/pc | 136 test(s) passed, 3 skipped, 0 failed. See all results in /labs/linux-lab/logging/nolibc/i386-pc-nolibc-test.log
>             x86_64/pc | 138 test(s) passed, 1 skipped, 0 failed. See all results in /labs/linux-lab/logging/nolibc/x86_64-pc-nolibc-test.log
>          mipsel/malta | 138 test(s) passed, 1 skipped, 0 failed. See all results in /labs/linux-lab/logging/nolibc/mipsel-malta-nolibc-test.log
>      loongarch64/virt | 138 test(s) passed, 1 skipped, 0 failed. See all results in /labs/linux-lab/logging/nolibc/loongarch64-virt-nolibc-test.log
>          riscv64/virt | 136 test(s) passed, 3 skipped, 0 failed. See all results in /labs/linux-lab/logging/nolibc/riscv64-virt-nolibc-test.log
>          riscv32/virt | no test log found
> s390x/s390-ccw-virtio | 138 test(s) passed, 1 skipped, 0 failed. See all results in /labs/linux-lab/logging/nolibc/s390x-s390-ccw-virtio-nolibc-test.log
> 
> Notes:
>   * The skipped ones are -fstackprotector, chmod_self and chown_self
> 
>     The -fstackprotector skip is due to gcc version.
>     chmod_self and chmod_self skips are due to procfs not enabled
> 
>   * ppc/g3beige support is added locally, but not added in this patchset
> 
>     will send ppc support as a new patchset, it depends on v2 test
>     report patchset [3] and the v5 rv32 support, require changes on
>     Makefile
> 
>   * riscv32/virt support is still in review, see v5 rv32 support [4]
> 
> This patchset doesn't depends on any of my other nolibc patch series,
> but the new rmdir() routine added in this patchset may be requird to
> apply the __sysret() from our v4 syscall helper series [2] after that
> series being merged, currently, we use the old method to let it compile
> without any dependency.
> 
> Here explains all of the patches:
> 
> * selftests/nolibc: stat_fault: silence NULL argument warning with glibc
>   selftests/nolibc: gettid: restore for glibc and musl
>   selftests/nolibc: add _LARGEFILE64_SOURCE for musl
> 
>   The above 3 patches adds musl compile support and improve glibc support.
> 
>   It is able to build and run nolibc-test with musl libc now, but there
>   are some failures/skips due to the musl its own issues/requirements:
> 
>     $ sudo ./nolibc-test  | grep -E 'FAIL|SKIP'
>     8 sbrk = 1 ENOMEM                                               [FAIL]
>     9 brk = -1 ENOMEM                                               [FAIL]
>     46 limit_int_fast16_min = -2147483648                           [FAIL]
>     47 limit_int_fast16_max = 2147483647                            [FAIL]
>     49 limit_int_fast32_min = -2147483648                           [FAIL]
>     50 limit_int_fast32_max = 2147483647                            [FAIL]
>     0 -fstackprotector not supported                                [SKIPPED]
> 
>   musl disabled sbrk and brk for some conflicts with its malloc and the
>   fast version of int types are defined in 32bit, which differs from nolibc
>   and glibc. musl reserved the sbrk(0) to allow get current brk, we
>   added a test for this in the v4 __sysret() helper series [2].

We could add new macros

#define UINT_MAX(t) (~(t)0)
#define SINT_MAX(t) (((t)1 << (sizeof(t) * 8 - 2)) - (t)1 + ((t)1 << (sizeof(t) * 8 - 2)))

to get whatever is appropriate for the respective type.

> 
> * selftests/nolibc: fix up kernel parameters support
> 
>   kernel cmdline allows pass two types of parameters, one is without
>   '=', another is with '=', the first one is passed as init arguments,
>   the sencond one is passed as init environment variables.
> 
>   Our nolibc-test prefer arguments to environment variables, this not
>   work when users add such parameters in the kernel cmdline:
> 
>     noapic NOLIBC_TEST=syscall
> 
>   So, this patch will verify the setting from arguments at first, if it
>   is no valid, will try the environment variables instead.

This would be much simpler as:

test = getenv("NOLIBC_TEST");
if (!test)
        test = argv[1];

It changes the semantics a bit, but it doesn't seem to be an issue.
(Maybe gated behind getpid() == 1).

> * selftests/nolibc: stat_timestamps: remove procfs dependency
> 
>   Use '/' instead of /proc/self, or we can add a 'has_proc' condition
>   for this test case, but it is not that necessary to skip the whole
>   stat_timestamps tests for such a subtest binding to /proc/self.
> 
>   Welcome suggestion from Thomas.

As above, I think the impact of depending on CONFIG_PROC_FS is
justifiable.

The usage of /proc/self was actually intentional.
This file has a timestamp of the start of the referenced process.
So each invocation of nolibc-test tests a new timestamp.

In contrast if nolibc-test is invocated from a prebaked filesystem the
timestamp of "/" will always be fixed, reducing the chance to find
errors.

> * tools/nolibc: add rmdir() support
>   selftests/nolibc: add a new rmdir() test case
> 
>   rmdir() routine and test case are added for the coming requirement.
> 
>   Note, if the __sysret() patchset [2] is applied before us, this patch
>   should be rebased on it and apply the __sysret() helper.
> 
> * selftests/nolibc: fix up failures when there is no procfs
> 
>   call rmdir() to remove /proc completely to rework the checking of
>   /proc, before, the existing of /proc not means the procfs is really
>   mounted.
> 
> * selftests/nolibc: rename proc variable to has_proc
>   selftests/nolibc: rename euid0 variable to is_root
> 
>   align with the has_gettid, has_xxx variables.
> 
> * selftests/nolibc: prepare tmpfs and hugetlbfs
>   selftests/nolibc: rename chmod_net to chmod_good
>   selftests/nolibc: link_cross: support tmpfs
>   selftests/nolibc: rename chroot_exe to chroot_file
> 
>   use file from /tmp instead of file from /proc when there is no procfs
>   this avoid skipping the chmod_net, link_cross, chroot_exe tests
> 
> * selftests/nolibc: vfprintf: silence memfd_create() warning
>   selftests/nolibc: vfprintf: skip if neither tmpfs nor hugetlbfs
>   selftests/nolibc: vfprintf: support tmpfs and hugetlbfs
> 
>   memfd_create from kernel >= v6.2 forcely warn on missing
>   MFD_NOEXEC_SEAL flag, the first one silence it with such flag, for
>   older kernels, use 0 flag as before.

Given this is only a problem when nolibc-test is PID1 and printing to
the system console, we could also just disable warnings on the system
console through syscall() or /proc/sys/kernel/printk.

It would also avoid cluttering the tests for limited gain.

>   since memfd_create() depends on TMPFS or HUGETLBFS, the second one
>   skip the whole vfprintf instead of simply fail if memfd_create() not
>   work.
> 
>   the 3rd one futher try the ramfs based tmpfs even when memfd_create()
>   not work.
> 
> At last, let's simply discuss about the configs, I have prepared minimal
> configs for all of the nolibc supported architectures but not sure where
> should we put them, what about tools/testing/selftests/nolibc/configs ?
> 
> Thanks!
> 
> Best regards,
> Zhangjin
> ---
> 
> [1]: https://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git/
> [2]: https://lore.kernel.org/linux-riscv/cover.1687187451.git.falcon@tinylab.org/
> [3]: https://lore.kernel.org/lkml/cover.1687156559.git.falcon@tinylab.org/
> [4]: https://lore.kernel.org/linux-riscv/cover.1687176996.git.falcon@tinylab.org/
> [5]: https://pastebin.com/5jq0Vxbz 
> 
> Zhangjin Wu (17):
>   selftests/nolibc: stat_fault: silence NULL argument warning with glibc
>   selftests/nolibc: gettid: restore for glibc and musl
>   selftests/nolibc: add _LARGEFILE64_SOURCE for musl
>   selftests/nolibc: fix up kernel parameters support
>   selftests/nolibc: stat_timestamps: remove procfs dependency
>   tools/nolibc: add rmdir() support
>   selftests/nolibc: add a new rmdir() test case
>   selftests/nolibc: fix up failures when there is no procfs
>   selftests/nolibc: rename proc variable to has_proc
>   selftests/nolibc: rename euid0 variable to is_root
>   selftests/nolibc: prepare tmpfs and hugetlbfs
>   selftests/nolibc: rename chmod_net to chmod_good
>   selftests/nolibc: link_cross: support tmpfs
>   selftests/nolibc: rename chroot_exe to chroot_file
>   selftests/nolibc: vfprintf: silence memfd_create() warning
>   selftests/nolibc: vfprintf: skip if neither tmpfs nor hugetlbfs
>   selftests/nolibc: vfprintf: support tmpfs and hugetlbfs
> 
>  tools/include/nolibc/sys.h                   |  28 ++++
>  tools/testing/selftests/nolibc/nolibc-test.c | 132 +++++++++++++++----
>  2 files changed, 138 insertions(+), 22 deletions(-)
> 
> -- 
> 2.25.1
> 

Re: [PATCH v1 00/17] selftests/nolibc: allow run with minimal kernel config

[Zhangjin Wu]

Hi, Thomas

> Hi Zhangjin,
> 
> some general comments for the whole series.
> 
> On 2023-06-21 20:52:30+0800, Zhangjin Wu wrote:
> > Hi, Willy
> > 
> > This patchset mainly allows speed up the nolibc test with a minimal
> > kernel config.
> > 
> > As the nolibc supported architectures become more and more, the 'run'
> > test with DEFCONFIG may cost several hours, which is not friendly to
> > develop testing and even for release testing, so, smaller kernel configs
> > may be required, and firstly, we should let nolibc-test work with less
> > kernel config options, this patchset aims to this goal.
> > 
> > This patchset mainly remove the dependency from procfs, tmpfs, net and
> > memfd_create, many failures have been fixed up.
> > 
> > When CONFIG_TMPFS and CONFIG_SHMEM are disabled, kernel will provide a
> > ramfs based tmpfs (mm/shmem.c), it will be used as a choice to fix up
> > some failures and also allow skip less tests.
> 
> Did you look into how much this duplicates from the kernels already
> existing "tinyconfig" and "kvm_guest.config" functionality?
>

Very good question and suggestion, thanks. it is just between tinyconfig
and kvm_guest.config, the former is not enough, the later provides more.
tinyconfig may be a very good base for us.

Just tested some architectures, based on tinyconfig, seems we only need
to enable very few options, for example, TTY, PRINTK, CONSOLE related
and target virtual board related options, but it requires more time to
just list the required options.

The 'minimal' ones I have prepared were shrunk from the 'defconfig', now
we need to add options from 'tinyconfig', with allnoconfig, it should be
smaller and therefore faster ;-)

Based on my local powerpc porting, I have prepared some changes like
this:

    # extra kernel configs by architecture
    EXTCONFIG_powerpc    = --enable SERIAL_PMACZILOG --enable CONFIG_SERIAL_PMACZILOG_CONSOLE
    EXTCONFIG            = --set-str CONFIG_INITRAMFS_SOURCE $(CURDIR)/initramfs $(EXTCONFIG_$(ARCH))

    ...

    menuconfig:
            $(Q)$(MAKE) -C $(srctree) ARCH=$(KARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) menuconfig
    
    extconfig:
            $(Q)$(srctree)/scripts/config --file $(srctree)/.config $(EXTCONFIG)
            $(Q)$(MAKE) -C $(srctree) ARCH=$(KARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) olddefconfig
    
    kernel: initramfs extconfig
            $(Q)$(MAKE) -C $(srctree) ARCH=$(KARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) $(IMAGE_NAME)


'menuconfig' is added for development, for example, find why something not work
and add the missing options.

'extconfig' is added to enable additional options (before, based on
defconfig) to let nolibc-test happy (for powerpc, add missing console
options which has been added as modules in default config).

Based on your suggestion, this may be a good new target:

    tinyconfig:
            $(Q)$(MAKE) -C $(srctree) ARCH=$(KARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) mrproper tinyconfig prepare

And this one, use 'allnoconfig' instead of 'olddefconfig':

    extconfig:
            $(Q)$(srctree)/scripts/config --file $(srctree)/.config $(EXTCONFIG)
            $(Q)$(MAKE) -C $(srctree) ARCH=$(KARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) KCONFIG_ALLCONFIG=$(srctree)/.config allnoconfig

So, the new 'tinyconfig' may function as the smallest test environment,
for faster compile and as a boundary test of nolibc-test itself.

But again, still need time to list the minimally required options, if they are
few, listing them in the EXTCONFIG_<ARCH> line may be acceptable, but if the
options are 'huge', standalone nolibc.config may be required, let's wait for
one or two days.

> And it would be interesting how much impact the enablement of procfs,
> tmpfs, net and memfd_create has in constrast to the minimal
> configuration.

For the test speed (mainly kernel compile) itself, when for one architecture on
a very good test host, the time cost increment is very little (see below), but it
does save some, especially when for lots of architectures ;-)

Comparing the rv64 testing speed on a Ubuntu 20.04 over '4G Mem + 4 Cores of
i7-8550U CPU @ 1.80GHz' Virtual Machne, this time include:

* nolibc-test sysroot install + build
* kernel config + build
* qemu boot with opensbi + u-boot + kernel v6.4-rcx

Testing results:

  'minimal':

      arch/board | result
     ------------|------------
    riscv64/virt | 136 test(s) passed, 3 skipped, 0 failed. See all results in /labs/linux-lab/logging/nolibc/riscv64-virt-nolibc-test.log
    
    LOG: see all results for all boards in /labs/linux-lab/logging/nolibc/nolibc-test.log
    
    
    real	1m57.395s
    user	4m50.002s
    sys	1m0.866s

  'minimal' + procfs, net, shmem/tmpfs, devtmpfs/devmtmpfs_mount ...:

      arch/board | result
     ------------|------------
    riscv64/virt | 138 test(s) passed, 1 skipped, 0 failed. See all results in /labs/linux-lab/logging/nolibc/riscv64-virt-nolibc-test.log
    
    LOG: see all results for all boards in /labs/linux-lab/logging/nolibc/nolibc-test.log
    
    
    real	2m17.812s
    user	6m4.695s
    sys	1m7.061s

It did save 20s (~17.1%) for us, not too much, but really faster. 

> It seems unfortunate to me to complicate the testsuite to handle such
> uncommon scenarios.

Yeah, such a config is not common, but as explained above, beside the compile
speedup improvement, it is really a good boundary test environment for
nolibc-test itself to make sure it work (no failure, less skips) at an
extremely worst-case scene, although our changes looks many, but every one is
as simple as CLOC ;-)

And that also means, nolibc is able to run with a very 'tiny' kernel
config and users could reuse our config fragments and add their own for
their embedded devices.

> 
> > Besides, it also adds musl support, improves glibc support and fixes up
> > a kernel cmdline passing use case.
> > 
> > This is based on the dev.2023.06.14a branch of linux-rcu [1], all of the
> > supported architectures are tested (with local minimal configs, [5]
> > pasted the one for i386) without failures:
> > 
> >            arch/board | result
> >           ------------|------------
> >       arm/vexpress-a9 | 138 test(s) passed, 1 skipped, 0 failed. See all results in /labs/linux-lab/logging/nolibc/arm-vexpress-a9-nolibc-test.log
> >          aarch64/virt | 138 test(s) passed, 1 skipped, 0 failed. See all results in /labs/linux-lab/logging/nolibc/aarch64-virt-nolibc-test.log
> >           ppc/g3beige | not supported
> >               i386/pc | 136 test(s) passed, 3 skipped, 0 failed. See all results in /labs/linux-lab/logging/nolibc/i386-pc-nolibc-test.log
> >             x86_64/pc | 138 test(s) passed, 1 skipped, 0 failed. See all results in /labs/linux-lab/logging/nolibc/x86_64-pc-nolibc-test.log
> >          mipsel/malta | 138 test(s) passed, 1 skipped, 0 failed. See all results in /labs/linux-lab/logging/nolibc/mipsel-malta-nolibc-test.log
> >      loongarch64/virt | 138 test(s) passed, 1 skipped, 0 failed. See all results in /labs/linux-lab/logging/nolibc/loongarch64-virt-nolibc-test.log
> >          riscv64/virt | 136 test(s) passed, 3 skipped, 0 failed. See all results in /labs/linux-lab/logging/nolibc/riscv64-virt-nolibc-test.log
> >          riscv32/virt | no test log found
> > s390x/s390-ccw-virtio | 138 test(s) passed, 1 skipped, 0 failed. See all results in /labs/linux-lab/logging/nolibc/s390x-s390-ccw-virtio-nolibc-test.log
> > 

(snipped)

> >   It is able to build and run nolibc-test with musl libc now, but there
> >   are some failures/skips due to the musl its own issues/requirements:
> > 
> >     $ sudo ./nolibc-test  | grep -E 'FAIL|SKIP'
> >     8 sbrk = 1 ENOMEM                                               [FAIL]
> >     9 brk = -1 ENOMEM                                               [FAIL]
> >     46 limit_int_fast16_min = -2147483648                           [FAIL]
> >     47 limit_int_fast16_max = 2147483647                            [FAIL]
> >     49 limit_int_fast32_min = -2147483648                           [FAIL]
> >     50 limit_int_fast32_max = 2147483647                            [FAIL]
> >     0 -fstackprotector not supported                                [SKIPPED]
> > 
> >   musl disabled sbrk and brk for some conflicts with its malloc and the
> >   fast version of int types are defined in 32bit, which differs from nolibc
> >   and glibc. musl reserved the sbrk(0) to allow get current brk, we
> >   added a test for this in the v4 __sysret() helper series [2].
> 
> We could add new macros
> 
> #define UINT_MAX(t) (~(t)0)
> #define SINT_MAX(t) (((t)1 << (sizeof(t) * 8 - 2)) - (t)1 + ((t)1 << (sizeof(t) * 8 - 2)))
> 
> to get whatever is appropriate for the respective type.
>

They work perfectly, thanks:

    /* for fast int test cases in stdlib test, musl use 32bit fast int */
    #undef UINT_MAX
    #define UINT_MAX(t)    (~(t)0)
    #undef SINT_MAX
    #define SINT_MAX(t)    (((t)1 << (sizeof(t) * 8 - 2)) - (t)1 + ((t)1 << (sizeof(t) * 8 - 2)))
    #undef SINT_MIN
    #define SINT_MIN(t)    (-SINT_MAX(t) - 1)

    ...

    CASE_TEST(limit_int_fast16_min);    EXPECT_EQ(1, INT_FAST16_MIN,   (int_fast16_t)    SINT_MIN(int_fast16_t)); break;
    CASE_TEST(limit_int_fast16_max);    EXPECT_EQ(1, INT_FAST16_MAX,   (int_fast16_t)    SINT_MAX(int_fast16_t)); break;
    CASE_TEST(limit_uint_fast16_max);   EXPECT_EQ(1, UINT_FAST16_MAX,  (uint_fast16_t)   UINT_MAX(uint_fast16_t)); break;
    CASE_TEST(limit_int_fast32_min);    EXPECT_EQ(1, INT_FAST32_MIN,   (int_fast32_t)    SINT_MIN(int_fast32_t)); break;
    CASE_TEST(limit_int_fast32_max);    EXPECT_EQ(1, INT_FAST32_MAX,   (int_fast32_t)    SINT_MAX(int_fast32_t)); break;
    CASE_TEST(limit_uint_fast32_max);   EXPECT_EQ(1, UINT_FAST32_MAX,  (uint_fast32_t)   UINT_MAX(uint_fast32_t)); break;

To avoid overriding the existing macros, perhaps we should add something
like UINT_TYPE_MAX(t), SINT_TYPE_MAX(t) and SINT_TYPE_MIN(t) ?

> > 
> > * selftests/nolibc: fix up kernel parameters support
> > 
> >   kernel cmdline allows pass two types of parameters, one is without
> >   '=', another is with '=', the first one is passed as init arguments,
> >   the sencond one is passed as init environment variables.
> > 
> >   Our nolibc-test prefer arguments to environment variables, this not
> >   work when users add such parameters in the kernel cmdline:
> > 
> >     noapic NOLIBC_TEST=syscall
> > 
> >   So, this patch will verify the setting from arguments at first, if it
> >   is no valid, will try the environment variables instead.
> 
> This would be much simpler as:
> 
> test = getenv("NOLIBC_TEST");
> if (!test)
>         test = argv[1];
>
> It changes the semantics a bit, but it doesn't seem to be an issue.
> (Maybe gated behind getpid() == 1).

Cool suggestion, it looks really better:

	if (getpid() == 1) {
		prepare();
		
		/* kernel cmdline may pass: "noapic NOLIBC_TEST=syscall",
                 * to init program:
		 *
		 *   "noapic" as arguments,
		 *   "NOLIBC_TEST=syscall" as environment variables,
                 *
		 * to avoid getting null test in this case, parsing
		 * environment variables at first.
		 */
		test = getenv("NOLIBC_TEST");
		if (!test)
			test = argv[1];
	} else {
		/* for normal nolibc-test program, prefer arguments */
		test = argv[1];
		if (!test)
			test = getenv("NOLIBC_TEST");
	}

> 
> > * selftests/nolibc: stat_timestamps: remove procfs dependency
> > 
> >   Use '/' instead of /proc/self, or we can add a 'has_proc' condition
> >   for this test case, but it is not that necessary to skip the whole
> >   stat_timestamps tests for such a subtest binding to /proc/self.
> > 
> >   Welcome suggestion from Thomas.
> 
> As above, I think the impact of depending on CONFIG_PROC_FS is
> justifiable.
> 
> The usage of /proc/self was actually intentional.
> This file has a timestamp of the start of the referenced process.
> So each invocation of nolibc-test tests a new timestamp.
>
> In contrast if nolibc-test is invocated from a prebaked filesystem the
> timestamp of "/" will always be fixed, reducing the chance to find
> errors.

Yeah, it is resonable to reserve this as /proc/self, for a balance, to
avoid failure in the worst-case scene, let's add a 'has_proc' condition
there instead, just like the other /proc/self based test cases do.

> 
> > * tools/nolibc: add rmdir() support
> >   selftests/nolibc: add a new rmdir() test case
> > 
(snipped)
> > 
> > * selftests/nolibc: vfprintf: silence memfd_create() warning
> >   selftests/nolibc: vfprintf: skip if neither tmpfs nor hugetlbfs
> >   selftests/nolibc: vfprintf: support tmpfs and hugetlbfs
> > 
> >   memfd_create from kernel >= v6.2 forcely warn on missing
> >   MFD_NOEXEC_SEAL flag, the first one silence it with such flag, for
> >   older kernels, use 0 flag as before.
> 
> Given this is only a problem when nolibc-test is PID1 and printing to
> the system console, we could also just disable warnings on the system
> console through syscall() or /proc/sys/kernel/printk.

Ok, I did think about disabling console for this call, but I was worried about
the requirement of root (euid0) to do so, limiting it under PID1 may solve the
root permission issue, but still need to find the right syscall to avoid the
dependency of /proc/sys/kernel/printk, otherwise, to avoid failure for !procfs,
the whole vfprintf will be skipped for such a warning, to be honest, it looks
not a good direction.

> 
> It would also avoid cluttering the tests for limited gain.
>

Hmm, if consider the more code lines about disabling/enabling console and the
dependency of /proc/sys/kernel/printk, I do prefer current change.

But I'm also interested in how the other applications developers to treat this
warning, from the new kernel version side, we should use the latest non
executable flags for security, but to let applications work with old kernels,
we must support old flags, checking the kernel versions may be another choice. 

Perhaps it's time for us to add the 'uname()' for nolibc, but the
version comparing may be not that easy when we are in c context ;-)

  https://www.man7.org/linux/man-pages/man2/uname.2.html

So, the current method may be still a 'balanced' solution, it tries supported
flags from new kernel to old kernel to get a better and working memfd_create()
without the version checking, is this cleaner?

	int i;
	/* kernel >= v6.2 require MFD_NOEXEC_SEAL (0x0008U), but older ones not support this flag */
	unsigned int flags[2] = {0x0008U, 0};

	for (i = 0; i < 2; i ++) {
		/* try supported flags from new kernels to old kernels */
		fd = memfd_create("vfprintf", flags[i]);
		if (fd != -1)
			break;
	}

	if (fd == -1) {
		...
	}

Thanks very much.

Best regards,
Zhangjin

> >   since memfd_create() depends on TMPFS or HUGETLBFS, the second one
> >   skip the whole vfprintf instead of simply fail if memfd_create() not
> >   work.
> > 
> >   the 3rd one futher try the ramfs based tmpfs even when memfd_create()
> >   not work.
> > 
> > At last, let's simply discuss about the configs, I have prepared minimal
> > configs for all of the nolibc supported architectures but not sure where
> > should we put them, what about tools/testing/selftests/nolibc/configs ?
> > 
> > Thanks!
> > 
> > Best regards,
> > Zhangjin
> > ---
> > 
> > [1]: https://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git/
> > [2]: https://lore.kernel.org/linux-riscv/cover.1687187451.git.falcon@tinylab.org/
> > [3]: https://lore.kernel.org/lkml/cover.1687156559.git.falcon@tinylab.org/
> > [4]: https://lore.kernel.org/linux-riscv/cover.1687176996.git.falcon@tinylab.org/
> > [5]: https://pastebin.com/5jq0Vxbz 
> > 
> > Zhangjin Wu (17):
> >   selftests/nolibc: stat_fault: silence NULL argument warning with glibc
> >   selftests/nolibc: gettid: restore for glibc and musl
> >   selftests/nolibc: add _LARGEFILE64_SOURCE for musl
> >   selftests/nolibc: fix up kernel parameters support
> >   selftests/nolibc: stat_timestamps: remove procfs dependency
> >   tools/nolibc: add rmdir() support
> >   selftests/nolibc: add a new rmdir() test case
> >   selftests/nolibc: fix up failures when there is no procfs
> >   selftests/nolibc: rename proc variable to has_proc
> >   selftests/nolibc: rename euid0 variable to is_root
> >   selftests/nolibc: prepare tmpfs and hugetlbfs
> >   selftests/nolibc: rename chmod_net to chmod_good
> >   selftests/nolibc: link_cross: support tmpfs
> >   selftests/nolibc: rename chroot_exe to chroot_file
> >   selftests/nolibc: vfprintf: silence memfd_create() warning
> >   selftests/nolibc: vfprintf: skip if neither tmpfs nor hugetlbfs
> >   selftests/nolibc: vfprintf: support tmpfs and hugetlbfs
> > 
> >  tools/include/nolibc/sys.h                   |  28 ++++
> >  tools/testing/selftests/nolibc/nolibc-test.c | 132 +++++++++++++++----
> >  2 files changed, 138 insertions(+), 22 deletions(-)
> > 
> > -- 
> > 2.25.1
> > 

Re: [PATCH v1 00/17] selftests/nolibc: allow run with minimal kernel config

[Thomas Weißschuh]

On 2023-06-23 02:45:59+0800, Zhangjin Wu wrote:
> > some general comments for the whole series.
> > 
> > On 2023-06-21 20:52:30+0800, Zhangjin Wu wrote:
> > > Hi, Willy
> > > 
> > > This patchset mainly allows speed up the nolibc test with a minimal
> > > kernel config.
> > > 
> > > As the nolibc supported architectures become more and more, the 'run'
> > > test with DEFCONFIG may cost several hours, which is not friendly to
> > > develop testing and even for release testing, so, smaller kernel configs
> > > may be required, and firstly, we should let nolibc-test work with less
> > > kernel config options, this patchset aims to this goal.
> > > 
> > > This patchset mainly remove the dependency from procfs, tmpfs, net and
> > > memfd_create, many failures have been fixed up.
> > > 
> > > When CONFIG_TMPFS and CONFIG_SHMEM are disabled, kernel will provide a
> > > ramfs based tmpfs (mm/shmem.c), it will be used as a choice to fix up
> > > some failures and also allow skip less tests.
> > 
> > Did you look into how much this duplicates from the kernels already
> > existing "tinyconfig" and "kvm_guest.config" functionality?
> >
> 
> Very good question and suggestion, thanks. it is just between tinyconfig
> and kvm_guest.config, the former is not enough, the later provides more.
> tinyconfig may be a very good base for us.
> 
> Just tested some architectures, based on tinyconfig, seems we only need
> to enable very few options, for example, TTY, PRINTK, CONSOLE related
> and target virtual board related options, but it requires more time to
> just list the required options.
> 
> The 'minimal' ones I have prepared were shrunk from the 'defconfig', now
> we need to add options from 'tinyconfig', with allnoconfig, it should be
> smaller and therefore faster ;-)
> 
> Based on my local powerpc porting, I have prepared some changes like
> this:
> 
>     # extra kernel configs by architecture
>     EXTCONFIG_powerpc    = --enable SERIAL_PMACZILOG --enable CONFIG_SERIAL_PMACZILOG_CONSOLE
>     EXTCONFIG            = --set-str CONFIG_INITRAMFS_SOURCE $(CURDIR)/initramfs $(EXTCONFIG_$(ARCH))
> 
>     ...
>also  
>     menuconfig:
>             $(Q)$(MAKE) -C $(srctree) ARCH=$(KARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) menuconfig
>     
>     extconfig:
>             $(Q)$(srctree)/scripts/config --file $(srctree)/.config $(EXTCONFIG)
>             $(Q)$(MAKE) -C $(srctree) ARCH=$(KARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) olddefconfig
>     
>     kernel: initramfs extconfig
>             $(Q)$(MAKE) -C $(srctree) ARCH=$(KARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) $(IMAGE_NAME)
> 
> 
> 'menuconfig' is added for development, for example, find why something not work
> and add the missing options.
> 
> 'extconfig' is added to enable additional options (before, based on
> defconfig) to let nolibc-test happy (for powerpc, add missing console
> options which has been added as modules in default config).
> 
> Based on your suggestion, this may be a good new target:
> 
>     tinyconfig:
>             $(Q)$(MAKE) -C $(srctree) ARCH=$(KARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) mrproper tinyconfig prepare
> 
> And this one, use 'allnoconfig' instead of 'olddefconfig':
> 
>     extconfig:
>             $(Q)$(srctree)/scripts/config --file $(srctree)/.config $(EXTCONFIG)
>             $(Q)$(MAKE) -C $(srctree) ARCH=$(KARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) KCONFIG_ALLCONFIG=$(srctree)/.config allnoconfig
> 
> So, the new 'tinyconfig' may function as the smallest test environment,
> for faster compile and as a boundary test of nolibc-test itself.
> 
> But again, still need time to list the minimally required options, if they are
> few, listing them in the EXTCONFIG_<ARCH> line may be acceptable, but if the
> options are 'huge', standalone nolibc.config may be required, let's wait for
> one or two days.

FYI there are many more tests in tools/testing/selftests/ that need
custom configs to run. Maybe we can reuse some of their configuration
machinery.
(And qemu machinery maybe)

> > And it would be interesting how much impact the enablement of procfs,
> > tmpfs, net and memfd_create has in constrast to the minimal
> > configuration.
> 
> For the test speed (mainly kernel compile) itself, when for one architecture on
> a very good test host, the time cost increment is very little (see below), but it
> does save some, especially when for lots of architectures ;-)
> 
> Comparing the rv64 testing speed on a Ubuntu 20.04 over '4G Mem + 4 Cores of
> i7-8550U CPU @ 1.80GHz' Virtual Machne, this time include:
> 
> * nolibc-test sysroot install + build
> * kernel config + build
> * qemu boot with opensbi + u-boot + kernel v6.4-rcx
> 
> Testing results:
> 
>   'minimal':
> 
>       arch/board | result
>      ------------|------------
>     riscv64/virt | 136 test(s) passed, 3 skipped, 0 failed. See all results in /labs/linux-lab/logging/nolibc/riscv64-virt-nolibc-test.log
>     
>     LOG: see all results for all boards in /labs/linux-lab/logging/nolibc/nolibc-test.log
>     
>     
>     real	1m57.395s
>     user	4m50.002s
>     sys	1m0.866s
> 
>   'minimal' + procfs, net, shmem/tmpfs, devtmpfs/devmtmpfs_mount ...:
> 
>       arch/board | result
>      ------------|------------
>     riscv64/virt | 138 test(s) passed, 1 skipped, 0 failed. See all results in /labs/linux-lab/logging/nolibc/riscv64-virt-nolibc-test.log
>     
>     LOG: see all results for all boards in /labs/linux-lab/logging/nolibc/nolibc-test.log
>     
>     
>     real	2m17.812s
>     user	6m4.695s
>     sys	1m7.061s
> 
> It did save 20s (~17.1%) for us, not too much, but really faster. 
> 
> > It seems unfortunate to me to complicate the testsuite to handle such
> > uncommon scenarios.
> 
> Yeah, such a config is not common, but as explained above, beside the compile
> speedup improvement, it is really a good boundary test environment for
> nolibc-test itself to make sure it work (no failure, less skips) at an
> extremely worst-case scene, although our changes looks many, but every one is
> as simple as CLOC ;-)
> 
> And that also means, nolibc is able to run with a very 'tiny' kernel
> config and users could reuse our config fragments and add their own for
> their embedded devices.

It would mean that nolibc-test is able to run on *really* trimmed down
systems, which seems of limited use.
If the testsuite has more dependencies it would not stop nolibc itself
to run on them.

As for the CONFIG_NET dependency, which I would guess is one of the more
expensive configs to enable:

link_cross can be easily adapted to instead use /proc/self.

chmod_net relies on /proc/$PID/net accepting chmod().
It is the only file in /proc/$PID/ that works that way.

Maybe its a kernel bug anyways and we shouldn't rely on it anyways?
I'm taking a look.

> > > Besides, it also adds musl support, improves glibc support and fixes up
> > > a kernel cmdline passing use case.
> > > 
> > > This is based on the dev.2023.06.14a branch of linux-rcu [1], all of the
> > > supported architectures are tested (with local minimal configs, [5]
> > > pasted the one for i386) without failures:
> > > 
> > >            arch/board | result
> > >           ------------|------------
> > >       arm/vexpress-a9 | 138 test(s) passed, 1 skipped, 0 failed. See all results in /labs/linux-lab/logging/nolibc/arm-vexpress-a9-nolibc-test.log
> > >          aarch64/virt | 138 test(s) passed, 1 skipped, 0 failed. See all results in /labs/linux-lab/logging/nolibc/aarch64-virt-nolibc-test.log
> > >           ppc/g3beige | not supported
> > >               i386/pc | 136 test(s) passed, 3 skipped, 0 failed. See all results in /labs/linux-lab/logging/nolibc/i386-pc-nolibc-test.log
> > >             x86_64/pc | 138 test(s) passed, 1 skipped, 0 failed. See all results in /labs/linux-lab/logging/nolibc/x86_64-pc-nolibc-test.log
> > >          mipsel/malta | 138 test(s) passed, 1 skipped, 0 failed. See all results in /labs/linux-lab/logging/nolibc/mipsel-malta-nolibc-test.log
> > >      loongarch64/virt | 138 test(s) passed, 1 skipped, 0 failed. See all results in /labs/linux-lab/logging/nolibc/loongarch64-virt-nolibc-test.log
> > >          riscv64/virt | 136 test(s) passed, 3 skipped, 0 failed. See all results in /labs/linux-lab/logging/nolibc/riscv64-virt-nolibc-test.log
> > >          riscv32/virt | no test log found
> > > s390x/s390-ccw-virtio | 138 test(s) passed, 1 skipped, 0 failed. See all results in /labs/linux-lab/logging/nolibc/s390x-s390-ccw-virtio-nolibc-test.log
> > > 
> 
> (snipped)
> 
> > >   It is able to build and run nolibc-test with musl libc now, but there
> > >   are some failures/skips due to the musl its own issues/requirements:
> > > 
> > >     $ sudo ./nolibc-test  | grep -E 'FAIL|SKIP'
> > >     8 sbrk = 1 ENOMEM                                               [FAIL]
> > >     9 brk = -1 ENOMEM                                               [FAIL]
> > >     46 limit_int_fast16_min = -2147483648                           [FAIL]
> > >     47 limit_int_fast16_max = 2147483647                            [FAIL]
> > >     49 limit_int_fast32_min = -2147483648                           [FAIL]
> > >     50 limit_int_fast32_max = 2147483647                            [FAIL]
> > >     0 -fstackprotector not supported                                [SKIPPED]
> > > 
> > >   musl disabled sbrk and brk for some conflicts with its malloc and the
> > >   fast version of int types are defined in 32bit, which differs from nolibc
> > >   and glibc. musl reserved the sbrk(0) to allow get current brk, we
> > >   added a test for this in the v4 __sysret() helper series [2].
> > 
> > We could add new macros
> > 
> > #define UINT_MAX(t) (~(t)0)
> > #define SINT_MAX(t) (((t)1 << (sizeof(t) * 8 - 2)) - (t)1 + ((t)1 << (sizeof(t) * 8 - 2)))
> > 
> > to get whatever is appropriate for the respective type.
> >
> 
> They work perfectly, thanks:
> 
>     /* for fast int test cases in stdlib test, musl use 32bit fast int */
>     #undef UINT_MAX
>     #define UINT_MAX(t)    (~(t)0)
>     #undef SINT_MAX
>     #define SINT_MAX(t)    (((t)1 << (sizeof(t) * 8 - 2)) - (t)1 + ((t)1 << (sizeof(t) * 8 - 2)))
>     #undef SINT_MIN
>     #define SINT_MIN(t)    (-SINT_MAX(t) - 1)
> 
>     ...
> 
>     CASE_TEST(limit_int_fast16_min);    EXPECT_EQ(1, INT_FAST16_MIN,   (int_fast16_t)    SINT_MIN(int_fast16_t)); break;
>     CASE_TEST(limit_int_fast16_max);    EXPECT_EQ(1, INT_FAST16_MAX,   (int_fast16_t)    SINT_MAX(int_fast16_t)); break;
>     CASE_TEST(limit_uint_fast16_max);   EXPECT_EQ(1, UINT_FAST16_MAX,  (uint_fast16_t)   UINT_MAX(uint_fast16_t)); break;
>     CASE_TEST(limit_int_fast32_min);    EXPECT_EQ(1, INT_FAST32_MIN,   (int_fast32_t)    SINT_MIN(int_fast32_t)); break;
>     CASE_TEST(limit_int_fast32_max);    EXPECT_EQ(1, INT_FAST32_MAX,   (int_fast32_t)    SINT_MAX(int_fast32_t)); break;
>     CASE_TEST(limit_uint_fast32_max);   EXPECT_EQ(1, UINT_FAST32_MAX,  (uint_fast32_t)   UINT_MAX(uint_fast32_t)); break;
> 
> To avoid overriding the existing macros, perhaps we should add something
> like UINT_TYPE_MAX(t), SINT_TYPE_MAX(t) and SINT_TYPE_MIN(t) ?

They should only be visible inside nolibc-test.c I think.
But yes the UINT_MAX naming is bad.

Also when going away from testing constants maybe we can get back some
test strength by validating the sizeof() of the datatypes.

<snip>

> > > 
> > > * selftests/nolibc: vfprintf: silence memfd_create() warning
> > >   selftests/nolibc: vfprintf: skip if neither tmpfs nor hugetlbfs
> > >   selftests/nolibc: vfprintf: support tmpfs and hugetlbfs
> > > 
> > >   memfd_create from kernel >= v6.2 forcely warn on missing
> > >   MFD_NOEXEC_SEAL flag, the first one silence it with such flag, for
> > >   older kernels, use 0 flag as before.
> > 
> > Given this is only a problem when nolibc-test is PID1 and printing to
> > the system console, we could also just disable warnings on the system
> > console through syscall() or /proc/sys/kernel/printk.
> 
> Ok, I did think about disabling console for this call, but I was worried about
> the requirement of root (euid0) to do so, limiting it under PID1 may solve the
> root permission issue, but still need to find the right syscall to avoid the
> dependency of /proc/sys/kernel/printk, otherwise, to avoid failure for !procfs,
> the whole vfprintf will be skipped for such a warning, to be honest, it looks
> not a good direction.

This should work:

syslog(__NR_syslog, 6 /* SYSLOG_ACTION_CONSOLE_OFF */);

> > 
> > It would also avoid cluttering the tests for limited gain.
> >
> 
> Hmm, if consider the more code lines about disabling/enabling console and the
> dependency of /proc/sys/kernel/printk, I do prefer current change.

It should really only be the single line above.

> But I'm also interested in how the other applications developers to treat this
> warning, from the new kernel version side, we should use the latest non
> executable flags for security, but to let applications work with old kernels,
> we must support old flags, checking the kernel versions may be another choice. 

I know that systemd does it the same way as you proposed it, with
non-negligible code overhead.

But for nolibc-test I really don't see any security issue.

> Perhaps it's time for us to add the 'uname()' for nolibc, but the
> version comparing may be not that easy when we are in c context ;-)
> 
>   https://www.man7.org/linux/man-pages/man2/uname.2.html

Please no :-)

> So, the current method may be still a 'balanced' solution, it tries supported
> flags from new kernel to old kernel to get a better and working memfd_create()
> without the version checking, is this cleaner?
> 
> 	int i;
> 	/* kernel >= v6.2 require MFD_NOEXEC_SEAL (0x0008U), but older ones not support this flag */

It is not required, only desired. The functionality still works as
expected. I don't think the "old" way can ever stop working as it would
break userspace ABI.

> 	unsigned int flags[2] = {0x0008U, 0};
> 
> 	for (i = 0; i < 2; i ++) {

Loops like this should use ARRAY_SIZE() to calculate the termination
condition.

> 		/* try supported flags from new kernels to old kernels */
> 		fd = memfd_create("vfprintf", flags[i]);
> 		if (fd != -1)
> 			break;
> 	}
> 
> 	if (fd == -1) {
> 		...
> 	}

<snip>


Thomas

Re: [PATCH v1 00/17] selftests/nolibc: allow run with minimal kernel config

[Thomas Weißschuh]

On 2023-06-24 08:52:55+0200, Thomas Weißschuh wrote:
> As for the CONFIG_NET dependency, which I would guess is one of the more
> expensive configs to enable:
> 
> link_cross can be easily adapted to instead use /proc/self.
> 
> chmod_net relies on /proc/$PID/net accepting chmod().
> It is the only file in /proc/$PID/ that works that way.
> 
> Maybe its a kernel bug anyways and we shouldn't rely on it anyways?
> I'm taking a look.

It indeed seems to be a kernel bug. The following patch aligns
/proc/$PID/net with all the other /proc/$PID stuff.

diff --git a/fs/proc/proc_net.c b/fs/proc/proc_net.c
index a0c0419872e3..8c5e9abf4380 100644
--- a/fs/proc/proc_net.c
+++ b/fs/proc/proc_net.c
@@ -320,6 +320,7 @@ static int proc_tgid_net_getattr(struct mnt_idmap *idmap,
 
 const struct inode_operations proc_net_inode_operations = {
 	.lookup		= proc_tgid_net_lookup,
+	.setattr	= proc_setattr,
 	.getattr	= proc_tgid_net_getattr,
 };

I'm not entirely sure about the process to synchronize the application
of the fix in the procfs tree and the fix/removal of the testcase in the
nolibc tree so we avoid broken states.
Or if this would technically be a (relevant) break of userspace ABI and
therefore has to stay as it is.

Any ideas?

Thomas

Re: [PATCH v1 00/17] selftests/nolibc: allow run with minimal kernel config

[Zhangjin Wu]

> Hi, Thomas
> 
> On 2023-06-23 02:45:59+0800, Zhangjin Wu wrote:
> > > some general comments for the whole series.
> > > 
> > > On 2023-06-21 20:52:30+0800, Zhangjin Wu wrote:
> > > > Hi, Willy
> > > > 
> > > > This patchset mainly allows speed up the nolibc test with a minimal
> > > > kernel config.
> > > > 

(snip)

> > Based on my local powerpc porting, I have prepared some changes like
> > this:
> > 
> >     # extra kernel configs by architecture
> >     EXTCONFIG_powerpc    = --enable SERIAL_PMACZILOG --enable CONFIG_SERIAL_PMACZILOG_CONSOLE
> >     EXTCONFIG            = --set-str CONFIG_INITRAMFS_SOURCE $(CURDIR)/initramfs $(EXTCONFIG_$(ARCH))
> > 
> >     ...
> >also  
> >     menuconfig:
> >             $(Q)$(MAKE) -C $(srctree) ARCH=$(KARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) menuconfig
> >     
> >     extconfig:
> >             $(Q)$(srctree)/scripts/config --file $(srctree)/.config $(EXTCONFIG)
> >             $(Q)$(MAKE) -C $(srctree) ARCH=$(KARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) olddefconfig
> >     
> >     kernel: initramfs extconfig
> >             $(Q)$(MAKE) -C $(srctree) ARCH=$(KARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) $(IMAGE_NAME)
> > 
> > 
> > 'menuconfig' is added for development, for example, find why something not work
> > and add the missing options.
> > 
> > 'extconfig' is added to enable additional options (before, based on
> > defconfig) to let nolibc-test happy (for powerpc, add missing console
> > options which has been added as modules in default config).
> > 
> > Based on your suggestion, this may be a good new target:
> > 
> >     tinyconfig:
> >             $(Q)$(MAKE) -C $(srctree) ARCH=$(KARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) mrproper tinyconfig prepare
> > 
> > And this one, use 'allnoconfig' instead of 'olddefconfig':
> > 
> >     extconfig:
> >             $(Q)$(srctree)/scripts/config --file $(srctree)/.config $(EXTCONFIG)
> >             $(Q)$(MAKE) -C $(srctree) ARCH=$(KARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) KCONFIG_ALLCONFIG=$(srctree)/.config allnoconfig
> > 
> > So, the new 'tinyconfig' may function as the smallest test environment,
> > for faster compile and as a boundary test of nolibc-test itself.
> > 
> > But again, still need time to list the minimally required options, if they are
> > few, listing them in the EXTCONFIG_<ARCH> line may be acceptable, but if the
> > options are 'huge', standalone nolibc.config may be required, let's wait for
> > one or two days.
> 
> FYI there are many more tests in tools/testing/selftests/ that need
> custom configs to run. Maybe we can reuse some of their configuration
> machinery.
> (And qemu machinery maybe)
>

Yeah, thanks, these files may be very good references:

    $ find tools/testing/selftests/ -name "*config" | grep qemu
    tools/testing/selftests/wireguard/qemu/arch/aarch64.config
    tools/testing/selftests/wireguard/qemu/arch/aarch64_be.config
    tools/testing/selftests/wireguard/qemu/arch/arm.config
    tools/testing/selftests/wireguard/qemu/arch/armeb.config
    tools/testing/selftests/wireguard/qemu/arch/i686.config
    tools/testing/selftests/wireguard/qemu/arch/m68k.config
    tools/testing/selftests/wireguard/qemu/arch/mips.config
    tools/testing/selftests/wireguard/qemu/arch/mips64.config
    tools/testing/selftests/wireguard/qemu/arch/mips64el.config
    tools/testing/selftests/wireguard/qemu/arch/mipsel.config
    tools/testing/selftests/wireguard/qemu/arch/powerpc.config
    tools/testing/selftests/wireguard/qemu/arch/powerpc64.config
    tools/testing/selftests/wireguard/qemu/arch/powerpc64le.config
    tools/testing/selftests/wireguard/qemu/arch/riscv32.config
    tools/testing/selftests/wireguard/qemu/arch/riscv64.config
    tools/testing/selftests/wireguard/qemu/arch/s390x.config
    tools/testing/selftests/wireguard/qemu/arch/um.config
    tools/testing/selftests/wireguard/qemu/arch/x86_64.config
    tools/testing/selftests/wireguard/qemu/debug.config
    tools/testing/selftests/wireguard/qemu/kernel.config

And I have prepared most of them, just left 2-3 architectures.

> > > And it would be interesting how much impact the enablement of procfs,
> > > tmpfs, net and memfd_create has in constrast to the minimal
> > > configuration.
> > 
(snip)
> > 
> > It did save 20s (~17.1%) for us, not too much, but really faster. 
> > 
> > > It seems unfortunate to me to complicate the testsuite to handle such
> > > uncommon scenarios.
> > 
> > Yeah, such a config is not common, but as explained above, beside the compile
> > speedup improvement, it is really a good boundary test environment for
> > nolibc-test itself to make sure it work (no failure, less skips) at an
> > extremely worst-case scene, although our changes looks many, but every one is
> > as simple as CLOC ;-)
> > 
> > And that also means, nolibc is able to run with a very 'tiny' kernel
> > config and users could reuse our config fragments and add their own for
> > their embedded devices.
> 
> It would mean that nolibc-test is able to run on *really* trimmed down
> systems, which seems of limited use.
> If the testsuite has more dependencies it would not stop nolibc itself
> to run on them.
> 
> As for the CONFIG_NET dependency, which I would guess is one of the more
> expensive configs to enable:
> 
> link_cross can be easily adapted to instead use /proc/self.
>

Yes, we can simply use /proc/self or proc/self/cmdline.

> chmod_net relies on /proc/$PID/net accepting chmod().
> It is the only file in /proc/$PID/ that works that way.
>

Something like /tmp/xxx has been used in our patchset, to get a chmodable file,
tmpfs is a good choice, even when the TMPFS (and SHMEM) option is disabled, a
ramfs based tmpfs will be provided by kernel.

> Maybe its a kernel bug anyways and we shouldn't rely on it anyways?
> I'm taking a look.
>

Good catch, I have been also interested in such a curious difference,
because all of the other proc interfaces are not chmodable, not sure if
it is 'intentional', let's discuss in your patch thread, perhaps we
should cc the /proc/self/net maintainers/authors ;-)

> > 
> > (snipped)
> > 
> > > >   It is able to build and run nolibc-test with musl libc now, but there
> > > >   are some failures/skips due to the musl its own issues/requirements:
> > > > 
> > > >     $ sudo ./nolibc-test  | grep -E 'FAIL|SKIP'
> > > >     8 sbrk = 1 ENOMEM                                               [FAIL]
> > > >     9 brk = -1 ENOMEM                                               [FAIL]
> > > >     46 limit_int_fast16_min = -2147483648                           [FAIL]
> > > >     47 limit_int_fast16_max = 2147483647                            [FAIL]
> > > >     49 limit_int_fast32_min = -2147483648                           [FAIL]
> > > >     50 limit_int_fast32_max = 2147483647                            [FAIL]
> > > >     0 -fstackprotector not supported                                [SKIPPED]
> > > > 
> > > >   musl disabled sbrk and brk for some conflicts with its malloc and the
> > > >   fast version of int types are defined in 32bit, which differs from nolibc
> > > >   and glibc. musl reserved the sbrk(0) to allow get current brk, we
> > > >   added a test for this in the v4 __sysret() helper series [2].
> > > 
> > > We could add new macros
> > > 
> > > #define UINT_MAX(t) (~(t)0)
> > > #define SINT_MAX(t) (((t)1 << (sizeof(t) * 8 - 2)) - (t)1 + ((t)1 << (sizeof(t) * 8 - 2)))
> > > 
> > > to get whatever is appropriate for the respective type.
> > >
> > 
> > They work perfectly, thanks:
> > 
> >     /* for fast int test cases in stdlib test, musl use 32bit fast int */
> >     #undef UINT_MAX
> >     #define UINT_MAX(t)    (~(t)0)
> >     #undef SINT_MAX
> >     #define SINT_MAX(t)    (((t)1 << (sizeof(t) * 8 - 2)) - (t)1 + ((t)1 << (sizeof(t) * 8 - 2)))
> >     #undef SINT_MIN
> >     #define SINT_MIN(t)    (-SINT_MAX(t) - 1)
> > 
> >     ...
> > 
> >     CASE_TEST(limit_int_fast16_min);    EXPECT_EQ(1, INT_FAST16_MIN,   (int_fast16_t)    SINT_MIN(int_fast16_t)); break;
> >     CASE_TEST(limit_int_fast16_max);    EXPECT_EQ(1, INT_FAST16_MAX,   (int_fast16_t)    SINT_MAX(int_fast16_t)); break;
> >     CASE_TEST(limit_uint_fast16_max);   EXPECT_EQ(1, UINT_FAST16_MAX,  (uint_fast16_t)   UINT_MAX(uint_fast16_t)); break;
> >     CASE_TEST(limit_int_fast32_min);    EXPECT_EQ(1, INT_FAST32_MIN,   (int_fast32_t)    SINT_MIN(int_fast32_t)); break;
> >     CASE_TEST(limit_int_fast32_max);    EXPECT_EQ(1, INT_FAST32_MAX,   (int_fast32_t)    SINT_MAX(int_fast32_t)); break;
> >     CASE_TEST(limit_uint_fast32_max);   EXPECT_EQ(1, UINT_FAST32_MAX,  (uint_fast32_t)   UINT_MAX(uint_fast32_t)); break;
> > 
> > To avoid overriding the existing macros, perhaps we should add something
> > like UINT_TYPE_MAX(t), SINT_TYPE_MAX(t) and SINT_TYPE_MIN(t) ?
> 
> They should only be visible inside nolibc-test.c I think.

Yeah, I did so.

> But yes the UINT_MAX naming is bad.
>
> Also when going away from testing constants maybe we can get back some
> test strength by validating the sizeof() of the datatypes.
>

It seems easier.

> <snip>
> 
> > > > 
> > > > * selftests/nolibc: vfprintf: silence memfd_create() warning
> > > >   selftests/nolibc: vfprintf: skip if neither tmpfs nor hugetlbfs
> > > >   selftests/nolibc: vfprintf: support tmpfs and hugetlbfs
> > > > 
> > > >   memfd_create from kernel >= v6.2 forcely warn on missing
> > > >   MFD_NOEXEC_SEAL flag, the first one silence it with such flag, for
> > > >   older kernels, use 0 flag as before.
> > > 
> > > Given this is only a problem when nolibc-test is PID1 and printing to
> > > the system console, we could also just disable warnings on the system
> > > console through syscall() or /proc/sys/kernel/printk.
> > 
> > Ok, I did think about disabling console for this call, but I was worried about
> > the requirement of root (euid0) to do so, limiting it under PID1 may solve the
> > root permission issue, but still need to find the right syscall to avoid the
> > dependency of /proc/sys/kernel/printk, otherwise, to avoid failure for !procfs,
> > the whole vfprintf will be skipped for such a warning, to be honest, it looks
> > not a good direction.
> 
> This should work:
> 
> syslog(__NR_syslog, 6 /* SYSLOG_ACTION_CONSOLE_OFF */);
>

Thanks, will try it like this:

    syslog(__NR_syslog, 6 /* SYSLOG_ACTION_CONSOLE_OFF */);
    memfd_create()
    syslog(__NR_syslog, 7 /* SYSLOG_ACTION_CONSOLE_ON */);

Putting it in prepare() may hide potential issues reported by kernel.

> > > 
> > > It would also avoid cluttering the tests for limited gain.
> > >
> > 
> > Hmm, if consider the more code lines about disabling/enabling console and the
> > dependency of /proc/sys/kernel/printk, I do prefer current change.
> 
> It should really only be the single line above.
> 
> > But I'm also interested in how the other applications developers to treat this
> > warning, from the new kernel version side, we should use the latest non
> > executable flags for security, but to let applications work with old kernels,
> > we must support old flags, checking the kernel versions may be another choice. 
> 
> I know that systemd does it the same way as you proposed it, with
> non-negligible code overhead.
> 
> But for nolibc-test I really don't see any security issue.
>

Yes, but at least as a good reference when people want to reuse some
codes from nolibc-test.c ;-)

> > Perhaps it's time for us to add the 'uname()' for nolibc, but the
> > version comparing may be not that easy when we are in c context ;-)
> > 
> >   https://www.man7.org/linux/man-pages/man2/uname.2.html
> 
> Please no :-)
> 
> > So, the current method may be still a 'balanced' solution, it tries supported
> > flags from new kernel to old kernel to get a better and working memfd_create()
> > without the version checking, is this cleaner?
> > 
> > 	int i;
> > 	/* kernel >= v6.2 require MFD_NOEXEC_SEAL (0x0008U), but older ones not support this flag */
> 
> It is not required, only desired. The functionality still works as
> expected. I don't think the "old" way can ever stop working as it would
> break userspace ABI.
> 
> > 	unsigned int flags[2] = {0x0008U, 0};
> > 
> > 	for (i = 0; i < 2; i ++) {
> 
> Loops like this should use ARRAY_SIZE() to calculate the termination
> condition.

Yeah, it should be:

    unsigned int flags[] = {0x0008U, 0};
   
    for (i = 0; i < sizeof(flags)/sizeof(flags[0]); i ++)
	    ...

Best regards,
Zhangjin

> 
> > 		/* try supported flags from new kernels to old kernels */
> > 		fd = memfd_create("vfprintf", flags[i]);
> > 		if (fd != -1)
> > 			break;
> > 	}
> > 
> > 	if (fd == -1) {
> > 		...
> > 	}
> 
> <snip>
> 
> 
> Thomas

Re: [PATCH v1 00/17] selftests/nolibc: allow run with minimal kernel config

[Zhangjin Wu]

Hi, Thomas

> > Hi Zhangjin,
> > 
> > some general comments for the whole series.
> > 
> > On 2023-06-21 20:52:30+0800, Zhangjin Wu wrote:
> > > Hi, Willy
> > > 
> > > This patchset mainly allows speed up the nolibc test with a minimal
> > > kernel config.
> > > 
> (snip)
> > > 
> > > * selftests/nolibc: fix up kernel parameters support
> > > 
> > >   kernel cmdline allows pass two types of parameters, one is without
> > >   '=', another is with '=', the first one is passed as init arguments,
> > >   the sencond one is passed as init environment variables.
> > > 
> > >   Our nolibc-test prefer arguments to environment variables, this not
> > >   work when users add such parameters in the kernel cmdline:
> > > 
> > >     noapic NOLIBC_TEST=syscall
> > > 
> > >   So, this patch will verify the setting from arguments at first, if it
> > >   is no valid, will try the environment variables instead.
> > 
> > This would be much simpler as:
> > 
> > test = getenv("NOLIBC_TEST");
> > if (!test)
> >         test = argv[1];
> >
> > It changes the semantics a bit, but it doesn't seem to be an issue.
> > (Maybe gated behind getpid() == 1).
> 
> Cool suggestion, it looks really better:
> 
> 	if (getpid() == 1) {
> 		prepare();
> 		
> 		/* kernel cmdline may pass: "noapic NOLIBC_TEST=syscall",
>                  * to init program:
> 		 *
> 		 *   "noapic" as arguments,
> 		 *   "NOLIBC_TEST=syscall" as environment variables,
>                  *
> 		 * to avoid getting null test in this case, parsing
> 		 * environment variables at first.
> 		 */
> 		test = getenv("NOLIBC_TEST");
> 		if (!test)
> 			test = argv[1];
> 	} else {
> 		/* for normal nolibc-test program, prefer arguments */
> 		test = argv[1];
> 		if (!test)
> 			test = getenv("NOLIBC_TEST");
> 	}
> 

Test shows, when no NOLIBC_TEST environment variable passed to kernel cmdline,
it will still branch to this code:

    test = argv[1]; /* nopaic ... */

And therefore report the whole test is ignored and no test will be run:

    Ignoring unknown test name 'noapic'

So, we may still need to verify it like my originally proposed method, but
let's further verify the one from NOLIBC_TEST=, we should tune the code a
litle.

Thanks,
Zhangjin

[PATCH v1 05/17] selftests/nolibc: stat_timestamps: remove procfs dependency

[Zhangjin Wu]

Hi, Thomas

I'm preparing a revision for this series, in the past days, when I was
working on testing our new 'minimal' kernel config support for all of
the architectures, the time cost (and wait) is really appreciable and the
repeated develop and test is really a big pain, I can also image when you
was working on stack-protector and Willy was working on lots of old
features ;-)

As you explained before, I knew the idea of using '/proc/self' here is
important to not using a fixed-time file, besides our proposed method (make
sure it at least not fail, just skip for !procfs):

    - CASE_TEST(stat_timestamps);   EXPECT_SYSZR(1, test_stat_timestamps()); break;
    + CASE_TEST(stat_timestamps);   EXPECT_SYSZR(proc, test_stat_timestamps()); break;

To further avoid skip it for !procfs (I don't mean relaly disable it for the
default tinyconfig support, which need more discuss, at least provide the
possibility to pass without procfs), do you like this change? it doesn't depend
on 'proc' now.

    -	if (stat("/proc/self/", &st))
    +	if (stat("/proc/self/", &st) && stat("/init", &st) && stat("/", &st))

The "/init" is compiled for 'run' target every time, so, the time stamp should
be dynamic enough, for libc-test, the /proc/self should be always there (if
still not enough, we can reuse the init file list here), the "/" here is only
for the worst-case scene ;-)

Thanks,
Zhangjin

> Since it is not really necessary to use /proc/self here, instead of
> adding a condition check, we use the always existing '/' path instead of
> /proc/self, this eventually let it work without procfs.
> 
> Signed-off-by: Zhangjin Wu <falcon@tinylab.org>
> ---
>  tools/testing/selftests/nolibc/nolibc-test.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tools/testing/selftests/nolibc/nolibc-test.c b/tools/testing/selftests/nolibc/nolibc-test.c
> index ebec948ec808..2ef44176f7a9 100644
> --- a/tools/testing/selftests/nolibc/nolibc-test.c
> +++ b/tools/testing/selftests/nolibc/nolibc-test.c
> @@ -520,7 +520,7 @@ static int test_stat_timestamps(void)
>  	if (sizeof(st.st_atim.tv_sec) != sizeof(st.st_atime))
>  		return 1;
>  
> -	if (stat("/proc/self/", &st))
> +	if (stat("/", &st))
>  		return 1;
>  
>  	if (st.st_atim.tv_sec != st.st_atime || st.st_atim.tv_nsec > 1000000000)
> -- 
> 2.25.1

Re: [PATCH v1 05/17] selftests/nolibc: stat_timestamps: remove procfs dependency

[Thomas Weißschuh]

Hi Zhangjin,

On 2023-06-28 21:59:22+0800, Zhangjin Wu wrote:
> I'm preparing a revision for this series, in the past days, when I was
> working on testing our new 'minimal' kernel config support for all of
> the architectures, the time cost (and wait) is really appreciable and the
> repeated develop and test is really a big pain, I can also image when you
> was working on stack-protector and Willy was working on lots of old
> features ;-)

To be honest I almost never built a kernel.
Most of the time I tested my stuff with qemu-user.
This made the dev-cycle really fast, especially with a binfmt setup that
launches foreign binaries automatically with qemu-user.

> As you explained before, I knew the idea of using '/proc/self' here is
> important to not using a fixed-time file, besides our proposed method (make
> sure it at least not fail, just skip for !procfs):
> 
>     - CASE_TEST(stat_timestamps);   EXPECT_SYSZR(1, test_stat_timestamps()); break;
>     + CASE_TEST(stat_timestamps);   EXPECT_SYSZR(proc, test_stat_timestamps()); break;
> 
> To further avoid skip it for !procfs (I don't mean relaly disable it for the
> default tinyconfig support, which need more discuss, at least provide the
> possibility to pass without procfs), do you like this change? it doesn't depend
> on 'proc' now.
> 
>     -	if (stat("/proc/self/", &st))
>     +	if (stat("/proc/self/", &st) && stat("/init", &st) && stat("/", &st))
> 
> The "/init" is compiled for 'run' target every time, so, the time stamp should
> be dynamic enough, for libc-test, the /proc/self should be always there (if
> still not enough, we can reuse the init file list here), the "/" here is only
> for the worst-case scene ;-)

Both aproaches seem fine. Just skipping on !proc seems good enough.

As for enabling proc in the test configs I just tested a plain
tinyconfig vs one with CONFIG_PROC_FS enabled:

tinyconfig:                  375.06user 53.21system 2:05.80elapsed
tinyconfig + CONFIG_PROC_FS: 397.77user 56.84system 2:09.24elapsed

The overhead seems acceptable.


Note as for disabling memfd:

It seems currently MEMFD_CREATE is hardwired to only be enabled when
either TMPFS or HUGETLBFS is enabled.

But the memfd code and syscalls seem to work perfectly fine with those
options disabled. I'll send a patch to fix up the Kconfigs to enable
that usecase.

> Thanks,
> Zhangjin
> 
> > Since it is not really necessary to use /proc/self here, instead of
> > adding a condition check, we use the always existing '/' path instead of
> > /proc/self, this eventually let it work without procfs.
> > 
> > Signed-off-by: Zhangjin Wu <falcon@tinylab.org>
> > ---
> >  tools/testing/selftests/nolibc/nolibc-test.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/tools/testing/selftests/nolibc/nolibc-test.c b/tools/testing/selftests/nolibc/nolibc-test.c
> > index ebec948ec808..2ef44176f7a9 100644
> > --- a/tools/testing/selftests/nolibc/nolibc-test.c
> > +++ b/tools/testing/selftests/nolibc/nolibc-test.c
> > @@ -520,7 +520,7 @@ static int test_stat_timestamps(void)
> >  	if (sizeof(st.st_atim.tv_sec) != sizeof(st.st_atime))
> >  		return 1;
> >  
> > -	if (stat("/proc/self/", &st))
> > +	if (stat("/", &st))
> >  		return 1;
> >  
> >  	if (st.st_atim.tv_sec != st.st_atime || st.st_atim.tv_nsec > 1000000000)
> > -- 
> > 2.25.1

Re: [PATCH v1 05/17] selftests/nolibc: stat_timestamps: remove procfs dependency

[Zhangjin Wu]

> Hi Zhangjin,
> 
> On 2023-06-28 21:59:22+0800, Zhangjin Wu wrote:
> > I'm preparing a revision for this series, in the past days, when I was
> > working on testing our new 'minimal' kernel config support for all of
> > the architectures, the time cost (and wait) is really appreciable and the
> > repeated develop and test is really a big pain, I can also image when you
> > was working on stack-protector and Willy was working on lots of old
> > features ;-)
> 
> To be honest I almost never built a kernel.
> Most of the time I tested my stuff with qemu-user.
> This made the dev-cycle really fast, especially with a binfmt setup that
> launches foreign binaries automatically with qemu-user.
>

Yeah, qemu-user-static + binfmt_misc work perfectly, but my host kernel
is not that new, so, I'm still a little worried about that there may be
some hidden issues.

> > As you explained before, I knew the idea of using '/proc/self' here is
> > important to not using a fixed-time file, besides our proposed method (make
> > sure it at least not fail, just skip for !procfs):
> > 
> >     - CASE_TEST(stat_timestamps);   EXPECT_SYSZR(1, test_stat_timestamps()); break;
> >     + CASE_TEST(stat_timestamps);   EXPECT_SYSZR(proc, test_stat_timestamps()); break;
> > 
> > To further avoid skip it for !procfs (I don't mean relaly disable it for the
> > default tinyconfig support, which need more discuss, at least provide the
> > possibility to pass without procfs), do you like this change? it doesn't depend
> > on 'proc' now.
> > 
> >     -	if (stat("/proc/self/", &st))
> >     +	if (stat("/proc/self/", &st) && stat("/init", &st) && stat("/", &st))
> > 
> > The "/init" is compiled for 'run' target every time, so, the time stamp should
> > be dynamic enough, for libc-test, the /proc/self should be always there (if
> > still not enough, we can reuse the init file list here), the "/" here is only
> > for the worst-case scene ;-)
> 
> Both aproaches seem fine. Just skipping on !proc seems good enough.
>

To get less skips, let's use the second method, just updated my local
patches ;-)

> As for enabling proc in the test configs I just tested a plain
> tinyconfig vs one with CONFIG_PROC_FS enabled:
> 
> tinyconfig:                  375.06user 53.21system 2:05.80elapsed
> tinyconfig + CONFIG_PROC_FS: 397.77user 56.84system 2:09.24elapsed
> 
> The overhead seems acceptable.
>

Yeah, only one option is ok, but "multiple options x multiple
architectures x multiple repeated runs", that is 'huge' ;-)

> 
> Note as for disabling memfd:
> 
> It seems currently MEMFD_CREATE is hardwired to only be enabled when
> either TMPFS or HUGETLBFS is enabled.
> 
> But the memfd code and syscalls seem to work perfectly fine with those
> options disabled. I'll send a patch to fix up the Kconfigs to enable
> that usecase.

Good catch!

but for the vfprintf test cases, It is able to open a file from tmpfs
directly. If no tmpfs, use the default ramfs (initramfs uses) instead,
this will also avoid the new flags trying (to silence the warning).

     static int expect_vfprintf(int llen, size_t c, const char *expected, const char *fmt, ...)
     {
    +       static const char *tmpfile = "/tmp/nolibc-vfprintf";
    +       struct stat stat_buf;
            int ret, fd, w, r;
            char buf[100];
            FILE *memfile;
            va_list args;

    -       fd = memfd_create("vfprintf", 0);
    +       if (stat("/tmp/.", &stat_buf)) {
    +               pad_spc(llen, 64, "[SKIPPED]\n");
    +               return 0;
    +       }
    +
    +       fd = open(tmpfile, O_CREAT | O_TRUNC | O_RDWR, 0755);
    ...
    +       unlink(tmpfile);
    ...

tmpfs is mounted (in another patch) like procfs in prepare() for pid==1.

I plan to use this method in the revision, do you like this?

memfd_create() was designed to do this work, but in current stage,
opening tmpfile ourselves may be better.

Thanks,
Zhangjin

Re: [PATCH v1 05/17] selftests/nolibc: stat_timestamps: remove procfs dependency

[Thomas Weißschuh]

On 2023-06-30 05:23:35+0800, Zhangjin Wu wrote:
> > Hi Zhangjin,
> > 
> > On 2023-06-28 21:59:22+0800, Zhangjin Wu wrote:
> > > I'm preparing a revision for this series, in the past days, when I was
> > > working on testing our new 'minimal' kernel config support for all of
> > > the architectures, the time cost (and wait) is really appreciable and the
> > > repeated develop and test is really a big pain, I can also image when you
> > > was working on stack-protector and Willy was working on lots of old
> > > features ;-)
> > 
> > To be honest I almost never built a kernel.
> > Most of the time I tested my stuff with qemu-user.
> > This made the dev-cycle really fast, especially with a binfmt setup that
> > launches foreign binaries automatically with qemu-user.
> >
> 
> Yeah, qemu-user-static + binfmt_misc work perfectly, but my host kernel
> is not that new, so, I'm still a little worried about that there may be
> some hidden issues.

qemu-user shouldn't have specific requirements for the host kernel.
Could you elaborate?

> > > As you explained before, I knew the idea of using '/proc/self' here is
> > > important to not using a fixed-time file, besides our proposed method (make
> > > sure it at least not fail, just skip for !procfs):
> > > 
> > >     - CASE_TEST(stat_timestamps);   EXPECT_SYSZR(1, test_stat_timestamps()); break;
> > >     + CASE_TEST(stat_timestamps);   EXPECT_SYSZR(proc, test_stat_timestamps()); break;
> > > 
> > > To further avoid skip it for !procfs (I don't mean relaly disable it for the
> > > default tinyconfig support, which need more discuss, at least provide the
> > > possibility to pass without procfs), do you like this change? it doesn't depend
> > > on 'proc' now.
> > > 
> > >     -	if (stat("/proc/self/", &st))
> > >     +	if (stat("/proc/self/", &st) && stat("/init", &st) && stat("/", &st))
> > > 
> > > The "/init" is compiled for 'run' target every time, so, the time stamp should
> > > be dynamic enough, for libc-test, the /proc/self should be always there (if
> > > still not enough, we can reuse the init file list here), the "/" here is only
> > > for the worst-case scene ;-)
> > 
> > Both aproaches seem fine. Just skipping on !proc seems good enough.
> >
> 
> To get less skips, let's use the second method, just updated my local
> patches ;-)
> 
> > As for enabling proc in the test configs I just tested a plain
> > tinyconfig vs one with CONFIG_PROC_FS enabled:
> > 
> > tinyconfig:                  375.06user 53.21system 2:05.80elapsed
> > tinyconfig + CONFIG_PROC_FS: 397.77user 56.84system 2:09.24elapsed
> > 
> > The overhead seems acceptable.
> >
> 
> Yeah, only one option is ok, but "multiple options x multiple
> architectures x multiple repeated runs", that is 'huge' ;-)

In your other patchset you mentioned a few options that were needed.
But if we can drop CONFIG_NET completely, reduce CONFIG_TMPFS to
CONFIG_MEMFD I would like to reevaluate the overall impact.

> > 
> > Note as for disabling memfd:
> > 
> > It seems currently MEMFD_CREATE is hardwired to only be enabled when
> > either TMPFS or HUGETLBFS is enabled.
> > 
> > But the memfd code and syscalls seem to work perfectly fine with those
> > options disabled. I'll send a patch to fix up the Kconfigs to enable
> > that usecase.
> 
> Good catch!
> 
> but for the vfprintf test cases, It is able to open a file from tmpfs
> directly. If no tmpfs, use the default ramfs (initramfs uses) instead,
> this will also avoid the new flags trying (to silence the warning).
> 
>      static int expect_vfprintf(int llen, size_t c, const char *expected, const char *fmt, ...)
>      {
>     +       static const char *tmpfile = "/tmp/nolibc-vfprintf";
>     +       struct stat stat_buf;

This should not be static.

>             int ret, fd, w, r;
>             char buf[100];
>             FILE *memfile;
>             va_list args;
> 
>     -       fd = memfd_create("vfprintf", 0);
>     +       if (stat("/tmp/.", &stat_buf)) {
>     +               pad_spc(llen, 64, "[SKIPPED]\n");
>     +               return 0;
>     +       }

Instead of checking with stat() here it would be nicer to check the
result of open() below.

>     +
>     +       fd = open(tmpfile, O_CREAT | O_TRUNC | O_RDWR, 0755);

Seems like a good usecase for open("/tmp", O_TMPFILE | O_EXCL | ...)

>     ...
>     +       unlink(tmpfile);

... and drop this.

  fd = memfd_open();
+ if (fd == -1)
+        fd = open("/tmp", O_TMPFILE | O_EXCL | O_CREAT | O_RDWR, 0600);
+ if (fd == -1)
+        skip()

May be enough.

>     ...
> 
> tmpfs is mounted (in another patch) like procfs in prepare() for pid==1.
> 
> I plan to use this method in the revision, do you like this?
> 
> memfd_create() was designed to do this work, but in current stage,
> opening tmpfile ourselves may be better.

I'm not really a fan of having so much fallback code for features that
should always be available.
But if we can keep it straightforward it's probably fine.
Or Willy has another opinion :-)