All of lore.kernel.org
 help / color / mirror / Atom feed
* [RFC v3 00/26] Unifying LKL into UML
@ 2020-02-05  7:30 Hajime Tazaki
  2020-02-05  7:30 ` [RFC v3 01/26] asm-generic: atomic64: allow using generic atomic64 on 64bit platforms Hajime Tazaki
                   ` (26 more replies)
  0 siblings, 27 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05  7:30 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Hajime Tazaki

This patchset is to ask opinions from folks, whether LKL codes is good to
integrate into UML code base.  We wish to have any kind of feedback from
your kind reviews.  There are numbers of commits which should be asked for
reviews to other mailing lists; we will do it later once we got discussed
in this mailing list.

Changes in rfc v3:
- use UML drivers (net, block) from LKL programs
- drop virtio device implementations
- drop mingw32 (Windows) host
- drop android (arm/aarch64) host
- drop FreeBSD (x86_64) host
- drop LD_PRELOAD (hijack) support
- update milestone

rfc v2:
- use UMMODE instead of SUBARCH to switch UML or LKL
- tools/lkl directory is still there. I confirmed we can move under arch/um
  (e.g., arch/um/lkl/hosts).  I will move it IF this is preferable.
- drop several patches involved non-uml directory
- drop several patches which are not required
- refine commit logs
- document updated




LKL (Linux Kernel Library) is aiming to allow reusing the Linux kernel code
as extensively as possible with minimal effort and reduced maintenance
overhead.

Examples of how LKL can be used are: creating userspace applications
(running on Linux and other operating systems) that can read or write Linux
filesystems or can use the Linux networking stack, creating kernel drivers
for other operating systems that can read Linux filesystems, bootloaders
support for reading/writing Linux filesystems, etc.

With LKL, the kernel code is compiled into an object file that can be
directly linked by applications. The API offered by LKL is based on the
Linux system call interface.

LKL is originally implemented as an architecture port in arch/lkl, but this
series of commits tries to integrate this into arch/um as one of the mode
of UML.  This was discussed during RFC email of LKL (*1).

The latest LKL version can be found at https://github.com/lkl/linux

Milestone
=========
This patches is a first step toward upstreaming *library mode* of Linux kernel,
but we think we need to have several steps toward our goal, describing in the
below.

1. Put LKL code under arch/um (arch/um/lkl), and build it in a separate way
from UML.
2. Use UML driver implementations in LKL as a minimum set of patches
-  Only support x86 sub architecture (dependency to UML drivers)
3. Support broader host supports
- add virtio device features

For the step 1, we put LKL as one of UMMODE in order to make less effort to
integrate (make ARCH=um UMMODE=library).  The modification to existing UML
code is trying to be minimized.



Building LKL the host library and LKL applications
==================================================

% cd tools/lkl
% make

will build LKL as a object file, it will install it in tools/lkl/lib together
with the headers files in tools/lkl/include then will build the host library,
tests and a few of application examples:

* tests/boot - a simple applications that uses LKL and exercises the basic
  LKL APIs

* tests/net-test - a simple applications that uses network feature of
  LKL and exercises the basic network-related APIs

* fs2tar - a tool that converts a filesystem image to a tar archive

* cptofs/cpfromfs - a tool that copies files to/from a filesystem image

* lklfuse - a tool that can mount a filesystem image in userspace,
  without root priviledges, using FUSE

% make run-tests

should run the above `tests/boot` and `tests/net-test` and report errors if
there are any.

Supported hosts
===============

Currently LKL supports Linux userspace applications. New hosts can be added
relatively easy if the host supports gcc and GNU ld. Previous versions of
LKL supported Windows kernel and Haiku kernel hosts, and we also have WIP
patches with rump-hypercall interface, used in UEFI, as well as macOS
userspace (part of POSIX).

There is also musl-libc port for LKL, which might be interested in for some
folks.


Further readings about LKL
=========================

- Discussion in github LKL issue
https://github.com/lkl/linux/issues/304

- LKL (an article)
https://www.researchgate.net/profile/Nicolae_Tapus2/publication/224164682_LKL_The_Linux_kernel_library/links/02bfe50fd921ab4f7c000000.pdf

*1 RFC email to LKML (back in 2015)
https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1012277.html



Please review the following changes for suitability for inclusion. If you have
any objections or suggestions for improvement, please respond to the patches. If
you agree with the changes, please provide your Acked-by.

The following changes since commit d65197ad52494bed3b5e64708281b8295f76c391:

  um: Fix time-travel=inf-cpu with xor/raid6 (2020-01-19 22:42:06 +0100)

are available in the Git repository at:

  git://github.com/thehajime/linux b3072caaef01f2cef5f34bc012e066826798500c
  https://github.com/thehajime/linux/tree/upstream-to-uml-5.6-rc1-v3

Hajime Tazaki (6):
  lkl tools: host lib: filesystem helpers
  lkl tools: host lib: networking helpers
  um lkl: add CI scripts to conduct regression tests
  um lkl: add UML network driver for lkl
  um lkl: add UML block device driver (ubd) for lkl
  um: fix clone flags to be familar with valgrind

Octavian Purdila (20):
  asm-generic: atomic64: allow using generic atomic64 on 64bit platforms
  arch: add __SYSCALL_DEFINE_ARCH
  um lkl: architecture skeleton for Linux kernel library
  um lkl: host interface
  um lkl: memory handling
  um lkl: kernel threads support
  um lkl: interrupt support
  um lkl: system call interface and application API
  um lkl: timers, time and delay support
  um lkl: basic kernel console support
  um lkl: initialization and cleanup
  um lkl: plug in the build system
  lkl tools: skeleton for host side library
  lkl tools: host lib: add utilities functions
  lkl tools: host lib: posix host operations
  lkl tools: add test programs
  lkl tools: cptofs that reads/writes to/from a filesystem image
  lkl tools: fs2tar that converts a filesystem image to tar
  lkl tools: add lklfuse
  um lkl: add documentation

 .circleci/config.yml                       |  198 ++++
 Documentation/virt/uml/lkl.txt             |   64 ++
 MAINTAINERS                                |    8 +
 arch/um/Kconfig                            |   13 +
 arch/um/Makefile                           |   19 +-
 arch/um/Makefile.um                        |   10 +
 arch/um/drivers/Makefile                   |    2 +
 arch/um/drivers/ubd_kern.c                 |    6 +-
 arch/um/drivers/ubd_user.c                 |    3 +-
 arch/um/kernel/Makefile                    |    4 +
 arch/um/kernel/irq.c                       |    4 +
 arch/um/lkl/.gitignore                     |    2 +
 arch/um/lkl/Kconfig                        |   88 ++
 arch/um/lkl/Kconfig.debug                  |    0
 arch/um/lkl/Makefile                       |   69 ++
 arch/um/lkl/auto.conf                      |    1 +
 arch/um/lkl/configs/lkl_defconfig          | 1052 ++++++++++++++++++++
 arch/um/lkl/include/asm/Kbuild             |   80 ++
 arch/um/lkl/include/asm/bitsperlong.h      |   11 +
 arch/um/lkl/include/asm/byteorder.h        |    7 +
 arch/um/lkl/include/asm/cpu.h              |   14 +
 arch/um/lkl/include/asm/elf.h              |   15 +
 arch/um/lkl/include/asm/host_ops.h         |   10 +
 arch/um/lkl/include/asm/irq.h              |   13 +
 arch/um/lkl/include/asm/page.h             |   14 +
 arch/um/lkl/include/asm/pgtable.h          |   57 ++
 arch/um/lkl/include/asm/processor.h        |   60 ++
 arch/um/lkl/include/asm/ptrace.h           |   25 +
 arch/um/lkl/include/asm/sched.h            |   23 +
 arch/um/lkl/include/asm/setup.h            |    7 +
 arch/um/lkl/include/asm/syscalls.h         |   18 +
 arch/um/lkl/include/asm/syscalls_32.h      |   43 +
 arch/um/lkl/include/asm/thread_info.h      |   70 ++
 arch/um/lkl/include/asm/tlb.h              |   12 +
 arch/um/lkl/include/asm/uaccess.h          |   64 ++
 arch/um/lkl/include/asm/unistd.h           |   27 +
 arch/um/lkl/include/asm/unistd_32.h        |   31 +
 arch/um/lkl/include/asm/vmlinux.lds.h      |   14 +
 arch/um/lkl/include/asm/xor.h              |    9 +
 arch/um/lkl/include/uapi/asm/Kbuild        |    9 +
 arch/um/lkl/include/uapi/asm/bitsperlong.h |   13 +
 arch/um/lkl/include/uapi/asm/byteorder.h   |   11 +
 arch/um/lkl/include/uapi/asm/host_ops.h    |  143 +++
 arch/um/lkl/include/uapi/asm/irq.h         |   36 +
 arch/um/lkl/include/uapi/asm/sigcontext.h  |   16 +
 arch/um/lkl/include/uapi/asm/siginfo.h     |   11 +
 arch/um/lkl/include/uapi/asm/swab.h        |   11 +
 arch/um/lkl/include/uapi/asm/syscalls.h    |  349 +++++++
 arch/um/lkl/include/uapi/asm/unistd.h      |   16 +
 arch/um/lkl/kernel/Makefile                |    4 +
 arch/um/lkl/kernel/asm-offsets.c           |    3 +
 arch/um/lkl/kernel/console.c               |   42 +
 arch/um/lkl/kernel/cpu.c                   |  223 +++++
 arch/um/lkl/kernel/irq.c                   |  209 ++++
 arch/um/lkl/kernel/misc.c                  |   60 ++
 arch/um/lkl/kernel/setup.c                 |  198 ++++
 arch/um/lkl/kernel/syscalls.c              |  192 ++++
 arch/um/lkl/kernel/syscalls_32.c           |  159 +++
 arch/um/lkl/kernel/threads.c               |  227 +++++
 arch/um/lkl/kernel/time.c                  |  145 +++
 arch/um/lkl/kernel/vmlinux.lds.S           |   51 +
 arch/um/lkl/mm/Makefile                    |    1 +
 arch/um/lkl/mm/bootmem.c                   |   99 ++
 arch/um/lkl/scripts/headers_install.py     |  196 ++++
 arch/um/os-Linux/Makefile                  |    5 +
 arch/um/os-Linux/file.c                    |    9 +
 include/asm-generic/atomic64.h             |    2 +
 include/linux/syscalls.h                   |    6 +
 tools/lkl/.gitignore                       |   13 +
 tools/lkl/Build                            |    6 +
 tools/lkl/Makefile                         |  131 +++
 tools/lkl/Makefile.autoconf                |   67 ++
 tools/lkl/Targets                          |   17 +
 tools/lkl/cptofs.c                         |  636 ++++++++++++
 tools/lkl/fs2tar.c                         |  412 ++++++++
 tools/lkl/include/.gitignore               |    1 +
 tools/lkl/include/lkl.h                    |  788 +++++++++++++++
 tools/lkl/include/lkl_config.h             |   61 ++
 tools/lkl/include/lkl_host.h               |  112 +++
 tools/lkl/lib/.gitignore                   |    3 +
 tools/lkl/lib/Build                        |   13 +
 tools/lkl/lib/config.c                     |  750 ++++++++++++++
 tools/lkl/lib/dbg.c                        |  300 ++++++
 tools/lkl/lib/dbg_handler.c                |   44 +
 tools/lkl/lib/endian.h                     |   31 +
 tools/lkl/lib/fs.c                         |  471 +++++++++
 tools/lkl/lib/jmp_buf.c                    |   14 +
 tools/lkl/lib/jmp_buf.h                    |    8 +
 tools/lkl/lib/net.c                        |  818 +++++++++++++++
 tools/lkl/lib/posix-host.c                 |  356 +++++++
 tools/lkl/lib/um/Build                     |    3 +
 tools/lkl/lib/um/um_block.c                |   17 +
 tools/lkl/lib/um/um_glue.c                 |   58 ++
 tools/lkl/lib/um/um_net.c                  |   25 +
 tools/lkl/lib/utils.c                      |  266 +++++
 tools/lkl/lklfuse.c                        |  659 ++++++++++++
 tools/lkl/scripts/checkpatch.sh            |   60 ++
 tools/lkl/scripts/lkl-jenkins.sh           |   21 +
 tools/lkl/tests/Build                      |    3 +
 tools/lkl/tests/boot.c                     |  562 +++++++++++
 tools/lkl/tests/boot.sh                    |    9 +
 tools/lkl/tests/cla.c                      |  159 +++
 tools/lkl/tests/cla.h                      |   33 +
 tools/lkl/tests/disk.c                     |  203 ++++
 tools/lkl/tests/disk.sh                    |   61 ++
 tools/lkl/tests/lklfuse.sh                 |  110 ++
 tools/lkl/tests/net-setup.sh               |   74 ++
 tools/lkl/tests/net-test.c                 |  291 ++++++
 tools/lkl/tests/net.sh                     |  108 ++
 tools/lkl/tests/run.py                     |  180 ++++
 tools/lkl/tests/tap13.py                   |  209 ++++
 tools/lkl/tests/test.c                     |  126 +++
 tools/lkl/tests/test.h                     |   72 ++
 tools/lkl/tests/test.sh                    |  182 ++++
 tools/lkl/tests/valgrind.supp              |  105 ++
 tools/lkl/tests/valgrind2xunit.py          |   69 ++
 116 files changed, 13327 insertions(+), 3 deletions(-)
 create mode 100644 .circleci/config.yml
 create mode 100644 Documentation/virt/uml/lkl.txt
 create mode 100644 arch/um/Makefile.um
 create mode 100644 arch/um/lkl/.gitignore
 create mode 100644 arch/um/lkl/Kconfig
 create mode 100644 arch/um/lkl/Kconfig.debug
 create mode 100644 arch/um/lkl/Makefile
 create mode 100644 arch/um/lkl/auto.conf
 create mode 100644 arch/um/lkl/configs/lkl_defconfig
 create mode 100644 arch/um/lkl/include/asm/Kbuild
 create mode 100644 arch/um/lkl/include/asm/bitsperlong.h
 create mode 100644 arch/um/lkl/include/asm/byteorder.h
 create mode 100644 arch/um/lkl/include/asm/cpu.h
 create mode 100644 arch/um/lkl/include/asm/elf.h
 create mode 100644 arch/um/lkl/include/asm/host_ops.h
 create mode 100644 arch/um/lkl/include/asm/irq.h
 create mode 100644 arch/um/lkl/include/asm/page.h
 create mode 100644 arch/um/lkl/include/asm/pgtable.h
 create mode 100644 arch/um/lkl/include/asm/processor.h
 create mode 100644 arch/um/lkl/include/asm/ptrace.h
 create mode 100644 arch/um/lkl/include/asm/sched.h
 create mode 100644 arch/um/lkl/include/asm/setup.h
 create mode 100644 arch/um/lkl/include/asm/syscalls.h
 create mode 100644 arch/um/lkl/include/asm/syscalls_32.h
 create mode 100644 arch/um/lkl/include/asm/thread_info.h
 create mode 100644 arch/um/lkl/include/asm/tlb.h
 create mode 100644 arch/um/lkl/include/asm/uaccess.h
 create mode 100644 arch/um/lkl/include/asm/unistd.h
 create mode 100644 arch/um/lkl/include/asm/unistd_32.h
 create mode 100644 arch/um/lkl/include/asm/vmlinux.lds.h
 create mode 100644 arch/um/lkl/include/asm/xor.h
 create mode 100644 arch/um/lkl/include/uapi/asm/Kbuild
 create mode 100644 arch/um/lkl/include/uapi/asm/bitsperlong.h
 create mode 100644 arch/um/lkl/include/uapi/asm/byteorder.h
 create mode 100644 arch/um/lkl/include/uapi/asm/host_ops.h
 create mode 100644 arch/um/lkl/include/uapi/asm/irq.h
 create mode 100644 arch/um/lkl/include/uapi/asm/sigcontext.h
 create mode 100644 arch/um/lkl/include/uapi/asm/siginfo.h
 create mode 100644 arch/um/lkl/include/uapi/asm/swab.h
 create mode 100644 arch/um/lkl/include/uapi/asm/syscalls.h
 create mode 100644 arch/um/lkl/include/uapi/asm/unistd.h
 create mode 100644 arch/um/lkl/kernel/Makefile
 create mode 100644 arch/um/lkl/kernel/asm-offsets.c
 create mode 100644 arch/um/lkl/kernel/console.c
 create mode 100644 arch/um/lkl/kernel/cpu.c
 create mode 100644 arch/um/lkl/kernel/irq.c
 create mode 100644 arch/um/lkl/kernel/misc.c
 create mode 100644 arch/um/lkl/kernel/setup.c
 create mode 100644 arch/um/lkl/kernel/syscalls.c
 create mode 100644 arch/um/lkl/kernel/syscalls_32.c
 create mode 100644 arch/um/lkl/kernel/threads.c
 create mode 100644 arch/um/lkl/kernel/time.c
 create mode 100644 arch/um/lkl/kernel/vmlinux.lds.S
 create mode 100644 arch/um/lkl/mm/Makefile
 create mode 100644 arch/um/lkl/mm/bootmem.c
 create mode 100755 arch/um/lkl/scripts/headers_install.py
 create mode 100644 tools/lkl/.gitignore
 create mode 100644 tools/lkl/Build
 create mode 100644 tools/lkl/Makefile
 create mode 100644 tools/lkl/Makefile.autoconf
 create mode 100644 tools/lkl/Targets
 create mode 100644 tools/lkl/cptofs.c
 create mode 100644 tools/lkl/fs2tar.c
 create mode 100644 tools/lkl/include/.gitignore
 create mode 100644 tools/lkl/include/lkl.h
 create mode 100644 tools/lkl/include/lkl_config.h
 create mode 100644 tools/lkl/include/lkl_host.h
 create mode 100644 tools/lkl/lib/.gitignore
 create mode 100644 tools/lkl/lib/Build
 create mode 100644 tools/lkl/lib/config.c
 create mode 100644 tools/lkl/lib/dbg.c
 create mode 100644 tools/lkl/lib/dbg_handler.c
 create mode 100644 tools/lkl/lib/endian.h
 create mode 100644 tools/lkl/lib/fs.c
 create mode 100644 tools/lkl/lib/jmp_buf.c
 create mode 100644 tools/lkl/lib/jmp_buf.h
 create mode 100644 tools/lkl/lib/net.c
 create mode 100644 tools/lkl/lib/posix-host.c
 create mode 100644 tools/lkl/lib/um/Build
 create mode 100644 tools/lkl/lib/um/um_block.c
 create mode 100644 tools/lkl/lib/um/um_glue.c
 create mode 100644 tools/lkl/lib/um/um_net.c
 create mode 100644 tools/lkl/lib/utils.c
 create mode 100644 tools/lkl/lklfuse.c
 create mode 100755 tools/lkl/scripts/checkpatch.sh
 create mode 100755 tools/lkl/scripts/lkl-jenkins.sh
 create mode 100644 tools/lkl/tests/Build
 create mode 100644 tools/lkl/tests/boot.c
 create mode 100755 tools/lkl/tests/boot.sh
 create mode 100644 tools/lkl/tests/cla.c
 create mode 100644 tools/lkl/tests/cla.h
 create mode 100644 tools/lkl/tests/disk.c
 create mode 100755 tools/lkl/tests/disk.sh
 create mode 100755 tools/lkl/tests/lklfuse.sh
 create mode 100644 tools/lkl/tests/net-setup.sh
 create mode 100644 tools/lkl/tests/net-test.c
 create mode 100755 tools/lkl/tests/net.sh
 create mode 100755 tools/lkl/tests/run.py
 create mode 100644 tools/lkl/tests/tap13.py
 create mode 100644 tools/lkl/tests/test.c
 create mode 100644 tools/lkl/tests/test.h
 create mode 100644 tools/lkl/tests/test.sh
 create mode 100644 tools/lkl/tests/valgrind.supp
 create mode 100755 tools/lkl/tests/valgrind2xunit.py

-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply	[flat|nested] 476+ messages in thread
* [RFC v3 01/26] asm-generic: atomic64: allow using generic atomic64 on 64bit platforms
  2020-02-05  7:30 [RFC v3 00/26] Unifying LKL into UML Hajime Tazaki
@ 2020-02-05  7:30 ` Hajime Tazaki
  2020-02-05  9:34   ` Peter Zijlstra
       [not found] ` <cover.1580882335.git.thehajime-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
                   ` (25 subsequent siblings)
  26 siblings, 1 reply; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05  7:30 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Will Deacon, Peter Zijlstra, Boqun Feng

From: Octavian Purdila <tavi.purdila@gmail.com>

With CONFIG_64BIT enabled, atomic64 via CONFIG_GENERIC_ATOMIC64 options
are not compiled due to type conflict of atomic64_t defined in
linux/type.h.

This commit fixes the issue and allow using generic atomic64 ops.

Currently, LKL is only the user which defines GENERIC_ATOMIC64
(lib/atomic64.c) under CONFIG_64BIT environment.  Thus, there is no
issues before this commit.

Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
Cc: Will Deacon <will@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Boqun Feng <boqun.feng@gmail.com>
---
 include/asm-generic/atomic64.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/include/asm-generic/atomic64.h b/include/asm-generic/atomic64.h
index 370f01d4450f..9b15847baae5 100644
--- a/include/asm-generic/atomic64.h
+++ b/include/asm-generic/atomic64.h
@@ -9,9 +9,11 @@
 #define _ASM_GENERIC_ATOMIC64_H
 #include <linux/types.h>
 
+#ifndef CONFIG_64BIT
 typedef struct {
 	s64 counter;
 } atomic64_t;
+#endif
 
 #define ATOMIC64_INIT(i)	{ (i) }
 
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v3 02/26] arch: add __SYSCALL_DEFINE_ARCH
@ 2020-02-05  7:30     ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05  7:30 UTC (permalink / raw)
  To: linux-um-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r
  Cc: Octavian Purdila, Akira Moroo,
	linux-kernel-library-uGLqWuYN4qMgsBAKwltoeQ,
	linux-arch-u79uwXL29TY76Z2rM5mHXA,
	linux-api-u79uwXL29TY76Z2rM5mHXA

From: Octavian Purdila <tavi.purdila-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>

This allows the architecture code to process the system call
definitions. It is used by LKL to create strong typed function
definitions for system calls.

Signed-off-by: Octavian Purdila <tavi.purdila-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Cc: linux-api-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
---
 include/linux/syscalls.h | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/include/linux/syscalls.h b/include/linux/syscalls.h
index 5262b7a76d39..d61069aa869a 100644
--- a/include/linux/syscalls.h
+++ b/include/linux/syscalls.h
@@ -203,9 +203,14 @@ static inline int is_syscall_trace_event(struct trace_event_call *tp_event)
 }
 #endif
 
+#ifndef __SYSCALL_DEFINE_ARCH
+#define __SYSCALL_DEFINE_ARCH(x, sname, ...)
+#endif
+
 #ifndef SYSCALL_DEFINE0
 #define SYSCALL_DEFINE0(sname)					\
 	SYSCALL_METADATA(_##sname, 0);				\
+	__SYSCALL_DEFINE_ARCH(0, _##sname);			\
 	asmlinkage long sys_##sname(void);			\
 	ALLOW_ERROR_INJECTION(sys_##sname, ERRNO);		\
 	asmlinkage long sys_##sname(void)
@@ -222,6 +227,7 @@ static inline int is_syscall_trace_event(struct trace_event_call *tp_event)
 
 #define SYSCALL_DEFINEx(x, sname, ...)				\
 	SYSCALL_METADATA(sname, x, __VA_ARGS__)			\
+	__SYSCALL_DEFINE_ARCH(x, sname, __VA_ARGS__)		\
 	__SYSCALL_DEFINEx(x, sname, __VA_ARGS__)
 
 #define __PROTECT(...) asmlinkage_protect(__VA_ARGS__)
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v3 02/26] arch: add __SYSCALL_DEFINE_ARCH
@ 2020-02-05  7:30     ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05  7:30 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	linux-api

From: Octavian Purdila <tavi.purdila@gmail.com>

This allows the architecture code to process the system call
definitions. It is used by LKL to create strong typed function
definitions for system calls.

Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
Cc: linux-api@vger.kernel.org
---
 include/linux/syscalls.h | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/include/linux/syscalls.h b/include/linux/syscalls.h
index 5262b7a76d39..d61069aa869a 100644
--- a/include/linux/syscalls.h
+++ b/include/linux/syscalls.h
@@ -203,9 +203,14 @@ static inline int is_syscall_trace_event(struct trace_event_call *tp_event)
 }
 #endif
 
+#ifndef __SYSCALL_DEFINE_ARCH
+#define __SYSCALL_DEFINE_ARCH(x, sname, ...)
+#endif
+
 #ifndef SYSCALL_DEFINE0
 #define SYSCALL_DEFINE0(sname)					\
 	SYSCALL_METADATA(_##sname, 0);				\
+	__SYSCALL_DEFINE_ARCH(0, _##sname);			\
 	asmlinkage long sys_##sname(void);			\
 	ALLOW_ERROR_INJECTION(sys_##sname, ERRNO);		\
 	asmlinkage long sys_##sname(void)
@@ -222,6 +227,7 @@ static inline int is_syscall_trace_event(struct trace_event_call *tp_event)
 
 #define SYSCALL_DEFINEx(x, sname, ...)				\
 	SYSCALL_METADATA(sname, x, __VA_ARGS__)			\
+	__SYSCALL_DEFINE_ARCH(x, sname, __VA_ARGS__)		\
 	__SYSCALL_DEFINEx(x, sname, __VA_ARGS__)
 
 #define __PROTECT(...) asmlinkage_protect(__VA_ARGS__)
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v3 03/26] um lkl: architecture skeleton for Linux kernel library
  2020-02-05  7:30 [RFC v3 00/26] Unifying LKL into UML Hajime Tazaki
@ 2020-02-05  7:30   ` Hajime Tazaki
       [not found] ` <cover.1580882335.git.thehajime-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
                     ` (25 subsequent siblings)
  26 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05  7:30 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Conrad Meyer, Edison M . Castro, Jens Staal, Lai Jiangshan,
	Levente Kurusa, Luca Dariz, Mark Stillwell, Matthieu Coudron,
	Michael Zimmermann, Motomu Utsumi, Patrick Collins,
	Petros Angelatos, Pierre-Hugues Husson, Xiao Jia, Yuan Liu,
	Hajime Tazaki

From: Octavian Purdila <tavi.purdila@gmail.com>

Adds the LKL Kconfig, vmlinux linker script, basic architecture
headers and miscellaneous basic functions or stubs such as
dump_stack(), show_regs() and cpuinfo proc ops.

The headers we introduce in this patch are simple wrappers to the
asm-generic headers or stubs for things we don't support, such as
ptrace, DMA, signals, ELF handling and low level processor operations.

The kernel configuration is automatically updated to reflect the
endianness of the host, 64bit support or the output format for
vmlinux's linker script. We do this by looking at the ld's default
output format.

Cc: Andreas Abel <aabel@google.com>
Cc: Conrad Meyer <cem@FreeBSD.org>
Cc: Edison M. Castro <edisonmcastro@hotmail.com>
Cc: H.K. Jerry Chu <hkchu@google.com>
Cc: Jens Staal <staal1978@gmail.com>
Cc: Lai Jiangshan <jiangshanlai@gmail.com>
Cc: Levente Kurusa <levex@linux.com>
Cc: Luca Dariz <luca.dariz@gmail.com>
Cc: Mark Stillwell <mark@stillwell.me>
Cc: Matthieu Coudron <mattator@gmail.com>
Cc: Michael Zimmermann <sigmaepsilon92@gmail.com>
Cc: Motomu Utsumi <motomuman@gmail.com>
Cc: Patrick Collins <pscollins@google.com>
Cc: Petros Angelatos <petrosagg@gmail.com>
Cc: Pierre-Hugues Husson <phh@phh.me>
Cc: Xiao Jia <xiaoj@google.com>
Cc: Yuan Liu <liuyuan@google.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 MAINTAINERS                                |    8 +
 arch/um/lkl/.gitignore                     |    2 +
 arch/um/lkl/Kconfig                        |   56 ++
 arch/um/lkl/Kconfig.debug                  |    0
 arch/um/lkl/configs/lkl_defconfig          | 1052 ++++++++++++++++++++
 arch/um/lkl/include/asm/Kbuild             |   80 ++
 arch/um/lkl/include/asm/bitsperlong.h      |   11 +
 arch/um/lkl/include/asm/byteorder.h        |    7 +
 arch/um/lkl/include/asm/cpu.h              |   14 +
 arch/um/lkl/include/asm/elf.h              |   15 +
 arch/um/lkl/include/asm/processor.h        |   60 ++
 arch/um/lkl/include/asm/ptrace.h           |   25 +
 arch/um/lkl/include/asm/sched.h            |   23 +
 arch/um/lkl/include/asm/syscalls.h         |   18 +
 arch/um/lkl/include/asm/syscalls_32.h      |   43 +
 arch/um/lkl/include/asm/tlb.h              |   12 +
 arch/um/lkl/include/asm/uaccess.h          |   64 ++
 arch/um/lkl/include/asm/unistd_32.h        |   31 +
 arch/um/lkl/include/asm/vmlinux.lds.h      |   14 +
 arch/um/lkl/include/asm/xor.h              |    9 +
 arch/um/lkl/include/uapi/asm/Kbuild        |    9 +
 arch/um/lkl/include/uapi/asm/bitsperlong.h |   13 +
 arch/um/lkl/include/uapi/asm/byteorder.h   |   11 +
 arch/um/lkl/include/uapi/asm/siginfo.h     |   11 +
 arch/um/lkl/include/uapi/asm/swab.h        |   11 +
 arch/um/lkl/include/uapi/asm/syscalls.h    |  348 +++++++
 arch/um/lkl/kernel/asm-offsets.c           |    2 +
 arch/um/lkl/kernel/misc.c                  |   60 ++
 arch/um/lkl/kernel/vmlinux.lds.S           |   51 +
 29 files changed, 2060 insertions(+)
 create mode 100644 arch/um/lkl/.gitignore
 create mode 100644 arch/um/lkl/Kconfig
 create mode 100644 arch/um/lkl/Kconfig.debug
 create mode 100644 arch/um/lkl/configs/lkl_defconfig
 create mode 100644 arch/um/lkl/include/asm/Kbuild
 create mode 100644 arch/um/lkl/include/asm/bitsperlong.h
 create mode 100644 arch/um/lkl/include/asm/byteorder.h
 create mode 100644 arch/um/lkl/include/asm/cpu.h
 create mode 100644 arch/um/lkl/include/asm/elf.h
 create mode 100644 arch/um/lkl/include/asm/processor.h
 create mode 100644 arch/um/lkl/include/asm/ptrace.h
 create mode 100644 arch/um/lkl/include/asm/sched.h
 create mode 100644 arch/um/lkl/include/asm/syscalls.h
 create mode 100644 arch/um/lkl/include/asm/syscalls_32.h
 create mode 100644 arch/um/lkl/include/asm/tlb.h
 create mode 100644 arch/um/lkl/include/asm/uaccess.h
 create mode 100644 arch/um/lkl/include/asm/unistd_32.h
 create mode 100644 arch/um/lkl/include/asm/vmlinux.lds.h
 create mode 100644 arch/um/lkl/include/asm/xor.h
 create mode 100644 arch/um/lkl/include/uapi/asm/Kbuild
 create mode 100644 arch/um/lkl/include/uapi/asm/bitsperlong.h
 create mode 100644 arch/um/lkl/include/uapi/asm/byteorder.h
 create mode 100644 arch/um/lkl/include/uapi/asm/siginfo.h
 create mode 100644 arch/um/lkl/include/uapi/asm/swab.h
 create mode 100644 arch/um/lkl/include/uapi/asm/syscalls.h
 create mode 100644 arch/um/lkl/kernel/asm-offsets.c
 create mode 100644 arch/um/lkl/kernel/misc.c
 create mode 100644 arch/um/lkl/kernel/vmlinux.lds.S

diff --git a/MAINTAINERS b/MAINTAINERS
index 4017e6b760be..12afa00402a1 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -9606,6 +9606,14 @@ F:	Documentation/core-api/atomic_ops.rst
 F:	Documentation/core-api/refcount-vs-atomic.rst
 F:	Documentation/memory-barriers.txt
 
+LINUX KERNEL LIBRARY
+M:	Octavian Purdila <tavi.purdila@gmail.com>
+M:	Hajime Tazaki <thehajime@gmail.com>
+L:	linux-kernel-library@freelists.org
+S:	Maintained
+F:	arch/um/lkl/
+F:	tools/lkl/
+
 LIS3LV02D ACCELEROMETER DRIVER
 M:	Eric Piel <eric.piel@tremplin-utc.net>
 S:	Maintained
diff --git a/arch/um/lkl/.gitignore b/arch/um/lkl/.gitignore
new file mode 100644
index 000000000000..ced1c60d8235
--- /dev/null
+++ b/arch/um/lkl/.gitignore
@@ -0,0 +1,2 @@
+kernel/vmlinux.lds
+include/generated
diff --git a/arch/um/lkl/Kconfig b/arch/um/lkl/Kconfig
new file mode 100644
index 000000000000..f7b641ea7aef
--- /dev/null
+++ b/arch/um/lkl/Kconfig
@@ -0,0 +1,56 @@
+# SPDX-License-Identifier: GPL-2.0
+
+config UML_LKL
+       def_bool y
+       depends on !SMP && !MMU && !COREDUMP && !SECCOMP && !UPROBES && !COMPAT && !USER_RETURN_NOTIFIER
+       select ARCH_THREAD_STACK_ALLOCATOR
+       select RWSEM_GENERIC_SPINLOCK
+       select GENERIC_ATOMIC64
+       select GENERIC_HWEIGHT
+       select FLATMEM
+       select FLAT_NODE_MEM_MAP
+       select GENERIC_CLOCKEVENTS
+       select GENERIC_CPU_DEVICES
+       select NO_HZ_IDLE
+       select NO_PREEMPT
+       select ARCH_WANT_FRAME_POINTERS
+       select HAS_DMA
+       select DMA_DIRECT_OPS
+       select PHYS_ADDR_T_64BIT if 64BIT
+       select 64BIT if "$(OUTPUT_FORMAT)" = "elf64-x86-64"
+       select 64BIT if "$(OUTPUT_FORMAT)" = "elf64-x86-64-freebsd"
+       select NET
+       select MULTIUSER
+
+config OUTPUT_FORMAT
+       string "Output format"
+       default "$(OUTPUT_FORMAT)"
+
+config ARCH_DMA_ADDR_T_64BIT
+       def_bool 64BIT
+
+config 64BIT
+       def_bool n
+
+config COREDUMP
+       def_bool n
+
+config BIG_ENDIAN
+       def_bool n
+
+config GENERIC_CSUM
+       def_bool y
+
+config GENERIC_HWEIGHT
+       def_bool y
+
+config NO_IOPORT_MAP
+       def_bool y
+
+config RWSEM_GENERIC_SPINLOCK
+	bool
+	default y
+
+config HZ
+        int
+        default 100
diff --git a/arch/um/lkl/Kconfig.debug b/arch/um/lkl/Kconfig.debug
new file mode 100644
index 000000000000..e69de29bb2d1
diff --git a/arch/um/lkl/configs/lkl_defconfig b/arch/um/lkl/configs/lkl_defconfig
new file mode 100644
index 000000000000..7050c233a17e
--- /dev/null
+++ b/arch/um/lkl/configs/lkl_defconfig
@@ -0,0 +1,1052 @@
+CONFIG_CC_IS_GCC=y
+CONFIG_GCC_VERSION=80301
+CONFIG_CLANG_VERSION=0
+CONFIG_CC_CAN_LINK=y
+CONFIG_CC_HAS_ASM_GOTO=y
+CONFIG_CC_HAS_ASM_INLINE=y
+CONFIG_CC_HAS_WARN_MAYBE_UNINITIALIZED=y
+CONFIG_IRQ_WORK=y
+CONFIG_BROKEN_ON_SMP=y
+CONFIG_INIT_ENV_ARG_LIMIT=32
+# CONFIG_COMPILE_TEST is not set
+CONFIG_LOCALVERSION=""
+# CONFIG_LOCALVERSION_AUTO is not set
+CONFIG_BUILD_SALT=""
+CONFIG_DEFAULT_HOSTNAME="(none)"
+# CONFIG_SYSVIPC is not set
+# CONFIG_POSIX_MQUEUE is not set
+# CONFIG_USELIB is not set
+# CONFIG_AUDIT is not set
+CONFIG_GENERIC_CLOCKEVENTS=y
+CONFIG_TICK_ONESHOT=y
+CONFIG_NO_HZ_COMMON=y
+# CONFIG_HZ_PERIODIC is not set
+CONFIG_NO_HZ_IDLE=y
+# CONFIG_NO_HZ is not set
+CONFIG_HIGH_RES_TIMERS=y
+CONFIG_PREEMPT_NONE=y
+# CONFIG_PREEMPT_VOLUNTARY is not set
+# CONFIG_PREEMPT is not set
+CONFIG_TICK_CPU_ACCOUNTING=y
+# CONFIG_BSD_PROCESS_ACCT is not set
+# CONFIG_TASKSTATS is not set
+# CONFIG_PSI is not set
+CONFIG_TINY_RCU=y
+# CONFIG_RCU_EXPERT is not set
+CONFIG_SRCU=y
+CONFIG_TINY_SRCU=y
+# CONFIG_IKCONFIG is not set
+# CONFIG_IKHEADERS is not set
+CONFIG_LOG_BUF_SHIFT=17
+CONFIG_PRINTK_SAFE_LOG_BUF_SHIFT=13
+CONFIG_CC_HAS_INT128=y
+# CONFIG_CGROUPS is not set
+# CONFIG_NAMESPACES is not set
+# CONFIG_CHECKPOINT_RESTORE is not set
+# CONFIG_SCHED_AUTOGROUP is not set
+# CONFIG_SYSFS_DEPRECATED is not set
+# CONFIG_RELAY is not set
+# CONFIG_BLK_DEV_INITRD is not set
+CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE=y
+# CONFIG_CC_OPTIMIZE_FOR_SIZE is not set
+CONFIG_SYSCTL=y
+CONFIG_BPF=y
+CONFIG_EXPERT=y
+CONFIG_MULTIUSER=y
+# CONFIG_SGETMASK_SYSCALL is not set
+# CONFIG_SYSFS_SYSCALL is not set
+CONFIG_FHANDLE=y
+CONFIG_POSIX_TIMERS=y
+# CONFIG_KALLSYMS_USE_DATA_SECTION is not set
+CONFIG_PRINTK=y
+CONFIG_BUG=y
+# CONFIG_BASE_FULL is not set
+# CONFIG_FUTEX is not set
+CONFIG_EPOLL=y
+# CONFIG_SIGNALFD is not set
+# CONFIG_TIMERFD is not set
+CONFIG_EVENTFD=y
+# CONFIG_AIO is not set
+CONFIG_IO_URING=y
+# CONFIG_ADVISE_SYSCALLS is not set
+CONFIG_MEMBARRIER=y
+CONFIG_KALLSYMS=y
+CONFIG_KALLSYMS_ALL=y
+CONFIG_KALLSYMS_BASE_RELATIVE=y
+# CONFIG_BPF_SYSCALL is not set
+CONFIG_EMBEDDED=y
+# CONFIG_PC104 is not set
+# CONFIG_VM_EVENT_COUNTERS is not set
+CONFIG_SLUB_DEBUG=y
+# CONFIG_COMPAT_BRK is not set
+# CONFIG_SLAB is not set
+CONFIG_SLUB=y
+# CONFIG_SLOB is not set
+CONFIG_SLAB_MERGE_DEFAULT=y
+# CONFIG_SLAB_FREELIST_RANDOM is not set
+# CONFIG_SLAB_FREELIST_HARDENED is not set
+# CONFIG_SHUFFLE_PAGE_ALLOCATOR is not set
+# CONFIG_MMAP_ALLOW_UNINITIALIZED is not set
+# CONFIG_PROFILING is not set
+CONFIG_LKL=y
+# CONFIG_COREDUMP is not set
+CONFIG_GENERIC_CSUM=y
+CONFIG_GENERIC_HWEIGHT=y
+CONFIG_NO_IOPORT_MAP=y
+CONFIG_RWSEM_GENERIC_SPINLOCK=y
+CONFIG_HZ=100
+# CONFIG_STDERR_CONSOLE is not set
+# CONFIG_SSL is not set
+# CONFIG_NULL_CHAN is not set
+# CONFIG_PORT_CHAN is not set
+# CONFIG_PTY_CHAN is not set
+# CONFIG_TTY_CHAN is not set
+# CONFIG_XTERM_CHAN is not set
+CONFIG_NOCONFIG_CHAN=y
+CONFIG_CON_ZERO_CHAN="fd:0,fd:1"
+CONFIG_CON_CHAN="xterm"
+CONFIG_SSL_CHAN="pty"
+# CONFIG_UML_SOUND is not set
+# CONFIG_SOUND is not set
+CONFIG_UML_NET=y
+# CONFIG_UML_NET_ETHERTAP is not set
+CONFIG_UML_NET_TUNTAP=y
+# CONFIG_UML_NET_SLIP is not set
+# CONFIG_UML_NET_DAEMON is not set
+# CONFIG_UML_NET_VECTOR is not set
+# CONFIG_UML_NET_VDE is not set
+# CONFIG_UML_NET_MCAST is not set
+# CONFIG_UML_NET_PCAP is not set
+CONFIG_UML_NET_SLIRP=y
+# CONFIG_VIRTIO_UML is not set
+CONFIG_ARCH_THREAD_STACK_ALLOCATOR=y
+CONFIG_CC_HAS_STACKPROTECTOR_NONE=y
+CONFIG_HAVE_VIRT_CPU_ACCOUNTING_GEN=y
+CONFIG_PGTABLE_LEVELS=2
+CONFIG_PLUGIN_HOSTCC=""
+CONFIG_BASE_SMALL=1
+# CONFIG_MODULES is not set
+CONFIG_BLOCK=y
+# CONFIG_BLK_DEV_BSG is not set
+# CONFIG_BLK_DEV_BSGLIB is not set
+# CONFIG_BLK_DEV_INTEGRITY is not set
+# CONFIG_BLK_DEV_ZONED is not set
+# CONFIG_BLK_CMDLINE_PARSER is not set
+# CONFIG_BLK_WBT is not set
+# CONFIG_BLK_SED_OPAL is not set
+# CONFIG_PARTITION_ADVANCED is not set
+CONFIG_MSDOS_PARTITION=y
+CONFIG_EFI_PARTITION=y
+CONFIG_MQ_IOSCHED_DEADLINE=y
+CONFIG_MQ_IOSCHED_KYBER=y
+# CONFIG_IOSCHED_BFQ is not set
+CONFIG_INLINE_SPIN_UNLOCK_IRQ=y
+CONFIG_INLINE_READ_UNLOCK=y
+CONFIG_INLINE_READ_UNLOCK_IRQ=y
+CONFIG_INLINE_WRITE_UNLOCK=y
+CONFIG_INLINE_WRITE_UNLOCK_IRQ=y
+CONFIG_BINFMT_SCRIPT=y
+# CONFIG_BINFMT_MISC is not set
+CONFIG_FLATMEM=y
+CONFIG_FLAT_NODE_MEM_MAP=y
+CONFIG_SPLIT_PTLOCK_CPUS=999999
+CONFIG_PHYS_ADDR_T_64BIT=y
+CONFIG_NOMMU_INITIAL_TRIM_EXCESS=1
+CONFIG_NEED_PER_CPU_KM=y
+# CONFIG_CLEANCACHE is not set
+# CONFIG_ZPOOL is not set
+# CONFIG_ZBUD is not set
+# CONFIG_PERCPU_STATS is not set
+# CONFIG_GUP_BENCHMARK is not set
+CONFIG_NET=y
+# CONFIG_PACKET is not set
+# CONFIG_UNIX is not set
+# CONFIG_TLS is not set
+# CONFIG_XFRM_USER is not set
+# CONFIG_NET_KEY is not set
+CONFIG_INET=y
+CONFIG_IP_MULTICAST=y
+CONFIG_IP_ADVANCED_ROUTER=y
+# CONFIG_IP_FIB_TRIE_STATS is not set
+CONFIG_IP_MULTIPLE_TABLES=y
+CONFIG_IP_ROUTE_MULTIPATH=y
+# CONFIG_IP_ROUTE_VERBOSE is not set
+CONFIG_IP_PNP=y
+CONFIG_IP_PNP_DHCP=y
+# CONFIG_IP_PNP_BOOTP is not set
+# CONFIG_IP_PNP_RARP is not set
+# CONFIG_NET_IPIP is not set
+# CONFIG_NET_IPGRE_DEMUX is not set
+CONFIG_NET_IP_TUNNEL=y
+# CONFIG_IP_MROUTE is not set
+# CONFIG_SYN_COOKIES is not set
+# CONFIG_NET_IPVTI is not set
+# CONFIG_NET_FOU is not set
+# CONFIG_NET_FOU_IP_TUNNELS is not set
+# CONFIG_INET_AH is not set
+# CONFIG_INET_ESP is not set
+# CONFIG_INET_IPCOMP is not set
+CONFIG_INET_TUNNEL=y
+CONFIG_INET_DIAG=y
+CONFIG_INET_TCP_DIAG=y
+# CONFIG_INET_UDP_DIAG is not set
+# CONFIG_INET_RAW_DIAG is not set
+# CONFIG_INET_DIAG_DESTROY is not set
+CONFIG_TCP_CONG_ADVANCED=y
+CONFIG_TCP_CONG_BIC=y
+CONFIG_TCP_CONG_CUBIC=y
+CONFIG_TCP_CONG_WESTWOOD=y
+CONFIG_TCP_CONG_HTCP=y
+# CONFIG_TCP_CONG_HSTCP is not set
+# CONFIG_TCP_CONG_HYBLA is not set
+# CONFIG_TCP_CONG_VEGAS is not set
+# CONFIG_TCP_CONG_NV is not set
+# CONFIG_TCP_CONG_SCALABLE is not set
+# CONFIG_TCP_CONG_LP is not set
+# CONFIG_TCP_CONG_VENO is not set
+# CONFIG_TCP_CONG_YEAH is not set
+# CONFIG_TCP_CONG_ILLINOIS is not set
+# CONFIG_TCP_CONG_DCTCP is not set
+# CONFIG_TCP_CONG_CDG is not set
+CONFIG_TCP_CONG_BBR=y
+# CONFIG_DEFAULT_BIC is not set
+CONFIG_DEFAULT_CUBIC=y
+# CONFIG_DEFAULT_HTCP is not set
+# CONFIG_DEFAULT_WESTWOOD is not set
+# CONFIG_DEFAULT_BBR is not set
+# CONFIG_DEFAULT_RENO is not set
+CONFIG_DEFAULT_TCP_CONG="cubic"
+# CONFIG_TCP_MD5SIG is not set
+CONFIG_IPV6=y
+# CONFIG_IPV6_ROUTER_PREF is not set
+# CONFIG_IPV6_OPTIMISTIC_DAD is not set
+# CONFIG_INET6_AH is not set
+# CONFIG_INET6_ESP is not set
+# CONFIG_INET6_IPCOMP is not set
+# CONFIG_IPV6_MIP6 is not set
+# CONFIG_IPV6_VTI is not set
+CONFIG_IPV6_SIT=y
+# CONFIG_IPV6_SIT_6RD is not set
+CONFIG_IPV6_NDISC_NODETYPE=y
+# CONFIG_IPV6_TUNNEL is not set
+CONFIG_IPV6_MULTIPLE_TABLES=y
+# CONFIG_IPV6_SUBTREES is not set
+# CONFIG_IPV6_MROUTE is not set
+# CONFIG_IPV6_SEG6_LWTUNNEL is not set
+# CONFIG_IPV6_SEG6_HMAC is not set
+# CONFIG_NETWORK_SECMARK is not set
+# CONFIG_NETWORK_PHY_TIMESTAMPING is not set
+# CONFIG_NETFILTER is not set
+# CONFIG_BPFILTER is not set
+# CONFIG_IP_DCCP is not set
+# CONFIG_IP_SCTP is not set
+# CONFIG_RDS is not set
+# CONFIG_TIPC is not set
+# CONFIG_ATM is not set
+# CONFIG_L2TP is not set
+# CONFIG_BRIDGE is not set
+CONFIG_HAVE_NET_DSA=y
+# CONFIG_NET_DSA is not set
+# CONFIG_VLAN_8021Q is not set
+# CONFIG_DECNET is not set
+# CONFIG_LLC2 is not set
+# CONFIG_ATALK is not set
+# CONFIG_X25 is not set
+# CONFIG_LAPB is not set
+# CONFIG_PHONET is not set
+# CONFIG_6LOWPAN is not set
+# CONFIG_IEEE802154 is not set
+CONFIG_NET_SCHED=y
+# CONFIG_NET_SCH_CBQ is not set
+# CONFIG_NET_SCH_HTB is not set
+# CONFIG_NET_SCH_HFSC is not set
+# CONFIG_NET_SCH_PRIO is not set
+# CONFIG_NET_SCH_MULTIQ is not set
+# CONFIG_NET_SCH_RED is not set
+# CONFIG_NET_SCH_SFB is not set
+# CONFIG_NET_SCH_SFQ is not set
+# CONFIG_NET_SCH_TEQL is not set
+# CONFIG_NET_SCH_TBF is not set
+# CONFIG_NET_SCH_CBS is not set
+# CONFIG_NET_SCH_ETF is not set
+# CONFIG_NET_SCH_TAPRIO is not set
+# CONFIG_NET_SCH_GRED is not set
+# CONFIG_NET_SCH_DSMARK is not set
+# CONFIG_NET_SCH_NETEM is not set
+# CONFIG_NET_SCH_DRR is not set
+# CONFIG_NET_SCH_MQPRIO is not set
+# CONFIG_NET_SCH_SKBPRIO is not set
+# CONFIG_NET_SCH_CHOKE is not set
+# CONFIG_NET_SCH_QFQ is not set
+# CONFIG_NET_SCH_CODEL is not set
+# CONFIG_NET_SCH_FQ_CODEL is not set
+# CONFIG_NET_SCH_CAKE is not set
+CONFIG_NET_SCH_FQ=y
+# CONFIG_NET_SCH_HHF is not set
+# CONFIG_NET_SCH_PIE is not set
+# CONFIG_NET_SCH_PLUG is not set
+# CONFIG_NET_SCH_DEFAULT is not set
+# CONFIG_NET_CLS_BASIC is not set
+# CONFIG_NET_CLS_TCINDEX is not set
+# CONFIG_NET_CLS_ROUTE4 is not set
+# CONFIG_NET_CLS_FW is not set
+# CONFIG_NET_CLS_U32 is not set
+# CONFIG_NET_CLS_RSVP is not set
+# CONFIG_NET_CLS_RSVP6 is not set
+# CONFIG_NET_CLS_FLOW is not set
+# CONFIG_NET_CLS_BPF is not set
+# CONFIG_NET_CLS_FLOWER is not set
+# CONFIG_NET_CLS_MATCHALL is not set
+# CONFIG_NET_EMATCH is not set
+# CONFIG_NET_CLS_ACT is not set
+CONFIG_NET_SCH_FIFO=y
+# CONFIG_DCB is not set
+# CONFIG_BATMAN_ADV is not set
+# CONFIG_OPENVSWITCH is not set
+# CONFIG_VSOCKETS is not set
+# CONFIG_NETLINK_DIAG is not set
+# CONFIG_MPLS is not set
+# CONFIG_NET_NSH is not set
+# CONFIG_HSR is not set
+# CONFIG_NET_SWITCHDEV is not set
+# CONFIG_NET_L3_MASTER_DEV is not set
+# CONFIG_NET_NCSI is not set
+CONFIG_NET_RX_BUSY_POLL=y
+CONFIG_BQL=y
+# CONFIG_NET_PKTGEN is not set
+# CONFIG_HAMRADIO is not set
+# CONFIG_CAN is not set
+# CONFIG_BT is not set
+# CONFIG_AF_RXRPC is not set
+# CONFIG_AF_KCM is not set
+CONFIG_FIB_RULES=y
+# CONFIG_WIRELESS is not set
+# CONFIG_WIMAX is not set
+# CONFIG_RFKILL is not set
+# CONFIG_NET_9P is not set
+# CONFIG_CAIF is not set
+# CONFIG_CEPH_LIB is not set
+# CONFIG_NFC is not set
+# CONFIG_PSAMPLE is not set
+# CONFIG_NET_IFE is not set
+# CONFIG_LWTUNNEL is not set
+CONFIG_DST_CACHE=y
+CONFIG_GRO_CELLS=y
+CONFIG_FAILOVER=y
+# CONFIG_PCCARD is not set
+# CONFIG_UEVENT_HELPER is not set
+# CONFIG_DEVTMPFS is not set
+CONFIG_STANDALONE=y
+CONFIG_PREVENT_FIRMWARE_BUILD=y
+# CONFIG_FW_LOADER is not set
+CONFIG_ALLOW_DEV_COREDUMP=y
+# CONFIG_DEBUG_DRIVER is not set
+# CONFIG_DEBUG_DEVRES is not set
+# CONFIG_DEBUG_TEST_DRIVER_REMOVE is not set
+CONFIG_GENERIC_CPU_DEVICES=y
+# CONFIG_CONNECTOR is not set
+# CONFIG_GNSS is not set
+# CONFIG_MTD is not set
+# CONFIG_OF is not set
+# CONFIG_PARPORT is not set
+CONFIG_BLK_DEV=y
+# CONFIG_BLK_DEV_NULL_BLK is not set
+# CONFIG_BLK_DEV_UBD_SYNC is not set
+CONFIG_BLK_DEV_COW_COMMON=y
+# CONFIG_BLK_DEV_LOOP is not set
+# CONFIG_BLK_DEV_DRBD is not set
+# CONFIG_BLK_DEV_NBD is not set
+# CONFIG_BLK_DEV_RAM is not set
+# CONFIG_CDROM_PKTCDVD is not set
+# CONFIG_ATA_OVER_ETH is not set
+# CONFIG_BLK_DEV_RBD is not set
+# CONFIG_NVME_FC is not set
+# CONFIG_DUMMY_IRQ is not set
+# CONFIG_ENCLOSURE_SERVICES is not set
+# CONFIG_SRAM is not set
+# CONFIG_XILINX_SDFEC is not set
+# CONFIG_C2PORT is not set
+# CONFIG_EEPROM_93CX6 is not set
+# CONFIG_VOP_BUS is not set
+# CONFIG_ECHO is not set
+CONFIG_SCSI_MOD=y
+# CONFIG_RAID_ATTRS is not set
+# CONFIG_SCSI is not set
+# CONFIG_ATA is not set
+# CONFIG_MD is not set
+# CONFIG_TARGET_CORE is not set
+CONFIG_NETDEVICES=y
+CONFIG_NET_CORE=y
+# CONFIG_BONDING is not set
+# CONFIG_DUMMY is not set
+# CONFIG_EQUALIZER is not set
+# CONFIG_NET_TEAM is not set
+# CONFIG_MACVLAN is not set
+# CONFIG_IPVLAN is not set
+# CONFIG_VXLAN is not set
+# CONFIG_GENEVE is not set
+# CONFIG_GTP is not set
+# CONFIG_MACSEC is not set
+# CONFIG_NETCONSOLE is not set
+# CONFIG_TUN is not set
+# CONFIG_TUN_VNET_CROSS_LE is not set
+# CONFIG_VETH is not set
+# CONFIG_NLMON is not set
+# CONFIG_ETHERNET is not set
+# CONFIG_MDIO_DEVICE is not set
+# CONFIG_PHYLIB is not set
+# CONFIG_PPP is not set
+# CONFIG_SLIP is not set
+# CONFIG_WLAN is not set
+# CONFIG_WAN is not set
+CONFIG_NET_FAILOVER=y
+# CONFIG_ISDN is not set
+# CONFIG_NVM is not set
+CONFIG_INPUT=y
+# CONFIG_INPUT_FF_MEMLESS is not set
+# CONFIG_INPUT_POLLDEV is not set
+# CONFIG_INPUT_SPARSEKMAP is not set
+# CONFIG_INPUT_MATRIXKMAP is not set
+# CONFIG_INPUT_MOUSEDEV is not set
+# CONFIG_INPUT_JOYDEV is not set
+# CONFIG_INPUT_EVDEV is not set
+# CONFIG_INPUT_EVBUG is not set
+CONFIG_INPUT_KEYBOARD=y
+CONFIG_KEYBOARD_ATKBD=y
+# CONFIG_KEYBOARD_LKKBD is not set
+# CONFIG_KEYBOARD_NEWTON is not set
+# CONFIG_KEYBOARD_OPENCORES is not set
+# CONFIG_KEYBOARD_STOWAWAY is not set
+# CONFIG_KEYBOARD_SUNKBD is not set
+# CONFIG_KEYBOARD_XTKBD is not set
+CONFIG_INPUT_MOUSE=y
+CONFIG_MOUSE_PS2=y
+CONFIG_MOUSE_PS2_ALPS=y
+CONFIG_MOUSE_PS2_BYD=y
+CONFIG_MOUSE_PS2_LOGIPS2PP=y
+CONFIG_MOUSE_PS2_SYNAPTICS=y
+CONFIG_MOUSE_PS2_CYPRESS=y
+CONFIG_MOUSE_PS2_TRACKPOINT=y
+# CONFIG_MOUSE_PS2_ELANTECH is not set
+# CONFIG_MOUSE_PS2_SENTELIC is not set
+# CONFIG_MOUSE_PS2_TOUCHKIT is not set
+CONFIG_MOUSE_PS2_FOCALTECH=y
+# CONFIG_MOUSE_SERIAL is not set
+# CONFIG_MOUSE_APPLETOUCH is not set
+# CONFIG_MOUSE_BCM5974 is not set
+# CONFIG_MOUSE_VSXXXAA is not set
+# CONFIG_MOUSE_SYNAPTICS_USB is not set
+# CONFIG_INPUT_JOYSTICK is not set
+# CONFIG_INPUT_TABLET is not set
+# CONFIG_INPUT_TOUCHSCREEN is not set
+# CONFIG_INPUT_MISC is not set
+# CONFIG_RMI4_CORE is not set
+CONFIG_SERIO=y
+CONFIG_SERIO_SERPORT=y
+CONFIG_SERIO_LIBPS2=y
+# CONFIG_SERIO_RAW is not set
+# CONFIG_SERIO_ALTERA_PS2 is not set
+# CONFIG_SERIO_PS2MULT is not set
+# CONFIG_SERIO_ARC_PS2 is not set
+# CONFIG_USERIO is not set
+# CONFIG_GAMEPORT is not set
+CONFIG_TTY=y
+# CONFIG_VT is not set
+CONFIG_UNIX98_PTYS=y
+CONFIG_LEGACY_PTYS=y
+CONFIG_LEGACY_PTY_COUNT=256
+# CONFIG_SERIAL_NONSTANDARD is not set
+# CONFIG_N_GSM is not set
+# CONFIG_TRACE_SINK is not set
+# CONFIG_NULL_TTY is not set
+CONFIG_LDISC_AUTOLOAD=y
+CONFIG_DEVMEM=y
+# CONFIG_DEVKMEM is not set
+# CONFIG_SERIAL_8250 is not set
+# CONFIG_SERIAL_UARTLITE is not set
+# CONFIG_SERIAL_SCCNXP is not set
+# CONFIG_SERIAL_ALTERA_JTAGUART is not set
+# CONFIG_SERIAL_ALTERA_UART is not set
+# CONFIG_SERIAL_ARC is not set
+# CONFIG_SERIAL_FSL_LPUART is not set
+# CONFIG_SERIAL_FSL_LINFLEXUART is not set
+# CONFIG_SERIAL_DEV_BUS is not set
+# CONFIG_TTY_PRINTK is not set
+# CONFIG_IPMI_HANDLER is not set
+CONFIG_HW_RANDOM=y
+# CONFIG_HW_RANDOM_TIMERIOMEM is not set
+# CONFIG_RAW_DRIVER is not set
+# CONFIG_TCG_TPM is not set
+# CONFIG_RANDOM_TRUST_BOOTLOADER is not set
+# CONFIG_I2C is not set
+# CONFIG_I3C is not set
+# CONFIG_SPI is not set
+# CONFIG_SPMI is not set
+# CONFIG_HSI is not set
+# CONFIG_PPS is not set
+# CONFIG_PTP_1588_CLOCK is not set
+# CONFIG_PINCTRL is not set
+# CONFIG_GPIOLIB is not set
+# CONFIG_W1 is not set
+# CONFIG_POWER_AVS is not set
+# CONFIG_POWER_RESET is not set
+# CONFIG_POWER_SUPPLY is not set
+CONFIG_HWMON=y
+# CONFIG_HWMON_DEBUG_CHIP is not set
+# CONFIG_SENSORS_AS370 is not set
+# CONFIG_SENSORS_ASPEED is not set
+# CONFIG_SENSORS_F71805F is not set
+# CONFIG_SENSORS_F71882FG is not set
+# CONFIG_SENSORS_IT87 is not set
+# CONFIG_SENSORS_MAX197 is not set
+# CONFIG_SENSORS_PC87360 is not set
+# CONFIG_SENSORS_PC87427 is not set
+# CONFIG_SENSORS_NTC_THERMISTOR is not set
+# CONFIG_SENSORS_NCT6683 is not set
+# CONFIG_SENSORS_NCT6775 is not set
+# CONFIG_SENSORS_NPCM7XX is not set
+# CONFIG_SENSORS_SMSC47M1 is not set
+# CONFIG_SENSORS_SMSC47B397 is not set
+# CONFIG_SENSORS_VT1211 is not set
+# CONFIG_SENSORS_W83627HF is not set
+# CONFIG_SENSORS_W83627EHF is not set
+# CONFIG_THERMAL is not set
+# CONFIG_WATCHDOG is not set
+CONFIG_SSB_POSSIBLE=y
+# CONFIG_SSB is not set
+CONFIG_BCMA_POSSIBLE=y
+# CONFIG_BCMA is not set
+# CONFIG_MFD_MADERA is not set
+# CONFIG_HTC_PASIC3 is not set
+# CONFIG_MFD_KEMPLD is not set
+# CONFIG_MFD_MT6397 is not set
+# CONFIG_MFD_SM501 is not set
+# CONFIG_ABX500_CORE is not set
+# CONFIG_MFD_SYSCON is not set
+# CONFIG_MFD_TI_AM335X_TSCADC is not set
+# CONFIG_MFD_TQMX86 is not set
+# CONFIG_REGULATOR is not set
+# CONFIG_RC_CORE is not set
+# CONFIG_MEDIA_SUPPORT is not set
+# CONFIG_DRM is not set
+# CONFIG_DRM_DP_CEC is not set
+# CONFIG_FB is not set
+# CONFIG_LCD_CLASS_DEVICE is not set
+# CONFIG_BACKLIGHT_CLASS_DEVICE is not set
+CONFIG_HID=y
+# CONFIG_HID_BATTERY_STRENGTH is not set
+# CONFIG_HIDRAW is not set
+# CONFIG_UHID is not set
+CONFIG_HID_GENERIC=y
+# CONFIG_HID_A4TECH is not set
+# CONFIG_HID_ACRUX is not set
+# CONFIG_HID_APPLE is not set
+# CONFIG_HID_AUREAL is not set
+# CONFIG_HID_BELKIN is not set
+# CONFIG_HID_CHERRY is not set
+# CONFIG_HID_CHICONY is not set
+# CONFIG_HID_COUGAR is not set
+# CONFIG_HID_MACALLY is not set
+# CONFIG_HID_CMEDIA is not set
+# CONFIG_HID_CYPRESS is not set
+# CONFIG_HID_DRAGONRISE is not set
+# CONFIG_HID_EMS_FF is not set
+# CONFIG_HID_ELECOM is not set
+# CONFIG_HID_EZKEY is not set
+# CONFIG_HID_GEMBIRD is not set
+# CONFIG_HID_GFRM is not set
+# CONFIG_HID_KEYTOUCH is not set
+# CONFIG_HID_KYE is not set
+# CONFIG_HID_WALTOP is not set
+# CONFIG_HID_VIEWSONIC is not set
+# CONFIG_HID_GYRATION is not set
+# CONFIG_HID_ICADE is not set
+# CONFIG_HID_ITE is not set
+# CONFIG_HID_JABRA is not set
+# CONFIG_HID_TWINHAN is not set
+# CONFIG_HID_KENSINGTON is not set
+# CONFIG_HID_LCPOWER is not set
+# CONFIG_HID_LENOVO is not set
+# CONFIG_HID_MAGICMOUSE is not set
+# CONFIG_HID_MALTRON is not set
+# CONFIG_HID_MAYFLASH is not set
+# CONFIG_HID_REDRAGON is not set
+# CONFIG_HID_MICROSOFT is not set
+# CONFIG_HID_MONTEREY is not set
+# CONFIG_HID_MULTITOUCH is not set
+# CONFIG_HID_NTI is not set
+# CONFIG_HID_ORTEK is not set
+# CONFIG_HID_PANTHERLORD is not set
+# CONFIG_HID_PETALYNX is not set
+# CONFIG_HID_PICOLCD is not set
+# CONFIG_HID_PLANTRONICS is not set
+# CONFIG_HID_PRIMAX is not set
+# CONFIG_HID_SAITEK is not set
+# CONFIG_HID_SAMSUNG is not set
+# CONFIG_HID_SPEEDLINK is not set
+# CONFIG_HID_STEAM is not set
+# CONFIG_HID_STEELSERIES is not set
+# CONFIG_HID_SUNPLUS is not set
+# CONFIG_HID_RMI is not set
+# CONFIG_HID_GREENASIA is not set
+# CONFIG_HID_SMARTJOYPLUS is not set
+# CONFIG_HID_TIVO is not set
+# CONFIG_HID_TOPSEED is not set
+# CONFIG_HID_THRUSTMASTER is not set
+# CONFIG_HID_UDRAW_PS3 is not set
+# CONFIG_HID_XINMO is not set
+# CONFIG_HID_ZEROPLUS is not set
+# CONFIG_HID_ZYDACRON is not set
+# CONFIG_HID_SENSOR_HUB is not set
+# CONFIG_HID_ALPS is not set
+CONFIG_USB_OHCI_LITTLE_ENDIAN=y
+CONFIG_USB_SUPPORT=y
+# CONFIG_USB_ULPI_BUS is not set
+CONFIG_USB_ARCH_HAS_HCD=y
+# CONFIG_USB is not set
+# CONFIG_NOP_USB_XCEIV is not set
+# CONFIG_USB_GADGET is not set
+# CONFIG_TYPEC is not set
+# CONFIG_USB_ROLE_SWITCH is not set
+# CONFIG_MMC is not set
+# CONFIG_MEMSTICK is not set
+# CONFIG_NEW_LEDS is not set
+# CONFIG_ACCESSIBILITY is not set
+# CONFIG_INFINIBAND is not set
+# CONFIG_RTC_CLASS is not set
+# CONFIG_DMADEVICES is not set
+# CONFIG_SYNC_FILE is not set
+# CONFIG_AUXDISPLAY is not set
+# CONFIG_VIRT_DRIVERS is not set
+CONFIG_VIRTIO_MENU=y
+# CONFIG_VIRTIO_MMIO is not set
+# CONFIG_GREYBUS is not set
+# CONFIG_STAGING is not set
+# CONFIG_GOLDFISH is not set
+# CONFIG_HWSPINLOCK is not set
+# CONFIG_MAILBOX is not set
+# CONFIG_REMOTEPROC is not set
+# CONFIG_RPMSG_VIRTIO is not set
+# CONFIG_SOC_TI is not set
+# CONFIG_XILINX_VCU is not set
+# CONFIG_PM_DEVFREQ is not set
+# CONFIG_EXTCON is not set
+# CONFIG_MEMORY is not set
+# CONFIG_IIO is not set
+# CONFIG_PWM is not set
+# CONFIG_IPACK_BUS is not set
+# CONFIG_RESET_CONTROLLER is not set
+# CONFIG_GENERIC_PHY is not set
+# CONFIG_BCM_KONA_USB2_PHY is not set
+# CONFIG_PHY_PXA_28NM_HSIC is not set
+# CONFIG_PHY_PXA_28NM_USB2 is not set
+# CONFIG_POWERCAP is not set
+# CONFIG_MCB is not set
+# CONFIG_RAS is not set
+# CONFIG_ANDROID is not set
+# CONFIG_LIBNVDIMM is not set
+# CONFIG_DAX is not set
+# CONFIG_NVMEM is not set
+# CONFIG_STM is not set
+# CONFIG_INTEL_TH is not set
+# CONFIG_FPGA is not set
+# CONFIG_SIOX is not set
+# CONFIG_SLIMBUS is not set
+# CONFIG_INTERCONNECT is not set
+# CONFIG_COUNTER is not set
+# CONFIG_VALIDATE_FS_PARSER is not set
+CONFIG_FS_IOMAP=y
+# CONFIG_EXT2_FS is not set
+# CONFIG_EXT3_FS is not set
+CONFIG_EXT4_FS=y
+CONFIG_EXT4_USE_FOR_EXT2=y
+CONFIG_EXT4_FS_POSIX_ACL=y
+CONFIG_EXT4_FS_SECURITY=y
+# CONFIG_EXT4_DEBUG is not set
+CONFIG_JBD2=y
+# CONFIG_JBD2_DEBUG is not set
+CONFIG_FS_MBCACHE=y
+# CONFIG_REISERFS_FS is not set
+# CONFIG_JFS_FS is not set
+# CONFIG_XFS_FS is not set
+# CONFIG_GFS2_FS is not set
+CONFIG_BTRFS_FS=y
+# CONFIG_BTRFS_FS_POSIX_ACL is not set
+# CONFIG_BTRFS_FS_CHECK_INTEGRITY is not set
+# CONFIG_BTRFS_FS_RUN_SANITY_TESTS is not set
+# CONFIG_BTRFS_DEBUG is not set
+# CONFIG_BTRFS_ASSERT is not set
+# CONFIG_BTRFS_FS_REF_VERIFY is not set
+# CONFIG_NILFS2_FS is not set
+# CONFIG_F2FS_FS is not set
+CONFIG_FS_POSIX_ACL=y
+CONFIG_EXPORTFS=y
+# CONFIG_EXPORTFS_BLOCK_OPS is not set
+# CONFIG_FILE_LOCKING is not set
+# CONFIG_FS_ENCRYPTION is not set
+# CONFIG_FS_VERITY is not set
+# CONFIG_DNOTIFY is not set
+# CONFIG_INOTIFY_USER is not set
+# CONFIG_FANOTIFY is not set
+# CONFIG_QUOTA is not set
+# CONFIG_AUTOFS4_FS is not set
+# CONFIG_AUTOFS_FS is not set
+# CONFIG_FUSE_FS is not set
+# CONFIG_OVERLAY_FS is not set
+# CONFIG_FSCACHE is not set
+# CONFIG_ISO9660_FS is not set
+# CONFIG_UDF_FS is not set
+CONFIG_FAT_FS=y
+# CONFIG_MSDOS_FS is not set
+CONFIG_VFAT_FS=y
+CONFIG_FAT_DEFAULT_CODEPAGE=437
+CONFIG_FAT_DEFAULT_IOCHARSET="iso8859-1"
+# CONFIG_FAT_DEFAULT_UTF8 is not set
+# CONFIG_NTFS_FS is not set
+CONFIG_PROC_FS=y
+CONFIG_PROC_SYSCTL=y
+# CONFIG_PROC_CHILDREN is not set
+CONFIG_KERNFS=y
+CONFIG_SYSFS=y
+# CONFIG_CONFIGFS_FS is not set
+CONFIG_MISC_FILESYSTEMS=y
+# CONFIG_ORANGEFS_FS is not set
+# CONFIG_ADFS_FS is not set
+# CONFIG_AFFS_FS is not set
+# CONFIG_HFS_FS is not set
+# CONFIG_HFSPLUS_FS is not set
+# CONFIG_BEFS_FS is not set
+# CONFIG_BFS_FS is not set
+# CONFIG_EFS_FS is not set
+# CONFIG_CRAMFS is not set
+# CONFIG_SQUASHFS is not set
+# CONFIG_VXFS_FS is not set
+# CONFIG_MINIX_FS is not set
+# CONFIG_OMFS_FS is not set
+# CONFIG_HPFS_FS is not set
+# CONFIG_QNX4FS_FS is not set
+# CONFIG_QNX6FS_FS is not set
+# CONFIG_ROMFS_FS is not set
+# CONFIG_PSTORE is not set
+# CONFIG_SYSV_FS is not set
+# CONFIG_UFS_FS is not set
+# CONFIG_EROFS_FS is not set
+CONFIG_NETWORK_FILESYSTEMS=y
+# CONFIG_CEPH_FS is not set
+# CONFIG_CIFS is not set
+# CONFIG_CODA_FS is not set
+# CONFIG_AFS_FS is not set
+CONFIG_NLS=y
+CONFIG_NLS_DEFAULT="iso8859-1"
+CONFIG_NLS_CODEPAGE_437=y
+CONFIG_NLS_CODEPAGE_737=y
+CONFIG_NLS_CODEPAGE_775=y
+CONFIG_NLS_CODEPAGE_850=y
+CONFIG_NLS_CODEPAGE_852=y
+CONFIG_NLS_CODEPAGE_855=y
+CONFIG_NLS_CODEPAGE_857=y
+CONFIG_NLS_CODEPAGE_860=y
+CONFIG_NLS_CODEPAGE_861=y
+CONFIG_NLS_CODEPAGE_862=y
+CONFIG_NLS_CODEPAGE_863=y
+CONFIG_NLS_CODEPAGE_864=y
+CONFIG_NLS_CODEPAGE_865=y
+CONFIG_NLS_CODEPAGE_866=y
+CONFIG_NLS_CODEPAGE_869=y
+CONFIG_NLS_CODEPAGE_936=y
+CONFIG_NLS_CODEPAGE_950=y
+CONFIG_NLS_CODEPAGE_932=y
+CONFIG_NLS_CODEPAGE_949=y
+CONFIG_NLS_CODEPAGE_874=y
+CONFIG_NLS_ISO8859_8=y
+CONFIG_NLS_CODEPAGE_1250=y
+CONFIG_NLS_CODEPAGE_1251=y
+CONFIG_NLS_ASCII=y
+CONFIG_NLS_ISO8859_1=y
+CONFIG_NLS_ISO8859_2=y
+CONFIG_NLS_ISO8859_3=y
+CONFIG_NLS_ISO8859_4=y
+CONFIG_NLS_ISO8859_5=y
+CONFIG_NLS_ISO8859_6=y
+CONFIG_NLS_ISO8859_7=y
+CONFIG_NLS_ISO8859_9=y
+CONFIG_NLS_ISO8859_13=y
+CONFIG_NLS_ISO8859_14=y
+CONFIG_NLS_ISO8859_15=y
+CONFIG_NLS_KOI8_R=y
+CONFIG_NLS_KOI8_U=y
+CONFIG_NLS_MAC_ROMAN=y
+CONFIG_NLS_MAC_CELTIC=y
+CONFIG_NLS_MAC_CENTEURO=y
+CONFIG_NLS_MAC_CROATIAN=y
+CONFIG_NLS_MAC_CYRILLIC=y
+CONFIG_NLS_MAC_GAELIC=y
+CONFIG_NLS_MAC_GREEK=y
+CONFIG_NLS_MAC_ICELAND=y
+CONFIG_NLS_MAC_INUIT=y
+CONFIG_NLS_MAC_ROMANIAN=y
+CONFIG_NLS_MAC_TURKISH=y
+CONFIG_NLS_UTF8=y
+# CONFIG_UNICODE is not set
+CONFIG_IO_WQ=y
+# CONFIG_KEYS is not set
+# CONFIG_SECURITY_DMESG_RESTRICT is not set
+# CONFIG_SECURITY is not set
+# CONFIG_SECURITYFS is not set
+CONFIG_HAVE_HARDENED_USERCOPY_ALLOCATOR=y
+# CONFIG_HARDENED_USERCOPY is not set
+# CONFIG_STATIC_USERMODEHELPER is not set
+CONFIG_DEFAULT_SECURITY_DAC=y
+CONFIG_LSM="lockdown,yama,loadpin,safesetid,integrity"
+CONFIG_INIT_STACK_NONE=y
+# CONFIG_INIT_ON_ALLOC_DEFAULT_ON is not set
+# CONFIG_INIT_ON_FREE_DEFAULT_ON is not set
+CONFIG_XOR_BLOCKS=y
+CONFIG_CRYPTO=y
+# CONFIG_CRYPTO_FIPS is not set
+CONFIG_CRYPTO_ALGAPI=y
+CONFIG_CRYPTO_ALGAPI2=y
+CONFIG_CRYPTO_HASH=y
+CONFIG_CRYPTO_HASH2=y
+CONFIG_CRYPTO_RNG=y
+CONFIG_CRYPTO_RNG2=y
+# CONFIG_CRYPTO_MANAGER is not set
+# CONFIG_CRYPTO_USER is not set
+# CONFIG_CRYPTO_NULL is not set
+# CONFIG_CRYPTO_CRYPTD is not set
+# CONFIG_CRYPTO_AUTHENC is not set
+# CONFIG_CRYPTO_RSA is not set
+# CONFIG_CRYPTO_DH is not set
+# CONFIG_CRYPTO_ECDH is not set
+# CONFIG_CRYPTO_ECRDSA is not set
+# CONFIG_CRYPTO_CURVE25519 is not set
+# CONFIG_CRYPTO_CCM is not set
+# CONFIG_CRYPTO_GCM is not set
+# CONFIG_CRYPTO_CHACHA20POLY1305 is not set
+# CONFIG_CRYPTO_AEGIS128 is not set
+# CONFIG_CRYPTO_SEQIV is not set
+# CONFIG_CRYPTO_ECHAINIV is not set
+# CONFIG_CRYPTO_CBC is not set
+# CONFIG_CRYPTO_CFB is not set
+# CONFIG_CRYPTO_CTR is not set
+# CONFIG_CRYPTO_CTS is not set
+# CONFIG_CRYPTO_ECB is not set
+# CONFIG_CRYPTO_LRW is not set
+# CONFIG_CRYPTO_OFB is not set
+# CONFIG_CRYPTO_PCBC is not set
+# CONFIG_CRYPTO_XTS is not set
+# CONFIG_CRYPTO_KEYWRAP is not set
+# CONFIG_CRYPTO_ADIANTUM is not set
+# CONFIG_CRYPTO_ESSIV is not set
+# CONFIG_CRYPTO_CMAC is not set
+# CONFIG_CRYPTO_HMAC is not set
+# CONFIG_CRYPTO_XCBC is not set
+# CONFIG_CRYPTO_VMAC is not set
+CONFIG_CRYPTO_CRC32C=y
+# CONFIG_CRYPTO_CRC32 is not set
+CONFIG_CRYPTO_XXHASH=y
+CONFIG_CRYPTO_BLAKE2B=y
+# CONFIG_CRYPTO_BLAKE2S is not set
+# CONFIG_CRYPTO_CRCT10DIF is not set
+# CONFIG_CRYPTO_GHASH is not set
+# CONFIG_CRYPTO_POLY1305 is not set
+# CONFIG_CRYPTO_MD4 is not set
+# CONFIG_CRYPTO_MD5 is not set
+# CONFIG_CRYPTO_MICHAEL_MIC is not set
+# CONFIG_CRYPTO_RMD128 is not set
+# CONFIG_CRYPTO_RMD160 is not set
+# CONFIG_CRYPTO_RMD256 is not set
+# CONFIG_CRYPTO_RMD320 is not set
+# CONFIG_CRYPTO_SHA1 is not set
+CONFIG_CRYPTO_SHA256=y
+# CONFIG_CRYPTO_SHA512 is not set
+# CONFIG_CRYPTO_SHA3 is not set
+# CONFIG_CRYPTO_SM3 is not set
+# CONFIG_CRYPTO_STREEBOG is not set
+# CONFIG_CRYPTO_TGR192 is not set
+# CONFIG_CRYPTO_WP512 is not set
+CONFIG_CRYPTO_AES=y
+# CONFIG_CRYPTO_AES_TI is not set
+# CONFIG_CRYPTO_ANUBIS is not set
+# CONFIG_CRYPTO_ARC4 is not set
+# CONFIG_CRYPTO_BLOWFISH is not set
+# CONFIG_CRYPTO_CAMELLIA is not set
+# CONFIG_CRYPTO_CAST5 is not set
+# CONFIG_CRYPTO_CAST6 is not set
+# CONFIG_CRYPTO_DES is not set
+# CONFIG_CRYPTO_FCRYPT is not set
+# CONFIG_CRYPTO_KHAZAD is not set
+# CONFIG_CRYPTO_SALSA20 is not set
+# CONFIG_CRYPTO_CHACHA20 is not set
+# CONFIG_CRYPTO_SEED is not set
+# CONFIG_CRYPTO_SERPENT is not set
+# CONFIG_CRYPTO_SM4 is not set
+# CONFIG_CRYPTO_TEA is not set
+# CONFIG_CRYPTO_TWOFISH is not set
+# CONFIG_CRYPTO_DEFLATE is not set
+# CONFIG_CRYPTO_LZO is not set
+# CONFIG_CRYPTO_842 is not set
+# CONFIG_CRYPTO_LZ4 is not set
+# CONFIG_CRYPTO_LZ4HC is not set
+# CONFIG_CRYPTO_ZSTD is not set
+CONFIG_CRYPTO_ANSI_CPRNG=y
+# CONFIG_CRYPTO_DRBG_MENU is not set
+# CONFIG_CRYPTO_JITTERENTROPY is not set
+# CONFIG_CRYPTO_USER_API_HASH is not set
+# CONFIG_CRYPTO_USER_API_SKCIPHER is not set
+# CONFIG_CRYPTO_USER_API_RNG is not set
+# CONFIG_CRYPTO_USER_API_AEAD is not set
+CONFIG_CRYPTO_LIB_AES=y
+# CONFIG_CRYPTO_LIB_BLAKE2S is not set
+# CONFIG_CRYPTO_LIB_CHACHA is not set
+# CONFIG_CRYPTO_LIB_CURVE25519 is not set
+CONFIG_CRYPTO_LIB_POLY1305_RSIZE=1
+# CONFIG_CRYPTO_LIB_POLY1305 is not set
+# CONFIG_CRYPTO_LIB_CHACHA20POLY1305 is not set
+CONFIG_CRYPTO_LIB_SHA256=y
+CONFIG_CRYPTO_HW=y
+# CONFIG_CRYPTO_DEV_AMLOGIC_GXL is not set
+CONFIG_RAID6_PQ=y
+# CONFIG_RAID6_PQ_BENCHMARK is not set
+# CONFIG_PACKING is not set
+CONFIG_BITREVERSE=y
+CONFIG_GENERIC_NET_UTILS=y
+# CONFIG_CORDIC is not set
+# CONFIG_CRC_CCITT is not set
+CONFIG_CRC16=y
+# CONFIG_CRC_T10DIF is not set
+# CONFIG_CRC_ITU_T is not set
+CONFIG_CRC32=y
+# CONFIG_CRC32_SELFTEST is not set
+CONFIG_CRC32_SLICEBY8=y
+# CONFIG_CRC32_SLICEBY4 is not set
+# CONFIG_CRC32_SARWATE is not set
+# CONFIG_CRC32_BIT is not set
+# CONFIG_CRC64 is not set
+# CONFIG_CRC4 is not set
+# CONFIG_CRC7 is not set
+CONFIG_LIBCRC32C=y
+# CONFIG_CRC8 is not set
+CONFIG_XXHASH=y
+# CONFIG_RANDOM32_SELFTEST is not set
+CONFIG_ZLIB_INFLATE=y
+CONFIG_ZLIB_DEFLATE=y
+CONFIG_LZO_COMPRESS=y
+CONFIG_LZO_DECOMPRESS=y
+CONFIG_ZSTD_COMPRESS=y
+CONFIG_ZSTD_DECOMPRESS=y
+# CONFIG_XZ_DEC is not set
+CONFIG_HAS_IOMEM=y
+CONFIG_HAS_DMA=y
+# CONFIG_DMA_API_DEBUG is not set
+CONFIG_DQL=y
+CONFIG_NLATTR=y
+CONFIG_GENERIC_ATOMIC64=y
+# CONFIG_IRQ_POLL is not set
+CONFIG_SBITMAP=y
+# CONFIG_STRING_SELFTEST is not set
+CONFIG_PRINTK_TIME=y
+# CONFIG_PRINTK_CALLER is not set
+CONFIG_CONSOLE_LOGLEVEL_DEFAULT=7
+CONFIG_CONSOLE_LOGLEVEL_QUIET=4
+CONFIG_MESSAGE_LOGLEVEL_DEFAULT=4
+CONFIG_SYMBOLIC_ERRNAME=y
+CONFIG_DEBUG_INFO=y
+CONFIG_DEBUG_INFO_REDUCED=y
+# CONFIG_DEBUG_INFO_SPLIT is not set
+# CONFIG_DEBUG_INFO_DWARF4 is not set
+# CONFIG_DEBUG_INFO_BTF is not set
+# CONFIG_GDB_SCRIPTS is not set
+# CONFIG_ENABLE_MUST_CHECK is not set
+# CONFIG_STRIP_ASM_SYMS is not set
+# CONFIG_READABLE_ASM is not set
+# CONFIG_HEADERS_INSTALL is not set
+CONFIG_OPTIMIZE_INLINING=y
+# CONFIG_DEBUG_SECTION_MISMATCH is not set
+CONFIG_SECTION_MISMATCH_WARN_ONLY=y
+CONFIG_ARCH_WANT_FRAME_POINTERS=y
+CONFIG_FRAME_POINTER=y
+# CONFIG_DEBUG_FORCE_WEAK_PER_CPU is not set
+# CONFIG_MAGIC_SYSRQ is not set
+# CONFIG_DEBUG_FS is not set
+# CONFIG_UBSAN is not set
+CONFIG_UBSAN_ALIGNMENT=y
+CONFIG_DEBUG_KERNEL=y
+CONFIG_DEBUG_MISC=y
+# CONFIG_PAGE_EXTENSION is not set
+# CONFIG_DEBUG_PAGEALLOC is not set
+# CONFIG_PAGE_POISONING is not set
+# CONFIG_DEBUG_OBJECTS is not set
+# CONFIG_SLUB_DEBUG_ON is not set
+# CONFIG_SLUB_STATS is not set
+# CONFIG_DEBUG_STACK_USAGE is not set
+# CONFIG_SCHED_STACK_END_CHECK is not set
+# CONFIG_DEBUG_VM is not set
+# CONFIG_DEBUG_NOMMU_REGIONS is not set
+# CONFIG_DEBUG_MEMORY_INIT is not set
+CONFIG_CC_HAS_KASAN_GENERIC=y
+CONFIG_KASAN_STACK=1
+# CONFIG_DEBUG_SHIRQ is not set
+# CONFIG_PANIC_ON_OOPS is not set
+CONFIG_PANIC_ON_OOPS_VALUE=0
+CONFIG_PANIC_TIMEOUT=0
+# CONFIG_SOFTLOCKUP_DETECTOR is not set
+# CONFIG_DETECT_HUNG_TASK is not set
+# CONFIG_WQ_WATCHDOG is not set
+CONFIG_SCHED_DEBUG=y
+# CONFIG_SCHEDSTATS is not set
+# CONFIG_DEBUG_TIMEKEEPING is not set
+# CONFIG_DEBUG_SPINLOCK is not set
+# CONFIG_DEBUG_MUTEXES is not set
+# CONFIG_DEBUG_RWSEMS is not set
+# CONFIG_DEBUG_ATOMIC_SLEEP is not set
+# CONFIG_DEBUG_LOCKING_API_SELFTESTS is not set
+# CONFIG_LOCK_TORTURE_TEST is not set
+# CONFIG_WW_MUTEX_SELFTEST is not set
+# CONFIG_WARN_ALL_UNSEEDED_RANDOM is not set
+# CONFIG_DEBUG_KOBJECT is not set
+# CONFIG_DEBUG_LIST is not set
+# CONFIG_DEBUG_PLIST is not set
+# CONFIG_DEBUG_SG is not set
+# CONFIG_DEBUG_NOTIFIERS is not set
+# CONFIG_BUG_ON_DATA_CORRUPTION is not set
+# CONFIG_DEBUG_CREDENTIALS is not set
+# CONFIG_RCU_PERF_TEST is not set
+# CONFIG_RCU_TORTURE_TEST is not set
+# CONFIG_RCU_TRACE is not set
+# CONFIG_RCU_EQS_DEBUG is not set
+# CONFIG_DEBUG_WQ_FORCE_RR_CPU is not set
+# CONFIG_DEBUG_BLOCK_EXT_DEVT is not set
+# CONFIG_SAMPLES is not set
+# CONFIG_KUNIT is not set
+# CONFIG_NOTIFIER_ERROR_INJECTION is not set
+# CONFIG_FAULT_INJECTION is not set
+CONFIG_CC_HAS_SANCOV_TRACE_PC=y
+CONFIG_RUNTIME_TESTING_MENU=y
+# CONFIG_TEST_LIST_SORT is not set
+# CONFIG_TEST_SORT is not set
+# CONFIG_BACKTRACE_SELF_TEST is not set
+# CONFIG_RBTREE_TEST is not set
+# CONFIG_REED_SOLOMON_TEST is not set
+# CONFIG_INTERVAL_TREE_TEST is not set
+# CONFIG_ATOMIC64_SELFTEST is not set
+# CONFIG_TEST_HEXDUMP is not set
+# CONFIG_TEST_STRING_HELPERS is not set
+# CONFIG_TEST_STRSCPY is not set
+# CONFIG_TEST_KSTRTOX is not set
+# CONFIG_TEST_PRINTF is not set
+# CONFIG_TEST_BITMAP is not set
+# CONFIG_TEST_BITFIELD is not set
+# CONFIG_TEST_UUID is not set
+# CONFIG_TEST_XARRAY is not set
+# CONFIG_TEST_OVERFLOW is not set
+# CONFIG_TEST_RHASHTABLE is not set
+# CONFIG_TEST_HASH is not set
+# CONFIG_TEST_IDA is not set
+# CONFIG_FIND_BIT_BENCHMARK is not set
+# CONFIG_TEST_SYSCTL is not set
+# CONFIG_TEST_UDELAY is not set
+# CONFIG_TEST_MEMCAT_P is not set
+# CONFIG_TEST_STACKINIT is not set
+# CONFIG_TEST_MEMINIT is not set
+# CONFIG_MEMTEST is not set
diff --git a/arch/um/lkl/include/asm/Kbuild b/arch/um/lkl/include/asm/Kbuild
new file mode 100644
index 000000000000..7f360f641f19
--- /dev/null
+++ b/arch/um/lkl/include/asm/Kbuild
@@ -0,0 +1,80 @@
+generic-y += atomic.h
+generic-y += barrier.h
+generic-y += bitops.h
+generic-y += bug.h
+generic-y += bugs.h
+generic-y += cache.h
+generic-y += cacheflush.h
+generic-y += checksum.h
+generic-y += cmpxchg-local.h
+generic-y += cmpxchg.h
+generic-y += compat.h
+generic-y += cputime.h
+generic-y += current.h
+generic-y += delay.h
+generic-y += device.h
+generic-y += div64.h
+generic-y += dma.h
+generic-y += dma-mapping.h
+generic-y += emergency-restart.h
+generic-y += errno.h
+generic-y += extable.h
+generic-y += exec.h
+generic-y += ftrace.h
+generic-y += futex.h
+generic-y += hardirq.h
+generic-y += hw_irq.h
+generic-y += io.h
+generic-y += ioctl.h
+generic-y += ipcbuf.h
+generic-y += irq_regs.h
+generic-y += irqflags.h
+generic-y += irq_work.h
+generic-y += kdebug.h
+generic-y += kmap_types.h
+generic-y += linkage.h
+generic-y += local.h
+generic-y += local64.h
+generic-y += mcs_spinlock.h
+generic-y += mmiowb.h
+generic-y += mmu.h
+generic-y += mmu_context.h
+generic-y += module.h
+generic-y += msgbuf.h
+generic-y += param.h
+generic-y += parport.h
+generic-y += pci.h
+generic-y += percpu.h
+generic-y += pgalloc.h
+generic-y += poll.h
+generic-y += preempt.h
+generic-y += resource.h
+generic-y += rwsem.h
+generic-y += scatterlist.h
+generic-y += seccomp.h
+generic-y += sections.h
+generic-y += segment.h
+generic-y += sembuf.h
+generic-y += serial.h
+generic-y += shmbuf.h
+generic-y += signal.h
+generic-y += sizes.h
+generic-y += socket.h
+generic-y += sockios.h
+generic-y += stat.h
+generic-y += statfs.h
+generic-y += string.h
+generic-y += swab.h
+generic-y += switch_to.h
+generic-y += syscall.h
+generic-y += termbits.h
+generic-y += termios.h
+generic-y += time.h
+generic-y += timex.h
+generic-y += tlbflush.h
+generic-y += topology.h
+generic-y += trace_clock.h
+generic-y += unaligned.h
+generic-y += user.h
+generic-y += word-at-a-time.h
+generic-y += kprobes.h
diff --git a/arch/um/lkl/include/asm/bitsperlong.h b/arch/um/lkl/include/asm/bitsperlong.h
new file mode 100644
index 000000000000..5745d5e51274
--- /dev/null
+++ b/arch/um/lkl/include/asm/bitsperlong.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __LKL_BITSPERLONG_H
+#define __LKL_BITSPERLONG_H
+
+#include <uapi/asm/bitsperlong.h>
+
+#define BITS_PER_LONG __BITS_PER_LONG
+
+#define BITS_PER_LONG_LONG 64
+
+#endif
diff --git a/arch/um/lkl/include/asm/byteorder.h b/arch/um/lkl/include/asm/byteorder.h
new file mode 100644
index 000000000000..5d0c4efaa44b
--- /dev/null
+++ b/arch/um/lkl/include/asm/byteorder.h
@@ -0,0 +1,7 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_BYTEORDER_H
+#define _ASM_LKL_BYTEORDER_H
+
+#include <uapi/asm/byteorder.h>
+
+#endif /* _ASM_LKL_BYTEORDER_H */
diff --git a/arch/um/lkl/include/asm/cpu.h b/arch/um/lkl/include/asm/cpu.h
new file mode 100644
index 000000000000..d2b8c501c7b1
--- /dev/null
+++ b/arch/um/lkl/include/asm/cpu.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_CPU_H
+#define _ASM_LKL_CPU_H
+
+int lkl_cpu_get(void);
+void lkl_cpu_put(void);
+int lkl_cpu_try_run_irq(int irq);
+int lkl_cpu_init(void);
+void lkl_cpu_shutdown(void);
+void lkl_cpu_wait_shutdown(void);
+void lkl_cpu_change_owner(lkl_thread_t owner);
+void lkl_cpu_set_irqs_pending(void);
+
+#endif /* _ASM_LKL_CPU_H */
diff --git a/arch/um/lkl/include/asm/elf.h b/arch/um/lkl/include/asm/elf.h
new file mode 100644
index 000000000000..bb2456d638f4
--- /dev/null
+++ b/arch/um/lkl/include/asm/elf.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_ELF_H
+#define _ASM_LKL_ELF_H
+
+#define elf_check_arch(x) 0
+
+#ifdef CONFIG_64BIT
+#define ELF_CLASS ELFCLASS64
+#else
+#define ELF_CLASS ELFCLASS32
+#endif
+
+#define elf_gregset_t long
+#define elf_fpregset_t double
+#endif
diff --git a/arch/um/lkl/include/asm/processor.h b/arch/um/lkl/include/asm/processor.h
new file mode 100644
index 000000000000..c1aa8b3a266e
--- /dev/null
+++ b/arch/um/lkl/include/asm/processor.h
@@ -0,0 +1,60 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_PROCESSOR_H
+#define _ASM_LKL_PROCESSOR_H
+
+struct task_struct;
+
+static inline void cpu_relax(void)
+{
+	unsigned long flags;
+
+	/* since this is usually called in a tight loop waiting for some
+	 * external condition (e.g. jiffies) lets run interrupts now to allow
+	 * the external condition to propagate
+	 */
+	local_irq_save(flags);
+	local_irq_restore(flags);
+}
+
+#define current_text_addr() ({ __label__ _l; _l: &&_l; })
+
+static inline unsigned long thread_saved_pc(struct task_struct *tsk)
+{
+	return 0;
+}
+
+static inline void release_thread(struct task_struct *dead_task)
+{
+}
+
+static inline void prepare_to_copy(struct task_struct *tsk)
+{
+}
+
+static inline unsigned long get_wchan(struct task_struct *p)
+{
+	return 0;
+}
+
+static inline void flush_thread(void)
+{
+}
+
+static inline void trap_init(void)
+{
+}
+
+struct thread_struct { };
+
+#define INIT_THREAD { }
+
+#define task_pt_regs(tsk) (struct pt_regs *)(NULL)
+
+/* We don't have strict user/kernel spaces */
+#define TASK_SIZE ((unsigned long)-1)
+#define TASK_UNMAPPED_BASE 0
+
+#define KSTK_EIP(tsk) (0)
+#define KSTK_ESP(tsk) (0)
+
+#endif
diff --git a/arch/um/lkl/include/asm/ptrace.h b/arch/um/lkl/include/asm/ptrace.h
new file mode 100644
index 000000000000..28199be26dc0
--- /dev/null
+++ b/arch/um/lkl/include/asm/ptrace.h
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_PTRACE_H
+#define _ASM_LKL_PTRACE_H
+
+#include <linux/errno.h>
+
+struct task_struct;
+
+#define user_mode(regs) 0
+#define kernel_mode(regs) 1
+#define profile_pc(regs) 0
+#define instruction_pointer(regs) 0
+#define user_stack_pointer(regs) 0
+
+static inline long arch_ptrace(struct task_struct *child, long request,
+			       unsigned long addr, unsigned long data)
+{
+	return -EINVAL;
+}
+
+static inline void ptrace_disable(struct task_struct *child)
+{
+}
+
+#endif
diff --git a/arch/um/lkl/include/asm/sched.h b/arch/um/lkl/include/asm/sched.h
new file mode 100644
index 000000000000..4c2635921ec8
--- /dev/null
+++ b/arch/um/lkl/include/asm/sched.h
@@ -0,0 +1,23 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_SCHED_H
+#define _ASM_LKL_SCHED_H
+
+#include <linux/sched.h>
+#include <uapi/asm/host_ops.h>
+
+static inline void thread_sched_jb(void)
+{
+	if (test_ti_thread_flag(current_thread_info(), TIF_HOST_THREAD)) {
+		set_ti_thread_flag(current_thread_info(), TIF_SCHED_JB);
+		set_current_state(TASK_UNINTERRUPTIBLE);
+		lkl_ops->jmp_buf_set(&current_thread_info()->sched_jb,
+				     schedule);
+	} else {
+		lkl_bug("%s() can be used only for host task\n", __func__);
+	}
+}
+
+void switch_to_host_task(struct task_struct *);
+int host_task_stub(void *unused);
+
+#endif /*  _ASM_LKL_SCHED_H */
diff --git a/arch/um/lkl/include/asm/syscalls.h b/arch/um/lkl/include/asm/syscalls.h
new file mode 100644
index 000000000000..2eaa870a9020
--- /dev/null
+++ b/arch/um/lkl/include/asm/syscalls.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_SYSCALLS_H
+#define _ASM_LKL_SYSCALLS_H
+
+int syscalls_init(void);
+void syscalls_cleanup(void);
+long lkl_syscall(long no, long *params);
+void wakeup_idle_host_task(void);
+
+#define sys_mmap sys_mmap_pgoff
+#define sys_mmap2 sys_mmap_pgoff
+#define sys_clone sys_ni_syscall
+#define sys_vfork sys_ni_syscall
+#define sys_rt_sigreturn sys_ni_syscall
+
+#include <asm-generic/syscalls.h>
+
+#endif /* _ASM_LKL_SYSCALLS_H */
diff --git a/arch/um/lkl/include/asm/syscalls_32.h b/arch/um/lkl/include/asm/syscalls_32.h
new file mode 100644
index 000000000000..0e1a7649c81b
--- /dev/null
+++ b/arch/um/lkl/include/asm/syscalls_32.h
@@ -0,0 +1,43 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_SYSCALLS_32_H
+#define _ASM_SYSCALLS_32_H
+
+#include <linux/compiler.h>
+#include <linux/linkage.h>
+#include <linux/types.h>
+#include <linux/signal.h>
+
+#if __BITS_PER_LONG == 32
+
+/* kernel/syscalls_32.c */
+asmlinkage long sys32_truncate64(const char __user *, unsigned long,
+				 unsigned long);
+asmlinkage long sys32_ftruncate64(unsigned int, unsigned long, unsigned long);
+
+#ifdef CONFIG_MMU
+struct mmap_arg_struct32;
+asmlinkage long sys32_mmap(struct mmap_arg_struct32 __user *);
+#endif
+
+asmlinkage long sys32_wait4(pid_t, unsigned int __user *, int,
+			    struct rusage __user *);
+
+asmlinkage long sys32_pread64(unsigned int, char __user *, u32, u32, u32);
+asmlinkage long sys32_pwrite64(unsigned int, const char __user *, u32, u32,
+			       u32);
+
+long sys32_fadvise64_64(int a, __u32 b, __u32 c, __u32 d, __u32 e, int f);
+
+asmlinkage ssize_t sys32_readahead(int, unsigned int, unsigned int, size_t);
+asmlinkage long sys32_sync_file_range(int, unsigned int, unsigned int,
+				      unsigned int, unsigned int, unsigned int);
+asmlinkage long sys32_sync_file_range2(int, unsigned int, unsigned int,
+				       unsigned int, unsigned int,
+				       unsigned int);
+asmlinkage long sys32_fadvise64(int, unsigned int, unsigned int, size_t, int);
+asmlinkage long sys32_fallocate(int, int, unsigned int, unsigned int,
+				unsigned int, unsigned int);
+
+#endif /* __BITS_PER_LONG */
+
+#endif /* _ASM_SYSCALLS_32_H */
diff --git a/arch/um/lkl/include/asm/tlb.h b/arch/um/lkl/include/asm/tlb.h
new file mode 100644
index 000000000000..d474890d317d
--- /dev/null
+++ b/arch/um/lkl/include/asm/tlb.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_TLB_H
+#define _ASM_LKL_TLB_H
+
+#define tlb_start_vma(tlb, vma)				do { } while (0)
+#define tlb_end_vma(tlb, vma)				do { } while (0)
+#define __tlb_remove_tlb_entry(tlb, pte, address)	do { } while (0)
+#define tlb_flush(tlb)					do { } while (0)
+
+#include <asm-generic/tlb.h>
+
+#endif /* _ASM_LKL_TLB_H */
diff --git a/arch/um/lkl/include/asm/uaccess.h b/arch/um/lkl/include/asm/uaccess.h
new file mode 100644
index 000000000000..f267ac3be8b3
--- /dev/null
+++ b/arch/um/lkl/include/asm/uaccess.h
@@ -0,0 +1,64 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_UACCESS_H
+#define _ASM_LKL_UACCESS_H
+
+/* copied from old include/asm-generic/uaccess.h */
+static inline __must_check long
+raw_copy_from_user(void *to, const void __user *from, unsigned long n)
+{
+	if (__builtin_constant_p(n)) {
+		switch (n) {
+		case 1:
+			*(u8 *)to = *(u8 __force *)from;
+			return 0;
+		case 2:
+			*(u16 *)to = *(u16 __force *)from;
+			return 0;
+		case 4:
+			*(u32 *)to = *(u32 __force *)from;
+			return 0;
+#ifdef CONFIG_64BIT
+		case 8:
+			*(u64 *)to = *(u64 __force *)from;
+			return 0;
+#endif
+		default:
+			break;
+		}
+	}
+
+	memcpy(to, (const void __force *)from, n);
+	return 0;
+}
+
+static inline __must_check long
+raw_copy_to_user(void __user *to, const void *from, unsigned long n)
+{
+	if (__builtin_constant_p(n)) {
+		switch (n) {
+		case 1:
+			*(u8 __force *)to = *(u8 *)from;
+			return 0;
+		case 2:
+			*(u16 __force *)to = *(u16 *)from;
+			return 0;
+		case 4:
+			*(u32 __force *)to = *(u32 *)from;
+			return 0;
+#ifdef CONFIG_64BIT
+		case 8:
+			*(u64 __force *)to = *(u64 *)from;
+			return 0;
+#endif
+		default:
+			break;
+		}
+	}
+
+	memcpy((void __force *)to, from, n);
+	return 0;
+}
+
+#include <asm-generic/uaccess.h>
+
+#endif
diff --git a/arch/um/lkl/include/asm/unistd_32.h b/arch/um/lkl/include/asm/unistd_32.h
new file mode 100644
index 000000000000..8582a55e61e2
--- /dev/null
+++ b/arch/um/lkl/include/asm/unistd_32.h
@@ -0,0 +1,31 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <asm/bitsperlong.h>
+
+#ifndef __SYSCALL
+#define __SYSCALL(x, y)
+#endif
+
+#if __BITS_PER_LONG == 32
+__SYSCALL(__NR3264_truncate, sys32_truncate64)
+__SYSCALL(__NR3264_ftruncate, sys32_ftruncate64)
+
+#ifdef CONFIG_MMU
+__SYSCALL(__NR3264_mmap, sys32_mmap)
+#endif
+
+__SYSCALL(__NR_wait4, sys32_wait4)
+
+__SYSCALL(__NR_pread64, sys32_pread64)
+__SYSCALL(__NR_pwrite64, sys32_pwrite64)
+
+__SYSCALL(__NR_readahead, sys32_readahead)
+#ifdef __ARCH_WANT_SYNC_FILE_RANGE2
+__SYSCALL(__NR_sync_file_range2, sys32_sync_file_range2)
+#else
+__SYSCALL(__NR_sync_file_range, sys32_sync_file_range)
+#endif
+/* mm/fadvise.c */
+__SYSCALL(__NR3264_fadvise64, sys32_fadvise64_64)
+__SYSCALL(__NR_fallocate, sys32_fallocate)
+
+#endif
diff --git a/arch/um/lkl/include/asm/vmlinux.lds.h b/arch/um/lkl/include/asm/vmlinux.lds.h
new file mode 100644
index 000000000000..a3c285882dc4
--- /dev/null
+++ b/arch/um/lkl/include/asm/vmlinux.lds.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_VMLINUX_LDS_H
+#define _LKL_VMLINUX_LDS_H
+
+/* we encode our own __ro_after_init section */
+#define RO_AFTER_INIT_DATA
+
+#ifdef __MINGW32__
+#define RODATA_SECTION .rdata
+#endif
+
+#include <asm-generic/vmlinux.lds.h>
+
+#endif
diff --git a/arch/um/lkl/include/asm/xor.h b/arch/um/lkl/include/asm/xor.h
new file mode 100644
index 000000000000..286ce75b5d9d
--- /dev/null
+++ b/arch/um/lkl/include/asm/xor.h
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_XOR_H
+#define _ASM_LKL_XOR_H
+
+#include <asm-generic/xor.h>
+
+#define XOR_SELECT_TEMPLATE(x) (&xor_block_8regs)
+
+#endif /* _ASM_LKL_XOR_H */
diff --git a/arch/um/lkl/include/uapi/asm/Kbuild b/arch/um/lkl/include/uapi/asm/Kbuild
new file mode 100644
index 000000000000..39d9a1f2e8f5
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/Kbuild
@@ -0,0 +1,9 @@
+# UAPI Header export list
+
+generic-y += elf.h
+generic-y += kvm_para.h
+generic-y += shmparam.h
+generic-y += timex.h
+
+# no header-y since we need special user headers handling
+# see arch/lkl/script/headers.py
diff --git a/arch/um/lkl/include/uapi/asm/bitsperlong.h b/arch/um/lkl/include/uapi/asm/bitsperlong.h
new file mode 100644
index 000000000000..8b4ebf2b0264
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/bitsperlong.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _ASM_UAPI_LKL_BITSPERLONG_H
+#define _ASM_UAPI_LKL_BITSPERLONG_H
+
+#ifdef CONFIG_64BIT
+#define __BITS_PER_LONG 64
+#else
+#define __BITS_PER_LONG 32
+#endif
+
+#define __ARCH_WANT_STAT64
+
+#endif /* _ASM_UAPI_LKL_BITSPERLONG_H */
diff --git a/arch/um/lkl/include/uapi/asm/byteorder.h b/arch/um/lkl/include/uapi/asm/byteorder.h
new file mode 100644
index 000000000000..3c4a58d2062f
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/byteorder.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _ASM_UAPI_LKL_BYTEORDER_H
+#define _ASM_UAPI_LKL_BYTEORDER_H
+
+#if defined(CONFIG_BIG_ENDIAN)
+#include <linux/byteorder/big_endian.h>
+#else
+#include <linux/byteorder/little_endian.h>
+#endif
+
+#endif /* _ASM_UAPI_LKL_BYTEORDER_H */
diff --git a/arch/um/lkl/include/uapi/asm/siginfo.h b/arch/um/lkl/include/uapi/asm/siginfo.h
new file mode 100644
index 000000000000..811916cf42c8
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/siginfo.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _ASM_LKL_SIGINFO_H
+#define _ASM_LKL_SIGINFO_H
+
+#ifdef CONFIG_64BIT
+#define __ARCH_SI_PREAMBLE_SIZE	(4 * sizeof(int))
+#endif
+
+#include <asm-generic/siginfo.h>
+
+#endif /* _ASM_LKL_SIGINFO_H */
diff --git a/arch/um/lkl/include/uapi/asm/swab.h b/arch/um/lkl/include/uapi/asm/swab.h
new file mode 100644
index 000000000000..1a1773e1bd35
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/swab.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _ASM_LKL_SWAB_H
+#define _ASM_LKL_SWAB_H
+
+#ifndef __arch_swab32
+#define __arch_swab32(x) ___constant_swab32(x)
+#endif
+
+#include <asm-generic/swab.h>
+
+#endif /* _ASM_LKL_SWAB_H */
diff --git a/arch/um/lkl/include/uapi/asm/syscalls.h b/arch/um/lkl/include/uapi/asm/syscalls.h
new file mode 100644
index 000000000000..a81534ffccb7
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/syscalls.h
@@ -0,0 +1,348 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _ASM_UAPI_LKL_SYSCALLS_H
+#define _ASM_UAPI_LKL_SYSCALLS_H
+
+#include <autoconf.h>
+#include <linux/types.h>
+
+typedef __kernel_uid32_t	qid_t;
+typedef __kernel_fd_set		fd_set;
+typedef __kernel_mode_t		mode_t;
+typedef unsigned short		umode_t;
+typedef __u32			nlink_t;
+typedef __kernel_off_t		off_t;
+typedef __kernel_pid_t		pid_t;
+typedef __kernel_key_t		key_t;
+typedef __kernel_suseconds_t	suseconds_t;
+typedef __kernel_timer_t	timer_t;
+typedef __kernel_clockid_t	clockid_t;
+typedef __kernel_mqd_t		mqd_t;
+typedef __kernel_uid32_t	uid_t;
+typedef __kernel_gid32_t	gid_t;
+typedef __kernel_uid16_t        uid16_t;
+typedef __kernel_gid16_t        gid16_t;
+typedef unsigned long		uintptr_t;
+#ifdef CONFIG_UID16
+typedef __kernel_old_uid_t	old_uid_t;
+typedef __kernel_old_gid_t	old_gid_t;
+#endif
+typedef __kernel_loff_t		loff_t;
+typedef __kernel_size_t		size_t;
+typedef __kernel_ssize_t	ssize_t;
+typedef __kernel_time_t		time_t;
+typedef __kernel_clock_t	clock_t;
+typedef __u32			u32;
+typedef __s32			s32;
+typedef __u64			u64;
+typedef __s64			s64;
+
+#define __user
+
+#include <asm/unistd.h>
+/* Temporary undefine system calls that don't have data types defined in UAPI
+ * headers
+ */
+#undef __NR_kexec_load
+#undef __NR_getcpu
+#undef __NR_sched_getattr
+#undef __NR_sched_setattr
+#undef __NR_sched_setparam
+#undef __NR_sched_getparam
+#undef __NR_sched_setscheduler
+#undef __NR_name_to_handle_at
+#undef __NR_open_by_handle_at
+
+/* deprecated system calls */
+#undef __NR_epoll_create
+#undef __NR_epoll_wait
+#undef __NR_access
+#undef __NR_chmod
+#undef __NR_chown
+#undef __NR_lchown
+#undef __NR_open
+#undef __NR_creat
+#undef __NR_readlink
+#undef __NR_pipe
+#undef __NR_mknod
+#undef __NR_mkdir
+#undef __NR_rmdir
+#undef __NR_unlink
+#undef __NR_symlink
+#undef __NR_link
+#undef __NR_rename
+#undef __NR_getdents
+#undef __NR_select
+#undef __NR_poll
+#undef __NR_dup2
+#undef __NR_futimesat
+#undef __NR_utimes
+#undef __NR_ustat
+#undef __NR_eventfd
+#undef __NR_bdflush
+#undef __NR_send
+#undef __NR_recv
+
+#undef __NR_umount
+#define __NR_umount __NR_umount2
+
+#ifdef CONFIG_64BIT
+#define __NR_newfstat __NR3264_fstat
+#define __NR_newfstatat __NR3264_fstatat
+#endif
+
+#define __NR_mmap_pgoff __NR3264_mmap
+
+#include <linux/time.h>
+#include <linux/times.h>
+#include <linux/timex.h>
+#include <linux/capability.h>
+#define __KERNEL__ /* to pull in S_ definitions */
+#include <linux/stat.h>
+#undef __KERNEL__
+#include <linux/errno.h>
+#include <linux/fcntl.h>
+#include <linux/fs.h>
+#include <asm/statfs.h>
+#include <asm/stat.h>
+#include <linux/bpf.h>
+#include <linux/msg.h>
+#include <linux/resource.h>
+#include <linux/sysinfo.h>
+#include <linux/shm.h>
+#include <linux/aio_abi.h>
+#include <linux/socket.h>
+#include <linux/perf_event.h>
+#include <linux/sem.h>
+#include <linux/futex.h>
+#include <linux/poll.h>
+#include <linux/mqueue.h>
+#include <linux/eventpoll.h>
+#include <linux/uio.h>
+#include <asm/signal.h>
+#include <asm/siginfo.h>
+#include <linux/utime.h>
+#include <asm/socket.h>
+#include <linux/icmp.h>
+#include <linux/ip.h>
+
+/* Define data structures used in system calls that are not defined in UAPI
+ * headers
+ */
+struct sockaddr {
+	unsigned short int sa_family;
+	char sa_data[14];
+};
+
+#define __UAPI_DEF_IF_NET_DEVICE_FLAGS_LOWER_UP_DORMANT_ECHO 1
+#define __UAPI_DEF_IF_IFNAMSIZ	1
+#define __UAPI_DEF_IF_NET_DEVICE_FLAGS 1
+#define __UAPI_DEF_IF_IFREQ	1
+#define __UAPI_DEF_IF_IFMAP	1
+#include <linux/if.h>
+#define __UAPI_DEF_IN_IPPROTO	1
+#define __UAPI_DEF_IN_ADDR	1
+#define __UAPI_DEF_IN6_ADDR	1
+#define __UAPI_DEF_IP_MREQ	1
+#define __UAPI_DEF_IN_PKTINFO	1
+#define __UAPI_DEF_SOCKADDR_IN	1
+#define __UAPI_DEF_IN_CLASS	1
+#include <linux/in.h>
+#include <linux/in6.h>
+#include <linux/sockios.h>
+#include <linux/route.h>
+#include <linux/ipv6_route.h>
+#include <linux/ipv6.h>
+#include <linux/netlink.h>
+#include <linux/neighbour.h>
+#include <linux/rtnetlink.h>
+#include <linux/fib_rules.h>
+
+#include <linux/kdev_t.h>
+#include <asm/irq.h>
+#include <linux/virtio_blk.h>
+#include <linux/virtio_net.h>
+#include <linux/virtio_ring.h>
+#include <linux/pkt_sched.h>
+#include <linux/io_uring.h>
+
+struct user_msghdr {
+	void		__user *msg_name;
+	int		msg_namelen;
+	struct iovec	__user *msg_iov;
+	__kernel_size_t	msg_iovlen;
+	void		__user *msg_control;
+	__kernel_size_t	msg_controllen;
+	unsigned int	msg_flags;
+};
+
+typedef __u32 key_serial_t;
+
+struct mmsghdr {
+	struct user_msghdr  msg_hdr;
+	unsigned int        msg_len;
+};
+
+struct linux_dirent64 {
+	u64		d_ino;
+	s64		d_off;
+	unsigned short	d_reclen;
+	unsigned char	d_type;
+	char		d_name[0];
+};
+
+struct linux_dirent {
+	unsigned long	d_ino;
+	unsigned long	d_off;
+	unsigned short	d_reclen;
+	char		d_name[1];
+};
+
+struct ustat {
+	__kernel_daddr_t	f_tfree;
+	__kernel_ino_t		f_tinode;
+	char			f_fname[6];
+	char			f_fpack[6];
+};
+
+typedef __kernel_rwf_t		rwf_t;
+
+#define AF_UNSPEC       0
+#define AF_UNIX         1
+#define AF_LOCAL        1
+#define AF_INET         2
+#define AF_AX25         3
+#define AF_IPX          4
+#define AF_APPLETALK    5
+#define AF_NETROM       6
+#define AF_BRIDGE       7
+#define AF_ATMPVC       8
+#define AF_X25          9
+#define AF_INET6        10
+#define AF_ROSE         11
+#define AF_DECnet       12
+#define AF_NETBEUI      13
+#define AF_SECURITY     14
+#define AF_KEY          15
+#define AF_NETLINK      16
+#define AF_ROUTE        AF_NETLINK
+#define AF_PACKET       17
+#define AF_ASH          18
+#define AF_ECONET       19
+#define AF_ATMSVC       20
+#define AF_RDS          21
+#define AF_SNA          22
+#define AF_IRDA         23
+#define AF_PPPOX        24
+#define AF_WANPIPE      25
+#define AF_LLC          26
+#define AF_IB           27
+#define AF_MPLS         28
+#define AF_CAN          29
+#define AF_TIPC         30
+#define AF_BLUETOOTH    31
+#define AF_IUCV         32
+#define AF_RXRPC        33
+#define AF_ISDN         34
+#define AF_PHONET       35
+#define AF_IEEE802154   36
+#define AF_CAIF         37
+#define AF_ALG          38
+#define AF_NFC          39
+#define AF_VSOCK        40
+
+#define SOCK_STREAM		1
+#define SOCK_DGRAM		2
+#define SOCK_RAW		3
+#define SOCK_RDM		4
+#define SOCK_SEQPACKET		5
+#define SOCK_DCCP		6
+#define SOCK_PACKET		10
+
+#define MSG_TRUNC 0x20
+#define MSG_DONTWAIT 0x40
+
+/* avoid colision with system headers defines */
+#define sa_handler sa_handler
+#define st_atime st_atime
+#define st_mtime st_mtime
+#define st_ctime st_ctime
+#define s_addr s_addr
+
+long lkl_syscall(long no, long *params);
+long lkl_sys_halt(void);
+
+#define __MAP0(m, ...)
+#define __MAP1(m, t, a) m(t, a)
+#define __MAP2(m, t, a, ...) m(t, a), __MAP1(m, __VA_ARGS__)
+#define __MAP3(m, t, a, ...) m(t, a), __MAP2(m, __VA_ARGS__)
+#define __MAP4(m, t, a, ...) m(t, a), __MAP3(m, __VA_ARGS__)
+#define __MAP5(m, t, a, ...) m(t, a), __MAP4(m, __VA_ARGS__)
+#define __MAP6(m, t, a, ...) m(t, a), __MAP5(m, __VA_ARGS__)
+#define __MAP(n, ...) __MAP##n(__VA_ARGS__)
+
+#define __SC_LONG(t, a) (long)a
+#define __SC_TABLE(t, a) {sizeof(t), (long long)(a)}
+#define __SC_DECL(t, a) t a
+
+#define LKL_SYSCALL0(name)					       \
+	static inline long lkl_sys##name(void)			       \
+	{							       \
+		long params[6];					       \
+		return lkl_syscall(__lkl__NR##name, params);	       \
+	}
+
+#if __BITS_PER_LONG == 32
+#define LKL_SYSCALLx(x, name, ...)					\
+	static inline							\
+	long lkl_sys##name(__MAP(x, __SC_DECL, __VA_ARGS__))		\
+	{								\
+		struct {						\
+			unsigned int size;				\
+			long long value;				\
+		} lkl_params[x] = { __MAP(x, __SC_TABLE, __VA_ARGS__) }; \
+		long sys_params[6], i, k;				\
+		for (i = k = 0; i < x && k < 6; i++, k++) {		\
+			if (lkl_params[i].size > sizeof(long) &&	\
+			    k + 1 < 6) {				\
+				sys_params[k] =				\
+					(long)(lkl_params[i].value & (-1UL)); \
+				k++;					\
+				sys_params[k] =				\
+					(long)(lkl_params[i].value >>	\
+					       __BITS_PER_LONG);	\
+			} else {					\
+				sys_params[k] = (long)(lkl_params[i].value); \
+			}						\
+		}							\
+		return lkl_syscall(__lkl__NR##name, sys_params);	\
+	}
+#else
+#define LKL_SYSCALLx(x, name, ...)					\
+	static inline							\
+	long lkl_sys##name(__MAP(x, __SC_DECL, __VA_ARGS__))		\
+	{								\
+		long lkl_params[6] = { __MAP(x, __SC_LONG, __VA_ARGS__) }; \
+		return lkl_syscall(__lkl__NR##name, lkl_params);	\
+	}
+#endif
+
+#define SYSCALL_DEFINE0(name, ...) LKL_SYSCALL0(name)
+#define SYSCALL_DEFINE1(name, ...) LKL_SYSCALLx(1, name, __VA_ARGS__)
+#define SYSCALL_DEFINE2(name, ...) LKL_SYSCALLx(2, name, __VA_ARGS__)
+#define SYSCALL_DEFINE3(name, ...) LKL_SYSCALLx(3, name, __VA_ARGS__)
+#define SYSCALL_DEFINE4(name, ...) LKL_SYSCALLx(4, name, __VA_ARGS__)
+#define SYSCALL_DEFINE5(name, ...) LKL_SYSCALLx(5, name, __VA_ARGS__)
+#define SYSCALL_DEFINE6(name, ...) LKL_SYSCALLx(6, name, __VA_ARGS__)
+
+#if __BITS_PER_LONG == 32
+#pragma GCC diagnostic push
+#pragma GCC diagnostic ignored "-Wpointer-to-int-cast"
+#endif
+
+#include <asm/syscall_defs.h>
+
+#if __BITS_PER_LONG == 32
+#pragma GCC diagnostic pop
+#endif
+
+#endif
diff --git a/arch/um/lkl/kernel/asm-offsets.c b/arch/um/lkl/kernel/asm-offsets.c
new file mode 100644
index 000000000000..6be0763698dc
--- /dev/null
+++ b/arch/um/lkl/kernel/asm-offsets.c
@@ -0,0 +1,2 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Dummy asm-offsets.c file. Required by kbuild and ready to be used - hint! */
diff --git a/arch/um/lkl/kernel/misc.c b/arch/um/lkl/kernel/misc.c
new file mode 100644
index 000000000000..60f048f02ae6
--- /dev/null
+++ b/arch/um/lkl/kernel/misc.c
@@ -0,0 +1,60 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/kallsyms.h>
+#include <linux/module.h>
+#include <linux/sched.h>
+#include <linux/seq_file.h>
+#include <asm/ptrace.h>
+#include <asm/host_ops.h>
+
+#ifdef CONFIG_PRINTK
+void dump_stack(void)
+{
+	unsigned long dummy;
+	unsigned long *stack = &dummy;
+	unsigned long addr;
+
+	pr_info("Call Trace:\n");
+	while (((long)stack & (THREAD_SIZE - 1)) != 0) {
+		addr = *stack;
+		if (__kernel_text_address(addr)) {
+			pr_info("%p:  [<%08lx>] %pS", stack, addr,
+				(void *)addr);
+			pr_cont("\n");
+		}
+		stack++;
+	}
+	pr_info("\n");
+}
+#endif
+
+void show_regs(struct pt_regs *regs)
+{
+}
+
+#ifdef CONFIG_PROC_FS
+static void *cpuinfo_start(struct seq_file *m, loff_t *pos)
+{
+	return NULL;
+}
+
+static void *cpuinfo_next(struct seq_file *m, void *v, loff_t *pos)
+{
+	return NULL;
+}
+
+static void cpuinfo_stop(struct seq_file *m, void *v)
+{
+}
+
+static int show_cpuinfo(struct seq_file *m, void *v)
+{
+	return 0;
+}
+
+const struct seq_operations cpuinfo_op = {
+	.start	= cpuinfo_start,
+	.next	= cpuinfo_next,
+	.stop	= cpuinfo_stop,
+	.show	= show_cpuinfo,
+};
+#endif
diff --git a/arch/um/lkl/kernel/vmlinux.lds.S b/arch/um/lkl/kernel/vmlinux.lds.S
new file mode 100644
index 000000000000..f57c563522cb
--- /dev/null
+++ b/arch/um/lkl/kernel/vmlinux.lds.S
@@ -0,0 +1,51 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <asm/vmlinux.lds.h>
+#include <asm/thread_info.h>
+#include <asm/page.h>
+#include <asm/cache.h>
+#include <linux/export.h>
+
+OUTPUT_FORMAT(CONFIG_OUTPUT_FORMAT)
+
+jiffies = jiffies_64;
+
+SECTIONS
+{
+	__init_begin = .;
+	HEAD_TEXT_SECTION
+	INIT_TEXT_SECTION(PAGE_SIZE)
+	INIT_DATA_SECTION(16)
+	PERCPU_SECTION(L1_CACHE_BYTES)
+	__init_end = .;
+
+	_stext = .;
+	_text = . ;
+	text = . ;
+	.text      :
+	{
+		TEXT_TEXT
+		SCHED_TEXT
+		LOCK_TEXT
+		CPUIDLE_TEXT
+	}
+	_etext = .;
+
+	_sdata = .;
+	RO_DATA(PAGE_SIZE)
+	RW_DATA(L1_CACHE_BYTES, PAGE_SIZE, THREAD_SIZE)
+	_edata = .;
+
+	__start_ro_after_init = .;
+	.data..ro_after_init : { *(.data..ro_after_init)}
+	EXCEPTION_TABLE(16)
+	__end_ro_after_init = .;
+	NOTES
+
+	BSS_SECTION(0, 0, 0)
+	_end = .;
+
+	STABS_DEBUG
+	DWARF_DEBUG
+
+	DISCARDS
+}
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v3 03/26] um lkl: architecture skeleton for Linux kernel library
@ 2020-02-05  7:30   ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05  7:30 UTC (permalink / raw)
  To: linux-um
  Cc: linux-arch, Levente Kurusa, Matthieu Coudron, Hajime Tazaki,
	Conrad Meyer, Octavian Purdila, Jens Staal, Motomu Utsumi,
	Lai Jiangshan, Akira Moroo, Petros Angelatos, Yuan Liu, Xiao Jia,
	Mark Stillwell, Patrick Collins, linux-kernel-library,
	Pierre-Hugues Husson, Michael Zimmermann, Luca Dariz,
	Edison M . Castro

From: Octavian Purdila <tavi.purdila@gmail.com>

Adds the LKL Kconfig, vmlinux linker script, basic architecture
headers and miscellaneous basic functions or stubs such as
dump_stack(), show_regs() and cpuinfo proc ops.

The headers we introduce in this patch are simple wrappers to the
asm-generic headers or stubs for things we don't support, such as
ptrace, DMA, signals, ELF handling and low level processor operations.

The kernel configuration is automatically updated to reflect the
endianness of the host, 64bit support or the output format for
vmlinux's linker script. We do this by looking at the ld's default
output format.

Cc: Andreas Abel <aabel@google.com>
Cc: Conrad Meyer <cem@FreeBSD.org>
Cc: Edison M. Castro <edisonmcastro@hotmail.com>
Cc: H.K. Jerry Chu <hkchu@google.com>
Cc: Jens Staal <staal1978@gmail.com>
Cc: Lai Jiangshan <jiangshanlai@gmail.com>
Cc: Levente Kurusa <levex@linux.com>
Cc: Luca Dariz <luca.dariz@gmail.com>
Cc: Mark Stillwell <mark@stillwell.me>
Cc: Matthieu Coudron <mattator@gmail.com>
Cc: Michael Zimmermann <sigmaepsilon92@gmail.com>
Cc: Motomu Utsumi <motomuman@gmail.com>
Cc: Patrick Collins <pscollins@google.com>
Cc: Petros Angelatos <petrosagg@gmail.com>
Cc: Pierre-Hugues Husson <phh@phh.me>
Cc: Xiao Jia <xiaoj@google.com>
Cc: Yuan Liu <liuyuan@google.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 MAINTAINERS                                |    8 +
 arch/um/lkl/.gitignore                     |    2 +
 arch/um/lkl/Kconfig                        |   56 ++
 arch/um/lkl/Kconfig.debug                  |    0
 arch/um/lkl/configs/lkl_defconfig          | 1052 ++++++++++++++++++++
 arch/um/lkl/include/asm/Kbuild             |   80 ++
 arch/um/lkl/include/asm/bitsperlong.h      |   11 +
 arch/um/lkl/include/asm/byteorder.h        |    7 +
 arch/um/lkl/include/asm/cpu.h              |   14 +
 arch/um/lkl/include/asm/elf.h              |   15 +
 arch/um/lkl/include/asm/processor.h        |   60 ++
 arch/um/lkl/include/asm/ptrace.h           |   25 +
 arch/um/lkl/include/asm/sched.h            |   23 +
 arch/um/lkl/include/asm/syscalls.h         |   18 +
 arch/um/lkl/include/asm/syscalls_32.h      |   43 +
 arch/um/lkl/include/asm/tlb.h              |   12 +
 arch/um/lkl/include/asm/uaccess.h          |   64 ++
 arch/um/lkl/include/asm/unistd_32.h        |   31 +
 arch/um/lkl/include/asm/vmlinux.lds.h      |   14 +
 arch/um/lkl/include/asm/xor.h              |    9 +
 arch/um/lkl/include/uapi/asm/Kbuild        |    9 +
 arch/um/lkl/include/uapi/asm/bitsperlong.h |   13 +
 arch/um/lkl/include/uapi/asm/byteorder.h   |   11 +
 arch/um/lkl/include/uapi/asm/siginfo.h     |   11 +
 arch/um/lkl/include/uapi/asm/swab.h        |   11 +
 arch/um/lkl/include/uapi/asm/syscalls.h    |  348 +++++++
 arch/um/lkl/kernel/asm-offsets.c           |    2 +
 arch/um/lkl/kernel/misc.c                  |   60 ++
 arch/um/lkl/kernel/vmlinux.lds.S           |   51 +
 29 files changed, 2060 insertions(+)
 create mode 100644 arch/um/lkl/.gitignore
 create mode 100644 arch/um/lkl/Kconfig
 create mode 100644 arch/um/lkl/Kconfig.debug
 create mode 100644 arch/um/lkl/configs/lkl_defconfig
 create mode 100644 arch/um/lkl/include/asm/Kbuild
 create mode 100644 arch/um/lkl/include/asm/bitsperlong.h
 create mode 100644 arch/um/lkl/include/asm/byteorder.h
 create mode 100644 arch/um/lkl/include/asm/cpu.h
 create mode 100644 arch/um/lkl/include/asm/elf.h
 create mode 100644 arch/um/lkl/include/asm/processor.h
 create mode 100644 arch/um/lkl/include/asm/ptrace.h
 create mode 100644 arch/um/lkl/include/asm/sched.h
 create mode 100644 arch/um/lkl/include/asm/syscalls.h
 create mode 100644 arch/um/lkl/include/asm/syscalls_32.h
 create mode 100644 arch/um/lkl/include/asm/tlb.h
 create mode 100644 arch/um/lkl/include/asm/uaccess.h
 create mode 100644 arch/um/lkl/include/asm/unistd_32.h
 create mode 100644 arch/um/lkl/include/asm/vmlinux.lds.h
 create mode 100644 arch/um/lkl/include/asm/xor.h
 create mode 100644 arch/um/lkl/include/uapi/asm/Kbuild
 create mode 100644 arch/um/lkl/include/uapi/asm/bitsperlong.h
 create mode 100644 arch/um/lkl/include/uapi/asm/byteorder.h
 create mode 100644 arch/um/lkl/include/uapi/asm/siginfo.h
 create mode 100644 arch/um/lkl/include/uapi/asm/swab.h
 create mode 100644 arch/um/lkl/include/uapi/asm/syscalls.h
 create mode 100644 arch/um/lkl/kernel/asm-offsets.c
 create mode 100644 arch/um/lkl/kernel/misc.c
 create mode 100644 arch/um/lkl/kernel/vmlinux.lds.S

diff --git a/MAINTAINERS b/MAINTAINERS
index 4017e6b760be..12afa00402a1 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -9606,6 +9606,14 @@ F:	Documentation/core-api/atomic_ops.rst
 F:	Documentation/core-api/refcount-vs-atomic.rst
 F:	Documentation/memory-barriers.txt
 
+LINUX KERNEL LIBRARY
+M:	Octavian Purdila <tavi.purdila@gmail.com>
+M:	Hajime Tazaki <thehajime@gmail.com>
+L:	linux-kernel-library@freelists.org
+S:	Maintained
+F:	arch/um/lkl/
+F:	tools/lkl/
+
 LIS3LV02D ACCELEROMETER DRIVER
 M:	Eric Piel <eric.piel@tremplin-utc.net>
 S:	Maintained
diff --git a/arch/um/lkl/.gitignore b/arch/um/lkl/.gitignore
new file mode 100644
index 000000000000..ced1c60d8235
--- /dev/null
+++ b/arch/um/lkl/.gitignore
@@ -0,0 +1,2 @@
+kernel/vmlinux.lds
+include/generated
diff --git a/arch/um/lkl/Kconfig b/arch/um/lkl/Kconfig
new file mode 100644
index 000000000000..f7b641ea7aef
--- /dev/null
+++ b/arch/um/lkl/Kconfig
@@ -0,0 +1,56 @@
+# SPDX-License-Identifier: GPL-2.0
+
+config UML_LKL
+       def_bool y
+       depends on !SMP && !MMU && !COREDUMP && !SECCOMP && !UPROBES && !COMPAT && !USER_RETURN_NOTIFIER
+       select ARCH_THREAD_STACK_ALLOCATOR
+       select RWSEM_GENERIC_SPINLOCK
+       select GENERIC_ATOMIC64
+       select GENERIC_HWEIGHT
+       select FLATMEM
+       select FLAT_NODE_MEM_MAP
+       select GENERIC_CLOCKEVENTS
+       select GENERIC_CPU_DEVICES
+       select NO_HZ_IDLE
+       select NO_PREEMPT
+       select ARCH_WANT_FRAME_POINTERS
+       select HAS_DMA
+       select DMA_DIRECT_OPS
+       select PHYS_ADDR_T_64BIT if 64BIT
+       select 64BIT if "$(OUTPUT_FORMAT)" = "elf64-x86-64"
+       select 64BIT if "$(OUTPUT_FORMAT)" = "elf64-x86-64-freebsd"
+       select NET
+       select MULTIUSER
+
+config OUTPUT_FORMAT
+       string "Output format"
+       default "$(OUTPUT_FORMAT)"
+
+config ARCH_DMA_ADDR_T_64BIT
+       def_bool 64BIT
+
+config 64BIT
+       def_bool n
+
+config COREDUMP
+       def_bool n
+
+config BIG_ENDIAN
+       def_bool n
+
+config GENERIC_CSUM
+       def_bool y
+
+config GENERIC_HWEIGHT
+       def_bool y
+
+config NO_IOPORT_MAP
+       def_bool y
+
+config RWSEM_GENERIC_SPINLOCK
+	bool
+	default y
+
+config HZ
+        int
+        default 100
diff --git a/arch/um/lkl/Kconfig.debug b/arch/um/lkl/Kconfig.debug
new file mode 100644
index 000000000000..e69de29bb2d1
diff --git a/arch/um/lkl/configs/lkl_defconfig b/arch/um/lkl/configs/lkl_defconfig
new file mode 100644
index 000000000000..7050c233a17e
--- /dev/null
+++ b/arch/um/lkl/configs/lkl_defconfig
@@ -0,0 +1,1052 @@
+CONFIG_CC_IS_GCC=y
+CONFIG_GCC_VERSION=80301
+CONFIG_CLANG_VERSION=0
+CONFIG_CC_CAN_LINK=y
+CONFIG_CC_HAS_ASM_GOTO=y
+CONFIG_CC_HAS_ASM_INLINE=y
+CONFIG_CC_HAS_WARN_MAYBE_UNINITIALIZED=y
+CONFIG_IRQ_WORK=y
+CONFIG_BROKEN_ON_SMP=y
+CONFIG_INIT_ENV_ARG_LIMIT=32
+# CONFIG_COMPILE_TEST is not set
+CONFIG_LOCALVERSION=""
+# CONFIG_LOCALVERSION_AUTO is not set
+CONFIG_BUILD_SALT=""
+CONFIG_DEFAULT_HOSTNAME="(none)"
+# CONFIG_SYSVIPC is not set
+# CONFIG_POSIX_MQUEUE is not set
+# CONFIG_USELIB is not set
+# CONFIG_AUDIT is not set
+CONFIG_GENERIC_CLOCKEVENTS=y
+CONFIG_TICK_ONESHOT=y
+CONFIG_NO_HZ_COMMON=y
+# CONFIG_HZ_PERIODIC is not set
+CONFIG_NO_HZ_IDLE=y
+# CONFIG_NO_HZ is not set
+CONFIG_HIGH_RES_TIMERS=y
+CONFIG_PREEMPT_NONE=y
+# CONFIG_PREEMPT_VOLUNTARY is not set
+# CONFIG_PREEMPT is not set
+CONFIG_TICK_CPU_ACCOUNTING=y
+# CONFIG_BSD_PROCESS_ACCT is not set
+# CONFIG_TASKSTATS is not set
+# CONFIG_PSI is not set
+CONFIG_TINY_RCU=y
+# CONFIG_RCU_EXPERT is not set
+CONFIG_SRCU=y
+CONFIG_TINY_SRCU=y
+# CONFIG_IKCONFIG is not set
+# CONFIG_IKHEADERS is not set
+CONFIG_LOG_BUF_SHIFT=17
+CONFIG_PRINTK_SAFE_LOG_BUF_SHIFT=13
+CONFIG_CC_HAS_INT128=y
+# CONFIG_CGROUPS is not set
+# CONFIG_NAMESPACES is not set
+# CONFIG_CHECKPOINT_RESTORE is not set
+# CONFIG_SCHED_AUTOGROUP is not set
+# CONFIG_SYSFS_DEPRECATED is not set
+# CONFIG_RELAY is not set
+# CONFIG_BLK_DEV_INITRD is not set
+CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE=y
+# CONFIG_CC_OPTIMIZE_FOR_SIZE is not set
+CONFIG_SYSCTL=y
+CONFIG_BPF=y
+CONFIG_EXPERT=y
+CONFIG_MULTIUSER=y
+# CONFIG_SGETMASK_SYSCALL is not set
+# CONFIG_SYSFS_SYSCALL is not set
+CONFIG_FHANDLE=y
+CONFIG_POSIX_TIMERS=y
+# CONFIG_KALLSYMS_USE_DATA_SECTION is not set
+CONFIG_PRINTK=y
+CONFIG_BUG=y
+# CONFIG_BASE_FULL is not set
+# CONFIG_FUTEX is not set
+CONFIG_EPOLL=y
+# CONFIG_SIGNALFD is not set
+# CONFIG_TIMERFD is not set
+CONFIG_EVENTFD=y
+# CONFIG_AIO is not set
+CONFIG_IO_URING=y
+# CONFIG_ADVISE_SYSCALLS is not set
+CONFIG_MEMBARRIER=y
+CONFIG_KALLSYMS=y
+CONFIG_KALLSYMS_ALL=y
+CONFIG_KALLSYMS_BASE_RELATIVE=y
+# CONFIG_BPF_SYSCALL is not set
+CONFIG_EMBEDDED=y
+# CONFIG_PC104 is not set
+# CONFIG_VM_EVENT_COUNTERS is not set
+CONFIG_SLUB_DEBUG=y
+# CONFIG_COMPAT_BRK is not set
+# CONFIG_SLAB is not set
+CONFIG_SLUB=y
+# CONFIG_SLOB is not set
+CONFIG_SLAB_MERGE_DEFAULT=y
+# CONFIG_SLAB_FREELIST_RANDOM is not set
+# CONFIG_SLAB_FREELIST_HARDENED is not set
+# CONFIG_SHUFFLE_PAGE_ALLOCATOR is not set
+# CONFIG_MMAP_ALLOW_UNINITIALIZED is not set
+# CONFIG_PROFILING is not set
+CONFIG_LKL=y
+# CONFIG_COREDUMP is not set
+CONFIG_GENERIC_CSUM=y
+CONFIG_GENERIC_HWEIGHT=y
+CONFIG_NO_IOPORT_MAP=y
+CONFIG_RWSEM_GENERIC_SPINLOCK=y
+CONFIG_HZ=100
+# CONFIG_STDERR_CONSOLE is not set
+# CONFIG_SSL is not set
+# CONFIG_NULL_CHAN is not set
+# CONFIG_PORT_CHAN is not set
+# CONFIG_PTY_CHAN is not set
+# CONFIG_TTY_CHAN is not set
+# CONFIG_XTERM_CHAN is not set
+CONFIG_NOCONFIG_CHAN=y
+CONFIG_CON_ZERO_CHAN="fd:0,fd:1"
+CONFIG_CON_CHAN="xterm"
+CONFIG_SSL_CHAN="pty"
+# CONFIG_UML_SOUND is not set
+# CONFIG_SOUND is not set
+CONFIG_UML_NET=y
+# CONFIG_UML_NET_ETHERTAP is not set
+CONFIG_UML_NET_TUNTAP=y
+# CONFIG_UML_NET_SLIP is not set
+# CONFIG_UML_NET_DAEMON is not set
+# CONFIG_UML_NET_VECTOR is not set
+# CONFIG_UML_NET_VDE is not set
+# CONFIG_UML_NET_MCAST is not set
+# CONFIG_UML_NET_PCAP is not set
+CONFIG_UML_NET_SLIRP=y
+# CONFIG_VIRTIO_UML is not set
+CONFIG_ARCH_THREAD_STACK_ALLOCATOR=y
+CONFIG_CC_HAS_STACKPROTECTOR_NONE=y
+CONFIG_HAVE_VIRT_CPU_ACCOUNTING_GEN=y
+CONFIG_PGTABLE_LEVELS=2
+CONFIG_PLUGIN_HOSTCC=""
+CONFIG_BASE_SMALL=1
+# CONFIG_MODULES is not set
+CONFIG_BLOCK=y
+# CONFIG_BLK_DEV_BSG is not set
+# CONFIG_BLK_DEV_BSGLIB is not set
+# CONFIG_BLK_DEV_INTEGRITY is not set
+# CONFIG_BLK_DEV_ZONED is not set
+# CONFIG_BLK_CMDLINE_PARSER is not set
+# CONFIG_BLK_WBT is not set
+# CONFIG_BLK_SED_OPAL is not set
+# CONFIG_PARTITION_ADVANCED is not set
+CONFIG_MSDOS_PARTITION=y
+CONFIG_EFI_PARTITION=y
+CONFIG_MQ_IOSCHED_DEADLINE=y
+CONFIG_MQ_IOSCHED_KYBER=y
+# CONFIG_IOSCHED_BFQ is not set
+CONFIG_INLINE_SPIN_UNLOCK_IRQ=y
+CONFIG_INLINE_READ_UNLOCK=y
+CONFIG_INLINE_READ_UNLOCK_IRQ=y
+CONFIG_INLINE_WRITE_UNLOCK=y
+CONFIG_INLINE_WRITE_UNLOCK_IRQ=y
+CONFIG_BINFMT_SCRIPT=y
+# CONFIG_BINFMT_MISC is not set
+CONFIG_FLATMEM=y
+CONFIG_FLAT_NODE_MEM_MAP=y
+CONFIG_SPLIT_PTLOCK_CPUS=999999
+CONFIG_PHYS_ADDR_T_64BIT=y
+CONFIG_NOMMU_INITIAL_TRIM_EXCESS=1
+CONFIG_NEED_PER_CPU_KM=y
+# CONFIG_CLEANCACHE is not set
+# CONFIG_ZPOOL is not set
+# CONFIG_ZBUD is not set
+# CONFIG_PERCPU_STATS is not set
+# CONFIG_GUP_BENCHMARK is not set
+CONFIG_NET=y
+# CONFIG_PACKET is not set
+# CONFIG_UNIX is not set
+# CONFIG_TLS is not set
+# CONFIG_XFRM_USER is not set
+# CONFIG_NET_KEY is not set
+CONFIG_INET=y
+CONFIG_IP_MULTICAST=y
+CONFIG_IP_ADVANCED_ROUTER=y
+# CONFIG_IP_FIB_TRIE_STATS is not set
+CONFIG_IP_MULTIPLE_TABLES=y
+CONFIG_IP_ROUTE_MULTIPATH=y
+# CONFIG_IP_ROUTE_VERBOSE is not set
+CONFIG_IP_PNP=y
+CONFIG_IP_PNP_DHCP=y
+# CONFIG_IP_PNP_BOOTP is not set
+# CONFIG_IP_PNP_RARP is not set
+# CONFIG_NET_IPIP is not set
+# CONFIG_NET_IPGRE_DEMUX is not set
+CONFIG_NET_IP_TUNNEL=y
+# CONFIG_IP_MROUTE is not set
+# CONFIG_SYN_COOKIES is not set
+# CONFIG_NET_IPVTI is not set
+# CONFIG_NET_FOU is not set
+# CONFIG_NET_FOU_IP_TUNNELS is not set
+# CONFIG_INET_AH is not set
+# CONFIG_INET_ESP is not set
+# CONFIG_INET_IPCOMP is not set
+CONFIG_INET_TUNNEL=y
+CONFIG_INET_DIAG=y
+CONFIG_INET_TCP_DIAG=y
+# CONFIG_INET_UDP_DIAG is not set
+# CONFIG_INET_RAW_DIAG is not set
+# CONFIG_INET_DIAG_DESTROY is not set
+CONFIG_TCP_CONG_ADVANCED=y
+CONFIG_TCP_CONG_BIC=y
+CONFIG_TCP_CONG_CUBIC=y
+CONFIG_TCP_CONG_WESTWOOD=y
+CONFIG_TCP_CONG_HTCP=y
+# CONFIG_TCP_CONG_HSTCP is not set
+# CONFIG_TCP_CONG_HYBLA is not set
+# CONFIG_TCP_CONG_VEGAS is not set
+# CONFIG_TCP_CONG_NV is not set
+# CONFIG_TCP_CONG_SCALABLE is not set
+# CONFIG_TCP_CONG_LP is not set
+# CONFIG_TCP_CONG_VENO is not set
+# CONFIG_TCP_CONG_YEAH is not set
+# CONFIG_TCP_CONG_ILLINOIS is not set
+# CONFIG_TCP_CONG_DCTCP is not set
+# CONFIG_TCP_CONG_CDG is not set
+CONFIG_TCP_CONG_BBR=y
+# CONFIG_DEFAULT_BIC is not set
+CONFIG_DEFAULT_CUBIC=y
+# CONFIG_DEFAULT_HTCP is not set
+# CONFIG_DEFAULT_WESTWOOD is not set
+# CONFIG_DEFAULT_BBR is not set
+# CONFIG_DEFAULT_RENO is not set
+CONFIG_DEFAULT_TCP_CONG="cubic"
+# CONFIG_TCP_MD5SIG is not set
+CONFIG_IPV6=y
+# CONFIG_IPV6_ROUTER_PREF is not set
+# CONFIG_IPV6_OPTIMISTIC_DAD is not set
+# CONFIG_INET6_AH is not set
+# CONFIG_INET6_ESP is not set
+# CONFIG_INET6_IPCOMP is not set
+# CONFIG_IPV6_MIP6 is not set
+# CONFIG_IPV6_VTI is not set
+CONFIG_IPV6_SIT=y
+# CONFIG_IPV6_SIT_6RD is not set
+CONFIG_IPV6_NDISC_NODETYPE=y
+# CONFIG_IPV6_TUNNEL is not set
+CONFIG_IPV6_MULTIPLE_TABLES=y
+# CONFIG_IPV6_SUBTREES is not set
+# CONFIG_IPV6_MROUTE is not set
+# CONFIG_IPV6_SEG6_LWTUNNEL is not set
+# CONFIG_IPV6_SEG6_HMAC is not set
+# CONFIG_NETWORK_SECMARK is not set
+# CONFIG_NETWORK_PHY_TIMESTAMPING is not set
+# CONFIG_NETFILTER is not set
+# CONFIG_BPFILTER is not set
+# CONFIG_IP_DCCP is not set
+# CONFIG_IP_SCTP is not set
+# CONFIG_RDS is not set
+# CONFIG_TIPC is not set
+# CONFIG_ATM is not set
+# CONFIG_L2TP is not set
+# CONFIG_BRIDGE is not set
+CONFIG_HAVE_NET_DSA=y
+# CONFIG_NET_DSA is not set
+# CONFIG_VLAN_8021Q is not set
+# CONFIG_DECNET is not set
+# CONFIG_LLC2 is not set
+# CONFIG_ATALK is not set
+# CONFIG_X25 is not set
+# CONFIG_LAPB is not set
+# CONFIG_PHONET is not set
+# CONFIG_6LOWPAN is not set
+# CONFIG_IEEE802154 is not set
+CONFIG_NET_SCHED=y
+# CONFIG_NET_SCH_CBQ is not set
+# CONFIG_NET_SCH_HTB is not set
+# CONFIG_NET_SCH_HFSC is not set
+# CONFIG_NET_SCH_PRIO is not set
+# CONFIG_NET_SCH_MULTIQ is not set
+# CONFIG_NET_SCH_RED is not set
+# CONFIG_NET_SCH_SFB is not set
+# CONFIG_NET_SCH_SFQ is not set
+# CONFIG_NET_SCH_TEQL is not set
+# CONFIG_NET_SCH_TBF is not set
+# CONFIG_NET_SCH_CBS is not set
+# CONFIG_NET_SCH_ETF is not set
+# CONFIG_NET_SCH_TAPRIO is not set
+# CONFIG_NET_SCH_GRED is not set
+# CONFIG_NET_SCH_DSMARK is not set
+# CONFIG_NET_SCH_NETEM is not set
+# CONFIG_NET_SCH_DRR is not set
+# CONFIG_NET_SCH_MQPRIO is not set
+# CONFIG_NET_SCH_SKBPRIO is not set
+# CONFIG_NET_SCH_CHOKE is not set
+# CONFIG_NET_SCH_QFQ is not set
+# CONFIG_NET_SCH_CODEL is not set
+# CONFIG_NET_SCH_FQ_CODEL is not set
+# CONFIG_NET_SCH_CAKE is not set
+CONFIG_NET_SCH_FQ=y
+# CONFIG_NET_SCH_HHF is not set
+# CONFIG_NET_SCH_PIE is not set
+# CONFIG_NET_SCH_PLUG is not set
+# CONFIG_NET_SCH_DEFAULT is not set
+# CONFIG_NET_CLS_BASIC is not set
+# CONFIG_NET_CLS_TCINDEX is not set
+# CONFIG_NET_CLS_ROUTE4 is not set
+# CONFIG_NET_CLS_FW is not set
+# CONFIG_NET_CLS_U32 is not set
+# CONFIG_NET_CLS_RSVP is not set
+# CONFIG_NET_CLS_RSVP6 is not set
+# CONFIG_NET_CLS_FLOW is not set
+# CONFIG_NET_CLS_BPF is not set
+# CONFIG_NET_CLS_FLOWER is not set
+# CONFIG_NET_CLS_MATCHALL is not set
+# CONFIG_NET_EMATCH is not set
+# CONFIG_NET_CLS_ACT is not set
+CONFIG_NET_SCH_FIFO=y
+# CONFIG_DCB is not set
+# CONFIG_BATMAN_ADV is not set
+# CONFIG_OPENVSWITCH is not set
+# CONFIG_VSOCKETS is not set
+# CONFIG_NETLINK_DIAG is not set
+# CONFIG_MPLS is not set
+# CONFIG_NET_NSH is not set
+# CONFIG_HSR is not set
+# CONFIG_NET_SWITCHDEV is not set
+# CONFIG_NET_L3_MASTER_DEV is not set
+# CONFIG_NET_NCSI is not set
+CONFIG_NET_RX_BUSY_POLL=y
+CONFIG_BQL=y
+# CONFIG_NET_PKTGEN is not set
+# CONFIG_HAMRADIO is not set
+# CONFIG_CAN is not set
+# CONFIG_BT is not set
+# CONFIG_AF_RXRPC is not set
+# CONFIG_AF_KCM is not set
+CONFIG_FIB_RULES=y
+# CONFIG_WIRELESS is not set
+# CONFIG_WIMAX is not set
+# CONFIG_RFKILL is not set
+# CONFIG_NET_9P is not set
+# CONFIG_CAIF is not set
+# CONFIG_CEPH_LIB is not set
+# CONFIG_NFC is not set
+# CONFIG_PSAMPLE is not set
+# CONFIG_NET_IFE is not set
+# CONFIG_LWTUNNEL is not set
+CONFIG_DST_CACHE=y
+CONFIG_GRO_CELLS=y
+CONFIG_FAILOVER=y
+# CONFIG_PCCARD is not set
+# CONFIG_UEVENT_HELPER is not set
+# CONFIG_DEVTMPFS is not set
+CONFIG_STANDALONE=y
+CONFIG_PREVENT_FIRMWARE_BUILD=y
+# CONFIG_FW_LOADER is not set
+CONFIG_ALLOW_DEV_COREDUMP=y
+# CONFIG_DEBUG_DRIVER is not set
+# CONFIG_DEBUG_DEVRES is not set
+# CONFIG_DEBUG_TEST_DRIVER_REMOVE is not set
+CONFIG_GENERIC_CPU_DEVICES=y
+# CONFIG_CONNECTOR is not set
+# CONFIG_GNSS is not set
+# CONFIG_MTD is not set
+# CONFIG_OF is not set
+# CONFIG_PARPORT is not set
+CONFIG_BLK_DEV=y
+# CONFIG_BLK_DEV_NULL_BLK is not set
+# CONFIG_BLK_DEV_UBD_SYNC is not set
+CONFIG_BLK_DEV_COW_COMMON=y
+# CONFIG_BLK_DEV_LOOP is not set
+# CONFIG_BLK_DEV_DRBD is not set
+# CONFIG_BLK_DEV_NBD is not set
+# CONFIG_BLK_DEV_RAM is not set
+# CONFIG_CDROM_PKTCDVD is not set
+# CONFIG_ATA_OVER_ETH is not set
+# CONFIG_BLK_DEV_RBD is not set
+# CONFIG_NVME_FC is not set
+# CONFIG_DUMMY_IRQ is not set
+# CONFIG_ENCLOSURE_SERVICES is not set
+# CONFIG_SRAM is not set
+# CONFIG_XILINX_SDFEC is not set
+# CONFIG_C2PORT is not set
+# CONFIG_EEPROM_93CX6 is not set
+# CONFIG_VOP_BUS is not set
+# CONFIG_ECHO is not set
+CONFIG_SCSI_MOD=y
+# CONFIG_RAID_ATTRS is not set
+# CONFIG_SCSI is not set
+# CONFIG_ATA is not set
+# CONFIG_MD is not set
+# CONFIG_TARGET_CORE is not set
+CONFIG_NETDEVICES=y
+CONFIG_NET_CORE=y
+# CONFIG_BONDING is not set
+# CONFIG_DUMMY is not set
+# CONFIG_EQUALIZER is not set
+# CONFIG_NET_TEAM is not set
+# CONFIG_MACVLAN is not set
+# CONFIG_IPVLAN is not set
+# CONFIG_VXLAN is not set
+# CONFIG_GENEVE is not set
+# CONFIG_GTP is not set
+# CONFIG_MACSEC is not set
+# CONFIG_NETCONSOLE is not set
+# CONFIG_TUN is not set
+# CONFIG_TUN_VNET_CROSS_LE is not set
+# CONFIG_VETH is not set
+# CONFIG_NLMON is not set
+# CONFIG_ETHERNET is not set
+# CONFIG_MDIO_DEVICE is not set
+# CONFIG_PHYLIB is not set
+# CONFIG_PPP is not set
+# CONFIG_SLIP is not set
+# CONFIG_WLAN is not set
+# CONFIG_WAN is not set
+CONFIG_NET_FAILOVER=y
+# CONFIG_ISDN is not set
+# CONFIG_NVM is not set
+CONFIG_INPUT=y
+# CONFIG_INPUT_FF_MEMLESS is not set
+# CONFIG_INPUT_POLLDEV is not set
+# CONFIG_INPUT_SPARSEKMAP is not set
+# CONFIG_INPUT_MATRIXKMAP is not set
+# CONFIG_INPUT_MOUSEDEV is not set
+# CONFIG_INPUT_JOYDEV is not set
+# CONFIG_INPUT_EVDEV is not set
+# CONFIG_INPUT_EVBUG is not set
+CONFIG_INPUT_KEYBOARD=y
+CONFIG_KEYBOARD_ATKBD=y
+# CONFIG_KEYBOARD_LKKBD is not set
+# CONFIG_KEYBOARD_NEWTON is not set
+# CONFIG_KEYBOARD_OPENCORES is not set
+# CONFIG_KEYBOARD_STOWAWAY is not set
+# CONFIG_KEYBOARD_SUNKBD is not set
+# CONFIG_KEYBOARD_XTKBD is not set
+CONFIG_INPUT_MOUSE=y
+CONFIG_MOUSE_PS2=y
+CONFIG_MOUSE_PS2_ALPS=y
+CONFIG_MOUSE_PS2_BYD=y
+CONFIG_MOUSE_PS2_LOGIPS2PP=y
+CONFIG_MOUSE_PS2_SYNAPTICS=y
+CONFIG_MOUSE_PS2_CYPRESS=y
+CONFIG_MOUSE_PS2_TRACKPOINT=y
+# CONFIG_MOUSE_PS2_ELANTECH is not set
+# CONFIG_MOUSE_PS2_SENTELIC is not set
+# CONFIG_MOUSE_PS2_TOUCHKIT is not set
+CONFIG_MOUSE_PS2_FOCALTECH=y
+# CONFIG_MOUSE_SERIAL is not set
+# CONFIG_MOUSE_APPLETOUCH is not set
+# CONFIG_MOUSE_BCM5974 is not set
+# CONFIG_MOUSE_VSXXXAA is not set
+# CONFIG_MOUSE_SYNAPTICS_USB is not set
+# CONFIG_INPUT_JOYSTICK is not set
+# CONFIG_INPUT_TABLET is not set
+# CONFIG_INPUT_TOUCHSCREEN is not set
+# CONFIG_INPUT_MISC is not set
+# CONFIG_RMI4_CORE is not set
+CONFIG_SERIO=y
+CONFIG_SERIO_SERPORT=y
+CONFIG_SERIO_LIBPS2=y
+# CONFIG_SERIO_RAW is not set
+# CONFIG_SERIO_ALTERA_PS2 is not set
+# CONFIG_SERIO_PS2MULT is not set
+# CONFIG_SERIO_ARC_PS2 is not set
+# CONFIG_USERIO is not set
+# CONFIG_GAMEPORT is not set
+CONFIG_TTY=y
+# CONFIG_VT is not set
+CONFIG_UNIX98_PTYS=y
+CONFIG_LEGACY_PTYS=y
+CONFIG_LEGACY_PTY_COUNT=256
+# CONFIG_SERIAL_NONSTANDARD is not set
+# CONFIG_N_GSM is not set
+# CONFIG_TRACE_SINK is not set
+# CONFIG_NULL_TTY is not set
+CONFIG_LDISC_AUTOLOAD=y
+CONFIG_DEVMEM=y
+# CONFIG_DEVKMEM is not set
+# CONFIG_SERIAL_8250 is not set
+# CONFIG_SERIAL_UARTLITE is not set
+# CONFIG_SERIAL_SCCNXP is not set
+# CONFIG_SERIAL_ALTERA_JTAGUART is not set
+# CONFIG_SERIAL_ALTERA_UART is not set
+# CONFIG_SERIAL_ARC is not set
+# CONFIG_SERIAL_FSL_LPUART is not set
+# CONFIG_SERIAL_FSL_LINFLEXUART is not set
+# CONFIG_SERIAL_DEV_BUS is not set
+# CONFIG_TTY_PRINTK is not set
+# CONFIG_IPMI_HANDLER is not set
+CONFIG_HW_RANDOM=y
+# CONFIG_HW_RANDOM_TIMERIOMEM is not set
+# CONFIG_RAW_DRIVER is not set
+# CONFIG_TCG_TPM is not set
+# CONFIG_RANDOM_TRUST_BOOTLOADER is not set
+# CONFIG_I2C is not set
+# CONFIG_I3C is not set
+# CONFIG_SPI is not set
+# CONFIG_SPMI is not set
+# CONFIG_HSI is not set
+# CONFIG_PPS is not set
+# CONFIG_PTP_1588_CLOCK is not set
+# CONFIG_PINCTRL is not set
+# CONFIG_GPIOLIB is not set
+# CONFIG_W1 is not set
+# CONFIG_POWER_AVS is not set
+# CONFIG_POWER_RESET is not set
+# CONFIG_POWER_SUPPLY is not set
+CONFIG_HWMON=y
+# CONFIG_HWMON_DEBUG_CHIP is not set
+# CONFIG_SENSORS_AS370 is not set
+# CONFIG_SENSORS_ASPEED is not set
+# CONFIG_SENSORS_F71805F is not set
+# CONFIG_SENSORS_F71882FG is not set
+# CONFIG_SENSORS_IT87 is not set
+# CONFIG_SENSORS_MAX197 is not set
+# CONFIG_SENSORS_PC87360 is not set
+# CONFIG_SENSORS_PC87427 is not set
+# CONFIG_SENSORS_NTC_THERMISTOR is not set
+# CONFIG_SENSORS_NCT6683 is not set
+# CONFIG_SENSORS_NCT6775 is not set
+# CONFIG_SENSORS_NPCM7XX is not set
+# CONFIG_SENSORS_SMSC47M1 is not set
+# CONFIG_SENSORS_SMSC47B397 is not set
+# CONFIG_SENSORS_VT1211 is not set
+# CONFIG_SENSORS_W83627HF is not set
+# CONFIG_SENSORS_W83627EHF is not set
+# CONFIG_THERMAL is not set
+# CONFIG_WATCHDOG is not set
+CONFIG_SSB_POSSIBLE=y
+# CONFIG_SSB is not set
+CONFIG_BCMA_POSSIBLE=y
+# CONFIG_BCMA is not set
+# CONFIG_MFD_MADERA is not set
+# CONFIG_HTC_PASIC3 is not set
+# CONFIG_MFD_KEMPLD is not set
+# CONFIG_MFD_MT6397 is not set
+# CONFIG_MFD_SM501 is not set
+# CONFIG_ABX500_CORE is not set
+# CONFIG_MFD_SYSCON is not set
+# CONFIG_MFD_TI_AM335X_TSCADC is not set
+# CONFIG_MFD_TQMX86 is not set
+# CONFIG_REGULATOR is not set
+# CONFIG_RC_CORE is not set
+# CONFIG_MEDIA_SUPPORT is not set
+# CONFIG_DRM is not set
+# CONFIG_DRM_DP_CEC is not set
+# CONFIG_FB is not set
+# CONFIG_LCD_CLASS_DEVICE is not set
+# CONFIG_BACKLIGHT_CLASS_DEVICE is not set
+CONFIG_HID=y
+# CONFIG_HID_BATTERY_STRENGTH is not set
+# CONFIG_HIDRAW is not set
+# CONFIG_UHID is not set
+CONFIG_HID_GENERIC=y
+# CONFIG_HID_A4TECH is not set
+# CONFIG_HID_ACRUX is not set
+# CONFIG_HID_APPLE is not set
+# CONFIG_HID_AUREAL is not set
+# CONFIG_HID_BELKIN is not set
+# CONFIG_HID_CHERRY is not set
+# CONFIG_HID_CHICONY is not set
+# CONFIG_HID_COUGAR is not set
+# CONFIG_HID_MACALLY is not set
+# CONFIG_HID_CMEDIA is not set
+# CONFIG_HID_CYPRESS is not set
+# CONFIG_HID_DRAGONRISE is not set
+# CONFIG_HID_EMS_FF is not set
+# CONFIG_HID_ELECOM is not set
+# CONFIG_HID_EZKEY is not set
+# CONFIG_HID_GEMBIRD is not set
+# CONFIG_HID_GFRM is not set
+# CONFIG_HID_KEYTOUCH is not set
+# CONFIG_HID_KYE is not set
+# CONFIG_HID_WALTOP is not set
+# CONFIG_HID_VIEWSONIC is not set
+# CONFIG_HID_GYRATION is not set
+# CONFIG_HID_ICADE is not set
+# CONFIG_HID_ITE is not set
+# CONFIG_HID_JABRA is not set
+# CONFIG_HID_TWINHAN is not set
+# CONFIG_HID_KENSINGTON is not set
+# CONFIG_HID_LCPOWER is not set
+# CONFIG_HID_LENOVO is not set
+# CONFIG_HID_MAGICMOUSE is not set
+# CONFIG_HID_MALTRON is not set
+# CONFIG_HID_MAYFLASH is not set
+# CONFIG_HID_REDRAGON is not set
+# CONFIG_HID_MICROSOFT is not set
+# CONFIG_HID_MONTEREY is not set
+# CONFIG_HID_MULTITOUCH is not set
+# CONFIG_HID_NTI is not set
+# CONFIG_HID_ORTEK is not set
+# CONFIG_HID_PANTHERLORD is not set
+# CONFIG_HID_PETALYNX is not set
+# CONFIG_HID_PICOLCD is not set
+# CONFIG_HID_PLANTRONICS is not set
+# CONFIG_HID_PRIMAX is not set
+# CONFIG_HID_SAITEK is not set
+# CONFIG_HID_SAMSUNG is not set
+# CONFIG_HID_SPEEDLINK is not set
+# CONFIG_HID_STEAM is not set
+# CONFIG_HID_STEELSERIES is not set
+# CONFIG_HID_SUNPLUS is not set
+# CONFIG_HID_RMI is not set
+# CONFIG_HID_GREENASIA is not set
+# CONFIG_HID_SMARTJOYPLUS is not set
+# CONFIG_HID_TIVO is not set
+# CONFIG_HID_TOPSEED is not set
+# CONFIG_HID_THRUSTMASTER is not set
+# CONFIG_HID_UDRAW_PS3 is not set
+# CONFIG_HID_XINMO is not set
+# CONFIG_HID_ZEROPLUS is not set
+# CONFIG_HID_ZYDACRON is not set
+# CONFIG_HID_SENSOR_HUB is not set
+# CONFIG_HID_ALPS is not set
+CONFIG_USB_OHCI_LITTLE_ENDIAN=y
+CONFIG_USB_SUPPORT=y
+# CONFIG_USB_ULPI_BUS is not set
+CONFIG_USB_ARCH_HAS_HCD=y
+# CONFIG_USB is not set
+# CONFIG_NOP_USB_XCEIV is not set
+# CONFIG_USB_GADGET is not set
+# CONFIG_TYPEC is not set
+# CONFIG_USB_ROLE_SWITCH is not set
+# CONFIG_MMC is not set
+# CONFIG_MEMSTICK is not set
+# CONFIG_NEW_LEDS is not set
+# CONFIG_ACCESSIBILITY is not set
+# CONFIG_INFINIBAND is not set
+# CONFIG_RTC_CLASS is not set
+# CONFIG_DMADEVICES is not set
+# CONFIG_SYNC_FILE is not set
+# CONFIG_AUXDISPLAY is not set
+# CONFIG_VIRT_DRIVERS is not set
+CONFIG_VIRTIO_MENU=y
+# CONFIG_VIRTIO_MMIO is not set
+# CONFIG_GREYBUS is not set
+# CONFIG_STAGING is not set
+# CONFIG_GOLDFISH is not set
+# CONFIG_HWSPINLOCK is not set
+# CONFIG_MAILBOX is not set
+# CONFIG_REMOTEPROC is not set
+# CONFIG_RPMSG_VIRTIO is not set
+# CONFIG_SOC_TI is not set
+# CONFIG_XILINX_VCU is not set
+# CONFIG_PM_DEVFREQ is not set
+# CONFIG_EXTCON is not set
+# CONFIG_MEMORY is not set
+# CONFIG_IIO is not set
+# CONFIG_PWM is not set
+# CONFIG_IPACK_BUS is not set
+# CONFIG_RESET_CONTROLLER is not set
+# CONFIG_GENERIC_PHY is not set
+# CONFIG_BCM_KONA_USB2_PHY is not set
+# CONFIG_PHY_PXA_28NM_HSIC is not set
+# CONFIG_PHY_PXA_28NM_USB2 is not set
+# CONFIG_POWERCAP is not set
+# CONFIG_MCB is not set
+# CONFIG_RAS is not set
+# CONFIG_ANDROID is not set
+# CONFIG_LIBNVDIMM is not set
+# CONFIG_DAX is not set
+# CONFIG_NVMEM is not set
+# CONFIG_STM is not set
+# CONFIG_INTEL_TH is not set
+# CONFIG_FPGA is not set
+# CONFIG_SIOX is not set
+# CONFIG_SLIMBUS is not set
+# CONFIG_INTERCONNECT is not set
+# CONFIG_COUNTER is not set
+# CONFIG_VALIDATE_FS_PARSER is not set
+CONFIG_FS_IOMAP=y
+# CONFIG_EXT2_FS is not set
+# CONFIG_EXT3_FS is not set
+CONFIG_EXT4_FS=y
+CONFIG_EXT4_USE_FOR_EXT2=y
+CONFIG_EXT4_FS_POSIX_ACL=y
+CONFIG_EXT4_FS_SECURITY=y
+# CONFIG_EXT4_DEBUG is not set
+CONFIG_JBD2=y
+# CONFIG_JBD2_DEBUG is not set
+CONFIG_FS_MBCACHE=y
+# CONFIG_REISERFS_FS is not set
+# CONFIG_JFS_FS is not set
+# CONFIG_XFS_FS is not set
+# CONFIG_GFS2_FS is not set
+CONFIG_BTRFS_FS=y
+# CONFIG_BTRFS_FS_POSIX_ACL is not set
+# CONFIG_BTRFS_FS_CHECK_INTEGRITY is not set
+# CONFIG_BTRFS_FS_RUN_SANITY_TESTS is not set
+# CONFIG_BTRFS_DEBUG is not set
+# CONFIG_BTRFS_ASSERT is not set
+# CONFIG_BTRFS_FS_REF_VERIFY is not set
+# CONFIG_NILFS2_FS is not set
+# CONFIG_F2FS_FS is not set
+CONFIG_FS_POSIX_ACL=y
+CONFIG_EXPORTFS=y
+# CONFIG_EXPORTFS_BLOCK_OPS is not set
+# CONFIG_FILE_LOCKING is not set
+# CONFIG_FS_ENCRYPTION is not set
+# CONFIG_FS_VERITY is not set
+# CONFIG_DNOTIFY is not set
+# CONFIG_INOTIFY_USER is not set
+# CONFIG_FANOTIFY is not set
+# CONFIG_QUOTA is not set
+# CONFIG_AUTOFS4_FS is not set
+# CONFIG_AUTOFS_FS is not set
+# CONFIG_FUSE_FS is not set
+# CONFIG_OVERLAY_FS is not set
+# CONFIG_FSCACHE is not set
+# CONFIG_ISO9660_FS is not set
+# CONFIG_UDF_FS is not set
+CONFIG_FAT_FS=y
+# CONFIG_MSDOS_FS is not set
+CONFIG_VFAT_FS=y
+CONFIG_FAT_DEFAULT_CODEPAGE=437
+CONFIG_FAT_DEFAULT_IOCHARSET="iso8859-1"
+# CONFIG_FAT_DEFAULT_UTF8 is not set
+# CONFIG_NTFS_FS is not set
+CONFIG_PROC_FS=y
+CONFIG_PROC_SYSCTL=y
+# CONFIG_PROC_CHILDREN is not set
+CONFIG_KERNFS=y
+CONFIG_SYSFS=y
+# CONFIG_CONFIGFS_FS is not set
+CONFIG_MISC_FILESYSTEMS=y
+# CONFIG_ORANGEFS_FS is not set
+# CONFIG_ADFS_FS is not set
+# CONFIG_AFFS_FS is not set
+# CONFIG_HFS_FS is not set
+# CONFIG_HFSPLUS_FS is not set
+# CONFIG_BEFS_FS is not set
+# CONFIG_BFS_FS is not set
+# CONFIG_EFS_FS is not set
+# CONFIG_CRAMFS is not set
+# CONFIG_SQUASHFS is not set
+# CONFIG_VXFS_FS is not set
+# CONFIG_MINIX_FS is not set
+# CONFIG_OMFS_FS is not set
+# CONFIG_HPFS_FS is not set
+# CONFIG_QNX4FS_FS is not set
+# CONFIG_QNX6FS_FS is not set
+# CONFIG_ROMFS_FS is not set
+# CONFIG_PSTORE is not set
+# CONFIG_SYSV_FS is not set
+# CONFIG_UFS_FS is not set
+# CONFIG_EROFS_FS is not set
+CONFIG_NETWORK_FILESYSTEMS=y
+# CONFIG_CEPH_FS is not set
+# CONFIG_CIFS is not set
+# CONFIG_CODA_FS is not set
+# CONFIG_AFS_FS is not set
+CONFIG_NLS=y
+CONFIG_NLS_DEFAULT="iso8859-1"
+CONFIG_NLS_CODEPAGE_437=y
+CONFIG_NLS_CODEPAGE_737=y
+CONFIG_NLS_CODEPAGE_775=y
+CONFIG_NLS_CODEPAGE_850=y
+CONFIG_NLS_CODEPAGE_852=y
+CONFIG_NLS_CODEPAGE_855=y
+CONFIG_NLS_CODEPAGE_857=y
+CONFIG_NLS_CODEPAGE_860=y
+CONFIG_NLS_CODEPAGE_861=y
+CONFIG_NLS_CODEPAGE_862=y
+CONFIG_NLS_CODEPAGE_863=y
+CONFIG_NLS_CODEPAGE_864=y
+CONFIG_NLS_CODEPAGE_865=y
+CONFIG_NLS_CODEPAGE_866=y
+CONFIG_NLS_CODEPAGE_869=y
+CONFIG_NLS_CODEPAGE_936=y
+CONFIG_NLS_CODEPAGE_950=y
+CONFIG_NLS_CODEPAGE_932=y
+CONFIG_NLS_CODEPAGE_949=y
+CONFIG_NLS_CODEPAGE_874=y
+CONFIG_NLS_ISO8859_8=y
+CONFIG_NLS_CODEPAGE_1250=y
+CONFIG_NLS_CODEPAGE_1251=y
+CONFIG_NLS_ASCII=y
+CONFIG_NLS_ISO8859_1=y
+CONFIG_NLS_ISO8859_2=y
+CONFIG_NLS_ISO8859_3=y
+CONFIG_NLS_ISO8859_4=y
+CONFIG_NLS_ISO8859_5=y
+CONFIG_NLS_ISO8859_6=y
+CONFIG_NLS_ISO8859_7=y
+CONFIG_NLS_ISO8859_9=y
+CONFIG_NLS_ISO8859_13=y
+CONFIG_NLS_ISO8859_14=y
+CONFIG_NLS_ISO8859_15=y
+CONFIG_NLS_KOI8_R=y
+CONFIG_NLS_KOI8_U=y
+CONFIG_NLS_MAC_ROMAN=y
+CONFIG_NLS_MAC_CELTIC=y
+CONFIG_NLS_MAC_CENTEURO=y
+CONFIG_NLS_MAC_CROATIAN=y
+CONFIG_NLS_MAC_CYRILLIC=y
+CONFIG_NLS_MAC_GAELIC=y
+CONFIG_NLS_MAC_GREEK=y
+CONFIG_NLS_MAC_ICELAND=y
+CONFIG_NLS_MAC_INUIT=y
+CONFIG_NLS_MAC_ROMANIAN=y
+CONFIG_NLS_MAC_TURKISH=y
+CONFIG_NLS_UTF8=y
+# CONFIG_UNICODE is not set
+CONFIG_IO_WQ=y
+# CONFIG_KEYS is not set
+# CONFIG_SECURITY_DMESG_RESTRICT is not set
+# CONFIG_SECURITY is not set
+# CONFIG_SECURITYFS is not set
+CONFIG_HAVE_HARDENED_USERCOPY_ALLOCATOR=y
+# CONFIG_HARDENED_USERCOPY is not set
+# CONFIG_STATIC_USERMODEHELPER is not set
+CONFIG_DEFAULT_SECURITY_DAC=y
+CONFIG_LSM="lockdown,yama,loadpin,safesetid,integrity"
+CONFIG_INIT_STACK_NONE=y
+# CONFIG_INIT_ON_ALLOC_DEFAULT_ON is not set
+# CONFIG_INIT_ON_FREE_DEFAULT_ON is not set
+CONFIG_XOR_BLOCKS=y
+CONFIG_CRYPTO=y
+# CONFIG_CRYPTO_FIPS is not set
+CONFIG_CRYPTO_ALGAPI=y
+CONFIG_CRYPTO_ALGAPI2=y
+CONFIG_CRYPTO_HASH=y
+CONFIG_CRYPTO_HASH2=y
+CONFIG_CRYPTO_RNG=y
+CONFIG_CRYPTO_RNG2=y
+# CONFIG_CRYPTO_MANAGER is not set
+# CONFIG_CRYPTO_USER is not set
+# CONFIG_CRYPTO_NULL is not set
+# CONFIG_CRYPTO_CRYPTD is not set
+# CONFIG_CRYPTO_AUTHENC is not set
+# CONFIG_CRYPTO_RSA is not set
+# CONFIG_CRYPTO_DH is not set
+# CONFIG_CRYPTO_ECDH is not set
+# CONFIG_CRYPTO_ECRDSA is not set
+# CONFIG_CRYPTO_CURVE25519 is not set
+# CONFIG_CRYPTO_CCM is not set
+# CONFIG_CRYPTO_GCM is not set
+# CONFIG_CRYPTO_CHACHA20POLY1305 is not set
+# CONFIG_CRYPTO_AEGIS128 is not set
+# CONFIG_CRYPTO_SEQIV is not set
+# CONFIG_CRYPTO_ECHAINIV is not set
+# CONFIG_CRYPTO_CBC is not set
+# CONFIG_CRYPTO_CFB is not set
+# CONFIG_CRYPTO_CTR is not set
+# CONFIG_CRYPTO_CTS is not set
+# CONFIG_CRYPTO_ECB is not set
+# CONFIG_CRYPTO_LRW is not set
+# CONFIG_CRYPTO_OFB is not set
+# CONFIG_CRYPTO_PCBC is not set
+# CONFIG_CRYPTO_XTS is not set
+# CONFIG_CRYPTO_KEYWRAP is not set
+# CONFIG_CRYPTO_ADIANTUM is not set
+# CONFIG_CRYPTO_ESSIV is not set
+# CONFIG_CRYPTO_CMAC is not set
+# CONFIG_CRYPTO_HMAC is not set
+# CONFIG_CRYPTO_XCBC is not set
+# CONFIG_CRYPTO_VMAC is not set
+CONFIG_CRYPTO_CRC32C=y
+# CONFIG_CRYPTO_CRC32 is not set
+CONFIG_CRYPTO_XXHASH=y
+CONFIG_CRYPTO_BLAKE2B=y
+# CONFIG_CRYPTO_BLAKE2S is not set
+# CONFIG_CRYPTO_CRCT10DIF is not set
+# CONFIG_CRYPTO_GHASH is not set
+# CONFIG_CRYPTO_POLY1305 is not set
+# CONFIG_CRYPTO_MD4 is not set
+# CONFIG_CRYPTO_MD5 is not set
+# CONFIG_CRYPTO_MICHAEL_MIC is not set
+# CONFIG_CRYPTO_RMD128 is not set
+# CONFIG_CRYPTO_RMD160 is not set
+# CONFIG_CRYPTO_RMD256 is not set
+# CONFIG_CRYPTO_RMD320 is not set
+# CONFIG_CRYPTO_SHA1 is not set
+CONFIG_CRYPTO_SHA256=y
+# CONFIG_CRYPTO_SHA512 is not set
+# CONFIG_CRYPTO_SHA3 is not set
+# CONFIG_CRYPTO_SM3 is not set
+# CONFIG_CRYPTO_STREEBOG is not set
+# CONFIG_CRYPTO_TGR192 is not set
+# CONFIG_CRYPTO_WP512 is not set
+CONFIG_CRYPTO_AES=y
+# CONFIG_CRYPTO_AES_TI is not set
+# CONFIG_CRYPTO_ANUBIS is not set
+# CONFIG_CRYPTO_ARC4 is not set
+# CONFIG_CRYPTO_BLOWFISH is not set
+# CONFIG_CRYPTO_CAMELLIA is not set
+# CONFIG_CRYPTO_CAST5 is not set
+# CONFIG_CRYPTO_CAST6 is not set
+# CONFIG_CRYPTO_DES is not set
+# CONFIG_CRYPTO_FCRYPT is not set
+# CONFIG_CRYPTO_KHAZAD is not set
+# CONFIG_CRYPTO_SALSA20 is not set
+# CONFIG_CRYPTO_CHACHA20 is not set
+# CONFIG_CRYPTO_SEED is not set
+# CONFIG_CRYPTO_SERPENT is not set
+# CONFIG_CRYPTO_SM4 is not set
+# CONFIG_CRYPTO_TEA is not set
+# CONFIG_CRYPTO_TWOFISH is not set
+# CONFIG_CRYPTO_DEFLATE is not set
+# CONFIG_CRYPTO_LZO is not set
+# CONFIG_CRYPTO_842 is not set
+# CONFIG_CRYPTO_LZ4 is not set
+# CONFIG_CRYPTO_LZ4HC is not set
+# CONFIG_CRYPTO_ZSTD is not set
+CONFIG_CRYPTO_ANSI_CPRNG=y
+# CONFIG_CRYPTO_DRBG_MENU is not set
+# CONFIG_CRYPTO_JITTERENTROPY is not set
+# CONFIG_CRYPTO_USER_API_HASH is not set
+# CONFIG_CRYPTO_USER_API_SKCIPHER is not set
+# CONFIG_CRYPTO_USER_API_RNG is not set
+# CONFIG_CRYPTO_USER_API_AEAD is not set
+CONFIG_CRYPTO_LIB_AES=y
+# CONFIG_CRYPTO_LIB_BLAKE2S is not set
+# CONFIG_CRYPTO_LIB_CHACHA is not set
+# CONFIG_CRYPTO_LIB_CURVE25519 is not set
+CONFIG_CRYPTO_LIB_POLY1305_RSIZE=1
+# CONFIG_CRYPTO_LIB_POLY1305 is not set
+# CONFIG_CRYPTO_LIB_CHACHA20POLY1305 is not set
+CONFIG_CRYPTO_LIB_SHA256=y
+CONFIG_CRYPTO_HW=y
+# CONFIG_CRYPTO_DEV_AMLOGIC_GXL is not set
+CONFIG_RAID6_PQ=y
+# CONFIG_RAID6_PQ_BENCHMARK is not set
+# CONFIG_PACKING is not set
+CONFIG_BITREVERSE=y
+CONFIG_GENERIC_NET_UTILS=y
+# CONFIG_CORDIC is not set
+# CONFIG_CRC_CCITT is not set
+CONFIG_CRC16=y
+# CONFIG_CRC_T10DIF is not set
+# CONFIG_CRC_ITU_T is not set
+CONFIG_CRC32=y
+# CONFIG_CRC32_SELFTEST is not set
+CONFIG_CRC32_SLICEBY8=y
+# CONFIG_CRC32_SLICEBY4 is not set
+# CONFIG_CRC32_SARWATE is not set
+# CONFIG_CRC32_BIT is not set
+# CONFIG_CRC64 is not set
+# CONFIG_CRC4 is not set
+# CONFIG_CRC7 is not set
+CONFIG_LIBCRC32C=y
+# CONFIG_CRC8 is not set
+CONFIG_XXHASH=y
+# CONFIG_RANDOM32_SELFTEST is not set
+CONFIG_ZLIB_INFLATE=y
+CONFIG_ZLIB_DEFLATE=y
+CONFIG_LZO_COMPRESS=y
+CONFIG_LZO_DECOMPRESS=y
+CONFIG_ZSTD_COMPRESS=y
+CONFIG_ZSTD_DECOMPRESS=y
+# CONFIG_XZ_DEC is not set
+CONFIG_HAS_IOMEM=y
+CONFIG_HAS_DMA=y
+# CONFIG_DMA_API_DEBUG is not set
+CONFIG_DQL=y
+CONFIG_NLATTR=y
+CONFIG_GENERIC_ATOMIC64=y
+# CONFIG_IRQ_POLL is not set
+CONFIG_SBITMAP=y
+# CONFIG_STRING_SELFTEST is not set
+CONFIG_PRINTK_TIME=y
+# CONFIG_PRINTK_CALLER is not set
+CONFIG_CONSOLE_LOGLEVEL_DEFAULT=7
+CONFIG_CONSOLE_LOGLEVEL_QUIET=4
+CONFIG_MESSAGE_LOGLEVEL_DEFAULT=4
+CONFIG_SYMBOLIC_ERRNAME=y
+CONFIG_DEBUG_INFO=y
+CONFIG_DEBUG_INFO_REDUCED=y
+# CONFIG_DEBUG_INFO_SPLIT is not set
+# CONFIG_DEBUG_INFO_DWARF4 is not set
+# CONFIG_DEBUG_INFO_BTF is not set
+# CONFIG_GDB_SCRIPTS is not set
+# CONFIG_ENABLE_MUST_CHECK is not set
+# CONFIG_STRIP_ASM_SYMS is not set
+# CONFIG_READABLE_ASM is not set
+# CONFIG_HEADERS_INSTALL is not set
+CONFIG_OPTIMIZE_INLINING=y
+# CONFIG_DEBUG_SECTION_MISMATCH is not set
+CONFIG_SECTION_MISMATCH_WARN_ONLY=y
+CONFIG_ARCH_WANT_FRAME_POINTERS=y
+CONFIG_FRAME_POINTER=y
+# CONFIG_DEBUG_FORCE_WEAK_PER_CPU is not set
+# CONFIG_MAGIC_SYSRQ is not set
+# CONFIG_DEBUG_FS is not set
+# CONFIG_UBSAN is not set
+CONFIG_UBSAN_ALIGNMENT=y
+CONFIG_DEBUG_KERNEL=y
+CONFIG_DEBUG_MISC=y
+# CONFIG_PAGE_EXTENSION is not set
+# CONFIG_DEBUG_PAGEALLOC is not set
+# CONFIG_PAGE_POISONING is not set
+# CONFIG_DEBUG_OBJECTS is not set
+# CONFIG_SLUB_DEBUG_ON is not set
+# CONFIG_SLUB_STATS is not set
+# CONFIG_DEBUG_STACK_USAGE is not set
+# CONFIG_SCHED_STACK_END_CHECK is not set
+# CONFIG_DEBUG_VM is not set
+# CONFIG_DEBUG_NOMMU_REGIONS is not set
+# CONFIG_DEBUG_MEMORY_INIT is not set
+CONFIG_CC_HAS_KASAN_GENERIC=y
+CONFIG_KASAN_STACK=1
+# CONFIG_DEBUG_SHIRQ is not set
+# CONFIG_PANIC_ON_OOPS is not set
+CONFIG_PANIC_ON_OOPS_VALUE=0
+CONFIG_PANIC_TIMEOUT=0
+# CONFIG_SOFTLOCKUP_DETECTOR is not set
+# CONFIG_DETECT_HUNG_TASK is not set
+# CONFIG_WQ_WATCHDOG is not set
+CONFIG_SCHED_DEBUG=y
+# CONFIG_SCHEDSTATS is not set
+# CONFIG_DEBUG_TIMEKEEPING is not set
+# CONFIG_DEBUG_SPINLOCK is not set
+# CONFIG_DEBUG_MUTEXES is not set
+# CONFIG_DEBUG_RWSEMS is not set
+# CONFIG_DEBUG_ATOMIC_SLEEP is not set
+# CONFIG_DEBUG_LOCKING_API_SELFTESTS is not set
+# CONFIG_LOCK_TORTURE_TEST is not set
+# CONFIG_WW_MUTEX_SELFTEST is not set
+# CONFIG_WARN_ALL_UNSEEDED_RANDOM is not set
+# CONFIG_DEBUG_KOBJECT is not set
+# CONFIG_DEBUG_LIST is not set
+# CONFIG_DEBUG_PLIST is not set
+# CONFIG_DEBUG_SG is not set
+# CONFIG_DEBUG_NOTIFIERS is not set
+# CONFIG_BUG_ON_DATA_CORRUPTION is not set
+# CONFIG_DEBUG_CREDENTIALS is not set
+# CONFIG_RCU_PERF_TEST is not set
+# CONFIG_RCU_TORTURE_TEST is not set
+# CONFIG_RCU_TRACE is not set
+# CONFIG_RCU_EQS_DEBUG is not set
+# CONFIG_DEBUG_WQ_FORCE_RR_CPU is not set
+# CONFIG_DEBUG_BLOCK_EXT_DEVT is not set
+# CONFIG_SAMPLES is not set
+# CONFIG_KUNIT is not set
+# CONFIG_NOTIFIER_ERROR_INJECTION is not set
+# CONFIG_FAULT_INJECTION is not set
+CONFIG_CC_HAS_SANCOV_TRACE_PC=y
+CONFIG_RUNTIME_TESTING_MENU=y
+# CONFIG_TEST_LIST_SORT is not set
+# CONFIG_TEST_SORT is not set
+# CONFIG_BACKTRACE_SELF_TEST is not set
+# CONFIG_RBTREE_TEST is not set
+# CONFIG_REED_SOLOMON_TEST is not set
+# CONFIG_INTERVAL_TREE_TEST is not set
+# CONFIG_ATOMIC64_SELFTEST is not set
+# CONFIG_TEST_HEXDUMP is not set
+# CONFIG_TEST_STRING_HELPERS is not set
+# CONFIG_TEST_STRSCPY is not set
+# CONFIG_TEST_KSTRTOX is not set
+# CONFIG_TEST_PRINTF is not set
+# CONFIG_TEST_BITMAP is not set
+# CONFIG_TEST_BITFIELD is not set
+# CONFIG_TEST_UUID is not set
+# CONFIG_TEST_XARRAY is not set
+# CONFIG_TEST_OVERFLOW is not set
+# CONFIG_TEST_RHASHTABLE is not set
+# CONFIG_TEST_HASH is not set
+# CONFIG_TEST_IDA is not set
+# CONFIG_FIND_BIT_BENCHMARK is not set
+# CONFIG_TEST_SYSCTL is not set
+# CONFIG_TEST_UDELAY is not set
+# CONFIG_TEST_MEMCAT_P is not set
+# CONFIG_TEST_STACKINIT is not set
+# CONFIG_TEST_MEMINIT is not set
+# CONFIG_MEMTEST is not set
diff --git a/arch/um/lkl/include/asm/Kbuild b/arch/um/lkl/include/asm/Kbuild
new file mode 100644
index 000000000000..7f360f641f19
--- /dev/null
+++ b/arch/um/lkl/include/asm/Kbuild
@@ -0,0 +1,80 @@
+generic-y += atomic.h
+generic-y += barrier.h
+generic-y += bitops.h
+generic-y += bug.h
+generic-y += bugs.h
+generic-y += cache.h
+generic-y += cacheflush.h
+generic-y += checksum.h
+generic-y += cmpxchg-local.h
+generic-y += cmpxchg.h
+generic-y += compat.h
+generic-y += cputime.h
+generic-y += current.h
+generic-y += delay.h
+generic-y += device.h
+generic-y += div64.h
+generic-y += dma.h
+generic-y += dma-mapping.h
+generic-y += emergency-restart.h
+generic-y += errno.h
+generic-y += extable.h
+generic-y += exec.h
+generic-y += ftrace.h
+generic-y += futex.h
+generic-y += hardirq.h
+generic-y += hw_irq.h
+generic-y += io.h
+generic-y += ioctl.h
+generic-y += ipcbuf.h
+generic-y += irq_regs.h
+generic-y += irqflags.h
+generic-y += irq_work.h
+generic-y += kdebug.h
+generic-y += kmap_types.h
+generic-y += linkage.h
+generic-y += local.h
+generic-y += local64.h
+generic-y += mcs_spinlock.h
+generic-y += mmiowb.h
+generic-y += mmu.h
+generic-y += mmu_context.h
+generic-y += module.h
+generic-y += msgbuf.h
+generic-y += param.h
+generic-y += parport.h
+generic-y += pci.h
+generic-y += percpu.h
+generic-y += pgalloc.h
+generic-y += poll.h
+generic-y += preempt.h
+generic-y += resource.h
+generic-y += rwsem.h
+generic-y += scatterlist.h
+generic-y += seccomp.h
+generic-y += sections.h
+generic-y += segment.h
+generic-y += sembuf.h
+generic-y += serial.h
+generic-y += shmbuf.h
+generic-y += signal.h
+generic-y += sizes.h
+generic-y += socket.h
+generic-y += sockios.h
+generic-y += stat.h
+generic-y += statfs.h
+generic-y += string.h
+generic-y += swab.h
+generic-y += switch_to.h
+generic-y += syscall.h
+generic-y += termbits.h
+generic-y += termios.h
+generic-y += time.h
+generic-y += timex.h
+generic-y += tlbflush.h
+generic-y += topology.h
+generic-y += trace_clock.h
+generic-y += unaligned.h
+generic-y += user.h
+generic-y += word-at-a-time.h
+generic-y += kprobes.h
diff --git a/arch/um/lkl/include/asm/bitsperlong.h b/arch/um/lkl/include/asm/bitsperlong.h
new file mode 100644
index 000000000000..5745d5e51274
--- /dev/null
+++ b/arch/um/lkl/include/asm/bitsperlong.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __LKL_BITSPERLONG_H
+#define __LKL_BITSPERLONG_H
+
+#include <uapi/asm/bitsperlong.h>
+
+#define BITS_PER_LONG __BITS_PER_LONG
+
+#define BITS_PER_LONG_LONG 64
+
+#endif
diff --git a/arch/um/lkl/include/asm/byteorder.h b/arch/um/lkl/include/asm/byteorder.h
new file mode 100644
index 000000000000..5d0c4efaa44b
--- /dev/null
+++ b/arch/um/lkl/include/asm/byteorder.h
@@ -0,0 +1,7 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_BYTEORDER_H
+#define _ASM_LKL_BYTEORDER_H
+
+#include <uapi/asm/byteorder.h>
+
+#endif /* _ASM_LKL_BYTEORDER_H */
diff --git a/arch/um/lkl/include/asm/cpu.h b/arch/um/lkl/include/asm/cpu.h
new file mode 100644
index 000000000000..d2b8c501c7b1
--- /dev/null
+++ b/arch/um/lkl/include/asm/cpu.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_CPU_H
+#define _ASM_LKL_CPU_H
+
+int lkl_cpu_get(void);
+void lkl_cpu_put(void);
+int lkl_cpu_try_run_irq(int irq);
+int lkl_cpu_init(void);
+void lkl_cpu_shutdown(void);
+void lkl_cpu_wait_shutdown(void);
+void lkl_cpu_change_owner(lkl_thread_t owner);
+void lkl_cpu_set_irqs_pending(void);
+
+#endif /* _ASM_LKL_CPU_H */
diff --git a/arch/um/lkl/include/asm/elf.h b/arch/um/lkl/include/asm/elf.h
new file mode 100644
index 000000000000..bb2456d638f4
--- /dev/null
+++ b/arch/um/lkl/include/asm/elf.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_ELF_H
+#define _ASM_LKL_ELF_H
+
+#define elf_check_arch(x) 0
+
+#ifdef CONFIG_64BIT
+#define ELF_CLASS ELFCLASS64
+#else
+#define ELF_CLASS ELFCLASS32
+#endif
+
+#define elf_gregset_t long
+#define elf_fpregset_t double
+#endif
diff --git a/arch/um/lkl/include/asm/processor.h b/arch/um/lkl/include/asm/processor.h
new file mode 100644
index 000000000000..c1aa8b3a266e
--- /dev/null
+++ b/arch/um/lkl/include/asm/processor.h
@@ -0,0 +1,60 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_PROCESSOR_H
+#define _ASM_LKL_PROCESSOR_H
+
+struct task_struct;
+
+static inline void cpu_relax(void)
+{
+	unsigned long flags;
+
+	/* since this is usually called in a tight loop waiting for some
+	 * external condition (e.g. jiffies) lets run interrupts now to allow
+	 * the external condition to propagate
+	 */
+	local_irq_save(flags);
+	local_irq_restore(flags);
+}
+
+#define current_text_addr() ({ __label__ _l; _l: &&_l; })
+
+static inline unsigned long thread_saved_pc(struct task_struct *tsk)
+{
+	return 0;
+}
+
+static inline void release_thread(struct task_struct *dead_task)
+{
+}
+
+static inline void prepare_to_copy(struct task_struct *tsk)
+{
+}
+
+static inline unsigned long get_wchan(struct task_struct *p)
+{
+	return 0;
+}
+
+static inline void flush_thread(void)
+{
+}
+
+static inline void trap_init(void)
+{
+}
+
+struct thread_struct { };
+
+#define INIT_THREAD { }
+
+#define task_pt_regs(tsk) (struct pt_regs *)(NULL)
+
+/* We don't have strict user/kernel spaces */
+#define TASK_SIZE ((unsigned long)-1)
+#define TASK_UNMAPPED_BASE 0
+
+#define KSTK_EIP(tsk) (0)
+#define KSTK_ESP(tsk) (0)
+
+#endif
diff --git a/arch/um/lkl/include/asm/ptrace.h b/arch/um/lkl/include/asm/ptrace.h
new file mode 100644
index 000000000000..28199be26dc0
--- /dev/null
+++ b/arch/um/lkl/include/asm/ptrace.h
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_PTRACE_H
+#define _ASM_LKL_PTRACE_H
+
+#include <linux/errno.h>
+
+struct task_struct;
+
+#define user_mode(regs) 0
+#define kernel_mode(regs) 1
+#define profile_pc(regs) 0
+#define instruction_pointer(regs) 0
+#define user_stack_pointer(regs) 0
+
+static inline long arch_ptrace(struct task_struct *child, long request,
+			       unsigned long addr, unsigned long data)
+{
+	return -EINVAL;
+}
+
+static inline void ptrace_disable(struct task_struct *child)
+{
+}
+
+#endif
diff --git a/arch/um/lkl/include/asm/sched.h b/arch/um/lkl/include/asm/sched.h
new file mode 100644
index 000000000000..4c2635921ec8
--- /dev/null
+++ b/arch/um/lkl/include/asm/sched.h
@@ -0,0 +1,23 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_SCHED_H
+#define _ASM_LKL_SCHED_H
+
+#include <linux/sched.h>
+#include <uapi/asm/host_ops.h>
+
+static inline void thread_sched_jb(void)
+{
+	if (test_ti_thread_flag(current_thread_info(), TIF_HOST_THREAD)) {
+		set_ti_thread_flag(current_thread_info(), TIF_SCHED_JB);
+		set_current_state(TASK_UNINTERRUPTIBLE);
+		lkl_ops->jmp_buf_set(&current_thread_info()->sched_jb,
+				     schedule);
+	} else {
+		lkl_bug("%s() can be used only for host task\n", __func__);
+	}
+}
+
+void switch_to_host_task(struct task_struct *);
+int host_task_stub(void *unused);
+
+#endif /*  _ASM_LKL_SCHED_H */
diff --git a/arch/um/lkl/include/asm/syscalls.h b/arch/um/lkl/include/asm/syscalls.h
new file mode 100644
index 000000000000..2eaa870a9020
--- /dev/null
+++ b/arch/um/lkl/include/asm/syscalls.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_SYSCALLS_H
+#define _ASM_LKL_SYSCALLS_H
+
+int syscalls_init(void);
+void syscalls_cleanup(void);
+long lkl_syscall(long no, long *params);
+void wakeup_idle_host_task(void);
+
+#define sys_mmap sys_mmap_pgoff
+#define sys_mmap2 sys_mmap_pgoff
+#define sys_clone sys_ni_syscall
+#define sys_vfork sys_ni_syscall
+#define sys_rt_sigreturn sys_ni_syscall
+
+#include <asm-generic/syscalls.h>
+
+#endif /* _ASM_LKL_SYSCALLS_H */
diff --git a/arch/um/lkl/include/asm/syscalls_32.h b/arch/um/lkl/include/asm/syscalls_32.h
new file mode 100644
index 000000000000..0e1a7649c81b
--- /dev/null
+++ b/arch/um/lkl/include/asm/syscalls_32.h
@@ -0,0 +1,43 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_SYSCALLS_32_H
+#define _ASM_SYSCALLS_32_H
+
+#include <linux/compiler.h>
+#include <linux/linkage.h>
+#include <linux/types.h>
+#include <linux/signal.h>
+
+#if __BITS_PER_LONG == 32
+
+/* kernel/syscalls_32.c */
+asmlinkage long sys32_truncate64(const char __user *, unsigned long,
+				 unsigned long);
+asmlinkage long sys32_ftruncate64(unsigned int, unsigned long, unsigned long);
+
+#ifdef CONFIG_MMU
+struct mmap_arg_struct32;
+asmlinkage long sys32_mmap(struct mmap_arg_struct32 __user *);
+#endif
+
+asmlinkage long sys32_wait4(pid_t, unsigned int __user *, int,
+			    struct rusage __user *);
+
+asmlinkage long sys32_pread64(unsigned int, char __user *, u32, u32, u32);
+asmlinkage long sys32_pwrite64(unsigned int, const char __user *, u32, u32,
+			       u32);
+
+long sys32_fadvise64_64(int a, __u32 b, __u32 c, __u32 d, __u32 e, int f);
+
+asmlinkage ssize_t sys32_readahead(int, unsigned int, unsigned int, size_t);
+asmlinkage long sys32_sync_file_range(int, unsigned int, unsigned int,
+				      unsigned int, unsigned int, unsigned int);
+asmlinkage long sys32_sync_file_range2(int, unsigned int, unsigned int,
+				       unsigned int, unsigned int,
+				       unsigned int);
+asmlinkage long sys32_fadvise64(int, unsigned int, unsigned int, size_t, int);
+asmlinkage long sys32_fallocate(int, int, unsigned int, unsigned int,
+				unsigned int, unsigned int);
+
+#endif /* __BITS_PER_LONG */
+
+#endif /* _ASM_SYSCALLS_32_H */
diff --git a/arch/um/lkl/include/asm/tlb.h b/arch/um/lkl/include/asm/tlb.h
new file mode 100644
index 000000000000..d474890d317d
--- /dev/null
+++ b/arch/um/lkl/include/asm/tlb.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_TLB_H
+#define _ASM_LKL_TLB_H
+
+#define tlb_start_vma(tlb, vma)				do { } while (0)
+#define tlb_end_vma(tlb, vma)				do { } while (0)
+#define __tlb_remove_tlb_entry(tlb, pte, address)	do { } while (0)
+#define tlb_flush(tlb)					do { } while (0)
+
+#include <asm-generic/tlb.h>
+
+#endif /* _ASM_LKL_TLB_H */
diff --git a/arch/um/lkl/include/asm/uaccess.h b/arch/um/lkl/include/asm/uaccess.h
new file mode 100644
index 000000000000..f267ac3be8b3
--- /dev/null
+++ b/arch/um/lkl/include/asm/uaccess.h
@@ -0,0 +1,64 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_UACCESS_H
+#define _ASM_LKL_UACCESS_H
+
+/* copied from old include/asm-generic/uaccess.h */
+static inline __must_check long
+raw_copy_from_user(void *to, const void __user *from, unsigned long n)
+{
+	if (__builtin_constant_p(n)) {
+		switch (n) {
+		case 1:
+			*(u8 *)to = *(u8 __force *)from;
+			return 0;
+		case 2:
+			*(u16 *)to = *(u16 __force *)from;
+			return 0;
+		case 4:
+			*(u32 *)to = *(u32 __force *)from;
+			return 0;
+#ifdef CONFIG_64BIT
+		case 8:
+			*(u64 *)to = *(u64 __force *)from;
+			return 0;
+#endif
+		default:
+			break;
+		}
+	}
+
+	memcpy(to, (const void __force *)from, n);
+	return 0;
+}
+
+static inline __must_check long
+raw_copy_to_user(void __user *to, const void *from, unsigned long n)
+{
+	if (__builtin_constant_p(n)) {
+		switch (n) {
+		case 1:
+			*(u8 __force *)to = *(u8 *)from;
+			return 0;
+		case 2:
+			*(u16 __force *)to = *(u16 *)from;
+			return 0;
+		case 4:
+			*(u32 __force *)to = *(u32 *)from;
+			return 0;
+#ifdef CONFIG_64BIT
+		case 8:
+			*(u64 __force *)to = *(u64 *)from;
+			return 0;
+#endif
+		default:
+			break;
+		}
+	}
+
+	memcpy((void __force *)to, from, n);
+	return 0;
+}
+
+#include <asm-generic/uaccess.h>
+
+#endif
diff --git a/arch/um/lkl/include/asm/unistd_32.h b/arch/um/lkl/include/asm/unistd_32.h
new file mode 100644
index 000000000000..8582a55e61e2
--- /dev/null
+++ b/arch/um/lkl/include/asm/unistd_32.h
@@ -0,0 +1,31 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <asm/bitsperlong.h>
+
+#ifndef __SYSCALL
+#define __SYSCALL(x, y)
+#endif
+
+#if __BITS_PER_LONG == 32
+__SYSCALL(__NR3264_truncate, sys32_truncate64)
+__SYSCALL(__NR3264_ftruncate, sys32_ftruncate64)
+
+#ifdef CONFIG_MMU
+__SYSCALL(__NR3264_mmap, sys32_mmap)
+#endif
+
+__SYSCALL(__NR_wait4, sys32_wait4)
+
+__SYSCALL(__NR_pread64, sys32_pread64)
+__SYSCALL(__NR_pwrite64, sys32_pwrite64)
+
+__SYSCALL(__NR_readahead, sys32_readahead)
+#ifdef __ARCH_WANT_SYNC_FILE_RANGE2
+__SYSCALL(__NR_sync_file_range2, sys32_sync_file_range2)
+#else
+__SYSCALL(__NR_sync_file_range, sys32_sync_file_range)
+#endif
+/* mm/fadvise.c */
+__SYSCALL(__NR3264_fadvise64, sys32_fadvise64_64)
+__SYSCALL(__NR_fallocate, sys32_fallocate)
+
+#endif
diff --git a/arch/um/lkl/include/asm/vmlinux.lds.h b/arch/um/lkl/include/asm/vmlinux.lds.h
new file mode 100644
index 000000000000..a3c285882dc4
--- /dev/null
+++ b/arch/um/lkl/include/asm/vmlinux.lds.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_VMLINUX_LDS_H
+#define _LKL_VMLINUX_LDS_H
+
+/* we encode our own __ro_after_init section */
+#define RO_AFTER_INIT_DATA
+
+#ifdef __MINGW32__
+#define RODATA_SECTION .rdata
+#endif
+
+#include <asm-generic/vmlinux.lds.h>
+
+#endif
diff --git a/arch/um/lkl/include/asm/xor.h b/arch/um/lkl/include/asm/xor.h
new file mode 100644
index 000000000000..286ce75b5d9d
--- /dev/null
+++ b/arch/um/lkl/include/asm/xor.h
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_XOR_H
+#define _ASM_LKL_XOR_H
+
+#include <asm-generic/xor.h>
+
+#define XOR_SELECT_TEMPLATE(x) (&xor_block_8regs)
+
+#endif /* _ASM_LKL_XOR_H */
diff --git a/arch/um/lkl/include/uapi/asm/Kbuild b/arch/um/lkl/include/uapi/asm/Kbuild
new file mode 100644
index 000000000000..39d9a1f2e8f5
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/Kbuild
@@ -0,0 +1,9 @@
+# UAPI Header export list
+
+generic-y += elf.h
+generic-y += kvm_para.h
+generic-y += shmparam.h
+generic-y += timex.h
+
+# no header-y since we need special user headers handling
+# see arch/lkl/script/headers.py
diff --git a/arch/um/lkl/include/uapi/asm/bitsperlong.h b/arch/um/lkl/include/uapi/asm/bitsperlong.h
new file mode 100644
index 000000000000..8b4ebf2b0264
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/bitsperlong.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _ASM_UAPI_LKL_BITSPERLONG_H
+#define _ASM_UAPI_LKL_BITSPERLONG_H
+
+#ifdef CONFIG_64BIT
+#define __BITS_PER_LONG 64
+#else
+#define __BITS_PER_LONG 32
+#endif
+
+#define __ARCH_WANT_STAT64
+
+#endif /* _ASM_UAPI_LKL_BITSPERLONG_H */
diff --git a/arch/um/lkl/include/uapi/asm/byteorder.h b/arch/um/lkl/include/uapi/asm/byteorder.h
new file mode 100644
index 000000000000..3c4a58d2062f
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/byteorder.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _ASM_UAPI_LKL_BYTEORDER_H
+#define _ASM_UAPI_LKL_BYTEORDER_H
+
+#if defined(CONFIG_BIG_ENDIAN)
+#include <linux/byteorder/big_endian.h>
+#else
+#include <linux/byteorder/little_endian.h>
+#endif
+
+#endif /* _ASM_UAPI_LKL_BYTEORDER_H */
diff --git a/arch/um/lkl/include/uapi/asm/siginfo.h b/arch/um/lkl/include/uapi/asm/siginfo.h
new file mode 100644
index 000000000000..811916cf42c8
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/siginfo.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _ASM_LKL_SIGINFO_H
+#define _ASM_LKL_SIGINFO_H
+
+#ifdef CONFIG_64BIT
+#define __ARCH_SI_PREAMBLE_SIZE	(4 * sizeof(int))
+#endif
+
+#include <asm-generic/siginfo.h>
+
+#endif /* _ASM_LKL_SIGINFO_H */
diff --git a/arch/um/lkl/include/uapi/asm/swab.h b/arch/um/lkl/include/uapi/asm/swab.h
new file mode 100644
index 000000000000..1a1773e1bd35
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/swab.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _ASM_LKL_SWAB_H
+#define _ASM_LKL_SWAB_H
+
+#ifndef __arch_swab32
+#define __arch_swab32(x) ___constant_swab32(x)
+#endif
+
+#include <asm-generic/swab.h>
+
+#endif /* _ASM_LKL_SWAB_H */
diff --git a/arch/um/lkl/include/uapi/asm/syscalls.h b/arch/um/lkl/include/uapi/asm/syscalls.h
new file mode 100644
index 000000000000..a81534ffccb7
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/syscalls.h
@@ -0,0 +1,348 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _ASM_UAPI_LKL_SYSCALLS_H
+#define _ASM_UAPI_LKL_SYSCALLS_H
+
+#include <autoconf.h>
+#include <linux/types.h>
+
+typedef __kernel_uid32_t	qid_t;
+typedef __kernel_fd_set		fd_set;
+typedef __kernel_mode_t		mode_t;
+typedef unsigned short		umode_t;
+typedef __u32			nlink_t;
+typedef __kernel_off_t		off_t;
+typedef __kernel_pid_t		pid_t;
+typedef __kernel_key_t		key_t;
+typedef __kernel_suseconds_t	suseconds_t;
+typedef __kernel_timer_t	timer_t;
+typedef __kernel_clockid_t	clockid_t;
+typedef __kernel_mqd_t		mqd_t;
+typedef __kernel_uid32_t	uid_t;
+typedef __kernel_gid32_t	gid_t;
+typedef __kernel_uid16_t        uid16_t;
+typedef __kernel_gid16_t        gid16_t;
+typedef unsigned long		uintptr_t;
+#ifdef CONFIG_UID16
+typedef __kernel_old_uid_t	old_uid_t;
+typedef __kernel_old_gid_t	old_gid_t;
+#endif
+typedef __kernel_loff_t		loff_t;
+typedef __kernel_size_t		size_t;
+typedef __kernel_ssize_t	ssize_t;
+typedef __kernel_time_t		time_t;
+typedef __kernel_clock_t	clock_t;
+typedef __u32			u32;
+typedef __s32			s32;
+typedef __u64			u64;
+typedef __s64			s64;
+
+#define __user
+
+#include <asm/unistd.h>
+/* Temporary undefine system calls that don't have data types defined in UAPI
+ * headers
+ */
+#undef __NR_kexec_load
+#undef __NR_getcpu
+#undef __NR_sched_getattr
+#undef __NR_sched_setattr
+#undef __NR_sched_setparam
+#undef __NR_sched_getparam
+#undef __NR_sched_setscheduler
+#undef __NR_name_to_handle_at
+#undef __NR_open_by_handle_at
+
+/* deprecated system calls */
+#undef __NR_epoll_create
+#undef __NR_epoll_wait
+#undef __NR_access
+#undef __NR_chmod
+#undef __NR_chown
+#undef __NR_lchown
+#undef __NR_open
+#undef __NR_creat
+#undef __NR_readlink
+#undef __NR_pipe
+#undef __NR_mknod
+#undef __NR_mkdir
+#undef __NR_rmdir
+#undef __NR_unlink
+#undef __NR_symlink
+#undef __NR_link
+#undef __NR_rename
+#undef __NR_getdents
+#undef __NR_select
+#undef __NR_poll
+#undef __NR_dup2
+#undef __NR_futimesat
+#undef __NR_utimes
+#undef __NR_ustat
+#undef __NR_eventfd
+#undef __NR_bdflush
+#undef __NR_send
+#undef __NR_recv
+
+#undef __NR_umount
+#define __NR_umount __NR_umount2
+
+#ifdef CONFIG_64BIT
+#define __NR_newfstat __NR3264_fstat
+#define __NR_newfstatat __NR3264_fstatat
+#endif
+
+#define __NR_mmap_pgoff __NR3264_mmap
+
+#include <linux/time.h>
+#include <linux/times.h>
+#include <linux/timex.h>
+#include <linux/capability.h>
+#define __KERNEL__ /* to pull in S_ definitions */
+#include <linux/stat.h>
+#undef __KERNEL__
+#include <linux/errno.h>
+#include <linux/fcntl.h>
+#include <linux/fs.h>
+#include <asm/statfs.h>
+#include <asm/stat.h>
+#include <linux/bpf.h>
+#include <linux/msg.h>
+#include <linux/resource.h>
+#include <linux/sysinfo.h>
+#include <linux/shm.h>
+#include <linux/aio_abi.h>
+#include <linux/socket.h>
+#include <linux/perf_event.h>
+#include <linux/sem.h>
+#include <linux/futex.h>
+#include <linux/poll.h>
+#include <linux/mqueue.h>
+#include <linux/eventpoll.h>
+#include <linux/uio.h>
+#include <asm/signal.h>
+#include <asm/siginfo.h>
+#include <linux/utime.h>
+#include <asm/socket.h>
+#include <linux/icmp.h>
+#include <linux/ip.h>
+
+/* Define data structures used in system calls that are not defined in UAPI
+ * headers
+ */
+struct sockaddr {
+	unsigned short int sa_family;
+	char sa_data[14];
+};
+
+#define __UAPI_DEF_IF_NET_DEVICE_FLAGS_LOWER_UP_DORMANT_ECHO 1
+#define __UAPI_DEF_IF_IFNAMSIZ	1
+#define __UAPI_DEF_IF_NET_DEVICE_FLAGS 1
+#define __UAPI_DEF_IF_IFREQ	1
+#define __UAPI_DEF_IF_IFMAP	1
+#include <linux/if.h>
+#define __UAPI_DEF_IN_IPPROTO	1
+#define __UAPI_DEF_IN_ADDR	1
+#define __UAPI_DEF_IN6_ADDR	1
+#define __UAPI_DEF_IP_MREQ	1
+#define __UAPI_DEF_IN_PKTINFO	1
+#define __UAPI_DEF_SOCKADDR_IN	1
+#define __UAPI_DEF_IN_CLASS	1
+#include <linux/in.h>
+#include <linux/in6.h>
+#include <linux/sockios.h>
+#include <linux/route.h>
+#include <linux/ipv6_route.h>
+#include <linux/ipv6.h>
+#include <linux/netlink.h>
+#include <linux/neighbour.h>
+#include <linux/rtnetlink.h>
+#include <linux/fib_rules.h>
+
+#include <linux/kdev_t.h>
+#include <asm/irq.h>
+#include <linux/virtio_blk.h>
+#include <linux/virtio_net.h>
+#include <linux/virtio_ring.h>
+#include <linux/pkt_sched.h>
+#include <linux/io_uring.h>
+
+struct user_msghdr {
+	void		__user *msg_name;
+	int		msg_namelen;
+	struct iovec	__user *msg_iov;
+	__kernel_size_t	msg_iovlen;
+	void		__user *msg_control;
+	__kernel_size_t	msg_controllen;
+	unsigned int	msg_flags;
+};
+
+typedef __u32 key_serial_t;
+
+struct mmsghdr {
+	struct user_msghdr  msg_hdr;
+	unsigned int        msg_len;
+};
+
+struct linux_dirent64 {
+	u64		d_ino;
+	s64		d_off;
+	unsigned short	d_reclen;
+	unsigned char	d_type;
+	char		d_name[0];
+};
+
+struct linux_dirent {
+	unsigned long	d_ino;
+	unsigned long	d_off;
+	unsigned short	d_reclen;
+	char		d_name[1];
+};
+
+struct ustat {
+	__kernel_daddr_t	f_tfree;
+	__kernel_ino_t		f_tinode;
+	char			f_fname[6];
+	char			f_fpack[6];
+};
+
+typedef __kernel_rwf_t		rwf_t;
+
+#define AF_UNSPEC       0
+#define AF_UNIX         1
+#define AF_LOCAL        1
+#define AF_INET         2
+#define AF_AX25         3
+#define AF_IPX          4
+#define AF_APPLETALK    5
+#define AF_NETROM       6
+#define AF_BRIDGE       7
+#define AF_ATMPVC       8
+#define AF_X25          9
+#define AF_INET6        10
+#define AF_ROSE         11
+#define AF_DECnet       12
+#define AF_NETBEUI      13
+#define AF_SECURITY     14
+#define AF_KEY          15
+#define AF_NETLINK      16
+#define AF_ROUTE        AF_NETLINK
+#define AF_PACKET       17
+#define AF_ASH          18
+#define AF_ECONET       19
+#define AF_ATMSVC       20
+#define AF_RDS          21
+#define AF_SNA          22
+#define AF_IRDA         23
+#define AF_PPPOX        24
+#define AF_WANPIPE      25
+#define AF_LLC          26
+#define AF_IB           27
+#define AF_MPLS         28
+#define AF_CAN          29
+#define AF_TIPC         30
+#define AF_BLUETOOTH    31
+#define AF_IUCV         32
+#define AF_RXRPC        33
+#define AF_ISDN         34
+#define AF_PHONET       35
+#define AF_IEEE802154   36
+#define AF_CAIF         37
+#define AF_ALG          38
+#define AF_NFC          39
+#define AF_VSOCK        40
+
+#define SOCK_STREAM		1
+#define SOCK_DGRAM		2
+#define SOCK_RAW		3
+#define SOCK_RDM		4
+#define SOCK_SEQPACKET		5
+#define SOCK_DCCP		6
+#define SOCK_PACKET		10
+
+#define MSG_TRUNC 0x20
+#define MSG_DONTWAIT 0x40
+
+/* avoid colision with system headers defines */
+#define sa_handler sa_handler
+#define st_atime st_atime
+#define st_mtime st_mtime
+#define st_ctime st_ctime
+#define s_addr s_addr
+
+long lkl_syscall(long no, long *params);
+long lkl_sys_halt(void);
+
+#define __MAP0(m, ...)
+#define __MAP1(m, t, a) m(t, a)
+#define __MAP2(m, t, a, ...) m(t, a), __MAP1(m, __VA_ARGS__)
+#define __MAP3(m, t, a, ...) m(t, a), __MAP2(m, __VA_ARGS__)
+#define __MAP4(m, t, a, ...) m(t, a), __MAP3(m, __VA_ARGS__)
+#define __MAP5(m, t, a, ...) m(t, a), __MAP4(m, __VA_ARGS__)
+#define __MAP6(m, t, a, ...) m(t, a), __MAP5(m, __VA_ARGS__)
+#define __MAP(n, ...) __MAP##n(__VA_ARGS__)
+
+#define __SC_LONG(t, a) (long)a
+#define __SC_TABLE(t, a) {sizeof(t), (long long)(a)}
+#define __SC_DECL(t, a) t a
+
+#define LKL_SYSCALL0(name)					       \
+	static inline long lkl_sys##name(void)			       \
+	{							       \
+		long params[6];					       \
+		return lkl_syscall(__lkl__NR##name, params);	       \
+	}
+
+#if __BITS_PER_LONG == 32
+#define LKL_SYSCALLx(x, name, ...)					\
+	static inline							\
+	long lkl_sys##name(__MAP(x, __SC_DECL, __VA_ARGS__))		\
+	{								\
+		struct {						\
+			unsigned int size;				\
+			long long value;				\
+		} lkl_params[x] = { __MAP(x, __SC_TABLE, __VA_ARGS__) }; \
+		long sys_params[6], i, k;				\
+		for (i = k = 0; i < x && k < 6; i++, k++) {		\
+			if (lkl_params[i].size > sizeof(long) &&	\
+			    k + 1 < 6) {				\
+				sys_params[k] =				\
+					(long)(lkl_params[i].value & (-1UL)); \
+				k++;					\
+				sys_params[k] =				\
+					(long)(lkl_params[i].value >>	\
+					       __BITS_PER_LONG);	\
+			} else {					\
+				sys_params[k] = (long)(lkl_params[i].value); \
+			}						\
+		}							\
+		return lkl_syscall(__lkl__NR##name, sys_params);	\
+	}
+#else
+#define LKL_SYSCALLx(x, name, ...)					\
+	static inline							\
+	long lkl_sys##name(__MAP(x, __SC_DECL, __VA_ARGS__))		\
+	{								\
+		long lkl_params[6] = { __MAP(x, __SC_LONG, __VA_ARGS__) }; \
+		return lkl_syscall(__lkl__NR##name, lkl_params);	\
+	}
+#endif
+
+#define SYSCALL_DEFINE0(name, ...) LKL_SYSCALL0(name)
+#define SYSCALL_DEFINE1(name, ...) LKL_SYSCALLx(1, name, __VA_ARGS__)
+#define SYSCALL_DEFINE2(name, ...) LKL_SYSCALLx(2, name, __VA_ARGS__)
+#define SYSCALL_DEFINE3(name, ...) LKL_SYSCALLx(3, name, __VA_ARGS__)
+#define SYSCALL_DEFINE4(name, ...) LKL_SYSCALLx(4, name, __VA_ARGS__)
+#define SYSCALL_DEFINE5(name, ...) LKL_SYSCALLx(5, name, __VA_ARGS__)
+#define SYSCALL_DEFINE6(name, ...) LKL_SYSCALLx(6, name, __VA_ARGS__)
+
+#if __BITS_PER_LONG == 32
+#pragma GCC diagnostic push
+#pragma GCC diagnostic ignored "-Wpointer-to-int-cast"
+#endif
+
+#include <asm/syscall_defs.h>
+
+#if __BITS_PER_LONG == 32
+#pragma GCC diagnostic pop
+#endif
+
+#endif
diff --git a/arch/um/lkl/kernel/asm-offsets.c b/arch/um/lkl/kernel/asm-offsets.c
new file mode 100644
index 000000000000..6be0763698dc
--- /dev/null
+++ b/arch/um/lkl/kernel/asm-offsets.c
@@ -0,0 +1,2 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Dummy asm-offsets.c file. Required by kbuild and ready to be used - hint! */
diff --git a/arch/um/lkl/kernel/misc.c b/arch/um/lkl/kernel/misc.c
new file mode 100644
index 000000000000..60f048f02ae6
--- /dev/null
+++ b/arch/um/lkl/kernel/misc.c
@@ -0,0 +1,60 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/kallsyms.h>
+#include <linux/module.h>
+#include <linux/sched.h>
+#include <linux/seq_file.h>
+#include <asm/ptrace.h>
+#include <asm/host_ops.h>
+
+#ifdef CONFIG_PRINTK
+void dump_stack(void)
+{
+	unsigned long dummy;
+	unsigned long *stack = &dummy;
+	unsigned long addr;
+
+	pr_info("Call Trace:\n");
+	while (((long)stack & (THREAD_SIZE - 1)) != 0) {
+		addr = *stack;
+		if (__kernel_text_address(addr)) {
+			pr_info("%p:  [<%08lx>] %pS", stack, addr,
+				(void *)addr);
+			pr_cont("\n");
+		}
+		stack++;
+	}
+	pr_info("\n");
+}
+#endif
+
+void show_regs(struct pt_regs *regs)
+{
+}
+
+#ifdef CONFIG_PROC_FS
+static void *cpuinfo_start(struct seq_file *m, loff_t *pos)
+{
+	return NULL;
+}
+
+static void *cpuinfo_next(struct seq_file *m, void *v, loff_t *pos)
+{
+	return NULL;
+}
+
+static void cpuinfo_stop(struct seq_file *m, void *v)
+{
+}
+
+static int show_cpuinfo(struct seq_file *m, void *v)
+{
+	return 0;
+}
+
+const struct seq_operations cpuinfo_op = {
+	.start	= cpuinfo_start,
+	.next	= cpuinfo_next,
+	.stop	= cpuinfo_stop,
+	.show	= show_cpuinfo,
+};
+#endif
diff --git a/arch/um/lkl/kernel/vmlinux.lds.S b/arch/um/lkl/kernel/vmlinux.lds.S
new file mode 100644
index 000000000000..f57c563522cb
--- /dev/null
+++ b/arch/um/lkl/kernel/vmlinux.lds.S
@@ -0,0 +1,51 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <asm/vmlinux.lds.h>
+#include <asm/thread_info.h>
+#include <asm/page.h>
+#include <asm/cache.h>
+#include <linux/export.h>
+
+OUTPUT_FORMAT(CONFIG_OUTPUT_FORMAT)
+
+jiffies = jiffies_64;
+
+SECTIONS
+{
+	__init_begin = .;
+	HEAD_TEXT_SECTION
+	INIT_TEXT_SECTION(PAGE_SIZE)
+	INIT_DATA_SECTION(16)
+	PERCPU_SECTION(L1_CACHE_BYTES)
+	__init_end = .;
+
+	_stext = .;
+	_text = . ;
+	text = . ;
+	.text      :
+	{
+		TEXT_TEXT
+		SCHED_TEXT
+		LOCK_TEXT
+		CPUIDLE_TEXT
+	}
+	_etext = .;
+
+	_sdata = .;
+	RO_DATA(PAGE_SIZE)
+	RW_DATA(L1_CACHE_BYTES, PAGE_SIZE, THREAD_SIZE)
+	_edata = .;
+
+	__start_ro_after_init = .;
+	.data..ro_after_init : { *(.data..ro_after_init)}
+	EXCEPTION_TABLE(16)
+	__end_ro_after_init = .;
+	NOTES
+
+	BSS_SECTION(0, 0, 0)
+	_end = .;
+
+	STABS_DEBUG
+	DWARF_DEBUG
+
+	DISCARDS
+}
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v3 04/26] um lkl: host interface
  2020-02-05  7:30 [RFC v3 00/26] Unifying LKL into UML Hajime Tazaki
@ 2020-02-05  7:30   ` Hajime Tazaki
       [not found] ` <cover.1580882335.git.thehajime-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
                     ` (25 subsequent siblings)
  26 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05  7:30 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Michael Zimmermann, Patrick Collins, Pierre-Hugues Husson,
	Yuan Liu, Hajime Tazaki

From: Octavian Purdila <tavi.purdila@gmail.com>

This patch introduces the host operations that define the interface
between the LKL and the host. These operations must be provided either
by a host library or by the application itself.

Cc: Michael Zimmermann <sigmaepsilon92@gmail.com>
Cc: Patrick Collins <pscollins@google.com>
Cc: Pierre-Hugues Husson <phh@phh.me>
Cc: Yuan Liu <liuyuan@google.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/lkl/include/asm/host_ops.h      | 10 ++++++++++
 arch/um/lkl/include/uapi/asm/host_ops.h | 26 +++++++++++++++++++++++++
 2 files changed, 36 insertions(+)
 create mode 100644 arch/um/lkl/include/asm/host_ops.h
 create mode 100644 arch/um/lkl/include/uapi/asm/host_ops.h

diff --git a/arch/um/lkl/include/asm/host_ops.h b/arch/um/lkl/include/asm/host_ops.h
new file mode 100644
index 000000000000..65850f394b79
--- /dev/null
+++ b/arch/um/lkl/include/asm/host_ops.h
@@ -0,0 +1,10 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_HOST_OPS_H
+#define _ASM_LKL_HOST_OPS_H
+
+#include "irq.h"
+#include <uapi/asm/host_ops.h>
+
+extern struct lkl_host_operations *lkl_ops;
+
+#endif
diff --git a/arch/um/lkl/include/uapi/asm/host_ops.h b/arch/um/lkl/include/uapi/asm/host_ops.h
new file mode 100644
index 000000000000..7cfb0a93e6a6
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/host_ops.h
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _ASM_UAPI_LKL_HOST_OPS_H
+#define _ASM_UAPI_LKL_HOST_OPS_H
+
+/* Defined in {posix,nt}-host.c */
+struct lkl_mutex;
+struct lkl_sem;
+struct lkl_tls_key;
+typedef unsigned long lkl_thread_t;
+struct lkl_jmp_buf {
+	unsigned long buf[32];
+};
+
+/**
+ * lkl_host_operations - host operations used by the Linux kernel
+ *
+ * These operations must be provided by a host library or by the application
+ * itself.
+ *
+ */
+struct lkl_host_operations {
+};
+
+void lkl_bug(const char *fmt, ...);
+
+#endif
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v3 04/26] um lkl: host interface
@ 2020-02-05  7:30   ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05  7:30 UTC (permalink / raw)
  To: linux-um
  Cc: linux-arch, Hajime Tazaki, Octavian Purdila, Akira Moroo,
	Patrick Collins, linux-kernel-library, Pierre-Hugues Husson,
	Michael Zimmermann, Yuan Liu

From: Octavian Purdila <tavi.purdila@gmail.com>

This patch introduces the host operations that define the interface
between the LKL and the host. These operations must be provided either
by a host library or by the application itself.

Cc: Michael Zimmermann <sigmaepsilon92@gmail.com>
Cc: Patrick Collins <pscollins@google.com>
Cc: Pierre-Hugues Husson <phh@phh.me>
Cc: Yuan Liu <liuyuan@google.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/lkl/include/asm/host_ops.h      | 10 ++++++++++
 arch/um/lkl/include/uapi/asm/host_ops.h | 26 +++++++++++++++++++++++++
 2 files changed, 36 insertions(+)
 create mode 100644 arch/um/lkl/include/asm/host_ops.h
 create mode 100644 arch/um/lkl/include/uapi/asm/host_ops.h

diff --git a/arch/um/lkl/include/asm/host_ops.h b/arch/um/lkl/include/asm/host_ops.h
new file mode 100644
index 000000000000..65850f394b79
--- /dev/null
+++ b/arch/um/lkl/include/asm/host_ops.h
@@ -0,0 +1,10 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_HOST_OPS_H
+#define _ASM_LKL_HOST_OPS_H
+
+#include "irq.h"
+#include <uapi/asm/host_ops.h>
+
+extern struct lkl_host_operations *lkl_ops;
+
+#endif
diff --git a/arch/um/lkl/include/uapi/asm/host_ops.h b/arch/um/lkl/include/uapi/asm/host_ops.h
new file mode 100644
index 000000000000..7cfb0a93e6a6
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/host_ops.h
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _ASM_UAPI_LKL_HOST_OPS_H
+#define _ASM_UAPI_LKL_HOST_OPS_H
+
+/* Defined in {posix,nt}-host.c */
+struct lkl_mutex;
+struct lkl_sem;
+struct lkl_tls_key;
+typedef unsigned long lkl_thread_t;
+struct lkl_jmp_buf {
+	unsigned long buf[32];
+};
+
+/**
+ * lkl_host_operations - host operations used by the Linux kernel
+ *
+ * These operations must be provided by a host library or by the application
+ * itself.
+ *
+ */
+struct lkl_host_operations {
+};
+
+void lkl_bug(const char *fmt, ...);
+
+#endif
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v3 05/26] um lkl: memory handling
  2020-02-05  7:30 [RFC v3 00/26] Unifying LKL into UML Hajime Tazaki
@ 2020-02-05  7:30   ` Hajime Tazaki
       [not found] ` <cover.1580882335.git.thehajime-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
                     ` (25 subsequent siblings)
  26 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05  7:30 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Levente Kurusa, Yuan Liu, Hajime Tazaki

From: Octavian Purdila <tavi.purdila@gmail.com>

LKL is a non MMU architecture and hence there is not much work left to
do other than initializing the boot allocator and providing the page
and page table definitions.

The backstore memory is allocated via a host operation and the memory
size to be used is specified when the kernel is started, in the
lkl_start_kernel call.

Cc: H.K. Jerry Chu <hkchu@google.com>
Cc: Levente Kurusa <levex@linux.com>
Cc: Yuan Liu <liuyuan@google.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/lkl/include/asm/page.h          | 14 ++++++
 arch/um/lkl/include/asm/pgtable.h       | 57 +++++++++++++++++++++
 arch/um/lkl/include/uapi/asm/host_ops.h |  5 ++
 arch/um/lkl/mm/bootmem.c                | 66 +++++++++++++++++++++++++
 4 files changed, 142 insertions(+)
 create mode 100644 arch/um/lkl/include/asm/page.h
 create mode 100644 arch/um/lkl/include/asm/pgtable.h
 create mode 100644 arch/um/lkl/mm/bootmem.c

diff --git a/arch/um/lkl/include/asm/page.h b/arch/um/lkl/include/asm/page.h
new file mode 100644
index 000000000000..e77f3da22031
--- /dev/null
+++ b/arch/um/lkl/include/asm/page.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_PAGE_H
+#define _ASM_LKL_PAGE_H
+
+#define CONFIG_KERNEL_RAM_BASE_ADDRESS memory_start
+
+#ifndef __ASSEMBLY__
+void free_mem(void);
+void bootmem_init(unsigned long mem_size);
+#endif
+
+#include <asm-generic/page.h>
+
+#endif /* _ASM_LKL_PAGE_H */
diff --git a/arch/um/lkl/include/asm/pgtable.h b/arch/um/lkl/include/asm/pgtable.h
new file mode 100644
index 000000000000..733beb6d53f6
--- /dev/null
+++ b/arch/um/lkl/include/asm/pgtable.h
@@ -0,0 +1,57 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_PGTABLE_H
+#define _LKL_PGTABLE_H
+
+#include <asm-generic/pgtable-nopud.h>
+
+/*
+ * (C) Copyright 2000-2002, Greg Ungerer <gerg@snapgear.com>
+ */
+
+#include <linux/slab.h>
+#include <asm/processor.h>
+#include <asm/io.h>
+
+#define pgd_present(pgd)	(1)
+#define pgd_none(pgd)		(0)
+#define pgd_bad(pgd)		(0)
+#define pgd_clear(pgdp)
+#define kern_addr_valid(addr)	(1)
+#define	pmd_offset(a, b)	((void *)0)
+
+#define PAGE_NONE		__pgprot(0)
+#define PAGE_SHARED		__pgprot(0)
+#define PAGE_COPY		__pgprot(0)
+#define PAGE_READONLY		__pgprot(0)
+#define PAGE_KERNEL		__pgprot(0)
+
+void paging_init(void);
+#define swapper_pg_dir		((pgd_t *)0)
+
+#define __swp_type(x)		(0)
+#define __swp_offset(x)		(0)
+#define __swp_entry(typ, off)	((swp_entry_t) { ((typ) | ((off) << 7)) })
+#define __pte_to_swp_entry(pte)	((swp_entry_t) { pte_val(pte) })
+#define __swp_entry_to_pte(x)	((pte_t) { (x).val })
+
+/*
+ * ZERO_PAGE is a global shared page that is always zero: used
+ * for zero-mapped memory areas etc..
+ */
+extern void *empty_zero_page;
+#define ZERO_PAGE(vaddr)	(virt_to_page(empty_zero_page))
+
+/*
+ * All 32bit addresses are effectively valid for vmalloc...
+ * Sort of meaningless for non-VM targets.
+ */
+#define	VMALLOC_START		0
+#define	VMALLOC_END		0xffffffff
+#define	KMAP_START		0
+#define	KMAP_END		0xffffffff
+
+#include <asm-generic/pgtable.h>
+
+#define check_pgt_cache()	do { } while (0)
+
+#endif
diff --git a/arch/um/lkl/include/uapi/asm/host_ops.h b/arch/um/lkl/include/uapi/asm/host_ops.h
index 7cfb0a93e6a6..6bbc94c120be 100644
--- a/arch/um/lkl/include/uapi/asm/host_ops.h
+++ b/arch/um/lkl/include/uapi/asm/host_ops.h
@@ -17,8 +17,13 @@ struct lkl_jmp_buf {
  * These operations must be provided by a host library or by the application
  * itself.
  *
+ * @mem_alloc - allocate memory
+ * @mem_free - free memory
+ *
  */
 struct lkl_host_operations {
+	void *(*mem_alloc)(unsigned long mem);
+	void (*mem_free)(void *mem);
 };
 
 void lkl_bug(const char *fmt, ...);
diff --git a/arch/um/lkl/mm/bootmem.c b/arch/um/lkl/mm/bootmem.c
new file mode 100644
index 000000000000..39dd0d22b44e
--- /dev/null
+++ b/arch/um/lkl/mm/bootmem.c
@@ -0,0 +1,66 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/memblock.h>
+#include <linux/mm.h>
+#include <linux/swap.h>
+
+unsigned long memory_start, memory_end;
+static unsigned long _memory_start, mem_size;
+
+void *empty_zero_page;
+
+void __init bootmem_init(unsigned long mem_sz)
+{
+	mem_size = mem_sz;
+
+	_memory_start = (unsigned long)lkl_ops->mem_alloc(mem_size);
+	memory_start = _memory_start;
+	WARN_ON(!memory_start);
+	memory_end = memory_start + mem_size;
+
+	if (PAGE_ALIGN(memory_start) != memory_start) {
+		mem_size -= PAGE_ALIGN(memory_start) - memory_start;
+		memory_start = PAGE_ALIGN(memory_start);
+		mem_size = (mem_size / PAGE_SIZE) * PAGE_SIZE;
+	}
+	pr_info("memblock address range: 0x%lx - 0x%lx\n", memory_start,
+		memory_start + mem_size);
+	/*
+	 * Give all the memory to the bootmap allocator, tell it to put the
+	 * boot mem_map at the start of memory.
+	 */
+	max_low_pfn = virt_to_pfn(memory_end);
+	min_low_pfn = virt_to_pfn(memory_start);
+	memblock_add(memory_start, mem_size);
+
+	empty_zero_page = memblock_alloc(PAGE_SIZE, PAGE_SIZE);
+	memset((void *)empty_zero_page, 0, PAGE_SIZE);
+
+	{
+		unsigned long zones_size[MAX_NR_ZONES] = {0, };
+
+		zones_size[ZONE_NORMAL] = (mem_size) >> PAGE_SHIFT;
+		free_area_init(zones_size);
+	}
+}
+
+void __init mem_init(void)
+{
+	max_mapnr = (((unsigned long)high_memory) - PAGE_OFFSET) >> PAGE_SHIFT;
+	/* this will put all memory onto the freelists */
+	totalram_pages_add(memblock_free_all());
+	pr_info("Memory available: %luk/%luk RAM\n",
+		(nr_free_pages() << PAGE_SHIFT) >> 10, mem_size >> 10);
+}
+
+/*
+ * In our case __init memory is not part of the page allocator so there is
+ * nothing to free.
+ */
+void free_initmem(void)
+{
+}
+
+void free_mem(void)
+{
+	lkl_ops->mem_free((void *)_memory_start);
+}
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v3 05/26] um lkl: memory handling
@ 2020-02-05  7:30   ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05  7:30 UTC (permalink / raw)
  To: linux-um
  Cc: linux-arch, Levente Kurusa, Octavian Purdila, Akira Moroo,
	Hajime Tazaki, linux-kernel-library, Yuan Liu

From: Octavian Purdila <tavi.purdila@gmail.com>

LKL is a non MMU architecture and hence there is not much work left to
do other than initializing the boot allocator and providing the page
and page table definitions.

The backstore memory is allocated via a host operation and the memory
size to be used is specified when the kernel is started, in the
lkl_start_kernel call.

Cc: H.K. Jerry Chu <hkchu@google.com>
Cc: Levente Kurusa <levex@linux.com>
Cc: Yuan Liu <liuyuan@google.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/lkl/include/asm/page.h          | 14 ++++++
 arch/um/lkl/include/asm/pgtable.h       | 57 +++++++++++++++++++++
 arch/um/lkl/include/uapi/asm/host_ops.h |  5 ++
 arch/um/lkl/mm/bootmem.c                | 66 +++++++++++++++++++++++++
 4 files changed, 142 insertions(+)
 create mode 100644 arch/um/lkl/include/asm/page.h
 create mode 100644 arch/um/lkl/include/asm/pgtable.h
 create mode 100644 arch/um/lkl/mm/bootmem.c

diff --git a/arch/um/lkl/include/asm/page.h b/arch/um/lkl/include/asm/page.h
new file mode 100644
index 000000000000..e77f3da22031
--- /dev/null
+++ b/arch/um/lkl/include/asm/page.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_PAGE_H
+#define _ASM_LKL_PAGE_H
+
+#define CONFIG_KERNEL_RAM_BASE_ADDRESS memory_start
+
+#ifndef __ASSEMBLY__
+void free_mem(void);
+void bootmem_init(unsigned long mem_size);
+#endif
+
+#include <asm-generic/page.h>
+
+#endif /* _ASM_LKL_PAGE_H */
diff --git a/arch/um/lkl/include/asm/pgtable.h b/arch/um/lkl/include/asm/pgtable.h
new file mode 100644
index 000000000000..733beb6d53f6
--- /dev/null
+++ b/arch/um/lkl/include/asm/pgtable.h
@@ -0,0 +1,57 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_PGTABLE_H
+#define _LKL_PGTABLE_H
+
+#include <asm-generic/pgtable-nopud.h>
+
+/*
+ * (C) Copyright 2000-2002, Greg Ungerer <gerg@snapgear.com>
+ */
+
+#include <linux/slab.h>
+#include <asm/processor.h>
+#include <asm/io.h>
+
+#define pgd_present(pgd)	(1)
+#define pgd_none(pgd)		(0)
+#define pgd_bad(pgd)		(0)
+#define pgd_clear(pgdp)
+#define kern_addr_valid(addr)	(1)
+#define	pmd_offset(a, b)	((void *)0)
+
+#define PAGE_NONE		__pgprot(0)
+#define PAGE_SHARED		__pgprot(0)
+#define PAGE_COPY		__pgprot(0)
+#define PAGE_READONLY		__pgprot(0)
+#define PAGE_KERNEL		__pgprot(0)
+
+void paging_init(void);
+#define swapper_pg_dir		((pgd_t *)0)
+
+#define __swp_type(x)		(0)
+#define __swp_offset(x)		(0)
+#define __swp_entry(typ, off)	((swp_entry_t) { ((typ) | ((off) << 7)) })
+#define __pte_to_swp_entry(pte)	((swp_entry_t) { pte_val(pte) })
+#define __swp_entry_to_pte(x)	((pte_t) { (x).val })
+
+/*
+ * ZERO_PAGE is a global shared page that is always zero: used
+ * for zero-mapped memory areas etc..
+ */
+extern void *empty_zero_page;
+#define ZERO_PAGE(vaddr)	(virt_to_page(empty_zero_page))
+
+/*
+ * All 32bit addresses are effectively valid for vmalloc...
+ * Sort of meaningless for non-VM targets.
+ */
+#define	VMALLOC_START		0
+#define	VMALLOC_END		0xffffffff
+#define	KMAP_START		0
+#define	KMAP_END		0xffffffff
+
+#include <asm-generic/pgtable.h>
+
+#define check_pgt_cache()	do { } while (0)
+
+#endif
diff --git a/arch/um/lkl/include/uapi/asm/host_ops.h b/arch/um/lkl/include/uapi/asm/host_ops.h
index 7cfb0a93e6a6..6bbc94c120be 100644
--- a/arch/um/lkl/include/uapi/asm/host_ops.h
+++ b/arch/um/lkl/include/uapi/asm/host_ops.h
@@ -17,8 +17,13 @@ struct lkl_jmp_buf {
  * These operations must be provided by a host library or by the application
  * itself.
  *
+ * @mem_alloc - allocate memory
+ * @mem_free - free memory
+ *
  */
 struct lkl_host_operations {
+	void *(*mem_alloc)(unsigned long mem);
+	void (*mem_free)(void *mem);
 };
 
 void lkl_bug(const char *fmt, ...);
diff --git a/arch/um/lkl/mm/bootmem.c b/arch/um/lkl/mm/bootmem.c
new file mode 100644
index 000000000000..39dd0d22b44e
--- /dev/null
+++ b/arch/um/lkl/mm/bootmem.c
@@ -0,0 +1,66 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/memblock.h>
+#include <linux/mm.h>
+#include <linux/swap.h>
+
+unsigned long memory_start, memory_end;
+static unsigned long _memory_start, mem_size;
+
+void *empty_zero_page;
+
+void __init bootmem_init(unsigned long mem_sz)
+{
+	mem_size = mem_sz;
+
+	_memory_start = (unsigned long)lkl_ops->mem_alloc(mem_size);
+	memory_start = _memory_start;
+	WARN_ON(!memory_start);
+	memory_end = memory_start + mem_size;
+
+	if (PAGE_ALIGN(memory_start) != memory_start) {
+		mem_size -= PAGE_ALIGN(memory_start) - memory_start;
+		memory_start = PAGE_ALIGN(memory_start);
+		mem_size = (mem_size / PAGE_SIZE) * PAGE_SIZE;
+	}
+	pr_info("memblock address range: 0x%lx - 0x%lx\n", memory_start,
+		memory_start + mem_size);
+	/*
+	 * Give all the memory to the bootmap allocator, tell it to put the
+	 * boot mem_map at the start of memory.
+	 */
+	max_low_pfn = virt_to_pfn(memory_end);
+	min_low_pfn = virt_to_pfn(memory_start);
+	memblock_add(memory_start, mem_size);
+
+	empty_zero_page = memblock_alloc(PAGE_SIZE, PAGE_SIZE);
+	memset((void *)empty_zero_page, 0, PAGE_SIZE);
+
+	{
+		unsigned long zones_size[MAX_NR_ZONES] = {0, };
+
+		zones_size[ZONE_NORMAL] = (mem_size) >> PAGE_SHIFT;
+		free_area_init(zones_size);
+	}
+}
+
+void __init mem_init(void)
+{
+	max_mapnr = (((unsigned long)high_memory) - PAGE_OFFSET) >> PAGE_SHIFT;
+	/* this will put all memory onto the freelists */
+	totalram_pages_add(memblock_free_all());
+	pr_info("Memory available: %luk/%luk RAM\n",
+		(nr_free_pages() << PAGE_SHIFT) >> 10, mem_size >> 10);
+}
+
+/*
+ * In our case __init memory is not part of the page allocator so there is
+ * nothing to free.
+ */
+void free_initmem(void)
+{
+}
+
+void free_mem(void)
+{
+	lkl_ops->mem_free((void *)_memory_start);
+}
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v3 06/26] um lkl: kernel threads support
  2020-02-05  7:30 [RFC v3 00/26] Unifying LKL into UML Hajime Tazaki
@ 2020-02-05  7:30   ` Hajime Tazaki
       [not found] ` <cover.1580882335.git.thehajime-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
                     ` (25 subsequent siblings)
  26 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05  7:30 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Lai Jiangshan, Patrick Collins, Yuan Liu, Hajime Tazaki

From: Octavian Purdila <tavi.purdila@gmail.com>

LKL does not support user processes but it must support kernel threads
as part as the normal kernel work-flow. It uses host operations to
create and terminate host threads that are going to run the kernel
threads. It also uses semaphores to synchronize those threads and to
allow the Linux kernel scheduler to control how the kernel threads
run.

Each kernel thread runs in a host threads and has a host semaphore
associated with it - the thread's scheduling semaphore. The semaphore
counter is initialized to 0. The first thing a kernel thread does
after getting spawned, before running any kernel code, is to perform a
down operation to block the thread.

The kernel controls host threads scheduling by performing up and down
operations on the scheduling semaphore. In __switch_context an up
operation on the next thread is performed to wake up a blocked thread,
and a down operation is performed on the prev thread to block it.

A thread is terminated by marking it in free_thread_info and
performing an up operation on the scheduling semaphore at which point
the marked thread will terminate itself.

Cc: Lai Jiangshan <jiangshanlai@gmail.com>
Cc: Patrick Collins <pscollins@google.com>
Cc: Yuan Liu <liuyuan@google.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/lkl/include/asm/thread_info.h   |  70 ++++++++
 arch/um/lkl/include/uapi/asm/host_ops.h |  55 ++++++
 arch/um/lkl/kernel/cpu.c                | 223 +++++++++++++++++++++++
 arch/um/lkl/kernel/threads.c            | 227 ++++++++++++++++++++++++
 4 files changed, 575 insertions(+)
 create mode 100644 arch/um/lkl/include/asm/thread_info.h
 create mode 100644 arch/um/lkl/kernel/cpu.c
 create mode 100644 arch/um/lkl/kernel/threads.c

diff --git a/arch/um/lkl/include/asm/thread_info.h b/arch/um/lkl/include/asm/thread_info.h
new file mode 100644
index 000000000000..da4e75fc7b10
--- /dev/null
+++ b/arch/um/lkl/include/asm/thread_info.h
@@ -0,0 +1,70 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_THREAD_INFO_H
+#define _ASM_LKL_THREAD_INFO_H
+
+#define THREAD_SIZE	       (4096)
+
+#ifndef __ASSEMBLY__
+#include <asm/types.h>
+#include <asm/processor.h>
+#include <asm/host_ops.h>
+
+typedef struct {
+	unsigned long seg;
+} mm_segment_t;
+
+struct thread_info {
+	struct task_struct *task;
+	unsigned long flags;
+	int preempt_count;
+	mm_segment_t addr_limit;
+	struct lkl_sem *sched_sem;
+	struct lkl_jmp_buf sched_jb;
+	bool dead;
+	lkl_thread_t tid;
+	struct task_struct *prev_sched;
+	unsigned long stackend;
+};
+
+#define INIT_THREAD_INFO(tsk)				\
+{							\
+	.task		= &tsk,				\
+	.preempt_count	= INIT_PREEMPT_COUNT,		\
+	.flags		= 0,				\
+	.addr_limit	= KERNEL_DS,			\
+}
+
+/* how to get the thread information struct from C */
+extern struct thread_info *_current_thread_info;
+static inline struct thread_info *current_thread_info(void)
+{
+	return _current_thread_info;
+}
+
+/* thread information allocation */
+unsigned long *alloc_thread_stack_node(struct task_struct *, int node);
+void free_thread_stack(struct task_struct *tsk);
+
+void threads_init(void);
+void threads_cleanup(void);
+
+#define TIF_SYSCALL_TRACE		0
+#define TIF_NOTIFY_RESUME		1
+#define TIF_SIGPENDING			2
+#define TIF_NEED_RESCHED		3
+#define TIF_RESTORE_SIGMASK		4
+#define TIF_MEMDIE			5
+#define TIF_NOHZ			6
+#define TIF_SCHED_JB			7
+#define TIF_HOST_THREAD			8
+
+#define __HAVE_THREAD_FUNCTIONS
+
+#define task_thread_info(task)	((struct thread_info *)(task)->stack)
+#define task_stack_page(task)	((task)->stack)
+void setup_thread_stack(struct task_struct *p, struct task_struct *org);
+#define end_of_stack(p) (&task_thread_info(p)->stackend)
+
+#endif /* __ASSEMBLY__ */
+
+#endif
diff --git a/arch/um/lkl/include/uapi/asm/host_ops.h b/arch/um/lkl/include/uapi/asm/host_ops.h
index 6bbc94c120be..19924fc7c718 100644
--- a/arch/um/lkl/include/uapi/asm/host_ops.h
+++ b/arch/um/lkl/include/uapi/asm/host_ops.h
@@ -17,15 +17,70 @@ struct lkl_jmp_buf {
  * These operations must be provided by a host library or by the application
  * itself.
  *
+ * @sem_alloc - allocate a host semaphore an initialize it to count
+ * @sem_free - free a host semaphore
+ * @sem_up - perform an up operation on the semaphore
+ * @sem_down - perform a down operation on the semaphore
+ *
+ * @mutex_alloc - allocate and initialize a host mutex; the recursive parameter
+ * determines if the mutex is recursive or not
+ * @mutex_free - free a host mutex
+ * @mutex_lock - acquire the mutex
+ * @mutex_unlock - release the mutex
+ *
+ * @thread_create - create a new thread and run f(arg) in its context; returns a
+ * thread handle or 0 if the thread could not be created
+ * @thread_detach - on POSIX systems, free up resources held by
+ * pthreads. Noop on Win32.
+ * @thread_exit - terminates the current thread
+ * @thread_join - wait for the given thread to terminate. Returns 0
+ * for success, -1 otherwise
+ *
  * @mem_alloc - allocate memory
  * @mem_free - free memory
  *
+ * @gettid - returns the host thread id of the caller, which need not
+ * be the same as the handle returned by thread_create
+ *
+ * @jmp_buf_set - runs the give function and setups a jump back point by saving
+ * the context in the jump buffer; jmp_buf_longjmp can be called from the give
+ * function or any callee in that function to return back to the jump back
+ * point
+ *
+ * NOTE: we can't return from jmp_buf_set before calling jmp_buf_longjmp or
+ * otherwise the saved context (stack) is not going to be valid, so we must pass
+ * the function that will eventually call longjmp here
+ *
+ * @jmp_buf_longjmp - perform a jump back to the saved jump buffer
  */
 struct lkl_host_operations {
+	struct lkl_sem *(*sem_alloc)(int count);
+	void (*sem_free)(struct lkl_sem *sem);
+	void (*sem_up)(struct lkl_sem *sem);
+	void (*sem_down)(struct lkl_sem *sem);
+
+	struct lkl_mutex *(*mutex_alloc)(int recursive);
+	void (*mutex_free)(struct lkl_mutex *mutex);
+	void (*mutex_lock)(struct lkl_mutex *mutex);
+	void (*mutex_unlock)(struct lkl_mutex *mutex);
+
+	lkl_thread_t (*thread_create)(void (*f)(void *), void *arg);
+	void (*thread_detach)(void);
+	void (*thread_exit)(void);
+	int (*thread_join)(lkl_thread_t tid);
+	lkl_thread_t (*thread_self)(void);
+	int (*thread_equal)(lkl_thread_t a, lkl_thread_t b);
+
 	void *(*mem_alloc)(unsigned long mem);
 	void (*mem_free)(void *mem);
+
+	long (*gettid)(void);
+
+	void (*jmp_buf_set)(struct lkl_jmp_buf *jmpb, void (*f)(void));
+	void (*jmp_buf_longjmp)(struct lkl_jmp_buf *jmpb, int val);
 };
 
+int lkl_printf(const char *fmt, ...);
 void lkl_bug(const char *fmt, ...);
 
 #endif
diff --git a/arch/um/lkl/kernel/cpu.c b/arch/um/lkl/kernel/cpu.c
new file mode 100644
index 000000000000..125af3b2d5dd
--- /dev/null
+++ b/arch/um/lkl/kernel/cpu.c
@@ -0,0 +1,223 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/kernel.h>
+#include <linux/sched/stat.h>
+#include <asm/host_ops.h>
+#include <asm/cpu.h>
+#include <asm/thread_info.h>
+#include <asm/unistd.h>
+#include <asm/sched.h>
+#include <asm/syscalls.h>
+
+/*
+ * This structure is used to get access to the "LKL CPU" that allows us to run
+ * Linux code. Because we have to deal with various synchronization requirements
+ * between idle thread, system calls, interrupts, "reentrancy", CPU shutdown,
+ * imbalance wake up (i.e. acquire the CPU from one thread and release it from
+ * another), we can't use a simple synchronization mechanism such as (recursive)
+ * mutex or semaphore. Instead, we use a mutex and a bunch of status data plus a
+ * semaphore.
+ */
+static struct lkl_cpu {
+	/* lock that protects the CPU status data */
+	struct lkl_mutex *lock;
+	/*
+	 * Since we must free the cpu lock during shutdown we need a
+	 * synchronization algorithm between lkl_cpu_shutdown() and the CPU
+	 * access functions since lkl_cpu_get() gets called from thread
+	 * destructor callback functions which may be scheduled after
+	 * lkl_cpu_shutdown() has freed the cpu lock.
+	 *
+	 * An atomic counter is used to keep track of the number of running
+	 * CPU access functions and allow the shutdown function to wait for
+	 * them.
+	 *
+	 * The shutdown functions adds MAX_THREADS to this counter which allows
+	 * the CPU access functions to check if the shutdown process has
+	 * started.
+	 *
+	 * This algorithm assumes that we never have more the MAX_THREADS
+	 * requesting CPU access.
+	 */
+	#define MAX_THREADS 1000000
+	unsigned int shutdown_gate;
+	bool irqs_pending;
+	/* no of threads waiting the CPU */
+	unsigned int sleepers;
+	/* no of times the current thread got the CPU */
+	unsigned int count;
+	/* current thread that owns the CPU */
+	lkl_thread_t owner;
+	/* semaphore for threads waiting the CPU */
+	struct lkl_sem *sem;
+	/* semaphore used for shutdown */
+	struct lkl_sem *shutdown_sem;
+} cpu;
+
+static int __cpu_try_get_lock(int n)
+{
+	lkl_thread_t self;
+
+	if (__sync_fetch_and_add(&cpu.shutdown_gate, n) >= MAX_THREADS)
+		return -2;
+
+	lkl_ops->mutex_lock(cpu.lock);
+
+	if (cpu.shutdown_gate >= MAX_THREADS)
+		return -1;
+
+	self = lkl_ops->thread_self();
+
+	if (cpu.owner && !lkl_ops->thread_equal(cpu.owner, self))
+		return 0;
+
+	cpu.owner = self;
+	cpu.count++;
+
+	return 1;
+}
+
+static void __cpu_try_get_unlock(int lock_ret, int n)
+{
+	if (lock_ret >= -1)
+		lkl_ops->mutex_unlock(cpu.lock);
+	__sync_fetch_and_sub(&cpu.shutdown_gate, n);
+}
+
+void lkl_cpu_change_owner(lkl_thread_t owner)
+{
+	lkl_ops->mutex_lock(cpu.lock);
+	if (cpu.count > 1)
+		lkl_bug("bad count while changing owner\n");
+	cpu.owner = owner;
+	lkl_ops->mutex_unlock(cpu.lock);
+}
+
+int lkl_cpu_get(void)
+{
+	int ret;
+
+	ret = __cpu_try_get_lock(1);
+
+	while (ret == 0) {
+		cpu.sleepers++;
+		__cpu_try_get_unlock(ret, 0);
+		lkl_ops->sem_down(cpu.sem);
+		ret = __cpu_try_get_lock(0);
+	}
+
+	__cpu_try_get_unlock(ret, 1);
+
+	return ret;
+}
+
+void lkl_cpu_put(void)
+{
+	lkl_ops->mutex_lock(cpu.lock);
+
+	if (!cpu.count || !cpu.owner ||
+	    !lkl_ops->thread_equal(cpu.owner, lkl_ops->thread_self()))
+		lkl_bug("%s: unbalanced put\n", __func__);
+
+	while (cpu.irqs_pending && !irqs_disabled()) {
+		cpu.irqs_pending = false;
+		lkl_ops->mutex_unlock(cpu.lock);
+		run_irqs();
+		lkl_ops->mutex_lock(cpu.lock);
+	}
+
+	if (test_ti_thread_flag(current_thread_info(), TIF_HOST_THREAD) &&
+	    !single_task_running() && cpu.count == 1) {
+		if (in_interrupt())
+			lkl_bug("%s: in interrupt\n", __func__);
+		lkl_ops->mutex_unlock(cpu.lock);
+		thread_sched_jb();
+		return;
+	}
+
+	if (--cpu.count > 0) {
+		lkl_ops->mutex_unlock(cpu.lock);
+		return;
+	}
+
+	if (cpu.sleepers) {
+		cpu.sleepers--;
+		lkl_ops->sem_up(cpu.sem);
+	}
+
+	cpu.owner = 0;
+
+	lkl_ops->mutex_unlock(cpu.lock);
+}
+
+int lkl_cpu_try_run_irq(int irq)
+{
+	int ret;
+
+	ret = __cpu_try_get_lock(1);
+	if (!ret) {
+		set_irq_pending(irq);
+		cpu.irqs_pending = true;
+	}
+	__cpu_try_get_unlock(ret, 1);
+
+	return ret;
+}
+
+void lkl_cpu_shutdown(void)
+{
+	__sync_fetch_and_add(&cpu.shutdown_gate, MAX_THREADS);
+}
+
+void lkl_cpu_wait_shutdown(void)
+{
+	lkl_ops->sem_down(cpu.shutdown_sem);
+	lkl_ops->sem_free(cpu.shutdown_sem);
+}
+
+static void lkl_cpu_cleanup(bool shutdown)
+{
+	while (__sync_fetch_and_add(&cpu.shutdown_gate, 0) > MAX_THREADS)
+		;
+
+	if (shutdown)
+		lkl_ops->sem_up(cpu.shutdown_sem);
+	else if (cpu.shutdown_sem)
+		lkl_ops->sem_free(cpu.shutdown_sem);
+	if (cpu.sem)
+		lkl_ops->sem_free(cpu.sem);
+	if (cpu.lock)
+		lkl_ops->mutex_free(cpu.lock);
+}
+
+void arch_cpu_idle(void)
+{
+	if (cpu.shutdown_gate >= MAX_THREADS) {
+		lkl_ops->mutex_lock(cpu.lock);
+		while (cpu.sleepers--)
+			lkl_ops->sem_up(cpu.sem);
+		lkl_ops->mutex_unlock(cpu.lock);
+
+		lkl_cpu_cleanup(true);
+
+		lkl_ops->thread_exit();
+	}
+	/* enable irqs now to allow direct irqs to run */
+	local_irq_enable();
+
+	/* switch to idle_host_task */
+	wakeup_idle_host_task();
+}
+
+int lkl_cpu_init(void)
+{
+	cpu.lock = lkl_ops->mutex_alloc(0);
+	cpu.sem = lkl_ops->sem_alloc(0);
+	cpu.shutdown_sem = lkl_ops->sem_alloc(0);
+
+	if (!cpu.lock || !cpu.sem || !cpu.shutdown_sem) {
+		lkl_cpu_cleanup(false);
+		return -ENOMEM;
+	}
+
+	return 0;
+}
diff --git a/arch/um/lkl/kernel/threads.c b/arch/um/lkl/kernel/threads.c
new file mode 100644
index 000000000000..4fe8c56ae5e0
--- /dev/null
+++ b/arch/um/lkl/kernel/threads.c
@@ -0,0 +1,227 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/module.h>
+#include <linux/slab.h>
+#include <linux/sched/task.h>
+#include <linux/sched/signal.h>
+#include <asm/host_ops.h>
+#include <asm/cpu.h>
+#include <asm/sched.h>
+
+static int init_ti(struct thread_info *ti)
+{
+	ti->sched_sem = lkl_ops->sem_alloc(0);
+	if (!ti->sched_sem)
+		return -ENOMEM;
+
+	ti->dead = false;
+	ti->prev_sched = NULL;
+	ti->tid = 0;
+
+	return 0;
+}
+
+unsigned long *alloc_thread_stack_node(struct task_struct *task, int node)
+{
+	struct thread_info *ti;
+
+	ti = kmalloc(sizeof(*ti), GFP_KERNEL);
+	if (!ti)
+		return NULL;
+
+	if (init_ti(ti)) {
+		kfree(ti);
+		return NULL;
+	}
+	ti->task = task;
+
+	return (unsigned long *)ti;
+}
+
+/*
+ * The only new tasks created are kernel threads that have a predefined starting
+ * point thus no stack copy is required.
+ */
+void setup_thread_stack(struct task_struct *p, struct task_struct *org)
+{
+	struct thread_info *ti = task_thread_info(p);
+	struct thread_info *org_ti = task_thread_info(org);
+
+	ti->flags = org_ti->flags;
+	ti->preempt_count = org_ti->preempt_count;
+	ti->addr_limit = org_ti->addr_limit;
+}
+
+static void kill_thread(struct thread_info *ti)
+{
+	if (!test_ti_thread_flag(ti, TIF_HOST_THREAD)) {
+		ti->dead = true;
+		lkl_ops->sem_up(ti->sched_sem);
+		lkl_ops->thread_join(ti->tid);
+	}
+	lkl_ops->sem_free(ti->sched_sem);
+}
+
+void free_thread_stack(struct task_struct *tsk)
+{
+	struct thread_info *ti = task_thread_info(tsk);
+
+	kill_thread(ti);
+	kfree(ti);
+}
+
+struct thread_info *_current_thread_info = &init_thread_union.thread_info;
+
+/*
+ * schedule() expects the return of this function to be the task that we
+ * switched away from. Returning prev is not going to work because we are
+ * actually going to return the previous taks that was scheduled before the
+ * task we are going to wake up, and not the current task, e.g.:
+ *
+ * swapper -> init: saved prev on swapper stack is swapper
+ * init -> ksoftirqd0: saved prev on init stack is init
+ * ksoftirqd0 -> swapper: returned prev is swapper
+ */
+static struct task_struct *abs_prev = &init_task;
+
+struct task_struct *__switch_to(struct task_struct *prev,
+				struct task_struct *next)
+{
+	struct thread_info *_prev = task_thread_info(prev);
+	struct thread_info *_next = task_thread_info(next);
+	unsigned long _prev_flags = _prev->flags;
+	struct lkl_jmp_buf _prev_jb;
+
+	_current_thread_info = task_thread_info(next);
+	_next->prev_sched = prev;
+	abs_prev = prev;
+
+	BUG_ON(!_next->tid);
+	lkl_cpu_change_owner(_next->tid);
+
+	if (test_bit(TIF_SCHED_JB, &_prev_flags)) {
+		/* Atomic. Must be done before wakeup next */
+		clear_ti_thread_flag(_prev, TIF_SCHED_JB);
+		_prev_jb = _prev->sched_jb;
+	}
+
+	lkl_ops->sem_up(_next->sched_sem);
+	if (test_bit(TIF_SCHED_JB, &_prev_flags))
+		lkl_ops->jmp_buf_longjmp(&_prev_jb, 1);
+	else
+		lkl_ops->sem_down(_prev->sched_sem);
+
+	if (_prev->dead)
+		lkl_ops->thread_exit();
+
+	return abs_prev;
+}
+
+int host_task_stub(void *unused)
+{
+	return 0;
+}
+
+void switch_to_host_task(struct task_struct *task)
+{
+	if (WARN_ON(!test_tsk_thread_flag(task, TIF_HOST_THREAD)))
+		return;
+
+	task_thread_info(task)->tid = lkl_ops->thread_self();
+
+	if (current == task)
+		return;
+
+	wake_up_process(task);
+	thread_sched_jb();
+	lkl_ops->sem_down(task_thread_info(task)->sched_sem);
+	schedule_tail(abs_prev);
+}
+
+struct thread_bootstrap_arg {
+	struct thread_info *ti;
+	int (*f)(void *arg);
+	void *arg;
+};
+
+static void thread_bootstrap(void *_tba)
+{
+	struct thread_bootstrap_arg *tba = (struct thread_bootstrap_arg *)_tba;
+	struct thread_info *ti = tba->ti;
+	int (*f)(void *) = tba->f;
+	void *arg = tba->arg;
+
+	lkl_ops->sem_down(ti->sched_sem);
+	kfree(tba);
+	if (ti->prev_sched)
+		schedule_tail(ti->prev_sched);
+
+	f(arg);
+	do_exit(0);
+}
+
+int copy_thread(unsigned long clone_flags, unsigned long esp,
+		unsigned long unused, struct task_struct *p)
+{
+	struct thread_info *ti = task_thread_info(p);
+	struct thread_bootstrap_arg *tba;
+
+	if ((int (*)(void *))esp == host_task_stub) {
+		set_ti_thread_flag(ti, TIF_HOST_THREAD);
+		return 0;
+	}
+
+	tba = kmalloc(sizeof(*tba), GFP_KERNEL);
+	if (!tba)
+		return -ENOMEM;
+
+	tba->f = (int (*)(void *))esp;
+	tba->arg = (void *)unused;
+	tba->ti = ti;
+
+	ti->tid = lkl_ops->thread_create(thread_bootstrap, tba);
+	if (!ti->tid) {
+		kfree(tba);
+		return -ENOMEM;
+	}
+
+	return 0;
+}
+
+void show_stack(struct task_struct *task, unsigned long *esp)
+{
+}
+
+/**
+ * This is called before the kernel initializes, so no kernel calls (including
+ * printk) can't be made yet.
+ */
+void threads_init(void)
+{
+	int ret;
+	struct thread_info *ti = &init_thread_union.thread_info;
+
+	ret = init_ti(ti);
+	if (ret < 0)
+		lkl_printf("lkl: failed to allocate init schedule semaphore\n");
+
+	ti->tid = lkl_ops->thread_self();
+}
+
+void threads_cleanup(void)
+{
+	struct task_struct *p, *t;
+
+	for_each_process_thread(p, t) {
+		struct thread_info *ti = task_thread_info(t);
+
+		if (t->pid != 1 && !test_ti_thread_flag(ti, TIF_HOST_THREAD))
+			WARN(!(t->flags & PF_KTHREAD),
+			     "non kernel thread task %s\n", t->comm);
+		WARN(t->state == TASK_RUNNING,
+		     "thread %s still running while halting\n", t->comm);
+
+		kill_thread(ti);
+	}
+
+	lkl_ops->sem_free(init_thread_union.thread_info.sched_sem);
+}
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v3 06/26] um lkl: kernel threads support
@ 2020-02-05  7:30   ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05  7:30 UTC (permalink / raw)
  To: linux-um
  Cc: linux-arch, Hajime Tazaki, Octavian Purdila, Lai Jiangshan,
	Akira Moroo, Patrick Collins, linux-kernel-library, Yuan Liu

From: Octavian Purdila <tavi.purdila@gmail.com>

LKL does not support user processes but it must support kernel threads
as part as the normal kernel work-flow. It uses host operations to
create and terminate host threads that are going to run the kernel
threads. It also uses semaphores to synchronize those threads and to
allow the Linux kernel scheduler to control how the kernel threads
run.

Each kernel thread runs in a host threads and has a host semaphore
associated with it - the thread's scheduling semaphore. The semaphore
counter is initialized to 0. The first thing a kernel thread does
after getting spawned, before running any kernel code, is to perform a
down operation to block the thread.

The kernel controls host threads scheduling by performing up and down
operations on the scheduling semaphore. In __switch_context an up
operation on the next thread is performed to wake up a blocked thread,
and a down operation is performed on the prev thread to block it.

A thread is terminated by marking it in free_thread_info and
performing an up operation on the scheduling semaphore at which point
the marked thread will terminate itself.

Cc: Lai Jiangshan <jiangshanlai@gmail.com>
Cc: Patrick Collins <pscollins@google.com>
Cc: Yuan Liu <liuyuan@google.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/lkl/include/asm/thread_info.h   |  70 ++++++++
 arch/um/lkl/include/uapi/asm/host_ops.h |  55 ++++++
 arch/um/lkl/kernel/cpu.c                | 223 +++++++++++++++++++++++
 arch/um/lkl/kernel/threads.c            | 227 ++++++++++++++++++++++++
 4 files changed, 575 insertions(+)
 create mode 100644 arch/um/lkl/include/asm/thread_info.h
 create mode 100644 arch/um/lkl/kernel/cpu.c
 create mode 100644 arch/um/lkl/kernel/threads.c

diff --git a/arch/um/lkl/include/asm/thread_info.h b/arch/um/lkl/include/asm/thread_info.h
new file mode 100644
index 000000000000..da4e75fc7b10
--- /dev/null
+++ b/arch/um/lkl/include/asm/thread_info.h
@@ -0,0 +1,70 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_THREAD_INFO_H
+#define _ASM_LKL_THREAD_INFO_H
+
+#define THREAD_SIZE	       (4096)
+
+#ifndef __ASSEMBLY__
+#include <asm/types.h>
+#include <asm/processor.h>
+#include <asm/host_ops.h>
+
+typedef struct {
+	unsigned long seg;
+} mm_segment_t;
+
+struct thread_info {
+	struct task_struct *task;
+	unsigned long flags;
+	int preempt_count;
+	mm_segment_t addr_limit;
+	struct lkl_sem *sched_sem;
+	struct lkl_jmp_buf sched_jb;
+	bool dead;
+	lkl_thread_t tid;
+	struct task_struct *prev_sched;
+	unsigned long stackend;
+};
+
+#define INIT_THREAD_INFO(tsk)				\
+{							\
+	.task		= &tsk,				\
+	.preempt_count	= INIT_PREEMPT_COUNT,		\
+	.flags		= 0,				\
+	.addr_limit	= KERNEL_DS,			\
+}
+
+/* how to get the thread information struct from C */
+extern struct thread_info *_current_thread_info;
+static inline struct thread_info *current_thread_info(void)
+{
+	return _current_thread_info;
+}
+
+/* thread information allocation */
+unsigned long *alloc_thread_stack_node(struct task_struct *, int node);
+void free_thread_stack(struct task_struct *tsk);
+
+void threads_init(void);
+void threads_cleanup(void);
+
+#define TIF_SYSCALL_TRACE		0
+#define TIF_NOTIFY_RESUME		1
+#define TIF_SIGPENDING			2
+#define TIF_NEED_RESCHED		3
+#define TIF_RESTORE_SIGMASK		4
+#define TIF_MEMDIE			5
+#define TIF_NOHZ			6
+#define TIF_SCHED_JB			7
+#define TIF_HOST_THREAD			8
+
+#define __HAVE_THREAD_FUNCTIONS
+
+#define task_thread_info(task)	((struct thread_info *)(task)->stack)
+#define task_stack_page(task)	((task)->stack)
+void setup_thread_stack(struct task_struct *p, struct task_struct *org);
+#define end_of_stack(p) (&task_thread_info(p)->stackend)
+
+#endif /* __ASSEMBLY__ */
+
+#endif
diff --git a/arch/um/lkl/include/uapi/asm/host_ops.h b/arch/um/lkl/include/uapi/asm/host_ops.h
index 6bbc94c120be..19924fc7c718 100644
--- a/arch/um/lkl/include/uapi/asm/host_ops.h
+++ b/arch/um/lkl/include/uapi/asm/host_ops.h
@@ -17,15 +17,70 @@ struct lkl_jmp_buf {
  * These operations must be provided by a host library or by the application
  * itself.
  *
+ * @sem_alloc - allocate a host semaphore an initialize it to count
+ * @sem_free - free a host semaphore
+ * @sem_up - perform an up operation on the semaphore
+ * @sem_down - perform a down operation on the semaphore
+ *
+ * @mutex_alloc - allocate and initialize a host mutex; the recursive parameter
+ * determines if the mutex is recursive or not
+ * @mutex_free - free a host mutex
+ * @mutex_lock - acquire the mutex
+ * @mutex_unlock - release the mutex
+ *
+ * @thread_create - create a new thread and run f(arg) in its context; returns a
+ * thread handle or 0 if the thread could not be created
+ * @thread_detach - on POSIX systems, free up resources held by
+ * pthreads. Noop on Win32.
+ * @thread_exit - terminates the current thread
+ * @thread_join - wait for the given thread to terminate. Returns 0
+ * for success, -1 otherwise
+ *
  * @mem_alloc - allocate memory
  * @mem_free - free memory
  *
+ * @gettid - returns the host thread id of the caller, which need not
+ * be the same as the handle returned by thread_create
+ *
+ * @jmp_buf_set - runs the give function and setups a jump back point by saving
+ * the context in the jump buffer; jmp_buf_longjmp can be called from the give
+ * function or any callee in that function to return back to the jump back
+ * point
+ *
+ * NOTE: we can't return from jmp_buf_set before calling jmp_buf_longjmp or
+ * otherwise the saved context (stack) is not going to be valid, so we must pass
+ * the function that will eventually call longjmp here
+ *
+ * @jmp_buf_longjmp - perform a jump back to the saved jump buffer
  */
 struct lkl_host_operations {
+	struct lkl_sem *(*sem_alloc)(int count);
+	void (*sem_free)(struct lkl_sem *sem);
+	void (*sem_up)(struct lkl_sem *sem);
+	void (*sem_down)(struct lkl_sem *sem);
+
+	struct lkl_mutex *(*mutex_alloc)(int recursive);
+	void (*mutex_free)(struct lkl_mutex *mutex);
+	void (*mutex_lock)(struct lkl_mutex *mutex);
+	void (*mutex_unlock)(struct lkl_mutex *mutex);
+
+	lkl_thread_t (*thread_create)(void (*f)(void *), void *arg);
+	void (*thread_detach)(void);
+	void (*thread_exit)(void);
+	int (*thread_join)(lkl_thread_t tid);
+	lkl_thread_t (*thread_self)(void);
+	int (*thread_equal)(lkl_thread_t a, lkl_thread_t b);
+
 	void *(*mem_alloc)(unsigned long mem);
 	void (*mem_free)(void *mem);
+
+	long (*gettid)(void);
+
+	void (*jmp_buf_set)(struct lkl_jmp_buf *jmpb, void (*f)(void));
+	void (*jmp_buf_longjmp)(struct lkl_jmp_buf *jmpb, int val);
 };
 
+int lkl_printf(const char *fmt, ...);
 void lkl_bug(const char *fmt, ...);
 
 #endif
diff --git a/arch/um/lkl/kernel/cpu.c b/arch/um/lkl/kernel/cpu.c
new file mode 100644
index 000000000000..125af3b2d5dd
--- /dev/null
+++ b/arch/um/lkl/kernel/cpu.c
@@ -0,0 +1,223 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/kernel.h>
+#include <linux/sched/stat.h>
+#include <asm/host_ops.h>
+#include <asm/cpu.h>
+#include <asm/thread_info.h>
+#include <asm/unistd.h>
+#include <asm/sched.h>
+#include <asm/syscalls.h>
+
+/*
+ * This structure is used to get access to the "LKL CPU" that allows us to run
+ * Linux code. Because we have to deal with various synchronization requirements
+ * between idle thread, system calls, interrupts, "reentrancy", CPU shutdown,
+ * imbalance wake up (i.e. acquire the CPU from one thread and release it from
+ * another), we can't use a simple synchronization mechanism such as (recursive)
+ * mutex or semaphore. Instead, we use a mutex and a bunch of status data plus a
+ * semaphore.
+ */
+static struct lkl_cpu {
+	/* lock that protects the CPU status data */
+	struct lkl_mutex *lock;
+	/*
+	 * Since we must free the cpu lock during shutdown we need a
+	 * synchronization algorithm between lkl_cpu_shutdown() and the CPU
+	 * access functions since lkl_cpu_get() gets called from thread
+	 * destructor callback functions which may be scheduled after
+	 * lkl_cpu_shutdown() has freed the cpu lock.
+	 *
+	 * An atomic counter is used to keep track of the number of running
+	 * CPU access functions and allow the shutdown function to wait for
+	 * them.
+	 *
+	 * The shutdown functions adds MAX_THREADS to this counter which allows
+	 * the CPU access functions to check if the shutdown process has
+	 * started.
+	 *
+	 * This algorithm assumes that we never have more the MAX_THREADS
+	 * requesting CPU access.
+	 */
+	#define MAX_THREADS 1000000
+	unsigned int shutdown_gate;
+	bool irqs_pending;
+	/* no of threads waiting the CPU */
+	unsigned int sleepers;
+	/* no of times the current thread got the CPU */
+	unsigned int count;
+	/* current thread that owns the CPU */
+	lkl_thread_t owner;
+	/* semaphore for threads waiting the CPU */
+	struct lkl_sem *sem;
+	/* semaphore used for shutdown */
+	struct lkl_sem *shutdown_sem;
+} cpu;
+
+static int __cpu_try_get_lock(int n)
+{
+	lkl_thread_t self;
+
+	if (__sync_fetch_and_add(&cpu.shutdown_gate, n) >= MAX_THREADS)
+		return -2;
+
+	lkl_ops->mutex_lock(cpu.lock);
+
+	if (cpu.shutdown_gate >= MAX_THREADS)
+		return -1;
+
+	self = lkl_ops->thread_self();
+
+	if (cpu.owner && !lkl_ops->thread_equal(cpu.owner, self))
+		return 0;
+
+	cpu.owner = self;
+	cpu.count++;
+
+	return 1;
+}
+
+static void __cpu_try_get_unlock(int lock_ret, int n)
+{
+	if (lock_ret >= -1)
+		lkl_ops->mutex_unlock(cpu.lock);
+	__sync_fetch_and_sub(&cpu.shutdown_gate, n);
+}
+
+void lkl_cpu_change_owner(lkl_thread_t owner)
+{
+	lkl_ops->mutex_lock(cpu.lock);
+	if (cpu.count > 1)
+		lkl_bug("bad count while changing owner\n");
+	cpu.owner = owner;
+	lkl_ops->mutex_unlock(cpu.lock);
+}
+
+int lkl_cpu_get(void)
+{
+	int ret;
+
+	ret = __cpu_try_get_lock(1);
+
+	while (ret == 0) {
+		cpu.sleepers++;
+		__cpu_try_get_unlock(ret, 0);
+		lkl_ops->sem_down(cpu.sem);
+		ret = __cpu_try_get_lock(0);
+	}
+
+	__cpu_try_get_unlock(ret, 1);
+
+	return ret;
+}
+
+void lkl_cpu_put(void)
+{
+	lkl_ops->mutex_lock(cpu.lock);
+
+	if (!cpu.count || !cpu.owner ||
+	    !lkl_ops->thread_equal(cpu.owner, lkl_ops->thread_self()))
+		lkl_bug("%s: unbalanced put\n", __func__);
+
+	while (cpu.irqs_pending && !irqs_disabled()) {
+		cpu.irqs_pending = false;
+		lkl_ops->mutex_unlock(cpu.lock);
+		run_irqs();
+		lkl_ops->mutex_lock(cpu.lock);
+	}
+
+	if (test_ti_thread_flag(current_thread_info(), TIF_HOST_THREAD) &&
+	    !single_task_running() && cpu.count == 1) {
+		if (in_interrupt())
+			lkl_bug("%s: in interrupt\n", __func__);
+		lkl_ops->mutex_unlock(cpu.lock);
+		thread_sched_jb();
+		return;
+	}
+
+	if (--cpu.count > 0) {
+		lkl_ops->mutex_unlock(cpu.lock);
+		return;
+	}
+
+	if (cpu.sleepers) {
+		cpu.sleepers--;
+		lkl_ops->sem_up(cpu.sem);
+	}
+
+	cpu.owner = 0;
+
+	lkl_ops->mutex_unlock(cpu.lock);
+}
+
+int lkl_cpu_try_run_irq(int irq)
+{
+	int ret;
+
+	ret = __cpu_try_get_lock(1);
+	if (!ret) {
+		set_irq_pending(irq);
+		cpu.irqs_pending = true;
+	}
+	__cpu_try_get_unlock(ret, 1);
+
+	return ret;
+}
+
+void lkl_cpu_shutdown(void)
+{
+	__sync_fetch_and_add(&cpu.shutdown_gate, MAX_THREADS);
+}
+
+void lkl_cpu_wait_shutdown(void)
+{
+	lkl_ops->sem_down(cpu.shutdown_sem);
+	lkl_ops->sem_free(cpu.shutdown_sem);
+}
+
+static void lkl_cpu_cleanup(bool shutdown)
+{
+	while (__sync_fetch_and_add(&cpu.shutdown_gate, 0) > MAX_THREADS)
+		;
+
+	if (shutdown)
+		lkl_ops->sem_up(cpu.shutdown_sem);
+	else if (cpu.shutdown_sem)
+		lkl_ops->sem_free(cpu.shutdown_sem);
+	if (cpu.sem)
+		lkl_ops->sem_free(cpu.sem);
+	if (cpu.lock)
+		lkl_ops->mutex_free(cpu.lock);
+}
+
+void arch_cpu_idle(void)
+{
+	if (cpu.shutdown_gate >= MAX_THREADS) {
+		lkl_ops->mutex_lock(cpu.lock);
+		while (cpu.sleepers--)
+			lkl_ops->sem_up(cpu.sem);
+		lkl_ops->mutex_unlock(cpu.lock);
+
+		lkl_cpu_cleanup(true);
+
+		lkl_ops->thread_exit();
+	}
+	/* enable irqs now to allow direct irqs to run */
+	local_irq_enable();
+
+	/* switch to idle_host_task */
+	wakeup_idle_host_task();
+}
+
+int lkl_cpu_init(void)
+{
+	cpu.lock = lkl_ops->mutex_alloc(0);
+	cpu.sem = lkl_ops->sem_alloc(0);
+	cpu.shutdown_sem = lkl_ops->sem_alloc(0);
+
+	if (!cpu.lock || !cpu.sem || !cpu.shutdown_sem) {
+		lkl_cpu_cleanup(false);
+		return -ENOMEM;
+	}
+
+	return 0;
+}
diff --git a/arch/um/lkl/kernel/threads.c b/arch/um/lkl/kernel/threads.c
new file mode 100644
index 000000000000..4fe8c56ae5e0
--- /dev/null
+++ b/arch/um/lkl/kernel/threads.c
@@ -0,0 +1,227 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/module.h>
+#include <linux/slab.h>
+#include <linux/sched/task.h>
+#include <linux/sched/signal.h>
+#include <asm/host_ops.h>
+#include <asm/cpu.h>
+#include <asm/sched.h>
+
+static int init_ti(struct thread_info *ti)
+{
+	ti->sched_sem = lkl_ops->sem_alloc(0);
+	if (!ti->sched_sem)
+		return -ENOMEM;
+
+	ti->dead = false;
+	ti->prev_sched = NULL;
+	ti->tid = 0;
+
+	return 0;
+}
+
+unsigned long *alloc_thread_stack_node(struct task_struct *task, int node)
+{
+	struct thread_info *ti;
+
+	ti = kmalloc(sizeof(*ti), GFP_KERNEL);
+	if (!ti)
+		return NULL;
+
+	if (init_ti(ti)) {
+		kfree(ti);
+		return NULL;
+	}
+	ti->task = task;
+
+	return (unsigned long *)ti;
+}
+
+/*
+ * The only new tasks created are kernel threads that have a predefined starting
+ * point thus no stack copy is required.
+ */
+void setup_thread_stack(struct task_struct *p, struct task_struct *org)
+{
+	struct thread_info *ti = task_thread_info(p);
+	struct thread_info *org_ti = task_thread_info(org);
+
+	ti->flags = org_ti->flags;
+	ti->preempt_count = org_ti->preempt_count;
+	ti->addr_limit = org_ti->addr_limit;
+}
+
+static void kill_thread(struct thread_info *ti)
+{
+	if (!test_ti_thread_flag(ti, TIF_HOST_THREAD)) {
+		ti->dead = true;
+		lkl_ops->sem_up(ti->sched_sem);
+		lkl_ops->thread_join(ti->tid);
+	}
+	lkl_ops->sem_free(ti->sched_sem);
+}
+
+void free_thread_stack(struct task_struct *tsk)
+{
+	struct thread_info *ti = task_thread_info(tsk);
+
+	kill_thread(ti);
+	kfree(ti);
+}
+
+struct thread_info *_current_thread_info = &init_thread_union.thread_info;
+
+/*
+ * schedule() expects the return of this function to be the task that we
+ * switched away from. Returning prev is not going to work because we are
+ * actually going to return the previous taks that was scheduled before the
+ * task we are going to wake up, and not the current task, e.g.:
+ *
+ * swapper -> init: saved prev on swapper stack is swapper
+ * init -> ksoftirqd0: saved prev on init stack is init
+ * ksoftirqd0 -> swapper: returned prev is swapper
+ */
+static struct task_struct *abs_prev = &init_task;
+
+struct task_struct *__switch_to(struct task_struct *prev,
+				struct task_struct *next)
+{
+	struct thread_info *_prev = task_thread_info(prev);
+	struct thread_info *_next = task_thread_info(next);
+	unsigned long _prev_flags = _prev->flags;
+	struct lkl_jmp_buf _prev_jb;
+
+	_current_thread_info = task_thread_info(next);
+	_next->prev_sched = prev;
+	abs_prev = prev;
+
+	BUG_ON(!_next->tid);
+	lkl_cpu_change_owner(_next->tid);
+
+	if (test_bit(TIF_SCHED_JB, &_prev_flags)) {
+		/* Atomic. Must be done before wakeup next */
+		clear_ti_thread_flag(_prev, TIF_SCHED_JB);
+		_prev_jb = _prev->sched_jb;
+	}
+
+	lkl_ops->sem_up(_next->sched_sem);
+	if (test_bit(TIF_SCHED_JB, &_prev_flags))
+		lkl_ops->jmp_buf_longjmp(&_prev_jb, 1);
+	else
+		lkl_ops->sem_down(_prev->sched_sem);
+
+	if (_prev->dead)
+		lkl_ops->thread_exit();
+
+	return abs_prev;
+}
+
+int host_task_stub(void *unused)
+{
+	return 0;
+}
+
+void switch_to_host_task(struct task_struct *task)
+{
+	if (WARN_ON(!test_tsk_thread_flag(task, TIF_HOST_THREAD)))
+		return;
+
+	task_thread_info(task)->tid = lkl_ops->thread_self();
+
+	if (current == task)
+		return;
+
+	wake_up_process(task);
+	thread_sched_jb();
+	lkl_ops->sem_down(task_thread_info(task)->sched_sem);
+	schedule_tail(abs_prev);
+}
+
+struct thread_bootstrap_arg {
+	struct thread_info *ti;
+	int (*f)(void *arg);
+	void *arg;
+};
+
+static void thread_bootstrap(void *_tba)
+{
+	struct thread_bootstrap_arg *tba = (struct thread_bootstrap_arg *)_tba;
+	struct thread_info *ti = tba->ti;
+	int (*f)(void *) = tba->f;
+	void *arg = tba->arg;
+
+	lkl_ops->sem_down(ti->sched_sem);
+	kfree(tba);
+	if (ti->prev_sched)
+		schedule_tail(ti->prev_sched);
+
+	f(arg);
+	do_exit(0);
+}
+
+int copy_thread(unsigned long clone_flags, unsigned long esp,
+		unsigned long unused, struct task_struct *p)
+{
+	struct thread_info *ti = task_thread_info(p);
+	struct thread_bootstrap_arg *tba;
+
+	if ((int (*)(void *))esp == host_task_stub) {
+		set_ti_thread_flag(ti, TIF_HOST_THREAD);
+		return 0;
+	}
+
+	tba = kmalloc(sizeof(*tba), GFP_KERNEL);
+	if (!tba)
+		return -ENOMEM;
+
+	tba->f = (int (*)(void *))esp;
+	tba->arg = (void *)unused;
+	tba->ti = ti;
+
+	ti->tid = lkl_ops->thread_create(thread_bootstrap, tba);
+	if (!ti->tid) {
+		kfree(tba);
+		return -ENOMEM;
+	}
+
+	return 0;
+}
+
+void show_stack(struct task_struct *task, unsigned long *esp)
+{
+}
+
+/**
+ * This is called before the kernel initializes, so no kernel calls (including
+ * printk) can't be made yet.
+ */
+void threads_init(void)
+{
+	int ret;
+	struct thread_info *ti = &init_thread_union.thread_info;
+
+	ret = init_ti(ti);
+	if (ret < 0)
+		lkl_printf("lkl: failed to allocate init schedule semaphore\n");
+
+	ti->tid = lkl_ops->thread_self();
+}
+
+void threads_cleanup(void)
+{
+	struct task_struct *p, *t;
+
+	for_each_process_thread(p, t) {
+		struct thread_info *ti = task_thread_info(t);
+
+		if (t->pid != 1 && !test_ti_thread_flag(ti, TIF_HOST_THREAD))
+			WARN(!(t->flags & PF_KTHREAD),
+			     "non kernel thread task %s\n", t->comm);
+		WARN(t->state == TASK_RUNNING,
+		     "thread %s still running while halting\n", t->comm);
+
+		kill_thread(ti);
+	}
+
+	lkl_ops->sem_free(init_thread_union.thread_info.sched_sem);
+}
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v3 07/26] um lkl: interrupt support
  2020-02-05  7:30 [RFC v3 00/26] Unifying LKL into UML Hajime Tazaki
@ 2020-02-05  7:30   ` Hajime Tazaki
       [not found] ` <cover.1580882335.git.thehajime-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
                     ` (25 subsequent siblings)
  26 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05  7:30 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Michael Zimmermann, Hajime Tazaki

From: Octavian Purdila <tavi.purdila@gmail.com>

Add APIs that allows the host to reserve and free and interrupt number
and also to trigger an interrupt.

The trigger operation will simply store the interrupt data in
queue. The interrupt handler is run later, at the first opportunity it
has to avoid races with any kernel threads.

Currently, interrupts are run on the first interrupt enable operation
if interrupts are disabled and if we are not already in interrupt
context.

When triggering an interrupt, it uses GCC's built-in functions for
atomic memory access to synchronize and simple boolean flags.

Cc: Michael Zimmermann <sigmaepsilon92@gmail.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/lkl/include/asm/irq.h             |  13 ++
 arch/um/lkl/include/uapi/asm/irq.h        |  36 ++++
 arch/um/lkl/include/uapi/asm/sigcontext.h |  16 ++
 arch/um/lkl/kernel/irq.c                  | 193 ++++++++++++++++++++++
 4 files changed, 258 insertions(+)
 create mode 100644 arch/um/lkl/include/asm/irq.h
 create mode 100644 arch/um/lkl/include/uapi/asm/irq.h
 create mode 100644 arch/um/lkl/include/uapi/asm/sigcontext.h
 create mode 100644 arch/um/lkl/kernel/irq.c

diff --git a/arch/um/lkl/include/asm/irq.h b/arch/um/lkl/include/asm/irq.h
new file mode 100644
index 000000000000..948fc54cb76c
--- /dev/null
+++ b/arch/um/lkl/include/asm/irq.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_IRQ_H
+#define _ASM_LKL_IRQ_H
+
+#define IRQ_STATUS_BITS		(sizeof(long) * 8)
+#define NR_IRQS			((int)(IRQ_STATUS_BITS * IRQ_STATUS_BITS))
+
+void run_irqs(void);
+void set_irq_pending(int irq);
+
+#include <uapi/asm/irq.h>
+
+#endif
diff --git a/arch/um/lkl/include/uapi/asm/irq.h b/arch/um/lkl/include/uapi/asm/irq.h
new file mode 100644
index 000000000000..666628b233eb
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/irq.h
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _ASM_UAPI_LKL_IRQ_H
+#define _ASM_UAPI_LKL_IRQ_H
+
+/**
+ * lkl_trigger_irq - generate an interrupt
+ *
+ * This function is used by the device host side to signal its Linux counterpart
+ * that some event happened.
+ *
+ * @irq - the irq number to signal
+ */
+int lkl_trigger_irq(int irq);
+
+/**
+ * lkl_get_free_irq - find and reserve a free IRQ number
+ *
+ * This function is called by the host device code to find an unused IRQ number
+ * and reserved it for its own use.
+ *
+ * @user - a string to identify the user
+ * @returns - and irq number that can be used by request_irq or an negative
+ * value in case of an error
+ */
+int lkl_get_free_irq(const char *user);
+
+/**
+ * lkl_put_irq - release an IRQ number previously obtained with lkl_get_free_irq
+ *
+ * @irq - irq number to release
+ * @user - string identifying the user; should be the same as the one passed to
+ * lkl_get_free_irq when the irq number was obtained
+ */
+void lkl_put_irq(int irq, const char *name);
+
+#endif
diff --git a/arch/um/lkl/include/uapi/asm/sigcontext.h b/arch/um/lkl/include/uapi/asm/sigcontext.h
new file mode 100644
index 000000000000..2f4848843d1d
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/sigcontext.h
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _ASM_UAPI_LKL_SIGCONTEXT_H
+#define _ASM_UAPI_LKL_SIGCONTEXT_H
+
+#include <asm/ptrace.h>
+
+struct pt_regs {
+	void *irq_data;
+};
+
+struct sigcontext {
+	struct pt_regs regs;
+	unsigned long oldmask;
+};
+
+#endif
diff --git a/arch/um/lkl/kernel/irq.c b/arch/um/lkl/kernel/irq.c
new file mode 100644
index 000000000000..e3b59e46ca50
--- /dev/null
+++ b/arch/um/lkl/kernel/irq.c
@@ -0,0 +1,193 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/kernel.h>
+#include <linux/init.h>
+#include <linux/irq.h>
+#include <linux/hardirq.h>
+#include <asm/irq_regs.h>
+#include <linux/sched.h>
+#include <linux/seq_file.h>
+#include <linux/tick.h>
+#include <asm/irqflags.h>
+#include <asm/host_ops.h>
+#include <asm/cpu.h>
+
+/*
+ * To avoid much overhead we use an indirect approach: the irqs are marked using
+ * a bitmap (array of longs) and a summary of the modified bits is kept in a
+ * separate "index" long - one bit for each sizeof(long). Thus we can support
+ * 4096 irqs on 64bit platforms and 1024 irqs on 32bit platforms.
+ *
+ * Whenever an irq is trigger both the array and the index is updated. To find
+ * which irqs were triggered we first search the index and then the
+ * corresponding part of the arrary.
+ */
+static unsigned long irq_status[NR_IRQS / IRQ_STATUS_BITS];
+static unsigned long irq_index_status;
+
+static inline unsigned long test_and_clear_irq_index_status(void)
+{
+	if (!irq_index_status)
+		return 0;
+	return __sync_fetch_and_and(&irq_index_status, 0);
+}
+
+static inline unsigned long test_and_clear_irq_status(int index)
+{
+	if (!&irq_status[index])
+		return 0;
+	return __sync_fetch_and_and(&irq_status[index], 0);
+}
+
+void set_irq_pending(int irq)
+{
+	int index = irq / IRQ_STATUS_BITS;
+	int bit = irq % IRQ_STATUS_BITS;
+
+	__sync_fetch_and_or(&irq_status[index], BIT(bit));
+	__sync_fetch_and_or(&irq_index_status, BIT(index));
+}
+
+static struct irq_info {
+	const char *user;
+} irqs[NR_IRQS];
+
+static bool irqs_enabled;
+
+static struct pt_regs dummy;
+
+static void run_irq(int irq)
+{
+	unsigned long flags;
+	struct pt_regs *old_regs = set_irq_regs((struct pt_regs *)&dummy);
+
+	/* interrupt handlers need to run with interrupts disabled */
+	local_irq_save(flags);
+	irq_enter();
+	generic_handle_irq(irq);
+	irq_exit();
+	set_irq_regs(old_regs);
+	local_irq_restore(flags);
+}
+
+/**
+ * This function can be called from arbitrary host threads, so do not
+ * issue any Linux calls (e.g. prink) if lkl_cpu_get() was not issued
+ * before.
+ */
+int lkl_trigger_irq(int irq)
+{
+	int ret;
+
+	if (!irq || irq > NR_IRQS)
+		return -EINVAL;
+
+	ret = lkl_cpu_try_run_irq(irq);
+	if (ret <= 0)
+		return ret;
+
+	/*
+	 * Since this can be called from Linux context (e.g. lkl_trigger_irq ->
+	 * IRQ -> softirq -> lkl_trigger_irq) make sure we are actually allowed
+	 * to run irqs at this point
+	 */
+	if (!irqs_enabled) {
+		set_irq_pending(irq);
+		lkl_cpu_put();
+		return 0;
+	}
+
+	run_irq(irq);
+
+	lkl_cpu_put();
+
+	return 0;
+}
+
+static inline void for_each_bit(unsigned long word, void (*f)(int, int), int j)
+{
+	int i = 0;
+
+	while (word) {
+		if (word & 1)
+			f(i, j);
+		word >>= 1;
+		i++;
+	}
+}
+
+static inline void deliver_irq(int bit, int index)
+{
+	run_irq(index * IRQ_STATUS_BITS + bit);
+}
+
+static inline void check_irq_status(int i, int unused)
+{
+	for_each_bit(test_and_clear_irq_status(i), deliver_irq, i);
+}
+
+void run_irqs(void)
+{
+	for_each_bit(test_and_clear_irq_index_status(), check_irq_status, 0);
+}
+
+int show_interrupts(struct seq_file *p, void *v)
+{
+	return 0;
+}
+
+int lkl_get_free_irq(const char *user)
+{
+	int i;
+	int ret = -EBUSY;
+
+	/* 0 is not a valid IRQ */
+	for (i = 1; i < NR_IRQS; i++) {
+		if (!irqs[i].user) {
+			irqs[i].user = user;
+			irq_set_chip_and_handler(i, &dummy_irq_chip,
+						 handle_simple_irq);
+			ret = i;
+			break;
+		}
+	}
+
+	return ret;
+}
+
+void lkl_put_irq(int i, const char *user)
+{
+	if (!irqs[i].user || strcmp(irqs[i].user, user) != 0) {
+		WARN("%s tried to release %s's irq %d", user, irqs[i].user, i);
+		return;
+	}
+
+	irqs[i].user = NULL;
+}
+
+unsigned long arch_local_save_flags(void)
+{
+	return irqs_enabled;
+}
+
+void arch_local_irq_restore(unsigned long flags)
+{
+	if (flags == ARCH_IRQ_ENABLED && irqs_enabled == ARCH_IRQ_DISABLED &&
+	    !in_interrupt())
+		run_irqs();
+	irqs_enabled = flags;
+}
+
+void init_IRQ(void)
+{
+	int i;
+
+	for (i = 0; i < NR_IRQS; i++)
+		irq_set_chip_and_handler(i, &dummy_irq_chip, handle_simple_irq);
+
+	pr_info("lkl: irqs initialized\n");
+}
+
+void cpu_yield_to_irqs(void)
+{
+	cpu_relax();
+}
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v3 07/26] um lkl: interrupt support
@ 2020-02-05  7:30   ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05  7:30 UTC (permalink / raw)
  To: linux-um
  Cc: linux-arch, Octavian Purdila, Akira Moroo, linux-kernel-library,
	Michael Zimmermann, Hajime Tazaki

From: Octavian Purdila <tavi.purdila@gmail.com>

Add APIs that allows the host to reserve and free and interrupt number
and also to trigger an interrupt.

The trigger operation will simply store the interrupt data in
queue. The interrupt handler is run later, at the first opportunity it
has to avoid races with any kernel threads.

Currently, interrupts are run on the first interrupt enable operation
if interrupts are disabled and if we are not already in interrupt
context.

When triggering an interrupt, it uses GCC's built-in functions for
atomic memory access to synchronize and simple boolean flags.

Cc: Michael Zimmermann <sigmaepsilon92@gmail.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/lkl/include/asm/irq.h             |  13 ++
 arch/um/lkl/include/uapi/asm/irq.h        |  36 ++++
 arch/um/lkl/include/uapi/asm/sigcontext.h |  16 ++
 arch/um/lkl/kernel/irq.c                  | 193 ++++++++++++++++++++++
 4 files changed, 258 insertions(+)
 create mode 100644 arch/um/lkl/include/asm/irq.h
 create mode 100644 arch/um/lkl/include/uapi/asm/irq.h
 create mode 100644 arch/um/lkl/include/uapi/asm/sigcontext.h
 create mode 100644 arch/um/lkl/kernel/irq.c

diff --git a/arch/um/lkl/include/asm/irq.h b/arch/um/lkl/include/asm/irq.h
new file mode 100644
index 000000000000..948fc54cb76c
--- /dev/null
+++ b/arch/um/lkl/include/asm/irq.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_IRQ_H
+#define _ASM_LKL_IRQ_H
+
+#define IRQ_STATUS_BITS		(sizeof(long) * 8)
+#define NR_IRQS			((int)(IRQ_STATUS_BITS * IRQ_STATUS_BITS))
+
+void run_irqs(void);
+void set_irq_pending(int irq);
+
+#include <uapi/asm/irq.h>
+
+#endif
diff --git a/arch/um/lkl/include/uapi/asm/irq.h b/arch/um/lkl/include/uapi/asm/irq.h
new file mode 100644
index 000000000000..666628b233eb
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/irq.h
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _ASM_UAPI_LKL_IRQ_H
+#define _ASM_UAPI_LKL_IRQ_H
+
+/**
+ * lkl_trigger_irq - generate an interrupt
+ *
+ * This function is used by the device host side to signal its Linux counterpart
+ * that some event happened.
+ *
+ * @irq - the irq number to signal
+ */
+int lkl_trigger_irq(int irq);
+
+/**
+ * lkl_get_free_irq - find and reserve a free IRQ number
+ *
+ * This function is called by the host device code to find an unused IRQ number
+ * and reserved it for its own use.
+ *
+ * @user - a string to identify the user
+ * @returns - and irq number that can be used by request_irq or an negative
+ * value in case of an error
+ */
+int lkl_get_free_irq(const char *user);
+
+/**
+ * lkl_put_irq - release an IRQ number previously obtained with lkl_get_free_irq
+ *
+ * @irq - irq number to release
+ * @user - string identifying the user; should be the same as the one passed to
+ * lkl_get_free_irq when the irq number was obtained
+ */
+void lkl_put_irq(int irq, const char *name);
+
+#endif
diff --git a/arch/um/lkl/include/uapi/asm/sigcontext.h b/arch/um/lkl/include/uapi/asm/sigcontext.h
new file mode 100644
index 000000000000..2f4848843d1d
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/sigcontext.h
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _ASM_UAPI_LKL_SIGCONTEXT_H
+#define _ASM_UAPI_LKL_SIGCONTEXT_H
+
+#include <asm/ptrace.h>
+
+struct pt_regs {
+	void *irq_data;
+};
+
+struct sigcontext {
+	struct pt_regs regs;
+	unsigned long oldmask;
+};
+
+#endif
diff --git a/arch/um/lkl/kernel/irq.c b/arch/um/lkl/kernel/irq.c
new file mode 100644
index 000000000000..e3b59e46ca50
--- /dev/null
+++ b/arch/um/lkl/kernel/irq.c
@@ -0,0 +1,193 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/kernel.h>
+#include <linux/init.h>
+#include <linux/irq.h>
+#include <linux/hardirq.h>
+#include <asm/irq_regs.h>
+#include <linux/sched.h>
+#include <linux/seq_file.h>
+#include <linux/tick.h>
+#include <asm/irqflags.h>
+#include <asm/host_ops.h>
+#include <asm/cpu.h>
+
+/*
+ * To avoid much overhead we use an indirect approach: the irqs are marked using
+ * a bitmap (array of longs) and a summary of the modified bits is kept in a
+ * separate "index" long - one bit for each sizeof(long). Thus we can support
+ * 4096 irqs on 64bit platforms and 1024 irqs on 32bit platforms.
+ *
+ * Whenever an irq is trigger both the array and the index is updated. To find
+ * which irqs were triggered we first search the index and then the
+ * corresponding part of the arrary.
+ */
+static unsigned long irq_status[NR_IRQS / IRQ_STATUS_BITS];
+static unsigned long irq_index_status;
+
+static inline unsigned long test_and_clear_irq_index_status(void)
+{
+	if (!irq_index_status)
+		return 0;
+	return __sync_fetch_and_and(&irq_index_status, 0);
+}
+
+static inline unsigned long test_and_clear_irq_status(int index)
+{
+	if (!&irq_status[index])
+		return 0;
+	return __sync_fetch_and_and(&irq_status[index], 0);
+}
+
+void set_irq_pending(int irq)
+{
+	int index = irq / IRQ_STATUS_BITS;
+	int bit = irq % IRQ_STATUS_BITS;
+
+	__sync_fetch_and_or(&irq_status[index], BIT(bit));
+	__sync_fetch_and_or(&irq_index_status, BIT(index));
+}
+
+static struct irq_info {
+	const char *user;
+} irqs[NR_IRQS];
+
+static bool irqs_enabled;
+
+static struct pt_regs dummy;
+
+static void run_irq(int irq)
+{
+	unsigned long flags;
+	struct pt_regs *old_regs = set_irq_regs((struct pt_regs *)&dummy);
+
+	/* interrupt handlers need to run with interrupts disabled */
+	local_irq_save(flags);
+	irq_enter();
+	generic_handle_irq(irq);
+	irq_exit();
+	set_irq_regs(old_regs);
+	local_irq_restore(flags);
+}
+
+/**
+ * This function can be called from arbitrary host threads, so do not
+ * issue any Linux calls (e.g. prink) if lkl_cpu_get() was not issued
+ * before.
+ */
+int lkl_trigger_irq(int irq)
+{
+	int ret;
+
+	if (!irq || irq > NR_IRQS)
+		return -EINVAL;
+
+	ret = lkl_cpu_try_run_irq(irq);
+	if (ret <= 0)
+		return ret;
+
+	/*
+	 * Since this can be called from Linux context (e.g. lkl_trigger_irq ->
+	 * IRQ -> softirq -> lkl_trigger_irq) make sure we are actually allowed
+	 * to run irqs at this point
+	 */
+	if (!irqs_enabled) {
+		set_irq_pending(irq);
+		lkl_cpu_put();
+		return 0;
+	}
+
+	run_irq(irq);
+
+	lkl_cpu_put();
+
+	return 0;
+}
+
+static inline void for_each_bit(unsigned long word, void (*f)(int, int), int j)
+{
+	int i = 0;
+
+	while (word) {
+		if (word & 1)
+			f(i, j);
+		word >>= 1;
+		i++;
+	}
+}
+
+static inline void deliver_irq(int bit, int index)
+{
+	run_irq(index * IRQ_STATUS_BITS + bit);
+}
+
+static inline void check_irq_status(int i, int unused)
+{
+	for_each_bit(test_and_clear_irq_status(i), deliver_irq, i);
+}
+
+void run_irqs(void)
+{
+	for_each_bit(test_and_clear_irq_index_status(), check_irq_status, 0);
+}
+
+int show_interrupts(struct seq_file *p, void *v)
+{
+	return 0;
+}
+
+int lkl_get_free_irq(const char *user)
+{
+	int i;
+	int ret = -EBUSY;
+
+	/* 0 is not a valid IRQ */
+	for (i = 1; i < NR_IRQS; i++) {
+		if (!irqs[i].user) {
+			irqs[i].user = user;
+			irq_set_chip_and_handler(i, &dummy_irq_chip,
+						 handle_simple_irq);
+			ret = i;
+			break;
+		}
+	}
+
+	return ret;
+}
+
+void lkl_put_irq(int i, const char *user)
+{
+	if (!irqs[i].user || strcmp(irqs[i].user, user) != 0) {
+		WARN("%s tried to release %s's irq %d", user, irqs[i].user, i);
+		return;
+	}
+
+	irqs[i].user = NULL;
+}
+
+unsigned long arch_local_save_flags(void)
+{
+	return irqs_enabled;
+}
+
+void arch_local_irq_restore(unsigned long flags)
+{
+	if (flags == ARCH_IRQ_ENABLED && irqs_enabled == ARCH_IRQ_DISABLED &&
+	    !in_interrupt())
+		run_irqs();
+	irqs_enabled = flags;
+}
+
+void init_IRQ(void)
+{
+	int i;
+
+	for (i = 0; i < NR_IRQS; i++)
+		irq_set_chip_and_handler(i, &dummy_irq_chip, handle_simple_irq);
+
+	pr_info("lkl: irqs initialized\n");
+}
+
+void cpu_yield_to_irqs(void)
+{
+	cpu_relax();
+}
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v3 08/26] um lkl: system call interface and application API
  2020-02-05  7:30 [RFC v3 00/26] Unifying LKL into UML Hajime Tazaki
@ 2020-02-05  7:30   ` Hajime Tazaki
       [not found] ` <cover.1580882335.git.thehajime-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
                     ` (25 subsequent siblings)
  26 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05  7:30 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Conrad Meyer, Jens Staal, Lai Jiangshan, Luca Dariz,
	Michael Zimmermann, Patrick Collins, Pierre-Hugues Husson,
	Yuan Liu, Hajime Tazaki

From: Octavian Purdila <tavi.purdila@gmail.com>

The LKL application API is based on the kernel system call interface
in order to offer a stable API to applications. Note that we can't
offer the full Linux system call interface due to LKL limitations such
as lack of virtual memory, signal, user processes, etc.

The host is using the LKL interrupt mechanism (lkl_trigger_irq) to
initiate a system call. The system call is executed in the context of
the init process.

To avoid collisions between the Linux API and the LKL API (e.g.  struct
stat, MKNOD, etc.) we use a python script to modify the user headers
and to prefix all of the global symbols (structures, typedefs,
defines) with LKL, lkl, _LKL, _lkl, __LKL or __lkl.

Cc: Conrad Meyer <cem@FreeBSD.org>
Cc: Jens Staal <staal1978@gmail.com>
Cc: Lai Jiangshan <jiangshanlai@gmail.com>
Cc: Luca Dariz <luca.dariz@gmail.com>
Cc: Michael Zimmermann <sigmaepsilon92@gmail.com>
Cc: Patrick Collins <pscollins@google.com>
Cc: Pierre-Hugues Husson <phh@phh.me>
Cc: Yuan Liu <liuyuan@google.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/lkl/include/asm/unistd.h        |  27 ++++
 arch/um/lkl/include/uapi/asm/host_ops.h |  14 ++
 arch/um/lkl/include/uapi/asm/unistd.h   |  16 ++
 arch/um/lkl/kernel/syscalls.c           | 192 +++++++++++++++++++++++
 arch/um/lkl/kernel/syscalls_32.c        | 159 +++++++++++++++++++
 arch/um/lkl/scripts/headers_install.py  | 196 ++++++++++++++++++++++++
 6 files changed, 604 insertions(+)
 create mode 100644 arch/um/lkl/include/asm/unistd.h
 create mode 100644 arch/um/lkl/include/uapi/asm/unistd.h
 create mode 100644 arch/um/lkl/kernel/syscalls.c
 create mode 100644 arch/um/lkl/kernel/syscalls_32.c
 create mode 100755 arch/um/lkl/scripts/headers_install.py

diff --git a/arch/um/lkl/include/asm/unistd.h b/arch/um/lkl/include/asm/unistd.h
new file mode 100644
index 000000000000..ca0eac0f2827
--- /dev/null
+++ b/arch/um/lkl/include/asm/unistd.h
@@ -0,0 +1,27 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <uapi/asm/unistd.h>
+
+#define __SC_ASCII(t, a) #t "," #a
+
+#define __ASCII_MAP0(m, ...)
+#define __ASCII_MAP1(m, t, a) m(t, a)
+#define __ASCII_MAP2(m, t, a, ...) m(t, a) "," __ASCII_MAP1(m, __VA_ARGS__)
+#define __ASCII_MAP3(m, t, a, ...) m(t, a) "," __ASCII_MAP2(m, __VA_ARGS__)
+#define __ASCII_MAP4(m, t, a, ...) m(t, a) "," __ASCII_MAP3(m, __VA_ARGS__)
+#define __ASCII_MAP5(m, t, a, ...) m(t, a) "," __ASCII_MAP4(m, __VA_ARGS__)
+#define __ASCII_MAP6(m, t, a, ...) m(t, a) "," __ASCII_MAP5(m, __VA_ARGS__)
+#define __ASCII_MAP(n, ...) __ASCII_MAP##n(__VA_ARGS__)
+
+#ifdef __MINGW32__
+#define SECTION_ATTRS "n0"
+#else
+#define SECTION_ATTRS "a"
+#endif
+
+#define __SYSCALL_DEFINE_ARCH(x, name, ...)				\
+	asm(".section .syscall_defs,\"" SECTION_ATTRS "\"\n"		\
+	    ".ascii \"#ifdef __NR" #name "\\n\"\n"			\
+	    ".ascii \"SYSCALL_DEFINE" #x "(" #name ","			\
+	    __ASCII_MAP(x, __SC_ASCII, __VA_ARGS__) ")\\n\"\n"		\
+	    ".ascii \"#endif\\n\"\n"					\
+	    ".section .text\n");
diff --git a/arch/um/lkl/include/uapi/asm/host_ops.h b/arch/um/lkl/include/uapi/asm/host_ops.h
index 19924fc7c718..1c839d7139f8 100644
--- a/arch/um/lkl/include/uapi/asm/host_ops.h
+++ b/arch/um/lkl/include/uapi/asm/host_ops.h
@@ -36,6 +36,15 @@ struct lkl_jmp_buf {
  * @thread_join - wait for the given thread to terminate. Returns 0
  * for success, -1 otherwise
  *
+ * @tls_alloc - allocate a thread local storage key; returns 0 if successful; if
+ * destructor is not NULL it will be called when a thread terminates with its
+ * argument set to the current thread local storage value
+ * @tls_free - frees a thread local storage key; returns 0 if successful
+ * @tls_set - associate data to the thread local storage key; returns 0 if
+ * successful
+ * @tls_get - return data associated with the thread local storage key or NULL
+ * on error
+ *
  * @mem_alloc - allocate memory
  * @mem_free - free memory
  *
@@ -71,6 +80,11 @@ struct lkl_host_operations {
 	lkl_thread_t (*thread_self)(void);
 	int (*thread_equal)(lkl_thread_t a, lkl_thread_t b);
 
+	struct lkl_tls_key *(*tls_alloc)(void (*destructor)(void *));
+	void (*tls_free)(struct lkl_tls_key *key);
+	int (*tls_set)(struct lkl_tls_key *key, void *data);
+	void *(*tls_get)(struct lkl_tls_key *key);
+
 	void *(*mem_alloc)(unsigned long mem);
 	void (*mem_free)(void *mem);
 
diff --git a/arch/um/lkl/include/uapi/asm/unistd.h b/arch/um/lkl/include/uapi/asm/unistd.h
new file mode 100644
index 000000000000..072474a4060d
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/unistd.h
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#define __ARCH_WANT_SYSCALL_NO_AT
+#define __ARCH_WANT_SYSCALL_DEPRECATED
+#define __ARCH_WANT_SYSCALL_NO_FLAGS
+#define __ARCH_WANT_RENAMEAT
+#define __ARCH_WANT_NEW_STAT
+#define __ARCH_WANT_SET_GET_RLIMIT
+#define __ARCH_WANT_TIME32_SYSCALLS
+
+#include <asm/bitsperlong.h>
+
+#if __BITS_PER_LONG == 64
+#define __ARCH_WANT_SYS_NEWFSTATAT
+#endif
+
+#include <asm-generic/unistd.h>
diff --git a/arch/um/lkl/kernel/syscalls.c b/arch/um/lkl/kernel/syscalls.c
new file mode 100644
index 000000000000..3ab1d776ca4b
--- /dev/null
+++ b/arch/um/lkl/kernel/syscalls.c
@@ -0,0 +1,192 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/stat.h>
+#include <linux/irq.h>
+#include <linux/sched.h>
+#include <linux/interrupt.h>
+#include <linux/jhash.h>
+#include <linux/slab.h>
+#include <linux/types.h>
+#include <linux/net.h>
+#include <linux/task_work.h>
+#include <linux/syscalls.h>
+#include <linux/kthread.h>
+#include <linux/platform_device.h>
+#include <asm/host_ops.h>
+#include <asm/syscalls.h>
+#include <asm/syscalls_32.h>
+#include <asm/cpu.h>
+#include <asm/sched.h>
+
+typedef long (*syscall_handler_t)(long arg1, ...);
+
+#undef __SYSCALL
+#define __SYSCALL(nr, sym)[nr] = (syscall_handler_t)sym,
+
+static syscall_handler_t syscall_table[__NR_syscalls] = {
+	[0 ... __NR_syscalls - 1] = (syscall_handler_t)sys_ni_syscall,
+#include <asm/unistd.h>
+
+#if __BITS_PER_LONG == 32
+#include <asm/unistd_32.h>
+#endif
+};
+
+static long run_syscall(long no, long *params)
+{
+	long ret;
+
+	if (no < 0 || no >= __NR_syscalls)
+		return -ENOSYS;
+
+	ret = syscall_table[no](params[0], params[1], params[2], params[3],
+				params[4], params[5]);
+
+	task_work_run();
+
+	return ret;
+}
+
+
+#define CLONE_FLAGS (CLONE_VM | CLONE_FS | CLONE_FILES | CLONE_THREAD |	\
+		     CLONE_SIGHAND | SIGCHLD)
+
+static int host_task_id;
+static struct task_struct *host0;
+
+static int new_host_task(struct task_struct **task)
+{
+	pid_t pid;
+
+	switch_to_host_task(host0);
+
+	pid = kernel_thread(host_task_stub, NULL, CLONE_FLAGS);
+	if (pid < 0)
+		return pid;
+
+	rcu_read_lock();
+	*task = find_task_by_pid_ns(pid, &init_pid_ns);
+	rcu_read_unlock();
+
+	host_task_id++;
+
+	snprintf((*task)->comm, sizeof((*task)->comm), "host%d", host_task_id);
+
+	return 0;
+}
+static void exit_task(void)
+{
+	do_exit(0);
+}
+
+static void del_host_task(void *arg)
+{
+	struct task_struct *task = (struct task_struct *)arg;
+	struct thread_info *ti = task_thread_info(task);
+
+	if (lkl_cpu_get() < 0)
+		return;
+
+	switch_to_host_task(task);
+	host_task_id--;
+	set_ti_thread_flag(ti, TIF_SCHED_JB);
+	lkl_ops->jmp_buf_set(&ti->sched_jb, exit_task);
+}
+
+static struct lkl_tls_key *task_key;
+
+long lkl_syscall(long no, long *params)
+{
+	struct task_struct *task = host0;
+	long ret;
+
+	ret = lkl_cpu_get();
+	if (ret < 0)
+		return ret;
+
+	if (lkl_ops->tls_get) {
+		task = lkl_ops->tls_get(task_key);
+		if (!task) {
+			ret = new_host_task(&task);
+			if (ret)
+				goto out;
+			lkl_ops->tls_set(task_key, task);
+		}
+	}
+
+	switch_to_host_task(task);
+
+	ret = run_syscall(no, params);
+
+	if (no == __NR_reboot) {
+		thread_sched_jb();
+		return ret;
+	}
+
+out:
+	lkl_cpu_put();
+
+	return ret;
+}
+
+static struct task_struct *idle_host_task;
+
+/* called from idle, don't failed, don't block */
+void wakeup_idle_host_task(void)
+{
+	if (!need_resched() && idle_host_task)
+		wake_up_process(idle_host_task);
+}
+
+static int idle_host_task_loop(void *unused)
+{
+	struct thread_info *ti = task_thread_info(current);
+
+	snprintf(current->comm, sizeof(current->comm), "idle_host_task");
+	set_thread_flag(TIF_HOST_THREAD);
+	idle_host_task = current;
+
+	for (;;) {
+		lkl_cpu_put();
+		lkl_ops->sem_down(ti->sched_sem);
+		if (idle_host_task == NULL) {
+			lkl_ops->thread_exit();
+			return 0;
+		}
+		schedule_tail(ti->prev_sched);
+	}
+}
+
+int syscalls_init(void)
+{
+	snprintf(current->comm, sizeof(current->comm), "host0");
+	set_thread_flag(TIF_HOST_THREAD);
+	host0 = current;
+
+	if (lkl_ops->tls_alloc) {
+		task_key = lkl_ops->tls_alloc(del_host_task);
+		if (!task_key)
+			return -1;
+	}
+
+	if (kernel_thread(idle_host_task_loop, NULL, CLONE_FLAGS) < 0) {
+		if (lkl_ops->tls_free)
+			lkl_ops->tls_free(task_key);
+		return -1;
+	}
+
+	return 0;
+}
+
+void syscalls_cleanup(void)
+{
+	if (idle_host_task) {
+		struct thread_info *ti = task_thread_info(idle_host_task);
+
+		idle_host_task = NULL;
+		lkl_ops->sem_up(ti->sched_sem);
+		lkl_ops->thread_join(ti->tid);
+	}
+
+	if (lkl_ops->tls_free)
+		lkl_ops->tls_free(task_key);
+}
diff --git a/arch/um/lkl/kernel/syscalls_32.c b/arch/um/lkl/kernel/syscalls_32.c
new file mode 100644
index 000000000000..a4271593c338
--- /dev/null
+++ b/arch/um/lkl/kernel/syscalls_32.c
@@ -0,0 +1,159 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * sys_ia32.c: Conversion between 32bit and 64bit native syscalls. Based on
+ *             sys_sparc32
+ *
+ * Copyright (C) 2000		VA Linux Co
+ * Copyright (C) 2000		Don Dugger <n0ano@valinux.com>
+ * Copyright (C) 1999		Arun Sharma <arun.sharma@intel.com>
+ * Copyright (C) 1997,1998	Jakub Jelinek (jj@sunsite.mff.cuni.cz)
+ * Copyright (C) 1997		David S. Miller (davem@caip.rutgers.edu)
+ * Copyright (C) 2000		Hewlett-Packard Co.
+ * Copyright (C) 2000		David Mosberger-Tang <davidm@hpl.hp.com>
+ * Copyright (C) 2000,2001,2002	Andi Kleen, SuSE Labs (x86-64 port)
+ *
+ * These routines maintain argument size conversion between 32bit and 64bit
+ * environment. In 2.5 most of this should be moved to a generic directory.
+ *
+ * This file assumes that there is a hole at the end of user address space.
+ *
+ * Some of the functions are LE specific currently. These are
+ * hopefully all marked.  This should be fixed.
+ */
+
+#include <linux/kernel.h>
+#include <linux/sched.h>
+#include <linux/fs.h>
+#include <linux/file.h>
+#include <linux/signal.h>
+#include <linux/syscalls.h>
+#include <linux/times.h>
+#include <linux/utsname.h>
+#include <linux/mm.h>
+#include <linux/uio.h>
+#include <linux/poll.h>
+#include <linux/personality.h>
+#include <linux/stat.h>
+#include <linux/rwsem.h>
+#include <linux/compat.h>
+#include <linux/vfs.h>
+#include <linux/ptrace.h>
+#include <linux/highuid.h>
+#include <linux/sysctl.h>
+#include <linux/slab.h>
+#include <asm/types.h>
+#include <linux/atomic.h>
+#include <asm/syscalls_32.h>
+
+#define AA(__x)		((unsigned long)(__x))
+
+#if __BITS_PER_LONG == 32
+
+asmlinkage long sys32_truncate64(const char __user *filename,
+				 unsigned long offset_low,
+				 unsigned long offset_high)
+{
+	return sys_truncate64(filename,
+			      ((loff_t)offset_high << 32) | offset_low);
+}
+
+asmlinkage long sys32_ftruncate64(unsigned int fd, unsigned long offset_low,
+				  unsigned long offset_high)
+{
+	return sys_ftruncate64(fd, ((loff_t)offset_high << 32) | offset_low);
+}
+
+#ifdef CONFIG_MMU
+/*
+ * Linux/i386 didn't use to be able to handle more than
+ * 4 system call parameters, so these system calls used a memory
+ * block for parameter passing..
+ */
+
+struct mmap_arg_struct32 {
+	unsigned int addr;
+	unsigned int len;
+	unsigned int prot;
+	unsigned int flags;
+	unsigned int fd;
+	unsigned int offset;
+};
+
+asmlinkage long sys32_mmap(struct mmap_arg_struct32 __user *arg)
+{
+	struct mmap_arg_struct32 a;
+
+	if (copy_from_user(&a, arg, sizeof(a)))
+		return -EFAULT;
+
+	if (a.offset & ~PAGE_MASK)
+		return -EINVAL;
+
+	return sys_mmap_pgoff(a.addr, a.len, a.prot, a.flags, a.fd,
+			      a.offset >> PAGE_SHIFT);
+}
+#endif
+
+asmlinkage long sys32_wait4(pid_t pid, unsigned int __user *stat_addr,
+			    int options, struct rusage __user *ru)
+{
+	return sys_wait4(pid, stat_addr, options, ru);
+}
+
+asmlinkage long sys32_pread64(unsigned int fd, char __user *ubuf, u32 count,
+			      u32 poslo, u32 poshi)
+{
+	return sys_pread64(fd, ubuf, count,
+			   ((loff_t)AA(poshi) << 32) | AA(poslo));
+}
+
+asmlinkage long sys32_pwrite64(unsigned int fd, const char __user *ubuf,
+			       u32 count, u32 poslo, u32 poshi)
+{
+	return sys_pwrite64(fd, ubuf, count,
+			    ((loff_t)AA(poshi) << 32) | AA(poslo));
+}
+
+/*
+ * Some system calls that need sign extended arguments. This could be
+ * done by a generic wrapper.
+ */
+long sys32_fadvise64_64(int fd, __u32 offset_low, __u32 offset_high,
+			__u32 len_low, __u32 len_high, int advice)
+{
+	return sys_fadvise64_64(fd, (((u64)offset_high) << 32) | offset_low,
+				(((u64)len_high) << 32) | len_low, advice);
+}
+
+asmlinkage ssize_t sys32_readahead(int fd, unsigned int off_lo,
+				   unsigned int off_hi, size_t count)
+{
+	return sys_readahead(fd, ((u64)off_hi << 32) | off_lo, count);
+}
+
+asmlinkage long sys32_sync_file_range(int fd, unsigned int off_low,
+				      unsigned int off_hi, unsigned int n_low,
+				      unsigned int n_hi, unsigned int flags)
+{
+	return sys_sync_file_range(fd, ((u64)off_hi << 32) | off_low,
+				   ((u64)n_hi << 32) | n_low, flags);
+}
+
+asmlinkage long sys32_sync_file_range2(int fd, unsigned int flags,
+				       unsigned int off_low,
+				       unsigned int off_hi, unsigned int n_low,
+				       unsigned int n_hi)
+{
+	return sys_sync_file_range(fd, ((u64)off_hi << 32) | off_low,
+				   ((u64)n_hi << 32) | n_low, flags);
+}
+
+asmlinkage long sys32_fallocate(int fd, int mode, unsigned int offset_lo,
+				unsigned int offset_hi, unsigned int len_lo,
+				unsigned int len_hi)
+{
+	return sys_fallocate(fd, mode, ((u64)offset_hi << 32) | offset_lo,
+			     ((u64)len_hi << 32) | len_lo);
+}
+
+#endif
diff --git a/arch/um/lkl/scripts/headers_install.py b/arch/um/lkl/scripts/headers_install.py
new file mode 100755
index 000000000000..30101bd1d4bf
--- /dev/null
+++ b/arch/um/lkl/scripts/headers_install.py
@@ -0,0 +1,196 @@
+#!/usr/bin/env python
+# SPDX-License-Identifier: GPL-2.0
+import re, os, sys, argparse, multiprocessing, fnmatch
+
+srctree = os.environ["srctree"]
+objtree = os.environ["objtree"]
+install_hdr_path = os.environ["INSTALL_HDR_PATH"]
+header_paths = [ install_hdr_path + "/include/",
+                 "arch/um/lkl/include/generated/uapi/", "include/generated/" ]
+
+headers = set()
+includes = set()
+
+def relpath2abspath(relpath):
+    if "generated" in relpath:
+        return objtree + "/" + relpath
+    else:
+        return objtree + "/" + relpath
+
+def find_headers(path):
+    headers.add(path)
+    f = open(relpath2abspath(path))
+    for l in f.readlines():
+        m = re.search("#include <(.*)>", l)
+        try:
+            i = m.group(1)
+            for p in header_paths:
+                if os.access(relpath2abspath(p + i), os.R_OK):
+                    if p + i not in headers:
+                        includes.add(i)
+                        headers.add(p + i)
+                        find_headers(p + i)
+        except:
+            pass
+    f.close()
+
+def has_lkl_prefix(w):
+  return w.startswith("lkl") or w.startswith("_lkl") or w.startswith("__lkl") \
+         or w.startswith("LKL") or w.startswith("_LKL") or w.startswith("__LKL")
+
+def find_symbols(regexp, store):
+    for h in headers:
+        f = open(h)
+        for l in f.readlines():
+            m = regexp.search(l)
+            if not m:
+                continue
+            for e in reversed(m.groups()):
+                if e:
+                    if not has_lkl_prefix(e):
+                        store.add(e)
+                    break
+        f.close()
+
+def find_ml_symbols(regexp, store):
+    for h in headers:
+        for i in regexp.finditer(open(h).read()):
+            for j in reversed(i.groups()):
+                if j:
+                    if not has_lkl_prefix(j):
+                        store.add(j)
+                    break
+
+def find_enums(block_regexp, symbol_regexp, store):
+    for h in headers:
+        # remove comments
+        content = re.sub(re.compile("(\/\*(\*(?!\/)|[^*])*\*\/)", re.S|re.M), " ", open(h).read())
+        # remove preprocesor lines
+        clean_content = ""
+        for l in content.split("\n"):
+            if re.match("\s*#", l):
+                continue
+            clean_content += l + "\n"
+        for i in block_regexp.finditer(clean_content):
+            for j in reversed(i.groups()):
+                if j:
+                    for k in symbol_regexp.finditer(j):
+                        for l in k.groups():
+                            if l:
+                                if not has_lkl_prefix(l):
+                                    store.add(l)
+                                break
+
+def lkl_prefix(w):
+    r = ""
+
+    if w.startswith("__"):
+        r = "__"
+    elif w.startswith("_"):
+        r = "_"
+
+    if w.isupper():
+        r += "LKL"
+    else:
+        r += "lkl"
+
+    if not w.startswith("_"):
+        r += "_"
+
+    r += w
+
+    return r
+
+def replace(h):
+    content = open(h).read()
+    for i in includes:
+        search_str = "(#[ \t]*include[ \t]*[<\"][ \t]*)" + i + "([ \t]*[>\"])"
+        replace_str = "\\1" + "lkl/" + i + "\\2"
+        content = re.sub(search_str, replace_str, content)
+    tmp = ""
+    for w in re.split("(\W+)", content):
+        if w in defines:
+            w = lkl_prefix(w)
+        tmp += w
+    content = tmp
+    for s in structs:
+        search_str = "(\W?struct\s+)" + s + "(\W)"
+        replace_str = "\\1" + lkl_prefix(s) + "\\2"
+        content = re.sub(search_str, replace_str, content, flags = re.MULTILINE)
+    for s in unions:
+        search_str = "(\W?union\s+)" + s + "(\W)"
+        replace_str = "\\1" + lkl_prefix(s) + "\\2"
+        content = re.sub(search_str, replace_str, content, flags = re.MULTILINE)
+    open(h, 'w').write(content)
+
+parser = argparse.ArgumentParser(description='install lkl headers')
+parser.add_argument('path', help='path to install to', )
+parser.add_argument('-j', '--jobs', help='number of parallel jobs', default=1, type=int)
+args = parser.parse_args()
+
+find_headers(install_hdr_path + "/include/asm/syscalls.h")
+headers.add(install_hdr_path + "/include/asm/host_ops.h")
+
+if 'LKL_INSTALL_ADDITIONAL_HEADERS' in os.environ:
+    with open(os.environ['LKL_INSTALL_ADDITIONAL_HEADERS'], 'rU') as f:
+        for line in f.readlines():
+            line = line.split('#', 1)[0].strip()
+            if line != '':
+                headers.add(line)
+
+new_headers = set()
+
+for h in headers:
+    dir = os.path.dirname(h)
+    out_dir = args.path + "/" + re.sub("(arch/um/lkl/include/generated/uapi/|include/generated/uapi/|include/generated|" + install_hdr_path + "/include/)(.*)", "lkl/\\2", dir)
+    try:
+        os.makedirs(out_dir)
+    except:
+        pass
+    print("  INSTALL\t%s" % (out_dir + "/" + os.path.basename(h)))
+    os.system(srctree+"/scripts/headers_install.sh %s %s" % (os.path.abspath(h),
+                                                       out_dir + "/" + os.path.basename(h)))
+    new_headers.add(out_dir + "/" + os.path.basename(h))
+
+headers = new_headers
+
+defines = set()
+structs = set()
+unions = set()
+
+p = re.compile("#[ \t]*define[ \t]*(\w+)")
+find_symbols(p, defines)
+p = re.compile("typedef.*(\(\*(\w+)\)\(.*\)\s*|\W+(\w+)\s*|\s+(\w+)\(.*\)\s*);")
+find_symbols(p, defines)
+p = re.compile("typedef\s+(struct|union)\s+\w*\s*{[^\\{\}]*}\W*(\w+)\s*;", re.M|re.S)
+find_ml_symbols(p, defines)
+defines.add("siginfo_t")
+defines.add("sigevent_t")
+p = re.compile("struct\s+(\w+)\s*\{")
+find_symbols(p, structs)
+structs.add("iovec")
+p = re.compile("union\s+(\w+)\s*\{")
+find_symbols(p, unions)
+p = re.compile("static\s+__inline__(\s+\w+)+\s+(\w+)\([^)]*\)\s")
+find_symbols(p, defines)
+p = re.compile("static\s+__always_inline(\s+\w+)+\s+(\w+)\([^)]*\)\s")
+find_symbols(p, defines)
+p = re.compile("enum\s+(\w*)\s*{([^}]*)}", re.M|re.S)
+q = re.compile("(\w+)\s*(,|=[^,]*|$)", re.M|re.S)
+find_enums(p, q, defines)
+
+# needed for i386
+defines.add("__NR_stime")
+
+def process_header(h):
+    print("  REPLACE\t%s" % (out_dir + "/" + os.path.basename(h)))
+    replace(h)
+
+p = multiprocessing.Pool(args.jobs)
+try:
+    p.map_async(process_header, headers).wait(999999)
+    p.close()
+except:
+    p.terminate()
+finally:
+    p.join()
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v3 08/26] um lkl: system call interface and application API
@ 2020-02-05  7:30   ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05  7:30 UTC (permalink / raw)
  To: linux-um
  Cc: linux-arch, Hajime Tazaki, Conrad Meyer, Octavian Purdila,
	Jens Staal, Lai Jiangshan, Akira Moroo, Patrick Collins,
	linux-kernel-library, Pierre-Hugues Husson, Michael Zimmermann,
	Luca Dariz, Yuan Liu

From: Octavian Purdila <tavi.purdila@gmail.com>

The LKL application API is based on the kernel system call interface
in order to offer a stable API to applications. Note that we can't
offer the full Linux system call interface due to LKL limitations such
as lack of virtual memory, signal, user processes, etc.

The host is using the LKL interrupt mechanism (lkl_trigger_irq) to
initiate a system call. The system call is executed in the context of
the init process.

To avoid collisions between the Linux API and the LKL API (e.g.  struct
stat, MKNOD, etc.) we use a python script to modify the user headers
and to prefix all of the global symbols (structures, typedefs,
defines) with LKL, lkl, _LKL, _lkl, __LKL or __lkl.

Cc: Conrad Meyer <cem@FreeBSD.org>
Cc: Jens Staal <staal1978@gmail.com>
Cc: Lai Jiangshan <jiangshanlai@gmail.com>
Cc: Luca Dariz <luca.dariz@gmail.com>
Cc: Michael Zimmermann <sigmaepsilon92@gmail.com>
Cc: Patrick Collins <pscollins@google.com>
Cc: Pierre-Hugues Husson <phh@phh.me>
Cc: Yuan Liu <liuyuan@google.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/lkl/include/asm/unistd.h        |  27 ++++
 arch/um/lkl/include/uapi/asm/host_ops.h |  14 ++
 arch/um/lkl/include/uapi/asm/unistd.h   |  16 ++
 arch/um/lkl/kernel/syscalls.c           | 192 +++++++++++++++++++++++
 arch/um/lkl/kernel/syscalls_32.c        | 159 +++++++++++++++++++
 arch/um/lkl/scripts/headers_install.py  | 196 ++++++++++++++++++++++++
 6 files changed, 604 insertions(+)
 create mode 100644 arch/um/lkl/include/asm/unistd.h
 create mode 100644 arch/um/lkl/include/uapi/asm/unistd.h
 create mode 100644 arch/um/lkl/kernel/syscalls.c
 create mode 100644 arch/um/lkl/kernel/syscalls_32.c
 create mode 100755 arch/um/lkl/scripts/headers_install.py

diff --git a/arch/um/lkl/include/asm/unistd.h b/arch/um/lkl/include/asm/unistd.h
new file mode 100644
index 000000000000..ca0eac0f2827
--- /dev/null
+++ b/arch/um/lkl/include/asm/unistd.h
@@ -0,0 +1,27 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <uapi/asm/unistd.h>
+
+#define __SC_ASCII(t, a) #t "," #a
+
+#define __ASCII_MAP0(m, ...)
+#define __ASCII_MAP1(m, t, a) m(t, a)
+#define __ASCII_MAP2(m, t, a, ...) m(t, a) "," __ASCII_MAP1(m, __VA_ARGS__)
+#define __ASCII_MAP3(m, t, a, ...) m(t, a) "," __ASCII_MAP2(m, __VA_ARGS__)
+#define __ASCII_MAP4(m, t, a, ...) m(t, a) "," __ASCII_MAP3(m, __VA_ARGS__)
+#define __ASCII_MAP5(m, t, a, ...) m(t, a) "," __ASCII_MAP4(m, __VA_ARGS__)
+#define __ASCII_MAP6(m, t, a, ...) m(t, a) "," __ASCII_MAP5(m, __VA_ARGS__)
+#define __ASCII_MAP(n, ...) __ASCII_MAP##n(__VA_ARGS__)
+
+#ifdef __MINGW32__
+#define SECTION_ATTRS "n0"
+#else
+#define SECTION_ATTRS "a"
+#endif
+
+#define __SYSCALL_DEFINE_ARCH(x, name, ...)				\
+	asm(".section .syscall_defs,\"" SECTION_ATTRS "\"\n"		\
+	    ".ascii \"#ifdef __NR" #name "\\n\"\n"			\
+	    ".ascii \"SYSCALL_DEFINE" #x "(" #name ","			\
+	    __ASCII_MAP(x, __SC_ASCII, __VA_ARGS__) ")\\n\"\n"		\
+	    ".ascii \"#endif\\n\"\n"					\
+	    ".section .text\n");
diff --git a/arch/um/lkl/include/uapi/asm/host_ops.h b/arch/um/lkl/include/uapi/asm/host_ops.h
index 19924fc7c718..1c839d7139f8 100644
--- a/arch/um/lkl/include/uapi/asm/host_ops.h
+++ b/arch/um/lkl/include/uapi/asm/host_ops.h
@@ -36,6 +36,15 @@ struct lkl_jmp_buf {
  * @thread_join - wait for the given thread to terminate. Returns 0
  * for success, -1 otherwise
  *
+ * @tls_alloc - allocate a thread local storage key; returns 0 if successful; if
+ * destructor is not NULL it will be called when a thread terminates with its
+ * argument set to the current thread local storage value
+ * @tls_free - frees a thread local storage key; returns 0 if successful
+ * @tls_set - associate data to the thread local storage key; returns 0 if
+ * successful
+ * @tls_get - return data associated with the thread local storage key or NULL
+ * on error
+ *
  * @mem_alloc - allocate memory
  * @mem_free - free memory
  *
@@ -71,6 +80,11 @@ struct lkl_host_operations {
 	lkl_thread_t (*thread_self)(void);
 	int (*thread_equal)(lkl_thread_t a, lkl_thread_t b);
 
+	struct lkl_tls_key *(*tls_alloc)(void (*destructor)(void *));
+	void (*tls_free)(struct lkl_tls_key *key);
+	int (*tls_set)(struct lkl_tls_key *key, void *data);
+	void *(*tls_get)(struct lkl_tls_key *key);
+
 	void *(*mem_alloc)(unsigned long mem);
 	void (*mem_free)(void *mem);
 
diff --git a/arch/um/lkl/include/uapi/asm/unistd.h b/arch/um/lkl/include/uapi/asm/unistd.h
new file mode 100644
index 000000000000..072474a4060d
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/unistd.h
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#define __ARCH_WANT_SYSCALL_NO_AT
+#define __ARCH_WANT_SYSCALL_DEPRECATED
+#define __ARCH_WANT_SYSCALL_NO_FLAGS
+#define __ARCH_WANT_RENAMEAT
+#define __ARCH_WANT_NEW_STAT
+#define __ARCH_WANT_SET_GET_RLIMIT
+#define __ARCH_WANT_TIME32_SYSCALLS
+
+#include <asm/bitsperlong.h>
+
+#if __BITS_PER_LONG == 64
+#define __ARCH_WANT_SYS_NEWFSTATAT
+#endif
+
+#include <asm-generic/unistd.h>
diff --git a/arch/um/lkl/kernel/syscalls.c b/arch/um/lkl/kernel/syscalls.c
new file mode 100644
index 000000000000..3ab1d776ca4b
--- /dev/null
+++ b/arch/um/lkl/kernel/syscalls.c
@@ -0,0 +1,192 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/stat.h>
+#include <linux/irq.h>
+#include <linux/sched.h>
+#include <linux/interrupt.h>
+#include <linux/jhash.h>
+#include <linux/slab.h>
+#include <linux/types.h>
+#include <linux/net.h>
+#include <linux/task_work.h>
+#include <linux/syscalls.h>
+#include <linux/kthread.h>
+#include <linux/platform_device.h>
+#include <asm/host_ops.h>
+#include <asm/syscalls.h>
+#include <asm/syscalls_32.h>
+#include <asm/cpu.h>
+#include <asm/sched.h>
+
+typedef long (*syscall_handler_t)(long arg1, ...);
+
+#undef __SYSCALL
+#define __SYSCALL(nr, sym)[nr] = (syscall_handler_t)sym,
+
+static syscall_handler_t syscall_table[__NR_syscalls] = {
+	[0 ... __NR_syscalls - 1] = (syscall_handler_t)sys_ni_syscall,
+#include <asm/unistd.h>
+
+#if __BITS_PER_LONG == 32
+#include <asm/unistd_32.h>
+#endif
+};
+
+static long run_syscall(long no, long *params)
+{
+	long ret;
+
+	if (no < 0 || no >= __NR_syscalls)
+		return -ENOSYS;
+
+	ret = syscall_table[no](params[0], params[1], params[2], params[3],
+				params[4], params[5]);
+
+	task_work_run();
+
+	return ret;
+}
+
+
+#define CLONE_FLAGS (CLONE_VM | CLONE_FS | CLONE_FILES | CLONE_THREAD |	\
+		     CLONE_SIGHAND | SIGCHLD)
+
+static int host_task_id;
+static struct task_struct *host0;
+
+static int new_host_task(struct task_struct **task)
+{
+	pid_t pid;
+
+	switch_to_host_task(host0);
+
+	pid = kernel_thread(host_task_stub, NULL, CLONE_FLAGS);
+	if (pid < 0)
+		return pid;
+
+	rcu_read_lock();
+	*task = find_task_by_pid_ns(pid, &init_pid_ns);
+	rcu_read_unlock();
+
+	host_task_id++;
+
+	snprintf((*task)->comm, sizeof((*task)->comm), "host%d", host_task_id);
+
+	return 0;
+}
+static void exit_task(void)
+{
+	do_exit(0);
+}
+
+static void del_host_task(void *arg)
+{
+	struct task_struct *task = (struct task_struct *)arg;
+	struct thread_info *ti = task_thread_info(task);
+
+	if (lkl_cpu_get() < 0)
+		return;
+
+	switch_to_host_task(task);
+	host_task_id--;
+	set_ti_thread_flag(ti, TIF_SCHED_JB);
+	lkl_ops->jmp_buf_set(&ti->sched_jb, exit_task);
+}
+
+static struct lkl_tls_key *task_key;
+
+long lkl_syscall(long no, long *params)
+{
+	struct task_struct *task = host0;
+	long ret;
+
+	ret = lkl_cpu_get();
+	if (ret < 0)
+		return ret;
+
+	if (lkl_ops->tls_get) {
+		task = lkl_ops->tls_get(task_key);
+		if (!task) {
+			ret = new_host_task(&task);
+			if (ret)
+				goto out;
+			lkl_ops->tls_set(task_key, task);
+		}
+	}
+
+	switch_to_host_task(task);
+
+	ret = run_syscall(no, params);
+
+	if (no == __NR_reboot) {
+		thread_sched_jb();
+		return ret;
+	}
+
+out:
+	lkl_cpu_put();
+
+	return ret;
+}
+
+static struct task_struct *idle_host_task;
+
+/* called from idle, don't failed, don't block */
+void wakeup_idle_host_task(void)
+{
+	if (!need_resched() && idle_host_task)
+		wake_up_process(idle_host_task);
+}
+
+static int idle_host_task_loop(void *unused)
+{
+	struct thread_info *ti = task_thread_info(current);
+
+	snprintf(current->comm, sizeof(current->comm), "idle_host_task");
+	set_thread_flag(TIF_HOST_THREAD);
+	idle_host_task = current;
+
+	for (;;) {
+		lkl_cpu_put();
+		lkl_ops->sem_down(ti->sched_sem);
+		if (idle_host_task == NULL) {
+			lkl_ops->thread_exit();
+			return 0;
+		}
+		schedule_tail(ti->prev_sched);
+	}
+}
+
+int syscalls_init(void)
+{
+	snprintf(current->comm, sizeof(current->comm), "host0");
+	set_thread_flag(TIF_HOST_THREAD);
+	host0 = current;
+
+	if (lkl_ops->tls_alloc) {
+		task_key = lkl_ops->tls_alloc(del_host_task);
+		if (!task_key)
+			return -1;
+	}
+
+	if (kernel_thread(idle_host_task_loop, NULL, CLONE_FLAGS) < 0) {
+		if (lkl_ops->tls_free)
+			lkl_ops->tls_free(task_key);
+		return -1;
+	}
+
+	return 0;
+}
+
+void syscalls_cleanup(void)
+{
+	if (idle_host_task) {
+		struct thread_info *ti = task_thread_info(idle_host_task);
+
+		idle_host_task = NULL;
+		lkl_ops->sem_up(ti->sched_sem);
+		lkl_ops->thread_join(ti->tid);
+	}
+
+	if (lkl_ops->tls_free)
+		lkl_ops->tls_free(task_key);
+}
diff --git a/arch/um/lkl/kernel/syscalls_32.c b/arch/um/lkl/kernel/syscalls_32.c
new file mode 100644
index 000000000000..a4271593c338
--- /dev/null
+++ b/arch/um/lkl/kernel/syscalls_32.c
@@ -0,0 +1,159 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * sys_ia32.c: Conversion between 32bit and 64bit native syscalls. Based on
+ *             sys_sparc32
+ *
+ * Copyright (C) 2000		VA Linux Co
+ * Copyright (C) 2000		Don Dugger <n0ano@valinux.com>
+ * Copyright (C) 1999		Arun Sharma <arun.sharma@intel.com>
+ * Copyright (C) 1997,1998	Jakub Jelinek (jj@sunsite.mff.cuni.cz)
+ * Copyright (C) 1997		David S. Miller (davem@caip.rutgers.edu)
+ * Copyright (C) 2000		Hewlett-Packard Co.
+ * Copyright (C) 2000		David Mosberger-Tang <davidm@hpl.hp.com>
+ * Copyright (C) 2000,2001,2002	Andi Kleen, SuSE Labs (x86-64 port)
+ *
+ * These routines maintain argument size conversion between 32bit and 64bit
+ * environment. In 2.5 most of this should be moved to a generic directory.
+ *
+ * This file assumes that there is a hole at the end of user address space.
+ *
+ * Some of the functions are LE specific currently. These are
+ * hopefully all marked.  This should be fixed.
+ */
+
+#include <linux/kernel.h>
+#include <linux/sched.h>
+#include <linux/fs.h>
+#include <linux/file.h>
+#include <linux/signal.h>
+#include <linux/syscalls.h>
+#include <linux/times.h>
+#include <linux/utsname.h>
+#include <linux/mm.h>
+#include <linux/uio.h>
+#include <linux/poll.h>
+#include <linux/personality.h>
+#include <linux/stat.h>
+#include <linux/rwsem.h>
+#include <linux/compat.h>
+#include <linux/vfs.h>
+#include <linux/ptrace.h>
+#include <linux/highuid.h>
+#include <linux/sysctl.h>
+#include <linux/slab.h>
+#include <asm/types.h>
+#include <linux/atomic.h>
+#include <asm/syscalls_32.h>
+
+#define AA(__x)		((unsigned long)(__x))
+
+#if __BITS_PER_LONG == 32
+
+asmlinkage long sys32_truncate64(const char __user *filename,
+				 unsigned long offset_low,
+				 unsigned long offset_high)
+{
+	return sys_truncate64(filename,
+			      ((loff_t)offset_high << 32) | offset_low);
+}
+
+asmlinkage long sys32_ftruncate64(unsigned int fd, unsigned long offset_low,
+				  unsigned long offset_high)
+{
+	return sys_ftruncate64(fd, ((loff_t)offset_high << 32) | offset_low);
+}
+
+#ifdef CONFIG_MMU
+/*
+ * Linux/i386 didn't use to be able to handle more than
+ * 4 system call parameters, so these system calls used a memory
+ * block for parameter passing..
+ */
+
+struct mmap_arg_struct32 {
+	unsigned int addr;
+	unsigned int len;
+	unsigned int prot;
+	unsigned int flags;
+	unsigned int fd;
+	unsigned int offset;
+};
+
+asmlinkage long sys32_mmap(struct mmap_arg_struct32 __user *arg)
+{
+	struct mmap_arg_struct32 a;
+
+	if (copy_from_user(&a, arg, sizeof(a)))
+		return -EFAULT;
+
+	if (a.offset & ~PAGE_MASK)
+		return -EINVAL;
+
+	return sys_mmap_pgoff(a.addr, a.len, a.prot, a.flags, a.fd,
+			      a.offset >> PAGE_SHIFT);
+}
+#endif
+
+asmlinkage long sys32_wait4(pid_t pid, unsigned int __user *stat_addr,
+			    int options, struct rusage __user *ru)
+{
+	return sys_wait4(pid, stat_addr, options, ru);
+}
+
+asmlinkage long sys32_pread64(unsigned int fd, char __user *ubuf, u32 count,
+			      u32 poslo, u32 poshi)
+{
+	return sys_pread64(fd, ubuf, count,
+			   ((loff_t)AA(poshi) << 32) | AA(poslo));
+}
+
+asmlinkage long sys32_pwrite64(unsigned int fd, const char __user *ubuf,
+			       u32 count, u32 poslo, u32 poshi)
+{
+	return sys_pwrite64(fd, ubuf, count,
+			    ((loff_t)AA(poshi) << 32) | AA(poslo));
+}
+
+/*
+ * Some system calls that need sign extended arguments. This could be
+ * done by a generic wrapper.
+ */
+long sys32_fadvise64_64(int fd, __u32 offset_low, __u32 offset_high,
+			__u32 len_low, __u32 len_high, int advice)
+{
+	return sys_fadvise64_64(fd, (((u64)offset_high) << 32) | offset_low,
+				(((u64)len_high) << 32) | len_low, advice);
+}
+
+asmlinkage ssize_t sys32_readahead(int fd, unsigned int off_lo,
+				   unsigned int off_hi, size_t count)
+{
+	return sys_readahead(fd, ((u64)off_hi << 32) | off_lo, count);
+}
+
+asmlinkage long sys32_sync_file_range(int fd, unsigned int off_low,
+				      unsigned int off_hi, unsigned int n_low,
+				      unsigned int n_hi, unsigned int flags)
+{
+	return sys_sync_file_range(fd, ((u64)off_hi << 32) | off_low,
+				   ((u64)n_hi << 32) | n_low, flags);
+}
+
+asmlinkage long sys32_sync_file_range2(int fd, unsigned int flags,
+				       unsigned int off_low,
+				       unsigned int off_hi, unsigned int n_low,
+				       unsigned int n_hi)
+{
+	return sys_sync_file_range(fd, ((u64)off_hi << 32) | off_low,
+				   ((u64)n_hi << 32) | n_low, flags);
+}
+
+asmlinkage long sys32_fallocate(int fd, int mode, unsigned int offset_lo,
+				unsigned int offset_hi, unsigned int len_lo,
+				unsigned int len_hi)
+{
+	return sys_fallocate(fd, mode, ((u64)offset_hi << 32) | offset_lo,
+			     ((u64)len_hi << 32) | len_lo);
+}
+
+#endif
diff --git a/arch/um/lkl/scripts/headers_install.py b/arch/um/lkl/scripts/headers_install.py
new file mode 100755
index 000000000000..30101bd1d4bf
--- /dev/null
+++ b/arch/um/lkl/scripts/headers_install.py
@@ -0,0 +1,196 @@
+#!/usr/bin/env python
+# SPDX-License-Identifier: GPL-2.0
+import re, os, sys, argparse, multiprocessing, fnmatch
+
+srctree = os.environ["srctree"]
+objtree = os.environ["objtree"]
+install_hdr_path = os.environ["INSTALL_HDR_PATH"]
+header_paths = [ install_hdr_path + "/include/",
+                 "arch/um/lkl/include/generated/uapi/", "include/generated/" ]
+
+headers = set()
+includes = set()
+
+def relpath2abspath(relpath):
+    if "generated" in relpath:
+        return objtree + "/" + relpath
+    else:
+        return objtree + "/" + relpath
+
+def find_headers(path):
+    headers.add(path)
+    f = open(relpath2abspath(path))
+    for l in f.readlines():
+        m = re.search("#include <(.*)>", l)
+        try:
+            i = m.group(1)
+            for p in header_paths:
+                if os.access(relpath2abspath(p + i), os.R_OK):
+                    if p + i not in headers:
+                        includes.add(i)
+                        headers.add(p + i)
+                        find_headers(p + i)
+        except:
+            pass
+    f.close()
+
+def has_lkl_prefix(w):
+  return w.startswith("lkl") or w.startswith("_lkl") or w.startswith("__lkl") \
+         or w.startswith("LKL") or w.startswith("_LKL") or w.startswith("__LKL")
+
+def find_symbols(regexp, store):
+    for h in headers:
+        f = open(h)
+        for l in f.readlines():
+            m = regexp.search(l)
+            if not m:
+                continue
+            for e in reversed(m.groups()):
+                if e:
+                    if not has_lkl_prefix(e):
+                        store.add(e)
+                    break
+        f.close()
+
+def find_ml_symbols(regexp, store):
+    for h in headers:
+        for i in regexp.finditer(open(h).read()):
+            for j in reversed(i.groups()):
+                if j:
+                    if not has_lkl_prefix(j):
+                        store.add(j)
+                    break
+
+def find_enums(block_regexp, symbol_regexp, store):
+    for h in headers:
+        # remove comments
+        content = re.sub(re.compile("(\/\*(\*(?!\/)|[^*])*\*\/)", re.S|re.M), " ", open(h).read())
+        # remove preprocesor lines
+        clean_content = ""
+        for l in content.split("\n"):
+            if re.match("\s*#", l):
+                continue
+            clean_content += l + "\n"
+        for i in block_regexp.finditer(clean_content):
+            for j in reversed(i.groups()):
+                if j:
+                    for k in symbol_regexp.finditer(j):
+                        for l in k.groups():
+                            if l:
+                                if not has_lkl_prefix(l):
+                                    store.add(l)
+                                break
+
+def lkl_prefix(w):
+    r = ""
+
+    if w.startswith("__"):
+        r = "__"
+    elif w.startswith("_"):
+        r = "_"
+
+    if w.isupper():
+        r += "LKL"
+    else:
+        r += "lkl"
+
+    if not w.startswith("_"):
+        r += "_"
+
+    r += w
+
+    return r
+
+def replace(h):
+    content = open(h).read()
+    for i in includes:
+        search_str = "(#[ \t]*include[ \t]*[<\"][ \t]*)" + i + "([ \t]*[>\"])"
+        replace_str = "\\1" + "lkl/" + i + "\\2"
+        content = re.sub(search_str, replace_str, content)
+    tmp = ""
+    for w in re.split("(\W+)", content):
+        if w in defines:
+            w = lkl_prefix(w)
+        tmp += w
+    content = tmp
+    for s in structs:
+        search_str = "(\W?struct\s+)" + s + "(\W)"
+        replace_str = "\\1" + lkl_prefix(s) + "\\2"
+        content = re.sub(search_str, replace_str, content, flags = re.MULTILINE)
+    for s in unions:
+        search_str = "(\W?union\s+)" + s + "(\W)"
+        replace_str = "\\1" + lkl_prefix(s) + "\\2"
+        content = re.sub(search_str, replace_str, content, flags = re.MULTILINE)
+    open(h, 'w').write(content)
+
+parser = argparse.ArgumentParser(description='install lkl headers')
+parser.add_argument('path', help='path to install to', )
+parser.add_argument('-j', '--jobs', help='number of parallel jobs', default=1, type=int)
+args = parser.parse_args()
+
+find_headers(install_hdr_path + "/include/asm/syscalls.h")
+headers.add(install_hdr_path + "/include/asm/host_ops.h")
+
+if 'LKL_INSTALL_ADDITIONAL_HEADERS' in os.environ:
+    with open(os.environ['LKL_INSTALL_ADDITIONAL_HEADERS'], 'rU') as f:
+        for line in f.readlines():
+            line = line.split('#', 1)[0].strip()
+            if line != '':
+                headers.add(line)
+
+new_headers = set()
+
+for h in headers:
+    dir = os.path.dirname(h)
+    out_dir = args.path + "/" + re.sub("(arch/um/lkl/include/generated/uapi/|include/generated/uapi/|include/generated|" + install_hdr_path + "/include/)(.*)", "lkl/\\2", dir)
+    try:
+        os.makedirs(out_dir)
+    except:
+        pass
+    print("  INSTALL\t%s" % (out_dir + "/" + os.path.basename(h)))
+    os.system(srctree+"/scripts/headers_install.sh %s %s" % (os.path.abspath(h),
+                                                       out_dir + "/" + os.path.basename(h)))
+    new_headers.add(out_dir + "/" + os.path.basename(h))
+
+headers = new_headers
+
+defines = set()
+structs = set()
+unions = set()
+
+p = re.compile("#[ \t]*define[ \t]*(\w+)")
+find_symbols(p, defines)
+p = re.compile("typedef.*(\(\*(\w+)\)\(.*\)\s*|\W+(\w+)\s*|\s+(\w+)\(.*\)\s*);")
+find_symbols(p, defines)
+p = re.compile("typedef\s+(struct|union)\s+\w*\s*{[^\\{\}]*}\W*(\w+)\s*;", re.M|re.S)
+find_ml_symbols(p, defines)
+defines.add("siginfo_t")
+defines.add("sigevent_t")
+p = re.compile("struct\s+(\w+)\s*\{")
+find_symbols(p, structs)
+structs.add("iovec")
+p = re.compile("union\s+(\w+)\s*\{")
+find_symbols(p, unions)
+p = re.compile("static\s+__inline__(\s+\w+)+\s+(\w+)\([^)]*\)\s")
+find_symbols(p, defines)
+p = re.compile("static\s+__always_inline(\s+\w+)+\s+(\w+)\([^)]*\)\s")
+find_symbols(p, defines)
+p = re.compile("enum\s+(\w*)\s*{([^}]*)}", re.M|re.S)
+q = re.compile("(\w+)\s*(,|=[^,]*|$)", re.M|re.S)
+find_enums(p, q, defines)
+
+# needed for i386
+defines.add("__NR_stime")
+
+def process_header(h):
+    print("  REPLACE\t%s" % (out_dir + "/" + os.path.basename(h)))
+    replace(h)
+
+p = multiprocessing.Pool(args.jobs)
+try:
+    p.map_async(process_header, headers).wait(999999)
+    p.close()
+except:
+    p.terminate()
+finally:
+    p.join()
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v3 09/26] um lkl: timers, time and delay support
  2020-02-05  7:30 [RFC v3 00/26] Unifying LKL into UML Hajime Tazaki
@ 2020-02-05  7:30   ` Hajime Tazaki
       [not found] ` <cover.1580882335.git.thehajime-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
                     ` (25 subsequent siblings)
  26 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05  7:30 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Michael Zimmermann, Hajime Tazaki

From: Octavian Purdila <tavi.purdila@gmail.com>

Clockevent driver based on host timer operations and clocksource
driver and udelay support based on host time operations.

Cc: Michael Zimmermann <sigmaepsilon92@gmail.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/lkl/include/uapi/asm/host_ops.h |  13 +++
 arch/um/lkl/kernel/time.c               | 145 ++++++++++++++++++++++++
 2 files changed, 158 insertions(+)
 create mode 100644 arch/um/lkl/kernel/time.c

diff --git a/arch/um/lkl/include/uapi/asm/host_ops.h b/arch/um/lkl/include/uapi/asm/host_ops.h
index 1c839d7139f8..c9f77dd7fbe7 100644
--- a/arch/um/lkl/include/uapi/asm/host_ops.h
+++ b/arch/um/lkl/include/uapi/asm/host_ops.h
@@ -48,6 +48,13 @@ struct lkl_jmp_buf {
  * @mem_alloc - allocate memory
  * @mem_free - free memory
  *
+ * @timer_create - allocate a host timer that runs fn(arg) when the timer
+ * fires.
+ * @timer_free - disarms and free the timer
+ * @timer_set_oneshot - arm the timer to fire once, after delta ns.
+ * @timer_set_periodic - arm the timer to fire periodically, with a period of
+ * delta ns.
+ *
  * @gettid - returns the host thread id of the caller, which need not
  * be the same as the handle returned by thread_create
  *
@@ -88,6 +95,12 @@ struct lkl_host_operations {
 	void *(*mem_alloc)(unsigned long mem);
 	void (*mem_free)(void *mem);
 
+	unsigned long long (*time)(void);
+
+	void *(*timer_alloc)(void (*fn)(void *), void *arg);
+	int (*timer_set_oneshot)(void *timer, unsigned long delta);
+	void (*timer_free)(void *timer);
+
 	long (*gettid)(void);
 
 	void (*jmp_buf_set)(struct lkl_jmp_buf *jmpb, void (*f)(void));
diff --git a/arch/um/lkl/kernel/time.c b/arch/um/lkl/kernel/time.c
new file mode 100644
index 000000000000..b8320e1bfa53
--- /dev/null
+++ b/arch/um/lkl/kernel/time.c
@@ -0,0 +1,145 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/clocksource.h>
+#include <linux/clockchips.h>
+#include <linux/jiffies.h>
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/interrupt.h>
+#include <linux/irq.h>
+#include <asm/host_ops.h>
+
+static unsigned long long boot_time;
+
+void __ndelay(unsigned long nsecs)
+{
+	unsigned long long start = lkl_ops->time();
+
+	while (lkl_ops->time() < start + nsecs)
+		;
+}
+
+void __udelay(unsigned long usecs)
+{
+	__ndelay(usecs * NSEC_PER_USEC);
+}
+
+void __const_udelay(unsigned long xloops)
+{
+	__udelay(xloops / 0x10c7ul);
+}
+
+void calibrate_delay(void)
+{
+}
+
+void read_persistent_clock(struct timespec *ts)
+{
+	*ts = ns_to_timespec(lkl_ops->time());
+}
+
+/*
+ * Scheduler clock - returns current time in nanosec units.
+ *
+ */
+unsigned long long sched_clock(void)
+{
+	if (!boot_time)
+		return 0;
+
+	return lkl_ops->time() - boot_time;
+}
+
+static u64 clock_read(struct clocksource *cs)
+{
+	return lkl_ops->time();
+}
+
+static struct clocksource clocksource = {
+	.name	= "lkl",
+	.rating = 499,
+	.read	= clock_read,
+	.flags	= CLOCK_SOURCE_IS_CONTINUOUS,
+	.mask	= CLOCKSOURCE_MASK(64),
+};
+
+static void *timer;
+
+static int timer_irq;
+
+static void timer_fn(void *arg)
+{
+	lkl_trigger_irq(timer_irq);
+}
+
+static int clockevent_set_state_shutdown(struct clock_event_device *evt)
+{
+	if (timer) {
+		lkl_ops->timer_free(timer);
+		timer = NULL;
+	}
+
+	return 0;
+}
+
+static int clockevent_set_state_oneshot(struct clock_event_device *evt)
+{
+	timer = lkl_ops->timer_alloc(timer_fn, NULL);
+	if (!timer)
+		return -ENOMEM;
+
+	return 0;
+}
+
+static irqreturn_t timer_irq_handler(int irq, void *dev_id)
+{
+	struct clock_event_device *dev = (struct clock_event_device *)dev_id;
+
+	dev->event_handler(dev);
+
+	return IRQ_HANDLED;
+}
+
+static int clockevent_next_event(unsigned long ns,
+				 struct clock_event_device *evt)
+{
+	return lkl_ops->timer_set_oneshot(timer, ns);
+}
+
+static struct clock_event_device clockevent = {
+	.name			= "lkl",
+	.features		= CLOCK_EVT_FEAT_ONESHOT,
+	.set_state_oneshot	= clockevent_set_state_oneshot,
+	.set_next_event		= clockevent_next_event,
+	.set_state_shutdown	= clockevent_set_state_shutdown,
+};
+
+static struct irqaction irq0  = {
+	.handler	= timer_irq_handler,
+	.flags		= IRQF_NOBALANCING | IRQF_TIMER,
+	.dev_id		= &clockevent,
+	.name		= "timer"
+};
+
+void __init time_init(void)
+{
+	int ret;
+
+	if (!lkl_ops->timer_alloc || !lkl_ops->timer_free ||
+	    !lkl_ops->timer_set_oneshot || !lkl_ops->time) {
+		pr_err("lkl: no time or timer support provided by host\n");
+		return;
+	}
+
+	timer_irq = lkl_get_free_irq("timer");
+	setup_irq(timer_irq, &irq0);
+
+	ret = clocksource_register_khz(&clocksource, 1000000);
+	if (ret)
+		pr_err("lkl: unable to register clocksource\n");
+
+	clockevents_config_and_register(&clockevent, NSEC_PER_SEC, 1,
+					ULONG_MAX);
+
+	boot_time = lkl_ops->time();
+	pr_info("lkl: time and timers initialized (irq%d)\n", timer_irq);
+}
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v3 09/26] um lkl: timers, time and delay support
@ 2020-02-05  7:30   ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05  7:30 UTC (permalink / raw)
  To: linux-um
  Cc: linux-arch, Octavian Purdila, Akira Moroo, linux-kernel-library,
	Michael Zimmermann, Hajime Tazaki

From: Octavian Purdila <tavi.purdila@gmail.com>

Clockevent driver based on host timer operations and clocksource
driver and udelay support based on host time operations.

Cc: Michael Zimmermann <sigmaepsilon92@gmail.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/lkl/include/uapi/asm/host_ops.h |  13 +++
 arch/um/lkl/kernel/time.c               | 145 ++++++++++++++++++++++++
 2 files changed, 158 insertions(+)
 create mode 100644 arch/um/lkl/kernel/time.c

diff --git a/arch/um/lkl/include/uapi/asm/host_ops.h b/arch/um/lkl/include/uapi/asm/host_ops.h
index 1c839d7139f8..c9f77dd7fbe7 100644
--- a/arch/um/lkl/include/uapi/asm/host_ops.h
+++ b/arch/um/lkl/include/uapi/asm/host_ops.h
@@ -48,6 +48,13 @@ struct lkl_jmp_buf {
  * @mem_alloc - allocate memory
  * @mem_free - free memory
  *
+ * @timer_create - allocate a host timer that runs fn(arg) when the timer
+ * fires.
+ * @timer_free - disarms and free the timer
+ * @timer_set_oneshot - arm the timer to fire once, after delta ns.
+ * @timer_set_periodic - arm the timer to fire periodically, with a period of
+ * delta ns.
+ *
  * @gettid - returns the host thread id of the caller, which need not
  * be the same as the handle returned by thread_create
  *
@@ -88,6 +95,12 @@ struct lkl_host_operations {
 	void *(*mem_alloc)(unsigned long mem);
 	void (*mem_free)(void *mem);
 
+	unsigned long long (*time)(void);
+
+	void *(*timer_alloc)(void (*fn)(void *), void *arg);
+	int (*timer_set_oneshot)(void *timer, unsigned long delta);
+	void (*timer_free)(void *timer);
+
 	long (*gettid)(void);
 
 	void (*jmp_buf_set)(struct lkl_jmp_buf *jmpb, void (*f)(void));
diff --git a/arch/um/lkl/kernel/time.c b/arch/um/lkl/kernel/time.c
new file mode 100644
index 000000000000..b8320e1bfa53
--- /dev/null
+++ b/arch/um/lkl/kernel/time.c
@@ -0,0 +1,145 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/clocksource.h>
+#include <linux/clockchips.h>
+#include <linux/jiffies.h>
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/interrupt.h>
+#include <linux/irq.h>
+#include <asm/host_ops.h>
+
+static unsigned long long boot_time;
+
+void __ndelay(unsigned long nsecs)
+{
+	unsigned long long start = lkl_ops->time();
+
+	while (lkl_ops->time() < start + nsecs)
+		;
+}
+
+void __udelay(unsigned long usecs)
+{
+	__ndelay(usecs * NSEC_PER_USEC);
+}
+
+void __const_udelay(unsigned long xloops)
+{
+	__udelay(xloops / 0x10c7ul);
+}
+
+void calibrate_delay(void)
+{
+}
+
+void read_persistent_clock(struct timespec *ts)
+{
+	*ts = ns_to_timespec(lkl_ops->time());
+}
+
+/*
+ * Scheduler clock - returns current time in nanosec units.
+ *
+ */
+unsigned long long sched_clock(void)
+{
+	if (!boot_time)
+		return 0;
+
+	return lkl_ops->time() - boot_time;
+}
+
+static u64 clock_read(struct clocksource *cs)
+{
+	return lkl_ops->time();
+}
+
+static struct clocksource clocksource = {
+	.name	= "lkl",
+	.rating = 499,
+	.read	= clock_read,
+	.flags	= CLOCK_SOURCE_IS_CONTINUOUS,
+	.mask	= CLOCKSOURCE_MASK(64),
+};
+
+static void *timer;
+
+static int timer_irq;
+
+static void timer_fn(void *arg)
+{
+	lkl_trigger_irq(timer_irq);
+}
+
+static int clockevent_set_state_shutdown(struct clock_event_device *evt)
+{
+	if (timer) {
+		lkl_ops->timer_free(timer);
+		timer = NULL;
+	}
+
+	return 0;
+}
+
+static int clockevent_set_state_oneshot(struct clock_event_device *evt)
+{
+	timer = lkl_ops->timer_alloc(timer_fn, NULL);
+	if (!timer)
+		return -ENOMEM;
+
+	return 0;
+}
+
+static irqreturn_t timer_irq_handler(int irq, void *dev_id)
+{
+	struct clock_event_device *dev = (struct clock_event_device *)dev_id;
+
+	dev->event_handler(dev);
+
+	return IRQ_HANDLED;
+}
+
+static int clockevent_next_event(unsigned long ns,
+				 struct clock_event_device *evt)
+{
+	return lkl_ops->timer_set_oneshot(timer, ns);
+}
+
+static struct clock_event_device clockevent = {
+	.name			= "lkl",
+	.features		= CLOCK_EVT_FEAT_ONESHOT,
+	.set_state_oneshot	= clockevent_set_state_oneshot,
+	.set_next_event		= clockevent_next_event,
+	.set_state_shutdown	= clockevent_set_state_shutdown,
+};
+
+static struct irqaction irq0  = {
+	.handler	= timer_irq_handler,
+	.flags		= IRQF_NOBALANCING | IRQF_TIMER,
+	.dev_id		= &clockevent,
+	.name		= "timer"
+};
+
+void __init time_init(void)
+{
+	int ret;
+
+	if (!lkl_ops->timer_alloc || !lkl_ops->timer_free ||
+	    !lkl_ops->timer_set_oneshot || !lkl_ops->time) {
+		pr_err("lkl: no time or timer support provided by host\n");
+		return;
+	}
+
+	timer_irq = lkl_get_free_irq("timer");
+	setup_irq(timer_irq, &irq0);
+
+	ret = clocksource_register_khz(&clocksource, 1000000);
+	if (ret)
+		pr_err("lkl: unable to register clocksource\n");
+
+	clockevents_config_and_register(&clockevent, NSEC_PER_SEC, 1,
+					ULONG_MAX);
+
+	boot_time = lkl_ops->time();
+	pr_info("lkl: time and timers initialized (irq%d)\n", timer_irq);
+}
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v3 10/26] um lkl: basic kernel console support
  2020-02-05  7:30 [RFC v3 00/26] Unifying LKL into UML Hajime Tazaki
@ 2020-02-05  7:30   ` Hajime Tazaki
       [not found] ` <cover.1580882335.git.thehajime-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
                     ` (25 subsequent siblings)
  26 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05  7:30 UTC (permalink / raw)
  To: linux-um; +Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch

From: Octavian Purdila <tavi.purdila@gmail.com>

Write operations are deferred to the host print operation.

Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/lkl/include/uapi/asm/host_ops.h |  4 +++
 arch/um/lkl/kernel/console.c            | 42 +++++++++++++++++++++++++
 2 files changed, 46 insertions(+)
 create mode 100644 arch/um/lkl/kernel/console.c

diff --git a/arch/um/lkl/include/uapi/asm/host_ops.h b/arch/um/lkl/include/uapi/asm/host_ops.h
index c9f77dd7fbe7..986340ba9d8d 100644
--- a/arch/um/lkl/include/uapi/asm/host_ops.h
+++ b/arch/um/lkl/include/uapi/asm/host_ops.h
@@ -17,6 +17,8 @@ struct lkl_jmp_buf {
  * These operations must be provided by a host library or by the application
  * itself.
  *
+ * @print - optional operation that receives console messages
+ *
  * @sem_alloc - allocate a host semaphore an initialize it to count
  * @sem_free - free a host semaphore
  * @sem_up - perform an up operation on the semaphore
@@ -70,6 +72,8 @@ struct lkl_jmp_buf {
  * @jmp_buf_longjmp - perform a jump back to the saved jump buffer
  */
 struct lkl_host_operations {
+	void (*print)(const char *str, int len);
+
 	struct lkl_sem *(*sem_alloc)(int count);
 	void (*sem_free)(struct lkl_sem *sem);
 	void (*sem_up)(struct lkl_sem *sem);
diff --git a/arch/um/lkl/kernel/console.c b/arch/um/lkl/kernel/console.c
new file mode 100644
index 000000000000..54d7f756c6da
--- /dev/null
+++ b/arch/um/lkl/kernel/console.c
@@ -0,0 +1,42 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/console.h>
+#include <asm/host_ops.h>
+
+static void console_write(struct console *con, const char *str,
+			  unsigned int len)
+{
+	if (lkl_ops->print)
+		lkl_ops->print(str, len);
+}
+
+#ifdef CONFIG_LKL_EARLY_CONSOLE
+static struct console lkl_boot_console = {
+	.name	= "lkl_boot_console",
+	.write	= console_write,
+	.flags	= CON_PRINTBUFFER | CON_BOOT,
+	.index	= -1,
+};
+
+int __init lkl_boot_console_init(void)
+{
+	register_console(&lkl_boot_console);
+	return 0;
+}
+early_initcall(lkl_boot_console_init);
+#endif
+
+static struct console lkl_console = {
+	.name	= "lkl_console",
+	.write	= console_write,
+	.flags	= CON_PRINTBUFFER,
+	.index	= -1,
+};
+
+static int __init lkl_console_init(void)
+{
+	register_console(&lkl_console);
+	return 0;
+}
+core_initcall(lkl_console_init);
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v3 10/26] um lkl: basic kernel console support
@ 2020-02-05  7:30   ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05  7:30 UTC (permalink / raw)
  To: linux-um; +Cc: Octavian Purdila, linux-kernel-library, linux-arch, Akira Moroo

From: Octavian Purdila <tavi.purdila@gmail.com>

Write operations are deferred to the host print operation.

Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/lkl/include/uapi/asm/host_ops.h |  4 +++
 arch/um/lkl/kernel/console.c            | 42 +++++++++++++++++++++++++
 2 files changed, 46 insertions(+)
 create mode 100644 arch/um/lkl/kernel/console.c

diff --git a/arch/um/lkl/include/uapi/asm/host_ops.h b/arch/um/lkl/include/uapi/asm/host_ops.h
index c9f77dd7fbe7..986340ba9d8d 100644
--- a/arch/um/lkl/include/uapi/asm/host_ops.h
+++ b/arch/um/lkl/include/uapi/asm/host_ops.h
@@ -17,6 +17,8 @@ struct lkl_jmp_buf {
  * These operations must be provided by a host library or by the application
  * itself.
  *
+ * @print - optional operation that receives console messages
+ *
  * @sem_alloc - allocate a host semaphore an initialize it to count
  * @sem_free - free a host semaphore
  * @sem_up - perform an up operation on the semaphore
@@ -70,6 +72,8 @@ struct lkl_jmp_buf {
  * @jmp_buf_longjmp - perform a jump back to the saved jump buffer
  */
 struct lkl_host_operations {
+	void (*print)(const char *str, int len);
+
 	struct lkl_sem *(*sem_alloc)(int count);
 	void (*sem_free)(struct lkl_sem *sem);
 	void (*sem_up)(struct lkl_sem *sem);
diff --git a/arch/um/lkl/kernel/console.c b/arch/um/lkl/kernel/console.c
new file mode 100644
index 000000000000..54d7f756c6da
--- /dev/null
+++ b/arch/um/lkl/kernel/console.c
@@ -0,0 +1,42 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/console.h>
+#include <asm/host_ops.h>
+
+static void console_write(struct console *con, const char *str,
+			  unsigned int len)
+{
+	if (lkl_ops->print)
+		lkl_ops->print(str, len);
+}
+
+#ifdef CONFIG_LKL_EARLY_CONSOLE
+static struct console lkl_boot_console = {
+	.name	= "lkl_boot_console",
+	.write	= console_write,
+	.flags	= CON_PRINTBUFFER | CON_BOOT,
+	.index	= -1,
+};
+
+int __init lkl_boot_console_init(void)
+{
+	register_console(&lkl_boot_console);
+	return 0;
+}
+early_initcall(lkl_boot_console_init);
+#endif
+
+static struct console lkl_console = {
+	.name	= "lkl_console",
+	.write	= console_write,
+	.flags	= CON_PRINTBUFFER,
+	.index	= -1,
+};
+
+static int __init lkl_console_init(void)
+{
+	register_console(&lkl_console);
+	return 0;
+}
+core_initcall(lkl_console_init);
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v3 11/26] um lkl: initialization and cleanup
  2020-02-05  7:30 [RFC v3 00/26] Unifying LKL into UML Hajime Tazaki
@ 2020-02-05  7:30   ` Hajime Tazaki
       [not found] ` <cover.1580882335.git.thehajime-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
                     ` (25 subsequent siblings)
  26 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05  7:30 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Michael Zimmermann, Patrick Collins, Yuan Liu, Hajime Tazaki

From: Octavian Purdila <tavi.purdila@gmail.com>

Add the lkl_start_kernel and lkl_sys_halt APIs that start and
respectively stops the Linux kernel.

lkl_start_kernel creates a separate threads that will run the initial
and idle kernel thread. It waits for the kernel to complete
initialization before returning, to avoid races with system calls
issues by the host application.

During the setup phase, we create "/init" in initial ramfs root
filesystem to avoid mounting the "real" rootfs since ramfs is good
enough for now.

lkl_sys_halt will shutdown the kernel, terminate all threads and
free all host resources used by the kernel before returning.

This patch also introduces idle CPU handling since it is closely
related to the shutdown process. A host semaphore is used to wait for
new interrupts when the kernel switches the CPU to idle to avoid
wasting host CPU cycles. When the kernel is shutdown we terminate the
idle thread at the first CPU idle event.

Cc: Michael Zimmermann <sigmaepsilon92@gmail.com>
Cc: Patrick Collins <pscollins@google.com>
Cc: Yuan Liu <liuyuan@google.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/lkl/include/asm/setup.h         |   7 +
 arch/um/lkl/include/uapi/asm/host_ops.h |  20 +++
 arch/um/lkl/kernel/setup.c              | 189 ++++++++++++++++++++++++
 3 files changed, 216 insertions(+)
 create mode 100644 arch/um/lkl/include/asm/setup.h
 create mode 100644 arch/um/lkl/kernel/setup.c

diff --git a/arch/um/lkl/include/asm/setup.h b/arch/um/lkl/include/asm/setup.h
new file mode 100644
index 000000000000..b40955208cc6
--- /dev/null
+++ b/arch/um/lkl/include/asm/setup.h
@@ -0,0 +1,7 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_SETUP_H
+#define _ASM_LKL_SETUP_H
+
+#define COMMAND_LINE_SIZE 4096
+
+#endif
diff --git a/arch/um/lkl/include/uapi/asm/host_ops.h b/arch/um/lkl/include/uapi/asm/host_ops.h
index 986340ba9d8d..fe4382c3050a 100644
--- a/arch/um/lkl/include/uapi/asm/host_ops.h
+++ b/arch/um/lkl/include/uapi/asm/host_ops.h
@@ -19,6 +19,8 @@ struct lkl_jmp_buf {
  *
  * @print - optional operation that receives console messages
  *
+ * @panic - called during a kernel panic
+ *
  * @sem_alloc - allocate a host semaphore an initialize it to count
  * @sem_free - free a host semaphore
  * @sem_up - perform an up operation on the semaphore
@@ -73,6 +75,7 @@ struct lkl_jmp_buf {
  */
 struct lkl_host_operations {
 	void (*print)(const char *str, int len);
+	void (*panic)(void);
 
 	struct lkl_sem *(*sem_alloc)(int count);
 	void (*sem_free)(struct lkl_sem *sem);
@@ -111,6 +114,23 @@ struct lkl_host_operations {
 	void (*jmp_buf_longjmp)(struct lkl_jmp_buf *jmpb, int val);
 };
 
+/**
+ * lkl_start_kernel - registers the host operations and starts the kernel
+ *
+ * The function returns only after the kernel is shutdown with lkl_sys_halt.
+ *
+ * @lkl_ops - pointer to host operations
+ * @cmd_line - format for command line string that is going to be used to
+ * generate the Linux kernel command line
+ */
+int lkl_start_kernel(struct lkl_host_operations *lkl_ops, const char *cmd_line,
+		     ...);
+
+/**
+ * lkl_is_running - returns 1 if the kernel is currently running
+ */
+int lkl_is_running(void);
+
 int lkl_printf(const char *fmt, ...);
 void lkl_bug(const char *fmt, ...);
 
diff --git a/arch/um/lkl/kernel/setup.c b/arch/um/lkl/kernel/setup.c
new file mode 100644
index 000000000000..36c199d3aa22
--- /dev/null
+++ b/arch/um/lkl/kernel/setup.c
@@ -0,0 +1,189 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/binfmts.h>
+#include <linux/init.h>
+#include <linux/init_task.h>
+#include <linux/personality.h>
+#include <linux/reboot.h>
+#include <linux/fs.h>
+#include <linux/start_kernel.h>
+#include <linux/syscalls.h>
+#include <linux/tick.h>
+#include <asm/host_ops.h>
+#include <asm/irq.h>
+#include <asm/unistd.h>
+#include <asm/syscalls.h>
+#include <asm/cpu.h>
+
+struct lkl_host_operations *lkl_ops;
+static char cmd_line[COMMAND_LINE_SIZE];
+static void *init_sem;
+static int is_running;
+void (*pm_power_off)(void) = NULL;
+static unsigned long mem_size = 64 * 1024 * 1024;
+
+static long lkl_panic_blink(int state)
+{
+	lkl_ops->panic();
+	return 0;
+}
+
+static int __init setup_mem_size(char *str)
+{
+	mem_size = memparse(str, NULL);
+	return 0;
+}
+early_param("mem", setup_mem_size);
+
+void __init setup_arch(char **cl)
+{
+	*cl = cmd_line;
+	panic_blink = lkl_panic_blink;
+	parse_early_param();
+	bootmem_init(mem_size);
+}
+
+static void __init lkl_run_kernel(void *arg)
+{
+	threads_init();
+	lkl_cpu_get();
+	start_kernel();
+}
+
+int __init lkl_start_kernel(struct lkl_host_operations *ops, const char *fmt,
+			    ...)
+{
+	va_list ap;
+	int ret;
+
+	lkl_ops = ops;
+
+	va_start(ap, fmt);
+	ret = vsnprintf(boot_command_line, COMMAND_LINE_SIZE, fmt, ap);
+	va_end(ap);
+
+	memcpy(cmd_line, boot_command_line, COMMAND_LINE_SIZE);
+
+	init_sem = lkl_ops->sem_alloc(0);
+	if (!init_sem)
+		return -ENOMEM;
+
+	ret = lkl_cpu_init();
+	if (ret)
+		goto out_free_init_sem;
+
+	ret = lkl_ops->thread_create(lkl_run_kernel, NULL);
+	if (!ret) {
+		ret = -ENOMEM;
+		goto out_free_init_sem;
+	}
+
+	lkl_ops->sem_down(init_sem);
+	lkl_ops->sem_free(init_sem);
+	current_thread_info()->tid = lkl_ops->thread_self();
+	lkl_cpu_change_owner(current_thread_info()->tid);
+
+	lkl_cpu_put();
+	is_running = 1;
+
+	return 0;
+
+out_free_init_sem:
+	lkl_ops->sem_free(init_sem);
+
+	return ret;
+}
+
+int lkl_is_running(void)
+{
+	return is_running;
+}
+
+void machine_halt(void)
+{
+	lkl_cpu_shutdown();
+}
+
+void machine_power_off(void)
+{
+	machine_halt();
+}
+
+void machine_restart(char *unused)
+{
+	machine_halt();
+}
+
+long lkl_sys_halt(void)
+{
+	long err;
+	long params[6] = {
+		LINUX_REBOOT_MAGIC1,
+		LINUX_REBOOT_MAGIC2,
+		LINUX_REBOOT_CMD_RESTART,
+	};
+
+	err = lkl_syscall(__NR_reboot, params);
+	if (err < 0)
+		return err;
+
+	is_running = false;
+
+	lkl_cpu_wait_shutdown();
+
+	syscalls_cleanup();
+	threads_cleanup();
+	/* Shutdown the clockevents source. */
+	tick_suspend_local();
+	free_mem();
+	lkl_ops->thread_join(current_thread_info()->tid);
+
+	return 0;
+}
+
+static int lkl_run_init(struct linux_binprm *bprm);
+
+static struct linux_binfmt lkl_run_init_binfmt = {
+	.module		= THIS_MODULE,
+	.load_binary	= lkl_run_init,
+};
+
+static int lkl_run_init(struct linux_binprm *bprm)
+{
+	int ret;
+
+	if (strcmp("/init", bprm->filename) != 0)
+		return -EINVAL;
+
+	ret = flush_old_exec(bprm);
+	if (ret)
+		return ret;
+	set_personality(PER_LINUX);
+	setup_new_exec(bprm);
+	install_exec_creds(bprm);
+
+	set_binfmt(&lkl_run_init_binfmt);
+
+	init_pid_ns.child_reaper = NULL;
+
+	syscalls_init();
+
+	lkl_ops->sem_up(init_sem);
+	lkl_ops->thread_exit();
+
+	return 0;
+}
+
+/* skip mounting the "real" rootfs. ramfs is good enough. */
+static int __init fs_setup(void)
+{
+	int fd;
+
+	fd = sys_open("/init", O_CREAT, 0700);
+	WARN_ON(fd < 0);
+	sys_close(fd);
+
+	register_binfmt(&lkl_run_init_binfmt);
+
+	return 0;
+}
+late_initcall(fs_setup);
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v3 11/26] um lkl: initialization and cleanup
@ 2020-02-05  7:30   ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05  7:30 UTC (permalink / raw)
  To: linux-um
  Cc: linux-arch, Hajime Tazaki, Octavian Purdila, Akira Moroo,
	Patrick Collins, linux-kernel-library, Michael Zimmermann,
	Yuan Liu

From: Octavian Purdila <tavi.purdila@gmail.com>

Add the lkl_start_kernel and lkl_sys_halt APIs that start and
respectively stops the Linux kernel.

lkl_start_kernel creates a separate threads that will run the initial
and idle kernel thread. It waits for the kernel to complete
initialization before returning, to avoid races with system calls
issues by the host application.

During the setup phase, we create "/init" in initial ramfs root
filesystem to avoid mounting the "real" rootfs since ramfs is good
enough for now.

lkl_sys_halt will shutdown the kernel, terminate all threads and
free all host resources used by the kernel before returning.

This patch also introduces idle CPU handling since it is closely
related to the shutdown process. A host semaphore is used to wait for
new interrupts when the kernel switches the CPU to idle to avoid
wasting host CPU cycles. When the kernel is shutdown we terminate the
idle thread at the first CPU idle event.

Cc: Michael Zimmermann <sigmaepsilon92@gmail.com>
Cc: Patrick Collins <pscollins@google.com>
Cc: Yuan Liu <liuyuan@google.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/lkl/include/asm/setup.h         |   7 +
 arch/um/lkl/include/uapi/asm/host_ops.h |  20 +++
 arch/um/lkl/kernel/setup.c              | 189 ++++++++++++++++++++++++
 3 files changed, 216 insertions(+)
 create mode 100644 arch/um/lkl/include/asm/setup.h
 create mode 100644 arch/um/lkl/kernel/setup.c

diff --git a/arch/um/lkl/include/asm/setup.h b/arch/um/lkl/include/asm/setup.h
new file mode 100644
index 000000000000..b40955208cc6
--- /dev/null
+++ b/arch/um/lkl/include/asm/setup.h
@@ -0,0 +1,7 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_SETUP_H
+#define _ASM_LKL_SETUP_H
+
+#define COMMAND_LINE_SIZE 4096
+
+#endif
diff --git a/arch/um/lkl/include/uapi/asm/host_ops.h b/arch/um/lkl/include/uapi/asm/host_ops.h
index 986340ba9d8d..fe4382c3050a 100644
--- a/arch/um/lkl/include/uapi/asm/host_ops.h
+++ b/arch/um/lkl/include/uapi/asm/host_ops.h
@@ -19,6 +19,8 @@ struct lkl_jmp_buf {
  *
  * @print - optional operation that receives console messages
  *
+ * @panic - called during a kernel panic
+ *
  * @sem_alloc - allocate a host semaphore an initialize it to count
  * @sem_free - free a host semaphore
  * @sem_up - perform an up operation on the semaphore
@@ -73,6 +75,7 @@ struct lkl_jmp_buf {
  */
 struct lkl_host_operations {
 	void (*print)(const char *str, int len);
+	void (*panic)(void);
 
 	struct lkl_sem *(*sem_alloc)(int count);
 	void (*sem_free)(struct lkl_sem *sem);
@@ -111,6 +114,23 @@ struct lkl_host_operations {
 	void (*jmp_buf_longjmp)(struct lkl_jmp_buf *jmpb, int val);
 };
 
+/**
+ * lkl_start_kernel - registers the host operations and starts the kernel
+ *
+ * The function returns only after the kernel is shutdown with lkl_sys_halt.
+ *
+ * @lkl_ops - pointer to host operations
+ * @cmd_line - format for command line string that is going to be used to
+ * generate the Linux kernel command line
+ */
+int lkl_start_kernel(struct lkl_host_operations *lkl_ops, const char *cmd_line,
+		     ...);
+
+/**
+ * lkl_is_running - returns 1 if the kernel is currently running
+ */
+int lkl_is_running(void);
+
 int lkl_printf(const char *fmt, ...);
 void lkl_bug(const char *fmt, ...);
 
diff --git a/arch/um/lkl/kernel/setup.c b/arch/um/lkl/kernel/setup.c
new file mode 100644
index 000000000000..36c199d3aa22
--- /dev/null
+++ b/arch/um/lkl/kernel/setup.c
@@ -0,0 +1,189 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/binfmts.h>
+#include <linux/init.h>
+#include <linux/init_task.h>
+#include <linux/personality.h>
+#include <linux/reboot.h>
+#include <linux/fs.h>
+#include <linux/start_kernel.h>
+#include <linux/syscalls.h>
+#include <linux/tick.h>
+#include <asm/host_ops.h>
+#include <asm/irq.h>
+#include <asm/unistd.h>
+#include <asm/syscalls.h>
+#include <asm/cpu.h>
+
+struct lkl_host_operations *lkl_ops;
+static char cmd_line[COMMAND_LINE_SIZE];
+static void *init_sem;
+static int is_running;
+void (*pm_power_off)(void) = NULL;
+static unsigned long mem_size = 64 * 1024 * 1024;
+
+static long lkl_panic_blink(int state)
+{
+	lkl_ops->panic();
+	return 0;
+}
+
+static int __init setup_mem_size(char *str)
+{
+	mem_size = memparse(str, NULL);
+	return 0;
+}
+early_param("mem", setup_mem_size);
+
+void __init setup_arch(char **cl)
+{
+	*cl = cmd_line;
+	panic_blink = lkl_panic_blink;
+	parse_early_param();
+	bootmem_init(mem_size);
+}
+
+static void __init lkl_run_kernel(void *arg)
+{
+	threads_init();
+	lkl_cpu_get();
+	start_kernel();
+}
+
+int __init lkl_start_kernel(struct lkl_host_operations *ops, const char *fmt,
+			    ...)
+{
+	va_list ap;
+	int ret;
+
+	lkl_ops = ops;
+
+	va_start(ap, fmt);
+	ret = vsnprintf(boot_command_line, COMMAND_LINE_SIZE, fmt, ap);
+	va_end(ap);
+
+	memcpy(cmd_line, boot_command_line, COMMAND_LINE_SIZE);
+
+	init_sem = lkl_ops->sem_alloc(0);
+	if (!init_sem)
+		return -ENOMEM;
+
+	ret = lkl_cpu_init();
+	if (ret)
+		goto out_free_init_sem;
+
+	ret = lkl_ops->thread_create(lkl_run_kernel, NULL);
+	if (!ret) {
+		ret = -ENOMEM;
+		goto out_free_init_sem;
+	}
+
+	lkl_ops->sem_down(init_sem);
+	lkl_ops->sem_free(init_sem);
+	current_thread_info()->tid = lkl_ops->thread_self();
+	lkl_cpu_change_owner(current_thread_info()->tid);
+
+	lkl_cpu_put();
+	is_running = 1;
+
+	return 0;
+
+out_free_init_sem:
+	lkl_ops->sem_free(init_sem);
+
+	return ret;
+}
+
+int lkl_is_running(void)
+{
+	return is_running;
+}
+
+void machine_halt(void)
+{
+	lkl_cpu_shutdown();
+}
+
+void machine_power_off(void)
+{
+	machine_halt();
+}
+
+void machine_restart(char *unused)
+{
+	machine_halt();
+}
+
+long lkl_sys_halt(void)
+{
+	long err;
+	long params[6] = {
+		LINUX_REBOOT_MAGIC1,
+		LINUX_REBOOT_MAGIC2,
+		LINUX_REBOOT_CMD_RESTART,
+	};
+
+	err = lkl_syscall(__NR_reboot, params);
+	if (err < 0)
+		return err;
+
+	is_running = false;
+
+	lkl_cpu_wait_shutdown();
+
+	syscalls_cleanup();
+	threads_cleanup();
+	/* Shutdown the clockevents source. */
+	tick_suspend_local();
+	free_mem();
+	lkl_ops->thread_join(current_thread_info()->tid);
+
+	return 0;
+}
+
+static int lkl_run_init(struct linux_binprm *bprm);
+
+static struct linux_binfmt lkl_run_init_binfmt = {
+	.module		= THIS_MODULE,
+	.load_binary	= lkl_run_init,
+};
+
+static int lkl_run_init(struct linux_binprm *bprm)
+{
+	int ret;
+
+	if (strcmp("/init", bprm->filename) != 0)
+		return -EINVAL;
+
+	ret = flush_old_exec(bprm);
+	if (ret)
+		return ret;
+	set_personality(PER_LINUX);
+	setup_new_exec(bprm);
+	install_exec_creds(bprm);
+
+	set_binfmt(&lkl_run_init_binfmt);
+
+	init_pid_ns.child_reaper = NULL;
+
+	syscalls_init();
+
+	lkl_ops->sem_up(init_sem);
+	lkl_ops->thread_exit();
+
+	return 0;
+}
+
+/* skip mounting the "real" rootfs. ramfs is good enough. */
+static int __init fs_setup(void)
+{
+	int fd;
+
+	fd = sys_open("/init", O_CREAT, 0700);
+	WARN_ON(fd < 0);
+	sys_close(fd);
+
+	register_binfmt(&lkl_run_init_binfmt);
+
+	return 0;
+}
+late_initcall(fs_setup);
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v3 12/26] um lkl: plug in the build system
  2020-02-05  7:30 [RFC v3 00/26] Unifying LKL into UML Hajime Tazaki
@ 2020-02-05  7:30   ` Hajime Tazaki
       [not found] ` <cover.1580882335.git.thehajime-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
                     ` (25 subsequent siblings)
  26 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05  7:30 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Hajime Tazaki

From: Octavian Purdila <tavi.purdila@gmail.com>

Basic Makefiles for building LKL. Add a new architecture specific
target for installing the resulting library files and headers.

To make LKL binaries build, UML introduced an additional option, UMMODE
variable, to switch the output file of build: kernel (default), or
library (LKL).  Those modes are not able to be ON at the same time.

To build on library mode, users do the following:

  make defconfig ARCH=um UMMODE=library
  make ARCH=um UMMODE=library

Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
Signed-off-by: Akira Moroo <retrage01@gmail.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/Kconfig             | 13 +++++++
 arch/um/Makefile            | 19 ++++++++++-
 arch/um/Makefile.um         | 10 ++++++
 arch/um/lkl/Kconfig         | 31 +++++++++++++++--
 arch/um/lkl/Makefile        | 67 +++++++++++++++++++++++++++++++++++++
 arch/um/lkl/auto.conf       |  1 +
 arch/um/lkl/kernel/Makefile |  4 +++
 arch/um/lkl/mm/Makefile     |  1 +
 8 files changed, 143 insertions(+), 3 deletions(-)
 create mode 100644 arch/um/Makefile.um
 create mode 100644 arch/um/lkl/Makefile
 create mode 100644 arch/um/lkl/auto.conf
 create mode 100644 arch/um/lkl/kernel/Makefile
 create mode 100644 arch/um/lkl/mm/Makefile

diff --git a/arch/um/Kconfig b/arch/um/Kconfig
index 0917f8443c28..fce63e42198d 100644
--- a/arch/um/Kconfig
+++ b/arch/um/Kconfig
@@ -5,6 +5,10 @@ menu "UML-specific options"
 config UML
 	bool
 	default y
+
+config UMMODE_KERN
+	bool "UML mode: kernel mode"
+	default y if "$(UMMODE)" = "kernel"
 	select ARCH_HAS_KCOV
 	select ARCH_NO_PREEMPT
 	select HAVE_ARCH_AUDITSYSCALL
@@ -20,7 +24,12 @@ config UML
 	select GENERIC_CLOCKEVENTS
 	select HAVE_GCC_PLUGINS
 	select TTY # Needed for line.c
+        help
+	  This mode switches a mode to build a regular kernel executable
+          of UML.
+
 
+if UMMODE_KERN
 config MMU
 	bool
 	default y
@@ -200,6 +209,10 @@ config UML_TIME_TRAVEL_SUPPORT
 
 	  It is safe to say Y, but you probably don't need this.
 
+endif #UMMODE_KERN
+
 endmenu
 
 source "arch/um/drivers/Kconfig"
+
+source "arch/um/lkl/Kconfig"
diff --git a/arch/um/Makefile b/arch/um/Makefile
index d2daa206872d..f2a537f700c2 100644
--- a/arch/um/Makefile
+++ b/arch/um/Makefile
@@ -1,3 +1,4 @@
+# SPDX-License-Identifier: GPL-2.0
 #
 # This file is included by the global makefile so that you can add your own
 # architecture-specific flags and dependencies.
@@ -44,7 +45,7 @@ HOST_DIR := arch/$(HEADER_ARCH)
 include $(ARCH_DIR)/Makefile-skas
 include $(HOST_DIR)/Makefile.um
 
-core-y += $(HOST_DIR)/um/
+
 
 SHARED_HEADERS	:= $(ARCH_DIR)/include/shared
 ARCH_INCLUDE	:= -I$(srctree)/$(SHARED_HEADERS)
@@ -144,5 +145,21 @@ CLEAN_FILES += linux x.i gmon.out
 archclean:
 	@find . \( -name '*.bb' -o -name '*.bbg' -o -name '*.da' \
 		-o -name '*.gcov' \) -type f -print | xargs rm -f
+	$(Q)rm -rf $(srctree)/$(LKL_DIR)/include/generated
 
 export HEADER_ARCH SUBARCH USER_CFLAGS CFLAGS_NO_HARDENING OS DEV_NULL_PATH
+
+
+
+# SPDX-License-Identifier: GPL-2.0
+# select mode of UML build
+UMMODE ?= kernel
+LKL_DIR := $(ARCH_DIR)/lkl
+
+ifeq ($(UMMODE),kernel)
+	include $(ARCH_DIR)/Makefile.um
+else ifeq ($(UMMODE),library)
+	include $(ARCH_DIR)/lkl/Makefile
+endif
+
+export UMMODE LKL_DIR
diff --git a/arch/um/Makefile.um b/arch/um/Makefile.um
new file mode 100644
index 000000000000..be7ee4d1adde
--- /dev/null
+++ b/arch/um/Makefile.um
@@ -0,0 +1,10 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# This file is included by the global makefile so that you can add your own
+# architecture-specific flags and dependencies.
+#
+# Copyright (C) 2002 - 2007 Jeff Dike (jdike@{addtoit,linux.intel}.com)
+# Licensed under the GPL
+#
+
+core-y += $(HOST_DIR)/um/
diff --git a/arch/um/lkl/Kconfig b/arch/um/lkl/Kconfig
index f7b641ea7aef..f72b423fad5b 100644
--- a/arch/um/lkl/Kconfig
+++ b/arch/um/lkl/Kconfig
@@ -1,6 +1,25 @@
 # SPDX-License-Identifier: GPL-2.0
 
-config UML_LKL
+menu "LKL-specific options"
+
+config UML
+	bool
+	default y
+
+config UMMODE_LIB
+	bool "UML mode: library mode"
+	depends on !UMMODE_KERN
+	select LKL
+	default y if "$(UMMODE)" = "library"
+	help
+	  This mode switches a mode to build a library of UML (Linux
+	  Kernel Library/LKL).  This switch is exclusive to "kernel mode"
+	  of UML, which is traditional mode of UML.
+
+	  For more detail about LKL, see
+	  <file:Documentation/virt/uml/lkl.txt>.
+
+config LKL
        def_bool y
        depends on !SMP && !MMU && !COREDUMP && !SECCOMP && !UPROBES && !COMPAT && !USER_RETURN_NOTIFIER
        select ARCH_THREAD_STACK_ALLOCATOR
@@ -29,6 +48,12 @@ config OUTPUT_FORMAT
 config ARCH_DMA_ADDR_T_64BIT
        def_bool 64BIT
 
+config X86_64
+       def_bool y if "$(OUTPUT_FORMAT)" = "elf64-x86-64"
+
+config X86_32
+       def_bool y if "$(OUTPUT_FORMAT)" = "elf32-i386"
+
 config 64BIT
        def_bool n
 
@@ -39,7 +64,7 @@ config BIG_ENDIAN
        def_bool n
 
 config GENERIC_CSUM
-       def_bool y
+       def_bool LKL
 
 config GENERIC_HWEIGHT
        def_bool y
@@ -54,3 +79,5 @@ config RWSEM_GENERIC_SPINLOCK
 config HZ
         int
         default 100
+
+endmenu
diff --git a/arch/um/lkl/Makefile b/arch/um/lkl/Makefile
new file mode 100644
index 000000000000..e1161fa3fb63
--- /dev/null
+++ b/arch/um/lkl/Makefile
@@ -0,0 +1,67 @@
+# SPDX-License-Identifier: GPL-2.0
+
+include $(LKL_DIR)/auto.conf
+
+# fixup CFLAGS of um build
+KBUILD_CFLAGS := $(subst $(CFLAGS),,$(KBUILD_CFLAGS))
+
+SRCARCH := um/lkl
+ARCH_INCLUDE += -I$(srctree)/$(LKL_DIR)/um/include
+LINUXINCLUDE := $(subst $(ARCH_DIR),$(LKL_DIR),$(LINUXINCLUDE)) $(ARCH_INCLUDE)
+KBUILD_CFLAGS += -fno-builtin
+KBUILD_DEFCONFIG := lkl_defconfig
+
+ifneq (,$(filter $(OUTPUT_FORMAT),elf64-x86-64 elf32-i386 elf64-x86-64-freebsd elf32-littlearm elf64-littleaarch64))
+KBUILD_CFLAGS += -fPIC
+else ifneq (,$(filter $(OUTPUT_FORMAT),pe-i386 pe-x86-64 ))
+ifneq ($(OUTPUT_FORMAT),pe-x86-64)
+prefix=_
+endif
+# workaround for #include_next<stdarg.h> errors
+LINUXINCLUDE := -isystem $(LKL_DIR)/include/system $(LINUXINCLUDE)
+# workaround for https://gcc.gnu.org/bugzilla/show_bug.cgi?id=52991
+KBUILD_CFLAGS += -mno-ms-bitfields
+else
+$(error Unrecognized platform: $(OUTPUT_FORMAT))
+endif
+
+ifeq ($(shell uname -s), Linux)
+NPROC=$(shell nproc)
+else # e.g., FreeBSD
+NPROC=$(shell sysctl -n hw.ncpu)
+endif
+
+LDFLAGS_vmlinux += -r
+LKL_ENTRY_POINTS := lkl_start_kernel lkl_sys_halt lkl_syscall lkl_trigger_irq \
+	lkl_get_free_irq lkl_put_irq lkl_is_running lkl_bug lkl_printf
+
+ifeq ($(OUTPUT_FORMAT),elf32-i386)
+LKL_ENTRY_POINTS += \
+	__x86.get_pc_thunk.bx __x86.get_pc_thunk.dx __x86.get_pc_thunk.ax \
+	__x86.get_pc_thunk.cx __x86.get_pc_thunk.si __x86.get_pc_thunk.di
+endif
+
+core-y += $(LKL_DIR)/kernel/
+core-y += $(LKL_DIR)/mm/
+
+all: lkl.o
+
+lkl.o: vmlinux
+	$(OBJCOPY) -R .eh_frame -R .syscall_defs $(foreach sym,$(LKL_ENTRY_POINTS),-G$(prefix)$(sym)) vmlinux lkl.o
+
+$(LKL_DIR)/include/generated/uapi/asm/syscall_defs.h: vmlinux
+	$(OBJCOPY) -j .syscall_defs -O binary --set-section-flags .syscall_defs=alloc $< $@
+	$(Q) export tmpfile=$(shell mktemp); \
+	sed 's/\x0//g' $@ > $$tmpfile; mv $$tmpfile $@ ; rm -f $$tmpfile
+
+install: lkl.o headers $(LKL_DIR)/include/generated/uapi/asm/syscall_defs.h
+	@echo "  INSTALL	$(INSTALL_PATH)/lib/lkl.o"
+	@mkdir -p $(INSTALL_PATH)/lib/
+	@cp lkl.o $(INSTALL_PATH)/lib/
+	$(Q)$(srctree)/$(LKL_DIR)/scripts/headers_install.py \
+		$(subst -j,-j$(NPROC),$(findstring -j,$(MAKEFLAGS))) \
+		$(INSTALL_PATH)/include
+
+define archhelp
+  echo '  install	- Install library and headers to INSTALL_PATH/{lib,include}'
+endef
diff --git a/arch/um/lkl/auto.conf b/arch/um/lkl/auto.conf
new file mode 100644
index 000000000000..4bfd65a02d73
--- /dev/null
+++ b/arch/um/lkl/auto.conf
@@ -0,0 +1 @@
+export OUTPUT_FORMAT=$(shell $(LD) -r -print-output-format)
diff --git a/arch/um/lkl/kernel/Makefile b/arch/um/lkl/kernel/Makefile
new file mode 100644
index 000000000000..ef489f2f7176
--- /dev/null
+++ b/arch/um/lkl/kernel/Makefile
@@ -0,0 +1,4 @@
+extra-y := vmlinux.lds
+
+obj-y = setup.o threads.o irq.o time.o syscalls.o misc.o console.o \
+	syscalls_32.o cpu.o
diff --git a/arch/um/lkl/mm/Makefile b/arch/um/lkl/mm/Makefile
new file mode 100644
index 000000000000..2af6e3051897
--- /dev/null
+++ b/arch/um/lkl/mm/Makefile
@@ -0,0 +1 @@
+obj-y = bootmem.o
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v3 12/26] um lkl: plug in the build system
@ 2020-02-05  7:30   ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05  7:30 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, linux-kernel-library, linux-arch,
	Hajime Tazaki, Akira Moroo

From: Octavian Purdila <tavi.purdila@gmail.com>

Basic Makefiles for building LKL. Add a new architecture specific
target for installing the resulting library files and headers.

To make LKL binaries build, UML introduced an additional option, UMMODE
variable, to switch the output file of build: kernel (default), or
library (LKL).  Those modes are not able to be ON at the same time.

To build on library mode, users do the following:

  make defconfig ARCH=um UMMODE=library
  make ARCH=um UMMODE=library

Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
Signed-off-by: Akira Moroo <retrage01@gmail.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/Kconfig             | 13 +++++++
 arch/um/Makefile            | 19 ++++++++++-
 arch/um/Makefile.um         | 10 ++++++
 arch/um/lkl/Kconfig         | 31 +++++++++++++++--
 arch/um/lkl/Makefile        | 67 +++++++++++++++++++++++++++++++++++++
 arch/um/lkl/auto.conf       |  1 +
 arch/um/lkl/kernel/Makefile |  4 +++
 arch/um/lkl/mm/Makefile     |  1 +
 8 files changed, 143 insertions(+), 3 deletions(-)
 create mode 100644 arch/um/Makefile.um
 create mode 100644 arch/um/lkl/Makefile
 create mode 100644 arch/um/lkl/auto.conf
 create mode 100644 arch/um/lkl/kernel/Makefile
 create mode 100644 arch/um/lkl/mm/Makefile

diff --git a/arch/um/Kconfig b/arch/um/Kconfig
index 0917f8443c28..fce63e42198d 100644
--- a/arch/um/Kconfig
+++ b/arch/um/Kconfig
@@ -5,6 +5,10 @@ menu "UML-specific options"
 config UML
 	bool
 	default y
+
+config UMMODE_KERN
+	bool "UML mode: kernel mode"
+	default y if "$(UMMODE)" = "kernel"
 	select ARCH_HAS_KCOV
 	select ARCH_NO_PREEMPT
 	select HAVE_ARCH_AUDITSYSCALL
@@ -20,7 +24,12 @@ config UML
 	select GENERIC_CLOCKEVENTS
 	select HAVE_GCC_PLUGINS
 	select TTY # Needed for line.c
+        help
+	  This mode switches a mode to build a regular kernel executable
+          of UML.
+
 
+if UMMODE_KERN
 config MMU
 	bool
 	default y
@@ -200,6 +209,10 @@ config UML_TIME_TRAVEL_SUPPORT
 
 	  It is safe to say Y, but you probably don't need this.
 
+endif #UMMODE_KERN
+
 endmenu
 
 source "arch/um/drivers/Kconfig"
+
+source "arch/um/lkl/Kconfig"
diff --git a/arch/um/Makefile b/arch/um/Makefile
index d2daa206872d..f2a537f700c2 100644
--- a/arch/um/Makefile
+++ b/arch/um/Makefile
@@ -1,3 +1,4 @@
+# SPDX-License-Identifier: GPL-2.0
 #
 # This file is included by the global makefile so that you can add your own
 # architecture-specific flags and dependencies.
@@ -44,7 +45,7 @@ HOST_DIR := arch/$(HEADER_ARCH)
 include $(ARCH_DIR)/Makefile-skas
 include $(HOST_DIR)/Makefile.um
 
-core-y += $(HOST_DIR)/um/
+
 
 SHARED_HEADERS	:= $(ARCH_DIR)/include/shared
 ARCH_INCLUDE	:= -I$(srctree)/$(SHARED_HEADERS)
@@ -144,5 +145,21 @@ CLEAN_FILES += linux x.i gmon.out
 archclean:
 	@find . \( -name '*.bb' -o -name '*.bbg' -o -name '*.da' \
 		-o -name '*.gcov' \) -type f -print | xargs rm -f
+	$(Q)rm -rf $(srctree)/$(LKL_DIR)/include/generated
 
 export HEADER_ARCH SUBARCH USER_CFLAGS CFLAGS_NO_HARDENING OS DEV_NULL_PATH
+
+
+
+# SPDX-License-Identifier: GPL-2.0
+# select mode of UML build
+UMMODE ?= kernel
+LKL_DIR := $(ARCH_DIR)/lkl
+
+ifeq ($(UMMODE),kernel)
+	include $(ARCH_DIR)/Makefile.um
+else ifeq ($(UMMODE),library)
+	include $(ARCH_DIR)/lkl/Makefile
+endif
+
+export UMMODE LKL_DIR
diff --git a/arch/um/Makefile.um b/arch/um/Makefile.um
new file mode 100644
index 000000000000..be7ee4d1adde
--- /dev/null
+++ b/arch/um/Makefile.um
@@ -0,0 +1,10 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# This file is included by the global makefile so that you can add your own
+# architecture-specific flags and dependencies.
+#
+# Copyright (C) 2002 - 2007 Jeff Dike (jdike@{addtoit,linux.intel}.com)
+# Licensed under the GPL
+#
+
+core-y += $(HOST_DIR)/um/
diff --git a/arch/um/lkl/Kconfig b/arch/um/lkl/Kconfig
index f7b641ea7aef..f72b423fad5b 100644
--- a/arch/um/lkl/Kconfig
+++ b/arch/um/lkl/Kconfig
@@ -1,6 +1,25 @@
 # SPDX-License-Identifier: GPL-2.0
 
-config UML_LKL
+menu "LKL-specific options"
+
+config UML
+	bool
+	default y
+
+config UMMODE_LIB
+	bool "UML mode: library mode"
+	depends on !UMMODE_KERN
+	select LKL
+	default y if "$(UMMODE)" = "library"
+	help
+	  This mode switches a mode to build a library of UML (Linux
+	  Kernel Library/LKL).  This switch is exclusive to "kernel mode"
+	  of UML, which is traditional mode of UML.
+
+	  For more detail about LKL, see
+	  <file:Documentation/virt/uml/lkl.txt>.
+
+config LKL
        def_bool y
        depends on !SMP && !MMU && !COREDUMP && !SECCOMP && !UPROBES && !COMPAT && !USER_RETURN_NOTIFIER
        select ARCH_THREAD_STACK_ALLOCATOR
@@ -29,6 +48,12 @@ config OUTPUT_FORMAT
 config ARCH_DMA_ADDR_T_64BIT
        def_bool 64BIT
 
+config X86_64
+       def_bool y if "$(OUTPUT_FORMAT)" = "elf64-x86-64"
+
+config X86_32
+       def_bool y if "$(OUTPUT_FORMAT)" = "elf32-i386"
+
 config 64BIT
        def_bool n
 
@@ -39,7 +64,7 @@ config BIG_ENDIAN
        def_bool n
 
 config GENERIC_CSUM
-       def_bool y
+       def_bool LKL
 
 config GENERIC_HWEIGHT
        def_bool y
@@ -54,3 +79,5 @@ config RWSEM_GENERIC_SPINLOCK
 config HZ
         int
         default 100
+
+endmenu
diff --git a/arch/um/lkl/Makefile b/arch/um/lkl/Makefile
new file mode 100644
index 000000000000..e1161fa3fb63
--- /dev/null
+++ b/arch/um/lkl/Makefile
@@ -0,0 +1,67 @@
+# SPDX-License-Identifier: GPL-2.0
+
+include $(LKL_DIR)/auto.conf
+
+# fixup CFLAGS of um build
+KBUILD_CFLAGS := $(subst $(CFLAGS),,$(KBUILD_CFLAGS))
+
+SRCARCH := um/lkl
+ARCH_INCLUDE += -I$(srctree)/$(LKL_DIR)/um/include
+LINUXINCLUDE := $(subst $(ARCH_DIR),$(LKL_DIR),$(LINUXINCLUDE)) $(ARCH_INCLUDE)
+KBUILD_CFLAGS += -fno-builtin
+KBUILD_DEFCONFIG := lkl_defconfig
+
+ifneq (,$(filter $(OUTPUT_FORMAT),elf64-x86-64 elf32-i386 elf64-x86-64-freebsd elf32-littlearm elf64-littleaarch64))
+KBUILD_CFLAGS += -fPIC
+else ifneq (,$(filter $(OUTPUT_FORMAT),pe-i386 pe-x86-64 ))
+ifneq ($(OUTPUT_FORMAT),pe-x86-64)
+prefix=_
+endif
+# workaround for #include_next<stdarg.h> errors
+LINUXINCLUDE := -isystem $(LKL_DIR)/include/system $(LINUXINCLUDE)
+# workaround for https://gcc.gnu.org/bugzilla/show_bug.cgi?id=52991
+KBUILD_CFLAGS += -mno-ms-bitfields
+else
+$(error Unrecognized platform: $(OUTPUT_FORMAT))
+endif
+
+ifeq ($(shell uname -s), Linux)
+NPROC=$(shell nproc)
+else # e.g., FreeBSD
+NPROC=$(shell sysctl -n hw.ncpu)
+endif
+
+LDFLAGS_vmlinux += -r
+LKL_ENTRY_POINTS := lkl_start_kernel lkl_sys_halt lkl_syscall lkl_trigger_irq \
+	lkl_get_free_irq lkl_put_irq lkl_is_running lkl_bug lkl_printf
+
+ifeq ($(OUTPUT_FORMAT),elf32-i386)
+LKL_ENTRY_POINTS += \
+	__x86.get_pc_thunk.bx __x86.get_pc_thunk.dx __x86.get_pc_thunk.ax \
+	__x86.get_pc_thunk.cx __x86.get_pc_thunk.si __x86.get_pc_thunk.di
+endif
+
+core-y += $(LKL_DIR)/kernel/
+core-y += $(LKL_DIR)/mm/
+
+all: lkl.o
+
+lkl.o: vmlinux
+	$(OBJCOPY) -R .eh_frame -R .syscall_defs $(foreach sym,$(LKL_ENTRY_POINTS),-G$(prefix)$(sym)) vmlinux lkl.o
+
+$(LKL_DIR)/include/generated/uapi/asm/syscall_defs.h: vmlinux
+	$(OBJCOPY) -j .syscall_defs -O binary --set-section-flags .syscall_defs=alloc $< $@
+	$(Q) export tmpfile=$(shell mktemp); \
+	sed 's/\x0//g' $@ > $$tmpfile; mv $$tmpfile $@ ; rm -f $$tmpfile
+
+install: lkl.o headers $(LKL_DIR)/include/generated/uapi/asm/syscall_defs.h
+	@echo "  INSTALL	$(INSTALL_PATH)/lib/lkl.o"
+	@mkdir -p $(INSTALL_PATH)/lib/
+	@cp lkl.o $(INSTALL_PATH)/lib/
+	$(Q)$(srctree)/$(LKL_DIR)/scripts/headers_install.py \
+		$(subst -j,-j$(NPROC),$(findstring -j,$(MAKEFLAGS))) \
+		$(INSTALL_PATH)/include
+
+define archhelp
+  echo '  install	- Install library and headers to INSTALL_PATH/{lib,include}'
+endef
diff --git a/arch/um/lkl/auto.conf b/arch/um/lkl/auto.conf
new file mode 100644
index 000000000000..4bfd65a02d73
--- /dev/null
+++ b/arch/um/lkl/auto.conf
@@ -0,0 +1 @@
+export OUTPUT_FORMAT=$(shell $(LD) -r -print-output-format)
diff --git a/arch/um/lkl/kernel/Makefile b/arch/um/lkl/kernel/Makefile
new file mode 100644
index 000000000000..ef489f2f7176
--- /dev/null
+++ b/arch/um/lkl/kernel/Makefile
@@ -0,0 +1,4 @@
+extra-y := vmlinux.lds
+
+obj-y = setup.o threads.o irq.o time.o syscalls.o misc.o console.o \
+	syscalls_32.o cpu.o
diff --git a/arch/um/lkl/mm/Makefile b/arch/um/lkl/mm/Makefile
new file mode 100644
index 000000000000..2af6e3051897
--- /dev/null
+++ b/arch/um/lkl/mm/Makefile
@@ -0,0 +1 @@
+obj-y = bootmem.o
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v3 13/26] lkl tools: skeleton for host side library
  2020-02-05  7:30 [RFC v3 00/26] Unifying LKL into UML Hajime Tazaki
@ 2020-02-05  7:30   ` Hajime Tazaki
       [not found] ` <cover.1580882335.git.thehajime-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
                     ` (25 subsequent siblings)
  26 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05  7:30 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Ben Wolsieffer, Conrad Meyer, Luca Dariz, Mark Stillwell,
	Michael Zimmermann, Motomu Utsumi, Patrick Collins,
	Petros Angelatos, Thomas Liebetraut, Xiao Jia, Yuan Liu,
	Hajime Tazaki

From: Octavian Purdila <tavi.purdila@gmail.com>

This patch adds the skeleton for the host library.

The host library is implementing the host operations needed by LKL and
is split into host dependent (depends on a specific host, e.g. POSIX
hosts) and host independent parts (will work on all supported hosts).

Cc: Ben Wolsieffer <benwolsieffer@gmail.com>
Cc: Conrad Meyer <cem@FreeBSD.org>
Cc: H.K. Jerry Chu <hkchu@google.com>
Cc: Luca Dariz <luca.dariz@gmail.com>
Cc: Mark Stillwell <mark@stillwell.me>
Cc: Michael Zimmermann <sigmaepsilon92@gmail.com>
Cc: Motomu Utsumi <motomuman@gmail.com>
Cc: Patrick Collins <pscollins@google.com>
Cc: Petros Angelatos <petrosagg@gmail.com>
Cc: Thomas Liebetraut <thomas@tommie-lie.de>
Cc: Xiao Jia <xiaoj@google.com>
Cc: Yuan Liu <liuyuan@google.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/lkl/.gitignore         |   4 +
 tools/lkl/Build              |   0
 tools/lkl/Makefile           | 125 ++++++++++++
 tools/lkl/Makefile.autoconf  |  65 +++++++
 tools/lkl/Targets            |   3 +
 tools/lkl/include/.gitignore |   1 +
 tools/lkl/include/lkl.h      | 358 +++++++++++++++++++++++++++++++++++
 tools/lkl/include/lkl_host.h |  19 ++
 tools/lkl/lib/.gitignore     |   3 +
 tools/lkl/lib/Build          |   1 +
 10 files changed, 579 insertions(+)
 create mode 100644 tools/lkl/.gitignore
 create mode 100644 tools/lkl/Build
 create mode 100644 tools/lkl/Makefile
 create mode 100644 tools/lkl/Makefile.autoconf
 create mode 100644 tools/lkl/Targets
 create mode 100644 tools/lkl/include/.gitignore
 create mode 100644 tools/lkl/include/lkl.h
 create mode 100644 tools/lkl/include/lkl_host.h
 create mode 100644 tools/lkl/lib/.gitignore
 create mode 100644 tools/lkl/lib/Build

diff --git a/tools/lkl/.gitignore b/tools/lkl/.gitignore
new file mode 100644
index 000000000000..1aed58bfe171
--- /dev/null
+++ b/tools/lkl/.gitignore
@@ -0,0 +1,4 @@
+Makefile.conf
+include/lkl_autoconf.h
+tests/autoconf.sh
+bin/stat
diff --git a/tools/lkl/Build b/tools/lkl/Build
new file mode 100644
index 000000000000..e69de29bb2d1
diff --git a/tools/lkl/Makefile b/tools/lkl/Makefile
new file mode 100644
index 000000000000..11ea7e9095bb
--- /dev/null
+++ b/tools/lkl/Makefile
@@ -0,0 +1,125 @@
+# Do not use make's built-in rules
+# (this improves performance and avoids hard-to-debug behaviour);
+# also do not print "Entering directory..." messages from make
+.SUFFIXES:
+MAKEFLAGS += -r --no-print-directory
+
+KCONFIG?=defconfig
+
+ifneq ($(silent),1)
+  ifneq ($(V),1)
+	QUIET_AUTOCONF       = @echo '  AUTOCONF '$@;
+	Q = @
+  endif
+endif
+
+PREFIX   := /usr
+
+ifeq (,$(srctree))
+  srctree := $(patsubst %/,%,$(dir $(shell pwd)))
+  srctree := $(patsubst %/,%,$(dir $(srctree)))
+endif
+export srctree
+
+-include $(srctree)/tools/scripts/Makefile.include
+
+# OUTPUT fixup should be *after* include ../scripts/Makefile.include
+ifneq ($(OUTPUT),)
+  OUTPUT := $(OUTPUT)/tools/lkl/
+else
+  OUTPUT := $(CURDIR)/
+endif
+export OUTPUT
+
+
+all:
+
+conf: $(OUTPUT)Makefile.conf
+
+$(OUTPUT)Makefile.conf: Makefile.autoconf
+	$(call QUIET_AUTOCONF, headers)$(MAKE) -f Makefile.autoconf -s
+
+-include $(OUTPUT)Makefile.conf
+
+export CFLAGS += -I$(OUTPUT)/include -Iinclude -Wall -g -O2 -Wextra \
+	 -Wno-unused-parameter \
+	 -Wno-missing-field-initializers -fno-strict-aliasing
+
+-include Targets
+
+TARGETS := $(progs-y:%=$(OUTPUT)%$(EXESUF))
+TARGETS += $(libs-y:%=$(OUTPUT)%$(SOSUF))
+all: $(TARGETS)
+
+# this workaround is for FreeBSD
+bin/stat:
+	$(Q)mkdir -p bin/
+ifeq ($(LKL_HOST_CONFIG_BSD),y)
+	$(Q)ln -sf `which gnustat` bin/stat
+	$(Q)ln -sf `which gsed` bin/sed
+else
+	$(Q)touch bin/stat
+endif
+
+# rule to build lkl.o
+$(OUTPUT)lib/lkl.o: bin/stat
+	$(Q)$(MAKE) -C $(srctree) ARCH=um UMMODE=library $(KOPT) $(KCONFIG)
+# this workaround is for arm32 linker (ld.gold)
+	$(Q)export PATH=$(shell pwd)/bin/:${PATH} ;\
+	$(MAKE) -C $(srctree) ARCH=um UMMODE=library $(KOPT) install INSTALL_PATH=$(OUTPUT)
+
+# rules to link libs
+$(OUTPUT)%$(SOSUF): LDFLAGS += -shared
+$(OUTPUT)%$(SOSUF): $(OUTPUT)%-in.o $(OUTPUT)liblkl.a
+	$(QUIET_LINK)$(CC) $(LDFLAGS) $(LDFLAGS_$*-y) -o $@ $^ $(LDLIBS) $(LDLIBS_$*-y)
+
+# liblkl is special
+$(OUTPUT)liblkl$(SOSUF): $(OUTPUT)%-in.o $(OUTPUT)lib/lkl.o
+$(OUTPUT)liblkl.a: $(OUTPUT)lib/liblkl-in.o $(OUTPUT)lib/lkl.o
+	$(QUIET_AR)$(AR) -rc $@ $^
+
+# rule to link programs
+$(OUTPUT)%$(EXESUF): $(OUTPUT)%-in.o $(OUTPUT)liblkl.a
+	$(QUIET_LINK)$(CC) $(LDFLAGS) $(LDFLAGS_$*-y) -o $@ $^ $(LDLIBS) $(LDLIBS_$*-y)
+
+# rule to build objects
+$(OUTPUT)%-in.o: $(OUTPUT)lib/lkl.o FORCE
+	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
+
+
+clean:
+	$(call QUIET_CLEAN, objects)find $(OUTPUT) -name '*.o' -delete -o -name '\.*.cmd'\
+	 -delete -o -name '\.*.d' -delete
+	$(call QUIET_CLEAN, headers)$(RM) -r $(OUTPUT)/include/lkl/
+	$(call QUIET_CLEAN, liblkl.a)$(RM) $(OUTPUT)/liblkl.a
+	$(call QUIET_CLEAN, targets)$(RM) $(TARGETS) bin/stat
+
+clean-conf: clean
+	$(call QUIET_CLEAN, Makefile.conf)$(RM) $(OUTPUT)/Makefile.conf
+
+headers_install: $(TARGETS)
+	$(call QUIET_INSTALL, headers) \
+	    install -d $(DESTDIR)$(PREFIX)/include ; \
+	    install -m 644 include/lkl.h include/lkl_host.h $(OUTPUT)include/lkl_autoconf.h \
+	      include/lkl_config.h $(DESTDIR)$(PREFIX)/include ; \
+	    cp -r $(OUTPUT)include/lkl $(DESTDIR)$(PREFIX)/include
+
+libraries_install: $(libs-y:%=$(OUTPUT)%$(SOSUF)) $(OUTPUT)liblkl.a
+	$(call QUIET_INSTALL, libraries) \
+	    install -d $(DESTDIR)$(PREFIX)/lib ; \
+	    install -m 644 $^ $(DESTDIR)$(PREFIX)/lib
+
+programs_install: $(progs-y:%=$(OUTPUT)%$(EXESUF))
+	$(call QUIET_INSTALL, programs) \
+	    install -d $(DESTDIR)$(PREFIX)/bin ; \
+	    install -m 755 $^ $(DESTDIR)$(PREFIX)/bin
+
+install: headers_install libraries_install programs_install
+
+
+FORCE: ;
+.PHONY: all clean FORCE
+.PHONY: headers_install libraries_install programs_install install
+.NOTPARALLEL : lib/lkl.o
+.SECONDARY:
+
diff --git a/tools/lkl/Makefile.autoconf b/tools/lkl/Makefile.autoconf
new file mode 100644
index 000000000000..268c367d9962
--- /dev/null
+++ b/tools/lkl/Makefile.autoconf
@@ -0,0 +1,65 @@
+POSIX_HOSTS=elf64-x86-64 elf32-i386 elf64-x86-64-freebsd
+
+define set_autoconf_var
+  $(shell echo "#define LKL_HOST_CONFIG_$(1) $(2)" \
+	  >> $(OUTPUT)/include/lkl_autoconf.h)
+  $(shell echo "LKL_HOST_CONFIG_$(1)=$(2)" >> $(OUTPUT)/tests/autoconf.sh)
+  export LKL_HOST_CONFIG_$(1)=$(2)
+endef
+
+define find_include
+  $(eval include_paths=$(shell $(CC) -E -Wp,-v -xc /dev/null 2>&1 | grep '^ '))
+  $(foreach f, $(include_paths), $(wildcard $(f)/$(1)))
+endef
+
+define is_defined
+$(shell $(CC) -dM -E - </dev/null | grep $(1))
+endef
+
+define bsd_host
+  $(call set_autoconf_var,BSD,y)
+endef
+
+define arm_host
+  $(call set_autoconf_var,ARM,y)
+endef
+
+define aarch64_host
+  $(call set_autoconf_var,AARCH64,y)
+endef
+
+define posix_host
+  $(call set_autoconf_var,POSIX,y)
+  LDFLAGS += -pie
+  CFLAGS += -fPIC -pthread
+  SOSUF := .so
+  LDLIBS += -lrt -lpthread
+  $(if $(filter $(1),elf64-x86-64-freebsd),$(call bsd_host))
+  $(if $(filter $(1),elf32-littlearm),$(call arm_host))
+  $(if $(filter $(1),elf64-littleaarch64),$(call aarch64_host))
+  $(if $(strip $(call find_include,fuse.h)),$(call set_autoconf_var,FUSE,y))
+  $(if $(strip $(call find_include,archive.h)),$(call set_autoconf_var,ARCHIVE,y))
+  $(if $(filter $(1),elf64-x86-64-freebsd),$(call set_autoconf_var,NEEDS_LARGP,y))
+  $(if $(filter $(1),elf32-i386),$(call set_autoconf_var,I386,y))
+endef
+
+define do_autoconf
+  export CROSS_COMPILE := $(CROSS_COMPILE)
+  export CC := $(CROSS_COMPILE)gcc
+  export LD := $(CROSS_COMPILE)ld
+  export AR := $(CROSS_COMPILE)ar
+  $(eval LD := $(CROSS_COMPILE)ld)
+  $(eval CC := $(CROSS_COMPILE)gcc)
+  $(eval LD_FMT := $(shell $(LD) -r -print-output-format))
+  $(if $(filter $(LD_FMT),$(POSIX_HOSTS)),$(call posix_host,$(LD_FMT)))
+endef
+
+export do_autoconf
+
+
+$(OUTPUT)Makefile.conf: Makefile.autoconf
+	$(shell mkdir -p $(OUTPUT)/include)
+	$(shell mkdir -p $(OUTPUT)/tests)
+	$(shell echo -n "" > $(OUTPUT)/include/lkl_autoconf.h)
+	$(shell echo -n "" > $(OUTPUT)/tests/autoconf.sh)
+	@echo "$$do_autoconf" > $(OUTPUT)/Makefile.conf
diff --git a/tools/lkl/Targets b/tools/lkl/Targets
new file mode 100644
index 000000000000..24c985e64638
--- /dev/null
+++ b/tools/lkl/Targets
@@ -0,0 +1,3 @@
+libs-y += lib/liblkl
+
+
diff --git a/tools/lkl/include/.gitignore b/tools/lkl/include/.gitignore
new file mode 100644
index 000000000000..c41a463c898d
--- /dev/null
+++ b/tools/lkl/include/.gitignore
@@ -0,0 +1 @@
+lkl/
\ No newline at end of file
diff --git a/tools/lkl/include/lkl.h b/tools/lkl/include/lkl.h
new file mode 100644
index 000000000000..4b95d0ef8e5b
--- /dev/null
+++ b/tools/lkl/include/lkl.h
@@ -0,0 +1,358 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_H
+#define _LKL_H
+
+#include "lkl_autoconf.h"
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#define _LKL_LIBC_COMPAT_H
+
+#ifdef __cplusplus
+#define class __lkl__class
+#endif
+
+/*
+ * Avoid collisions between Android which defines __unused and
+ * linux/icmp.h which uses __unused as a structure field.
+ */
+#pragma push_macro("__unused")
+#undef __unused
+
+#include <lkl/asm/syscalls.h>
+
+#pragma pop_macro("__unused")
+
+#ifdef __cplusplus
+#undef class
+#endif
+
+#if defined(__MINGW32__)
+#define strtok_r strtok_s
+#define inet_pton lkl_inet_pton
+
+int inet_pton(int af, const char *src, void *dst);
+#endif
+
+#if __LKL__BITS_PER_LONG == 64
+#define lkl_sys_fstatat lkl_sys_newfstatat
+#define lkl_sys_fstat lkl_sys_newfstat
+
+#else
+#define __lkl__NR_fcntl __lkl__NR_fcntl64
+
+#define lkl_stat lkl_stat64
+#define lkl_sys_stat lkl_sys_stat64
+#define lkl_sys_lstat lkl_sys_lstat64
+#define lkl_sys_truncate lkl_sys_truncate64
+#define lkl_sys_ftruncate lkl_sys_ftruncate64
+#define lkl_sys_sendfile lkl_sys_sendfile64
+#define lkl_sys_fstatat lkl_sys_fstatat64
+#define lkl_sys_fstat lkl_sys_fstat64
+#define lkl_sys_fcntl lkl_sys_fcntl64
+
+#define lkl_statfs lkl_statfs64
+
+static inline int lkl_sys_statfs(const char *path, struct lkl_statfs *buf)
+{
+	return lkl_sys_statfs64(path, sizeof(*buf), buf);
+}
+
+static inline int lkl_sys_fstatfs(unsigned int fd, struct lkl_statfs *buf)
+{
+	return lkl_sys_fstatfs64(fd, sizeof(*buf), buf);
+}
+
+#define lkl_sys_nanosleep lkl_sys_nanosleep_time32
+static inline int lkl_sys_nanosleep_time32(struct __lkl__kernel_timespec *rqtp,
+					   struct __lkl__kernel_timespec *rmtp)
+{
+	long p[6] = {(long)rqtp, (long)rmtp, 0, 0, 0, 0};
+
+	return lkl_syscall(__lkl__NR_nanosleep, p);
+}
+
+#endif
+
+static inline int lkl_sys_stat(const char *path, struct lkl_stat *buf)
+{
+	return lkl_sys_fstatat(LKL_AT_FDCWD, path, buf, 0);
+}
+
+static inline int lkl_sys_lstat(const char *path, struct lkl_stat *buf)
+{
+	return lkl_sys_fstatat(LKL_AT_FDCWD, path, buf,
+			       LKL_AT_SYMLINK_NOFOLLOW);
+}
+
+#ifdef __lkl__NR_llseek
+/**
+ * lkl_sys_lseek - wrapper for lkl_sys_llseek
+ */
+static inline long long lkl_sys_lseek(unsigned int fd, __lkl__kernel_loff_t off,
+				      unsigned int whence)
+{
+	long long res;
+	long ret = lkl_sys_llseek(fd, off >> 32, off & 0xffffffff, &res,
+				  whence);
+
+	return ret < 0 ? ret : res;
+}
+#endif
+
+static inline void *lkl_sys_mmap(void *addr, size_t length, int prot, int flags,
+				 int fd, off_t offset)
+{
+	return (void *)lkl_sys_mmap_pgoff((long)addr, length, prot, flags, fd,
+					  offset >> 12);
+}
+
+#define lkl_sys_mmap2 lkl_sys_mmap_pgoff
+
+#ifdef __lkl__NR_openat
+/**
+ * lkl_sys_open - wrapper for lkl_sys_openat
+ */
+static inline long lkl_sys_open(const char *file, int flags, int mode)
+{
+	return lkl_sys_openat(LKL_AT_FDCWD, file, flags, mode);
+}
+
+/**
+ * lkl_sys_creat - wrapper for lkl_sys_openat
+ */
+static inline long lkl_sys_creat(const char *file, int mode)
+{
+	return lkl_sys_openat(LKL_AT_FDCWD, file,
+			      LKL_O_CREAT|LKL_O_WRONLY|LKL_O_TRUNC, mode);
+}
+#endif
+
+
+#ifdef __lkl__NR_faccessat
+/**
+ * lkl_sys_access - wrapper for lkl_sys_faccessat
+ */
+static inline long lkl_sys_access(const char *file, int mode)
+{
+	return lkl_sys_faccessat(LKL_AT_FDCWD, file, mode);
+}
+#endif
+
+#ifdef __lkl__NR_fchownat
+/**
+ * lkl_sys_chown - wrapper for lkl_sys_fchownat
+ */
+static inline long lkl_sys_chown(const char *path, lkl_uid_t uid, lkl_gid_t gid)
+{
+	return lkl_sys_fchownat(LKL_AT_FDCWD, path, uid, gid, 0);
+}
+#endif
+
+#ifdef __lkl__NR_fchmodat
+/**
+ * lkl_sys_chmod - wrapper for lkl_sys_fchmodat
+ */
+static inline long lkl_sys_chmod(const char *path, mode_t mode)
+{
+	return lkl_sys_fchmodat(LKL_AT_FDCWD, path, mode);
+}
+#endif
+
+#ifdef __lkl__NR_linkat
+/**
+ * lkl_sys_link - wrapper for lkl_sys_linkat
+ */
+static inline long lkl_sys_link(const char *existing, const char *new)
+{
+	return lkl_sys_linkat(LKL_AT_FDCWD, existing, LKL_AT_FDCWD, new, 0);
+}
+#endif
+
+#ifdef __lkl__NR_unlinkat
+/**
+ * lkl_sys_unlink - wrapper for lkl_sys_unlinkat
+ */
+static inline long lkl_sys_unlink(const char *path)
+{
+	return lkl_sys_unlinkat(LKL_AT_FDCWD, path, 0);
+}
+#endif
+
+#ifdef __lkl__NR_symlinkat
+/**
+ * lkl_sys_symlink - wrapper for lkl_sys_symlinkat
+ */
+static inline long lkl_sys_symlink(const char *existing, const char *new)
+{
+	return lkl_sys_symlinkat(existing, LKL_AT_FDCWD, new);
+}
+#endif
+
+#ifdef __lkl__NR_readlinkat
+/**
+ * lkl_sys_readlink - wrapper for lkl_sys_readlinkat
+ */
+static inline long lkl_sys_readlink(const char *path, char *buf, size_t bufsize)
+{
+	return lkl_sys_readlinkat(LKL_AT_FDCWD, path, buf, bufsize);
+}
+#endif
+
+#ifdef __lkl__NR_renameat
+/**
+ * lkl_sys_rename - wrapper for lkl_sys_renameat
+ */
+static inline long lkl_sys_rename(const char *old, const char *new)
+{
+	return lkl_sys_renameat(LKL_AT_FDCWD, old, LKL_AT_FDCWD, new);
+}
+#endif
+
+#ifdef __lkl__NR_mkdirat
+/**
+ * lkl_sys_mkdir - wrapper for lkl_sys_mkdirat
+ */
+static inline long lkl_sys_mkdir(const char *path, mode_t mode)
+{
+	return lkl_sys_mkdirat(LKL_AT_FDCWD, path, mode);
+}
+#endif
+
+#ifdef __lkl__NR_unlinkat
+/**
+ * lkl_sys_rmdir - wrapper for lkl_sys_unlinkrat
+ */
+static inline long lkl_sys_rmdir(const char *path)
+{
+	return lkl_sys_unlinkat(LKL_AT_FDCWD, path, LKL_AT_REMOVEDIR);
+}
+#endif
+
+#ifdef __lkl__NR_mknodat
+/**
+ * lkl_sys_mknod - wrapper for lkl_sys_mknodat
+ */
+static inline long lkl_sys_mknod(const char *path, mode_t mode, dev_t dev)
+{
+	return lkl_sys_mknodat(LKL_AT_FDCWD, path, mode, dev);
+}
+#endif
+
+#ifdef __lkl__NR_pipe2
+/**
+ * lkl_sys_pipe - wrapper for lkl_sys_pipe2
+ */
+static inline long lkl_sys_pipe(int fd[2])
+{
+	return lkl_sys_pipe2(fd, 0);
+}
+#endif
+
+#ifdef __lkl__NR_sendto
+/**
+ * lkl_sys_send - wrapper for lkl_sys_sendto
+ */
+static inline long lkl_sys_send(int fd, void *buf, size_t len, int flags)
+{
+	return lkl_sys_sendto(fd, buf, len, flags, 0, 0);
+}
+#endif
+
+#ifdef __lkl__NR_recvfrom
+/**
+ * lkl_sys_recv - wrapper for lkl_sys_recvfrom
+ */
+static inline long lkl_sys_recv(int fd, void *buf, size_t len, int flags)
+{
+	return lkl_sys_recvfrom(fd, buf, len, flags, 0, 0);
+}
+#endif
+
+#ifdef __lkl__NR_pselect6
+/**
+ * lkl_sys_select - wrapper for lkl_sys_pselect
+ */
+static inline long lkl_sys_select(int n, lkl_fd_set *rfds, lkl_fd_set *wfds,
+				  lkl_fd_set *efds, struct lkl_timeval *tv)
+{
+	long data[2] = { 0, _LKL_NSIG/8 };
+	struct lkl_timespec ts;
+	lkl_time_t extra_secs;
+	const lkl_time_t max_time = ((1ULL<<8)*sizeof(time_t)-1)-1;
+
+	if (tv) {
+		if (tv->tv_sec < 0 || tv->tv_usec < 0)
+			return -LKL_EINVAL;
+
+		extra_secs = tv->tv_usec / 1000000;
+		ts.tv_nsec = tv->tv_usec % 1000000 * 1000;
+		ts.tv_sec = extra_secs > max_time - tv->tv_sec ?
+			max_time : tv->tv_sec + extra_secs;
+	}
+	return lkl_sys_pselect6(n, rfds, wfds, efds, tv ?
+				(struct __lkl__kernel_timespec *)&ts : 0, data);
+}
+#endif
+
+#ifdef __lkl__NR_ppoll
+/**
+ * lkl_sys_poll - wrapper for lkl_sys_ppoll
+ */
+static inline long lkl_sys_poll(struct lkl_pollfd *fds, int n, int timeout)
+{
+	return lkl_sys_ppoll(fds, n, timeout >= 0 ?
+			     (struct __lkl__kernel_timespec *)
+			     &((struct lkl_timespec){ .tv_sec = timeout/1000,
+				   .tv_nsec = timeout%1000*1000000 }) : 0,
+			     0, _LKL_NSIG/8);
+}
+#endif
+
+#ifdef __lkl__NR_epoll_create1
+/**
+ * lkl_sys_epoll_create - wrapper for lkl_sys_epoll_create1
+ */
+static inline long lkl_sys_epoll_create(int size)
+{
+	return lkl_sys_epoll_create1(0);
+}
+#endif
+
+#ifdef __lkl__NR_epoll_pwait
+/**
+ * lkl_sys_epoll_wait - wrapper for lkl_sys_epoll_pwait
+ */
+static inline long lkl_sys_epoll_wait(int fd, struct lkl_epoll_event *ev,
+				      int cnt, int to)
+{
+	return lkl_sys_epoll_pwait(fd, ev, cnt, to, 0, _LKL_NSIG/8);
+}
+#endif
+
+
+
+/**
+ * lkl_strerror - returns a string describing the given error code
+ *
+ * @err - error code
+ * @returns - string for the given error code
+ */
+const char *lkl_strerror(int err);
+
+/**
+ * lkl_perror - prints a string describing the given error code
+ *
+ * @msg - prefix for the error message
+ * @err - error code
+ */
+void lkl_perror(char *msg, int err);
+
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif
diff --git a/tools/lkl/include/lkl_host.h b/tools/lkl/include/lkl_host.h
new file mode 100644
index 000000000000..b5f96096fe69
--- /dev/null
+++ b/tools/lkl/include/lkl_host.h
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_HOST_H
+#define _LKL_HOST_H
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include <lkl/asm/host_ops.h>
+#include <lkl.h>
+
+extern struct lkl_host_operations lkl_host_ops;
+
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif
diff --git a/tools/lkl/lib/.gitignore b/tools/lkl/lib/.gitignore
new file mode 100644
index 000000000000..427ae0273fdd
--- /dev/null
+++ b/tools/lkl/lib/.gitignore
@@ -0,0 +1,3 @@
+lkl.o
+liblkl.a
+
diff --git a/tools/lkl/lib/Build b/tools/lkl/lib/Build
new file mode 100644
index 000000000000..8b137891791f
--- /dev/null
+++ b/tools/lkl/lib/Build
@@ -0,0 +1 @@
+
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v3 13/26] lkl tools: skeleton for host side library
@ 2020-02-05  7:30   ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05  7:30 UTC (permalink / raw)
  To: linux-um
  Cc: linux-arch, Patrick Collins, Xiao Jia, Hajime Tazaki,
	Conrad Meyer, Octavian Purdila, Motomu Utsumi, Akira Moroo,
	Petros Angelatos, Thomas Liebetraut, Mark Stillwell,
	Ben Wolsieffer, linux-kernel-library, Michael Zimmermann,
	Luca Dariz, Yuan Liu

From: Octavian Purdila <tavi.purdila@gmail.com>

This patch adds the skeleton for the host library.

The host library is implementing the host operations needed by LKL and
is split into host dependent (depends on a specific host, e.g. POSIX
hosts) and host independent parts (will work on all supported hosts).

Cc: Ben Wolsieffer <benwolsieffer@gmail.com>
Cc: Conrad Meyer <cem@FreeBSD.org>
Cc: H.K. Jerry Chu <hkchu@google.com>
Cc: Luca Dariz <luca.dariz@gmail.com>
Cc: Mark Stillwell <mark@stillwell.me>
Cc: Michael Zimmermann <sigmaepsilon92@gmail.com>
Cc: Motomu Utsumi <motomuman@gmail.com>
Cc: Patrick Collins <pscollins@google.com>
Cc: Petros Angelatos <petrosagg@gmail.com>
Cc: Thomas Liebetraut <thomas@tommie-lie.de>
Cc: Xiao Jia <xiaoj@google.com>
Cc: Yuan Liu <liuyuan@google.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/lkl/.gitignore         |   4 +
 tools/lkl/Build              |   0
 tools/lkl/Makefile           | 125 ++++++++++++
 tools/lkl/Makefile.autoconf  |  65 +++++++
 tools/lkl/Targets            |   3 +
 tools/lkl/include/.gitignore |   1 +
 tools/lkl/include/lkl.h      | 358 +++++++++++++++++++++++++++++++++++
 tools/lkl/include/lkl_host.h |  19 ++
 tools/lkl/lib/.gitignore     |   3 +
 tools/lkl/lib/Build          |   1 +
 10 files changed, 579 insertions(+)
 create mode 100644 tools/lkl/.gitignore
 create mode 100644 tools/lkl/Build
 create mode 100644 tools/lkl/Makefile
 create mode 100644 tools/lkl/Makefile.autoconf
 create mode 100644 tools/lkl/Targets
 create mode 100644 tools/lkl/include/.gitignore
 create mode 100644 tools/lkl/include/lkl.h
 create mode 100644 tools/lkl/include/lkl_host.h
 create mode 100644 tools/lkl/lib/.gitignore
 create mode 100644 tools/lkl/lib/Build

diff --git a/tools/lkl/.gitignore b/tools/lkl/.gitignore
new file mode 100644
index 000000000000..1aed58bfe171
--- /dev/null
+++ b/tools/lkl/.gitignore
@@ -0,0 +1,4 @@
+Makefile.conf
+include/lkl_autoconf.h
+tests/autoconf.sh
+bin/stat
diff --git a/tools/lkl/Build b/tools/lkl/Build
new file mode 100644
index 000000000000..e69de29bb2d1
diff --git a/tools/lkl/Makefile b/tools/lkl/Makefile
new file mode 100644
index 000000000000..11ea7e9095bb
--- /dev/null
+++ b/tools/lkl/Makefile
@@ -0,0 +1,125 @@
+# Do not use make's built-in rules
+# (this improves performance and avoids hard-to-debug behaviour);
+# also do not print "Entering directory..." messages from make
+.SUFFIXES:
+MAKEFLAGS += -r --no-print-directory
+
+KCONFIG?=defconfig
+
+ifneq ($(silent),1)
+  ifneq ($(V),1)
+	QUIET_AUTOCONF       = @echo '  AUTOCONF '$@;
+	Q = @
+  endif
+endif
+
+PREFIX   := /usr
+
+ifeq (,$(srctree))
+  srctree := $(patsubst %/,%,$(dir $(shell pwd)))
+  srctree := $(patsubst %/,%,$(dir $(srctree)))
+endif
+export srctree
+
+-include $(srctree)/tools/scripts/Makefile.include
+
+# OUTPUT fixup should be *after* include ../scripts/Makefile.include
+ifneq ($(OUTPUT),)
+  OUTPUT := $(OUTPUT)/tools/lkl/
+else
+  OUTPUT := $(CURDIR)/
+endif
+export OUTPUT
+
+
+all:
+
+conf: $(OUTPUT)Makefile.conf
+
+$(OUTPUT)Makefile.conf: Makefile.autoconf
+	$(call QUIET_AUTOCONF, headers)$(MAKE) -f Makefile.autoconf -s
+
+-include $(OUTPUT)Makefile.conf
+
+export CFLAGS += -I$(OUTPUT)/include -Iinclude -Wall -g -O2 -Wextra \
+	 -Wno-unused-parameter \
+	 -Wno-missing-field-initializers -fno-strict-aliasing
+
+-include Targets
+
+TARGETS := $(progs-y:%=$(OUTPUT)%$(EXESUF))
+TARGETS += $(libs-y:%=$(OUTPUT)%$(SOSUF))
+all: $(TARGETS)
+
+# this workaround is for FreeBSD
+bin/stat:
+	$(Q)mkdir -p bin/
+ifeq ($(LKL_HOST_CONFIG_BSD),y)
+	$(Q)ln -sf `which gnustat` bin/stat
+	$(Q)ln -sf `which gsed` bin/sed
+else
+	$(Q)touch bin/stat
+endif
+
+# rule to build lkl.o
+$(OUTPUT)lib/lkl.o: bin/stat
+	$(Q)$(MAKE) -C $(srctree) ARCH=um UMMODE=library $(KOPT) $(KCONFIG)
+# this workaround is for arm32 linker (ld.gold)
+	$(Q)export PATH=$(shell pwd)/bin/:${PATH} ;\
+	$(MAKE) -C $(srctree) ARCH=um UMMODE=library $(KOPT) install INSTALL_PATH=$(OUTPUT)
+
+# rules to link libs
+$(OUTPUT)%$(SOSUF): LDFLAGS += -shared
+$(OUTPUT)%$(SOSUF): $(OUTPUT)%-in.o $(OUTPUT)liblkl.a
+	$(QUIET_LINK)$(CC) $(LDFLAGS) $(LDFLAGS_$*-y) -o $@ $^ $(LDLIBS) $(LDLIBS_$*-y)
+
+# liblkl is special
+$(OUTPUT)liblkl$(SOSUF): $(OUTPUT)%-in.o $(OUTPUT)lib/lkl.o
+$(OUTPUT)liblkl.a: $(OUTPUT)lib/liblkl-in.o $(OUTPUT)lib/lkl.o
+	$(QUIET_AR)$(AR) -rc $@ $^
+
+# rule to link programs
+$(OUTPUT)%$(EXESUF): $(OUTPUT)%-in.o $(OUTPUT)liblkl.a
+	$(QUIET_LINK)$(CC) $(LDFLAGS) $(LDFLAGS_$*-y) -o $@ $^ $(LDLIBS) $(LDLIBS_$*-y)
+
+# rule to build objects
+$(OUTPUT)%-in.o: $(OUTPUT)lib/lkl.o FORCE
+	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
+
+
+clean:
+	$(call QUIET_CLEAN, objects)find $(OUTPUT) -name '*.o' -delete -o -name '\.*.cmd'\
+	 -delete -o -name '\.*.d' -delete
+	$(call QUIET_CLEAN, headers)$(RM) -r $(OUTPUT)/include/lkl/
+	$(call QUIET_CLEAN, liblkl.a)$(RM) $(OUTPUT)/liblkl.a
+	$(call QUIET_CLEAN, targets)$(RM) $(TARGETS) bin/stat
+
+clean-conf: clean
+	$(call QUIET_CLEAN, Makefile.conf)$(RM) $(OUTPUT)/Makefile.conf
+
+headers_install: $(TARGETS)
+	$(call QUIET_INSTALL, headers) \
+	    install -d $(DESTDIR)$(PREFIX)/include ; \
+	    install -m 644 include/lkl.h include/lkl_host.h $(OUTPUT)include/lkl_autoconf.h \
+	      include/lkl_config.h $(DESTDIR)$(PREFIX)/include ; \
+	    cp -r $(OUTPUT)include/lkl $(DESTDIR)$(PREFIX)/include
+
+libraries_install: $(libs-y:%=$(OUTPUT)%$(SOSUF)) $(OUTPUT)liblkl.a
+	$(call QUIET_INSTALL, libraries) \
+	    install -d $(DESTDIR)$(PREFIX)/lib ; \
+	    install -m 644 $^ $(DESTDIR)$(PREFIX)/lib
+
+programs_install: $(progs-y:%=$(OUTPUT)%$(EXESUF))
+	$(call QUIET_INSTALL, programs) \
+	    install -d $(DESTDIR)$(PREFIX)/bin ; \
+	    install -m 755 $^ $(DESTDIR)$(PREFIX)/bin
+
+install: headers_install libraries_install programs_install
+
+
+FORCE: ;
+.PHONY: all clean FORCE
+.PHONY: headers_install libraries_install programs_install install
+.NOTPARALLEL : lib/lkl.o
+.SECONDARY:
+
diff --git a/tools/lkl/Makefile.autoconf b/tools/lkl/Makefile.autoconf
new file mode 100644
index 000000000000..268c367d9962
--- /dev/null
+++ b/tools/lkl/Makefile.autoconf
@@ -0,0 +1,65 @@
+POSIX_HOSTS=elf64-x86-64 elf32-i386 elf64-x86-64-freebsd
+
+define set_autoconf_var
+  $(shell echo "#define LKL_HOST_CONFIG_$(1) $(2)" \
+	  >> $(OUTPUT)/include/lkl_autoconf.h)
+  $(shell echo "LKL_HOST_CONFIG_$(1)=$(2)" >> $(OUTPUT)/tests/autoconf.sh)
+  export LKL_HOST_CONFIG_$(1)=$(2)
+endef
+
+define find_include
+  $(eval include_paths=$(shell $(CC) -E -Wp,-v -xc /dev/null 2>&1 | grep '^ '))
+  $(foreach f, $(include_paths), $(wildcard $(f)/$(1)))
+endef
+
+define is_defined
+$(shell $(CC) -dM -E - </dev/null | grep $(1))
+endef
+
+define bsd_host
+  $(call set_autoconf_var,BSD,y)
+endef
+
+define arm_host
+  $(call set_autoconf_var,ARM,y)
+endef
+
+define aarch64_host
+  $(call set_autoconf_var,AARCH64,y)
+endef
+
+define posix_host
+  $(call set_autoconf_var,POSIX,y)
+  LDFLAGS += -pie
+  CFLAGS += -fPIC -pthread
+  SOSUF := .so
+  LDLIBS += -lrt -lpthread
+  $(if $(filter $(1),elf64-x86-64-freebsd),$(call bsd_host))
+  $(if $(filter $(1),elf32-littlearm),$(call arm_host))
+  $(if $(filter $(1),elf64-littleaarch64),$(call aarch64_host))
+  $(if $(strip $(call find_include,fuse.h)),$(call set_autoconf_var,FUSE,y))
+  $(if $(strip $(call find_include,archive.h)),$(call set_autoconf_var,ARCHIVE,y))
+  $(if $(filter $(1),elf64-x86-64-freebsd),$(call set_autoconf_var,NEEDS_LARGP,y))
+  $(if $(filter $(1),elf32-i386),$(call set_autoconf_var,I386,y))
+endef
+
+define do_autoconf
+  export CROSS_COMPILE := $(CROSS_COMPILE)
+  export CC := $(CROSS_COMPILE)gcc
+  export LD := $(CROSS_COMPILE)ld
+  export AR := $(CROSS_COMPILE)ar
+  $(eval LD := $(CROSS_COMPILE)ld)
+  $(eval CC := $(CROSS_COMPILE)gcc)
+  $(eval LD_FMT := $(shell $(LD) -r -print-output-format))
+  $(if $(filter $(LD_FMT),$(POSIX_HOSTS)),$(call posix_host,$(LD_FMT)))
+endef
+
+export do_autoconf
+
+
+$(OUTPUT)Makefile.conf: Makefile.autoconf
+	$(shell mkdir -p $(OUTPUT)/include)
+	$(shell mkdir -p $(OUTPUT)/tests)
+	$(shell echo -n "" > $(OUTPUT)/include/lkl_autoconf.h)
+	$(shell echo -n "" > $(OUTPUT)/tests/autoconf.sh)
+	@echo "$$do_autoconf" > $(OUTPUT)/Makefile.conf
diff --git a/tools/lkl/Targets b/tools/lkl/Targets
new file mode 100644
index 000000000000..24c985e64638
--- /dev/null
+++ b/tools/lkl/Targets
@@ -0,0 +1,3 @@
+libs-y += lib/liblkl
+
+
diff --git a/tools/lkl/include/.gitignore b/tools/lkl/include/.gitignore
new file mode 100644
index 000000000000..c41a463c898d
--- /dev/null
+++ b/tools/lkl/include/.gitignore
@@ -0,0 +1 @@
+lkl/
\ No newline at end of file
diff --git a/tools/lkl/include/lkl.h b/tools/lkl/include/lkl.h
new file mode 100644
index 000000000000..4b95d0ef8e5b
--- /dev/null
+++ b/tools/lkl/include/lkl.h
@@ -0,0 +1,358 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_H
+#define _LKL_H
+
+#include "lkl_autoconf.h"
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#define _LKL_LIBC_COMPAT_H
+
+#ifdef __cplusplus
+#define class __lkl__class
+#endif
+
+/*
+ * Avoid collisions between Android which defines __unused and
+ * linux/icmp.h which uses __unused as a structure field.
+ */
+#pragma push_macro("__unused")
+#undef __unused
+
+#include <lkl/asm/syscalls.h>
+
+#pragma pop_macro("__unused")
+
+#ifdef __cplusplus
+#undef class
+#endif
+
+#if defined(__MINGW32__)
+#define strtok_r strtok_s
+#define inet_pton lkl_inet_pton
+
+int inet_pton(int af, const char *src, void *dst);
+#endif
+
+#if __LKL__BITS_PER_LONG == 64
+#define lkl_sys_fstatat lkl_sys_newfstatat
+#define lkl_sys_fstat lkl_sys_newfstat
+
+#else
+#define __lkl__NR_fcntl __lkl__NR_fcntl64
+
+#define lkl_stat lkl_stat64
+#define lkl_sys_stat lkl_sys_stat64
+#define lkl_sys_lstat lkl_sys_lstat64
+#define lkl_sys_truncate lkl_sys_truncate64
+#define lkl_sys_ftruncate lkl_sys_ftruncate64
+#define lkl_sys_sendfile lkl_sys_sendfile64
+#define lkl_sys_fstatat lkl_sys_fstatat64
+#define lkl_sys_fstat lkl_sys_fstat64
+#define lkl_sys_fcntl lkl_sys_fcntl64
+
+#define lkl_statfs lkl_statfs64
+
+static inline int lkl_sys_statfs(const char *path, struct lkl_statfs *buf)
+{
+	return lkl_sys_statfs64(path, sizeof(*buf), buf);
+}
+
+static inline int lkl_sys_fstatfs(unsigned int fd, struct lkl_statfs *buf)
+{
+	return lkl_sys_fstatfs64(fd, sizeof(*buf), buf);
+}
+
+#define lkl_sys_nanosleep lkl_sys_nanosleep_time32
+static inline int lkl_sys_nanosleep_time32(struct __lkl__kernel_timespec *rqtp,
+					   struct __lkl__kernel_timespec *rmtp)
+{
+	long p[6] = {(long)rqtp, (long)rmtp, 0, 0, 0, 0};
+
+	return lkl_syscall(__lkl__NR_nanosleep, p);
+}
+
+#endif
+
+static inline int lkl_sys_stat(const char *path, struct lkl_stat *buf)
+{
+	return lkl_sys_fstatat(LKL_AT_FDCWD, path, buf, 0);
+}
+
+static inline int lkl_sys_lstat(const char *path, struct lkl_stat *buf)
+{
+	return lkl_sys_fstatat(LKL_AT_FDCWD, path, buf,
+			       LKL_AT_SYMLINK_NOFOLLOW);
+}
+
+#ifdef __lkl__NR_llseek
+/**
+ * lkl_sys_lseek - wrapper for lkl_sys_llseek
+ */
+static inline long long lkl_sys_lseek(unsigned int fd, __lkl__kernel_loff_t off,
+				      unsigned int whence)
+{
+	long long res;
+	long ret = lkl_sys_llseek(fd, off >> 32, off & 0xffffffff, &res,
+				  whence);
+
+	return ret < 0 ? ret : res;
+}
+#endif
+
+static inline void *lkl_sys_mmap(void *addr, size_t length, int prot, int flags,
+				 int fd, off_t offset)
+{
+	return (void *)lkl_sys_mmap_pgoff((long)addr, length, prot, flags, fd,
+					  offset >> 12);
+}
+
+#define lkl_sys_mmap2 lkl_sys_mmap_pgoff
+
+#ifdef __lkl__NR_openat
+/**
+ * lkl_sys_open - wrapper for lkl_sys_openat
+ */
+static inline long lkl_sys_open(const char *file, int flags, int mode)
+{
+	return lkl_sys_openat(LKL_AT_FDCWD, file, flags, mode);
+}
+
+/**
+ * lkl_sys_creat - wrapper for lkl_sys_openat
+ */
+static inline long lkl_sys_creat(const char *file, int mode)
+{
+	return lkl_sys_openat(LKL_AT_FDCWD, file,
+			      LKL_O_CREAT|LKL_O_WRONLY|LKL_O_TRUNC, mode);
+}
+#endif
+
+
+#ifdef __lkl__NR_faccessat
+/**
+ * lkl_sys_access - wrapper for lkl_sys_faccessat
+ */
+static inline long lkl_sys_access(const char *file, int mode)
+{
+	return lkl_sys_faccessat(LKL_AT_FDCWD, file, mode);
+}
+#endif
+
+#ifdef __lkl__NR_fchownat
+/**
+ * lkl_sys_chown - wrapper for lkl_sys_fchownat
+ */
+static inline long lkl_sys_chown(const char *path, lkl_uid_t uid, lkl_gid_t gid)
+{
+	return lkl_sys_fchownat(LKL_AT_FDCWD, path, uid, gid, 0);
+}
+#endif
+
+#ifdef __lkl__NR_fchmodat
+/**
+ * lkl_sys_chmod - wrapper for lkl_sys_fchmodat
+ */
+static inline long lkl_sys_chmod(const char *path, mode_t mode)
+{
+	return lkl_sys_fchmodat(LKL_AT_FDCWD, path, mode);
+}
+#endif
+
+#ifdef __lkl__NR_linkat
+/**
+ * lkl_sys_link - wrapper for lkl_sys_linkat
+ */
+static inline long lkl_sys_link(const char *existing, const char *new)
+{
+	return lkl_sys_linkat(LKL_AT_FDCWD, existing, LKL_AT_FDCWD, new, 0);
+}
+#endif
+
+#ifdef __lkl__NR_unlinkat
+/**
+ * lkl_sys_unlink - wrapper for lkl_sys_unlinkat
+ */
+static inline long lkl_sys_unlink(const char *path)
+{
+	return lkl_sys_unlinkat(LKL_AT_FDCWD, path, 0);
+}
+#endif
+
+#ifdef __lkl__NR_symlinkat
+/**
+ * lkl_sys_symlink - wrapper for lkl_sys_symlinkat
+ */
+static inline long lkl_sys_symlink(const char *existing, const char *new)
+{
+	return lkl_sys_symlinkat(existing, LKL_AT_FDCWD, new);
+}
+#endif
+
+#ifdef __lkl__NR_readlinkat
+/**
+ * lkl_sys_readlink - wrapper for lkl_sys_readlinkat
+ */
+static inline long lkl_sys_readlink(const char *path, char *buf, size_t bufsize)
+{
+	return lkl_sys_readlinkat(LKL_AT_FDCWD, path, buf, bufsize);
+}
+#endif
+
+#ifdef __lkl__NR_renameat
+/**
+ * lkl_sys_rename - wrapper for lkl_sys_renameat
+ */
+static inline long lkl_sys_rename(const char *old, const char *new)
+{
+	return lkl_sys_renameat(LKL_AT_FDCWD, old, LKL_AT_FDCWD, new);
+}
+#endif
+
+#ifdef __lkl__NR_mkdirat
+/**
+ * lkl_sys_mkdir - wrapper for lkl_sys_mkdirat
+ */
+static inline long lkl_sys_mkdir(const char *path, mode_t mode)
+{
+	return lkl_sys_mkdirat(LKL_AT_FDCWD, path, mode);
+}
+#endif
+
+#ifdef __lkl__NR_unlinkat
+/**
+ * lkl_sys_rmdir - wrapper for lkl_sys_unlinkrat
+ */
+static inline long lkl_sys_rmdir(const char *path)
+{
+	return lkl_sys_unlinkat(LKL_AT_FDCWD, path, LKL_AT_REMOVEDIR);
+}
+#endif
+
+#ifdef __lkl__NR_mknodat
+/**
+ * lkl_sys_mknod - wrapper for lkl_sys_mknodat
+ */
+static inline long lkl_sys_mknod(const char *path, mode_t mode, dev_t dev)
+{
+	return lkl_sys_mknodat(LKL_AT_FDCWD, path, mode, dev);
+}
+#endif
+
+#ifdef __lkl__NR_pipe2
+/**
+ * lkl_sys_pipe - wrapper for lkl_sys_pipe2
+ */
+static inline long lkl_sys_pipe(int fd[2])
+{
+	return lkl_sys_pipe2(fd, 0);
+}
+#endif
+
+#ifdef __lkl__NR_sendto
+/**
+ * lkl_sys_send - wrapper for lkl_sys_sendto
+ */
+static inline long lkl_sys_send(int fd, void *buf, size_t len, int flags)
+{
+	return lkl_sys_sendto(fd, buf, len, flags, 0, 0);
+}
+#endif
+
+#ifdef __lkl__NR_recvfrom
+/**
+ * lkl_sys_recv - wrapper for lkl_sys_recvfrom
+ */
+static inline long lkl_sys_recv(int fd, void *buf, size_t len, int flags)
+{
+	return lkl_sys_recvfrom(fd, buf, len, flags, 0, 0);
+}
+#endif
+
+#ifdef __lkl__NR_pselect6
+/**
+ * lkl_sys_select - wrapper for lkl_sys_pselect
+ */
+static inline long lkl_sys_select(int n, lkl_fd_set *rfds, lkl_fd_set *wfds,
+				  lkl_fd_set *efds, struct lkl_timeval *tv)
+{
+	long data[2] = { 0, _LKL_NSIG/8 };
+	struct lkl_timespec ts;
+	lkl_time_t extra_secs;
+	const lkl_time_t max_time = ((1ULL<<8)*sizeof(time_t)-1)-1;
+
+	if (tv) {
+		if (tv->tv_sec < 0 || tv->tv_usec < 0)
+			return -LKL_EINVAL;
+
+		extra_secs = tv->tv_usec / 1000000;
+		ts.tv_nsec = tv->tv_usec % 1000000 * 1000;
+		ts.tv_sec = extra_secs > max_time - tv->tv_sec ?
+			max_time : tv->tv_sec + extra_secs;
+	}
+	return lkl_sys_pselect6(n, rfds, wfds, efds, tv ?
+				(struct __lkl__kernel_timespec *)&ts : 0, data);
+}
+#endif
+
+#ifdef __lkl__NR_ppoll
+/**
+ * lkl_sys_poll - wrapper for lkl_sys_ppoll
+ */
+static inline long lkl_sys_poll(struct lkl_pollfd *fds, int n, int timeout)
+{
+	return lkl_sys_ppoll(fds, n, timeout >= 0 ?
+			     (struct __lkl__kernel_timespec *)
+			     &((struct lkl_timespec){ .tv_sec = timeout/1000,
+				   .tv_nsec = timeout%1000*1000000 }) : 0,
+			     0, _LKL_NSIG/8);
+}
+#endif
+
+#ifdef __lkl__NR_epoll_create1
+/**
+ * lkl_sys_epoll_create - wrapper for lkl_sys_epoll_create1
+ */
+static inline long lkl_sys_epoll_create(int size)
+{
+	return lkl_sys_epoll_create1(0);
+}
+#endif
+
+#ifdef __lkl__NR_epoll_pwait
+/**
+ * lkl_sys_epoll_wait - wrapper for lkl_sys_epoll_pwait
+ */
+static inline long lkl_sys_epoll_wait(int fd, struct lkl_epoll_event *ev,
+				      int cnt, int to)
+{
+	return lkl_sys_epoll_pwait(fd, ev, cnt, to, 0, _LKL_NSIG/8);
+}
+#endif
+
+
+
+/**
+ * lkl_strerror - returns a string describing the given error code
+ *
+ * @err - error code
+ * @returns - string for the given error code
+ */
+const char *lkl_strerror(int err);
+
+/**
+ * lkl_perror - prints a string describing the given error code
+ *
+ * @msg - prefix for the error message
+ * @err - error code
+ */
+void lkl_perror(char *msg, int err);
+
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif
diff --git a/tools/lkl/include/lkl_host.h b/tools/lkl/include/lkl_host.h
new file mode 100644
index 000000000000..b5f96096fe69
--- /dev/null
+++ b/tools/lkl/include/lkl_host.h
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_HOST_H
+#define _LKL_HOST_H
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include <lkl/asm/host_ops.h>
+#include <lkl.h>
+
+extern struct lkl_host_operations lkl_host_ops;
+
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif
diff --git a/tools/lkl/lib/.gitignore b/tools/lkl/lib/.gitignore
new file mode 100644
index 000000000000..427ae0273fdd
--- /dev/null
+++ b/tools/lkl/lib/.gitignore
@@ -0,0 +1,3 @@
+lkl.o
+liblkl.a
+
diff --git a/tools/lkl/lib/Build b/tools/lkl/lib/Build
new file mode 100644
index 000000000000..8b137891791f
--- /dev/null
+++ b/tools/lkl/lib/Build
@@ -0,0 +1 @@
+
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v3 14/26] lkl tools: host lib: add utilities functions
  2020-02-05  7:30 [RFC v3 00/26] Unifying LKL into UML Hajime Tazaki
@ 2020-02-05  7:30   ` Hajime Tazaki
       [not found] ` <cover.1580882335.git.thehajime-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
                     ` (25 subsequent siblings)
  26 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05  7:30 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Conrad Meyer, Patrick Collins, Yuan Liu, Motomu Utsumi,
	Hajime Tazaki

From: Octavian Purdila <tavi.purdila@gmail.com>

Add basic utility functions for getting a string from a kernel error
code and a fprintf like function that uses the host print
operation. The latter is useful for informing the user about errors
that occur in the host library.

Other configuration and debug utilities are also added.

Cc: Conrad Meyer <cem@FreeBSD.org>
Cc: Patrick Collins <pscollins@google.com>
Cc: Yuan Liu <liuyuan@google.com>
Cc: Motomu Utsumi <motomuman@gmail.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/lkl/include/lkl_config.h |  61 +++
 tools/lkl/include/lkl_host.h   |   7 +
 tools/lkl/lib/Build            |   7 +
 tools/lkl/lib/config.c         | 744 +++++++++++++++++++++++++++++++++
 tools/lkl/lib/dbg.c            | 300 +++++++++++++
 tools/lkl/lib/dbg_handler.c    |  44 ++
 tools/lkl/lib/endian.h         |  31 ++
 tools/lkl/lib/jmp_buf.c        |  14 +
 tools/lkl/lib/jmp_buf.h        |   8 +
 tools/lkl/lib/utils.c          | 266 ++++++++++++
 10 files changed, 1482 insertions(+)
 create mode 100644 tools/lkl/include/lkl_config.h
 create mode 100644 tools/lkl/lib/config.c
 create mode 100644 tools/lkl/lib/dbg.c
 create mode 100644 tools/lkl/lib/dbg_handler.c
 create mode 100644 tools/lkl/lib/endian.h
 create mode 100644 tools/lkl/lib/jmp_buf.c
 create mode 100644 tools/lkl/lib/jmp_buf.h
 create mode 100644 tools/lkl/lib/utils.c

diff --git a/tools/lkl/include/lkl_config.h b/tools/lkl/include/lkl_config.h
new file mode 100644
index 000000000000..d3edf8b414cf
--- /dev/null
+++ b/tools/lkl/include/lkl_config.h
@@ -0,0 +1,61 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_LIB_CONFIG_H
+#define _LKL_LIB_CONFIG_H
+
+#define LKL_CONFIG_JSON_TOKEN_MAX 300
+
+struct lkl_config_iface {
+	struct lkl_config_iface *next;
+	struct lkl_netdev *nd;
+
+	/* OBSOLETE: should use IFTYPE and IFPARAMS */
+	char *iftap;
+	char *iftype;
+	char *ifparams;
+	char *ifmtu_str;
+	char *ifip;
+	char *ifipv6;
+	char *ifgateway;
+	char *ifgateway6;
+	char *ifmac_str;
+	char *ifnetmask_len;
+	char *ifnetmask6_len;
+	char *ifoffload_str;
+	char *ifneigh_entries;
+	char *ifqdisc_entries;
+};
+
+struct lkl_config {
+	int ifnum;
+	struct lkl_config_iface *ifaces;
+
+	char *gateway;
+	char *gateway6;
+	char *debug;
+	char *mount;
+	/* single_cpu mode:
+	 * 0: Don't pin to single CPU (default).
+	 * 1: Pin only LKL kernel threads to single CPU.
+	 * 2: Pin all LKL threads to single CPU including all LKL kernel threads
+	 * and device polling threads. Avoid this mode if having busy polling
+	 * threads.
+	 *
+	 * mode 2 can achieve better TCP_RR but worse TCP_STREAM than mode 1.
+	 * You should choose the best for your application and virtio device
+	 * type.
+	 */
+	char *single_cpu;
+	char *sysctls;
+	char *boot_cmdline;
+	char *dump;
+	char *delay_main;
+};
+
+int lkl_load_config_json(struct lkl_config *cfg, char *jstr);
+int lkl_load_config_env(struct lkl_config *cfg);
+void lkl_show_config(struct lkl_config *cfg);
+int lkl_load_config_pre(struct lkl_config *cfg);
+int lkl_load_config_post(struct lkl_config *cfg);
+int lkl_unload_config(struct lkl_config *cfg);
+
+#endif /* _LKL_LIB_CONFIG_H */
diff --git a/tools/lkl/include/lkl_host.h b/tools/lkl/include/lkl_host.h
index b5f96096fe69..85e80eb4ad0d 100644
--- a/tools/lkl/include/lkl_host.h
+++ b/tools/lkl/include/lkl_host.h
@@ -11,6 +11,13 @@ extern "C" {
 
 extern struct lkl_host_operations lkl_host_ops;
 
+/**
+ * lkl_printf - print a message via the host print operation
+ *
+ * @fmt: printf like format string
+ */
+int lkl_printf(const char *fmt, ...);
+
 
 #ifdef __cplusplus
 }
diff --git a/tools/lkl/lib/Build b/tools/lkl/lib/Build
index 8b137891791f..658bfa865b9c 100644
--- a/tools/lkl/lib/Build
+++ b/tools/lkl/lib/Build
@@ -1 +1,8 @@
+CFLAGS_config.o += -I$(srctree)/tools/perf/pmu-events
 
+liblkl-y += jmp_buf.o
+liblkl-y += utils.o
+liblkl-y += dbg.o
+liblkl-y += dbg_handler.o
+liblkl-y += ../../perf/pmu-events/jsmn.o
+liblkl-y += config.o
diff --git a/tools/lkl/lib/config.c b/tools/lkl/lib/config.c
new file mode 100644
index 000000000000..37f8ac4d942a
--- /dev/null
+++ b/tools/lkl/lib/config.c
@@ -0,0 +1,744 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdlib.h>
+#define _HAVE_STRING_ARCH_strtok_r
+#include <string.h>
+#ifndef __MINGW32__
+#include <arpa/inet.h>
+#endif
+#include <lkl_host.h>
+#include <lkl_config.h>
+
+#include "jsmn.h"
+
+static int jsoneq(const char *json, jsmntok_t *tok, const char *s)
+{
+	if (tok->type == JSMN_STRING &&
+		(int) strlen(s) == tok->end - tok->start &&
+		strncmp(json + tok->start, s, tok->end - tok->start) == 0) {
+		return 0;
+	}
+	return -1;
+}
+
+static int cfgcpy(char **to, char *from)
+{
+	if (!from)
+		return 0;
+	if (*to)
+		free(*to);
+	*to = (char *)malloc((strlen(from) + 1) * sizeof(char));
+	if (*to == NULL) {
+		lkl_printf("malloc failed\n");
+		return -1;
+	}
+	strcpy(*to, from);
+	return 0;
+}
+
+static int cfgncpy(char **to, char *from, int len)
+{
+	if (!from)
+		return 0;
+	if (*to)
+		free(*to);
+	*to = (char *)malloc((len + 1) * sizeof(char));
+	if (*to == NULL) {
+		lkl_printf("malloc failed\n");
+		return -1;
+	}
+	strncpy(*to, from, len + 1);
+	(*to)[len] = '\0';
+	return 0;
+}
+
+static int parse_ifarr(struct lkl_config *cfg,
+		jsmntok_t *toks, char *jstr, int startpos)
+{
+	int ifidx, pos, posend, ret;
+	char **cfgptr;
+	struct lkl_config_iface *iface, *prev = NULL;
+
+	if (!cfg || !toks || !jstr)
+		return -1;
+	pos = startpos;
+	pos++;
+	if (toks[pos].type != JSMN_ARRAY) {
+		lkl_printf("unexpected json type, json array expected\n");
+		return -1;
+	}
+
+	cfg->ifnum = toks[pos].size;
+	pos++;
+	iface = cfg->ifaces;
+
+	for (ifidx = 0; ifidx < cfg->ifnum; ifidx++) {
+		if (toks[pos].type != JSMN_OBJECT) {
+			lkl_printf("object json type expected\n");
+			return -1;
+		}
+
+		posend = pos + toks[pos].size;
+		pos++;
+		iface = malloc(sizeof(struct lkl_config_iface));
+		memset(iface, 0, sizeof(struct lkl_config_iface));
+
+		if (prev)
+			prev->next = iface;
+		else
+			cfg->ifaces = iface;
+		prev = iface;
+
+		for (; pos < posend; pos += 2) {
+			if (toks[pos].type != JSMN_STRING) {
+				lkl_printf("object json type expected\n");
+				return -1;
+			}
+			if (jsoneq(jstr, &toks[pos], "type") == 0) {
+				cfgptr = &iface->iftype;
+			} else if (jsoneq(jstr, &toks[pos], "param") == 0) {
+				cfgptr = &iface->ifparams;
+			} else if (jsoneq(jstr, &toks[pos], "mtu") == 0) {
+				cfgptr = &iface->ifmtu_str;
+			} else if (jsoneq(jstr, &toks[pos], "ip") == 0) {
+				cfgptr = &iface->ifip;
+			} else if (jsoneq(jstr, &toks[pos], "ipv6") == 0) {
+				cfgptr = &iface->ifipv6;
+			} else if (jsoneq(jstr, &toks[pos], "ifgateway") == 0) {
+				cfgptr = &iface->ifgateway;
+			} else if (jsoneq(jstr, &toks[pos],
+							"ifgateway6") == 0) {
+				cfgptr = &iface->ifgateway6;
+			} else if (jsoneq(jstr, &toks[pos], "mac") == 0) {
+				cfgptr = &iface->ifmac_str;
+			} else if (jsoneq(jstr, &toks[pos], "masklen") == 0) {
+				cfgptr = &iface->ifnetmask_len;
+			} else if (jsoneq(jstr, &toks[pos], "masklen6") == 0) {
+				cfgptr = &iface->ifnetmask6_len;
+			} else if (jsoneq(jstr, &toks[pos], "neigh") == 0) {
+				cfgptr = &iface->ifneigh_entries;
+			} else if (jsoneq(jstr, &toks[pos], "qdisc") == 0) {
+				cfgptr = &iface->ifqdisc_entries;
+			} else if (jsoneq(jstr, &toks[pos], "offload") == 0) {
+				cfgptr = &iface->ifoffload_str;
+			} else {
+				lkl_printf("unexpected key: %.*s\n",
+						toks[pos].end-toks[pos].start,
+						jstr + toks[pos].start);
+				return -1;
+			}
+			ret = cfgncpy(cfgptr, jstr + toks[pos+1].start,
+					toks[pos+1].end-toks[pos+1].start);
+			if (ret < 0)
+				return ret;
+		}
+	}
+	return pos - startpos;
+}
+
+#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]))
+
+int lkl_load_config_json(struct lkl_config *cfg, char *jstr)
+{
+	int pos, ret;
+	char **cfgptr;
+	jsmn_parser jp;
+	jsmntok_t toks[LKL_CONFIG_JSON_TOKEN_MAX];
+
+	if (!cfg || !jstr)
+		return -1;
+	jsmn_init(&jp);
+	ret = jsmn_parse(&jp, jstr, strlen(jstr), toks, ARRAY_SIZE(toks));
+	if (ret != JSMN_SUCCESS) {
+		lkl_printf("failed to parse json\n");
+		return -1;
+	}
+	if (toks[0].type != JSMN_OBJECT) {
+		lkl_printf("object json type expected\n");
+		return -1;
+	}
+	for (pos = 1; pos < jp.toknext; pos++) {
+		if (toks[pos].type != JSMN_STRING) {
+			lkl_printf("string json type expected\n");
+			return -1;
+		}
+		if (jsoneq(jstr, &toks[pos], "interfaces") == 0) {
+			ret = parse_ifarr(cfg, toks, jstr, pos);
+			if (ret < 0)
+				return ret;
+			pos += ret;
+			pos--;
+			continue;
+		}
+		if (jsoneq(jstr, &toks[pos], "gateway") == 0) {
+			cfgptr = &cfg->gateway;
+		} else if (jsoneq(jstr, &toks[pos], "gateway6") == 0) {
+			cfgptr = &cfg->gateway6;
+		} else if (jsoneq(jstr, &toks[pos], "debug") == 0) {
+			cfgptr = &cfg->debug;
+		} else if (jsoneq(jstr, &toks[pos], "mount") == 0) {
+			cfgptr = &cfg->mount;
+		} else if (jsoneq(jstr, &toks[pos], "singlecpu") == 0) {
+			cfgptr = &cfg->single_cpu;
+		} else if (jsoneq(jstr, &toks[pos], "sysctl") == 0) {
+			cfgptr = &cfg->sysctls;
+		} else if (jsoneq(jstr, &toks[pos], "boot_cmdline") == 0) {
+			cfgptr = &cfg->boot_cmdline;
+		} else if (jsoneq(jstr, &toks[pos], "dump") == 0) {
+			cfgptr = &cfg->dump;
+		} else if (jsoneq(jstr, &toks[pos], "delay_main") == 0) {
+			cfgptr = &cfg->delay_main;
+		} else {
+			lkl_printf("unexpected key in json %.*s\n",
+					toks[pos].end-toks[pos].start,
+					jstr + toks[pos].start);
+			return -1;
+		}
+		pos++;
+		ret = cfgncpy(cfgptr, jstr + toks[pos].start,
+				toks[pos].end-toks[pos].start);
+		if (ret < 0)
+			return ret;
+	}
+	return 0;
+}
+
+void lkl_show_config(struct lkl_config *cfg)
+{
+	struct lkl_config_iface *iface;
+	int i = 0;
+
+	if (!cfg)
+		return;
+	lkl_printf("gateway: %s\n", cfg->gateway);
+	lkl_printf("gateway6: %s\n", cfg->gateway6);
+	lkl_printf("debug: %s\n", cfg->debug);
+	lkl_printf("mount: %s\n", cfg->mount);
+	lkl_printf("singlecpu: %s\n", cfg->single_cpu);
+	lkl_printf("sysctl: %s\n", cfg->sysctls);
+	lkl_printf("cmdline: %s\n", cfg->boot_cmdline);
+	lkl_printf("dump: %s\n", cfg->dump);
+	lkl_printf("delay: %s\n", cfg->delay_main);
+
+	for (iface = cfg->ifaces; iface; iface = iface->next, i++) {
+		lkl_printf("ifmac[%d] = %s\n", i, iface->ifmac_str);
+		lkl_printf("ifmtu[%d] = %s\n", i, iface->ifmtu_str);
+		lkl_printf("iftype[%d] = %s\n", i, iface->iftype);
+		lkl_printf("ifparam[%d] = %s\n", i, iface->ifparams);
+		lkl_printf("ifip[%d] = %s\n", i, iface->ifip);
+		lkl_printf("ifmasklen[%d] = %s\n", i, iface->ifnetmask_len);
+		lkl_printf("ifgateway[%d] = %s\n", i, iface->ifgateway);
+		lkl_printf("ifip6[%d] = %s\n", i, iface->ifipv6);
+		lkl_printf("ifmasklen6[%d] = %s\n", i, iface->ifnetmask6_len);
+		lkl_printf("ifgateway6[%d] = %s\n", i, iface->ifgateway6);
+		lkl_printf("ifoffload[%d] = %s\n", i, iface->ifoffload_str);
+		lkl_printf("ifneigh[%d] = %s\n", i, iface->ifneigh_entries);
+		lkl_printf("ifqdisk[%d] = %s\n", i, iface->ifqdisc_entries);
+	}
+}
+
+int lkl_load_config_env(struct lkl_config *cfg)
+{
+	int ret;
+	char *enviftype = getenv("LKL_HIJACK_NET_IFTYPE");
+	char *envifparams = getenv("LKL_HIJACK_NET_IFPARAMS");
+	char *envmtu_str = getenv("LKL_HIJACK_NET_MTU");
+	char *envip = getenv("LKL_HIJACK_NET_IP");
+	char *envipv6 = getenv("LKL_HIJACK_NET_IPV6");
+	char *envifgateway = getenv("LKL_HIJACK_NET_IFGATEWAY");
+	char *envifgateway6 = getenv("LKL_HIJACK_NET_IFGATEWAY6");
+	char *envmac_str = getenv("LKL_HIJACK_NET_MAC");
+	char *envnetmask_len = getenv("LKL_HIJACK_NET_NETMASK_LEN");
+	char *envnetmask6_len = getenv("LKL_HIJACK_NET_NETMASK6_LEN");
+	char *envgateway = getenv("LKL_HIJACK_NET_GATEWAY");
+	char *envgateway6 = getenv("LKL_HIJACK_NET_GATEWAY6");
+	char *envdebug = getenv("LKL_HIJACK_DEBUG");
+	char *envmount = getenv("LKL_HIJACK_MOUNT");
+	char *envneigh_entries = getenv("LKL_HIJACK_NET_NEIGHBOR");
+	char *envqdisc_entries = getenv("LKL_HIJACK_NET_QDISC");
+	char *envsingle_cpu = getenv("LKL_HIJACK_SINGLE_CPU");
+	char *envoffload_str = getenv("LKL_HIJACK_OFFLOAD");
+	char *envsysctls = getenv("LKL_HIJACK_SYSCTL");
+	char *envboot_cmdline = getenv("LKL_HIJACK_BOOT_CMDLINE") ? : "";
+	char *envdump = getenv("LKL_HIJACK_DUMP");
+	struct lkl_config_iface *iface;
+
+	if (!cfg)
+		return -1;
+	if (enviftype)
+		cfg->ifnum = 1;
+
+	iface = malloc(sizeof(struct lkl_config_iface));
+	memset(iface, 0, sizeof(struct lkl_config_iface));
+
+	ret = cfgcpy(&iface->iftype, enviftype);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifparams, envifparams);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifmtu_str, envmtu_str);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifip, envip);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifipv6, envipv6);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifgateway, envifgateway);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifgateway6, envifgateway6);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifmac_str, envmac_str);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifnetmask_len, envnetmask_len);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifnetmask6_len, envnetmask6_len);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifoffload_str, envoffload_str);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifneigh_entries, envneigh_entries);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifqdisc_entries, envqdisc_entries);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&cfg->gateway, envgateway);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&cfg->gateway6, envgateway6);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&cfg->debug, envdebug);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&cfg->mount, envmount);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&cfg->single_cpu, envsingle_cpu);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&cfg->sysctls, envsysctls);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&cfg->boot_cmdline, envboot_cmdline);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&cfg->dump, envdump);
+	if (ret < 0)
+		return ret;
+	return 0;
+}
+
+static int parse_mac_str(char *mac_str, __lkl__u8 mac[LKL_ETH_ALEN])
+{
+	char delim[] = ":";
+	char *saveptr = NULL, *token = NULL;
+	int i = 0;
+
+	if (!mac_str)
+		return 0;
+
+	for (token = strtok_r(mac_str, delim, &saveptr);
+	     i < LKL_ETH_ALEN; i++) {
+		if (!token) {
+			/* The address is too short */
+			return -1;
+		}
+
+		mac[i] = (__lkl__u8) strtol(token, NULL, 16);
+		token = strtok_r(NULL, delim, &saveptr);
+	}
+
+	if (strtok_r(NULL, delim, &saveptr)) {
+		/* The address is too long */
+		return -1;
+	}
+
+	return 1;
+}
+
+/* Add permanent neighbor entries in the form of "ip|mac;ip|mac;..." */
+static void add_neighbor(int ifindex, char *entries)
+{
+	char *saveptr = NULL, *token = NULL;
+	char *ip = NULL, *mac_str = NULL;
+	int ret = 0;
+	__lkl__u8 mac[LKL_ETH_ALEN];
+	char ip_addr[16];
+	int af;
+
+	for (token = strtok_r(entries, ";", &saveptr); token;
+	     token = strtok_r(NULL, ";", &saveptr)) {
+		ip = strtok(token, "|");
+		mac_str = strtok(NULL, "|");
+		if (ip == NULL || mac_str == NULL || strtok(NULL, "|") != NULL)
+			return;
+
+		af = LKL_AF_INET;
+		ret = inet_pton(LKL_AF_INET, ip, ip_addr);
+		if (ret == 0) {
+			ret = inet_pton(LKL_AF_INET6, ip, ip_addr);
+			af = LKL_AF_INET6;
+		}
+		if (ret != 1) {
+			lkl_printf("Bad ip address: %s\n", ip);
+			return;
+		}
+
+		ret = parse_mac_str(mac_str, mac);
+		if (ret != 1) {
+			lkl_printf("Failed to parse mac: %s\n", mac_str);
+			return;
+		}
+		ret = lkl_add_neighbor(ifindex, af, ip_addr, mac);
+		if (ret) {
+			lkl_printf("Failed to add neighbor entry: %s\n",
+				   lkl_strerror(ret));
+			return;
+		}
+	}
+}
+
+/* We don't have an easy way to make FILE*s out of our fds, so we
+ * can't use e.g. fgets
+ */
+static int dump_file(char *path)
+{
+	int ret = -1, bytes_read = 0;
+	char str[1024] = { 0 };
+	int fd;
+
+	fd = lkl_sys_open(path, LKL_O_RDONLY, 0);
+
+	if (fd < 0) {
+		lkl_printf("%s lkl_sys_open %s: %s\n",
+			   __func__, path, lkl_strerror(fd));
+		return -1;
+	}
+
+	/* Need to print this out in order to make sense of the output */
+	lkl_printf("Reading from %s:\n==========\n", path);
+	while ((ret = lkl_sys_read(fd, str, sizeof(str) - 1)) > 0)
+		bytes_read += lkl_printf("%s", str);
+	lkl_printf("==========\n");
+
+	if (ret) {
+		lkl_printf("%s lkl_sys_read %s: %s\n",
+			   __func__, path, lkl_strerror(ret));
+		return -1;
+	}
+
+	return 0;
+}
+
+static void mount_cmds_exec(char *_cmds, int (*callback)(char *))
+{
+	char *saveptr = NULL, *token;
+	int ret = 0;
+	char *cmds = strdup(_cmds);
+
+	token = strtok_r(cmds, ",", &saveptr);
+
+	while (token && ret >= 0) {
+		ret = callback(token);
+		token = strtok_r(NULL, ",", &saveptr);
+	}
+
+	if (ret < 0)
+		lkl_printf("%s: failed parsing %s\n", __func__, _cmds);
+
+	free(cmds);
+}
+
+static int lkl_config_netdev_create(struct lkl_config *cfg,
+				    struct lkl_config_iface *iface)
+{
+	int ret, offload = 0;
+	struct lkl_netdev_args nd_args;
+	__lkl__u8 mac[LKL_ETH_ALEN] = {0};
+	struct lkl_netdev *nd = NULL;
+
+	if (iface->ifoffload_str)
+		offload = strtol(iface->ifoffload_str, NULL, 0);
+	memset(&nd_args, 0, sizeof(struct lkl_netdev_args));
+
+	if (!nd && iface->iftype && iface->ifparams) {
+	}
+
+	if (nd) {
+		if ((mac[0] != 0) || (mac[1] != 0) ||
+				(mac[2] != 0) || (mac[3] != 0) ||
+				(mac[4] != 0) || (mac[5] != 0)) {
+			nd_args.mac = mac;
+		} else {
+			ret = parse_mac_str(iface->ifmac_str, mac);
+
+			if (ret < 0) {
+				lkl_printf("failed to parse mac\n");
+				return -1;
+			} else if (ret > 0) {
+				nd_args.mac = mac;
+			} else {
+				nd_args.mac = NULL;
+			}
+		}
+
+		nd_args.offload = offload;
+		iface->nd = nd;
+	}
+	return 0;
+}
+
+static int lkl_config_netdev_configure(struct lkl_config *cfg,
+				       struct lkl_config_iface *iface)
+{
+	int ret, nd_ifindex = -1;
+	struct lkl_netdev *nd = iface->nd;
+
+	if (!nd) {
+		lkl_printf("no netdev available %s\n", iface ? iface->ifparams
+			   : "(null)");
+		return -1;
+	}
+
+	if (nd->id >= 0) {
+		nd_ifindex = lkl_netdev_get_ifindex(nd->id);
+		if (nd_ifindex > 0)
+			lkl_if_up(nd_ifindex);
+		else
+			lkl_printf(
+				"failed to get ifindex for netdev id %d: %s\n",
+				nd->id, lkl_strerror(nd_ifindex));
+	}
+
+	if (nd_ifindex >= 0 && iface->ifmtu_str) {
+		int mtu = atoi(iface->ifmtu_str);
+
+		ret = lkl_if_set_mtu(nd_ifindex, mtu);
+		if (ret < 0)
+			lkl_printf("failed to set MTU: %s\n",
+				   lkl_strerror(ret));
+	}
+
+	if (nd_ifindex >= 0 && iface->ifip && iface->ifnetmask_len) {
+		unsigned int addr;
+
+		if (inet_pton(LKL_AF_INET, iface->ifip,
+			      (struct lkl_in_addr *)&addr) != 1)
+			lkl_printf("Invalid ipv4 address: %s\n", iface->ifip);
+
+		int nmlen = atoi(iface->ifnetmask_len);
+
+		if (addr != LKL_INADDR_NONE && nmlen > 0 && nmlen < 32) {
+			ret = lkl_if_set_ipv4(nd_ifindex, addr, nmlen);
+			if (ret < 0)
+				lkl_printf("failed to set IPv4 address: %s\n",
+					   lkl_strerror(ret));
+		}
+		if (iface->ifgateway) {
+			unsigned int gwaddr;
+
+			if (inet_pton(LKL_AF_INET, iface->ifgateway,
+				      (struct lkl_in_addr *)&gwaddr) != 1)
+				lkl_printf("Invalid ipv4 gateway: %s\n",
+					   iface->ifgateway);
+
+			if (gwaddr != LKL_INADDR_NONE) {
+				ret = lkl_if_set_ipv4_gateway(nd_ifindex,
+						addr, nmlen, gwaddr);
+				if (ret < 0)
+					lkl_printf(
+						"failed to set v4 if gw: %s\n",
+						lkl_strerror(ret));
+			}
+		}
+	}
+
+	if (nd_ifindex >= 0 && iface->ifipv6 &&
+			iface->ifnetmask6_len) {
+		struct lkl_in6_addr addr;
+		unsigned int pflen = atoi(iface->ifnetmask6_len);
+
+		if (inet_pton(LKL_AF_INET6, iface->ifipv6,
+			      (struct lkl_in6_addr *)&addr) != 1) {
+			lkl_printf("Invalid ipv6 addr: %s\n",
+				   iface->ifipv6);
+		}  else {
+			ret = lkl_if_set_ipv6(nd_ifindex, &addr, pflen);
+			if (ret < 0)
+				lkl_printf("failed to set IPv6 address: %s\n",
+					   lkl_strerror(ret));
+		}
+		if (iface->ifgateway6) {
+			char gwaddr[16];
+
+			if (inet_pton(LKL_AF_INET6, iface->ifgateway6,
+								gwaddr) != 1) {
+				lkl_printf("Invalid ipv6 gateway: %s\n",
+					   iface->ifgateway6);
+			} else {
+				ret = lkl_if_set_ipv6_gateway(nd_ifindex,
+						&addr, pflen, gwaddr);
+				if (ret < 0)
+					lkl_printf(
+						"failed to set v6 if gw: %s\n",
+						lkl_strerror(ret));
+			}
+		}
+	}
+
+	if (nd_ifindex >= 0 && iface->ifneigh_entries)
+		add_neighbor(nd_ifindex, iface->ifneigh_entries);
+
+	if (nd_ifindex >= 0 && iface->ifqdisc_entries)
+		lkl_qdisc_parse_add(nd_ifindex, iface->ifqdisc_entries);
+
+	return 0;
+}
+
+static void free_cfgparam(char *cfgparam)
+{
+	if (cfgparam)
+		free(cfgparam);
+}
+
+static int lkl_clean_config(struct lkl_config *cfg)
+{
+	struct lkl_config_iface *iface;
+
+	if (!cfg)
+		return -1;
+
+	for (iface = cfg->ifaces; iface; iface = iface->next) {
+		free_cfgparam(iface->iftype);
+		free_cfgparam(iface->ifparams);
+		free_cfgparam(iface->ifmtu_str);
+		free_cfgparam(iface->ifip);
+		free_cfgparam(iface->ifipv6);
+		free_cfgparam(iface->ifgateway);
+		free_cfgparam(iface->ifgateway6);
+		free_cfgparam(iface->ifmac_str);
+		free_cfgparam(iface->ifnetmask_len);
+		free_cfgparam(iface->ifnetmask6_len);
+		free_cfgparam(iface->ifoffload_str);
+		free_cfgparam(iface->ifneigh_entries);
+		free_cfgparam(iface->ifqdisc_entries);
+	}
+	free_cfgparam(cfg->gateway);
+	free_cfgparam(cfg->gateway6);
+	free_cfgparam(cfg->debug);
+	free_cfgparam(cfg->mount);
+	free_cfgparam(cfg->single_cpu);
+	free_cfgparam(cfg->sysctls);
+	free_cfgparam(cfg->boot_cmdline);
+	free_cfgparam(cfg->dump);
+	free_cfgparam(cfg->delay_main);
+	return 0;
+}
+
+
+int lkl_load_config_pre(struct lkl_config *cfg)
+{
+	int lkl_debug, ret;
+	struct lkl_config_iface *iface;
+
+	if (!cfg)
+		return 0;
+
+	if (cfg->debug)
+		lkl_debug = strtol(cfg->debug, NULL, 0);
+
+	if (!cfg->debug || (lkl_debug == 0))
+		lkl_host_ops.print = NULL;
+
+	for (iface = cfg->ifaces; iface; iface = iface->next) {
+		ret = lkl_config_netdev_create(cfg, iface);
+		if (ret < 0)
+			return -1;
+	}
+
+	return 0;
+}
+
+int lkl_load_config_post(struct lkl_config *cfg)
+{
+	int ret;
+	struct lkl_config_iface *iface;
+
+	if (!cfg)
+		return 0;
+
+	if (cfg->mount)
+		mount_cmds_exec(cfg->mount, lkl_mount_fs);
+
+	for (iface = cfg->ifaces; iface; iface = iface->next) {
+		ret = lkl_config_netdev_configure(cfg, iface);
+		if (ret < 0)
+			break;
+	}
+
+	if (cfg->gateway) {
+		unsigned int gwaddr;
+
+		if (inet_pton(LKL_AF_INET, cfg->gateway,
+			      (struct lkl_in_addr *)&gwaddr) != 1)
+			lkl_printf("Invalid ipv4 gateway: %s\n", cfg->gateway);
+
+		if (gwaddr != LKL_INADDR_NONE) {
+			ret = lkl_set_ipv4_gateway(gwaddr);
+			if (ret < 0)
+				lkl_printf("failed to set IPv4 gateway: %s\n",
+					   lkl_strerror(ret));
+		}
+	}
+
+	if (cfg->gateway6) {
+		char gw[16];
+
+		if (inet_pton(LKL_AF_INET6, cfg->gateway6, gw) != 1) {
+			lkl_printf("Invalid ipv6 gateway: %s\n", cfg->gateway6);
+		} else {
+			ret = lkl_set_ipv6_gateway(gw);
+			if (ret < 0)
+				lkl_printf("failed to set IPv6 gateway: %s\n",
+					   lkl_strerror(ret));
+		}
+	}
+
+	if (cfg->sysctls)
+		lkl_sysctl_parse_write(cfg->sysctls);
+
+	/* put a delay before calling main() */
+	if (cfg->delay_main) {
+		unsigned long delay = strtoul(cfg->delay_main, NULL, 10);
+
+		if (delay == ~0UL)
+			lkl_printf("got invalid delay_main value (%s)\n",
+				   cfg->delay_main);
+		else {
+			lkl_printf("sleeping %lu usec\n", delay);
+			usleep(delay);
+		}
+	}
+
+	return 0;
+}
+
+int lkl_unload_config(struct lkl_config *cfg)
+{
+	if (cfg) {
+		if (cfg->dump)
+			mount_cmds_exec(cfg->dump, dump_file);
+
+		lkl_clean_config(cfg);
+	}
+
+	return 0;
+}
diff --git a/tools/lkl/lib/dbg.c b/tools/lkl/lib/dbg.c
new file mode 100644
index 000000000000..b613353bce5c
--- /dev/null
+++ b/tools/lkl/lib/dbg.c
@@ -0,0 +1,300 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <errno.h>
+#include <lkl.h>
+#include <limits.h>
+#include <string.h>
+#include <stdio.h>
+#include <stdlib.h>
+
+static const char *PROMOTE = "$";
+#define str(x) #x
+#define xstr(s) str(s)
+#define MAX_BUF 100
+static char cmd[MAX_BUF];
+static char argv[10][MAX_BUF];
+static int argc;
+static char cur_dir[MAX_BUF] = "/";
+
+static char *normalize_path(const char *src, size_t src_len)
+{
+	char *res;
+	unsigned int res_len;
+	const char *ptr = src;
+	const char *end = &src[src_len];
+	const char *next;
+
+	res = malloc((src_len > 0 ? src_len : 1) + 1);
+	res_len = 0;
+
+	for (ptr = src; ptr < end; ptr = next+1) {
+		size_t len;
+
+		next = memchr(ptr, '/', end-ptr);
+		if (next == NULL)
+			next = end;
+
+		len = next-ptr;
+		switch (len) {
+		case 2:
+			if (ptr[0] == '.' && ptr[1] == '.') {
+				const char *slash = strrchr(res, '/');
+
+				if (slash != NULL)
+					res_len = slash - res;
+				continue;
+			}
+			break;
+		case 1:
+			if (ptr[0] == '.')
+				continue;
+			break;
+		case 0:
+			continue;
+		}
+		res[res_len++] = '/';
+		memcpy(&res[res_len], ptr, len);
+		res_len += len;
+	}
+	if (res_len == 0)
+		res[res_len++] = '/';
+	res[res_len] = '\0';
+	return res;
+}
+
+static void build_path(char *path)
+{
+	char *npath;
+
+	strcpy(path, cur_dir);
+	if (argc >= 1) {
+		if (argv[0][0] == '/')
+			strncpy(path, argv[0], LKL_PATH_MAX);
+		else {
+			strncat(path, "/", LKL_PATH_MAX - strlen(path) - 1);
+			strncat(path, argv[0], LKL_PATH_MAX - strlen(path) - 1);
+		}
+	}
+	npath = normalize_path(path, strlen(path));
+	strcpy(path, npath);
+	free(npath);
+}
+
+static void help(void)
+{
+	const char *msg =
+		"cat FILE\n"
+		"\tShow content of FILE\n"
+		"cd [DIR]\n"
+		"\tChange directory to DIR\n"
+		"exit\n"
+		"\tExit the debug session\n"
+		"help\n"
+		"\tShow this message\n"
+		"ls [DIR]\n"
+		"\tList files in DIR\n"
+		"mount FSTYPE\n"
+		"\tMount FSTYPE as /FSTYPE\n"
+		"overwrite FILE\n"
+		"\tOverwrite content of FILE from stdin\n"
+		"pwd\n"
+		"\tShow current directory\n"
+		;
+	printf("%s", msg);
+}
+
+static void ls(void)
+{
+	char path[LKL_PATH_MAX];
+	struct lkl_dir *dir;
+	struct lkl_linux_dirent64 *de;
+	int err;
+
+	build_path(path);
+	dir = lkl_opendir(path, &err);
+	if (dir) {
+		do {
+			de = lkl_readdir(dir);
+			if (de) {
+				printf("%s\n", de->d_name);
+			} else {
+				err = lkl_errdir(dir);
+				if (err != 0) {
+					fprintf(stderr, "%s\n",
+						lkl_strerror(err));
+				}
+				break;
+			}
+		} while (1);
+		lkl_closedir(dir);
+	} else {
+		fprintf(stderr, "%s: %s\n", path, lkl_strerror(err));
+	}
+}
+
+static void cd(void)
+{
+	char path[LKL_PATH_MAX];
+	struct lkl_dir *dir;
+	int err;
+
+	build_path(path);
+	dir = lkl_opendir(path, &err);
+	if (dir) {
+		strcpy(cur_dir, path);
+		lkl_closedir(dir);
+	} else {
+		fprintf(stderr, "%s: %s\n", path, lkl_strerror(err));
+	}
+}
+
+static void mount(void)
+{
+	char *fstype;
+	int ret = 0;
+
+	if (argc != 1) {
+		fprintf(stderr, "%s\n", "One argument is needed.");
+		return;
+	}
+
+	fstype = argv[0];
+	ret = lkl_mount_fs(fstype);
+	if (ret == 1)
+		fprintf(stderr, "%s is already mounted.\n", fstype);
+}
+
+static void cat(void)
+{
+	char path[LKL_PATH_MAX];
+	int ret;
+	char buf[1024];
+	int fd;
+
+	if (argc != 1) {
+		fprintf(stderr, "%s\n", "One argument is needed.");
+		return;
+	}
+
+	build_path(path);
+	fd = lkl_sys_open(path, LKL_O_RDONLY, 0);
+
+	if (fd < 0) {
+		fprintf(stderr, "lkl_sys_open %s: %s\n",
+			path, lkl_strerror(fd));
+		return;
+	}
+
+	while ((ret = lkl_sys_read(fd, buf, sizeof(buf) - 1)) > 0) {
+		buf[ret] = '\0';
+		printf("%s", buf);
+	}
+
+	if (ret) {
+		fprintf(stderr, "lkl_sys_read %s: %s\n",
+			path, lkl_strerror(ret));
+	}
+	lkl_sys_close(fd);
+}
+
+static void overwrite(void)
+{
+	char path[LKL_PATH_MAX];
+	int ret;
+	int fd;
+	char buf[1024];
+
+	build_path(path);
+	fd = lkl_sys_open(path, LKL_O_WRONLY | LKL_O_CREAT, 0);
+	if (fd < 0) {
+		fprintf(stderr, "lkl_sys_open %s: %s\n",
+			path, lkl_strerror(fd));
+		return;
+	}
+	printf("Input the content and stop by hitting Ctrl-D:\n");
+	while (fgets(buf, 1023, stdin)) {
+		ret = lkl_sys_write(fd, buf, strlen(buf));
+		if (ret < 0) {
+			fprintf(stderr, "lkl_sys_write %s: %s\n",
+				path, lkl_strerror(fd));
+		}
+	}
+	lkl_sys_close(fd);
+}
+
+static void pwd(void)
+{
+	printf("%s\n", cur_dir);
+}
+
+static int parse_cmd(char *input)
+{
+	char *token;
+
+	token = strtok(input, " ");
+	if (token)
+		strcpy(cmd, token);
+	else
+		return -1;
+
+	argc = 0;
+	token = strtok(NULL, " ");
+	while (token) {
+		if (argc >= 10) {
+			fprintf(stderr, "To many args > 10\n");
+			return -1;
+		}
+		strcpy(argv[argc++], token);
+		token = strtok(NULL, " ");
+	}
+	return 0;
+}
+
+static void run_cmd(void)
+{
+	if (strcmp(cmd, "cat") == 0)
+		cat();
+	else if (strcmp(cmd, "cd") == 0)
+		cd();
+	else if (strcmp(cmd, "help") == 0)
+		help();
+	else if (strcmp(cmd, "ls") == 0)
+		ls();
+	else if (strcmp(cmd, "mount") == 0)
+		mount();
+	else if (strcmp(cmd, "overwrite") == 0)
+		overwrite();
+	else if (strcmp(cmd, "pwd") == 0)
+		pwd();
+	else
+		fprintf(stderr, "Unknown command: %s\n", cmd);
+}
+
+void dbg_entrance(void)
+{
+	char input[MAX_BUF + 1];
+	int ret;
+	int c;
+
+	printf("Type help to see a list of commands\n");
+	do {
+		printf("%s ", PROMOTE);
+		ret = scanf("%" xstr(MAX_BUF) "[^\n]s", input);
+		while ((c = getchar()) != '\n' && c != EOF)
+			;
+		if (ret == 0)
+			continue;
+		if (ret != 1 && errno != EINTR) {
+			perror("scanf");
+			continue;
+		}
+		if (strlen(input) == MAX_BUF) {
+			fprintf(stderr, "Too long input > %d\n", MAX_BUF - 1);
+			continue;
+		}
+		if (parse_cmd(input))
+			continue;
+		if (strcmp(cmd, "exit") == 0)
+			break;
+		run_cmd();
+	} while (1);
+}
diff --git a/tools/lkl/lib/dbg_handler.c b/tools/lkl/lib/dbg_handler.c
new file mode 100644
index 000000000000..01d165a5fc1e
--- /dev/null
+++ b/tools/lkl/lib/dbg_handler.c
@@ -0,0 +1,44 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <lkl_host.h>
+
+extern void dbg_entrance(void);
+static int dbg_running;
+
+static void dbg_thread(void *arg)
+{
+	lkl_host_ops.thread_detach();
+	printf("======Enter Debug======\n");
+	dbg_entrance();
+	printf("======Exit Debug======\n");
+	dbg_running = 0;
+}
+
+void dbg_handler(int signum)
+{
+	/* We don't care about the possible race on dbg_running. */
+	if (dbg_running) {
+		fprintf(stderr, "A debug lib is running\n");
+		return;
+	}
+	dbg_running = 1;
+	lkl_host_ops.thread_create(&dbg_thread, NULL);
+}
+
+#ifndef __MINGW32__
+#include <signal.h>
+void lkl_register_dbg_handler(void)
+{
+	struct sigaction sa;
+
+	sigemptyset(&sa.sa_mask);
+	sa.sa_handler = dbg_handler;
+	if (sigaction(SIGTSTP, &sa, NULL) == -1)
+		perror("sigaction");
+}
+#else
+void lkl_register_dbg_handler(void)
+{
+	fprintf(stderr, "%s is not implemented.\n", __func__);
+}
+#endif
diff --git a/tools/lkl/lib/endian.h b/tools/lkl/lib/endian.h
new file mode 100644
index 000000000000..aaccfa0edb65
--- /dev/null
+++ b/tools/lkl/lib/endian.h
@@ -0,0 +1,31 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_LIB_ENDIAN_H
+#define _LKL_LIB_ENDIAN_H
+
+#if defined(__FreeBSD__)
+#include <sys/endian.h>
+#elif defined(__ANDROID__)
+#include <sys/endian.h>
+#elif defined(__MINGW32__)
+#include <winsock.h>
+#define le32toh(x) (x)
+#define le16toh(x) (x)
+#define htole32(x) (x)
+#define htole16(x) (x)
+#define le64toh(x) (x)
+#define htobe32(x) htonl(x)
+#define htobe16(x) htons(x)
+#define be32toh(x) ntohl(x)
+#define be16toh(x) ntohs(x)
+#else
+#include <endian.h>
+#endif
+
+#ifndef htonl
+#define htonl(x) htobe32(x)
+#define htons(x) htobe16(x)
+#define ntohl(x) be32toh(x)
+#define ntohs(x) be16toh(x)
+#endif
+
+#endif /* _LKL_LIB_ENDIAN_H */
diff --git a/tools/lkl/lib/jmp_buf.c b/tools/lkl/lib/jmp_buf.c
new file mode 100644
index 000000000000..f6bdd7e4bd83
--- /dev/null
+++ b/tools/lkl/lib/jmp_buf.c
@@ -0,0 +1,14 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <setjmp.h>
+#include <lkl_host.h>
+
+void jmp_buf_set(struct lkl_jmp_buf *jmpb, void (*f)(void))
+{
+	if (!setjmp(*((jmp_buf *)jmpb->buf)))
+		f();
+}
+
+void jmp_buf_longjmp(struct lkl_jmp_buf *jmpb, int val)
+{
+	longjmp(*((jmp_buf *)jmpb->buf), val);
+}
diff --git a/tools/lkl/lib/jmp_buf.h b/tools/lkl/lib/jmp_buf.h
new file mode 100644
index 000000000000..8782cbaaf51f
--- /dev/null
+++ b/tools/lkl/lib/jmp_buf.h
@@ -0,0 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_LIB_JMP_BUF_H
+#define _LKL_LIB_JMP_BUF_H
+
+void jmp_buf_set(struct lkl_jmp_buf *jmpb, void (*f)(void));
+void jmp_buf_longjmp(struct lkl_jmp_buf *jmpb, int val);
+
+#endif
diff --git a/tools/lkl/lib/utils.c b/tools/lkl/lib/utils.c
new file mode 100644
index 000000000000..7de92bbe5475
--- /dev/null
+++ b/tools/lkl/lib/utils.c
@@ -0,0 +1,266 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdarg.h>
+#include <stdio.h>
+#include <string.h>
+#include <lkl_host.h>
+
+static const char * const lkl_err_strings[] = {
+	"Success",
+	"Operation not permitted",
+	"No such file or directory",
+	"No such process",
+	"Interrupted system call",
+	"I/O error",
+	"No such device or address",
+	"Argument list too long",
+	"Exec format error",
+	"Bad file number",
+	"No child processes",
+	"Try again",
+	"Out of memory",
+	"Permission denied",
+	"Bad address",
+	"Block device required",
+	"Device or resource busy",
+	"File exists",
+	"Cross-device link",
+	"No such device",
+	"Not a directory",
+	"Is a directory",
+	"Invalid argument",
+	"File table overflow",
+	"Too many open files",
+	"Not a typewriter",
+	"Text file busy",
+	"File too large",
+	"No space left on device",
+	"Illegal seek",
+	"Read-only file system",
+	"Too many links",
+	"Broken pipe",
+	"Math argument out of domain of func",
+	"Math result not representable",
+	"Resource deadlock would occur",
+	"File name too long",
+	"No record locks available",
+	"Invalid system call number",
+	"Directory not empty",
+	"Too many symbolic links encountered",
+	"Bad error code", /* EWOULDBLOCK is EAGAIN */
+	"No message of desired type",
+	"Identifier removed",
+	"Channel number out of range",
+	"Level 2 not synchronized",
+	"Level 3 halted",
+	"Level 3 reset",
+	"Link number out of range",
+	"Protocol driver not attached",
+	"No CSI structure available",
+	"Level 2 halted",
+	"Invalid exchange",
+	"Invalid request descriptor",
+	"Exchange full",
+	"No anode",
+	"Invalid request code",
+	"Invalid slot",
+	"Bad error code", /* EDEADLOCK is EDEADLK */
+	"Bad font file format",
+	"Device not a stream",
+	"No data available",
+	"Timer expired",
+	"Out of streams resources",
+	"Machine is not on the network",
+	"Package not installed",
+	"Object is remote",
+	"Link has been severed",
+	"Advertise error",
+	"Srmount error",
+	"Communication error on send",
+	"Protocol error",
+	"Multihop attempted",
+	"RFS specific error",
+	"Not a data message",
+	"Value too large for defined data type",
+	"Name not unique on network",
+	"File descriptor in bad state",
+	"Remote address changed",
+	"Can not access a needed shared library",
+	"Accessing a corrupted shared library",
+	".lib section in a.out corrupted",
+	"Attempting to link in too many shared libraries",
+	"Cannot exec a shared library directly",
+	"Illegal byte sequence",
+	"Interrupted system call should be restarted",
+	"Streams pipe error",
+	"Too many users",
+	"Socket operation on non-socket",
+	"Destination address required",
+	"Message too long",
+	"Protocol wrong type for socket",
+	"Protocol not available",
+	"Protocol not supported",
+	"Socket type not supported",
+	"Operation not supported on transport endpoint",
+	"Protocol family not supported",
+	"Address family not supported by protocol",
+	"Address already in use",
+	"Cannot assign requested address",
+	"Network is down",
+	"Network is unreachable",
+	"Network dropped connection because of reset",
+	"Software caused connection abort",
+	"Connection reset by peer",
+	"No buffer space available",
+	"Transport endpoint is already connected",
+	"Transport endpoint is not connected",
+	"Cannot send after transport endpoint shutdown",
+	"Too many references: cannot splice",
+	"Connection timed out",
+	"Connection refused",
+	"Host is down",
+	"No route to host",
+	"Operation already in progress",
+	"Operation now in progress",
+	"Stale file handle",
+	"Structure needs cleaning",
+	"Not a XENIX named type file",
+	"No XENIX semaphores available",
+	"Is a named type file",
+	"Remote I/O error",
+	"Quota exceeded",
+	"No medium found",
+	"Wrong medium type",
+	"Operation Canceled",
+	"Required key not available",
+	"Key has expired",
+	"Key has been revoked",
+	"Key was rejected by service",
+	"Owner died",
+	"State not recoverable",
+	"Operation not possible due to RF-kill",
+	"Memory page has hardware error",
+};
+
+const char *lkl_strerror(int err)
+{
+	if (err < 0)
+		err = -err;
+
+	if ((size_t)err >= sizeof(lkl_err_strings) / sizeof(const char *))
+		return "Bad error code";
+
+	return lkl_err_strings[err];
+}
+
+void lkl_perror(char *msg, int err)
+{
+	const char *err_msg = lkl_strerror(err);
+	/* We need to use 'real' printf because lkl_host_ops.print can
+	 * be turned off when debugging is off.
+	 */
+	lkl_printf("%s: %s\n", msg, err_msg);
+}
+
+static int lkl_vprintf(const char *fmt, va_list args)
+{
+	int n;
+	char *buffer;
+	va_list copy;
+
+	if (!lkl_host_ops.print)
+		return 0;
+
+	va_copy(copy, args);
+	n = vsnprintf(NULL, 0, fmt, copy);
+	va_end(copy);
+
+	buffer = lkl_host_ops.mem_alloc(n + 1);
+	if (!buffer)
+		return -1;
+
+	vsnprintf(buffer, n + 1, fmt, args);
+
+	lkl_host_ops.print(buffer, n);
+	lkl_host_ops.mem_free(buffer);
+
+	return n;
+}
+
+int lkl_printf(const char *fmt, ...)
+{
+	int n;
+	va_list args;
+
+	va_start(args, fmt);
+	n = lkl_vprintf(fmt, args);
+	va_end(args);
+
+	return n;
+}
+
+void lkl_bug(const char *fmt, ...)
+{
+	va_list args;
+
+	va_start(args, fmt);
+	lkl_vprintf(fmt, args);
+	va_end(args);
+
+	lkl_host_ops.panic();
+}
+#ifndef __arch_um__
+int lkl_sysctl(const char *path, const char *value)
+{
+	int ret;
+	int fd;
+	char *delim, *p;
+	char full_path[256];
+
+	lkl_mount_fs("proc");
+
+	snprintf(full_path, sizeof(full_path), "/proc/sys/%s", path);
+	p = full_path;
+	while ((delim = strstr(p, "."))) {
+		*delim = '/';
+		p = delim + 1;
+	}
+
+	fd = lkl_sys_open(full_path, LKL_O_WRONLY | LKL_O_CREAT, 0);
+	if (fd < 0) {
+		lkl_printf("lkl_sys_open %s: %s\n",
+			   full_path, lkl_strerror(fd));
+		return -1;
+	}
+	ret = lkl_sys_write(fd, value, strlen(value));
+	if (ret < 0) {
+		lkl_printf("lkl_sys_write %s: %s\n",
+			full_path, lkl_strerror(fd));
+	}
+
+	lkl_sys_close(fd);
+
+	return 0;
+}
+
+/* Configure sysctl parameters as the form of "key=value;key=value;..." */
+void lkl_sysctl_parse_write(const char *sysctls)
+{
+	char *saveptr = NULL, *token = NULL;
+	char *key = NULL, *value = NULL;
+	char strings[256];
+	int ret = 0;
+
+	strcpy(strings, sysctls);
+	for (token = strtok_r(strings, ";", &saveptr); token;
+	     token = strtok_r(NULL, ";", &saveptr)) {
+		key = strtok(token, "=");
+		value = strtok(NULL, "=");
+		ret = lkl_sysctl(key, value);
+		if (ret) {
+			lkl_printf("Failed to configure sysctl entries: %s\n",
+				   lkl_strerror(ret));
+			return;
+		}
+	}
+}
+#endif
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v3 14/26] lkl tools: host lib: add utilities functions
@ 2020-02-05  7:30   ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05  7:30 UTC (permalink / raw)
  To: linux-um
  Cc: linux-arch, Hajime Tazaki, Conrad Meyer, Octavian Purdila,
	Motomu Utsumi, Akira Moroo, Patrick Collins,
	linux-kernel-library, Yuan Liu

From: Octavian Purdila <tavi.purdila@gmail.com>

Add basic utility functions for getting a string from a kernel error
code and a fprintf like function that uses the host print
operation. The latter is useful for informing the user about errors
that occur in the host library.

Other configuration and debug utilities are also added.

Cc: Conrad Meyer <cem@FreeBSD.org>
Cc: Patrick Collins <pscollins@google.com>
Cc: Yuan Liu <liuyuan@google.com>
Cc: Motomu Utsumi <motomuman@gmail.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/lkl/include/lkl_config.h |  61 +++
 tools/lkl/include/lkl_host.h   |   7 +
 tools/lkl/lib/Build            |   7 +
 tools/lkl/lib/config.c         | 744 +++++++++++++++++++++++++++++++++
 tools/lkl/lib/dbg.c            | 300 +++++++++++++
 tools/lkl/lib/dbg_handler.c    |  44 ++
 tools/lkl/lib/endian.h         |  31 ++
 tools/lkl/lib/jmp_buf.c        |  14 +
 tools/lkl/lib/jmp_buf.h        |   8 +
 tools/lkl/lib/utils.c          | 266 ++++++++++++
 10 files changed, 1482 insertions(+)
 create mode 100644 tools/lkl/include/lkl_config.h
 create mode 100644 tools/lkl/lib/config.c
 create mode 100644 tools/lkl/lib/dbg.c
 create mode 100644 tools/lkl/lib/dbg_handler.c
 create mode 100644 tools/lkl/lib/endian.h
 create mode 100644 tools/lkl/lib/jmp_buf.c
 create mode 100644 tools/lkl/lib/jmp_buf.h
 create mode 100644 tools/lkl/lib/utils.c

diff --git a/tools/lkl/include/lkl_config.h b/tools/lkl/include/lkl_config.h
new file mode 100644
index 000000000000..d3edf8b414cf
--- /dev/null
+++ b/tools/lkl/include/lkl_config.h
@@ -0,0 +1,61 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_LIB_CONFIG_H
+#define _LKL_LIB_CONFIG_H
+
+#define LKL_CONFIG_JSON_TOKEN_MAX 300
+
+struct lkl_config_iface {
+	struct lkl_config_iface *next;
+	struct lkl_netdev *nd;
+
+	/* OBSOLETE: should use IFTYPE and IFPARAMS */
+	char *iftap;
+	char *iftype;
+	char *ifparams;
+	char *ifmtu_str;
+	char *ifip;
+	char *ifipv6;
+	char *ifgateway;
+	char *ifgateway6;
+	char *ifmac_str;
+	char *ifnetmask_len;
+	char *ifnetmask6_len;
+	char *ifoffload_str;
+	char *ifneigh_entries;
+	char *ifqdisc_entries;
+};
+
+struct lkl_config {
+	int ifnum;
+	struct lkl_config_iface *ifaces;
+
+	char *gateway;
+	char *gateway6;
+	char *debug;
+	char *mount;
+	/* single_cpu mode:
+	 * 0: Don't pin to single CPU (default).
+	 * 1: Pin only LKL kernel threads to single CPU.
+	 * 2: Pin all LKL threads to single CPU including all LKL kernel threads
+	 * and device polling threads. Avoid this mode if having busy polling
+	 * threads.
+	 *
+	 * mode 2 can achieve better TCP_RR but worse TCP_STREAM than mode 1.
+	 * You should choose the best for your application and virtio device
+	 * type.
+	 */
+	char *single_cpu;
+	char *sysctls;
+	char *boot_cmdline;
+	char *dump;
+	char *delay_main;
+};
+
+int lkl_load_config_json(struct lkl_config *cfg, char *jstr);
+int lkl_load_config_env(struct lkl_config *cfg);
+void lkl_show_config(struct lkl_config *cfg);
+int lkl_load_config_pre(struct lkl_config *cfg);
+int lkl_load_config_post(struct lkl_config *cfg);
+int lkl_unload_config(struct lkl_config *cfg);
+
+#endif /* _LKL_LIB_CONFIG_H */
diff --git a/tools/lkl/include/lkl_host.h b/tools/lkl/include/lkl_host.h
index b5f96096fe69..85e80eb4ad0d 100644
--- a/tools/lkl/include/lkl_host.h
+++ b/tools/lkl/include/lkl_host.h
@@ -11,6 +11,13 @@ extern "C" {
 
 extern struct lkl_host_operations lkl_host_ops;
 
+/**
+ * lkl_printf - print a message via the host print operation
+ *
+ * @fmt: printf like format string
+ */
+int lkl_printf(const char *fmt, ...);
+
 
 #ifdef __cplusplus
 }
diff --git a/tools/lkl/lib/Build b/tools/lkl/lib/Build
index 8b137891791f..658bfa865b9c 100644
--- a/tools/lkl/lib/Build
+++ b/tools/lkl/lib/Build
@@ -1 +1,8 @@
+CFLAGS_config.o += -I$(srctree)/tools/perf/pmu-events
 
+liblkl-y += jmp_buf.o
+liblkl-y += utils.o
+liblkl-y += dbg.o
+liblkl-y += dbg_handler.o
+liblkl-y += ../../perf/pmu-events/jsmn.o
+liblkl-y += config.o
diff --git a/tools/lkl/lib/config.c b/tools/lkl/lib/config.c
new file mode 100644
index 000000000000..37f8ac4d942a
--- /dev/null
+++ b/tools/lkl/lib/config.c
@@ -0,0 +1,744 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdlib.h>
+#define _HAVE_STRING_ARCH_strtok_r
+#include <string.h>
+#ifndef __MINGW32__
+#include <arpa/inet.h>
+#endif
+#include <lkl_host.h>
+#include <lkl_config.h>
+
+#include "jsmn.h"
+
+static int jsoneq(const char *json, jsmntok_t *tok, const char *s)
+{
+	if (tok->type == JSMN_STRING &&
+		(int) strlen(s) == tok->end - tok->start &&
+		strncmp(json + tok->start, s, tok->end - tok->start) == 0) {
+		return 0;
+	}
+	return -1;
+}
+
+static int cfgcpy(char **to, char *from)
+{
+	if (!from)
+		return 0;
+	if (*to)
+		free(*to);
+	*to = (char *)malloc((strlen(from) + 1) * sizeof(char));
+	if (*to == NULL) {
+		lkl_printf("malloc failed\n");
+		return -1;
+	}
+	strcpy(*to, from);
+	return 0;
+}
+
+static int cfgncpy(char **to, char *from, int len)
+{
+	if (!from)
+		return 0;
+	if (*to)
+		free(*to);
+	*to = (char *)malloc((len + 1) * sizeof(char));
+	if (*to == NULL) {
+		lkl_printf("malloc failed\n");
+		return -1;
+	}
+	strncpy(*to, from, len + 1);
+	(*to)[len] = '\0';
+	return 0;
+}
+
+static int parse_ifarr(struct lkl_config *cfg,
+		jsmntok_t *toks, char *jstr, int startpos)
+{
+	int ifidx, pos, posend, ret;
+	char **cfgptr;
+	struct lkl_config_iface *iface, *prev = NULL;
+
+	if (!cfg || !toks || !jstr)
+		return -1;
+	pos = startpos;
+	pos++;
+	if (toks[pos].type != JSMN_ARRAY) {
+		lkl_printf("unexpected json type, json array expected\n");
+		return -1;
+	}
+
+	cfg->ifnum = toks[pos].size;
+	pos++;
+	iface = cfg->ifaces;
+
+	for (ifidx = 0; ifidx < cfg->ifnum; ifidx++) {
+		if (toks[pos].type != JSMN_OBJECT) {
+			lkl_printf("object json type expected\n");
+			return -1;
+		}
+
+		posend = pos + toks[pos].size;
+		pos++;
+		iface = malloc(sizeof(struct lkl_config_iface));
+		memset(iface, 0, sizeof(struct lkl_config_iface));
+
+		if (prev)
+			prev->next = iface;
+		else
+			cfg->ifaces = iface;
+		prev = iface;
+
+		for (; pos < posend; pos += 2) {
+			if (toks[pos].type != JSMN_STRING) {
+				lkl_printf("object json type expected\n");
+				return -1;
+			}
+			if (jsoneq(jstr, &toks[pos], "type") == 0) {
+				cfgptr = &iface->iftype;
+			} else if (jsoneq(jstr, &toks[pos], "param") == 0) {
+				cfgptr = &iface->ifparams;
+			} else if (jsoneq(jstr, &toks[pos], "mtu") == 0) {
+				cfgptr = &iface->ifmtu_str;
+			} else if (jsoneq(jstr, &toks[pos], "ip") == 0) {
+				cfgptr = &iface->ifip;
+			} else if (jsoneq(jstr, &toks[pos], "ipv6") == 0) {
+				cfgptr = &iface->ifipv6;
+			} else if (jsoneq(jstr, &toks[pos], "ifgateway") == 0) {
+				cfgptr = &iface->ifgateway;
+			} else if (jsoneq(jstr, &toks[pos],
+							"ifgateway6") == 0) {
+				cfgptr = &iface->ifgateway6;
+			} else if (jsoneq(jstr, &toks[pos], "mac") == 0) {
+				cfgptr = &iface->ifmac_str;
+			} else if (jsoneq(jstr, &toks[pos], "masklen") == 0) {
+				cfgptr = &iface->ifnetmask_len;
+			} else if (jsoneq(jstr, &toks[pos], "masklen6") == 0) {
+				cfgptr = &iface->ifnetmask6_len;
+			} else if (jsoneq(jstr, &toks[pos], "neigh") == 0) {
+				cfgptr = &iface->ifneigh_entries;
+			} else if (jsoneq(jstr, &toks[pos], "qdisc") == 0) {
+				cfgptr = &iface->ifqdisc_entries;
+			} else if (jsoneq(jstr, &toks[pos], "offload") == 0) {
+				cfgptr = &iface->ifoffload_str;
+			} else {
+				lkl_printf("unexpected key: %.*s\n",
+						toks[pos].end-toks[pos].start,
+						jstr + toks[pos].start);
+				return -1;
+			}
+			ret = cfgncpy(cfgptr, jstr + toks[pos+1].start,
+					toks[pos+1].end-toks[pos+1].start);
+			if (ret < 0)
+				return ret;
+		}
+	}
+	return pos - startpos;
+}
+
+#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]))
+
+int lkl_load_config_json(struct lkl_config *cfg, char *jstr)
+{
+	int pos, ret;
+	char **cfgptr;
+	jsmn_parser jp;
+	jsmntok_t toks[LKL_CONFIG_JSON_TOKEN_MAX];
+
+	if (!cfg || !jstr)
+		return -1;
+	jsmn_init(&jp);
+	ret = jsmn_parse(&jp, jstr, strlen(jstr), toks, ARRAY_SIZE(toks));
+	if (ret != JSMN_SUCCESS) {
+		lkl_printf("failed to parse json\n");
+		return -1;
+	}
+	if (toks[0].type != JSMN_OBJECT) {
+		lkl_printf("object json type expected\n");
+		return -1;
+	}
+	for (pos = 1; pos < jp.toknext; pos++) {
+		if (toks[pos].type != JSMN_STRING) {
+			lkl_printf("string json type expected\n");
+			return -1;
+		}
+		if (jsoneq(jstr, &toks[pos], "interfaces") == 0) {
+			ret = parse_ifarr(cfg, toks, jstr, pos);
+			if (ret < 0)
+				return ret;
+			pos += ret;
+			pos--;
+			continue;
+		}
+		if (jsoneq(jstr, &toks[pos], "gateway") == 0) {
+			cfgptr = &cfg->gateway;
+		} else if (jsoneq(jstr, &toks[pos], "gateway6") == 0) {
+			cfgptr = &cfg->gateway6;
+		} else if (jsoneq(jstr, &toks[pos], "debug") == 0) {
+			cfgptr = &cfg->debug;
+		} else if (jsoneq(jstr, &toks[pos], "mount") == 0) {
+			cfgptr = &cfg->mount;
+		} else if (jsoneq(jstr, &toks[pos], "singlecpu") == 0) {
+			cfgptr = &cfg->single_cpu;
+		} else if (jsoneq(jstr, &toks[pos], "sysctl") == 0) {
+			cfgptr = &cfg->sysctls;
+		} else if (jsoneq(jstr, &toks[pos], "boot_cmdline") == 0) {
+			cfgptr = &cfg->boot_cmdline;
+		} else if (jsoneq(jstr, &toks[pos], "dump") == 0) {
+			cfgptr = &cfg->dump;
+		} else if (jsoneq(jstr, &toks[pos], "delay_main") == 0) {
+			cfgptr = &cfg->delay_main;
+		} else {
+			lkl_printf("unexpected key in json %.*s\n",
+					toks[pos].end-toks[pos].start,
+					jstr + toks[pos].start);
+			return -1;
+		}
+		pos++;
+		ret = cfgncpy(cfgptr, jstr + toks[pos].start,
+				toks[pos].end-toks[pos].start);
+		if (ret < 0)
+			return ret;
+	}
+	return 0;
+}
+
+void lkl_show_config(struct lkl_config *cfg)
+{
+	struct lkl_config_iface *iface;
+	int i = 0;
+
+	if (!cfg)
+		return;
+	lkl_printf("gateway: %s\n", cfg->gateway);
+	lkl_printf("gateway6: %s\n", cfg->gateway6);
+	lkl_printf("debug: %s\n", cfg->debug);
+	lkl_printf("mount: %s\n", cfg->mount);
+	lkl_printf("singlecpu: %s\n", cfg->single_cpu);
+	lkl_printf("sysctl: %s\n", cfg->sysctls);
+	lkl_printf("cmdline: %s\n", cfg->boot_cmdline);
+	lkl_printf("dump: %s\n", cfg->dump);
+	lkl_printf("delay: %s\n", cfg->delay_main);
+
+	for (iface = cfg->ifaces; iface; iface = iface->next, i++) {
+		lkl_printf("ifmac[%d] = %s\n", i, iface->ifmac_str);
+		lkl_printf("ifmtu[%d] = %s\n", i, iface->ifmtu_str);
+		lkl_printf("iftype[%d] = %s\n", i, iface->iftype);
+		lkl_printf("ifparam[%d] = %s\n", i, iface->ifparams);
+		lkl_printf("ifip[%d] = %s\n", i, iface->ifip);
+		lkl_printf("ifmasklen[%d] = %s\n", i, iface->ifnetmask_len);
+		lkl_printf("ifgateway[%d] = %s\n", i, iface->ifgateway);
+		lkl_printf("ifip6[%d] = %s\n", i, iface->ifipv6);
+		lkl_printf("ifmasklen6[%d] = %s\n", i, iface->ifnetmask6_len);
+		lkl_printf("ifgateway6[%d] = %s\n", i, iface->ifgateway6);
+		lkl_printf("ifoffload[%d] = %s\n", i, iface->ifoffload_str);
+		lkl_printf("ifneigh[%d] = %s\n", i, iface->ifneigh_entries);
+		lkl_printf("ifqdisk[%d] = %s\n", i, iface->ifqdisc_entries);
+	}
+}
+
+int lkl_load_config_env(struct lkl_config *cfg)
+{
+	int ret;
+	char *enviftype = getenv("LKL_HIJACK_NET_IFTYPE");
+	char *envifparams = getenv("LKL_HIJACK_NET_IFPARAMS");
+	char *envmtu_str = getenv("LKL_HIJACK_NET_MTU");
+	char *envip = getenv("LKL_HIJACK_NET_IP");
+	char *envipv6 = getenv("LKL_HIJACK_NET_IPV6");
+	char *envifgateway = getenv("LKL_HIJACK_NET_IFGATEWAY");
+	char *envifgateway6 = getenv("LKL_HIJACK_NET_IFGATEWAY6");
+	char *envmac_str = getenv("LKL_HIJACK_NET_MAC");
+	char *envnetmask_len = getenv("LKL_HIJACK_NET_NETMASK_LEN");
+	char *envnetmask6_len = getenv("LKL_HIJACK_NET_NETMASK6_LEN");
+	char *envgateway = getenv("LKL_HIJACK_NET_GATEWAY");
+	char *envgateway6 = getenv("LKL_HIJACK_NET_GATEWAY6");
+	char *envdebug = getenv("LKL_HIJACK_DEBUG");
+	char *envmount = getenv("LKL_HIJACK_MOUNT");
+	char *envneigh_entries = getenv("LKL_HIJACK_NET_NEIGHBOR");
+	char *envqdisc_entries = getenv("LKL_HIJACK_NET_QDISC");
+	char *envsingle_cpu = getenv("LKL_HIJACK_SINGLE_CPU");
+	char *envoffload_str = getenv("LKL_HIJACK_OFFLOAD");
+	char *envsysctls = getenv("LKL_HIJACK_SYSCTL");
+	char *envboot_cmdline = getenv("LKL_HIJACK_BOOT_CMDLINE") ? : "";
+	char *envdump = getenv("LKL_HIJACK_DUMP");
+	struct lkl_config_iface *iface;
+
+	if (!cfg)
+		return -1;
+	if (enviftype)
+		cfg->ifnum = 1;
+
+	iface = malloc(sizeof(struct lkl_config_iface));
+	memset(iface, 0, sizeof(struct lkl_config_iface));
+
+	ret = cfgcpy(&iface->iftype, enviftype);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifparams, envifparams);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifmtu_str, envmtu_str);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifip, envip);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifipv6, envipv6);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifgateway, envifgateway);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifgateway6, envifgateway6);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifmac_str, envmac_str);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifnetmask_len, envnetmask_len);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifnetmask6_len, envnetmask6_len);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifoffload_str, envoffload_str);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifneigh_entries, envneigh_entries);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifqdisc_entries, envqdisc_entries);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&cfg->gateway, envgateway);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&cfg->gateway6, envgateway6);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&cfg->debug, envdebug);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&cfg->mount, envmount);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&cfg->single_cpu, envsingle_cpu);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&cfg->sysctls, envsysctls);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&cfg->boot_cmdline, envboot_cmdline);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&cfg->dump, envdump);
+	if (ret < 0)
+		return ret;
+	return 0;
+}
+
+static int parse_mac_str(char *mac_str, __lkl__u8 mac[LKL_ETH_ALEN])
+{
+	char delim[] = ":";
+	char *saveptr = NULL, *token = NULL;
+	int i = 0;
+
+	if (!mac_str)
+		return 0;
+
+	for (token = strtok_r(mac_str, delim, &saveptr);
+	     i < LKL_ETH_ALEN; i++) {
+		if (!token) {
+			/* The address is too short */
+			return -1;
+		}
+
+		mac[i] = (__lkl__u8) strtol(token, NULL, 16);
+		token = strtok_r(NULL, delim, &saveptr);
+	}
+
+	if (strtok_r(NULL, delim, &saveptr)) {
+		/* The address is too long */
+		return -1;
+	}
+
+	return 1;
+}
+
+/* Add permanent neighbor entries in the form of "ip|mac;ip|mac;..." */
+static void add_neighbor(int ifindex, char *entries)
+{
+	char *saveptr = NULL, *token = NULL;
+	char *ip = NULL, *mac_str = NULL;
+	int ret = 0;
+	__lkl__u8 mac[LKL_ETH_ALEN];
+	char ip_addr[16];
+	int af;
+
+	for (token = strtok_r(entries, ";", &saveptr); token;
+	     token = strtok_r(NULL, ";", &saveptr)) {
+		ip = strtok(token, "|");
+		mac_str = strtok(NULL, "|");
+		if (ip == NULL || mac_str == NULL || strtok(NULL, "|") != NULL)
+			return;
+
+		af = LKL_AF_INET;
+		ret = inet_pton(LKL_AF_INET, ip, ip_addr);
+		if (ret == 0) {
+			ret = inet_pton(LKL_AF_INET6, ip, ip_addr);
+			af = LKL_AF_INET6;
+		}
+		if (ret != 1) {
+			lkl_printf("Bad ip address: %s\n", ip);
+			return;
+		}
+
+		ret = parse_mac_str(mac_str, mac);
+		if (ret != 1) {
+			lkl_printf("Failed to parse mac: %s\n", mac_str);
+			return;
+		}
+		ret = lkl_add_neighbor(ifindex, af, ip_addr, mac);
+		if (ret) {
+			lkl_printf("Failed to add neighbor entry: %s\n",
+				   lkl_strerror(ret));
+			return;
+		}
+	}
+}
+
+/* We don't have an easy way to make FILE*s out of our fds, so we
+ * can't use e.g. fgets
+ */
+static int dump_file(char *path)
+{
+	int ret = -1, bytes_read = 0;
+	char str[1024] = { 0 };
+	int fd;
+
+	fd = lkl_sys_open(path, LKL_O_RDONLY, 0);
+
+	if (fd < 0) {
+		lkl_printf("%s lkl_sys_open %s: %s\n",
+			   __func__, path, lkl_strerror(fd));
+		return -1;
+	}
+
+	/* Need to print this out in order to make sense of the output */
+	lkl_printf("Reading from %s:\n==========\n", path);
+	while ((ret = lkl_sys_read(fd, str, sizeof(str) - 1)) > 0)
+		bytes_read += lkl_printf("%s", str);
+	lkl_printf("==========\n");
+
+	if (ret) {
+		lkl_printf("%s lkl_sys_read %s: %s\n",
+			   __func__, path, lkl_strerror(ret));
+		return -1;
+	}
+
+	return 0;
+}
+
+static void mount_cmds_exec(char *_cmds, int (*callback)(char *))
+{
+	char *saveptr = NULL, *token;
+	int ret = 0;
+	char *cmds = strdup(_cmds);
+
+	token = strtok_r(cmds, ",", &saveptr);
+
+	while (token && ret >= 0) {
+		ret = callback(token);
+		token = strtok_r(NULL, ",", &saveptr);
+	}
+
+	if (ret < 0)
+		lkl_printf("%s: failed parsing %s\n", __func__, _cmds);
+
+	free(cmds);
+}
+
+static int lkl_config_netdev_create(struct lkl_config *cfg,
+				    struct lkl_config_iface *iface)
+{
+	int ret, offload = 0;
+	struct lkl_netdev_args nd_args;
+	__lkl__u8 mac[LKL_ETH_ALEN] = {0};
+	struct lkl_netdev *nd = NULL;
+
+	if (iface->ifoffload_str)
+		offload = strtol(iface->ifoffload_str, NULL, 0);
+	memset(&nd_args, 0, sizeof(struct lkl_netdev_args));
+
+	if (!nd && iface->iftype && iface->ifparams) {
+	}
+
+	if (nd) {
+		if ((mac[0] != 0) || (mac[1] != 0) ||
+				(mac[2] != 0) || (mac[3] != 0) ||
+				(mac[4] != 0) || (mac[5] != 0)) {
+			nd_args.mac = mac;
+		} else {
+			ret = parse_mac_str(iface->ifmac_str, mac);
+
+			if (ret < 0) {
+				lkl_printf("failed to parse mac\n");
+				return -1;
+			} else if (ret > 0) {
+				nd_args.mac = mac;
+			} else {
+				nd_args.mac = NULL;
+			}
+		}
+
+		nd_args.offload = offload;
+		iface->nd = nd;
+	}
+	return 0;
+}
+
+static int lkl_config_netdev_configure(struct lkl_config *cfg,
+				       struct lkl_config_iface *iface)
+{
+	int ret, nd_ifindex = -1;
+	struct lkl_netdev *nd = iface->nd;
+
+	if (!nd) {
+		lkl_printf("no netdev available %s\n", iface ? iface->ifparams
+			   : "(null)");
+		return -1;
+	}
+
+	if (nd->id >= 0) {
+		nd_ifindex = lkl_netdev_get_ifindex(nd->id);
+		if (nd_ifindex > 0)
+			lkl_if_up(nd_ifindex);
+		else
+			lkl_printf(
+				"failed to get ifindex for netdev id %d: %s\n",
+				nd->id, lkl_strerror(nd_ifindex));
+	}
+
+	if (nd_ifindex >= 0 && iface->ifmtu_str) {
+		int mtu = atoi(iface->ifmtu_str);
+
+		ret = lkl_if_set_mtu(nd_ifindex, mtu);
+		if (ret < 0)
+			lkl_printf("failed to set MTU: %s\n",
+				   lkl_strerror(ret));
+	}
+
+	if (nd_ifindex >= 0 && iface->ifip && iface->ifnetmask_len) {
+		unsigned int addr;
+
+		if (inet_pton(LKL_AF_INET, iface->ifip,
+			      (struct lkl_in_addr *)&addr) != 1)
+			lkl_printf("Invalid ipv4 address: %s\n", iface->ifip);
+
+		int nmlen = atoi(iface->ifnetmask_len);
+
+		if (addr != LKL_INADDR_NONE && nmlen > 0 && nmlen < 32) {
+			ret = lkl_if_set_ipv4(nd_ifindex, addr, nmlen);
+			if (ret < 0)
+				lkl_printf("failed to set IPv4 address: %s\n",
+					   lkl_strerror(ret));
+		}
+		if (iface->ifgateway) {
+			unsigned int gwaddr;
+
+			if (inet_pton(LKL_AF_INET, iface->ifgateway,
+				      (struct lkl_in_addr *)&gwaddr) != 1)
+				lkl_printf("Invalid ipv4 gateway: %s\n",
+					   iface->ifgateway);
+
+			if (gwaddr != LKL_INADDR_NONE) {
+				ret = lkl_if_set_ipv4_gateway(nd_ifindex,
+						addr, nmlen, gwaddr);
+				if (ret < 0)
+					lkl_printf(
+						"failed to set v4 if gw: %s\n",
+						lkl_strerror(ret));
+			}
+		}
+	}
+
+	if (nd_ifindex >= 0 && iface->ifipv6 &&
+			iface->ifnetmask6_len) {
+		struct lkl_in6_addr addr;
+		unsigned int pflen = atoi(iface->ifnetmask6_len);
+
+		if (inet_pton(LKL_AF_INET6, iface->ifipv6,
+			      (struct lkl_in6_addr *)&addr) != 1) {
+			lkl_printf("Invalid ipv6 addr: %s\n",
+				   iface->ifipv6);
+		}  else {
+			ret = lkl_if_set_ipv6(nd_ifindex, &addr, pflen);
+			if (ret < 0)
+				lkl_printf("failed to set IPv6 address: %s\n",
+					   lkl_strerror(ret));
+		}
+		if (iface->ifgateway6) {
+			char gwaddr[16];
+
+			if (inet_pton(LKL_AF_INET6, iface->ifgateway6,
+								gwaddr) != 1) {
+				lkl_printf("Invalid ipv6 gateway: %s\n",
+					   iface->ifgateway6);
+			} else {
+				ret = lkl_if_set_ipv6_gateway(nd_ifindex,
+						&addr, pflen, gwaddr);
+				if (ret < 0)
+					lkl_printf(
+						"failed to set v6 if gw: %s\n",
+						lkl_strerror(ret));
+			}
+		}
+	}
+
+	if (nd_ifindex >= 0 && iface->ifneigh_entries)
+		add_neighbor(nd_ifindex, iface->ifneigh_entries);
+
+	if (nd_ifindex >= 0 && iface->ifqdisc_entries)
+		lkl_qdisc_parse_add(nd_ifindex, iface->ifqdisc_entries);
+
+	return 0;
+}
+
+static void free_cfgparam(char *cfgparam)
+{
+	if (cfgparam)
+		free(cfgparam);
+}
+
+static int lkl_clean_config(struct lkl_config *cfg)
+{
+	struct lkl_config_iface *iface;
+
+	if (!cfg)
+		return -1;
+
+	for (iface = cfg->ifaces; iface; iface = iface->next) {
+		free_cfgparam(iface->iftype);
+		free_cfgparam(iface->ifparams);
+		free_cfgparam(iface->ifmtu_str);
+		free_cfgparam(iface->ifip);
+		free_cfgparam(iface->ifipv6);
+		free_cfgparam(iface->ifgateway);
+		free_cfgparam(iface->ifgateway6);
+		free_cfgparam(iface->ifmac_str);
+		free_cfgparam(iface->ifnetmask_len);
+		free_cfgparam(iface->ifnetmask6_len);
+		free_cfgparam(iface->ifoffload_str);
+		free_cfgparam(iface->ifneigh_entries);
+		free_cfgparam(iface->ifqdisc_entries);
+	}
+	free_cfgparam(cfg->gateway);
+	free_cfgparam(cfg->gateway6);
+	free_cfgparam(cfg->debug);
+	free_cfgparam(cfg->mount);
+	free_cfgparam(cfg->single_cpu);
+	free_cfgparam(cfg->sysctls);
+	free_cfgparam(cfg->boot_cmdline);
+	free_cfgparam(cfg->dump);
+	free_cfgparam(cfg->delay_main);
+	return 0;
+}
+
+
+int lkl_load_config_pre(struct lkl_config *cfg)
+{
+	int lkl_debug, ret;
+	struct lkl_config_iface *iface;
+
+	if (!cfg)
+		return 0;
+
+	if (cfg->debug)
+		lkl_debug = strtol(cfg->debug, NULL, 0);
+
+	if (!cfg->debug || (lkl_debug == 0))
+		lkl_host_ops.print = NULL;
+
+	for (iface = cfg->ifaces; iface; iface = iface->next) {
+		ret = lkl_config_netdev_create(cfg, iface);
+		if (ret < 0)
+			return -1;
+	}
+
+	return 0;
+}
+
+int lkl_load_config_post(struct lkl_config *cfg)
+{
+	int ret;
+	struct lkl_config_iface *iface;
+
+	if (!cfg)
+		return 0;
+
+	if (cfg->mount)
+		mount_cmds_exec(cfg->mount, lkl_mount_fs);
+
+	for (iface = cfg->ifaces; iface; iface = iface->next) {
+		ret = lkl_config_netdev_configure(cfg, iface);
+		if (ret < 0)
+			break;
+	}
+
+	if (cfg->gateway) {
+		unsigned int gwaddr;
+
+		if (inet_pton(LKL_AF_INET, cfg->gateway,
+			      (struct lkl_in_addr *)&gwaddr) != 1)
+			lkl_printf("Invalid ipv4 gateway: %s\n", cfg->gateway);
+
+		if (gwaddr != LKL_INADDR_NONE) {
+			ret = lkl_set_ipv4_gateway(gwaddr);
+			if (ret < 0)
+				lkl_printf("failed to set IPv4 gateway: %s\n",
+					   lkl_strerror(ret));
+		}
+	}
+
+	if (cfg->gateway6) {
+		char gw[16];
+
+		if (inet_pton(LKL_AF_INET6, cfg->gateway6, gw) != 1) {
+			lkl_printf("Invalid ipv6 gateway: %s\n", cfg->gateway6);
+		} else {
+			ret = lkl_set_ipv6_gateway(gw);
+			if (ret < 0)
+				lkl_printf("failed to set IPv6 gateway: %s\n",
+					   lkl_strerror(ret));
+		}
+	}
+
+	if (cfg->sysctls)
+		lkl_sysctl_parse_write(cfg->sysctls);
+
+	/* put a delay before calling main() */
+	if (cfg->delay_main) {
+		unsigned long delay = strtoul(cfg->delay_main, NULL, 10);
+
+		if (delay == ~0UL)
+			lkl_printf("got invalid delay_main value (%s)\n",
+				   cfg->delay_main);
+		else {
+			lkl_printf("sleeping %lu usec\n", delay);
+			usleep(delay);
+		}
+	}
+
+	return 0;
+}
+
+int lkl_unload_config(struct lkl_config *cfg)
+{
+	if (cfg) {
+		if (cfg->dump)
+			mount_cmds_exec(cfg->dump, dump_file);
+
+		lkl_clean_config(cfg);
+	}
+
+	return 0;
+}
diff --git a/tools/lkl/lib/dbg.c b/tools/lkl/lib/dbg.c
new file mode 100644
index 000000000000..b613353bce5c
--- /dev/null
+++ b/tools/lkl/lib/dbg.c
@@ -0,0 +1,300 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <errno.h>
+#include <lkl.h>
+#include <limits.h>
+#include <string.h>
+#include <stdio.h>
+#include <stdlib.h>
+
+static const char *PROMOTE = "$";
+#define str(x) #x
+#define xstr(s) str(s)
+#define MAX_BUF 100
+static char cmd[MAX_BUF];
+static char argv[10][MAX_BUF];
+static int argc;
+static char cur_dir[MAX_BUF] = "/";
+
+static char *normalize_path(const char *src, size_t src_len)
+{
+	char *res;
+	unsigned int res_len;
+	const char *ptr = src;
+	const char *end = &src[src_len];
+	const char *next;
+
+	res = malloc((src_len > 0 ? src_len : 1) + 1);
+	res_len = 0;
+
+	for (ptr = src; ptr < end; ptr = next+1) {
+		size_t len;
+
+		next = memchr(ptr, '/', end-ptr);
+		if (next == NULL)
+			next = end;
+
+		len = next-ptr;
+		switch (len) {
+		case 2:
+			if (ptr[0] == '.' && ptr[1] == '.') {
+				const char *slash = strrchr(res, '/');
+
+				if (slash != NULL)
+					res_len = slash - res;
+				continue;
+			}
+			break;
+		case 1:
+			if (ptr[0] == '.')
+				continue;
+			break;
+		case 0:
+			continue;
+		}
+		res[res_len++] = '/';
+		memcpy(&res[res_len], ptr, len);
+		res_len += len;
+	}
+	if (res_len == 0)
+		res[res_len++] = '/';
+	res[res_len] = '\0';
+	return res;
+}
+
+static void build_path(char *path)
+{
+	char *npath;
+
+	strcpy(path, cur_dir);
+	if (argc >= 1) {
+		if (argv[0][0] == '/')
+			strncpy(path, argv[0], LKL_PATH_MAX);
+		else {
+			strncat(path, "/", LKL_PATH_MAX - strlen(path) - 1);
+			strncat(path, argv[0], LKL_PATH_MAX - strlen(path) - 1);
+		}
+	}
+	npath = normalize_path(path, strlen(path));
+	strcpy(path, npath);
+	free(npath);
+}
+
+static void help(void)
+{
+	const char *msg =
+		"cat FILE\n"
+		"\tShow content of FILE\n"
+		"cd [DIR]\n"
+		"\tChange directory to DIR\n"
+		"exit\n"
+		"\tExit the debug session\n"
+		"help\n"
+		"\tShow this message\n"
+		"ls [DIR]\n"
+		"\tList files in DIR\n"
+		"mount FSTYPE\n"
+		"\tMount FSTYPE as /FSTYPE\n"
+		"overwrite FILE\n"
+		"\tOverwrite content of FILE from stdin\n"
+		"pwd\n"
+		"\tShow current directory\n"
+		;
+	printf("%s", msg);
+}
+
+static void ls(void)
+{
+	char path[LKL_PATH_MAX];
+	struct lkl_dir *dir;
+	struct lkl_linux_dirent64 *de;
+	int err;
+
+	build_path(path);
+	dir = lkl_opendir(path, &err);
+	if (dir) {
+		do {
+			de = lkl_readdir(dir);
+			if (de) {
+				printf("%s\n", de->d_name);
+			} else {
+				err = lkl_errdir(dir);
+				if (err != 0) {
+					fprintf(stderr, "%s\n",
+						lkl_strerror(err));
+				}
+				break;
+			}
+		} while (1);
+		lkl_closedir(dir);
+	} else {
+		fprintf(stderr, "%s: %s\n", path, lkl_strerror(err));
+	}
+}
+
+static void cd(void)
+{
+	char path[LKL_PATH_MAX];
+	struct lkl_dir *dir;
+	int err;
+
+	build_path(path);
+	dir = lkl_opendir(path, &err);
+	if (dir) {
+		strcpy(cur_dir, path);
+		lkl_closedir(dir);
+	} else {
+		fprintf(stderr, "%s: %s\n", path, lkl_strerror(err));
+	}
+}
+
+static void mount(void)
+{
+	char *fstype;
+	int ret = 0;
+
+	if (argc != 1) {
+		fprintf(stderr, "%s\n", "One argument is needed.");
+		return;
+	}
+
+	fstype = argv[0];
+	ret = lkl_mount_fs(fstype);
+	if (ret == 1)
+		fprintf(stderr, "%s is already mounted.\n", fstype);
+}
+
+static void cat(void)
+{
+	char path[LKL_PATH_MAX];
+	int ret;
+	char buf[1024];
+	int fd;
+
+	if (argc != 1) {
+		fprintf(stderr, "%s\n", "One argument is needed.");
+		return;
+	}
+
+	build_path(path);
+	fd = lkl_sys_open(path, LKL_O_RDONLY, 0);
+
+	if (fd < 0) {
+		fprintf(stderr, "lkl_sys_open %s: %s\n",
+			path, lkl_strerror(fd));
+		return;
+	}
+
+	while ((ret = lkl_sys_read(fd, buf, sizeof(buf) - 1)) > 0) {
+		buf[ret] = '\0';
+		printf("%s", buf);
+	}
+
+	if (ret) {
+		fprintf(stderr, "lkl_sys_read %s: %s\n",
+			path, lkl_strerror(ret));
+	}
+	lkl_sys_close(fd);
+}
+
+static void overwrite(void)
+{
+	char path[LKL_PATH_MAX];
+	int ret;
+	int fd;
+	char buf[1024];
+
+	build_path(path);
+	fd = lkl_sys_open(path, LKL_O_WRONLY | LKL_O_CREAT, 0);
+	if (fd < 0) {
+		fprintf(stderr, "lkl_sys_open %s: %s\n",
+			path, lkl_strerror(fd));
+		return;
+	}
+	printf("Input the content and stop by hitting Ctrl-D:\n");
+	while (fgets(buf, 1023, stdin)) {
+		ret = lkl_sys_write(fd, buf, strlen(buf));
+		if (ret < 0) {
+			fprintf(stderr, "lkl_sys_write %s: %s\n",
+				path, lkl_strerror(fd));
+		}
+	}
+	lkl_sys_close(fd);
+}
+
+static void pwd(void)
+{
+	printf("%s\n", cur_dir);
+}
+
+static int parse_cmd(char *input)
+{
+	char *token;
+
+	token = strtok(input, " ");
+	if (token)
+		strcpy(cmd, token);
+	else
+		return -1;
+
+	argc = 0;
+	token = strtok(NULL, " ");
+	while (token) {
+		if (argc >= 10) {
+			fprintf(stderr, "To many args > 10\n");
+			return -1;
+		}
+		strcpy(argv[argc++], token);
+		token = strtok(NULL, " ");
+	}
+	return 0;
+}
+
+static void run_cmd(void)
+{
+	if (strcmp(cmd, "cat") == 0)
+		cat();
+	else if (strcmp(cmd, "cd") == 0)
+		cd();
+	else if (strcmp(cmd, "help") == 0)
+		help();
+	else if (strcmp(cmd, "ls") == 0)
+		ls();
+	else if (strcmp(cmd, "mount") == 0)
+		mount();
+	else if (strcmp(cmd, "overwrite") == 0)
+		overwrite();
+	else if (strcmp(cmd, "pwd") == 0)
+		pwd();
+	else
+		fprintf(stderr, "Unknown command: %s\n", cmd);
+}
+
+void dbg_entrance(void)
+{
+	char input[MAX_BUF + 1];
+	int ret;
+	int c;
+
+	printf("Type help to see a list of commands\n");
+	do {
+		printf("%s ", PROMOTE);
+		ret = scanf("%" xstr(MAX_BUF) "[^\n]s", input);
+		while ((c = getchar()) != '\n' && c != EOF)
+			;
+		if (ret == 0)
+			continue;
+		if (ret != 1 && errno != EINTR) {
+			perror("scanf");
+			continue;
+		}
+		if (strlen(input) == MAX_BUF) {
+			fprintf(stderr, "Too long input > %d\n", MAX_BUF - 1);
+			continue;
+		}
+		if (parse_cmd(input))
+			continue;
+		if (strcmp(cmd, "exit") == 0)
+			break;
+		run_cmd();
+	} while (1);
+}
diff --git a/tools/lkl/lib/dbg_handler.c b/tools/lkl/lib/dbg_handler.c
new file mode 100644
index 000000000000..01d165a5fc1e
--- /dev/null
+++ b/tools/lkl/lib/dbg_handler.c
@@ -0,0 +1,44 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <lkl_host.h>
+
+extern void dbg_entrance(void);
+static int dbg_running;
+
+static void dbg_thread(void *arg)
+{
+	lkl_host_ops.thread_detach();
+	printf("======Enter Debug======\n");
+	dbg_entrance();
+	printf("======Exit Debug======\n");
+	dbg_running = 0;
+}
+
+void dbg_handler(int signum)
+{
+	/* We don't care about the possible race on dbg_running. */
+	if (dbg_running) {
+		fprintf(stderr, "A debug lib is running\n");
+		return;
+	}
+	dbg_running = 1;
+	lkl_host_ops.thread_create(&dbg_thread, NULL);
+}
+
+#ifndef __MINGW32__
+#include <signal.h>
+void lkl_register_dbg_handler(void)
+{
+	struct sigaction sa;
+
+	sigemptyset(&sa.sa_mask);
+	sa.sa_handler = dbg_handler;
+	if (sigaction(SIGTSTP, &sa, NULL) == -1)
+		perror("sigaction");
+}
+#else
+void lkl_register_dbg_handler(void)
+{
+	fprintf(stderr, "%s is not implemented.\n", __func__);
+}
+#endif
diff --git a/tools/lkl/lib/endian.h b/tools/lkl/lib/endian.h
new file mode 100644
index 000000000000..aaccfa0edb65
--- /dev/null
+++ b/tools/lkl/lib/endian.h
@@ -0,0 +1,31 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_LIB_ENDIAN_H
+#define _LKL_LIB_ENDIAN_H
+
+#if defined(__FreeBSD__)
+#include <sys/endian.h>
+#elif defined(__ANDROID__)
+#include <sys/endian.h>
+#elif defined(__MINGW32__)
+#include <winsock.h>
+#define le32toh(x) (x)
+#define le16toh(x) (x)
+#define htole32(x) (x)
+#define htole16(x) (x)
+#define le64toh(x) (x)
+#define htobe32(x) htonl(x)
+#define htobe16(x) htons(x)
+#define be32toh(x) ntohl(x)
+#define be16toh(x) ntohs(x)
+#else
+#include <endian.h>
+#endif
+
+#ifndef htonl
+#define htonl(x) htobe32(x)
+#define htons(x) htobe16(x)
+#define ntohl(x) be32toh(x)
+#define ntohs(x) be16toh(x)
+#endif
+
+#endif /* _LKL_LIB_ENDIAN_H */
diff --git a/tools/lkl/lib/jmp_buf.c b/tools/lkl/lib/jmp_buf.c
new file mode 100644
index 000000000000..f6bdd7e4bd83
--- /dev/null
+++ b/tools/lkl/lib/jmp_buf.c
@@ -0,0 +1,14 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <setjmp.h>
+#include <lkl_host.h>
+
+void jmp_buf_set(struct lkl_jmp_buf *jmpb, void (*f)(void))
+{
+	if (!setjmp(*((jmp_buf *)jmpb->buf)))
+		f();
+}
+
+void jmp_buf_longjmp(struct lkl_jmp_buf *jmpb, int val)
+{
+	longjmp(*((jmp_buf *)jmpb->buf), val);
+}
diff --git a/tools/lkl/lib/jmp_buf.h b/tools/lkl/lib/jmp_buf.h
new file mode 100644
index 000000000000..8782cbaaf51f
--- /dev/null
+++ b/tools/lkl/lib/jmp_buf.h
@@ -0,0 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_LIB_JMP_BUF_H
+#define _LKL_LIB_JMP_BUF_H
+
+void jmp_buf_set(struct lkl_jmp_buf *jmpb, void (*f)(void));
+void jmp_buf_longjmp(struct lkl_jmp_buf *jmpb, int val);
+
+#endif
diff --git a/tools/lkl/lib/utils.c b/tools/lkl/lib/utils.c
new file mode 100644
index 000000000000..7de92bbe5475
--- /dev/null
+++ b/tools/lkl/lib/utils.c
@@ -0,0 +1,266 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdarg.h>
+#include <stdio.h>
+#include <string.h>
+#include <lkl_host.h>
+
+static const char * const lkl_err_strings[] = {
+	"Success",
+	"Operation not permitted",
+	"No such file or directory",
+	"No such process",
+	"Interrupted system call",
+	"I/O error",
+	"No such device or address",
+	"Argument list too long",
+	"Exec format error",
+	"Bad file number",
+	"No child processes",
+	"Try again",
+	"Out of memory",
+	"Permission denied",
+	"Bad address",
+	"Block device required",
+	"Device or resource busy",
+	"File exists",
+	"Cross-device link",
+	"No such device",
+	"Not a directory",
+	"Is a directory",
+	"Invalid argument",
+	"File table overflow",
+	"Too many open files",
+	"Not a typewriter",
+	"Text file busy",
+	"File too large",
+	"No space left on device",
+	"Illegal seek",
+	"Read-only file system",
+	"Too many links",
+	"Broken pipe",
+	"Math argument out of domain of func",
+	"Math result not representable",
+	"Resource deadlock would occur",
+	"File name too long",
+	"No record locks available",
+	"Invalid system call number",
+	"Directory not empty",
+	"Too many symbolic links encountered",
+	"Bad error code", /* EWOULDBLOCK is EAGAIN */
+	"No message of desired type",
+	"Identifier removed",
+	"Channel number out of range",
+	"Level 2 not synchronized",
+	"Level 3 halted",
+	"Level 3 reset",
+	"Link number out of range",
+	"Protocol driver not attached",
+	"No CSI structure available",
+	"Level 2 halted",
+	"Invalid exchange",
+	"Invalid request descriptor",
+	"Exchange full",
+	"No anode",
+	"Invalid request code",
+	"Invalid slot",
+	"Bad error code", /* EDEADLOCK is EDEADLK */
+	"Bad font file format",
+	"Device not a stream",
+	"No data available",
+	"Timer expired",
+	"Out of streams resources",
+	"Machine is not on the network",
+	"Package not installed",
+	"Object is remote",
+	"Link has been severed",
+	"Advertise error",
+	"Srmount error",
+	"Communication error on send",
+	"Protocol error",
+	"Multihop attempted",
+	"RFS specific error",
+	"Not a data message",
+	"Value too large for defined data type",
+	"Name not unique on network",
+	"File descriptor in bad state",
+	"Remote address changed",
+	"Can not access a needed shared library",
+	"Accessing a corrupted shared library",
+	".lib section in a.out corrupted",
+	"Attempting to link in too many shared libraries",
+	"Cannot exec a shared library directly",
+	"Illegal byte sequence",
+	"Interrupted system call should be restarted",
+	"Streams pipe error",
+	"Too many users",
+	"Socket operation on non-socket",
+	"Destination address required",
+	"Message too long",
+	"Protocol wrong type for socket",
+	"Protocol not available",
+	"Protocol not supported",
+	"Socket type not supported",
+	"Operation not supported on transport endpoint",
+	"Protocol family not supported",
+	"Address family not supported by protocol",
+	"Address already in use",
+	"Cannot assign requested address",
+	"Network is down",
+	"Network is unreachable",
+	"Network dropped connection because of reset",
+	"Software caused connection abort",
+	"Connection reset by peer",
+	"No buffer space available",
+	"Transport endpoint is already connected",
+	"Transport endpoint is not connected",
+	"Cannot send after transport endpoint shutdown",
+	"Too many references: cannot splice",
+	"Connection timed out",
+	"Connection refused",
+	"Host is down",
+	"No route to host",
+	"Operation already in progress",
+	"Operation now in progress",
+	"Stale file handle",
+	"Structure needs cleaning",
+	"Not a XENIX named type file",
+	"No XENIX semaphores available",
+	"Is a named type file",
+	"Remote I/O error",
+	"Quota exceeded",
+	"No medium found",
+	"Wrong medium type",
+	"Operation Canceled",
+	"Required key not available",
+	"Key has expired",
+	"Key has been revoked",
+	"Key was rejected by service",
+	"Owner died",
+	"State not recoverable",
+	"Operation not possible due to RF-kill",
+	"Memory page has hardware error",
+};
+
+const char *lkl_strerror(int err)
+{
+	if (err < 0)
+		err = -err;
+
+	if ((size_t)err >= sizeof(lkl_err_strings) / sizeof(const char *))
+		return "Bad error code";
+
+	return lkl_err_strings[err];
+}
+
+void lkl_perror(char *msg, int err)
+{
+	const char *err_msg = lkl_strerror(err);
+	/* We need to use 'real' printf because lkl_host_ops.print can
+	 * be turned off when debugging is off.
+	 */
+	lkl_printf("%s: %s\n", msg, err_msg);
+}
+
+static int lkl_vprintf(const char *fmt, va_list args)
+{
+	int n;
+	char *buffer;
+	va_list copy;
+
+	if (!lkl_host_ops.print)
+		return 0;
+
+	va_copy(copy, args);
+	n = vsnprintf(NULL, 0, fmt, copy);
+	va_end(copy);
+
+	buffer = lkl_host_ops.mem_alloc(n + 1);
+	if (!buffer)
+		return -1;
+
+	vsnprintf(buffer, n + 1, fmt, args);
+
+	lkl_host_ops.print(buffer, n);
+	lkl_host_ops.mem_free(buffer);
+
+	return n;
+}
+
+int lkl_printf(const char *fmt, ...)
+{
+	int n;
+	va_list args;
+
+	va_start(args, fmt);
+	n = lkl_vprintf(fmt, args);
+	va_end(args);
+
+	return n;
+}
+
+void lkl_bug(const char *fmt, ...)
+{
+	va_list args;
+
+	va_start(args, fmt);
+	lkl_vprintf(fmt, args);
+	va_end(args);
+
+	lkl_host_ops.panic();
+}
+#ifndef __arch_um__
+int lkl_sysctl(const char *path, const char *value)
+{
+	int ret;
+	int fd;
+	char *delim, *p;
+	char full_path[256];
+
+	lkl_mount_fs("proc");
+
+	snprintf(full_path, sizeof(full_path), "/proc/sys/%s", path);
+	p = full_path;
+	while ((delim = strstr(p, "."))) {
+		*delim = '/';
+		p = delim + 1;
+	}
+
+	fd = lkl_sys_open(full_path, LKL_O_WRONLY | LKL_O_CREAT, 0);
+	if (fd < 0) {
+		lkl_printf("lkl_sys_open %s: %s\n",
+			   full_path, lkl_strerror(fd));
+		return -1;
+	}
+	ret = lkl_sys_write(fd, value, strlen(value));
+	if (ret < 0) {
+		lkl_printf("lkl_sys_write %s: %s\n",
+			full_path, lkl_strerror(fd));
+	}
+
+	lkl_sys_close(fd);
+
+	return 0;
+}
+
+/* Configure sysctl parameters as the form of "key=value;key=value;..." */
+void lkl_sysctl_parse_write(const char *sysctls)
+{
+	char *saveptr = NULL, *token = NULL;
+	char *key = NULL, *value = NULL;
+	char strings[256];
+	int ret = 0;
+
+	strcpy(strings, sysctls);
+	for (token = strtok_r(strings, ";", &saveptr); token;
+	     token = strtok_r(NULL, ";", &saveptr)) {
+		key = strtok(token, "=");
+		value = strtok(NULL, "=");
+		ret = lkl_sysctl(key, value);
+		if (ret) {
+			lkl_printf("Failed to configure sysctl entries: %s\n",
+				   lkl_strerror(ret));
+			return;
+		}
+	}
+}
+#endif
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v3 15/26] lkl tools: host lib: filesystem helpers
  2020-02-05  7:30 [RFC v3 00/26] Unifying LKL into UML Hajime Tazaki
@ 2020-02-05  7:30   ` Hajime Tazaki
       [not found] ` <cover.1580882335.git.thehajime-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
                     ` (25 subsequent siblings)
  26 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05  7:30 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Hajime Tazaki, Conrad Meyer, Michael Zimmermann, Yuan Liu

Add LKL applications APIs to mount and unmount a filesystem from a
disk added via lkl_disk_add().

Also add open/close/read directory wrappers on top of
lkl_sys_getdents64.

Cc: Conrad Meyer <cem@FreeBSD.org>
Cc: Michael Zimmermann <sigmaepsilon92@gmail.com>
Cc: Yuan Liu <liuyuan@google.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/lkl/include/lkl.h | 165 ++++++++++++++
 tools/lkl/lib/Build     |   1 +
 tools/lkl/lib/fs.c      | 471 ++++++++++++++++++++++++++++++++++++++++
 3 files changed, 637 insertions(+)
 create mode 100644 tools/lkl/lib/fs.c

diff --git a/tools/lkl/include/lkl.h b/tools/lkl/include/lkl.h
index 4b95d0ef8e5b..cdae92300320 100644
--- a/tools/lkl/include/lkl.h
+++ b/tools/lkl/include/lkl.h
@@ -350,6 +350,171 @@ const char *lkl_strerror(int err);
  */
 void lkl_perror(char *msg, int err);
 
+/**
+ * struct lkl_dev_blk_ops - block device host operations, defined in lkl_host.h.
+ */
+struct lkl_dev_blk_ops;
+
+/**
+ * lkl_disk - host disk handle
+ *
+ * @dev - a pointer to private information for this disk backend
+ * @fd - a POSIX file descriptor that can be used by preadv/pwritev
+ * @handle - an NT file handle that can be used by ReadFile/WriteFile
+ */
+struct lkl_disk {
+	void *dev;
+	union {
+		int fd;
+		void *handle;
+	};
+	struct lkl_dev_blk_ops *ops;
+};
+
+/**
+ * lkl_disk_add - add a new disk
+ *
+ * @disk - the host disk handle
+ * @returns a disk id (0 is valid) or a strictly negative value in case of error
+ */
+int lkl_disk_add(struct lkl_disk *disk);
+
+/**
+ * lkl_disk_remove - remove a disk
+ *
+ * This function makes a cleanup of the @disk's private information
+ * that was initialized by lkl_disk_add before.
+ *
+ * @disk - the host disk handle
+ */
+int lkl_disk_remove(struct lkl_disk disk);
+
+/**
+ * lkl_encode_dev_from_sysfs_blkdev - extract device id from sysfs
+ *
+ * This function returns the device id for the given sysfs dev node.
+ * The content of the node has to be in the form 'MAJOR:MINOR'.
+ * Also, this function expects an absolute path which means that sysfs
+ * already has to be mounted at the given path
+ *
+ * @sysfs_path - absolute path to the sysfs dev node
+ * @pdevid - pointer to memory where dev id will be returned
+ * @returns - 0 on success, a negative value on error
+ */
+int lkl_encode_dev_from_sysfs(const char *sysfs_path, uint32_t *pdevid);
+
+/**
+ * lkl_mount_dev - mount a disk
+ *
+ * This functions creates a device file for the given disk, creates a mount
+ * point and mounts the device over the mount point.
+ *
+ * @disk_id - the disk id identifying the disk to be mounted
+ * @part - disk partition or zero for full disk
+ * @fs_type - filesystem type
+ * @flags - mount flags
+ * @opts - additional filesystem specific mount options
+ * @mnt_str - a string that will be filled by this function with the path where
+ * the filesystem has been mounted
+ * @mnt_str_len - size of mnt_str
+ * @returns - 0 on success, a negative value on error
+ */
+long lkl_mount_dev(unsigned int disk_id, unsigned int part, const char *fs_type,
+		   int flags, const char *opts,
+		   char *mnt_str, unsigned int mnt_str_len);
+
+/**
+ * lkl_umount_dev - umount a disk
+ *
+ * This functions umounts the given disks and removes the device file and the
+ * mount point.
+ *
+ * @disk_id - the disk id identifying the disk to be mounted
+ * @part - disk partition or zero for full disk
+ * @flags - umount flags
+ * @timeout_ms - timeout to wait for the kernel to flush closed files so that
+ * umount can succeed
+ * @returns - 0 on success, a negative value on error
+ */
+long lkl_umount_dev(unsigned int disk_id, unsigned int part, int flags,
+		    long timeout_ms);
+
+/**
+ * lkl_umount_timeout - umount filesystem with timeout
+ *
+ * @path - the path to unmount
+ * @flags - umount flags
+ * @timeout_ms - timeout to wait for the kernel to flush closed files so that
+ * umount can succeed
+ * @returns - 0 on success, a negative value on error
+ */
+long lkl_umount_timeout(char *path, int flags, long timeout_ms);
+
+/**
+ * lkl_opendir - open a directory
+ *
+ * @path - directory path
+ * @err - pointer to store the error in case of failure
+ * @returns - a handle to be used when calling lkl_readdir
+ */
+struct lkl_dir *lkl_opendir(const char *path, int *err);
+
+/**
+ * lkl_fdopendir - open a directory
+ *
+ * @fd - file descriptor
+ * @err - pointer to store the error in case of failure
+ * @returns - a handle to be used when calling lkl_readdir
+ */
+struct lkl_dir *lkl_fdopendir(int fd, int *err);
+
+/**
+ * lkl_rewinddir - reset directory stream
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ */
+void lkl_rewinddir(struct lkl_dir *dir);
+
+/**
+ * lkl_closedir - close the directory
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ */
+int lkl_closedir(struct lkl_dir *dir);
+
+/**
+ * lkl_readdir - get the next available entry of the directory
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ * @returns - a lkl_dirent64 entry or NULL if the end of the directory stream is
+ * reached or if an error occurred; check lkl_errdir() to distinguish between
+ * errors or end of the directory stream
+ */
+struct lkl_linux_dirent64 *lkl_readdir(struct lkl_dir *dir);
+
+/**
+ * lkl_errdir - checks if an error occurred during the last lkl_readdir call
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ * @returns - 0 if no error occurred, or a negative value otherwise
+ */
+int lkl_errdir(struct lkl_dir *dir);
+
+/**
+ * lkl_dirfd - gets the file descriptor associated with the directory handle
+ *
+ * @dir - the directory handle as returned by lkl_opendir
+ * @returns - a positive value,which is the LKL file descriptor associated with
+ * the directory handle, or a negative value otherwise
+ */
+int lkl_dirfd(struct lkl_dir *dir);
+
+/**
+ * lkl_mount_fs - mount a file system type like proc, sys
+ * @fstype - file system type. e.g. proc, sys
+ * @returns - 0 on success. 1 if it's already mounted. negative on failure.
+ */
+int lkl_mount_fs(char *fstype);
 
 #ifdef __cplusplus
 }
diff --git a/tools/lkl/lib/Build b/tools/lkl/lib/Build
index 658bfa865b9c..4a444337b0e7 100644
--- a/tools/lkl/lib/Build
+++ b/tools/lkl/lib/Build
@@ -1,5 +1,6 @@
 CFLAGS_config.o += -I$(srctree)/tools/perf/pmu-events
 
+liblkl-y += fs.o
 liblkl-y += jmp_buf.o
 liblkl-y += utils.o
 liblkl-y += dbg.o
diff --git a/tools/lkl/lib/fs.c b/tools/lkl/lib/fs.c
new file mode 100644
index 000000000000..d51d928c14b6
--- /dev/null
+++ b/tools/lkl/lib/fs.c
@@ -0,0 +1,471 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdarg.h>
+#include <stdio.h>
+#include <string.h>
+#include <stdlib.h>
+#include <lkl_host.h>
+
+#define MAX_FSTYPE_LEN 50
+
+static struct lkl_disk *lkl_disks[16];
+
+int lkl_disk_add(struct lkl_disk *disk)
+{
+	int ret = -1;
+
+	switch (disk->backend) {
+	case BLK_BACKEND_UM:
+		ret = lkl_disk_um_add(disk, disk->dev);
+		break;
+	default:
+		break;
+	}
+
+	lkl_disks[ret] = disk;
+
+	return ret;
+}
+
+int lkl_disk_remove(struct lkl_disk disk)
+{
+	switch (disk.backend) {
+	case BLK_BACKEND_UM:
+	default:
+		break;
+	}
+
+	return 0;
+}
+
+int lkl_mount_fs(char *fstype)
+{
+	char dir[MAX_FSTYPE_LEN+2] = "/";
+	int flags = 0, ret = 0;
+
+	strncat(dir, fstype, MAX_FSTYPE_LEN);
+
+	/* Create with regular umask */
+	ret = lkl_sys_mkdir(dir, 0xff);
+	if (ret && ret != -LKL_EEXIST) {
+		lkl_perror("mount_fs mkdir", ret);
+		return ret;
+	}
+
+	/* We have no use for nonzero flags right now */
+	ret = lkl_sys_mount("none", dir, fstype, flags, NULL);
+	if (ret && ret != -LKL_EBUSY) {
+		lkl_sys_rmdir(dir);
+		return ret;
+	}
+
+	if (ret == -LKL_EBUSY)
+		return 1;
+	return 0;
+}
+
+static uint32_t new_encode_dev(unsigned int major, unsigned int minor)
+{
+	return (minor & 0xff) | (major << 8) | ((minor & ~0xff) << 12);
+}
+
+static int startswith(const char *str, const char *pre)
+{
+	return strncmp(pre, str, strlen(pre)) == 0;
+}
+
+static int get_node_with_prefix(const char *path, const char *prefix,
+				char *result, unsigned int result_len)
+{
+	struct lkl_dir *dir = NULL;
+	struct lkl_linux_dirent64 *dirent;
+	int ret;
+
+	dir = lkl_opendir(path, &ret);
+	if (!dir)
+		return ret;
+
+	ret = -LKL_ENOENT;
+
+	while ((dirent = lkl_readdir(dir))) {
+		if (startswith(dirent->d_name, prefix)) {
+			if (strlen(dirent->d_name) + 1 > result_len) {
+				ret = -LKL_ENOMEM;
+				break;
+			}
+			memcpy(result, dirent->d_name, strlen(dirent->d_name));
+			result[strlen(dirent->d_name)] = '\0';
+			ret = 0;
+			break;
+		}
+	}
+
+	lkl_closedir(dir);
+
+	return ret;
+}
+
+int lkl_encode_dev_from_sysfs(const char *sysfs_path, uint32_t *pdevid)
+{
+	int ret;
+	long fd;
+	int major, minor;
+	char buf[16] = { 0, };
+	char *bufptr;
+
+	fd = lkl_sys_open(sysfs_path, LKL_O_RDONLY, 0);
+	if (fd < 0)
+		return fd;
+
+	ret = lkl_sys_read(fd, buf, sizeof(buf));
+	if (ret < 0)
+		goto out_close;
+
+	if (ret == sizeof(buf)) {
+		ret = -LKL_ENOBUFS;
+		goto out_close;
+	}
+
+	bufptr = strchr(buf, ':');
+	if (bufptr == NULL) {
+		ret = -LKL_EINVAL;
+		goto out_close;
+	}
+	bufptr[0] = '\0';
+	bufptr++;
+
+	major = atoi(buf);
+	minor = atoi(bufptr);
+
+	*pdevid = new_encode_dev(major, minor);
+	ret = 0;
+
+out_close:
+	lkl_sys_close(fd);
+
+	return ret;
+}
+
+#define SYSFS_DEV_UMBLK_CMDLINE_PATH \
+	"/sysfs/devices/platform/uml-blkdev.%d"
+
+struct abuf {
+	char *mem, *ptr;
+	unsigned int len;
+};
+
+static int snprintf_append(struct abuf *buf, const char *fmt, ...)
+{
+	int ret;
+	va_list args;
+
+	if (!buf->ptr)
+		buf->ptr = buf->mem;
+
+	va_start(args, fmt);
+	ret = vsnprintf(buf->ptr, buf->len - (buf->ptr - buf->mem), fmt, args);
+	va_end(args);
+
+	if (ret < 0 || (ret >= (int)(buf->len - (buf->ptr - buf->mem))))
+		return -LKL_ENOMEM;
+
+	buf->ptr += ret;
+
+	return 0;
+}
+
+static int __lkl_get_blkdev(int disk_id, unsigned int part, uint32_t *pdevid,
+			    const char *sysfs_path_fmt, const char *drv_prefix,
+			    const char *disk_prefix)
+{
+	char sysfs_path[LKL_PATH_MAX];
+	char drv_name[LKL_PATH_MAX];
+	char disk_name[LKL_PATH_MAX];
+	struct abuf sysfs_path_buf = {
+		.mem = sysfs_path,
+		.len = sizeof(sysfs_path),
+	};
+	int ret;
+
+	if (disk_id < 0)
+		return -LKL_EINVAL;
+
+	ret = lkl_mount_fs("sysfs");
+	if (ret < 0)
+		return ret;
+
+	ret = snprintf_append(&sysfs_path_buf, sysfs_path_fmt, disk_id);
+	if (ret)
+		return ret;
+
+	ret = get_node_with_prefix(sysfs_path, drv_prefix, drv_name,
+				   sizeof(drv_name));
+	if (ret)
+		return ret;
+
+	ret = snprintf_append(&sysfs_path_buf, "/%s/block", drv_name);
+	if (ret)
+		return ret;
+
+	ret = get_node_with_prefix(sysfs_path, disk_prefix, disk_name,
+				   sizeof(disk_name));
+	if (ret)
+		return ret;
+
+	if (!part)
+		ret = snprintf_append(&sysfs_path_buf, "/%s/dev", disk_name);
+	else
+		ret = snprintf_append(&sysfs_path_buf, "/%s/%s%d/dev",
+				      disk_name, disk_name, part);
+	if (ret)
+		return ret;
+
+	return lkl_encode_dev_from_sysfs(sysfs_path, pdevid);
+}
+
+int lkl_get_blkdev(int disk_id, unsigned int part, uint32_t *pdevid)
+{
+	char *fmt;
+	struct lkl_disk *disk = lkl_disks[disk_id];
+
+	switch (disk->backend) {
+	case BLK_BACKEND_UM:
+		fmt = SYSFS_DEV_UMBLK_CMDLINE_PATH;
+		return __lkl_get_blkdev(disk_id, part, pdevid, fmt,
+					"", "ubd");
+	default:
+		break;
+	}
+
+	return -1;
+}
+
+long lkl_mount_dev(unsigned int disk_id, unsigned int part,
+		   const char *fs_type, int flags,
+		   const char *data, char *mnt_str, unsigned int mnt_str_len)
+{
+	char dev_str[] = { "/dev/xxxxxxxx" };
+	unsigned int dev;
+	int err;
+	char _data[4096]; /* FIXME: PAGE_SIZE is not exported by LKL */
+
+	if (mnt_str_len < sizeof(dev_str))
+		return -LKL_ENOMEM;
+
+	err = lkl_get_blkdev(disk_id, part, &dev);
+	if (err < 0)
+		return err;
+
+	snprintf(dev_str, sizeof(dev_str), "/dev/%08x", dev);
+	snprintf(mnt_str, mnt_str_len, "/mnt/%08x", dev);
+
+	err = lkl_sys_access("/dev", LKL_S_IRWXO);
+	if (err < 0) {
+		if (err == -LKL_ENOENT)
+			err = lkl_sys_mkdir("/dev", 0700);
+		if (err < 0)
+			return err;
+	}
+
+	err = lkl_sys_mknod(dev_str, LKL_S_IFBLK | 0600, dev);
+	if (err < 0)
+		return err;
+
+	err = lkl_sys_access("/mnt", LKL_S_IRWXO);
+	if (err < 0) {
+		if (err == -LKL_ENOENT)
+			err = lkl_sys_mkdir("/mnt", 0700);
+		if (err < 0)
+			return err;
+	}
+
+	err = lkl_sys_mkdir(mnt_str, 0700);
+	if (err < 0) {
+		lkl_sys_unlink(dev_str);
+		return err;
+	}
+
+	/* kernel always copies a full page */
+	if (data) {
+		strncpy(_data, data, sizeof(_data));
+		_data[sizeof(_data) - 1] = 0;
+	} else {
+		_data[0] = 0;
+	}
+
+	err = lkl_sys_mount(dev_str, mnt_str, (char *)fs_type, flags, _data);
+	if (err < 0) {
+		lkl_sys_unlink(dev_str);
+		lkl_sys_rmdir(mnt_str);
+		return err;
+	}
+
+	return 0;
+}
+
+long lkl_umount_timeout(char *path, int flags, long timeout_ms)
+{
+	long incr = 10000000; /* 10 ms */
+	struct lkl_timespec ts = {
+		.tv_sec = 0,
+		.tv_nsec = incr,
+	};
+	long err;
+
+	do {
+		err = lkl_sys_umount(path, flags);
+		if (err == -LKL_EBUSY) {
+			lkl_sys_nanosleep((struct __lkl__kernel_timespec *)&ts,
+					  NULL);
+			timeout_ms -= incr / 1000000;
+		}
+	} while (err == -LKL_EBUSY && timeout_ms > 0);
+
+	return err;
+}
+
+long lkl_umount_dev(unsigned int disk_id, unsigned int part, int flags,
+		    long timeout_ms)
+{
+	char dev_str[] = { "/dev/xxxxxxxx" };
+	char mnt_str[] = { "/mnt/xxxxxxxx" };
+	unsigned int dev;
+	int err;
+
+	err = lkl_get_blkdev(disk_id, part, &dev);
+	if (err < 0)
+		return err;
+
+	snprintf(dev_str, sizeof(dev_str), "/dev/%08x", dev);
+	snprintf(mnt_str, sizeof(mnt_str), "/mnt/%08x", dev);
+
+	err = lkl_umount_timeout(mnt_str, flags, timeout_ms);
+	if (err)
+		return err;
+
+	err = lkl_sys_unlink(dev_str);
+	if (err)
+		return err;
+
+	return lkl_sys_rmdir(mnt_str);
+}
+
+struct lkl_dir {
+	int fd;
+	char buf[1024];
+	char *pos;
+	int len;
+};
+
+static struct lkl_dir *lkl_dir_alloc(int *err)
+{
+	struct lkl_dir *dir = lkl_host_ops.mem_alloc(sizeof(struct lkl_dir));
+
+	if (!dir) {
+		*err = -LKL_ENOMEM;
+		return NULL;
+	}
+
+	dir->len = 0;
+	dir->pos = NULL;
+
+	return dir;
+}
+
+struct lkl_dir *lkl_opendir(const char *path, int *err)
+{
+	struct lkl_dir *dir = lkl_dir_alloc(err);
+
+	if (!dir) {
+		*err = -LKL_ENOMEM;
+		return NULL;
+	}
+
+	dir->fd = lkl_sys_open(path, LKL_O_RDONLY | LKL_O_DIRECTORY, 0);
+	if (dir->fd < 0) {
+		*err = dir->fd;
+		lkl_host_ops.mem_free(dir);
+		return NULL;
+	}
+
+	*err = 0;
+
+	return dir;
+}
+
+struct lkl_dir *lkl_fdopendir(int fd, int *err)
+{
+	struct lkl_dir *dir = lkl_dir_alloc(err);
+
+	if (!dir)
+		return NULL;
+
+	dir->fd = fd;
+
+	return dir;
+}
+
+void lkl_rewinddir(struct lkl_dir *dir)
+{
+	lkl_sys_lseek(dir->fd, 0, LKL_SEEK_SET);
+	dir->len = 0;
+	dir->pos = NULL;
+}
+
+int lkl_closedir(struct lkl_dir *dir)
+{
+	int ret;
+
+	ret = lkl_sys_close(dir->fd);
+	lkl_host_ops.mem_free(dir);
+
+	return ret;
+}
+
+struct lkl_linux_dirent64 *lkl_readdir(struct lkl_dir *dir)
+{
+	struct lkl_linux_dirent64 *de;
+
+	if (dir->len < 0)
+		return NULL;
+
+	if (!dir->pos || dir->pos - dir->buf >= dir->len)
+		goto read_buf;
+
+return_de:
+	de = (struct lkl_linux_dirent64 *)dir->pos;
+	dir->pos += de->d_reclen;
+
+	return de;
+
+read_buf:
+	dir->pos = NULL;
+	de = (struct lkl_linux_dirent64 *)dir->buf;
+	dir->len = lkl_sys_getdents64(dir->fd, de, sizeof(dir->buf));
+	if (dir->len <= 0)
+		return NULL;
+
+	dir->pos = dir->buf;
+	goto return_de;
+}
+
+int lkl_errdir(struct lkl_dir *dir)
+{
+	if (dir->len >= 0)
+		return 0;
+
+	return dir->len;
+}
+
+int lkl_dirfd(struct lkl_dir *dir)
+{
+	return dir->fd;
+}
+
+int lkl_set_fd_limit(unsigned int fd_limit)
+{
+	struct lkl_rlimit rlim = {
+		.rlim_cur = fd_limit,
+		.rlim_max = fd_limit,
+	};
+	return lkl_sys_setrlimit(LKL_RLIMIT_NOFILE, &rlim);
+}
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v3 15/26] lkl tools: host lib: filesystem helpers
@ 2020-02-05  7:30   ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05  7:30 UTC (permalink / raw)
  To: linux-um
  Cc: linux-arch, Conrad Meyer, Octavian Purdila, Akira Moroo,
	Yuan Liu, linux-kernel-library, Michael Zimmermann,
	Hajime Tazaki

Add LKL applications APIs to mount and unmount a filesystem from a
disk added via lkl_disk_add().

Also add open/close/read directory wrappers on top of
lkl_sys_getdents64.

Cc: Conrad Meyer <cem@FreeBSD.org>
Cc: Michael Zimmermann <sigmaepsilon92@gmail.com>
Cc: Yuan Liu <liuyuan@google.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/lkl/include/lkl.h | 165 ++++++++++++++
 tools/lkl/lib/Build     |   1 +
 tools/lkl/lib/fs.c      | 471 ++++++++++++++++++++++++++++++++++++++++
 3 files changed, 637 insertions(+)
 create mode 100644 tools/lkl/lib/fs.c

diff --git a/tools/lkl/include/lkl.h b/tools/lkl/include/lkl.h
index 4b95d0ef8e5b..cdae92300320 100644
--- a/tools/lkl/include/lkl.h
+++ b/tools/lkl/include/lkl.h
@@ -350,6 +350,171 @@ const char *lkl_strerror(int err);
  */
 void lkl_perror(char *msg, int err);
 
+/**
+ * struct lkl_dev_blk_ops - block device host operations, defined in lkl_host.h.
+ */
+struct lkl_dev_blk_ops;
+
+/**
+ * lkl_disk - host disk handle
+ *
+ * @dev - a pointer to private information for this disk backend
+ * @fd - a POSIX file descriptor that can be used by preadv/pwritev
+ * @handle - an NT file handle that can be used by ReadFile/WriteFile
+ */
+struct lkl_disk {
+	void *dev;
+	union {
+		int fd;
+		void *handle;
+	};
+	struct lkl_dev_blk_ops *ops;
+};
+
+/**
+ * lkl_disk_add - add a new disk
+ *
+ * @disk - the host disk handle
+ * @returns a disk id (0 is valid) or a strictly negative value in case of error
+ */
+int lkl_disk_add(struct lkl_disk *disk);
+
+/**
+ * lkl_disk_remove - remove a disk
+ *
+ * This function makes a cleanup of the @disk's private information
+ * that was initialized by lkl_disk_add before.
+ *
+ * @disk - the host disk handle
+ */
+int lkl_disk_remove(struct lkl_disk disk);
+
+/**
+ * lkl_encode_dev_from_sysfs_blkdev - extract device id from sysfs
+ *
+ * This function returns the device id for the given sysfs dev node.
+ * The content of the node has to be in the form 'MAJOR:MINOR'.
+ * Also, this function expects an absolute path which means that sysfs
+ * already has to be mounted at the given path
+ *
+ * @sysfs_path - absolute path to the sysfs dev node
+ * @pdevid - pointer to memory where dev id will be returned
+ * @returns - 0 on success, a negative value on error
+ */
+int lkl_encode_dev_from_sysfs(const char *sysfs_path, uint32_t *pdevid);
+
+/**
+ * lkl_mount_dev - mount a disk
+ *
+ * This functions creates a device file for the given disk, creates a mount
+ * point and mounts the device over the mount point.
+ *
+ * @disk_id - the disk id identifying the disk to be mounted
+ * @part - disk partition or zero for full disk
+ * @fs_type - filesystem type
+ * @flags - mount flags
+ * @opts - additional filesystem specific mount options
+ * @mnt_str - a string that will be filled by this function with the path where
+ * the filesystem has been mounted
+ * @mnt_str_len - size of mnt_str
+ * @returns - 0 on success, a negative value on error
+ */
+long lkl_mount_dev(unsigned int disk_id, unsigned int part, const char *fs_type,
+		   int flags, const char *opts,
+		   char *mnt_str, unsigned int mnt_str_len);
+
+/**
+ * lkl_umount_dev - umount a disk
+ *
+ * This functions umounts the given disks and removes the device file and the
+ * mount point.
+ *
+ * @disk_id - the disk id identifying the disk to be mounted
+ * @part - disk partition or zero for full disk
+ * @flags - umount flags
+ * @timeout_ms - timeout to wait for the kernel to flush closed files so that
+ * umount can succeed
+ * @returns - 0 on success, a negative value on error
+ */
+long lkl_umount_dev(unsigned int disk_id, unsigned int part, int flags,
+		    long timeout_ms);
+
+/**
+ * lkl_umount_timeout - umount filesystem with timeout
+ *
+ * @path - the path to unmount
+ * @flags - umount flags
+ * @timeout_ms - timeout to wait for the kernel to flush closed files so that
+ * umount can succeed
+ * @returns - 0 on success, a negative value on error
+ */
+long lkl_umount_timeout(char *path, int flags, long timeout_ms);
+
+/**
+ * lkl_opendir - open a directory
+ *
+ * @path - directory path
+ * @err - pointer to store the error in case of failure
+ * @returns - a handle to be used when calling lkl_readdir
+ */
+struct lkl_dir *lkl_opendir(const char *path, int *err);
+
+/**
+ * lkl_fdopendir - open a directory
+ *
+ * @fd - file descriptor
+ * @err - pointer to store the error in case of failure
+ * @returns - a handle to be used when calling lkl_readdir
+ */
+struct lkl_dir *lkl_fdopendir(int fd, int *err);
+
+/**
+ * lkl_rewinddir - reset directory stream
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ */
+void lkl_rewinddir(struct lkl_dir *dir);
+
+/**
+ * lkl_closedir - close the directory
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ */
+int lkl_closedir(struct lkl_dir *dir);
+
+/**
+ * lkl_readdir - get the next available entry of the directory
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ * @returns - a lkl_dirent64 entry or NULL if the end of the directory stream is
+ * reached or if an error occurred; check lkl_errdir() to distinguish between
+ * errors or end of the directory stream
+ */
+struct lkl_linux_dirent64 *lkl_readdir(struct lkl_dir *dir);
+
+/**
+ * lkl_errdir - checks if an error occurred during the last lkl_readdir call
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ * @returns - 0 if no error occurred, or a negative value otherwise
+ */
+int lkl_errdir(struct lkl_dir *dir);
+
+/**
+ * lkl_dirfd - gets the file descriptor associated with the directory handle
+ *
+ * @dir - the directory handle as returned by lkl_opendir
+ * @returns - a positive value,which is the LKL file descriptor associated with
+ * the directory handle, or a negative value otherwise
+ */
+int lkl_dirfd(struct lkl_dir *dir);
+
+/**
+ * lkl_mount_fs - mount a file system type like proc, sys
+ * @fstype - file system type. e.g. proc, sys
+ * @returns - 0 on success. 1 if it's already mounted. negative on failure.
+ */
+int lkl_mount_fs(char *fstype);
 
 #ifdef __cplusplus
 }
diff --git a/tools/lkl/lib/Build b/tools/lkl/lib/Build
index 658bfa865b9c..4a444337b0e7 100644
--- a/tools/lkl/lib/Build
+++ b/tools/lkl/lib/Build
@@ -1,5 +1,6 @@
 CFLAGS_config.o += -I$(srctree)/tools/perf/pmu-events
 
+liblkl-y += fs.o
 liblkl-y += jmp_buf.o
 liblkl-y += utils.o
 liblkl-y += dbg.o
diff --git a/tools/lkl/lib/fs.c b/tools/lkl/lib/fs.c
new file mode 100644
index 000000000000..d51d928c14b6
--- /dev/null
+++ b/tools/lkl/lib/fs.c
@@ -0,0 +1,471 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdarg.h>
+#include <stdio.h>
+#include <string.h>
+#include <stdlib.h>
+#include <lkl_host.h>
+
+#define MAX_FSTYPE_LEN 50
+
+static struct lkl_disk *lkl_disks[16];
+
+int lkl_disk_add(struct lkl_disk *disk)
+{
+	int ret = -1;
+
+	switch (disk->backend) {
+	case BLK_BACKEND_UM:
+		ret = lkl_disk_um_add(disk, disk->dev);
+		break;
+	default:
+		break;
+	}
+
+	lkl_disks[ret] = disk;
+
+	return ret;
+}
+
+int lkl_disk_remove(struct lkl_disk disk)
+{
+	switch (disk.backend) {
+	case BLK_BACKEND_UM:
+	default:
+		break;
+	}
+
+	return 0;
+}
+
+int lkl_mount_fs(char *fstype)
+{
+	char dir[MAX_FSTYPE_LEN+2] = "/";
+	int flags = 0, ret = 0;
+
+	strncat(dir, fstype, MAX_FSTYPE_LEN);
+
+	/* Create with regular umask */
+	ret = lkl_sys_mkdir(dir, 0xff);
+	if (ret && ret != -LKL_EEXIST) {
+		lkl_perror("mount_fs mkdir", ret);
+		return ret;
+	}
+
+	/* We have no use for nonzero flags right now */
+	ret = lkl_sys_mount("none", dir, fstype, flags, NULL);
+	if (ret && ret != -LKL_EBUSY) {
+		lkl_sys_rmdir(dir);
+		return ret;
+	}
+
+	if (ret == -LKL_EBUSY)
+		return 1;
+	return 0;
+}
+
+static uint32_t new_encode_dev(unsigned int major, unsigned int minor)
+{
+	return (minor & 0xff) | (major << 8) | ((minor & ~0xff) << 12);
+}
+
+static int startswith(const char *str, const char *pre)
+{
+	return strncmp(pre, str, strlen(pre)) == 0;
+}
+
+static int get_node_with_prefix(const char *path, const char *prefix,
+				char *result, unsigned int result_len)
+{
+	struct lkl_dir *dir = NULL;
+	struct lkl_linux_dirent64 *dirent;
+	int ret;
+
+	dir = lkl_opendir(path, &ret);
+	if (!dir)
+		return ret;
+
+	ret = -LKL_ENOENT;
+
+	while ((dirent = lkl_readdir(dir))) {
+		if (startswith(dirent->d_name, prefix)) {
+			if (strlen(dirent->d_name) + 1 > result_len) {
+				ret = -LKL_ENOMEM;
+				break;
+			}
+			memcpy(result, dirent->d_name, strlen(dirent->d_name));
+			result[strlen(dirent->d_name)] = '\0';
+			ret = 0;
+			break;
+		}
+	}
+
+	lkl_closedir(dir);
+
+	return ret;
+}
+
+int lkl_encode_dev_from_sysfs(const char *sysfs_path, uint32_t *pdevid)
+{
+	int ret;
+	long fd;
+	int major, minor;
+	char buf[16] = { 0, };
+	char *bufptr;
+
+	fd = lkl_sys_open(sysfs_path, LKL_O_RDONLY, 0);
+	if (fd < 0)
+		return fd;
+
+	ret = lkl_sys_read(fd, buf, sizeof(buf));
+	if (ret < 0)
+		goto out_close;
+
+	if (ret == sizeof(buf)) {
+		ret = -LKL_ENOBUFS;
+		goto out_close;
+	}
+
+	bufptr = strchr(buf, ':');
+	if (bufptr == NULL) {
+		ret = -LKL_EINVAL;
+		goto out_close;
+	}
+	bufptr[0] = '\0';
+	bufptr++;
+
+	major = atoi(buf);
+	minor = atoi(bufptr);
+
+	*pdevid = new_encode_dev(major, minor);
+	ret = 0;
+
+out_close:
+	lkl_sys_close(fd);
+
+	return ret;
+}
+
+#define SYSFS_DEV_UMBLK_CMDLINE_PATH \
+	"/sysfs/devices/platform/uml-blkdev.%d"
+
+struct abuf {
+	char *mem, *ptr;
+	unsigned int len;
+};
+
+static int snprintf_append(struct abuf *buf, const char *fmt, ...)
+{
+	int ret;
+	va_list args;
+
+	if (!buf->ptr)
+		buf->ptr = buf->mem;
+
+	va_start(args, fmt);
+	ret = vsnprintf(buf->ptr, buf->len - (buf->ptr - buf->mem), fmt, args);
+	va_end(args);
+
+	if (ret < 0 || (ret >= (int)(buf->len - (buf->ptr - buf->mem))))
+		return -LKL_ENOMEM;
+
+	buf->ptr += ret;
+
+	return 0;
+}
+
+static int __lkl_get_blkdev(int disk_id, unsigned int part, uint32_t *pdevid,
+			    const char *sysfs_path_fmt, const char *drv_prefix,
+			    const char *disk_prefix)
+{
+	char sysfs_path[LKL_PATH_MAX];
+	char drv_name[LKL_PATH_MAX];
+	char disk_name[LKL_PATH_MAX];
+	struct abuf sysfs_path_buf = {
+		.mem = sysfs_path,
+		.len = sizeof(sysfs_path),
+	};
+	int ret;
+
+	if (disk_id < 0)
+		return -LKL_EINVAL;
+
+	ret = lkl_mount_fs("sysfs");
+	if (ret < 0)
+		return ret;
+
+	ret = snprintf_append(&sysfs_path_buf, sysfs_path_fmt, disk_id);
+	if (ret)
+		return ret;
+
+	ret = get_node_with_prefix(sysfs_path, drv_prefix, drv_name,
+				   sizeof(drv_name));
+	if (ret)
+		return ret;
+
+	ret = snprintf_append(&sysfs_path_buf, "/%s/block", drv_name);
+	if (ret)
+		return ret;
+
+	ret = get_node_with_prefix(sysfs_path, disk_prefix, disk_name,
+				   sizeof(disk_name));
+	if (ret)
+		return ret;
+
+	if (!part)
+		ret = snprintf_append(&sysfs_path_buf, "/%s/dev", disk_name);
+	else
+		ret = snprintf_append(&sysfs_path_buf, "/%s/%s%d/dev",
+				      disk_name, disk_name, part);
+	if (ret)
+		return ret;
+
+	return lkl_encode_dev_from_sysfs(sysfs_path, pdevid);
+}
+
+int lkl_get_blkdev(int disk_id, unsigned int part, uint32_t *pdevid)
+{
+	char *fmt;
+	struct lkl_disk *disk = lkl_disks[disk_id];
+
+	switch (disk->backend) {
+	case BLK_BACKEND_UM:
+		fmt = SYSFS_DEV_UMBLK_CMDLINE_PATH;
+		return __lkl_get_blkdev(disk_id, part, pdevid, fmt,
+					"", "ubd");
+	default:
+		break;
+	}
+
+	return -1;
+}
+
+long lkl_mount_dev(unsigned int disk_id, unsigned int part,
+		   const char *fs_type, int flags,
+		   const char *data, char *mnt_str, unsigned int mnt_str_len)
+{
+	char dev_str[] = { "/dev/xxxxxxxx" };
+	unsigned int dev;
+	int err;
+	char _data[4096]; /* FIXME: PAGE_SIZE is not exported by LKL */
+
+	if (mnt_str_len < sizeof(dev_str))
+		return -LKL_ENOMEM;
+
+	err = lkl_get_blkdev(disk_id, part, &dev);
+	if (err < 0)
+		return err;
+
+	snprintf(dev_str, sizeof(dev_str), "/dev/%08x", dev);
+	snprintf(mnt_str, mnt_str_len, "/mnt/%08x", dev);
+
+	err = lkl_sys_access("/dev", LKL_S_IRWXO);
+	if (err < 0) {
+		if (err == -LKL_ENOENT)
+			err = lkl_sys_mkdir("/dev", 0700);
+		if (err < 0)
+			return err;
+	}
+
+	err = lkl_sys_mknod(dev_str, LKL_S_IFBLK | 0600, dev);
+	if (err < 0)
+		return err;
+
+	err = lkl_sys_access("/mnt", LKL_S_IRWXO);
+	if (err < 0) {
+		if (err == -LKL_ENOENT)
+			err = lkl_sys_mkdir("/mnt", 0700);
+		if (err < 0)
+			return err;
+	}
+
+	err = lkl_sys_mkdir(mnt_str, 0700);
+	if (err < 0) {
+		lkl_sys_unlink(dev_str);
+		return err;
+	}
+
+	/* kernel always copies a full page */
+	if (data) {
+		strncpy(_data, data, sizeof(_data));
+		_data[sizeof(_data) - 1] = 0;
+	} else {
+		_data[0] = 0;
+	}
+
+	err = lkl_sys_mount(dev_str, mnt_str, (char *)fs_type, flags, _data);
+	if (err < 0) {
+		lkl_sys_unlink(dev_str);
+		lkl_sys_rmdir(mnt_str);
+		return err;
+	}
+
+	return 0;
+}
+
+long lkl_umount_timeout(char *path, int flags, long timeout_ms)
+{
+	long incr = 10000000; /* 10 ms */
+	struct lkl_timespec ts = {
+		.tv_sec = 0,
+		.tv_nsec = incr,
+	};
+	long err;
+
+	do {
+		err = lkl_sys_umount(path, flags);
+		if (err == -LKL_EBUSY) {
+			lkl_sys_nanosleep((struct __lkl__kernel_timespec *)&ts,
+					  NULL);
+			timeout_ms -= incr / 1000000;
+		}
+	} while (err == -LKL_EBUSY && timeout_ms > 0);
+
+	return err;
+}
+
+long lkl_umount_dev(unsigned int disk_id, unsigned int part, int flags,
+		    long timeout_ms)
+{
+	char dev_str[] = { "/dev/xxxxxxxx" };
+	char mnt_str[] = { "/mnt/xxxxxxxx" };
+	unsigned int dev;
+	int err;
+
+	err = lkl_get_blkdev(disk_id, part, &dev);
+	if (err < 0)
+		return err;
+
+	snprintf(dev_str, sizeof(dev_str), "/dev/%08x", dev);
+	snprintf(mnt_str, sizeof(mnt_str), "/mnt/%08x", dev);
+
+	err = lkl_umount_timeout(mnt_str, flags, timeout_ms);
+	if (err)
+		return err;
+
+	err = lkl_sys_unlink(dev_str);
+	if (err)
+		return err;
+
+	return lkl_sys_rmdir(mnt_str);
+}
+
+struct lkl_dir {
+	int fd;
+	char buf[1024];
+	char *pos;
+	int len;
+};
+
+static struct lkl_dir *lkl_dir_alloc(int *err)
+{
+	struct lkl_dir *dir = lkl_host_ops.mem_alloc(sizeof(struct lkl_dir));
+
+	if (!dir) {
+		*err = -LKL_ENOMEM;
+		return NULL;
+	}
+
+	dir->len = 0;
+	dir->pos = NULL;
+
+	return dir;
+}
+
+struct lkl_dir *lkl_opendir(const char *path, int *err)
+{
+	struct lkl_dir *dir = lkl_dir_alloc(err);
+
+	if (!dir) {
+		*err = -LKL_ENOMEM;
+		return NULL;
+	}
+
+	dir->fd = lkl_sys_open(path, LKL_O_RDONLY | LKL_O_DIRECTORY, 0);
+	if (dir->fd < 0) {
+		*err = dir->fd;
+		lkl_host_ops.mem_free(dir);
+		return NULL;
+	}
+
+	*err = 0;
+
+	return dir;
+}
+
+struct lkl_dir *lkl_fdopendir(int fd, int *err)
+{
+	struct lkl_dir *dir = lkl_dir_alloc(err);
+
+	if (!dir)
+		return NULL;
+
+	dir->fd = fd;
+
+	return dir;
+}
+
+void lkl_rewinddir(struct lkl_dir *dir)
+{
+	lkl_sys_lseek(dir->fd, 0, LKL_SEEK_SET);
+	dir->len = 0;
+	dir->pos = NULL;
+}
+
+int lkl_closedir(struct lkl_dir *dir)
+{
+	int ret;
+
+	ret = lkl_sys_close(dir->fd);
+	lkl_host_ops.mem_free(dir);
+
+	return ret;
+}
+
+struct lkl_linux_dirent64 *lkl_readdir(struct lkl_dir *dir)
+{
+	struct lkl_linux_dirent64 *de;
+
+	if (dir->len < 0)
+		return NULL;
+
+	if (!dir->pos || dir->pos - dir->buf >= dir->len)
+		goto read_buf;
+
+return_de:
+	de = (struct lkl_linux_dirent64 *)dir->pos;
+	dir->pos += de->d_reclen;
+
+	return de;
+
+read_buf:
+	dir->pos = NULL;
+	de = (struct lkl_linux_dirent64 *)dir->buf;
+	dir->len = lkl_sys_getdents64(dir->fd, de, sizeof(dir->buf));
+	if (dir->len <= 0)
+		return NULL;
+
+	dir->pos = dir->buf;
+	goto return_de;
+}
+
+int lkl_errdir(struct lkl_dir *dir)
+{
+	if (dir->len >= 0)
+		return 0;
+
+	return dir->len;
+}
+
+int lkl_dirfd(struct lkl_dir *dir)
+{
+	return dir->fd;
+}
+
+int lkl_set_fd_limit(unsigned int fd_limit)
+{
+	struct lkl_rlimit rlim = {
+		.rlim_cur = fd_limit,
+		.rlim_max = fd_limit,
+	};
+	return lkl_sys_setrlimit(LKL_RLIMIT_NOFILE, &rlim);
+}
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v3 16/26] lkl tools: host lib: networking helpers
  2020-02-05  7:30 [RFC v3 00/26] Unifying LKL into UML Hajime Tazaki
                   ` (14 preceding siblings ...)
  2020-02-05  7:30   ` Hajime Tazaki
@ 2020-02-05  7:30 ` Hajime Tazaki
  2020-02-05  7:30   ` Hajime Tazaki
                   ` (10 subsequent siblings)
  26 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05  7:30 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Hajime Tazaki

This commit adds LKL application APIs to control network related
resources of kernel via LKL syscall, ioctl, and via netlink messages.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 tools/lkl/include/lkl.h      | 241 +++++++++++
 tools/lkl/include/lkl_host.h |  85 ++++
 tools/lkl/lib/Build          |   1 +
 tools/lkl/lib/net.c          | 818 +++++++++++++++++++++++++++++++++++
 4 files changed, 1145 insertions(+)
 create mode 100644 tools/lkl/lib/net.c

diff --git a/tools/lkl/include/lkl.h b/tools/lkl/include/lkl.h
index cdae92300320..1f4291ad9455 100644
--- a/tools/lkl/include/lkl.h
+++ b/tools/lkl/include/lkl.h
@@ -516,6 +516,247 @@ int lkl_dirfd(struct lkl_dir *dir);
  */
 int lkl_mount_fs(char *fstype);
 
+/**
+ * lkl_if_up - activate network interface
+ *
+ * @ifindex - the ifindex of the interface
+ * @returns - return 0 if no error: otherwise negative value returns
+ */
+int lkl_if_up(int ifindex);
+
+/**
+ * lkl_if_down - deactivate network interface
+ *
+ * @ifindex - the ifindex of the interface
+ * @returns - return 0 if no error: otherwise negative value returns
+ */
+int lkl_if_down(int ifindex);
+
+/**
+ * lkl_if_set_mtu - set MTU on interface
+ *
+ * @ifindex - the ifindex of the interface
+ * @mtu - the requested MTU size
+ * @returns - return 0 if no error: otherwise negative value returns
+ */
+int lkl_if_set_mtu(int ifindex, int mtu);
+
+/**
+ * lkl_if_set_ipv4 - set IPv4 address on interface
+ *
+ * @ifindex - the ifindex of the interface
+ * @addr - 4-byte IP address (i.e., struct in_addr)
+ * @netmask_len - prefix length of the @addr
+ * @returns - return 0 if no error: otherwise negative value returns
+ */
+int lkl_if_set_ipv4(int ifindex, unsigned int addr, unsigned int netmask_len);
+
+/**
+ * lkl_set_ipv4_gateway - add an IPv4 default route
+ *
+ * @addr - 4-byte IP address of the gateway (i.e., struct in_addr)
+ * @returns - return 0 if no error: otherwise negative value returns
+ */
+int lkl_set_ipv4_gateway(unsigned int addr);
+
+/**
+ * lkl_if_set_ipv4_gateway - add an IPv4 default route in rule table
+ *
+ * @ifindex - the ifindex of the interface, used for tableid calculation
+ * @addr - 4-byte IP address of the interface
+ * @netmask_len - prefix length of the @addr
+ * @gw_addr - 4-byte IP address of the gateway
+ * @returns - return 0 if no error: otherwise negative value returns
+ */
+int lkl_if_set_ipv4_gateway(int ifindex, unsigned int addr,
+		unsigned int netmask_len, unsigned int gw_addr);
+
+/**
+ * lkl_if_set_ipv6 - set IPv6 address on interface
+ * must be called after interface is up.
+ *
+ * @ifindex - the ifindex of the interface
+ * @addr - 16-byte IPv6 address (i.e., struct in6_addr)
+ * @netprefix_len - prefix length of the @addr
+ * @returns - return 0 if no error: otherwise negative value returns
+ */
+int lkl_if_set_ipv6(int ifindex, void *addr, unsigned int netprefix_len);
+
+/**
+ * lkl_set_ipv6_gateway - add an IPv6 default route
+ *
+ * @addr - 16-byte IPv6 address of the gateway (i.e., struct in6_addr)
+ * @returns - return 0 if no error: otherwise negative value returns
+ */
+int lkl_set_ipv6_gateway(void *addr);
+
+/**
+ * lkl_if_set_ipv6_gateway - add an IPv6 default route in rule table
+ *
+ * @ifindex - the ifindex of the interface, used for tableid calculation
+ * @addr - 16-byte IP address of the interface
+ * @netmask_len - prefix length of the @addr
+ * @gw_addr - 16-byte IP address of the gateway (i.e., struct in_addr)
+ * @returns - return 0 if no error: otherwise negative value returns
+ */
+int lkl_if_set_ipv6_gateway(int ifindex, void *addr,
+		unsigned int netmask_len, void *gw_addr);
+
+/**
+ * lkl_ifname_to_ifindex - obtain ifindex of an interface by name
+ *
+ * @name - string of an interface
+ * @returns - return an integer of ifindex if no error
+ */
+int lkl_ifname_to_ifindex(const char *name);
+
+/**
+ * lkl_netdev_get_ifindex - retrieve the interface index for a given network
+ * device id
+ *
+ * @id - the network device id
+ * @returns the interface index or a stricly negative value in case of error
+ */
+int lkl_netdev_get_ifindex(int id);
+
+/**
+ * lkl_add_neighbor - add a permanent arp entry
+ * @ifindex - the ifindex of the interface
+ * @af - address family of the ip address. Must be LKL_AF_INET or LKL_AF_INET6
+ * @ip - ip address of the entry in network byte order
+ * @mac - mac address of the entry
+ */
+int lkl_add_neighbor(int ifindex, int af, void *addr, void *mac);
+
+/**
+ * lkl_if_add_ip - add an ip address
+ * @ifindex - the ifindex of the interface
+ * @af - address family of the ip address. Must be LKL_AF_INET or LKL_AF_INET6
+ * @addr - ip address of the entry in network byte order
+ * @netprefix_len - prefix length of the @addr
+ */
+int lkl_if_add_ip(int ifindex, int af, void *addr, unsigned int netprefix_len);
+
+/**
+ * lkl_if_del_ip - add an ip address
+ * @ifindex - the ifindex of the interface
+ * @af - address family of the ip address. Must be LKL_AF_INET or LKL_AF_INET6
+ * @addr - ip address of the entry in network byte order
+ * @netprefix_len - prefix length of the @addr
+ */
+int lkl_if_del_ip(int ifindex, int af, void *addr, unsigned int netprefix_len);
+
+/**
+ * lkl_add_gateway - add a gateway
+ * @af - address family of the ip address. Must be LKL_AF_INET or LKL_AF_INET6
+ * @gwaddr - 4-byte IP address of the gateway (i.e., struct in_addr)
+ */
+int lkl_add_gateway(int af, void *gwaddr);
+
+/**
+ * XXX Should I use OIF selector?
+ * temporary table idx = ifindex * 2 + 0 <- ipv4
+ * temporary table idx = ifindex * 2 + 1 <- ipv6
+ */
+/**
+ * lkl_if_add_rule_from_addr - create an ip rule table with "from" selector
+ * @ifindex - the ifindex of the interface, used for table id calculation
+ * @af - address family of the ip address. Must be LKL_AF_INET or LKL_AF_INET6
+ * @saddr - network byte order ip address, "from" selector address of this rule
+ */
+int lkl_if_add_rule_from_saddr(int ifindex, int af, void *saddr);
+
+/**
+ * lkl_if_add_gateway - add gateway to rule table
+ * @ifindex - the ifindex of the interface, used for table id calculation
+ * @af - address family of the ip address. Must be LKL_AF_INET or LKL_AF_INET6
+ * @gwaddr - 4-byte IP address of the gateway (i.e., struct in_addr)
+ */
+int lkl_if_add_gateway(int ifindex, int af, void *gwaddr);
+
+/**
+ * lkl_if_add_linklocal - add linklocal route to rule table
+ * @ifindex - the ifindex of the interface, used for table id calculation
+ * @af - address family of the ip address. Must be LKL_AF_INET or LKL_AF_INET6
+ * @addr - ip address of the entry in network byte order
+ * @netprefix_len - prefix length of the @addr
+ */
+int lkl_if_add_linklocal(int ifindex, int af,  void *addr, int netprefix_len);
+
+/**
+ * lkl_if_wait_ipv6_dad - wait for DAD to be done for a ipv6 address
+ * must be called after interface is up
+ *
+ * @ifindex - the ifindex of the interface
+ * @addr - ip address of the entry in network byte order
+ */
+int lkl_if_wait_ipv6_dad(int ifindex, void *addr);
+
+/**
+ * lkl_set_fd_limit - set the maximum number of file descriptors allowed
+ * @fd_limit - fd max limit
+ */
+int lkl_set_fd_limit(unsigned int fd_limit);
+
+/**
+ * lkl_qdisc_add - set qdisc rule onto an interface
+ *
+ * @ifindex - the ifindex of the interface
+ * @root - the name of root class (e.g., "root");
+ * @type - the type of qdisc (e.g., "fq")
+ */
+int lkl_qdisc_add(int ifindex, const char *root, const char *type);
+
+/**
+ * lkl_qdisc_parse_add - Add a qdisc entry for an interface with strings
+ *
+ * @ifindex - the ifindex of the interface
+ * @entries - strings of qdisc configurations in the form of
+ *            "root|type;root|type;..."
+ */
+void lkl_qdisc_parse_add(int ifindex, const char *entries);
+
+/**
+ * lkl_netdev - host network device handle, defined in lkl_host.h.
+ */
+struct lkl_netdev;
+
+/**
+ * lkl_netdev_args - arguments to lkl_netdev_add
+ * @mac - optional MAC address for the device
+ * @offload - offload bits for the device
+ */
+struct lkl_netdev_args {
+	void *mac;
+	unsigned int offload;
+};
+
+/*
+ * lkl_register_dbg_handler- register a signal handler that loads a debug lib.
+ *
+ * The signal handler is triggered by Ctrl-Z. It creates a new pthread which
+ * call dbg_entrance().
+ *
+ * If you run the program from shell script, make sure you ignore SIGTSTP by
+ * "trap '' TSTP" in the shell script.
+ */
+void lkl_register_dbg_handler(void);
+
+/**
+ * lkl_sysctl - write a sysctl value
+ *
+ * @path - the path to an sysctl entry (e.g., "net.ipv4.tcp_wmem");
+ * @value - the value of the sysctl (e.g., "4096 87380 2147483647")
+ */
+int lkl_sysctl(const char *path, const char *value);
+
+/**
+ * lkl_sysctl_parse_write - Configure sysctl parameters with strings
+ *
+ * @sysctls - Configure sysctl parameters as the form of "key=value;..."
+ */
+void lkl_sysctl_parse_write(const char *sysctls);
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/tools/lkl/include/lkl_host.h b/tools/lkl/include/lkl_host.h
index 85e80eb4ad0d..4e6d6e031498 100644
--- a/tools/lkl/include/lkl_host.h
+++ b/tools/lkl/include/lkl_host.h
@@ -18,6 +18,91 @@ extern struct lkl_host_operations lkl_host_ops;
  */
 int lkl_printf(const char *fmt, ...);
 
+#ifdef LKL_HOST_CONFIG_POSIX
+#include <sys/uio.h>
+#else
+struct iovec {
+	void *iov_base;
+	size_t iov_len;
+};
+#endif
+
+struct lkl_netdev {
+	struct lkl_dev_net_ops *ops;
+	int id;
+	uint8_t has_vnet_hdr: 1;
+};
+
+/**
+ * struct lkl_dev_net_ops - network device host operations
+ */
+struct lkl_dev_net_ops {
+	/**
+	 * @tx: writes a L2 packet into the net device
+	 *
+	 * The data buffer can only hold 0 or 1 complete packets.
+	 *
+	 * @nd - pointer to the network device;
+	 * @iov - pointer to the buffer vector;
+	 * @cnt - # of vectors in iov.
+	 *
+	 * @returns number of bytes transmitted
+	 */
+	int (*tx)(struct lkl_netdev *nd, struct iovec *iov, int cnt);
+
+	/**
+	 * @rx: reads a packet from the net device.
+	 *
+	 * It must only read one complete packet if present.
+	 *
+	 * If the buffer is too small for the packet, the implementation may
+	 * decide to drop it or trim it.
+	 *
+	 * @nd - pointer to the network device
+	 * @iov - pointer to the buffer vector to store the packet
+	 * @cnt - # of vectors in iov.
+	 *
+	 * @returns number of bytes read for success or < 0 if error
+	 */
+	int (*rx)(struct lkl_netdev *nd, struct iovec *iov, int cnt);
+
+#define LKL_DEV_NET_POLL_RX		1
+#define LKL_DEV_NET_POLL_TX		2
+#define LKL_DEV_NET_POLL_HUP		4
+
+	/**
+	 * @poll: polls a net device
+	 *
+	 * Supports the following events: LKL_DEV_NET_POLL_RX
+	 * (readable), LKL_DEV_NET_POLL_TX (writable) or
+	 * LKL_DEV_NET_POLL_HUP (the close operations has been issued
+	 * and we need to clean up). Blocks until one event is
+	 * available.
+	 *
+	 * @nd - pointer to the network device
+	 *
+	 * @returns - LKL_DEV_NET_POLL_RX, LKL_DEV_NET_POLL_TX,
+	 * LKL_DEV_NET_POLL_HUP or a negative value for errors
+	 */
+	int (*poll)(struct lkl_netdev *nd);
+
+	/**
+	 * @poll_hup: make poll wakeup and return LKL_DEV_NET_POLL_HUP
+	 *
+	 * @nd - pointer to the network device
+	 */
+	void (*poll_hup)(struct lkl_netdev *nd);
+
+	/**
+	 * @free: frees a network device
+	 *
+	 * Implementation must release its resources and free the network device
+	 * structure.
+	 *
+	 * @nd - pointer to the network device
+	 */
+	void (*free)(struct lkl_netdev *nd);
+};
 
 #ifdef __cplusplus
 }
diff --git a/tools/lkl/lib/Build b/tools/lkl/lib/Build
index 4a444337b0e7..dba530424e4a 100644
--- a/tools/lkl/lib/Build
+++ b/tools/lkl/lib/Build
@@ -1,6 +1,7 @@
 CFLAGS_config.o += -I$(srctree)/tools/perf/pmu-events
 
 liblkl-y += fs.o
+liblkl-y += net.o
 liblkl-y += jmp_buf.o
 liblkl-y += utils.o
 liblkl-y += dbg.o
diff --git a/tools/lkl/lib/net.c b/tools/lkl/lib/net.c
new file mode 100644
index 000000000000..316965ffd21e
--- /dev/null
+++ b/tools/lkl/lib/net.c
@@ -0,0 +1,818 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <string.h>
+#include <stdio.h>
+#include "endian.h"
+#include <lkl_host.h>
+
+#ifdef __MINGW32__
+#include <ws2tcpip.h>
+
+int lkl_inet_pton(int af, const char *src, void *dst)
+{
+	struct addrinfo hint, *res = NULL;
+	int err;
+
+	memset(&hint, 0, sizeof(struct addrinfo));
+
+	hint.ai_family = af;
+	hint.ai_flags = AI_NUMERICHOST;
+
+	err = getaddrinfo(src, NULL, &hint, &res);
+	if (err)
+		return 0;
+
+	switch (af) {
+	case AF_INET:
+		*(struct in_addr *)dst =
+			((struct sockaddr_in *)&res->ai_addr)->sin_addr;
+		break;
+	case AF_INET6:
+		*(struct in6_addr *)dst =
+			((struct sockaddr_in6 *)&res->ai_addr)->sin6_addr;
+		break;
+	default:
+		freeaddrinfo(res);
+		return 0;
+	}
+
+	freeaddrinfo(res);
+	return 1;
+}
+#endif
+
+static inline void set_sockaddr(struct lkl_sockaddr_in *sin, unsigned int addr,
+				unsigned short port)
+{
+	sin->sin_family = LKL_AF_INET;
+	sin->sin_addr.lkl_s_addr = addr;
+	sin->sin_port = port;
+}
+
+static inline int ifindex_to_name(int sock, struct lkl_ifreq *ifr, int ifindex)
+{
+	ifr->lkl_ifr_ifindex = ifindex;
+	return lkl_sys_ioctl(sock, LKL_SIOCGIFNAME, (long)ifr);
+}
+
+int lkl_ifname_to_ifindex(const char *name)
+{
+	struct lkl_ifreq ifr;
+	int fd, ret;
+
+	fd = lkl_sys_socket(LKL_AF_INET, LKL_SOCK_DGRAM, 0);
+	if (fd < 0)
+		return fd;
+
+	strcpy(ifr.lkl_ifr_name, name);
+
+	ret = lkl_sys_ioctl(fd, LKL_SIOCGIFINDEX, (long)&ifr);
+	if (ret < 0)
+		return ret;
+
+	return ifr.lkl_ifr_ifindex;
+}
+
+int lkl_if_up(int ifindex)
+{
+	struct lkl_ifreq ifr;
+	int err, sock = lkl_sys_socket(LKL_AF_INET, LKL_SOCK_DGRAM, 0);
+
+	if (sock < 0)
+		return sock;
+	err = ifindex_to_name(sock, &ifr, ifindex);
+	if (err < 0)
+		return err;
+
+	err = lkl_sys_ioctl(sock, LKL_SIOCGIFFLAGS, (long)&ifr);
+	if (!err) {
+		ifr.lkl_ifr_flags |= LKL_IFF_UP;
+		err = lkl_sys_ioctl(sock, LKL_SIOCSIFFLAGS, (long)&ifr);
+	}
+
+	lkl_sys_close(sock);
+
+	return err;
+}
+
+int lkl_if_down(int ifindex)
+{
+	struct lkl_ifreq ifr;
+	int err, sock;
+
+	sock = lkl_sys_socket(LKL_AF_INET, LKL_SOCK_DGRAM, 0);
+	if (sock < 0)
+		return sock;
+
+	err = ifindex_to_name(sock, &ifr, ifindex);
+	if (err < 0)
+		return err;
+
+	err = lkl_sys_ioctl(sock, LKL_SIOCGIFFLAGS, (long)&ifr);
+	if (!err) {
+		ifr.lkl_ifr_flags &= ~LKL_IFF_UP;
+		err = lkl_sys_ioctl(sock, LKL_SIOCSIFFLAGS, (long)&ifr);
+	}
+
+	lkl_sys_close(sock);
+
+	return err;
+}
+
+int lkl_if_set_mtu(int ifindex, int mtu)
+{
+	struct lkl_ifreq ifr;
+	int err, sock;
+
+	sock = lkl_sys_socket(LKL_AF_INET, LKL_SOCK_DGRAM, 0);
+	if (sock < 0)
+		return sock;
+
+	err = ifindex_to_name(sock, &ifr, ifindex);
+	if (err < 0)
+		return err;
+
+	ifr.lkl_ifr_mtu = mtu;
+
+	err = lkl_sys_ioctl(sock, LKL_SIOCSIFMTU, (long)&ifr);
+
+	lkl_sys_close(sock);
+
+	return err;
+}
+
+int lkl_if_set_ipv4(int ifindex, unsigned int addr, unsigned int netmask_len)
+{
+	return lkl_if_add_ip(ifindex, LKL_AF_INET, &addr, netmask_len);
+}
+
+int lkl_if_set_ipv4_gateway(int ifindex, unsigned int src_addr,
+		unsigned int src_masklen, unsigned int via_addr)
+{
+	int err;
+
+	err = lkl_if_add_rule_from_saddr(ifindex, LKL_AF_INET, &src_addr);
+	if (err)
+		return err;
+	err = lkl_if_add_linklocal(ifindex, LKL_AF_INET,
+					&src_addr, src_masklen);
+	if (err)
+		return err;
+	return lkl_if_add_gateway(ifindex, LKL_AF_INET, &via_addr);
+}
+
+int lkl_set_ipv4_gateway(unsigned int addr)
+{
+	return lkl_add_gateway(LKL_AF_INET, &addr);
+}
+
+int lkl_netdev_get_ifindex(int id)
+{
+	struct lkl_ifreq ifr;
+	int sock, ret;
+
+	sock = lkl_sys_socket(LKL_AF_INET, LKL_SOCK_DGRAM, 0);
+	if (sock < 0)
+		return sock;
+
+	snprintf(ifr.lkl_ifr_name, sizeof(ifr.lkl_ifr_name), "eth%d", id);
+	ret = lkl_sys_ioctl(sock, LKL_SIOCGIFINDEX, (long)&ifr);
+	lkl_sys_close(sock);
+
+	return ret < 0 ? ret : ifr.lkl_ifr_ifindex;
+}
+
+static int netlink_sock(unsigned int groups)
+{
+	struct lkl_sockaddr_nl la;
+	int fd, err;
+
+	fd = lkl_sys_socket(LKL_AF_NETLINK, LKL_SOCK_DGRAM, LKL_NETLINK_ROUTE);
+	if (fd < 0)
+		return fd;
+
+	memset(&la, 0, sizeof(la));
+	la.nl_family = LKL_AF_NETLINK;
+	la.nl_groups = groups;
+	err = lkl_sys_bind(fd, (struct lkl_sockaddr *)&la, sizeof(la));
+	if (err < 0)
+		return err;
+
+	return fd;
+}
+
+static int parse_rtattr(struct lkl_rtattr *tb[], int max,
+			struct lkl_rtattr *rta, int len)
+{
+	unsigned short type;
+
+	memset(tb, 0, sizeof(struct lkl_rtattr *) * (max + 1));
+	while (LKL_RTA_OK(rta, len)) {
+		type = rta->rta_type;
+		if ((type <= max) && (!tb[type]))
+			tb[type] = rta;
+		rta = LKL_RTA_NEXT(rta, len);
+	}
+	if (len)
+		lkl_printf("!!!Deficit %d, rta_len=%d\n", len,
+			rta->rta_len);
+	return 0;
+}
+
+struct addr_filter {
+	unsigned int ifindex;
+	void *addr;
+};
+
+static unsigned int get_ifa_flags(struct lkl_ifaddrmsg *ifa,
+				  struct lkl_rtattr *ifa_flags_attr)
+{
+	return ifa_flags_attr ? *(unsigned int *)LKL_RTA_DATA(ifa_flags_attr) :
+				ifa->ifa_flags;
+}
+
+/* returns:
+ * 0 - dad succeed.
+ * -1 - dad failed or other error.
+ * 1 - should wait for new msg.
+ */
+static int check_ipv6_dad(struct lkl_sockaddr_nl *nladdr,
+			  struct lkl_nlmsghdr *n, void *arg)
+{
+	struct addr_filter *filter = arg;
+	struct lkl_ifaddrmsg *ifa = LKL_NLMSG_DATA(n);
+	struct lkl_rtattr *rta_tb[LKL_IFA_MAX+1];
+	unsigned int ifa_flags;
+	int len = n->nlmsg_len;
+
+	if (n->nlmsg_type != LKL_RTM_NEWADDR)
+		return 1;
+
+	len -= LKL_NLMSG_LENGTH(sizeof(*ifa));
+	if (len < 0) {
+		lkl_printf("BUG: wrong nlmsg len %d\n", len);
+		return -1;
+	}
+
+	parse_rtattr(rta_tb, LKL_IFA_MAX, LKL_IFA_RTA(ifa),
+		     n->nlmsg_len - LKL_NLMSG_LENGTH(sizeof(*ifa)));
+
+	ifa_flags = get_ifa_flags(ifa, rta_tb[LKL_IFA_FLAGS]);
+
+	if (ifa->ifa_index != filter->ifindex)
+		return 1;
+	if (ifa->ifa_family != LKL_AF_INET6)
+		return 1;
+
+	if (!rta_tb[LKL_IFA_LOCAL])
+		rta_tb[LKL_IFA_LOCAL] = rta_tb[LKL_IFA_ADDRESS];
+
+	if (!rta_tb[LKL_IFA_LOCAL] ||
+	    (filter->addr && memcmp(LKL_RTA_DATA(rta_tb[LKL_IFA_LOCAL]),
+				    filter->addr, 16))) {
+		return 1;
+	}
+	if (ifa_flags & LKL_IFA_F_DADFAILED) {
+		lkl_printf("IPV6 DAD failed.\n");
+		return -1;
+	}
+	if (!(ifa_flags & LKL_IFA_F_TENTATIVE))
+		return 0;
+	return 1;
+}
+
+/* Copied from iproute2/lib/ */
+static int rtnl_listen(int fd, int (*handler)(struct lkl_sockaddr_nl *nladdr,
+					      struct lkl_nlmsghdr *, void *),
+		       void *arg)
+{
+	int status;
+	struct lkl_nlmsghdr *h;
+	struct lkl_sockaddr_nl nladdr = { .nl_family = LKL_AF_NETLINK };
+	struct lkl_iovec iov;
+	struct lkl_user_msghdr msg = {
+		.msg_name = &nladdr,
+		.msg_namelen = sizeof(nladdr),
+		.msg_iov = &iov,
+		.msg_iovlen = 1,
+	};
+	char   buf[16384];
+
+	iov.iov_base = buf;
+	while (1) {
+		iov.iov_len = sizeof(buf);
+		status = lkl_sys_recvmsg(fd, &msg, 0);
+
+		if (status < 0) {
+			if (status == -LKL_EINTR || status == -LKL_EAGAIN)
+				continue;
+			lkl_printf("netlink receive error %s (%d)\n",
+				lkl_strerror(status), status);
+			if (status == -LKL_ENOBUFS)
+				continue;
+			return status;
+		}
+		if (status == 0) {
+			lkl_printf("EOF on netlink\n");
+			return -1;
+		}
+		if (msg.msg_namelen != sizeof(nladdr)) {
+			lkl_printf("Sender address length == %d\n",
+				msg.msg_namelen);
+			return -1;
+		}
+
+		for (h = (struct lkl_nlmsghdr *)buf;
+		     (unsigned int)status >= sizeof(*h);) {
+			int err;
+			int len = h->nlmsg_len;
+			int l = len - sizeof(*h);
+
+			if (l < 0 || len > status) {
+				if (msg.msg_flags & LKL_MSG_TRUNC) {
+					lkl_printf("Truncated message\n");
+					return -1;
+				}
+				lkl_printf("!!!malformed message: len=%d\n",
+					len);
+				return -1;
+			}
+
+			err = handler(&nladdr, h, arg);
+			if (err <= 0)
+				return err;
+
+			status -= LKL_NLMSG_ALIGN(len);
+			h = (struct lkl_nlmsghdr *)((char *)h +
+						    LKL_NLMSG_ALIGN(len));
+		}
+		if (msg.msg_flags & LKL_MSG_TRUNC) {
+			lkl_printf("Message truncated\n");
+			continue;
+		}
+		if (status) {
+			lkl_printf("!!!Remnant of size %d\n", status);
+			return -1;
+		}
+	}
+}
+
+int lkl_if_wait_ipv6_dad(int ifindex, void *addr)
+{
+	struct addr_filter filter = {.ifindex = ifindex, .addr = addr};
+	int fd, ret;
+	struct {
+		struct lkl_nlmsghdr		nlmsg_info;
+		struct lkl_ifaddrmsg	ifaddrmsg_info;
+	} req;
+
+	fd = netlink_sock(1 << (LKL_RTNLGRP_IPV6_IFADDR - 1));
+	if (fd < 0)
+		return fd;
+
+	memset(&req, 0, sizeof(req));
+	req.nlmsg_info.nlmsg_len =
+			LKL_NLMSG_LENGTH(sizeof(struct lkl_ifaddrmsg));
+	req.nlmsg_info.nlmsg_flags = LKL_NLM_F_REQUEST | LKL_NLM_F_DUMP;
+	req.nlmsg_info.nlmsg_type = LKL_RTM_GETADDR;
+	req.ifaddrmsg_info.ifa_family = LKL_AF_INET6;
+	req.ifaddrmsg_info.ifa_index = ifindex;
+	ret = lkl_sys_send(fd, &req, req.nlmsg_info.nlmsg_len, 0);
+	if (ret < 0) {
+		lkl_perror("lkl_sys_send", ret);
+		return ret;
+	}
+	ret = rtnl_listen(fd, check_ipv6_dad, (void *)&filter);
+	lkl_sys_close(fd);
+	return ret;
+}
+
+int lkl_if_set_ipv6(int ifindex, void *addr, unsigned int netprefix_len)
+{
+	int err = lkl_if_add_ip(ifindex, LKL_AF_INET6, addr, netprefix_len);
+
+	if (err)
+		return err;
+	return lkl_if_wait_ipv6_dad(ifindex, addr);
+}
+
+int lkl_if_set_ipv6_gateway(int ifindex, void *src_addr,
+		unsigned int src_masklen, void *via_addr)
+{
+	int err;
+
+	err = lkl_if_add_rule_from_saddr(ifindex, LKL_AF_INET6, src_addr);
+	if (err)
+		return err;
+	err = lkl_if_add_linklocal(ifindex, LKL_AF_INET6,
+					src_addr, src_masklen);
+	if (err)
+		return err;
+	return lkl_if_add_gateway(ifindex, LKL_AF_INET6, via_addr);
+}
+
+int lkl_set_ipv6_gateway(void *addr)
+{
+	return lkl_add_gateway(LKL_AF_INET6, addr);
+}
+
+/* returns:
+ * 0 - succeed.
+ * < 0 - error number.
+ * 1 - should wait for new msg.
+ */
+static int check_error(struct lkl_sockaddr_nl *nladdr, struct lkl_nlmsghdr *n,
+		       void *arg)
+{
+	unsigned int s = *(unsigned int *)arg;
+
+	if (nladdr->nl_pid != 0 || n->nlmsg_seq != s) {
+		/* Don't forget to skip that message. */
+		return 1;
+	}
+
+	if (n->nlmsg_type == LKL_NLMSG_ERROR) {
+		struct lkl_nlmsgerr *err =
+			(struct lkl_nlmsgerr *)LKL_NLMSG_DATA(n);
+		int l = n->nlmsg_len - sizeof(*n);
+
+		if (l < (int)sizeof(struct lkl_nlmsgerr))
+			lkl_printf("ERROR truncated\n");
+		else if (!err->error)
+			return 0;
+
+		lkl_printf("RTNETLINK answers: %s\n",
+			lkl_strerror(-err->error));
+		return err->error;
+	}
+	lkl_printf("Unexpected reply!!!\n");
+	return -1;
+}
+
+static unsigned int seq;
+static int rtnl_talk(int fd, struct lkl_nlmsghdr *n)
+{
+	int status;
+	struct lkl_sockaddr_nl nladdr = {.nl_family = LKL_AF_NETLINK};
+	struct lkl_iovec iov = {.iov_base = (void *)n, .iov_len = n->nlmsg_len};
+	struct lkl_user_msghdr msg = {
+			.msg_name = &nladdr,
+			.msg_namelen = sizeof(nladdr),
+			.msg_iov = &iov,
+			.msg_iovlen = 1,
+	};
+
+	n->nlmsg_seq = seq;
+	n->nlmsg_flags |= LKL_NLM_F_ACK;
+
+	status = lkl_sys_sendmsg(fd, &msg, 0);
+	if (status < 0) {
+		lkl_perror("Cannot talk to rtnetlink", status);
+		return status;
+	}
+
+	status = rtnl_listen(fd, check_error, (void *)&seq);
+	seq++;
+	return status;
+}
+
+static int addattr_l(struct lkl_nlmsghdr *n, unsigned int maxlen,
+		     int type, const void *data, int alen)
+{
+	int len = LKL_RTA_LENGTH(alen);
+	struct lkl_rtattr *rta;
+
+	if (LKL_NLMSG_ALIGN(n->nlmsg_len) + LKL_RTA_ALIGN(len) > maxlen) {
+		lkl_printf("%s ERROR: message exceeded bound of %d\n", __func__,
+			   maxlen);
+		return -1;
+	}
+	rta = ((struct lkl_rtattr *) (((void *) (n)) +
+				      LKL_NLMSG_ALIGN(n->nlmsg_len)));
+	rta->rta_type = type;
+	rta->rta_len = len;
+	memcpy(LKL_RTA_DATA(rta), data, alen);
+	n->nlmsg_len = LKL_NLMSG_ALIGN(n->nlmsg_len) + LKL_RTA_ALIGN(len);
+	return 0;
+}
+
+int lkl_add_neighbor(int ifindex, int af, void *ip, void *mac)
+{
+	struct {
+		struct lkl_nlmsghdr n;
+		struct lkl_ndmsg r;
+		char buf[1024];
+	} req = {
+		.n.nlmsg_len = LKL_NLMSG_LENGTH(sizeof(struct lkl_ndmsg)),
+		.n.nlmsg_type = LKL_RTM_NEWNEIGH,
+		.n.nlmsg_flags = LKL_NLM_F_REQUEST |
+				 LKL_NLM_F_CREATE | LKL_NLM_F_REPLACE,
+		.r.ndm_family = af,
+		.r.ndm_ifindex = ifindex,
+		.r.ndm_state = LKL_NUD_PERMANENT,
+
+	};
+	int err, addr_sz;
+	int fd;
+
+	if (af == LKL_AF_INET)
+		addr_sz = 4;
+	else if (af == LKL_AF_INET6)
+		addr_sz = 16;
+	else {
+		lkl_printf("Bad address family: %d\n", af);
+		return -1;
+	}
+
+	fd = netlink_sock(0);
+	if (fd < 0)
+		return fd;
+
+	// create the IP attribute
+	addattr_l(&req.n, sizeof(req), LKL_NDA_DST, ip, addr_sz);
+
+	// create the MAC attribute
+	addattr_l(&req.n, sizeof(req), LKL_NDA_LLADDR, mac, 6);
+
+	err = rtnl_talk(fd, &req.n);
+	lkl_sys_close(fd);
+	return err;
+}
+
+static int ipaddr_modify(int cmd, int flags, int ifindex, int af, void *addr,
+			 unsigned int netprefix_len)
+{
+	struct {
+		struct lkl_nlmsghdr n;
+		struct lkl_ifaddrmsg ifa;
+		char buf[256];
+	} req = {
+		.n.nlmsg_len = LKL_NLMSG_LENGTH(sizeof(struct lkl_ifaddrmsg)),
+		.n.nlmsg_flags = LKL_NLM_F_REQUEST | flags,
+		.n.nlmsg_type = cmd,
+		.ifa.ifa_family = af,
+		.ifa.ifa_prefixlen = netprefix_len,
+		.ifa.ifa_index = ifindex,
+	};
+	int err, addr_sz;
+	int fd;
+
+	if (af == LKL_AF_INET)
+		addr_sz = 4;
+	else if (af == LKL_AF_INET6)
+		addr_sz = 16;
+	else {
+		lkl_printf("Bad address family: %d\n", af);
+		return -1;
+	}
+
+	fd = netlink_sock(0);
+	if (fd < 0)
+		return fd;
+
+	// create the IP attribute
+	addattr_l(&req.n, sizeof(req), LKL_IFA_LOCAL, addr, addr_sz);
+
+	err = rtnl_talk(fd, &req.n);
+
+	lkl_sys_close(fd);
+	return err;
+}
+
+int lkl_if_add_ip(int ifindex, int af, void *addr, unsigned int netprefix_len)
+{
+	return ipaddr_modify(LKL_RTM_NEWADDR, LKL_NLM_F_CREATE | LKL_NLM_F_EXCL,
+			     ifindex, af, addr, netprefix_len);
+}
+
+int lkl_if_del_ip(int ifindex, int af, void *addr, unsigned int netprefix_len)
+{
+	return ipaddr_modify(LKL_RTM_DELADDR, 0, ifindex, af,
+			     addr, netprefix_len);
+}
+
+static int iproute_modify(int cmd, unsigned int flags, int ifindex, int af,
+		void *route_addr, int route_masklen, void *gwaddr)
+{
+	struct {
+		struct lkl_nlmsghdr	n;
+		struct lkl_rtmsg	r;
+		char			buf[1024];
+	} req = {
+		.n.nlmsg_len = LKL_NLMSG_LENGTH(sizeof(struct lkl_rtmsg)),
+		.n.nlmsg_flags = LKL_NLM_F_REQUEST | flags,
+		.n.nlmsg_type = cmd,
+		.r.rtm_family = af,
+		.r.rtm_table = LKL_RT_TABLE_MAIN,
+		.r.rtm_scope = LKL_RT_SCOPE_UNIVERSE,
+	};
+	int err, addr_sz;
+	int i, fd;
+
+	fd = netlink_sock(0);
+	if (fd < 0) {
+		lkl_printf("netlink_sock error: %d\n", fd);
+		return fd;
+	}
+
+	if (af == LKL_AF_INET)
+		addr_sz = 4;
+	else if (af == LKL_AF_INET6)
+		addr_sz = 16;
+	else {
+		lkl_printf("Bad address family: %d\n", af);
+		return -1;
+	}
+
+	if (cmd != LKL_RTM_DELROUTE) {
+		req.r.rtm_protocol = LKL_RTPROT_BOOT;
+		req.r.rtm_scope = LKL_RT_SCOPE_UNIVERSE;
+		req.r.rtm_type = LKL_RTN_UNICAST;
+	}
+
+	if (gwaddr)
+		addattr_l(&req.n, sizeof(req),
+				LKL_RTA_GATEWAY, gwaddr, addr_sz);
+
+	if (af == LKL_AF_INET && route_addr) {
+		unsigned int netaddr = *(unsigned int *)route_addr;
+
+		netaddr = ntohl(netaddr);
+		netaddr = (netaddr >> (32 - route_masklen));
+		netaddr = (netaddr << (32 - route_masklen));
+		netaddr =  htonl(netaddr);
+		*(unsigned int *)route_addr = netaddr;
+		req.r.rtm_dst_len = route_masklen;
+		addattr_l(&req.n, sizeof(req), LKL_RTA_DST,
+					route_addr, addr_sz);
+	}
+
+	if (af == LKL_AF_INET6 && route_addr) {
+		struct lkl_in6_addr netaddr =
+			*(struct lkl_in6_addr *)route_addr;
+		int rmbyte = route_masklen/8;
+		int rmbit = route_masklen%8;
+
+		for (i = 0; i < rmbyte; i++)
+			netaddr.in6_u.u6_addr8[15-i] = 0;
+		netaddr.in6_u.u6_addr8[15-rmbyte] =
+			(netaddr.in6_u.u6_addr8[15-rmbyte] >> rmbit);
+		netaddr.in6_u.u6_addr8[15-rmbyte] =
+			(netaddr.in6_u.u6_addr8[15-rmbyte] << rmbit);
+		*(struct lkl_in6_addr *)route_addr = netaddr;
+		req.r.rtm_dst_len = route_masklen;
+		addattr_l(&req.n, sizeof(req), LKL_RTA_DST,
+					route_addr, addr_sz);
+	}
+
+	if (ifindex != LKL_RT_TABLE_MAIN) {
+		if (af == LKL_AF_INET)
+			req.r.rtm_table = ifindex * 2;
+		else if (af == LKL_AF_INET6)
+			req.r.rtm_table = ifindex * 2 + 1;
+		addattr_l(&req.n, sizeof(req), LKL_RTA_OIF, &ifindex, addr_sz);
+	}
+	err = rtnl_talk(fd, &req.n);
+	lkl_sys_close(fd);
+	return err;
+}
+
+int lkl_if_add_linklocal(int ifindex, int af,  void *addr, int netprefix_len)
+{
+	return iproute_modify(LKL_RTM_NEWROUTE, LKL_NLM_F_CREATE|LKL_NLM_F_EXCL,
+			ifindex, af, addr, netprefix_len, NULL);
+}
+
+int lkl_if_add_gateway(int ifindex, int af, void *gwaddr)
+{
+	return iproute_modify(LKL_RTM_NEWROUTE, LKL_NLM_F_CREATE|LKL_NLM_F_EXCL,
+			ifindex, af, NULL, 0, gwaddr);
+}
+
+int lkl_add_gateway(int af, void *gwaddr)
+{
+	return iproute_modify(LKL_RTM_NEWROUTE, LKL_NLM_F_CREATE|LKL_NLM_F_EXCL,
+			LKL_RT_TABLE_MAIN, af, NULL, 0, gwaddr);
+}
+
+static int iprule_modify(int cmd, int ifindex, int af, void *saddr)
+{
+	struct {
+		struct lkl_nlmsghdr	n;
+		struct lkl_rtmsg		r;
+		char			buf[1024];
+	} req = {
+		.n.nlmsg_type = cmd,
+		.n.nlmsg_len = LKL_NLMSG_LENGTH(sizeof(struct lkl_rtmsg)),
+		.n.nlmsg_flags = LKL_NLM_F_REQUEST,
+		.r.rtm_protocol = LKL_RTPROT_BOOT,
+		.r.rtm_scope = LKL_RT_SCOPE_UNIVERSE,
+		.r.rtm_family = af,
+		.r.rtm_type = LKL_RTN_UNSPEC,
+	};
+	int fd, err;
+	int addr_sz;
+
+	if (af == LKL_AF_INET)
+		addr_sz = 4;
+	else if (af == LKL_AF_INET6)
+		addr_sz = 16;
+	else {
+		lkl_printf("Bad address family: %d\n", af);
+		return -1;
+	}
+
+	fd = netlink_sock(0);
+	if (fd < 0)
+		return fd;
+
+	if (cmd == LKL_RTM_NEWRULE) {
+		req.n.nlmsg_flags |= LKL_NLM_F_CREATE|LKL_NLM_F_EXCL;
+		req.r.rtm_type = LKL_RTN_UNICAST;
+	}
+
+	//set from address
+	req.r.rtm_src_len = 8 * addr_sz;
+	addattr_l(&req.n, sizeof(req), LKL_FRA_SRC, saddr, addr_sz);
+
+	//use ifindex as table id
+	if (af == LKL_AF_INET)
+		req.r.rtm_table = ifindex * 2;
+	else if (af == LKL_AF_INET6)
+		req.r.rtm_table = ifindex * 2 + 1;
+	err = rtnl_talk(fd, &req.n);
+
+	lkl_sys_close(fd);
+	return err;
+}
+
+int lkl_if_add_rule_from_saddr(int ifindex, int af, void *saddr)
+{
+	return iprule_modify(LKL_RTM_NEWRULE, ifindex, af, saddr);
+}
+
+static int qdisc_add(int cmd, int flags, int ifindex,
+		     const char *root, const char *type)
+{
+	struct {
+		struct lkl_nlmsghdr n;
+		struct lkl_tcmsg tc;
+		char buf[2*1024];
+	} req = {
+		.n.nlmsg_len = LKL_NLMSG_LENGTH(sizeof(struct lkl_tcmsg)),
+		.n.nlmsg_flags = LKL_NLM_F_REQUEST|flags,
+		.n.nlmsg_type = cmd,
+		.tc.tcm_family = LKL_AF_UNSPEC,
+	};
+	int err, fd;
+
+	if (!root || !type) {
+		lkl_printf("root and type arguments\n");
+		return -1;
+	}
+
+	if (strcmp(root, "root") == 0)
+		req.tc.tcm_parent = LKL_TC_H_ROOT;
+	req.tc.tcm_ifindex = ifindex;
+
+	fd = netlink_sock(0);
+	if (fd < 0)
+		return fd;
+
+	// create the qdisc attribute
+	addattr_l(&req.n, sizeof(req), LKL_TCA_KIND, type, strlen(type)+1);
+
+	err = rtnl_talk(fd, &req.n);
+	lkl_sys_close(fd);
+	return err;
+}
+
+int lkl_qdisc_add(int ifindex, const char *root, const char *type)
+{
+	return qdisc_add(LKL_RTM_NEWQDISC, LKL_NLM_F_CREATE | LKL_NLM_F_EXCL,
+			 ifindex, root, type);
+}
+
+/* Add a qdisc entry for an interface in the form of
+ * "root|type;root|type;..."
+ */
+void lkl_qdisc_parse_add(int ifindex, const char *entries)
+{
+	char *saveptr = NULL, *token = NULL;
+	char *root = NULL, *type = NULL;
+	char strings[256];
+	int ret = 0;
+
+	strcpy(strings, entries);
+
+	for (token = strtok_r(strings, ";", &saveptr); token;
+	     token = strtok_r(NULL, ";", &saveptr)) {
+		root = strtok(token, "|");
+		type = strtok(NULL, "|");
+		ret = lkl_qdisc_add(ifindex, root, type);
+		if (ret) {
+			lkl_printf("Failed to add qdisc entry: %s\n",
+				   lkl_strerror(ret));
+			return;
+		}
+	}
+}
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v3 17/26] lkl tools: host lib: posix host operations
  2020-02-05  7:30 [RFC v3 00/26] Unifying LKL into UML Hajime Tazaki
@ 2020-02-05  7:30   ` Hajime Tazaki
       [not found] ` <cover.1580882335.git.thehajime-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
                     ` (25 subsequent siblings)
  26 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05  7:30 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Conrad Meyer, Mark Stillwell, Patrick Collins,
	Pierre-Hugues Husson, Thomas Liebetraut, Yuan Liu, Hajime Tazaki

From: Octavian Purdila <tavi.purdila@gmail.com>

Implement LKL host operations for POSIX hosts.

Cc: Conrad Meyer <cem@FreeBSD.org>
Cc: Mark Stillwell <mark@stillwell.me>
Cc: Patrick Collins <pscollins@google.com>
Cc: Pierre-Hugues Husson <phh@phh.me>
Cc: Thomas Liebetraut <thomas@tommie-lie.de>
Cc: Yuan Liu <liuyuan@google.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/lkl/lib/Build        |   2 +
 tools/lkl/lib/posix-host.c | 353 +++++++++++++++++++++++++++++++++++++
 2 files changed, 355 insertions(+)
 create mode 100644 tools/lkl/lib/posix-host.c

diff --git a/tools/lkl/lib/Build b/tools/lkl/lib/Build
index dba530424e4a..0e711e260a3a 100644
--- a/tools/lkl/lib/Build
+++ b/tools/lkl/lib/Build
@@ -1,8 +1,10 @@
 CFLAGS_config.o += -I$(srctree)/tools/perf/pmu-events
+CFLAGS_posix-host.o += -D_FILE_OFFSET_BITS=64
 
 liblkl-y += fs.o
 liblkl-y += net.o
 liblkl-y += jmp_buf.o
+liblkl-$(LKL_HOST_CONFIG_POSIX) += posix-host.o
 liblkl-y += utils.o
 liblkl-y += dbg.o
 liblkl-y += dbg_handler.o
diff --git a/tools/lkl/lib/posix-host.c b/tools/lkl/lib/posix-host.c
new file mode 100644
index 000000000000..4be1611a8942
--- /dev/null
+++ b/tools/lkl/lib/posix-host.c
@@ -0,0 +1,353 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <pthread.h>
+#include <stdlib.h>
+#include <sys/time.h>
+#include <time.h>
+#include <signal.h>
+#include <assert.h>
+#include <unistd.h>
+#include <errno.h>
+#include <string.h>
+#include <time.h>
+#include <stdint.h>
+#include <sys/uio.h>
+#include <sys/types.h>
+#include <sys/socket.h>
+#include <sys/syscall.h>
+#include <poll.h>
+#include <lkl_host.h>
+#include "jmp_buf.h"
+
+/* Let's see if the host has semaphore.h */
+#include <unistd.h>
+
+#ifdef _POSIX_SEMAPHORES
+#include <semaphore.h>
+/* TODO(pscollins): We don't support fork() for now, but maybe one day
+ * we will?
+ */
+#define SHARE_SEM 0
+#endif /* _POSIX_SEMAPHORES */
+
+static void print(const char *str, int len)
+{
+	int ret __attribute__((unused));
+
+	ret = write(STDOUT_FILENO, str, len);
+}
+
+struct lkl_mutex {
+	pthread_mutex_t mutex;
+};
+
+struct lkl_sem {
+#ifdef _POSIX_SEMAPHORES
+	sem_t sem;
+#else
+	pthread_mutex_t lock;
+	int count;
+	pthread_cond_t cond;
+#endif /* _POSIX_SEMAPHORES */
+};
+
+struct lkl_tls_key {
+	pthread_key_t key;
+};
+
+#define WARN_UNLESS(exp) do {						\
+		if (exp < 0)						\
+			lkl_printf("%s: %s\n", #exp, strerror(errno));	\
+	} while (0)
+
+static int _warn_pthread(int ret, char *str_exp)
+{
+	if (ret > 0)
+		lkl_printf("%s: %s\n", str_exp, strerror(ret));
+
+	return ret;
+}
+
+
+/* pthread_* functions use the reverse convention */
+#define WARN_PTHREAD(exp) _warn_pthread(exp, #exp)
+
+static struct lkl_sem *sem_alloc(int count)
+{
+	struct lkl_sem *sem;
+
+	sem = malloc(sizeof(*sem));
+	if (!sem)
+		return NULL;
+
+#ifdef _POSIX_SEMAPHORES
+	if (sem_init(&sem->sem, SHARE_SEM, count) < 0) {
+		lkl_printf("sem_init: %s\n", strerror(errno));
+		free(sem);
+		return NULL;
+	}
+#else
+	pthread_mutex_init(&sem->lock, NULL);
+	sem->count = count;
+	WARN_PTHREAD(pthread_cond_init(&sem->cond, NULL));
+#endif /* _POSIX_SEMAPHORES */
+
+	return sem;
+}
+
+static void sem_free(struct lkl_sem *sem)
+{
+#ifdef _POSIX_SEMAPHORES
+	WARN_UNLESS(sem_destroy(&sem->sem));
+#else
+	WARN_PTHREAD(pthread_cond_destroy(&sem->cond));
+	WARN_PTHREAD(pthread_mutex_destroy(&sem->lock));
+#endif /* _POSIX_SEMAPHORES */
+	free(sem);
+}
+
+static void sem_up(struct lkl_sem *sem)
+{
+#ifdef _POSIX_SEMAPHORES
+	WARN_UNLESS(sem_post(&sem->sem));
+#else
+	WARN_PTHREAD(pthread_mutex_lock(&sem->lock));
+	sem->count++;
+	if (sem->count > 0)
+		WARN_PTHREAD(pthread_cond_signal(&sem->cond));
+	WARN_PTHREAD(pthread_mutex_unlock(&sem->lock));
+#endif /* _POSIX_SEMAPHORES */
+
+}
+
+static void sem_down(struct lkl_sem *sem)
+{
+#ifdef _POSIX_SEMAPHORES
+	int err;
+
+	do {
+		err = sem_wait(&sem->sem);
+	} while (err < 0 && errno == EINTR);
+	if (err < 0 && errno != EINTR)
+		lkl_printf("sem_wait: %s\n", strerror(errno));
+#else
+	WARN_PTHREAD(pthread_mutex_lock(&sem->lock));
+	while (sem->count <= 0)
+		WARN_PTHREAD(pthread_cond_wait(&sem->cond, &sem->lock));
+	sem->count--;
+	WARN_PTHREAD(pthread_mutex_unlock(&sem->lock));
+#endif /* _POSIX_SEMAPHORES */
+}
+
+static struct lkl_mutex *mutex_alloc(int recursive)
+{
+	struct lkl_mutex *_mutex = malloc(sizeof(struct lkl_mutex));
+	pthread_mutex_t *mutex = NULL;
+	pthread_mutexattr_t attr;
+
+	if (!_mutex)
+		return NULL;
+
+	mutex = &_mutex->mutex;
+	WARN_PTHREAD(pthread_mutexattr_init(&attr));
+
+	/* PTHREAD_MUTEX_ERRORCHECK is *very* useful for debugging,
+	 * but has some overhead, so we provide an option to turn it
+	 * off.
+	 */
+#ifdef DEBUG
+	if (!recursive)
+		WARN_PTHREAD(pthread_mutexattr_settype(
+				     &attr, PTHREAD_MUTEX_ERRORCHECK));
+#endif /* DEBUG */
+
+	if (recursive)
+		WARN_PTHREAD(pthread_mutexattr_settype(
+				     &attr, PTHREAD_MUTEX_RECURSIVE));
+
+	WARN_PTHREAD(pthread_mutex_init(mutex, &attr));
+
+	return _mutex;
+}
+
+static void mutex_lock(struct lkl_mutex *mutex)
+{
+	WARN_PTHREAD(pthread_mutex_lock(&mutex->mutex));
+}
+
+static void mutex_unlock(struct lkl_mutex *_mutex)
+{
+	pthread_mutex_t *mutex = &_mutex->mutex;
+
+	WARN_PTHREAD(pthread_mutex_unlock(mutex));
+}
+
+static void mutex_free(struct lkl_mutex *_mutex)
+{
+	pthread_mutex_t *mutex = &_mutex->mutex;
+
+	WARN_PTHREAD(pthread_mutex_destroy(mutex));
+	free(_mutex);
+}
+
+static lkl_thread_t thread_create(void (*fn)(void *), void *arg)
+{
+	pthread_t thread;
+
+	if (WARN_PTHREAD(pthread_create(&thread, NULL, (void *)fn,
+					arg)))
+		return 0;
+	else
+		return (lkl_thread_t) thread;
+}
+
+static void thread_detach(void)
+{
+	WARN_PTHREAD(pthread_detach(pthread_self()));
+}
+
+static void thread_exit(void)
+{
+	pthread_exit(NULL);
+}
+
+static int thread_join(lkl_thread_t tid)
+{
+	if (WARN_PTHREAD(pthread_join((pthread_t)tid, NULL)))
+		return (-1);
+	else
+		return 0;
+}
+
+static lkl_thread_t thread_self(void)
+{
+	return (lkl_thread_t)pthread_self();
+}
+
+static int thread_equal(lkl_thread_t a, lkl_thread_t b)
+{
+	return pthread_equal((pthread_t)a, (pthread_t)b);
+}
+
+static struct lkl_tls_key *tls_alloc(void (*destructor)(void *))
+{
+	struct lkl_tls_key *ret = malloc(sizeof(struct lkl_tls_key));
+
+	if (WARN_PTHREAD(pthread_key_create(&ret->key, destructor))) {
+		free(ret);
+		return NULL;
+	}
+	return ret;
+}
+
+static void tls_free(struct lkl_tls_key *key)
+{
+	WARN_PTHREAD(pthread_key_delete(key->key));
+	free(key);
+}
+
+static int tls_set(struct lkl_tls_key *key, void *data)
+{
+	if (WARN_PTHREAD(pthread_setspecific(key->key, data)))
+		return (-1);
+	return 0;
+}
+
+static void *tls_get(struct lkl_tls_key *key)
+{
+	return pthread_getspecific(key->key);
+}
+
+static unsigned long long time_ns(void)
+{
+	struct timespec ts;
+
+	clock_gettime(CLOCK_MONOTONIC, &ts);
+
+	return 1e9*ts.tv_sec + ts.tv_nsec;
+}
+
+static void *timer_alloc(void (*fn)(void *), void *arg)
+{
+	int err;
+	timer_t timer;
+	struct sigevent se =  {
+		.sigev_notify = SIGEV_THREAD,
+		.sigev_value = {
+			.sival_ptr = arg,
+		},
+		.sigev_notify_function = (void *)fn,
+	};
+
+	err = timer_create(CLOCK_REALTIME, &se, &timer);
+	if (err)
+		return NULL;
+
+	return (void *)(long)timer;
+}
+
+static int timer_set_oneshot(void *_timer, unsigned long ns)
+{
+	timer_t timer = (timer_t)(long)_timer;
+	struct itimerspec ts = {
+		.it_value = {
+			.tv_sec = ns / 1000000000,
+			.tv_nsec = ns % 1000000000,
+		},
+	};
+
+	return timer_settime(timer, 0, &ts, NULL);
+}
+
+static void timer_free(void *_timer)
+{
+	timer_t timer = (timer_t)(long)_timer;
+
+	timer_delete(timer);
+}
+
+static void panic(void)
+{
+	assert(0);
+}
+
+static long _gettid(void)
+{
+#ifdef	__FreeBSD__
+	return (long)pthread_self();
+#else
+	return syscall(SYS_gettid);
+#endif
+}
+
+struct lkl_host_operations lkl_host_ops = {
+	.panic = panic,
+	.thread_create = thread_create,
+	.thread_detach = thread_detach,
+	.thread_exit = thread_exit,
+	.thread_join = thread_join,
+	.thread_self = thread_self,
+	.thread_equal = thread_equal,
+	.sem_alloc = sem_alloc,
+	.sem_free = sem_free,
+	.sem_up = sem_up,
+	.sem_down = sem_down,
+	.mutex_alloc = mutex_alloc,
+	.mutex_free = mutex_free,
+	.mutex_lock = mutex_lock,
+	.mutex_unlock = mutex_unlock,
+	.tls_alloc = tls_alloc,
+	.tls_free = tls_free,
+	.tls_set = tls_set,
+	.tls_get = tls_get,
+	.time = time_ns,
+	.timer_alloc = timer_alloc,
+	.timer_set_oneshot = timer_set_oneshot,
+	.timer_free = timer_free,
+	.print = print,
+	.mem_alloc = (void *)malloc,
+	.mem_free = free,
+	.gettid = _gettid,
+	.jmp_buf_set = jmp_buf_set,
+	.jmp_buf_longjmp = jmp_buf_longjmp,
+};
+
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v3 17/26] lkl tools: host lib: posix host operations
@ 2020-02-05  7:30   ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05  7:30 UTC (permalink / raw)
  To: linux-um
  Cc: linux-arch, Hajime Tazaki, Conrad Meyer, Octavian Purdila,
	Akira Moroo, Thomas Liebetraut, Mark Stillwell, Patrick Collins,
	linux-kernel-library, Pierre-Hugues Husson, Yuan Liu

From: Octavian Purdila <tavi.purdila@gmail.com>

Implement LKL host operations for POSIX hosts.

Cc: Conrad Meyer <cem@FreeBSD.org>
Cc: Mark Stillwell <mark@stillwell.me>
Cc: Patrick Collins <pscollins@google.com>
Cc: Pierre-Hugues Husson <phh@phh.me>
Cc: Thomas Liebetraut <thomas@tommie-lie.de>
Cc: Yuan Liu <liuyuan@google.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/lkl/lib/Build        |   2 +
 tools/lkl/lib/posix-host.c | 353 +++++++++++++++++++++++++++++++++++++
 2 files changed, 355 insertions(+)
 create mode 100644 tools/lkl/lib/posix-host.c

diff --git a/tools/lkl/lib/Build b/tools/lkl/lib/Build
index dba530424e4a..0e711e260a3a 100644
--- a/tools/lkl/lib/Build
+++ b/tools/lkl/lib/Build
@@ -1,8 +1,10 @@
 CFLAGS_config.o += -I$(srctree)/tools/perf/pmu-events
+CFLAGS_posix-host.o += -D_FILE_OFFSET_BITS=64
 
 liblkl-y += fs.o
 liblkl-y += net.o
 liblkl-y += jmp_buf.o
+liblkl-$(LKL_HOST_CONFIG_POSIX) += posix-host.o
 liblkl-y += utils.o
 liblkl-y += dbg.o
 liblkl-y += dbg_handler.o
diff --git a/tools/lkl/lib/posix-host.c b/tools/lkl/lib/posix-host.c
new file mode 100644
index 000000000000..4be1611a8942
--- /dev/null
+++ b/tools/lkl/lib/posix-host.c
@@ -0,0 +1,353 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <pthread.h>
+#include <stdlib.h>
+#include <sys/time.h>
+#include <time.h>
+#include <signal.h>
+#include <assert.h>
+#include <unistd.h>
+#include <errno.h>
+#include <string.h>
+#include <time.h>
+#include <stdint.h>
+#include <sys/uio.h>
+#include <sys/types.h>
+#include <sys/socket.h>
+#include <sys/syscall.h>
+#include <poll.h>
+#include <lkl_host.h>
+#include "jmp_buf.h"
+
+/* Let's see if the host has semaphore.h */
+#include <unistd.h>
+
+#ifdef _POSIX_SEMAPHORES
+#include <semaphore.h>
+/* TODO(pscollins): We don't support fork() for now, but maybe one day
+ * we will?
+ */
+#define SHARE_SEM 0
+#endif /* _POSIX_SEMAPHORES */
+
+static void print(const char *str, int len)
+{
+	int ret __attribute__((unused));
+
+	ret = write(STDOUT_FILENO, str, len);
+}
+
+struct lkl_mutex {
+	pthread_mutex_t mutex;
+};
+
+struct lkl_sem {
+#ifdef _POSIX_SEMAPHORES
+	sem_t sem;
+#else
+	pthread_mutex_t lock;
+	int count;
+	pthread_cond_t cond;
+#endif /* _POSIX_SEMAPHORES */
+};
+
+struct lkl_tls_key {
+	pthread_key_t key;
+};
+
+#define WARN_UNLESS(exp) do {						\
+		if (exp < 0)						\
+			lkl_printf("%s: %s\n", #exp, strerror(errno));	\
+	} while (0)
+
+static int _warn_pthread(int ret, char *str_exp)
+{
+	if (ret > 0)
+		lkl_printf("%s: %s\n", str_exp, strerror(ret));
+
+	return ret;
+}
+
+
+/* pthread_* functions use the reverse convention */
+#define WARN_PTHREAD(exp) _warn_pthread(exp, #exp)
+
+static struct lkl_sem *sem_alloc(int count)
+{
+	struct lkl_sem *sem;
+
+	sem = malloc(sizeof(*sem));
+	if (!sem)
+		return NULL;
+
+#ifdef _POSIX_SEMAPHORES
+	if (sem_init(&sem->sem, SHARE_SEM, count) < 0) {
+		lkl_printf("sem_init: %s\n", strerror(errno));
+		free(sem);
+		return NULL;
+	}
+#else
+	pthread_mutex_init(&sem->lock, NULL);
+	sem->count = count;
+	WARN_PTHREAD(pthread_cond_init(&sem->cond, NULL));
+#endif /* _POSIX_SEMAPHORES */
+
+	return sem;
+}
+
+static void sem_free(struct lkl_sem *sem)
+{
+#ifdef _POSIX_SEMAPHORES
+	WARN_UNLESS(sem_destroy(&sem->sem));
+#else
+	WARN_PTHREAD(pthread_cond_destroy(&sem->cond));
+	WARN_PTHREAD(pthread_mutex_destroy(&sem->lock));
+#endif /* _POSIX_SEMAPHORES */
+	free(sem);
+}
+
+static void sem_up(struct lkl_sem *sem)
+{
+#ifdef _POSIX_SEMAPHORES
+	WARN_UNLESS(sem_post(&sem->sem));
+#else
+	WARN_PTHREAD(pthread_mutex_lock(&sem->lock));
+	sem->count++;
+	if (sem->count > 0)
+		WARN_PTHREAD(pthread_cond_signal(&sem->cond));
+	WARN_PTHREAD(pthread_mutex_unlock(&sem->lock));
+#endif /* _POSIX_SEMAPHORES */
+
+}
+
+static void sem_down(struct lkl_sem *sem)
+{
+#ifdef _POSIX_SEMAPHORES
+	int err;
+
+	do {
+		err = sem_wait(&sem->sem);
+	} while (err < 0 && errno == EINTR);
+	if (err < 0 && errno != EINTR)
+		lkl_printf("sem_wait: %s\n", strerror(errno));
+#else
+	WARN_PTHREAD(pthread_mutex_lock(&sem->lock));
+	while (sem->count <= 0)
+		WARN_PTHREAD(pthread_cond_wait(&sem->cond, &sem->lock));
+	sem->count--;
+	WARN_PTHREAD(pthread_mutex_unlock(&sem->lock));
+#endif /* _POSIX_SEMAPHORES */
+}
+
+static struct lkl_mutex *mutex_alloc(int recursive)
+{
+	struct lkl_mutex *_mutex = malloc(sizeof(struct lkl_mutex));
+	pthread_mutex_t *mutex = NULL;
+	pthread_mutexattr_t attr;
+
+	if (!_mutex)
+		return NULL;
+
+	mutex = &_mutex->mutex;
+	WARN_PTHREAD(pthread_mutexattr_init(&attr));
+
+	/* PTHREAD_MUTEX_ERRORCHECK is *very* useful for debugging,
+	 * but has some overhead, so we provide an option to turn it
+	 * off.
+	 */
+#ifdef DEBUG
+	if (!recursive)
+		WARN_PTHREAD(pthread_mutexattr_settype(
+				     &attr, PTHREAD_MUTEX_ERRORCHECK));
+#endif /* DEBUG */
+
+	if (recursive)
+		WARN_PTHREAD(pthread_mutexattr_settype(
+				     &attr, PTHREAD_MUTEX_RECURSIVE));
+
+	WARN_PTHREAD(pthread_mutex_init(mutex, &attr));
+
+	return _mutex;
+}
+
+static void mutex_lock(struct lkl_mutex *mutex)
+{
+	WARN_PTHREAD(pthread_mutex_lock(&mutex->mutex));
+}
+
+static void mutex_unlock(struct lkl_mutex *_mutex)
+{
+	pthread_mutex_t *mutex = &_mutex->mutex;
+
+	WARN_PTHREAD(pthread_mutex_unlock(mutex));
+}
+
+static void mutex_free(struct lkl_mutex *_mutex)
+{
+	pthread_mutex_t *mutex = &_mutex->mutex;
+
+	WARN_PTHREAD(pthread_mutex_destroy(mutex));
+	free(_mutex);
+}
+
+static lkl_thread_t thread_create(void (*fn)(void *), void *arg)
+{
+	pthread_t thread;
+
+	if (WARN_PTHREAD(pthread_create(&thread, NULL, (void *)fn,
+					arg)))
+		return 0;
+	else
+		return (lkl_thread_t) thread;
+}
+
+static void thread_detach(void)
+{
+	WARN_PTHREAD(pthread_detach(pthread_self()));
+}
+
+static void thread_exit(void)
+{
+	pthread_exit(NULL);
+}
+
+static int thread_join(lkl_thread_t tid)
+{
+	if (WARN_PTHREAD(pthread_join((pthread_t)tid, NULL)))
+		return (-1);
+	else
+		return 0;
+}
+
+static lkl_thread_t thread_self(void)
+{
+	return (lkl_thread_t)pthread_self();
+}
+
+static int thread_equal(lkl_thread_t a, lkl_thread_t b)
+{
+	return pthread_equal((pthread_t)a, (pthread_t)b);
+}
+
+static struct lkl_tls_key *tls_alloc(void (*destructor)(void *))
+{
+	struct lkl_tls_key *ret = malloc(sizeof(struct lkl_tls_key));
+
+	if (WARN_PTHREAD(pthread_key_create(&ret->key, destructor))) {
+		free(ret);
+		return NULL;
+	}
+	return ret;
+}
+
+static void tls_free(struct lkl_tls_key *key)
+{
+	WARN_PTHREAD(pthread_key_delete(key->key));
+	free(key);
+}
+
+static int tls_set(struct lkl_tls_key *key, void *data)
+{
+	if (WARN_PTHREAD(pthread_setspecific(key->key, data)))
+		return (-1);
+	return 0;
+}
+
+static void *tls_get(struct lkl_tls_key *key)
+{
+	return pthread_getspecific(key->key);
+}
+
+static unsigned long long time_ns(void)
+{
+	struct timespec ts;
+
+	clock_gettime(CLOCK_MONOTONIC, &ts);
+
+	return 1e9*ts.tv_sec + ts.tv_nsec;
+}
+
+static void *timer_alloc(void (*fn)(void *), void *arg)
+{
+	int err;
+	timer_t timer;
+	struct sigevent se =  {
+		.sigev_notify = SIGEV_THREAD,
+		.sigev_value = {
+			.sival_ptr = arg,
+		},
+		.sigev_notify_function = (void *)fn,
+	};
+
+	err = timer_create(CLOCK_REALTIME, &se, &timer);
+	if (err)
+		return NULL;
+
+	return (void *)(long)timer;
+}
+
+static int timer_set_oneshot(void *_timer, unsigned long ns)
+{
+	timer_t timer = (timer_t)(long)_timer;
+	struct itimerspec ts = {
+		.it_value = {
+			.tv_sec = ns / 1000000000,
+			.tv_nsec = ns % 1000000000,
+		},
+	};
+
+	return timer_settime(timer, 0, &ts, NULL);
+}
+
+static void timer_free(void *_timer)
+{
+	timer_t timer = (timer_t)(long)_timer;
+
+	timer_delete(timer);
+}
+
+static void panic(void)
+{
+	assert(0);
+}
+
+static long _gettid(void)
+{
+#ifdef	__FreeBSD__
+	return (long)pthread_self();
+#else
+	return syscall(SYS_gettid);
+#endif
+}
+
+struct lkl_host_operations lkl_host_ops = {
+	.panic = panic,
+	.thread_create = thread_create,
+	.thread_detach = thread_detach,
+	.thread_exit = thread_exit,
+	.thread_join = thread_join,
+	.thread_self = thread_self,
+	.thread_equal = thread_equal,
+	.sem_alloc = sem_alloc,
+	.sem_free = sem_free,
+	.sem_up = sem_up,
+	.sem_down = sem_down,
+	.mutex_alloc = mutex_alloc,
+	.mutex_free = mutex_free,
+	.mutex_lock = mutex_lock,
+	.mutex_unlock = mutex_unlock,
+	.tls_alloc = tls_alloc,
+	.tls_free = tls_free,
+	.tls_set = tls_set,
+	.tls_get = tls_get,
+	.time = time_ns,
+	.timer_alloc = timer_alloc,
+	.timer_set_oneshot = timer_set_oneshot,
+	.timer_free = timer_free,
+	.print = print,
+	.mem_alloc = (void *)malloc,
+	.mem_free = free,
+	.gettid = _gettid,
+	.jmp_buf_set = jmp_buf_set,
+	.jmp_buf_longjmp = jmp_buf_longjmp,
+};
+
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v3 18/26] lkl tools: add test programs
  2020-02-05  7:30 [RFC v3 00/26] Unifying LKL into UML Hajime Tazaki
                   ` (16 preceding siblings ...)
  2020-02-05  7:30   ` Hajime Tazaki
@ 2020-02-05  7:30 ` Hajime Tazaki
  2020-02-05  7:30   ` Hajime Tazaki
                   ` (8 subsequent siblings)
  26 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05  7:30 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Conrad Meyer, David Disseldorp, Lai Jiangshan, Luca Dariz,
	Mark Stillwell, Motomu Utsumi, Patrick Collins, Petros Angelatos,
	Thomas Liebetraut, Yuan Liu, Hajime Tazaki

From: Octavian Purdila <tavi.purdila@gmail.com>

Add a simple LKL test application (boot) that starts the kernel and
performs simple tests that minimally exercise the LKL API. Networking
and block device tests are also added which are useful to detect
regressions.

Cc: Conrad Meyer <cem@FreeBSD.org>
Cc: David Disseldorp <ddiss@suse.de>
Cc: H.K. Jerry Chu <hkchu@google.com>
Cc: Lai Jiangshan <jiangshanlai@gmail.com>
Cc: Luca Dariz <luca.dariz@gmail.com>
Cc: Mark Stillwell <mark@stillwell.me>
Cc: Motomu Utsumi <motomuman@gmail.com>
Cc: Patrick Collins <pscollins@google.com>
Cc: Petros Angelatos <petrosagg@gmail.com>
Cc: Thomas Liebetraut <thomas@tommie-lie.de>
Cc: Yuan Liu <liuyuan@google.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/lkl/.gitignore              |   5 +
 tools/lkl/Makefile                |   5 +-
 tools/lkl/Targets                 |   3 +
 tools/lkl/tests/Build             |   3 +
 tools/lkl/tests/boot.c            | 562 ++++++++++++++++++++++++++++++
 tools/lkl/tests/boot.sh           |   9 +
 tools/lkl/tests/cla.c             | 159 +++++++++
 tools/lkl/tests/cla.h             |  33 ++
 tools/lkl/tests/disk.c            | 189 ++++++++++
 tools/lkl/tests/disk.sh           |  61 ++++
 tools/lkl/tests/net-setup.sh      |  73 ++++
 tools/lkl/tests/net-test.c        | 287 +++++++++++++++
 tools/lkl/tests/net.sh            |  90 +++++
 tools/lkl/tests/run.py            | 179 ++++++++++
 tools/lkl/tests/tap13.py          | 209 +++++++++++
 tools/lkl/tests/test.c            | 126 +++++++
 tools/lkl/tests/test.h            |  72 ++++
 tools/lkl/tests/test.sh           | 182 ++++++++++
 tools/lkl/tests/valgrind.supp     | 105 ++++++
 tools/lkl/tests/valgrind2xunit.py |  69 ++++
 20 files changed, 2420 insertions(+), 1 deletion(-)
 create mode 100644 tools/lkl/tests/Build
 create mode 100644 tools/lkl/tests/boot.c
 create mode 100755 tools/lkl/tests/boot.sh
 create mode 100644 tools/lkl/tests/cla.c
 create mode 100644 tools/lkl/tests/cla.h
 create mode 100644 tools/lkl/tests/disk.c
 create mode 100755 tools/lkl/tests/disk.sh
 create mode 100644 tools/lkl/tests/net-setup.sh
 create mode 100644 tools/lkl/tests/net-test.c
 create mode 100755 tools/lkl/tests/net.sh
 create mode 100755 tools/lkl/tests/run.py
 create mode 100644 tools/lkl/tests/tap13.py
 create mode 100644 tools/lkl/tests/test.c
 create mode 100644 tools/lkl/tests/test.h
 create mode 100644 tools/lkl/tests/test.sh
 create mode 100644 tools/lkl/tests/valgrind.supp
 create mode 100755 tools/lkl/tests/valgrind2xunit.py

diff --git a/tools/lkl/.gitignore b/tools/lkl/.gitignore
index 1aed58bfe171..4e08254dbd46 100644
--- a/tools/lkl/.gitignore
+++ b/tools/lkl/.gitignore
@@ -2,3 +2,8 @@ Makefile.conf
 include/lkl_autoconf.h
 tests/autoconf.sh
 bin/stat
+tests/net-test
+tests/disk
+tests/boot
+tests/valgrind*.xml
+*.pyc
diff --git a/tools/lkl/Makefile b/tools/lkl/Makefile
index 11ea7e9095bb..0abccb9d86b6 100644
--- a/tools/lkl/Makefile
+++ b/tools/lkl/Makefile
@@ -117,8 +117,11 @@ programs_install: $(progs-y:%=$(OUTPUT)%$(EXESUF))
 install: headers_install libraries_install programs_install
 
 
+run-tests:
+	./tests/run.py $(tests)
+
 FORCE: ;
-.PHONY: all clean FORCE
+.PHONY: all clean FORCE run-tests
 .PHONY: headers_install libraries_install programs_install install
 .NOTPARALLEL : lib/lkl.o
 .SECONDARY:
diff --git a/tools/lkl/Targets b/tools/lkl/Targets
index 24c985e64638..a9f74c3cc8fb 100644
--- a/tools/lkl/Targets
+++ b/tools/lkl/Targets
@@ -1,3 +1,6 @@
 libs-y += lib/liblkl
 
+progs-y += tests/boot
+progs-y += tests/disk
+progs-y += tests/net-test
 
diff --git a/tools/lkl/tests/Build b/tools/lkl/tests/Build
new file mode 100644
index 000000000000..ace86a3d3438
--- /dev/null
+++ b/tools/lkl/tests/Build
@@ -0,0 +1,3 @@
+boot-y += boot.o test.o
+disk-y += disk.o cla.o test.o
+net-test-y += net-test.o cla.o test.o
diff --git a/tools/lkl/tests/boot.c b/tools/lkl/tests/boot.c
new file mode 100644
index 000000000000..b021e9540147
--- /dev/null
+++ b/tools/lkl/tests/boot.c
@@ -0,0 +1,562 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <unistd.h>
+#include <string.h>
+#include <time.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <lkl.h>
+#include <lkl_host.h>
+
+#include <sys/stat.h>
+#include <fcntl.h>
+#if defined(__FreeBSD__)
+#include <net/if.h>
+#include <sys/ioctl.h>
+#elif __linux
+#include <sys/epoll.h>
+#include <sys/ioctl.h>
+#elif __MINGW32__
+#include <windows.h>
+#endif
+
+#include "test.h"
+
+#ifndef __MINGW32__
+#define sleep_ns 87654321
+int lkl_test_nanosleep(void)
+{
+	struct lkl_timespec ts = {
+		.tv_sec = 0,
+		.tv_nsec = sleep_ns,
+	};
+	struct timespec start, stop;
+	long delta;
+	long ret;
+
+	clock_gettime(CLOCK_MONOTONIC, &start);
+	ret = lkl_sys_nanosleep((struct __lkl__kernel_timespec *)&ts, NULL);
+	clock_gettime(CLOCK_MONOTONIC, &stop);
+
+	delta = 1e9*(stop.tv_sec - start.tv_sec) +
+		(stop.tv_nsec - start.tv_nsec);
+
+	lkl_test_logf("sleep %ld, expected sleep %d\n", delta, sleep_ns);
+
+	if (ret == 0 && delta > sleep_ns * 0.9)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+#endif
+
+LKL_TEST_CALL(getpid, lkl_sys_getpid, 1)
+
+void check_latency(long (*f)(void), long *min, long *max, long *avg)
+{
+	int i;
+	unsigned long long start, stop, sum = 0;
+	static const int count = 1000;
+	long delta;
+
+	*min = 1000000000;
+	*max = -1;
+
+	for (i = 0; i < count; i++) {
+		start = lkl_host_ops.time();
+		f();
+		stop = lkl_host_ops.time();
+		delta = stop - start;
+		if (*min > delta)
+			*min = delta;
+		if (*max < delta)
+			*max = delta;
+		sum += delta;
+	}
+	*avg = sum / count;
+}
+
+static long native_getpid(void)
+{
+#ifdef __MINGW32__
+	GetCurrentProcessId();
+#else
+	getpid();
+#endif
+	return 0;
+}
+
+int lkl_test_syscall_latency(void)
+{
+	long min, max, avg;
+
+	lkl_test_logf("avg/min/max: ");
+
+	check_latency(lkl_sys_getpid, &min, &max, &avg);
+
+	lkl_test_logf("lkl:%ld/%ld/%ld ", avg, min, max);
+
+	check_latency(native_getpid, &min, &max, &avg);
+
+	lkl_test_logf("native:%ld/%ld/%ld\n", avg, min, max);
+
+	return TEST_SUCCESS;
+}
+
+#define access_rights 0721
+
+LKL_TEST_CALL(creat, lkl_sys_creat, 0, "/file", access_rights)
+LKL_TEST_CALL(close, lkl_sys_close, 0, 0);
+LKL_TEST_CALL(failopen, lkl_sys_open, -LKL_ENOENT, "/file2", 0, 0);
+LKL_TEST_CALL(umask, lkl_sys_umask, 022,  0777);
+LKL_TEST_CALL(umask2, lkl_sys_umask, 0777, 0);
+LKL_TEST_CALL(open, lkl_sys_open, 0, "/file", LKL_O_RDWR, 0);
+static const char wrbuf[] = "test";
+LKL_TEST_CALL(write, lkl_sys_write, sizeof(wrbuf), 0, wrbuf, sizeof(wrbuf));
+LKL_TEST_CALL(lseek_cur, lkl_sys_lseek, sizeof(wrbuf), 0, 0, LKL_SEEK_CUR);
+LKL_TEST_CALL(lseek_end, lkl_sys_lseek, sizeof(wrbuf), 0, 0, LKL_SEEK_END);
+LKL_TEST_CALL(lseek_set, lkl_sys_lseek, 0, 0, 0, LKL_SEEK_SET);
+
+int lkl_test_read(void)
+{
+	char buf[10] = { 0, };
+	long ret;
+
+	ret = lkl_sys_read(0, buf, sizeof(buf));
+
+	lkl_test_logf("lkl_sys_read=%ld buf=%s\n", ret, buf);
+
+	if (ret == sizeof(wrbuf) && !strcmp(wrbuf, buf))
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+int lkl_test_fstat(void)
+{
+	struct lkl_stat stat;
+	long ret;
+
+	ret = lkl_sys_fstat(0, &stat);
+
+	lkl_test_logf("lkl_sys_fstat=%ld mode=%o size=%zd\n", ret, stat.st_mode,
+		      stat.st_size);
+
+	if (ret == 0 && stat.st_size == sizeof(wrbuf) &&
+	    stat.st_mode == (access_rights | LKL_S_IFREG))
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+LKL_TEST_CALL(mkdir, lkl_sys_mkdir, 0, "/mnt", access_rights)
+
+int lkl_test_stat(void)
+{
+	struct lkl_stat stat;
+	long ret;
+
+	ret = lkl_sys_stat("/mnt", &stat);
+
+	lkl_test_logf("lkl_sys_stat(\"/mnt\")=%ld mode=%o\n", ret,
+		      stat.st_mode);
+
+	if (ret == 0 && stat.st_mode == (access_rights | LKL_S_IFDIR))
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+static int lkl_test_pipe2(void)
+{
+	int pipe_fds[2];
+	int READ_IDX = 0, WRITE_IDX = 1;
+	static const char msg[] = "Hello world!";
+	char str[20];
+	int msg_len_bytes = strlen(msg) + 1;
+	int cmp_res;
+	long ret;
+
+	ret = lkl_sys_pipe2(pipe_fds, LKL_O_NONBLOCK);
+	if (ret) {
+		lkl_test_logf("pipe2: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_write(pipe_fds[WRITE_IDX], msg, msg_len_bytes);
+	if (ret != msg_len_bytes) {
+		if (ret < 0)
+			lkl_test_logf("write error: %s\n", lkl_strerror(ret));
+		else
+			lkl_test_logf("short write: %ld\n", ret);
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_read(pipe_fds[READ_IDX], str, msg_len_bytes);
+	if (ret != msg_len_bytes) {
+		if (ret < 0)
+			lkl_test_logf("read error: %s\n", lkl_strerror(ret));
+		else
+			lkl_test_logf("short read: %ld\n", ret);
+		return TEST_FAILURE;
+	}
+
+	cmp_res = memcmp(msg, str, msg_len_bytes);
+	if (cmp_res) {
+		lkl_test_logf("memcmp failed: %d\n", cmp_res);
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_close(pipe_fds[0]);
+	if (ret) {
+		lkl_test_logf("close error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_close(pipe_fds[1]);
+	if (ret) {
+		lkl_test_logf("close error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	return TEST_SUCCESS;
+}
+
+static int lkl_test_epoll(void)
+{
+	int epoll_fd, pipe_fds[2];
+	int READ_IDX = 0, WRITE_IDX = 1;
+	struct lkl_epoll_event wait_on, read_result;
+	static const char msg[] = "Hello world!";
+	long ret;
+
+	memset(&wait_on, 0, sizeof(wait_on));
+	memset(&read_result, 0, sizeof(read_result));
+
+	ret = lkl_sys_pipe2(pipe_fds, LKL_O_NONBLOCK);
+	if (ret) {
+		lkl_test_logf("pipe2 error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	epoll_fd = lkl_sys_epoll_create(1);
+	if (epoll_fd < 0) {
+		lkl_test_logf("epoll_create error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	wait_on.events = LKL_POLLIN | LKL_POLLOUT;
+	wait_on.data = pipe_fds[READ_IDX];
+
+	ret = lkl_sys_epoll_ctl(epoll_fd, LKL_EPOLL_CTL_ADD, pipe_fds[READ_IDX],
+				&wait_on);
+	if (ret < 0) {
+		lkl_test_logf("epoll_ctl error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	/* Shouldn't be ready before we have written something */
+	ret = lkl_sys_epoll_wait(epoll_fd, &read_result, 1, 0);
+	if (ret != 0) {
+		if (ret < 0)
+			lkl_test_logf("epoll_wait error: %s\n",
+				      lkl_strerror(ret));
+		else
+			lkl_test_logf("epoll_wait: bad event: 0x%lx\n", ret);
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_write(pipe_fds[WRITE_IDX], msg, strlen(msg) + 1);
+	if (ret < 0) {
+		lkl_test_logf("write error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	/* We expect exactly 1 fd to be ready immediately */
+	ret = lkl_sys_epoll_wait(epoll_fd, &read_result, 1, 0);
+	if (ret != 1) {
+		if (ret < 0)
+			lkl_test_logf("epoll_wait error: %s\n",
+				      lkl_strerror(ret));
+		else
+			lkl_test_logf("epoll_wait: bad ev no %ld\n", ret);
+		return TEST_FAILURE;
+	}
+
+	/* Already tested reading from pipe2 so no need to do it
+	 * here
+	 */
+
+	return TEST_SUCCESS;
+}
+
+LKL_TEST_CALL(chdir_proc, lkl_sys_chdir, 0, "proc");
+
+static int dir_fd;
+
+static int lkl_test_open_cwd(void)
+{
+	dir_fd = lkl_sys_open(".", LKL_O_RDONLY | LKL_O_DIRECTORY, 0);
+	if (dir_fd < 0) {
+		lkl_test_logf("failed to open current directory: %s\n",
+			      lkl_strerror(dir_fd));
+		return TEST_FAILURE;
+	}
+
+	return TEST_SUCCESS;
+}
+
+/* column where to insert a line break for the list file tests below. */
+#define COL_LINE_BREAK 70
+
+static int lkl_test_getdents64(void)
+{
+	long ret;
+	char buf[1024], *pos;
+	struct lkl_linux_dirent64 *de;
+	int wr;
+
+	de = (struct lkl_linux_dirent64 *)buf;
+	ret = lkl_sys_getdents64(dir_fd, de, sizeof(buf));
+
+	wr = lkl_test_logf("%d ", dir_fd);
+
+	if (ret < 0)
+		return TEST_FAILURE;
+
+	for (pos = buf; pos - buf < ret; pos += de->d_reclen) {
+		de = (struct lkl_linux_dirent64 *)pos;
+
+		wr += lkl_test_logf("%s ", de->d_name);
+		if (wr >= COL_LINE_BREAK) {
+			lkl_test_logf("\n");
+			wr = 0;
+		}
+	}
+
+	return TEST_SUCCESS;
+}
+
+LKL_TEST_CALL(close_dir_fd, lkl_sys_close, 0, dir_fd);
+LKL_TEST_CALL(chdir_root, lkl_sys_chdir, 0, "/");
+LKL_TEST_CALL(mount_fs_proc, lkl_mount_fs, 0, "proc");
+LKL_TEST_CALL(umount_fs_proc, lkl_umount_timeout, 0, "proc", 0, 1000);
+LKL_TEST_CALL(lo_ifup, lkl_if_up, 0, 1);
+
+static int lkl_test_mutex(void)
+{
+	long ret = TEST_SUCCESS;
+	/*
+	 * Can't do much to verify that this works, so we'll just let Valgrind
+	 * warn us on CI if we've made bad memory accesses.
+	 */
+
+	struct lkl_mutex *mutex;
+
+	mutex = lkl_host_ops.mutex_alloc(0);
+	lkl_host_ops.mutex_lock(mutex);
+	lkl_host_ops.mutex_unlock(mutex);
+	lkl_host_ops.mutex_free(mutex);
+
+	mutex = lkl_host_ops.mutex_alloc(1);
+	lkl_host_ops.mutex_lock(mutex);
+	lkl_host_ops.mutex_lock(mutex);
+	lkl_host_ops.mutex_unlock(mutex);
+	lkl_host_ops.mutex_unlock(mutex);
+	lkl_host_ops.mutex_free(mutex);
+
+	return ret;
+}
+
+static int lkl_test_semaphore(void)
+{
+	long ret = TEST_SUCCESS;
+	/*
+	 * Can't do much to verify that this works, so we'll just let Valgrind
+	 * warn us on CI if we've made bad memory accesses.
+	 */
+
+	struct lkl_sem *sem = lkl_host_ops.sem_alloc(1);
+
+	lkl_host_ops.sem_down(sem);
+	lkl_host_ops.sem_up(sem);
+	lkl_host_ops.sem_free(sem);
+
+	return ret;
+}
+
+static int lkl_test_gettid(void)
+{
+	long tid = lkl_host_ops.gettid();
+
+	lkl_test_logf("%ld", tid);
+
+	/* As far as I know, thread IDs are non-zero on all reasonable
+	 * systems.
+	 */
+	if (tid)
+		return TEST_SUCCESS;
+	else
+		return TEST_FAILURE;
+}
+
+static void test_thread(void *data)
+{
+	int *pipe_fds = (int *) data;
+	char tmp[LKL_PIPE_BUF+1];
+	int ret;
+
+	ret = lkl_sys_read(pipe_fds[0], tmp, sizeof(tmp));
+	if (ret < 0)
+		lkl_test_logf("%s: %s\n", __func__, lkl_strerror(ret));
+}
+
+static int lkl_test_syscall_thread(void)
+{
+	int pipe_fds[2];
+	char tmp[LKL_PIPE_BUF+1];
+	long ret;
+	lkl_thread_t tid;
+
+	ret = lkl_sys_pipe2(pipe_fds, 0);
+	if (ret) {
+		lkl_test_logf("pipe2: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_fcntl(pipe_fds[0], LKL_F_SETPIPE_SZ, 1);
+	if (ret < 0) {
+		lkl_test_logf("fcntl setpipe_sz: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	tid = lkl_host_ops.thread_create(test_thread, pipe_fds);
+	if (!tid) {
+		lkl_test_logf("failed to create thread\n");
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_write(pipe_fds[1], tmp, sizeof(tmp));
+	if (ret != sizeof(tmp)) {
+		if (ret < 0)
+			lkl_test_logf("write error: %s\n", lkl_strerror(ret));
+		else
+			lkl_test_logf("short write: %ld\n", ret);
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_host_ops.thread_join(tid);
+	if (ret) {
+		lkl_test_logf("failed to join thread\n");
+		return TEST_FAILURE;
+	}
+
+	return TEST_SUCCESS;
+}
+
+#ifndef __MINGW32__
+static void thread_get_pid(void *unused)
+{
+	lkl_sys_getpid();
+}
+
+static int lkl_test_many_syscall_threads(void)
+{
+	lkl_thread_t tid;
+	int count = 65, ret;
+
+	while (--count > 0) {
+		tid = lkl_host_ops.thread_create(thread_get_pid, NULL);
+		if (!tid) {
+			lkl_test_logf("failed to create thread\n");
+			return TEST_FAILURE;
+		}
+
+		ret = lkl_host_ops.thread_join(tid);
+		if (ret) {
+			lkl_test_logf("failed to join thread\n");
+			return TEST_FAILURE;
+		}
+	}
+
+	return TEST_SUCCESS;
+}
+#endif
+
+static void thread_quit_immediately(void *unused)
+{
+}
+
+static int lkl_test_join(void)
+{
+	lkl_thread_t tid = lkl_host_ops.thread_create(thread_quit_immediately,
+						      NULL);
+	int ret = lkl_host_ops.thread_join(tid);
+
+	if (ret == 0) {
+		lkl_test_logf("joined %ld\n", tid);
+		return TEST_SUCCESS;
+	}
+
+	lkl_test_logf("failed joining %ld\n", tid);
+	return TEST_FAILURE;
+}
+
+LKL_TEST_CALL(start_kernel, lkl_start_kernel, 0, &lkl_host_ops,
+	     "mem=16M loglevel=8");
+LKL_TEST_CALL(stop_kernel, lkl_sys_halt, 0);
+
+struct lkl_test tests[] = {
+	LKL_TEST(mutex),
+	LKL_TEST(semaphore),
+	LKL_TEST(join),
+	LKL_TEST(start_kernel),
+	LKL_TEST(getpid),
+	LKL_TEST(syscall_latency),
+	LKL_TEST(umask),
+	LKL_TEST(umask2),
+	LKL_TEST(creat),
+	LKL_TEST(close),
+	LKL_TEST(failopen),
+	LKL_TEST(open),
+	LKL_TEST(write),
+	LKL_TEST(lseek_cur),
+	LKL_TEST(lseek_end),
+	LKL_TEST(lseek_set),
+	LKL_TEST(read),
+	LKL_TEST(fstat),
+	LKL_TEST(mkdir),
+	LKL_TEST(stat),
+#ifndef __MINGW32__
+	LKL_TEST(nanosleep),
+#endif
+	LKL_TEST(pipe2),
+	LKL_TEST(epoll),
+	LKL_TEST(mount_fs_proc),
+	LKL_TEST(chdir_proc),
+	LKL_TEST(open_cwd),
+	LKL_TEST(getdents64),
+	LKL_TEST(close_dir_fd),
+	LKL_TEST(chdir_root),
+	LKL_TEST(umount_fs_proc),
+	LKL_TEST(lo_ifup),
+	LKL_TEST(gettid),
+	LKL_TEST(syscall_thread),
+	/*
+	 * Wine has an issue where the FlsCallback is not called when
+	 * the thread terminates which makes testing the automatic
+	 * syscall threads cleanup impossible under wine.
+	 */
+#ifndef __MINGW32__
+	LKL_TEST(many_syscall_threads),
+#endif
+	LKL_TEST(stop_kernel),
+};
+
+int main(int argc, const char **argv)
+{
+	lkl_host_ops.print = lkl_test_log;
+
+	return lkl_test_run(tests, sizeof(tests)/sizeof(struct lkl_test),
+			    "boot");
+}
diff --git a/tools/lkl/tests/boot.sh b/tools/lkl/tests/boot.sh
new file mode 100755
index 000000000000..d985c04b0ac1
--- /dev/null
+++ b/tools/lkl/tests/boot.sh
@@ -0,0 +1,9 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+source $script_dir/test.sh
+
+lkl_test_plan 1 "boot"
+lkl_test_run 1
+lkl_test_exec $script_dir/boot
diff --git a/tools/lkl/tests/cla.c b/tools/lkl/tests/cla.c
new file mode 100644
index 000000000000..a34badeb5f06
--- /dev/null
+++ b/tools/lkl/tests/cla.c
@@ -0,0 +1,159 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <string.h>
+#include <errno.h>
+#include <stdlib.h>
+#ifdef __MINGW32__
+#include <winsock2.h>
+#else
+#include <sys/socket.h>
+#include <netinet/in.h>
+#include <arpa/inet.h>
+#endif
+
+#include "cla.h"
+
+static int cl_arg_parse_bool(struct cl_arg *arg, const char *value)
+{
+	*((int *)arg->store) = 1;
+	return 0;
+}
+
+static int cl_arg_parse_str(struct cl_arg *arg, const char *value)
+{
+	*((const char **)arg->store) = value;
+	return 0;
+}
+
+static int cl_arg_parse_int(struct cl_arg *arg, const char *value)
+{
+	errno = 0;
+	*((int *)arg->store) = strtol(value, NULL, 0);
+	return errno == 0;
+}
+
+static int cl_arg_parse_str_set(struct cl_arg *arg, const char *value)
+{
+	const char **set = arg->set;
+	int i;
+
+	for (i = 0; set[i] != NULL; i++) {
+		if (strcmp(set[i], value) == 0) {
+			*((int *)arg->store) = i;
+			return 0;
+		}
+	}
+
+	return -1;
+}
+
+static int cl_arg_parse_ipv4(struct cl_arg *arg, const char *value)
+{
+	unsigned int addr;
+
+	if (!value)
+		return -1;
+
+	addr = inet_addr(value);
+	if (addr == INADDR_NONE)
+		return -1;
+	*((unsigned int *)arg->store) = addr;
+	return 0;
+}
+
+static cl_arg_parser_t parsers[] = {
+	[CL_ARG_BOOL] = cl_arg_parse_bool,
+	[CL_ARG_INT] = cl_arg_parse_int,
+	[CL_ARG_STR] = cl_arg_parse_str,
+	[CL_ARG_STR_SET] = cl_arg_parse_str_set,
+	[CL_ARG_IPV4] = cl_arg_parse_ipv4,
+};
+
+static struct cl_arg *find_short_arg(char name, struct cl_arg *args)
+{
+	struct cl_arg *arg;
+
+	for (arg = args; arg->short_name != 0; arg++) {
+		if (arg->short_name == name)
+			return arg;
+	}
+
+	return NULL;
+}
+
+static struct cl_arg *find_long_arg(const char *name, struct cl_arg *args)
+{
+	struct cl_arg *arg;
+
+	for (arg = args; arg->long_name; arg++) {
+		if (strcmp(arg->long_name, name) == 0)
+			return arg;
+	}
+
+	return NULL;
+}
+
+static void print_help(struct cl_arg *args)
+{
+	struct cl_arg *arg;
+
+	fprintf(stderr, "usage:\n");
+	for (arg = args; arg->long_name; arg++) {
+		fprintf(stderr, "-%c, --%-20s %s", arg->short_name,
+			arg->long_name, arg->help);
+		if (arg->type == CL_ARG_STR_SET) {
+			const char **set = arg->set;
+
+			fprintf(stderr, " [ ");
+			while (*set != NULL)
+				fprintf(stderr, "%s ", *(set++));
+			fprintf(stderr, "]");
+		}
+		fprintf(stderr, "\n");
+	}
+}
+
+int parse_args(int argc, const char **argv, struct cl_arg *args)
+{
+	int i;
+
+	for (i = 1; i < argc; i++) {
+		struct cl_arg *arg = NULL;
+		cl_arg_parser_t parser;
+
+		if (argv[i][0] == '-') {
+			if (argv[i][1] != '-')
+				arg = find_short_arg(argv[i][1], args);
+			else
+				arg = find_long_arg(&argv[i][2], args);
+		}
+
+		if (!arg) {
+			fprintf(stderr, "unknown option '%s'\n", argv[i]);
+			print_help(args);
+			return -1;
+		}
+
+		if (arg->type == CL_ARG_USER || arg->type >= CL_ARG_END)
+			parser = arg->parser;
+		else
+			parser = parsers[arg->type];
+
+		if (!parser) {
+			fprintf(stderr, "can't parse --'%s'/-'%c'\n",
+				arg->long_name, args->short_name);
+			return -1;
+		}
+
+		if (parser(arg, argv[i + 1]) < 0) {
+			fprintf(stderr, "can't parse '%s'\n", argv[i]);
+			print_help(args);
+			return -1;
+		}
+
+		if (arg->has_arg)
+			i++;
+	}
+
+	return 0;
+}
diff --git a/tools/lkl/tests/cla.h b/tools/lkl/tests/cla.h
new file mode 100644
index 000000000000..f8369be02e5a
--- /dev/null
+++ b/tools/lkl/tests/cla.h
@@ -0,0 +1,33 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_TEST_CLA_H
+#define _LKL_TEST_CLA_H
+
+enum cl_arg_type {
+	CL_ARG_USER = 0,
+	CL_ARG_BOOL,
+	CL_ARG_INT,
+	CL_ARG_STR,
+	CL_ARG_STR_SET,
+	CL_ARG_IPV4,
+	CL_ARG_END,
+};
+
+struct cl_arg;
+
+typedef int (*cl_arg_parser_t)(struct cl_arg *arg, const char *value);
+
+struct cl_arg {
+	const char *long_name;
+	char short_name;
+	const char *help;
+	int has_arg;
+	enum cl_arg_type type;
+	void *store;
+	void *set;
+	cl_arg_parser_t parser;
+};
+
+int parse_args(int argc, const char **argv, struct cl_arg *args);
+
+
+#endif /* _LKL_TEST_CLA_H */
diff --git a/tools/lkl/tests/disk.c b/tools/lkl/tests/disk.c
new file mode 100644
index 000000000000..0aa039876b54
--- /dev/null
+++ b/tools/lkl/tests/disk.c
@@ -0,0 +1,189 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <unistd.h>
+#include <string.h>
+#include <time.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <lkl.h>
+#include <lkl_host.h>
+#ifndef __MINGW32__
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <sys/ioctl.h>
+#else
+#include <windows.h>
+#endif
+
+#include "test.h"
+#include "cla.h"
+
+static struct {
+	int printk;
+	const char *disk;
+	const char *fstype;
+	int partition;
+} cla;
+
+struct cl_arg args[] = {
+	{"disk", 'd', "disk file to use", 1, CL_ARG_STR, &cla.disk},
+	{"partition", 'P', "partition to mount", 1, CL_ARG_INT, &cla.partition},
+	{"type", 't', "filesystem type", 1, CL_ARG_STR, &cla.fstype},
+	{0},
+};
+
+
+static struct lkl_disk disk;
+static int disk_id = -1;
+
+int lkl_test_disk_add(void)
+{
+#ifdef __MINGW32__
+	disk.handle = CreateFile(cla.disk, GENERIC_READ | GENERIC_WRITE,
+			       0, NULL, OPEN_EXISTING, 0, NULL);
+	if (!disk.handle)
+#else
+	disk.fd = open(cla.disk, O_RDWR);
+	if (disk.fd < 0)
+#endif
+		goto out_unlink;
+
+	disk.ops = NULL;
+
+	disk_id = lkl_disk_add(&disk);
+	if (disk_id < 0)
+		goto out_close;
+
+	goto out;
+
+out_close:
+#ifdef __MINGW32__
+	CloseHandle(disk.handle);
+#else
+	close(disk.fd);
+#endif
+
+out_unlink:
+#ifdef __MINGW32__
+	DeleteFile(cla.disk);
+#else
+	unlink(cla.disk);
+#endif
+
+out:
+	lkl_test_logf("disk fd/handle %x disk_id %d", disk.fd, disk_id);
+
+	if (disk_id >= 0)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+int lkl_test_disk_remove(void)
+{
+	int ret;
+
+	ret = lkl_disk_remove(disk);
+
+#ifdef __MINGW32__
+	CloseHandle(disk.handle);
+#else
+	close(disk.fd);
+#endif
+
+	if (ret == 0)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+
+static char mnt_point[32];
+
+LKL_TEST_CALL(mount_dev, lkl_mount_dev, 0, disk_id, cla.partition, cla.fstype,
+	      0, NULL, mnt_point, sizeof(mnt_point))
+
+static int lkl_test_umount_dev(void)
+{
+	long ret, ret2;
+
+	ret = lkl_sys_chdir("/");
+
+	ret2 = lkl_umount_dev(disk_id, cla.partition, 0, 1000);
+
+	lkl_test_logf("%ld %ld", ret, ret2);
+
+	if (!ret && !ret2)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+struct lkl_dir *dir;
+
+static int lkl_test_opendir(void)
+{
+	int err;
+
+	dir = lkl_opendir(mnt_point, &err);
+
+	lkl_test_logf("lkl_opedir(%s) = %d %s\n", mnt_point, err,
+		      lkl_strerror(err));
+
+	if (err == 0)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+static int lkl_test_readdir(void)
+{
+	struct lkl_linux_dirent64 *de = lkl_readdir(dir);
+	int wr = 0;
+
+	while (de) {
+		wr += lkl_test_logf("%s ", de->d_name);
+		if (wr >= 70) {
+			lkl_test_logf("\n");
+			wr = 0;
+			break;
+		}
+		de = lkl_readdir(dir);
+	}
+
+	if (lkl_errdir(dir) == 0)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+LKL_TEST_CALL(closedir, lkl_closedir, 0, dir);
+LKL_TEST_CALL(chdir_mnt_point, lkl_sys_chdir, 0, mnt_point);
+LKL_TEST_CALL(start_kernel, lkl_start_kernel, 0, &lkl_host_ops,
+	     "mem=16M loglevel=8");
+LKL_TEST_CALL(stop_kernel, lkl_sys_halt, 0);
+
+struct lkl_test tests[] = {
+	LKL_TEST(disk_add),
+	LKL_TEST(start_kernel),
+	LKL_TEST(mount_dev),
+	LKL_TEST(chdir_mnt_point),
+	LKL_TEST(opendir),
+	LKL_TEST(readdir),
+	LKL_TEST(closedir),
+	LKL_TEST(umount_dev),
+	LKL_TEST(stop_kernel),
+	LKL_TEST(disk_remove),
+
+};
+
+int main(int argc, const char **argv)
+{
+	if (parse_args(argc, argv, args) < 0)
+		return -1;
+
+	lkl_host_ops.print = lkl_test_log;
+
+	return lkl_test_run(tests, sizeof(tests)/sizeof(struct lkl_test),
+			    "disk %s", cla.fstype);
+}
diff --git a/tools/lkl/tests/disk.sh b/tools/lkl/tests/disk.sh
new file mode 100755
index 000000000000..9bdcb16f2d5c
--- /dev/null
+++ b/tools/lkl/tests/disk.sh
@@ -0,0 +1,61 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+
+source $script_dir/test.sh
+
+function prepfs()
+{
+    set -e
+
+    file=`mktemp`
+
+    dd if=/dev/zero of=$file bs=1024 count=204800
+
+    yes | mkfs.$1 $file
+
+    if ! [ -z $BSD_WDIR ]; then
+        $MYSSH mkdir -p $BSD_WDIR
+        ssh_copy $file $BSD_WDIR
+        rm $file
+        file=$BSD_WDIR/$(basename $file)
+    fi
+
+    export_vars file
+}
+
+function cleanfs()
+{
+    set -e
+
+    if ! [ -z $BSD_WDIR ]; then
+        $MYSSH rm $1
+        $MYSSH rm $BSD_WDIR/disk
+    else
+        rm $1
+    fi
+}
+
+if [ "$1" = "-t" ]; then
+    shift
+    fstype=$1
+    shift
+fi
+
+if [ -z "$fstype" ]; then
+    fstype="ext4"
+fi
+
+if [ -z $(which mkfs.$fstype) ]; then
+    lkl_test_plan 0 "disk $fstype"
+    echo "no mkfs.$fstype command"
+    exit 0
+fi
+
+lkl_test_plan 1 "disk $fstype"
+lkl_test_run 1 prepfs $fstype
+lkl_test_exec $script_dir/disk -d $file -t $fstype $@
+lkl_test_plan 1 "disk $fstype"
+lkl_test_run 1 cleanfs $file
+
diff --git a/tools/lkl/tests/net-setup.sh b/tools/lkl/tests/net-setup.sh
new file mode 100644
index 000000000000..0cfc42a30a54
--- /dev/null
+++ b/tools/lkl/tests/net-setup.sh
@@ -0,0 +1,73 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+TEST_IP_NETWORK=192.168.113.0
+TEST_IP_NETMASK=24
+TEST_IP6_NETWORK=fc03::0
+TEST_IP6_NETMASK=64
+TEST_MAC0="aa:bb:cc:dd:ee:ff"
+TEST_MAC1="aa:bb:cc:dd:ee:aa"
+TEST_NETSERVER_PORT=11223
+
+# $1 - count
+# $2 - netcount
+ip_add()
+{
+    IP_HEX=$(printf '%.2X%.2X%.2X%.2X\n' \
+         `echo $TEST_IP_NETWORK | sed -e 's/\./ /g'`)
+    NET_COUNT=$(( 1 << (32 - $TEST_IP_NETMASK) ))
+    NEXT_IP_HEX=$(printf %.8X `echo $((0x$IP_HEX + $1 + ${2:-0} * $NET_COUNT))`)
+    NEXT_IP=$(printf '%d.%d.%d.%d\n' \
+          `echo $NEXT_IP_HEX | sed -r 's/(..)/0x\1 /g'`)
+    echo -n "$NEXT_IP"
+}
+
+# $1 - count
+# $2 - netcount
+ip6_add()
+{
+    IP6_PREFIX=${TEST_IP6_NETWORK%*::*}
+    IP6_HOST=${TEST_IP6_NETWORK#*::*}
+    echo -n "$(printf "%x" $((0x$IP6_PREFIX+${2:-0})))::$(($IP6_HOST+$1))"
+}
+
+ip_host()
+{
+
+    ip_add 1 $1
+}
+
+ip_lkl()
+{
+    ip_add 2 $1
+}
+
+ip_host_mask()
+{
+    echo -n "$(ip_host $1)/$TEST_IP_NETMASK"
+}
+
+ip_net_mask()
+{
+    echo "$(ip_add 0 $1)/$TEST_IP_NETMASK"
+}
+
+ip6_host()
+{
+    ip6_add 1 $1
+}
+
+ip6_lkl()
+{
+    ip6_add 2 $1
+}
+
+ip6_host_mask()
+{
+    echo -n "$(ip6_host $1)/$TEST_IP6_NETMASK"
+}
+
+ip6_net_mask()
+{
+    echo "$(ip6_add 0 $1)/$TEST_IP6_NETMASK"
+}
diff --git a/tools/lkl/tests/net-test.c b/tools/lkl/tests/net-test.c
new file mode 100644
index 000000000000..edc37150270b
--- /dev/null
+++ b/tools/lkl/tests/net-test.c
@@ -0,0 +1,287 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <string.h>
+#include <stdlib.h>
+#include <errno.h>
+#ifdef __FreeBSD__
+#include <sys/types.h>
+#endif
+#ifdef __MINGW32__
+#include <winsock2.h>
+#else
+#include <sys/socket.h>
+#include <netinet/in.h>
+#include <arpa/inet.h>
+#endif
+
+#include <lkl.h>
+#include <lkl_host.h>
+
+#include "cla.h"
+#include "test.h"
+
+enum {
+	BACKEND_NONE,
+};
+
+const char *backends[] = { "loopback", NULL };
+static struct {
+	int backend;
+	const char *ifname;
+	int dhcp, nmlen;
+	unsigned int ip, dst, gateway, sleep;
+} cla = {
+	.backend = BACKEND_NONE,
+	.ip = INADDR_NONE,
+	.gateway = INADDR_NONE,
+	.dst = INADDR_NONE,
+	.sleep = 0,
+};
+
+
+struct cl_arg args[] = {
+	{"backend", 'b', "network backend type", 1, CL_ARG_STR_SET,
+	 &cla.backend, backends},
+	{"ifname", 'i', "interface name", 1, CL_ARG_STR, &cla.ifname},
+	{"dhcp", 'd', "use dhcp to configure LKL", 0, CL_ARG_BOOL, &cla.dhcp},
+	{"ip", 'I', "IPv4 address to use", 1, CL_ARG_IPV4, &cla.ip},
+	{"netmask-len", 'n', "IPv4 netmask length", 1, CL_ARG_INT,
+	 &cla.nmlen},
+	{"gateway", 'g', "IPv4 gateway to use", 1, CL_ARG_IPV4, &cla.gateway},
+	{"dst", 'D', "IPv4 destination address", 1, CL_ARG_IPV4, &cla.dst},
+	{"sleep", 's', "sleep", 1, CL_ARG_INT, &cla.sleep},
+	{0},
+};
+
+u_short
+in_cksum(const u_short *addr, register int len, u_short csum)
+{
+	int nleft = len;
+	const u_short *w = addr;
+	u_short answer;
+	int sum = csum;
+
+	while (nleft > 1)  {
+		sum += *w++;
+		nleft -= 2;
+	}
+
+	if (nleft == 1)
+		sum += htons(*(u_char *)w << 8);
+
+	sum = (sum >> 16) + (sum & 0xffff);
+	sum += (sum >> 16);
+	answer = ~sum;
+	return answer;
+}
+
+static int lkl_test_sleep(void)
+{
+	struct lkl_timespec ts = {
+		.tv_sec = cla.sleep,
+	};
+	int ret;
+
+	ret = lkl_sys_nanosleep((struct __lkl__kernel_timespec *)&ts, NULL);
+	if (ret < 0) {
+		lkl_test_logf("nanosleep error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	return TEST_SUCCESS;
+}
+
+static int lkl_test_icmp(void)
+{
+	int sock, ret;
+	struct lkl_iphdr *iph;
+	struct lkl_icmphdr *icmp;
+	struct lkl_sockaddr_in saddr;
+	struct lkl_pollfd pfd;
+	char buf[32];
+
+	if (cla.dst == INADDR_NONE)
+		return TEST_SKIP;
+
+	memset(&saddr, 0, sizeof(saddr));
+	saddr.sin_family = AF_INET;
+	saddr.sin_addr.lkl_s_addr = cla.dst;
+
+	lkl_test_logf("pinging %s\n",
+		      inet_ntoa(*(struct in_addr *)&saddr.sin_addr));
+
+	sock = lkl_sys_socket(LKL_AF_INET, LKL_SOCK_RAW, LKL_IPPROTO_ICMP);
+	if (sock < 0) {
+		lkl_test_logf("socket error (%s)\n", lkl_strerror(sock));
+		return TEST_FAILURE;
+	}
+
+	icmp = malloc(sizeof(struct lkl_icmphdr *));
+	icmp->type = LKL_ICMP_ECHO;
+	icmp->code = 0;
+	icmp->checksum = 0;
+	icmp->un.echo.sequence = 0;
+	icmp->un.echo.id = 0;
+	icmp->checksum = in_cksum((u_short *)icmp, sizeof(*icmp), 0);
+
+	ret = lkl_sys_sendto(sock, icmp, sizeof(*icmp), 0,
+			     (struct lkl_sockaddr *)&saddr,
+			     sizeof(saddr));
+	if (ret < 0) {
+		lkl_test_logf("sendto error (%s)\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	free(icmp);
+
+	pfd.fd = sock;
+	pfd.events = LKL_POLLIN;
+	pfd.revents = 0;
+
+	ret = lkl_sys_poll(&pfd, 1, 1000);
+	if (ret < 0) {
+		lkl_test_logf("poll error (%s)\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_recv(sock, buf, sizeof(buf), LKL_MSG_DONTWAIT);
+	if (ret < 0) {
+		lkl_test_logf("recv error (%s)\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	iph = (struct lkl_iphdr *)buf;
+	icmp = (struct lkl_icmphdr *)(buf + iph->ihl * 4);
+	/* DHCP server may issue an ICMP echo request to a dhcp client */
+	if ((icmp->type != LKL_ICMP_ECHOREPLY || icmp->code != 0) &&
+	    (icmp->type != LKL_ICMP_ECHO)) {
+		lkl_test_logf("no ICMP echo reply (type=%d, code=%d)\n",
+			      icmp->type, icmp->code);
+		return TEST_FAILURE;
+	}
+
+	return TEST_SUCCESS;
+}
+
+static struct lkl_netdev *nd;
+
+static int lkl_test_nd_create(void)
+{
+	switch (cla.backend) {
+	case BACKEND_NONE:
+		return TEST_SKIP;
+	}
+
+	if (!nd) {
+		lkl_test_logf("failed to create netdev\n");
+		return TEST_BAILOUT;
+	}
+
+	return TEST_SUCCESS;
+}
+
+static int nd_id;
+
+static int lkl_test_nd_add(void)
+{
+	if (cla.backend == BACKEND_NONE)
+		return TEST_SKIP;
+
+	return TEST_SUCCESS;
+}
+
+static int lkl_test_nd_remove(void)
+{
+	if (cla.backend == BACKEND_NONE)
+		return TEST_SKIP;
+
+	return TEST_SUCCESS;
+}
+
+LKL_TEST_CALL(start_kernel, lkl_start_kernel, 0, &lkl_host_ops,
+	"mem=16M loglevel=8 %s", cla.dhcp ? "ip=dhcp" : "");
+LKL_TEST_CALL(stop_kernel, lkl_sys_halt, 0);
+
+static int nd_ifindex;
+
+static int lkl_test_nd_ifindex(void)
+{
+	if (cla.backend == BACKEND_NONE)
+		return TEST_SKIP;
+
+	nd_ifindex = lkl_netdev_get_ifindex(nd_id);
+	if (nd_ifindex < 0) {
+		lkl_test_logf("failed to get ifindex for netdev id %d: %s\n",
+			      nd_id, lkl_strerror(nd_ifindex));
+		return TEST_BAILOUT;
+	}
+
+	return TEST_SUCCESS;
+}
+
+LKL_TEST_CALL(if_up, lkl_if_up, 0,
+	      cla.backend == BACKEND_NONE ? 1 : nd_ifindex);
+
+static int lkl_test_set_ipv4(void)
+{
+	int ret;
+
+	if (cla.backend == BACKEND_NONE || cla.ip == LKL_INADDR_NONE)
+		return TEST_SKIP;
+
+	ret = lkl_if_set_ipv4(nd_ifindex, cla.ip, cla.nmlen);
+	if (ret < 0) {
+		lkl_test_logf("failed to set IPv4 address: %s\n",
+			      lkl_strerror(ret));
+		return TEST_BAILOUT;
+	}
+
+	return TEST_SUCCESS;
+}
+
+static int lkl_test_set_gateway(void)
+{
+	int ret;
+
+	if (cla.backend == BACKEND_NONE || cla.gateway == LKL_INADDR_NONE)
+		return TEST_SKIP;
+
+	ret = lkl_set_ipv4_gateway(cla.gateway);
+	if (ret < 0) {
+		lkl_test_logf("failed to set IPv4 gateway: %s\n",
+			      lkl_strerror(ret));
+		return TEST_BAILOUT;
+	}
+
+	return TEST_SUCCESS;
+}
+
+struct lkl_test tests[] = {
+	LKL_TEST(nd_create),
+	LKL_TEST(nd_add),
+	LKL_TEST(start_kernel),
+	LKL_TEST(nd_ifindex),
+	LKL_TEST(if_up),
+	LKL_TEST(set_ipv4),
+	LKL_TEST(set_gateway),
+	LKL_TEST(sleep),
+	LKL_TEST(icmp),
+	LKL_TEST(nd_remove),
+	LKL_TEST(stop_kernel),
+};
+
+int main(int argc, const char **argv)
+{
+	if (parse_args(argc, argv, args) < 0)
+		return -1;
+
+	if (cla.ip != LKL_INADDR_NONE && (cla.nmlen < 0 || cla.nmlen > 32)) {
+		fprintf(stderr, "invalid netmask length %d\n", cla.nmlen);
+		return -1;
+	}
+
+	lkl_host_ops.print = lkl_test_log;
+
+	return lkl_test_run(tests, sizeof(tests)/sizeof(struct lkl_test),
+			    "net %s", backends[cla.backend]);
+}
diff --git a/tools/lkl/tests/net.sh b/tools/lkl/tests/net.sh
new file mode 100755
index 000000000000..32e8452b0406
--- /dev/null
+++ b/tools/lkl/tests/net.sh
@@ -0,0 +1,90 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+
+source $script_dir/test.sh
+source $script_dir/net-setup.sh
+
+cleanup_backend()
+{
+    set -e
+
+    case "$1" in
+    "loopback")
+        ;;
+    esac
+}
+
+get_test_ip()
+{
+    # DHCP test parameters
+    TEST_HOST=8.8.8.8
+    HOST_IF=$(lkl_test_cmd ip route get $TEST_HOST | head -n1 |cut -d ' ' -f5)
+    HOST_GW=$(lkl_test_cmd ip route get $TEST_HOST | head -n1 | cut -d ' ' -f3)
+    if lkl_test_cmd ping -c1 -w1 $HOST_GW; then
+        TEST_IP_REMOTE=$HOST_GW
+    elif lkl_test_cmd ping -c1 -w1 $TEST_HOST; then
+        TEST_IP_REMOTE=$TEST_HOST
+    else
+        echo "could not find remote test ip"
+        return $TEST_SKIP
+    fi
+
+    export_vars HOST_IF TEST_IP_REMOTE
+}
+
+setup_backend()
+{
+    set -e
+
+    if [ "$LKL_HOST_CONFIG_POSIX" != "y" ] &&
+       [ "$1" != "loopback" ]; then
+        echo "not a posix environment"
+        return $TEST_SKIP
+    fi
+
+    case "$1" in
+    "loopback")
+        ;;
+    *)
+        echo "don't know how to setup backend $1"
+        return $TEST_FAILED
+        ;;
+    esac
+}
+
+run_tests()
+{
+    case "$1" in
+    "loopback")
+        lkl_test_exec $script_dir/net-test --dst 127.0.0.1
+        ;;
+    esac
+}
+
+if [ "$1" = "-b" ]; then
+    shift
+    backend=$1
+    shift
+fi
+
+if [ -z "$backend" ]; then
+    backend="loopback"
+fi
+
+lkl_test_plan 1 "net $backend"
+lkl_test_run 1 setup_backend $backend
+
+if [ $? = $TEST_SKIP ]; then
+    exit 0
+fi
+
+trap "cleanup_backend $backend" EXIT
+
+run_tests $backend
+
+trap : EXIT
+lkl_test_plan 1 "net $backend"
+lkl_test_run 1 cleanup_backend $backend
+
diff --git a/tools/lkl/tests/run.py b/tools/lkl/tests/run.py
new file mode 100755
index 000000000000..b72299aaabee
--- /dev/null
+++ b/tools/lkl/tests/run.py
@@ -0,0 +1,179 @@
+#!/usr/bin/env python
+# SPDX-License-Identifier: GPL-2.0
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 2 of the License
+#
+# Author: Octavian Purdila <tavi@cs.pub.ro>
+#
+
+from __future__ import print_function
+
+import argparse
+import os
+import subprocess
+import sys
+import tap13
+import xml.etree.ElementTree as ET
+
+from junit_xml import TestSuite, TestCase
+
+
+class Reporter(tap13.Reporter):
+    def start(self, obj):
+        if type(obj) is tap13.Test:
+            if obj.result == "*":
+                end='\r'
+            else:
+                end='\n'
+            print("  TEST       %-8s %.50s" %
+                  (obj.result, obj.description + " " + obj.comment), end=end)
+
+        elif type(obj) is tap13.Suite:
+            if obj.tests_planned == 0:
+                status = "skip"
+            else:
+                status = ""
+            print("  SUITE      %-8s %s" % (status, obj.name))
+
+    def end(self, obj):
+        if type(obj) is tap13.Test:
+            if obj.result != "ok":
+                try:
+                    print(obj.yaml["log"], end='')
+                except:
+                    None
+
+
+mydir=os.path.dirname(os.path.realpath(__file__))
+
+tests = [
+    'boot.sh',
+    'disk.sh -t ext4',
+    'disk.sh -t vfat',
+    'disk.sh -t btrfs',
+    'net.sh -b loopback',
+    'lklfuse.sh -t ext4',
+    'lklfuse.sh -t vfat',
+    'lklfuse.sh -t btrfs',
+]
+
+parser = argparse.ArgumentParser(description='LKL test runner')
+parser.add_argument('tests', nargs='?', action='append',
+                    help='tests to run %s' % tests)
+parser.add_argument('--junit-dir',
+                    help='directory where to store the juni suites')
+parser.add_argument('--gdb', action='store_true', default=False,
+                    help='run simple tests under gdb; implies --pass-through')
+parser.add_argument('--pass-through', action='store_true',  default=False,
+                    help='run the test without interpeting the test output')
+parser.add_argument('--valgrind', action='store_true', default=False,
+                    help='run simple tests under valgrind')
+
+args = parser.parse_args()
+if args.tests == [None]:
+    args.tests = tests
+
+if args.gdb:
+    args.pass_through=True
+    os.environ['GDB']="yes"
+
+if args.valgrind:
+    os.environ['VALGRIND']="yes"
+
+tap = tap13.Parser(Reporter())
+
+os.environ['PATH'] += ":" + mydir
+
+exit_code = 0
+
+for t in args.tests:
+    if not t:
+        continue
+    if args.pass_through:
+        print(t)
+        if subprocess.call(t, shell=True) != 0:
+            exit_code = 1
+    else:
+        p = subprocess.Popen(t, shell=True, stdout=subprocess.PIPE)
+        tap.parse(p.stdout)
+
+if args.pass_through:
+    sys.exit(exit_code)
+
+suites_count = 0
+tests_total = 0
+tests_not_ok = 0
+tests_ok = 0
+tests_skip = 0
+val_errs = 0
+val_fails = 0
+val_skips = 0
+
+for s in tap.run.suites:
+
+    junit_tests = []
+    suites_count += 1
+
+    for t in s.tests:
+        try:
+            secs = t.yaml["time_us"] / 1000000.0
+        except:
+            secs = 0
+        try:
+            log = t.yaml['log']
+        except:
+            log = ""
+
+        jt = TestCase(t.description, elapsed_sec=secs, stdout=log)
+        if t.result == 'skip':
+            jt.add_skipped_info(output=log)
+        elif t.result == 'not ok':
+            jt.add_error_info(output=log)
+
+        junit_tests.append(jt)
+
+        tests_total += 1
+        if t.result == "ok":
+            tests_ok += 1
+        elif t.result == "not ok":
+            tests_not_ok += 1
+            exit_code = 1
+        elif t.result == "skip":
+            tests_skip += 1
+
+    if args.junit_dir:
+        js = TestSuite(s.name, junit_tests)
+        with open(os.path.join(args.junit_dir, os.path.basename(s.name) + '.xml'), 'w') as f:
+            js.to_file(f, [js])
+
+        if os.getenv('VALGRIND') is not None:
+            val_xml = 'valgrind-%s.xml' % os.path.basename(s.name).replace(' ','-')
+            # skipped tests don't generate xml file
+            if os.path.exists(val_xml) is False:
+                continue
+
+            cmd = 'mv %s %s' % (val_xml, args.junit_dir)
+            subprocess.call(cmd, shell=True, )
+
+            cmd = mydir + '/valgrind2xunit.py ' + val_xml
+            subprocess.call(cmd, shell=True, cwd=args.junit_dir)
+
+            # count valgrind results
+            doc = ET.parse(os.path.join(args.junit_dir, 'valgrind-%s_xunit.xml' \
+                                        % (os.path.basename(s.name).replace(' ','-'))))
+            ts = doc.getroot()
+            val_errs += int(ts.get('errors'))
+            val_fails += int(ts.get('failures'))
+            val_skips += int(ts.get('skip'))
+
+print("Summary: %d suites run, %d tests, %d ok, %d not ok, %d skipped" %
+      (suites_count, tests_total, tests_ok, tests_not_ok, tests_skip))
+
+if os.getenv('VALGRIND') is not None:
+    print(" valgrind (memcheck): %d failures, %d skipped" % (val_fails, val_skips))
+    if val_errs or val_fails:
+        exit_code = 1
+
+sys.exit(exit_code)
diff --git a/tools/lkl/tests/tap13.py b/tools/lkl/tests/tap13.py
new file mode 100644
index 000000000000..65c73cda7ca1
--- /dev/null
+++ b/tools/lkl/tests/tap13.py
@@ -0,0 +1,209 @@
+#!/usr/bin/env python
+# SPDX-License-Identifier: GPL-2.0
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 2 of the License
+#
+# Author: Octavian Purdila <tavi@cs.pub.ro>
+#
+# Based on TAP13:
+#
+# Copyright 2013, Red Hat, Inc.
+# Author: Josef Skladanka <jskladan@redhat.com>
+#
+from __future__ import print_function
+
+import re
+import sys
+import yamlish
+
+
+class Reporter(object):
+
+    def start(self, obj):
+        None
+
+    def end(self, obj):
+        None
+
+
+class Test(object):
+    def __init__(self, reporter, result, id, description=None, directive=None,
+                 comment=None):
+        self.reporter = reporter
+        self.result = result
+        if directive:
+            self.result = directive.lower()
+        if id:
+            self.id = int(id)
+        else:
+            self.id = None
+        if description:
+            self.description = description
+        else:
+            self.description = ""
+        if comment:
+            self.comment = "# " + comment
+        else:
+            self.comment = ""
+        self.yaml = None
+        self._yaml_buffer = None
+        self.diagnostics = []
+
+        self.reporter.start(self)
+
+    def end(self):
+        if not self.yaml:
+            self.yaml = yamlish.load(self._yaml_buffer)
+            self.reporter.end(self)
+
+
+class Suite(object):
+    def __init__(self, reporter, start, end, explanation):
+        self.reporter = reporter
+        self.tests = []
+        self.name = explanation
+        self.tests_planned = int(end)
+
+        self.__tests_counter = 0
+        self.__tests_base = 0
+
+        self.reporter.start(self)
+
+    def newTest(self, args):
+        try:
+            self.tests[-1].end()
+        except IndexError:
+            None
+
+        if 'id' not in args or not args['id']:
+            args['id'] = self.__tests_counter
+        else:
+            args['id'] = int(args['id']) + self.__tests_base
+
+        if args['id'] < self.__tests_counter:
+            print("error: bad test id %d, fixing it" % (args['id']))
+            args['id'] = self.__tests_counter
+        # according to TAP13 specs, missing tests must be handled as 'not ok'
+        # here we add the missing tests in sequence
+        while args['id'] > (self.__tests_counter + 1):
+            comment = 'test %d not present' % self.__tests_counter
+            self.tests.append(Test(self.reporter, 'not ok',
+                                   self.__tests_counter, comment=comment))
+            self.__tests_counter += 1
+
+        if args['id'] == self.__tests_counter:
+            if args['directive']:
+                self.test().result = args['directive'].lower()
+            else:
+                self.test().result = args['result']
+            self.reporter.start(self.test())
+        else:
+            self.tests.append(Test(self.reporter, **args))
+            self.__tests_counter += 1
+
+    def test(self):
+        return self.tests[-1]
+
+    def end(self, name, planned):
+        if name == self.name:
+            self.tests_planned += int(planned)
+            self.__tests_base = self.__tests_counter
+            return False
+        try:
+            self.test().end()
+        except IndexError:
+            None
+        if len(self.tests) != self.tests_planned:
+            for i in range(len(self.tests), self.tests_planned):
+                self.tests.append(Test(self.reporter, 'not ok', i+1,
+                                       comment='test not present'))
+        return True
+
+
+class Run(object):
+
+    def __init__(self, reporter):
+        self.reporter = reporter
+        self.suites = []
+
+    def suite(self):
+        return self.suites[-1]
+
+    def test(self):
+        return self.suites[-1].tests[-1]
+
+    def newSuite(self, args):
+        new = False
+        try:
+            if self.suite().end(args['explanation'], args['end']):
+                new = True
+        except IndexError:
+            new = True
+        if new:
+            self.suites.append(Suite(self.reporter, **args))
+
+    def newTest(self, args):
+        self.suite().newTest(args)
+
+
+class Parser(object):
+    RE_PLAN = re.compile(r"^\s*(?P<start>\d+)\.\.(?P<end>\d+)\s*(#\s*(?P<explanation>.*))?\s*$")
+    RE_TEST_LINE = re.compile(r"^\s*(?P<result>(not\s+)?ok|[*]+)\s*(?P<id>\d+)?\s*(?P<description>[^#]+)?\s*(#\s*(?P<directive>TODO|SKIP)?\s*(?P<comment>.+)?)?\s*$",  re.IGNORECASE)
+    RE_EXPLANATION = re.compile(r"^\s*#\s*(?P<explanation>.+)?\s*$")
+    RE_YAMLISH_START = re.compile(r"^\s*---.*$")
+    RE_YAMLISH_END = re.compile(r"^\s*\.\.\.\s*$")
+
+    def __init__(self, reporter):
+        self.seek_test = False
+        self.in_test = False
+        self.in_yaml = False
+        self.run = Run(reporter)
+
+    def parse(self, source):
+        # to avoid input buffering
+        while True:
+            line = source.readline()
+            if not line:
+                break
+
+            if self.in_yaml:
+                if Parser.RE_YAMLISH_END.match(line):
+                    self.run.test()._yaml_buffer.append(line.strip())
+                    self.in_yaml = False
+                else:
+                    self.run.test()._yaml_buffer.append(line.rstrip())
+                continue
+
+            line = line.strip()
+
+            if self.in_test:
+                if Parser.RE_EXPLANATION.match(line):
+                    self.run.test().diagnostics.append(line)
+                    continue
+                if Parser.RE_YAMLISH_START.match(line):
+                    self.run.test()._yaml_buffer = [line.strip()]
+                    self.in_yaml = True
+                    continue
+
+            m = Parser.RE_PLAN.match(line)
+            if m:
+                self.seek_test = True
+                args = m.groupdict()
+                self.run.newSuite(args)
+                continue
+
+            if self.seek_test:
+                m = Parser.RE_TEST_LINE.match(line)
+                if m:
+                    args = m.groupdict()
+                    self.run.newTest(args)
+                    self.in_test = True
+                    continue
+
+            print(line)
+        try:
+            self.run.suite().end(None, 0)
+        except IndexError:
+            None
diff --git a/tools/lkl/tests/test.c b/tools/lkl/tests/test.c
new file mode 100644
index 000000000000..3e334d106c48
--- /dev/null
+++ b/tools/lkl/tests/test.c
@@ -0,0 +1,126 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <stdarg.h>
+#include <time.h>
+
+#include "test.h"
+
+/* circular log buffer */
+
+static char log_buf[0x10000];
+static char *head = log_buf, *tail = log_buf;
+
+static inline void advance(char **ptr)
+{
+	if ((unsigned int)(*ptr - log_buf) >= sizeof(log_buf))
+		*ptr = log_buf;
+	else
+		*ptr = *ptr + 1;
+}
+
+static void log_char(char c)
+{
+	*tail = c;
+	advance(&tail);
+	if (tail == head)
+		advance(&head);
+}
+
+static void print_log(void)
+{
+	char last;
+
+	printf(" log: |\n");
+	last = '\n';
+	while (head != tail) {
+		if (last == '\n')
+			printf("  ");
+		last = *head;
+		putchar(last);
+		advance(&head);
+	}
+	if (last != '\n')
+		putchar('\n');
+}
+
+int lkl_test_run(const struct lkl_test *tests, int nr, const char *fmt, ...)
+{
+	int i, ret, status = TEST_SUCCESS;
+	clock_t start, stop;
+	char name[1024];
+	va_list args;
+
+	va_start(args, fmt);
+	vsnprintf(name, sizeof(name), fmt, args);
+	va_end(args);
+
+	printf("1..%d # %s\n", nr, name);
+	for (i = 1; i <= nr; i++) {
+		const struct lkl_test *t = &tests[i-1];
+		unsigned long delta_us;
+
+		printf("* %d %s\n", i, t->name);
+		fflush(stdout);
+
+		start = clock();
+
+		ret = t->fn(t->arg1, t->arg2, t->arg3);
+
+		stop = clock();
+
+		switch (ret) {
+		case TEST_SUCCESS:
+			printf("ok %d %s\n", i, t->name);
+			break;
+		case TEST_SKIP:
+			printf("ok %d %s # SKIP\n", i, t->name);
+			break;
+		case TEST_BAILOUT:
+			status = TEST_BAILOUT;
+			/* fall through */
+		case TEST_FAILURE:
+		default:
+			if (status != TEST_BAILOUT)
+				status = TEST_FAILURE;
+			printf("not ok %d %s\n", i, t->name);
+		}
+
+		printf(" ---\n");
+		delta_us = (stop - start) * 1000000 / CLOCKS_PER_SEC;
+		printf(" time_us: %ld\n", delta_us);
+		print_log();
+		printf(" ...\n");
+
+		if (status == TEST_BAILOUT) {
+			printf("Bail out!\n");
+			return TEST_FAILURE;
+		}
+
+		fflush(stdout);
+	}
+
+	return status;
+}
+
+
+void lkl_test_log(const char *str, int len)
+{
+	while (len--)
+		log_char(*(str++));
+}
+
+int lkl_test_logf(const char *fmt, ...)
+{
+	char tmp[1024], *c;
+	va_list args;
+	unsigned int n;
+
+	va_start(args, fmt);
+	n = vsnprintf(tmp, sizeof(tmp), fmt, args);
+	va_end(args);
+
+	for (c = tmp; *c != 0; c++)
+		log_char(*c);
+
+	return n > sizeof(tmp) ? sizeof(tmp) : n;
+}
diff --git a/tools/lkl/tests/test.h b/tools/lkl/tests/test.h
new file mode 100644
index 000000000000..f63ad6d419cb
--- /dev/null
+++ b/tools/lkl/tests/test.h
@@ -0,0 +1,72 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_TEST_H
+#define _LKL_TEST_H
+
+#define TEST_SUCCESS	0
+#define TEST_FAILURE	1
+#define TEST_SKIP	2
+#define TEST_TODO	3
+#define TEST_BAILOUT	4
+
+struct lkl_test {
+	const char *name;
+	int (*fn)();
+	void *arg1, *arg2, *arg3;
+};
+
+/**
+ * Simple wrapper to initialize a test entry.
+ * @name - test name, it assume test function is named test_@name
+ * @vargs - arguments to be passed to the function
+ */
+#define LKL_TEST(name, ...) { #name, lkl_test_##name, __VA_ARGS__ }
+
+/**
+ * lkl_test_run - run a test suite
+ *
+ * @tests - the list of tests to run
+ * @nr - number of tests
+ * @fmt - format string to be used for suite name
+ */
+int lkl_test_run(const struct lkl_test *tests, int nr, const char *fmt, ...);
+
+/**
+ * lkl_test_log - store a string in the test log buffer
+ * @str - the string to log (can be non-NULL terminated)
+ * @len - the string length
+ */
+void lkl_test_log(const char *str, int len);
+
+/**
+ * lkl_test_logf - printf like function to store into the test log buffer
+ * @fmt - printf format string
+ * @vargs - arguments to the format string
+ */
+int lkl_test_logf(const char *fmt, ...) __attribute__((format(printf, 1, 2)));
+
+/**
+ * LKL_TEST_CALL - create a test function as for a LKL call
+ *
+ * The test function will be named lkl_test_@name and will return
+ * TEST_SUCCESS if the called functions returns @expect. Otherwise
+ * will return TEST_FAILUIRE.
+ *
+ * @name - test name; must be unique because it is part of the the
+ * test function; the test function will be named
+ * @call - function to call
+ * @expect - expected return value for success
+ * @args - arguments to pass to the LKL call
+ */
+#define LKL_TEST_CALL(name, call, expect, ...)				\
+	static int lkl_test_##name(void)				\
+	{								\
+		long ret;						\
+									\
+		ret = call(__VA_ARGS__);				\
+		lkl_test_logf("%s(%s) = %ld %s\n", #call, #__VA_ARGS__, \
+			ret, ret < 0 ? lkl_strerror(ret) : "");		\
+		return (ret == expect) ? TEST_SUCCESS : TEST_FAILURE;	\
+	}
+
+
+#endif /* _LKL_TEST_H */
diff --git a/tools/lkl/tests/test.sh b/tools/lkl/tests/test.sh
new file mode 100644
index 000000000000..f4d652b9fbe8
--- /dev/null
+++ b/tools/lkl/tests/test.sh
@@ -0,0 +1,182 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+basedir=$(cd $script_dir/..; pwd)
+base_objdir=$(cd ${OUTPUT}/; pwd)
+source ${script_dir}/autoconf.sh
+
+TEST_SUCCESS=0
+TEST_FAILURE=1
+TEST_SKIP=113
+TEST_TODO=114
+TEST_BAILOUT=115
+
+print_log()
+{
+    echo " log: |"
+    while read line; do
+        echo "  $line"
+    done < $1
+}
+
+export_vars()
+{
+    if [ -z "$var_file" ]; then
+        return
+    fi
+
+    for i in $@; do
+        echo "$i=${!i}" >> $var_file
+    done
+}
+
+lkl_test_run()
+{
+    log_file=$(mktemp)
+    export var_file=$(mktemp)
+
+    tid=$1 && shift && tname=$@
+
+    echo "* $tid $tname"
+
+    start=$(date '+%s%9N')
+    # run in a separate shell to avoid -e terminating us
+    $@ 2>&1 | strings >$log_file
+    exit=${PIPESTATUS[0]}
+    stop=$(date '+%s%9N')
+
+    case $exit in
+    $TEST_SUCCESS)
+        echo "ok $tid $tname"
+        ;;
+    $TEST_SKIP)
+        echo "ok $tid $tname # SKIP"
+        ;;
+    $TEST_BAILOUT)
+        echo "not ok $tid $tname"
+        echo "Bail out!"
+        ;;
+    $TEST_FAILURE|*)
+        echo "not ok $tid $tname"
+        ;;
+    esac
+
+    delta=$(((stop-start)/1000))
+
+    echo " ---"
+    echo " time_us: $delta"
+    print_log $log_file
+    echo -e " ..."
+
+    rm $log_file
+    . $var_file
+    rm $var_file
+
+    return $exit
+}
+
+lkl_test_plan()
+{
+    echo "1..$1 # $2"
+    export suite_name="${2// /\-}"
+}
+
+lkl_test_exec()
+{
+    local SUDO=""
+    local WRAPPER=""
+
+    if [ "$1" = "sudo" ]; then
+        SUDO=sudo
+        shift
+    fi
+
+    local file=$1
+    shift
+
+    if [ -n "$LKL_HOST_CONFIG_NT" ]; then
+        file=$file.exe
+    fi
+
+    file=${OUTPUT}/tests/$(basename $file)
+
+    if file $file | grep ARM; then
+        WRAPPER="qemu-arm-static"
+    elif file $file | grep "FreeBSD" ; then
+        ssh_copy "$file" $BSD_WDIR
+        if [ -n "$SUDO" ]; then
+            SUDO=""
+        fi
+        WRAPPER="$MYSSH $SU"
+        # ssh will mess up with pipes ('|') so, escape the pipe char.
+        args="${@//\|/\\\|}"
+        set - $BSD_WDIR/$(basename $file) $args
+        file=""
+    elif [ -n "$GDB" ]; then
+        WRAPPER="gdb"
+        args="$@"
+        set - -ex "run $args" -ex quit $file
+        file=""
+    elif [ -n "$VALGRIND" ]; then
+        WRAPPER="valgrind --suppressions=$script_dir/valgrind.supp \
+                  --leak-check=full --show-leak-kinds=all --xml=yes \
+                  --xml-file=valgrind-$suite_name.xml"
+    fi
+
+    $SUDO $WRAPPER $file "$@"
+}
+
+lkl_test_cmd()
+{
+    local WRAPPER=""
+
+    if [ -z "$QUIET" ]; then
+        SHOPTS="-x"
+    fi
+
+    if [ -n "$LKL_HOST_CONFIG_BSD" ]; then
+        WRAPPER="$MYSSH $SU"
+    fi
+
+    echo "$@" | $WRAPPER sh $SHOPTS
+}
+
+# XXX: $MYSSH and $MYSCP are defined in a circleci docker image.
+# see the definitions in lkl/lkl-docker:circleci/freebsd11/Dockerfile
+ssh_push()
+{
+    while [ -n "$1" ]; do
+        if [[ "$1" = *.sh ]]; then
+            type="script"
+        else
+            type="file"
+        fi
+
+        dir=$(dirname $1)
+        $MYSSH mkdir -p $BSD_WDIR/$dir
+
+        $MYSCP -P 7722 -r $basedir/$1 root@localhost:$BSD_WDIR/$dir
+        if [ "$type" = "script" ]; then
+            $MYSSH chmod a+x $BSD_WDIR/$1
+        fi
+
+        shift
+    done
+}
+
+ssh_copy()
+{
+    $MYSCP -P 7722 -r $1 root@localhost:$2
+}
+
+lkl_test_bsd_cleanup()
+{
+    $MYSSH rm -rf $BSD_WDIR
+}
+
+if [ -n "$LKL_HOST_CONFIG_BSD" ]; then
+    trap lkl_test_bsd_cleanup EXIT
+    export BSD_WDIR=/root/lkl
+    $MYSSH mkdir -p $BSD_WDIR
+fi
diff --git a/tools/lkl/tests/valgrind.supp b/tools/lkl/tests/valgrind.supp
new file mode 100644
index 000000000000..2a0e270f2292
--- /dev/null
+++ b/tools/lkl/tests/valgrind.supp
@@ -0,0 +1,105 @@
+{
+   <unfinished timer 1>
+   Memcheck:Leak
+   match-leak-kinds: possible
+   ...
+   fun:pthread_create@@GLIBC_2.2.5
+   fun:__start_helper_thread
+   fun:__pthread_once_slow
+   fun:timer_create@@GLIBC_2.3.3
+   fun:timer_alloc
+   fun:clockevent_set_state_oneshot
+   ...
+   fun:__clockevents_switch_state
+   fun:clockevents_switch_state
+   fun:tick_setup_periodic
+   ...
+}
+
+{
+   <unfinished timer 2>
+   Memcheck:Leak
+   match-leak-kinds: possible
+   ...
+   fun:allocate_stack
+   fun:pthread_create@@GLIBC_2.2.5
+   fun:timer_helper_thread
+   fun:start_thread
+   ...
+}
+
+{
+   <pid1 kernel thread>
+   Memcheck:Leak
+   match-leak-kinds: possible
+   ...
+   fun:thread_create
+   fun:copy_thread
+   fun:copy_thread_tls
+   ...
+   fun:rest_init
+   ...
+   fun:start_kernel
+   ...
+}
+
+{
+   <xfs uninitialized buf error: delete this once upstream is fixed>
+   Memcheck:Value8
+   fun:crc32_body
+   fun:crc32_le_generic
+   fun:__crc32c_le
+   fun:chksum_update
+   fun:crypto_shash_update
+   fun:crc32c
+   fun:xlog_cksum
+}
+
+{
+   <xfs pwrite64 issue: delete this once upstream is fixed>
+   Memcheck:Param
+   pwrite64(buf)
+   ...
+   fun:blk_request
+   fun:blk_enqueue
+   fun:virtio_process_one
+   fun:virtio_process_queue
+   fun:virtio_write
+   fun:__raw_writel
+   fun:writel
+   fun:vm_notify
+   fun:virtqueue_notify
+   fun:virtio_queue_rq
+   fun:blk_mq_dispatch_rq_list
+   fun:blk_mq_sched_dispatch_requests
+}
+
+{
+   <virtio_net_pipe xmits>
+   Memcheck:Param
+   writev(vector[...])
+   ...
+   fun:fd_net_tx
+   fun:net_enqueue
+   fun:virtio_process_one
+   fun:virtio_process_queue
+   fun:virtio_write
+   fun:__raw_writel
+   fun:writel
+   fun:vm_notify
+   ...
+   fun:netdev_start_xmit
+   ...
+}
+
+{
+   <ubd_kern.c>
+   Memcheck:Param
+   read(buf)
+   ...
+   fun:read
+   fun:os_read_file
+   fun:bulk_req_safe_read.constprop.11
+   fun:io_thread
+   ...
+}
diff --git a/tools/lkl/tests/valgrind2xunit.py b/tools/lkl/tests/valgrind2xunit.py
new file mode 100755
index 000000000000..ab7c12b83377
--- /dev/null
+++ b/tools/lkl/tests/valgrind2xunit.py
@@ -0,0 +1,69 @@
+#!/usr/bin/env python
+# SPDX-License-Identifier: GPL-2.0
+
+##
+## Downloader from
+## http://humdi.net/wiki/tips/valgrind-to-xunit-xml-converter
+##
+
+import xml.etree.ElementTree as ET
+import sys
+import os
+
+fname = sys.argv[1]
+if fname is None:
+    fname = 'valgrind.xml'
+
+doc = ET.parse(fname)
+errors = doc.findall('.//error')
+
+out = open(os.path.splitext(os.path.basename(fname))[0]+'_xunit.xml',"w")
+out.write('<?xml version="1.0" encoding="UTF-8"?>\n')
+out.write('<testsuite name="valgrind" tests="'+str(len(errors))+'" errors="0" failures="'+str(len(errors))+'" skip="0">\n')
+errorcount=0
+for error in errors:
+    errorcount=errorcount+1
+
+    kind = error.find('kind')
+    what = error.find('what')
+    if  what == None:
+        what = error.find('xwhat/text')
+
+    stack = error.find('stack')
+    frames = stack.findall('frame')
+
+    for frame in frames:
+        fi = frame.find('file')
+        li = frame.find('line')
+        if fi != None and li != None:
+            break
+
+    if fi != None and li != None:
+        out.write('    <testcase classname="ValgrindMemoryCheck" name="Memory check '+str(errorcount)+' ('+kind.text+', '+fi.text+':'+li.text+')" time="0">\n')
+    else:
+        out.write('    <testcase classname="ValgrindMemoryCheck" name="Memory check '+str(errorcount)+' ('+kind.text+')" time="0">\n')
+    out.write('        <error type="'+kind.text+'">\n')
+    out.write('  '+what.text+'\n\n')
+
+    for frame in frames:
+        ip = frame.find('ip')
+        fn = frame.find('fn')
+        fi = frame.find('file')
+        li = frame.find('line')
+
+        if fn is None:
+            bodytext = '(unresolved symbol)'
+        else:
+            bodytext = fn.text
+        bodytext = bodytext.replace("&","&amp;")
+        bodytext = bodytext.replace("<","&lt;")
+        bodytext = bodytext.replace(">","&gt;")
+        if fi != None and li != None:
+            out.write('  '+ip.text+': '+bodytext+' ('+fi.text+':'+li.text+')\n')
+        else:
+            out.write('  '+ip.text+': '+bodytext+'\n')
+
+    out.write('        </error>\n')
+    out.write('    </testcase>\n')
+out.write('</testsuite>\n')
+out.close()
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v3 19/26] lkl tools: cptofs that reads/writes to/from a filesystem image
  2020-02-05  7:30 [RFC v3 00/26] Unifying LKL into UML Hajime Tazaki
@ 2020-02-05  7:30   ` Hajime Tazaki
       [not found] ` <cover.1580882335.git.thehajime-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
                     ` (25 subsequent siblings)
  26 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05  7:30 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Conrad Meyer, Dan Peebles, Petros Angelatos, Tuomas Tynkkynen,
	Yuriy Taraday, Hajime Tazaki

From: Octavian Purdila <tavi.purdila@gmail.com>

A tool to read/write to/from a filesystem image without mounting the
file to host filesystem.  Thus there is no root privilege to modify the
contents.

Cc: Conrad Meyer <cem@FreeBSD.org>
Cc: Dan Peebles <pumpkin@me.com>
Cc: Petros Angelatos <petrosagg@gmail.com>
Cc: Tuomas Tynkkynen <tuomas.tynkkynen@iki.fi>
Cc: Yuriy Taraday <yorik.sar@gmail.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/lkl/.gitignore |   2 +
 tools/lkl/Build      |   2 +
 tools/lkl/Makefile   |   3 +
 tools/lkl/Targets    |   4 +
 tools/lkl/cptofs.c   | 636 +++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 647 insertions(+)
 create mode 100644 tools/lkl/cptofs.c

diff --git a/tools/lkl/.gitignore b/tools/lkl/.gitignore
index 4e08254dbd46..1a8210f4d9c4 100644
--- a/tools/lkl/.gitignore
+++ b/tools/lkl/.gitignore
@@ -7,3 +7,5 @@ tests/disk
 tests/boot
 tests/valgrind*.xml
 *.pyc
+cptofs
+cpfromfs
diff --git a/tools/lkl/Build b/tools/lkl/Build
index e69de29bb2d1..a9d12c5ca260 100644
--- a/tools/lkl/Build
+++ b/tools/lkl/Build
@@ -0,0 +1,2 @@
+cptofs-$(LKL_HOST_CONFIG_ARCHIVE) += cptofs.o
+
diff --git a/tools/lkl/Makefile b/tools/lkl/Makefile
index 0abccb9d86b6..03750aab98d8 100644
--- a/tools/lkl/Makefile
+++ b/tools/lkl/Makefile
@@ -86,6 +86,9 @@ $(OUTPUT)%$(EXESUF): $(OUTPUT)%-in.o $(OUTPUT)liblkl.a
 $(OUTPUT)%-in.o: $(OUTPUT)lib/lkl.o FORCE
 	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
 
+$(OUTPUT)cpfromfs$(EXESUF): $(OUTPUT)cptofs$(EXESUF)
+	$(Q)if ! [ -e $@ ]; then ln -s $< $@; fi
+
 
 clean:
 	$(call QUIET_CLEAN, objects)find $(OUTPUT) -name '*.o' -delete -o -name '\.*.cmd'\
diff --git a/tools/lkl/Targets b/tools/lkl/Targets
index a9f74c3cc8fb..17de965f92b3 100644
--- a/tools/lkl/Targets
+++ b/tools/lkl/Targets
@@ -4,3 +4,7 @@ progs-y += tests/boot
 progs-y += tests/disk
 progs-y += tests/net-test
 
+progs-$(LKL_HOST_CONFIG_ARCHIVE) += cptofs
+progs-$(LKL_HOST_CONFIG_ARCHIVE) += cpfromfs
+LDLIBS_cptofs-y := -larchive
+LDLIBS_cptofs-$(LKL_HOST_CONFIG_NEEDS_LARGP) += -largp
diff --git a/tools/lkl/cptofs.c b/tools/lkl/cptofs.c
new file mode 100644
index 000000000000..fb34f570fa1e
--- /dev/null
+++ b/tools/lkl/cptofs.c
@@ -0,0 +1,636 @@
+// SPDX-License-Identifier: GPL-2.0
+#ifdef __FreeBSD__
+#include <sys/param.h>
+#endif
+
+#include <stdio.h>
+#include <time.h>
+#include <argp.h>
+#include <unistd.h>
+#include <string.h>
+#include <stdlib.h>
+#include <libgen.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <fnmatch.h>
+#include <dirent.h>
+#include <lkl.h>
+#include <lkl_host.h>
+
+static const char doc_cptofs[] = "Copy files to a filesystem image";
+static const char doc_cpfromfs[] = "Copy files from a filesystem image";
+static const char args_doc_cptofs[] = "-t fstype -i fsimage path... fs_path";
+static const char args_doc_cpfromfs[] = "-t fstype -i fsimage fs_path... path";
+
+static struct argp_option options[] = {
+	{"enable-printk", 'p', 0, 0, "show Linux printks"},
+	{"partition", 'P', "int", 0, "partition number"},
+	{"filesystem-type", 't', "string", 0,
+	 "select filesystem type - mandatory"},
+	{"filesystem-image", 'i', "string", 0,
+	 "path to the filesystem image - mandatory"},
+	{"selinux", 's', "string", 0, "selinux attributes for destination"},
+	{0},
+};
+
+static struct cl_args {
+	int printk;
+	int part;
+	const char *fsimg_type;
+	const char *fsimg_path;
+	int npaths;
+	char **paths;
+	const char *selinux;
+} cla;
+
+static int cptofs;
+
+static error_t parse_opt(int key, char *arg, struct argp_state *state)
+{
+	struct cl_args *cla = state->input;
+
+	switch (key) {
+	case 'p':
+		cla->printk = 1;
+		break;
+	case 'P':
+		cla->part = atoi(arg);
+		break;
+	case 't':
+		cla->fsimg_type = arg;
+		break;
+	case 'i':
+		cla->fsimg_path = arg;
+		break;
+	case 's':
+		cla->selinux = arg;
+		break;
+	case ARGP_KEY_ARG:
+		// Capture all remaining arguments in our paths array and stop
+		// parsing here. We treat the last one as the destination and
+		// everything before it as sources, just like cp does.
+		cla->paths = &state->argv[state->next - 1];
+		cla->npaths = state->argc - state->next + 1;
+		state->next = state->argc;
+		break;
+	default:
+		return ARGP_ERR_UNKNOWN;
+	}
+
+	return 0;
+}
+
+static struct argp argp_cptofs = {
+	.options = options,
+	.parser = parse_opt,
+	.args_doc = args_doc_cptofs,
+	.doc = doc_cptofs,
+};
+
+static struct argp argp_cpfromfs = {
+	.options = options,
+	.parser = parse_opt,
+	.args_doc = args_doc_cpfromfs,
+	.doc = doc_cpfromfs,
+};
+
+static int searchdir(const char *fs_path, const char *path, const char *match);
+
+static int open_src(const char *path)
+{
+	int fd;
+
+	if (cptofs)
+		fd = open(path, O_RDONLY, 0);
+	else
+		fd = lkl_sys_open(path, LKL_O_RDONLY, 0);
+
+	if (fd < 0)
+		fprintf(stderr, "unable to open file %s for reading: %s\n",
+			path, cptofs ? strerror(errno) : lkl_strerror(fd));
+
+	return fd;
+}
+
+static int open_dst(const char *path, int mode)
+{
+	int fd;
+
+	if (cptofs)
+		fd = lkl_sys_open(path, LKL_O_RDWR | LKL_O_TRUNC | LKL_O_CREAT,
+				  mode);
+	else
+		fd = open(path, O_RDWR | O_TRUNC | O_CREAT, mode);
+
+	if (fd < 0)
+		fprintf(stderr, "unable to open file %s for writing: %s\n",
+			path, cptofs ? lkl_strerror(fd) : strerror(errno));
+
+	if (cla.selinux && cptofs) {
+		int ret = lkl_sys_fsetxattr(fd, "security.selinux", cla.selinux,
+					    strlen(cla.selinux), 0);
+		if (ret)
+			fprintf(stderr,
+				"unable to set selinux attribute on %s: %s\n",
+				path, lkl_strerror(ret));
+	}
+
+	return fd;
+}
+
+static int read_src(int fd, char *buf, int len)
+{
+	int ret;
+
+	if (cptofs)
+		ret = read(fd, buf, len);
+	else
+		ret = lkl_sys_read(fd, buf, len);
+
+	if (ret < 0)
+		fprintf(stderr, "error reading file: %s\n",
+			cptofs ? strerror(errno) : lkl_strerror(ret));
+
+	return ret;
+}
+
+static int write_dst(int fd, char *buf, int len)
+{
+	int ret;
+
+	if (cptofs)
+		ret = lkl_sys_write(fd, buf, len);
+	else
+		ret = write(fd, buf, len);
+
+	if (ret < 0)
+		fprintf(stderr, "error writing file: %s\n",
+			cptofs ? lkl_strerror(ret) : strerror(errno));
+
+	return ret;
+}
+
+static void close_src(int fd)
+{
+	if (cptofs)
+		close(fd);
+	else
+		lkl_sys_close(fd);
+}
+
+static void close_dst(int fd)
+{
+	if (cptofs)
+		lkl_sys_close(fd);
+	else
+		close(fd);
+}
+
+static int copy_file(const char *src, const char *dst, int mode)
+{
+	long len, to_write, wrote;
+	char buf[4096], *ptr;
+	int ret = 0;
+	int fd_src, fd_dst;
+
+	fd_src = open_src(src);
+	if (fd_src < 0)
+		return fd_src;
+
+	fd_dst = open_dst(dst, mode);
+	if (fd_dst < 0)
+		return fd_dst;
+
+	do {
+		len = read_src(fd_src, buf, sizeof(buf));
+
+		if (len > 0) {
+			ptr = buf;
+			to_write = len;
+			do {
+				wrote = write_dst(fd_dst, ptr, to_write);
+
+				if (wrote < 0) {
+					ret = wrote;
+					goto out;
+				}
+
+				to_write -= wrote;
+				ptr += len;
+
+			} while (to_write > 0);
+		}
+
+		if (len < 0)
+			ret = len;
+
+	} while (len > 0);
+
+out:
+	close_src(fd_src);
+	close_dst(fd_dst);
+
+	return ret;
+}
+
+static int stat_src(const char *path, unsigned int *type, unsigned int *mode,
+		    long long *size, struct lkl_timespec *mtime,
+		    struct lkl_timespec *atime)
+{
+	struct stat stat;
+	struct lkl_stat lkl_stat;
+	int ret;
+
+	if (cptofs) {
+		ret = lstat(path, &stat);
+		if (type)
+			*type = stat.st_mode & S_IFMT;
+		if (mode)
+			*mode = stat.st_mode & ~S_IFMT;
+		if (size)
+			*size = stat.st_size;
+		if (mtime) {
+			mtime->tv_sec = stat.st_mtim.tv_sec;
+			mtime->tv_nsec = stat.st_mtim.tv_nsec;
+		}
+		if (atime) {
+			atime->tv_sec = stat.st_atim.tv_sec;
+			atime->tv_nsec = stat.st_atim.tv_nsec;
+		}
+	} else {
+		ret = lkl_sys_lstat(path, &lkl_stat);
+		if (type)
+			*type = lkl_stat.st_mode & S_IFMT;
+		if (mode)
+			*mode = lkl_stat.st_mode & ~S_IFMT;
+		if (size)
+			*size = lkl_stat.st_size;
+		if (mtime) {
+			mtime->tv_sec = lkl_stat.lkl_st_mtime;
+			mtime->tv_nsec = lkl_stat.st_mtime_nsec;
+		}
+		if (atime) {
+			atime->tv_sec = lkl_stat.lkl_st_atime;
+			atime->tv_nsec = lkl_stat.st_atime_nsec;
+		}
+	}
+
+	if (ret)
+		fprintf(stderr, "fsimg lstat(%s) error: %s\n",
+			path, cptofs ? strerror(errno) : lkl_strerror(ret));
+
+	return ret;
+}
+
+static int mkdir_dst(const char *path, unsigned int mode)
+{
+	int ret;
+
+	if (cptofs) {
+		ret = lkl_sys_mkdir(path, mode);
+		if (ret == -LKL_EEXIST)
+			ret = 0;
+	} else {
+		ret = mkdir(path, mode);
+		if (ret < 0 && errno == EEXIST)
+			ret = 0;
+	}
+
+	if (ret)
+		fprintf(stderr, "unable to create directory %s: %s\n",
+			path, cptofs ? strerror(errno) : lkl_strerror(ret));
+
+	return ret;
+}
+
+static int readlink_src(const char *src, char *out, int outsize)
+{
+	int ret;
+
+	if (cptofs)
+		ret = readlink(src, out, outsize);
+	else
+		ret = lkl_sys_readlink(src, out, outsize);
+
+	if (ret < 0)
+		fprintf(stderr, "unable to readlink '%s': %s\n", src,
+			cptofs ? strerror(errno) : lkl_strerror(ret));
+
+	return ret;
+}
+
+static int symlink_dst(const char *path, const char *target)
+{
+	int ret;
+
+	if (cptofs)
+		ret = lkl_sys_symlink(target, path);
+	else
+		ret = symlink(target, path);
+
+	if (ret)
+		fprintf(stderr, "unable to symlink '%s' with target '%s': %s\n",
+			path, target, cptofs ? lkl_strerror(ret) :
+			strerror(errno));
+
+	return ret;
+}
+
+static int copy_symlink(const char *src, const char *dst)
+{
+	int ret;
+	long long size, actual_size;
+	char *target = NULL;
+
+	ret = stat_src(src, NULL, NULL, &size, NULL, NULL);
+	if (ret) {
+		ret = -1;
+		goto out;
+	}
+
+	target = malloc(size + 1);
+	if (!target) {
+		fprintf(stderr, "Unable to allocate memory (%lld bytes)\n",
+			size + 1);
+		ret = -1;
+		goto out;
+	}
+
+	actual_size = readlink_src(src, target, size);
+	if (actual_size != size) {
+		fprintf(stderr,
+			"readlink(%s) bad size: got %lld, expected %lld\n",
+			src, actual_size, size);
+		ret = -1;
+		goto out;
+	}
+	target[size] = 0; // readlink doesn't append the trailing null byte
+
+	ret = symlink_dst(dst, target);
+	if (ret)
+		ret = -1;
+
+out:
+	if (target)
+		free(target);
+
+	return ret;
+}
+
+static int do_entry(const char *_src, const char *_dst, const char *name)
+{
+	char src[PATH_MAX], dst[PATH_MAX];
+	struct lkl_timespec mtime, atime;
+	unsigned int type, mode;
+	int ret;
+
+	snprintf(src, sizeof(src), "%s/%s", _src, name);
+	snprintf(dst, sizeof(dst), "%s/%s", _dst, name);
+
+	ret = stat_src(src, &type, &mode, NULL, &mtime, &atime);
+
+	switch (type) {
+	case S_IFREG:
+	{
+		ret = copy_file(src, dst, mode);
+		break;
+	}
+	case S_IFDIR:
+		ret = mkdir_dst(dst, mode);
+		if (ret)
+			break;
+		ret = searchdir(src, dst, NULL);
+		break;
+	case S_IFLNK:
+		ret = copy_symlink(src, dst);
+		break;
+	case S_IFSOCK:
+	case S_IFBLK:
+	case S_IFCHR:
+	case S_IFIFO:
+	default:
+		printf("skipping %s: unsupported entry type %d\n", src, type);
+	}
+
+	if (!ret) {
+		if (cptofs) {
+			struct lkl_timespec lkl_ts[] = { atime, mtime };
+
+			ret = lkl_sys_utimensat(-1, dst,
+						(struct __lkl__kernel_timespec
+						 *)lkl_ts,
+						LKL_AT_SYMLINK_NOFOLLOW);
+		} else {
+			struct timespec ts[] = {
+				{ .tv_sec = atime.tv_sec,
+				  .tv_nsec = atime.tv_nsec, },
+				{ .tv_sec = mtime.tv_sec,
+				  .tv_nsec = mtime.tv_nsec, },
+			};
+
+			ret = utimensat(-1, dst, ts, AT_SYMLINK_NOFOLLOW);
+		}
+	}
+
+	if (ret)
+		printf("error processing entry %s, aborting\n", src);
+
+	return ret;
+}
+
+static DIR *open_dir(const char *path)
+{
+	DIR *dir;
+	int err;
+
+	if (cptofs)
+		dir = opendir(path);
+	else
+		dir = (DIR *)lkl_opendir(path, &err);
+
+	if (!dir)
+		fprintf(stderr, "unable to open directory %s: %s\n",
+			path, cptofs ? strerror(errno) : lkl_strerror(err));
+	return dir;
+}
+
+static const char *read_dir(DIR *dir, const char *path)
+{
+	struct lkl_dir *lkl_dir = (struct lkl_dir *)dir;
+	const char *name = NULL;
+	const char *err = NULL;
+
+	if (cptofs) {
+		struct dirent *de = readdir(dir);
+
+		if (de)
+			name = de->d_name;
+	} else {
+		struct lkl_linux_dirent64 *de = lkl_readdir(lkl_dir);
+
+		if (de)
+			name = de->d_name;
+	}
+
+	if (!name) {
+		if (cptofs) {
+			if (errno)
+				err = strerror(errno);
+		} else {
+			if (lkl_errdir(lkl_dir))
+				err = lkl_strerror(lkl_errdir(lkl_dir));
+		}
+	}
+
+	if (err)
+		fprintf(stderr, "error while reading directory %s: %s\n",
+			path, err);
+	return name;
+}
+
+static void close_dir(DIR *dir)
+{
+	if (cptofs)
+		closedir(dir);
+	else
+		lkl_closedir((struct lkl_dir *)dir);
+}
+
+static int searchdir(const char *src, const char *dst, const char *match)
+{
+	DIR *dir;
+	const char *name;
+	int ret = 0;
+
+	dir = open_dir(src);
+	if (!dir)
+		return -1;
+
+	while ((name = read_dir(dir, src))) {
+		if (!strcmp(name, ".") || !strcmp(name, "..") ||
+		    (match && fnmatch(match, name, 0) != 0))
+			continue;
+
+		ret = do_entry(src, dst, name);
+		if (ret)
+			goto out;
+	}
+
+out:
+	close_dir(dir);
+
+	return ret;
+}
+
+static int match_root(const char *src)
+{
+	const char *c = src;
+
+	while (*c) {
+		switch (*c) {
+		case '.':
+			if (c > src && c[-1] == '.')
+				return 0;
+			break;
+		case '/':
+			break;
+		default:
+			return 0;
+		}
+		c++;
+	}
+
+	return 1;
+}
+
+int copy_one(const char *src, const char *mpoint, const char *dst)
+{
+	char *src_path_dir, *src_path_base;
+	char src_path[PATH_MAX], dst_path[PATH_MAX];
+
+	if (cptofs) {
+		snprintf(src_path, sizeof(src_path),  "%s", src);
+		snprintf(dst_path, sizeof(dst_path),  "%s/%s", mpoint, dst);
+	} else {
+		snprintf(src_path, sizeof(src_path),  "%s/%s", mpoint, src);
+		snprintf(dst_path, sizeof(dst_path),  "%s", dst);
+	}
+
+	if (match_root(src))
+		return searchdir(src_path, dst, NULL);
+
+	src_path_dir = dirname(strdup(src_path));
+	src_path_base = basename(strdup(src_path));
+
+	return searchdir(src_path_dir, dst_path, src_path_base);
+}
+
+int main(int argc, char **argv)
+{
+	struct lkl_disk disk;
+	long ret, umount_ret;
+	int i;
+	char mpoint[32];
+	unsigned int disk_id;
+
+	if (strstr(argv[0], "cptofs")) {
+		cptofs = 1;
+		ret = argp_parse(&argp_cptofs, argc, argv, 0, 0, &cla);
+	} else {
+		ret = argp_parse(&argp_cpfromfs, argc, argv, 0, 0, &cla);
+	}
+
+	if (ret < 0)
+		return -1;
+
+	if (!cla.printk)
+		lkl_host_ops.print = NULL;
+
+	disk.fd = open(cla.fsimg_path, cptofs ? O_RDWR : O_RDONLY);
+	if (disk.fd < 0) {
+		fprintf(stderr, "can't open fsimg %s: %s\n", cla.fsimg_path,
+			strerror(errno));
+		ret = 1;
+		goto out;
+	}
+
+	disk.ops = NULL;
+	disk.dev = (char *)cla.fsimg_path;
+
+	ret = lkl_disk_add(&disk);
+	if (ret < 0) {
+		fprintf(stderr, "can't add disk: %s\n", lkl_strerror(ret));
+		goto out_close;
+	}
+	disk_id = ret;
+
+	lkl_start_kernel(&lkl_host_ops, "mem=100M");
+
+	ret = lkl_mount_dev(disk_id, cla.part, cla.fsimg_type,
+			    cptofs ? 0 : LKL_MS_RDONLY,
+			    NULL, mpoint, sizeof(mpoint));
+	if (ret) {
+		fprintf(stderr, "can't mount disk: %s\n", lkl_strerror(ret));
+		goto out_close;
+	}
+
+	lkl_sys_umask(0);
+
+	for (i = 0; i < cla.npaths - 1; i++) {
+		ret = copy_one(cla.paths[i], mpoint, cla.paths[cla.npaths - 1]);
+		if (ret)
+			break;
+	}
+
+	umount_ret = lkl_umount_dev(disk_id, cla.part, 0, 1000);
+	if (ret == 0)
+		ret = umount_ret;
+
+out_close:
+	close(disk.fd);
+
+out:
+	lkl_sys_halt();
+
+	return ret;
+}
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v3 19/26] lkl tools: cptofs that reads/writes to/from a filesystem image
@ 2020-02-05  7:30   ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05  7:30 UTC (permalink / raw)
  To: linux-um
  Cc: linux-arch, Conrad Meyer, Octavian Purdila, Akira Moroo,
	Petros Angelatos, Dan Peebles, linux-kernel-library,
	Yuriy Taraday, Tuomas Tynkkynen, Hajime Tazaki

From: Octavian Purdila <tavi.purdila@gmail.com>

A tool to read/write to/from a filesystem image without mounting the
file to host filesystem.  Thus there is no root privilege to modify the
contents.

Cc: Conrad Meyer <cem@FreeBSD.org>
Cc: Dan Peebles <pumpkin@me.com>
Cc: Petros Angelatos <petrosagg@gmail.com>
Cc: Tuomas Tynkkynen <tuomas.tynkkynen@iki.fi>
Cc: Yuriy Taraday <yorik.sar@gmail.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/lkl/.gitignore |   2 +
 tools/lkl/Build      |   2 +
 tools/lkl/Makefile   |   3 +
 tools/lkl/Targets    |   4 +
 tools/lkl/cptofs.c   | 636 +++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 647 insertions(+)
 create mode 100644 tools/lkl/cptofs.c

diff --git a/tools/lkl/.gitignore b/tools/lkl/.gitignore
index 4e08254dbd46..1a8210f4d9c4 100644
--- a/tools/lkl/.gitignore
+++ b/tools/lkl/.gitignore
@@ -7,3 +7,5 @@ tests/disk
 tests/boot
 tests/valgrind*.xml
 *.pyc
+cptofs
+cpfromfs
diff --git a/tools/lkl/Build b/tools/lkl/Build
index e69de29bb2d1..a9d12c5ca260 100644
--- a/tools/lkl/Build
+++ b/tools/lkl/Build
@@ -0,0 +1,2 @@
+cptofs-$(LKL_HOST_CONFIG_ARCHIVE) += cptofs.o
+
diff --git a/tools/lkl/Makefile b/tools/lkl/Makefile
index 0abccb9d86b6..03750aab98d8 100644
--- a/tools/lkl/Makefile
+++ b/tools/lkl/Makefile
@@ -86,6 +86,9 @@ $(OUTPUT)%$(EXESUF): $(OUTPUT)%-in.o $(OUTPUT)liblkl.a
 $(OUTPUT)%-in.o: $(OUTPUT)lib/lkl.o FORCE
 	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
 
+$(OUTPUT)cpfromfs$(EXESUF): $(OUTPUT)cptofs$(EXESUF)
+	$(Q)if ! [ -e $@ ]; then ln -s $< $@; fi
+
 
 clean:
 	$(call QUIET_CLEAN, objects)find $(OUTPUT) -name '*.o' -delete -o -name '\.*.cmd'\
diff --git a/tools/lkl/Targets b/tools/lkl/Targets
index a9f74c3cc8fb..17de965f92b3 100644
--- a/tools/lkl/Targets
+++ b/tools/lkl/Targets
@@ -4,3 +4,7 @@ progs-y += tests/boot
 progs-y += tests/disk
 progs-y += tests/net-test
 
+progs-$(LKL_HOST_CONFIG_ARCHIVE) += cptofs
+progs-$(LKL_HOST_CONFIG_ARCHIVE) += cpfromfs
+LDLIBS_cptofs-y := -larchive
+LDLIBS_cptofs-$(LKL_HOST_CONFIG_NEEDS_LARGP) += -largp
diff --git a/tools/lkl/cptofs.c b/tools/lkl/cptofs.c
new file mode 100644
index 000000000000..fb34f570fa1e
--- /dev/null
+++ b/tools/lkl/cptofs.c
@@ -0,0 +1,636 @@
+// SPDX-License-Identifier: GPL-2.0
+#ifdef __FreeBSD__
+#include <sys/param.h>
+#endif
+
+#include <stdio.h>
+#include <time.h>
+#include <argp.h>
+#include <unistd.h>
+#include <string.h>
+#include <stdlib.h>
+#include <libgen.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <fnmatch.h>
+#include <dirent.h>
+#include <lkl.h>
+#include <lkl_host.h>
+
+static const char doc_cptofs[] = "Copy files to a filesystem image";
+static const char doc_cpfromfs[] = "Copy files from a filesystem image";
+static const char args_doc_cptofs[] = "-t fstype -i fsimage path... fs_path";
+static const char args_doc_cpfromfs[] = "-t fstype -i fsimage fs_path... path";
+
+static struct argp_option options[] = {
+	{"enable-printk", 'p', 0, 0, "show Linux printks"},
+	{"partition", 'P', "int", 0, "partition number"},
+	{"filesystem-type", 't', "string", 0,
+	 "select filesystem type - mandatory"},
+	{"filesystem-image", 'i', "string", 0,
+	 "path to the filesystem image - mandatory"},
+	{"selinux", 's', "string", 0, "selinux attributes for destination"},
+	{0},
+};
+
+static struct cl_args {
+	int printk;
+	int part;
+	const char *fsimg_type;
+	const char *fsimg_path;
+	int npaths;
+	char **paths;
+	const char *selinux;
+} cla;
+
+static int cptofs;
+
+static error_t parse_opt(int key, char *arg, struct argp_state *state)
+{
+	struct cl_args *cla = state->input;
+
+	switch (key) {
+	case 'p':
+		cla->printk = 1;
+		break;
+	case 'P':
+		cla->part = atoi(arg);
+		break;
+	case 't':
+		cla->fsimg_type = arg;
+		break;
+	case 'i':
+		cla->fsimg_path = arg;
+		break;
+	case 's':
+		cla->selinux = arg;
+		break;
+	case ARGP_KEY_ARG:
+		// Capture all remaining arguments in our paths array and stop
+		// parsing here. We treat the last one as the destination and
+		// everything before it as sources, just like cp does.
+		cla->paths = &state->argv[state->next - 1];
+		cla->npaths = state->argc - state->next + 1;
+		state->next = state->argc;
+		break;
+	default:
+		return ARGP_ERR_UNKNOWN;
+	}
+
+	return 0;
+}
+
+static struct argp argp_cptofs = {
+	.options = options,
+	.parser = parse_opt,
+	.args_doc = args_doc_cptofs,
+	.doc = doc_cptofs,
+};
+
+static struct argp argp_cpfromfs = {
+	.options = options,
+	.parser = parse_opt,
+	.args_doc = args_doc_cpfromfs,
+	.doc = doc_cpfromfs,
+};
+
+static int searchdir(const char *fs_path, const char *path, const char *match);
+
+static int open_src(const char *path)
+{
+	int fd;
+
+	if (cptofs)
+		fd = open(path, O_RDONLY, 0);
+	else
+		fd = lkl_sys_open(path, LKL_O_RDONLY, 0);
+
+	if (fd < 0)
+		fprintf(stderr, "unable to open file %s for reading: %s\n",
+			path, cptofs ? strerror(errno) : lkl_strerror(fd));
+
+	return fd;
+}
+
+static int open_dst(const char *path, int mode)
+{
+	int fd;
+
+	if (cptofs)
+		fd = lkl_sys_open(path, LKL_O_RDWR | LKL_O_TRUNC | LKL_O_CREAT,
+				  mode);
+	else
+		fd = open(path, O_RDWR | O_TRUNC | O_CREAT, mode);
+
+	if (fd < 0)
+		fprintf(stderr, "unable to open file %s for writing: %s\n",
+			path, cptofs ? lkl_strerror(fd) : strerror(errno));
+
+	if (cla.selinux && cptofs) {
+		int ret = lkl_sys_fsetxattr(fd, "security.selinux", cla.selinux,
+					    strlen(cla.selinux), 0);
+		if (ret)
+			fprintf(stderr,
+				"unable to set selinux attribute on %s: %s\n",
+				path, lkl_strerror(ret));
+	}
+
+	return fd;
+}
+
+static int read_src(int fd, char *buf, int len)
+{
+	int ret;
+
+	if (cptofs)
+		ret = read(fd, buf, len);
+	else
+		ret = lkl_sys_read(fd, buf, len);
+
+	if (ret < 0)
+		fprintf(stderr, "error reading file: %s\n",
+			cptofs ? strerror(errno) : lkl_strerror(ret));
+
+	return ret;
+}
+
+static int write_dst(int fd, char *buf, int len)
+{
+	int ret;
+
+	if (cptofs)
+		ret = lkl_sys_write(fd, buf, len);
+	else
+		ret = write(fd, buf, len);
+
+	if (ret < 0)
+		fprintf(stderr, "error writing file: %s\n",
+			cptofs ? lkl_strerror(ret) : strerror(errno));
+
+	return ret;
+}
+
+static void close_src(int fd)
+{
+	if (cptofs)
+		close(fd);
+	else
+		lkl_sys_close(fd);
+}
+
+static void close_dst(int fd)
+{
+	if (cptofs)
+		lkl_sys_close(fd);
+	else
+		close(fd);
+}
+
+static int copy_file(const char *src, const char *dst, int mode)
+{
+	long len, to_write, wrote;
+	char buf[4096], *ptr;
+	int ret = 0;
+	int fd_src, fd_dst;
+
+	fd_src = open_src(src);
+	if (fd_src < 0)
+		return fd_src;
+
+	fd_dst = open_dst(dst, mode);
+	if (fd_dst < 0)
+		return fd_dst;
+
+	do {
+		len = read_src(fd_src, buf, sizeof(buf));
+
+		if (len > 0) {
+			ptr = buf;
+			to_write = len;
+			do {
+				wrote = write_dst(fd_dst, ptr, to_write);
+
+				if (wrote < 0) {
+					ret = wrote;
+					goto out;
+				}
+
+				to_write -= wrote;
+				ptr += len;
+
+			} while (to_write > 0);
+		}
+
+		if (len < 0)
+			ret = len;
+
+	} while (len > 0);
+
+out:
+	close_src(fd_src);
+	close_dst(fd_dst);
+
+	return ret;
+}
+
+static int stat_src(const char *path, unsigned int *type, unsigned int *mode,
+		    long long *size, struct lkl_timespec *mtime,
+		    struct lkl_timespec *atime)
+{
+	struct stat stat;
+	struct lkl_stat lkl_stat;
+	int ret;
+
+	if (cptofs) {
+		ret = lstat(path, &stat);
+		if (type)
+			*type = stat.st_mode & S_IFMT;
+		if (mode)
+			*mode = stat.st_mode & ~S_IFMT;
+		if (size)
+			*size = stat.st_size;
+		if (mtime) {
+			mtime->tv_sec = stat.st_mtim.tv_sec;
+			mtime->tv_nsec = stat.st_mtim.tv_nsec;
+		}
+		if (atime) {
+			atime->tv_sec = stat.st_atim.tv_sec;
+			atime->tv_nsec = stat.st_atim.tv_nsec;
+		}
+	} else {
+		ret = lkl_sys_lstat(path, &lkl_stat);
+		if (type)
+			*type = lkl_stat.st_mode & S_IFMT;
+		if (mode)
+			*mode = lkl_stat.st_mode & ~S_IFMT;
+		if (size)
+			*size = lkl_stat.st_size;
+		if (mtime) {
+			mtime->tv_sec = lkl_stat.lkl_st_mtime;
+			mtime->tv_nsec = lkl_stat.st_mtime_nsec;
+		}
+		if (atime) {
+			atime->tv_sec = lkl_stat.lkl_st_atime;
+			atime->tv_nsec = lkl_stat.st_atime_nsec;
+		}
+	}
+
+	if (ret)
+		fprintf(stderr, "fsimg lstat(%s) error: %s\n",
+			path, cptofs ? strerror(errno) : lkl_strerror(ret));
+
+	return ret;
+}
+
+static int mkdir_dst(const char *path, unsigned int mode)
+{
+	int ret;
+
+	if (cptofs) {
+		ret = lkl_sys_mkdir(path, mode);
+		if (ret == -LKL_EEXIST)
+			ret = 0;
+	} else {
+		ret = mkdir(path, mode);
+		if (ret < 0 && errno == EEXIST)
+			ret = 0;
+	}
+
+	if (ret)
+		fprintf(stderr, "unable to create directory %s: %s\n",
+			path, cptofs ? strerror(errno) : lkl_strerror(ret));
+
+	return ret;
+}
+
+static int readlink_src(const char *src, char *out, int outsize)
+{
+	int ret;
+
+	if (cptofs)
+		ret = readlink(src, out, outsize);
+	else
+		ret = lkl_sys_readlink(src, out, outsize);
+
+	if (ret < 0)
+		fprintf(stderr, "unable to readlink '%s': %s\n", src,
+			cptofs ? strerror(errno) : lkl_strerror(ret));
+
+	return ret;
+}
+
+static int symlink_dst(const char *path, const char *target)
+{
+	int ret;
+
+	if (cptofs)
+		ret = lkl_sys_symlink(target, path);
+	else
+		ret = symlink(target, path);
+
+	if (ret)
+		fprintf(stderr, "unable to symlink '%s' with target '%s': %s\n",
+			path, target, cptofs ? lkl_strerror(ret) :
+			strerror(errno));
+
+	return ret;
+}
+
+static int copy_symlink(const char *src, const char *dst)
+{
+	int ret;
+	long long size, actual_size;
+	char *target = NULL;
+
+	ret = stat_src(src, NULL, NULL, &size, NULL, NULL);
+	if (ret) {
+		ret = -1;
+		goto out;
+	}
+
+	target = malloc(size + 1);
+	if (!target) {
+		fprintf(stderr, "Unable to allocate memory (%lld bytes)\n",
+			size + 1);
+		ret = -1;
+		goto out;
+	}
+
+	actual_size = readlink_src(src, target, size);
+	if (actual_size != size) {
+		fprintf(stderr,
+			"readlink(%s) bad size: got %lld, expected %lld\n",
+			src, actual_size, size);
+		ret = -1;
+		goto out;
+	}
+	target[size] = 0; // readlink doesn't append the trailing null byte
+
+	ret = symlink_dst(dst, target);
+	if (ret)
+		ret = -1;
+
+out:
+	if (target)
+		free(target);
+
+	return ret;
+}
+
+static int do_entry(const char *_src, const char *_dst, const char *name)
+{
+	char src[PATH_MAX], dst[PATH_MAX];
+	struct lkl_timespec mtime, atime;
+	unsigned int type, mode;
+	int ret;
+
+	snprintf(src, sizeof(src), "%s/%s", _src, name);
+	snprintf(dst, sizeof(dst), "%s/%s", _dst, name);
+
+	ret = stat_src(src, &type, &mode, NULL, &mtime, &atime);
+
+	switch (type) {
+	case S_IFREG:
+	{
+		ret = copy_file(src, dst, mode);
+		break;
+	}
+	case S_IFDIR:
+		ret = mkdir_dst(dst, mode);
+		if (ret)
+			break;
+		ret = searchdir(src, dst, NULL);
+		break;
+	case S_IFLNK:
+		ret = copy_symlink(src, dst);
+		break;
+	case S_IFSOCK:
+	case S_IFBLK:
+	case S_IFCHR:
+	case S_IFIFO:
+	default:
+		printf("skipping %s: unsupported entry type %d\n", src, type);
+	}
+
+	if (!ret) {
+		if (cptofs) {
+			struct lkl_timespec lkl_ts[] = { atime, mtime };
+
+			ret = lkl_sys_utimensat(-1, dst,
+						(struct __lkl__kernel_timespec
+						 *)lkl_ts,
+						LKL_AT_SYMLINK_NOFOLLOW);
+		} else {
+			struct timespec ts[] = {
+				{ .tv_sec = atime.tv_sec,
+				  .tv_nsec = atime.tv_nsec, },
+				{ .tv_sec = mtime.tv_sec,
+				  .tv_nsec = mtime.tv_nsec, },
+			};
+
+			ret = utimensat(-1, dst, ts, AT_SYMLINK_NOFOLLOW);
+		}
+	}
+
+	if (ret)
+		printf("error processing entry %s, aborting\n", src);
+
+	return ret;
+}
+
+static DIR *open_dir(const char *path)
+{
+	DIR *dir;
+	int err;
+
+	if (cptofs)
+		dir = opendir(path);
+	else
+		dir = (DIR *)lkl_opendir(path, &err);
+
+	if (!dir)
+		fprintf(stderr, "unable to open directory %s: %s\n",
+			path, cptofs ? strerror(errno) : lkl_strerror(err));
+	return dir;
+}
+
+static const char *read_dir(DIR *dir, const char *path)
+{
+	struct lkl_dir *lkl_dir = (struct lkl_dir *)dir;
+	const char *name = NULL;
+	const char *err = NULL;
+
+	if (cptofs) {
+		struct dirent *de = readdir(dir);
+
+		if (de)
+			name = de->d_name;
+	} else {
+		struct lkl_linux_dirent64 *de = lkl_readdir(lkl_dir);
+
+		if (de)
+			name = de->d_name;
+	}
+
+	if (!name) {
+		if (cptofs) {
+			if (errno)
+				err = strerror(errno);
+		} else {
+			if (lkl_errdir(lkl_dir))
+				err = lkl_strerror(lkl_errdir(lkl_dir));
+		}
+	}
+
+	if (err)
+		fprintf(stderr, "error while reading directory %s: %s\n",
+			path, err);
+	return name;
+}
+
+static void close_dir(DIR *dir)
+{
+	if (cptofs)
+		closedir(dir);
+	else
+		lkl_closedir((struct lkl_dir *)dir);
+}
+
+static int searchdir(const char *src, const char *dst, const char *match)
+{
+	DIR *dir;
+	const char *name;
+	int ret = 0;
+
+	dir = open_dir(src);
+	if (!dir)
+		return -1;
+
+	while ((name = read_dir(dir, src))) {
+		if (!strcmp(name, ".") || !strcmp(name, "..") ||
+		    (match && fnmatch(match, name, 0) != 0))
+			continue;
+
+		ret = do_entry(src, dst, name);
+		if (ret)
+			goto out;
+	}
+
+out:
+	close_dir(dir);
+
+	return ret;
+}
+
+static int match_root(const char *src)
+{
+	const char *c = src;
+
+	while (*c) {
+		switch (*c) {
+		case '.':
+			if (c > src && c[-1] == '.')
+				return 0;
+			break;
+		case '/':
+			break;
+		default:
+			return 0;
+		}
+		c++;
+	}
+
+	return 1;
+}
+
+int copy_one(const char *src, const char *mpoint, const char *dst)
+{
+	char *src_path_dir, *src_path_base;
+	char src_path[PATH_MAX], dst_path[PATH_MAX];
+
+	if (cptofs) {
+		snprintf(src_path, sizeof(src_path),  "%s", src);
+		snprintf(dst_path, sizeof(dst_path),  "%s/%s", mpoint, dst);
+	} else {
+		snprintf(src_path, sizeof(src_path),  "%s/%s", mpoint, src);
+		snprintf(dst_path, sizeof(dst_path),  "%s", dst);
+	}
+
+	if (match_root(src))
+		return searchdir(src_path, dst, NULL);
+
+	src_path_dir = dirname(strdup(src_path));
+	src_path_base = basename(strdup(src_path));
+
+	return searchdir(src_path_dir, dst_path, src_path_base);
+}
+
+int main(int argc, char **argv)
+{
+	struct lkl_disk disk;
+	long ret, umount_ret;
+	int i;
+	char mpoint[32];
+	unsigned int disk_id;
+
+	if (strstr(argv[0], "cptofs")) {
+		cptofs = 1;
+		ret = argp_parse(&argp_cptofs, argc, argv, 0, 0, &cla);
+	} else {
+		ret = argp_parse(&argp_cpfromfs, argc, argv, 0, 0, &cla);
+	}
+
+	if (ret < 0)
+		return -1;
+
+	if (!cla.printk)
+		lkl_host_ops.print = NULL;
+
+	disk.fd = open(cla.fsimg_path, cptofs ? O_RDWR : O_RDONLY);
+	if (disk.fd < 0) {
+		fprintf(stderr, "can't open fsimg %s: %s\n", cla.fsimg_path,
+			strerror(errno));
+		ret = 1;
+		goto out;
+	}
+
+	disk.ops = NULL;
+	disk.dev = (char *)cla.fsimg_path;
+
+	ret = lkl_disk_add(&disk);
+	if (ret < 0) {
+		fprintf(stderr, "can't add disk: %s\n", lkl_strerror(ret));
+		goto out_close;
+	}
+	disk_id = ret;
+
+	lkl_start_kernel(&lkl_host_ops, "mem=100M");
+
+	ret = lkl_mount_dev(disk_id, cla.part, cla.fsimg_type,
+			    cptofs ? 0 : LKL_MS_RDONLY,
+			    NULL, mpoint, sizeof(mpoint));
+	if (ret) {
+		fprintf(stderr, "can't mount disk: %s\n", lkl_strerror(ret));
+		goto out_close;
+	}
+
+	lkl_sys_umask(0);
+
+	for (i = 0; i < cla.npaths - 1; i++) {
+		ret = copy_one(cla.paths[i], mpoint, cla.paths[cla.npaths - 1]);
+		if (ret)
+			break;
+	}
+
+	umount_ret = lkl_umount_dev(disk_id, cla.part, 0, 1000);
+	if (ret == 0)
+		ret = umount_ret;
+
+out_close:
+	close(disk.fd);
+
+out:
+	lkl_sys_halt();
+
+	return ret;
+}
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v3 20/26] lkl tools: fs2tar that converts a filesystem image to tar
  2020-02-05  7:30 [RFC v3 00/26] Unifying LKL into UML Hajime Tazaki
@ 2020-02-05  7:30   ` Hajime Tazaki
       [not found] ` <cover.1580882335.git.thehajime-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
                     ` (25 subsequent siblings)
  26 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05  7:30 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Conrad Meyer, Petros Angelatos, Hajime Tazaki

From: Octavian Purdila <tavi.purdila@gmail.com>

Simple utility that converts a filesystem image to a tar file,
preserving file rights and extended attributes.

Cc: Conrad Meyer <cem@FreeBSD.org>
Cc: Petros Angelatos <petrosagg@gmail.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/lkl/.gitignore |   1 +
 tools/lkl/Build      |   1 +
 tools/lkl/Targets    |   4 +
 tools/lkl/fs2tar.c   | 412 +++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 418 insertions(+)
 create mode 100644 tools/lkl/fs2tar.c

diff --git a/tools/lkl/.gitignore b/tools/lkl/.gitignore
index 1a8210f4d9c4..138e65efcad2 100644
--- a/tools/lkl/.gitignore
+++ b/tools/lkl/.gitignore
@@ -9,3 +9,4 @@ tests/valgrind*.xml
 *.pyc
 cptofs
 cpfromfs
+fs2tar
diff --git a/tools/lkl/Build b/tools/lkl/Build
index a9d12c5ca260..73b37363a6de 100644
--- a/tools/lkl/Build
+++ b/tools/lkl/Build
@@ -1,2 +1,3 @@
 cptofs-$(LKL_HOST_CONFIG_ARCHIVE) += cptofs.o
+fs2tar-$(LKL_HOST_CONFIG_ARCHIVE) += fs2tar.o
 
diff --git a/tools/lkl/Targets b/tools/lkl/Targets
index 17de965f92b3..3451a1856955 100644
--- a/tools/lkl/Targets
+++ b/tools/lkl/Targets
@@ -8,3 +8,7 @@ progs-$(LKL_HOST_CONFIG_ARCHIVE) += cptofs
 progs-$(LKL_HOST_CONFIG_ARCHIVE) += cpfromfs
 LDLIBS_cptofs-y := -larchive
 LDLIBS_cptofs-$(LKL_HOST_CONFIG_NEEDS_LARGP) += -largp
+
+progs-$(LKL_HOST_CONFIG_ARCHIVE) += fs2tar
+LDLIBS_fs2tar-y := -larchive
+LDLIBS_fs2tar-$(LKL_HOST_CONFIG_NEEDS_LARGP) += -largp
diff --git a/tools/lkl/fs2tar.c b/tools/lkl/fs2tar.c
new file mode 100644
index 000000000000..b7dbcf2028a3
--- /dev/null
+++ b/tools/lkl/fs2tar.c
@@ -0,0 +1,412 @@
+// SPDX-License-Identifier: GPL-2.0
+#ifdef __FreeBSD__
+#include <sys/param.h>
+#endif
+
+#include <stdio.h>
+#include <time.h>
+#include <argp.h>
+#include <unistd.h>
+#include <string.h>
+#include <stdlib.h>
+#include <libgen.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <archive.h>
+#include <archive_entry.h>
+#include <lkl.h>
+#include <lkl_host.h>
+
+char doc[] = "";
+char args_doc[] = "-t fstype fsimage_path tar_path";
+static struct argp_option options[] = {
+	{"enable-printk", 'p', 0, 0, "show Linux printks"},
+	{"partition", 'P', "int", 0, "partition number"},
+	{"filesystem-type", 't', "string", 0,
+	 "select filesystem type - mandatory"},
+	{"selinux-contexts", 's', "file", 0,
+	 "export selinux contexts to file"},
+	{0},
+};
+
+static struct cl_args {
+	int printk;
+	int part;
+	const char *fsimg_type;
+	const char *fsimg_path;
+	const char *tar_path;
+	FILE *selinux;
+} cla;
+
+static error_t parse_opt(int key, char *arg, struct argp_state *state)
+{
+	struct cl_args *cla = state->input;
+
+	switch (key) {
+	case 'p':
+		cla->printk = 1;
+		break;
+	case 'P':
+		cla->part = atoi(arg);
+		break;
+	case 't':
+		cla->fsimg_type = arg;
+		break;
+	case 's':
+		cla->selinux = fopen(arg, "w");
+		if (!cla->selinux) {
+			fprintf(stderr,
+				"failed to open selinux contexts file: %s\n",
+				strerror(errno));
+			return -1;
+		}
+		break;
+	case ARGP_KEY_ARG:
+		if (!cla->fsimg_path)
+			cla->fsimg_path = arg;
+		else if (!cla->tar_path)
+			cla->tar_path = arg;
+		else
+			return -1;
+		break;
+	case ARGP_KEY_END:
+		if (state->arg_num < 2 || !cla->fsimg_type)
+			argp_usage(state);
+		break;
+	default:
+		return ARGP_ERR_UNKNOWN;
+	}
+
+	return 0;
+}
+
+static struct argp argp = { options, parse_opt, args_doc, doc };
+
+static struct archive *tar;
+
+static int searchdir(const char *fsimg_path, const char *path);
+
+static int copy_file(const char *fsimg_path, const char *path)
+{
+	long fsimg_fd;
+	char buff[4096];
+	long len, wrote;
+	int ret = 0;
+
+	fsimg_fd = lkl_sys_open(fsimg_path, LKL_O_RDONLY, 0);
+	if (fsimg_fd < 0) {
+		fprintf(stderr, "fsimg error opening %s: %s\n", fsimg_path,
+			lkl_strerror(fsimg_fd));
+		return fsimg_fd;
+	}
+
+	do {
+		len = lkl_sys_read(fsimg_fd, buff, sizeof(buff));
+		if (len > 0) {
+			wrote = archive_write_data(tar, buff, len);
+			if (wrote != len) {
+				fprintf(stderr,
+					"error writing file %s to archive: %s [%d %ld]\n",
+					path, archive_error_string(tar), ret,
+					len);
+				ret = -archive_errno(tar);
+				break;
+			}
+		}
+
+		if (len < 0) {
+			fprintf(stderr, "error reading fsimg file %s: %s\n",
+				fsimg_path, lkl_strerror(len));
+			ret = len;
+		}
+
+	} while (len > 0);
+
+	lkl_sys_close(fsimg_fd);
+
+	return ret;
+}
+
+static int add_link(const char *fsimg_path, const char *path,
+		    struct archive_entry *entry)
+{
+	char buf[4096] = { 0, };
+	long len;
+
+	len = lkl_sys_readlink(fsimg_path, buf, sizeof(buf));
+	if (len < 0) {
+		fprintf(stderr, "fsimg readlink error %s: %s\n",
+			fsimg_path, lkl_strerror(len));
+		return len;
+	}
+
+	archive_entry_set_symlink(entry, buf);
+
+	return 0;
+}
+
+static inline void fsimg_copy_stat(struct stat *st, struct lkl_stat *fst)
+{
+	st->st_dev = fst->st_dev;
+	st->st_ino = fst->st_ino;
+	st->st_mode = fst->st_mode;
+	st->st_nlink = fst->st_nlink;
+	st->st_uid = fst->st_uid;
+	st->st_gid = fst->st_gid;
+	st->st_rdev = fst->st_rdev;
+	st->st_size = fst->st_size;
+	st->st_blksize = fst->st_blksize;
+	st->st_blocks = fst->st_blocks;
+	st->st_atim.tv_sec = fst->lkl_st_atime;
+	st->st_atim.tv_nsec = fst->st_atime_nsec;
+	st->st_mtim.tv_sec = fst->lkl_st_mtime;
+	st->st_mtim.tv_nsec = fst->st_mtime_nsec;
+	st->st_ctim.tv_sec = fst->lkl_st_ctime;
+	st->st_ctim.tv_nsec = fst->st_ctime_nsec;
+}
+
+static int copy_xattr(const char *fsimg_path, const char *path,
+		      struct archive_entry *entry)
+{
+	long ret;
+	char *xattr_list, *i;
+	long xattr_list_size;
+
+	ret = lkl_sys_llistxattr(fsimg_path, NULL, 0);
+	if (ret < 0) {
+		fprintf(stderr, "fsimg llistxattr(%s) error: %s\n",
+			path, lkl_strerror(ret));
+		return ret;
+	}
+
+	if (!ret)
+		return 0;
+
+	xattr_list = malloc(ret);
+
+	ret = lkl_sys_llistxattr(fsimg_path, xattr_list, ret);
+	if (ret < 0) {
+		fprintf(stderr, "fsimg llistxattr(%s) error: %s\n", path,
+			lkl_strerror(ret));
+		free(xattr_list);
+		return ret;
+	}
+
+	xattr_list_size = ret;
+
+	for (i = xattr_list; i - xattr_list < xattr_list_size;
+	     i += strlen(i) + 1) {
+		void *xattr_buf;
+
+		ret = lkl_sys_lgetxattr(fsimg_path, i, NULL, 0);
+		if (ret < 0) {
+			fprintf(stderr, "fsimg lgetxattr(%s) error: %s\n", path,
+				lkl_strerror(ret));
+			free(xattr_list);
+			return ret;
+		}
+
+		xattr_buf = malloc(ret);
+
+		ret = lkl_sys_lgetxattr(fsimg_path, i, xattr_buf, ret);
+		if (ret < 0) {
+			fprintf(stderr, "fsimg lgetxattr2(%s) error: %s\n",
+				path, lkl_strerror(ret));
+			free(xattr_list);
+			free(xattr_buf);
+			return ret;
+		}
+
+		if (cla.selinux && strcmp(i, "security.selinux") == 0)
+			fprintf(cla.selinux, "%s %s\n", path,
+				(char *)xattr_buf);
+
+		archive_entry_xattr_clear(entry);
+		archive_entry_xattr_add_entry(entry, i, xattr_buf, ret);
+
+		free(xattr_buf);
+	}
+
+	free(xattr_list);
+
+	return 0;
+}
+
+static int do_entry(const char *fsimg_path, const char *path,
+		    const struct lkl_linux_dirent64 *de)
+{
+	char fsimg_new_path[PATH_MAX], new_path[PATH_MAX];
+	struct lkl_stat fsimg_stat;
+	struct stat stat;
+	struct archive_entry *entry;
+	int ftype;
+	long ret;
+
+	snprintf(new_path, sizeof(new_path), "%s/%s", path, de->d_name);
+	snprintf(fsimg_new_path, sizeof(fsimg_new_path), "%s/%s", fsimg_path,
+		 de->d_name);
+
+	ret = lkl_sys_lstat(fsimg_new_path, &fsimg_stat);
+	if (ret) {
+		fprintf(stderr, "fsimg lstat(%s) error: %s\n",
+			path, lkl_strerror(ret));
+		return ret;
+	}
+
+	entry = archive_entry_new();
+
+	archive_entry_set_pathname(entry, new_path);
+	fsimg_copy_stat(&stat, &fsimg_stat);
+	archive_entry_copy_stat(entry, &stat);
+	ret = copy_xattr(fsimg_new_path, new_path, entry);
+	if (ret)
+		return ret;
+	/* TODO: ACLs */
+
+	ftype = stat.st_mode & S_IFMT;
+
+	switch (ftype) {
+	case S_IFREG:
+		archive_write_header(tar, entry);
+		ret = copy_file(fsimg_new_path, new_path);
+		break;
+	case S_IFDIR:
+		archive_write_header(tar, entry);
+		ret = searchdir(fsimg_new_path, new_path);
+		break;
+	case S_IFLNK:
+		ret = add_link(fsimg_new_path, new_path, entry);
+		/* fall through */
+	case S_IFSOCK:
+	case S_IFBLK:
+	case S_IFCHR:
+	case S_IFIFO:
+		if (ret)
+			break;
+		archive_write_header(tar, entry);
+		break;
+	default:
+		printf("skipping %s: unsupported entry type %d\n", new_path,
+		       ftype);
+	}
+
+	archive_entry_free(entry);
+
+	if (ret)
+		printf("error processing entry %s, aborting\n", new_path);
+
+	return ret;
+}
+
+static int searchdir(const char *fsimg_path, const char *path)
+{
+	long ret, fd;
+	char buf[1024], *pos;
+	long buf_len;
+
+	fd = lkl_sys_open(fsimg_path, LKL_O_RDONLY | LKL_O_DIRECTORY, 0);
+	if (fd < 0) {
+		fprintf(stderr, "failed to open dir %s: %s", fsimg_path,
+			lkl_strerror(fd));
+		return fd;
+	}
+
+	do {
+		struct lkl_linux_dirent64 *de;
+
+		de = (struct lkl_linux_dirent64 *) buf;
+		buf_len = lkl_sys_getdents64(fd, de, sizeof(buf));
+		if (buf_len < 0) {
+			fprintf(stderr, "gentdents64 error: %s\n",
+				lkl_strerror(buf_len));
+			break;
+		}
+
+		for (pos = buf; pos - buf < buf_len; pos += de->d_reclen) {
+			de = (struct lkl_linux_dirent64 *)pos;
+
+			if (!strcmp(de->d_name, ".") ||
+			    !strcmp(de->d_name, ".."))
+				continue;
+
+			ret = do_entry(fsimg_path, path, de);
+			if (ret)
+				goto out;
+		}
+
+	} while (buf_len > 0);
+
+out:
+	lkl_sys_close(fd);
+	return ret;
+}
+
+int main(int argc, char **argv)
+{
+	struct lkl_disk disk;
+	long ret;
+	char mpoint[32];
+	unsigned int disk_id;
+
+	if (argp_parse(&argp, argc, argv, 0, 0, &cla) < 0)
+		return -1;
+
+	if (!cla.printk)
+		lkl_host_ops.print = NULL;
+
+	disk.fd = open(cla.fsimg_path, O_RDONLY);
+	if (disk.fd < 0) {
+		fprintf(stderr, "can't open fsimg %s: %s\n", cla.fsimg_path,
+			strerror(errno));
+		ret = 1;
+		goto out;
+	}
+
+	disk.ops = NULL;
+	disk.dev = (char *)cla.fsimg_path;
+
+	ret = lkl_disk_add(&disk);
+	if (ret < 0) {
+		fprintf(stderr, "can't add disk: %s\n", lkl_strerror(ret));
+		goto out_close;
+	}
+	disk_id = ret;
+
+	lkl_start_kernel(&lkl_host_ops, "mem=10M");
+
+	ret = lkl_mount_dev(disk_id, cla.part, cla.fsimg_type, LKL_MS_RDONLY,
+			    NULL, mpoint, sizeof(mpoint));
+	if (ret) {
+		fprintf(stderr, "can't mount disk: %s\n", lkl_strerror(ret));
+		goto out_close;
+	}
+
+	ret = lkl_sys_chdir(mpoint);
+	if (ret) {
+		fprintf(stderr, "can't chdir to %s: %s\n", mpoint,
+			lkl_strerror(ret));
+		goto out_umount;
+	}
+
+	tar = archive_write_new();
+	archive_write_set_format_pax_restricted(tar);
+	archive_write_open_filename(tar, cla.tar_path);
+
+	ret = searchdir(mpoint, "");
+
+	archive_write_free(tar);
+
+	if (cla.selinux)
+		fclose(cla.selinux);
+
+out_umount:
+	lkl_umount_dev(disk_id, cla.part, 0, 1000);
+
+out_close:
+	close(disk.fd);
+
+out:
+	lkl_sys_halt();
+
+	return ret;
+}
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v3 20/26] lkl tools: fs2tar that converts a filesystem image to tar
@ 2020-02-05  7:30   ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05  7:30 UTC (permalink / raw)
  To: linux-um
  Cc: linux-arch, Conrad Meyer, Octavian Purdila, Akira Moroo,
	Petros Angelatos, linux-kernel-library, Hajime Tazaki

From: Octavian Purdila <tavi.purdila@gmail.com>

Simple utility that converts a filesystem image to a tar file,
preserving file rights and extended attributes.

Cc: Conrad Meyer <cem@FreeBSD.org>
Cc: Petros Angelatos <petrosagg@gmail.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/lkl/.gitignore |   1 +
 tools/lkl/Build      |   1 +
 tools/lkl/Targets    |   4 +
 tools/lkl/fs2tar.c   | 412 +++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 418 insertions(+)
 create mode 100644 tools/lkl/fs2tar.c

diff --git a/tools/lkl/.gitignore b/tools/lkl/.gitignore
index 1a8210f4d9c4..138e65efcad2 100644
--- a/tools/lkl/.gitignore
+++ b/tools/lkl/.gitignore
@@ -9,3 +9,4 @@ tests/valgrind*.xml
 *.pyc
 cptofs
 cpfromfs
+fs2tar
diff --git a/tools/lkl/Build b/tools/lkl/Build
index a9d12c5ca260..73b37363a6de 100644
--- a/tools/lkl/Build
+++ b/tools/lkl/Build
@@ -1,2 +1,3 @@
 cptofs-$(LKL_HOST_CONFIG_ARCHIVE) += cptofs.o
+fs2tar-$(LKL_HOST_CONFIG_ARCHIVE) += fs2tar.o
 
diff --git a/tools/lkl/Targets b/tools/lkl/Targets
index 17de965f92b3..3451a1856955 100644
--- a/tools/lkl/Targets
+++ b/tools/lkl/Targets
@@ -8,3 +8,7 @@ progs-$(LKL_HOST_CONFIG_ARCHIVE) += cptofs
 progs-$(LKL_HOST_CONFIG_ARCHIVE) += cpfromfs
 LDLIBS_cptofs-y := -larchive
 LDLIBS_cptofs-$(LKL_HOST_CONFIG_NEEDS_LARGP) += -largp
+
+progs-$(LKL_HOST_CONFIG_ARCHIVE) += fs2tar
+LDLIBS_fs2tar-y := -larchive
+LDLIBS_fs2tar-$(LKL_HOST_CONFIG_NEEDS_LARGP) += -largp
diff --git a/tools/lkl/fs2tar.c b/tools/lkl/fs2tar.c
new file mode 100644
index 000000000000..b7dbcf2028a3
--- /dev/null
+++ b/tools/lkl/fs2tar.c
@@ -0,0 +1,412 @@
+// SPDX-License-Identifier: GPL-2.0
+#ifdef __FreeBSD__
+#include <sys/param.h>
+#endif
+
+#include <stdio.h>
+#include <time.h>
+#include <argp.h>
+#include <unistd.h>
+#include <string.h>
+#include <stdlib.h>
+#include <libgen.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <archive.h>
+#include <archive_entry.h>
+#include <lkl.h>
+#include <lkl_host.h>
+
+char doc[] = "";
+char args_doc[] = "-t fstype fsimage_path tar_path";
+static struct argp_option options[] = {
+	{"enable-printk", 'p', 0, 0, "show Linux printks"},
+	{"partition", 'P', "int", 0, "partition number"},
+	{"filesystem-type", 't', "string", 0,
+	 "select filesystem type - mandatory"},
+	{"selinux-contexts", 's', "file", 0,
+	 "export selinux contexts to file"},
+	{0},
+};
+
+static struct cl_args {
+	int printk;
+	int part;
+	const char *fsimg_type;
+	const char *fsimg_path;
+	const char *tar_path;
+	FILE *selinux;
+} cla;
+
+static error_t parse_opt(int key, char *arg, struct argp_state *state)
+{
+	struct cl_args *cla = state->input;
+
+	switch (key) {
+	case 'p':
+		cla->printk = 1;
+		break;
+	case 'P':
+		cla->part = atoi(arg);
+		break;
+	case 't':
+		cla->fsimg_type = arg;
+		break;
+	case 's':
+		cla->selinux = fopen(arg, "w");
+		if (!cla->selinux) {
+			fprintf(stderr,
+				"failed to open selinux contexts file: %s\n",
+				strerror(errno));
+			return -1;
+		}
+		break;
+	case ARGP_KEY_ARG:
+		if (!cla->fsimg_path)
+			cla->fsimg_path = arg;
+		else if (!cla->tar_path)
+			cla->tar_path = arg;
+		else
+			return -1;
+		break;
+	case ARGP_KEY_END:
+		if (state->arg_num < 2 || !cla->fsimg_type)
+			argp_usage(state);
+		break;
+	default:
+		return ARGP_ERR_UNKNOWN;
+	}
+
+	return 0;
+}
+
+static struct argp argp = { options, parse_opt, args_doc, doc };
+
+static struct archive *tar;
+
+static int searchdir(const char *fsimg_path, const char *path);
+
+static int copy_file(const char *fsimg_path, const char *path)
+{
+	long fsimg_fd;
+	char buff[4096];
+	long len, wrote;
+	int ret = 0;
+
+	fsimg_fd = lkl_sys_open(fsimg_path, LKL_O_RDONLY, 0);
+	if (fsimg_fd < 0) {
+		fprintf(stderr, "fsimg error opening %s: %s\n", fsimg_path,
+			lkl_strerror(fsimg_fd));
+		return fsimg_fd;
+	}
+
+	do {
+		len = lkl_sys_read(fsimg_fd, buff, sizeof(buff));
+		if (len > 0) {
+			wrote = archive_write_data(tar, buff, len);
+			if (wrote != len) {
+				fprintf(stderr,
+					"error writing file %s to archive: %s [%d %ld]\n",
+					path, archive_error_string(tar), ret,
+					len);
+				ret = -archive_errno(tar);
+				break;
+			}
+		}
+
+		if (len < 0) {
+			fprintf(stderr, "error reading fsimg file %s: %s\n",
+				fsimg_path, lkl_strerror(len));
+			ret = len;
+		}
+
+	} while (len > 0);
+
+	lkl_sys_close(fsimg_fd);
+
+	return ret;
+}
+
+static int add_link(const char *fsimg_path, const char *path,
+		    struct archive_entry *entry)
+{
+	char buf[4096] = { 0, };
+	long len;
+
+	len = lkl_sys_readlink(fsimg_path, buf, sizeof(buf));
+	if (len < 0) {
+		fprintf(stderr, "fsimg readlink error %s: %s\n",
+			fsimg_path, lkl_strerror(len));
+		return len;
+	}
+
+	archive_entry_set_symlink(entry, buf);
+
+	return 0;
+}
+
+static inline void fsimg_copy_stat(struct stat *st, struct lkl_stat *fst)
+{
+	st->st_dev = fst->st_dev;
+	st->st_ino = fst->st_ino;
+	st->st_mode = fst->st_mode;
+	st->st_nlink = fst->st_nlink;
+	st->st_uid = fst->st_uid;
+	st->st_gid = fst->st_gid;
+	st->st_rdev = fst->st_rdev;
+	st->st_size = fst->st_size;
+	st->st_blksize = fst->st_blksize;
+	st->st_blocks = fst->st_blocks;
+	st->st_atim.tv_sec = fst->lkl_st_atime;
+	st->st_atim.tv_nsec = fst->st_atime_nsec;
+	st->st_mtim.tv_sec = fst->lkl_st_mtime;
+	st->st_mtim.tv_nsec = fst->st_mtime_nsec;
+	st->st_ctim.tv_sec = fst->lkl_st_ctime;
+	st->st_ctim.tv_nsec = fst->st_ctime_nsec;
+}
+
+static int copy_xattr(const char *fsimg_path, const char *path,
+		      struct archive_entry *entry)
+{
+	long ret;
+	char *xattr_list, *i;
+	long xattr_list_size;
+
+	ret = lkl_sys_llistxattr(fsimg_path, NULL, 0);
+	if (ret < 0) {
+		fprintf(stderr, "fsimg llistxattr(%s) error: %s\n",
+			path, lkl_strerror(ret));
+		return ret;
+	}
+
+	if (!ret)
+		return 0;
+
+	xattr_list = malloc(ret);
+
+	ret = lkl_sys_llistxattr(fsimg_path, xattr_list, ret);
+	if (ret < 0) {
+		fprintf(stderr, "fsimg llistxattr(%s) error: %s\n", path,
+			lkl_strerror(ret));
+		free(xattr_list);
+		return ret;
+	}
+
+	xattr_list_size = ret;
+
+	for (i = xattr_list; i - xattr_list < xattr_list_size;
+	     i += strlen(i) + 1) {
+		void *xattr_buf;
+
+		ret = lkl_sys_lgetxattr(fsimg_path, i, NULL, 0);
+		if (ret < 0) {
+			fprintf(stderr, "fsimg lgetxattr(%s) error: %s\n", path,
+				lkl_strerror(ret));
+			free(xattr_list);
+			return ret;
+		}
+
+		xattr_buf = malloc(ret);
+
+		ret = lkl_sys_lgetxattr(fsimg_path, i, xattr_buf, ret);
+		if (ret < 0) {
+			fprintf(stderr, "fsimg lgetxattr2(%s) error: %s\n",
+				path, lkl_strerror(ret));
+			free(xattr_list);
+			free(xattr_buf);
+			return ret;
+		}
+
+		if (cla.selinux && strcmp(i, "security.selinux") == 0)
+			fprintf(cla.selinux, "%s %s\n", path,
+				(char *)xattr_buf);
+
+		archive_entry_xattr_clear(entry);
+		archive_entry_xattr_add_entry(entry, i, xattr_buf, ret);
+
+		free(xattr_buf);
+	}
+
+	free(xattr_list);
+
+	return 0;
+}
+
+static int do_entry(const char *fsimg_path, const char *path,
+		    const struct lkl_linux_dirent64 *de)
+{
+	char fsimg_new_path[PATH_MAX], new_path[PATH_MAX];
+	struct lkl_stat fsimg_stat;
+	struct stat stat;
+	struct archive_entry *entry;
+	int ftype;
+	long ret;
+
+	snprintf(new_path, sizeof(new_path), "%s/%s", path, de->d_name);
+	snprintf(fsimg_new_path, sizeof(fsimg_new_path), "%s/%s", fsimg_path,
+		 de->d_name);
+
+	ret = lkl_sys_lstat(fsimg_new_path, &fsimg_stat);
+	if (ret) {
+		fprintf(stderr, "fsimg lstat(%s) error: %s\n",
+			path, lkl_strerror(ret));
+		return ret;
+	}
+
+	entry = archive_entry_new();
+
+	archive_entry_set_pathname(entry, new_path);
+	fsimg_copy_stat(&stat, &fsimg_stat);
+	archive_entry_copy_stat(entry, &stat);
+	ret = copy_xattr(fsimg_new_path, new_path, entry);
+	if (ret)
+		return ret;
+	/* TODO: ACLs */
+
+	ftype = stat.st_mode & S_IFMT;
+
+	switch (ftype) {
+	case S_IFREG:
+		archive_write_header(tar, entry);
+		ret = copy_file(fsimg_new_path, new_path);
+		break;
+	case S_IFDIR:
+		archive_write_header(tar, entry);
+		ret = searchdir(fsimg_new_path, new_path);
+		break;
+	case S_IFLNK:
+		ret = add_link(fsimg_new_path, new_path, entry);
+		/* fall through */
+	case S_IFSOCK:
+	case S_IFBLK:
+	case S_IFCHR:
+	case S_IFIFO:
+		if (ret)
+			break;
+		archive_write_header(tar, entry);
+		break;
+	default:
+		printf("skipping %s: unsupported entry type %d\n", new_path,
+		       ftype);
+	}
+
+	archive_entry_free(entry);
+
+	if (ret)
+		printf("error processing entry %s, aborting\n", new_path);
+
+	return ret;
+}
+
+static int searchdir(const char *fsimg_path, const char *path)
+{
+	long ret, fd;
+	char buf[1024], *pos;
+	long buf_len;
+
+	fd = lkl_sys_open(fsimg_path, LKL_O_RDONLY | LKL_O_DIRECTORY, 0);
+	if (fd < 0) {
+		fprintf(stderr, "failed to open dir %s: %s", fsimg_path,
+			lkl_strerror(fd));
+		return fd;
+	}
+
+	do {
+		struct lkl_linux_dirent64 *de;
+
+		de = (struct lkl_linux_dirent64 *) buf;
+		buf_len = lkl_sys_getdents64(fd, de, sizeof(buf));
+		if (buf_len < 0) {
+			fprintf(stderr, "gentdents64 error: %s\n",
+				lkl_strerror(buf_len));
+			break;
+		}
+
+		for (pos = buf; pos - buf < buf_len; pos += de->d_reclen) {
+			de = (struct lkl_linux_dirent64 *)pos;
+
+			if (!strcmp(de->d_name, ".") ||
+			    !strcmp(de->d_name, ".."))
+				continue;
+
+			ret = do_entry(fsimg_path, path, de);
+			if (ret)
+				goto out;
+		}
+
+	} while (buf_len > 0);
+
+out:
+	lkl_sys_close(fd);
+	return ret;
+}
+
+int main(int argc, char **argv)
+{
+	struct lkl_disk disk;
+	long ret;
+	char mpoint[32];
+	unsigned int disk_id;
+
+	if (argp_parse(&argp, argc, argv, 0, 0, &cla) < 0)
+		return -1;
+
+	if (!cla.printk)
+		lkl_host_ops.print = NULL;
+
+	disk.fd = open(cla.fsimg_path, O_RDONLY);
+	if (disk.fd < 0) {
+		fprintf(stderr, "can't open fsimg %s: %s\n", cla.fsimg_path,
+			strerror(errno));
+		ret = 1;
+		goto out;
+	}
+
+	disk.ops = NULL;
+	disk.dev = (char *)cla.fsimg_path;
+
+	ret = lkl_disk_add(&disk);
+	if (ret < 0) {
+		fprintf(stderr, "can't add disk: %s\n", lkl_strerror(ret));
+		goto out_close;
+	}
+	disk_id = ret;
+
+	lkl_start_kernel(&lkl_host_ops, "mem=10M");
+
+	ret = lkl_mount_dev(disk_id, cla.part, cla.fsimg_type, LKL_MS_RDONLY,
+			    NULL, mpoint, sizeof(mpoint));
+	if (ret) {
+		fprintf(stderr, "can't mount disk: %s\n", lkl_strerror(ret));
+		goto out_close;
+	}
+
+	ret = lkl_sys_chdir(mpoint);
+	if (ret) {
+		fprintf(stderr, "can't chdir to %s: %s\n", mpoint,
+			lkl_strerror(ret));
+		goto out_umount;
+	}
+
+	tar = archive_write_new();
+	archive_write_set_format_pax_restricted(tar);
+	archive_write_open_filename(tar, cla.tar_path);
+
+	ret = searchdir(mpoint, "");
+
+	archive_write_free(tar);
+
+	if (cla.selinux)
+		fclose(cla.selinux);
+
+out_umount:
+	lkl_umount_dev(disk_id, cla.part, 0, 1000);
+
+out_close:
+	close(disk.fd);
+
+out:
+	lkl_sys_halt();
+
+	return ret;
+}
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v3 21/26] lkl tools: add lklfuse
  2020-02-05  7:30 [RFC v3 00/26] Unifying LKL into UML Hajime Tazaki
                   ` (19 preceding siblings ...)
  2020-02-05  7:30   ` Hajime Tazaki
@ 2020-02-05  7:30 ` Hajime Tazaki
  2020-02-05  7:30   ` Hajime Tazaki
                   ` (5 subsequent siblings)
  26 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05  7:30 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Conrad Meyer, Quentin Anglade, Rafael Gieschke, Hajime Tazaki

From: Octavian Purdila <tavi.purdila@gmail.com>

Add a simple fuse based program that can mount filesystem in userspace
using LKL.

Cc: Conrad Meyer <cem@FreeBSD.org>
Cc: Quentin Anglade <quentin.anglade@objectif-libre.com>
Cc: Rafael Gieschke <rafael.gieschke@rz.uni-freiburg.de>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/lkl/.gitignore       |   1 +
 tools/lkl/Build            |   3 +
 tools/lkl/Targets          |   3 +
 tools/lkl/lklfuse.c        | 659 +++++++++++++++++++++++++++++++++++++
 tools/lkl/tests/lklfuse.sh | 110 +++++++
 5 files changed, 776 insertions(+)
 create mode 100644 tools/lkl/lklfuse.c
 create mode 100755 tools/lkl/tests/lklfuse.sh

diff --git a/tools/lkl/.gitignore b/tools/lkl/.gitignore
index 138e65efcad2..c78ec268e4b0 100644
--- a/tools/lkl/.gitignore
+++ b/tools/lkl/.gitignore
@@ -10,3 +10,4 @@ tests/valgrind*.xml
 cptofs
 cpfromfs
 fs2tar
+lklfuse
diff --git a/tools/lkl/Build b/tools/lkl/Build
index 73b37363a6de..6048440d0e1b 100644
--- a/tools/lkl/Build
+++ b/tools/lkl/Build
@@ -1,3 +1,6 @@
+CFLAGS_lklfuse.o += -D_FILE_OFFSET_BITS=64
+
 cptofs-$(LKL_HOST_CONFIG_ARCHIVE) += cptofs.o
 fs2tar-$(LKL_HOST_CONFIG_ARCHIVE) += fs2tar.o
+lklfuse-$(LKL_HOST_CONFIG_FUSE) += lklfuse.o
 
diff --git a/tools/lkl/Targets b/tools/lkl/Targets
index 3451a1856955..03d24362a195 100644
--- a/tools/lkl/Targets
+++ b/tools/lkl/Targets
@@ -12,3 +12,6 @@ LDLIBS_cptofs-$(LKL_HOST_CONFIG_NEEDS_LARGP) += -largp
 progs-$(LKL_HOST_CONFIG_ARCHIVE) += fs2tar
 LDLIBS_fs2tar-y := -larchive
 LDLIBS_fs2tar-$(LKL_HOST_CONFIG_NEEDS_LARGP) += -largp
+
+progs-$(LKL_HOST_CONFIG_FUSE) += lklfuse
+LDLIBS_lklfuse-y := -lfuse
diff --git a/tools/lkl/lklfuse.c b/tools/lkl/lklfuse.c
new file mode 100644
index 000000000000..f6e14778e354
--- /dev/null
+++ b/tools/lkl/lklfuse.c
@@ -0,0 +1,659 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdbool.h>
+#include <stddef.h>
+#include <stdio.h>
+#include <string.h>
+#include <stdlib.h>
+#include <sys/stat.h>
+#include <string.h>
+#include <errno.h>
+#include <unistd.h>
+#define FUSE_USE_VERSION 26
+#include <fuse.h>
+#include <fuse/fuse_opt.h>
+#include <fuse/fuse_lowlevel.h>
+#include <lkl.h>
+#include <lkl_host.h>
+
+#define LKLFUSE_VERSION "0.1"
+
+struct lklfuse {
+	const char *file;
+	const char *log;
+	const char *type;
+	const char *opts;
+	struct lkl_disk disk;
+	int disk_id;
+	int part;
+	int ro;
+	int mb;
+} lklfuse = {
+	.mb = 64,
+};
+
+#define LKLFUSE_OPT(t, p, v) { t, offsetof(struct lklfuse, p), v }
+
+enum {
+	KEY_HELP,
+	KEY_VERSION,
+};
+
+static struct fuse_opt lklfuse_opts[] = {
+	LKLFUSE_OPT("log=%s", log, 0),
+	LKLFUSE_OPT("type=%s", type, 0),
+	LKLFUSE_OPT("mb=%d", mb, 0),
+	LKLFUSE_OPT("opts=%s", opts, 0),
+	LKLFUSE_OPT("part=%d", part, 0),
+	FUSE_OPT_KEY("-h", KEY_HELP),
+	FUSE_OPT_KEY("--help", KEY_HELP),
+	FUSE_OPT_KEY("-V",             KEY_VERSION),
+	FUSE_OPT_KEY("--version",      KEY_VERSION),
+	FUSE_OPT_END
+};
+
+static void usage(void)
+{
+	printf(
+"usage: lklfuse file mountpoint [options]\n"
+"\n"
+"general options:\n"
+"    -o opt,[opt...]        mount options\n"
+"    -h   --help            print help\n"
+"    -V   --version         print version\n"
+"\n"
+"lklfuse options:\n"
+"    -o log=FILE            log file\n"
+"    -o type=fstype         filesystem type\n"
+"    -o mb=memory in mb     ammount of memory to allocate\n"
+"    -o part=parition       partition to mount\n"
+"    -o ro                  open file read-only\n"
+"    -o opts=options        mount options (use \\ to escape , and =)\n"
+);
+}
+
+static int lklfuse_opt_proc(void *data, const char *arg, int key,
+			  struct fuse_args *args)
+{
+	switch (key) {
+	case FUSE_OPT_KEY_OPT:
+		if (strcmp(arg, "ro") == 0)
+			lklfuse.ro = 1;
+		return 1;
+
+	case FUSE_OPT_KEY_NONOPT:
+		if (!lklfuse.file) {
+			lklfuse.file = strdup(arg);
+			return 0;
+		}
+		return 1;
+
+	case KEY_HELP:
+		usage();
+		fuse_opt_add_arg(args, "-ho");
+		fuse_main(args->argc, args->argv, NULL, NULL);
+		exit(1);
+
+	case KEY_VERSION:
+		printf("lklfuse version %s\n", LKLFUSE_VERSION);
+		fuse_opt_add_arg(args, "--version");
+		fuse_main(args->argc, args->argv, NULL, NULL);
+		exit(0);
+
+	default:
+		fprintf(stderr, "internal error\n");
+		abort();
+	}
+}
+
+static void lklfuse_xlat_stat(const struct lkl_stat *in, struct stat *st)
+{
+	st->st_dev = in->st_dev;
+	st->st_ino = in->st_ino;
+	st->st_mode = in->st_mode;
+	st->st_nlink = in->st_nlink;
+	st->st_uid = in->st_uid;
+	st->st_gid = in->st_gid;
+	st->st_rdev = in->st_rdev;
+	st->st_size = in->st_size;
+	st->st_blksize = in->st_blksize;
+	st->st_blocks = in->st_blocks;
+	st->st_atim.tv_sec = in->lkl_st_atime;
+	st->st_atim.tv_nsec = in->st_atime_nsec;
+	st->st_mtim.tv_sec = in->lkl_st_mtime;
+	st->st_mtim.tv_nsec = in->st_mtime_nsec;
+	st->st_ctim.tv_sec = in->lkl_st_ctime;
+	st->st_ctim.tv_nsec = in->st_ctime_nsec;
+}
+
+static int lklfuse_fgetattr(const char *path, struct stat *st,
+			    struct fuse_file_info *fi)
+{
+	long ret;
+	struct lkl_stat lkl_stat;
+
+	ret = lkl_sys_fstat(fi->fh, &lkl_stat);
+	if (ret)
+		return ret;
+
+	lklfuse_xlat_stat(&lkl_stat, st);
+	return 0;
+}
+
+static int lklfuse_getattr(const char *path, struct stat *st)
+{
+	long ret;
+	struct lkl_stat lkl_stat;
+
+	ret = lkl_sys_lstat(path, &lkl_stat);
+	if (ret)
+		return ret;
+
+	lklfuse_xlat_stat(&lkl_stat, st);
+	return 0;
+}
+
+static int lklfuse_readlink(const char *path, char *buf, size_t len)
+{
+	long ret;
+
+	ret = lkl_sys_readlink(path, buf, len);
+	if (ret < 0)
+		return ret;
+
+	if ((size_t)ret == len)
+		ret = len - 1;
+
+	buf[ret] = 0;
+
+	return 0;
+}
+
+static int lklfuse_mknod(const char *path, mode_t mode, dev_t dev)
+{
+	return lkl_sys_mknod(path, mode, dev);
+}
+
+static int lklfuse_mkdir(const char *path, mode_t mode)
+{
+	return lkl_sys_mkdir(path, mode);
+}
+
+static int lklfuse_unlink(const char *path)
+{
+	return lkl_sys_unlink(path);
+}
+
+static int lklfuse_rmdir(const char *path)
+{
+	return lkl_sys_rmdir(path);
+}
+
+static int lklfuse_symlink(const char *oldname, const char *newname)
+{
+	return lkl_sys_symlink(oldname, newname);
+}
+
+
+static int lklfuse_rename(const char *oldname, const char *newname)
+{
+	return lkl_sys_rename(oldname, newname);
+}
+
+static int lklfuse_link(const char *oldname, const char *newname)
+{
+	return lkl_sys_link(oldname, newname);
+}
+
+static int lklfuse_chmod(const char *path, mode_t mode)
+{
+	return lkl_sys_chmod(path, mode);
+}
+
+
+static int lklfuse_chown(const char *path, uid_t uid, gid_t gid)
+{
+	return lkl_sys_fchownat(LKL_AT_FDCWD, path, uid, gid,
+				LKL_AT_SYMLINK_NOFOLLOW);
+}
+
+static int lklfuse_truncate(const char *path, off_t off)
+{
+	return lkl_sys_truncate(path, off);
+}
+
+static int lklfuse_open3(const char *path, bool create, mode_t mode,
+			 struct fuse_file_info *fi)
+{
+	long ret;
+	int flags;
+
+	if ((fi->flags & O_ACCMODE) == O_RDONLY)
+		flags = LKL_O_RDONLY;
+	else if ((fi->flags & O_ACCMODE) == O_WRONLY)
+		flags = LKL_O_WRONLY;
+	else if ((fi->flags & O_ACCMODE) == O_RDWR)
+		flags = LKL_O_RDWR;
+	else
+		return -EINVAL;
+
+	if (create)
+		flags |= LKL_O_CREAT;
+
+	ret = lkl_sys_open(path, flags, mode);
+	if (ret < 0)
+		return ret;
+
+	fi->fh = ret;
+
+	return 0;
+}
+
+static int lklfuse_create(const char *path, mode_t mode,
+			  struct fuse_file_info *fi)
+{
+	return lklfuse_open3(path, true, mode, fi);
+}
+
+static int lklfuse_open(const char *path, struct fuse_file_info *fi)
+{
+	return lklfuse_open3(path, false, 0, fi);
+}
+
+static int lklfuse_read(const char *path, char *buf, size_t size, off_t offset,
+		      struct fuse_file_info *fi)
+{
+	long ret;
+	ssize_t orig_size = size;
+
+	do {
+		ret = lkl_sys_pread64(fi->fh, buf, size, offset);
+		if (ret <= 0)
+			break;
+		size -= ret;
+		offset += ret;
+		buf += ret;
+	} while (size > 0);
+
+	return ret < 0 ? ret : orig_size - (ssize_t)size;
+
+}
+
+static int lklfuse_write(const char *path, const char *buf, size_t size,
+		       off_t offset, struct fuse_file_info *fi)
+{
+	long ret;
+	ssize_t orig_size = size;
+
+	do {
+		ret = lkl_sys_pwrite64(fi->fh, buf, size, offset);
+		if (ret <= 0)
+			break;
+		size -= ret;
+		offset += ret;
+		buf += ret;
+	} while (size > 0);
+
+	return ret < 0 ? ret : orig_size - (ssize_t)size;
+}
+
+
+static int lklfuse_statfs(const char *path, struct statvfs *stat)
+{
+	long ret;
+	struct lkl_statfs lkl_statfs;
+
+	ret = lkl_sys_statfs(path, &lkl_statfs);
+	if (ret < 0)
+		return ret;
+
+	stat->f_bsize = lkl_statfs.f_bsize;
+	stat->f_frsize = lkl_statfs.f_frsize;
+	stat->f_blocks = lkl_statfs.f_blocks;
+	stat->f_bfree = lkl_statfs.f_bfree;
+	stat->f_bavail = lkl_statfs.f_bavail;
+	stat->f_files = lkl_statfs.f_files;
+	stat->f_ffree = lkl_statfs.f_ffree;
+	stat->f_favail = stat->f_ffree;
+	stat->f_fsid = *(unsigned long *)&lkl_statfs.f_fsid.val[0];
+	stat->f_flag = lkl_statfs.f_flags;
+	stat->f_namemax = lkl_statfs.f_namelen;
+
+	return 0;
+}
+
+static int lklfuse_flush(const char *path, struct fuse_file_info *fi)
+{
+	return 0;
+}
+
+static int lklfuse_release(const char *path, struct fuse_file_info *fi)
+{
+	return lkl_sys_close(fi->fh);
+}
+
+static int lklfuse_fsync(const char *path, int datasync,
+		       struct fuse_file_info *fi)
+{
+	if (datasync)
+		return lkl_sys_fdatasync(fi->fh);
+	else
+		return lkl_sys_fsync(fi->fh);
+}
+
+static int lklfuse_setxattr(const char *path, const char *name, const char *val,
+		   size_t size, int flags)
+{
+	return lkl_sys_setxattr(path, name, val, size, flags);
+}
+
+static int lklfuse_getxattr(const char *path, const char *name, char *val,
+			  size_t size)
+{
+	return lkl_sys_getxattr(path, name, val, size);
+}
+
+static int lklfuse_listxattr(const char *path, char *list, size_t size)
+{
+	return lkl_sys_listxattr(path, list, size);
+}
+
+static int lklfuse_removexattr(const char *path, const char *name)
+{
+	return lkl_sys_removexattr(path, name);
+}
+
+static int lklfuse_opendir(const char *path, struct fuse_file_info *fi)
+{
+	struct lkl_dir *dir;
+	int err;
+
+	dir = lkl_opendir(path, &err);
+	if (!dir)
+		return err;
+
+	fi->fh = (uintptr_t)dir;
+
+	return 0;
+}
+
+/** Read directory
+ *
+ * This supersedes the old getdir() interface.  New applications
+ * should use this.
+ *
+ * The filesystem may choose between two modes of operation:
+ *
+ * 1) The readdir implementation ignores the offset parameter, and
+ * passes zero to the filler function's offset.  The filler
+ * function will not return '1' (unless an error happens), so the
+ * whole directory is read in a single readdir operation.  This
+ * works just like the old getdir() method.
+ *
+ * 2) The readdir implementation keeps track of the offsets of the
+ * directory entries.  It uses the offset parameter and always
+ * passes non-zero offset to the filler function.  When the buffer
+ * is full (or an error happens) the filler function will return
+ * '1'.
+ *
+ * Introduced in version 2.3
+ */
+static int lklfuse_readdir(const char *path, void *buf, fuse_fill_dir_t fill,
+			 off_t off, struct fuse_file_info *fi)
+{
+	struct lkl_dir *dir = (struct lkl_dir *)(uintptr_t)fi->fh;
+	struct lkl_linux_dirent64 *de;
+
+	while ((de = lkl_readdir(dir))) {
+		struct stat st = { 0, };
+
+		st.st_ino = de->d_ino;
+		st.st_mode = de->d_type << 12;
+
+		if (fill(buf, de->d_name, &st, 0))
+			break;
+	}
+
+	if (!de)
+		return lkl_errdir(dir);
+
+	return 0;
+}
+
+static int lklfuse_releasedir(const char *path, struct fuse_file_info *fi)
+{
+	struct lkl_dir *dir = (struct lkl_dir *)(uintptr_t)fi->fh;
+
+	return lkl_closedir(dir);
+}
+
+static int lklfuse_fsyncdir(const char *path, int datasync,
+			  struct fuse_file_info *fi)
+{
+	struct lkl_dir *dir = (struct lkl_dir *)(uintptr_t)fi->fh;
+	int fd = lkl_dirfd(dir);
+
+	if (datasync)
+		return lkl_sys_fdatasync(fd);
+	else
+		return lkl_sys_fsync(fd);
+}
+
+static int lklfuse_access(const char *path, int mode)
+{
+	return lkl_sys_access(path, mode);
+}
+
+static int lklfuse_utimens(const char *path, const struct timespec tv[2])
+{
+	struct lkl_timespec ts[2];
+
+	ts[0].tv_sec = tv[0].tv_sec;
+	ts[0].tv_nsec = tv[0].tv_nsec;
+	ts[1].tv_sec = tv[0].tv_sec;
+	ts[1].tv_nsec = tv[0].tv_nsec;
+
+	return lkl_sys_utimensat(-1, path, (struct __lkl__kernel_timespec *)ts,
+				 LKL_AT_SYMLINK_NOFOLLOW);
+}
+
+static int lklfuse_fallocate(const char *path, int mode, off_t offset,
+			     off_t len, struct fuse_file_info *fi)
+{
+	return lkl_sys_fallocate(fi->fh, mode, offset, len);
+}
+
+const struct fuse_operations lklfuse_ops = {
+	.flag_nullpath_ok = 1,
+	.flag_nopath = 1,
+	.flag_utime_omit_ok = 1,
+
+	.getattr = lklfuse_getattr,
+	.readlink = lklfuse_readlink,
+	.mknod = lklfuse_mknod,
+	.mkdir = lklfuse_mkdir,
+	.unlink = lklfuse_unlink,
+	.rmdir = lklfuse_rmdir,
+	.symlink = lklfuse_symlink,
+	.rename = lklfuse_rename,
+	.link = lklfuse_link,
+	.chmod = lklfuse_chmod,
+	.chown = lklfuse_chown,
+	.truncate = lklfuse_truncate,
+	.open = lklfuse_open,
+	.read = lklfuse_read,
+	.write = lklfuse_write,
+	.statfs = lklfuse_statfs,
+	.flush = lklfuse_flush,
+	.release = lklfuse_release,
+	.fsync = lklfuse_fsync,
+	.setxattr = lklfuse_setxattr,
+	.getxattr = lklfuse_getxattr,
+	.listxattr = lklfuse_listxattr,
+	.removexattr = lklfuse_removexattr,
+	.opendir = lklfuse_opendir,
+	.readdir = lklfuse_readdir,
+	.releasedir = lklfuse_releasedir,
+	.fsyncdir = lklfuse_fsyncdir,
+	.access = lklfuse_access,
+	.create = lklfuse_create,
+	.fgetattr = lklfuse_fgetattr,
+	/* .lock, */
+	.utimens = lklfuse_utimens,
+	/* .bmap, */
+	/* .ioctl, */
+	/* .poll, */
+	/* .write_buf, (SG io) */
+	/* .read_buf, (SG io) */
+	/* .flock, */
+	.fallocate = lklfuse_fallocate,
+};
+
+static int start_lkl(void)
+{
+	long ret;
+	char mpoint[32], cmdline[16];
+
+	snprintf(cmdline, sizeof(cmdline), "mem=%dM", lklfuse.mb);
+	ret = lkl_start_kernel(&lkl_host_ops, cmdline);
+	if (ret) {
+		fprintf(stderr, "can't start kernel: %s\n", lkl_strerror(ret));
+		goto out;
+	}
+
+	ret = lkl_mount_dev(lklfuse.disk_id, lklfuse.part, lklfuse.type,
+			    lklfuse.ro ? LKL_MS_RDONLY : 0, lklfuse.opts,
+			    mpoint, sizeof(mpoint));
+
+	if (ret) {
+		fprintf(stderr, "can't mount disk: %s\n", lkl_strerror(ret));
+		goto out_halt;
+	}
+
+	ret = lkl_sys_chroot(mpoint);
+	if (ret) {
+		fprintf(stderr, "can't chdir to %s: %s\n", mpoint,
+			lkl_strerror(ret));
+		goto out_umount;
+	}
+
+	return 0;
+
+out_umount:
+	lkl_umount_dev(lklfuse.disk_id, lklfuse.part, 0, 1000);
+
+out_halt:
+	lkl_sys_halt();
+
+out:
+	return ret;
+}
+
+static void stop_lkl(void)
+{
+	int ret;
+
+	ret = lkl_sys_chdir("/");
+	if (ret)
+		fprintf(stderr, "can't chdir to /: %s\n", lkl_strerror(ret));
+	ret = lkl_sys_umount("/", 0);
+	if (ret)
+		fprintf(stderr, "failed to umount disk: %d: %s\n",
+			lklfuse.disk_id, lkl_strerror(ret));
+	lkl_sys_halt();
+}
+
+int main(int argc, char **argv)
+{
+	struct fuse_args args = FUSE_ARGS_INIT(argc, argv);
+	struct fuse_chan *ch;
+	struct fuse *fuse;
+	struct stat st;
+	char *mnt;
+	int fg, mt, ret;
+
+	if (fuse_opt_parse(&args, &lklfuse, lklfuse_opts, lklfuse_opt_proc))
+		return 1;
+
+	if (!lklfuse.file || !lklfuse.type) {
+		fprintf(stderr, "no file or filesystem type specified\n");
+		return 1;
+	}
+
+	if (fuse_parse_cmdline(&args, &mnt, &mt, &fg))
+		return 1;
+
+	ret = stat(mnt, &st);
+	if (ret) {
+		perror(mnt);
+		goto out_free;
+	}
+
+	ret = open(lklfuse.file, lklfuse.ro ? O_RDONLY : O_RDWR);
+	if (ret < 0) {
+		perror(lklfuse.file);
+		goto out_free;
+	}
+
+	lklfuse.disk.fd = ret;
+	lklfuse.disk.dev = (char *)lklfuse.file;
+
+	ret = lkl_disk_add(&lklfuse.disk);
+	if (ret < 0) {
+		fprintf(stderr, "can't add disk: %s\n", lkl_strerror(ret));
+		goto out_close_disk;
+	}
+
+	lklfuse.disk_id = ret;
+
+	ch = fuse_mount(mnt, &args);
+	if (!ch) {
+		ret = -1;
+		goto out_close_disk;
+	}
+
+	fuse = fuse_new(ch, &args, &lklfuse_ops, sizeof(lklfuse_ops), NULL);
+	if (!fuse) {
+		ret = -1;
+		goto out_fuse_unmount;
+	}
+
+	fuse_opt_free_args(&args);
+
+	if (fuse_daemonize(fg) ||
+	    fuse_set_signal_handlers(fuse_get_session(fuse))) {
+		ret = -1;
+		goto out_fuse_destroy;
+	}
+
+	ret = start_lkl();
+	if (ret) {
+		ret = -1;
+		goto out_remove_signals;
+	}
+
+	if (mt)
+		fprintf(stderr, "warning: multithreaded mode not supported\n");
+
+	ret = fuse_loop(fuse);
+
+	stop_lkl();
+
+out_remove_signals:
+	fuse_remove_signal_handlers(fuse_get_session(fuse));
+
+out_fuse_unmount:
+	if (ch)
+		fuse_unmount(mnt, ch);
+
+out_fuse_destroy:
+	if (fuse)
+		fuse_destroy(fuse);
+
+out_close_disk:
+	close(lklfuse.disk.fd);
+
+out_free:
+	free(mnt);
+
+	return ret < 0 ? 1 : 0;
+}
diff --git a/tools/lkl/tests/lklfuse.sh b/tools/lkl/tests/lklfuse.sh
new file mode 100755
index 000000000000..7f35dd53fc4e
--- /dev/null
+++ b/tools/lkl/tests/lklfuse.sh
@@ -0,0 +1,110 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+
+source $script_dir/test.sh
+
+cleanup()
+{
+    set -e
+
+    sleep 1
+    fusermount -u $dir
+    rm $file
+    rmdir $dir
+}
+
+
+# $1 - disk image
+# $2 - fstype
+function prepfs()
+{
+    set -e
+
+    dd if=/dev/zero of=$1 bs=1024 count=102400
+
+    yes | mkfs.$2 $1
+}
+
+# $1 - disk image
+# $2 - mount point
+# $3 - filesystem type
+lklfuse_mount()
+{
+    ${script_dir}/../lklfuse $1 $2 -o type=$3
+}
+
+# $1 - mount point
+lklfuse_basic()
+{
+    set -e
+
+    cd $1
+    touch a
+    if ! [ -e ]; then exit 1; fi
+    rm a
+    mkdir a
+    if ! [ -d ]; then exit 1; fi
+    rmdir a
+}
+
+# $1 - dir
+# $2 - filesystem type
+lklfuse_stressng()
+{
+    set -e
+
+    if [ -z $(which stress-ng) ]; then
+        echo "missing stress-ng"
+        return $TEST_SKIP
+    fi
+
+    cd $1
+
+    if [ "$2" = "vfat" ]; then
+        exclude="chmod,filename,link,mknod,symlink,xattr"
+    fi
+
+    stress-ng --class filesystem --all 0 --timeout 10 \
+	      --exclude fiemap,$exclude --fallocate-bytes 10m \
+	      --sync-file-bytes 10m
+}
+
+if [ "$1" = "-t" ]; then
+    shift
+    fstype=$1
+    shift
+fi
+
+if [ -z "$fstype" ]; then
+    fstype="ext4"
+fi
+
+if ! [ -x $script_dir/../lklfuse ]; then
+    lkl_test_plan 0 "lklfuse.sh $fstype"
+    echo "lklfuse not available"
+    exit 0
+fi
+
+if ! [ -e /dev/fuse ]; then
+    lkl_test_plan 0 "lklfuse.sh $fstype"
+    echo "/dev/fuse not available"
+    exit 0
+fi
+
+
+file=`mktemp`
+dir=`mktemp -d`
+
+trap cleanup EXIT
+
+lkl_test_plan 4 "lklfuse $fstype"
+
+lkl_test_run 1 prepfs $file $fstype
+lkl_test_run 2 lklfuse_mount $file $dir $fstype
+lkl_test_run 3 lklfuse_basic $dir
+# stress-ng returns 2 with no apparent failures so skip it for now
+#lkl_test_run 4 lklfuse_stressng $dir $fstype
+trap : EXIT
+lkl_test_run 4 cleanup
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v3 22/26] um lkl: add documentation
  2020-02-05  7:30 [RFC v3 00/26] Unifying LKL into UML Hajime Tazaki
@ 2020-02-05  7:30   ` Hajime Tazaki
       [not found] ` <cover.1580882335.git.thehajime-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
                     ` (25 subsequent siblings)
  26 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05  7:30 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Chenyang Zhong, Conrad Meyer, Gustavo Bittencourt, Motomu Utsumi,
	Patrick Collins, Thomas Liebetraut, Yuan Liu

From: Octavian Purdila <tavi.purdila@gmail.com>

A document describing brief introduction of LKL, why it is needed, and
how it is used is added.  The document is located under uml/ directory.

Cc: Chenyang Zhong <zhongcy95@gmail.com>
Cc: Conrad Meyer <cem@FreeBSD.org>
Cc: Gustavo Bittencourt <gbitten@gmail.com>
Cc: H.K. Jerry Chu <hkchu@google.com>
Cc: Motomu Utsumi <motomuman@gmail.com>
Cc: Patrick Collins <pscollins@google.com>
Cc: Thomas Liebetraut <thomas@tommie-lie.de>
Cc: Yuan Liu <liuyuan@google.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 Documentation/virt/uml/lkl.txt | 64 ++++++++++++++++++++++++++++++++++
 1 file changed, 64 insertions(+)
 create mode 100644 Documentation/virt/uml/lkl.txt

diff --git a/Documentation/virt/uml/lkl.txt b/Documentation/virt/uml/lkl.txt
new file mode 100644
index 000000000000..f6d581092331
--- /dev/null
+++ b/Documentation/virt/uml/lkl.txt
@@ -0,0 +1,64 @@
+
+Introduction
+============
+
+LKL (Linux Kernel Library) is aiming to allow reusing the Linux kernel code as
+extensively as possible with minimal effort and reduced maintenance overhead.
+
+Examples of how LKL can be used are: creating userspace applications (running on
+Linux and other operating systems) that can read or write Linux filesystems or
+can use the Linux networking stack, creating kernel drivers for other operating
+systems that can read Linux filesystems, bootloaders support for reading/writing
+Linux filesystems, etc.
+
+With LKL, the kernel code is compiled into an object file that can be directly
+linked by applications. The API offered by LKL is based on the Linux system call
+interface.
+
+LKL is implemented as one of the mode of UML (arch/um). It uses host operations
+defined by the application or a host library (tools/lkl/lib).
+
+
+Supported hosts
+===============
+
+The supported host for now is Linux (x86 architecture) userspace applications.
+
+
+Building LKL the host library and LKL applications
+==================================================
+
+    $ make -C tools/lkl
+
+will build LKL as a object file, it will install it in tools/lkl/lib together
+with the headers files in tools/lkl/include then will build the host library,
+tests and a few of application examples:
+
+* tests/boot - a simple applications that uses LKL and exercises the basic LKL
+  APIs
+
+* tests/net-test - a simple applications that uses network feature of LKL and
+  exercises the basic network-related APIs
+
+* fs2tar - a tool that converts a filesystem image to a tar archive
+
+* cptofs/cpfromfs - a tool that copies files to/from a filesystem image
+
+* lklfuse - a tool that can mount a filesystem image in userspace,
+  without root privileges, using FUSE
+
+
+Building LKL on Ubuntu
+-----------------------
+
+    $ sudo apt-get install libfuse-dev libarchive-dev xfsprogs
+
+    # Optional, if you would like to be able to run tests
+    $ sudo apt-get install btrfs-tools
+    $ pip install yamlish junit_xml
+
+    $ make -C tools/lkl
+
+    # To check that everything works:
+    $ cd tools/lkl
+    $ make run-tests
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v3 22/26] um lkl: add documentation
@ 2020-02-05  7:30   ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05  7:30 UTC (permalink / raw)
  To: linux-um
  Cc: linux-arch, Conrad Meyer, Octavian Purdila, Motomu Utsumi,
	Akira Moroo, Thomas Liebetraut, Patrick Collins,
	linux-kernel-library, Chenyang Zhong, Yuan Liu,
	Gustavo Bittencourt

From: Octavian Purdila <tavi.purdila@gmail.com>

A document describing brief introduction of LKL, why it is needed, and
how it is used is added.  The document is located under uml/ directory.

Cc: Chenyang Zhong <zhongcy95@gmail.com>
Cc: Conrad Meyer <cem@FreeBSD.org>
Cc: Gustavo Bittencourt <gbitten@gmail.com>
Cc: H.K. Jerry Chu <hkchu@google.com>
Cc: Motomu Utsumi <motomuman@gmail.com>
Cc: Patrick Collins <pscollins@google.com>
Cc: Thomas Liebetraut <thomas@tommie-lie.de>
Cc: Yuan Liu <liuyuan@google.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 Documentation/virt/uml/lkl.txt | 64 ++++++++++++++++++++++++++++++++++
 1 file changed, 64 insertions(+)
 create mode 100644 Documentation/virt/uml/lkl.txt

diff --git a/Documentation/virt/uml/lkl.txt b/Documentation/virt/uml/lkl.txt
new file mode 100644
index 000000000000..f6d581092331
--- /dev/null
+++ b/Documentation/virt/uml/lkl.txt
@@ -0,0 +1,64 @@
+
+Introduction
+============
+
+LKL (Linux Kernel Library) is aiming to allow reusing the Linux kernel code as
+extensively as possible with minimal effort and reduced maintenance overhead.
+
+Examples of how LKL can be used are: creating userspace applications (running on
+Linux and other operating systems) that can read or write Linux filesystems or
+can use the Linux networking stack, creating kernel drivers for other operating
+systems that can read Linux filesystems, bootloaders support for reading/writing
+Linux filesystems, etc.
+
+With LKL, the kernel code is compiled into an object file that can be directly
+linked by applications. The API offered by LKL is based on the Linux system call
+interface.
+
+LKL is implemented as one of the mode of UML (arch/um). It uses host operations
+defined by the application or a host library (tools/lkl/lib).
+
+
+Supported hosts
+===============
+
+The supported host for now is Linux (x86 architecture) userspace applications.
+
+
+Building LKL the host library and LKL applications
+==================================================
+
+    $ make -C tools/lkl
+
+will build LKL as a object file, it will install it in tools/lkl/lib together
+with the headers files in tools/lkl/include then will build the host library,
+tests and a few of application examples:
+
+* tests/boot - a simple applications that uses LKL and exercises the basic LKL
+  APIs
+
+* tests/net-test - a simple applications that uses network feature of LKL and
+  exercises the basic network-related APIs
+
+* fs2tar - a tool that converts a filesystem image to a tar archive
+
+* cptofs/cpfromfs - a tool that copies files to/from a filesystem image
+
+* lklfuse - a tool that can mount a filesystem image in userspace,
+  without root privileges, using FUSE
+
+
+Building LKL on Ubuntu
+-----------------------
+
+    $ sudo apt-get install libfuse-dev libarchive-dev xfsprogs
+
+    # Optional, if you would like to be able to run tests
+    $ sudo apt-get install btrfs-tools
+    $ pip install yamlish junit_xml
+
+    $ make -C tools/lkl
+
+    # To check that everything works:
+    $ cd tools/lkl
+    $ make run-tests
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v3 23/26] um lkl: add CI scripts to conduct regression tests
  2020-02-05  7:30 [RFC v3 00/26] Unifying LKL into UML Hajime Tazaki
                   ` (21 preceding siblings ...)
  2020-02-05  7:30   ` Hajime Tazaki
@ 2020-02-05  7:30 ` Hajime Tazaki
  2020-02-05  7:30 ` [RFC v3 24/26] um lkl: add UML network driver for lkl Hajime Tazaki
                   ` (3 subsequent siblings)
  26 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05  7:30 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Hajime Tazaki

For the continuous integration (CI), we use CircleCI with multiple test
targets triggered with a single commit.  The test also conducts memory
checks using valgrind in order to prevent enbug.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 .circleci/config.yml             | 196 +++++++++++++++++++++++++++++++
 tools/lkl/scripts/checkpatch.sh  |  60 ++++++++++
 tools/lkl/scripts/lkl-jenkins.sh |  21 ++++
 3 files changed, 277 insertions(+)
 create mode 100644 .circleci/config.yml
 create mode 100755 tools/lkl/scripts/checkpatch.sh
 create mode 100755 tools/lkl/scripts/lkl-jenkins.sh

diff --git a/.circleci/config.yml b/.circleci/config.yml
new file mode 100644
index 000000000000..5bdf059014c0
--- /dev/null
+++ b/.circleci/config.yml
@@ -0,0 +1,196 @@
+version: 2
+general:
+  artifacts:
+
+do_steps: &do_steps
+ steps:
+  - run: echo "$CROSS_COMPILE" > ~/_cross_compile
+  - restore_cache:
+      key: code-tree-shallow-{{ .Environment.CACHE_VERSION }}
+  - run:
+      name: checkout build tree
+      command: |
+        mkdir -p ~/.ssh/
+        ssh-keyscan -H github.com >> ~/.ssh/known_hosts
+        if ! [ -d .git ]; then
+          git clone --depth=1 $CIRCLE_REPOSITORY_URL .;
+        fi
+        if [[ $CIRCLE_BRANCH == pull/* ]]; then
+           git fetch --depth=1 origin $CIRCLE_BRANCH/head;
+        else
+           git fetch --depth=1 origin $CIRCLE_BRANCH;
+        fi
+        git reset --hard $CIRCLE_SHA1
+  - save_cache:
+      key: code-tree-shallow-{{ .Environment.CACHE_VERSION }}-{{ epoch }}
+      paths:
+        - /home/ubuntu/project/.git
+  - run:
+      name: clean
+      command: |
+        make mrproper
+        cd tools/lkl && make clean-conf
+        rm -rf ~/junit
+  - run: mkdir -p /home/ubuntu/.ccache
+  - restore_cache:
+      key: compiler-cache-{{ checksum "~/_cross_compile" }}-{{ .Environment.CACHE_VERSION }}
+  - run: cd tools/lkl && make -j8 ${MKARG}
+  - run: mkdir -p ~/destdir && cd tools/lkl && make DESTDIR=~/destdir
+  - save_cache:
+     paths:
+       - /home/ubuntu/.ccache
+     key: compiler-cache-{{ checksum "~/_cross_compile" }}-{{ .Environment.CACHE_VERSION }}-{{ epoch }}
+  - run:
+      name: run tests
+      command: |
+        mkdir -p ~/junit
+        make -C tools/lkl run-tests tests="--junit-dir ~/junit"
+      no_output_timeout: "90m"
+  - run:
+      name: collecting test results
+      command: |
+        find ./tools/lkl/ -type f -name "*.xml" -exec mv {} ~/junit/ \;
+  - store_test_results:
+      path: ~/junit
+  - store_artifacts:
+      path: ~/junit
+
+
+do_uml_steps: &do_uml_steps
+ steps:
+  - run: echo "$CROSS_COMPILE" > ~/_cross_compile
+  - restore_cache:
+      key: code-tree-shallow-{{ .Environment.CACHE_VERSION }}
+  - run:
+      name: checkout build tree
+      command: |
+        mkdir -p ~/.ssh/
+        ssh-keyscan -H github.com >> ~/.ssh/known_hosts
+        if ! [ -d .git ]; then
+          git clone --depth=1 $CIRCLE_REPOSITORY_URL .;
+        fi
+        if [[ $CIRCLE_BRANCH == pull/* ]]; then
+           git fetch --depth=1 origin $CIRCLE_BRANCH/head;
+        else
+           git fetch --depth=1 origin $CIRCLE_BRANCH;
+        fi
+        git reset --hard $CIRCLE_SHA1
+  - save_cache:
+      key: code-tree-shallow-{{ .Environment.CACHE_VERSION }}-{{ epoch }}
+      paths:
+        - /home/ubuntu/project/.git
+  - run: mkdir -p /home/ubuntu/.ccache
+  - restore_cache:
+      key: compiler-cache-{{ checksum "~/_cross_compile" }}-{{ .Environment.CACHE_VERSION }}
+  - run:
+      name: build
+      command: |
+        sudo apt-get update
+        sudo apt-get install -y gcc-multilib g++-multilib
+        make defconfig ARCH=um SUBARCH=$SUBARCH
+        make ARCH=um SUBARCH=$SUBARCH
+  - save_cache:
+     paths:
+       - /home/ubuntu/.ccache
+     key: compiler-cache-{{ checksum "~/_cross_compile" }}-{{ .Environment.CACHE_VERSION }}-{{ epoch }}
+  - run:
+      name: test
+      command: |
+        # XXX: i386 build doesn't work with the test
+        if [ $CIRCLE_STAGE = "i386_uml" ] || [ $CIRCLE_STAGE = "i386_uml_on_x86_64" ]; then
+          exit 0
+        fi
+        ./linux rootfstype=hostfs ro mem=1g loglevel=10 init="/bin/bash -c exit" || export RETVAL=$?
+        # SIGABRT=6 => 128+6
+        if [ $RETVAL != "134" ]; then
+          exit 1
+        fi
+
+## Customize the test machine
+jobs:
+  x86_64:
+   docker:
+     - image: lkldocker/circleci-x86_64:0.7
+   environment:
+     CROSS_COMPILE: ""
+   <<: *do_steps
+
+  i386:
+   docker:
+     - image: lkldocker/circleci-i386:0.1
+   environment:
+     CROSS_COMPILE: ""
+   <<: *do_steps
+
+  x86_64_valgrind:
+   docker:
+     - image: lkldocker/circleci-x86_64:0.7
+   environment:
+     CROSS_COMPILE: ""
+     VALGRIND: 1
+   <<: *do_steps
+
+  x86_64_uml:
+   docker:
+     - image: lkldocker/circleci-x86_64:0.7
+   environment:
+     CROSS_COMPILE: ""
+     TMPDIR: "/tmp" # required for not using /dev/shm
+     SUBARCH: "x86_64"
+   <<: *do_uml_steps
+
+  i386_uml:
+   docker:
+     - image: lkldocker/circleci-i386:0.1
+   environment:
+     CROSS_COMPILE: ""
+     SUBARCH: "i386"
+     TMPDIR: "/tmp" # required for not using /dev/shm
+   <<: *do_uml_steps
+
+  i386_uml_on_x86_64:
+   docker:
+     - image: lkldocker/circleci-x86_64:0.7
+   environment:
+     CROSS_COMPILE: ""
+     TMPDIR: "/tmp" # required for not using /dev/shm
+     SUBARCH: "i386"
+   <<: *do_uml_steps
+
+  checkpatch:
+   docker:
+     - image: lkldocker/circleci:0.5
+   environment:
+   steps:
+     - restore_cache:
+        key: code-tree-full-history-{{ .Environment.CACHE_VERSION }}
+     - checkout
+     - run: sudo pip install ply
+     - run: tools/lkl/scripts/checkpatch.sh
+     - save_cache:
+        key: code-tree-full-history-{{ .Environment.CACHE_VERSION }}-{{ epoch }}
+        paths:
+          - /home/ubuntu/project/.git
+        when: always
+
+workflows:
+  version: 2
+  build:
+    jobs:
+     - x86_64
+     - x86_64_valgrind
+     - checkpatch
+     - i386
+     - x86_64_uml
+     - i386_uml
+     - i386_uml_on_x86_64
+  nightly:
+    triggers:
+      - schedule:
+          cron: "0 0 * * *"
+          filters:
+            branches:
+              only:
+                - master
+    jobs:
+      - x86_64_valgrind
diff --git a/tools/lkl/scripts/checkpatch.sh b/tools/lkl/scripts/checkpatch.sh
new file mode 100755
index 000000000000..0c02ca6b21a2
--- /dev/null
+++ b/tools/lkl/scripts/checkpatch.sh
@@ -0,0 +1,60 @@
+#!/bin/sh -ex
+# SPDX-License-Identifier: GPL-2.0
+
+if [ -z "$origin_master" ]; then
+    origin_master="origin/master"
+fi
+
+UPSTREAM=git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
+LKL=github.com:lkl/linux.git
+
+upstream=`git remote -v | grep $UPSTREAM | cut -f1 | head -n1`
+lkl=`git remote -v | grep $LKL | cut -f1 | head -n1`
+
+if [ -z "$upstream" ]; then
+    git fetch --tags --progress git://$UPSTREAM
+else
+    git fetch --tags $upstream
+fi
+
+if [ -z "$lkl" ]; then
+    git remote add lkl-upstream git@$LKL || true
+    lkl=`git remote -v | grep $LKL | cut -f1 | head -n1`
+fi
+
+if [ -z "$lkl" ]; then
+    echo "can't find lkl remote, quiting"
+    exit 1
+fi
+
+git fetch $lkl
+git fetch --tags $upstream
+
+# find the last upstream tag to avoid checking upstream commits during
+# upstream merges
+tag=`git tag --sort='-*authordate' | grep ^v | head -n1`
+tmp=`mktemp -d`
+
+commits=$(git log --no-merges --pretty=format:%h HEAD ^$lkl/master ^$tag)
+for c in $commits; do
+    git format-patch -1 -o $tmp $c
+done
+
+if [ -z "$c" ]; then
+    echo "there are not commits/patches to check, quiting."
+    rmdir $tmp
+    exit 0
+fi
+
+./scripts/checkpatch.pl --ignore FILE_PATH_CHANGES $tmp/*.patch
+rm $tmp/*.patch
+
+# checkpatch.pl does not know how to deal with 3 way diffs which would
+# be useful to check the conflict resolutions during merges...
+#for c in `git log --merges --pretty=format:%h HEAD ^$origin_master ^$tag`; do
+#    git log --pretty=email $c -1 > $tmp/$c.patch
+#    git diff $c $c^1 $c^2 >> $tmp/$c.patch
+#done
+
+rmdir $tmp
+
diff --git a/tools/lkl/scripts/lkl-jenkins.sh b/tools/lkl/scripts/lkl-jenkins.sh
new file mode 100755
index 000000000000..eaadc6e90143
--- /dev/null
+++ b/tools/lkl/scripts/lkl-jenkins.sh
@@ -0,0 +1,21 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+set -e
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+basedir=$(cd $script_dir/../../..; pwd)
+
+export PATH=$PATH:/sbin
+
+build_and_test()
+{
+    cd $basedir
+    make mrproper
+    cd tools/lkl
+    make clean-conf
+    make -j4
+    make run-tests
+}
+
+build_and_test
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v3 24/26] um lkl: add UML network driver for lkl
  2020-02-05  7:30 [RFC v3 00/26] Unifying LKL into UML Hajime Tazaki
                   ` (22 preceding siblings ...)
  2020-02-05  7:30 ` [RFC v3 23/26] um lkl: add CI scripts to conduct regression tests Hajime Tazaki
@ 2020-02-05  7:30 ` Hajime Tazaki
  2020-02-05  7:30 ` [RFC v3 25/26] um lkl: add UML block device driver (ubd) " Hajime Tazaki
                   ` (2 subsequent siblings)
  26 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05  7:30 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Hajime Tazaki

This commit adds the use of the UML drivers of network interfaces which
should be able to use any backend configured by boot command line
parameters.  Currently slirp and tuntap backends are tested.

Since the UML drivers use LKL irq infrastructure, the build system only
picks required object files of UML, and LKL adds trivial glue code and
ifdefs into UML code so that existing driver codes can be used for LKL
as-is.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 .circleci/config.yml                    |  2 ++
 arch/um/drivers/Makefile                |  2 ++
 arch/um/kernel/Makefile                 |  4 +++
 arch/um/kernel/irq.c                    |  4 +++
 arch/um/lkl/Kconfig                     |  2 ++
 arch/um/lkl/Makefile                    |  2 ++
 arch/um/lkl/include/uapi/asm/host_ops.h |  6 +++++
 arch/um/lkl/kernel/asm-offsets.c        |  1 +
 arch/um/lkl/kernel/irq.c                | 16 ++++++++++++
 arch/um/lkl/kernel/setup.c              |  4 +++
 arch/um/lkl/mm/bootmem.c                | 33 +++++++++++++++++++++++++
 arch/um/os-Linux/Makefile               |  5 ++++
 arch/um/os-Linux/file.c                 |  9 +++++++
 tools/lkl/Makefile.autoconf             |  2 ++
 tools/lkl/include/lkl.h                 |  9 +++++++
 tools/lkl/include/lkl_host.h            |  1 +
 tools/lkl/lib/Build                     |  1 +
 tools/lkl/lib/config.c                  |  6 +++++
 tools/lkl/lib/posix-host.c              |  3 +++
 tools/lkl/lib/um/Build                  |  2 ++
 tools/lkl/lib/um/um_glue.c              | 33 +++++++++++++++++++++++++
 tools/lkl/lib/um/um_net.c               | 25 +++++++++++++++++++
 tools/lkl/tests/net-setup.sh            |  1 +
 tools/lkl/tests/net-test.c              | 14 +++++++----
 tools/lkl/tests/net.sh                  | 18 ++++++++++++++
 tools/lkl/tests/run.py                  |  1 +
 26 files changed, 201 insertions(+), 5 deletions(-)
 create mode 100644 tools/lkl/lib/um/Build
 create mode 100644 tools/lkl/lib/um/um_glue.c
 create mode 100644 tools/lkl/lib/um/um_net.c

diff --git a/.circleci/config.yml b/.circleci/config.yml
index 5bdf059014c0..df1554078a4b 100644
--- a/.circleci/config.yml
+++ b/.circleci/config.yml
@@ -43,6 +43,8 @@ do_steps: &do_steps
   - run:
       name: run tests
       command: |
+        sudo apt-get update
+        sudo apt-get install -y slirp
         mkdir -p ~/junit
         make -C tools/lkl run-tests tests="--junit-dir ~/junit"
       no_output_timeout: "90m"
diff --git a/arch/um/drivers/Makefile b/arch/um/drivers/Makefile
index a290821e355c..3cce6c81079e 100644
--- a/arch/um/drivers/Makefile
+++ b/arch/um/drivers/Makefile
@@ -37,7 +37,9 @@ $(obj)/vde.o: $(obj)/vde_kern.o $(obj)/vde_user.o
 # When the above is fixed, don't forget to add this too!
 #targets += $(obj)/pcap.o
 
+ifeq ($(UMMODE),kernel)
 obj-y := stdio_console.o fd.o chan_kern.o chan_user.o line.o
+endif
 obj-$(CONFIG_SSL) += ssl.o
 obj-$(CONFIG_STDERR_CONSOLE) += stderr_console.o
 
diff --git a/arch/um/kernel/Makefile b/arch/um/kernel/Makefile
index 5aa882011e04..78b49eb9bf81 100644
--- a/arch/um/kernel/Makefile
+++ b/arch/um/kernel/Makefile
@@ -12,12 +12,16 @@ CPPFLAGS_vmlinux.lds := -DSTART=$(LDS_START)		\
                         -DELF_ARCH=$(LDS_ELF_ARCH)	\
                         -DELF_FORMAT=$(LDS_ELF_FORMAT)	\
 			$(LDS_EXTRA)
+ifeq ($(UMMODE),kernel)
 extra-y := vmlinux.lds
 
 obj-y = config.o exec.o exitcode.o irq.o ksyms.o mem.o \
 	physmem.o process.o ptrace.o reboot.o sigio.o \
 	signal.o syscall.o sysrq.o time.o tlb.o trap.o \
 	um_arch.o umid.o maccess.o kmsg_dump.o skas/
+else ifeq ($(UMMODE),library)
+obj-y = irq.o
+endif
 
 obj-$(CONFIG_BLK_DEV_INITRD) += initrd.o
 obj-$(CONFIG_GPROF)	+= gprof_syms.o
diff --git a/arch/um/kernel/irq.c b/arch/um/kernel/irq.c
index 3577118bb4a5..580a889c2277 100644
--- a/arch/um/kernel/irq.c
+++ b/arch/um/kernel/irq.c
@@ -406,6 +406,7 @@ int deactivate_all_fds(void)
 	return 0;
 }
 
+#ifdef CONFIG_UMMODE_KERN
 /*
  * do_IRQ handles all normal device IRQs (the special
  * SMP cross-CPU interrupts have their own specific
@@ -420,6 +421,7 @@ unsigned int do_IRQ(int irq, struct uml_pt_regs *regs)
 	set_irq_regs(old_regs);
 	return 1;
 }
+#endif /* !CONFIG_UMMODE_KERN */
 
 void um_free_irq(unsigned int irq, void *dev)
 {
@@ -446,6 +448,7 @@ int um_request_irq(unsigned int irq, int fd, int type,
 
 EXPORT_SYMBOL(um_request_irq);
 
+#ifdef CONFIG_UMMODE_KERN
 /*
  * irq_chip must define at least enable/disable and ack when
  * the edge handler is used.
@@ -597,3 +600,4 @@ unsigned long from_irq_stack(int nested)
 	return mask & ~1;
 }
 
+#endif /* !CONFIG_UMMODE_KERN */
diff --git a/arch/um/lkl/Kconfig b/arch/um/lkl/Kconfig
index f72b423fad5b..3568f33d5ccf 100644
--- a/arch/um/lkl/Kconfig
+++ b/arch/um/lkl/Kconfig
@@ -81,3 +81,5 @@ config HZ
         default 100
 
 endmenu
+
+source "arch/um/drivers/Kconfig"
diff --git a/arch/um/lkl/Makefile b/arch/um/lkl/Makefile
index e1161fa3fb63..17ff6d00d1db 100644
--- a/arch/um/lkl/Makefile
+++ b/arch/um/lkl/Makefile
@@ -4,6 +4,8 @@ include $(LKL_DIR)/auto.conf
 
 # fixup CFLAGS of um build
 KBUILD_CFLAGS := $(subst $(CFLAGS),,$(KBUILD_CFLAGS))
+# XXX
+KBUILD_CFLAGS	+= -DTIMER_IRQ=0 -DUM_ETH_IRQ=5 -DLAST_IRQ=15
 
 SRCARCH := um/lkl
 ARCH_INCLUDE += -I$(srctree)/$(LKL_DIR)/um/include
diff --git a/arch/um/lkl/include/uapi/asm/host_ops.h b/arch/um/lkl/include/uapi/asm/host_ops.h
index fe4382c3050a..0230885f4f64 100644
--- a/arch/um/lkl/include/uapi/asm/host_ops.h
+++ b/arch/um/lkl/include/uapi/asm/host_ops.h
@@ -17,6 +17,10 @@ struct lkl_jmp_buf {
  * These operations must be provided by a host library or by the application
  * itself.
  *
+ * @um_devices - string containg the list of UML devices in command line
+ * format. This string is appended to the kernel command line and
+ * is provided here for convenience to be implemented by the host library.
+ *
  * @print - optional operation that receives console messages
  *
  * @panic - called during a kernel panic
@@ -74,6 +78,8 @@ struct lkl_jmp_buf {
  * @jmp_buf_longjmp - perform a jump back to the saved jump buffer
  */
 struct lkl_host_operations {
+	const char *um_devices;
+
 	void (*print)(const char *str, int len);
 	void (*panic)(void);
 
diff --git a/arch/um/lkl/kernel/asm-offsets.c b/arch/um/lkl/kernel/asm-offsets.c
index 6be0763698dc..6fdca6df21a2 100644
--- a/arch/um/lkl/kernel/asm-offsets.c
+++ b/arch/um/lkl/kernel/asm-offsets.c
@@ -1,2 +1,3 @@
 // SPDX-License-Identifier: GPL-2.0
 /* Dummy asm-offsets.c file. Required by kbuild and ready to be used - hint! */
+#include <sysdep/kernel-offsets.h>
diff --git a/arch/um/lkl/kernel/irq.c b/arch/um/lkl/kernel/irq.c
index e3b59e46ca50..7de65b8f699d 100644
--- a/arch/um/lkl/kernel/irq.c
+++ b/arch/um/lkl/kernel/irq.c
@@ -11,6 +11,10 @@
 #include <asm/host_ops.h>
 #include <asm/cpu.h>
 
+#if defined(__linux) && (defined(__i386) || defined(__x86_64))
+#include <os.h>
+#endif
+
 /*
  * To avoid much overhead we use an indirect approach: the irqs are marked using
  * a bitmap (array of longs) and a summary of the modified bits is kept in a
@@ -184,6 +188,10 @@ void init_IRQ(void)
 	for (i = 0; i < NR_IRQS; i++)
 		irq_set_chip_and_handler(i, &dummy_irq_chip, handle_simple_irq);
 
+#if defined(__linux) && (defined(__i386) || defined(__x86_64))
+	/* Initialize EPOLL Loop */
+	os_setup_epoll();
+#endif
 	pr_info("lkl: irqs initialized\n");
 }
 
@@ -191,3 +199,11 @@ void cpu_yield_to_irqs(void)
 {
 	cpu_relax();
 }
+
+#if defined(__linux) && (defined(__i386) || defined(__x86_64))
+unsigned int do_IRQ(int irq, struct uml_pt_regs *regs)
+{
+	lkl_trigger_irq(irq);
+	return 1;
+}
+#endif
diff --git a/arch/um/lkl/kernel/setup.c b/arch/um/lkl/kernel/setup.c
index 36c199d3aa22..39b7b7c79581 100644
--- a/arch/um/lkl/kernel/setup.c
+++ b/arch/um/lkl/kernel/setup.c
@@ -61,6 +61,10 @@ int __init lkl_start_kernel(struct lkl_host_operations *ops, const char *fmt,
 	ret = vsnprintf(boot_command_line, COMMAND_LINE_SIZE, fmt, ap);
 	va_end(ap);
 
+	if (ops->um_devices)
+		strscpy(boot_command_line + ret, ops->um_devices,
+			COMMAND_LINE_SIZE - ret);
+
 	memcpy(cmd_line, boot_command_line, COMMAND_LINE_SIZE);
 
 	init_sem = lkl_ops->sem_alloc(0);
diff --git a/arch/um/lkl/mm/bootmem.c b/arch/um/lkl/mm/bootmem.c
index 39dd0d22b44e..3175dadee13f 100644
--- a/arch/um/lkl/mm/bootmem.c
+++ b/arch/um/lkl/mm/bootmem.c
@@ -64,3 +64,36 @@ void free_mem(void)
 {
 	lkl_ops->mem_free((void *)_memory_start);
 }
+
+void *uml_kmalloc(int size, int flags)
+{
+	return kmalloc(size, flags);
+}
+
+char *uml_strdup(const char *string)
+{
+	return kstrdup(string, GFP_KERNEL);
+}
+
+void free_stack(unsigned long stack, int order)
+{
+	free_pages(stack, order);
+}
+
+unsigned long alloc_stack(int order, int atomic)
+{
+	unsigned long page;
+	gfp_t flags = GFP_KERNEL;
+
+	if (atomic)
+		flags = GFP_ATOMIC;
+	page = __get_free_pages(flags, order);
+
+	return page;
+}
+
+int __cant_sleep(void)
+{
+	return in_atomic() || irqs_disabled() || in_interrupt();
+	/* Is in_interrupt() really needed? */
+}
diff --git a/arch/um/os-Linux/Makefile b/arch/um/os-Linux/Makefile
index 839915b8c31c..a74a2486e178 100644
--- a/arch/um/os-Linux/Makefile
+++ b/arch/um/os-Linux/Makefile
@@ -6,9 +6,14 @@
 # Don't instrument UML-specific code
 KCOV_INSTRUMENT                := n
 
+ifeq ($(UMMODE),kernel)
 obj-y = execvp.o file.o helper.o irq.o main.o mem.o process.o \
 	registers.o sigio.o signal.o start_up.o time.o tty.o \
 	umid.o user_syms.o util.o drivers/ skas/
+else
+obj-y = execvp.o file.o helper.o irq.o drivers/
+endif
+
 
 obj-$(CONFIG_ARCH_REUSE_HOST_VSYSCALL_AREA) += elf_aux.o
 
diff --git a/arch/um/os-Linux/file.c b/arch/um/os-Linux/file.c
index 5133e3afb96f..6656a2ec418a 100644
--- a/arch/um/os-Linux/file.c
+++ b/arch/um/os-Linux/file.c
@@ -397,6 +397,14 @@ int os_pipe(int *fds, int stream, int close_on_exec)
 	return err;
 }
 
+static void sig_handler(int sig)
+{
+	if (sig != SIGIO)
+		return;
+
+	sigio_handler(sig, NULL, NULL);
+}
+
 int os_set_fd_async(int fd)
 {
 	int err, flags;
@@ -413,6 +421,7 @@ int os_set_fd_async(int fd)
 		return err;
 	}
 
+	signal(SIGIO, sig_handler);
 	if ((fcntl(fd, F_SETSIG, SIGIO) < 0) ||
 	    (fcntl(fd, F_SETOWN, os_getpid()) < 0)) {
 		err = -errno;
diff --git a/tools/lkl/Makefile.autoconf b/tools/lkl/Makefile.autoconf
index 268c367d9962..d2dcaa3b85f2 100644
--- a/tools/lkl/Makefile.autoconf
+++ b/tools/lkl/Makefile.autoconf
@@ -41,6 +41,8 @@ define posix_host
   $(if $(strip $(call find_include,archive.h)),$(call set_autoconf_var,ARCHIVE,y))
   $(if $(filter $(1),elf64-x86-64-freebsd),$(call set_autoconf_var,NEEDS_LARGP,y))
   $(if $(filter $(1),elf32-i386),$(call set_autoconf_var,I386,y))
+  $(if $(filter $(1),elf64-x86-64),$(call set_autoconf_var,UML_DEV,y))
+  $(if $(filter $(1),elf32-i386),$(call set_autoconf_var,UML_DEV,y))
 endef
 
 define do_autoconf
diff --git a/tools/lkl/include/lkl.h b/tools/lkl/include/lkl.h
index 1f4291ad9455..952d11e30868 100644
--- a/tools/lkl/include/lkl.h
+++ b/tools/lkl/include/lkl.h
@@ -731,6 +731,15 @@ struct lkl_netdev_args {
 	unsigned int offload;
 };
 
+#ifdef LKL_HOST_CONFIG_UML_DEV
+struct lkl_netdev *lkl_um_netdev_create(const char *ifparams);
+#else
+static inline struct lkl_netdev *lkl_um_netdev_create(const char *ifparams)
+{
+	return NULL;
+}
+#endif
+
 /*
  * lkl_register_dbg_handler- register a signal handler that loads a debug lib.
  *
diff --git a/tools/lkl/include/lkl_host.h b/tools/lkl/include/lkl_host.h
index 4e6d6e031498..7ac30bdcaf0a 100644
--- a/tools/lkl/include/lkl_host.h
+++ b/tools/lkl/include/lkl_host.h
@@ -10,6 +10,7 @@ extern "C" {
 #include <lkl.h>
 
 extern struct lkl_host_operations lkl_host_ops;
+extern char lkl_um_devs[4096];
 
 /**
  * lkl_printf - print a message via the host print operation
diff --git a/tools/lkl/lib/Build b/tools/lkl/lib/Build
index 0e711e260a3a..ad8a4b806375 100644
--- a/tools/lkl/lib/Build
+++ b/tools/lkl/lib/Build
@@ -10,3 +10,4 @@ liblkl-y += dbg.o
 liblkl-y += dbg_handler.o
 liblkl-y += ../../perf/pmu-events/jsmn.o
 liblkl-y += config.o
+liblkl-$(LKL_HOST_CONFIG_UML_DEV) += um/
diff --git a/tools/lkl/lib/config.c b/tools/lkl/lib/config.c
index 37f8ac4d942a..db3af68446d3 100644
--- a/tools/lkl/lib/config.c
+++ b/tools/lkl/lib/config.c
@@ -470,6 +470,12 @@ static int lkl_config_netdev_create(struct lkl_config *cfg,
 	memset(&nd_args, 0, sizeof(struct lkl_netdev_args));
 
 	if (!nd && iface->iftype && iface->ifparams) {
+		if ((strcmp(iface->iftype, "um") == 0)) {
+			nd = lkl_um_netdev_create(iface->ifparams);
+			iface->nd = nd;
+			/* um_netdev doesn't use virtio device */
+			return 0;
+		}
 	}
 
 	if (nd) {
diff --git a/tools/lkl/lib/posix-host.c b/tools/lkl/lib/posix-host.c
index 4be1611a8942..d6394d41e125 100644
--- a/tools/lkl/lib/posix-host.c
+++ b/tools/lkl/lib/posix-host.c
@@ -346,6 +346,9 @@ struct lkl_host_operations lkl_host_ops = {
 	.print = print,
 	.mem_alloc = (void *)malloc,
 	.mem_free = free,
+#ifdef LKL_HOST_CONFIG_UML_DEV
+	.um_devices = lkl_um_devs,
+#endif
 	.gettid = _gettid,
 	.jmp_buf_set = jmp_buf_set,
 	.jmp_buf_longjmp = jmp_buf_longjmp,
diff --git a/tools/lkl/lib/um/Build b/tools/lkl/lib/um/Build
new file mode 100644
index 000000000000..09a60975ecd6
--- /dev/null
+++ b/tools/lkl/lib/um/Build
@@ -0,0 +1,2 @@
+liblkl-y += um_glue.o
+liblkl-y += um_net.o
diff --git a/tools/lkl/lib/um/um_glue.c b/tools/lkl/lib/um/um_glue.c
new file mode 100644
index 000000000000..10ca8524e682
--- /dev/null
+++ b/tools/lkl/lib/um/um_glue.c
@@ -0,0 +1,33 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
+#include <errno.h>
+
+
+
+char lkl_um_devs[4096];
+
+/* from sigio.c */
+void maybe_sigio_broken(int fd, int read)
+{
+}
+
+/* from process.c */
+int os_getpid(void)
+{
+	return getpid();
+}
+
+
+/* from chan_kern.c */
+void free_irqs(void)
+{
+}
+
+/* from sigio.c */
+int ignore_sigio_fd(int fd)
+{
+	return 0;
+}
diff --git a/tools/lkl/lib/um/um_net.c b/tools/lkl/lib/um/um_net.c
new file mode 100644
index 000000000000..edd0c65fc08e
--- /dev/null
+++ b/tools/lkl/lib/um/um_net.c
@@ -0,0 +1,25 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <lkl_host.h>
+
+static int registered_net_dev_idx;
+
+struct lkl_netdev *lkl_um_netdev_create(const char *ifparams)
+{
+	struct lkl_netdev *nd;
+
+	nd = lkl_host_ops.mem_alloc(sizeof(struct lkl_netdev));
+	if (!nd)
+		return NULL;
+
+	memset(nd, 0, sizeof(struct lkl_netdev));
+
+	nd->id = registered_net_dev_idx++;
+	/* concat strings */
+	snprintf(lkl_um_devs + strlen(lkl_um_devs), sizeof(lkl_um_devs),
+		 " eth%d=%s", nd->id, ifparams);
+
+	return nd;
+}
diff --git a/tools/lkl/tests/net-setup.sh b/tools/lkl/tests/net-setup.sh
index 0cfc42a30a54..fa390fb904b5 100644
--- a/tools/lkl/tests/net-setup.sh
+++ b/tools/lkl/tests/net-setup.sh
@@ -8,6 +8,7 @@ TEST_IP6_NETMASK=64
 TEST_MAC0="aa:bb:cc:dd:ee:ff"
 TEST_MAC1="aa:bb:cc:dd:ee:aa"
 TEST_NETSERVER_PORT=11223
+TEST_UM_SLIRP_PARMS="slirp,,`which slirp`"
 
 # $1 - count
 # $2 - netcount
diff --git a/tools/lkl/tests/net-test.c b/tools/lkl/tests/net-test.c
index edc37150270b..c3f818742263 100644
--- a/tools/lkl/tests/net-test.c
+++ b/tools/lkl/tests/net-test.c
@@ -22,9 +22,10 @@
 
 enum {
 	BACKEND_NONE,
+	BACKEND_UM,
 };
 
-const char *backends[] = { "loopback", NULL };
+const char *backends[] = { "loopback", "um", NULL };
 static struct {
 	int backend;
 	const char *ifname;
@@ -164,12 +165,17 @@ static int lkl_test_icmp(void)
 }
 
 static struct lkl_netdev *nd;
+static int nd_id;
 
 static int lkl_test_nd_create(void)
 {
 	switch (cla.backend) {
 	case BACKEND_NONE:
 		return TEST_SKIP;
+	case BACKEND_UM:
+		nd = lkl_um_netdev_create(cla.ifname);
+		nd_id = nd->id;
+		break;
 	}
 
 	if (!nd) {
@@ -180,11 +186,9 @@ static int lkl_test_nd_create(void)
 	return TEST_SUCCESS;
 }
 
-static int nd_id;
-
 static int lkl_test_nd_add(void)
 {
-	if (cla.backend == BACKEND_NONE)
+	if (cla.backend == BACKEND_NONE || cla.backend == BACKEND_UM)
 		return TEST_SKIP;
 
 	return TEST_SUCCESS;
@@ -192,7 +196,7 @@ static int lkl_test_nd_add(void)
 
 static int lkl_test_nd_remove(void)
 {
-	if (cla.backend == BACKEND_NONE)
+	if (cla.backend == BACKEND_NONE || cla.backend == BACKEND_UM)
 		return TEST_SKIP;
 
 	return TEST_SUCCESS;
diff --git a/tools/lkl/tests/net.sh b/tools/lkl/tests/net.sh
index 32e8452b0406..089ebfb332da 100755
--- a/tools/lkl/tests/net.sh
+++ b/tools/lkl/tests/net.sh
@@ -13,6 +13,8 @@ cleanup_backend()
     case "$1" in
     "loopback")
         ;;
+    "um")
+        ;;
     esac
 }
 
@@ -47,6 +49,16 @@ setup_backend()
     case "$1" in
     "loopback")
         ;;
+    "um")
+	# only intel arch is capable with um-net backent
+	if [ -z "$LKL_HOST_CONFIG_UML_DEV" ]; then
+            return $TEST_SKIP
+	fi
+	# slirp's helper process doesn't work with valgrind
+	if [ -n "$VALGRIND" ]; then
+            return $TEST_SKIP
+	fi
+        ;;
     *)
         echo "don't know how to setup backend $1"
         return $TEST_FAILED
@@ -60,6 +72,12 @@ run_tests()
     "loopback")
         lkl_test_exec $script_dir/net-test --dst 127.0.0.1
         ;;
+    "um")
+        lkl_test_exec $script_dir/net-test --backend um \
+                      --ifname $TEST_UM_SLIRP_PARMS \
+                      --ip 10.0.2.15 --netmask-len 8 \
+                      --dst 10.0.2.2
+        ;;
     esac
 }
 
diff --git a/tools/lkl/tests/run.py b/tools/lkl/tests/run.py
index b72299aaabee..23d5c5489e2c 100755
--- a/tools/lkl/tests/run.py
+++ b/tools/lkl/tests/run.py
@@ -54,6 +54,7 @@ tests = [
     'disk.sh -t vfat',
     'disk.sh -t btrfs',
     'net.sh -b loopback',
+    'net.sh -b um',
     'lklfuse.sh -t ext4',
     'lklfuse.sh -t vfat',
     'lklfuse.sh -t btrfs',
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v3 25/26] um lkl: add UML block device driver (ubd) for lkl
  2020-02-05  7:30 [RFC v3 00/26] Unifying LKL into UML Hajime Tazaki
                   ` (23 preceding siblings ...)
  2020-02-05  7:30 ` [RFC v3 24/26] um lkl: add UML network driver for lkl Hajime Tazaki
@ 2020-02-05  7:30 ` Hajime Tazaki
  2020-02-05  7:30 ` [RFC v3 26/26] um: fix clone flags to be familar with valgrind Hajime Tazaki
  2020-03-30 14:45   ` Hajime Tazaki
  26 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05  7:30 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Hajime Tazaki

This commit adds a support to utilize exising UML drivers of block
devices for LKL.  The goal is similar to networking drivers, add minimum
amount of code and use drivers code as-is.

This commit also adds a test code with several filesystems (ext4, vfat,
btrfs at the moment) to exercise the basic operations of disk.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/drivers/ubd_kern.c              |  6 +++++-
 arch/um/lkl/Kconfig                     |  3 +++
 arch/um/lkl/Makefile                    |  2 +-
 arch/um/lkl/include/uapi/asm/syscalls.h |  1 +
 arch/um/lkl/kernel/setup.c              |  5 +++++
 tools/lkl/include/lkl.h                 | 15 ++++++++++++++
 tools/lkl/lib/um/Build                  |  1 +
 tools/lkl/lib/um/um_block.c             | 17 ++++++++++++++++
 tools/lkl/lib/um/um_glue.c              | 27 ++++++++++++++++++++++++-
 tools/lkl/tests/disk.c                  | 14 +++++++++++++
 10 files changed, 88 insertions(+), 3 deletions(-)
 create mode 100644 tools/lkl/lib/um/um_block.c

diff --git a/arch/um/drivers/ubd_kern.c b/arch/um/drivers/ubd_kern.c
index 6627d7c30f37..be1dc01de963 100644
--- a/arch/um/drivers/ubd_kern.c
+++ b/arch/um/drivers/ubd_kern.c
@@ -544,7 +544,7 @@ static irqreturn_t ubd_intr(int irq, void *dev)
 /* Only changed by ubd_init, which is an initcall. */
 static int io_pid = -1;
 
-static void kill_io_thread(void)
+void kill_io_thread(void)
 {
 	if(io_pid != -1)
 		os_kill_process(io_pid, 1);
@@ -818,6 +818,7 @@ static int ubd_open_dev(struct ubd *ubd_dev)
 	}
 	ubd_dev->fd = fd;
 
+#ifdef CONFIG_UMMODE_KERN
 	if(ubd_dev->cow.file != NULL){
 		blk_queue_max_hw_sectors(ubd_dev->queue, 8 * sizeof(long));
 
@@ -842,6 +843,7 @@ static int ubd_open_dev(struct ubd *ubd_dev)
 		if(err < 0) goto error;
 		ubd_dev->cow.fd = err;
 	}
+#endif	/* CONFIG_UMMODE_KERN */
 	if (ubd_dev->no_trim == 0) {
 		ubd_dev->queue->limits.discard_granularity = SECTOR_SIZE;
 		ubd_dev->queue->limits.discard_alignment = SECTOR_SIZE;
@@ -851,9 +853,11 @@ static int ubd_open_dev(struct ubd *ubd_dev)
 	}
 	blk_queue_flag_set(QUEUE_FLAG_NONROT, ubd_dev->queue);
 	return 0;
+#ifdef CONFIG_UMMODE_KERN
  error:
 	os_close_file(ubd_dev->fd);
 	return err;
+#endif
 }
 
 static void ubd_device_release(struct device *dev)
diff --git a/arch/um/lkl/Kconfig b/arch/um/lkl/Kconfig
index 3568f33d5ccf..fdd7ef2653a3 100644
--- a/arch/um/lkl/Kconfig
+++ b/arch/um/lkl/Kconfig
@@ -48,6 +48,9 @@ config OUTPUT_FORMAT
 config ARCH_DMA_ADDR_T_64BIT
        def_bool 64BIT
 
+config BLK_DEV_UBD
+       def_bool y if "$(OUTPUT_FORMAT)" = "elf32-i386" || "$(OUTPUT_FORMAT)" = "elf64-x86-64"
+
 config X86_64
        def_bool y if "$(OUTPUT_FORMAT)" = "elf64-x86-64"
 
diff --git a/arch/um/lkl/Makefile b/arch/um/lkl/Makefile
index 17ff6d00d1db..0ed34f87848b 100644
--- a/arch/um/lkl/Makefile
+++ b/arch/um/lkl/Makefile
@@ -5,7 +5,7 @@ include $(LKL_DIR)/auto.conf
 # fixup CFLAGS of um build
 KBUILD_CFLAGS := $(subst $(CFLAGS),,$(KBUILD_CFLAGS))
 # XXX
-KBUILD_CFLAGS	+= -DTIMER_IRQ=0 -DUM_ETH_IRQ=5 -DLAST_IRQ=15
+KBUILD_CFLAGS	+= -DTIMER_IRQ=0 -DUBD_IRQ=4 -DUM_ETH_IRQ=5 -DLAST_IRQ=15
 
 SRCARCH := um/lkl
 ARCH_INCLUDE += -I$(srctree)/$(LKL_DIR)/um/include
diff --git a/arch/um/lkl/include/uapi/asm/syscalls.h b/arch/um/lkl/include/uapi/asm/syscalls.h
index a81534ffccb7..1384c34137d6 100644
--- a/arch/um/lkl/include/uapi/asm/syscalls.h
+++ b/arch/um/lkl/include/uapi/asm/syscalls.h
@@ -164,6 +164,7 @@ struct sockaddr {
 #include <linux/virtio_ring.h>
 #include <linux/pkt_sched.h>
 #include <linux/io_uring.h>
+#include <linux/major.h>
 
 struct user_msghdr {
 	void		__user *msg_name;
diff --git a/arch/um/lkl/kernel/setup.c b/arch/um/lkl/kernel/setup.c
index 39b7b7c79581..daf55c216db9 100644
--- a/arch/um/lkl/kernel/setup.c
+++ b/arch/um/lkl/kernel/setup.c
@@ -126,6 +126,11 @@ long lkl_sys_halt(void)
 		LINUX_REBOOT_CMD_RESTART,
 	};
 
+#ifdef CONFIG_BLK_DEV_UBD
+void kill_io_thread(void);
+	kill_io_thread();
+#endif
+
 	err = lkl_syscall(__NR_reboot, params);
 	if (err < 0)
 		return err;
diff --git a/tools/lkl/include/lkl.h b/tools/lkl/include/lkl.h
index 952d11e30868..7d164e4dcad1 100644
--- a/tools/lkl/include/lkl.h
+++ b/tools/lkl/include/lkl.h
@@ -355,6 +355,10 @@ void lkl_perror(char *msg, int err);
  */
 struct lkl_dev_blk_ops;
 
+enum {
+	BLK_BACKEND_UM,
+};
+
 /**
  * lkl_disk - host disk handle
  *
@@ -368,6 +372,7 @@ struct lkl_disk {
 		int fd;
 		void *handle;
 	};
+	int backend;
 	struct lkl_dev_blk_ops *ops;
 };
 
@@ -403,6 +408,16 @@ int lkl_disk_remove(struct lkl_disk disk);
  */
 int lkl_encode_dev_from_sysfs(const char *sysfs_path, uint32_t *pdevid);
 
+
+#ifdef LKL_HOST_CONFIG_UML_DEV
+int lkl_disk_um_add(struct lkl_disk *disk, const char *blkparams);
+#else
+static inline int lkl_disk_um_add(struct lkl_disk *disk, const char *blkparams)
+{
+	return -LKL_ENOSYS;
+}
+#endif
+
 /**
  * lkl_mount_dev - mount a disk
  *
diff --git a/tools/lkl/lib/um/Build b/tools/lkl/lib/um/Build
index 09a60975ecd6..52573c05a797 100644
--- a/tools/lkl/lib/um/Build
+++ b/tools/lkl/lib/um/Build
@@ -1,2 +1,3 @@
 liblkl-y += um_glue.o
 liblkl-y += um_net.o
+liblkl-y += um_block.o
diff --git a/tools/lkl/lib/um/um_block.c b/tools/lkl/lib/um/um_block.c
new file mode 100644
index 000000000000..70dc624ec43c
--- /dev/null
+++ b/tools/lkl/lib/um/um_block.c
@@ -0,0 +1,17 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <lkl_host.h>
+
+
+static int registered_blk_dev_idx;
+
+int lkl_disk_um_add(struct lkl_disk *disk, const char *blkparams)
+{
+	/* concat strings */
+	snprintf(lkl_um_devs + strlen(lkl_um_devs), sizeof(lkl_um_devs),
+		 " ubd%d=%s", registered_blk_dev_idx, blkparams);
+
+	return registered_blk_dev_idx++;
+}
diff --git a/tools/lkl/lib/um/um_glue.c b/tools/lkl/lib/um/um_glue.c
index 10ca8524e682..ed82ff8ced52 100644
--- a/tools/lkl/lib/um/um_glue.c
+++ b/tools/lkl/lib/um/um_glue.c
@@ -2,9 +2,10 @@
 #include <stdio.h>
 #include <stdlib.h>
 #include <string.h>
+#include <signal.h>
 #include <unistd.h>
 #include <errno.h>
-
+#include <sys/wait.h>
 
 
 char lkl_um_devs[4096];
@@ -31,3 +32,27 @@ int ignore_sigio_fd(int fd)
 {
 	return 0;
 }
+
+/* from util.c */
+/*
+ * UML helper threads must not handle SIGWINCH/INT/TERM
+ */
+void os_fix_helper_signals(void)
+{
+	signal(SIGWINCH, SIG_IGN);
+	signal(SIGINT, SIG_DFL);
+	signal(SIGTERM, SIG_DFL);
+}
+
+
+
+void os_kill_process(int pid, int reap_child)
+{
+	kill(pid, SIGKILL);
+	if (reap_child) {
+		while ((errno = 0, ((waitpid(pid, NULL, __WALL)) < 0))
+		       && (errno == EINTR))
+			;
+	}
+
+}
diff --git a/tools/lkl/tests/disk.c b/tools/lkl/tests/disk.c
index 0aa039876b54..f7801014d054 100644
--- a/tools/lkl/tests/disk.c
+++ b/tools/lkl/tests/disk.c
@@ -23,12 +23,17 @@ static struct {
 	const char *disk;
 	const char *fstype;
 	int partition;
+	int backend;
 } cla;
 
+const char *backends[] = { "um", NULL };
+
 struct cl_arg args[] = {
 	{"disk", 'd', "disk file to use", 1, CL_ARG_STR, &cla.disk},
 	{"partition", 'P', "partition to mount", 1, CL_ARG_INT, &cla.partition},
 	{"type", 't', "filesystem type", 1, CL_ARG_STR, &cla.fstype},
+	{"backend", 'b', "blockd evice backend type", 1, CL_ARG_STR_SET,
+	 &cla.backend, backends},
 	{0},
 };
 
@@ -50,6 +55,15 @@ int lkl_test_disk_add(void)
 
 	disk.ops = NULL;
 
+	disk.backend = cla.backend;
+
+	switch (disk.backend) {
+	case BLK_BACKEND_UM:
+		disk.dev = (char *)cla.disk;
+	default:
+		break;
+	}
+
 	disk_id = lkl_disk_add(&disk);
 	if (disk_id < 0)
 		goto out_close;
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v3 26/26] um: fix clone flags to be familar with valgrind
  2020-02-05  7:30 [RFC v3 00/26] Unifying LKL into UML Hajime Tazaki
                   ` (24 preceding siblings ...)
  2020-02-05  7:30 ` [RFC v3 25/26] um lkl: add UML block device driver (ubd) " Hajime Tazaki
@ 2020-02-05  7:30 ` Hajime Tazaki
  2020-03-30 14:45   ` Hajime Tazaki
  26 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05  7:30 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Hajime Tazaki

Valgrind reports the following error with UML when ubd driver is
compiled in.

==29588== Unsupported clone() flags: 0x500
==29588==
==29588== The only supported clone() uses are:
==29588==  - via a threads library (LinuxThreads or NPTL)
==29588==  - via the implementation of fork or vfork

Adding CLONE_FS flags silences this issue.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/drivers/ubd_user.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/arch/um/drivers/ubd_user.c b/arch/um/drivers/ubd_user.c
index a1afe414ce48..fabb50e91c37 100644
--- a/arch/um/drivers/ubd_user.c
+++ b/arch/um/drivers/ubd_user.c
@@ -47,7 +47,8 @@ int start_io_thread(unsigned long sp, int *fd_out)
 		goto out_close;
 	}
 
-	pid = clone(io_thread, (void *) sp, CLONE_FILES | CLONE_VM, NULL);
+	pid = clone(io_thread, (void *) sp,
+		    CLONE_FILES | CLONE_VM | CLONE_FS, NULL);
 	if(pid < 0){
 		err = -errno;
 		printk("start_io_thread - clone failed : errno = %d\n", errno);
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* Re: [RFC v3 01/26] asm-generic: atomic64: allow using generic atomic64 on 64bit platforms
  2020-02-05  7:30 ` [RFC v3 01/26] asm-generic: atomic64: allow using generic atomic64 on 64bit platforms Hajime Tazaki
@ 2020-02-05  9:34   ` Peter Zijlstra
  2020-02-05 12:24       ` Octavian Purdila
  0 siblings, 1 reply; 476+ messages in thread
From: Peter Zijlstra @ 2020-02-05  9:34 UTC (permalink / raw)
  To: Hajime Tazaki
  Cc: linux-um, Octavian Purdila, Akira Moroo, linux-kernel-library,
	linux-arch, Will Deacon, Boqun Feng

On Wed, Feb 05, 2020 at 04:30:10PM +0900, Hajime Tazaki wrote:
> From: Octavian Purdila <tavi.purdila@gmail.com>
> 
> With CONFIG_64BIT enabled, atomic64 via CONFIG_GENERIC_ATOMIC64 options
> are not compiled due to type conflict of atomic64_t defined in
> linux/type.h.
> 
> This commit fixes the issue and allow using generic atomic64 ops.
> 
> Currently, LKL is only the user which defines GENERIC_ATOMIC64
> (lib/atomic64.c) under CONFIG_64BIT environment.  Thus, there is no
> issues before this commit.

Uhhhhh, no.

Not allowing GENERIC_ATOMIC64 on 64BIT is a *feature*.

Any 64bit arch that needs GENERIC_ATOMIC64 is an utter piece of crap.

Please explain more.

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v3 07/26] um lkl: interrupt support
  2020-02-05  7:30   ` Hajime Tazaki
@ 2020-02-05 10:47     ` Anton Ivanov
  -1 siblings, 0 replies; 476+ messages in thread
From: Anton Ivanov @ 2020-02-05 10:47 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um
  Cc: linux-arch, Octavian Purdila, Akira Moroo, linux-kernel-library,
	Michael Zimmermann



On 05/02/2020 07:30, Hajime Tazaki wrote:
> From: Octavian Purdila <tavi.purdila@gmail.com>
> 
> Add APIs that allows the host to reserve and free and interrupt number
> and also to trigger an interrupt.
> 
> The trigger operation will simply store the interrupt data in
> queue. The interrupt handler is run later, at the first opportunity it
> has to avoid races with any kernel threads.
> 
> Currently, interrupts are run on the first interrupt enable operation
> if interrupts are disabled and if we are not already in interrupt
> context.
> 
> When triggering an interrupt, it uses GCC's built-in functions for
> atomic memory access to synchronize and simple boolean flags.
> 
> Cc: Michael Zimmermann <sigmaepsilon92@gmail.com>
> Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
> Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
> ---
>   arch/um/lkl/include/asm/irq.h             |  13 ++
>   arch/um/lkl/include/uapi/asm/irq.h        |  36 ++++
>   arch/um/lkl/include/uapi/asm/sigcontext.h |  16 ++
>   arch/um/lkl/kernel/irq.c                  | 193 ++++++++++++++++++++++
>   4 files changed, 258 insertions(+)
>   create mode 100644 arch/um/lkl/include/asm/irq.h
>   create mode 100644 arch/um/lkl/include/uapi/asm/irq.h
>   create mode 100644 arch/um/lkl/include/uapi/asm/sigcontext.h
>   create mode 100644 arch/um/lkl/kernel/irq.c
> 
> diff --git a/arch/um/lkl/include/asm/irq.h b/arch/um/lkl/include/asm/irq.h
> new file mode 100644
> index 000000000000..948fc54cb76c
> --- /dev/null
> +++ b/arch/um/lkl/include/asm/irq.h
> @@ -0,0 +1,13 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef _ASM_LKL_IRQ_H
> +#define _ASM_LKL_IRQ_H
> +
> +#define IRQ_STATUS_BITS		(sizeof(long) * 8)
> +#define NR_IRQS			((int)(IRQ_STATUS_BITS * IRQ_STATUS_BITS))
> +
> +void run_irqs(void);
> +void set_irq_pending(int irq);
> +
> +#include <uapi/asm/irq.h>
> +
> +#endif
> diff --git a/arch/um/lkl/include/uapi/asm/irq.h b/arch/um/lkl/include/uapi/asm/irq.h
> new file mode 100644
> index 000000000000..666628b233eb
> --- /dev/null
> +++ b/arch/um/lkl/include/uapi/asm/irq.h
> @@ -0,0 +1,36 @@
> +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
> +#ifndef _ASM_UAPI_LKL_IRQ_H
> +#define _ASM_UAPI_LKL_IRQ_H
> +
> +/**
> + * lkl_trigger_irq - generate an interrupt
> + *
> + * This function is used by the device host side to signal its Linux counterpart
> + * that some event happened.
> + *
> + * @irq - the irq number to signal
> + */
> +int lkl_trigger_irq(int irq);
> +
> +/**
> + * lkl_get_free_irq - find and reserve a free IRQ number
> + *
> + * This function is called by the host device code to find an unused IRQ number
> + * and reserved it for its own use.
> + *
> + * @user - a string to identify the user
> + * @returns - and irq number that can be used by request_irq or an negative
> + * value in case of an error
> + */
> +int lkl_get_free_irq(const char *user);
> +
> +/**
> + * lkl_put_irq - release an IRQ number previously obtained with lkl_get_free_irq
> + *
> + * @irq - irq number to release
> + * @user - string identifying the user; should be the same as the one passed to
> + * lkl_get_free_irq when the irq number was obtained
> + */
> +void lkl_put_irq(int irq, const char *name);
> +
> +#endif
> diff --git a/arch/um/lkl/include/uapi/asm/sigcontext.h b/arch/um/lkl/include/uapi/asm/sigcontext.h
> new file mode 100644
> index 000000000000..2f4848843d1d
> --- /dev/null
> +++ b/arch/um/lkl/include/uapi/asm/sigcontext.h
> @@ -0,0 +1,16 @@
> +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
> +#ifndef _ASM_UAPI_LKL_SIGCONTEXT_H
> +#define _ASM_UAPI_LKL_SIGCONTEXT_H
> +
> +#include <asm/ptrace.h>
> +
> +struct pt_regs {
> +	void *irq_data;
> +};
> +
> +struct sigcontext {
> +	struct pt_regs regs;
> +	unsigned long oldmask;
> +};
> +
> +#endif
> diff --git a/arch/um/lkl/kernel/irq.c b/arch/um/lkl/kernel/irq.c
> new file mode 100644
> index 000000000000..e3b59e46ca50
> --- /dev/null
> +++ b/arch/um/lkl/kernel/irq.c
> @@ -0,0 +1,193 @@
> +// SPDX-License-Identifier: GPL-2.0
> +#include <linux/kernel.h>
> +#include <linux/init.h>
> +#include <linux/irq.h>
> +#include <linux/hardirq.h>
> +#include <asm/irq_regs.h>
> +#include <linux/sched.h>
> +#include <linux/seq_file.h>
> +#include <linux/tick.h>
> +#include <asm/irqflags.h>
> +#include <asm/host_ops.h>
> +#include <asm/cpu.h>
> +
> +/*
> + * To avoid much overhead we use an indirect approach: the irqs are marked using
> + * a bitmap (array of longs) and a summary of the modified bits is kept in a
> + * separate "index" long - one bit for each sizeof(long). Thus we can support
> + * 4096 irqs on 64bit platforms and 1024 irqs on 32bit platforms.
> + *
> + * Whenever an irq is trigger both the array and the index is updated. To find
> + * which irqs were triggered we first search the index and then the
> + * corresponding part of the arrary.
> + */
> +static unsigned long irq_status[NR_IRQS / IRQ_STATUS_BITS];
> +static unsigned long irq_index_status;
> +
> +static inline unsigned long test_and_clear_irq_index_status(void)
> +{
> +	if (!irq_index_status)
> +		return 0;
> +	return __sync_fetch_and_and(&irq_index_status, 0);
> +}
> +
> +static inline unsigned long test_and_clear_irq_status(int index)
> +{
> +	if (!&irq_status[index])
> +		return 0;
> +	return __sync_fetch_and_and(&irq_status[index], 0);
> +}
> +
> +void set_irq_pending(int irq)
> +{
> +	int index = irq / IRQ_STATUS_BITS;
> +	int bit = irq % IRQ_STATUS_BITS;
> +
> +	__sync_fetch_and_or(&irq_status[index], BIT(bit));
> +	__sync_fetch_and_or(&irq_index_status, BIT(index));
> +}
> +
> +static struct irq_info {
> +	const char *user;
> +} irqs[NR_IRQS];
> +
> +static bool irqs_enabled;
> +
> +static struct pt_regs dummy;
> +
> +static void run_irq(int irq)
> +{
> +	unsigned long flags;
> +	struct pt_regs *old_regs = set_irq_regs((struct pt_regs *)&dummy);
> +
> +	/* interrupt handlers need to run with interrupts disabled */
> +	local_irq_save(flags);
> +	irq_enter();
> +	generic_handle_irq(irq);
> +	irq_exit();
> +	set_irq_regs(old_regs);
> +	local_irq_restore(flags);
> +}
> +
> +/**
> + * This function can be called from arbitrary host threads, so do not
> + * issue any Linux calls (e.g. prink) if lkl_cpu_get() was not issued
> + * before.
> + */
> +int lkl_trigger_irq(int irq)
> +{
> +	int ret;
> +
> +	if (!irq || irq > NR_IRQS)
> +		return -EINVAL;
> +
> +	ret = lkl_cpu_try_run_irq(irq);
> +	if (ret <= 0)
> +		return ret;
> +
> +	/*
> +	 * Since this can be called from Linux context (e.g. lkl_trigger_irq ->
> +	 * IRQ -> softirq -> lkl_trigger_irq) make sure we are actually allowed
> +	 * to run irqs at this point
> +	 */
> +	if (!irqs_enabled) {
> +		set_irq_pending(irq);
> +		lkl_cpu_put();
> +		return 0;
> +	}
> +
> +	run_irq(irq);
> +
> +	lkl_cpu_put();
> +
> +	return 0;

Isn't that just:

	if (irqs_enabled)
		run_irq(irq);
	else
		set_irq_pending(irq);

	lkl_cpu_put();

	return 0;

> +}
> +
> +static inline void for_each_bit(unsigned long word, void (*f)(int, int), int j)
> +{
> +	int i = 0;
> +
> +	while (word) {
> +		if (word & 1)
> +			f(i, j);
> +		word >>= 1;
> +		i++;
> +	}
> +}
> +
> +static inline void deliver_irq(int bit, int index)
> +{
> +	run_irq(index * IRQ_STATUS_BITS + bit);
> +}
> +
> +static inline void check_irq_status(int i, int unused)
> +{
> +	for_each_bit(test_and_clear_irq_status(i), deliver_irq, i);
> +}
> +
> +void run_irqs(void)
> +{
> +	for_each_bit(test_and_clear_irq_index_status(), check_irq_status, 0);
> +}
> +
> +int show_interrupts(struct seq_file *p, void *v)
> +{
> +	return 0;
> +}
> +
> +int lkl_get_free_irq(const char *user)
> +{
> +	int i;
> +	int ret = -EBUSY;
> +
> +	/* 0 is not a valid IRQ */
> +	for (i = 1; i < NR_IRQS; i++) {
> +		if (!irqs[i].user) {
> +			irqs[i].user = user;
> +			irq_set_chip_and_handler(i, &dummy_irq_chip,
> +						 handle_simple_irq);
> +			ret = i;
> +			break;
> +		}
> +	}
> +
> +	return ret;
> +}
> +
> +void lkl_put_irq(int i, const char *user)
> +{
> +	if (!irqs[i].user || strcmp(irqs[i].user, user) != 0) {
> +		WARN("%s tried to release %s's irq %d", user, irqs[i].user, i);
> +		return;
> +	}
> +
> +	irqs[i].user = NULL;
> +}
> +
> +unsigned long arch_local_save_flags(void)
> +{
> +	return irqs_enabled;
> +}
> +
> +void arch_local_irq_restore(unsigned long flags)
> +{
> +	if (flags == ARCH_IRQ_ENABLED && irqs_enabled == ARCH_IRQ_DISABLED &&
> +	    !in_interrupt())
> +		run_irqs();
> +	irqs_enabled = flags;
> +}
> +
> +void init_IRQ(void)
> +{
> +	int i;
> +
> +	for (i = 0; i < NR_IRQS; i++)
> +		irq_set_chip_and_handler(i, &dummy_irq_chip, handle_simple_irq);
> +
> +	pr_info("lkl: irqs initialized\n");
> +}
> +
> +void cpu_yield_to_irqs(void)
> +{
> +	cpu_relax();
> +}
> 

-- 
Anton R. Ivanov
https://www.kot-begemot.co.uk/

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v3 07/26] um lkl: interrupt support
@ 2020-02-05 10:47     ` Anton Ivanov
  0 siblings, 0 replies; 476+ messages in thread
From: Anton Ivanov @ 2020-02-05 10:47 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um
  Cc: linux-arch, Octavian Purdila, Michael Zimmermann,
	linux-kernel-library, Akira Moroo



On 05/02/2020 07:30, Hajime Tazaki wrote:
> From: Octavian Purdila <tavi.purdila@gmail.com>
> 
> Add APIs that allows the host to reserve and free and interrupt number
> and also to trigger an interrupt.
> 
> The trigger operation will simply store the interrupt data in
> queue. The interrupt handler is run later, at the first opportunity it
> has to avoid races with any kernel threads.
> 
> Currently, interrupts are run on the first interrupt enable operation
> if interrupts are disabled and if we are not already in interrupt
> context.
> 
> When triggering an interrupt, it uses GCC's built-in functions for
> atomic memory access to synchronize and simple boolean flags.
> 
> Cc: Michael Zimmermann <sigmaepsilon92@gmail.com>
> Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
> Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
> ---
>   arch/um/lkl/include/asm/irq.h             |  13 ++
>   arch/um/lkl/include/uapi/asm/irq.h        |  36 ++++
>   arch/um/lkl/include/uapi/asm/sigcontext.h |  16 ++
>   arch/um/lkl/kernel/irq.c                  | 193 ++++++++++++++++++++++
>   4 files changed, 258 insertions(+)
>   create mode 100644 arch/um/lkl/include/asm/irq.h
>   create mode 100644 arch/um/lkl/include/uapi/asm/irq.h
>   create mode 100644 arch/um/lkl/include/uapi/asm/sigcontext.h
>   create mode 100644 arch/um/lkl/kernel/irq.c
> 
> diff --git a/arch/um/lkl/include/asm/irq.h b/arch/um/lkl/include/asm/irq.h
> new file mode 100644
> index 000000000000..948fc54cb76c
> --- /dev/null
> +++ b/arch/um/lkl/include/asm/irq.h
> @@ -0,0 +1,13 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef _ASM_LKL_IRQ_H
> +#define _ASM_LKL_IRQ_H
> +
> +#define IRQ_STATUS_BITS		(sizeof(long) * 8)
> +#define NR_IRQS			((int)(IRQ_STATUS_BITS * IRQ_STATUS_BITS))
> +
> +void run_irqs(void);
> +void set_irq_pending(int irq);
> +
> +#include <uapi/asm/irq.h>
> +
> +#endif
> diff --git a/arch/um/lkl/include/uapi/asm/irq.h b/arch/um/lkl/include/uapi/asm/irq.h
> new file mode 100644
> index 000000000000..666628b233eb
> --- /dev/null
> +++ b/arch/um/lkl/include/uapi/asm/irq.h
> @@ -0,0 +1,36 @@
> +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
> +#ifndef _ASM_UAPI_LKL_IRQ_H
> +#define _ASM_UAPI_LKL_IRQ_H
> +
> +/**
> + * lkl_trigger_irq - generate an interrupt
> + *
> + * This function is used by the device host side to signal its Linux counterpart
> + * that some event happened.
> + *
> + * @irq - the irq number to signal
> + */
> +int lkl_trigger_irq(int irq);
> +
> +/**
> + * lkl_get_free_irq - find and reserve a free IRQ number
> + *
> + * This function is called by the host device code to find an unused IRQ number
> + * and reserved it for its own use.
> + *
> + * @user - a string to identify the user
> + * @returns - and irq number that can be used by request_irq or an negative
> + * value in case of an error
> + */
> +int lkl_get_free_irq(const char *user);
> +
> +/**
> + * lkl_put_irq - release an IRQ number previously obtained with lkl_get_free_irq
> + *
> + * @irq - irq number to release
> + * @user - string identifying the user; should be the same as the one passed to
> + * lkl_get_free_irq when the irq number was obtained
> + */
> +void lkl_put_irq(int irq, const char *name);
> +
> +#endif
> diff --git a/arch/um/lkl/include/uapi/asm/sigcontext.h b/arch/um/lkl/include/uapi/asm/sigcontext.h
> new file mode 100644
> index 000000000000..2f4848843d1d
> --- /dev/null
> +++ b/arch/um/lkl/include/uapi/asm/sigcontext.h
> @@ -0,0 +1,16 @@
> +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
> +#ifndef _ASM_UAPI_LKL_SIGCONTEXT_H
> +#define _ASM_UAPI_LKL_SIGCONTEXT_H
> +
> +#include <asm/ptrace.h>
> +
> +struct pt_regs {
> +	void *irq_data;
> +};
> +
> +struct sigcontext {
> +	struct pt_regs regs;
> +	unsigned long oldmask;
> +};
> +
> +#endif
> diff --git a/arch/um/lkl/kernel/irq.c b/arch/um/lkl/kernel/irq.c
> new file mode 100644
> index 000000000000..e3b59e46ca50
> --- /dev/null
> +++ b/arch/um/lkl/kernel/irq.c
> @@ -0,0 +1,193 @@
> +// SPDX-License-Identifier: GPL-2.0
> +#include <linux/kernel.h>
> +#include <linux/init.h>
> +#include <linux/irq.h>
> +#include <linux/hardirq.h>
> +#include <asm/irq_regs.h>
> +#include <linux/sched.h>
> +#include <linux/seq_file.h>
> +#include <linux/tick.h>
> +#include <asm/irqflags.h>
> +#include <asm/host_ops.h>
> +#include <asm/cpu.h>
> +
> +/*
> + * To avoid much overhead we use an indirect approach: the irqs are marked using
> + * a bitmap (array of longs) and a summary of the modified bits is kept in a
> + * separate "index" long - one bit for each sizeof(long). Thus we can support
> + * 4096 irqs on 64bit platforms and 1024 irqs on 32bit platforms.
> + *
> + * Whenever an irq is trigger both the array and the index is updated. To find
> + * which irqs were triggered we first search the index and then the
> + * corresponding part of the arrary.
> + */
> +static unsigned long irq_status[NR_IRQS / IRQ_STATUS_BITS];
> +static unsigned long irq_index_status;
> +
> +static inline unsigned long test_and_clear_irq_index_status(void)
> +{
> +	if (!irq_index_status)
> +		return 0;
> +	return __sync_fetch_and_and(&irq_index_status, 0);
> +}
> +
> +static inline unsigned long test_and_clear_irq_status(int index)
> +{
> +	if (!&irq_status[index])
> +		return 0;
> +	return __sync_fetch_and_and(&irq_status[index], 0);
> +}
> +
> +void set_irq_pending(int irq)
> +{
> +	int index = irq / IRQ_STATUS_BITS;
> +	int bit = irq % IRQ_STATUS_BITS;
> +
> +	__sync_fetch_and_or(&irq_status[index], BIT(bit));
> +	__sync_fetch_and_or(&irq_index_status, BIT(index));
> +}
> +
> +static struct irq_info {
> +	const char *user;
> +} irqs[NR_IRQS];
> +
> +static bool irqs_enabled;
> +
> +static struct pt_regs dummy;
> +
> +static void run_irq(int irq)
> +{
> +	unsigned long flags;
> +	struct pt_regs *old_regs = set_irq_regs((struct pt_regs *)&dummy);
> +
> +	/* interrupt handlers need to run with interrupts disabled */
> +	local_irq_save(flags);
> +	irq_enter();
> +	generic_handle_irq(irq);
> +	irq_exit();
> +	set_irq_regs(old_regs);
> +	local_irq_restore(flags);
> +}
> +
> +/**
> + * This function can be called from arbitrary host threads, so do not
> + * issue any Linux calls (e.g. prink) if lkl_cpu_get() was not issued
> + * before.
> + */
> +int lkl_trigger_irq(int irq)
> +{
> +	int ret;
> +
> +	if (!irq || irq > NR_IRQS)
> +		return -EINVAL;
> +
> +	ret = lkl_cpu_try_run_irq(irq);
> +	if (ret <= 0)
> +		return ret;
> +
> +	/*
> +	 * Since this can be called from Linux context (e.g. lkl_trigger_irq ->
> +	 * IRQ -> softirq -> lkl_trigger_irq) make sure we are actually allowed
> +	 * to run irqs at this point
> +	 */
> +	if (!irqs_enabled) {
> +		set_irq_pending(irq);
> +		lkl_cpu_put();
> +		return 0;
> +	}
> +
> +	run_irq(irq);
> +
> +	lkl_cpu_put();
> +
> +	return 0;

Isn't that just:

	if (irqs_enabled)
		run_irq(irq);
	else
		set_irq_pending(irq);

	lkl_cpu_put();

	return 0;

> +}
> +
> +static inline void for_each_bit(unsigned long word, void (*f)(int, int), int j)
> +{
> +	int i = 0;
> +
> +	while (word) {
> +		if (word & 1)
> +			f(i, j);
> +		word >>= 1;
> +		i++;
> +	}
> +}
> +
> +static inline void deliver_irq(int bit, int index)
> +{
> +	run_irq(index * IRQ_STATUS_BITS + bit);
> +}
> +
> +static inline void check_irq_status(int i, int unused)
> +{
> +	for_each_bit(test_and_clear_irq_status(i), deliver_irq, i);
> +}
> +
> +void run_irqs(void)
> +{
> +	for_each_bit(test_and_clear_irq_index_status(), check_irq_status, 0);
> +}
> +
> +int show_interrupts(struct seq_file *p, void *v)
> +{
> +	return 0;
> +}
> +
> +int lkl_get_free_irq(const char *user)
> +{
> +	int i;
> +	int ret = -EBUSY;
> +
> +	/* 0 is not a valid IRQ */
> +	for (i = 1; i < NR_IRQS; i++) {
> +		if (!irqs[i].user) {
> +			irqs[i].user = user;
> +			irq_set_chip_and_handler(i, &dummy_irq_chip,
> +						 handle_simple_irq);
> +			ret = i;
> +			break;
> +		}
> +	}
> +
> +	return ret;
> +}
> +
> +void lkl_put_irq(int i, const char *user)
> +{
> +	if (!irqs[i].user || strcmp(irqs[i].user, user) != 0) {
> +		WARN("%s tried to release %s's irq %d", user, irqs[i].user, i);
> +		return;
> +	}
> +
> +	irqs[i].user = NULL;
> +}
> +
> +unsigned long arch_local_save_flags(void)
> +{
> +	return irqs_enabled;
> +}
> +
> +void arch_local_irq_restore(unsigned long flags)
> +{
> +	if (flags == ARCH_IRQ_ENABLED && irqs_enabled == ARCH_IRQ_DISABLED &&
> +	    !in_interrupt())
> +		run_irqs();
> +	irqs_enabled = flags;
> +}
> +
> +void init_IRQ(void)
> +{
> +	int i;
> +
> +	for (i = 0; i < NR_IRQS; i++)
> +		irq_set_chip_and_handler(i, &dummy_irq_chip, handle_simple_irq);
> +
> +	pr_info("lkl: irqs initialized\n");
> +}
> +
> +void cpu_yield_to_irqs(void)
> +{
> +	cpu_relax();
> +}
> 

-- 
Anton R. Ivanov
https://www.kot-begemot.co.uk/

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v3 01/26] asm-generic: atomic64: allow using generic atomic64 on 64bit platforms
  2020-02-05  9:34   ` Peter Zijlstra
@ 2020-02-05 12:24       ` Octavian Purdila
  0 siblings, 0 replies; 476+ messages in thread
From: Octavian Purdila @ 2020-02-05 12:24 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: Hajime Tazaki, linux-um, Akira Moroo, linux-kernel-library,
	linux-arch, Will Deacon, Boqun Feng

On Wed, Feb 5, 2020 at 11:34 AM Peter Zijlstra <peterz@infradead.org> wrote:
>
> On Wed, Feb 05, 2020 at 04:30:10PM +0900, Hajime Tazaki wrote:
> > From: Octavian Purdila <tavi.purdila@gmail.com>
> >
> > With CONFIG_64BIT enabled, atomic64 via CONFIG_GENERIC_ATOMIC64 options
> > are not compiled due to type conflict of atomic64_t defined in
> > linux/type.h.
> >
> > This commit fixes the issue and allow using generic atomic64 ops.
> >
> > Currently, LKL is only the user which defines GENERIC_ATOMIC64
> > (lib/atomic64.c) under CONFIG_64BIT environment.  Thus, there is no
> > issues before this commit.
>
> Uhhhhh, no.
>
> Not allowing GENERIC_ATOMIC64 on 64BIT is a *feature*.
>
> Any 64bit arch that needs GENERIC_ATOMIC64 is an utter piece of crap.
>
> Please explain more.
>

Hi Peter,

I was not aware that not allowing GENERIC_ATOMIC64 was intentional. I
understand your point that a 64 bit architecture that can't handle 64
bit atomic operation is broken.

One way to deal with this in LKL would be to use GCC atomic builtins
or if that doesn't work expose them as host operations. This would
keep LKL as a meta-arch that can run on multiple physical
architectures. I'll give it a try.

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v3 01/26] asm-generic: atomic64: allow using generic atomic64 on 64bit platforms
@ 2020-02-05 12:24       ` Octavian Purdila
  0 siblings, 0 replies; 476+ messages in thread
From: Octavian Purdila @ 2020-02-05 12:24 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: linux-arch, Boqun Feng, linux-um, Akira Moroo,
	linux-kernel-library, Will Deacon, Hajime Tazaki

On Wed, Feb 5, 2020 at 11:34 AM Peter Zijlstra <peterz@infradead.org> wrote:
>
> On Wed, Feb 05, 2020 at 04:30:10PM +0900, Hajime Tazaki wrote:
> > From: Octavian Purdila <tavi.purdila@gmail.com>
> >
> > With CONFIG_64BIT enabled, atomic64 via CONFIG_GENERIC_ATOMIC64 options
> > are not compiled due to type conflict of atomic64_t defined in
> > linux/type.h.
> >
> > This commit fixes the issue and allow using generic atomic64 ops.
> >
> > Currently, LKL is only the user which defines GENERIC_ATOMIC64
> > (lib/atomic64.c) under CONFIG_64BIT environment.  Thus, there is no
> > issues before this commit.
>
> Uhhhhh, no.
>
> Not allowing GENERIC_ATOMIC64 on 64BIT is a *feature*.
>
> Any 64bit arch that needs GENERIC_ATOMIC64 is an utter piece of crap.
>
> Please explain more.
>

Hi Peter,

I was not aware that not allowing GENERIC_ATOMIC64 was intentional. I
understand your point that a 64 bit architecture that can't handle 64
bit atomic operation is broken.

One way to deal with this in LKL would be to use GCC atomic builtins
or if that doesn't work expose them as host operations. This would
keep LKL as a meta-arch that can run on multiple physical
architectures. I'll give it a try.

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v3 01/26] asm-generic: atomic64: allow using generic atomic64 on 64bit platforms
  2020-02-05 12:24       ` Octavian Purdila
@ 2020-02-05 12:29         ` Anton Ivanov
  -1 siblings, 0 replies; 476+ messages in thread
From: Anton Ivanov @ 2020-02-05 12:29 UTC (permalink / raw)
  To: Octavian Purdila, Peter Zijlstra
  Cc: linux-arch, Boqun Feng, linux-um, Akira Moroo,
	linux-kernel-library, Will Deacon, Hajime Tazaki

On 05/02/2020 12:24, Octavian Purdila wrote:
> On Wed, Feb 5, 2020 at 11:34 AM Peter Zijlstra <peterz@infradead.org> wrote:
>>
>> On Wed, Feb 05, 2020 at 04:30:10PM +0900, Hajime Tazaki wrote:
>>> From: Octavian Purdila <tavi.purdila@gmail.com>
>>>
>>> With CONFIG_64BIT enabled, atomic64 via CONFIG_GENERIC_ATOMIC64 options
>>> are not compiled due to type conflict of atomic64_t defined in
>>> linux/type.h.
>>>
>>> This commit fixes the issue and allow using generic atomic64 ops.
>>>
>>> Currently, LKL is only the user which defines GENERIC_ATOMIC64
>>> (lib/atomic64.c) under CONFIG_64BIT environment.  Thus, there is no
>>> issues before this commit.
>>
>> Uhhhhh, no.
>>
>> Not allowing GENERIC_ATOMIC64 on 64BIT is a *feature*.
>>
>> Any 64bit arch that needs GENERIC_ATOMIC64 is an utter piece of crap.
>>
>> Please explain more.
>>
> 
> Hi Peter,
> 
> I was not aware that not allowing GENERIC_ATOMIC64 was intentional. I
> understand your point that a 64 bit architecture that can't handle 64
> bit atomic operation is broken.
> 
> One way to deal with this in LKL would be to use GCC atomic builtins
> or if that doesn't work expose them as host operations. This would
> keep LKL as a meta-arch that can run on multiple physical
> architectures. I'll give it a try.
> 
> _______________________________________________
> linux-um mailing list
> linux-um@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-um
> 


You can lift a set of defines which do that for most compilers/arches 
out of OVS code.

Have a look at lib/ovs-atomic*.h

It should do exactly what you want (+/- cutting it down to things you need).

-- 
Anton R. Ivanov
https://www.kot-begemot.co.uk/

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v3 01/26] asm-generic: atomic64: allow using generic atomic64 on 64bit platforms
@ 2020-02-05 12:29         ` Anton Ivanov
  0 siblings, 0 replies; 476+ messages in thread
From: Anton Ivanov @ 2020-02-05 12:29 UTC (permalink / raw)
  To: Octavian Purdila, Peter Zijlstra
  Cc: linux-arch, Boqun Feng, linux-um, Akira Moroo,
	linux-kernel-library, Will Deacon, Hajime Tazaki

On 05/02/2020 12:24, Octavian Purdila wrote:
> On Wed, Feb 5, 2020 at 11:34 AM Peter Zijlstra <peterz@infradead.org> wrote:
>>
>> On Wed, Feb 05, 2020 at 04:30:10PM +0900, Hajime Tazaki wrote:
>>> From: Octavian Purdila <tavi.purdila@gmail.com>
>>>
>>> With CONFIG_64BIT enabled, atomic64 via CONFIG_GENERIC_ATOMIC64 options
>>> are not compiled due to type conflict of atomic64_t defined in
>>> linux/type.h.
>>>
>>> This commit fixes the issue and allow using generic atomic64 ops.
>>>
>>> Currently, LKL is only the user which defines GENERIC_ATOMIC64
>>> (lib/atomic64.c) under CONFIG_64BIT environment.  Thus, there is no
>>> issues before this commit.
>>
>> Uhhhhh, no.
>>
>> Not allowing GENERIC_ATOMIC64 on 64BIT is a *feature*.
>>
>> Any 64bit arch that needs GENERIC_ATOMIC64 is an utter piece of crap.
>>
>> Please explain more.
>>
> 
> Hi Peter,
> 
> I was not aware that not allowing GENERIC_ATOMIC64 was intentional. I
> understand your point that a 64 bit architecture that can't handle 64
> bit atomic operation is broken.
> 
> One way to deal with this in LKL would be to use GCC atomic builtins
> or if that doesn't work expose them as host operations. This would
> keep LKL as a meta-arch that can run on multiple physical
> architectures. I'll give it a try.
> 
> _______________________________________________
> linux-um mailing list
> linux-um@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-um
> 


You can lift a set of defines which do that for most compilers/arches 
out of OVS code.

Have a look at lib/ovs-atomic*.h

It should do exactly what you want (+/- cutting it down to things you need).

-- 
Anton R. Ivanov
https://www.kot-begemot.co.uk/

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v3 01/26] asm-generic: atomic64: allow using generic atomic64 on 64bit platforms
  2020-02-05 12:24       ` Octavian Purdila
@ 2020-02-05 12:49         ` Peter Zijlstra
  -1 siblings, 0 replies; 476+ messages in thread
From: Peter Zijlstra @ 2020-02-05 12:49 UTC (permalink / raw)
  To: Octavian Purdila
  Cc: Hajime Tazaki, linux-um, Akira Moroo, linux-kernel-library,
	linux-arch, Will Deacon, Boqun Feng

On Wed, Feb 05, 2020 at 02:24:38PM +0200, Octavian Purdila wrote:
> I was not aware that not allowing GENERIC_ATOMIC64 was intentional. I

It might not have been, but presented with this patch, I feel like it
should've been :-)

> understand your point that a 64 bit architecture that can't handle 64
> bit atomic operation is broken.

(sadly they actually exist, I shall name no names)

> One way to deal with this in LKL would be to use GCC atomic builtins
> or if that doesn't work expose them as host operations. This would
> keep LKL as a meta-arch that can run on multiple physical
> architectures. I'll give it a try.

What is this LKL you speak of and how does it do the 32bit atomics?

One thing to keep in mind is that the C11 atomics (_Atomic) don't
trivially map to the LKMM -- although I keep forgetting the exact
details, there is a paper on it somewhere.  Also, once you're limited to
a specific arch the issue also becomes much easier than C11 vs LKMM in
general.

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v3 01/26] asm-generic: atomic64: allow using generic atomic64 on 64bit platforms
@ 2020-02-05 12:49         ` Peter Zijlstra
  0 siblings, 0 replies; 476+ messages in thread
From: Peter Zijlstra @ 2020-02-05 12:49 UTC (permalink / raw)
  To: Octavian Purdila
  Cc: linux-arch, Boqun Feng, linux-um, Akira Moroo,
	linux-kernel-library, Will Deacon, Hajime Tazaki

On Wed, Feb 05, 2020 at 02:24:38PM +0200, Octavian Purdila wrote:
> I was not aware that not allowing GENERIC_ATOMIC64 was intentional. I

It might not have been, but presented with this patch, I feel like it
should've been :-)

> understand your point that a 64 bit architecture that can't handle 64
> bit atomic operation is broken.

(sadly they actually exist, I shall name no names)

> One way to deal with this in LKL would be to use GCC atomic builtins
> or if that doesn't work expose them as host operations. This would
> keep LKL as a meta-arch that can run on multiple physical
> architectures. I'll give it a try.

What is this LKL you speak of and how does it do the 32bit atomics?

One thing to keep in mind is that the C11 atomics (_Atomic) don't
trivially map to the LKMM -- although I keep forgetting the exact
details, there is a paper on it somewhere.  Also, once you're limited to
a specific arch the issue also becomes much easier than C11 vs LKMM in
general.

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v3 01/26] asm-generic: atomic64: allow using generic atomic64 on 64bit platforms
  2020-02-05 12:49         ` Peter Zijlstra
@ 2020-02-05 14:00           ` Octavian Purdila
  -1 siblings, 0 replies; 476+ messages in thread
From: Octavian Purdila @ 2020-02-05 14:00 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: Hajime Tazaki, linux-um, Akira Moroo, linux-kernel-library,
	linux-arch, Will Deacon, Boqun Feng

On Wed, Feb 5, 2020 at 2:49 PM Peter Zijlstra <peterz@infradead.org> wrote:
>
> On Wed, Feb 05, 2020 at 02:24:38PM +0200, Octavian Purdila wrote:
> > I was not aware that not allowing GENERIC_ATOMIC64 was intentional. I
>
> It might not have been, but presented with this patch, I feel like it
> should've been :-)
>
> > understand your point that a 64 bit architecture that can't handle 64
> > bit atomic operation is broken.
>
> (sadly they actually exist, I shall name no names)
>
> > One way to deal with this in LKL would be to use GCC atomic builtins
> > or if that doesn't work expose them as host operations. This would
> > keep LKL as a meta-arch that can run on multiple physical
> > architectures. I'll give it a try.
>
> What is this LKL you speak of and how does it do the 32bit atomics?
>

LKL is a build of the Linux kernel as a library that can run in many
environments including multiple architectures and OSes [1]

For 32bit atomics LKL also uses the asm-generic implementation. It is
very similar with generic 64bit atomic implementation and it is used
by multiple 32bit arches. I think this was my original reasoning for
this patch and not going with C11 atomics.

> One thing to keep in mind is that the C11 atomics (_Atomic) don't
> trivially map to the LKMM -- although I keep forgetting the exact
> details, there is a paper on it somewhere.  Also, once you're limited to
> a specific arch the issue also becomes much easier than C11 vs LKMM in
> general.

Thanks, I will look it up.

[1] https://github.com/lkl/linux

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v3 01/26] asm-generic: atomic64: allow using generic atomic64 on 64bit platforms
@ 2020-02-05 14:00           ` Octavian Purdila
  0 siblings, 0 replies; 476+ messages in thread
From: Octavian Purdila @ 2020-02-05 14:00 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: linux-arch, Boqun Feng, linux-um, Akira Moroo,
	linux-kernel-library, Will Deacon, Hajime Tazaki

On Wed, Feb 5, 2020 at 2:49 PM Peter Zijlstra <peterz@infradead.org> wrote:
>
> On Wed, Feb 05, 2020 at 02:24:38PM +0200, Octavian Purdila wrote:
> > I was not aware that not allowing GENERIC_ATOMIC64 was intentional. I
>
> It might not have been, but presented with this patch, I feel like it
> should've been :-)
>
> > understand your point that a 64 bit architecture that can't handle 64
> > bit atomic operation is broken.
>
> (sadly they actually exist, I shall name no names)
>
> > One way to deal with this in LKL would be to use GCC atomic builtins
> > or if that doesn't work expose them as host operations. This would
> > keep LKL as a meta-arch that can run on multiple physical
> > architectures. I'll give it a try.
>
> What is this LKL you speak of and how does it do the 32bit atomics?
>

LKL is a build of the Linux kernel as a library that can run in many
environments including multiple architectures and OSes [1]

For 32bit atomics LKL also uses the asm-generic implementation. It is
very similar with generic 64bit atomic implementation and it is used
by multiple 32bit arches. I think this was my original reasoning for
this patch and not going with C11 atomics.

> One thing to keep in mind is that the C11 atomics (_Atomic) don't
> trivially map to the LKMM -- although I keep forgetting the exact
> details, there is a paper on it somewhere.  Also, once you're limited to
> a specific arch the issue also becomes much easier than C11 vs LKMM in
> general.

Thanks, I will look it up.

[1] https://github.com/lkl/linux

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v3 07/26] um lkl: interrupt support
  2020-02-05 10:47     ` Anton Ivanov
@ 2020-02-05 14:46       ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05 14:46 UTC (permalink / raw)
  To: anton.ivanov
  Cc: linux-um, linux-arch, tavi.purdila, retrage01,
	linux-kernel-library, sigmaepsilon92


On Wed, 05 Feb 2020 19:47:36 +0900,
Anton Ivanov wrote:

> > +/**
> > + * This function can be called from arbitrary host threads, so do not
> > + * issue any Linux calls (e.g. prink) if lkl_cpu_get() was not issued
> > + * before.
> > + */
> > +int lkl_trigger_irq(int irq)
> > +{
> > +	int ret;
> > +
> > +	if (!irq || irq > NR_IRQS)
> > +		return -EINVAL;
> > +
> > +	ret = lkl_cpu_try_run_irq(irq);
> > +	if (ret <= 0)
> > +		return ret;
> > +
> > +	/*
> > +	 * Since this can be called from Linux context (e.g. lkl_trigger_irq ->
> > +	 * IRQ -> softirq -> lkl_trigger_irq) make sure we are actually allowed
> > +	 * to run irqs at this point
> > +	 */
> > +	if (!irqs_enabled) {
> > +		set_irq_pending(irq);
> > +		lkl_cpu_put();
> > +		return 0;
> > +	}
> > +
> > +	run_irq(irq);
> > +
> > +	lkl_cpu_put();
> > +
> > +	return 0;
> 
> Isn't that just:
> 
> 	if (irqs_enabled)
> 		run_irq(irq);
> 	else
> 		set_irq_pending(irq);
> 
> 	lkl_cpu_put();
> 
> 	return 0;

Thanks, this is much cleaner.  I will fix this in the next turn.

-- Hajime

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v3 07/26] um lkl: interrupt support
@ 2020-02-05 14:46       ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-02-05 14:46 UTC (permalink / raw)
  To: anton.ivanov
  Cc: linux-arch, tavi.purdila, linux-um, retrage01,
	linux-kernel-library, sigmaepsilon92


On Wed, 05 Feb 2020 19:47:36 +0900,
Anton Ivanov wrote:

> > +/**
> > + * This function can be called from arbitrary host threads, so do not
> > + * issue any Linux calls (e.g. prink) if lkl_cpu_get() was not issued
> > + * before.
> > + */
> > +int lkl_trigger_irq(int irq)
> > +{
> > +	int ret;
> > +
> > +	if (!irq || irq > NR_IRQS)
> > +		return -EINVAL;
> > +
> > +	ret = lkl_cpu_try_run_irq(irq);
> > +	if (ret <= 0)
> > +		return ret;
> > +
> > +	/*
> > +	 * Since this can be called from Linux context (e.g. lkl_trigger_irq ->
> > +	 * IRQ -> softirq -> lkl_trigger_irq) make sure we are actually allowed
> > +	 * to run irqs at this point
> > +	 */
> > +	if (!irqs_enabled) {
> > +		set_irq_pending(irq);
> > +		lkl_cpu_put();
> > +		return 0;
> > +	}
> > +
> > +	run_irq(irq);
> > +
> > +	lkl_cpu_put();
> > +
> > +	return 0;
> 
> Isn't that just:
> 
> 	if (irqs_enabled)
> 		run_irq(irq);
> 	else
> 		set_irq_pending(irq);
> 
> 	lkl_cpu_put();
> 
> 	return 0;

Thanks, this is much cleaner.  I will fix this in the next turn.

-- Hajime


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v3 01/26] asm-generic: atomic64: allow using generic atomic64 on 64bit platforms
  2020-02-05 14:00           ` Octavian Purdila
@ 2020-02-05 17:13             ` Peter Zijlstra
  -1 siblings, 0 replies; 476+ messages in thread
From: Peter Zijlstra @ 2020-02-05 17:13 UTC (permalink / raw)
  To: Octavian Purdila
  Cc: Hajime Tazaki, linux-um, Akira Moroo, linux-kernel-library,
	linux-arch, Will Deacon, Boqun Feng

On Wed, Feb 05, 2020 at 04:00:41PM +0200, Octavian Purdila wrote:
> On Wed, Feb 5, 2020 at 2:49 PM Peter Zijlstra <peterz@infradead.org> wrote:
> >
> > On Wed, Feb 05, 2020 at 02:24:38PM +0200, Octavian Purdila wrote:
> > > I was not aware that not allowing GENERIC_ATOMIC64 was intentional. I
> >
> > It might not have been, but presented with this patch, I feel like it
> > should've been :-)
> >
> > > understand your point that a 64 bit architecture that can't handle 64
> > > bit atomic operation is broken.
> >
> > (sadly they actually exist, I shall name no names)
> >
> > > One way to deal with this in LKL would be to use GCC atomic builtins
> > > or if that doesn't work expose them as host operations. This would
> > > keep LKL as a meta-arch that can run on multiple physical
> > > architectures. I'll give it a try.
> >
> > What is this LKL you speak of and how does it do the 32bit atomics?
> >
> 
> LKL is a build of the Linux kernel as a library that can run in many
> environments including multiple architectures and OSes [1]

Thanks, I'll put it on the to-read list.

> For 32bit atomics LKL also uses the asm-generic implementation. It is
> very similar with generic 64bit atomic implementation and it is used
> by multiple 32bit arches. I think this was my original reasoning for
> this patch and not going with C11 atomics.

Uh no, asm-generic/atomic.h is radically different from lib/atomic64.c.

asm-generic/atomic.h builds all required atomic operations from
cmpxchg() (loops), while lib/atomic64.c builds 64bit atomics by using a
hashed set of spinlocks.

The asm-generic stuff gives you real atomic ops, albeit sub-optimal,
lib/atomic64.c gives you a turd.

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v3 01/26] asm-generic: atomic64: allow using generic atomic64 on 64bit platforms
@ 2020-02-05 17:13             ` Peter Zijlstra
  0 siblings, 0 replies; 476+ messages in thread
From: Peter Zijlstra @ 2020-02-05 17:13 UTC (permalink / raw)
  To: Octavian Purdila
  Cc: linux-arch, Boqun Feng, linux-um, Akira Moroo,
	linux-kernel-library, Will Deacon, Hajime Tazaki

On Wed, Feb 05, 2020 at 04:00:41PM +0200, Octavian Purdila wrote:
> On Wed, Feb 5, 2020 at 2:49 PM Peter Zijlstra <peterz@infradead.org> wrote:
> >
> > On Wed, Feb 05, 2020 at 02:24:38PM +0200, Octavian Purdila wrote:
> > > I was not aware that not allowing GENERIC_ATOMIC64 was intentional. I
> >
> > It might not have been, but presented with this patch, I feel like it
> > should've been :-)
> >
> > > understand your point that a 64 bit architecture that can't handle 64
> > > bit atomic operation is broken.
> >
> > (sadly they actually exist, I shall name no names)
> >
> > > One way to deal with this in LKL would be to use GCC atomic builtins
> > > or if that doesn't work expose them as host operations. This would
> > > keep LKL as a meta-arch that can run on multiple physical
> > > architectures. I'll give it a try.
> >
> > What is this LKL you speak of and how does it do the 32bit atomics?
> >
> 
> LKL is a build of the Linux kernel as a library that can run in many
> environments including multiple architectures and OSes [1]

Thanks, I'll put it on the to-read list.

> For 32bit atomics LKL also uses the asm-generic implementation. It is
> very similar with generic 64bit atomic implementation and it is used
> by multiple 32bit arches. I think this was my original reasoning for
> this patch and not going with C11 atomics.

Uh no, asm-generic/atomic.h is radically different from lib/atomic64.c.

asm-generic/atomic.h builds all required atomic operations from
cmpxchg() (loops), while lib/atomic64.c builds 64bit atomics by using a
hashed set of spinlocks.

The asm-generic stuff gives you real atomic ops, albeit sub-optimal,
lib/atomic64.c gives you a turd.

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v3 01/26] asm-generic: atomic64: allow using generic atomic64 on 64bit platforms
  2020-02-05 17:13             ` Peter Zijlstra
@ 2020-02-07 12:32               ` Octavian Purdila
  -1 siblings, 0 replies; 476+ messages in thread
From: Octavian Purdila @ 2020-02-07 12:32 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: Hajime Tazaki, linux-um, Akira Moroo, linux-kernel-library,
	linux-arch, Will Deacon, Boqun Feng

On Wed, Feb 5, 2020 at 7:13 PM Peter Zijlstra <peterz@infradead.org> wrote:
>
> On Wed, Feb 05, 2020 at 04:00:41PM +0200, Octavian Purdila wrote:
> > On Wed, Feb 5, 2020 at 2:49 PM Peter Zijlstra <peterz@infradead.org> wrote:
> > >
> > > On Wed, Feb 05, 2020 at 02:24:38PM +0200, Octavian Purdila wrote:
> > > > I was not aware that not allowing GENERIC_ATOMIC64 was intentional. I
> > >
> > > It might not have been, but presented with this patch, I feel like it
> > > should've been :-)
> > >
> > > > understand your point that a 64 bit architecture that can't handle 64
> > > > bit atomic operation is broken.
> > >
> > > (sadly they actually exist, I shall name no names)
> > >
> > > > One way to deal with this in LKL would be to use GCC atomic builtins
> > > > or if that doesn't work expose them as host operations. This would
> > > > keep LKL as a meta-arch that can run on multiple physical
> > > > architectures. I'll give it a try.
> > >
> > > What is this LKL you speak of and how does it do the 32bit atomics?
> > >
> >
> > LKL is a build of the Linux kernel as a library that can run in many
> > environments including multiple architectures and OSes [1]
>
> Thanks, I'll put it on the to-read list.
>
> > For 32bit atomics LKL also uses the asm-generic implementation. It is
> > very similar with generic 64bit atomic implementation and it is used
> > by multiple 32bit arches. I think this was my original reasoning for
> > this patch and not going with C11 atomics.
>
> Uh no, asm-generic/atomic.h is radically different from lib/atomic64.c.
>
> asm-generic/atomic.h builds all required atomic operations from
> cmpxchg() (loops), while lib/atomic64.c builds 64bit atomics by using a
> hashed set of spinlocks.
>
> The asm-generic stuff gives you real atomic ops, albeit sub-optimal,
> lib/atomic64.c gives you a turd.

Ah, yes, I overlooked that.

(I think that they are equivalent for LKL because it is a non-SMP arch
only at this moment, but I agree that the better approach is to have a
proper implementation in LKL)

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v3 01/26] asm-generic: atomic64: allow using generic atomic64 on 64bit platforms
@ 2020-02-07 12:32               ` Octavian Purdila
  0 siblings, 0 replies; 476+ messages in thread
From: Octavian Purdila @ 2020-02-07 12:32 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: linux-arch, Boqun Feng, linux-um, Akira Moroo,
	linux-kernel-library, Will Deacon, Hajime Tazaki

On Wed, Feb 5, 2020 at 7:13 PM Peter Zijlstra <peterz@infradead.org> wrote:
>
> On Wed, Feb 05, 2020 at 04:00:41PM +0200, Octavian Purdila wrote:
> > On Wed, Feb 5, 2020 at 2:49 PM Peter Zijlstra <peterz@infradead.org> wrote:
> > >
> > > On Wed, Feb 05, 2020 at 02:24:38PM +0200, Octavian Purdila wrote:
> > > > I was not aware that not allowing GENERIC_ATOMIC64 was intentional. I
> > >
> > > It might not have been, but presented with this patch, I feel like it
> > > should've been :-)
> > >
> > > > understand your point that a 64 bit architecture that can't handle 64
> > > > bit atomic operation is broken.
> > >
> > > (sadly they actually exist, I shall name no names)
> > >
> > > > One way to deal with this in LKL would be to use GCC atomic builtins
> > > > or if that doesn't work expose them as host operations. This would
> > > > keep LKL as a meta-arch that can run on multiple physical
> > > > architectures. I'll give it a try.
> > >
> > > What is this LKL you speak of and how does it do the 32bit atomics?
> > >
> >
> > LKL is a build of the Linux kernel as a library that can run in many
> > environments including multiple architectures and OSes [1]
>
> Thanks, I'll put it on the to-read list.
>
> > For 32bit atomics LKL also uses the asm-generic implementation. It is
> > very similar with generic 64bit atomic implementation and it is used
> > by multiple 32bit arches. I think this was my original reasoning for
> > this patch and not going with C11 atomics.
>
> Uh no, asm-generic/atomic.h is radically different from lib/atomic64.c.
>
> asm-generic/atomic.h builds all required atomic operations from
> cmpxchg() (loops), while lib/atomic64.c builds 64bit atomics by using a
> hashed set of spinlocks.
>
> The asm-generic stuff gives you real atomic ops, albeit sub-optimal,
> lib/atomic64.c gives you a turd.

Ah, yes, I overlooked that.

(I think that they are equivalent for LKL because it is a non-SMP arch
only at this moment, but I agree that the better approach is to have a
proper implementation in LKL)

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* [RFC v4 00/25] Unifying LKL into UML
  2020-02-05  7:30 [RFC v3 00/26] Unifying LKL into UML Hajime Tazaki
@ 2020-03-30 14:45   ` Hajime Tazaki
       [not found] ` <cover.1580882335.git.thehajime-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
                     ` (25 subsequent siblings)
  26 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Hajime Tazaki

This is another spin of the unification of LKL into UML.  I've updated
the patchset by addressing comments received so far.


Changes in rfc v4:
- Rebase on the current uml/master branch
- Fix IRQ handling (bug fix)
- drop a patch for CONFIG_GENERIC_ATOMIC64 support (comment by Peter Zijlstra)
- implement vector net driver for UMMODE_LIB (comment by Anton)
- clean up uapi headers to avoid duplicates
- clean up IRQ handling code (comment by Anton)
- fix error handling in test code (comments by David Disseldorp)

Changes in rfc v3:
- use UML drivers (net, block) from LKL programs
- drop virtio device implementations
- drop mingw32 (Windows) host
- drop android (arm/aarch64) host
- drop FreeBSD (x86_64) host
- drop LD_PRELOAD (hijack) support
- update milestone

rfc v2:
- use UMMODE instead of SUBARCH to switch UML or LKL
- tools/lkl directory is still there. I confirmed we can move under arch/um
  (e.g., arch/um/lkl/hosts).  I will move it IF this is preferable.
- drop several patches involved non-uml directory
- drop several patches which are not required
- refine commit logs
- document updated




LKL (Linux Kernel Library) is aiming to allow reusing the Linux kernel code
as extensively as possible with minimal effort and reduced maintenance
overhead.

Examples of how LKL can be used are: creating userspace applications
(running on Linux and other operating systems) that can read or write Linux
filesystems or can use the Linux networking stack, creating kernel drivers
for other operating systems that can read Linux filesystems, bootloaders
support for reading/writing Linux filesystems, etc.

With LKL, the kernel code is compiled into an object file that can be
directly linked by applications. The API offered by LKL is based on the
Linux system call interface.

LKL is originally implemented as an architecture port in arch/lkl, but this
series of commits tries to integrate this into arch/um as one of the mode
of UML.  This was discussed during RFC email of LKL (*1).

The latest LKL version can be found at https://github.com/lkl/linux

Milestone
=========
This patches is a first step toward upstreaming *library mode* of Linux kernel,
but we think we need to have several steps toward our goal, describing in the
below.

1. Put LKL code under arch/um (arch/um/lkl), and build it in a separate way
from UML.
2. Use UML driver implementations in LKL as a minimum set of patches
-  Only support x86 sub architecture (dependency to UML drivers)
3. Support broader host supports
- add virtio device features

For the step 1, we put LKL as one of UMMODE in order to make less effort to
integrate (make ARCH=um UMMODE=library).  The modification to existing UML
code is trying to be minimized.



Building LKL the host library and LKL applications
==================================================

% cd tools/lkl
% make

will build LKL as a object file, it will install it in tools/lkl/lib together
with the headers files in tools/lkl/include then will build the host library,
tests and a few of application examples:

* tests/boot - a simple applications that uses LKL and exercises the basic
  LKL APIs

* tests/net-test - a simple applications that uses network feature of
  LKL and exercises the basic network-related APIs

* fs2tar - a tool that converts a filesystem image to a tar archive

* cptofs/cpfromfs - a tool that copies files to/from a filesystem image

* lklfuse - a tool that can mount a filesystem image in userspace,
  without root priviledges, using FUSE

% make run-tests

should run the above `tests/boot` and `tests/net-test` and report errors if
there are any.

Supported hosts
===============

Currently LKL supports Linux userspace applications. New hosts can be added
relatively easy if the host supports gcc and GNU ld. Previous versions of
LKL supported Windows kernel and Haiku kernel hosts, and we also have WIP
patches with rump-hypercall interface, used in UEFI, as well as macOS
userspace (part of POSIX).

There is also musl-libc port for LKL, which might be interested in for some
folks.


Further readings about LKL
=========================

- Discussion in github LKL issue
https://github.com/lkl/linux/issues/304

- LKL (an article)
https://www.researchgate.net/profile/Nicolae_Tapus2/publication/224164682_LKL_The_Linux_kernel_library/links/02bfe50fd921ab4f7c000000.pdf

*1 RFC email to LKML (back in 2015)
https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1012277.html



Please review the following changes for suitability for inclusion. If you have
any objections or suggestions for improvement, please respond to the patches. If
you agree with the changes, please provide your Acked-by.

The following changes since commit 4a7c46247f9c620c0390a15cb00b6ef9576b9c23:

  um: Remove some unnecessary NULL checks in vector_user.c (2020-03-29 23:56:47 +0200)

are available in the Git repository at:

  git://github.com/thehajime/linux 59ed4e23efa7abc9c7d52e6b5fb31fb8ce795c5c
  https://github.com/thehajime/linux/tree/upstream-to-uml-5.6-rc7-v4

Hajime Tazaki (6):
  lkl tools: host lib: filesystem helpers
  lkl tools: host lib: networking helpers
  um lkl: add CI scripts to conduct regression tests
  um lkl: add UML network driver for lkl
  um lkl: add UML block device driver (ubd) for lkl
  um: fix clone flags to be familiar with valgrind

Octavian Purdila (19):
  arch: add __SYSCALL_DEFINE_ARCH
  um lkl: architecture skeleton for Linux kernel library
  um lkl: host interface
  um lkl: memory handling
  um lkl: kernel threads support
  um lkl: interrupt support
  um lkl: system call interface and application API
  um lkl: timers, time and delay support
  um lkl: basic kernel console support
  um lkl: initialization and cleanup
  um lkl: plug in the build system
  lkl tools: skeleton for host side library
  lkl tools: host lib: add utilities functions
  lkl tools: host lib: posix host operations
  lkl tools: add test programs
  lkl tools: cptofs that reads/writes to/from a filesystem image
  lkl tools: fs2tar that converts a filesystem image to tar
  lkl tools: add lklfuse
  um lkl: add documentation

 .circleci/config.yml                       |  198 ++++
 Documentation/virt/uml/lkl.txt             |   64 ++
 MAINTAINERS                                |    8 +
 arch/um/Kconfig                            |   13 +
 arch/um/Makefile                           |   19 +-
 arch/um/Makefile.um                        |   10 +
 arch/um/drivers/Makefile                   |    2 +
 arch/um/drivers/ubd_kern.c                 |    6 +-
 arch/um/drivers/ubd_user.c                 |    3 +-
 arch/um/include/asm/irq.h                  |    2 +
 arch/um/kernel/Makefile                    |    4 +
 arch/um/kernel/irq.c                       |    4 +
 arch/um/lkl/.gitignore                     |    2 +
 arch/um/lkl/Kconfig                        |   88 ++
 arch/um/lkl/Kconfig.debug                  |    0
 arch/um/lkl/Makefile                       |   67 ++
 arch/um/lkl/auto.conf                      |    1 +
 arch/um/lkl/configs/lkl_defconfig          | 1052 ++++++++++++++++++++
 arch/um/lkl/include/asm/Kbuild             |   79 ++
 arch/um/lkl/include/asm/atomic.h           |   11 +
 arch/um/lkl/include/asm/atomic64.h         |  114 +++
 arch/um/lkl/include/asm/bitsperlong.h      |   11 +
 arch/um/lkl/include/asm/byteorder.h        |    7 +
 arch/um/lkl/include/asm/cpu.h              |   14 +
 arch/um/lkl/include/asm/elf.h              |   15 +
 arch/um/lkl/include/asm/host_ops.h         |   10 +
 arch/um/lkl/include/asm/irq.h              |   16 +
 arch/um/lkl/include/asm/page.h             |   14 +
 arch/um/lkl/include/asm/pgtable.h          |   57 ++
 arch/um/lkl/include/asm/processor.h        |   60 ++
 arch/um/lkl/include/asm/ptrace.h           |   25 +
 arch/um/lkl/include/asm/sched.h            |   23 +
 arch/um/lkl/include/asm/setup.h            |    7 +
 arch/um/lkl/include/asm/syscalls.h         |   18 +
 arch/um/lkl/include/asm/syscalls_32.h      |   43 +
 arch/um/lkl/include/asm/thread_info.h      |   70 ++
 arch/um/lkl/include/asm/tlb.h              |   12 +
 arch/um/lkl/include/asm/uaccess.h          |   64 ++
 arch/um/lkl/include/asm/unistd.h           |   27 +
 arch/um/lkl/include/asm/unistd_32.h        |   31 +
 arch/um/lkl/include/asm/vmalloc.h          |    5 +
 arch/um/lkl/include/asm/vmlinux.lds.h      |   14 +
 arch/um/lkl/include/asm/xor.h              |    9 +
 arch/um/lkl/include/uapi/asm/Kbuild        |   11 +
 arch/um/lkl/include/uapi/asm/bitsperlong.h |   13 +
 arch/um/lkl/include/uapi/asm/byteorder.h   |   11 +
 arch/um/lkl/include/uapi/asm/host_ops.h    |  143 +++
 arch/um/lkl/include/uapi/asm/irq.h         |   36 +
 arch/um/lkl/include/uapi/asm/sigcontext.h  |   16 +
 arch/um/lkl/include/uapi/asm/syscalls.h    |  349 +++++++
 arch/um/lkl/include/uapi/asm/unistd.h      |   13 +
 arch/um/lkl/kernel/Makefile                |    4 +
 arch/um/lkl/kernel/asm-offsets.c           |    3 +
 arch/um/lkl/kernel/console.c               |   42 +
 arch/um/lkl/kernel/cpu.c                   |  223 +++++
 arch/um/lkl/kernel/irq.c                   |  223 +++++
 arch/um/lkl/kernel/misc.c                  |   60 ++
 arch/um/lkl/kernel/setup.c                 |  198 ++++
 arch/um/lkl/kernel/syscalls.c              |  192 ++++
 arch/um/lkl/kernel/syscalls_32.c           |  159 +++
 arch/um/lkl/kernel/threads.c               |  227 +++++
 arch/um/lkl/kernel/time.c                  |  145 +++
 arch/um/lkl/kernel/vmlinux.lds.S           |   51 +
 arch/um/lkl/mm/Makefile                    |    1 +
 arch/um/lkl/mm/bootmem.c                   |   99 ++
 arch/um/lkl/scripts/headers_install.py     |  196 ++++
 arch/um/os-Linux/Makefile                  |    5 +
 include/linux/syscalls.h                   |    6 +
 tools/lkl/.gitignore                       |   13 +
 tools/lkl/Build                            |    6 +
 tools/lkl/Makefile                         |  131 +++
 tools/lkl/Makefile.autoconf                |   67 ++
 tools/lkl/Targets                          |   17 +
 tools/lkl/cptofs.c                         |  636 ++++++++++++
 tools/lkl/fs2tar.c                         |  412 ++++++++
 tools/lkl/include/.gitignore               |    1 +
 tools/lkl/include/lkl.h                    |  788 +++++++++++++++
 tools/lkl/include/lkl_config.h             |   61 ++
 tools/lkl/include/lkl_host.h               |  112 +++
 tools/lkl/lib/.gitignore                   |    3 +
 tools/lkl/lib/Build                        |   13 +
 tools/lkl/lib/config.c                     |  750 ++++++++++++++
 tools/lkl/lib/dbg.c                        |  300 ++++++
 tools/lkl/lib/dbg_handler.c                |   44 +
 tools/lkl/lib/endian.h                     |   31 +
 tools/lkl/lib/fs.c                         |  471 +++++++++
 tools/lkl/lib/jmp_buf.c                    |   14 +
 tools/lkl/lib/jmp_buf.h                    |    8 +
 tools/lkl/lib/net.c                        |  834 ++++++++++++++++
 tools/lkl/lib/posix-host.c                 |  356 +++++++
 tools/lkl/lib/um/Build                     |    3 +
 tools/lkl/lib/um/um_block.c                |   17 +
 tools/lkl/lib/um/um_glue.c                 |   61 ++
 tools/lkl/lib/um/um_net.c                  |   30 +
 tools/lkl/lib/utils.c                      |  266 +++++
 tools/lkl/lklfuse.c                        |  659 ++++++++++++
 tools/lkl/scripts/checkpatch.sh            |   60 ++
 tools/lkl/scripts/lkl-jenkins.sh           |   21 +
 tools/lkl/tests/Build                      |    3 +
 tools/lkl/tests/boot.c                     |  562 +++++++++++
 tools/lkl/tests/boot.sh                    |    9 +
 tools/lkl/tests/cla.c                      |  159 +++
 tools/lkl/tests/cla.h                      |   33 +
 tools/lkl/tests/disk.c                     |  203 ++++
 tools/lkl/tests/disk.sh                    |   61 ++
 tools/lkl/tests/lklfuse.sh                 |  110 ++
 tools/lkl/tests/net-setup.sh               |  136 +++
 tools/lkl/tests/net-test.c                 |  307 ++++++
 tools/lkl/tests/net.sh                     |  135 +++
 tools/lkl/tests/run.py                     |  181 ++++
 tools/lkl/tests/tap13.py                   |  209 ++++
 tools/lkl/tests/test.c                     |  126 +++
 tools/lkl/tests/test.h                     |   72 ++
 tools/lkl/tests/test.sh                    |  182 ++++
 tools/lkl/tests/valgrind.supp              |  105 ++
 tools/lkl/tests/valgrind2xunit.py          |   69 ++
 116 files changed, 13569 insertions(+), 3 deletions(-)
 create mode 100644 .circleci/config.yml
 create mode 100644 Documentation/virt/uml/lkl.txt
 create mode 100644 arch/um/Makefile.um
 create mode 100644 arch/um/lkl/.gitignore
 create mode 100644 arch/um/lkl/Kconfig
 create mode 100644 arch/um/lkl/Kconfig.debug
 create mode 100644 arch/um/lkl/Makefile
 create mode 100644 arch/um/lkl/auto.conf
 create mode 100644 arch/um/lkl/configs/lkl_defconfig
 create mode 100644 arch/um/lkl/include/asm/Kbuild
 create mode 100644 arch/um/lkl/include/asm/atomic.h
 create mode 100644 arch/um/lkl/include/asm/atomic64.h
 create mode 100644 arch/um/lkl/include/asm/bitsperlong.h
 create mode 100644 arch/um/lkl/include/asm/byteorder.h
 create mode 100644 arch/um/lkl/include/asm/cpu.h
 create mode 100644 arch/um/lkl/include/asm/elf.h
 create mode 100644 arch/um/lkl/include/asm/host_ops.h
 create mode 100644 arch/um/lkl/include/asm/irq.h
 create mode 100644 arch/um/lkl/include/asm/page.h
 create mode 100644 arch/um/lkl/include/asm/pgtable.h
 create mode 100644 arch/um/lkl/include/asm/processor.h
 create mode 100644 arch/um/lkl/include/asm/ptrace.h
 create mode 100644 arch/um/lkl/include/asm/sched.h
 create mode 100644 arch/um/lkl/include/asm/setup.h
 create mode 100644 arch/um/lkl/include/asm/syscalls.h
 create mode 100644 arch/um/lkl/include/asm/syscalls_32.h
 create mode 100644 arch/um/lkl/include/asm/thread_info.h
 create mode 100644 arch/um/lkl/include/asm/tlb.h
 create mode 100644 arch/um/lkl/include/asm/uaccess.h
 create mode 100644 arch/um/lkl/include/asm/unistd.h
 create mode 100644 arch/um/lkl/include/asm/unistd_32.h
 create mode 100644 arch/um/lkl/include/asm/vmalloc.h
 create mode 100644 arch/um/lkl/include/asm/vmlinux.lds.h
 create mode 100644 arch/um/lkl/include/asm/xor.h
 create mode 100644 arch/um/lkl/include/uapi/asm/Kbuild
 create mode 100644 arch/um/lkl/include/uapi/asm/bitsperlong.h
 create mode 100644 arch/um/lkl/include/uapi/asm/byteorder.h
 create mode 100644 arch/um/lkl/include/uapi/asm/host_ops.h
 create mode 100644 arch/um/lkl/include/uapi/asm/irq.h
 create mode 100644 arch/um/lkl/include/uapi/asm/sigcontext.h
 create mode 100644 arch/um/lkl/include/uapi/asm/syscalls.h
 create mode 100644 arch/um/lkl/include/uapi/asm/unistd.h
 create mode 100644 arch/um/lkl/kernel/Makefile
 create mode 100644 arch/um/lkl/kernel/asm-offsets.c
 create mode 100644 arch/um/lkl/kernel/console.c
 create mode 100644 arch/um/lkl/kernel/cpu.c
 create mode 100644 arch/um/lkl/kernel/irq.c
 create mode 100644 arch/um/lkl/kernel/misc.c
 create mode 100644 arch/um/lkl/kernel/setup.c
 create mode 100644 arch/um/lkl/kernel/syscalls.c
 create mode 100644 arch/um/lkl/kernel/syscalls_32.c
 create mode 100644 arch/um/lkl/kernel/threads.c
 create mode 100644 arch/um/lkl/kernel/time.c
 create mode 100644 arch/um/lkl/kernel/vmlinux.lds.S
 create mode 100644 arch/um/lkl/mm/Makefile
 create mode 100644 arch/um/lkl/mm/bootmem.c
 create mode 100755 arch/um/lkl/scripts/headers_install.py
 create mode 100644 tools/lkl/.gitignore
 create mode 100644 tools/lkl/Build
 create mode 100644 tools/lkl/Makefile
 create mode 100644 tools/lkl/Makefile.autoconf
 create mode 100644 tools/lkl/Targets
 create mode 100644 tools/lkl/cptofs.c
 create mode 100644 tools/lkl/fs2tar.c
 create mode 100644 tools/lkl/include/.gitignore
 create mode 100644 tools/lkl/include/lkl.h
 create mode 100644 tools/lkl/include/lkl_config.h
 create mode 100644 tools/lkl/include/lkl_host.h
 create mode 100644 tools/lkl/lib/.gitignore
 create mode 100644 tools/lkl/lib/Build
 create mode 100644 tools/lkl/lib/config.c
 create mode 100644 tools/lkl/lib/dbg.c
 create mode 100644 tools/lkl/lib/dbg_handler.c
 create mode 100644 tools/lkl/lib/endian.h
 create mode 100644 tools/lkl/lib/fs.c
 create mode 100644 tools/lkl/lib/jmp_buf.c
 create mode 100644 tools/lkl/lib/jmp_buf.h
 create mode 100644 tools/lkl/lib/net.c
 create mode 100644 tools/lkl/lib/posix-host.c
 create mode 100644 tools/lkl/lib/um/Build
 create mode 100644 tools/lkl/lib/um/um_block.c
 create mode 100644 tools/lkl/lib/um/um_glue.c
 create mode 100644 tools/lkl/lib/um/um_net.c
 create mode 100644 tools/lkl/lib/utils.c
 create mode 100644 tools/lkl/lklfuse.c
 create mode 100755 tools/lkl/scripts/checkpatch.sh
 create mode 100755 tools/lkl/scripts/lkl-jenkins.sh
 create mode 100644 tools/lkl/tests/Build
 create mode 100644 tools/lkl/tests/boot.c
 create mode 100755 tools/lkl/tests/boot.sh
 create mode 100644 tools/lkl/tests/cla.c
 create mode 100644 tools/lkl/tests/cla.h
 create mode 100644 tools/lkl/tests/disk.c
 create mode 100755 tools/lkl/tests/disk.sh
 create mode 100755 tools/lkl/tests/lklfuse.sh
 create mode 100644 tools/lkl/tests/net-setup.sh
 create mode 100644 tools/lkl/tests/net-test.c
 create mode 100755 tools/lkl/tests/net.sh
 create mode 100755 tools/lkl/tests/run.py
 create mode 100644 tools/lkl/tests/tap13.py
 create mode 100644 tools/lkl/tests/test.c
 create mode 100644 tools/lkl/tests/test.h
 create mode 100644 tools/lkl/tests/test.sh
 create mode 100644 tools/lkl/tests/valgrind.supp
 create mode 100755 tools/lkl/tests/valgrind2xunit.py

-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply	[flat|nested] 476+ messages in thread
* [RFC v4 00/25] Unifying LKL into UML
@ 2020-03-30 14:45   ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, linux-kernel-library, linux-arch,
	Hajime Tazaki, Akira Moroo

This is another spin of the unification of LKL into UML.  I've updated
the patchset by addressing comments received so far.


Changes in rfc v4:
- Rebase on the current uml/master branch
- Fix IRQ handling (bug fix)
- drop a patch for CONFIG_GENERIC_ATOMIC64 support (comment by Peter Zijlstra)
- implement vector net driver for UMMODE_LIB (comment by Anton)
- clean up uapi headers to avoid duplicates
- clean up IRQ handling code (comment by Anton)
- fix error handling in test code (comments by David Disseldorp)

Changes in rfc v3:
- use UML drivers (net, block) from LKL programs
- drop virtio device implementations
- drop mingw32 (Windows) host
- drop android (arm/aarch64) host
- drop FreeBSD (x86_64) host
- drop LD_PRELOAD (hijack) support
- update milestone

rfc v2:
- use UMMODE instead of SUBARCH to switch UML or LKL
- tools/lkl directory is still there. I confirmed we can move under arch/um
  (e.g., arch/um/lkl/hosts).  I will move it IF this is preferable.
- drop several patches involved non-uml directory
- drop several patches which are not required
- refine commit logs
- document updated




LKL (Linux Kernel Library) is aiming to allow reusing the Linux kernel code
as extensively as possible with minimal effort and reduced maintenance
overhead.

Examples of how LKL can be used are: creating userspace applications
(running on Linux and other operating systems) that can read or write Linux
filesystems or can use the Linux networking stack, creating kernel drivers
for other operating systems that can read Linux filesystems, bootloaders
support for reading/writing Linux filesystems, etc.

With LKL, the kernel code is compiled into an object file that can be
directly linked by applications. The API offered by LKL is based on the
Linux system call interface.

LKL is originally implemented as an architecture port in arch/lkl, but this
series of commits tries to integrate this into arch/um as one of the mode
of UML.  This was discussed during RFC email of LKL (*1).

The latest LKL version can be found at https://github.com/lkl/linux

Milestone
=========
This patches is a first step toward upstreaming *library mode* of Linux kernel,
but we think we need to have several steps toward our goal, describing in the
below.

1. Put LKL code under arch/um (arch/um/lkl), and build it in a separate way
from UML.
2. Use UML driver implementations in LKL as a minimum set of patches
-  Only support x86 sub architecture (dependency to UML drivers)
3. Support broader host supports
- add virtio device features

For the step 1, we put LKL as one of UMMODE in order to make less effort to
integrate (make ARCH=um UMMODE=library).  The modification to existing UML
code is trying to be minimized.



Building LKL the host library and LKL applications
==================================================

% cd tools/lkl
% make

will build LKL as a object file, it will install it in tools/lkl/lib together
with the headers files in tools/lkl/include then will build the host library,
tests and a few of application examples:

* tests/boot - a simple applications that uses LKL and exercises the basic
  LKL APIs

* tests/net-test - a simple applications that uses network feature of
  LKL and exercises the basic network-related APIs

* fs2tar - a tool that converts a filesystem image to a tar archive

* cptofs/cpfromfs - a tool that copies files to/from a filesystem image

* lklfuse - a tool that can mount a filesystem image in userspace,
  without root priviledges, using FUSE

% make run-tests

should run the above `tests/boot` and `tests/net-test` and report errors if
there are any.

Supported hosts
===============

Currently LKL supports Linux userspace applications. New hosts can be added
relatively easy if the host supports gcc and GNU ld. Previous versions of
LKL supported Windows kernel and Haiku kernel hosts, and we also have WIP
patches with rump-hypercall interface, used in UEFI, as well as macOS
userspace (part of POSIX).

There is also musl-libc port for LKL, which might be interested in for some
folks.


Further readings about LKL
=========================

- Discussion in github LKL issue
https://github.com/lkl/linux/issues/304

- LKL (an article)
https://www.researchgate.net/profile/Nicolae_Tapus2/publication/224164682_LKL_The_Linux_kernel_library/links/02bfe50fd921ab4f7c000000.pdf

*1 RFC email to LKML (back in 2015)
https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1012277.html



Please review the following changes for suitability for inclusion. If you have
any objections or suggestions for improvement, please respond to the patches. If
you agree with the changes, please provide your Acked-by.

The following changes since commit 4a7c46247f9c620c0390a15cb00b6ef9576b9c23:

  um: Remove some unnecessary NULL checks in vector_user.c (2020-03-29 23:56:47 +0200)

are available in the Git repository at:

  git://github.com/thehajime/linux 59ed4e23efa7abc9c7d52e6b5fb31fb8ce795c5c
  https://github.com/thehajime/linux/tree/upstream-to-uml-5.6-rc7-v4

Hajime Tazaki (6):
  lkl tools: host lib: filesystem helpers
  lkl tools: host lib: networking helpers
  um lkl: add CI scripts to conduct regression tests
  um lkl: add UML network driver for lkl
  um lkl: add UML block device driver (ubd) for lkl
  um: fix clone flags to be familiar with valgrind

Octavian Purdila (19):
  arch: add __SYSCALL_DEFINE_ARCH
  um lkl: architecture skeleton for Linux kernel library
  um lkl: host interface
  um lkl: memory handling
  um lkl: kernel threads support
  um lkl: interrupt support
  um lkl: system call interface and application API
  um lkl: timers, time and delay support
  um lkl: basic kernel console support
  um lkl: initialization and cleanup
  um lkl: plug in the build system
  lkl tools: skeleton for host side library
  lkl tools: host lib: add utilities functions
  lkl tools: host lib: posix host operations
  lkl tools: add test programs
  lkl tools: cptofs that reads/writes to/from a filesystem image
  lkl tools: fs2tar that converts a filesystem image to tar
  lkl tools: add lklfuse
  um lkl: add documentation

 .circleci/config.yml                       |  198 ++++
 Documentation/virt/uml/lkl.txt             |   64 ++
 MAINTAINERS                                |    8 +
 arch/um/Kconfig                            |   13 +
 arch/um/Makefile                           |   19 +-
 arch/um/Makefile.um                        |   10 +
 arch/um/drivers/Makefile                   |    2 +
 arch/um/drivers/ubd_kern.c                 |    6 +-
 arch/um/drivers/ubd_user.c                 |    3 +-
 arch/um/include/asm/irq.h                  |    2 +
 arch/um/kernel/Makefile                    |    4 +
 arch/um/kernel/irq.c                       |    4 +
 arch/um/lkl/.gitignore                     |    2 +
 arch/um/lkl/Kconfig                        |   88 ++
 arch/um/lkl/Kconfig.debug                  |    0
 arch/um/lkl/Makefile                       |   67 ++
 arch/um/lkl/auto.conf                      |    1 +
 arch/um/lkl/configs/lkl_defconfig          | 1052 ++++++++++++++++++++
 arch/um/lkl/include/asm/Kbuild             |   79 ++
 arch/um/lkl/include/asm/atomic.h           |   11 +
 arch/um/lkl/include/asm/atomic64.h         |  114 +++
 arch/um/lkl/include/asm/bitsperlong.h      |   11 +
 arch/um/lkl/include/asm/byteorder.h        |    7 +
 arch/um/lkl/include/asm/cpu.h              |   14 +
 arch/um/lkl/include/asm/elf.h              |   15 +
 arch/um/lkl/include/asm/host_ops.h         |   10 +
 arch/um/lkl/include/asm/irq.h              |   16 +
 arch/um/lkl/include/asm/page.h             |   14 +
 arch/um/lkl/include/asm/pgtable.h          |   57 ++
 arch/um/lkl/include/asm/processor.h        |   60 ++
 arch/um/lkl/include/asm/ptrace.h           |   25 +
 arch/um/lkl/include/asm/sched.h            |   23 +
 arch/um/lkl/include/asm/setup.h            |    7 +
 arch/um/lkl/include/asm/syscalls.h         |   18 +
 arch/um/lkl/include/asm/syscalls_32.h      |   43 +
 arch/um/lkl/include/asm/thread_info.h      |   70 ++
 arch/um/lkl/include/asm/tlb.h              |   12 +
 arch/um/lkl/include/asm/uaccess.h          |   64 ++
 arch/um/lkl/include/asm/unistd.h           |   27 +
 arch/um/lkl/include/asm/unistd_32.h        |   31 +
 arch/um/lkl/include/asm/vmalloc.h          |    5 +
 arch/um/lkl/include/asm/vmlinux.lds.h      |   14 +
 arch/um/lkl/include/asm/xor.h              |    9 +
 arch/um/lkl/include/uapi/asm/Kbuild        |   11 +
 arch/um/lkl/include/uapi/asm/bitsperlong.h |   13 +
 arch/um/lkl/include/uapi/asm/byteorder.h   |   11 +
 arch/um/lkl/include/uapi/asm/host_ops.h    |  143 +++
 arch/um/lkl/include/uapi/asm/irq.h         |   36 +
 arch/um/lkl/include/uapi/asm/sigcontext.h  |   16 +
 arch/um/lkl/include/uapi/asm/syscalls.h    |  349 +++++++
 arch/um/lkl/include/uapi/asm/unistd.h      |   13 +
 arch/um/lkl/kernel/Makefile                |    4 +
 arch/um/lkl/kernel/asm-offsets.c           |    3 +
 arch/um/lkl/kernel/console.c               |   42 +
 arch/um/lkl/kernel/cpu.c                   |  223 +++++
 arch/um/lkl/kernel/irq.c                   |  223 +++++
 arch/um/lkl/kernel/misc.c                  |   60 ++
 arch/um/lkl/kernel/setup.c                 |  198 ++++
 arch/um/lkl/kernel/syscalls.c              |  192 ++++
 arch/um/lkl/kernel/syscalls_32.c           |  159 +++
 arch/um/lkl/kernel/threads.c               |  227 +++++
 arch/um/lkl/kernel/time.c                  |  145 +++
 arch/um/lkl/kernel/vmlinux.lds.S           |   51 +
 arch/um/lkl/mm/Makefile                    |    1 +
 arch/um/lkl/mm/bootmem.c                   |   99 ++
 arch/um/lkl/scripts/headers_install.py     |  196 ++++
 arch/um/os-Linux/Makefile                  |    5 +
 include/linux/syscalls.h                   |    6 +
 tools/lkl/.gitignore                       |   13 +
 tools/lkl/Build                            |    6 +
 tools/lkl/Makefile                         |  131 +++
 tools/lkl/Makefile.autoconf                |   67 ++
 tools/lkl/Targets                          |   17 +
 tools/lkl/cptofs.c                         |  636 ++++++++++++
 tools/lkl/fs2tar.c                         |  412 ++++++++
 tools/lkl/include/.gitignore               |    1 +
 tools/lkl/include/lkl.h                    |  788 +++++++++++++++
 tools/lkl/include/lkl_config.h             |   61 ++
 tools/lkl/include/lkl_host.h               |  112 +++
 tools/lkl/lib/.gitignore                   |    3 +
 tools/lkl/lib/Build                        |   13 +
 tools/lkl/lib/config.c                     |  750 ++++++++++++++
 tools/lkl/lib/dbg.c                        |  300 ++++++
 tools/lkl/lib/dbg_handler.c                |   44 +
 tools/lkl/lib/endian.h                     |   31 +
 tools/lkl/lib/fs.c                         |  471 +++++++++
 tools/lkl/lib/jmp_buf.c                    |   14 +
 tools/lkl/lib/jmp_buf.h                    |    8 +
 tools/lkl/lib/net.c                        |  834 ++++++++++++++++
 tools/lkl/lib/posix-host.c                 |  356 +++++++
 tools/lkl/lib/um/Build                     |    3 +
 tools/lkl/lib/um/um_block.c                |   17 +
 tools/lkl/lib/um/um_glue.c                 |   61 ++
 tools/lkl/lib/um/um_net.c                  |   30 +
 tools/lkl/lib/utils.c                      |  266 +++++
 tools/lkl/lklfuse.c                        |  659 ++++++++++++
 tools/lkl/scripts/checkpatch.sh            |   60 ++
 tools/lkl/scripts/lkl-jenkins.sh           |   21 +
 tools/lkl/tests/Build                      |    3 +
 tools/lkl/tests/boot.c                     |  562 +++++++++++
 tools/lkl/tests/boot.sh                    |    9 +
 tools/lkl/tests/cla.c                      |  159 +++
 tools/lkl/tests/cla.h                      |   33 +
 tools/lkl/tests/disk.c                     |  203 ++++
 tools/lkl/tests/disk.sh                    |   61 ++
 tools/lkl/tests/lklfuse.sh                 |  110 ++
 tools/lkl/tests/net-setup.sh               |  136 +++
 tools/lkl/tests/net-test.c                 |  307 ++++++
 tools/lkl/tests/net.sh                     |  135 +++
 tools/lkl/tests/run.py                     |  181 ++++
 tools/lkl/tests/tap13.py                   |  209 ++++
 tools/lkl/tests/test.c                     |  126 +++
 tools/lkl/tests/test.h                     |   72 ++
 tools/lkl/tests/test.sh                    |  182 ++++
 tools/lkl/tests/valgrind.supp              |  105 ++
 tools/lkl/tests/valgrind2xunit.py          |   69 ++
 116 files changed, 13569 insertions(+), 3 deletions(-)
 create mode 100644 .circleci/config.yml
 create mode 100644 Documentation/virt/uml/lkl.txt
 create mode 100644 arch/um/Makefile.um
 create mode 100644 arch/um/lkl/.gitignore
 create mode 100644 arch/um/lkl/Kconfig
 create mode 100644 arch/um/lkl/Kconfig.debug
 create mode 100644 arch/um/lkl/Makefile
 create mode 100644 arch/um/lkl/auto.conf
 create mode 100644 arch/um/lkl/configs/lkl_defconfig
 create mode 100644 arch/um/lkl/include/asm/Kbuild
 create mode 100644 arch/um/lkl/include/asm/atomic.h
 create mode 100644 arch/um/lkl/include/asm/atomic64.h
 create mode 100644 arch/um/lkl/include/asm/bitsperlong.h
 create mode 100644 arch/um/lkl/include/asm/byteorder.h
 create mode 100644 arch/um/lkl/include/asm/cpu.h
 create mode 100644 arch/um/lkl/include/asm/elf.h
 create mode 100644 arch/um/lkl/include/asm/host_ops.h
 create mode 100644 arch/um/lkl/include/asm/irq.h
 create mode 100644 arch/um/lkl/include/asm/page.h
 create mode 100644 arch/um/lkl/include/asm/pgtable.h
 create mode 100644 arch/um/lkl/include/asm/processor.h
 create mode 100644 arch/um/lkl/include/asm/ptrace.h
 create mode 100644 arch/um/lkl/include/asm/sched.h
 create mode 100644 arch/um/lkl/include/asm/setup.h
 create mode 100644 arch/um/lkl/include/asm/syscalls.h
 create mode 100644 arch/um/lkl/include/asm/syscalls_32.h
 create mode 100644 arch/um/lkl/include/asm/thread_info.h
 create mode 100644 arch/um/lkl/include/asm/tlb.h
 create mode 100644 arch/um/lkl/include/asm/uaccess.h
 create mode 100644 arch/um/lkl/include/asm/unistd.h
 create mode 100644 arch/um/lkl/include/asm/unistd_32.h
 create mode 100644 arch/um/lkl/include/asm/vmalloc.h
 create mode 100644 arch/um/lkl/include/asm/vmlinux.lds.h
 create mode 100644 arch/um/lkl/include/asm/xor.h
 create mode 100644 arch/um/lkl/include/uapi/asm/Kbuild
 create mode 100644 arch/um/lkl/include/uapi/asm/bitsperlong.h
 create mode 100644 arch/um/lkl/include/uapi/asm/byteorder.h
 create mode 100644 arch/um/lkl/include/uapi/asm/host_ops.h
 create mode 100644 arch/um/lkl/include/uapi/asm/irq.h
 create mode 100644 arch/um/lkl/include/uapi/asm/sigcontext.h
 create mode 100644 arch/um/lkl/include/uapi/asm/syscalls.h
 create mode 100644 arch/um/lkl/include/uapi/asm/unistd.h
 create mode 100644 arch/um/lkl/kernel/Makefile
 create mode 100644 arch/um/lkl/kernel/asm-offsets.c
 create mode 100644 arch/um/lkl/kernel/console.c
 create mode 100644 arch/um/lkl/kernel/cpu.c
 create mode 100644 arch/um/lkl/kernel/irq.c
 create mode 100644 arch/um/lkl/kernel/misc.c
 create mode 100644 arch/um/lkl/kernel/setup.c
 create mode 100644 arch/um/lkl/kernel/syscalls.c
 create mode 100644 arch/um/lkl/kernel/syscalls_32.c
 create mode 100644 arch/um/lkl/kernel/threads.c
 create mode 100644 arch/um/lkl/kernel/time.c
 create mode 100644 arch/um/lkl/kernel/vmlinux.lds.S
 create mode 100644 arch/um/lkl/mm/Makefile
 create mode 100644 arch/um/lkl/mm/bootmem.c
 create mode 100755 arch/um/lkl/scripts/headers_install.py
 create mode 100644 tools/lkl/.gitignore
 create mode 100644 tools/lkl/Build
 create mode 100644 tools/lkl/Makefile
 create mode 100644 tools/lkl/Makefile.autoconf
 create mode 100644 tools/lkl/Targets
 create mode 100644 tools/lkl/cptofs.c
 create mode 100644 tools/lkl/fs2tar.c
 create mode 100644 tools/lkl/include/.gitignore
 create mode 100644 tools/lkl/include/lkl.h
 create mode 100644 tools/lkl/include/lkl_config.h
 create mode 100644 tools/lkl/include/lkl_host.h
 create mode 100644 tools/lkl/lib/.gitignore
 create mode 100644 tools/lkl/lib/Build
 create mode 100644 tools/lkl/lib/config.c
 create mode 100644 tools/lkl/lib/dbg.c
 create mode 100644 tools/lkl/lib/dbg_handler.c
 create mode 100644 tools/lkl/lib/endian.h
 create mode 100644 tools/lkl/lib/fs.c
 create mode 100644 tools/lkl/lib/jmp_buf.c
 create mode 100644 tools/lkl/lib/jmp_buf.h
 create mode 100644 tools/lkl/lib/net.c
 create mode 100644 tools/lkl/lib/posix-host.c
 create mode 100644 tools/lkl/lib/um/Build
 create mode 100644 tools/lkl/lib/um/um_block.c
 create mode 100644 tools/lkl/lib/um/um_glue.c
 create mode 100644 tools/lkl/lib/um/um_net.c
 create mode 100644 tools/lkl/lib/utils.c
 create mode 100644 tools/lkl/lklfuse.c
 create mode 100755 tools/lkl/scripts/checkpatch.sh
 create mode 100755 tools/lkl/scripts/lkl-jenkins.sh
 create mode 100644 tools/lkl/tests/Build
 create mode 100644 tools/lkl/tests/boot.c
 create mode 100755 tools/lkl/tests/boot.sh
 create mode 100644 tools/lkl/tests/cla.c
 create mode 100644 tools/lkl/tests/cla.h
 create mode 100644 tools/lkl/tests/disk.c
 create mode 100755 tools/lkl/tests/disk.sh
 create mode 100755 tools/lkl/tests/lklfuse.sh
 create mode 100644 tools/lkl/tests/net-setup.sh
 create mode 100644 tools/lkl/tests/net-test.c
 create mode 100755 tools/lkl/tests/net.sh
 create mode 100755 tools/lkl/tests/run.py
 create mode 100644 tools/lkl/tests/tap13.py
 create mode 100644 tools/lkl/tests/test.c
 create mode 100644 tools/lkl/tests/test.h
 create mode 100644 tools/lkl/tests/test.sh
 create mode 100644 tools/lkl/tests/valgrind.supp
 create mode 100755 tools/lkl/tests/valgrind2xunit.py

-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* [RFC v4 01/25] arch: add __SYSCALL_DEFINE_ARCH
  2020-03-30 14:45   ` Hajime Tazaki
@ 2020-03-30 14:45     ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	linux-api

From: Octavian Purdila <tavi.purdila@gmail.com>

This allows the architecture code to process the system call
definitions. It is used by LKL to create strong typed function
definitions for system calls.

Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
Cc: linux-api@vger.kernel.org
---
 include/linux/syscalls.h | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/include/linux/syscalls.h b/include/linux/syscalls.h
index 1815065d52f3..e45815a3ee10 100644
--- a/include/linux/syscalls.h
+++ b/include/linux/syscalls.h
@@ -203,9 +203,14 @@ static inline int is_syscall_trace_event(struct trace_event_call *tp_event)
 }
 #endif
 
+#ifndef __SYSCALL_DEFINE_ARCH
+#define __SYSCALL_DEFINE_ARCH(x, sname, ...)
+#endif
+
 #ifndef SYSCALL_DEFINE0
 #define SYSCALL_DEFINE0(sname)					\
 	SYSCALL_METADATA(_##sname, 0);				\
+	__SYSCALL_DEFINE_ARCH(0, _##sname);			\
 	asmlinkage long sys_##sname(void);			\
 	ALLOW_ERROR_INJECTION(sys_##sname, ERRNO);		\
 	asmlinkage long sys_##sname(void)
@@ -222,6 +227,7 @@ static inline int is_syscall_trace_event(struct trace_event_call *tp_event)
 
 #define SYSCALL_DEFINEx(x, sname, ...)				\
 	SYSCALL_METADATA(sname, x, __VA_ARGS__)			\
+	__SYSCALL_DEFINE_ARCH(x, sname, __VA_ARGS__)		\
 	__SYSCALL_DEFINEx(x, sname, __VA_ARGS__)
 
 #define __PROTECT(...) asmlinkage_protect(__VA_ARGS__)
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 01/25] arch: add __SYSCALL_DEFINE_ARCH
@ 2020-03-30 14:45     ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, linux-kernel-library, linux-arch, linux-api,
	Akira Moroo

From: Octavian Purdila <tavi.purdila@gmail.com>

This allows the architecture code to process the system call
definitions. It is used by LKL to create strong typed function
definitions for system calls.

Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
Cc: linux-api@vger.kernel.org
---
 include/linux/syscalls.h | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/include/linux/syscalls.h b/include/linux/syscalls.h
index 1815065d52f3..e45815a3ee10 100644
--- a/include/linux/syscalls.h
+++ b/include/linux/syscalls.h
@@ -203,9 +203,14 @@ static inline int is_syscall_trace_event(struct trace_event_call *tp_event)
 }
 #endif
 
+#ifndef __SYSCALL_DEFINE_ARCH
+#define __SYSCALL_DEFINE_ARCH(x, sname, ...)
+#endif
+
 #ifndef SYSCALL_DEFINE0
 #define SYSCALL_DEFINE0(sname)					\
 	SYSCALL_METADATA(_##sname, 0);				\
+	__SYSCALL_DEFINE_ARCH(0, _##sname);			\
 	asmlinkage long sys_##sname(void);			\
 	ALLOW_ERROR_INJECTION(sys_##sname, ERRNO);		\
 	asmlinkage long sys_##sname(void)
@@ -222,6 +227,7 @@ static inline int is_syscall_trace_event(struct trace_event_call *tp_event)
 
 #define SYSCALL_DEFINEx(x, sname, ...)				\
 	SYSCALL_METADATA(sname, x, __VA_ARGS__)			\
+	__SYSCALL_DEFINE_ARCH(x, sname, __VA_ARGS__)		\
 	__SYSCALL_DEFINEx(x, sname, __VA_ARGS__)
 
 #define __PROTECT(...) asmlinkage_protect(__VA_ARGS__)
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 02/25] um lkl: architecture skeleton for Linux kernel library
  2020-03-30 14:45   ` Hajime Tazaki
@ 2020-03-30 14:45     ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Conrad Meyer, Edison M . Castro, Jens Staal, Lai Jiangshan,
	Levente Kurusa, Luca Dariz, Mark Stillwell, Matthieu Coudron,
	Michael Zimmermann, Motomu Utsumi, Patrick Collins,
	Petros Angelatos, Pierre-Hugues Husson, Xiao Jia, Yuan Liu,
	Hajime Tazaki

From: Octavian Purdila <tavi.purdila@gmail.com>

Adds the LKL Kconfig, vmlinux linker script, basic architecture
headers and miscellaneous basic functions or stubs such as
dump_stack(), show_regs() and cpuinfo proc ops.

The headers we introduce in this patch are simple wrappers to the
asm-generic headers or stubs for things we don't support, such as
ptrace, DMA, signals, ELF handling and low level processor operations.

The kernel configuration is automatically updated to reflect the
endianness of the host, 64bit support or the output format for
vmlinux's linker script. We do this by looking at the ld's default
output format.

Cc: Andreas Abel <aabel@google.com>
Cc: Conrad Meyer <cem@FreeBSD.org>
Cc: Edison M. Castro <edisonmcastro@hotmail.com>
Cc: H.K. Jerry Chu <hkchu@google.com>
Cc: Jens Staal <staal1978@gmail.com>
Cc: Lai Jiangshan <jiangshanlai@gmail.com>
Cc: Levente Kurusa <levex@linux.com>
Cc: Luca Dariz <luca.dariz@gmail.com>
Cc: Mark Stillwell <mark@stillwell.me>
Cc: Matthieu Coudron <mattator@gmail.com>
Cc: Michael Zimmermann <sigmaepsilon92@gmail.com>
Cc: Motomu Utsumi <motomuman@gmail.com>
Cc: Patrick Collins <pscollins@google.com>
Cc: Petros Angelatos <petrosagg@gmail.com>
Cc: Pierre-Hugues Husson <phh@phh.me>
Cc: Xiao Jia <xiaoj@google.com>
Cc: Yuan Liu <liuyuan@google.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 MAINTAINERS                                |    8 +
 arch/um/lkl/.gitignore                     |    2 +
 arch/um/lkl/Kconfig                        |   56 ++
 arch/um/lkl/Kconfig.debug                  |    0
 arch/um/lkl/configs/lkl_defconfig          | 1052 ++++++++++++++++++++
 arch/um/lkl/include/asm/Kbuild             |   79 ++
 arch/um/lkl/include/asm/atomic.h           |   11 +
 arch/um/lkl/include/asm/atomic64.h         |  114 +++
 arch/um/lkl/include/asm/bitsperlong.h      |   11 +
 arch/um/lkl/include/asm/byteorder.h        |    7 +
 arch/um/lkl/include/asm/cpu.h              |   14 +
 arch/um/lkl/include/asm/elf.h              |   15 +
 arch/um/lkl/include/asm/processor.h        |   60 ++
 arch/um/lkl/include/asm/ptrace.h           |   25 +
 arch/um/lkl/include/asm/sched.h            |   23 +
 arch/um/lkl/include/asm/syscalls.h         |   18 +
 arch/um/lkl/include/asm/syscalls_32.h      |   43 +
 arch/um/lkl/include/asm/tlb.h              |   12 +
 arch/um/lkl/include/asm/uaccess.h          |   64 ++
 arch/um/lkl/include/asm/unistd_32.h        |   31 +
 arch/um/lkl/include/asm/vmalloc.h          |    5 +
 arch/um/lkl/include/asm/vmlinux.lds.h      |   14 +
 arch/um/lkl/include/asm/xor.h              |    9 +
 arch/um/lkl/include/uapi/asm/Kbuild        |   11 +
 arch/um/lkl/include/uapi/asm/bitsperlong.h |   13 +
 arch/um/lkl/include/uapi/asm/byteorder.h   |   11 +
 arch/um/lkl/include/uapi/asm/syscalls.h    |  348 +++++++
 arch/um/lkl/kernel/asm-offsets.c           |    2 +
 arch/um/lkl/kernel/misc.c                  |   60 ++
 arch/um/lkl/kernel/vmlinux.lds.S           |   51 +
 30 files changed, 2169 insertions(+)
 create mode 100644 arch/um/lkl/.gitignore
 create mode 100644 arch/um/lkl/Kconfig
 create mode 100644 arch/um/lkl/Kconfig.debug
 create mode 100644 arch/um/lkl/configs/lkl_defconfig
 create mode 100644 arch/um/lkl/include/asm/Kbuild
 create mode 100644 arch/um/lkl/include/asm/atomic.h
 create mode 100644 arch/um/lkl/include/asm/atomic64.h
 create mode 100644 arch/um/lkl/include/asm/bitsperlong.h
 create mode 100644 arch/um/lkl/include/asm/byteorder.h
 create mode 100644 arch/um/lkl/include/asm/cpu.h
 create mode 100644 arch/um/lkl/include/asm/elf.h
 create mode 100644 arch/um/lkl/include/asm/processor.h
 create mode 100644 arch/um/lkl/include/asm/ptrace.h
 create mode 100644 arch/um/lkl/include/asm/sched.h
 create mode 100644 arch/um/lkl/include/asm/syscalls.h
 create mode 100644 arch/um/lkl/include/asm/syscalls_32.h
 create mode 100644 arch/um/lkl/include/asm/tlb.h
 create mode 100644 arch/um/lkl/include/asm/uaccess.h
 create mode 100644 arch/um/lkl/include/asm/unistd_32.h
 create mode 100644 arch/um/lkl/include/asm/vmalloc.h
 create mode 100644 arch/um/lkl/include/asm/vmlinux.lds.h
 create mode 100644 arch/um/lkl/include/asm/xor.h
 create mode 100644 arch/um/lkl/include/uapi/asm/Kbuild
 create mode 100644 arch/um/lkl/include/uapi/asm/bitsperlong.h
 create mode 100644 arch/um/lkl/include/uapi/asm/byteorder.h
 create mode 100644 arch/um/lkl/include/uapi/asm/syscalls.h
 create mode 100644 arch/um/lkl/kernel/asm-offsets.c
 create mode 100644 arch/um/lkl/kernel/misc.c
 create mode 100644 arch/um/lkl/kernel/vmlinux.lds.S

diff --git a/MAINTAINERS b/MAINTAINERS
index cc1d18cb5d18..f3913964eabe 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -9710,6 +9710,14 @@ F:	Documentation/core-api/atomic_ops.rst
 F:	Documentation/core-api/refcount-vs-atomic.rst
 F:	Documentation/memory-barriers.txt
 
+LINUX KERNEL LIBRARY
+M:	Octavian Purdila <tavi.purdila@gmail.com>
+M:	Hajime Tazaki <thehajime@gmail.com>
+L:	linux-kernel-library@freelists.org
+S:	Maintained
+F:	arch/um/lkl/
+F:	tools/lkl/
+
 LIS3LV02D ACCELEROMETER DRIVER
 M:	Eric Piel <eric.piel@tremplin-utc.net>
 S:	Maintained
diff --git a/arch/um/lkl/.gitignore b/arch/um/lkl/.gitignore
new file mode 100644
index 000000000000..ced1c60d8235
--- /dev/null
+++ b/arch/um/lkl/.gitignore
@@ -0,0 +1,2 @@
+kernel/vmlinux.lds
+include/generated
diff --git a/arch/um/lkl/Kconfig b/arch/um/lkl/Kconfig
new file mode 100644
index 000000000000..f7e8d707a12b
--- /dev/null
+++ b/arch/um/lkl/Kconfig
@@ -0,0 +1,56 @@
+# SPDX-License-Identifier: GPL-2.0
+
+config UML_LKL
+       def_bool y
+       depends on !SMP && !MMU && !COREDUMP && !SECCOMP && !UPROBES && !COMPAT && !USER_RETURN_NOTIFIER
+       select ARCH_THREAD_STACK_ALLOCATOR
+       select RWSEM_GENERIC_SPINLOCK
+       select GENERIC_ATOMIC64 if !64BIT
+       select GENERIC_HWEIGHT
+       select FLATMEM
+       select FLAT_NODE_MEM_MAP
+       select GENERIC_CLOCKEVENTS
+       select GENERIC_CPU_DEVICES
+       select NO_HZ_IDLE
+       select NO_PREEMPT
+       select ARCH_WANT_FRAME_POINTERS
+       select HAS_DMA
+       select DMA_DIRECT_OPS
+       select PHYS_ADDR_T_64BIT if 64BIT
+       select 64BIT if "$(OUTPUT_FORMAT)" = "elf64-x86-64"
+       select 64BIT if "$(OUTPUT_FORMAT)" = "elf64-x86-64-freebsd"
+       select NET
+       select MULTIUSER
+
+config OUTPUT_FORMAT
+       string "Output format"
+       default "$(OUTPUT_FORMAT)"
+
+config ARCH_DMA_ADDR_T_64BIT
+       def_bool 64BIT
+
+config 64BIT
+       def_bool n
+
+config COREDUMP
+       def_bool n
+
+config BIG_ENDIAN
+       def_bool n
+
+config GENERIC_CSUM
+       def_bool y
+
+config GENERIC_HWEIGHT
+       def_bool y
+
+config NO_IOPORT_MAP
+       def_bool y
+
+config RWSEM_GENERIC_SPINLOCK
+	bool
+	default y
+
+config HZ
+        int
+        default 100
diff --git a/arch/um/lkl/Kconfig.debug b/arch/um/lkl/Kconfig.debug
new file mode 100644
index 000000000000..e69de29bb2d1
diff --git a/arch/um/lkl/configs/lkl_defconfig b/arch/um/lkl/configs/lkl_defconfig
new file mode 100644
index 000000000000..7050c233a17e
--- /dev/null
+++ b/arch/um/lkl/configs/lkl_defconfig
@@ -0,0 +1,1052 @@
+CONFIG_CC_IS_GCC=y
+CONFIG_GCC_VERSION=80301
+CONFIG_CLANG_VERSION=0
+CONFIG_CC_CAN_LINK=y
+CONFIG_CC_HAS_ASM_GOTO=y
+CONFIG_CC_HAS_ASM_INLINE=y
+CONFIG_CC_HAS_WARN_MAYBE_UNINITIALIZED=y
+CONFIG_IRQ_WORK=y
+CONFIG_BROKEN_ON_SMP=y
+CONFIG_INIT_ENV_ARG_LIMIT=32
+# CONFIG_COMPILE_TEST is not set
+CONFIG_LOCALVERSION=""
+# CONFIG_LOCALVERSION_AUTO is not set
+CONFIG_BUILD_SALT=""
+CONFIG_DEFAULT_HOSTNAME="(none)"
+# CONFIG_SYSVIPC is not set
+# CONFIG_POSIX_MQUEUE is not set
+# CONFIG_USELIB is not set
+# CONFIG_AUDIT is not set
+CONFIG_GENERIC_CLOCKEVENTS=y
+CONFIG_TICK_ONESHOT=y
+CONFIG_NO_HZ_COMMON=y
+# CONFIG_HZ_PERIODIC is not set
+CONFIG_NO_HZ_IDLE=y
+# CONFIG_NO_HZ is not set
+CONFIG_HIGH_RES_TIMERS=y
+CONFIG_PREEMPT_NONE=y
+# CONFIG_PREEMPT_VOLUNTARY is not set
+# CONFIG_PREEMPT is not set
+CONFIG_TICK_CPU_ACCOUNTING=y
+# CONFIG_BSD_PROCESS_ACCT is not set
+# CONFIG_TASKSTATS is not set
+# CONFIG_PSI is not set
+CONFIG_TINY_RCU=y
+# CONFIG_RCU_EXPERT is not set
+CONFIG_SRCU=y
+CONFIG_TINY_SRCU=y
+# CONFIG_IKCONFIG is not set
+# CONFIG_IKHEADERS is not set
+CONFIG_LOG_BUF_SHIFT=17
+CONFIG_PRINTK_SAFE_LOG_BUF_SHIFT=13
+CONFIG_CC_HAS_INT128=y
+# CONFIG_CGROUPS is not set
+# CONFIG_NAMESPACES is not set
+# CONFIG_CHECKPOINT_RESTORE is not set
+# CONFIG_SCHED_AUTOGROUP is not set
+# CONFIG_SYSFS_DEPRECATED is not set
+# CONFIG_RELAY is not set
+# CONFIG_BLK_DEV_INITRD is not set
+CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE=y
+# CONFIG_CC_OPTIMIZE_FOR_SIZE is not set
+CONFIG_SYSCTL=y
+CONFIG_BPF=y
+CONFIG_EXPERT=y
+CONFIG_MULTIUSER=y
+# CONFIG_SGETMASK_SYSCALL is not set
+# CONFIG_SYSFS_SYSCALL is not set
+CONFIG_FHANDLE=y
+CONFIG_POSIX_TIMERS=y
+# CONFIG_KALLSYMS_USE_DATA_SECTION is not set
+CONFIG_PRINTK=y
+CONFIG_BUG=y
+# CONFIG_BASE_FULL is not set
+# CONFIG_FUTEX is not set
+CONFIG_EPOLL=y
+# CONFIG_SIGNALFD is not set
+# CONFIG_TIMERFD is not set
+CONFIG_EVENTFD=y
+# CONFIG_AIO is not set
+CONFIG_IO_URING=y
+# CONFIG_ADVISE_SYSCALLS is not set
+CONFIG_MEMBARRIER=y
+CONFIG_KALLSYMS=y
+CONFIG_KALLSYMS_ALL=y
+CONFIG_KALLSYMS_BASE_RELATIVE=y
+# CONFIG_BPF_SYSCALL is not set
+CONFIG_EMBEDDED=y
+# CONFIG_PC104 is not set
+# CONFIG_VM_EVENT_COUNTERS is not set
+CONFIG_SLUB_DEBUG=y
+# CONFIG_COMPAT_BRK is not set
+# CONFIG_SLAB is not set
+CONFIG_SLUB=y
+# CONFIG_SLOB is not set
+CONFIG_SLAB_MERGE_DEFAULT=y
+# CONFIG_SLAB_FREELIST_RANDOM is not set
+# CONFIG_SLAB_FREELIST_HARDENED is not set
+# CONFIG_SHUFFLE_PAGE_ALLOCATOR is not set
+# CONFIG_MMAP_ALLOW_UNINITIALIZED is not set
+# CONFIG_PROFILING is not set
+CONFIG_LKL=y
+# CONFIG_COREDUMP is not set
+CONFIG_GENERIC_CSUM=y
+CONFIG_GENERIC_HWEIGHT=y
+CONFIG_NO_IOPORT_MAP=y
+CONFIG_RWSEM_GENERIC_SPINLOCK=y
+CONFIG_HZ=100
+# CONFIG_STDERR_CONSOLE is not set
+# CONFIG_SSL is not set
+# CONFIG_NULL_CHAN is not set
+# CONFIG_PORT_CHAN is not set
+# CONFIG_PTY_CHAN is not set
+# CONFIG_TTY_CHAN is not set
+# CONFIG_XTERM_CHAN is not set
+CONFIG_NOCONFIG_CHAN=y
+CONFIG_CON_ZERO_CHAN="fd:0,fd:1"
+CONFIG_CON_CHAN="xterm"
+CONFIG_SSL_CHAN="pty"
+# CONFIG_UML_SOUND is not set
+# CONFIG_SOUND is not set
+CONFIG_UML_NET=y
+# CONFIG_UML_NET_ETHERTAP is not set
+CONFIG_UML_NET_TUNTAP=y
+# CONFIG_UML_NET_SLIP is not set
+# CONFIG_UML_NET_DAEMON is not set
+# CONFIG_UML_NET_VECTOR is not set
+# CONFIG_UML_NET_VDE is not set
+# CONFIG_UML_NET_MCAST is not set
+# CONFIG_UML_NET_PCAP is not set
+CONFIG_UML_NET_SLIRP=y
+# CONFIG_VIRTIO_UML is not set
+CONFIG_ARCH_THREAD_STACK_ALLOCATOR=y
+CONFIG_CC_HAS_STACKPROTECTOR_NONE=y
+CONFIG_HAVE_VIRT_CPU_ACCOUNTING_GEN=y
+CONFIG_PGTABLE_LEVELS=2
+CONFIG_PLUGIN_HOSTCC=""
+CONFIG_BASE_SMALL=1
+# CONFIG_MODULES is not set
+CONFIG_BLOCK=y
+# CONFIG_BLK_DEV_BSG is not set
+# CONFIG_BLK_DEV_BSGLIB is not set
+# CONFIG_BLK_DEV_INTEGRITY is not set
+# CONFIG_BLK_DEV_ZONED is not set
+# CONFIG_BLK_CMDLINE_PARSER is not set
+# CONFIG_BLK_WBT is not set
+# CONFIG_BLK_SED_OPAL is not set
+# CONFIG_PARTITION_ADVANCED is not set
+CONFIG_MSDOS_PARTITION=y
+CONFIG_EFI_PARTITION=y
+CONFIG_MQ_IOSCHED_DEADLINE=y
+CONFIG_MQ_IOSCHED_KYBER=y
+# CONFIG_IOSCHED_BFQ is not set
+CONFIG_INLINE_SPIN_UNLOCK_IRQ=y
+CONFIG_INLINE_READ_UNLOCK=y
+CONFIG_INLINE_READ_UNLOCK_IRQ=y
+CONFIG_INLINE_WRITE_UNLOCK=y
+CONFIG_INLINE_WRITE_UNLOCK_IRQ=y
+CONFIG_BINFMT_SCRIPT=y
+# CONFIG_BINFMT_MISC is not set
+CONFIG_FLATMEM=y
+CONFIG_FLAT_NODE_MEM_MAP=y
+CONFIG_SPLIT_PTLOCK_CPUS=999999
+CONFIG_PHYS_ADDR_T_64BIT=y
+CONFIG_NOMMU_INITIAL_TRIM_EXCESS=1
+CONFIG_NEED_PER_CPU_KM=y
+# CONFIG_CLEANCACHE is not set
+# CONFIG_ZPOOL is not set
+# CONFIG_ZBUD is not set
+# CONFIG_PERCPU_STATS is not set
+# CONFIG_GUP_BENCHMARK is not set
+CONFIG_NET=y
+# CONFIG_PACKET is not set
+# CONFIG_UNIX is not set
+# CONFIG_TLS is not set
+# CONFIG_XFRM_USER is not set
+# CONFIG_NET_KEY is not set
+CONFIG_INET=y
+CONFIG_IP_MULTICAST=y
+CONFIG_IP_ADVANCED_ROUTER=y
+# CONFIG_IP_FIB_TRIE_STATS is not set
+CONFIG_IP_MULTIPLE_TABLES=y
+CONFIG_IP_ROUTE_MULTIPATH=y
+# CONFIG_IP_ROUTE_VERBOSE is not set
+CONFIG_IP_PNP=y
+CONFIG_IP_PNP_DHCP=y
+# CONFIG_IP_PNP_BOOTP is not set
+# CONFIG_IP_PNP_RARP is not set
+# CONFIG_NET_IPIP is not set
+# CONFIG_NET_IPGRE_DEMUX is not set
+CONFIG_NET_IP_TUNNEL=y
+# CONFIG_IP_MROUTE is not set
+# CONFIG_SYN_COOKIES is not set
+# CONFIG_NET_IPVTI is not set
+# CONFIG_NET_FOU is not set
+# CONFIG_NET_FOU_IP_TUNNELS is not set
+# CONFIG_INET_AH is not set
+# CONFIG_INET_ESP is not set
+# CONFIG_INET_IPCOMP is not set
+CONFIG_INET_TUNNEL=y
+CONFIG_INET_DIAG=y
+CONFIG_INET_TCP_DIAG=y
+# CONFIG_INET_UDP_DIAG is not set
+# CONFIG_INET_RAW_DIAG is not set
+# CONFIG_INET_DIAG_DESTROY is not set
+CONFIG_TCP_CONG_ADVANCED=y
+CONFIG_TCP_CONG_BIC=y
+CONFIG_TCP_CONG_CUBIC=y
+CONFIG_TCP_CONG_WESTWOOD=y
+CONFIG_TCP_CONG_HTCP=y
+# CONFIG_TCP_CONG_HSTCP is not set
+# CONFIG_TCP_CONG_HYBLA is not set
+# CONFIG_TCP_CONG_VEGAS is not set
+# CONFIG_TCP_CONG_NV is not set
+# CONFIG_TCP_CONG_SCALABLE is not set
+# CONFIG_TCP_CONG_LP is not set
+# CONFIG_TCP_CONG_VENO is not set
+# CONFIG_TCP_CONG_YEAH is not set
+# CONFIG_TCP_CONG_ILLINOIS is not set
+# CONFIG_TCP_CONG_DCTCP is not set
+# CONFIG_TCP_CONG_CDG is not set
+CONFIG_TCP_CONG_BBR=y
+# CONFIG_DEFAULT_BIC is not set
+CONFIG_DEFAULT_CUBIC=y
+# CONFIG_DEFAULT_HTCP is not set
+# CONFIG_DEFAULT_WESTWOOD is not set
+# CONFIG_DEFAULT_BBR is not set
+# CONFIG_DEFAULT_RENO is not set
+CONFIG_DEFAULT_TCP_CONG="cubic"
+# CONFIG_TCP_MD5SIG is not set
+CONFIG_IPV6=y
+# CONFIG_IPV6_ROUTER_PREF is not set
+# CONFIG_IPV6_OPTIMISTIC_DAD is not set
+# CONFIG_INET6_AH is not set
+# CONFIG_INET6_ESP is not set
+# CONFIG_INET6_IPCOMP is not set
+# CONFIG_IPV6_MIP6 is not set
+# CONFIG_IPV6_VTI is not set
+CONFIG_IPV6_SIT=y
+# CONFIG_IPV6_SIT_6RD is not set
+CONFIG_IPV6_NDISC_NODETYPE=y
+# CONFIG_IPV6_TUNNEL is not set
+CONFIG_IPV6_MULTIPLE_TABLES=y
+# CONFIG_IPV6_SUBTREES is not set
+# CONFIG_IPV6_MROUTE is not set
+# CONFIG_IPV6_SEG6_LWTUNNEL is not set
+# CONFIG_IPV6_SEG6_HMAC is not set
+# CONFIG_NETWORK_SECMARK is not set
+# CONFIG_NETWORK_PHY_TIMESTAMPING is not set
+# CONFIG_NETFILTER is not set
+# CONFIG_BPFILTER is not set
+# CONFIG_IP_DCCP is not set
+# CONFIG_IP_SCTP is not set
+# CONFIG_RDS is not set
+# CONFIG_TIPC is not set
+# CONFIG_ATM is not set
+# CONFIG_L2TP is not set
+# CONFIG_BRIDGE is not set
+CONFIG_HAVE_NET_DSA=y
+# CONFIG_NET_DSA is not set
+# CONFIG_VLAN_8021Q is not set
+# CONFIG_DECNET is not set
+# CONFIG_LLC2 is not set
+# CONFIG_ATALK is not set
+# CONFIG_X25 is not set
+# CONFIG_LAPB is not set
+# CONFIG_PHONET is not set
+# CONFIG_6LOWPAN is not set
+# CONFIG_IEEE802154 is not set
+CONFIG_NET_SCHED=y
+# CONFIG_NET_SCH_CBQ is not set
+# CONFIG_NET_SCH_HTB is not set
+# CONFIG_NET_SCH_HFSC is not set
+# CONFIG_NET_SCH_PRIO is not set
+# CONFIG_NET_SCH_MULTIQ is not set
+# CONFIG_NET_SCH_RED is not set
+# CONFIG_NET_SCH_SFB is not set
+# CONFIG_NET_SCH_SFQ is not set
+# CONFIG_NET_SCH_TEQL is not set
+# CONFIG_NET_SCH_TBF is not set
+# CONFIG_NET_SCH_CBS is not set
+# CONFIG_NET_SCH_ETF is not set
+# CONFIG_NET_SCH_TAPRIO is not set
+# CONFIG_NET_SCH_GRED is not set
+# CONFIG_NET_SCH_DSMARK is not set
+# CONFIG_NET_SCH_NETEM is not set
+# CONFIG_NET_SCH_DRR is not set
+# CONFIG_NET_SCH_MQPRIO is not set
+# CONFIG_NET_SCH_SKBPRIO is not set
+# CONFIG_NET_SCH_CHOKE is not set
+# CONFIG_NET_SCH_QFQ is not set
+# CONFIG_NET_SCH_CODEL is not set
+# CONFIG_NET_SCH_FQ_CODEL is not set
+# CONFIG_NET_SCH_CAKE is not set
+CONFIG_NET_SCH_FQ=y
+# CONFIG_NET_SCH_HHF is not set
+# CONFIG_NET_SCH_PIE is not set
+# CONFIG_NET_SCH_PLUG is not set
+# CONFIG_NET_SCH_DEFAULT is not set
+# CONFIG_NET_CLS_BASIC is not set
+# CONFIG_NET_CLS_TCINDEX is not set
+# CONFIG_NET_CLS_ROUTE4 is not set
+# CONFIG_NET_CLS_FW is not set
+# CONFIG_NET_CLS_U32 is not set
+# CONFIG_NET_CLS_RSVP is not set
+# CONFIG_NET_CLS_RSVP6 is not set
+# CONFIG_NET_CLS_FLOW is not set
+# CONFIG_NET_CLS_BPF is not set
+# CONFIG_NET_CLS_FLOWER is not set
+# CONFIG_NET_CLS_MATCHALL is not set
+# CONFIG_NET_EMATCH is not set
+# CONFIG_NET_CLS_ACT is not set
+CONFIG_NET_SCH_FIFO=y
+# CONFIG_DCB is not set
+# CONFIG_BATMAN_ADV is not set
+# CONFIG_OPENVSWITCH is not set
+# CONFIG_VSOCKETS is not set
+# CONFIG_NETLINK_DIAG is not set
+# CONFIG_MPLS is not set
+# CONFIG_NET_NSH is not set
+# CONFIG_HSR is not set
+# CONFIG_NET_SWITCHDEV is not set
+# CONFIG_NET_L3_MASTER_DEV is not set
+# CONFIG_NET_NCSI is not set
+CONFIG_NET_RX_BUSY_POLL=y
+CONFIG_BQL=y
+# CONFIG_NET_PKTGEN is not set
+# CONFIG_HAMRADIO is not set
+# CONFIG_CAN is not set
+# CONFIG_BT is not set
+# CONFIG_AF_RXRPC is not set
+# CONFIG_AF_KCM is not set
+CONFIG_FIB_RULES=y
+# CONFIG_WIRELESS is not set
+# CONFIG_WIMAX is not set
+# CONFIG_RFKILL is not set
+# CONFIG_NET_9P is not set
+# CONFIG_CAIF is not set
+# CONFIG_CEPH_LIB is not set
+# CONFIG_NFC is not set
+# CONFIG_PSAMPLE is not set
+# CONFIG_NET_IFE is not set
+# CONFIG_LWTUNNEL is not set
+CONFIG_DST_CACHE=y
+CONFIG_GRO_CELLS=y
+CONFIG_FAILOVER=y
+# CONFIG_PCCARD is not set
+# CONFIG_UEVENT_HELPER is not set
+# CONFIG_DEVTMPFS is not set
+CONFIG_STANDALONE=y
+CONFIG_PREVENT_FIRMWARE_BUILD=y
+# CONFIG_FW_LOADER is not set
+CONFIG_ALLOW_DEV_COREDUMP=y
+# CONFIG_DEBUG_DRIVER is not set
+# CONFIG_DEBUG_DEVRES is not set
+# CONFIG_DEBUG_TEST_DRIVER_REMOVE is not set
+CONFIG_GENERIC_CPU_DEVICES=y
+# CONFIG_CONNECTOR is not set
+# CONFIG_GNSS is not set
+# CONFIG_MTD is not set
+# CONFIG_OF is not set
+# CONFIG_PARPORT is not set
+CONFIG_BLK_DEV=y
+# CONFIG_BLK_DEV_NULL_BLK is not set
+# CONFIG_BLK_DEV_UBD_SYNC is not set
+CONFIG_BLK_DEV_COW_COMMON=y
+# CONFIG_BLK_DEV_LOOP is not set
+# CONFIG_BLK_DEV_DRBD is not set
+# CONFIG_BLK_DEV_NBD is not set
+# CONFIG_BLK_DEV_RAM is not set
+# CONFIG_CDROM_PKTCDVD is not set
+# CONFIG_ATA_OVER_ETH is not set
+# CONFIG_BLK_DEV_RBD is not set
+# CONFIG_NVME_FC is not set
+# CONFIG_DUMMY_IRQ is not set
+# CONFIG_ENCLOSURE_SERVICES is not set
+# CONFIG_SRAM is not set
+# CONFIG_XILINX_SDFEC is not set
+# CONFIG_C2PORT is not set
+# CONFIG_EEPROM_93CX6 is not set
+# CONFIG_VOP_BUS is not set
+# CONFIG_ECHO is not set
+CONFIG_SCSI_MOD=y
+# CONFIG_RAID_ATTRS is not set
+# CONFIG_SCSI is not set
+# CONFIG_ATA is not set
+# CONFIG_MD is not set
+# CONFIG_TARGET_CORE is not set
+CONFIG_NETDEVICES=y
+CONFIG_NET_CORE=y
+# CONFIG_BONDING is not set
+# CONFIG_DUMMY is not set
+# CONFIG_EQUALIZER is not set
+# CONFIG_NET_TEAM is not set
+# CONFIG_MACVLAN is not set
+# CONFIG_IPVLAN is not set
+# CONFIG_VXLAN is not set
+# CONFIG_GENEVE is not set
+# CONFIG_GTP is not set
+# CONFIG_MACSEC is not set
+# CONFIG_NETCONSOLE is not set
+# CONFIG_TUN is not set
+# CONFIG_TUN_VNET_CROSS_LE is not set
+# CONFIG_VETH is not set
+# CONFIG_NLMON is not set
+# CONFIG_ETHERNET is not set
+# CONFIG_MDIO_DEVICE is not set
+# CONFIG_PHYLIB is not set
+# CONFIG_PPP is not set
+# CONFIG_SLIP is not set
+# CONFIG_WLAN is not set
+# CONFIG_WAN is not set
+CONFIG_NET_FAILOVER=y
+# CONFIG_ISDN is not set
+# CONFIG_NVM is not set
+CONFIG_INPUT=y
+# CONFIG_INPUT_FF_MEMLESS is not set
+# CONFIG_INPUT_POLLDEV is not set
+# CONFIG_INPUT_SPARSEKMAP is not set
+# CONFIG_INPUT_MATRIXKMAP is not set
+# CONFIG_INPUT_MOUSEDEV is not set
+# CONFIG_INPUT_JOYDEV is not set
+# CONFIG_INPUT_EVDEV is not set
+# CONFIG_INPUT_EVBUG is not set
+CONFIG_INPUT_KEYBOARD=y
+CONFIG_KEYBOARD_ATKBD=y
+# CONFIG_KEYBOARD_LKKBD is not set
+# CONFIG_KEYBOARD_NEWTON is not set
+# CONFIG_KEYBOARD_OPENCORES is not set
+# CONFIG_KEYBOARD_STOWAWAY is not set
+# CONFIG_KEYBOARD_SUNKBD is not set
+# CONFIG_KEYBOARD_XTKBD is not set
+CONFIG_INPUT_MOUSE=y
+CONFIG_MOUSE_PS2=y
+CONFIG_MOUSE_PS2_ALPS=y
+CONFIG_MOUSE_PS2_BYD=y
+CONFIG_MOUSE_PS2_LOGIPS2PP=y
+CONFIG_MOUSE_PS2_SYNAPTICS=y
+CONFIG_MOUSE_PS2_CYPRESS=y
+CONFIG_MOUSE_PS2_TRACKPOINT=y
+# CONFIG_MOUSE_PS2_ELANTECH is not set
+# CONFIG_MOUSE_PS2_SENTELIC is not set
+# CONFIG_MOUSE_PS2_TOUCHKIT is not set
+CONFIG_MOUSE_PS2_FOCALTECH=y
+# CONFIG_MOUSE_SERIAL is not set
+# CONFIG_MOUSE_APPLETOUCH is not set
+# CONFIG_MOUSE_BCM5974 is not set
+# CONFIG_MOUSE_VSXXXAA is not set
+# CONFIG_MOUSE_SYNAPTICS_USB is not set
+# CONFIG_INPUT_JOYSTICK is not set
+# CONFIG_INPUT_TABLET is not set
+# CONFIG_INPUT_TOUCHSCREEN is not set
+# CONFIG_INPUT_MISC is not set
+# CONFIG_RMI4_CORE is not set
+CONFIG_SERIO=y
+CONFIG_SERIO_SERPORT=y
+CONFIG_SERIO_LIBPS2=y
+# CONFIG_SERIO_RAW is not set
+# CONFIG_SERIO_ALTERA_PS2 is not set
+# CONFIG_SERIO_PS2MULT is not set
+# CONFIG_SERIO_ARC_PS2 is not set
+# CONFIG_USERIO is not set
+# CONFIG_GAMEPORT is not set
+CONFIG_TTY=y
+# CONFIG_VT is not set
+CONFIG_UNIX98_PTYS=y
+CONFIG_LEGACY_PTYS=y
+CONFIG_LEGACY_PTY_COUNT=256
+# CONFIG_SERIAL_NONSTANDARD is not set
+# CONFIG_N_GSM is not set
+# CONFIG_TRACE_SINK is not set
+# CONFIG_NULL_TTY is not set
+CONFIG_LDISC_AUTOLOAD=y
+CONFIG_DEVMEM=y
+# CONFIG_DEVKMEM is not set
+# CONFIG_SERIAL_8250 is not set
+# CONFIG_SERIAL_UARTLITE is not set
+# CONFIG_SERIAL_SCCNXP is not set
+# CONFIG_SERIAL_ALTERA_JTAGUART is not set
+# CONFIG_SERIAL_ALTERA_UART is not set
+# CONFIG_SERIAL_ARC is not set
+# CONFIG_SERIAL_FSL_LPUART is not set
+# CONFIG_SERIAL_FSL_LINFLEXUART is not set
+# CONFIG_SERIAL_DEV_BUS is not set
+# CONFIG_TTY_PRINTK is not set
+# CONFIG_IPMI_HANDLER is not set
+CONFIG_HW_RANDOM=y
+# CONFIG_HW_RANDOM_TIMERIOMEM is not set
+# CONFIG_RAW_DRIVER is not set
+# CONFIG_TCG_TPM is not set
+# CONFIG_RANDOM_TRUST_BOOTLOADER is not set
+# CONFIG_I2C is not set
+# CONFIG_I3C is not set
+# CONFIG_SPI is not set
+# CONFIG_SPMI is not set
+# CONFIG_HSI is not set
+# CONFIG_PPS is not set
+# CONFIG_PTP_1588_CLOCK is not set
+# CONFIG_PINCTRL is not set
+# CONFIG_GPIOLIB is not set
+# CONFIG_W1 is not set
+# CONFIG_POWER_AVS is not set
+# CONFIG_POWER_RESET is not set
+# CONFIG_POWER_SUPPLY is not set
+CONFIG_HWMON=y
+# CONFIG_HWMON_DEBUG_CHIP is not set
+# CONFIG_SENSORS_AS370 is not set
+# CONFIG_SENSORS_ASPEED is not set
+# CONFIG_SENSORS_F71805F is not set
+# CONFIG_SENSORS_F71882FG is not set
+# CONFIG_SENSORS_IT87 is not set
+# CONFIG_SENSORS_MAX197 is not set
+# CONFIG_SENSORS_PC87360 is not set
+# CONFIG_SENSORS_PC87427 is not set
+# CONFIG_SENSORS_NTC_THERMISTOR is not set
+# CONFIG_SENSORS_NCT6683 is not set
+# CONFIG_SENSORS_NCT6775 is not set
+# CONFIG_SENSORS_NPCM7XX is not set
+# CONFIG_SENSORS_SMSC47M1 is not set
+# CONFIG_SENSORS_SMSC47B397 is not set
+# CONFIG_SENSORS_VT1211 is not set
+# CONFIG_SENSORS_W83627HF is not set
+# CONFIG_SENSORS_W83627EHF is not set
+# CONFIG_THERMAL is not set
+# CONFIG_WATCHDOG is not set
+CONFIG_SSB_POSSIBLE=y
+# CONFIG_SSB is not set
+CONFIG_BCMA_POSSIBLE=y
+# CONFIG_BCMA is not set
+# CONFIG_MFD_MADERA is not set
+# CONFIG_HTC_PASIC3 is not set
+# CONFIG_MFD_KEMPLD is not set
+# CONFIG_MFD_MT6397 is not set
+# CONFIG_MFD_SM501 is not set
+# CONFIG_ABX500_CORE is not set
+# CONFIG_MFD_SYSCON is not set
+# CONFIG_MFD_TI_AM335X_TSCADC is not set
+# CONFIG_MFD_TQMX86 is not set
+# CONFIG_REGULATOR is not set
+# CONFIG_RC_CORE is not set
+# CONFIG_MEDIA_SUPPORT is not set
+# CONFIG_DRM is not set
+# CONFIG_DRM_DP_CEC is not set
+# CONFIG_FB is not set
+# CONFIG_LCD_CLASS_DEVICE is not set
+# CONFIG_BACKLIGHT_CLASS_DEVICE is not set
+CONFIG_HID=y
+# CONFIG_HID_BATTERY_STRENGTH is not set
+# CONFIG_HIDRAW is not set
+# CONFIG_UHID is not set
+CONFIG_HID_GENERIC=y
+# CONFIG_HID_A4TECH is not set
+# CONFIG_HID_ACRUX is not set
+# CONFIG_HID_APPLE is not set
+# CONFIG_HID_AUREAL is not set
+# CONFIG_HID_BELKIN is not set
+# CONFIG_HID_CHERRY is not set
+# CONFIG_HID_CHICONY is not set
+# CONFIG_HID_COUGAR is not set
+# CONFIG_HID_MACALLY is not set
+# CONFIG_HID_CMEDIA is not set
+# CONFIG_HID_CYPRESS is not set
+# CONFIG_HID_DRAGONRISE is not set
+# CONFIG_HID_EMS_FF is not set
+# CONFIG_HID_ELECOM is not set
+# CONFIG_HID_EZKEY is not set
+# CONFIG_HID_GEMBIRD is not set
+# CONFIG_HID_GFRM is not set
+# CONFIG_HID_KEYTOUCH is not set
+# CONFIG_HID_KYE is not set
+# CONFIG_HID_WALTOP is not set
+# CONFIG_HID_VIEWSONIC is not set
+# CONFIG_HID_GYRATION is not set
+# CONFIG_HID_ICADE is not set
+# CONFIG_HID_ITE is not set
+# CONFIG_HID_JABRA is not set
+# CONFIG_HID_TWINHAN is not set
+# CONFIG_HID_KENSINGTON is not set
+# CONFIG_HID_LCPOWER is not set
+# CONFIG_HID_LENOVO is not set
+# CONFIG_HID_MAGICMOUSE is not set
+# CONFIG_HID_MALTRON is not set
+# CONFIG_HID_MAYFLASH is not set
+# CONFIG_HID_REDRAGON is not set
+# CONFIG_HID_MICROSOFT is not set
+# CONFIG_HID_MONTEREY is not set
+# CONFIG_HID_MULTITOUCH is not set
+# CONFIG_HID_NTI is not set
+# CONFIG_HID_ORTEK is not set
+# CONFIG_HID_PANTHERLORD is not set
+# CONFIG_HID_PETALYNX is not set
+# CONFIG_HID_PICOLCD is not set
+# CONFIG_HID_PLANTRONICS is not set
+# CONFIG_HID_PRIMAX is not set
+# CONFIG_HID_SAITEK is not set
+# CONFIG_HID_SAMSUNG is not set
+# CONFIG_HID_SPEEDLINK is not set
+# CONFIG_HID_STEAM is not set
+# CONFIG_HID_STEELSERIES is not set
+# CONFIG_HID_SUNPLUS is not set
+# CONFIG_HID_RMI is not set
+# CONFIG_HID_GREENASIA is not set
+# CONFIG_HID_SMARTJOYPLUS is not set
+# CONFIG_HID_TIVO is not set
+# CONFIG_HID_TOPSEED is not set
+# CONFIG_HID_THRUSTMASTER is not set
+# CONFIG_HID_UDRAW_PS3 is not set
+# CONFIG_HID_XINMO is not set
+# CONFIG_HID_ZEROPLUS is not set
+# CONFIG_HID_ZYDACRON is not set
+# CONFIG_HID_SENSOR_HUB is not set
+# CONFIG_HID_ALPS is not set
+CONFIG_USB_OHCI_LITTLE_ENDIAN=y
+CONFIG_USB_SUPPORT=y
+# CONFIG_USB_ULPI_BUS is not set
+CONFIG_USB_ARCH_HAS_HCD=y
+# CONFIG_USB is not set
+# CONFIG_NOP_USB_XCEIV is not set
+# CONFIG_USB_GADGET is not set
+# CONFIG_TYPEC is not set
+# CONFIG_USB_ROLE_SWITCH is not set
+# CONFIG_MMC is not set
+# CONFIG_MEMSTICK is not set
+# CONFIG_NEW_LEDS is not set
+# CONFIG_ACCESSIBILITY is not set
+# CONFIG_INFINIBAND is not set
+# CONFIG_RTC_CLASS is not set
+# CONFIG_DMADEVICES is not set
+# CONFIG_SYNC_FILE is not set
+# CONFIG_AUXDISPLAY is not set
+# CONFIG_VIRT_DRIVERS is not set
+CONFIG_VIRTIO_MENU=y
+# CONFIG_VIRTIO_MMIO is not set
+# CONFIG_GREYBUS is not set
+# CONFIG_STAGING is not set
+# CONFIG_GOLDFISH is not set
+# CONFIG_HWSPINLOCK is not set
+# CONFIG_MAILBOX is not set
+# CONFIG_REMOTEPROC is not set
+# CONFIG_RPMSG_VIRTIO is not set
+# CONFIG_SOC_TI is not set
+# CONFIG_XILINX_VCU is not set
+# CONFIG_PM_DEVFREQ is not set
+# CONFIG_EXTCON is not set
+# CONFIG_MEMORY is not set
+# CONFIG_IIO is not set
+# CONFIG_PWM is not set
+# CONFIG_IPACK_BUS is not set
+# CONFIG_RESET_CONTROLLER is not set
+# CONFIG_GENERIC_PHY is not set
+# CONFIG_BCM_KONA_USB2_PHY is not set
+# CONFIG_PHY_PXA_28NM_HSIC is not set
+# CONFIG_PHY_PXA_28NM_USB2 is not set
+# CONFIG_POWERCAP is not set
+# CONFIG_MCB is not set
+# CONFIG_RAS is not set
+# CONFIG_ANDROID is not set
+# CONFIG_LIBNVDIMM is not set
+# CONFIG_DAX is not set
+# CONFIG_NVMEM is not set
+# CONFIG_STM is not set
+# CONFIG_INTEL_TH is not set
+# CONFIG_FPGA is not set
+# CONFIG_SIOX is not set
+# CONFIG_SLIMBUS is not set
+# CONFIG_INTERCONNECT is not set
+# CONFIG_COUNTER is not set
+# CONFIG_VALIDATE_FS_PARSER is not set
+CONFIG_FS_IOMAP=y
+# CONFIG_EXT2_FS is not set
+# CONFIG_EXT3_FS is not set
+CONFIG_EXT4_FS=y
+CONFIG_EXT4_USE_FOR_EXT2=y
+CONFIG_EXT4_FS_POSIX_ACL=y
+CONFIG_EXT4_FS_SECURITY=y
+# CONFIG_EXT4_DEBUG is not set
+CONFIG_JBD2=y
+# CONFIG_JBD2_DEBUG is not set
+CONFIG_FS_MBCACHE=y
+# CONFIG_REISERFS_FS is not set
+# CONFIG_JFS_FS is not set
+# CONFIG_XFS_FS is not set
+# CONFIG_GFS2_FS is not set
+CONFIG_BTRFS_FS=y
+# CONFIG_BTRFS_FS_POSIX_ACL is not set
+# CONFIG_BTRFS_FS_CHECK_INTEGRITY is not set
+# CONFIG_BTRFS_FS_RUN_SANITY_TESTS is not set
+# CONFIG_BTRFS_DEBUG is not set
+# CONFIG_BTRFS_ASSERT is not set
+# CONFIG_BTRFS_FS_REF_VERIFY is not set
+# CONFIG_NILFS2_FS is not set
+# CONFIG_F2FS_FS is not set
+CONFIG_FS_POSIX_ACL=y
+CONFIG_EXPORTFS=y
+# CONFIG_EXPORTFS_BLOCK_OPS is not set
+# CONFIG_FILE_LOCKING is not set
+# CONFIG_FS_ENCRYPTION is not set
+# CONFIG_FS_VERITY is not set
+# CONFIG_DNOTIFY is not set
+# CONFIG_INOTIFY_USER is not set
+# CONFIG_FANOTIFY is not set
+# CONFIG_QUOTA is not set
+# CONFIG_AUTOFS4_FS is not set
+# CONFIG_AUTOFS_FS is not set
+# CONFIG_FUSE_FS is not set
+# CONFIG_OVERLAY_FS is not set
+# CONFIG_FSCACHE is not set
+# CONFIG_ISO9660_FS is not set
+# CONFIG_UDF_FS is not set
+CONFIG_FAT_FS=y
+# CONFIG_MSDOS_FS is not set
+CONFIG_VFAT_FS=y
+CONFIG_FAT_DEFAULT_CODEPAGE=437
+CONFIG_FAT_DEFAULT_IOCHARSET="iso8859-1"
+# CONFIG_FAT_DEFAULT_UTF8 is not set
+# CONFIG_NTFS_FS is not set
+CONFIG_PROC_FS=y
+CONFIG_PROC_SYSCTL=y
+# CONFIG_PROC_CHILDREN is not set
+CONFIG_KERNFS=y
+CONFIG_SYSFS=y
+# CONFIG_CONFIGFS_FS is not set
+CONFIG_MISC_FILESYSTEMS=y
+# CONFIG_ORANGEFS_FS is not set
+# CONFIG_ADFS_FS is not set
+# CONFIG_AFFS_FS is not set
+# CONFIG_HFS_FS is not set
+# CONFIG_HFSPLUS_FS is not set
+# CONFIG_BEFS_FS is not set
+# CONFIG_BFS_FS is not set
+# CONFIG_EFS_FS is not set
+# CONFIG_CRAMFS is not set
+# CONFIG_SQUASHFS is not set
+# CONFIG_VXFS_FS is not set
+# CONFIG_MINIX_FS is not set
+# CONFIG_OMFS_FS is not set
+# CONFIG_HPFS_FS is not set
+# CONFIG_QNX4FS_FS is not set
+# CONFIG_QNX6FS_FS is not set
+# CONFIG_ROMFS_FS is not set
+# CONFIG_PSTORE is not set
+# CONFIG_SYSV_FS is not set
+# CONFIG_UFS_FS is not set
+# CONFIG_EROFS_FS is not set
+CONFIG_NETWORK_FILESYSTEMS=y
+# CONFIG_CEPH_FS is not set
+# CONFIG_CIFS is not set
+# CONFIG_CODA_FS is not set
+# CONFIG_AFS_FS is not set
+CONFIG_NLS=y
+CONFIG_NLS_DEFAULT="iso8859-1"
+CONFIG_NLS_CODEPAGE_437=y
+CONFIG_NLS_CODEPAGE_737=y
+CONFIG_NLS_CODEPAGE_775=y
+CONFIG_NLS_CODEPAGE_850=y
+CONFIG_NLS_CODEPAGE_852=y
+CONFIG_NLS_CODEPAGE_855=y
+CONFIG_NLS_CODEPAGE_857=y
+CONFIG_NLS_CODEPAGE_860=y
+CONFIG_NLS_CODEPAGE_861=y
+CONFIG_NLS_CODEPAGE_862=y
+CONFIG_NLS_CODEPAGE_863=y
+CONFIG_NLS_CODEPAGE_864=y
+CONFIG_NLS_CODEPAGE_865=y
+CONFIG_NLS_CODEPAGE_866=y
+CONFIG_NLS_CODEPAGE_869=y
+CONFIG_NLS_CODEPAGE_936=y
+CONFIG_NLS_CODEPAGE_950=y
+CONFIG_NLS_CODEPAGE_932=y
+CONFIG_NLS_CODEPAGE_949=y
+CONFIG_NLS_CODEPAGE_874=y
+CONFIG_NLS_ISO8859_8=y
+CONFIG_NLS_CODEPAGE_1250=y
+CONFIG_NLS_CODEPAGE_1251=y
+CONFIG_NLS_ASCII=y
+CONFIG_NLS_ISO8859_1=y
+CONFIG_NLS_ISO8859_2=y
+CONFIG_NLS_ISO8859_3=y
+CONFIG_NLS_ISO8859_4=y
+CONFIG_NLS_ISO8859_5=y
+CONFIG_NLS_ISO8859_6=y
+CONFIG_NLS_ISO8859_7=y
+CONFIG_NLS_ISO8859_9=y
+CONFIG_NLS_ISO8859_13=y
+CONFIG_NLS_ISO8859_14=y
+CONFIG_NLS_ISO8859_15=y
+CONFIG_NLS_KOI8_R=y
+CONFIG_NLS_KOI8_U=y
+CONFIG_NLS_MAC_ROMAN=y
+CONFIG_NLS_MAC_CELTIC=y
+CONFIG_NLS_MAC_CENTEURO=y
+CONFIG_NLS_MAC_CROATIAN=y
+CONFIG_NLS_MAC_CYRILLIC=y
+CONFIG_NLS_MAC_GAELIC=y
+CONFIG_NLS_MAC_GREEK=y
+CONFIG_NLS_MAC_ICELAND=y
+CONFIG_NLS_MAC_INUIT=y
+CONFIG_NLS_MAC_ROMANIAN=y
+CONFIG_NLS_MAC_TURKISH=y
+CONFIG_NLS_UTF8=y
+# CONFIG_UNICODE is not set
+CONFIG_IO_WQ=y
+# CONFIG_KEYS is not set
+# CONFIG_SECURITY_DMESG_RESTRICT is not set
+# CONFIG_SECURITY is not set
+# CONFIG_SECURITYFS is not set
+CONFIG_HAVE_HARDENED_USERCOPY_ALLOCATOR=y
+# CONFIG_HARDENED_USERCOPY is not set
+# CONFIG_STATIC_USERMODEHELPER is not set
+CONFIG_DEFAULT_SECURITY_DAC=y
+CONFIG_LSM="lockdown,yama,loadpin,safesetid,integrity"
+CONFIG_INIT_STACK_NONE=y
+# CONFIG_INIT_ON_ALLOC_DEFAULT_ON is not set
+# CONFIG_INIT_ON_FREE_DEFAULT_ON is not set
+CONFIG_XOR_BLOCKS=y
+CONFIG_CRYPTO=y
+# CONFIG_CRYPTO_FIPS is not set
+CONFIG_CRYPTO_ALGAPI=y
+CONFIG_CRYPTO_ALGAPI2=y
+CONFIG_CRYPTO_HASH=y
+CONFIG_CRYPTO_HASH2=y
+CONFIG_CRYPTO_RNG=y
+CONFIG_CRYPTO_RNG2=y
+# CONFIG_CRYPTO_MANAGER is not set
+# CONFIG_CRYPTO_USER is not set
+# CONFIG_CRYPTO_NULL is not set
+# CONFIG_CRYPTO_CRYPTD is not set
+# CONFIG_CRYPTO_AUTHENC is not set
+# CONFIG_CRYPTO_RSA is not set
+# CONFIG_CRYPTO_DH is not set
+# CONFIG_CRYPTO_ECDH is not set
+# CONFIG_CRYPTO_ECRDSA is not set
+# CONFIG_CRYPTO_CURVE25519 is not set
+# CONFIG_CRYPTO_CCM is not set
+# CONFIG_CRYPTO_GCM is not set
+# CONFIG_CRYPTO_CHACHA20POLY1305 is not set
+# CONFIG_CRYPTO_AEGIS128 is not set
+# CONFIG_CRYPTO_SEQIV is not set
+# CONFIG_CRYPTO_ECHAINIV is not set
+# CONFIG_CRYPTO_CBC is not set
+# CONFIG_CRYPTO_CFB is not set
+# CONFIG_CRYPTO_CTR is not set
+# CONFIG_CRYPTO_CTS is not set
+# CONFIG_CRYPTO_ECB is not set
+# CONFIG_CRYPTO_LRW is not set
+# CONFIG_CRYPTO_OFB is not set
+# CONFIG_CRYPTO_PCBC is not set
+# CONFIG_CRYPTO_XTS is not set
+# CONFIG_CRYPTO_KEYWRAP is not set
+# CONFIG_CRYPTO_ADIANTUM is not set
+# CONFIG_CRYPTO_ESSIV is not set
+# CONFIG_CRYPTO_CMAC is not set
+# CONFIG_CRYPTO_HMAC is not set
+# CONFIG_CRYPTO_XCBC is not set
+# CONFIG_CRYPTO_VMAC is not set
+CONFIG_CRYPTO_CRC32C=y
+# CONFIG_CRYPTO_CRC32 is not set
+CONFIG_CRYPTO_XXHASH=y
+CONFIG_CRYPTO_BLAKE2B=y
+# CONFIG_CRYPTO_BLAKE2S is not set
+# CONFIG_CRYPTO_CRCT10DIF is not set
+# CONFIG_CRYPTO_GHASH is not set
+# CONFIG_CRYPTO_POLY1305 is not set
+# CONFIG_CRYPTO_MD4 is not set
+# CONFIG_CRYPTO_MD5 is not set
+# CONFIG_CRYPTO_MICHAEL_MIC is not set
+# CONFIG_CRYPTO_RMD128 is not set
+# CONFIG_CRYPTO_RMD160 is not set
+# CONFIG_CRYPTO_RMD256 is not set
+# CONFIG_CRYPTO_RMD320 is not set
+# CONFIG_CRYPTO_SHA1 is not set
+CONFIG_CRYPTO_SHA256=y
+# CONFIG_CRYPTO_SHA512 is not set
+# CONFIG_CRYPTO_SHA3 is not set
+# CONFIG_CRYPTO_SM3 is not set
+# CONFIG_CRYPTO_STREEBOG is not set
+# CONFIG_CRYPTO_TGR192 is not set
+# CONFIG_CRYPTO_WP512 is not set
+CONFIG_CRYPTO_AES=y
+# CONFIG_CRYPTO_AES_TI is not set
+# CONFIG_CRYPTO_ANUBIS is not set
+# CONFIG_CRYPTO_ARC4 is not set
+# CONFIG_CRYPTO_BLOWFISH is not set
+# CONFIG_CRYPTO_CAMELLIA is not set
+# CONFIG_CRYPTO_CAST5 is not set
+# CONFIG_CRYPTO_CAST6 is not set
+# CONFIG_CRYPTO_DES is not set
+# CONFIG_CRYPTO_FCRYPT is not set
+# CONFIG_CRYPTO_KHAZAD is not set
+# CONFIG_CRYPTO_SALSA20 is not set
+# CONFIG_CRYPTO_CHACHA20 is not set
+# CONFIG_CRYPTO_SEED is not set
+# CONFIG_CRYPTO_SERPENT is not set
+# CONFIG_CRYPTO_SM4 is not set
+# CONFIG_CRYPTO_TEA is not set
+# CONFIG_CRYPTO_TWOFISH is not set
+# CONFIG_CRYPTO_DEFLATE is not set
+# CONFIG_CRYPTO_LZO is not set
+# CONFIG_CRYPTO_842 is not set
+# CONFIG_CRYPTO_LZ4 is not set
+# CONFIG_CRYPTO_LZ4HC is not set
+# CONFIG_CRYPTO_ZSTD is not set
+CONFIG_CRYPTO_ANSI_CPRNG=y
+# CONFIG_CRYPTO_DRBG_MENU is not set
+# CONFIG_CRYPTO_JITTERENTROPY is not set
+# CONFIG_CRYPTO_USER_API_HASH is not set
+# CONFIG_CRYPTO_USER_API_SKCIPHER is not set
+# CONFIG_CRYPTO_USER_API_RNG is not set
+# CONFIG_CRYPTO_USER_API_AEAD is not set
+CONFIG_CRYPTO_LIB_AES=y
+# CONFIG_CRYPTO_LIB_BLAKE2S is not set
+# CONFIG_CRYPTO_LIB_CHACHA is not set
+# CONFIG_CRYPTO_LIB_CURVE25519 is not set
+CONFIG_CRYPTO_LIB_POLY1305_RSIZE=1
+# CONFIG_CRYPTO_LIB_POLY1305 is not set
+# CONFIG_CRYPTO_LIB_CHACHA20POLY1305 is not set
+CONFIG_CRYPTO_LIB_SHA256=y
+CONFIG_CRYPTO_HW=y
+# CONFIG_CRYPTO_DEV_AMLOGIC_GXL is not set
+CONFIG_RAID6_PQ=y
+# CONFIG_RAID6_PQ_BENCHMARK is not set
+# CONFIG_PACKING is not set
+CONFIG_BITREVERSE=y
+CONFIG_GENERIC_NET_UTILS=y
+# CONFIG_CORDIC is not set
+# CONFIG_CRC_CCITT is not set
+CONFIG_CRC16=y
+# CONFIG_CRC_T10DIF is not set
+# CONFIG_CRC_ITU_T is not set
+CONFIG_CRC32=y
+# CONFIG_CRC32_SELFTEST is not set
+CONFIG_CRC32_SLICEBY8=y
+# CONFIG_CRC32_SLICEBY4 is not set
+# CONFIG_CRC32_SARWATE is not set
+# CONFIG_CRC32_BIT is not set
+# CONFIG_CRC64 is not set
+# CONFIG_CRC4 is not set
+# CONFIG_CRC7 is not set
+CONFIG_LIBCRC32C=y
+# CONFIG_CRC8 is not set
+CONFIG_XXHASH=y
+# CONFIG_RANDOM32_SELFTEST is not set
+CONFIG_ZLIB_INFLATE=y
+CONFIG_ZLIB_DEFLATE=y
+CONFIG_LZO_COMPRESS=y
+CONFIG_LZO_DECOMPRESS=y
+CONFIG_ZSTD_COMPRESS=y
+CONFIG_ZSTD_DECOMPRESS=y
+# CONFIG_XZ_DEC is not set
+CONFIG_HAS_IOMEM=y
+CONFIG_HAS_DMA=y
+# CONFIG_DMA_API_DEBUG is not set
+CONFIG_DQL=y
+CONFIG_NLATTR=y
+CONFIG_GENERIC_ATOMIC64=y
+# CONFIG_IRQ_POLL is not set
+CONFIG_SBITMAP=y
+# CONFIG_STRING_SELFTEST is not set
+CONFIG_PRINTK_TIME=y
+# CONFIG_PRINTK_CALLER is not set
+CONFIG_CONSOLE_LOGLEVEL_DEFAULT=7
+CONFIG_CONSOLE_LOGLEVEL_QUIET=4
+CONFIG_MESSAGE_LOGLEVEL_DEFAULT=4
+CONFIG_SYMBOLIC_ERRNAME=y
+CONFIG_DEBUG_INFO=y
+CONFIG_DEBUG_INFO_REDUCED=y
+# CONFIG_DEBUG_INFO_SPLIT is not set
+# CONFIG_DEBUG_INFO_DWARF4 is not set
+# CONFIG_DEBUG_INFO_BTF is not set
+# CONFIG_GDB_SCRIPTS is not set
+# CONFIG_ENABLE_MUST_CHECK is not set
+# CONFIG_STRIP_ASM_SYMS is not set
+# CONFIG_READABLE_ASM is not set
+# CONFIG_HEADERS_INSTALL is not set
+CONFIG_OPTIMIZE_INLINING=y
+# CONFIG_DEBUG_SECTION_MISMATCH is not set
+CONFIG_SECTION_MISMATCH_WARN_ONLY=y
+CONFIG_ARCH_WANT_FRAME_POINTERS=y
+CONFIG_FRAME_POINTER=y
+# CONFIG_DEBUG_FORCE_WEAK_PER_CPU is not set
+# CONFIG_MAGIC_SYSRQ is not set
+# CONFIG_DEBUG_FS is not set
+# CONFIG_UBSAN is not set
+CONFIG_UBSAN_ALIGNMENT=y
+CONFIG_DEBUG_KERNEL=y
+CONFIG_DEBUG_MISC=y
+# CONFIG_PAGE_EXTENSION is not set
+# CONFIG_DEBUG_PAGEALLOC is not set
+# CONFIG_PAGE_POISONING is not set
+# CONFIG_DEBUG_OBJECTS is not set
+# CONFIG_SLUB_DEBUG_ON is not set
+# CONFIG_SLUB_STATS is not set
+# CONFIG_DEBUG_STACK_USAGE is not set
+# CONFIG_SCHED_STACK_END_CHECK is not set
+# CONFIG_DEBUG_VM is not set
+# CONFIG_DEBUG_NOMMU_REGIONS is not set
+# CONFIG_DEBUG_MEMORY_INIT is not set
+CONFIG_CC_HAS_KASAN_GENERIC=y
+CONFIG_KASAN_STACK=1
+# CONFIG_DEBUG_SHIRQ is not set
+# CONFIG_PANIC_ON_OOPS is not set
+CONFIG_PANIC_ON_OOPS_VALUE=0
+CONFIG_PANIC_TIMEOUT=0
+# CONFIG_SOFTLOCKUP_DETECTOR is not set
+# CONFIG_DETECT_HUNG_TASK is not set
+# CONFIG_WQ_WATCHDOG is not set
+CONFIG_SCHED_DEBUG=y
+# CONFIG_SCHEDSTATS is not set
+# CONFIG_DEBUG_TIMEKEEPING is not set
+# CONFIG_DEBUG_SPINLOCK is not set
+# CONFIG_DEBUG_MUTEXES is not set
+# CONFIG_DEBUG_RWSEMS is not set
+# CONFIG_DEBUG_ATOMIC_SLEEP is not set
+# CONFIG_DEBUG_LOCKING_API_SELFTESTS is not set
+# CONFIG_LOCK_TORTURE_TEST is not set
+# CONFIG_WW_MUTEX_SELFTEST is not set
+# CONFIG_WARN_ALL_UNSEEDED_RANDOM is not set
+# CONFIG_DEBUG_KOBJECT is not set
+# CONFIG_DEBUG_LIST is not set
+# CONFIG_DEBUG_PLIST is not set
+# CONFIG_DEBUG_SG is not set
+# CONFIG_DEBUG_NOTIFIERS is not set
+# CONFIG_BUG_ON_DATA_CORRUPTION is not set
+# CONFIG_DEBUG_CREDENTIALS is not set
+# CONFIG_RCU_PERF_TEST is not set
+# CONFIG_RCU_TORTURE_TEST is not set
+# CONFIG_RCU_TRACE is not set
+# CONFIG_RCU_EQS_DEBUG is not set
+# CONFIG_DEBUG_WQ_FORCE_RR_CPU is not set
+# CONFIG_DEBUG_BLOCK_EXT_DEVT is not set
+# CONFIG_SAMPLES is not set
+# CONFIG_KUNIT is not set
+# CONFIG_NOTIFIER_ERROR_INJECTION is not set
+# CONFIG_FAULT_INJECTION is not set
+CONFIG_CC_HAS_SANCOV_TRACE_PC=y
+CONFIG_RUNTIME_TESTING_MENU=y
+# CONFIG_TEST_LIST_SORT is not set
+# CONFIG_TEST_SORT is not set
+# CONFIG_BACKTRACE_SELF_TEST is not set
+# CONFIG_RBTREE_TEST is not set
+# CONFIG_REED_SOLOMON_TEST is not set
+# CONFIG_INTERVAL_TREE_TEST is not set
+# CONFIG_ATOMIC64_SELFTEST is not set
+# CONFIG_TEST_HEXDUMP is not set
+# CONFIG_TEST_STRING_HELPERS is not set
+# CONFIG_TEST_STRSCPY is not set
+# CONFIG_TEST_KSTRTOX is not set
+# CONFIG_TEST_PRINTF is not set
+# CONFIG_TEST_BITMAP is not set
+# CONFIG_TEST_BITFIELD is not set
+# CONFIG_TEST_UUID is not set
+# CONFIG_TEST_XARRAY is not set
+# CONFIG_TEST_OVERFLOW is not set
+# CONFIG_TEST_RHASHTABLE is not set
+# CONFIG_TEST_HASH is not set
+# CONFIG_TEST_IDA is not set
+# CONFIG_FIND_BIT_BENCHMARK is not set
+# CONFIG_TEST_SYSCTL is not set
+# CONFIG_TEST_UDELAY is not set
+# CONFIG_TEST_MEMCAT_P is not set
+# CONFIG_TEST_STACKINIT is not set
+# CONFIG_TEST_MEMINIT is not set
+# CONFIG_MEMTEST is not set
diff --git a/arch/um/lkl/include/asm/Kbuild b/arch/um/lkl/include/asm/Kbuild
new file mode 100644
index 000000000000..6ae3104fa285
--- /dev/null
+++ b/arch/um/lkl/include/asm/Kbuild
@@ -0,0 +1,79 @@
+generic-y += barrier.h
+generic-y += bitops.h
+generic-y += bug.h
+generic-y += bugs.h
+generic-y += cache.h
+generic-y += cacheflush.h
+generic-y += checksum.h
+generic-y += cmpxchg-local.h
+generic-y += cmpxchg.h
+generic-y += compat.h
+generic-y += cputime.h
+generic-y += current.h
+generic-y += delay.h
+generic-y += device.h
+generic-y += div64.h
+generic-y += dma.h
+generic-y += dma-mapping.h
+generic-y += emergency-restart.h
+generic-y += errno.h
+generic-y += extable.h
+generic-y += exec.h
+generic-y += ftrace.h
+generic-y += futex.h
+generic-y += hardirq.h
+generic-y += hw_irq.h
+generic-y += io.h
+generic-y += ioctl.h
+generic-y += ipcbuf.h
+generic-y += irq_regs.h
+generic-y += irqflags.h
+generic-y += irq_work.h
+generic-y += kdebug.h
+generic-y += kmap_types.h
+generic-y += linkage.h
+generic-y += local.h
+generic-y += local64.h
+generic-y += mcs_spinlock.h
+generic-y += mmiowb.h
+generic-y += mmu.h
+generic-y += mmu_context.h
+generic-y += module.h
+generic-y += msgbuf.h
+generic-y += param.h
+generic-y += parport.h
+generic-y += pci.h
+generic-y += percpu.h
+generic-y += pgalloc.h
+generic-y += poll.h
+generic-y += preempt.h
+generic-y += resource.h
+generic-y += rwsem.h
+generic-y += scatterlist.h
+generic-y += seccomp.h
+generic-y += sections.h
+generic-y += segment.h
+generic-y += sembuf.h
+generic-y += serial.h
+generic-y += shmbuf.h
+generic-y += signal.h
+generic-y += sizes.h
+generic-y += socket.h
+generic-y += sockios.h
+generic-y += stat.h
+generic-y += statfs.h
+generic-y += string.h
+generic-y += swab.h
+generic-y += switch_to.h
+generic-y += syscall.h
+generic-y += termbits.h
+generic-y += termios.h
+generic-y += time.h
+generic-y += timex.h
+generic-y += tlbflush.h
+generic-y += topology.h
+generic-y += trace_clock.h
+generic-y += unaligned.h
+generic-y += user.h
+generic-y += word-at-a-time.h
+generic-y += kprobes.h
diff --git a/arch/um/lkl/include/asm/atomic.h b/arch/um/lkl/include/asm/atomic.h
new file mode 100644
index 000000000000..0c5f9f8e4b30
--- /dev/null
+++ b/arch/um/lkl/include/asm/atomic.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __LKL_ATOMIC_H
+#define __LKL_ATOMIC_H
+
+#include <asm-generic/atomic.h>
+
+#ifndef CONFIG_GENERIC_ATOMIC64
+#include "atomic64.h"
+#endif /* CONFIG_GENERIC_ATOMIC64 */
+
+#endif
diff --git a/arch/um/lkl/include/asm/atomic64.h b/arch/um/lkl/include/asm/atomic64.h
new file mode 100644
index 000000000000..87d7956c29eb
--- /dev/null
+++ b/arch/um/lkl/include/asm/atomic64.h
@@ -0,0 +1,114 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __LKL_ATOMIC64_H
+#define __LKL_ATOMIC64_H
+
+#include <linux/types.h>
+
+#ifdef CONFIG_SMP
+#error "SMP is not supported on this platform"
+#else
+#define ATOMIC64_OP(op, c_op)						\
+static inline void atomic64_##op(s64 i, atomic64_t *v)			\
+{									\
+	unsigned long flags;						\
+									\
+	raw_local_irq_save(flags);					\
+	v->counter = v->counter c_op i;					\
+	raw_local_irq_restore(flags);					\
+}
+
+#define ATOMIC64_OP_RETURN(op, c_op)					\
+static inline s64 atomic64_##op##_return(s64 i, atomic64_t *v)		\
+{									\
+	unsigned long flags;						\
+	s64 ret;							\
+									\
+	raw_local_irq_save(flags);					\
+	ret = (v->counter = v->counter c_op i);				\
+	raw_local_irq_restore(flags);					\
+									\
+	return ret;							\
+}
+
+#define ATOMIC64_FETCH_OP(op, c_op)					\
+static inline s64 atomic64_fetch_##op(s64 i, atomic64_t *v)		\
+{									\
+	unsigned long flags;						\
+	s64 ret;							\
+									\
+	raw_local_irq_save(flags);					\
+	ret = v->counter;						\
+	v->counter = v->counter c_op i;					\
+	raw_local_irq_restore(flags);					\
+									\
+	return ret;							\
+}
+#endif /* CONFIG_SMP */
+
+#ifndef atomic64_add_return
+ATOMIC64_OP_RETURN(add, +)
+#endif
+
+#ifndef atomic64_sub_return
+ATOMIC64_OP_RETURN(sub, -)
+#endif
+
+#ifndef atomic64_fetch_add
+ATOMIC64_FETCH_OP(add, +)
+#endif
+
+#ifndef atomic64_fetch_sub
+ATOMIC64_FETCH_OP(sub, -)
+#endif
+
+#ifndef atomic64_fetch_and
+ATOMIC64_FETCH_OP(and, &)
+#endif
+
+#ifndef atomic64_fetch_or
+ATOMIC64_FETCH_OP(or, |)
+#endif
+
+#ifndef atomic64_fetch_xor
+ATOMIC64_FETCH_OP(xor, ^)
+#endif
+
+#ifndef atomic64_and
+ATOMIC64_OP(and, &)
+#endif
+
+#ifndef atomic64_or
+ATOMIC64_OP(or, |)
+#endif
+
+#ifndef atomic64_xor
+ATOMIC64_OP(xor, ^)
+#endif
+
+#undef ATOMIC64_FETCH_OP
+#undef ATOMIC64_OP_RETURN
+#undef ATOMIC64_OP
+
+
+#define ATOMIC64_INIT(i)	{ (i) }
+
+static inline void atomic64_add(s64 i, atomic64_t *v)
+{
+	atomic64_add_return(i, v);
+}
+
+static inline void atomic64_sub(s64 i, atomic64_t *v)
+{
+	atomic64_sub_return(i, v);
+}
+
+#ifndef atomic64_read
+#define atomic64_read(v)	READ_ONCE((v)->counter)
+#endif
+
+#define atomic64_set(v, i) WRITE_ONCE(((v)->counter), (i))
+
+#define atomic64_xchg(ptr, v)		(xchg(&(ptr)->counter, (v)))
+#define atomic64_cmpxchg(v, old, new)	(cmpxchg(&((v)->counter), (old), (new)))
+
+#endif /* __LKL_ATOMIC64_H */
diff --git a/arch/um/lkl/include/asm/bitsperlong.h b/arch/um/lkl/include/asm/bitsperlong.h
new file mode 100644
index 000000000000..5745d5e51274
--- /dev/null
+++ b/arch/um/lkl/include/asm/bitsperlong.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __LKL_BITSPERLONG_H
+#define __LKL_BITSPERLONG_H
+
+#include <uapi/asm/bitsperlong.h>
+
+#define BITS_PER_LONG __BITS_PER_LONG
+
+#define BITS_PER_LONG_LONG 64
+
+#endif
diff --git a/arch/um/lkl/include/asm/byteorder.h b/arch/um/lkl/include/asm/byteorder.h
new file mode 100644
index 000000000000..5d0c4efaa44b
--- /dev/null
+++ b/arch/um/lkl/include/asm/byteorder.h
@@ -0,0 +1,7 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_BYTEORDER_H
+#define _ASM_LKL_BYTEORDER_H
+
+#include <uapi/asm/byteorder.h>
+
+#endif /* _ASM_LKL_BYTEORDER_H */
diff --git a/arch/um/lkl/include/asm/cpu.h b/arch/um/lkl/include/asm/cpu.h
new file mode 100644
index 000000000000..d2b8c501c7b1
--- /dev/null
+++ b/arch/um/lkl/include/asm/cpu.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_CPU_H
+#define _ASM_LKL_CPU_H
+
+int lkl_cpu_get(void);
+void lkl_cpu_put(void);
+int lkl_cpu_try_run_irq(int irq);
+int lkl_cpu_init(void);
+void lkl_cpu_shutdown(void);
+void lkl_cpu_wait_shutdown(void);
+void lkl_cpu_change_owner(lkl_thread_t owner);
+void lkl_cpu_set_irqs_pending(void);
+
+#endif /* _ASM_LKL_CPU_H */
diff --git a/arch/um/lkl/include/asm/elf.h b/arch/um/lkl/include/asm/elf.h
new file mode 100644
index 000000000000..bb2456d638f4
--- /dev/null
+++ b/arch/um/lkl/include/asm/elf.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_ELF_H
+#define _ASM_LKL_ELF_H
+
+#define elf_check_arch(x) 0
+
+#ifdef CONFIG_64BIT
+#define ELF_CLASS ELFCLASS64
+#else
+#define ELF_CLASS ELFCLASS32
+#endif
+
+#define elf_gregset_t long
+#define elf_fpregset_t double
+#endif
diff --git a/arch/um/lkl/include/asm/processor.h b/arch/um/lkl/include/asm/processor.h
new file mode 100644
index 000000000000..c1aa8b3a266e
--- /dev/null
+++ b/arch/um/lkl/include/asm/processor.h
@@ -0,0 +1,60 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_PROCESSOR_H
+#define _ASM_LKL_PROCESSOR_H
+
+struct task_struct;
+
+static inline void cpu_relax(void)
+{
+	unsigned long flags;
+
+	/* since this is usually called in a tight loop waiting for some
+	 * external condition (e.g. jiffies) lets run interrupts now to allow
+	 * the external condition to propagate
+	 */
+	local_irq_save(flags);
+	local_irq_restore(flags);
+}
+
+#define current_text_addr() ({ __label__ _l; _l: &&_l; })
+
+static inline unsigned long thread_saved_pc(struct task_struct *tsk)
+{
+	return 0;
+}
+
+static inline void release_thread(struct task_struct *dead_task)
+{
+}
+
+static inline void prepare_to_copy(struct task_struct *tsk)
+{
+}
+
+static inline unsigned long get_wchan(struct task_struct *p)
+{
+	return 0;
+}
+
+static inline void flush_thread(void)
+{
+}
+
+static inline void trap_init(void)
+{
+}
+
+struct thread_struct { };
+
+#define INIT_THREAD { }
+
+#define task_pt_regs(tsk) (struct pt_regs *)(NULL)
+
+/* We don't have strict user/kernel spaces */
+#define TASK_SIZE ((unsigned long)-1)
+#define TASK_UNMAPPED_BASE 0
+
+#define KSTK_EIP(tsk) (0)
+#define KSTK_ESP(tsk) (0)
+
+#endif
diff --git a/arch/um/lkl/include/asm/ptrace.h b/arch/um/lkl/include/asm/ptrace.h
new file mode 100644
index 000000000000..28199be26dc0
--- /dev/null
+++ b/arch/um/lkl/include/asm/ptrace.h
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_PTRACE_H
+#define _ASM_LKL_PTRACE_H
+
+#include <linux/errno.h>
+
+struct task_struct;
+
+#define user_mode(regs) 0
+#define kernel_mode(regs) 1
+#define profile_pc(regs) 0
+#define instruction_pointer(regs) 0
+#define user_stack_pointer(regs) 0
+
+static inline long arch_ptrace(struct task_struct *child, long request,
+			       unsigned long addr, unsigned long data)
+{
+	return -EINVAL;
+}
+
+static inline void ptrace_disable(struct task_struct *child)
+{
+}
+
+#endif
diff --git a/arch/um/lkl/include/asm/sched.h b/arch/um/lkl/include/asm/sched.h
new file mode 100644
index 000000000000..4c2635921ec8
--- /dev/null
+++ b/arch/um/lkl/include/asm/sched.h
@@ -0,0 +1,23 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_SCHED_H
+#define _ASM_LKL_SCHED_H
+
+#include <linux/sched.h>
+#include <uapi/asm/host_ops.h>
+
+static inline void thread_sched_jb(void)
+{
+	if (test_ti_thread_flag(current_thread_info(), TIF_HOST_THREAD)) {
+		set_ti_thread_flag(current_thread_info(), TIF_SCHED_JB);
+		set_current_state(TASK_UNINTERRUPTIBLE);
+		lkl_ops->jmp_buf_set(&current_thread_info()->sched_jb,
+				     schedule);
+	} else {
+		lkl_bug("%s() can be used only for host task\n", __func__);
+	}
+}
+
+void switch_to_host_task(struct task_struct *);
+int host_task_stub(void *unused);
+
+#endif /*  _ASM_LKL_SCHED_H */
diff --git a/arch/um/lkl/include/asm/syscalls.h b/arch/um/lkl/include/asm/syscalls.h
new file mode 100644
index 000000000000..2eaa870a9020
--- /dev/null
+++ b/arch/um/lkl/include/asm/syscalls.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_SYSCALLS_H
+#define _ASM_LKL_SYSCALLS_H
+
+int syscalls_init(void);
+void syscalls_cleanup(void);
+long lkl_syscall(long no, long *params);
+void wakeup_idle_host_task(void);
+
+#define sys_mmap sys_mmap_pgoff
+#define sys_mmap2 sys_mmap_pgoff
+#define sys_clone sys_ni_syscall
+#define sys_vfork sys_ni_syscall
+#define sys_rt_sigreturn sys_ni_syscall
+
+#include <asm-generic/syscalls.h>
+
+#endif /* _ASM_LKL_SYSCALLS_H */
diff --git a/arch/um/lkl/include/asm/syscalls_32.h b/arch/um/lkl/include/asm/syscalls_32.h
new file mode 100644
index 000000000000..0e1a7649c81b
--- /dev/null
+++ b/arch/um/lkl/include/asm/syscalls_32.h
@@ -0,0 +1,43 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_SYSCALLS_32_H
+#define _ASM_SYSCALLS_32_H
+
+#include <linux/compiler.h>
+#include <linux/linkage.h>
+#include <linux/types.h>
+#include <linux/signal.h>
+
+#if __BITS_PER_LONG == 32
+
+/* kernel/syscalls_32.c */
+asmlinkage long sys32_truncate64(const char __user *, unsigned long,
+				 unsigned long);
+asmlinkage long sys32_ftruncate64(unsigned int, unsigned long, unsigned long);
+
+#ifdef CONFIG_MMU
+struct mmap_arg_struct32;
+asmlinkage long sys32_mmap(struct mmap_arg_struct32 __user *);
+#endif
+
+asmlinkage long sys32_wait4(pid_t, unsigned int __user *, int,
+			    struct rusage __user *);
+
+asmlinkage long sys32_pread64(unsigned int, char __user *, u32, u32, u32);
+asmlinkage long sys32_pwrite64(unsigned int, const char __user *, u32, u32,
+			       u32);
+
+long sys32_fadvise64_64(int a, __u32 b, __u32 c, __u32 d, __u32 e, int f);
+
+asmlinkage ssize_t sys32_readahead(int, unsigned int, unsigned int, size_t);
+asmlinkage long sys32_sync_file_range(int, unsigned int, unsigned int,
+				      unsigned int, unsigned int, unsigned int);
+asmlinkage long sys32_sync_file_range2(int, unsigned int, unsigned int,
+				       unsigned int, unsigned int,
+				       unsigned int);
+asmlinkage long sys32_fadvise64(int, unsigned int, unsigned int, size_t, int);
+asmlinkage long sys32_fallocate(int, int, unsigned int, unsigned int,
+				unsigned int, unsigned int);
+
+#endif /* __BITS_PER_LONG */
+
+#endif /* _ASM_SYSCALLS_32_H */
diff --git a/arch/um/lkl/include/asm/tlb.h b/arch/um/lkl/include/asm/tlb.h
new file mode 100644
index 000000000000..d474890d317d
--- /dev/null
+++ b/arch/um/lkl/include/asm/tlb.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_TLB_H
+#define _ASM_LKL_TLB_H
+
+#define tlb_start_vma(tlb, vma)				do { } while (0)
+#define tlb_end_vma(tlb, vma)				do { } while (0)
+#define __tlb_remove_tlb_entry(tlb, pte, address)	do { } while (0)
+#define tlb_flush(tlb)					do { } while (0)
+
+#include <asm-generic/tlb.h>
+
+#endif /* _ASM_LKL_TLB_H */
diff --git a/arch/um/lkl/include/asm/uaccess.h b/arch/um/lkl/include/asm/uaccess.h
new file mode 100644
index 000000000000..f267ac3be8b3
--- /dev/null
+++ b/arch/um/lkl/include/asm/uaccess.h
@@ -0,0 +1,64 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_UACCESS_H
+#define _ASM_LKL_UACCESS_H
+
+/* copied from old include/asm-generic/uaccess.h */
+static inline __must_check long
+raw_copy_from_user(void *to, const void __user *from, unsigned long n)
+{
+	if (__builtin_constant_p(n)) {
+		switch (n) {
+		case 1:
+			*(u8 *)to = *(u8 __force *)from;
+			return 0;
+		case 2:
+			*(u16 *)to = *(u16 __force *)from;
+			return 0;
+		case 4:
+			*(u32 *)to = *(u32 __force *)from;
+			return 0;
+#ifdef CONFIG_64BIT
+		case 8:
+			*(u64 *)to = *(u64 __force *)from;
+			return 0;
+#endif
+		default:
+			break;
+		}
+	}
+
+	memcpy(to, (const void __force *)from, n);
+	return 0;
+}
+
+static inline __must_check long
+raw_copy_to_user(void __user *to, const void *from, unsigned long n)
+{
+	if (__builtin_constant_p(n)) {
+		switch (n) {
+		case 1:
+			*(u8 __force *)to = *(u8 *)from;
+			return 0;
+		case 2:
+			*(u16 __force *)to = *(u16 *)from;
+			return 0;
+		case 4:
+			*(u32 __force *)to = *(u32 *)from;
+			return 0;
+#ifdef CONFIG_64BIT
+		case 8:
+			*(u64 __force *)to = *(u64 *)from;
+			return 0;
+#endif
+		default:
+			break;
+		}
+	}
+
+	memcpy((void __force *)to, from, n);
+	return 0;
+}
+
+#include <asm-generic/uaccess.h>
+
+#endif
diff --git a/arch/um/lkl/include/asm/unistd_32.h b/arch/um/lkl/include/asm/unistd_32.h
new file mode 100644
index 000000000000..8582a55e61e2
--- /dev/null
+++ b/arch/um/lkl/include/asm/unistd_32.h
@@ -0,0 +1,31 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <asm/bitsperlong.h>
+
+#ifndef __SYSCALL
+#define __SYSCALL(x, y)
+#endif
+
+#if __BITS_PER_LONG == 32
+__SYSCALL(__NR3264_truncate, sys32_truncate64)
+__SYSCALL(__NR3264_ftruncate, sys32_ftruncate64)
+
+#ifdef CONFIG_MMU
+__SYSCALL(__NR3264_mmap, sys32_mmap)
+#endif
+
+__SYSCALL(__NR_wait4, sys32_wait4)
+
+__SYSCALL(__NR_pread64, sys32_pread64)
+__SYSCALL(__NR_pwrite64, sys32_pwrite64)
+
+__SYSCALL(__NR_readahead, sys32_readahead)
+#ifdef __ARCH_WANT_SYNC_FILE_RANGE2
+__SYSCALL(__NR_sync_file_range2, sys32_sync_file_range2)
+#else
+__SYSCALL(__NR_sync_file_range, sys32_sync_file_range)
+#endif
+/* mm/fadvise.c */
+__SYSCALL(__NR3264_fadvise64, sys32_fadvise64_64)
+__SYSCALL(__NR_fallocate, sys32_fallocate)
+
+#endif
diff --git a/arch/um/lkl/include/asm/vmalloc.h b/arch/um/lkl/include/asm/vmalloc.h
new file mode 100644
index 000000000000..46de2e4f03c7
--- /dev/null
+++ b/arch/um/lkl/include/asm/vmalloc.h
@@ -0,0 +1,5 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_VMALLOC_H
+#define _LKL_VMALLOC_H
+
+#endif /* _LKL_VMALLOC_H */
diff --git a/arch/um/lkl/include/asm/vmlinux.lds.h b/arch/um/lkl/include/asm/vmlinux.lds.h
new file mode 100644
index 000000000000..a3c285882dc4
--- /dev/null
+++ b/arch/um/lkl/include/asm/vmlinux.lds.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_VMLINUX_LDS_H
+#define _LKL_VMLINUX_LDS_H
+
+/* we encode our own __ro_after_init section */
+#define RO_AFTER_INIT_DATA
+
+#ifdef __MINGW32__
+#define RODATA_SECTION .rdata
+#endif
+
+#include <asm-generic/vmlinux.lds.h>
+
+#endif
diff --git a/arch/um/lkl/include/asm/xor.h b/arch/um/lkl/include/asm/xor.h
new file mode 100644
index 000000000000..286ce75b5d9d
--- /dev/null
+++ b/arch/um/lkl/include/asm/xor.h
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_XOR_H
+#define _ASM_LKL_XOR_H
+
+#include <asm-generic/xor.h>
+
+#define XOR_SELECT_TEMPLATE(x) (&xor_block_8regs)
+
+#endif /* _ASM_LKL_XOR_H */
diff --git a/arch/um/lkl/include/uapi/asm/Kbuild b/arch/um/lkl/include/uapi/asm/Kbuild
new file mode 100644
index 000000000000..e1f554364a56
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/Kbuild
@@ -0,0 +1,11 @@
+# UAPI Header export list
+
+generic-y += elf.h
+generic-y += kvm_para.h
+generic-y += shmparam.h
+generic-y += siginfo.h
+generic-y += swab.h
+generic-y += timex.h
+
+# no header-y since we need special user headers handling
+# see arch/lkl/script/headers.py
diff --git a/arch/um/lkl/include/uapi/asm/bitsperlong.h b/arch/um/lkl/include/uapi/asm/bitsperlong.h
new file mode 100644
index 000000000000..8b4ebf2b0264
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/bitsperlong.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _ASM_UAPI_LKL_BITSPERLONG_H
+#define _ASM_UAPI_LKL_BITSPERLONG_H
+
+#ifdef CONFIG_64BIT
+#define __BITS_PER_LONG 64
+#else
+#define __BITS_PER_LONG 32
+#endif
+
+#define __ARCH_WANT_STAT64
+
+#endif /* _ASM_UAPI_LKL_BITSPERLONG_H */
diff --git a/arch/um/lkl/include/uapi/asm/byteorder.h b/arch/um/lkl/include/uapi/asm/byteorder.h
new file mode 100644
index 000000000000..3c4a58d2062f
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/byteorder.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _ASM_UAPI_LKL_BYTEORDER_H
+#define _ASM_UAPI_LKL_BYTEORDER_H
+
+#if defined(CONFIG_BIG_ENDIAN)
+#include <linux/byteorder/big_endian.h>
+#else
+#include <linux/byteorder/little_endian.h>
+#endif
+
+#endif /* _ASM_UAPI_LKL_BYTEORDER_H */
diff --git a/arch/um/lkl/include/uapi/asm/syscalls.h b/arch/um/lkl/include/uapi/asm/syscalls.h
new file mode 100644
index 000000000000..a81534ffccb7
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/syscalls.h
@@ -0,0 +1,348 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _ASM_UAPI_LKL_SYSCALLS_H
+#define _ASM_UAPI_LKL_SYSCALLS_H
+
+#include <autoconf.h>
+#include <linux/types.h>
+
+typedef __kernel_uid32_t	qid_t;
+typedef __kernel_fd_set		fd_set;
+typedef __kernel_mode_t		mode_t;
+typedef unsigned short		umode_t;
+typedef __u32			nlink_t;
+typedef __kernel_off_t		off_t;
+typedef __kernel_pid_t		pid_t;
+typedef __kernel_key_t		key_t;
+typedef __kernel_suseconds_t	suseconds_t;
+typedef __kernel_timer_t	timer_t;
+typedef __kernel_clockid_t	clockid_t;
+typedef __kernel_mqd_t		mqd_t;
+typedef __kernel_uid32_t	uid_t;
+typedef __kernel_gid32_t	gid_t;
+typedef __kernel_uid16_t        uid16_t;
+typedef __kernel_gid16_t        gid16_t;
+typedef unsigned long		uintptr_t;
+#ifdef CONFIG_UID16
+typedef __kernel_old_uid_t	old_uid_t;
+typedef __kernel_old_gid_t	old_gid_t;
+#endif
+typedef __kernel_loff_t		loff_t;
+typedef __kernel_size_t		size_t;
+typedef __kernel_ssize_t	ssize_t;
+typedef __kernel_time_t		time_t;
+typedef __kernel_clock_t	clock_t;
+typedef __u32			u32;
+typedef __s32			s32;
+typedef __u64			u64;
+typedef __s64			s64;
+
+#define __user
+
+#include <asm/unistd.h>
+/* Temporary undefine system calls that don't have data types defined in UAPI
+ * headers
+ */
+#undef __NR_kexec_load
+#undef __NR_getcpu
+#undef __NR_sched_getattr
+#undef __NR_sched_setattr
+#undef __NR_sched_setparam
+#undef __NR_sched_getparam
+#undef __NR_sched_setscheduler
+#undef __NR_name_to_handle_at
+#undef __NR_open_by_handle_at
+
+/* deprecated system calls */
+#undef __NR_epoll_create
+#undef __NR_epoll_wait
+#undef __NR_access
+#undef __NR_chmod
+#undef __NR_chown
+#undef __NR_lchown
+#undef __NR_open
+#undef __NR_creat
+#undef __NR_readlink
+#undef __NR_pipe
+#undef __NR_mknod
+#undef __NR_mkdir
+#undef __NR_rmdir
+#undef __NR_unlink
+#undef __NR_symlink
+#undef __NR_link
+#undef __NR_rename
+#undef __NR_getdents
+#undef __NR_select
+#undef __NR_poll
+#undef __NR_dup2
+#undef __NR_futimesat
+#undef __NR_utimes
+#undef __NR_ustat
+#undef __NR_eventfd
+#undef __NR_bdflush
+#undef __NR_send
+#undef __NR_recv
+
+#undef __NR_umount
+#define __NR_umount __NR_umount2
+
+#ifdef CONFIG_64BIT
+#define __NR_newfstat __NR3264_fstat
+#define __NR_newfstatat __NR3264_fstatat
+#endif
+
+#define __NR_mmap_pgoff __NR3264_mmap
+
+#include <linux/time.h>
+#include <linux/times.h>
+#include <linux/timex.h>
+#include <linux/capability.h>
+#define __KERNEL__ /* to pull in S_ definitions */
+#include <linux/stat.h>
+#undef __KERNEL__
+#include <linux/errno.h>
+#include <linux/fcntl.h>
+#include <linux/fs.h>
+#include <asm/statfs.h>
+#include <asm/stat.h>
+#include <linux/bpf.h>
+#include <linux/msg.h>
+#include <linux/resource.h>
+#include <linux/sysinfo.h>
+#include <linux/shm.h>
+#include <linux/aio_abi.h>
+#include <linux/socket.h>
+#include <linux/perf_event.h>
+#include <linux/sem.h>
+#include <linux/futex.h>
+#include <linux/poll.h>
+#include <linux/mqueue.h>
+#include <linux/eventpoll.h>
+#include <linux/uio.h>
+#include <asm/signal.h>
+#include <asm/siginfo.h>
+#include <linux/utime.h>
+#include <asm/socket.h>
+#include <linux/icmp.h>
+#include <linux/ip.h>
+
+/* Define data structures used in system calls that are not defined in UAPI
+ * headers
+ */
+struct sockaddr {
+	unsigned short int sa_family;
+	char sa_data[14];
+};
+
+#define __UAPI_DEF_IF_NET_DEVICE_FLAGS_LOWER_UP_DORMANT_ECHO 1
+#define __UAPI_DEF_IF_IFNAMSIZ	1
+#define __UAPI_DEF_IF_NET_DEVICE_FLAGS 1
+#define __UAPI_DEF_IF_IFREQ	1
+#define __UAPI_DEF_IF_IFMAP	1
+#include <linux/if.h>
+#define __UAPI_DEF_IN_IPPROTO	1
+#define __UAPI_DEF_IN_ADDR	1
+#define __UAPI_DEF_IN6_ADDR	1
+#define __UAPI_DEF_IP_MREQ	1
+#define __UAPI_DEF_IN_PKTINFO	1
+#define __UAPI_DEF_SOCKADDR_IN	1
+#define __UAPI_DEF_IN_CLASS	1
+#include <linux/in.h>
+#include <linux/in6.h>
+#include <linux/sockios.h>
+#include <linux/route.h>
+#include <linux/ipv6_route.h>
+#include <linux/ipv6.h>
+#include <linux/netlink.h>
+#include <linux/neighbour.h>
+#include <linux/rtnetlink.h>
+#include <linux/fib_rules.h>
+
+#include <linux/kdev_t.h>
+#include <asm/irq.h>
+#include <linux/virtio_blk.h>
+#include <linux/virtio_net.h>
+#include <linux/virtio_ring.h>
+#include <linux/pkt_sched.h>
+#include <linux/io_uring.h>
+
+struct user_msghdr {
+	void		__user *msg_name;
+	int		msg_namelen;
+	struct iovec	__user *msg_iov;
+	__kernel_size_t	msg_iovlen;
+	void		__user *msg_control;
+	__kernel_size_t	msg_controllen;
+	unsigned int	msg_flags;
+};
+
+typedef __u32 key_serial_t;
+
+struct mmsghdr {
+	struct user_msghdr  msg_hdr;
+	unsigned int        msg_len;
+};
+
+struct linux_dirent64 {
+	u64		d_ino;
+	s64		d_off;
+	unsigned short	d_reclen;
+	unsigned char	d_type;
+	char		d_name[0];
+};
+
+struct linux_dirent {
+	unsigned long	d_ino;
+	unsigned long	d_off;
+	unsigned short	d_reclen;
+	char		d_name[1];
+};
+
+struct ustat {
+	__kernel_daddr_t	f_tfree;
+	__kernel_ino_t		f_tinode;
+	char			f_fname[6];
+	char			f_fpack[6];
+};
+
+typedef __kernel_rwf_t		rwf_t;
+
+#define AF_UNSPEC       0
+#define AF_UNIX         1
+#define AF_LOCAL        1
+#define AF_INET         2
+#define AF_AX25         3
+#define AF_IPX          4
+#define AF_APPLETALK    5
+#define AF_NETROM       6
+#define AF_BRIDGE       7
+#define AF_ATMPVC       8
+#define AF_X25          9
+#define AF_INET6        10
+#define AF_ROSE         11
+#define AF_DECnet       12
+#define AF_NETBEUI      13
+#define AF_SECURITY     14
+#define AF_KEY          15
+#define AF_NETLINK      16
+#define AF_ROUTE        AF_NETLINK
+#define AF_PACKET       17
+#define AF_ASH          18
+#define AF_ECONET       19
+#define AF_ATMSVC       20
+#define AF_RDS          21
+#define AF_SNA          22
+#define AF_IRDA         23
+#define AF_PPPOX        24
+#define AF_WANPIPE      25
+#define AF_LLC          26
+#define AF_IB           27
+#define AF_MPLS         28
+#define AF_CAN          29
+#define AF_TIPC         30
+#define AF_BLUETOOTH    31
+#define AF_IUCV         32
+#define AF_RXRPC        33
+#define AF_ISDN         34
+#define AF_PHONET       35
+#define AF_IEEE802154   36
+#define AF_CAIF         37
+#define AF_ALG          38
+#define AF_NFC          39
+#define AF_VSOCK        40
+
+#define SOCK_STREAM		1
+#define SOCK_DGRAM		2
+#define SOCK_RAW		3
+#define SOCK_RDM		4
+#define SOCK_SEQPACKET		5
+#define SOCK_DCCP		6
+#define SOCK_PACKET		10
+
+#define MSG_TRUNC 0x20
+#define MSG_DONTWAIT 0x40
+
+/* avoid colision with system headers defines */
+#define sa_handler sa_handler
+#define st_atime st_atime
+#define st_mtime st_mtime
+#define st_ctime st_ctime
+#define s_addr s_addr
+
+long lkl_syscall(long no, long *params);
+long lkl_sys_halt(void);
+
+#define __MAP0(m, ...)
+#define __MAP1(m, t, a) m(t, a)
+#define __MAP2(m, t, a, ...) m(t, a), __MAP1(m, __VA_ARGS__)
+#define __MAP3(m, t, a, ...) m(t, a), __MAP2(m, __VA_ARGS__)
+#define __MAP4(m, t, a, ...) m(t, a), __MAP3(m, __VA_ARGS__)
+#define __MAP5(m, t, a, ...) m(t, a), __MAP4(m, __VA_ARGS__)
+#define __MAP6(m, t, a, ...) m(t, a), __MAP5(m, __VA_ARGS__)
+#define __MAP(n, ...) __MAP##n(__VA_ARGS__)
+
+#define __SC_LONG(t, a) (long)a
+#define __SC_TABLE(t, a) {sizeof(t), (long long)(a)}
+#define __SC_DECL(t, a) t a
+
+#define LKL_SYSCALL0(name)					       \
+	static inline long lkl_sys##name(void)			       \
+	{							       \
+		long params[6];					       \
+		return lkl_syscall(__lkl__NR##name, params);	       \
+	}
+
+#if __BITS_PER_LONG == 32
+#define LKL_SYSCALLx(x, name, ...)					\
+	static inline							\
+	long lkl_sys##name(__MAP(x, __SC_DECL, __VA_ARGS__))		\
+	{								\
+		struct {						\
+			unsigned int size;				\
+			long long value;				\
+		} lkl_params[x] = { __MAP(x, __SC_TABLE, __VA_ARGS__) }; \
+		long sys_params[6], i, k;				\
+		for (i = k = 0; i < x && k < 6; i++, k++) {		\
+			if (lkl_params[i].size > sizeof(long) &&	\
+			    k + 1 < 6) {				\
+				sys_params[k] =				\
+					(long)(lkl_params[i].value & (-1UL)); \
+				k++;					\
+				sys_params[k] =				\
+					(long)(lkl_params[i].value >>	\
+					       __BITS_PER_LONG);	\
+			} else {					\
+				sys_params[k] = (long)(lkl_params[i].value); \
+			}						\
+		}							\
+		return lkl_syscall(__lkl__NR##name, sys_params);	\
+	}
+#else
+#define LKL_SYSCALLx(x, name, ...)					\
+	static inline							\
+	long lkl_sys##name(__MAP(x, __SC_DECL, __VA_ARGS__))		\
+	{								\
+		long lkl_params[6] = { __MAP(x, __SC_LONG, __VA_ARGS__) }; \
+		return lkl_syscall(__lkl__NR##name, lkl_params);	\
+	}
+#endif
+
+#define SYSCALL_DEFINE0(name, ...) LKL_SYSCALL0(name)
+#define SYSCALL_DEFINE1(name, ...) LKL_SYSCALLx(1, name, __VA_ARGS__)
+#define SYSCALL_DEFINE2(name, ...) LKL_SYSCALLx(2, name, __VA_ARGS__)
+#define SYSCALL_DEFINE3(name, ...) LKL_SYSCALLx(3, name, __VA_ARGS__)
+#define SYSCALL_DEFINE4(name, ...) LKL_SYSCALLx(4, name, __VA_ARGS__)
+#define SYSCALL_DEFINE5(name, ...) LKL_SYSCALLx(5, name, __VA_ARGS__)
+#define SYSCALL_DEFINE6(name, ...) LKL_SYSCALLx(6, name, __VA_ARGS__)
+
+#if __BITS_PER_LONG == 32
+#pragma GCC diagnostic push
+#pragma GCC diagnostic ignored "-Wpointer-to-int-cast"
+#endif
+
+#include <asm/syscall_defs.h>
+
+#if __BITS_PER_LONG == 32
+#pragma GCC diagnostic pop
+#endif
+
+#endif
diff --git a/arch/um/lkl/kernel/asm-offsets.c b/arch/um/lkl/kernel/asm-offsets.c
new file mode 100644
index 000000000000..6be0763698dc
--- /dev/null
+++ b/arch/um/lkl/kernel/asm-offsets.c
@@ -0,0 +1,2 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Dummy asm-offsets.c file. Required by kbuild and ready to be used - hint! */
diff --git a/arch/um/lkl/kernel/misc.c b/arch/um/lkl/kernel/misc.c
new file mode 100644
index 000000000000..60f048f02ae6
--- /dev/null
+++ b/arch/um/lkl/kernel/misc.c
@@ -0,0 +1,60 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/kallsyms.h>
+#include <linux/module.h>
+#include <linux/sched.h>
+#include <linux/seq_file.h>
+#include <asm/ptrace.h>
+#include <asm/host_ops.h>
+
+#ifdef CONFIG_PRINTK
+void dump_stack(void)
+{
+	unsigned long dummy;
+	unsigned long *stack = &dummy;
+	unsigned long addr;
+
+	pr_info("Call Trace:\n");
+	while (((long)stack & (THREAD_SIZE - 1)) != 0) {
+		addr = *stack;
+		if (__kernel_text_address(addr)) {
+			pr_info("%p:  [<%08lx>] %pS", stack, addr,
+				(void *)addr);
+			pr_cont("\n");
+		}
+		stack++;
+	}
+	pr_info("\n");
+}
+#endif
+
+void show_regs(struct pt_regs *regs)
+{
+}
+
+#ifdef CONFIG_PROC_FS
+static void *cpuinfo_start(struct seq_file *m, loff_t *pos)
+{
+	return NULL;
+}
+
+static void *cpuinfo_next(struct seq_file *m, void *v, loff_t *pos)
+{
+	return NULL;
+}
+
+static void cpuinfo_stop(struct seq_file *m, void *v)
+{
+}
+
+static int show_cpuinfo(struct seq_file *m, void *v)
+{
+	return 0;
+}
+
+const struct seq_operations cpuinfo_op = {
+	.start	= cpuinfo_start,
+	.next	= cpuinfo_next,
+	.stop	= cpuinfo_stop,
+	.show	= show_cpuinfo,
+};
+#endif
diff --git a/arch/um/lkl/kernel/vmlinux.lds.S b/arch/um/lkl/kernel/vmlinux.lds.S
new file mode 100644
index 000000000000..f57c563522cb
--- /dev/null
+++ b/arch/um/lkl/kernel/vmlinux.lds.S
@@ -0,0 +1,51 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <asm/vmlinux.lds.h>
+#include <asm/thread_info.h>
+#include <asm/page.h>
+#include <asm/cache.h>
+#include <linux/export.h>
+
+OUTPUT_FORMAT(CONFIG_OUTPUT_FORMAT)
+
+jiffies = jiffies_64;
+
+SECTIONS
+{
+	__init_begin = .;
+	HEAD_TEXT_SECTION
+	INIT_TEXT_SECTION(PAGE_SIZE)
+	INIT_DATA_SECTION(16)
+	PERCPU_SECTION(L1_CACHE_BYTES)
+	__init_end = .;
+
+	_stext = .;
+	_text = . ;
+	text = . ;
+	.text      :
+	{
+		TEXT_TEXT
+		SCHED_TEXT
+		LOCK_TEXT
+		CPUIDLE_TEXT
+	}
+	_etext = .;
+
+	_sdata = .;
+	RO_DATA(PAGE_SIZE)
+	RW_DATA(L1_CACHE_BYTES, PAGE_SIZE, THREAD_SIZE)
+	_edata = .;
+
+	__start_ro_after_init = .;
+	.data..ro_after_init : { *(.data..ro_after_init)}
+	EXCEPTION_TABLE(16)
+	__end_ro_after_init = .;
+	NOTES
+
+	BSS_SECTION(0, 0, 0)
+	_end = .;
+
+	STABS_DEBUG
+	DWARF_DEBUG
+
+	DISCARDS
+}
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 02/25] um lkl: architecture skeleton for Linux kernel library
@ 2020-03-30 14:45     ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: linux-arch, Levente Kurusa, Matthieu Coudron, Hajime Tazaki,
	Conrad Meyer, Octavian Purdila, Jens Staal, Motomu Utsumi,
	Lai Jiangshan, Akira Moroo, Petros Angelatos, Yuan Liu, Xiao Jia,
	Mark Stillwell, Patrick Collins, linux-kernel-library,
	Pierre-Hugues Husson, Michael Zimmermann, Luca Dariz,
	Edison M . Castro

From: Octavian Purdila <tavi.purdila@gmail.com>

Adds the LKL Kconfig, vmlinux linker script, basic architecture
headers and miscellaneous basic functions or stubs such as
dump_stack(), show_regs() and cpuinfo proc ops.

The headers we introduce in this patch are simple wrappers to the
asm-generic headers or stubs for things we don't support, such as
ptrace, DMA, signals, ELF handling and low level processor operations.

The kernel configuration is automatically updated to reflect the
endianness of the host, 64bit support or the output format for
vmlinux's linker script. We do this by looking at the ld's default
output format.

Cc: Andreas Abel <aabel@google.com>
Cc: Conrad Meyer <cem@FreeBSD.org>
Cc: Edison M. Castro <edisonmcastro@hotmail.com>
Cc: H.K. Jerry Chu <hkchu@google.com>
Cc: Jens Staal <staal1978@gmail.com>
Cc: Lai Jiangshan <jiangshanlai@gmail.com>
Cc: Levente Kurusa <levex@linux.com>
Cc: Luca Dariz <luca.dariz@gmail.com>
Cc: Mark Stillwell <mark@stillwell.me>
Cc: Matthieu Coudron <mattator@gmail.com>
Cc: Michael Zimmermann <sigmaepsilon92@gmail.com>
Cc: Motomu Utsumi <motomuman@gmail.com>
Cc: Patrick Collins <pscollins@google.com>
Cc: Petros Angelatos <petrosagg@gmail.com>
Cc: Pierre-Hugues Husson <phh@phh.me>
Cc: Xiao Jia <xiaoj@google.com>
Cc: Yuan Liu <liuyuan@google.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 MAINTAINERS                                |    8 +
 arch/um/lkl/.gitignore                     |    2 +
 arch/um/lkl/Kconfig                        |   56 ++
 arch/um/lkl/Kconfig.debug                  |    0
 arch/um/lkl/configs/lkl_defconfig          | 1052 ++++++++++++++++++++
 arch/um/lkl/include/asm/Kbuild             |   79 ++
 arch/um/lkl/include/asm/atomic.h           |   11 +
 arch/um/lkl/include/asm/atomic64.h         |  114 +++
 arch/um/lkl/include/asm/bitsperlong.h      |   11 +
 arch/um/lkl/include/asm/byteorder.h        |    7 +
 arch/um/lkl/include/asm/cpu.h              |   14 +
 arch/um/lkl/include/asm/elf.h              |   15 +
 arch/um/lkl/include/asm/processor.h        |   60 ++
 arch/um/lkl/include/asm/ptrace.h           |   25 +
 arch/um/lkl/include/asm/sched.h            |   23 +
 arch/um/lkl/include/asm/syscalls.h         |   18 +
 arch/um/lkl/include/asm/syscalls_32.h      |   43 +
 arch/um/lkl/include/asm/tlb.h              |   12 +
 arch/um/lkl/include/asm/uaccess.h          |   64 ++
 arch/um/lkl/include/asm/unistd_32.h        |   31 +
 arch/um/lkl/include/asm/vmalloc.h          |    5 +
 arch/um/lkl/include/asm/vmlinux.lds.h      |   14 +
 arch/um/lkl/include/asm/xor.h              |    9 +
 arch/um/lkl/include/uapi/asm/Kbuild        |   11 +
 arch/um/lkl/include/uapi/asm/bitsperlong.h |   13 +
 arch/um/lkl/include/uapi/asm/byteorder.h   |   11 +
 arch/um/lkl/include/uapi/asm/syscalls.h    |  348 +++++++
 arch/um/lkl/kernel/asm-offsets.c           |    2 +
 arch/um/lkl/kernel/misc.c                  |   60 ++
 arch/um/lkl/kernel/vmlinux.lds.S           |   51 +
 30 files changed, 2169 insertions(+)
 create mode 100644 arch/um/lkl/.gitignore
 create mode 100644 arch/um/lkl/Kconfig
 create mode 100644 arch/um/lkl/Kconfig.debug
 create mode 100644 arch/um/lkl/configs/lkl_defconfig
 create mode 100644 arch/um/lkl/include/asm/Kbuild
 create mode 100644 arch/um/lkl/include/asm/atomic.h
 create mode 100644 arch/um/lkl/include/asm/atomic64.h
 create mode 100644 arch/um/lkl/include/asm/bitsperlong.h
 create mode 100644 arch/um/lkl/include/asm/byteorder.h
 create mode 100644 arch/um/lkl/include/asm/cpu.h
 create mode 100644 arch/um/lkl/include/asm/elf.h
 create mode 100644 arch/um/lkl/include/asm/processor.h
 create mode 100644 arch/um/lkl/include/asm/ptrace.h
 create mode 100644 arch/um/lkl/include/asm/sched.h
 create mode 100644 arch/um/lkl/include/asm/syscalls.h
 create mode 100644 arch/um/lkl/include/asm/syscalls_32.h
 create mode 100644 arch/um/lkl/include/asm/tlb.h
 create mode 100644 arch/um/lkl/include/asm/uaccess.h
 create mode 100644 arch/um/lkl/include/asm/unistd_32.h
 create mode 100644 arch/um/lkl/include/asm/vmalloc.h
 create mode 100644 arch/um/lkl/include/asm/vmlinux.lds.h
 create mode 100644 arch/um/lkl/include/asm/xor.h
 create mode 100644 arch/um/lkl/include/uapi/asm/Kbuild
 create mode 100644 arch/um/lkl/include/uapi/asm/bitsperlong.h
 create mode 100644 arch/um/lkl/include/uapi/asm/byteorder.h
 create mode 100644 arch/um/lkl/include/uapi/asm/syscalls.h
 create mode 100644 arch/um/lkl/kernel/asm-offsets.c
 create mode 100644 arch/um/lkl/kernel/misc.c
 create mode 100644 arch/um/lkl/kernel/vmlinux.lds.S

diff --git a/MAINTAINERS b/MAINTAINERS
index cc1d18cb5d18..f3913964eabe 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -9710,6 +9710,14 @@ F:	Documentation/core-api/atomic_ops.rst
 F:	Documentation/core-api/refcount-vs-atomic.rst
 F:	Documentation/memory-barriers.txt
 
+LINUX KERNEL LIBRARY
+M:	Octavian Purdila <tavi.purdila@gmail.com>
+M:	Hajime Tazaki <thehajime@gmail.com>
+L:	linux-kernel-library@freelists.org
+S:	Maintained
+F:	arch/um/lkl/
+F:	tools/lkl/
+
 LIS3LV02D ACCELEROMETER DRIVER
 M:	Eric Piel <eric.piel@tremplin-utc.net>
 S:	Maintained
diff --git a/arch/um/lkl/.gitignore b/arch/um/lkl/.gitignore
new file mode 100644
index 000000000000..ced1c60d8235
--- /dev/null
+++ b/arch/um/lkl/.gitignore
@@ -0,0 +1,2 @@
+kernel/vmlinux.lds
+include/generated
diff --git a/arch/um/lkl/Kconfig b/arch/um/lkl/Kconfig
new file mode 100644
index 000000000000..f7e8d707a12b
--- /dev/null
+++ b/arch/um/lkl/Kconfig
@@ -0,0 +1,56 @@
+# SPDX-License-Identifier: GPL-2.0
+
+config UML_LKL
+       def_bool y
+       depends on !SMP && !MMU && !COREDUMP && !SECCOMP && !UPROBES && !COMPAT && !USER_RETURN_NOTIFIER
+       select ARCH_THREAD_STACK_ALLOCATOR
+       select RWSEM_GENERIC_SPINLOCK
+       select GENERIC_ATOMIC64 if !64BIT
+       select GENERIC_HWEIGHT
+       select FLATMEM
+       select FLAT_NODE_MEM_MAP
+       select GENERIC_CLOCKEVENTS
+       select GENERIC_CPU_DEVICES
+       select NO_HZ_IDLE
+       select NO_PREEMPT
+       select ARCH_WANT_FRAME_POINTERS
+       select HAS_DMA
+       select DMA_DIRECT_OPS
+       select PHYS_ADDR_T_64BIT if 64BIT
+       select 64BIT if "$(OUTPUT_FORMAT)" = "elf64-x86-64"
+       select 64BIT if "$(OUTPUT_FORMAT)" = "elf64-x86-64-freebsd"
+       select NET
+       select MULTIUSER
+
+config OUTPUT_FORMAT
+       string "Output format"
+       default "$(OUTPUT_FORMAT)"
+
+config ARCH_DMA_ADDR_T_64BIT
+       def_bool 64BIT
+
+config 64BIT
+       def_bool n
+
+config COREDUMP
+       def_bool n
+
+config BIG_ENDIAN
+       def_bool n
+
+config GENERIC_CSUM
+       def_bool y
+
+config GENERIC_HWEIGHT
+       def_bool y
+
+config NO_IOPORT_MAP
+       def_bool y
+
+config RWSEM_GENERIC_SPINLOCK
+	bool
+	default y
+
+config HZ
+        int
+        default 100
diff --git a/arch/um/lkl/Kconfig.debug b/arch/um/lkl/Kconfig.debug
new file mode 100644
index 000000000000..e69de29bb2d1
diff --git a/arch/um/lkl/configs/lkl_defconfig b/arch/um/lkl/configs/lkl_defconfig
new file mode 100644
index 000000000000..7050c233a17e
--- /dev/null
+++ b/arch/um/lkl/configs/lkl_defconfig
@@ -0,0 +1,1052 @@
+CONFIG_CC_IS_GCC=y
+CONFIG_GCC_VERSION=80301
+CONFIG_CLANG_VERSION=0
+CONFIG_CC_CAN_LINK=y
+CONFIG_CC_HAS_ASM_GOTO=y
+CONFIG_CC_HAS_ASM_INLINE=y
+CONFIG_CC_HAS_WARN_MAYBE_UNINITIALIZED=y
+CONFIG_IRQ_WORK=y
+CONFIG_BROKEN_ON_SMP=y
+CONFIG_INIT_ENV_ARG_LIMIT=32
+# CONFIG_COMPILE_TEST is not set
+CONFIG_LOCALVERSION=""
+# CONFIG_LOCALVERSION_AUTO is not set
+CONFIG_BUILD_SALT=""
+CONFIG_DEFAULT_HOSTNAME="(none)"
+# CONFIG_SYSVIPC is not set
+# CONFIG_POSIX_MQUEUE is not set
+# CONFIG_USELIB is not set
+# CONFIG_AUDIT is not set
+CONFIG_GENERIC_CLOCKEVENTS=y
+CONFIG_TICK_ONESHOT=y
+CONFIG_NO_HZ_COMMON=y
+# CONFIG_HZ_PERIODIC is not set
+CONFIG_NO_HZ_IDLE=y
+# CONFIG_NO_HZ is not set
+CONFIG_HIGH_RES_TIMERS=y
+CONFIG_PREEMPT_NONE=y
+# CONFIG_PREEMPT_VOLUNTARY is not set
+# CONFIG_PREEMPT is not set
+CONFIG_TICK_CPU_ACCOUNTING=y
+# CONFIG_BSD_PROCESS_ACCT is not set
+# CONFIG_TASKSTATS is not set
+# CONFIG_PSI is not set
+CONFIG_TINY_RCU=y
+# CONFIG_RCU_EXPERT is not set
+CONFIG_SRCU=y
+CONFIG_TINY_SRCU=y
+# CONFIG_IKCONFIG is not set
+# CONFIG_IKHEADERS is not set
+CONFIG_LOG_BUF_SHIFT=17
+CONFIG_PRINTK_SAFE_LOG_BUF_SHIFT=13
+CONFIG_CC_HAS_INT128=y
+# CONFIG_CGROUPS is not set
+# CONFIG_NAMESPACES is not set
+# CONFIG_CHECKPOINT_RESTORE is not set
+# CONFIG_SCHED_AUTOGROUP is not set
+# CONFIG_SYSFS_DEPRECATED is not set
+# CONFIG_RELAY is not set
+# CONFIG_BLK_DEV_INITRD is not set
+CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE=y
+# CONFIG_CC_OPTIMIZE_FOR_SIZE is not set
+CONFIG_SYSCTL=y
+CONFIG_BPF=y
+CONFIG_EXPERT=y
+CONFIG_MULTIUSER=y
+# CONFIG_SGETMASK_SYSCALL is not set
+# CONFIG_SYSFS_SYSCALL is not set
+CONFIG_FHANDLE=y
+CONFIG_POSIX_TIMERS=y
+# CONFIG_KALLSYMS_USE_DATA_SECTION is not set
+CONFIG_PRINTK=y
+CONFIG_BUG=y
+# CONFIG_BASE_FULL is not set
+# CONFIG_FUTEX is not set
+CONFIG_EPOLL=y
+# CONFIG_SIGNALFD is not set
+# CONFIG_TIMERFD is not set
+CONFIG_EVENTFD=y
+# CONFIG_AIO is not set
+CONFIG_IO_URING=y
+# CONFIG_ADVISE_SYSCALLS is not set
+CONFIG_MEMBARRIER=y
+CONFIG_KALLSYMS=y
+CONFIG_KALLSYMS_ALL=y
+CONFIG_KALLSYMS_BASE_RELATIVE=y
+# CONFIG_BPF_SYSCALL is not set
+CONFIG_EMBEDDED=y
+# CONFIG_PC104 is not set
+# CONFIG_VM_EVENT_COUNTERS is not set
+CONFIG_SLUB_DEBUG=y
+# CONFIG_COMPAT_BRK is not set
+# CONFIG_SLAB is not set
+CONFIG_SLUB=y
+# CONFIG_SLOB is not set
+CONFIG_SLAB_MERGE_DEFAULT=y
+# CONFIG_SLAB_FREELIST_RANDOM is not set
+# CONFIG_SLAB_FREELIST_HARDENED is not set
+# CONFIG_SHUFFLE_PAGE_ALLOCATOR is not set
+# CONFIG_MMAP_ALLOW_UNINITIALIZED is not set
+# CONFIG_PROFILING is not set
+CONFIG_LKL=y
+# CONFIG_COREDUMP is not set
+CONFIG_GENERIC_CSUM=y
+CONFIG_GENERIC_HWEIGHT=y
+CONFIG_NO_IOPORT_MAP=y
+CONFIG_RWSEM_GENERIC_SPINLOCK=y
+CONFIG_HZ=100
+# CONFIG_STDERR_CONSOLE is not set
+# CONFIG_SSL is not set
+# CONFIG_NULL_CHAN is not set
+# CONFIG_PORT_CHAN is not set
+# CONFIG_PTY_CHAN is not set
+# CONFIG_TTY_CHAN is not set
+# CONFIG_XTERM_CHAN is not set
+CONFIG_NOCONFIG_CHAN=y
+CONFIG_CON_ZERO_CHAN="fd:0,fd:1"
+CONFIG_CON_CHAN="xterm"
+CONFIG_SSL_CHAN="pty"
+# CONFIG_UML_SOUND is not set
+# CONFIG_SOUND is not set
+CONFIG_UML_NET=y
+# CONFIG_UML_NET_ETHERTAP is not set
+CONFIG_UML_NET_TUNTAP=y
+# CONFIG_UML_NET_SLIP is not set
+# CONFIG_UML_NET_DAEMON is not set
+# CONFIG_UML_NET_VECTOR is not set
+# CONFIG_UML_NET_VDE is not set
+# CONFIG_UML_NET_MCAST is not set
+# CONFIG_UML_NET_PCAP is not set
+CONFIG_UML_NET_SLIRP=y
+# CONFIG_VIRTIO_UML is not set
+CONFIG_ARCH_THREAD_STACK_ALLOCATOR=y
+CONFIG_CC_HAS_STACKPROTECTOR_NONE=y
+CONFIG_HAVE_VIRT_CPU_ACCOUNTING_GEN=y
+CONFIG_PGTABLE_LEVELS=2
+CONFIG_PLUGIN_HOSTCC=""
+CONFIG_BASE_SMALL=1
+# CONFIG_MODULES is not set
+CONFIG_BLOCK=y
+# CONFIG_BLK_DEV_BSG is not set
+# CONFIG_BLK_DEV_BSGLIB is not set
+# CONFIG_BLK_DEV_INTEGRITY is not set
+# CONFIG_BLK_DEV_ZONED is not set
+# CONFIG_BLK_CMDLINE_PARSER is not set
+# CONFIG_BLK_WBT is not set
+# CONFIG_BLK_SED_OPAL is not set
+# CONFIG_PARTITION_ADVANCED is not set
+CONFIG_MSDOS_PARTITION=y
+CONFIG_EFI_PARTITION=y
+CONFIG_MQ_IOSCHED_DEADLINE=y
+CONFIG_MQ_IOSCHED_KYBER=y
+# CONFIG_IOSCHED_BFQ is not set
+CONFIG_INLINE_SPIN_UNLOCK_IRQ=y
+CONFIG_INLINE_READ_UNLOCK=y
+CONFIG_INLINE_READ_UNLOCK_IRQ=y
+CONFIG_INLINE_WRITE_UNLOCK=y
+CONFIG_INLINE_WRITE_UNLOCK_IRQ=y
+CONFIG_BINFMT_SCRIPT=y
+# CONFIG_BINFMT_MISC is not set
+CONFIG_FLATMEM=y
+CONFIG_FLAT_NODE_MEM_MAP=y
+CONFIG_SPLIT_PTLOCK_CPUS=999999
+CONFIG_PHYS_ADDR_T_64BIT=y
+CONFIG_NOMMU_INITIAL_TRIM_EXCESS=1
+CONFIG_NEED_PER_CPU_KM=y
+# CONFIG_CLEANCACHE is not set
+# CONFIG_ZPOOL is not set
+# CONFIG_ZBUD is not set
+# CONFIG_PERCPU_STATS is not set
+# CONFIG_GUP_BENCHMARK is not set
+CONFIG_NET=y
+# CONFIG_PACKET is not set
+# CONFIG_UNIX is not set
+# CONFIG_TLS is not set
+# CONFIG_XFRM_USER is not set
+# CONFIG_NET_KEY is not set
+CONFIG_INET=y
+CONFIG_IP_MULTICAST=y
+CONFIG_IP_ADVANCED_ROUTER=y
+# CONFIG_IP_FIB_TRIE_STATS is not set
+CONFIG_IP_MULTIPLE_TABLES=y
+CONFIG_IP_ROUTE_MULTIPATH=y
+# CONFIG_IP_ROUTE_VERBOSE is not set
+CONFIG_IP_PNP=y
+CONFIG_IP_PNP_DHCP=y
+# CONFIG_IP_PNP_BOOTP is not set
+# CONFIG_IP_PNP_RARP is not set
+# CONFIG_NET_IPIP is not set
+# CONFIG_NET_IPGRE_DEMUX is not set
+CONFIG_NET_IP_TUNNEL=y
+# CONFIG_IP_MROUTE is not set
+# CONFIG_SYN_COOKIES is not set
+# CONFIG_NET_IPVTI is not set
+# CONFIG_NET_FOU is not set
+# CONFIG_NET_FOU_IP_TUNNELS is not set
+# CONFIG_INET_AH is not set
+# CONFIG_INET_ESP is not set
+# CONFIG_INET_IPCOMP is not set
+CONFIG_INET_TUNNEL=y
+CONFIG_INET_DIAG=y
+CONFIG_INET_TCP_DIAG=y
+# CONFIG_INET_UDP_DIAG is not set
+# CONFIG_INET_RAW_DIAG is not set
+# CONFIG_INET_DIAG_DESTROY is not set
+CONFIG_TCP_CONG_ADVANCED=y
+CONFIG_TCP_CONG_BIC=y
+CONFIG_TCP_CONG_CUBIC=y
+CONFIG_TCP_CONG_WESTWOOD=y
+CONFIG_TCP_CONG_HTCP=y
+# CONFIG_TCP_CONG_HSTCP is not set
+# CONFIG_TCP_CONG_HYBLA is not set
+# CONFIG_TCP_CONG_VEGAS is not set
+# CONFIG_TCP_CONG_NV is not set
+# CONFIG_TCP_CONG_SCALABLE is not set
+# CONFIG_TCP_CONG_LP is not set
+# CONFIG_TCP_CONG_VENO is not set
+# CONFIG_TCP_CONG_YEAH is not set
+# CONFIG_TCP_CONG_ILLINOIS is not set
+# CONFIG_TCP_CONG_DCTCP is not set
+# CONFIG_TCP_CONG_CDG is not set
+CONFIG_TCP_CONG_BBR=y
+# CONFIG_DEFAULT_BIC is not set
+CONFIG_DEFAULT_CUBIC=y
+# CONFIG_DEFAULT_HTCP is not set
+# CONFIG_DEFAULT_WESTWOOD is not set
+# CONFIG_DEFAULT_BBR is not set
+# CONFIG_DEFAULT_RENO is not set
+CONFIG_DEFAULT_TCP_CONG="cubic"
+# CONFIG_TCP_MD5SIG is not set
+CONFIG_IPV6=y
+# CONFIG_IPV6_ROUTER_PREF is not set
+# CONFIG_IPV6_OPTIMISTIC_DAD is not set
+# CONFIG_INET6_AH is not set
+# CONFIG_INET6_ESP is not set
+# CONFIG_INET6_IPCOMP is not set
+# CONFIG_IPV6_MIP6 is not set
+# CONFIG_IPV6_VTI is not set
+CONFIG_IPV6_SIT=y
+# CONFIG_IPV6_SIT_6RD is not set
+CONFIG_IPV6_NDISC_NODETYPE=y
+# CONFIG_IPV6_TUNNEL is not set
+CONFIG_IPV6_MULTIPLE_TABLES=y
+# CONFIG_IPV6_SUBTREES is not set
+# CONFIG_IPV6_MROUTE is not set
+# CONFIG_IPV6_SEG6_LWTUNNEL is not set
+# CONFIG_IPV6_SEG6_HMAC is not set
+# CONFIG_NETWORK_SECMARK is not set
+# CONFIG_NETWORK_PHY_TIMESTAMPING is not set
+# CONFIG_NETFILTER is not set
+# CONFIG_BPFILTER is not set
+# CONFIG_IP_DCCP is not set
+# CONFIG_IP_SCTP is not set
+# CONFIG_RDS is not set
+# CONFIG_TIPC is not set
+# CONFIG_ATM is not set
+# CONFIG_L2TP is not set
+# CONFIG_BRIDGE is not set
+CONFIG_HAVE_NET_DSA=y
+# CONFIG_NET_DSA is not set
+# CONFIG_VLAN_8021Q is not set
+# CONFIG_DECNET is not set
+# CONFIG_LLC2 is not set
+# CONFIG_ATALK is not set
+# CONFIG_X25 is not set
+# CONFIG_LAPB is not set
+# CONFIG_PHONET is not set
+# CONFIG_6LOWPAN is not set
+# CONFIG_IEEE802154 is not set
+CONFIG_NET_SCHED=y
+# CONFIG_NET_SCH_CBQ is not set
+# CONFIG_NET_SCH_HTB is not set
+# CONFIG_NET_SCH_HFSC is not set
+# CONFIG_NET_SCH_PRIO is not set
+# CONFIG_NET_SCH_MULTIQ is not set
+# CONFIG_NET_SCH_RED is not set
+# CONFIG_NET_SCH_SFB is not set
+# CONFIG_NET_SCH_SFQ is not set
+# CONFIG_NET_SCH_TEQL is not set
+# CONFIG_NET_SCH_TBF is not set
+# CONFIG_NET_SCH_CBS is not set
+# CONFIG_NET_SCH_ETF is not set
+# CONFIG_NET_SCH_TAPRIO is not set
+# CONFIG_NET_SCH_GRED is not set
+# CONFIG_NET_SCH_DSMARK is not set
+# CONFIG_NET_SCH_NETEM is not set
+# CONFIG_NET_SCH_DRR is not set
+# CONFIG_NET_SCH_MQPRIO is not set
+# CONFIG_NET_SCH_SKBPRIO is not set
+# CONFIG_NET_SCH_CHOKE is not set
+# CONFIG_NET_SCH_QFQ is not set
+# CONFIG_NET_SCH_CODEL is not set
+# CONFIG_NET_SCH_FQ_CODEL is not set
+# CONFIG_NET_SCH_CAKE is not set
+CONFIG_NET_SCH_FQ=y
+# CONFIG_NET_SCH_HHF is not set
+# CONFIG_NET_SCH_PIE is not set
+# CONFIG_NET_SCH_PLUG is not set
+# CONFIG_NET_SCH_DEFAULT is not set
+# CONFIG_NET_CLS_BASIC is not set
+# CONFIG_NET_CLS_TCINDEX is not set
+# CONFIG_NET_CLS_ROUTE4 is not set
+# CONFIG_NET_CLS_FW is not set
+# CONFIG_NET_CLS_U32 is not set
+# CONFIG_NET_CLS_RSVP is not set
+# CONFIG_NET_CLS_RSVP6 is not set
+# CONFIG_NET_CLS_FLOW is not set
+# CONFIG_NET_CLS_BPF is not set
+# CONFIG_NET_CLS_FLOWER is not set
+# CONFIG_NET_CLS_MATCHALL is not set
+# CONFIG_NET_EMATCH is not set
+# CONFIG_NET_CLS_ACT is not set
+CONFIG_NET_SCH_FIFO=y
+# CONFIG_DCB is not set
+# CONFIG_BATMAN_ADV is not set
+# CONFIG_OPENVSWITCH is not set
+# CONFIG_VSOCKETS is not set
+# CONFIG_NETLINK_DIAG is not set
+# CONFIG_MPLS is not set
+# CONFIG_NET_NSH is not set
+# CONFIG_HSR is not set
+# CONFIG_NET_SWITCHDEV is not set
+# CONFIG_NET_L3_MASTER_DEV is not set
+# CONFIG_NET_NCSI is not set
+CONFIG_NET_RX_BUSY_POLL=y
+CONFIG_BQL=y
+# CONFIG_NET_PKTGEN is not set
+# CONFIG_HAMRADIO is not set
+# CONFIG_CAN is not set
+# CONFIG_BT is not set
+# CONFIG_AF_RXRPC is not set
+# CONFIG_AF_KCM is not set
+CONFIG_FIB_RULES=y
+# CONFIG_WIRELESS is not set
+# CONFIG_WIMAX is not set
+# CONFIG_RFKILL is not set
+# CONFIG_NET_9P is not set
+# CONFIG_CAIF is not set
+# CONFIG_CEPH_LIB is not set
+# CONFIG_NFC is not set
+# CONFIG_PSAMPLE is not set
+# CONFIG_NET_IFE is not set
+# CONFIG_LWTUNNEL is not set
+CONFIG_DST_CACHE=y
+CONFIG_GRO_CELLS=y
+CONFIG_FAILOVER=y
+# CONFIG_PCCARD is not set
+# CONFIG_UEVENT_HELPER is not set
+# CONFIG_DEVTMPFS is not set
+CONFIG_STANDALONE=y
+CONFIG_PREVENT_FIRMWARE_BUILD=y
+# CONFIG_FW_LOADER is not set
+CONFIG_ALLOW_DEV_COREDUMP=y
+# CONFIG_DEBUG_DRIVER is not set
+# CONFIG_DEBUG_DEVRES is not set
+# CONFIG_DEBUG_TEST_DRIVER_REMOVE is not set
+CONFIG_GENERIC_CPU_DEVICES=y
+# CONFIG_CONNECTOR is not set
+# CONFIG_GNSS is not set
+# CONFIG_MTD is not set
+# CONFIG_OF is not set
+# CONFIG_PARPORT is not set
+CONFIG_BLK_DEV=y
+# CONFIG_BLK_DEV_NULL_BLK is not set
+# CONFIG_BLK_DEV_UBD_SYNC is not set
+CONFIG_BLK_DEV_COW_COMMON=y
+# CONFIG_BLK_DEV_LOOP is not set
+# CONFIG_BLK_DEV_DRBD is not set
+# CONFIG_BLK_DEV_NBD is not set
+# CONFIG_BLK_DEV_RAM is not set
+# CONFIG_CDROM_PKTCDVD is not set
+# CONFIG_ATA_OVER_ETH is not set
+# CONFIG_BLK_DEV_RBD is not set
+# CONFIG_NVME_FC is not set
+# CONFIG_DUMMY_IRQ is not set
+# CONFIG_ENCLOSURE_SERVICES is not set
+# CONFIG_SRAM is not set
+# CONFIG_XILINX_SDFEC is not set
+# CONFIG_C2PORT is not set
+# CONFIG_EEPROM_93CX6 is not set
+# CONFIG_VOP_BUS is not set
+# CONFIG_ECHO is not set
+CONFIG_SCSI_MOD=y
+# CONFIG_RAID_ATTRS is not set
+# CONFIG_SCSI is not set
+# CONFIG_ATA is not set
+# CONFIG_MD is not set
+# CONFIG_TARGET_CORE is not set
+CONFIG_NETDEVICES=y
+CONFIG_NET_CORE=y
+# CONFIG_BONDING is not set
+# CONFIG_DUMMY is not set
+# CONFIG_EQUALIZER is not set
+# CONFIG_NET_TEAM is not set
+# CONFIG_MACVLAN is not set
+# CONFIG_IPVLAN is not set
+# CONFIG_VXLAN is not set
+# CONFIG_GENEVE is not set
+# CONFIG_GTP is not set
+# CONFIG_MACSEC is not set
+# CONFIG_NETCONSOLE is not set
+# CONFIG_TUN is not set
+# CONFIG_TUN_VNET_CROSS_LE is not set
+# CONFIG_VETH is not set
+# CONFIG_NLMON is not set
+# CONFIG_ETHERNET is not set
+# CONFIG_MDIO_DEVICE is not set
+# CONFIG_PHYLIB is not set
+# CONFIG_PPP is not set
+# CONFIG_SLIP is not set
+# CONFIG_WLAN is not set
+# CONFIG_WAN is not set
+CONFIG_NET_FAILOVER=y
+# CONFIG_ISDN is not set
+# CONFIG_NVM is not set
+CONFIG_INPUT=y
+# CONFIG_INPUT_FF_MEMLESS is not set
+# CONFIG_INPUT_POLLDEV is not set
+# CONFIG_INPUT_SPARSEKMAP is not set
+# CONFIG_INPUT_MATRIXKMAP is not set
+# CONFIG_INPUT_MOUSEDEV is not set
+# CONFIG_INPUT_JOYDEV is not set
+# CONFIG_INPUT_EVDEV is not set
+# CONFIG_INPUT_EVBUG is not set
+CONFIG_INPUT_KEYBOARD=y
+CONFIG_KEYBOARD_ATKBD=y
+# CONFIG_KEYBOARD_LKKBD is not set
+# CONFIG_KEYBOARD_NEWTON is not set
+# CONFIG_KEYBOARD_OPENCORES is not set
+# CONFIG_KEYBOARD_STOWAWAY is not set
+# CONFIG_KEYBOARD_SUNKBD is not set
+# CONFIG_KEYBOARD_XTKBD is not set
+CONFIG_INPUT_MOUSE=y
+CONFIG_MOUSE_PS2=y
+CONFIG_MOUSE_PS2_ALPS=y
+CONFIG_MOUSE_PS2_BYD=y
+CONFIG_MOUSE_PS2_LOGIPS2PP=y
+CONFIG_MOUSE_PS2_SYNAPTICS=y
+CONFIG_MOUSE_PS2_CYPRESS=y
+CONFIG_MOUSE_PS2_TRACKPOINT=y
+# CONFIG_MOUSE_PS2_ELANTECH is not set
+# CONFIG_MOUSE_PS2_SENTELIC is not set
+# CONFIG_MOUSE_PS2_TOUCHKIT is not set
+CONFIG_MOUSE_PS2_FOCALTECH=y
+# CONFIG_MOUSE_SERIAL is not set
+# CONFIG_MOUSE_APPLETOUCH is not set
+# CONFIG_MOUSE_BCM5974 is not set
+# CONFIG_MOUSE_VSXXXAA is not set
+# CONFIG_MOUSE_SYNAPTICS_USB is not set
+# CONFIG_INPUT_JOYSTICK is not set
+# CONFIG_INPUT_TABLET is not set
+# CONFIG_INPUT_TOUCHSCREEN is not set
+# CONFIG_INPUT_MISC is not set
+# CONFIG_RMI4_CORE is not set
+CONFIG_SERIO=y
+CONFIG_SERIO_SERPORT=y
+CONFIG_SERIO_LIBPS2=y
+# CONFIG_SERIO_RAW is not set
+# CONFIG_SERIO_ALTERA_PS2 is not set
+# CONFIG_SERIO_PS2MULT is not set
+# CONFIG_SERIO_ARC_PS2 is not set
+# CONFIG_USERIO is not set
+# CONFIG_GAMEPORT is not set
+CONFIG_TTY=y
+# CONFIG_VT is not set
+CONFIG_UNIX98_PTYS=y
+CONFIG_LEGACY_PTYS=y
+CONFIG_LEGACY_PTY_COUNT=256
+# CONFIG_SERIAL_NONSTANDARD is not set
+# CONFIG_N_GSM is not set
+# CONFIG_TRACE_SINK is not set
+# CONFIG_NULL_TTY is not set
+CONFIG_LDISC_AUTOLOAD=y
+CONFIG_DEVMEM=y
+# CONFIG_DEVKMEM is not set
+# CONFIG_SERIAL_8250 is not set
+# CONFIG_SERIAL_UARTLITE is not set
+# CONFIG_SERIAL_SCCNXP is not set
+# CONFIG_SERIAL_ALTERA_JTAGUART is not set
+# CONFIG_SERIAL_ALTERA_UART is not set
+# CONFIG_SERIAL_ARC is not set
+# CONFIG_SERIAL_FSL_LPUART is not set
+# CONFIG_SERIAL_FSL_LINFLEXUART is not set
+# CONFIG_SERIAL_DEV_BUS is not set
+# CONFIG_TTY_PRINTK is not set
+# CONFIG_IPMI_HANDLER is not set
+CONFIG_HW_RANDOM=y
+# CONFIG_HW_RANDOM_TIMERIOMEM is not set
+# CONFIG_RAW_DRIVER is not set
+# CONFIG_TCG_TPM is not set
+# CONFIG_RANDOM_TRUST_BOOTLOADER is not set
+# CONFIG_I2C is not set
+# CONFIG_I3C is not set
+# CONFIG_SPI is not set
+# CONFIG_SPMI is not set
+# CONFIG_HSI is not set
+# CONFIG_PPS is not set
+# CONFIG_PTP_1588_CLOCK is not set
+# CONFIG_PINCTRL is not set
+# CONFIG_GPIOLIB is not set
+# CONFIG_W1 is not set
+# CONFIG_POWER_AVS is not set
+# CONFIG_POWER_RESET is not set
+# CONFIG_POWER_SUPPLY is not set
+CONFIG_HWMON=y
+# CONFIG_HWMON_DEBUG_CHIP is not set
+# CONFIG_SENSORS_AS370 is not set
+# CONFIG_SENSORS_ASPEED is not set
+# CONFIG_SENSORS_F71805F is not set
+# CONFIG_SENSORS_F71882FG is not set
+# CONFIG_SENSORS_IT87 is not set
+# CONFIG_SENSORS_MAX197 is not set
+# CONFIG_SENSORS_PC87360 is not set
+# CONFIG_SENSORS_PC87427 is not set
+# CONFIG_SENSORS_NTC_THERMISTOR is not set
+# CONFIG_SENSORS_NCT6683 is not set
+# CONFIG_SENSORS_NCT6775 is not set
+# CONFIG_SENSORS_NPCM7XX is not set
+# CONFIG_SENSORS_SMSC47M1 is not set
+# CONFIG_SENSORS_SMSC47B397 is not set
+# CONFIG_SENSORS_VT1211 is not set
+# CONFIG_SENSORS_W83627HF is not set
+# CONFIG_SENSORS_W83627EHF is not set
+# CONFIG_THERMAL is not set
+# CONFIG_WATCHDOG is not set
+CONFIG_SSB_POSSIBLE=y
+# CONFIG_SSB is not set
+CONFIG_BCMA_POSSIBLE=y
+# CONFIG_BCMA is not set
+# CONFIG_MFD_MADERA is not set
+# CONFIG_HTC_PASIC3 is not set
+# CONFIG_MFD_KEMPLD is not set
+# CONFIG_MFD_MT6397 is not set
+# CONFIG_MFD_SM501 is not set
+# CONFIG_ABX500_CORE is not set
+# CONFIG_MFD_SYSCON is not set
+# CONFIG_MFD_TI_AM335X_TSCADC is not set
+# CONFIG_MFD_TQMX86 is not set
+# CONFIG_REGULATOR is not set
+# CONFIG_RC_CORE is not set
+# CONFIG_MEDIA_SUPPORT is not set
+# CONFIG_DRM is not set
+# CONFIG_DRM_DP_CEC is not set
+# CONFIG_FB is not set
+# CONFIG_LCD_CLASS_DEVICE is not set
+# CONFIG_BACKLIGHT_CLASS_DEVICE is not set
+CONFIG_HID=y
+# CONFIG_HID_BATTERY_STRENGTH is not set
+# CONFIG_HIDRAW is not set
+# CONFIG_UHID is not set
+CONFIG_HID_GENERIC=y
+# CONFIG_HID_A4TECH is not set
+# CONFIG_HID_ACRUX is not set
+# CONFIG_HID_APPLE is not set
+# CONFIG_HID_AUREAL is not set
+# CONFIG_HID_BELKIN is not set
+# CONFIG_HID_CHERRY is not set
+# CONFIG_HID_CHICONY is not set
+# CONFIG_HID_COUGAR is not set
+# CONFIG_HID_MACALLY is not set
+# CONFIG_HID_CMEDIA is not set
+# CONFIG_HID_CYPRESS is not set
+# CONFIG_HID_DRAGONRISE is not set
+# CONFIG_HID_EMS_FF is not set
+# CONFIG_HID_ELECOM is not set
+# CONFIG_HID_EZKEY is not set
+# CONFIG_HID_GEMBIRD is not set
+# CONFIG_HID_GFRM is not set
+# CONFIG_HID_KEYTOUCH is not set
+# CONFIG_HID_KYE is not set
+# CONFIG_HID_WALTOP is not set
+# CONFIG_HID_VIEWSONIC is not set
+# CONFIG_HID_GYRATION is not set
+# CONFIG_HID_ICADE is not set
+# CONFIG_HID_ITE is not set
+# CONFIG_HID_JABRA is not set
+# CONFIG_HID_TWINHAN is not set
+# CONFIG_HID_KENSINGTON is not set
+# CONFIG_HID_LCPOWER is not set
+# CONFIG_HID_LENOVO is not set
+# CONFIG_HID_MAGICMOUSE is not set
+# CONFIG_HID_MALTRON is not set
+# CONFIG_HID_MAYFLASH is not set
+# CONFIG_HID_REDRAGON is not set
+# CONFIG_HID_MICROSOFT is not set
+# CONFIG_HID_MONTEREY is not set
+# CONFIG_HID_MULTITOUCH is not set
+# CONFIG_HID_NTI is not set
+# CONFIG_HID_ORTEK is not set
+# CONFIG_HID_PANTHERLORD is not set
+# CONFIG_HID_PETALYNX is not set
+# CONFIG_HID_PICOLCD is not set
+# CONFIG_HID_PLANTRONICS is not set
+# CONFIG_HID_PRIMAX is not set
+# CONFIG_HID_SAITEK is not set
+# CONFIG_HID_SAMSUNG is not set
+# CONFIG_HID_SPEEDLINK is not set
+# CONFIG_HID_STEAM is not set
+# CONFIG_HID_STEELSERIES is not set
+# CONFIG_HID_SUNPLUS is not set
+# CONFIG_HID_RMI is not set
+# CONFIG_HID_GREENASIA is not set
+# CONFIG_HID_SMARTJOYPLUS is not set
+# CONFIG_HID_TIVO is not set
+# CONFIG_HID_TOPSEED is not set
+# CONFIG_HID_THRUSTMASTER is not set
+# CONFIG_HID_UDRAW_PS3 is not set
+# CONFIG_HID_XINMO is not set
+# CONFIG_HID_ZEROPLUS is not set
+# CONFIG_HID_ZYDACRON is not set
+# CONFIG_HID_SENSOR_HUB is not set
+# CONFIG_HID_ALPS is not set
+CONFIG_USB_OHCI_LITTLE_ENDIAN=y
+CONFIG_USB_SUPPORT=y
+# CONFIG_USB_ULPI_BUS is not set
+CONFIG_USB_ARCH_HAS_HCD=y
+# CONFIG_USB is not set
+# CONFIG_NOP_USB_XCEIV is not set
+# CONFIG_USB_GADGET is not set
+# CONFIG_TYPEC is not set
+# CONFIG_USB_ROLE_SWITCH is not set
+# CONFIG_MMC is not set
+# CONFIG_MEMSTICK is not set
+# CONFIG_NEW_LEDS is not set
+# CONFIG_ACCESSIBILITY is not set
+# CONFIG_INFINIBAND is not set
+# CONFIG_RTC_CLASS is not set
+# CONFIG_DMADEVICES is not set
+# CONFIG_SYNC_FILE is not set
+# CONFIG_AUXDISPLAY is not set
+# CONFIG_VIRT_DRIVERS is not set
+CONFIG_VIRTIO_MENU=y
+# CONFIG_VIRTIO_MMIO is not set
+# CONFIG_GREYBUS is not set
+# CONFIG_STAGING is not set
+# CONFIG_GOLDFISH is not set
+# CONFIG_HWSPINLOCK is not set
+# CONFIG_MAILBOX is not set
+# CONFIG_REMOTEPROC is not set
+# CONFIG_RPMSG_VIRTIO is not set
+# CONFIG_SOC_TI is not set
+# CONFIG_XILINX_VCU is not set
+# CONFIG_PM_DEVFREQ is not set
+# CONFIG_EXTCON is not set
+# CONFIG_MEMORY is not set
+# CONFIG_IIO is not set
+# CONFIG_PWM is not set
+# CONFIG_IPACK_BUS is not set
+# CONFIG_RESET_CONTROLLER is not set
+# CONFIG_GENERIC_PHY is not set
+# CONFIG_BCM_KONA_USB2_PHY is not set
+# CONFIG_PHY_PXA_28NM_HSIC is not set
+# CONFIG_PHY_PXA_28NM_USB2 is not set
+# CONFIG_POWERCAP is not set
+# CONFIG_MCB is not set
+# CONFIG_RAS is not set
+# CONFIG_ANDROID is not set
+# CONFIG_LIBNVDIMM is not set
+# CONFIG_DAX is not set
+# CONFIG_NVMEM is not set
+# CONFIG_STM is not set
+# CONFIG_INTEL_TH is not set
+# CONFIG_FPGA is not set
+# CONFIG_SIOX is not set
+# CONFIG_SLIMBUS is not set
+# CONFIG_INTERCONNECT is not set
+# CONFIG_COUNTER is not set
+# CONFIG_VALIDATE_FS_PARSER is not set
+CONFIG_FS_IOMAP=y
+# CONFIG_EXT2_FS is not set
+# CONFIG_EXT3_FS is not set
+CONFIG_EXT4_FS=y
+CONFIG_EXT4_USE_FOR_EXT2=y
+CONFIG_EXT4_FS_POSIX_ACL=y
+CONFIG_EXT4_FS_SECURITY=y
+# CONFIG_EXT4_DEBUG is not set
+CONFIG_JBD2=y
+# CONFIG_JBD2_DEBUG is not set
+CONFIG_FS_MBCACHE=y
+# CONFIG_REISERFS_FS is not set
+# CONFIG_JFS_FS is not set
+# CONFIG_XFS_FS is not set
+# CONFIG_GFS2_FS is not set
+CONFIG_BTRFS_FS=y
+# CONFIG_BTRFS_FS_POSIX_ACL is not set
+# CONFIG_BTRFS_FS_CHECK_INTEGRITY is not set
+# CONFIG_BTRFS_FS_RUN_SANITY_TESTS is not set
+# CONFIG_BTRFS_DEBUG is not set
+# CONFIG_BTRFS_ASSERT is not set
+# CONFIG_BTRFS_FS_REF_VERIFY is not set
+# CONFIG_NILFS2_FS is not set
+# CONFIG_F2FS_FS is not set
+CONFIG_FS_POSIX_ACL=y
+CONFIG_EXPORTFS=y
+# CONFIG_EXPORTFS_BLOCK_OPS is not set
+# CONFIG_FILE_LOCKING is not set
+# CONFIG_FS_ENCRYPTION is not set
+# CONFIG_FS_VERITY is not set
+# CONFIG_DNOTIFY is not set
+# CONFIG_INOTIFY_USER is not set
+# CONFIG_FANOTIFY is not set
+# CONFIG_QUOTA is not set
+# CONFIG_AUTOFS4_FS is not set
+# CONFIG_AUTOFS_FS is not set
+# CONFIG_FUSE_FS is not set
+# CONFIG_OVERLAY_FS is not set
+# CONFIG_FSCACHE is not set
+# CONFIG_ISO9660_FS is not set
+# CONFIG_UDF_FS is not set
+CONFIG_FAT_FS=y
+# CONFIG_MSDOS_FS is not set
+CONFIG_VFAT_FS=y
+CONFIG_FAT_DEFAULT_CODEPAGE=437
+CONFIG_FAT_DEFAULT_IOCHARSET="iso8859-1"
+# CONFIG_FAT_DEFAULT_UTF8 is not set
+# CONFIG_NTFS_FS is not set
+CONFIG_PROC_FS=y
+CONFIG_PROC_SYSCTL=y
+# CONFIG_PROC_CHILDREN is not set
+CONFIG_KERNFS=y
+CONFIG_SYSFS=y
+# CONFIG_CONFIGFS_FS is not set
+CONFIG_MISC_FILESYSTEMS=y
+# CONFIG_ORANGEFS_FS is not set
+# CONFIG_ADFS_FS is not set
+# CONFIG_AFFS_FS is not set
+# CONFIG_HFS_FS is not set
+# CONFIG_HFSPLUS_FS is not set
+# CONFIG_BEFS_FS is not set
+# CONFIG_BFS_FS is not set
+# CONFIG_EFS_FS is not set
+# CONFIG_CRAMFS is not set
+# CONFIG_SQUASHFS is not set
+# CONFIG_VXFS_FS is not set
+# CONFIG_MINIX_FS is not set
+# CONFIG_OMFS_FS is not set
+# CONFIG_HPFS_FS is not set
+# CONFIG_QNX4FS_FS is not set
+# CONFIG_QNX6FS_FS is not set
+# CONFIG_ROMFS_FS is not set
+# CONFIG_PSTORE is not set
+# CONFIG_SYSV_FS is not set
+# CONFIG_UFS_FS is not set
+# CONFIG_EROFS_FS is not set
+CONFIG_NETWORK_FILESYSTEMS=y
+# CONFIG_CEPH_FS is not set
+# CONFIG_CIFS is not set
+# CONFIG_CODA_FS is not set
+# CONFIG_AFS_FS is not set
+CONFIG_NLS=y
+CONFIG_NLS_DEFAULT="iso8859-1"
+CONFIG_NLS_CODEPAGE_437=y
+CONFIG_NLS_CODEPAGE_737=y
+CONFIG_NLS_CODEPAGE_775=y
+CONFIG_NLS_CODEPAGE_850=y
+CONFIG_NLS_CODEPAGE_852=y
+CONFIG_NLS_CODEPAGE_855=y
+CONFIG_NLS_CODEPAGE_857=y
+CONFIG_NLS_CODEPAGE_860=y
+CONFIG_NLS_CODEPAGE_861=y
+CONFIG_NLS_CODEPAGE_862=y
+CONFIG_NLS_CODEPAGE_863=y
+CONFIG_NLS_CODEPAGE_864=y
+CONFIG_NLS_CODEPAGE_865=y
+CONFIG_NLS_CODEPAGE_866=y
+CONFIG_NLS_CODEPAGE_869=y
+CONFIG_NLS_CODEPAGE_936=y
+CONFIG_NLS_CODEPAGE_950=y
+CONFIG_NLS_CODEPAGE_932=y
+CONFIG_NLS_CODEPAGE_949=y
+CONFIG_NLS_CODEPAGE_874=y
+CONFIG_NLS_ISO8859_8=y
+CONFIG_NLS_CODEPAGE_1250=y
+CONFIG_NLS_CODEPAGE_1251=y
+CONFIG_NLS_ASCII=y
+CONFIG_NLS_ISO8859_1=y
+CONFIG_NLS_ISO8859_2=y
+CONFIG_NLS_ISO8859_3=y
+CONFIG_NLS_ISO8859_4=y
+CONFIG_NLS_ISO8859_5=y
+CONFIG_NLS_ISO8859_6=y
+CONFIG_NLS_ISO8859_7=y
+CONFIG_NLS_ISO8859_9=y
+CONFIG_NLS_ISO8859_13=y
+CONFIG_NLS_ISO8859_14=y
+CONFIG_NLS_ISO8859_15=y
+CONFIG_NLS_KOI8_R=y
+CONFIG_NLS_KOI8_U=y
+CONFIG_NLS_MAC_ROMAN=y
+CONFIG_NLS_MAC_CELTIC=y
+CONFIG_NLS_MAC_CENTEURO=y
+CONFIG_NLS_MAC_CROATIAN=y
+CONFIG_NLS_MAC_CYRILLIC=y
+CONFIG_NLS_MAC_GAELIC=y
+CONFIG_NLS_MAC_GREEK=y
+CONFIG_NLS_MAC_ICELAND=y
+CONFIG_NLS_MAC_INUIT=y
+CONFIG_NLS_MAC_ROMANIAN=y
+CONFIG_NLS_MAC_TURKISH=y
+CONFIG_NLS_UTF8=y
+# CONFIG_UNICODE is not set
+CONFIG_IO_WQ=y
+# CONFIG_KEYS is not set
+# CONFIG_SECURITY_DMESG_RESTRICT is not set
+# CONFIG_SECURITY is not set
+# CONFIG_SECURITYFS is not set
+CONFIG_HAVE_HARDENED_USERCOPY_ALLOCATOR=y
+# CONFIG_HARDENED_USERCOPY is not set
+# CONFIG_STATIC_USERMODEHELPER is not set
+CONFIG_DEFAULT_SECURITY_DAC=y
+CONFIG_LSM="lockdown,yama,loadpin,safesetid,integrity"
+CONFIG_INIT_STACK_NONE=y
+# CONFIG_INIT_ON_ALLOC_DEFAULT_ON is not set
+# CONFIG_INIT_ON_FREE_DEFAULT_ON is not set
+CONFIG_XOR_BLOCKS=y
+CONFIG_CRYPTO=y
+# CONFIG_CRYPTO_FIPS is not set
+CONFIG_CRYPTO_ALGAPI=y
+CONFIG_CRYPTO_ALGAPI2=y
+CONFIG_CRYPTO_HASH=y
+CONFIG_CRYPTO_HASH2=y
+CONFIG_CRYPTO_RNG=y
+CONFIG_CRYPTO_RNG2=y
+# CONFIG_CRYPTO_MANAGER is not set
+# CONFIG_CRYPTO_USER is not set
+# CONFIG_CRYPTO_NULL is not set
+# CONFIG_CRYPTO_CRYPTD is not set
+# CONFIG_CRYPTO_AUTHENC is not set
+# CONFIG_CRYPTO_RSA is not set
+# CONFIG_CRYPTO_DH is not set
+# CONFIG_CRYPTO_ECDH is not set
+# CONFIG_CRYPTO_ECRDSA is not set
+# CONFIG_CRYPTO_CURVE25519 is not set
+# CONFIG_CRYPTO_CCM is not set
+# CONFIG_CRYPTO_GCM is not set
+# CONFIG_CRYPTO_CHACHA20POLY1305 is not set
+# CONFIG_CRYPTO_AEGIS128 is not set
+# CONFIG_CRYPTO_SEQIV is not set
+# CONFIG_CRYPTO_ECHAINIV is not set
+# CONFIG_CRYPTO_CBC is not set
+# CONFIG_CRYPTO_CFB is not set
+# CONFIG_CRYPTO_CTR is not set
+# CONFIG_CRYPTO_CTS is not set
+# CONFIG_CRYPTO_ECB is not set
+# CONFIG_CRYPTO_LRW is not set
+# CONFIG_CRYPTO_OFB is not set
+# CONFIG_CRYPTO_PCBC is not set
+# CONFIG_CRYPTO_XTS is not set
+# CONFIG_CRYPTO_KEYWRAP is not set
+# CONFIG_CRYPTO_ADIANTUM is not set
+# CONFIG_CRYPTO_ESSIV is not set
+# CONFIG_CRYPTO_CMAC is not set
+# CONFIG_CRYPTO_HMAC is not set
+# CONFIG_CRYPTO_XCBC is not set
+# CONFIG_CRYPTO_VMAC is not set
+CONFIG_CRYPTO_CRC32C=y
+# CONFIG_CRYPTO_CRC32 is not set
+CONFIG_CRYPTO_XXHASH=y
+CONFIG_CRYPTO_BLAKE2B=y
+# CONFIG_CRYPTO_BLAKE2S is not set
+# CONFIG_CRYPTO_CRCT10DIF is not set
+# CONFIG_CRYPTO_GHASH is not set
+# CONFIG_CRYPTO_POLY1305 is not set
+# CONFIG_CRYPTO_MD4 is not set
+# CONFIG_CRYPTO_MD5 is not set
+# CONFIG_CRYPTO_MICHAEL_MIC is not set
+# CONFIG_CRYPTO_RMD128 is not set
+# CONFIG_CRYPTO_RMD160 is not set
+# CONFIG_CRYPTO_RMD256 is not set
+# CONFIG_CRYPTO_RMD320 is not set
+# CONFIG_CRYPTO_SHA1 is not set
+CONFIG_CRYPTO_SHA256=y
+# CONFIG_CRYPTO_SHA512 is not set
+# CONFIG_CRYPTO_SHA3 is not set
+# CONFIG_CRYPTO_SM3 is not set
+# CONFIG_CRYPTO_STREEBOG is not set
+# CONFIG_CRYPTO_TGR192 is not set
+# CONFIG_CRYPTO_WP512 is not set
+CONFIG_CRYPTO_AES=y
+# CONFIG_CRYPTO_AES_TI is not set
+# CONFIG_CRYPTO_ANUBIS is not set
+# CONFIG_CRYPTO_ARC4 is not set
+# CONFIG_CRYPTO_BLOWFISH is not set
+# CONFIG_CRYPTO_CAMELLIA is not set
+# CONFIG_CRYPTO_CAST5 is not set
+# CONFIG_CRYPTO_CAST6 is not set
+# CONFIG_CRYPTO_DES is not set
+# CONFIG_CRYPTO_FCRYPT is not set
+# CONFIG_CRYPTO_KHAZAD is not set
+# CONFIG_CRYPTO_SALSA20 is not set
+# CONFIG_CRYPTO_CHACHA20 is not set
+# CONFIG_CRYPTO_SEED is not set
+# CONFIG_CRYPTO_SERPENT is not set
+# CONFIG_CRYPTO_SM4 is not set
+# CONFIG_CRYPTO_TEA is not set
+# CONFIG_CRYPTO_TWOFISH is not set
+# CONFIG_CRYPTO_DEFLATE is not set
+# CONFIG_CRYPTO_LZO is not set
+# CONFIG_CRYPTO_842 is not set
+# CONFIG_CRYPTO_LZ4 is not set
+# CONFIG_CRYPTO_LZ4HC is not set
+# CONFIG_CRYPTO_ZSTD is not set
+CONFIG_CRYPTO_ANSI_CPRNG=y
+# CONFIG_CRYPTO_DRBG_MENU is not set
+# CONFIG_CRYPTO_JITTERENTROPY is not set
+# CONFIG_CRYPTO_USER_API_HASH is not set
+# CONFIG_CRYPTO_USER_API_SKCIPHER is not set
+# CONFIG_CRYPTO_USER_API_RNG is not set
+# CONFIG_CRYPTO_USER_API_AEAD is not set
+CONFIG_CRYPTO_LIB_AES=y
+# CONFIG_CRYPTO_LIB_BLAKE2S is not set
+# CONFIG_CRYPTO_LIB_CHACHA is not set
+# CONFIG_CRYPTO_LIB_CURVE25519 is not set
+CONFIG_CRYPTO_LIB_POLY1305_RSIZE=1
+# CONFIG_CRYPTO_LIB_POLY1305 is not set
+# CONFIG_CRYPTO_LIB_CHACHA20POLY1305 is not set
+CONFIG_CRYPTO_LIB_SHA256=y
+CONFIG_CRYPTO_HW=y
+# CONFIG_CRYPTO_DEV_AMLOGIC_GXL is not set
+CONFIG_RAID6_PQ=y
+# CONFIG_RAID6_PQ_BENCHMARK is not set
+# CONFIG_PACKING is not set
+CONFIG_BITREVERSE=y
+CONFIG_GENERIC_NET_UTILS=y
+# CONFIG_CORDIC is not set
+# CONFIG_CRC_CCITT is not set
+CONFIG_CRC16=y
+# CONFIG_CRC_T10DIF is not set
+# CONFIG_CRC_ITU_T is not set
+CONFIG_CRC32=y
+# CONFIG_CRC32_SELFTEST is not set
+CONFIG_CRC32_SLICEBY8=y
+# CONFIG_CRC32_SLICEBY4 is not set
+# CONFIG_CRC32_SARWATE is not set
+# CONFIG_CRC32_BIT is not set
+# CONFIG_CRC64 is not set
+# CONFIG_CRC4 is not set
+# CONFIG_CRC7 is not set
+CONFIG_LIBCRC32C=y
+# CONFIG_CRC8 is not set
+CONFIG_XXHASH=y
+# CONFIG_RANDOM32_SELFTEST is not set
+CONFIG_ZLIB_INFLATE=y
+CONFIG_ZLIB_DEFLATE=y
+CONFIG_LZO_COMPRESS=y
+CONFIG_LZO_DECOMPRESS=y
+CONFIG_ZSTD_COMPRESS=y
+CONFIG_ZSTD_DECOMPRESS=y
+# CONFIG_XZ_DEC is not set
+CONFIG_HAS_IOMEM=y
+CONFIG_HAS_DMA=y
+# CONFIG_DMA_API_DEBUG is not set
+CONFIG_DQL=y
+CONFIG_NLATTR=y
+CONFIG_GENERIC_ATOMIC64=y
+# CONFIG_IRQ_POLL is not set
+CONFIG_SBITMAP=y
+# CONFIG_STRING_SELFTEST is not set
+CONFIG_PRINTK_TIME=y
+# CONFIG_PRINTK_CALLER is not set
+CONFIG_CONSOLE_LOGLEVEL_DEFAULT=7
+CONFIG_CONSOLE_LOGLEVEL_QUIET=4
+CONFIG_MESSAGE_LOGLEVEL_DEFAULT=4
+CONFIG_SYMBOLIC_ERRNAME=y
+CONFIG_DEBUG_INFO=y
+CONFIG_DEBUG_INFO_REDUCED=y
+# CONFIG_DEBUG_INFO_SPLIT is not set
+# CONFIG_DEBUG_INFO_DWARF4 is not set
+# CONFIG_DEBUG_INFO_BTF is not set
+# CONFIG_GDB_SCRIPTS is not set
+# CONFIG_ENABLE_MUST_CHECK is not set
+# CONFIG_STRIP_ASM_SYMS is not set
+# CONFIG_READABLE_ASM is not set
+# CONFIG_HEADERS_INSTALL is not set
+CONFIG_OPTIMIZE_INLINING=y
+# CONFIG_DEBUG_SECTION_MISMATCH is not set
+CONFIG_SECTION_MISMATCH_WARN_ONLY=y
+CONFIG_ARCH_WANT_FRAME_POINTERS=y
+CONFIG_FRAME_POINTER=y
+# CONFIG_DEBUG_FORCE_WEAK_PER_CPU is not set
+# CONFIG_MAGIC_SYSRQ is not set
+# CONFIG_DEBUG_FS is not set
+# CONFIG_UBSAN is not set
+CONFIG_UBSAN_ALIGNMENT=y
+CONFIG_DEBUG_KERNEL=y
+CONFIG_DEBUG_MISC=y
+# CONFIG_PAGE_EXTENSION is not set
+# CONFIG_DEBUG_PAGEALLOC is not set
+# CONFIG_PAGE_POISONING is not set
+# CONFIG_DEBUG_OBJECTS is not set
+# CONFIG_SLUB_DEBUG_ON is not set
+# CONFIG_SLUB_STATS is not set
+# CONFIG_DEBUG_STACK_USAGE is not set
+# CONFIG_SCHED_STACK_END_CHECK is not set
+# CONFIG_DEBUG_VM is not set
+# CONFIG_DEBUG_NOMMU_REGIONS is not set
+# CONFIG_DEBUG_MEMORY_INIT is not set
+CONFIG_CC_HAS_KASAN_GENERIC=y
+CONFIG_KASAN_STACK=1
+# CONFIG_DEBUG_SHIRQ is not set
+# CONFIG_PANIC_ON_OOPS is not set
+CONFIG_PANIC_ON_OOPS_VALUE=0
+CONFIG_PANIC_TIMEOUT=0
+# CONFIG_SOFTLOCKUP_DETECTOR is not set
+# CONFIG_DETECT_HUNG_TASK is not set
+# CONFIG_WQ_WATCHDOG is not set
+CONFIG_SCHED_DEBUG=y
+# CONFIG_SCHEDSTATS is not set
+# CONFIG_DEBUG_TIMEKEEPING is not set
+# CONFIG_DEBUG_SPINLOCK is not set
+# CONFIG_DEBUG_MUTEXES is not set
+# CONFIG_DEBUG_RWSEMS is not set
+# CONFIG_DEBUG_ATOMIC_SLEEP is not set
+# CONFIG_DEBUG_LOCKING_API_SELFTESTS is not set
+# CONFIG_LOCK_TORTURE_TEST is not set
+# CONFIG_WW_MUTEX_SELFTEST is not set
+# CONFIG_WARN_ALL_UNSEEDED_RANDOM is not set
+# CONFIG_DEBUG_KOBJECT is not set
+# CONFIG_DEBUG_LIST is not set
+# CONFIG_DEBUG_PLIST is not set
+# CONFIG_DEBUG_SG is not set
+# CONFIG_DEBUG_NOTIFIERS is not set
+# CONFIG_BUG_ON_DATA_CORRUPTION is not set
+# CONFIG_DEBUG_CREDENTIALS is not set
+# CONFIG_RCU_PERF_TEST is not set
+# CONFIG_RCU_TORTURE_TEST is not set
+# CONFIG_RCU_TRACE is not set
+# CONFIG_RCU_EQS_DEBUG is not set
+# CONFIG_DEBUG_WQ_FORCE_RR_CPU is not set
+# CONFIG_DEBUG_BLOCK_EXT_DEVT is not set
+# CONFIG_SAMPLES is not set
+# CONFIG_KUNIT is not set
+# CONFIG_NOTIFIER_ERROR_INJECTION is not set
+# CONFIG_FAULT_INJECTION is not set
+CONFIG_CC_HAS_SANCOV_TRACE_PC=y
+CONFIG_RUNTIME_TESTING_MENU=y
+# CONFIG_TEST_LIST_SORT is not set
+# CONFIG_TEST_SORT is not set
+# CONFIG_BACKTRACE_SELF_TEST is not set
+# CONFIG_RBTREE_TEST is not set
+# CONFIG_REED_SOLOMON_TEST is not set
+# CONFIG_INTERVAL_TREE_TEST is not set
+# CONFIG_ATOMIC64_SELFTEST is not set
+# CONFIG_TEST_HEXDUMP is not set
+# CONFIG_TEST_STRING_HELPERS is not set
+# CONFIG_TEST_STRSCPY is not set
+# CONFIG_TEST_KSTRTOX is not set
+# CONFIG_TEST_PRINTF is not set
+# CONFIG_TEST_BITMAP is not set
+# CONFIG_TEST_BITFIELD is not set
+# CONFIG_TEST_UUID is not set
+# CONFIG_TEST_XARRAY is not set
+# CONFIG_TEST_OVERFLOW is not set
+# CONFIG_TEST_RHASHTABLE is not set
+# CONFIG_TEST_HASH is not set
+# CONFIG_TEST_IDA is not set
+# CONFIG_FIND_BIT_BENCHMARK is not set
+# CONFIG_TEST_SYSCTL is not set
+# CONFIG_TEST_UDELAY is not set
+# CONFIG_TEST_MEMCAT_P is not set
+# CONFIG_TEST_STACKINIT is not set
+# CONFIG_TEST_MEMINIT is not set
+# CONFIG_MEMTEST is not set
diff --git a/arch/um/lkl/include/asm/Kbuild b/arch/um/lkl/include/asm/Kbuild
new file mode 100644
index 000000000000..6ae3104fa285
--- /dev/null
+++ b/arch/um/lkl/include/asm/Kbuild
@@ -0,0 +1,79 @@
+generic-y += barrier.h
+generic-y += bitops.h
+generic-y += bug.h
+generic-y += bugs.h
+generic-y += cache.h
+generic-y += cacheflush.h
+generic-y += checksum.h
+generic-y += cmpxchg-local.h
+generic-y += cmpxchg.h
+generic-y += compat.h
+generic-y += cputime.h
+generic-y += current.h
+generic-y += delay.h
+generic-y += device.h
+generic-y += div64.h
+generic-y += dma.h
+generic-y += dma-mapping.h
+generic-y += emergency-restart.h
+generic-y += errno.h
+generic-y += extable.h
+generic-y += exec.h
+generic-y += ftrace.h
+generic-y += futex.h
+generic-y += hardirq.h
+generic-y += hw_irq.h
+generic-y += io.h
+generic-y += ioctl.h
+generic-y += ipcbuf.h
+generic-y += irq_regs.h
+generic-y += irqflags.h
+generic-y += irq_work.h
+generic-y += kdebug.h
+generic-y += kmap_types.h
+generic-y += linkage.h
+generic-y += local.h
+generic-y += local64.h
+generic-y += mcs_spinlock.h
+generic-y += mmiowb.h
+generic-y += mmu.h
+generic-y += mmu_context.h
+generic-y += module.h
+generic-y += msgbuf.h
+generic-y += param.h
+generic-y += parport.h
+generic-y += pci.h
+generic-y += percpu.h
+generic-y += pgalloc.h
+generic-y += poll.h
+generic-y += preempt.h
+generic-y += resource.h
+generic-y += rwsem.h
+generic-y += scatterlist.h
+generic-y += seccomp.h
+generic-y += sections.h
+generic-y += segment.h
+generic-y += sembuf.h
+generic-y += serial.h
+generic-y += shmbuf.h
+generic-y += signal.h
+generic-y += sizes.h
+generic-y += socket.h
+generic-y += sockios.h
+generic-y += stat.h
+generic-y += statfs.h
+generic-y += string.h
+generic-y += swab.h
+generic-y += switch_to.h
+generic-y += syscall.h
+generic-y += termbits.h
+generic-y += termios.h
+generic-y += time.h
+generic-y += timex.h
+generic-y += tlbflush.h
+generic-y += topology.h
+generic-y += trace_clock.h
+generic-y += unaligned.h
+generic-y += user.h
+generic-y += word-at-a-time.h
+generic-y += kprobes.h
diff --git a/arch/um/lkl/include/asm/atomic.h b/arch/um/lkl/include/asm/atomic.h
new file mode 100644
index 000000000000..0c5f9f8e4b30
--- /dev/null
+++ b/arch/um/lkl/include/asm/atomic.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __LKL_ATOMIC_H
+#define __LKL_ATOMIC_H
+
+#include <asm-generic/atomic.h>
+
+#ifndef CONFIG_GENERIC_ATOMIC64
+#include "atomic64.h"
+#endif /* CONFIG_GENERIC_ATOMIC64 */
+
+#endif
diff --git a/arch/um/lkl/include/asm/atomic64.h b/arch/um/lkl/include/asm/atomic64.h
new file mode 100644
index 000000000000..87d7956c29eb
--- /dev/null
+++ b/arch/um/lkl/include/asm/atomic64.h
@@ -0,0 +1,114 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __LKL_ATOMIC64_H
+#define __LKL_ATOMIC64_H
+
+#include <linux/types.h>
+
+#ifdef CONFIG_SMP
+#error "SMP is not supported on this platform"
+#else
+#define ATOMIC64_OP(op, c_op)						\
+static inline void atomic64_##op(s64 i, atomic64_t *v)			\
+{									\
+	unsigned long flags;						\
+									\
+	raw_local_irq_save(flags);					\
+	v->counter = v->counter c_op i;					\
+	raw_local_irq_restore(flags);					\
+}
+
+#define ATOMIC64_OP_RETURN(op, c_op)					\
+static inline s64 atomic64_##op##_return(s64 i, atomic64_t *v)		\
+{									\
+	unsigned long flags;						\
+	s64 ret;							\
+									\
+	raw_local_irq_save(flags);					\
+	ret = (v->counter = v->counter c_op i);				\
+	raw_local_irq_restore(flags);					\
+									\
+	return ret;							\
+}
+
+#define ATOMIC64_FETCH_OP(op, c_op)					\
+static inline s64 atomic64_fetch_##op(s64 i, atomic64_t *v)		\
+{									\
+	unsigned long flags;						\
+	s64 ret;							\
+									\
+	raw_local_irq_save(flags);					\
+	ret = v->counter;						\
+	v->counter = v->counter c_op i;					\
+	raw_local_irq_restore(flags);					\
+									\
+	return ret;							\
+}
+#endif /* CONFIG_SMP */
+
+#ifndef atomic64_add_return
+ATOMIC64_OP_RETURN(add, +)
+#endif
+
+#ifndef atomic64_sub_return
+ATOMIC64_OP_RETURN(sub, -)
+#endif
+
+#ifndef atomic64_fetch_add
+ATOMIC64_FETCH_OP(add, +)
+#endif
+
+#ifndef atomic64_fetch_sub
+ATOMIC64_FETCH_OP(sub, -)
+#endif
+
+#ifndef atomic64_fetch_and
+ATOMIC64_FETCH_OP(and, &)
+#endif
+
+#ifndef atomic64_fetch_or
+ATOMIC64_FETCH_OP(or, |)
+#endif
+
+#ifndef atomic64_fetch_xor
+ATOMIC64_FETCH_OP(xor, ^)
+#endif
+
+#ifndef atomic64_and
+ATOMIC64_OP(and, &)
+#endif
+
+#ifndef atomic64_or
+ATOMIC64_OP(or, |)
+#endif
+
+#ifndef atomic64_xor
+ATOMIC64_OP(xor, ^)
+#endif
+
+#undef ATOMIC64_FETCH_OP
+#undef ATOMIC64_OP_RETURN
+#undef ATOMIC64_OP
+
+
+#define ATOMIC64_INIT(i)	{ (i) }
+
+static inline void atomic64_add(s64 i, atomic64_t *v)
+{
+	atomic64_add_return(i, v);
+}
+
+static inline void atomic64_sub(s64 i, atomic64_t *v)
+{
+	atomic64_sub_return(i, v);
+}
+
+#ifndef atomic64_read
+#define atomic64_read(v)	READ_ONCE((v)->counter)
+#endif
+
+#define atomic64_set(v, i) WRITE_ONCE(((v)->counter), (i))
+
+#define atomic64_xchg(ptr, v)		(xchg(&(ptr)->counter, (v)))
+#define atomic64_cmpxchg(v, old, new)	(cmpxchg(&((v)->counter), (old), (new)))
+
+#endif /* __LKL_ATOMIC64_H */
diff --git a/arch/um/lkl/include/asm/bitsperlong.h b/arch/um/lkl/include/asm/bitsperlong.h
new file mode 100644
index 000000000000..5745d5e51274
--- /dev/null
+++ b/arch/um/lkl/include/asm/bitsperlong.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __LKL_BITSPERLONG_H
+#define __LKL_BITSPERLONG_H
+
+#include <uapi/asm/bitsperlong.h>
+
+#define BITS_PER_LONG __BITS_PER_LONG
+
+#define BITS_PER_LONG_LONG 64
+
+#endif
diff --git a/arch/um/lkl/include/asm/byteorder.h b/arch/um/lkl/include/asm/byteorder.h
new file mode 100644
index 000000000000..5d0c4efaa44b
--- /dev/null
+++ b/arch/um/lkl/include/asm/byteorder.h
@@ -0,0 +1,7 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_BYTEORDER_H
+#define _ASM_LKL_BYTEORDER_H
+
+#include <uapi/asm/byteorder.h>
+
+#endif /* _ASM_LKL_BYTEORDER_H */
diff --git a/arch/um/lkl/include/asm/cpu.h b/arch/um/lkl/include/asm/cpu.h
new file mode 100644
index 000000000000..d2b8c501c7b1
--- /dev/null
+++ b/arch/um/lkl/include/asm/cpu.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_CPU_H
+#define _ASM_LKL_CPU_H
+
+int lkl_cpu_get(void);
+void lkl_cpu_put(void);
+int lkl_cpu_try_run_irq(int irq);
+int lkl_cpu_init(void);
+void lkl_cpu_shutdown(void);
+void lkl_cpu_wait_shutdown(void);
+void lkl_cpu_change_owner(lkl_thread_t owner);
+void lkl_cpu_set_irqs_pending(void);
+
+#endif /* _ASM_LKL_CPU_H */
diff --git a/arch/um/lkl/include/asm/elf.h b/arch/um/lkl/include/asm/elf.h
new file mode 100644
index 000000000000..bb2456d638f4
--- /dev/null
+++ b/arch/um/lkl/include/asm/elf.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_ELF_H
+#define _ASM_LKL_ELF_H
+
+#define elf_check_arch(x) 0
+
+#ifdef CONFIG_64BIT
+#define ELF_CLASS ELFCLASS64
+#else
+#define ELF_CLASS ELFCLASS32
+#endif
+
+#define elf_gregset_t long
+#define elf_fpregset_t double
+#endif
diff --git a/arch/um/lkl/include/asm/processor.h b/arch/um/lkl/include/asm/processor.h
new file mode 100644
index 000000000000..c1aa8b3a266e
--- /dev/null
+++ b/arch/um/lkl/include/asm/processor.h
@@ -0,0 +1,60 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_PROCESSOR_H
+#define _ASM_LKL_PROCESSOR_H
+
+struct task_struct;
+
+static inline void cpu_relax(void)
+{
+	unsigned long flags;
+
+	/* since this is usually called in a tight loop waiting for some
+	 * external condition (e.g. jiffies) lets run interrupts now to allow
+	 * the external condition to propagate
+	 */
+	local_irq_save(flags);
+	local_irq_restore(flags);
+}
+
+#define current_text_addr() ({ __label__ _l; _l: &&_l; })
+
+static inline unsigned long thread_saved_pc(struct task_struct *tsk)
+{
+	return 0;
+}
+
+static inline void release_thread(struct task_struct *dead_task)
+{
+}
+
+static inline void prepare_to_copy(struct task_struct *tsk)
+{
+}
+
+static inline unsigned long get_wchan(struct task_struct *p)
+{
+	return 0;
+}
+
+static inline void flush_thread(void)
+{
+}
+
+static inline void trap_init(void)
+{
+}
+
+struct thread_struct { };
+
+#define INIT_THREAD { }
+
+#define task_pt_regs(tsk) (struct pt_regs *)(NULL)
+
+/* We don't have strict user/kernel spaces */
+#define TASK_SIZE ((unsigned long)-1)
+#define TASK_UNMAPPED_BASE 0
+
+#define KSTK_EIP(tsk) (0)
+#define KSTK_ESP(tsk) (0)
+
+#endif
diff --git a/arch/um/lkl/include/asm/ptrace.h b/arch/um/lkl/include/asm/ptrace.h
new file mode 100644
index 000000000000..28199be26dc0
--- /dev/null
+++ b/arch/um/lkl/include/asm/ptrace.h
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_PTRACE_H
+#define _ASM_LKL_PTRACE_H
+
+#include <linux/errno.h>
+
+struct task_struct;
+
+#define user_mode(regs) 0
+#define kernel_mode(regs) 1
+#define profile_pc(regs) 0
+#define instruction_pointer(regs) 0
+#define user_stack_pointer(regs) 0
+
+static inline long arch_ptrace(struct task_struct *child, long request,
+			       unsigned long addr, unsigned long data)
+{
+	return -EINVAL;
+}
+
+static inline void ptrace_disable(struct task_struct *child)
+{
+}
+
+#endif
diff --git a/arch/um/lkl/include/asm/sched.h b/arch/um/lkl/include/asm/sched.h
new file mode 100644
index 000000000000..4c2635921ec8
--- /dev/null
+++ b/arch/um/lkl/include/asm/sched.h
@@ -0,0 +1,23 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_SCHED_H
+#define _ASM_LKL_SCHED_H
+
+#include <linux/sched.h>
+#include <uapi/asm/host_ops.h>
+
+static inline void thread_sched_jb(void)
+{
+	if (test_ti_thread_flag(current_thread_info(), TIF_HOST_THREAD)) {
+		set_ti_thread_flag(current_thread_info(), TIF_SCHED_JB);
+		set_current_state(TASK_UNINTERRUPTIBLE);
+		lkl_ops->jmp_buf_set(&current_thread_info()->sched_jb,
+				     schedule);
+	} else {
+		lkl_bug("%s() can be used only for host task\n", __func__);
+	}
+}
+
+void switch_to_host_task(struct task_struct *);
+int host_task_stub(void *unused);
+
+#endif /*  _ASM_LKL_SCHED_H */
diff --git a/arch/um/lkl/include/asm/syscalls.h b/arch/um/lkl/include/asm/syscalls.h
new file mode 100644
index 000000000000..2eaa870a9020
--- /dev/null
+++ b/arch/um/lkl/include/asm/syscalls.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_SYSCALLS_H
+#define _ASM_LKL_SYSCALLS_H
+
+int syscalls_init(void);
+void syscalls_cleanup(void);
+long lkl_syscall(long no, long *params);
+void wakeup_idle_host_task(void);
+
+#define sys_mmap sys_mmap_pgoff
+#define sys_mmap2 sys_mmap_pgoff
+#define sys_clone sys_ni_syscall
+#define sys_vfork sys_ni_syscall
+#define sys_rt_sigreturn sys_ni_syscall
+
+#include <asm-generic/syscalls.h>
+
+#endif /* _ASM_LKL_SYSCALLS_H */
diff --git a/arch/um/lkl/include/asm/syscalls_32.h b/arch/um/lkl/include/asm/syscalls_32.h
new file mode 100644
index 000000000000..0e1a7649c81b
--- /dev/null
+++ b/arch/um/lkl/include/asm/syscalls_32.h
@@ -0,0 +1,43 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_SYSCALLS_32_H
+#define _ASM_SYSCALLS_32_H
+
+#include <linux/compiler.h>
+#include <linux/linkage.h>
+#include <linux/types.h>
+#include <linux/signal.h>
+
+#if __BITS_PER_LONG == 32
+
+/* kernel/syscalls_32.c */
+asmlinkage long sys32_truncate64(const char __user *, unsigned long,
+				 unsigned long);
+asmlinkage long sys32_ftruncate64(unsigned int, unsigned long, unsigned long);
+
+#ifdef CONFIG_MMU
+struct mmap_arg_struct32;
+asmlinkage long sys32_mmap(struct mmap_arg_struct32 __user *);
+#endif
+
+asmlinkage long sys32_wait4(pid_t, unsigned int __user *, int,
+			    struct rusage __user *);
+
+asmlinkage long sys32_pread64(unsigned int, char __user *, u32, u32, u32);
+asmlinkage long sys32_pwrite64(unsigned int, const char __user *, u32, u32,
+			       u32);
+
+long sys32_fadvise64_64(int a, __u32 b, __u32 c, __u32 d, __u32 e, int f);
+
+asmlinkage ssize_t sys32_readahead(int, unsigned int, unsigned int, size_t);
+asmlinkage long sys32_sync_file_range(int, unsigned int, unsigned int,
+				      unsigned int, unsigned int, unsigned int);
+asmlinkage long sys32_sync_file_range2(int, unsigned int, unsigned int,
+				       unsigned int, unsigned int,
+				       unsigned int);
+asmlinkage long sys32_fadvise64(int, unsigned int, unsigned int, size_t, int);
+asmlinkage long sys32_fallocate(int, int, unsigned int, unsigned int,
+				unsigned int, unsigned int);
+
+#endif /* __BITS_PER_LONG */
+
+#endif /* _ASM_SYSCALLS_32_H */
diff --git a/arch/um/lkl/include/asm/tlb.h b/arch/um/lkl/include/asm/tlb.h
new file mode 100644
index 000000000000..d474890d317d
--- /dev/null
+++ b/arch/um/lkl/include/asm/tlb.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_TLB_H
+#define _ASM_LKL_TLB_H
+
+#define tlb_start_vma(tlb, vma)				do { } while (0)
+#define tlb_end_vma(tlb, vma)				do { } while (0)
+#define __tlb_remove_tlb_entry(tlb, pte, address)	do { } while (0)
+#define tlb_flush(tlb)					do { } while (0)
+
+#include <asm-generic/tlb.h>
+
+#endif /* _ASM_LKL_TLB_H */
diff --git a/arch/um/lkl/include/asm/uaccess.h b/arch/um/lkl/include/asm/uaccess.h
new file mode 100644
index 000000000000..f267ac3be8b3
--- /dev/null
+++ b/arch/um/lkl/include/asm/uaccess.h
@@ -0,0 +1,64 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_UACCESS_H
+#define _ASM_LKL_UACCESS_H
+
+/* copied from old include/asm-generic/uaccess.h */
+static inline __must_check long
+raw_copy_from_user(void *to, const void __user *from, unsigned long n)
+{
+	if (__builtin_constant_p(n)) {
+		switch (n) {
+		case 1:
+			*(u8 *)to = *(u8 __force *)from;
+			return 0;
+		case 2:
+			*(u16 *)to = *(u16 __force *)from;
+			return 0;
+		case 4:
+			*(u32 *)to = *(u32 __force *)from;
+			return 0;
+#ifdef CONFIG_64BIT
+		case 8:
+			*(u64 *)to = *(u64 __force *)from;
+			return 0;
+#endif
+		default:
+			break;
+		}
+	}
+
+	memcpy(to, (const void __force *)from, n);
+	return 0;
+}
+
+static inline __must_check long
+raw_copy_to_user(void __user *to, const void *from, unsigned long n)
+{
+	if (__builtin_constant_p(n)) {
+		switch (n) {
+		case 1:
+			*(u8 __force *)to = *(u8 *)from;
+			return 0;
+		case 2:
+			*(u16 __force *)to = *(u16 *)from;
+			return 0;
+		case 4:
+			*(u32 __force *)to = *(u32 *)from;
+			return 0;
+#ifdef CONFIG_64BIT
+		case 8:
+			*(u64 __force *)to = *(u64 *)from;
+			return 0;
+#endif
+		default:
+			break;
+		}
+	}
+
+	memcpy((void __force *)to, from, n);
+	return 0;
+}
+
+#include <asm-generic/uaccess.h>
+
+#endif
diff --git a/arch/um/lkl/include/asm/unistd_32.h b/arch/um/lkl/include/asm/unistd_32.h
new file mode 100644
index 000000000000..8582a55e61e2
--- /dev/null
+++ b/arch/um/lkl/include/asm/unistd_32.h
@@ -0,0 +1,31 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <asm/bitsperlong.h>
+
+#ifndef __SYSCALL
+#define __SYSCALL(x, y)
+#endif
+
+#if __BITS_PER_LONG == 32
+__SYSCALL(__NR3264_truncate, sys32_truncate64)
+__SYSCALL(__NR3264_ftruncate, sys32_ftruncate64)
+
+#ifdef CONFIG_MMU
+__SYSCALL(__NR3264_mmap, sys32_mmap)
+#endif
+
+__SYSCALL(__NR_wait4, sys32_wait4)
+
+__SYSCALL(__NR_pread64, sys32_pread64)
+__SYSCALL(__NR_pwrite64, sys32_pwrite64)
+
+__SYSCALL(__NR_readahead, sys32_readahead)
+#ifdef __ARCH_WANT_SYNC_FILE_RANGE2
+__SYSCALL(__NR_sync_file_range2, sys32_sync_file_range2)
+#else
+__SYSCALL(__NR_sync_file_range, sys32_sync_file_range)
+#endif
+/* mm/fadvise.c */
+__SYSCALL(__NR3264_fadvise64, sys32_fadvise64_64)
+__SYSCALL(__NR_fallocate, sys32_fallocate)
+
+#endif
diff --git a/arch/um/lkl/include/asm/vmalloc.h b/arch/um/lkl/include/asm/vmalloc.h
new file mode 100644
index 000000000000..46de2e4f03c7
--- /dev/null
+++ b/arch/um/lkl/include/asm/vmalloc.h
@@ -0,0 +1,5 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_VMALLOC_H
+#define _LKL_VMALLOC_H
+
+#endif /* _LKL_VMALLOC_H */
diff --git a/arch/um/lkl/include/asm/vmlinux.lds.h b/arch/um/lkl/include/asm/vmlinux.lds.h
new file mode 100644
index 000000000000..a3c285882dc4
--- /dev/null
+++ b/arch/um/lkl/include/asm/vmlinux.lds.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_VMLINUX_LDS_H
+#define _LKL_VMLINUX_LDS_H
+
+/* we encode our own __ro_after_init section */
+#define RO_AFTER_INIT_DATA
+
+#ifdef __MINGW32__
+#define RODATA_SECTION .rdata
+#endif
+
+#include <asm-generic/vmlinux.lds.h>
+
+#endif
diff --git a/arch/um/lkl/include/asm/xor.h b/arch/um/lkl/include/asm/xor.h
new file mode 100644
index 000000000000..286ce75b5d9d
--- /dev/null
+++ b/arch/um/lkl/include/asm/xor.h
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_XOR_H
+#define _ASM_LKL_XOR_H
+
+#include <asm-generic/xor.h>
+
+#define XOR_SELECT_TEMPLATE(x) (&xor_block_8regs)
+
+#endif /* _ASM_LKL_XOR_H */
diff --git a/arch/um/lkl/include/uapi/asm/Kbuild b/arch/um/lkl/include/uapi/asm/Kbuild
new file mode 100644
index 000000000000..e1f554364a56
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/Kbuild
@@ -0,0 +1,11 @@
+# UAPI Header export list
+
+generic-y += elf.h
+generic-y += kvm_para.h
+generic-y += shmparam.h
+generic-y += siginfo.h
+generic-y += swab.h
+generic-y += timex.h
+
+# no header-y since we need special user headers handling
+# see arch/lkl/script/headers.py
diff --git a/arch/um/lkl/include/uapi/asm/bitsperlong.h b/arch/um/lkl/include/uapi/asm/bitsperlong.h
new file mode 100644
index 000000000000..8b4ebf2b0264
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/bitsperlong.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _ASM_UAPI_LKL_BITSPERLONG_H
+#define _ASM_UAPI_LKL_BITSPERLONG_H
+
+#ifdef CONFIG_64BIT
+#define __BITS_PER_LONG 64
+#else
+#define __BITS_PER_LONG 32
+#endif
+
+#define __ARCH_WANT_STAT64
+
+#endif /* _ASM_UAPI_LKL_BITSPERLONG_H */
diff --git a/arch/um/lkl/include/uapi/asm/byteorder.h b/arch/um/lkl/include/uapi/asm/byteorder.h
new file mode 100644
index 000000000000..3c4a58d2062f
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/byteorder.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _ASM_UAPI_LKL_BYTEORDER_H
+#define _ASM_UAPI_LKL_BYTEORDER_H
+
+#if defined(CONFIG_BIG_ENDIAN)
+#include <linux/byteorder/big_endian.h>
+#else
+#include <linux/byteorder/little_endian.h>
+#endif
+
+#endif /* _ASM_UAPI_LKL_BYTEORDER_H */
diff --git a/arch/um/lkl/include/uapi/asm/syscalls.h b/arch/um/lkl/include/uapi/asm/syscalls.h
new file mode 100644
index 000000000000..a81534ffccb7
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/syscalls.h
@@ -0,0 +1,348 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _ASM_UAPI_LKL_SYSCALLS_H
+#define _ASM_UAPI_LKL_SYSCALLS_H
+
+#include <autoconf.h>
+#include <linux/types.h>
+
+typedef __kernel_uid32_t	qid_t;
+typedef __kernel_fd_set		fd_set;
+typedef __kernel_mode_t		mode_t;
+typedef unsigned short		umode_t;
+typedef __u32			nlink_t;
+typedef __kernel_off_t		off_t;
+typedef __kernel_pid_t		pid_t;
+typedef __kernel_key_t		key_t;
+typedef __kernel_suseconds_t	suseconds_t;
+typedef __kernel_timer_t	timer_t;
+typedef __kernel_clockid_t	clockid_t;
+typedef __kernel_mqd_t		mqd_t;
+typedef __kernel_uid32_t	uid_t;
+typedef __kernel_gid32_t	gid_t;
+typedef __kernel_uid16_t        uid16_t;
+typedef __kernel_gid16_t        gid16_t;
+typedef unsigned long		uintptr_t;
+#ifdef CONFIG_UID16
+typedef __kernel_old_uid_t	old_uid_t;
+typedef __kernel_old_gid_t	old_gid_t;
+#endif
+typedef __kernel_loff_t		loff_t;
+typedef __kernel_size_t		size_t;
+typedef __kernel_ssize_t	ssize_t;
+typedef __kernel_time_t		time_t;
+typedef __kernel_clock_t	clock_t;
+typedef __u32			u32;
+typedef __s32			s32;
+typedef __u64			u64;
+typedef __s64			s64;
+
+#define __user
+
+#include <asm/unistd.h>
+/* Temporary undefine system calls that don't have data types defined in UAPI
+ * headers
+ */
+#undef __NR_kexec_load
+#undef __NR_getcpu
+#undef __NR_sched_getattr
+#undef __NR_sched_setattr
+#undef __NR_sched_setparam
+#undef __NR_sched_getparam
+#undef __NR_sched_setscheduler
+#undef __NR_name_to_handle_at
+#undef __NR_open_by_handle_at
+
+/* deprecated system calls */
+#undef __NR_epoll_create
+#undef __NR_epoll_wait
+#undef __NR_access
+#undef __NR_chmod
+#undef __NR_chown
+#undef __NR_lchown
+#undef __NR_open
+#undef __NR_creat
+#undef __NR_readlink
+#undef __NR_pipe
+#undef __NR_mknod
+#undef __NR_mkdir
+#undef __NR_rmdir
+#undef __NR_unlink
+#undef __NR_symlink
+#undef __NR_link
+#undef __NR_rename
+#undef __NR_getdents
+#undef __NR_select
+#undef __NR_poll
+#undef __NR_dup2
+#undef __NR_futimesat
+#undef __NR_utimes
+#undef __NR_ustat
+#undef __NR_eventfd
+#undef __NR_bdflush
+#undef __NR_send
+#undef __NR_recv
+
+#undef __NR_umount
+#define __NR_umount __NR_umount2
+
+#ifdef CONFIG_64BIT
+#define __NR_newfstat __NR3264_fstat
+#define __NR_newfstatat __NR3264_fstatat
+#endif
+
+#define __NR_mmap_pgoff __NR3264_mmap
+
+#include <linux/time.h>
+#include <linux/times.h>
+#include <linux/timex.h>
+#include <linux/capability.h>
+#define __KERNEL__ /* to pull in S_ definitions */
+#include <linux/stat.h>
+#undef __KERNEL__
+#include <linux/errno.h>
+#include <linux/fcntl.h>
+#include <linux/fs.h>
+#include <asm/statfs.h>
+#include <asm/stat.h>
+#include <linux/bpf.h>
+#include <linux/msg.h>
+#include <linux/resource.h>
+#include <linux/sysinfo.h>
+#include <linux/shm.h>
+#include <linux/aio_abi.h>
+#include <linux/socket.h>
+#include <linux/perf_event.h>
+#include <linux/sem.h>
+#include <linux/futex.h>
+#include <linux/poll.h>
+#include <linux/mqueue.h>
+#include <linux/eventpoll.h>
+#include <linux/uio.h>
+#include <asm/signal.h>
+#include <asm/siginfo.h>
+#include <linux/utime.h>
+#include <asm/socket.h>
+#include <linux/icmp.h>
+#include <linux/ip.h>
+
+/* Define data structures used in system calls that are not defined in UAPI
+ * headers
+ */
+struct sockaddr {
+	unsigned short int sa_family;
+	char sa_data[14];
+};
+
+#define __UAPI_DEF_IF_NET_DEVICE_FLAGS_LOWER_UP_DORMANT_ECHO 1
+#define __UAPI_DEF_IF_IFNAMSIZ	1
+#define __UAPI_DEF_IF_NET_DEVICE_FLAGS 1
+#define __UAPI_DEF_IF_IFREQ	1
+#define __UAPI_DEF_IF_IFMAP	1
+#include <linux/if.h>
+#define __UAPI_DEF_IN_IPPROTO	1
+#define __UAPI_DEF_IN_ADDR	1
+#define __UAPI_DEF_IN6_ADDR	1
+#define __UAPI_DEF_IP_MREQ	1
+#define __UAPI_DEF_IN_PKTINFO	1
+#define __UAPI_DEF_SOCKADDR_IN	1
+#define __UAPI_DEF_IN_CLASS	1
+#include <linux/in.h>
+#include <linux/in6.h>
+#include <linux/sockios.h>
+#include <linux/route.h>
+#include <linux/ipv6_route.h>
+#include <linux/ipv6.h>
+#include <linux/netlink.h>
+#include <linux/neighbour.h>
+#include <linux/rtnetlink.h>
+#include <linux/fib_rules.h>
+
+#include <linux/kdev_t.h>
+#include <asm/irq.h>
+#include <linux/virtio_blk.h>
+#include <linux/virtio_net.h>
+#include <linux/virtio_ring.h>
+#include <linux/pkt_sched.h>
+#include <linux/io_uring.h>
+
+struct user_msghdr {
+	void		__user *msg_name;
+	int		msg_namelen;
+	struct iovec	__user *msg_iov;
+	__kernel_size_t	msg_iovlen;
+	void		__user *msg_control;
+	__kernel_size_t	msg_controllen;
+	unsigned int	msg_flags;
+};
+
+typedef __u32 key_serial_t;
+
+struct mmsghdr {
+	struct user_msghdr  msg_hdr;
+	unsigned int        msg_len;
+};
+
+struct linux_dirent64 {
+	u64		d_ino;
+	s64		d_off;
+	unsigned short	d_reclen;
+	unsigned char	d_type;
+	char		d_name[0];
+};
+
+struct linux_dirent {
+	unsigned long	d_ino;
+	unsigned long	d_off;
+	unsigned short	d_reclen;
+	char		d_name[1];
+};
+
+struct ustat {
+	__kernel_daddr_t	f_tfree;
+	__kernel_ino_t		f_tinode;
+	char			f_fname[6];
+	char			f_fpack[6];
+};
+
+typedef __kernel_rwf_t		rwf_t;
+
+#define AF_UNSPEC       0
+#define AF_UNIX         1
+#define AF_LOCAL        1
+#define AF_INET         2
+#define AF_AX25         3
+#define AF_IPX          4
+#define AF_APPLETALK    5
+#define AF_NETROM       6
+#define AF_BRIDGE       7
+#define AF_ATMPVC       8
+#define AF_X25          9
+#define AF_INET6        10
+#define AF_ROSE         11
+#define AF_DECnet       12
+#define AF_NETBEUI      13
+#define AF_SECURITY     14
+#define AF_KEY          15
+#define AF_NETLINK      16
+#define AF_ROUTE        AF_NETLINK
+#define AF_PACKET       17
+#define AF_ASH          18
+#define AF_ECONET       19
+#define AF_ATMSVC       20
+#define AF_RDS          21
+#define AF_SNA          22
+#define AF_IRDA         23
+#define AF_PPPOX        24
+#define AF_WANPIPE      25
+#define AF_LLC          26
+#define AF_IB           27
+#define AF_MPLS         28
+#define AF_CAN          29
+#define AF_TIPC         30
+#define AF_BLUETOOTH    31
+#define AF_IUCV         32
+#define AF_RXRPC        33
+#define AF_ISDN         34
+#define AF_PHONET       35
+#define AF_IEEE802154   36
+#define AF_CAIF         37
+#define AF_ALG          38
+#define AF_NFC          39
+#define AF_VSOCK        40
+
+#define SOCK_STREAM		1
+#define SOCK_DGRAM		2
+#define SOCK_RAW		3
+#define SOCK_RDM		4
+#define SOCK_SEQPACKET		5
+#define SOCK_DCCP		6
+#define SOCK_PACKET		10
+
+#define MSG_TRUNC 0x20
+#define MSG_DONTWAIT 0x40
+
+/* avoid colision with system headers defines */
+#define sa_handler sa_handler
+#define st_atime st_atime
+#define st_mtime st_mtime
+#define st_ctime st_ctime
+#define s_addr s_addr
+
+long lkl_syscall(long no, long *params);
+long lkl_sys_halt(void);
+
+#define __MAP0(m, ...)
+#define __MAP1(m, t, a) m(t, a)
+#define __MAP2(m, t, a, ...) m(t, a), __MAP1(m, __VA_ARGS__)
+#define __MAP3(m, t, a, ...) m(t, a), __MAP2(m, __VA_ARGS__)
+#define __MAP4(m, t, a, ...) m(t, a), __MAP3(m, __VA_ARGS__)
+#define __MAP5(m, t, a, ...) m(t, a), __MAP4(m, __VA_ARGS__)
+#define __MAP6(m, t, a, ...) m(t, a), __MAP5(m, __VA_ARGS__)
+#define __MAP(n, ...) __MAP##n(__VA_ARGS__)
+
+#define __SC_LONG(t, a) (long)a
+#define __SC_TABLE(t, a) {sizeof(t), (long long)(a)}
+#define __SC_DECL(t, a) t a
+
+#define LKL_SYSCALL0(name)					       \
+	static inline long lkl_sys##name(void)			       \
+	{							       \
+		long params[6];					       \
+		return lkl_syscall(__lkl__NR##name, params);	       \
+	}
+
+#if __BITS_PER_LONG == 32
+#define LKL_SYSCALLx(x, name, ...)					\
+	static inline							\
+	long lkl_sys##name(__MAP(x, __SC_DECL, __VA_ARGS__))		\
+	{								\
+		struct {						\
+			unsigned int size;				\
+			long long value;				\
+		} lkl_params[x] = { __MAP(x, __SC_TABLE, __VA_ARGS__) }; \
+		long sys_params[6], i, k;				\
+		for (i = k = 0; i < x && k < 6; i++, k++) {		\
+			if (lkl_params[i].size > sizeof(long) &&	\
+			    k + 1 < 6) {				\
+				sys_params[k] =				\
+					(long)(lkl_params[i].value & (-1UL)); \
+				k++;					\
+				sys_params[k] =				\
+					(long)(lkl_params[i].value >>	\
+					       __BITS_PER_LONG);	\
+			} else {					\
+				sys_params[k] = (long)(lkl_params[i].value); \
+			}						\
+		}							\
+		return lkl_syscall(__lkl__NR##name, sys_params);	\
+	}
+#else
+#define LKL_SYSCALLx(x, name, ...)					\
+	static inline							\
+	long lkl_sys##name(__MAP(x, __SC_DECL, __VA_ARGS__))		\
+	{								\
+		long lkl_params[6] = { __MAP(x, __SC_LONG, __VA_ARGS__) }; \
+		return lkl_syscall(__lkl__NR##name, lkl_params);	\
+	}
+#endif
+
+#define SYSCALL_DEFINE0(name, ...) LKL_SYSCALL0(name)
+#define SYSCALL_DEFINE1(name, ...) LKL_SYSCALLx(1, name, __VA_ARGS__)
+#define SYSCALL_DEFINE2(name, ...) LKL_SYSCALLx(2, name, __VA_ARGS__)
+#define SYSCALL_DEFINE3(name, ...) LKL_SYSCALLx(3, name, __VA_ARGS__)
+#define SYSCALL_DEFINE4(name, ...) LKL_SYSCALLx(4, name, __VA_ARGS__)
+#define SYSCALL_DEFINE5(name, ...) LKL_SYSCALLx(5, name, __VA_ARGS__)
+#define SYSCALL_DEFINE6(name, ...) LKL_SYSCALLx(6, name, __VA_ARGS__)
+
+#if __BITS_PER_LONG == 32
+#pragma GCC diagnostic push
+#pragma GCC diagnostic ignored "-Wpointer-to-int-cast"
+#endif
+
+#include <asm/syscall_defs.h>
+
+#if __BITS_PER_LONG == 32
+#pragma GCC diagnostic pop
+#endif
+
+#endif
diff --git a/arch/um/lkl/kernel/asm-offsets.c b/arch/um/lkl/kernel/asm-offsets.c
new file mode 100644
index 000000000000..6be0763698dc
--- /dev/null
+++ b/arch/um/lkl/kernel/asm-offsets.c
@@ -0,0 +1,2 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Dummy asm-offsets.c file. Required by kbuild and ready to be used - hint! */
diff --git a/arch/um/lkl/kernel/misc.c b/arch/um/lkl/kernel/misc.c
new file mode 100644
index 000000000000..60f048f02ae6
--- /dev/null
+++ b/arch/um/lkl/kernel/misc.c
@@ -0,0 +1,60 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/kallsyms.h>
+#include <linux/module.h>
+#include <linux/sched.h>
+#include <linux/seq_file.h>
+#include <asm/ptrace.h>
+#include <asm/host_ops.h>
+
+#ifdef CONFIG_PRINTK
+void dump_stack(void)
+{
+	unsigned long dummy;
+	unsigned long *stack = &dummy;
+	unsigned long addr;
+
+	pr_info("Call Trace:\n");
+	while (((long)stack & (THREAD_SIZE - 1)) != 0) {
+		addr = *stack;
+		if (__kernel_text_address(addr)) {
+			pr_info("%p:  [<%08lx>] %pS", stack, addr,
+				(void *)addr);
+			pr_cont("\n");
+		}
+		stack++;
+	}
+	pr_info("\n");
+}
+#endif
+
+void show_regs(struct pt_regs *regs)
+{
+}
+
+#ifdef CONFIG_PROC_FS
+static void *cpuinfo_start(struct seq_file *m, loff_t *pos)
+{
+	return NULL;
+}
+
+static void *cpuinfo_next(struct seq_file *m, void *v, loff_t *pos)
+{
+	return NULL;
+}
+
+static void cpuinfo_stop(struct seq_file *m, void *v)
+{
+}
+
+static int show_cpuinfo(struct seq_file *m, void *v)
+{
+	return 0;
+}
+
+const struct seq_operations cpuinfo_op = {
+	.start	= cpuinfo_start,
+	.next	= cpuinfo_next,
+	.stop	= cpuinfo_stop,
+	.show	= show_cpuinfo,
+};
+#endif
diff --git a/arch/um/lkl/kernel/vmlinux.lds.S b/arch/um/lkl/kernel/vmlinux.lds.S
new file mode 100644
index 000000000000..f57c563522cb
--- /dev/null
+++ b/arch/um/lkl/kernel/vmlinux.lds.S
@@ -0,0 +1,51 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <asm/vmlinux.lds.h>
+#include <asm/thread_info.h>
+#include <asm/page.h>
+#include <asm/cache.h>
+#include <linux/export.h>
+
+OUTPUT_FORMAT(CONFIG_OUTPUT_FORMAT)
+
+jiffies = jiffies_64;
+
+SECTIONS
+{
+	__init_begin = .;
+	HEAD_TEXT_SECTION
+	INIT_TEXT_SECTION(PAGE_SIZE)
+	INIT_DATA_SECTION(16)
+	PERCPU_SECTION(L1_CACHE_BYTES)
+	__init_end = .;
+
+	_stext = .;
+	_text = . ;
+	text = . ;
+	.text      :
+	{
+		TEXT_TEXT
+		SCHED_TEXT
+		LOCK_TEXT
+		CPUIDLE_TEXT
+	}
+	_etext = .;
+
+	_sdata = .;
+	RO_DATA(PAGE_SIZE)
+	RW_DATA(L1_CACHE_BYTES, PAGE_SIZE, THREAD_SIZE)
+	_edata = .;
+
+	__start_ro_after_init = .;
+	.data..ro_after_init : { *(.data..ro_after_init)}
+	EXCEPTION_TABLE(16)
+	__end_ro_after_init = .;
+	NOTES
+
+	BSS_SECTION(0, 0, 0)
+	_end = .;
+
+	STABS_DEBUG
+	DWARF_DEBUG
+
+	DISCARDS
+}
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 03/25] um lkl: host interface
  2020-03-30 14:45   ` Hajime Tazaki
@ 2020-03-30 14:45     ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Michael Zimmermann, Patrick Collins, Pierre-Hugues Husson,
	Yuan Liu, Hajime Tazaki

From: Octavian Purdila <tavi.purdila@gmail.com>

This patch introduces the host operations that define the interface
between the LKL and the host. These operations must be provided either
by a host library or by the application itself.

Cc: Michael Zimmermann <sigmaepsilon92@gmail.com>
Cc: Patrick Collins <pscollins@google.com>
Cc: Pierre-Hugues Husson <phh@phh.me>
Cc: Yuan Liu <liuyuan@google.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/lkl/include/asm/host_ops.h      | 10 ++++++++++
 arch/um/lkl/include/uapi/asm/host_ops.h | 26 +++++++++++++++++++++++++
 2 files changed, 36 insertions(+)
 create mode 100644 arch/um/lkl/include/asm/host_ops.h
 create mode 100644 arch/um/lkl/include/uapi/asm/host_ops.h

diff --git a/arch/um/lkl/include/asm/host_ops.h b/arch/um/lkl/include/asm/host_ops.h
new file mode 100644
index 000000000000..65850f394b79
--- /dev/null
+++ b/arch/um/lkl/include/asm/host_ops.h
@@ -0,0 +1,10 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_HOST_OPS_H
+#define _ASM_LKL_HOST_OPS_H
+
+#include "irq.h"
+#include <uapi/asm/host_ops.h>
+
+extern struct lkl_host_operations *lkl_ops;
+
+#endif
diff --git a/arch/um/lkl/include/uapi/asm/host_ops.h b/arch/um/lkl/include/uapi/asm/host_ops.h
new file mode 100644
index 000000000000..7cfb0a93e6a6
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/host_ops.h
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _ASM_UAPI_LKL_HOST_OPS_H
+#define _ASM_UAPI_LKL_HOST_OPS_H
+
+/* Defined in {posix,nt}-host.c */
+struct lkl_mutex;
+struct lkl_sem;
+struct lkl_tls_key;
+typedef unsigned long lkl_thread_t;
+struct lkl_jmp_buf {
+	unsigned long buf[32];
+};
+
+/**
+ * lkl_host_operations - host operations used by the Linux kernel
+ *
+ * These operations must be provided by a host library or by the application
+ * itself.
+ *
+ */
+struct lkl_host_operations {
+};
+
+void lkl_bug(const char *fmt, ...);
+
+#endif
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 03/25] um lkl: host interface
@ 2020-03-30 14:45     ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: linux-arch, Hajime Tazaki, Octavian Purdila, Akira Moroo,
	Patrick Collins, linux-kernel-library, Pierre-Hugues Husson,
	Michael Zimmermann, Yuan Liu

From: Octavian Purdila <tavi.purdila@gmail.com>

This patch introduces the host operations that define the interface
between the LKL and the host. These operations must be provided either
by a host library or by the application itself.

Cc: Michael Zimmermann <sigmaepsilon92@gmail.com>
Cc: Patrick Collins <pscollins@google.com>
Cc: Pierre-Hugues Husson <phh@phh.me>
Cc: Yuan Liu <liuyuan@google.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/lkl/include/asm/host_ops.h      | 10 ++++++++++
 arch/um/lkl/include/uapi/asm/host_ops.h | 26 +++++++++++++++++++++++++
 2 files changed, 36 insertions(+)
 create mode 100644 arch/um/lkl/include/asm/host_ops.h
 create mode 100644 arch/um/lkl/include/uapi/asm/host_ops.h

diff --git a/arch/um/lkl/include/asm/host_ops.h b/arch/um/lkl/include/asm/host_ops.h
new file mode 100644
index 000000000000..65850f394b79
--- /dev/null
+++ b/arch/um/lkl/include/asm/host_ops.h
@@ -0,0 +1,10 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_HOST_OPS_H
+#define _ASM_LKL_HOST_OPS_H
+
+#include "irq.h"
+#include <uapi/asm/host_ops.h>
+
+extern struct lkl_host_operations *lkl_ops;
+
+#endif
diff --git a/arch/um/lkl/include/uapi/asm/host_ops.h b/arch/um/lkl/include/uapi/asm/host_ops.h
new file mode 100644
index 000000000000..7cfb0a93e6a6
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/host_ops.h
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _ASM_UAPI_LKL_HOST_OPS_H
+#define _ASM_UAPI_LKL_HOST_OPS_H
+
+/* Defined in {posix,nt}-host.c */
+struct lkl_mutex;
+struct lkl_sem;
+struct lkl_tls_key;
+typedef unsigned long lkl_thread_t;
+struct lkl_jmp_buf {
+	unsigned long buf[32];
+};
+
+/**
+ * lkl_host_operations - host operations used by the Linux kernel
+ *
+ * These operations must be provided by a host library or by the application
+ * itself.
+ *
+ */
+struct lkl_host_operations {
+};
+
+void lkl_bug(const char *fmt, ...);
+
+#endif
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 04/25] um lkl: memory handling
  2020-03-30 14:45   ` Hajime Tazaki
@ 2020-03-30 14:45     ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Levente Kurusa, Yuan Liu, Hajime Tazaki

From: Octavian Purdila <tavi.purdila@gmail.com>

LKL is a non MMU architecture and hence there is not much work left to
do other than initializing the boot allocator and providing the page
and page table definitions.

The backstore memory is allocated via a host operation and the memory
size to be used is specified when the kernel is started, in the
lkl_start_kernel call.

Cc: H.K. Jerry Chu <hkchu@google.com>
Cc: Levente Kurusa <levex@linux.com>
Cc: Yuan Liu <liuyuan@google.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/lkl/include/asm/page.h          | 14 ++++++
 arch/um/lkl/include/asm/pgtable.h       | 57 +++++++++++++++++++++
 arch/um/lkl/include/uapi/asm/host_ops.h |  5 ++
 arch/um/lkl/mm/bootmem.c                | 66 +++++++++++++++++++++++++
 4 files changed, 142 insertions(+)
 create mode 100644 arch/um/lkl/include/asm/page.h
 create mode 100644 arch/um/lkl/include/asm/pgtable.h
 create mode 100644 arch/um/lkl/mm/bootmem.c

diff --git a/arch/um/lkl/include/asm/page.h b/arch/um/lkl/include/asm/page.h
new file mode 100644
index 000000000000..e77f3da22031
--- /dev/null
+++ b/arch/um/lkl/include/asm/page.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_PAGE_H
+#define _ASM_LKL_PAGE_H
+
+#define CONFIG_KERNEL_RAM_BASE_ADDRESS memory_start
+
+#ifndef __ASSEMBLY__
+void free_mem(void);
+void bootmem_init(unsigned long mem_size);
+#endif
+
+#include <asm-generic/page.h>
+
+#endif /* _ASM_LKL_PAGE_H */
diff --git a/arch/um/lkl/include/asm/pgtable.h b/arch/um/lkl/include/asm/pgtable.h
new file mode 100644
index 000000000000..733beb6d53f6
--- /dev/null
+++ b/arch/um/lkl/include/asm/pgtable.h
@@ -0,0 +1,57 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_PGTABLE_H
+#define _LKL_PGTABLE_H
+
+#include <asm-generic/pgtable-nopud.h>
+
+/*
+ * (C) Copyright 2000-2002, Greg Ungerer <gerg@snapgear.com>
+ */
+
+#include <linux/slab.h>
+#include <asm/processor.h>
+#include <asm/io.h>
+
+#define pgd_present(pgd)	(1)
+#define pgd_none(pgd)		(0)
+#define pgd_bad(pgd)		(0)
+#define pgd_clear(pgdp)
+#define kern_addr_valid(addr)	(1)
+#define	pmd_offset(a, b)	((void *)0)
+
+#define PAGE_NONE		__pgprot(0)
+#define PAGE_SHARED		__pgprot(0)
+#define PAGE_COPY		__pgprot(0)
+#define PAGE_READONLY		__pgprot(0)
+#define PAGE_KERNEL		__pgprot(0)
+
+void paging_init(void);
+#define swapper_pg_dir		((pgd_t *)0)
+
+#define __swp_type(x)		(0)
+#define __swp_offset(x)		(0)
+#define __swp_entry(typ, off)	((swp_entry_t) { ((typ) | ((off) << 7)) })
+#define __pte_to_swp_entry(pte)	((swp_entry_t) { pte_val(pte) })
+#define __swp_entry_to_pte(x)	((pte_t) { (x).val })
+
+/*
+ * ZERO_PAGE is a global shared page that is always zero: used
+ * for zero-mapped memory areas etc..
+ */
+extern void *empty_zero_page;
+#define ZERO_PAGE(vaddr)	(virt_to_page(empty_zero_page))
+
+/*
+ * All 32bit addresses are effectively valid for vmalloc...
+ * Sort of meaningless for non-VM targets.
+ */
+#define	VMALLOC_START		0
+#define	VMALLOC_END		0xffffffff
+#define	KMAP_START		0
+#define	KMAP_END		0xffffffff
+
+#include <asm-generic/pgtable.h>
+
+#define check_pgt_cache()	do { } while (0)
+
+#endif
diff --git a/arch/um/lkl/include/uapi/asm/host_ops.h b/arch/um/lkl/include/uapi/asm/host_ops.h
index 7cfb0a93e6a6..6bbc94c120be 100644
--- a/arch/um/lkl/include/uapi/asm/host_ops.h
+++ b/arch/um/lkl/include/uapi/asm/host_ops.h
@@ -17,8 +17,13 @@ struct lkl_jmp_buf {
  * These operations must be provided by a host library or by the application
  * itself.
  *
+ * @mem_alloc - allocate memory
+ * @mem_free - free memory
+ *
  */
 struct lkl_host_operations {
+	void *(*mem_alloc)(unsigned long mem);
+	void (*mem_free)(void *mem);
 };
 
 void lkl_bug(const char *fmt, ...);
diff --git a/arch/um/lkl/mm/bootmem.c b/arch/um/lkl/mm/bootmem.c
new file mode 100644
index 000000000000..39dd0d22b44e
--- /dev/null
+++ b/arch/um/lkl/mm/bootmem.c
@@ -0,0 +1,66 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/memblock.h>
+#include <linux/mm.h>
+#include <linux/swap.h>
+
+unsigned long memory_start, memory_end;
+static unsigned long _memory_start, mem_size;
+
+void *empty_zero_page;
+
+void __init bootmem_init(unsigned long mem_sz)
+{
+	mem_size = mem_sz;
+
+	_memory_start = (unsigned long)lkl_ops->mem_alloc(mem_size);
+	memory_start = _memory_start;
+	WARN_ON(!memory_start);
+	memory_end = memory_start + mem_size;
+
+	if (PAGE_ALIGN(memory_start) != memory_start) {
+		mem_size -= PAGE_ALIGN(memory_start) - memory_start;
+		memory_start = PAGE_ALIGN(memory_start);
+		mem_size = (mem_size / PAGE_SIZE) * PAGE_SIZE;
+	}
+	pr_info("memblock address range: 0x%lx - 0x%lx\n", memory_start,
+		memory_start + mem_size);
+	/*
+	 * Give all the memory to the bootmap allocator, tell it to put the
+	 * boot mem_map at the start of memory.
+	 */
+	max_low_pfn = virt_to_pfn(memory_end);
+	min_low_pfn = virt_to_pfn(memory_start);
+	memblock_add(memory_start, mem_size);
+
+	empty_zero_page = memblock_alloc(PAGE_SIZE, PAGE_SIZE);
+	memset((void *)empty_zero_page, 0, PAGE_SIZE);
+
+	{
+		unsigned long zones_size[MAX_NR_ZONES] = {0, };
+
+		zones_size[ZONE_NORMAL] = (mem_size) >> PAGE_SHIFT;
+		free_area_init(zones_size);
+	}
+}
+
+void __init mem_init(void)
+{
+	max_mapnr = (((unsigned long)high_memory) - PAGE_OFFSET) >> PAGE_SHIFT;
+	/* this will put all memory onto the freelists */
+	totalram_pages_add(memblock_free_all());
+	pr_info("Memory available: %luk/%luk RAM\n",
+		(nr_free_pages() << PAGE_SHIFT) >> 10, mem_size >> 10);
+}
+
+/*
+ * In our case __init memory is not part of the page allocator so there is
+ * nothing to free.
+ */
+void free_initmem(void)
+{
+}
+
+void free_mem(void)
+{
+	lkl_ops->mem_free((void *)_memory_start);
+}
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 04/25] um lkl: memory handling
@ 2020-03-30 14:45     ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: linux-arch, Levente Kurusa, Octavian Purdila, Akira Moroo,
	Hajime Tazaki, linux-kernel-library, Yuan Liu

From: Octavian Purdila <tavi.purdila@gmail.com>

LKL is a non MMU architecture and hence there is not much work left to
do other than initializing the boot allocator and providing the page
and page table definitions.

The backstore memory is allocated via a host operation and the memory
size to be used is specified when the kernel is started, in the
lkl_start_kernel call.

Cc: H.K. Jerry Chu <hkchu@google.com>
Cc: Levente Kurusa <levex@linux.com>
Cc: Yuan Liu <liuyuan@google.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/lkl/include/asm/page.h          | 14 ++++++
 arch/um/lkl/include/asm/pgtable.h       | 57 +++++++++++++++++++++
 arch/um/lkl/include/uapi/asm/host_ops.h |  5 ++
 arch/um/lkl/mm/bootmem.c                | 66 +++++++++++++++++++++++++
 4 files changed, 142 insertions(+)
 create mode 100644 arch/um/lkl/include/asm/page.h
 create mode 100644 arch/um/lkl/include/asm/pgtable.h
 create mode 100644 arch/um/lkl/mm/bootmem.c

diff --git a/arch/um/lkl/include/asm/page.h b/arch/um/lkl/include/asm/page.h
new file mode 100644
index 000000000000..e77f3da22031
--- /dev/null
+++ b/arch/um/lkl/include/asm/page.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_PAGE_H
+#define _ASM_LKL_PAGE_H
+
+#define CONFIG_KERNEL_RAM_BASE_ADDRESS memory_start
+
+#ifndef __ASSEMBLY__
+void free_mem(void);
+void bootmem_init(unsigned long mem_size);
+#endif
+
+#include <asm-generic/page.h>
+
+#endif /* _ASM_LKL_PAGE_H */
diff --git a/arch/um/lkl/include/asm/pgtable.h b/arch/um/lkl/include/asm/pgtable.h
new file mode 100644
index 000000000000..733beb6d53f6
--- /dev/null
+++ b/arch/um/lkl/include/asm/pgtable.h
@@ -0,0 +1,57 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_PGTABLE_H
+#define _LKL_PGTABLE_H
+
+#include <asm-generic/pgtable-nopud.h>
+
+/*
+ * (C) Copyright 2000-2002, Greg Ungerer <gerg@snapgear.com>
+ */
+
+#include <linux/slab.h>
+#include <asm/processor.h>
+#include <asm/io.h>
+
+#define pgd_present(pgd)	(1)
+#define pgd_none(pgd)		(0)
+#define pgd_bad(pgd)		(0)
+#define pgd_clear(pgdp)
+#define kern_addr_valid(addr)	(1)
+#define	pmd_offset(a, b)	((void *)0)
+
+#define PAGE_NONE		__pgprot(0)
+#define PAGE_SHARED		__pgprot(0)
+#define PAGE_COPY		__pgprot(0)
+#define PAGE_READONLY		__pgprot(0)
+#define PAGE_KERNEL		__pgprot(0)
+
+void paging_init(void);
+#define swapper_pg_dir		((pgd_t *)0)
+
+#define __swp_type(x)		(0)
+#define __swp_offset(x)		(0)
+#define __swp_entry(typ, off)	((swp_entry_t) { ((typ) | ((off) << 7)) })
+#define __pte_to_swp_entry(pte)	((swp_entry_t) { pte_val(pte) })
+#define __swp_entry_to_pte(x)	((pte_t) { (x).val })
+
+/*
+ * ZERO_PAGE is a global shared page that is always zero: used
+ * for zero-mapped memory areas etc..
+ */
+extern void *empty_zero_page;
+#define ZERO_PAGE(vaddr)	(virt_to_page(empty_zero_page))
+
+/*
+ * All 32bit addresses are effectively valid for vmalloc...
+ * Sort of meaningless for non-VM targets.
+ */
+#define	VMALLOC_START		0
+#define	VMALLOC_END		0xffffffff
+#define	KMAP_START		0
+#define	KMAP_END		0xffffffff
+
+#include <asm-generic/pgtable.h>
+
+#define check_pgt_cache()	do { } while (0)
+
+#endif
diff --git a/arch/um/lkl/include/uapi/asm/host_ops.h b/arch/um/lkl/include/uapi/asm/host_ops.h
index 7cfb0a93e6a6..6bbc94c120be 100644
--- a/arch/um/lkl/include/uapi/asm/host_ops.h
+++ b/arch/um/lkl/include/uapi/asm/host_ops.h
@@ -17,8 +17,13 @@ struct lkl_jmp_buf {
  * These operations must be provided by a host library or by the application
  * itself.
  *
+ * @mem_alloc - allocate memory
+ * @mem_free - free memory
+ *
  */
 struct lkl_host_operations {
+	void *(*mem_alloc)(unsigned long mem);
+	void (*mem_free)(void *mem);
 };
 
 void lkl_bug(const char *fmt, ...);
diff --git a/arch/um/lkl/mm/bootmem.c b/arch/um/lkl/mm/bootmem.c
new file mode 100644
index 000000000000..39dd0d22b44e
--- /dev/null
+++ b/arch/um/lkl/mm/bootmem.c
@@ -0,0 +1,66 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/memblock.h>
+#include <linux/mm.h>
+#include <linux/swap.h>
+
+unsigned long memory_start, memory_end;
+static unsigned long _memory_start, mem_size;
+
+void *empty_zero_page;
+
+void __init bootmem_init(unsigned long mem_sz)
+{
+	mem_size = mem_sz;
+
+	_memory_start = (unsigned long)lkl_ops->mem_alloc(mem_size);
+	memory_start = _memory_start;
+	WARN_ON(!memory_start);
+	memory_end = memory_start + mem_size;
+
+	if (PAGE_ALIGN(memory_start) != memory_start) {
+		mem_size -= PAGE_ALIGN(memory_start) - memory_start;
+		memory_start = PAGE_ALIGN(memory_start);
+		mem_size = (mem_size / PAGE_SIZE) * PAGE_SIZE;
+	}
+	pr_info("memblock address range: 0x%lx - 0x%lx\n", memory_start,
+		memory_start + mem_size);
+	/*
+	 * Give all the memory to the bootmap allocator, tell it to put the
+	 * boot mem_map at the start of memory.
+	 */
+	max_low_pfn = virt_to_pfn(memory_end);
+	min_low_pfn = virt_to_pfn(memory_start);
+	memblock_add(memory_start, mem_size);
+
+	empty_zero_page = memblock_alloc(PAGE_SIZE, PAGE_SIZE);
+	memset((void *)empty_zero_page, 0, PAGE_SIZE);
+
+	{
+		unsigned long zones_size[MAX_NR_ZONES] = {0, };
+
+		zones_size[ZONE_NORMAL] = (mem_size) >> PAGE_SHIFT;
+		free_area_init(zones_size);
+	}
+}
+
+void __init mem_init(void)
+{
+	max_mapnr = (((unsigned long)high_memory) - PAGE_OFFSET) >> PAGE_SHIFT;
+	/* this will put all memory onto the freelists */
+	totalram_pages_add(memblock_free_all());
+	pr_info("Memory available: %luk/%luk RAM\n",
+		(nr_free_pages() << PAGE_SHIFT) >> 10, mem_size >> 10);
+}
+
+/*
+ * In our case __init memory is not part of the page allocator so there is
+ * nothing to free.
+ */
+void free_initmem(void)
+{
+}
+
+void free_mem(void)
+{
+	lkl_ops->mem_free((void *)_memory_start);
+}
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 05/25] um lkl: kernel threads support
  2020-03-30 14:45   ` Hajime Tazaki
@ 2020-03-30 14:45     ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Lai Jiangshan, Patrick Collins, Yuan Liu, Hajime Tazaki

From: Octavian Purdila <tavi.purdila@gmail.com>

LKL does not support user processes but it must support kernel threads
as part as the normal kernel work-flow. It uses host operations to
create and terminate host threads that are going to run the kernel
threads. It also uses semaphores to synchronize those threads and to
allow the Linux kernel scheduler to control how the kernel threads
run.

Each kernel thread runs in a host threads and has a host semaphore
associated with it - the thread's scheduling semaphore. The semaphore
counter is initialized to 0. The first thing a kernel thread does
after getting spawned, before running any kernel code, is to perform a
down operation to block the thread.

The kernel controls host threads scheduling by performing up and down
operations on the scheduling semaphore. In __switch_context an up
operation on the next thread is performed to wake up a blocked thread,
and a down operation is performed on the prev thread to block it.

A thread is terminated by marking it in free_thread_info and
performing an up operation on the scheduling semaphore at which point
the marked thread will terminate itself.

Cc: Lai Jiangshan <jiangshanlai@gmail.com>
Cc: Patrick Collins <pscollins@google.com>
Cc: Yuan Liu <liuyuan@google.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/lkl/include/asm/thread_info.h   |  70 ++++++++
 arch/um/lkl/include/uapi/asm/host_ops.h |  55 ++++++
 arch/um/lkl/kernel/cpu.c                | 223 +++++++++++++++++++++++
 arch/um/lkl/kernel/threads.c            | 227 ++++++++++++++++++++++++
 4 files changed, 575 insertions(+)
 create mode 100644 arch/um/lkl/include/asm/thread_info.h
 create mode 100644 arch/um/lkl/kernel/cpu.c
 create mode 100644 arch/um/lkl/kernel/threads.c

diff --git a/arch/um/lkl/include/asm/thread_info.h b/arch/um/lkl/include/asm/thread_info.h
new file mode 100644
index 000000000000..da4e75fc7b10
--- /dev/null
+++ b/arch/um/lkl/include/asm/thread_info.h
@@ -0,0 +1,70 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_THREAD_INFO_H
+#define _ASM_LKL_THREAD_INFO_H
+
+#define THREAD_SIZE	       (4096)
+
+#ifndef __ASSEMBLY__
+#include <asm/types.h>
+#include <asm/processor.h>
+#include <asm/host_ops.h>
+
+typedef struct {
+	unsigned long seg;
+} mm_segment_t;
+
+struct thread_info {
+	struct task_struct *task;
+	unsigned long flags;
+	int preempt_count;
+	mm_segment_t addr_limit;
+	struct lkl_sem *sched_sem;
+	struct lkl_jmp_buf sched_jb;
+	bool dead;
+	lkl_thread_t tid;
+	struct task_struct *prev_sched;
+	unsigned long stackend;
+};
+
+#define INIT_THREAD_INFO(tsk)				\
+{							\
+	.task		= &tsk,				\
+	.preempt_count	= INIT_PREEMPT_COUNT,		\
+	.flags		= 0,				\
+	.addr_limit	= KERNEL_DS,			\
+}
+
+/* how to get the thread information struct from C */
+extern struct thread_info *_current_thread_info;
+static inline struct thread_info *current_thread_info(void)
+{
+	return _current_thread_info;
+}
+
+/* thread information allocation */
+unsigned long *alloc_thread_stack_node(struct task_struct *, int node);
+void free_thread_stack(struct task_struct *tsk);
+
+void threads_init(void);
+void threads_cleanup(void);
+
+#define TIF_SYSCALL_TRACE		0
+#define TIF_NOTIFY_RESUME		1
+#define TIF_SIGPENDING			2
+#define TIF_NEED_RESCHED		3
+#define TIF_RESTORE_SIGMASK		4
+#define TIF_MEMDIE			5
+#define TIF_NOHZ			6
+#define TIF_SCHED_JB			7
+#define TIF_HOST_THREAD			8
+
+#define __HAVE_THREAD_FUNCTIONS
+
+#define task_thread_info(task)	((struct thread_info *)(task)->stack)
+#define task_stack_page(task)	((task)->stack)
+void setup_thread_stack(struct task_struct *p, struct task_struct *org);
+#define end_of_stack(p) (&task_thread_info(p)->stackend)
+
+#endif /* __ASSEMBLY__ */
+
+#endif
diff --git a/arch/um/lkl/include/uapi/asm/host_ops.h b/arch/um/lkl/include/uapi/asm/host_ops.h
index 6bbc94c120be..19924fc7c718 100644
--- a/arch/um/lkl/include/uapi/asm/host_ops.h
+++ b/arch/um/lkl/include/uapi/asm/host_ops.h
@@ -17,15 +17,70 @@ struct lkl_jmp_buf {
  * These operations must be provided by a host library or by the application
  * itself.
  *
+ * @sem_alloc - allocate a host semaphore an initialize it to count
+ * @sem_free - free a host semaphore
+ * @sem_up - perform an up operation on the semaphore
+ * @sem_down - perform a down operation on the semaphore
+ *
+ * @mutex_alloc - allocate and initialize a host mutex; the recursive parameter
+ * determines if the mutex is recursive or not
+ * @mutex_free - free a host mutex
+ * @mutex_lock - acquire the mutex
+ * @mutex_unlock - release the mutex
+ *
+ * @thread_create - create a new thread and run f(arg) in its context; returns a
+ * thread handle or 0 if the thread could not be created
+ * @thread_detach - on POSIX systems, free up resources held by
+ * pthreads. Noop on Win32.
+ * @thread_exit - terminates the current thread
+ * @thread_join - wait for the given thread to terminate. Returns 0
+ * for success, -1 otherwise
+ *
  * @mem_alloc - allocate memory
  * @mem_free - free memory
  *
+ * @gettid - returns the host thread id of the caller, which need not
+ * be the same as the handle returned by thread_create
+ *
+ * @jmp_buf_set - runs the give function and setups a jump back point by saving
+ * the context in the jump buffer; jmp_buf_longjmp can be called from the give
+ * function or any callee in that function to return back to the jump back
+ * point
+ *
+ * NOTE: we can't return from jmp_buf_set before calling jmp_buf_longjmp or
+ * otherwise the saved context (stack) is not going to be valid, so we must pass
+ * the function that will eventually call longjmp here
+ *
+ * @jmp_buf_longjmp - perform a jump back to the saved jump buffer
  */
 struct lkl_host_operations {
+	struct lkl_sem *(*sem_alloc)(int count);
+	void (*sem_free)(struct lkl_sem *sem);
+	void (*sem_up)(struct lkl_sem *sem);
+	void (*sem_down)(struct lkl_sem *sem);
+
+	struct lkl_mutex *(*mutex_alloc)(int recursive);
+	void (*mutex_free)(struct lkl_mutex *mutex);
+	void (*mutex_lock)(struct lkl_mutex *mutex);
+	void (*mutex_unlock)(struct lkl_mutex *mutex);
+
+	lkl_thread_t (*thread_create)(void (*f)(void *), void *arg);
+	void (*thread_detach)(void);
+	void (*thread_exit)(void);
+	int (*thread_join)(lkl_thread_t tid);
+	lkl_thread_t (*thread_self)(void);
+	int (*thread_equal)(lkl_thread_t a, lkl_thread_t b);
+
 	void *(*mem_alloc)(unsigned long mem);
 	void (*mem_free)(void *mem);
+
+	long (*gettid)(void);
+
+	void (*jmp_buf_set)(struct lkl_jmp_buf *jmpb, void (*f)(void));
+	void (*jmp_buf_longjmp)(struct lkl_jmp_buf *jmpb, int val);
 };
 
+int lkl_printf(const char *fmt, ...);
 void lkl_bug(const char *fmt, ...);
 
 #endif
diff --git a/arch/um/lkl/kernel/cpu.c b/arch/um/lkl/kernel/cpu.c
new file mode 100644
index 000000000000..125af3b2d5dd
--- /dev/null
+++ b/arch/um/lkl/kernel/cpu.c
@@ -0,0 +1,223 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/kernel.h>
+#include <linux/sched/stat.h>
+#include <asm/host_ops.h>
+#include <asm/cpu.h>
+#include <asm/thread_info.h>
+#include <asm/unistd.h>
+#include <asm/sched.h>
+#include <asm/syscalls.h>
+
+/*
+ * This structure is used to get access to the "LKL CPU" that allows us to run
+ * Linux code. Because we have to deal with various synchronization requirements
+ * between idle thread, system calls, interrupts, "reentrancy", CPU shutdown,
+ * imbalance wake up (i.e. acquire the CPU from one thread and release it from
+ * another), we can't use a simple synchronization mechanism such as (recursive)
+ * mutex or semaphore. Instead, we use a mutex and a bunch of status data plus a
+ * semaphore.
+ */
+static struct lkl_cpu {
+	/* lock that protects the CPU status data */
+	struct lkl_mutex *lock;
+	/*
+	 * Since we must free the cpu lock during shutdown we need a
+	 * synchronization algorithm between lkl_cpu_shutdown() and the CPU
+	 * access functions since lkl_cpu_get() gets called from thread
+	 * destructor callback functions which may be scheduled after
+	 * lkl_cpu_shutdown() has freed the cpu lock.
+	 *
+	 * An atomic counter is used to keep track of the number of running
+	 * CPU access functions and allow the shutdown function to wait for
+	 * them.
+	 *
+	 * The shutdown functions adds MAX_THREADS to this counter which allows
+	 * the CPU access functions to check if the shutdown process has
+	 * started.
+	 *
+	 * This algorithm assumes that we never have more the MAX_THREADS
+	 * requesting CPU access.
+	 */
+	#define MAX_THREADS 1000000
+	unsigned int shutdown_gate;
+	bool irqs_pending;
+	/* no of threads waiting the CPU */
+	unsigned int sleepers;
+	/* no of times the current thread got the CPU */
+	unsigned int count;
+	/* current thread that owns the CPU */
+	lkl_thread_t owner;
+	/* semaphore for threads waiting the CPU */
+	struct lkl_sem *sem;
+	/* semaphore used for shutdown */
+	struct lkl_sem *shutdown_sem;
+} cpu;
+
+static int __cpu_try_get_lock(int n)
+{
+	lkl_thread_t self;
+
+	if (__sync_fetch_and_add(&cpu.shutdown_gate, n) >= MAX_THREADS)
+		return -2;
+
+	lkl_ops->mutex_lock(cpu.lock);
+
+	if (cpu.shutdown_gate >= MAX_THREADS)
+		return -1;
+
+	self = lkl_ops->thread_self();
+
+	if (cpu.owner && !lkl_ops->thread_equal(cpu.owner, self))
+		return 0;
+
+	cpu.owner = self;
+	cpu.count++;
+
+	return 1;
+}
+
+static void __cpu_try_get_unlock(int lock_ret, int n)
+{
+	if (lock_ret >= -1)
+		lkl_ops->mutex_unlock(cpu.lock);
+	__sync_fetch_and_sub(&cpu.shutdown_gate, n);
+}
+
+void lkl_cpu_change_owner(lkl_thread_t owner)
+{
+	lkl_ops->mutex_lock(cpu.lock);
+	if (cpu.count > 1)
+		lkl_bug("bad count while changing owner\n");
+	cpu.owner = owner;
+	lkl_ops->mutex_unlock(cpu.lock);
+}
+
+int lkl_cpu_get(void)
+{
+	int ret;
+
+	ret = __cpu_try_get_lock(1);
+
+	while (ret == 0) {
+		cpu.sleepers++;
+		__cpu_try_get_unlock(ret, 0);
+		lkl_ops->sem_down(cpu.sem);
+		ret = __cpu_try_get_lock(0);
+	}
+
+	__cpu_try_get_unlock(ret, 1);
+
+	return ret;
+}
+
+void lkl_cpu_put(void)
+{
+	lkl_ops->mutex_lock(cpu.lock);
+
+	if (!cpu.count || !cpu.owner ||
+	    !lkl_ops->thread_equal(cpu.owner, lkl_ops->thread_self()))
+		lkl_bug("%s: unbalanced put\n", __func__);
+
+	while (cpu.irqs_pending && !irqs_disabled()) {
+		cpu.irqs_pending = false;
+		lkl_ops->mutex_unlock(cpu.lock);
+		run_irqs();
+		lkl_ops->mutex_lock(cpu.lock);
+	}
+
+	if (test_ti_thread_flag(current_thread_info(), TIF_HOST_THREAD) &&
+	    !single_task_running() && cpu.count == 1) {
+		if (in_interrupt())
+			lkl_bug("%s: in interrupt\n", __func__);
+		lkl_ops->mutex_unlock(cpu.lock);
+		thread_sched_jb();
+		return;
+	}
+
+	if (--cpu.count > 0) {
+		lkl_ops->mutex_unlock(cpu.lock);
+		return;
+	}
+
+	if (cpu.sleepers) {
+		cpu.sleepers--;
+		lkl_ops->sem_up(cpu.sem);
+	}
+
+	cpu.owner = 0;
+
+	lkl_ops->mutex_unlock(cpu.lock);
+}
+
+int lkl_cpu_try_run_irq(int irq)
+{
+	int ret;
+
+	ret = __cpu_try_get_lock(1);
+	if (!ret) {
+		set_irq_pending(irq);
+		cpu.irqs_pending = true;
+	}
+	__cpu_try_get_unlock(ret, 1);
+
+	return ret;
+}
+
+void lkl_cpu_shutdown(void)
+{
+	__sync_fetch_and_add(&cpu.shutdown_gate, MAX_THREADS);
+}
+
+void lkl_cpu_wait_shutdown(void)
+{
+	lkl_ops->sem_down(cpu.shutdown_sem);
+	lkl_ops->sem_free(cpu.shutdown_sem);
+}
+
+static void lkl_cpu_cleanup(bool shutdown)
+{
+	while (__sync_fetch_and_add(&cpu.shutdown_gate, 0) > MAX_THREADS)
+		;
+
+	if (shutdown)
+		lkl_ops->sem_up(cpu.shutdown_sem);
+	else if (cpu.shutdown_sem)
+		lkl_ops->sem_free(cpu.shutdown_sem);
+	if (cpu.sem)
+		lkl_ops->sem_free(cpu.sem);
+	if (cpu.lock)
+		lkl_ops->mutex_free(cpu.lock);
+}
+
+void arch_cpu_idle(void)
+{
+	if (cpu.shutdown_gate >= MAX_THREADS) {
+		lkl_ops->mutex_lock(cpu.lock);
+		while (cpu.sleepers--)
+			lkl_ops->sem_up(cpu.sem);
+		lkl_ops->mutex_unlock(cpu.lock);
+
+		lkl_cpu_cleanup(true);
+
+		lkl_ops->thread_exit();
+	}
+	/* enable irqs now to allow direct irqs to run */
+	local_irq_enable();
+
+	/* switch to idle_host_task */
+	wakeup_idle_host_task();
+}
+
+int lkl_cpu_init(void)
+{
+	cpu.lock = lkl_ops->mutex_alloc(0);
+	cpu.sem = lkl_ops->sem_alloc(0);
+	cpu.shutdown_sem = lkl_ops->sem_alloc(0);
+
+	if (!cpu.lock || !cpu.sem || !cpu.shutdown_sem) {
+		lkl_cpu_cleanup(false);
+		return -ENOMEM;
+	}
+
+	return 0;
+}
diff --git a/arch/um/lkl/kernel/threads.c b/arch/um/lkl/kernel/threads.c
new file mode 100644
index 000000000000..4fe8c56ae5e0
--- /dev/null
+++ b/arch/um/lkl/kernel/threads.c
@@ -0,0 +1,227 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/module.h>
+#include <linux/slab.h>
+#include <linux/sched/task.h>
+#include <linux/sched/signal.h>
+#include <asm/host_ops.h>
+#include <asm/cpu.h>
+#include <asm/sched.h>
+
+static int init_ti(struct thread_info *ti)
+{
+	ti->sched_sem = lkl_ops->sem_alloc(0);
+	if (!ti->sched_sem)
+		return -ENOMEM;
+
+	ti->dead = false;
+	ti->prev_sched = NULL;
+	ti->tid = 0;
+
+	return 0;
+}
+
+unsigned long *alloc_thread_stack_node(struct task_struct *task, int node)
+{
+	struct thread_info *ti;
+
+	ti = kmalloc(sizeof(*ti), GFP_KERNEL);
+	if (!ti)
+		return NULL;
+
+	if (init_ti(ti)) {
+		kfree(ti);
+		return NULL;
+	}
+	ti->task = task;
+
+	return (unsigned long *)ti;
+}
+
+/*
+ * The only new tasks created are kernel threads that have a predefined starting
+ * point thus no stack copy is required.
+ */
+void setup_thread_stack(struct task_struct *p, struct task_struct *org)
+{
+	struct thread_info *ti = task_thread_info(p);
+	struct thread_info *org_ti = task_thread_info(org);
+
+	ti->flags = org_ti->flags;
+	ti->preempt_count = org_ti->preempt_count;
+	ti->addr_limit = org_ti->addr_limit;
+}
+
+static void kill_thread(struct thread_info *ti)
+{
+	if (!test_ti_thread_flag(ti, TIF_HOST_THREAD)) {
+		ti->dead = true;
+		lkl_ops->sem_up(ti->sched_sem);
+		lkl_ops->thread_join(ti->tid);
+	}
+	lkl_ops->sem_free(ti->sched_sem);
+}
+
+void free_thread_stack(struct task_struct *tsk)
+{
+	struct thread_info *ti = task_thread_info(tsk);
+
+	kill_thread(ti);
+	kfree(ti);
+}
+
+struct thread_info *_current_thread_info = &init_thread_union.thread_info;
+
+/*
+ * schedule() expects the return of this function to be the task that we
+ * switched away from. Returning prev is not going to work because we are
+ * actually going to return the previous taks that was scheduled before the
+ * task we are going to wake up, and not the current task, e.g.:
+ *
+ * swapper -> init: saved prev on swapper stack is swapper
+ * init -> ksoftirqd0: saved prev on init stack is init
+ * ksoftirqd0 -> swapper: returned prev is swapper
+ */
+static struct task_struct *abs_prev = &init_task;
+
+struct task_struct *__switch_to(struct task_struct *prev,
+				struct task_struct *next)
+{
+	struct thread_info *_prev = task_thread_info(prev);
+	struct thread_info *_next = task_thread_info(next);
+	unsigned long _prev_flags = _prev->flags;
+	struct lkl_jmp_buf _prev_jb;
+
+	_current_thread_info = task_thread_info(next);
+	_next->prev_sched = prev;
+	abs_prev = prev;
+
+	BUG_ON(!_next->tid);
+	lkl_cpu_change_owner(_next->tid);
+
+	if (test_bit(TIF_SCHED_JB, &_prev_flags)) {
+		/* Atomic. Must be done before wakeup next */
+		clear_ti_thread_flag(_prev, TIF_SCHED_JB);
+		_prev_jb = _prev->sched_jb;
+	}
+
+	lkl_ops->sem_up(_next->sched_sem);
+	if (test_bit(TIF_SCHED_JB, &_prev_flags))
+		lkl_ops->jmp_buf_longjmp(&_prev_jb, 1);
+	else
+		lkl_ops->sem_down(_prev->sched_sem);
+
+	if (_prev->dead)
+		lkl_ops->thread_exit();
+
+	return abs_prev;
+}
+
+int host_task_stub(void *unused)
+{
+	return 0;
+}
+
+void switch_to_host_task(struct task_struct *task)
+{
+	if (WARN_ON(!test_tsk_thread_flag(task, TIF_HOST_THREAD)))
+		return;
+
+	task_thread_info(task)->tid = lkl_ops->thread_self();
+
+	if (current == task)
+		return;
+
+	wake_up_process(task);
+	thread_sched_jb();
+	lkl_ops->sem_down(task_thread_info(task)->sched_sem);
+	schedule_tail(abs_prev);
+}
+
+struct thread_bootstrap_arg {
+	struct thread_info *ti;
+	int (*f)(void *arg);
+	void *arg;
+};
+
+static void thread_bootstrap(void *_tba)
+{
+	struct thread_bootstrap_arg *tba = (struct thread_bootstrap_arg *)_tba;
+	struct thread_info *ti = tba->ti;
+	int (*f)(void *) = tba->f;
+	void *arg = tba->arg;
+
+	lkl_ops->sem_down(ti->sched_sem);
+	kfree(tba);
+	if (ti->prev_sched)
+		schedule_tail(ti->prev_sched);
+
+	f(arg);
+	do_exit(0);
+}
+
+int copy_thread(unsigned long clone_flags, unsigned long esp,
+		unsigned long unused, struct task_struct *p)
+{
+	struct thread_info *ti = task_thread_info(p);
+	struct thread_bootstrap_arg *tba;
+
+	if ((int (*)(void *))esp == host_task_stub) {
+		set_ti_thread_flag(ti, TIF_HOST_THREAD);
+		return 0;
+	}
+
+	tba = kmalloc(sizeof(*tba), GFP_KERNEL);
+	if (!tba)
+		return -ENOMEM;
+
+	tba->f = (int (*)(void *))esp;
+	tba->arg = (void *)unused;
+	tba->ti = ti;
+
+	ti->tid = lkl_ops->thread_create(thread_bootstrap, tba);
+	if (!ti->tid) {
+		kfree(tba);
+		return -ENOMEM;
+	}
+
+	return 0;
+}
+
+void show_stack(struct task_struct *task, unsigned long *esp)
+{
+}
+
+/**
+ * This is called before the kernel initializes, so no kernel calls (including
+ * printk) can't be made yet.
+ */
+void threads_init(void)
+{
+	int ret;
+	struct thread_info *ti = &init_thread_union.thread_info;
+
+	ret = init_ti(ti);
+	if (ret < 0)
+		lkl_printf("lkl: failed to allocate init schedule semaphore\n");
+
+	ti->tid = lkl_ops->thread_self();
+}
+
+void threads_cleanup(void)
+{
+	struct task_struct *p, *t;
+
+	for_each_process_thread(p, t) {
+		struct thread_info *ti = task_thread_info(t);
+
+		if (t->pid != 1 && !test_ti_thread_flag(ti, TIF_HOST_THREAD))
+			WARN(!(t->flags & PF_KTHREAD),
+			     "non kernel thread task %s\n", t->comm);
+		WARN(t->state == TASK_RUNNING,
+		     "thread %s still running while halting\n", t->comm);
+
+		kill_thread(ti);
+	}
+
+	lkl_ops->sem_free(init_thread_union.thread_info.sched_sem);
+}
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 05/25] um lkl: kernel threads support
@ 2020-03-30 14:45     ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: linux-arch, Hajime Tazaki, Octavian Purdila, Lai Jiangshan,
	Akira Moroo, Patrick Collins, linux-kernel-library, Yuan Liu

From: Octavian Purdila <tavi.purdila@gmail.com>

LKL does not support user processes but it must support kernel threads
as part as the normal kernel work-flow. It uses host operations to
create and terminate host threads that are going to run the kernel
threads. It also uses semaphores to synchronize those threads and to
allow the Linux kernel scheduler to control how the kernel threads
run.

Each kernel thread runs in a host threads and has a host semaphore
associated with it - the thread's scheduling semaphore. The semaphore
counter is initialized to 0. The first thing a kernel thread does
after getting spawned, before running any kernel code, is to perform a
down operation to block the thread.

The kernel controls host threads scheduling by performing up and down
operations on the scheduling semaphore. In __switch_context an up
operation on the next thread is performed to wake up a blocked thread,
and a down operation is performed on the prev thread to block it.

A thread is terminated by marking it in free_thread_info and
performing an up operation on the scheduling semaphore at which point
the marked thread will terminate itself.

Cc: Lai Jiangshan <jiangshanlai@gmail.com>
Cc: Patrick Collins <pscollins@google.com>
Cc: Yuan Liu <liuyuan@google.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/lkl/include/asm/thread_info.h   |  70 ++++++++
 arch/um/lkl/include/uapi/asm/host_ops.h |  55 ++++++
 arch/um/lkl/kernel/cpu.c                | 223 +++++++++++++++++++++++
 arch/um/lkl/kernel/threads.c            | 227 ++++++++++++++++++++++++
 4 files changed, 575 insertions(+)
 create mode 100644 arch/um/lkl/include/asm/thread_info.h
 create mode 100644 arch/um/lkl/kernel/cpu.c
 create mode 100644 arch/um/lkl/kernel/threads.c

diff --git a/arch/um/lkl/include/asm/thread_info.h b/arch/um/lkl/include/asm/thread_info.h
new file mode 100644
index 000000000000..da4e75fc7b10
--- /dev/null
+++ b/arch/um/lkl/include/asm/thread_info.h
@@ -0,0 +1,70 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_THREAD_INFO_H
+#define _ASM_LKL_THREAD_INFO_H
+
+#define THREAD_SIZE	       (4096)
+
+#ifndef __ASSEMBLY__
+#include <asm/types.h>
+#include <asm/processor.h>
+#include <asm/host_ops.h>
+
+typedef struct {
+	unsigned long seg;
+} mm_segment_t;
+
+struct thread_info {
+	struct task_struct *task;
+	unsigned long flags;
+	int preempt_count;
+	mm_segment_t addr_limit;
+	struct lkl_sem *sched_sem;
+	struct lkl_jmp_buf sched_jb;
+	bool dead;
+	lkl_thread_t tid;
+	struct task_struct *prev_sched;
+	unsigned long stackend;
+};
+
+#define INIT_THREAD_INFO(tsk)				\
+{							\
+	.task		= &tsk,				\
+	.preempt_count	= INIT_PREEMPT_COUNT,		\
+	.flags		= 0,				\
+	.addr_limit	= KERNEL_DS,			\
+}
+
+/* how to get the thread information struct from C */
+extern struct thread_info *_current_thread_info;
+static inline struct thread_info *current_thread_info(void)
+{
+	return _current_thread_info;
+}
+
+/* thread information allocation */
+unsigned long *alloc_thread_stack_node(struct task_struct *, int node);
+void free_thread_stack(struct task_struct *tsk);
+
+void threads_init(void);
+void threads_cleanup(void);
+
+#define TIF_SYSCALL_TRACE		0
+#define TIF_NOTIFY_RESUME		1
+#define TIF_SIGPENDING			2
+#define TIF_NEED_RESCHED		3
+#define TIF_RESTORE_SIGMASK		4
+#define TIF_MEMDIE			5
+#define TIF_NOHZ			6
+#define TIF_SCHED_JB			7
+#define TIF_HOST_THREAD			8
+
+#define __HAVE_THREAD_FUNCTIONS
+
+#define task_thread_info(task)	((struct thread_info *)(task)->stack)
+#define task_stack_page(task)	((task)->stack)
+void setup_thread_stack(struct task_struct *p, struct task_struct *org);
+#define end_of_stack(p) (&task_thread_info(p)->stackend)
+
+#endif /* __ASSEMBLY__ */
+
+#endif
diff --git a/arch/um/lkl/include/uapi/asm/host_ops.h b/arch/um/lkl/include/uapi/asm/host_ops.h
index 6bbc94c120be..19924fc7c718 100644
--- a/arch/um/lkl/include/uapi/asm/host_ops.h
+++ b/arch/um/lkl/include/uapi/asm/host_ops.h
@@ -17,15 +17,70 @@ struct lkl_jmp_buf {
  * These operations must be provided by a host library or by the application
  * itself.
  *
+ * @sem_alloc - allocate a host semaphore an initialize it to count
+ * @sem_free - free a host semaphore
+ * @sem_up - perform an up operation on the semaphore
+ * @sem_down - perform a down operation on the semaphore
+ *
+ * @mutex_alloc - allocate and initialize a host mutex; the recursive parameter
+ * determines if the mutex is recursive or not
+ * @mutex_free - free a host mutex
+ * @mutex_lock - acquire the mutex
+ * @mutex_unlock - release the mutex
+ *
+ * @thread_create - create a new thread and run f(arg) in its context; returns a
+ * thread handle or 0 if the thread could not be created
+ * @thread_detach - on POSIX systems, free up resources held by
+ * pthreads. Noop on Win32.
+ * @thread_exit - terminates the current thread
+ * @thread_join - wait for the given thread to terminate. Returns 0
+ * for success, -1 otherwise
+ *
  * @mem_alloc - allocate memory
  * @mem_free - free memory
  *
+ * @gettid - returns the host thread id of the caller, which need not
+ * be the same as the handle returned by thread_create
+ *
+ * @jmp_buf_set - runs the give function and setups a jump back point by saving
+ * the context in the jump buffer; jmp_buf_longjmp can be called from the give
+ * function or any callee in that function to return back to the jump back
+ * point
+ *
+ * NOTE: we can't return from jmp_buf_set before calling jmp_buf_longjmp or
+ * otherwise the saved context (stack) is not going to be valid, so we must pass
+ * the function that will eventually call longjmp here
+ *
+ * @jmp_buf_longjmp - perform a jump back to the saved jump buffer
  */
 struct lkl_host_operations {
+	struct lkl_sem *(*sem_alloc)(int count);
+	void (*sem_free)(struct lkl_sem *sem);
+	void (*sem_up)(struct lkl_sem *sem);
+	void (*sem_down)(struct lkl_sem *sem);
+
+	struct lkl_mutex *(*mutex_alloc)(int recursive);
+	void (*mutex_free)(struct lkl_mutex *mutex);
+	void (*mutex_lock)(struct lkl_mutex *mutex);
+	void (*mutex_unlock)(struct lkl_mutex *mutex);
+
+	lkl_thread_t (*thread_create)(void (*f)(void *), void *arg);
+	void (*thread_detach)(void);
+	void (*thread_exit)(void);
+	int (*thread_join)(lkl_thread_t tid);
+	lkl_thread_t (*thread_self)(void);
+	int (*thread_equal)(lkl_thread_t a, lkl_thread_t b);
+
 	void *(*mem_alloc)(unsigned long mem);
 	void (*mem_free)(void *mem);
+
+	long (*gettid)(void);
+
+	void (*jmp_buf_set)(struct lkl_jmp_buf *jmpb, void (*f)(void));
+	void (*jmp_buf_longjmp)(struct lkl_jmp_buf *jmpb, int val);
 };
 
+int lkl_printf(const char *fmt, ...);
 void lkl_bug(const char *fmt, ...);
 
 #endif
diff --git a/arch/um/lkl/kernel/cpu.c b/arch/um/lkl/kernel/cpu.c
new file mode 100644
index 000000000000..125af3b2d5dd
--- /dev/null
+++ b/arch/um/lkl/kernel/cpu.c
@@ -0,0 +1,223 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/kernel.h>
+#include <linux/sched/stat.h>
+#include <asm/host_ops.h>
+#include <asm/cpu.h>
+#include <asm/thread_info.h>
+#include <asm/unistd.h>
+#include <asm/sched.h>
+#include <asm/syscalls.h>
+
+/*
+ * This structure is used to get access to the "LKL CPU" that allows us to run
+ * Linux code. Because we have to deal with various synchronization requirements
+ * between idle thread, system calls, interrupts, "reentrancy", CPU shutdown,
+ * imbalance wake up (i.e. acquire the CPU from one thread and release it from
+ * another), we can't use a simple synchronization mechanism such as (recursive)
+ * mutex or semaphore. Instead, we use a mutex and a bunch of status data plus a
+ * semaphore.
+ */
+static struct lkl_cpu {
+	/* lock that protects the CPU status data */
+	struct lkl_mutex *lock;
+	/*
+	 * Since we must free the cpu lock during shutdown we need a
+	 * synchronization algorithm between lkl_cpu_shutdown() and the CPU
+	 * access functions since lkl_cpu_get() gets called from thread
+	 * destructor callback functions which may be scheduled after
+	 * lkl_cpu_shutdown() has freed the cpu lock.
+	 *
+	 * An atomic counter is used to keep track of the number of running
+	 * CPU access functions and allow the shutdown function to wait for
+	 * them.
+	 *
+	 * The shutdown functions adds MAX_THREADS to this counter which allows
+	 * the CPU access functions to check if the shutdown process has
+	 * started.
+	 *
+	 * This algorithm assumes that we never have more the MAX_THREADS
+	 * requesting CPU access.
+	 */
+	#define MAX_THREADS 1000000
+	unsigned int shutdown_gate;
+	bool irqs_pending;
+	/* no of threads waiting the CPU */
+	unsigned int sleepers;
+	/* no of times the current thread got the CPU */
+	unsigned int count;
+	/* current thread that owns the CPU */
+	lkl_thread_t owner;
+	/* semaphore for threads waiting the CPU */
+	struct lkl_sem *sem;
+	/* semaphore used for shutdown */
+	struct lkl_sem *shutdown_sem;
+} cpu;
+
+static int __cpu_try_get_lock(int n)
+{
+	lkl_thread_t self;
+
+	if (__sync_fetch_and_add(&cpu.shutdown_gate, n) >= MAX_THREADS)
+		return -2;
+
+	lkl_ops->mutex_lock(cpu.lock);
+
+	if (cpu.shutdown_gate >= MAX_THREADS)
+		return -1;
+
+	self = lkl_ops->thread_self();
+
+	if (cpu.owner && !lkl_ops->thread_equal(cpu.owner, self))
+		return 0;
+
+	cpu.owner = self;
+	cpu.count++;
+
+	return 1;
+}
+
+static void __cpu_try_get_unlock(int lock_ret, int n)
+{
+	if (lock_ret >= -1)
+		lkl_ops->mutex_unlock(cpu.lock);
+	__sync_fetch_and_sub(&cpu.shutdown_gate, n);
+}
+
+void lkl_cpu_change_owner(lkl_thread_t owner)
+{
+	lkl_ops->mutex_lock(cpu.lock);
+	if (cpu.count > 1)
+		lkl_bug("bad count while changing owner\n");
+	cpu.owner = owner;
+	lkl_ops->mutex_unlock(cpu.lock);
+}
+
+int lkl_cpu_get(void)
+{
+	int ret;
+
+	ret = __cpu_try_get_lock(1);
+
+	while (ret == 0) {
+		cpu.sleepers++;
+		__cpu_try_get_unlock(ret, 0);
+		lkl_ops->sem_down(cpu.sem);
+		ret = __cpu_try_get_lock(0);
+	}
+
+	__cpu_try_get_unlock(ret, 1);
+
+	return ret;
+}
+
+void lkl_cpu_put(void)
+{
+	lkl_ops->mutex_lock(cpu.lock);
+
+	if (!cpu.count || !cpu.owner ||
+	    !lkl_ops->thread_equal(cpu.owner, lkl_ops->thread_self()))
+		lkl_bug("%s: unbalanced put\n", __func__);
+
+	while (cpu.irqs_pending && !irqs_disabled()) {
+		cpu.irqs_pending = false;
+		lkl_ops->mutex_unlock(cpu.lock);
+		run_irqs();
+		lkl_ops->mutex_lock(cpu.lock);
+	}
+
+	if (test_ti_thread_flag(current_thread_info(), TIF_HOST_THREAD) &&
+	    !single_task_running() && cpu.count == 1) {
+		if (in_interrupt())
+			lkl_bug("%s: in interrupt\n", __func__);
+		lkl_ops->mutex_unlock(cpu.lock);
+		thread_sched_jb();
+		return;
+	}
+
+	if (--cpu.count > 0) {
+		lkl_ops->mutex_unlock(cpu.lock);
+		return;
+	}
+
+	if (cpu.sleepers) {
+		cpu.sleepers--;
+		lkl_ops->sem_up(cpu.sem);
+	}
+
+	cpu.owner = 0;
+
+	lkl_ops->mutex_unlock(cpu.lock);
+}
+
+int lkl_cpu_try_run_irq(int irq)
+{
+	int ret;
+
+	ret = __cpu_try_get_lock(1);
+	if (!ret) {
+		set_irq_pending(irq);
+		cpu.irqs_pending = true;
+	}
+	__cpu_try_get_unlock(ret, 1);
+
+	return ret;
+}
+
+void lkl_cpu_shutdown(void)
+{
+	__sync_fetch_and_add(&cpu.shutdown_gate, MAX_THREADS);
+}
+
+void lkl_cpu_wait_shutdown(void)
+{
+	lkl_ops->sem_down(cpu.shutdown_sem);
+	lkl_ops->sem_free(cpu.shutdown_sem);
+}
+
+static void lkl_cpu_cleanup(bool shutdown)
+{
+	while (__sync_fetch_and_add(&cpu.shutdown_gate, 0) > MAX_THREADS)
+		;
+
+	if (shutdown)
+		lkl_ops->sem_up(cpu.shutdown_sem);
+	else if (cpu.shutdown_sem)
+		lkl_ops->sem_free(cpu.shutdown_sem);
+	if (cpu.sem)
+		lkl_ops->sem_free(cpu.sem);
+	if (cpu.lock)
+		lkl_ops->mutex_free(cpu.lock);
+}
+
+void arch_cpu_idle(void)
+{
+	if (cpu.shutdown_gate >= MAX_THREADS) {
+		lkl_ops->mutex_lock(cpu.lock);
+		while (cpu.sleepers--)
+			lkl_ops->sem_up(cpu.sem);
+		lkl_ops->mutex_unlock(cpu.lock);
+
+		lkl_cpu_cleanup(true);
+
+		lkl_ops->thread_exit();
+	}
+	/* enable irqs now to allow direct irqs to run */
+	local_irq_enable();
+
+	/* switch to idle_host_task */
+	wakeup_idle_host_task();
+}
+
+int lkl_cpu_init(void)
+{
+	cpu.lock = lkl_ops->mutex_alloc(0);
+	cpu.sem = lkl_ops->sem_alloc(0);
+	cpu.shutdown_sem = lkl_ops->sem_alloc(0);
+
+	if (!cpu.lock || !cpu.sem || !cpu.shutdown_sem) {
+		lkl_cpu_cleanup(false);
+		return -ENOMEM;
+	}
+
+	return 0;
+}
diff --git a/arch/um/lkl/kernel/threads.c b/arch/um/lkl/kernel/threads.c
new file mode 100644
index 000000000000..4fe8c56ae5e0
--- /dev/null
+++ b/arch/um/lkl/kernel/threads.c
@@ -0,0 +1,227 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/module.h>
+#include <linux/slab.h>
+#include <linux/sched/task.h>
+#include <linux/sched/signal.h>
+#include <asm/host_ops.h>
+#include <asm/cpu.h>
+#include <asm/sched.h>
+
+static int init_ti(struct thread_info *ti)
+{
+	ti->sched_sem = lkl_ops->sem_alloc(0);
+	if (!ti->sched_sem)
+		return -ENOMEM;
+
+	ti->dead = false;
+	ti->prev_sched = NULL;
+	ti->tid = 0;
+
+	return 0;
+}
+
+unsigned long *alloc_thread_stack_node(struct task_struct *task, int node)
+{
+	struct thread_info *ti;
+
+	ti = kmalloc(sizeof(*ti), GFP_KERNEL);
+	if (!ti)
+		return NULL;
+
+	if (init_ti(ti)) {
+		kfree(ti);
+		return NULL;
+	}
+	ti->task = task;
+
+	return (unsigned long *)ti;
+}
+
+/*
+ * The only new tasks created are kernel threads that have a predefined starting
+ * point thus no stack copy is required.
+ */
+void setup_thread_stack(struct task_struct *p, struct task_struct *org)
+{
+	struct thread_info *ti = task_thread_info(p);
+	struct thread_info *org_ti = task_thread_info(org);
+
+	ti->flags = org_ti->flags;
+	ti->preempt_count = org_ti->preempt_count;
+	ti->addr_limit = org_ti->addr_limit;
+}
+
+static void kill_thread(struct thread_info *ti)
+{
+	if (!test_ti_thread_flag(ti, TIF_HOST_THREAD)) {
+		ti->dead = true;
+		lkl_ops->sem_up(ti->sched_sem);
+		lkl_ops->thread_join(ti->tid);
+	}
+	lkl_ops->sem_free(ti->sched_sem);
+}
+
+void free_thread_stack(struct task_struct *tsk)
+{
+	struct thread_info *ti = task_thread_info(tsk);
+
+	kill_thread(ti);
+	kfree(ti);
+}
+
+struct thread_info *_current_thread_info = &init_thread_union.thread_info;
+
+/*
+ * schedule() expects the return of this function to be the task that we
+ * switched away from. Returning prev is not going to work because we are
+ * actually going to return the previous taks that was scheduled before the
+ * task we are going to wake up, and not the current task, e.g.:
+ *
+ * swapper -> init: saved prev on swapper stack is swapper
+ * init -> ksoftirqd0: saved prev on init stack is init
+ * ksoftirqd0 -> swapper: returned prev is swapper
+ */
+static struct task_struct *abs_prev = &init_task;
+
+struct task_struct *__switch_to(struct task_struct *prev,
+				struct task_struct *next)
+{
+	struct thread_info *_prev = task_thread_info(prev);
+	struct thread_info *_next = task_thread_info(next);
+	unsigned long _prev_flags = _prev->flags;
+	struct lkl_jmp_buf _prev_jb;
+
+	_current_thread_info = task_thread_info(next);
+	_next->prev_sched = prev;
+	abs_prev = prev;
+
+	BUG_ON(!_next->tid);
+	lkl_cpu_change_owner(_next->tid);
+
+	if (test_bit(TIF_SCHED_JB, &_prev_flags)) {
+		/* Atomic. Must be done before wakeup next */
+		clear_ti_thread_flag(_prev, TIF_SCHED_JB);
+		_prev_jb = _prev->sched_jb;
+	}
+
+	lkl_ops->sem_up(_next->sched_sem);
+	if (test_bit(TIF_SCHED_JB, &_prev_flags))
+		lkl_ops->jmp_buf_longjmp(&_prev_jb, 1);
+	else
+		lkl_ops->sem_down(_prev->sched_sem);
+
+	if (_prev->dead)
+		lkl_ops->thread_exit();
+
+	return abs_prev;
+}
+
+int host_task_stub(void *unused)
+{
+	return 0;
+}
+
+void switch_to_host_task(struct task_struct *task)
+{
+	if (WARN_ON(!test_tsk_thread_flag(task, TIF_HOST_THREAD)))
+		return;
+
+	task_thread_info(task)->tid = lkl_ops->thread_self();
+
+	if (current == task)
+		return;
+
+	wake_up_process(task);
+	thread_sched_jb();
+	lkl_ops->sem_down(task_thread_info(task)->sched_sem);
+	schedule_tail(abs_prev);
+}
+
+struct thread_bootstrap_arg {
+	struct thread_info *ti;
+	int (*f)(void *arg);
+	void *arg;
+};
+
+static void thread_bootstrap(void *_tba)
+{
+	struct thread_bootstrap_arg *tba = (struct thread_bootstrap_arg *)_tba;
+	struct thread_info *ti = tba->ti;
+	int (*f)(void *) = tba->f;
+	void *arg = tba->arg;
+
+	lkl_ops->sem_down(ti->sched_sem);
+	kfree(tba);
+	if (ti->prev_sched)
+		schedule_tail(ti->prev_sched);
+
+	f(arg);
+	do_exit(0);
+}
+
+int copy_thread(unsigned long clone_flags, unsigned long esp,
+		unsigned long unused, struct task_struct *p)
+{
+	struct thread_info *ti = task_thread_info(p);
+	struct thread_bootstrap_arg *tba;
+
+	if ((int (*)(void *))esp == host_task_stub) {
+		set_ti_thread_flag(ti, TIF_HOST_THREAD);
+		return 0;
+	}
+
+	tba = kmalloc(sizeof(*tba), GFP_KERNEL);
+	if (!tba)
+		return -ENOMEM;
+
+	tba->f = (int (*)(void *))esp;
+	tba->arg = (void *)unused;
+	tba->ti = ti;
+
+	ti->tid = lkl_ops->thread_create(thread_bootstrap, tba);
+	if (!ti->tid) {
+		kfree(tba);
+		return -ENOMEM;
+	}
+
+	return 0;
+}
+
+void show_stack(struct task_struct *task, unsigned long *esp)
+{
+}
+
+/**
+ * This is called before the kernel initializes, so no kernel calls (including
+ * printk) can't be made yet.
+ */
+void threads_init(void)
+{
+	int ret;
+	struct thread_info *ti = &init_thread_union.thread_info;
+
+	ret = init_ti(ti);
+	if (ret < 0)
+		lkl_printf("lkl: failed to allocate init schedule semaphore\n");
+
+	ti->tid = lkl_ops->thread_self();
+}
+
+void threads_cleanup(void)
+{
+	struct task_struct *p, *t;
+
+	for_each_process_thread(p, t) {
+		struct thread_info *ti = task_thread_info(t);
+
+		if (t->pid != 1 && !test_ti_thread_flag(ti, TIF_HOST_THREAD))
+			WARN(!(t->flags & PF_KTHREAD),
+			     "non kernel thread task %s\n", t->comm);
+		WARN(t->state == TASK_RUNNING,
+		     "thread %s still running while halting\n", t->comm);
+
+		kill_thread(ti);
+	}
+
+	lkl_ops->sem_free(init_thread_union.thread_info.sched_sem);
+}
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 06/25] um lkl: interrupt support
  2020-03-30 14:45   ` Hajime Tazaki
@ 2020-03-30 14:45     ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Michael Zimmermann, Hajime Tazaki

From: Octavian Purdila <tavi.purdila@gmail.com>

Add APIs that allows the host to reserve and free and interrupt number
and also to trigger an interrupt.

The trigger operation will simply store the interrupt data in
queue. The interrupt handler is run later, at the first opportunity it
has to avoid races with any kernel threads.

Currently, interrupts are run on the first interrupt enable operation
if interrupts are disabled and if we are not already in interrupt
context.

When triggering an interrupt, it uses GCC's built-in functions for
atomic memory access to synchronize and simple boolean flags.

Cc: Michael Zimmermann <sigmaepsilon92@gmail.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/lkl/include/asm/irq.h             |  13 ++
 arch/um/lkl/include/uapi/asm/irq.h        |  36 ++++
 arch/um/lkl/include/uapi/asm/sigcontext.h |  16 ++
 arch/um/lkl/kernel/irq.c                  | 190 ++++++++++++++++++++++
 4 files changed, 255 insertions(+)
 create mode 100644 arch/um/lkl/include/asm/irq.h
 create mode 100644 arch/um/lkl/include/uapi/asm/irq.h
 create mode 100644 arch/um/lkl/include/uapi/asm/sigcontext.h
 create mode 100644 arch/um/lkl/kernel/irq.c

diff --git a/arch/um/lkl/include/asm/irq.h b/arch/um/lkl/include/asm/irq.h
new file mode 100644
index 000000000000..948fc54cb76c
--- /dev/null
+++ b/arch/um/lkl/include/asm/irq.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_IRQ_H
+#define _ASM_LKL_IRQ_H
+
+#define IRQ_STATUS_BITS		(sizeof(long) * 8)
+#define NR_IRQS			((int)(IRQ_STATUS_BITS * IRQ_STATUS_BITS))
+
+void run_irqs(void);
+void set_irq_pending(int irq);
+
+#include <uapi/asm/irq.h>
+
+#endif
diff --git a/arch/um/lkl/include/uapi/asm/irq.h b/arch/um/lkl/include/uapi/asm/irq.h
new file mode 100644
index 000000000000..666628b233eb
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/irq.h
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _ASM_UAPI_LKL_IRQ_H
+#define _ASM_UAPI_LKL_IRQ_H
+
+/**
+ * lkl_trigger_irq - generate an interrupt
+ *
+ * This function is used by the device host side to signal its Linux counterpart
+ * that some event happened.
+ *
+ * @irq - the irq number to signal
+ */
+int lkl_trigger_irq(int irq);
+
+/**
+ * lkl_get_free_irq - find and reserve a free IRQ number
+ *
+ * This function is called by the host device code to find an unused IRQ number
+ * and reserved it for its own use.
+ *
+ * @user - a string to identify the user
+ * @returns - and irq number that can be used by request_irq or an negative
+ * value in case of an error
+ */
+int lkl_get_free_irq(const char *user);
+
+/**
+ * lkl_put_irq - release an IRQ number previously obtained with lkl_get_free_irq
+ *
+ * @irq - irq number to release
+ * @user - string identifying the user; should be the same as the one passed to
+ * lkl_get_free_irq when the irq number was obtained
+ */
+void lkl_put_irq(int irq, const char *name);
+
+#endif
diff --git a/arch/um/lkl/include/uapi/asm/sigcontext.h b/arch/um/lkl/include/uapi/asm/sigcontext.h
new file mode 100644
index 000000000000..2f4848843d1d
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/sigcontext.h
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _ASM_UAPI_LKL_SIGCONTEXT_H
+#define _ASM_UAPI_LKL_SIGCONTEXT_H
+
+#include <asm/ptrace.h>
+
+struct pt_regs {
+	void *irq_data;
+};
+
+struct sigcontext {
+	struct pt_regs regs;
+	unsigned long oldmask;
+};
+
+#endif
diff --git a/arch/um/lkl/kernel/irq.c b/arch/um/lkl/kernel/irq.c
new file mode 100644
index 000000000000..c794412f85d9
--- /dev/null
+++ b/arch/um/lkl/kernel/irq.c
@@ -0,0 +1,190 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/kernel.h>
+#include <linux/init.h>
+#include <linux/irq.h>
+#include <linux/hardirq.h>
+#include <asm/irq_regs.h>
+#include <linux/sched.h>
+#include <linux/seq_file.h>
+#include <linux/tick.h>
+#include <asm/irqflags.h>
+#include <asm/host_ops.h>
+#include <asm/cpu.h>
+
+/*
+ * To avoid much overhead we use an indirect approach: the irqs are marked using
+ * a bitmap (array of longs) and a summary of the modified bits is kept in a
+ * separate "index" long - one bit for each sizeof(long). Thus we can support
+ * 4096 irqs on 64bit platforms and 1024 irqs on 32bit platforms.
+ *
+ * Whenever an irq is trigger both the array and the index is updated. To find
+ * which irqs were triggered we first search the index and then the
+ * corresponding part of the arrary.
+ */
+static unsigned long irq_status[NR_IRQS / IRQ_STATUS_BITS];
+static unsigned long irq_index_status;
+
+static inline unsigned long test_and_clear_irq_index_status(void)
+{
+	if (!irq_index_status)
+		return 0;
+	return __sync_fetch_and_and(&irq_index_status, 0);
+}
+
+static inline unsigned long test_and_clear_irq_status(int index)
+{
+	if (!&irq_status[index])
+		return 0;
+	return __sync_fetch_and_and(&irq_status[index], 0);
+}
+
+void set_irq_pending(int irq)
+{
+	int index = irq / IRQ_STATUS_BITS;
+	int bit = irq % IRQ_STATUS_BITS;
+
+	__sync_fetch_and_or(&irq_status[index], BIT(bit));
+	__sync_fetch_and_or(&irq_index_status, BIT(index));
+}
+
+static struct irq_info {
+	const char *user;
+} irqs[NR_IRQS];
+
+static bool irqs_enabled;
+
+static struct pt_regs dummy;
+
+static void run_irq(int irq)
+{
+	unsigned long flags;
+	struct pt_regs *old_regs = set_irq_regs((struct pt_regs *)&dummy);
+
+	/* interrupt handlers need to run with interrupts disabled */
+	local_irq_save(flags);
+	irq_enter();
+	generic_handle_irq(irq);
+	irq_exit();
+	set_irq_regs(old_regs);
+	local_irq_restore(flags);
+}
+
+/**
+ * This function can be called from arbitrary host threads, so do not
+ * issue any Linux calls (e.g. prink) if lkl_cpu_get() was not issued
+ * before.
+ */
+int lkl_trigger_irq(int irq)
+{
+	int ret;
+
+	if (!irq || irq > NR_IRQS)
+		return -EINVAL;
+
+	ret = lkl_cpu_try_run_irq(irq);
+	if (ret <= 0)
+		return ret;
+
+	/*
+	 * Since this can be called from Linux context (e.g. lkl_trigger_irq ->
+	 * IRQ -> softirq -> lkl_trigger_irq) make sure we are actually allowed
+	 * to run irqs at this point
+	 */
+	if (!irqs_enabled)
+		set_irq_pending(irq);
+	else
+		run_irq(irq);
+
+	lkl_cpu_put();
+
+	return 0;
+}
+
+static inline void for_each_bit(unsigned long word, void (*f)(int, int), int j)
+{
+	int i = 0;
+
+	while (word) {
+		if (word & 1)
+			f(i, j);
+		word >>= 1;
+		i++;
+	}
+}
+
+static inline void deliver_irq(int bit, int index)
+{
+	run_irq(index * IRQ_STATUS_BITS + bit);
+}
+
+static inline void check_irq_status(int i, int unused)
+{
+	for_each_bit(test_and_clear_irq_status(i), deliver_irq, i);
+}
+
+void run_irqs(void)
+{
+	for_each_bit(test_and_clear_irq_index_status(), check_irq_status, 0);
+}
+
+int show_interrupts(struct seq_file *p, void *v)
+{
+	return 0;
+}
+
+int lkl_get_free_irq(const char *user)
+{
+	int i;
+	int ret = -EBUSY;
+
+	/* 0 is not a valid IRQ */
+	for (i = 1; i < NR_IRQS; i++) {
+		if (!irqs[i].user) {
+			irqs[i].user = user;
+			irq_set_chip_and_handler(i, &dummy_irq_chip,
+						 handle_simple_irq);
+			ret = i;
+			break;
+		}
+	}
+
+	return ret;
+}
+
+void lkl_put_irq(int i, const char *user)
+{
+	if (!irqs[i].user || strcmp(irqs[i].user, user) != 0) {
+		WARN("%s tried to release %s's irq %d", user, irqs[i].user, i);
+		return;
+	}
+
+	irqs[i].user = NULL;
+}
+
+unsigned long arch_local_save_flags(void)
+{
+	return irqs_enabled;
+}
+
+void arch_local_irq_restore(unsigned long flags)
+{
+	if (flags == ARCH_IRQ_ENABLED && irqs_enabled == ARCH_IRQ_DISABLED &&
+	    !in_interrupt())
+		run_irqs();
+	irqs_enabled = flags;
+}
+
+void init_IRQ(void)
+{
+	int i;
+
+	for (i = 0; i < NR_IRQS; i++)
+		irq_set_chip_and_handler(i, &dummy_irq_chip, handle_simple_irq);
+
+	pr_info("lkl: irqs initialized\n");
+}
+
+void cpu_yield_to_irqs(void)
+{
+	cpu_relax();
+}
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 06/25] um lkl: interrupt support
@ 2020-03-30 14:45     ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: linux-arch, Octavian Purdila, Akira Moroo, linux-kernel-library,
	Michael Zimmermann, Hajime Tazaki

From: Octavian Purdila <tavi.purdila@gmail.com>

Add APIs that allows the host to reserve and free and interrupt number
and also to trigger an interrupt.

The trigger operation will simply store the interrupt data in
queue. The interrupt handler is run later, at the first opportunity it
has to avoid races with any kernel threads.

Currently, interrupts are run on the first interrupt enable operation
if interrupts are disabled and if we are not already in interrupt
context.

When triggering an interrupt, it uses GCC's built-in functions for
atomic memory access to synchronize and simple boolean flags.

Cc: Michael Zimmermann <sigmaepsilon92@gmail.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/lkl/include/asm/irq.h             |  13 ++
 arch/um/lkl/include/uapi/asm/irq.h        |  36 ++++
 arch/um/lkl/include/uapi/asm/sigcontext.h |  16 ++
 arch/um/lkl/kernel/irq.c                  | 190 ++++++++++++++++++++++
 4 files changed, 255 insertions(+)
 create mode 100644 arch/um/lkl/include/asm/irq.h
 create mode 100644 arch/um/lkl/include/uapi/asm/irq.h
 create mode 100644 arch/um/lkl/include/uapi/asm/sigcontext.h
 create mode 100644 arch/um/lkl/kernel/irq.c

diff --git a/arch/um/lkl/include/asm/irq.h b/arch/um/lkl/include/asm/irq.h
new file mode 100644
index 000000000000..948fc54cb76c
--- /dev/null
+++ b/arch/um/lkl/include/asm/irq.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_IRQ_H
+#define _ASM_LKL_IRQ_H
+
+#define IRQ_STATUS_BITS		(sizeof(long) * 8)
+#define NR_IRQS			((int)(IRQ_STATUS_BITS * IRQ_STATUS_BITS))
+
+void run_irqs(void);
+void set_irq_pending(int irq);
+
+#include <uapi/asm/irq.h>
+
+#endif
diff --git a/arch/um/lkl/include/uapi/asm/irq.h b/arch/um/lkl/include/uapi/asm/irq.h
new file mode 100644
index 000000000000..666628b233eb
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/irq.h
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _ASM_UAPI_LKL_IRQ_H
+#define _ASM_UAPI_LKL_IRQ_H
+
+/**
+ * lkl_trigger_irq - generate an interrupt
+ *
+ * This function is used by the device host side to signal its Linux counterpart
+ * that some event happened.
+ *
+ * @irq - the irq number to signal
+ */
+int lkl_trigger_irq(int irq);
+
+/**
+ * lkl_get_free_irq - find and reserve a free IRQ number
+ *
+ * This function is called by the host device code to find an unused IRQ number
+ * and reserved it for its own use.
+ *
+ * @user - a string to identify the user
+ * @returns - and irq number that can be used by request_irq or an negative
+ * value in case of an error
+ */
+int lkl_get_free_irq(const char *user);
+
+/**
+ * lkl_put_irq - release an IRQ number previously obtained with lkl_get_free_irq
+ *
+ * @irq - irq number to release
+ * @user - string identifying the user; should be the same as the one passed to
+ * lkl_get_free_irq when the irq number was obtained
+ */
+void lkl_put_irq(int irq, const char *name);
+
+#endif
diff --git a/arch/um/lkl/include/uapi/asm/sigcontext.h b/arch/um/lkl/include/uapi/asm/sigcontext.h
new file mode 100644
index 000000000000..2f4848843d1d
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/sigcontext.h
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _ASM_UAPI_LKL_SIGCONTEXT_H
+#define _ASM_UAPI_LKL_SIGCONTEXT_H
+
+#include <asm/ptrace.h>
+
+struct pt_regs {
+	void *irq_data;
+};
+
+struct sigcontext {
+	struct pt_regs regs;
+	unsigned long oldmask;
+};
+
+#endif
diff --git a/arch/um/lkl/kernel/irq.c b/arch/um/lkl/kernel/irq.c
new file mode 100644
index 000000000000..c794412f85d9
--- /dev/null
+++ b/arch/um/lkl/kernel/irq.c
@@ -0,0 +1,190 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/kernel.h>
+#include <linux/init.h>
+#include <linux/irq.h>
+#include <linux/hardirq.h>
+#include <asm/irq_regs.h>
+#include <linux/sched.h>
+#include <linux/seq_file.h>
+#include <linux/tick.h>
+#include <asm/irqflags.h>
+#include <asm/host_ops.h>
+#include <asm/cpu.h>
+
+/*
+ * To avoid much overhead we use an indirect approach: the irqs are marked using
+ * a bitmap (array of longs) and a summary of the modified bits is kept in a
+ * separate "index" long - one bit for each sizeof(long). Thus we can support
+ * 4096 irqs on 64bit platforms and 1024 irqs on 32bit platforms.
+ *
+ * Whenever an irq is trigger both the array and the index is updated. To find
+ * which irqs were triggered we first search the index and then the
+ * corresponding part of the arrary.
+ */
+static unsigned long irq_status[NR_IRQS / IRQ_STATUS_BITS];
+static unsigned long irq_index_status;
+
+static inline unsigned long test_and_clear_irq_index_status(void)
+{
+	if (!irq_index_status)
+		return 0;
+	return __sync_fetch_and_and(&irq_index_status, 0);
+}
+
+static inline unsigned long test_and_clear_irq_status(int index)
+{
+	if (!&irq_status[index])
+		return 0;
+	return __sync_fetch_and_and(&irq_status[index], 0);
+}
+
+void set_irq_pending(int irq)
+{
+	int index = irq / IRQ_STATUS_BITS;
+	int bit = irq % IRQ_STATUS_BITS;
+
+	__sync_fetch_and_or(&irq_status[index], BIT(bit));
+	__sync_fetch_and_or(&irq_index_status, BIT(index));
+}
+
+static struct irq_info {
+	const char *user;
+} irqs[NR_IRQS];
+
+static bool irqs_enabled;
+
+static struct pt_regs dummy;
+
+static void run_irq(int irq)
+{
+	unsigned long flags;
+	struct pt_regs *old_regs = set_irq_regs((struct pt_regs *)&dummy);
+
+	/* interrupt handlers need to run with interrupts disabled */
+	local_irq_save(flags);
+	irq_enter();
+	generic_handle_irq(irq);
+	irq_exit();
+	set_irq_regs(old_regs);
+	local_irq_restore(flags);
+}
+
+/**
+ * This function can be called from arbitrary host threads, so do not
+ * issue any Linux calls (e.g. prink) if lkl_cpu_get() was not issued
+ * before.
+ */
+int lkl_trigger_irq(int irq)
+{
+	int ret;
+
+	if (!irq || irq > NR_IRQS)
+		return -EINVAL;
+
+	ret = lkl_cpu_try_run_irq(irq);
+	if (ret <= 0)
+		return ret;
+
+	/*
+	 * Since this can be called from Linux context (e.g. lkl_trigger_irq ->
+	 * IRQ -> softirq -> lkl_trigger_irq) make sure we are actually allowed
+	 * to run irqs at this point
+	 */
+	if (!irqs_enabled)
+		set_irq_pending(irq);
+	else
+		run_irq(irq);
+
+	lkl_cpu_put();
+
+	return 0;
+}
+
+static inline void for_each_bit(unsigned long word, void (*f)(int, int), int j)
+{
+	int i = 0;
+
+	while (word) {
+		if (word & 1)
+			f(i, j);
+		word >>= 1;
+		i++;
+	}
+}
+
+static inline void deliver_irq(int bit, int index)
+{
+	run_irq(index * IRQ_STATUS_BITS + bit);
+}
+
+static inline void check_irq_status(int i, int unused)
+{
+	for_each_bit(test_and_clear_irq_status(i), deliver_irq, i);
+}
+
+void run_irqs(void)
+{
+	for_each_bit(test_and_clear_irq_index_status(), check_irq_status, 0);
+}
+
+int show_interrupts(struct seq_file *p, void *v)
+{
+	return 0;
+}
+
+int lkl_get_free_irq(const char *user)
+{
+	int i;
+	int ret = -EBUSY;
+
+	/* 0 is not a valid IRQ */
+	for (i = 1; i < NR_IRQS; i++) {
+		if (!irqs[i].user) {
+			irqs[i].user = user;
+			irq_set_chip_and_handler(i, &dummy_irq_chip,
+						 handle_simple_irq);
+			ret = i;
+			break;
+		}
+	}
+
+	return ret;
+}
+
+void lkl_put_irq(int i, const char *user)
+{
+	if (!irqs[i].user || strcmp(irqs[i].user, user) != 0) {
+		WARN("%s tried to release %s's irq %d", user, irqs[i].user, i);
+		return;
+	}
+
+	irqs[i].user = NULL;
+}
+
+unsigned long arch_local_save_flags(void)
+{
+	return irqs_enabled;
+}
+
+void arch_local_irq_restore(unsigned long flags)
+{
+	if (flags == ARCH_IRQ_ENABLED && irqs_enabled == ARCH_IRQ_DISABLED &&
+	    !in_interrupt())
+		run_irqs();
+	irqs_enabled = flags;
+}
+
+void init_IRQ(void)
+{
+	int i;
+
+	for (i = 0; i < NR_IRQS; i++)
+		irq_set_chip_and_handler(i, &dummy_irq_chip, handle_simple_irq);
+
+	pr_info("lkl: irqs initialized\n");
+}
+
+void cpu_yield_to_irqs(void)
+{
+	cpu_relax();
+}
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 07/25] um lkl: system call interface and application API
  2020-03-30 14:45   ` Hajime Tazaki
@ 2020-03-30 14:45     ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Conrad Meyer, Jens Staal, Lai Jiangshan, Luca Dariz,
	Michael Zimmermann, Patrick Collins, Pierre-Hugues Husson,
	Yuan Liu, Hajime Tazaki

From: Octavian Purdila <tavi.purdila@gmail.com>

The LKL application API is based on the kernel system call interface
in order to offer a stable API to applications. Note that we can't
offer the full Linux system call interface due to LKL limitations such
as lack of virtual memory, signal, user processes, etc.

The host is using the LKL interrupt mechanism (lkl_trigger_irq) to
initiate a system call. The system call is executed in the context of
the init process.

To avoid collisions between the Linux API and the LKL API (e.g.  struct
stat, MKNOD, etc.) we use a python script to modify the user headers
and to prefix all of the global symbols (structures, typedefs,
defines) with LKL, lkl, _LKL, _lkl, __LKL or __lkl.

Cc: Conrad Meyer <cem@FreeBSD.org>
Cc: Jens Staal <staal1978@gmail.com>
Cc: Lai Jiangshan <jiangshanlai@gmail.com>
Cc: Luca Dariz <luca.dariz@gmail.com>
Cc: Michael Zimmermann <sigmaepsilon92@gmail.com>
Cc: Patrick Collins <pscollins@google.com>
Cc: Pierre-Hugues Husson <phh@phh.me>
Cc: Yuan Liu <liuyuan@google.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/lkl/include/asm/unistd.h        |  27 ++++
 arch/um/lkl/include/uapi/asm/host_ops.h |  14 ++
 arch/um/lkl/include/uapi/asm/unistd.h   |  13 ++
 arch/um/lkl/kernel/syscalls.c           | 192 +++++++++++++++++++++++
 arch/um/lkl/kernel/syscalls_32.c        | 159 +++++++++++++++++++
 arch/um/lkl/scripts/headers_install.py  | 196 ++++++++++++++++++++++++
 6 files changed, 601 insertions(+)
 create mode 100644 arch/um/lkl/include/asm/unistd.h
 create mode 100644 arch/um/lkl/include/uapi/asm/unistd.h
 create mode 100644 arch/um/lkl/kernel/syscalls.c
 create mode 100644 arch/um/lkl/kernel/syscalls_32.c
 create mode 100755 arch/um/lkl/scripts/headers_install.py

diff --git a/arch/um/lkl/include/asm/unistd.h b/arch/um/lkl/include/asm/unistd.h
new file mode 100644
index 000000000000..ca0eac0f2827
--- /dev/null
+++ b/arch/um/lkl/include/asm/unistd.h
@@ -0,0 +1,27 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <uapi/asm/unistd.h>
+
+#define __SC_ASCII(t, a) #t "," #a
+
+#define __ASCII_MAP0(m, ...)
+#define __ASCII_MAP1(m, t, a) m(t, a)
+#define __ASCII_MAP2(m, t, a, ...) m(t, a) "," __ASCII_MAP1(m, __VA_ARGS__)
+#define __ASCII_MAP3(m, t, a, ...) m(t, a) "," __ASCII_MAP2(m, __VA_ARGS__)
+#define __ASCII_MAP4(m, t, a, ...) m(t, a) "," __ASCII_MAP3(m, __VA_ARGS__)
+#define __ASCII_MAP5(m, t, a, ...) m(t, a) "," __ASCII_MAP4(m, __VA_ARGS__)
+#define __ASCII_MAP6(m, t, a, ...) m(t, a) "," __ASCII_MAP5(m, __VA_ARGS__)
+#define __ASCII_MAP(n, ...) __ASCII_MAP##n(__VA_ARGS__)
+
+#ifdef __MINGW32__
+#define SECTION_ATTRS "n0"
+#else
+#define SECTION_ATTRS "a"
+#endif
+
+#define __SYSCALL_DEFINE_ARCH(x, name, ...)				\
+	asm(".section .syscall_defs,\"" SECTION_ATTRS "\"\n"		\
+	    ".ascii \"#ifdef __NR" #name "\\n\"\n"			\
+	    ".ascii \"SYSCALL_DEFINE" #x "(" #name ","			\
+	    __ASCII_MAP(x, __SC_ASCII, __VA_ARGS__) ")\\n\"\n"		\
+	    ".ascii \"#endif\\n\"\n"					\
+	    ".section .text\n");
diff --git a/arch/um/lkl/include/uapi/asm/host_ops.h b/arch/um/lkl/include/uapi/asm/host_ops.h
index 19924fc7c718..1c839d7139f8 100644
--- a/arch/um/lkl/include/uapi/asm/host_ops.h
+++ b/arch/um/lkl/include/uapi/asm/host_ops.h
@@ -36,6 +36,15 @@ struct lkl_jmp_buf {
  * @thread_join - wait for the given thread to terminate. Returns 0
  * for success, -1 otherwise
  *
+ * @tls_alloc - allocate a thread local storage key; returns 0 if successful; if
+ * destructor is not NULL it will be called when a thread terminates with its
+ * argument set to the current thread local storage value
+ * @tls_free - frees a thread local storage key; returns 0 if successful
+ * @tls_set - associate data to the thread local storage key; returns 0 if
+ * successful
+ * @tls_get - return data associated with the thread local storage key or NULL
+ * on error
+ *
  * @mem_alloc - allocate memory
  * @mem_free - free memory
  *
@@ -71,6 +80,11 @@ struct lkl_host_operations {
 	lkl_thread_t (*thread_self)(void);
 	int (*thread_equal)(lkl_thread_t a, lkl_thread_t b);
 
+	struct lkl_tls_key *(*tls_alloc)(void (*destructor)(void *));
+	void (*tls_free)(struct lkl_tls_key *key);
+	int (*tls_set)(struct lkl_tls_key *key, void *data);
+	void *(*tls_get)(struct lkl_tls_key *key);
+
 	void *(*mem_alloc)(unsigned long mem);
 	void (*mem_free)(void *mem);
 
diff --git a/arch/um/lkl/include/uapi/asm/unistd.h b/arch/um/lkl/include/uapi/asm/unistd.h
new file mode 100644
index 000000000000..7670e93f0152
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/unistd.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#define __ARCH_WANT_RENAMEAT
+#define __ARCH_WANT_NEW_STAT
+#define __ARCH_WANT_SET_GET_RLIMIT
+#define __ARCH_WANT_TIME32_SYSCALLS
+
+#include <asm/bitsperlong.h>
+
+#if __BITS_PER_LONG == 64
+#define __ARCH_WANT_SYS_NEWFSTATAT
+#endif
+
+#include <asm-generic/unistd.h>
diff --git a/arch/um/lkl/kernel/syscalls.c b/arch/um/lkl/kernel/syscalls.c
new file mode 100644
index 000000000000..3ab1d776ca4b
--- /dev/null
+++ b/arch/um/lkl/kernel/syscalls.c
@@ -0,0 +1,192 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/stat.h>
+#include <linux/irq.h>
+#include <linux/sched.h>
+#include <linux/interrupt.h>
+#include <linux/jhash.h>
+#include <linux/slab.h>
+#include <linux/types.h>
+#include <linux/net.h>
+#include <linux/task_work.h>
+#include <linux/syscalls.h>
+#include <linux/kthread.h>
+#include <linux/platform_device.h>
+#include <asm/host_ops.h>
+#include <asm/syscalls.h>
+#include <asm/syscalls_32.h>
+#include <asm/cpu.h>
+#include <asm/sched.h>
+
+typedef long (*syscall_handler_t)(long arg1, ...);
+
+#undef __SYSCALL
+#define __SYSCALL(nr, sym)[nr] = (syscall_handler_t)sym,
+
+static syscall_handler_t syscall_table[__NR_syscalls] = {
+	[0 ... __NR_syscalls - 1] = (syscall_handler_t)sys_ni_syscall,
+#include <asm/unistd.h>
+
+#if __BITS_PER_LONG == 32
+#include <asm/unistd_32.h>
+#endif
+};
+
+static long run_syscall(long no, long *params)
+{
+	long ret;
+
+	if (no < 0 || no >= __NR_syscalls)
+		return -ENOSYS;
+
+	ret = syscall_table[no](params[0], params[1], params[2], params[3],
+				params[4], params[5]);
+
+	task_work_run();
+
+	return ret;
+}
+
+
+#define CLONE_FLAGS (CLONE_VM | CLONE_FS | CLONE_FILES | CLONE_THREAD |	\
+		     CLONE_SIGHAND | SIGCHLD)
+
+static int host_task_id;
+static struct task_struct *host0;
+
+static int new_host_task(struct task_struct **task)
+{
+	pid_t pid;
+
+	switch_to_host_task(host0);
+
+	pid = kernel_thread(host_task_stub, NULL, CLONE_FLAGS);
+	if (pid < 0)
+		return pid;
+
+	rcu_read_lock();
+	*task = find_task_by_pid_ns(pid, &init_pid_ns);
+	rcu_read_unlock();
+
+	host_task_id++;
+
+	snprintf((*task)->comm, sizeof((*task)->comm), "host%d", host_task_id);
+
+	return 0;
+}
+static void exit_task(void)
+{
+	do_exit(0);
+}
+
+static void del_host_task(void *arg)
+{
+	struct task_struct *task = (struct task_struct *)arg;
+	struct thread_info *ti = task_thread_info(task);
+
+	if (lkl_cpu_get() < 0)
+		return;
+
+	switch_to_host_task(task);
+	host_task_id--;
+	set_ti_thread_flag(ti, TIF_SCHED_JB);
+	lkl_ops->jmp_buf_set(&ti->sched_jb, exit_task);
+}
+
+static struct lkl_tls_key *task_key;
+
+long lkl_syscall(long no, long *params)
+{
+	struct task_struct *task = host0;
+	long ret;
+
+	ret = lkl_cpu_get();
+	if (ret < 0)
+		return ret;
+
+	if (lkl_ops->tls_get) {
+		task = lkl_ops->tls_get(task_key);
+		if (!task) {
+			ret = new_host_task(&task);
+			if (ret)
+				goto out;
+			lkl_ops->tls_set(task_key, task);
+		}
+	}
+
+	switch_to_host_task(task);
+
+	ret = run_syscall(no, params);
+
+	if (no == __NR_reboot) {
+		thread_sched_jb();
+		return ret;
+	}
+
+out:
+	lkl_cpu_put();
+
+	return ret;
+}
+
+static struct task_struct *idle_host_task;
+
+/* called from idle, don't failed, don't block */
+void wakeup_idle_host_task(void)
+{
+	if (!need_resched() && idle_host_task)
+		wake_up_process(idle_host_task);
+}
+
+static int idle_host_task_loop(void *unused)
+{
+	struct thread_info *ti = task_thread_info(current);
+
+	snprintf(current->comm, sizeof(current->comm), "idle_host_task");
+	set_thread_flag(TIF_HOST_THREAD);
+	idle_host_task = current;
+
+	for (;;) {
+		lkl_cpu_put();
+		lkl_ops->sem_down(ti->sched_sem);
+		if (idle_host_task == NULL) {
+			lkl_ops->thread_exit();
+			return 0;
+		}
+		schedule_tail(ti->prev_sched);
+	}
+}
+
+int syscalls_init(void)
+{
+	snprintf(current->comm, sizeof(current->comm), "host0");
+	set_thread_flag(TIF_HOST_THREAD);
+	host0 = current;
+
+	if (lkl_ops->tls_alloc) {
+		task_key = lkl_ops->tls_alloc(del_host_task);
+		if (!task_key)
+			return -1;
+	}
+
+	if (kernel_thread(idle_host_task_loop, NULL, CLONE_FLAGS) < 0) {
+		if (lkl_ops->tls_free)
+			lkl_ops->tls_free(task_key);
+		return -1;
+	}
+
+	return 0;
+}
+
+void syscalls_cleanup(void)
+{
+	if (idle_host_task) {
+		struct thread_info *ti = task_thread_info(idle_host_task);
+
+		idle_host_task = NULL;
+		lkl_ops->sem_up(ti->sched_sem);
+		lkl_ops->thread_join(ti->tid);
+	}
+
+	if (lkl_ops->tls_free)
+		lkl_ops->tls_free(task_key);
+}
diff --git a/arch/um/lkl/kernel/syscalls_32.c b/arch/um/lkl/kernel/syscalls_32.c
new file mode 100644
index 000000000000..a4271593c338
--- /dev/null
+++ b/arch/um/lkl/kernel/syscalls_32.c
@@ -0,0 +1,159 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * sys_ia32.c: Conversion between 32bit and 64bit native syscalls. Based on
+ *             sys_sparc32
+ *
+ * Copyright (C) 2000		VA Linux Co
+ * Copyright (C) 2000		Don Dugger <n0ano@valinux.com>
+ * Copyright (C) 1999		Arun Sharma <arun.sharma@intel.com>
+ * Copyright (C) 1997,1998	Jakub Jelinek (jj@sunsite.mff.cuni.cz)
+ * Copyright (C) 1997		David S. Miller (davem@caip.rutgers.edu)
+ * Copyright (C) 2000		Hewlett-Packard Co.
+ * Copyright (C) 2000		David Mosberger-Tang <davidm@hpl.hp.com>
+ * Copyright (C) 2000,2001,2002	Andi Kleen, SuSE Labs (x86-64 port)
+ *
+ * These routines maintain argument size conversion between 32bit and 64bit
+ * environment. In 2.5 most of this should be moved to a generic directory.
+ *
+ * This file assumes that there is a hole at the end of user address space.
+ *
+ * Some of the functions are LE specific currently. These are
+ * hopefully all marked.  This should be fixed.
+ */
+
+#include <linux/kernel.h>
+#include <linux/sched.h>
+#include <linux/fs.h>
+#include <linux/file.h>
+#include <linux/signal.h>
+#include <linux/syscalls.h>
+#include <linux/times.h>
+#include <linux/utsname.h>
+#include <linux/mm.h>
+#include <linux/uio.h>
+#include <linux/poll.h>
+#include <linux/personality.h>
+#include <linux/stat.h>
+#include <linux/rwsem.h>
+#include <linux/compat.h>
+#include <linux/vfs.h>
+#include <linux/ptrace.h>
+#include <linux/highuid.h>
+#include <linux/sysctl.h>
+#include <linux/slab.h>
+#include <asm/types.h>
+#include <linux/atomic.h>
+#include <asm/syscalls_32.h>
+
+#define AA(__x)		((unsigned long)(__x))
+
+#if __BITS_PER_LONG == 32
+
+asmlinkage long sys32_truncate64(const char __user *filename,
+				 unsigned long offset_low,
+				 unsigned long offset_high)
+{
+	return sys_truncate64(filename,
+			      ((loff_t)offset_high << 32) | offset_low);
+}
+
+asmlinkage long sys32_ftruncate64(unsigned int fd, unsigned long offset_low,
+				  unsigned long offset_high)
+{
+	return sys_ftruncate64(fd, ((loff_t)offset_high << 32) | offset_low);
+}
+
+#ifdef CONFIG_MMU
+/*
+ * Linux/i386 didn't use to be able to handle more than
+ * 4 system call parameters, so these system calls used a memory
+ * block for parameter passing..
+ */
+
+struct mmap_arg_struct32 {
+	unsigned int addr;
+	unsigned int len;
+	unsigned int prot;
+	unsigned int flags;
+	unsigned int fd;
+	unsigned int offset;
+};
+
+asmlinkage long sys32_mmap(struct mmap_arg_struct32 __user *arg)
+{
+	struct mmap_arg_struct32 a;
+
+	if (copy_from_user(&a, arg, sizeof(a)))
+		return -EFAULT;
+
+	if (a.offset & ~PAGE_MASK)
+		return -EINVAL;
+
+	return sys_mmap_pgoff(a.addr, a.len, a.prot, a.flags, a.fd,
+			      a.offset >> PAGE_SHIFT);
+}
+#endif
+
+asmlinkage long sys32_wait4(pid_t pid, unsigned int __user *stat_addr,
+			    int options, struct rusage __user *ru)
+{
+	return sys_wait4(pid, stat_addr, options, ru);
+}
+
+asmlinkage long sys32_pread64(unsigned int fd, char __user *ubuf, u32 count,
+			      u32 poslo, u32 poshi)
+{
+	return sys_pread64(fd, ubuf, count,
+			   ((loff_t)AA(poshi) << 32) | AA(poslo));
+}
+
+asmlinkage long sys32_pwrite64(unsigned int fd, const char __user *ubuf,
+			       u32 count, u32 poslo, u32 poshi)
+{
+	return sys_pwrite64(fd, ubuf, count,
+			    ((loff_t)AA(poshi) << 32) | AA(poslo));
+}
+
+/*
+ * Some system calls that need sign extended arguments. This could be
+ * done by a generic wrapper.
+ */
+long sys32_fadvise64_64(int fd, __u32 offset_low, __u32 offset_high,
+			__u32 len_low, __u32 len_high, int advice)
+{
+	return sys_fadvise64_64(fd, (((u64)offset_high) << 32) | offset_low,
+				(((u64)len_high) << 32) | len_low, advice);
+}
+
+asmlinkage ssize_t sys32_readahead(int fd, unsigned int off_lo,
+				   unsigned int off_hi, size_t count)
+{
+	return sys_readahead(fd, ((u64)off_hi << 32) | off_lo, count);
+}
+
+asmlinkage long sys32_sync_file_range(int fd, unsigned int off_low,
+				      unsigned int off_hi, unsigned int n_low,
+				      unsigned int n_hi, unsigned int flags)
+{
+	return sys_sync_file_range(fd, ((u64)off_hi << 32) | off_low,
+				   ((u64)n_hi << 32) | n_low, flags);
+}
+
+asmlinkage long sys32_sync_file_range2(int fd, unsigned int flags,
+				       unsigned int off_low,
+				       unsigned int off_hi, unsigned int n_low,
+				       unsigned int n_hi)
+{
+	return sys_sync_file_range(fd, ((u64)off_hi << 32) | off_low,
+				   ((u64)n_hi << 32) | n_low, flags);
+}
+
+asmlinkage long sys32_fallocate(int fd, int mode, unsigned int offset_lo,
+				unsigned int offset_hi, unsigned int len_lo,
+				unsigned int len_hi)
+{
+	return sys_fallocate(fd, mode, ((u64)offset_hi << 32) | offset_lo,
+			     ((u64)len_hi << 32) | len_lo);
+}
+
+#endif
diff --git a/arch/um/lkl/scripts/headers_install.py b/arch/um/lkl/scripts/headers_install.py
new file mode 100755
index 000000000000..30101bd1d4bf
--- /dev/null
+++ b/arch/um/lkl/scripts/headers_install.py
@@ -0,0 +1,196 @@
+#!/usr/bin/env python
+# SPDX-License-Identifier: GPL-2.0
+import re, os, sys, argparse, multiprocessing, fnmatch
+
+srctree = os.environ["srctree"]
+objtree = os.environ["objtree"]
+install_hdr_path = os.environ["INSTALL_HDR_PATH"]
+header_paths = [ install_hdr_path + "/include/",
+                 "arch/um/lkl/include/generated/uapi/", "include/generated/" ]
+
+headers = set()
+includes = set()
+
+def relpath2abspath(relpath):
+    if "generated" in relpath:
+        return objtree + "/" + relpath
+    else:
+        return objtree + "/" + relpath
+
+def find_headers(path):
+    headers.add(path)
+    f = open(relpath2abspath(path))
+    for l in f.readlines():
+        m = re.search("#include <(.*)>", l)
+        try:
+            i = m.group(1)
+            for p in header_paths:
+                if os.access(relpath2abspath(p + i), os.R_OK):
+                    if p + i not in headers:
+                        includes.add(i)
+                        headers.add(p + i)
+                        find_headers(p + i)
+        except:
+            pass
+    f.close()
+
+def has_lkl_prefix(w):
+  return w.startswith("lkl") or w.startswith("_lkl") or w.startswith("__lkl") \
+         or w.startswith("LKL") or w.startswith("_LKL") or w.startswith("__LKL")
+
+def find_symbols(regexp, store):
+    for h in headers:
+        f = open(h)
+        for l in f.readlines():
+            m = regexp.search(l)
+            if not m:
+                continue
+            for e in reversed(m.groups()):
+                if e:
+                    if not has_lkl_prefix(e):
+                        store.add(e)
+                    break
+        f.close()
+
+def find_ml_symbols(regexp, store):
+    for h in headers:
+        for i in regexp.finditer(open(h).read()):
+            for j in reversed(i.groups()):
+                if j:
+                    if not has_lkl_prefix(j):
+                        store.add(j)
+                    break
+
+def find_enums(block_regexp, symbol_regexp, store):
+    for h in headers:
+        # remove comments
+        content = re.sub(re.compile("(\/\*(\*(?!\/)|[^*])*\*\/)", re.S|re.M), " ", open(h).read())
+        # remove preprocesor lines
+        clean_content = ""
+        for l in content.split("\n"):
+            if re.match("\s*#", l):
+                continue
+            clean_content += l + "\n"
+        for i in block_regexp.finditer(clean_content):
+            for j in reversed(i.groups()):
+                if j:
+                    for k in symbol_regexp.finditer(j):
+                        for l in k.groups():
+                            if l:
+                                if not has_lkl_prefix(l):
+                                    store.add(l)
+                                break
+
+def lkl_prefix(w):
+    r = ""
+
+    if w.startswith("__"):
+        r = "__"
+    elif w.startswith("_"):
+        r = "_"
+
+    if w.isupper():
+        r += "LKL"
+    else:
+        r += "lkl"
+
+    if not w.startswith("_"):
+        r += "_"
+
+    r += w
+
+    return r
+
+def replace(h):
+    content = open(h).read()
+    for i in includes:
+        search_str = "(#[ \t]*include[ \t]*[<\"][ \t]*)" + i + "([ \t]*[>\"])"
+        replace_str = "\\1" + "lkl/" + i + "\\2"
+        content = re.sub(search_str, replace_str, content)
+    tmp = ""
+    for w in re.split("(\W+)", content):
+        if w in defines:
+            w = lkl_prefix(w)
+        tmp += w
+    content = tmp
+    for s in structs:
+        search_str = "(\W?struct\s+)" + s + "(\W)"
+        replace_str = "\\1" + lkl_prefix(s) + "\\2"
+        content = re.sub(search_str, replace_str, content, flags = re.MULTILINE)
+    for s in unions:
+        search_str = "(\W?union\s+)" + s + "(\W)"
+        replace_str = "\\1" + lkl_prefix(s) + "\\2"
+        content = re.sub(search_str, replace_str, content, flags = re.MULTILINE)
+    open(h, 'w').write(content)
+
+parser = argparse.ArgumentParser(description='install lkl headers')
+parser.add_argument('path', help='path to install to', )
+parser.add_argument('-j', '--jobs', help='number of parallel jobs', default=1, type=int)
+args = parser.parse_args()
+
+find_headers(install_hdr_path + "/include/asm/syscalls.h")
+headers.add(install_hdr_path + "/include/asm/host_ops.h")
+
+if 'LKL_INSTALL_ADDITIONAL_HEADERS' in os.environ:
+    with open(os.environ['LKL_INSTALL_ADDITIONAL_HEADERS'], 'rU') as f:
+        for line in f.readlines():
+            line = line.split('#', 1)[0].strip()
+            if line != '':
+                headers.add(line)
+
+new_headers = set()
+
+for h in headers:
+    dir = os.path.dirname(h)
+    out_dir = args.path + "/" + re.sub("(arch/um/lkl/include/generated/uapi/|include/generated/uapi/|include/generated|" + install_hdr_path + "/include/)(.*)", "lkl/\\2", dir)
+    try:
+        os.makedirs(out_dir)
+    except:
+        pass
+    print("  INSTALL\t%s" % (out_dir + "/" + os.path.basename(h)))
+    os.system(srctree+"/scripts/headers_install.sh %s %s" % (os.path.abspath(h),
+                                                       out_dir + "/" + os.path.basename(h)))
+    new_headers.add(out_dir + "/" + os.path.basename(h))
+
+headers = new_headers
+
+defines = set()
+structs = set()
+unions = set()
+
+p = re.compile("#[ \t]*define[ \t]*(\w+)")
+find_symbols(p, defines)
+p = re.compile("typedef.*(\(\*(\w+)\)\(.*\)\s*|\W+(\w+)\s*|\s+(\w+)\(.*\)\s*);")
+find_symbols(p, defines)
+p = re.compile("typedef\s+(struct|union)\s+\w*\s*{[^\\{\}]*}\W*(\w+)\s*;", re.M|re.S)
+find_ml_symbols(p, defines)
+defines.add("siginfo_t")
+defines.add("sigevent_t")
+p = re.compile("struct\s+(\w+)\s*\{")
+find_symbols(p, structs)
+structs.add("iovec")
+p = re.compile("union\s+(\w+)\s*\{")
+find_symbols(p, unions)
+p = re.compile("static\s+__inline__(\s+\w+)+\s+(\w+)\([^)]*\)\s")
+find_symbols(p, defines)
+p = re.compile("static\s+__always_inline(\s+\w+)+\s+(\w+)\([^)]*\)\s")
+find_symbols(p, defines)
+p = re.compile("enum\s+(\w*)\s*{([^}]*)}", re.M|re.S)
+q = re.compile("(\w+)\s*(,|=[^,]*|$)", re.M|re.S)
+find_enums(p, q, defines)
+
+# needed for i386
+defines.add("__NR_stime")
+
+def process_header(h):
+    print("  REPLACE\t%s" % (out_dir + "/" + os.path.basename(h)))
+    replace(h)
+
+p = multiprocessing.Pool(args.jobs)
+try:
+    p.map_async(process_header, headers).wait(999999)
+    p.close()
+except:
+    p.terminate()
+finally:
+    p.join()
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 07/25] um lkl: system call interface and application API
@ 2020-03-30 14:45     ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: linux-arch, Hajime Tazaki, Conrad Meyer, Octavian Purdila,
	Jens Staal, Lai Jiangshan, Akira Moroo, Patrick Collins,
	linux-kernel-library, Pierre-Hugues Husson, Michael Zimmermann,
	Luca Dariz, Yuan Liu

From: Octavian Purdila <tavi.purdila@gmail.com>

The LKL application API is based on the kernel system call interface
in order to offer a stable API to applications. Note that we can't
offer the full Linux system call interface due to LKL limitations such
as lack of virtual memory, signal, user processes, etc.

The host is using the LKL interrupt mechanism (lkl_trigger_irq) to
initiate a system call. The system call is executed in the context of
the init process.

To avoid collisions between the Linux API and the LKL API (e.g.  struct
stat, MKNOD, etc.) we use a python script to modify the user headers
and to prefix all of the global symbols (structures, typedefs,
defines) with LKL, lkl, _LKL, _lkl, __LKL or __lkl.

Cc: Conrad Meyer <cem@FreeBSD.org>
Cc: Jens Staal <staal1978@gmail.com>
Cc: Lai Jiangshan <jiangshanlai@gmail.com>
Cc: Luca Dariz <luca.dariz@gmail.com>
Cc: Michael Zimmermann <sigmaepsilon92@gmail.com>
Cc: Patrick Collins <pscollins@google.com>
Cc: Pierre-Hugues Husson <phh@phh.me>
Cc: Yuan Liu <liuyuan@google.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/lkl/include/asm/unistd.h        |  27 ++++
 arch/um/lkl/include/uapi/asm/host_ops.h |  14 ++
 arch/um/lkl/include/uapi/asm/unistd.h   |  13 ++
 arch/um/lkl/kernel/syscalls.c           | 192 +++++++++++++++++++++++
 arch/um/lkl/kernel/syscalls_32.c        | 159 +++++++++++++++++++
 arch/um/lkl/scripts/headers_install.py  | 196 ++++++++++++++++++++++++
 6 files changed, 601 insertions(+)
 create mode 100644 arch/um/lkl/include/asm/unistd.h
 create mode 100644 arch/um/lkl/include/uapi/asm/unistd.h
 create mode 100644 arch/um/lkl/kernel/syscalls.c
 create mode 100644 arch/um/lkl/kernel/syscalls_32.c
 create mode 100755 arch/um/lkl/scripts/headers_install.py

diff --git a/arch/um/lkl/include/asm/unistd.h b/arch/um/lkl/include/asm/unistd.h
new file mode 100644
index 000000000000..ca0eac0f2827
--- /dev/null
+++ b/arch/um/lkl/include/asm/unistd.h
@@ -0,0 +1,27 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <uapi/asm/unistd.h>
+
+#define __SC_ASCII(t, a) #t "," #a
+
+#define __ASCII_MAP0(m, ...)
+#define __ASCII_MAP1(m, t, a) m(t, a)
+#define __ASCII_MAP2(m, t, a, ...) m(t, a) "," __ASCII_MAP1(m, __VA_ARGS__)
+#define __ASCII_MAP3(m, t, a, ...) m(t, a) "," __ASCII_MAP2(m, __VA_ARGS__)
+#define __ASCII_MAP4(m, t, a, ...) m(t, a) "," __ASCII_MAP3(m, __VA_ARGS__)
+#define __ASCII_MAP5(m, t, a, ...) m(t, a) "," __ASCII_MAP4(m, __VA_ARGS__)
+#define __ASCII_MAP6(m, t, a, ...) m(t, a) "," __ASCII_MAP5(m, __VA_ARGS__)
+#define __ASCII_MAP(n, ...) __ASCII_MAP##n(__VA_ARGS__)
+
+#ifdef __MINGW32__
+#define SECTION_ATTRS "n0"
+#else
+#define SECTION_ATTRS "a"
+#endif
+
+#define __SYSCALL_DEFINE_ARCH(x, name, ...)				\
+	asm(".section .syscall_defs,\"" SECTION_ATTRS "\"\n"		\
+	    ".ascii \"#ifdef __NR" #name "\\n\"\n"			\
+	    ".ascii \"SYSCALL_DEFINE" #x "(" #name ","			\
+	    __ASCII_MAP(x, __SC_ASCII, __VA_ARGS__) ")\\n\"\n"		\
+	    ".ascii \"#endif\\n\"\n"					\
+	    ".section .text\n");
diff --git a/arch/um/lkl/include/uapi/asm/host_ops.h b/arch/um/lkl/include/uapi/asm/host_ops.h
index 19924fc7c718..1c839d7139f8 100644
--- a/arch/um/lkl/include/uapi/asm/host_ops.h
+++ b/arch/um/lkl/include/uapi/asm/host_ops.h
@@ -36,6 +36,15 @@ struct lkl_jmp_buf {
  * @thread_join - wait for the given thread to terminate. Returns 0
  * for success, -1 otherwise
  *
+ * @tls_alloc - allocate a thread local storage key; returns 0 if successful; if
+ * destructor is not NULL it will be called when a thread terminates with its
+ * argument set to the current thread local storage value
+ * @tls_free - frees a thread local storage key; returns 0 if successful
+ * @tls_set - associate data to the thread local storage key; returns 0 if
+ * successful
+ * @tls_get - return data associated with the thread local storage key or NULL
+ * on error
+ *
  * @mem_alloc - allocate memory
  * @mem_free - free memory
  *
@@ -71,6 +80,11 @@ struct lkl_host_operations {
 	lkl_thread_t (*thread_self)(void);
 	int (*thread_equal)(lkl_thread_t a, lkl_thread_t b);
 
+	struct lkl_tls_key *(*tls_alloc)(void (*destructor)(void *));
+	void (*tls_free)(struct lkl_tls_key *key);
+	int (*tls_set)(struct lkl_tls_key *key, void *data);
+	void *(*tls_get)(struct lkl_tls_key *key);
+
 	void *(*mem_alloc)(unsigned long mem);
 	void (*mem_free)(void *mem);
 
diff --git a/arch/um/lkl/include/uapi/asm/unistd.h b/arch/um/lkl/include/uapi/asm/unistd.h
new file mode 100644
index 000000000000..7670e93f0152
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/unistd.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#define __ARCH_WANT_RENAMEAT
+#define __ARCH_WANT_NEW_STAT
+#define __ARCH_WANT_SET_GET_RLIMIT
+#define __ARCH_WANT_TIME32_SYSCALLS
+
+#include <asm/bitsperlong.h>
+
+#if __BITS_PER_LONG == 64
+#define __ARCH_WANT_SYS_NEWFSTATAT
+#endif
+
+#include <asm-generic/unistd.h>
diff --git a/arch/um/lkl/kernel/syscalls.c b/arch/um/lkl/kernel/syscalls.c
new file mode 100644
index 000000000000..3ab1d776ca4b
--- /dev/null
+++ b/arch/um/lkl/kernel/syscalls.c
@@ -0,0 +1,192 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/stat.h>
+#include <linux/irq.h>
+#include <linux/sched.h>
+#include <linux/interrupt.h>
+#include <linux/jhash.h>
+#include <linux/slab.h>
+#include <linux/types.h>
+#include <linux/net.h>
+#include <linux/task_work.h>
+#include <linux/syscalls.h>
+#include <linux/kthread.h>
+#include <linux/platform_device.h>
+#include <asm/host_ops.h>
+#include <asm/syscalls.h>
+#include <asm/syscalls_32.h>
+#include <asm/cpu.h>
+#include <asm/sched.h>
+
+typedef long (*syscall_handler_t)(long arg1, ...);
+
+#undef __SYSCALL
+#define __SYSCALL(nr, sym)[nr] = (syscall_handler_t)sym,
+
+static syscall_handler_t syscall_table[__NR_syscalls] = {
+	[0 ... __NR_syscalls - 1] = (syscall_handler_t)sys_ni_syscall,
+#include <asm/unistd.h>
+
+#if __BITS_PER_LONG == 32
+#include <asm/unistd_32.h>
+#endif
+};
+
+static long run_syscall(long no, long *params)
+{
+	long ret;
+
+	if (no < 0 || no >= __NR_syscalls)
+		return -ENOSYS;
+
+	ret = syscall_table[no](params[0], params[1], params[2], params[3],
+				params[4], params[5]);
+
+	task_work_run();
+
+	return ret;
+}
+
+
+#define CLONE_FLAGS (CLONE_VM | CLONE_FS | CLONE_FILES | CLONE_THREAD |	\
+		     CLONE_SIGHAND | SIGCHLD)
+
+static int host_task_id;
+static struct task_struct *host0;
+
+static int new_host_task(struct task_struct **task)
+{
+	pid_t pid;
+
+	switch_to_host_task(host0);
+
+	pid = kernel_thread(host_task_stub, NULL, CLONE_FLAGS);
+	if (pid < 0)
+		return pid;
+
+	rcu_read_lock();
+	*task = find_task_by_pid_ns(pid, &init_pid_ns);
+	rcu_read_unlock();
+
+	host_task_id++;
+
+	snprintf((*task)->comm, sizeof((*task)->comm), "host%d", host_task_id);
+
+	return 0;
+}
+static void exit_task(void)
+{
+	do_exit(0);
+}
+
+static void del_host_task(void *arg)
+{
+	struct task_struct *task = (struct task_struct *)arg;
+	struct thread_info *ti = task_thread_info(task);
+
+	if (lkl_cpu_get() < 0)
+		return;
+
+	switch_to_host_task(task);
+	host_task_id--;
+	set_ti_thread_flag(ti, TIF_SCHED_JB);
+	lkl_ops->jmp_buf_set(&ti->sched_jb, exit_task);
+}
+
+static struct lkl_tls_key *task_key;
+
+long lkl_syscall(long no, long *params)
+{
+	struct task_struct *task = host0;
+	long ret;
+
+	ret = lkl_cpu_get();
+	if (ret < 0)
+		return ret;
+
+	if (lkl_ops->tls_get) {
+		task = lkl_ops->tls_get(task_key);
+		if (!task) {
+			ret = new_host_task(&task);
+			if (ret)
+				goto out;
+			lkl_ops->tls_set(task_key, task);
+		}
+	}
+
+	switch_to_host_task(task);
+
+	ret = run_syscall(no, params);
+
+	if (no == __NR_reboot) {
+		thread_sched_jb();
+		return ret;
+	}
+
+out:
+	lkl_cpu_put();
+
+	return ret;
+}
+
+static struct task_struct *idle_host_task;
+
+/* called from idle, don't failed, don't block */
+void wakeup_idle_host_task(void)
+{
+	if (!need_resched() && idle_host_task)
+		wake_up_process(idle_host_task);
+}
+
+static int idle_host_task_loop(void *unused)
+{
+	struct thread_info *ti = task_thread_info(current);
+
+	snprintf(current->comm, sizeof(current->comm), "idle_host_task");
+	set_thread_flag(TIF_HOST_THREAD);
+	idle_host_task = current;
+
+	for (;;) {
+		lkl_cpu_put();
+		lkl_ops->sem_down(ti->sched_sem);
+		if (idle_host_task == NULL) {
+			lkl_ops->thread_exit();
+			return 0;
+		}
+		schedule_tail(ti->prev_sched);
+	}
+}
+
+int syscalls_init(void)
+{
+	snprintf(current->comm, sizeof(current->comm), "host0");
+	set_thread_flag(TIF_HOST_THREAD);
+	host0 = current;
+
+	if (lkl_ops->tls_alloc) {
+		task_key = lkl_ops->tls_alloc(del_host_task);
+		if (!task_key)
+			return -1;
+	}
+
+	if (kernel_thread(idle_host_task_loop, NULL, CLONE_FLAGS) < 0) {
+		if (lkl_ops->tls_free)
+			lkl_ops->tls_free(task_key);
+		return -1;
+	}
+
+	return 0;
+}
+
+void syscalls_cleanup(void)
+{
+	if (idle_host_task) {
+		struct thread_info *ti = task_thread_info(idle_host_task);
+
+		idle_host_task = NULL;
+		lkl_ops->sem_up(ti->sched_sem);
+		lkl_ops->thread_join(ti->tid);
+	}
+
+	if (lkl_ops->tls_free)
+		lkl_ops->tls_free(task_key);
+}
diff --git a/arch/um/lkl/kernel/syscalls_32.c b/arch/um/lkl/kernel/syscalls_32.c
new file mode 100644
index 000000000000..a4271593c338
--- /dev/null
+++ b/arch/um/lkl/kernel/syscalls_32.c
@@ -0,0 +1,159 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * sys_ia32.c: Conversion between 32bit and 64bit native syscalls. Based on
+ *             sys_sparc32
+ *
+ * Copyright (C) 2000		VA Linux Co
+ * Copyright (C) 2000		Don Dugger <n0ano@valinux.com>
+ * Copyright (C) 1999		Arun Sharma <arun.sharma@intel.com>
+ * Copyright (C) 1997,1998	Jakub Jelinek (jj@sunsite.mff.cuni.cz)
+ * Copyright (C) 1997		David S. Miller (davem@caip.rutgers.edu)
+ * Copyright (C) 2000		Hewlett-Packard Co.
+ * Copyright (C) 2000		David Mosberger-Tang <davidm@hpl.hp.com>
+ * Copyright (C) 2000,2001,2002	Andi Kleen, SuSE Labs (x86-64 port)
+ *
+ * These routines maintain argument size conversion between 32bit and 64bit
+ * environment. In 2.5 most of this should be moved to a generic directory.
+ *
+ * This file assumes that there is a hole at the end of user address space.
+ *
+ * Some of the functions are LE specific currently. These are
+ * hopefully all marked.  This should be fixed.
+ */
+
+#include <linux/kernel.h>
+#include <linux/sched.h>
+#include <linux/fs.h>
+#include <linux/file.h>
+#include <linux/signal.h>
+#include <linux/syscalls.h>
+#include <linux/times.h>
+#include <linux/utsname.h>
+#include <linux/mm.h>
+#include <linux/uio.h>
+#include <linux/poll.h>
+#include <linux/personality.h>
+#include <linux/stat.h>
+#include <linux/rwsem.h>
+#include <linux/compat.h>
+#include <linux/vfs.h>
+#include <linux/ptrace.h>
+#include <linux/highuid.h>
+#include <linux/sysctl.h>
+#include <linux/slab.h>
+#include <asm/types.h>
+#include <linux/atomic.h>
+#include <asm/syscalls_32.h>
+
+#define AA(__x)		((unsigned long)(__x))
+
+#if __BITS_PER_LONG == 32
+
+asmlinkage long sys32_truncate64(const char __user *filename,
+				 unsigned long offset_low,
+				 unsigned long offset_high)
+{
+	return sys_truncate64(filename,
+			      ((loff_t)offset_high << 32) | offset_low);
+}
+
+asmlinkage long sys32_ftruncate64(unsigned int fd, unsigned long offset_low,
+				  unsigned long offset_high)
+{
+	return sys_ftruncate64(fd, ((loff_t)offset_high << 32) | offset_low);
+}
+
+#ifdef CONFIG_MMU
+/*
+ * Linux/i386 didn't use to be able to handle more than
+ * 4 system call parameters, so these system calls used a memory
+ * block for parameter passing..
+ */
+
+struct mmap_arg_struct32 {
+	unsigned int addr;
+	unsigned int len;
+	unsigned int prot;
+	unsigned int flags;
+	unsigned int fd;
+	unsigned int offset;
+};
+
+asmlinkage long sys32_mmap(struct mmap_arg_struct32 __user *arg)
+{
+	struct mmap_arg_struct32 a;
+
+	if (copy_from_user(&a, arg, sizeof(a)))
+		return -EFAULT;
+
+	if (a.offset & ~PAGE_MASK)
+		return -EINVAL;
+
+	return sys_mmap_pgoff(a.addr, a.len, a.prot, a.flags, a.fd,
+			      a.offset >> PAGE_SHIFT);
+}
+#endif
+
+asmlinkage long sys32_wait4(pid_t pid, unsigned int __user *stat_addr,
+			    int options, struct rusage __user *ru)
+{
+	return sys_wait4(pid, stat_addr, options, ru);
+}
+
+asmlinkage long sys32_pread64(unsigned int fd, char __user *ubuf, u32 count,
+			      u32 poslo, u32 poshi)
+{
+	return sys_pread64(fd, ubuf, count,
+			   ((loff_t)AA(poshi) << 32) | AA(poslo));
+}
+
+asmlinkage long sys32_pwrite64(unsigned int fd, const char __user *ubuf,
+			       u32 count, u32 poslo, u32 poshi)
+{
+	return sys_pwrite64(fd, ubuf, count,
+			    ((loff_t)AA(poshi) << 32) | AA(poslo));
+}
+
+/*
+ * Some system calls that need sign extended arguments. This could be
+ * done by a generic wrapper.
+ */
+long sys32_fadvise64_64(int fd, __u32 offset_low, __u32 offset_high,
+			__u32 len_low, __u32 len_high, int advice)
+{
+	return sys_fadvise64_64(fd, (((u64)offset_high) << 32) | offset_low,
+				(((u64)len_high) << 32) | len_low, advice);
+}
+
+asmlinkage ssize_t sys32_readahead(int fd, unsigned int off_lo,
+				   unsigned int off_hi, size_t count)
+{
+	return sys_readahead(fd, ((u64)off_hi << 32) | off_lo, count);
+}
+
+asmlinkage long sys32_sync_file_range(int fd, unsigned int off_low,
+				      unsigned int off_hi, unsigned int n_low,
+				      unsigned int n_hi, unsigned int flags)
+{
+	return sys_sync_file_range(fd, ((u64)off_hi << 32) | off_low,
+				   ((u64)n_hi << 32) | n_low, flags);
+}
+
+asmlinkage long sys32_sync_file_range2(int fd, unsigned int flags,
+				       unsigned int off_low,
+				       unsigned int off_hi, unsigned int n_low,
+				       unsigned int n_hi)
+{
+	return sys_sync_file_range(fd, ((u64)off_hi << 32) | off_low,
+				   ((u64)n_hi << 32) | n_low, flags);
+}
+
+asmlinkage long sys32_fallocate(int fd, int mode, unsigned int offset_lo,
+				unsigned int offset_hi, unsigned int len_lo,
+				unsigned int len_hi)
+{
+	return sys_fallocate(fd, mode, ((u64)offset_hi << 32) | offset_lo,
+			     ((u64)len_hi << 32) | len_lo);
+}
+
+#endif
diff --git a/arch/um/lkl/scripts/headers_install.py b/arch/um/lkl/scripts/headers_install.py
new file mode 100755
index 000000000000..30101bd1d4bf
--- /dev/null
+++ b/arch/um/lkl/scripts/headers_install.py
@@ -0,0 +1,196 @@
+#!/usr/bin/env python
+# SPDX-License-Identifier: GPL-2.0
+import re, os, sys, argparse, multiprocessing, fnmatch
+
+srctree = os.environ["srctree"]
+objtree = os.environ["objtree"]
+install_hdr_path = os.environ["INSTALL_HDR_PATH"]
+header_paths = [ install_hdr_path + "/include/",
+                 "arch/um/lkl/include/generated/uapi/", "include/generated/" ]
+
+headers = set()
+includes = set()
+
+def relpath2abspath(relpath):
+    if "generated" in relpath:
+        return objtree + "/" + relpath
+    else:
+        return objtree + "/" + relpath
+
+def find_headers(path):
+    headers.add(path)
+    f = open(relpath2abspath(path))
+    for l in f.readlines():
+        m = re.search("#include <(.*)>", l)
+        try:
+            i = m.group(1)
+            for p in header_paths:
+                if os.access(relpath2abspath(p + i), os.R_OK):
+                    if p + i not in headers:
+                        includes.add(i)
+                        headers.add(p + i)
+                        find_headers(p + i)
+        except:
+            pass
+    f.close()
+
+def has_lkl_prefix(w):
+  return w.startswith("lkl") or w.startswith("_lkl") or w.startswith("__lkl") \
+         or w.startswith("LKL") or w.startswith("_LKL") or w.startswith("__LKL")
+
+def find_symbols(regexp, store):
+    for h in headers:
+        f = open(h)
+        for l in f.readlines():
+            m = regexp.search(l)
+            if not m:
+                continue
+            for e in reversed(m.groups()):
+                if e:
+                    if not has_lkl_prefix(e):
+                        store.add(e)
+                    break
+        f.close()
+
+def find_ml_symbols(regexp, store):
+    for h in headers:
+        for i in regexp.finditer(open(h).read()):
+            for j in reversed(i.groups()):
+                if j:
+                    if not has_lkl_prefix(j):
+                        store.add(j)
+                    break
+
+def find_enums(block_regexp, symbol_regexp, store):
+    for h in headers:
+        # remove comments
+        content = re.sub(re.compile("(\/\*(\*(?!\/)|[^*])*\*\/)", re.S|re.M), " ", open(h).read())
+        # remove preprocesor lines
+        clean_content = ""
+        for l in content.split("\n"):
+            if re.match("\s*#", l):
+                continue
+            clean_content += l + "\n"
+        for i in block_regexp.finditer(clean_content):
+            for j in reversed(i.groups()):
+                if j:
+                    for k in symbol_regexp.finditer(j):
+                        for l in k.groups():
+                            if l:
+                                if not has_lkl_prefix(l):
+                                    store.add(l)
+                                break
+
+def lkl_prefix(w):
+    r = ""
+
+    if w.startswith("__"):
+        r = "__"
+    elif w.startswith("_"):
+        r = "_"
+
+    if w.isupper():
+        r += "LKL"
+    else:
+        r += "lkl"
+
+    if not w.startswith("_"):
+        r += "_"
+
+    r += w
+
+    return r
+
+def replace(h):
+    content = open(h).read()
+    for i in includes:
+        search_str = "(#[ \t]*include[ \t]*[<\"][ \t]*)" + i + "([ \t]*[>\"])"
+        replace_str = "\\1" + "lkl/" + i + "\\2"
+        content = re.sub(search_str, replace_str, content)
+    tmp = ""
+    for w in re.split("(\W+)", content):
+        if w in defines:
+            w = lkl_prefix(w)
+        tmp += w
+    content = tmp
+    for s in structs:
+        search_str = "(\W?struct\s+)" + s + "(\W)"
+        replace_str = "\\1" + lkl_prefix(s) + "\\2"
+        content = re.sub(search_str, replace_str, content, flags = re.MULTILINE)
+    for s in unions:
+        search_str = "(\W?union\s+)" + s + "(\W)"
+        replace_str = "\\1" + lkl_prefix(s) + "\\2"
+        content = re.sub(search_str, replace_str, content, flags = re.MULTILINE)
+    open(h, 'w').write(content)
+
+parser = argparse.ArgumentParser(description='install lkl headers')
+parser.add_argument('path', help='path to install to', )
+parser.add_argument('-j', '--jobs', help='number of parallel jobs', default=1, type=int)
+args = parser.parse_args()
+
+find_headers(install_hdr_path + "/include/asm/syscalls.h")
+headers.add(install_hdr_path + "/include/asm/host_ops.h")
+
+if 'LKL_INSTALL_ADDITIONAL_HEADERS' in os.environ:
+    with open(os.environ['LKL_INSTALL_ADDITIONAL_HEADERS'], 'rU') as f:
+        for line in f.readlines():
+            line = line.split('#', 1)[0].strip()
+            if line != '':
+                headers.add(line)
+
+new_headers = set()
+
+for h in headers:
+    dir = os.path.dirname(h)
+    out_dir = args.path + "/" + re.sub("(arch/um/lkl/include/generated/uapi/|include/generated/uapi/|include/generated|" + install_hdr_path + "/include/)(.*)", "lkl/\\2", dir)
+    try:
+        os.makedirs(out_dir)
+    except:
+        pass
+    print("  INSTALL\t%s" % (out_dir + "/" + os.path.basename(h)))
+    os.system(srctree+"/scripts/headers_install.sh %s %s" % (os.path.abspath(h),
+                                                       out_dir + "/" + os.path.basename(h)))
+    new_headers.add(out_dir + "/" + os.path.basename(h))
+
+headers = new_headers
+
+defines = set()
+structs = set()
+unions = set()
+
+p = re.compile("#[ \t]*define[ \t]*(\w+)")
+find_symbols(p, defines)
+p = re.compile("typedef.*(\(\*(\w+)\)\(.*\)\s*|\W+(\w+)\s*|\s+(\w+)\(.*\)\s*);")
+find_symbols(p, defines)
+p = re.compile("typedef\s+(struct|union)\s+\w*\s*{[^\\{\}]*}\W*(\w+)\s*;", re.M|re.S)
+find_ml_symbols(p, defines)
+defines.add("siginfo_t")
+defines.add("sigevent_t")
+p = re.compile("struct\s+(\w+)\s*\{")
+find_symbols(p, structs)
+structs.add("iovec")
+p = re.compile("union\s+(\w+)\s*\{")
+find_symbols(p, unions)
+p = re.compile("static\s+__inline__(\s+\w+)+\s+(\w+)\([^)]*\)\s")
+find_symbols(p, defines)
+p = re.compile("static\s+__always_inline(\s+\w+)+\s+(\w+)\([^)]*\)\s")
+find_symbols(p, defines)
+p = re.compile("enum\s+(\w*)\s*{([^}]*)}", re.M|re.S)
+q = re.compile("(\w+)\s*(,|=[^,]*|$)", re.M|re.S)
+find_enums(p, q, defines)
+
+# needed for i386
+defines.add("__NR_stime")
+
+def process_header(h):
+    print("  REPLACE\t%s" % (out_dir + "/" + os.path.basename(h)))
+    replace(h)
+
+p = multiprocessing.Pool(args.jobs)
+try:
+    p.map_async(process_header, headers).wait(999999)
+    p.close()
+except:
+    p.terminate()
+finally:
+    p.join()
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 08/25] um lkl: timers, time and delay support
  2020-03-30 14:45   ` Hajime Tazaki
@ 2020-03-30 14:45     ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Michael Zimmermann, Hajime Tazaki

From: Octavian Purdila <tavi.purdila@gmail.com>

Clockevent driver based on host timer operations and clocksource
driver and udelay support based on host time operations.

Cc: Michael Zimmermann <sigmaepsilon92@gmail.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/lkl/include/uapi/asm/host_ops.h |  13 +++
 arch/um/lkl/kernel/time.c               | 145 ++++++++++++++++++++++++
 2 files changed, 158 insertions(+)
 create mode 100644 arch/um/lkl/kernel/time.c

diff --git a/arch/um/lkl/include/uapi/asm/host_ops.h b/arch/um/lkl/include/uapi/asm/host_ops.h
index 1c839d7139f8..c9f77dd7fbe7 100644
--- a/arch/um/lkl/include/uapi/asm/host_ops.h
+++ b/arch/um/lkl/include/uapi/asm/host_ops.h
@@ -48,6 +48,13 @@ struct lkl_jmp_buf {
  * @mem_alloc - allocate memory
  * @mem_free - free memory
  *
+ * @timer_create - allocate a host timer that runs fn(arg) when the timer
+ * fires.
+ * @timer_free - disarms and free the timer
+ * @timer_set_oneshot - arm the timer to fire once, after delta ns.
+ * @timer_set_periodic - arm the timer to fire periodically, with a period of
+ * delta ns.
+ *
  * @gettid - returns the host thread id of the caller, which need not
  * be the same as the handle returned by thread_create
  *
@@ -88,6 +95,12 @@ struct lkl_host_operations {
 	void *(*mem_alloc)(unsigned long mem);
 	void (*mem_free)(void *mem);
 
+	unsigned long long (*time)(void);
+
+	void *(*timer_alloc)(void (*fn)(void *), void *arg);
+	int (*timer_set_oneshot)(void *timer, unsigned long delta);
+	void (*timer_free)(void *timer);
+
 	long (*gettid)(void);
 
 	void (*jmp_buf_set)(struct lkl_jmp_buf *jmpb, void (*f)(void));
diff --git a/arch/um/lkl/kernel/time.c b/arch/um/lkl/kernel/time.c
new file mode 100644
index 000000000000..dade9717a986
--- /dev/null
+++ b/arch/um/lkl/kernel/time.c
@@ -0,0 +1,145 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/clocksource.h>
+#include <linux/clockchips.h>
+#include <linux/jiffies.h>
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/interrupt.h>
+#include <linux/irq.h>
+#include <asm/host_ops.h>
+
+static unsigned long long boot_time;
+
+void __ndelay(unsigned long nsecs)
+{
+	unsigned long long start = lkl_ops->time();
+
+	while (lkl_ops->time() < start + nsecs)
+		;
+}
+
+void __udelay(unsigned long usecs)
+{
+	__ndelay(usecs * NSEC_PER_USEC);
+}
+
+void __const_udelay(unsigned long xloops)
+{
+	__udelay(xloops / 0x10c7ul);
+}
+
+void calibrate_delay(void)
+{
+}
+
+void read_persistent_clock(struct timespec64 *ts)
+{
+	*ts = ns_to_timespec64(lkl_ops->time());
+}
+
+/*
+ * Scheduler clock - returns current time in nanosec units.
+ *
+ */
+unsigned long long sched_clock(void)
+{
+	if (!boot_time)
+		return 0;
+
+	return lkl_ops->time() - boot_time;
+}
+
+static u64 clock_read(struct clocksource *cs)
+{
+	return lkl_ops->time();
+}
+
+static struct clocksource clocksource = {
+	.name	= "lkl",
+	.rating = 499,
+	.read	= clock_read,
+	.flags	= CLOCK_SOURCE_IS_CONTINUOUS,
+	.mask	= CLOCKSOURCE_MASK(64),
+};
+
+static void *timer;
+
+static int timer_irq;
+
+static void timer_fn(void *arg)
+{
+	lkl_trigger_irq(timer_irq);
+}
+
+static int clockevent_set_state_shutdown(struct clock_event_device *evt)
+{
+	if (timer) {
+		lkl_ops->timer_free(timer);
+		timer = NULL;
+	}
+
+	return 0;
+}
+
+static int clockevent_set_state_oneshot(struct clock_event_device *evt)
+{
+	timer = lkl_ops->timer_alloc(timer_fn, NULL);
+	if (!timer)
+		return -ENOMEM;
+
+	return 0;
+}
+
+static irqreturn_t timer_irq_handler(int irq, void *dev_id)
+{
+	struct clock_event_device *dev = (struct clock_event_device *)dev_id;
+
+	dev->event_handler(dev);
+
+	return IRQ_HANDLED;
+}
+
+static int clockevent_next_event(unsigned long ns,
+				 struct clock_event_device *evt)
+{
+	return lkl_ops->timer_set_oneshot(timer, ns);
+}
+
+static struct clock_event_device clockevent = {
+	.name			= "lkl",
+	.features		= CLOCK_EVT_FEAT_ONESHOT,
+	.set_state_oneshot	= clockevent_set_state_oneshot,
+	.set_next_event		= clockevent_next_event,
+	.set_state_shutdown	= clockevent_set_state_shutdown,
+};
+
+static struct irqaction irq0  = {
+	.handler	= timer_irq_handler,
+	.flags		= IRQF_NOBALANCING | IRQF_TIMER,
+	.dev_id		= &clockevent,
+	.name		= "timer"
+};
+
+void __init time_init(void)
+{
+	int ret;
+
+	if (!lkl_ops->timer_alloc || !lkl_ops->timer_free ||
+	    !lkl_ops->timer_set_oneshot || !lkl_ops->time) {
+		pr_err("lkl: no time or timer support provided by host\n");
+		return;
+	}
+
+	timer_irq = lkl_get_free_irq("timer");
+	setup_irq(timer_irq, &irq0);
+
+	ret = clocksource_register_khz(&clocksource, 1000000);
+	if (ret)
+		pr_err("lkl: unable to register clocksource\n");
+
+	clockevents_config_and_register(&clockevent, NSEC_PER_SEC, 1,
+					ULONG_MAX);
+
+	boot_time = lkl_ops->time();
+	pr_info("lkl: time and timers initialized (irq%d)\n", timer_irq);
+}
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 08/25] um lkl: timers, time and delay support
@ 2020-03-30 14:45     ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: linux-arch, Octavian Purdila, Akira Moroo, linux-kernel-library,
	Michael Zimmermann, Hajime Tazaki

From: Octavian Purdila <tavi.purdila@gmail.com>

Clockevent driver based on host timer operations and clocksource
driver and udelay support based on host time operations.

Cc: Michael Zimmermann <sigmaepsilon92@gmail.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/lkl/include/uapi/asm/host_ops.h |  13 +++
 arch/um/lkl/kernel/time.c               | 145 ++++++++++++++++++++++++
 2 files changed, 158 insertions(+)
 create mode 100644 arch/um/lkl/kernel/time.c

diff --git a/arch/um/lkl/include/uapi/asm/host_ops.h b/arch/um/lkl/include/uapi/asm/host_ops.h
index 1c839d7139f8..c9f77dd7fbe7 100644
--- a/arch/um/lkl/include/uapi/asm/host_ops.h
+++ b/arch/um/lkl/include/uapi/asm/host_ops.h
@@ -48,6 +48,13 @@ struct lkl_jmp_buf {
  * @mem_alloc - allocate memory
  * @mem_free - free memory
  *
+ * @timer_create - allocate a host timer that runs fn(arg) when the timer
+ * fires.
+ * @timer_free - disarms and free the timer
+ * @timer_set_oneshot - arm the timer to fire once, after delta ns.
+ * @timer_set_periodic - arm the timer to fire periodically, with a period of
+ * delta ns.
+ *
  * @gettid - returns the host thread id of the caller, which need not
  * be the same as the handle returned by thread_create
  *
@@ -88,6 +95,12 @@ struct lkl_host_operations {
 	void *(*mem_alloc)(unsigned long mem);
 	void (*mem_free)(void *mem);
 
+	unsigned long long (*time)(void);
+
+	void *(*timer_alloc)(void (*fn)(void *), void *arg);
+	int (*timer_set_oneshot)(void *timer, unsigned long delta);
+	void (*timer_free)(void *timer);
+
 	long (*gettid)(void);
 
 	void (*jmp_buf_set)(struct lkl_jmp_buf *jmpb, void (*f)(void));
diff --git a/arch/um/lkl/kernel/time.c b/arch/um/lkl/kernel/time.c
new file mode 100644
index 000000000000..dade9717a986
--- /dev/null
+++ b/arch/um/lkl/kernel/time.c
@@ -0,0 +1,145 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/clocksource.h>
+#include <linux/clockchips.h>
+#include <linux/jiffies.h>
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/interrupt.h>
+#include <linux/irq.h>
+#include <asm/host_ops.h>
+
+static unsigned long long boot_time;
+
+void __ndelay(unsigned long nsecs)
+{
+	unsigned long long start = lkl_ops->time();
+
+	while (lkl_ops->time() < start + nsecs)
+		;
+}
+
+void __udelay(unsigned long usecs)
+{
+	__ndelay(usecs * NSEC_PER_USEC);
+}
+
+void __const_udelay(unsigned long xloops)
+{
+	__udelay(xloops / 0x10c7ul);
+}
+
+void calibrate_delay(void)
+{
+}
+
+void read_persistent_clock(struct timespec64 *ts)
+{
+	*ts = ns_to_timespec64(lkl_ops->time());
+}
+
+/*
+ * Scheduler clock - returns current time in nanosec units.
+ *
+ */
+unsigned long long sched_clock(void)
+{
+	if (!boot_time)
+		return 0;
+
+	return lkl_ops->time() - boot_time;
+}
+
+static u64 clock_read(struct clocksource *cs)
+{
+	return lkl_ops->time();
+}
+
+static struct clocksource clocksource = {
+	.name	= "lkl",
+	.rating = 499,
+	.read	= clock_read,
+	.flags	= CLOCK_SOURCE_IS_CONTINUOUS,
+	.mask	= CLOCKSOURCE_MASK(64),
+};
+
+static void *timer;
+
+static int timer_irq;
+
+static void timer_fn(void *arg)
+{
+	lkl_trigger_irq(timer_irq);
+}
+
+static int clockevent_set_state_shutdown(struct clock_event_device *evt)
+{
+	if (timer) {
+		lkl_ops->timer_free(timer);
+		timer = NULL;
+	}
+
+	return 0;
+}
+
+static int clockevent_set_state_oneshot(struct clock_event_device *evt)
+{
+	timer = lkl_ops->timer_alloc(timer_fn, NULL);
+	if (!timer)
+		return -ENOMEM;
+
+	return 0;
+}
+
+static irqreturn_t timer_irq_handler(int irq, void *dev_id)
+{
+	struct clock_event_device *dev = (struct clock_event_device *)dev_id;
+
+	dev->event_handler(dev);
+
+	return IRQ_HANDLED;
+}
+
+static int clockevent_next_event(unsigned long ns,
+				 struct clock_event_device *evt)
+{
+	return lkl_ops->timer_set_oneshot(timer, ns);
+}
+
+static struct clock_event_device clockevent = {
+	.name			= "lkl",
+	.features		= CLOCK_EVT_FEAT_ONESHOT,
+	.set_state_oneshot	= clockevent_set_state_oneshot,
+	.set_next_event		= clockevent_next_event,
+	.set_state_shutdown	= clockevent_set_state_shutdown,
+};
+
+static struct irqaction irq0  = {
+	.handler	= timer_irq_handler,
+	.flags		= IRQF_NOBALANCING | IRQF_TIMER,
+	.dev_id		= &clockevent,
+	.name		= "timer"
+};
+
+void __init time_init(void)
+{
+	int ret;
+
+	if (!lkl_ops->timer_alloc || !lkl_ops->timer_free ||
+	    !lkl_ops->timer_set_oneshot || !lkl_ops->time) {
+		pr_err("lkl: no time or timer support provided by host\n");
+		return;
+	}
+
+	timer_irq = lkl_get_free_irq("timer");
+	setup_irq(timer_irq, &irq0);
+
+	ret = clocksource_register_khz(&clocksource, 1000000);
+	if (ret)
+		pr_err("lkl: unable to register clocksource\n");
+
+	clockevents_config_and_register(&clockevent, NSEC_PER_SEC, 1,
+					ULONG_MAX);
+
+	boot_time = lkl_ops->time();
+	pr_info("lkl: time and timers initialized (irq%d)\n", timer_irq);
+}
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 09/25] um lkl: basic kernel console support
  2020-03-30 14:45   ` Hajime Tazaki
@ 2020-03-30 14:45     ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um; +Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch

From: Octavian Purdila <tavi.purdila@gmail.com>

This patch adds a basic structure of console support in kernel.  Write
operations are deferred to the host print operation.

Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/lkl/include/uapi/asm/host_ops.h |  4 +++
 arch/um/lkl/kernel/console.c            | 42 +++++++++++++++++++++++++
 2 files changed, 46 insertions(+)
 create mode 100644 arch/um/lkl/kernel/console.c

diff --git a/arch/um/lkl/include/uapi/asm/host_ops.h b/arch/um/lkl/include/uapi/asm/host_ops.h
index c9f77dd7fbe7..986340ba9d8d 100644
--- a/arch/um/lkl/include/uapi/asm/host_ops.h
+++ b/arch/um/lkl/include/uapi/asm/host_ops.h
@@ -17,6 +17,8 @@ struct lkl_jmp_buf {
  * These operations must be provided by a host library or by the application
  * itself.
  *
+ * @print - optional operation that receives console messages
+ *
  * @sem_alloc - allocate a host semaphore an initialize it to count
  * @sem_free - free a host semaphore
  * @sem_up - perform an up operation on the semaphore
@@ -70,6 +72,8 @@ struct lkl_jmp_buf {
  * @jmp_buf_longjmp - perform a jump back to the saved jump buffer
  */
 struct lkl_host_operations {
+	void (*print)(const char *str, int len);
+
 	struct lkl_sem *(*sem_alloc)(int count);
 	void (*sem_free)(struct lkl_sem *sem);
 	void (*sem_up)(struct lkl_sem *sem);
diff --git a/arch/um/lkl/kernel/console.c b/arch/um/lkl/kernel/console.c
new file mode 100644
index 000000000000..54d7f756c6da
--- /dev/null
+++ b/arch/um/lkl/kernel/console.c
@@ -0,0 +1,42 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/console.h>
+#include <asm/host_ops.h>
+
+static void console_write(struct console *con, const char *str,
+			  unsigned int len)
+{
+	if (lkl_ops->print)
+		lkl_ops->print(str, len);
+}
+
+#ifdef CONFIG_LKL_EARLY_CONSOLE
+static struct console lkl_boot_console = {
+	.name	= "lkl_boot_console",
+	.write	= console_write,
+	.flags	= CON_PRINTBUFFER | CON_BOOT,
+	.index	= -1,
+};
+
+int __init lkl_boot_console_init(void)
+{
+	register_console(&lkl_boot_console);
+	return 0;
+}
+early_initcall(lkl_boot_console_init);
+#endif
+
+static struct console lkl_console = {
+	.name	= "lkl_console",
+	.write	= console_write,
+	.flags	= CON_PRINTBUFFER,
+	.index	= -1,
+};
+
+static int __init lkl_console_init(void)
+{
+	register_console(&lkl_console);
+	return 0;
+}
+core_initcall(lkl_console_init);
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 09/25] um lkl: basic kernel console support
@ 2020-03-30 14:45     ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um; +Cc: Octavian Purdila, linux-kernel-library, linux-arch, Akira Moroo

From: Octavian Purdila <tavi.purdila@gmail.com>

This patch adds a basic structure of console support in kernel.  Write
operations are deferred to the host print operation.

Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/lkl/include/uapi/asm/host_ops.h |  4 +++
 arch/um/lkl/kernel/console.c            | 42 +++++++++++++++++++++++++
 2 files changed, 46 insertions(+)
 create mode 100644 arch/um/lkl/kernel/console.c

diff --git a/arch/um/lkl/include/uapi/asm/host_ops.h b/arch/um/lkl/include/uapi/asm/host_ops.h
index c9f77dd7fbe7..986340ba9d8d 100644
--- a/arch/um/lkl/include/uapi/asm/host_ops.h
+++ b/arch/um/lkl/include/uapi/asm/host_ops.h
@@ -17,6 +17,8 @@ struct lkl_jmp_buf {
  * These operations must be provided by a host library or by the application
  * itself.
  *
+ * @print - optional operation that receives console messages
+ *
  * @sem_alloc - allocate a host semaphore an initialize it to count
  * @sem_free - free a host semaphore
  * @sem_up - perform an up operation on the semaphore
@@ -70,6 +72,8 @@ struct lkl_jmp_buf {
  * @jmp_buf_longjmp - perform a jump back to the saved jump buffer
  */
 struct lkl_host_operations {
+	void (*print)(const char *str, int len);
+
 	struct lkl_sem *(*sem_alloc)(int count);
 	void (*sem_free)(struct lkl_sem *sem);
 	void (*sem_up)(struct lkl_sem *sem);
diff --git a/arch/um/lkl/kernel/console.c b/arch/um/lkl/kernel/console.c
new file mode 100644
index 000000000000..54d7f756c6da
--- /dev/null
+++ b/arch/um/lkl/kernel/console.c
@@ -0,0 +1,42 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/console.h>
+#include <asm/host_ops.h>
+
+static void console_write(struct console *con, const char *str,
+			  unsigned int len)
+{
+	if (lkl_ops->print)
+		lkl_ops->print(str, len);
+}
+
+#ifdef CONFIG_LKL_EARLY_CONSOLE
+static struct console lkl_boot_console = {
+	.name	= "lkl_boot_console",
+	.write	= console_write,
+	.flags	= CON_PRINTBUFFER | CON_BOOT,
+	.index	= -1,
+};
+
+int __init lkl_boot_console_init(void)
+{
+	register_console(&lkl_boot_console);
+	return 0;
+}
+early_initcall(lkl_boot_console_init);
+#endif
+
+static struct console lkl_console = {
+	.name	= "lkl_console",
+	.write	= console_write,
+	.flags	= CON_PRINTBUFFER,
+	.index	= -1,
+};
+
+static int __init lkl_console_init(void)
+{
+	register_console(&lkl_console);
+	return 0;
+}
+core_initcall(lkl_console_init);
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 10/25] um lkl: initialization and cleanup
  2020-03-30 14:45   ` Hajime Tazaki
@ 2020-03-30 14:45     ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Michael Zimmermann, Patrick Collins, Yuan Liu, Hajime Tazaki

From: Octavian Purdila <tavi.purdila@gmail.com>

Add the lkl_start_kernel and lkl_sys_halt APIs that start and
respectively stops the Linux kernel.

lkl_start_kernel creates a separate threads that will run the initial
and idle kernel thread. It waits for the kernel to complete
initialization before returning, to avoid races with system calls
issues by the host application.

During the setup phase, we create "/init" in initial ramfs root
filesystem to avoid mounting the "real" rootfs since ramfs is good
enough for now.

lkl_sys_halt will shutdown the kernel, terminate all threads and
free all host resources used by the kernel before returning.

This patch also introduces idle CPU handling since it is closely
related to the shutdown process. A host semaphore is used to wait for
new interrupts when the kernel switches the CPU to idle to avoid
wasting host CPU cycles. When the kernel is shutdown we terminate the
idle thread at the first CPU idle event.

Cc: Michael Zimmermann <sigmaepsilon92@gmail.com>
Cc: Patrick Collins <pscollins@google.com>
Cc: Yuan Liu <liuyuan@google.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/lkl/include/asm/setup.h         |   7 +
 arch/um/lkl/include/uapi/asm/host_ops.h |  20 +++
 arch/um/lkl/kernel/setup.c              | 189 ++++++++++++++++++++++++
 3 files changed, 216 insertions(+)
 create mode 100644 arch/um/lkl/include/asm/setup.h
 create mode 100644 arch/um/lkl/kernel/setup.c

diff --git a/arch/um/lkl/include/asm/setup.h b/arch/um/lkl/include/asm/setup.h
new file mode 100644
index 000000000000..b40955208cc6
--- /dev/null
+++ b/arch/um/lkl/include/asm/setup.h
@@ -0,0 +1,7 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_SETUP_H
+#define _ASM_LKL_SETUP_H
+
+#define COMMAND_LINE_SIZE 4096
+
+#endif
diff --git a/arch/um/lkl/include/uapi/asm/host_ops.h b/arch/um/lkl/include/uapi/asm/host_ops.h
index 986340ba9d8d..fe4382c3050a 100644
--- a/arch/um/lkl/include/uapi/asm/host_ops.h
+++ b/arch/um/lkl/include/uapi/asm/host_ops.h
@@ -19,6 +19,8 @@ struct lkl_jmp_buf {
  *
  * @print - optional operation that receives console messages
  *
+ * @panic - called during a kernel panic
+ *
  * @sem_alloc - allocate a host semaphore an initialize it to count
  * @sem_free - free a host semaphore
  * @sem_up - perform an up operation on the semaphore
@@ -73,6 +75,7 @@ struct lkl_jmp_buf {
  */
 struct lkl_host_operations {
 	void (*print)(const char *str, int len);
+	void (*panic)(void);
 
 	struct lkl_sem *(*sem_alloc)(int count);
 	void (*sem_free)(struct lkl_sem *sem);
@@ -111,6 +114,23 @@ struct lkl_host_operations {
 	void (*jmp_buf_longjmp)(struct lkl_jmp_buf *jmpb, int val);
 };
 
+/**
+ * lkl_start_kernel - registers the host operations and starts the kernel
+ *
+ * The function returns only after the kernel is shutdown with lkl_sys_halt.
+ *
+ * @lkl_ops - pointer to host operations
+ * @cmd_line - format for command line string that is going to be used to
+ * generate the Linux kernel command line
+ */
+int lkl_start_kernel(struct lkl_host_operations *lkl_ops, const char *cmd_line,
+		     ...);
+
+/**
+ * lkl_is_running - returns 1 if the kernel is currently running
+ */
+int lkl_is_running(void);
+
 int lkl_printf(const char *fmt, ...);
 void lkl_bug(const char *fmt, ...);
 
diff --git a/arch/um/lkl/kernel/setup.c b/arch/um/lkl/kernel/setup.c
new file mode 100644
index 000000000000..36c199d3aa22
--- /dev/null
+++ b/arch/um/lkl/kernel/setup.c
@@ -0,0 +1,189 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/binfmts.h>
+#include <linux/init.h>
+#include <linux/init_task.h>
+#include <linux/personality.h>
+#include <linux/reboot.h>
+#include <linux/fs.h>
+#include <linux/start_kernel.h>
+#include <linux/syscalls.h>
+#include <linux/tick.h>
+#include <asm/host_ops.h>
+#include <asm/irq.h>
+#include <asm/unistd.h>
+#include <asm/syscalls.h>
+#include <asm/cpu.h>
+
+struct lkl_host_operations *lkl_ops;
+static char cmd_line[COMMAND_LINE_SIZE];
+static void *init_sem;
+static int is_running;
+void (*pm_power_off)(void) = NULL;
+static unsigned long mem_size = 64 * 1024 * 1024;
+
+static long lkl_panic_blink(int state)
+{
+	lkl_ops->panic();
+	return 0;
+}
+
+static int __init setup_mem_size(char *str)
+{
+	mem_size = memparse(str, NULL);
+	return 0;
+}
+early_param("mem", setup_mem_size);
+
+void __init setup_arch(char **cl)
+{
+	*cl = cmd_line;
+	panic_blink = lkl_panic_blink;
+	parse_early_param();
+	bootmem_init(mem_size);
+}
+
+static void __init lkl_run_kernel(void *arg)
+{
+	threads_init();
+	lkl_cpu_get();
+	start_kernel();
+}
+
+int __init lkl_start_kernel(struct lkl_host_operations *ops, const char *fmt,
+			    ...)
+{
+	va_list ap;
+	int ret;
+
+	lkl_ops = ops;
+
+	va_start(ap, fmt);
+	ret = vsnprintf(boot_command_line, COMMAND_LINE_SIZE, fmt, ap);
+	va_end(ap);
+
+	memcpy(cmd_line, boot_command_line, COMMAND_LINE_SIZE);
+
+	init_sem = lkl_ops->sem_alloc(0);
+	if (!init_sem)
+		return -ENOMEM;
+
+	ret = lkl_cpu_init();
+	if (ret)
+		goto out_free_init_sem;
+
+	ret = lkl_ops->thread_create(lkl_run_kernel, NULL);
+	if (!ret) {
+		ret = -ENOMEM;
+		goto out_free_init_sem;
+	}
+
+	lkl_ops->sem_down(init_sem);
+	lkl_ops->sem_free(init_sem);
+	current_thread_info()->tid = lkl_ops->thread_self();
+	lkl_cpu_change_owner(current_thread_info()->tid);
+
+	lkl_cpu_put();
+	is_running = 1;
+
+	return 0;
+
+out_free_init_sem:
+	lkl_ops->sem_free(init_sem);
+
+	return ret;
+}
+
+int lkl_is_running(void)
+{
+	return is_running;
+}
+
+void machine_halt(void)
+{
+	lkl_cpu_shutdown();
+}
+
+void machine_power_off(void)
+{
+	machine_halt();
+}
+
+void machine_restart(char *unused)
+{
+	machine_halt();
+}
+
+long lkl_sys_halt(void)
+{
+	long err;
+	long params[6] = {
+		LINUX_REBOOT_MAGIC1,
+		LINUX_REBOOT_MAGIC2,
+		LINUX_REBOOT_CMD_RESTART,
+	};
+
+	err = lkl_syscall(__NR_reboot, params);
+	if (err < 0)
+		return err;
+
+	is_running = false;
+
+	lkl_cpu_wait_shutdown();
+
+	syscalls_cleanup();
+	threads_cleanup();
+	/* Shutdown the clockevents source. */
+	tick_suspend_local();
+	free_mem();
+	lkl_ops->thread_join(current_thread_info()->tid);
+
+	return 0;
+}
+
+static int lkl_run_init(struct linux_binprm *bprm);
+
+static struct linux_binfmt lkl_run_init_binfmt = {
+	.module		= THIS_MODULE,
+	.load_binary	= lkl_run_init,
+};
+
+static int lkl_run_init(struct linux_binprm *bprm)
+{
+	int ret;
+
+	if (strcmp("/init", bprm->filename) != 0)
+		return -EINVAL;
+
+	ret = flush_old_exec(bprm);
+	if (ret)
+		return ret;
+	set_personality(PER_LINUX);
+	setup_new_exec(bprm);
+	install_exec_creds(bprm);
+
+	set_binfmt(&lkl_run_init_binfmt);
+
+	init_pid_ns.child_reaper = NULL;
+
+	syscalls_init();
+
+	lkl_ops->sem_up(init_sem);
+	lkl_ops->thread_exit();
+
+	return 0;
+}
+
+/* skip mounting the "real" rootfs. ramfs is good enough. */
+static int __init fs_setup(void)
+{
+	int fd;
+
+	fd = sys_open("/init", O_CREAT, 0700);
+	WARN_ON(fd < 0);
+	sys_close(fd);
+
+	register_binfmt(&lkl_run_init_binfmt);
+
+	return 0;
+}
+late_initcall(fs_setup);
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 10/25] um lkl: initialization and cleanup
@ 2020-03-30 14:45     ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: linux-arch, Hajime Tazaki, Octavian Purdila, Akira Moroo,
	Patrick Collins, linux-kernel-library, Michael Zimmermann,
	Yuan Liu

From: Octavian Purdila <tavi.purdila@gmail.com>

Add the lkl_start_kernel and lkl_sys_halt APIs that start and
respectively stops the Linux kernel.

lkl_start_kernel creates a separate threads that will run the initial
and idle kernel thread. It waits for the kernel to complete
initialization before returning, to avoid races with system calls
issues by the host application.

During the setup phase, we create "/init" in initial ramfs root
filesystem to avoid mounting the "real" rootfs since ramfs is good
enough for now.

lkl_sys_halt will shutdown the kernel, terminate all threads and
free all host resources used by the kernel before returning.

This patch also introduces idle CPU handling since it is closely
related to the shutdown process. A host semaphore is used to wait for
new interrupts when the kernel switches the CPU to idle to avoid
wasting host CPU cycles. When the kernel is shutdown we terminate the
idle thread at the first CPU idle event.

Cc: Michael Zimmermann <sigmaepsilon92@gmail.com>
Cc: Patrick Collins <pscollins@google.com>
Cc: Yuan Liu <liuyuan@google.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/lkl/include/asm/setup.h         |   7 +
 arch/um/lkl/include/uapi/asm/host_ops.h |  20 +++
 arch/um/lkl/kernel/setup.c              | 189 ++++++++++++++++++++++++
 3 files changed, 216 insertions(+)
 create mode 100644 arch/um/lkl/include/asm/setup.h
 create mode 100644 arch/um/lkl/kernel/setup.c

diff --git a/arch/um/lkl/include/asm/setup.h b/arch/um/lkl/include/asm/setup.h
new file mode 100644
index 000000000000..b40955208cc6
--- /dev/null
+++ b/arch/um/lkl/include/asm/setup.h
@@ -0,0 +1,7 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_SETUP_H
+#define _ASM_LKL_SETUP_H
+
+#define COMMAND_LINE_SIZE 4096
+
+#endif
diff --git a/arch/um/lkl/include/uapi/asm/host_ops.h b/arch/um/lkl/include/uapi/asm/host_ops.h
index 986340ba9d8d..fe4382c3050a 100644
--- a/arch/um/lkl/include/uapi/asm/host_ops.h
+++ b/arch/um/lkl/include/uapi/asm/host_ops.h
@@ -19,6 +19,8 @@ struct lkl_jmp_buf {
  *
  * @print - optional operation that receives console messages
  *
+ * @panic - called during a kernel panic
+ *
  * @sem_alloc - allocate a host semaphore an initialize it to count
  * @sem_free - free a host semaphore
  * @sem_up - perform an up operation on the semaphore
@@ -73,6 +75,7 @@ struct lkl_jmp_buf {
  */
 struct lkl_host_operations {
 	void (*print)(const char *str, int len);
+	void (*panic)(void);
 
 	struct lkl_sem *(*sem_alloc)(int count);
 	void (*sem_free)(struct lkl_sem *sem);
@@ -111,6 +114,23 @@ struct lkl_host_operations {
 	void (*jmp_buf_longjmp)(struct lkl_jmp_buf *jmpb, int val);
 };
 
+/**
+ * lkl_start_kernel - registers the host operations and starts the kernel
+ *
+ * The function returns only after the kernel is shutdown with lkl_sys_halt.
+ *
+ * @lkl_ops - pointer to host operations
+ * @cmd_line - format for command line string that is going to be used to
+ * generate the Linux kernel command line
+ */
+int lkl_start_kernel(struct lkl_host_operations *lkl_ops, const char *cmd_line,
+		     ...);
+
+/**
+ * lkl_is_running - returns 1 if the kernel is currently running
+ */
+int lkl_is_running(void);
+
 int lkl_printf(const char *fmt, ...);
 void lkl_bug(const char *fmt, ...);
 
diff --git a/arch/um/lkl/kernel/setup.c b/arch/um/lkl/kernel/setup.c
new file mode 100644
index 000000000000..36c199d3aa22
--- /dev/null
+++ b/arch/um/lkl/kernel/setup.c
@@ -0,0 +1,189 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/binfmts.h>
+#include <linux/init.h>
+#include <linux/init_task.h>
+#include <linux/personality.h>
+#include <linux/reboot.h>
+#include <linux/fs.h>
+#include <linux/start_kernel.h>
+#include <linux/syscalls.h>
+#include <linux/tick.h>
+#include <asm/host_ops.h>
+#include <asm/irq.h>
+#include <asm/unistd.h>
+#include <asm/syscalls.h>
+#include <asm/cpu.h>
+
+struct lkl_host_operations *lkl_ops;
+static char cmd_line[COMMAND_LINE_SIZE];
+static void *init_sem;
+static int is_running;
+void (*pm_power_off)(void) = NULL;
+static unsigned long mem_size = 64 * 1024 * 1024;
+
+static long lkl_panic_blink(int state)
+{
+	lkl_ops->panic();
+	return 0;
+}
+
+static int __init setup_mem_size(char *str)
+{
+	mem_size = memparse(str, NULL);
+	return 0;
+}
+early_param("mem", setup_mem_size);
+
+void __init setup_arch(char **cl)
+{
+	*cl = cmd_line;
+	panic_blink = lkl_panic_blink;
+	parse_early_param();
+	bootmem_init(mem_size);
+}
+
+static void __init lkl_run_kernel(void *arg)
+{
+	threads_init();
+	lkl_cpu_get();
+	start_kernel();
+}
+
+int __init lkl_start_kernel(struct lkl_host_operations *ops, const char *fmt,
+			    ...)
+{
+	va_list ap;
+	int ret;
+
+	lkl_ops = ops;
+
+	va_start(ap, fmt);
+	ret = vsnprintf(boot_command_line, COMMAND_LINE_SIZE, fmt, ap);
+	va_end(ap);
+
+	memcpy(cmd_line, boot_command_line, COMMAND_LINE_SIZE);
+
+	init_sem = lkl_ops->sem_alloc(0);
+	if (!init_sem)
+		return -ENOMEM;
+
+	ret = lkl_cpu_init();
+	if (ret)
+		goto out_free_init_sem;
+
+	ret = lkl_ops->thread_create(lkl_run_kernel, NULL);
+	if (!ret) {
+		ret = -ENOMEM;
+		goto out_free_init_sem;
+	}
+
+	lkl_ops->sem_down(init_sem);
+	lkl_ops->sem_free(init_sem);
+	current_thread_info()->tid = lkl_ops->thread_self();
+	lkl_cpu_change_owner(current_thread_info()->tid);
+
+	lkl_cpu_put();
+	is_running = 1;
+
+	return 0;
+
+out_free_init_sem:
+	lkl_ops->sem_free(init_sem);
+
+	return ret;
+}
+
+int lkl_is_running(void)
+{
+	return is_running;
+}
+
+void machine_halt(void)
+{
+	lkl_cpu_shutdown();
+}
+
+void machine_power_off(void)
+{
+	machine_halt();
+}
+
+void machine_restart(char *unused)
+{
+	machine_halt();
+}
+
+long lkl_sys_halt(void)
+{
+	long err;
+	long params[6] = {
+		LINUX_REBOOT_MAGIC1,
+		LINUX_REBOOT_MAGIC2,
+		LINUX_REBOOT_CMD_RESTART,
+	};
+
+	err = lkl_syscall(__NR_reboot, params);
+	if (err < 0)
+		return err;
+
+	is_running = false;
+
+	lkl_cpu_wait_shutdown();
+
+	syscalls_cleanup();
+	threads_cleanup();
+	/* Shutdown the clockevents source. */
+	tick_suspend_local();
+	free_mem();
+	lkl_ops->thread_join(current_thread_info()->tid);
+
+	return 0;
+}
+
+static int lkl_run_init(struct linux_binprm *bprm);
+
+static struct linux_binfmt lkl_run_init_binfmt = {
+	.module		= THIS_MODULE,
+	.load_binary	= lkl_run_init,
+};
+
+static int lkl_run_init(struct linux_binprm *bprm)
+{
+	int ret;
+
+	if (strcmp("/init", bprm->filename) != 0)
+		return -EINVAL;
+
+	ret = flush_old_exec(bprm);
+	if (ret)
+		return ret;
+	set_personality(PER_LINUX);
+	setup_new_exec(bprm);
+	install_exec_creds(bprm);
+
+	set_binfmt(&lkl_run_init_binfmt);
+
+	init_pid_ns.child_reaper = NULL;
+
+	syscalls_init();
+
+	lkl_ops->sem_up(init_sem);
+	lkl_ops->thread_exit();
+
+	return 0;
+}
+
+/* skip mounting the "real" rootfs. ramfs is good enough. */
+static int __init fs_setup(void)
+{
+	int fd;
+
+	fd = sys_open("/init", O_CREAT, 0700);
+	WARN_ON(fd < 0);
+	sys_close(fd);
+
+	register_binfmt(&lkl_run_init_binfmt);
+
+	return 0;
+}
+late_initcall(fs_setup);
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 11/25] um lkl: plug in the build system
  2020-03-30 14:45   ` Hajime Tazaki
@ 2020-03-30 14:45     ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Hajime Tazaki

From: Octavian Purdila <tavi.purdila@gmail.com>

Basic Makefiles for building LKL. Add a new architecture specific
target for installing the resulting library files and headers.

To make LKL binaries build, UML introduced an additional option, UMMODE
variable, to switch the output file of build: kernel (default), or
library (LKL).  Those modes are not able to be ON at the same time.

To build on library mode, users do the following:

  make defconfig ARCH=um UMMODE=library
  make ARCH=um UMMODE=library

Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
Signed-off-by: Akira Moroo <retrage01@gmail.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/Kconfig             | 13 +++++++
 arch/um/Makefile            | 19 ++++++++++-
 arch/um/Makefile.um         | 10 ++++++
 arch/um/lkl/Kconfig         | 31 +++++++++++++++--
 arch/um/lkl/Makefile        | 67 +++++++++++++++++++++++++++++++++++++
 arch/um/lkl/auto.conf       |  1 +
 arch/um/lkl/kernel/Makefile |  4 +++
 arch/um/lkl/mm/Makefile     |  1 +
 8 files changed, 143 insertions(+), 3 deletions(-)
 create mode 100644 arch/um/Makefile.um
 create mode 100644 arch/um/lkl/Makefile
 create mode 100644 arch/um/lkl/auto.conf
 create mode 100644 arch/um/lkl/kernel/Makefile
 create mode 100644 arch/um/lkl/mm/Makefile

diff --git a/arch/um/Kconfig b/arch/um/Kconfig
index 96ab7026b037..17a6bc25b615 100644
--- a/arch/um/Kconfig
+++ b/arch/um/Kconfig
@@ -5,6 +5,10 @@ menu "UML-specific options"
 config UML
 	bool
 	default y
+
+config UMMODE_KERN
+	bool "UML mode: kernel mode"
+	default y if "$(UMMODE)" = "kernel"
 	select ARCH_HAS_KCOV
 	select ARCH_NO_PREEMPT
 	select HAVE_ARCH_AUDITSYSCALL
@@ -20,7 +24,12 @@ config UML
 	select GENERIC_CLOCKEVENTS
 	select HAVE_GCC_PLUGINS
 	select TTY # Needed for line.c
+        help
+	  This mode switches a mode to build a regular kernel executable
+          of UML.
+
 
+if UMMODE_KERN
 config MMU
 	bool
 	default y
@@ -207,6 +216,10 @@ config UML_TIME_TRAVEL_SUPPORT
 
 	  It is safe to say Y, but you probably don't need this.
 
+endif #UMMODE_KERN
+
 endmenu
 
 source "arch/um/drivers/Kconfig"
+
+source "arch/um/lkl/Kconfig"
diff --git a/arch/um/Makefile b/arch/um/Makefile
index d2daa206872d..f2a537f700c2 100644
--- a/arch/um/Makefile
+++ b/arch/um/Makefile
@@ -1,3 +1,4 @@
+# SPDX-License-Identifier: GPL-2.0
 #
 # This file is included by the global makefile so that you can add your own
 # architecture-specific flags and dependencies.
@@ -44,7 +45,7 @@ HOST_DIR := arch/$(HEADER_ARCH)
 include $(ARCH_DIR)/Makefile-skas
 include $(HOST_DIR)/Makefile.um
 
-core-y += $(HOST_DIR)/um/
+
 
 SHARED_HEADERS	:= $(ARCH_DIR)/include/shared
 ARCH_INCLUDE	:= -I$(srctree)/$(SHARED_HEADERS)
@@ -144,5 +145,21 @@ CLEAN_FILES += linux x.i gmon.out
 archclean:
 	@find . \( -name '*.bb' -o -name '*.bbg' -o -name '*.da' \
 		-o -name '*.gcov' \) -type f -print | xargs rm -f
+	$(Q)rm -rf $(srctree)/$(LKL_DIR)/include/generated
 
 export HEADER_ARCH SUBARCH USER_CFLAGS CFLAGS_NO_HARDENING OS DEV_NULL_PATH
+
+
+
+# SPDX-License-Identifier: GPL-2.0
+# select mode of UML build
+UMMODE ?= kernel
+LKL_DIR := $(ARCH_DIR)/lkl
+
+ifeq ($(UMMODE),kernel)
+	include $(ARCH_DIR)/Makefile.um
+else ifeq ($(UMMODE),library)
+	include $(ARCH_DIR)/lkl/Makefile
+endif
+
+export UMMODE LKL_DIR
diff --git a/arch/um/Makefile.um b/arch/um/Makefile.um
new file mode 100644
index 000000000000..be7ee4d1adde
--- /dev/null
+++ b/arch/um/Makefile.um
@@ -0,0 +1,10 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# This file is included by the global makefile so that you can add your own
+# architecture-specific flags and dependencies.
+#
+# Copyright (C) 2002 - 2007 Jeff Dike (jdike@{addtoit,linux.intel}.com)
+# Licensed under the GPL
+#
+
+core-y += $(HOST_DIR)/um/
diff --git a/arch/um/lkl/Kconfig b/arch/um/lkl/Kconfig
index f7e8d707a12b..c72c40226509 100644
--- a/arch/um/lkl/Kconfig
+++ b/arch/um/lkl/Kconfig
@@ -1,6 +1,25 @@
 # SPDX-License-Identifier: GPL-2.0
 
-config UML_LKL
+menu "LKL-specific options"
+
+config UML
+	bool
+	default y
+
+config UMMODE_LIB
+	bool "UML mode: library mode"
+	depends on !UMMODE_KERN
+	select LKL
+	default y if "$(UMMODE)" = "library"
+	help
+	  This mode switches a mode to build a library of UML (Linux
+	  Kernel Library/LKL).  This switch is exclusive to "kernel mode"
+	  of UML, which is traditional mode of UML.
+
+	  For more detail about LKL, see
+	  <file:Documentation/virt/uml/lkl.txt>.
+
+config LKL
        def_bool y
        depends on !SMP && !MMU && !COREDUMP && !SECCOMP && !UPROBES && !COMPAT && !USER_RETURN_NOTIFIER
        select ARCH_THREAD_STACK_ALLOCATOR
@@ -29,6 +48,12 @@ config OUTPUT_FORMAT
 config ARCH_DMA_ADDR_T_64BIT
        def_bool 64BIT
 
+config X86_64
+       def_bool y if "$(OUTPUT_FORMAT)" = "elf64-x86-64"
+
+config X86_32
+       def_bool y if "$(OUTPUT_FORMAT)" = "elf32-i386"
+
 config 64BIT
        def_bool n
 
@@ -39,7 +64,7 @@ config BIG_ENDIAN
        def_bool n
 
 config GENERIC_CSUM
-       def_bool y
+       def_bool LKL
 
 config GENERIC_HWEIGHT
        def_bool y
@@ -54,3 +79,5 @@ config RWSEM_GENERIC_SPINLOCK
 config HZ
         int
         default 100
+
+endmenu
diff --git a/arch/um/lkl/Makefile b/arch/um/lkl/Makefile
new file mode 100644
index 000000000000..e1161fa3fb63
--- /dev/null
+++ b/arch/um/lkl/Makefile
@@ -0,0 +1,67 @@
+# SPDX-License-Identifier: GPL-2.0
+
+include $(LKL_DIR)/auto.conf
+
+# fixup CFLAGS of um build
+KBUILD_CFLAGS := $(subst $(CFLAGS),,$(KBUILD_CFLAGS))
+
+SRCARCH := um/lkl
+ARCH_INCLUDE += -I$(srctree)/$(LKL_DIR)/um/include
+LINUXINCLUDE := $(subst $(ARCH_DIR),$(LKL_DIR),$(LINUXINCLUDE)) $(ARCH_INCLUDE)
+KBUILD_CFLAGS += -fno-builtin
+KBUILD_DEFCONFIG := lkl_defconfig
+
+ifneq (,$(filter $(OUTPUT_FORMAT),elf64-x86-64 elf32-i386 elf64-x86-64-freebsd elf32-littlearm elf64-littleaarch64))
+KBUILD_CFLAGS += -fPIC
+else ifneq (,$(filter $(OUTPUT_FORMAT),pe-i386 pe-x86-64 ))
+ifneq ($(OUTPUT_FORMAT),pe-x86-64)
+prefix=_
+endif
+# workaround for #include_next<stdarg.h> errors
+LINUXINCLUDE := -isystem $(LKL_DIR)/include/system $(LINUXINCLUDE)
+# workaround for https://gcc.gnu.org/bugzilla/show_bug.cgi?id=52991
+KBUILD_CFLAGS += -mno-ms-bitfields
+else
+$(error Unrecognized platform: $(OUTPUT_FORMAT))
+endif
+
+ifeq ($(shell uname -s), Linux)
+NPROC=$(shell nproc)
+else # e.g., FreeBSD
+NPROC=$(shell sysctl -n hw.ncpu)
+endif
+
+LDFLAGS_vmlinux += -r
+LKL_ENTRY_POINTS := lkl_start_kernel lkl_sys_halt lkl_syscall lkl_trigger_irq \
+	lkl_get_free_irq lkl_put_irq lkl_is_running lkl_bug lkl_printf
+
+ifeq ($(OUTPUT_FORMAT),elf32-i386)
+LKL_ENTRY_POINTS += \
+	__x86.get_pc_thunk.bx __x86.get_pc_thunk.dx __x86.get_pc_thunk.ax \
+	__x86.get_pc_thunk.cx __x86.get_pc_thunk.si __x86.get_pc_thunk.di
+endif
+
+core-y += $(LKL_DIR)/kernel/
+core-y += $(LKL_DIR)/mm/
+
+all: lkl.o
+
+lkl.o: vmlinux
+	$(OBJCOPY) -R .eh_frame -R .syscall_defs $(foreach sym,$(LKL_ENTRY_POINTS),-G$(prefix)$(sym)) vmlinux lkl.o
+
+$(LKL_DIR)/include/generated/uapi/asm/syscall_defs.h: vmlinux
+	$(OBJCOPY) -j .syscall_defs -O binary --set-section-flags .syscall_defs=alloc $< $@
+	$(Q) export tmpfile=$(shell mktemp); \
+	sed 's/\x0//g' $@ > $$tmpfile; mv $$tmpfile $@ ; rm -f $$tmpfile
+
+install: lkl.o headers $(LKL_DIR)/include/generated/uapi/asm/syscall_defs.h
+	@echo "  INSTALL	$(INSTALL_PATH)/lib/lkl.o"
+	@mkdir -p $(INSTALL_PATH)/lib/
+	@cp lkl.o $(INSTALL_PATH)/lib/
+	$(Q)$(srctree)/$(LKL_DIR)/scripts/headers_install.py \
+		$(subst -j,-j$(NPROC),$(findstring -j,$(MAKEFLAGS))) \
+		$(INSTALL_PATH)/include
+
+define archhelp
+  echo '  install	- Install library and headers to INSTALL_PATH/{lib,include}'
+endef
diff --git a/arch/um/lkl/auto.conf b/arch/um/lkl/auto.conf
new file mode 100644
index 000000000000..4bfd65a02d73
--- /dev/null
+++ b/arch/um/lkl/auto.conf
@@ -0,0 +1 @@
+export OUTPUT_FORMAT=$(shell $(LD) -r -print-output-format)
diff --git a/arch/um/lkl/kernel/Makefile b/arch/um/lkl/kernel/Makefile
new file mode 100644
index 000000000000..ef489f2f7176
--- /dev/null
+++ b/arch/um/lkl/kernel/Makefile
@@ -0,0 +1,4 @@
+extra-y := vmlinux.lds
+
+obj-y = setup.o threads.o irq.o time.o syscalls.o misc.o console.o \
+	syscalls_32.o cpu.o
diff --git a/arch/um/lkl/mm/Makefile b/arch/um/lkl/mm/Makefile
new file mode 100644
index 000000000000..2af6e3051897
--- /dev/null
+++ b/arch/um/lkl/mm/Makefile
@@ -0,0 +1 @@
+obj-y = bootmem.o
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 11/25] um lkl: plug in the build system
@ 2020-03-30 14:45     ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, linux-kernel-library, linux-arch,
	Hajime Tazaki, Akira Moroo

From: Octavian Purdila <tavi.purdila@gmail.com>

Basic Makefiles for building LKL. Add a new architecture specific
target for installing the resulting library files and headers.

To make LKL binaries build, UML introduced an additional option, UMMODE
variable, to switch the output file of build: kernel (default), or
library (LKL).  Those modes are not able to be ON at the same time.

To build on library mode, users do the following:

  make defconfig ARCH=um UMMODE=library
  make ARCH=um UMMODE=library

Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
Signed-off-by: Akira Moroo <retrage01@gmail.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/Kconfig             | 13 +++++++
 arch/um/Makefile            | 19 ++++++++++-
 arch/um/Makefile.um         | 10 ++++++
 arch/um/lkl/Kconfig         | 31 +++++++++++++++--
 arch/um/lkl/Makefile        | 67 +++++++++++++++++++++++++++++++++++++
 arch/um/lkl/auto.conf       |  1 +
 arch/um/lkl/kernel/Makefile |  4 +++
 arch/um/lkl/mm/Makefile     |  1 +
 8 files changed, 143 insertions(+), 3 deletions(-)
 create mode 100644 arch/um/Makefile.um
 create mode 100644 arch/um/lkl/Makefile
 create mode 100644 arch/um/lkl/auto.conf
 create mode 100644 arch/um/lkl/kernel/Makefile
 create mode 100644 arch/um/lkl/mm/Makefile

diff --git a/arch/um/Kconfig b/arch/um/Kconfig
index 96ab7026b037..17a6bc25b615 100644
--- a/arch/um/Kconfig
+++ b/arch/um/Kconfig
@@ -5,6 +5,10 @@ menu "UML-specific options"
 config UML
 	bool
 	default y
+
+config UMMODE_KERN
+	bool "UML mode: kernel mode"
+	default y if "$(UMMODE)" = "kernel"
 	select ARCH_HAS_KCOV
 	select ARCH_NO_PREEMPT
 	select HAVE_ARCH_AUDITSYSCALL
@@ -20,7 +24,12 @@ config UML
 	select GENERIC_CLOCKEVENTS
 	select HAVE_GCC_PLUGINS
 	select TTY # Needed for line.c
+        help
+	  This mode switches a mode to build a regular kernel executable
+          of UML.
+
 
+if UMMODE_KERN
 config MMU
 	bool
 	default y
@@ -207,6 +216,10 @@ config UML_TIME_TRAVEL_SUPPORT
 
 	  It is safe to say Y, but you probably don't need this.
 
+endif #UMMODE_KERN
+
 endmenu
 
 source "arch/um/drivers/Kconfig"
+
+source "arch/um/lkl/Kconfig"
diff --git a/arch/um/Makefile b/arch/um/Makefile
index d2daa206872d..f2a537f700c2 100644
--- a/arch/um/Makefile
+++ b/arch/um/Makefile
@@ -1,3 +1,4 @@
+# SPDX-License-Identifier: GPL-2.0
 #
 # This file is included by the global makefile so that you can add your own
 # architecture-specific flags and dependencies.
@@ -44,7 +45,7 @@ HOST_DIR := arch/$(HEADER_ARCH)
 include $(ARCH_DIR)/Makefile-skas
 include $(HOST_DIR)/Makefile.um
 
-core-y += $(HOST_DIR)/um/
+
 
 SHARED_HEADERS	:= $(ARCH_DIR)/include/shared
 ARCH_INCLUDE	:= -I$(srctree)/$(SHARED_HEADERS)
@@ -144,5 +145,21 @@ CLEAN_FILES += linux x.i gmon.out
 archclean:
 	@find . \( -name '*.bb' -o -name '*.bbg' -o -name '*.da' \
 		-o -name '*.gcov' \) -type f -print | xargs rm -f
+	$(Q)rm -rf $(srctree)/$(LKL_DIR)/include/generated
 
 export HEADER_ARCH SUBARCH USER_CFLAGS CFLAGS_NO_HARDENING OS DEV_NULL_PATH
+
+
+
+# SPDX-License-Identifier: GPL-2.0
+# select mode of UML build
+UMMODE ?= kernel
+LKL_DIR := $(ARCH_DIR)/lkl
+
+ifeq ($(UMMODE),kernel)
+	include $(ARCH_DIR)/Makefile.um
+else ifeq ($(UMMODE),library)
+	include $(ARCH_DIR)/lkl/Makefile
+endif
+
+export UMMODE LKL_DIR
diff --git a/arch/um/Makefile.um b/arch/um/Makefile.um
new file mode 100644
index 000000000000..be7ee4d1adde
--- /dev/null
+++ b/arch/um/Makefile.um
@@ -0,0 +1,10 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# This file is included by the global makefile so that you can add your own
+# architecture-specific flags and dependencies.
+#
+# Copyright (C) 2002 - 2007 Jeff Dike (jdike@{addtoit,linux.intel}.com)
+# Licensed under the GPL
+#
+
+core-y += $(HOST_DIR)/um/
diff --git a/arch/um/lkl/Kconfig b/arch/um/lkl/Kconfig
index f7e8d707a12b..c72c40226509 100644
--- a/arch/um/lkl/Kconfig
+++ b/arch/um/lkl/Kconfig
@@ -1,6 +1,25 @@
 # SPDX-License-Identifier: GPL-2.0
 
-config UML_LKL
+menu "LKL-specific options"
+
+config UML
+	bool
+	default y
+
+config UMMODE_LIB
+	bool "UML mode: library mode"
+	depends on !UMMODE_KERN
+	select LKL
+	default y if "$(UMMODE)" = "library"
+	help
+	  This mode switches a mode to build a library of UML (Linux
+	  Kernel Library/LKL).  This switch is exclusive to "kernel mode"
+	  of UML, which is traditional mode of UML.
+
+	  For more detail about LKL, see
+	  <file:Documentation/virt/uml/lkl.txt>.
+
+config LKL
        def_bool y
        depends on !SMP && !MMU && !COREDUMP && !SECCOMP && !UPROBES && !COMPAT && !USER_RETURN_NOTIFIER
        select ARCH_THREAD_STACK_ALLOCATOR
@@ -29,6 +48,12 @@ config OUTPUT_FORMAT
 config ARCH_DMA_ADDR_T_64BIT
        def_bool 64BIT
 
+config X86_64
+       def_bool y if "$(OUTPUT_FORMAT)" = "elf64-x86-64"
+
+config X86_32
+       def_bool y if "$(OUTPUT_FORMAT)" = "elf32-i386"
+
 config 64BIT
        def_bool n
 
@@ -39,7 +64,7 @@ config BIG_ENDIAN
        def_bool n
 
 config GENERIC_CSUM
-       def_bool y
+       def_bool LKL
 
 config GENERIC_HWEIGHT
        def_bool y
@@ -54,3 +79,5 @@ config RWSEM_GENERIC_SPINLOCK
 config HZ
         int
         default 100
+
+endmenu
diff --git a/arch/um/lkl/Makefile b/arch/um/lkl/Makefile
new file mode 100644
index 000000000000..e1161fa3fb63
--- /dev/null
+++ b/arch/um/lkl/Makefile
@@ -0,0 +1,67 @@
+# SPDX-License-Identifier: GPL-2.0
+
+include $(LKL_DIR)/auto.conf
+
+# fixup CFLAGS of um build
+KBUILD_CFLAGS := $(subst $(CFLAGS),,$(KBUILD_CFLAGS))
+
+SRCARCH := um/lkl
+ARCH_INCLUDE += -I$(srctree)/$(LKL_DIR)/um/include
+LINUXINCLUDE := $(subst $(ARCH_DIR),$(LKL_DIR),$(LINUXINCLUDE)) $(ARCH_INCLUDE)
+KBUILD_CFLAGS += -fno-builtin
+KBUILD_DEFCONFIG := lkl_defconfig
+
+ifneq (,$(filter $(OUTPUT_FORMAT),elf64-x86-64 elf32-i386 elf64-x86-64-freebsd elf32-littlearm elf64-littleaarch64))
+KBUILD_CFLAGS += -fPIC
+else ifneq (,$(filter $(OUTPUT_FORMAT),pe-i386 pe-x86-64 ))
+ifneq ($(OUTPUT_FORMAT),pe-x86-64)
+prefix=_
+endif
+# workaround for #include_next<stdarg.h> errors
+LINUXINCLUDE := -isystem $(LKL_DIR)/include/system $(LINUXINCLUDE)
+# workaround for https://gcc.gnu.org/bugzilla/show_bug.cgi?id=52991
+KBUILD_CFLAGS += -mno-ms-bitfields
+else
+$(error Unrecognized platform: $(OUTPUT_FORMAT))
+endif
+
+ifeq ($(shell uname -s), Linux)
+NPROC=$(shell nproc)
+else # e.g., FreeBSD
+NPROC=$(shell sysctl -n hw.ncpu)
+endif
+
+LDFLAGS_vmlinux += -r
+LKL_ENTRY_POINTS := lkl_start_kernel lkl_sys_halt lkl_syscall lkl_trigger_irq \
+	lkl_get_free_irq lkl_put_irq lkl_is_running lkl_bug lkl_printf
+
+ifeq ($(OUTPUT_FORMAT),elf32-i386)
+LKL_ENTRY_POINTS += \
+	__x86.get_pc_thunk.bx __x86.get_pc_thunk.dx __x86.get_pc_thunk.ax \
+	__x86.get_pc_thunk.cx __x86.get_pc_thunk.si __x86.get_pc_thunk.di
+endif
+
+core-y += $(LKL_DIR)/kernel/
+core-y += $(LKL_DIR)/mm/
+
+all: lkl.o
+
+lkl.o: vmlinux
+	$(OBJCOPY) -R .eh_frame -R .syscall_defs $(foreach sym,$(LKL_ENTRY_POINTS),-G$(prefix)$(sym)) vmlinux lkl.o
+
+$(LKL_DIR)/include/generated/uapi/asm/syscall_defs.h: vmlinux
+	$(OBJCOPY) -j .syscall_defs -O binary --set-section-flags .syscall_defs=alloc $< $@
+	$(Q) export tmpfile=$(shell mktemp); \
+	sed 's/\x0//g' $@ > $$tmpfile; mv $$tmpfile $@ ; rm -f $$tmpfile
+
+install: lkl.o headers $(LKL_DIR)/include/generated/uapi/asm/syscall_defs.h
+	@echo "  INSTALL	$(INSTALL_PATH)/lib/lkl.o"
+	@mkdir -p $(INSTALL_PATH)/lib/
+	@cp lkl.o $(INSTALL_PATH)/lib/
+	$(Q)$(srctree)/$(LKL_DIR)/scripts/headers_install.py \
+		$(subst -j,-j$(NPROC),$(findstring -j,$(MAKEFLAGS))) \
+		$(INSTALL_PATH)/include
+
+define archhelp
+  echo '  install	- Install library and headers to INSTALL_PATH/{lib,include}'
+endef
diff --git a/arch/um/lkl/auto.conf b/arch/um/lkl/auto.conf
new file mode 100644
index 000000000000..4bfd65a02d73
--- /dev/null
+++ b/arch/um/lkl/auto.conf
@@ -0,0 +1 @@
+export OUTPUT_FORMAT=$(shell $(LD) -r -print-output-format)
diff --git a/arch/um/lkl/kernel/Makefile b/arch/um/lkl/kernel/Makefile
new file mode 100644
index 000000000000..ef489f2f7176
--- /dev/null
+++ b/arch/um/lkl/kernel/Makefile
@@ -0,0 +1,4 @@
+extra-y := vmlinux.lds
+
+obj-y = setup.o threads.o irq.o time.o syscalls.o misc.o console.o \
+	syscalls_32.o cpu.o
diff --git a/arch/um/lkl/mm/Makefile b/arch/um/lkl/mm/Makefile
new file mode 100644
index 000000000000..2af6e3051897
--- /dev/null
+++ b/arch/um/lkl/mm/Makefile
@@ -0,0 +1 @@
+obj-y = bootmem.o
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 12/25] lkl tools: skeleton for host side library
  2020-03-30 14:45   ` Hajime Tazaki
@ 2020-03-30 14:45     ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Ben Wolsieffer, Conrad Meyer, Luca Dariz, Mark Stillwell,
	Michael Zimmermann, Motomu Utsumi, Patrick Collins,
	Petros Angelatos, Thomas Liebetraut, Xiao Jia, Yuan Liu,
	Hajime Tazaki

From: Octavian Purdila <tavi.purdila@gmail.com>

This patch adds the skeleton for the host library.

The host library is implementing the host operations needed by LKL and
is split into host dependent (depends on a specific host, e.g. POSIX
hosts) and host independent parts (will work on all supported hosts).

Cc: Ben Wolsieffer <benwolsieffer@gmail.com>
Cc: Conrad Meyer <cem@FreeBSD.org>
Cc: H.K. Jerry Chu <hkchu@google.com>
Cc: Luca Dariz <luca.dariz@gmail.com>
Cc: Mark Stillwell <mark@stillwell.me>
Cc: Michael Zimmermann <sigmaepsilon92@gmail.com>
Cc: Motomu Utsumi <motomuman@gmail.com>
Cc: Patrick Collins <pscollins@google.com>
Cc: Petros Angelatos <petrosagg@gmail.com>
Cc: Thomas Liebetraut <thomas@tommie-lie.de>
Cc: Xiao Jia <xiaoj@google.com>
Cc: Yuan Liu <liuyuan@google.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/lkl/.gitignore         |   4 +
 tools/lkl/Build              |   0
 tools/lkl/Makefile           | 125 ++++++++++++
 tools/lkl/Makefile.autoconf  |  65 +++++++
 tools/lkl/Targets            |   3 +
 tools/lkl/include/.gitignore |   1 +
 tools/lkl/include/lkl.h      | 358 +++++++++++++++++++++++++++++++++++
 tools/lkl/include/lkl_host.h |  19 ++
 tools/lkl/lib/.gitignore     |   3 +
 tools/lkl/lib/Build          |   1 +
 10 files changed, 579 insertions(+)
 create mode 100644 tools/lkl/.gitignore
 create mode 100644 tools/lkl/Build
 create mode 100644 tools/lkl/Makefile
 create mode 100644 tools/lkl/Makefile.autoconf
 create mode 100644 tools/lkl/Targets
 create mode 100644 tools/lkl/include/.gitignore
 create mode 100644 tools/lkl/include/lkl.h
 create mode 100644 tools/lkl/include/lkl_host.h
 create mode 100644 tools/lkl/lib/.gitignore
 create mode 100644 tools/lkl/lib/Build

diff --git a/tools/lkl/.gitignore b/tools/lkl/.gitignore
new file mode 100644
index 000000000000..1aed58bfe171
--- /dev/null
+++ b/tools/lkl/.gitignore
@@ -0,0 +1,4 @@
+Makefile.conf
+include/lkl_autoconf.h
+tests/autoconf.sh
+bin/stat
diff --git a/tools/lkl/Build b/tools/lkl/Build
new file mode 100644
index 000000000000..e69de29bb2d1
diff --git a/tools/lkl/Makefile b/tools/lkl/Makefile
new file mode 100644
index 000000000000..11ea7e9095bb
--- /dev/null
+++ b/tools/lkl/Makefile
@@ -0,0 +1,125 @@
+# Do not use make's built-in rules
+# (this improves performance and avoids hard-to-debug behaviour);
+# also do not print "Entering directory..." messages from make
+.SUFFIXES:
+MAKEFLAGS += -r --no-print-directory
+
+KCONFIG?=defconfig
+
+ifneq ($(silent),1)
+  ifneq ($(V),1)
+	QUIET_AUTOCONF       = @echo '  AUTOCONF '$@;
+	Q = @
+  endif
+endif
+
+PREFIX   := /usr
+
+ifeq (,$(srctree))
+  srctree := $(patsubst %/,%,$(dir $(shell pwd)))
+  srctree := $(patsubst %/,%,$(dir $(srctree)))
+endif
+export srctree
+
+-include $(srctree)/tools/scripts/Makefile.include
+
+# OUTPUT fixup should be *after* include ../scripts/Makefile.include
+ifneq ($(OUTPUT),)
+  OUTPUT := $(OUTPUT)/tools/lkl/
+else
+  OUTPUT := $(CURDIR)/
+endif
+export OUTPUT
+
+
+all:
+
+conf: $(OUTPUT)Makefile.conf
+
+$(OUTPUT)Makefile.conf: Makefile.autoconf
+	$(call QUIET_AUTOCONF, headers)$(MAKE) -f Makefile.autoconf -s
+
+-include $(OUTPUT)Makefile.conf
+
+export CFLAGS += -I$(OUTPUT)/include -Iinclude -Wall -g -O2 -Wextra \
+	 -Wno-unused-parameter \
+	 -Wno-missing-field-initializers -fno-strict-aliasing
+
+-include Targets
+
+TARGETS := $(progs-y:%=$(OUTPUT)%$(EXESUF))
+TARGETS += $(libs-y:%=$(OUTPUT)%$(SOSUF))
+all: $(TARGETS)
+
+# this workaround is for FreeBSD
+bin/stat:
+	$(Q)mkdir -p bin/
+ifeq ($(LKL_HOST_CONFIG_BSD),y)
+	$(Q)ln -sf `which gnustat` bin/stat
+	$(Q)ln -sf `which gsed` bin/sed
+else
+	$(Q)touch bin/stat
+endif
+
+# rule to build lkl.o
+$(OUTPUT)lib/lkl.o: bin/stat
+	$(Q)$(MAKE) -C $(srctree) ARCH=um UMMODE=library $(KOPT) $(KCONFIG)
+# this workaround is for arm32 linker (ld.gold)
+	$(Q)export PATH=$(shell pwd)/bin/:${PATH} ;\
+	$(MAKE) -C $(srctree) ARCH=um UMMODE=library $(KOPT) install INSTALL_PATH=$(OUTPUT)
+
+# rules to link libs
+$(OUTPUT)%$(SOSUF): LDFLAGS += -shared
+$(OUTPUT)%$(SOSUF): $(OUTPUT)%-in.o $(OUTPUT)liblkl.a
+	$(QUIET_LINK)$(CC) $(LDFLAGS) $(LDFLAGS_$*-y) -o $@ $^ $(LDLIBS) $(LDLIBS_$*-y)
+
+# liblkl is special
+$(OUTPUT)liblkl$(SOSUF): $(OUTPUT)%-in.o $(OUTPUT)lib/lkl.o
+$(OUTPUT)liblkl.a: $(OUTPUT)lib/liblkl-in.o $(OUTPUT)lib/lkl.o
+	$(QUIET_AR)$(AR) -rc $@ $^
+
+# rule to link programs
+$(OUTPUT)%$(EXESUF): $(OUTPUT)%-in.o $(OUTPUT)liblkl.a
+	$(QUIET_LINK)$(CC) $(LDFLAGS) $(LDFLAGS_$*-y) -o $@ $^ $(LDLIBS) $(LDLIBS_$*-y)
+
+# rule to build objects
+$(OUTPUT)%-in.o: $(OUTPUT)lib/lkl.o FORCE
+	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
+
+
+clean:
+	$(call QUIET_CLEAN, objects)find $(OUTPUT) -name '*.o' -delete -o -name '\.*.cmd'\
+	 -delete -o -name '\.*.d' -delete
+	$(call QUIET_CLEAN, headers)$(RM) -r $(OUTPUT)/include/lkl/
+	$(call QUIET_CLEAN, liblkl.a)$(RM) $(OUTPUT)/liblkl.a
+	$(call QUIET_CLEAN, targets)$(RM) $(TARGETS) bin/stat
+
+clean-conf: clean
+	$(call QUIET_CLEAN, Makefile.conf)$(RM) $(OUTPUT)/Makefile.conf
+
+headers_install: $(TARGETS)
+	$(call QUIET_INSTALL, headers) \
+	    install -d $(DESTDIR)$(PREFIX)/include ; \
+	    install -m 644 include/lkl.h include/lkl_host.h $(OUTPUT)include/lkl_autoconf.h \
+	      include/lkl_config.h $(DESTDIR)$(PREFIX)/include ; \
+	    cp -r $(OUTPUT)include/lkl $(DESTDIR)$(PREFIX)/include
+
+libraries_install: $(libs-y:%=$(OUTPUT)%$(SOSUF)) $(OUTPUT)liblkl.a
+	$(call QUIET_INSTALL, libraries) \
+	    install -d $(DESTDIR)$(PREFIX)/lib ; \
+	    install -m 644 $^ $(DESTDIR)$(PREFIX)/lib
+
+programs_install: $(progs-y:%=$(OUTPUT)%$(EXESUF))
+	$(call QUIET_INSTALL, programs) \
+	    install -d $(DESTDIR)$(PREFIX)/bin ; \
+	    install -m 755 $^ $(DESTDIR)$(PREFIX)/bin
+
+install: headers_install libraries_install programs_install
+
+
+FORCE: ;
+.PHONY: all clean FORCE
+.PHONY: headers_install libraries_install programs_install install
+.NOTPARALLEL : lib/lkl.o
+.SECONDARY:
+
diff --git a/tools/lkl/Makefile.autoconf b/tools/lkl/Makefile.autoconf
new file mode 100644
index 000000000000..268c367d9962
--- /dev/null
+++ b/tools/lkl/Makefile.autoconf
@@ -0,0 +1,65 @@
+POSIX_HOSTS=elf64-x86-64 elf32-i386 elf64-x86-64-freebsd
+
+define set_autoconf_var
+  $(shell echo "#define LKL_HOST_CONFIG_$(1) $(2)" \
+	  >> $(OUTPUT)/include/lkl_autoconf.h)
+  $(shell echo "LKL_HOST_CONFIG_$(1)=$(2)" >> $(OUTPUT)/tests/autoconf.sh)
+  export LKL_HOST_CONFIG_$(1)=$(2)
+endef
+
+define find_include
+  $(eval include_paths=$(shell $(CC) -E -Wp,-v -xc /dev/null 2>&1 | grep '^ '))
+  $(foreach f, $(include_paths), $(wildcard $(f)/$(1)))
+endef
+
+define is_defined
+$(shell $(CC) -dM -E - </dev/null | grep $(1))
+endef
+
+define bsd_host
+  $(call set_autoconf_var,BSD,y)
+endef
+
+define arm_host
+  $(call set_autoconf_var,ARM,y)
+endef
+
+define aarch64_host
+  $(call set_autoconf_var,AARCH64,y)
+endef
+
+define posix_host
+  $(call set_autoconf_var,POSIX,y)
+  LDFLAGS += -pie
+  CFLAGS += -fPIC -pthread
+  SOSUF := .so
+  LDLIBS += -lrt -lpthread
+  $(if $(filter $(1),elf64-x86-64-freebsd),$(call bsd_host))
+  $(if $(filter $(1),elf32-littlearm),$(call arm_host))
+  $(if $(filter $(1),elf64-littleaarch64),$(call aarch64_host))
+  $(if $(strip $(call find_include,fuse.h)),$(call set_autoconf_var,FUSE,y))
+  $(if $(strip $(call find_include,archive.h)),$(call set_autoconf_var,ARCHIVE,y))
+  $(if $(filter $(1),elf64-x86-64-freebsd),$(call set_autoconf_var,NEEDS_LARGP,y))
+  $(if $(filter $(1),elf32-i386),$(call set_autoconf_var,I386,y))
+endef
+
+define do_autoconf
+  export CROSS_COMPILE := $(CROSS_COMPILE)
+  export CC := $(CROSS_COMPILE)gcc
+  export LD := $(CROSS_COMPILE)ld
+  export AR := $(CROSS_COMPILE)ar
+  $(eval LD := $(CROSS_COMPILE)ld)
+  $(eval CC := $(CROSS_COMPILE)gcc)
+  $(eval LD_FMT := $(shell $(LD) -r -print-output-format))
+  $(if $(filter $(LD_FMT),$(POSIX_HOSTS)),$(call posix_host,$(LD_FMT)))
+endef
+
+export do_autoconf
+
+
+$(OUTPUT)Makefile.conf: Makefile.autoconf
+	$(shell mkdir -p $(OUTPUT)/include)
+	$(shell mkdir -p $(OUTPUT)/tests)
+	$(shell echo -n "" > $(OUTPUT)/include/lkl_autoconf.h)
+	$(shell echo -n "" > $(OUTPUT)/tests/autoconf.sh)
+	@echo "$$do_autoconf" > $(OUTPUT)/Makefile.conf
diff --git a/tools/lkl/Targets b/tools/lkl/Targets
new file mode 100644
index 000000000000..24c985e64638
--- /dev/null
+++ b/tools/lkl/Targets
@@ -0,0 +1,3 @@
+libs-y += lib/liblkl
+
+
diff --git a/tools/lkl/include/.gitignore b/tools/lkl/include/.gitignore
new file mode 100644
index 000000000000..c41a463c898d
--- /dev/null
+++ b/tools/lkl/include/.gitignore
@@ -0,0 +1 @@
+lkl/
\ No newline at end of file
diff --git a/tools/lkl/include/lkl.h b/tools/lkl/include/lkl.h
new file mode 100644
index 000000000000..4b95d0ef8e5b
--- /dev/null
+++ b/tools/lkl/include/lkl.h
@@ -0,0 +1,358 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_H
+#define _LKL_H
+
+#include "lkl_autoconf.h"
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#define _LKL_LIBC_COMPAT_H
+
+#ifdef __cplusplus
+#define class __lkl__class
+#endif
+
+/*
+ * Avoid collisions between Android which defines __unused and
+ * linux/icmp.h which uses __unused as a structure field.
+ */
+#pragma push_macro("__unused")
+#undef __unused
+
+#include <lkl/asm/syscalls.h>
+
+#pragma pop_macro("__unused")
+
+#ifdef __cplusplus
+#undef class
+#endif
+
+#if defined(__MINGW32__)
+#define strtok_r strtok_s
+#define inet_pton lkl_inet_pton
+
+int inet_pton(int af, const char *src, void *dst);
+#endif
+
+#if __LKL__BITS_PER_LONG == 64
+#define lkl_sys_fstatat lkl_sys_newfstatat
+#define lkl_sys_fstat lkl_sys_newfstat
+
+#else
+#define __lkl__NR_fcntl __lkl__NR_fcntl64
+
+#define lkl_stat lkl_stat64
+#define lkl_sys_stat lkl_sys_stat64
+#define lkl_sys_lstat lkl_sys_lstat64
+#define lkl_sys_truncate lkl_sys_truncate64
+#define lkl_sys_ftruncate lkl_sys_ftruncate64
+#define lkl_sys_sendfile lkl_sys_sendfile64
+#define lkl_sys_fstatat lkl_sys_fstatat64
+#define lkl_sys_fstat lkl_sys_fstat64
+#define lkl_sys_fcntl lkl_sys_fcntl64
+
+#define lkl_statfs lkl_statfs64
+
+static inline int lkl_sys_statfs(const char *path, struct lkl_statfs *buf)
+{
+	return lkl_sys_statfs64(path, sizeof(*buf), buf);
+}
+
+static inline int lkl_sys_fstatfs(unsigned int fd, struct lkl_statfs *buf)
+{
+	return lkl_sys_fstatfs64(fd, sizeof(*buf), buf);
+}
+
+#define lkl_sys_nanosleep lkl_sys_nanosleep_time32
+static inline int lkl_sys_nanosleep_time32(struct __lkl__kernel_timespec *rqtp,
+					   struct __lkl__kernel_timespec *rmtp)
+{
+	long p[6] = {(long)rqtp, (long)rmtp, 0, 0, 0, 0};
+
+	return lkl_syscall(__lkl__NR_nanosleep, p);
+}
+
+#endif
+
+static inline int lkl_sys_stat(const char *path, struct lkl_stat *buf)
+{
+	return lkl_sys_fstatat(LKL_AT_FDCWD, path, buf, 0);
+}
+
+static inline int lkl_sys_lstat(const char *path, struct lkl_stat *buf)
+{
+	return lkl_sys_fstatat(LKL_AT_FDCWD, path, buf,
+			       LKL_AT_SYMLINK_NOFOLLOW);
+}
+
+#ifdef __lkl__NR_llseek
+/**
+ * lkl_sys_lseek - wrapper for lkl_sys_llseek
+ */
+static inline long long lkl_sys_lseek(unsigned int fd, __lkl__kernel_loff_t off,
+				      unsigned int whence)
+{
+	long long res;
+	long ret = lkl_sys_llseek(fd, off >> 32, off & 0xffffffff, &res,
+				  whence);
+
+	return ret < 0 ? ret : res;
+}
+#endif
+
+static inline void *lkl_sys_mmap(void *addr, size_t length, int prot, int flags,
+				 int fd, off_t offset)
+{
+	return (void *)lkl_sys_mmap_pgoff((long)addr, length, prot, flags, fd,
+					  offset >> 12);
+}
+
+#define lkl_sys_mmap2 lkl_sys_mmap_pgoff
+
+#ifdef __lkl__NR_openat
+/**
+ * lkl_sys_open - wrapper for lkl_sys_openat
+ */
+static inline long lkl_sys_open(const char *file, int flags, int mode)
+{
+	return lkl_sys_openat(LKL_AT_FDCWD, file, flags, mode);
+}
+
+/**
+ * lkl_sys_creat - wrapper for lkl_sys_openat
+ */
+static inline long lkl_sys_creat(const char *file, int mode)
+{
+	return lkl_sys_openat(LKL_AT_FDCWD, file,
+			      LKL_O_CREAT|LKL_O_WRONLY|LKL_O_TRUNC, mode);
+}
+#endif
+
+
+#ifdef __lkl__NR_faccessat
+/**
+ * lkl_sys_access - wrapper for lkl_sys_faccessat
+ */
+static inline long lkl_sys_access(const char *file, int mode)
+{
+	return lkl_sys_faccessat(LKL_AT_FDCWD, file, mode);
+}
+#endif
+
+#ifdef __lkl__NR_fchownat
+/**
+ * lkl_sys_chown - wrapper for lkl_sys_fchownat
+ */
+static inline long lkl_sys_chown(const char *path, lkl_uid_t uid, lkl_gid_t gid)
+{
+	return lkl_sys_fchownat(LKL_AT_FDCWD, path, uid, gid, 0);
+}
+#endif
+
+#ifdef __lkl__NR_fchmodat
+/**
+ * lkl_sys_chmod - wrapper for lkl_sys_fchmodat
+ */
+static inline long lkl_sys_chmod(const char *path, mode_t mode)
+{
+	return lkl_sys_fchmodat(LKL_AT_FDCWD, path, mode);
+}
+#endif
+
+#ifdef __lkl__NR_linkat
+/**
+ * lkl_sys_link - wrapper for lkl_sys_linkat
+ */
+static inline long lkl_sys_link(const char *existing, const char *new)
+{
+	return lkl_sys_linkat(LKL_AT_FDCWD, existing, LKL_AT_FDCWD, new, 0);
+}
+#endif
+
+#ifdef __lkl__NR_unlinkat
+/**
+ * lkl_sys_unlink - wrapper for lkl_sys_unlinkat
+ */
+static inline long lkl_sys_unlink(const char *path)
+{
+	return lkl_sys_unlinkat(LKL_AT_FDCWD, path, 0);
+}
+#endif
+
+#ifdef __lkl__NR_symlinkat
+/**
+ * lkl_sys_symlink - wrapper for lkl_sys_symlinkat
+ */
+static inline long lkl_sys_symlink(const char *existing, const char *new)
+{
+	return lkl_sys_symlinkat(existing, LKL_AT_FDCWD, new);
+}
+#endif
+
+#ifdef __lkl__NR_readlinkat
+/**
+ * lkl_sys_readlink - wrapper for lkl_sys_readlinkat
+ */
+static inline long lkl_sys_readlink(const char *path, char *buf, size_t bufsize)
+{
+	return lkl_sys_readlinkat(LKL_AT_FDCWD, path, buf, bufsize);
+}
+#endif
+
+#ifdef __lkl__NR_renameat
+/**
+ * lkl_sys_rename - wrapper for lkl_sys_renameat
+ */
+static inline long lkl_sys_rename(const char *old, const char *new)
+{
+	return lkl_sys_renameat(LKL_AT_FDCWD, old, LKL_AT_FDCWD, new);
+}
+#endif
+
+#ifdef __lkl__NR_mkdirat
+/**
+ * lkl_sys_mkdir - wrapper for lkl_sys_mkdirat
+ */
+static inline long lkl_sys_mkdir(const char *path, mode_t mode)
+{
+	return lkl_sys_mkdirat(LKL_AT_FDCWD, path, mode);
+}
+#endif
+
+#ifdef __lkl__NR_unlinkat
+/**
+ * lkl_sys_rmdir - wrapper for lkl_sys_unlinkrat
+ */
+static inline long lkl_sys_rmdir(const char *path)
+{
+	return lkl_sys_unlinkat(LKL_AT_FDCWD, path, LKL_AT_REMOVEDIR);
+}
+#endif
+
+#ifdef __lkl__NR_mknodat
+/**
+ * lkl_sys_mknod - wrapper for lkl_sys_mknodat
+ */
+static inline long lkl_sys_mknod(const char *path, mode_t mode, dev_t dev)
+{
+	return lkl_sys_mknodat(LKL_AT_FDCWD, path, mode, dev);
+}
+#endif
+
+#ifdef __lkl__NR_pipe2
+/**
+ * lkl_sys_pipe - wrapper for lkl_sys_pipe2
+ */
+static inline long lkl_sys_pipe(int fd[2])
+{
+	return lkl_sys_pipe2(fd, 0);
+}
+#endif
+
+#ifdef __lkl__NR_sendto
+/**
+ * lkl_sys_send - wrapper for lkl_sys_sendto
+ */
+static inline long lkl_sys_send(int fd, void *buf, size_t len, int flags)
+{
+	return lkl_sys_sendto(fd, buf, len, flags, 0, 0);
+}
+#endif
+
+#ifdef __lkl__NR_recvfrom
+/**
+ * lkl_sys_recv - wrapper for lkl_sys_recvfrom
+ */
+static inline long lkl_sys_recv(int fd, void *buf, size_t len, int flags)
+{
+	return lkl_sys_recvfrom(fd, buf, len, flags, 0, 0);
+}
+#endif
+
+#ifdef __lkl__NR_pselect6
+/**
+ * lkl_sys_select - wrapper for lkl_sys_pselect
+ */
+static inline long lkl_sys_select(int n, lkl_fd_set *rfds, lkl_fd_set *wfds,
+				  lkl_fd_set *efds, struct lkl_timeval *tv)
+{
+	long data[2] = { 0, _LKL_NSIG/8 };
+	struct lkl_timespec ts;
+	lkl_time_t extra_secs;
+	const lkl_time_t max_time = ((1ULL<<8)*sizeof(time_t)-1)-1;
+
+	if (tv) {
+		if (tv->tv_sec < 0 || tv->tv_usec < 0)
+			return -LKL_EINVAL;
+
+		extra_secs = tv->tv_usec / 1000000;
+		ts.tv_nsec = tv->tv_usec % 1000000 * 1000;
+		ts.tv_sec = extra_secs > max_time - tv->tv_sec ?
+			max_time : tv->tv_sec + extra_secs;
+	}
+	return lkl_sys_pselect6(n, rfds, wfds, efds, tv ?
+				(struct __lkl__kernel_timespec *)&ts : 0, data);
+}
+#endif
+
+#ifdef __lkl__NR_ppoll
+/**
+ * lkl_sys_poll - wrapper for lkl_sys_ppoll
+ */
+static inline long lkl_sys_poll(struct lkl_pollfd *fds, int n, int timeout)
+{
+	return lkl_sys_ppoll(fds, n, timeout >= 0 ?
+			     (struct __lkl__kernel_timespec *)
+			     &((struct lkl_timespec){ .tv_sec = timeout/1000,
+				   .tv_nsec = timeout%1000*1000000 }) : 0,
+			     0, _LKL_NSIG/8);
+}
+#endif
+
+#ifdef __lkl__NR_epoll_create1
+/**
+ * lkl_sys_epoll_create - wrapper for lkl_sys_epoll_create1
+ */
+static inline long lkl_sys_epoll_create(int size)
+{
+	return lkl_sys_epoll_create1(0);
+}
+#endif
+
+#ifdef __lkl__NR_epoll_pwait
+/**
+ * lkl_sys_epoll_wait - wrapper for lkl_sys_epoll_pwait
+ */
+static inline long lkl_sys_epoll_wait(int fd, struct lkl_epoll_event *ev,
+				      int cnt, int to)
+{
+	return lkl_sys_epoll_pwait(fd, ev, cnt, to, 0, _LKL_NSIG/8);
+}
+#endif
+
+
+
+/**
+ * lkl_strerror - returns a string describing the given error code
+ *
+ * @err - error code
+ * @returns - string for the given error code
+ */
+const char *lkl_strerror(int err);
+
+/**
+ * lkl_perror - prints a string describing the given error code
+ *
+ * @msg - prefix for the error message
+ * @err - error code
+ */
+void lkl_perror(char *msg, int err);
+
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif
diff --git a/tools/lkl/include/lkl_host.h b/tools/lkl/include/lkl_host.h
new file mode 100644
index 000000000000..b5f96096fe69
--- /dev/null
+++ b/tools/lkl/include/lkl_host.h
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_HOST_H
+#define _LKL_HOST_H
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include <lkl/asm/host_ops.h>
+#include <lkl.h>
+
+extern struct lkl_host_operations lkl_host_ops;
+
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif
diff --git a/tools/lkl/lib/.gitignore b/tools/lkl/lib/.gitignore
new file mode 100644
index 000000000000..427ae0273fdd
--- /dev/null
+++ b/tools/lkl/lib/.gitignore
@@ -0,0 +1,3 @@
+lkl.o
+liblkl.a
+
diff --git a/tools/lkl/lib/Build b/tools/lkl/lib/Build
new file mode 100644
index 000000000000..8b137891791f
--- /dev/null
+++ b/tools/lkl/lib/Build
@@ -0,0 +1 @@
+
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 12/25] lkl tools: skeleton for host side library
@ 2020-03-30 14:45     ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: linux-arch, Patrick Collins, Xiao Jia, Hajime Tazaki,
	Conrad Meyer, Octavian Purdila, Motomu Utsumi, Akira Moroo,
	Petros Angelatos, Thomas Liebetraut, Mark Stillwell,
	Ben Wolsieffer, linux-kernel-library, Michael Zimmermann,
	Luca Dariz, Yuan Liu

From: Octavian Purdila <tavi.purdila@gmail.com>

This patch adds the skeleton for the host library.

The host library is implementing the host operations needed by LKL and
is split into host dependent (depends on a specific host, e.g. POSIX
hosts) and host independent parts (will work on all supported hosts).

Cc: Ben Wolsieffer <benwolsieffer@gmail.com>
Cc: Conrad Meyer <cem@FreeBSD.org>
Cc: H.K. Jerry Chu <hkchu@google.com>
Cc: Luca Dariz <luca.dariz@gmail.com>
Cc: Mark Stillwell <mark@stillwell.me>
Cc: Michael Zimmermann <sigmaepsilon92@gmail.com>
Cc: Motomu Utsumi <motomuman@gmail.com>
Cc: Patrick Collins <pscollins@google.com>
Cc: Petros Angelatos <petrosagg@gmail.com>
Cc: Thomas Liebetraut <thomas@tommie-lie.de>
Cc: Xiao Jia <xiaoj@google.com>
Cc: Yuan Liu <liuyuan@google.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/lkl/.gitignore         |   4 +
 tools/lkl/Build              |   0
 tools/lkl/Makefile           | 125 ++++++++++++
 tools/lkl/Makefile.autoconf  |  65 +++++++
 tools/lkl/Targets            |   3 +
 tools/lkl/include/.gitignore |   1 +
 tools/lkl/include/lkl.h      | 358 +++++++++++++++++++++++++++++++++++
 tools/lkl/include/lkl_host.h |  19 ++
 tools/lkl/lib/.gitignore     |   3 +
 tools/lkl/lib/Build          |   1 +
 10 files changed, 579 insertions(+)
 create mode 100644 tools/lkl/.gitignore
 create mode 100644 tools/lkl/Build
 create mode 100644 tools/lkl/Makefile
 create mode 100644 tools/lkl/Makefile.autoconf
 create mode 100644 tools/lkl/Targets
 create mode 100644 tools/lkl/include/.gitignore
 create mode 100644 tools/lkl/include/lkl.h
 create mode 100644 tools/lkl/include/lkl_host.h
 create mode 100644 tools/lkl/lib/.gitignore
 create mode 100644 tools/lkl/lib/Build

diff --git a/tools/lkl/.gitignore b/tools/lkl/.gitignore
new file mode 100644
index 000000000000..1aed58bfe171
--- /dev/null
+++ b/tools/lkl/.gitignore
@@ -0,0 +1,4 @@
+Makefile.conf
+include/lkl_autoconf.h
+tests/autoconf.sh
+bin/stat
diff --git a/tools/lkl/Build b/tools/lkl/Build
new file mode 100644
index 000000000000..e69de29bb2d1
diff --git a/tools/lkl/Makefile b/tools/lkl/Makefile
new file mode 100644
index 000000000000..11ea7e9095bb
--- /dev/null
+++ b/tools/lkl/Makefile
@@ -0,0 +1,125 @@
+# Do not use make's built-in rules
+# (this improves performance and avoids hard-to-debug behaviour);
+# also do not print "Entering directory..." messages from make
+.SUFFIXES:
+MAKEFLAGS += -r --no-print-directory
+
+KCONFIG?=defconfig
+
+ifneq ($(silent),1)
+  ifneq ($(V),1)
+	QUIET_AUTOCONF       = @echo '  AUTOCONF '$@;
+	Q = @
+  endif
+endif
+
+PREFIX   := /usr
+
+ifeq (,$(srctree))
+  srctree := $(patsubst %/,%,$(dir $(shell pwd)))
+  srctree := $(patsubst %/,%,$(dir $(srctree)))
+endif
+export srctree
+
+-include $(srctree)/tools/scripts/Makefile.include
+
+# OUTPUT fixup should be *after* include ../scripts/Makefile.include
+ifneq ($(OUTPUT),)
+  OUTPUT := $(OUTPUT)/tools/lkl/
+else
+  OUTPUT := $(CURDIR)/
+endif
+export OUTPUT
+
+
+all:
+
+conf: $(OUTPUT)Makefile.conf
+
+$(OUTPUT)Makefile.conf: Makefile.autoconf
+	$(call QUIET_AUTOCONF, headers)$(MAKE) -f Makefile.autoconf -s
+
+-include $(OUTPUT)Makefile.conf
+
+export CFLAGS += -I$(OUTPUT)/include -Iinclude -Wall -g -O2 -Wextra \
+	 -Wno-unused-parameter \
+	 -Wno-missing-field-initializers -fno-strict-aliasing
+
+-include Targets
+
+TARGETS := $(progs-y:%=$(OUTPUT)%$(EXESUF))
+TARGETS += $(libs-y:%=$(OUTPUT)%$(SOSUF))
+all: $(TARGETS)
+
+# this workaround is for FreeBSD
+bin/stat:
+	$(Q)mkdir -p bin/
+ifeq ($(LKL_HOST_CONFIG_BSD),y)
+	$(Q)ln -sf `which gnustat` bin/stat
+	$(Q)ln -sf `which gsed` bin/sed
+else
+	$(Q)touch bin/stat
+endif
+
+# rule to build lkl.o
+$(OUTPUT)lib/lkl.o: bin/stat
+	$(Q)$(MAKE) -C $(srctree) ARCH=um UMMODE=library $(KOPT) $(KCONFIG)
+# this workaround is for arm32 linker (ld.gold)
+	$(Q)export PATH=$(shell pwd)/bin/:${PATH} ;\
+	$(MAKE) -C $(srctree) ARCH=um UMMODE=library $(KOPT) install INSTALL_PATH=$(OUTPUT)
+
+# rules to link libs
+$(OUTPUT)%$(SOSUF): LDFLAGS += -shared
+$(OUTPUT)%$(SOSUF): $(OUTPUT)%-in.o $(OUTPUT)liblkl.a
+	$(QUIET_LINK)$(CC) $(LDFLAGS) $(LDFLAGS_$*-y) -o $@ $^ $(LDLIBS) $(LDLIBS_$*-y)
+
+# liblkl is special
+$(OUTPUT)liblkl$(SOSUF): $(OUTPUT)%-in.o $(OUTPUT)lib/lkl.o
+$(OUTPUT)liblkl.a: $(OUTPUT)lib/liblkl-in.o $(OUTPUT)lib/lkl.o
+	$(QUIET_AR)$(AR) -rc $@ $^
+
+# rule to link programs
+$(OUTPUT)%$(EXESUF): $(OUTPUT)%-in.o $(OUTPUT)liblkl.a
+	$(QUIET_LINK)$(CC) $(LDFLAGS) $(LDFLAGS_$*-y) -o $@ $^ $(LDLIBS) $(LDLIBS_$*-y)
+
+# rule to build objects
+$(OUTPUT)%-in.o: $(OUTPUT)lib/lkl.o FORCE
+	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
+
+
+clean:
+	$(call QUIET_CLEAN, objects)find $(OUTPUT) -name '*.o' -delete -o -name '\.*.cmd'\
+	 -delete -o -name '\.*.d' -delete
+	$(call QUIET_CLEAN, headers)$(RM) -r $(OUTPUT)/include/lkl/
+	$(call QUIET_CLEAN, liblkl.a)$(RM) $(OUTPUT)/liblkl.a
+	$(call QUIET_CLEAN, targets)$(RM) $(TARGETS) bin/stat
+
+clean-conf: clean
+	$(call QUIET_CLEAN, Makefile.conf)$(RM) $(OUTPUT)/Makefile.conf
+
+headers_install: $(TARGETS)
+	$(call QUIET_INSTALL, headers) \
+	    install -d $(DESTDIR)$(PREFIX)/include ; \
+	    install -m 644 include/lkl.h include/lkl_host.h $(OUTPUT)include/lkl_autoconf.h \
+	      include/lkl_config.h $(DESTDIR)$(PREFIX)/include ; \
+	    cp -r $(OUTPUT)include/lkl $(DESTDIR)$(PREFIX)/include
+
+libraries_install: $(libs-y:%=$(OUTPUT)%$(SOSUF)) $(OUTPUT)liblkl.a
+	$(call QUIET_INSTALL, libraries) \
+	    install -d $(DESTDIR)$(PREFIX)/lib ; \
+	    install -m 644 $^ $(DESTDIR)$(PREFIX)/lib
+
+programs_install: $(progs-y:%=$(OUTPUT)%$(EXESUF))
+	$(call QUIET_INSTALL, programs) \
+	    install -d $(DESTDIR)$(PREFIX)/bin ; \
+	    install -m 755 $^ $(DESTDIR)$(PREFIX)/bin
+
+install: headers_install libraries_install programs_install
+
+
+FORCE: ;
+.PHONY: all clean FORCE
+.PHONY: headers_install libraries_install programs_install install
+.NOTPARALLEL : lib/lkl.o
+.SECONDARY:
+
diff --git a/tools/lkl/Makefile.autoconf b/tools/lkl/Makefile.autoconf
new file mode 100644
index 000000000000..268c367d9962
--- /dev/null
+++ b/tools/lkl/Makefile.autoconf
@@ -0,0 +1,65 @@
+POSIX_HOSTS=elf64-x86-64 elf32-i386 elf64-x86-64-freebsd
+
+define set_autoconf_var
+  $(shell echo "#define LKL_HOST_CONFIG_$(1) $(2)" \
+	  >> $(OUTPUT)/include/lkl_autoconf.h)
+  $(shell echo "LKL_HOST_CONFIG_$(1)=$(2)" >> $(OUTPUT)/tests/autoconf.sh)
+  export LKL_HOST_CONFIG_$(1)=$(2)
+endef
+
+define find_include
+  $(eval include_paths=$(shell $(CC) -E -Wp,-v -xc /dev/null 2>&1 | grep '^ '))
+  $(foreach f, $(include_paths), $(wildcard $(f)/$(1)))
+endef
+
+define is_defined
+$(shell $(CC) -dM -E - </dev/null | grep $(1))
+endef
+
+define bsd_host
+  $(call set_autoconf_var,BSD,y)
+endef
+
+define arm_host
+  $(call set_autoconf_var,ARM,y)
+endef
+
+define aarch64_host
+  $(call set_autoconf_var,AARCH64,y)
+endef
+
+define posix_host
+  $(call set_autoconf_var,POSIX,y)
+  LDFLAGS += -pie
+  CFLAGS += -fPIC -pthread
+  SOSUF := .so
+  LDLIBS += -lrt -lpthread
+  $(if $(filter $(1),elf64-x86-64-freebsd),$(call bsd_host))
+  $(if $(filter $(1),elf32-littlearm),$(call arm_host))
+  $(if $(filter $(1),elf64-littleaarch64),$(call aarch64_host))
+  $(if $(strip $(call find_include,fuse.h)),$(call set_autoconf_var,FUSE,y))
+  $(if $(strip $(call find_include,archive.h)),$(call set_autoconf_var,ARCHIVE,y))
+  $(if $(filter $(1),elf64-x86-64-freebsd),$(call set_autoconf_var,NEEDS_LARGP,y))
+  $(if $(filter $(1),elf32-i386),$(call set_autoconf_var,I386,y))
+endef
+
+define do_autoconf
+  export CROSS_COMPILE := $(CROSS_COMPILE)
+  export CC := $(CROSS_COMPILE)gcc
+  export LD := $(CROSS_COMPILE)ld
+  export AR := $(CROSS_COMPILE)ar
+  $(eval LD := $(CROSS_COMPILE)ld)
+  $(eval CC := $(CROSS_COMPILE)gcc)
+  $(eval LD_FMT := $(shell $(LD) -r -print-output-format))
+  $(if $(filter $(LD_FMT),$(POSIX_HOSTS)),$(call posix_host,$(LD_FMT)))
+endef
+
+export do_autoconf
+
+
+$(OUTPUT)Makefile.conf: Makefile.autoconf
+	$(shell mkdir -p $(OUTPUT)/include)
+	$(shell mkdir -p $(OUTPUT)/tests)
+	$(shell echo -n "" > $(OUTPUT)/include/lkl_autoconf.h)
+	$(shell echo -n "" > $(OUTPUT)/tests/autoconf.sh)
+	@echo "$$do_autoconf" > $(OUTPUT)/Makefile.conf
diff --git a/tools/lkl/Targets b/tools/lkl/Targets
new file mode 100644
index 000000000000..24c985e64638
--- /dev/null
+++ b/tools/lkl/Targets
@@ -0,0 +1,3 @@
+libs-y += lib/liblkl
+
+
diff --git a/tools/lkl/include/.gitignore b/tools/lkl/include/.gitignore
new file mode 100644
index 000000000000..c41a463c898d
--- /dev/null
+++ b/tools/lkl/include/.gitignore
@@ -0,0 +1 @@
+lkl/
\ No newline at end of file
diff --git a/tools/lkl/include/lkl.h b/tools/lkl/include/lkl.h
new file mode 100644
index 000000000000..4b95d0ef8e5b
--- /dev/null
+++ b/tools/lkl/include/lkl.h
@@ -0,0 +1,358 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_H
+#define _LKL_H
+
+#include "lkl_autoconf.h"
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#define _LKL_LIBC_COMPAT_H
+
+#ifdef __cplusplus
+#define class __lkl__class
+#endif
+
+/*
+ * Avoid collisions between Android which defines __unused and
+ * linux/icmp.h which uses __unused as a structure field.
+ */
+#pragma push_macro("__unused")
+#undef __unused
+
+#include <lkl/asm/syscalls.h>
+
+#pragma pop_macro("__unused")
+
+#ifdef __cplusplus
+#undef class
+#endif
+
+#if defined(__MINGW32__)
+#define strtok_r strtok_s
+#define inet_pton lkl_inet_pton
+
+int inet_pton(int af, const char *src, void *dst);
+#endif
+
+#if __LKL__BITS_PER_LONG == 64
+#define lkl_sys_fstatat lkl_sys_newfstatat
+#define lkl_sys_fstat lkl_sys_newfstat
+
+#else
+#define __lkl__NR_fcntl __lkl__NR_fcntl64
+
+#define lkl_stat lkl_stat64
+#define lkl_sys_stat lkl_sys_stat64
+#define lkl_sys_lstat lkl_sys_lstat64
+#define lkl_sys_truncate lkl_sys_truncate64
+#define lkl_sys_ftruncate lkl_sys_ftruncate64
+#define lkl_sys_sendfile lkl_sys_sendfile64
+#define lkl_sys_fstatat lkl_sys_fstatat64
+#define lkl_sys_fstat lkl_sys_fstat64
+#define lkl_sys_fcntl lkl_sys_fcntl64
+
+#define lkl_statfs lkl_statfs64
+
+static inline int lkl_sys_statfs(const char *path, struct lkl_statfs *buf)
+{
+	return lkl_sys_statfs64(path, sizeof(*buf), buf);
+}
+
+static inline int lkl_sys_fstatfs(unsigned int fd, struct lkl_statfs *buf)
+{
+	return lkl_sys_fstatfs64(fd, sizeof(*buf), buf);
+}
+
+#define lkl_sys_nanosleep lkl_sys_nanosleep_time32
+static inline int lkl_sys_nanosleep_time32(struct __lkl__kernel_timespec *rqtp,
+					   struct __lkl__kernel_timespec *rmtp)
+{
+	long p[6] = {(long)rqtp, (long)rmtp, 0, 0, 0, 0};
+
+	return lkl_syscall(__lkl__NR_nanosleep, p);
+}
+
+#endif
+
+static inline int lkl_sys_stat(const char *path, struct lkl_stat *buf)
+{
+	return lkl_sys_fstatat(LKL_AT_FDCWD, path, buf, 0);
+}
+
+static inline int lkl_sys_lstat(const char *path, struct lkl_stat *buf)
+{
+	return lkl_sys_fstatat(LKL_AT_FDCWD, path, buf,
+			       LKL_AT_SYMLINK_NOFOLLOW);
+}
+
+#ifdef __lkl__NR_llseek
+/**
+ * lkl_sys_lseek - wrapper for lkl_sys_llseek
+ */
+static inline long long lkl_sys_lseek(unsigned int fd, __lkl__kernel_loff_t off,
+				      unsigned int whence)
+{
+	long long res;
+	long ret = lkl_sys_llseek(fd, off >> 32, off & 0xffffffff, &res,
+				  whence);
+
+	return ret < 0 ? ret : res;
+}
+#endif
+
+static inline void *lkl_sys_mmap(void *addr, size_t length, int prot, int flags,
+				 int fd, off_t offset)
+{
+	return (void *)lkl_sys_mmap_pgoff((long)addr, length, prot, flags, fd,
+					  offset >> 12);
+}
+
+#define lkl_sys_mmap2 lkl_sys_mmap_pgoff
+
+#ifdef __lkl__NR_openat
+/**
+ * lkl_sys_open - wrapper for lkl_sys_openat
+ */
+static inline long lkl_sys_open(const char *file, int flags, int mode)
+{
+	return lkl_sys_openat(LKL_AT_FDCWD, file, flags, mode);
+}
+
+/**
+ * lkl_sys_creat - wrapper for lkl_sys_openat
+ */
+static inline long lkl_sys_creat(const char *file, int mode)
+{
+	return lkl_sys_openat(LKL_AT_FDCWD, file,
+			      LKL_O_CREAT|LKL_O_WRONLY|LKL_O_TRUNC, mode);
+}
+#endif
+
+
+#ifdef __lkl__NR_faccessat
+/**
+ * lkl_sys_access - wrapper for lkl_sys_faccessat
+ */
+static inline long lkl_sys_access(const char *file, int mode)
+{
+	return lkl_sys_faccessat(LKL_AT_FDCWD, file, mode);
+}
+#endif
+
+#ifdef __lkl__NR_fchownat
+/**
+ * lkl_sys_chown - wrapper for lkl_sys_fchownat
+ */
+static inline long lkl_sys_chown(const char *path, lkl_uid_t uid, lkl_gid_t gid)
+{
+	return lkl_sys_fchownat(LKL_AT_FDCWD, path, uid, gid, 0);
+}
+#endif
+
+#ifdef __lkl__NR_fchmodat
+/**
+ * lkl_sys_chmod - wrapper for lkl_sys_fchmodat
+ */
+static inline long lkl_sys_chmod(const char *path, mode_t mode)
+{
+	return lkl_sys_fchmodat(LKL_AT_FDCWD, path, mode);
+}
+#endif
+
+#ifdef __lkl__NR_linkat
+/**
+ * lkl_sys_link - wrapper for lkl_sys_linkat
+ */
+static inline long lkl_sys_link(const char *existing, const char *new)
+{
+	return lkl_sys_linkat(LKL_AT_FDCWD, existing, LKL_AT_FDCWD, new, 0);
+}
+#endif
+
+#ifdef __lkl__NR_unlinkat
+/**
+ * lkl_sys_unlink - wrapper for lkl_sys_unlinkat
+ */
+static inline long lkl_sys_unlink(const char *path)
+{
+	return lkl_sys_unlinkat(LKL_AT_FDCWD, path, 0);
+}
+#endif
+
+#ifdef __lkl__NR_symlinkat
+/**
+ * lkl_sys_symlink - wrapper for lkl_sys_symlinkat
+ */
+static inline long lkl_sys_symlink(const char *existing, const char *new)
+{
+	return lkl_sys_symlinkat(existing, LKL_AT_FDCWD, new);
+}
+#endif
+
+#ifdef __lkl__NR_readlinkat
+/**
+ * lkl_sys_readlink - wrapper for lkl_sys_readlinkat
+ */
+static inline long lkl_sys_readlink(const char *path, char *buf, size_t bufsize)
+{
+	return lkl_sys_readlinkat(LKL_AT_FDCWD, path, buf, bufsize);
+}
+#endif
+
+#ifdef __lkl__NR_renameat
+/**
+ * lkl_sys_rename - wrapper for lkl_sys_renameat
+ */
+static inline long lkl_sys_rename(const char *old, const char *new)
+{
+	return lkl_sys_renameat(LKL_AT_FDCWD, old, LKL_AT_FDCWD, new);
+}
+#endif
+
+#ifdef __lkl__NR_mkdirat
+/**
+ * lkl_sys_mkdir - wrapper for lkl_sys_mkdirat
+ */
+static inline long lkl_sys_mkdir(const char *path, mode_t mode)
+{
+	return lkl_sys_mkdirat(LKL_AT_FDCWD, path, mode);
+}
+#endif
+
+#ifdef __lkl__NR_unlinkat
+/**
+ * lkl_sys_rmdir - wrapper for lkl_sys_unlinkrat
+ */
+static inline long lkl_sys_rmdir(const char *path)
+{
+	return lkl_sys_unlinkat(LKL_AT_FDCWD, path, LKL_AT_REMOVEDIR);
+}
+#endif
+
+#ifdef __lkl__NR_mknodat
+/**
+ * lkl_sys_mknod - wrapper for lkl_sys_mknodat
+ */
+static inline long lkl_sys_mknod(const char *path, mode_t mode, dev_t dev)
+{
+	return lkl_sys_mknodat(LKL_AT_FDCWD, path, mode, dev);
+}
+#endif
+
+#ifdef __lkl__NR_pipe2
+/**
+ * lkl_sys_pipe - wrapper for lkl_sys_pipe2
+ */
+static inline long lkl_sys_pipe(int fd[2])
+{
+	return lkl_sys_pipe2(fd, 0);
+}
+#endif
+
+#ifdef __lkl__NR_sendto
+/**
+ * lkl_sys_send - wrapper for lkl_sys_sendto
+ */
+static inline long lkl_sys_send(int fd, void *buf, size_t len, int flags)
+{
+	return lkl_sys_sendto(fd, buf, len, flags, 0, 0);
+}
+#endif
+
+#ifdef __lkl__NR_recvfrom
+/**
+ * lkl_sys_recv - wrapper for lkl_sys_recvfrom
+ */
+static inline long lkl_sys_recv(int fd, void *buf, size_t len, int flags)
+{
+	return lkl_sys_recvfrom(fd, buf, len, flags, 0, 0);
+}
+#endif
+
+#ifdef __lkl__NR_pselect6
+/**
+ * lkl_sys_select - wrapper for lkl_sys_pselect
+ */
+static inline long lkl_sys_select(int n, lkl_fd_set *rfds, lkl_fd_set *wfds,
+				  lkl_fd_set *efds, struct lkl_timeval *tv)
+{
+	long data[2] = { 0, _LKL_NSIG/8 };
+	struct lkl_timespec ts;
+	lkl_time_t extra_secs;
+	const lkl_time_t max_time = ((1ULL<<8)*sizeof(time_t)-1)-1;
+
+	if (tv) {
+		if (tv->tv_sec < 0 || tv->tv_usec < 0)
+			return -LKL_EINVAL;
+
+		extra_secs = tv->tv_usec / 1000000;
+		ts.tv_nsec = tv->tv_usec % 1000000 * 1000;
+		ts.tv_sec = extra_secs > max_time - tv->tv_sec ?
+			max_time : tv->tv_sec + extra_secs;
+	}
+	return lkl_sys_pselect6(n, rfds, wfds, efds, tv ?
+				(struct __lkl__kernel_timespec *)&ts : 0, data);
+}
+#endif
+
+#ifdef __lkl__NR_ppoll
+/**
+ * lkl_sys_poll - wrapper for lkl_sys_ppoll
+ */
+static inline long lkl_sys_poll(struct lkl_pollfd *fds, int n, int timeout)
+{
+	return lkl_sys_ppoll(fds, n, timeout >= 0 ?
+			     (struct __lkl__kernel_timespec *)
+			     &((struct lkl_timespec){ .tv_sec = timeout/1000,
+				   .tv_nsec = timeout%1000*1000000 }) : 0,
+			     0, _LKL_NSIG/8);
+}
+#endif
+
+#ifdef __lkl__NR_epoll_create1
+/**
+ * lkl_sys_epoll_create - wrapper for lkl_sys_epoll_create1
+ */
+static inline long lkl_sys_epoll_create(int size)
+{
+	return lkl_sys_epoll_create1(0);
+}
+#endif
+
+#ifdef __lkl__NR_epoll_pwait
+/**
+ * lkl_sys_epoll_wait - wrapper for lkl_sys_epoll_pwait
+ */
+static inline long lkl_sys_epoll_wait(int fd, struct lkl_epoll_event *ev,
+				      int cnt, int to)
+{
+	return lkl_sys_epoll_pwait(fd, ev, cnt, to, 0, _LKL_NSIG/8);
+}
+#endif
+
+
+
+/**
+ * lkl_strerror - returns a string describing the given error code
+ *
+ * @err - error code
+ * @returns - string for the given error code
+ */
+const char *lkl_strerror(int err);
+
+/**
+ * lkl_perror - prints a string describing the given error code
+ *
+ * @msg - prefix for the error message
+ * @err - error code
+ */
+void lkl_perror(char *msg, int err);
+
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif
diff --git a/tools/lkl/include/lkl_host.h b/tools/lkl/include/lkl_host.h
new file mode 100644
index 000000000000..b5f96096fe69
--- /dev/null
+++ b/tools/lkl/include/lkl_host.h
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_HOST_H
+#define _LKL_HOST_H
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include <lkl/asm/host_ops.h>
+#include <lkl.h>
+
+extern struct lkl_host_operations lkl_host_ops;
+
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif
diff --git a/tools/lkl/lib/.gitignore b/tools/lkl/lib/.gitignore
new file mode 100644
index 000000000000..427ae0273fdd
--- /dev/null
+++ b/tools/lkl/lib/.gitignore
@@ -0,0 +1,3 @@
+lkl.o
+liblkl.a
+
diff --git a/tools/lkl/lib/Build b/tools/lkl/lib/Build
new file mode 100644
index 000000000000..8b137891791f
--- /dev/null
+++ b/tools/lkl/lib/Build
@@ -0,0 +1 @@
+
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 13/25] lkl tools: host lib: add utilities functions
  2020-03-30 14:45   ` Hajime Tazaki
@ 2020-03-30 14:45     ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Conrad Meyer, Patrick Collins, Yuan Liu, Motomu Utsumi,
	Hajime Tazaki

From: Octavian Purdila <tavi.purdila@gmail.com>

Add basic utility functions for getting a string from a kernel error
code and a fprintf like function that uses the host print
operation. The latter is useful for informing the user about errors
that occur in the host library.

Other configuration and debug utilities are also added.

Cc: Conrad Meyer <cem@FreeBSD.org>
Cc: Patrick Collins <pscollins@google.com>
Cc: Yuan Liu <liuyuan@google.com>
Cc: Motomu Utsumi <motomuman@gmail.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/lkl/include/lkl_config.h |  61 +++
 tools/lkl/include/lkl_host.h   |   7 +
 tools/lkl/lib/Build            |   7 +
 tools/lkl/lib/config.c         | 744 +++++++++++++++++++++++++++++++++
 tools/lkl/lib/dbg.c            | 300 +++++++++++++
 tools/lkl/lib/dbg_handler.c    |  44 ++
 tools/lkl/lib/endian.h         |  31 ++
 tools/lkl/lib/jmp_buf.c        |  14 +
 tools/lkl/lib/jmp_buf.h        |   8 +
 tools/lkl/lib/utils.c          | 266 ++++++++++++
 10 files changed, 1482 insertions(+)
 create mode 100644 tools/lkl/include/lkl_config.h
 create mode 100644 tools/lkl/lib/config.c
 create mode 100644 tools/lkl/lib/dbg.c
 create mode 100644 tools/lkl/lib/dbg_handler.c
 create mode 100644 tools/lkl/lib/endian.h
 create mode 100644 tools/lkl/lib/jmp_buf.c
 create mode 100644 tools/lkl/lib/jmp_buf.h
 create mode 100644 tools/lkl/lib/utils.c

diff --git a/tools/lkl/include/lkl_config.h b/tools/lkl/include/lkl_config.h
new file mode 100644
index 000000000000..d3edf8b414cf
--- /dev/null
+++ b/tools/lkl/include/lkl_config.h
@@ -0,0 +1,61 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_LIB_CONFIG_H
+#define _LKL_LIB_CONFIG_H
+
+#define LKL_CONFIG_JSON_TOKEN_MAX 300
+
+struct lkl_config_iface {
+	struct lkl_config_iface *next;
+	struct lkl_netdev *nd;
+
+	/* OBSOLETE: should use IFTYPE and IFPARAMS */
+	char *iftap;
+	char *iftype;
+	char *ifparams;
+	char *ifmtu_str;
+	char *ifip;
+	char *ifipv6;
+	char *ifgateway;
+	char *ifgateway6;
+	char *ifmac_str;
+	char *ifnetmask_len;
+	char *ifnetmask6_len;
+	char *ifoffload_str;
+	char *ifneigh_entries;
+	char *ifqdisc_entries;
+};
+
+struct lkl_config {
+	int ifnum;
+	struct lkl_config_iface *ifaces;
+
+	char *gateway;
+	char *gateway6;
+	char *debug;
+	char *mount;
+	/* single_cpu mode:
+	 * 0: Don't pin to single CPU (default).
+	 * 1: Pin only LKL kernel threads to single CPU.
+	 * 2: Pin all LKL threads to single CPU including all LKL kernel threads
+	 * and device polling threads. Avoid this mode if having busy polling
+	 * threads.
+	 *
+	 * mode 2 can achieve better TCP_RR but worse TCP_STREAM than mode 1.
+	 * You should choose the best for your application and virtio device
+	 * type.
+	 */
+	char *single_cpu;
+	char *sysctls;
+	char *boot_cmdline;
+	char *dump;
+	char *delay_main;
+};
+
+int lkl_load_config_json(struct lkl_config *cfg, char *jstr);
+int lkl_load_config_env(struct lkl_config *cfg);
+void lkl_show_config(struct lkl_config *cfg);
+int lkl_load_config_pre(struct lkl_config *cfg);
+int lkl_load_config_post(struct lkl_config *cfg);
+int lkl_unload_config(struct lkl_config *cfg);
+
+#endif /* _LKL_LIB_CONFIG_H */
diff --git a/tools/lkl/include/lkl_host.h b/tools/lkl/include/lkl_host.h
index b5f96096fe69..85e80eb4ad0d 100644
--- a/tools/lkl/include/lkl_host.h
+++ b/tools/lkl/include/lkl_host.h
@@ -11,6 +11,13 @@ extern "C" {
 
 extern struct lkl_host_operations lkl_host_ops;
 
+/**
+ * lkl_printf - print a message via the host print operation
+ *
+ * @fmt: printf like format string
+ */
+int lkl_printf(const char *fmt, ...);
+
 
 #ifdef __cplusplus
 }
diff --git a/tools/lkl/lib/Build b/tools/lkl/lib/Build
index 8b137891791f..658bfa865b9c 100644
--- a/tools/lkl/lib/Build
+++ b/tools/lkl/lib/Build
@@ -1 +1,8 @@
+CFLAGS_config.o += -I$(srctree)/tools/perf/pmu-events
 
+liblkl-y += jmp_buf.o
+liblkl-y += utils.o
+liblkl-y += dbg.o
+liblkl-y += dbg_handler.o
+liblkl-y += ../../perf/pmu-events/jsmn.o
+liblkl-y += config.o
diff --git a/tools/lkl/lib/config.c b/tools/lkl/lib/config.c
new file mode 100644
index 000000000000..37f8ac4d942a
--- /dev/null
+++ b/tools/lkl/lib/config.c
@@ -0,0 +1,744 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdlib.h>
+#define _HAVE_STRING_ARCH_strtok_r
+#include <string.h>
+#ifndef __MINGW32__
+#include <arpa/inet.h>
+#endif
+#include <lkl_host.h>
+#include <lkl_config.h>
+
+#include "jsmn.h"
+
+static int jsoneq(const char *json, jsmntok_t *tok, const char *s)
+{
+	if (tok->type == JSMN_STRING &&
+		(int) strlen(s) == tok->end - tok->start &&
+		strncmp(json + tok->start, s, tok->end - tok->start) == 0) {
+		return 0;
+	}
+	return -1;
+}
+
+static int cfgcpy(char **to, char *from)
+{
+	if (!from)
+		return 0;
+	if (*to)
+		free(*to);
+	*to = (char *)malloc((strlen(from) + 1) * sizeof(char));
+	if (*to == NULL) {
+		lkl_printf("malloc failed\n");
+		return -1;
+	}
+	strcpy(*to, from);
+	return 0;
+}
+
+static int cfgncpy(char **to, char *from, int len)
+{
+	if (!from)
+		return 0;
+	if (*to)
+		free(*to);
+	*to = (char *)malloc((len + 1) * sizeof(char));
+	if (*to == NULL) {
+		lkl_printf("malloc failed\n");
+		return -1;
+	}
+	strncpy(*to, from, len + 1);
+	(*to)[len] = '\0';
+	return 0;
+}
+
+static int parse_ifarr(struct lkl_config *cfg,
+		jsmntok_t *toks, char *jstr, int startpos)
+{
+	int ifidx, pos, posend, ret;
+	char **cfgptr;
+	struct lkl_config_iface *iface, *prev = NULL;
+
+	if (!cfg || !toks || !jstr)
+		return -1;
+	pos = startpos;
+	pos++;
+	if (toks[pos].type != JSMN_ARRAY) {
+		lkl_printf("unexpected json type, json array expected\n");
+		return -1;
+	}
+
+	cfg->ifnum = toks[pos].size;
+	pos++;
+	iface = cfg->ifaces;
+
+	for (ifidx = 0; ifidx < cfg->ifnum; ifidx++) {
+		if (toks[pos].type != JSMN_OBJECT) {
+			lkl_printf("object json type expected\n");
+			return -1;
+		}
+
+		posend = pos + toks[pos].size;
+		pos++;
+		iface = malloc(sizeof(struct lkl_config_iface));
+		memset(iface, 0, sizeof(struct lkl_config_iface));
+
+		if (prev)
+			prev->next = iface;
+		else
+			cfg->ifaces = iface;
+		prev = iface;
+
+		for (; pos < posend; pos += 2) {
+			if (toks[pos].type != JSMN_STRING) {
+				lkl_printf("object json type expected\n");
+				return -1;
+			}
+			if (jsoneq(jstr, &toks[pos], "type") == 0) {
+				cfgptr = &iface->iftype;
+			} else if (jsoneq(jstr, &toks[pos], "param") == 0) {
+				cfgptr = &iface->ifparams;
+			} else if (jsoneq(jstr, &toks[pos], "mtu") == 0) {
+				cfgptr = &iface->ifmtu_str;
+			} else if (jsoneq(jstr, &toks[pos], "ip") == 0) {
+				cfgptr = &iface->ifip;
+			} else if (jsoneq(jstr, &toks[pos], "ipv6") == 0) {
+				cfgptr = &iface->ifipv6;
+			} else if (jsoneq(jstr, &toks[pos], "ifgateway") == 0) {
+				cfgptr = &iface->ifgateway;
+			} else if (jsoneq(jstr, &toks[pos],
+							"ifgateway6") == 0) {
+				cfgptr = &iface->ifgateway6;
+			} else if (jsoneq(jstr, &toks[pos], "mac") == 0) {
+				cfgptr = &iface->ifmac_str;
+			} else if (jsoneq(jstr, &toks[pos], "masklen") == 0) {
+				cfgptr = &iface->ifnetmask_len;
+			} else if (jsoneq(jstr, &toks[pos], "masklen6") == 0) {
+				cfgptr = &iface->ifnetmask6_len;
+			} else if (jsoneq(jstr, &toks[pos], "neigh") == 0) {
+				cfgptr = &iface->ifneigh_entries;
+			} else if (jsoneq(jstr, &toks[pos], "qdisc") == 0) {
+				cfgptr = &iface->ifqdisc_entries;
+			} else if (jsoneq(jstr, &toks[pos], "offload") == 0) {
+				cfgptr = &iface->ifoffload_str;
+			} else {
+				lkl_printf("unexpected key: %.*s\n",
+						toks[pos].end-toks[pos].start,
+						jstr + toks[pos].start);
+				return -1;
+			}
+			ret = cfgncpy(cfgptr, jstr + toks[pos+1].start,
+					toks[pos+1].end-toks[pos+1].start);
+			if (ret < 0)
+				return ret;
+		}
+	}
+	return pos - startpos;
+}
+
+#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]))
+
+int lkl_load_config_json(struct lkl_config *cfg, char *jstr)
+{
+	int pos, ret;
+	char **cfgptr;
+	jsmn_parser jp;
+	jsmntok_t toks[LKL_CONFIG_JSON_TOKEN_MAX];
+
+	if (!cfg || !jstr)
+		return -1;
+	jsmn_init(&jp);
+	ret = jsmn_parse(&jp, jstr, strlen(jstr), toks, ARRAY_SIZE(toks));
+	if (ret != JSMN_SUCCESS) {
+		lkl_printf("failed to parse json\n");
+		return -1;
+	}
+	if (toks[0].type != JSMN_OBJECT) {
+		lkl_printf("object json type expected\n");
+		return -1;
+	}
+	for (pos = 1; pos < jp.toknext; pos++) {
+		if (toks[pos].type != JSMN_STRING) {
+			lkl_printf("string json type expected\n");
+			return -1;
+		}
+		if (jsoneq(jstr, &toks[pos], "interfaces") == 0) {
+			ret = parse_ifarr(cfg, toks, jstr, pos);
+			if (ret < 0)
+				return ret;
+			pos += ret;
+			pos--;
+			continue;
+		}
+		if (jsoneq(jstr, &toks[pos], "gateway") == 0) {
+			cfgptr = &cfg->gateway;
+		} else if (jsoneq(jstr, &toks[pos], "gateway6") == 0) {
+			cfgptr = &cfg->gateway6;
+		} else if (jsoneq(jstr, &toks[pos], "debug") == 0) {
+			cfgptr = &cfg->debug;
+		} else if (jsoneq(jstr, &toks[pos], "mount") == 0) {
+			cfgptr = &cfg->mount;
+		} else if (jsoneq(jstr, &toks[pos], "singlecpu") == 0) {
+			cfgptr = &cfg->single_cpu;
+		} else if (jsoneq(jstr, &toks[pos], "sysctl") == 0) {
+			cfgptr = &cfg->sysctls;
+		} else if (jsoneq(jstr, &toks[pos], "boot_cmdline") == 0) {
+			cfgptr = &cfg->boot_cmdline;
+		} else if (jsoneq(jstr, &toks[pos], "dump") == 0) {
+			cfgptr = &cfg->dump;
+		} else if (jsoneq(jstr, &toks[pos], "delay_main") == 0) {
+			cfgptr = &cfg->delay_main;
+		} else {
+			lkl_printf("unexpected key in json %.*s\n",
+					toks[pos].end-toks[pos].start,
+					jstr + toks[pos].start);
+			return -1;
+		}
+		pos++;
+		ret = cfgncpy(cfgptr, jstr + toks[pos].start,
+				toks[pos].end-toks[pos].start);
+		if (ret < 0)
+			return ret;
+	}
+	return 0;
+}
+
+void lkl_show_config(struct lkl_config *cfg)
+{
+	struct lkl_config_iface *iface;
+	int i = 0;
+
+	if (!cfg)
+		return;
+	lkl_printf("gateway: %s\n", cfg->gateway);
+	lkl_printf("gateway6: %s\n", cfg->gateway6);
+	lkl_printf("debug: %s\n", cfg->debug);
+	lkl_printf("mount: %s\n", cfg->mount);
+	lkl_printf("singlecpu: %s\n", cfg->single_cpu);
+	lkl_printf("sysctl: %s\n", cfg->sysctls);
+	lkl_printf("cmdline: %s\n", cfg->boot_cmdline);
+	lkl_printf("dump: %s\n", cfg->dump);
+	lkl_printf("delay: %s\n", cfg->delay_main);
+
+	for (iface = cfg->ifaces; iface; iface = iface->next, i++) {
+		lkl_printf("ifmac[%d] = %s\n", i, iface->ifmac_str);
+		lkl_printf("ifmtu[%d] = %s\n", i, iface->ifmtu_str);
+		lkl_printf("iftype[%d] = %s\n", i, iface->iftype);
+		lkl_printf("ifparam[%d] = %s\n", i, iface->ifparams);
+		lkl_printf("ifip[%d] = %s\n", i, iface->ifip);
+		lkl_printf("ifmasklen[%d] = %s\n", i, iface->ifnetmask_len);
+		lkl_printf("ifgateway[%d] = %s\n", i, iface->ifgateway);
+		lkl_printf("ifip6[%d] = %s\n", i, iface->ifipv6);
+		lkl_printf("ifmasklen6[%d] = %s\n", i, iface->ifnetmask6_len);
+		lkl_printf("ifgateway6[%d] = %s\n", i, iface->ifgateway6);
+		lkl_printf("ifoffload[%d] = %s\n", i, iface->ifoffload_str);
+		lkl_printf("ifneigh[%d] = %s\n", i, iface->ifneigh_entries);
+		lkl_printf("ifqdisk[%d] = %s\n", i, iface->ifqdisc_entries);
+	}
+}
+
+int lkl_load_config_env(struct lkl_config *cfg)
+{
+	int ret;
+	char *enviftype = getenv("LKL_HIJACK_NET_IFTYPE");
+	char *envifparams = getenv("LKL_HIJACK_NET_IFPARAMS");
+	char *envmtu_str = getenv("LKL_HIJACK_NET_MTU");
+	char *envip = getenv("LKL_HIJACK_NET_IP");
+	char *envipv6 = getenv("LKL_HIJACK_NET_IPV6");
+	char *envifgateway = getenv("LKL_HIJACK_NET_IFGATEWAY");
+	char *envifgateway6 = getenv("LKL_HIJACK_NET_IFGATEWAY6");
+	char *envmac_str = getenv("LKL_HIJACK_NET_MAC");
+	char *envnetmask_len = getenv("LKL_HIJACK_NET_NETMASK_LEN");
+	char *envnetmask6_len = getenv("LKL_HIJACK_NET_NETMASK6_LEN");
+	char *envgateway = getenv("LKL_HIJACK_NET_GATEWAY");
+	char *envgateway6 = getenv("LKL_HIJACK_NET_GATEWAY6");
+	char *envdebug = getenv("LKL_HIJACK_DEBUG");
+	char *envmount = getenv("LKL_HIJACK_MOUNT");
+	char *envneigh_entries = getenv("LKL_HIJACK_NET_NEIGHBOR");
+	char *envqdisc_entries = getenv("LKL_HIJACK_NET_QDISC");
+	char *envsingle_cpu = getenv("LKL_HIJACK_SINGLE_CPU");
+	char *envoffload_str = getenv("LKL_HIJACK_OFFLOAD");
+	char *envsysctls = getenv("LKL_HIJACK_SYSCTL");
+	char *envboot_cmdline = getenv("LKL_HIJACK_BOOT_CMDLINE") ? : "";
+	char *envdump = getenv("LKL_HIJACK_DUMP");
+	struct lkl_config_iface *iface;
+
+	if (!cfg)
+		return -1;
+	if (enviftype)
+		cfg->ifnum = 1;
+
+	iface = malloc(sizeof(struct lkl_config_iface));
+	memset(iface, 0, sizeof(struct lkl_config_iface));
+
+	ret = cfgcpy(&iface->iftype, enviftype);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifparams, envifparams);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifmtu_str, envmtu_str);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifip, envip);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifipv6, envipv6);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifgateway, envifgateway);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifgateway6, envifgateway6);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifmac_str, envmac_str);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifnetmask_len, envnetmask_len);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifnetmask6_len, envnetmask6_len);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifoffload_str, envoffload_str);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifneigh_entries, envneigh_entries);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifqdisc_entries, envqdisc_entries);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&cfg->gateway, envgateway);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&cfg->gateway6, envgateway6);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&cfg->debug, envdebug);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&cfg->mount, envmount);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&cfg->single_cpu, envsingle_cpu);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&cfg->sysctls, envsysctls);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&cfg->boot_cmdline, envboot_cmdline);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&cfg->dump, envdump);
+	if (ret < 0)
+		return ret;
+	return 0;
+}
+
+static int parse_mac_str(char *mac_str, __lkl__u8 mac[LKL_ETH_ALEN])
+{
+	char delim[] = ":";
+	char *saveptr = NULL, *token = NULL;
+	int i = 0;
+
+	if (!mac_str)
+		return 0;
+
+	for (token = strtok_r(mac_str, delim, &saveptr);
+	     i < LKL_ETH_ALEN; i++) {
+		if (!token) {
+			/* The address is too short */
+			return -1;
+		}
+
+		mac[i] = (__lkl__u8) strtol(token, NULL, 16);
+		token = strtok_r(NULL, delim, &saveptr);
+	}
+
+	if (strtok_r(NULL, delim, &saveptr)) {
+		/* The address is too long */
+		return -1;
+	}
+
+	return 1;
+}
+
+/* Add permanent neighbor entries in the form of "ip|mac;ip|mac;..." */
+static void add_neighbor(int ifindex, char *entries)
+{
+	char *saveptr = NULL, *token = NULL;
+	char *ip = NULL, *mac_str = NULL;
+	int ret = 0;
+	__lkl__u8 mac[LKL_ETH_ALEN];
+	char ip_addr[16];
+	int af;
+
+	for (token = strtok_r(entries, ";", &saveptr); token;
+	     token = strtok_r(NULL, ";", &saveptr)) {
+		ip = strtok(token, "|");
+		mac_str = strtok(NULL, "|");
+		if (ip == NULL || mac_str == NULL || strtok(NULL, "|") != NULL)
+			return;
+
+		af = LKL_AF_INET;
+		ret = inet_pton(LKL_AF_INET, ip, ip_addr);
+		if (ret == 0) {
+			ret = inet_pton(LKL_AF_INET6, ip, ip_addr);
+			af = LKL_AF_INET6;
+		}
+		if (ret != 1) {
+			lkl_printf("Bad ip address: %s\n", ip);
+			return;
+		}
+
+		ret = parse_mac_str(mac_str, mac);
+		if (ret != 1) {
+			lkl_printf("Failed to parse mac: %s\n", mac_str);
+			return;
+		}
+		ret = lkl_add_neighbor(ifindex, af, ip_addr, mac);
+		if (ret) {
+			lkl_printf("Failed to add neighbor entry: %s\n",
+				   lkl_strerror(ret));
+			return;
+		}
+	}
+}
+
+/* We don't have an easy way to make FILE*s out of our fds, so we
+ * can't use e.g. fgets
+ */
+static int dump_file(char *path)
+{
+	int ret = -1, bytes_read = 0;
+	char str[1024] = { 0 };
+	int fd;
+
+	fd = lkl_sys_open(path, LKL_O_RDONLY, 0);
+
+	if (fd < 0) {
+		lkl_printf("%s lkl_sys_open %s: %s\n",
+			   __func__, path, lkl_strerror(fd));
+		return -1;
+	}
+
+	/* Need to print this out in order to make sense of the output */
+	lkl_printf("Reading from %s:\n==========\n", path);
+	while ((ret = lkl_sys_read(fd, str, sizeof(str) - 1)) > 0)
+		bytes_read += lkl_printf("%s", str);
+	lkl_printf("==========\n");
+
+	if (ret) {
+		lkl_printf("%s lkl_sys_read %s: %s\n",
+			   __func__, path, lkl_strerror(ret));
+		return -1;
+	}
+
+	return 0;
+}
+
+static void mount_cmds_exec(char *_cmds, int (*callback)(char *))
+{
+	char *saveptr = NULL, *token;
+	int ret = 0;
+	char *cmds = strdup(_cmds);
+
+	token = strtok_r(cmds, ",", &saveptr);
+
+	while (token && ret >= 0) {
+		ret = callback(token);
+		token = strtok_r(NULL, ",", &saveptr);
+	}
+
+	if (ret < 0)
+		lkl_printf("%s: failed parsing %s\n", __func__, _cmds);
+
+	free(cmds);
+}
+
+static int lkl_config_netdev_create(struct lkl_config *cfg,
+				    struct lkl_config_iface *iface)
+{
+	int ret, offload = 0;
+	struct lkl_netdev_args nd_args;
+	__lkl__u8 mac[LKL_ETH_ALEN] = {0};
+	struct lkl_netdev *nd = NULL;
+
+	if (iface->ifoffload_str)
+		offload = strtol(iface->ifoffload_str, NULL, 0);
+	memset(&nd_args, 0, sizeof(struct lkl_netdev_args));
+
+	if (!nd && iface->iftype && iface->ifparams) {
+	}
+
+	if (nd) {
+		if ((mac[0] != 0) || (mac[1] != 0) ||
+				(mac[2] != 0) || (mac[3] != 0) ||
+				(mac[4] != 0) || (mac[5] != 0)) {
+			nd_args.mac = mac;
+		} else {
+			ret = parse_mac_str(iface->ifmac_str, mac);
+
+			if (ret < 0) {
+				lkl_printf("failed to parse mac\n");
+				return -1;
+			} else if (ret > 0) {
+				nd_args.mac = mac;
+			} else {
+				nd_args.mac = NULL;
+			}
+		}
+
+		nd_args.offload = offload;
+		iface->nd = nd;
+	}
+	return 0;
+}
+
+static int lkl_config_netdev_configure(struct lkl_config *cfg,
+				       struct lkl_config_iface *iface)
+{
+	int ret, nd_ifindex = -1;
+	struct lkl_netdev *nd = iface->nd;
+
+	if (!nd) {
+		lkl_printf("no netdev available %s\n", iface ? iface->ifparams
+			   : "(null)");
+		return -1;
+	}
+
+	if (nd->id >= 0) {
+		nd_ifindex = lkl_netdev_get_ifindex(nd->id);
+		if (nd_ifindex > 0)
+			lkl_if_up(nd_ifindex);
+		else
+			lkl_printf(
+				"failed to get ifindex for netdev id %d: %s\n",
+				nd->id, lkl_strerror(nd_ifindex));
+	}
+
+	if (nd_ifindex >= 0 && iface->ifmtu_str) {
+		int mtu = atoi(iface->ifmtu_str);
+
+		ret = lkl_if_set_mtu(nd_ifindex, mtu);
+		if (ret < 0)
+			lkl_printf("failed to set MTU: %s\n",
+				   lkl_strerror(ret));
+	}
+
+	if (nd_ifindex >= 0 && iface->ifip && iface->ifnetmask_len) {
+		unsigned int addr;
+
+		if (inet_pton(LKL_AF_INET, iface->ifip,
+			      (struct lkl_in_addr *)&addr) != 1)
+			lkl_printf("Invalid ipv4 address: %s\n", iface->ifip);
+
+		int nmlen = atoi(iface->ifnetmask_len);
+
+		if (addr != LKL_INADDR_NONE && nmlen > 0 && nmlen < 32) {
+			ret = lkl_if_set_ipv4(nd_ifindex, addr, nmlen);
+			if (ret < 0)
+				lkl_printf("failed to set IPv4 address: %s\n",
+					   lkl_strerror(ret));
+		}
+		if (iface->ifgateway) {
+			unsigned int gwaddr;
+
+			if (inet_pton(LKL_AF_INET, iface->ifgateway,
+				      (struct lkl_in_addr *)&gwaddr) != 1)
+				lkl_printf("Invalid ipv4 gateway: %s\n",
+					   iface->ifgateway);
+
+			if (gwaddr != LKL_INADDR_NONE) {
+				ret = lkl_if_set_ipv4_gateway(nd_ifindex,
+						addr, nmlen, gwaddr);
+				if (ret < 0)
+					lkl_printf(
+						"failed to set v4 if gw: %s\n",
+						lkl_strerror(ret));
+			}
+		}
+	}
+
+	if (nd_ifindex >= 0 && iface->ifipv6 &&
+			iface->ifnetmask6_len) {
+		struct lkl_in6_addr addr;
+		unsigned int pflen = atoi(iface->ifnetmask6_len);
+
+		if (inet_pton(LKL_AF_INET6, iface->ifipv6,
+			      (struct lkl_in6_addr *)&addr) != 1) {
+			lkl_printf("Invalid ipv6 addr: %s\n",
+				   iface->ifipv6);
+		}  else {
+			ret = lkl_if_set_ipv6(nd_ifindex, &addr, pflen);
+			if (ret < 0)
+				lkl_printf("failed to set IPv6 address: %s\n",
+					   lkl_strerror(ret));
+		}
+		if (iface->ifgateway6) {
+			char gwaddr[16];
+
+			if (inet_pton(LKL_AF_INET6, iface->ifgateway6,
+								gwaddr) != 1) {
+				lkl_printf("Invalid ipv6 gateway: %s\n",
+					   iface->ifgateway6);
+			} else {
+				ret = lkl_if_set_ipv6_gateway(nd_ifindex,
+						&addr, pflen, gwaddr);
+				if (ret < 0)
+					lkl_printf(
+						"failed to set v6 if gw: %s\n",
+						lkl_strerror(ret));
+			}
+		}
+	}
+
+	if (nd_ifindex >= 0 && iface->ifneigh_entries)
+		add_neighbor(nd_ifindex, iface->ifneigh_entries);
+
+	if (nd_ifindex >= 0 && iface->ifqdisc_entries)
+		lkl_qdisc_parse_add(nd_ifindex, iface->ifqdisc_entries);
+
+	return 0;
+}
+
+static void free_cfgparam(char *cfgparam)
+{
+	if (cfgparam)
+		free(cfgparam);
+}
+
+static int lkl_clean_config(struct lkl_config *cfg)
+{
+	struct lkl_config_iface *iface;
+
+	if (!cfg)
+		return -1;
+
+	for (iface = cfg->ifaces; iface; iface = iface->next) {
+		free_cfgparam(iface->iftype);
+		free_cfgparam(iface->ifparams);
+		free_cfgparam(iface->ifmtu_str);
+		free_cfgparam(iface->ifip);
+		free_cfgparam(iface->ifipv6);
+		free_cfgparam(iface->ifgateway);
+		free_cfgparam(iface->ifgateway6);
+		free_cfgparam(iface->ifmac_str);
+		free_cfgparam(iface->ifnetmask_len);
+		free_cfgparam(iface->ifnetmask6_len);
+		free_cfgparam(iface->ifoffload_str);
+		free_cfgparam(iface->ifneigh_entries);
+		free_cfgparam(iface->ifqdisc_entries);
+	}
+	free_cfgparam(cfg->gateway);
+	free_cfgparam(cfg->gateway6);
+	free_cfgparam(cfg->debug);
+	free_cfgparam(cfg->mount);
+	free_cfgparam(cfg->single_cpu);
+	free_cfgparam(cfg->sysctls);
+	free_cfgparam(cfg->boot_cmdline);
+	free_cfgparam(cfg->dump);
+	free_cfgparam(cfg->delay_main);
+	return 0;
+}
+
+
+int lkl_load_config_pre(struct lkl_config *cfg)
+{
+	int lkl_debug, ret;
+	struct lkl_config_iface *iface;
+
+	if (!cfg)
+		return 0;
+
+	if (cfg->debug)
+		lkl_debug = strtol(cfg->debug, NULL, 0);
+
+	if (!cfg->debug || (lkl_debug == 0))
+		lkl_host_ops.print = NULL;
+
+	for (iface = cfg->ifaces; iface; iface = iface->next) {
+		ret = lkl_config_netdev_create(cfg, iface);
+		if (ret < 0)
+			return -1;
+	}
+
+	return 0;
+}
+
+int lkl_load_config_post(struct lkl_config *cfg)
+{
+	int ret;
+	struct lkl_config_iface *iface;
+
+	if (!cfg)
+		return 0;
+
+	if (cfg->mount)
+		mount_cmds_exec(cfg->mount, lkl_mount_fs);
+
+	for (iface = cfg->ifaces; iface; iface = iface->next) {
+		ret = lkl_config_netdev_configure(cfg, iface);
+		if (ret < 0)
+			break;
+	}
+
+	if (cfg->gateway) {
+		unsigned int gwaddr;
+
+		if (inet_pton(LKL_AF_INET, cfg->gateway,
+			      (struct lkl_in_addr *)&gwaddr) != 1)
+			lkl_printf("Invalid ipv4 gateway: %s\n", cfg->gateway);
+
+		if (gwaddr != LKL_INADDR_NONE) {
+			ret = lkl_set_ipv4_gateway(gwaddr);
+			if (ret < 0)
+				lkl_printf("failed to set IPv4 gateway: %s\n",
+					   lkl_strerror(ret));
+		}
+	}
+
+	if (cfg->gateway6) {
+		char gw[16];
+
+		if (inet_pton(LKL_AF_INET6, cfg->gateway6, gw) != 1) {
+			lkl_printf("Invalid ipv6 gateway: %s\n", cfg->gateway6);
+		} else {
+			ret = lkl_set_ipv6_gateway(gw);
+			if (ret < 0)
+				lkl_printf("failed to set IPv6 gateway: %s\n",
+					   lkl_strerror(ret));
+		}
+	}
+
+	if (cfg->sysctls)
+		lkl_sysctl_parse_write(cfg->sysctls);
+
+	/* put a delay before calling main() */
+	if (cfg->delay_main) {
+		unsigned long delay = strtoul(cfg->delay_main, NULL, 10);
+
+		if (delay == ~0UL)
+			lkl_printf("got invalid delay_main value (%s)\n",
+				   cfg->delay_main);
+		else {
+			lkl_printf("sleeping %lu usec\n", delay);
+			usleep(delay);
+		}
+	}
+
+	return 0;
+}
+
+int lkl_unload_config(struct lkl_config *cfg)
+{
+	if (cfg) {
+		if (cfg->dump)
+			mount_cmds_exec(cfg->dump, dump_file);
+
+		lkl_clean_config(cfg);
+	}
+
+	return 0;
+}
diff --git a/tools/lkl/lib/dbg.c b/tools/lkl/lib/dbg.c
new file mode 100644
index 000000000000..b613353bce5c
--- /dev/null
+++ b/tools/lkl/lib/dbg.c
@@ -0,0 +1,300 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <errno.h>
+#include <lkl.h>
+#include <limits.h>
+#include <string.h>
+#include <stdio.h>
+#include <stdlib.h>
+
+static const char *PROMOTE = "$";
+#define str(x) #x
+#define xstr(s) str(s)
+#define MAX_BUF 100
+static char cmd[MAX_BUF];
+static char argv[10][MAX_BUF];
+static int argc;
+static char cur_dir[MAX_BUF] = "/";
+
+static char *normalize_path(const char *src, size_t src_len)
+{
+	char *res;
+	unsigned int res_len;
+	const char *ptr = src;
+	const char *end = &src[src_len];
+	const char *next;
+
+	res = malloc((src_len > 0 ? src_len : 1) + 1);
+	res_len = 0;
+
+	for (ptr = src; ptr < end; ptr = next+1) {
+		size_t len;
+
+		next = memchr(ptr, '/', end-ptr);
+		if (next == NULL)
+			next = end;
+
+		len = next-ptr;
+		switch (len) {
+		case 2:
+			if (ptr[0] == '.' && ptr[1] == '.') {
+				const char *slash = strrchr(res, '/');
+
+				if (slash != NULL)
+					res_len = slash - res;
+				continue;
+			}
+			break;
+		case 1:
+			if (ptr[0] == '.')
+				continue;
+			break;
+		case 0:
+			continue;
+		}
+		res[res_len++] = '/';
+		memcpy(&res[res_len], ptr, len);
+		res_len += len;
+	}
+	if (res_len == 0)
+		res[res_len++] = '/';
+	res[res_len] = '\0';
+	return res;
+}
+
+static void build_path(char *path)
+{
+	char *npath;
+
+	strcpy(path, cur_dir);
+	if (argc >= 1) {
+		if (argv[0][0] == '/')
+			strncpy(path, argv[0], LKL_PATH_MAX);
+		else {
+			strncat(path, "/", LKL_PATH_MAX - strlen(path) - 1);
+			strncat(path, argv[0], LKL_PATH_MAX - strlen(path) - 1);
+		}
+	}
+	npath = normalize_path(path, strlen(path));
+	strcpy(path, npath);
+	free(npath);
+}
+
+static void help(void)
+{
+	const char *msg =
+		"cat FILE\n"
+		"\tShow content of FILE\n"
+		"cd [DIR]\n"
+		"\tChange directory to DIR\n"
+		"exit\n"
+		"\tExit the debug session\n"
+		"help\n"
+		"\tShow this message\n"
+		"ls [DIR]\n"
+		"\tList files in DIR\n"
+		"mount FSTYPE\n"
+		"\tMount FSTYPE as /FSTYPE\n"
+		"overwrite FILE\n"
+		"\tOverwrite content of FILE from stdin\n"
+		"pwd\n"
+		"\tShow current directory\n"
+		;
+	printf("%s", msg);
+}
+
+static void ls(void)
+{
+	char path[LKL_PATH_MAX];
+	struct lkl_dir *dir;
+	struct lkl_linux_dirent64 *de;
+	int err;
+
+	build_path(path);
+	dir = lkl_opendir(path, &err);
+	if (dir) {
+		do {
+			de = lkl_readdir(dir);
+			if (de) {
+				printf("%s\n", de->d_name);
+			} else {
+				err = lkl_errdir(dir);
+				if (err != 0) {
+					fprintf(stderr, "%s\n",
+						lkl_strerror(err));
+				}
+				break;
+			}
+		} while (1);
+		lkl_closedir(dir);
+	} else {
+		fprintf(stderr, "%s: %s\n", path, lkl_strerror(err));
+	}
+}
+
+static void cd(void)
+{
+	char path[LKL_PATH_MAX];
+	struct lkl_dir *dir;
+	int err;
+
+	build_path(path);
+	dir = lkl_opendir(path, &err);
+	if (dir) {
+		strcpy(cur_dir, path);
+		lkl_closedir(dir);
+	} else {
+		fprintf(stderr, "%s: %s\n", path, lkl_strerror(err));
+	}
+}
+
+static void mount(void)
+{
+	char *fstype;
+	int ret = 0;
+
+	if (argc != 1) {
+		fprintf(stderr, "%s\n", "One argument is needed.");
+		return;
+	}
+
+	fstype = argv[0];
+	ret = lkl_mount_fs(fstype);
+	if (ret == 1)
+		fprintf(stderr, "%s is already mounted.\n", fstype);
+}
+
+static void cat(void)
+{
+	char path[LKL_PATH_MAX];
+	int ret;
+	char buf[1024];
+	int fd;
+
+	if (argc != 1) {
+		fprintf(stderr, "%s\n", "One argument is needed.");
+		return;
+	}
+
+	build_path(path);
+	fd = lkl_sys_open(path, LKL_O_RDONLY, 0);
+
+	if (fd < 0) {
+		fprintf(stderr, "lkl_sys_open %s: %s\n",
+			path, lkl_strerror(fd));
+		return;
+	}
+
+	while ((ret = lkl_sys_read(fd, buf, sizeof(buf) - 1)) > 0) {
+		buf[ret] = '\0';
+		printf("%s", buf);
+	}
+
+	if (ret) {
+		fprintf(stderr, "lkl_sys_read %s: %s\n",
+			path, lkl_strerror(ret));
+	}
+	lkl_sys_close(fd);
+}
+
+static void overwrite(void)
+{
+	char path[LKL_PATH_MAX];
+	int ret;
+	int fd;
+	char buf[1024];
+
+	build_path(path);
+	fd = lkl_sys_open(path, LKL_O_WRONLY | LKL_O_CREAT, 0);
+	if (fd < 0) {
+		fprintf(stderr, "lkl_sys_open %s: %s\n",
+			path, lkl_strerror(fd));
+		return;
+	}
+	printf("Input the content and stop by hitting Ctrl-D:\n");
+	while (fgets(buf, 1023, stdin)) {
+		ret = lkl_sys_write(fd, buf, strlen(buf));
+		if (ret < 0) {
+			fprintf(stderr, "lkl_sys_write %s: %s\n",
+				path, lkl_strerror(fd));
+		}
+	}
+	lkl_sys_close(fd);
+}
+
+static void pwd(void)
+{
+	printf("%s\n", cur_dir);
+}
+
+static int parse_cmd(char *input)
+{
+	char *token;
+
+	token = strtok(input, " ");
+	if (token)
+		strcpy(cmd, token);
+	else
+		return -1;
+
+	argc = 0;
+	token = strtok(NULL, " ");
+	while (token) {
+		if (argc >= 10) {
+			fprintf(stderr, "To many args > 10\n");
+			return -1;
+		}
+		strcpy(argv[argc++], token);
+		token = strtok(NULL, " ");
+	}
+	return 0;
+}
+
+static void run_cmd(void)
+{
+	if (strcmp(cmd, "cat") == 0)
+		cat();
+	else if (strcmp(cmd, "cd") == 0)
+		cd();
+	else if (strcmp(cmd, "help") == 0)
+		help();
+	else if (strcmp(cmd, "ls") == 0)
+		ls();
+	else if (strcmp(cmd, "mount") == 0)
+		mount();
+	else if (strcmp(cmd, "overwrite") == 0)
+		overwrite();
+	else if (strcmp(cmd, "pwd") == 0)
+		pwd();
+	else
+		fprintf(stderr, "Unknown command: %s\n", cmd);
+}
+
+void dbg_entrance(void)
+{
+	char input[MAX_BUF + 1];
+	int ret;
+	int c;
+
+	printf("Type help to see a list of commands\n");
+	do {
+		printf("%s ", PROMOTE);
+		ret = scanf("%" xstr(MAX_BUF) "[^\n]s", input);
+		while ((c = getchar()) != '\n' && c != EOF)
+			;
+		if (ret == 0)
+			continue;
+		if (ret != 1 && errno != EINTR) {
+			perror("scanf");
+			continue;
+		}
+		if (strlen(input) == MAX_BUF) {
+			fprintf(stderr, "Too long input > %d\n", MAX_BUF - 1);
+			continue;
+		}
+		if (parse_cmd(input))
+			continue;
+		if (strcmp(cmd, "exit") == 0)
+			break;
+		run_cmd();
+	} while (1);
+}
diff --git a/tools/lkl/lib/dbg_handler.c b/tools/lkl/lib/dbg_handler.c
new file mode 100644
index 000000000000..01d165a5fc1e
--- /dev/null
+++ b/tools/lkl/lib/dbg_handler.c
@@ -0,0 +1,44 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <lkl_host.h>
+
+extern void dbg_entrance(void);
+static int dbg_running;
+
+static void dbg_thread(void *arg)
+{
+	lkl_host_ops.thread_detach();
+	printf("======Enter Debug======\n");
+	dbg_entrance();
+	printf("======Exit Debug======\n");
+	dbg_running = 0;
+}
+
+void dbg_handler(int signum)
+{
+	/* We don't care about the possible race on dbg_running. */
+	if (dbg_running) {
+		fprintf(stderr, "A debug lib is running\n");
+		return;
+	}
+	dbg_running = 1;
+	lkl_host_ops.thread_create(&dbg_thread, NULL);
+}
+
+#ifndef __MINGW32__
+#include <signal.h>
+void lkl_register_dbg_handler(void)
+{
+	struct sigaction sa;
+
+	sigemptyset(&sa.sa_mask);
+	sa.sa_handler = dbg_handler;
+	if (sigaction(SIGTSTP, &sa, NULL) == -1)
+		perror("sigaction");
+}
+#else
+void lkl_register_dbg_handler(void)
+{
+	fprintf(stderr, "%s is not implemented.\n", __func__);
+}
+#endif
diff --git a/tools/lkl/lib/endian.h b/tools/lkl/lib/endian.h
new file mode 100644
index 000000000000..aaccfa0edb65
--- /dev/null
+++ b/tools/lkl/lib/endian.h
@@ -0,0 +1,31 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_LIB_ENDIAN_H
+#define _LKL_LIB_ENDIAN_H
+
+#if defined(__FreeBSD__)
+#include <sys/endian.h>
+#elif defined(__ANDROID__)
+#include <sys/endian.h>
+#elif defined(__MINGW32__)
+#include <winsock.h>
+#define le32toh(x) (x)
+#define le16toh(x) (x)
+#define htole32(x) (x)
+#define htole16(x) (x)
+#define le64toh(x) (x)
+#define htobe32(x) htonl(x)
+#define htobe16(x) htons(x)
+#define be32toh(x) ntohl(x)
+#define be16toh(x) ntohs(x)
+#else
+#include <endian.h>
+#endif
+
+#ifndef htonl
+#define htonl(x) htobe32(x)
+#define htons(x) htobe16(x)
+#define ntohl(x) be32toh(x)
+#define ntohs(x) be16toh(x)
+#endif
+
+#endif /* _LKL_LIB_ENDIAN_H */
diff --git a/tools/lkl/lib/jmp_buf.c b/tools/lkl/lib/jmp_buf.c
new file mode 100644
index 000000000000..f6bdd7e4bd83
--- /dev/null
+++ b/tools/lkl/lib/jmp_buf.c
@@ -0,0 +1,14 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <setjmp.h>
+#include <lkl_host.h>
+
+void jmp_buf_set(struct lkl_jmp_buf *jmpb, void (*f)(void))
+{
+	if (!setjmp(*((jmp_buf *)jmpb->buf)))
+		f();
+}
+
+void jmp_buf_longjmp(struct lkl_jmp_buf *jmpb, int val)
+{
+	longjmp(*((jmp_buf *)jmpb->buf), val);
+}
diff --git a/tools/lkl/lib/jmp_buf.h b/tools/lkl/lib/jmp_buf.h
new file mode 100644
index 000000000000..8782cbaaf51f
--- /dev/null
+++ b/tools/lkl/lib/jmp_buf.h
@@ -0,0 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_LIB_JMP_BUF_H
+#define _LKL_LIB_JMP_BUF_H
+
+void jmp_buf_set(struct lkl_jmp_buf *jmpb, void (*f)(void));
+void jmp_buf_longjmp(struct lkl_jmp_buf *jmpb, int val);
+
+#endif
diff --git a/tools/lkl/lib/utils.c b/tools/lkl/lib/utils.c
new file mode 100644
index 000000000000..7de92bbe5475
--- /dev/null
+++ b/tools/lkl/lib/utils.c
@@ -0,0 +1,266 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdarg.h>
+#include <stdio.h>
+#include <string.h>
+#include <lkl_host.h>
+
+static const char * const lkl_err_strings[] = {
+	"Success",
+	"Operation not permitted",
+	"No such file or directory",
+	"No such process",
+	"Interrupted system call",
+	"I/O error",
+	"No such device or address",
+	"Argument list too long",
+	"Exec format error",
+	"Bad file number",
+	"No child processes",
+	"Try again",
+	"Out of memory",
+	"Permission denied",
+	"Bad address",
+	"Block device required",
+	"Device or resource busy",
+	"File exists",
+	"Cross-device link",
+	"No such device",
+	"Not a directory",
+	"Is a directory",
+	"Invalid argument",
+	"File table overflow",
+	"Too many open files",
+	"Not a typewriter",
+	"Text file busy",
+	"File too large",
+	"No space left on device",
+	"Illegal seek",
+	"Read-only file system",
+	"Too many links",
+	"Broken pipe",
+	"Math argument out of domain of func",
+	"Math result not representable",
+	"Resource deadlock would occur",
+	"File name too long",
+	"No record locks available",
+	"Invalid system call number",
+	"Directory not empty",
+	"Too many symbolic links encountered",
+	"Bad error code", /* EWOULDBLOCK is EAGAIN */
+	"No message of desired type",
+	"Identifier removed",
+	"Channel number out of range",
+	"Level 2 not synchronized",
+	"Level 3 halted",
+	"Level 3 reset",
+	"Link number out of range",
+	"Protocol driver not attached",
+	"No CSI structure available",
+	"Level 2 halted",
+	"Invalid exchange",
+	"Invalid request descriptor",
+	"Exchange full",
+	"No anode",
+	"Invalid request code",
+	"Invalid slot",
+	"Bad error code", /* EDEADLOCK is EDEADLK */
+	"Bad font file format",
+	"Device not a stream",
+	"No data available",
+	"Timer expired",
+	"Out of streams resources",
+	"Machine is not on the network",
+	"Package not installed",
+	"Object is remote",
+	"Link has been severed",
+	"Advertise error",
+	"Srmount error",
+	"Communication error on send",
+	"Protocol error",
+	"Multihop attempted",
+	"RFS specific error",
+	"Not a data message",
+	"Value too large for defined data type",
+	"Name not unique on network",
+	"File descriptor in bad state",
+	"Remote address changed",
+	"Can not access a needed shared library",
+	"Accessing a corrupted shared library",
+	".lib section in a.out corrupted",
+	"Attempting to link in too many shared libraries",
+	"Cannot exec a shared library directly",
+	"Illegal byte sequence",
+	"Interrupted system call should be restarted",
+	"Streams pipe error",
+	"Too many users",
+	"Socket operation on non-socket",
+	"Destination address required",
+	"Message too long",
+	"Protocol wrong type for socket",
+	"Protocol not available",
+	"Protocol not supported",
+	"Socket type not supported",
+	"Operation not supported on transport endpoint",
+	"Protocol family not supported",
+	"Address family not supported by protocol",
+	"Address already in use",
+	"Cannot assign requested address",
+	"Network is down",
+	"Network is unreachable",
+	"Network dropped connection because of reset",
+	"Software caused connection abort",
+	"Connection reset by peer",
+	"No buffer space available",
+	"Transport endpoint is already connected",
+	"Transport endpoint is not connected",
+	"Cannot send after transport endpoint shutdown",
+	"Too many references: cannot splice",
+	"Connection timed out",
+	"Connection refused",
+	"Host is down",
+	"No route to host",
+	"Operation already in progress",
+	"Operation now in progress",
+	"Stale file handle",
+	"Structure needs cleaning",
+	"Not a XENIX named type file",
+	"No XENIX semaphores available",
+	"Is a named type file",
+	"Remote I/O error",
+	"Quota exceeded",
+	"No medium found",
+	"Wrong medium type",
+	"Operation Canceled",
+	"Required key not available",
+	"Key has expired",
+	"Key has been revoked",
+	"Key was rejected by service",
+	"Owner died",
+	"State not recoverable",
+	"Operation not possible due to RF-kill",
+	"Memory page has hardware error",
+};
+
+const char *lkl_strerror(int err)
+{
+	if (err < 0)
+		err = -err;
+
+	if ((size_t)err >= sizeof(lkl_err_strings) / sizeof(const char *))
+		return "Bad error code";
+
+	return lkl_err_strings[err];
+}
+
+void lkl_perror(char *msg, int err)
+{
+	const char *err_msg = lkl_strerror(err);
+	/* We need to use 'real' printf because lkl_host_ops.print can
+	 * be turned off when debugging is off.
+	 */
+	lkl_printf("%s: %s\n", msg, err_msg);
+}
+
+static int lkl_vprintf(const char *fmt, va_list args)
+{
+	int n;
+	char *buffer;
+	va_list copy;
+
+	if (!lkl_host_ops.print)
+		return 0;
+
+	va_copy(copy, args);
+	n = vsnprintf(NULL, 0, fmt, copy);
+	va_end(copy);
+
+	buffer = lkl_host_ops.mem_alloc(n + 1);
+	if (!buffer)
+		return -1;
+
+	vsnprintf(buffer, n + 1, fmt, args);
+
+	lkl_host_ops.print(buffer, n);
+	lkl_host_ops.mem_free(buffer);
+
+	return n;
+}
+
+int lkl_printf(const char *fmt, ...)
+{
+	int n;
+	va_list args;
+
+	va_start(args, fmt);
+	n = lkl_vprintf(fmt, args);
+	va_end(args);
+
+	return n;
+}
+
+void lkl_bug(const char *fmt, ...)
+{
+	va_list args;
+
+	va_start(args, fmt);
+	lkl_vprintf(fmt, args);
+	va_end(args);
+
+	lkl_host_ops.panic();
+}
+#ifndef __arch_um__
+int lkl_sysctl(const char *path, const char *value)
+{
+	int ret;
+	int fd;
+	char *delim, *p;
+	char full_path[256];
+
+	lkl_mount_fs("proc");
+
+	snprintf(full_path, sizeof(full_path), "/proc/sys/%s", path);
+	p = full_path;
+	while ((delim = strstr(p, "."))) {
+		*delim = '/';
+		p = delim + 1;
+	}
+
+	fd = lkl_sys_open(full_path, LKL_O_WRONLY | LKL_O_CREAT, 0);
+	if (fd < 0) {
+		lkl_printf("lkl_sys_open %s: %s\n",
+			   full_path, lkl_strerror(fd));
+		return -1;
+	}
+	ret = lkl_sys_write(fd, value, strlen(value));
+	if (ret < 0) {
+		lkl_printf("lkl_sys_write %s: %s\n",
+			full_path, lkl_strerror(fd));
+	}
+
+	lkl_sys_close(fd);
+
+	return 0;
+}
+
+/* Configure sysctl parameters as the form of "key=value;key=value;..." */
+void lkl_sysctl_parse_write(const char *sysctls)
+{
+	char *saveptr = NULL, *token = NULL;
+	char *key = NULL, *value = NULL;
+	char strings[256];
+	int ret = 0;
+
+	strcpy(strings, sysctls);
+	for (token = strtok_r(strings, ";", &saveptr); token;
+	     token = strtok_r(NULL, ";", &saveptr)) {
+		key = strtok(token, "=");
+		value = strtok(NULL, "=");
+		ret = lkl_sysctl(key, value);
+		if (ret) {
+			lkl_printf("Failed to configure sysctl entries: %s\n",
+				   lkl_strerror(ret));
+			return;
+		}
+	}
+}
+#endif
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 13/25] lkl tools: host lib: add utilities functions
@ 2020-03-30 14:45     ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: linux-arch, Hajime Tazaki, Conrad Meyer, Octavian Purdila,
	Motomu Utsumi, Akira Moroo, Patrick Collins,
	linux-kernel-library, Yuan Liu

From: Octavian Purdila <tavi.purdila@gmail.com>

Add basic utility functions for getting a string from a kernel error
code and a fprintf like function that uses the host print
operation. The latter is useful for informing the user about errors
that occur in the host library.

Other configuration and debug utilities are also added.

Cc: Conrad Meyer <cem@FreeBSD.org>
Cc: Patrick Collins <pscollins@google.com>
Cc: Yuan Liu <liuyuan@google.com>
Cc: Motomu Utsumi <motomuman@gmail.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/lkl/include/lkl_config.h |  61 +++
 tools/lkl/include/lkl_host.h   |   7 +
 tools/lkl/lib/Build            |   7 +
 tools/lkl/lib/config.c         | 744 +++++++++++++++++++++++++++++++++
 tools/lkl/lib/dbg.c            | 300 +++++++++++++
 tools/lkl/lib/dbg_handler.c    |  44 ++
 tools/lkl/lib/endian.h         |  31 ++
 tools/lkl/lib/jmp_buf.c        |  14 +
 tools/lkl/lib/jmp_buf.h        |   8 +
 tools/lkl/lib/utils.c          | 266 ++++++++++++
 10 files changed, 1482 insertions(+)
 create mode 100644 tools/lkl/include/lkl_config.h
 create mode 100644 tools/lkl/lib/config.c
 create mode 100644 tools/lkl/lib/dbg.c
 create mode 100644 tools/lkl/lib/dbg_handler.c
 create mode 100644 tools/lkl/lib/endian.h
 create mode 100644 tools/lkl/lib/jmp_buf.c
 create mode 100644 tools/lkl/lib/jmp_buf.h
 create mode 100644 tools/lkl/lib/utils.c

diff --git a/tools/lkl/include/lkl_config.h b/tools/lkl/include/lkl_config.h
new file mode 100644
index 000000000000..d3edf8b414cf
--- /dev/null
+++ b/tools/lkl/include/lkl_config.h
@@ -0,0 +1,61 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_LIB_CONFIG_H
+#define _LKL_LIB_CONFIG_H
+
+#define LKL_CONFIG_JSON_TOKEN_MAX 300
+
+struct lkl_config_iface {
+	struct lkl_config_iface *next;
+	struct lkl_netdev *nd;
+
+	/* OBSOLETE: should use IFTYPE and IFPARAMS */
+	char *iftap;
+	char *iftype;
+	char *ifparams;
+	char *ifmtu_str;
+	char *ifip;
+	char *ifipv6;
+	char *ifgateway;
+	char *ifgateway6;
+	char *ifmac_str;
+	char *ifnetmask_len;
+	char *ifnetmask6_len;
+	char *ifoffload_str;
+	char *ifneigh_entries;
+	char *ifqdisc_entries;
+};
+
+struct lkl_config {
+	int ifnum;
+	struct lkl_config_iface *ifaces;
+
+	char *gateway;
+	char *gateway6;
+	char *debug;
+	char *mount;
+	/* single_cpu mode:
+	 * 0: Don't pin to single CPU (default).
+	 * 1: Pin only LKL kernel threads to single CPU.
+	 * 2: Pin all LKL threads to single CPU including all LKL kernel threads
+	 * and device polling threads. Avoid this mode if having busy polling
+	 * threads.
+	 *
+	 * mode 2 can achieve better TCP_RR but worse TCP_STREAM than mode 1.
+	 * You should choose the best for your application and virtio device
+	 * type.
+	 */
+	char *single_cpu;
+	char *sysctls;
+	char *boot_cmdline;
+	char *dump;
+	char *delay_main;
+};
+
+int lkl_load_config_json(struct lkl_config *cfg, char *jstr);
+int lkl_load_config_env(struct lkl_config *cfg);
+void lkl_show_config(struct lkl_config *cfg);
+int lkl_load_config_pre(struct lkl_config *cfg);
+int lkl_load_config_post(struct lkl_config *cfg);
+int lkl_unload_config(struct lkl_config *cfg);
+
+#endif /* _LKL_LIB_CONFIG_H */
diff --git a/tools/lkl/include/lkl_host.h b/tools/lkl/include/lkl_host.h
index b5f96096fe69..85e80eb4ad0d 100644
--- a/tools/lkl/include/lkl_host.h
+++ b/tools/lkl/include/lkl_host.h
@@ -11,6 +11,13 @@ extern "C" {
 
 extern struct lkl_host_operations lkl_host_ops;
 
+/**
+ * lkl_printf - print a message via the host print operation
+ *
+ * @fmt: printf like format string
+ */
+int lkl_printf(const char *fmt, ...);
+
 
 #ifdef __cplusplus
 }
diff --git a/tools/lkl/lib/Build b/tools/lkl/lib/Build
index 8b137891791f..658bfa865b9c 100644
--- a/tools/lkl/lib/Build
+++ b/tools/lkl/lib/Build
@@ -1 +1,8 @@
+CFLAGS_config.o += -I$(srctree)/tools/perf/pmu-events
 
+liblkl-y += jmp_buf.o
+liblkl-y += utils.o
+liblkl-y += dbg.o
+liblkl-y += dbg_handler.o
+liblkl-y += ../../perf/pmu-events/jsmn.o
+liblkl-y += config.o
diff --git a/tools/lkl/lib/config.c b/tools/lkl/lib/config.c
new file mode 100644
index 000000000000..37f8ac4d942a
--- /dev/null
+++ b/tools/lkl/lib/config.c
@@ -0,0 +1,744 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdlib.h>
+#define _HAVE_STRING_ARCH_strtok_r
+#include <string.h>
+#ifndef __MINGW32__
+#include <arpa/inet.h>
+#endif
+#include <lkl_host.h>
+#include <lkl_config.h>
+
+#include "jsmn.h"
+
+static int jsoneq(const char *json, jsmntok_t *tok, const char *s)
+{
+	if (tok->type == JSMN_STRING &&
+		(int) strlen(s) == tok->end - tok->start &&
+		strncmp(json + tok->start, s, tok->end - tok->start) == 0) {
+		return 0;
+	}
+	return -1;
+}
+
+static int cfgcpy(char **to, char *from)
+{
+	if (!from)
+		return 0;
+	if (*to)
+		free(*to);
+	*to = (char *)malloc((strlen(from) + 1) * sizeof(char));
+	if (*to == NULL) {
+		lkl_printf("malloc failed\n");
+		return -1;
+	}
+	strcpy(*to, from);
+	return 0;
+}
+
+static int cfgncpy(char **to, char *from, int len)
+{
+	if (!from)
+		return 0;
+	if (*to)
+		free(*to);
+	*to = (char *)malloc((len + 1) * sizeof(char));
+	if (*to == NULL) {
+		lkl_printf("malloc failed\n");
+		return -1;
+	}
+	strncpy(*to, from, len + 1);
+	(*to)[len] = '\0';
+	return 0;
+}
+
+static int parse_ifarr(struct lkl_config *cfg,
+		jsmntok_t *toks, char *jstr, int startpos)
+{
+	int ifidx, pos, posend, ret;
+	char **cfgptr;
+	struct lkl_config_iface *iface, *prev = NULL;
+
+	if (!cfg || !toks || !jstr)
+		return -1;
+	pos = startpos;
+	pos++;
+	if (toks[pos].type != JSMN_ARRAY) {
+		lkl_printf("unexpected json type, json array expected\n");
+		return -1;
+	}
+
+	cfg->ifnum = toks[pos].size;
+	pos++;
+	iface = cfg->ifaces;
+
+	for (ifidx = 0; ifidx < cfg->ifnum; ifidx++) {
+		if (toks[pos].type != JSMN_OBJECT) {
+			lkl_printf("object json type expected\n");
+			return -1;
+		}
+
+		posend = pos + toks[pos].size;
+		pos++;
+		iface = malloc(sizeof(struct lkl_config_iface));
+		memset(iface, 0, sizeof(struct lkl_config_iface));
+
+		if (prev)
+			prev->next = iface;
+		else
+			cfg->ifaces = iface;
+		prev = iface;
+
+		for (; pos < posend; pos += 2) {
+			if (toks[pos].type != JSMN_STRING) {
+				lkl_printf("object json type expected\n");
+				return -1;
+			}
+			if (jsoneq(jstr, &toks[pos], "type") == 0) {
+				cfgptr = &iface->iftype;
+			} else if (jsoneq(jstr, &toks[pos], "param") == 0) {
+				cfgptr = &iface->ifparams;
+			} else if (jsoneq(jstr, &toks[pos], "mtu") == 0) {
+				cfgptr = &iface->ifmtu_str;
+			} else if (jsoneq(jstr, &toks[pos], "ip") == 0) {
+				cfgptr = &iface->ifip;
+			} else if (jsoneq(jstr, &toks[pos], "ipv6") == 0) {
+				cfgptr = &iface->ifipv6;
+			} else if (jsoneq(jstr, &toks[pos], "ifgateway") == 0) {
+				cfgptr = &iface->ifgateway;
+			} else if (jsoneq(jstr, &toks[pos],
+							"ifgateway6") == 0) {
+				cfgptr = &iface->ifgateway6;
+			} else if (jsoneq(jstr, &toks[pos], "mac") == 0) {
+				cfgptr = &iface->ifmac_str;
+			} else if (jsoneq(jstr, &toks[pos], "masklen") == 0) {
+				cfgptr = &iface->ifnetmask_len;
+			} else if (jsoneq(jstr, &toks[pos], "masklen6") == 0) {
+				cfgptr = &iface->ifnetmask6_len;
+			} else if (jsoneq(jstr, &toks[pos], "neigh") == 0) {
+				cfgptr = &iface->ifneigh_entries;
+			} else if (jsoneq(jstr, &toks[pos], "qdisc") == 0) {
+				cfgptr = &iface->ifqdisc_entries;
+			} else if (jsoneq(jstr, &toks[pos], "offload") == 0) {
+				cfgptr = &iface->ifoffload_str;
+			} else {
+				lkl_printf("unexpected key: %.*s\n",
+						toks[pos].end-toks[pos].start,
+						jstr + toks[pos].start);
+				return -1;
+			}
+			ret = cfgncpy(cfgptr, jstr + toks[pos+1].start,
+					toks[pos+1].end-toks[pos+1].start);
+			if (ret < 0)
+				return ret;
+		}
+	}
+	return pos - startpos;
+}
+
+#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]))
+
+int lkl_load_config_json(struct lkl_config *cfg, char *jstr)
+{
+	int pos, ret;
+	char **cfgptr;
+	jsmn_parser jp;
+	jsmntok_t toks[LKL_CONFIG_JSON_TOKEN_MAX];
+
+	if (!cfg || !jstr)
+		return -1;
+	jsmn_init(&jp);
+	ret = jsmn_parse(&jp, jstr, strlen(jstr), toks, ARRAY_SIZE(toks));
+	if (ret != JSMN_SUCCESS) {
+		lkl_printf("failed to parse json\n");
+		return -1;
+	}
+	if (toks[0].type != JSMN_OBJECT) {
+		lkl_printf("object json type expected\n");
+		return -1;
+	}
+	for (pos = 1; pos < jp.toknext; pos++) {
+		if (toks[pos].type != JSMN_STRING) {
+			lkl_printf("string json type expected\n");
+			return -1;
+		}
+		if (jsoneq(jstr, &toks[pos], "interfaces") == 0) {
+			ret = parse_ifarr(cfg, toks, jstr, pos);
+			if (ret < 0)
+				return ret;
+			pos += ret;
+			pos--;
+			continue;
+		}
+		if (jsoneq(jstr, &toks[pos], "gateway") == 0) {
+			cfgptr = &cfg->gateway;
+		} else if (jsoneq(jstr, &toks[pos], "gateway6") == 0) {
+			cfgptr = &cfg->gateway6;
+		} else if (jsoneq(jstr, &toks[pos], "debug") == 0) {
+			cfgptr = &cfg->debug;
+		} else if (jsoneq(jstr, &toks[pos], "mount") == 0) {
+			cfgptr = &cfg->mount;
+		} else if (jsoneq(jstr, &toks[pos], "singlecpu") == 0) {
+			cfgptr = &cfg->single_cpu;
+		} else if (jsoneq(jstr, &toks[pos], "sysctl") == 0) {
+			cfgptr = &cfg->sysctls;
+		} else if (jsoneq(jstr, &toks[pos], "boot_cmdline") == 0) {
+			cfgptr = &cfg->boot_cmdline;
+		} else if (jsoneq(jstr, &toks[pos], "dump") == 0) {
+			cfgptr = &cfg->dump;
+		} else if (jsoneq(jstr, &toks[pos], "delay_main") == 0) {
+			cfgptr = &cfg->delay_main;
+		} else {
+			lkl_printf("unexpected key in json %.*s\n",
+					toks[pos].end-toks[pos].start,
+					jstr + toks[pos].start);
+			return -1;
+		}
+		pos++;
+		ret = cfgncpy(cfgptr, jstr + toks[pos].start,
+				toks[pos].end-toks[pos].start);
+		if (ret < 0)
+			return ret;
+	}
+	return 0;
+}
+
+void lkl_show_config(struct lkl_config *cfg)
+{
+	struct lkl_config_iface *iface;
+	int i = 0;
+
+	if (!cfg)
+		return;
+	lkl_printf("gateway: %s\n", cfg->gateway);
+	lkl_printf("gateway6: %s\n", cfg->gateway6);
+	lkl_printf("debug: %s\n", cfg->debug);
+	lkl_printf("mount: %s\n", cfg->mount);
+	lkl_printf("singlecpu: %s\n", cfg->single_cpu);
+	lkl_printf("sysctl: %s\n", cfg->sysctls);
+	lkl_printf("cmdline: %s\n", cfg->boot_cmdline);
+	lkl_printf("dump: %s\n", cfg->dump);
+	lkl_printf("delay: %s\n", cfg->delay_main);
+
+	for (iface = cfg->ifaces; iface; iface = iface->next, i++) {
+		lkl_printf("ifmac[%d] = %s\n", i, iface->ifmac_str);
+		lkl_printf("ifmtu[%d] = %s\n", i, iface->ifmtu_str);
+		lkl_printf("iftype[%d] = %s\n", i, iface->iftype);
+		lkl_printf("ifparam[%d] = %s\n", i, iface->ifparams);
+		lkl_printf("ifip[%d] = %s\n", i, iface->ifip);
+		lkl_printf("ifmasklen[%d] = %s\n", i, iface->ifnetmask_len);
+		lkl_printf("ifgateway[%d] = %s\n", i, iface->ifgateway);
+		lkl_printf("ifip6[%d] = %s\n", i, iface->ifipv6);
+		lkl_printf("ifmasklen6[%d] = %s\n", i, iface->ifnetmask6_len);
+		lkl_printf("ifgateway6[%d] = %s\n", i, iface->ifgateway6);
+		lkl_printf("ifoffload[%d] = %s\n", i, iface->ifoffload_str);
+		lkl_printf("ifneigh[%d] = %s\n", i, iface->ifneigh_entries);
+		lkl_printf("ifqdisk[%d] = %s\n", i, iface->ifqdisc_entries);
+	}
+}
+
+int lkl_load_config_env(struct lkl_config *cfg)
+{
+	int ret;
+	char *enviftype = getenv("LKL_HIJACK_NET_IFTYPE");
+	char *envifparams = getenv("LKL_HIJACK_NET_IFPARAMS");
+	char *envmtu_str = getenv("LKL_HIJACK_NET_MTU");
+	char *envip = getenv("LKL_HIJACK_NET_IP");
+	char *envipv6 = getenv("LKL_HIJACK_NET_IPV6");
+	char *envifgateway = getenv("LKL_HIJACK_NET_IFGATEWAY");
+	char *envifgateway6 = getenv("LKL_HIJACK_NET_IFGATEWAY6");
+	char *envmac_str = getenv("LKL_HIJACK_NET_MAC");
+	char *envnetmask_len = getenv("LKL_HIJACK_NET_NETMASK_LEN");
+	char *envnetmask6_len = getenv("LKL_HIJACK_NET_NETMASK6_LEN");
+	char *envgateway = getenv("LKL_HIJACK_NET_GATEWAY");
+	char *envgateway6 = getenv("LKL_HIJACK_NET_GATEWAY6");
+	char *envdebug = getenv("LKL_HIJACK_DEBUG");
+	char *envmount = getenv("LKL_HIJACK_MOUNT");
+	char *envneigh_entries = getenv("LKL_HIJACK_NET_NEIGHBOR");
+	char *envqdisc_entries = getenv("LKL_HIJACK_NET_QDISC");
+	char *envsingle_cpu = getenv("LKL_HIJACK_SINGLE_CPU");
+	char *envoffload_str = getenv("LKL_HIJACK_OFFLOAD");
+	char *envsysctls = getenv("LKL_HIJACK_SYSCTL");
+	char *envboot_cmdline = getenv("LKL_HIJACK_BOOT_CMDLINE") ? : "";
+	char *envdump = getenv("LKL_HIJACK_DUMP");
+	struct lkl_config_iface *iface;
+
+	if (!cfg)
+		return -1;
+	if (enviftype)
+		cfg->ifnum = 1;
+
+	iface = malloc(sizeof(struct lkl_config_iface));
+	memset(iface, 0, sizeof(struct lkl_config_iface));
+
+	ret = cfgcpy(&iface->iftype, enviftype);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifparams, envifparams);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifmtu_str, envmtu_str);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifip, envip);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifipv6, envipv6);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifgateway, envifgateway);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifgateway6, envifgateway6);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifmac_str, envmac_str);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifnetmask_len, envnetmask_len);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifnetmask6_len, envnetmask6_len);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifoffload_str, envoffload_str);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifneigh_entries, envneigh_entries);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&iface->ifqdisc_entries, envqdisc_entries);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&cfg->gateway, envgateway);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&cfg->gateway6, envgateway6);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&cfg->debug, envdebug);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&cfg->mount, envmount);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&cfg->single_cpu, envsingle_cpu);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&cfg->sysctls, envsysctls);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&cfg->boot_cmdline, envboot_cmdline);
+	if (ret < 0)
+		return ret;
+	ret = cfgcpy(&cfg->dump, envdump);
+	if (ret < 0)
+		return ret;
+	return 0;
+}
+
+static int parse_mac_str(char *mac_str, __lkl__u8 mac[LKL_ETH_ALEN])
+{
+	char delim[] = ":";
+	char *saveptr = NULL, *token = NULL;
+	int i = 0;
+
+	if (!mac_str)
+		return 0;
+
+	for (token = strtok_r(mac_str, delim, &saveptr);
+	     i < LKL_ETH_ALEN; i++) {
+		if (!token) {
+			/* The address is too short */
+			return -1;
+		}
+
+		mac[i] = (__lkl__u8) strtol(token, NULL, 16);
+		token = strtok_r(NULL, delim, &saveptr);
+	}
+
+	if (strtok_r(NULL, delim, &saveptr)) {
+		/* The address is too long */
+		return -1;
+	}
+
+	return 1;
+}
+
+/* Add permanent neighbor entries in the form of "ip|mac;ip|mac;..." */
+static void add_neighbor(int ifindex, char *entries)
+{
+	char *saveptr = NULL, *token = NULL;
+	char *ip = NULL, *mac_str = NULL;
+	int ret = 0;
+	__lkl__u8 mac[LKL_ETH_ALEN];
+	char ip_addr[16];
+	int af;
+
+	for (token = strtok_r(entries, ";", &saveptr); token;
+	     token = strtok_r(NULL, ";", &saveptr)) {
+		ip = strtok(token, "|");
+		mac_str = strtok(NULL, "|");
+		if (ip == NULL || mac_str == NULL || strtok(NULL, "|") != NULL)
+			return;
+
+		af = LKL_AF_INET;
+		ret = inet_pton(LKL_AF_INET, ip, ip_addr);
+		if (ret == 0) {
+			ret = inet_pton(LKL_AF_INET6, ip, ip_addr);
+			af = LKL_AF_INET6;
+		}
+		if (ret != 1) {
+			lkl_printf("Bad ip address: %s\n", ip);
+			return;
+		}
+
+		ret = parse_mac_str(mac_str, mac);
+		if (ret != 1) {
+			lkl_printf("Failed to parse mac: %s\n", mac_str);
+			return;
+		}
+		ret = lkl_add_neighbor(ifindex, af, ip_addr, mac);
+		if (ret) {
+			lkl_printf("Failed to add neighbor entry: %s\n",
+				   lkl_strerror(ret));
+			return;
+		}
+	}
+}
+
+/* We don't have an easy way to make FILE*s out of our fds, so we
+ * can't use e.g. fgets
+ */
+static int dump_file(char *path)
+{
+	int ret = -1, bytes_read = 0;
+	char str[1024] = { 0 };
+	int fd;
+
+	fd = lkl_sys_open(path, LKL_O_RDONLY, 0);
+
+	if (fd < 0) {
+		lkl_printf("%s lkl_sys_open %s: %s\n",
+			   __func__, path, lkl_strerror(fd));
+		return -1;
+	}
+
+	/* Need to print this out in order to make sense of the output */
+	lkl_printf("Reading from %s:\n==========\n", path);
+	while ((ret = lkl_sys_read(fd, str, sizeof(str) - 1)) > 0)
+		bytes_read += lkl_printf("%s", str);
+	lkl_printf("==========\n");
+
+	if (ret) {
+		lkl_printf("%s lkl_sys_read %s: %s\n",
+			   __func__, path, lkl_strerror(ret));
+		return -1;
+	}
+
+	return 0;
+}
+
+static void mount_cmds_exec(char *_cmds, int (*callback)(char *))
+{
+	char *saveptr = NULL, *token;
+	int ret = 0;
+	char *cmds = strdup(_cmds);
+
+	token = strtok_r(cmds, ",", &saveptr);
+
+	while (token && ret >= 0) {
+		ret = callback(token);
+		token = strtok_r(NULL, ",", &saveptr);
+	}
+
+	if (ret < 0)
+		lkl_printf("%s: failed parsing %s\n", __func__, _cmds);
+
+	free(cmds);
+}
+
+static int lkl_config_netdev_create(struct lkl_config *cfg,
+				    struct lkl_config_iface *iface)
+{
+	int ret, offload = 0;
+	struct lkl_netdev_args nd_args;
+	__lkl__u8 mac[LKL_ETH_ALEN] = {0};
+	struct lkl_netdev *nd = NULL;
+
+	if (iface->ifoffload_str)
+		offload = strtol(iface->ifoffload_str, NULL, 0);
+	memset(&nd_args, 0, sizeof(struct lkl_netdev_args));
+
+	if (!nd && iface->iftype && iface->ifparams) {
+	}
+
+	if (nd) {
+		if ((mac[0] != 0) || (mac[1] != 0) ||
+				(mac[2] != 0) || (mac[3] != 0) ||
+				(mac[4] != 0) || (mac[5] != 0)) {
+			nd_args.mac = mac;
+		} else {
+			ret = parse_mac_str(iface->ifmac_str, mac);
+
+			if (ret < 0) {
+				lkl_printf("failed to parse mac\n");
+				return -1;
+			} else if (ret > 0) {
+				nd_args.mac = mac;
+			} else {
+				nd_args.mac = NULL;
+			}
+		}
+
+		nd_args.offload = offload;
+		iface->nd = nd;
+	}
+	return 0;
+}
+
+static int lkl_config_netdev_configure(struct lkl_config *cfg,
+				       struct lkl_config_iface *iface)
+{
+	int ret, nd_ifindex = -1;
+	struct lkl_netdev *nd = iface->nd;
+
+	if (!nd) {
+		lkl_printf("no netdev available %s\n", iface ? iface->ifparams
+			   : "(null)");
+		return -1;
+	}
+
+	if (nd->id >= 0) {
+		nd_ifindex = lkl_netdev_get_ifindex(nd->id);
+		if (nd_ifindex > 0)
+			lkl_if_up(nd_ifindex);
+		else
+			lkl_printf(
+				"failed to get ifindex for netdev id %d: %s\n",
+				nd->id, lkl_strerror(nd_ifindex));
+	}
+
+	if (nd_ifindex >= 0 && iface->ifmtu_str) {
+		int mtu = atoi(iface->ifmtu_str);
+
+		ret = lkl_if_set_mtu(nd_ifindex, mtu);
+		if (ret < 0)
+			lkl_printf("failed to set MTU: %s\n",
+				   lkl_strerror(ret));
+	}
+
+	if (nd_ifindex >= 0 && iface->ifip && iface->ifnetmask_len) {
+		unsigned int addr;
+
+		if (inet_pton(LKL_AF_INET, iface->ifip,
+			      (struct lkl_in_addr *)&addr) != 1)
+			lkl_printf("Invalid ipv4 address: %s\n", iface->ifip);
+
+		int nmlen = atoi(iface->ifnetmask_len);
+
+		if (addr != LKL_INADDR_NONE && nmlen > 0 && nmlen < 32) {
+			ret = lkl_if_set_ipv4(nd_ifindex, addr, nmlen);
+			if (ret < 0)
+				lkl_printf("failed to set IPv4 address: %s\n",
+					   lkl_strerror(ret));
+		}
+		if (iface->ifgateway) {
+			unsigned int gwaddr;
+
+			if (inet_pton(LKL_AF_INET, iface->ifgateway,
+				      (struct lkl_in_addr *)&gwaddr) != 1)
+				lkl_printf("Invalid ipv4 gateway: %s\n",
+					   iface->ifgateway);
+
+			if (gwaddr != LKL_INADDR_NONE) {
+				ret = lkl_if_set_ipv4_gateway(nd_ifindex,
+						addr, nmlen, gwaddr);
+				if (ret < 0)
+					lkl_printf(
+						"failed to set v4 if gw: %s\n",
+						lkl_strerror(ret));
+			}
+		}
+	}
+
+	if (nd_ifindex >= 0 && iface->ifipv6 &&
+			iface->ifnetmask6_len) {
+		struct lkl_in6_addr addr;
+		unsigned int pflen = atoi(iface->ifnetmask6_len);
+
+		if (inet_pton(LKL_AF_INET6, iface->ifipv6,
+			      (struct lkl_in6_addr *)&addr) != 1) {
+			lkl_printf("Invalid ipv6 addr: %s\n",
+				   iface->ifipv6);
+		}  else {
+			ret = lkl_if_set_ipv6(nd_ifindex, &addr, pflen);
+			if (ret < 0)
+				lkl_printf("failed to set IPv6 address: %s\n",
+					   lkl_strerror(ret));
+		}
+		if (iface->ifgateway6) {
+			char gwaddr[16];
+
+			if (inet_pton(LKL_AF_INET6, iface->ifgateway6,
+								gwaddr) != 1) {
+				lkl_printf("Invalid ipv6 gateway: %s\n",
+					   iface->ifgateway6);
+			} else {
+				ret = lkl_if_set_ipv6_gateway(nd_ifindex,
+						&addr, pflen, gwaddr);
+				if (ret < 0)
+					lkl_printf(
+						"failed to set v6 if gw: %s\n",
+						lkl_strerror(ret));
+			}
+		}
+	}
+
+	if (nd_ifindex >= 0 && iface->ifneigh_entries)
+		add_neighbor(nd_ifindex, iface->ifneigh_entries);
+
+	if (nd_ifindex >= 0 && iface->ifqdisc_entries)
+		lkl_qdisc_parse_add(nd_ifindex, iface->ifqdisc_entries);
+
+	return 0;
+}
+
+static void free_cfgparam(char *cfgparam)
+{
+	if (cfgparam)
+		free(cfgparam);
+}
+
+static int lkl_clean_config(struct lkl_config *cfg)
+{
+	struct lkl_config_iface *iface;
+
+	if (!cfg)
+		return -1;
+
+	for (iface = cfg->ifaces; iface; iface = iface->next) {
+		free_cfgparam(iface->iftype);
+		free_cfgparam(iface->ifparams);
+		free_cfgparam(iface->ifmtu_str);
+		free_cfgparam(iface->ifip);
+		free_cfgparam(iface->ifipv6);
+		free_cfgparam(iface->ifgateway);
+		free_cfgparam(iface->ifgateway6);
+		free_cfgparam(iface->ifmac_str);
+		free_cfgparam(iface->ifnetmask_len);
+		free_cfgparam(iface->ifnetmask6_len);
+		free_cfgparam(iface->ifoffload_str);
+		free_cfgparam(iface->ifneigh_entries);
+		free_cfgparam(iface->ifqdisc_entries);
+	}
+	free_cfgparam(cfg->gateway);
+	free_cfgparam(cfg->gateway6);
+	free_cfgparam(cfg->debug);
+	free_cfgparam(cfg->mount);
+	free_cfgparam(cfg->single_cpu);
+	free_cfgparam(cfg->sysctls);
+	free_cfgparam(cfg->boot_cmdline);
+	free_cfgparam(cfg->dump);
+	free_cfgparam(cfg->delay_main);
+	return 0;
+}
+
+
+int lkl_load_config_pre(struct lkl_config *cfg)
+{
+	int lkl_debug, ret;
+	struct lkl_config_iface *iface;
+
+	if (!cfg)
+		return 0;
+
+	if (cfg->debug)
+		lkl_debug = strtol(cfg->debug, NULL, 0);
+
+	if (!cfg->debug || (lkl_debug == 0))
+		lkl_host_ops.print = NULL;
+
+	for (iface = cfg->ifaces; iface; iface = iface->next) {
+		ret = lkl_config_netdev_create(cfg, iface);
+		if (ret < 0)
+			return -1;
+	}
+
+	return 0;
+}
+
+int lkl_load_config_post(struct lkl_config *cfg)
+{
+	int ret;
+	struct lkl_config_iface *iface;
+
+	if (!cfg)
+		return 0;
+
+	if (cfg->mount)
+		mount_cmds_exec(cfg->mount, lkl_mount_fs);
+
+	for (iface = cfg->ifaces; iface; iface = iface->next) {
+		ret = lkl_config_netdev_configure(cfg, iface);
+		if (ret < 0)
+			break;
+	}
+
+	if (cfg->gateway) {
+		unsigned int gwaddr;
+
+		if (inet_pton(LKL_AF_INET, cfg->gateway,
+			      (struct lkl_in_addr *)&gwaddr) != 1)
+			lkl_printf("Invalid ipv4 gateway: %s\n", cfg->gateway);
+
+		if (gwaddr != LKL_INADDR_NONE) {
+			ret = lkl_set_ipv4_gateway(gwaddr);
+			if (ret < 0)
+				lkl_printf("failed to set IPv4 gateway: %s\n",
+					   lkl_strerror(ret));
+		}
+	}
+
+	if (cfg->gateway6) {
+		char gw[16];
+
+		if (inet_pton(LKL_AF_INET6, cfg->gateway6, gw) != 1) {
+			lkl_printf("Invalid ipv6 gateway: %s\n", cfg->gateway6);
+		} else {
+			ret = lkl_set_ipv6_gateway(gw);
+			if (ret < 0)
+				lkl_printf("failed to set IPv6 gateway: %s\n",
+					   lkl_strerror(ret));
+		}
+	}
+
+	if (cfg->sysctls)
+		lkl_sysctl_parse_write(cfg->sysctls);
+
+	/* put a delay before calling main() */
+	if (cfg->delay_main) {
+		unsigned long delay = strtoul(cfg->delay_main, NULL, 10);
+
+		if (delay == ~0UL)
+			lkl_printf("got invalid delay_main value (%s)\n",
+				   cfg->delay_main);
+		else {
+			lkl_printf("sleeping %lu usec\n", delay);
+			usleep(delay);
+		}
+	}
+
+	return 0;
+}
+
+int lkl_unload_config(struct lkl_config *cfg)
+{
+	if (cfg) {
+		if (cfg->dump)
+			mount_cmds_exec(cfg->dump, dump_file);
+
+		lkl_clean_config(cfg);
+	}
+
+	return 0;
+}
diff --git a/tools/lkl/lib/dbg.c b/tools/lkl/lib/dbg.c
new file mode 100644
index 000000000000..b613353bce5c
--- /dev/null
+++ b/tools/lkl/lib/dbg.c
@@ -0,0 +1,300 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <errno.h>
+#include <lkl.h>
+#include <limits.h>
+#include <string.h>
+#include <stdio.h>
+#include <stdlib.h>
+
+static const char *PROMOTE = "$";
+#define str(x) #x
+#define xstr(s) str(s)
+#define MAX_BUF 100
+static char cmd[MAX_BUF];
+static char argv[10][MAX_BUF];
+static int argc;
+static char cur_dir[MAX_BUF] = "/";
+
+static char *normalize_path(const char *src, size_t src_len)
+{
+	char *res;
+	unsigned int res_len;
+	const char *ptr = src;
+	const char *end = &src[src_len];
+	const char *next;
+
+	res = malloc((src_len > 0 ? src_len : 1) + 1);
+	res_len = 0;
+
+	for (ptr = src; ptr < end; ptr = next+1) {
+		size_t len;
+
+		next = memchr(ptr, '/', end-ptr);
+		if (next == NULL)
+			next = end;
+
+		len = next-ptr;
+		switch (len) {
+		case 2:
+			if (ptr[0] == '.' && ptr[1] == '.') {
+				const char *slash = strrchr(res, '/');
+
+				if (slash != NULL)
+					res_len = slash - res;
+				continue;
+			}
+			break;
+		case 1:
+			if (ptr[0] == '.')
+				continue;
+			break;
+		case 0:
+			continue;
+		}
+		res[res_len++] = '/';
+		memcpy(&res[res_len], ptr, len);
+		res_len += len;
+	}
+	if (res_len == 0)
+		res[res_len++] = '/';
+	res[res_len] = '\0';
+	return res;
+}
+
+static void build_path(char *path)
+{
+	char *npath;
+
+	strcpy(path, cur_dir);
+	if (argc >= 1) {
+		if (argv[0][0] == '/')
+			strncpy(path, argv[0], LKL_PATH_MAX);
+		else {
+			strncat(path, "/", LKL_PATH_MAX - strlen(path) - 1);
+			strncat(path, argv[0], LKL_PATH_MAX - strlen(path) - 1);
+		}
+	}
+	npath = normalize_path(path, strlen(path));
+	strcpy(path, npath);
+	free(npath);
+}
+
+static void help(void)
+{
+	const char *msg =
+		"cat FILE\n"
+		"\tShow content of FILE\n"
+		"cd [DIR]\n"
+		"\tChange directory to DIR\n"
+		"exit\n"
+		"\tExit the debug session\n"
+		"help\n"
+		"\tShow this message\n"
+		"ls [DIR]\n"
+		"\tList files in DIR\n"
+		"mount FSTYPE\n"
+		"\tMount FSTYPE as /FSTYPE\n"
+		"overwrite FILE\n"
+		"\tOverwrite content of FILE from stdin\n"
+		"pwd\n"
+		"\tShow current directory\n"
+		;
+	printf("%s", msg);
+}
+
+static void ls(void)
+{
+	char path[LKL_PATH_MAX];
+	struct lkl_dir *dir;
+	struct lkl_linux_dirent64 *de;
+	int err;
+
+	build_path(path);
+	dir = lkl_opendir(path, &err);
+	if (dir) {
+		do {
+			de = lkl_readdir(dir);
+			if (de) {
+				printf("%s\n", de->d_name);
+			} else {
+				err = lkl_errdir(dir);
+				if (err != 0) {
+					fprintf(stderr, "%s\n",
+						lkl_strerror(err));
+				}
+				break;
+			}
+		} while (1);
+		lkl_closedir(dir);
+	} else {
+		fprintf(stderr, "%s: %s\n", path, lkl_strerror(err));
+	}
+}
+
+static void cd(void)
+{
+	char path[LKL_PATH_MAX];
+	struct lkl_dir *dir;
+	int err;
+
+	build_path(path);
+	dir = lkl_opendir(path, &err);
+	if (dir) {
+		strcpy(cur_dir, path);
+		lkl_closedir(dir);
+	} else {
+		fprintf(stderr, "%s: %s\n", path, lkl_strerror(err));
+	}
+}
+
+static void mount(void)
+{
+	char *fstype;
+	int ret = 0;
+
+	if (argc != 1) {
+		fprintf(stderr, "%s\n", "One argument is needed.");
+		return;
+	}
+
+	fstype = argv[0];
+	ret = lkl_mount_fs(fstype);
+	if (ret == 1)
+		fprintf(stderr, "%s is already mounted.\n", fstype);
+}
+
+static void cat(void)
+{
+	char path[LKL_PATH_MAX];
+	int ret;
+	char buf[1024];
+	int fd;
+
+	if (argc != 1) {
+		fprintf(stderr, "%s\n", "One argument is needed.");
+		return;
+	}
+
+	build_path(path);
+	fd = lkl_sys_open(path, LKL_O_RDONLY, 0);
+
+	if (fd < 0) {
+		fprintf(stderr, "lkl_sys_open %s: %s\n",
+			path, lkl_strerror(fd));
+		return;
+	}
+
+	while ((ret = lkl_sys_read(fd, buf, sizeof(buf) - 1)) > 0) {
+		buf[ret] = '\0';
+		printf("%s", buf);
+	}
+
+	if (ret) {
+		fprintf(stderr, "lkl_sys_read %s: %s\n",
+			path, lkl_strerror(ret));
+	}
+	lkl_sys_close(fd);
+}
+
+static void overwrite(void)
+{
+	char path[LKL_PATH_MAX];
+	int ret;
+	int fd;
+	char buf[1024];
+
+	build_path(path);
+	fd = lkl_sys_open(path, LKL_O_WRONLY | LKL_O_CREAT, 0);
+	if (fd < 0) {
+		fprintf(stderr, "lkl_sys_open %s: %s\n",
+			path, lkl_strerror(fd));
+		return;
+	}
+	printf("Input the content and stop by hitting Ctrl-D:\n");
+	while (fgets(buf, 1023, stdin)) {
+		ret = lkl_sys_write(fd, buf, strlen(buf));
+		if (ret < 0) {
+			fprintf(stderr, "lkl_sys_write %s: %s\n",
+				path, lkl_strerror(fd));
+		}
+	}
+	lkl_sys_close(fd);
+}
+
+static void pwd(void)
+{
+	printf("%s\n", cur_dir);
+}
+
+static int parse_cmd(char *input)
+{
+	char *token;
+
+	token = strtok(input, " ");
+	if (token)
+		strcpy(cmd, token);
+	else
+		return -1;
+
+	argc = 0;
+	token = strtok(NULL, " ");
+	while (token) {
+		if (argc >= 10) {
+			fprintf(stderr, "To many args > 10\n");
+			return -1;
+		}
+		strcpy(argv[argc++], token);
+		token = strtok(NULL, " ");
+	}
+	return 0;
+}
+
+static void run_cmd(void)
+{
+	if (strcmp(cmd, "cat") == 0)
+		cat();
+	else if (strcmp(cmd, "cd") == 0)
+		cd();
+	else if (strcmp(cmd, "help") == 0)
+		help();
+	else if (strcmp(cmd, "ls") == 0)
+		ls();
+	else if (strcmp(cmd, "mount") == 0)
+		mount();
+	else if (strcmp(cmd, "overwrite") == 0)
+		overwrite();
+	else if (strcmp(cmd, "pwd") == 0)
+		pwd();
+	else
+		fprintf(stderr, "Unknown command: %s\n", cmd);
+}
+
+void dbg_entrance(void)
+{
+	char input[MAX_BUF + 1];
+	int ret;
+	int c;
+
+	printf("Type help to see a list of commands\n");
+	do {
+		printf("%s ", PROMOTE);
+		ret = scanf("%" xstr(MAX_BUF) "[^\n]s", input);
+		while ((c = getchar()) != '\n' && c != EOF)
+			;
+		if (ret == 0)
+			continue;
+		if (ret != 1 && errno != EINTR) {
+			perror("scanf");
+			continue;
+		}
+		if (strlen(input) == MAX_BUF) {
+			fprintf(stderr, "Too long input > %d\n", MAX_BUF - 1);
+			continue;
+		}
+		if (parse_cmd(input))
+			continue;
+		if (strcmp(cmd, "exit") == 0)
+			break;
+		run_cmd();
+	} while (1);
+}
diff --git a/tools/lkl/lib/dbg_handler.c b/tools/lkl/lib/dbg_handler.c
new file mode 100644
index 000000000000..01d165a5fc1e
--- /dev/null
+++ b/tools/lkl/lib/dbg_handler.c
@@ -0,0 +1,44 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <lkl_host.h>
+
+extern void dbg_entrance(void);
+static int dbg_running;
+
+static void dbg_thread(void *arg)
+{
+	lkl_host_ops.thread_detach();
+	printf("======Enter Debug======\n");
+	dbg_entrance();
+	printf("======Exit Debug======\n");
+	dbg_running = 0;
+}
+
+void dbg_handler(int signum)
+{
+	/* We don't care about the possible race on dbg_running. */
+	if (dbg_running) {
+		fprintf(stderr, "A debug lib is running\n");
+		return;
+	}
+	dbg_running = 1;
+	lkl_host_ops.thread_create(&dbg_thread, NULL);
+}
+
+#ifndef __MINGW32__
+#include <signal.h>
+void lkl_register_dbg_handler(void)
+{
+	struct sigaction sa;
+
+	sigemptyset(&sa.sa_mask);
+	sa.sa_handler = dbg_handler;
+	if (sigaction(SIGTSTP, &sa, NULL) == -1)
+		perror("sigaction");
+}
+#else
+void lkl_register_dbg_handler(void)
+{
+	fprintf(stderr, "%s is not implemented.\n", __func__);
+}
+#endif
diff --git a/tools/lkl/lib/endian.h b/tools/lkl/lib/endian.h
new file mode 100644
index 000000000000..aaccfa0edb65
--- /dev/null
+++ b/tools/lkl/lib/endian.h
@@ -0,0 +1,31 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_LIB_ENDIAN_H
+#define _LKL_LIB_ENDIAN_H
+
+#if defined(__FreeBSD__)
+#include <sys/endian.h>
+#elif defined(__ANDROID__)
+#include <sys/endian.h>
+#elif defined(__MINGW32__)
+#include <winsock.h>
+#define le32toh(x) (x)
+#define le16toh(x) (x)
+#define htole32(x) (x)
+#define htole16(x) (x)
+#define le64toh(x) (x)
+#define htobe32(x) htonl(x)
+#define htobe16(x) htons(x)
+#define be32toh(x) ntohl(x)
+#define be16toh(x) ntohs(x)
+#else
+#include <endian.h>
+#endif
+
+#ifndef htonl
+#define htonl(x) htobe32(x)
+#define htons(x) htobe16(x)
+#define ntohl(x) be32toh(x)
+#define ntohs(x) be16toh(x)
+#endif
+
+#endif /* _LKL_LIB_ENDIAN_H */
diff --git a/tools/lkl/lib/jmp_buf.c b/tools/lkl/lib/jmp_buf.c
new file mode 100644
index 000000000000..f6bdd7e4bd83
--- /dev/null
+++ b/tools/lkl/lib/jmp_buf.c
@@ -0,0 +1,14 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <setjmp.h>
+#include <lkl_host.h>
+
+void jmp_buf_set(struct lkl_jmp_buf *jmpb, void (*f)(void))
+{
+	if (!setjmp(*((jmp_buf *)jmpb->buf)))
+		f();
+}
+
+void jmp_buf_longjmp(struct lkl_jmp_buf *jmpb, int val)
+{
+	longjmp(*((jmp_buf *)jmpb->buf), val);
+}
diff --git a/tools/lkl/lib/jmp_buf.h b/tools/lkl/lib/jmp_buf.h
new file mode 100644
index 000000000000..8782cbaaf51f
--- /dev/null
+++ b/tools/lkl/lib/jmp_buf.h
@@ -0,0 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_LIB_JMP_BUF_H
+#define _LKL_LIB_JMP_BUF_H
+
+void jmp_buf_set(struct lkl_jmp_buf *jmpb, void (*f)(void));
+void jmp_buf_longjmp(struct lkl_jmp_buf *jmpb, int val);
+
+#endif
diff --git a/tools/lkl/lib/utils.c b/tools/lkl/lib/utils.c
new file mode 100644
index 000000000000..7de92bbe5475
--- /dev/null
+++ b/tools/lkl/lib/utils.c
@@ -0,0 +1,266 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdarg.h>
+#include <stdio.h>
+#include <string.h>
+#include <lkl_host.h>
+
+static const char * const lkl_err_strings[] = {
+	"Success",
+	"Operation not permitted",
+	"No such file or directory",
+	"No such process",
+	"Interrupted system call",
+	"I/O error",
+	"No such device or address",
+	"Argument list too long",
+	"Exec format error",
+	"Bad file number",
+	"No child processes",
+	"Try again",
+	"Out of memory",
+	"Permission denied",
+	"Bad address",
+	"Block device required",
+	"Device or resource busy",
+	"File exists",
+	"Cross-device link",
+	"No such device",
+	"Not a directory",
+	"Is a directory",
+	"Invalid argument",
+	"File table overflow",
+	"Too many open files",
+	"Not a typewriter",
+	"Text file busy",
+	"File too large",
+	"No space left on device",
+	"Illegal seek",
+	"Read-only file system",
+	"Too many links",
+	"Broken pipe",
+	"Math argument out of domain of func",
+	"Math result not representable",
+	"Resource deadlock would occur",
+	"File name too long",
+	"No record locks available",
+	"Invalid system call number",
+	"Directory not empty",
+	"Too many symbolic links encountered",
+	"Bad error code", /* EWOULDBLOCK is EAGAIN */
+	"No message of desired type",
+	"Identifier removed",
+	"Channel number out of range",
+	"Level 2 not synchronized",
+	"Level 3 halted",
+	"Level 3 reset",
+	"Link number out of range",
+	"Protocol driver not attached",
+	"No CSI structure available",
+	"Level 2 halted",
+	"Invalid exchange",
+	"Invalid request descriptor",
+	"Exchange full",
+	"No anode",
+	"Invalid request code",
+	"Invalid slot",
+	"Bad error code", /* EDEADLOCK is EDEADLK */
+	"Bad font file format",
+	"Device not a stream",
+	"No data available",
+	"Timer expired",
+	"Out of streams resources",
+	"Machine is not on the network",
+	"Package not installed",
+	"Object is remote",
+	"Link has been severed",
+	"Advertise error",
+	"Srmount error",
+	"Communication error on send",
+	"Protocol error",
+	"Multihop attempted",
+	"RFS specific error",
+	"Not a data message",
+	"Value too large for defined data type",
+	"Name not unique on network",
+	"File descriptor in bad state",
+	"Remote address changed",
+	"Can not access a needed shared library",
+	"Accessing a corrupted shared library",
+	".lib section in a.out corrupted",
+	"Attempting to link in too many shared libraries",
+	"Cannot exec a shared library directly",
+	"Illegal byte sequence",
+	"Interrupted system call should be restarted",
+	"Streams pipe error",
+	"Too many users",
+	"Socket operation on non-socket",
+	"Destination address required",
+	"Message too long",
+	"Protocol wrong type for socket",
+	"Protocol not available",
+	"Protocol not supported",
+	"Socket type not supported",
+	"Operation not supported on transport endpoint",
+	"Protocol family not supported",
+	"Address family not supported by protocol",
+	"Address already in use",
+	"Cannot assign requested address",
+	"Network is down",
+	"Network is unreachable",
+	"Network dropped connection because of reset",
+	"Software caused connection abort",
+	"Connection reset by peer",
+	"No buffer space available",
+	"Transport endpoint is already connected",
+	"Transport endpoint is not connected",
+	"Cannot send after transport endpoint shutdown",
+	"Too many references: cannot splice",
+	"Connection timed out",
+	"Connection refused",
+	"Host is down",
+	"No route to host",
+	"Operation already in progress",
+	"Operation now in progress",
+	"Stale file handle",
+	"Structure needs cleaning",
+	"Not a XENIX named type file",
+	"No XENIX semaphores available",
+	"Is a named type file",
+	"Remote I/O error",
+	"Quota exceeded",
+	"No medium found",
+	"Wrong medium type",
+	"Operation Canceled",
+	"Required key not available",
+	"Key has expired",
+	"Key has been revoked",
+	"Key was rejected by service",
+	"Owner died",
+	"State not recoverable",
+	"Operation not possible due to RF-kill",
+	"Memory page has hardware error",
+};
+
+const char *lkl_strerror(int err)
+{
+	if (err < 0)
+		err = -err;
+
+	if ((size_t)err >= sizeof(lkl_err_strings) / sizeof(const char *))
+		return "Bad error code";
+
+	return lkl_err_strings[err];
+}
+
+void lkl_perror(char *msg, int err)
+{
+	const char *err_msg = lkl_strerror(err);
+	/* We need to use 'real' printf because lkl_host_ops.print can
+	 * be turned off when debugging is off.
+	 */
+	lkl_printf("%s: %s\n", msg, err_msg);
+}
+
+static int lkl_vprintf(const char *fmt, va_list args)
+{
+	int n;
+	char *buffer;
+	va_list copy;
+
+	if (!lkl_host_ops.print)
+		return 0;
+
+	va_copy(copy, args);
+	n = vsnprintf(NULL, 0, fmt, copy);
+	va_end(copy);
+
+	buffer = lkl_host_ops.mem_alloc(n + 1);
+	if (!buffer)
+		return -1;
+
+	vsnprintf(buffer, n + 1, fmt, args);
+
+	lkl_host_ops.print(buffer, n);
+	lkl_host_ops.mem_free(buffer);
+
+	return n;
+}
+
+int lkl_printf(const char *fmt, ...)
+{
+	int n;
+	va_list args;
+
+	va_start(args, fmt);
+	n = lkl_vprintf(fmt, args);
+	va_end(args);
+
+	return n;
+}
+
+void lkl_bug(const char *fmt, ...)
+{
+	va_list args;
+
+	va_start(args, fmt);
+	lkl_vprintf(fmt, args);
+	va_end(args);
+
+	lkl_host_ops.panic();
+}
+#ifndef __arch_um__
+int lkl_sysctl(const char *path, const char *value)
+{
+	int ret;
+	int fd;
+	char *delim, *p;
+	char full_path[256];
+
+	lkl_mount_fs("proc");
+
+	snprintf(full_path, sizeof(full_path), "/proc/sys/%s", path);
+	p = full_path;
+	while ((delim = strstr(p, "."))) {
+		*delim = '/';
+		p = delim + 1;
+	}
+
+	fd = lkl_sys_open(full_path, LKL_O_WRONLY | LKL_O_CREAT, 0);
+	if (fd < 0) {
+		lkl_printf("lkl_sys_open %s: %s\n",
+			   full_path, lkl_strerror(fd));
+		return -1;
+	}
+	ret = lkl_sys_write(fd, value, strlen(value));
+	if (ret < 0) {
+		lkl_printf("lkl_sys_write %s: %s\n",
+			full_path, lkl_strerror(fd));
+	}
+
+	lkl_sys_close(fd);
+
+	return 0;
+}
+
+/* Configure sysctl parameters as the form of "key=value;key=value;..." */
+void lkl_sysctl_parse_write(const char *sysctls)
+{
+	char *saveptr = NULL, *token = NULL;
+	char *key = NULL, *value = NULL;
+	char strings[256];
+	int ret = 0;
+
+	strcpy(strings, sysctls);
+	for (token = strtok_r(strings, ";", &saveptr); token;
+	     token = strtok_r(NULL, ";", &saveptr)) {
+		key = strtok(token, "=");
+		value = strtok(NULL, "=");
+		ret = lkl_sysctl(key, value);
+		if (ret) {
+			lkl_printf("Failed to configure sysctl entries: %s\n",
+				   lkl_strerror(ret));
+			return;
+		}
+	}
+}
+#endif
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 14/25] lkl tools: host lib: filesystem helpers
  2020-03-30 14:45   ` Hajime Tazaki
@ 2020-03-30 14:45     ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Hajime Tazaki, Conrad Meyer, Michael Zimmermann, Yuan Liu

Add LKL applications APIs to mount and unmount a filesystem from a
disk added via lkl_disk_add().

Also add open/close/read directory wrappers on top of
lkl_sys_getdents64.

Cc: Conrad Meyer <cem@FreeBSD.org>
Cc: Michael Zimmermann <sigmaepsilon92@gmail.com>
Cc: Yuan Liu <liuyuan@google.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/lkl/include/lkl.h | 165 ++++++++++++++
 tools/lkl/lib/Build     |   1 +
 tools/lkl/lib/fs.c      | 471 ++++++++++++++++++++++++++++++++++++++++
 3 files changed, 637 insertions(+)
 create mode 100644 tools/lkl/lib/fs.c

diff --git a/tools/lkl/include/lkl.h b/tools/lkl/include/lkl.h
index 4b95d0ef8e5b..cdae92300320 100644
--- a/tools/lkl/include/lkl.h
+++ b/tools/lkl/include/lkl.h
@@ -350,6 +350,171 @@ const char *lkl_strerror(int err);
  */
 void lkl_perror(char *msg, int err);
 
+/**
+ * struct lkl_dev_blk_ops - block device host operations, defined in lkl_host.h.
+ */
+struct lkl_dev_blk_ops;
+
+/**
+ * lkl_disk - host disk handle
+ *
+ * @dev - a pointer to private information for this disk backend
+ * @fd - a POSIX file descriptor that can be used by preadv/pwritev
+ * @handle - an NT file handle that can be used by ReadFile/WriteFile
+ */
+struct lkl_disk {
+	void *dev;
+	union {
+		int fd;
+		void *handle;
+	};
+	struct lkl_dev_blk_ops *ops;
+};
+
+/**
+ * lkl_disk_add - add a new disk
+ *
+ * @disk - the host disk handle
+ * @returns a disk id (0 is valid) or a strictly negative value in case of error
+ */
+int lkl_disk_add(struct lkl_disk *disk);
+
+/**
+ * lkl_disk_remove - remove a disk
+ *
+ * This function makes a cleanup of the @disk's private information
+ * that was initialized by lkl_disk_add before.
+ *
+ * @disk - the host disk handle
+ */
+int lkl_disk_remove(struct lkl_disk disk);
+
+/**
+ * lkl_encode_dev_from_sysfs_blkdev - extract device id from sysfs
+ *
+ * This function returns the device id for the given sysfs dev node.
+ * The content of the node has to be in the form 'MAJOR:MINOR'.
+ * Also, this function expects an absolute path which means that sysfs
+ * already has to be mounted at the given path
+ *
+ * @sysfs_path - absolute path to the sysfs dev node
+ * @pdevid - pointer to memory where dev id will be returned
+ * @returns - 0 on success, a negative value on error
+ */
+int lkl_encode_dev_from_sysfs(const char *sysfs_path, uint32_t *pdevid);
+
+/**
+ * lkl_mount_dev - mount a disk
+ *
+ * This functions creates a device file for the given disk, creates a mount
+ * point and mounts the device over the mount point.
+ *
+ * @disk_id - the disk id identifying the disk to be mounted
+ * @part - disk partition or zero for full disk
+ * @fs_type - filesystem type
+ * @flags - mount flags
+ * @opts - additional filesystem specific mount options
+ * @mnt_str - a string that will be filled by this function with the path where
+ * the filesystem has been mounted
+ * @mnt_str_len - size of mnt_str
+ * @returns - 0 on success, a negative value on error
+ */
+long lkl_mount_dev(unsigned int disk_id, unsigned int part, const char *fs_type,
+		   int flags, const char *opts,
+		   char *mnt_str, unsigned int mnt_str_len);
+
+/**
+ * lkl_umount_dev - umount a disk
+ *
+ * This functions umounts the given disks and removes the device file and the
+ * mount point.
+ *
+ * @disk_id - the disk id identifying the disk to be mounted
+ * @part - disk partition or zero for full disk
+ * @flags - umount flags
+ * @timeout_ms - timeout to wait for the kernel to flush closed files so that
+ * umount can succeed
+ * @returns - 0 on success, a negative value on error
+ */
+long lkl_umount_dev(unsigned int disk_id, unsigned int part, int flags,
+		    long timeout_ms);
+
+/**
+ * lkl_umount_timeout - umount filesystem with timeout
+ *
+ * @path - the path to unmount
+ * @flags - umount flags
+ * @timeout_ms - timeout to wait for the kernel to flush closed files so that
+ * umount can succeed
+ * @returns - 0 on success, a negative value on error
+ */
+long lkl_umount_timeout(char *path, int flags, long timeout_ms);
+
+/**
+ * lkl_opendir - open a directory
+ *
+ * @path - directory path
+ * @err - pointer to store the error in case of failure
+ * @returns - a handle to be used when calling lkl_readdir
+ */
+struct lkl_dir *lkl_opendir(const char *path, int *err);
+
+/**
+ * lkl_fdopendir - open a directory
+ *
+ * @fd - file descriptor
+ * @err - pointer to store the error in case of failure
+ * @returns - a handle to be used when calling lkl_readdir
+ */
+struct lkl_dir *lkl_fdopendir(int fd, int *err);
+
+/**
+ * lkl_rewinddir - reset directory stream
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ */
+void lkl_rewinddir(struct lkl_dir *dir);
+
+/**
+ * lkl_closedir - close the directory
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ */
+int lkl_closedir(struct lkl_dir *dir);
+
+/**
+ * lkl_readdir - get the next available entry of the directory
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ * @returns - a lkl_dirent64 entry or NULL if the end of the directory stream is
+ * reached or if an error occurred; check lkl_errdir() to distinguish between
+ * errors or end of the directory stream
+ */
+struct lkl_linux_dirent64 *lkl_readdir(struct lkl_dir *dir);
+
+/**
+ * lkl_errdir - checks if an error occurred during the last lkl_readdir call
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ * @returns - 0 if no error occurred, or a negative value otherwise
+ */
+int lkl_errdir(struct lkl_dir *dir);
+
+/**
+ * lkl_dirfd - gets the file descriptor associated with the directory handle
+ *
+ * @dir - the directory handle as returned by lkl_opendir
+ * @returns - a positive value,which is the LKL file descriptor associated with
+ * the directory handle, or a negative value otherwise
+ */
+int lkl_dirfd(struct lkl_dir *dir);
+
+/**
+ * lkl_mount_fs - mount a file system type like proc, sys
+ * @fstype - file system type. e.g. proc, sys
+ * @returns - 0 on success. 1 if it's already mounted. negative on failure.
+ */
+int lkl_mount_fs(char *fstype);
 
 #ifdef __cplusplus
 }
diff --git a/tools/lkl/lib/Build b/tools/lkl/lib/Build
index 658bfa865b9c..4a444337b0e7 100644
--- a/tools/lkl/lib/Build
+++ b/tools/lkl/lib/Build
@@ -1,5 +1,6 @@
 CFLAGS_config.o += -I$(srctree)/tools/perf/pmu-events
 
+liblkl-y += fs.o
 liblkl-y += jmp_buf.o
 liblkl-y += utils.o
 liblkl-y += dbg.o
diff --git a/tools/lkl/lib/fs.c b/tools/lkl/lib/fs.c
new file mode 100644
index 000000000000..d51d928c14b6
--- /dev/null
+++ b/tools/lkl/lib/fs.c
@@ -0,0 +1,471 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdarg.h>
+#include <stdio.h>
+#include <string.h>
+#include <stdlib.h>
+#include <lkl_host.h>
+
+#define MAX_FSTYPE_LEN 50
+
+static struct lkl_disk *lkl_disks[16];
+
+int lkl_disk_add(struct lkl_disk *disk)
+{
+	int ret = -1;
+
+	switch (disk->backend) {
+	case BLK_BACKEND_UM:
+		ret = lkl_disk_um_add(disk, disk->dev);
+		break;
+	default:
+		break;
+	}
+
+	lkl_disks[ret] = disk;
+
+	return ret;
+}
+
+int lkl_disk_remove(struct lkl_disk disk)
+{
+	switch (disk.backend) {
+	case BLK_BACKEND_UM:
+	default:
+		break;
+	}
+
+	return 0;
+}
+
+int lkl_mount_fs(char *fstype)
+{
+	char dir[MAX_FSTYPE_LEN+2] = "/";
+	int flags = 0, ret = 0;
+
+	strncat(dir, fstype, MAX_FSTYPE_LEN);
+
+	/* Create with regular umask */
+	ret = lkl_sys_mkdir(dir, 0xff);
+	if (ret && ret != -LKL_EEXIST) {
+		lkl_perror("mount_fs mkdir", ret);
+		return ret;
+	}
+
+	/* We have no use for nonzero flags right now */
+	ret = lkl_sys_mount("none", dir, fstype, flags, NULL);
+	if (ret && ret != -LKL_EBUSY) {
+		lkl_sys_rmdir(dir);
+		return ret;
+	}
+
+	if (ret == -LKL_EBUSY)
+		return 1;
+	return 0;
+}
+
+static uint32_t new_encode_dev(unsigned int major, unsigned int minor)
+{
+	return (minor & 0xff) | (major << 8) | ((minor & ~0xff) << 12);
+}
+
+static int startswith(const char *str, const char *pre)
+{
+	return strncmp(pre, str, strlen(pre)) == 0;
+}
+
+static int get_node_with_prefix(const char *path, const char *prefix,
+				char *result, unsigned int result_len)
+{
+	struct lkl_dir *dir = NULL;
+	struct lkl_linux_dirent64 *dirent;
+	int ret;
+
+	dir = lkl_opendir(path, &ret);
+	if (!dir)
+		return ret;
+
+	ret = -LKL_ENOENT;
+
+	while ((dirent = lkl_readdir(dir))) {
+		if (startswith(dirent->d_name, prefix)) {
+			if (strlen(dirent->d_name) + 1 > result_len) {
+				ret = -LKL_ENOMEM;
+				break;
+			}
+			memcpy(result, dirent->d_name, strlen(dirent->d_name));
+			result[strlen(dirent->d_name)] = '\0';
+			ret = 0;
+			break;
+		}
+	}
+
+	lkl_closedir(dir);
+
+	return ret;
+}
+
+int lkl_encode_dev_from_sysfs(const char *sysfs_path, uint32_t *pdevid)
+{
+	int ret;
+	long fd;
+	int major, minor;
+	char buf[16] = { 0, };
+	char *bufptr;
+
+	fd = lkl_sys_open(sysfs_path, LKL_O_RDONLY, 0);
+	if (fd < 0)
+		return fd;
+
+	ret = lkl_sys_read(fd, buf, sizeof(buf));
+	if (ret < 0)
+		goto out_close;
+
+	if (ret == sizeof(buf)) {
+		ret = -LKL_ENOBUFS;
+		goto out_close;
+	}
+
+	bufptr = strchr(buf, ':');
+	if (bufptr == NULL) {
+		ret = -LKL_EINVAL;
+		goto out_close;
+	}
+	bufptr[0] = '\0';
+	bufptr++;
+
+	major = atoi(buf);
+	minor = atoi(bufptr);
+
+	*pdevid = new_encode_dev(major, minor);
+	ret = 0;
+
+out_close:
+	lkl_sys_close(fd);
+
+	return ret;
+}
+
+#define SYSFS_DEV_UMBLK_CMDLINE_PATH \
+	"/sysfs/devices/platform/uml-blkdev.%d"
+
+struct abuf {
+	char *mem, *ptr;
+	unsigned int len;
+};
+
+static int snprintf_append(struct abuf *buf, const char *fmt, ...)
+{
+	int ret;
+	va_list args;
+
+	if (!buf->ptr)
+		buf->ptr = buf->mem;
+
+	va_start(args, fmt);
+	ret = vsnprintf(buf->ptr, buf->len - (buf->ptr - buf->mem), fmt, args);
+	va_end(args);
+
+	if (ret < 0 || (ret >= (int)(buf->len - (buf->ptr - buf->mem))))
+		return -LKL_ENOMEM;
+
+	buf->ptr += ret;
+
+	return 0;
+}
+
+static int __lkl_get_blkdev(int disk_id, unsigned int part, uint32_t *pdevid,
+			    const char *sysfs_path_fmt, const char *drv_prefix,
+			    const char *disk_prefix)
+{
+	char sysfs_path[LKL_PATH_MAX];
+	char drv_name[LKL_PATH_MAX];
+	char disk_name[LKL_PATH_MAX];
+	struct abuf sysfs_path_buf = {
+		.mem = sysfs_path,
+		.len = sizeof(sysfs_path),
+	};
+	int ret;
+
+	if (disk_id < 0)
+		return -LKL_EINVAL;
+
+	ret = lkl_mount_fs("sysfs");
+	if (ret < 0)
+		return ret;
+
+	ret = snprintf_append(&sysfs_path_buf, sysfs_path_fmt, disk_id);
+	if (ret)
+		return ret;
+
+	ret = get_node_with_prefix(sysfs_path, drv_prefix, drv_name,
+				   sizeof(drv_name));
+	if (ret)
+		return ret;
+
+	ret = snprintf_append(&sysfs_path_buf, "/%s/block", drv_name);
+	if (ret)
+		return ret;
+
+	ret = get_node_with_prefix(sysfs_path, disk_prefix, disk_name,
+				   sizeof(disk_name));
+	if (ret)
+		return ret;
+
+	if (!part)
+		ret = snprintf_append(&sysfs_path_buf, "/%s/dev", disk_name);
+	else
+		ret = snprintf_append(&sysfs_path_buf, "/%s/%s%d/dev",
+				      disk_name, disk_name, part);
+	if (ret)
+		return ret;
+
+	return lkl_encode_dev_from_sysfs(sysfs_path, pdevid);
+}
+
+int lkl_get_blkdev(int disk_id, unsigned int part, uint32_t *pdevid)
+{
+	char *fmt;
+	struct lkl_disk *disk = lkl_disks[disk_id];
+
+	switch (disk->backend) {
+	case BLK_BACKEND_UM:
+		fmt = SYSFS_DEV_UMBLK_CMDLINE_PATH;
+		return __lkl_get_blkdev(disk_id, part, pdevid, fmt,
+					"", "ubd");
+	default:
+		break;
+	}
+
+	return -1;
+}
+
+long lkl_mount_dev(unsigned int disk_id, unsigned int part,
+		   const char *fs_type, int flags,
+		   const char *data, char *mnt_str, unsigned int mnt_str_len)
+{
+	char dev_str[] = { "/dev/xxxxxxxx" };
+	unsigned int dev;
+	int err;
+	char _data[4096]; /* FIXME: PAGE_SIZE is not exported by LKL */
+
+	if (mnt_str_len < sizeof(dev_str))
+		return -LKL_ENOMEM;
+
+	err = lkl_get_blkdev(disk_id, part, &dev);
+	if (err < 0)
+		return err;
+
+	snprintf(dev_str, sizeof(dev_str), "/dev/%08x", dev);
+	snprintf(mnt_str, mnt_str_len, "/mnt/%08x", dev);
+
+	err = lkl_sys_access("/dev", LKL_S_IRWXO);
+	if (err < 0) {
+		if (err == -LKL_ENOENT)
+			err = lkl_sys_mkdir("/dev", 0700);
+		if (err < 0)
+			return err;
+	}
+
+	err = lkl_sys_mknod(dev_str, LKL_S_IFBLK | 0600, dev);
+	if (err < 0)
+		return err;
+
+	err = lkl_sys_access("/mnt", LKL_S_IRWXO);
+	if (err < 0) {
+		if (err == -LKL_ENOENT)
+			err = lkl_sys_mkdir("/mnt", 0700);
+		if (err < 0)
+			return err;
+	}
+
+	err = lkl_sys_mkdir(mnt_str, 0700);
+	if (err < 0) {
+		lkl_sys_unlink(dev_str);
+		return err;
+	}
+
+	/* kernel always copies a full page */
+	if (data) {
+		strncpy(_data, data, sizeof(_data));
+		_data[sizeof(_data) - 1] = 0;
+	} else {
+		_data[0] = 0;
+	}
+
+	err = lkl_sys_mount(dev_str, mnt_str, (char *)fs_type, flags, _data);
+	if (err < 0) {
+		lkl_sys_unlink(dev_str);
+		lkl_sys_rmdir(mnt_str);
+		return err;
+	}
+
+	return 0;
+}
+
+long lkl_umount_timeout(char *path, int flags, long timeout_ms)
+{
+	long incr = 10000000; /* 10 ms */
+	struct lkl_timespec ts = {
+		.tv_sec = 0,
+		.tv_nsec = incr,
+	};
+	long err;
+
+	do {
+		err = lkl_sys_umount(path, flags);
+		if (err == -LKL_EBUSY) {
+			lkl_sys_nanosleep((struct __lkl__kernel_timespec *)&ts,
+					  NULL);
+			timeout_ms -= incr / 1000000;
+		}
+	} while (err == -LKL_EBUSY && timeout_ms > 0);
+
+	return err;
+}
+
+long lkl_umount_dev(unsigned int disk_id, unsigned int part, int flags,
+		    long timeout_ms)
+{
+	char dev_str[] = { "/dev/xxxxxxxx" };
+	char mnt_str[] = { "/mnt/xxxxxxxx" };
+	unsigned int dev;
+	int err;
+
+	err = lkl_get_blkdev(disk_id, part, &dev);
+	if (err < 0)
+		return err;
+
+	snprintf(dev_str, sizeof(dev_str), "/dev/%08x", dev);
+	snprintf(mnt_str, sizeof(mnt_str), "/mnt/%08x", dev);
+
+	err = lkl_umount_timeout(mnt_str, flags, timeout_ms);
+	if (err)
+		return err;
+
+	err = lkl_sys_unlink(dev_str);
+	if (err)
+		return err;
+
+	return lkl_sys_rmdir(mnt_str);
+}
+
+struct lkl_dir {
+	int fd;
+	char buf[1024];
+	char *pos;
+	int len;
+};
+
+static struct lkl_dir *lkl_dir_alloc(int *err)
+{
+	struct lkl_dir *dir = lkl_host_ops.mem_alloc(sizeof(struct lkl_dir));
+
+	if (!dir) {
+		*err = -LKL_ENOMEM;
+		return NULL;
+	}
+
+	dir->len = 0;
+	dir->pos = NULL;
+
+	return dir;
+}
+
+struct lkl_dir *lkl_opendir(const char *path, int *err)
+{
+	struct lkl_dir *dir = lkl_dir_alloc(err);
+
+	if (!dir) {
+		*err = -LKL_ENOMEM;
+		return NULL;
+	}
+
+	dir->fd = lkl_sys_open(path, LKL_O_RDONLY | LKL_O_DIRECTORY, 0);
+	if (dir->fd < 0) {
+		*err = dir->fd;
+		lkl_host_ops.mem_free(dir);
+		return NULL;
+	}
+
+	*err = 0;
+
+	return dir;
+}
+
+struct lkl_dir *lkl_fdopendir(int fd, int *err)
+{
+	struct lkl_dir *dir = lkl_dir_alloc(err);
+
+	if (!dir)
+		return NULL;
+
+	dir->fd = fd;
+
+	return dir;
+}
+
+void lkl_rewinddir(struct lkl_dir *dir)
+{
+	lkl_sys_lseek(dir->fd, 0, LKL_SEEK_SET);
+	dir->len = 0;
+	dir->pos = NULL;
+}
+
+int lkl_closedir(struct lkl_dir *dir)
+{
+	int ret;
+
+	ret = lkl_sys_close(dir->fd);
+	lkl_host_ops.mem_free(dir);
+
+	return ret;
+}
+
+struct lkl_linux_dirent64 *lkl_readdir(struct lkl_dir *dir)
+{
+	struct lkl_linux_dirent64 *de;
+
+	if (dir->len < 0)
+		return NULL;
+
+	if (!dir->pos || dir->pos - dir->buf >= dir->len)
+		goto read_buf;
+
+return_de:
+	de = (struct lkl_linux_dirent64 *)dir->pos;
+	dir->pos += de->d_reclen;
+
+	return de;
+
+read_buf:
+	dir->pos = NULL;
+	de = (struct lkl_linux_dirent64 *)dir->buf;
+	dir->len = lkl_sys_getdents64(dir->fd, de, sizeof(dir->buf));
+	if (dir->len <= 0)
+		return NULL;
+
+	dir->pos = dir->buf;
+	goto return_de;
+}
+
+int lkl_errdir(struct lkl_dir *dir)
+{
+	if (dir->len >= 0)
+		return 0;
+
+	return dir->len;
+}
+
+int lkl_dirfd(struct lkl_dir *dir)
+{
+	return dir->fd;
+}
+
+int lkl_set_fd_limit(unsigned int fd_limit)
+{
+	struct lkl_rlimit rlim = {
+		.rlim_cur = fd_limit,
+		.rlim_max = fd_limit,
+	};
+	return lkl_sys_setrlimit(LKL_RLIMIT_NOFILE, &rlim);
+}
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 14/25] lkl tools: host lib: filesystem helpers
@ 2020-03-30 14:45     ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: linux-arch, Conrad Meyer, Octavian Purdila, Akira Moroo,
	Yuan Liu, linux-kernel-library, Michael Zimmermann,
	Hajime Tazaki

Add LKL applications APIs to mount and unmount a filesystem from a
disk added via lkl_disk_add().

Also add open/close/read directory wrappers on top of
lkl_sys_getdents64.

Cc: Conrad Meyer <cem@FreeBSD.org>
Cc: Michael Zimmermann <sigmaepsilon92@gmail.com>
Cc: Yuan Liu <liuyuan@google.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/lkl/include/lkl.h | 165 ++++++++++++++
 tools/lkl/lib/Build     |   1 +
 tools/lkl/lib/fs.c      | 471 ++++++++++++++++++++++++++++++++++++++++
 3 files changed, 637 insertions(+)
 create mode 100644 tools/lkl/lib/fs.c

diff --git a/tools/lkl/include/lkl.h b/tools/lkl/include/lkl.h
index 4b95d0ef8e5b..cdae92300320 100644
--- a/tools/lkl/include/lkl.h
+++ b/tools/lkl/include/lkl.h
@@ -350,6 +350,171 @@ const char *lkl_strerror(int err);
  */
 void lkl_perror(char *msg, int err);
 
+/**
+ * struct lkl_dev_blk_ops - block device host operations, defined in lkl_host.h.
+ */
+struct lkl_dev_blk_ops;
+
+/**
+ * lkl_disk - host disk handle
+ *
+ * @dev - a pointer to private information for this disk backend
+ * @fd - a POSIX file descriptor that can be used by preadv/pwritev
+ * @handle - an NT file handle that can be used by ReadFile/WriteFile
+ */
+struct lkl_disk {
+	void *dev;
+	union {
+		int fd;
+		void *handle;
+	};
+	struct lkl_dev_blk_ops *ops;
+};
+
+/**
+ * lkl_disk_add - add a new disk
+ *
+ * @disk - the host disk handle
+ * @returns a disk id (0 is valid) or a strictly negative value in case of error
+ */
+int lkl_disk_add(struct lkl_disk *disk);
+
+/**
+ * lkl_disk_remove - remove a disk
+ *
+ * This function makes a cleanup of the @disk's private information
+ * that was initialized by lkl_disk_add before.
+ *
+ * @disk - the host disk handle
+ */
+int lkl_disk_remove(struct lkl_disk disk);
+
+/**
+ * lkl_encode_dev_from_sysfs_blkdev - extract device id from sysfs
+ *
+ * This function returns the device id for the given sysfs dev node.
+ * The content of the node has to be in the form 'MAJOR:MINOR'.
+ * Also, this function expects an absolute path which means that sysfs
+ * already has to be mounted at the given path
+ *
+ * @sysfs_path - absolute path to the sysfs dev node
+ * @pdevid - pointer to memory where dev id will be returned
+ * @returns - 0 on success, a negative value on error
+ */
+int lkl_encode_dev_from_sysfs(const char *sysfs_path, uint32_t *pdevid);
+
+/**
+ * lkl_mount_dev - mount a disk
+ *
+ * This functions creates a device file for the given disk, creates a mount
+ * point and mounts the device over the mount point.
+ *
+ * @disk_id - the disk id identifying the disk to be mounted
+ * @part - disk partition or zero for full disk
+ * @fs_type - filesystem type
+ * @flags - mount flags
+ * @opts - additional filesystem specific mount options
+ * @mnt_str - a string that will be filled by this function with the path where
+ * the filesystem has been mounted
+ * @mnt_str_len - size of mnt_str
+ * @returns - 0 on success, a negative value on error
+ */
+long lkl_mount_dev(unsigned int disk_id, unsigned int part, const char *fs_type,
+		   int flags, const char *opts,
+		   char *mnt_str, unsigned int mnt_str_len);
+
+/**
+ * lkl_umount_dev - umount a disk
+ *
+ * This functions umounts the given disks and removes the device file and the
+ * mount point.
+ *
+ * @disk_id - the disk id identifying the disk to be mounted
+ * @part - disk partition or zero for full disk
+ * @flags - umount flags
+ * @timeout_ms - timeout to wait for the kernel to flush closed files so that
+ * umount can succeed
+ * @returns - 0 on success, a negative value on error
+ */
+long lkl_umount_dev(unsigned int disk_id, unsigned int part, int flags,
+		    long timeout_ms);
+
+/**
+ * lkl_umount_timeout - umount filesystem with timeout
+ *
+ * @path - the path to unmount
+ * @flags - umount flags
+ * @timeout_ms - timeout to wait for the kernel to flush closed files so that
+ * umount can succeed
+ * @returns - 0 on success, a negative value on error
+ */
+long lkl_umount_timeout(char *path, int flags, long timeout_ms);
+
+/**
+ * lkl_opendir - open a directory
+ *
+ * @path - directory path
+ * @err - pointer to store the error in case of failure
+ * @returns - a handle to be used when calling lkl_readdir
+ */
+struct lkl_dir *lkl_opendir(const char *path, int *err);
+
+/**
+ * lkl_fdopendir - open a directory
+ *
+ * @fd - file descriptor
+ * @err - pointer to store the error in case of failure
+ * @returns - a handle to be used when calling lkl_readdir
+ */
+struct lkl_dir *lkl_fdopendir(int fd, int *err);
+
+/**
+ * lkl_rewinddir - reset directory stream
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ */
+void lkl_rewinddir(struct lkl_dir *dir);
+
+/**
+ * lkl_closedir - close the directory
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ */
+int lkl_closedir(struct lkl_dir *dir);
+
+/**
+ * lkl_readdir - get the next available entry of the directory
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ * @returns - a lkl_dirent64 entry or NULL if the end of the directory stream is
+ * reached or if an error occurred; check lkl_errdir() to distinguish between
+ * errors or end of the directory stream
+ */
+struct lkl_linux_dirent64 *lkl_readdir(struct lkl_dir *dir);
+
+/**
+ * lkl_errdir - checks if an error occurred during the last lkl_readdir call
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ * @returns - 0 if no error occurred, or a negative value otherwise
+ */
+int lkl_errdir(struct lkl_dir *dir);
+
+/**
+ * lkl_dirfd - gets the file descriptor associated with the directory handle
+ *
+ * @dir - the directory handle as returned by lkl_opendir
+ * @returns - a positive value,which is the LKL file descriptor associated with
+ * the directory handle, or a negative value otherwise
+ */
+int lkl_dirfd(struct lkl_dir *dir);
+
+/**
+ * lkl_mount_fs - mount a file system type like proc, sys
+ * @fstype - file system type. e.g. proc, sys
+ * @returns - 0 on success. 1 if it's already mounted. negative on failure.
+ */
+int lkl_mount_fs(char *fstype);
 
 #ifdef __cplusplus
 }
diff --git a/tools/lkl/lib/Build b/tools/lkl/lib/Build
index 658bfa865b9c..4a444337b0e7 100644
--- a/tools/lkl/lib/Build
+++ b/tools/lkl/lib/Build
@@ -1,5 +1,6 @@
 CFLAGS_config.o += -I$(srctree)/tools/perf/pmu-events
 
+liblkl-y += fs.o
 liblkl-y += jmp_buf.o
 liblkl-y += utils.o
 liblkl-y += dbg.o
diff --git a/tools/lkl/lib/fs.c b/tools/lkl/lib/fs.c
new file mode 100644
index 000000000000..d51d928c14b6
--- /dev/null
+++ b/tools/lkl/lib/fs.c
@@ -0,0 +1,471 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdarg.h>
+#include <stdio.h>
+#include <string.h>
+#include <stdlib.h>
+#include <lkl_host.h>
+
+#define MAX_FSTYPE_LEN 50
+
+static struct lkl_disk *lkl_disks[16];
+
+int lkl_disk_add(struct lkl_disk *disk)
+{
+	int ret = -1;
+
+	switch (disk->backend) {
+	case BLK_BACKEND_UM:
+		ret = lkl_disk_um_add(disk, disk->dev);
+		break;
+	default:
+		break;
+	}
+
+	lkl_disks[ret] = disk;
+
+	return ret;
+}
+
+int lkl_disk_remove(struct lkl_disk disk)
+{
+	switch (disk.backend) {
+	case BLK_BACKEND_UM:
+	default:
+		break;
+	}
+
+	return 0;
+}
+
+int lkl_mount_fs(char *fstype)
+{
+	char dir[MAX_FSTYPE_LEN+2] = "/";
+	int flags = 0, ret = 0;
+
+	strncat(dir, fstype, MAX_FSTYPE_LEN);
+
+	/* Create with regular umask */
+	ret = lkl_sys_mkdir(dir, 0xff);
+	if (ret && ret != -LKL_EEXIST) {
+		lkl_perror("mount_fs mkdir", ret);
+		return ret;
+	}
+
+	/* We have no use for nonzero flags right now */
+	ret = lkl_sys_mount("none", dir, fstype, flags, NULL);
+	if (ret && ret != -LKL_EBUSY) {
+		lkl_sys_rmdir(dir);
+		return ret;
+	}
+
+	if (ret == -LKL_EBUSY)
+		return 1;
+	return 0;
+}
+
+static uint32_t new_encode_dev(unsigned int major, unsigned int minor)
+{
+	return (minor & 0xff) | (major << 8) | ((minor & ~0xff) << 12);
+}
+
+static int startswith(const char *str, const char *pre)
+{
+	return strncmp(pre, str, strlen(pre)) == 0;
+}
+
+static int get_node_with_prefix(const char *path, const char *prefix,
+				char *result, unsigned int result_len)
+{
+	struct lkl_dir *dir = NULL;
+	struct lkl_linux_dirent64 *dirent;
+	int ret;
+
+	dir = lkl_opendir(path, &ret);
+	if (!dir)
+		return ret;
+
+	ret = -LKL_ENOENT;
+
+	while ((dirent = lkl_readdir(dir))) {
+		if (startswith(dirent->d_name, prefix)) {
+			if (strlen(dirent->d_name) + 1 > result_len) {
+				ret = -LKL_ENOMEM;
+				break;
+			}
+			memcpy(result, dirent->d_name, strlen(dirent->d_name));
+			result[strlen(dirent->d_name)] = '\0';
+			ret = 0;
+			break;
+		}
+	}
+
+	lkl_closedir(dir);
+
+	return ret;
+}
+
+int lkl_encode_dev_from_sysfs(const char *sysfs_path, uint32_t *pdevid)
+{
+	int ret;
+	long fd;
+	int major, minor;
+	char buf[16] = { 0, };
+	char *bufptr;
+
+	fd = lkl_sys_open(sysfs_path, LKL_O_RDONLY, 0);
+	if (fd < 0)
+		return fd;
+
+	ret = lkl_sys_read(fd, buf, sizeof(buf));
+	if (ret < 0)
+		goto out_close;
+
+	if (ret == sizeof(buf)) {
+		ret = -LKL_ENOBUFS;
+		goto out_close;
+	}
+
+	bufptr = strchr(buf, ':');
+	if (bufptr == NULL) {
+		ret = -LKL_EINVAL;
+		goto out_close;
+	}
+	bufptr[0] = '\0';
+	bufptr++;
+
+	major = atoi(buf);
+	minor = atoi(bufptr);
+
+	*pdevid = new_encode_dev(major, minor);
+	ret = 0;
+
+out_close:
+	lkl_sys_close(fd);
+
+	return ret;
+}
+
+#define SYSFS_DEV_UMBLK_CMDLINE_PATH \
+	"/sysfs/devices/platform/uml-blkdev.%d"
+
+struct abuf {
+	char *mem, *ptr;
+	unsigned int len;
+};
+
+static int snprintf_append(struct abuf *buf, const char *fmt, ...)
+{
+	int ret;
+	va_list args;
+
+	if (!buf->ptr)
+		buf->ptr = buf->mem;
+
+	va_start(args, fmt);
+	ret = vsnprintf(buf->ptr, buf->len - (buf->ptr - buf->mem), fmt, args);
+	va_end(args);
+
+	if (ret < 0 || (ret >= (int)(buf->len - (buf->ptr - buf->mem))))
+		return -LKL_ENOMEM;
+
+	buf->ptr += ret;
+
+	return 0;
+}
+
+static int __lkl_get_blkdev(int disk_id, unsigned int part, uint32_t *pdevid,
+			    const char *sysfs_path_fmt, const char *drv_prefix,
+			    const char *disk_prefix)
+{
+	char sysfs_path[LKL_PATH_MAX];
+	char drv_name[LKL_PATH_MAX];
+	char disk_name[LKL_PATH_MAX];
+	struct abuf sysfs_path_buf = {
+		.mem = sysfs_path,
+		.len = sizeof(sysfs_path),
+	};
+	int ret;
+
+	if (disk_id < 0)
+		return -LKL_EINVAL;
+
+	ret = lkl_mount_fs("sysfs");
+	if (ret < 0)
+		return ret;
+
+	ret = snprintf_append(&sysfs_path_buf, sysfs_path_fmt, disk_id);
+	if (ret)
+		return ret;
+
+	ret = get_node_with_prefix(sysfs_path, drv_prefix, drv_name,
+				   sizeof(drv_name));
+	if (ret)
+		return ret;
+
+	ret = snprintf_append(&sysfs_path_buf, "/%s/block", drv_name);
+	if (ret)
+		return ret;
+
+	ret = get_node_with_prefix(sysfs_path, disk_prefix, disk_name,
+				   sizeof(disk_name));
+	if (ret)
+		return ret;
+
+	if (!part)
+		ret = snprintf_append(&sysfs_path_buf, "/%s/dev", disk_name);
+	else
+		ret = snprintf_append(&sysfs_path_buf, "/%s/%s%d/dev",
+				      disk_name, disk_name, part);
+	if (ret)
+		return ret;
+
+	return lkl_encode_dev_from_sysfs(sysfs_path, pdevid);
+}
+
+int lkl_get_blkdev(int disk_id, unsigned int part, uint32_t *pdevid)
+{
+	char *fmt;
+	struct lkl_disk *disk = lkl_disks[disk_id];
+
+	switch (disk->backend) {
+	case BLK_BACKEND_UM:
+		fmt = SYSFS_DEV_UMBLK_CMDLINE_PATH;
+		return __lkl_get_blkdev(disk_id, part, pdevid, fmt,
+					"", "ubd");
+	default:
+		break;
+	}
+
+	return -1;
+}
+
+long lkl_mount_dev(unsigned int disk_id, unsigned int part,
+		   const char *fs_type, int flags,
+		   const char *data, char *mnt_str, unsigned int mnt_str_len)
+{
+	char dev_str[] = { "/dev/xxxxxxxx" };
+	unsigned int dev;
+	int err;
+	char _data[4096]; /* FIXME: PAGE_SIZE is not exported by LKL */
+
+	if (mnt_str_len < sizeof(dev_str))
+		return -LKL_ENOMEM;
+
+	err = lkl_get_blkdev(disk_id, part, &dev);
+	if (err < 0)
+		return err;
+
+	snprintf(dev_str, sizeof(dev_str), "/dev/%08x", dev);
+	snprintf(mnt_str, mnt_str_len, "/mnt/%08x", dev);
+
+	err = lkl_sys_access("/dev", LKL_S_IRWXO);
+	if (err < 0) {
+		if (err == -LKL_ENOENT)
+			err = lkl_sys_mkdir("/dev", 0700);
+		if (err < 0)
+			return err;
+	}
+
+	err = lkl_sys_mknod(dev_str, LKL_S_IFBLK | 0600, dev);
+	if (err < 0)
+		return err;
+
+	err = lkl_sys_access("/mnt", LKL_S_IRWXO);
+	if (err < 0) {
+		if (err == -LKL_ENOENT)
+			err = lkl_sys_mkdir("/mnt", 0700);
+		if (err < 0)
+			return err;
+	}
+
+	err = lkl_sys_mkdir(mnt_str, 0700);
+	if (err < 0) {
+		lkl_sys_unlink(dev_str);
+		return err;
+	}
+
+	/* kernel always copies a full page */
+	if (data) {
+		strncpy(_data, data, sizeof(_data));
+		_data[sizeof(_data) - 1] = 0;
+	} else {
+		_data[0] = 0;
+	}
+
+	err = lkl_sys_mount(dev_str, mnt_str, (char *)fs_type, flags, _data);
+	if (err < 0) {
+		lkl_sys_unlink(dev_str);
+		lkl_sys_rmdir(mnt_str);
+		return err;
+	}
+
+	return 0;
+}
+
+long lkl_umount_timeout(char *path, int flags, long timeout_ms)
+{
+	long incr = 10000000; /* 10 ms */
+	struct lkl_timespec ts = {
+		.tv_sec = 0,
+		.tv_nsec = incr,
+	};
+	long err;
+
+	do {
+		err = lkl_sys_umount(path, flags);
+		if (err == -LKL_EBUSY) {
+			lkl_sys_nanosleep((struct __lkl__kernel_timespec *)&ts,
+					  NULL);
+			timeout_ms -= incr / 1000000;
+		}
+	} while (err == -LKL_EBUSY && timeout_ms > 0);
+
+	return err;
+}
+
+long lkl_umount_dev(unsigned int disk_id, unsigned int part, int flags,
+		    long timeout_ms)
+{
+	char dev_str[] = { "/dev/xxxxxxxx" };
+	char mnt_str[] = { "/mnt/xxxxxxxx" };
+	unsigned int dev;
+	int err;
+
+	err = lkl_get_blkdev(disk_id, part, &dev);
+	if (err < 0)
+		return err;
+
+	snprintf(dev_str, sizeof(dev_str), "/dev/%08x", dev);
+	snprintf(mnt_str, sizeof(mnt_str), "/mnt/%08x", dev);
+
+	err = lkl_umount_timeout(mnt_str, flags, timeout_ms);
+	if (err)
+		return err;
+
+	err = lkl_sys_unlink(dev_str);
+	if (err)
+		return err;
+
+	return lkl_sys_rmdir(mnt_str);
+}
+
+struct lkl_dir {
+	int fd;
+	char buf[1024];
+	char *pos;
+	int len;
+};
+
+static struct lkl_dir *lkl_dir_alloc(int *err)
+{
+	struct lkl_dir *dir = lkl_host_ops.mem_alloc(sizeof(struct lkl_dir));
+
+	if (!dir) {
+		*err = -LKL_ENOMEM;
+		return NULL;
+	}
+
+	dir->len = 0;
+	dir->pos = NULL;
+
+	return dir;
+}
+
+struct lkl_dir *lkl_opendir(const char *path, int *err)
+{
+	struct lkl_dir *dir = lkl_dir_alloc(err);
+
+	if (!dir) {
+		*err = -LKL_ENOMEM;
+		return NULL;
+	}
+
+	dir->fd = lkl_sys_open(path, LKL_O_RDONLY | LKL_O_DIRECTORY, 0);
+	if (dir->fd < 0) {
+		*err = dir->fd;
+		lkl_host_ops.mem_free(dir);
+		return NULL;
+	}
+
+	*err = 0;
+
+	return dir;
+}
+
+struct lkl_dir *lkl_fdopendir(int fd, int *err)
+{
+	struct lkl_dir *dir = lkl_dir_alloc(err);
+
+	if (!dir)
+		return NULL;
+
+	dir->fd = fd;
+
+	return dir;
+}
+
+void lkl_rewinddir(struct lkl_dir *dir)
+{
+	lkl_sys_lseek(dir->fd, 0, LKL_SEEK_SET);
+	dir->len = 0;
+	dir->pos = NULL;
+}
+
+int lkl_closedir(struct lkl_dir *dir)
+{
+	int ret;
+
+	ret = lkl_sys_close(dir->fd);
+	lkl_host_ops.mem_free(dir);
+
+	return ret;
+}
+
+struct lkl_linux_dirent64 *lkl_readdir(struct lkl_dir *dir)
+{
+	struct lkl_linux_dirent64 *de;
+
+	if (dir->len < 0)
+		return NULL;
+
+	if (!dir->pos || dir->pos - dir->buf >= dir->len)
+		goto read_buf;
+
+return_de:
+	de = (struct lkl_linux_dirent64 *)dir->pos;
+	dir->pos += de->d_reclen;
+
+	return de;
+
+read_buf:
+	dir->pos = NULL;
+	de = (struct lkl_linux_dirent64 *)dir->buf;
+	dir->len = lkl_sys_getdents64(dir->fd, de, sizeof(dir->buf));
+	if (dir->len <= 0)
+		return NULL;
+
+	dir->pos = dir->buf;
+	goto return_de;
+}
+
+int lkl_errdir(struct lkl_dir *dir)
+{
+	if (dir->len >= 0)
+		return 0;
+
+	return dir->len;
+}
+
+int lkl_dirfd(struct lkl_dir *dir)
+{
+	return dir->fd;
+}
+
+int lkl_set_fd_limit(unsigned int fd_limit)
+{
+	struct lkl_rlimit rlim = {
+		.rlim_cur = fd_limit,
+		.rlim_max = fd_limit,
+	};
+	return lkl_sys_setrlimit(LKL_RLIMIT_NOFILE, &rlim);
+}
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 15/25] lkl tools: host lib: networking helpers
  2020-03-30 14:45   ` Hajime Tazaki
@ 2020-03-30 14:45     ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Hajime Tazaki

This commit adds LKL application APIs to control network related
resources of kernel via LKL syscall, ioctl, and via netlink messages.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 tools/lkl/include/lkl.h      | 241 ++++++++++
 tools/lkl/include/lkl_host.h |  85 ++++
 tools/lkl/lib/Build          |   1 +
 tools/lkl/lib/net.c          | 826 +++++++++++++++++++++++++++++++++++
 4 files changed, 1153 insertions(+)
 create mode 100644 tools/lkl/lib/net.c

diff --git a/tools/lkl/include/lkl.h b/tools/lkl/include/lkl.h
index cdae92300320..1f4291ad9455 100644
--- a/tools/lkl/include/lkl.h
+++ b/tools/lkl/include/lkl.h
@@ -516,6 +516,247 @@ int lkl_dirfd(struct lkl_dir *dir);
  */
 int lkl_mount_fs(char *fstype);
 
+/**
+ * lkl_if_up - activate network interface
+ *
+ * @ifindex - the ifindex of the interface
+ * @returns - return 0 if no error: otherwise negative value returns
+ */
+int lkl_if_up(int ifindex);
+
+/**
+ * lkl_if_down - deactivate network interface
+ *
+ * @ifindex - the ifindex of the interface
+ * @returns - return 0 if no error: otherwise negative value returns
+ */
+int lkl_if_down(int ifindex);
+
+/**
+ * lkl_if_set_mtu - set MTU on interface
+ *
+ * @ifindex - the ifindex of the interface
+ * @mtu - the requested MTU size
+ * @returns - return 0 if no error: otherwise negative value returns
+ */
+int lkl_if_set_mtu(int ifindex, int mtu);
+
+/**
+ * lkl_if_set_ipv4 - set IPv4 address on interface
+ *
+ * @ifindex - the ifindex of the interface
+ * @addr - 4-byte IP address (i.e., struct in_addr)
+ * @netmask_len - prefix length of the @addr
+ * @returns - return 0 if no error: otherwise negative value returns
+ */
+int lkl_if_set_ipv4(int ifindex, unsigned int addr, unsigned int netmask_len);
+
+/**
+ * lkl_set_ipv4_gateway - add an IPv4 default route
+ *
+ * @addr - 4-byte IP address of the gateway (i.e., struct in_addr)
+ * @returns - return 0 if no error: otherwise negative value returns
+ */
+int lkl_set_ipv4_gateway(unsigned int addr);
+
+/**
+ * lkl_if_set_ipv4_gateway - add an IPv4 default route in rule table
+ *
+ * @ifindex - the ifindex of the interface, used for tableid calculation
+ * @addr - 4-byte IP address of the interface
+ * @netmask_len - prefix length of the @addr
+ * @gw_addr - 4-byte IP address of the gateway
+ * @returns - return 0 if no error: otherwise negative value returns
+ */
+int lkl_if_set_ipv4_gateway(int ifindex, unsigned int addr,
+		unsigned int netmask_len, unsigned int gw_addr);
+
+/**
+ * lkl_if_set_ipv6 - set IPv6 address on interface
+ * must be called after interface is up.
+ *
+ * @ifindex - the ifindex of the interface
+ * @addr - 16-byte IPv6 address (i.e., struct in6_addr)
+ * @netprefix_len - prefix length of the @addr
+ * @returns - return 0 if no error: otherwise negative value returns
+ */
+int lkl_if_set_ipv6(int ifindex, void *addr, unsigned int netprefix_len);
+
+/**
+ * lkl_set_ipv6_gateway - add an IPv6 default route
+ *
+ * @addr - 16-byte IPv6 address of the gateway (i.e., struct in6_addr)
+ * @returns - return 0 if no error: otherwise negative value returns
+ */
+int lkl_set_ipv6_gateway(void *addr);
+
+/**
+ * lkl_if_set_ipv6_gateway - add an IPv6 default route in rule table
+ *
+ * @ifindex - the ifindex of the interface, used for tableid calculation
+ * @addr - 16-byte IP address of the interface
+ * @netmask_len - prefix length of the @addr
+ * @gw_addr - 16-byte IP address of the gateway (i.e., struct in_addr)
+ * @returns - return 0 if no error: otherwise negative value returns
+ */
+int lkl_if_set_ipv6_gateway(int ifindex, void *addr,
+		unsigned int netmask_len, void *gw_addr);
+
+/**
+ * lkl_ifname_to_ifindex - obtain ifindex of an interface by name
+ *
+ * @name - string of an interface
+ * @returns - return an integer of ifindex if no error
+ */
+int lkl_ifname_to_ifindex(const char *name);
+
+/**
+ * lkl_netdev_get_ifindex - retrieve the interface index for a given network
+ * device id
+ *
+ * @id - the network device id
+ * @returns the interface index or a stricly negative value in case of error
+ */
+int lkl_netdev_get_ifindex(int id);
+
+/**
+ * lkl_add_neighbor - add a permanent arp entry
+ * @ifindex - the ifindex of the interface
+ * @af - address family of the ip address. Must be LKL_AF_INET or LKL_AF_INET6
+ * @ip - ip address of the entry in network byte order
+ * @mac - mac address of the entry
+ */
+int lkl_add_neighbor(int ifindex, int af, void *addr, void *mac);
+
+/**
+ * lkl_if_add_ip - add an ip address
+ * @ifindex - the ifindex of the interface
+ * @af - address family of the ip address. Must be LKL_AF_INET or LKL_AF_INET6
+ * @addr - ip address of the entry in network byte order
+ * @netprefix_len - prefix length of the @addr
+ */
+int lkl_if_add_ip(int ifindex, int af, void *addr, unsigned int netprefix_len);
+
+/**
+ * lkl_if_del_ip - add an ip address
+ * @ifindex - the ifindex of the interface
+ * @af - address family of the ip address. Must be LKL_AF_INET or LKL_AF_INET6
+ * @addr - ip address of the entry in network byte order
+ * @netprefix_len - prefix length of the @addr
+ */
+int lkl_if_del_ip(int ifindex, int af, void *addr, unsigned int netprefix_len);
+
+/**
+ * lkl_add_gateway - add a gateway
+ * @af - address family of the ip address. Must be LKL_AF_INET or LKL_AF_INET6
+ * @gwaddr - 4-byte IP address of the gateway (i.e., struct in_addr)
+ */
+int lkl_add_gateway(int af, void *gwaddr);
+
+/**
+ * XXX Should I use OIF selector?
+ * temporary table idx = ifindex * 2 + 0 <- ipv4
+ * temporary table idx = ifindex * 2 + 1 <- ipv6
+ */
+/**
+ * lkl_if_add_rule_from_addr - create an ip rule table with "from" selector
+ * @ifindex - the ifindex of the interface, used for table id calculation
+ * @af - address family of the ip address. Must be LKL_AF_INET or LKL_AF_INET6
+ * @saddr - network byte order ip address, "from" selector address of this rule
+ */
+int lkl_if_add_rule_from_saddr(int ifindex, int af, void *saddr);
+
+/**
+ * lkl_if_add_gateway - add gateway to rule table
+ * @ifindex - the ifindex of the interface, used for table id calculation
+ * @af - address family of the ip address. Must be LKL_AF_INET or LKL_AF_INET6
+ * @gwaddr - 4-byte IP address of the gateway (i.e., struct in_addr)
+ */
+int lkl_if_add_gateway(int ifindex, int af, void *gwaddr);
+
+/**
+ * lkl_if_add_linklocal - add linklocal route to rule table
+ * @ifindex - the ifindex of the interface, used for table id calculation
+ * @af - address family of the ip address. Must be LKL_AF_INET or LKL_AF_INET6
+ * @addr - ip address of the entry in network byte order
+ * @netprefix_len - prefix length of the @addr
+ */
+int lkl_if_add_linklocal(int ifindex, int af,  void *addr, int netprefix_len);
+
+/**
+ * lkl_if_wait_ipv6_dad - wait for DAD to be done for a ipv6 address
+ * must be called after interface is up
+ *
+ * @ifindex - the ifindex of the interface
+ * @addr - ip address of the entry in network byte order
+ */
+int lkl_if_wait_ipv6_dad(int ifindex, void *addr);
+
+/**
+ * lkl_set_fd_limit - set the maximum number of file descriptors allowed
+ * @fd_limit - fd max limit
+ */
+int lkl_set_fd_limit(unsigned int fd_limit);
+
+/**
+ * lkl_qdisc_add - set qdisc rule onto an interface
+ *
+ * @ifindex - the ifindex of the interface
+ * @root - the name of root class (e.g., "root");
+ * @type - the type of qdisc (e.g., "fq")
+ */
+int lkl_qdisc_add(int ifindex, const char *root, const char *type);
+
+/**
+ * lkl_qdisc_parse_add - Add a qdisc entry for an interface with strings
+ *
+ * @ifindex - the ifindex of the interface
+ * @entries - strings of qdisc configurations in the form of
+ *            "root|type;root|type;..."
+ */
+void lkl_qdisc_parse_add(int ifindex, const char *entries);
+
+/**
+ * lkl_netdev - host network device handle, defined in lkl_host.h.
+ */
+struct lkl_netdev;
+
+/**
+ * lkl_netdev_args - arguments to lkl_netdev_add
+ * @mac - optional MAC address for the device
+ * @offload - offload bits for the device
+ */
+struct lkl_netdev_args {
+	void *mac;
+	unsigned int offload;
+};
+
+/*
+ * lkl_register_dbg_handler- register a signal handler that loads a debug lib.
+ *
+ * The signal handler is triggered by Ctrl-Z. It creates a new pthread which
+ * call dbg_entrance().
+ *
+ * If you run the program from shell script, make sure you ignore SIGTSTP by
+ * "trap '' TSTP" in the shell script.
+ */
+void lkl_register_dbg_handler(void);
+
+/**
+ * lkl_sysctl - write a sysctl value
+ *
+ * @path - the path to an sysctl entry (e.g., "net.ipv4.tcp_wmem");
+ * @value - the value of the sysctl (e.g., "4096 87380 2147483647")
+ */
+int lkl_sysctl(const char *path, const char *value);
+
+/**
+ * lkl_sysctl_parse_write - Configure sysctl parameters with strings
+ *
+ * @sysctls - Configure sysctl parameters as the form of "key=value;..."
+ */
+void lkl_sysctl_parse_write(const char *sysctls);
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/tools/lkl/include/lkl_host.h b/tools/lkl/include/lkl_host.h
index 85e80eb4ad0d..4e6d6e031498 100644
--- a/tools/lkl/include/lkl_host.h
+++ b/tools/lkl/include/lkl_host.h
@@ -18,6 +18,91 @@ extern struct lkl_host_operations lkl_host_ops;
  */
 int lkl_printf(const char *fmt, ...);
 
+#ifdef LKL_HOST_CONFIG_POSIX
+#include <sys/uio.h>
+#else
+struct iovec {
+	void *iov_base;
+	size_t iov_len;
+};
+#endif
+
+struct lkl_netdev {
+	struct lkl_dev_net_ops *ops;
+	int id;
+	uint8_t has_vnet_hdr: 1;
+};
+
+/**
+ * struct lkl_dev_net_ops - network device host operations
+ */
+struct lkl_dev_net_ops {
+	/**
+	 * @tx: writes a L2 packet into the net device
+	 *
+	 * The data buffer can only hold 0 or 1 complete packets.
+	 *
+	 * @nd - pointer to the network device;
+	 * @iov - pointer to the buffer vector;
+	 * @cnt - # of vectors in iov.
+	 *
+	 * @returns number of bytes transmitted
+	 */
+	int (*tx)(struct lkl_netdev *nd, struct iovec *iov, int cnt);
+
+	/**
+	 * @rx: reads a packet from the net device.
+	 *
+	 * It must only read one complete packet if present.
+	 *
+	 * If the buffer is too small for the packet, the implementation may
+	 * decide to drop it or trim it.
+	 *
+	 * @nd - pointer to the network device
+	 * @iov - pointer to the buffer vector to store the packet
+	 * @cnt - # of vectors in iov.
+	 *
+	 * @returns number of bytes read for success or < 0 if error
+	 */
+	int (*rx)(struct lkl_netdev *nd, struct iovec *iov, int cnt);
+
+#define LKL_DEV_NET_POLL_RX		1
+#define LKL_DEV_NET_POLL_TX		2
+#define LKL_DEV_NET_POLL_HUP		4
+
+	/**
+	 * @poll: polls a net device
+	 *
+	 * Supports the following events: LKL_DEV_NET_POLL_RX
+	 * (readable), LKL_DEV_NET_POLL_TX (writable) or
+	 * LKL_DEV_NET_POLL_HUP (the close operations has been issued
+	 * and we need to clean up). Blocks until one event is
+	 * available.
+	 *
+	 * @nd - pointer to the network device
+	 *
+	 * @returns - LKL_DEV_NET_POLL_RX, LKL_DEV_NET_POLL_TX,
+	 * LKL_DEV_NET_POLL_HUP or a negative value for errors
+	 */
+	int (*poll)(struct lkl_netdev *nd);
+
+	/**
+	 * @poll_hup: make poll wakeup and return LKL_DEV_NET_POLL_HUP
+	 *
+	 * @nd - pointer to the network device
+	 */
+	void (*poll_hup)(struct lkl_netdev *nd);
+
+	/**
+	 * @free: frees a network device
+	 *
+	 * Implementation must release its resources and free the network device
+	 * structure.
+	 *
+	 * @nd - pointer to the network device
+	 */
+	void (*free)(struct lkl_netdev *nd);
+};
 
 #ifdef __cplusplus
 }
diff --git a/tools/lkl/lib/Build b/tools/lkl/lib/Build
index 4a444337b0e7..dba530424e4a 100644
--- a/tools/lkl/lib/Build
+++ b/tools/lkl/lib/Build
@@ -1,6 +1,7 @@
 CFLAGS_config.o += -I$(srctree)/tools/perf/pmu-events
 
 liblkl-y += fs.o
+liblkl-y += net.o
 liblkl-y += jmp_buf.o
 liblkl-y += utils.o
 liblkl-y += dbg.o
diff --git a/tools/lkl/lib/net.c b/tools/lkl/lib/net.c
new file mode 100644
index 000000000000..cf6894c35e46
--- /dev/null
+++ b/tools/lkl/lib/net.c
@@ -0,0 +1,826 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <string.h>
+#include <stdio.h>
+#include "endian.h"
+#include <lkl_host.h>
+
+#ifdef __MINGW32__
+#include <ws2tcpip.h>
+
+int lkl_inet_pton(int af, const char *src, void *dst)
+{
+	struct addrinfo hint, *res = NULL;
+	int err;
+
+	memset(&hint, 0, sizeof(struct addrinfo));
+
+	hint.ai_family = af;
+	hint.ai_flags = AI_NUMERICHOST;
+
+	err = getaddrinfo(src, NULL, &hint, &res);
+	if (err)
+		return 0;
+
+	switch (af) {
+	case AF_INET:
+		*(struct in_addr *)dst =
+			((struct sockaddr_in *)&res->ai_addr)->sin_addr;
+		break;
+	case AF_INET6:
+		*(struct in6_addr *)dst =
+			((struct sockaddr_in6 *)&res->ai_addr)->sin6_addr;
+		break;
+	default:
+		freeaddrinfo(res);
+		return 0;
+	}
+
+	freeaddrinfo(res);
+	return 1;
+}
+#endif
+
+static inline void set_sockaddr(struct lkl_sockaddr_in *sin, unsigned int addr,
+				unsigned short port)
+{
+	sin->sin_family = LKL_AF_INET;
+	sin->sin_addr.lkl_s_addr = addr;
+	sin->sin_port = port;
+}
+
+static inline int ifindex_to_name(int sock, struct lkl_ifreq *ifr, int ifindex)
+{
+	ifr->lkl_ifr_ifindex = ifindex;
+	return lkl_sys_ioctl(sock, LKL_SIOCGIFNAME, (long)ifr);
+}
+
+int lkl_ifname_to_ifindex(const char *name)
+{
+	struct lkl_ifreq ifr;
+	int fd, ret;
+
+	fd = lkl_sys_socket(LKL_AF_INET, LKL_SOCK_DGRAM, 0);
+	if (fd < 0)
+		return fd;
+
+	if (strlen(name) >= LKL_IFNAMSIZ)
+		return -LKL_ENAMETOOLONG;
+
+	strcpy(ifr.lkl_ifr_name, name);
+
+	ret = lkl_sys_ioctl(fd, LKL_SIOCGIFINDEX, (long)&ifr);
+	if (ret < 0)
+		return ret;
+
+	return ifr.lkl_ifr_ifindex;
+}
+
+int lkl_if_up(int ifindex)
+{
+	struct lkl_ifreq ifr;
+	int err, sock = lkl_sys_socket(LKL_AF_INET, LKL_SOCK_DGRAM, 0);
+
+	if (sock < 0)
+		return sock;
+	err = ifindex_to_name(sock, &ifr, ifindex);
+	if (err < 0)
+		return err;
+
+	err = lkl_sys_ioctl(sock, LKL_SIOCGIFFLAGS, (long)&ifr);
+	if (!err) {
+		ifr.lkl_ifr_flags |= LKL_IFF_UP;
+		err = lkl_sys_ioctl(sock, LKL_SIOCSIFFLAGS, (long)&ifr);
+	}
+
+	lkl_sys_close(sock);
+
+	return err;
+}
+
+int lkl_if_down(int ifindex)
+{
+	struct lkl_ifreq ifr;
+	int err, sock;
+
+	sock = lkl_sys_socket(LKL_AF_INET, LKL_SOCK_DGRAM, 0);
+	if (sock < 0)
+		return sock;
+
+	err = ifindex_to_name(sock, &ifr, ifindex);
+	if (err < 0)
+		return err;
+
+	err = lkl_sys_ioctl(sock, LKL_SIOCGIFFLAGS, (long)&ifr);
+	if (!err) {
+		ifr.lkl_ifr_flags &= ~LKL_IFF_UP;
+		err = lkl_sys_ioctl(sock, LKL_SIOCSIFFLAGS, (long)&ifr);
+	}
+
+	lkl_sys_close(sock);
+
+	return err;
+}
+
+int lkl_if_set_mtu(int ifindex, int mtu)
+{
+	struct lkl_ifreq ifr;
+	int err, sock;
+
+	sock = lkl_sys_socket(LKL_AF_INET, LKL_SOCK_DGRAM, 0);
+	if (sock < 0)
+		return sock;
+
+	err = ifindex_to_name(sock, &ifr, ifindex);
+	if (err < 0)
+		return err;
+
+	ifr.lkl_ifr_mtu = mtu;
+
+	err = lkl_sys_ioctl(sock, LKL_SIOCSIFMTU, (long)&ifr);
+
+	lkl_sys_close(sock);
+
+	return err;
+}
+
+int lkl_if_set_ipv4(int ifindex, unsigned int addr, unsigned int netmask_len)
+{
+	return lkl_if_add_ip(ifindex, LKL_AF_INET, &addr, netmask_len);
+}
+
+int lkl_if_set_ipv4_gateway(int ifindex, unsigned int src_addr,
+		unsigned int src_masklen, unsigned int via_addr)
+{
+	int err;
+
+	err = lkl_if_add_rule_from_saddr(ifindex, LKL_AF_INET, &src_addr);
+	if (err)
+		return err;
+	err = lkl_if_add_linklocal(ifindex, LKL_AF_INET,
+					&src_addr, src_masklen);
+	if (err)
+		return err;
+	return lkl_if_add_gateway(ifindex, LKL_AF_INET, &via_addr);
+}
+
+int lkl_set_ipv4_gateway(unsigned int addr)
+{
+	return lkl_add_gateway(LKL_AF_INET, &addr);
+}
+
+int lkl_netdev_get_ifindex(int id)
+{
+	struct lkl_ifreq ifr;
+	int sock, ret;
+
+	sock = lkl_sys_socket(LKL_AF_INET, LKL_SOCK_DGRAM, 0);
+	if (sock < 0)
+		return sock;
+
+	snprintf(ifr.lkl_ifr_name, sizeof(ifr.lkl_ifr_name), "eth%d", id);
+	ret = lkl_sys_ioctl(sock, LKL_SIOCGIFINDEX, (long)&ifr);
+	lkl_sys_close(sock);
+
+	return ret < 0 ? ret : ifr.lkl_ifr_ifindex;
+}
+
+static int netlink_sock(unsigned int groups)
+{
+	struct lkl_sockaddr_nl la;
+	int fd, err;
+
+	fd = lkl_sys_socket(LKL_AF_NETLINK, LKL_SOCK_DGRAM, LKL_NETLINK_ROUTE);
+	if (fd < 0)
+		return fd;
+
+	memset(&la, 0, sizeof(la));
+	la.nl_family = LKL_AF_NETLINK;
+	la.nl_groups = groups;
+	err = lkl_sys_bind(fd, (struct lkl_sockaddr *)&la, sizeof(la));
+	if (err < 0)
+		return err;
+
+	return fd;
+}
+
+static int parse_rtattr(struct lkl_rtattr *tb[], int max,
+			struct lkl_rtattr *rta, int len)
+{
+	unsigned short type;
+
+	memset(tb, 0, sizeof(struct lkl_rtattr *) * (max + 1));
+	while (LKL_RTA_OK(rta, len)) {
+		type = rta->rta_type;
+		if ((type <= max) && (!tb[type]))
+			tb[type] = rta;
+		rta = LKL_RTA_NEXT(rta, len);
+	}
+	if (len)
+		lkl_printf("!!!Deficit %d, rta_len=%d\n", len,
+			rta->rta_len);
+	return 0;
+}
+
+struct addr_filter {
+	unsigned int ifindex;
+	void *addr;
+};
+
+static unsigned int get_ifa_flags(struct lkl_ifaddrmsg *ifa,
+				  struct lkl_rtattr *ifa_flags_attr)
+{
+	return ifa_flags_attr ? *(unsigned int *)LKL_RTA_DATA(ifa_flags_attr) :
+				ifa->ifa_flags;
+}
+
+/* returns:
+ * 0 - dad succeed.
+ * -1 - dad failed or other error.
+ * 1 - should wait for new msg.
+ */
+static int check_ipv6_dad(struct lkl_sockaddr_nl *nladdr,
+			  struct lkl_nlmsghdr *n, void *arg)
+{
+	struct addr_filter *filter = arg;
+	struct lkl_ifaddrmsg *ifa = LKL_NLMSG_DATA(n);
+	struct lkl_rtattr *rta_tb[LKL_IFA_MAX+1];
+	unsigned int ifa_flags;
+	int len = n->nlmsg_len;
+
+	if (n->nlmsg_type != LKL_RTM_NEWADDR)
+		return 1;
+
+	len -= LKL_NLMSG_LENGTH(sizeof(*ifa));
+	if (len < 0) {
+		lkl_printf("BUG: wrong nlmsg len %d\n", len);
+		return -1;
+	}
+
+	parse_rtattr(rta_tb, LKL_IFA_MAX, LKL_IFA_RTA(ifa),
+		     n->nlmsg_len - LKL_NLMSG_LENGTH(sizeof(*ifa)));
+
+	ifa_flags = get_ifa_flags(ifa, rta_tb[LKL_IFA_FLAGS]);
+
+	if (ifa->ifa_index != filter->ifindex)
+		return 1;
+	if (ifa->ifa_family != LKL_AF_INET6)
+		return 1;
+
+	if (!rta_tb[LKL_IFA_LOCAL])
+		rta_tb[LKL_IFA_LOCAL] = rta_tb[LKL_IFA_ADDRESS];
+
+	if (!rta_tb[LKL_IFA_LOCAL] ||
+	    (filter->addr && memcmp(LKL_RTA_DATA(rta_tb[LKL_IFA_LOCAL]),
+				    filter->addr, 16))) {
+		return 1;
+	}
+	if (ifa_flags & LKL_IFA_F_DADFAILED) {
+		lkl_printf("IPV6 DAD failed.\n");
+		return -1;
+	}
+	if (!(ifa_flags & LKL_IFA_F_TENTATIVE))
+		return 0;
+	return 1;
+}
+
+/* Copied from iproute2/lib/ */
+static int rtnl_listen(int fd, int (*handler)(struct lkl_sockaddr_nl *nladdr,
+					      struct lkl_nlmsghdr *, void *),
+		       void *arg)
+{
+	int status;
+	struct lkl_nlmsghdr *h;
+	struct lkl_sockaddr_nl nladdr = { .nl_family = LKL_AF_NETLINK };
+	struct lkl_iovec iov;
+	struct lkl_user_msghdr msg = {
+		.msg_name = &nladdr,
+		.msg_namelen = sizeof(nladdr),
+		.msg_iov = &iov,
+		.msg_iovlen = 1,
+	};
+	char   buf[16384];
+
+	iov.iov_base = buf;
+	while (1) {
+		iov.iov_len = sizeof(buf);
+		status = lkl_sys_recvmsg(fd, &msg, 0);
+
+		if (status < 0) {
+			if (status == -LKL_EINTR || status == -LKL_EAGAIN)
+				continue;
+			lkl_printf("netlink receive error %s (%d)\n",
+				lkl_strerror(status), status);
+			if (status == -LKL_ENOBUFS)
+				continue;
+			return status;
+		}
+		if (status == 0) {
+			lkl_printf("EOF on netlink\n");
+			return -1;
+		}
+		if (msg.msg_namelen != sizeof(nladdr)) {
+			lkl_printf("Sender address length == %d\n",
+				msg.msg_namelen);
+			return -1;
+		}
+
+		for (h = (struct lkl_nlmsghdr *)buf;
+		     (unsigned int)status >= sizeof(*h);) {
+			int err;
+			int len = h->nlmsg_len;
+			int l = len - sizeof(*h);
+
+			if (l < 0 || len > status) {
+				if (msg.msg_flags & LKL_MSG_TRUNC) {
+					lkl_printf("Truncated message\n");
+					return -1;
+				}
+				lkl_printf("!!!malformed message: len=%d\n",
+					len);
+				return -1;
+			}
+
+			err = handler(&nladdr, h, arg);
+			if (err <= 0)
+				return err;
+
+			status -= LKL_NLMSG_ALIGN(len);
+			h = (struct lkl_nlmsghdr *)((char *)h +
+						    LKL_NLMSG_ALIGN(len));
+		}
+		if (msg.msg_flags & LKL_MSG_TRUNC) {
+			lkl_printf("Message truncated\n");
+			continue;
+		}
+		if (status) {
+			lkl_printf("!!!Remnant of size %d\n", status);
+			return -1;
+		}
+	}
+}
+
+int lkl_if_wait_ipv6_dad(int ifindex, void *addr)
+{
+	struct addr_filter filter = {.ifindex = ifindex, .addr = addr};
+	int fd, ret;
+	struct {
+		struct lkl_nlmsghdr		nlmsg_info;
+		struct lkl_ifaddrmsg	ifaddrmsg_info;
+	} req;
+
+	fd = netlink_sock(1 << (LKL_RTNLGRP_IPV6_IFADDR - 1));
+	if (fd < 0)
+		return fd;
+
+	memset(&req, 0, sizeof(req));
+	req.nlmsg_info.nlmsg_len =
+			LKL_NLMSG_LENGTH(sizeof(struct lkl_ifaddrmsg));
+	req.nlmsg_info.nlmsg_flags = LKL_NLM_F_REQUEST | LKL_NLM_F_DUMP;
+	req.nlmsg_info.nlmsg_type = LKL_RTM_GETADDR;
+	req.ifaddrmsg_info.ifa_family = LKL_AF_INET6;
+	req.ifaddrmsg_info.ifa_index = ifindex;
+	ret = lkl_sys_send(fd, &req, req.nlmsg_info.nlmsg_len, 0);
+	if (ret < 0) {
+		lkl_perror("lkl_sys_send", ret);
+		return ret;
+	}
+	ret = rtnl_listen(fd, check_ipv6_dad, (void *)&filter);
+	lkl_sys_close(fd);
+	return ret;
+}
+
+int lkl_if_set_ipv6(int ifindex, void *addr, unsigned int netprefix_len)
+{
+	int err = lkl_if_add_ip(ifindex, LKL_AF_INET6, addr, netprefix_len);
+
+	if (err)
+		return err;
+	return lkl_if_wait_ipv6_dad(ifindex, addr);
+}
+
+int lkl_if_set_ipv6_gateway(int ifindex, void *src_addr,
+		unsigned int src_masklen, void *via_addr)
+{
+	int err;
+
+	err = lkl_if_add_rule_from_saddr(ifindex, LKL_AF_INET6, src_addr);
+	if (err)
+		return err;
+	err = lkl_if_add_linklocal(ifindex, LKL_AF_INET6,
+					src_addr, src_masklen);
+	if (err)
+		return err;
+	return lkl_if_add_gateway(ifindex, LKL_AF_INET6, via_addr);
+}
+
+int lkl_set_ipv6_gateway(void *addr)
+{
+	return lkl_add_gateway(LKL_AF_INET6, addr);
+}
+
+/* returns:
+ * 0 - succeed.
+ * < 0 - error number.
+ * 1 - should wait for new msg.
+ */
+static int check_error(struct lkl_sockaddr_nl *nladdr, struct lkl_nlmsghdr *n,
+		       void *arg)
+{
+	unsigned int s = *(unsigned int *)arg;
+
+	if (nladdr->nl_pid != 0 || n->nlmsg_seq != s) {
+		/* Don't forget to skip that message. */
+		return 1;
+	}
+
+	if (n->nlmsg_type == LKL_NLMSG_ERROR) {
+		struct lkl_nlmsgerr *err =
+			(struct lkl_nlmsgerr *)LKL_NLMSG_DATA(n);
+		int l = n->nlmsg_len - sizeof(*n);
+
+		if (l < (int)sizeof(struct lkl_nlmsgerr))
+			lkl_printf("ERROR truncated\n");
+		else if (!err->error)
+			return 0;
+
+		lkl_printf("RTNETLINK answers: %s\n",
+			lkl_strerror(-err->error));
+		return err->error;
+	}
+	lkl_printf("Unexpected reply!!!\n");
+	return -1;
+}
+
+static unsigned int seq;
+static int rtnl_talk(int fd, struct lkl_nlmsghdr *n)
+{
+	int status;
+	struct lkl_sockaddr_nl nladdr = {.nl_family = LKL_AF_NETLINK};
+	struct lkl_iovec iov = {.iov_base = (void *)n, .iov_len = n->nlmsg_len};
+	struct lkl_user_msghdr msg = {
+			.msg_name = &nladdr,
+			.msg_namelen = sizeof(nladdr),
+			.msg_iov = &iov,
+			.msg_iovlen = 1,
+	};
+
+	n->nlmsg_seq = seq;
+	n->nlmsg_flags |= LKL_NLM_F_ACK;
+
+	status = lkl_sys_sendmsg(fd, &msg, 0);
+	if (status < 0) {
+		lkl_perror("Cannot talk to rtnetlink", status);
+		return status;
+	}
+
+	status = rtnl_listen(fd, check_error, (void *)&seq);
+	seq++;
+	return status;
+}
+
+static int addattr_l(struct lkl_nlmsghdr *n, unsigned int maxlen,
+		     int type, const void *data, int alen)
+{
+	int len = LKL_RTA_LENGTH(alen);
+	struct lkl_rtattr *rta;
+
+	if (LKL_NLMSG_ALIGN(n->nlmsg_len) + LKL_RTA_ALIGN(len) > maxlen) {
+		lkl_printf("%s ERROR: message exceeded bound of %d\n", __func__,
+			   maxlen);
+		return -1;
+	}
+	rta = ((struct lkl_rtattr *) (((void *) (n)) +
+				      LKL_NLMSG_ALIGN(n->nlmsg_len)));
+	rta->rta_type = type;
+	rta->rta_len = len;
+	memcpy(LKL_RTA_DATA(rta), data, alen);
+	n->nlmsg_len = LKL_NLMSG_ALIGN(n->nlmsg_len) + LKL_RTA_ALIGN(len);
+	return 0;
+}
+
+int lkl_add_neighbor(int ifindex, int af, void *ip, void *mac)
+{
+	struct {
+		struct lkl_nlmsghdr n;
+		struct lkl_ndmsg r;
+		char buf[1024];
+	} req = {
+		.n.nlmsg_len = LKL_NLMSG_LENGTH(sizeof(struct lkl_ndmsg)),
+		.n.nlmsg_type = LKL_RTM_NEWNEIGH,
+		.n.nlmsg_flags = LKL_NLM_F_REQUEST |
+				 LKL_NLM_F_CREATE | LKL_NLM_F_REPLACE,
+		.r.ndm_family = af,
+		.r.ndm_ifindex = ifindex,
+		.r.ndm_state = LKL_NUD_PERMANENT,
+
+	};
+	int err, addr_sz;
+	int fd;
+
+	if (af == LKL_AF_INET)
+		addr_sz = 4;
+	else if (af == LKL_AF_INET6)
+		addr_sz = 16;
+	else {
+		lkl_printf("Bad address family: %d\n", af);
+		return -1;
+	}
+
+	fd = netlink_sock(0);
+	if (fd < 0)
+		return fd;
+
+	// create the IP attribute
+	addattr_l(&req.n, sizeof(req), LKL_NDA_DST, ip, addr_sz);
+
+	// create the MAC attribute
+	addattr_l(&req.n, sizeof(req), LKL_NDA_LLADDR, mac, 6);
+
+	err = rtnl_talk(fd, &req.n);
+	lkl_sys_close(fd);
+	return err;
+}
+
+static int ipaddr_modify(int cmd, int flags, int ifindex, int af, void *addr,
+			 unsigned int netprefix_len)
+{
+	struct {
+		struct lkl_nlmsghdr n;
+		struct lkl_ifaddrmsg ifa;
+		char buf[256];
+	} req = {
+		.n.nlmsg_len = LKL_NLMSG_LENGTH(sizeof(struct lkl_ifaddrmsg)),
+		.n.nlmsg_flags = LKL_NLM_F_REQUEST | flags,
+		.n.nlmsg_type = cmd,
+		.ifa.ifa_family = af,
+		.ifa.ifa_prefixlen = netprefix_len,
+		.ifa.ifa_index = ifindex,
+	};
+	int err, addr_sz;
+	int fd;
+
+	if (af == LKL_AF_INET)
+		addr_sz = 4;
+	else if (af == LKL_AF_INET6)
+		addr_sz = 16;
+	else {
+		lkl_printf("Bad address family: %d\n", af);
+		return -1;
+	}
+
+	fd = netlink_sock(0);
+	if (fd < 0)
+		return fd;
+
+	// create the IP attribute
+	addattr_l(&req.n, sizeof(req), LKL_IFA_LOCAL, addr, addr_sz);
+
+	err = rtnl_talk(fd, &req.n);
+
+	lkl_sys_close(fd);
+	return err;
+}
+
+int lkl_if_add_ip(int ifindex, int af, void *addr, unsigned int netprefix_len)
+{
+	return ipaddr_modify(LKL_RTM_NEWADDR, LKL_NLM_F_CREATE | LKL_NLM_F_EXCL,
+			     ifindex, af, addr, netprefix_len);
+}
+
+int lkl_if_del_ip(int ifindex, int af, void *addr, unsigned int netprefix_len)
+{
+	return ipaddr_modify(LKL_RTM_DELADDR, 0, ifindex, af,
+			     addr, netprefix_len);
+}
+
+static int iproute_modify(int cmd, unsigned int flags, int ifindex, int af,
+		void *route_addr, int route_masklen, void *gwaddr)
+{
+	struct {
+		struct lkl_nlmsghdr	n;
+		struct lkl_rtmsg	r;
+		char			buf[1024];
+	} req = {
+		.n.nlmsg_len = LKL_NLMSG_LENGTH(sizeof(struct lkl_rtmsg)),
+		.n.nlmsg_flags = LKL_NLM_F_REQUEST | flags,
+		.n.nlmsg_type = cmd,
+		.r.rtm_family = af,
+		.r.rtm_table = LKL_RT_TABLE_MAIN,
+		.r.rtm_scope = LKL_RT_SCOPE_UNIVERSE,
+	};
+	int err, addr_sz;
+	int i, fd;
+
+	fd = netlink_sock(0);
+	if (fd < 0) {
+		lkl_printf("netlink_sock error: %d\n", fd);
+		return fd;
+	}
+
+	if (af == LKL_AF_INET)
+		addr_sz = 4;
+	else if (af == LKL_AF_INET6)
+		addr_sz = 16;
+	else {
+		lkl_printf("Bad address family: %d\n", af);
+		return -1;
+	}
+
+	if (cmd != LKL_RTM_DELROUTE) {
+		req.r.rtm_protocol = LKL_RTPROT_BOOT;
+		req.r.rtm_scope = LKL_RT_SCOPE_UNIVERSE;
+		req.r.rtm_type = LKL_RTN_UNICAST;
+	}
+
+	if (gwaddr)
+		addattr_l(&req.n, sizeof(req),
+				LKL_RTA_GATEWAY, gwaddr, addr_sz);
+
+	if (af == LKL_AF_INET && route_addr) {
+		unsigned int netaddr = *(unsigned int *)route_addr;
+
+		netaddr = ntohl(netaddr);
+		netaddr = (netaddr >> (32 - route_masklen));
+		netaddr = (netaddr << (32 - route_masklen));
+		netaddr =  htonl(netaddr);
+		*(unsigned int *)route_addr = netaddr;
+		req.r.rtm_dst_len = route_masklen;
+		addattr_l(&req.n, sizeof(req), LKL_RTA_DST,
+					route_addr, addr_sz);
+	}
+
+	if (af == LKL_AF_INET6 && route_addr) {
+		struct lkl_in6_addr netaddr =
+			*(struct lkl_in6_addr *)route_addr;
+		int rmbyte = route_masklen/8;
+		int rmbit = route_masklen%8;
+
+		for (i = 0; i < rmbyte; i++)
+			netaddr.in6_u.u6_addr8[15-i] = 0;
+		netaddr.in6_u.u6_addr8[15-rmbyte] =
+			(netaddr.in6_u.u6_addr8[15-rmbyte] >> rmbit);
+		netaddr.in6_u.u6_addr8[15-rmbyte] =
+			(netaddr.in6_u.u6_addr8[15-rmbyte] << rmbit);
+		*(struct lkl_in6_addr *)route_addr = netaddr;
+		req.r.rtm_dst_len = route_masklen;
+		addattr_l(&req.n, sizeof(req), LKL_RTA_DST,
+					route_addr, addr_sz);
+	}
+
+	if (ifindex != LKL_RT_TABLE_MAIN) {
+		if (af == LKL_AF_INET)
+			req.r.rtm_table = ifindex * 2;
+		else if (af == LKL_AF_INET6)
+			req.r.rtm_table = ifindex * 2 + 1;
+		addattr_l(&req.n, sizeof(req), LKL_RTA_OIF, &ifindex, addr_sz);
+	}
+	err = rtnl_talk(fd, &req.n);
+	lkl_sys_close(fd);
+	return err;
+}
+
+int lkl_if_add_linklocal(int ifindex, int af,  void *addr, int netprefix_len)
+{
+	return iproute_modify(LKL_RTM_NEWROUTE, LKL_NLM_F_CREATE|LKL_NLM_F_EXCL,
+			ifindex, af, addr, netprefix_len, NULL);
+}
+
+int lkl_if_add_gateway(int ifindex, int af, void *gwaddr)
+{
+	return iproute_modify(LKL_RTM_NEWROUTE, LKL_NLM_F_CREATE|LKL_NLM_F_EXCL,
+			ifindex, af, NULL, 0, gwaddr);
+}
+
+int lkl_add_gateway(int af, void *gwaddr)
+{
+	return iproute_modify(LKL_RTM_NEWROUTE, LKL_NLM_F_CREATE|LKL_NLM_F_EXCL,
+			LKL_RT_TABLE_MAIN, af, NULL, 0, gwaddr);
+}
+
+static int iprule_modify(int cmd, int ifindex, int af, void *saddr)
+{
+	struct {
+		struct lkl_nlmsghdr	n;
+		struct lkl_rtmsg		r;
+		char			buf[1024];
+	} req = {
+		.n.nlmsg_type = cmd,
+		.n.nlmsg_len = LKL_NLMSG_LENGTH(sizeof(struct lkl_rtmsg)),
+		.n.nlmsg_flags = LKL_NLM_F_REQUEST,
+		.r.rtm_protocol = LKL_RTPROT_BOOT,
+		.r.rtm_scope = LKL_RT_SCOPE_UNIVERSE,
+		.r.rtm_family = af,
+		.r.rtm_type = LKL_RTN_UNSPEC,
+	};
+	int fd, err;
+	int addr_sz;
+
+	if (af == LKL_AF_INET)
+		addr_sz = 4;
+	else if (af == LKL_AF_INET6)
+		addr_sz = 16;
+	else {
+		lkl_printf("Bad address family: %d\n", af);
+		return -1;
+	}
+
+	fd = netlink_sock(0);
+	if (fd < 0)
+		return fd;
+
+	if (cmd == LKL_RTM_NEWRULE) {
+		req.n.nlmsg_flags |= LKL_NLM_F_CREATE|LKL_NLM_F_EXCL;
+		req.r.rtm_type = LKL_RTN_UNICAST;
+	}
+
+	//set from address
+	req.r.rtm_src_len = 8 * addr_sz;
+	addattr_l(&req.n, sizeof(req), LKL_FRA_SRC, saddr, addr_sz);
+
+	//use ifindex as table id
+	if (af == LKL_AF_INET)
+		req.r.rtm_table = ifindex * 2;
+	else if (af == LKL_AF_INET6)
+		req.r.rtm_table = ifindex * 2 + 1;
+	err = rtnl_talk(fd, &req.n);
+
+	lkl_sys_close(fd);
+	return err;
+}
+
+int lkl_if_add_rule_from_saddr(int ifindex, int af, void *saddr)
+{
+	return iprule_modify(LKL_RTM_NEWRULE, ifindex, af, saddr);
+}
+
+static int qdisc_add(int cmd, int flags, int ifindex,
+		     const char *root, const char *type)
+{
+	struct {
+		struct lkl_nlmsghdr n;
+		struct lkl_tcmsg tc;
+		char buf[2*1024];
+	} req = {
+		.n.nlmsg_len = LKL_NLMSG_LENGTH(sizeof(struct lkl_tcmsg)),
+		.n.nlmsg_flags = LKL_NLM_F_REQUEST|flags,
+		.n.nlmsg_type = cmd,
+		.tc.tcm_family = LKL_AF_UNSPEC,
+	};
+	int err, fd;
+
+	if (!root || !type) {
+		lkl_printf("root and type arguments\n");
+		return -1;
+	}
+
+	if (strcmp(root, "root") == 0)
+		req.tc.tcm_parent = LKL_TC_H_ROOT;
+	req.tc.tcm_ifindex = ifindex;
+
+	fd = netlink_sock(0);
+	if (fd < 0)
+		return fd;
+
+	// create the qdisc attribute
+	addattr_l(&req.n, sizeof(req), LKL_TCA_KIND, type, strlen(type)+1);
+
+	err = rtnl_talk(fd, &req.n);
+	lkl_sys_close(fd);
+	return err;
+}
+
+int lkl_qdisc_add(int ifindex, const char *root, const char *type)
+{
+	return qdisc_add(LKL_RTM_NEWQDISC, LKL_NLM_F_CREATE | LKL_NLM_F_EXCL,
+			 ifindex, root, type);
+}
+
+/* Add a qdisc entry for an interface in the form of
+ * "root|type;root|type;..."
+ */
+void lkl_qdisc_parse_add(int ifindex, const char *entries)
+{
+	char *saveptr = NULL, *token = NULL;
+	char *root = NULL, *type = NULL;
+	char strings[256];
+	int ret = 0;
+
+	if (strlen(entries) >= sizeof(strings)) {
+		lkl_printf("Error: long strings of input: %s\n", entries);
+		return;
+	}
+
+	strcpy(strings, entries);
+
+	for (token = strtok_r(strings, ";", &saveptr); token;
+	     token = strtok_r(NULL, ";", &saveptr)) {
+		root = strtok(token, "|");
+		type = strtok(NULL, "|");
+		ret = lkl_qdisc_add(ifindex, root, type);
+		if (ret) {
+			lkl_printf("Failed to add qdisc entry: %s\n",
+				   lkl_strerror(ret));
+			return;
+		}
+	}
+}
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 15/25] lkl tools: host lib: networking helpers
@ 2020-03-30 14:45     ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, linux-kernel-library, linux-arch,
	Hajime Tazaki, Akira Moroo

This commit adds LKL application APIs to control network related
resources of kernel via LKL syscall, ioctl, and via netlink messages.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 tools/lkl/include/lkl.h      | 241 ++++++++++
 tools/lkl/include/lkl_host.h |  85 ++++
 tools/lkl/lib/Build          |   1 +
 tools/lkl/lib/net.c          | 826 +++++++++++++++++++++++++++++++++++
 4 files changed, 1153 insertions(+)
 create mode 100644 tools/lkl/lib/net.c

diff --git a/tools/lkl/include/lkl.h b/tools/lkl/include/lkl.h
index cdae92300320..1f4291ad9455 100644
--- a/tools/lkl/include/lkl.h
+++ b/tools/lkl/include/lkl.h
@@ -516,6 +516,247 @@ int lkl_dirfd(struct lkl_dir *dir);
  */
 int lkl_mount_fs(char *fstype);
 
+/**
+ * lkl_if_up - activate network interface
+ *
+ * @ifindex - the ifindex of the interface
+ * @returns - return 0 if no error: otherwise negative value returns
+ */
+int lkl_if_up(int ifindex);
+
+/**
+ * lkl_if_down - deactivate network interface
+ *
+ * @ifindex - the ifindex of the interface
+ * @returns - return 0 if no error: otherwise negative value returns
+ */
+int lkl_if_down(int ifindex);
+
+/**
+ * lkl_if_set_mtu - set MTU on interface
+ *
+ * @ifindex - the ifindex of the interface
+ * @mtu - the requested MTU size
+ * @returns - return 0 if no error: otherwise negative value returns
+ */
+int lkl_if_set_mtu(int ifindex, int mtu);
+
+/**
+ * lkl_if_set_ipv4 - set IPv4 address on interface
+ *
+ * @ifindex - the ifindex of the interface
+ * @addr - 4-byte IP address (i.e., struct in_addr)
+ * @netmask_len - prefix length of the @addr
+ * @returns - return 0 if no error: otherwise negative value returns
+ */
+int lkl_if_set_ipv4(int ifindex, unsigned int addr, unsigned int netmask_len);
+
+/**
+ * lkl_set_ipv4_gateway - add an IPv4 default route
+ *
+ * @addr - 4-byte IP address of the gateway (i.e., struct in_addr)
+ * @returns - return 0 if no error: otherwise negative value returns
+ */
+int lkl_set_ipv4_gateway(unsigned int addr);
+
+/**
+ * lkl_if_set_ipv4_gateway - add an IPv4 default route in rule table
+ *
+ * @ifindex - the ifindex of the interface, used for tableid calculation
+ * @addr - 4-byte IP address of the interface
+ * @netmask_len - prefix length of the @addr
+ * @gw_addr - 4-byte IP address of the gateway
+ * @returns - return 0 if no error: otherwise negative value returns
+ */
+int lkl_if_set_ipv4_gateway(int ifindex, unsigned int addr,
+		unsigned int netmask_len, unsigned int gw_addr);
+
+/**
+ * lkl_if_set_ipv6 - set IPv6 address on interface
+ * must be called after interface is up.
+ *
+ * @ifindex - the ifindex of the interface
+ * @addr - 16-byte IPv6 address (i.e., struct in6_addr)
+ * @netprefix_len - prefix length of the @addr
+ * @returns - return 0 if no error: otherwise negative value returns
+ */
+int lkl_if_set_ipv6(int ifindex, void *addr, unsigned int netprefix_len);
+
+/**
+ * lkl_set_ipv6_gateway - add an IPv6 default route
+ *
+ * @addr - 16-byte IPv6 address of the gateway (i.e., struct in6_addr)
+ * @returns - return 0 if no error: otherwise negative value returns
+ */
+int lkl_set_ipv6_gateway(void *addr);
+
+/**
+ * lkl_if_set_ipv6_gateway - add an IPv6 default route in rule table
+ *
+ * @ifindex - the ifindex of the interface, used for tableid calculation
+ * @addr - 16-byte IP address of the interface
+ * @netmask_len - prefix length of the @addr
+ * @gw_addr - 16-byte IP address of the gateway (i.e., struct in_addr)
+ * @returns - return 0 if no error: otherwise negative value returns
+ */
+int lkl_if_set_ipv6_gateway(int ifindex, void *addr,
+		unsigned int netmask_len, void *gw_addr);
+
+/**
+ * lkl_ifname_to_ifindex - obtain ifindex of an interface by name
+ *
+ * @name - string of an interface
+ * @returns - return an integer of ifindex if no error
+ */
+int lkl_ifname_to_ifindex(const char *name);
+
+/**
+ * lkl_netdev_get_ifindex - retrieve the interface index for a given network
+ * device id
+ *
+ * @id - the network device id
+ * @returns the interface index or a stricly negative value in case of error
+ */
+int lkl_netdev_get_ifindex(int id);
+
+/**
+ * lkl_add_neighbor - add a permanent arp entry
+ * @ifindex - the ifindex of the interface
+ * @af - address family of the ip address. Must be LKL_AF_INET or LKL_AF_INET6
+ * @ip - ip address of the entry in network byte order
+ * @mac - mac address of the entry
+ */
+int lkl_add_neighbor(int ifindex, int af, void *addr, void *mac);
+
+/**
+ * lkl_if_add_ip - add an ip address
+ * @ifindex - the ifindex of the interface
+ * @af - address family of the ip address. Must be LKL_AF_INET or LKL_AF_INET6
+ * @addr - ip address of the entry in network byte order
+ * @netprefix_len - prefix length of the @addr
+ */
+int lkl_if_add_ip(int ifindex, int af, void *addr, unsigned int netprefix_len);
+
+/**
+ * lkl_if_del_ip - add an ip address
+ * @ifindex - the ifindex of the interface
+ * @af - address family of the ip address. Must be LKL_AF_INET or LKL_AF_INET6
+ * @addr - ip address of the entry in network byte order
+ * @netprefix_len - prefix length of the @addr
+ */
+int lkl_if_del_ip(int ifindex, int af, void *addr, unsigned int netprefix_len);
+
+/**
+ * lkl_add_gateway - add a gateway
+ * @af - address family of the ip address. Must be LKL_AF_INET or LKL_AF_INET6
+ * @gwaddr - 4-byte IP address of the gateway (i.e., struct in_addr)
+ */
+int lkl_add_gateway(int af, void *gwaddr);
+
+/**
+ * XXX Should I use OIF selector?
+ * temporary table idx = ifindex * 2 + 0 <- ipv4
+ * temporary table idx = ifindex * 2 + 1 <- ipv6
+ */
+/**
+ * lkl_if_add_rule_from_addr - create an ip rule table with "from" selector
+ * @ifindex - the ifindex of the interface, used for table id calculation
+ * @af - address family of the ip address. Must be LKL_AF_INET or LKL_AF_INET6
+ * @saddr - network byte order ip address, "from" selector address of this rule
+ */
+int lkl_if_add_rule_from_saddr(int ifindex, int af, void *saddr);
+
+/**
+ * lkl_if_add_gateway - add gateway to rule table
+ * @ifindex - the ifindex of the interface, used for table id calculation
+ * @af - address family of the ip address. Must be LKL_AF_INET or LKL_AF_INET6
+ * @gwaddr - 4-byte IP address of the gateway (i.e., struct in_addr)
+ */
+int lkl_if_add_gateway(int ifindex, int af, void *gwaddr);
+
+/**
+ * lkl_if_add_linklocal - add linklocal route to rule table
+ * @ifindex - the ifindex of the interface, used for table id calculation
+ * @af - address family of the ip address. Must be LKL_AF_INET or LKL_AF_INET6
+ * @addr - ip address of the entry in network byte order
+ * @netprefix_len - prefix length of the @addr
+ */
+int lkl_if_add_linklocal(int ifindex, int af,  void *addr, int netprefix_len);
+
+/**
+ * lkl_if_wait_ipv6_dad - wait for DAD to be done for a ipv6 address
+ * must be called after interface is up
+ *
+ * @ifindex - the ifindex of the interface
+ * @addr - ip address of the entry in network byte order
+ */
+int lkl_if_wait_ipv6_dad(int ifindex, void *addr);
+
+/**
+ * lkl_set_fd_limit - set the maximum number of file descriptors allowed
+ * @fd_limit - fd max limit
+ */
+int lkl_set_fd_limit(unsigned int fd_limit);
+
+/**
+ * lkl_qdisc_add - set qdisc rule onto an interface
+ *
+ * @ifindex - the ifindex of the interface
+ * @root - the name of root class (e.g., "root");
+ * @type - the type of qdisc (e.g., "fq")
+ */
+int lkl_qdisc_add(int ifindex, const char *root, const char *type);
+
+/**
+ * lkl_qdisc_parse_add - Add a qdisc entry for an interface with strings
+ *
+ * @ifindex - the ifindex of the interface
+ * @entries - strings of qdisc configurations in the form of
+ *            "root|type;root|type;..."
+ */
+void lkl_qdisc_parse_add(int ifindex, const char *entries);
+
+/**
+ * lkl_netdev - host network device handle, defined in lkl_host.h.
+ */
+struct lkl_netdev;
+
+/**
+ * lkl_netdev_args - arguments to lkl_netdev_add
+ * @mac - optional MAC address for the device
+ * @offload - offload bits for the device
+ */
+struct lkl_netdev_args {
+	void *mac;
+	unsigned int offload;
+};
+
+/*
+ * lkl_register_dbg_handler- register a signal handler that loads a debug lib.
+ *
+ * The signal handler is triggered by Ctrl-Z. It creates a new pthread which
+ * call dbg_entrance().
+ *
+ * If you run the program from shell script, make sure you ignore SIGTSTP by
+ * "trap '' TSTP" in the shell script.
+ */
+void lkl_register_dbg_handler(void);
+
+/**
+ * lkl_sysctl - write a sysctl value
+ *
+ * @path - the path to an sysctl entry (e.g., "net.ipv4.tcp_wmem");
+ * @value - the value of the sysctl (e.g., "4096 87380 2147483647")
+ */
+int lkl_sysctl(const char *path, const char *value);
+
+/**
+ * lkl_sysctl_parse_write - Configure sysctl parameters with strings
+ *
+ * @sysctls - Configure sysctl parameters as the form of "key=value;..."
+ */
+void lkl_sysctl_parse_write(const char *sysctls);
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/tools/lkl/include/lkl_host.h b/tools/lkl/include/lkl_host.h
index 85e80eb4ad0d..4e6d6e031498 100644
--- a/tools/lkl/include/lkl_host.h
+++ b/tools/lkl/include/lkl_host.h
@@ -18,6 +18,91 @@ extern struct lkl_host_operations lkl_host_ops;
  */
 int lkl_printf(const char *fmt, ...);
 
+#ifdef LKL_HOST_CONFIG_POSIX
+#include <sys/uio.h>
+#else
+struct iovec {
+	void *iov_base;
+	size_t iov_len;
+};
+#endif
+
+struct lkl_netdev {
+	struct lkl_dev_net_ops *ops;
+	int id;
+	uint8_t has_vnet_hdr: 1;
+};
+
+/**
+ * struct lkl_dev_net_ops - network device host operations
+ */
+struct lkl_dev_net_ops {
+	/**
+	 * @tx: writes a L2 packet into the net device
+	 *
+	 * The data buffer can only hold 0 or 1 complete packets.
+	 *
+	 * @nd - pointer to the network device;
+	 * @iov - pointer to the buffer vector;
+	 * @cnt - # of vectors in iov.
+	 *
+	 * @returns number of bytes transmitted
+	 */
+	int (*tx)(struct lkl_netdev *nd, struct iovec *iov, int cnt);
+
+	/**
+	 * @rx: reads a packet from the net device.
+	 *
+	 * It must only read one complete packet if present.
+	 *
+	 * If the buffer is too small for the packet, the implementation may
+	 * decide to drop it or trim it.
+	 *
+	 * @nd - pointer to the network device
+	 * @iov - pointer to the buffer vector to store the packet
+	 * @cnt - # of vectors in iov.
+	 *
+	 * @returns number of bytes read for success or < 0 if error
+	 */
+	int (*rx)(struct lkl_netdev *nd, struct iovec *iov, int cnt);
+
+#define LKL_DEV_NET_POLL_RX		1
+#define LKL_DEV_NET_POLL_TX		2
+#define LKL_DEV_NET_POLL_HUP		4
+
+	/**
+	 * @poll: polls a net device
+	 *
+	 * Supports the following events: LKL_DEV_NET_POLL_RX
+	 * (readable), LKL_DEV_NET_POLL_TX (writable) or
+	 * LKL_DEV_NET_POLL_HUP (the close operations has been issued
+	 * and we need to clean up). Blocks until one event is
+	 * available.
+	 *
+	 * @nd - pointer to the network device
+	 *
+	 * @returns - LKL_DEV_NET_POLL_RX, LKL_DEV_NET_POLL_TX,
+	 * LKL_DEV_NET_POLL_HUP or a negative value for errors
+	 */
+	int (*poll)(struct lkl_netdev *nd);
+
+	/**
+	 * @poll_hup: make poll wakeup and return LKL_DEV_NET_POLL_HUP
+	 *
+	 * @nd - pointer to the network device
+	 */
+	void (*poll_hup)(struct lkl_netdev *nd);
+
+	/**
+	 * @free: frees a network device
+	 *
+	 * Implementation must release its resources and free the network device
+	 * structure.
+	 *
+	 * @nd - pointer to the network device
+	 */
+	void (*free)(struct lkl_netdev *nd);
+};
 
 #ifdef __cplusplus
 }
diff --git a/tools/lkl/lib/Build b/tools/lkl/lib/Build
index 4a444337b0e7..dba530424e4a 100644
--- a/tools/lkl/lib/Build
+++ b/tools/lkl/lib/Build
@@ -1,6 +1,7 @@
 CFLAGS_config.o += -I$(srctree)/tools/perf/pmu-events
 
 liblkl-y += fs.o
+liblkl-y += net.o
 liblkl-y += jmp_buf.o
 liblkl-y += utils.o
 liblkl-y += dbg.o
diff --git a/tools/lkl/lib/net.c b/tools/lkl/lib/net.c
new file mode 100644
index 000000000000..cf6894c35e46
--- /dev/null
+++ b/tools/lkl/lib/net.c
@@ -0,0 +1,826 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <string.h>
+#include <stdio.h>
+#include "endian.h"
+#include <lkl_host.h>
+
+#ifdef __MINGW32__
+#include <ws2tcpip.h>
+
+int lkl_inet_pton(int af, const char *src, void *dst)
+{
+	struct addrinfo hint, *res = NULL;
+	int err;
+
+	memset(&hint, 0, sizeof(struct addrinfo));
+
+	hint.ai_family = af;
+	hint.ai_flags = AI_NUMERICHOST;
+
+	err = getaddrinfo(src, NULL, &hint, &res);
+	if (err)
+		return 0;
+
+	switch (af) {
+	case AF_INET:
+		*(struct in_addr *)dst =
+			((struct sockaddr_in *)&res->ai_addr)->sin_addr;
+		break;
+	case AF_INET6:
+		*(struct in6_addr *)dst =
+			((struct sockaddr_in6 *)&res->ai_addr)->sin6_addr;
+		break;
+	default:
+		freeaddrinfo(res);
+		return 0;
+	}
+
+	freeaddrinfo(res);
+	return 1;
+}
+#endif
+
+static inline void set_sockaddr(struct lkl_sockaddr_in *sin, unsigned int addr,
+				unsigned short port)
+{
+	sin->sin_family = LKL_AF_INET;
+	sin->sin_addr.lkl_s_addr = addr;
+	sin->sin_port = port;
+}
+
+static inline int ifindex_to_name(int sock, struct lkl_ifreq *ifr, int ifindex)
+{
+	ifr->lkl_ifr_ifindex = ifindex;
+	return lkl_sys_ioctl(sock, LKL_SIOCGIFNAME, (long)ifr);
+}
+
+int lkl_ifname_to_ifindex(const char *name)
+{
+	struct lkl_ifreq ifr;
+	int fd, ret;
+
+	fd = lkl_sys_socket(LKL_AF_INET, LKL_SOCK_DGRAM, 0);
+	if (fd < 0)
+		return fd;
+
+	if (strlen(name) >= LKL_IFNAMSIZ)
+		return -LKL_ENAMETOOLONG;
+
+	strcpy(ifr.lkl_ifr_name, name);
+
+	ret = lkl_sys_ioctl(fd, LKL_SIOCGIFINDEX, (long)&ifr);
+	if (ret < 0)
+		return ret;
+
+	return ifr.lkl_ifr_ifindex;
+}
+
+int lkl_if_up(int ifindex)
+{
+	struct lkl_ifreq ifr;
+	int err, sock = lkl_sys_socket(LKL_AF_INET, LKL_SOCK_DGRAM, 0);
+
+	if (sock < 0)
+		return sock;
+	err = ifindex_to_name(sock, &ifr, ifindex);
+	if (err < 0)
+		return err;
+
+	err = lkl_sys_ioctl(sock, LKL_SIOCGIFFLAGS, (long)&ifr);
+	if (!err) {
+		ifr.lkl_ifr_flags |= LKL_IFF_UP;
+		err = lkl_sys_ioctl(sock, LKL_SIOCSIFFLAGS, (long)&ifr);
+	}
+
+	lkl_sys_close(sock);
+
+	return err;
+}
+
+int lkl_if_down(int ifindex)
+{
+	struct lkl_ifreq ifr;
+	int err, sock;
+
+	sock = lkl_sys_socket(LKL_AF_INET, LKL_SOCK_DGRAM, 0);
+	if (sock < 0)
+		return sock;
+
+	err = ifindex_to_name(sock, &ifr, ifindex);
+	if (err < 0)
+		return err;
+
+	err = lkl_sys_ioctl(sock, LKL_SIOCGIFFLAGS, (long)&ifr);
+	if (!err) {
+		ifr.lkl_ifr_flags &= ~LKL_IFF_UP;
+		err = lkl_sys_ioctl(sock, LKL_SIOCSIFFLAGS, (long)&ifr);
+	}
+
+	lkl_sys_close(sock);
+
+	return err;
+}
+
+int lkl_if_set_mtu(int ifindex, int mtu)
+{
+	struct lkl_ifreq ifr;
+	int err, sock;
+
+	sock = lkl_sys_socket(LKL_AF_INET, LKL_SOCK_DGRAM, 0);
+	if (sock < 0)
+		return sock;
+
+	err = ifindex_to_name(sock, &ifr, ifindex);
+	if (err < 0)
+		return err;
+
+	ifr.lkl_ifr_mtu = mtu;
+
+	err = lkl_sys_ioctl(sock, LKL_SIOCSIFMTU, (long)&ifr);
+
+	lkl_sys_close(sock);
+
+	return err;
+}
+
+int lkl_if_set_ipv4(int ifindex, unsigned int addr, unsigned int netmask_len)
+{
+	return lkl_if_add_ip(ifindex, LKL_AF_INET, &addr, netmask_len);
+}
+
+int lkl_if_set_ipv4_gateway(int ifindex, unsigned int src_addr,
+		unsigned int src_masklen, unsigned int via_addr)
+{
+	int err;
+
+	err = lkl_if_add_rule_from_saddr(ifindex, LKL_AF_INET, &src_addr);
+	if (err)
+		return err;
+	err = lkl_if_add_linklocal(ifindex, LKL_AF_INET,
+					&src_addr, src_masklen);
+	if (err)
+		return err;
+	return lkl_if_add_gateway(ifindex, LKL_AF_INET, &via_addr);
+}
+
+int lkl_set_ipv4_gateway(unsigned int addr)
+{
+	return lkl_add_gateway(LKL_AF_INET, &addr);
+}
+
+int lkl_netdev_get_ifindex(int id)
+{
+	struct lkl_ifreq ifr;
+	int sock, ret;
+
+	sock = lkl_sys_socket(LKL_AF_INET, LKL_SOCK_DGRAM, 0);
+	if (sock < 0)
+		return sock;
+
+	snprintf(ifr.lkl_ifr_name, sizeof(ifr.lkl_ifr_name), "eth%d", id);
+	ret = lkl_sys_ioctl(sock, LKL_SIOCGIFINDEX, (long)&ifr);
+	lkl_sys_close(sock);
+
+	return ret < 0 ? ret : ifr.lkl_ifr_ifindex;
+}
+
+static int netlink_sock(unsigned int groups)
+{
+	struct lkl_sockaddr_nl la;
+	int fd, err;
+
+	fd = lkl_sys_socket(LKL_AF_NETLINK, LKL_SOCK_DGRAM, LKL_NETLINK_ROUTE);
+	if (fd < 0)
+		return fd;
+
+	memset(&la, 0, sizeof(la));
+	la.nl_family = LKL_AF_NETLINK;
+	la.nl_groups = groups;
+	err = lkl_sys_bind(fd, (struct lkl_sockaddr *)&la, sizeof(la));
+	if (err < 0)
+		return err;
+
+	return fd;
+}
+
+static int parse_rtattr(struct lkl_rtattr *tb[], int max,
+			struct lkl_rtattr *rta, int len)
+{
+	unsigned short type;
+
+	memset(tb, 0, sizeof(struct lkl_rtattr *) * (max + 1));
+	while (LKL_RTA_OK(rta, len)) {
+		type = rta->rta_type;
+		if ((type <= max) && (!tb[type]))
+			tb[type] = rta;
+		rta = LKL_RTA_NEXT(rta, len);
+	}
+	if (len)
+		lkl_printf("!!!Deficit %d, rta_len=%d\n", len,
+			rta->rta_len);
+	return 0;
+}
+
+struct addr_filter {
+	unsigned int ifindex;
+	void *addr;
+};
+
+static unsigned int get_ifa_flags(struct lkl_ifaddrmsg *ifa,
+				  struct lkl_rtattr *ifa_flags_attr)
+{
+	return ifa_flags_attr ? *(unsigned int *)LKL_RTA_DATA(ifa_flags_attr) :
+				ifa->ifa_flags;
+}
+
+/* returns:
+ * 0 - dad succeed.
+ * -1 - dad failed or other error.
+ * 1 - should wait for new msg.
+ */
+static int check_ipv6_dad(struct lkl_sockaddr_nl *nladdr,
+			  struct lkl_nlmsghdr *n, void *arg)
+{
+	struct addr_filter *filter = arg;
+	struct lkl_ifaddrmsg *ifa = LKL_NLMSG_DATA(n);
+	struct lkl_rtattr *rta_tb[LKL_IFA_MAX+1];
+	unsigned int ifa_flags;
+	int len = n->nlmsg_len;
+
+	if (n->nlmsg_type != LKL_RTM_NEWADDR)
+		return 1;
+
+	len -= LKL_NLMSG_LENGTH(sizeof(*ifa));
+	if (len < 0) {
+		lkl_printf("BUG: wrong nlmsg len %d\n", len);
+		return -1;
+	}
+
+	parse_rtattr(rta_tb, LKL_IFA_MAX, LKL_IFA_RTA(ifa),
+		     n->nlmsg_len - LKL_NLMSG_LENGTH(sizeof(*ifa)));
+
+	ifa_flags = get_ifa_flags(ifa, rta_tb[LKL_IFA_FLAGS]);
+
+	if (ifa->ifa_index != filter->ifindex)
+		return 1;
+	if (ifa->ifa_family != LKL_AF_INET6)
+		return 1;
+
+	if (!rta_tb[LKL_IFA_LOCAL])
+		rta_tb[LKL_IFA_LOCAL] = rta_tb[LKL_IFA_ADDRESS];
+
+	if (!rta_tb[LKL_IFA_LOCAL] ||
+	    (filter->addr && memcmp(LKL_RTA_DATA(rta_tb[LKL_IFA_LOCAL]),
+				    filter->addr, 16))) {
+		return 1;
+	}
+	if (ifa_flags & LKL_IFA_F_DADFAILED) {
+		lkl_printf("IPV6 DAD failed.\n");
+		return -1;
+	}
+	if (!(ifa_flags & LKL_IFA_F_TENTATIVE))
+		return 0;
+	return 1;
+}
+
+/* Copied from iproute2/lib/ */
+static int rtnl_listen(int fd, int (*handler)(struct lkl_sockaddr_nl *nladdr,
+					      struct lkl_nlmsghdr *, void *),
+		       void *arg)
+{
+	int status;
+	struct lkl_nlmsghdr *h;
+	struct lkl_sockaddr_nl nladdr = { .nl_family = LKL_AF_NETLINK };
+	struct lkl_iovec iov;
+	struct lkl_user_msghdr msg = {
+		.msg_name = &nladdr,
+		.msg_namelen = sizeof(nladdr),
+		.msg_iov = &iov,
+		.msg_iovlen = 1,
+	};
+	char   buf[16384];
+
+	iov.iov_base = buf;
+	while (1) {
+		iov.iov_len = sizeof(buf);
+		status = lkl_sys_recvmsg(fd, &msg, 0);
+
+		if (status < 0) {
+			if (status == -LKL_EINTR || status == -LKL_EAGAIN)
+				continue;
+			lkl_printf("netlink receive error %s (%d)\n",
+				lkl_strerror(status), status);
+			if (status == -LKL_ENOBUFS)
+				continue;
+			return status;
+		}
+		if (status == 0) {
+			lkl_printf("EOF on netlink\n");
+			return -1;
+		}
+		if (msg.msg_namelen != sizeof(nladdr)) {
+			lkl_printf("Sender address length == %d\n",
+				msg.msg_namelen);
+			return -1;
+		}
+
+		for (h = (struct lkl_nlmsghdr *)buf;
+		     (unsigned int)status >= sizeof(*h);) {
+			int err;
+			int len = h->nlmsg_len;
+			int l = len - sizeof(*h);
+
+			if (l < 0 || len > status) {
+				if (msg.msg_flags & LKL_MSG_TRUNC) {
+					lkl_printf("Truncated message\n");
+					return -1;
+				}
+				lkl_printf("!!!malformed message: len=%d\n",
+					len);
+				return -1;
+			}
+
+			err = handler(&nladdr, h, arg);
+			if (err <= 0)
+				return err;
+
+			status -= LKL_NLMSG_ALIGN(len);
+			h = (struct lkl_nlmsghdr *)((char *)h +
+						    LKL_NLMSG_ALIGN(len));
+		}
+		if (msg.msg_flags & LKL_MSG_TRUNC) {
+			lkl_printf("Message truncated\n");
+			continue;
+		}
+		if (status) {
+			lkl_printf("!!!Remnant of size %d\n", status);
+			return -1;
+		}
+	}
+}
+
+int lkl_if_wait_ipv6_dad(int ifindex, void *addr)
+{
+	struct addr_filter filter = {.ifindex = ifindex, .addr = addr};
+	int fd, ret;
+	struct {
+		struct lkl_nlmsghdr		nlmsg_info;
+		struct lkl_ifaddrmsg	ifaddrmsg_info;
+	} req;
+
+	fd = netlink_sock(1 << (LKL_RTNLGRP_IPV6_IFADDR - 1));
+	if (fd < 0)
+		return fd;
+
+	memset(&req, 0, sizeof(req));
+	req.nlmsg_info.nlmsg_len =
+			LKL_NLMSG_LENGTH(sizeof(struct lkl_ifaddrmsg));
+	req.nlmsg_info.nlmsg_flags = LKL_NLM_F_REQUEST | LKL_NLM_F_DUMP;
+	req.nlmsg_info.nlmsg_type = LKL_RTM_GETADDR;
+	req.ifaddrmsg_info.ifa_family = LKL_AF_INET6;
+	req.ifaddrmsg_info.ifa_index = ifindex;
+	ret = lkl_sys_send(fd, &req, req.nlmsg_info.nlmsg_len, 0);
+	if (ret < 0) {
+		lkl_perror("lkl_sys_send", ret);
+		return ret;
+	}
+	ret = rtnl_listen(fd, check_ipv6_dad, (void *)&filter);
+	lkl_sys_close(fd);
+	return ret;
+}
+
+int lkl_if_set_ipv6(int ifindex, void *addr, unsigned int netprefix_len)
+{
+	int err = lkl_if_add_ip(ifindex, LKL_AF_INET6, addr, netprefix_len);
+
+	if (err)
+		return err;
+	return lkl_if_wait_ipv6_dad(ifindex, addr);
+}
+
+int lkl_if_set_ipv6_gateway(int ifindex, void *src_addr,
+		unsigned int src_masklen, void *via_addr)
+{
+	int err;
+
+	err = lkl_if_add_rule_from_saddr(ifindex, LKL_AF_INET6, src_addr);
+	if (err)
+		return err;
+	err = lkl_if_add_linklocal(ifindex, LKL_AF_INET6,
+					src_addr, src_masklen);
+	if (err)
+		return err;
+	return lkl_if_add_gateway(ifindex, LKL_AF_INET6, via_addr);
+}
+
+int lkl_set_ipv6_gateway(void *addr)
+{
+	return lkl_add_gateway(LKL_AF_INET6, addr);
+}
+
+/* returns:
+ * 0 - succeed.
+ * < 0 - error number.
+ * 1 - should wait for new msg.
+ */
+static int check_error(struct lkl_sockaddr_nl *nladdr, struct lkl_nlmsghdr *n,
+		       void *arg)
+{
+	unsigned int s = *(unsigned int *)arg;
+
+	if (nladdr->nl_pid != 0 || n->nlmsg_seq != s) {
+		/* Don't forget to skip that message. */
+		return 1;
+	}
+
+	if (n->nlmsg_type == LKL_NLMSG_ERROR) {
+		struct lkl_nlmsgerr *err =
+			(struct lkl_nlmsgerr *)LKL_NLMSG_DATA(n);
+		int l = n->nlmsg_len - sizeof(*n);
+
+		if (l < (int)sizeof(struct lkl_nlmsgerr))
+			lkl_printf("ERROR truncated\n");
+		else if (!err->error)
+			return 0;
+
+		lkl_printf("RTNETLINK answers: %s\n",
+			lkl_strerror(-err->error));
+		return err->error;
+	}
+	lkl_printf("Unexpected reply!!!\n");
+	return -1;
+}
+
+static unsigned int seq;
+static int rtnl_talk(int fd, struct lkl_nlmsghdr *n)
+{
+	int status;
+	struct lkl_sockaddr_nl nladdr = {.nl_family = LKL_AF_NETLINK};
+	struct lkl_iovec iov = {.iov_base = (void *)n, .iov_len = n->nlmsg_len};
+	struct lkl_user_msghdr msg = {
+			.msg_name = &nladdr,
+			.msg_namelen = sizeof(nladdr),
+			.msg_iov = &iov,
+			.msg_iovlen = 1,
+	};
+
+	n->nlmsg_seq = seq;
+	n->nlmsg_flags |= LKL_NLM_F_ACK;
+
+	status = lkl_sys_sendmsg(fd, &msg, 0);
+	if (status < 0) {
+		lkl_perror("Cannot talk to rtnetlink", status);
+		return status;
+	}
+
+	status = rtnl_listen(fd, check_error, (void *)&seq);
+	seq++;
+	return status;
+}
+
+static int addattr_l(struct lkl_nlmsghdr *n, unsigned int maxlen,
+		     int type, const void *data, int alen)
+{
+	int len = LKL_RTA_LENGTH(alen);
+	struct lkl_rtattr *rta;
+
+	if (LKL_NLMSG_ALIGN(n->nlmsg_len) + LKL_RTA_ALIGN(len) > maxlen) {
+		lkl_printf("%s ERROR: message exceeded bound of %d\n", __func__,
+			   maxlen);
+		return -1;
+	}
+	rta = ((struct lkl_rtattr *) (((void *) (n)) +
+				      LKL_NLMSG_ALIGN(n->nlmsg_len)));
+	rta->rta_type = type;
+	rta->rta_len = len;
+	memcpy(LKL_RTA_DATA(rta), data, alen);
+	n->nlmsg_len = LKL_NLMSG_ALIGN(n->nlmsg_len) + LKL_RTA_ALIGN(len);
+	return 0;
+}
+
+int lkl_add_neighbor(int ifindex, int af, void *ip, void *mac)
+{
+	struct {
+		struct lkl_nlmsghdr n;
+		struct lkl_ndmsg r;
+		char buf[1024];
+	} req = {
+		.n.nlmsg_len = LKL_NLMSG_LENGTH(sizeof(struct lkl_ndmsg)),
+		.n.nlmsg_type = LKL_RTM_NEWNEIGH,
+		.n.nlmsg_flags = LKL_NLM_F_REQUEST |
+				 LKL_NLM_F_CREATE | LKL_NLM_F_REPLACE,
+		.r.ndm_family = af,
+		.r.ndm_ifindex = ifindex,
+		.r.ndm_state = LKL_NUD_PERMANENT,
+
+	};
+	int err, addr_sz;
+	int fd;
+
+	if (af == LKL_AF_INET)
+		addr_sz = 4;
+	else if (af == LKL_AF_INET6)
+		addr_sz = 16;
+	else {
+		lkl_printf("Bad address family: %d\n", af);
+		return -1;
+	}
+
+	fd = netlink_sock(0);
+	if (fd < 0)
+		return fd;
+
+	// create the IP attribute
+	addattr_l(&req.n, sizeof(req), LKL_NDA_DST, ip, addr_sz);
+
+	// create the MAC attribute
+	addattr_l(&req.n, sizeof(req), LKL_NDA_LLADDR, mac, 6);
+
+	err = rtnl_talk(fd, &req.n);
+	lkl_sys_close(fd);
+	return err;
+}
+
+static int ipaddr_modify(int cmd, int flags, int ifindex, int af, void *addr,
+			 unsigned int netprefix_len)
+{
+	struct {
+		struct lkl_nlmsghdr n;
+		struct lkl_ifaddrmsg ifa;
+		char buf[256];
+	} req = {
+		.n.nlmsg_len = LKL_NLMSG_LENGTH(sizeof(struct lkl_ifaddrmsg)),
+		.n.nlmsg_flags = LKL_NLM_F_REQUEST | flags,
+		.n.nlmsg_type = cmd,
+		.ifa.ifa_family = af,
+		.ifa.ifa_prefixlen = netprefix_len,
+		.ifa.ifa_index = ifindex,
+	};
+	int err, addr_sz;
+	int fd;
+
+	if (af == LKL_AF_INET)
+		addr_sz = 4;
+	else if (af == LKL_AF_INET6)
+		addr_sz = 16;
+	else {
+		lkl_printf("Bad address family: %d\n", af);
+		return -1;
+	}
+
+	fd = netlink_sock(0);
+	if (fd < 0)
+		return fd;
+
+	// create the IP attribute
+	addattr_l(&req.n, sizeof(req), LKL_IFA_LOCAL, addr, addr_sz);
+
+	err = rtnl_talk(fd, &req.n);
+
+	lkl_sys_close(fd);
+	return err;
+}
+
+int lkl_if_add_ip(int ifindex, int af, void *addr, unsigned int netprefix_len)
+{
+	return ipaddr_modify(LKL_RTM_NEWADDR, LKL_NLM_F_CREATE | LKL_NLM_F_EXCL,
+			     ifindex, af, addr, netprefix_len);
+}
+
+int lkl_if_del_ip(int ifindex, int af, void *addr, unsigned int netprefix_len)
+{
+	return ipaddr_modify(LKL_RTM_DELADDR, 0, ifindex, af,
+			     addr, netprefix_len);
+}
+
+static int iproute_modify(int cmd, unsigned int flags, int ifindex, int af,
+		void *route_addr, int route_masklen, void *gwaddr)
+{
+	struct {
+		struct lkl_nlmsghdr	n;
+		struct lkl_rtmsg	r;
+		char			buf[1024];
+	} req = {
+		.n.nlmsg_len = LKL_NLMSG_LENGTH(sizeof(struct lkl_rtmsg)),
+		.n.nlmsg_flags = LKL_NLM_F_REQUEST | flags,
+		.n.nlmsg_type = cmd,
+		.r.rtm_family = af,
+		.r.rtm_table = LKL_RT_TABLE_MAIN,
+		.r.rtm_scope = LKL_RT_SCOPE_UNIVERSE,
+	};
+	int err, addr_sz;
+	int i, fd;
+
+	fd = netlink_sock(0);
+	if (fd < 0) {
+		lkl_printf("netlink_sock error: %d\n", fd);
+		return fd;
+	}
+
+	if (af == LKL_AF_INET)
+		addr_sz = 4;
+	else if (af == LKL_AF_INET6)
+		addr_sz = 16;
+	else {
+		lkl_printf("Bad address family: %d\n", af);
+		return -1;
+	}
+
+	if (cmd != LKL_RTM_DELROUTE) {
+		req.r.rtm_protocol = LKL_RTPROT_BOOT;
+		req.r.rtm_scope = LKL_RT_SCOPE_UNIVERSE;
+		req.r.rtm_type = LKL_RTN_UNICAST;
+	}
+
+	if (gwaddr)
+		addattr_l(&req.n, sizeof(req),
+				LKL_RTA_GATEWAY, gwaddr, addr_sz);
+
+	if (af == LKL_AF_INET && route_addr) {
+		unsigned int netaddr = *(unsigned int *)route_addr;
+
+		netaddr = ntohl(netaddr);
+		netaddr = (netaddr >> (32 - route_masklen));
+		netaddr = (netaddr << (32 - route_masklen));
+		netaddr =  htonl(netaddr);
+		*(unsigned int *)route_addr = netaddr;
+		req.r.rtm_dst_len = route_masklen;
+		addattr_l(&req.n, sizeof(req), LKL_RTA_DST,
+					route_addr, addr_sz);
+	}
+
+	if (af == LKL_AF_INET6 && route_addr) {
+		struct lkl_in6_addr netaddr =
+			*(struct lkl_in6_addr *)route_addr;
+		int rmbyte = route_masklen/8;
+		int rmbit = route_masklen%8;
+
+		for (i = 0; i < rmbyte; i++)
+			netaddr.in6_u.u6_addr8[15-i] = 0;
+		netaddr.in6_u.u6_addr8[15-rmbyte] =
+			(netaddr.in6_u.u6_addr8[15-rmbyte] >> rmbit);
+		netaddr.in6_u.u6_addr8[15-rmbyte] =
+			(netaddr.in6_u.u6_addr8[15-rmbyte] << rmbit);
+		*(struct lkl_in6_addr *)route_addr = netaddr;
+		req.r.rtm_dst_len = route_masklen;
+		addattr_l(&req.n, sizeof(req), LKL_RTA_DST,
+					route_addr, addr_sz);
+	}
+
+	if (ifindex != LKL_RT_TABLE_MAIN) {
+		if (af == LKL_AF_INET)
+			req.r.rtm_table = ifindex * 2;
+		else if (af == LKL_AF_INET6)
+			req.r.rtm_table = ifindex * 2 + 1;
+		addattr_l(&req.n, sizeof(req), LKL_RTA_OIF, &ifindex, addr_sz);
+	}
+	err = rtnl_talk(fd, &req.n);
+	lkl_sys_close(fd);
+	return err;
+}
+
+int lkl_if_add_linklocal(int ifindex, int af,  void *addr, int netprefix_len)
+{
+	return iproute_modify(LKL_RTM_NEWROUTE, LKL_NLM_F_CREATE|LKL_NLM_F_EXCL,
+			ifindex, af, addr, netprefix_len, NULL);
+}
+
+int lkl_if_add_gateway(int ifindex, int af, void *gwaddr)
+{
+	return iproute_modify(LKL_RTM_NEWROUTE, LKL_NLM_F_CREATE|LKL_NLM_F_EXCL,
+			ifindex, af, NULL, 0, gwaddr);
+}
+
+int lkl_add_gateway(int af, void *gwaddr)
+{
+	return iproute_modify(LKL_RTM_NEWROUTE, LKL_NLM_F_CREATE|LKL_NLM_F_EXCL,
+			LKL_RT_TABLE_MAIN, af, NULL, 0, gwaddr);
+}
+
+static int iprule_modify(int cmd, int ifindex, int af, void *saddr)
+{
+	struct {
+		struct lkl_nlmsghdr	n;
+		struct lkl_rtmsg		r;
+		char			buf[1024];
+	} req = {
+		.n.nlmsg_type = cmd,
+		.n.nlmsg_len = LKL_NLMSG_LENGTH(sizeof(struct lkl_rtmsg)),
+		.n.nlmsg_flags = LKL_NLM_F_REQUEST,
+		.r.rtm_protocol = LKL_RTPROT_BOOT,
+		.r.rtm_scope = LKL_RT_SCOPE_UNIVERSE,
+		.r.rtm_family = af,
+		.r.rtm_type = LKL_RTN_UNSPEC,
+	};
+	int fd, err;
+	int addr_sz;
+
+	if (af == LKL_AF_INET)
+		addr_sz = 4;
+	else if (af == LKL_AF_INET6)
+		addr_sz = 16;
+	else {
+		lkl_printf("Bad address family: %d\n", af);
+		return -1;
+	}
+
+	fd = netlink_sock(0);
+	if (fd < 0)
+		return fd;
+
+	if (cmd == LKL_RTM_NEWRULE) {
+		req.n.nlmsg_flags |= LKL_NLM_F_CREATE|LKL_NLM_F_EXCL;
+		req.r.rtm_type = LKL_RTN_UNICAST;
+	}
+
+	//set from address
+	req.r.rtm_src_len = 8 * addr_sz;
+	addattr_l(&req.n, sizeof(req), LKL_FRA_SRC, saddr, addr_sz);
+
+	//use ifindex as table id
+	if (af == LKL_AF_INET)
+		req.r.rtm_table = ifindex * 2;
+	else if (af == LKL_AF_INET6)
+		req.r.rtm_table = ifindex * 2 + 1;
+	err = rtnl_talk(fd, &req.n);
+
+	lkl_sys_close(fd);
+	return err;
+}
+
+int lkl_if_add_rule_from_saddr(int ifindex, int af, void *saddr)
+{
+	return iprule_modify(LKL_RTM_NEWRULE, ifindex, af, saddr);
+}
+
+static int qdisc_add(int cmd, int flags, int ifindex,
+		     const char *root, const char *type)
+{
+	struct {
+		struct lkl_nlmsghdr n;
+		struct lkl_tcmsg tc;
+		char buf[2*1024];
+	} req = {
+		.n.nlmsg_len = LKL_NLMSG_LENGTH(sizeof(struct lkl_tcmsg)),
+		.n.nlmsg_flags = LKL_NLM_F_REQUEST|flags,
+		.n.nlmsg_type = cmd,
+		.tc.tcm_family = LKL_AF_UNSPEC,
+	};
+	int err, fd;
+
+	if (!root || !type) {
+		lkl_printf("root and type arguments\n");
+		return -1;
+	}
+
+	if (strcmp(root, "root") == 0)
+		req.tc.tcm_parent = LKL_TC_H_ROOT;
+	req.tc.tcm_ifindex = ifindex;
+
+	fd = netlink_sock(0);
+	if (fd < 0)
+		return fd;
+
+	// create the qdisc attribute
+	addattr_l(&req.n, sizeof(req), LKL_TCA_KIND, type, strlen(type)+1);
+
+	err = rtnl_talk(fd, &req.n);
+	lkl_sys_close(fd);
+	return err;
+}
+
+int lkl_qdisc_add(int ifindex, const char *root, const char *type)
+{
+	return qdisc_add(LKL_RTM_NEWQDISC, LKL_NLM_F_CREATE | LKL_NLM_F_EXCL,
+			 ifindex, root, type);
+}
+
+/* Add a qdisc entry for an interface in the form of
+ * "root|type;root|type;..."
+ */
+void lkl_qdisc_parse_add(int ifindex, const char *entries)
+{
+	char *saveptr = NULL, *token = NULL;
+	char *root = NULL, *type = NULL;
+	char strings[256];
+	int ret = 0;
+
+	if (strlen(entries) >= sizeof(strings)) {
+		lkl_printf("Error: long strings of input: %s\n", entries);
+		return;
+	}
+
+	strcpy(strings, entries);
+
+	for (token = strtok_r(strings, ";", &saveptr); token;
+	     token = strtok_r(NULL, ";", &saveptr)) {
+		root = strtok(token, "|");
+		type = strtok(NULL, "|");
+		ret = lkl_qdisc_add(ifindex, root, type);
+		if (ret) {
+			lkl_printf("Failed to add qdisc entry: %s\n",
+				   lkl_strerror(ret));
+			return;
+		}
+	}
+}
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 16/25] lkl tools: host lib: posix host operations
  2020-03-30 14:45   ` Hajime Tazaki
@ 2020-03-30 14:45     ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Conrad Meyer, Mark Stillwell, Patrick Collins,
	Pierre-Hugues Husson, Thomas Liebetraut, Yuan Liu, Hajime Tazaki

From: Octavian Purdila <tavi.purdila@gmail.com>

This commit implements LKL host operations for POSIX hosts.  The
operations are implemented to be portable in various POSIX OSs.

Cc: Conrad Meyer <cem@FreeBSD.org>
Cc: Mark Stillwell <mark@stillwell.me>
Cc: Patrick Collins <pscollins@google.com>
Cc: Pierre-Hugues Husson <phh@phh.me>
Cc: Thomas Liebetraut <thomas@tommie-lie.de>
Cc: Yuan Liu <liuyuan@google.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/lkl/lib/Build        |   2 +
 tools/lkl/lib/posix-host.c | 353 +++++++++++++++++++++++++++++++++++++
 2 files changed, 355 insertions(+)
 create mode 100644 tools/lkl/lib/posix-host.c

diff --git a/tools/lkl/lib/Build b/tools/lkl/lib/Build
index dba530424e4a..0e711e260a3a 100644
--- a/tools/lkl/lib/Build
+++ b/tools/lkl/lib/Build
@@ -1,8 +1,10 @@
 CFLAGS_config.o += -I$(srctree)/tools/perf/pmu-events
+CFLAGS_posix-host.o += -D_FILE_OFFSET_BITS=64
 
 liblkl-y += fs.o
 liblkl-y += net.o
 liblkl-y += jmp_buf.o
+liblkl-$(LKL_HOST_CONFIG_POSIX) += posix-host.o
 liblkl-y += utils.o
 liblkl-y += dbg.o
 liblkl-y += dbg_handler.o
diff --git a/tools/lkl/lib/posix-host.c b/tools/lkl/lib/posix-host.c
new file mode 100644
index 000000000000..4be1611a8942
--- /dev/null
+++ b/tools/lkl/lib/posix-host.c
@@ -0,0 +1,353 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <pthread.h>
+#include <stdlib.h>
+#include <sys/time.h>
+#include <time.h>
+#include <signal.h>
+#include <assert.h>
+#include <unistd.h>
+#include <errno.h>
+#include <string.h>
+#include <time.h>
+#include <stdint.h>
+#include <sys/uio.h>
+#include <sys/types.h>
+#include <sys/socket.h>
+#include <sys/syscall.h>
+#include <poll.h>
+#include <lkl_host.h>
+#include "jmp_buf.h"
+
+/* Let's see if the host has semaphore.h */
+#include <unistd.h>
+
+#ifdef _POSIX_SEMAPHORES
+#include <semaphore.h>
+/* TODO(pscollins): We don't support fork() for now, but maybe one day
+ * we will?
+ */
+#define SHARE_SEM 0
+#endif /* _POSIX_SEMAPHORES */
+
+static void print(const char *str, int len)
+{
+	int ret __attribute__((unused));
+
+	ret = write(STDOUT_FILENO, str, len);
+}
+
+struct lkl_mutex {
+	pthread_mutex_t mutex;
+};
+
+struct lkl_sem {
+#ifdef _POSIX_SEMAPHORES
+	sem_t sem;
+#else
+	pthread_mutex_t lock;
+	int count;
+	pthread_cond_t cond;
+#endif /* _POSIX_SEMAPHORES */
+};
+
+struct lkl_tls_key {
+	pthread_key_t key;
+};
+
+#define WARN_UNLESS(exp) do {						\
+		if (exp < 0)						\
+			lkl_printf("%s: %s\n", #exp, strerror(errno));	\
+	} while (0)
+
+static int _warn_pthread(int ret, char *str_exp)
+{
+	if (ret > 0)
+		lkl_printf("%s: %s\n", str_exp, strerror(ret));
+
+	return ret;
+}
+
+
+/* pthread_* functions use the reverse convention */
+#define WARN_PTHREAD(exp) _warn_pthread(exp, #exp)
+
+static struct lkl_sem *sem_alloc(int count)
+{
+	struct lkl_sem *sem;
+
+	sem = malloc(sizeof(*sem));
+	if (!sem)
+		return NULL;
+
+#ifdef _POSIX_SEMAPHORES
+	if (sem_init(&sem->sem, SHARE_SEM, count) < 0) {
+		lkl_printf("sem_init: %s\n", strerror(errno));
+		free(sem);
+		return NULL;
+	}
+#else
+	pthread_mutex_init(&sem->lock, NULL);
+	sem->count = count;
+	WARN_PTHREAD(pthread_cond_init(&sem->cond, NULL));
+#endif /* _POSIX_SEMAPHORES */
+
+	return sem;
+}
+
+static void sem_free(struct lkl_sem *sem)
+{
+#ifdef _POSIX_SEMAPHORES
+	WARN_UNLESS(sem_destroy(&sem->sem));
+#else
+	WARN_PTHREAD(pthread_cond_destroy(&sem->cond));
+	WARN_PTHREAD(pthread_mutex_destroy(&sem->lock));
+#endif /* _POSIX_SEMAPHORES */
+	free(sem);
+}
+
+static void sem_up(struct lkl_sem *sem)
+{
+#ifdef _POSIX_SEMAPHORES
+	WARN_UNLESS(sem_post(&sem->sem));
+#else
+	WARN_PTHREAD(pthread_mutex_lock(&sem->lock));
+	sem->count++;
+	if (sem->count > 0)
+		WARN_PTHREAD(pthread_cond_signal(&sem->cond));
+	WARN_PTHREAD(pthread_mutex_unlock(&sem->lock));
+#endif /* _POSIX_SEMAPHORES */
+
+}
+
+static void sem_down(struct lkl_sem *sem)
+{
+#ifdef _POSIX_SEMAPHORES
+	int err;
+
+	do {
+		err = sem_wait(&sem->sem);
+	} while (err < 0 && errno == EINTR);
+	if (err < 0 && errno != EINTR)
+		lkl_printf("sem_wait: %s\n", strerror(errno));
+#else
+	WARN_PTHREAD(pthread_mutex_lock(&sem->lock));
+	while (sem->count <= 0)
+		WARN_PTHREAD(pthread_cond_wait(&sem->cond, &sem->lock));
+	sem->count--;
+	WARN_PTHREAD(pthread_mutex_unlock(&sem->lock));
+#endif /* _POSIX_SEMAPHORES */
+}
+
+static struct lkl_mutex *mutex_alloc(int recursive)
+{
+	struct lkl_mutex *_mutex = malloc(sizeof(struct lkl_mutex));
+	pthread_mutex_t *mutex = NULL;
+	pthread_mutexattr_t attr;
+
+	if (!_mutex)
+		return NULL;
+
+	mutex = &_mutex->mutex;
+	WARN_PTHREAD(pthread_mutexattr_init(&attr));
+
+	/* PTHREAD_MUTEX_ERRORCHECK is *very* useful for debugging,
+	 * but has some overhead, so we provide an option to turn it
+	 * off.
+	 */
+#ifdef DEBUG
+	if (!recursive)
+		WARN_PTHREAD(pthread_mutexattr_settype(
+				     &attr, PTHREAD_MUTEX_ERRORCHECK));
+#endif /* DEBUG */
+
+	if (recursive)
+		WARN_PTHREAD(pthread_mutexattr_settype(
+				     &attr, PTHREAD_MUTEX_RECURSIVE));
+
+	WARN_PTHREAD(pthread_mutex_init(mutex, &attr));
+
+	return _mutex;
+}
+
+static void mutex_lock(struct lkl_mutex *mutex)
+{
+	WARN_PTHREAD(pthread_mutex_lock(&mutex->mutex));
+}
+
+static void mutex_unlock(struct lkl_mutex *_mutex)
+{
+	pthread_mutex_t *mutex = &_mutex->mutex;
+
+	WARN_PTHREAD(pthread_mutex_unlock(mutex));
+}
+
+static void mutex_free(struct lkl_mutex *_mutex)
+{
+	pthread_mutex_t *mutex = &_mutex->mutex;
+
+	WARN_PTHREAD(pthread_mutex_destroy(mutex));
+	free(_mutex);
+}
+
+static lkl_thread_t thread_create(void (*fn)(void *), void *arg)
+{
+	pthread_t thread;
+
+	if (WARN_PTHREAD(pthread_create(&thread, NULL, (void *)fn,
+					arg)))
+		return 0;
+	else
+		return (lkl_thread_t) thread;
+}
+
+static void thread_detach(void)
+{
+	WARN_PTHREAD(pthread_detach(pthread_self()));
+}
+
+static void thread_exit(void)
+{
+	pthread_exit(NULL);
+}
+
+static int thread_join(lkl_thread_t tid)
+{
+	if (WARN_PTHREAD(pthread_join((pthread_t)tid, NULL)))
+		return (-1);
+	else
+		return 0;
+}
+
+static lkl_thread_t thread_self(void)
+{
+	return (lkl_thread_t)pthread_self();
+}
+
+static int thread_equal(lkl_thread_t a, lkl_thread_t b)
+{
+	return pthread_equal((pthread_t)a, (pthread_t)b);
+}
+
+static struct lkl_tls_key *tls_alloc(void (*destructor)(void *))
+{
+	struct lkl_tls_key *ret = malloc(sizeof(struct lkl_tls_key));
+
+	if (WARN_PTHREAD(pthread_key_create(&ret->key, destructor))) {
+		free(ret);
+		return NULL;
+	}
+	return ret;
+}
+
+static void tls_free(struct lkl_tls_key *key)
+{
+	WARN_PTHREAD(pthread_key_delete(key->key));
+	free(key);
+}
+
+static int tls_set(struct lkl_tls_key *key, void *data)
+{
+	if (WARN_PTHREAD(pthread_setspecific(key->key, data)))
+		return (-1);
+	return 0;
+}
+
+static void *tls_get(struct lkl_tls_key *key)
+{
+	return pthread_getspecific(key->key);
+}
+
+static unsigned long long time_ns(void)
+{
+	struct timespec ts;
+
+	clock_gettime(CLOCK_MONOTONIC, &ts);
+
+	return 1e9*ts.tv_sec + ts.tv_nsec;
+}
+
+static void *timer_alloc(void (*fn)(void *), void *arg)
+{
+	int err;
+	timer_t timer;
+	struct sigevent se =  {
+		.sigev_notify = SIGEV_THREAD,
+		.sigev_value = {
+			.sival_ptr = arg,
+		},
+		.sigev_notify_function = (void *)fn,
+	};
+
+	err = timer_create(CLOCK_REALTIME, &se, &timer);
+	if (err)
+		return NULL;
+
+	return (void *)(long)timer;
+}
+
+static int timer_set_oneshot(void *_timer, unsigned long ns)
+{
+	timer_t timer = (timer_t)(long)_timer;
+	struct itimerspec ts = {
+		.it_value = {
+			.tv_sec = ns / 1000000000,
+			.tv_nsec = ns % 1000000000,
+		},
+	};
+
+	return timer_settime(timer, 0, &ts, NULL);
+}
+
+static void timer_free(void *_timer)
+{
+	timer_t timer = (timer_t)(long)_timer;
+
+	timer_delete(timer);
+}
+
+static void panic(void)
+{
+	assert(0);
+}
+
+static long _gettid(void)
+{
+#ifdef	__FreeBSD__
+	return (long)pthread_self();
+#else
+	return syscall(SYS_gettid);
+#endif
+}
+
+struct lkl_host_operations lkl_host_ops = {
+	.panic = panic,
+	.thread_create = thread_create,
+	.thread_detach = thread_detach,
+	.thread_exit = thread_exit,
+	.thread_join = thread_join,
+	.thread_self = thread_self,
+	.thread_equal = thread_equal,
+	.sem_alloc = sem_alloc,
+	.sem_free = sem_free,
+	.sem_up = sem_up,
+	.sem_down = sem_down,
+	.mutex_alloc = mutex_alloc,
+	.mutex_free = mutex_free,
+	.mutex_lock = mutex_lock,
+	.mutex_unlock = mutex_unlock,
+	.tls_alloc = tls_alloc,
+	.tls_free = tls_free,
+	.tls_set = tls_set,
+	.tls_get = tls_get,
+	.time = time_ns,
+	.timer_alloc = timer_alloc,
+	.timer_set_oneshot = timer_set_oneshot,
+	.timer_free = timer_free,
+	.print = print,
+	.mem_alloc = (void *)malloc,
+	.mem_free = free,
+	.gettid = _gettid,
+	.jmp_buf_set = jmp_buf_set,
+	.jmp_buf_longjmp = jmp_buf_longjmp,
+};
+
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 16/25] lkl tools: host lib: posix host operations
@ 2020-03-30 14:45     ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: linux-arch, Hajime Tazaki, Conrad Meyer, Octavian Purdila,
	Akira Moroo, Thomas Liebetraut, Mark Stillwell, Patrick Collins,
	linux-kernel-library, Pierre-Hugues Husson, Yuan Liu

From: Octavian Purdila <tavi.purdila@gmail.com>

This commit implements LKL host operations for POSIX hosts.  The
operations are implemented to be portable in various POSIX OSs.

Cc: Conrad Meyer <cem@FreeBSD.org>
Cc: Mark Stillwell <mark@stillwell.me>
Cc: Patrick Collins <pscollins@google.com>
Cc: Pierre-Hugues Husson <phh@phh.me>
Cc: Thomas Liebetraut <thomas@tommie-lie.de>
Cc: Yuan Liu <liuyuan@google.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/lkl/lib/Build        |   2 +
 tools/lkl/lib/posix-host.c | 353 +++++++++++++++++++++++++++++++++++++
 2 files changed, 355 insertions(+)
 create mode 100644 tools/lkl/lib/posix-host.c

diff --git a/tools/lkl/lib/Build b/tools/lkl/lib/Build
index dba530424e4a..0e711e260a3a 100644
--- a/tools/lkl/lib/Build
+++ b/tools/lkl/lib/Build
@@ -1,8 +1,10 @@
 CFLAGS_config.o += -I$(srctree)/tools/perf/pmu-events
+CFLAGS_posix-host.o += -D_FILE_OFFSET_BITS=64
 
 liblkl-y += fs.o
 liblkl-y += net.o
 liblkl-y += jmp_buf.o
+liblkl-$(LKL_HOST_CONFIG_POSIX) += posix-host.o
 liblkl-y += utils.o
 liblkl-y += dbg.o
 liblkl-y += dbg_handler.o
diff --git a/tools/lkl/lib/posix-host.c b/tools/lkl/lib/posix-host.c
new file mode 100644
index 000000000000..4be1611a8942
--- /dev/null
+++ b/tools/lkl/lib/posix-host.c
@@ -0,0 +1,353 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <pthread.h>
+#include <stdlib.h>
+#include <sys/time.h>
+#include <time.h>
+#include <signal.h>
+#include <assert.h>
+#include <unistd.h>
+#include <errno.h>
+#include <string.h>
+#include <time.h>
+#include <stdint.h>
+#include <sys/uio.h>
+#include <sys/types.h>
+#include <sys/socket.h>
+#include <sys/syscall.h>
+#include <poll.h>
+#include <lkl_host.h>
+#include "jmp_buf.h"
+
+/* Let's see if the host has semaphore.h */
+#include <unistd.h>
+
+#ifdef _POSIX_SEMAPHORES
+#include <semaphore.h>
+/* TODO(pscollins): We don't support fork() for now, but maybe one day
+ * we will?
+ */
+#define SHARE_SEM 0
+#endif /* _POSIX_SEMAPHORES */
+
+static void print(const char *str, int len)
+{
+	int ret __attribute__((unused));
+
+	ret = write(STDOUT_FILENO, str, len);
+}
+
+struct lkl_mutex {
+	pthread_mutex_t mutex;
+};
+
+struct lkl_sem {
+#ifdef _POSIX_SEMAPHORES
+	sem_t sem;
+#else
+	pthread_mutex_t lock;
+	int count;
+	pthread_cond_t cond;
+#endif /* _POSIX_SEMAPHORES */
+};
+
+struct lkl_tls_key {
+	pthread_key_t key;
+};
+
+#define WARN_UNLESS(exp) do {						\
+		if (exp < 0)						\
+			lkl_printf("%s: %s\n", #exp, strerror(errno));	\
+	} while (0)
+
+static int _warn_pthread(int ret, char *str_exp)
+{
+	if (ret > 0)
+		lkl_printf("%s: %s\n", str_exp, strerror(ret));
+
+	return ret;
+}
+
+
+/* pthread_* functions use the reverse convention */
+#define WARN_PTHREAD(exp) _warn_pthread(exp, #exp)
+
+static struct lkl_sem *sem_alloc(int count)
+{
+	struct lkl_sem *sem;
+
+	sem = malloc(sizeof(*sem));
+	if (!sem)
+		return NULL;
+
+#ifdef _POSIX_SEMAPHORES
+	if (sem_init(&sem->sem, SHARE_SEM, count) < 0) {
+		lkl_printf("sem_init: %s\n", strerror(errno));
+		free(sem);
+		return NULL;
+	}
+#else
+	pthread_mutex_init(&sem->lock, NULL);
+	sem->count = count;
+	WARN_PTHREAD(pthread_cond_init(&sem->cond, NULL));
+#endif /* _POSIX_SEMAPHORES */
+
+	return sem;
+}
+
+static void sem_free(struct lkl_sem *sem)
+{
+#ifdef _POSIX_SEMAPHORES
+	WARN_UNLESS(sem_destroy(&sem->sem));
+#else
+	WARN_PTHREAD(pthread_cond_destroy(&sem->cond));
+	WARN_PTHREAD(pthread_mutex_destroy(&sem->lock));
+#endif /* _POSIX_SEMAPHORES */
+	free(sem);
+}
+
+static void sem_up(struct lkl_sem *sem)
+{
+#ifdef _POSIX_SEMAPHORES
+	WARN_UNLESS(sem_post(&sem->sem));
+#else
+	WARN_PTHREAD(pthread_mutex_lock(&sem->lock));
+	sem->count++;
+	if (sem->count > 0)
+		WARN_PTHREAD(pthread_cond_signal(&sem->cond));
+	WARN_PTHREAD(pthread_mutex_unlock(&sem->lock));
+#endif /* _POSIX_SEMAPHORES */
+
+}
+
+static void sem_down(struct lkl_sem *sem)
+{
+#ifdef _POSIX_SEMAPHORES
+	int err;
+
+	do {
+		err = sem_wait(&sem->sem);
+	} while (err < 0 && errno == EINTR);
+	if (err < 0 && errno != EINTR)
+		lkl_printf("sem_wait: %s\n", strerror(errno));
+#else
+	WARN_PTHREAD(pthread_mutex_lock(&sem->lock));
+	while (sem->count <= 0)
+		WARN_PTHREAD(pthread_cond_wait(&sem->cond, &sem->lock));
+	sem->count--;
+	WARN_PTHREAD(pthread_mutex_unlock(&sem->lock));
+#endif /* _POSIX_SEMAPHORES */
+}
+
+static struct lkl_mutex *mutex_alloc(int recursive)
+{
+	struct lkl_mutex *_mutex = malloc(sizeof(struct lkl_mutex));
+	pthread_mutex_t *mutex = NULL;
+	pthread_mutexattr_t attr;
+
+	if (!_mutex)
+		return NULL;
+
+	mutex = &_mutex->mutex;
+	WARN_PTHREAD(pthread_mutexattr_init(&attr));
+
+	/* PTHREAD_MUTEX_ERRORCHECK is *very* useful for debugging,
+	 * but has some overhead, so we provide an option to turn it
+	 * off.
+	 */
+#ifdef DEBUG
+	if (!recursive)
+		WARN_PTHREAD(pthread_mutexattr_settype(
+				     &attr, PTHREAD_MUTEX_ERRORCHECK));
+#endif /* DEBUG */
+
+	if (recursive)
+		WARN_PTHREAD(pthread_mutexattr_settype(
+				     &attr, PTHREAD_MUTEX_RECURSIVE));
+
+	WARN_PTHREAD(pthread_mutex_init(mutex, &attr));
+
+	return _mutex;
+}
+
+static void mutex_lock(struct lkl_mutex *mutex)
+{
+	WARN_PTHREAD(pthread_mutex_lock(&mutex->mutex));
+}
+
+static void mutex_unlock(struct lkl_mutex *_mutex)
+{
+	pthread_mutex_t *mutex = &_mutex->mutex;
+
+	WARN_PTHREAD(pthread_mutex_unlock(mutex));
+}
+
+static void mutex_free(struct lkl_mutex *_mutex)
+{
+	pthread_mutex_t *mutex = &_mutex->mutex;
+
+	WARN_PTHREAD(pthread_mutex_destroy(mutex));
+	free(_mutex);
+}
+
+static lkl_thread_t thread_create(void (*fn)(void *), void *arg)
+{
+	pthread_t thread;
+
+	if (WARN_PTHREAD(pthread_create(&thread, NULL, (void *)fn,
+					arg)))
+		return 0;
+	else
+		return (lkl_thread_t) thread;
+}
+
+static void thread_detach(void)
+{
+	WARN_PTHREAD(pthread_detach(pthread_self()));
+}
+
+static void thread_exit(void)
+{
+	pthread_exit(NULL);
+}
+
+static int thread_join(lkl_thread_t tid)
+{
+	if (WARN_PTHREAD(pthread_join((pthread_t)tid, NULL)))
+		return (-1);
+	else
+		return 0;
+}
+
+static lkl_thread_t thread_self(void)
+{
+	return (lkl_thread_t)pthread_self();
+}
+
+static int thread_equal(lkl_thread_t a, lkl_thread_t b)
+{
+	return pthread_equal((pthread_t)a, (pthread_t)b);
+}
+
+static struct lkl_tls_key *tls_alloc(void (*destructor)(void *))
+{
+	struct lkl_tls_key *ret = malloc(sizeof(struct lkl_tls_key));
+
+	if (WARN_PTHREAD(pthread_key_create(&ret->key, destructor))) {
+		free(ret);
+		return NULL;
+	}
+	return ret;
+}
+
+static void tls_free(struct lkl_tls_key *key)
+{
+	WARN_PTHREAD(pthread_key_delete(key->key));
+	free(key);
+}
+
+static int tls_set(struct lkl_tls_key *key, void *data)
+{
+	if (WARN_PTHREAD(pthread_setspecific(key->key, data)))
+		return (-1);
+	return 0;
+}
+
+static void *tls_get(struct lkl_tls_key *key)
+{
+	return pthread_getspecific(key->key);
+}
+
+static unsigned long long time_ns(void)
+{
+	struct timespec ts;
+
+	clock_gettime(CLOCK_MONOTONIC, &ts);
+
+	return 1e9*ts.tv_sec + ts.tv_nsec;
+}
+
+static void *timer_alloc(void (*fn)(void *), void *arg)
+{
+	int err;
+	timer_t timer;
+	struct sigevent se =  {
+		.sigev_notify = SIGEV_THREAD,
+		.sigev_value = {
+			.sival_ptr = arg,
+		},
+		.sigev_notify_function = (void *)fn,
+	};
+
+	err = timer_create(CLOCK_REALTIME, &se, &timer);
+	if (err)
+		return NULL;
+
+	return (void *)(long)timer;
+}
+
+static int timer_set_oneshot(void *_timer, unsigned long ns)
+{
+	timer_t timer = (timer_t)(long)_timer;
+	struct itimerspec ts = {
+		.it_value = {
+			.tv_sec = ns / 1000000000,
+			.tv_nsec = ns % 1000000000,
+		},
+	};
+
+	return timer_settime(timer, 0, &ts, NULL);
+}
+
+static void timer_free(void *_timer)
+{
+	timer_t timer = (timer_t)(long)_timer;
+
+	timer_delete(timer);
+}
+
+static void panic(void)
+{
+	assert(0);
+}
+
+static long _gettid(void)
+{
+#ifdef	__FreeBSD__
+	return (long)pthread_self();
+#else
+	return syscall(SYS_gettid);
+#endif
+}
+
+struct lkl_host_operations lkl_host_ops = {
+	.panic = panic,
+	.thread_create = thread_create,
+	.thread_detach = thread_detach,
+	.thread_exit = thread_exit,
+	.thread_join = thread_join,
+	.thread_self = thread_self,
+	.thread_equal = thread_equal,
+	.sem_alloc = sem_alloc,
+	.sem_free = sem_free,
+	.sem_up = sem_up,
+	.sem_down = sem_down,
+	.mutex_alloc = mutex_alloc,
+	.mutex_free = mutex_free,
+	.mutex_lock = mutex_lock,
+	.mutex_unlock = mutex_unlock,
+	.tls_alloc = tls_alloc,
+	.tls_free = tls_free,
+	.tls_set = tls_set,
+	.tls_get = tls_get,
+	.time = time_ns,
+	.timer_alloc = timer_alloc,
+	.timer_set_oneshot = timer_set_oneshot,
+	.timer_free = timer_free,
+	.print = print,
+	.mem_alloc = (void *)malloc,
+	.mem_free = free,
+	.gettid = _gettid,
+	.jmp_buf_set = jmp_buf_set,
+	.jmp_buf_longjmp = jmp_buf_longjmp,
+};
+
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 17/25] lkl tools: add test programs
  2020-03-30 14:45   ` Hajime Tazaki
@ 2020-03-30 14:45     ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Conrad Meyer, David Disseldorp, Lai Jiangshan, Luca Dariz,
	Mark Stillwell, Motomu Utsumi, Patrick Collins, Petros Angelatos,
	Thomas Liebetraut, Yuan Liu, Hajime Tazaki

From: Octavian Purdila <tavi.purdila@gmail.com>

Add a simple LKL test application (boot) that starts the kernel and
performs simple tests that minimally exercise the LKL API. Networking
and block device tests are also added which are useful to detect
regressions.

Cc: Conrad Meyer <cem@FreeBSD.org>
Cc: David Disseldorp <ddiss@suse.de>
Cc: H.K. Jerry Chu <hkchu@google.com>
Cc: Lai Jiangshan <jiangshanlai@gmail.com>
Cc: Luca Dariz <luca.dariz@gmail.com>
Cc: Mark Stillwell <mark@stillwell.me>
Cc: Motomu Utsumi <motomuman@gmail.com>
Cc: Patrick Collins <pscollins@google.com>
Cc: Petros Angelatos <petrosagg@gmail.com>
Cc: Thomas Liebetraut <thomas@tommie-lie.de>
Cc: Yuan Liu <liuyuan@google.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/lkl/.gitignore              |   5 +
 tools/lkl/Makefile                |   5 +-
 tools/lkl/Targets                 |   3 +
 tools/lkl/tests/Build             |   3 +
 tools/lkl/tests/boot.c            | 562 ++++++++++++++++++++++++++++++
 tools/lkl/tests/boot.sh           |   9 +
 tools/lkl/tests/cla.c             | 159 +++++++++
 tools/lkl/tests/cla.h             |  33 ++
 tools/lkl/tests/disk.c            | 189 ++++++++++
 tools/lkl/tests/disk.sh           |  61 ++++
 tools/lkl/tests/net-setup.sh      |  73 ++++
 tools/lkl/tests/net-test.c        | 299 ++++++++++++++++
 tools/lkl/tests/net.sh            |  90 +++++
 tools/lkl/tests/run.py            | 179 ++++++++++
 tools/lkl/tests/tap13.py          | 209 +++++++++++
 tools/lkl/tests/test.c            | 126 +++++++
 tools/lkl/tests/test.h            |  72 ++++
 tools/lkl/tests/test.sh           | 182 ++++++++++
 tools/lkl/tests/valgrind.supp     | 105 ++++++
 tools/lkl/tests/valgrind2xunit.py |  69 ++++
 20 files changed, 2432 insertions(+), 1 deletion(-)
 create mode 100644 tools/lkl/tests/Build
 create mode 100644 tools/lkl/tests/boot.c
 create mode 100755 tools/lkl/tests/boot.sh
 create mode 100644 tools/lkl/tests/cla.c
 create mode 100644 tools/lkl/tests/cla.h
 create mode 100644 tools/lkl/tests/disk.c
 create mode 100755 tools/lkl/tests/disk.sh
 create mode 100644 tools/lkl/tests/net-setup.sh
 create mode 100644 tools/lkl/tests/net-test.c
 create mode 100755 tools/lkl/tests/net.sh
 create mode 100755 tools/lkl/tests/run.py
 create mode 100644 tools/lkl/tests/tap13.py
 create mode 100644 tools/lkl/tests/test.c
 create mode 100644 tools/lkl/tests/test.h
 create mode 100644 tools/lkl/tests/test.sh
 create mode 100644 tools/lkl/tests/valgrind.supp
 create mode 100755 tools/lkl/tests/valgrind2xunit.py

diff --git a/tools/lkl/.gitignore b/tools/lkl/.gitignore
index 1aed58bfe171..4e08254dbd46 100644
--- a/tools/lkl/.gitignore
+++ b/tools/lkl/.gitignore
@@ -2,3 +2,8 @@ Makefile.conf
 include/lkl_autoconf.h
 tests/autoconf.sh
 bin/stat
+tests/net-test
+tests/disk
+tests/boot
+tests/valgrind*.xml
+*.pyc
diff --git a/tools/lkl/Makefile b/tools/lkl/Makefile
index 11ea7e9095bb..0abccb9d86b6 100644
--- a/tools/lkl/Makefile
+++ b/tools/lkl/Makefile
@@ -117,8 +117,11 @@ programs_install: $(progs-y:%=$(OUTPUT)%$(EXESUF))
 install: headers_install libraries_install programs_install
 
 
+run-tests:
+	./tests/run.py $(tests)
+
 FORCE: ;
-.PHONY: all clean FORCE
+.PHONY: all clean FORCE run-tests
 .PHONY: headers_install libraries_install programs_install install
 .NOTPARALLEL : lib/lkl.o
 .SECONDARY:
diff --git a/tools/lkl/Targets b/tools/lkl/Targets
index 24c985e64638..a9f74c3cc8fb 100644
--- a/tools/lkl/Targets
+++ b/tools/lkl/Targets
@@ -1,3 +1,6 @@
 libs-y += lib/liblkl
 
+progs-y += tests/boot
+progs-y += tests/disk
+progs-y += tests/net-test
 
diff --git a/tools/lkl/tests/Build b/tools/lkl/tests/Build
new file mode 100644
index 000000000000..ace86a3d3438
--- /dev/null
+++ b/tools/lkl/tests/Build
@@ -0,0 +1,3 @@
+boot-y += boot.o test.o
+disk-y += disk.o cla.o test.o
+net-test-y += net-test.o cla.o test.o
diff --git a/tools/lkl/tests/boot.c b/tools/lkl/tests/boot.c
new file mode 100644
index 000000000000..b021e9540147
--- /dev/null
+++ b/tools/lkl/tests/boot.c
@@ -0,0 +1,562 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <unistd.h>
+#include <string.h>
+#include <time.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <lkl.h>
+#include <lkl_host.h>
+
+#include <sys/stat.h>
+#include <fcntl.h>
+#if defined(__FreeBSD__)
+#include <net/if.h>
+#include <sys/ioctl.h>
+#elif __linux
+#include <sys/epoll.h>
+#include <sys/ioctl.h>
+#elif __MINGW32__
+#include <windows.h>
+#endif
+
+#include "test.h"
+
+#ifndef __MINGW32__
+#define sleep_ns 87654321
+int lkl_test_nanosleep(void)
+{
+	struct lkl_timespec ts = {
+		.tv_sec = 0,
+		.tv_nsec = sleep_ns,
+	};
+	struct timespec start, stop;
+	long delta;
+	long ret;
+
+	clock_gettime(CLOCK_MONOTONIC, &start);
+	ret = lkl_sys_nanosleep((struct __lkl__kernel_timespec *)&ts, NULL);
+	clock_gettime(CLOCK_MONOTONIC, &stop);
+
+	delta = 1e9*(stop.tv_sec - start.tv_sec) +
+		(stop.tv_nsec - start.tv_nsec);
+
+	lkl_test_logf("sleep %ld, expected sleep %d\n", delta, sleep_ns);
+
+	if (ret == 0 && delta > sleep_ns * 0.9)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+#endif
+
+LKL_TEST_CALL(getpid, lkl_sys_getpid, 1)
+
+void check_latency(long (*f)(void), long *min, long *max, long *avg)
+{
+	int i;
+	unsigned long long start, stop, sum = 0;
+	static const int count = 1000;
+	long delta;
+
+	*min = 1000000000;
+	*max = -1;
+
+	for (i = 0; i < count; i++) {
+		start = lkl_host_ops.time();
+		f();
+		stop = lkl_host_ops.time();
+		delta = stop - start;
+		if (*min > delta)
+			*min = delta;
+		if (*max < delta)
+			*max = delta;
+		sum += delta;
+	}
+	*avg = sum / count;
+}
+
+static long native_getpid(void)
+{
+#ifdef __MINGW32__
+	GetCurrentProcessId();
+#else
+	getpid();
+#endif
+	return 0;
+}
+
+int lkl_test_syscall_latency(void)
+{
+	long min, max, avg;
+
+	lkl_test_logf("avg/min/max: ");
+
+	check_latency(lkl_sys_getpid, &min, &max, &avg);
+
+	lkl_test_logf("lkl:%ld/%ld/%ld ", avg, min, max);
+
+	check_latency(native_getpid, &min, &max, &avg);
+
+	lkl_test_logf("native:%ld/%ld/%ld\n", avg, min, max);
+
+	return TEST_SUCCESS;
+}
+
+#define access_rights 0721
+
+LKL_TEST_CALL(creat, lkl_sys_creat, 0, "/file", access_rights)
+LKL_TEST_CALL(close, lkl_sys_close, 0, 0);
+LKL_TEST_CALL(failopen, lkl_sys_open, -LKL_ENOENT, "/file2", 0, 0);
+LKL_TEST_CALL(umask, lkl_sys_umask, 022,  0777);
+LKL_TEST_CALL(umask2, lkl_sys_umask, 0777, 0);
+LKL_TEST_CALL(open, lkl_sys_open, 0, "/file", LKL_O_RDWR, 0);
+static const char wrbuf[] = "test";
+LKL_TEST_CALL(write, lkl_sys_write, sizeof(wrbuf), 0, wrbuf, sizeof(wrbuf));
+LKL_TEST_CALL(lseek_cur, lkl_sys_lseek, sizeof(wrbuf), 0, 0, LKL_SEEK_CUR);
+LKL_TEST_CALL(lseek_end, lkl_sys_lseek, sizeof(wrbuf), 0, 0, LKL_SEEK_END);
+LKL_TEST_CALL(lseek_set, lkl_sys_lseek, 0, 0, 0, LKL_SEEK_SET);
+
+int lkl_test_read(void)
+{
+	char buf[10] = { 0, };
+	long ret;
+
+	ret = lkl_sys_read(0, buf, sizeof(buf));
+
+	lkl_test_logf("lkl_sys_read=%ld buf=%s\n", ret, buf);
+
+	if (ret == sizeof(wrbuf) && !strcmp(wrbuf, buf))
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+int lkl_test_fstat(void)
+{
+	struct lkl_stat stat;
+	long ret;
+
+	ret = lkl_sys_fstat(0, &stat);
+
+	lkl_test_logf("lkl_sys_fstat=%ld mode=%o size=%zd\n", ret, stat.st_mode,
+		      stat.st_size);
+
+	if (ret == 0 && stat.st_size == sizeof(wrbuf) &&
+	    stat.st_mode == (access_rights | LKL_S_IFREG))
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+LKL_TEST_CALL(mkdir, lkl_sys_mkdir, 0, "/mnt", access_rights)
+
+int lkl_test_stat(void)
+{
+	struct lkl_stat stat;
+	long ret;
+
+	ret = lkl_sys_stat("/mnt", &stat);
+
+	lkl_test_logf("lkl_sys_stat(\"/mnt\")=%ld mode=%o\n", ret,
+		      stat.st_mode);
+
+	if (ret == 0 && stat.st_mode == (access_rights | LKL_S_IFDIR))
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+static int lkl_test_pipe2(void)
+{
+	int pipe_fds[2];
+	int READ_IDX = 0, WRITE_IDX = 1;
+	static const char msg[] = "Hello world!";
+	char str[20];
+	int msg_len_bytes = strlen(msg) + 1;
+	int cmp_res;
+	long ret;
+
+	ret = lkl_sys_pipe2(pipe_fds, LKL_O_NONBLOCK);
+	if (ret) {
+		lkl_test_logf("pipe2: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_write(pipe_fds[WRITE_IDX], msg, msg_len_bytes);
+	if (ret != msg_len_bytes) {
+		if (ret < 0)
+			lkl_test_logf("write error: %s\n", lkl_strerror(ret));
+		else
+			lkl_test_logf("short write: %ld\n", ret);
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_read(pipe_fds[READ_IDX], str, msg_len_bytes);
+	if (ret != msg_len_bytes) {
+		if (ret < 0)
+			lkl_test_logf("read error: %s\n", lkl_strerror(ret));
+		else
+			lkl_test_logf("short read: %ld\n", ret);
+		return TEST_FAILURE;
+	}
+
+	cmp_res = memcmp(msg, str, msg_len_bytes);
+	if (cmp_res) {
+		lkl_test_logf("memcmp failed: %d\n", cmp_res);
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_close(pipe_fds[0]);
+	if (ret) {
+		lkl_test_logf("close error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_close(pipe_fds[1]);
+	if (ret) {
+		lkl_test_logf("close error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	return TEST_SUCCESS;
+}
+
+static int lkl_test_epoll(void)
+{
+	int epoll_fd, pipe_fds[2];
+	int READ_IDX = 0, WRITE_IDX = 1;
+	struct lkl_epoll_event wait_on, read_result;
+	static const char msg[] = "Hello world!";
+	long ret;
+
+	memset(&wait_on, 0, sizeof(wait_on));
+	memset(&read_result, 0, sizeof(read_result));
+
+	ret = lkl_sys_pipe2(pipe_fds, LKL_O_NONBLOCK);
+	if (ret) {
+		lkl_test_logf("pipe2 error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	epoll_fd = lkl_sys_epoll_create(1);
+	if (epoll_fd < 0) {
+		lkl_test_logf("epoll_create error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	wait_on.events = LKL_POLLIN | LKL_POLLOUT;
+	wait_on.data = pipe_fds[READ_IDX];
+
+	ret = lkl_sys_epoll_ctl(epoll_fd, LKL_EPOLL_CTL_ADD, pipe_fds[READ_IDX],
+				&wait_on);
+	if (ret < 0) {
+		lkl_test_logf("epoll_ctl error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	/* Shouldn't be ready before we have written something */
+	ret = lkl_sys_epoll_wait(epoll_fd, &read_result, 1, 0);
+	if (ret != 0) {
+		if (ret < 0)
+			lkl_test_logf("epoll_wait error: %s\n",
+				      lkl_strerror(ret));
+		else
+			lkl_test_logf("epoll_wait: bad event: 0x%lx\n", ret);
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_write(pipe_fds[WRITE_IDX], msg, strlen(msg) + 1);
+	if (ret < 0) {
+		lkl_test_logf("write error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	/* We expect exactly 1 fd to be ready immediately */
+	ret = lkl_sys_epoll_wait(epoll_fd, &read_result, 1, 0);
+	if (ret != 1) {
+		if (ret < 0)
+			lkl_test_logf("epoll_wait error: %s\n",
+				      lkl_strerror(ret));
+		else
+			lkl_test_logf("epoll_wait: bad ev no %ld\n", ret);
+		return TEST_FAILURE;
+	}
+
+	/* Already tested reading from pipe2 so no need to do it
+	 * here
+	 */
+
+	return TEST_SUCCESS;
+}
+
+LKL_TEST_CALL(chdir_proc, lkl_sys_chdir, 0, "proc");
+
+static int dir_fd;
+
+static int lkl_test_open_cwd(void)
+{
+	dir_fd = lkl_sys_open(".", LKL_O_RDONLY | LKL_O_DIRECTORY, 0);
+	if (dir_fd < 0) {
+		lkl_test_logf("failed to open current directory: %s\n",
+			      lkl_strerror(dir_fd));
+		return TEST_FAILURE;
+	}
+
+	return TEST_SUCCESS;
+}
+
+/* column where to insert a line break for the list file tests below. */
+#define COL_LINE_BREAK 70
+
+static int lkl_test_getdents64(void)
+{
+	long ret;
+	char buf[1024], *pos;
+	struct lkl_linux_dirent64 *de;
+	int wr;
+
+	de = (struct lkl_linux_dirent64 *)buf;
+	ret = lkl_sys_getdents64(dir_fd, de, sizeof(buf));
+
+	wr = lkl_test_logf("%d ", dir_fd);
+
+	if (ret < 0)
+		return TEST_FAILURE;
+
+	for (pos = buf; pos - buf < ret; pos += de->d_reclen) {
+		de = (struct lkl_linux_dirent64 *)pos;
+
+		wr += lkl_test_logf("%s ", de->d_name);
+		if (wr >= COL_LINE_BREAK) {
+			lkl_test_logf("\n");
+			wr = 0;
+		}
+	}
+
+	return TEST_SUCCESS;
+}
+
+LKL_TEST_CALL(close_dir_fd, lkl_sys_close, 0, dir_fd);
+LKL_TEST_CALL(chdir_root, lkl_sys_chdir, 0, "/");
+LKL_TEST_CALL(mount_fs_proc, lkl_mount_fs, 0, "proc");
+LKL_TEST_CALL(umount_fs_proc, lkl_umount_timeout, 0, "proc", 0, 1000);
+LKL_TEST_CALL(lo_ifup, lkl_if_up, 0, 1);
+
+static int lkl_test_mutex(void)
+{
+	long ret = TEST_SUCCESS;
+	/*
+	 * Can't do much to verify that this works, so we'll just let Valgrind
+	 * warn us on CI if we've made bad memory accesses.
+	 */
+
+	struct lkl_mutex *mutex;
+
+	mutex = lkl_host_ops.mutex_alloc(0);
+	lkl_host_ops.mutex_lock(mutex);
+	lkl_host_ops.mutex_unlock(mutex);
+	lkl_host_ops.mutex_free(mutex);
+
+	mutex = lkl_host_ops.mutex_alloc(1);
+	lkl_host_ops.mutex_lock(mutex);
+	lkl_host_ops.mutex_lock(mutex);
+	lkl_host_ops.mutex_unlock(mutex);
+	lkl_host_ops.mutex_unlock(mutex);
+	lkl_host_ops.mutex_free(mutex);
+
+	return ret;
+}
+
+static int lkl_test_semaphore(void)
+{
+	long ret = TEST_SUCCESS;
+	/*
+	 * Can't do much to verify that this works, so we'll just let Valgrind
+	 * warn us on CI if we've made bad memory accesses.
+	 */
+
+	struct lkl_sem *sem = lkl_host_ops.sem_alloc(1);
+
+	lkl_host_ops.sem_down(sem);
+	lkl_host_ops.sem_up(sem);
+	lkl_host_ops.sem_free(sem);
+
+	return ret;
+}
+
+static int lkl_test_gettid(void)
+{
+	long tid = lkl_host_ops.gettid();
+
+	lkl_test_logf("%ld", tid);
+
+	/* As far as I know, thread IDs are non-zero on all reasonable
+	 * systems.
+	 */
+	if (tid)
+		return TEST_SUCCESS;
+	else
+		return TEST_FAILURE;
+}
+
+static void test_thread(void *data)
+{
+	int *pipe_fds = (int *) data;
+	char tmp[LKL_PIPE_BUF+1];
+	int ret;
+
+	ret = lkl_sys_read(pipe_fds[0], tmp, sizeof(tmp));
+	if (ret < 0)
+		lkl_test_logf("%s: %s\n", __func__, lkl_strerror(ret));
+}
+
+static int lkl_test_syscall_thread(void)
+{
+	int pipe_fds[2];
+	char tmp[LKL_PIPE_BUF+1];
+	long ret;
+	lkl_thread_t tid;
+
+	ret = lkl_sys_pipe2(pipe_fds, 0);
+	if (ret) {
+		lkl_test_logf("pipe2: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_fcntl(pipe_fds[0], LKL_F_SETPIPE_SZ, 1);
+	if (ret < 0) {
+		lkl_test_logf("fcntl setpipe_sz: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	tid = lkl_host_ops.thread_create(test_thread, pipe_fds);
+	if (!tid) {
+		lkl_test_logf("failed to create thread\n");
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_write(pipe_fds[1], tmp, sizeof(tmp));
+	if (ret != sizeof(tmp)) {
+		if (ret < 0)
+			lkl_test_logf("write error: %s\n", lkl_strerror(ret));
+		else
+			lkl_test_logf("short write: %ld\n", ret);
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_host_ops.thread_join(tid);
+	if (ret) {
+		lkl_test_logf("failed to join thread\n");
+		return TEST_FAILURE;
+	}
+
+	return TEST_SUCCESS;
+}
+
+#ifndef __MINGW32__
+static void thread_get_pid(void *unused)
+{
+	lkl_sys_getpid();
+}
+
+static int lkl_test_many_syscall_threads(void)
+{
+	lkl_thread_t tid;
+	int count = 65, ret;
+
+	while (--count > 0) {
+		tid = lkl_host_ops.thread_create(thread_get_pid, NULL);
+		if (!tid) {
+			lkl_test_logf("failed to create thread\n");
+			return TEST_FAILURE;
+		}
+
+		ret = lkl_host_ops.thread_join(tid);
+		if (ret) {
+			lkl_test_logf("failed to join thread\n");
+			return TEST_FAILURE;
+		}
+	}
+
+	return TEST_SUCCESS;
+}
+#endif
+
+static void thread_quit_immediately(void *unused)
+{
+}
+
+static int lkl_test_join(void)
+{
+	lkl_thread_t tid = lkl_host_ops.thread_create(thread_quit_immediately,
+						      NULL);
+	int ret = lkl_host_ops.thread_join(tid);
+
+	if (ret == 0) {
+		lkl_test_logf("joined %ld\n", tid);
+		return TEST_SUCCESS;
+	}
+
+	lkl_test_logf("failed joining %ld\n", tid);
+	return TEST_FAILURE;
+}
+
+LKL_TEST_CALL(start_kernel, lkl_start_kernel, 0, &lkl_host_ops,
+	     "mem=16M loglevel=8");
+LKL_TEST_CALL(stop_kernel, lkl_sys_halt, 0);
+
+struct lkl_test tests[] = {
+	LKL_TEST(mutex),
+	LKL_TEST(semaphore),
+	LKL_TEST(join),
+	LKL_TEST(start_kernel),
+	LKL_TEST(getpid),
+	LKL_TEST(syscall_latency),
+	LKL_TEST(umask),
+	LKL_TEST(umask2),
+	LKL_TEST(creat),
+	LKL_TEST(close),
+	LKL_TEST(failopen),
+	LKL_TEST(open),
+	LKL_TEST(write),
+	LKL_TEST(lseek_cur),
+	LKL_TEST(lseek_end),
+	LKL_TEST(lseek_set),
+	LKL_TEST(read),
+	LKL_TEST(fstat),
+	LKL_TEST(mkdir),
+	LKL_TEST(stat),
+#ifndef __MINGW32__
+	LKL_TEST(nanosleep),
+#endif
+	LKL_TEST(pipe2),
+	LKL_TEST(epoll),
+	LKL_TEST(mount_fs_proc),
+	LKL_TEST(chdir_proc),
+	LKL_TEST(open_cwd),
+	LKL_TEST(getdents64),
+	LKL_TEST(close_dir_fd),
+	LKL_TEST(chdir_root),
+	LKL_TEST(umount_fs_proc),
+	LKL_TEST(lo_ifup),
+	LKL_TEST(gettid),
+	LKL_TEST(syscall_thread),
+	/*
+	 * Wine has an issue where the FlsCallback is not called when
+	 * the thread terminates which makes testing the automatic
+	 * syscall threads cleanup impossible under wine.
+	 */
+#ifndef __MINGW32__
+	LKL_TEST(many_syscall_threads),
+#endif
+	LKL_TEST(stop_kernel),
+};
+
+int main(int argc, const char **argv)
+{
+	lkl_host_ops.print = lkl_test_log;
+
+	return lkl_test_run(tests, sizeof(tests)/sizeof(struct lkl_test),
+			    "boot");
+}
diff --git a/tools/lkl/tests/boot.sh b/tools/lkl/tests/boot.sh
new file mode 100755
index 000000000000..d985c04b0ac1
--- /dev/null
+++ b/tools/lkl/tests/boot.sh
@@ -0,0 +1,9 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+source $script_dir/test.sh
+
+lkl_test_plan 1 "boot"
+lkl_test_run 1
+lkl_test_exec $script_dir/boot
diff --git a/tools/lkl/tests/cla.c b/tools/lkl/tests/cla.c
new file mode 100644
index 000000000000..a34badeb5f06
--- /dev/null
+++ b/tools/lkl/tests/cla.c
@@ -0,0 +1,159 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <string.h>
+#include <errno.h>
+#include <stdlib.h>
+#ifdef __MINGW32__
+#include <winsock2.h>
+#else
+#include <sys/socket.h>
+#include <netinet/in.h>
+#include <arpa/inet.h>
+#endif
+
+#include "cla.h"
+
+static int cl_arg_parse_bool(struct cl_arg *arg, const char *value)
+{
+	*((int *)arg->store) = 1;
+	return 0;
+}
+
+static int cl_arg_parse_str(struct cl_arg *arg, const char *value)
+{
+	*((const char **)arg->store) = value;
+	return 0;
+}
+
+static int cl_arg_parse_int(struct cl_arg *arg, const char *value)
+{
+	errno = 0;
+	*((int *)arg->store) = strtol(value, NULL, 0);
+	return errno == 0;
+}
+
+static int cl_arg_parse_str_set(struct cl_arg *arg, const char *value)
+{
+	const char **set = arg->set;
+	int i;
+
+	for (i = 0; set[i] != NULL; i++) {
+		if (strcmp(set[i], value) == 0) {
+			*((int *)arg->store) = i;
+			return 0;
+		}
+	}
+
+	return -1;
+}
+
+static int cl_arg_parse_ipv4(struct cl_arg *arg, const char *value)
+{
+	unsigned int addr;
+
+	if (!value)
+		return -1;
+
+	addr = inet_addr(value);
+	if (addr == INADDR_NONE)
+		return -1;
+	*((unsigned int *)arg->store) = addr;
+	return 0;
+}
+
+static cl_arg_parser_t parsers[] = {
+	[CL_ARG_BOOL] = cl_arg_parse_bool,
+	[CL_ARG_INT] = cl_arg_parse_int,
+	[CL_ARG_STR] = cl_arg_parse_str,
+	[CL_ARG_STR_SET] = cl_arg_parse_str_set,
+	[CL_ARG_IPV4] = cl_arg_parse_ipv4,
+};
+
+static struct cl_arg *find_short_arg(char name, struct cl_arg *args)
+{
+	struct cl_arg *arg;
+
+	for (arg = args; arg->short_name != 0; arg++) {
+		if (arg->short_name == name)
+			return arg;
+	}
+
+	return NULL;
+}
+
+static struct cl_arg *find_long_arg(const char *name, struct cl_arg *args)
+{
+	struct cl_arg *arg;
+
+	for (arg = args; arg->long_name; arg++) {
+		if (strcmp(arg->long_name, name) == 0)
+			return arg;
+	}
+
+	return NULL;
+}
+
+static void print_help(struct cl_arg *args)
+{
+	struct cl_arg *arg;
+
+	fprintf(stderr, "usage:\n");
+	for (arg = args; arg->long_name; arg++) {
+		fprintf(stderr, "-%c, --%-20s %s", arg->short_name,
+			arg->long_name, arg->help);
+		if (arg->type == CL_ARG_STR_SET) {
+			const char **set = arg->set;
+
+			fprintf(stderr, " [ ");
+			while (*set != NULL)
+				fprintf(stderr, "%s ", *(set++));
+			fprintf(stderr, "]");
+		}
+		fprintf(stderr, "\n");
+	}
+}
+
+int parse_args(int argc, const char **argv, struct cl_arg *args)
+{
+	int i;
+
+	for (i = 1; i < argc; i++) {
+		struct cl_arg *arg = NULL;
+		cl_arg_parser_t parser;
+
+		if (argv[i][0] == '-') {
+			if (argv[i][1] != '-')
+				arg = find_short_arg(argv[i][1], args);
+			else
+				arg = find_long_arg(&argv[i][2], args);
+		}
+
+		if (!arg) {
+			fprintf(stderr, "unknown option '%s'\n", argv[i]);
+			print_help(args);
+			return -1;
+		}
+
+		if (arg->type == CL_ARG_USER || arg->type >= CL_ARG_END)
+			parser = arg->parser;
+		else
+			parser = parsers[arg->type];
+
+		if (!parser) {
+			fprintf(stderr, "can't parse --'%s'/-'%c'\n",
+				arg->long_name, args->short_name);
+			return -1;
+		}
+
+		if (parser(arg, argv[i + 1]) < 0) {
+			fprintf(stderr, "can't parse '%s'\n", argv[i]);
+			print_help(args);
+			return -1;
+		}
+
+		if (arg->has_arg)
+			i++;
+	}
+
+	return 0;
+}
diff --git a/tools/lkl/tests/cla.h b/tools/lkl/tests/cla.h
new file mode 100644
index 000000000000..f8369be02e5a
--- /dev/null
+++ b/tools/lkl/tests/cla.h
@@ -0,0 +1,33 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_TEST_CLA_H
+#define _LKL_TEST_CLA_H
+
+enum cl_arg_type {
+	CL_ARG_USER = 0,
+	CL_ARG_BOOL,
+	CL_ARG_INT,
+	CL_ARG_STR,
+	CL_ARG_STR_SET,
+	CL_ARG_IPV4,
+	CL_ARG_END,
+};
+
+struct cl_arg;
+
+typedef int (*cl_arg_parser_t)(struct cl_arg *arg, const char *value);
+
+struct cl_arg {
+	const char *long_name;
+	char short_name;
+	const char *help;
+	int has_arg;
+	enum cl_arg_type type;
+	void *store;
+	void *set;
+	cl_arg_parser_t parser;
+};
+
+int parse_args(int argc, const char **argv, struct cl_arg *args);
+
+
+#endif /* _LKL_TEST_CLA_H */
diff --git a/tools/lkl/tests/disk.c b/tools/lkl/tests/disk.c
new file mode 100644
index 000000000000..0aa039876b54
--- /dev/null
+++ b/tools/lkl/tests/disk.c
@@ -0,0 +1,189 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <unistd.h>
+#include <string.h>
+#include <time.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <lkl.h>
+#include <lkl_host.h>
+#ifndef __MINGW32__
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <sys/ioctl.h>
+#else
+#include <windows.h>
+#endif
+
+#include "test.h"
+#include "cla.h"
+
+static struct {
+	int printk;
+	const char *disk;
+	const char *fstype;
+	int partition;
+} cla;
+
+struct cl_arg args[] = {
+	{"disk", 'd', "disk file to use", 1, CL_ARG_STR, &cla.disk},
+	{"partition", 'P', "partition to mount", 1, CL_ARG_INT, &cla.partition},
+	{"type", 't', "filesystem type", 1, CL_ARG_STR, &cla.fstype},
+	{0},
+};
+
+
+static struct lkl_disk disk;
+static int disk_id = -1;
+
+int lkl_test_disk_add(void)
+{
+#ifdef __MINGW32__
+	disk.handle = CreateFile(cla.disk, GENERIC_READ | GENERIC_WRITE,
+			       0, NULL, OPEN_EXISTING, 0, NULL);
+	if (!disk.handle)
+#else
+	disk.fd = open(cla.disk, O_RDWR);
+	if (disk.fd < 0)
+#endif
+		goto out_unlink;
+
+	disk.ops = NULL;
+
+	disk_id = lkl_disk_add(&disk);
+	if (disk_id < 0)
+		goto out_close;
+
+	goto out;
+
+out_close:
+#ifdef __MINGW32__
+	CloseHandle(disk.handle);
+#else
+	close(disk.fd);
+#endif
+
+out_unlink:
+#ifdef __MINGW32__
+	DeleteFile(cla.disk);
+#else
+	unlink(cla.disk);
+#endif
+
+out:
+	lkl_test_logf("disk fd/handle %x disk_id %d", disk.fd, disk_id);
+
+	if (disk_id >= 0)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+int lkl_test_disk_remove(void)
+{
+	int ret;
+
+	ret = lkl_disk_remove(disk);
+
+#ifdef __MINGW32__
+	CloseHandle(disk.handle);
+#else
+	close(disk.fd);
+#endif
+
+	if (ret == 0)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+
+static char mnt_point[32];
+
+LKL_TEST_CALL(mount_dev, lkl_mount_dev, 0, disk_id, cla.partition, cla.fstype,
+	      0, NULL, mnt_point, sizeof(mnt_point))
+
+static int lkl_test_umount_dev(void)
+{
+	long ret, ret2;
+
+	ret = lkl_sys_chdir("/");
+
+	ret2 = lkl_umount_dev(disk_id, cla.partition, 0, 1000);
+
+	lkl_test_logf("%ld %ld", ret, ret2);
+
+	if (!ret && !ret2)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+struct lkl_dir *dir;
+
+static int lkl_test_opendir(void)
+{
+	int err;
+
+	dir = lkl_opendir(mnt_point, &err);
+
+	lkl_test_logf("lkl_opedir(%s) = %d %s\n", mnt_point, err,
+		      lkl_strerror(err));
+
+	if (err == 0)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+static int lkl_test_readdir(void)
+{
+	struct lkl_linux_dirent64 *de = lkl_readdir(dir);
+	int wr = 0;
+
+	while (de) {
+		wr += lkl_test_logf("%s ", de->d_name);
+		if (wr >= 70) {
+			lkl_test_logf("\n");
+			wr = 0;
+			break;
+		}
+		de = lkl_readdir(dir);
+	}
+
+	if (lkl_errdir(dir) == 0)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+LKL_TEST_CALL(closedir, lkl_closedir, 0, dir);
+LKL_TEST_CALL(chdir_mnt_point, lkl_sys_chdir, 0, mnt_point);
+LKL_TEST_CALL(start_kernel, lkl_start_kernel, 0, &lkl_host_ops,
+	     "mem=16M loglevel=8");
+LKL_TEST_CALL(stop_kernel, lkl_sys_halt, 0);
+
+struct lkl_test tests[] = {
+	LKL_TEST(disk_add),
+	LKL_TEST(start_kernel),
+	LKL_TEST(mount_dev),
+	LKL_TEST(chdir_mnt_point),
+	LKL_TEST(opendir),
+	LKL_TEST(readdir),
+	LKL_TEST(closedir),
+	LKL_TEST(umount_dev),
+	LKL_TEST(stop_kernel),
+	LKL_TEST(disk_remove),
+
+};
+
+int main(int argc, const char **argv)
+{
+	if (parse_args(argc, argv, args) < 0)
+		return -1;
+
+	lkl_host_ops.print = lkl_test_log;
+
+	return lkl_test_run(tests, sizeof(tests)/sizeof(struct lkl_test),
+			    "disk %s", cla.fstype);
+}
diff --git a/tools/lkl/tests/disk.sh b/tools/lkl/tests/disk.sh
new file mode 100755
index 000000000000..9bdcb16f2d5c
--- /dev/null
+++ b/tools/lkl/tests/disk.sh
@@ -0,0 +1,61 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+
+source $script_dir/test.sh
+
+function prepfs()
+{
+    set -e
+
+    file=`mktemp`
+
+    dd if=/dev/zero of=$file bs=1024 count=204800
+
+    yes | mkfs.$1 $file
+
+    if ! [ -z $BSD_WDIR ]; then
+        $MYSSH mkdir -p $BSD_WDIR
+        ssh_copy $file $BSD_WDIR
+        rm $file
+        file=$BSD_WDIR/$(basename $file)
+    fi
+
+    export_vars file
+}
+
+function cleanfs()
+{
+    set -e
+
+    if ! [ -z $BSD_WDIR ]; then
+        $MYSSH rm $1
+        $MYSSH rm $BSD_WDIR/disk
+    else
+        rm $1
+    fi
+}
+
+if [ "$1" = "-t" ]; then
+    shift
+    fstype=$1
+    shift
+fi
+
+if [ -z "$fstype" ]; then
+    fstype="ext4"
+fi
+
+if [ -z $(which mkfs.$fstype) ]; then
+    lkl_test_plan 0 "disk $fstype"
+    echo "no mkfs.$fstype command"
+    exit 0
+fi
+
+lkl_test_plan 1 "disk $fstype"
+lkl_test_run 1 prepfs $fstype
+lkl_test_exec $script_dir/disk -d $file -t $fstype $@
+lkl_test_plan 1 "disk $fstype"
+lkl_test_run 1 cleanfs $file
+
diff --git a/tools/lkl/tests/net-setup.sh b/tools/lkl/tests/net-setup.sh
new file mode 100644
index 000000000000..0cfc42a30a54
--- /dev/null
+++ b/tools/lkl/tests/net-setup.sh
@@ -0,0 +1,73 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+TEST_IP_NETWORK=192.168.113.0
+TEST_IP_NETMASK=24
+TEST_IP6_NETWORK=fc03::0
+TEST_IP6_NETMASK=64
+TEST_MAC0="aa:bb:cc:dd:ee:ff"
+TEST_MAC1="aa:bb:cc:dd:ee:aa"
+TEST_NETSERVER_PORT=11223
+
+# $1 - count
+# $2 - netcount
+ip_add()
+{
+    IP_HEX=$(printf '%.2X%.2X%.2X%.2X\n' \
+         `echo $TEST_IP_NETWORK | sed -e 's/\./ /g'`)
+    NET_COUNT=$(( 1 << (32 - $TEST_IP_NETMASK) ))
+    NEXT_IP_HEX=$(printf %.8X `echo $((0x$IP_HEX + $1 + ${2:-0} * $NET_COUNT))`)
+    NEXT_IP=$(printf '%d.%d.%d.%d\n' \
+          `echo $NEXT_IP_HEX | sed -r 's/(..)/0x\1 /g'`)
+    echo -n "$NEXT_IP"
+}
+
+# $1 - count
+# $2 - netcount
+ip6_add()
+{
+    IP6_PREFIX=${TEST_IP6_NETWORK%*::*}
+    IP6_HOST=${TEST_IP6_NETWORK#*::*}
+    echo -n "$(printf "%x" $((0x$IP6_PREFIX+${2:-0})))::$(($IP6_HOST+$1))"
+}
+
+ip_host()
+{
+
+    ip_add 1 $1
+}
+
+ip_lkl()
+{
+    ip_add 2 $1
+}
+
+ip_host_mask()
+{
+    echo -n "$(ip_host $1)/$TEST_IP_NETMASK"
+}
+
+ip_net_mask()
+{
+    echo "$(ip_add 0 $1)/$TEST_IP_NETMASK"
+}
+
+ip6_host()
+{
+    ip6_add 1 $1
+}
+
+ip6_lkl()
+{
+    ip6_add 2 $1
+}
+
+ip6_host_mask()
+{
+    echo -n "$(ip6_host $1)/$TEST_IP6_NETMASK"
+}
+
+ip6_net_mask()
+{
+    echo "$(ip6_add 0 $1)/$TEST_IP6_NETMASK"
+}
diff --git a/tools/lkl/tests/net-test.c b/tools/lkl/tests/net-test.c
new file mode 100644
index 000000000000..5a8e39a52417
--- /dev/null
+++ b/tools/lkl/tests/net-test.c
@@ -0,0 +1,299 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <string.h>
+#include <stdlib.h>
+#include <errno.h>
+#ifdef __FreeBSD__
+#include <sys/types.h>
+#endif
+#ifdef __MINGW32__
+#include <winsock2.h>
+#else
+#include <sys/socket.h>
+#include <netinet/in.h>
+#include <arpa/inet.h>
+#endif
+
+#include <lkl.h>
+#include <lkl_host.h>
+
+#include "cla.h"
+#include "test.h"
+
+enum {
+	BACKEND_NONE,
+};
+
+const char *backends[] = { "loopback", NULL };
+static struct {
+	int backend;
+	const char *ifname;
+	int dhcp, nmlen;
+	unsigned int ip, dst, gateway, sleep;
+} cla = {
+	.backend = BACKEND_NONE,
+	.ip = INADDR_NONE,
+	.gateway = INADDR_NONE,
+	.dst = INADDR_NONE,
+	.sleep = 0,
+};
+
+
+struct cl_arg args[] = {
+	{"backend", 'b', "network backend type", 1, CL_ARG_STR_SET,
+	 &cla.backend, backends},
+	{"ifname", 'i', "interface name", 1, CL_ARG_STR, &cla.ifname},
+	{"dhcp", 'd', "use dhcp to configure LKL", 0, CL_ARG_BOOL, &cla.dhcp},
+	{"ip", 'I', "IPv4 address to use", 1, CL_ARG_IPV4, &cla.ip},
+	{"netmask-len", 'n', "IPv4 netmask length", 1, CL_ARG_INT,
+	 &cla.nmlen},
+	{"gateway", 'g', "IPv4 gateway to use", 1, CL_ARG_IPV4, &cla.gateway},
+	{"dst", 'D', "IPv4 destination address", 1, CL_ARG_IPV4, &cla.dst},
+	{"sleep", 's', "sleep", 1, CL_ARG_INT, &cla.sleep},
+	{0},
+};
+
+u_short
+in_cksum(const u_short *addr, register int len, u_short csum)
+{
+	int nleft = len;
+	const u_short *w = addr;
+	u_short answer;
+	int sum = csum;
+
+	while (nleft > 1)  {
+		sum += *w++;
+		nleft -= 2;
+	}
+
+	if (nleft == 1)
+		sum += htons(*(u_char *)w << 8);
+
+	sum = (sum >> 16) + (sum & 0xffff);
+	sum += (sum >> 16);
+	answer = ~sum;
+	return answer;
+}
+
+static int lkl_test_sleep(void)
+{
+	struct lkl_timespec ts = {
+		.tv_sec = cla.sleep,
+	};
+	int ret;
+
+	ret = lkl_sys_nanosleep((struct __lkl__kernel_timespec *)&ts, NULL);
+	if (ret < 0) {
+		lkl_test_logf("nanosleep error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	return TEST_SUCCESS;
+}
+
+static int lkl_test_icmp(void)
+{
+	int sock, ret;
+	struct lkl_iphdr *iph;
+	struct lkl_icmphdr *icmp;
+	struct lkl_sockaddr_in saddr;
+	struct lkl_pollfd pfd;
+	char buf[32];
+
+	if (cla.dst == INADDR_NONE)
+		return TEST_SKIP;
+
+	memset(&saddr, 0, sizeof(saddr));
+	saddr.sin_family = AF_INET;
+	saddr.sin_addr.lkl_s_addr = cla.dst;
+
+	lkl_test_logf("pinging %s\n",
+		      inet_ntoa(*(struct in_addr *)&saddr.sin_addr));
+
+	sock = lkl_sys_socket(LKL_AF_INET, LKL_SOCK_RAW, LKL_IPPROTO_ICMP);
+	if (sock < 0) {
+		lkl_test_logf("socket error (%s)\n", lkl_strerror(sock));
+		return TEST_FAILURE;
+	}
+
+	icmp = malloc(sizeof(struct lkl_icmphdr));
+	icmp->type = LKL_ICMP_ECHO;
+	icmp->code = 0;
+	icmp->checksum = 0;
+	icmp->un.echo.sequence = 0;
+	icmp->un.echo.id = 0;
+	icmp->checksum = in_cksum((u_short *)icmp, sizeof(*icmp), 0);
+
+	ret = lkl_sys_sendto(sock, icmp, sizeof(*icmp), 0,
+			     (struct lkl_sockaddr *)&saddr,
+			     sizeof(saddr));
+	if (ret < 0) {
+		lkl_test_logf("sendto error (%s)\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	free(icmp);
+
+	pfd.fd = sock;
+	pfd.events = LKL_POLLIN;
+	pfd.revents = 0;
+
+	ret = lkl_sys_poll(&pfd, 1, 1000);
+	if (ret < 0) {
+		lkl_test_logf("poll error (%s)\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_recv(sock, buf, sizeof(buf), LKL_MSG_DONTWAIT);
+	if (ret < 0) {
+		lkl_test_logf("recv error (%s)\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	if (ret < (int)sizeof(struct lkl_iphdr)) {
+		lkl_test_logf("short length of received ip packet (len=%d)\n",
+			      ret);
+		return TEST_FAILURE;
+	}
+	iph = (struct lkl_iphdr *)buf;
+
+	if (ret < (int)((iph->ihl * 4) + sizeof(struct lkl_icmphdr))) {
+		lkl_test_logf("short length of received icmp packet (len=%d)\n",
+			      ret);
+		return TEST_FAILURE;
+	}
+	icmp = (struct lkl_icmphdr *)(buf + iph->ihl * 4);
+
+	/* DHCP server may issue an ICMP echo request to a dhcp client */
+	if ((icmp->type != LKL_ICMP_ECHOREPLY || icmp->code != 0) &&
+	    (icmp->type != LKL_ICMP_ECHO)) {
+		lkl_test_logf("no ICMP echo reply (type=%d, code=%d)\n",
+			      icmp->type, icmp->code);
+		return TEST_FAILURE;
+	}
+
+	return TEST_SUCCESS;
+}
+
+static struct lkl_netdev *nd;
+
+static int lkl_test_nd_create(void)
+{
+	switch (cla.backend) {
+	case BACKEND_NONE:
+		return TEST_SKIP;
+	}
+
+	if (!nd) {
+		lkl_test_logf("failed to create netdev\n");
+		return TEST_BAILOUT;
+	}
+
+	return TEST_SUCCESS;
+}
+
+static int nd_id;
+
+static int lkl_test_nd_add(void)
+{
+	if (cla.backend == BACKEND_NONE)
+		return TEST_SKIP;
+
+	return TEST_SUCCESS;
+}
+
+static int lkl_test_nd_remove(void)
+{
+	if (cla.backend == BACKEND_NONE)
+		return TEST_SKIP;
+
+	return TEST_SUCCESS;
+}
+
+LKL_TEST_CALL(start_kernel, lkl_start_kernel, 0, &lkl_host_ops,
+	"mem=16M loglevel=8 %s", cla.dhcp ? "ip=dhcp" : "");
+LKL_TEST_CALL(stop_kernel, lkl_sys_halt, 0);
+
+static int nd_ifindex;
+
+static int lkl_test_nd_ifindex(void)
+{
+	if (cla.backend == BACKEND_NONE)
+		return TEST_SKIP;
+
+	nd_ifindex = lkl_netdev_get_ifindex(nd_id);
+	if (nd_ifindex < 0) {
+		lkl_test_logf("failed to get ifindex for netdev id %d: %s\n",
+			      nd_id, lkl_strerror(nd_ifindex));
+		return TEST_BAILOUT;
+	}
+
+	return TEST_SUCCESS;
+}
+
+LKL_TEST_CALL(if_up, lkl_if_up, 0,
+	      cla.backend == BACKEND_NONE ? 1 : nd_ifindex);
+
+static int lkl_test_set_ipv4(void)
+{
+	int ret;
+
+	if (cla.backend == BACKEND_NONE || cla.ip == LKL_INADDR_NONE)
+		return TEST_SKIP;
+
+	ret = lkl_if_set_ipv4(nd_ifindex, cla.ip, cla.nmlen);
+	if (ret < 0) {
+		lkl_test_logf("failed to set IPv4 address: %s\n",
+			      lkl_strerror(ret));
+		return TEST_BAILOUT;
+	}
+
+	return TEST_SUCCESS;
+}
+
+static int lkl_test_set_gateway(void)
+{
+	int ret;
+
+	if (cla.backend == BACKEND_NONE || cla.gateway == LKL_INADDR_NONE)
+		return TEST_SKIP;
+
+	ret = lkl_set_ipv4_gateway(cla.gateway);
+	if (ret < 0) {
+		lkl_test_logf("failed to set IPv4 gateway: %s\n",
+			      lkl_strerror(ret));
+		return TEST_BAILOUT;
+	}
+
+	return TEST_SUCCESS;
+}
+
+struct lkl_test tests[] = {
+	LKL_TEST(nd_create),
+	LKL_TEST(nd_add),
+	LKL_TEST(start_kernel),
+	LKL_TEST(nd_ifindex),
+	LKL_TEST(if_up),
+	LKL_TEST(set_ipv4),
+	LKL_TEST(set_gateway),
+	LKL_TEST(sleep),
+	LKL_TEST(icmp),
+	LKL_TEST(nd_remove),
+	LKL_TEST(stop_kernel),
+};
+
+int main(int argc, const char **argv)
+{
+	if (parse_args(argc, argv, args) < 0)
+		return -1;
+
+	if (cla.ip != LKL_INADDR_NONE && (cla.nmlen < 0 || cla.nmlen > 32)) {
+		fprintf(stderr, "invalid netmask length %d\n", cla.nmlen);
+		return -1;
+	}
+
+	lkl_host_ops.print = lkl_test_log;
+
+	return lkl_test_run(tests, sizeof(tests)/sizeof(struct lkl_test),
+			    "net %s", backends[cla.backend]);
+}
diff --git a/tools/lkl/tests/net.sh b/tools/lkl/tests/net.sh
new file mode 100755
index 000000000000..32e8452b0406
--- /dev/null
+++ b/tools/lkl/tests/net.sh
@@ -0,0 +1,90 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+
+source $script_dir/test.sh
+source $script_dir/net-setup.sh
+
+cleanup_backend()
+{
+    set -e
+
+    case "$1" in
+    "loopback")
+        ;;
+    esac
+}
+
+get_test_ip()
+{
+    # DHCP test parameters
+    TEST_HOST=8.8.8.8
+    HOST_IF=$(lkl_test_cmd ip route get $TEST_HOST | head -n1 |cut -d ' ' -f5)
+    HOST_GW=$(lkl_test_cmd ip route get $TEST_HOST | head -n1 | cut -d ' ' -f3)
+    if lkl_test_cmd ping -c1 -w1 $HOST_GW; then
+        TEST_IP_REMOTE=$HOST_GW
+    elif lkl_test_cmd ping -c1 -w1 $TEST_HOST; then
+        TEST_IP_REMOTE=$TEST_HOST
+    else
+        echo "could not find remote test ip"
+        return $TEST_SKIP
+    fi
+
+    export_vars HOST_IF TEST_IP_REMOTE
+}
+
+setup_backend()
+{
+    set -e
+
+    if [ "$LKL_HOST_CONFIG_POSIX" != "y" ] &&
+       [ "$1" != "loopback" ]; then
+        echo "not a posix environment"
+        return $TEST_SKIP
+    fi
+
+    case "$1" in
+    "loopback")
+        ;;
+    *)
+        echo "don't know how to setup backend $1"
+        return $TEST_FAILED
+        ;;
+    esac
+}
+
+run_tests()
+{
+    case "$1" in
+    "loopback")
+        lkl_test_exec $script_dir/net-test --dst 127.0.0.1
+        ;;
+    esac
+}
+
+if [ "$1" = "-b" ]; then
+    shift
+    backend=$1
+    shift
+fi
+
+if [ -z "$backend" ]; then
+    backend="loopback"
+fi
+
+lkl_test_plan 1 "net $backend"
+lkl_test_run 1 setup_backend $backend
+
+if [ $? = $TEST_SKIP ]; then
+    exit 0
+fi
+
+trap "cleanup_backend $backend" EXIT
+
+run_tests $backend
+
+trap : EXIT
+lkl_test_plan 1 "net $backend"
+lkl_test_run 1 cleanup_backend $backend
+
diff --git a/tools/lkl/tests/run.py b/tools/lkl/tests/run.py
new file mode 100755
index 000000000000..b72299aaabee
--- /dev/null
+++ b/tools/lkl/tests/run.py
@@ -0,0 +1,179 @@
+#!/usr/bin/env python
+# SPDX-License-Identifier: GPL-2.0
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 2 of the License
+#
+# Author: Octavian Purdila <tavi@cs.pub.ro>
+#
+
+from __future__ import print_function
+
+import argparse
+import os
+import subprocess
+import sys
+import tap13
+import xml.etree.ElementTree as ET
+
+from junit_xml import TestSuite, TestCase
+
+
+class Reporter(tap13.Reporter):
+    def start(self, obj):
+        if type(obj) is tap13.Test:
+            if obj.result == "*":
+                end='\r'
+            else:
+                end='\n'
+            print("  TEST       %-8s %.50s" %
+                  (obj.result, obj.description + " " + obj.comment), end=end)
+
+        elif type(obj) is tap13.Suite:
+            if obj.tests_planned == 0:
+                status = "skip"
+            else:
+                status = ""
+            print("  SUITE      %-8s %s" % (status, obj.name))
+
+    def end(self, obj):
+        if type(obj) is tap13.Test:
+            if obj.result != "ok":
+                try:
+                    print(obj.yaml["log"], end='')
+                except:
+                    None
+
+
+mydir=os.path.dirname(os.path.realpath(__file__))
+
+tests = [
+    'boot.sh',
+    'disk.sh -t ext4',
+    'disk.sh -t vfat',
+    'disk.sh -t btrfs',
+    'net.sh -b loopback',
+    'lklfuse.sh -t ext4',
+    'lklfuse.sh -t vfat',
+    'lklfuse.sh -t btrfs',
+]
+
+parser = argparse.ArgumentParser(description='LKL test runner')
+parser.add_argument('tests', nargs='?', action='append',
+                    help='tests to run %s' % tests)
+parser.add_argument('--junit-dir',
+                    help='directory where to store the juni suites')
+parser.add_argument('--gdb', action='store_true', default=False,
+                    help='run simple tests under gdb; implies --pass-through')
+parser.add_argument('--pass-through', action='store_true',  default=False,
+                    help='run the test without interpeting the test output')
+parser.add_argument('--valgrind', action='store_true', default=False,
+                    help='run simple tests under valgrind')
+
+args = parser.parse_args()
+if args.tests == [None]:
+    args.tests = tests
+
+if args.gdb:
+    args.pass_through=True
+    os.environ['GDB']="yes"
+
+if args.valgrind:
+    os.environ['VALGRIND']="yes"
+
+tap = tap13.Parser(Reporter())
+
+os.environ['PATH'] += ":" + mydir
+
+exit_code = 0
+
+for t in args.tests:
+    if not t:
+        continue
+    if args.pass_through:
+        print(t)
+        if subprocess.call(t, shell=True) != 0:
+            exit_code = 1
+    else:
+        p = subprocess.Popen(t, shell=True, stdout=subprocess.PIPE)
+        tap.parse(p.stdout)
+
+if args.pass_through:
+    sys.exit(exit_code)
+
+suites_count = 0
+tests_total = 0
+tests_not_ok = 0
+tests_ok = 0
+tests_skip = 0
+val_errs = 0
+val_fails = 0
+val_skips = 0
+
+for s in tap.run.suites:
+
+    junit_tests = []
+    suites_count += 1
+
+    for t in s.tests:
+        try:
+            secs = t.yaml["time_us"] / 1000000.0
+        except:
+            secs = 0
+        try:
+            log = t.yaml['log']
+        except:
+            log = ""
+
+        jt = TestCase(t.description, elapsed_sec=secs, stdout=log)
+        if t.result == 'skip':
+            jt.add_skipped_info(output=log)
+        elif t.result == 'not ok':
+            jt.add_error_info(output=log)
+
+        junit_tests.append(jt)
+
+        tests_total += 1
+        if t.result == "ok":
+            tests_ok += 1
+        elif t.result == "not ok":
+            tests_not_ok += 1
+            exit_code = 1
+        elif t.result == "skip":
+            tests_skip += 1
+
+    if args.junit_dir:
+        js = TestSuite(s.name, junit_tests)
+        with open(os.path.join(args.junit_dir, os.path.basename(s.name) + '.xml'), 'w') as f:
+            js.to_file(f, [js])
+
+        if os.getenv('VALGRIND') is not None:
+            val_xml = 'valgrind-%s.xml' % os.path.basename(s.name).replace(' ','-')
+            # skipped tests don't generate xml file
+            if os.path.exists(val_xml) is False:
+                continue
+
+            cmd = 'mv %s %s' % (val_xml, args.junit_dir)
+            subprocess.call(cmd, shell=True, )
+
+            cmd = mydir + '/valgrind2xunit.py ' + val_xml
+            subprocess.call(cmd, shell=True, cwd=args.junit_dir)
+
+            # count valgrind results
+            doc = ET.parse(os.path.join(args.junit_dir, 'valgrind-%s_xunit.xml' \
+                                        % (os.path.basename(s.name).replace(' ','-'))))
+            ts = doc.getroot()
+            val_errs += int(ts.get('errors'))
+            val_fails += int(ts.get('failures'))
+            val_skips += int(ts.get('skip'))
+
+print("Summary: %d suites run, %d tests, %d ok, %d not ok, %d skipped" %
+      (suites_count, tests_total, tests_ok, tests_not_ok, tests_skip))
+
+if os.getenv('VALGRIND') is not None:
+    print(" valgrind (memcheck): %d failures, %d skipped" % (val_fails, val_skips))
+    if val_errs or val_fails:
+        exit_code = 1
+
+sys.exit(exit_code)
diff --git a/tools/lkl/tests/tap13.py b/tools/lkl/tests/tap13.py
new file mode 100644
index 000000000000..65c73cda7ca1
--- /dev/null
+++ b/tools/lkl/tests/tap13.py
@@ -0,0 +1,209 @@
+#!/usr/bin/env python
+# SPDX-License-Identifier: GPL-2.0
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 2 of the License
+#
+# Author: Octavian Purdila <tavi@cs.pub.ro>
+#
+# Based on TAP13:
+#
+# Copyright 2013, Red Hat, Inc.
+# Author: Josef Skladanka <jskladan@redhat.com>
+#
+from __future__ import print_function
+
+import re
+import sys
+import yamlish
+
+
+class Reporter(object):
+
+    def start(self, obj):
+        None
+
+    def end(self, obj):
+        None
+
+
+class Test(object):
+    def __init__(self, reporter, result, id, description=None, directive=None,
+                 comment=None):
+        self.reporter = reporter
+        self.result = result
+        if directive:
+            self.result = directive.lower()
+        if id:
+            self.id = int(id)
+        else:
+            self.id = None
+        if description:
+            self.description = description
+        else:
+            self.description = ""
+        if comment:
+            self.comment = "# " + comment
+        else:
+            self.comment = ""
+        self.yaml = None
+        self._yaml_buffer = None
+        self.diagnostics = []
+
+        self.reporter.start(self)
+
+    def end(self):
+        if not self.yaml:
+            self.yaml = yamlish.load(self._yaml_buffer)
+            self.reporter.end(self)
+
+
+class Suite(object):
+    def __init__(self, reporter, start, end, explanation):
+        self.reporter = reporter
+        self.tests = []
+        self.name = explanation
+        self.tests_planned = int(end)
+
+        self.__tests_counter = 0
+        self.__tests_base = 0
+
+        self.reporter.start(self)
+
+    def newTest(self, args):
+        try:
+            self.tests[-1].end()
+        except IndexError:
+            None
+
+        if 'id' not in args or not args['id']:
+            args['id'] = self.__tests_counter
+        else:
+            args['id'] = int(args['id']) + self.__tests_base
+
+        if args['id'] < self.__tests_counter:
+            print("error: bad test id %d, fixing it" % (args['id']))
+            args['id'] = self.__tests_counter
+        # according to TAP13 specs, missing tests must be handled as 'not ok'
+        # here we add the missing tests in sequence
+        while args['id'] > (self.__tests_counter + 1):
+            comment = 'test %d not present' % self.__tests_counter
+            self.tests.append(Test(self.reporter, 'not ok',
+                                   self.__tests_counter, comment=comment))
+            self.__tests_counter += 1
+
+        if args['id'] == self.__tests_counter:
+            if args['directive']:
+                self.test().result = args['directive'].lower()
+            else:
+                self.test().result = args['result']
+            self.reporter.start(self.test())
+        else:
+            self.tests.append(Test(self.reporter, **args))
+            self.__tests_counter += 1
+
+    def test(self):
+        return self.tests[-1]
+
+    def end(self, name, planned):
+        if name == self.name:
+            self.tests_planned += int(planned)
+            self.__tests_base = self.__tests_counter
+            return False
+        try:
+            self.test().end()
+        except IndexError:
+            None
+        if len(self.tests) != self.tests_planned:
+            for i in range(len(self.tests), self.tests_planned):
+                self.tests.append(Test(self.reporter, 'not ok', i+1,
+                                       comment='test not present'))
+        return True
+
+
+class Run(object):
+
+    def __init__(self, reporter):
+        self.reporter = reporter
+        self.suites = []
+
+    def suite(self):
+        return self.suites[-1]
+
+    def test(self):
+        return self.suites[-1].tests[-1]
+
+    def newSuite(self, args):
+        new = False
+        try:
+            if self.suite().end(args['explanation'], args['end']):
+                new = True
+        except IndexError:
+            new = True
+        if new:
+            self.suites.append(Suite(self.reporter, **args))
+
+    def newTest(self, args):
+        self.suite().newTest(args)
+
+
+class Parser(object):
+    RE_PLAN = re.compile(r"^\s*(?P<start>\d+)\.\.(?P<end>\d+)\s*(#\s*(?P<explanation>.*))?\s*$")
+    RE_TEST_LINE = re.compile(r"^\s*(?P<result>(not\s+)?ok|[*]+)\s*(?P<id>\d+)?\s*(?P<description>[^#]+)?\s*(#\s*(?P<directive>TODO|SKIP)?\s*(?P<comment>.+)?)?\s*$",  re.IGNORECASE)
+    RE_EXPLANATION = re.compile(r"^\s*#\s*(?P<explanation>.+)?\s*$")
+    RE_YAMLISH_START = re.compile(r"^\s*---.*$")
+    RE_YAMLISH_END = re.compile(r"^\s*\.\.\.\s*$")
+
+    def __init__(self, reporter):
+        self.seek_test = False
+        self.in_test = False
+        self.in_yaml = False
+        self.run = Run(reporter)
+
+    def parse(self, source):
+        # to avoid input buffering
+        while True:
+            line = source.readline()
+            if not line:
+                break
+
+            if self.in_yaml:
+                if Parser.RE_YAMLISH_END.match(line):
+                    self.run.test()._yaml_buffer.append(line.strip())
+                    self.in_yaml = False
+                else:
+                    self.run.test()._yaml_buffer.append(line.rstrip())
+                continue
+
+            line = line.strip()
+
+            if self.in_test:
+                if Parser.RE_EXPLANATION.match(line):
+                    self.run.test().diagnostics.append(line)
+                    continue
+                if Parser.RE_YAMLISH_START.match(line):
+                    self.run.test()._yaml_buffer = [line.strip()]
+                    self.in_yaml = True
+                    continue
+
+            m = Parser.RE_PLAN.match(line)
+            if m:
+                self.seek_test = True
+                args = m.groupdict()
+                self.run.newSuite(args)
+                continue
+
+            if self.seek_test:
+                m = Parser.RE_TEST_LINE.match(line)
+                if m:
+                    args = m.groupdict()
+                    self.run.newTest(args)
+                    self.in_test = True
+                    continue
+
+            print(line)
+        try:
+            self.run.suite().end(None, 0)
+        except IndexError:
+            None
diff --git a/tools/lkl/tests/test.c b/tools/lkl/tests/test.c
new file mode 100644
index 000000000000..3e334d106c48
--- /dev/null
+++ b/tools/lkl/tests/test.c
@@ -0,0 +1,126 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <stdarg.h>
+#include <time.h>
+
+#include "test.h"
+
+/* circular log buffer */
+
+static char log_buf[0x10000];
+static char *head = log_buf, *tail = log_buf;
+
+static inline void advance(char **ptr)
+{
+	if ((unsigned int)(*ptr - log_buf) >= sizeof(log_buf))
+		*ptr = log_buf;
+	else
+		*ptr = *ptr + 1;
+}
+
+static void log_char(char c)
+{
+	*tail = c;
+	advance(&tail);
+	if (tail == head)
+		advance(&head);
+}
+
+static void print_log(void)
+{
+	char last;
+
+	printf(" log: |\n");
+	last = '\n';
+	while (head != tail) {
+		if (last == '\n')
+			printf("  ");
+		last = *head;
+		putchar(last);
+		advance(&head);
+	}
+	if (last != '\n')
+		putchar('\n');
+}
+
+int lkl_test_run(const struct lkl_test *tests, int nr, const char *fmt, ...)
+{
+	int i, ret, status = TEST_SUCCESS;
+	clock_t start, stop;
+	char name[1024];
+	va_list args;
+
+	va_start(args, fmt);
+	vsnprintf(name, sizeof(name), fmt, args);
+	va_end(args);
+
+	printf("1..%d # %s\n", nr, name);
+	for (i = 1; i <= nr; i++) {
+		const struct lkl_test *t = &tests[i-1];
+		unsigned long delta_us;
+
+		printf("* %d %s\n", i, t->name);
+		fflush(stdout);
+
+		start = clock();
+
+		ret = t->fn(t->arg1, t->arg2, t->arg3);
+
+		stop = clock();
+
+		switch (ret) {
+		case TEST_SUCCESS:
+			printf("ok %d %s\n", i, t->name);
+			break;
+		case TEST_SKIP:
+			printf("ok %d %s # SKIP\n", i, t->name);
+			break;
+		case TEST_BAILOUT:
+			status = TEST_BAILOUT;
+			/* fall through */
+		case TEST_FAILURE:
+		default:
+			if (status != TEST_BAILOUT)
+				status = TEST_FAILURE;
+			printf("not ok %d %s\n", i, t->name);
+		}
+
+		printf(" ---\n");
+		delta_us = (stop - start) * 1000000 / CLOCKS_PER_SEC;
+		printf(" time_us: %ld\n", delta_us);
+		print_log();
+		printf(" ...\n");
+
+		if (status == TEST_BAILOUT) {
+			printf("Bail out!\n");
+			return TEST_FAILURE;
+		}
+
+		fflush(stdout);
+	}
+
+	return status;
+}
+
+
+void lkl_test_log(const char *str, int len)
+{
+	while (len--)
+		log_char(*(str++));
+}
+
+int lkl_test_logf(const char *fmt, ...)
+{
+	char tmp[1024], *c;
+	va_list args;
+	unsigned int n;
+
+	va_start(args, fmt);
+	n = vsnprintf(tmp, sizeof(tmp), fmt, args);
+	va_end(args);
+
+	for (c = tmp; *c != 0; c++)
+		log_char(*c);
+
+	return n > sizeof(tmp) ? sizeof(tmp) : n;
+}
diff --git a/tools/lkl/tests/test.h b/tools/lkl/tests/test.h
new file mode 100644
index 000000000000..f63ad6d419cb
--- /dev/null
+++ b/tools/lkl/tests/test.h
@@ -0,0 +1,72 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_TEST_H
+#define _LKL_TEST_H
+
+#define TEST_SUCCESS	0
+#define TEST_FAILURE	1
+#define TEST_SKIP	2
+#define TEST_TODO	3
+#define TEST_BAILOUT	4
+
+struct lkl_test {
+	const char *name;
+	int (*fn)();
+	void *arg1, *arg2, *arg3;
+};
+
+/**
+ * Simple wrapper to initialize a test entry.
+ * @name - test name, it assume test function is named test_@name
+ * @vargs - arguments to be passed to the function
+ */
+#define LKL_TEST(name, ...) { #name, lkl_test_##name, __VA_ARGS__ }
+
+/**
+ * lkl_test_run - run a test suite
+ *
+ * @tests - the list of tests to run
+ * @nr - number of tests
+ * @fmt - format string to be used for suite name
+ */
+int lkl_test_run(const struct lkl_test *tests, int nr, const char *fmt, ...);
+
+/**
+ * lkl_test_log - store a string in the test log buffer
+ * @str - the string to log (can be non-NULL terminated)
+ * @len - the string length
+ */
+void lkl_test_log(const char *str, int len);
+
+/**
+ * lkl_test_logf - printf like function to store into the test log buffer
+ * @fmt - printf format string
+ * @vargs - arguments to the format string
+ */
+int lkl_test_logf(const char *fmt, ...) __attribute__((format(printf, 1, 2)));
+
+/**
+ * LKL_TEST_CALL - create a test function as for a LKL call
+ *
+ * The test function will be named lkl_test_@name and will return
+ * TEST_SUCCESS if the called functions returns @expect. Otherwise
+ * will return TEST_FAILUIRE.
+ *
+ * @name - test name; must be unique because it is part of the the
+ * test function; the test function will be named
+ * @call - function to call
+ * @expect - expected return value for success
+ * @args - arguments to pass to the LKL call
+ */
+#define LKL_TEST_CALL(name, call, expect, ...)				\
+	static int lkl_test_##name(void)				\
+	{								\
+		long ret;						\
+									\
+		ret = call(__VA_ARGS__);				\
+		lkl_test_logf("%s(%s) = %ld %s\n", #call, #__VA_ARGS__, \
+			ret, ret < 0 ? lkl_strerror(ret) : "");		\
+		return (ret == expect) ? TEST_SUCCESS : TEST_FAILURE;	\
+	}
+
+
+#endif /* _LKL_TEST_H */
diff --git a/tools/lkl/tests/test.sh b/tools/lkl/tests/test.sh
new file mode 100644
index 000000000000..f4d652b9fbe8
--- /dev/null
+++ b/tools/lkl/tests/test.sh
@@ -0,0 +1,182 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+basedir=$(cd $script_dir/..; pwd)
+base_objdir=$(cd ${OUTPUT}/; pwd)
+source ${script_dir}/autoconf.sh
+
+TEST_SUCCESS=0
+TEST_FAILURE=1
+TEST_SKIP=113
+TEST_TODO=114
+TEST_BAILOUT=115
+
+print_log()
+{
+    echo " log: |"
+    while read line; do
+        echo "  $line"
+    done < $1
+}
+
+export_vars()
+{
+    if [ -z "$var_file" ]; then
+        return
+    fi
+
+    for i in $@; do
+        echo "$i=${!i}" >> $var_file
+    done
+}
+
+lkl_test_run()
+{
+    log_file=$(mktemp)
+    export var_file=$(mktemp)
+
+    tid=$1 && shift && tname=$@
+
+    echo "* $tid $tname"
+
+    start=$(date '+%s%9N')
+    # run in a separate shell to avoid -e terminating us
+    $@ 2>&1 | strings >$log_file
+    exit=${PIPESTATUS[0]}
+    stop=$(date '+%s%9N')
+
+    case $exit in
+    $TEST_SUCCESS)
+        echo "ok $tid $tname"
+        ;;
+    $TEST_SKIP)
+        echo "ok $tid $tname # SKIP"
+        ;;
+    $TEST_BAILOUT)
+        echo "not ok $tid $tname"
+        echo "Bail out!"
+        ;;
+    $TEST_FAILURE|*)
+        echo "not ok $tid $tname"
+        ;;
+    esac
+
+    delta=$(((stop-start)/1000))
+
+    echo " ---"
+    echo " time_us: $delta"
+    print_log $log_file
+    echo -e " ..."
+
+    rm $log_file
+    . $var_file
+    rm $var_file
+
+    return $exit
+}
+
+lkl_test_plan()
+{
+    echo "1..$1 # $2"
+    export suite_name="${2// /\-}"
+}
+
+lkl_test_exec()
+{
+    local SUDO=""
+    local WRAPPER=""
+
+    if [ "$1" = "sudo" ]; then
+        SUDO=sudo
+        shift
+    fi
+
+    local file=$1
+    shift
+
+    if [ -n "$LKL_HOST_CONFIG_NT" ]; then
+        file=$file.exe
+    fi
+
+    file=${OUTPUT}/tests/$(basename $file)
+
+    if file $file | grep ARM; then
+        WRAPPER="qemu-arm-static"
+    elif file $file | grep "FreeBSD" ; then
+        ssh_copy "$file" $BSD_WDIR
+        if [ -n "$SUDO" ]; then
+            SUDO=""
+        fi
+        WRAPPER="$MYSSH $SU"
+        # ssh will mess up with pipes ('|') so, escape the pipe char.
+        args="${@//\|/\\\|}"
+        set - $BSD_WDIR/$(basename $file) $args
+        file=""
+    elif [ -n "$GDB" ]; then
+        WRAPPER="gdb"
+        args="$@"
+        set - -ex "run $args" -ex quit $file
+        file=""
+    elif [ -n "$VALGRIND" ]; then
+        WRAPPER="valgrind --suppressions=$script_dir/valgrind.supp \
+                  --leak-check=full --show-leak-kinds=all --xml=yes \
+                  --xml-file=valgrind-$suite_name.xml"
+    fi
+
+    $SUDO $WRAPPER $file "$@"
+}
+
+lkl_test_cmd()
+{
+    local WRAPPER=""
+
+    if [ -z "$QUIET" ]; then
+        SHOPTS="-x"
+    fi
+
+    if [ -n "$LKL_HOST_CONFIG_BSD" ]; then
+        WRAPPER="$MYSSH $SU"
+    fi
+
+    echo "$@" | $WRAPPER sh $SHOPTS
+}
+
+# XXX: $MYSSH and $MYSCP are defined in a circleci docker image.
+# see the definitions in lkl/lkl-docker:circleci/freebsd11/Dockerfile
+ssh_push()
+{
+    while [ -n "$1" ]; do
+        if [[ "$1" = *.sh ]]; then
+            type="script"
+        else
+            type="file"
+        fi
+
+        dir=$(dirname $1)
+        $MYSSH mkdir -p $BSD_WDIR/$dir
+
+        $MYSCP -P 7722 -r $basedir/$1 root@localhost:$BSD_WDIR/$dir
+        if [ "$type" = "script" ]; then
+            $MYSSH chmod a+x $BSD_WDIR/$1
+        fi
+
+        shift
+    done
+}
+
+ssh_copy()
+{
+    $MYSCP -P 7722 -r $1 root@localhost:$2
+}
+
+lkl_test_bsd_cleanup()
+{
+    $MYSSH rm -rf $BSD_WDIR
+}
+
+if [ -n "$LKL_HOST_CONFIG_BSD" ]; then
+    trap lkl_test_bsd_cleanup EXIT
+    export BSD_WDIR=/root/lkl
+    $MYSSH mkdir -p $BSD_WDIR
+fi
diff --git a/tools/lkl/tests/valgrind.supp b/tools/lkl/tests/valgrind.supp
new file mode 100644
index 000000000000..2a0e270f2292
--- /dev/null
+++ b/tools/lkl/tests/valgrind.supp
@@ -0,0 +1,105 @@
+{
+   <unfinished timer 1>
+   Memcheck:Leak
+   match-leak-kinds: possible
+   ...
+   fun:pthread_create@@GLIBC_2.2.5
+   fun:__start_helper_thread
+   fun:__pthread_once_slow
+   fun:timer_create@@GLIBC_2.3.3
+   fun:timer_alloc
+   fun:clockevent_set_state_oneshot
+   ...
+   fun:__clockevents_switch_state
+   fun:clockevents_switch_state
+   fun:tick_setup_periodic
+   ...
+}
+
+{
+   <unfinished timer 2>
+   Memcheck:Leak
+   match-leak-kinds: possible
+   ...
+   fun:allocate_stack
+   fun:pthread_create@@GLIBC_2.2.5
+   fun:timer_helper_thread
+   fun:start_thread
+   ...
+}
+
+{
+   <pid1 kernel thread>
+   Memcheck:Leak
+   match-leak-kinds: possible
+   ...
+   fun:thread_create
+   fun:copy_thread
+   fun:copy_thread_tls
+   ...
+   fun:rest_init
+   ...
+   fun:start_kernel
+   ...
+}
+
+{
+   <xfs uninitialized buf error: delete this once upstream is fixed>
+   Memcheck:Value8
+   fun:crc32_body
+   fun:crc32_le_generic
+   fun:__crc32c_le
+   fun:chksum_update
+   fun:crypto_shash_update
+   fun:crc32c
+   fun:xlog_cksum
+}
+
+{
+   <xfs pwrite64 issue: delete this once upstream is fixed>
+   Memcheck:Param
+   pwrite64(buf)
+   ...
+   fun:blk_request
+   fun:blk_enqueue
+   fun:virtio_process_one
+   fun:virtio_process_queue
+   fun:virtio_write
+   fun:__raw_writel
+   fun:writel
+   fun:vm_notify
+   fun:virtqueue_notify
+   fun:virtio_queue_rq
+   fun:blk_mq_dispatch_rq_list
+   fun:blk_mq_sched_dispatch_requests
+}
+
+{
+   <virtio_net_pipe xmits>
+   Memcheck:Param
+   writev(vector[...])
+   ...
+   fun:fd_net_tx
+   fun:net_enqueue
+   fun:virtio_process_one
+   fun:virtio_process_queue
+   fun:virtio_write
+   fun:__raw_writel
+   fun:writel
+   fun:vm_notify
+   ...
+   fun:netdev_start_xmit
+   ...
+}
+
+{
+   <ubd_kern.c>
+   Memcheck:Param
+   read(buf)
+   ...
+   fun:read
+   fun:os_read_file
+   fun:bulk_req_safe_read.constprop.11
+   fun:io_thread
+   ...
+}
diff --git a/tools/lkl/tests/valgrind2xunit.py b/tools/lkl/tests/valgrind2xunit.py
new file mode 100755
index 000000000000..ab7c12b83377
--- /dev/null
+++ b/tools/lkl/tests/valgrind2xunit.py
@@ -0,0 +1,69 @@
+#!/usr/bin/env python
+# SPDX-License-Identifier: GPL-2.0
+
+##
+## Downloader from
+## http://humdi.net/wiki/tips/valgrind-to-xunit-xml-converter
+##
+
+import xml.etree.ElementTree as ET
+import sys
+import os
+
+fname = sys.argv[1]
+if fname is None:
+    fname = 'valgrind.xml'
+
+doc = ET.parse(fname)
+errors = doc.findall('.//error')
+
+out = open(os.path.splitext(os.path.basename(fname))[0]+'_xunit.xml',"w")
+out.write('<?xml version="1.0" encoding="UTF-8"?>\n')
+out.write('<testsuite name="valgrind" tests="'+str(len(errors))+'" errors="0" failures="'+str(len(errors))+'" skip="0">\n')
+errorcount=0
+for error in errors:
+    errorcount=errorcount+1
+
+    kind = error.find('kind')
+    what = error.find('what')
+    if  what == None:
+        what = error.find('xwhat/text')
+
+    stack = error.find('stack')
+    frames = stack.findall('frame')
+
+    for frame in frames:
+        fi = frame.find('file')
+        li = frame.find('line')
+        if fi != None and li != None:
+            break
+
+    if fi != None and li != None:
+        out.write('    <testcase classname="ValgrindMemoryCheck" name="Memory check '+str(errorcount)+' ('+kind.text+', '+fi.text+':'+li.text+')" time="0">\n')
+    else:
+        out.write('    <testcase classname="ValgrindMemoryCheck" name="Memory check '+str(errorcount)+' ('+kind.text+')" time="0">\n')
+    out.write('        <error type="'+kind.text+'">\n')
+    out.write('  '+what.text+'\n\n')
+
+    for frame in frames:
+        ip = frame.find('ip')
+        fn = frame.find('fn')
+        fi = frame.find('file')
+        li = frame.find('line')
+
+        if fn is None:
+            bodytext = '(unresolved symbol)'
+        else:
+            bodytext = fn.text
+        bodytext = bodytext.replace("&","&amp;")
+        bodytext = bodytext.replace("<","&lt;")
+        bodytext = bodytext.replace(">","&gt;")
+        if fi != None and li != None:
+            out.write('  '+ip.text+': '+bodytext+' ('+fi.text+':'+li.text+')\n')
+        else:
+            out.write('  '+ip.text+': '+bodytext+'\n')
+
+    out.write('        </error>\n')
+    out.write('    </testcase>\n')
+out.write('</testsuite>\n')
+out.close()
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 17/25] lkl tools: add test programs
@ 2020-03-30 14:45     ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: linux-arch, Patrick Collins, Hajime Tazaki, Conrad Meyer,
	Octavian Purdila, Motomu Utsumi, Lai Jiangshan, Akira Moroo,
	Petros Angelatos, Thomas Liebetraut, Mark Stillwell,
	David Disseldorp, linux-kernel-library, Luca Dariz, Yuan Liu

From: Octavian Purdila <tavi.purdila@gmail.com>

Add a simple LKL test application (boot) that starts the kernel and
performs simple tests that minimally exercise the LKL API. Networking
and block device tests are also added which are useful to detect
regressions.

Cc: Conrad Meyer <cem@FreeBSD.org>
Cc: David Disseldorp <ddiss@suse.de>
Cc: H.K. Jerry Chu <hkchu@google.com>
Cc: Lai Jiangshan <jiangshanlai@gmail.com>
Cc: Luca Dariz <luca.dariz@gmail.com>
Cc: Mark Stillwell <mark@stillwell.me>
Cc: Motomu Utsumi <motomuman@gmail.com>
Cc: Patrick Collins <pscollins@google.com>
Cc: Petros Angelatos <petrosagg@gmail.com>
Cc: Thomas Liebetraut <thomas@tommie-lie.de>
Cc: Yuan Liu <liuyuan@google.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/lkl/.gitignore              |   5 +
 tools/lkl/Makefile                |   5 +-
 tools/lkl/Targets                 |   3 +
 tools/lkl/tests/Build             |   3 +
 tools/lkl/tests/boot.c            | 562 ++++++++++++++++++++++++++++++
 tools/lkl/tests/boot.sh           |   9 +
 tools/lkl/tests/cla.c             | 159 +++++++++
 tools/lkl/tests/cla.h             |  33 ++
 tools/lkl/tests/disk.c            | 189 ++++++++++
 tools/lkl/tests/disk.sh           |  61 ++++
 tools/lkl/tests/net-setup.sh      |  73 ++++
 tools/lkl/tests/net-test.c        | 299 ++++++++++++++++
 tools/lkl/tests/net.sh            |  90 +++++
 tools/lkl/tests/run.py            | 179 ++++++++++
 tools/lkl/tests/tap13.py          | 209 +++++++++++
 tools/lkl/tests/test.c            | 126 +++++++
 tools/lkl/tests/test.h            |  72 ++++
 tools/lkl/tests/test.sh           | 182 ++++++++++
 tools/lkl/tests/valgrind.supp     | 105 ++++++
 tools/lkl/tests/valgrind2xunit.py |  69 ++++
 20 files changed, 2432 insertions(+), 1 deletion(-)
 create mode 100644 tools/lkl/tests/Build
 create mode 100644 tools/lkl/tests/boot.c
 create mode 100755 tools/lkl/tests/boot.sh
 create mode 100644 tools/lkl/tests/cla.c
 create mode 100644 tools/lkl/tests/cla.h
 create mode 100644 tools/lkl/tests/disk.c
 create mode 100755 tools/lkl/tests/disk.sh
 create mode 100644 tools/lkl/tests/net-setup.sh
 create mode 100644 tools/lkl/tests/net-test.c
 create mode 100755 tools/lkl/tests/net.sh
 create mode 100755 tools/lkl/tests/run.py
 create mode 100644 tools/lkl/tests/tap13.py
 create mode 100644 tools/lkl/tests/test.c
 create mode 100644 tools/lkl/tests/test.h
 create mode 100644 tools/lkl/tests/test.sh
 create mode 100644 tools/lkl/tests/valgrind.supp
 create mode 100755 tools/lkl/tests/valgrind2xunit.py

diff --git a/tools/lkl/.gitignore b/tools/lkl/.gitignore
index 1aed58bfe171..4e08254dbd46 100644
--- a/tools/lkl/.gitignore
+++ b/tools/lkl/.gitignore
@@ -2,3 +2,8 @@ Makefile.conf
 include/lkl_autoconf.h
 tests/autoconf.sh
 bin/stat
+tests/net-test
+tests/disk
+tests/boot
+tests/valgrind*.xml
+*.pyc
diff --git a/tools/lkl/Makefile b/tools/lkl/Makefile
index 11ea7e9095bb..0abccb9d86b6 100644
--- a/tools/lkl/Makefile
+++ b/tools/lkl/Makefile
@@ -117,8 +117,11 @@ programs_install: $(progs-y:%=$(OUTPUT)%$(EXESUF))
 install: headers_install libraries_install programs_install
 
 
+run-tests:
+	./tests/run.py $(tests)
+
 FORCE: ;
-.PHONY: all clean FORCE
+.PHONY: all clean FORCE run-tests
 .PHONY: headers_install libraries_install programs_install install
 .NOTPARALLEL : lib/lkl.o
 .SECONDARY:
diff --git a/tools/lkl/Targets b/tools/lkl/Targets
index 24c985e64638..a9f74c3cc8fb 100644
--- a/tools/lkl/Targets
+++ b/tools/lkl/Targets
@@ -1,3 +1,6 @@
 libs-y += lib/liblkl
 
+progs-y += tests/boot
+progs-y += tests/disk
+progs-y += tests/net-test
 
diff --git a/tools/lkl/tests/Build b/tools/lkl/tests/Build
new file mode 100644
index 000000000000..ace86a3d3438
--- /dev/null
+++ b/tools/lkl/tests/Build
@@ -0,0 +1,3 @@
+boot-y += boot.o test.o
+disk-y += disk.o cla.o test.o
+net-test-y += net-test.o cla.o test.o
diff --git a/tools/lkl/tests/boot.c b/tools/lkl/tests/boot.c
new file mode 100644
index 000000000000..b021e9540147
--- /dev/null
+++ b/tools/lkl/tests/boot.c
@@ -0,0 +1,562 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <unistd.h>
+#include <string.h>
+#include <time.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <lkl.h>
+#include <lkl_host.h>
+
+#include <sys/stat.h>
+#include <fcntl.h>
+#if defined(__FreeBSD__)
+#include <net/if.h>
+#include <sys/ioctl.h>
+#elif __linux
+#include <sys/epoll.h>
+#include <sys/ioctl.h>
+#elif __MINGW32__
+#include <windows.h>
+#endif
+
+#include "test.h"
+
+#ifndef __MINGW32__
+#define sleep_ns 87654321
+int lkl_test_nanosleep(void)
+{
+	struct lkl_timespec ts = {
+		.tv_sec = 0,
+		.tv_nsec = sleep_ns,
+	};
+	struct timespec start, stop;
+	long delta;
+	long ret;
+
+	clock_gettime(CLOCK_MONOTONIC, &start);
+	ret = lkl_sys_nanosleep((struct __lkl__kernel_timespec *)&ts, NULL);
+	clock_gettime(CLOCK_MONOTONIC, &stop);
+
+	delta = 1e9*(stop.tv_sec - start.tv_sec) +
+		(stop.tv_nsec - start.tv_nsec);
+
+	lkl_test_logf("sleep %ld, expected sleep %d\n", delta, sleep_ns);
+
+	if (ret == 0 && delta > sleep_ns * 0.9)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+#endif
+
+LKL_TEST_CALL(getpid, lkl_sys_getpid, 1)
+
+void check_latency(long (*f)(void), long *min, long *max, long *avg)
+{
+	int i;
+	unsigned long long start, stop, sum = 0;
+	static const int count = 1000;
+	long delta;
+
+	*min = 1000000000;
+	*max = -1;
+
+	for (i = 0; i < count; i++) {
+		start = lkl_host_ops.time();
+		f();
+		stop = lkl_host_ops.time();
+		delta = stop - start;
+		if (*min > delta)
+			*min = delta;
+		if (*max < delta)
+			*max = delta;
+		sum += delta;
+	}
+	*avg = sum / count;
+}
+
+static long native_getpid(void)
+{
+#ifdef __MINGW32__
+	GetCurrentProcessId();
+#else
+	getpid();
+#endif
+	return 0;
+}
+
+int lkl_test_syscall_latency(void)
+{
+	long min, max, avg;
+
+	lkl_test_logf("avg/min/max: ");
+
+	check_latency(lkl_sys_getpid, &min, &max, &avg);
+
+	lkl_test_logf("lkl:%ld/%ld/%ld ", avg, min, max);
+
+	check_latency(native_getpid, &min, &max, &avg);
+
+	lkl_test_logf("native:%ld/%ld/%ld\n", avg, min, max);
+
+	return TEST_SUCCESS;
+}
+
+#define access_rights 0721
+
+LKL_TEST_CALL(creat, lkl_sys_creat, 0, "/file", access_rights)
+LKL_TEST_CALL(close, lkl_sys_close, 0, 0);
+LKL_TEST_CALL(failopen, lkl_sys_open, -LKL_ENOENT, "/file2", 0, 0);
+LKL_TEST_CALL(umask, lkl_sys_umask, 022,  0777);
+LKL_TEST_CALL(umask2, lkl_sys_umask, 0777, 0);
+LKL_TEST_CALL(open, lkl_sys_open, 0, "/file", LKL_O_RDWR, 0);
+static const char wrbuf[] = "test";
+LKL_TEST_CALL(write, lkl_sys_write, sizeof(wrbuf), 0, wrbuf, sizeof(wrbuf));
+LKL_TEST_CALL(lseek_cur, lkl_sys_lseek, sizeof(wrbuf), 0, 0, LKL_SEEK_CUR);
+LKL_TEST_CALL(lseek_end, lkl_sys_lseek, sizeof(wrbuf), 0, 0, LKL_SEEK_END);
+LKL_TEST_CALL(lseek_set, lkl_sys_lseek, 0, 0, 0, LKL_SEEK_SET);
+
+int lkl_test_read(void)
+{
+	char buf[10] = { 0, };
+	long ret;
+
+	ret = lkl_sys_read(0, buf, sizeof(buf));
+
+	lkl_test_logf("lkl_sys_read=%ld buf=%s\n", ret, buf);
+
+	if (ret == sizeof(wrbuf) && !strcmp(wrbuf, buf))
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+int lkl_test_fstat(void)
+{
+	struct lkl_stat stat;
+	long ret;
+
+	ret = lkl_sys_fstat(0, &stat);
+
+	lkl_test_logf("lkl_sys_fstat=%ld mode=%o size=%zd\n", ret, stat.st_mode,
+		      stat.st_size);
+
+	if (ret == 0 && stat.st_size == sizeof(wrbuf) &&
+	    stat.st_mode == (access_rights | LKL_S_IFREG))
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+LKL_TEST_CALL(mkdir, lkl_sys_mkdir, 0, "/mnt", access_rights)
+
+int lkl_test_stat(void)
+{
+	struct lkl_stat stat;
+	long ret;
+
+	ret = lkl_sys_stat("/mnt", &stat);
+
+	lkl_test_logf("lkl_sys_stat(\"/mnt\")=%ld mode=%o\n", ret,
+		      stat.st_mode);
+
+	if (ret == 0 && stat.st_mode == (access_rights | LKL_S_IFDIR))
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+static int lkl_test_pipe2(void)
+{
+	int pipe_fds[2];
+	int READ_IDX = 0, WRITE_IDX = 1;
+	static const char msg[] = "Hello world!";
+	char str[20];
+	int msg_len_bytes = strlen(msg) + 1;
+	int cmp_res;
+	long ret;
+
+	ret = lkl_sys_pipe2(pipe_fds, LKL_O_NONBLOCK);
+	if (ret) {
+		lkl_test_logf("pipe2: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_write(pipe_fds[WRITE_IDX], msg, msg_len_bytes);
+	if (ret != msg_len_bytes) {
+		if (ret < 0)
+			lkl_test_logf("write error: %s\n", lkl_strerror(ret));
+		else
+			lkl_test_logf("short write: %ld\n", ret);
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_read(pipe_fds[READ_IDX], str, msg_len_bytes);
+	if (ret != msg_len_bytes) {
+		if (ret < 0)
+			lkl_test_logf("read error: %s\n", lkl_strerror(ret));
+		else
+			lkl_test_logf("short read: %ld\n", ret);
+		return TEST_FAILURE;
+	}
+
+	cmp_res = memcmp(msg, str, msg_len_bytes);
+	if (cmp_res) {
+		lkl_test_logf("memcmp failed: %d\n", cmp_res);
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_close(pipe_fds[0]);
+	if (ret) {
+		lkl_test_logf("close error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_close(pipe_fds[1]);
+	if (ret) {
+		lkl_test_logf("close error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	return TEST_SUCCESS;
+}
+
+static int lkl_test_epoll(void)
+{
+	int epoll_fd, pipe_fds[2];
+	int READ_IDX = 0, WRITE_IDX = 1;
+	struct lkl_epoll_event wait_on, read_result;
+	static const char msg[] = "Hello world!";
+	long ret;
+
+	memset(&wait_on, 0, sizeof(wait_on));
+	memset(&read_result, 0, sizeof(read_result));
+
+	ret = lkl_sys_pipe2(pipe_fds, LKL_O_NONBLOCK);
+	if (ret) {
+		lkl_test_logf("pipe2 error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	epoll_fd = lkl_sys_epoll_create(1);
+	if (epoll_fd < 0) {
+		lkl_test_logf("epoll_create error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	wait_on.events = LKL_POLLIN | LKL_POLLOUT;
+	wait_on.data = pipe_fds[READ_IDX];
+
+	ret = lkl_sys_epoll_ctl(epoll_fd, LKL_EPOLL_CTL_ADD, pipe_fds[READ_IDX],
+				&wait_on);
+	if (ret < 0) {
+		lkl_test_logf("epoll_ctl error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	/* Shouldn't be ready before we have written something */
+	ret = lkl_sys_epoll_wait(epoll_fd, &read_result, 1, 0);
+	if (ret != 0) {
+		if (ret < 0)
+			lkl_test_logf("epoll_wait error: %s\n",
+				      lkl_strerror(ret));
+		else
+			lkl_test_logf("epoll_wait: bad event: 0x%lx\n", ret);
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_write(pipe_fds[WRITE_IDX], msg, strlen(msg) + 1);
+	if (ret < 0) {
+		lkl_test_logf("write error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	/* We expect exactly 1 fd to be ready immediately */
+	ret = lkl_sys_epoll_wait(epoll_fd, &read_result, 1, 0);
+	if (ret != 1) {
+		if (ret < 0)
+			lkl_test_logf("epoll_wait error: %s\n",
+				      lkl_strerror(ret));
+		else
+			lkl_test_logf("epoll_wait: bad ev no %ld\n", ret);
+		return TEST_FAILURE;
+	}
+
+	/* Already tested reading from pipe2 so no need to do it
+	 * here
+	 */
+
+	return TEST_SUCCESS;
+}
+
+LKL_TEST_CALL(chdir_proc, lkl_sys_chdir, 0, "proc");
+
+static int dir_fd;
+
+static int lkl_test_open_cwd(void)
+{
+	dir_fd = lkl_sys_open(".", LKL_O_RDONLY | LKL_O_DIRECTORY, 0);
+	if (dir_fd < 0) {
+		lkl_test_logf("failed to open current directory: %s\n",
+			      lkl_strerror(dir_fd));
+		return TEST_FAILURE;
+	}
+
+	return TEST_SUCCESS;
+}
+
+/* column where to insert a line break for the list file tests below. */
+#define COL_LINE_BREAK 70
+
+static int lkl_test_getdents64(void)
+{
+	long ret;
+	char buf[1024], *pos;
+	struct lkl_linux_dirent64 *de;
+	int wr;
+
+	de = (struct lkl_linux_dirent64 *)buf;
+	ret = lkl_sys_getdents64(dir_fd, de, sizeof(buf));
+
+	wr = lkl_test_logf("%d ", dir_fd);
+
+	if (ret < 0)
+		return TEST_FAILURE;
+
+	for (pos = buf; pos - buf < ret; pos += de->d_reclen) {
+		de = (struct lkl_linux_dirent64 *)pos;
+
+		wr += lkl_test_logf("%s ", de->d_name);
+		if (wr >= COL_LINE_BREAK) {
+			lkl_test_logf("\n");
+			wr = 0;
+		}
+	}
+
+	return TEST_SUCCESS;
+}
+
+LKL_TEST_CALL(close_dir_fd, lkl_sys_close, 0, dir_fd);
+LKL_TEST_CALL(chdir_root, lkl_sys_chdir, 0, "/");
+LKL_TEST_CALL(mount_fs_proc, lkl_mount_fs, 0, "proc");
+LKL_TEST_CALL(umount_fs_proc, lkl_umount_timeout, 0, "proc", 0, 1000);
+LKL_TEST_CALL(lo_ifup, lkl_if_up, 0, 1);
+
+static int lkl_test_mutex(void)
+{
+	long ret = TEST_SUCCESS;
+	/*
+	 * Can't do much to verify that this works, so we'll just let Valgrind
+	 * warn us on CI if we've made bad memory accesses.
+	 */
+
+	struct lkl_mutex *mutex;
+
+	mutex = lkl_host_ops.mutex_alloc(0);
+	lkl_host_ops.mutex_lock(mutex);
+	lkl_host_ops.mutex_unlock(mutex);
+	lkl_host_ops.mutex_free(mutex);
+
+	mutex = lkl_host_ops.mutex_alloc(1);
+	lkl_host_ops.mutex_lock(mutex);
+	lkl_host_ops.mutex_lock(mutex);
+	lkl_host_ops.mutex_unlock(mutex);
+	lkl_host_ops.mutex_unlock(mutex);
+	lkl_host_ops.mutex_free(mutex);
+
+	return ret;
+}
+
+static int lkl_test_semaphore(void)
+{
+	long ret = TEST_SUCCESS;
+	/*
+	 * Can't do much to verify that this works, so we'll just let Valgrind
+	 * warn us on CI if we've made bad memory accesses.
+	 */
+
+	struct lkl_sem *sem = lkl_host_ops.sem_alloc(1);
+
+	lkl_host_ops.sem_down(sem);
+	lkl_host_ops.sem_up(sem);
+	lkl_host_ops.sem_free(sem);
+
+	return ret;
+}
+
+static int lkl_test_gettid(void)
+{
+	long tid = lkl_host_ops.gettid();
+
+	lkl_test_logf("%ld", tid);
+
+	/* As far as I know, thread IDs are non-zero on all reasonable
+	 * systems.
+	 */
+	if (tid)
+		return TEST_SUCCESS;
+	else
+		return TEST_FAILURE;
+}
+
+static void test_thread(void *data)
+{
+	int *pipe_fds = (int *) data;
+	char tmp[LKL_PIPE_BUF+1];
+	int ret;
+
+	ret = lkl_sys_read(pipe_fds[0], tmp, sizeof(tmp));
+	if (ret < 0)
+		lkl_test_logf("%s: %s\n", __func__, lkl_strerror(ret));
+}
+
+static int lkl_test_syscall_thread(void)
+{
+	int pipe_fds[2];
+	char tmp[LKL_PIPE_BUF+1];
+	long ret;
+	lkl_thread_t tid;
+
+	ret = lkl_sys_pipe2(pipe_fds, 0);
+	if (ret) {
+		lkl_test_logf("pipe2: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_fcntl(pipe_fds[0], LKL_F_SETPIPE_SZ, 1);
+	if (ret < 0) {
+		lkl_test_logf("fcntl setpipe_sz: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	tid = lkl_host_ops.thread_create(test_thread, pipe_fds);
+	if (!tid) {
+		lkl_test_logf("failed to create thread\n");
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_write(pipe_fds[1], tmp, sizeof(tmp));
+	if (ret != sizeof(tmp)) {
+		if (ret < 0)
+			lkl_test_logf("write error: %s\n", lkl_strerror(ret));
+		else
+			lkl_test_logf("short write: %ld\n", ret);
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_host_ops.thread_join(tid);
+	if (ret) {
+		lkl_test_logf("failed to join thread\n");
+		return TEST_FAILURE;
+	}
+
+	return TEST_SUCCESS;
+}
+
+#ifndef __MINGW32__
+static void thread_get_pid(void *unused)
+{
+	lkl_sys_getpid();
+}
+
+static int lkl_test_many_syscall_threads(void)
+{
+	lkl_thread_t tid;
+	int count = 65, ret;
+
+	while (--count > 0) {
+		tid = lkl_host_ops.thread_create(thread_get_pid, NULL);
+		if (!tid) {
+			lkl_test_logf("failed to create thread\n");
+			return TEST_FAILURE;
+		}
+
+		ret = lkl_host_ops.thread_join(tid);
+		if (ret) {
+			lkl_test_logf("failed to join thread\n");
+			return TEST_FAILURE;
+		}
+	}
+
+	return TEST_SUCCESS;
+}
+#endif
+
+static void thread_quit_immediately(void *unused)
+{
+}
+
+static int lkl_test_join(void)
+{
+	lkl_thread_t tid = lkl_host_ops.thread_create(thread_quit_immediately,
+						      NULL);
+	int ret = lkl_host_ops.thread_join(tid);
+
+	if (ret == 0) {
+		lkl_test_logf("joined %ld\n", tid);
+		return TEST_SUCCESS;
+	}
+
+	lkl_test_logf("failed joining %ld\n", tid);
+	return TEST_FAILURE;
+}
+
+LKL_TEST_CALL(start_kernel, lkl_start_kernel, 0, &lkl_host_ops,
+	     "mem=16M loglevel=8");
+LKL_TEST_CALL(stop_kernel, lkl_sys_halt, 0);
+
+struct lkl_test tests[] = {
+	LKL_TEST(mutex),
+	LKL_TEST(semaphore),
+	LKL_TEST(join),
+	LKL_TEST(start_kernel),
+	LKL_TEST(getpid),
+	LKL_TEST(syscall_latency),
+	LKL_TEST(umask),
+	LKL_TEST(umask2),
+	LKL_TEST(creat),
+	LKL_TEST(close),
+	LKL_TEST(failopen),
+	LKL_TEST(open),
+	LKL_TEST(write),
+	LKL_TEST(lseek_cur),
+	LKL_TEST(lseek_end),
+	LKL_TEST(lseek_set),
+	LKL_TEST(read),
+	LKL_TEST(fstat),
+	LKL_TEST(mkdir),
+	LKL_TEST(stat),
+#ifndef __MINGW32__
+	LKL_TEST(nanosleep),
+#endif
+	LKL_TEST(pipe2),
+	LKL_TEST(epoll),
+	LKL_TEST(mount_fs_proc),
+	LKL_TEST(chdir_proc),
+	LKL_TEST(open_cwd),
+	LKL_TEST(getdents64),
+	LKL_TEST(close_dir_fd),
+	LKL_TEST(chdir_root),
+	LKL_TEST(umount_fs_proc),
+	LKL_TEST(lo_ifup),
+	LKL_TEST(gettid),
+	LKL_TEST(syscall_thread),
+	/*
+	 * Wine has an issue where the FlsCallback is not called when
+	 * the thread terminates which makes testing the automatic
+	 * syscall threads cleanup impossible under wine.
+	 */
+#ifndef __MINGW32__
+	LKL_TEST(many_syscall_threads),
+#endif
+	LKL_TEST(stop_kernel),
+};
+
+int main(int argc, const char **argv)
+{
+	lkl_host_ops.print = lkl_test_log;
+
+	return lkl_test_run(tests, sizeof(tests)/sizeof(struct lkl_test),
+			    "boot");
+}
diff --git a/tools/lkl/tests/boot.sh b/tools/lkl/tests/boot.sh
new file mode 100755
index 000000000000..d985c04b0ac1
--- /dev/null
+++ b/tools/lkl/tests/boot.sh
@@ -0,0 +1,9 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+source $script_dir/test.sh
+
+lkl_test_plan 1 "boot"
+lkl_test_run 1
+lkl_test_exec $script_dir/boot
diff --git a/tools/lkl/tests/cla.c b/tools/lkl/tests/cla.c
new file mode 100644
index 000000000000..a34badeb5f06
--- /dev/null
+++ b/tools/lkl/tests/cla.c
@@ -0,0 +1,159 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <string.h>
+#include <errno.h>
+#include <stdlib.h>
+#ifdef __MINGW32__
+#include <winsock2.h>
+#else
+#include <sys/socket.h>
+#include <netinet/in.h>
+#include <arpa/inet.h>
+#endif
+
+#include "cla.h"
+
+static int cl_arg_parse_bool(struct cl_arg *arg, const char *value)
+{
+	*((int *)arg->store) = 1;
+	return 0;
+}
+
+static int cl_arg_parse_str(struct cl_arg *arg, const char *value)
+{
+	*((const char **)arg->store) = value;
+	return 0;
+}
+
+static int cl_arg_parse_int(struct cl_arg *arg, const char *value)
+{
+	errno = 0;
+	*((int *)arg->store) = strtol(value, NULL, 0);
+	return errno == 0;
+}
+
+static int cl_arg_parse_str_set(struct cl_arg *arg, const char *value)
+{
+	const char **set = arg->set;
+	int i;
+
+	for (i = 0; set[i] != NULL; i++) {
+		if (strcmp(set[i], value) == 0) {
+			*((int *)arg->store) = i;
+			return 0;
+		}
+	}
+
+	return -1;
+}
+
+static int cl_arg_parse_ipv4(struct cl_arg *arg, const char *value)
+{
+	unsigned int addr;
+
+	if (!value)
+		return -1;
+
+	addr = inet_addr(value);
+	if (addr == INADDR_NONE)
+		return -1;
+	*((unsigned int *)arg->store) = addr;
+	return 0;
+}
+
+static cl_arg_parser_t parsers[] = {
+	[CL_ARG_BOOL] = cl_arg_parse_bool,
+	[CL_ARG_INT] = cl_arg_parse_int,
+	[CL_ARG_STR] = cl_arg_parse_str,
+	[CL_ARG_STR_SET] = cl_arg_parse_str_set,
+	[CL_ARG_IPV4] = cl_arg_parse_ipv4,
+};
+
+static struct cl_arg *find_short_arg(char name, struct cl_arg *args)
+{
+	struct cl_arg *arg;
+
+	for (arg = args; arg->short_name != 0; arg++) {
+		if (arg->short_name == name)
+			return arg;
+	}
+
+	return NULL;
+}
+
+static struct cl_arg *find_long_arg(const char *name, struct cl_arg *args)
+{
+	struct cl_arg *arg;
+
+	for (arg = args; arg->long_name; arg++) {
+		if (strcmp(arg->long_name, name) == 0)
+			return arg;
+	}
+
+	return NULL;
+}
+
+static void print_help(struct cl_arg *args)
+{
+	struct cl_arg *arg;
+
+	fprintf(stderr, "usage:\n");
+	for (arg = args; arg->long_name; arg++) {
+		fprintf(stderr, "-%c, --%-20s %s", arg->short_name,
+			arg->long_name, arg->help);
+		if (arg->type == CL_ARG_STR_SET) {
+			const char **set = arg->set;
+
+			fprintf(stderr, " [ ");
+			while (*set != NULL)
+				fprintf(stderr, "%s ", *(set++));
+			fprintf(stderr, "]");
+		}
+		fprintf(stderr, "\n");
+	}
+}
+
+int parse_args(int argc, const char **argv, struct cl_arg *args)
+{
+	int i;
+
+	for (i = 1; i < argc; i++) {
+		struct cl_arg *arg = NULL;
+		cl_arg_parser_t parser;
+
+		if (argv[i][0] == '-') {
+			if (argv[i][1] != '-')
+				arg = find_short_arg(argv[i][1], args);
+			else
+				arg = find_long_arg(&argv[i][2], args);
+		}
+
+		if (!arg) {
+			fprintf(stderr, "unknown option '%s'\n", argv[i]);
+			print_help(args);
+			return -1;
+		}
+
+		if (arg->type == CL_ARG_USER || arg->type >= CL_ARG_END)
+			parser = arg->parser;
+		else
+			parser = parsers[arg->type];
+
+		if (!parser) {
+			fprintf(stderr, "can't parse --'%s'/-'%c'\n",
+				arg->long_name, args->short_name);
+			return -1;
+		}
+
+		if (parser(arg, argv[i + 1]) < 0) {
+			fprintf(stderr, "can't parse '%s'\n", argv[i]);
+			print_help(args);
+			return -1;
+		}
+
+		if (arg->has_arg)
+			i++;
+	}
+
+	return 0;
+}
diff --git a/tools/lkl/tests/cla.h b/tools/lkl/tests/cla.h
new file mode 100644
index 000000000000..f8369be02e5a
--- /dev/null
+++ b/tools/lkl/tests/cla.h
@@ -0,0 +1,33 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_TEST_CLA_H
+#define _LKL_TEST_CLA_H
+
+enum cl_arg_type {
+	CL_ARG_USER = 0,
+	CL_ARG_BOOL,
+	CL_ARG_INT,
+	CL_ARG_STR,
+	CL_ARG_STR_SET,
+	CL_ARG_IPV4,
+	CL_ARG_END,
+};
+
+struct cl_arg;
+
+typedef int (*cl_arg_parser_t)(struct cl_arg *arg, const char *value);
+
+struct cl_arg {
+	const char *long_name;
+	char short_name;
+	const char *help;
+	int has_arg;
+	enum cl_arg_type type;
+	void *store;
+	void *set;
+	cl_arg_parser_t parser;
+};
+
+int parse_args(int argc, const char **argv, struct cl_arg *args);
+
+
+#endif /* _LKL_TEST_CLA_H */
diff --git a/tools/lkl/tests/disk.c b/tools/lkl/tests/disk.c
new file mode 100644
index 000000000000..0aa039876b54
--- /dev/null
+++ b/tools/lkl/tests/disk.c
@@ -0,0 +1,189 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <unistd.h>
+#include <string.h>
+#include <time.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <lkl.h>
+#include <lkl_host.h>
+#ifndef __MINGW32__
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <sys/ioctl.h>
+#else
+#include <windows.h>
+#endif
+
+#include "test.h"
+#include "cla.h"
+
+static struct {
+	int printk;
+	const char *disk;
+	const char *fstype;
+	int partition;
+} cla;
+
+struct cl_arg args[] = {
+	{"disk", 'd', "disk file to use", 1, CL_ARG_STR, &cla.disk},
+	{"partition", 'P', "partition to mount", 1, CL_ARG_INT, &cla.partition},
+	{"type", 't', "filesystem type", 1, CL_ARG_STR, &cla.fstype},
+	{0},
+};
+
+
+static struct lkl_disk disk;
+static int disk_id = -1;
+
+int lkl_test_disk_add(void)
+{
+#ifdef __MINGW32__
+	disk.handle = CreateFile(cla.disk, GENERIC_READ | GENERIC_WRITE,
+			       0, NULL, OPEN_EXISTING, 0, NULL);
+	if (!disk.handle)
+#else
+	disk.fd = open(cla.disk, O_RDWR);
+	if (disk.fd < 0)
+#endif
+		goto out_unlink;
+
+	disk.ops = NULL;
+
+	disk_id = lkl_disk_add(&disk);
+	if (disk_id < 0)
+		goto out_close;
+
+	goto out;
+
+out_close:
+#ifdef __MINGW32__
+	CloseHandle(disk.handle);
+#else
+	close(disk.fd);
+#endif
+
+out_unlink:
+#ifdef __MINGW32__
+	DeleteFile(cla.disk);
+#else
+	unlink(cla.disk);
+#endif
+
+out:
+	lkl_test_logf("disk fd/handle %x disk_id %d", disk.fd, disk_id);
+
+	if (disk_id >= 0)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+int lkl_test_disk_remove(void)
+{
+	int ret;
+
+	ret = lkl_disk_remove(disk);
+
+#ifdef __MINGW32__
+	CloseHandle(disk.handle);
+#else
+	close(disk.fd);
+#endif
+
+	if (ret == 0)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+
+static char mnt_point[32];
+
+LKL_TEST_CALL(mount_dev, lkl_mount_dev, 0, disk_id, cla.partition, cla.fstype,
+	      0, NULL, mnt_point, sizeof(mnt_point))
+
+static int lkl_test_umount_dev(void)
+{
+	long ret, ret2;
+
+	ret = lkl_sys_chdir("/");
+
+	ret2 = lkl_umount_dev(disk_id, cla.partition, 0, 1000);
+
+	lkl_test_logf("%ld %ld", ret, ret2);
+
+	if (!ret && !ret2)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+struct lkl_dir *dir;
+
+static int lkl_test_opendir(void)
+{
+	int err;
+
+	dir = lkl_opendir(mnt_point, &err);
+
+	lkl_test_logf("lkl_opedir(%s) = %d %s\n", mnt_point, err,
+		      lkl_strerror(err));
+
+	if (err == 0)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+static int lkl_test_readdir(void)
+{
+	struct lkl_linux_dirent64 *de = lkl_readdir(dir);
+	int wr = 0;
+
+	while (de) {
+		wr += lkl_test_logf("%s ", de->d_name);
+		if (wr >= 70) {
+			lkl_test_logf("\n");
+			wr = 0;
+			break;
+		}
+		de = lkl_readdir(dir);
+	}
+
+	if (lkl_errdir(dir) == 0)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+LKL_TEST_CALL(closedir, lkl_closedir, 0, dir);
+LKL_TEST_CALL(chdir_mnt_point, lkl_sys_chdir, 0, mnt_point);
+LKL_TEST_CALL(start_kernel, lkl_start_kernel, 0, &lkl_host_ops,
+	     "mem=16M loglevel=8");
+LKL_TEST_CALL(stop_kernel, lkl_sys_halt, 0);
+
+struct lkl_test tests[] = {
+	LKL_TEST(disk_add),
+	LKL_TEST(start_kernel),
+	LKL_TEST(mount_dev),
+	LKL_TEST(chdir_mnt_point),
+	LKL_TEST(opendir),
+	LKL_TEST(readdir),
+	LKL_TEST(closedir),
+	LKL_TEST(umount_dev),
+	LKL_TEST(stop_kernel),
+	LKL_TEST(disk_remove),
+
+};
+
+int main(int argc, const char **argv)
+{
+	if (parse_args(argc, argv, args) < 0)
+		return -1;
+
+	lkl_host_ops.print = lkl_test_log;
+
+	return lkl_test_run(tests, sizeof(tests)/sizeof(struct lkl_test),
+			    "disk %s", cla.fstype);
+}
diff --git a/tools/lkl/tests/disk.sh b/tools/lkl/tests/disk.sh
new file mode 100755
index 000000000000..9bdcb16f2d5c
--- /dev/null
+++ b/tools/lkl/tests/disk.sh
@@ -0,0 +1,61 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+
+source $script_dir/test.sh
+
+function prepfs()
+{
+    set -e
+
+    file=`mktemp`
+
+    dd if=/dev/zero of=$file bs=1024 count=204800
+
+    yes | mkfs.$1 $file
+
+    if ! [ -z $BSD_WDIR ]; then
+        $MYSSH mkdir -p $BSD_WDIR
+        ssh_copy $file $BSD_WDIR
+        rm $file
+        file=$BSD_WDIR/$(basename $file)
+    fi
+
+    export_vars file
+}
+
+function cleanfs()
+{
+    set -e
+
+    if ! [ -z $BSD_WDIR ]; then
+        $MYSSH rm $1
+        $MYSSH rm $BSD_WDIR/disk
+    else
+        rm $1
+    fi
+}
+
+if [ "$1" = "-t" ]; then
+    shift
+    fstype=$1
+    shift
+fi
+
+if [ -z "$fstype" ]; then
+    fstype="ext4"
+fi
+
+if [ -z $(which mkfs.$fstype) ]; then
+    lkl_test_plan 0 "disk $fstype"
+    echo "no mkfs.$fstype command"
+    exit 0
+fi
+
+lkl_test_plan 1 "disk $fstype"
+lkl_test_run 1 prepfs $fstype
+lkl_test_exec $script_dir/disk -d $file -t $fstype $@
+lkl_test_plan 1 "disk $fstype"
+lkl_test_run 1 cleanfs $file
+
diff --git a/tools/lkl/tests/net-setup.sh b/tools/lkl/tests/net-setup.sh
new file mode 100644
index 000000000000..0cfc42a30a54
--- /dev/null
+++ b/tools/lkl/tests/net-setup.sh
@@ -0,0 +1,73 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+TEST_IP_NETWORK=192.168.113.0
+TEST_IP_NETMASK=24
+TEST_IP6_NETWORK=fc03::0
+TEST_IP6_NETMASK=64
+TEST_MAC0="aa:bb:cc:dd:ee:ff"
+TEST_MAC1="aa:bb:cc:dd:ee:aa"
+TEST_NETSERVER_PORT=11223
+
+# $1 - count
+# $2 - netcount
+ip_add()
+{
+    IP_HEX=$(printf '%.2X%.2X%.2X%.2X\n' \
+         `echo $TEST_IP_NETWORK | sed -e 's/\./ /g'`)
+    NET_COUNT=$(( 1 << (32 - $TEST_IP_NETMASK) ))
+    NEXT_IP_HEX=$(printf %.8X `echo $((0x$IP_HEX + $1 + ${2:-0} * $NET_COUNT))`)
+    NEXT_IP=$(printf '%d.%d.%d.%d\n' \
+          `echo $NEXT_IP_HEX | sed -r 's/(..)/0x\1 /g'`)
+    echo -n "$NEXT_IP"
+}
+
+# $1 - count
+# $2 - netcount
+ip6_add()
+{
+    IP6_PREFIX=${TEST_IP6_NETWORK%*::*}
+    IP6_HOST=${TEST_IP6_NETWORK#*::*}
+    echo -n "$(printf "%x" $((0x$IP6_PREFIX+${2:-0})))::$(($IP6_HOST+$1))"
+}
+
+ip_host()
+{
+
+    ip_add 1 $1
+}
+
+ip_lkl()
+{
+    ip_add 2 $1
+}
+
+ip_host_mask()
+{
+    echo -n "$(ip_host $1)/$TEST_IP_NETMASK"
+}
+
+ip_net_mask()
+{
+    echo "$(ip_add 0 $1)/$TEST_IP_NETMASK"
+}
+
+ip6_host()
+{
+    ip6_add 1 $1
+}
+
+ip6_lkl()
+{
+    ip6_add 2 $1
+}
+
+ip6_host_mask()
+{
+    echo -n "$(ip6_host $1)/$TEST_IP6_NETMASK"
+}
+
+ip6_net_mask()
+{
+    echo "$(ip6_add 0 $1)/$TEST_IP6_NETMASK"
+}
diff --git a/tools/lkl/tests/net-test.c b/tools/lkl/tests/net-test.c
new file mode 100644
index 000000000000..5a8e39a52417
--- /dev/null
+++ b/tools/lkl/tests/net-test.c
@@ -0,0 +1,299 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <string.h>
+#include <stdlib.h>
+#include <errno.h>
+#ifdef __FreeBSD__
+#include <sys/types.h>
+#endif
+#ifdef __MINGW32__
+#include <winsock2.h>
+#else
+#include <sys/socket.h>
+#include <netinet/in.h>
+#include <arpa/inet.h>
+#endif
+
+#include <lkl.h>
+#include <lkl_host.h>
+
+#include "cla.h"
+#include "test.h"
+
+enum {
+	BACKEND_NONE,
+};
+
+const char *backends[] = { "loopback", NULL };
+static struct {
+	int backend;
+	const char *ifname;
+	int dhcp, nmlen;
+	unsigned int ip, dst, gateway, sleep;
+} cla = {
+	.backend = BACKEND_NONE,
+	.ip = INADDR_NONE,
+	.gateway = INADDR_NONE,
+	.dst = INADDR_NONE,
+	.sleep = 0,
+};
+
+
+struct cl_arg args[] = {
+	{"backend", 'b', "network backend type", 1, CL_ARG_STR_SET,
+	 &cla.backend, backends},
+	{"ifname", 'i', "interface name", 1, CL_ARG_STR, &cla.ifname},
+	{"dhcp", 'd', "use dhcp to configure LKL", 0, CL_ARG_BOOL, &cla.dhcp},
+	{"ip", 'I', "IPv4 address to use", 1, CL_ARG_IPV4, &cla.ip},
+	{"netmask-len", 'n', "IPv4 netmask length", 1, CL_ARG_INT,
+	 &cla.nmlen},
+	{"gateway", 'g', "IPv4 gateway to use", 1, CL_ARG_IPV4, &cla.gateway},
+	{"dst", 'D', "IPv4 destination address", 1, CL_ARG_IPV4, &cla.dst},
+	{"sleep", 's', "sleep", 1, CL_ARG_INT, &cla.sleep},
+	{0},
+};
+
+u_short
+in_cksum(const u_short *addr, register int len, u_short csum)
+{
+	int nleft = len;
+	const u_short *w = addr;
+	u_short answer;
+	int sum = csum;
+
+	while (nleft > 1)  {
+		sum += *w++;
+		nleft -= 2;
+	}
+
+	if (nleft == 1)
+		sum += htons(*(u_char *)w << 8);
+
+	sum = (sum >> 16) + (sum & 0xffff);
+	sum += (sum >> 16);
+	answer = ~sum;
+	return answer;
+}
+
+static int lkl_test_sleep(void)
+{
+	struct lkl_timespec ts = {
+		.tv_sec = cla.sleep,
+	};
+	int ret;
+
+	ret = lkl_sys_nanosleep((struct __lkl__kernel_timespec *)&ts, NULL);
+	if (ret < 0) {
+		lkl_test_logf("nanosleep error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	return TEST_SUCCESS;
+}
+
+static int lkl_test_icmp(void)
+{
+	int sock, ret;
+	struct lkl_iphdr *iph;
+	struct lkl_icmphdr *icmp;
+	struct lkl_sockaddr_in saddr;
+	struct lkl_pollfd pfd;
+	char buf[32];
+
+	if (cla.dst == INADDR_NONE)
+		return TEST_SKIP;
+
+	memset(&saddr, 0, sizeof(saddr));
+	saddr.sin_family = AF_INET;
+	saddr.sin_addr.lkl_s_addr = cla.dst;
+
+	lkl_test_logf("pinging %s\n",
+		      inet_ntoa(*(struct in_addr *)&saddr.sin_addr));
+
+	sock = lkl_sys_socket(LKL_AF_INET, LKL_SOCK_RAW, LKL_IPPROTO_ICMP);
+	if (sock < 0) {
+		lkl_test_logf("socket error (%s)\n", lkl_strerror(sock));
+		return TEST_FAILURE;
+	}
+
+	icmp = malloc(sizeof(struct lkl_icmphdr));
+	icmp->type = LKL_ICMP_ECHO;
+	icmp->code = 0;
+	icmp->checksum = 0;
+	icmp->un.echo.sequence = 0;
+	icmp->un.echo.id = 0;
+	icmp->checksum = in_cksum((u_short *)icmp, sizeof(*icmp), 0);
+
+	ret = lkl_sys_sendto(sock, icmp, sizeof(*icmp), 0,
+			     (struct lkl_sockaddr *)&saddr,
+			     sizeof(saddr));
+	if (ret < 0) {
+		lkl_test_logf("sendto error (%s)\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	free(icmp);
+
+	pfd.fd = sock;
+	pfd.events = LKL_POLLIN;
+	pfd.revents = 0;
+
+	ret = lkl_sys_poll(&pfd, 1, 1000);
+	if (ret < 0) {
+		lkl_test_logf("poll error (%s)\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_recv(sock, buf, sizeof(buf), LKL_MSG_DONTWAIT);
+	if (ret < 0) {
+		lkl_test_logf("recv error (%s)\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	if (ret < (int)sizeof(struct lkl_iphdr)) {
+		lkl_test_logf("short length of received ip packet (len=%d)\n",
+			      ret);
+		return TEST_FAILURE;
+	}
+	iph = (struct lkl_iphdr *)buf;
+
+	if (ret < (int)((iph->ihl * 4) + sizeof(struct lkl_icmphdr))) {
+		lkl_test_logf("short length of received icmp packet (len=%d)\n",
+			      ret);
+		return TEST_FAILURE;
+	}
+	icmp = (struct lkl_icmphdr *)(buf + iph->ihl * 4);
+
+	/* DHCP server may issue an ICMP echo request to a dhcp client */
+	if ((icmp->type != LKL_ICMP_ECHOREPLY || icmp->code != 0) &&
+	    (icmp->type != LKL_ICMP_ECHO)) {
+		lkl_test_logf("no ICMP echo reply (type=%d, code=%d)\n",
+			      icmp->type, icmp->code);
+		return TEST_FAILURE;
+	}
+
+	return TEST_SUCCESS;
+}
+
+static struct lkl_netdev *nd;
+
+static int lkl_test_nd_create(void)
+{
+	switch (cla.backend) {
+	case BACKEND_NONE:
+		return TEST_SKIP;
+	}
+
+	if (!nd) {
+		lkl_test_logf("failed to create netdev\n");
+		return TEST_BAILOUT;
+	}
+
+	return TEST_SUCCESS;
+}
+
+static int nd_id;
+
+static int lkl_test_nd_add(void)
+{
+	if (cla.backend == BACKEND_NONE)
+		return TEST_SKIP;
+
+	return TEST_SUCCESS;
+}
+
+static int lkl_test_nd_remove(void)
+{
+	if (cla.backend == BACKEND_NONE)
+		return TEST_SKIP;
+
+	return TEST_SUCCESS;
+}
+
+LKL_TEST_CALL(start_kernel, lkl_start_kernel, 0, &lkl_host_ops,
+	"mem=16M loglevel=8 %s", cla.dhcp ? "ip=dhcp" : "");
+LKL_TEST_CALL(stop_kernel, lkl_sys_halt, 0);
+
+static int nd_ifindex;
+
+static int lkl_test_nd_ifindex(void)
+{
+	if (cla.backend == BACKEND_NONE)
+		return TEST_SKIP;
+
+	nd_ifindex = lkl_netdev_get_ifindex(nd_id);
+	if (nd_ifindex < 0) {
+		lkl_test_logf("failed to get ifindex for netdev id %d: %s\n",
+			      nd_id, lkl_strerror(nd_ifindex));
+		return TEST_BAILOUT;
+	}
+
+	return TEST_SUCCESS;
+}
+
+LKL_TEST_CALL(if_up, lkl_if_up, 0,
+	      cla.backend == BACKEND_NONE ? 1 : nd_ifindex);
+
+static int lkl_test_set_ipv4(void)
+{
+	int ret;
+
+	if (cla.backend == BACKEND_NONE || cla.ip == LKL_INADDR_NONE)
+		return TEST_SKIP;
+
+	ret = lkl_if_set_ipv4(nd_ifindex, cla.ip, cla.nmlen);
+	if (ret < 0) {
+		lkl_test_logf("failed to set IPv4 address: %s\n",
+			      lkl_strerror(ret));
+		return TEST_BAILOUT;
+	}
+
+	return TEST_SUCCESS;
+}
+
+static int lkl_test_set_gateway(void)
+{
+	int ret;
+
+	if (cla.backend == BACKEND_NONE || cla.gateway == LKL_INADDR_NONE)
+		return TEST_SKIP;
+
+	ret = lkl_set_ipv4_gateway(cla.gateway);
+	if (ret < 0) {
+		lkl_test_logf("failed to set IPv4 gateway: %s\n",
+			      lkl_strerror(ret));
+		return TEST_BAILOUT;
+	}
+
+	return TEST_SUCCESS;
+}
+
+struct lkl_test tests[] = {
+	LKL_TEST(nd_create),
+	LKL_TEST(nd_add),
+	LKL_TEST(start_kernel),
+	LKL_TEST(nd_ifindex),
+	LKL_TEST(if_up),
+	LKL_TEST(set_ipv4),
+	LKL_TEST(set_gateway),
+	LKL_TEST(sleep),
+	LKL_TEST(icmp),
+	LKL_TEST(nd_remove),
+	LKL_TEST(stop_kernel),
+};
+
+int main(int argc, const char **argv)
+{
+	if (parse_args(argc, argv, args) < 0)
+		return -1;
+
+	if (cla.ip != LKL_INADDR_NONE && (cla.nmlen < 0 || cla.nmlen > 32)) {
+		fprintf(stderr, "invalid netmask length %d\n", cla.nmlen);
+		return -1;
+	}
+
+	lkl_host_ops.print = lkl_test_log;
+
+	return lkl_test_run(tests, sizeof(tests)/sizeof(struct lkl_test),
+			    "net %s", backends[cla.backend]);
+}
diff --git a/tools/lkl/tests/net.sh b/tools/lkl/tests/net.sh
new file mode 100755
index 000000000000..32e8452b0406
--- /dev/null
+++ b/tools/lkl/tests/net.sh
@@ -0,0 +1,90 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+
+source $script_dir/test.sh
+source $script_dir/net-setup.sh
+
+cleanup_backend()
+{
+    set -e
+
+    case "$1" in
+    "loopback")
+        ;;
+    esac
+}
+
+get_test_ip()
+{
+    # DHCP test parameters
+    TEST_HOST=8.8.8.8
+    HOST_IF=$(lkl_test_cmd ip route get $TEST_HOST | head -n1 |cut -d ' ' -f5)
+    HOST_GW=$(lkl_test_cmd ip route get $TEST_HOST | head -n1 | cut -d ' ' -f3)
+    if lkl_test_cmd ping -c1 -w1 $HOST_GW; then
+        TEST_IP_REMOTE=$HOST_GW
+    elif lkl_test_cmd ping -c1 -w1 $TEST_HOST; then
+        TEST_IP_REMOTE=$TEST_HOST
+    else
+        echo "could not find remote test ip"
+        return $TEST_SKIP
+    fi
+
+    export_vars HOST_IF TEST_IP_REMOTE
+}
+
+setup_backend()
+{
+    set -e
+
+    if [ "$LKL_HOST_CONFIG_POSIX" != "y" ] &&
+       [ "$1" != "loopback" ]; then
+        echo "not a posix environment"
+        return $TEST_SKIP
+    fi
+
+    case "$1" in
+    "loopback")
+        ;;
+    *)
+        echo "don't know how to setup backend $1"
+        return $TEST_FAILED
+        ;;
+    esac
+}
+
+run_tests()
+{
+    case "$1" in
+    "loopback")
+        lkl_test_exec $script_dir/net-test --dst 127.0.0.1
+        ;;
+    esac
+}
+
+if [ "$1" = "-b" ]; then
+    shift
+    backend=$1
+    shift
+fi
+
+if [ -z "$backend" ]; then
+    backend="loopback"
+fi
+
+lkl_test_plan 1 "net $backend"
+lkl_test_run 1 setup_backend $backend
+
+if [ $? = $TEST_SKIP ]; then
+    exit 0
+fi
+
+trap "cleanup_backend $backend" EXIT
+
+run_tests $backend
+
+trap : EXIT
+lkl_test_plan 1 "net $backend"
+lkl_test_run 1 cleanup_backend $backend
+
diff --git a/tools/lkl/tests/run.py b/tools/lkl/tests/run.py
new file mode 100755
index 000000000000..b72299aaabee
--- /dev/null
+++ b/tools/lkl/tests/run.py
@@ -0,0 +1,179 @@
+#!/usr/bin/env python
+# SPDX-License-Identifier: GPL-2.0
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 2 of the License
+#
+# Author: Octavian Purdila <tavi@cs.pub.ro>
+#
+
+from __future__ import print_function
+
+import argparse
+import os
+import subprocess
+import sys
+import tap13
+import xml.etree.ElementTree as ET
+
+from junit_xml import TestSuite, TestCase
+
+
+class Reporter(tap13.Reporter):
+    def start(self, obj):
+        if type(obj) is tap13.Test:
+            if obj.result == "*":
+                end='\r'
+            else:
+                end='\n'
+            print("  TEST       %-8s %.50s" %
+                  (obj.result, obj.description + " " + obj.comment), end=end)
+
+        elif type(obj) is tap13.Suite:
+            if obj.tests_planned == 0:
+                status = "skip"
+            else:
+                status = ""
+            print("  SUITE      %-8s %s" % (status, obj.name))
+
+    def end(self, obj):
+        if type(obj) is tap13.Test:
+            if obj.result != "ok":
+                try:
+                    print(obj.yaml["log"], end='')
+                except:
+                    None
+
+
+mydir=os.path.dirname(os.path.realpath(__file__))
+
+tests = [
+    'boot.sh',
+    'disk.sh -t ext4',
+    'disk.sh -t vfat',
+    'disk.sh -t btrfs',
+    'net.sh -b loopback',
+    'lklfuse.sh -t ext4',
+    'lklfuse.sh -t vfat',
+    'lklfuse.sh -t btrfs',
+]
+
+parser = argparse.ArgumentParser(description='LKL test runner')
+parser.add_argument('tests', nargs='?', action='append',
+                    help='tests to run %s' % tests)
+parser.add_argument('--junit-dir',
+                    help='directory where to store the juni suites')
+parser.add_argument('--gdb', action='store_true', default=False,
+                    help='run simple tests under gdb; implies --pass-through')
+parser.add_argument('--pass-through', action='store_true',  default=False,
+                    help='run the test without interpeting the test output')
+parser.add_argument('--valgrind', action='store_true', default=False,
+                    help='run simple tests under valgrind')
+
+args = parser.parse_args()
+if args.tests == [None]:
+    args.tests = tests
+
+if args.gdb:
+    args.pass_through=True
+    os.environ['GDB']="yes"
+
+if args.valgrind:
+    os.environ['VALGRIND']="yes"
+
+tap = tap13.Parser(Reporter())
+
+os.environ['PATH'] += ":" + mydir
+
+exit_code = 0
+
+for t in args.tests:
+    if not t:
+        continue
+    if args.pass_through:
+        print(t)
+        if subprocess.call(t, shell=True) != 0:
+            exit_code = 1
+    else:
+        p = subprocess.Popen(t, shell=True, stdout=subprocess.PIPE)
+        tap.parse(p.stdout)
+
+if args.pass_through:
+    sys.exit(exit_code)
+
+suites_count = 0
+tests_total = 0
+tests_not_ok = 0
+tests_ok = 0
+tests_skip = 0
+val_errs = 0
+val_fails = 0
+val_skips = 0
+
+for s in tap.run.suites:
+
+    junit_tests = []
+    suites_count += 1
+
+    for t in s.tests:
+        try:
+            secs = t.yaml["time_us"] / 1000000.0
+        except:
+            secs = 0
+        try:
+            log = t.yaml['log']
+        except:
+            log = ""
+
+        jt = TestCase(t.description, elapsed_sec=secs, stdout=log)
+        if t.result == 'skip':
+            jt.add_skipped_info(output=log)
+        elif t.result == 'not ok':
+            jt.add_error_info(output=log)
+
+        junit_tests.append(jt)
+
+        tests_total += 1
+        if t.result == "ok":
+            tests_ok += 1
+        elif t.result == "not ok":
+            tests_not_ok += 1
+            exit_code = 1
+        elif t.result == "skip":
+            tests_skip += 1
+
+    if args.junit_dir:
+        js = TestSuite(s.name, junit_tests)
+        with open(os.path.join(args.junit_dir, os.path.basename(s.name) + '.xml'), 'w') as f:
+            js.to_file(f, [js])
+
+        if os.getenv('VALGRIND') is not None:
+            val_xml = 'valgrind-%s.xml' % os.path.basename(s.name).replace(' ','-')
+            # skipped tests don't generate xml file
+            if os.path.exists(val_xml) is False:
+                continue
+
+            cmd = 'mv %s %s' % (val_xml, args.junit_dir)
+            subprocess.call(cmd, shell=True, )
+
+            cmd = mydir + '/valgrind2xunit.py ' + val_xml
+            subprocess.call(cmd, shell=True, cwd=args.junit_dir)
+
+            # count valgrind results
+            doc = ET.parse(os.path.join(args.junit_dir, 'valgrind-%s_xunit.xml' \
+                                        % (os.path.basename(s.name).replace(' ','-'))))
+            ts = doc.getroot()
+            val_errs += int(ts.get('errors'))
+            val_fails += int(ts.get('failures'))
+            val_skips += int(ts.get('skip'))
+
+print("Summary: %d suites run, %d tests, %d ok, %d not ok, %d skipped" %
+      (suites_count, tests_total, tests_ok, tests_not_ok, tests_skip))
+
+if os.getenv('VALGRIND') is not None:
+    print(" valgrind (memcheck): %d failures, %d skipped" % (val_fails, val_skips))
+    if val_errs or val_fails:
+        exit_code = 1
+
+sys.exit(exit_code)
diff --git a/tools/lkl/tests/tap13.py b/tools/lkl/tests/tap13.py
new file mode 100644
index 000000000000..65c73cda7ca1
--- /dev/null
+++ b/tools/lkl/tests/tap13.py
@@ -0,0 +1,209 @@
+#!/usr/bin/env python
+# SPDX-License-Identifier: GPL-2.0
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 2 of the License
+#
+# Author: Octavian Purdila <tavi@cs.pub.ro>
+#
+# Based on TAP13:
+#
+# Copyright 2013, Red Hat, Inc.
+# Author: Josef Skladanka <jskladan@redhat.com>
+#
+from __future__ import print_function
+
+import re
+import sys
+import yamlish
+
+
+class Reporter(object):
+
+    def start(self, obj):
+        None
+
+    def end(self, obj):
+        None
+
+
+class Test(object):
+    def __init__(self, reporter, result, id, description=None, directive=None,
+                 comment=None):
+        self.reporter = reporter
+        self.result = result
+        if directive:
+            self.result = directive.lower()
+        if id:
+            self.id = int(id)
+        else:
+            self.id = None
+        if description:
+            self.description = description
+        else:
+            self.description = ""
+        if comment:
+            self.comment = "# " + comment
+        else:
+            self.comment = ""
+        self.yaml = None
+        self._yaml_buffer = None
+        self.diagnostics = []
+
+        self.reporter.start(self)
+
+    def end(self):
+        if not self.yaml:
+            self.yaml = yamlish.load(self._yaml_buffer)
+            self.reporter.end(self)
+
+
+class Suite(object):
+    def __init__(self, reporter, start, end, explanation):
+        self.reporter = reporter
+        self.tests = []
+        self.name = explanation
+        self.tests_planned = int(end)
+
+        self.__tests_counter = 0
+        self.__tests_base = 0
+
+        self.reporter.start(self)
+
+    def newTest(self, args):
+        try:
+            self.tests[-1].end()
+        except IndexError:
+            None
+
+        if 'id' not in args or not args['id']:
+            args['id'] = self.__tests_counter
+        else:
+            args['id'] = int(args['id']) + self.__tests_base
+
+        if args['id'] < self.__tests_counter:
+            print("error: bad test id %d, fixing it" % (args['id']))
+            args['id'] = self.__tests_counter
+        # according to TAP13 specs, missing tests must be handled as 'not ok'
+        # here we add the missing tests in sequence
+        while args['id'] > (self.__tests_counter + 1):
+            comment = 'test %d not present' % self.__tests_counter
+            self.tests.append(Test(self.reporter, 'not ok',
+                                   self.__tests_counter, comment=comment))
+            self.__tests_counter += 1
+
+        if args['id'] == self.__tests_counter:
+            if args['directive']:
+                self.test().result = args['directive'].lower()
+            else:
+                self.test().result = args['result']
+            self.reporter.start(self.test())
+        else:
+            self.tests.append(Test(self.reporter, **args))
+            self.__tests_counter += 1
+
+    def test(self):
+        return self.tests[-1]
+
+    def end(self, name, planned):
+        if name == self.name:
+            self.tests_planned += int(planned)
+            self.__tests_base = self.__tests_counter
+            return False
+        try:
+            self.test().end()
+        except IndexError:
+            None
+        if len(self.tests) != self.tests_planned:
+            for i in range(len(self.tests), self.tests_planned):
+                self.tests.append(Test(self.reporter, 'not ok', i+1,
+                                       comment='test not present'))
+        return True
+
+
+class Run(object):
+
+    def __init__(self, reporter):
+        self.reporter = reporter
+        self.suites = []
+
+    def suite(self):
+        return self.suites[-1]
+
+    def test(self):
+        return self.suites[-1].tests[-1]
+
+    def newSuite(self, args):
+        new = False
+        try:
+            if self.suite().end(args['explanation'], args['end']):
+                new = True
+        except IndexError:
+            new = True
+        if new:
+            self.suites.append(Suite(self.reporter, **args))
+
+    def newTest(self, args):
+        self.suite().newTest(args)
+
+
+class Parser(object):
+    RE_PLAN = re.compile(r"^\s*(?P<start>\d+)\.\.(?P<end>\d+)\s*(#\s*(?P<explanation>.*))?\s*$")
+    RE_TEST_LINE = re.compile(r"^\s*(?P<result>(not\s+)?ok|[*]+)\s*(?P<id>\d+)?\s*(?P<description>[^#]+)?\s*(#\s*(?P<directive>TODO|SKIP)?\s*(?P<comment>.+)?)?\s*$",  re.IGNORECASE)
+    RE_EXPLANATION = re.compile(r"^\s*#\s*(?P<explanation>.+)?\s*$")
+    RE_YAMLISH_START = re.compile(r"^\s*---.*$")
+    RE_YAMLISH_END = re.compile(r"^\s*\.\.\.\s*$")
+
+    def __init__(self, reporter):
+        self.seek_test = False
+        self.in_test = False
+        self.in_yaml = False
+        self.run = Run(reporter)
+
+    def parse(self, source):
+        # to avoid input buffering
+        while True:
+            line = source.readline()
+            if not line:
+                break
+
+            if self.in_yaml:
+                if Parser.RE_YAMLISH_END.match(line):
+                    self.run.test()._yaml_buffer.append(line.strip())
+                    self.in_yaml = False
+                else:
+                    self.run.test()._yaml_buffer.append(line.rstrip())
+                continue
+
+            line = line.strip()
+
+            if self.in_test:
+                if Parser.RE_EXPLANATION.match(line):
+                    self.run.test().diagnostics.append(line)
+                    continue
+                if Parser.RE_YAMLISH_START.match(line):
+                    self.run.test()._yaml_buffer = [line.strip()]
+                    self.in_yaml = True
+                    continue
+
+            m = Parser.RE_PLAN.match(line)
+            if m:
+                self.seek_test = True
+                args = m.groupdict()
+                self.run.newSuite(args)
+                continue
+
+            if self.seek_test:
+                m = Parser.RE_TEST_LINE.match(line)
+                if m:
+                    args = m.groupdict()
+                    self.run.newTest(args)
+                    self.in_test = True
+                    continue
+
+            print(line)
+        try:
+            self.run.suite().end(None, 0)
+        except IndexError:
+            None
diff --git a/tools/lkl/tests/test.c b/tools/lkl/tests/test.c
new file mode 100644
index 000000000000..3e334d106c48
--- /dev/null
+++ b/tools/lkl/tests/test.c
@@ -0,0 +1,126 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <stdarg.h>
+#include <time.h>
+
+#include "test.h"
+
+/* circular log buffer */
+
+static char log_buf[0x10000];
+static char *head = log_buf, *tail = log_buf;
+
+static inline void advance(char **ptr)
+{
+	if ((unsigned int)(*ptr - log_buf) >= sizeof(log_buf))
+		*ptr = log_buf;
+	else
+		*ptr = *ptr + 1;
+}
+
+static void log_char(char c)
+{
+	*tail = c;
+	advance(&tail);
+	if (tail == head)
+		advance(&head);
+}
+
+static void print_log(void)
+{
+	char last;
+
+	printf(" log: |\n");
+	last = '\n';
+	while (head != tail) {
+		if (last == '\n')
+			printf("  ");
+		last = *head;
+		putchar(last);
+		advance(&head);
+	}
+	if (last != '\n')
+		putchar('\n');
+}
+
+int lkl_test_run(const struct lkl_test *tests, int nr, const char *fmt, ...)
+{
+	int i, ret, status = TEST_SUCCESS;
+	clock_t start, stop;
+	char name[1024];
+	va_list args;
+
+	va_start(args, fmt);
+	vsnprintf(name, sizeof(name), fmt, args);
+	va_end(args);
+
+	printf("1..%d # %s\n", nr, name);
+	for (i = 1; i <= nr; i++) {
+		const struct lkl_test *t = &tests[i-1];
+		unsigned long delta_us;
+
+		printf("* %d %s\n", i, t->name);
+		fflush(stdout);
+
+		start = clock();
+
+		ret = t->fn(t->arg1, t->arg2, t->arg3);
+
+		stop = clock();
+
+		switch (ret) {
+		case TEST_SUCCESS:
+			printf("ok %d %s\n", i, t->name);
+			break;
+		case TEST_SKIP:
+			printf("ok %d %s # SKIP\n", i, t->name);
+			break;
+		case TEST_BAILOUT:
+			status = TEST_BAILOUT;
+			/* fall through */
+		case TEST_FAILURE:
+		default:
+			if (status != TEST_BAILOUT)
+				status = TEST_FAILURE;
+			printf("not ok %d %s\n", i, t->name);
+		}
+
+		printf(" ---\n");
+		delta_us = (stop - start) * 1000000 / CLOCKS_PER_SEC;
+		printf(" time_us: %ld\n", delta_us);
+		print_log();
+		printf(" ...\n");
+
+		if (status == TEST_BAILOUT) {
+			printf("Bail out!\n");
+			return TEST_FAILURE;
+		}
+
+		fflush(stdout);
+	}
+
+	return status;
+}
+
+
+void lkl_test_log(const char *str, int len)
+{
+	while (len--)
+		log_char(*(str++));
+}
+
+int lkl_test_logf(const char *fmt, ...)
+{
+	char tmp[1024], *c;
+	va_list args;
+	unsigned int n;
+
+	va_start(args, fmt);
+	n = vsnprintf(tmp, sizeof(tmp), fmt, args);
+	va_end(args);
+
+	for (c = tmp; *c != 0; c++)
+		log_char(*c);
+
+	return n > sizeof(tmp) ? sizeof(tmp) : n;
+}
diff --git a/tools/lkl/tests/test.h b/tools/lkl/tests/test.h
new file mode 100644
index 000000000000..f63ad6d419cb
--- /dev/null
+++ b/tools/lkl/tests/test.h
@@ -0,0 +1,72 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_TEST_H
+#define _LKL_TEST_H
+
+#define TEST_SUCCESS	0
+#define TEST_FAILURE	1
+#define TEST_SKIP	2
+#define TEST_TODO	3
+#define TEST_BAILOUT	4
+
+struct lkl_test {
+	const char *name;
+	int (*fn)();
+	void *arg1, *arg2, *arg3;
+};
+
+/**
+ * Simple wrapper to initialize a test entry.
+ * @name - test name, it assume test function is named test_@name
+ * @vargs - arguments to be passed to the function
+ */
+#define LKL_TEST(name, ...) { #name, lkl_test_##name, __VA_ARGS__ }
+
+/**
+ * lkl_test_run - run a test suite
+ *
+ * @tests - the list of tests to run
+ * @nr - number of tests
+ * @fmt - format string to be used for suite name
+ */
+int lkl_test_run(const struct lkl_test *tests, int nr, const char *fmt, ...);
+
+/**
+ * lkl_test_log - store a string in the test log buffer
+ * @str - the string to log (can be non-NULL terminated)
+ * @len - the string length
+ */
+void lkl_test_log(const char *str, int len);
+
+/**
+ * lkl_test_logf - printf like function to store into the test log buffer
+ * @fmt - printf format string
+ * @vargs - arguments to the format string
+ */
+int lkl_test_logf(const char *fmt, ...) __attribute__((format(printf, 1, 2)));
+
+/**
+ * LKL_TEST_CALL - create a test function as for a LKL call
+ *
+ * The test function will be named lkl_test_@name and will return
+ * TEST_SUCCESS if the called functions returns @expect. Otherwise
+ * will return TEST_FAILUIRE.
+ *
+ * @name - test name; must be unique because it is part of the the
+ * test function; the test function will be named
+ * @call - function to call
+ * @expect - expected return value for success
+ * @args - arguments to pass to the LKL call
+ */
+#define LKL_TEST_CALL(name, call, expect, ...)				\
+	static int lkl_test_##name(void)				\
+	{								\
+		long ret;						\
+									\
+		ret = call(__VA_ARGS__);				\
+		lkl_test_logf("%s(%s) = %ld %s\n", #call, #__VA_ARGS__, \
+			ret, ret < 0 ? lkl_strerror(ret) : "");		\
+		return (ret == expect) ? TEST_SUCCESS : TEST_FAILURE;	\
+	}
+
+
+#endif /* _LKL_TEST_H */
diff --git a/tools/lkl/tests/test.sh b/tools/lkl/tests/test.sh
new file mode 100644
index 000000000000..f4d652b9fbe8
--- /dev/null
+++ b/tools/lkl/tests/test.sh
@@ -0,0 +1,182 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+basedir=$(cd $script_dir/..; pwd)
+base_objdir=$(cd ${OUTPUT}/; pwd)
+source ${script_dir}/autoconf.sh
+
+TEST_SUCCESS=0
+TEST_FAILURE=1
+TEST_SKIP=113
+TEST_TODO=114
+TEST_BAILOUT=115
+
+print_log()
+{
+    echo " log: |"
+    while read line; do
+        echo "  $line"
+    done < $1
+}
+
+export_vars()
+{
+    if [ -z "$var_file" ]; then
+        return
+    fi
+
+    for i in $@; do
+        echo "$i=${!i}" >> $var_file
+    done
+}
+
+lkl_test_run()
+{
+    log_file=$(mktemp)
+    export var_file=$(mktemp)
+
+    tid=$1 && shift && tname=$@
+
+    echo "* $tid $tname"
+
+    start=$(date '+%s%9N')
+    # run in a separate shell to avoid -e terminating us
+    $@ 2>&1 | strings >$log_file
+    exit=${PIPESTATUS[0]}
+    stop=$(date '+%s%9N')
+
+    case $exit in
+    $TEST_SUCCESS)
+        echo "ok $tid $tname"
+        ;;
+    $TEST_SKIP)
+        echo "ok $tid $tname # SKIP"
+        ;;
+    $TEST_BAILOUT)
+        echo "not ok $tid $tname"
+        echo "Bail out!"
+        ;;
+    $TEST_FAILURE|*)
+        echo "not ok $tid $tname"
+        ;;
+    esac
+
+    delta=$(((stop-start)/1000))
+
+    echo " ---"
+    echo " time_us: $delta"
+    print_log $log_file
+    echo -e " ..."
+
+    rm $log_file
+    . $var_file
+    rm $var_file
+
+    return $exit
+}
+
+lkl_test_plan()
+{
+    echo "1..$1 # $2"
+    export suite_name="${2// /\-}"
+}
+
+lkl_test_exec()
+{
+    local SUDO=""
+    local WRAPPER=""
+
+    if [ "$1" = "sudo" ]; then
+        SUDO=sudo
+        shift
+    fi
+
+    local file=$1
+    shift
+
+    if [ -n "$LKL_HOST_CONFIG_NT" ]; then
+        file=$file.exe
+    fi
+
+    file=${OUTPUT}/tests/$(basename $file)
+
+    if file $file | grep ARM; then
+        WRAPPER="qemu-arm-static"
+    elif file $file | grep "FreeBSD" ; then
+        ssh_copy "$file" $BSD_WDIR
+        if [ -n "$SUDO" ]; then
+            SUDO=""
+        fi
+        WRAPPER="$MYSSH $SU"
+        # ssh will mess up with pipes ('|') so, escape the pipe char.
+        args="${@//\|/\\\|}"
+        set - $BSD_WDIR/$(basename $file) $args
+        file=""
+    elif [ -n "$GDB" ]; then
+        WRAPPER="gdb"
+        args="$@"
+        set - -ex "run $args" -ex quit $file
+        file=""
+    elif [ -n "$VALGRIND" ]; then
+        WRAPPER="valgrind --suppressions=$script_dir/valgrind.supp \
+                  --leak-check=full --show-leak-kinds=all --xml=yes \
+                  --xml-file=valgrind-$suite_name.xml"
+    fi
+
+    $SUDO $WRAPPER $file "$@"
+}
+
+lkl_test_cmd()
+{
+    local WRAPPER=""
+
+    if [ -z "$QUIET" ]; then
+        SHOPTS="-x"
+    fi
+
+    if [ -n "$LKL_HOST_CONFIG_BSD" ]; then
+        WRAPPER="$MYSSH $SU"
+    fi
+
+    echo "$@" | $WRAPPER sh $SHOPTS
+}
+
+# XXX: $MYSSH and $MYSCP are defined in a circleci docker image.
+# see the definitions in lkl/lkl-docker:circleci/freebsd11/Dockerfile
+ssh_push()
+{
+    while [ -n "$1" ]; do
+        if [[ "$1" = *.sh ]]; then
+            type="script"
+        else
+            type="file"
+        fi
+
+        dir=$(dirname $1)
+        $MYSSH mkdir -p $BSD_WDIR/$dir
+
+        $MYSCP -P 7722 -r $basedir/$1 root@localhost:$BSD_WDIR/$dir
+        if [ "$type" = "script" ]; then
+            $MYSSH chmod a+x $BSD_WDIR/$1
+        fi
+
+        shift
+    done
+}
+
+ssh_copy()
+{
+    $MYSCP -P 7722 -r $1 root@localhost:$2
+}
+
+lkl_test_bsd_cleanup()
+{
+    $MYSSH rm -rf $BSD_WDIR
+}
+
+if [ -n "$LKL_HOST_CONFIG_BSD" ]; then
+    trap lkl_test_bsd_cleanup EXIT
+    export BSD_WDIR=/root/lkl
+    $MYSSH mkdir -p $BSD_WDIR
+fi
diff --git a/tools/lkl/tests/valgrind.supp b/tools/lkl/tests/valgrind.supp
new file mode 100644
index 000000000000..2a0e270f2292
--- /dev/null
+++ b/tools/lkl/tests/valgrind.supp
@@ -0,0 +1,105 @@
+{
+   <unfinished timer 1>
+   Memcheck:Leak
+   match-leak-kinds: possible
+   ...
+   fun:pthread_create@@GLIBC_2.2.5
+   fun:__start_helper_thread
+   fun:__pthread_once_slow
+   fun:timer_create@@GLIBC_2.3.3
+   fun:timer_alloc
+   fun:clockevent_set_state_oneshot
+   ...
+   fun:__clockevents_switch_state
+   fun:clockevents_switch_state
+   fun:tick_setup_periodic
+   ...
+}
+
+{
+   <unfinished timer 2>
+   Memcheck:Leak
+   match-leak-kinds: possible
+   ...
+   fun:allocate_stack
+   fun:pthread_create@@GLIBC_2.2.5
+   fun:timer_helper_thread
+   fun:start_thread
+   ...
+}
+
+{
+   <pid1 kernel thread>
+   Memcheck:Leak
+   match-leak-kinds: possible
+   ...
+   fun:thread_create
+   fun:copy_thread
+   fun:copy_thread_tls
+   ...
+   fun:rest_init
+   ...
+   fun:start_kernel
+   ...
+}
+
+{
+   <xfs uninitialized buf error: delete this once upstream is fixed>
+   Memcheck:Value8
+   fun:crc32_body
+   fun:crc32_le_generic
+   fun:__crc32c_le
+   fun:chksum_update
+   fun:crypto_shash_update
+   fun:crc32c
+   fun:xlog_cksum
+}
+
+{
+   <xfs pwrite64 issue: delete this once upstream is fixed>
+   Memcheck:Param
+   pwrite64(buf)
+   ...
+   fun:blk_request
+   fun:blk_enqueue
+   fun:virtio_process_one
+   fun:virtio_process_queue
+   fun:virtio_write
+   fun:__raw_writel
+   fun:writel
+   fun:vm_notify
+   fun:virtqueue_notify
+   fun:virtio_queue_rq
+   fun:blk_mq_dispatch_rq_list
+   fun:blk_mq_sched_dispatch_requests
+}
+
+{
+   <virtio_net_pipe xmits>
+   Memcheck:Param
+   writev(vector[...])
+   ...
+   fun:fd_net_tx
+   fun:net_enqueue
+   fun:virtio_process_one
+   fun:virtio_process_queue
+   fun:virtio_write
+   fun:__raw_writel
+   fun:writel
+   fun:vm_notify
+   ...
+   fun:netdev_start_xmit
+   ...
+}
+
+{
+   <ubd_kern.c>
+   Memcheck:Param
+   read(buf)
+   ...
+   fun:read
+   fun:os_read_file
+   fun:bulk_req_safe_read.constprop.11
+   fun:io_thread
+   ...
+}
diff --git a/tools/lkl/tests/valgrind2xunit.py b/tools/lkl/tests/valgrind2xunit.py
new file mode 100755
index 000000000000..ab7c12b83377
--- /dev/null
+++ b/tools/lkl/tests/valgrind2xunit.py
@@ -0,0 +1,69 @@
+#!/usr/bin/env python
+# SPDX-License-Identifier: GPL-2.0
+
+##
+## Downloader from
+## http://humdi.net/wiki/tips/valgrind-to-xunit-xml-converter
+##
+
+import xml.etree.ElementTree as ET
+import sys
+import os
+
+fname = sys.argv[1]
+if fname is None:
+    fname = 'valgrind.xml'
+
+doc = ET.parse(fname)
+errors = doc.findall('.//error')
+
+out = open(os.path.splitext(os.path.basename(fname))[0]+'_xunit.xml',"w")
+out.write('<?xml version="1.0" encoding="UTF-8"?>\n')
+out.write('<testsuite name="valgrind" tests="'+str(len(errors))+'" errors="0" failures="'+str(len(errors))+'" skip="0">\n')
+errorcount=0
+for error in errors:
+    errorcount=errorcount+1
+
+    kind = error.find('kind')
+    what = error.find('what')
+    if  what == None:
+        what = error.find('xwhat/text')
+
+    stack = error.find('stack')
+    frames = stack.findall('frame')
+
+    for frame in frames:
+        fi = frame.find('file')
+        li = frame.find('line')
+        if fi != None and li != None:
+            break
+
+    if fi != None and li != None:
+        out.write('    <testcase classname="ValgrindMemoryCheck" name="Memory check '+str(errorcount)+' ('+kind.text+', '+fi.text+':'+li.text+')" time="0">\n')
+    else:
+        out.write('    <testcase classname="ValgrindMemoryCheck" name="Memory check '+str(errorcount)+' ('+kind.text+')" time="0">\n')
+    out.write('        <error type="'+kind.text+'">\n')
+    out.write('  '+what.text+'\n\n')
+
+    for frame in frames:
+        ip = frame.find('ip')
+        fn = frame.find('fn')
+        fi = frame.find('file')
+        li = frame.find('line')
+
+        if fn is None:
+            bodytext = '(unresolved symbol)'
+        else:
+            bodytext = fn.text
+        bodytext = bodytext.replace("&","&amp;")
+        bodytext = bodytext.replace("<","&lt;")
+        bodytext = bodytext.replace(">","&gt;")
+        if fi != None and li != None:
+            out.write('  '+ip.text+': '+bodytext+' ('+fi.text+':'+li.text+')\n')
+        else:
+            out.write('  '+ip.text+': '+bodytext+'\n')
+
+    out.write('        </error>\n')
+    out.write('    </testcase>\n')
+out.write('</testsuite>\n')
+out.close()
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 18/25] lkl tools: cptofs that reads/writes to/from a filesystem image
  2020-03-30 14:45   ` Hajime Tazaki
@ 2020-03-30 14:45     ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Conrad Meyer, Dan Peebles, Petros Angelatos, Tuomas Tynkkynen,
	Yuriy Taraday, Hajime Tazaki

From: Octavian Purdila <tavi.purdila@gmail.com>

A tool to read/write to/from a filesystem image without mounting the
file to host filesystem.  Thus there is no root privilege to modify the
contents.

Cc: Conrad Meyer <cem@FreeBSD.org>
Cc: Dan Peebles <pumpkin@me.com>
Cc: Petros Angelatos <petrosagg@gmail.com>
Cc: Tuomas Tynkkynen <tuomas.tynkkynen@iki.fi>
Cc: Yuriy Taraday <yorik.sar@gmail.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/lkl/.gitignore |   2 +
 tools/lkl/Build      |   2 +
 tools/lkl/Makefile   |   3 +
 tools/lkl/Targets    |   4 +
 tools/lkl/cptofs.c   | 636 +++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 647 insertions(+)
 create mode 100644 tools/lkl/cptofs.c

diff --git a/tools/lkl/.gitignore b/tools/lkl/.gitignore
index 4e08254dbd46..1a8210f4d9c4 100644
--- a/tools/lkl/.gitignore
+++ b/tools/lkl/.gitignore
@@ -7,3 +7,5 @@ tests/disk
 tests/boot
 tests/valgrind*.xml
 *.pyc
+cptofs
+cpfromfs
diff --git a/tools/lkl/Build b/tools/lkl/Build
index e69de29bb2d1..a9d12c5ca260 100644
--- a/tools/lkl/Build
+++ b/tools/lkl/Build
@@ -0,0 +1,2 @@
+cptofs-$(LKL_HOST_CONFIG_ARCHIVE) += cptofs.o
+
diff --git a/tools/lkl/Makefile b/tools/lkl/Makefile
index 0abccb9d86b6..03750aab98d8 100644
--- a/tools/lkl/Makefile
+++ b/tools/lkl/Makefile
@@ -86,6 +86,9 @@ $(OUTPUT)%$(EXESUF): $(OUTPUT)%-in.o $(OUTPUT)liblkl.a
 $(OUTPUT)%-in.o: $(OUTPUT)lib/lkl.o FORCE
 	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
 
+$(OUTPUT)cpfromfs$(EXESUF): $(OUTPUT)cptofs$(EXESUF)
+	$(Q)if ! [ -e $@ ]; then ln -s $< $@; fi
+
 
 clean:
 	$(call QUIET_CLEAN, objects)find $(OUTPUT) -name '*.o' -delete -o -name '\.*.cmd'\
diff --git a/tools/lkl/Targets b/tools/lkl/Targets
index a9f74c3cc8fb..17de965f92b3 100644
--- a/tools/lkl/Targets
+++ b/tools/lkl/Targets
@@ -4,3 +4,7 @@ progs-y += tests/boot
 progs-y += tests/disk
 progs-y += tests/net-test
 
+progs-$(LKL_HOST_CONFIG_ARCHIVE) += cptofs
+progs-$(LKL_HOST_CONFIG_ARCHIVE) += cpfromfs
+LDLIBS_cptofs-y := -larchive
+LDLIBS_cptofs-$(LKL_HOST_CONFIG_NEEDS_LARGP) += -largp
diff --git a/tools/lkl/cptofs.c b/tools/lkl/cptofs.c
new file mode 100644
index 000000000000..fb34f570fa1e
--- /dev/null
+++ b/tools/lkl/cptofs.c
@@ -0,0 +1,636 @@
+// SPDX-License-Identifier: GPL-2.0
+#ifdef __FreeBSD__
+#include <sys/param.h>
+#endif
+
+#include <stdio.h>
+#include <time.h>
+#include <argp.h>
+#include <unistd.h>
+#include <string.h>
+#include <stdlib.h>
+#include <libgen.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <fnmatch.h>
+#include <dirent.h>
+#include <lkl.h>
+#include <lkl_host.h>
+
+static const char doc_cptofs[] = "Copy files to a filesystem image";
+static const char doc_cpfromfs[] = "Copy files from a filesystem image";
+static const char args_doc_cptofs[] = "-t fstype -i fsimage path... fs_path";
+static const char args_doc_cpfromfs[] = "-t fstype -i fsimage fs_path... path";
+
+static struct argp_option options[] = {
+	{"enable-printk", 'p', 0, 0, "show Linux printks"},
+	{"partition", 'P', "int", 0, "partition number"},
+	{"filesystem-type", 't', "string", 0,
+	 "select filesystem type - mandatory"},
+	{"filesystem-image", 'i', "string", 0,
+	 "path to the filesystem image - mandatory"},
+	{"selinux", 's', "string", 0, "selinux attributes for destination"},
+	{0},
+};
+
+static struct cl_args {
+	int printk;
+	int part;
+	const char *fsimg_type;
+	const char *fsimg_path;
+	int npaths;
+	char **paths;
+	const char *selinux;
+} cla;
+
+static int cptofs;
+
+static error_t parse_opt(int key, char *arg, struct argp_state *state)
+{
+	struct cl_args *cla = state->input;
+
+	switch (key) {
+	case 'p':
+		cla->printk = 1;
+		break;
+	case 'P':
+		cla->part = atoi(arg);
+		break;
+	case 't':
+		cla->fsimg_type = arg;
+		break;
+	case 'i':
+		cla->fsimg_path = arg;
+		break;
+	case 's':
+		cla->selinux = arg;
+		break;
+	case ARGP_KEY_ARG:
+		// Capture all remaining arguments in our paths array and stop
+		// parsing here. We treat the last one as the destination and
+		// everything before it as sources, just like cp does.
+		cla->paths = &state->argv[state->next - 1];
+		cla->npaths = state->argc - state->next + 1;
+		state->next = state->argc;
+		break;
+	default:
+		return ARGP_ERR_UNKNOWN;
+	}
+
+	return 0;
+}
+
+static struct argp argp_cptofs = {
+	.options = options,
+	.parser = parse_opt,
+	.args_doc = args_doc_cptofs,
+	.doc = doc_cptofs,
+};
+
+static struct argp argp_cpfromfs = {
+	.options = options,
+	.parser = parse_opt,
+	.args_doc = args_doc_cpfromfs,
+	.doc = doc_cpfromfs,
+};
+
+static int searchdir(const char *fs_path, const char *path, const char *match);
+
+static int open_src(const char *path)
+{
+	int fd;
+
+	if (cptofs)
+		fd = open(path, O_RDONLY, 0);
+	else
+		fd = lkl_sys_open(path, LKL_O_RDONLY, 0);
+
+	if (fd < 0)
+		fprintf(stderr, "unable to open file %s for reading: %s\n",
+			path, cptofs ? strerror(errno) : lkl_strerror(fd));
+
+	return fd;
+}
+
+static int open_dst(const char *path, int mode)
+{
+	int fd;
+
+	if (cptofs)
+		fd = lkl_sys_open(path, LKL_O_RDWR | LKL_O_TRUNC | LKL_O_CREAT,
+				  mode);
+	else
+		fd = open(path, O_RDWR | O_TRUNC | O_CREAT, mode);
+
+	if (fd < 0)
+		fprintf(stderr, "unable to open file %s for writing: %s\n",
+			path, cptofs ? lkl_strerror(fd) : strerror(errno));
+
+	if (cla.selinux && cptofs) {
+		int ret = lkl_sys_fsetxattr(fd, "security.selinux", cla.selinux,
+					    strlen(cla.selinux), 0);
+		if (ret)
+			fprintf(stderr,
+				"unable to set selinux attribute on %s: %s\n",
+				path, lkl_strerror(ret));
+	}
+
+	return fd;
+}
+
+static int read_src(int fd, char *buf, int len)
+{
+	int ret;
+
+	if (cptofs)
+		ret = read(fd, buf, len);
+	else
+		ret = lkl_sys_read(fd, buf, len);
+
+	if (ret < 0)
+		fprintf(stderr, "error reading file: %s\n",
+			cptofs ? strerror(errno) : lkl_strerror(ret));
+
+	return ret;
+}
+
+static int write_dst(int fd, char *buf, int len)
+{
+	int ret;
+
+	if (cptofs)
+		ret = lkl_sys_write(fd, buf, len);
+	else
+		ret = write(fd, buf, len);
+
+	if (ret < 0)
+		fprintf(stderr, "error writing file: %s\n",
+			cptofs ? lkl_strerror(ret) : strerror(errno));
+
+	return ret;
+}
+
+static void close_src(int fd)
+{
+	if (cptofs)
+		close(fd);
+	else
+		lkl_sys_close(fd);
+}
+
+static void close_dst(int fd)
+{
+	if (cptofs)
+		lkl_sys_close(fd);
+	else
+		close(fd);
+}
+
+static int copy_file(const char *src, const char *dst, int mode)
+{
+	long len, to_write, wrote;
+	char buf[4096], *ptr;
+	int ret = 0;
+	int fd_src, fd_dst;
+
+	fd_src = open_src(src);
+	if (fd_src < 0)
+		return fd_src;
+
+	fd_dst = open_dst(dst, mode);
+	if (fd_dst < 0)
+		return fd_dst;
+
+	do {
+		len = read_src(fd_src, buf, sizeof(buf));
+
+		if (len > 0) {
+			ptr = buf;
+			to_write = len;
+			do {
+				wrote = write_dst(fd_dst, ptr, to_write);
+
+				if (wrote < 0) {
+					ret = wrote;
+					goto out;
+				}
+
+				to_write -= wrote;
+				ptr += len;
+
+			} while (to_write > 0);
+		}
+
+		if (len < 0)
+			ret = len;
+
+	} while (len > 0);
+
+out:
+	close_src(fd_src);
+	close_dst(fd_dst);
+
+	return ret;
+}
+
+static int stat_src(const char *path, unsigned int *type, unsigned int *mode,
+		    long long *size, struct lkl_timespec *mtime,
+		    struct lkl_timespec *atime)
+{
+	struct stat stat;
+	struct lkl_stat lkl_stat;
+	int ret;
+
+	if (cptofs) {
+		ret = lstat(path, &stat);
+		if (type)
+			*type = stat.st_mode & S_IFMT;
+		if (mode)
+			*mode = stat.st_mode & ~S_IFMT;
+		if (size)
+			*size = stat.st_size;
+		if (mtime) {
+			mtime->tv_sec = stat.st_mtim.tv_sec;
+			mtime->tv_nsec = stat.st_mtim.tv_nsec;
+		}
+		if (atime) {
+			atime->tv_sec = stat.st_atim.tv_sec;
+			atime->tv_nsec = stat.st_atim.tv_nsec;
+		}
+	} else {
+		ret = lkl_sys_lstat(path, &lkl_stat);
+		if (type)
+			*type = lkl_stat.st_mode & S_IFMT;
+		if (mode)
+			*mode = lkl_stat.st_mode & ~S_IFMT;
+		if (size)
+			*size = lkl_stat.st_size;
+		if (mtime) {
+			mtime->tv_sec = lkl_stat.lkl_st_mtime;
+			mtime->tv_nsec = lkl_stat.st_mtime_nsec;
+		}
+		if (atime) {
+			atime->tv_sec = lkl_stat.lkl_st_atime;
+			atime->tv_nsec = lkl_stat.st_atime_nsec;
+		}
+	}
+
+	if (ret)
+		fprintf(stderr, "fsimg lstat(%s) error: %s\n",
+			path, cptofs ? strerror(errno) : lkl_strerror(ret));
+
+	return ret;
+}
+
+static int mkdir_dst(const char *path, unsigned int mode)
+{
+	int ret;
+
+	if (cptofs) {
+		ret = lkl_sys_mkdir(path, mode);
+		if (ret == -LKL_EEXIST)
+			ret = 0;
+	} else {
+		ret = mkdir(path, mode);
+		if (ret < 0 && errno == EEXIST)
+			ret = 0;
+	}
+
+	if (ret)
+		fprintf(stderr, "unable to create directory %s: %s\n",
+			path, cptofs ? strerror(errno) : lkl_strerror(ret));
+
+	return ret;
+}
+
+static int readlink_src(const char *src, char *out, int outsize)
+{
+	int ret;
+
+	if (cptofs)
+		ret = readlink(src, out, outsize);
+	else
+		ret = lkl_sys_readlink(src, out, outsize);
+
+	if (ret < 0)
+		fprintf(stderr, "unable to readlink '%s': %s\n", src,
+			cptofs ? strerror(errno) : lkl_strerror(ret));
+
+	return ret;
+}
+
+static int symlink_dst(const char *path, const char *target)
+{
+	int ret;
+
+	if (cptofs)
+		ret = lkl_sys_symlink(target, path);
+	else
+		ret = symlink(target, path);
+
+	if (ret)
+		fprintf(stderr, "unable to symlink '%s' with target '%s': %s\n",
+			path, target, cptofs ? lkl_strerror(ret) :
+			strerror(errno));
+
+	return ret;
+}
+
+static int copy_symlink(const char *src, const char *dst)
+{
+	int ret;
+	long long size, actual_size;
+	char *target = NULL;
+
+	ret = stat_src(src, NULL, NULL, &size, NULL, NULL);
+	if (ret) {
+		ret = -1;
+		goto out;
+	}
+
+	target = malloc(size + 1);
+	if (!target) {
+		fprintf(stderr, "Unable to allocate memory (%lld bytes)\n",
+			size + 1);
+		ret = -1;
+		goto out;
+	}
+
+	actual_size = readlink_src(src, target, size);
+	if (actual_size != size) {
+		fprintf(stderr,
+			"readlink(%s) bad size: got %lld, expected %lld\n",
+			src, actual_size, size);
+		ret = -1;
+		goto out;
+	}
+	target[size] = 0; // readlink doesn't append the trailing null byte
+
+	ret = symlink_dst(dst, target);
+	if (ret)
+		ret = -1;
+
+out:
+	if (target)
+		free(target);
+
+	return ret;
+}
+
+static int do_entry(const char *_src, const char *_dst, const char *name)
+{
+	char src[PATH_MAX], dst[PATH_MAX];
+	struct lkl_timespec mtime, atime;
+	unsigned int type, mode;
+	int ret;
+
+	snprintf(src, sizeof(src), "%s/%s", _src, name);
+	snprintf(dst, sizeof(dst), "%s/%s", _dst, name);
+
+	ret = stat_src(src, &type, &mode, NULL, &mtime, &atime);
+
+	switch (type) {
+	case S_IFREG:
+	{
+		ret = copy_file(src, dst, mode);
+		break;
+	}
+	case S_IFDIR:
+		ret = mkdir_dst(dst, mode);
+		if (ret)
+			break;
+		ret = searchdir(src, dst, NULL);
+		break;
+	case S_IFLNK:
+		ret = copy_symlink(src, dst);
+		break;
+	case S_IFSOCK:
+	case S_IFBLK:
+	case S_IFCHR:
+	case S_IFIFO:
+	default:
+		printf("skipping %s: unsupported entry type %d\n", src, type);
+	}
+
+	if (!ret) {
+		if (cptofs) {
+			struct lkl_timespec lkl_ts[] = { atime, mtime };
+
+			ret = lkl_sys_utimensat(-1, dst,
+						(struct __lkl__kernel_timespec
+						 *)lkl_ts,
+						LKL_AT_SYMLINK_NOFOLLOW);
+		} else {
+			struct timespec ts[] = {
+				{ .tv_sec = atime.tv_sec,
+				  .tv_nsec = atime.tv_nsec, },
+				{ .tv_sec = mtime.tv_sec,
+				  .tv_nsec = mtime.tv_nsec, },
+			};
+
+			ret = utimensat(-1, dst, ts, AT_SYMLINK_NOFOLLOW);
+		}
+	}
+
+	if (ret)
+		printf("error processing entry %s, aborting\n", src);
+
+	return ret;
+}
+
+static DIR *open_dir(const char *path)
+{
+	DIR *dir;
+	int err;
+
+	if (cptofs)
+		dir = opendir(path);
+	else
+		dir = (DIR *)lkl_opendir(path, &err);
+
+	if (!dir)
+		fprintf(stderr, "unable to open directory %s: %s\n",
+			path, cptofs ? strerror(errno) : lkl_strerror(err));
+	return dir;
+}
+
+static const char *read_dir(DIR *dir, const char *path)
+{
+	struct lkl_dir *lkl_dir = (struct lkl_dir *)dir;
+	const char *name = NULL;
+	const char *err = NULL;
+
+	if (cptofs) {
+		struct dirent *de = readdir(dir);
+
+		if (de)
+			name = de->d_name;
+	} else {
+		struct lkl_linux_dirent64 *de = lkl_readdir(lkl_dir);
+
+		if (de)
+			name = de->d_name;
+	}
+
+	if (!name) {
+		if (cptofs) {
+			if (errno)
+				err = strerror(errno);
+		} else {
+			if (lkl_errdir(lkl_dir))
+				err = lkl_strerror(lkl_errdir(lkl_dir));
+		}
+	}
+
+	if (err)
+		fprintf(stderr, "error while reading directory %s: %s\n",
+			path, err);
+	return name;
+}
+
+static void close_dir(DIR *dir)
+{
+	if (cptofs)
+		closedir(dir);
+	else
+		lkl_closedir((struct lkl_dir *)dir);
+}
+
+static int searchdir(const char *src, const char *dst, const char *match)
+{
+	DIR *dir;
+	const char *name;
+	int ret = 0;
+
+	dir = open_dir(src);
+	if (!dir)
+		return -1;
+
+	while ((name = read_dir(dir, src))) {
+		if (!strcmp(name, ".") || !strcmp(name, "..") ||
+		    (match && fnmatch(match, name, 0) != 0))
+			continue;
+
+		ret = do_entry(src, dst, name);
+		if (ret)
+			goto out;
+	}
+
+out:
+	close_dir(dir);
+
+	return ret;
+}
+
+static int match_root(const char *src)
+{
+	const char *c = src;
+
+	while (*c) {
+		switch (*c) {
+		case '.':
+			if (c > src && c[-1] == '.')
+				return 0;
+			break;
+		case '/':
+			break;
+		default:
+			return 0;
+		}
+		c++;
+	}
+
+	return 1;
+}
+
+int copy_one(const char *src, const char *mpoint, const char *dst)
+{
+	char *src_path_dir, *src_path_base;
+	char src_path[PATH_MAX], dst_path[PATH_MAX];
+
+	if (cptofs) {
+		snprintf(src_path, sizeof(src_path),  "%s", src);
+		snprintf(dst_path, sizeof(dst_path),  "%s/%s", mpoint, dst);
+	} else {
+		snprintf(src_path, sizeof(src_path),  "%s/%s", mpoint, src);
+		snprintf(dst_path, sizeof(dst_path),  "%s", dst);
+	}
+
+	if (match_root(src))
+		return searchdir(src_path, dst, NULL);
+
+	src_path_dir = dirname(strdup(src_path));
+	src_path_base = basename(strdup(src_path));
+
+	return searchdir(src_path_dir, dst_path, src_path_base);
+}
+
+int main(int argc, char **argv)
+{
+	struct lkl_disk disk;
+	long ret, umount_ret;
+	int i;
+	char mpoint[32];
+	unsigned int disk_id;
+
+	if (strstr(argv[0], "cptofs")) {
+		cptofs = 1;
+		ret = argp_parse(&argp_cptofs, argc, argv, 0, 0, &cla);
+	} else {
+		ret = argp_parse(&argp_cpfromfs, argc, argv, 0, 0, &cla);
+	}
+
+	if (ret < 0)
+		return -1;
+
+	if (!cla.printk)
+		lkl_host_ops.print = NULL;
+
+	disk.fd = open(cla.fsimg_path, cptofs ? O_RDWR : O_RDONLY);
+	if (disk.fd < 0) {
+		fprintf(stderr, "can't open fsimg %s: %s\n", cla.fsimg_path,
+			strerror(errno));
+		ret = 1;
+		goto out;
+	}
+
+	disk.ops = NULL;
+	disk.dev = (char *)cla.fsimg_path;
+
+	ret = lkl_disk_add(&disk);
+	if (ret < 0) {
+		fprintf(stderr, "can't add disk: %s\n", lkl_strerror(ret));
+		goto out_close;
+	}
+	disk_id = ret;
+
+	lkl_start_kernel(&lkl_host_ops, "mem=100M");
+
+	ret = lkl_mount_dev(disk_id, cla.part, cla.fsimg_type,
+			    cptofs ? 0 : LKL_MS_RDONLY,
+			    NULL, mpoint, sizeof(mpoint));
+	if (ret) {
+		fprintf(stderr, "can't mount disk: %s\n", lkl_strerror(ret));
+		goto out_close;
+	}
+
+	lkl_sys_umask(0);
+
+	for (i = 0; i < cla.npaths - 1; i++) {
+		ret = copy_one(cla.paths[i], mpoint, cla.paths[cla.npaths - 1]);
+		if (ret)
+			break;
+	}
+
+	umount_ret = lkl_umount_dev(disk_id, cla.part, 0, 1000);
+	if (ret == 0)
+		ret = umount_ret;
+
+out_close:
+	close(disk.fd);
+
+out:
+	lkl_sys_halt();
+
+	return ret;
+}
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 18/25] lkl tools: cptofs that reads/writes to/from a filesystem image
@ 2020-03-30 14:45     ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: linux-arch, Conrad Meyer, Octavian Purdila, Akira Moroo,
	Petros Angelatos, Dan Peebles, linux-kernel-library,
	Yuriy Taraday, Tuomas Tynkkynen, Hajime Tazaki

From: Octavian Purdila <tavi.purdila@gmail.com>

A tool to read/write to/from a filesystem image without mounting the
file to host filesystem.  Thus there is no root privilege to modify the
contents.

Cc: Conrad Meyer <cem@FreeBSD.org>
Cc: Dan Peebles <pumpkin@me.com>
Cc: Petros Angelatos <petrosagg@gmail.com>
Cc: Tuomas Tynkkynen <tuomas.tynkkynen@iki.fi>
Cc: Yuriy Taraday <yorik.sar@gmail.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/lkl/.gitignore |   2 +
 tools/lkl/Build      |   2 +
 tools/lkl/Makefile   |   3 +
 tools/lkl/Targets    |   4 +
 tools/lkl/cptofs.c   | 636 +++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 647 insertions(+)
 create mode 100644 tools/lkl/cptofs.c

diff --git a/tools/lkl/.gitignore b/tools/lkl/.gitignore
index 4e08254dbd46..1a8210f4d9c4 100644
--- a/tools/lkl/.gitignore
+++ b/tools/lkl/.gitignore
@@ -7,3 +7,5 @@ tests/disk
 tests/boot
 tests/valgrind*.xml
 *.pyc
+cptofs
+cpfromfs
diff --git a/tools/lkl/Build b/tools/lkl/Build
index e69de29bb2d1..a9d12c5ca260 100644
--- a/tools/lkl/Build
+++ b/tools/lkl/Build
@@ -0,0 +1,2 @@
+cptofs-$(LKL_HOST_CONFIG_ARCHIVE) += cptofs.o
+
diff --git a/tools/lkl/Makefile b/tools/lkl/Makefile
index 0abccb9d86b6..03750aab98d8 100644
--- a/tools/lkl/Makefile
+++ b/tools/lkl/Makefile
@@ -86,6 +86,9 @@ $(OUTPUT)%$(EXESUF): $(OUTPUT)%-in.o $(OUTPUT)liblkl.a
 $(OUTPUT)%-in.o: $(OUTPUT)lib/lkl.o FORCE
 	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
 
+$(OUTPUT)cpfromfs$(EXESUF): $(OUTPUT)cptofs$(EXESUF)
+	$(Q)if ! [ -e $@ ]; then ln -s $< $@; fi
+
 
 clean:
 	$(call QUIET_CLEAN, objects)find $(OUTPUT) -name '*.o' -delete -o -name '\.*.cmd'\
diff --git a/tools/lkl/Targets b/tools/lkl/Targets
index a9f74c3cc8fb..17de965f92b3 100644
--- a/tools/lkl/Targets
+++ b/tools/lkl/Targets
@@ -4,3 +4,7 @@ progs-y += tests/boot
 progs-y += tests/disk
 progs-y += tests/net-test
 
+progs-$(LKL_HOST_CONFIG_ARCHIVE) += cptofs
+progs-$(LKL_HOST_CONFIG_ARCHIVE) += cpfromfs
+LDLIBS_cptofs-y := -larchive
+LDLIBS_cptofs-$(LKL_HOST_CONFIG_NEEDS_LARGP) += -largp
diff --git a/tools/lkl/cptofs.c b/tools/lkl/cptofs.c
new file mode 100644
index 000000000000..fb34f570fa1e
--- /dev/null
+++ b/tools/lkl/cptofs.c
@@ -0,0 +1,636 @@
+// SPDX-License-Identifier: GPL-2.0
+#ifdef __FreeBSD__
+#include <sys/param.h>
+#endif
+
+#include <stdio.h>
+#include <time.h>
+#include <argp.h>
+#include <unistd.h>
+#include <string.h>
+#include <stdlib.h>
+#include <libgen.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <fnmatch.h>
+#include <dirent.h>
+#include <lkl.h>
+#include <lkl_host.h>
+
+static const char doc_cptofs[] = "Copy files to a filesystem image";
+static const char doc_cpfromfs[] = "Copy files from a filesystem image";
+static const char args_doc_cptofs[] = "-t fstype -i fsimage path... fs_path";
+static const char args_doc_cpfromfs[] = "-t fstype -i fsimage fs_path... path";
+
+static struct argp_option options[] = {
+	{"enable-printk", 'p', 0, 0, "show Linux printks"},
+	{"partition", 'P', "int", 0, "partition number"},
+	{"filesystem-type", 't', "string", 0,
+	 "select filesystem type - mandatory"},
+	{"filesystem-image", 'i', "string", 0,
+	 "path to the filesystem image - mandatory"},
+	{"selinux", 's', "string", 0, "selinux attributes for destination"},
+	{0},
+};
+
+static struct cl_args {
+	int printk;
+	int part;
+	const char *fsimg_type;
+	const char *fsimg_path;
+	int npaths;
+	char **paths;
+	const char *selinux;
+} cla;
+
+static int cptofs;
+
+static error_t parse_opt(int key, char *arg, struct argp_state *state)
+{
+	struct cl_args *cla = state->input;
+
+	switch (key) {
+	case 'p':
+		cla->printk = 1;
+		break;
+	case 'P':
+		cla->part = atoi(arg);
+		break;
+	case 't':
+		cla->fsimg_type = arg;
+		break;
+	case 'i':
+		cla->fsimg_path = arg;
+		break;
+	case 's':
+		cla->selinux = arg;
+		break;
+	case ARGP_KEY_ARG:
+		// Capture all remaining arguments in our paths array and stop
+		// parsing here. We treat the last one as the destination and
+		// everything before it as sources, just like cp does.
+		cla->paths = &state->argv[state->next - 1];
+		cla->npaths = state->argc - state->next + 1;
+		state->next = state->argc;
+		break;
+	default:
+		return ARGP_ERR_UNKNOWN;
+	}
+
+	return 0;
+}
+
+static struct argp argp_cptofs = {
+	.options = options,
+	.parser = parse_opt,
+	.args_doc = args_doc_cptofs,
+	.doc = doc_cptofs,
+};
+
+static struct argp argp_cpfromfs = {
+	.options = options,
+	.parser = parse_opt,
+	.args_doc = args_doc_cpfromfs,
+	.doc = doc_cpfromfs,
+};
+
+static int searchdir(const char *fs_path, const char *path, const char *match);
+
+static int open_src(const char *path)
+{
+	int fd;
+
+	if (cptofs)
+		fd = open(path, O_RDONLY, 0);
+	else
+		fd = lkl_sys_open(path, LKL_O_RDONLY, 0);
+
+	if (fd < 0)
+		fprintf(stderr, "unable to open file %s for reading: %s\n",
+			path, cptofs ? strerror(errno) : lkl_strerror(fd));
+
+	return fd;
+}
+
+static int open_dst(const char *path, int mode)
+{
+	int fd;
+
+	if (cptofs)
+		fd = lkl_sys_open(path, LKL_O_RDWR | LKL_O_TRUNC | LKL_O_CREAT,
+				  mode);
+	else
+		fd = open(path, O_RDWR | O_TRUNC | O_CREAT, mode);
+
+	if (fd < 0)
+		fprintf(stderr, "unable to open file %s for writing: %s\n",
+			path, cptofs ? lkl_strerror(fd) : strerror(errno));
+
+	if (cla.selinux && cptofs) {
+		int ret = lkl_sys_fsetxattr(fd, "security.selinux", cla.selinux,
+					    strlen(cla.selinux), 0);
+		if (ret)
+			fprintf(stderr,
+				"unable to set selinux attribute on %s: %s\n",
+				path, lkl_strerror(ret));
+	}
+
+	return fd;
+}
+
+static int read_src(int fd, char *buf, int len)
+{
+	int ret;
+
+	if (cptofs)
+		ret = read(fd, buf, len);
+	else
+		ret = lkl_sys_read(fd, buf, len);
+
+	if (ret < 0)
+		fprintf(stderr, "error reading file: %s\n",
+			cptofs ? strerror(errno) : lkl_strerror(ret));
+
+	return ret;
+}
+
+static int write_dst(int fd, char *buf, int len)
+{
+	int ret;
+
+	if (cptofs)
+		ret = lkl_sys_write(fd, buf, len);
+	else
+		ret = write(fd, buf, len);
+
+	if (ret < 0)
+		fprintf(stderr, "error writing file: %s\n",
+			cptofs ? lkl_strerror(ret) : strerror(errno));
+
+	return ret;
+}
+
+static void close_src(int fd)
+{
+	if (cptofs)
+		close(fd);
+	else
+		lkl_sys_close(fd);
+}
+
+static void close_dst(int fd)
+{
+	if (cptofs)
+		lkl_sys_close(fd);
+	else
+		close(fd);
+}
+
+static int copy_file(const char *src, const char *dst, int mode)
+{
+	long len, to_write, wrote;
+	char buf[4096], *ptr;
+	int ret = 0;
+	int fd_src, fd_dst;
+
+	fd_src = open_src(src);
+	if (fd_src < 0)
+		return fd_src;
+
+	fd_dst = open_dst(dst, mode);
+	if (fd_dst < 0)
+		return fd_dst;
+
+	do {
+		len = read_src(fd_src, buf, sizeof(buf));
+
+		if (len > 0) {
+			ptr = buf;
+			to_write = len;
+			do {
+				wrote = write_dst(fd_dst, ptr, to_write);
+
+				if (wrote < 0) {
+					ret = wrote;
+					goto out;
+				}
+
+				to_write -= wrote;
+				ptr += len;
+
+			} while (to_write > 0);
+		}
+
+		if (len < 0)
+			ret = len;
+
+	} while (len > 0);
+
+out:
+	close_src(fd_src);
+	close_dst(fd_dst);
+
+	return ret;
+}
+
+static int stat_src(const char *path, unsigned int *type, unsigned int *mode,
+		    long long *size, struct lkl_timespec *mtime,
+		    struct lkl_timespec *atime)
+{
+	struct stat stat;
+	struct lkl_stat lkl_stat;
+	int ret;
+
+	if (cptofs) {
+		ret = lstat(path, &stat);
+		if (type)
+			*type = stat.st_mode & S_IFMT;
+		if (mode)
+			*mode = stat.st_mode & ~S_IFMT;
+		if (size)
+			*size = stat.st_size;
+		if (mtime) {
+			mtime->tv_sec = stat.st_mtim.tv_sec;
+			mtime->tv_nsec = stat.st_mtim.tv_nsec;
+		}
+		if (atime) {
+			atime->tv_sec = stat.st_atim.tv_sec;
+			atime->tv_nsec = stat.st_atim.tv_nsec;
+		}
+	} else {
+		ret = lkl_sys_lstat(path, &lkl_stat);
+		if (type)
+			*type = lkl_stat.st_mode & S_IFMT;
+		if (mode)
+			*mode = lkl_stat.st_mode & ~S_IFMT;
+		if (size)
+			*size = lkl_stat.st_size;
+		if (mtime) {
+			mtime->tv_sec = lkl_stat.lkl_st_mtime;
+			mtime->tv_nsec = lkl_stat.st_mtime_nsec;
+		}
+		if (atime) {
+			atime->tv_sec = lkl_stat.lkl_st_atime;
+			atime->tv_nsec = lkl_stat.st_atime_nsec;
+		}
+	}
+
+	if (ret)
+		fprintf(stderr, "fsimg lstat(%s) error: %s\n",
+			path, cptofs ? strerror(errno) : lkl_strerror(ret));
+
+	return ret;
+}
+
+static int mkdir_dst(const char *path, unsigned int mode)
+{
+	int ret;
+
+	if (cptofs) {
+		ret = lkl_sys_mkdir(path, mode);
+		if (ret == -LKL_EEXIST)
+			ret = 0;
+	} else {
+		ret = mkdir(path, mode);
+		if (ret < 0 && errno == EEXIST)
+			ret = 0;
+	}
+
+	if (ret)
+		fprintf(stderr, "unable to create directory %s: %s\n",
+			path, cptofs ? strerror(errno) : lkl_strerror(ret));
+
+	return ret;
+}
+
+static int readlink_src(const char *src, char *out, int outsize)
+{
+	int ret;
+
+	if (cptofs)
+		ret = readlink(src, out, outsize);
+	else
+		ret = lkl_sys_readlink(src, out, outsize);
+
+	if (ret < 0)
+		fprintf(stderr, "unable to readlink '%s': %s\n", src,
+			cptofs ? strerror(errno) : lkl_strerror(ret));
+
+	return ret;
+}
+
+static int symlink_dst(const char *path, const char *target)
+{
+	int ret;
+
+	if (cptofs)
+		ret = lkl_sys_symlink(target, path);
+	else
+		ret = symlink(target, path);
+
+	if (ret)
+		fprintf(stderr, "unable to symlink '%s' with target '%s': %s\n",
+			path, target, cptofs ? lkl_strerror(ret) :
+			strerror(errno));
+
+	return ret;
+}
+
+static int copy_symlink(const char *src, const char *dst)
+{
+	int ret;
+	long long size, actual_size;
+	char *target = NULL;
+
+	ret = stat_src(src, NULL, NULL, &size, NULL, NULL);
+	if (ret) {
+		ret = -1;
+		goto out;
+	}
+
+	target = malloc(size + 1);
+	if (!target) {
+		fprintf(stderr, "Unable to allocate memory (%lld bytes)\n",
+			size + 1);
+		ret = -1;
+		goto out;
+	}
+
+	actual_size = readlink_src(src, target, size);
+	if (actual_size != size) {
+		fprintf(stderr,
+			"readlink(%s) bad size: got %lld, expected %lld\n",
+			src, actual_size, size);
+		ret = -1;
+		goto out;
+	}
+	target[size] = 0; // readlink doesn't append the trailing null byte
+
+	ret = symlink_dst(dst, target);
+	if (ret)
+		ret = -1;
+
+out:
+	if (target)
+		free(target);
+
+	return ret;
+}
+
+static int do_entry(const char *_src, const char *_dst, const char *name)
+{
+	char src[PATH_MAX], dst[PATH_MAX];
+	struct lkl_timespec mtime, atime;
+	unsigned int type, mode;
+	int ret;
+
+	snprintf(src, sizeof(src), "%s/%s", _src, name);
+	snprintf(dst, sizeof(dst), "%s/%s", _dst, name);
+
+	ret = stat_src(src, &type, &mode, NULL, &mtime, &atime);
+
+	switch (type) {
+	case S_IFREG:
+	{
+		ret = copy_file(src, dst, mode);
+		break;
+	}
+	case S_IFDIR:
+		ret = mkdir_dst(dst, mode);
+		if (ret)
+			break;
+		ret = searchdir(src, dst, NULL);
+		break;
+	case S_IFLNK:
+		ret = copy_symlink(src, dst);
+		break;
+	case S_IFSOCK:
+	case S_IFBLK:
+	case S_IFCHR:
+	case S_IFIFO:
+	default:
+		printf("skipping %s: unsupported entry type %d\n", src, type);
+	}
+
+	if (!ret) {
+		if (cptofs) {
+			struct lkl_timespec lkl_ts[] = { atime, mtime };
+
+			ret = lkl_sys_utimensat(-1, dst,
+						(struct __lkl__kernel_timespec
+						 *)lkl_ts,
+						LKL_AT_SYMLINK_NOFOLLOW);
+		} else {
+			struct timespec ts[] = {
+				{ .tv_sec = atime.tv_sec,
+				  .tv_nsec = atime.tv_nsec, },
+				{ .tv_sec = mtime.tv_sec,
+				  .tv_nsec = mtime.tv_nsec, },
+			};
+
+			ret = utimensat(-1, dst, ts, AT_SYMLINK_NOFOLLOW);
+		}
+	}
+
+	if (ret)
+		printf("error processing entry %s, aborting\n", src);
+
+	return ret;
+}
+
+static DIR *open_dir(const char *path)
+{
+	DIR *dir;
+	int err;
+
+	if (cptofs)
+		dir = opendir(path);
+	else
+		dir = (DIR *)lkl_opendir(path, &err);
+
+	if (!dir)
+		fprintf(stderr, "unable to open directory %s: %s\n",
+			path, cptofs ? strerror(errno) : lkl_strerror(err));
+	return dir;
+}
+
+static const char *read_dir(DIR *dir, const char *path)
+{
+	struct lkl_dir *lkl_dir = (struct lkl_dir *)dir;
+	const char *name = NULL;
+	const char *err = NULL;
+
+	if (cptofs) {
+		struct dirent *de = readdir(dir);
+
+		if (de)
+			name = de->d_name;
+	} else {
+		struct lkl_linux_dirent64 *de = lkl_readdir(lkl_dir);
+
+		if (de)
+			name = de->d_name;
+	}
+
+	if (!name) {
+		if (cptofs) {
+			if (errno)
+				err = strerror(errno);
+		} else {
+			if (lkl_errdir(lkl_dir))
+				err = lkl_strerror(lkl_errdir(lkl_dir));
+		}
+	}
+
+	if (err)
+		fprintf(stderr, "error while reading directory %s: %s\n",
+			path, err);
+	return name;
+}
+
+static void close_dir(DIR *dir)
+{
+	if (cptofs)
+		closedir(dir);
+	else
+		lkl_closedir((struct lkl_dir *)dir);
+}
+
+static int searchdir(const char *src, const char *dst, const char *match)
+{
+	DIR *dir;
+	const char *name;
+	int ret = 0;
+
+	dir = open_dir(src);
+	if (!dir)
+		return -1;
+
+	while ((name = read_dir(dir, src))) {
+		if (!strcmp(name, ".") || !strcmp(name, "..") ||
+		    (match && fnmatch(match, name, 0) != 0))
+			continue;
+
+		ret = do_entry(src, dst, name);
+		if (ret)
+			goto out;
+	}
+
+out:
+	close_dir(dir);
+
+	return ret;
+}
+
+static int match_root(const char *src)
+{
+	const char *c = src;
+
+	while (*c) {
+		switch (*c) {
+		case '.':
+			if (c > src && c[-1] == '.')
+				return 0;
+			break;
+		case '/':
+			break;
+		default:
+			return 0;
+		}
+		c++;
+	}
+
+	return 1;
+}
+
+int copy_one(const char *src, const char *mpoint, const char *dst)
+{
+	char *src_path_dir, *src_path_base;
+	char src_path[PATH_MAX], dst_path[PATH_MAX];
+
+	if (cptofs) {
+		snprintf(src_path, sizeof(src_path),  "%s", src);
+		snprintf(dst_path, sizeof(dst_path),  "%s/%s", mpoint, dst);
+	} else {
+		snprintf(src_path, sizeof(src_path),  "%s/%s", mpoint, src);
+		snprintf(dst_path, sizeof(dst_path),  "%s", dst);
+	}
+
+	if (match_root(src))
+		return searchdir(src_path, dst, NULL);
+
+	src_path_dir = dirname(strdup(src_path));
+	src_path_base = basename(strdup(src_path));
+
+	return searchdir(src_path_dir, dst_path, src_path_base);
+}
+
+int main(int argc, char **argv)
+{
+	struct lkl_disk disk;
+	long ret, umount_ret;
+	int i;
+	char mpoint[32];
+	unsigned int disk_id;
+
+	if (strstr(argv[0], "cptofs")) {
+		cptofs = 1;
+		ret = argp_parse(&argp_cptofs, argc, argv, 0, 0, &cla);
+	} else {
+		ret = argp_parse(&argp_cpfromfs, argc, argv, 0, 0, &cla);
+	}
+
+	if (ret < 0)
+		return -1;
+
+	if (!cla.printk)
+		lkl_host_ops.print = NULL;
+
+	disk.fd = open(cla.fsimg_path, cptofs ? O_RDWR : O_RDONLY);
+	if (disk.fd < 0) {
+		fprintf(stderr, "can't open fsimg %s: %s\n", cla.fsimg_path,
+			strerror(errno));
+		ret = 1;
+		goto out;
+	}
+
+	disk.ops = NULL;
+	disk.dev = (char *)cla.fsimg_path;
+
+	ret = lkl_disk_add(&disk);
+	if (ret < 0) {
+		fprintf(stderr, "can't add disk: %s\n", lkl_strerror(ret));
+		goto out_close;
+	}
+	disk_id = ret;
+
+	lkl_start_kernel(&lkl_host_ops, "mem=100M");
+
+	ret = lkl_mount_dev(disk_id, cla.part, cla.fsimg_type,
+			    cptofs ? 0 : LKL_MS_RDONLY,
+			    NULL, mpoint, sizeof(mpoint));
+	if (ret) {
+		fprintf(stderr, "can't mount disk: %s\n", lkl_strerror(ret));
+		goto out_close;
+	}
+
+	lkl_sys_umask(0);
+
+	for (i = 0; i < cla.npaths - 1; i++) {
+		ret = copy_one(cla.paths[i], mpoint, cla.paths[cla.npaths - 1]);
+		if (ret)
+			break;
+	}
+
+	umount_ret = lkl_umount_dev(disk_id, cla.part, 0, 1000);
+	if (ret == 0)
+		ret = umount_ret;
+
+out_close:
+	close(disk.fd);
+
+out:
+	lkl_sys_halt();
+
+	return ret;
+}
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 19/25] lkl tools: fs2tar that converts a filesystem image to tar
  2020-03-30 14:45   ` Hajime Tazaki
@ 2020-03-30 14:45     ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Conrad Meyer, Petros Angelatos, Hajime Tazaki

From: Octavian Purdila <tavi.purdila@gmail.com>

Simple utility that converts a filesystem image to a tar file,
preserving file rights and extended attributes.

Cc: Conrad Meyer <cem@FreeBSD.org>
Cc: Petros Angelatos <petrosagg@gmail.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/lkl/.gitignore |   1 +
 tools/lkl/Build      |   1 +
 tools/lkl/Targets    |   4 +
 tools/lkl/fs2tar.c   | 412 +++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 418 insertions(+)
 create mode 100644 tools/lkl/fs2tar.c

diff --git a/tools/lkl/.gitignore b/tools/lkl/.gitignore
index 1a8210f4d9c4..138e65efcad2 100644
--- a/tools/lkl/.gitignore
+++ b/tools/lkl/.gitignore
@@ -9,3 +9,4 @@ tests/valgrind*.xml
 *.pyc
 cptofs
 cpfromfs
+fs2tar
diff --git a/tools/lkl/Build b/tools/lkl/Build
index a9d12c5ca260..73b37363a6de 100644
--- a/tools/lkl/Build
+++ b/tools/lkl/Build
@@ -1,2 +1,3 @@
 cptofs-$(LKL_HOST_CONFIG_ARCHIVE) += cptofs.o
+fs2tar-$(LKL_HOST_CONFIG_ARCHIVE) += fs2tar.o
 
diff --git a/tools/lkl/Targets b/tools/lkl/Targets
index 17de965f92b3..3451a1856955 100644
--- a/tools/lkl/Targets
+++ b/tools/lkl/Targets
@@ -8,3 +8,7 @@ progs-$(LKL_HOST_CONFIG_ARCHIVE) += cptofs
 progs-$(LKL_HOST_CONFIG_ARCHIVE) += cpfromfs
 LDLIBS_cptofs-y := -larchive
 LDLIBS_cptofs-$(LKL_HOST_CONFIG_NEEDS_LARGP) += -largp
+
+progs-$(LKL_HOST_CONFIG_ARCHIVE) += fs2tar
+LDLIBS_fs2tar-y := -larchive
+LDLIBS_fs2tar-$(LKL_HOST_CONFIG_NEEDS_LARGP) += -largp
diff --git a/tools/lkl/fs2tar.c b/tools/lkl/fs2tar.c
new file mode 100644
index 000000000000..b7dbcf2028a3
--- /dev/null
+++ b/tools/lkl/fs2tar.c
@@ -0,0 +1,412 @@
+// SPDX-License-Identifier: GPL-2.0
+#ifdef __FreeBSD__
+#include <sys/param.h>
+#endif
+
+#include <stdio.h>
+#include <time.h>
+#include <argp.h>
+#include <unistd.h>
+#include <string.h>
+#include <stdlib.h>
+#include <libgen.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <archive.h>
+#include <archive_entry.h>
+#include <lkl.h>
+#include <lkl_host.h>
+
+char doc[] = "";
+char args_doc[] = "-t fstype fsimage_path tar_path";
+static struct argp_option options[] = {
+	{"enable-printk", 'p', 0, 0, "show Linux printks"},
+	{"partition", 'P', "int", 0, "partition number"},
+	{"filesystem-type", 't', "string", 0,
+	 "select filesystem type - mandatory"},
+	{"selinux-contexts", 's', "file", 0,
+	 "export selinux contexts to file"},
+	{0},
+};
+
+static struct cl_args {
+	int printk;
+	int part;
+	const char *fsimg_type;
+	const char *fsimg_path;
+	const char *tar_path;
+	FILE *selinux;
+} cla;
+
+static error_t parse_opt(int key, char *arg, struct argp_state *state)
+{
+	struct cl_args *cla = state->input;
+
+	switch (key) {
+	case 'p':
+		cla->printk = 1;
+		break;
+	case 'P':
+		cla->part = atoi(arg);
+		break;
+	case 't':
+		cla->fsimg_type = arg;
+		break;
+	case 's':
+		cla->selinux = fopen(arg, "w");
+		if (!cla->selinux) {
+			fprintf(stderr,
+				"failed to open selinux contexts file: %s\n",
+				strerror(errno));
+			return -1;
+		}
+		break;
+	case ARGP_KEY_ARG:
+		if (!cla->fsimg_path)
+			cla->fsimg_path = arg;
+		else if (!cla->tar_path)
+			cla->tar_path = arg;
+		else
+			return -1;
+		break;
+	case ARGP_KEY_END:
+		if (state->arg_num < 2 || !cla->fsimg_type)
+			argp_usage(state);
+		break;
+	default:
+		return ARGP_ERR_UNKNOWN;
+	}
+
+	return 0;
+}
+
+static struct argp argp = { options, parse_opt, args_doc, doc };
+
+static struct archive *tar;
+
+static int searchdir(const char *fsimg_path, const char *path);
+
+static int copy_file(const char *fsimg_path, const char *path)
+{
+	long fsimg_fd;
+	char buff[4096];
+	long len, wrote;
+	int ret = 0;
+
+	fsimg_fd = lkl_sys_open(fsimg_path, LKL_O_RDONLY, 0);
+	if (fsimg_fd < 0) {
+		fprintf(stderr, "fsimg error opening %s: %s\n", fsimg_path,
+			lkl_strerror(fsimg_fd));
+		return fsimg_fd;
+	}
+
+	do {
+		len = lkl_sys_read(fsimg_fd, buff, sizeof(buff));
+		if (len > 0) {
+			wrote = archive_write_data(tar, buff, len);
+			if (wrote != len) {
+				fprintf(stderr,
+					"error writing file %s to archive: %s [%d %ld]\n",
+					path, archive_error_string(tar), ret,
+					len);
+				ret = -archive_errno(tar);
+				break;
+			}
+		}
+
+		if (len < 0) {
+			fprintf(stderr, "error reading fsimg file %s: %s\n",
+				fsimg_path, lkl_strerror(len));
+			ret = len;
+		}
+
+	} while (len > 0);
+
+	lkl_sys_close(fsimg_fd);
+
+	return ret;
+}
+
+static int add_link(const char *fsimg_path, const char *path,
+		    struct archive_entry *entry)
+{
+	char buf[4096] = { 0, };
+	long len;
+
+	len = lkl_sys_readlink(fsimg_path, buf, sizeof(buf));
+	if (len < 0) {
+		fprintf(stderr, "fsimg readlink error %s: %s\n",
+			fsimg_path, lkl_strerror(len));
+		return len;
+	}
+
+	archive_entry_set_symlink(entry, buf);
+
+	return 0;
+}
+
+static inline void fsimg_copy_stat(struct stat *st, struct lkl_stat *fst)
+{
+	st->st_dev = fst->st_dev;
+	st->st_ino = fst->st_ino;
+	st->st_mode = fst->st_mode;
+	st->st_nlink = fst->st_nlink;
+	st->st_uid = fst->st_uid;
+	st->st_gid = fst->st_gid;
+	st->st_rdev = fst->st_rdev;
+	st->st_size = fst->st_size;
+	st->st_blksize = fst->st_blksize;
+	st->st_blocks = fst->st_blocks;
+	st->st_atim.tv_sec = fst->lkl_st_atime;
+	st->st_atim.tv_nsec = fst->st_atime_nsec;
+	st->st_mtim.tv_sec = fst->lkl_st_mtime;
+	st->st_mtim.tv_nsec = fst->st_mtime_nsec;
+	st->st_ctim.tv_sec = fst->lkl_st_ctime;
+	st->st_ctim.tv_nsec = fst->st_ctime_nsec;
+}
+
+static int copy_xattr(const char *fsimg_path, const char *path,
+		      struct archive_entry *entry)
+{
+	long ret;
+	char *xattr_list, *i;
+	long xattr_list_size;
+
+	ret = lkl_sys_llistxattr(fsimg_path, NULL, 0);
+	if (ret < 0) {
+		fprintf(stderr, "fsimg llistxattr(%s) error: %s\n",
+			path, lkl_strerror(ret));
+		return ret;
+	}
+
+	if (!ret)
+		return 0;
+
+	xattr_list = malloc(ret);
+
+	ret = lkl_sys_llistxattr(fsimg_path, xattr_list, ret);
+	if (ret < 0) {
+		fprintf(stderr, "fsimg llistxattr(%s) error: %s\n", path,
+			lkl_strerror(ret));
+		free(xattr_list);
+		return ret;
+	}
+
+	xattr_list_size = ret;
+
+	for (i = xattr_list; i - xattr_list < xattr_list_size;
+	     i += strlen(i) + 1) {
+		void *xattr_buf;
+
+		ret = lkl_sys_lgetxattr(fsimg_path, i, NULL, 0);
+		if (ret < 0) {
+			fprintf(stderr, "fsimg lgetxattr(%s) error: %s\n", path,
+				lkl_strerror(ret));
+			free(xattr_list);
+			return ret;
+		}
+
+		xattr_buf = malloc(ret);
+
+		ret = lkl_sys_lgetxattr(fsimg_path, i, xattr_buf, ret);
+		if (ret < 0) {
+			fprintf(stderr, "fsimg lgetxattr2(%s) error: %s\n",
+				path, lkl_strerror(ret));
+			free(xattr_list);
+			free(xattr_buf);
+			return ret;
+		}
+
+		if (cla.selinux && strcmp(i, "security.selinux") == 0)
+			fprintf(cla.selinux, "%s %s\n", path,
+				(char *)xattr_buf);
+
+		archive_entry_xattr_clear(entry);
+		archive_entry_xattr_add_entry(entry, i, xattr_buf, ret);
+
+		free(xattr_buf);
+	}
+
+	free(xattr_list);
+
+	return 0;
+}
+
+static int do_entry(const char *fsimg_path, const char *path,
+		    const struct lkl_linux_dirent64 *de)
+{
+	char fsimg_new_path[PATH_MAX], new_path[PATH_MAX];
+	struct lkl_stat fsimg_stat;
+	struct stat stat;
+	struct archive_entry *entry;
+	int ftype;
+	long ret;
+
+	snprintf(new_path, sizeof(new_path), "%s/%s", path, de->d_name);
+	snprintf(fsimg_new_path, sizeof(fsimg_new_path), "%s/%s", fsimg_path,
+		 de->d_name);
+
+	ret = lkl_sys_lstat(fsimg_new_path, &fsimg_stat);
+	if (ret) {
+		fprintf(stderr, "fsimg lstat(%s) error: %s\n",
+			path, lkl_strerror(ret));
+		return ret;
+	}
+
+	entry = archive_entry_new();
+
+	archive_entry_set_pathname(entry, new_path);
+	fsimg_copy_stat(&stat, &fsimg_stat);
+	archive_entry_copy_stat(entry, &stat);
+	ret = copy_xattr(fsimg_new_path, new_path, entry);
+	if (ret)
+		return ret;
+	/* TODO: ACLs */
+
+	ftype = stat.st_mode & S_IFMT;
+
+	switch (ftype) {
+	case S_IFREG:
+		archive_write_header(tar, entry);
+		ret = copy_file(fsimg_new_path, new_path);
+		break;
+	case S_IFDIR:
+		archive_write_header(tar, entry);
+		ret = searchdir(fsimg_new_path, new_path);
+		break;
+	case S_IFLNK:
+		ret = add_link(fsimg_new_path, new_path, entry);
+		/* fall through */
+	case S_IFSOCK:
+	case S_IFBLK:
+	case S_IFCHR:
+	case S_IFIFO:
+		if (ret)
+			break;
+		archive_write_header(tar, entry);
+		break;
+	default:
+		printf("skipping %s: unsupported entry type %d\n", new_path,
+		       ftype);
+	}
+
+	archive_entry_free(entry);
+
+	if (ret)
+		printf("error processing entry %s, aborting\n", new_path);
+
+	return ret;
+}
+
+static int searchdir(const char *fsimg_path, const char *path)
+{
+	long ret, fd;
+	char buf[1024], *pos;
+	long buf_len;
+
+	fd = lkl_sys_open(fsimg_path, LKL_O_RDONLY | LKL_O_DIRECTORY, 0);
+	if (fd < 0) {
+		fprintf(stderr, "failed to open dir %s: %s", fsimg_path,
+			lkl_strerror(fd));
+		return fd;
+	}
+
+	do {
+		struct lkl_linux_dirent64 *de;
+
+		de = (struct lkl_linux_dirent64 *) buf;
+		buf_len = lkl_sys_getdents64(fd, de, sizeof(buf));
+		if (buf_len < 0) {
+			fprintf(stderr, "gentdents64 error: %s\n",
+				lkl_strerror(buf_len));
+			break;
+		}
+
+		for (pos = buf; pos - buf < buf_len; pos += de->d_reclen) {
+			de = (struct lkl_linux_dirent64 *)pos;
+
+			if (!strcmp(de->d_name, ".") ||
+			    !strcmp(de->d_name, ".."))
+				continue;
+
+			ret = do_entry(fsimg_path, path, de);
+			if (ret)
+				goto out;
+		}
+
+	} while (buf_len > 0);
+
+out:
+	lkl_sys_close(fd);
+	return ret;
+}
+
+int main(int argc, char **argv)
+{
+	struct lkl_disk disk;
+	long ret;
+	char mpoint[32];
+	unsigned int disk_id;
+
+	if (argp_parse(&argp, argc, argv, 0, 0, &cla) < 0)
+		return -1;
+
+	if (!cla.printk)
+		lkl_host_ops.print = NULL;
+
+	disk.fd = open(cla.fsimg_path, O_RDONLY);
+	if (disk.fd < 0) {
+		fprintf(stderr, "can't open fsimg %s: %s\n", cla.fsimg_path,
+			strerror(errno));
+		ret = 1;
+		goto out;
+	}
+
+	disk.ops = NULL;
+	disk.dev = (char *)cla.fsimg_path;
+
+	ret = lkl_disk_add(&disk);
+	if (ret < 0) {
+		fprintf(stderr, "can't add disk: %s\n", lkl_strerror(ret));
+		goto out_close;
+	}
+	disk_id = ret;
+
+	lkl_start_kernel(&lkl_host_ops, "mem=10M");
+
+	ret = lkl_mount_dev(disk_id, cla.part, cla.fsimg_type, LKL_MS_RDONLY,
+			    NULL, mpoint, sizeof(mpoint));
+	if (ret) {
+		fprintf(stderr, "can't mount disk: %s\n", lkl_strerror(ret));
+		goto out_close;
+	}
+
+	ret = lkl_sys_chdir(mpoint);
+	if (ret) {
+		fprintf(stderr, "can't chdir to %s: %s\n", mpoint,
+			lkl_strerror(ret));
+		goto out_umount;
+	}
+
+	tar = archive_write_new();
+	archive_write_set_format_pax_restricted(tar);
+	archive_write_open_filename(tar, cla.tar_path);
+
+	ret = searchdir(mpoint, "");
+
+	archive_write_free(tar);
+
+	if (cla.selinux)
+		fclose(cla.selinux);
+
+out_umount:
+	lkl_umount_dev(disk_id, cla.part, 0, 1000);
+
+out_close:
+	close(disk.fd);
+
+out:
+	lkl_sys_halt();
+
+	return ret;
+}
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 19/25] lkl tools: fs2tar that converts a filesystem image to tar
@ 2020-03-30 14:45     ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: linux-arch, Conrad Meyer, Octavian Purdila, Akira Moroo,
	Petros Angelatos, linux-kernel-library, Hajime Tazaki

From: Octavian Purdila <tavi.purdila@gmail.com>

Simple utility that converts a filesystem image to a tar file,
preserving file rights and extended attributes.

Cc: Conrad Meyer <cem@FreeBSD.org>
Cc: Petros Angelatos <petrosagg@gmail.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/lkl/.gitignore |   1 +
 tools/lkl/Build      |   1 +
 tools/lkl/Targets    |   4 +
 tools/lkl/fs2tar.c   | 412 +++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 418 insertions(+)
 create mode 100644 tools/lkl/fs2tar.c

diff --git a/tools/lkl/.gitignore b/tools/lkl/.gitignore
index 1a8210f4d9c4..138e65efcad2 100644
--- a/tools/lkl/.gitignore
+++ b/tools/lkl/.gitignore
@@ -9,3 +9,4 @@ tests/valgrind*.xml
 *.pyc
 cptofs
 cpfromfs
+fs2tar
diff --git a/tools/lkl/Build b/tools/lkl/Build
index a9d12c5ca260..73b37363a6de 100644
--- a/tools/lkl/Build
+++ b/tools/lkl/Build
@@ -1,2 +1,3 @@
 cptofs-$(LKL_HOST_CONFIG_ARCHIVE) += cptofs.o
+fs2tar-$(LKL_HOST_CONFIG_ARCHIVE) += fs2tar.o
 
diff --git a/tools/lkl/Targets b/tools/lkl/Targets
index 17de965f92b3..3451a1856955 100644
--- a/tools/lkl/Targets
+++ b/tools/lkl/Targets
@@ -8,3 +8,7 @@ progs-$(LKL_HOST_CONFIG_ARCHIVE) += cptofs
 progs-$(LKL_HOST_CONFIG_ARCHIVE) += cpfromfs
 LDLIBS_cptofs-y := -larchive
 LDLIBS_cptofs-$(LKL_HOST_CONFIG_NEEDS_LARGP) += -largp
+
+progs-$(LKL_HOST_CONFIG_ARCHIVE) += fs2tar
+LDLIBS_fs2tar-y := -larchive
+LDLIBS_fs2tar-$(LKL_HOST_CONFIG_NEEDS_LARGP) += -largp
diff --git a/tools/lkl/fs2tar.c b/tools/lkl/fs2tar.c
new file mode 100644
index 000000000000..b7dbcf2028a3
--- /dev/null
+++ b/tools/lkl/fs2tar.c
@@ -0,0 +1,412 @@
+// SPDX-License-Identifier: GPL-2.0
+#ifdef __FreeBSD__
+#include <sys/param.h>
+#endif
+
+#include <stdio.h>
+#include <time.h>
+#include <argp.h>
+#include <unistd.h>
+#include <string.h>
+#include <stdlib.h>
+#include <libgen.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <archive.h>
+#include <archive_entry.h>
+#include <lkl.h>
+#include <lkl_host.h>
+
+char doc[] = "";
+char args_doc[] = "-t fstype fsimage_path tar_path";
+static struct argp_option options[] = {
+	{"enable-printk", 'p', 0, 0, "show Linux printks"},
+	{"partition", 'P', "int", 0, "partition number"},
+	{"filesystem-type", 't', "string", 0,
+	 "select filesystem type - mandatory"},
+	{"selinux-contexts", 's', "file", 0,
+	 "export selinux contexts to file"},
+	{0},
+};
+
+static struct cl_args {
+	int printk;
+	int part;
+	const char *fsimg_type;
+	const char *fsimg_path;
+	const char *tar_path;
+	FILE *selinux;
+} cla;
+
+static error_t parse_opt(int key, char *arg, struct argp_state *state)
+{
+	struct cl_args *cla = state->input;
+
+	switch (key) {
+	case 'p':
+		cla->printk = 1;
+		break;
+	case 'P':
+		cla->part = atoi(arg);
+		break;
+	case 't':
+		cla->fsimg_type = arg;
+		break;
+	case 's':
+		cla->selinux = fopen(arg, "w");
+		if (!cla->selinux) {
+			fprintf(stderr,
+				"failed to open selinux contexts file: %s\n",
+				strerror(errno));
+			return -1;
+		}
+		break;
+	case ARGP_KEY_ARG:
+		if (!cla->fsimg_path)
+			cla->fsimg_path = arg;
+		else if (!cla->tar_path)
+			cla->tar_path = arg;
+		else
+			return -1;
+		break;
+	case ARGP_KEY_END:
+		if (state->arg_num < 2 || !cla->fsimg_type)
+			argp_usage(state);
+		break;
+	default:
+		return ARGP_ERR_UNKNOWN;
+	}
+
+	return 0;
+}
+
+static struct argp argp = { options, parse_opt, args_doc, doc };
+
+static struct archive *tar;
+
+static int searchdir(const char *fsimg_path, const char *path);
+
+static int copy_file(const char *fsimg_path, const char *path)
+{
+	long fsimg_fd;
+	char buff[4096];
+	long len, wrote;
+	int ret = 0;
+
+	fsimg_fd = lkl_sys_open(fsimg_path, LKL_O_RDONLY, 0);
+	if (fsimg_fd < 0) {
+		fprintf(stderr, "fsimg error opening %s: %s\n", fsimg_path,
+			lkl_strerror(fsimg_fd));
+		return fsimg_fd;
+	}
+
+	do {
+		len = lkl_sys_read(fsimg_fd, buff, sizeof(buff));
+		if (len > 0) {
+			wrote = archive_write_data(tar, buff, len);
+			if (wrote != len) {
+				fprintf(stderr,
+					"error writing file %s to archive: %s [%d %ld]\n",
+					path, archive_error_string(tar), ret,
+					len);
+				ret = -archive_errno(tar);
+				break;
+			}
+		}
+
+		if (len < 0) {
+			fprintf(stderr, "error reading fsimg file %s: %s\n",
+				fsimg_path, lkl_strerror(len));
+			ret = len;
+		}
+
+	} while (len > 0);
+
+	lkl_sys_close(fsimg_fd);
+
+	return ret;
+}
+
+static int add_link(const char *fsimg_path, const char *path,
+		    struct archive_entry *entry)
+{
+	char buf[4096] = { 0, };
+	long len;
+
+	len = lkl_sys_readlink(fsimg_path, buf, sizeof(buf));
+	if (len < 0) {
+		fprintf(stderr, "fsimg readlink error %s: %s\n",
+			fsimg_path, lkl_strerror(len));
+		return len;
+	}
+
+	archive_entry_set_symlink(entry, buf);
+
+	return 0;
+}
+
+static inline void fsimg_copy_stat(struct stat *st, struct lkl_stat *fst)
+{
+	st->st_dev = fst->st_dev;
+	st->st_ino = fst->st_ino;
+	st->st_mode = fst->st_mode;
+	st->st_nlink = fst->st_nlink;
+	st->st_uid = fst->st_uid;
+	st->st_gid = fst->st_gid;
+	st->st_rdev = fst->st_rdev;
+	st->st_size = fst->st_size;
+	st->st_blksize = fst->st_blksize;
+	st->st_blocks = fst->st_blocks;
+	st->st_atim.tv_sec = fst->lkl_st_atime;
+	st->st_atim.tv_nsec = fst->st_atime_nsec;
+	st->st_mtim.tv_sec = fst->lkl_st_mtime;
+	st->st_mtim.tv_nsec = fst->st_mtime_nsec;
+	st->st_ctim.tv_sec = fst->lkl_st_ctime;
+	st->st_ctim.tv_nsec = fst->st_ctime_nsec;
+}
+
+static int copy_xattr(const char *fsimg_path, const char *path,
+		      struct archive_entry *entry)
+{
+	long ret;
+	char *xattr_list, *i;
+	long xattr_list_size;
+
+	ret = lkl_sys_llistxattr(fsimg_path, NULL, 0);
+	if (ret < 0) {
+		fprintf(stderr, "fsimg llistxattr(%s) error: %s\n",
+			path, lkl_strerror(ret));
+		return ret;
+	}
+
+	if (!ret)
+		return 0;
+
+	xattr_list = malloc(ret);
+
+	ret = lkl_sys_llistxattr(fsimg_path, xattr_list, ret);
+	if (ret < 0) {
+		fprintf(stderr, "fsimg llistxattr(%s) error: %s\n", path,
+			lkl_strerror(ret));
+		free(xattr_list);
+		return ret;
+	}
+
+	xattr_list_size = ret;
+
+	for (i = xattr_list; i - xattr_list < xattr_list_size;
+	     i += strlen(i) + 1) {
+		void *xattr_buf;
+
+		ret = lkl_sys_lgetxattr(fsimg_path, i, NULL, 0);
+		if (ret < 0) {
+			fprintf(stderr, "fsimg lgetxattr(%s) error: %s\n", path,
+				lkl_strerror(ret));
+			free(xattr_list);
+			return ret;
+		}
+
+		xattr_buf = malloc(ret);
+
+		ret = lkl_sys_lgetxattr(fsimg_path, i, xattr_buf, ret);
+		if (ret < 0) {
+			fprintf(stderr, "fsimg lgetxattr2(%s) error: %s\n",
+				path, lkl_strerror(ret));
+			free(xattr_list);
+			free(xattr_buf);
+			return ret;
+		}
+
+		if (cla.selinux && strcmp(i, "security.selinux") == 0)
+			fprintf(cla.selinux, "%s %s\n", path,
+				(char *)xattr_buf);
+
+		archive_entry_xattr_clear(entry);
+		archive_entry_xattr_add_entry(entry, i, xattr_buf, ret);
+
+		free(xattr_buf);
+	}
+
+	free(xattr_list);
+
+	return 0;
+}
+
+static int do_entry(const char *fsimg_path, const char *path,
+		    const struct lkl_linux_dirent64 *de)
+{
+	char fsimg_new_path[PATH_MAX], new_path[PATH_MAX];
+	struct lkl_stat fsimg_stat;
+	struct stat stat;
+	struct archive_entry *entry;
+	int ftype;
+	long ret;
+
+	snprintf(new_path, sizeof(new_path), "%s/%s", path, de->d_name);
+	snprintf(fsimg_new_path, sizeof(fsimg_new_path), "%s/%s", fsimg_path,
+		 de->d_name);
+
+	ret = lkl_sys_lstat(fsimg_new_path, &fsimg_stat);
+	if (ret) {
+		fprintf(stderr, "fsimg lstat(%s) error: %s\n",
+			path, lkl_strerror(ret));
+		return ret;
+	}
+
+	entry = archive_entry_new();
+
+	archive_entry_set_pathname(entry, new_path);
+	fsimg_copy_stat(&stat, &fsimg_stat);
+	archive_entry_copy_stat(entry, &stat);
+	ret = copy_xattr(fsimg_new_path, new_path, entry);
+	if (ret)
+		return ret;
+	/* TODO: ACLs */
+
+	ftype = stat.st_mode & S_IFMT;
+
+	switch (ftype) {
+	case S_IFREG:
+		archive_write_header(tar, entry);
+		ret = copy_file(fsimg_new_path, new_path);
+		break;
+	case S_IFDIR:
+		archive_write_header(tar, entry);
+		ret = searchdir(fsimg_new_path, new_path);
+		break;
+	case S_IFLNK:
+		ret = add_link(fsimg_new_path, new_path, entry);
+		/* fall through */
+	case S_IFSOCK:
+	case S_IFBLK:
+	case S_IFCHR:
+	case S_IFIFO:
+		if (ret)
+			break;
+		archive_write_header(tar, entry);
+		break;
+	default:
+		printf("skipping %s: unsupported entry type %d\n", new_path,
+		       ftype);
+	}
+
+	archive_entry_free(entry);
+
+	if (ret)
+		printf("error processing entry %s, aborting\n", new_path);
+
+	return ret;
+}
+
+static int searchdir(const char *fsimg_path, const char *path)
+{
+	long ret, fd;
+	char buf[1024], *pos;
+	long buf_len;
+
+	fd = lkl_sys_open(fsimg_path, LKL_O_RDONLY | LKL_O_DIRECTORY, 0);
+	if (fd < 0) {
+		fprintf(stderr, "failed to open dir %s: %s", fsimg_path,
+			lkl_strerror(fd));
+		return fd;
+	}
+
+	do {
+		struct lkl_linux_dirent64 *de;
+
+		de = (struct lkl_linux_dirent64 *) buf;
+		buf_len = lkl_sys_getdents64(fd, de, sizeof(buf));
+		if (buf_len < 0) {
+			fprintf(stderr, "gentdents64 error: %s\n",
+				lkl_strerror(buf_len));
+			break;
+		}
+
+		for (pos = buf; pos - buf < buf_len; pos += de->d_reclen) {
+			de = (struct lkl_linux_dirent64 *)pos;
+
+			if (!strcmp(de->d_name, ".") ||
+			    !strcmp(de->d_name, ".."))
+				continue;
+
+			ret = do_entry(fsimg_path, path, de);
+			if (ret)
+				goto out;
+		}
+
+	} while (buf_len > 0);
+
+out:
+	lkl_sys_close(fd);
+	return ret;
+}
+
+int main(int argc, char **argv)
+{
+	struct lkl_disk disk;
+	long ret;
+	char mpoint[32];
+	unsigned int disk_id;
+
+	if (argp_parse(&argp, argc, argv, 0, 0, &cla) < 0)
+		return -1;
+
+	if (!cla.printk)
+		lkl_host_ops.print = NULL;
+
+	disk.fd = open(cla.fsimg_path, O_RDONLY);
+	if (disk.fd < 0) {
+		fprintf(stderr, "can't open fsimg %s: %s\n", cla.fsimg_path,
+			strerror(errno));
+		ret = 1;
+		goto out;
+	}
+
+	disk.ops = NULL;
+	disk.dev = (char *)cla.fsimg_path;
+
+	ret = lkl_disk_add(&disk);
+	if (ret < 0) {
+		fprintf(stderr, "can't add disk: %s\n", lkl_strerror(ret));
+		goto out_close;
+	}
+	disk_id = ret;
+
+	lkl_start_kernel(&lkl_host_ops, "mem=10M");
+
+	ret = lkl_mount_dev(disk_id, cla.part, cla.fsimg_type, LKL_MS_RDONLY,
+			    NULL, mpoint, sizeof(mpoint));
+	if (ret) {
+		fprintf(stderr, "can't mount disk: %s\n", lkl_strerror(ret));
+		goto out_close;
+	}
+
+	ret = lkl_sys_chdir(mpoint);
+	if (ret) {
+		fprintf(stderr, "can't chdir to %s: %s\n", mpoint,
+			lkl_strerror(ret));
+		goto out_umount;
+	}
+
+	tar = archive_write_new();
+	archive_write_set_format_pax_restricted(tar);
+	archive_write_open_filename(tar, cla.tar_path);
+
+	ret = searchdir(mpoint, "");
+
+	archive_write_free(tar);
+
+	if (cla.selinux)
+		fclose(cla.selinux);
+
+out_umount:
+	lkl_umount_dev(disk_id, cla.part, 0, 1000);
+
+out_close:
+	close(disk.fd);
+
+out:
+	lkl_sys_halt();
+
+	return ret;
+}
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 20/25] lkl tools: add lklfuse
  2020-03-30 14:45   ` Hajime Tazaki
@ 2020-03-30 14:45     ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Conrad Meyer, Quentin Anglade, Rafael Gieschke, Hajime Tazaki

From: Octavian Purdila <tavi.purdila@gmail.com>

Add a simple fuse based program that can mount filesystem in userspace
using LKL.

Cc: Conrad Meyer <cem@FreeBSD.org>
Cc: Quentin Anglade <quentin.anglade@objectif-libre.com>
Cc: Rafael Gieschke <rafael.gieschke@rz.uni-freiburg.de>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/lkl/.gitignore       |   1 +
 tools/lkl/Build            |   3 +
 tools/lkl/Targets          |   3 +
 tools/lkl/lklfuse.c        | 659 +++++++++++++++++++++++++++++++++++++
 tools/lkl/tests/lklfuse.sh | 110 +++++++
 5 files changed, 776 insertions(+)
 create mode 100644 tools/lkl/lklfuse.c
 create mode 100755 tools/lkl/tests/lklfuse.sh

diff --git a/tools/lkl/.gitignore b/tools/lkl/.gitignore
index 138e65efcad2..c78ec268e4b0 100644
--- a/tools/lkl/.gitignore
+++ b/tools/lkl/.gitignore
@@ -10,3 +10,4 @@ tests/valgrind*.xml
 cptofs
 cpfromfs
 fs2tar
+lklfuse
diff --git a/tools/lkl/Build b/tools/lkl/Build
index 73b37363a6de..6048440d0e1b 100644
--- a/tools/lkl/Build
+++ b/tools/lkl/Build
@@ -1,3 +1,6 @@
+CFLAGS_lklfuse.o += -D_FILE_OFFSET_BITS=64
+
 cptofs-$(LKL_HOST_CONFIG_ARCHIVE) += cptofs.o
 fs2tar-$(LKL_HOST_CONFIG_ARCHIVE) += fs2tar.o
+lklfuse-$(LKL_HOST_CONFIG_FUSE) += lklfuse.o
 
diff --git a/tools/lkl/Targets b/tools/lkl/Targets
index 3451a1856955..03d24362a195 100644
--- a/tools/lkl/Targets
+++ b/tools/lkl/Targets
@@ -12,3 +12,6 @@ LDLIBS_cptofs-$(LKL_HOST_CONFIG_NEEDS_LARGP) += -largp
 progs-$(LKL_HOST_CONFIG_ARCHIVE) += fs2tar
 LDLIBS_fs2tar-y := -larchive
 LDLIBS_fs2tar-$(LKL_HOST_CONFIG_NEEDS_LARGP) += -largp
+
+progs-$(LKL_HOST_CONFIG_FUSE) += lklfuse
+LDLIBS_lklfuse-y := -lfuse
diff --git a/tools/lkl/lklfuse.c b/tools/lkl/lklfuse.c
new file mode 100644
index 000000000000..f6e14778e354
--- /dev/null
+++ b/tools/lkl/lklfuse.c
@@ -0,0 +1,659 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdbool.h>
+#include <stddef.h>
+#include <stdio.h>
+#include <string.h>
+#include <stdlib.h>
+#include <sys/stat.h>
+#include <string.h>
+#include <errno.h>
+#include <unistd.h>
+#define FUSE_USE_VERSION 26
+#include <fuse.h>
+#include <fuse/fuse_opt.h>
+#include <fuse/fuse_lowlevel.h>
+#include <lkl.h>
+#include <lkl_host.h>
+
+#define LKLFUSE_VERSION "0.1"
+
+struct lklfuse {
+	const char *file;
+	const char *log;
+	const char *type;
+	const char *opts;
+	struct lkl_disk disk;
+	int disk_id;
+	int part;
+	int ro;
+	int mb;
+} lklfuse = {
+	.mb = 64,
+};
+
+#define LKLFUSE_OPT(t, p, v) { t, offsetof(struct lklfuse, p), v }
+
+enum {
+	KEY_HELP,
+	KEY_VERSION,
+};
+
+static struct fuse_opt lklfuse_opts[] = {
+	LKLFUSE_OPT("log=%s", log, 0),
+	LKLFUSE_OPT("type=%s", type, 0),
+	LKLFUSE_OPT("mb=%d", mb, 0),
+	LKLFUSE_OPT("opts=%s", opts, 0),
+	LKLFUSE_OPT("part=%d", part, 0),
+	FUSE_OPT_KEY("-h", KEY_HELP),
+	FUSE_OPT_KEY("--help", KEY_HELP),
+	FUSE_OPT_KEY("-V",             KEY_VERSION),
+	FUSE_OPT_KEY("--version",      KEY_VERSION),
+	FUSE_OPT_END
+};
+
+static void usage(void)
+{
+	printf(
+"usage: lklfuse file mountpoint [options]\n"
+"\n"
+"general options:\n"
+"    -o opt,[opt...]        mount options\n"
+"    -h   --help            print help\n"
+"    -V   --version         print version\n"
+"\n"
+"lklfuse options:\n"
+"    -o log=FILE            log file\n"
+"    -o type=fstype         filesystem type\n"
+"    -o mb=memory in mb     ammount of memory to allocate\n"
+"    -o part=parition       partition to mount\n"
+"    -o ro                  open file read-only\n"
+"    -o opts=options        mount options (use \\ to escape , and =)\n"
+);
+}
+
+static int lklfuse_opt_proc(void *data, const char *arg, int key,
+			  struct fuse_args *args)
+{
+	switch (key) {
+	case FUSE_OPT_KEY_OPT:
+		if (strcmp(arg, "ro") == 0)
+			lklfuse.ro = 1;
+		return 1;
+
+	case FUSE_OPT_KEY_NONOPT:
+		if (!lklfuse.file) {
+			lklfuse.file = strdup(arg);
+			return 0;
+		}
+		return 1;
+
+	case KEY_HELP:
+		usage();
+		fuse_opt_add_arg(args, "-ho");
+		fuse_main(args->argc, args->argv, NULL, NULL);
+		exit(1);
+
+	case KEY_VERSION:
+		printf("lklfuse version %s\n", LKLFUSE_VERSION);
+		fuse_opt_add_arg(args, "--version");
+		fuse_main(args->argc, args->argv, NULL, NULL);
+		exit(0);
+
+	default:
+		fprintf(stderr, "internal error\n");
+		abort();
+	}
+}
+
+static void lklfuse_xlat_stat(const struct lkl_stat *in, struct stat *st)
+{
+	st->st_dev = in->st_dev;
+	st->st_ino = in->st_ino;
+	st->st_mode = in->st_mode;
+	st->st_nlink = in->st_nlink;
+	st->st_uid = in->st_uid;
+	st->st_gid = in->st_gid;
+	st->st_rdev = in->st_rdev;
+	st->st_size = in->st_size;
+	st->st_blksize = in->st_blksize;
+	st->st_blocks = in->st_blocks;
+	st->st_atim.tv_sec = in->lkl_st_atime;
+	st->st_atim.tv_nsec = in->st_atime_nsec;
+	st->st_mtim.tv_sec = in->lkl_st_mtime;
+	st->st_mtim.tv_nsec = in->st_mtime_nsec;
+	st->st_ctim.tv_sec = in->lkl_st_ctime;
+	st->st_ctim.tv_nsec = in->st_ctime_nsec;
+}
+
+static int lklfuse_fgetattr(const char *path, struct stat *st,
+			    struct fuse_file_info *fi)
+{
+	long ret;
+	struct lkl_stat lkl_stat;
+
+	ret = lkl_sys_fstat(fi->fh, &lkl_stat);
+	if (ret)
+		return ret;
+
+	lklfuse_xlat_stat(&lkl_stat, st);
+	return 0;
+}
+
+static int lklfuse_getattr(const char *path, struct stat *st)
+{
+	long ret;
+	struct lkl_stat lkl_stat;
+
+	ret = lkl_sys_lstat(path, &lkl_stat);
+	if (ret)
+		return ret;
+
+	lklfuse_xlat_stat(&lkl_stat, st);
+	return 0;
+}
+
+static int lklfuse_readlink(const char *path, char *buf, size_t len)
+{
+	long ret;
+
+	ret = lkl_sys_readlink(path, buf, len);
+	if (ret < 0)
+		return ret;
+
+	if ((size_t)ret == len)
+		ret = len - 1;
+
+	buf[ret] = 0;
+
+	return 0;
+}
+
+static int lklfuse_mknod(const char *path, mode_t mode, dev_t dev)
+{
+	return lkl_sys_mknod(path, mode, dev);
+}
+
+static int lklfuse_mkdir(const char *path, mode_t mode)
+{
+	return lkl_sys_mkdir(path, mode);
+}
+
+static int lklfuse_unlink(const char *path)
+{
+	return lkl_sys_unlink(path);
+}
+
+static int lklfuse_rmdir(const char *path)
+{
+	return lkl_sys_rmdir(path);
+}
+
+static int lklfuse_symlink(const char *oldname, const char *newname)
+{
+	return lkl_sys_symlink(oldname, newname);
+}
+
+
+static int lklfuse_rename(const char *oldname, const char *newname)
+{
+	return lkl_sys_rename(oldname, newname);
+}
+
+static int lklfuse_link(const char *oldname, const char *newname)
+{
+	return lkl_sys_link(oldname, newname);
+}
+
+static int lklfuse_chmod(const char *path, mode_t mode)
+{
+	return lkl_sys_chmod(path, mode);
+}
+
+
+static int lklfuse_chown(const char *path, uid_t uid, gid_t gid)
+{
+	return lkl_sys_fchownat(LKL_AT_FDCWD, path, uid, gid,
+				LKL_AT_SYMLINK_NOFOLLOW);
+}
+
+static int lklfuse_truncate(const char *path, off_t off)
+{
+	return lkl_sys_truncate(path, off);
+}
+
+static int lklfuse_open3(const char *path, bool create, mode_t mode,
+			 struct fuse_file_info *fi)
+{
+	long ret;
+	int flags;
+
+	if ((fi->flags & O_ACCMODE) == O_RDONLY)
+		flags = LKL_O_RDONLY;
+	else if ((fi->flags & O_ACCMODE) == O_WRONLY)
+		flags = LKL_O_WRONLY;
+	else if ((fi->flags & O_ACCMODE) == O_RDWR)
+		flags = LKL_O_RDWR;
+	else
+		return -EINVAL;
+
+	if (create)
+		flags |= LKL_O_CREAT;
+
+	ret = lkl_sys_open(path, flags, mode);
+	if (ret < 0)
+		return ret;
+
+	fi->fh = ret;
+
+	return 0;
+}
+
+static int lklfuse_create(const char *path, mode_t mode,
+			  struct fuse_file_info *fi)
+{
+	return lklfuse_open3(path, true, mode, fi);
+}
+
+static int lklfuse_open(const char *path, struct fuse_file_info *fi)
+{
+	return lklfuse_open3(path, false, 0, fi);
+}
+
+static int lklfuse_read(const char *path, char *buf, size_t size, off_t offset,
+		      struct fuse_file_info *fi)
+{
+	long ret;
+	ssize_t orig_size = size;
+
+	do {
+		ret = lkl_sys_pread64(fi->fh, buf, size, offset);
+		if (ret <= 0)
+			break;
+		size -= ret;
+		offset += ret;
+		buf += ret;
+	} while (size > 0);
+
+	return ret < 0 ? ret : orig_size - (ssize_t)size;
+
+}
+
+static int lklfuse_write(const char *path, const char *buf, size_t size,
+		       off_t offset, struct fuse_file_info *fi)
+{
+	long ret;
+	ssize_t orig_size = size;
+
+	do {
+		ret = lkl_sys_pwrite64(fi->fh, buf, size, offset);
+		if (ret <= 0)
+			break;
+		size -= ret;
+		offset += ret;
+		buf += ret;
+	} while (size > 0);
+
+	return ret < 0 ? ret : orig_size - (ssize_t)size;
+}
+
+
+static int lklfuse_statfs(const char *path, struct statvfs *stat)
+{
+	long ret;
+	struct lkl_statfs lkl_statfs;
+
+	ret = lkl_sys_statfs(path, &lkl_statfs);
+	if (ret < 0)
+		return ret;
+
+	stat->f_bsize = lkl_statfs.f_bsize;
+	stat->f_frsize = lkl_statfs.f_frsize;
+	stat->f_blocks = lkl_statfs.f_blocks;
+	stat->f_bfree = lkl_statfs.f_bfree;
+	stat->f_bavail = lkl_statfs.f_bavail;
+	stat->f_files = lkl_statfs.f_files;
+	stat->f_ffree = lkl_statfs.f_ffree;
+	stat->f_favail = stat->f_ffree;
+	stat->f_fsid = *(unsigned long *)&lkl_statfs.f_fsid.val[0];
+	stat->f_flag = lkl_statfs.f_flags;
+	stat->f_namemax = lkl_statfs.f_namelen;
+
+	return 0;
+}
+
+static int lklfuse_flush(const char *path, struct fuse_file_info *fi)
+{
+	return 0;
+}
+
+static int lklfuse_release(const char *path, struct fuse_file_info *fi)
+{
+	return lkl_sys_close(fi->fh);
+}
+
+static int lklfuse_fsync(const char *path, int datasync,
+		       struct fuse_file_info *fi)
+{
+	if (datasync)
+		return lkl_sys_fdatasync(fi->fh);
+	else
+		return lkl_sys_fsync(fi->fh);
+}
+
+static int lklfuse_setxattr(const char *path, const char *name, const char *val,
+		   size_t size, int flags)
+{
+	return lkl_sys_setxattr(path, name, val, size, flags);
+}
+
+static int lklfuse_getxattr(const char *path, const char *name, char *val,
+			  size_t size)
+{
+	return lkl_sys_getxattr(path, name, val, size);
+}
+
+static int lklfuse_listxattr(const char *path, char *list, size_t size)
+{
+	return lkl_sys_listxattr(path, list, size);
+}
+
+static int lklfuse_removexattr(const char *path, const char *name)
+{
+	return lkl_sys_removexattr(path, name);
+}
+
+static int lklfuse_opendir(const char *path, struct fuse_file_info *fi)
+{
+	struct lkl_dir *dir;
+	int err;
+
+	dir = lkl_opendir(path, &err);
+	if (!dir)
+		return err;
+
+	fi->fh = (uintptr_t)dir;
+
+	return 0;
+}
+
+/** Read directory
+ *
+ * This supersedes the old getdir() interface.  New applications
+ * should use this.
+ *
+ * The filesystem may choose between two modes of operation:
+ *
+ * 1) The readdir implementation ignores the offset parameter, and
+ * passes zero to the filler function's offset.  The filler
+ * function will not return '1' (unless an error happens), so the
+ * whole directory is read in a single readdir operation.  This
+ * works just like the old getdir() method.
+ *
+ * 2) The readdir implementation keeps track of the offsets of the
+ * directory entries.  It uses the offset parameter and always
+ * passes non-zero offset to the filler function.  When the buffer
+ * is full (or an error happens) the filler function will return
+ * '1'.
+ *
+ * Introduced in version 2.3
+ */
+static int lklfuse_readdir(const char *path, void *buf, fuse_fill_dir_t fill,
+			 off_t off, struct fuse_file_info *fi)
+{
+	struct lkl_dir *dir = (struct lkl_dir *)(uintptr_t)fi->fh;
+	struct lkl_linux_dirent64 *de;
+
+	while ((de = lkl_readdir(dir))) {
+		struct stat st = { 0, };
+
+		st.st_ino = de->d_ino;
+		st.st_mode = de->d_type << 12;
+
+		if (fill(buf, de->d_name, &st, 0))
+			break;
+	}
+
+	if (!de)
+		return lkl_errdir(dir);
+
+	return 0;
+}
+
+static int lklfuse_releasedir(const char *path, struct fuse_file_info *fi)
+{
+	struct lkl_dir *dir = (struct lkl_dir *)(uintptr_t)fi->fh;
+
+	return lkl_closedir(dir);
+}
+
+static int lklfuse_fsyncdir(const char *path, int datasync,
+			  struct fuse_file_info *fi)
+{
+	struct lkl_dir *dir = (struct lkl_dir *)(uintptr_t)fi->fh;
+	int fd = lkl_dirfd(dir);
+
+	if (datasync)
+		return lkl_sys_fdatasync(fd);
+	else
+		return lkl_sys_fsync(fd);
+}
+
+static int lklfuse_access(const char *path, int mode)
+{
+	return lkl_sys_access(path, mode);
+}
+
+static int lklfuse_utimens(const char *path, const struct timespec tv[2])
+{
+	struct lkl_timespec ts[2];
+
+	ts[0].tv_sec = tv[0].tv_sec;
+	ts[0].tv_nsec = tv[0].tv_nsec;
+	ts[1].tv_sec = tv[0].tv_sec;
+	ts[1].tv_nsec = tv[0].tv_nsec;
+
+	return lkl_sys_utimensat(-1, path, (struct __lkl__kernel_timespec *)ts,
+				 LKL_AT_SYMLINK_NOFOLLOW);
+}
+
+static int lklfuse_fallocate(const char *path, int mode, off_t offset,
+			     off_t len, struct fuse_file_info *fi)
+{
+	return lkl_sys_fallocate(fi->fh, mode, offset, len);
+}
+
+const struct fuse_operations lklfuse_ops = {
+	.flag_nullpath_ok = 1,
+	.flag_nopath = 1,
+	.flag_utime_omit_ok = 1,
+
+	.getattr = lklfuse_getattr,
+	.readlink = lklfuse_readlink,
+	.mknod = lklfuse_mknod,
+	.mkdir = lklfuse_mkdir,
+	.unlink = lklfuse_unlink,
+	.rmdir = lklfuse_rmdir,
+	.symlink = lklfuse_symlink,
+	.rename = lklfuse_rename,
+	.link = lklfuse_link,
+	.chmod = lklfuse_chmod,
+	.chown = lklfuse_chown,
+	.truncate = lklfuse_truncate,
+	.open = lklfuse_open,
+	.read = lklfuse_read,
+	.write = lklfuse_write,
+	.statfs = lklfuse_statfs,
+	.flush = lklfuse_flush,
+	.release = lklfuse_release,
+	.fsync = lklfuse_fsync,
+	.setxattr = lklfuse_setxattr,
+	.getxattr = lklfuse_getxattr,
+	.listxattr = lklfuse_listxattr,
+	.removexattr = lklfuse_removexattr,
+	.opendir = lklfuse_opendir,
+	.readdir = lklfuse_readdir,
+	.releasedir = lklfuse_releasedir,
+	.fsyncdir = lklfuse_fsyncdir,
+	.access = lklfuse_access,
+	.create = lklfuse_create,
+	.fgetattr = lklfuse_fgetattr,
+	/* .lock, */
+	.utimens = lklfuse_utimens,
+	/* .bmap, */
+	/* .ioctl, */
+	/* .poll, */
+	/* .write_buf, (SG io) */
+	/* .read_buf, (SG io) */
+	/* .flock, */
+	.fallocate = lklfuse_fallocate,
+};
+
+static int start_lkl(void)
+{
+	long ret;
+	char mpoint[32], cmdline[16];
+
+	snprintf(cmdline, sizeof(cmdline), "mem=%dM", lklfuse.mb);
+	ret = lkl_start_kernel(&lkl_host_ops, cmdline);
+	if (ret) {
+		fprintf(stderr, "can't start kernel: %s\n", lkl_strerror(ret));
+		goto out;
+	}
+
+	ret = lkl_mount_dev(lklfuse.disk_id, lklfuse.part, lklfuse.type,
+			    lklfuse.ro ? LKL_MS_RDONLY : 0, lklfuse.opts,
+			    mpoint, sizeof(mpoint));
+
+	if (ret) {
+		fprintf(stderr, "can't mount disk: %s\n", lkl_strerror(ret));
+		goto out_halt;
+	}
+
+	ret = lkl_sys_chroot(mpoint);
+	if (ret) {
+		fprintf(stderr, "can't chdir to %s: %s\n", mpoint,
+			lkl_strerror(ret));
+		goto out_umount;
+	}
+
+	return 0;
+
+out_umount:
+	lkl_umount_dev(lklfuse.disk_id, lklfuse.part, 0, 1000);
+
+out_halt:
+	lkl_sys_halt();
+
+out:
+	return ret;
+}
+
+static void stop_lkl(void)
+{
+	int ret;
+
+	ret = lkl_sys_chdir("/");
+	if (ret)
+		fprintf(stderr, "can't chdir to /: %s\n", lkl_strerror(ret));
+	ret = lkl_sys_umount("/", 0);
+	if (ret)
+		fprintf(stderr, "failed to umount disk: %d: %s\n",
+			lklfuse.disk_id, lkl_strerror(ret));
+	lkl_sys_halt();
+}
+
+int main(int argc, char **argv)
+{
+	struct fuse_args args = FUSE_ARGS_INIT(argc, argv);
+	struct fuse_chan *ch;
+	struct fuse *fuse;
+	struct stat st;
+	char *mnt;
+	int fg, mt, ret;
+
+	if (fuse_opt_parse(&args, &lklfuse, lklfuse_opts, lklfuse_opt_proc))
+		return 1;
+
+	if (!lklfuse.file || !lklfuse.type) {
+		fprintf(stderr, "no file or filesystem type specified\n");
+		return 1;
+	}
+
+	if (fuse_parse_cmdline(&args, &mnt, &mt, &fg))
+		return 1;
+
+	ret = stat(mnt, &st);
+	if (ret) {
+		perror(mnt);
+		goto out_free;
+	}
+
+	ret = open(lklfuse.file, lklfuse.ro ? O_RDONLY : O_RDWR);
+	if (ret < 0) {
+		perror(lklfuse.file);
+		goto out_free;
+	}
+
+	lklfuse.disk.fd = ret;
+	lklfuse.disk.dev = (char *)lklfuse.file;
+
+	ret = lkl_disk_add(&lklfuse.disk);
+	if (ret < 0) {
+		fprintf(stderr, "can't add disk: %s\n", lkl_strerror(ret));
+		goto out_close_disk;
+	}
+
+	lklfuse.disk_id = ret;
+
+	ch = fuse_mount(mnt, &args);
+	if (!ch) {
+		ret = -1;
+		goto out_close_disk;
+	}
+
+	fuse = fuse_new(ch, &args, &lklfuse_ops, sizeof(lklfuse_ops), NULL);
+	if (!fuse) {
+		ret = -1;
+		goto out_fuse_unmount;
+	}
+
+	fuse_opt_free_args(&args);
+
+	if (fuse_daemonize(fg) ||
+	    fuse_set_signal_handlers(fuse_get_session(fuse))) {
+		ret = -1;
+		goto out_fuse_destroy;
+	}
+
+	ret = start_lkl();
+	if (ret) {
+		ret = -1;
+		goto out_remove_signals;
+	}
+
+	if (mt)
+		fprintf(stderr, "warning: multithreaded mode not supported\n");
+
+	ret = fuse_loop(fuse);
+
+	stop_lkl();
+
+out_remove_signals:
+	fuse_remove_signal_handlers(fuse_get_session(fuse));
+
+out_fuse_unmount:
+	if (ch)
+		fuse_unmount(mnt, ch);
+
+out_fuse_destroy:
+	if (fuse)
+		fuse_destroy(fuse);
+
+out_close_disk:
+	close(lklfuse.disk.fd);
+
+out_free:
+	free(mnt);
+
+	return ret < 0 ? 1 : 0;
+}
diff --git a/tools/lkl/tests/lklfuse.sh b/tools/lkl/tests/lklfuse.sh
new file mode 100755
index 000000000000..7f35dd53fc4e
--- /dev/null
+++ b/tools/lkl/tests/lklfuse.sh
@@ -0,0 +1,110 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+
+source $script_dir/test.sh
+
+cleanup()
+{
+    set -e
+
+    sleep 1
+    fusermount -u $dir
+    rm $file
+    rmdir $dir
+}
+
+
+# $1 - disk image
+# $2 - fstype
+function prepfs()
+{
+    set -e
+
+    dd if=/dev/zero of=$1 bs=1024 count=102400
+
+    yes | mkfs.$2 $1
+}
+
+# $1 - disk image
+# $2 - mount point
+# $3 - filesystem type
+lklfuse_mount()
+{
+    ${script_dir}/../lklfuse $1 $2 -o type=$3
+}
+
+# $1 - mount point
+lklfuse_basic()
+{
+    set -e
+
+    cd $1
+    touch a
+    if ! [ -e ]; then exit 1; fi
+    rm a
+    mkdir a
+    if ! [ -d ]; then exit 1; fi
+    rmdir a
+}
+
+# $1 - dir
+# $2 - filesystem type
+lklfuse_stressng()
+{
+    set -e
+
+    if [ -z $(which stress-ng) ]; then
+        echo "missing stress-ng"
+        return $TEST_SKIP
+    fi
+
+    cd $1
+
+    if [ "$2" = "vfat" ]; then
+        exclude="chmod,filename,link,mknod,symlink,xattr"
+    fi
+
+    stress-ng --class filesystem --all 0 --timeout 10 \
+	      --exclude fiemap,$exclude --fallocate-bytes 10m \
+	      --sync-file-bytes 10m
+}
+
+if [ "$1" = "-t" ]; then
+    shift
+    fstype=$1
+    shift
+fi
+
+if [ -z "$fstype" ]; then
+    fstype="ext4"
+fi
+
+if ! [ -x $script_dir/../lklfuse ]; then
+    lkl_test_plan 0 "lklfuse.sh $fstype"
+    echo "lklfuse not available"
+    exit 0
+fi
+
+if ! [ -e /dev/fuse ]; then
+    lkl_test_plan 0 "lklfuse.sh $fstype"
+    echo "/dev/fuse not available"
+    exit 0
+fi
+
+
+file=`mktemp`
+dir=`mktemp -d`
+
+trap cleanup EXIT
+
+lkl_test_plan 4 "lklfuse $fstype"
+
+lkl_test_run 1 prepfs $file $fstype
+lkl_test_run 2 lklfuse_mount $file $dir $fstype
+lkl_test_run 3 lklfuse_basic $dir
+# stress-ng returns 2 with no apparent failures so skip it for now
+#lkl_test_run 4 lklfuse_stressng $dir $fstype
+trap : EXIT
+lkl_test_run 4 cleanup
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 20/25] lkl tools: add lklfuse
@ 2020-03-30 14:45     ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: linux-arch, Rafael Gieschke, Conrad Meyer, Octavian Purdila,
	Akira Moroo, Quentin Anglade, linux-kernel-library,
	Hajime Tazaki

From: Octavian Purdila <tavi.purdila@gmail.com>

Add a simple fuse based program that can mount filesystem in userspace
using LKL.

Cc: Conrad Meyer <cem@FreeBSD.org>
Cc: Quentin Anglade <quentin.anglade@objectif-libre.com>
Cc: Rafael Gieschke <rafael.gieschke@rz.uni-freiburg.de>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/lkl/.gitignore       |   1 +
 tools/lkl/Build            |   3 +
 tools/lkl/Targets          |   3 +
 tools/lkl/lklfuse.c        | 659 +++++++++++++++++++++++++++++++++++++
 tools/lkl/tests/lklfuse.sh | 110 +++++++
 5 files changed, 776 insertions(+)
 create mode 100644 tools/lkl/lklfuse.c
 create mode 100755 tools/lkl/tests/lklfuse.sh

diff --git a/tools/lkl/.gitignore b/tools/lkl/.gitignore
index 138e65efcad2..c78ec268e4b0 100644
--- a/tools/lkl/.gitignore
+++ b/tools/lkl/.gitignore
@@ -10,3 +10,4 @@ tests/valgrind*.xml
 cptofs
 cpfromfs
 fs2tar
+lklfuse
diff --git a/tools/lkl/Build b/tools/lkl/Build
index 73b37363a6de..6048440d0e1b 100644
--- a/tools/lkl/Build
+++ b/tools/lkl/Build
@@ -1,3 +1,6 @@
+CFLAGS_lklfuse.o += -D_FILE_OFFSET_BITS=64
+
 cptofs-$(LKL_HOST_CONFIG_ARCHIVE) += cptofs.o
 fs2tar-$(LKL_HOST_CONFIG_ARCHIVE) += fs2tar.o
+lklfuse-$(LKL_HOST_CONFIG_FUSE) += lklfuse.o
 
diff --git a/tools/lkl/Targets b/tools/lkl/Targets
index 3451a1856955..03d24362a195 100644
--- a/tools/lkl/Targets
+++ b/tools/lkl/Targets
@@ -12,3 +12,6 @@ LDLIBS_cptofs-$(LKL_HOST_CONFIG_NEEDS_LARGP) += -largp
 progs-$(LKL_HOST_CONFIG_ARCHIVE) += fs2tar
 LDLIBS_fs2tar-y := -larchive
 LDLIBS_fs2tar-$(LKL_HOST_CONFIG_NEEDS_LARGP) += -largp
+
+progs-$(LKL_HOST_CONFIG_FUSE) += lklfuse
+LDLIBS_lklfuse-y := -lfuse
diff --git a/tools/lkl/lklfuse.c b/tools/lkl/lklfuse.c
new file mode 100644
index 000000000000..f6e14778e354
--- /dev/null
+++ b/tools/lkl/lklfuse.c
@@ -0,0 +1,659 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdbool.h>
+#include <stddef.h>
+#include <stdio.h>
+#include <string.h>
+#include <stdlib.h>
+#include <sys/stat.h>
+#include <string.h>
+#include <errno.h>
+#include <unistd.h>
+#define FUSE_USE_VERSION 26
+#include <fuse.h>
+#include <fuse/fuse_opt.h>
+#include <fuse/fuse_lowlevel.h>
+#include <lkl.h>
+#include <lkl_host.h>
+
+#define LKLFUSE_VERSION "0.1"
+
+struct lklfuse {
+	const char *file;
+	const char *log;
+	const char *type;
+	const char *opts;
+	struct lkl_disk disk;
+	int disk_id;
+	int part;
+	int ro;
+	int mb;
+} lklfuse = {
+	.mb = 64,
+};
+
+#define LKLFUSE_OPT(t, p, v) { t, offsetof(struct lklfuse, p), v }
+
+enum {
+	KEY_HELP,
+	KEY_VERSION,
+};
+
+static struct fuse_opt lklfuse_opts[] = {
+	LKLFUSE_OPT("log=%s", log, 0),
+	LKLFUSE_OPT("type=%s", type, 0),
+	LKLFUSE_OPT("mb=%d", mb, 0),
+	LKLFUSE_OPT("opts=%s", opts, 0),
+	LKLFUSE_OPT("part=%d", part, 0),
+	FUSE_OPT_KEY("-h", KEY_HELP),
+	FUSE_OPT_KEY("--help", KEY_HELP),
+	FUSE_OPT_KEY("-V",             KEY_VERSION),
+	FUSE_OPT_KEY("--version",      KEY_VERSION),
+	FUSE_OPT_END
+};
+
+static void usage(void)
+{
+	printf(
+"usage: lklfuse file mountpoint [options]\n"
+"\n"
+"general options:\n"
+"    -o opt,[opt...]        mount options\n"
+"    -h   --help            print help\n"
+"    -V   --version         print version\n"
+"\n"
+"lklfuse options:\n"
+"    -o log=FILE            log file\n"
+"    -o type=fstype         filesystem type\n"
+"    -o mb=memory in mb     ammount of memory to allocate\n"
+"    -o part=parition       partition to mount\n"
+"    -o ro                  open file read-only\n"
+"    -o opts=options        mount options (use \\ to escape , and =)\n"
+);
+}
+
+static int lklfuse_opt_proc(void *data, const char *arg, int key,
+			  struct fuse_args *args)
+{
+	switch (key) {
+	case FUSE_OPT_KEY_OPT:
+		if (strcmp(arg, "ro") == 0)
+			lklfuse.ro = 1;
+		return 1;
+
+	case FUSE_OPT_KEY_NONOPT:
+		if (!lklfuse.file) {
+			lklfuse.file = strdup(arg);
+			return 0;
+		}
+		return 1;
+
+	case KEY_HELP:
+		usage();
+		fuse_opt_add_arg(args, "-ho");
+		fuse_main(args->argc, args->argv, NULL, NULL);
+		exit(1);
+
+	case KEY_VERSION:
+		printf("lklfuse version %s\n", LKLFUSE_VERSION);
+		fuse_opt_add_arg(args, "--version");
+		fuse_main(args->argc, args->argv, NULL, NULL);
+		exit(0);
+
+	default:
+		fprintf(stderr, "internal error\n");
+		abort();
+	}
+}
+
+static void lklfuse_xlat_stat(const struct lkl_stat *in, struct stat *st)
+{
+	st->st_dev = in->st_dev;
+	st->st_ino = in->st_ino;
+	st->st_mode = in->st_mode;
+	st->st_nlink = in->st_nlink;
+	st->st_uid = in->st_uid;
+	st->st_gid = in->st_gid;
+	st->st_rdev = in->st_rdev;
+	st->st_size = in->st_size;
+	st->st_blksize = in->st_blksize;
+	st->st_blocks = in->st_blocks;
+	st->st_atim.tv_sec = in->lkl_st_atime;
+	st->st_atim.tv_nsec = in->st_atime_nsec;
+	st->st_mtim.tv_sec = in->lkl_st_mtime;
+	st->st_mtim.tv_nsec = in->st_mtime_nsec;
+	st->st_ctim.tv_sec = in->lkl_st_ctime;
+	st->st_ctim.tv_nsec = in->st_ctime_nsec;
+}
+
+static int lklfuse_fgetattr(const char *path, struct stat *st,
+			    struct fuse_file_info *fi)
+{
+	long ret;
+	struct lkl_stat lkl_stat;
+
+	ret = lkl_sys_fstat(fi->fh, &lkl_stat);
+	if (ret)
+		return ret;
+
+	lklfuse_xlat_stat(&lkl_stat, st);
+	return 0;
+}
+
+static int lklfuse_getattr(const char *path, struct stat *st)
+{
+	long ret;
+	struct lkl_stat lkl_stat;
+
+	ret = lkl_sys_lstat(path, &lkl_stat);
+	if (ret)
+		return ret;
+
+	lklfuse_xlat_stat(&lkl_stat, st);
+	return 0;
+}
+
+static int lklfuse_readlink(const char *path, char *buf, size_t len)
+{
+	long ret;
+
+	ret = lkl_sys_readlink(path, buf, len);
+	if (ret < 0)
+		return ret;
+
+	if ((size_t)ret == len)
+		ret = len - 1;
+
+	buf[ret] = 0;
+
+	return 0;
+}
+
+static int lklfuse_mknod(const char *path, mode_t mode, dev_t dev)
+{
+	return lkl_sys_mknod(path, mode, dev);
+}
+
+static int lklfuse_mkdir(const char *path, mode_t mode)
+{
+	return lkl_sys_mkdir(path, mode);
+}
+
+static int lklfuse_unlink(const char *path)
+{
+	return lkl_sys_unlink(path);
+}
+
+static int lklfuse_rmdir(const char *path)
+{
+	return lkl_sys_rmdir(path);
+}
+
+static int lklfuse_symlink(const char *oldname, const char *newname)
+{
+	return lkl_sys_symlink(oldname, newname);
+}
+
+
+static int lklfuse_rename(const char *oldname, const char *newname)
+{
+	return lkl_sys_rename(oldname, newname);
+}
+
+static int lklfuse_link(const char *oldname, const char *newname)
+{
+	return lkl_sys_link(oldname, newname);
+}
+
+static int lklfuse_chmod(const char *path, mode_t mode)
+{
+	return lkl_sys_chmod(path, mode);
+}
+
+
+static int lklfuse_chown(const char *path, uid_t uid, gid_t gid)
+{
+	return lkl_sys_fchownat(LKL_AT_FDCWD, path, uid, gid,
+				LKL_AT_SYMLINK_NOFOLLOW);
+}
+
+static int lklfuse_truncate(const char *path, off_t off)
+{
+	return lkl_sys_truncate(path, off);
+}
+
+static int lklfuse_open3(const char *path, bool create, mode_t mode,
+			 struct fuse_file_info *fi)
+{
+	long ret;
+	int flags;
+
+	if ((fi->flags & O_ACCMODE) == O_RDONLY)
+		flags = LKL_O_RDONLY;
+	else if ((fi->flags & O_ACCMODE) == O_WRONLY)
+		flags = LKL_O_WRONLY;
+	else if ((fi->flags & O_ACCMODE) == O_RDWR)
+		flags = LKL_O_RDWR;
+	else
+		return -EINVAL;
+
+	if (create)
+		flags |= LKL_O_CREAT;
+
+	ret = lkl_sys_open(path, flags, mode);
+	if (ret < 0)
+		return ret;
+
+	fi->fh = ret;
+
+	return 0;
+}
+
+static int lklfuse_create(const char *path, mode_t mode,
+			  struct fuse_file_info *fi)
+{
+	return lklfuse_open3(path, true, mode, fi);
+}
+
+static int lklfuse_open(const char *path, struct fuse_file_info *fi)
+{
+	return lklfuse_open3(path, false, 0, fi);
+}
+
+static int lklfuse_read(const char *path, char *buf, size_t size, off_t offset,
+		      struct fuse_file_info *fi)
+{
+	long ret;
+	ssize_t orig_size = size;
+
+	do {
+		ret = lkl_sys_pread64(fi->fh, buf, size, offset);
+		if (ret <= 0)
+			break;
+		size -= ret;
+		offset += ret;
+		buf += ret;
+	} while (size > 0);
+
+	return ret < 0 ? ret : orig_size - (ssize_t)size;
+
+}
+
+static int lklfuse_write(const char *path, const char *buf, size_t size,
+		       off_t offset, struct fuse_file_info *fi)
+{
+	long ret;
+	ssize_t orig_size = size;
+
+	do {
+		ret = lkl_sys_pwrite64(fi->fh, buf, size, offset);
+		if (ret <= 0)
+			break;
+		size -= ret;
+		offset += ret;
+		buf += ret;
+	} while (size > 0);
+
+	return ret < 0 ? ret : orig_size - (ssize_t)size;
+}
+
+
+static int lklfuse_statfs(const char *path, struct statvfs *stat)
+{
+	long ret;
+	struct lkl_statfs lkl_statfs;
+
+	ret = lkl_sys_statfs(path, &lkl_statfs);
+	if (ret < 0)
+		return ret;
+
+	stat->f_bsize = lkl_statfs.f_bsize;
+	stat->f_frsize = lkl_statfs.f_frsize;
+	stat->f_blocks = lkl_statfs.f_blocks;
+	stat->f_bfree = lkl_statfs.f_bfree;
+	stat->f_bavail = lkl_statfs.f_bavail;
+	stat->f_files = lkl_statfs.f_files;
+	stat->f_ffree = lkl_statfs.f_ffree;
+	stat->f_favail = stat->f_ffree;
+	stat->f_fsid = *(unsigned long *)&lkl_statfs.f_fsid.val[0];
+	stat->f_flag = lkl_statfs.f_flags;
+	stat->f_namemax = lkl_statfs.f_namelen;
+
+	return 0;
+}
+
+static int lklfuse_flush(const char *path, struct fuse_file_info *fi)
+{
+	return 0;
+}
+
+static int lklfuse_release(const char *path, struct fuse_file_info *fi)
+{
+	return lkl_sys_close(fi->fh);
+}
+
+static int lklfuse_fsync(const char *path, int datasync,
+		       struct fuse_file_info *fi)
+{
+	if (datasync)
+		return lkl_sys_fdatasync(fi->fh);
+	else
+		return lkl_sys_fsync(fi->fh);
+}
+
+static int lklfuse_setxattr(const char *path, const char *name, const char *val,
+		   size_t size, int flags)
+{
+	return lkl_sys_setxattr(path, name, val, size, flags);
+}
+
+static int lklfuse_getxattr(const char *path, const char *name, char *val,
+			  size_t size)
+{
+	return lkl_sys_getxattr(path, name, val, size);
+}
+
+static int lklfuse_listxattr(const char *path, char *list, size_t size)
+{
+	return lkl_sys_listxattr(path, list, size);
+}
+
+static int lklfuse_removexattr(const char *path, const char *name)
+{
+	return lkl_sys_removexattr(path, name);
+}
+
+static int lklfuse_opendir(const char *path, struct fuse_file_info *fi)
+{
+	struct lkl_dir *dir;
+	int err;
+
+	dir = lkl_opendir(path, &err);
+	if (!dir)
+		return err;
+
+	fi->fh = (uintptr_t)dir;
+
+	return 0;
+}
+
+/** Read directory
+ *
+ * This supersedes the old getdir() interface.  New applications
+ * should use this.
+ *
+ * The filesystem may choose between two modes of operation:
+ *
+ * 1) The readdir implementation ignores the offset parameter, and
+ * passes zero to the filler function's offset.  The filler
+ * function will not return '1' (unless an error happens), so the
+ * whole directory is read in a single readdir operation.  This
+ * works just like the old getdir() method.
+ *
+ * 2) The readdir implementation keeps track of the offsets of the
+ * directory entries.  It uses the offset parameter and always
+ * passes non-zero offset to the filler function.  When the buffer
+ * is full (or an error happens) the filler function will return
+ * '1'.
+ *
+ * Introduced in version 2.3
+ */
+static int lklfuse_readdir(const char *path, void *buf, fuse_fill_dir_t fill,
+			 off_t off, struct fuse_file_info *fi)
+{
+	struct lkl_dir *dir = (struct lkl_dir *)(uintptr_t)fi->fh;
+	struct lkl_linux_dirent64 *de;
+
+	while ((de = lkl_readdir(dir))) {
+		struct stat st = { 0, };
+
+		st.st_ino = de->d_ino;
+		st.st_mode = de->d_type << 12;
+
+		if (fill(buf, de->d_name, &st, 0))
+			break;
+	}
+
+	if (!de)
+		return lkl_errdir(dir);
+
+	return 0;
+}
+
+static int lklfuse_releasedir(const char *path, struct fuse_file_info *fi)
+{
+	struct lkl_dir *dir = (struct lkl_dir *)(uintptr_t)fi->fh;
+
+	return lkl_closedir(dir);
+}
+
+static int lklfuse_fsyncdir(const char *path, int datasync,
+			  struct fuse_file_info *fi)
+{
+	struct lkl_dir *dir = (struct lkl_dir *)(uintptr_t)fi->fh;
+	int fd = lkl_dirfd(dir);
+
+	if (datasync)
+		return lkl_sys_fdatasync(fd);
+	else
+		return lkl_sys_fsync(fd);
+}
+
+static int lklfuse_access(const char *path, int mode)
+{
+	return lkl_sys_access(path, mode);
+}
+
+static int lklfuse_utimens(const char *path, const struct timespec tv[2])
+{
+	struct lkl_timespec ts[2];
+
+	ts[0].tv_sec = tv[0].tv_sec;
+	ts[0].tv_nsec = tv[0].tv_nsec;
+	ts[1].tv_sec = tv[0].tv_sec;
+	ts[1].tv_nsec = tv[0].tv_nsec;
+
+	return lkl_sys_utimensat(-1, path, (struct __lkl__kernel_timespec *)ts,
+				 LKL_AT_SYMLINK_NOFOLLOW);
+}
+
+static int lklfuse_fallocate(const char *path, int mode, off_t offset,
+			     off_t len, struct fuse_file_info *fi)
+{
+	return lkl_sys_fallocate(fi->fh, mode, offset, len);
+}
+
+const struct fuse_operations lklfuse_ops = {
+	.flag_nullpath_ok = 1,
+	.flag_nopath = 1,
+	.flag_utime_omit_ok = 1,
+
+	.getattr = lklfuse_getattr,
+	.readlink = lklfuse_readlink,
+	.mknod = lklfuse_mknod,
+	.mkdir = lklfuse_mkdir,
+	.unlink = lklfuse_unlink,
+	.rmdir = lklfuse_rmdir,
+	.symlink = lklfuse_symlink,
+	.rename = lklfuse_rename,
+	.link = lklfuse_link,
+	.chmod = lklfuse_chmod,
+	.chown = lklfuse_chown,
+	.truncate = lklfuse_truncate,
+	.open = lklfuse_open,
+	.read = lklfuse_read,
+	.write = lklfuse_write,
+	.statfs = lklfuse_statfs,
+	.flush = lklfuse_flush,
+	.release = lklfuse_release,
+	.fsync = lklfuse_fsync,
+	.setxattr = lklfuse_setxattr,
+	.getxattr = lklfuse_getxattr,
+	.listxattr = lklfuse_listxattr,
+	.removexattr = lklfuse_removexattr,
+	.opendir = lklfuse_opendir,
+	.readdir = lklfuse_readdir,
+	.releasedir = lklfuse_releasedir,
+	.fsyncdir = lklfuse_fsyncdir,
+	.access = lklfuse_access,
+	.create = lklfuse_create,
+	.fgetattr = lklfuse_fgetattr,
+	/* .lock, */
+	.utimens = lklfuse_utimens,
+	/* .bmap, */
+	/* .ioctl, */
+	/* .poll, */
+	/* .write_buf, (SG io) */
+	/* .read_buf, (SG io) */
+	/* .flock, */
+	.fallocate = lklfuse_fallocate,
+};
+
+static int start_lkl(void)
+{
+	long ret;
+	char mpoint[32], cmdline[16];
+
+	snprintf(cmdline, sizeof(cmdline), "mem=%dM", lklfuse.mb);
+	ret = lkl_start_kernel(&lkl_host_ops, cmdline);
+	if (ret) {
+		fprintf(stderr, "can't start kernel: %s\n", lkl_strerror(ret));
+		goto out;
+	}
+
+	ret = lkl_mount_dev(lklfuse.disk_id, lklfuse.part, lklfuse.type,
+			    lklfuse.ro ? LKL_MS_RDONLY : 0, lklfuse.opts,
+			    mpoint, sizeof(mpoint));
+
+	if (ret) {
+		fprintf(stderr, "can't mount disk: %s\n", lkl_strerror(ret));
+		goto out_halt;
+	}
+
+	ret = lkl_sys_chroot(mpoint);
+	if (ret) {
+		fprintf(stderr, "can't chdir to %s: %s\n", mpoint,
+			lkl_strerror(ret));
+		goto out_umount;
+	}
+
+	return 0;
+
+out_umount:
+	lkl_umount_dev(lklfuse.disk_id, lklfuse.part, 0, 1000);
+
+out_halt:
+	lkl_sys_halt();
+
+out:
+	return ret;
+}
+
+static void stop_lkl(void)
+{
+	int ret;
+
+	ret = lkl_sys_chdir("/");
+	if (ret)
+		fprintf(stderr, "can't chdir to /: %s\n", lkl_strerror(ret));
+	ret = lkl_sys_umount("/", 0);
+	if (ret)
+		fprintf(stderr, "failed to umount disk: %d: %s\n",
+			lklfuse.disk_id, lkl_strerror(ret));
+	lkl_sys_halt();
+}
+
+int main(int argc, char **argv)
+{
+	struct fuse_args args = FUSE_ARGS_INIT(argc, argv);
+	struct fuse_chan *ch;
+	struct fuse *fuse;
+	struct stat st;
+	char *mnt;
+	int fg, mt, ret;
+
+	if (fuse_opt_parse(&args, &lklfuse, lklfuse_opts, lklfuse_opt_proc))
+		return 1;
+
+	if (!lklfuse.file || !lklfuse.type) {
+		fprintf(stderr, "no file or filesystem type specified\n");
+		return 1;
+	}
+
+	if (fuse_parse_cmdline(&args, &mnt, &mt, &fg))
+		return 1;
+
+	ret = stat(mnt, &st);
+	if (ret) {
+		perror(mnt);
+		goto out_free;
+	}
+
+	ret = open(lklfuse.file, lklfuse.ro ? O_RDONLY : O_RDWR);
+	if (ret < 0) {
+		perror(lklfuse.file);
+		goto out_free;
+	}
+
+	lklfuse.disk.fd = ret;
+	lklfuse.disk.dev = (char *)lklfuse.file;
+
+	ret = lkl_disk_add(&lklfuse.disk);
+	if (ret < 0) {
+		fprintf(stderr, "can't add disk: %s\n", lkl_strerror(ret));
+		goto out_close_disk;
+	}
+
+	lklfuse.disk_id = ret;
+
+	ch = fuse_mount(mnt, &args);
+	if (!ch) {
+		ret = -1;
+		goto out_close_disk;
+	}
+
+	fuse = fuse_new(ch, &args, &lklfuse_ops, sizeof(lklfuse_ops), NULL);
+	if (!fuse) {
+		ret = -1;
+		goto out_fuse_unmount;
+	}
+
+	fuse_opt_free_args(&args);
+
+	if (fuse_daemonize(fg) ||
+	    fuse_set_signal_handlers(fuse_get_session(fuse))) {
+		ret = -1;
+		goto out_fuse_destroy;
+	}
+
+	ret = start_lkl();
+	if (ret) {
+		ret = -1;
+		goto out_remove_signals;
+	}
+
+	if (mt)
+		fprintf(stderr, "warning: multithreaded mode not supported\n");
+
+	ret = fuse_loop(fuse);
+
+	stop_lkl();
+
+out_remove_signals:
+	fuse_remove_signal_handlers(fuse_get_session(fuse));
+
+out_fuse_unmount:
+	if (ch)
+		fuse_unmount(mnt, ch);
+
+out_fuse_destroy:
+	if (fuse)
+		fuse_destroy(fuse);
+
+out_close_disk:
+	close(lklfuse.disk.fd);
+
+out_free:
+	free(mnt);
+
+	return ret < 0 ? 1 : 0;
+}
diff --git a/tools/lkl/tests/lklfuse.sh b/tools/lkl/tests/lklfuse.sh
new file mode 100755
index 000000000000..7f35dd53fc4e
--- /dev/null
+++ b/tools/lkl/tests/lklfuse.sh
@@ -0,0 +1,110 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+
+source $script_dir/test.sh
+
+cleanup()
+{
+    set -e
+
+    sleep 1
+    fusermount -u $dir
+    rm $file
+    rmdir $dir
+}
+
+
+# $1 - disk image
+# $2 - fstype
+function prepfs()
+{
+    set -e
+
+    dd if=/dev/zero of=$1 bs=1024 count=102400
+
+    yes | mkfs.$2 $1
+}
+
+# $1 - disk image
+# $2 - mount point
+# $3 - filesystem type
+lklfuse_mount()
+{
+    ${script_dir}/../lklfuse $1 $2 -o type=$3
+}
+
+# $1 - mount point
+lklfuse_basic()
+{
+    set -e
+
+    cd $1
+    touch a
+    if ! [ -e ]; then exit 1; fi
+    rm a
+    mkdir a
+    if ! [ -d ]; then exit 1; fi
+    rmdir a
+}
+
+# $1 - dir
+# $2 - filesystem type
+lklfuse_stressng()
+{
+    set -e
+
+    if [ -z $(which stress-ng) ]; then
+        echo "missing stress-ng"
+        return $TEST_SKIP
+    fi
+
+    cd $1
+
+    if [ "$2" = "vfat" ]; then
+        exclude="chmod,filename,link,mknod,symlink,xattr"
+    fi
+
+    stress-ng --class filesystem --all 0 --timeout 10 \
+	      --exclude fiemap,$exclude --fallocate-bytes 10m \
+	      --sync-file-bytes 10m
+}
+
+if [ "$1" = "-t" ]; then
+    shift
+    fstype=$1
+    shift
+fi
+
+if [ -z "$fstype" ]; then
+    fstype="ext4"
+fi
+
+if ! [ -x $script_dir/../lklfuse ]; then
+    lkl_test_plan 0 "lklfuse.sh $fstype"
+    echo "lklfuse not available"
+    exit 0
+fi
+
+if ! [ -e /dev/fuse ]; then
+    lkl_test_plan 0 "lklfuse.sh $fstype"
+    echo "/dev/fuse not available"
+    exit 0
+fi
+
+
+file=`mktemp`
+dir=`mktemp -d`
+
+trap cleanup EXIT
+
+lkl_test_plan 4 "lklfuse $fstype"
+
+lkl_test_run 1 prepfs $file $fstype
+lkl_test_run 2 lklfuse_mount $file $dir $fstype
+lkl_test_run 3 lklfuse_basic $dir
+# stress-ng returns 2 with no apparent failures so skip it for now
+#lkl_test_run 4 lklfuse_stressng $dir $fstype
+trap : EXIT
+lkl_test_run 4 cleanup
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 21/25] um lkl: add documentation
  2020-03-30 14:45   ` Hajime Tazaki
@ 2020-03-30 14:45     ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Chenyang Zhong, Conrad Meyer, Gustavo Bittencourt, Motomu Utsumi,
	Patrick Collins, Thomas Liebetraut, Yuan Liu

From: Octavian Purdila <tavi.purdila@gmail.com>

A document describing brief introduction of LKL, why it is needed, and
how it is used is added.  The document is located under uml/ directory.

Cc: Chenyang Zhong <zhongcy95@gmail.com>
Cc: Conrad Meyer <cem@FreeBSD.org>
Cc: Gustavo Bittencourt <gbitten@gmail.com>
Cc: H.K. Jerry Chu <hkchu@google.com>
Cc: Motomu Utsumi <motomuman@gmail.com>
Cc: Patrick Collins <pscollins@google.com>
Cc: Thomas Liebetraut <thomas@tommie-lie.de>
Cc: Yuan Liu <liuyuan@google.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 Documentation/virt/uml/lkl.txt | 64 ++++++++++++++++++++++++++++++++++
 1 file changed, 64 insertions(+)
 create mode 100644 Documentation/virt/uml/lkl.txt

diff --git a/Documentation/virt/uml/lkl.txt b/Documentation/virt/uml/lkl.txt
new file mode 100644
index 000000000000..f6d581092331
--- /dev/null
+++ b/Documentation/virt/uml/lkl.txt
@@ -0,0 +1,64 @@
+
+Introduction
+============
+
+LKL (Linux Kernel Library) is aiming to allow reusing the Linux kernel code as
+extensively as possible with minimal effort and reduced maintenance overhead.
+
+Examples of how LKL can be used are: creating userspace applications (running on
+Linux and other operating systems) that can read or write Linux filesystems or
+can use the Linux networking stack, creating kernel drivers for other operating
+systems that can read Linux filesystems, bootloaders support for reading/writing
+Linux filesystems, etc.
+
+With LKL, the kernel code is compiled into an object file that can be directly
+linked by applications. The API offered by LKL is based on the Linux system call
+interface.
+
+LKL is implemented as one of the mode of UML (arch/um). It uses host operations
+defined by the application or a host library (tools/lkl/lib).
+
+
+Supported hosts
+===============
+
+The supported host for now is Linux (x86 architecture) userspace applications.
+
+
+Building LKL the host library and LKL applications
+==================================================
+
+    $ make -C tools/lkl
+
+will build LKL as a object file, it will install it in tools/lkl/lib together
+with the headers files in tools/lkl/include then will build the host library,
+tests and a few of application examples:
+
+* tests/boot - a simple applications that uses LKL and exercises the basic LKL
+  APIs
+
+* tests/net-test - a simple applications that uses network feature of LKL and
+  exercises the basic network-related APIs
+
+* fs2tar - a tool that converts a filesystem image to a tar archive
+
+* cptofs/cpfromfs - a tool that copies files to/from a filesystem image
+
+* lklfuse - a tool that can mount a filesystem image in userspace,
+  without root privileges, using FUSE
+
+
+Building LKL on Ubuntu
+-----------------------
+
+    $ sudo apt-get install libfuse-dev libarchive-dev xfsprogs
+
+    # Optional, if you would like to be able to run tests
+    $ sudo apt-get install btrfs-tools
+    $ pip install yamlish junit_xml
+
+    $ make -C tools/lkl
+
+    # To check that everything works:
+    $ cd tools/lkl
+    $ make run-tests
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 21/25] um lkl: add documentation
@ 2020-03-30 14:45     ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: linux-arch, Conrad Meyer, Octavian Purdila, Motomu Utsumi,
	Akira Moroo, Thomas Liebetraut, Patrick Collins,
	linux-kernel-library, Chenyang Zhong, Yuan Liu,
	Gustavo Bittencourt

From: Octavian Purdila <tavi.purdila@gmail.com>

A document describing brief introduction of LKL, why it is needed, and
how it is used is added.  The document is located under uml/ directory.

Cc: Chenyang Zhong <zhongcy95@gmail.com>
Cc: Conrad Meyer <cem@FreeBSD.org>
Cc: Gustavo Bittencourt <gbitten@gmail.com>
Cc: H.K. Jerry Chu <hkchu@google.com>
Cc: Motomu Utsumi <motomuman@gmail.com>
Cc: Patrick Collins <pscollins@google.com>
Cc: Thomas Liebetraut <thomas@tommie-lie.de>
Cc: Yuan Liu <liuyuan@google.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 Documentation/virt/uml/lkl.txt | 64 ++++++++++++++++++++++++++++++++++
 1 file changed, 64 insertions(+)
 create mode 100644 Documentation/virt/uml/lkl.txt

diff --git a/Documentation/virt/uml/lkl.txt b/Documentation/virt/uml/lkl.txt
new file mode 100644
index 000000000000..f6d581092331
--- /dev/null
+++ b/Documentation/virt/uml/lkl.txt
@@ -0,0 +1,64 @@
+
+Introduction
+============
+
+LKL (Linux Kernel Library) is aiming to allow reusing the Linux kernel code as
+extensively as possible with minimal effort and reduced maintenance overhead.
+
+Examples of how LKL can be used are: creating userspace applications (running on
+Linux and other operating systems) that can read or write Linux filesystems or
+can use the Linux networking stack, creating kernel drivers for other operating
+systems that can read Linux filesystems, bootloaders support for reading/writing
+Linux filesystems, etc.
+
+With LKL, the kernel code is compiled into an object file that can be directly
+linked by applications. The API offered by LKL is based on the Linux system call
+interface.
+
+LKL is implemented as one of the mode of UML (arch/um). It uses host operations
+defined by the application or a host library (tools/lkl/lib).
+
+
+Supported hosts
+===============
+
+The supported host for now is Linux (x86 architecture) userspace applications.
+
+
+Building LKL the host library and LKL applications
+==================================================
+
+    $ make -C tools/lkl
+
+will build LKL as a object file, it will install it in tools/lkl/lib together
+with the headers files in tools/lkl/include then will build the host library,
+tests and a few of application examples:
+
+* tests/boot - a simple applications that uses LKL and exercises the basic LKL
+  APIs
+
+* tests/net-test - a simple applications that uses network feature of LKL and
+  exercises the basic network-related APIs
+
+* fs2tar - a tool that converts a filesystem image to a tar archive
+
+* cptofs/cpfromfs - a tool that copies files to/from a filesystem image
+
+* lklfuse - a tool that can mount a filesystem image in userspace,
+  without root privileges, using FUSE
+
+
+Building LKL on Ubuntu
+-----------------------
+
+    $ sudo apt-get install libfuse-dev libarchive-dev xfsprogs
+
+    # Optional, if you would like to be able to run tests
+    $ sudo apt-get install btrfs-tools
+    $ pip install yamlish junit_xml
+
+    $ make -C tools/lkl
+
+    # To check that everything works:
+    $ cd tools/lkl
+    $ make run-tests
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 22/25] um lkl: add CI scripts to conduct regression tests
  2020-03-30 14:45   ` Hajime Tazaki
@ 2020-03-30 14:45     ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Hajime Tazaki

For the continuous integration (CI), we use CircleCI with multiple test
targets triggered with a single commit.  The test also conducts memory
checks using valgrind in order to prevent enbug.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 .circleci/config.yml             | 196 +++++++++++++++++++++++++++++++
 tools/lkl/scripts/checkpatch.sh  |  60 ++++++++++
 tools/lkl/scripts/lkl-jenkins.sh |  21 ++++
 3 files changed, 277 insertions(+)
 create mode 100644 .circleci/config.yml
 create mode 100755 tools/lkl/scripts/checkpatch.sh
 create mode 100755 tools/lkl/scripts/lkl-jenkins.sh

diff --git a/.circleci/config.yml b/.circleci/config.yml
new file mode 100644
index 000000000000..5bdf059014c0
--- /dev/null
+++ b/.circleci/config.yml
@@ -0,0 +1,196 @@
+version: 2
+general:
+  artifacts:
+
+do_steps: &do_steps
+ steps:
+  - run: echo "$CROSS_COMPILE" > ~/_cross_compile
+  - restore_cache:
+      key: code-tree-shallow-{{ .Environment.CACHE_VERSION }}
+  - run:
+      name: checkout build tree
+      command: |
+        mkdir -p ~/.ssh/
+        ssh-keyscan -H github.com >> ~/.ssh/known_hosts
+        if ! [ -d .git ]; then
+          git clone --depth=1 $CIRCLE_REPOSITORY_URL .;
+        fi
+        if [[ $CIRCLE_BRANCH == pull/* ]]; then
+           git fetch --depth=1 origin $CIRCLE_BRANCH/head;
+        else
+           git fetch --depth=1 origin $CIRCLE_BRANCH;
+        fi
+        git reset --hard $CIRCLE_SHA1
+  - save_cache:
+      key: code-tree-shallow-{{ .Environment.CACHE_VERSION }}-{{ epoch }}
+      paths:
+        - /home/ubuntu/project/.git
+  - run:
+      name: clean
+      command: |
+        make mrproper
+        cd tools/lkl && make clean-conf
+        rm -rf ~/junit
+  - run: mkdir -p /home/ubuntu/.ccache
+  - restore_cache:
+      key: compiler-cache-{{ checksum "~/_cross_compile" }}-{{ .Environment.CACHE_VERSION }}
+  - run: cd tools/lkl && make -j8 ${MKARG}
+  - run: mkdir -p ~/destdir && cd tools/lkl && make DESTDIR=~/destdir
+  - save_cache:
+     paths:
+       - /home/ubuntu/.ccache
+     key: compiler-cache-{{ checksum "~/_cross_compile" }}-{{ .Environment.CACHE_VERSION }}-{{ epoch }}
+  - run:
+      name: run tests
+      command: |
+        mkdir -p ~/junit
+        make -C tools/lkl run-tests tests="--junit-dir ~/junit"
+      no_output_timeout: "90m"
+  - run:
+      name: collecting test results
+      command: |
+        find ./tools/lkl/ -type f -name "*.xml" -exec mv {} ~/junit/ \;
+  - store_test_results:
+      path: ~/junit
+  - store_artifacts:
+      path: ~/junit
+
+
+do_uml_steps: &do_uml_steps
+ steps:
+  - run: echo "$CROSS_COMPILE" > ~/_cross_compile
+  - restore_cache:
+      key: code-tree-shallow-{{ .Environment.CACHE_VERSION }}
+  - run:
+      name: checkout build tree
+      command: |
+        mkdir -p ~/.ssh/
+        ssh-keyscan -H github.com >> ~/.ssh/known_hosts
+        if ! [ -d .git ]; then
+          git clone --depth=1 $CIRCLE_REPOSITORY_URL .;
+        fi
+        if [[ $CIRCLE_BRANCH == pull/* ]]; then
+           git fetch --depth=1 origin $CIRCLE_BRANCH/head;
+        else
+           git fetch --depth=1 origin $CIRCLE_BRANCH;
+        fi
+        git reset --hard $CIRCLE_SHA1
+  - save_cache:
+      key: code-tree-shallow-{{ .Environment.CACHE_VERSION }}-{{ epoch }}
+      paths:
+        - /home/ubuntu/project/.git
+  - run: mkdir -p /home/ubuntu/.ccache
+  - restore_cache:
+      key: compiler-cache-{{ checksum "~/_cross_compile" }}-{{ .Environment.CACHE_VERSION }}
+  - run:
+      name: build
+      command: |
+        sudo apt-get update
+        sudo apt-get install -y gcc-multilib g++-multilib
+        make defconfig ARCH=um SUBARCH=$SUBARCH
+        make ARCH=um SUBARCH=$SUBARCH
+  - save_cache:
+     paths:
+       - /home/ubuntu/.ccache
+     key: compiler-cache-{{ checksum "~/_cross_compile" }}-{{ .Environment.CACHE_VERSION }}-{{ epoch }}
+  - run:
+      name: test
+      command: |
+        # XXX: i386 build doesn't work with the test
+        if [ $CIRCLE_STAGE = "i386_uml" ] || [ $CIRCLE_STAGE = "i386_uml_on_x86_64" ]; then
+          exit 0
+        fi
+        ./linux rootfstype=hostfs ro mem=1g loglevel=10 init="/bin/bash -c exit" || export RETVAL=$?
+        # SIGABRT=6 => 128+6
+        if [ $RETVAL != "134" ]; then
+          exit 1
+        fi
+
+## Customize the test machine
+jobs:
+  x86_64:
+   docker:
+     - image: lkldocker/circleci-x86_64:0.7
+   environment:
+     CROSS_COMPILE: ""
+   <<: *do_steps
+
+  i386:
+   docker:
+     - image: lkldocker/circleci-i386:0.1
+   environment:
+     CROSS_COMPILE: ""
+   <<: *do_steps
+
+  x86_64_valgrind:
+   docker:
+     - image: lkldocker/circleci-x86_64:0.7
+   environment:
+     CROSS_COMPILE: ""
+     VALGRIND: 1
+   <<: *do_steps
+
+  x86_64_uml:
+   docker:
+     - image: lkldocker/circleci-x86_64:0.7
+   environment:
+     CROSS_COMPILE: ""
+     TMPDIR: "/tmp" # required for not using /dev/shm
+     SUBARCH: "x86_64"
+   <<: *do_uml_steps
+
+  i386_uml:
+   docker:
+     - image: lkldocker/circleci-i386:0.1
+   environment:
+     CROSS_COMPILE: ""
+     SUBARCH: "i386"
+     TMPDIR: "/tmp" # required for not using /dev/shm
+   <<: *do_uml_steps
+
+  i386_uml_on_x86_64:
+   docker:
+     - image: lkldocker/circleci-x86_64:0.7
+   environment:
+     CROSS_COMPILE: ""
+     TMPDIR: "/tmp" # required for not using /dev/shm
+     SUBARCH: "i386"
+   <<: *do_uml_steps
+
+  checkpatch:
+   docker:
+     - image: lkldocker/circleci:0.5
+   environment:
+   steps:
+     - restore_cache:
+        key: code-tree-full-history-{{ .Environment.CACHE_VERSION }}
+     - checkout
+     - run: sudo pip install ply
+     - run: tools/lkl/scripts/checkpatch.sh
+     - save_cache:
+        key: code-tree-full-history-{{ .Environment.CACHE_VERSION }}-{{ epoch }}
+        paths:
+          - /home/ubuntu/project/.git
+        when: always
+
+workflows:
+  version: 2
+  build:
+    jobs:
+     - x86_64
+     - x86_64_valgrind
+     - checkpatch
+     - i386
+     - x86_64_uml
+     - i386_uml
+     - i386_uml_on_x86_64
+  nightly:
+    triggers:
+      - schedule:
+          cron: "0 0 * * *"
+          filters:
+            branches:
+              only:
+                - master
+    jobs:
+      - x86_64_valgrind
diff --git a/tools/lkl/scripts/checkpatch.sh b/tools/lkl/scripts/checkpatch.sh
new file mode 100755
index 000000000000..0c02ca6b21a2
--- /dev/null
+++ b/tools/lkl/scripts/checkpatch.sh
@@ -0,0 +1,60 @@
+#!/bin/sh -ex
+# SPDX-License-Identifier: GPL-2.0
+
+if [ -z "$origin_master" ]; then
+    origin_master="origin/master"
+fi
+
+UPSTREAM=git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
+LKL=github.com:lkl/linux.git
+
+upstream=`git remote -v | grep $UPSTREAM | cut -f1 | head -n1`
+lkl=`git remote -v | grep $LKL | cut -f1 | head -n1`
+
+if [ -z "$upstream" ]; then
+    git fetch --tags --progress git://$UPSTREAM
+else
+    git fetch --tags $upstream
+fi
+
+if [ -z "$lkl" ]; then
+    git remote add lkl-upstream git@$LKL || true
+    lkl=`git remote -v | grep $LKL | cut -f1 | head -n1`
+fi
+
+if [ -z "$lkl" ]; then
+    echo "can't find lkl remote, quiting"
+    exit 1
+fi
+
+git fetch $lkl
+git fetch --tags $upstream
+
+# find the last upstream tag to avoid checking upstream commits during
+# upstream merges
+tag=`git tag --sort='-*authordate' | grep ^v | head -n1`
+tmp=`mktemp -d`
+
+commits=$(git log --no-merges --pretty=format:%h HEAD ^$lkl/master ^$tag)
+for c in $commits; do
+    git format-patch -1 -o $tmp $c
+done
+
+if [ -z "$c" ]; then
+    echo "there are not commits/patches to check, quiting."
+    rmdir $tmp
+    exit 0
+fi
+
+./scripts/checkpatch.pl --ignore FILE_PATH_CHANGES $tmp/*.patch
+rm $tmp/*.patch
+
+# checkpatch.pl does not know how to deal with 3 way diffs which would
+# be useful to check the conflict resolutions during merges...
+#for c in `git log --merges --pretty=format:%h HEAD ^$origin_master ^$tag`; do
+#    git log --pretty=email $c -1 > $tmp/$c.patch
+#    git diff $c $c^1 $c^2 >> $tmp/$c.patch
+#done
+
+rmdir $tmp
+
diff --git a/tools/lkl/scripts/lkl-jenkins.sh b/tools/lkl/scripts/lkl-jenkins.sh
new file mode 100755
index 000000000000..eaadc6e90143
--- /dev/null
+++ b/tools/lkl/scripts/lkl-jenkins.sh
@@ -0,0 +1,21 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+set -e
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+basedir=$(cd $script_dir/../../..; pwd)
+
+export PATH=$PATH:/sbin
+
+build_and_test()
+{
+    cd $basedir
+    make mrproper
+    cd tools/lkl
+    make clean-conf
+    make -j4
+    make run-tests
+}
+
+build_and_test
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 22/25] um lkl: add CI scripts to conduct regression tests
@ 2020-03-30 14:45     ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, linux-kernel-library, linux-arch,
	Hajime Tazaki, Akira Moroo

For the continuous integration (CI), we use CircleCI with multiple test
targets triggered with a single commit.  The test also conducts memory
checks using valgrind in order to prevent enbug.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 .circleci/config.yml             | 196 +++++++++++++++++++++++++++++++
 tools/lkl/scripts/checkpatch.sh  |  60 ++++++++++
 tools/lkl/scripts/lkl-jenkins.sh |  21 ++++
 3 files changed, 277 insertions(+)
 create mode 100644 .circleci/config.yml
 create mode 100755 tools/lkl/scripts/checkpatch.sh
 create mode 100755 tools/lkl/scripts/lkl-jenkins.sh

diff --git a/.circleci/config.yml b/.circleci/config.yml
new file mode 100644
index 000000000000..5bdf059014c0
--- /dev/null
+++ b/.circleci/config.yml
@@ -0,0 +1,196 @@
+version: 2
+general:
+  artifacts:
+
+do_steps: &do_steps
+ steps:
+  - run: echo "$CROSS_COMPILE" > ~/_cross_compile
+  - restore_cache:
+      key: code-tree-shallow-{{ .Environment.CACHE_VERSION }}
+  - run:
+      name: checkout build tree
+      command: |
+        mkdir -p ~/.ssh/
+        ssh-keyscan -H github.com >> ~/.ssh/known_hosts
+        if ! [ -d .git ]; then
+          git clone --depth=1 $CIRCLE_REPOSITORY_URL .;
+        fi
+        if [[ $CIRCLE_BRANCH == pull/* ]]; then
+           git fetch --depth=1 origin $CIRCLE_BRANCH/head;
+        else
+           git fetch --depth=1 origin $CIRCLE_BRANCH;
+        fi
+        git reset --hard $CIRCLE_SHA1
+  - save_cache:
+      key: code-tree-shallow-{{ .Environment.CACHE_VERSION }}-{{ epoch }}
+      paths:
+        - /home/ubuntu/project/.git
+  - run:
+      name: clean
+      command: |
+        make mrproper
+        cd tools/lkl && make clean-conf
+        rm -rf ~/junit
+  - run: mkdir -p /home/ubuntu/.ccache
+  - restore_cache:
+      key: compiler-cache-{{ checksum "~/_cross_compile" }}-{{ .Environment.CACHE_VERSION }}
+  - run: cd tools/lkl && make -j8 ${MKARG}
+  - run: mkdir -p ~/destdir && cd tools/lkl && make DESTDIR=~/destdir
+  - save_cache:
+     paths:
+       - /home/ubuntu/.ccache
+     key: compiler-cache-{{ checksum "~/_cross_compile" }}-{{ .Environment.CACHE_VERSION }}-{{ epoch }}
+  - run:
+      name: run tests
+      command: |
+        mkdir -p ~/junit
+        make -C tools/lkl run-tests tests="--junit-dir ~/junit"
+      no_output_timeout: "90m"
+  - run:
+      name: collecting test results
+      command: |
+        find ./tools/lkl/ -type f -name "*.xml" -exec mv {} ~/junit/ \;
+  - store_test_results:
+      path: ~/junit
+  - store_artifacts:
+      path: ~/junit
+
+
+do_uml_steps: &do_uml_steps
+ steps:
+  - run: echo "$CROSS_COMPILE" > ~/_cross_compile
+  - restore_cache:
+      key: code-tree-shallow-{{ .Environment.CACHE_VERSION }}
+  - run:
+      name: checkout build tree
+      command: |
+        mkdir -p ~/.ssh/
+        ssh-keyscan -H github.com >> ~/.ssh/known_hosts
+        if ! [ -d .git ]; then
+          git clone --depth=1 $CIRCLE_REPOSITORY_URL .;
+        fi
+        if [[ $CIRCLE_BRANCH == pull/* ]]; then
+           git fetch --depth=1 origin $CIRCLE_BRANCH/head;
+        else
+           git fetch --depth=1 origin $CIRCLE_BRANCH;
+        fi
+        git reset --hard $CIRCLE_SHA1
+  - save_cache:
+      key: code-tree-shallow-{{ .Environment.CACHE_VERSION }}-{{ epoch }}
+      paths:
+        - /home/ubuntu/project/.git
+  - run: mkdir -p /home/ubuntu/.ccache
+  - restore_cache:
+      key: compiler-cache-{{ checksum "~/_cross_compile" }}-{{ .Environment.CACHE_VERSION }}
+  - run:
+      name: build
+      command: |
+        sudo apt-get update
+        sudo apt-get install -y gcc-multilib g++-multilib
+        make defconfig ARCH=um SUBARCH=$SUBARCH
+        make ARCH=um SUBARCH=$SUBARCH
+  - save_cache:
+     paths:
+       - /home/ubuntu/.ccache
+     key: compiler-cache-{{ checksum "~/_cross_compile" }}-{{ .Environment.CACHE_VERSION }}-{{ epoch }}
+  - run:
+      name: test
+      command: |
+        # XXX: i386 build doesn't work with the test
+        if [ $CIRCLE_STAGE = "i386_uml" ] || [ $CIRCLE_STAGE = "i386_uml_on_x86_64" ]; then
+          exit 0
+        fi
+        ./linux rootfstype=hostfs ro mem=1g loglevel=10 init="/bin/bash -c exit" || export RETVAL=$?
+        # SIGABRT=6 => 128+6
+        if [ $RETVAL != "134" ]; then
+          exit 1
+        fi
+
+## Customize the test machine
+jobs:
+  x86_64:
+   docker:
+     - image: lkldocker/circleci-x86_64:0.7
+   environment:
+     CROSS_COMPILE: ""
+   <<: *do_steps
+
+  i386:
+   docker:
+     - image: lkldocker/circleci-i386:0.1
+   environment:
+     CROSS_COMPILE: ""
+   <<: *do_steps
+
+  x86_64_valgrind:
+   docker:
+     - image: lkldocker/circleci-x86_64:0.7
+   environment:
+     CROSS_COMPILE: ""
+     VALGRIND: 1
+   <<: *do_steps
+
+  x86_64_uml:
+   docker:
+     - image: lkldocker/circleci-x86_64:0.7
+   environment:
+     CROSS_COMPILE: ""
+     TMPDIR: "/tmp" # required for not using /dev/shm
+     SUBARCH: "x86_64"
+   <<: *do_uml_steps
+
+  i386_uml:
+   docker:
+     - image: lkldocker/circleci-i386:0.1
+   environment:
+     CROSS_COMPILE: ""
+     SUBARCH: "i386"
+     TMPDIR: "/tmp" # required for not using /dev/shm
+   <<: *do_uml_steps
+
+  i386_uml_on_x86_64:
+   docker:
+     - image: lkldocker/circleci-x86_64:0.7
+   environment:
+     CROSS_COMPILE: ""
+     TMPDIR: "/tmp" # required for not using /dev/shm
+     SUBARCH: "i386"
+   <<: *do_uml_steps
+
+  checkpatch:
+   docker:
+     - image: lkldocker/circleci:0.5
+   environment:
+   steps:
+     - restore_cache:
+        key: code-tree-full-history-{{ .Environment.CACHE_VERSION }}
+     - checkout
+     - run: sudo pip install ply
+     - run: tools/lkl/scripts/checkpatch.sh
+     - save_cache:
+        key: code-tree-full-history-{{ .Environment.CACHE_VERSION }}-{{ epoch }}
+        paths:
+          - /home/ubuntu/project/.git
+        when: always
+
+workflows:
+  version: 2
+  build:
+    jobs:
+     - x86_64
+     - x86_64_valgrind
+     - checkpatch
+     - i386
+     - x86_64_uml
+     - i386_uml
+     - i386_uml_on_x86_64
+  nightly:
+    triggers:
+      - schedule:
+          cron: "0 0 * * *"
+          filters:
+            branches:
+              only:
+                - master
+    jobs:
+      - x86_64_valgrind
diff --git a/tools/lkl/scripts/checkpatch.sh b/tools/lkl/scripts/checkpatch.sh
new file mode 100755
index 000000000000..0c02ca6b21a2
--- /dev/null
+++ b/tools/lkl/scripts/checkpatch.sh
@@ -0,0 +1,60 @@
+#!/bin/sh -ex
+# SPDX-License-Identifier: GPL-2.0
+
+if [ -z "$origin_master" ]; then
+    origin_master="origin/master"
+fi
+
+UPSTREAM=git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
+LKL=github.com:lkl/linux.git
+
+upstream=`git remote -v | grep $UPSTREAM | cut -f1 | head -n1`
+lkl=`git remote -v | grep $LKL | cut -f1 | head -n1`
+
+if [ -z "$upstream" ]; then
+    git fetch --tags --progress git://$UPSTREAM
+else
+    git fetch --tags $upstream
+fi
+
+if [ -z "$lkl" ]; then
+    git remote add lkl-upstream git@$LKL || true
+    lkl=`git remote -v | grep $LKL | cut -f1 | head -n1`
+fi
+
+if [ -z "$lkl" ]; then
+    echo "can't find lkl remote, quiting"
+    exit 1
+fi
+
+git fetch $lkl
+git fetch --tags $upstream
+
+# find the last upstream tag to avoid checking upstream commits during
+# upstream merges
+tag=`git tag --sort='-*authordate' | grep ^v | head -n1`
+tmp=`mktemp -d`
+
+commits=$(git log --no-merges --pretty=format:%h HEAD ^$lkl/master ^$tag)
+for c in $commits; do
+    git format-patch -1 -o $tmp $c
+done
+
+if [ -z "$c" ]; then
+    echo "there are not commits/patches to check, quiting."
+    rmdir $tmp
+    exit 0
+fi
+
+./scripts/checkpatch.pl --ignore FILE_PATH_CHANGES $tmp/*.patch
+rm $tmp/*.patch
+
+# checkpatch.pl does not know how to deal with 3 way diffs which would
+# be useful to check the conflict resolutions during merges...
+#for c in `git log --merges --pretty=format:%h HEAD ^$origin_master ^$tag`; do
+#    git log --pretty=email $c -1 > $tmp/$c.patch
+#    git diff $c $c^1 $c^2 >> $tmp/$c.patch
+#done
+
+rmdir $tmp
+
diff --git a/tools/lkl/scripts/lkl-jenkins.sh b/tools/lkl/scripts/lkl-jenkins.sh
new file mode 100755
index 000000000000..eaadc6e90143
--- /dev/null
+++ b/tools/lkl/scripts/lkl-jenkins.sh
@@ -0,0 +1,21 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+set -e
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+basedir=$(cd $script_dir/../../..; pwd)
+
+export PATH=$PATH:/sbin
+
+build_and_test()
+{
+    cd $basedir
+    make mrproper
+    cd tools/lkl
+    make clean-conf
+    make -j4
+    make run-tests
+}
+
+build_and_test
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 23/25] um lkl: add UML network driver for lkl
  2020-03-30 14:45   ` Hajime Tazaki
@ 2020-03-30 14:45     ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Hajime Tazaki

This commit adds the use of the UML drivers of network interfaces which
should be able to use any backend configured by boot command line
parameters.  Currently vector tap backend and (obsolte) slirp and tuntap
backends are tested.

Since the UML drivers use LKL irq infrastructure, the build system only
picks required object files of UML, and LKL adds trivial glue code and
ifdefs into UML code so that existing driver codes can be used for LKL
as-is.

Here is a benchmark number with netperf (TCP_STREAM/TCP_MAERTS in Mbps)
over 10Gbps ethernet.

                      |TCP_STREAM  | TCP_MAERTS
 -------------------- --------------------------
 UMMODE_LIB (um-tap)  | 1932.71    |    8.58
 UMMODE_LIB (vec-tap) | 7565.97    | 9216.49
 UMMODE_LIB (virtio)  | 8743.91    | 9310.35
 UMMODE_KERN (um-tap) | 1819.09    |    0.84
 UMMODE_KERN (vec-tap)| 5418.51    | 9412.07

- setup

We varied client side (netperf) with different net drivers/devices.
tso,tx/rx csum are enabled if possible.

               +--docker0--+
               |           |     10GbEth
 netperf +---tap0        eth0 +==========+ eth0 +---+ netserver
 (client)                (ixgbe)          (ixgbe)

<-- Linux box (4.18.5) -->              <-- Linux box (4.17.19) -->

- summary
With vector driver over tap backend, UMMODE_LIB can reach similar results
with the one of virtio.  And outperforming to UMMODE_KERN is the result of
simplified user/kernel-space transision (just function calls rather than
interposision by ptrace&co).

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 .circleci/config.yml                    |  2 +
 arch/um/drivers/Makefile                |  2 +
 arch/um/include/asm/irq.h               |  2 +
 arch/um/kernel/Makefile                 |  4 ++
 arch/um/kernel/irq.c                    |  4 ++
 arch/um/lkl/Kconfig                     |  2 +
 arch/um/lkl/configs/lkl_defconfig       |  2 +-
 arch/um/lkl/include/asm/irq.h           |  3 ++
 arch/um/lkl/include/uapi/asm/host_ops.h |  6 +++
 arch/um/lkl/kernel/asm-offsets.c        |  1 +
 arch/um/lkl/kernel/irq.c                | 33 +++++++++++++
 arch/um/lkl/kernel/setup.c              |  4 ++
 arch/um/lkl/mm/bootmem.c                | 33 +++++++++++++
 arch/um/os-Linux/Makefile               |  5 ++
 tools/lkl/Makefile.autoconf             |  2 +
 tools/lkl/include/lkl.h                 |  9 ++++
 tools/lkl/include/lkl_host.h            |  1 +
 tools/lkl/lib/Build                     |  1 +
 tools/lkl/lib/config.c                  |  6 +++
 tools/lkl/lib/net.c                     |  8 ++++
 tools/lkl/lib/posix-host.c              |  3 ++
 tools/lkl/lib/um/Build                  |  2 +
 tools/lkl/lib/um/um_glue.c              | 39 +++++++++++++++
 tools/lkl/lib/um/um_net.c               | 30 ++++++++++++
 tools/lkl/tests/net-setup.sh            | 63 +++++++++++++++++++++++++
 tools/lkl/tests/net-test.c              | 18 +++++--
 tools/lkl/tests/net.sh                  | 45 ++++++++++++++++++
 tools/lkl/tests/run.py                  |  2 +
 28 files changed, 326 insertions(+), 6 deletions(-)
 create mode 100644 tools/lkl/lib/um/Build
 create mode 100644 tools/lkl/lib/um/um_glue.c
 create mode 100644 tools/lkl/lib/um/um_net.c

diff --git a/.circleci/config.yml b/.circleci/config.yml
index 5bdf059014c0..df1554078a4b 100644
--- a/.circleci/config.yml
+++ b/.circleci/config.yml
@@ -43,6 +43,8 @@ do_steps: &do_steps
   - run:
       name: run tests
       command: |
+        sudo apt-get update
+        sudo apt-get install -y slirp
         mkdir -p ~/junit
         make -C tools/lkl run-tests tests="--junit-dir ~/junit"
       no_output_timeout: "90m"
diff --git a/arch/um/drivers/Makefile b/arch/um/drivers/Makefile
index a290821e355c..3cce6c81079e 100644
--- a/arch/um/drivers/Makefile
+++ b/arch/um/drivers/Makefile
@@ -37,7 +37,9 @@ $(obj)/vde.o: $(obj)/vde_kern.o $(obj)/vde_user.o
 # When the above is fixed, don't forget to add this too!
 #targets += $(obj)/pcap.o
 
+ifeq ($(UMMODE),kernel)
 obj-y := stdio_console.o fd.o chan_kern.o chan_user.o line.o
+endif
 obj-$(CONFIG_SSL) += ssl.o
 obj-$(CONFIG_STDERR_CONSOLE) += stderr_console.o
 
diff --git a/arch/um/include/asm/irq.h b/arch/um/include/asm/irq.h
index 42c6205e2dc4..839b86f29b8a 100644
--- a/arch/um/include/asm/irq.h
+++ b/arch/um/include/asm/irq.h
@@ -32,6 +32,8 @@
 
 #endif
 
+#ifdef CONFIG_UMMODE_KERN
 #define NR_IRQS (LAST_IRQ + 1)
+#endif /* CONFIG_UMMODE_KERN */
 
 #endif
diff --git a/arch/um/kernel/Makefile b/arch/um/kernel/Makefile
index 5aa882011e04..78b49eb9bf81 100644
--- a/arch/um/kernel/Makefile
+++ b/arch/um/kernel/Makefile
@@ -12,12 +12,16 @@ CPPFLAGS_vmlinux.lds := -DSTART=$(LDS_START)		\
                         -DELF_ARCH=$(LDS_ELF_ARCH)	\
                         -DELF_FORMAT=$(LDS_ELF_FORMAT)	\
 			$(LDS_EXTRA)
+ifeq ($(UMMODE),kernel)
 extra-y := vmlinux.lds
 
 obj-y = config.o exec.o exitcode.o irq.o ksyms.o mem.o \
 	physmem.o process.o ptrace.o reboot.o sigio.o \
 	signal.o syscall.o sysrq.o time.o tlb.o trap.o \
 	um_arch.o umid.o maccess.o kmsg_dump.o skas/
+else ifeq ($(UMMODE),library)
+obj-y = irq.o
+endif
 
 obj-$(CONFIG_BLK_DEV_INITRD) += initrd.o
 obj-$(CONFIG_GPROF)	+= gprof_syms.o
diff --git a/arch/um/kernel/irq.c b/arch/um/kernel/irq.c
index 3577118bb4a5..26e6bd804f99 100644
--- a/arch/um/kernel/irq.c
+++ b/arch/um/kernel/irq.c
@@ -406,6 +406,7 @@ int deactivate_all_fds(void)
 	return 0;
 }
 
+#ifdef CONFIG_UMMODE_KERN
 /*
  * do_IRQ handles all normal device IRQs (the special
  * SMP cross-CPU interrupts have their own specific
@@ -420,6 +421,7 @@ unsigned int do_IRQ(int irq, struct uml_pt_regs *regs)
 	set_irq_regs(old_regs);
 	return 1;
 }
+#endif /* CONFIG_UMMODE_KERN */
 
 void um_free_irq(unsigned int irq, void *dev)
 {
@@ -446,6 +448,7 @@ int um_request_irq(unsigned int irq, int fd, int type,
 
 EXPORT_SYMBOL(um_request_irq);
 
+#ifdef CONFIG_UMMODE_KERN
 /*
  * irq_chip must define at least enable/disable and ack when
  * the edge handler is used.
@@ -597,3 +600,4 @@ unsigned long from_irq_stack(int nested)
 	return mask & ~1;
 }
 
+#endif /* CONFIG_UMMODE_KERN */
diff --git a/arch/um/lkl/Kconfig b/arch/um/lkl/Kconfig
index c72c40226509..2b6b4063045c 100644
--- a/arch/um/lkl/Kconfig
+++ b/arch/um/lkl/Kconfig
@@ -81,3 +81,5 @@ config HZ
         default 100
 
 endmenu
+
+source "arch/um/drivers/Kconfig"
diff --git a/arch/um/lkl/configs/lkl_defconfig b/arch/um/lkl/configs/lkl_defconfig
index 7050c233a17e..f2f218a17185 100644
--- a/arch/um/lkl/configs/lkl_defconfig
+++ b/arch/um/lkl/configs/lkl_defconfig
@@ -113,7 +113,7 @@ CONFIG_UML_NET=y
 CONFIG_UML_NET_TUNTAP=y
 # CONFIG_UML_NET_SLIP is not set
 # CONFIG_UML_NET_DAEMON is not set
-# CONFIG_UML_NET_VECTOR is not set
+CONFIG_UML_NET_VECTOR=y
 # CONFIG_UML_NET_VDE is not set
 # CONFIG_UML_NET_MCAST is not set
 # CONFIG_UML_NET_PCAP is not set
diff --git a/arch/um/lkl/include/asm/irq.h b/arch/um/lkl/include/asm/irq.h
index 948fc54cb76c..2ee00ffde21c 100644
--- a/arch/um/lkl/include/asm/irq.h
+++ b/arch/um/lkl/include/asm/irq.h
@@ -2,6 +2,9 @@
 #ifndef _ASM_LKL_IRQ_H
 #define _ASM_LKL_IRQ_H
 
+/* pull UML's definitions */
+#include "../../../include/asm/irq.h"
+
 #define IRQ_STATUS_BITS		(sizeof(long) * 8)
 #define NR_IRQS			((int)(IRQ_STATUS_BITS * IRQ_STATUS_BITS))
 
diff --git a/arch/um/lkl/include/uapi/asm/host_ops.h b/arch/um/lkl/include/uapi/asm/host_ops.h
index fe4382c3050a..0230885f4f64 100644
--- a/arch/um/lkl/include/uapi/asm/host_ops.h
+++ b/arch/um/lkl/include/uapi/asm/host_ops.h
@@ -17,6 +17,10 @@ struct lkl_jmp_buf {
  * These operations must be provided by a host library or by the application
  * itself.
  *
+ * @um_devices - string containg the list of UML devices in command line
+ * format. This string is appended to the kernel command line and
+ * is provided here for convenience to be implemented by the host library.
+ *
  * @print - optional operation that receives console messages
  *
  * @panic - called during a kernel panic
@@ -74,6 +78,8 @@ struct lkl_jmp_buf {
  * @jmp_buf_longjmp - perform a jump back to the saved jump buffer
  */
 struct lkl_host_operations {
+	const char *um_devices;
+
 	void (*print)(const char *str, int len);
 	void (*panic)(void);
 
diff --git a/arch/um/lkl/kernel/asm-offsets.c b/arch/um/lkl/kernel/asm-offsets.c
index 6be0763698dc..6fdca6df21a2 100644
--- a/arch/um/lkl/kernel/asm-offsets.c
+++ b/arch/um/lkl/kernel/asm-offsets.c
@@ -1,2 +1,3 @@
 // SPDX-License-Identifier: GPL-2.0
 /* Dummy asm-offsets.c file. Required by kbuild and ready to be used - hint! */
+#include <sysdep/kernel-offsets.h>
diff --git a/arch/um/lkl/kernel/irq.c b/arch/um/lkl/kernel/irq.c
index c794412f85d9..59e8572862a2 100644
--- a/arch/um/lkl/kernel/irq.c
+++ b/arch/um/lkl/kernel/irq.c
@@ -11,6 +11,11 @@
 #include <asm/host_ops.h>
 #include <asm/cpu.h>
 
+#if defined(__linux) && (defined(__i386) || defined(__x86_64))
+#include <os.h>
+#endif
+void *um_os_signal(int signum, void *handler);
+
 /*
  * To avoid much overhead we use an indirect approach: the irqs are marked using
  * a bitmap (array of longs) and a summary of the modified bits is kept in a
@@ -174,6 +179,16 @@ void arch_local_irq_restore(unsigned long flags)
 	irqs_enabled = flags;
 }
 
+#if defined(__linux) && (defined(__i386) || defined(__x86_64))
+static void sig_handler(int sig)
+{
+	if (sig != SIGIO)
+		return;
+
+	sigio_handler(sig, NULL, NULL);
+}
+#endif
+
 void init_IRQ(void)
 {
 	int i;
@@ -181,6 +196,11 @@ void init_IRQ(void)
 	for (i = 0; i < NR_IRQS; i++)
 		irq_set_chip_and_handler(i, &dummy_irq_chip, handle_simple_irq);
 
+#if defined(__linux) && (defined(__i386) || defined(__x86_64))
+	/* Initialize EPOLL Loop */
+	os_setup_epoll();
+	um_os_signal(SIGIO, sig_handler);
+#endif
 	pr_info("lkl: irqs initialized\n");
 }
 
@@ -188,3 +208,16 @@ void cpu_yield_to_irqs(void)
 {
 	cpu_relax();
 }
+
+#if defined(__linux) && (defined(__i386) || defined(__x86_64))
+unsigned int do_IRQ(int irq, struct uml_pt_regs *regs)
+{
+	/*
+	 * this might be called in signal handler, so dispatch to different
+	 * thread to avoid race
+	 */
+	set_irq_pending(irq);
+
+	return 1;
+}
+#endif
diff --git a/arch/um/lkl/kernel/setup.c b/arch/um/lkl/kernel/setup.c
index 36c199d3aa22..39b7b7c79581 100644
--- a/arch/um/lkl/kernel/setup.c
+++ b/arch/um/lkl/kernel/setup.c
@@ -61,6 +61,10 @@ int __init lkl_start_kernel(struct lkl_host_operations *ops, const char *fmt,
 	ret = vsnprintf(boot_command_line, COMMAND_LINE_SIZE, fmt, ap);
 	va_end(ap);
 
+	if (ops->um_devices)
+		strscpy(boot_command_line + ret, ops->um_devices,
+			COMMAND_LINE_SIZE - ret);
+
 	memcpy(cmd_line, boot_command_line, COMMAND_LINE_SIZE);
 
 	init_sem = lkl_ops->sem_alloc(0);
diff --git a/arch/um/lkl/mm/bootmem.c b/arch/um/lkl/mm/bootmem.c
index 39dd0d22b44e..3175dadee13f 100644
--- a/arch/um/lkl/mm/bootmem.c
+++ b/arch/um/lkl/mm/bootmem.c
@@ -64,3 +64,36 @@ void free_mem(void)
 {
 	lkl_ops->mem_free((void *)_memory_start);
 }
+
+void *uml_kmalloc(int size, int flags)
+{
+	return kmalloc(size, flags);
+}
+
+char *uml_strdup(const char *string)
+{
+	return kstrdup(string, GFP_KERNEL);
+}
+
+void free_stack(unsigned long stack, int order)
+{
+	free_pages(stack, order);
+}
+
+unsigned long alloc_stack(int order, int atomic)
+{
+	unsigned long page;
+	gfp_t flags = GFP_KERNEL;
+
+	if (atomic)
+		flags = GFP_ATOMIC;
+	page = __get_free_pages(flags, order);
+
+	return page;
+}
+
+int __cant_sleep(void)
+{
+	return in_atomic() || irqs_disabled() || in_interrupt();
+	/* Is in_interrupt() really needed? */
+}
diff --git a/arch/um/os-Linux/Makefile b/arch/um/os-Linux/Makefile
index 839915b8c31c..a74a2486e178 100644
--- a/arch/um/os-Linux/Makefile
+++ b/arch/um/os-Linux/Makefile
@@ -6,9 +6,14 @@
 # Don't instrument UML-specific code
 KCOV_INSTRUMENT                := n
 
+ifeq ($(UMMODE),kernel)
 obj-y = execvp.o file.o helper.o irq.o main.o mem.o process.o \
 	registers.o sigio.o signal.o start_up.o time.o tty.o \
 	umid.o user_syms.o util.o drivers/ skas/
+else
+obj-y = execvp.o file.o helper.o irq.o drivers/
+endif
+
 
 obj-$(CONFIG_ARCH_REUSE_HOST_VSYSCALL_AREA) += elf_aux.o
 
diff --git a/tools/lkl/Makefile.autoconf b/tools/lkl/Makefile.autoconf
index 268c367d9962..d2dcaa3b85f2 100644
--- a/tools/lkl/Makefile.autoconf
+++ b/tools/lkl/Makefile.autoconf
@@ -41,6 +41,8 @@ define posix_host
   $(if $(strip $(call find_include,archive.h)),$(call set_autoconf_var,ARCHIVE,y))
   $(if $(filter $(1),elf64-x86-64-freebsd),$(call set_autoconf_var,NEEDS_LARGP,y))
   $(if $(filter $(1),elf32-i386),$(call set_autoconf_var,I386,y))
+  $(if $(filter $(1),elf64-x86-64),$(call set_autoconf_var,UML_DEV,y))
+  $(if $(filter $(1),elf32-i386),$(call set_autoconf_var,UML_DEV,y))
 endef
 
 define do_autoconf
diff --git a/tools/lkl/include/lkl.h b/tools/lkl/include/lkl.h
index 1f4291ad9455..952d11e30868 100644
--- a/tools/lkl/include/lkl.h
+++ b/tools/lkl/include/lkl.h
@@ -731,6 +731,15 @@ struct lkl_netdev_args {
 	unsigned int offload;
 };
 
+#ifdef LKL_HOST_CONFIG_UML_DEV
+struct lkl_netdev *lkl_um_netdev_create(const char *ifparams);
+#else
+static inline struct lkl_netdev *lkl_um_netdev_create(const char *ifparams)
+{
+	return NULL;
+}
+#endif
+
 /*
  * lkl_register_dbg_handler- register a signal handler that loads a debug lib.
  *
diff --git a/tools/lkl/include/lkl_host.h b/tools/lkl/include/lkl_host.h
index 4e6d6e031498..7ac30bdcaf0a 100644
--- a/tools/lkl/include/lkl_host.h
+++ b/tools/lkl/include/lkl_host.h
@@ -10,6 +10,7 @@ extern "C" {
 #include <lkl.h>
 
 extern struct lkl_host_operations lkl_host_ops;
+extern char lkl_um_devs[4096];
 
 /**
  * lkl_printf - print a message via the host print operation
diff --git a/tools/lkl/lib/Build b/tools/lkl/lib/Build
index 0e711e260a3a..ad8a4b806375 100644
--- a/tools/lkl/lib/Build
+++ b/tools/lkl/lib/Build
@@ -10,3 +10,4 @@ liblkl-y += dbg.o
 liblkl-y += dbg_handler.o
 liblkl-y += ../../perf/pmu-events/jsmn.o
 liblkl-y += config.o
+liblkl-$(LKL_HOST_CONFIG_UML_DEV) += um/
diff --git a/tools/lkl/lib/config.c b/tools/lkl/lib/config.c
index 37f8ac4d942a..db3af68446d3 100644
--- a/tools/lkl/lib/config.c
+++ b/tools/lkl/lib/config.c
@@ -470,6 +470,12 @@ static int lkl_config_netdev_create(struct lkl_config *cfg,
 	memset(&nd_args, 0, sizeof(struct lkl_netdev_args));
 
 	if (!nd && iface->iftype && iface->ifparams) {
+		if ((strcmp(iface->iftype, "um") == 0)) {
+			nd = lkl_um_netdev_create(iface->ifparams);
+			iface->nd = nd;
+			/* um_netdev doesn't use virtio device */
+			return 0;
+		}
 	}
 
 	if (nd) {
diff --git a/tools/lkl/lib/net.c b/tools/lkl/lib/net.c
index cf6894c35e46..4d4cd2382b2a 100644
--- a/tools/lkl/lib/net.c
+++ b/tools/lkl/lib/net.c
@@ -179,6 +179,14 @@ int lkl_netdev_get_ifindex(int id)
 
 	snprintf(ifr.lkl_ifr_name, sizeof(ifr.lkl_ifr_name), "eth%d", id);
 	ret = lkl_sys_ioctl(sock, LKL_SIOCGIFINDEX, (long)&ifr);
+
+	/* retry with vector device */
+	if (ret < 0) {
+		snprintf(ifr.lkl_ifr_name, sizeof(ifr.lkl_ifr_name),
+			 "vec%d",id);
+		ret = lkl_sys_ioctl(sock, LKL_SIOCGIFINDEX, (long)&ifr);
+	}
+
 	lkl_sys_close(sock);
 
 	return ret < 0 ? ret : ifr.lkl_ifr_ifindex;
diff --git a/tools/lkl/lib/posix-host.c b/tools/lkl/lib/posix-host.c
index 4be1611a8942..d6394d41e125 100644
--- a/tools/lkl/lib/posix-host.c
+++ b/tools/lkl/lib/posix-host.c
@@ -346,6 +346,9 @@ struct lkl_host_operations lkl_host_ops = {
 	.print = print,
 	.mem_alloc = (void *)malloc,
 	.mem_free = free,
+#ifdef LKL_HOST_CONFIG_UML_DEV
+	.um_devices = lkl_um_devs,
+#endif
 	.gettid = _gettid,
 	.jmp_buf_set = jmp_buf_set,
 	.jmp_buf_longjmp = jmp_buf_longjmp,
diff --git a/tools/lkl/lib/um/Build b/tools/lkl/lib/um/Build
new file mode 100644
index 000000000000..09a60975ecd6
--- /dev/null
+++ b/tools/lkl/lib/um/Build
@@ -0,0 +1,2 @@
+liblkl-y += um_glue.o
+liblkl-y += um_net.o
diff --git a/tools/lkl/lib/um/um_glue.c b/tools/lkl/lib/um/um_glue.c
new file mode 100644
index 000000000000..f59e31a706de
--- /dev/null
+++ b/tools/lkl/lib/um/um_glue.c
@@ -0,0 +1,39 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
+#include <errno.h>
+
+
+
+char lkl_um_devs[4096];
+
+/* from sigio.c */
+void maybe_sigio_broken(int fd, int read)
+{
+}
+
+/* from process.c */
+int os_getpid(void)
+{
+	return getpid();
+}
+
+
+/* from chan_kern.c */
+void free_irqs(void)
+{
+}
+
+/* from sigio.c */
+int ignore_sigio_fd(int fd)
+{
+	return 0;
+}
+
+/* new functions */
+void *um_os_signal(int signum, void *handler)
+{
+	return signal(signum, handler);
+}
diff --git a/tools/lkl/lib/um/um_net.c b/tools/lkl/lib/um/um_net.c
new file mode 100644
index 000000000000..eed2dcd9a91c
--- /dev/null
+++ b/tools/lkl/lib/um/um_net.c
@@ -0,0 +1,30 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <lkl_host.h>
+
+static int registered_net_dev_idx;
+
+struct lkl_netdev *lkl_um_netdev_create(const char *ifparams)
+{
+	struct lkl_netdev *nd;
+
+	nd = lkl_host_ops.mem_alloc(sizeof(struct lkl_netdev));
+	if (!nd)
+		return NULL;
+
+	memset(nd, 0, sizeof(struct lkl_netdev));
+
+	nd->id = registered_net_dev_idx++;
+	/* concat strings */
+	/* XXX: better vector device detection ? */
+	if (strncmp(ifparams, "trans", 5) == 0)
+		snprintf(lkl_um_devs + strlen(lkl_um_devs), sizeof(lkl_um_devs),
+			 " vec%d:%s", nd->id, ifparams);
+	else
+		snprintf(lkl_um_devs + strlen(lkl_um_devs), sizeof(lkl_um_devs),
+			 " eth%d=%s", nd->id, ifparams);
+
+	return nd;
+}
diff --git a/tools/lkl/tests/net-setup.sh b/tools/lkl/tests/net-setup.sh
index 0cfc42a30a54..7aef0afc2df9 100644
--- a/tools/lkl/tests/net-setup.sh
+++ b/tools/lkl/tests/net-setup.sh
@@ -1,6 +1,11 @@
 #!/usr/bin/env bash
 # SPDX-License-Identifier: GPL-2.0
 
+if [ -n "$LKL_HOST_CONFIG_BSD" ]; then
+TEST_TAP_IFNAME=tap
+else
+TEST_TAP_IFNAME=lkl_test_tap
+fi
 TEST_IP_NETWORK=192.168.113.0
 TEST_IP_NETMASK=24
 TEST_IP6_NETWORK=fc03::0
@@ -8,6 +13,8 @@ TEST_IP6_NETMASK=64
 TEST_MAC0="aa:bb:cc:dd:ee:ff"
 TEST_MAC1="aa:bb:cc:dd:ee:aa"
 TEST_NETSERVER_PORT=11223
+TEST_UM_SLIRP_PARMS="slirp,,`which slirp`"
+TEST_UM_VECTOR_TAP_PARMS="transport=tap,ifname=${TEST_TAP_IFNAME}0"
 
 # $1 - count
 # $2 - netcount
@@ -71,3 +78,59 @@ ip6_net_mask()
 {
     echo "$(ip6_add 0 $1)/$TEST_IP6_NETMASK"
 }
+
+tap_ifname()
+{
+    echo -n "$TEST_TAP_IFNAME${1:-0}"
+}
+
+tap_prepare()
+{
+    if [ -n "$LKL_HOST_CONFIG_ANDROID" ]; then
+        if ! lkl_test_cmd test -d /dev/net &>/dev/null; then
+            lkl_test_cmd sudo mkdir /dev/net
+            lkl_test_cmd sudo ln -s /dev/tun /dev/net/tun
+        fi
+        TAP_USER="vpn"
+        ANDROID_USER="vpn,vpn,net_admin,inet"
+        export_vars ANDROID_USER
+    else
+        TAP_USER=$USER
+    fi
+}
+
+tap_setup()
+{
+    if [ -n "$LKL_HOST_CONFIG_BSD" ]; then
+        lkl_test_cmd sudo ifconfig tap create
+        lkl_test_cmd sudo sysctl net.link.tap.up_on_open=1
+        lkl_test_cmd sudo sysctl net.link.tap.user_open=1
+        lkl_test_cmd sudo ifconfig $(tap_ifname) $(ip_host)
+        lkl_test_cmd sudo ifconfig $(tap_ifname) inet6 $(ip6_host)
+        return
+    fi
+
+    lkl_test_cmd sudo ip tuntap add dev $(tap_ifname $1) mode tap user $TAP_USER
+    lkl_test_cmd sudo ip link set dev $(tap_ifname $1) up
+    lkl_test_cmd sudo ip addr add dev $(tap_ifname $1) $(ip_host_mask $1)
+    lkl_test_cmd sudo ip -6 addr add dev $(tap_ifname $1) $(ip6_host_mask $1)
+
+    if [ -n "$LKL_HOST_CONFIG_ANDROID" ]; then
+        lkl_test_cmd sudo ip route add $(ip_net_mask $1) \
+                     dev $(tap_ifname $1) proto kernel scope link \
+                     src $(ip_host $1) table local
+        lkl_test_cmd sudo ip -6 route add $(ip6_net_mask $1) \
+                     dev $(tap_ifname $1) table local
+    fi
+}
+
+tap_cleanup()
+{
+    if [ -n "$LKL_HOST_CONFIG_BSD" ]; then
+        lkl_test_cmd sudo ifconfig $(tap_ifname) destroy
+        return
+    fi
+
+    lkl_test_cmd sudo ip link set dev $(tap_ifname $1) down
+    lkl_test_cmd sudo ip tuntap del dev $(tap_ifname $1) mode tap
+}
diff --git a/tools/lkl/tests/net-test.c b/tools/lkl/tests/net-test.c
index 5a8e39a52417..1050b0df3134 100644
--- a/tools/lkl/tests/net-test.c
+++ b/tools/lkl/tests/net-test.c
@@ -22,9 +22,11 @@
 
 enum {
 	BACKEND_NONE,
+	BACKEND_UM,
+	BACKEND_UM_VECTOR_TAP,
 };
 
-const char *backends[] = { "loopback", NULL };
+const char *backends[] = { "loopback", "um", "um-vector-tap", NULL };
 static struct {
 	int backend;
 	const char *ifname;
@@ -176,12 +178,18 @@ static int lkl_test_icmp(void)
 }
 
 static struct lkl_netdev *nd;
+static int nd_id;
 
 static int lkl_test_nd_create(void)
 {
 	switch (cla.backend) {
 	case BACKEND_NONE:
 		return TEST_SKIP;
+	case BACKEND_UM:
+	case BACKEND_UM_VECTOR_TAP:
+		nd = lkl_um_netdev_create(cla.ifname);
+		nd_id = nd->id;
+		break;
 	}
 
 	if (!nd) {
@@ -192,11 +200,10 @@ static int lkl_test_nd_create(void)
 	return TEST_SUCCESS;
 }
 
-static int nd_id;
-
 static int lkl_test_nd_add(void)
 {
-	if (cla.backend == BACKEND_NONE)
+	if (cla.backend == BACKEND_NONE || cla.backend == BACKEND_UM
+	    || cla.backend == BACKEND_UM_VECTOR_TAP)
 		return TEST_SKIP;
 
 	return TEST_SUCCESS;
@@ -204,7 +211,8 @@ static int lkl_test_nd_add(void)
 
 static int lkl_test_nd_remove(void)
 {
-	if (cla.backend == BACKEND_NONE)
+	if (cla.backend == BACKEND_NONE || cla.backend == BACKEND_UM
+	    || cla.backend == BACKEND_UM_VECTOR_TAP)
 		return TEST_SKIP;
 
 	return TEST_SUCCESS;
diff --git a/tools/lkl/tests/net.sh b/tools/lkl/tests/net.sh
index 32e8452b0406..d964c6a2321d 100755
--- a/tools/lkl/tests/net.sh
+++ b/tools/lkl/tests/net.sh
@@ -11,8 +11,17 @@ cleanup_backend()
     set -e
 
     case "$1" in
+    "um-vector-tap")
+	# only intel arch is capable with um-net backent
+	if [ -z "$LKL_HOST_CONFIG_UML_DEV" ]; then
+            return $TEST_SKIP
+	fi
+        tap_cleanup
+        ;;
     "loopback")
         ;;
+    "um")
+        ;;
     esac
 }
 
@@ -47,6 +56,30 @@ setup_backend()
     case "$1" in
     "loopback")
         ;;
+    "um")
+	# only intel arch is capable with um-net backent
+	if [ -z "$LKL_HOST_CONFIG_UML_DEV" ]; then
+            return $TEST_SKIP
+	fi
+	# slirp's helper process doesn't work with valgrind
+	if [ -n "$VALGRIND" ]; then
+            return $TEST_SKIP
+	fi
+        ;;
+    "um-vector-tap")
+	# only intel arch is capable with um-net backent
+	if [ -z "$LKL_HOST_CONFIG_UML_DEV" ]; then
+            return $TEST_SKIP
+	fi
+        tap_prepare
+        if ! lkl_test_cmd test -c /dev/net/tun; then
+            if [ -z "$LKL_HOST_CONFIG_BSD" ]; then
+                echo "missing /dev/net/tun"
+                return $TEST_SKIP
+            fi
+        fi
+        tap_setup
+        ;;
     *)
         echo "don't know how to setup backend $1"
         return $TEST_FAILED
@@ -60,6 +93,18 @@ run_tests()
     "loopback")
         lkl_test_exec $script_dir/net-test --dst 127.0.0.1
         ;;
+    "um")
+        lkl_test_exec $script_dir/net-test --backend um \
+                      --ifname $TEST_UM_SLIRP_PARMS \
+                      --ip 10.0.2.15 --netmask-len 8 \
+                      --dst 10.0.2.2
+        ;;
+    "um-vector-tap")
+        lkl_test_exec $script_dir/net-test --backend um-vector-tap \
+                      --ifname $TEST_UM_VECTOR_TAP_PARMS \
+                      --ip $(ip_lkl) --netmask-len $TEST_IP_NETMASK \
+                      --dst $(ip_host)
+        ;;
     esac
 }
 
diff --git a/tools/lkl/tests/run.py b/tools/lkl/tests/run.py
index b72299aaabee..2cc918a6e11a 100755
--- a/tools/lkl/tests/run.py
+++ b/tools/lkl/tests/run.py
@@ -54,6 +54,8 @@ tests = [
     'disk.sh -t vfat',
     'disk.sh -t btrfs',
     'net.sh -b loopback',
+    'net.sh -b um',
+    'net.sh -b um-vector-tap',
     'lklfuse.sh -t ext4',
     'lklfuse.sh -t vfat',
     'lklfuse.sh -t btrfs',
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 23/25] um lkl: add UML network driver for lkl
@ 2020-03-30 14:45     ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, linux-kernel-library, linux-arch,
	Hajime Tazaki, Akira Moroo

This commit adds the use of the UML drivers of network interfaces which
should be able to use any backend configured by boot command line
parameters.  Currently vector tap backend and (obsolte) slirp and tuntap
backends are tested.

Since the UML drivers use LKL irq infrastructure, the build system only
picks required object files of UML, and LKL adds trivial glue code and
ifdefs into UML code so that existing driver codes can be used for LKL
as-is.

Here is a benchmark number with netperf (TCP_STREAM/TCP_MAERTS in Mbps)
over 10Gbps ethernet.

                      |TCP_STREAM  | TCP_MAERTS
 -------------------- --------------------------
 UMMODE_LIB (um-tap)  | 1932.71    |    8.58
 UMMODE_LIB (vec-tap) | 7565.97    | 9216.49
 UMMODE_LIB (virtio)  | 8743.91    | 9310.35
 UMMODE_KERN (um-tap) | 1819.09    |    0.84
 UMMODE_KERN (vec-tap)| 5418.51    | 9412.07

- setup

We varied client side (netperf) with different net drivers/devices.
tso,tx/rx csum are enabled if possible.

               +--docker0--+
               |           |     10GbEth
 netperf +---tap0        eth0 +==========+ eth0 +---+ netserver
 (client)                (ixgbe)          (ixgbe)

<-- Linux box (4.18.5) -->              <-- Linux box (4.17.19) -->

- summary
With vector driver over tap backend, UMMODE_LIB can reach similar results
with the one of virtio.  And outperforming to UMMODE_KERN is the result of
simplified user/kernel-space transision (just function calls rather than
interposision by ptrace&co).

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 .circleci/config.yml                    |  2 +
 arch/um/drivers/Makefile                |  2 +
 arch/um/include/asm/irq.h               |  2 +
 arch/um/kernel/Makefile                 |  4 ++
 arch/um/kernel/irq.c                    |  4 ++
 arch/um/lkl/Kconfig                     |  2 +
 arch/um/lkl/configs/lkl_defconfig       |  2 +-
 arch/um/lkl/include/asm/irq.h           |  3 ++
 arch/um/lkl/include/uapi/asm/host_ops.h |  6 +++
 arch/um/lkl/kernel/asm-offsets.c        |  1 +
 arch/um/lkl/kernel/irq.c                | 33 +++++++++++++
 arch/um/lkl/kernel/setup.c              |  4 ++
 arch/um/lkl/mm/bootmem.c                | 33 +++++++++++++
 arch/um/os-Linux/Makefile               |  5 ++
 tools/lkl/Makefile.autoconf             |  2 +
 tools/lkl/include/lkl.h                 |  9 ++++
 tools/lkl/include/lkl_host.h            |  1 +
 tools/lkl/lib/Build                     |  1 +
 tools/lkl/lib/config.c                  |  6 +++
 tools/lkl/lib/net.c                     |  8 ++++
 tools/lkl/lib/posix-host.c              |  3 ++
 tools/lkl/lib/um/Build                  |  2 +
 tools/lkl/lib/um/um_glue.c              | 39 +++++++++++++++
 tools/lkl/lib/um/um_net.c               | 30 ++++++++++++
 tools/lkl/tests/net-setup.sh            | 63 +++++++++++++++++++++++++
 tools/lkl/tests/net-test.c              | 18 +++++--
 tools/lkl/tests/net.sh                  | 45 ++++++++++++++++++
 tools/lkl/tests/run.py                  |  2 +
 28 files changed, 326 insertions(+), 6 deletions(-)
 create mode 100644 tools/lkl/lib/um/Build
 create mode 100644 tools/lkl/lib/um/um_glue.c
 create mode 100644 tools/lkl/lib/um/um_net.c

diff --git a/.circleci/config.yml b/.circleci/config.yml
index 5bdf059014c0..df1554078a4b 100644
--- a/.circleci/config.yml
+++ b/.circleci/config.yml
@@ -43,6 +43,8 @@ do_steps: &do_steps
   - run:
       name: run tests
       command: |
+        sudo apt-get update
+        sudo apt-get install -y slirp
         mkdir -p ~/junit
         make -C tools/lkl run-tests tests="--junit-dir ~/junit"
       no_output_timeout: "90m"
diff --git a/arch/um/drivers/Makefile b/arch/um/drivers/Makefile
index a290821e355c..3cce6c81079e 100644
--- a/arch/um/drivers/Makefile
+++ b/arch/um/drivers/Makefile
@@ -37,7 +37,9 @@ $(obj)/vde.o: $(obj)/vde_kern.o $(obj)/vde_user.o
 # When the above is fixed, don't forget to add this too!
 #targets += $(obj)/pcap.o
 
+ifeq ($(UMMODE),kernel)
 obj-y := stdio_console.o fd.o chan_kern.o chan_user.o line.o
+endif
 obj-$(CONFIG_SSL) += ssl.o
 obj-$(CONFIG_STDERR_CONSOLE) += stderr_console.o
 
diff --git a/arch/um/include/asm/irq.h b/arch/um/include/asm/irq.h
index 42c6205e2dc4..839b86f29b8a 100644
--- a/arch/um/include/asm/irq.h
+++ b/arch/um/include/asm/irq.h
@@ -32,6 +32,8 @@
 
 #endif
 
+#ifdef CONFIG_UMMODE_KERN
 #define NR_IRQS (LAST_IRQ + 1)
+#endif /* CONFIG_UMMODE_KERN */
 
 #endif
diff --git a/arch/um/kernel/Makefile b/arch/um/kernel/Makefile
index 5aa882011e04..78b49eb9bf81 100644
--- a/arch/um/kernel/Makefile
+++ b/arch/um/kernel/Makefile
@@ -12,12 +12,16 @@ CPPFLAGS_vmlinux.lds := -DSTART=$(LDS_START)		\
                         -DELF_ARCH=$(LDS_ELF_ARCH)	\
                         -DELF_FORMAT=$(LDS_ELF_FORMAT)	\
 			$(LDS_EXTRA)
+ifeq ($(UMMODE),kernel)
 extra-y := vmlinux.lds
 
 obj-y = config.o exec.o exitcode.o irq.o ksyms.o mem.o \
 	physmem.o process.o ptrace.o reboot.o sigio.o \
 	signal.o syscall.o sysrq.o time.o tlb.o trap.o \
 	um_arch.o umid.o maccess.o kmsg_dump.o skas/
+else ifeq ($(UMMODE),library)
+obj-y = irq.o
+endif
 
 obj-$(CONFIG_BLK_DEV_INITRD) += initrd.o
 obj-$(CONFIG_GPROF)	+= gprof_syms.o
diff --git a/arch/um/kernel/irq.c b/arch/um/kernel/irq.c
index 3577118bb4a5..26e6bd804f99 100644
--- a/arch/um/kernel/irq.c
+++ b/arch/um/kernel/irq.c
@@ -406,6 +406,7 @@ int deactivate_all_fds(void)
 	return 0;
 }
 
+#ifdef CONFIG_UMMODE_KERN
 /*
  * do_IRQ handles all normal device IRQs (the special
  * SMP cross-CPU interrupts have their own specific
@@ -420,6 +421,7 @@ unsigned int do_IRQ(int irq, struct uml_pt_regs *regs)
 	set_irq_regs(old_regs);
 	return 1;
 }
+#endif /* CONFIG_UMMODE_KERN */
 
 void um_free_irq(unsigned int irq, void *dev)
 {
@@ -446,6 +448,7 @@ int um_request_irq(unsigned int irq, int fd, int type,
 
 EXPORT_SYMBOL(um_request_irq);
 
+#ifdef CONFIG_UMMODE_KERN
 /*
  * irq_chip must define at least enable/disable and ack when
  * the edge handler is used.
@@ -597,3 +600,4 @@ unsigned long from_irq_stack(int nested)
 	return mask & ~1;
 }
 
+#endif /* CONFIG_UMMODE_KERN */
diff --git a/arch/um/lkl/Kconfig b/arch/um/lkl/Kconfig
index c72c40226509..2b6b4063045c 100644
--- a/arch/um/lkl/Kconfig
+++ b/arch/um/lkl/Kconfig
@@ -81,3 +81,5 @@ config HZ
         default 100
 
 endmenu
+
+source "arch/um/drivers/Kconfig"
diff --git a/arch/um/lkl/configs/lkl_defconfig b/arch/um/lkl/configs/lkl_defconfig
index 7050c233a17e..f2f218a17185 100644
--- a/arch/um/lkl/configs/lkl_defconfig
+++ b/arch/um/lkl/configs/lkl_defconfig
@@ -113,7 +113,7 @@ CONFIG_UML_NET=y
 CONFIG_UML_NET_TUNTAP=y
 # CONFIG_UML_NET_SLIP is not set
 # CONFIG_UML_NET_DAEMON is not set
-# CONFIG_UML_NET_VECTOR is not set
+CONFIG_UML_NET_VECTOR=y
 # CONFIG_UML_NET_VDE is not set
 # CONFIG_UML_NET_MCAST is not set
 # CONFIG_UML_NET_PCAP is not set
diff --git a/arch/um/lkl/include/asm/irq.h b/arch/um/lkl/include/asm/irq.h
index 948fc54cb76c..2ee00ffde21c 100644
--- a/arch/um/lkl/include/asm/irq.h
+++ b/arch/um/lkl/include/asm/irq.h
@@ -2,6 +2,9 @@
 #ifndef _ASM_LKL_IRQ_H
 #define _ASM_LKL_IRQ_H
 
+/* pull UML's definitions */
+#include "../../../include/asm/irq.h"
+
 #define IRQ_STATUS_BITS		(sizeof(long) * 8)
 #define NR_IRQS			((int)(IRQ_STATUS_BITS * IRQ_STATUS_BITS))
 
diff --git a/arch/um/lkl/include/uapi/asm/host_ops.h b/arch/um/lkl/include/uapi/asm/host_ops.h
index fe4382c3050a..0230885f4f64 100644
--- a/arch/um/lkl/include/uapi/asm/host_ops.h
+++ b/arch/um/lkl/include/uapi/asm/host_ops.h
@@ -17,6 +17,10 @@ struct lkl_jmp_buf {
  * These operations must be provided by a host library or by the application
  * itself.
  *
+ * @um_devices - string containg the list of UML devices in command line
+ * format. This string is appended to the kernel command line and
+ * is provided here for convenience to be implemented by the host library.
+ *
  * @print - optional operation that receives console messages
  *
  * @panic - called during a kernel panic
@@ -74,6 +78,8 @@ struct lkl_jmp_buf {
  * @jmp_buf_longjmp - perform a jump back to the saved jump buffer
  */
 struct lkl_host_operations {
+	const char *um_devices;
+
 	void (*print)(const char *str, int len);
 	void (*panic)(void);
 
diff --git a/arch/um/lkl/kernel/asm-offsets.c b/arch/um/lkl/kernel/asm-offsets.c
index 6be0763698dc..6fdca6df21a2 100644
--- a/arch/um/lkl/kernel/asm-offsets.c
+++ b/arch/um/lkl/kernel/asm-offsets.c
@@ -1,2 +1,3 @@
 // SPDX-License-Identifier: GPL-2.0
 /* Dummy asm-offsets.c file. Required by kbuild and ready to be used - hint! */
+#include <sysdep/kernel-offsets.h>
diff --git a/arch/um/lkl/kernel/irq.c b/arch/um/lkl/kernel/irq.c
index c794412f85d9..59e8572862a2 100644
--- a/arch/um/lkl/kernel/irq.c
+++ b/arch/um/lkl/kernel/irq.c
@@ -11,6 +11,11 @@
 #include <asm/host_ops.h>
 #include <asm/cpu.h>
 
+#if defined(__linux) && (defined(__i386) || defined(__x86_64))
+#include <os.h>
+#endif
+void *um_os_signal(int signum, void *handler);
+
 /*
  * To avoid much overhead we use an indirect approach: the irqs are marked using
  * a bitmap (array of longs) and a summary of the modified bits is kept in a
@@ -174,6 +179,16 @@ void arch_local_irq_restore(unsigned long flags)
 	irqs_enabled = flags;
 }
 
+#if defined(__linux) && (defined(__i386) || defined(__x86_64))
+static void sig_handler(int sig)
+{
+	if (sig != SIGIO)
+		return;
+
+	sigio_handler(sig, NULL, NULL);
+}
+#endif
+
 void init_IRQ(void)
 {
 	int i;
@@ -181,6 +196,11 @@ void init_IRQ(void)
 	for (i = 0; i < NR_IRQS; i++)
 		irq_set_chip_and_handler(i, &dummy_irq_chip, handle_simple_irq);
 
+#if defined(__linux) && (defined(__i386) || defined(__x86_64))
+	/* Initialize EPOLL Loop */
+	os_setup_epoll();
+	um_os_signal(SIGIO, sig_handler);
+#endif
 	pr_info("lkl: irqs initialized\n");
 }
 
@@ -188,3 +208,16 @@ void cpu_yield_to_irqs(void)
 {
 	cpu_relax();
 }
+
+#if defined(__linux) && (defined(__i386) || defined(__x86_64))
+unsigned int do_IRQ(int irq, struct uml_pt_regs *regs)
+{
+	/*
+	 * this might be called in signal handler, so dispatch to different
+	 * thread to avoid race
+	 */
+	set_irq_pending(irq);
+
+	return 1;
+}
+#endif
diff --git a/arch/um/lkl/kernel/setup.c b/arch/um/lkl/kernel/setup.c
index 36c199d3aa22..39b7b7c79581 100644
--- a/arch/um/lkl/kernel/setup.c
+++ b/arch/um/lkl/kernel/setup.c
@@ -61,6 +61,10 @@ int __init lkl_start_kernel(struct lkl_host_operations *ops, const char *fmt,
 	ret = vsnprintf(boot_command_line, COMMAND_LINE_SIZE, fmt, ap);
 	va_end(ap);
 
+	if (ops->um_devices)
+		strscpy(boot_command_line + ret, ops->um_devices,
+			COMMAND_LINE_SIZE - ret);
+
 	memcpy(cmd_line, boot_command_line, COMMAND_LINE_SIZE);
 
 	init_sem = lkl_ops->sem_alloc(0);
diff --git a/arch/um/lkl/mm/bootmem.c b/arch/um/lkl/mm/bootmem.c
index 39dd0d22b44e..3175dadee13f 100644
--- a/arch/um/lkl/mm/bootmem.c
+++ b/arch/um/lkl/mm/bootmem.c
@@ -64,3 +64,36 @@ void free_mem(void)
 {
 	lkl_ops->mem_free((void *)_memory_start);
 }
+
+void *uml_kmalloc(int size, int flags)
+{
+	return kmalloc(size, flags);
+}
+
+char *uml_strdup(const char *string)
+{
+	return kstrdup(string, GFP_KERNEL);
+}
+
+void free_stack(unsigned long stack, int order)
+{
+	free_pages(stack, order);
+}
+
+unsigned long alloc_stack(int order, int atomic)
+{
+	unsigned long page;
+	gfp_t flags = GFP_KERNEL;
+
+	if (atomic)
+		flags = GFP_ATOMIC;
+	page = __get_free_pages(flags, order);
+
+	return page;
+}
+
+int __cant_sleep(void)
+{
+	return in_atomic() || irqs_disabled() || in_interrupt();
+	/* Is in_interrupt() really needed? */
+}
diff --git a/arch/um/os-Linux/Makefile b/arch/um/os-Linux/Makefile
index 839915b8c31c..a74a2486e178 100644
--- a/arch/um/os-Linux/Makefile
+++ b/arch/um/os-Linux/Makefile
@@ -6,9 +6,14 @@
 # Don't instrument UML-specific code
 KCOV_INSTRUMENT                := n
 
+ifeq ($(UMMODE),kernel)
 obj-y = execvp.o file.o helper.o irq.o main.o mem.o process.o \
 	registers.o sigio.o signal.o start_up.o time.o tty.o \
 	umid.o user_syms.o util.o drivers/ skas/
+else
+obj-y = execvp.o file.o helper.o irq.o drivers/
+endif
+
 
 obj-$(CONFIG_ARCH_REUSE_HOST_VSYSCALL_AREA) += elf_aux.o
 
diff --git a/tools/lkl/Makefile.autoconf b/tools/lkl/Makefile.autoconf
index 268c367d9962..d2dcaa3b85f2 100644
--- a/tools/lkl/Makefile.autoconf
+++ b/tools/lkl/Makefile.autoconf
@@ -41,6 +41,8 @@ define posix_host
   $(if $(strip $(call find_include,archive.h)),$(call set_autoconf_var,ARCHIVE,y))
   $(if $(filter $(1),elf64-x86-64-freebsd),$(call set_autoconf_var,NEEDS_LARGP,y))
   $(if $(filter $(1),elf32-i386),$(call set_autoconf_var,I386,y))
+  $(if $(filter $(1),elf64-x86-64),$(call set_autoconf_var,UML_DEV,y))
+  $(if $(filter $(1),elf32-i386),$(call set_autoconf_var,UML_DEV,y))
 endef
 
 define do_autoconf
diff --git a/tools/lkl/include/lkl.h b/tools/lkl/include/lkl.h
index 1f4291ad9455..952d11e30868 100644
--- a/tools/lkl/include/lkl.h
+++ b/tools/lkl/include/lkl.h
@@ -731,6 +731,15 @@ struct lkl_netdev_args {
 	unsigned int offload;
 };
 
+#ifdef LKL_HOST_CONFIG_UML_DEV
+struct lkl_netdev *lkl_um_netdev_create(const char *ifparams);
+#else
+static inline struct lkl_netdev *lkl_um_netdev_create(const char *ifparams)
+{
+	return NULL;
+}
+#endif
+
 /*
  * lkl_register_dbg_handler- register a signal handler that loads a debug lib.
  *
diff --git a/tools/lkl/include/lkl_host.h b/tools/lkl/include/lkl_host.h
index 4e6d6e031498..7ac30bdcaf0a 100644
--- a/tools/lkl/include/lkl_host.h
+++ b/tools/lkl/include/lkl_host.h
@@ -10,6 +10,7 @@ extern "C" {
 #include <lkl.h>
 
 extern struct lkl_host_operations lkl_host_ops;
+extern char lkl_um_devs[4096];
 
 /**
  * lkl_printf - print a message via the host print operation
diff --git a/tools/lkl/lib/Build b/tools/lkl/lib/Build
index 0e711e260a3a..ad8a4b806375 100644
--- a/tools/lkl/lib/Build
+++ b/tools/lkl/lib/Build
@@ -10,3 +10,4 @@ liblkl-y += dbg.o
 liblkl-y += dbg_handler.o
 liblkl-y += ../../perf/pmu-events/jsmn.o
 liblkl-y += config.o
+liblkl-$(LKL_HOST_CONFIG_UML_DEV) += um/
diff --git a/tools/lkl/lib/config.c b/tools/lkl/lib/config.c
index 37f8ac4d942a..db3af68446d3 100644
--- a/tools/lkl/lib/config.c
+++ b/tools/lkl/lib/config.c
@@ -470,6 +470,12 @@ static int lkl_config_netdev_create(struct lkl_config *cfg,
 	memset(&nd_args, 0, sizeof(struct lkl_netdev_args));
 
 	if (!nd && iface->iftype && iface->ifparams) {
+		if ((strcmp(iface->iftype, "um") == 0)) {
+			nd = lkl_um_netdev_create(iface->ifparams);
+			iface->nd = nd;
+			/* um_netdev doesn't use virtio device */
+			return 0;
+		}
 	}
 
 	if (nd) {
diff --git a/tools/lkl/lib/net.c b/tools/lkl/lib/net.c
index cf6894c35e46..4d4cd2382b2a 100644
--- a/tools/lkl/lib/net.c
+++ b/tools/lkl/lib/net.c
@@ -179,6 +179,14 @@ int lkl_netdev_get_ifindex(int id)
 
 	snprintf(ifr.lkl_ifr_name, sizeof(ifr.lkl_ifr_name), "eth%d", id);
 	ret = lkl_sys_ioctl(sock, LKL_SIOCGIFINDEX, (long)&ifr);
+
+	/* retry with vector device */
+	if (ret < 0) {
+		snprintf(ifr.lkl_ifr_name, sizeof(ifr.lkl_ifr_name),
+			 "vec%d",id);
+		ret = lkl_sys_ioctl(sock, LKL_SIOCGIFINDEX, (long)&ifr);
+	}
+
 	lkl_sys_close(sock);
 
 	return ret < 0 ? ret : ifr.lkl_ifr_ifindex;
diff --git a/tools/lkl/lib/posix-host.c b/tools/lkl/lib/posix-host.c
index 4be1611a8942..d6394d41e125 100644
--- a/tools/lkl/lib/posix-host.c
+++ b/tools/lkl/lib/posix-host.c
@@ -346,6 +346,9 @@ struct lkl_host_operations lkl_host_ops = {
 	.print = print,
 	.mem_alloc = (void *)malloc,
 	.mem_free = free,
+#ifdef LKL_HOST_CONFIG_UML_DEV
+	.um_devices = lkl_um_devs,
+#endif
 	.gettid = _gettid,
 	.jmp_buf_set = jmp_buf_set,
 	.jmp_buf_longjmp = jmp_buf_longjmp,
diff --git a/tools/lkl/lib/um/Build b/tools/lkl/lib/um/Build
new file mode 100644
index 000000000000..09a60975ecd6
--- /dev/null
+++ b/tools/lkl/lib/um/Build
@@ -0,0 +1,2 @@
+liblkl-y += um_glue.o
+liblkl-y += um_net.o
diff --git a/tools/lkl/lib/um/um_glue.c b/tools/lkl/lib/um/um_glue.c
new file mode 100644
index 000000000000..f59e31a706de
--- /dev/null
+++ b/tools/lkl/lib/um/um_glue.c
@@ -0,0 +1,39 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
+#include <errno.h>
+
+
+
+char lkl_um_devs[4096];
+
+/* from sigio.c */
+void maybe_sigio_broken(int fd, int read)
+{
+}
+
+/* from process.c */
+int os_getpid(void)
+{
+	return getpid();
+}
+
+
+/* from chan_kern.c */
+void free_irqs(void)
+{
+}
+
+/* from sigio.c */
+int ignore_sigio_fd(int fd)
+{
+	return 0;
+}
+
+/* new functions */
+void *um_os_signal(int signum, void *handler)
+{
+	return signal(signum, handler);
+}
diff --git a/tools/lkl/lib/um/um_net.c b/tools/lkl/lib/um/um_net.c
new file mode 100644
index 000000000000..eed2dcd9a91c
--- /dev/null
+++ b/tools/lkl/lib/um/um_net.c
@@ -0,0 +1,30 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <lkl_host.h>
+
+static int registered_net_dev_idx;
+
+struct lkl_netdev *lkl_um_netdev_create(const char *ifparams)
+{
+	struct lkl_netdev *nd;
+
+	nd = lkl_host_ops.mem_alloc(sizeof(struct lkl_netdev));
+	if (!nd)
+		return NULL;
+
+	memset(nd, 0, sizeof(struct lkl_netdev));
+
+	nd->id = registered_net_dev_idx++;
+	/* concat strings */
+	/* XXX: better vector device detection ? */
+	if (strncmp(ifparams, "trans", 5) == 0)
+		snprintf(lkl_um_devs + strlen(lkl_um_devs), sizeof(lkl_um_devs),
+			 " vec%d:%s", nd->id, ifparams);
+	else
+		snprintf(lkl_um_devs + strlen(lkl_um_devs), sizeof(lkl_um_devs),
+			 " eth%d=%s", nd->id, ifparams);
+
+	return nd;
+}
diff --git a/tools/lkl/tests/net-setup.sh b/tools/lkl/tests/net-setup.sh
index 0cfc42a30a54..7aef0afc2df9 100644
--- a/tools/lkl/tests/net-setup.sh
+++ b/tools/lkl/tests/net-setup.sh
@@ -1,6 +1,11 @@
 #!/usr/bin/env bash
 # SPDX-License-Identifier: GPL-2.0
 
+if [ -n "$LKL_HOST_CONFIG_BSD" ]; then
+TEST_TAP_IFNAME=tap
+else
+TEST_TAP_IFNAME=lkl_test_tap
+fi
 TEST_IP_NETWORK=192.168.113.0
 TEST_IP_NETMASK=24
 TEST_IP6_NETWORK=fc03::0
@@ -8,6 +13,8 @@ TEST_IP6_NETMASK=64
 TEST_MAC0="aa:bb:cc:dd:ee:ff"
 TEST_MAC1="aa:bb:cc:dd:ee:aa"
 TEST_NETSERVER_PORT=11223
+TEST_UM_SLIRP_PARMS="slirp,,`which slirp`"
+TEST_UM_VECTOR_TAP_PARMS="transport=tap,ifname=${TEST_TAP_IFNAME}0"
 
 # $1 - count
 # $2 - netcount
@@ -71,3 +78,59 @@ ip6_net_mask()
 {
     echo "$(ip6_add 0 $1)/$TEST_IP6_NETMASK"
 }
+
+tap_ifname()
+{
+    echo -n "$TEST_TAP_IFNAME${1:-0}"
+}
+
+tap_prepare()
+{
+    if [ -n "$LKL_HOST_CONFIG_ANDROID" ]; then
+        if ! lkl_test_cmd test -d /dev/net &>/dev/null; then
+            lkl_test_cmd sudo mkdir /dev/net
+            lkl_test_cmd sudo ln -s /dev/tun /dev/net/tun
+        fi
+        TAP_USER="vpn"
+        ANDROID_USER="vpn,vpn,net_admin,inet"
+        export_vars ANDROID_USER
+    else
+        TAP_USER=$USER
+    fi
+}
+
+tap_setup()
+{
+    if [ -n "$LKL_HOST_CONFIG_BSD" ]; then
+        lkl_test_cmd sudo ifconfig tap create
+        lkl_test_cmd sudo sysctl net.link.tap.up_on_open=1
+        lkl_test_cmd sudo sysctl net.link.tap.user_open=1
+        lkl_test_cmd sudo ifconfig $(tap_ifname) $(ip_host)
+        lkl_test_cmd sudo ifconfig $(tap_ifname) inet6 $(ip6_host)
+        return
+    fi
+
+    lkl_test_cmd sudo ip tuntap add dev $(tap_ifname $1) mode tap user $TAP_USER
+    lkl_test_cmd sudo ip link set dev $(tap_ifname $1) up
+    lkl_test_cmd sudo ip addr add dev $(tap_ifname $1) $(ip_host_mask $1)
+    lkl_test_cmd sudo ip -6 addr add dev $(tap_ifname $1) $(ip6_host_mask $1)
+
+    if [ -n "$LKL_HOST_CONFIG_ANDROID" ]; then
+        lkl_test_cmd sudo ip route add $(ip_net_mask $1) \
+                     dev $(tap_ifname $1) proto kernel scope link \
+                     src $(ip_host $1) table local
+        lkl_test_cmd sudo ip -6 route add $(ip6_net_mask $1) \
+                     dev $(tap_ifname $1) table local
+    fi
+}
+
+tap_cleanup()
+{
+    if [ -n "$LKL_HOST_CONFIG_BSD" ]; then
+        lkl_test_cmd sudo ifconfig $(tap_ifname) destroy
+        return
+    fi
+
+    lkl_test_cmd sudo ip link set dev $(tap_ifname $1) down
+    lkl_test_cmd sudo ip tuntap del dev $(tap_ifname $1) mode tap
+}
diff --git a/tools/lkl/tests/net-test.c b/tools/lkl/tests/net-test.c
index 5a8e39a52417..1050b0df3134 100644
--- a/tools/lkl/tests/net-test.c
+++ b/tools/lkl/tests/net-test.c
@@ -22,9 +22,11 @@
 
 enum {
 	BACKEND_NONE,
+	BACKEND_UM,
+	BACKEND_UM_VECTOR_TAP,
 };
 
-const char *backends[] = { "loopback", NULL };
+const char *backends[] = { "loopback", "um", "um-vector-tap", NULL };
 static struct {
 	int backend;
 	const char *ifname;
@@ -176,12 +178,18 @@ static int lkl_test_icmp(void)
 }
 
 static struct lkl_netdev *nd;
+static int nd_id;
 
 static int lkl_test_nd_create(void)
 {
 	switch (cla.backend) {
 	case BACKEND_NONE:
 		return TEST_SKIP;
+	case BACKEND_UM:
+	case BACKEND_UM_VECTOR_TAP:
+		nd = lkl_um_netdev_create(cla.ifname);
+		nd_id = nd->id;
+		break;
 	}
 
 	if (!nd) {
@@ -192,11 +200,10 @@ static int lkl_test_nd_create(void)
 	return TEST_SUCCESS;
 }
 
-static int nd_id;
-
 static int lkl_test_nd_add(void)
 {
-	if (cla.backend == BACKEND_NONE)
+	if (cla.backend == BACKEND_NONE || cla.backend == BACKEND_UM
+	    || cla.backend == BACKEND_UM_VECTOR_TAP)
 		return TEST_SKIP;
 
 	return TEST_SUCCESS;
@@ -204,7 +211,8 @@ static int lkl_test_nd_add(void)
 
 static int lkl_test_nd_remove(void)
 {
-	if (cla.backend == BACKEND_NONE)
+	if (cla.backend == BACKEND_NONE || cla.backend == BACKEND_UM
+	    || cla.backend == BACKEND_UM_VECTOR_TAP)
 		return TEST_SKIP;
 
 	return TEST_SUCCESS;
diff --git a/tools/lkl/tests/net.sh b/tools/lkl/tests/net.sh
index 32e8452b0406..d964c6a2321d 100755
--- a/tools/lkl/tests/net.sh
+++ b/tools/lkl/tests/net.sh
@@ -11,8 +11,17 @@ cleanup_backend()
     set -e
 
     case "$1" in
+    "um-vector-tap")
+	# only intel arch is capable with um-net backent
+	if [ -z "$LKL_HOST_CONFIG_UML_DEV" ]; then
+            return $TEST_SKIP
+	fi
+        tap_cleanup
+        ;;
     "loopback")
         ;;
+    "um")
+        ;;
     esac
 }
 
@@ -47,6 +56,30 @@ setup_backend()
     case "$1" in
     "loopback")
         ;;
+    "um")
+	# only intel arch is capable with um-net backent
+	if [ -z "$LKL_HOST_CONFIG_UML_DEV" ]; then
+            return $TEST_SKIP
+	fi
+	# slirp's helper process doesn't work with valgrind
+	if [ -n "$VALGRIND" ]; then
+            return $TEST_SKIP
+	fi
+        ;;
+    "um-vector-tap")
+	# only intel arch is capable with um-net backent
+	if [ -z "$LKL_HOST_CONFIG_UML_DEV" ]; then
+            return $TEST_SKIP
+	fi
+        tap_prepare
+        if ! lkl_test_cmd test -c /dev/net/tun; then
+            if [ -z "$LKL_HOST_CONFIG_BSD" ]; then
+                echo "missing /dev/net/tun"
+                return $TEST_SKIP
+            fi
+        fi
+        tap_setup
+        ;;
     *)
         echo "don't know how to setup backend $1"
         return $TEST_FAILED
@@ -60,6 +93,18 @@ run_tests()
     "loopback")
         lkl_test_exec $script_dir/net-test --dst 127.0.0.1
         ;;
+    "um")
+        lkl_test_exec $script_dir/net-test --backend um \
+                      --ifname $TEST_UM_SLIRP_PARMS \
+                      --ip 10.0.2.15 --netmask-len 8 \
+                      --dst 10.0.2.2
+        ;;
+    "um-vector-tap")
+        lkl_test_exec $script_dir/net-test --backend um-vector-tap \
+                      --ifname $TEST_UM_VECTOR_TAP_PARMS \
+                      --ip $(ip_lkl) --netmask-len $TEST_IP_NETMASK \
+                      --dst $(ip_host)
+        ;;
     esac
 }
 
diff --git a/tools/lkl/tests/run.py b/tools/lkl/tests/run.py
index b72299aaabee..2cc918a6e11a 100755
--- a/tools/lkl/tests/run.py
+++ b/tools/lkl/tests/run.py
@@ -54,6 +54,8 @@ tests = [
     'disk.sh -t vfat',
     'disk.sh -t btrfs',
     'net.sh -b loopback',
+    'net.sh -b um',
+    'net.sh -b um-vector-tap',
     'lklfuse.sh -t ext4',
     'lklfuse.sh -t vfat',
     'lklfuse.sh -t btrfs',
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 24/25] um lkl: add UML block device driver (ubd) for lkl
  2020-03-30 14:45   ` Hajime Tazaki
@ 2020-03-30 14:45     ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Hajime Tazaki

This commit adds a support to utilize exising UML drivers of block
devices for LKL.  The goal is similar to networking drivers, add minimum
amount of code and use drivers code as-is.

This commit also adds a test code with several filesystems (ext4, vfat,
btrfs at the moment) to exercise the basic operations of disk.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/drivers/ubd_kern.c              |  6 +++++-
 arch/um/lkl/Kconfig                     |  3 +++
 arch/um/lkl/include/uapi/asm/syscalls.h |  1 +
 arch/um/lkl/kernel/setup.c              |  5 +++++
 tools/lkl/include/lkl.h                 | 15 +++++++++++++++
 tools/lkl/lib/um/Build                  |  1 +
 tools/lkl/lib/um/um_block.c             | 17 +++++++++++++++++
 tools/lkl/lib/um/um_glue.c              | 24 +++++++++++++++++++++++-
 tools/lkl/tests/disk.c                  | 14 ++++++++++++++
 9 files changed, 84 insertions(+), 2 deletions(-)
 create mode 100644 tools/lkl/lib/um/um_block.c

diff --git a/arch/um/drivers/ubd_kern.c b/arch/um/drivers/ubd_kern.c
index eae8c83364f7..248bac8e36b8 100644
--- a/arch/um/drivers/ubd_kern.c
+++ b/arch/um/drivers/ubd_kern.c
@@ -545,7 +545,7 @@ static irqreturn_t ubd_intr(int irq, void *dev)
 /* Only changed by ubd_init, which is an initcall. */
 static int io_pid = -1;
 
-static void kill_io_thread(void)
+void kill_io_thread(void)
 {
 	if(io_pid != -1)
 		os_kill_process(io_pid, 1);
@@ -819,6 +819,7 @@ static int ubd_open_dev(struct ubd *ubd_dev)
 	}
 	ubd_dev->fd = fd;
 
+#ifdef CONFIG_UMMODE_KERN
 	if(ubd_dev->cow.file != NULL){
 		blk_queue_max_hw_sectors(ubd_dev->queue, 8 * sizeof(long));
 
@@ -843,6 +844,7 @@ static int ubd_open_dev(struct ubd *ubd_dev)
 		if(err < 0) goto error;
 		ubd_dev->cow.fd = err;
 	}
+#endif	/* CONFIG_UMMODE_KERN */
 	if (ubd_dev->no_trim == 0) {
 		ubd_dev->queue->limits.discard_granularity = SECTOR_SIZE;
 		ubd_dev->queue->limits.discard_alignment = SECTOR_SIZE;
@@ -852,9 +854,11 @@ static int ubd_open_dev(struct ubd *ubd_dev)
 	}
 	blk_queue_flag_set(QUEUE_FLAG_NONROT, ubd_dev->queue);
 	return 0;
+#ifdef CONFIG_UMMODE_KERN
  error:
 	os_close_file(ubd_dev->fd);
 	return err;
+#endif
 }
 
 static void ubd_device_release(struct device *dev)
diff --git a/arch/um/lkl/Kconfig b/arch/um/lkl/Kconfig
index 2b6b4063045c..8e4e0a465fc8 100644
--- a/arch/um/lkl/Kconfig
+++ b/arch/um/lkl/Kconfig
@@ -48,6 +48,9 @@ config OUTPUT_FORMAT
 config ARCH_DMA_ADDR_T_64BIT
        def_bool 64BIT
 
+config BLK_DEV_UBD
+       def_bool y if "$(OUTPUT_FORMAT)" = "elf32-i386" || "$(OUTPUT_FORMAT)" = "elf64-x86-64"
+
 config X86_64
        def_bool y if "$(OUTPUT_FORMAT)" = "elf64-x86-64"
 
diff --git a/arch/um/lkl/include/uapi/asm/syscalls.h b/arch/um/lkl/include/uapi/asm/syscalls.h
index a81534ffccb7..1384c34137d6 100644
--- a/arch/um/lkl/include/uapi/asm/syscalls.h
+++ b/arch/um/lkl/include/uapi/asm/syscalls.h
@@ -164,6 +164,7 @@ struct sockaddr {
 #include <linux/virtio_ring.h>
 #include <linux/pkt_sched.h>
 #include <linux/io_uring.h>
+#include <linux/major.h>
 
 struct user_msghdr {
 	void		__user *msg_name;
diff --git a/arch/um/lkl/kernel/setup.c b/arch/um/lkl/kernel/setup.c
index 39b7b7c79581..daf55c216db9 100644
--- a/arch/um/lkl/kernel/setup.c
+++ b/arch/um/lkl/kernel/setup.c
@@ -126,6 +126,11 @@ long lkl_sys_halt(void)
 		LINUX_REBOOT_CMD_RESTART,
 	};
 
+#ifdef CONFIG_BLK_DEV_UBD
+void kill_io_thread(void);
+	kill_io_thread();
+#endif
+
 	err = lkl_syscall(__NR_reboot, params);
 	if (err < 0)
 		return err;
diff --git a/tools/lkl/include/lkl.h b/tools/lkl/include/lkl.h
index 952d11e30868..7d164e4dcad1 100644
--- a/tools/lkl/include/lkl.h
+++ b/tools/lkl/include/lkl.h
@@ -355,6 +355,10 @@ void lkl_perror(char *msg, int err);
  */
 struct lkl_dev_blk_ops;
 
+enum {
+	BLK_BACKEND_UM,
+};
+
 /**
  * lkl_disk - host disk handle
  *
@@ -368,6 +372,7 @@ struct lkl_disk {
 		int fd;
 		void *handle;
 	};
+	int backend;
 	struct lkl_dev_blk_ops *ops;
 };
 
@@ -403,6 +408,16 @@ int lkl_disk_remove(struct lkl_disk disk);
  */
 int lkl_encode_dev_from_sysfs(const char *sysfs_path, uint32_t *pdevid);
 
+
+#ifdef LKL_HOST_CONFIG_UML_DEV
+int lkl_disk_um_add(struct lkl_disk *disk, const char *blkparams);
+#else
+static inline int lkl_disk_um_add(struct lkl_disk *disk, const char *blkparams)
+{
+	return -LKL_ENOSYS;
+}
+#endif
+
 /**
  * lkl_mount_dev - mount a disk
  *
diff --git a/tools/lkl/lib/um/Build b/tools/lkl/lib/um/Build
index 09a60975ecd6..52573c05a797 100644
--- a/tools/lkl/lib/um/Build
+++ b/tools/lkl/lib/um/Build
@@ -1,2 +1,3 @@
 liblkl-y += um_glue.o
 liblkl-y += um_net.o
+liblkl-y += um_block.o
diff --git a/tools/lkl/lib/um/um_block.c b/tools/lkl/lib/um/um_block.c
new file mode 100644
index 000000000000..70dc624ec43c
--- /dev/null
+++ b/tools/lkl/lib/um/um_block.c
@@ -0,0 +1,17 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <lkl_host.h>
+
+
+static int registered_blk_dev_idx;
+
+int lkl_disk_um_add(struct lkl_disk *disk, const char *blkparams)
+{
+	/* concat strings */
+	snprintf(lkl_um_devs + strlen(lkl_um_devs), sizeof(lkl_um_devs),
+		 " ubd%d=%s", registered_blk_dev_idx, blkparams);
+
+	return registered_blk_dev_idx++;
+}
diff --git a/tools/lkl/lib/um/um_glue.c b/tools/lkl/lib/um/um_glue.c
index f59e31a706de..fd5a338bcb9f 100644
--- a/tools/lkl/lib/um/um_glue.c
+++ b/tools/lkl/lib/um/um_glue.c
@@ -2,9 +2,10 @@
 #include <stdio.h>
 #include <stdlib.h>
 #include <string.h>
+#include <signal.h>
 #include <unistd.h>
 #include <errno.h>
-
+#include <sys/wait.h>
 
 
 char lkl_um_devs[4096];
@@ -37,3 +38,24 @@ void *um_os_signal(int signum, void *handler)
 {
 	return signal(signum, handler);
 }
+
+/* from util.c */
+/*
+ * UML helper threads must not handle SIGWINCH/INT/TERM
+ */
+void os_fix_helper_signals(void)
+{
+	signal(SIGWINCH, SIG_IGN);
+	signal(SIGINT, SIG_DFL);
+	signal(SIGTERM, SIG_DFL);
+}
+
+void os_kill_process(int pid, int reap_child)
+{
+	kill(pid, SIGKILL);
+	if (reap_child) {
+		while ((errno = 0, ((waitpid(pid, NULL, __WALL)) < 0))
+		       && (errno == EINTR))
+			;
+	}
+}
diff --git a/tools/lkl/tests/disk.c b/tools/lkl/tests/disk.c
index 0aa039876b54..f7801014d054 100644
--- a/tools/lkl/tests/disk.c
+++ b/tools/lkl/tests/disk.c
@@ -23,12 +23,17 @@ static struct {
 	const char *disk;
 	const char *fstype;
 	int partition;
+	int backend;
 } cla;
 
+const char *backends[] = { "um", NULL };
+
 struct cl_arg args[] = {
 	{"disk", 'd', "disk file to use", 1, CL_ARG_STR, &cla.disk},
 	{"partition", 'P', "partition to mount", 1, CL_ARG_INT, &cla.partition},
 	{"type", 't', "filesystem type", 1, CL_ARG_STR, &cla.fstype},
+	{"backend", 'b', "blockd evice backend type", 1, CL_ARG_STR_SET,
+	 &cla.backend, backends},
 	{0},
 };
 
@@ -50,6 +55,15 @@ int lkl_test_disk_add(void)
 
 	disk.ops = NULL;
 
+	disk.backend = cla.backend;
+
+	switch (disk.backend) {
+	case BLK_BACKEND_UM:
+		disk.dev = (char *)cla.disk;
+	default:
+		break;
+	}
+
 	disk_id = lkl_disk_add(&disk);
 	if (disk_id < 0)
 		goto out_close;
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 24/25] um lkl: add UML block device driver (ubd) for lkl
@ 2020-03-30 14:45     ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, linux-kernel-library, linux-arch,
	Hajime Tazaki, Akira Moroo

This commit adds a support to utilize exising UML drivers of block
devices for LKL.  The goal is similar to networking drivers, add minimum
amount of code and use drivers code as-is.

This commit also adds a test code with several filesystems (ext4, vfat,
btrfs at the moment) to exercise the basic operations of disk.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/drivers/ubd_kern.c              |  6 +++++-
 arch/um/lkl/Kconfig                     |  3 +++
 arch/um/lkl/include/uapi/asm/syscalls.h |  1 +
 arch/um/lkl/kernel/setup.c              |  5 +++++
 tools/lkl/include/lkl.h                 | 15 +++++++++++++++
 tools/lkl/lib/um/Build                  |  1 +
 tools/lkl/lib/um/um_block.c             | 17 +++++++++++++++++
 tools/lkl/lib/um/um_glue.c              | 24 +++++++++++++++++++++++-
 tools/lkl/tests/disk.c                  | 14 ++++++++++++++
 9 files changed, 84 insertions(+), 2 deletions(-)
 create mode 100644 tools/lkl/lib/um/um_block.c

diff --git a/arch/um/drivers/ubd_kern.c b/arch/um/drivers/ubd_kern.c
index eae8c83364f7..248bac8e36b8 100644
--- a/arch/um/drivers/ubd_kern.c
+++ b/arch/um/drivers/ubd_kern.c
@@ -545,7 +545,7 @@ static irqreturn_t ubd_intr(int irq, void *dev)
 /* Only changed by ubd_init, which is an initcall. */
 static int io_pid = -1;
 
-static void kill_io_thread(void)
+void kill_io_thread(void)
 {
 	if(io_pid != -1)
 		os_kill_process(io_pid, 1);
@@ -819,6 +819,7 @@ static int ubd_open_dev(struct ubd *ubd_dev)
 	}
 	ubd_dev->fd = fd;
 
+#ifdef CONFIG_UMMODE_KERN
 	if(ubd_dev->cow.file != NULL){
 		blk_queue_max_hw_sectors(ubd_dev->queue, 8 * sizeof(long));
 
@@ -843,6 +844,7 @@ static int ubd_open_dev(struct ubd *ubd_dev)
 		if(err < 0) goto error;
 		ubd_dev->cow.fd = err;
 	}
+#endif	/* CONFIG_UMMODE_KERN */
 	if (ubd_dev->no_trim == 0) {
 		ubd_dev->queue->limits.discard_granularity = SECTOR_SIZE;
 		ubd_dev->queue->limits.discard_alignment = SECTOR_SIZE;
@@ -852,9 +854,11 @@ static int ubd_open_dev(struct ubd *ubd_dev)
 	}
 	blk_queue_flag_set(QUEUE_FLAG_NONROT, ubd_dev->queue);
 	return 0;
+#ifdef CONFIG_UMMODE_KERN
  error:
 	os_close_file(ubd_dev->fd);
 	return err;
+#endif
 }
 
 static void ubd_device_release(struct device *dev)
diff --git a/arch/um/lkl/Kconfig b/arch/um/lkl/Kconfig
index 2b6b4063045c..8e4e0a465fc8 100644
--- a/arch/um/lkl/Kconfig
+++ b/arch/um/lkl/Kconfig
@@ -48,6 +48,9 @@ config OUTPUT_FORMAT
 config ARCH_DMA_ADDR_T_64BIT
        def_bool 64BIT
 
+config BLK_DEV_UBD
+       def_bool y if "$(OUTPUT_FORMAT)" = "elf32-i386" || "$(OUTPUT_FORMAT)" = "elf64-x86-64"
+
 config X86_64
        def_bool y if "$(OUTPUT_FORMAT)" = "elf64-x86-64"
 
diff --git a/arch/um/lkl/include/uapi/asm/syscalls.h b/arch/um/lkl/include/uapi/asm/syscalls.h
index a81534ffccb7..1384c34137d6 100644
--- a/arch/um/lkl/include/uapi/asm/syscalls.h
+++ b/arch/um/lkl/include/uapi/asm/syscalls.h
@@ -164,6 +164,7 @@ struct sockaddr {
 #include <linux/virtio_ring.h>
 #include <linux/pkt_sched.h>
 #include <linux/io_uring.h>
+#include <linux/major.h>
 
 struct user_msghdr {
 	void		__user *msg_name;
diff --git a/arch/um/lkl/kernel/setup.c b/arch/um/lkl/kernel/setup.c
index 39b7b7c79581..daf55c216db9 100644
--- a/arch/um/lkl/kernel/setup.c
+++ b/arch/um/lkl/kernel/setup.c
@@ -126,6 +126,11 @@ long lkl_sys_halt(void)
 		LINUX_REBOOT_CMD_RESTART,
 	};
 
+#ifdef CONFIG_BLK_DEV_UBD
+void kill_io_thread(void);
+	kill_io_thread();
+#endif
+
 	err = lkl_syscall(__NR_reboot, params);
 	if (err < 0)
 		return err;
diff --git a/tools/lkl/include/lkl.h b/tools/lkl/include/lkl.h
index 952d11e30868..7d164e4dcad1 100644
--- a/tools/lkl/include/lkl.h
+++ b/tools/lkl/include/lkl.h
@@ -355,6 +355,10 @@ void lkl_perror(char *msg, int err);
  */
 struct lkl_dev_blk_ops;
 
+enum {
+	BLK_BACKEND_UM,
+};
+
 /**
  * lkl_disk - host disk handle
  *
@@ -368,6 +372,7 @@ struct lkl_disk {
 		int fd;
 		void *handle;
 	};
+	int backend;
 	struct lkl_dev_blk_ops *ops;
 };
 
@@ -403,6 +408,16 @@ int lkl_disk_remove(struct lkl_disk disk);
  */
 int lkl_encode_dev_from_sysfs(const char *sysfs_path, uint32_t *pdevid);
 
+
+#ifdef LKL_HOST_CONFIG_UML_DEV
+int lkl_disk_um_add(struct lkl_disk *disk, const char *blkparams);
+#else
+static inline int lkl_disk_um_add(struct lkl_disk *disk, const char *blkparams)
+{
+	return -LKL_ENOSYS;
+}
+#endif
+
 /**
  * lkl_mount_dev - mount a disk
  *
diff --git a/tools/lkl/lib/um/Build b/tools/lkl/lib/um/Build
index 09a60975ecd6..52573c05a797 100644
--- a/tools/lkl/lib/um/Build
+++ b/tools/lkl/lib/um/Build
@@ -1,2 +1,3 @@
 liblkl-y += um_glue.o
 liblkl-y += um_net.o
+liblkl-y += um_block.o
diff --git a/tools/lkl/lib/um/um_block.c b/tools/lkl/lib/um/um_block.c
new file mode 100644
index 000000000000..70dc624ec43c
--- /dev/null
+++ b/tools/lkl/lib/um/um_block.c
@@ -0,0 +1,17 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <lkl_host.h>
+
+
+static int registered_blk_dev_idx;
+
+int lkl_disk_um_add(struct lkl_disk *disk, const char *blkparams)
+{
+	/* concat strings */
+	snprintf(lkl_um_devs + strlen(lkl_um_devs), sizeof(lkl_um_devs),
+		 " ubd%d=%s", registered_blk_dev_idx, blkparams);
+
+	return registered_blk_dev_idx++;
+}
diff --git a/tools/lkl/lib/um/um_glue.c b/tools/lkl/lib/um/um_glue.c
index f59e31a706de..fd5a338bcb9f 100644
--- a/tools/lkl/lib/um/um_glue.c
+++ b/tools/lkl/lib/um/um_glue.c
@@ -2,9 +2,10 @@
 #include <stdio.h>
 #include <stdlib.h>
 #include <string.h>
+#include <signal.h>
 #include <unistd.h>
 #include <errno.h>
-
+#include <sys/wait.h>
 
 
 char lkl_um_devs[4096];
@@ -37,3 +38,24 @@ void *um_os_signal(int signum, void *handler)
 {
 	return signal(signum, handler);
 }
+
+/* from util.c */
+/*
+ * UML helper threads must not handle SIGWINCH/INT/TERM
+ */
+void os_fix_helper_signals(void)
+{
+	signal(SIGWINCH, SIG_IGN);
+	signal(SIGINT, SIG_DFL);
+	signal(SIGTERM, SIG_DFL);
+}
+
+void os_kill_process(int pid, int reap_child)
+{
+	kill(pid, SIGKILL);
+	if (reap_child) {
+		while ((errno = 0, ((waitpid(pid, NULL, __WALL)) < 0))
+		       && (errno == EINTR))
+			;
+	}
+}
diff --git a/tools/lkl/tests/disk.c b/tools/lkl/tests/disk.c
index 0aa039876b54..f7801014d054 100644
--- a/tools/lkl/tests/disk.c
+++ b/tools/lkl/tests/disk.c
@@ -23,12 +23,17 @@ static struct {
 	const char *disk;
 	const char *fstype;
 	int partition;
+	int backend;
 } cla;
 
+const char *backends[] = { "um", NULL };
+
 struct cl_arg args[] = {
 	{"disk", 'd', "disk file to use", 1, CL_ARG_STR, &cla.disk},
 	{"partition", 'P', "partition to mount", 1, CL_ARG_INT, &cla.partition},
 	{"type", 't', "filesystem type", 1, CL_ARG_STR, &cla.fstype},
+	{"backend", 'b', "blockd evice backend type", 1, CL_ARG_STR_SET,
+	 &cla.backend, backends},
 	{0},
 };
 
@@ -50,6 +55,15 @@ int lkl_test_disk_add(void)
 
 	disk.ops = NULL;
 
+	disk.backend = cla.backend;
+
+	switch (disk.backend) {
+	case BLK_BACKEND_UM:
+		disk.dev = (char *)cla.disk;
+	default:
+		break;
+	}
+
 	disk_id = lkl_disk_add(&disk);
 	if (disk_id < 0)
 		goto out_close;
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 25/25] um: fix clone flags to be familiar with valgrind
  2020-03-30 14:45   ` Hajime Tazaki
@ 2020-03-30 14:45     ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Hajime Tazaki

Valgrind reports the following error with UML when ubd driver is
compiled in.

==29588== Unsupported clone() flags: 0x500
==29588==
==29588== The only supported clone() uses are:
==29588==  - via a threads library (LinuxThreads or NPTL)
==29588==  - via the implementation of fork or vfork

Adding CLONE_FS flags silences this issue.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/drivers/ubd_user.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/arch/um/drivers/ubd_user.c b/arch/um/drivers/ubd_user.c
index a1afe414ce48..fabb50e91c37 100644
--- a/arch/um/drivers/ubd_user.c
+++ b/arch/um/drivers/ubd_user.c
@@ -47,7 +47,8 @@ int start_io_thread(unsigned long sp, int *fd_out)
 		goto out_close;
 	}
 
-	pid = clone(io_thread, (void *) sp, CLONE_FILES | CLONE_VM, NULL);
+	pid = clone(io_thread, (void *) sp,
+		    CLONE_FILES | CLONE_VM | CLONE_FS, NULL);
 	if(pid < 0){
 		err = -errno;
 		printk("start_io_thread - clone failed : errno = %d\n", errno);
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v4 25/25] um: fix clone flags to be familiar with valgrind
@ 2020-03-30 14:45     ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-30 14:45 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, linux-kernel-library, linux-arch,
	Hajime Tazaki, Akira Moroo

Valgrind reports the following error with UML when ubd driver is
compiled in.

==29588== Unsupported clone() flags: 0x500
==29588==
==29588== The only supported clone() uses are:
==29588==  - via a threads library (LinuxThreads or NPTL)
==29588==  - via the implementation of fork or vfork

Adding CLONE_FS flags silences this issue.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/drivers/ubd_user.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/arch/um/drivers/ubd_user.c b/arch/um/drivers/ubd_user.c
index a1afe414ce48..fabb50e91c37 100644
--- a/arch/um/drivers/ubd_user.c
+++ b/arch/um/drivers/ubd_user.c
@@ -47,7 +47,8 @@ int start_io_thread(unsigned long sp, int *fd_out)
 		goto out_close;
 	}
 
-	pid = clone(io_thread, (void *) sp, CLONE_FILES | CLONE_VM, NULL);
+	pid = clone(io_thread, (void *) sp,
+		    CLONE_FILES | CLONE_VM | CLONE_FS, NULL);
 	if(pid < 0){
 		err = -errno;
 		printk("start_io_thread - clone failed : errno = %d\n", errno);
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* Re: [RFC v4 23/25] um lkl: add UML network driver for lkl
  2020-03-30 14:45     ` Hajime Tazaki
@ 2020-03-30 21:31       ` Johannes Berg
  -1 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-03-30 21:31 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um
  Cc: Octavian Purdila, linux-kernel-library, linux-arch, Akira Moroo


> +++ b/arch/um/lkl/include/asm/irq.h
> @@ -2,6 +2,9 @@
>  #ifndef _ASM_LKL_IRQ_H
>  #define _ASM_LKL_IRQ_H
>  
> +/* pull UML's definitions */
> +#include "../../../include/asm/irq.h"

This is _really_ ugly.

> +#if defined(__linux) && (defined(__i386) || defined(__x86_64))
> +#include <os.h>
> +#endif
> +void *um_os_signal(int signum, void *handler);

and arguably those random declarations you're sprinkling are worse.

> @@ -181,6 +196,11 @@ void init_IRQ(void)
>  	for (i = 0; i < NR_IRQS; i++)
>  		irq_set_chip_and_handler(i, &dummy_irq_chip, handle_simple_irq);
>  
> +#if defined(__linux) && (defined(__i386) || defined(__x86_64))

What's with all those ifdefs with this condition?

> +++ b/tools/lkl/lib/um/um_glue.c
> @@ -0,0 +1,39 @@
> +// SPDX-License-Identifier: GPL-2.0
> +#include <stdio.h>
> +#include <stdlib.h>
> +#include <string.h>
> +#include <unistd.h>
> +#include <errno.h>
> +
> +
> +
> +char lkl_um_devs[4096];
> +
> +/* from sigio.c */
> +void maybe_sigio_broken(int fd, int read)
> +{
> +}
> +
> +/* from process.c */
> +int os_getpid(void)
> +{
> +	return getpid();
> +}

All of this really is quite ugly - are you sure it's needed for just the
vector network driver??

johannes

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v4 23/25] um lkl: add UML network driver for lkl
@ 2020-03-30 21:31       ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-03-30 21:31 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um
  Cc: Octavian Purdila, linux-kernel-library, Akira Moroo, linux-arch


> +++ b/arch/um/lkl/include/asm/irq.h
> @@ -2,6 +2,9 @@
>  #ifndef _ASM_LKL_IRQ_H
>  #define _ASM_LKL_IRQ_H
>  
> +/* pull UML's definitions */
> +#include "../../../include/asm/irq.h"

This is _really_ ugly.

> +#if defined(__linux) && (defined(__i386) || defined(__x86_64))
> +#include <os.h>
> +#endif
> +void *um_os_signal(int signum, void *handler);

and arguably those random declarations you're sprinkling are worse.

> @@ -181,6 +196,11 @@ void init_IRQ(void)
>  	for (i = 0; i < NR_IRQS; i++)
>  		irq_set_chip_and_handler(i, &dummy_irq_chip, handle_simple_irq);
>  
> +#if defined(__linux) && (defined(__i386) || defined(__x86_64))

What's with all those ifdefs with this condition?

> +++ b/tools/lkl/lib/um/um_glue.c
> @@ -0,0 +1,39 @@
> +// SPDX-License-Identifier: GPL-2.0
> +#include <stdio.h>
> +#include <stdlib.h>
> +#include <string.h>
> +#include <unistd.h>
> +#include <errno.h>
> +
> +
> +
> +char lkl_um_devs[4096];
> +
> +/* from sigio.c */
> +void maybe_sigio_broken(int fd, int read)
> +{
> +}
> +
> +/* from process.c */
> +int os_getpid(void)
> +{
> +	return getpid();
> +}

All of this really is quite ugly - are you sure it's needed for just the
vector network driver??

johannes


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v4 02/25] um lkl: architecture skeleton for Linux kernel library
  2020-03-30 14:45     ` Hajime Tazaki
@ 2020-03-30 21:53       ` Johannes Berg
  -1 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-03-30 21:53 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um
  Cc: linux-arch, Levente Kurusa, Matthieu Coudron, Conrad Meyer,
	Octavian Purdila, Jens Staal, Motomu Utsumi, Lai Jiangshan,
	Akira Moroo, Petros Angelatos, Yuan Liu, Xiao Jia,
	Mark Stillwell, Patrick Collins, linux-kernel-library,
	Pierre-Hugues Husson, Michael Zimmermann, Luca Dariz,
	Edison M . Castro

On Mon, 2020-03-30 at 23:45 +0900, Hajime Tazaki wrote:
> From: Octavian Purdila <tavi.purdila@gmail.com>
> 
> Adds the LKL Kconfig, vmlinux linker script, basic architecture
> headers and miscellaneous basic functions or stubs such as
> dump_stack(), show_regs() and cpuinfo proc ops.
> 
> The headers we introduce in this patch are simple wrappers to the
> asm-generic headers or stubs for things we don't support, such as
> ptrace, DMA, signals, ELF handling and low level processor operations.
> 
> The kernel configuration is automatically updated to reflect the
> endianness of the host, 64bit support or the output format for
> vmlinux's linker script. We do this by looking at the ld's default
> output format.

Can you elaborate what the plan is here?

I mean, you're not actually "unifying" anything with ARCH=um, you're
just basically splitting ARCH=um into ARCH=um-um and ARCH=um-lkl or
something. Is there much point?

Even the basic underlying building blocks are _very_ different, e.g. in
UML the host interface is just a bunch of functions that must be
implemented (os_*()), while you have a "struct lkl_host_operations". If
we can't even unify at that trivial level, is there any point in it at
all? I'm not even really sure what those ops are used for - are all of
those things that the *application* using LKL necessarily must provide?

Similarly with the IRQ routing mechanism - two completely different
concepts. Where's the "unifying"?

Ultimately, I can see two ways this goes:

1) We give up, and get ARCH=lkl, sharing just (some of) the drivers
   while moving them into the right drivers/somewhere/ place. Even that
   looks somewhat awkward looking at the later patches in this set, but
   seems like that at *least* should be done.

2) Ideally, instead, we actually unify: LKL grows support for userspace
   processes using UML infrastructure, the "in-kernel" IRQ mechanisms
   are unified, UML stuff moves into lkl-ops, and the UML binary
   basically becomes a user of the LKL library to start everything up.
   There may be some bits remaining that are just not interesting (e.g.
   some drivers you don't care about would continue to make direct calls
   to the user side instead of lkl-ops, and then they're just not
   compatible with lkl, only with the uml special case of lkl), but then
   things are clean.


Now, of course it seems like (2) would actually be better - LKL would
actually get support for userspace processes using UML's tricks, most of
the code is unified, and even LKL's users can take advantage of new
things. At the same time, all of the duplication is avoided.

However, I just don't know if it's actually _possible_ to do that
though. Conceptually, it seems like it should be - why shouldn't a
library be able to spawn other userspace processes - I mean, it's not
really in the model that LKL really _wants_ since it's supposed to offer
the syscall API, but you could make a "bool run_normal_init" or
something in the lkl-ops for the user of the library to determine what
should happen, right?

However, there clearly are huge differences between LKL and UML in all
respects, and I don't know if this wouldn't conflict with the library
model, e.g. there may be linker issues etc. Or maybe the specific UML
interrupt handling is required in UML and cannot be done in LKL (but
then perhaps it could be put into the hypothetical UML-application vs.
UML-the-LKL-library?)


Ultimately, personally I think it's going to have to be one or the other
of those two options though, I don't really see much value in what
you're doing here in the patchset now. This way just messes up
everything, it's not clear what's UML and what's LKL, and they're
intertwined with ifdefs everywhere; just look at where you have to add
ifdefs in patch 23 - how would anyone later understand which part gets
compiled for which of them?

johannes

PS: actually having something like lkl-ops in UML would've been nice
also for my "time-travel" work, since it neatly abstracts the timers
out. I do wonder a bit about the overhead of jumping through function
pointers all the time though, it may be worth rethinking that overall
anyway!

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v4 02/25] um lkl: architecture skeleton for Linux kernel library
@ 2020-03-30 21:53       ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-03-30 21:53 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um
  Cc: linux-arch, Levente Kurusa, Matthieu Coudron, Conrad Meyer,
	Octavian Purdila, Jens Staal, Motomu Utsumi, Lai Jiangshan,
	Akira Moroo, Petros Angelatos, Edison M . Castro, Xiao Jia,
	Mark Stillwell, linux-kernel-library, Patrick Collins,
	Pierre-Hugues Husson, Michael Zimmermann, Luca Dariz, Yuan Liu

On Mon, 2020-03-30 at 23:45 +0900, Hajime Tazaki wrote:
> From: Octavian Purdila <tavi.purdila@gmail.com>
> 
> Adds the LKL Kconfig, vmlinux linker script, basic architecture
> headers and miscellaneous basic functions or stubs such as
> dump_stack(), show_regs() and cpuinfo proc ops.
> 
> The headers we introduce in this patch are simple wrappers to the
> asm-generic headers or stubs for things we don't support, such as
> ptrace, DMA, signals, ELF handling and low level processor operations.
> 
> The kernel configuration is automatically updated to reflect the
> endianness of the host, 64bit support or the output format for
> vmlinux's linker script. We do this by looking at the ld's default
> output format.

Can you elaborate what the plan is here?

I mean, you're not actually "unifying" anything with ARCH=um, you're
just basically splitting ARCH=um into ARCH=um-um and ARCH=um-lkl or
something. Is there much point?

Even the basic underlying building blocks are _very_ different, e.g. in
UML the host interface is just a bunch of functions that must be
implemented (os_*()), while you have a "struct lkl_host_operations". If
we can't even unify at that trivial level, is there any point in it at
all? I'm not even really sure what those ops are used for - are all of
those things that the *application* using LKL necessarily must provide?

Similarly with the IRQ routing mechanism - two completely different
concepts. Where's the "unifying"?

Ultimately, I can see two ways this goes:

1) We give up, and get ARCH=lkl, sharing just (some of) the drivers
   while moving them into the right drivers/somewhere/ place. Even that
   looks somewhat awkward looking at the later patches in this set, but
   seems like that at *least* should be done.

2) Ideally, instead, we actually unify: LKL grows support for userspace
   processes using UML infrastructure, the "in-kernel" IRQ mechanisms
   are unified, UML stuff moves into lkl-ops, and the UML binary
   basically becomes a user of the LKL library to start everything up.
   There may be some bits remaining that are just not interesting (e.g.
   some drivers you don't care about would continue to make direct calls
   to the user side instead of lkl-ops, and then they're just not
   compatible with lkl, only with the uml special case of lkl), but then
   things are clean.


Now, of course it seems like (2) would actually be better - LKL would
actually get support for userspace processes using UML's tricks, most of
the code is unified, and even LKL's users can take advantage of new
things. At the same time, all of the duplication is avoided.

However, I just don't know if it's actually _possible_ to do that
though. Conceptually, it seems like it should be - why shouldn't a
library be able to spawn other userspace processes - I mean, it's not
really in the model that LKL really _wants_ since it's supposed to offer
the syscall API, but you could make a "bool run_normal_init" or
something in the lkl-ops for the user of the library to determine what
should happen, right?

However, there clearly are huge differences between LKL and UML in all
respects, and I don't know if this wouldn't conflict with the library
model, e.g. there may be linker issues etc. Or maybe the specific UML
interrupt handling is required in UML and cannot be done in LKL (but
then perhaps it could be put into the hypothetical UML-application vs.
UML-the-LKL-library?)


Ultimately, personally I think it's going to have to be one or the other
of those two options though, I don't really see much value in what
you're doing here in the patchset now. This way just messes up
everything, it's not clear what's UML and what's LKL, and they're
intertwined with ifdefs everywhere; just look at where you have to add
ifdefs in patch 23 - how would anyone later understand which part gets
compiled for which of them?

johannes

PS: actually having something like lkl-ops in UML would've been nice
also for my "time-travel" work, since it neatly abstracts the timers
out. I do wonder a bit about the overhead of jumping through function
pointers all the time though, it may be worth rethinking that overall
anyway!


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v4 02/25] um lkl: architecture skeleton for Linux kernel library
@ 2020-03-30 22:12         ` Richard Weinberger
  0 siblings, 0 replies; 476+ messages in thread
From: Richard Weinberger @ 2020-03-30 22:12 UTC (permalink / raw)
  To: Johannes Berg
  Cc: Hajime Tazaki, linux-um, Linux-Arch, Levente Kurusa,
	Matthieu Coudron, Conrad Meyer, Octavian Purdila, Jens Staal,
	Motomu Utsumi, Lai Jiangshan, Akira Moroo, Petros Angelatos,
	Edison M . Castro, Xiao Jia, Mark Stillwell,
	linux-kernel-library, Patrick Collins, Pierre-Hugues Husson,
	Michael Zimmermann, Luca Dariz

Johannes, Hajime,

On Mon, Mar 30, 2020 at 11:53 PM Johannes Berg
<johannes@sipsolutions.net> wrote:
>
> On Mon, 2020-03-30 at 23:45 +0900, Hajime Tazaki wrote:
> > From: Octavian Purdila <tavi.purdila@gmail.com>
> >
> > Adds the LKL Kconfig, vmlinux linker script, basic architecture
> > headers and miscellaneous basic functions or stubs such as
> > dump_stack(), show_regs() and cpuinfo proc ops.
> >
> > The headers we introduce in this patch are simple wrappers to the
> > asm-generic headers or stubs for things we don't support, such as
> > ptrace, DMA, signals, ELF handling and low level processor operations.
> >
> > The kernel configuration is automatically updated to reflect the
> > endianness of the host, 64bit support or the output format for
> > vmlinux's linker script. We do this by looking at the ld's default
> > output format.
>
> Can you elaborate what the plan is here?
>
> I mean, you're not actually "unifying" anything with ARCH=um, you're
> just basically splitting ARCH=um into ARCH=um-um and ARCH=um-lkl or
> something. Is there much point?
>
> Even the basic underlying building blocks are _very_ different, e.g. in
> UML the host interface is just a bunch of functions that must be
> implemented (os_*()), while you have a "struct lkl_host_operations". If
> we can't even unify at that trivial level, is there any point in it at
> all? I'm not even really sure what those ops are used for - are all of
> those things that the *application* using LKL necessarily must provide?
>
> Similarly with the IRQ routing mechanism - two completely different
> concepts. Where's the "unifying"?
>
> Ultimately, I can see two ways this goes:
>
> 1) We give up, and get ARCH=lkl, sharing just (some of) the drivers
>    while moving them into the right drivers/somewhere/ place. Even that
>    looks somewhat awkward looking at the later patches in this set, but
>    seems like that at *least* should be done.

Yeah, this would be a goal.
UML and LKL are quite different but they should share at least their userspace
drivers.
I also don't mind if we don't share every driver at the beginning but
it should be
a feasible goal for the future.

> 2) Ideally, instead, we actually unify: LKL grows support for userspace
>    processes using UML infrastructure, the "in-kernel" IRQ mechanisms
>    are unified, UML stuff moves into lkl-ops, and the UML binary
>    basically becomes a user of the LKL library to start everything up.
>    There may be some bits remaining that are just not interesting (e.g.
>    some drivers you don't care about would continue to make direct calls
>    to the user side instead of lkl-ops, and then they're just not
>    compatible with lkl, only with the uml special case of lkl), but then
>    things are clean.

A few months ago I though this is doable but now I'm not so sure anymore.

>
> Now, of course it seems like (2) would actually be better - LKL would
> actually get support for userspace processes using UML's tricks, most of
> the code is unified, and even LKL's users can take advantage of new
> things. At the same time, all of the duplication is avoided.
>
> However, I just don't know if it's actually _possible_ to do that
> though. Conceptually, it seems like it should be - why shouldn't a
> library be able to spawn other userspace processes - I mean, it's not
> really in the model that LKL really _wants_ since it's supposed to offer
> the syscall API, but you could make a "bool run_normal_init" or
> something in the lkl-ops for the user of the library to determine what
> should happen, right?
>
> However, there clearly are huge differences between LKL and UML in all
> respects, and I don't know if this wouldn't conflict with the library
> model, e.g. there may be linker issues etc. Or maybe the specific UML
> interrupt handling is required in UML and cannot be done in LKL (but
> then perhaps it could be put into the hypothetical UML-application vs.
> UML-the-LKL-library?)
>
>
> Ultimately, personally I think it's going to have to be one or the other
> of those two options though, I don't really see much value in what
> you're doing here in the patchset now. This way just messes up
> everything, it's not clear what's UML and what's LKL, and they're
> intertwined with ifdefs everywhere; just look at where you have to add
> ifdefs in patch 23 - how would anyone later understand which part gets
> compiled for which of them?
>
> johannes
>
> PS: actually having something like lkl-ops in UML would've been nice
> also for my "time-travel" work, since it neatly abstracts the timers
> out. I do wonder a bit about the overhead of jumping through function
> pointers all the time though, it may be worth rethinking that overall
> anyway!

Agreed. UML can also learn from LKL. :-)

-- 
Thanks,
//richard

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v4 02/25] um lkl: architecture skeleton for Linux kernel library
@ 2020-03-30 22:12         ` Richard Weinberger
  0 siblings, 0 replies; 476+ messages in thread
From: Richard Weinberger @ 2020-03-30 22:12 UTC (permalink / raw)
  To: Johannes Berg
  Cc: Hajime Tazaki, linux-um, Linux-Arch, Levente Kurusa,
	Matthieu Coudron, Conrad Meyer, Octavian Purdila, Jens Staal,
	Motomu Utsumi, Lai Jiangshan, Akira Moroo, Petros Angelatos,
	Edison M . Castro, Xiao Jia, Mark Stillwell,
	linux-kernel-library, Patrick Collins, Pierre-Hugues Husson,
	Michael Zimmermann, Luca Dariz, Yuan Liu

Johannes, Hajime,

On Mon, Mar 30, 2020 at 11:53 PM Johannes Berg
<johannes@sipsolutions.net> wrote:
>
> On Mon, 2020-03-30 at 23:45 +0900, Hajime Tazaki wrote:
> > From: Octavian Purdila <tavi.purdila@gmail.com>
> >
> > Adds the LKL Kconfig, vmlinux linker script, basic architecture
> > headers and miscellaneous basic functions or stubs such as
> > dump_stack(), show_regs() and cpuinfo proc ops.
> >
> > The headers we introduce in this patch are simple wrappers to the
> > asm-generic headers or stubs for things we don't support, such as
> > ptrace, DMA, signals, ELF handling and low level processor operations.
> >
> > The kernel configuration is automatically updated to reflect the
> > endianness of the host, 64bit support or the output format for
> > vmlinux's linker script. We do this by looking at the ld's default
> > output format.
>
> Can you elaborate what the plan is here?
>
> I mean, you're not actually "unifying" anything with ARCH=um, you're
> just basically splitting ARCH=um into ARCH=um-um and ARCH=um-lkl or
> something. Is there much point?
>
> Even the basic underlying building blocks are _very_ different, e.g. in
> UML the host interface is just a bunch of functions that must be
> implemented (os_*()), while you have a "struct lkl_host_operations". If
> we can't even unify at that trivial level, is there any point in it at
> all? I'm not even really sure what those ops are used for - are all of
> those things that the *application* using LKL necessarily must provide?
>
> Similarly with the IRQ routing mechanism - two completely different
> concepts. Where's the "unifying"?
>
> Ultimately, I can see two ways this goes:
>
> 1) We give up, and get ARCH=lkl, sharing just (some of) the drivers
>    while moving them into the right drivers/somewhere/ place. Even that
>    looks somewhat awkward looking at the later patches in this set, but
>    seems like that at *least* should be done.

Yeah, this would be a goal.
UML and LKL are quite different but they should share at least their userspace
drivers.
I also don't mind if we don't share every driver at the beginning but
it should be
a feasible goal for the future.

> 2) Ideally, instead, we actually unify: LKL grows support for userspace
>    processes using UML infrastructure, the "in-kernel" IRQ mechanisms
>    are unified, UML stuff moves into lkl-ops, and the UML binary
>    basically becomes a user of the LKL library to start everything up.
>    There may be some bits remaining that are just not interesting (e.g.
>    some drivers you don't care about would continue to make direct calls
>    to the user side instead of lkl-ops, and then they're just not
>    compatible with lkl, only with the uml special case of lkl), but then
>    things are clean.

A few months ago I though this is doable but now I'm not so sure anymore.

>
> Now, of course it seems like (2) would actually be better - LKL would
> actually get support for userspace processes using UML's tricks, most of
> the code is unified, and even LKL's users can take advantage of new
> things. At the same time, all of the duplication is avoided.
>
> However, I just don't know if it's actually _possible_ to do that
> though. Conceptually, it seems like it should be - why shouldn't a
> library be able to spawn other userspace processes - I mean, it's not
> really in the model that LKL really _wants_ since it's supposed to offer
> the syscall API, but you could make a "bool run_normal_init" or
> something in the lkl-ops for the user of the library to determine what
> should happen, right?
>
> However, there clearly are huge differences between LKL and UML in all
> respects, and I don't know if this wouldn't conflict with the library
> model, e.g. there may be linker issues etc. Or maybe the specific UML
> interrupt handling is required in UML and cannot be done in LKL (but
> then perhaps it could be put into the hypothetical UML-application vs.
> UML-the-LKL-library?)
>
>
> Ultimately, personally I think it's going to have to be one or the other
> of those two options though, I don't really see much value in what
> you're doing here in the patchset now. This way just messes up
> everything, it's not clear what's UML and what's LKL, and they're
> intertwined with ifdefs everywhere; just look at where you have to add
> ifdefs in patch 23 - how would anyone later understand which part gets
> compiled for which of them?
>
> johannes
>
> PS: actually having something like lkl-ops in UML would've been nice
> also for my "time-travel" work, since it neatly abstracts the timers
> out. I do wonder a bit about the overhead of jumping through function
> pointers all the time though, it may be worth rethinking that overall
> anyway!

Agreed. UML can also learn from LKL. :-)

-- 
Thanks,
//richard

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v4 23/25] um lkl: add UML network driver for lkl
  2020-03-30 21:31       ` Johannes Berg
@ 2020-03-31  2:38         ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-31  2:38 UTC (permalink / raw)
  To: johannes
  Cc: linux-um, tavi.purdila, linux-kernel-library, linux-arch, retrage01


Thanks for the comments, Johannes.

On Tue, 31 Mar 2020 06:31:15 +0900,
Johannes Berg wrote:
> 
> 
> > +++ b/arch/um/lkl/include/asm/irq.h
> > @@ -2,6 +2,9 @@
> >  #ifndef _ASM_LKL_IRQ_H
> >  #define _ASM_LKL_IRQ_H
> >  
> > +/* pull UML's definitions */
> > +#include "../../../include/asm/irq.h"
> 
> This is _really_ ugly.

Hmm, in previous patchset (until v3), I was using the worse approach
(I thought) to avoid this include.

+KBUILD_CFLAGS  += -DTIMER_IRQ=0 -DUBD_IRQ=4 -DUM_ETH_IRQ=5 -DLAST_IRQ=15

And I thought the current way is better than before.


> > +#if defined(__linux) && (defined(__i386) || defined(__x86_64))
> > +#include <os.h>
> > +#endif
> > +void *um_os_signal(int signum, void *handler);
> 
> and arguably those random declarations you're sprinkling are worse.

It means that those are only used when UML drivers (net, block) are
enabled, which are only available on Intel/linux hosts.  OTOH,
UMMODE_LIB _will_ support non-Intel/Linux hosts in the future thus,
this ifdefs exist.

> > @@ -181,6 +196,11 @@ void init_IRQ(void)
> >  	for (i = 0; i < NR_IRQS; i++)
> >  		irq_set_chip_and_handler(i, &dummy_irq_chip, handle_simple_irq);
> >  
> > +#if defined(__linux) && (defined(__i386) || defined(__x86_64))
> 
> What's with all those ifdefs with this condition?

Same as above.
but I agree that the ifdefs are cryptic; I'll try to make it more
understandable if I use ifdefs.

> > +++ b/tools/lkl/lib/um/um_glue.c
> > @@ -0,0 +1,39 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +#include <stdio.h>
> > +#include <stdlib.h>
> > +#include <string.h>
> > +#include <unistd.h>
> > +#include <errno.h>
> > +
> > +
> > +
> > +char lkl_um_devs[4096];
> > +
> > +/* from sigio.c */
> > +void maybe_sigio_broken(int fd, int read)
> > +{
> > +}
> > +
> > +/* from process.c */
> > +int os_getpid(void)
> > +{
> > +	return getpid();
> > +}
> 
> All of this really is quite ugly - are you sure it's needed for just the
> vector network driver??

Those are needed when we use UML_NET or BLK_DEV_UBD is enabled.  I was
trying to minimize those glue code as much as possible; I need another
try to check if we can remove more or eliminate completely by other
ways.

-- Hajime

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v4 23/25] um lkl: add UML network driver for lkl
@ 2020-03-31  2:38         ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-31  2:38 UTC (permalink / raw)
  To: johannes
  Cc: tavi.purdila, linux-kernel-library, retrage01, linux-um, linux-arch


Thanks for the comments, Johannes.

On Tue, 31 Mar 2020 06:31:15 +0900,
Johannes Berg wrote:
> 
> 
> > +++ b/arch/um/lkl/include/asm/irq.h
> > @@ -2,6 +2,9 @@
> >  #ifndef _ASM_LKL_IRQ_H
> >  #define _ASM_LKL_IRQ_H
> >  
> > +/* pull UML's definitions */
> > +#include "../../../include/asm/irq.h"
> 
> This is _really_ ugly.

Hmm, in previous patchset (until v3), I was using the worse approach
(I thought) to avoid this include.

+KBUILD_CFLAGS  += -DTIMER_IRQ=0 -DUBD_IRQ=4 -DUM_ETH_IRQ=5 -DLAST_IRQ=15

And I thought the current way is better than before.


> > +#if defined(__linux) && (defined(__i386) || defined(__x86_64))
> > +#include <os.h>
> > +#endif
> > +void *um_os_signal(int signum, void *handler);
> 
> and arguably those random declarations you're sprinkling are worse.

It means that those are only used when UML drivers (net, block) are
enabled, which are only available on Intel/linux hosts.  OTOH,
UMMODE_LIB _will_ support non-Intel/Linux hosts in the future thus,
this ifdefs exist.

> > @@ -181,6 +196,11 @@ void init_IRQ(void)
> >  	for (i = 0; i < NR_IRQS; i++)
> >  		irq_set_chip_and_handler(i, &dummy_irq_chip, handle_simple_irq);
> >  
> > +#if defined(__linux) && (defined(__i386) || defined(__x86_64))
> 
> What's with all those ifdefs with this condition?

Same as above.
but I agree that the ifdefs are cryptic; I'll try to make it more
understandable if I use ifdefs.

> > +++ b/tools/lkl/lib/um/um_glue.c
> > @@ -0,0 +1,39 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +#include <stdio.h>
> > +#include <stdlib.h>
> > +#include <string.h>
> > +#include <unistd.h>
> > +#include <errno.h>
> > +
> > +
> > +
> > +char lkl_um_devs[4096];
> > +
> > +/* from sigio.c */
> > +void maybe_sigio_broken(int fd, int read)
> > +{
> > +}
> > +
> > +/* from process.c */
> > +int os_getpid(void)
> > +{
> > +	return getpid();
> > +}
> 
> All of this really is quite ugly - are you sure it's needed for just the
> vector network driver??

Those are needed when we use UML_NET or BLK_DEV_UBD is enabled.  I was
trying to minimize those glue code as much as possible; I need another
try to check if we can remove more or eliminate completely by other
ways.

-- Hajime

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v4 02/25] um lkl: architecture skeleton for Linux kernel library
  2020-03-30 22:12         ` Richard Weinberger
@ 2020-03-31  7:08           ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-31  7:08 UTC (permalink / raw)
  To: richard.weinberger
  Cc: johannes, linux-um, linux-arch, levex, mattator, cem,
	tavi.purdila, staal1978, motomuman, jiangshanlai, retrage01,
	petrosagg, edisonmcastro, xiaoj, mark, linux-kernel-library,
	pscollins, phh, sigmaepsilon92, luca.dariz, liuyuan


Hello Johannes, Richard

On Tue, 31 Mar 2020 07:12:38 +0900,
Richard Weinberger wrote:
> 
> Johannes, Hajime,
> 
> On Mon, Mar 30, 2020 at 11:53 PM Johannes Berg
> <johannes@sipsolutions.net> wrote:
> >
> > On Mon, 2020-03-30 at 23:45 +0900, Hajime Tazaki wrote:
> > > From: Octavian Purdila <tavi.purdila@gmail.com>
> > >
> > > Adds the LKL Kconfig, vmlinux linker script, basic architecture
> > > headers and miscellaneous basic functions or stubs such as
> > > dump_stack(), show_regs() and cpuinfo proc ops.
> > >
> > > The headers we introduce in this patch are simple wrappers to the
> > > asm-generic headers or stubs for things we don't support, such as
> > > ptrace, DMA, signals, ELF handling and low level processor operations.
> > >
> > > The kernel configuration is automatically updated to reflect the
> > > endianness of the host, 64bit support or the output format for
> > > vmlinux's linker script. We do this by looking at the ld's default
> > > output format.
> >
> > Can you elaborate what the plan is here?

I always wrote down the current plan that I have in my mind in the
cover letter (patch 00); please take a look at the "Milestone" block.

Quoted:

> Milestone
> =========
> This patches is a first step toward upstreaming *library mode* of Linux kernel,
> but we think we need to have several steps toward our goal, describing in the
> below.
> 
> 1. Put LKL code under arch/um (arch/um/lkl), and build it in a separate way
> from UML.
> 2. Use UML driver implementations in LKL as a minimum set of patches
> -  Only support x86 sub architecture (dependency to UML drivers)
> 3. Support broader host supports
> - add virtio device features
> 
> For the step 1, we put LKL as one of UMMODE in order to make less effort to
> integrate (make ARCH=um UMMODE=library).  The modification to existing UML
> code is trying to be minimized.


> > I mean, you're not actually "unifying" anything with ARCH=um, you're
> > just basically splitting ARCH=um into ARCH=um-um and ARCH=um-lkl or
> > something. Is there much point?

The current step (1 in the milestone) tries to cover this goal:
splitting ARCH=um into UMMODE_KERNEL and UMMODE_LIB.

> > Even the basic underlying building blocks are _very_ different, e.g. in
> > UML the host interface is just a bunch of functions that must be
> > implemented (os_*()), while you have a "struct lkl_host_operations". If
> > we can't even unify at that trivial level, is there any point in it at
> > all? I'm not even really sure what those ops are used for - are all of
> > those things that the *application* using LKL necessarily must provide?
> >
> > Similarly with the IRQ routing mechanism - two completely different
> > concepts. Where's the "unifying"?
> >
> > Ultimately, I can see two ways this goes:
> >
> > 1) We give up, and get ARCH=lkl, sharing just (some of) the drivers
> >    while moving them into the right drivers/somewhere/ place. Even that
> >    looks somewhat awkward looking at the later patches in this set, but
> >    seems like that at *least* should be done.
> 
> Yeah, this would be a goal.
> UML and LKL are quite different but they should share at least their userspace
> drivers.
> I also don't mind if we don't share every driver at the beginning but
> it should be
> a feasible goal for the future.

Sharing drivers code is also included in this patchset, step 2 in the
milestone.

I was thinking that implementing os_*() functions with lkl_host_ops
would be the further goal (e.g., step 3 or 4).

> > 2) Ideally, instead, we actually unify: LKL grows support for userspace
> >    processes using UML infrastructure, the "in-kernel" IRQ mechanisms
> >    are unified, UML stuff moves into lkl-ops, and the UML binary
> >    basically becomes a user of the LKL library to start everything up.
> >    There may be some bits remaining that are just not interesting (e.g.
> >    some drivers you don't care about would continue to make direct calls
> >    to the user side instead of lkl-ops, and then they're just not
> >    compatible with lkl, only with the uml special case of lkl), but then
> >    things are clean.
> 
> A few months ago I though this is doable but now I'm not so sure anymore.

For the part of (2) which Johannes pointed out (I mean the part "UML
stuff moves into lkl-ops"), I become to think that implementing os_*()
functions using lkl_host_ops would be also interesting if those
re-implementation makes the glue code eliminated.

I'll work on that.

> > Now, of course it seems like (2) would actually be better - LKL would
> > actually get support for userspace processes using UML's tricks, most of
> > the code is unified, and even LKL's users can take advantage of new
> > things. At the same time, all of the duplication is avoided.
> >
> > However, I just don't know if it's actually _possible_ to do that
> > though. Conceptually, it seems like it should be - why shouldn't a
> > library be able to spawn other userspace processes - I mean, it's not
> > really in the model that LKL really _wants_ since it's supposed to offer
> > the syscall API, but you could make a "bool run_normal_init" or
> > something in the lkl-ops for the user of the library to determine what
> > should happen, right?
> >
> > However, there clearly are huge differences between LKL and UML in all
> > respects, and I don't know if this wouldn't conflict with the library
> > model, e.g. there may be linker issues etc. Or maybe the specific UML
> > interrupt handling is required in UML and cannot be done in LKL (but
> > then perhaps it could be put into the hypothetical UML-application vs.
> > UML-the-LKL-library?)


For the other part of (2), I agree that your definition of the
unification will be the best and final goal of this integration.

But, especially the support for UML userspace processes in LKL is not
easy as far as I can see, at least the current LKL doesn't have, I
treat those extensions (from the LKL pov) as further goals.

Or the title of the cover letter is somehow overstatement: instead,
"Minimum integration of UML with LKL" or something like this would be
better ?

> > Ultimately, personally I think it's going to have to be one or the other
> > of those two options though, I don't really see much value in what
> > you're doing here in the patchset now. This way just messes up
> > everything, it's not clear what's UML and what's LKL, and they're
> > intertwined with ifdefs everywhere; just look at where you have to add
> > ifdefs in patch 23 - how would anyone later understand which part gets
> > compiled for which of them?

Since the patchset of LKL is relatively huge, I was trying to make a
minimum patchset for the first step.  Because having #ifdefs and glue
code makes existing arch/um code untouched, I took this way to make
the patchset as small as possible.

But if this is not the case, I will look for the other way and touch
arch/um code to fit the current LKL implementation.

What do you think ?

# For the particular ifdefs in patch 23, I will try to make it easy to
  understand.

> > johannes
> >
> > PS: actually having something like lkl-ops in UML would've been nice
> > also for my "time-travel" work, since it neatly abstracts the timers
> > out. I do wonder a bit about the overhead of jumping through function
> > pointers all the time though, it may be worth rethinking that overall
> > anyway!
> 
> Agreed. UML can also learn from LKL. :-)

Ditto;  this is also my wish.

-- Hajime

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v4 02/25] um lkl: architecture skeleton for Linux kernel library
@ 2020-03-31  7:08           ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-03-31  7:08 UTC (permalink / raw)
  To: richard.weinberger
  Cc: linux-arch, levex, mattator, cem, tavi.purdila, jiangshanlai,
	staal1978, motomuman, linux-um, retrage01, petrosagg, liuyuan,
	phh, xiaoj, mark, pscollins, linux-kernel-library, johannes,
	sigmaepsilon92, luca.dariz, edisonmcastro


Hello Johannes, Richard

On Tue, 31 Mar 2020 07:12:38 +0900,
Richard Weinberger wrote:
> 
> Johannes, Hajime,
> 
> On Mon, Mar 30, 2020 at 11:53 PM Johannes Berg
> <johannes@sipsolutions.net> wrote:
> >
> > On Mon, 2020-03-30 at 23:45 +0900, Hajime Tazaki wrote:
> > > From: Octavian Purdila <tavi.purdila@gmail.com>
> > >
> > > Adds the LKL Kconfig, vmlinux linker script, basic architecture
> > > headers and miscellaneous basic functions or stubs such as
> > > dump_stack(), show_regs() and cpuinfo proc ops.
> > >
> > > The headers we introduce in this patch are simple wrappers to the
> > > asm-generic headers or stubs for things we don't support, such as
> > > ptrace, DMA, signals, ELF handling and low level processor operations.
> > >
> > > The kernel configuration is automatically updated to reflect the
> > > endianness of the host, 64bit support or the output format for
> > > vmlinux's linker script. We do this by looking at the ld's default
> > > output format.
> >
> > Can you elaborate what the plan is here?

I always wrote down the current plan that I have in my mind in the
cover letter (patch 00); please take a look at the "Milestone" block.

Quoted:

> Milestone
> =========
> This patches is a first step toward upstreaming *library mode* of Linux kernel,
> but we think we need to have several steps toward our goal, describing in the
> below.
> 
> 1. Put LKL code under arch/um (arch/um/lkl), and build it in a separate way
> from UML.
> 2. Use UML driver implementations in LKL as a minimum set of patches
> -  Only support x86 sub architecture (dependency to UML drivers)
> 3. Support broader host supports
> - add virtio device features
> 
> For the step 1, we put LKL as one of UMMODE in order to make less effort to
> integrate (make ARCH=um UMMODE=library).  The modification to existing UML
> code is trying to be minimized.


> > I mean, you're not actually "unifying" anything with ARCH=um, you're
> > just basically splitting ARCH=um into ARCH=um-um and ARCH=um-lkl or
> > something. Is there much point?

The current step (1 in the milestone) tries to cover this goal:
splitting ARCH=um into UMMODE_KERNEL and UMMODE_LIB.

> > Even the basic underlying building blocks are _very_ different, e.g. in
> > UML the host interface is just a bunch of functions that must be
> > implemented (os_*()), while you have a "struct lkl_host_operations". If
> > we can't even unify at that trivial level, is there any point in it at
> > all? I'm not even really sure what those ops are used for - are all of
> > those things that the *application* using LKL necessarily must provide?
> >
> > Similarly with the IRQ routing mechanism - two completely different
> > concepts. Where's the "unifying"?
> >
> > Ultimately, I can see two ways this goes:
> >
> > 1) We give up, and get ARCH=lkl, sharing just (some of) the drivers
> >    while moving them into the right drivers/somewhere/ place. Even that
> >    looks somewhat awkward looking at the later patches in this set, but
> >    seems like that at *least* should be done.
> 
> Yeah, this would be a goal.
> UML and LKL are quite different but they should share at least their userspace
> drivers.
> I also don't mind if we don't share every driver at the beginning but
> it should be
> a feasible goal for the future.

Sharing drivers code is also included in this patchset, step 2 in the
milestone.

I was thinking that implementing os_*() functions with lkl_host_ops
would be the further goal (e.g., step 3 or 4).

> > 2) Ideally, instead, we actually unify: LKL grows support for userspace
> >    processes using UML infrastructure, the "in-kernel" IRQ mechanisms
> >    are unified, UML stuff moves into lkl-ops, and the UML binary
> >    basically becomes a user of the LKL library to start everything up.
> >    There may be some bits remaining that are just not interesting (e.g.
> >    some drivers you don't care about would continue to make direct calls
> >    to the user side instead of lkl-ops, and then they're just not
> >    compatible with lkl, only with the uml special case of lkl), but then
> >    things are clean.
> 
> A few months ago I though this is doable but now I'm not so sure anymore.

For the part of (2) which Johannes pointed out (I mean the part "UML
stuff moves into lkl-ops"), I become to think that implementing os_*()
functions using lkl_host_ops would be also interesting if those
re-implementation makes the glue code eliminated.

I'll work on that.

> > Now, of course it seems like (2) would actually be better - LKL would
> > actually get support for userspace processes using UML's tricks, most of
> > the code is unified, and even LKL's users can take advantage of new
> > things. At the same time, all of the duplication is avoided.
> >
> > However, I just don't know if it's actually _possible_ to do that
> > though. Conceptually, it seems like it should be - why shouldn't a
> > library be able to spawn other userspace processes - I mean, it's not
> > really in the model that LKL really _wants_ since it's supposed to offer
> > the syscall API, but you could make a "bool run_normal_init" or
> > something in the lkl-ops for the user of the library to determine what
> > should happen, right?
> >
> > However, there clearly are huge differences between LKL and UML in all
> > respects, and I don't know if this wouldn't conflict with the library
> > model, e.g. there may be linker issues etc. Or maybe the specific UML
> > interrupt handling is required in UML and cannot be done in LKL (but
> > then perhaps it could be put into the hypothetical UML-application vs.
> > UML-the-LKL-library?)


For the other part of (2), I agree that your definition of the
unification will be the best and final goal of this integration.

But, especially the support for UML userspace processes in LKL is not
easy as far as I can see, at least the current LKL doesn't have, I
treat those extensions (from the LKL pov) as further goals.

Or the title of the cover letter is somehow overstatement: instead,
"Minimum integration of UML with LKL" or something like this would be
better ?

> > Ultimately, personally I think it's going to have to be one or the other
> > of those two options though, I don't really see much value in what
> > you're doing here in the patchset now. This way just messes up
> > everything, it's not clear what's UML and what's LKL, and they're
> > intertwined with ifdefs everywhere; just look at where you have to add
> > ifdefs in patch 23 - how would anyone later understand which part gets
> > compiled for which of them?

Since the patchset of LKL is relatively huge, I was trying to make a
minimum patchset for the first step.  Because having #ifdefs and glue
code makes existing arch/um code untouched, I took this way to make
the patchset as small as possible.

But if this is not the case, I will look for the other way and touch
arch/um code to fit the current LKL implementation.

What do you think ?

# For the particular ifdefs in patch 23, I will try to make it easy to
  understand.

> > johannes
> >
> > PS: actually having something like lkl-ops in UML would've been nice
> > also for my "time-travel" work, since it neatly abstracts the timers
> > out. I do wonder a bit about the overhead of jumping through function
> > pointers all the time though, it may be worth rethinking that overall
> > anyway!
> 
> Agreed. UML can also learn from LKL. :-)

Ditto;  this is also my wish.

-- Hajime

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v4 23/25] um lkl: add UML network driver for lkl
  2020-03-31  2:38         ` Hajime Tazaki
@ 2020-03-31 19:52           ` Johannes Berg
  -1 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-03-31 19:52 UTC (permalink / raw)
  To: Hajime Tazaki
  Cc: linux-um, tavi.purdila, linux-kernel-library, linux-arch, retrage01

On Tue, 2020-03-31 at 11:38 +0900, Hajime Tazaki wrote:
> Thanks for the comments, Johannes.
> 
> On Tue, 31 Mar 2020 06:31:15 +0900,
> Johannes Berg wrote:
> > 
> > > +++ b/arch/um/lkl/include/asm/irq.h
> > > @@ -2,6 +2,9 @@
> > >  #ifndef _ASM_LKL_IRQ_H
> > >  #define _ASM_LKL_IRQ_H
> > >  
> > > +/* pull UML's definitions */
> > > +#include "../../../include/asm/irq.h"
> > 
> > This is _really_ ugly.
> 
> Hmm, in previous patchset (until v3), I was using the worse approach
> (I thought) to avoid this include.
> 
> +KBUILD_CFLAGS  += -DTIMER_IRQ=0 -DUBD_IRQ=4 -DUM_ETH_IRQ=5 -DLAST_IRQ=15
> 
> And I thought the current way is better than before.

Yeah, ok, that's worse :)

But why is it even needed? It kinda seems to me that this means we're
not splitting the code well.

IMHO, if we even want to treat LKL/UML as sub-arches, then we should
still split the driver code off in a cleaner way, rather than linking
half here half there?

And maybe reorg the code... but I'll reply over in your other email
more.

> > > @@ -181,6 +196,11 @@ void init_IRQ(void)
> > >  	for (i = 0; i < NR_IRQS; i++)
> > >  		irq_set_chip_and_handler(i, &dummy_irq_chip, handle_simple_irq);
> > >  
> > > +#if defined(__linux) && (defined(__i386) || defined(__x86_64))
> > 
> > What's with all those ifdefs with this condition?
> 
> Same as above.
> but I agree that the ifdefs are cryptic; I'll try to make it more
> understandable if I use ifdefs.

I'm also generally not convinced that it's a good idea to sprinkle these
kinds of ifdefs over the place.

johannes

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v4 23/25] um lkl: add UML network driver for lkl
@ 2020-03-31 19:52           ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-03-31 19:52 UTC (permalink / raw)
  To: Hajime Tazaki
  Cc: tavi.purdila, linux-kernel-library, retrage01, linux-um, linux-arch

On Tue, 2020-03-31 at 11:38 +0900, Hajime Tazaki wrote:
> Thanks for the comments, Johannes.
> 
> On Tue, 31 Mar 2020 06:31:15 +0900,
> Johannes Berg wrote:
> > 
> > > +++ b/arch/um/lkl/include/asm/irq.h
> > > @@ -2,6 +2,9 @@
> > >  #ifndef _ASM_LKL_IRQ_H
> > >  #define _ASM_LKL_IRQ_H
> > >  
> > > +/* pull UML's definitions */
> > > +#include "../../../include/asm/irq.h"
> > 
> > This is _really_ ugly.
> 
> Hmm, in previous patchset (until v3), I was using the worse approach
> (I thought) to avoid this include.
> 
> +KBUILD_CFLAGS  += -DTIMER_IRQ=0 -DUBD_IRQ=4 -DUM_ETH_IRQ=5 -DLAST_IRQ=15
> 
> And I thought the current way is better than before.

Yeah, ok, that's worse :)

But why is it even needed? It kinda seems to me that this means we're
not splitting the code well.

IMHO, if we even want to treat LKL/UML as sub-arches, then we should
still split the driver code off in a cleaner way, rather than linking
half here half there?

And maybe reorg the code... but I'll reply over in your other email
more.

> > > @@ -181,6 +196,11 @@ void init_IRQ(void)
> > >  	for (i = 0; i < NR_IRQS; i++)
> > >  		irq_set_chip_and_handler(i, &dummy_irq_chip, handle_simple_irq);
> > >  
> > > +#if defined(__linux) && (defined(__i386) || defined(__x86_64))
> > 
> > What's with all those ifdefs with this condition?
> 
> Same as above.
> but I agree that the ifdefs are cryptic; I'll try to make it more
> understandable if I use ifdefs.

I'm also generally not convinced that it's a good idea to sprinkle these
kinds of ifdefs over the place.

johannes


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v4 02/25] um lkl: architecture skeleton for Linux kernel library
  2020-03-31  7:08           ` Hajime Tazaki
@ 2020-03-31 20:16             ` Johannes Berg
  -1 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-03-31 20:16 UTC (permalink / raw)
  To: Hajime Tazaki, richard.weinberger
  Cc: linux-um, linux-arch, levex, mattator, cem, tavi.purdila,
	staal1978, motomuman, jiangshanlai, retrage01, petrosagg,
	edisonmcastro, xiaoj, mark, linux-kernel-library, pscollins, phh,
	sigmaepsilon92, luca.dariz, liuyuan

Hi,

> I always wrote down the current plan that I have in my mind in the
> cover letter (patch 00); please take a look at the "Milestone" block.

Right, sorry.

> > This patches is a first step toward upstreaming *library mode* of Linux kernel,
> > but we think we need to have several steps toward our goal, describing in the
> > below.
> > 
> > 1. Put LKL code under arch/um (arch/um/lkl), and build it in a separate way
> > from UML.
> > 2. Use UML driver implementations in LKL as a minimum set of patches
> > -  Only support x86 sub architecture (dependency to UML drivers)
> > 3. Support broader host supports
> > - add virtio device features
> > 
> > For the step 1, we put LKL as one of UMMODE in order to make less effort to
> > integrate (make ARCH=um UMMODE=library).  The modification to existing UML
> > code is trying to be minimized.

> The current step (1 in the milestone) tries to cover this goal:
> splitting ARCH=um into UMMODE_KERNEL and UMMODE_LIB.

So maybe we're doing this backwards?

I mean ... you're trying to minimize the UML code changes, while I'm
sort of arguing for maximizing them, to achieve a cleaner split.

In a sense, I think if this is to happen, then we're in it for the long
haul. Meaning that we don't actually need all of this working
immediately.

So I think conceptually we should answer the questions that I raised
below first (basically a kind of "can it be done?" question), and then
work towards that goal? IMHO.

> > > 1) We give up, and get ARCH=lkl, sharing just (some of) the drivers
> > >    while moving them into the right drivers/somewhere/ place. Even that
> > >    looks somewhat awkward looking at the later patches in this set, but
> > >    seems like that at *least* should be done.
> > 
> > Yeah, this would be a goal.
> > UML and LKL are quite different but they should share at least their userspace
> > drivers.
> > I also don't mind if we don't share every driver at the beginning but
> > it should be
> > a feasible goal for the future.
> 
> Sharing drivers code is also included in this patchset, step 2 in the
> milestone.
> 
> I was thinking that implementing os_*() functions with lkl_host_ops
> would be the further goal (e.g., step 3 or 4).

Personally, I think this is backwards. That step is the actually
*interesting* part, because if this turns out not to be possible, then
we should pick option (1) instead of trying to do option (2), failing,
and leaving the code a mess (at least personally I think that after this
patchset, the code is kinda a mess with all the ifdefs, duplication,
etc.) Yes, I know you're in this for the long haul, but still - it'd be
a shame to have to do that.

So in a sense, I myself would actually prefer to have an LKL _without_
drivers, but integrated well with UML, over the one that you have now.

> > > 2) Ideally, instead, we actually unify: LKL grows support for userspace
> > >    processes using UML infrastructure, the "in-kernel" IRQ mechanisms
> > >    are unified, UML stuff moves into lkl-ops, and the UML binary
> > >    basically becomes a user of the LKL library to start everything up.
> > >    There may be some bits remaining that are just not interesting (e.g.
> > >    some drivers you don't care about would continue to make direct calls
> > >    to the user side instead of lkl-ops, and then they're just not
> > >    compatible with lkl, only with the uml special case of lkl), but then
> > >    things are clean.
> > 
> > A few months ago I though this is doable but now I'm not so sure anymore.
> 
> For the part of (2) which Johannes pointed out (I mean the part "UML
> stuff moves into lkl-ops"), I become to think that implementing os_*()
> functions using lkl_host_ops would be also interesting if those
> re-implementation makes the glue code eliminated.
> 
> I'll work on that.

Don't go too fast :-)

I really think that this only makes sense if we can also share much of
the other code, e.g. the interrupt processing, thread model, etc. If we
just share the lkl ops underneath, and then end up implementing two IRQ
models and all on top of those, IMHO we've won nothing.

So I (at least) really see it as a choice between these two options:

1) add LKL as arch/lkl/ and share the drivers, but not the arch code

2) really unify LKL and UML, and have them share the arch code, and make
   UML a special case of LKL, but not in the sense that it has vastly
   different arch code (like special interrupt handling, etc.)

Now, you allude to the fact that UML is pretty much x86 only, and
perhaps that's a point where we can do (2) but only support userspace
programs on x86, or such? I don't know where the host architecture
actually comes in much in UML, and where that may or may not be the case
in LKL.

> For the other part of (2), I agree that your definition of the
> unification will be the best and final goal of this integration.

Fair, but the problem is that we have to decide *now* whether it's
actually possible or not. If not, then IMHO it's a bad choice to even
put LKL under arch/um/.

> But, especially the support for UML userspace processes in LKL is not
> easy as far as I can see

OK, I'll bite - why not? I mean, what's different enough in LKL and UML
to make this "not easy"?

I'm not trying to paint you into a corner here with that, I'm just
trying to understand the innards of LKL better. I have a _bit_ of a
grasp of the UML internals by now, but of course not LKL.

So where do they differ? Conceptually, they seem very similar, but the
details actually are different.

But I have the same question on e.g. the IRQ model. I mean, OK, I
understand that LKL started from a different base and all, but is it
actually *necessary* for LKL to have a different IRQ model? Or is that
"just" intertia?

> Or the title of the cover letter is somehow overstatement: instead,
> "Minimum integration of UML with LKL" or something like this would be
> better ?

Heh, well, doesn't really matter?

But again, there are a few different aspects here:
 - what's technically feasible
 - what this patchset achieves
 - where we want to be in the end

I think right now these are diverging enough that we can't even answer
that last question, much less find the road to get there.

> Since the patchset of LKL is relatively huge, I was trying to make a
> minimum patchset for the first step.  Because having #ifdefs and glue
> code makes existing arch/um code untouched, I took this way to make
> the patchset as small as possible.
> 
> But if this is not the case, I will look for the other way and touch
> arch/um code to fit the current LKL implementation.
> 
> What do you think ?

I think that'd be fine, if indeed that's what we want to do.

I really think we're beating around the bush, and need to first figure
out the technical differences between UML and LKL and decide between the
options (1) and (2) above. Maybe there's a compromise there somewhere,
where some small bits of code still _are_ different, but IMHO having two
(IRQ, thread, memory) models, two host interfaces (lkl-ops vs. os_*
functions), even two include/asm/ source trees (and so on) is not
appropriate.

This may take some patches, and some experimentation. I'd leave drivers
out of this initially - you should be able to test LKL with something
simpler, right? The API surface is basically the syscall interface as
functions, so you can start the library and call something simple
initially? Though I guess you need some driver for the IRQ model to make
sense, etc.

And like I said before, that decision will frame everything else. I
really don't think we can make significant progress here without having
decided whether this is possible.

Perhaps UML *can* become a "special case" of LKL, with a special API
function (that's not part of the syscall surface) to "boot(cmdline)" or
something. But if it can't, and has to remain as separated as the two
are today, I would argue we're better off just not calling them the same
architecture.

johannes

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v4 02/25] um lkl: architecture skeleton for Linux kernel library
@ 2020-03-31 20:16             ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-03-31 20:16 UTC (permalink / raw)
  To: Hajime Tazaki, richard.weinberger
  Cc: linux-arch, levex, mattator, cem, tavi.purdila, jiangshanlai,
	staal1978, motomuman, linux-um, retrage01, petrosagg, liuyuan,
	xiaoj, mark, pscollins, linux-kernel-library, phh,
	sigmaepsilon92, luca.dariz, edisonmcastro

Hi,

> I always wrote down the current plan that I have in my mind in the
> cover letter (patch 00); please take a look at the "Milestone" block.

Right, sorry.

> > This patches is a first step toward upstreaming *library mode* of Linux kernel,
> > but we think we need to have several steps toward our goal, describing in the
> > below.
> > 
> > 1. Put LKL code under arch/um (arch/um/lkl), and build it in a separate way
> > from UML.
> > 2. Use UML driver implementations in LKL as a minimum set of patches
> > -  Only support x86 sub architecture (dependency to UML drivers)
> > 3. Support broader host supports
> > - add virtio device features
> > 
> > For the step 1, we put LKL as one of UMMODE in order to make less effort to
> > integrate (make ARCH=um UMMODE=library).  The modification to existing UML
> > code is trying to be minimized.

> The current step (1 in the milestone) tries to cover this goal:
> splitting ARCH=um into UMMODE_KERNEL and UMMODE_LIB.

So maybe we're doing this backwards?

I mean ... you're trying to minimize the UML code changes, while I'm
sort of arguing for maximizing them, to achieve a cleaner split.

In a sense, I think if this is to happen, then we're in it for the long
haul. Meaning that we don't actually need all of this working
immediately.

So I think conceptually we should answer the questions that I raised
below first (basically a kind of "can it be done?" question), and then
work towards that goal? IMHO.

> > > 1) We give up, and get ARCH=lkl, sharing just (some of) the drivers
> > >    while moving them into the right drivers/somewhere/ place. Even that
> > >    looks somewhat awkward looking at the later patches in this set, but
> > >    seems like that at *least* should be done.
> > 
> > Yeah, this would be a goal.
> > UML and LKL are quite different but they should share at least their userspace
> > drivers.
> > I also don't mind if we don't share every driver at the beginning but
> > it should be
> > a feasible goal for the future.
> 
> Sharing drivers code is also included in this patchset, step 2 in the
> milestone.
> 
> I was thinking that implementing os_*() functions with lkl_host_ops
> would be the further goal (e.g., step 3 or 4).

Personally, I think this is backwards. That step is the actually
*interesting* part, because if this turns out not to be possible, then
we should pick option (1) instead of trying to do option (2), failing,
and leaving the code a mess (at least personally I think that after this
patchset, the code is kinda a mess with all the ifdefs, duplication,
etc.) Yes, I know you're in this for the long haul, but still - it'd be
a shame to have to do that.

So in a sense, I myself would actually prefer to have an LKL _without_
drivers, but integrated well with UML, over the one that you have now.

> > > 2) Ideally, instead, we actually unify: LKL grows support for userspace
> > >    processes using UML infrastructure, the "in-kernel" IRQ mechanisms
> > >    are unified, UML stuff moves into lkl-ops, and the UML binary
> > >    basically becomes a user of the LKL library to start everything up.
> > >    There may be some bits remaining that are just not interesting (e.g.
> > >    some drivers you don't care about would continue to make direct calls
> > >    to the user side instead of lkl-ops, and then they're just not
> > >    compatible with lkl, only with the uml special case of lkl), but then
> > >    things are clean.
> > 
> > A few months ago I though this is doable but now I'm not so sure anymore.
> 
> For the part of (2) which Johannes pointed out (I mean the part "UML
> stuff moves into lkl-ops"), I become to think that implementing os_*()
> functions using lkl_host_ops would be also interesting if those
> re-implementation makes the glue code eliminated.
> 
> I'll work on that.

Don't go too fast :-)

I really think that this only makes sense if we can also share much of
the other code, e.g. the interrupt processing, thread model, etc. If we
just share the lkl ops underneath, and then end up implementing two IRQ
models and all on top of those, IMHO we've won nothing.

So I (at least) really see it as a choice between these two options:

1) add LKL as arch/lkl/ and share the drivers, but not the arch code

2) really unify LKL and UML, and have them share the arch code, and make
   UML a special case of LKL, but not in the sense that it has vastly
   different arch code (like special interrupt handling, etc.)

Now, you allude to the fact that UML is pretty much x86 only, and
perhaps that's a point where we can do (2) but only support userspace
programs on x86, or such? I don't know where the host architecture
actually comes in much in UML, and where that may or may not be the case
in LKL.

> For the other part of (2), I agree that your definition of the
> unification will be the best and final goal of this integration.

Fair, but the problem is that we have to decide *now* whether it's
actually possible or not. If not, then IMHO it's a bad choice to even
put LKL under arch/um/.

> But, especially the support for UML userspace processes in LKL is not
> easy as far as I can see

OK, I'll bite - why not? I mean, what's different enough in LKL and UML
to make this "not easy"?

I'm not trying to paint you into a corner here with that, I'm just
trying to understand the innards of LKL better. I have a _bit_ of a
grasp of the UML internals by now, but of course not LKL.

So where do they differ? Conceptually, they seem very similar, but the
details actually are different.

But I have the same question on e.g. the IRQ model. I mean, OK, I
understand that LKL started from a different base and all, but is it
actually *necessary* for LKL to have a different IRQ model? Or is that
"just" intertia?

> Or the title of the cover letter is somehow overstatement: instead,
> "Minimum integration of UML with LKL" or something like this would be
> better ?

Heh, well, doesn't really matter?

But again, there are a few different aspects here:
 - what's technically feasible
 - what this patchset achieves
 - where we want to be in the end

I think right now these are diverging enough that we can't even answer
that last question, much less find the road to get there.

> Since the patchset of LKL is relatively huge, I was trying to make a
> minimum patchset for the first step.  Because having #ifdefs and glue
> code makes existing arch/um code untouched, I took this way to make
> the patchset as small as possible.
> 
> But if this is not the case, I will look for the other way and touch
> arch/um code to fit the current LKL implementation.
> 
> What do you think ?

I think that'd be fine, if indeed that's what we want to do.

I really think we're beating around the bush, and need to first figure
out the technical differences between UML and LKL and decide between the
options (1) and (2) above. Maybe there's a compromise there somewhere,
where some small bits of code still _are_ different, but IMHO having two
(IRQ, thread, memory) models, two host interfaces (lkl-ops vs. os_*
functions), even two include/asm/ source trees (and so on) is not
appropriate.

This may take some patches, and some experimentation. I'd leave drivers
out of this initially - you should be able to test LKL with something
simpler, right? The API surface is basically the syscall interface as
functions, so you can start the library and call something simple
initially? Though I guess you need some driver for the IRQ model to make
sense, etc.

And like I said before, that decision will frame everything else. I
really don't think we can make significant progress here without having
decided whether this is possible.

Perhaps UML *can* become a "special case" of LKL, with a special API
function (that's not part of the syscall surface) to "boot(cmdline)" or
something. But if it can't, and has to remain as separated as the two
are today, I would argue we're better off just not calling them the same
architecture.

johannes


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v4 02/25] um lkl: architecture skeleton for Linux kernel library
  2020-03-31 20:16             ` Johannes Berg
@ 2020-04-02  6:44               ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-04-02  6:44 UTC (permalink / raw)
  To: johannes
  Cc: richard.weinberger, linux-arch, levex, mattator, cem,
	tavi.purdila, jiangshanlai, staal1978, motomuman, linux-um,
	retrage01, petrosagg, liuyuan, mark, pscollins,
	linux-kernel-library, phh, sigmaepsilon92, luca.dariz,
	edisonmcastro


Hello,

# dropped an address from Cc as it's not reachable.

On Wed, 01 Apr 2020 05:16:58 +0900,
Johannes Berg wrote:

> > > For the step 1, we put LKL as one of UMMODE in order to make less effort to
> > > integrate (make ARCH=um UMMODE=library).  The modification to existing UML
> > > code is trying to be minimized.
> 
> > The current step (1 in the milestone) tries to cover this goal:
> > splitting ARCH=um into UMMODE_KERNEL and UMMODE_LIB.
> 
> So maybe we're doing this backwards?
> 
> I mean ... you're trying to minimize the UML code changes, while I'm
> sort of arguing for maximizing them, to achieve a cleaner split.

I see the point.

> In a sense, I think if this is to happen, then we're in it for the long
> haul. Meaning that we don't actually need all of this working
> immediately.
> 
> So I think conceptually we should answer the questions that I raised
> below first (basically a kind of "can it be done?" question), and then
> work towards that goal? IMHO.

okay, agree.

> > > > 1) We give up, and get ARCH=lkl, sharing just (some of) the drivers
> > > >    while moving them into the right drivers/somewhere/ place. Even that
> > > >    looks somewhat awkward looking at the later patches in this set, but
> > > >    seems like that at *least* should be done.
> > > 
> > > Yeah, this would be a goal.
> > > UML and LKL are quite different but they should share at least their userspace
> > > drivers.
> > > I also don't mind if we don't share every driver at the beginning but
> > > it should be
> > > a feasible goal for the future.
> > 
> > Sharing drivers code is also included in this patchset, step 2 in the
> > milestone.
> > 
> > I was thinking that implementing os_*() functions with lkl_host_ops
> > would be the further goal (e.g., step 3 or 4).
> 
> Personally, I think this is backwards. That step is the actually
> *interesting* part, because if this turns out not to be possible, then
> we should pick option (1) instead of trying to do option (2), failing,
> and leaving the code a mess (at least personally I think that after this
> patchset, the code is kinda a mess with all the ifdefs, duplication,
> etc.) Yes, I know you're in this for the long haul, but still - it'd be
> a shame to have to do that.
> 
> So in a sense, I myself would actually prefer to have an LKL _without_
> drivers, but integrated well with UML, over the one that you have now.

LKL without drivers might be nothing, but let's see if this will end a
clean, and minimal viable patchset.

> > > > 2) Ideally, instead, we actually unify: LKL grows support for userspace
> > > >    processes using UML infrastructure, the "in-kernel" IRQ mechanisms
> > > >    are unified, UML stuff moves into lkl-ops, and the UML binary
> > > >    basically becomes a user of the LKL library to start everything up.
> > > >    There may be some bits remaining that are just not interesting (e.g.
> > > >    some drivers you don't care about would continue to make direct calls
> > > >    to the user side instead of lkl-ops, and then they're just not
> > > >    compatible with lkl, only with the uml special case of lkl), but then
> > > >    things are clean.
> > > 
> > > A few months ago I though this is doable but now I'm not so sure anymore.
> > 
> > For the part of (2) which Johannes pointed out (I mean the part "UML
> > stuff moves into lkl-ops"), I become to think that implementing os_*()
> > functions using lkl_host_ops would be also interesting if those
> > re-implementation makes the glue code eliminated.
> > 
> > I'll work on that.
> 
> Don't go too fast :-)
> 
> I really think that this only makes sense if we can also share much of
> the other code, e.g. the interrupt processing, thread model, etc. If we
> just share the lkl ops underneath, and then end up implementing two IRQ
> models and all on top of those, IMHO we've won nothing.
> 
> So I (at least) really see it as a choice between these two options:
> 
> 1) add LKL as arch/lkl/ and share the drivers, but not the arch code
> 
> 2) really unify LKL and UML, and have them share the arch code, and make
>    UML a special case of LKL, but not in the sense that it has vastly
>    different arch code (like special interrupt handling, etc.)

1) is not an option; we discussed this before not to have similar
archs in different directories.

# Richard, correct me if I'm wrong.

2) may be doable, the followings would be the list of things what we need:
- unify the driver codes
- unify interrupt handling, need to be x86/linux-independent
- unify thread model (i.e., struct thread_info)
- unify code scheduling  (e.g., __switch_to())
- unify memory management (mmu v.s. nommu)
- unify host interface (os_*() v.s. lkl_host_ops)
- support multiple syscall handlings (ptrace-based interception,
  direct func call, etc)
- (may still miss something..)

Those are actually the list of differences between UML and LKL.
And each item is inter-related: interrupt handling depends on how the
code/thread is executed, thread implementation interacts with memory
management, etc.

Thus for instance, unifying IRQ mechanism may involve several points
of the above list to be reorganized/reimplemented UML code to support
the library mode.  I thought this makes broader changes to UML, which
I was trying to avoid in v4 patch.

> Now, you allude to the fact that UML is pretty much x86 only, and
> perhaps that's a point where we can do (2) but only support userspace
> programs on x86, or such?

x86/linux only (LKL also has x86/win32 support).

> I don't know where the host architecture
> actually comes in much in UML, and where that may or may not be the case
> in LKL.
> 
> > For the other part of (2), I agree that your definition of the
> > unification will be the best and final goal of this integration.
> 
> Fair, but the problem is that we have to decide *now* whether it's
> actually possible or not. If not, then IMHO it's a bad choice to even
> put LKL under arch/um/.
> 
> > But, especially the support for UML userspace processes in LKL is not
> > easy as far as I can see
> 
> OK, I'll bite - why not? I mean, what's different enough in LKL and UML
> to make this "not easy"?
> 
> I'm not trying to paint you into a corner here with that, I'm just
> trying to understand the innards of LKL better. I have a _bit_ of a
> grasp of the UML internals by now, but of course not LKL.
> 
> So where do they differ? Conceptually, they seem very similar, but the
> details actually are different.
> 

The differences (and also conflicts) between UML and LKL are pretty
much the unification list of above 2).

The current LKL design assumes that there is only a single
process/application (but can have multiple threads) in a single LKL
kernel instance.

This assumption makes the design simple in several places of the
kernel:

- no memory protection across multiple users/processes needed
- no address space separation between user- and kernel-code

as a result, userspace code can directly call syscalls as function
calls.

This part is needed to expand more to support multiple processes
running/spawned (as UML does), which I mentioned the current LKL
doesn't have.

> But I have the same question on e.g. the IRQ model. I mean, OK, I
> understand that LKL started from a different base and all, but is it
> actually *necessary* for LKL to have a different IRQ model? Or is that
> "just" intertia?

As for the interrupt handling, the currently LKL's interrupt is
triggered by the function call (lkl_trigger_irq or equivalent) while
UML's one is triggered by fd notification (epoll).
# LKL actually uses fd events but encapsulates into the lkl_host_ops.

So now we have three options for interrupt handling:
- use UML IRQ model in LKL (and UML)
- use LKL IRQ model in UML (and LKL)
- use two IRQ models (the current v4 patch)

I gave up to take the 1st approach as it drops host-independent
characteristics of LKL, thus took 3rd approach.  So to answer your
question, yes, it was needed to have a different IRQ model.

But maybe I should try the 2nd approach as well to avoid duplication.
I will conduct another experiment if this sounds the right direction.

> > Or the title of the cover letter is somehow overstatement: instead,
> > "Minimum integration of UML with LKL" or something like this would be
> > better ?
> 
> Heh, well, doesn't really matter?
>
> But again, there are a few different aspects here:
>  - what's technically feasible
>  - what this patchset achieves
>  - where we want to be in the end
> 
> I think right now these are diverging enough that we can't even answer
> that last question, much less find the road to get there.

I think the option 2) is the final goal.

> > Since the patchset of LKL is relatively huge, I was trying to make a
> > minimum patchset for the first step.  Because having #ifdefs and glue
> > code makes existing arch/um code untouched, I took this way to make
> > the patchset as small as possible.
> > 
> > But if this is not the case, I will look for the other way and touch
> > arch/um code to fit the current LKL implementation.
> > 
> > What do you think ?
> 
> I think that'd be fine, if indeed that's what we want to do.
> 
> I really think we're beating around the bush, and need to first figure
> out the technical differences between UML and LKL and decide between the
> options (1) and (2) above. Maybe there's a compromise there somewhere,
> where some small bits of code still _are_ different, but IMHO having two
> (IRQ, thread, memory) models, two host interfaces (lkl-ops vs. os_*
> functions), even two include/asm/ source trees (and so on) is not
> appropriate.
> 
> This may take some patches, and some experimentation. I'd leave drivers
> out of this initially - you should be able to test LKL with something
> simpler, right? The API surface is basically the syscall interface as
> functions, so you can start the library and call something simple
> initially? Though I guess you need some driver for the IRQ model to make
> sense, etc.

okay.

> And like I said before, that decision will frame everything else. I
> really don't think we can make significant progress here without having
> decided whether this is possible.
>
> Perhaps UML *can* become a "special case" of LKL, with a special API
> function (that's not part of the syscall surface) to "boot(cmdline)" or
> something. But if it can't, and has to remain as separated as the two
> are today, I would argue we're better off just not calling them the same
> architecture.

I agree with this if the unification has all completed.

Thanks,
-- Hajime

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v4 02/25] um lkl: architecture skeleton for Linux kernel library
@ 2020-04-02  6:44               ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-04-02  6:44 UTC (permalink / raw)
  To: johannes
  Cc: linux-arch, levex, mattator, cem, richard.weinberger, linux-um,
	staal1978, motomuman, jiangshanlai, retrage01, petrosagg,
	tavi.purdila, mark, linux-kernel-library, edisonmcastro,
	pscollins, phh, sigmaepsilon92, luca.dariz, liuyuan


Hello,

# dropped an address from Cc as it's not reachable.

On Wed, 01 Apr 2020 05:16:58 +0900,
Johannes Berg wrote:

> > > For the step 1, we put LKL as one of UMMODE in order to make less effort to
> > > integrate (make ARCH=um UMMODE=library).  The modification to existing UML
> > > code is trying to be minimized.
> 
> > The current step (1 in the milestone) tries to cover this goal:
> > splitting ARCH=um into UMMODE_KERNEL and UMMODE_LIB.
> 
> So maybe we're doing this backwards?
> 
> I mean ... you're trying to minimize the UML code changes, while I'm
> sort of arguing for maximizing them, to achieve a cleaner split.

I see the point.

> In a sense, I think if this is to happen, then we're in it for the long
> haul. Meaning that we don't actually need all of this working
> immediately.
> 
> So I think conceptually we should answer the questions that I raised
> below first (basically a kind of "can it be done?" question), and then
> work towards that goal? IMHO.

okay, agree.

> > > > 1) We give up, and get ARCH=lkl, sharing just (some of) the drivers
> > > >    while moving them into the right drivers/somewhere/ place. Even that
> > > >    looks somewhat awkward looking at the later patches in this set, but
> > > >    seems like that at *least* should be done.
> > > 
> > > Yeah, this would be a goal.
> > > UML and LKL are quite different but they should share at least their userspace
> > > drivers.
> > > I also don't mind if we don't share every driver at the beginning but
> > > it should be
> > > a feasible goal for the future.
> > 
> > Sharing drivers code is also included in this patchset, step 2 in the
> > milestone.
> > 
> > I was thinking that implementing os_*() functions with lkl_host_ops
> > would be the further goal (e.g., step 3 or 4).
> 
> Personally, I think this is backwards. That step is the actually
> *interesting* part, because if this turns out not to be possible, then
> we should pick option (1) instead of trying to do option (2), failing,
> and leaving the code a mess (at least personally I think that after this
> patchset, the code is kinda a mess with all the ifdefs, duplication,
> etc.) Yes, I know you're in this for the long haul, but still - it'd be
> a shame to have to do that.
> 
> So in a sense, I myself would actually prefer to have an LKL _without_
> drivers, but integrated well with UML, over the one that you have now.

LKL without drivers might be nothing, but let's see if this will end a
clean, and minimal viable patchset.

> > > > 2) Ideally, instead, we actually unify: LKL grows support for userspace
> > > >    processes using UML infrastructure, the "in-kernel" IRQ mechanisms
> > > >    are unified, UML stuff moves into lkl-ops, and the UML binary
> > > >    basically becomes a user of the LKL library to start everything up.
> > > >    There may be some bits remaining that are just not interesting (e.g.
> > > >    some drivers you don't care about would continue to make direct calls
> > > >    to the user side instead of lkl-ops, and then they're just not
> > > >    compatible with lkl, only with the uml special case of lkl), but then
> > > >    things are clean.
> > > 
> > > A few months ago I though this is doable but now I'm not so sure anymore.
> > 
> > For the part of (2) which Johannes pointed out (I mean the part "UML
> > stuff moves into lkl-ops"), I become to think that implementing os_*()
> > functions using lkl_host_ops would be also interesting if those
> > re-implementation makes the glue code eliminated.
> > 
> > I'll work on that.
> 
> Don't go too fast :-)
> 
> I really think that this only makes sense if we can also share much of
> the other code, e.g. the interrupt processing, thread model, etc. If we
> just share the lkl ops underneath, and then end up implementing two IRQ
> models and all on top of those, IMHO we've won nothing.
> 
> So I (at least) really see it as a choice between these two options:
> 
> 1) add LKL as arch/lkl/ and share the drivers, but not the arch code
> 
> 2) really unify LKL and UML, and have them share the arch code, and make
>    UML a special case of LKL, but not in the sense that it has vastly
>    different arch code (like special interrupt handling, etc.)

1) is not an option; we discussed this before not to have similar
archs in different directories.

# Richard, correct me if I'm wrong.

2) may be doable, the followings would be the list of things what we need:
- unify the driver codes
- unify interrupt handling, need to be x86/linux-independent
- unify thread model (i.e., struct thread_info)
- unify code scheduling  (e.g., __switch_to())
- unify memory management (mmu v.s. nommu)
- unify host interface (os_*() v.s. lkl_host_ops)
- support multiple syscall handlings (ptrace-based interception,
  direct func call, etc)
- (may still miss something..)

Those are actually the list of differences between UML and LKL.
And each item is inter-related: interrupt handling depends on how the
code/thread is executed, thread implementation interacts with memory
management, etc.

Thus for instance, unifying IRQ mechanism may involve several points
of the above list to be reorganized/reimplemented UML code to support
the library mode.  I thought this makes broader changes to UML, which
I was trying to avoid in v4 patch.

> Now, you allude to the fact that UML is pretty much x86 only, and
> perhaps that's a point where we can do (2) but only support userspace
> programs on x86, or such?

x86/linux only (LKL also has x86/win32 support).

> I don't know where the host architecture
> actually comes in much in UML, and where that may or may not be the case
> in LKL.
> 
> > For the other part of (2), I agree that your definition of the
> > unification will be the best and final goal of this integration.
> 
> Fair, but the problem is that we have to decide *now* whether it's
> actually possible or not. If not, then IMHO it's a bad choice to even
> put LKL under arch/um/.
> 
> > But, especially the support for UML userspace processes in LKL is not
> > easy as far as I can see
> 
> OK, I'll bite - why not? I mean, what's different enough in LKL and UML
> to make this "not easy"?
> 
> I'm not trying to paint you into a corner here with that, I'm just
> trying to understand the innards of LKL better. I have a _bit_ of a
> grasp of the UML internals by now, but of course not LKL.
> 
> So where do they differ? Conceptually, they seem very similar, but the
> details actually are different.
> 

The differences (and also conflicts) between UML and LKL are pretty
much the unification list of above 2).

The current LKL design assumes that there is only a single
process/application (but can have multiple threads) in a single LKL
kernel instance.

This assumption makes the design simple in several places of the
kernel:

- no memory protection across multiple users/processes needed
- no address space separation between user- and kernel-code

as a result, userspace code can directly call syscalls as function
calls.

This part is needed to expand more to support multiple processes
running/spawned (as UML does), which I mentioned the current LKL
doesn't have.

> But I have the same question on e.g. the IRQ model. I mean, OK, I
> understand that LKL started from a different base and all, but is it
> actually *necessary* for LKL to have a different IRQ model? Or is that
> "just" intertia?

As for the interrupt handling, the currently LKL's interrupt is
triggered by the function call (lkl_trigger_irq or equivalent) while
UML's one is triggered by fd notification (epoll).
# LKL actually uses fd events but encapsulates into the lkl_host_ops.

So now we have three options for interrupt handling:
- use UML IRQ model in LKL (and UML)
- use LKL IRQ model in UML (and LKL)
- use two IRQ models (the current v4 patch)

I gave up to take the 1st approach as it drops host-independent
characteristics of LKL, thus took 3rd approach.  So to answer your
question, yes, it was needed to have a different IRQ model.

But maybe I should try the 2nd approach as well to avoid duplication.
I will conduct another experiment if this sounds the right direction.

> > Or the title of the cover letter is somehow overstatement: instead,
> > "Minimum integration of UML with LKL" or something like this would be
> > better ?
> 
> Heh, well, doesn't really matter?
>
> But again, there are a few different aspects here:
>  - what's technically feasible
>  - what this patchset achieves
>  - where we want to be in the end
> 
> I think right now these are diverging enough that we can't even answer
> that last question, much less find the road to get there.

I think the option 2) is the final goal.

> > Since the patchset of LKL is relatively huge, I was trying to make a
> > minimum patchset for the first step.  Because having #ifdefs and glue
> > code makes existing arch/um code untouched, I took this way to make
> > the patchset as small as possible.
> > 
> > But if this is not the case, I will look for the other way and touch
> > arch/um code to fit the current LKL implementation.
> > 
> > What do you think ?
> 
> I think that'd be fine, if indeed that's what we want to do.
> 
> I really think we're beating around the bush, and need to first figure
> out the technical differences between UML and LKL and decide between the
> options (1) and (2) above. Maybe there's a compromise there somewhere,
> where some small bits of code still _are_ different, but IMHO having two
> (IRQ, thread, memory) models, two host interfaces (lkl-ops vs. os_*
> functions), even two include/asm/ source trees (and so on) is not
> appropriate.
> 
> This may take some patches, and some experimentation. I'd leave drivers
> out of this initially - you should be able to test LKL with something
> simpler, right? The API surface is basically the syscall interface as
> functions, so you can start the library and call something simple
> initially? Though I guess you need some driver for the IRQ model to make
> sense, etc.

okay.

> And like I said before, that decision will frame everything else. I
> really don't think we can make significant progress here without having
> decided whether this is possible.
>
> Perhaps UML *can* become a "special case" of LKL, with a special API
> function (that's not part of the syscall surface) to "boot(cmdline)" or
> something. But if it can't, and has to remain as separated as the two
> are today, I would argue we're better off just not calling them the same
> architecture.

I agree with this if the unification has all completed.

Thanks,
-- Hajime

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v4 02/25] um lkl: architecture skeleton for Linux kernel library
@ 2020-04-07 19:25                 ` Octavian Purdila
  0 siblings, 0 replies; 476+ messages in thread
From: Octavian Purdila @ 2020-04-07 19:25 UTC (permalink / raw)
  To: Hajime Tazaki
  Cc: Johannes Berg, Richard Weinberger, linux-arch, Levente Kurusa,
	Matthieu Coudron, cem, Lai Jiangshan, Jens Staal, Motomu Utsumi,
	linux-um, Akira Moroo, Petros Angelatos, Yuan Liu,
	Mark Stillwell, pscollins, linux-kernel-library,
	Pierre-Hugues Husson, sigmaepsilon92, Luca Dariz

<snip>


> > And like I said before, that decision will frame everything else. I
> > really don't think we can make significant progress here without having
> > decided whether this is possible.
> >
> > Perhaps UML *can* become a "special case" of LKL, with a special API
> > function (that's not part of the syscall surface) to "boot(cmdline)" or
> > something. But if it can't, and has to remain as separated as the two
> > are today, I would argue we're better off just not calling them the same
> > architecture.
>
> I agree with this if the unification has all completed.
>

I tought a lot about this and I agree with Johannes that if we want to
unity UML and LKL we should start with the hard parts.

I am also starting to think that it is unlikely to be able to merge the
two "nicely" and that we should probably turn this on its head and start
with reworking UML towards the LKL features. We will end up
re-implementing *some* of the LKL concepts from scratch in UML but I
think at this point this is unavoidable.

Here are my thoughts on a very rough plan for doing that:

Milestone 1: LKL lib on top of UML
 * Kernel - Host build split
   Build UML as a relocatable object using the UML’s kernel linker
   script.
   Move the ptrace and other well isolated os code out of arch/um to
   tools/um (or maybe start directly with tools/lkl?)
   Use standard host toolchain to create a static library stripped of
   the ptrace code. Use standard host toolchain to build the main UML
   executable.
   Add library init API that creates the UML kernel process and starts
   UML.
* System calls APIs
   Add new system call interface dbased on fd irq.
   Use the LKL scripts to export the required headers to create system
   calls APIs that use the UML system calls infrastructure.
   Keep the underlying host and driver operations (threads, irqs, etc.)
   as they are now in UML.
 * Boot test
   Port the LKL boot test to verify that we are able to programatically
   issue system calls.

Milestone 2: add virtio disk support
 * Export asm/io.h operations to host/os. Create IO access operations
   and redirect them to weak os_ variants that use the current UML
   implementation.
 * Add the LKL IO access layer including generic virtio handling and the
   virtio block device code.
 * Port LKL disk test and disk apps (lklfuse, fs2tar, cptofs)

Milestone 3: new arch ports
  * Abstrac the system call / IRQ mode the move the implementation to host
  * Abstract the thread model and move the implementation to host
  * Add LKL thread model and LKL ports

Thanks,
Tavi

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v4 02/25] um lkl: architecture skeleton for Linux kernel library
@ 2020-04-07 19:25                 ` Octavian Purdila
  0 siblings, 0 replies; 476+ messages in thread
From: Octavian Purdila @ 2020-04-07 19:25 UTC (permalink / raw)
  To: Hajime Tazaki
  Cc: Johannes Berg, Richard Weinberger, linux-arch, Levente Kurusa,
	Matthieu Coudron, cem, Lai Jiangshan, Jens Staal, Motomu Utsumi,
	linux-um, Akira Moroo, Petros Angelatos, Yuan Liu,
	Mark Stillwell, pscollins, linux-kernel-library,
	Pierre-Hugues Husson, sigmaepsilon92, Luca Dariz,
	Edison M . Castro

<snip>


> > And like I said before, that decision will frame everything else. I
> > really don't think we can make significant progress here without having
> > decided whether this is possible.
> >
> > Perhaps UML *can* become a "special case" of LKL, with a special API
> > function (that's not part of the syscall surface) to "boot(cmdline)" or
> > something. But if it can't, and has to remain as separated as the two
> > are today, I would argue we're better off just not calling them the same
> > architecture.
>
> I agree with this if the unification has all completed.
>

I tought a lot about this and I agree with Johannes that if we want to
unity UML and LKL we should start with the hard parts.

I am also starting to think that it is unlikely to be able to merge the
two "nicely" and that we should probably turn this on its head and start
with reworking UML towards the LKL features. We will end up
re-implementing *some* of the LKL concepts from scratch in UML but I
think at this point this is unavoidable.

Here are my thoughts on a very rough plan for doing that:

Milestone 1: LKL lib on top of UML
 * Kernel - Host build split
   Build UML as a relocatable object using the UML’s kernel linker
   script.
   Move the ptrace and other well isolated os code out of arch/um to
   tools/um (or maybe start directly with tools/lkl?)
   Use standard host toolchain to create a static library stripped of
   the ptrace code. Use standard host toolchain to build the main UML
   executable.
   Add library init API that creates the UML kernel process and starts
   UML.
* System calls APIs
   Add new system call interface dbased on fd irq.
   Use the LKL scripts to export the required headers to create system
   calls APIs that use the UML system calls infrastructure.
   Keep the underlying host and driver operations (threads, irqs, etc.)
   as they are now in UML.
 * Boot test
   Port the LKL boot test to verify that we are able to programatically
   issue system calls.

Milestone 2: add virtio disk support
 * Export asm/io.h operations to host/os. Create IO access operations
   and redirect them to weak os_ variants that use the current UML
   implementation.
 * Add the LKL IO access layer including generic virtio handling and the
   virtio block device code.
 * Port LKL disk test and disk apps (lklfuse, fs2tar, cptofs)

Milestone 3: new arch ports
  * Abstrac the system call / IRQ mode the move the implementation to host
  * Abstract the thread model and move the implementation to host
  * Add LKL thread model and LKL ports

Thanks,
Tavi

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v4 02/25] um lkl: architecture skeleton for Linux kernel library
@ 2020-04-07 19:25                 ` Octavian Purdila
  0 siblings, 0 replies; 476+ messages in thread
From: Octavian Purdila @ 2020-04-07 19:25 UTC (permalink / raw)
  To: Hajime Tazaki
  Cc: linux-arch, Levente Kurusa, Matthieu Coudron, cem,
	Richard Weinberger, linux-um, Jens Staal, Motomu Utsumi,
	Lai Jiangshan, Akira Moroo, Petros Angelatos, Edison M . Castro,
	Pierre-Hugues Husson, Mark Stillwell, linux-kernel-library,
	pscollins, Johannes Berg, sigmaepsilon92, Luca Dariz, Yuan Liu

<snip>


> > And like I said before, that decision will frame everything else. I
> > really don't think we can make significant progress here without having
> > decided whether this is possible.
> >
> > Perhaps UML *can* become a "special case" of LKL, with a special API
> > function (that's not part of the syscall surface) to "boot(cmdline)" or
> > something. But if it can't, and has to remain as separated as the two
> > are today, I would argue we're better off just not calling them the same
> > architecture.
>
> I agree with this if the unification has all completed.
>

I tought a lot about this and I agree with Johannes that if we want to
unity UML and LKL we should start with the hard parts.

I am also starting to think that it is unlikely to be able to merge the
two "nicely" and that we should probably turn this on its head and start
with reworking UML towards the LKL features. We will end up
re-implementing *some* of the LKL concepts from scratch in UML but I
think at this point this is unavoidable.

Here are my thoughts on a very rough plan for doing that:

Milestone 1: LKL lib on top of UML
 * Kernel - Host build split
   Build UML as a relocatable object using the UML’s kernel linker
   script.
   Move the ptrace and other well isolated os code out of arch/um to
   tools/um (or maybe start directly with tools/lkl?)
   Use standard host toolchain to create a static library stripped of
   the ptrace code. Use standard host toolchain to build the main UML
   executable.
   Add library init API that creates the UML kernel process and starts
   UML.
* System calls APIs
   Add new system call interface dbased on fd irq.
   Use the LKL scripts to export the required headers to create system
   calls APIs that use the UML system calls infrastructure.
   Keep the underlying host and driver operations (threads, irqs, etc.)
   as they are now in UML.
 * Boot test
   Port the LKL boot test to verify that we are able to programatically
   issue system calls.

Milestone 2: add virtio disk support
 * Export asm/io.h operations to host/os. Create IO access operations
   and redirect them to weak os_ variants that use the current UML
   implementation.
 * Add the LKL IO access layer including generic virtio handling and the
   virtio block device code.
 * Port LKL disk test and disk apps (lklfuse, fs2tar, cptofs)

Milestone 3: new arch ports
  * Abstrac the system call / IRQ mode the move the implementation to host
  * Abstract the thread model and move the implementation to host
  * Add LKL thread model and LKL ports

Thanks,
Tavi

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

^ permalink raw reply	[flat|nested] 476+ messages in thread
* [RFC v5 00/21] Unifying LKL into UML
  2020-03-30 14:45   ` Hajime Tazaki
@ 2020-07-02 14:06     ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-07-02 14:06 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Hajime Tazaki

This is another spin of the unification of LKL into UML.  Based on the
discussion of v4 patchset, we have tried to address issue raised and
rewrote the patchset from scratch.  The summary is listed in the
changelog below.

Although there are still bugs in the patchset, we'd like to ask your
opinions on the design we changed.

The milestone section is also updated: this patchset is for the
milestone 1, though the common init API is still not implemented yet.


Changes in rfc v5:
- rewrite whole patchset from scratch
- move arch-dependent code of arch/um and arch/x86/um to tools/um
 - mainly code under os-Linux/ involved
 - introduce 2-stage build (kernel, and host-dependent parts)
 - clean up vmlinux.lds.S
- put LKL-specific implementations as a SUBARCH under arch/um/nommu
- introduce !CONFIG_MMU in arch/um
- use struct arch_thread and arch_switch_to() for subarch-specific
  thread implementation
- integrate with the IRQ infrastructure of UML
- tested with block device drivers (ubd) for the proof

Changes in rfc v4: (https://lwn.net/Articles/816276/)
- Rebase on the current uml/master branch
- Fix IRQ handling (bug fix)
- drop a patch for CONFIG_GENERIC_ATOMIC64 (comment by Peter Zijlstra)
- implement vector net driver for UMMODE_LIB (comment by Anton)
- clean up uapi headers to avoid duplicates
- clean up IRQ handling code (comment by Anton)
- fix error handling in test code (comments by David Disseldorp)

Changes in rfc v3:
- use UML drivers (net, block) from LKL programs
- drop virtio device implementations
- drop mingw32 (Windows) host
- drop android (arm/aarch64) host
- drop FreeBSD (x86_64) host
- drop LD_PRELOAD (hijack) support
- update milestone

rfc v2:
- use UMMODE instead of SUBARCH to switch UML or LKL
- tools/lkl directory is still there. I confirmed we can move under arch/um
  (e.g., arch/um/lkl/hosts).  I will move it IF this is preferable.
- drop several patches involved non-uml directory
- drop several patches which are not required
- refine commit logs
- document updated




LKL (Linux Kernel Library) is aiming to allow reusing the Linux kernel code
as extensively as possible with minimal effort and reduced maintenance
overhead.

Examples of how LKL can be used are: creating userspace applications
(running on Linux and other operating systems) that can read or write Linux
filesystems or can use the Linux networking stack, creating kernel drivers
for other operating systems that can read Linux filesystems, bootloaders
support for reading/writing Linux filesystems, etc.

With LKL, the kernel code is compiled into an object file that can be
directly linked by applications. The API offered by LKL is based on the
Linux system call interface.

LKL is originally implemented as an architecture port in arch/lkl, but this
series of commits tries to integrate this into arch/um as one of the mode
of UML.  This was discussed during RFC email of LKL (*1).

The latest LKL version can be found at https://github.com/lkl/linux

Milestone
=========
This patches is a first step toward upstreaming *library mode* of Linux kernel,
but we think we need to have several steps toward our goal, describing in the
below.


Milestone 1: LKL lib on top of UML
 * Kernel - Host build split
 -  Build UML as a relocatable object using the UML's kernel linker script.
 -  Move the ptrace and other well isolated os code out of arch/um to
    tools/um
 -  Use standard host toolchain to create a static library stripped of
    the ptrace code. Use standard host toolchain to build the main UML
    executable.
 -  Add library init API that creates the UML kernel process and starts
    UML.
 * System calls APIs
 -  Add new system call interface based on UML's irq facility.
 -  Use the LKL scripts to export the required headers to create system
    calls APIs that use the UML system calls infrastructure.
 -  Keep the underlying host and driver operations (threads, irqs, etc.)
    as they are now in UML.
 * Boot test
 -  Port the LKL boot test to verify that we are able to programatically
    issue system calls.

Milestone 2: add virtio disk support
 * Export asm/io.h operations to host/os. Create IO access operations
   and redirect them to weak os_ variants that use the current UML
   implementation.
 * Add the LKL IO access layer including generic virtio handling and the
   virtio block device code.
 * Port LKL disk test and disk apps (lklfuse, fs2tar, cptofs)

Milestone 3: new arch ports
  * Abstract the system call / IRQ mode the move the implementation to host
  * Abstract the thread model and move the implementation to host
  * Add LKL thread model and LKL ports


Building LKL the host library and LKL applications
==================================================

% cd tools/um
% make

will build LKL as a object file, it will install it in tools/um/lib together
with the headers files in tools/um/include then will build the host library,
tests and a few of application examples:

* tests/boot - a simple applications that uses LKL and exercises the basic
  LKL APIs

% make run-tests

should run the above `tests/boot` and `tests/net-test` and report errors if
there are any.

Supported hosts
===============

Currently LKL supports Linux userspace applications. New hosts can be added
relatively easy if the host supports gcc and GNU ld. Previous versions of
LKL supported Windows kernel and Haiku kernel hosts, and we also have WIP
patches with rump-hypercall interface, used in UEFI, as well as macOS
userspace (part of POSIX).

There is also musl-libc port for LKL, which might be interested in for some
folks.


Further readings about LKL
=========================

- Discussion in github LKL issue
https://github.com/lkl/linux/issues/304

- LKL (an article)
https://www.researchgate.net/profile/Nicolae_Tapus2/publication/224164682_LKL_The_Linux_kernel_library/links/02bfe50fd921ab4f7c000000.pdf

*1 RFC email to LKML (back in 2015)
https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1012277.html


Please review the following changes for suitability for inclusion. If you have
any objections or suggestions for improvement, please respond to the patches. If
you agree with the changes, please provide your Acked-by.

The following changes since commit f6e8c474390be2e6f5bd0b8966e19958214609ff:

  um: virtio: Replace zero-length array with flexible-array (2020-06-02 22:38:00 +0200)

are available in the Git repository at:

  git://github.com/thehajime/linux b6eab5512a8168bbb46a485e9386e8cbd812b511
  https://github.com/thehajime/linux/tree/uml-lkl-5.8rc1-v5

Hajime Tazaki (18):
  um: move arch/um/os-Linux dir to tools/um/uml
  um: move arch/x86/um/os-Linux to tools/um/uml/
  scritps: um: suppress warnings if SRCARCH=um
  um: extend arch_switch_to for alternate SUBARCH
  um: add nommu mode for UML library mode
  um: nommu: host interface
  um: nommu: memory handling
  um: nommu: kernel thread support
  um: nommu: system call interface and application API
  um: nommu: basic console support
  um: nommu: initialization and cleanup
  um: nommu: integrate with irq infrastructure of UML
  um: nommu: plug in the build system
  um: host: add nommu build for ARCH=um
  um: host: add utilities functions
  um: host: posix host operations
  um: host: add test programs
  um: nommu: add block device support of UML

Octavian Purdila (3):
  um: split build in kernel and host parts
  um: add os init and exit calls
  um: host: implement os_initcalls and os_exitcalls

 arch/um/Kconfig                               |  29 +-
 arch/um/Makefile                              |  25 +-
 arch/um/drivers/Makefile                      |  10 +-
 .../um/{os-Linux => }/drivers/ethertap_kern.c |   0
 arch/um/{os-Linux => }/drivers/tuntap_kern.c  |   0
 arch/um/include/asm/common.lds.S              |  99 ----
 arch/um/include/asm/host_ops.h                |   9 +
 arch/um/include/asm/mmu.h                     |   3 +
 arch/um/include/asm/mmu_context.h             |   8 +
 arch/um/include/asm/page.h                    |  15 +
 arch/um/include/asm/pgtable.h                 |  27 +
 arch/um/include/asm/thread_info.h             |  24 +
 arch/um/include/asm/uaccess.h                 |   6 +
 arch/um/include/asm/xor.h                     |   3 +-
 arch/um/include/shared/as-layout.h            |   1 +
 .../drivers => include/shared}/etap.h         |   0
 arch/um/include/shared/init.h                 |  18 +-
 arch/um/include/shared/os.h                   |   1 +
 .../drivers => include/shared}/tuntap.h       |   0
 arch/um/include/uapi/asm/Kbuild               |   2 +
 arch/um/kernel/Makefile                       |  12 +-
 arch/um/kernel/dyn.lds.S                      | 171 -------
 arch/um/kernel/irq.c                          |  13 +
 arch/um/kernel/process.c                      |  14 +-
 arch/um/kernel/reboot.c                       |   5 +
 arch/um/kernel/time.c                         |   2 +
 arch/um/kernel/um_arch.c                      |  16 +
 arch/um/kernel/uml.lds.S                      | 115 -----
 arch/um/kernel/vmlinux.lds.S                  |  91 +++-
 arch/um/nommu/Makefile                        |   1 +
 arch/um/nommu/Makefile.um                     |  18 +
 arch/um/nommu/include/asm/Kbuild              |   6 +
 arch/um/nommu/include/asm/archparam.h         |   1 +
 arch/um/nommu/include/asm/atomic.h            |  11 +
 arch/um/nommu/include/asm/atomic64.h          | 114 +++++
 arch/um/nommu/include/asm/bitsperlong.h       |  12 +
 arch/um/nommu/include/asm/byteorder.h         |   7 +
 arch/um/nommu/include/asm/cpu.h               |  16 +
 arch/um/nommu/include/asm/elf.h               |  15 +
 arch/um/nommu/include/asm/mm_context.h        |   8 +
 arch/um/nommu/include/asm/processor.h         |  46 ++
 arch/um/nommu/include/asm/ptrace.h            |  21 +
 arch/um/nommu/include/asm/sched.h             |  23 +
 arch/um/nommu/include/asm/segment.h           |   9 +
 arch/um/nommu/include/asm/syscall_wrapper.h   |  57 +++
 arch/um/nommu/include/asm/syscalls.h          |  15 +
 arch/um/nommu/include/uapi/asm/Kbuild         |   4 +
 arch/um/nommu/include/uapi/asm/bitsperlong.h  |  11 +
 arch/um/nommu/include/uapi/asm/byteorder.h    |  11 +
 arch/um/nommu/include/uapi/asm/host_ops.h     | 122 +++++
 arch/um/nommu/include/uapi/asm/sigcontext.h   |  12 +
 arch/um/nommu/include/uapi/asm/syscalls.h     | 287 +++++++++++
 arch/um/nommu/include/uapi/asm/unistd.h       |  17 +
 arch/um/nommu/um/Kconfig                      |  45 ++
 arch/um/nommu/um/Makefile                     |   4 +
 arch/um/nommu/um/bootmem.c                    |  86 ++++
 arch/um/nommu/um/console.c                    |  42 ++
 arch/um/nommu/um/cpu.c                        | 247 ++++++++++
 arch/um/nommu/um/delay.c                      |  31 ++
 arch/um/nommu/um/setup.c                      | 178 +++++++
 arch/um/nommu/um/shared/sysdep/archsetjmp.h   |  13 +
 arch/um/nommu/um/shared/sysdep/faultinfo.h    |   8 +
 .../nommu/um/shared/sysdep/kernel-offsets.h   |  12 +
 arch/um/nommu/um/shared/sysdep/mcontext.h     |   9 +
 arch/um/nommu/um/shared/sysdep/ptrace.h       |  42 ++
 arch/um/nommu/um/shared/sysdep/ptrace_user.h  |   7 +
 arch/um/nommu/um/syscalls.c                   | 199 ++++++++
 arch/um/nommu/um/threads.c                    | 261 ++++++++++
 arch/um/nommu/um/unimplemented.c              |  70 +++
 arch/um/nommu/um/user_constants.h             |  13 +
 arch/um/os-Linux/Makefile                     |  19 -
 arch/um/os-Linux/drivers/Makefile             |  13 -
 arch/um/scripts/headers_install.py            | 197 ++++++++
 arch/x86/um/Makefile                          |   2 +-
 arch/x86/um/os-Linux/Makefile                 |  13 -
 arch/x86/um/ptrace_32.c                       |   2 +-
 arch/x86/um/syscalls_64.c                     |   2 +-
 scripts/headers_install.sh                    |   3 +
 scripts/link-vmlinux.sh                       |  42 +-
 tools/um/Makefile                             |  82 ++++
 tools/um/Targets                              |  13 +
 tools/um/include/lkl.h                        | 357 ++++++++++++++
 tools/um/include/lkl_host.h                   |  27 +
 tools/um/lib/Build                            |   7 +
 tools/um/lib/fs.c                             | 461 ++++++++++++++++++
 tools/um/lib/jmp_buf.c                        |  14 +
 tools/um/lib/jmp_buf.h                        |   8 +
 tools/um/lib/posix-host.c                     | 293 +++++++++++
 tools/um/lib/utils.c                          | 213 ++++++++
 tools/um/tests/Build                          |   2 +
 tools/um/tests/boot.c                         | 393 +++++++++++++++
 tools/um/tests/boot.sh                        |   9 +
 tools/um/tests/cla.c                          | 159 ++++++
 tools/um/tests/cla.h                          |  33 ++
 tools/um/tests/disk.c                         | 168 +++++++
 tools/um/tests/disk.sh                        |  70 +++
 tools/um/tests/run.py                         | 174 +++++++
 tools/um/tests/tap13.py                       | 209 ++++++++
 tools/um/tests/test.c                         | 126 +++++
 tools/um/tests/test.h                         |  72 +++
 tools/um/tests/test.sh                        | 181 +++++++
 tools/um/uml/Build                            |  57 +++
 tools/um/uml/drivers/Build                    |  10 +
 .../um/uml}/drivers/ethertap_user.c           |   0
 .../um/uml}/drivers/tuntap_user.c             |   0
 {arch/um/os-Linux => tools/um/uml}/elf_aux.c  |   0
 {arch/um/os-Linux => tools/um/uml}/execvp.c   |   4 -
 {arch/um/os-Linux => tools/um/uml}/file.c     |   0
 {arch/um/os-Linux => tools/um/uml}/helper.c   |   0
 {arch/um/os-Linux => tools/um/uml}/irq.c      |   0
 {arch/um/os-Linux => tools/um/uml}/main.c     |   0
 {arch/um/os-Linux => tools/um/uml}/mem.c      |   0
 tools/um/uml/nommu/Build                      |   1 +
 tools/um/uml/nommu/registers.c                |  21 +
 tools/um/uml/nommu/unimplemented.c            |  21 +
 {arch/um/os-Linux => tools/um/uml}/process.c  |   2 +
 .../um/os-Linux => tools/um/uml}/registers.c  |   0
 {arch/um/os-Linux => tools/um/uml}/sigio.c    |   0
 {arch/um/os-Linux => tools/um/uml}/signal.c   |  12 +-
 .../skas/Makefile => tools/um/uml/skas/Build  |   6 +-
 {arch/um/os-Linux => tools/um/uml}/skas/mem.c |   0
 .../os-Linux => tools/um/uml}/skas/process.c  |   3 +-
 {arch/um/os-Linux => tools/um/uml}/start_up.c |   0
 {arch/um/os-Linux => tools/um/uml}/time.c     |   0
 {arch/um/os-Linux => tools/um/uml}/tty.c      |   0
 {arch/um/os-Linux => tools/um/uml}/umid.c     |   0
 .../um/os-Linux => tools/um/uml}/user_syms.c  |   1 +
 {arch/um/os-Linux => tools/um/uml}/util.c     |  26 +
 tools/um/uml/x86/Build                        |  11 +
 .../os-Linux => tools/um/uml/x86}/mcontext.c  |   0
 .../um/os-Linux => tools/um/uml/x86}/prctl.c  |   0
 .../os-Linux => tools/um/uml/x86}/registers.c |   0
 .../os-Linux => tools/um/uml/x86}/task_size.c |   0
 .../um/os-Linux => tools/um/uml/x86}/tls.c    |   0
 134 files changed, 5846 insertions(+), 523 deletions(-)
 rename arch/um/{os-Linux => }/drivers/ethertap_kern.c (100%)
 rename arch/um/{os-Linux => }/drivers/tuntap_kern.c (100%)
 delete mode 100644 arch/um/include/asm/common.lds.S
 create mode 100644 arch/um/include/asm/host_ops.h
 rename arch/um/{os-Linux/drivers => include/shared}/etap.h (100%)
 rename arch/um/{os-Linux/drivers => include/shared}/tuntap.h (100%)
 create mode 100644 arch/um/include/uapi/asm/Kbuild
 delete mode 100644 arch/um/kernel/dyn.lds.S
 delete mode 100644 arch/um/kernel/uml.lds.S
 create mode 100644 arch/um/nommu/Makefile
 create mode 100644 arch/um/nommu/Makefile.um
 create mode 100644 arch/um/nommu/include/asm/Kbuild
 create mode 100644 arch/um/nommu/include/asm/archparam.h
 create mode 100644 arch/um/nommu/include/asm/atomic.h
 create mode 100644 arch/um/nommu/include/asm/atomic64.h
 create mode 100644 arch/um/nommu/include/asm/bitsperlong.h
 create mode 100644 arch/um/nommu/include/asm/byteorder.h
 create mode 100644 arch/um/nommu/include/asm/cpu.h
 create mode 100644 arch/um/nommu/include/asm/elf.h
 create mode 100644 arch/um/nommu/include/asm/mm_context.h
 create mode 100644 arch/um/nommu/include/asm/processor.h
 create mode 100644 arch/um/nommu/include/asm/ptrace.h
 create mode 100644 arch/um/nommu/include/asm/sched.h
 create mode 100644 arch/um/nommu/include/asm/segment.h
 create mode 100644 arch/um/nommu/include/asm/syscall_wrapper.h
 create mode 100644 arch/um/nommu/include/asm/syscalls.h
 create mode 100644 arch/um/nommu/include/uapi/asm/Kbuild
 create mode 100644 arch/um/nommu/include/uapi/asm/bitsperlong.h
 create mode 100644 arch/um/nommu/include/uapi/asm/byteorder.h
 create mode 100644 arch/um/nommu/include/uapi/asm/host_ops.h
 create mode 100644 arch/um/nommu/include/uapi/asm/sigcontext.h
 create mode 100644 arch/um/nommu/include/uapi/asm/syscalls.h
 create mode 100644 arch/um/nommu/include/uapi/asm/unistd.h
 create mode 100644 arch/um/nommu/um/Kconfig
 create mode 100644 arch/um/nommu/um/Makefile
 create mode 100644 arch/um/nommu/um/bootmem.c
 create mode 100644 arch/um/nommu/um/console.c
 create mode 100644 arch/um/nommu/um/cpu.c
 create mode 100644 arch/um/nommu/um/delay.c
 create mode 100644 arch/um/nommu/um/setup.c
 create mode 100644 arch/um/nommu/um/shared/sysdep/archsetjmp.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/faultinfo.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/kernel-offsets.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/mcontext.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/ptrace.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/ptrace_user.h
 create mode 100644 arch/um/nommu/um/syscalls.c
 create mode 100644 arch/um/nommu/um/threads.c
 create mode 100644 arch/um/nommu/um/unimplemented.c
 create mode 100644 arch/um/nommu/um/user_constants.h
 delete mode 100644 arch/um/os-Linux/Makefile
 delete mode 100644 arch/um/os-Linux/drivers/Makefile
 create mode 100755 arch/um/scripts/headers_install.py
 delete mode 100644 arch/x86/um/os-Linux/Makefile
 create mode 100644 tools/um/Makefile
 create mode 100644 tools/um/Targets
 create mode 100644 tools/um/include/lkl.h
 create mode 100644 tools/um/include/lkl_host.h
 create mode 100644 tools/um/lib/Build
 create mode 100644 tools/um/lib/fs.c
 create mode 100644 tools/um/lib/jmp_buf.c
 create mode 100644 tools/um/lib/jmp_buf.h
 create mode 100644 tools/um/lib/posix-host.c
 create mode 100644 tools/um/lib/utils.c
 create mode 100644 tools/um/tests/Build
 create mode 100644 tools/um/tests/boot.c
 create mode 100755 tools/um/tests/boot.sh
 create mode 100644 tools/um/tests/cla.c
 create mode 100644 tools/um/tests/cla.h
 create mode 100644 tools/um/tests/disk.c
 create mode 100755 tools/um/tests/disk.sh
 create mode 100755 tools/um/tests/run.py
 create mode 100644 tools/um/tests/tap13.py
 create mode 100644 tools/um/tests/test.c
 create mode 100644 tools/um/tests/test.h
 create mode 100644 tools/um/tests/test.sh
 create mode 100644 tools/um/uml/Build
 create mode 100644 tools/um/uml/drivers/Build
 rename {arch/um/os-Linux => tools/um/uml}/drivers/ethertap_user.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/drivers/tuntap_user.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/elf_aux.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/execvp.c (98%)
 rename {arch/um/os-Linux => tools/um/uml}/file.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/helper.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/irq.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/main.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/mem.c (100%)
 create mode 100644 tools/um/uml/nommu/Build
 create mode 100644 tools/um/uml/nommu/registers.c
 create mode 100644 tools/um/uml/nommu/unimplemented.c
 rename {arch/um/os-Linux => tools/um/uml}/process.c (99%)
 rename {arch/um/os-Linux => tools/um/uml}/registers.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/sigio.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/signal.c (96%)
 rename arch/um/os-Linux/skas/Makefile => tools/um/uml/skas/Build (56%)
 rename {arch/um/os-Linux => tools/um/uml}/skas/mem.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/skas/process.c (99%)
 rename {arch/um/os-Linux => tools/um/uml}/start_up.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/time.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/tty.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/umid.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/user_syms.c (99%)
 rename {arch/um/os-Linux => tools/um/uml}/util.c (89%)
 create mode 100644 tools/um/uml/x86/Build
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/mcontext.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/prctl.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/registers.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/task_size.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/tls.c (100%)

-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply	[flat|nested] 476+ messages in thread
* [RFC v5 00/21] Unifying LKL into UML
@ 2020-07-02 14:06     ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-07-02 14:06 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, linux-kernel-library, linux-arch,
	Hajime Tazaki, Akira Moroo

This is another spin of the unification of LKL into UML.  Based on the
discussion of v4 patchset, we have tried to address issue raised and
rewrote the patchset from scratch.  The summary is listed in the
changelog below.

Although there are still bugs in the patchset, we'd like to ask your
opinions on the design we changed.

The milestone section is also updated: this patchset is for the
milestone 1, though the common init API is still not implemented yet.


Changes in rfc v5:
- rewrite whole patchset from scratch
- move arch-dependent code of arch/um and arch/x86/um to tools/um
 - mainly code under os-Linux/ involved
 - introduce 2-stage build (kernel, and host-dependent parts)
 - clean up vmlinux.lds.S
- put LKL-specific implementations as a SUBARCH under arch/um/nommu
- introduce !CONFIG_MMU in arch/um
- use struct arch_thread and arch_switch_to() for subarch-specific
  thread implementation
- integrate with the IRQ infrastructure of UML
- tested with block device drivers (ubd) for the proof

Changes in rfc v4: (https://lwn.net/Articles/816276/)
- Rebase on the current uml/master branch
- Fix IRQ handling (bug fix)
- drop a patch for CONFIG_GENERIC_ATOMIC64 (comment by Peter Zijlstra)
- implement vector net driver for UMMODE_LIB (comment by Anton)
- clean up uapi headers to avoid duplicates
- clean up IRQ handling code (comment by Anton)
- fix error handling in test code (comments by David Disseldorp)

Changes in rfc v3:
- use UML drivers (net, block) from LKL programs
- drop virtio device implementations
- drop mingw32 (Windows) host
- drop android (arm/aarch64) host
- drop FreeBSD (x86_64) host
- drop LD_PRELOAD (hijack) support
- update milestone

rfc v2:
- use UMMODE instead of SUBARCH to switch UML or LKL
- tools/lkl directory is still there. I confirmed we can move under arch/um
  (e.g., arch/um/lkl/hosts).  I will move it IF this is preferable.
- drop several patches involved non-uml directory
- drop several patches which are not required
- refine commit logs
- document updated




LKL (Linux Kernel Library) is aiming to allow reusing the Linux kernel code
as extensively as possible with minimal effort and reduced maintenance
overhead.

Examples of how LKL can be used are: creating userspace applications
(running on Linux and other operating systems) that can read or write Linux
filesystems or can use the Linux networking stack, creating kernel drivers
for other operating systems that can read Linux filesystems, bootloaders
support for reading/writing Linux filesystems, etc.

With LKL, the kernel code is compiled into an object file that can be
directly linked by applications. The API offered by LKL is based on the
Linux system call interface.

LKL is originally implemented as an architecture port in arch/lkl, but this
series of commits tries to integrate this into arch/um as one of the mode
of UML.  This was discussed during RFC email of LKL (*1).

The latest LKL version can be found at https://github.com/lkl/linux

Milestone
=========
This patches is a first step toward upstreaming *library mode* of Linux kernel,
but we think we need to have several steps toward our goal, describing in the
below.


Milestone 1: LKL lib on top of UML
 * Kernel - Host build split
 -  Build UML as a relocatable object using the UML's kernel linker script.
 -  Move the ptrace and other well isolated os code out of arch/um to
    tools/um
 -  Use standard host toolchain to create a static library stripped of
    the ptrace code. Use standard host toolchain to build the main UML
    executable.
 -  Add library init API that creates the UML kernel process and starts
    UML.
 * System calls APIs
 -  Add new system call interface based on UML's irq facility.
 -  Use the LKL scripts to export the required headers to create system
    calls APIs that use the UML system calls infrastructure.
 -  Keep the underlying host and driver operations (threads, irqs, etc.)
    as they are now in UML.
 * Boot test
 -  Port the LKL boot test to verify that we are able to programatically
    issue system calls.

Milestone 2: add virtio disk support
 * Export asm/io.h operations to host/os. Create IO access operations
   and redirect them to weak os_ variants that use the current UML
   implementation.
 * Add the LKL IO access layer including generic virtio handling and the
   virtio block device code.
 * Port LKL disk test and disk apps (lklfuse, fs2tar, cptofs)

Milestone 3: new arch ports
  * Abstract the system call / IRQ mode the move the implementation to host
  * Abstract the thread model and move the implementation to host
  * Add LKL thread model and LKL ports


Building LKL the host library and LKL applications
==================================================

% cd tools/um
% make

will build LKL as a object file, it will install it in tools/um/lib together
with the headers files in tools/um/include then will build the host library,
tests and a few of application examples:

* tests/boot - a simple applications that uses LKL and exercises the basic
  LKL APIs

% make run-tests

should run the above `tests/boot` and `tests/net-test` and report errors if
there are any.

Supported hosts
===============

Currently LKL supports Linux userspace applications. New hosts can be added
relatively easy if the host supports gcc and GNU ld. Previous versions of
LKL supported Windows kernel and Haiku kernel hosts, and we also have WIP
patches with rump-hypercall interface, used in UEFI, as well as macOS
userspace (part of POSIX).

There is also musl-libc port for LKL, which might be interested in for some
folks.


Further readings about LKL
=========================

- Discussion in github LKL issue
https://github.com/lkl/linux/issues/304

- LKL (an article)
https://www.researchgate.net/profile/Nicolae_Tapus2/publication/224164682_LKL_The_Linux_kernel_library/links/02bfe50fd921ab4f7c000000.pdf

*1 RFC email to LKML (back in 2015)
https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1012277.html


Please review the following changes for suitability for inclusion. If you have
any objections or suggestions for improvement, please respond to the patches. If
you agree with the changes, please provide your Acked-by.

The following changes since commit f6e8c474390be2e6f5bd0b8966e19958214609ff:

  um: virtio: Replace zero-length array with flexible-array (2020-06-02 22:38:00 +0200)

are available in the Git repository at:

  git://github.com/thehajime/linux b6eab5512a8168bbb46a485e9386e8cbd812b511
  https://github.com/thehajime/linux/tree/uml-lkl-5.8rc1-v5

Hajime Tazaki (18):
  um: move arch/um/os-Linux dir to tools/um/uml
  um: move arch/x86/um/os-Linux to tools/um/uml/
  scritps: um: suppress warnings if SRCARCH=um
  um: extend arch_switch_to for alternate SUBARCH
  um: add nommu mode for UML library mode
  um: nommu: host interface
  um: nommu: memory handling
  um: nommu: kernel thread support
  um: nommu: system call interface and application API
  um: nommu: basic console support
  um: nommu: initialization and cleanup
  um: nommu: integrate with irq infrastructure of UML
  um: nommu: plug in the build system
  um: host: add nommu build for ARCH=um
  um: host: add utilities functions
  um: host: posix host operations
  um: host: add test programs
  um: nommu: add block device support of UML

Octavian Purdila (3):
  um: split build in kernel and host parts
  um: add os init and exit calls
  um: host: implement os_initcalls and os_exitcalls

 arch/um/Kconfig                               |  29 +-
 arch/um/Makefile                              |  25 +-
 arch/um/drivers/Makefile                      |  10 +-
 .../um/{os-Linux => }/drivers/ethertap_kern.c |   0
 arch/um/{os-Linux => }/drivers/tuntap_kern.c  |   0
 arch/um/include/asm/common.lds.S              |  99 ----
 arch/um/include/asm/host_ops.h                |   9 +
 arch/um/include/asm/mmu.h                     |   3 +
 arch/um/include/asm/mmu_context.h             |   8 +
 arch/um/include/asm/page.h                    |  15 +
 arch/um/include/asm/pgtable.h                 |  27 +
 arch/um/include/asm/thread_info.h             |  24 +
 arch/um/include/asm/uaccess.h                 |   6 +
 arch/um/include/asm/xor.h                     |   3 +-
 arch/um/include/shared/as-layout.h            |   1 +
 .../drivers => include/shared}/etap.h         |   0
 arch/um/include/shared/init.h                 |  18 +-
 arch/um/include/shared/os.h                   |   1 +
 .../drivers => include/shared}/tuntap.h       |   0
 arch/um/include/uapi/asm/Kbuild               |   2 +
 arch/um/kernel/Makefile                       |  12 +-
 arch/um/kernel/dyn.lds.S                      | 171 -------
 arch/um/kernel/irq.c                          |  13 +
 arch/um/kernel/process.c                      |  14 +-
 arch/um/kernel/reboot.c                       |   5 +
 arch/um/kernel/time.c                         |   2 +
 arch/um/kernel/um_arch.c                      |  16 +
 arch/um/kernel/uml.lds.S                      | 115 -----
 arch/um/kernel/vmlinux.lds.S                  |  91 +++-
 arch/um/nommu/Makefile                        |   1 +
 arch/um/nommu/Makefile.um                     |  18 +
 arch/um/nommu/include/asm/Kbuild              |   6 +
 arch/um/nommu/include/asm/archparam.h         |   1 +
 arch/um/nommu/include/asm/atomic.h            |  11 +
 arch/um/nommu/include/asm/atomic64.h          | 114 +++++
 arch/um/nommu/include/asm/bitsperlong.h       |  12 +
 arch/um/nommu/include/asm/byteorder.h         |   7 +
 arch/um/nommu/include/asm/cpu.h               |  16 +
 arch/um/nommu/include/asm/elf.h               |  15 +
 arch/um/nommu/include/asm/mm_context.h        |   8 +
 arch/um/nommu/include/asm/processor.h         |  46 ++
 arch/um/nommu/include/asm/ptrace.h            |  21 +
 arch/um/nommu/include/asm/sched.h             |  23 +
 arch/um/nommu/include/asm/segment.h           |   9 +
 arch/um/nommu/include/asm/syscall_wrapper.h   |  57 +++
 arch/um/nommu/include/asm/syscalls.h          |  15 +
 arch/um/nommu/include/uapi/asm/Kbuild         |   4 +
 arch/um/nommu/include/uapi/asm/bitsperlong.h  |  11 +
 arch/um/nommu/include/uapi/asm/byteorder.h    |  11 +
 arch/um/nommu/include/uapi/asm/host_ops.h     | 122 +++++
 arch/um/nommu/include/uapi/asm/sigcontext.h   |  12 +
 arch/um/nommu/include/uapi/asm/syscalls.h     | 287 +++++++++++
 arch/um/nommu/include/uapi/asm/unistd.h       |  17 +
 arch/um/nommu/um/Kconfig                      |  45 ++
 arch/um/nommu/um/Makefile                     |   4 +
 arch/um/nommu/um/bootmem.c                    |  86 ++++
 arch/um/nommu/um/console.c                    |  42 ++
 arch/um/nommu/um/cpu.c                        | 247 ++++++++++
 arch/um/nommu/um/delay.c                      |  31 ++
 arch/um/nommu/um/setup.c                      | 178 +++++++
 arch/um/nommu/um/shared/sysdep/archsetjmp.h   |  13 +
 arch/um/nommu/um/shared/sysdep/faultinfo.h    |   8 +
 .../nommu/um/shared/sysdep/kernel-offsets.h   |  12 +
 arch/um/nommu/um/shared/sysdep/mcontext.h     |   9 +
 arch/um/nommu/um/shared/sysdep/ptrace.h       |  42 ++
 arch/um/nommu/um/shared/sysdep/ptrace_user.h  |   7 +
 arch/um/nommu/um/syscalls.c                   | 199 ++++++++
 arch/um/nommu/um/threads.c                    | 261 ++++++++++
 arch/um/nommu/um/unimplemented.c              |  70 +++
 arch/um/nommu/um/user_constants.h             |  13 +
 arch/um/os-Linux/Makefile                     |  19 -
 arch/um/os-Linux/drivers/Makefile             |  13 -
 arch/um/scripts/headers_install.py            | 197 ++++++++
 arch/x86/um/Makefile                          |   2 +-
 arch/x86/um/os-Linux/Makefile                 |  13 -
 arch/x86/um/ptrace_32.c                       |   2 +-
 arch/x86/um/syscalls_64.c                     |   2 +-
 scripts/headers_install.sh                    |   3 +
 scripts/link-vmlinux.sh                       |  42 +-
 tools/um/Makefile                             |  82 ++++
 tools/um/Targets                              |  13 +
 tools/um/include/lkl.h                        | 357 ++++++++++++++
 tools/um/include/lkl_host.h                   |  27 +
 tools/um/lib/Build                            |   7 +
 tools/um/lib/fs.c                             | 461 ++++++++++++++++++
 tools/um/lib/jmp_buf.c                        |  14 +
 tools/um/lib/jmp_buf.h                        |   8 +
 tools/um/lib/posix-host.c                     | 293 +++++++++++
 tools/um/lib/utils.c                          | 213 ++++++++
 tools/um/tests/Build                          |   2 +
 tools/um/tests/boot.c                         | 393 +++++++++++++++
 tools/um/tests/boot.sh                        |   9 +
 tools/um/tests/cla.c                          | 159 ++++++
 tools/um/tests/cla.h                          |  33 ++
 tools/um/tests/disk.c                         | 168 +++++++
 tools/um/tests/disk.sh                        |  70 +++
 tools/um/tests/run.py                         | 174 +++++++
 tools/um/tests/tap13.py                       | 209 ++++++++
 tools/um/tests/test.c                         | 126 +++++
 tools/um/tests/test.h                         |  72 +++
 tools/um/tests/test.sh                        | 181 +++++++
 tools/um/uml/Build                            |  57 +++
 tools/um/uml/drivers/Build                    |  10 +
 .../um/uml}/drivers/ethertap_user.c           |   0
 .../um/uml}/drivers/tuntap_user.c             |   0
 {arch/um/os-Linux => tools/um/uml}/elf_aux.c  |   0
 {arch/um/os-Linux => tools/um/uml}/execvp.c   |   4 -
 {arch/um/os-Linux => tools/um/uml}/file.c     |   0
 {arch/um/os-Linux => tools/um/uml}/helper.c   |   0
 {arch/um/os-Linux => tools/um/uml}/irq.c      |   0
 {arch/um/os-Linux => tools/um/uml}/main.c     |   0
 {arch/um/os-Linux => tools/um/uml}/mem.c      |   0
 tools/um/uml/nommu/Build                      |   1 +
 tools/um/uml/nommu/registers.c                |  21 +
 tools/um/uml/nommu/unimplemented.c            |  21 +
 {arch/um/os-Linux => tools/um/uml}/process.c  |   2 +
 .../um/os-Linux => tools/um/uml}/registers.c  |   0
 {arch/um/os-Linux => tools/um/uml}/sigio.c    |   0
 {arch/um/os-Linux => tools/um/uml}/signal.c   |  12 +-
 .../skas/Makefile => tools/um/uml/skas/Build  |   6 +-
 {arch/um/os-Linux => tools/um/uml}/skas/mem.c |   0
 .../os-Linux => tools/um/uml}/skas/process.c  |   3 +-
 {arch/um/os-Linux => tools/um/uml}/start_up.c |   0
 {arch/um/os-Linux => tools/um/uml}/time.c     |   0
 {arch/um/os-Linux => tools/um/uml}/tty.c      |   0
 {arch/um/os-Linux => tools/um/uml}/umid.c     |   0
 .../um/os-Linux => tools/um/uml}/user_syms.c  |   1 +
 {arch/um/os-Linux => tools/um/uml}/util.c     |  26 +
 tools/um/uml/x86/Build                        |  11 +
 .../os-Linux => tools/um/uml/x86}/mcontext.c  |   0
 .../um/os-Linux => tools/um/uml/x86}/prctl.c  |   0
 .../os-Linux => tools/um/uml/x86}/registers.c |   0
 .../os-Linux => tools/um/uml/x86}/task_size.c |   0
 .../um/os-Linux => tools/um/uml/x86}/tls.c    |   0
 134 files changed, 5846 insertions(+), 523 deletions(-)
 rename arch/um/{os-Linux => }/drivers/ethertap_kern.c (100%)
 rename arch/um/{os-Linux => }/drivers/tuntap_kern.c (100%)
 delete mode 100644 arch/um/include/asm/common.lds.S
 create mode 100644 arch/um/include/asm/host_ops.h
 rename arch/um/{os-Linux/drivers => include/shared}/etap.h (100%)
 rename arch/um/{os-Linux/drivers => include/shared}/tuntap.h (100%)
 create mode 100644 arch/um/include/uapi/asm/Kbuild
 delete mode 100644 arch/um/kernel/dyn.lds.S
 delete mode 100644 arch/um/kernel/uml.lds.S
 create mode 100644 arch/um/nommu/Makefile
 create mode 100644 arch/um/nommu/Makefile.um
 create mode 100644 arch/um/nommu/include/asm/Kbuild
 create mode 100644 arch/um/nommu/include/asm/archparam.h
 create mode 100644 arch/um/nommu/include/asm/atomic.h
 create mode 100644 arch/um/nommu/include/asm/atomic64.h
 create mode 100644 arch/um/nommu/include/asm/bitsperlong.h
 create mode 100644 arch/um/nommu/include/asm/byteorder.h
 create mode 100644 arch/um/nommu/include/asm/cpu.h
 create mode 100644 arch/um/nommu/include/asm/elf.h
 create mode 100644 arch/um/nommu/include/asm/mm_context.h
 create mode 100644 arch/um/nommu/include/asm/processor.h
 create mode 100644 arch/um/nommu/include/asm/ptrace.h
 create mode 100644 arch/um/nommu/include/asm/sched.h
 create mode 100644 arch/um/nommu/include/asm/segment.h
 create mode 100644 arch/um/nommu/include/asm/syscall_wrapper.h
 create mode 100644 arch/um/nommu/include/asm/syscalls.h
 create mode 100644 arch/um/nommu/include/uapi/asm/Kbuild
 create mode 100644 arch/um/nommu/include/uapi/asm/bitsperlong.h
 create mode 100644 arch/um/nommu/include/uapi/asm/byteorder.h
 create mode 100644 arch/um/nommu/include/uapi/asm/host_ops.h
 create mode 100644 arch/um/nommu/include/uapi/asm/sigcontext.h
 create mode 100644 arch/um/nommu/include/uapi/asm/syscalls.h
 create mode 100644 arch/um/nommu/include/uapi/asm/unistd.h
 create mode 100644 arch/um/nommu/um/Kconfig
 create mode 100644 arch/um/nommu/um/Makefile
 create mode 100644 arch/um/nommu/um/bootmem.c
 create mode 100644 arch/um/nommu/um/console.c
 create mode 100644 arch/um/nommu/um/cpu.c
 create mode 100644 arch/um/nommu/um/delay.c
 create mode 100644 arch/um/nommu/um/setup.c
 create mode 100644 arch/um/nommu/um/shared/sysdep/archsetjmp.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/faultinfo.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/kernel-offsets.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/mcontext.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/ptrace.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/ptrace_user.h
 create mode 100644 arch/um/nommu/um/syscalls.c
 create mode 100644 arch/um/nommu/um/threads.c
 create mode 100644 arch/um/nommu/um/unimplemented.c
 create mode 100644 arch/um/nommu/um/user_constants.h
 delete mode 100644 arch/um/os-Linux/Makefile
 delete mode 100644 arch/um/os-Linux/drivers/Makefile
 create mode 100755 arch/um/scripts/headers_install.py
 delete mode 100644 arch/x86/um/os-Linux/Makefile
 create mode 100644 tools/um/Makefile
 create mode 100644 tools/um/Targets
 create mode 100644 tools/um/include/lkl.h
 create mode 100644 tools/um/include/lkl_host.h
 create mode 100644 tools/um/lib/Build
 create mode 100644 tools/um/lib/fs.c
 create mode 100644 tools/um/lib/jmp_buf.c
 create mode 100644 tools/um/lib/jmp_buf.h
 create mode 100644 tools/um/lib/posix-host.c
 create mode 100644 tools/um/lib/utils.c
 create mode 100644 tools/um/tests/Build
 create mode 100644 tools/um/tests/boot.c
 create mode 100755 tools/um/tests/boot.sh
 create mode 100644 tools/um/tests/cla.c
 create mode 100644 tools/um/tests/cla.h
 create mode 100644 tools/um/tests/disk.c
 create mode 100755 tools/um/tests/disk.sh
 create mode 100755 tools/um/tests/run.py
 create mode 100644 tools/um/tests/tap13.py
 create mode 100644 tools/um/tests/test.c
 create mode 100644 tools/um/tests/test.h
 create mode 100644 tools/um/tests/test.sh
 create mode 100644 tools/um/uml/Build
 create mode 100644 tools/um/uml/drivers/Build
 rename {arch/um/os-Linux => tools/um/uml}/drivers/ethertap_user.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/drivers/tuntap_user.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/elf_aux.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/execvp.c (98%)
 rename {arch/um/os-Linux => tools/um/uml}/file.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/helper.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/irq.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/main.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/mem.c (100%)
 create mode 100644 tools/um/uml/nommu/Build
 create mode 100644 tools/um/uml/nommu/registers.c
 create mode 100644 tools/um/uml/nommu/unimplemented.c
 rename {arch/um/os-Linux => tools/um/uml}/process.c (99%)
 rename {arch/um/os-Linux => tools/um/uml}/registers.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/sigio.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/signal.c (96%)
 rename arch/um/os-Linux/skas/Makefile => tools/um/uml/skas/Build (56%)
 rename {arch/um/os-Linux => tools/um/uml}/skas/mem.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/skas/process.c (99%)
 rename {arch/um/os-Linux => tools/um/uml}/start_up.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/time.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/tty.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/umid.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/user_syms.c (99%)
 rename {arch/um/os-Linux => tools/um/uml}/util.c (89%)
 create mode 100644 tools/um/uml/x86/Build
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/mcontext.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/prctl.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/registers.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/task_size.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/tls.c (100%)

-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* [RFC v5 01/21] um: split build in kernel and host parts
  2020-07-02 14:06     ` Hajime Tazaki
@ 2020-07-02 14:06       ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-07-02 14:06 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Octavian Purdila, Hajime Tazaki

From: Octavian Purdila <tavi@cs.pub.ro>

This patch splits the UML build in two parts: a kernel part that
generates a relocatable object (linux.o) and a host build part that
links the relocatable object into an executable.

This allows us to simplify the linker script since we can now remove
the host bits (e.g. .rel sections). It also gives us greater
flexibility since it will be much easier to support other
architectures and OSes if we use the standard linker script.

The host build part has been implemented in tools/um so that we can
reuse the available host build infrastructure.

The patch also changes the UML build invocation, if before

 $ make ARCH=um defconfig
 $ make ARCH=um

was generating the executable now this only generates the relocatable
object.

To fully build UML use:

 $ make -C tools/um

which will generate tools/lkl/linux as the UML executable. To
statically compiled UML use:

 $ make -C tools/lkl UML_STATIC=y

Signed-off-by: Octavian Purdila <tavi@cs.pub.ro>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/Kconfig                  |  12 +--
 arch/um/Makefile                 |  14 ++-
 arch/um/include/asm/common.lds.S |  99 ------------------
 arch/um/kernel/dyn.lds.S         | 171 -------------------------------
 arch/um/kernel/uml.lds.S         | 115 ---------------------
 arch/um/kernel/vmlinux.lds.S     |  91 ++++++++++++++--
 scripts/link-vmlinux.sh          |  42 +++-----
 tools/um/Makefile                |  72 +++++++++++++
 tools/um/Targets                 |   4 +
 tools/um/uml/Build               |   0
 10 files changed, 184 insertions(+), 436 deletions(-)
 delete mode 100644 arch/um/include/asm/common.lds.S
 delete mode 100644 arch/um/kernel/dyn.lds.S
 delete mode 100644 arch/um/kernel/uml.lds.S
 create mode 100644 tools/um/Makefile
 create mode 100644 tools/um/Targets
 create mode 100644 tools/um/uml/Build

diff --git a/arch/um/Kconfig b/arch/um/Kconfig
index 96ab7026b037..a0a9fd3ab96c 100644
--- a/arch/um/Kconfig
+++ b/arch/um/Kconfig
@@ -20,6 +20,7 @@ config UML
 	select GENERIC_CLOCKEVENTS
 	select HAVE_GCC_PLUGINS
 	select TTY # Needed for line.c
+	select MODULE_REL_CRCS if MODVERSIONS
 
 config MMU
 	bool
@@ -79,17 +80,6 @@ config STATIC_LINK
 	  NOTE: This option is incompatible with some networking features which
 	  depend on features that require being dynamically loaded (like NSS).
 
-config LD_SCRIPT_STATIC
-	bool
-	default y
-	depends on STATIC_LINK
-
-config LD_SCRIPT_DYN
-	bool
-	default y
-	depends on !LD_SCRIPT_STATIC
-	select MODULE_REL_CRCS if MODVERSIONS
-
 config HOSTFS
 	tristate "Host filesystem"
 	help
diff --git a/arch/um/Makefile b/arch/um/Makefile
index 275f5ffdf6f0..c952ddefcc1c 100644
--- a/arch/um/Makefile
+++ b/arch/um/Makefile
@@ -95,14 +95,20 @@ KBUILD_CPPFLAGS += -I$(srctree)/$(HOST_DIR)/include \
 KERNEL_DEFINES = $(strip -Derrno=kernel_errno -Dsigprocmask=kernel_sigprocmask \
 			 -Dmktime=kernel_mktime $(ARCH_KERNEL_DEFINES))
 KBUILD_CFLAGS += $(KERNEL_DEFINES)
+LDFLAGS_vmlinux += -r
 
-PHONY += linux
+PHONY += linux.o
 
-all: linux
+all: linux.o
 
-linux: vmlinux
+linux.o: vmlinux
 	@echo '  LINK $@'
-	$(Q)ln -f $< $@
+	$(Q)$(OBJCOPY) -R .eh_frame $< $@
+
+install: linux.o
+	@echo "  INSTALL $(INSTALL_PATH)/lib/$<"
+	@mkdir -p $(INSTALL_PATH)/lib/
+	@cp $< $(INSTALL_PATH)/lib/
 
 define archhelp
   echo '* linux		- Binary kernel image (./linux) - for backward'
diff --git a/arch/um/include/asm/common.lds.S b/arch/um/include/asm/common.lds.S
deleted file mode 100644
index eca6c452a41b..000000000000
--- a/arch/um/include/asm/common.lds.S
+++ /dev/null
@@ -1,99 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#include <asm-generic/vmlinux.lds.h>
-
-  .fini      : { *(.fini)    } =0x9090
-  _etext = .;
-  PROVIDE (etext = .);
-
-  . = ALIGN(4096);
-  _sdata = .;
-  PROVIDE (sdata = .);
-
-  RO_DATA(4096)
-
-  .unprotected : { *(.unprotected) }
-  . = ALIGN(4096);
-  PROVIDE (_unprotected_end = .);
-
-  . = ALIGN(4096);
-  EXCEPTION_TABLE(0)
-
-  BUG_TABLE
-
-  .uml.setup.init : {
-	__uml_setup_start = .;
-	*(.uml.setup.init)
-	__uml_setup_end = .;
-  }
-	
-  .uml.help.init : {
-	__uml_help_start = .;
-	*(.uml.help.init)
-	__uml_help_end = .;
-  }
-	
-  .uml.postsetup.init : {
-	__uml_postsetup_start = .;
-	*(.uml.postsetup.init)
-	__uml_postsetup_end = .;
-  }
-	
-  .init.setup : {
-	INIT_SETUP(0)
-  }
-
-  PERCPU_SECTION(32)
-	
-  .initcall.init : {
-	INIT_CALLS
-  }
-
-  .con_initcall.init : {
-	CON_INITCALL
-  }
-
-  .exitcall : {
-	__exitcall_begin = .;
-	*(.exitcall.exit)
-	__exitcall_end = .;
-  }
-
-  .uml.exitcall : {
-	__uml_exitcall_begin = .;
-	*(.uml.exitcall.exit)
-	__uml_exitcall_end = .;
-  }
-
-  . = ALIGN(4);
-  .altinstructions : {
-	__alt_instructions = .;
-	*(.altinstructions)
-	__alt_instructions_end = .;
-  }
-  .altinstr_replacement : { *(.altinstr_replacement) }
-  /* .exit.text is discard at runtime, not link time, to deal with references
-     from .altinstructions and .eh_frame */
-  .exit.text : { EXIT_TEXT }
-  .exit.data : { *(.exit.data) }
-
-  .preinit_array : {
-	__preinit_array_start = .;
-	*(.preinit_array)
-	__preinit_array_end = .;
-  }
-  .init_array : {
-	__init_array_start = .;
-	*(.init_array)
-	__init_array_end = .;
-  }
-  .fini_array : {
-	__fini_array_start = .;
-	*(.fini_array)
-	__fini_array_end = .;
-  }
-
-   . = ALIGN(4096);
-  .init.ramfs : {
-	INIT_RAM_FS
-  }
-
diff --git a/arch/um/kernel/dyn.lds.S b/arch/um/kernel/dyn.lds.S
deleted file mode 100644
index f5001481010c..000000000000
--- a/arch/um/kernel/dyn.lds.S
+++ /dev/null
@@ -1,171 +0,0 @@
-#include <asm/vmlinux.lds.h>
-#include <asm/page.h>
-
-OUTPUT_FORMAT(ELF_FORMAT)
-OUTPUT_ARCH(ELF_ARCH)
-ENTRY(_start)
-jiffies = jiffies_64;
-
-SECTIONS
-{
-  PROVIDE (__executable_start = START);
-  . = START + SIZEOF_HEADERS;
-  .interp         : { *(.interp) }
-  __binary_start = .;
-  . = ALIGN(4096);		/* Init code and data */
-  _text = .;
-  INIT_TEXT_SECTION(PAGE_SIZE)
-
-  . = ALIGN(PAGE_SIZE);
-
-  /* Read-only sections, merged into text segment: */
-  .hash           : { *(.hash) }
-  .gnu.hash       : { *(.gnu.hash) }
-  .dynsym         : { *(.dynsym) }
-  .dynstr         : { *(.dynstr) }
-  .gnu.version    : { *(.gnu.version) }
-  .gnu.version_d  : { *(.gnu.version_d) }
-  .gnu.version_r  : { *(.gnu.version_r) }
-  .rel.init       : { *(.rel.init) }
-  .rela.init      : { *(.rela.init) }
-  .rel.text       : { *(.rel.text .rel.text.* .rel.gnu.linkonce.t.*) }
-  .rela.text      : { *(.rela.text .rela.text.* .rela.gnu.linkonce.t.*) }
-  .rel.fini       : { *(.rel.fini) }
-  .rela.fini      : { *(.rela.fini) }
-  .rel.rodata     : { *(.rel.rodata .rel.rodata.* .rel.gnu.linkonce.r.*) }
-  .rela.rodata    : { *(.rela.rodata .rela.rodata.* .rela.gnu.linkonce.r.*) }
-  .rel.data       : { *(.rel.data .rel.data.* .rel.gnu.linkonce.d.*) }
-  .rela.data      : { *(.rela.data .rela.data.* .rela.gnu.linkonce.d.*) }
-  .rel.tdata	  : { *(.rel.tdata .rel.tdata.* .rel.gnu.linkonce.td.*) }
-  .rela.tdata	  : { *(.rela.tdata .rela.tdata.* .rela.gnu.linkonce.td.*) }
-  .rel.tbss	  : { *(.rel.tbss .rel.tbss.* .rel.gnu.linkonce.tb.*) }
-  .rela.tbss	  : { *(.rela.tbss .rela.tbss.* .rela.gnu.linkonce.tb.*) }
-  .rel.ctors      : { *(.rel.ctors) }
-  .rela.ctors     : { *(.rela.ctors) }
-  .rel.dtors      : { *(.rel.dtors) }
-  .rela.dtors     : { *(.rela.dtors) }
-  .rel.got        : { *(.rel.got) }
-  .rela.got       : { *(.rela.got) }
-  .rel.bss        : { *(.rel.bss .rel.bss.* .rel.gnu.linkonce.b.*) }
-  .rela.bss       : { *(.rela.bss .rela.bss.* .rela.gnu.linkonce.b.*) }
-  .rel.plt : {
-	*(.rel.plt)
-	PROVIDE_HIDDEN(__rel_iplt_start = .);
-	*(.rel.iplt)
-	PROVIDE_HIDDEN(__rel_iplt_end = .);
-  }
-  .rela.plt : {
-	*(.rela.plt)
-	PROVIDE_HIDDEN(__rela_iplt_start = .);
-	*(.rela.iplt)
-	PROVIDE_HIDDEN(__rela_iplt_end = .);
-  }
-  .init           : {
-    KEEP (*(.init))
-  } =0x90909090
-  .plt            : { *(.plt) }
-  .text           : {
-    _stext = .;
-    TEXT_TEXT
-    SCHED_TEXT
-    CPUIDLE_TEXT
-    LOCK_TEXT
-    IRQENTRY_TEXT
-    SOFTIRQENTRY_TEXT
-    *(.fixup)
-    *(.stub .text.* .gnu.linkonce.t.*)
-    /* .gnu.warning sections are handled specially by elf32.em.  */
-    *(.gnu.warning)
-
-    . = ALIGN(PAGE_SIZE);
-  } =0x90909090
-  . = ALIGN(PAGE_SIZE);
-  .syscall_stub : {
-	__syscall_stub_start = .;
-	*(.__syscall_stub*)
-	__syscall_stub_end = .;
-  }
-  .fini           : {
-    KEEP (*(.fini))
-  } =0x90909090
-
-  .kstrtab : { *(.kstrtab) }
-
-  #include <asm/common.lds.S>
-
-  __init_begin = .;
-  init.data : { INIT_DATA }
-  __init_end = .;
-
-  /* Ensure the __preinit_array_start label is properly aligned.  We
-     could instead move the label definition inside the section, but
-     the linker would then create the section even if it turns out to
-     be empty, which isn't pretty.  */
-  . = ALIGN(32 / 8);
-  .preinit_array     : { *(.preinit_array) }
-  .init_array     : { *(.init_array) }
-  .fini_array     : { *(.fini_array) }
-  .data           : {
-    INIT_TASK_DATA(KERNEL_STACK_SIZE)
-    . = ALIGN(KERNEL_STACK_SIZE);
-    *(.data..init_irqstack)
-    DATA_DATA
-    *(.data.* .gnu.linkonce.d.*)
-    SORT(CONSTRUCTORS)
-  }
-  .data1          : { *(.data1) }
-  .tdata	  : { *(.tdata .tdata.* .gnu.linkonce.td.*) }
-  .tbss		  : { *(.tbss .tbss.* .gnu.linkonce.tb.*) *(.tcommon) }
-  .eh_frame       : { KEEP (*(.eh_frame)) }
-  .gcc_except_table   : { *(.gcc_except_table) }
-  .dynamic        : { *(.dynamic) }
-  .ctors          : {
-    /* gcc uses crtbegin.o to find the start of
-       the constructors, so we make sure it is
-       first.  Because this is a wildcard, it
-       doesn't matter if the user does not
-       actually link against crtbegin.o; the
-       linker won't look for a file to match a
-       wildcard.  The wildcard also means that it
-       doesn't matter which directory crtbegin.o
-       is in.  */
-    KEEP (*crtbegin.o(.ctors))
-    /* We don't want to include the .ctor section from
-       from the crtend.o file until after the sorted ctors.
-       The .ctor section from the crtend file contains the
-       end of ctors marker and it must be last */
-    KEEP (*(EXCLUDE_FILE (*crtend.o ) .ctors))
-    KEEP (*(SORT(.ctors.*)))
-    KEEP (*(.ctors))
-  }
-  .dtors          : {
-    KEEP (*crtbegin.o(.dtors))
-    KEEP (*(EXCLUDE_FILE (*crtend.o ) .dtors))
-    KEEP (*(SORT(.dtors.*)))
-    KEEP (*(.dtors))
-  }
-  .jcr            : { KEEP (*(.jcr)) }
-  .got            : { *(.got.plt) *(.got) }
-  _edata = .;
-  PROVIDE (edata = .);
-  .bss            : {
-   __bss_start = .;
-   *(.dynbss)
-   *(.bss .bss.* .gnu.linkonce.b.*)
-   *(COMMON)
-   /* Align here to ensure that the .bss section occupies space up to
-      _end.  Align after .bss to ensure correct alignment even if the
-      .bss section disappears because there are no input sections.  */
-   . = ALIGN(32 / 8);
-  . = ALIGN(32 / 8);
-  }
-   __bss_stop = .;
-  _end = .;
-  PROVIDE (end = .);
-
-  STABS_DEBUG
-
-  DWARF_DEBUG
-
-  DISCARDS
-}
diff --git a/arch/um/kernel/uml.lds.S b/arch/um/kernel/uml.lds.S
deleted file mode 100644
index 3b6dab3d4501..000000000000
--- a/arch/um/kernel/uml.lds.S
+++ /dev/null
@@ -1,115 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#include <asm/vmlinux.lds.h>
-#include <asm/page.h>
-
-OUTPUT_FORMAT(ELF_FORMAT)
-OUTPUT_ARCH(ELF_ARCH)
-ENTRY(_start)
-jiffies = jiffies_64;
-
-SECTIONS
-{
-  /* This must contain the right address - not quite the default ELF one.*/
-  PROVIDE (__executable_start = START);
-  /* Static binaries stick stuff here, like the sigreturn trampoline,
-   * invisibly to objdump.  So, just make __binary_start equal to the very
-   * beginning of the executable, and if there are unmapped pages after this,
-   * they are forever unusable.
-   */
-  __binary_start = START;
-
-  . = START + SIZEOF_HEADERS;
-  . = ALIGN(PAGE_SIZE);
-
-  _text = .;
-  INIT_TEXT_SECTION(0)
-
-  .text      :
-  {
-    _stext = .;
-    TEXT_TEXT
-    SCHED_TEXT
-    CPUIDLE_TEXT
-    LOCK_TEXT
-    IRQENTRY_TEXT
-    SOFTIRQENTRY_TEXT
-    *(.fixup)
-    /* .gnu.warning sections are handled specially by elf32.em.  */
-    *(.gnu.warning)
-    *(.gnu.linkonce.t*)
-  }
-
-  . = ALIGN(PAGE_SIZE);
-  .syscall_stub : {
-	__syscall_stub_start = .;
-	*(.__syscall_stub*)
-	__syscall_stub_end = .;
-  }
-
-  /*
-   * These are needed even in a static link, even if they wind up being empty.
-   * Newer glibc needs these __rel{,a}_iplt_{start,end} symbols.
-   */
-  .rel.plt : {
-	*(.rel.plt)
-	PROVIDE_HIDDEN(__rel_iplt_start = .);
-	*(.rel.iplt)
-	PROVIDE_HIDDEN(__rel_iplt_end = .);
-  }
-  .rela.plt : {
-	*(.rela.plt)
-	PROVIDE_HIDDEN(__rela_iplt_start = .);
-	*(.rela.iplt)
-	PROVIDE_HIDDEN(__rela_iplt_end = .);
-  }
-
-  #include <asm/common.lds.S>
-
-  __init_begin = .;
-  init.data : { INIT_DATA }
-  __init_end = .;
-
-  .data    :
-  {
-    INIT_TASK_DATA(KERNEL_STACK_SIZE)
-    . = ALIGN(KERNEL_STACK_SIZE);
-    *(.data..init_irqstack)
-    DATA_DATA
-    *(.gnu.linkonce.d*)
-    CONSTRUCTORS
-  }
-  .data1   : { *(.data1) }
-  .ctors         :
-  {
-    *(.ctors)
-  }
-  .dtors         :
-  {
-    *(.dtors)
-  }
-
-  .got           : { *(.got.plt) *(.got) }
-  .dynamic       : { *(.dynamic) }
-  .tdata	  : { *(.tdata .tdata.* .gnu.linkonce.td.*) }
-  .tbss		  : { *(.tbss .tbss.* .gnu.linkonce.tb.*) *(.tcommon) }
-  /* We want the small data sections together, so single-instruction offsets
-     can access them all, and initialized data all before uninitialized, so
-     we can shorten the on-disk segment size.  */
-  .sdata     : { *(.sdata) }
-  _edata  =  .;
-  PROVIDE (edata = .);
-  . = ALIGN(PAGE_SIZE);
-  __bss_start = .;
-  PROVIDE(_bss_start = .);
-  SBSS(0)
-  BSS(0)
-   __bss_stop = .;
-  _end = .;
-  PROVIDE (end = .);
-
-  STABS_DEBUG
-
-  DWARF_DEBUG
-
-  DISCARDS
-}
diff --git a/arch/um/kernel/vmlinux.lds.S b/arch/um/kernel/vmlinux.lds.S
index 16e49bfa2b42..aaedd66772fa 100644
--- a/arch/um/kernel/vmlinux.lds.S
+++ b/arch/um/kernel/vmlinux.lds.S
@@ -1,8 +1,87 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <asm/vmlinux.lds.h>
+#include <asm/thread_info.h>
+#include <asm/page.h>
+#include <asm/thread_info.h>
+#include <asm/cache.h>
+#include <linux/export.h>
 
-KERNEL_STACK_SIZE = 4096 * (1 << CONFIG_KERNEL_STACK_ORDER);
+OUTPUT_FORMAT(ELF_FORMAT)
+jiffies = jiffies_64;
 
-#ifdef CONFIG_LD_SCRIPT_STATIC
-#include "uml.lds.S"
-#else
-#include "dyn.lds.S"
-#endif
+
+SECTIONS
+{
+	__init_begin = . ;
+	HEAD_TEXT_SECTION
+	INIT_TEXT_SECTION(PAGE_SIZE)
+	INIT_DATA_SECTION(16)
+	PERCPU_SECTION(L1_CACHE_BYTES)
+	__init_end = . ;
+
+	_stext = . ;
+	_text = . ;
+	.text : ALIGN(THREAD_SIZE)
+	{
+		__binary_start = . ;
+		TEXT_TEXT
+		SCHED_TEXT
+		LOCK_TEXT
+		CPUIDLE_TEXT
+		IRQENTRY_TEXT
+		SOFTIRQENTRY_TEXT
+	}
+	_etext = . ;
+
+	.syscall_stub : ALIGN(PAGE_SIZE)
+	{
+		__syscall_stub_start = .;
+		*(.__syscall_stub*)
+		__syscall_stub_end = .;
+	}
+
+	_sdata = . ;
+	RO_DATA(PAGE_SIZE)
+	RW_DATA(L1_CACHE_BYTES, PAGE_SIZE, THREAD_SIZE)
+	.data : ALIGN(THREAD_SIZE)  { }
+
+	EXCEPTION_TABLE(16)
+
+	.uml.init_irqstack : ALIGN(THREAD_SIZE) {
+		*(.data..init_irqstack)
+	}
+
+	.uml.setup.init : ALIGN(8) {
+		__uml_setup_start = .;
+		*(.uml.setup.init)
+		__uml_setup_end = .;
+	}
+
+	.uml.help.init : ALIGN(8)  {
+		__uml_help_start = .;
+		*(.uml.help.init)
+		__uml_help_end = .;
+	}
+
+	.uml.postsetup.init : ALIGN(8) {
+		__uml_postsetup_start = .;
+		*(.uml.postsetup.init)
+		__uml_postsetup_end = .;
+	}
+
+	.uml.exitcall : ALIGN(8) {
+		__uml_exitcall_begin = .;
+		*(.uml.exitcall.exit)
+		__uml_exitcall_end = .;
+	}
+	_edata = . ;
+
+	BSS_SECTION(0, 0, 0)
+	_end = . ;
+
+	STABS_DEBUG
+
+	DWARF_DEBUG
+
+	DISCARDS
+}
diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
index d09ab4afbda4..0bbb626ace81 100755
--- a/scripts/link-vmlinux.sh
+++ b/scripts/link-vmlinux.sh
@@ -75,36 +75,18 @@ vmlinux_link()
 		strip_debug=-Wl,--strip-debug
 	fi
 
-	if [ "${SRCARCH}" != "um" ]; then
-		objects="--whole-archive			\
-			${KBUILD_VMLINUX_OBJS}			\
-			--no-whole-archive			\
-			--start-group				\
-			${KBUILD_VMLINUX_LIBS}			\
-			--end-group				\
-			${@}"
-
-		${LD} ${KBUILD_LDFLAGS} ${LDFLAGS_vmlinux}	\
-			${strip_debug#-Wl,}			\
-			-o ${output}				\
-			-T ${lds} ${objects}
-	else
-		objects="-Wl,--whole-archive			\
-			${KBUILD_VMLINUX_OBJS}			\
-			-Wl,--no-whole-archive			\
-			-Wl,--start-group			\
-			${KBUILD_VMLINUX_LIBS}			\
-			-Wl,--end-group				\
-			${@}"
-
-		${CC} ${CFLAGS_vmlinux}				\
-			${strip_debug}				\
-			-o ${output}				\
-			-Wl,-T,${lds}				\
-			${objects}				\
-			-lutil -lrt -lpthread
-		rm -f linux
-	fi
+	objects="--whole-archive			\
+		${KBUILD_VMLINUX_OBJS}			\
+		--no-whole-archive			\
+		--start-group				\
+		${KBUILD_VMLINUX_LIBS}			\
+		--end-group				\
+		${@}"
+
+	${LD} ${KBUILD_LDFLAGS} ${LDFLAGS_vmlinux}	\
+		${strip_debug#-Wl,}			\
+		-o ${output}				\
+		-T ${lds} ${objects}
 }
 
 # generate .BTF typeinfo from DWARF debuginfo
diff --git a/tools/um/Makefile b/tools/um/Makefile
new file mode 100644
index 000000000000..a63466ef7ef5
--- /dev/null
+++ b/tools/um/Makefile
@@ -0,0 +1,72 @@
+# Do not use make's built-in rules
+# (this improves performance and avoids hard-to-debug behaviour);
+# also do not print "Entering directory..." messages from make
+.SUFFIXES:
+MAKEFLAGS += -r --no-print-directory
+
+KCONFIG?=defconfig
+
+ifneq ($(silent),1)
+  ifneq ($(V),1)
+	QUIET_AUTOCONF       = @echo '  AUTOCONF '$@;
+	Q = @
+  endif
+endif
+
+PREFIX   := /usr
+
+ifeq (,$(srctree))
+  srctree := $(patsubst %/,%,$(dir $(shell pwd)))
+  srctree := $(patsubst %/,%,$(dir $(srctree)))
+endif
+export srctree
+
+-include ../scripts/Makefile.include
+
+# OUTPUT fixup should be *after* include ../scripts/Makefile.include
+ifneq ($(OUTPUT),)
+  OUTPUT := $(OUTPUT)/tools/um/
+else
+  OUTPUT := $(CURDIR)/
+endif
+export OUTPUT
+export objtree := $(OUTPUT)/../..
+
+
+all:
+
+export CFLAGS += -I$(OUTPUT)/include -Iinclude -Wall -g -O2 -Wextra -fPIC \
+	 -Wno-unused-parameter \
+	 -Wno-missing-field-initializers -fno-strict-aliasing
+
+-include Targets
+
+TARGETS := $(progs-y:%=$(OUTPUT)%)
+TARGETS += $(libs-y:%=$(OUTPUT)%)
+all: $(TARGETS)
+
+# rule to build linux.o
+$(OUTPUT)lib/linux.o:
+	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) $(KCONFIG)
+	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) install INSTALL_PATH=$(OUTPUT)
+
+$(OUTPUT)liblinux.a: $(OUTPUT)lib/linux.o $(OUTPUT)uml/liblinux-in.o
+	$(QUIET_AR)$(AR) -rc $@ $^
+
+# rule to link programs
+$(OUTPUT)%: $(OUTPUT)%-in.o $(OUTPUT)liblinux.a
+	$(QUIET_LINK)$(CC) $(LDFLAGS) $(LDFLAGS_$(notdir $*)-y) -o $@ $^ $(LDLIBS) $(LDLIBS_$(notdir $*)-y)
+
+$(OUTPUT)%-in.o: FORCE
+	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
+
+clean:
+	$(call QUIET_CLEAN, objects)find $(OUTPUT) -name '*.o' -delete -o -name '\.*.cmd'\
+	 -delete -o -name '\.*.d' -delete
+	$(call QUIET_CLEAN, liblinux.a)$(RM) $(OUTPUT)/liblinux.a
+	$(call QUIET_CLEAN, $(TARGETS))$(RM) $(TARGETS)
+
+FORCE: ;
+.PHONY: all clean FORCE
+.IGNORE: clean
+.NOTPARALLEL : lib/linux.o
diff --git a/tools/um/Targets b/tools/um/Targets
new file mode 100644
index 000000000000..a4711f1ef422
--- /dev/null
+++ b/tools/um/Targets
@@ -0,0 +1,4 @@
+progs-y += uml/linux
+LDLIBS_linux-y := -lrt -lpthread -lutil
+LDFLAGS_linux-y := -no-pie -Wl,--wrap,malloc -Wl,--wrap,free -Wl,--wrap,calloc
+LDFLAGS_linux-$(UML_STATIC) += -static
diff --git a/tools/um/uml/Build b/tools/um/uml/Build
new file mode 100644
index 000000000000..e69de29bb2d1
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v5 01/21] um: split build in kernel and host parts
@ 2020-07-02 14:06       ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-07-02 14:06 UTC (permalink / raw)
  To: linux-um
  Cc: linux-arch, Octavian Purdila, Octavian Purdila, Akira Moroo,
	linux-kernel-library, Hajime Tazaki

From: Octavian Purdila <tavi@cs.pub.ro>

This patch splits the UML build in two parts: a kernel part that
generates a relocatable object (linux.o) and a host build part that
links the relocatable object into an executable.

This allows us to simplify the linker script since we can now remove
the host bits (e.g. .rel sections). It also gives us greater
flexibility since it will be much easier to support other
architectures and OSes if we use the standard linker script.

The host build part has been implemented in tools/um so that we can
reuse the available host build infrastructure.

The patch also changes the UML build invocation, if before

 $ make ARCH=um defconfig
 $ make ARCH=um

was generating the executable now this only generates the relocatable
object.

To fully build UML use:

 $ make -C tools/um

which will generate tools/lkl/linux as the UML executable. To
statically compiled UML use:

 $ make -C tools/lkl UML_STATIC=y

Signed-off-by: Octavian Purdila <tavi@cs.pub.ro>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/Kconfig                  |  12 +--
 arch/um/Makefile                 |  14 ++-
 arch/um/include/asm/common.lds.S |  99 ------------------
 arch/um/kernel/dyn.lds.S         | 171 -------------------------------
 arch/um/kernel/uml.lds.S         | 115 ---------------------
 arch/um/kernel/vmlinux.lds.S     |  91 ++++++++++++++--
 scripts/link-vmlinux.sh          |  42 +++-----
 tools/um/Makefile                |  72 +++++++++++++
 tools/um/Targets                 |   4 +
 tools/um/uml/Build               |   0
 10 files changed, 184 insertions(+), 436 deletions(-)
 delete mode 100644 arch/um/include/asm/common.lds.S
 delete mode 100644 arch/um/kernel/dyn.lds.S
 delete mode 100644 arch/um/kernel/uml.lds.S
 create mode 100644 tools/um/Makefile
 create mode 100644 tools/um/Targets
 create mode 100644 tools/um/uml/Build

diff --git a/arch/um/Kconfig b/arch/um/Kconfig
index 96ab7026b037..a0a9fd3ab96c 100644
--- a/arch/um/Kconfig
+++ b/arch/um/Kconfig
@@ -20,6 +20,7 @@ config UML
 	select GENERIC_CLOCKEVENTS
 	select HAVE_GCC_PLUGINS
 	select TTY # Needed for line.c
+	select MODULE_REL_CRCS if MODVERSIONS
 
 config MMU
 	bool
@@ -79,17 +80,6 @@ config STATIC_LINK
 	  NOTE: This option is incompatible with some networking features which
 	  depend on features that require being dynamically loaded (like NSS).
 
-config LD_SCRIPT_STATIC
-	bool
-	default y
-	depends on STATIC_LINK
-
-config LD_SCRIPT_DYN
-	bool
-	default y
-	depends on !LD_SCRIPT_STATIC
-	select MODULE_REL_CRCS if MODVERSIONS
-
 config HOSTFS
 	tristate "Host filesystem"
 	help
diff --git a/arch/um/Makefile b/arch/um/Makefile
index 275f5ffdf6f0..c952ddefcc1c 100644
--- a/arch/um/Makefile
+++ b/arch/um/Makefile
@@ -95,14 +95,20 @@ KBUILD_CPPFLAGS += -I$(srctree)/$(HOST_DIR)/include \
 KERNEL_DEFINES = $(strip -Derrno=kernel_errno -Dsigprocmask=kernel_sigprocmask \
 			 -Dmktime=kernel_mktime $(ARCH_KERNEL_DEFINES))
 KBUILD_CFLAGS += $(KERNEL_DEFINES)
+LDFLAGS_vmlinux += -r
 
-PHONY += linux
+PHONY += linux.o
 
-all: linux
+all: linux.o
 
-linux: vmlinux
+linux.o: vmlinux
 	@echo '  LINK $@'
-	$(Q)ln -f $< $@
+	$(Q)$(OBJCOPY) -R .eh_frame $< $@
+
+install: linux.o
+	@echo "  INSTALL $(INSTALL_PATH)/lib/$<"
+	@mkdir -p $(INSTALL_PATH)/lib/
+	@cp $< $(INSTALL_PATH)/lib/
 
 define archhelp
   echo '* linux		- Binary kernel image (./linux) - for backward'
diff --git a/arch/um/include/asm/common.lds.S b/arch/um/include/asm/common.lds.S
deleted file mode 100644
index eca6c452a41b..000000000000
--- a/arch/um/include/asm/common.lds.S
+++ /dev/null
@@ -1,99 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#include <asm-generic/vmlinux.lds.h>
-
-  .fini      : { *(.fini)    } =0x9090
-  _etext = .;
-  PROVIDE (etext = .);
-
-  . = ALIGN(4096);
-  _sdata = .;
-  PROVIDE (sdata = .);
-
-  RO_DATA(4096)
-
-  .unprotected : { *(.unprotected) }
-  . = ALIGN(4096);
-  PROVIDE (_unprotected_end = .);
-
-  . = ALIGN(4096);
-  EXCEPTION_TABLE(0)
-
-  BUG_TABLE
-
-  .uml.setup.init : {
-	__uml_setup_start = .;
-	*(.uml.setup.init)
-	__uml_setup_end = .;
-  }
-	
-  .uml.help.init : {
-	__uml_help_start = .;
-	*(.uml.help.init)
-	__uml_help_end = .;
-  }
-	
-  .uml.postsetup.init : {
-	__uml_postsetup_start = .;
-	*(.uml.postsetup.init)
-	__uml_postsetup_end = .;
-  }
-	
-  .init.setup : {
-	INIT_SETUP(0)
-  }
-
-  PERCPU_SECTION(32)
-	
-  .initcall.init : {
-	INIT_CALLS
-  }
-
-  .con_initcall.init : {
-	CON_INITCALL
-  }
-
-  .exitcall : {
-	__exitcall_begin = .;
-	*(.exitcall.exit)
-	__exitcall_end = .;
-  }
-
-  .uml.exitcall : {
-	__uml_exitcall_begin = .;
-	*(.uml.exitcall.exit)
-	__uml_exitcall_end = .;
-  }
-
-  . = ALIGN(4);
-  .altinstructions : {
-	__alt_instructions = .;
-	*(.altinstructions)
-	__alt_instructions_end = .;
-  }
-  .altinstr_replacement : { *(.altinstr_replacement) }
-  /* .exit.text is discard at runtime, not link time, to deal with references
-     from .altinstructions and .eh_frame */
-  .exit.text : { EXIT_TEXT }
-  .exit.data : { *(.exit.data) }
-
-  .preinit_array : {
-	__preinit_array_start = .;
-	*(.preinit_array)
-	__preinit_array_end = .;
-  }
-  .init_array : {
-	__init_array_start = .;
-	*(.init_array)
-	__init_array_end = .;
-  }
-  .fini_array : {
-	__fini_array_start = .;
-	*(.fini_array)
-	__fini_array_end = .;
-  }
-
-   . = ALIGN(4096);
-  .init.ramfs : {
-	INIT_RAM_FS
-  }
-
diff --git a/arch/um/kernel/dyn.lds.S b/arch/um/kernel/dyn.lds.S
deleted file mode 100644
index f5001481010c..000000000000
--- a/arch/um/kernel/dyn.lds.S
+++ /dev/null
@@ -1,171 +0,0 @@
-#include <asm/vmlinux.lds.h>
-#include <asm/page.h>
-
-OUTPUT_FORMAT(ELF_FORMAT)
-OUTPUT_ARCH(ELF_ARCH)
-ENTRY(_start)
-jiffies = jiffies_64;
-
-SECTIONS
-{
-  PROVIDE (__executable_start = START);
-  . = START + SIZEOF_HEADERS;
-  .interp         : { *(.interp) }
-  __binary_start = .;
-  . = ALIGN(4096);		/* Init code and data */
-  _text = .;
-  INIT_TEXT_SECTION(PAGE_SIZE)
-
-  . = ALIGN(PAGE_SIZE);
-
-  /* Read-only sections, merged into text segment: */
-  .hash           : { *(.hash) }
-  .gnu.hash       : { *(.gnu.hash) }
-  .dynsym         : { *(.dynsym) }
-  .dynstr         : { *(.dynstr) }
-  .gnu.version    : { *(.gnu.version) }
-  .gnu.version_d  : { *(.gnu.version_d) }
-  .gnu.version_r  : { *(.gnu.version_r) }
-  .rel.init       : { *(.rel.init) }
-  .rela.init      : { *(.rela.init) }
-  .rel.text       : { *(.rel.text .rel.text.* .rel.gnu.linkonce.t.*) }
-  .rela.text      : { *(.rela.text .rela.text.* .rela.gnu.linkonce.t.*) }
-  .rel.fini       : { *(.rel.fini) }
-  .rela.fini      : { *(.rela.fini) }
-  .rel.rodata     : { *(.rel.rodata .rel.rodata.* .rel.gnu.linkonce.r.*) }
-  .rela.rodata    : { *(.rela.rodata .rela.rodata.* .rela.gnu.linkonce.r.*) }
-  .rel.data       : { *(.rel.data .rel.data.* .rel.gnu.linkonce.d.*) }
-  .rela.data      : { *(.rela.data .rela.data.* .rela.gnu.linkonce.d.*) }
-  .rel.tdata	  : { *(.rel.tdata .rel.tdata.* .rel.gnu.linkonce.td.*) }
-  .rela.tdata	  : { *(.rela.tdata .rela.tdata.* .rela.gnu.linkonce.td.*) }
-  .rel.tbss	  : { *(.rel.tbss .rel.tbss.* .rel.gnu.linkonce.tb.*) }
-  .rela.tbss	  : { *(.rela.tbss .rela.tbss.* .rela.gnu.linkonce.tb.*) }
-  .rel.ctors      : { *(.rel.ctors) }
-  .rela.ctors     : { *(.rela.ctors) }
-  .rel.dtors      : { *(.rel.dtors) }
-  .rela.dtors     : { *(.rela.dtors) }
-  .rel.got        : { *(.rel.got) }
-  .rela.got       : { *(.rela.got) }
-  .rel.bss        : { *(.rel.bss .rel.bss.* .rel.gnu.linkonce.b.*) }
-  .rela.bss       : { *(.rela.bss .rela.bss.* .rela.gnu.linkonce.b.*) }
-  .rel.plt : {
-	*(.rel.plt)
-	PROVIDE_HIDDEN(__rel_iplt_start = .);
-	*(.rel.iplt)
-	PROVIDE_HIDDEN(__rel_iplt_end = .);
-  }
-  .rela.plt : {
-	*(.rela.plt)
-	PROVIDE_HIDDEN(__rela_iplt_start = .);
-	*(.rela.iplt)
-	PROVIDE_HIDDEN(__rela_iplt_end = .);
-  }
-  .init           : {
-    KEEP (*(.init))
-  } =0x90909090
-  .plt            : { *(.plt) }
-  .text           : {
-    _stext = .;
-    TEXT_TEXT
-    SCHED_TEXT
-    CPUIDLE_TEXT
-    LOCK_TEXT
-    IRQENTRY_TEXT
-    SOFTIRQENTRY_TEXT
-    *(.fixup)
-    *(.stub .text.* .gnu.linkonce.t.*)
-    /* .gnu.warning sections are handled specially by elf32.em.  */
-    *(.gnu.warning)
-
-    . = ALIGN(PAGE_SIZE);
-  } =0x90909090
-  . = ALIGN(PAGE_SIZE);
-  .syscall_stub : {
-	__syscall_stub_start = .;
-	*(.__syscall_stub*)
-	__syscall_stub_end = .;
-  }
-  .fini           : {
-    KEEP (*(.fini))
-  } =0x90909090
-
-  .kstrtab : { *(.kstrtab) }
-
-  #include <asm/common.lds.S>
-
-  __init_begin = .;
-  init.data : { INIT_DATA }
-  __init_end = .;
-
-  /* Ensure the __preinit_array_start label is properly aligned.  We
-     could instead move the label definition inside the section, but
-     the linker would then create the section even if it turns out to
-     be empty, which isn't pretty.  */
-  . = ALIGN(32 / 8);
-  .preinit_array     : { *(.preinit_array) }
-  .init_array     : { *(.init_array) }
-  .fini_array     : { *(.fini_array) }
-  .data           : {
-    INIT_TASK_DATA(KERNEL_STACK_SIZE)
-    . = ALIGN(KERNEL_STACK_SIZE);
-    *(.data..init_irqstack)
-    DATA_DATA
-    *(.data.* .gnu.linkonce.d.*)
-    SORT(CONSTRUCTORS)
-  }
-  .data1          : { *(.data1) }
-  .tdata	  : { *(.tdata .tdata.* .gnu.linkonce.td.*) }
-  .tbss		  : { *(.tbss .tbss.* .gnu.linkonce.tb.*) *(.tcommon) }
-  .eh_frame       : { KEEP (*(.eh_frame)) }
-  .gcc_except_table   : { *(.gcc_except_table) }
-  .dynamic        : { *(.dynamic) }
-  .ctors          : {
-    /* gcc uses crtbegin.o to find the start of
-       the constructors, so we make sure it is
-       first.  Because this is a wildcard, it
-       doesn't matter if the user does not
-       actually link against crtbegin.o; the
-       linker won't look for a file to match a
-       wildcard.  The wildcard also means that it
-       doesn't matter which directory crtbegin.o
-       is in.  */
-    KEEP (*crtbegin.o(.ctors))
-    /* We don't want to include the .ctor section from
-       from the crtend.o file until after the sorted ctors.
-       The .ctor section from the crtend file contains the
-       end of ctors marker and it must be last */
-    KEEP (*(EXCLUDE_FILE (*crtend.o ) .ctors))
-    KEEP (*(SORT(.ctors.*)))
-    KEEP (*(.ctors))
-  }
-  .dtors          : {
-    KEEP (*crtbegin.o(.dtors))
-    KEEP (*(EXCLUDE_FILE (*crtend.o ) .dtors))
-    KEEP (*(SORT(.dtors.*)))
-    KEEP (*(.dtors))
-  }
-  .jcr            : { KEEP (*(.jcr)) }
-  .got            : { *(.got.plt) *(.got) }
-  _edata = .;
-  PROVIDE (edata = .);
-  .bss            : {
-   __bss_start = .;
-   *(.dynbss)
-   *(.bss .bss.* .gnu.linkonce.b.*)
-   *(COMMON)
-   /* Align here to ensure that the .bss section occupies space up to
-      _end.  Align after .bss to ensure correct alignment even if the
-      .bss section disappears because there are no input sections.  */
-   . = ALIGN(32 / 8);
-  . = ALIGN(32 / 8);
-  }
-   __bss_stop = .;
-  _end = .;
-  PROVIDE (end = .);
-
-  STABS_DEBUG
-
-  DWARF_DEBUG
-
-  DISCARDS
-}
diff --git a/arch/um/kernel/uml.lds.S b/arch/um/kernel/uml.lds.S
deleted file mode 100644
index 3b6dab3d4501..000000000000
--- a/arch/um/kernel/uml.lds.S
+++ /dev/null
@@ -1,115 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#include <asm/vmlinux.lds.h>
-#include <asm/page.h>
-
-OUTPUT_FORMAT(ELF_FORMAT)
-OUTPUT_ARCH(ELF_ARCH)
-ENTRY(_start)
-jiffies = jiffies_64;
-
-SECTIONS
-{
-  /* This must contain the right address - not quite the default ELF one.*/
-  PROVIDE (__executable_start = START);
-  /* Static binaries stick stuff here, like the sigreturn trampoline,
-   * invisibly to objdump.  So, just make __binary_start equal to the very
-   * beginning of the executable, and if there are unmapped pages after this,
-   * they are forever unusable.
-   */
-  __binary_start = START;
-
-  . = START + SIZEOF_HEADERS;
-  . = ALIGN(PAGE_SIZE);
-
-  _text = .;
-  INIT_TEXT_SECTION(0)
-
-  .text      :
-  {
-    _stext = .;
-    TEXT_TEXT
-    SCHED_TEXT
-    CPUIDLE_TEXT
-    LOCK_TEXT
-    IRQENTRY_TEXT
-    SOFTIRQENTRY_TEXT
-    *(.fixup)
-    /* .gnu.warning sections are handled specially by elf32.em.  */
-    *(.gnu.warning)
-    *(.gnu.linkonce.t*)
-  }
-
-  . = ALIGN(PAGE_SIZE);
-  .syscall_stub : {
-	__syscall_stub_start = .;
-	*(.__syscall_stub*)
-	__syscall_stub_end = .;
-  }
-
-  /*
-   * These are needed even in a static link, even if they wind up being empty.
-   * Newer glibc needs these __rel{,a}_iplt_{start,end} symbols.
-   */
-  .rel.plt : {
-	*(.rel.plt)
-	PROVIDE_HIDDEN(__rel_iplt_start = .);
-	*(.rel.iplt)
-	PROVIDE_HIDDEN(__rel_iplt_end = .);
-  }
-  .rela.plt : {
-	*(.rela.plt)
-	PROVIDE_HIDDEN(__rela_iplt_start = .);
-	*(.rela.iplt)
-	PROVIDE_HIDDEN(__rela_iplt_end = .);
-  }
-
-  #include <asm/common.lds.S>
-
-  __init_begin = .;
-  init.data : { INIT_DATA }
-  __init_end = .;
-
-  .data    :
-  {
-    INIT_TASK_DATA(KERNEL_STACK_SIZE)
-    . = ALIGN(KERNEL_STACK_SIZE);
-    *(.data..init_irqstack)
-    DATA_DATA
-    *(.gnu.linkonce.d*)
-    CONSTRUCTORS
-  }
-  .data1   : { *(.data1) }
-  .ctors         :
-  {
-    *(.ctors)
-  }
-  .dtors         :
-  {
-    *(.dtors)
-  }
-
-  .got           : { *(.got.plt) *(.got) }
-  .dynamic       : { *(.dynamic) }
-  .tdata	  : { *(.tdata .tdata.* .gnu.linkonce.td.*) }
-  .tbss		  : { *(.tbss .tbss.* .gnu.linkonce.tb.*) *(.tcommon) }
-  /* We want the small data sections together, so single-instruction offsets
-     can access them all, and initialized data all before uninitialized, so
-     we can shorten the on-disk segment size.  */
-  .sdata     : { *(.sdata) }
-  _edata  =  .;
-  PROVIDE (edata = .);
-  . = ALIGN(PAGE_SIZE);
-  __bss_start = .;
-  PROVIDE(_bss_start = .);
-  SBSS(0)
-  BSS(0)
-   __bss_stop = .;
-  _end = .;
-  PROVIDE (end = .);
-
-  STABS_DEBUG
-
-  DWARF_DEBUG
-
-  DISCARDS
-}
diff --git a/arch/um/kernel/vmlinux.lds.S b/arch/um/kernel/vmlinux.lds.S
index 16e49bfa2b42..aaedd66772fa 100644
--- a/arch/um/kernel/vmlinux.lds.S
+++ b/arch/um/kernel/vmlinux.lds.S
@@ -1,8 +1,87 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <asm/vmlinux.lds.h>
+#include <asm/thread_info.h>
+#include <asm/page.h>
+#include <asm/thread_info.h>
+#include <asm/cache.h>
+#include <linux/export.h>
 
-KERNEL_STACK_SIZE = 4096 * (1 << CONFIG_KERNEL_STACK_ORDER);
+OUTPUT_FORMAT(ELF_FORMAT)
+jiffies = jiffies_64;
 
-#ifdef CONFIG_LD_SCRIPT_STATIC
-#include "uml.lds.S"
-#else
-#include "dyn.lds.S"
-#endif
+
+SECTIONS
+{
+	__init_begin = . ;
+	HEAD_TEXT_SECTION
+	INIT_TEXT_SECTION(PAGE_SIZE)
+	INIT_DATA_SECTION(16)
+	PERCPU_SECTION(L1_CACHE_BYTES)
+	__init_end = . ;
+
+	_stext = . ;
+	_text = . ;
+	.text : ALIGN(THREAD_SIZE)
+	{
+		__binary_start = . ;
+		TEXT_TEXT
+		SCHED_TEXT
+		LOCK_TEXT
+		CPUIDLE_TEXT
+		IRQENTRY_TEXT
+		SOFTIRQENTRY_TEXT
+	}
+	_etext = . ;
+
+	.syscall_stub : ALIGN(PAGE_SIZE)
+	{
+		__syscall_stub_start = .;
+		*(.__syscall_stub*)
+		__syscall_stub_end = .;
+	}
+
+	_sdata = . ;
+	RO_DATA(PAGE_SIZE)
+	RW_DATA(L1_CACHE_BYTES, PAGE_SIZE, THREAD_SIZE)
+	.data : ALIGN(THREAD_SIZE)  { }
+
+	EXCEPTION_TABLE(16)
+
+	.uml.init_irqstack : ALIGN(THREAD_SIZE) {
+		*(.data..init_irqstack)
+	}
+
+	.uml.setup.init : ALIGN(8) {
+		__uml_setup_start = .;
+		*(.uml.setup.init)
+		__uml_setup_end = .;
+	}
+
+	.uml.help.init : ALIGN(8)  {
+		__uml_help_start = .;
+		*(.uml.help.init)
+		__uml_help_end = .;
+	}
+
+	.uml.postsetup.init : ALIGN(8) {
+		__uml_postsetup_start = .;
+		*(.uml.postsetup.init)
+		__uml_postsetup_end = .;
+	}
+
+	.uml.exitcall : ALIGN(8) {
+		__uml_exitcall_begin = .;
+		*(.uml.exitcall.exit)
+		__uml_exitcall_end = .;
+	}
+	_edata = . ;
+
+	BSS_SECTION(0, 0, 0)
+	_end = . ;
+
+	STABS_DEBUG
+
+	DWARF_DEBUG
+
+	DISCARDS
+}
diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
index d09ab4afbda4..0bbb626ace81 100755
--- a/scripts/link-vmlinux.sh
+++ b/scripts/link-vmlinux.sh
@@ -75,36 +75,18 @@ vmlinux_link()
 		strip_debug=-Wl,--strip-debug
 	fi
 
-	if [ "${SRCARCH}" != "um" ]; then
-		objects="--whole-archive			\
-			${KBUILD_VMLINUX_OBJS}			\
-			--no-whole-archive			\
-			--start-group				\
-			${KBUILD_VMLINUX_LIBS}			\
-			--end-group				\
-			${@}"
-
-		${LD} ${KBUILD_LDFLAGS} ${LDFLAGS_vmlinux}	\
-			${strip_debug#-Wl,}			\
-			-o ${output}				\
-			-T ${lds} ${objects}
-	else
-		objects="-Wl,--whole-archive			\
-			${KBUILD_VMLINUX_OBJS}			\
-			-Wl,--no-whole-archive			\
-			-Wl,--start-group			\
-			${KBUILD_VMLINUX_LIBS}			\
-			-Wl,--end-group				\
-			${@}"
-
-		${CC} ${CFLAGS_vmlinux}				\
-			${strip_debug}				\
-			-o ${output}				\
-			-Wl,-T,${lds}				\
-			${objects}				\
-			-lutil -lrt -lpthread
-		rm -f linux
-	fi
+	objects="--whole-archive			\
+		${KBUILD_VMLINUX_OBJS}			\
+		--no-whole-archive			\
+		--start-group				\
+		${KBUILD_VMLINUX_LIBS}			\
+		--end-group				\
+		${@}"
+
+	${LD} ${KBUILD_LDFLAGS} ${LDFLAGS_vmlinux}	\
+		${strip_debug#-Wl,}			\
+		-o ${output}				\
+		-T ${lds} ${objects}
 }
 
 # generate .BTF typeinfo from DWARF debuginfo
diff --git a/tools/um/Makefile b/tools/um/Makefile
new file mode 100644
index 000000000000..a63466ef7ef5
--- /dev/null
+++ b/tools/um/Makefile
@@ -0,0 +1,72 @@
+# Do not use make's built-in rules
+# (this improves performance and avoids hard-to-debug behaviour);
+# also do not print "Entering directory..." messages from make
+.SUFFIXES:
+MAKEFLAGS += -r --no-print-directory
+
+KCONFIG?=defconfig
+
+ifneq ($(silent),1)
+  ifneq ($(V),1)
+	QUIET_AUTOCONF       = @echo '  AUTOCONF '$@;
+	Q = @
+  endif
+endif
+
+PREFIX   := /usr
+
+ifeq (,$(srctree))
+  srctree := $(patsubst %/,%,$(dir $(shell pwd)))
+  srctree := $(patsubst %/,%,$(dir $(srctree)))
+endif
+export srctree
+
+-include ../scripts/Makefile.include
+
+# OUTPUT fixup should be *after* include ../scripts/Makefile.include
+ifneq ($(OUTPUT),)
+  OUTPUT := $(OUTPUT)/tools/um/
+else
+  OUTPUT := $(CURDIR)/
+endif
+export OUTPUT
+export objtree := $(OUTPUT)/../..
+
+
+all:
+
+export CFLAGS += -I$(OUTPUT)/include -Iinclude -Wall -g -O2 -Wextra -fPIC \
+	 -Wno-unused-parameter \
+	 -Wno-missing-field-initializers -fno-strict-aliasing
+
+-include Targets
+
+TARGETS := $(progs-y:%=$(OUTPUT)%)
+TARGETS += $(libs-y:%=$(OUTPUT)%)
+all: $(TARGETS)
+
+# rule to build linux.o
+$(OUTPUT)lib/linux.o:
+	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) $(KCONFIG)
+	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) install INSTALL_PATH=$(OUTPUT)
+
+$(OUTPUT)liblinux.a: $(OUTPUT)lib/linux.o $(OUTPUT)uml/liblinux-in.o
+	$(QUIET_AR)$(AR) -rc $@ $^
+
+# rule to link programs
+$(OUTPUT)%: $(OUTPUT)%-in.o $(OUTPUT)liblinux.a
+	$(QUIET_LINK)$(CC) $(LDFLAGS) $(LDFLAGS_$(notdir $*)-y) -o $@ $^ $(LDLIBS) $(LDLIBS_$(notdir $*)-y)
+
+$(OUTPUT)%-in.o: FORCE
+	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
+
+clean:
+	$(call QUIET_CLEAN, objects)find $(OUTPUT) -name '*.o' -delete -o -name '\.*.cmd'\
+	 -delete -o -name '\.*.d' -delete
+	$(call QUIET_CLEAN, liblinux.a)$(RM) $(OUTPUT)/liblinux.a
+	$(call QUIET_CLEAN, $(TARGETS))$(RM) $(TARGETS)
+
+FORCE: ;
+.PHONY: all clean FORCE
+.IGNORE: clean
+.NOTPARALLEL : lib/linux.o
diff --git a/tools/um/Targets b/tools/um/Targets
new file mode 100644
index 000000000000..a4711f1ef422
--- /dev/null
+++ b/tools/um/Targets
@@ -0,0 +1,4 @@
+progs-y += uml/linux
+LDLIBS_linux-y := -lrt -lpthread -lutil
+LDFLAGS_linux-y := -no-pie -Wl,--wrap,malloc -Wl,--wrap,free -Wl,--wrap,calloc
+LDFLAGS_linux-$(UML_STATIC) += -static
diff --git a/tools/um/uml/Build b/tools/um/uml/Build
new file mode 100644
index 000000000000..e69de29bb2d1
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v5 02/21] um: add os init and exit calls
  2020-07-02 14:06     ` Hajime Tazaki
@ 2020-07-02 14:06       ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-07-02 14:06 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Octavian Purdila

From: Octavian Purdila <tavi@cs.pub.ro>

These calls are added in preparation for moving some of the code from
the kernel to the host build.

Signed-off-by: Octavian Purdila <tavi@cs.pub.ro>
---
 arch/um/include/shared/init.h | 14 ++++----------
 arch/um/kernel/reboot.c       |  5 +++++
 arch/um/kernel/um_arch.c      | 11 +++++++++++
 3 files changed, 20 insertions(+), 10 deletions(-)

diff --git a/arch/um/include/shared/init.h b/arch/um/include/shared/init.h
index c66de434a983..d09308330ca5 100644
--- a/arch/um/include/shared/init.h
+++ b/arch/um/include/shared/init.h
@@ -109,19 +109,13 @@ extern struct uml_param __uml_setup_start, __uml_setup_end;
 
 #ifdef __UM_HOST__
 
-#define __define_initcall(level,fn) \
-	static initcall_t __initcall_##fn __used \
-	__attribute__((__section__(".initcall" level ".init"))) = fn
-
-/* Userspace initcalls shouldn't depend on anything in the kernel, so we'll
- * make them run first.
- */
-#define __initcall(fn) __define_initcall("1", fn)
+#undef __uml_exit_call
+#define __uml_exit_call		__used __section(os_exitcalls)
+#define __init_call		__used __section(os_initcalls)
 
+#define __initcall(fn) static initcall_t __initcall_##fn __init_call = fn
 #define __exitcall(fn) static exitcall_t __exitcall_##fn __exit_call = fn
 
-#define __init_call	__used __section(.initcall.init)
-
 #endif
 
 #endif /* _LINUX_UML_INIT_H */
diff --git a/arch/um/kernel/reboot.c b/arch/um/kernel/reboot.c
index 48c0610d506e..5420aec411f4 100644
--- a/arch/um/kernel/reboot.c
+++ b/arch/um/kernel/reboot.c
@@ -35,10 +35,15 @@ static void kill_off_processes(void)
 	read_unlock(&tasklist_lock);
 }
 
+void __weak os_exitcalls(void)
+{
+}
+
 void uml_cleanup(void)
 {
 	kmalloc_ok = 0;
 	do_uml_exitcalls();
+	os_exitcalls();
 	kill_off_processes();
 }
 
diff --git a/arch/um/kernel/um_arch.c b/arch/um/kernel/um_arch.c
index 0f40eccbd759..78d6042fd2e6 100644
--- a/arch/um/kernel/um_arch.c
+++ b/arch/um/kernel/um_arch.c
@@ -362,3 +362,14 @@ void __init check_bugs(void)
 void apply_alternatives(struct alt_instr *start, struct alt_instr *end)
 {
 }
+
+int __weak os_initcalls(void)
+{
+	return 0;
+}
+
+int __init run_os_initcalls(void)
+{
+	return os_initcalls();
+}
+__initcall(run_os_initcalls);
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v5 02/21] um: add os init and exit calls
@ 2020-07-02 14:06       ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-07-02 14:06 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, linux-kernel-library, linux-arch,
	Octavian Purdila, Akira Moroo

From: Octavian Purdila <tavi@cs.pub.ro>

These calls are added in preparation for moving some of the code from
the kernel to the host build.

Signed-off-by: Octavian Purdila <tavi@cs.pub.ro>
---
 arch/um/include/shared/init.h | 14 ++++----------
 arch/um/kernel/reboot.c       |  5 +++++
 arch/um/kernel/um_arch.c      | 11 +++++++++++
 3 files changed, 20 insertions(+), 10 deletions(-)

diff --git a/arch/um/include/shared/init.h b/arch/um/include/shared/init.h
index c66de434a983..d09308330ca5 100644
--- a/arch/um/include/shared/init.h
+++ b/arch/um/include/shared/init.h
@@ -109,19 +109,13 @@ extern struct uml_param __uml_setup_start, __uml_setup_end;
 
 #ifdef __UM_HOST__
 
-#define __define_initcall(level,fn) \
-	static initcall_t __initcall_##fn __used \
-	__attribute__((__section__(".initcall" level ".init"))) = fn
-
-/* Userspace initcalls shouldn't depend on anything in the kernel, so we'll
- * make them run first.
- */
-#define __initcall(fn) __define_initcall("1", fn)
+#undef __uml_exit_call
+#define __uml_exit_call		__used __section(os_exitcalls)
+#define __init_call		__used __section(os_initcalls)
 
+#define __initcall(fn) static initcall_t __initcall_##fn __init_call = fn
 #define __exitcall(fn) static exitcall_t __exitcall_##fn __exit_call = fn
 
-#define __init_call	__used __section(.initcall.init)
-
 #endif
 
 #endif /* _LINUX_UML_INIT_H */
diff --git a/arch/um/kernel/reboot.c b/arch/um/kernel/reboot.c
index 48c0610d506e..5420aec411f4 100644
--- a/arch/um/kernel/reboot.c
+++ b/arch/um/kernel/reboot.c
@@ -35,10 +35,15 @@ static void kill_off_processes(void)
 	read_unlock(&tasklist_lock);
 }
 
+void __weak os_exitcalls(void)
+{
+}
+
 void uml_cleanup(void)
 {
 	kmalloc_ok = 0;
 	do_uml_exitcalls();
+	os_exitcalls();
 	kill_off_processes();
 }
 
diff --git a/arch/um/kernel/um_arch.c b/arch/um/kernel/um_arch.c
index 0f40eccbd759..78d6042fd2e6 100644
--- a/arch/um/kernel/um_arch.c
+++ b/arch/um/kernel/um_arch.c
@@ -362,3 +362,14 @@ void __init check_bugs(void)
 void apply_alternatives(struct alt_instr *start, struct alt_instr *end)
 {
 }
+
+int __weak os_initcalls(void)
+{
+	return 0;
+}
+
+int __init run_os_initcalls(void)
+{
+	return os_initcalls();
+}
+__initcall(run_os_initcalls);
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v5 03/21] um: move arch/um/os-Linux dir to tools/um/uml
  2020-07-02 14:06     ` Hajime Tazaki
@ 2020-07-02 14:06       ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-07-02 14:06 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Hajime Tazaki

This patch moves underlying OS dependent part under arch/um to tools/um
directory so that arch/um code does not need to build host build
facilities (e.g., libc).

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/Makefile                              |  3 +-
 arch/um/drivers/Makefile                      |  2 +
 .../um/{os-Linux => }/drivers/ethertap_kern.c |  0
 arch/um/{os-Linux => }/drivers/tuntap_kern.c  |  0
 .../drivers => include/shared}/etap.h         |  0
 arch/um/include/shared/init.h                 |  4 +-
 .../drivers => include/shared}/tuntap.h       |  0
 arch/um/os-Linux/Makefile                     | 19 --------
 arch/um/os-Linux/drivers/Makefile             | 13 -----
 tools/um/Makefile                             |  6 ++-
 tools/um/uml/Build                            | 48 +++++++++++++++++++
 tools/um/uml/drivers/Build                    | 10 ++++
 .../um/uml}/drivers/ethertap_user.c           |  0
 .../um/uml}/drivers/tuntap_user.c             |  0
 {arch/um/os-Linux => tools/um/uml}/elf_aux.c  |  0
 {arch/um/os-Linux => tools/um/uml}/execvp.c   |  4 --
 {arch/um/os-Linux => tools/um/uml}/file.c     |  0
 {arch/um/os-Linux => tools/um/uml}/helper.c   |  0
 {arch/um/os-Linux => tools/um/uml}/irq.c      |  0
 {arch/um/os-Linux => tools/um/uml}/main.c     |  0
 {arch/um/os-Linux => tools/um/uml}/mem.c      |  0
 {arch/um/os-Linux => tools/um/uml}/process.c  |  0
 .../um/os-Linux => tools/um/uml}/registers.c  |  0
 {arch/um/os-Linux => tools/um/uml}/sigio.c    |  0
 {arch/um/os-Linux => tools/um/uml}/signal.c   |  0
 .../skas/Makefile => tools/um/uml/skas/Build  |  6 +--
 {arch/um/os-Linux => tools/um/uml}/skas/mem.c |  0
 .../os-Linux => tools/um/uml}/skas/process.c  |  3 +-
 {arch/um/os-Linux => tools/um/uml}/start_up.c |  0
 {arch/um/os-Linux => tools/um/uml}/time.c     |  0
 {arch/um/os-Linux => tools/um/uml}/tty.c      |  0
 {arch/um/os-Linux => tools/um/uml}/umid.c     |  0
 .../um/os-Linux => tools/um/uml}/user_syms.c  |  1 +
 {arch/um/os-Linux => tools/um/uml}/util.c     |  0
 34 files changed, 72 insertions(+), 47 deletions(-)
 rename arch/um/{os-Linux => }/drivers/ethertap_kern.c (100%)
 rename arch/um/{os-Linux => }/drivers/tuntap_kern.c (100%)
 rename arch/um/{os-Linux/drivers => include/shared}/etap.h (100%)
 rename arch/um/{os-Linux/drivers => include/shared}/tuntap.h (100%)
 delete mode 100644 arch/um/os-Linux/Makefile
 delete mode 100644 arch/um/os-Linux/drivers/Makefile
 create mode 100644 tools/um/uml/drivers/Build
 rename {arch/um/os-Linux => tools/um/uml}/drivers/ethertap_user.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/drivers/tuntap_user.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/elf_aux.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/execvp.c (98%)
 rename {arch/um/os-Linux => tools/um/uml}/file.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/helper.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/irq.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/main.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/mem.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/process.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/registers.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/sigio.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/signal.c (100%)
 rename arch/um/os-Linux/skas/Makefile => tools/um/uml/skas/Build (56%)
 rename {arch/um/os-Linux => tools/um/uml}/skas/mem.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/skas/process.c (99%)
 rename {arch/um/os-Linux => tools/um/uml}/start_up.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/time.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/tty.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/umid.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/user_syms.c (99%)
 rename {arch/um/os-Linux => tools/um/uml}/util.c (100%)

diff --git a/arch/um/Makefile b/arch/um/Makefile
index c952ddefcc1c..af0cf64c1428 100644
--- a/arch/um/Makefile
+++ b/arch/um/Makefile
@@ -24,8 +24,7 @@ OS := $(shell uname -s)
 SHELL := /bin/bash
 
 core-y			+= $(ARCH_DIR)/kernel/		\
-			   $(ARCH_DIR)/drivers/		\
-			   $(ARCH_DIR)/os-$(OS)/
+			   $(ARCH_DIR)/drivers/
 
 MODE_INCLUDE	+= -I$(srctree)/$(ARCH_DIR)/include/shared/skas
 
diff --git a/arch/um/drivers/Makefile b/arch/um/drivers/Makefile
index 2a249f619467..ae96c83e312d 100644
--- a/arch/um/drivers/Makefile
+++ b/arch/um/drivers/Makefile
@@ -48,6 +48,8 @@ obj-$(CONFIG_UML_NET_VECTOR) += vector.o
 obj-$(CONFIG_UML_NET_VDE) += vde.o
 obj-$(CONFIG_UML_NET_MCAST) += umcast.o
 obj-$(CONFIG_UML_NET_PCAP) += pcap.o
+obj-$(CONFIG_UML_NET_ETHERTAP) += ethertap_kern.o
+obj-$(CONFIG_UML_NET_TUNTAP) += tuntap_kern.o
 obj-$(CONFIG_UML_NET) += net.o 
 obj-$(CONFIG_MCONSOLE) += mconsole.o
 obj-$(CONFIG_MMAPPER) += mmapper_kern.o 
diff --git a/arch/um/os-Linux/drivers/ethertap_kern.c b/arch/um/drivers/ethertap_kern.c
similarity index 100%
rename from arch/um/os-Linux/drivers/ethertap_kern.c
rename to arch/um/drivers/ethertap_kern.c
diff --git a/arch/um/os-Linux/drivers/tuntap_kern.c b/arch/um/drivers/tuntap_kern.c
similarity index 100%
rename from arch/um/os-Linux/drivers/tuntap_kern.c
rename to arch/um/drivers/tuntap_kern.c
diff --git a/arch/um/os-Linux/drivers/etap.h b/arch/um/include/shared/etap.h
similarity index 100%
rename from arch/um/os-Linux/drivers/etap.h
rename to arch/um/include/shared/etap.h
diff --git a/arch/um/include/shared/init.h b/arch/um/include/shared/init.h
index d09308330ca5..15b52328e7aa 100644
--- a/arch/um/include/shared/init.h
+++ b/arch/um/include/shared/init.h
@@ -41,7 +41,9 @@
 typedef int (*initcall_t)(void);
 typedef void (*exitcall_t)(void);
 
-#include <linux/compiler_types.h>
+//#include <linux/compiler_types.h>
+#define __used                          __attribute__((__used__))
+#define __section(S)                    __attribute__((__section__(#S)))
 
 /* These are for everybody (although not all archs will actually
    discard it in modules) */
diff --git a/arch/um/os-Linux/drivers/tuntap.h b/arch/um/include/shared/tuntap.h
similarity index 100%
rename from arch/um/os-Linux/drivers/tuntap.h
rename to arch/um/include/shared/tuntap.h
diff --git a/arch/um/os-Linux/Makefile b/arch/um/os-Linux/Makefile
deleted file mode 100644
index 839915b8c31c..000000000000
--- a/arch/um/os-Linux/Makefile
+++ /dev/null
@@ -1,19 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0
-# 
-# Copyright (C) 2000 - 2007 Jeff Dike (jdike@{addtoit,linux.intel}.com)
-#
-
-# Don't instrument UML-specific code
-KCOV_INSTRUMENT                := n
-
-obj-y = execvp.o file.o helper.o irq.o main.o mem.o process.o \
-	registers.o sigio.o signal.o start_up.o time.o tty.o \
-	umid.o user_syms.o util.o drivers/ skas/
-
-obj-$(CONFIG_ARCH_REUSE_HOST_VSYSCALL_AREA) += elf_aux.o
-
-USER_OBJS := $(user-objs-y) elf_aux.o execvp.o file.o helper.o irq.o \
-	main.o mem.o process.o registers.o sigio.o signal.o start_up.o time.o \
-	tty.o umid.o util.o
-
-include arch/um/scripts/Makefile.rules
diff --git a/arch/um/os-Linux/drivers/Makefile b/arch/um/os-Linux/drivers/Makefile
deleted file mode 100644
index d79e75f1b69a..000000000000
--- a/arch/um/os-Linux/drivers/Makefile
+++ /dev/null
@@ -1,13 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0
-# 
-# Copyright (C) 2000, 2002 Jeff Dike (jdike@karaya.com)
-#
-
-ethertap-objs := ethertap_kern.o ethertap_user.o
-tuntap-objs := tuntap_kern.o tuntap_user.o
-
-obj-y = 
-obj-$(CONFIG_UML_NET_ETHERTAP) += ethertap.o
-obj-$(CONFIG_UML_NET_TUNTAP) += tuntap.o
-
-include arch/um/scripts/Makefile.rules
diff --git a/tools/um/Makefile b/tools/um/Makefile
index a63466ef7ef5..07b9b93ed817 100644
--- a/tools/um/Makefile
+++ b/tools/um/Makefile
@@ -50,7 +50,7 @@ $(OUTPUT)lib/linux.o:
 	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) $(KCONFIG)
 	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) install INSTALL_PATH=$(OUTPUT)
 
-$(OUTPUT)liblinux.a: $(OUTPUT)lib/linux.o $(OUTPUT)uml/liblinux-in.o
+$(OUTPUT)liblinux.a: $(OUTPUT)lib/linux.o $(OUTPUT)uml/liblinux-in.o $(OUTPUT)lib/liblinux-in.o
 	$(QUIET_AR)$(AR) -rc $@ $^
 
 # rule to link programs
@@ -60,6 +60,10 @@ $(OUTPUT)%: $(OUTPUT)%-in.o $(OUTPUT)liblinux.a
 $(OUTPUT)%-in.o: FORCE
 	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
 
+# rule to build objects
+$(OUTPUT)%-in.o: $(OUTPUT)lib/linux.o
+	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
+
 clean:
 	$(call QUIET_CLEAN, objects)find $(OUTPUT) -name '*.o' -delete -o -name '\.*.cmd'\
 	 -delete -o -name '\.*.d' -delete
diff --git a/tools/um/uml/Build b/tools/um/uml/Build
index e69de29bb2d1..8e0607ee8b0a 100644
--- a/tools/um/uml/Build
+++ b/tools/um/uml/Build
@@ -0,0 +1,48 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Copyright (C) 2000 - 2007 Jeff Dike (jdike@{addtoit,linux.intel}.com)
+#
+
+# Don't instrument UML-specific code
+KCOV_INSTRUMENT                := n
+
+include $(objtree)/include/config/auto.conf
+
+liblinux-y += execvp.o file.o helper.o irq.o main.o mem.o process.o \
+	registers.o sigio.o signal.o start_up.o time.o tty.o \
+	umid.o user_syms.o util.o drivers/ skas/
+
+liblinux-$(CONFIG_ARCH_REUSE_HOST_VSYSCALL_AREA) += elf_aux.o
+
+export O = $(srctree)
+objtree := $(O)
+# reset CFLAGS
+CFLAGS := -g -O2
+
+# from arch/um/Makefile
+ARCH_DIR := arch/um
+HEADER_ARCH 	:= x86
+HOST_DIR := arch/$(HEADER_ARCH)
+ifdef CONFIG_64BIT
+  KBUILD_CFLAGS += -mcmodel=large
+endif
+KBUILD_CFLAGS += $(CFLAGS) $(CFLAGS-y) -D__arch_um__ \
+	$(ARCH_INCLUDE) $(MODE_INCLUDE) -Dvmap=kernel_vmap	\
+	-Dlongjmp=kernel_longjmp -Dsetjmp=kernel_setjmp \
+	-Din6addr_loopback=kernel_in6addr_loopback \
+	-Din6addr_any=kernel_in6addr_any -Dstrrchr=kernel_strrchr
+SHARED_HEADERS	:= $(ARCH_DIR)/include/shared
+MODE_INCLUDE	+= -I$(srctree)/$(ARCH_DIR)/include/shared/skas
+ARCH_INCLUDE	:= -I$(srctree)/$(SHARED_HEADERS)
+ARCH_INCLUDE	+= -I$(srctree)/$(HOST_DIR)/um/shared
+KBUILD_CPPFLAGS += -I$(srctree)/$(HOST_DIR)/um
+USER_CFLAGS := $(patsubst $(KERNEL_DEFINES),,$(patsubst -I%,,$(KBUILD_CFLAGS))) \
+		$(ARCH_INCLUDE) $(MODE_INCLUDE) $(filter -I%,$(CFLAGS)) \
+		-D_FILE_OFFSET_BITS=64 -idirafter $(srctree)/include/uapi \
+		-idirafter $(objtree)/include -D__KERNEL__ -D__UM_HOST__
+
+# from Makefile-os-Linux
+USER_CFLAGS += -D_GNU_SOURCE -D_LARGEFILE64_SOURCE
+
+# from Makefile.rules
+CFLAGS += $(USER_CFLAGS) -include $(srctree)/tools/include/linux/kern_levels.h -include user.h $(CFLAGS_$(basetarget).o)
diff --git a/tools/um/uml/drivers/Build b/tools/um/uml/drivers/Build
new file mode 100644
index 000000000000..c7a8eaa97d72
--- /dev/null
+++ b/tools/um/uml/drivers/Build
@@ -0,0 +1,10 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Copyright (C) 2000, 2002 Jeff Dike (jdike@karaya.com)
+#
+
+include $(objtree)/include/config/auto.conf
+
+
+liblinux-$(CONFIG_UML_NET_ETHERTAP) += ethertap_user.o
+liblinux-$(CONFIG_UML_NET_TUNTAP) += tuntap_user.o
diff --git a/arch/um/os-Linux/drivers/ethertap_user.c b/tools/um/uml/drivers/ethertap_user.c
similarity index 100%
rename from arch/um/os-Linux/drivers/ethertap_user.c
rename to tools/um/uml/drivers/ethertap_user.c
diff --git a/arch/um/os-Linux/drivers/tuntap_user.c b/tools/um/uml/drivers/tuntap_user.c
similarity index 100%
rename from arch/um/os-Linux/drivers/tuntap_user.c
rename to tools/um/uml/drivers/tuntap_user.c
diff --git a/arch/um/os-Linux/elf_aux.c b/tools/um/uml/elf_aux.c
similarity index 100%
rename from arch/um/os-Linux/elf_aux.c
rename to tools/um/uml/elf_aux.c
diff --git a/arch/um/os-Linux/execvp.c b/tools/um/uml/execvp.c
similarity index 98%
rename from arch/um/os-Linux/execvp.c
rename to tools/um/uml/execvp.c
index 84a0777c2a45..c105e0e7b778 100644
--- a/arch/um/os-Linux/execvp.c
+++ b/tools/um/uml/execvp.c
@@ -26,12 +26,8 @@
 #include <errno.h>
 #include <limits.h>
 
-#ifndef TEST
-#include <um_malloc.h>
-#else
 #include <stdio.h>
 #define um_kmalloc malloc
-#endif
 #include <os.h>
 
 /* Execute FILE, searching in the `PATH' environment variable if it contains
diff --git a/arch/um/os-Linux/file.c b/tools/um/uml/file.c
similarity index 100%
rename from arch/um/os-Linux/file.c
rename to tools/um/uml/file.c
diff --git a/arch/um/os-Linux/helper.c b/tools/um/uml/helper.c
similarity index 100%
rename from arch/um/os-Linux/helper.c
rename to tools/um/uml/helper.c
diff --git a/arch/um/os-Linux/irq.c b/tools/um/uml/irq.c
similarity index 100%
rename from arch/um/os-Linux/irq.c
rename to tools/um/uml/irq.c
diff --git a/arch/um/os-Linux/main.c b/tools/um/uml/main.c
similarity index 100%
rename from arch/um/os-Linux/main.c
rename to tools/um/uml/main.c
diff --git a/arch/um/os-Linux/mem.c b/tools/um/uml/mem.c
similarity index 100%
rename from arch/um/os-Linux/mem.c
rename to tools/um/uml/mem.c
diff --git a/arch/um/os-Linux/process.c b/tools/um/uml/process.c
similarity index 100%
rename from arch/um/os-Linux/process.c
rename to tools/um/uml/process.c
diff --git a/arch/um/os-Linux/registers.c b/tools/um/uml/registers.c
similarity index 100%
rename from arch/um/os-Linux/registers.c
rename to tools/um/uml/registers.c
diff --git a/arch/um/os-Linux/sigio.c b/tools/um/uml/sigio.c
similarity index 100%
rename from arch/um/os-Linux/sigio.c
rename to tools/um/uml/sigio.c
diff --git a/arch/um/os-Linux/signal.c b/tools/um/uml/signal.c
similarity index 100%
rename from arch/um/os-Linux/signal.c
rename to tools/um/uml/signal.c
diff --git a/arch/um/os-Linux/skas/Makefile b/tools/um/uml/skas/Build
similarity index 56%
rename from arch/um/os-Linux/skas/Makefile
rename to tools/um/uml/skas/Build
index c4566e788815..5fc9d62df863 100644
--- a/arch/um/os-Linux/skas/Makefile
+++ b/tools/um/uml/skas/Build
@@ -3,8 +3,4 @@
 # Copyright (C) 2002 - 2007 Jeff Dike (jdike@{linux.intel,addtoit}.com)
 #
 
-obj-y := mem.o process.o
-
-USER_OBJS := $(obj-y)
-
-include arch/um/scripts/Makefile.rules
+liblinux-y += mem.o process.o
diff --git a/arch/um/os-Linux/skas/mem.c b/tools/um/uml/skas/mem.c
similarity index 100%
rename from arch/um/os-Linux/skas/mem.c
rename to tools/um/uml/skas/mem.c
diff --git a/arch/um/os-Linux/skas/process.c b/tools/um/uml/skas/process.c
similarity index 99%
rename from arch/um/os-Linux/skas/process.c
rename to tools/um/uml/skas/process.c
index 4fb877b99dde..fca6a08e81bd 100644
--- a/arch/um/os-Linux/skas/process.c
+++ b/tools/um/uml/skas/process.c
@@ -21,7 +21,6 @@
 #include <registers.h>
 #include <skas.h>
 #include <sysdep/stub.h>
-#include <linux/threads.h>
 
 int is_skas_winch(int pid, int fd, void *data)
 {
@@ -248,7 +247,7 @@ static int userspace_tramp(void *stack)
 	return 0;
 }
 
-int userspace_pid[NR_CPUS];
+int userspace_pid[UM_NR_CPUS];
 
 /**
  * start_userspace() - prepare a new userspace process
diff --git a/arch/um/os-Linux/start_up.c b/tools/um/uml/start_up.c
similarity index 100%
rename from arch/um/os-Linux/start_up.c
rename to tools/um/uml/start_up.c
diff --git a/arch/um/os-Linux/time.c b/tools/um/uml/time.c
similarity index 100%
rename from arch/um/os-Linux/time.c
rename to tools/um/uml/time.c
diff --git a/arch/um/os-Linux/tty.c b/tools/um/uml/tty.c
similarity index 100%
rename from arch/um/os-Linux/tty.c
rename to tools/um/uml/tty.c
diff --git a/arch/um/os-Linux/umid.c b/tools/um/uml/umid.c
similarity index 100%
rename from arch/um/os-Linux/umid.c
rename to tools/um/uml/umid.c
diff --git a/arch/um/os-Linux/user_syms.c b/tools/um/uml/user_syms.c
similarity index 99%
rename from arch/um/os-Linux/user_syms.c
rename to tools/um/uml/user_syms.c
index 715594fe5719..4047fd7bfdb7 100644
--- a/arch/um/os-Linux/user_syms.c
+++ b/tools/um/uml/user_syms.c
@@ -13,6 +13,7 @@
 #undef strstr
 #undef memcpy
 #undef memset
+#define EXPORT_SYMBOL(x)
 
 extern size_t strlen(const char *);
 extern void *memmove(void *, const void *, size_t);
diff --git a/arch/um/os-Linux/util.c b/tools/um/uml/util.c
similarity index 100%
rename from arch/um/os-Linux/util.c
rename to tools/um/uml/util.c
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v5 03/21] um: move arch/um/os-Linux dir to tools/um/uml
@ 2020-07-02 14:06       ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-07-02 14:06 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, linux-kernel-library, linux-arch,
	Hajime Tazaki, Akira Moroo

This patch moves underlying OS dependent part under arch/um to tools/um
directory so that arch/um code does not need to build host build
facilities (e.g., libc).

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/Makefile                              |  3 +-
 arch/um/drivers/Makefile                      |  2 +
 .../um/{os-Linux => }/drivers/ethertap_kern.c |  0
 arch/um/{os-Linux => }/drivers/tuntap_kern.c  |  0
 .../drivers => include/shared}/etap.h         |  0
 arch/um/include/shared/init.h                 |  4 +-
 .../drivers => include/shared}/tuntap.h       |  0
 arch/um/os-Linux/Makefile                     | 19 --------
 arch/um/os-Linux/drivers/Makefile             | 13 -----
 tools/um/Makefile                             |  6 ++-
 tools/um/uml/Build                            | 48 +++++++++++++++++++
 tools/um/uml/drivers/Build                    | 10 ++++
 .../um/uml}/drivers/ethertap_user.c           |  0
 .../um/uml}/drivers/tuntap_user.c             |  0
 {arch/um/os-Linux => tools/um/uml}/elf_aux.c  |  0
 {arch/um/os-Linux => tools/um/uml}/execvp.c   |  4 --
 {arch/um/os-Linux => tools/um/uml}/file.c     |  0
 {arch/um/os-Linux => tools/um/uml}/helper.c   |  0
 {arch/um/os-Linux => tools/um/uml}/irq.c      |  0
 {arch/um/os-Linux => tools/um/uml}/main.c     |  0
 {arch/um/os-Linux => tools/um/uml}/mem.c      |  0
 {arch/um/os-Linux => tools/um/uml}/process.c  |  0
 .../um/os-Linux => tools/um/uml}/registers.c  |  0
 {arch/um/os-Linux => tools/um/uml}/sigio.c    |  0
 {arch/um/os-Linux => tools/um/uml}/signal.c   |  0
 .../skas/Makefile => tools/um/uml/skas/Build  |  6 +--
 {arch/um/os-Linux => tools/um/uml}/skas/mem.c |  0
 .../os-Linux => tools/um/uml}/skas/process.c  |  3 +-
 {arch/um/os-Linux => tools/um/uml}/start_up.c |  0
 {arch/um/os-Linux => tools/um/uml}/time.c     |  0
 {arch/um/os-Linux => tools/um/uml}/tty.c      |  0
 {arch/um/os-Linux => tools/um/uml}/umid.c     |  0
 .../um/os-Linux => tools/um/uml}/user_syms.c  |  1 +
 {arch/um/os-Linux => tools/um/uml}/util.c     |  0
 34 files changed, 72 insertions(+), 47 deletions(-)
 rename arch/um/{os-Linux => }/drivers/ethertap_kern.c (100%)
 rename arch/um/{os-Linux => }/drivers/tuntap_kern.c (100%)
 rename arch/um/{os-Linux/drivers => include/shared}/etap.h (100%)
 rename arch/um/{os-Linux/drivers => include/shared}/tuntap.h (100%)
 delete mode 100644 arch/um/os-Linux/Makefile
 delete mode 100644 arch/um/os-Linux/drivers/Makefile
 create mode 100644 tools/um/uml/drivers/Build
 rename {arch/um/os-Linux => tools/um/uml}/drivers/ethertap_user.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/drivers/tuntap_user.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/elf_aux.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/execvp.c (98%)
 rename {arch/um/os-Linux => tools/um/uml}/file.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/helper.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/irq.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/main.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/mem.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/process.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/registers.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/sigio.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/signal.c (100%)
 rename arch/um/os-Linux/skas/Makefile => tools/um/uml/skas/Build (56%)
 rename {arch/um/os-Linux => tools/um/uml}/skas/mem.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/skas/process.c (99%)
 rename {arch/um/os-Linux => tools/um/uml}/start_up.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/time.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/tty.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/umid.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/user_syms.c (99%)
 rename {arch/um/os-Linux => tools/um/uml}/util.c (100%)

diff --git a/arch/um/Makefile b/arch/um/Makefile
index c952ddefcc1c..af0cf64c1428 100644
--- a/arch/um/Makefile
+++ b/arch/um/Makefile
@@ -24,8 +24,7 @@ OS := $(shell uname -s)
 SHELL := /bin/bash
 
 core-y			+= $(ARCH_DIR)/kernel/		\
-			   $(ARCH_DIR)/drivers/		\
-			   $(ARCH_DIR)/os-$(OS)/
+			   $(ARCH_DIR)/drivers/
 
 MODE_INCLUDE	+= -I$(srctree)/$(ARCH_DIR)/include/shared/skas
 
diff --git a/arch/um/drivers/Makefile b/arch/um/drivers/Makefile
index 2a249f619467..ae96c83e312d 100644
--- a/arch/um/drivers/Makefile
+++ b/arch/um/drivers/Makefile
@@ -48,6 +48,8 @@ obj-$(CONFIG_UML_NET_VECTOR) += vector.o
 obj-$(CONFIG_UML_NET_VDE) += vde.o
 obj-$(CONFIG_UML_NET_MCAST) += umcast.o
 obj-$(CONFIG_UML_NET_PCAP) += pcap.o
+obj-$(CONFIG_UML_NET_ETHERTAP) += ethertap_kern.o
+obj-$(CONFIG_UML_NET_TUNTAP) += tuntap_kern.o
 obj-$(CONFIG_UML_NET) += net.o 
 obj-$(CONFIG_MCONSOLE) += mconsole.o
 obj-$(CONFIG_MMAPPER) += mmapper_kern.o 
diff --git a/arch/um/os-Linux/drivers/ethertap_kern.c b/arch/um/drivers/ethertap_kern.c
similarity index 100%
rename from arch/um/os-Linux/drivers/ethertap_kern.c
rename to arch/um/drivers/ethertap_kern.c
diff --git a/arch/um/os-Linux/drivers/tuntap_kern.c b/arch/um/drivers/tuntap_kern.c
similarity index 100%
rename from arch/um/os-Linux/drivers/tuntap_kern.c
rename to arch/um/drivers/tuntap_kern.c
diff --git a/arch/um/os-Linux/drivers/etap.h b/arch/um/include/shared/etap.h
similarity index 100%
rename from arch/um/os-Linux/drivers/etap.h
rename to arch/um/include/shared/etap.h
diff --git a/arch/um/include/shared/init.h b/arch/um/include/shared/init.h
index d09308330ca5..15b52328e7aa 100644
--- a/arch/um/include/shared/init.h
+++ b/arch/um/include/shared/init.h
@@ -41,7 +41,9 @@
 typedef int (*initcall_t)(void);
 typedef void (*exitcall_t)(void);
 
-#include <linux/compiler_types.h>
+//#include <linux/compiler_types.h>
+#define __used                          __attribute__((__used__))
+#define __section(S)                    __attribute__((__section__(#S)))
 
 /* These are for everybody (although not all archs will actually
    discard it in modules) */
diff --git a/arch/um/os-Linux/drivers/tuntap.h b/arch/um/include/shared/tuntap.h
similarity index 100%
rename from arch/um/os-Linux/drivers/tuntap.h
rename to arch/um/include/shared/tuntap.h
diff --git a/arch/um/os-Linux/Makefile b/arch/um/os-Linux/Makefile
deleted file mode 100644
index 839915b8c31c..000000000000
--- a/arch/um/os-Linux/Makefile
+++ /dev/null
@@ -1,19 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0
-# 
-# Copyright (C) 2000 - 2007 Jeff Dike (jdike@{addtoit,linux.intel}.com)
-#
-
-# Don't instrument UML-specific code
-KCOV_INSTRUMENT                := n
-
-obj-y = execvp.o file.o helper.o irq.o main.o mem.o process.o \
-	registers.o sigio.o signal.o start_up.o time.o tty.o \
-	umid.o user_syms.o util.o drivers/ skas/
-
-obj-$(CONFIG_ARCH_REUSE_HOST_VSYSCALL_AREA) += elf_aux.o
-
-USER_OBJS := $(user-objs-y) elf_aux.o execvp.o file.o helper.o irq.o \
-	main.o mem.o process.o registers.o sigio.o signal.o start_up.o time.o \
-	tty.o umid.o util.o
-
-include arch/um/scripts/Makefile.rules
diff --git a/arch/um/os-Linux/drivers/Makefile b/arch/um/os-Linux/drivers/Makefile
deleted file mode 100644
index d79e75f1b69a..000000000000
--- a/arch/um/os-Linux/drivers/Makefile
+++ /dev/null
@@ -1,13 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0
-# 
-# Copyright (C) 2000, 2002 Jeff Dike (jdike@karaya.com)
-#
-
-ethertap-objs := ethertap_kern.o ethertap_user.o
-tuntap-objs := tuntap_kern.o tuntap_user.o
-
-obj-y = 
-obj-$(CONFIG_UML_NET_ETHERTAP) += ethertap.o
-obj-$(CONFIG_UML_NET_TUNTAP) += tuntap.o
-
-include arch/um/scripts/Makefile.rules
diff --git a/tools/um/Makefile b/tools/um/Makefile
index a63466ef7ef5..07b9b93ed817 100644
--- a/tools/um/Makefile
+++ b/tools/um/Makefile
@@ -50,7 +50,7 @@ $(OUTPUT)lib/linux.o:
 	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) $(KCONFIG)
 	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) install INSTALL_PATH=$(OUTPUT)
 
-$(OUTPUT)liblinux.a: $(OUTPUT)lib/linux.o $(OUTPUT)uml/liblinux-in.o
+$(OUTPUT)liblinux.a: $(OUTPUT)lib/linux.o $(OUTPUT)uml/liblinux-in.o $(OUTPUT)lib/liblinux-in.o
 	$(QUIET_AR)$(AR) -rc $@ $^
 
 # rule to link programs
@@ -60,6 +60,10 @@ $(OUTPUT)%: $(OUTPUT)%-in.o $(OUTPUT)liblinux.a
 $(OUTPUT)%-in.o: FORCE
 	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
 
+# rule to build objects
+$(OUTPUT)%-in.o: $(OUTPUT)lib/linux.o
+	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
+
 clean:
 	$(call QUIET_CLEAN, objects)find $(OUTPUT) -name '*.o' -delete -o -name '\.*.cmd'\
 	 -delete -o -name '\.*.d' -delete
diff --git a/tools/um/uml/Build b/tools/um/uml/Build
index e69de29bb2d1..8e0607ee8b0a 100644
--- a/tools/um/uml/Build
+++ b/tools/um/uml/Build
@@ -0,0 +1,48 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Copyright (C) 2000 - 2007 Jeff Dike (jdike@{addtoit,linux.intel}.com)
+#
+
+# Don't instrument UML-specific code
+KCOV_INSTRUMENT                := n
+
+include $(objtree)/include/config/auto.conf
+
+liblinux-y += execvp.o file.o helper.o irq.o main.o mem.o process.o \
+	registers.o sigio.o signal.o start_up.o time.o tty.o \
+	umid.o user_syms.o util.o drivers/ skas/
+
+liblinux-$(CONFIG_ARCH_REUSE_HOST_VSYSCALL_AREA) += elf_aux.o
+
+export O = $(srctree)
+objtree := $(O)
+# reset CFLAGS
+CFLAGS := -g -O2
+
+# from arch/um/Makefile
+ARCH_DIR := arch/um
+HEADER_ARCH 	:= x86
+HOST_DIR := arch/$(HEADER_ARCH)
+ifdef CONFIG_64BIT
+  KBUILD_CFLAGS += -mcmodel=large
+endif
+KBUILD_CFLAGS += $(CFLAGS) $(CFLAGS-y) -D__arch_um__ \
+	$(ARCH_INCLUDE) $(MODE_INCLUDE) -Dvmap=kernel_vmap	\
+	-Dlongjmp=kernel_longjmp -Dsetjmp=kernel_setjmp \
+	-Din6addr_loopback=kernel_in6addr_loopback \
+	-Din6addr_any=kernel_in6addr_any -Dstrrchr=kernel_strrchr
+SHARED_HEADERS	:= $(ARCH_DIR)/include/shared
+MODE_INCLUDE	+= -I$(srctree)/$(ARCH_DIR)/include/shared/skas
+ARCH_INCLUDE	:= -I$(srctree)/$(SHARED_HEADERS)
+ARCH_INCLUDE	+= -I$(srctree)/$(HOST_DIR)/um/shared
+KBUILD_CPPFLAGS += -I$(srctree)/$(HOST_DIR)/um
+USER_CFLAGS := $(patsubst $(KERNEL_DEFINES),,$(patsubst -I%,,$(KBUILD_CFLAGS))) \
+		$(ARCH_INCLUDE) $(MODE_INCLUDE) $(filter -I%,$(CFLAGS)) \
+		-D_FILE_OFFSET_BITS=64 -idirafter $(srctree)/include/uapi \
+		-idirafter $(objtree)/include -D__KERNEL__ -D__UM_HOST__
+
+# from Makefile-os-Linux
+USER_CFLAGS += -D_GNU_SOURCE -D_LARGEFILE64_SOURCE
+
+# from Makefile.rules
+CFLAGS += $(USER_CFLAGS) -include $(srctree)/tools/include/linux/kern_levels.h -include user.h $(CFLAGS_$(basetarget).o)
diff --git a/tools/um/uml/drivers/Build b/tools/um/uml/drivers/Build
new file mode 100644
index 000000000000..c7a8eaa97d72
--- /dev/null
+++ b/tools/um/uml/drivers/Build
@@ -0,0 +1,10 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Copyright (C) 2000, 2002 Jeff Dike (jdike@karaya.com)
+#
+
+include $(objtree)/include/config/auto.conf
+
+
+liblinux-$(CONFIG_UML_NET_ETHERTAP) += ethertap_user.o
+liblinux-$(CONFIG_UML_NET_TUNTAP) += tuntap_user.o
diff --git a/arch/um/os-Linux/drivers/ethertap_user.c b/tools/um/uml/drivers/ethertap_user.c
similarity index 100%
rename from arch/um/os-Linux/drivers/ethertap_user.c
rename to tools/um/uml/drivers/ethertap_user.c
diff --git a/arch/um/os-Linux/drivers/tuntap_user.c b/tools/um/uml/drivers/tuntap_user.c
similarity index 100%
rename from arch/um/os-Linux/drivers/tuntap_user.c
rename to tools/um/uml/drivers/tuntap_user.c
diff --git a/arch/um/os-Linux/elf_aux.c b/tools/um/uml/elf_aux.c
similarity index 100%
rename from arch/um/os-Linux/elf_aux.c
rename to tools/um/uml/elf_aux.c
diff --git a/arch/um/os-Linux/execvp.c b/tools/um/uml/execvp.c
similarity index 98%
rename from arch/um/os-Linux/execvp.c
rename to tools/um/uml/execvp.c
index 84a0777c2a45..c105e0e7b778 100644
--- a/arch/um/os-Linux/execvp.c
+++ b/tools/um/uml/execvp.c
@@ -26,12 +26,8 @@
 #include <errno.h>
 #include <limits.h>
 
-#ifndef TEST
-#include <um_malloc.h>
-#else
 #include <stdio.h>
 #define um_kmalloc malloc
-#endif
 #include <os.h>
 
 /* Execute FILE, searching in the `PATH' environment variable if it contains
diff --git a/arch/um/os-Linux/file.c b/tools/um/uml/file.c
similarity index 100%
rename from arch/um/os-Linux/file.c
rename to tools/um/uml/file.c
diff --git a/arch/um/os-Linux/helper.c b/tools/um/uml/helper.c
similarity index 100%
rename from arch/um/os-Linux/helper.c
rename to tools/um/uml/helper.c
diff --git a/arch/um/os-Linux/irq.c b/tools/um/uml/irq.c
similarity index 100%
rename from arch/um/os-Linux/irq.c
rename to tools/um/uml/irq.c
diff --git a/arch/um/os-Linux/main.c b/tools/um/uml/main.c
similarity index 100%
rename from arch/um/os-Linux/main.c
rename to tools/um/uml/main.c
diff --git a/arch/um/os-Linux/mem.c b/tools/um/uml/mem.c
similarity index 100%
rename from arch/um/os-Linux/mem.c
rename to tools/um/uml/mem.c
diff --git a/arch/um/os-Linux/process.c b/tools/um/uml/process.c
similarity index 100%
rename from arch/um/os-Linux/process.c
rename to tools/um/uml/process.c
diff --git a/arch/um/os-Linux/registers.c b/tools/um/uml/registers.c
similarity index 100%
rename from arch/um/os-Linux/registers.c
rename to tools/um/uml/registers.c
diff --git a/arch/um/os-Linux/sigio.c b/tools/um/uml/sigio.c
similarity index 100%
rename from arch/um/os-Linux/sigio.c
rename to tools/um/uml/sigio.c
diff --git a/arch/um/os-Linux/signal.c b/tools/um/uml/signal.c
similarity index 100%
rename from arch/um/os-Linux/signal.c
rename to tools/um/uml/signal.c
diff --git a/arch/um/os-Linux/skas/Makefile b/tools/um/uml/skas/Build
similarity index 56%
rename from arch/um/os-Linux/skas/Makefile
rename to tools/um/uml/skas/Build
index c4566e788815..5fc9d62df863 100644
--- a/arch/um/os-Linux/skas/Makefile
+++ b/tools/um/uml/skas/Build
@@ -3,8 +3,4 @@
 # Copyright (C) 2002 - 2007 Jeff Dike (jdike@{linux.intel,addtoit}.com)
 #
 
-obj-y := mem.o process.o
-
-USER_OBJS := $(obj-y)
-
-include arch/um/scripts/Makefile.rules
+liblinux-y += mem.o process.o
diff --git a/arch/um/os-Linux/skas/mem.c b/tools/um/uml/skas/mem.c
similarity index 100%
rename from arch/um/os-Linux/skas/mem.c
rename to tools/um/uml/skas/mem.c
diff --git a/arch/um/os-Linux/skas/process.c b/tools/um/uml/skas/process.c
similarity index 99%
rename from arch/um/os-Linux/skas/process.c
rename to tools/um/uml/skas/process.c
index 4fb877b99dde..fca6a08e81bd 100644
--- a/arch/um/os-Linux/skas/process.c
+++ b/tools/um/uml/skas/process.c
@@ -21,7 +21,6 @@
 #include <registers.h>
 #include <skas.h>
 #include <sysdep/stub.h>
-#include <linux/threads.h>
 
 int is_skas_winch(int pid, int fd, void *data)
 {
@@ -248,7 +247,7 @@ static int userspace_tramp(void *stack)
 	return 0;
 }
 
-int userspace_pid[NR_CPUS];
+int userspace_pid[UM_NR_CPUS];
 
 /**
  * start_userspace() - prepare a new userspace process
diff --git a/arch/um/os-Linux/start_up.c b/tools/um/uml/start_up.c
similarity index 100%
rename from arch/um/os-Linux/start_up.c
rename to tools/um/uml/start_up.c
diff --git a/arch/um/os-Linux/time.c b/tools/um/uml/time.c
similarity index 100%
rename from arch/um/os-Linux/time.c
rename to tools/um/uml/time.c
diff --git a/arch/um/os-Linux/tty.c b/tools/um/uml/tty.c
similarity index 100%
rename from arch/um/os-Linux/tty.c
rename to tools/um/uml/tty.c
diff --git a/arch/um/os-Linux/umid.c b/tools/um/uml/umid.c
similarity index 100%
rename from arch/um/os-Linux/umid.c
rename to tools/um/uml/umid.c
diff --git a/arch/um/os-Linux/user_syms.c b/tools/um/uml/user_syms.c
similarity index 99%
rename from arch/um/os-Linux/user_syms.c
rename to tools/um/uml/user_syms.c
index 715594fe5719..4047fd7bfdb7 100644
--- a/arch/um/os-Linux/user_syms.c
+++ b/tools/um/uml/user_syms.c
@@ -13,6 +13,7 @@
 #undef strstr
 #undef memcpy
 #undef memset
+#define EXPORT_SYMBOL(x)
 
 extern size_t strlen(const char *);
 extern void *memmove(void *, const void *, size_t);
diff --git a/arch/um/os-Linux/util.c b/tools/um/uml/util.c
similarity index 100%
rename from arch/um/os-Linux/util.c
rename to tools/um/uml/util.c
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v5 04/21] um: host: implement os_initcalls and os_exitcalls
  2020-07-02 14:06     ` Hajime Tazaki
@ 2020-07-02 14:06       ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-07-02 14:06 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Octavian Purdila

From: Octavian Purdila <tavi@cs.pub.ro>

This patch implements the init and exit calls for host code. It uses
the automatic __start_<section> and __stop_<section> variables that
are defined by gcc / ld when using custom sections.

Note that this patch should be merged with "um: move arch/um/os-Linux
dir to tools/um" but for now it is separate to make the review easier.

Signed-off-by: Octavian Purdila <tavi@cs.pub.ro>
---
 tools/um/uml/util.c | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)

diff --git a/tools/um/uml/util.c b/tools/um/uml/util.c
index ecf2f390fad2..4011b36fee7e 100644
--- a/tools/um/uml/util.c
+++ b/tools/um/uml/util.c
@@ -186,3 +186,29 @@ void os_warn(const char *fmt, ...)
 	vfprintf(stderr, fmt, list);
 	va_end(list);
 }
+
+extern void (*__start_os_exitcalls)(void);
+extern void (*__stop_os_exitcalls)(void);
+
+void os_exitcalls(void)
+{
+	exitcall_t *call;
+
+	call = &__stop_os_exitcalls;
+	while (--call >= &__start_os_exitcalls)
+		(*call)();
+}
+
+extern int (*__start_os_initcalls)(void);
+extern int (*__stop_os_initcalls)(void);
+
+int os_initcalls(void)
+{
+	initcall_t *call;
+
+	call = &__stop_os_initcalls;
+	while (--call >= &__start_os_initcalls)
+		(*call)();
+
+	return 0;
+}
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v5 04/21] um: host: implement os_initcalls and os_exitcalls
@ 2020-07-02 14:06       ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-07-02 14:06 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, linux-kernel-library, linux-arch,
	Octavian Purdila, Akira Moroo

From: Octavian Purdila <tavi@cs.pub.ro>

This patch implements the init and exit calls for host code. It uses
the automatic __start_<section> and __stop_<section> variables that
are defined by gcc / ld when using custom sections.

Note that this patch should be merged with "um: move arch/um/os-Linux
dir to tools/um" but for now it is separate to make the review easier.

Signed-off-by: Octavian Purdila <tavi@cs.pub.ro>
---
 tools/um/uml/util.c | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)

diff --git a/tools/um/uml/util.c b/tools/um/uml/util.c
index ecf2f390fad2..4011b36fee7e 100644
--- a/tools/um/uml/util.c
+++ b/tools/um/uml/util.c
@@ -186,3 +186,29 @@ void os_warn(const char *fmt, ...)
 	vfprintf(stderr, fmt, list);
 	va_end(list);
 }
+
+extern void (*__start_os_exitcalls)(void);
+extern void (*__stop_os_exitcalls)(void);
+
+void os_exitcalls(void)
+{
+	exitcall_t *call;
+
+	call = &__stop_os_exitcalls;
+	while (--call >= &__start_os_exitcalls)
+		(*call)();
+}
+
+extern int (*__start_os_initcalls)(void);
+extern int (*__stop_os_initcalls)(void);
+
+int os_initcalls(void)
+{
+	initcall_t *call;
+
+	call = &__stop_os_initcalls;
+	while (--call >= &__start_os_initcalls)
+		(*call)();
+
+	return 0;
+}
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v5 05/21] um: move arch/x86/um/os-Linux to tools/um/uml/
  2020-07-02 14:06     ` Hajime Tazaki
@ 2020-07-02 14:06       ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-07-02 14:06 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Hajime Tazaki

Similar to arch/um/os-Linux code, arch/x86 also contains OS dependent
part as well.  This patch moves underlying OS dependent part to tools/um
directory so that arch/x86 code does not need to build host build
facilities (e.g., libc).

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/x86/um/Makefile                                |  2 +-
 arch/x86/um/os-Linux/Makefile                       | 13 -------------
 tools/um/Makefile                                   |  2 +-
 tools/um/uml/Build                                  |  2 ++
 tools/um/uml/x86/Build                              | 11 +++++++++++
 .../x86/um/os-Linux => tools/um/uml/x86}/mcontext.c |  0
 {arch/x86/um/os-Linux => tools/um/uml/x86}/prctl.c  |  0
 .../um/os-Linux => tools/um/uml/x86}/registers.c    |  0
 .../um/os-Linux => tools/um/uml/x86}/task_size.c    |  0
 {arch/x86/um/os-Linux => tools/um/uml/x86}/tls.c    |  0
 10 files changed, 15 insertions(+), 15 deletions(-)
 delete mode 100644 arch/x86/um/os-Linux/Makefile
 create mode 100644 tools/um/uml/x86/Build
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/mcontext.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/prctl.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/registers.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/task_size.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/tls.c (100%)

diff --git a/arch/x86/um/Makefile b/arch/x86/um/Makefile
index 77f70b969d14..25e9846b36a2 100644
--- a/arch/x86/um/Makefile
+++ b/arch/x86/um/Makefile
@@ -13,7 +13,7 @@ obj-y = bugs_$(BITS).o delay.o fault.o ldt.o \
 	ptrace_$(BITS).o ptrace_user.o setjmp_$(BITS).o signal.o \
 	stub_$(BITS).o stub_segv.o \
 	sys_call_table_$(BITS).o sysrq_$(BITS).o tls_$(BITS).o \
-	mem_$(BITS).o subarch.o os-$(OS)/
+	mem_$(BITS).o subarch.o
 
 ifeq ($(CONFIG_X86_32),y)
 
diff --git a/arch/x86/um/os-Linux/Makefile b/arch/x86/um/os-Linux/Makefile
deleted file mode 100644
index 253bfb8cb702..000000000000
--- a/arch/x86/um/os-Linux/Makefile
+++ /dev/null
@@ -1,13 +0,0 @@
-#
-# Copyright (C) 2000 - 2007 Jeff Dike (jdike@{addtoit,linux.intel}.com)
-# Licensed under the GPL
-#
-
-obj-y = registers.o task_size.o mcontext.o
-
-obj-$(CONFIG_X86_32) += tls.o
-obj-$(CONFIG_64BIT) += prctl.o
-
-USER_OBJS := $(obj-y)
-
-include arch/um/scripts/Makefile.rules
diff --git a/tools/um/Makefile b/tools/um/Makefile
index 07b9b93ed817..552ed5f1edae 100644
--- a/tools/um/Makefile
+++ b/tools/um/Makefile
@@ -61,7 +61,7 @@ $(OUTPUT)%-in.o: FORCE
 	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
 
 # rule to build objects
-$(OUTPUT)%-in.o: $(OUTPUT)lib/linux.o
+$(OUTPUT)%-in.o: $(OUTPUT)lib/linux.o FORCE
 	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
 
 clean:
diff --git a/tools/um/uml/Build b/tools/um/uml/Build
index 8e0607ee8b0a..435a9670d3ff 100644
--- a/tools/um/uml/Build
+++ b/tools/um/uml/Build
@@ -12,6 +12,8 @@ liblinux-y += execvp.o file.o helper.o irq.o main.o mem.o process.o \
 	registers.o sigio.o signal.o start_up.o time.o tty.o \
 	umid.o user_syms.o util.o drivers/ skas/
 
+liblinux-y += x86/
+
 liblinux-$(CONFIG_ARCH_REUSE_HOST_VSYSCALL_AREA) += elf_aux.o
 
 export O = $(srctree)
diff --git a/tools/um/uml/x86/Build b/tools/um/uml/x86/Build
new file mode 100644
index 000000000000..16bbc7eb19d1
--- /dev/null
+++ b/tools/um/uml/x86/Build
@@ -0,0 +1,11 @@
+#
+# Copyright (C) 2000 - 2007 Jeff Dike (jdike@{addtoit,linux.intel}.com)
+# Licensed under the GPL
+#
+
+include $(objtree)/include/config/auto.conf
+
+liblinux-y = registers.o task_size.o mcontext.o
+
+liblinux-$(CONFIG_X86_32) += tls.o
+liblinux-$(CONFIG_64BIT) += prctl.o
diff --git a/arch/x86/um/os-Linux/mcontext.c b/tools/um/uml/x86/mcontext.c
similarity index 100%
rename from arch/x86/um/os-Linux/mcontext.c
rename to tools/um/uml/x86/mcontext.c
diff --git a/arch/x86/um/os-Linux/prctl.c b/tools/um/uml/x86/prctl.c
similarity index 100%
rename from arch/x86/um/os-Linux/prctl.c
rename to tools/um/uml/x86/prctl.c
diff --git a/arch/x86/um/os-Linux/registers.c b/tools/um/uml/x86/registers.c
similarity index 100%
rename from arch/x86/um/os-Linux/registers.c
rename to tools/um/uml/x86/registers.c
diff --git a/arch/x86/um/os-Linux/task_size.c b/tools/um/uml/x86/task_size.c
similarity index 100%
rename from arch/x86/um/os-Linux/task_size.c
rename to tools/um/uml/x86/task_size.c
diff --git a/arch/x86/um/os-Linux/tls.c b/tools/um/uml/x86/tls.c
similarity index 100%
rename from arch/x86/um/os-Linux/tls.c
rename to tools/um/uml/x86/tls.c
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v5 05/21] um: move arch/x86/um/os-Linux to tools/um/uml/
@ 2020-07-02 14:06       ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-07-02 14:06 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, linux-kernel-library, linux-arch,
	Hajime Tazaki, Akira Moroo

Similar to arch/um/os-Linux code, arch/x86 also contains OS dependent
part as well.  This patch moves underlying OS dependent part to tools/um
directory so that arch/x86 code does not need to build host build
facilities (e.g., libc).

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/x86/um/Makefile                                |  2 +-
 arch/x86/um/os-Linux/Makefile                       | 13 -------------
 tools/um/Makefile                                   |  2 +-
 tools/um/uml/Build                                  |  2 ++
 tools/um/uml/x86/Build                              | 11 +++++++++++
 .../x86/um/os-Linux => tools/um/uml/x86}/mcontext.c |  0
 {arch/x86/um/os-Linux => tools/um/uml/x86}/prctl.c  |  0
 .../um/os-Linux => tools/um/uml/x86}/registers.c    |  0
 .../um/os-Linux => tools/um/uml/x86}/task_size.c    |  0
 {arch/x86/um/os-Linux => tools/um/uml/x86}/tls.c    |  0
 10 files changed, 15 insertions(+), 15 deletions(-)
 delete mode 100644 arch/x86/um/os-Linux/Makefile
 create mode 100644 tools/um/uml/x86/Build
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/mcontext.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/prctl.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/registers.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/task_size.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/tls.c (100%)

diff --git a/arch/x86/um/Makefile b/arch/x86/um/Makefile
index 77f70b969d14..25e9846b36a2 100644
--- a/arch/x86/um/Makefile
+++ b/arch/x86/um/Makefile
@@ -13,7 +13,7 @@ obj-y = bugs_$(BITS).o delay.o fault.o ldt.o \
 	ptrace_$(BITS).o ptrace_user.o setjmp_$(BITS).o signal.o \
 	stub_$(BITS).o stub_segv.o \
 	sys_call_table_$(BITS).o sysrq_$(BITS).o tls_$(BITS).o \
-	mem_$(BITS).o subarch.o os-$(OS)/
+	mem_$(BITS).o subarch.o
 
 ifeq ($(CONFIG_X86_32),y)
 
diff --git a/arch/x86/um/os-Linux/Makefile b/arch/x86/um/os-Linux/Makefile
deleted file mode 100644
index 253bfb8cb702..000000000000
--- a/arch/x86/um/os-Linux/Makefile
+++ /dev/null
@@ -1,13 +0,0 @@
-#
-# Copyright (C) 2000 - 2007 Jeff Dike (jdike@{addtoit,linux.intel}.com)
-# Licensed under the GPL
-#
-
-obj-y = registers.o task_size.o mcontext.o
-
-obj-$(CONFIG_X86_32) += tls.o
-obj-$(CONFIG_64BIT) += prctl.o
-
-USER_OBJS := $(obj-y)
-
-include arch/um/scripts/Makefile.rules
diff --git a/tools/um/Makefile b/tools/um/Makefile
index 07b9b93ed817..552ed5f1edae 100644
--- a/tools/um/Makefile
+++ b/tools/um/Makefile
@@ -61,7 +61,7 @@ $(OUTPUT)%-in.o: FORCE
 	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
 
 # rule to build objects
-$(OUTPUT)%-in.o: $(OUTPUT)lib/linux.o
+$(OUTPUT)%-in.o: $(OUTPUT)lib/linux.o FORCE
 	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
 
 clean:
diff --git a/tools/um/uml/Build b/tools/um/uml/Build
index 8e0607ee8b0a..435a9670d3ff 100644
--- a/tools/um/uml/Build
+++ b/tools/um/uml/Build
@@ -12,6 +12,8 @@ liblinux-y += execvp.o file.o helper.o irq.o main.o mem.o process.o \
 	registers.o sigio.o signal.o start_up.o time.o tty.o \
 	umid.o user_syms.o util.o drivers/ skas/
 
+liblinux-y += x86/
+
 liblinux-$(CONFIG_ARCH_REUSE_HOST_VSYSCALL_AREA) += elf_aux.o
 
 export O = $(srctree)
diff --git a/tools/um/uml/x86/Build b/tools/um/uml/x86/Build
new file mode 100644
index 000000000000..16bbc7eb19d1
--- /dev/null
+++ b/tools/um/uml/x86/Build
@@ -0,0 +1,11 @@
+#
+# Copyright (C) 2000 - 2007 Jeff Dike (jdike@{addtoit,linux.intel}.com)
+# Licensed under the GPL
+#
+
+include $(objtree)/include/config/auto.conf
+
+liblinux-y = registers.o task_size.o mcontext.o
+
+liblinux-$(CONFIG_X86_32) += tls.o
+liblinux-$(CONFIG_64BIT) += prctl.o
diff --git a/arch/x86/um/os-Linux/mcontext.c b/tools/um/uml/x86/mcontext.c
similarity index 100%
rename from arch/x86/um/os-Linux/mcontext.c
rename to tools/um/uml/x86/mcontext.c
diff --git a/arch/x86/um/os-Linux/prctl.c b/tools/um/uml/x86/prctl.c
similarity index 100%
rename from arch/x86/um/os-Linux/prctl.c
rename to tools/um/uml/x86/prctl.c
diff --git a/arch/x86/um/os-Linux/registers.c b/tools/um/uml/x86/registers.c
similarity index 100%
rename from arch/x86/um/os-Linux/registers.c
rename to tools/um/uml/x86/registers.c
diff --git a/arch/x86/um/os-Linux/task_size.c b/tools/um/uml/x86/task_size.c
similarity index 100%
rename from arch/x86/um/os-Linux/task_size.c
rename to tools/um/uml/x86/task_size.c
diff --git a/arch/x86/um/os-Linux/tls.c b/tools/um/uml/x86/tls.c
similarity index 100%
rename from arch/x86/um/os-Linux/tls.c
rename to tools/um/uml/x86/tls.c
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v5 06/21] scritps: um: suppress warnings if SRCARCH=um
  2020-07-02 14:06     ` Hajime Tazaki
@ 2020-07-02 14:07       ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-07-02 14:07 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Hajime Tazaki

This commit fixes numbers of warning messages about leaked CONFIG
options.  nommu mode of UML requires copies of kernel headers to offer
syscall-like API for the library users.  Thus, the warnings are to be
avoided to function this exposure of API.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 scripts/headers_install.sh | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/scripts/headers_install.sh b/scripts/headers_install.sh
index a07668a5c36b..59966f3b4ebe 100755
--- a/scripts/headers_install.sh
+++ b/scripts/headers_install.sh
@@ -99,6 +99,9 @@ include/uapi/linux/raw.h:CONFIG_MAX_RAW_DEVS
 for c in $configs
 do
 	warn=1
+	if [ "$SRCARCH" = "um" ] ; then
+		break
+	fi
 
 	for ignore in $config_leak_ignores
 	do
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v5 06/21] scritps: um: suppress warnings if SRCARCH=um
@ 2020-07-02 14:07       ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-07-02 14:07 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, linux-kernel-library, linux-arch,
	Hajime Tazaki, Akira Moroo

This commit fixes numbers of warning messages about leaked CONFIG
options.  nommu mode of UML requires copies of kernel headers to offer
syscall-like API for the library users.  Thus, the warnings are to be
avoided to function this exposure of API.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 scripts/headers_install.sh | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/scripts/headers_install.sh b/scripts/headers_install.sh
index a07668a5c36b..59966f3b4ebe 100755
--- a/scripts/headers_install.sh
+++ b/scripts/headers_install.sh
@@ -99,6 +99,9 @@ include/uapi/linux/raw.h:CONFIG_MAX_RAW_DEVS
 for c in $configs
 do
 	warn=1
+	if [ "$SRCARCH" = "um" ] ; then
+		break
+	fi
 
 	for ignore in $config_leak_ignores
 	do
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v5 07/21] um: extend arch_switch_to for alternate SUBARCH
  2020-07-02 14:06     ` Hajime Tazaki
@ 2020-07-02 14:07       ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-07-02 14:07 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Hajime Tazaki

This commit introduces additional argument of previous task when
context switch happens.  New SUBARCH can use the new information to
switch tasks in a subarch-specific manner.

The patch is particularly required by nommu mode implemented as a
SUBARCH of UML.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/kernel/process.c  | 6 +++---
 arch/x86/um/ptrace_32.c   | 2 +-
 arch/x86/um/syscalls_64.c | 2 +-
 3 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/arch/um/kernel/process.c b/arch/um/kernel/process.c
index cbe33af2a880..e5238a42ea17 100644
--- a/arch/um/kernel/process.c
+++ b/arch/um/kernel/process.c
@@ -81,7 +81,7 @@ static inline void set_current(struct task_struct *task)
 		{ external_pid(), task });
 }
 
-extern void arch_switch_to(struct task_struct *to);
+extern void arch_switch_to(struct task_struct *from, struct task_struct *to);
 
 void *__switch_to(struct task_struct *from, struct task_struct *to)
 {
@@ -89,7 +89,7 @@ void *__switch_to(struct task_struct *from, struct task_struct *to)
 	set_current(to);
 
 	switch_threads(&from->thread.switch_buf, &to->thread.switch_buf);
-	arch_switch_to(current);
+	arch_switch_to(from, to);
 
 	return current->thread.prev_sched;
 }
@@ -146,7 +146,7 @@ void fork_handler(void)
 	 * arch_switch_to isn't needed. We could want to apply this to
 	 * improve performance. -bb
 	 */
-	arch_switch_to(current);
+	arch_switch_to(NULL, current);
 
 	current->thread.prev_sched = NULL;
 
diff --git a/arch/x86/um/ptrace_32.c b/arch/x86/um/ptrace_32.c
index 2497bac56066..0f184710d4ca 100644
--- a/arch/x86/um/ptrace_32.c
+++ b/arch/x86/um/ptrace_32.c
@@ -11,7 +11,7 @@
 
 extern int arch_switch_tls(struct task_struct *to);
 
-void arch_switch_to(struct task_struct *to)
+void arch_switch_to(struct task_struct *from, struct task_struct *to)
 {
 	int err = arch_switch_tls(to);
 	if (!err)
diff --git a/arch/x86/um/syscalls_64.c b/arch/x86/um/syscalls_64.c
index 58f51667e2e4..2ef9474d2bd2 100644
--- a/arch/x86/um/syscalls_64.c
+++ b/arch/x86/um/syscalls_64.c
@@ -80,7 +80,7 @@ SYSCALL_DEFINE2(arch_prctl, int, option, unsigned long, arg2)
 	return arch_prctl(current, option, (unsigned long __user *) arg2);
 }
 
-void arch_switch_to(struct task_struct *to)
+void arch_switch_to(struct task_struct *from, struct task_struct *to)
 {
 	if ((to->thread.arch.fs == 0) || (to->mm == NULL))
 		return;
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v5 07/21] um: extend arch_switch_to for alternate SUBARCH
@ 2020-07-02 14:07       ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-07-02 14:07 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, linux-kernel-library, linux-arch,
	Hajime Tazaki, Akira Moroo

This commit introduces additional argument of previous task when
context switch happens.  New SUBARCH can use the new information to
switch tasks in a subarch-specific manner.

The patch is particularly required by nommu mode implemented as a
SUBARCH of UML.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/kernel/process.c  | 6 +++---
 arch/x86/um/ptrace_32.c   | 2 +-
 arch/x86/um/syscalls_64.c | 2 +-
 3 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/arch/um/kernel/process.c b/arch/um/kernel/process.c
index cbe33af2a880..e5238a42ea17 100644
--- a/arch/um/kernel/process.c
+++ b/arch/um/kernel/process.c
@@ -81,7 +81,7 @@ static inline void set_current(struct task_struct *task)
 		{ external_pid(), task });
 }
 
-extern void arch_switch_to(struct task_struct *to);
+extern void arch_switch_to(struct task_struct *from, struct task_struct *to);
 
 void *__switch_to(struct task_struct *from, struct task_struct *to)
 {
@@ -89,7 +89,7 @@ void *__switch_to(struct task_struct *from, struct task_struct *to)
 	set_current(to);
 
 	switch_threads(&from->thread.switch_buf, &to->thread.switch_buf);
-	arch_switch_to(current);
+	arch_switch_to(from, to);
 
 	return current->thread.prev_sched;
 }
@@ -146,7 +146,7 @@ void fork_handler(void)
 	 * arch_switch_to isn't needed. We could want to apply this to
 	 * improve performance. -bb
 	 */
-	arch_switch_to(current);
+	arch_switch_to(NULL, current);
 
 	current->thread.prev_sched = NULL;
 
diff --git a/arch/x86/um/ptrace_32.c b/arch/x86/um/ptrace_32.c
index 2497bac56066..0f184710d4ca 100644
--- a/arch/x86/um/ptrace_32.c
+++ b/arch/x86/um/ptrace_32.c
@@ -11,7 +11,7 @@
 
 extern int arch_switch_tls(struct task_struct *to);
 
-void arch_switch_to(struct task_struct *to)
+void arch_switch_to(struct task_struct *from, struct task_struct *to)
 {
 	int err = arch_switch_tls(to);
 	if (!err)
diff --git a/arch/x86/um/syscalls_64.c b/arch/x86/um/syscalls_64.c
index 58f51667e2e4..2ef9474d2bd2 100644
--- a/arch/x86/um/syscalls_64.c
+++ b/arch/x86/um/syscalls_64.c
@@ -80,7 +80,7 @@ SYSCALL_DEFINE2(arch_prctl, int, option, unsigned long, arg2)
 	return arch_prctl(current, option, (unsigned long __user *) arg2);
 }
 
-void arch_switch_to(struct task_struct *to)
+void arch_switch_to(struct task_struct *from, struct task_struct *to)
 {
 	if ((to->thread.arch.fs == 0) || (to->mm == NULL))
 		return;
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v5 08/21] um: add nommu mode for UML library mode
  2020-07-02 14:06     ` Hajime Tazaki
@ 2020-07-02 14:07       ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-07-02 14:07 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Hajime Tazaki

This patch introduces the nommu operation with UML code so that host
interface can be shrineked for broader environments support.

The nommu mode is implemneted as SUBARCH of arch/um, which places at
arch/um/nommu. This SUBARCH defines mode-specific code of memory
management as well as thread implementation, along with the uapi headers
to be exported to users.

The headers we introduce in this patch are simple wrappers to the
asm-generic headers or stubs for things we don't support, such as
ptrace, DMA, signals, ELF handling and low level processor operations.

nommu mode shares most of arch/um code (irq, thread_info, process
scheduling) but implements its own facilities, which are memory
management (nommu), thread primitives (struct arch_thread), system call
interface, and console.

The outlook of updated directory structure is as follows:

   arch/um
   |-- configs         (untouched)
   |-- drivers         unuse stdio_console.c for !MMU
   |-- include
   |   |-- asm         updated with !CONFIG_MMU
   |   |-- linux       (untouched)
   |   `-- shareda     updated with new functions
   |       `-- skas    (untouched)
   |   `-- uapi        added for upai header installation
   |-- kernel          updated to integrate with !MMU
   |   `-- skas        (untouched, don't use for !MMU)
   |-- nommu           SUBARCH dir (internally =um/nommu)
   |   |-- include
   |   |   |-- asm     headers for subarch specific info
   |   |   `-- uapi    headers for user-visible APIs
   |   `-- um
   |       `-- shared  headers for subarch specific info
   `-- scripts         added a script for header installation

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/nommu/Makefile                        |   1 +
 arch/um/nommu/Makefile.um                     |  18 +++
 arch/um/nommu/include/asm/Kbuild              |   6 +
 arch/um/nommu/include/asm/archparam.h         |   1 +
 arch/um/nommu/include/asm/atomic.h            |  11 ++
 arch/um/nommu/include/asm/atomic64.h          | 114 ++++++++++++++++++
 arch/um/nommu/include/asm/bitsperlong.h       |  12 ++
 arch/um/nommu/include/asm/byteorder.h         |   7 ++
 arch/um/nommu/include/asm/cpu.h               |  13 ++
 arch/um/nommu/include/asm/elf.h               |  15 +++
 arch/um/nommu/include/asm/mm_context.h        |   8 ++
 arch/um/nommu/include/asm/processor.h         |  46 +++++++
 arch/um/nommu/include/asm/ptrace.h            |  21 ++++
 arch/um/nommu/include/asm/sched.h             |  23 ++++
 arch/um/nommu/include/asm/segment.h           |   9 ++
 arch/um/nommu/include/uapi/asm/Kbuild         |   4 +
 arch/um/nommu/include/uapi/asm/bitsperlong.h  |  11 ++
 arch/um/nommu/include/uapi/asm/byteorder.h    |  11 ++
 arch/um/nommu/include/uapi/asm/sigcontext.h   |  12 ++
 arch/um/nommu/um/delay.c                      |  31 +++++
 arch/um/nommu/um/shared/sysdep/archsetjmp.h   |  13 ++
 arch/um/nommu/um/shared/sysdep/faultinfo.h    |   8 ++
 .../nommu/um/shared/sysdep/kernel-offsets.h   |  12 ++
 arch/um/nommu/um/shared/sysdep/mcontext.h     |   9 ++
 arch/um/nommu/um/shared/sysdep/ptrace.h       |  42 +++++++
 arch/um/nommu/um/shared/sysdep/ptrace_user.h  |   7 ++
 arch/um/nommu/um/unimplemented.c              |  70 +++++++++++
 arch/um/nommu/um/user_constants.h             |  13 ++
 28 files changed, 548 insertions(+)
 create mode 100644 arch/um/nommu/Makefile
 create mode 100644 arch/um/nommu/Makefile.um
 create mode 100644 arch/um/nommu/include/asm/Kbuild
 create mode 100644 arch/um/nommu/include/asm/archparam.h
 create mode 100644 arch/um/nommu/include/asm/atomic.h
 create mode 100644 arch/um/nommu/include/asm/atomic64.h
 create mode 100644 arch/um/nommu/include/asm/bitsperlong.h
 create mode 100644 arch/um/nommu/include/asm/byteorder.h
 create mode 100644 arch/um/nommu/include/asm/cpu.h
 create mode 100644 arch/um/nommu/include/asm/elf.h
 create mode 100644 arch/um/nommu/include/asm/mm_context.h
 create mode 100644 arch/um/nommu/include/asm/processor.h
 create mode 100644 arch/um/nommu/include/asm/ptrace.h
 create mode 100644 arch/um/nommu/include/asm/sched.h
 create mode 100644 arch/um/nommu/include/asm/segment.h
 create mode 100644 arch/um/nommu/include/uapi/asm/Kbuild
 create mode 100644 arch/um/nommu/include/uapi/asm/bitsperlong.h
 create mode 100644 arch/um/nommu/include/uapi/asm/byteorder.h
 create mode 100644 arch/um/nommu/include/uapi/asm/sigcontext.h
 create mode 100644 arch/um/nommu/um/delay.c
 create mode 100644 arch/um/nommu/um/shared/sysdep/archsetjmp.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/faultinfo.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/kernel-offsets.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/mcontext.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/ptrace.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/ptrace_user.h
 create mode 100644 arch/um/nommu/um/unimplemented.c
 create mode 100644 arch/um/nommu/um/user_constants.h

diff --git a/arch/um/nommu/Makefile b/arch/um/nommu/Makefile
new file mode 100644
index 000000000000..792d6005489e
--- /dev/null
+++ b/arch/um/nommu/Makefile
@@ -0,0 +1 @@
+#
diff --git a/arch/um/nommu/Makefile.um b/arch/um/nommu/Makefile.um
new file mode 100644
index 000000000000..6cb0e9494d05
--- /dev/null
+++ b/arch/um/nommu/Makefile.um
@@ -0,0 +1,18 @@
+KBUILD_CFLAGS += -fno-builtin -fPIC
+ELF_FORMAT=$(shell $(LD) -r -print-output-format)
+
+ifeq ($(shell uname -s), Linux)
+NPROC=$(shell nproc)
+else # e.g., FreeBSD
+NPROC=$(shell sysctl -n hw.ncpu)
+endif
+
+um_headers_install: $(objtree)/$(HOST_DIR)/include/generated/uapi/asm/syscall_defs.h headers
+	$(Q)$(srctree)/$(ARCH_DIR)/scripts/headers_install.py \
+		$(subst -j,-j$(NPROC),$(findstring -j,$(MAKEFLAGS))) \
+		$(INSTALL_PATH)/include
+
+$(objtree)/$(HOST_DIR)/include/generated/uapi/asm/syscall_defs.h: vmlinux
+	$(Q)$(OBJCOPY) -j .syscall_defs -O binary --set-section-flags .syscall_defs=alloc $< $@
+	$(Q) export tmpfile=$(shell mktemp); \
+	sed 's/\x0//g' $@ > $$tmpfile; mv $$tmpfile $@ ; rm -f $$tmpfile
diff --git a/arch/um/nommu/include/asm/Kbuild b/arch/um/nommu/include/asm/Kbuild
new file mode 100644
index 000000000000..2532e1a0a0d1
--- /dev/null
+++ b/arch/um/nommu/include/asm/Kbuild
@@ -0,0 +1,6 @@
+generic-y += cmpxchg.h
+generic-y += local64.h
+generic-y += seccomp.h
+generic-y += string.h
+generic-y += syscall.h
+generic-y += user.h
diff --git a/arch/um/nommu/include/asm/archparam.h b/arch/um/nommu/include/asm/archparam.h
new file mode 100644
index 000000000000..ea32a7d3cf1b
--- /dev/null
+++ b/arch/um/nommu/include/asm/archparam.h
@@ -0,0 +1 @@
+/* SPDX-License-Identifier: GPL-2.0 */
diff --git a/arch/um/nommu/include/asm/atomic.h b/arch/um/nommu/include/asm/atomic.h
new file mode 100644
index 000000000000..63e2e16bda92
--- /dev/null
+++ b/arch/um/nommu/include/asm/atomic.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_ATOMIC_H
+#define __UM_NOMMU_ATOMIC_H
+
+#include <asm-generic/atomic.h>
+
+#ifndef CONFIG_GENERIC_ATOMIC64
+#include "atomic64.h"
+#endif /* !CONFIG_GENERIC_ATOMIC64 */
+
+#endif
diff --git a/arch/um/nommu/include/asm/atomic64.h b/arch/um/nommu/include/asm/atomic64.h
new file mode 100644
index 000000000000..949360dea7af
--- /dev/null
+++ b/arch/um/nommu/include/asm/atomic64.h
@@ -0,0 +1,114 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_ATOMIC64_H
+#define __UM_NOMMU_ATOMIC64_H
+
+#include <linux/types.h>
+
+#ifdef CONFIG_SMP
+#error "SMP is not supported on this platform"
+#else
+#define ATOMIC64_OP(op, c_op)					\
+	static inline void atomic64_##op(s64 i, atomic64_t *v)	\
+	{							\
+		unsigned long flags;				\
+								\
+		raw_local_irq_save(flags);			\
+		v->counter = v->counter c_op i;			\
+		raw_local_irq_restore(flags);			\
+	}
+
+#define ATOMIC64_OP_RETURN(op, c_op)					\
+	static inline s64 atomic64_##op##_return(s64 i, atomic64_t *v)	\
+	{								\
+		unsigned long flags;					\
+		s64 ret;						\
+									\
+		raw_local_irq_save(flags);				\
+		ret = (v->counter = v->counter c_op i);			\
+		raw_local_irq_restore(flags);				\
+									\
+		return ret;						\
+	}
+
+#define ATOMIC64_FETCH_OP(op, c_op)					\
+	static inline s64 atomic64_fetch_##op(s64 i, atomic64_t *v)	\
+	{								\
+		unsigned long flags;					\
+		s64 ret;						\
+									\
+		raw_local_irq_save(flags);				\
+		ret = v->counter;					\
+		v->counter = v->counter c_op i;				\
+		raw_local_irq_restore(flags);				\
+									\
+		return ret;						\
+	}
+#endif /* CONFIG_SMP */
+
+#ifndef atomic64_add_return
+ATOMIC64_OP_RETURN(add, +)
+#endif
+
+#ifndef atomic64_sub_return
+	ATOMIC64_OP_RETURN(sub, -)
+#endif
+
+#ifndef atomic64_fetch_add
+	ATOMIC64_FETCH_OP(add, +)
+#endif
+
+#ifndef atomic64_fetch_sub
+	ATOMIC64_FETCH_OP(sub, -)
+#endif
+
+#ifndef atomic64_fetch_and
+	ATOMIC64_FETCH_OP(and, &)
+#endif
+
+#ifndef atomic64_fetch_or
+	ATOMIC64_FETCH_OP(or, |)
+#endif
+
+#ifndef atomic64_fetch_xor
+	ATOMIC64_FETCH_OP(xor, ^)
+#endif
+
+#ifndef atomic64_and
+	ATOMIC64_OP(and, &)
+#endif
+
+#ifndef atomic64_or
+	ATOMIC64_OP(or, |)
+#endif
+
+#ifndef atomic64_xor
+	ATOMIC64_OP(xor, ^)
+#endif
+
+#undef ATOMIC64_FETCH_OP
+#undef ATOMIC64_OP_RETURN
+#undef ATOMIC64_OP
+
+
+#define ATOMIC64_INIT(i)       { (i) }
+
+	static inline void atomic64_add(s64 i, atomic64_t *v)
+{
+	atomic64_add_return(i, v);
+}
+
+static inline void atomic64_sub(s64 i, atomic64_t *v)
+{
+	atomic64_sub_return(i, v);
+}
+
+#ifndef atomic64_read
+#define atomic64_read(v)       READ_ONCE((v)->counter)
+#endif
+
+#define atomic64_set(v, i) WRITE_ONCE(((v)->counter), (i))
+
+#define atomic64_xchg(ptr, v)          (xchg(&(ptr)->counter, (v)))
+#define atomic64_cmpxchg(v, old, new)  (cmpxchg(&((v)->counter), (old), (new)))
+
+#endif /* __LKL_ATOMIC64_H */
diff --git a/arch/um/nommu/include/asm/bitsperlong.h b/arch/um/nommu/include/asm/bitsperlong.h
new file mode 100644
index 000000000000..a150cee41e75
--- /dev/null
+++ b/arch/um/nommu/include/asm/bitsperlong.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_BITSPERLONG_H
+#define __UM_NOMMU_BITSPERLONG_H
+
+#include <uapi/asm/bitsperlong.h>
+
+#define BITS_PER_LONG __BITS_PER_LONG
+
+#define BITS_PER_LONG_LONG 64
+
+#endif
+
diff --git a/arch/um/nommu/include/asm/byteorder.h b/arch/um/nommu/include/asm/byteorder.h
new file mode 100644
index 000000000000..920a5fd26cad
--- /dev/null
+++ b/arch/um/nommu/include/asm/byteorder.h
@@ -0,0 +1,7 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_BYTEORDER_H
+#define __UM_NOMMU_BYTEORDER_H
+
+#include <uapi/asm/byteorder.h>
+
+#endif
diff --git a/arch/um/nommu/include/asm/cpu.h b/arch/um/nommu/include/asm/cpu.h
new file mode 100644
index 000000000000..c101c078ef21
--- /dev/null
+++ b/arch/um/nommu/include/asm/cpu.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_CPU_H
+#define __UM_NOMMU_CPU_H
+
+int lkl_cpu_get(void);
+void lkl_cpu_put(void);
+int lkl_cpu_try_run_irq(int irq);
+int lkl_cpu_init(void);
+void lkl_cpu_wait_shutdown(void);
+void lkl_cpu_change_owner(lkl_thread_t owner);
+void lkl_cpu_set_irqs_pending(void);
+
+#endif
diff --git a/arch/um/nommu/include/asm/elf.h b/arch/um/nommu/include/asm/elf.h
new file mode 100644
index 000000000000..edcf63edeed1
--- /dev/null
+++ b/arch/um/nommu/include/asm/elf.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_ELF_H
+#define __UM_NOMMU_ELF_H
+
+#define elf_check_arch(x) 0
+
+#ifdef CONFIG_64BIT
+#define ELF_CLASS ELFCLASS64
+#else
+#define ELF_CLASS ELFCLASS32
+#endif
+
+#define elf_gregset_t long
+#define elf_fpregset_t double
+#endif
diff --git a/arch/um/nommu/include/asm/mm_context.h b/arch/um/nommu/include/asm/mm_context.h
new file mode 100644
index 000000000000..a2e53984aabd
--- /dev/null
+++ b/arch/um/nommu/include/asm/mm_context.h
@@ -0,0 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_MM_CONTEXT_H
+#define __UM_NOMMU_MM_CONTEXT_H
+
+struct uml_arch_mm_context {
+};
+
+#endif
diff --git a/arch/um/nommu/include/asm/processor.h b/arch/um/nommu/include/asm/processor.h
new file mode 100644
index 000000000000..3e8ba870caaf
--- /dev/null
+++ b/arch/um/nommu/include/asm/processor.h
@@ -0,0 +1,46 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_PROCESSOR_H
+#define __UM_NOMMU_PROCESSOR_H
+
+#include <asm/host_ops.h>
+
+struct arch_thread {
+	struct lkl_sem *sched_sem;
+	bool dead;
+	lkl_thread_t tid;
+	struct lkl_jmp_buf sched_jb;
+	unsigned long stackend;
+};
+
+#include <asm/ptrace-generic.h>
+#include <asm/processor-generic.h>
+
+#define INIT_ARCH_THREAD {}
+#define task_pt_regs(tsk) (struct pt_regs *)(NULL)
+
+static inline void cpu_relax(void)
+{
+	unsigned long flags;
+
+	/* since this is usually called in a tight loop waiting for some
+	 * external condition (e.g. jiffies) lets run interrupts now to allow
+	 * the external condition to propagate
+	 */
+	local_irq_save(flags);
+	local_irq_restore(flags);
+}
+
+#define KSTK_EIP(tsk)	(0)
+#define KSTK_ESP(tsk)	(0)
+
+static inline void trap_init(void)
+{
+}
+
+static inline void arch_copy_thread(struct arch_thread *from,
+				    struct arch_thread *to)
+{
+	panic("unimplemented %s: fork isn't supported yet", __func__);
+}
+
+#endif
diff --git a/arch/um/nommu/include/asm/ptrace.h b/arch/um/nommu/include/asm/ptrace.h
new file mode 100644
index 000000000000..b214410e9825
--- /dev/null
+++ b/arch/um/nommu/include/asm/ptrace.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_PTRACE_H
+#define __UM_NOMMU_PTRACE_H
+
+#include <linux/errno.h>
+
+static int reg_dummy __attribute__((unused));
+
+#define PT_REGS_ORIG_SYSCALL(r) (reg_dummy)
+#define PT_REGS_SYSCALL_RET(r) (reg_dummy)
+#define PT_REGS_SET_SYSCALL_RETURN(r, res) (reg_dummy = (res))
+#define REGS_SP(r) (reg_dummy)
+
+#define user_mode(regs) 0
+#define kernel_mode(regs) 1
+#define profile_pc(regs) 0
+#define user_stack_pointer(regs) 0
+
+extern void new_thread_handler(void);
+
+#endif
diff --git a/arch/um/nommu/include/asm/sched.h b/arch/um/nommu/include/asm/sched.h
new file mode 100644
index 000000000000..a4496f482633
--- /dev/null
+++ b/arch/um/nommu/include/asm/sched.h
@@ -0,0 +1,23 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_SCHED_H
+#define __UM_NOMMU_SCHED_H
+
+#include <linux/sched.h>
+#include <uapi/asm/host_ops.h>
+
+static inline void thread_sched_jb(void)
+{
+	if (test_ti_thread_flag(current_thread_info(), TIF_HOST_THREAD)) {
+		set_ti_thread_flag(current_thread_info(), TIF_SCHED_JB);
+		set_current_state(TASK_UNINTERRUPTIBLE);
+		lkl_ops->jmp_buf_set(&current_thread_info()->task->thread.arch.sched_jb,
+				     schedule);
+	} else {
+		lkl_bug("%s can be used only for host task", __func__);
+	}
+}
+
+void switch_to_host_task(struct task_struct *);
+int host_task_stub(void *unused);
+
+#endif
diff --git a/arch/um/nommu/include/asm/segment.h b/arch/um/nommu/include/asm/segment.h
new file mode 100644
index 000000000000..5608da95cb60
--- /dev/null
+++ b/arch/um/nommu/include/asm/segment.h
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_SEGMENT_H
+#define __UM_NOMMU_SEGMENT_H
+
+typedef struct {
+	unsigned long seg;
+} mm_segment_t;
+
+#endif /* _ASM_LKL_SEGMENT_H */
diff --git a/arch/um/nommu/include/uapi/asm/Kbuild b/arch/um/nommu/include/uapi/asm/Kbuild
new file mode 100644
index 000000000000..4f79e5c4846d
--- /dev/null
+++ b/arch/um/nommu/include/uapi/asm/Kbuild
@@ -0,0 +1,4 @@
+# UAPI Header export list
+
+# no generated-y since we need special user headers handling
+# see arch/um/script/headers_install.py
diff --git a/arch/um/nommu/include/uapi/asm/bitsperlong.h b/arch/um/nommu/include/uapi/asm/bitsperlong.h
new file mode 100644
index 000000000000..852566ac2e52
--- /dev/null
+++ b/arch/um/nommu/include/uapi/asm/bitsperlong.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef __UM_NOMMU_UAPI_BITSPERLONG_H
+#define __UM_NOMMU_UAPI_BITSPERLONG_H
+
+#ifdef CONFIG_64BIT
+#define __BITS_PER_LONG 64
+#else
+#define __BITS_PER_LONG 32
+#endif
+
+#endif
diff --git a/arch/um/nommu/include/uapi/asm/byteorder.h b/arch/um/nommu/include/uapi/asm/byteorder.h
new file mode 100644
index 000000000000..e7ad11a751cf
--- /dev/null
+++ b/arch/um/nommu/include/uapi/asm/byteorder.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef __UM_NOMMU_UAPI_BYTEORDER_H
+#define __UM_NOMMU_UAPI_BYTEORDER_H
+
+#if defined(CONFIG_BIG_ENDIAN)
+#include <linux/byteorder/big_endian.h>
+#else
+#include <linux/byteorder/little_endian.h>
+#endif
+
+#endif
diff --git a/arch/um/nommu/include/uapi/asm/sigcontext.h b/arch/um/nommu/include/uapi/asm/sigcontext.h
new file mode 100644
index 000000000000..b934ae7f5550
--- /dev/null
+++ b/arch/um/nommu/include/uapi/asm/sigcontext.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef __UM_NOMMU_UAPI_SIGCONTEXT_H
+#define __UM_NOMMU_UAPI_SIGCONTEXT_H
+
+#include <asm/ptrace-generic.h>
+
+struct sigcontext {
+	struct pt_regs regs;
+	unsigned long oldmask;
+};
+
+#endif
diff --git a/arch/um/nommu/um/delay.c b/arch/um/nommu/um/delay.c
new file mode 100644
index 000000000000..58a366d9b5f0
--- /dev/null
+++ b/arch/um/nommu/um/delay.c
@@ -0,0 +1,31 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/jiffies.h>
+#include <linux/delay.h>
+#include <os.h>
+
+void __ndelay(unsigned long nsecs)
+{
+	long long start = os_nsecs();
+
+	while (os_nsecs() < start + nsecs)
+		;
+}
+
+void __udelay(unsigned long usecs)
+{
+	__ndelay(usecs * NSEC_PER_USEC);
+}
+
+void __const_udelay(unsigned long xloops)
+{
+	__udelay(xloops / 0x10c7ul);
+}
+
+void __delay(unsigned long loops)
+{
+	__ndelay(loops / 5);
+}
+
+void calibrate_delay(void)
+{
+}
diff --git a/arch/um/nommu/um/shared/sysdep/archsetjmp.h b/arch/um/nommu/um/shared/sysdep/archsetjmp.h
new file mode 100644
index 000000000000..4a5c10d7521b
--- /dev/null
+++ b/arch/um/nommu/um/shared/sysdep/archsetjmp.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ARCH_UM_SETJMP_H
+#define __ARCH_UM_SETJMP_H
+
+struct __jmp_buf {
+	unsigned long __dummy;
+};
+#define JB_IP __dummy
+#define JB_SP __dummy
+
+typedef struct __jmp_buf jmp_buf[1];
+
+#endif /* __ARCH_UM_SETJMP_H */
diff --git a/arch/um/nommu/um/shared/sysdep/faultinfo.h b/arch/um/nommu/um/shared/sysdep/faultinfo.h
new file mode 100644
index 000000000000..49210742212d
--- /dev/null
+++ b/arch/um/nommu/um/shared/sysdep/faultinfo.h
@@ -0,0 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ARCH_UM_FAULTINFO_H
+#define __ARCH_UM_FAULTINFO_H
+
+struct faultinfo {
+};
+
+#endif /* __ARCH_UM_FAULTINFO_H */
diff --git a/arch/um/nommu/um/shared/sysdep/kernel-offsets.h b/arch/um/nommu/um/shared/sysdep/kernel-offsets.h
new file mode 100644
index 000000000000..a004bffb7b8d
--- /dev/null
+++ b/arch/um/nommu/um/shared/sysdep/kernel-offsets.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <linux/stddef.h>
+#include <linux/sched.h>
+#include <linux/elf.h>
+#include <linux/crypto.h>
+#include <linux/kbuild.h>
+#include <asm/mman.h>
+
+void foo(void)
+{
+#include <common-offsets.h>
+}
diff --git a/arch/um/nommu/um/shared/sysdep/mcontext.h b/arch/um/nommu/um/shared/sysdep/mcontext.h
new file mode 100644
index 000000000000..b734012a74da
--- /dev/null
+++ b/arch/um/nommu/um/shared/sysdep/mcontext.h
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ARCH_UM_MCONTEXT_H
+#define __ARCH_UM_MCONTEXT_H
+
+extern void get_regs_from_mc(struct uml_pt_regs *regs, mcontext_t *mc);
+
+#define GET_FAULTINFO_FROM_MC(fi, mc) (fi = fi)
+
+#endif /* __ARCH_UM_MCONTEXT_H */
diff --git a/arch/um/nommu/um/shared/sysdep/ptrace.h b/arch/um/nommu/um/shared/sysdep/ptrace.h
new file mode 100644
index 000000000000..bfdfb520a21d
--- /dev/null
+++ b/arch/um/nommu/um/shared/sysdep/ptrace.h
@@ -0,0 +1,42 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ARCH_UM_PTRACE_H
+#define __ARCH_UM_PTRACE_H
+
+#include <generated/user_constants.h>
+#include <linux/errno.h>
+
+struct task_struct;
+
+#define UPT_SYSCALL_NR(r) ((r)->syscall)
+#define UPT_RESTART_SYSCALL(r) ((r)->syscall--) /* XXX */
+
+#define UPT_SP(r) 0
+#define UPT_IP(r) 0
+#define EMPTY_UML_PT_REGS { }
+
+#define MAX_REG_OFFSET (UM_FRAME_SIZE)
+#define MAX_REG_NR ((MAX_REG_OFFSET) / sizeof(unsigned long))
+
+/* unused */
+struct uml_pt_regs {
+	unsigned long gp[1];
+	unsigned long fp[1];
+	long faultinfo;
+	long syscall;
+	int is_user;
+};
+
+extern void arch_init_registers(int pid);
+
+static inline long arch_ptrace(struct task_struct *child,
+			       long request, unsigned long addr,
+			       unsigned long data)
+{
+	return -EINVAL;
+}
+
+static inline void ptrace_disable(struct task_struct *child) {}
+static inline void user_enable_single_step(struct task_struct *child) {}
+static inline void user_disable_single_step(struct task_struct *child) {}
+
+#endif /* __ARCH_UM_PTRACE_H */
diff --git a/arch/um/nommu/um/shared/sysdep/ptrace_user.h b/arch/um/nommu/um/shared/sysdep/ptrace_user.h
new file mode 100644
index 000000000000..86d5cb20fb9f
--- /dev/null
+++ b/arch/um/nommu/um/shared/sysdep/ptrace_user.h
@@ -0,0 +1,7 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ARCH_UM_PTRACE_USER_H
+#define __ARCH_UM_PTRACE_USER_H
+
+#define FP_SIZE 1
+
+#endif
diff --git a/arch/um/nommu/um/unimplemented.c b/arch/um/nommu/um/unimplemented.c
new file mode 100644
index 000000000000..fe33e02e39e5
--- /dev/null
+++ b/arch/um/nommu/um/unimplemented.c
@@ -0,0 +1,70 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/signal.h>
+#include <sysdep/ptrace.h>
+#include <asm/ptrace.h>
+
+/* physmem.c  */
+unsigned long high_physmem;
+
+/* x86/um/setjmp*.S  */
+void kernel_longjmp(void)
+{}
+void kernel_setjmp(void)
+{}
+
+/* trap.c */
+void relay_signal(int sig, struct siginfo *si, struct uml_pt_regs *regs)
+{}
+void bus_handler(int sig, struct siginfo *si, struct uml_pt_regs *regs)
+{}
+void segv_handler(int sig, struct siginfo *unused_si, struct uml_pt_regs *regs)
+{}
+void winch(int sig, struct siginfo *unused_si, struct uml_pt_regs *regs)
+{}
+
+/* tlb.c */
+void flush_tlb_kernel_vm(void)
+{}
+void force_flush_all(void)
+{}
+
+/* skas/process.c */
+void halt_skas(void)
+{}
+int is_skas_winch(int pid, int fd, void *data)
+{
+	return 0;
+}
+void reboot_skas(void)
+{}
+
+int __init start_uml(void)
+{
+	return 0;
+}
+
+/* exec.c */
+void flush_thread(void)
+{}
+
+/* x86/ptrace_64.c */
+int is_syscall(unsigned long addr)
+{
+	return 0;
+}
+
+
+/* x86/sysrq.c */
+void show_regs(struct pt_regs *regs)
+{}
+
+/* x86/signal.c */
+int setup_signal_stack_si(unsigned long stack_top, struct ksignal *ksig,
+			  struct pt_regs *regs, sigset_t *mask)
+{
+	return 0;
+}
+
+/* x86/bugs.c */
+void arch_check_bugs(void)
+{}
diff --git a/arch/um/nommu/um/user_constants.h b/arch/um/nommu/um/user_constants.h
new file mode 100644
index 000000000000..2245d3d24120
--- /dev/null
+++ b/arch/um/nommu/um/user_constants.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_USER_CONSTANTS_H
+#define __UM_NOMMU_USER_CONSTANTS_H
+
+/* XXX: put dummy values */
+#define UM_FRAME_SIZE 4
+#define HOST_FP_SIZE 1
+#define HOST_IP 1
+#define HOST_SP 2
+#define HOST_BP 3
+#define UM_NR_CPUS 1
+
+#endif
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v5 08/21] um: add nommu mode for UML library mode
@ 2020-07-02 14:07       ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-07-02 14:07 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, linux-kernel-library, linux-arch,
	Hajime Tazaki, Akira Moroo

This patch introduces the nommu operation with UML code so that host
interface can be shrineked for broader environments support.

The nommu mode is implemneted as SUBARCH of arch/um, which places at
arch/um/nommu. This SUBARCH defines mode-specific code of memory
management as well as thread implementation, along with the uapi headers
to be exported to users.

The headers we introduce in this patch are simple wrappers to the
asm-generic headers or stubs for things we don't support, such as
ptrace, DMA, signals, ELF handling and low level processor operations.

nommu mode shares most of arch/um code (irq, thread_info, process
scheduling) but implements its own facilities, which are memory
management (nommu), thread primitives (struct arch_thread), system call
interface, and console.

The outlook of updated directory structure is as follows:

   arch/um
   |-- configs         (untouched)
   |-- drivers         unuse stdio_console.c for !MMU
   |-- include
   |   |-- asm         updated with !CONFIG_MMU
   |   |-- linux       (untouched)
   |   `-- shareda     updated with new functions
   |       `-- skas    (untouched)
   |   `-- uapi        added for upai header installation
   |-- kernel          updated to integrate with !MMU
   |   `-- skas        (untouched, don't use for !MMU)
   |-- nommu           SUBARCH dir (internally =um/nommu)
   |   |-- include
   |   |   |-- asm     headers for subarch specific info
   |   |   `-- uapi    headers for user-visible APIs
   |   `-- um
   |       `-- shared  headers for subarch specific info
   `-- scripts         added a script for header installation

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/nommu/Makefile                        |   1 +
 arch/um/nommu/Makefile.um                     |  18 +++
 arch/um/nommu/include/asm/Kbuild              |   6 +
 arch/um/nommu/include/asm/archparam.h         |   1 +
 arch/um/nommu/include/asm/atomic.h            |  11 ++
 arch/um/nommu/include/asm/atomic64.h          | 114 ++++++++++++++++++
 arch/um/nommu/include/asm/bitsperlong.h       |  12 ++
 arch/um/nommu/include/asm/byteorder.h         |   7 ++
 arch/um/nommu/include/asm/cpu.h               |  13 ++
 arch/um/nommu/include/asm/elf.h               |  15 +++
 arch/um/nommu/include/asm/mm_context.h        |   8 ++
 arch/um/nommu/include/asm/processor.h         |  46 +++++++
 arch/um/nommu/include/asm/ptrace.h            |  21 ++++
 arch/um/nommu/include/asm/sched.h             |  23 ++++
 arch/um/nommu/include/asm/segment.h           |   9 ++
 arch/um/nommu/include/uapi/asm/Kbuild         |   4 +
 arch/um/nommu/include/uapi/asm/bitsperlong.h  |  11 ++
 arch/um/nommu/include/uapi/asm/byteorder.h    |  11 ++
 arch/um/nommu/include/uapi/asm/sigcontext.h   |  12 ++
 arch/um/nommu/um/delay.c                      |  31 +++++
 arch/um/nommu/um/shared/sysdep/archsetjmp.h   |  13 ++
 arch/um/nommu/um/shared/sysdep/faultinfo.h    |   8 ++
 .../nommu/um/shared/sysdep/kernel-offsets.h   |  12 ++
 arch/um/nommu/um/shared/sysdep/mcontext.h     |   9 ++
 arch/um/nommu/um/shared/sysdep/ptrace.h       |  42 +++++++
 arch/um/nommu/um/shared/sysdep/ptrace_user.h  |   7 ++
 arch/um/nommu/um/unimplemented.c              |  70 +++++++++++
 arch/um/nommu/um/user_constants.h             |  13 ++
 28 files changed, 548 insertions(+)
 create mode 100644 arch/um/nommu/Makefile
 create mode 100644 arch/um/nommu/Makefile.um
 create mode 100644 arch/um/nommu/include/asm/Kbuild
 create mode 100644 arch/um/nommu/include/asm/archparam.h
 create mode 100644 arch/um/nommu/include/asm/atomic.h
 create mode 100644 arch/um/nommu/include/asm/atomic64.h
 create mode 100644 arch/um/nommu/include/asm/bitsperlong.h
 create mode 100644 arch/um/nommu/include/asm/byteorder.h
 create mode 100644 arch/um/nommu/include/asm/cpu.h
 create mode 100644 arch/um/nommu/include/asm/elf.h
 create mode 100644 arch/um/nommu/include/asm/mm_context.h
 create mode 100644 arch/um/nommu/include/asm/processor.h
 create mode 100644 arch/um/nommu/include/asm/ptrace.h
 create mode 100644 arch/um/nommu/include/asm/sched.h
 create mode 100644 arch/um/nommu/include/asm/segment.h
 create mode 100644 arch/um/nommu/include/uapi/asm/Kbuild
 create mode 100644 arch/um/nommu/include/uapi/asm/bitsperlong.h
 create mode 100644 arch/um/nommu/include/uapi/asm/byteorder.h
 create mode 100644 arch/um/nommu/include/uapi/asm/sigcontext.h
 create mode 100644 arch/um/nommu/um/delay.c
 create mode 100644 arch/um/nommu/um/shared/sysdep/archsetjmp.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/faultinfo.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/kernel-offsets.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/mcontext.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/ptrace.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/ptrace_user.h
 create mode 100644 arch/um/nommu/um/unimplemented.c
 create mode 100644 arch/um/nommu/um/user_constants.h

diff --git a/arch/um/nommu/Makefile b/arch/um/nommu/Makefile
new file mode 100644
index 000000000000..792d6005489e
--- /dev/null
+++ b/arch/um/nommu/Makefile
@@ -0,0 +1 @@
+#
diff --git a/arch/um/nommu/Makefile.um b/arch/um/nommu/Makefile.um
new file mode 100644
index 000000000000..6cb0e9494d05
--- /dev/null
+++ b/arch/um/nommu/Makefile.um
@@ -0,0 +1,18 @@
+KBUILD_CFLAGS += -fno-builtin -fPIC
+ELF_FORMAT=$(shell $(LD) -r -print-output-format)
+
+ifeq ($(shell uname -s), Linux)
+NPROC=$(shell nproc)
+else # e.g., FreeBSD
+NPROC=$(shell sysctl -n hw.ncpu)
+endif
+
+um_headers_install: $(objtree)/$(HOST_DIR)/include/generated/uapi/asm/syscall_defs.h headers
+	$(Q)$(srctree)/$(ARCH_DIR)/scripts/headers_install.py \
+		$(subst -j,-j$(NPROC),$(findstring -j,$(MAKEFLAGS))) \
+		$(INSTALL_PATH)/include
+
+$(objtree)/$(HOST_DIR)/include/generated/uapi/asm/syscall_defs.h: vmlinux
+	$(Q)$(OBJCOPY) -j .syscall_defs -O binary --set-section-flags .syscall_defs=alloc $< $@
+	$(Q) export tmpfile=$(shell mktemp); \
+	sed 's/\x0//g' $@ > $$tmpfile; mv $$tmpfile $@ ; rm -f $$tmpfile
diff --git a/arch/um/nommu/include/asm/Kbuild b/arch/um/nommu/include/asm/Kbuild
new file mode 100644
index 000000000000..2532e1a0a0d1
--- /dev/null
+++ b/arch/um/nommu/include/asm/Kbuild
@@ -0,0 +1,6 @@
+generic-y += cmpxchg.h
+generic-y += local64.h
+generic-y += seccomp.h
+generic-y += string.h
+generic-y += syscall.h
+generic-y += user.h
diff --git a/arch/um/nommu/include/asm/archparam.h b/arch/um/nommu/include/asm/archparam.h
new file mode 100644
index 000000000000..ea32a7d3cf1b
--- /dev/null
+++ b/arch/um/nommu/include/asm/archparam.h
@@ -0,0 +1 @@
+/* SPDX-License-Identifier: GPL-2.0 */
diff --git a/arch/um/nommu/include/asm/atomic.h b/arch/um/nommu/include/asm/atomic.h
new file mode 100644
index 000000000000..63e2e16bda92
--- /dev/null
+++ b/arch/um/nommu/include/asm/atomic.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_ATOMIC_H
+#define __UM_NOMMU_ATOMIC_H
+
+#include <asm-generic/atomic.h>
+
+#ifndef CONFIG_GENERIC_ATOMIC64
+#include "atomic64.h"
+#endif /* !CONFIG_GENERIC_ATOMIC64 */
+
+#endif
diff --git a/arch/um/nommu/include/asm/atomic64.h b/arch/um/nommu/include/asm/atomic64.h
new file mode 100644
index 000000000000..949360dea7af
--- /dev/null
+++ b/arch/um/nommu/include/asm/atomic64.h
@@ -0,0 +1,114 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_ATOMIC64_H
+#define __UM_NOMMU_ATOMIC64_H
+
+#include <linux/types.h>
+
+#ifdef CONFIG_SMP
+#error "SMP is not supported on this platform"
+#else
+#define ATOMIC64_OP(op, c_op)					\
+	static inline void atomic64_##op(s64 i, atomic64_t *v)	\
+	{							\
+		unsigned long flags;				\
+								\
+		raw_local_irq_save(flags);			\
+		v->counter = v->counter c_op i;			\
+		raw_local_irq_restore(flags);			\
+	}
+
+#define ATOMIC64_OP_RETURN(op, c_op)					\
+	static inline s64 atomic64_##op##_return(s64 i, atomic64_t *v)	\
+	{								\
+		unsigned long flags;					\
+		s64 ret;						\
+									\
+		raw_local_irq_save(flags);				\
+		ret = (v->counter = v->counter c_op i);			\
+		raw_local_irq_restore(flags);				\
+									\
+		return ret;						\
+	}
+
+#define ATOMIC64_FETCH_OP(op, c_op)					\
+	static inline s64 atomic64_fetch_##op(s64 i, atomic64_t *v)	\
+	{								\
+		unsigned long flags;					\
+		s64 ret;						\
+									\
+		raw_local_irq_save(flags);				\
+		ret = v->counter;					\
+		v->counter = v->counter c_op i;				\
+		raw_local_irq_restore(flags);				\
+									\
+		return ret;						\
+	}
+#endif /* CONFIG_SMP */
+
+#ifndef atomic64_add_return
+ATOMIC64_OP_RETURN(add, +)
+#endif
+
+#ifndef atomic64_sub_return
+	ATOMIC64_OP_RETURN(sub, -)
+#endif
+
+#ifndef atomic64_fetch_add
+	ATOMIC64_FETCH_OP(add, +)
+#endif
+
+#ifndef atomic64_fetch_sub
+	ATOMIC64_FETCH_OP(sub, -)
+#endif
+
+#ifndef atomic64_fetch_and
+	ATOMIC64_FETCH_OP(and, &)
+#endif
+
+#ifndef atomic64_fetch_or
+	ATOMIC64_FETCH_OP(or, |)
+#endif
+
+#ifndef atomic64_fetch_xor
+	ATOMIC64_FETCH_OP(xor, ^)
+#endif
+
+#ifndef atomic64_and
+	ATOMIC64_OP(and, &)
+#endif
+
+#ifndef atomic64_or
+	ATOMIC64_OP(or, |)
+#endif
+
+#ifndef atomic64_xor
+	ATOMIC64_OP(xor, ^)
+#endif
+
+#undef ATOMIC64_FETCH_OP
+#undef ATOMIC64_OP_RETURN
+#undef ATOMIC64_OP
+
+
+#define ATOMIC64_INIT(i)       { (i) }
+
+	static inline void atomic64_add(s64 i, atomic64_t *v)
+{
+	atomic64_add_return(i, v);
+}
+
+static inline void atomic64_sub(s64 i, atomic64_t *v)
+{
+	atomic64_sub_return(i, v);
+}
+
+#ifndef atomic64_read
+#define atomic64_read(v)       READ_ONCE((v)->counter)
+#endif
+
+#define atomic64_set(v, i) WRITE_ONCE(((v)->counter), (i))
+
+#define atomic64_xchg(ptr, v)          (xchg(&(ptr)->counter, (v)))
+#define atomic64_cmpxchg(v, old, new)  (cmpxchg(&((v)->counter), (old), (new)))
+
+#endif /* __LKL_ATOMIC64_H */
diff --git a/arch/um/nommu/include/asm/bitsperlong.h b/arch/um/nommu/include/asm/bitsperlong.h
new file mode 100644
index 000000000000..a150cee41e75
--- /dev/null
+++ b/arch/um/nommu/include/asm/bitsperlong.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_BITSPERLONG_H
+#define __UM_NOMMU_BITSPERLONG_H
+
+#include <uapi/asm/bitsperlong.h>
+
+#define BITS_PER_LONG __BITS_PER_LONG
+
+#define BITS_PER_LONG_LONG 64
+
+#endif
+
diff --git a/arch/um/nommu/include/asm/byteorder.h b/arch/um/nommu/include/asm/byteorder.h
new file mode 100644
index 000000000000..920a5fd26cad
--- /dev/null
+++ b/arch/um/nommu/include/asm/byteorder.h
@@ -0,0 +1,7 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_BYTEORDER_H
+#define __UM_NOMMU_BYTEORDER_H
+
+#include <uapi/asm/byteorder.h>
+
+#endif
diff --git a/arch/um/nommu/include/asm/cpu.h b/arch/um/nommu/include/asm/cpu.h
new file mode 100644
index 000000000000..c101c078ef21
--- /dev/null
+++ b/arch/um/nommu/include/asm/cpu.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_CPU_H
+#define __UM_NOMMU_CPU_H
+
+int lkl_cpu_get(void);
+void lkl_cpu_put(void);
+int lkl_cpu_try_run_irq(int irq);
+int lkl_cpu_init(void);
+void lkl_cpu_wait_shutdown(void);
+void lkl_cpu_change_owner(lkl_thread_t owner);
+void lkl_cpu_set_irqs_pending(void);
+
+#endif
diff --git a/arch/um/nommu/include/asm/elf.h b/arch/um/nommu/include/asm/elf.h
new file mode 100644
index 000000000000..edcf63edeed1
--- /dev/null
+++ b/arch/um/nommu/include/asm/elf.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_ELF_H
+#define __UM_NOMMU_ELF_H
+
+#define elf_check_arch(x) 0
+
+#ifdef CONFIG_64BIT
+#define ELF_CLASS ELFCLASS64
+#else
+#define ELF_CLASS ELFCLASS32
+#endif
+
+#define elf_gregset_t long
+#define elf_fpregset_t double
+#endif
diff --git a/arch/um/nommu/include/asm/mm_context.h b/arch/um/nommu/include/asm/mm_context.h
new file mode 100644
index 000000000000..a2e53984aabd
--- /dev/null
+++ b/arch/um/nommu/include/asm/mm_context.h
@@ -0,0 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_MM_CONTEXT_H
+#define __UM_NOMMU_MM_CONTEXT_H
+
+struct uml_arch_mm_context {
+};
+
+#endif
diff --git a/arch/um/nommu/include/asm/processor.h b/arch/um/nommu/include/asm/processor.h
new file mode 100644
index 000000000000..3e8ba870caaf
--- /dev/null
+++ b/arch/um/nommu/include/asm/processor.h
@@ -0,0 +1,46 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_PROCESSOR_H
+#define __UM_NOMMU_PROCESSOR_H
+
+#include <asm/host_ops.h>
+
+struct arch_thread {
+	struct lkl_sem *sched_sem;
+	bool dead;
+	lkl_thread_t tid;
+	struct lkl_jmp_buf sched_jb;
+	unsigned long stackend;
+};
+
+#include <asm/ptrace-generic.h>
+#include <asm/processor-generic.h>
+
+#define INIT_ARCH_THREAD {}
+#define task_pt_regs(tsk) (struct pt_regs *)(NULL)
+
+static inline void cpu_relax(void)
+{
+	unsigned long flags;
+
+	/* since this is usually called in a tight loop waiting for some
+	 * external condition (e.g. jiffies) lets run interrupts now to allow
+	 * the external condition to propagate
+	 */
+	local_irq_save(flags);
+	local_irq_restore(flags);
+}
+
+#define KSTK_EIP(tsk)	(0)
+#define KSTK_ESP(tsk)	(0)
+
+static inline void trap_init(void)
+{
+}
+
+static inline void arch_copy_thread(struct arch_thread *from,
+				    struct arch_thread *to)
+{
+	panic("unimplemented %s: fork isn't supported yet", __func__);
+}
+
+#endif
diff --git a/arch/um/nommu/include/asm/ptrace.h b/arch/um/nommu/include/asm/ptrace.h
new file mode 100644
index 000000000000..b214410e9825
--- /dev/null
+++ b/arch/um/nommu/include/asm/ptrace.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_PTRACE_H
+#define __UM_NOMMU_PTRACE_H
+
+#include <linux/errno.h>
+
+static int reg_dummy __attribute__((unused));
+
+#define PT_REGS_ORIG_SYSCALL(r) (reg_dummy)
+#define PT_REGS_SYSCALL_RET(r) (reg_dummy)
+#define PT_REGS_SET_SYSCALL_RETURN(r, res) (reg_dummy = (res))
+#define REGS_SP(r) (reg_dummy)
+
+#define user_mode(regs) 0
+#define kernel_mode(regs) 1
+#define profile_pc(regs) 0
+#define user_stack_pointer(regs) 0
+
+extern void new_thread_handler(void);
+
+#endif
diff --git a/arch/um/nommu/include/asm/sched.h b/arch/um/nommu/include/asm/sched.h
new file mode 100644
index 000000000000..a4496f482633
--- /dev/null
+++ b/arch/um/nommu/include/asm/sched.h
@@ -0,0 +1,23 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_SCHED_H
+#define __UM_NOMMU_SCHED_H
+
+#include <linux/sched.h>
+#include <uapi/asm/host_ops.h>
+
+static inline void thread_sched_jb(void)
+{
+	if (test_ti_thread_flag(current_thread_info(), TIF_HOST_THREAD)) {
+		set_ti_thread_flag(current_thread_info(), TIF_SCHED_JB);
+		set_current_state(TASK_UNINTERRUPTIBLE);
+		lkl_ops->jmp_buf_set(&current_thread_info()->task->thread.arch.sched_jb,
+				     schedule);
+	} else {
+		lkl_bug("%s can be used only for host task", __func__);
+	}
+}
+
+void switch_to_host_task(struct task_struct *);
+int host_task_stub(void *unused);
+
+#endif
diff --git a/arch/um/nommu/include/asm/segment.h b/arch/um/nommu/include/asm/segment.h
new file mode 100644
index 000000000000..5608da95cb60
--- /dev/null
+++ b/arch/um/nommu/include/asm/segment.h
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_SEGMENT_H
+#define __UM_NOMMU_SEGMENT_H
+
+typedef struct {
+	unsigned long seg;
+} mm_segment_t;
+
+#endif /* _ASM_LKL_SEGMENT_H */
diff --git a/arch/um/nommu/include/uapi/asm/Kbuild b/arch/um/nommu/include/uapi/asm/Kbuild
new file mode 100644
index 000000000000..4f79e5c4846d
--- /dev/null
+++ b/arch/um/nommu/include/uapi/asm/Kbuild
@@ -0,0 +1,4 @@
+# UAPI Header export list
+
+# no generated-y since we need special user headers handling
+# see arch/um/script/headers_install.py
diff --git a/arch/um/nommu/include/uapi/asm/bitsperlong.h b/arch/um/nommu/include/uapi/asm/bitsperlong.h
new file mode 100644
index 000000000000..852566ac2e52
--- /dev/null
+++ b/arch/um/nommu/include/uapi/asm/bitsperlong.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef __UM_NOMMU_UAPI_BITSPERLONG_H
+#define __UM_NOMMU_UAPI_BITSPERLONG_H
+
+#ifdef CONFIG_64BIT
+#define __BITS_PER_LONG 64
+#else
+#define __BITS_PER_LONG 32
+#endif
+
+#endif
diff --git a/arch/um/nommu/include/uapi/asm/byteorder.h b/arch/um/nommu/include/uapi/asm/byteorder.h
new file mode 100644
index 000000000000..e7ad11a751cf
--- /dev/null
+++ b/arch/um/nommu/include/uapi/asm/byteorder.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef __UM_NOMMU_UAPI_BYTEORDER_H
+#define __UM_NOMMU_UAPI_BYTEORDER_H
+
+#if defined(CONFIG_BIG_ENDIAN)
+#include <linux/byteorder/big_endian.h>
+#else
+#include <linux/byteorder/little_endian.h>
+#endif
+
+#endif
diff --git a/arch/um/nommu/include/uapi/asm/sigcontext.h b/arch/um/nommu/include/uapi/asm/sigcontext.h
new file mode 100644
index 000000000000..b934ae7f5550
--- /dev/null
+++ b/arch/um/nommu/include/uapi/asm/sigcontext.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef __UM_NOMMU_UAPI_SIGCONTEXT_H
+#define __UM_NOMMU_UAPI_SIGCONTEXT_H
+
+#include <asm/ptrace-generic.h>
+
+struct sigcontext {
+	struct pt_regs regs;
+	unsigned long oldmask;
+};
+
+#endif
diff --git a/arch/um/nommu/um/delay.c b/arch/um/nommu/um/delay.c
new file mode 100644
index 000000000000..58a366d9b5f0
--- /dev/null
+++ b/arch/um/nommu/um/delay.c
@@ -0,0 +1,31 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/jiffies.h>
+#include <linux/delay.h>
+#include <os.h>
+
+void __ndelay(unsigned long nsecs)
+{
+	long long start = os_nsecs();
+
+	while (os_nsecs() < start + nsecs)
+		;
+}
+
+void __udelay(unsigned long usecs)
+{
+	__ndelay(usecs * NSEC_PER_USEC);
+}
+
+void __const_udelay(unsigned long xloops)
+{
+	__udelay(xloops / 0x10c7ul);
+}
+
+void __delay(unsigned long loops)
+{
+	__ndelay(loops / 5);
+}
+
+void calibrate_delay(void)
+{
+}
diff --git a/arch/um/nommu/um/shared/sysdep/archsetjmp.h b/arch/um/nommu/um/shared/sysdep/archsetjmp.h
new file mode 100644
index 000000000000..4a5c10d7521b
--- /dev/null
+++ b/arch/um/nommu/um/shared/sysdep/archsetjmp.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ARCH_UM_SETJMP_H
+#define __ARCH_UM_SETJMP_H
+
+struct __jmp_buf {
+	unsigned long __dummy;
+};
+#define JB_IP __dummy
+#define JB_SP __dummy
+
+typedef struct __jmp_buf jmp_buf[1];
+
+#endif /* __ARCH_UM_SETJMP_H */
diff --git a/arch/um/nommu/um/shared/sysdep/faultinfo.h b/arch/um/nommu/um/shared/sysdep/faultinfo.h
new file mode 100644
index 000000000000..49210742212d
--- /dev/null
+++ b/arch/um/nommu/um/shared/sysdep/faultinfo.h
@@ -0,0 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ARCH_UM_FAULTINFO_H
+#define __ARCH_UM_FAULTINFO_H
+
+struct faultinfo {
+};
+
+#endif /* __ARCH_UM_FAULTINFO_H */
diff --git a/arch/um/nommu/um/shared/sysdep/kernel-offsets.h b/arch/um/nommu/um/shared/sysdep/kernel-offsets.h
new file mode 100644
index 000000000000..a004bffb7b8d
--- /dev/null
+++ b/arch/um/nommu/um/shared/sysdep/kernel-offsets.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <linux/stddef.h>
+#include <linux/sched.h>
+#include <linux/elf.h>
+#include <linux/crypto.h>
+#include <linux/kbuild.h>
+#include <asm/mman.h>
+
+void foo(void)
+{
+#include <common-offsets.h>
+}
diff --git a/arch/um/nommu/um/shared/sysdep/mcontext.h b/arch/um/nommu/um/shared/sysdep/mcontext.h
new file mode 100644
index 000000000000..b734012a74da
--- /dev/null
+++ b/arch/um/nommu/um/shared/sysdep/mcontext.h
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ARCH_UM_MCONTEXT_H
+#define __ARCH_UM_MCONTEXT_H
+
+extern void get_regs_from_mc(struct uml_pt_regs *regs, mcontext_t *mc);
+
+#define GET_FAULTINFO_FROM_MC(fi, mc) (fi = fi)
+
+#endif /* __ARCH_UM_MCONTEXT_H */
diff --git a/arch/um/nommu/um/shared/sysdep/ptrace.h b/arch/um/nommu/um/shared/sysdep/ptrace.h
new file mode 100644
index 000000000000..bfdfb520a21d
--- /dev/null
+++ b/arch/um/nommu/um/shared/sysdep/ptrace.h
@@ -0,0 +1,42 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ARCH_UM_PTRACE_H
+#define __ARCH_UM_PTRACE_H
+
+#include <generated/user_constants.h>
+#include <linux/errno.h>
+
+struct task_struct;
+
+#define UPT_SYSCALL_NR(r) ((r)->syscall)
+#define UPT_RESTART_SYSCALL(r) ((r)->syscall--) /* XXX */
+
+#define UPT_SP(r) 0
+#define UPT_IP(r) 0
+#define EMPTY_UML_PT_REGS { }
+
+#define MAX_REG_OFFSET (UM_FRAME_SIZE)
+#define MAX_REG_NR ((MAX_REG_OFFSET) / sizeof(unsigned long))
+
+/* unused */
+struct uml_pt_regs {
+	unsigned long gp[1];
+	unsigned long fp[1];
+	long faultinfo;
+	long syscall;
+	int is_user;
+};
+
+extern void arch_init_registers(int pid);
+
+static inline long arch_ptrace(struct task_struct *child,
+			       long request, unsigned long addr,
+			       unsigned long data)
+{
+	return -EINVAL;
+}
+
+static inline void ptrace_disable(struct task_struct *child) {}
+static inline void user_enable_single_step(struct task_struct *child) {}
+static inline void user_disable_single_step(struct task_struct *child) {}
+
+#endif /* __ARCH_UM_PTRACE_H */
diff --git a/arch/um/nommu/um/shared/sysdep/ptrace_user.h b/arch/um/nommu/um/shared/sysdep/ptrace_user.h
new file mode 100644
index 000000000000..86d5cb20fb9f
--- /dev/null
+++ b/arch/um/nommu/um/shared/sysdep/ptrace_user.h
@@ -0,0 +1,7 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ARCH_UM_PTRACE_USER_H
+#define __ARCH_UM_PTRACE_USER_H
+
+#define FP_SIZE 1
+
+#endif
diff --git a/arch/um/nommu/um/unimplemented.c b/arch/um/nommu/um/unimplemented.c
new file mode 100644
index 000000000000..fe33e02e39e5
--- /dev/null
+++ b/arch/um/nommu/um/unimplemented.c
@@ -0,0 +1,70 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/signal.h>
+#include <sysdep/ptrace.h>
+#include <asm/ptrace.h>
+
+/* physmem.c  */
+unsigned long high_physmem;
+
+/* x86/um/setjmp*.S  */
+void kernel_longjmp(void)
+{}
+void kernel_setjmp(void)
+{}
+
+/* trap.c */
+void relay_signal(int sig, struct siginfo *si, struct uml_pt_regs *regs)
+{}
+void bus_handler(int sig, struct siginfo *si, struct uml_pt_regs *regs)
+{}
+void segv_handler(int sig, struct siginfo *unused_si, struct uml_pt_regs *regs)
+{}
+void winch(int sig, struct siginfo *unused_si, struct uml_pt_regs *regs)
+{}
+
+/* tlb.c */
+void flush_tlb_kernel_vm(void)
+{}
+void force_flush_all(void)
+{}
+
+/* skas/process.c */
+void halt_skas(void)
+{}
+int is_skas_winch(int pid, int fd, void *data)
+{
+	return 0;
+}
+void reboot_skas(void)
+{}
+
+int __init start_uml(void)
+{
+	return 0;
+}
+
+/* exec.c */
+void flush_thread(void)
+{}
+
+/* x86/ptrace_64.c */
+int is_syscall(unsigned long addr)
+{
+	return 0;
+}
+
+
+/* x86/sysrq.c */
+void show_regs(struct pt_regs *regs)
+{}
+
+/* x86/signal.c */
+int setup_signal_stack_si(unsigned long stack_top, struct ksignal *ksig,
+			  struct pt_regs *regs, sigset_t *mask)
+{
+	return 0;
+}
+
+/* x86/bugs.c */
+void arch_check_bugs(void)
+{}
diff --git a/arch/um/nommu/um/user_constants.h b/arch/um/nommu/um/user_constants.h
new file mode 100644
index 000000000000..2245d3d24120
--- /dev/null
+++ b/arch/um/nommu/um/user_constants.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_USER_CONSTANTS_H
+#define __UM_NOMMU_USER_CONSTANTS_H
+
+/* XXX: put dummy values */
+#define UM_FRAME_SIZE 4
+#define HOST_FP_SIZE 1
+#define HOST_IP 1
+#define HOST_SP 2
+#define HOST_BP 3
+#define UM_NR_CPUS 1
+
+#endif
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v5 09/21] um: nommu: host interface
  2020-07-02 14:06     ` Hajime Tazaki
@ 2020-07-02 14:07       ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-07-02 14:07 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Hajime Tazaki

This patch introduces the host operations that define the interface
between the LKL and the host. These operations must be provided either
by a host library or by the application itself.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/include/asm/host_ops.h            |  9 ++++++++
 arch/um/nommu/include/uapi/asm/host_ops.h | 25 +++++++++++++++++++++++
 2 files changed, 34 insertions(+)
 create mode 100644 arch/um/include/asm/host_ops.h
 create mode 100644 arch/um/nommu/include/uapi/asm/host_ops.h

diff --git a/arch/um/include/asm/host_ops.h b/arch/um/include/asm/host_ops.h
new file mode 100644
index 000000000000..f52423cc4ced
--- /dev/null
+++ b/arch/um/include/asm/host_ops.h
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_HOST_OPS_H
+#define _ASM_LKL_HOST_OPS_H
+
+#include <uapi/asm/host_ops.h>
+
+extern struct lkl_host_operations *lkl_ops;
+
+#endif
diff --git a/arch/um/nommu/include/uapi/asm/host_ops.h b/arch/um/nommu/include/uapi/asm/host_ops.h
new file mode 100644
index 000000000000..d3dad11b459e
--- /dev/null
+++ b/arch/um/nommu/include/uapi/asm/host_ops.h
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef __UM_NOMMU_UAPI_HOST_OPS_H
+#define __UM_NOMMU_UAPI_HOST_OPS_H
+
+/* Defined in {posix,nt}-host.c */
+struct lkl_mutex;
+struct lkl_sem;
+typedef unsigned long lkl_thread_t;
+struct lkl_jmp_buf {
+	unsigned long buf[128];
+};
+
+/**
+ * lkl_host_operations - host operations used by the Linux kernel
+ *
+ * These operations must be provided by a host library or by the application
+ * itself.
+ *
+ */
+struct lkl_host_operations {
+};
+
+void lkl_bug(const char *fmt, ...);
+
+#endif
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v5 09/21] um: nommu: host interface
@ 2020-07-02 14:07       ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-07-02 14:07 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, linux-kernel-library, linux-arch,
	Hajime Tazaki, Akira Moroo

This patch introduces the host operations that define the interface
between the LKL and the host. These operations must be provided either
by a host library or by the application itself.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/include/asm/host_ops.h            |  9 ++++++++
 arch/um/nommu/include/uapi/asm/host_ops.h | 25 +++++++++++++++++++++++
 2 files changed, 34 insertions(+)
 create mode 100644 arch/um/include/asm/host_ops.h
 create mode 100644 arch/um/nommu/include/uapi/asm/host_ops.h

diff --git a/arch/um/include/asm/host_ops.h b/arch/um/include/asm/host_ops.h
new file mode 100644
index 000000000000..f52423cc4ced
--- /dev/null
+++ b/arch/um/include/asm/host_ops.h
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_HOST_OPS_H
+#define _ASM_LKL_HOST_OPS_H
+
+#include <uapi/asm/host_ops.h>
+
+extern struct lkl_host_operations *lkl_ops;
+
+#endif
diff --git a/arch/um/nommu/include/uapi/asm/host_ops.h b/arch/um/nommu/include/uapi/asm/host_ops.h
new file mode 100644
index 000000000000..d3dad11b459e
--- /dev/null
+++ b/arch/um/nommu/include/uapi/asm/host_ops.h
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef __UM_NOMMU_UAPI_HOST_OPS_H
+#define __UM_NOMMU_UAPI_HOST_OPS_H
+
+/* Defined in {posix,nt}-host.c */
+struct lkl_mutex;
+struct lkl_sem;
+typedef unsigned long lkl_thread_t;
+struct lkl_jmp_buf {
+	unsigned long buf[128];
+};
+
+/**
+ * lkl_host_operations - host operations used by the Linux kernel
+ *
+ * These operations must be provided by a host library or by the application
+ * itself.
+ *
+ */
+struct lkl_host_operations {
+};
+
+void lkl_bug(const char *fmt, ...);
+
+#endif
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v5 10/21] um: nommu: memory handling
  2020-07-02 14:06     ` Hajime Tazaki
@ 2020-07-02 14:07       ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-07-02 14:07 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Hajime Tazaki

nommu mode follows the way !CONFIG_MMU architecture of other linux
archs.  Ther is not much work left to do other than initializing the
boot allocator and providing the page and page table definitions.

The backstore memory is allocated via a host operation and the memory
size to be used is specified when the kernel is started, in the
lkl_start_kernel call.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/include/asm/mmu.h                 |  3 +
 arch/um/include/asm/mmu_context.h         |  8 +++
 arch/um/include/asm/page.h                | 15 ++++
 arch/um/include/asm/pgtable.h             | 27 +++++++
 arch/um/include/asm/uaccess.h             |  6 ++
 arch/um/nommu/include/uapi/asm/host_ops.h |  5 ++
 arch/um/nommu/um/bootmem.c                | 86 +++++++++++++++++++++++
 7 files changed, 150 insertions(+)
 create mode 100644 arch/um/nommu/um/bootmem.c

diff --git a/arch/um/include/asm/mmu.h b/arch/um/include/asm/mmu.h
index 5b072aba5b65..c06d6cb67dd7 100644
--- a/arch/um/include/asm/mmu.h
+++ b/arch/um/include/asm/mmu.h
@@ -13,6 +13,9 @@ typedef struct mm_context {
 	struct mm_id id;
 	struct uml_arch_mm_context arch;
 	struct page *stub_pages[2];
+#ifndef CONFIG_MMU
+	unsigned long		end_brk;
+#endif
 } mm_context_t;
 
 extern void __switch_mm(struct mm_id * mm_idp);
diff --git a/arch/um/include/asm/mmu_context.h b/arch/um/include/asm/mmu_context.h
index b4deb1bfbb68..92cc838bb477 100644
--- a/arch/um/include/asm/mmu_context.h
+++ b/arch/um/include/asm/mmu_context.h
@@ -6,6 +6,7 @@
 #ifndef __UM_MMU_CONTEXT_H
 #define __UM_MMU_CONTEXT_H
 
+#ifdef CONFIG_MMU
 #include <linux/sched.h>
 #include <linux/mm_types.h>
 
@@ -74,4 +75,11 @@ extern int init_new_context(struct task_struct *task, struct mm_struct *mm);
 
 extern void destroy_context(struct mm_struct *mm);
 
+#else
+#include <asm-generic/mmu_context.h>
+
+extern void force_flush_all(void);
+
+#endif /* CONFIG_MMU */
+
 #endif
diff --git a/arch/um/include/asm/page.h b/arch/um/include/asm/page.h
index 95af12e82a32..c9d5e487ac6b 100644
--- a/arch/um/include/asm/page.h
+++ b/arch/um/include/asm/page.h
@@ -7,6 +7,8 @@
 #ifndef __UM_PAGE_H
 #define __UM_PAGE_H
 
+#ifdef CONFIG_MMU
+
 #include <linux/const.h>
 
 /* PAGE_SHIFT determines the page size */
@@ -120,4 +122,17 @@ extern unsigned long uml_physmem;
 #define __HAVE_ARCH_GATE_AREA 1
 #endif
 
+#else  /* CONFIG_MMU */
+#define CONFIG_KERNEL_RAM_BASE_ADDRESS memory_start
+#include <asm-generic/page.h>
+
+#define __va_space (8*1024*1024)
+
+#ifndef __ASSEMBLY__
+#include <mem.h>
+void free_mem(void);
+#endif
+
+#endif /* !CONFIG_MMU  */
+
 #endif	/* __UM_PAGE_H */
diff --git a/arch/um/include/asm/pgtable.h b/arch/um/include/asm/pgtable.h
index b5ddf5d98bd5..5c19f6a58ef7 100644
--- a/arch/um/include/asm/pgtable.h
+++ b/arch/um/include/asm/pgtable.h
@@ -21,6 +21,8 @@
 #define _PAGE_PROTNONE	0x010	/* if the user mapped it with PROT_NONE;
 				   pte_present gives true */
 
+#ifdef CONFIG_MMU
+
 #ifdef CONFIG_3_LEVEL_PGTABLES
 #include <asm/pgtable-3level.h>
 #else
@@ -362,4 +364,29 @@ do {						\
 	__flush_tlb_one((vaddr));		\
 } while (0)
 
+#else  /* CONFIG_MMU */
+
+#include <asm-generic/pgtable-nopud.h>
+#include <asm-generic/pgtable.h>
+
+#define swapper_pg_dir		((pgd_t *)0)
+#define PAGE_KERNEL             __pgprot(0)
+#define PGDIR_SHIFT		21
+#define PGDIR_SIZE		(1UL << PGDIR_SHIFT)
+#define PGDIR_MASK		(~(PGDIR_SIZE-1))
+
+#define VMALLOC_START 0
+
+/*
+ * ZERO_PAGE is a global shared page that is always zero: used
+ * for zero-mapped memory areas etc..
+ */
+extern unsigned long *empty_zero_page;
+#define ZERO_PAGE(vaddr)	(virt_to_page(empty_zero_page))
+
+extern unsigned long end_iomem;
+
+#endif /* !CONFIG_MMU */
+
+
 #endif
diff --git a/arch/um/include/asm/uaccess.h b/arch/um/include/asm/uaccess.h
index fe66d659acad..95db8f06d295 100644
--- a/arch/um/include/asm/uaccess.h
+++ b/arch/um/include/asm/uaccess.h
@@ -21,6 +21,7 @@
 #define __addr_range_nowrap(addr, size) \
 	((unsigned long) (addr) <= ((unsigned long) (addr) + (size)))
 
+#ifdef CONFIG_MMU
 extern unsigned long raw_copy_from_user(void *to, const void __user *from, unsigned long n);
 extern unsigned long raw_copy_to_user(void __user *to, const void *from, unsigned long n);
 extern long __strncpy_from_user(char *dst, const char __user *src, long count);
@@ -46,4 +47,9 @@ static inline int __access_ok(unsigned long addr, unsigned long size)
 		uaccess_kernel());
 }
 
+#else
+#include <asm-generic/uaccess.h>
+#endif /* CONFIG_MMU */
+
+
 #endif
diff --git a/arch/um/nommu/include/uapi/asm/host_ops.h b/arch/um/nommu/include/uapi/asm/host_ops.h
index d3dad11b459e..5253c3f8de0e 100644
--- a/arch/um/nommu/include/uapi/asm/host_ops.h
+++ b/arch/um/nommu/include/uapi/asm/host_ops.h
@@ -16,8 +16,13 @@ struct lkl_jmp_buf {
  * These operations must be provided by a host library or by the application
  * itself.
  *
+ * @mem_alloc - allocate memory
+ * @mem_free - free memory
+ *
  */
 struct lkl_host_operations {
+	void *(*mem_alloc)(unsigned long mem);
+	void (*mem_free)(void *mem);
 };
 
 void lkl_bug(const char *fmt, ...);
diff --git a/arch/um/nommu/um/bootmem.c b/arch/um/nommu/um/bootmem.c
new file mode 100644
index 000000000000..84ce15402f19
--- /dev/null
+++ b/arch/um/nommu/um/bootmem.c
@@ -0,0 +1,86 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/memblock.h>
+#include <linux/mm.h>
+#include <linux/swap.h>
+
+unsigned long memory_start, memory_end;
+static unsigned long _memory_start, mem_size;
+
+unsigned long *empty_zero_page;
+
+/* XXX: unused */
+unsigned long long highmem;
+int iomem_size;
+int kmalloc_ok = 1;
+
+void __init setup_physmem(unsigned long start, unsigned long reserve_end,
+			  unsigned long mem_sz, unsigned long long _highmem)
+{
+	mem_size = mem_sz;
+
+	_memory_start = (unsigned long)lkl_ops->mem_alloc(mem_size);
+	memory_start = _memory_start;
+	WARN_ON(!memory_start);
+	memory_end = memory_start + mem_size;
+
+	if (PAGE_ALIGN(memory_start) != memory_start) {
+		mem_size -= PAGE_ALIGN(memory_start) - memory_start;
+		memory_start = PAGE_ALIGN(memory_start);
+		mem_size = (mem_size / PAGE_SIZE) * PAGE_SIZE;
+	}
+	pr_info("memblock address range: 0x%lx - 0x%lx\n", memory_start,
+		memory_start+mem_size);
+	/*
+	 * Give all the memory to the bootmap allocator, tell it to put the
+	 * boot mem_map at the start of memory.
+	 */
+	max_low_pfn = virt_to_pfn(memory_end);
+	min_low_pfn = virt_to_pfn(memory_start);
+	memblock_add(memory_start, mem_size);
+
+	empty_zero_page = memblock_alloc(PAGE_SIZE, PAGE_SIZE);
+	memset((void *)empty_zero_page, 0, PAGE_SIZE);
+
+	{
+		unsigned long zones_size[MAX_NR_ZONES] = {0, };
+
+		zones_size[ZONE_NORMAL] = (mem_size) >> PAGE_SHIFT;
+		free_area_init(zones_size);
+	}
+}
+
+void __init mem_init(void)
+{
+	max_mapnr = (((unsigned long)high_memory) - PAGE_OFFSET) >> PAGE_SHIFT;
+	/* this will put all memory onto the freelists */
+	totalram_pages_add(memblock_free_all());
+	pr_info("Memory available: %luk/%luk RAM\n",
+		(nr_free_pages() << PAGE_SHIFT) >> 10, mem_size >> 10);
+}
+
+/*
+ * In our case __init memory is not part of the page allocator so there is
+ * nothing to free.
+ */
+void free_initmem(void)
+{
+}
+
+void free_mem(void)
+{
+	lkl_ops->mem_free((void *)_memory_start);
+}
+
+void *uml_kmalloc(int size, int flags)
+{
+	return kmalloc(size, flags);
+}
+
+void __init mem_total_pages(unsigned long physmem, unsigned long iomem,
+		     unsigned long _highmem)
+{
+}
+
+void __init paging_init(void)
+{
+}
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v5 10/21] um: nommu: memory handling
@ 2020-07-02 14:07       ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-07-02 14:07 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, linux-kernel-library, linux-arch,
	Hajime Tazaki, Akira Moroo

nommu mode follows the way !CONFIG_MMU architecture of other linux
archs.  Ther is not much work left to do other than initializing the
boot allocator and providing the page and page table definitions.

The backstore memory is allocated via a host operation and the memory
size to be used is specified when the kernel is started, in the
lkl_start_kernel call.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/include/asm/mmu.h                 |  3 +
 arch/um/include/asm/mmu_context.h         |  8 +++
 arch/um/include/asm/page.h                | 15 ++++
 arch/um/include/asm/pgtable.h             | 27 +++++++
 arch/um/include/asm/uaccess.h             |  6 ++
 arch/um/nommu/include/uapi/asm/host_ops.h |  5 ++
 arch/um/nommu/um/bootmem.c                | 86 +++++++++++++++++++++++
 7 files changed, 150 insertions(+)
 create mode 100644 arch/um/nommu/um/bootmem.c

diff --git a/arch/um/include/asm/mmu.h b/arch/um/include/asm/mmu.h
index 5b072aba5b65..c06d6cb67dd7 100644
--- a/arch/um/include/asm/mmu.h
+++ b/arch/um/include/asm/mmu.h
@@ -13,6 +13,9 @@ typedef struct mm_context {
 	struct mm_id id;
 	struct uml_arch_mm_context arch;
 	struct page *stub_pages[2];
+#ifndef CONFIG_MMU
+	unsigned long		end_brk;
+#endif
 } mm_context_t;
 
 extern void __switch_mm(struct mm_id * mm_idp);
diff --git a/arch/um/include/asm/mmu_context.h b/arch/um/include/asm/mmu_context.h
index b4deb1bfbb68..92cc838bb477 100644
--- a/arch/um/include/asm/mmu_context.h
+++ b/arch/um/include/asm/mmu_context.h
@@ -6,6 +6,7 @@
 #ifndef __UM_MMU_CONTEXT_H
 #define __UM_MMU_CONTEXT_H
 
+#ifdef CONFIG_MMU
 #include <linux/sched.h>
 #include <linux/mm_types.h>
 
@@ -74,4 +75,11 @@ extern int init_new_context(struct task_struct *task, struct mm_struct *mm);
 
 extern void destroy_context(struct mm_struct *mm);
 
+#else
+#include <asm-generic/mmu_context.h>
+
+extern void force_flush_all(void);
+
+#endif /* CONFIG_MMU */
+
 #endif
diff --git a/arch/um/include/asm/page.h b/arch/um/include/asm/page.h
index 95af12e82a32..c9d5e487ac6b 100644
--- a/arch/um/include/asm/page.h
+++ b/arch/um/include/asm/page.h
@@ -7,6 +7,8 @@
 #ifndef __UM_PAGE_H
 #define __UM_PAGE_H
 
+#ifdef CONFIG_MMU
+
 #include <linux/const.h>
 
 /* PAGE_SHIFT determines the page size */
@@ -120,4 +122,17 @@ extern unsigned long uml_physmem;
 #define __HAVE_ARCH_GATE_AREA 1
 #endif
 
+#else  /* CONFIG_MMU */
+#define CONFIG_KERNEL_RAM_BASE_ADDRESS memory_start
+#include <asm-generic/page.h>
+
+#define __va_space (8*1024*1024)
+
+#ifndef __ASSEMBLY__
+#include <mem.h>
+void free_mem(void);
+#endif
+
+#endif /* !CONFIG_MMU  */
+
 #endif	/* __UM_PAGE_H */
diff --git a/arch/um/include/asm/pgtable.h b/arch/um/include/asm/pgtable.h
index b5ddf5d98bd5..5c19f6a58ef7 100644
--- a/arch/um/include/asm/pgtable.h
+++ b/arch/um/include/asm/pgtable.h
@@ -21,6 +21,8 @@
 #define _PAGE_PROTNONE	0x010	/* if the user mapped it with PROT_NONE;
 				   pte_present gives true */
 
+#ifdef CONFIG_MMU
+
 #ifdef CONFIG_3_LEVEL_PGTABLES
 #include <asm/pgtable-3level.h>
 #else
@@ -362,4 +364,29 @@ do {						\
 	__flush_tlb_one((vaddr));		\
 } while (0)
 
+#else  /* CONFIG_MMU */
+
+#include <asm-generic/pgtable-nopud.h>
+#include <asm-generic/pgtable.h>
+
+#define swapper_pg_dir		((pgd_t *)0)
+#define PAGE_KERNEL             __pgprot(0)
+#define PGDIR_SHIFT		21
+#define PGDIR_SIZE		(1UL << PGDIR_SHIFT)
+#define PGDIR_MASK		(~(PGDIR_SIZE-1))
+
+#define VMALLOC_START 0
+
+/*
+ * ZERO_PAGE is a global shared page that is always zero: used
+ * for zero-mapped memory areas etc..
+ */
+extern unsigned long *empty_zero_page;
+#define ZERO_PAGE(vaddr)	(virt_to_page(empty_zero_page))
+
+extern unsigned long end_iomem;
+
+#endif /* !CONFIG_MMU */
+
+
 #endif
diff --git a/arch/um/include/asm/uaccess.h b/arch/um/include/asm/uaccess.h
index fe66d659acad..95db8f06d295 100644
--- a/arch/um/include/asm/uaccess.h
+++ b/arch/um/include/asm/uaccess.h
@@ -21,6 +21,7 @@
 #define __addr_range_nowrap(addr, size) \
 	((unsigned long) (addr) <= ((unsigned long) (addr) + (size)))
 
+#ifdef CONFIG_MMU
 extern unsigned long raw_copy_from_user(void *to, const void __user *from, unsigned long n);
 extern unsigned long raw_copy_to_user(void __user *to, const void *from, unsigned long n);
 extern long __strncpy_from_user(char *dst, const char __user *src, long count);
@@ -46,4 +47,9 @@ static inline int __access_ok(unsigned long addr, unsigned long size)
 		uaccess_kernel());
 }
 
+#else
+#include <asm-generic/uaccess.h>
+#endif /* CONFIG_MMU */
+
+
 #endif
diff --git a/arch/um/nommu/include/uapi/asm/host_ops.h b/arch/um/nommu/include/uapi/asm/host_ops.h
index d3dad11b459e..5253c3f8de0e 100644
--- a/arch/um/nommu/include/uapi/asm/host_ops.h
+++ b/arch/um/nommu/include/uapi/asm/host_ops.h
@@ -16,8 +16,13 @@ struct lkl_jmp_buf {
  * These operations must be provided by a host library or by the application
  * itself.
  *
+ * @mem_alloc - allocate memory
+ * @mem_free - free memory
+ *
  */
 struct lkl_host_operations {
+	void *(*mem_alloc)(unsigned long mem);
+	void (*mem_free)(void *mem);
 };
 
 void lkl_bug(const char *fmt, ...);
diff --git a/arch/um/nommu/um/bootmem.c b/arch/um/nommu/um/bootmem.c
new file mode 100644
index 000000000000..84ce15402f19
--- /dev/null
+++ b/arch/um/nommu/um/bootmem.c
@@ -0,0 +1,86 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/memblock.h>
+#include <linux/mm.h>
+#include <linux/swap.h>
+
+unsigned long memory_start, memory_end;
+static unsigned long _memory_start, mem_size;
+
+unsigned long *empty_zero_page;
+
+/* XXX: unused */
+unsigned long long highmem;
+int iomem_size;
+int kmalloc_ok = 1;
+
+void __init setup_physmem(unsigned long start, unsigned long reserve_end,
+			  unsigned long mem_sz, unsigned long long _highmem)
+{
+	mem_size = mem_sz;
+
+	_memory_start = (unsigned long)lkl_ops->mem_alloc(mem_size);
+	memory_start = _memory_start;
+	WARN_ON(!memory_start);
+	memory_end = memory_start + mem_size;
+
+	if (PAGE_ALIGN(memory_start) != memory_start) {
+		mem_size -= PAGE_ALIGN(memory_start) - memory_start;
+		memory_start = PAGE_ALIGN(memory_start);
+		mem_size = (mem_size / PAGE_SIZE) * PAGE_SIZE;
+	}
+	pr_info("memblock address range: 0x%lx - 0x%lx\n", memory_start,
+		memory_start+mem_size);
+	/*
+	 * Give all the memory to the bootmap allocator, tell it to put the
+	 * boot mem_map at the start of memory.
+	 */
+	max_low_pfn = virt_to_pfn(memory_end);
+	min_low_pfn = virt_to_pfn(memory_start);
+	memblock_add(memory_start, mem_size);
+
+	empty_zero_page = memblock_alloc(PAGE_SIZE, PAGE_SIZE);
+	memset((void *)empty_zero_page, 0, PAGE_SIZE);
+
+	{
+		unsigned long zones_size[MAX_NR_ZONES] = {0, };
+
+		zones_size[ZONE_NORMAL] = (mem_size) >> PAGE_SHIFT;
+		free_area_init(zones_size);
+	}
+}
+
+void __init mem_init(void)
+{
+	max_mapnr = (((unsigned long)high_memory) - PAGE_OFFSET) >> PAGE_SHIFT;
+	/* this will put all memory onto the freelists */
+	totalram_pages_add(memblock_free_all());
+	pr_info("Memory available: %luk/%luk RAM\n",
+		(nr_free_pages() << PAGE_SHIFT) >> 10, mem_size >> 10);
+}
+
+/*
+ * In our case __init memory is not part of the page allocator so there is
+ * nothing to free.
+ */
+void free_initmem(void)
+{
+}
+
+void free_mem(void)
+{
+	lkl_ops->mem_free((void *)_memory_start);
+}
+
+void *uml_kmalloc(int size, int flags)
+{
+	return kmalloc(size, flags);
+}
+
+void __init mem_total_pages(unsigned long physmem, unsigned long iomem,
+		     unsigned long _highmem)
+{
+}
+
+void __init paging_init(void)
+{
+}
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v5 11/21] um: nommu: kernel thread support
  2020-07-02 14:06     ` Hajime Tazaki
@ 2020-07-02 14:07       ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-07-02 14:07 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Hajime Tazaki

nommu mode does not support user processes but it must support kernel
threads as part as the normal kernel work-flow. It uses host operations
to create and terminate host threads that are going to run the kernel
threads. It also uses semaphores to synchronize those threads and to
allow the Linux kernel scheduler to control how the kernel threads
run.

Each kernel thread runs in a host threads and has a host semaphore
associated with it - the thread's scheduling semaphore. The semaphore
counter is initialized to 0. The first thing a kernel thread does
after getting spawned, before running any kernel code, is to perform a
down operation to block the thread.

The kernel controls host threads scheduling by performing up and down
operations on the scheduling semaphore. In __switch_context an up
operation on the next thread is performed to wake up a blocked thread,
and a down operation is performed on the prev thread to block it.

A thread is terminated by marking it in free_thread_info and
performing an up operation on the scheduling semaphore at which point
the marked thread will terminate itself.

UML common part (process.c) is extended to call idle functions of
SUBARCH as well as kernel thread detection with a flag information of
TIF_HOST_THREAD.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/include/asm/thread_info.h         |  24 ++
 arch/um/kernel/process.c                  |   8 +-
 arch/um/nommu/include/uapi/asm/host_ops.h |  54 +++++
 arch/um/nommu/um/cpu.c                    | 236 ++++++++++++++++++++
 arch/um/nommu/um/threads.c                | 258 ++++++++++++++++++++++
 5 files changed, 579 insertions(+), 1 deletion(-)
 create mode 100644 arch/um/nommu/um/cpu.c
 create mode 100644 arch/um/nommu/um/threads.c

diff --git a/arch/um/include/asm/thread_info.h b/arch/um/include/asm/thread_info.h
index 4c19ce4c49f1..433584f6b9d0 100644
--- a/arch/um/include/asm/thread_info.h
+++ b/arch/um/include/asm/thread_info.h
@@ -40,6 +40,7 @@ struct thread_info {
 	.real_thread = NULL,			\
 }
 
+#ifdef CONFIG_MMU
 /* how to get the thread information struct from C */
 static inline struct thread_info *current_thread_info(void)
 {
@@ -52,6 +53,27 @@ static inline struct thread_info *current_thread_info(void)
 	return ti;
 }
 
+#else
+
+#define __HAVE_THREAD_FUNCTIONS
+#define task_thread_info(task)	((struct thread_info *)(task)->stack)
+#define task_stack_page(task)	((task)->stack)
+#define end_of_stack(p) (&task_thread_info(p)->aux_fp_regs[FP_SIZE-1])
+
+void threads_init(void);
+void threads_cleanup(void);
+
+unsigned long *alloc_thread_stack_node(struct task_struct *p, int node);
+void setup_thread_stack(struct task_struct *p, struct task_struct *org);
+void free_thread_stack(struct task_struct *tsk);
+
+extern struct thread_info *_current_thread_info;
+static inline struct thread_info *current_thread_info(void)
+{
+	return _current_thread_info;
+}
+#endif /* CONFIG_MMU */
+
 #endif
 
 #define TIF_SYSCALL_TRACE	0	/* syscall trace active */
@@ -63,6 +85,8 @@ static inline struct thread_info *current_thread_info(void)
 #define TIF_RESTORE_SIGMASK	7
 #define TIF_NOTIFY_RESUME	8
 #define TIF_SECCOMP		9	/* secure computing */
+#define TIF_SCHED_JB		10
+#define TIF_HOST_THREAD		11
 
 #define _TIF_SYSCALL_TRACE	(1 << TIF_SYSCALL_TRACE)
 #define _TIF_SIGPENDING		(1 << TIF_SIGPENDING)
diff --git a/arch/um/kernel/process.c b/arch/um/kernel/process.c
index e5238a42ea17..0dafded5947e 100644
--- a/arch/um/kernel/process.c
+++ b/arch/um/kernel/process.c
@@ -157,7 +157,8 @@ int copy_thread_tls(unsigned long clone_flags, unsigned long sp,
 		unsigned long arg, struct task_struct * p, unsigned long tls)
 {
 	void (*handler)(void);
-	int kthread = current->flags & PF_KTHREAD;
+	int kthread = current->flags & PF_KTHREAD ||
+		test_ti_thread_flag(current_thread_info(), TIF_HOST_THREAD);
 	int ret = 0;
 
 	p->thread = (struct thread_struct) INIT_THREAD;
@@ -214,10 +215,15 @@ static void um_idle_sleep(void)
 	}
 }
 
+void __weak subarch_cpu_idle(void)
+{
+}
+
 void arch_cpu_idle(void)
 {
 	cpu_tasks[current_thread_info()->cpu].pid = os_getpid();
 	um_idle_sleep();
+	subarch_cpu_idle();
 	local_irq_enable();
 }
 
diff --git a/arch/um/nommu/include/uapi/asm/host_ops.h b/arch/um/nommu/include/uapi/asm/host_ops.h
index 5253c3f8de0e..720385fccbdf 100644
--- a/arch/um/nommu/include/uapi/asm/host_ops.h
+++ b/arch/um/nommu/include/uapi/asm/host_ops.h
@@ -16,11 +16,65 @@ struct lkl_jmp_buf {
  * These operations must be provided by a host library or by the application
  * itself.
  *
+ * @sem_alloc - allocate a host semaphore an initialize it to count
+ * @sem_free - free a host semaphore
+ * @sem_up - perform an up operation on the semaphore
+ * @sem_down - perform a down operation on the semaphore
+ *
+ * @mutex_alloc - allocate and initialize a host mutex; the recursive parameter
+ * determines if the mutex is recursive or not
+ * @mutex_free - free a host mutex
+ * @mutex_lock - acquire the mutex
+ * @mutex_unlock - release the mutex
+ *
+ * @thread_create - create a new thread and run f(arg) in its context; returns a
+ * thread handle or 0 if the thread could not be created
+ * @thread_detach - on POSIX systems, free up resources held by
+ * pthreads. Noop on Win32.
+ * @thread_exit - terminates the current thread
+ * @thread_join - wait for the given thread to terminate. Returns 0
+ * for success, -1 otherwise
+ *
+ * @gettid - returns the host thread id of the caller, which need not
+ * be the same as the handle returned by thread_create
+ *
+ * @jmp_buf_set - runs the give function and setups a jump back point by saving
+ * the context in the jump buffer; jmp_buf_longjmp can be called from the give
+ * function or any callee in that function to return back to the jump back
+ * point
+ *
+ * NOTE: we can't return from jmp_buf_set before calling jmp_buf_longjmp or
+ * otherwise the saved context (stack) is not going to be valid, so we must pass
+ * the function that will eventually call longjmp here
+ *
+ * @jmp_buf_longjmp - perform a jump back to the saved jump buffer
+ *
  * @mem_alloc - allocate memory
  * @mem_free - free memory
  *
  */
 struct lkl_host_operations {
+	struct lkl_sem *(*sem_alloc)(int count);
+	void (*sem_free)(struct lkl_sem *sem);
+	void (*sem_up)(struct lkl_sem *sem);
+	void (*sem_down)(struct lkl_sem *sem);
+
+	struct lkl_mutex *(*mutex_alloc)(int recursive);
+	void (*mutex_free)(struct lkl_mutex *mutex);
+	void (*mutex_lock)(struct lkl_mutex *mutex);
+	void (*mutex_unlock)(struct lkl_mutex *mutex);
+
+	lkl_thread_t (*thread_create)(void *(*f)(void *), void *arg);
+	void (*thread_detach)(void);
+	void (*thread_exit)(void);
+	int (*thread_join)(lkl_thread_t tid);
+	lkl_thread_t (*thread_self)(void);
+	int (*thread_equal)(lkl_thread_t a, lkl_thread_t b);
+	long (*gettid)(void);
+
+	void (*jmp_buf_set)(struct lkl_jmp_buf *jmpb, void (*f)(void));
+	void (*jmp_buf_longjmp)(struct lkl_jmp_buf *jmpb, int val);
+
 	void *(*mem_alloc)(unsigned long mem);
 	void (*mem_free)(void *mem);
 };
diff --git a/arch/um/nommu/um/cpu.c b/arch/um/nommu/um/cpu.c
new file mode 100644
index 000000000000..9986a3f8c5dd
--- /dev/null
+++ b/arch/um/nommu/um/cpu.c
@@ -0,0 +1,236 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/kernel.h>
+#include <linux/sched/stat.h>
+#include <asm/host_ops.h>
+#include <asm/cpu.h>
+#include <asm/thread_info.h>
+#include <asm/unistd.h>
+#include <asm/sched.h>
+#include <asm/syscalls.h>
+#include <init.h>
+
+void run_irqs(void)
+{
+	panic("unimplemented %s", __func__);
+}
+
+int set_irq_pending(int irq)
+{
+	panic("unimplemented %s", __func__);
+}
+
+/*
+ * This structure is used to get access to the "LKL CPU" that allows us to run
+ * Linux code. Because we have to deal with various synchronization requirements
+ * between idle thread, system calls, interrupts, "reentrancy", CPU shutdown,
+ * imbalance wake up (i.e. acquire the CPU from one thread and release it from
+ * another), we can't use a simple synchronization mechanism such as (recursive)
+ * mutex or semaphore. Instead, we use a mutex and a bunch of status data plus a
+ * semaphore.
+ */
+static struct lkl_cpu {
+	/* lock that protects the CPU status data */
+	struct lkl_mutex *lock;
+	/*
+	 * Since we must free the cpu lock during shutdown we need a
+	 * synchronization algorithm between lkl_cpu_shutdown() and the CPU
+	 * access functions since lkl_cpu_get() gets called from thread
+	 * destructor callback functions which may be scheduled after
+	 * lkl_cpu_shutdown() has freed the cpu lock.
+	 *
+	 * An atomic counter is used to keep track of the number of running
+	 * CPU access functions and allow the shutdown function to wait for
+	 * them.
+	 *
+	 * The shutdown functions adds MAX_THREADS to this counter which allows
+	 * the CPU access functions to check if the shutdown process has
+	 * started.
+	 *
+	 * This algorithm assumes that we never have more the MAX_THREADS
+	 * requesting CPU access.
+	 */
+	#define MAX_THREADS 1000000
+	unsigned int shutdown_gate;
+	bool irqs_pending;
+	/* no of threads waiting the CPU */
+	unsigned int sleepers;
+	/* no of times the current thread got the CPU */
+	unsigned int count;
+	/* current thread that owns the CPU */
+	lkl_thread_t owner;
+	/* semaphore for threads waiting the CPU */
+	struct lkl_sem *sem;
+	/* semaphore used for shutdown */
+	struct lkl_sem *shutdown_sem;
+} cpu;
+
+static int __cpu_try_get_lock(int n)
+{
+	lkl_thread_t self;
+
+	if (__sync_fetch_and_add(&cpu.shutdown_gate, n) >= MAX_THREADS)
+		return -2;
+
+	lkl_ops->mutex_lock(cpu.lock);
+
+	if (cpu.shutdown_gate >= MAX_THREADS)
+		return -1;
+
+	self = lkl_ops->thread_self();
+
+	if (cpu.owner && !lkl_ops->thread_equal(cpu.owner, self))
+		return 0;
+
+	cpu.owner = self;
+	cpu.count++;
+
+	return 1;
+}
+
+static void __cpu_try_get_unlock(int lock_ret, int n)
+{
+	if (lock_ret >= -1)
+		lkl_ops->mutex_unlock(cpu.lock);
+	__sync_fetch_and_sub(&cpu.shutdown_gate, n);
+}
+
+void lkl_cpu_change_owner(lkl_thread_t owner)
+{
+	lkl_ops->mutex_lock(cpu.lock);
+	if (cpu.count > 1)
+		lkl_bug("bad count while changing owner\n");
+	cpu.owner = owner;
+	lkl_ops->mutex_unlock(cpu.lock);
+}
+
+int lkl_cpu_get(void)
+{
+	int ret;
+
+	ret = __cpu_try_get_lock(1);
+
+	while (ret == 0) {
+		cpu.sleepers++;
+		__cpu_try_get_unlock(ret, 0);
+		lkl_ops->sem_down(cpu.sem);
+		ret = __cpu_try_get_lock(0);
+	}
+
+	__cpu_try_get_unlock(ret, 1);
+
+	return ret;
+}
+
+void lkl_cpu_put(void)
+{
+	lkl_ops->mutex_lock(cpu.lock);
+
+	if (!cpu.count || !cpu.owner ||
+	    !lkl_ops->thread_equal(cpu.owner, lkl_ops->thread_self()))
+		lkl_bug("%s: unbalanced put\n", __func__);
+
+	while (cpu.irqs_pending && !irqs_disabled()) {
+		cpu.irqs_pending = false;
+		lkl_ops->mutex_unlock(cpu.lock);
+		run_irqs();
+		lkl_ops->mutex_lock(cpu.lock);
+	}
+
+	if (test_ti_thread_flag(current_thread_info(), TIF_HOST_THREAD) &&
+	    !single_task_running() && cpu.count == 1) {
+		if (in_interrupt())
+			lkl_bug("%s: in interrupt\n", __func__);
+		lkl_ops->mutex_unlock(cpu.lock);
+		thread_sched_jb();
+		return;
+	}
+
+	if (--cpu.count > 0) {
+		lkl_ops->mutex_unlock(cpu.lock);
+		return;
+	}
+
+	if (cpu.sleepers) {
+		cpu.sleepers--;
+		lkl_ops->sem_up(cpu.sem);
+	}
+
+	cpu.owner = 0;
+
+	lkl_ops->mutex_unlock(cpu.lock);
+}
+
+int lkl_cpu_try_run_irq(int irq)
+{
+	int ret;
+
+	ret = __cpu_try_get_lock(1);
+	if (!ret) {
+		set_irq_pending(irq);
+		cpu.irqs_pending = true;
+	}
+	__cpu_try_get_unlock(ret, 1);
+
+	return ret;
+}
+
+static void lkl_cpu_shutdown(void)
+{
+	__sync_fetch_and_add(&cpu.shutdown_gate, MAX_THREADS);
+}
+__uml_exitcall(lkl_cpu_shutdown);
+
+void lkl_cpu_wait_shutdown(void)
+{
+	lkl_ops->sem_down(cpu.shutdown_sem);
+	lkl_ops->sem_free(cpu.shutdown_sem);
+}
+
+static void lkl_cpu_cleanup(bool shutdown)
+{
+	while (__sync_fetch_and_add(&cpu.shutdown_gate, 0) > MAX_THREADS)
+		;
+
+	if (shutdown)
+		lkl_ops->sem_up(cpu.shutdown_sem);
+	else if (cpu.shutdown_sem)
+		lkl_ops->sem_free(cpu.shutdown_sem);
+	if (cpu.sem)
+		lkl_ops->sem_free(cpu.sem);
+	if (cpu.lock)
+		lkl_ops->mutex_free(cpu.lock);
+}
+
+void subarch_cpu_idle(void)
+{
+	if (cpu.shutdown_gate >= MAX_THREADS) {
+
+		lkl_ops->mutex_lock(cpu.lock);
+		while (cpu.sleepers--)
+			lkl_ops->sem_up(cpu.sem);
+		lkl_ops->mutex_unlock(cpu.lock);
+
+		lkl_cpu_cleanup(true);
+
+		lkl_ops->thread_exit();
+	}
+
+#ifdef doesntwork
+	/* switch to idle_host_task */
+	wakeup_idle_host_task();
+#endif
+}
+
+int lkl_cpu_init(void)
+{
+	cpu.lock = lkl_ops->mutex_alloc(0);
+	cpu.sem = lkl_ops->sem_alloc(0);
+	cpu.shutdown_sem = lkl_ops->sem_alloc(0);
+
+	if (!cpu.lock || !cpu.sem || !cpu.shutdown_sem) {
+		lkl_cpu_cleanup(false);
+		return -ENOMEM;
+	}
+
+	return 0;
+}
diff --git a/arch/um/nommu/um/threads.c b/arch/um/nommu/um/threads.c
new file mode 100644
index 000000000000..3e70eccc191a
--- /dev/null
+++ b/arch/um/nommu/um/threads.c
@@ -0,0 +1,258 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/slab.h>
+#include <linux/sched/task.h>
+#include <linux/sched/signal.h>
+#include <asm/host_ops.h>
+#include <asm/cpu.h>
+#include <asm/sched.h>
+
+#include <os.h>
+
+static int init_arch_thread(struct arch_thread *thread)
+{
+	thread->sched_sem = lkl_ops->sem_alloc(0);
+	if (!thread->sched_sem)
+		return -ENOMEM;
+
+	thread->dead = false;
+	thread->tid = 0;
+
+	return 0;
+}
+
+unsigned long *alloc_thread_stack_node(struct task_struct *task, int node)
+{
+	struct thread_info *ti;
+
+	ti = kmalloc(sizeof(*ti), GFP_KERNEL | __GFP_ZERO);
+	if (!ti)
+		return NULL;
+
+	ti->task = task;
+	return (unsigned long *)ti;
+}
+
+/*
+ * The only new tasks created are kernel threads that have a predefined starting
+ * point thus no stack copy is required.
+ */
+void setup_thread_stack(struct task_struct *p, struct task_struct *org)
+{
+	struct thread_info *ti = task_thread_info(p);
+	struct thread_info *org_ti = task_thread_info(org);
+
+	ti->flags = org_ti->flags;
+	ti->preempt_count = org_ti->preempt_count;
+	ti->addr_limit = org_ti->addr_limit;
+}
+
+static void kill_thread(struct thread_info *ti)
+{
+	if (!test_ti_thread_flag(ti, TIF_HOST_THREAD)) {
+		ti->task->thread.arch.dead = true;
+		lkl_ops->sem_up(ti->task->thread.arch.sched_sem);
+		lkl_ops->thread_join(ti->task->thread.arch.tid);
+	}
+	lkl_ops->sem_free(ti->task->thread.arch.sched_sem);
+}
+
+void free_thread_stack(struct task_struct *tsk)
+{
+	struct thread_info *ti = task_thread_info(tsk);
+
+	kill_thread(ti);
+	kfree(ti);
+}
+
+struct thread_info *_current_thread_info = &init_thread_union.thread_info;
+
+void switch_threads(jmp_buf *me, jmp_buf *you)
+{
+	/* NOP */
+}
+
+/*
+ * schedule() expects the return of this function to be the task that we
+ * switched away from. Returning prev is not going to work because we are
+ * actually going to return the previous taks that was scheduled before the
+ * task we are going to wake up, and not the current task, e.g.:
+ *
+ * swapper -> init: saved prev on swapper stack is swapper
+ * init -> ksoftirqd0: saved prev on init stack is init
+ * ksoftirqd0 -> swapper: returned prev is swapper
+ */
+static struct task_struct *abs_prev = &init_task;
+
+void arch_switch_to(struct task_struct *prev,
+		    struct task_struct *next)
+{
+	struct arch_thread *_prev = &prev->thread.arch;
+	struct arch_thread *_next = &next->thread.arch;
+	unsigned long _prev_flags = task_thread_info(prev)->flags;
+	struct lkl_jmp_buf *_prev_jb;
+
+	_current_thread_info = task_thread_info(next);
+	next->thread.prev_sched = prev;
+	abs_prev = prev;
+
+	WARN_ON(!_next->tid);
+	lkl_cpu_change_owner(_next->tid);
+
+	if (test_bit(TIF_SCHED_JB, &_prev_flags)) {
+		/* Atomic. Must be done before wakeup next */
+		clear_ti_thread_flag(task_thread_info(prev), TIF_SCHED_JB);
+		_prev_jb = &_prev->sched_jb;
+	}
+
+	lkl_ops->sem_up(_next->sched_sem);
+	if (test_bit(TIF_SCHED_JB, &_prev_flags))
+		lkl_ops->jmp_buf_longjmp(_prev_jb, 1);
+	else
+		lkl_ops->sem_down(_prev->sched_sem);
+
+	if (_prev->dead)
+		lkl_ops->thread_exit();
+
+	/* __switch_to (arch/um) returns this value */
+	current->thread.prev_sched = abs_prev;
+}
+
+int host_task_stub(void *unused)
+{
+	return 0;
+}
+
+void switch_to_host_task(struct task_struct *task)
+{
+	if (WARN_ON(!test_tsk_thread_flag(task, TIF_HOST_THREAD)))
+		return;
+
+	task->thread.arch.tid = lkl_ops->thread_self();
+
+	if (current == task)
+		return;
+
+	wake_up_process(task);
+	thread_sched_jb();
+	lkl_ops->sem_down(task->thread.arch.sched_sem);
+	schedule_tail(abs_prev);
+}
+
+struct thread_bootstrap_arg {
+	struct thread_info *ti;
+	int (*f)(void *a);
+	void *arg;
+};
+
+static void *thread_bootstrap(void *_tba)
+{
+	struct thread_bootstrap_arg *tba = (struct thread_bootstrap_arg *)_tba;
+	struct thread_info *ti = tba->ti;
+	int (*f)(void *) = tba->f;
+	void *arg = tba->arg;
+
+	lkl_ops->sem_down(ti->task->thread.arch.sched_sem);
+	kfree(tba);
+	if (ti->task->thread.prev_sched)
+		schedule_tail(ti->task->thread.prev_sched);
+
+	f(arg);
+	do_exit(0);
+
+	return NULL;
+}
+
+void new_thread(void *stack, jmp_buf *buf, void (*handler)(void))
+{
+	struct thread_info *ti = (struct thread_info *)stack;
+	struct task_struct *p = ti->task;
+	struct thread_bootstrap_arg *tba;
+	int ret;
+
+	unsigned long esp = (unsigned long)p->thread.request.u.thread.proc;
+	unsigned long unused = (unsigned long)p->thread.request.u.thread.arg;
+
+	ret = init_arch_thread(&p->thread.arch);
+	if (ret < 0)
+		panic("%s: init_arch_thread", __func__);
+
+	if ((int (*)(void *))esp == host_task_stub) {
+		set_ti_thread_flag(ti, TIF_HOST_THREAD);
+		return;
+	}
+
+	tba = kmalloc(sizeof(*tba), GFP_KERNEL);
+	if (!tba)
+		return;
+
+	tba->f = (int (*)(void *))esp;
+	tba->arg = (void *)unused;
+	tba->ti = ti;
+
+	p->thread.arch.tid = lkl_ops->thread_create(thread_bootstrap, tba);
+	if (!p->thread.arch.tid) {
+		kfree(tba);
+		return;
+	}
+}
+
+void show_stack(struct task_struct *task, unsigned long *esp)
+{
+}
+
+static inline void pr_early(const char *str)
+{
+	if (lkl_ops->print)
+		lkl_ops->print(str, strlen(str));
+}
+
+/**
+ * This is called before the kernel initializes, so no kernel calls (including
+ * printk) can't be made yet.
+ */
+void threads_init(void)
+{
+	int ret;
+	struct thread_info *ti = &init_thread_union.thread_info;
+
+	ti->task->thread = (struct thread_struct) INIT_THREAD;
+	ret = init_arch_thread(&ti->task->thread.arch);
+	if (ret < 0)
+		pr_early("lkl: failed to allocate init schedule semaphore\n");
+
+	ti->task->thread.arch.tid = lkl_ops->thread_self();
+}
+
+void threads_cleanup(void)
+{
+	struct task_struct *p, *t;
+
+	for_each_process_thread(p, t) {
+		struct thread_info *ti = task_thread_info(t);
+
+		if (t->pid != 1 && !test_ti_thread_flag(ti, TIF_HOST_THREAD))
+			WARN(!(t->flags & PF_KTHREAD),
+			     "non kernel thread task %s\n", t->comm);
+		WARN(t->state == TASK_RUNNING,
+		     "thread %s still running while halting\n", t->comm);
+
+		kill_thread(ti);
+	}
+
+	lkl_ops->sem_free(
+		init_thread_union.thread_info.task->thread.arch.sched_sem);
+}
+
+void initial_thread_cb_skas(void (*proc)(void *), void *arg)
+{
+	pr_warn("unimplemented %s", __func__);
+}
+
+int arch_set_tls(struct task_struct *new, unsigned long tls)
+{
+	panic("unimplemented %s", __func__);
+}
+void clear_flushed_tls(struct task_struct *task)
+{
+	panic("unimplemented %s", __func__);
+}
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v5 11/21] um: nommu: kernel thread support
@ 2020-07-02 14:07       ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-07-02 14:07 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, linux-kernel-library, linux-arch,
	Hajime Tazaki, Akira Moroo

nommu mode does not support user processes but it must support kernel
threads as part as the normal kernel work-flow. It uses host operations
to create and terminate host threads that are going to run the kernel
threads. It also uses semaphores to synchronize those threads and to
allow the Linux kernel scheduler to control how the kernel threads
run.

Each kernel thread runs in a host threads and has a host semaphore
associated with it - the thread's scheduling semaphore. The semaphore
counter is initialized to 0. The first thing a kernel thread does
after getting spawned, before running any kernel code, is to perform a
down operation to block the thread.

The kernel controls host threads scheduling by performing up and down
operations on the scheduling semaphore. In __switch_context an up
operation on the next thread is performed to wake up a blocked thread,
and a down operation is performed on the prev thread to block it.

A thread is terminated by marking it in free_thread_info and
performing an up operation on the scheduling semaphore at which point
the marked thread will terminate itself.

UML common part (process.c) is extended to call idle functions of
SUBARCH as well as kernel thread detection with a flag information of
TIF_HOST_THREAD.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/include/asm/thread_info.h         |  24 ++
 arch/um/kernel/process.c                  |   8 +-
 arch/um/nommu/include/uapi/asm/host_ops.h |  54 +++++
 arch/um/nommu/um/cpu.c                    | 236 ++++++++++++++++++++
 arch/um/nommu/um/threads.c                | 258 ++++++++++++++++++++++
 5 files changed, 579 insertions(+), 1 deletion(-)
 create mode 100644 arch/um/nommu/um/cpu.c
 create mode 100644 arch/um/nommu/um/threads.c

diff --git a/arch/um/include/asm/thread_info.h b/arch/um/include/asm/thread_info.h
index 4c19ce4c49f1..433584f6b9d0 100644
--- a/arch/um/include/asm/thread_info.h
+++ b/arch/um/include/asm/thread_info.h
@@ -40,6 +40,7 @@ struct thread_info {
 	.real_thread = NULL,			\
 }
 
+#ifdef CONFIG_MMU
 /* how to get the thread information struct from C */
 static inline struct thread_info *current_thread_info(void)
 {
@@ -52,6 +53,27 @@ static inline struct thread_info *current_thread_info(void)
 	return ti;
 }
 
+#else
+
+#define __HAVE_THREAD_FUNCTIONS
+#define task_thread_info(task)	((struct thread_info *)(task)->stack)
+#define task_stack_page(task)	((task)->stack)
+#define end_of_stack(p) (&task_thread_info(p)->aux_fp_regs[FP_SIZE-1])
+
+void threads_init(void);
+void threads_cleanup(void);
+
+unsigned long *alloc_thread_stack_node(struct task_struct *p, int node);
+void setup_thread_stack(struct task_struct *p, struct task_struct *org);
+void free_thread_stack(struct task_struct *tsk);
+
+extern struct thread_info *_current_thread_info;
+static inline struct thread_info *current_thread_info(void)
+{
+	return _current_thread_info;
+}
+#endif /* CONFIG_MMU */
+
 #endif
 
 #define TIF_SYSCALL_TRACE	0	/* syscall trace active */
@@ -63,6 +85,8 @@ static inline struct thread_info *current_thread_info(void)
 #define TIF_RESTORE_SIGMASK	7
 #define TIF_NOTIFY_RESUME	8
 #define TIF_SECCOMP		9	/* secure computing */
+#define TIF_SCHED_JB		10
+#define TIF_HOST_THREAD		11
 
 #define _TIF_SYSCALL_TRACE	(1 << TIF_SYSCALL_TRACE)
 #define _TIF_SIGPENDING		(1 << TIF_SIGPENDING)
diff --git a/arch/um/kernel/process.c b/arch/um/kernel/process.c
index e5238a42ea17..0dafded5947e 100644
--- a/arch/um/kernel/process.c
+++ b/arch/um/kernel/process.c
@@ -157,7 +157,8 @@ int copy_thread_tls(unsigned long clone_flags, unsigned long sp,
 		unsigned long arg, struct task_struct * p, unsigned long tls)
 {
 	void (*handler)(void);
-	int kthread = current->flags & PF_KTHREAD;
+	int kthread = current->flags & PF_KTHREAD ||
+		test_ti_thread_flag(current_thread_info(), TIF_HOST_THREAD);
 	int ret = 0;
 
 	p->thread = (struct thread_struct) INIT_THREAD;
@@ -214,10 +215,15 @@ static void um_idle_sleep(void)
 	}
 }
 
+void __weak subarch_cpu_idle(void)
+{
+}
+
 void arch_cpu_idle(void)
 {
 	cpu_tasks[current_thread_info()->cpu].pid = os_getpid();
 	um_idle_sleep();
+	subarch_cpu_idle();
 	local_irq_enable();
 }
 
diff --git a/arch/um/nommu/include/uapi/asm/host_ops.h b/arch/um/nommu/include/uapi/asm/host_ops.h
index 5253c3f8de0e..720385fccbdf 100644
--- a/arch/um/nommu/include/uapi/asm/host_ops.h
+++ b/arch/um/nommu/include/uapi/asm/host_ops.h
@@ -16,11 +16,65 @@ struct lkl_jmp_buf {
  * These operations must be provided by a host library or by the application
  * itself.
  *
+ * @sem_alloc - allocate a host semaphore an initialize it to count
+ * @sem_free - free a host semaphore
+ * @sem_up - perform an up operation on the semaphore
+ * @sem_down - perform a down operation on the semaphore
+ *
+ * @mutex_alloc - allocate and initialize a host mutex; the recursive parameter
+ * determines if the mutex is recursive or not
+ * @mutex_free - free a host mutex
+ * @mutex_lock - acquire the mutex
+ * @mutex_unlock - release the mutex
+ *
+ * @thread_create - create a new thread and run f(arg) in its context; returns a
+ * thread handle or 0 if the thread could not be created
+ * @thread_detach - on POSIX systems, free up resources held by
+ * pthreads. Noop on Win32.
+ * @thread_exit - terminates the current thread
+ * @thread_join - wait for the given thread to terminate. Returns 0
+ * for success, -1 otherwise
+ *
+ * @gettid - returns the host thread id of the caller, which need not
+ * be the same as the handle returned by thread_create
+ *
+ * @jmp_buf_set - runs the give function and setups a jump back point by saving
+ * the context in the jump buffer; jmp_buf_longjmp can be called from the give
+ * function or any callee in that function to return back to the jump back
+ * point
+ *
+ * NOTE: we can't return from jmp_buf_set before calling jmp_buf_longjmp or
+ * otherwise the saved context (stack) is not going to be valid, so we must pass
+ * the function that will eventually call longjmp here
+ *
+ * @jmp_buf_longjmp - perform a jump back to the saved jump buffer
+ *
  * @mem_alloc - allocate memory
  * @mem_free - free memory
  *
  */
 struct lkl_host_operations {
+	struct lkl_sem *(*sem_alloc)(int count);
+	void (*sem_free)(struct lkl_sem *sem);
+	void (*sem_up)(struct lkl_sem *sem);
+	void (*sem_down)(struct lkl_sem *sem);
+
+	struct lkl_mutex *(*mutex_alloc)(int recursive);
+	void (*mutex_free)(struct lkl_mutex *mutex);
+	void (*mutex_lock)(struct lkl_mutex *mutex);
+	void (*mutex_unlock)(struct lkl_mutex *mutex);
+
+	lkl_thread_t (*thread_create)(void *(*f)(void *), void *arg);
+	void (*thread_detach)(void);
+	void (*thread_exit)(void);
+	int (*thread_join)(lkl_thread_t tid);
+	lkl_thread_t (*thread_self)(void);
+	int (*thread_equal)(lkl_thread_t a, lkl_thread_t b);
+	long (*gettid)(void);
+
+	void (*jmp_buf_set)(struct lkl_jmp_buf *jmpb, void (*f)(void));
+	void (*jmp_buf_longjmp)(struct lkl_jmp_buf *jmpb, int val);
+
 	void *(*mem_alloc)(unsigned long mem);
 	void (*mem_free)(void *mem);
 };
diff --git a/arch/um/nommu/um/cpu.c b/arch/um/nommu/um/cpu.c
new file mode 100644
index 000000000000..9986a3f8c5dd
--- /dev/null
+++ b/arch/um/nommu/um/cpu.c
@@ -0,0 +1,236 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/kernel.h>
+#include <linux/sched/stat.h>
+#include <asm/host_ops.h>
+#include <asm/cpu.h>
+#include <asm/thread_info.h>
+#include <asm/unistd.h>
+#include <asm/sched.h>
+#include <asm/syscalls.h>
+#include <init.h>
+
+void run_irqs(void)
+{
+	panic("unimplemented %s", __func__);
+}
+
+int set_irq_pending(int irq)
+{
+	panic("unimplemented %s", __func__);
+}
+
+/*
+ * This structure is used to get access to the "LKL CPU" that allows us to run
+ * Linux code. Because we have to deal with various synchronization requirements
+ * between idle thread, system calls, interrupts, "reentrancy", CPU shutdown,
+ * imbalance wake up (i.e. acquire the CPU from one thread and release it from
+ * another), we can't use a simple synchronization mechanism such as (recursive)
+ * mutex or semaphore. Instead, we use a mutex and a bunch of status data plus a
+ * semaphore.
+ */
+static struct lkl_cpu {
+	/* lock that protects the CPU status data */
+	struct lkl_mutex *lock;
+	/*
+	 * Since we must free the cpu lock during shutdown we need a
+	 * synchronization algorithm between lkl_cpu_shutdown() and the CPU
+	 * access functions since lkl_cpu_get() gets called from thread
+	 * destructor callback functions which may be scheduled after
+	 * lkl_cpu_shutdown() has freed the cpu lock.
+	 *
+	 * An atomic counter is used to keep track of the number of running
+	 * CPU access functions and allow the shutdown function to wait for
+	 * them.
+	 *
+	 * The shutdown functions adds MAX_THREADS to this counter which allows
+	 * the CPU access functions to check if the shutdown process has
+	 * started.
+	 *
+	 * This algorithm assumes that we never have more the MAX_THREADS
+	 * requesting CPU access.
+	 */
+	#define MAX_THREADS 1000000
+	unsigned int shutdown_gate;
+	bool irqs_pending;
+	/* no of threads waiting the CPU */
+	unsigned int sleepers;
+	/* no of times the current thread got the CPU */
+	unsigned int count;
+	/* current thread that owns the CPU */
+	lkl_thread_t owner;
+	/* semaphore for threads waiting the CPU */
+	struct lkl_sem *sem;
+	/* semaphore used for shutdown */
+	struct lkl_sem *shutdown_sem;
+} cpu;
+
+static int __cpu_try_get_lock(int n)
+{
+	lkl_thread_t self;
+
+	if (__sync_fetch_and_add(&cpu.shutdown_gate, n) >= MAX_THREADS)
+		return -2;
+
+	lkl_ops->mutex_lock(cpu.lock);
+
+	if (cpu.shutdown_gate >= MAX_THREADS)
+		return -1;
+
+	self = lkl_ops->thread_self();
+
+	if (cpu.owner && !lkl_ops->thread_equal(cpu.owner, self))
+		return 0;
+
+	cpu.owner = self;
+	cpu.count++;
+
+	return 1;
+}
+
+static void __cpu_try_get_unlock(int lock_ret, int n)
+{
+	if (lock_ret >= -1)
+		lkl_ops->mutex_unlock(cpu.lock);
+	__sync_fetch_and_sub(&cpu.shutdown_gate, n);
+}
+
+void lkl_cpu_change_owner(lkl_thread_t owner)
+{
+	lkl_ops->mutex_lock(cpu.lock);
+	if (cpu.count > 1)
+		lkl_bug("bad count while changing owner\n");
+	cpu.owner = owner;
+	lkl_ops->mutex_unlock(cpu.lock);
+}
+
+int lkl_cpu_get(void)
+{
+	int ret;
+
+	ret = __cpu_try_get_lock(1);
+
+	while (ret == 0) {
+		cpu.sleepers++;
+		__cpu_try_get_unlock(ret, 0);
+		lkl_ops->sem_down(cpu.sem);
+		ret = __cpu_try_get_lock(0);
+	}
+
+	__cpu_try_get_unlock(ret, 1);
+
+	return ret;
+}
+
+void lkl_cpu_put(void)
+{
+	lkl_ops->mutex_lock(cpu.lock);
+
+	if (!cpu.count || !cpu.owner ||
+	    !lkl_ops->thread_equal(cpu.owner, lkl_ops->thread_self()))
+		lkl_bug("%s: unbalanced put\n", __func__);
+
+	while (cpu.irqs_pending && !irqs_disabled()) {
+		cpu.irqs_pending = false;
+		lkl_ops->mutex_unlock(cpu.lock);
+		run_irqs();
+		lkl_ops->mutex_lock(cpu.lock);
+	}
+
+	if (test_ti_thread_flag(current_thread_info(), TIF_HOST_THREAD) &&
+	    !single_task_running() && cpu.count == 1) {
+		if (in_interrupt())
+			lkl_bug("%s: in interrupt\n", __func__);
+		lkl_ops->mutex_unlock(cpu.lock);
+		thread_sched_jb();
+		return;
+	}
+
+	if (--cpu.count > 0) {
+		lkl_ops->mutex_unlock(cpu.lock);
+		return;
+	}
+
+	if (cpu.sleepers) {
+		cpu.sleepers--;
+		lkl_ops->sem_up(cpu.sem);
+	}
+
+	cpu.owner = 0;
+
+	lkl_ops->mutex_unlock(cpu.lock);
+}
+
+int lkl_cpu_try_run_irq(int irq)
+{
+	int ret;
+
+	ret = __cpu_try_get_lock(1);
+	if (!ret) {
+		set_irq_pending(irq);
+		cpu.irqs_pending = true;
+	}
+	__cpu_try_get_unlock(ret, 1);
+
+	return ret;
+}
+
+static void lkl_cpu_shutdown(void)
+{
+	__sync_fetch_and_add(&cpu.shutdown_gate, MAX_THREADS);
+}
+__uml_exitcall(lkl_cpu_shutdown);
+
+void lkl_cpu_wait_shutdown(void)
+{
+	lkl_ops->sem_down(cpu.shutdown_sem);
+	lkl_ops->sem_free(cpu.shutdown_sem);
+}
+
+static void lkl_cpu_cleanup(bool shutdown)
+{
+	while (__sync_fetch_and_add(&cpu.shutdown_gate, 0) > MAX_THREADS)
+		;
+
+	if (shutdown)
+		lkl_ops->sem_up(cpu.shutdown_sem);
+	else if (cpu.shutdown_sem)
+		lkl_ops->sem_free(cpu.shutdown_sem);
+	if (cpu.sem)
+		lkl_ops->sem_free(cpu.sem);
+	if (cpu.lock)
+		lkl_ops->mutex_free(cpu.lock);
+}
+
+void subarch_cpu_idle(void)
+{
+	if (cpu.shutdown_gate >= MAX_THREADS) {
+
+		lkl_ops->mutex_lock(cpu.lock);
+		while (cpu.sleepers--)
+			lkl_ops->sem_up(cpu.sem);
+		lkl_ops->mutex_unlock(cpu.lock);
+
+		lkl_cpu_cleanup(true);
+
+		lkl_ops->thread_exit();
+	}
+
+#ifdef doesntwork
+	/* switch to idle_host_task */
+	wakeup_idle_host_task();
+#endif
+}
+
+int lkl_cpu_init(void)
+{
+	cpu.lock = lkl_ops->mutex_alloc(0);
+	cpu.sem = lkl_ops->sem_alloc(0);
+	cpu.shutdown_sem = lkl_ops->sem_alloc(0);
+
+	if (!cpu.lock || !cpu.sem || !cpu.shutdown_sem) {
+		lkl_cpu_cleanup(false);
+		return -ENOMEM;
+	}
+
+	return 0;
+}
diff --git a/arch/um/nommu/um/threads.c b/arch/um/nommu/um/threads.c
new file mode 100644
index 000000000000..3e70eccc191a
--- /dev/null
+++ b/arch/um/nommu/um/threads.c
@@ -0,0 +1,258 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/slab.h>
+#include <linux/sched/task.h>
+#include <linux/sched/signal.h>
+#include <asm/host_ops.h>
+#include <asm/cpu.h>
+#include <asm/sched.h>
+
+#include <os.h>
+
+static int init_arch_thread(struct arch_thread *thread)
+{
+	thread->sched_sem = lkl_ops->sem_alloc(0);
+	if (!thread->sched_sem)
+		return -ENOMEM;
+
+	thread->dead = false;
+	thread->tid = 0;
+
+	return 0;
+}
+
+unsigned long *alloc_thread_stack_node(struct task_struct *task, int node)
+{
+	struct thread_info *ti;
+
+	ti = kmalloc(sizeof(*ti), GFP_KERNEL | __GFP_ZERO);
+	if (!ti)
+		return NULL;
+
+	ti->task = task;
+	return (unsigned long *)ti;
+}
+
+/*
+ * The only new tasks created are kernel threads that have a predefined starting
+ * point thus no stack copy is required.
+ */
+void setup_thread_stack(struct task_struct *p, struct task_struct *org)
+{
+	struct thread_info *ti = task_thread_info(p);
+	struct thread_info *org_ti = task_thread_info(org);
+
+	ti->flags = org_ti->flags;
+	ti->preempt_count = org_ti->preempt_count;
+	ti->addr_limit = org_ti->addr_limit;
+}
+
+static void kill_thread(struct thread_info *ti)
+{
+	if (!test_ti_thread_flag(ti, TIF_HOST_THREAD)) {
+		ti->task->thread.arch.dead = true;
+		lkl_ops->sem_up(ti->task->thread.arch.sched_sem);
+		lkl_ops->thread_join(ti->task->thread.arch.tid);
+	}
+	lkl_ops->sem_free(ti->task->thread.arch.sched_sem);
+}
+
+void free_thread_stack(struct task_struct *tsk)
+{
+	struct thread_info *ti = task_thread_info(tsk);
+
+	kill_thread(ti);
+	kfree(ti);
+}
+
+struct thread_info *_current_thread_info = &init_thread_union.thread_info;
+
+void switch_threads(jmp_buf *me, jmp_buf *you)
+{
+	/* NOP */
+}
+
+/*
+ * schedule() expects the return of this function to be the task that we
+ * switched away from. Returning prev is not going to work because we are
+ * actually going to return the previous taks that was scheduled before the
+ * task we are going to wake up, and not the current task, e.g.:
+ *
+ * swapper -> init: saved prev on swapper stack is swapper
+ * init -> ksoftirqd0: saved prev on init stack is init
+ * ksoftirqd0 -> swapper: returned prev is swapper
+ */
+static struct task_struct *abs_prev = &init_task;
+
+void arch_switch_to(struct task_struct *prev,
+		    struct task_struct *next)
+{
+	struct arch_thread *_prev = &prev->thread.arch;
+	struct arch_thread *_next = &next->thread.arch;
+	unsigned long _prev_flags = task_thread_info(prev)->flags;
+	struct lkl_jmp_buf *_prev_jb;
+
+	_current_thread_info = task_thread_info(next);
+	next->thread.prev_sched = prev;
+	abs_prev = prev;
+
+	WARN_ON(!_next->tid);
+	lkl_cpu_change_owner(_next->tid);
+
+	if (test_bit(TIF_SCHED_JB, &_prev_flags)) {
+		/* Atomic. Must be done before wakeup next */
+		clear_ti_thread_flag(task_thread_info(prev), TIF_SCHED_JB);
+		_prev_jb = &_prev->sched_jb;
+	}
+
+	lkl_ops->sem_up(_next->sched_sem);
+	if (test_bit(TIF_SCHED_JB, &_prev_flags))
+		lkl_ops->jmp_buf_longjmp(_prev_jb, 1);
+	else
+		lkl_ops->sem_down(_prev->sched_sem);
+
+	if (_prev->dead)
+		lkl_ops->thread_exit();
+
+	/* __switch_to (arch/um) returns this value */
+	current->thread.prev_sched = abs_prev;
+}
+
+int host_task_stub(void *unused)
+{
+	return 0;
+}
+
+void switch_to_host_task(struct task_struct *task)
+{
+	if (WARN_ON(!test_tsk_thread_flag(task, TIF_HOST_THREAD)))
+		return;
+
+	task->thread.arch.tid = lkl_ops->thread_self();
+
+	if (current == task)
+		return;
+
+	wake_up_process(task);
+	thread_sched_jb();
+	lkl_ops->sem_down(task->thread.arch.sched_sem);
+	schedule_tail(abs_prev);
+}
+
+struct thread_bootstrap_arg {
+	struct thread_info *ti;
+	int (*f)(void *a);
+	void *arg;
+};
+
+static void *thread_bootstrap(void *_tba)
+{
+	struct thread_bootstrap_arg *tba = (struct thread_bootstrap_arg *)_tba;
+	struct thread_info *ti = tba->ti;
+	int (*f)(void *) = tba->f;
+	void *arg = tba->arg;
+
+	lkl_ops->sem_down(ti->task->thread.arch.sched_sem);
+	kfree(tba);
+	if (ti->task->thread.prev_sched)
+		schedule_tail(ti->task->thread.prev_sched);
+
+	f(arg);
+	do_exit(0);
+
+	return NULL;
+}
+
+void new_thread(void *stack, jmp_buf *buf, void (*handler)(void))
+{
+	struct thread_info *ti = (struct thread_info *)stack;
+	struct task_struct *p = ti->task;
+	struct thread_bootstrap_arg *tba;
+	int ret;
+
+	unsigned long esp = (unsigned long)p->thread.request.u.thread.proc;
+	unsigned long unused = (unsigned long)p->thread.request.u.thread.arg;
+
+	ret = init_arch_thread(&p->thread.arch);
+	if (ret < 0)
+		panic("%s: init_arch_thread", __func__);
+
+	if ((int (*)(void *))esp == host_task_stub) {
+		set_ti_thread_flag(ti, TIF_HOST_THREAD);
+		return;
+	}
+
+	tba = kmalloc(sizeof(*tba), GFP_KERNEL);
+	if (!tba)
+		return;
+
+	tba->f = (int (*)(void *))esp;
+	tba->arg = (void *)unused;
+	tba->ti = ti;
+
+	p->thread.arch.tid = lkl_ops->thread_create(thread_bootstrap, tba);
+	if (!p->thread.arch.tid) {
+		kfree(tba);
+		return;
+	}
+}
+
+void show_stack(struct task_struct *task, unsigned long *esp)
+{
+}
+
+static inline void pr_early(const char *str)
+{
+	if (lkl_ops->print)
+		lkl_ops->print(str, strlen(str));
+}
+
+/**
+ * This is called before the kernel initializes, so no kernel calls (including
+ * printk) can't be made yet.
+ */
+void threads_init(void)
+{
+	int ret;
+	struct thread_info *ti = &init_thread_union.thread_info;
+
+	ti->task->thread = (struct thread_struct) INIT_THREAD;
+	ret = init_arch_thread(&ti->task->thread.arch);
+	if (ret < 0)
+		pr_early("lkl: failed to allocate init schedule semaphore\n");
+
+	ti->task->thread.arch.tid = lkl_ops->thread_self();
+}
+
+void threads_cleanup(void)
+{
+	struct task_struct *p, *t;
+
+	for_each_process_thread(p, t) {
+		struct thread_info *ti = task_thread_info(t);
+
+		if (t->pid != 1 && !test_ti_thread_flag(ti, TIF_HOST_THREAD))
+			WARN(!(t->flags & PF_KTHREAD),
+			     "non kernel thread task %s\n", t->comm);
+		WARN(t->state == TASK_RUNNING,
+		     "thread %s still running while halting\n", t->comm);
+
+		kill_thread(ti);
+	}
+
+	lkl_ops->sem_free(
+		init_thread_union.thread_info.task->thread.arch.sched_sem);
+}
+
+void initial_thread_cb_skas(void (*proc)(void *), void *arg)
+{
+	pr_warn("unimplemented %s", __func__);
+}
+
+int arch_set_tls(struct task_struct *new, unsigned long tls)
+{
+	panic("unimplemented %s", __func__);
+}
+void clear_flushed_tls(struct task_struct *task)
+{
+	panic("unimplemented %s", __func__);
+}
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v5 12/21] um: nommu: system call interface and application API
  2020-07-02 14:06     ` Hajime Tazaki
@ 2020-07-02 14:07       ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-07-02 14:07 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Hajime Tazaki

The application API is based on the kernel system call interface in
order to offer a stable API to applications. Note that we can't
offer the full Linux system call interface due to the limitations of
nommu mode such as lack of virtual memory, signal, user processes, etc.

The host is using dedicated kernel thread, which is switched upon the
entry of lkl_syscall so that numbers of context switches can be reduced
for the faster performance.

To expose the API definitions to applications, this commit uses
syscall_wrapper to define arch-specific information of syscall, and this
is used to generate syscall_defs.h for lkl so that it can be used
as a template of the list of syscall available for LKL apps.

To avoid collisions between the Linux API and the LKL API (e.g.  struct
stat, MKNOD, etc.) we use a python script to modify the user headers
and to prefix all of the global symbols (structures, typedefs,
defines) with LKL, lkl, _LKL, _lkl, __LKL or __lkl.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/include/uapi/asm/Kbuild             |   2 +
 arch/um/nommu/include/asm/syscall_wrapper.h |  57 ++++
 arch/um/nommu/include/asm/syscalls.h        |  15 +
 arch/um/nommu/include/uapi/asm/host_ops.h   |  14 +
 arch/um/nommu/include/uapi/asm/syscalls.h   | 287 ++++++++++++++++++++
 arch/um/nommu/include/uapi/asm/unistd.h     |  15 +
 arch/um/nommu/um/syscalls.c                 | 199 ++++++++++++++
 arch/um/scripts/headers_install.py          | 197 ++++++++++++++
 8 files changed, 786 insertions(+)
 create mode 100644 arch/um/include/uapi/asm/Kbuild
 create mode 100644 arch/um/nommu/include/asm/syscall_wrapper.h
 create mode 100644 arch/um/nommu/include/asm/syscalls.h
 create mode 100644 arch/um/nommu/include/uapi/asm/syscalls.h
 create mode 100644 arch/um/nommu/include/uapi/asm/unistd.h
 create mode 100644 arch/um/nommu/um/syscalls.c
 create mode 100755 arch/um/scripts/headers_install.py

diff --git a/arch/um/include/uapi/asm/Kbuild b/arch/um/include/uapi/asm/Kbuild
new file mode 100644
index 000000000000..a983c7f049bc
--- /dev/null
+++ b/arch/um/include/uapi/asm/Kbuild
@@ -0,0 +1,2 @@
+# no generated-y since we need special user headers handling
+# see arch/um/script/headers_install.py
diff --git a/arch/um/nommu/include/asm/syscall_wrapper.h b/arch/um/nommu/include/asm/syscall_wrapper.h
new file mode 100644
index 000000000000..bb2a7d4274f6
--- /dev/null
+++ b/arch/um/nommu/include/asm/syscall_wrapper.h
@@ -0,0 +1,57 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef __UM_SYSCALL_WRAPPER_H
+#define __UM_SYSCALL_WRAPPER_H
+
+#define __SC_ASCII(t, a) #t "," #a
+
+#define __ASCII_MAP0(m, ...)
+#define __ASCII_MAP1(m, t, a) m(t, a)
+#define __ASCII_MAP2(m, t, a, ...) m(t, a) "," __ASCII_MAP1(m, __VA_ARGS__)
+#define __ASCII_MAP3(m, t, a, ...) m(t, a) "," __ASCII_MAP2(m, __VA_ARGS__)
+#define __ASCII_MAP4(m, t, a, ...) m(t, a) "," __ASCII_MAP3(m, __VA_ARGS__)
+#define __ASCII_MAP5(m, t, a, ...) m(t, a) "," __ASCII_MAP4(m, __VA_ARGS__)
+#define __ASCII_MAP6(m, t, a, ...) m(t, a) "," __ASCII_MAP5(m, __VA_ARGS__)
+#define __ASCII_MAP(n, ...) __ASCII_MAP##n(__VA_ARGS__)
+
+#ifdef __MINGW32__
+#define SECTION_ATTRS "n0"
+#else
+#define SECTION_ATTRS "a"
+#endif
+
+#define __SYSCALL_DEFINE_ARCH(x, name, ...)				\
+	asm(".section .syscall_defs,\"" SECTION_ATTRS "\"\n"		\
+	    ".ascii \"#ifdef __NR" #name "\\n\"\n"			\
+	    ".ascii \"SYSCALL_DEFINE" #x "(" #name ","			\
+	    __ASCII_MAP(x, __SC_ASCII, __VA_ARGS__) ")\\n\"\n"		\
+	    ".ascii \"#endif\\n\"\n"					\
+	    ".section .text\n")
+
+#define SYSCALL_DEFINE0(sname)					\
+	SYSCALL_METADATA(_##sname, 0);				\
+	__SYSCALL_DEFINE_ARCH(0, _##sname);			\
+	asmlinkage long sys_##sname(void);			\
+	ALLOW_ERROR_INJECTION(sys_##sname, ERRNO);		\
+	asmlinkage long sys_##sname(void)
+
+#define __SYSCALL_DEFINEx(x, name, ...)					\
+	__SYSCALL_DEFINE_ARCH(x, name, __VA_ARGS__);			\
+	__diag_push();							\
+	__diag_ignore(GCC, 8, "-Wattribute-alias",			\
+		      "Type aliasing is used to sanitize syscall arguments");\
+	asmlinkage long sys##name(__MAP(x, __SC_DECL, __VA_ARGS__))	\
+		__attribute__((alias(__stringify(__se_sys##name))));	\
+	ALLOW_ERROR_INJECTION(sys##name, ERRNO);			\
+	static inline long __do_sys##name(__MAP(x, __SC_DECL, __VA_ARGS__)); \
+	asmlinkage long __se_sys##name(__MAP(x, __SC_LONG, __VA_ARGS__)); \
+	asmlinkage long __se_sys##name(__MAP(x, __SC_LONG, __VA_ARGS__)) \
+	{								\
+		long ret = __do_sys##name(__MAP(x, __SC_CAST, __VA_ARGS__)); \
+		__MAP(x, __SC_TEST, __VA_ARGS__);			\
+		__PROTECT(x, ret, __MAP(x, __SC_ARGS, __VA_ARGS__));	\
+		return ret;						\
+	}								\
+	__diag_pop();							\
+	static inline long __do_sys##name(__MAP(x, __SC_DECL, __VA_ARGS__))
+
+#endif /* __UM_SYSCALL_WRAPPER_H */
diff --git a/arch/um/nommu/include/asm/syscalls.h b/arch/um/nommu/include/asm/syscalls.h
new file mode 100644
index 000000000000..6061d9415dad
--- /dev/null
+++ b/arch/um/nommu/include/asm/syscalls.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+int syscalls_init(void);
+void syscalls_cleanup(void);
+long lkl_syscall(long no, long *params);
+void wakeup_idle_host_task(void);
+
+#define sys_mmap sys_ni_syscall
+#define sys_rt_sigreturn sys_ni_syscall
+#define sys_arch_prctl sys_ni_syscall
+#define sys_iopl sys_ni_syscall
+#define sys_ioperm sys_ni_syscall
+#define sys_clone sys_ni_syscall
+
+int run_syscalls(void);
diff --git a/arch/um/nommu/include/uapi/asm/host_ops.h b/arch/um/nommu/include/uapi/asm/host_ops.h
index 720385fccbdf..6ee9489fc47e 100644
--- a/arch/um/nommu/include/uapi/asm/host_ops.h
+++ b/arch/um/nommu/include/uapi/asm/host_ops.h
@@ -38,6 +38,15 @@ struct lkl_jmp_buf {
  * @gettid - returns the host thread id of the caller, which need not
  * be the same as the handle returned by thread_create
  *
+ * @tls_alloc - allocate a thread local storage key; returns 0 if successful; if
+ * destructor is not NULL it will be called when a thread terminates with its
+ * argument set to the current thread local storage value
+ * @tls_free - frees a thread local storage key; returns 0 if successful
+ * @tls_set - associate data to the thread local storage key; returns 0 if
+ * successful
+ * @tls_get - return data associated with the thread local storage key or NULL
+ * on error
+ *
  * @jmp_buf_set - runs the give function and setups a jump back point by saving
  * the context in the jump buffer; jmp_buf_longjmp can be called from the give
  * function or any callee in that function to return back to the jump back
@@ -72,6 +81,11 @@ struct lkl_host_operations {
 	int (*thread_equal)(lkl_thread_t a, lkl_thread_t b);
 	long (*gettid)(void);
 
+	struct lkl_tls_key *(*tls_alloc)(void (*destructor)(void *));
+	void (*tls_free)(struct lkl_tls_key *key);
+	int (*tls_set)(struct lkl_tls_key *key, void *data);
+	void *(*tls_get)(struct lkl_tls_key *key);
+
 	void (*jmp_buf_set)(struct lkl_jmp_buf *jmpb, void (*f)(void));
 	void (*jmp_buf_longjmp)(struct lkl_jmp_buf *jmpb, int val);
 
diff --git a/arch/um/nommu/include/uapi/asm/syscalls.h b/arch/um/nommu/include/uapi/asm/syscalls.h
new file mode 100644
index 000000000000..751957b978cd
--- /dev/null
+++ b/arch/um/nommu/include/uapi/asm/syscalls.h
@@ -0,0 +1,287 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef __UM_NOMMU_UAPI_SYSCALLS_H
+#define __UM_NOMMU_UAPI_SYSCALLS_H
+
+#include <autoconf.h>
+#include <linux/types.h>
+
+typedef __kernel_uid32_t	qid_t;
+typedef __kernel_fd_set		fd_set;
+typedef __kernel_mode_t		mode_t;
+typedef unsigned short		umode_t;
+typedef __u32			nlink_t;
+typedef __kernel_off_t		off_t;
+typedef __kernel_pid_t		pid_t;
+typedef __kernel_key_t		key_t;
+typedef __kernel_suseconds_t	suseconds_t;
+typedef __kernel_timer_t	timer_t;
+typedef __kernel_clockid_t	clockid_t;
+typedef __kernel_mqd_t		mqd_t;
+typedef __kernel_uid32_t	uid_t;
+typedef __kernel_gid32_t	gid_t;
+typedef __kernel_uid16_t        uid16_t;
+typedef __kernel_gid16_t        gid16_t;
+typedef unsigned long		uintptr_t;
+#ifdef CONFIG_UID16
+typedef __kernel_old_uid_t	old_uid_t;
+typedef __kernel_old_gid_t	old_gid_t;
+#endif
+typedef __kernel_loff_t		loff_t;
+typedef __kernel_size_t		size_t;
+typedef __kernel_ssize_t	ssize_t;
+typedef __kernel_time_t		time_t;
+typedef __kernel_clock_t	clock_t;
+typedef __u32			u32;
+typedef __s32			s32;
+typedef __u64			u64;
+typedef __s64			s64;
+
+typedef __kernel_long_t	__kernel_old_time_t;
+
+#define __user
+
+#include <asm/unistd.h>
+/* Temporary undefine system calls that don't have data types defined in UAPI
+ * headers
+ */
+#undef __NR_kexec_load
+#undef __NR_getcpu
+#undef __NR_sched_getattr
+#undef __NR_sched_setattr
+#undef __NR_sched_setparam
+#undef __NR_sched_getparam
+#undef __NR_sched_setscheduler
+#undef __NR_name_to_handle_at
+#undef __NR_open_by_handle_at
+
+/* deprecated system calls */
+#undef __NR_access
+#undef __NR_chmod
+#undef __NR_chown
+#undef __NR_lchown
+#undef __NR_open
+#undef __NR_creat
+#undef __NR_readlink
+#undef __NR_pipe
+#undef __NR_mknod
+#undef __NR_mkdir
+#undef __NR_rmdir
+#undef __NR_unlink
+#undef __NR_symlink
+#undef __NR_link
+#undef __NR_rename
+#undef __NR_getdents
+#undef __NR_select
+#undef __NR_poll
+#undef __NR_dup2
+#undef __NR_sysfs
+#undef __NR_ustat
+#undef __NR_inotify_init
+#undef __NR_epoll_create
+#undef __NR_epoll_wait
+#undef __NR_signalfd
+#undef __NR_eventfd
+
+#undef __NR_umount
+#define __NR_umount __NR_umount2
+
+#ifdef CONFIG_64BIT
+#define __NR_newfstat __NR3264_fstat
+#define __NR_newfstatat __NR3264_fstatat
+#endif
+
+#include <linux/time.h>
+#include <linux/times.h>
+#include <linux/timex.h>
+#include <linux/capability.h>
+#define __KERNEL__ /* to pull in S_ definitions */
+#include <linux/stat.h>
+#undef __KERNEL__
+#include <linux/errno.h>
+#include <linux/fcntl.h>
+#include <linux/fs.h>
+#include <asm/statfs.h>
+#include <asm/stat.h>
+#include <linux/bpf.h>
+#include <linux/msg.h>
+#include <linux/resource.h>
+#include <linux/sysinfo.h>
+#include <linux/shm.h>
+#include <linux/aio_abi.h>
+#include <linux/socket.h>
+#include <linux/perf_event.h>
+#include <linux/sem.h>
+#include <linux/futex.h>
+#include <linux/poll.h>
+#include <linux/mqueue.h>
+#include <linux/eventpoll.h>
+#include <linux/uio.h>
+#include <asm/signal.h>
+#include <asm/siginfo.h>
+#include <linux/utime.h>
+#include <asm/socket.h>
+#include <linux/icmp.h>
+#include <linux/ip.h>
+
+/* Define data structures used in system calls that are not defined in UAPI
+ * headers
+ */
+struct sockaddr {
+	unsigned short int sa_family;
+	char sa_data[14];
+};
+
+#define __UAPI_DEF_IF_NET_DEVICE_FLAGS_LOWER_UP_DORMANT_ECHO 1
+#define __UAPI_DEF_IF_IFNAMSIZ	1
+#define __UAPI_DEF_IF_NET_DEVICE_FLAGS 1
+#define __UAPI_DEF_IF_IFREQ	1
+#define __UAPI_DEF_IF_IFMAP	1
+#include <linux/if.h>
+#define __UAPI_DEF_IN_IPPROTO	1
+#define __UAPI_DEF_IN_ADDR	1
+#define __UAPI_DEF_IN6_ADDR	1
+#define __UAPI_DEF_IP_MREQ	1
+#define __UAPI_DEF_IN_PKTINFO	1
+#define __UAPI_DEF_SOCKADDR_IN	1
+#define __UAPI_DEF_IN_CLASS	1
+#include <linux/in.h>
+#include <linux/in6.h>
+#include <linux/sockios.h>
+#include <linux/route.h>
+#include <linux/ipv6_route.h>
+#include <linux/ipv6.h>
+#include <linux/netlink.h>
+#include <linux/neighbour.h>
+#include <linux/rtnetlink.h>
+#include <linux/fib_rules.h>
+
+#include <linux/kdev_t.h>
+#include <linux/virtio_blk.h>
+#include <linux/virtio_net.h>
+#include <linux/virtio_ring.h>
+#include <linux/pkt_sched.h>
+#include <linux/io_uring.h>
+#include <linux/utsname.h>
+
+struct user_msghdr {
+	void		__user *msg_name;
+	int		msg_namelen;
+	struct iovec	__user *msg_iov;
+	__kernel_size_t	msg_iovlen;
+	void		__user *msg_control;
+	__kernel_size_t	msg_controllen;
+	unsigned int	msg_flags;
+};
+
+typedef __u32 key_serial_t;
+
+struct mmsghdr {
+	struct user_msghdr  msg_hdr;
+	unsigned int        msg_len;
+};
+
+struct linux_dirent64 {
+	u64		d_ino;
+	s64		d_off;
+	unsigned short	d_reclen;
+	unsigned char	d_type;
+	char		d_name[0];
+};
+
+struct linux_dirent {
+	unsigned long	d_ino;
+	unsigned long	d_off;
+	unsigned short	d_reclen;
+	char		d_name[1];
+};
+
+struct ustat {
+	__kernel_daddr_t	f_tfree;
+	__kernel_ino_t		f_tinode;
+	char			f_fname[6];
+	char			f_fpack[6];
+};
+
+struct __aio_sigset;
+struct clone_args;
+
+typedef __kernel_rwf_t		rwf_t;
+
+long lkl_syscall(long no, long *params);
+long lkl_sys_halt(void);
+
+#define __MAP0(m, ...)
+#define __MAP1(m, t, a) m(t, a)
+#define __MAP2(m, t, a, ...) m(t, a), __MAP1(m, __VA_ARGS__)
+#define __MAP3(m, t, a, ...) m(t, a), __MAP2(m, __VA_ARGS__)
+#define __MAP4(m, t, a, ...) m(t, a), __MAP3(m, __VA_ARGS__)
+#define __MAP5(m, t, a, ...) m(t, a), __MAP4(m, __VA_ARGS__)
+#define __MAP6(m, t, a, ...) m(t, a), __MAP5(m, __VA_ARGS__)
+#define __MAP(n, ...) __MAP##n(__VA_ARGS__)
+
+#define __SC_LONG(t, a) (long)(a)
+#define __SC_TABLE(t, a) {sizeof(t), (long long)(a)}
+#define __SC_DECL(t, a) t a
+
+#define LKL_SYSCALL0(name)					       \
+	static inline long lkl_sys##name(void)			       \
+	{							       \
+		long params[6];					       \
+		return lkl_syscall(__lkl__NR##name, params);	       \
+	}
+
+#if __BITS_PER_LONG == 32
+#define LKL_SYSCALLx(x, name, ...)					\
+	static inline							\
+	long lkl_sys##name(__MAP(x, __SC_DECL, __VA_ARGS__))		\
+	{								\
+		struct {						\
+			unsigned int size;				\
+			long long value;				\
+		} lkl_params[x] = { __MAP(x, __SC_TABLE, __VA_ARGS__) }; \
+		long sys_params[6], i, k;				\
+		for (i = k = 0; i < x && k < 6; i++, k++) {		\
+			if (lkl_params[i].size > sizeof(long) &&	\
+			    k + 1 < 6) {				\
+				sys_params[k] =				\
+					(long)(lkl_params[i].value & (-1UL)); \
+				k++;					\
+				sys_params[k] =				\
+					(long)(lkl_params[i].value >>	\
+					       __BITS_PER_LONG);	\
+			} else {					\
+				sys_params[k] = (long)(lkl_params[i].value); \
+			}						\
+		}							\
+		return lkl_syscall(__lkl__NR##name, sys_params);	\
+	}
+#else
+#define LKL_SYSCALLx(x, name, ...)					\
+	static inline							\
+	long lkl_sys##name(__MAP(x, __SC_DECL, __VA_ARGS__))		\
+	{								\
+		long lkl_params[6] = { __MAP(x, __SC_LONG, __VA_ARGS__) }; \
+		return lkl_syscall(__lkl__NR##name, lkl_params);	\
+	}
+#endif
+
+#define SYSCALL_DEFINE0(name, ...) LKL_SYSCALL0(name)
+#define SYSCALL_DEFINE1(name, ...) LKL_SYSCALLx(1, name, __VA_ARGS__)
+#define SYSCALL_DEFINE2(name, ...) LKL_SYSCALLx(2, name, __VA_ARGS__)
+#define SYSCALL_DEFINE3(name, ...) LKL_SYSCALLx(3, name, __VA_ARGS__)
+#define SYSCALL_DEFINE4(name, ...) LKL_SYSCALLx(4, name, __VA_ARGS__)
+#define SYSCALL_DEFINE5(name, ...) LKL_SYSCALLx(5, name, __VA_ARGS__)
+#define SYSCALL_DEFINE6(name, ...) LKL_SYSCALLx(6, name, __VA_ARGS__)
+
+#if __BITS_PER_LONG == 32
+#pragma GCC diagnostic push
+#pragma GCC diagnostic ignored "-Wpointer-to-int-cast"
+#endif
+
+#include <asm/syscall_defs.h>
+
+#if __BITS_PER_LONG == 32
+#pragma GCC diagnostic pop
+#endif
+
+#endif
diff --git a/arch/um/nommu/include/uapi/asm/unistd.h b/arch/um/nommu/include/uapi/asm/unistd.h
new file mode 100644
index 000000000000..1762ebb829f5
--- /dev/null
+++ b/arch/um/nommu/include/uapi/asm/unistd.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef __UM_NOMMU_UAPI_UNISTD_H
+#define __UM_NOMMU_UAPI_UNISTD_H
+
+#define __ARCH_WANT_NEW_STAT
+#include <asm/bitsperlong.h>
+
+#if __BITS_PER_LONG == 64
+#define __ARCH_WANT_SYS_NEWFSTATAT
+#endif
+
+#include <asm-generic/unistd.h>
+
+
+#endif
diff --git a/arch/um/nommu/um/syscalls.c b/arch/um/nommu/um/syscalls.c
new file mode 100644
index 000000000000..fd343f855bad
--- /dev/null
+++ b/arch/um/nommu/um/syscalls.c
@@ -0,0 +1,199 @@
+// SPDX-License-Identifier: GPL-2.0
+#ifdef CONFIG_ARCH_HAS_SYSCALL_WRAPPER
+#undef CONFIG_ARCH_HAS_SYSCALL_WRAPPER
+#include <linux/syscalls.h>
+#define CONFIG_ARCH_HAS_SYSCALL_WRAPPER
+#endif
+
+#include <linux/stat.h>
+#include <linux/irq.h>
+#include <linux/sched.h>
+#include <linux/interrupt.h>
+#include <linux/jhash.h>
+#include <linux/slab.h>
+#include <linux/types.h>
+#include <linux/net.h>
+#include <linux/task_work.h>
+
+#include <asm/host_ops.h>
+#include <asm/syscalls.h>
+#include <asm/cpu.h>
+#include <asm/sched.h>
+#include <kern_util.h>
+#include <os.h>
+
+typedef long (*syscall_handler_t)(long arg1, ...);
+
+#undef __SYSCALL
+#define __SYSCALL(nr, sym)[nr] = (syscall_handler_t)sym,
+
+syscall_handler_t syscall_table[__NR_syscalls] = {
+	[0 ... __NR_syscalls - 1] =  (syscall_handler_t)sys_ni_syscall,
+#undef __UM_NOMMU_UAPI_UNISTD_H
+#include <asm/unistd.h>
+
+#if __BITS_PER_LONG == 32
+#include <asm/unistd_32.h>
+#endif
+};
+
+
+static long run_syscall(long no, long *params)
+{
+	long ret;
+
+	if (no < 0 || no >= __NR_syscalls)
+		return -ENOSYS;
+
+	ret = syscall_table[no](params[0], params[1], params[2], params[3],
+				params[4], params[5]);
+
+	task_work_run();
+
+	return ret;
+}
+
+
+#define CLONE_FLAGS (CLONE_VM | CLONE_FS | CLONE_FILES | CLONE_THREAD |	\
+		     CLONE_SIGHAND | SIGCHLD)
+
+static int host_task_id;
+static struct task_struct *host0;
+
+static int new_host_task(struct task_struct **task)
+{
+	pid_t pid;
+
+	switch_to_host_task(host0);
+
+	pid = kernel_thread(host_task_stub, NULL, CLONE_FLAGS);
+	if (pid < 0)
+		return pid;
+
+	rcu_read_lock();
+	*task = find_task_by_pid_ns(pid, &init_pid_ns);
+	rcu_read_unlock();
+
+	host_task_id++;
+
+	snprintf((*task)->comm, sizeof((*task)->comm), "host%d", host_task_id);
+
+	return 0;
+}
+static void exit_task(void)
+{
+	do_exit(0);
+}
+
+static void del_host_task(void *arg)
+{
+	struct task_struct *task = (struct task_struct *)arg;
+	struct thread_info *ti = task_thread_info(task);
+
+	if (lkl_cpu_get() < 0)
+		return;
+
+	switch_to_host_task(task);
+	host_task_id--;
+	set_ti_thread_flag(ti, TIF_SCHED_JB);
+	lkl_ops->jmp_buf_set(&ti->task->thread.arch.sched_jb, exit_task);
+}
+
+static struct lkl_tls_key *task_key;
+
+long lkl_syscall(long no, long *params)
+{
+	struct task_struct *task = host0;
+	long ret;
+
+	ret = lkl_cpu_get();
+	if (ret < 0)
+		return ret;
+
+	if (lkl_ops->tls_get) {
+		task = lkl_ops->tls_get(task_key);
+		if (!task) {
+			ret = new_host_task(&task);
+			if (ret)
+				goto out;
+			lkl_ops->tls_set(task_key, task);
+		}
+	}
+
+	switch_to_host_task(task);
+
+	ret = run_syscall(no, params);
+
+	if (no == __NR_reboot) {
+		thread_sched_jb();
+		return ret;
+	}
+
+out:
+	lkl_cpu_put();
+
+	return ret;
+}
+
+static struct task_struct *idle_host_task;
+
+/* called from idle, don't failed, don't block */
+void wakeup_idle_host_task(void)
+{
+	if (!need_resched() && idle_host_task)
+		wake_up_process(idle_host_task);
+}
+
+static int idle_host_task_loop(void *unused)
+{
+	struct thread_info *ti = task_thread_info(current);
+
+	snprintf(current->comm, sizeof(current->comm), "idle_host_task");
+	set_thread_flag(TIF_HOST_THREAD);
+	idle_host_task = current;
+
+	for (;;) {
+		lkl_cpu_put();
+		lkl_ops->sem_down(ti->task->thread.arch.sched_sem);
+		if (idle_host_task == NULL) {
+			lkl_ops->thread_exit();
+			return 0;
+		}
+		schedule_tail(ti->task->thread.prev_sched);
+	}
+}
+
+int syscalls_init(void)
+{
+	snprintf(current->comm, sizeof(current->comm), "host0");
+	set_thread_flag(TIF_HOST_THREAD);
+	host0 = current;
+
+	if (lkl_ops->tls_alloc) {
+		task_key = lkl_ops->tls_alloc(del_host_task);
+		if (!task_key)
+			return -1;
+	}
+
+	if (kernel_thread(idle_host_task_loop, NULL, CLONE_FLAGS) < 0) {
+		if (lkl_ops->tls_free)
+			lkl_ops->tls_free(task_key);
+		return -1;
+	}
+
+	return 0;
+}
+
+void syscalls_cleanup(void)
+{
+	if (idle_host_task) {
+		struct thread_info *ti = task_thread_info(idle_host_task);
+
+		idle_host_task = NULL;
+		lkl_ops->sem_up(ti->task->thread.arch.sched_sem);
+		lkl_ops->thread_join(ti->task->thread.arch.tid);
+	}
+
+	if (lkl_ops->tls_free)
+		lkl_ops->tls_free(task_key);
+}
diff --git a/arch/um/scripts/headers_install.py b/arch/um/scripts/headers_install.py
new file mode 100755
index 000000000000..ec7356c0dee9
--- /dev/null
+++ b/arch/um/scripts/headers_install.py
@@ -0,0 +1,197 @@
+#!/usr/bin/env python
+# SPDX-License-Identifier: GPL-2.0
+import re, os, sys, argparse, multiprocessing, fnmatch
+
+srctree = os.environ["srctree"]
+objtree = os.environ["objtree"]
+install_hdr_path = os.environ["INSTALL_HDR_PATH"]
+header_paths = [ install_hdr_path + "/include/", "arch/um/nommu/include/uapi/",
+                 "arch/um/nommu/include/generated/uapi/", "include/generated/" ]
+
+headers = set()
+includes = set()
+
+def relpath2abspath(relpath):
+    if "generated" in relpath or install_hdr_path in relpath:
+        return objtree + "/" + relpath
+    else:
+        return srctree + "/" + relpath
+
+def find_headers(path):
+    headers.add(path)
+    f = open(relpath2abspath(path))
+    for l in f.readlines():
+        m = re.search("#\s*include <(.*)>", l)
+        try:
+            i = m.group(1)
+            for p in header_paths:
+                if os.access(relpath2abspath(p + i), os.R_OK):
+                    if p + i not in headers:
+                        includes.add(i)
+                        headers.add(p + i)
+                        find_headers(p + i)
+        except:
+            pass
+    f.close()
+
+def has_lkl_prefix(w):
+  return w.startswith("lkl") or w.startswith("_lkl") or w.startswith("__lkl") \
+         or w.startswith("LKL") or w.startswith("_LKL") or w.startswith("__LKL")
+
+def find_symbols(regexp, store):
+    for h in headers:
+        f = open(h)
+        for l in f.readlines():
+            m = regexp.search(l)
+            if not m:
+                continue
+            for e in reversed(m.groups()):
+                if e:
+                    if not has_lkl_prefix(e):
+                        store.add(e)
+                    break
+        f.close()
+
+def find_ml_symbols(regexp, store):
+    for h in headers:
+        for i in regexp.finditer(open(h).read()):
+            for j in reversed(i.groups()):
+                if j:
+                    if not has_lkl_prefix(j):
+                        store.add(j)
+                    break
+
+def find_enums(block_regexp, symbol_regexp, store):
+    for h in headers:
+        # remove comments
+        content = re.sub(re.compile("(\/\*(\*(?!\/)|[^*])*\*\/)", re.S|re.M), " ", open(h).read())
+        # remove preprocesor lines
+        clean_content = ""
+        for l in content.split("\n"):
+            if re.match("\s*#", l):
+                continue
+            clean_content += l + "\n"
+        for i in block_regexp.finditer(clean_content):
+            for j in reversed(i.groups()):
+                if j:
+                    for k in symbol_regexp.finditer(j):
+                        for l in k.groups():
+                            if l:
+                                if not has_lkl_prefix(l):
+                                    store.add(l)
+                                break
+
+def lkl_prefix(w):
+    r = ""
+
+    if w.startswith("__"):
+        r = "__"
+    elif w.startswith("_"):
+        r = "_"
+
+    if w.isupper():
+        r += "LKL"
+    else:
+        r += "lkl"
+
+    if not w.startswith("_"):
+        r += "_"
+
+    r += w
+
+    return r
+
+def replace(h):
+    content = open(h).read()
+    for i in includes:
+        search_str = "(#[ \t]*include[ \t]*[<\"][ \t]*)" + i + "([ \t]*[>\"])"
+        replace_str = "\\1" + "lkl/" + i + "\\2"
+        content = re.sub(search_str, replace_str, content)
+    tmp = ""
+    for w in re.split("(\W+)", content):
+        if w in defines:
+            w = lkl_prefix(w)
+        tmp += w
+    content = tmp
+    for s in structs:
+        search_str = "(\W?struct\s+)" + s + "(\W)"
+        replace_str = "\\1" + lkl_prefix(s) + "\\2"
+        content = re.sub(search_str, replace_str, content, flags = re.MULTILINE)
+    for s in unions:
+        search_str = "(\W?union\s+)" + s + "(\W)"
+        replace_str = "\\1" + lkl_prefix(s) + "\\2"
+        content = re.sub(search_str, replace_str, content, flags = re.MULTILINE)
+    open(h, 'w').write(content)
+
+parser = argparse.ArgumentParser(description='install lkl headers')
+parser.add_argument('path', help='path to install to', )
+parser.add_argument('-j', '--jobs', help='number of parallel jobs', default=1, type=int)
+args = parser.parse_args()
+
+find_headers("arch/um/nommu/include/uapi/asm/syscalls.h")
+headers.add("arch/um/nommu/include/uapi/asm/host_ops.h")
+
+if 'LKL_INSTALL_ADDITIONAL_HEADERS' in os.environ:
+    with open(os.environ['LKL_INSTALL_ADDITIONAL_HEADERS'], 'rU') as f:
+        for line in f.readlines():
+            line = line.split('#', 1)[0].strip()
+            if line != '':
+                headers.add(line)
+
+new_headers = set()
+
+for h in headers:
+    h = relpath2abspath(h)
+    dir = os.path.dirname(h)
+    out_dir = args.path + "/" + re.sub("(" + srctree + "/arch/um/nommu/include/uapi/|arch/um/nommu/include/generated/uapi/|include/generated/uapi/|include/generated|" + install_hdr_path + "/include/)(.*)", "lkl/\\2", dir)
+    try:
+        os.makedirs(out_dir)
+    except:
+        pass
+    print("  INSTALL\t%s" % (out_dir + "/" + os.path.basename(h)))
+    os.system(srctree+"/scripts/headers_install.sh %s %s" % (os.path.abspath(h),
+                                                       out_dir + "/" + os.path.basename(h)))
+    new_headers.add(out_dir + "/" + os.path.basename(h))
+
+headers = new_headers
+
+defines = set()
+structs = set()
+unions = set()
+
+p = re.compile("#[ \t]*define[ \t]*(\w+)")
+find_symbols(p, defines)
+p = re.compile("typedef.*(\(\*(\w+)\)\(.*\)\s*|\W+(\w+)\s*|\s+(\w+)\(.*\)\s*);")
+find_symbols(p, defines)
+p = re.compile("typedef\s+(struct|union)\s+\w*\s*{[^\\{\}]*}\W*(\w+)\s*;", re.M|re.S)
+find_ml_symbols(p, defines)
+defines.add("siginfo_t")
+defines.add("sigevent_t")
+p = re.compile("struct\s+(\w+)\s*\{")
+find_symbols(p, structs)
+structs.add("iovec")
+p = re.compile("union\s+(\w+)\s*\{")
+find_symbols(p, unions)
+p = re.compile("static\s+__inline__(\s+\w+)+\s+(\w+)\([^)]*\)\s")
+find_symbols(p, defines)
+p = re.compile("static\s+__always_inline(\s+\w+)+\s+(\w+)\([^)]*\)\s")
+find_symbols(p, defines)
+p = re.compile("enum\s+(\w*)\s*{([^}]*)}", re.M|re.S)
+q = re.compile("(\w+)\s*(,|=[^,]*|$)", re.M|re.S)
+find_enums(p, q, defines)
+
+# needed for i386
+defines.add("__NR_stime")
+
+def process_header(h):
+    print("  REPLACE\t%s" % (out_dir + "/" + os.path.basename(h)))
+    replace(h)
+
+p = multiprocessing.Pool(args.jobs)
+try:
+    p.map_async(process_header, headers).wait(999999)
+    p.close()
+except:
+    p.terminate()
+finally:
+    p.join()
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v5 12/21] um: nommu: system call interface and application API
@ 2020-07-02 14:07       ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-07-02 14:07 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, linux-kernel-library, linux-arch,
	Hajime Tazaki, Akira Moroo

The application API is based on the kernel system call interface in
order to offer a stable API to applications. Note that we can't
offer the full Linux system call interface due to the limitations of
nommu mode such as lack of virtual memory, signal, user processes, etc.

The host is using dedicated kernel thread, which is switched upon the
entry of lkl_syscall so that numbers of context switches can be reduced
for the faster performance.

To expose the API definitions to applications, this commit uses
syscall_wrapper to define arch-specific information of syscall, and this
is used to generate syscall_defs.h for lkl so that it can be used
as a template of the list of syscall available for LKL apps.

To avoid collisions between the Linux API and the LKL API (e.g.  struct
stat, MKNOD, etc.) we use a python script to modify the user headers
and to prefix all of the global symbols (structures, typedefs,
defines) with LKL, lkl, _LKL, _lkl, __LKL or __lkl.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/include/uapi/asm/Kbuild             |   2 +
 arch/um/nommu/include/asm/syscall_wrapper.h |  57 ++++
 arch/um/nommu/include/asm/syscalls.h        |  15 +
 arch/um/nommu/include/uapi/asm/host_ops.h   |  14 +
 arch/um/nommu/include/uapi/asm/syscalls.h   | 287 ++++++++++++++++++++
 arch/um/nommu/include/uapi/asm/unistd.h     |  15 +
 arch/um/nommu/um/syscalls.c                 | 199 ++++++++++++++
 arch/um/scripts/headers_install.py          | 197 ++++++++++++++
 8 files changed, 786 insertions(+)
 create mode 100644 arch/um/include/uapi/asm/Kbuild
 create mode 100644 arch/um/nommu/include/asm/syscall_wrapper.h
 create mode 100644 arch/um/nommu/include/asm/syscalls.h
 create mode 100644 arch/um/nommu/include/uapi/asm/syscalls.h
 create mode 100644 arch/um/nommu/include/uapi/asm/unistd.h
 create mode 100644 arch/um/nommu/um/syscalls.c
 create mode 100755 arch/um/scripts/headers_install.py

diff --git a/arch/um/include/uapi/asm/Kbuild b/arch/um/include/uapi/asm/Kbuild
new file mode 100644
index 000000000000..a983c7f049bc
--- /dev/null
+++ b/arch/um/include/uapi/asm/Kbuild
@@ -0,0 +1,2 @@
+# no generated-y since we need special user headers handling
+# see arch/um/script/headers_install.py
diff --git a/arch/um/nommu/include/asm/syscall_wrapper.h b/arch/um/nommu/include/asm/syscall_wrapper.h
new file mode 100644
index 000000000000..bb2a7d4274f6
--- /dev/null
+++ b/arch/um/nommu/include/asm/syscall_wrapper.h
@@ -0,0 +1,57 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef __UM_SYSCALL_WRAPPER_H
+#define __UM_SYSCALL_WRAPPER_H
+
+#define __SC_ASCII(t, a) #t "," #a
+
+#define __ASCII_MAP0(m, ...)
+#define __ASCII_MAP1(m, t, a) m(t, a)
+#define __ASCII_MAP2(m, t, a, ...) m(t, a) "," __ASCII_MAP1(m, __VA_ARGS__)
+#define __ASCII_MAP3(m, t, a, ...) m(t, a) "," __ASCII_MAP2(m, __VA_ARGS__)
+#define __ASCII_MAP4(m, t, a, ...) m(t, a) "," __ASCII_MAP3(m, __VA_ARGS__)
+#define __ASCII_MAP5(m, t, a, ...) m(t, a) "," __ASCII_MAP4(m, __VA_ARGS__)
+#define __ASCII_MAP6(m, t, a, ...) m(t, a) "," __ASCII_MAP5(m, __VA_ARGS__)
+#define __ASCII_MAP(n, ...) __ASCII_MAP##n(__VA_ARGS__)
+
+#ifdef __MINGW32__
+#define SECTION_ATTRS "n0"
+#else
+#define SECTION_ATTRS "a"
+#endif
+
+#define __SYSCALL_DEFINE_ARCH(x, name, ...)				\
+	asm(".section .syscall_defs,\"" SECTION_ATTRS "\"\n"		\
+	    ".ascii \"#ifdef __NR" #name "\\n\"\n"			\
+	    ".ascii \"SYSCALL_DEFINE" #x "(" #name ","			\
+	    __ASCII_MAP(x, __SC_ASCII, __VA_ARGS__) ")\\n\"\n"		\
+	    ".ascii \"#endif\\n\"\n"					\
+	    ".section .text\n")
+
+#define SYSCALL_DEFINE0(sname)					\
+	SYSCALL_METADATA(_##sname, 0);				\
+	__SYSCALL_DEFINE_ARCH(0, _##sname);			\
+	asmlinkage long sys_##sname(void);			\
+	ALLOW_ERROR_INJECTION(sys_##sname, ERRNO);		\
+	asmlinkage long sys_##sname(void)
+
+#define __SYSCALL_DEFINEx(x, name, ...)					\
+	__SYSCALL_DEFINE_ARCH(x, name, __VA_ARGS__);			\
+	__diag_push();							\
+	__diag_ignore(GCC, 8, "-Wattribute-alias",			\
+		      "Type aliasing is used to sanitize syscall arguments");\
+	asmlinkage long sys##name(__MAP(x, __SC_DECL, __VA_ARGS__))	\
+		__attribute__((alias(__stringify(__se_sys##name))));	\
+	ALLOW_ERROR_INJECTION(sys##name, ERRNO);			\
+	static inline long __do_sys##name(__MAP(x, __SC_DECL, __VA_ARGS__)); \
+	asmlinkage long __se_sys##name(__MAP(x, __SC_LONG, __VA_ARGS__)); \
+	asmlinkage long __se_sys##name(__MAP(x, __SC_LONG, __VA_ARGS__)) \
+	{								\
+		long ret = __do_sys##name(__MAP(x, __SC_CAST, __VA_ARGS__)); \
+		__MAP(x, __SC_TEST, __VA_ARGS__);			\
+		__PROTECT(x, ret, __MAP(x, __SC_ARGS, __VA_ARGS__));	\
+		return ret;						\
+	}								\
+	__diag_pop();							\
+	static inline long __do_sys##name(__MAP(x, __SC_DECL, __VA_ARGS__))
+
+#endif /* __UM_SYSCALL_WRAPPER_H */
diff --git a/arch/um/nommu/include/asm/syscalls.h b/arch/um/nommu/include/asm/syscalls.h
new file mode 100644
index 000000000000..6061d9415dad
--- /dev/null
+++ b/arch/um/nommu/include/asm/syscalls.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+int syscalls_init(void);
+void syscalls_cleanup(void);
+long lkl_syscall(long no, long *params);
+void wakeup_idle_host_task(void);
+
+#define sys_mmap sys_ni_syscall
+#define sys_rt_sigreturn sys_ni_syscall
+#define sys_arch_prctl sys_ni_syscall
+#define sys_iopl sys_ni_syscall
+#define sys_ioperm sys_ni_syscall
+#define sys_clone sys_ni_syscall
+
+int run_syscalls(void);
diff --git a/arch/um/nommu/include/uapi/asm/host_ops.h b/arch/um/nommu/include/uapi/asm/host_ops.h
index 720385fccbdf..6ee9489fc47e 100644
--- a/arch/um/nommu/include/uapi/asm/host_ops.h
+++ b/arch/um/nommu/include/uapi/asm/host_ops.h
@@ -38,6 +38,15 @@ struct lkl_jmp_buf {
  * @gettid - returns the host thread id of the caller, which need not
  * be the same as the handle returned by thread_create
  *
+ * @tls_alloc - allocate a thread local storage key; returns 0 if successful; if
+ * destructor is not NULL it will be called when a thread terminates with its
+ * argument set to the current thread local storage value
+ * @tls_free - frees a thread local storage key; returns 0 if successful
+ * @tls_set - associate data to the thread local storage key; returns 0 if
+ * successful
+ * @tls_get - return data associated with the thread local storage key or NULL
+ * on error
+ *
  * @jmp_buf_set - runs the give function and setups a jump back point by saving
  * the context in the jump buffer; jmp_buf_longjmp can be called from the give
  * function or any callee in that function to return back to the jump back
@@ -72,6 +81,11 @@ struct lkl_host_operations {
 	int (*thread_equal)(lkl_thread_t a, lkl_thread_t b);
 	long (*gettid)(void);
 
+	struct lkl_tls_key *(*tls_alloc)(void (*destructor)(void *));
+	void (*tls_free)(struct lkl_tls_key *key);
+	int (*tls_set)(struct lkl_tls_key *key, void *data);
+	void *(*tls_get)(struct lkl_tls_key *key);
+
 	void (*jmp_buf_set)(struct lkl_jmp_buf *jmpb, void (*f)(void));
 	void (*jmp_buf_longjmp)(struct lkl_jmp_buf *jmpb, int val);
 
diff --git a/arch/um/nommu/include/uapi/asm/syscalls.h b/arch/um/nommu/include/uapi/asm/syscalls.h
new file mode 100644
index 000000000000..751957b978cd
--- /dev/null
+++ b/arch/um/nommu/include/uapi/asm/syscalls.h
@@ -0,0 +1,287 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef __UM_NOMMU_UAPI_SYSCALLS_H
+#define __UM_NOMMU_UAPI_SYSCALLS_H
+
+#include <autoconf.h>
+#include <linux/types.h>
+
+typedef __kernel_uid32_t	qid_t;
+typedef __kernel_fd_set		fd_set;
+typedef __kernel_mode_t		mode_t;
+typedef unsigned short		umode_t;
+typedef __u32			nlink_t;
+typedef __kernel_off_t		off_t;
+typedef __kernel_pid_t		pid_t;
+typedef __kernel_key_t		key_t;
+typedef __kernel_suseconds_t	suseconds_t;
+typedef __kernel_timer_t	timer_t;
+typedef __kernel_clockid_t	clockid_t;
+typedef __kernel_mqd_t		mqd_t;
+typedef __kernel_uid32_t	uid_t;
+typedef __kernel_gid32_t	gid_t;
+typedef __kernel_uid16_t        uid16_t;
+typedef __kernel_gid16_t        gid16_t;
+typedef unsigned long		uintptr_t;
+#ifdef CONFIG_UID16
+typedef __kernel_old_uid_t	old_uid_t;
+typedef __kernel_old_gid_t	old_gid_t;
+#endif
+typedef __kernel_loff_t		loff_t;
+typedef __kernel_size_t		size_t;
+typedef __kernel_ssize_t	ssize_t;
+typedef __kernel_time_t		time_t;
+typedef __kernel_clock_t	clock_t;
+typedef __u32			u32;
+typedef __s32			s32;
+typedef __u64			u64;
+typedef __s64			s64;
+
+typedef __kernel_long_t	__kernel_old_time_t;
+
+#define __user
+
+#include <asm/unistd.h>
+/* Temporary undefine system calls that don't have data types defined in UAPI
+ * headers
+ */
+#undef __NR_kexec_load
+#undef __NR_getcpu
+#undef __NR_sched_getattr
+#undef __NR_sched_setattr
+#undef __NR_sched_setparam
+#undef __NR_sched_getparam
+#undef __NR_sched_setscheduler
+#undef __NR_name_to_handle_at
+#undef __NR_open_by_handle_at
+
+/* deprecated system calls */
+#undef __NR_access
+#undef __NR_chmod
+#undef __NR_chown
+#undef __NR_lchown
+#undef __NR_open
+#undef __NR_creat
+#undef __NR_readlink
+#undef __NR_pipe
+#undef __NR_mknod
+#undef __NR_mkdir
+#undef __NR_rmdir
+#undef __NR_unlink
+#undef __NR_symlink
+#undef __NR_link
+#undef __NR_rename
+#undef __NR_getdents
+#undef __NR_select
+#undef __NR_poll
+#undef __NR_dup2
+#undef __NR_sysfs
+#undef __NR_ustat
+#undef __NR_inotify_init
+#undef __NR_epoll_create
+#undef __NR_epoll_wait
+#undef __NR_signalfd
+#undef __NR_eventfd
+
+#undef __NR_umount
+#define __NR_umount __NR_umount2
+
+#ifdef CONFIG_64BIT
+#define __NR_newfstat __NR3264_fstat
+#define __NR_newfstatat __NR3264_fstatat
+#endif
+
+#include <linux/time.h>
+#include <linux/times.h>
+#include <linux/timex.h>
+#include <linux/capability.h>
+#define __KERNEL__ /* to pull in S_ definitions */
+#include <linux/stat.h>
+#undef __KERNEL__
+#include <linux/errno.h>
+#include <linux/fcntl.h>
+#include <linux/fs.h>
+#include <asm/statfs.h>
+#include <asm/stat.h>
+#include <linux/bpf.h>
+#include <linux/msg.h>
+#include <linux/resource.h>
+#include <linux/sysinfo.h>
+#include <linux/shm.h>
+#include <linux/aio_abi.h>
+#include <linux/socket.h>
+#include <linux/perf_event.h>
+#include <linux/sem.h>
+#include <linux/futex.h>
+#include <linux/poll.h>
+#include <linux/mqueue.h>
+#include <linux/eventpoll.h>
+#include <linux/uio.h>
+#include <asm/signal.h>
+#include <asm/siginfo.h>
+#include <linux/utime.h>
+#include <asm/socket.h>
+#include <linux/icmp.h>
+#include <linux/ip.h>
+
+/* Define data structures used in system calls that are not defined in UAPI
+ * headers
+ */
+struct sockaddr {
+	unsigned short int sa_family;
+	char sa_data[14];
+};
+
+#define __UAPI_DEF_IF_NET_DEVICE_FLAGS_LOWER_UP_DORMANT_ECHO 1
+#define __UAPI_DEF_IF_IFNAMSIZ	1
+#define __UAPI_DEF_IF_NET_DEVICE_FLAGS 1
+#define __UAPI_DEF_IF_IFREQ	1
+#define __UAPI_DEF_IF_IFMAP	1
+#include <linux/if.h>
+#define __UAPI_DEF_IN_IPPROTO	1
+#define __UAPI_DEF_IN_ADDR	1
+#define __UAPI_DEF_IN6_ADDR	1
+#define __UAPI_DEF_IP_MREQ	1
+#define __UAPI_DEF_IN_PKTINFO	1
+#define __UAPI_DEF_SOCKADDR_IN	1
+#define __UAPI_DEF_IN_CLASS	1
+#include <linux/in.h>
+#include <linux/in6.h>
+#include <linux/sockios.h>
+#include <linux/route.h>
+#include <linux/ipv6_route.h>
+#include <linux/ipv6.h>
+#include <linux/netlink.h>
+#include <linux/neighbour.h>
+#include <linux/rtnetlink.h>
+#include <linux/fib_rules.h>
+
+#include <linux/kdev_t.h>
+#include <linux/virtio_blk.h>
+#include <linux/virtio_net.h>
+#include <linux/virtio_ring.h>
+#include <linux/pkt_sched.h>
+#include <linux/io_uring.h>
+#include <linux/utsname.h>
+
+struct user_msghdr {
+	void		__user *msg_name;
+	int		msg_namelen;
+	struct iovec	__user *msg_iov;
+	__kernel_size_t	msg_iovlen;
+	void		__user *msg_control;
+	__kernel_size_t	msg_controllen;
+	unsigned int	msg_flags;
+};
+
+typedef __u32 key_serial_t;
+
+struct mmsghdr {
+	struct user_msghdr  msg_hdr;
+	unsigned int        msg_len;
+};
+
+struct linux_dirent64 {
+	u64		d_ino;
+	s64		d_off;
+	unsigned short	d_reclen;
+	unsigned char	d_type;
+	char		d_name[0];
+};
+
+struct linux_dirent {
+	unsigned long	d_ino;
+	unsigned long	d_off;
+	unsigned short	d_reclen;
+	char		d_name[1];
+};
+
+struct ustat {
+	__kernel_daddr_t	f_tfree;
+	__kernel_ino_t		f_tinode;
+	char			f_fname[6];
+	char			f_fpack[6];
+};
+
+struct __aio_sigset;
+struct clone_args;
+
+typedef __kernel_rwf_t		rwf_t;
+
+long lkl_syscall(long no, long *params);
+long lkl_sys_halt(void);
+
+#define __MAP0(m, ...)
+#define __MAP1(m, t, a) m(t, a)
+#define __MAP2(m, t, a, ...) m(t, a), __MAP1(m, __VA_ARGS__)
+#define __MAP3(m, t, a, ...) m(t, a), __MAP2(m, __VA_ARGS__)
+#define __MAP4(m, t, a, ...) m(t, a), __MAP3(m, __VA_ARGS__)
+#define __MAP5(m, t, a, ...) m(t, a), __MAP4(m, __VA_ARGS__)
+#define __MAP6(m, t, a, ...) m(t, a), __MAP5(m, __VA_ARGS__)
+#define __MAP(n, ...) __MAP##n(__VA_ARGS__)
+
+#define __SC_LONG(t, a) (long)(a)
+#define __SC_TABLE(t, a) {sizeof(t), (long long)(a)}
+#define __SC_DECL(t, a) t a
+
+#define LKL_SYSCALL0(name)					       \
+	static inline long lkl_sys##name(void)			       \
+	{							       \
+		long params[6];					       \
+		return lkl_syscall(__lkl__NR##name, params);	       \
+	}
+
+#if __BITS_PER_LONG == 32
+#define LKL_SYSCALLx(x, name, ...)					\
+	static inline							\
+	long lkl_sys##name(__MAP(x, __SC_DECL, __VA_ARGS__))		\
+	{								\
+		struct {						\
+			unsigned int size;				\
+			long long value;				\
+		} lkl_params[x] = { __MAP(x, __SC_TABLE, __VA_ARGS__) }; \
+		long sys_params[6], i, k;				\
+		for (i = k = 0; i < x && k < 6; i++, k++) {		\
+			if (lkl_params[i].size > sizeof(long) &&	\
+			    k + 1 < 6) {				\
+				sys_params[k] =				\
+					(long)(lkl_params[i].value & (-1UL)); \
+				k++;					\
+				sys_params[k] =				\
+					(long)(lkl_params[i].value >>	\
+					       __BITS_PER_LONG);	\
+			} else {					\
+				sys_params[k] = (long)(lkl_params[i].value); \
+			}						\
+		}							\
+		return lkl_syscall(__lkl__NR##name, sys_params);	\
+	}
+#else
+#define LKL_SYSCALLx(x, name, ...)					\
+	static inline							\
+	long lkl_sys##name(__MAP(x, __SC_DECL, __VA_ARGS__))		\
+	{								\
+		long lkl_params[6] = { __MAP(x, __SC_LONG, __VA_ARGS__) }; \
+		return lkl_syscall(__lkl__NR##name, lkl_params);	\
+	}
+#endif
+
+#define SYSCALL_DEFINE0(name, ...) LKL_SYSCALL0(name)
+#define SYSCALL_DEFINE1(name, ...) LKL_SYSCALLx(1, name, __VA_ARGS__)
+#define SYSCALL_DEFINE2(name, ...) LKL_SYSCALLx(2, name, __VA_ARGS__)
+#define SYSCALL_DEFINE3(name, ...) LKL_SYSCALLx(3, name, __VA_ARGS__)
+#define SYSCALL_DEFINE4(name, ...) LKL_SYSCALLx(4, name, __VA_ARGS__)
+#define SYSCALL_DEFINE5(name, ...) LKL_SYSCALLx(5, name, __VA_ARGS__)
+#define SYSCALL_DEFINE6(name, ...) LKL_SYSCALLx(6, name, __VA_ARGS__)
+
+#if __BITS_PER_LONG == 32
+#pragma GCC diagnostic push
+#pragma GCC diagnostic ignored "-Wpointer-to-int-cast"
+#endif
+
+#include <asm/syscall_defs.h>
+
+#if __BITS_PER_LONG == 32
+#pragma GCC diagnostic pop
+#endif
+
+#endif
diff --git a/arch/um/nommu/include/uapi/asm/unistd.h b/arch/um/nommu/include/uapi/asm/unistd.h
new file mode 100644
index 000000000000..1762ebb829f5
--- /dev/null
+++ b/arch/um/nommu/include/uapi/asm/unistd.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef __UM_NOMMU_UAPI_UNISTD_H
+#define __UM_NOMMU_UAPI_UNISTD_H
+
+#define __ARCH_WANT_NEW_STAT
+#include <asm/bitsperlong.h>
+
+#if __BITS_PER_LONG == 64
+#define __ARCH_WANT_SYS_NEWFSTATAT
+#endif
+
+#include <asm-generic/unistd.h>
+
+
+#endif
diff --git a/arch/um/nommu/um/syscalls.c b/arch/um/nommu/um/syscalls.c
new file mode 100644
index 000000000000..fd343f855bad
--- /dev/null
+++ b/arch/um/nommu/um/syscalls.c
@@ -0,0 +1,199 @@
+// SPDX-License-Identifier: GPL-2.0
+#ifdef CONFIG_ARCH_HAS_SYSCALL_WRAPPER
+#undef CONFIG_ARCH_HAS_SYSCALL_WRAPPER
+#include <linux/syscalls.h>
+#define CONFIG_ARCH_HAS_SYSCALL_WRAPPER
+#endif
+
+#include <linux/stat.h>
+#include <linux/irq.h>
+#include <linux/sched.h>
+#include <linux/interrupt.h>
+#include <linux/jhash.h>
+#include <linux/slab.h>
+#include <linux/types.h>
+#include <linux/net.h>
+#include <linux/task_work.h>
+
+#include <asm/host_ops.h>
+#include <asm/syscalls.h>
+#include <asm/cpu.h>
+#include <asm/sched.h>
+#include <kern_util.h>
+#include <os.h>
+
+typedef long (*syscall_handler_t)(long arg1, ...);
+
+#undef __SYSCALL
+#define __SYSCALL(nr, sym)[nr] = (syscall_handler_t)sym,
+
+syscall_handler_t syscall_table[__NR_syscalls] = {
+	[0 ... __NR_syscalls - 1] =  (syscall_handler_t)sys_ni_syscall,
+#undef __UM_NOMMU_UAPI_UNISTD_H
+#include <asm/unistd.h>
+
+#if __BITS_PER_LONG == 32
+#include <asm/unistd_32.h>
+#endif
+};
+
+
+static long run_syscall(long no, long *params)
+{
+	long ret;
+
+	if (no < 0 || no >= __NR_syscalls)
+		return -ENOSYS;
+
+	ret = syscall_table[no](params[0], params[1], params[2], params[3],
+				params[4], params[5]);
+
+	task_work_run();
+
+	return ret;
+}
+
+
+#define CLONE_FLAGS (CLONE_VM | CLONE_FS | CLONE_FILES | CLONE_THREAD |	\
+		     CLONE_SIGHAND | SIGCHLD)
+
+static int host_task_id;
+static struct task_struct *host0;
+
+static int new_host_task(struct task_struct **task)
+{
+	pid_t pid;
+
+	switch_to_host_task(host0);
+
+	pid = kernel_thread(host_task_stub, NULL, CLONE_FLAGS);
+	if (pid < 0)
+		return pid;
+
+	rcu_read_lock();
+	*task = find_task_by_pid_ns(pid, &init_pid_ns);
+	rcu_read_unlock();
+
+	host_task_id++;
+
+	snprintf((*task)->comm, sizeof((*task)->comm), "host%d", host_task_id);
+
+	return 0;
+}
+static void exit_task(void)
+{
+	do_exit(0);
+}
+
+static void del_host_task(void *arg)
+{
+	struct task_struct *task = (struct task_struct *)arg;
+	struct thread_info *ti = task_thread_info(task);
+
+	if (lkl_cpu_get() < 0)
+		return;
+
+	switch_to_host_task(task);
+	host_task_id--;
+	set_ti_thread_flag(ti, TIF_SCHED_JB);
+	lkl_ops->jmp_buf_set(&ti->task->thread.arch.sched_jb, exit_task);
+}
+
+static struct lkl_tls_key *task_key;
+
+long lkl_syscall(long no, long *params)
+{
+	struct task_struct *task = host0;
+	long ret;
+
+	ret = lkl_cpu_get();
+	if (ret < 0)
+		return ret;
+
+	if (lkl_ops->tls_get) {
+		task = lkl_ops->tls_get(task_key);
+		if (!task) {
+			ret = new_host_task(&task);
+			if (ret)
+				goto out;
+			lkl_ops->tls_set(task_key, task);
+		}
+	}
+
+	switch_to_host_task(task);
+
+	ret = run_syscall(no, params);
+
+	if (no == __NR_reboot) {
+		thread_sched_jb();
+		return ret;
+	}
+
+out:
+	lkl_cpu_put();
+
+	return ret;
+}
+
+static struct task_struct *idle_host_task;
+
+/* called from idle, don't failed, don't block */
+void wakeup_idle_host_task(void)
+{
+	if (!need_resched() && idle_host_task)
+		wake_up_process(idle_host_task);
+}
+
+static int idle_host_task_loop(void *unused)
+{
+	struct thread_info *ti = task_thread_info(current);
+
+	snprintf(current->comm, sizeof(current->comm), "idle_host_task");
+	set_thread_flag(TIF_HOST_THREAD);
+	idle_host_task = current;
+
+	for (;;) {
+		lkl_cpu_put();
+		lkl_ops->sem_down(ti->task->thread.arch.sched_sem);
+		if (idle_host_task == NULL) {
+			lkl_ops->thread_exit();
+			return 0;
+		}
+		schedule_tail(ti->task->thread.prev_sched);
+	}
+}
+
+int syscalls_init(void)
+{
+	snprintf(current->comm, sizeof(current->comm), "host0");
+	set_thread_flag(TIF_HOST_THREAD);
+	host0 = current;
+
+	if (lkl_ops->tls_alloc) {
+		task_key = lkl_ops->tls_alloc(del_host_task);
+		if (!task_key)
+			return -1;
+	}
+
+	if (kernel_thread(idle_host_task_loop, NULL, CLONE_FLAGS) < 0) {
+		if (lkl_ops->tls_free)
+			lkl_ops->tls_free(task_key);
+		return -1;
+	}
+
+	return 0;
+}
+
+void syscalls_cleanup(void)
+{
+	if (idle_host_task) {
+		struct thread_info *ti = task_thread_info(idle_host_task);
+
+		idle_host_task = NULL;
+		lkl_ops->sem_up(ti->task->thread.arch.sched_sem);
+		lkl_ops->thread_join(ti->task->thread.arch.tid);
+	}
+
+	if (lkl_ops->tls_free)
+		lkl_ops->tls_free(task_key);
+}
diff --git a/arch/um/scripts/headers_install.py b/arch/um/scripts/headers_install.py
new file mode 100755
index 000000000000..ec7356c0dee9
--- /dev/null
+++ b/arch/um/scripts/headers_install.py
@@ -0,0 +1,197 @@
+#!/usr/bin/env python
+# SPDX-License-Identifier: GPL-2.0
+import re, os, sys, argparse, multiprocessing, fnmatch
+
+srctree = os.environ["srctree"]
+objtree = os.environ["objtree"]
+install_hdr_path = os.environ["INSTALL_HDR_PATH"]
+header_paths = [ install_hdr_path + "/include/", "arch/um/nommu/include/uapi/",
+                 "arch/um/nommu/include/generated/uapi/", "include/generated/" ]
+
+headers = set()
+includes = set()
+
+def relpath2abspath(relpath):
+    if "generated" in relpath or install_hdr_path in relpath:
+        return objtree + "/" + relpath
+    else:
+        return srctree + "/" + relpath
+
+def find_headers(path):
+    headers.add(path)
+    f = open(relpath2abspath(path))
+    for l in f.readlines():
+        m = re.search("#\s*include <(.*)>", l)
+        try:
+            i = m.group(1)
+            for p in header_paths:
+                if os.access(relpath2abspath(p + i), os.R_OK):
+                    if p + i not in headers:
+                        includes.add(i)
+                        headers.add(p + i)
+                        find_headers(p + i)
+        except:
+            pass
+    f.close()
+
+def has_lkl_prefix(w):
+  return w.startswith("lkl") or w.startswith("_lkl") or w.startswith("__lkl") \
+         or w.startswith("LKL") or w.startswith("_LKL") or w.startswith("__LKL")
+
+def find_symbols(regexp, store):
+    for h in headers:
+        f = open(h)
+        for l in f.readlines():
+            m = regexp.search(l)
+            if not m:
+                continue
+            for e in reversed(m.groups()):
+                if e:
+                    if not has_lkl_prefix(e):
+                        store.add(e)
+                    break
+        f.close()
+
+def find_ml_symbols(regexp, store):
+    for h in headers:
+        for i in regexp.finditer(open(h).read()):
+            for j in reversed(i.groups()):
+                if j:
+                    if not has_lkl_prefix(j):
+                        store.add(j)
+                    break
+
+def find_enums(block_regexp, symbol_regexp, store):
+    for h in headers:
+        # remove comments
+        content = re.sub(re.compile("(\/\*(\*(?!\/)|[^*])*\*\/)", re.S|re.M), " ", open(h).read())
+        # remove preprocesor lines
+        clean_content = ""
+        for l in content.split("\n"):
+            if re.match("\s*#", l):
+                continue
+            clean_content += l + "\n"
+        for i in block_regexp.finditer(clean_content):
+            for j in reversed(i.groups()):
+                if j:
+                    for k in symbol_regexp.finditer(j):
+                        for l in k.groups():
+                            if l:
+                                if not has_lkl_prefix(l):
+                                    store.add(l)
+                                break
+
+def lkl_prefix(w):
+    r = ""
+
+    if w.startswith("__"):
+        r = "__"
+    elif w.startswith("_"):
+        r = "_"
+
+    if w.isupper():
+        r += "LKL"
+    else:
+        r += "lkl"
+
+    if not w.startswith("_"):
+        r += "_"
+
+    r += w
+
+    return r
+
+def replace(h):
+    content = open(h).read()
+    for i in includes:
+        search_str = "(#[ \t]*include[ \t]*[<\"][ \t]*)" + i + "([ \t]*[>\"])"
+        replace_str = "\\1" + "lkl/" + i + "\\2"
+        content = re.sub(search_str, replace_str, content)
+    tmp = ""
+    for w in re.split("(\W+)", content):
+        if w in defines:
+            w = lkl_prefix(w)
+        tmp += w
+    content = tmp
+    for s in structs:
+        search_str = "(\W?struct\s+)" + s + "(\W)"
+        replace_str = "\\1" + lkl_prefix(s) + "\\2"
+        content = re.sub(search_str, replace_str, content, flags = re.MULTILINE)
+    for s in unions:
+        search_str = "(\W?union\s+)" + s + "(\W)"
+        replace_str = "\\1" + lkl_prefix(s) + "\\2"
+        content = re.sub(search_str, replace_str, content, flags = re.MULTILINE)
+    open(h, 'w').write(content)
+
+parser = argparse.ArgumentParser(description='install lkl headers')
+parser.add_argument('path', help='path to install to', )
+parser.add_argument('-j', '--jobs', help='number of parallel jobs', default=1, type=int)
+args = parser.parse_args()
+
+find_headers("arch/um/nommu/include/uapi/asm/syscalls.h")
+headers.add("arch/um/nommu/include/uapi/asm/host_ops.h")
+
+if 'LKL_INSTALL_ADDITIONAL_HEADERS' in os.environ:
+    with open(os.environ['LKL_INSTALL_ADDITIONAL_HEADERS'], 'rU') as f:
+        for line in f.readlines():
+            line = line.split('#', 1)[0].strip()
+            if line != '':
+                headers.add(line)
+
+new_headers = set()
+
+for h in headers:
+    h = relpath2abspath(h)
+    dir = os.path.dirname(h)
+    out_dir = args.path + "/" + re.sub("(" + srctree + "/arch/um/nommu/include/uapi/|arch/um/nommu/include/generated/uapi/|include/generated/uapi/|include/generated|" + install_hdr_path + "/include/)(.*)", "lkl/\\2", dir)
+    try:
+        os.makedirs(out_dir)
+    except:
+        pass
+    print("  INSTALL\t%s" % (out_dir + "/" + os.path.basename(h)))
+    os.system(srctree+"/scripts/headers_install.sh %s %s" % (os.path.abspath(h),
+                                                       out_dir + "/" + os.path.basename(h)))
+    new_headers.add(out_dir + "/" + os.path.basename(h))
+
+headers = new_headers
+
+defines = set()
+structs = set()
+unions = set()
+
+p = re.compile("#[ \t]*define[ \t]*(\w+)")
+find_symbols(p, defines)
+p = re.compile("typedef.*(\(\*(\w+)\)\(.*\)\s*|\W+(\w+)\s*|\s+(\w+)\(.*\)\s*);")
+find_symbols(p, defines)
+p = re.compile("typedef\s+(struct|union)\s+\w*\s*{[^\\{\}]*}\W*(\w+)\s*;", re.M|re.S)
+find_ml_symbols(p, defines)
+defines.add("siginfo_t")
+defines.add("sigevent_t")
+p = re.compile("struct\s+(\w+)\s*\{")
+find_symbols(p, structs)
+structs.add("iovec")
+p = re.compile("union\s+(\w+)\s*\{")
+find_symbols(p, unions)
+p = re.compile("static\s+__inline__(\s+\w+)+\s+(\w+)\([^)]*\)\s")
+find_symbols(p, defines)
+p = re.compile("static\s+__always_inline(\s+\w+)+\s+(\w+)\([^)]*\)\s")
+find_symbols(p, defines)
+p = re.compile("enum\s+(\w*)\s*{([^}]*)}", re.M|re.S)
+q = re.compile("(\w+)\s*(,|=[^,]*|$)", re.M|re.S)
+find_enums(p, q, defines)
+
+# needed for i386
+defines.add("__NR_stime")
+
+def process_header(h):
+    print("  REPLACE\t%s" % (out_dir + "/" + os.path.basename(h)))
+    replace(h)
+
+p = multiprocessing.Pool(args.jobs)
+try:
+    p.map_async(process_header, headers).wait(999999)
+    p.close()
+except:
+    p.terminate()
+finally:
+    p.join()
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v5 13/21] um: nommu: basic console support
  2020-07-02 14:06     ` Hajime Tazaki
@ 2020-07-02 14:07       ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-07-02 14:07 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Hajime Tazaki

This commit also disables stdio-console of UM if !CONFIG_MMU to avoid
conflicts between multiple consoles.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/drivers/Makefile                  |  8 ++++-
 arch/um/nommu/include/uapi/asm/host_ops.h |  3 ++
 arch/um/nommu/um/console.c                | 42 +++++++++++++++++++++++
 3 files changed, 52 insertions(+), 1 deletion(-)
 create mode 100644 arch/um/nommu/um/console.c

diff --git a/arch/um/drivers/Makefile b/arch/um/drivers/Makefile
index ae96c83e312d..694da4aa550d 100644
--- a/arch/um/drivers/Makefile
+++ b/arch/um/drivers/Makefile
@@ -37,7 +37,13 @@ $(obj)/vde.o: $(obj)/vde_kern.o $(obj)/vde_user.o
 # When the above is fixed, don't forget to add this too!
 #targets += $(obj)/pcap.o
 
-obj-y := stdio_console.o fd.o chan_kern.o chan_user.o line.o
+ifdef CONFIG_MMU
+obj-y := stdio_console.o
+else
+obj-y :=
+endif
+obj-y += fd.o chan_kern.o chan_user.o line.o
+
 obj-$(CONFIG_SSL) += ssl.o
 obj-$(CONFIG_STDERR_CONSOLE) += stderr_console.o
 
diff --git a/arch/um/nommu/include/uapi/asm/host_ops.h b/arch/um/nommu/include/uapi/asm/host_ops.h
index 6ee9489fc47e..133877c857a1 100644
--- a/arch/um/nommu/include/uapi/asm/host_ops.h
+++ b/arch/um/nommu/include/uapi/asm/host_ops.h
@@ -15,6 +15,7 @@ struct lkl_jmp_buf {
  *
  * These operations must be provided by a host library or by the application
  * itself.
+ * @print - optional operation that receives console messages
  *
  * @sem_alloc - allocate a host semaphore an initialize it to count
  * @sem_free - free a host semaphore
@@ -63,6 +64,8 @@ struct lkl_jmp_buf {
  *
  */
 struct lkl_host_operations {
+	void (*print)(const char *str, int len);
+
 	struct lkl_sem *(*sem_alloc)(int count);
 	void (*sem_free)(struct lkl_sem *sem);
 	void (*sem_up)(struct lkl_sem *sem);
diff --git a/arch/um/nommu/um/console.c b/arch/um/nommu/um/console.c
new file mode 100644
index 000000000000..e3194f7bb0dd
--- /dev/null
+++ b/arch/um/nommu/um/console.c
@@ -0,0 +1,42 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/console.h>
+#include <asm/host_ops.h>
+
+static void console_write(struct console *con, const char *str, unsigned len)
+{
+	if (lkl_ops->print)
+		lkl_ops->print(str, len);
+}
+
+#ifdef CONFIG_LKL_EARLY_CONSOLE
+static struct console lkl_boot_console = {
+	.name	= "lkl_boot_console",
+	.write	= console_write,
+	.flags	= CON_PRINTBUFFER | CON_BOOT,
+	.index	= -1,
+};
+
+int __init lkl_boot_console_init(void)
+{
+	register_console(&lkl_boot_console);
+	return 0;
+}
+early_initcall(lkl_boot_console_init);
+#endif
+
+static struct console lkl_console = {
+	.name	= "lkl_console",
+	.write	= console_write,
+	.flags	= CON_PRINTBUFFER,
+	.index	= -1,
+};
+
+int __init lkl_console_init(void)
+{
+	register_console(&lkl_console);
+	return 0;
+}
+core_initcall(lkl_console_init);
+
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v5 13/21] um: nommu: basic console support
@ 2020-07-02 14:07       ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-07-02 14:07 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, linux-kernel-library, linux-arch,
	Hajime Tazaki, Akira Moroo

This commit also disables stdio-console of UM if !CONFIG_MMU to avoid
conflicts between multiple consoles.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/drivers/Makefile                  |  8 ++++-
 arch/um/nommu/include/uapi/asm/host_ops.h |  3 ++
 arch/um/nommu/um/console.c                | 42 +++++++++++++++++++++++
 3 files changed, 52 insertions(+), 1 deletion(-)
 create mode 100644 arch/um/nommu/um/console.c

diff --git a/arch/um/drivers/Makefile b/arch/um/drivers/Makefile
index ae96c83e312d..694da4aa550d 100644
--- a/arch/um/drivers/Makefile
+++ b/arch/um/drivers/Makefile
@@ -37,7 +37,13 @@ $(obj)/vde.o: $(obj)/vde_kern.o $(obj)/vde_user.o
 # When the above is fixed, don't forget to add this too!
 #targets += $(obj)/pcap.o
 
-obj-y := stdio_console.o fd.o chan_kern.o chan_user.o line.o
+ifdef CONFIG_MMU
+obj-y := stdio_console.o
+else
+obj-y :=
+endif
+obj-y += fd.o chan_kern.o chan_user.o line.o
+
 obj-$(CONFIG_SSL) += ssl.o
 obj-$(CONFIG_STDERR_CONSOLE) += stderr_console.o
 
diff --git a/arch/um/nommu/include/uapi/asm/host_ops.h b/arch/um/nommu/include/uapi/asm/host_ops.h
index 6ee9489fc47e..133877c857a1 100644
--- a/arch/um/nommu/include/uapi/asm/host_ops.h
+++ b/arch/um/nommu/include/uapi/asm/host_ops.h
@@ -15,6 +15,7 @@ struct lkl_jmp_buf {
  *
  * These operations must be provided by a host library or by the application
  * itself.
+ * @print - optional operation that receives console messages
  *
  * @sem_alloc - allocate a host semaphore an initialize it to count
  * @sem_free - free a host semaphore
@@ -63,6 +64,8 @@ struct lkl_jmp_buf {
  *
  */
 struct lkl_host_operations {
+	void (*print)(const char *str, int len);
+
 	struct lkl_sem *(*sem_alloc)(int count);
 	void (*sem_free)(struct lkl_sem *sem);
 	void (*sem_up)(struct lkl_sem *sem);
diff --git a/arch/um/nommu/um/console.c b/arch/um/nommu/um/console.c
new file mode 100644
index 000000000000..e3194f7bb0dd
--- /dev/null
+++ b/arch/um/nommu/um/console.c
@@ -0,0 +1,42 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/console.h>
+#include <asm/host_ops.h>
+
+static void console_write(struct console *con, const char *str, unsigned len)
+{
+	if (lkl_ops->print)
+		lkl_ops->print(str, len);
+}
+
+#ifdef CONFIG_LKL_EARLY_CONSOLE
+static struct console lkl_boot_console = {
+	.name	= "lkl_boot_console",
+	.write	= console_write,
+	.flags	= CON_PRINTBUFFER | CON_BOOT,
+	.index	= -1,
+};
+
+int __init lkl_boot_console_init(void)
+{
+	register_console(&lkl_boot_console);
+	return 0;
+}
+early_initcall(lkl_boot_console_init);
+#endif
+
+static struct console lkl_console = {
+	.name	= "lkl_console",
+	.write	= console_write,
+	.flags	= CON_PRINTBUFFER,
+	.index	= -1,
+};
+
+int __init lkl_console_init(void)
+{
+	register_console(&lkl_console);
+	return 0;
+}
+core_initcall(lkl_console_init);
+
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v5 14/21] um: nommu: initialization and cleanup
  2020-07-02 14:06     ` Hajime Tazaki
@ 2020-07-02 14:07       ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-07-02 14:07 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Hajime Tazaki

Add the lkl_start_kernel and lkl_sys_halt APIs that start and
respectively stops the Linux kernel.

lkl_start_kernel creates a separate threads that will run the initial
and idle kernel thread. It waits for the kernel to complete
initialization before returning, to avoid races with system calls
issues by the host application.

During the setup phase, we create "/init" in initial ramfs root
filesystem to avoid mounting the "real" rootfs since ramfs is good
enough for now.

lkl_sys_halt will shutdown the kernel, terminate all threads and
free all host resources used by the kernel before returning.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/nommu/include/uapi/asm/host_ops.h |  14 ++
 arch/um/nommu/um/setup.c                  | 161 ++++++++++++++++++++++
 tools/um/uml/process.c                    |   2 +
 3 files changed, 177 insertions(+)
 create mode 100644 arch/um/nommu/um/setup.c

diff --git a/arch/um/nommu/include/uapi/asm/host_ops.h b/arch/um/nommu/include/uapi/asm/host_ops.h
index 133877c857a1..0811494c080c 100644
--- a/arch/um/nommu/include/uapi/asm/host_ops.h
+++ b/arch/um/nommu/include/uapi/asm/host_ops.h
@@ -16,6 +16,7 @@ struct lkl_jmp_buf {
  * These operations must be provided by a host library or by the application
  * itself.
  * @print - optional operation that receives console messages
+ * @panic - called during a kernel panic
  *
  * @sem_alloc - allocate a host semaphore an initialize it to count
  * @sem_free - free a host semaphore
@@ -65,6 +66,7 @@ struct lkl_jmp_buf {
  */
 struct lkl_host_operations {
 	void (*print)(const char *str, int len);
+	void (*panic)(void);
 
 	struct lkl_sem *(*sem_alloc)(int count);
 	void (*sem_free)(struct lkl_sem *sem);
@@ -96,6 +98,18 @@ struct lkl_host_operations {
 	void (*mem_free)(void *mem);
 };
 
+/**
+ * lkl_start_kernel - registers the host operations and starts the kernel
+ *
+ * The function returns only after the kernel is shutdown with lkl_sys_halt.
+ *
+ * @lkl_ops - pointer to host operations
+ * @cmd_line - format for command line string that is going to be used to
+ * generate the Linux kernel command line
+ */
+int lkl_start_kernel(struct lkl_host_operations *ops,
+		     const char *fmt, ...);
+
 void lkl_bug(const char *fmt, ...);
 
 #endif
diff --git a/arch/um/nommu/um/setup.c b/arch/um/nommu/um/setup.c
new file mode 100644
index 000000000000..25aa003792d4
--- /dev/null
+++ b/arch/um/nommu/um/setup.c
@@ -0,0 +1,161 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/binfmts.h>
+#include <linux/init.h>
+#include <linux/init_task.h>
+#include <linux/personality.h>
+#include <linux/reboot.h>
+#include <linux/fs.h>
+#include <linux/start_kernel.h>
+#include <linux/syscalls.h>
+#include <linux/tick.h>
+#include <linux/kernel.h>
+#include <linux/console.h>
+#include <asm/host_ops.h>
+#include <asm/irq.h>
+#include <asm/unistd.h>
+#include <asm/syscalls.h>
+#include <asm/cpu.h>
+#include <os.h>
+#include <as-layout.h>
+
+
+static void *init_sem;
+static int is_running;
+
+
+struct lkl_host_operations *lkl_ops;
+
+long lkl_panic_blink(int state)
+{
+	lkl_ops->panic();
+	return 0;
+}
+
+static void __init *lkl_run_kernel(void *arg)
+{
+
+	panic_blink = lkl_panic_blink;
+
+	threads_init();
+	lkl_cpu_get();
+	start_kernel();
+
+	return NULL;
+}
+
+int __init lkl_start_kernel(struct lkl_host_operations *ops,
+			    const char *fmt, ...)
+{
+	int ret;
+
+	lkl_ops = ops;
+
+	init_sem = lkl_ops->sem_alloc(0);
+	if (!init_sem)
+		return -ENOMEM;
+
+	ret = lkl_cpu_init();
+	if (ret)
+		goto out_free_init_sem;
+
+	ret = lkl_ops->thread_create(lkl_run_kernel, NULL);
+	if (!ret) {
+		ret = -ENOMEM;
+		goto out_free_init_sem;
+	}
+
+	lkl_ops->sem_down(init_sem);
+	lkl_ops->sem_free(init_sem);
+	current_thread_info()->task->thread.arch.tid = lkl_ops->thread_self();
+	lkl_cpu_change_owner(current_thread_info()->task->thread.arch.tid);
+
+	lkl_cpu_put();
+	is_running = 1;
+
+	return 0;
+
+out_free_init_sem:
+	lkl_ops->sem_free(init_sem);
+
+	return ret;
+}
+
+int lkl_is_running(void)
+{
+	return is_running;
+}
+
+
+long lkl_sys_halt(void)
+{
+	long err;
+	long params[6] = {LINUX_REBOOT_MAGIC1,
+		LINUX_REBOOT_MAGIC2, LINUX_REBOOT_CMD_RESTART, };
+
+	err = lkl_syscall(__NR_reboot, params);
+	if (err < 0)
+		return err;
+
+	is_running = false;
+
+	lkl_cpu_wait_shutdown();
+
+	syscalls_cleanup();
+	threads_cleanup();
+	/* Shutdown the clockevents source. */
+	tick_suspend_local();
+	free_mem();
+	lkl_ops->thread_join(current_thread_info()->task->thread.arch.tid);
+
+	return 0;
+}
+
+
+static int lkl_run_init(struct linux_binprm *bprm);
+
+static struct linux_binfmt lkl_run_init_binfmt = {
+	.module		= THIS_MODULE,
+	.load_binary	= lkl_run_init,
+};
+
+static int lkl_run_init(struct linux_binprm *bprm)
+{
+	int ret;
+
+	if (strcmp("/init", bprm->filename) != 0)
+		return -EINVAL;
+
+	ret = flush_old_exec(bprm);
+	if (ret)
+		return ret;
+	set_personality(PER_LINUX);
+	setup_new_exec(bprm);
+	install_exec_creds(bprm);
+
+	set_binfmt(&lkl_run_init_binfmt);
+
+	init_pid_ns.child_reaper = 0;
+
+	syscalls_init();
+
+	lkl_ops->sem_up(init_sem);
+	lkl_ops->thread_exit();
+
+	return 0;
+}
+
+
+/* skip mounting the "real" rootfs. ramfs is good enough. */
+static int __init fs_setup(void)
+{
+	int fd;
+
+	fd = ksys_open("/init", O_CREAT, 0700);
+	WARN_ON(fd < 0);
+	ksys_close(fd);
+
+	register_binfmt(&lkl_run_init_binfmt);
+
+	return 0;
+}
+late_initcall(fs_setup);
diff --git a/tools/um/uml/process.c b/tools/um/uml/process.c
index e52dd37ddadc..c39d914e80ac 100644
--- a/tools/um/uml/process.c
+++ b/tools/um/uml/process.c
@@ -114,6 +114,8 @@ void os_kill_process(int pid, int reap_child)
 
 void os_kill_ptraced_process(int pid, int reap_child)
 {
+	if (pid == 0)
+		return;
 	kill(pid, SIGKILL);
 	ptrace(PTRACE_KILL, pid);
 	ptrace(PTRACE_CONT, pid);
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v5 14/21] um: nommu: initialization and cleanup
@ 2020-07-02 14:07       ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-07-02 14:07 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, linux-kernel-library, linux-arch,
	Hajime Tazaki, Akira Moroo

Add the lkl_start_kernel and lkl_sys_halt APIs that start and
respectively stops the Linux kernel.

lkl_start_kernel creates a separate threads that will run the initial
and idle kernel thread. It waits for the kernel to complete
initialization before returning, to avoid races with system calls
issues by the host application.

During the setup phase, we create "/init" in initial ramfs root
filesystem to avoid mounting the "real" rootfs since ramfs is good
enough for now.

lkl_sys_halt will shutdown the kernel, terminate all threads and
free all host resources used by the kernel before returning.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/nommu/include/uapi/asm/host_ops.h |  14 ++
 arch/um/nommu/um/setup.c                  | 161 ++++++++++++++++++++++
 tools/um/uml/process.c                    |   2 +
 3 files changed, 177 insertions(+)
 create mode 100644 arch/um/nommu/um/setup.c

diff --git a/arch/um/nommu/include/uapi/asm/host_ops.h b/arch/um/nommu/include/uapi/asm/host_ops.h
index 133877c857a1..0811494c080c 100644
--- a/arch/um/nommu/include/uapi/asm/host_ops.h
+++ b/arch/um/nommu/include/uapi/asm/host_ops.h
@@ -16,6 +16,7 @@ struct lkl_jmp_buf {
  * These operations must be provided by a host library or by the application
  * itself.
  * @print - optional operation that receives console messages
+ * @panic - called during a kernel panic
  *
  * @sem_alloc - allocate a host semaphore an initialize it to count
  * @sem_free - free a host semaphore
@@ -65,6 +66,7 @@ struct lkl_jmp_buf {
  */
 struct lkl_host_operations {
 	void (*print)(const char *str, int len);
+	void (*panic)(void);
 
 	struct lkl_sem *(*sem_alloc)(int count);
 	void (*sem_free)(struct lkl_sem *sem);
@@ -96,6 +98,18 @@ struct lkl_host_operations {
 	void (*mem_free)(void *mem);
 };
 
+/**
+ * lkl_start_kernel - registers the host operations and starts the kernel
+ *
+ * The function returns only after the kernel is shutdown with lkl_sys_halt.
+ *
+ * @lkl_ops - pointer to host operations
+ * @cmd_line - format for command line string that is going to be used to
+ * generate the Linux kernel command line
+ */
+int lkl_start_kernel(struct lkl_host_operations *ops,
+		     const char *fmt, ...);
+
 void lkl_bug(const char *fmt, ...);
 
 #endif
diff --git a/arch/um/nommu/um/setup.c b/arch/um/nommu/um/setup.c
new file mode 100644
index 000000000000..25aa003792d4
--- /dev/null
+++ b/arch/um/nommu/um/setup.c
@@ -0,0 +1,161 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/binfmts.h>
+#include <linux/init.h>
+#include <linux/init_task.h>
+#include <linux/personality.h>
+#include <linux/reboot.h>
+#include <linux/fs.h>
+#include <linux/start_kernel.h>
+#include <linux/syscalls.h>
+#include <linux/tick.h>
+#include <linux/kernel.h>
+#include <linux/console.h>
+#include <asm/host_ops.h>
+#include <asm/irq.h>
+#include <asm/unistd.h>
+#include <asm/syscalls.h>
+#include <asm/cpu.h>
+#include <os.h>
+#include <as-layout.h>
+
+
+static void *init_sem;
+static int is_running;
+
+
+struct lkl_host_operations *lkl_ops;
+
+long lkl_panic_blink(int state)
+{
+	lkl_ops->panic();
+	return 0;
+}
+
+static void __init *lkl_run_kernel(void *arg)
+{
+
+	panic_blink = lkl_panic_blink;
+
+	threads_init();
+	lkl_cpu_get();
+	start_kernel();
+
+	return NULL;
+}
+
+int __init lkl_start_kernel(struct lkl_host_operations *ops,
+			    const char *fmt, ...)
+{
+	int ret;
+
+	lkl_ops = ops;
+
+	init_sem = lkl_ops->sem_alloc(0);
+	if (!init_sem)
+		return -ENOMEM;
+
+	ret = lkl_cpu_init();
+	if (ret)
+		goto out_free_init_sem;
+
+	ret = lkl_ops->thread_create(lkl_run_kernel, NULL);
+	if (!ret) {
+		ret = -ENOMEM;
+		goto out_free_init_sem;
+	}
+
+	lkl_ops->sem_down(init_sem);
+	lkl_ops->sem_free(init_sem);
+	current_thread_info()->task->thread.arch.tid = lkl_ops->thread_self();
+	lkl_cpu_change_owner(current_thread_info()->task->thread.arch.tid);
+
+	lkl_cpu_put();
+	is_running = 1;
+
+	return 0;
+
+out_free_init_sem:
+	lkl_ops->sem_free(init_sem);
+
+	return ret;
+}
+
+int lkl_is_running(void)
+{
+	return is_running;
+}
+
+
+long lkl_sys_halt(void)
+{
+	long err;
+	long params[6] = {LINUX_REBOOT_MAGIC1,
+		LINUX_REBOOT_MAGIC2, LINUX_REBOOT_CMD_RESTART, };
+
+	err = lkl_syscall(__NR_reboot, params);
+	if (err < 0)
+		return err;
+
+	is_running = false;
+
+	lkl_cpu_wait_shutdown();
+
+	syscalls_cleanup();
+	threads_cleanup();
+	/* Shutdown the clockevents source. */
+	tick_suspend_local();
+	free_mem();
+	lkl_ops->thread_join(current_thread_info()->task->thread.arch.tid);
+
+	return 0;
+}
+
+
+static int lkl_run_init(struct linux_binprm *bprm);
+
+static struct linux_binfmt lkl_run_init_binfmt = {
+	.module		= THIS_MODULE,
+	.load_binary	= lkl_run_init,
+};
+
+static int lkl_run_init(struct linux_binprm *bprm)
+{
+	int ret;
+
+	if (strcmp("/init", bprm->filename) != 0)
+		return -EINVAL;
+
+	ret = flush_old_exec(bprm);
+	if (ret)
+		return ret;
+	set_personality(PER_LINUX);
+	setup_new_exec(bprm);
+	install_exec_creds(bprm);
+
+	set_binfmt(&lkl_run_init_binfmt);
+
+	init_pid_ns.child_reaper = 0;
+
+	syscalls_init();
+
+	lkl_ops->sem_up(init_sem);
+	lkl_ops->thread_exit();
+
+	return 0;
+}
+
+
+/* skip mounting the "real" rootfs. ramfs is good enough. */
+static int __init fs_setup(void)
+{
+	int fd;
+
+	fd = ksys_open("/init", O_CREAT, 0700);
+	WARN_ON(fd < 0);
+	ksys_close(fd);
+
+	register_binfmt(&lkl_run_init_binfmt);
+
+	return 0;
+}
+late_initcall(fs_setup);
diff --git a/tools/um/uml/process.c b/tools/um/uml/process.c
index e52dd37ddadc..c39d914e80ac 100644
--- a/tools/um/uml/process.c
+++ b/tools/um/uml/process.c
@@ -114,6 +114,8 @@ void os_kill_process(int pid, int reap_child)
 
 void os_kill_ptraced_process(int pid, int reap_child)
 {
+	if (pid == 0)
+		return;
 	kill(pid, SIGKILL);
 	ptrace(PTRACE_KILL, pid);
 	ptrace(PTRACE_CONT, pid);
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v5 15/21] um: nommu: integrate with irq infrastructure of UML
  2020-07-02 14:06     ` Hajime Tazaki
@ 2020-07-02 14:07       ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-07-02 14:07 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Hajime Tazaki

In order to cooperate with UML's irq infrastructure and nommu threads
based on host threads, irq handlers shall synchronize the our
scheduler which is controlled by struct lkl_cpu.  To do that, the irq
infra notifies its entry of handlers by obtaining cpu access of thread
scheduler (lkl_cpu_try_run_irq) and its release (lkl_cpu_put).

In additon to that, in order to stick the signal handler's thread to the
one of the idle thread, several required configurations of (thread's)
signal mask are added: otherwise handlers running on arbitrary thread
cannot obtain cpu access and immediately fall into pending interrupt
which may slow down the delivery of signals.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/include/shared/os.h     |  1 +
 arch/um/kernel/irq.c            | 13 +++++++++++++
 arch/um/kernel/time.c           |  2 ++
 arch/um/nommu/include/asm/cpu.h |  3 +++
 arch/um/nommu/um/cpu.c          | 17 ++++++++++++++---
 arch/um/nommu/um/setup.c        |  5 +++++
 arch/um/nommu/um/threads.c      |  3 +++
 tools/um/uml/signal.c           | 12 ++++++++++--
 8 files changed, 51 insertions(+), 5 deletions(-)

diff --git a/arch/um/include/shared/os.h b/arch/um/include/shared/os.h
index f467d28fc0b4..0da3ce22e16f 100644
--- a/arch/um/include/shared/os.h
+++ b/arch/um/include/shared/os.h
@@ -241,6 +241,7 @@ extern int set_signals(int enable);
 extern int set_signals_trace(int enable);
 extern int os_is_signal_stack(void);
 extern void deliver_alarm(void);
+extern void set_pending_signals(int sig);
 
 /* util.c */
 extern void stack_protections(unsigned long address);
diff --git a/arch/um/kernel/irq.c b/arch/um/kernel/irq.c
index 3577118bb4a5..4ce7f5fef7f8 100644
--- a/arch/um/kernel/irq.c
+++ b/arch/um/kernel/irq.c
@@ -19,6 +19,7 @@
 #include <kern_util.h>
 #include <os.h>
 #include <irq_user.h>
+#include <asm/cpu.h>
 
 
 extern void free_irqs(void);
@@ -541,6 +542,11 @@ unsigned long to_irq_stack(unsigned long *mask_out)
 	unsigned long mask, old;
 	int nested;
 
+#ifndef CONFIG_MMU
+	if (!nommu_irq_enter(ffs(*mask_out) - 1))
+		return 1;
+#endif
+
 	mask = xchg(&pending_mask, *mask_out);
 	if (mask != 0) {
 		/*
@@ -557,6 +563,10 @@ unsigned long to_irq_stack(unsigned long *mask_out)
 			old |= mask;
 			mask = xchg(&pending_mask, old);
 		} while (mask != old);
+
+#ifndef CONFIG_MMU
+		nommu_irq_exit();
+#endif
 		return 1;
 	}
 
@@ -594,6 +604,9 @@ unsigned long from_irq_stack(int nested)
 	*to = *ti;
 
 	mask = xchg(&pending_mask, 0);
+#ifndef CONFIG_MMU
+	nommu_irq_exit();
+#endif
 	return mask & ~1;
 }
 
diff --git a/arch/um/kernel/time.c b/arch/um/kernel/time.c
index 25eaa6a0c658..9e141fe8fe27 100644
--- a/arch/um/kernel/time.c
+++ b/arch/um/kernel/time.c
@@ -597,11 +597,13 @@ static struct clock_event_device timer_clockevent = {
 
 static irqreturn_t um_timer(int irq, void *dev)
 {
+#ifdef CONFIG_MMU
 	if (get_current()->mm != NULL)
 	{
         /* userspace - relay signal, results in correct userspace timers */
 		os_alarm_process(get_current()->mm->context.id.u.pid);
 	}
+#endif
 
 	(*timer_clockevent.event_handler)(&timer_clockevent);
 
diff --git a/arch/um/nommu/include/asm/cpu.h b/arch/um/nommu/include/asm/cpu.h
index c101c078ef21..c52a78c77bea 100644
--- a/arch/um/nommu/include/asm/cpu.h
+++ b/arch/um/nommu/include/asm/cpu.h
@@ -10,4 +10,7 @@ void lkl_cpu_wait_shutdown(void);
 void lkl_cpu_change_owner(lkl_thread_t owner);
 void lkl_cpu_set_irqs_pending(void);
 
+void nommu_irq_exit(void);
+int nommu_irq_enter(int sig);
+
 #endif
diff --git a/arch/um/nommu/um/cpu.c b/arch/um/nommu/um/cpu.c
index 9986a3f8c5dd..75d8bdc1847e 100644
--- a/arch/um/nommu/um/cpu.c
+++ b/arch/um/nommu/um/cpu.c
@@ -8,15 +8,16 @@
 #include <asm/sched.h>
 #include <asm/syscalls.h>
 #include <init.h>
+#include <os.h>
 
 void run_irqs(void)
 {
-	panic("unimplemented %s", __func__);
+	unblock_signals();
 }
 
-int set_irq_pending(int irq)
+void set_irq_pending(int sig)
 {
-	panic("unimplemented %s", __func__);
+	set_pending_signals(sig);
 }
 
 /*
@@ -174,6 +175,16 @@ int lkl_cpu_try_run_irq(int irq)
 	return ret;
 }
 
+int nommu_irq_enter(int sig)
+{
+	return lkl_cpu_try_run_irq(sig);
+}
+
+void nommu_irq_exit(void)
+{
+	return lkl_cpu_put();
+}
+
 static void lkl_cpu_shutdown(void)
 {
 	__sync_fetch_and_add(&cpu.shutdown_gate, MAX_THREADS);
diff --git a/arch/um/nommu/um/setup.c b/arch/um/nommu/um/setup.c
index 25aa003792d4..f74baea50bd3 100644
--- a/arch/um/nommu/um/setup.c
+++ b/arch/um/nommu/um/setup.c
@@ -36,6 +36,8 @@ static void __init *lkl_run_kernel(void *arg)
 
 	panic_blink = lkl_panic_blink;
 
+	/* signal should be received at this thread (main and idle threads) */
+	init_new_thread_signals();
 	threads_init();
 	lkl_cpu_get();
 	start_kernel();
@@ -58,6 +60,9 @@ int __init lkl_start_kernel(struct lkl_host_operations *ops,
 	if (ret)
 		goto out_free_init_sem;
 
+	change_sig(SIGALRM, 0);
+	change_sig(SIGIO, 0);
+
 	ret = lkl_ops->thread_create(lkl_run_kernel, NULL);
 	if (!ret) {
 		ret = -ENOMEM;
diff --git a/arch/um/nommu/um/threads.c b/arch/um/nommu/um/threads.c
index 3e70eccc191a..df96057100fd 100644
--- a/arch/um/nommu/um/threads.c
+++ b/arch/um/nommu/um/threads.c
@@ -151,6 +151,9 @@ static void *thread_bootstrap(void *_tba)
 	int (*f)(void *) = tba->f;
 	void *arg = tba->arg;
 
+	change_sig(SIGALRM, 0);
+	change_sig(SIGIO, 0);
+
 	lkl_ops->sem_down(ti->task->thread.arch.sched_sem);
 	kfree(tba);
 	if (ti->task->thread.prev_sched)
diff --git a/tools/um/uml/signal.c b/tools/um/uml/signal.c
index b58bc68cbe64..2b5c98de21cd 100644
--- a/tools/um/uml/signal.c
+++ b/tools/um/uml/signal.c
@@ -218,8 +218,8 @@ void set_handler(int sig)
 
 	sigemptyset(&sig_mask);
 	sigaddset(&sig_mask, sig);
-	if (sigprocmask(SIG_UNBLOCK, &sig_mask, NULL) < 0)
-		panic("sigprocmask failed - errno = %d\n", errno);
+	if (pthread_sigmask(SIG_UNBLOCK, &sig_mask, NULL) < 0)
+		panic("pthread_sigmask failed - errno = %d\n", errno);
 }
 
 int change_sig(int signal, int on)
@@ -355,3 +355,11 @@ int os_is_signal_stack(void)
 
 	return ss.ss_flags & SS_ONSTACK;
 }
+
+void set_pending_signals(int sig)
+{
+	if (sig == SIGIO)
+		signals_pending |= SIGIO_MASK;
+	else if (sig == SIGALRM)
+		signals_pending |= SIGALRM_MASK;
+}
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v5 15/21] um: nommu: integrate with irq infrastructure of UML
@ 2020-07-02 14:07       ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-07-02 14:07 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, linux-kernel-library, linux-arch,
	Hajime Tazaki, Akira Moroo

In order to cooperate with UML's irq infrastructure and nommu threads
based on host threads, irq handlers shall synchronize the our
scheduler which is controlled by struct lkl_cpu.  To do that, the irq
infra notifies its entry of handlers by obtaining cpu access of thread
scheduler (lkl_cpu_try_run_irq) and its release (lkl_cpu_put).

In additon to that, in order to stick the signal handler's thread to the
one of the idle thread, several required configurations of (thread's)
signal mask are added: otherwise handlers running on arbitrary thread
cannot obtain cpu access and immediately fall into pending interrupt
which may slow down the delivery of signals.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/include/shared/os.h     |  1 +
 arch/um/kernel/irq.c            | 13 +++++++++++++
 arch/um/kernel/time.c           |  2 ++
 arch/um/nommu/include/asm/cpu.h |  3 +++
 arch/um/nommu/um/cpu.c          | 17 ++++++++++++++---
 arch/um/nommu/um/setup.c        |  5 +++++
 arch/um/nommu/um/threads.c      |  3 +++
 tools/um/uml/signal.c           | 12 ++++++++++--
 8 files changed, 51 insertions(+), 5 deletions(-)

diff --git a/arch/um/include/shared/os.h b/arch/um/include/shared/os.h
index f467d28fc0b4..0da3ce22e16f 100644
--- a/arch/um/include/shared/os.h
+++ b/arch/um/include/shared/os.h
@@ -241,6 +241,7 @@ extern int set_signals(int enable);
 extern int set_signals_trace(int enable);
 extern int os_is_signal_stack(void);
 extern void deliver_alarm(void);
+extern void set_pending_signals(int sig);
 
 /* util.c */
 extern void stack_protections(unsigned long address);
diff --git a/arch/um/kernel/irq.c b/arch/um/kernel/irq.c
index 3577118bb4a5..4ce7f5fef7f8 100644
--- a/arch/um/kernel/irq.c
+++ b/arch/um/kernel/irq.c
@@ -19,6 +19,7 @@
 #include <kern_util.h>
 #include <os.h>
 #include <irq_user.h>
+#include <asm/cpu.h>
 
 
 extern void free_irqs(void);
@@ -541,6 +542,11 @@ unsigned long to_irq_stack(unsigned long *mask_out)
 	unsigned long mask, old;
 	int nested;
 
+#ifndef CONFIG_MMU
+	if (!nommu_irq_enter(ffs(*mask_out) - 1))
+		return 1;
+#endif
+
 	mask = xchg(&pending_mask, *mask_out);
 	if (mask != 0) {
 		/*
@@ -557,6 +563,10 @@ unsigned long to_irq_stack(unsigned long *mask_out)
 			old |= mask;
 			mask = xchg(&pending_mask, old);
 		} while (mask != old);
+
+#ifndef CONFIG_MMU
+		nommu_irq_exit();
+#endif
 		return 1;
 	}
 
@@ -594,6 +604,9 @@ unsigned long from_irq_stack(int nested)
 	*to = *ti;
 
 	mask = xchg(&pending_mask, 0);
+#ifndef CONFIG_MMU
+	nommu_irq_exit();
+#endif
 	return mask & ~1;
 }
 
diff --git a/arch/um/kernel/time.c b/arch/um/kernel/time.c
index 25eaa6a0c658..9e141fe8fe27 100644
--- a/arch/um/kernel/time.c
+++ b/arch/um/kernel/time.c
@@ -597,11 +597,13 @@ static struct clock_event_device timer_clockevent = {
 
 static irqreturn_t um_timer(int irq, void *dev)
 {
+#ifdef CONFIG_MMU
 	if (get_current()->mm != NULL)
 	{
         /* userspace - relay signal, results in correct userspace timers */
 		os_alarm_process(get_current()->mm->context.id.u.pid);
 	}
+#endif
 
 	(*timer_clockevent.event_handler)(&timer_clockevent);
 
diff --git a/arch/um/nommu/include/asm/cpu.h b/arch/um/nommu/include/asm/cpu.h
index c101c078ef21..c52a78c77bea 100644
--- a/arch/um/nommu/include/asm/cpu.h
+++ b/arch/um/nommu/include/asm/cpu.h
@@ -10,4 +10,7 @@ void lkl_cpu_wait_shutdown(void);
 void lkl_cpu_change_owner(lkl_thread_t owner);
 void lkl_cpu_set_irqs_pending(void);
 
+void nommu_irq_exit(void);
+int nommu_irq_enter(int sig);
+
 #endif
diff --git a/arch/um/nommu/um/cpu.c b/arch/um/nommu/um/cpu.c
index 9986a3f8c5dd..75d8bdc1847e 100644
--- a/arch/um/nommu/um/cpu.c
+++ b/arch/um/nommu/um/cpu.c
@@ -8,15 +8,16 @@
 #include <asm/sched.h>
 #include <asm/syscalls.h>
 #include <init.h>
+#include <os.h>
 
 void run_irqs(void)
 {
-	panic("unimplemented %s", __func__);
+	unblock_signals();
 }
 
-int set_irq_pending(int irq)
+void set_irq_pending(int sig)
 {
-	panic("unimplemented %s", __func__);
+	set_pending_signals(sig);
 }
 
 /*
@@ -174,6 +175,16 @@ int lkl_cpu_try_run_irq(int irq)
 	return ret;
 }
 
+int nommu_irq_enter(int sig)
+{
+	return lkl_cpu_try_run_irq(sig);
+}
+
+void nommu_irq_exit(void)
+{
+	return lkl_cpu_put();
+}
+
 static void lkl_cpu_shutdown(void)
 {
 	__sync_fetch_and_add(&cpu.shutdown_gate, MAX_THREADS);
diff --git a/arch/um/nommu/um/setup.c b/arch/um/nommu/um/setup.c
index 25aa003792d4..f74baea50bd3 100644
--- a/arch/um/nommu/um/setup.c
+++ b/arch/um/nommu/um/setup.c
@@ -36,6 +36,8 @@ static void __init *lkl_run_kernel(void *arg)
 
 	panic_blink = lkl_panic_blink;
 
+	/* signal should be received at this thread (main and idle threads) */
+	init_new_thread_signals();
 	threads_init();
 	lkl_cpu_get();
 	start_kernel();
@@ -58,6 +60,9 @@ int __init lkl_start_kernel(struct lkl_host_operations *ops,
 	if (ret)
 		goto out_free_init_sem;
 
+	change_sig(SIGALRM, 0);
+	change_sig(SIGIO, 0);
+
 	ret = lkl_ops->thread_create(lkl_run_kernel, NULL);
 	if (!ret) {
 		ret = -ENOMEM;
diff --git a/arch/um/nommu/um/threads.c b/arch/um/nommu/um/threads.c
index 3e70eccc191a..df96057100fd 100644
--- a/arch/um/nommu/um/threads.c
+++ b/arch/um/nommu/um/threads.c
@@ -151,6 +151,9 @@ static void *thread_bootstrap(void *_tba)
 	int (*f)(void *) = tba->f;
 	void *arg = tba->arg;
 
+	change_sig(SIGALRM, 0);
+	change_sig(SIGIO, 0);
+
 	lkl_ops->sem_down(ti->task->thread.arch.sched_sem);
 	kfree(tba);
 	if (ti->task->thread.prev_sched)
diff --git a/tools/um/uml/signal.c b/tools/um/uml/signal.c
index b58bc68cbe64..2b5c98de21cd 100644
--- a/tools/um/uml/signal.c
+++ b/tools/um/uml/signal.c
@@ -218,8 +218,8 @@ void set_handler(int sig)
 
 	sigemptyset(&sig_mask);
 	sigaddset(&sig_mask, sig);
-	if (sigprocmask(SIG_UNBLOCK, &sig_mask, NULL) < 0)
-		panic("sigprocmask failed - errno = %d\n", errno);
+	if (pthread_sigmask(SIG_UNBLOCK, &sig_mask, NULL) < 0)
+		panic("pthread_sigmask failed - errno = %d\n", errno);
 }
 
 int change_sig(int signal, int on)
@@ -355,3 +355,11 @@ int os_is_signal_stack(void)
 
 	return ss.ss_flags & SS_ONSTACK;
 }
+
+void set_pending_signals(int sig)
+{
+	if (sig == SIGIO)
+		signals_pending |= SIGIO_MASK;
+	else if (sig == SIGALRM)
+		signals_pending |= SIGALRM_MASK;
+}
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v5 16/21] um: nommu: plug in the build system
  2020-07-02 14:06     ` Hajime Tazaki
@ 2020-07-02 14:07       ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-07-02 14:07 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Hajime Tazaki

Basic Makefiles for building library of nommu mode. Add a new architecture
specific target for installing the resulting library files and headers.

To make nommu binaries build, UML introduced an additional option, UMMODE
variable, to switch the output file of build: kernel (default), or
library (nommu).  Those modes are not able to be ON at the same time.

To build on library mode, users do the following:

  make defconfig ARCH=um UMMODE=library
  make ARCH=um UMMODE=library

Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/Kconfig           | 17 ++++++++++++++---
 arch/um/Makefile          | 10 +++++++++-
 arch/um/kernel/Makefile   | 12 ++++++++----
 arch/um/nommu/um/Kconfig  | 37 +++++++++++++++++++++++++++++++++++++
 arch/um/nommu/um/Makefile |  4 ++++
 5 files changed, 72 insertions(+), 8 deletions(-)
 create mode 100644 arch/um/nommu/um/Kconfig
 create mode 100644 arch/um/nommu/um/Makefile

diff --git a/arch/um/Kconfig b/arch/um/Kconfig
index a0a9fd3ab96c..300fcc90cff3 100644
--- a/arch/um/Kconfig
+++ b/arch/um/Kconfig
@@ -22,9 +22,20 @@ config UML
 	select TTY # Needed for line.c
 	select MODULE_REL_CRCS if MODVERSIONS
 
+config UMMODE_LIB
+	bool "UML mode: library mode"
+	default y if "$(UMMODE)" = "library"
+	help
+	 This mode switches a mode to build a library of UML (Linux
+	 Kernel Library/LKL).  This switch is exclusive to "kernel mode"
+	 of UML, which is traditional mode of UML.
+
+	 For more detail about LKL, see
+	 <file:Documentation/virt/uml/lkl.txt>.
+
 config MMU
 	bool
-	default y
+	default y if !UMMODE_LIB
 
 config NO_IOMEM
 	def_bool y
@@ -45,12 +56,12 @@ config LOCKDEP_SUPPORT
 
 config STACKTRACE_SUPPORT
 	bool
-	default y
+	default y if MMU
 	select STACKTRACE
 
 config GENERIC_CALIBRATE_DELAY
 	bool
-	default y
+	default y if MMU
 
 config HZ
 	int
diff --git a/arch/um/Makefile b/arch/um/Makefile
index af0cf64c1428..7e640e65a80e 100644
--- a/arch/um/Makefile
+++ b/arch/um/Makefile
@@ -17,6 +17,10 @@ else
         KBUILD_DEFCONFIG := $(SUBARCH)_defconfig
 endif
 
+ifeq ($(UMMODE),library)
+  SUBARCH := um/nommu
+endif
+
 ARCH_DIR := arch/um
 OS := $(shell uname -s)
 # We require bash because the vmlinux link and loader script cpp use bash
@@ -102,9 +106,13 @@ all: linux.o
 
 linux.o: vmlinux
 	@echo '  LINK $@'
-	$(Q)$(OBJCOPY) -R .eh_frame $< $@
+	$(Q)$(OBJCOPY) -R .eh_frame -L sem_init -L sem_post -L sem_wait -L sem_destroy $< $@
 
+ifeq ($(UMMODE),library)
+install: linux.o um_headers_install
+else
 install: linux.o
+endif
 	@echo "  INSTALL $(INSTALL_PATH)/lib/$<"
 	@mkdir -p $(INSTALL_PATH)/lib/
 	@cp $< $(INSTALL_PATH)/lib/
diff --git a/arch/um/kernel/Makefile b/arch/um/kernel/Makefile
index 5aa882011e04..bdc4108bc52e 100644
--- a/arch/um/kernel/Makefile
+++ b/arch/um/kernel/Makefile
@@ -14,10 +14,14 @@ CPPFLAGS_vmlinux.lds := -DSTART=$(LDS_START)		\
 			$(LDS_EXTRA)
 extra-y := vmlinux.lds
 
-obj-y = config.o exec.o exitcode.o irq.o ksyms.o mem.o \
-	physmem.o process.o ptrace.o reboot.o sigio.o \
-	signal.o syscall.o sysrq.o time.o tlb.o trap.o \
-	um_arch.o umid.o maccess.o kmsg_dump.o skas/
+obj-y = config.o exitcode.o irq.o ksyms.o \
+	process.o reboot.o sigio.o \
+	signal.o syscall.o time.o \
+	um_arch.o umid.o maccess.o kmsg_dump.o
+
+ifdef CONFIG_MMU
+obj-y += exec.o mem.o physmem.o ptrace.o sysrq.o tlb.o trap.o skas/
+endif
 
 obj-$(CONFIG_BLK_DEV_INITRD) += initrd.o
 obj-$(CONFIG_GPROF)	+= gprof_syms.o
diff --git a/arch/um/nommu/um/Kconfig b/arch/um/nommu/um/Kconfig
new file mode 100644
index 000000000000..20b3eaccb6f0
--- /dev/null
+++ b/arch/um/nommu/um/Kconfig
@@ -0,0 +1,37 @@
+config UML_NOMMU
+	def_bool y
+	depends on !SMP && !MMU
+	select UACCESS_MEMCPY
+	select ARCH_THREAD_STACK_ALLOCATOR
+	select ARCH_HAS_SYSCALL_WRAPPER
+
+config 64BIT
+	bool
+	default y
+
+config GENERIC_CSUM
+	def_bool y
+
+config GENERIC_ATOMIC64
+	bool
+	default y if !64BIT
+
+config SECCOMP
+	bool
+	default n
+
+config GENERIC_HWEIGHT
+	def_bool y
+
+config GENERIC_CALIBRATE_DELAY
+	bool
+	default n
+
+config STACKTRACE_SUPPORT
+	bool
+	default n
+
+# XXX: need this to work well with tap13.py
+config PRINTK_TIME
+	bool
+	default y
diff --git a/arch/um/nommu/um/Makefile b/arch/um/nommu/um/Makefile
new file mode 100644
index 000000000000..b580e0fe97b5
--- /dev/null
+++ b/arch/um/nommu/um/Makefile
@@ -0,0 +1,4 @@
+include/generated/user_constants.h: $(srctree)/arch/um/nommu/um/user_constants.h
+	$(Q)cp -f $^ $@
+
+obj-y = bootmem.o console.o cpu.o delay.o setup.o syscalls.o threads.o unimplemented.o
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v5 16/21] um: nommu: plug in the build system
@ 2020-07-02 14:07       ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-07-02 14:07 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, linux-kernel-library, linux-arch,
	Hajime Tazaki, Akira Moroo

Basic Makefiles for building library of nommu mode. Add a new architecture
specific target for installing the resulting library files and headers.

To make nommu binaries build, UML introduced an additional option, UMMODE
variable, to switch the output file of build: kernel (default), or
library (nommu).  Those modes are not able to be ON at the same time.

To build on library mode, users do the following:

  make defconfig ARCH=um UMMODE=library
  make ARCH=um UMMODE=library

Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/Kconfig           | 17 ++++++++++++++---
 arch/um/Makefile          | 10 +++++++++-
 arch/um/kernel/Makefile   | 12 ++++++++----
 arch/um/nommu/um/Kconfig  | 37 +++++++++++++++++++++++++++++++++++++
 arch/um/nommu/um/Makefile |  4 ++++
 5 files changed, 72 insertions(+), 8 deletions(-)
 create mode 100644 arch/um/nommu/um/Kconfig
 create mode 100644 arch/um/nommu/um/Makefile

diff --git a/arch/um/Kconfig b/arch/um/Kconfig
index a0a9fd3ab96c..300fcc90cff3 100644
--- a/arch/um/Kconfig
+++ b/arch/um/Kconfig
@@ -22,9 +22,20 @@ config UML
 	select TTY # Needed for line.c
 	select MODULE_REL_CRCS if MODVERSIONS
 
+config UMMODE_LIB
+	bool "UML mode: library mode"
+	default y if "$(UMMODE)" = "library"
+	help
+	 This mode switches a mode to build a library of UML (Linux
+	 Kernel Library/LKL).  This switch is exclusive to "kernel mode"
+	 of UML, which is traditional mode of UML.
+
+	 For more detail about LKL, see
+	 <file:Documentation/virt/uml/lkl.txt>.
+
 config MMU
 	bool
-	default y
+	default y if !UMMODE_LIB
 
 config NO_IOMEM
 	def_bool y
@@ -45,12 +56,12 @@ config LOCKDEP_SUPPORT
 
 config STACKTRACE_SUPPORT
 	bool
-	default y
+	default y if MMU
 	select STACKTRACE
 
 config GENERIC_CALIBRATE_DELAY
 	bool
-	default y
+	default y if MMU
 
 config HZ
 	int
diff --git a/arch/um/Makefile b/arch/um/Makefile
index af0cf64c1428..7e640e65a80e 100644
--- a/arch/um/Makefile
+++ b/arch/um/Makefile
@@ -17,6 +17,10 @@ else
         KBUILD_DEFCONFIG := $(SUBARCH)_defconfig
 endif
 
+ifeq ($(UMMODE),library)
+  SUBARCH := um/nommu
+endif
+
 ARCH_DIR := arch/um
 OS := $(shell uname -s)
 # We require bash because the vmlinux link and loader script cpp use bash
@@ -102,9 +106,13 @@ all: linux.o
 
 linux.o: vmlinux
 	@echo '  LINK $@'
-	$(Q)$(OBJCOPY) -R .eh_frame $< $@
+	$(Q)$(OBJCOPY) -R .eh_frame -L sem_init -L sem_post -L sem_wait -L sem_destroy $< $@
 
+ifeq ($(UMMODE),library)
+install: linux.o um_headers_install
+else
 install: linux.o
+endif
 	@echo "  INSTALL $(INSTALL_PATH)/lib/$<"
 	@mkdir -p $(INSTALL_PATH)/lib/
 	@cp $< $(INSTALL_PATH)/lib/
diff --git a/arch/um/kernel/Makefile b/arch/um/kernel/Makefile
index 5aa882011e04..bdc4108bc52e 100644
--- a/arch/um/kernel/Makefile
+++ b/arch/um/kernel/Makefile
@@ -14,10 +14,14 @@ CPPFLAGS_vmlinux.lds := -DSTART=$(LDS_START)		\
 			$(LDS_EXTRA)
 extra-y := vmlinux.lds
 
-obj-y = config.o exec.o exitcode.o irq.o ksyms.o mem.o \
-	physmem.o process.o ptrace.o reboot.o sigio.o \
-	signal.o syscall.o sysrq.o time.o tlb.o trap.o \
-	um_arch.o umid.o maccess.o kmsg_dump.o skas/
+obj-y = config.o exitcode.o irq.o ksyms.o \
+	process.o reboot.o sigio.o \
+	signal.o syscall.o time.o \
+	um_arch.o umid.o maccess.o kmsg_dump.o
+
+ifdef CONFIG_MMU
+obj-y += exec.o mem.o physmem.o ptrace.o sysrq.o tlb.o trap.o skas/
+endif
 
 obj-$(CONFIG_BLK_DEV_INITRD) += initrd.o
 obj-$(CONFIG_GPROF)	+= gprof_syms.o
diff --git a/arch/um/nommu/um/Kconfig b/arch/um/nommu/um/Kconfig
new file mode 100644
index 000000000000..20b3eaccb6f0
--- /dev/null
+++ b/arch/um/nommu/um/Kconfig
@@ -0,0 +1,37 @@
+config UML_NOMMU
+	def_bool y
+	depends on !SMP && !MMU
+	select UACCESS_MEMCPY
+	select ARCH_THREAD_STACK_ALLOCATOR
+	select ARCH_HAS_SYSCALL_WRAPPER
+
+config 64BIT
+	bool
+	default y
+
+config GENERIC_CSUM
+	def_bool y
+
+config GENERIC_ATOMIC64
+	bool
+	default y if !64BIT
+
+config SECCOMP
+	bool
+	default n
+
+config GENERIC_HWEIGHT
+	def_bool y
+
+config GENERIC_CALIBRATE_DELAY
+	bool
+	default n
+
+config STACKTRACE_SUPPORT
+	bool
+	default n
+
+# XXX: need this to work well with tap13.py
+config PRINTK_TIME
+	bool
+	default y
diff --git a/arch/um/nommu/um/Makefile b/arch/um/nommu/um/Makefile
new file mode 100644
index 000000000000..b580e0fe97b5
--- /dev/null
+++ b/arch/um/nommu/um/Makefile
@@ -0,0 +1,4 @@
+include/generated/user_constants.h: $(srctree)/arch/um/nommu/um/user_constants.h
+	$(Q)cp -f $^ $@
+
+obj-y = bootmem.o console.o cpu.o delay.o setup.o syscalls.o threads.o unimplemented.o
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v5 17/21] um: host: add nommu build for ARCH=um
  2020-07-02 14:06     ` Hajime Tazaki
@ 2020-07-02 14:07       ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-07-02 14:07 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Hajime Tazaki

This patch adds the skeleton for the host library.

The host library is implementing the host operations needed by nommu
mode and is split into host dependent (depends on a specific host, e.g.
POSIX hosts) and host independent parts (will work on all supported
hosts).

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/um/Makefile                  |   7 +-
 tools/um/Targets                   |   3 +-
 tools/um/include/lkl.h             | 151 +++++++++++++++++++++++++++++
 tools/um/include/lkl_host.h        |  26 +++++
 tools/um/uml/Build                 |  15 ++-
 tools/um/uml/nommu/Build           |   1 +
 tools/um/uml/nommu/registers.c     |  21 ++++
 tools/um/uml/nommu/unimplemented.c |  21 ++++
 8 files changed, 238 insertions(+), 7 deletions(-)
 create mode 100644 tools/um/include/lkl.h
 create mode 100644 tools/um/include/lkl_host.h
 create mode 100644 tools/um/uml/nommu/Build
 create mode 100644 tools/um/uml/nommu/registers.c
 create mode 100644 tools/um/uml/nommu/unimplemented.c

diff --git a/tools/um/Makefile b/tools/um/Makefile
index 552ed5f1edae..a93e060b1f89 100644
--- a/tools/um/Makefile
+++ b/tools/um/Makefile
@@ -5,6 +5,7 @@
 MAKEFLAGS += -r --no-print-directory
 
 KCONFIG?=defconfig
+UMMODE?=kernel
 
 ifneq ($(silent),1)
   ifneq ($(V),1)
@@ -47,8 +48,9 @@ all: $(TARGETS)
 
 # rule to build linux.o
 $(OUTPUT)lib/linux.o:
-	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) $(KCONFIG)
-	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) install INSTALL_PATH=$(OUTPUT)
+	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um UMMODE=$(UMMODE) $(KOPT) $(KCONFIG)
+	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um UMMODE=$(UMMODE) $(KOPT) install \
+	INSTALL_PATH=$(OUTPUT)
 
 $(OUTPUT)liblinux.a: $(OUTPUT)lib/linux.o $(OUTPUT)uml/liblinux-in.o $(OUTPUT)lib/liblinux-in.o
 	$(QUIET_AR)$(AR) -rc $@ $^
@@ -67,6 +69,7 @@ $(OUTPUT)%-in.o: $(OUTPUT)lib/linux.o FORCE
 clean:
 	$(call QUIET_CLEAN, objects)find $(OUTPUT) -name '*.o' -delete -o -name '\.*.cmd'\
 	 -delete -o -name '\.*.d' -delete
+	$(call QUIET_CLEAN, headers)$(RM) -r $(OUTPUT)/include/lkl/
 	$(call QUIET_CLEAN, liblinux.a)$(RM) $(OUTPUT)/liblinux.a
 	$(call QUIET_CLEAN, $(TARGETS))$(RM) $(TARGETS)
 
diff --git a/tools/um/Targets b/tools/um/Targets
index a4711f1ef422..4e1f0f4d81a3 100644
--- a/tools/um/Targets
+++ b/tools/um/Targets
@@ -1,4 +1,5 @@
 progs-y += uml/linux
-LDLIBS_linux-y := -lrt -lpthread -lutil
+
+LDLIBS := -lrt -lpthread -lutil
 LDFLAGS_linux-y := -no-pie -Wl,--wrap,malloc -Wl,--wrap,free -Wl,--wrap,calloc
 LDFLAGS_linux-$(UML_STATIC) += -static
diff --git a/tools/um/include/lkl.h b/tools/um/include/lkl.h
new file mode 100644
index 000000000000..707e01b64a70
--- /dev/null
+++ b/tools/um/include/lkl.h
@@ -0,0 +1,151 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_H
+#define _LKL_H
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#define _LKL_LIBC_COMPAT_H
+
+#ifdef __cplusplus
+#define class __lkl__class
+#endif
+
+/*
+ * Avoid collisions between Android which defines __unused and
+ * linux/icmp.h which uses __unused as a structure field.
+ */
+#pragma push_macro("__unused")
+#undef __unused
+
+#include <lkl/asm/syscalls.h>
+
+#pragma pop_macro("__unused")
+
+#ifdef __cplusplus
+#undef class
+#endif
+
+/**
+ * lkl_strerror - returns a string describing the given error code
+ *
+ * @err - error code
+ * @returns - string for the given error code
+ */
+const char *lkl_strerror(int err);
+
+/**
+ * lkl_perror - prints a string describing the given error code
+ *
+ * @msg - prefix for the error message
+ * @err - error code
+ */
+void lkl_perror(char *msg, int err);
+
+#if __LKL__BITS_PER_LONG == 64
+#define lkl_sys_fstatat lkl_sys_newfstatat
+#define lkl_sys_fstat lkl_sys_newfstat
+
+#else
+#define __lkl__NR_fcntl __lkl__NR_fcntl64
+
+#define lkl_stat lkl_stat64
+#define lkl_sys_stat lkl_sys_stat64
+#define lkl_sys_lstat lkl_sys_lstat64
+#define lkl_sys_truncate lkl_sys_truncate64
+#define lkl_sys_ftruncate lkl_sys_ftruncate64
+#define lkl_sys_sendfile lkl_sys_sendfile64
+#define lkl_sys_fstatat lkl_sys_fstatat64
+#define lkl_sys_fstat lkl_sys_fstat64
+#define lkl_sys_fcntl lkl_sys_fcntl64
+
+#define lkl_statfs lkl_statfs64
+
+static inline int lkl_sys_statfs(const char *path, struct lkl_statfs *buf)
+{
+	return lkl_sys_statfs64(path, sizeof(*buf), buf);
+}
+
+static inline int lkl_sys_fstatfs(unsigned int fd, struct lkl_statfs *buf)
+{
+	return lkl_sys_fstatfs64(fd, sizeof(*buf), buf);
+}
+
+#define lkl_sys_nanosleep lkl_sys_nanosleep_time32
+static inline int lkl_sys_nanosleep_time32(struct lkl_timespec *rqtp,
+					   struct lkl_timespec *rmtp)
+{
+	long p[6] = {(long)rqtp, (long)rmtp, 0, 0, 0, 0};
+
+	return lkl_syscall(__lkl__NR_nanosleep, p);
+}
+
+#endif
+
+static inline int lkl_sys_stat(const char *path, struct lkl_stat *buf)
+{
+	return lkl_sys_fstatat(LKL_AT_FDCWD, path, buf, 0);
+}
+
+static inline int lkl_sys_lstat(const char *path, struct lkl_stat *buf)
+{
+	return lkl_sys_fstatat(LKL_AT_FDCWD, path, buf,
+			       LKL_AT_SYMLINK_NOFOLLOW);
+}
+
+#ifdef __lkl__NR_openat
+/**
+ * lkl_sys_open - wrapper for lkl_sys_openat
+ */
+static inline long lkl_sys_open(const char *file, int flags, int mode)
+{
+	return lkl_sys_openat(LKL_AT_FDCWD, file, flags, mode);
+}
+
+/**
+ * lkl_sys_creat - wrapper for lkl_sys_openat
+ */
+static inline long lkl_sys_creat(const char *file, int mode)
+{
+	return lkl_sys_openat(LKL_AT_FDCWD, file,
+			      LKL_O_CREAT|LKL_O_WRONLY|LKL_O_TRUNC, mode);
+}
+#endif
+
+#ifdef __lkl__NR_mkdirat
+/**
+ * lkl_sys_mkdir - wrapper for lkl_sys_mkdirat
+ */
+static inline long lkl_sys_mkdir(const char *path, mode_t mode)
+{
+	return lkl_sys_mkdirat(LKL_AT_FDCWD, path, mode);
+}
+#endif
+
+#ifdef __lkl__NR_epoll_create1
+/**
+ * lkl_sys_epoll_create - wrapper for lkl_sys_epoll_create1
+ */
+static inline long lkl_sys_epoll_create(int size)
+{
+	return lkl_sys_epoll_create1(0);
+}
+#endif
+
+#ifdef __lkl__NR_epoll_pwait
+/**
+ * lkl_sys_epoll_wait - wrapper for lkl_sys_epoll_pwait
+ */
+static inline long lkl_sys_epoll_wait(int fd, struct lkl_epoll_event *ev,
+				      int cnt, int to)
+{
+	return lkl_sys_epoll_pwait(fd, ev, cnt, to, 0, _LKL_NSIG/8);
+}
+#endif
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif
diff --git a/tools/um/include/lkl_host.h b/tools/um/include/lkl_host.h
new file mode 100644
index 000000000000..85e80eb4ad0d
--- /dev/null
+++ b/tools/um/include/lkl_host.h
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_HOST_H
+#define _LKL_HOST_H
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include <lkl/asm/host_ops.h>
+#include <lkl.h>
+
+extern struct lkl_host_operations lkl_host_ops;
+
+/**
+ * lkl_printf - print a message via the host print operation
+ *
+ * @fmt: printf like format string
+ */
+int lkl_printf(const char *fmt, ...);
+
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif
diff --git a/tools/um/uml/Build b/tools/um/uml/Build
index 435a9670d3ff..191c6479ad72 100644
--- a/tools/um/uml/Build
+++ b/tools/um/uml/Build
@@ -8,11 +8,19 @@ KCOV_INSTRUMENT                := n
 
 include $(objtree)/include/config/auto.conf
 
-liblinux-y += execvp.o file.o helper.o irq.o main.o mem.o process.o \
-	registers.o sigio.o signal.o start_up.o time.o tty.o \
-	umid.o user_syms.o util.o drivers/ skas/
+liblinux-y += execvp.o file.o helper.o irq.o mem.o process.o \
+	registers.o sigio.o signal.o time.o tty.o \
+	umid.o user_syms.o util.o drivers/
 
+ifdef CONFIG_MMU
+liblinux-y += main.o start_up.o skas/
+HEADER_ARCH 	:= x86
 liblinux-y += x86/
+else
+HEADER_ARCH 	:= um/nommu
+liblinux-y += nommu/
+endif
+
 
 liblinux-$(CONFIG_ARCH_REUSE_HOST_VSYSCALL_AREA) += elf_aux.o
 
@@ -23,7 +31,6 @@ CFLAGS := -g -O2
 
 # from arch/um/Makefile
 ARCH_DIR := arch/um
-HEADER_ARCH 	:= x86
 HOST_DIR := arch/$(HEADER_ARCH)
 ifdef CONFIG_64BIT
   KBUILD_CFLAGS += -mcmodel=large
diff --git a/tools/um/uml/nommu/Build b/tools/um/uml/nommu/Build
new file mode 100644
index 000000000000..98fd6b86a085
--- /dev/null
+++ b/tools/um/uml/nommu/Build
@@ -0,0 +1 @@
+liblinux-y = registers.o unimplemented.o
diff --git a/tools/um/uml/nommu/registers.c b/tools/um/uml/nommu/registers.c
new file mode 100644
index 000000000000..11573a204720
--- /dev/null
+++ b/tools/um/uml/nommu/registers.c
@@ -0,0 +1,21 @@
+// SPDX-License-Identifier: GPL-2.0
+
+struct uml_pt_regs;
+
+int get_fp_registers(int pid, unsigned long *regs)
+{
+	return 0;
+}
+
+int save_i387_registers(int pid, unsigned long *fp_regs)
+{
+	return 0;
+}
+
+void arch_init_registers(int pid)
+{
+}
+
+void get_regs_from_mc(struct uml_pt_regs *regs, void *mc)
+{
+}
diff --git a/tools/um/uml/nommu/unimplemented.c b/tools/um/uml/nommu/unimplemented.c
new file mode 100644
index 000000000000..9da3e5c8bafb
--- /dev/null
+++ b/tools/um/uml/nommu/unimplemented.c
@@ -0,0 +1,21 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <generated/user_constants.h>
+
+struct uml_pt_regs;
+
+/* os-Linux/skas/process.c */
+int userspace_pid[UM_NR_CPUS];
+void userspace(struct uml_pt_regs *regs, unsigned long *aux_fp_regs)
+{}
+
+
+/* x86/os-Linux/task_size.c */
+unsigned long os_get_top_address(void)
+{
+	return 0;
+}
+
+/* start-up.c */
+void os_early_checks(void)
+{
+}
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v5 17/21] um: host: add nommu build for ARCH=um
@ 2020-07-02 14:07       ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-07-02 14:07 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, linux-kernel-library, linux-arch,
	Hajime Tazaki, Akira Moroo

This patch adds the skeleton for the host library.

The host library is implementing the host operations needed by nommu
mode and is split into host dependent (depends on a specific host, e.g.
POSIX hosts) and host independent parts (will work on all supported
hosts).

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/um/Makefile                  |   7 +-
 tools/um/Targets                   |   3 +-
 tools/um/include/lkl.h             | 151 +++++++++++++++++++++++++++++
 tools/um/include/lkl_host.h        |  26 +++++
 tools/um/uml/Build                 |  15 ++-
 tools/um/uml/nommu/Build           |   1 +
 tools/um/uml/nommu/registers.c     |  21 ++++
 tools/um/uml/nommu/unimplemented.c |  21 ++++
 8 files changed, 238 insertions(+), 7 deletions(-)
 create mode 100644 tools/um/include/lkl.h
 create mode 100644 tools/um/include/lkl_host.h
 create mode 100644 tools/um/uml/nommu/Build
 create mode 100644 tools/um/uml/nommu/registers.c
 create mode 100644 tools/um/uml/nommu/unimplemented.c

diff --git a/tools/um/Makefile b/tools/um/Makefile
index 552ed5f1edae..a93e060b1f89 100644
--- a/tools/um/Makefile
+++ b/tools/um/Makefile
@@ -5,6 +5,7 @@
 MAKEFLAGS += -r --no-print-directory
 
 KCONFIG?=defconfig
+UMMODE?=kernel
 
 ifneq ($(silent),1)
   ifneq ($(V),1)
@@ -47,8 +48,9 @@ all: $(TARGETS)
 
 # rule to build linux.o
 $(OUTPUT)lib/linux.o:
-	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) $(KCONFIG)
-	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) install INSTALL_PATH=$(OUTPUT)
+	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um UMMODE=$(UMMODE) $(KOPT) $(KCONFIG)
+	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um UMMODE=$(UMMODE) $(KOPT) install \
+	INSTALL_PATH=$(OUTPUT)
 
 $(OUTPUT)liblinux.a: $(OUTPUT)lib/linux.o $(OUTPUT)uml/liblinux-in.o $(OUTPUT)lib/liblinux-in.o
 	$(QUIET_AR)$(AR) -rc $@ $^
@@ -67,6 +69,7 @@ $(OUTPUT)%-in.o: $(OUTPUT)lib/linux.o FORCE
 clean:
 	$(call QUIET_CLEAN, objects)find $(OUTPUT) -name '*.o' -delete -o -name '\.*.cmd'\
 	 -delete -o -name '\.*.d' -delete
+	$(call QUIET_CLEAN, headers)$(RM) -r $(OUTPUT)/include/lkl/
 	$(call QUIET_CLEAN, liblinux.a)$(RM) $(OUTPUT)/liblinux.a
 	$(call QUIET_CLEAN, $(TARGETS))$(RM) $(TARGETS)
 
diff --git a/tools/um/Targets b/tools/um/Targets
index a4711f1ef422..4e1f0f4d81a3 100644
--- a/tools/um/Targets
+++ b/tools/um/Targets
@@ -1,4 +1,5 @@
 progs-y += uml/linux
-LDLIBS_linux-y := -lrt -lpthread -lutil
+
+LDLIBS := -lrt -lpthread -lutil
 LDFLAGS_linux-y := -no-pie -Wl,--wrap,malloc -Wl,--wrap,free -Wl,--wrap,calloc
 LDFLAGS_linux-$(UML_STATIC) += -static
diff --git a/tools/um/include/lkl.h b/tools/um/include/lkl.h
new file mode 100644
index 000000000000..707e01b64a70
--- /dev/null
+++ b/tools/um/include/lkl.h
@@ -0,0 +1,151 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_H
+#define _LKL_H
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#define _LKL_LIBC_COMPAT_H
+
+#ifdef __cplusplus
+#define class __lkl__class
+#endif
+
+/*
+ * Avoid collisions between Android which defines __unused and
+ * linux/icmp.h which uses __unused as a structure field.
+ */
+#pragma push_macro("__unused")
+#undef __unused
+
+#include <lkl/asm/syscalls.h>
+
+#pragma pop_macro("__unused")
+
+#ifdef __cplusplus
+#undef class
+#endif
+
+/**
+ * lkl_strerror - returns a string describing the given error code
+ *
+ * @err - error code
+ * @returns - string for the given error code
+ */
+const char *lkl_strerror(int err);
+
+/**
+ * lkl_perror - prints a string describing the given error code
+ *
+ * @msg - prefix for the error message
+ * @err - error code
+ */
+void lkl_perror(char *msg, int err);
+
+#if __LKL__BITS_PER_LONG == 64
+#define lkl_sys_fstatat lkl_sys_newfstatat
+#define lkl_sys_fstat lkl_sys_newfstat
+
+#else
+#define __lkl__NR_fcntl __lkl__NR_fcntl64
+
+#define lkl_stat lkl_stat64
+#define lkl_sys_stat lkl_sys_stat64
+#define lkl_sys_lstat lkl_sys_lstat64
+#define lkl_sys_truncate lkl_sys_truncate64
+#define lkl_sys_ftruncate lkl_sys_ftruncate64
+#define lkl_sys_sendfile lkl_sys_sendfile64
+#define lkl_sys_fstatat lkl_sys_fstatat64
+#define lkl_sys_fstat lkl_sys_fstat64
+#define lkl_sys_fcntl lkl_sys_fcntl64
+
+#define lkl_statfs lkl_statfs64
+
+static inline int lkl_sys_statfs(const char *path, struct lkl_statfs *buf)
+{
+	return lkl_sys_statfs64(path, sizeof(*buf), buf);
+}
+
+static inline int lkl_sys_fstatfs(unsigned int fd, struct lkl_statfs *buf)
+{
+	return lkl_sys_fstatfs64(fd, sizeof(*buf), buf);
+}
+
+#define lkl_sys_nanosleep lkl_sys_nanosleep_time32
+static inline int lkl_sys_nanosleep_time32(struct lkl_timespec *rqtp,
+					   struct lkl_timespec *rmtp)
+{
+	long p[6] = {(long)rqtp, (long)rmtp, 0, 0, 0, 0};
+
+	return lkl_syscall(__lkl__NR_nanosleep, p);
+}
+
+#endif
+
+static inline int lkl_sys_stat(const char *path, struct lkl_stat *buf)
+{
+	return lkl_sys_fstatat(LKL_AT_FDCWD, path, buf, 0);
+}
+
+static inline int lkl_sys_lstat(const char *path, struct lkl_stat *buf)
+{
+	return lkl_sys_fstatat(LKL_AT_FDCWD, path, buf,
+			       LKL_AT_SYMLINK_NOFOLLOW);
+}
+
+#ifdef __lkl__NR_openat
+/**
+ * lkl_sys_open - wrapper for lkl_sys_openat
+ */
+static inline long lkl_sys_open(const char *file, int flags, int mode)
+{
+	return lkl_sys_openat(LKL_AT_FDCWD, file, flags, mode);
+}
+
+/**
+ * lkl_sys_creat - wrapper for lkl_sys_openat
+ */
+static inline long lkl_sys_creat(const char *file, int mode)
+{
+	return lkl_sys_openat(LKL_AT_FDCWD, file,
+			      LKL_O_CREAT|LKL_O_WRONLY|LKL_O_TRUNC, mode);
+}
+#endif
+
+#ifdef __lkl__NR_mkdirat
+/**
+ * lkl_sys_mkdir - wrapper for lkl_sys_mkdirat
+ */
+static inline long lkl_sys_mkdir(const char *path, mode_t mode)
+{
+	return lkl_sys_mkdirat(LKL_AT_FDCWD, path, mode);
+}
+#endif
+
+#ifdef __lkl__NR_epoll_create1
+/**
+ * lkl_sys_epoll_create - wrapper for lkl_sys_epoll_create1
+ */
+static inline long lkl_sys_epoll_create(int size)
+{
+	return lkl_sys_epoll_create1(0);
+}
+#endif
+
+#ifdef __lkl__NR_epoll_pwait
+/**
+ * lkl_sys_epoll_wait - wrapper for lkl_sys_epoll_pwait
+ */
+static inline long lkl_sys_epoll_wait(int fd, struct lkl_epoll_event *ev,
+				      int cnt, int to)
+{
+	return lkl_sys_epoll_pwait(fd, ev, cnt, to, 0, _LKL_NSIG/8);
+}
+#endif
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif
diff --git a/tools/um/include/lkl_host.h b/tools/um/include/lkl_host.h
new file mode 100644
index 000000000000..85e80eb4ad0d
--- /dev/null
+++ b/tools/um/include/lkl_host.h
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_HOST_H
+#define _LKL_HOST_H
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include <lkl/asm/host_ops.h>
+#include <lkl.h>
+
+extern struct lkl_host_operations lkl_host_ops;
+
+/**
+ * lkl_printf - print a message via the host print operation
+ *
+ * @fmt: printf like format string
+ */
+int lkl_printf(const char *fmt, ...);
+
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif
diff --git a/tools/um/uml/Build b/tools/um/uml/Build
index 435a9670d3ff..191c6479ad72 100644
--- a/tools/um/uml/Build
+++ b/tools/um/uml/Build
@@ -8,11 +8,19 @@ KCOV_INSTRUMENT                := n
 
 include $(objtree)/include/config/auto.conf
 
-liblinux-y += execvp.o file.o helper.o irq.o main.o mem.o process.o \
-	registers.o sigio.o signal.o start_up.o time.o tty.o \
-	umid.o user_syms.o util.o drivers/ skas/
+liblinux-y += execvp.o file.o helper.o irq.o mem.o process.o \
+	registers.o sigio.o signal.o time.o tty.o \
+	umid.o user_syms.o util.o drivers/
 
+ifdef CONFIG_MMU
+liblinux-y += main.o start_up.o skas/
+HEADER_ARCH 	:= x86
 liblinux-y += x86/
+else
+HEADER_ARCH 	:= um/nommu
+liblinux-y += nommu/
+endif
+
 
 liblinux-$(CONFIG_ARCH_REUSE_HOST_VSYSCALL_AREA) += elf_aux.o
 
@@ -23,7 +31,6 @@ CFLAGS := -g -O2
 
 # from arch/um/Makefile
 ARCH_DIR := arch/um
-HEADER_ARCH 	:= x86
 HOST_DIR := arch/$(HEADER_ARCH)
 ifdef CONFIG_64BIT
   KBUILD_CFLAGS += -mcmodel=large
diff --git a/tools/um/uml/nommu/Build b/tools/um/uml/nommu/Build
new file mode 100644
index 000000000000..98fd6b86a085
--- /dev/null
+++ b/tools/um/uml/nommu/Build
@@ -0,0 +1 @@
+liblinux-y = registers.o unimplemented.o
diff --git a/tools/um/uml/nommu/registers.c b/tools/um/uml/nommu/registers.c
new file mode 100644
index 000000000000..11573a204720
--- /dev/null
+++ b/tools/um/uml/nommu/registers.c
@@ -0,0 +1,21 @@
+// SPDX-License-Identifier: GPL-2.0
+
+struct uml_pt_regs;
+
+int get_fp_registers(int pid, unsigned long *regs)
+{
+	return 0;
+}
+
+int save_i387_registers(int pid, unsigned long *fp_regs)
+{
+	return 0;
+}
+
+void arch_init_registers(int pid)
+{
+}
+
+void get_regs_from_mc(struct uml_pt_regs *regs, void *mc)
+{
+}
diff --git a/tools/um/uml/nommu/unimplemented.c b/tools/um/uml/nommu/unimplemented.c
new file mode 100644
index 000000000000..9da3e5c8bafb
--- /dev/null
+++ b/tools/um/uml/nommu/unimplemented.c
@@ -0,0 +1,21 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <generated/user_constants.h>
+
+struct uml_pt_regs;
+
+/* os-Linux/skas/process.c */
+int userspace_pid[UM_NR_CPUS];
+void userspace(struct uml_pt_regs *regs, unsigned long *aux_fp_regs)
+{}
+
+
+/* x86/os-Linux/task_size.c */
+unsigned long os_get_top_address(void)
+{
+	return 0;
+}
+
+/* start-up.c */
+void os_early_checks(void)
+{
+}
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v5 18/21] um: host: add utilities functions
  2020-07-02 14:06     ` Hajime Tazaki
@ 2020-07-02 14:07       ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-07-02 14:07 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Hajime Tazaki

Add basic utility functions for getting a string from a kernel error
code and a fprintf like function that uses the host print
operation. The latter is useful for informing the user about errors
that occur in the host library.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 tools/um/lib/Build     |   4 +
 tools/um/lib/jmp_buf.c |  14 +++
 tools/um/lib/jmp_buf.h |   8 ++
 tools/um/lib/utils.c   | 210 +++++++++++++++++++++++++++++++++++++++++
 4 files changed, 236 insertions(+)
 create mode 100644 tools/um/lib/Build
 create mode 100644 tools/um/lib/jmp_buf.c
 create mode 100644 tools/um/lib/jmp_buf.h
 create mode 100644 tools/um/lib/utils.c

diff --git a/tools/um/lib/Build b/tools/um/lib/Build
new file mode 100644
index 000000000000..fc967af4104a
--- /dev/null
+++ b/tools/um/lib/Build
@@ -0,0 +1,4 @@
+include $(objtree)/include/config/auto.conf
+
+liblinux-$(CONFIG_UMMODE_LIB) += utils.o
+liblinux-$(CONFIG_UMMODE_LIB) += jmp_buf.o
diff --git a/tools/um/lib/jmp_buf.c b/tools/um/lib/jmp_buf.c
new file mode 100644
index 000000000000..f6bdd7e4bd83
--- /dev/null
+++ b/tools/um/lib/jmp_buf.c
@@ -0,0 +1,14 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <setjmp.h>
+#include <lkl_host.h>
+
+void jmp_buf_set(struct lkl_jmp_buf *jmpb, void (*f)(void))
+{
+	if (!setjmp(*((jmp_buf *)jmpb->buf)))
+		f();
+}
+
+void jmp_buf_longjmp(struct lkl_jmp_buf *jmpb, int val)
+{
+	longjmp(*((jmp_buf *)jmpb->buf), val);
+}
diff --git a/tools/um/lib/jmp_buf.h b/tools/um/lib/jmp_buf.h
new file mode 100644
index 000000000000..8782cbaaf51f
--- /dev/null
+++ b/tools/um/lib/jmp_buf.h
@@ -0,0 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_LIB_JMP_BUF_H
+#define _LKL_LIB_JMP_BUF_H
+
+void jmp_buf_set(struct lkl_jmp_buf *jmpb, void (*f)(void));
+void jmp_buf_longjmp(struct lkl_jmp_buf *jmpb, int val);
+
+#endif
diff --git a/tools/um/lib/utils.c b/tools/um/lib/utils.c
new file mode 100644
index 000000000000..4930479a8a35
--- /dev/null
+++ b/tools/um/lib/utils.c
@@ -0,0 +1,210 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdarg.h>
+#include <stdio.h>
+#include <string.h>
+#include <lkl_host.h>
+
+static const char * const lkl_err_strings[] = {
+	"Success",
+	"Operation not permitted",
+	"No such file or directory",
+	"No such process",
+	"Interrupted system call",
+	"I/O error",
+	"No such device or address",
+	"Argument list too long",
+	"Exec format error",
+	"Bad file number",
+	"No child processes",
+	"Try again",
+	"Out of memory",
+	"Permission denied",
+	"Bad address",
+	"Block device required",
+	"Device or resource busy",
+	"File exists",
+	"Cross-device link",
+	"No such device",
+	"Not a directory",
+	"Is a directory",
+	"Invalid argument",
+	"File table overflow",
+	"Too many open files",
+	"Not a typewriter",
+	"Text file busy",
+	"File too large",
+	"No space left on device",
+	"Illegal seek",
+	"Read-only file system",
+	"Too many links",
+	"Broken pipe",
+	"Math argument out of domain of func",
+	"Math result not representable",
+	"Resource deadlock would occur",
+	"File name too long",
+	"No record locks available",
+	"Invalid system call number",
+	"Directory not empty",
+	"Too many symbolic links encountered",
+	"Bad error code", /* EWOULDBLOCK is EAGAIN */
+	"No message of desired type",
+	"Identifier removed",
+	"Channel number out of range",
+	"Level 2 not synchronized",
+	"Level 3 halted",
+	"Level 3 reset",
+	"Link number out of range",
+	"Protocol driver not attached",
+	"No CSI structure available",
+	"Level 2 halted",
+	"Invalid exchange",
+	"Invalid request descriptor",
+	"Exchange full",
+	"No anode",
+	"Invalid request code",
+	"Invalid slot",
+	"Bad error code", /* EDEADLOCK is EDEADLK */
+	"Bad font file format",
+	"Device not a stream",
+	"No data available",
+	"Timer expired",
+	"Out of streams resources",
+	"Machine is not on the network",
+	"Package not installed",
+	"Object is remote",
+	"Link has been severed",
+	"Advertise error",
+	"Srmount error",
+	"Communication error on send",
+	"Protocol error",
+	"Multihop attempted",
+	"RFS specific error",
+	"Not a data message",
+	"Value too large for defined data type",
+	"Name not unique on network",
+	"File descriptor in bad state",
+	"Remote address changed",
+	"Can not access a needed shared library",
+	"Accessing a corrupted shared library",
+	".lib section in a.out corrupted",
+	"Attempting to link in too many shared libraries",
+	"Cannot exec a shared library directly",
+	"Illegal byte sequence",
+	"Interrupted system call should be restarted",
+	"Streams pipe error",
+	"Too many users",
+	"Socket operation on non-socket",
+	"Destination address required",
+	"Message too long",
+	"Protocol wrong type for socket",
+	"Protocol not available",
+	"Protocol not supported",
+	"Socket type not supported",
+	"Operation not supported on transport endpoint",
+	"Protocol family not supported",
+	"Address family not supported by protocol",
+	"Address already in use",
+	"Cannot assign requested address",
+	"Network is down",
+	"Network is unreachable",
+	"Network dropped connection because of reset",
+	"Software caused connection abort",
+	"Connection reset by peer",
+	"No buffer space available",
+	"Transport endpoint is already connected",
+	"Transport endpoint is not connected",
+	"Cannot send after transport endpoint shutdown",
+	"Too many references: cannot splice",
+	"Connection timed out",
+	"Connection refused",
+	"Host is down",
+	"No route to host",
+	"Operation already in progress",
+	"Operation now in progress",
+	"Stale file handle",
+	"Structure needs cleaning",
+	"Not a XENIX named type file",
+	"No XENIX semaphores available",
+	"Is a named type file",
+	"Remote I/O error",
+	"Quota exceeded",
+	"No medium found",
+	"Wrong medium type",
+	"Operation Canceled",
+	"Required key not available",
+	"Key has expired",
+	"Key has been revoked",
+	"Key was rejected by service",
+	"Owner died",
+	"State not recoverable",
+	"Operation not possible due to RF-kill",
+	"Memory page has hardware error",
+};
+
+const char *lkl_strerror(int err)
+{
+	if (err < 0)
+		err = -err;
+
+	if ((size_t)err >= sizeof(lkl_err_strings) / sizeof(const char *))
+		return "Bad error code";
+
+	return lkl_err_strings[err];
+}
+
+void lkl_perror(char *msg, int err)
+{
+	const char *err_msg = lkl_strerror(err);
+	/* We need to use 'real' printf because lkl_host_ops.print can
+	 * be turned off when debugging is off.
+	 */
+	lkl_printf("%s: %s\n", msg, err_msg);
+}
+
+static int lkl_vprintf(const char *fmt, va_list args)
+{
+	int n;
+	char *buffer;
+	va_list copy;
+
+	if (!lkl_host_ops.print)
+		return 0;
+
+	va_copy(copy, args);
+	n = vsnprintf(NULL, 0, fmt, copy);
+	va_end(copy);
+
+	buffer = lkl_host_ops.mem_alloc(n + 1);
+	if (!buffer)
+		return (-1);
+
+	vsnprintf(buffer, n + 1, fmt, args);
+
+	lkl_host_ops.print(buffer, n);
+	lkl_host_ops.mem_free(buffer);
+
+	return n;
+}
+
+int lkl_printf(const char *fmt, ...)
+{
+	int n;
+	va_list args;
+
+	va_start(args, fmt);
+	n = lkl_vprintf(fmt, args);
+	va_end(args);
+
+	return n;
+}
+
+void lkl_bug(const char *fmt, ...)
+{
+	va_list args;
+
+	va_start(args, fmt);
+	lkl_vprintf(fmt, args);
+	va_end(args);
+
+	lkl_host_ops.panic();
+}
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v5 18/21] um: host: add utilities functions
@ 2020-07-02 14:07       ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-07-02 14:07 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, linux-kernel-library, linux-arch,
	Hajime Tazaki, Akira Moroo

Add basic utility functions for getting a string from a kernel error
code and a fprintf like function that uses the host print
operation. The latter is useful for informing the user about errors
that occur in the host library.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 tools/um/lib/Build     |   4 +
 tools/um/lib/jmp_buf.c |  14 +++
 tools/um/lib/jmp_buf.h |   8 ++
 tools/um/lib/utils.c   | 210 +++++++++++++++++++++++++++++++++++++++++
 4 files changed, 236 insertions(+)
 create mode 100644 tools/um/lib/Build
 create mode 100644 tools/um/lib/jmp_buf.c
 create mode 100644 tools/um/lib/jmp_buf.h
 create mode 100644 tools/um/lib/utils.c

diff --git a/tools/um/lib/Build b/tools/um/lib/Build
new file mode 100644
index 000000000000..fc967af4104a
--- /dev/null
+++ b/tools/um/lib/Build
@@ -0,0 +1,4 @@
+include $(objtree)/include/config/auto.conf
+
+liblinux-$(CONFIG_UMMODE_LIB) += utils.o
+liblinux-$(CONFIG_UMMODE_LIB) += jmp_buf.o
diff --git a/tools/um/lib/jmp_buf.c b/tools/um/lib/jmp_buf.c
new file mode 100644
index 000000000000..f6bdd7e4bd83
--- /dev/null
+++ b/tools/um/lib/jmp_buf.c
@@ -0,0 +1,14 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <setjmp.h>
+#include <lkl_host.h>
+
+void jmp_buf_set(struct lkl_jmp_buf *jmpb, void (*f)(void))
+{
+	if (!setjmp(*((jmp_buf *)jmpb->buf)))
+		f();
+}
+
+void jmp_buf_longjmp(struct lkl_jmp_buf *jmpb, int val)
+{
+	longjmp(*((jmp_buf *)jmpb->buf), val);
+}
diff --git a/tools/um/lib/jmp_buf.h b/tools/um/lib/jmp_buf.h
new file mode 100644
index 000000000000..8782cbaaf51f
--- /dev/null
+++ b/tools/um/lib/jmp_buf.h
@@ -0,0 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_LIB_JMP_BUF_H
+#define _LKL_LIB_JMP_BUF_H
+
+void jmp_buf_set(struct lkl_jmp_buf *jmpb, void (*f)(void));
+void jmp_buf_longjmp(struct lkl_jmp_buf *jmpb, int val);
+
+#endif
diff --git a/tools/um/lib/utils.c b/tools/um/lib/utils.c
new file mode 100644
index 000000000000..4930479a8a35
--- /dev/null
+++ b/tools/um/lib/utils.c
@@ -0,0 +1,210 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdarg.h>
+#include <stdio.h>
+#include <string.h>
+#include <lkl_host.h>
+
+static const char * const lkl_err_strings[] = {
+	"Success",
+	"Operation not permitted",
+	"No such file or directory",
+	"No such process",
+	"Interrupted system call",
+	"I/O error",
+	"No such device or address",
+	"Argument list too long",
+	"Exec format error",
+	"Bad file number",
+	"No child processes",
+	"Try again",
+	"Out of memory",
+	"Permission denied",
+	"Bad address",
+	"Block device required",
+	"Device or resource busy",
+	"File exists",
+	"Cross-device link",
+	"No such device",
+	"Not a directory",
+	"Is a directory",
+	"Invalid argument",
+	"File table overflow",
+	"Too many open files",
+	"Not a typewriter",
+	"Text file busy",
+	"File too large",
+	"No space left on device",
+	"Illegal seek",
+	"Read-only file system",
+	"Too many links",
+	"Broken pipe",
+	"Math argument out of domain of func",
+	"Math result not representable",
+	"Resource deadlock would occur",
+	"File name too long",
+	"No record locks available",
+	"Invalid system call number",
+	"Directory not empty",
+	"Too many symbolic links encountered",
+	"Bad error code", /* EWOULDBLOCK is EAGAIN */
+	"No message of desired type",
+	"Identifier removed",
+	"Channel number out of range",
+	"Level 2 not synchronized",
+	"Level 3 halted",
+	"Level 3 reset",
+	"Link number out of range",
+	"Protocol driver not attached",
+	"No CSI structure available",
+	"Level 2 halted",
+	"Invalid exchange",
+	"Invalid request descriptor",
+	"Exchange full",
+	"No anode",
+	"Invalid request code",
+	"Invalid slot",
+	"Bad error code", /* EDEADLOCK is EDEADLK */
+	"Bad font file format",
+	"Device not a stream",
+	"No data available",
+	"Timer expired",
+	"Out of streams resources",
+	"Machine is not on the network",
+	"Package not installed",
+	"Object is remote",
+	"Link has been severed",
+	"Advertise error",
+	"Srmount error",
+	"Communication error on send",
+	"Protocol error",
+	"Multihop attempted",
+	"RFS specific error",
+	"Not a data message",
+	"Value too large for defined data type",
+	"Name not unique on network",
+	"File descriptor in bad state",
+	"Remote address changed",
+	"Can not access a needed shared library",
+	"Accessing a corrupted shared library",
+	".lib section in a.out corrupted",
+	"Attempting to link in too many shared libraries",
+	"Cannot exec a shared library directly",
+	"Illegal byte sequence",
+	"Interrupted system call should be restarted",
+	"Streams pipe error",
+	"Too many users",
+	"Socket operation on non-socket",
+	"Destination address required",
+	"Message too long",
+	"Protocol wrong type for socket",
+	"Protocol not available",
+	"Protocol not supported",
+	"Socket type not supported",
+	"Operation not supported on transport endpoint",
+	"Protocol family not supported",
+	"Address family not supported by protocol",
+	"Address already in use",
+	"Cannot assign requested address",
+	"Network is down",
+	"Network is unreachable",
+	"Network dropped connection because of reset",
+	"Software caused connection abort",
+	"Connection reset by peer",
+	"No buffer space available",
+	"Transport endpoint is already connected",
+	"Transport endpoint is not connected",
+	"Cannot send after transport endpoint shutdown",
+	"Too many references: cannot splice",
+	"Connection timed out",
+	"Connection refused",
+	"Host is down",
+	"No route to host",
+	"Operation already in progress",
+	"Operation now in progress",
+	"Stale file handle",
+	"Structure needs cleaning",
+	"Not a XENIX named type file",
+	"No XENIX semaphores available",
+	"Is a named type file",
+	"Remote I/O error",
+	"Quota exceeded",
+	"No medium found",
+	"Wrong medium type",
+	"Operation Canceled",
+	"Required key not available",
+	"Key has expired",
+	"Key has been revoked",
+	"Key was rejected by service",
+	"Owner died",
+	"State not recoverable",
+	"Operation not possible due to RF-kill",
+	"Memory page has hardware error",
+};
+
+const char *lkl_strerror(int err)
+{
+	if (err < 0)
+		err = -err;
+
+	if ((size_t)err >= sizeof(lkl_err_strings) / sizeof(const char *))
+		return "Bad error code";
+
+	return lkl_err_strings[err];
+}
+
+void lkl_perror(char *msg, int err)
+{
+	const char *err_msg = lkl_strerror(err);
+	/* We need to use 'real' printf because lkl_host_ops.print can
+	 * be turned off when debugging is off.
+	 */
+	lkl_printf("%s: %s\n", msg, err_msg);
+}
+
+static int lkl_vprintf(const char *fmt, va_list args)
+{
+	int n;
+	char *buffer;
+	va_list copy;
+
+	if (!lkl_host_ops.print)
+		return 0;
+
+	va_copy(copy, args);
+	n = vsnprintf(NULL, 0, fmt, copy);
+	va_end(copy);
+
+	buffer = lkl_host_ops.mem_alloc(n + 1);
+	if (!buffer)
+		return (-1);
+
+	vsnprintf(buffer, n + 1, fmt, args);
+
+	lkl_host_ops.print(buffer, n);
+	lkl_host_ops.mem_free(buffer);
+
+	return n;
+}
+
+int lkl_printf(const char *fmt, ...)
+{
+	int n;
+	va_list args;
+
+	va_start(args, fmt);
+	n = lkl_vprintf(fmt, args);
+	va_end(args);
+
+	return n;
+}
+
+void lkl_bug(const char *fmt, ...)
+{
+	va_list args;
+
+	va_start(args, fmt);
+	lkl_vprintf(fmt, args);
+	va_end(args);
+
+	lkl_host_ops.panic();
+}
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v5 19/21] um: host: posix host operations
  2020-07-02 14:06     ` Hajime Tazaki
@ 2020-07-02 14:07       ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-07-02 14:07 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Hajime Tazaki

This commit implements LKL host operations for POSIX hosts.  The
operations are implemented to be portable in various POSIX OSs.

Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 tools/um/lib/Build        |   2 +
 tools/um/lib/posix-host.c | 292 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 294 insertions(+)
 create mode 100644 tools/um/lib/posix-host.c

diff --git a/tools/um/lib/Build b/tools/um/lib/Build
index fc967af4104a..dddff26a3b4e 100644
--- a/tools/um/lib/Build
+++ b/tools/um/lib/Build
@@ -1,4 +1,6 @@
 include $(objtree)/include/config/auto.conf
+CFLAGS_posix-host.o += -D_FILE_OFFSET_BITS=64
 
 liblinux-$(CONFIG_UMMODE_LIB) += utils.o
+liblinux-$(CONFIG_UMMODE_LIB) += posix-host.o
 liblinux-$(CONFIG_UMMODE_LIB) += jmp_buf.o
diff --git a/tools/um/lib/posix-host.c b/tools/um/lib/posix-host.c
new file mode 100644
index 000000000000..b6b5b2902254
--- /dev/null
+++ b/tools/um/lib/posix-host.c
@@ -0,0 +1,292 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <pthread.h>
+#include <limits.h>
+#include <errno.h>
+#include <string.h>
+#include <stdlib.h>
+#include <assert.h>
+#include <sys/syscall.h>
+#include <lkl_host.h>
+#include "jmp_buf.h"
+
+/* Let's see if the host has semaphore.h */
+#include <unistd.h>
+
+#ifdef _POSIX_SEMAPHORES
+#include <semaphore.h>
+/* TODO(pscollins): We don't support fork() for now, but maybe one day
+ * we will?
+ */
+#define SHARE_SEM 0
+#endif /* _POSIX_SEMAPHORES */
+
+
+struct lkl_mutex {
+	pthread_mutex_t mutex;
+};
+
+struct lkl_sem {
+#ifdef _POSIX_SEMAPHORES
+	sem_t sem;
+#else
+	pthread_mutex_t lock;
+	int count;
+	pthread_cond_t cond;
+#endif /* _POSIX_SEMAPHORES */
+};
+
+struct lkl_tls_key {
+	pthread_key_t key;
+};
+
+static void print(const char *str, int len)
+{
+	int ret __attribute__((unused));
+
+	ret = write(STDOUT_FILENO, str, len);
+}
+
+static void panic(void)
+{
+	assert(0);
+}
+
+#define WARN_UNLESS(exp) do {						\
+		if (exp < 0)						\
+			lkl_printf("%s: %s\n", #exp, strerror(errno));	\
+	} while (0)
+
+static int _warn_pthread(int ret, char *str_exp)
+{
+	if (ret > 0)
+		lkl_printf("%s: %s\n", str_exp, strerror(ret));
+
+	return ret;
+}
+
+
+/* pthread_* functions use the reverse convention */
+#define WARN_PTHREAD(exp) _warn_pthread(exp, #exp)
+
+static struct lkl_sem *sem_alloc(int count)
+{
+	struct lkl_sem *sem;
+
+	sem = malloc(sizeof(*sem));
+	if (!sem)
+		return NULL;
+
+#ifdef _POSIX_SEMAPHORES
+	if (sem_init(&sem->sem, SHARE_SEM, count) < 0) {
+		lkl_printf("sem_init: %s\n", strerror(errno));
+		free(sem);
+		return NULL;
+	}
+#else
+	pthread_mutex_init(&sem->lock, NULL);
+	sem->count = count;
+	WARN_PTHREAD(pthread_cond_init(&sem->cond, NULL));
+#endif /* _POSIX_SEMAPHORES */
+
+	return sem;
+}
+
+static void sem_free(struct lkl_sem *sem)
+{
+#ifdef _POSIX_SEMAPHORES
+	WARN_UNLESS(sem_destroy(&sem->sem));
+#else
+	WARN_PTHREAD(pthread_cond_destroy(&sem->cond));
+	WARN_PTHREAD(pthread_mutex_destroy(&sem->lock));
+#endif /* _POSIX_SEMAPHORES */
+	free(sem);
+}
+
+static void sem_up(struct lkl_sem *sem)
+{
+#ifdef _POSIX_SEMAPHORES
+	WARN_UNLESS(sem_post(&sem->sem));
+#else
+	WARN_PTHREAD(pthread_mutex_lock(&sem->lock));
+	sem->count++;
+	if (sem->count > 0)
+		WARN_PTHREAD(pthread_cond_signal(&sem->cond));
+	WARN_PTHREAD(pthread_mutex_unlock(&sem->lock));
+#endif /* _POSIX_SEMAPHORES */
+
+}
+
+static void sem_down(struct lkl_sem *sem)
+{
+#ifdef _POSIX_SEMAPHORES
+	int err;
+
+	do {
+		err = sem_wait(&sem->sem);
+	} while (err < 0 && errno == EINTR);
+	if (err < 0 && errno != EINTR)
+		lkl_printf("sem_wait: %s\n", strerror(errno));
+#else
+	WARN_PTHREAD(pthread_mutex_lock(&sem->lock));
+	while (sem->count <= 0)
+		WARN_PTHREAD(pthread_cond_wait(&sem->cond, &sem->lock));
+	sem->count--;
+	WARN_PTHREAD(pthread_mutex_unlock(&sem->lock));
+#endif /* _POSIX_SEMAPHORES */
+}
+
+static struct lkl_mutex *mutex_alloc(int recursive)
+{
+	struct lkl_mutex *_mutex = malloc(sizeof(struct lkl_mutex));
+	pthread_mutex_t *mutex = NULL;
+	pthread_mutexattr_t attr;
+
+	if (!_mutex)
+		return NULL;
+
+	mutex = &_mutex->mutex;
+	WARN_PTHREAD(pthread_mutexattr_init(&attr));
+
+	/* PTHREAD_MUTEX_ERRORCHECK is *very* useful for debugging,
+	 * but has some overhead, so we provide an option to turn it
+	 * off.
+	 */
+#ifdef DEBUG
+	if (!recursive)
+		WARN_PTHREAD(pthread_mutexattr_settype(
+				     &attr, PTHREAD_MUTEX_ERRORCHECK));
+#endif /* DEBUG */
+
+	if (recursive)
+		WARN_PTHREAD(pthread_mutexattr_settype(
+				     &attr, PTHREAD_MUTEX_RECURSIVE));
+
+	WARN_PTHREAD(pthread_mutex_init(mutex, &attr));
+
+	return _mutex;
+}
+
+static void mutex_lock(struct lkl_mutex *mutex)
+{
+	WARN_PTHREAD(pthread_mutex_lock(&mutex->mutex));
+}
+
+static void mutex_unlock(struct lkl_mutex *_mutex)
+{
+	pthread_mutex_t *mutex = &_mutex->mutex;
+
+	WARN_PTHREAD(pthread_mutex_unlock(mutex));
+}
+
+static void mutex_free(struct lkl_mutex *_mutex)
+{
+	pthread_mutex_t *mutex = &_mutex->mutex;
+
+	WARN_PTHREAD(pthread_mutex_destroy(mutex));
+	free(_mutex);
+}
+
+static lkl_thread_t thread_create(void* (*fn)(void *), void *arg)
+{
+	pthread_t thread;
+
+	if (WARN_PTHREAD(pthread_create(&thread, NULL, fn, arg)))
+		return 0;
+	else
+		return (lkl_thread_t) thread;
+}
+
+static void thread_detach(void)
+{
+	WARN_PTHREAD(pthread_detach(pthread_self()));
+}
+
+static void thread_exit(void)
+{
+	pthread_exit(NULL);
+}
+
+static int thread_join(lkl_thread_t tid)
+{
+	if (WARN_PTHREAD(pthread_join((pthread_t)tid, NULL)))
+		return (-1);
+	else
+		return 0;
+}
+
+static lkl_thread_t thread_self(void)
+{
+	return (lkl_thread_t)pthread_self();
+}
+
+static int thread_equal(lkl_thread_t a, lkl_thread_t b)
+{
+	return pthread_equal((pthread_t)a, (pthread_t)b);
+}
+
+static long _gettid(void)
+{
+#ifdef	__FreeBSD__
+	return (long)pthread_self();
+#else
+	return syscall(SYS_gettid);
+#endif
+}
+
+static struct lkl_tls_key *tls_alloc(void (*destructor)(void *))
+{
+	struct lkl_tls_key *ret = malloc(sizeof(struct lkl_tls_key));
+
+	if (WARN_PTHREAD(pthread_key_create(&ret->key, destructor))) {
+		free(ret);
+		return NULL;
+	}
+	return ret;
+}
+
+static void tls_free(struct lkl_tls_key *key)
+{
+	WARN_PTHREAD(pthread_key_delete(key->key));
+	free(key);
+}
+
+static int tls_set(struct lkl_tls_key *key, void *data)
+{
+	if (WARN_PTHREAD(pthread_setspecific(key->key, data)))
+		return (-1);
+	return 0;
+}
+
+static void *tls_get(struct lkl_tls_key *key)
+{
+	return pthread_getspecific(key->key);
+}
+
+struct lkl_host_operations lkl_host_ops = {
+	.panic = panic,
+	.print = print,
+	.mem_alloc = (void *)malloc,
+	.mem_free = free,
+	.thread_create = thread_create,
+	.thread_detach = thread_detach,
+	.thread_exit = thread_exit,
+	.thread_join = thread_join,
+	.thread_self = thread_self,
+	.thread_equal = thread_equal,
+	.sem_alloc = sem_alloc,
+	.sem_free = sem_free,
+	.sem_up = sem_up,
+	.sem_down = sem_down,
+	.mutex_alloc = mutex_alloc,
+	.mutex_free = mutex_free,
+	.mutex_lock = mutex_lock,
+	.mutex_unlock = mutex_unlock,
+	.gettid = _gettid,
+	.tls_alloc = tls_alloc,
+	.tls_free = tls_free,
+	.tls_set = tls_set,
+	.tls_get = tls_get,
+	.jmp_buf_set = jmp_buf_set,
+	.jmp_buf_longjmp = jmp_buf_longjmp,
+};
+
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v5 19/21] um: host: posix host operations
@ 2020-07-02 14:07       ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-07-02 14:07 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, linux-kernel-library, linux-arch,
	Hajime Tazaki, Akira Moroo

This commit implements LKL host operations for POSIX hosts.  The
operations are implemented to be portable in various POSIX OSs.

Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 tools/um/lib/Build        |   2 +
 tools/um/lib/posix-host.c | 292 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 294 insertions(+)
 create mode 100644 tools/um/lib/posix-host.c

diff --git a/tools/um/lib/Build b/tools/um/lib/Build
index fc967af4104a..dddff26a3b4e 100644
--- a/tools/um/lib/Build
+++ b/tools/um/lib/Build
@@ -1,4 +1,6 @@
 include $(objtree)/include/config/auto.conf
+CFLAGS_posix-host.o += -D_FILE_OFFSET_BITS=64
 
 liblinux-$(CONFIG_UMMODE_LIB) += utils.o
+liblinux-$(CONFIG_UMMODE_LIB) += posix-host.o
 liblinux-$(CONFIG_UMMODE_LIB) += jmp_buf.o
diff --git a/tools/um/lib/posix-host.c b/tools/um/lib/posix-host.c
new file mode 100644
index 000000000000..b6b5b2902254
--- /dev/null
+++ b/tools/um/lib/posix-host.c
@@ -0,0 +1,292 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <pthread.h>
+#include <limits.h>
+#include <errno.h>
+#include <string.h>
+#include <stdlib.h>
+#include <assert.h>
+#include <sys/syscall.h>
+#include <lkl_host.h>
+#include "jmp_buf.h"
+
+/* Let's see if the host has semaphore.h */
+#include <unistd.h>
+
+#ifdef _POSIX_SEMAPHORES
+#include <semaphore.h>
+/* TODO(pscollins): We don't support fork() for now, but maybe one day
+ * we will?
+ */
+#define SHARE_SEM 0
+#endif /* _POSIX_SEMAPHORES */
+
+
+struct lkl_mutex {
+	pthread_mutex_t mutex;
+};
+
+struct lkl_sem {
+#ifdef _POSIX_SEMAPHORES
+	sem_t sem;
+#else
+	pthread_mutex_t lock;
+	int count;
+	pthread_cond_t cond;
+#endif /* _POSIX_SEMAPHORES */
+};
+
+struct lkl_tls_key {
+	pthread_key_t key;
+};
+
+static void print(const char *str, int len)
+{
+	int ret __attribute__((unused));
+
+	ret = write(STDOUT_FILENO, str, len);
+}
+
+static void panic(void)
+{
+	assert(0);
+}
+
+#define WARN_UNLESS(exp) do {						\
+		if (exp < 0)						\
+			lkl_printf("%s: %s\n", #exp, strerror(errno));	\
+	} while (0)
+
+static int _warn_pthread(int ret, char *str_exp)
+{
+	if (ret > 0)
+		lkl_printf("%s: %s\n", str_exp, strerror(ret));
+
+	return ret;
+}
+
+
+/* pthread_* functions use the reverse convention */
+#define WARN_PTHREAD(exp) _warn_pthread(exp, #exp)
+
+static struct lkl_sem *sem_alloc(int count)
+{
+	struct lkl_sem *sem;
+
+	sem = malloc(sizeof(*sem));
+	if (!sem)
+		return NULL;
+
+#ifdef _POSIX_SEMAPHORES
+	if (sem_init(&sem->sem, SHARE_SEM, count) < 0) {
+		lkl_printf("sem_init: %s\n", strerror(errno));
+		free(sem);
+		return NULL;
+	}
+#else
+	pthread_mutex_init(&sem->lock, NULL);
+	sem->count = count;
+	WARN_PTHREAD(pthread_cond_init(&sem->cond, NULL));
+#endif /* _POSIX_SEMAPHORES */
+
+	return sem;
+}
+
+static void sem_free(struct lkl_sem *sem)
+{
+#ifdef _POSIX_SEMAPHORES
+	WARN_UNLESS(sem_destroy(&sem->sem));
+#else
+	WARN_PTHREAD(pthread_cond_destroy(&sem->cond));
+	WARN_PTHREAD(pthread_mutex_destroy(&sem->lock));
+#endif /* _POSIX_SEMAPHORES */
+	free(sem);
+}
+
+static void sem_up(struct lkl_sem *sem)
+{
+#ifdef _POSIX_SEMAPHORES
+	WARN_UNLESS(sem_post(&sem->sem));
+#else
+	WARN_PTHREAD(pthread_mutex_lock(&sem->lock));
+	sem->count++;
+	if (sem->count > 0)
+		WARN_PTHREAD(pthread_cond_signal(&sem->cond));
+	WARN_PTHREAD(pthread_mutex_unlock(&sem->lock));
+#endif /* _POSIX_SEMAPHORES */
+
+}
+
+static void sem_down(struct lkl_sem *sem)
+{
+#ifdef _POSIX_SEMAPHORES
+	int err;
+
+	do {
+		err = sem_wait(&sem->sem);
+	} while (err < 0 && errno == EINTR);
+	if (err < 0 && errno != EINTR)
+		lkl_printf("sem_wait: %s\n", strerror(errno));
+#else
+	WARN_PTHREAD(pthread_mutex_lock(&sem->lock));
+	while (sem->count <= 0)
+		WARN_PTHREAD(pthread_cond_wait(&sem->cond, &sem->lock));
+	sem->count--;
+	WARN_PTHREAD(pthread_mutex_unlock(&sem->lock));
+#endif /* _POSIX_SEMAPHORES */
+}
+
+static struct lkl_mutex *mutex_alloc(int recursive)
+{
+	struct lkl_mutex *_mutex = malloc(sizeof(struct lkl_mutex));
+	pthread_mutex_t *mutex = NULL;
+	pthread_mutexattr_t attr;
+
+	if (!_mutex)
+		return NULL;
+
+	mutex = &_mutex->mutex;
+	WARN_PTHREAD(pthread_mutexattr_init(&attr));
+
+	/* PTHREAD_MUTEX_ERRORCHECK is *very* useful for debugging,
+	 * but has some overhead, so we provide an option to turn it
+	 * off.
+	 */
+#ifdef DEBUG
+	if (!recursive)
+		WARN_PTHREAD(pthread_mutexattr_settype(
+				     &attr, PTHREAD_MUTEX_ERRORCHECK));
+#endif /* DEBUG */
+
+	if (recursive)
+		WARN_PTHREAD(pthread_mutexattr_settype(
+				     &attr, PTHREAD_MUTEX_RECURSIVE));
+
+	WARN_PTHREAD(pthread_mutex_init(mutex, &attr));
+
+	return _mutex;
+}
+
+static void mutex_lock(struct lkl_mutex *mutex)
+{
+	WARN_PTHREAD(pthread_mutex_lock(&mutex->mutex));
+}
+
+static void mutex_unlock(struct lkl_mutex *_mutex)
+{
+	pthread_mutex_t *mutex = &_mutex->mutex;
+
+	WARN_PTHREAD(pthread_mutex_unlock(mutex));
+}
+
+static void mutex_free(struct lkl_mutex *_mutex)
+{
+	pthread_mutex_t *mutex = &_mutex->mutex;
+
+	WARN_PTHREAD(pthread_mutex_destroy(mutex));
+	free(_mutex);
+}
+
+static lkl_thread_t thread_create(void* (*fn)(void *), void *arg)
+{
+	pthread_t thread;
+
+	if (WARN_PTHREAD(pthread_create(&thread, NULL, fn, arg)))
+		return 0;
+	else
+		return (lkl_thread_t) thread;
+}
+
+static void thread_detach(void)
+{
+	WARN_PTHREAD(pthread_detach(pthread_self()));
+}
+
+static void thread_exit(void)
+{
+	pthread_exit(NULL);
+}
+
+static int thread_join(lkl_thread_t tid)
+{
+	if (WARN_PTHREAD(pthread_join((pthread_t)tid, NULL)))
+		return (-1);
+	else
+		return 0;
+}
+
+static lkl_thread_t thread_self(void)
+{
+	return (lkl_thread_t)pthread_self();
+}
+
+static int thread_equal(lkl_thread_t a, lkl_thread_t b)
+{
+	return pthread_equal((pthread_t)a, (pthread_t)b);
+}
+
+static long _gettid(void)
+{
+#ifdef	__FreeBSD__
+	return (long)pthread_self();
+#else
+	return syscall(SYS_gettid);
+#endif
+}
+
+static struct lkl_tls_key *tls_alloc(void (*destructor)(void *))
+{
+	struct lkl_tls_key *ret = malloc(sizeof(struct lkl_tls_key));
+
+	if (WARN_PTHREAD(pthread_key_create(&ret->key, destructor))) {
+		free(ret);
+		return NULL;
+	}
+	return ret;
+}
+
+static void tls_free(struct lkl_tls_key *key)
+{
+	WARN_PTHREAD(pthread_key_delete(key->key));
+	free(key);
+}
+
+static int tls_set(struct lkl_tls_key *key, void *data)
+{
+	if (WARN_PTHREAD(pthread_setspecific(key->key, data)))
+		return (-1);
+	return 0;
+}
+
+static void *tls_get(struct lkl_tls_key *key)
+{
+	return pthread_getspecific(key->key);
+}
+
+struct lkl_host_operations lkl_host_ops = {
+	.panic = panic,
+	.print = print,
+	.mem_alloc = (void *)malloc,
+	.mem_free = free,
+	.thread_create = thread_create,
+	.thread_detach = thread_detach,
+	.thread_exit = thread_exit,
+	.thread_join = thread_join,
+	.thread_self = thread_self,
+	.thread_equal = thread_equal,
+	.sem_alloc = sem_alloc,
+	.sem_free = sem_free,
+	.sem_up = sem_up,
+	.sem_down = sem_down,
+	.mutex_alloc = mutex_alloc,
+	.mutex_free = mutex_free,
+	.mutex_lock = mutex_lock,
+	.mutex_unlock = mutex_unlock,
+	.gettid = _gettid,
+	.tls_alloc = tls_alloc,
+	.tls_free = tls_free,
+	.tls_set = tls_set,
+	.tls_get = tls_get,
+	.jmp_buf_set = jmp_buf_set,
+	.jmp_buf_longjmp = jmp_buf_longjmp,
+};
+
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v5 20/21] um: host: add test programs
  2020-07-02 14:06     ` Hajime Tazaki
@ 2020-07-02 14:07       ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-07-02 14:07 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Hajime Tazaki

Add a simple LKL test application (boot) that starts the kernel and
performs simple tests that minimally exercise the LKL API.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/um/Makefile       |   5 +-
 tools/um/Targets        |   6 +
 tools/um/tests/Build    |   1 +
 tools/um/tests/boot.c   | 393 ++++++++++++++++++++++++++++++++++++++++
 tools/um/tests/boot.sh  |   9 +
 tools/um/tests/run.py   | 172 ++++++++++++++++++
 tools/um/tests/tap13.py | 209 +++++++++++++++++++++
 tools/um/tests/test.c   | 126 +++++++++++++
 tools/um/tests/test.h   |  72 ++++++++
 tools/um/tests/test.sh  | 181 ++++++++++++++++++
 10 files changed, 1173 insertions(+), 1 deletion(-)
 create mode 100644 tools/um/tests/Build
 create mode 100644 tools/um/tests/boot.c
 create mode 100755 tools/um/tests/boot.sh
 create mode 100755 tools/um/tests/run.py
 create mode 100644 tools/um/tests/tap13.py
 create mode 100644 tools/um/tests/test.c
 create mode 100644 tools/um/tests/test.h
 create mode 100644 tools/um/tests/test.sh

diff --git a/tools/um/Makefile b/tools/um/Makefile
index a93e060b1f89..20de2e4c162f 100644
--- a/tools/um/Makefile
+++ b/tools/um/Makefile
@@ -73,7 +73,10 @@ clean:
 	$(call QUIET_CLEAN, liblinux.a)$(RM) $(OUTPUT)/liblinux.a
 	$(call QUIET_CLEAN, $(TARGETS))$(RM) $(TARGETS)
 
+run-tests:
+	./tests/run.py $(tests)
+
 FORCE: ;
-.PHONY: all clean FORCE
+.PHONY: all clean FORCE run-tests
 .IGNORE: clean
 .NOTPARALLEL : lib/linux.o
diff --git a/tools/um/Targets b/tools/um/Targets
index 4e1f0f4d81a3..2bb90381843c 100644
--- a/tools/um/Targets
+++ b/tools/um/Targets
@@ -1,4 +1,10 @@
+ifeq ($(UMMODE),library)
+progs-y += tests/boot
+else
 progs-y += uml/linux
+endif
+
+LDFLAGS_boot-y := -pie
 
 LDLIBS := -lrt -lpthread -lutil
 LDFLAGS_linux-y := -no-pie -Wl,--wrap,malloc -Wl,--wrap,free -Wl,--wrap,calloc
diff --git a/tools/um/tests/Build b/tools/um/tests/Build
new file mode 100644
index 000000000000..26a607f90dc8
--- /dev/null
+++ b/tools/um/tests/Build
@@ -0,0 +1 @@
+boot-y += boot.o test.o
diff --git a/tools/um/tests/boot.c b/tools/um/tests/boot.c
new file mode 100644
index 000000000000..45b9a7276de7
--- /dev/null
+++ b/tools/um/tests/boot.c
@@ -0,0 +1,393 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <unistd.h>
+#include <string.h>
+#include <time.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <lkl.h>
+#include <lkl_host.h>
+
+#include <sys/stat.h>
+#include <fcntl.h>
+#if defined(__FreeBSD__)
+#include <net/if.h>
+#include <sys/ioctl.h>
+#elif __linux
+#include <sys/epoll.h>
+#include <sys/ioctl.h>
+#elif __MINGW32__
+#include <windows.h>
+#endif
+
+#include "test.h"
+
+#ifndef __MINGW32__
+#define sleep_ns 87654321
+int lkl_test_nanosleep(void)
+{
+	struct lkl_timespec ts = {
+		.tv_sec = 0,
+		.tv_nsec = sleep_ns,
+	};
+	struct timespec start, stop;
+	long delta;
+	long ret;
+
+	clock_gettime(CLOCK_MONOTONIC, &start);
+	ret = lkl_sys_nanosleep((struct __lkl__kernel_timespec *)&ts, NULL);
+	clock_gettime(CLOCK_MONOTONIC, &stop);
+
+	delta = 1e9*(stop.tv_sec - start.tv_sec) +
+		(stop.tv_nsec - start.tv_nsec);
+
+	lkl_test_logf("sleep %ld, expected sleep %d\n", delta, sleep_ns);
+
+	if (ret == 0 && delta > sleep_ns * 0.9)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+#endif
+
+LKL_TEST_CALL(getpid, lkl_sys_getpid, 1)
+
+void check_latency(long (*f)(void), long *min, long *max, long *avg)
+{
+	int i;
+	struct timespec start, stop;
+	unsigned long long sum = 0;
+	static const int count = 20;
+	long delta;
+
+	*min = 1000000000;
+	*max = -1;
+
+	for (i = 0; i < count; i++) {
+		clock_gettime(CLOCK_MONOTONIC, &start);
+		f();
+		clock_gettime(CLOCK_MONOTONIC, &stop);
+
+		delta = 1e9*(stop.tv_sec - start.tv_sec) +
+			(stop.tv_nsec - start.tv_nsec);
+
+		if (*min > delta)
+			*min = delta;
+		if (*max < delta)
+			*max = delta;
+		sum += delta;
+	}
+	*avg = sum / count;
+}
+
+static long native_getpid(void)
+{
+#ifdef __MINGW32__
+	GetCurrentProcessId();
+#else
+	getpid();
+#endif
+	return 0;
+}
+
+int lkl_test_syscall_latency(void)
+{
+	long min, max, avg;
+
+	lkl_test_logf("avg/min/max: ");
+
+	check_latency(lkl_sys_getpid, &min, &max, &avg);
+
+	lkl_test_logf("lkl:%ld/%ld/%ld ", avg, min, max);
+
+	check_latency(native_getpid, &min, &max, &avg);
+
+	lkl_test_logf("native:%ld/%ld/%ld\n", avg, min, max);
+
+	return TEST_SUCCESS;
+}
+
+#define access_rights 0721
+
+LKL_TEST_CALL(creat, lkl_sys_creat, 3, "/file", access_rights)
+LKL_TEST_CALL(close, lkl_sys_close, 0, 0);
+LKL_TEST_CALL(failopen, lkl_sys_open, -LKL_ENOENT, "/file2", 0, 0);
+LKL_TEST_CALL(umask, lkl_sys_umask, 022,  0777);
+LKL_TEST_CALL(umask2, lkl_sys_umask, 0777, 0);
+LKL_TEST_CALL(open, lkl_sys_open, 0, "/file", LKL_O_RDWR, 0);
+static const char wrbuf[] = "test";
+LKL_TEST_CALL(write, lkl_sys_write, sizeof(wrbuf), 0, wrbuf, sizeof(wrbuf));
+LKL_TEST_CALL(lseek_cur, lkl_sys_lseek, sizeof(wrbuf), 0, 0, LKL_SEEK_CUR);
+LKL_TEST_CALL(lseek_end, lkl_sys_lseek, sizeof(wrbuf), 0, 0, LKL_SEEK_END);
+LKL_TEST_CALL(lseek_set, lkl_sys_lseek, 0, 0, 0, LKL_SEEK_SET);
+
+int lkl_test_read(void)
+{
+	char buf[10] = { 0, };
+	long ret;
+
+	ret = lkl_sys_read(0, buf, sizeof(buf));
+
+	lkl_test_logf("lkl_sys_read=%ld buf=%s\n", ret, buf);
+
+	if (ret == sizeof(wrbuf) && !strcmp(wrbuf, buf))
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+int lkl_test_fstat(void)
+{
+	struct lkl_stat stat;
+	long ret;
+
+	ret = lkl_sys_fstat(0, (void *)&stat);
+
+	lkl_test_logf("lkl_sys_fstat=%ld mode=%o size=%ld\n", ret, stat.st_mode,
+		      stat.st_size);
+
+	if (ret == 0 && stat.st_size == sizeof(wrbuf) &&
+	    stat.st_mode == (access_rights | LKL_S_IFREG))
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+LKL_TEST_CALL(mkdir, lkl_sys_mkdir, 0, "/proc", access_rights)
+
+int lkl_test_stat(void)
+{
+	struct lkl_stat stat;
+	long ret;
+
+	ret = lkl_sys_stat("/proc", (void *)&stat);
+
+	lkl_test_logf("lkl_sys_stat(\"/proc\")=%ld mode=%o\n", ret,
+		      stat.st_mode);
+
+	if (ret == 0 && stat.st_mode == (access_rights | LKL_S_IFDIR))
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+static int lkl_test_pipe2(void)
+{
+	int pipe_fds[2];
+	int READ_IDX = 0, WRITE_IDX = 1;
+	static const char msg[] = "Hello world!";
+	char str[20];
+	int msg_len_bytes = strlen(msg) + 1;
+	int cmp_res;
+	long ret;
+
+	ret = lkl_sys_pipe2(pipe_fds, LKL_O_NONBLOCK);
+	if (ret) {
+		lkl_test_logf("pipe2: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_write(pipe_fds[WRITE_IDX], msg, msg_len_bytes);
+	if (ret != msg_len_bytes) {
+		if (ret < 0)
+			lkl_test_logf("write error: %s\n", lkl_strerror(ret));
+		else
+			lkl_test_logf("short write: %ld\n", ret);
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_read(pipe_fds[READ_IDX], str, msg_len_bytes);
+	if (ret != msg_len_bytes) {
+		if (ret < 0)
+			lkl_test_logf("read error: %s\n", lkl_strerror(ret));
+		else
+			lkl_test_logf("short read: %ld\n", ret);
+		return TEST_FAILURE;
+	}
+
+	cmp_res = memcmp(msg, str, msg_len_bytes);
+	if (cmp_res) {
+		lkl_test_logf("memcmp failed: %d\n", cmp_res);
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_close(pipe_fds[0]);
+	if (ret) {
+		lkl_test_logf("close error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_close(pipe_fds[1]);
+	if (ret) {
+		lkl_test_logf("close error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	return TEST_SUCCESS;
+}
+
+static int lkl_test_epoll(void)
+{
+	int epoll_fd, pipe_fds[2];
+	int READ_IDX = 0, WRITE_IDX = 1;
+	struct lkl_epoll_event wait_on, read_result;
+	static const char msg[] = "Hello world!";
+	long ret;
+
+	memset(&wait_on, 0, sizeof(wait_on));
+	memset(&read_result, 0, sizeof(read_result));
+
+	ret = lkl_sys_pipe2(pipe_fds, LKL_O_NONBLOCK);
+	if (ret) {
+		lkl_test_logf("pipe2 error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	epoll_fd = lkl_sys_epoll_create(1);
+	if (epoll_fd < 0) {
+		lkl_test_logf("epoll_create error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	wait_on.events = LKL_POLLIN | LKL_POLLOUT;
+	wait_on.data = pipe_fds[READ_IDX];
+
+	ret = lkl_sys_epoll_ctl(epoll_fd, LKL_EPOLL_CTL_ADD, pipe_fds[READ_IDX],
+				&wait_on);
+	if (ret < 0) {
+		lkl_test_logf("epoll_ctl error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	/* Shouldn't be ready before we have written something */
+	ret = lkl_sys_epoll_wait(epoll_fd, &read_result, 1, 0);
+	if (ret != 0) {
+		if (ret < 0)
+			lkl_test_logf("epoll_wait error: %s\n",
+				      lkl_strerror(ret));
+		else
+			lkl_test_logf("epoll_wait: bad event: 0x%lx\n", ret);
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_write(pipe_fds[WRITE_IDX], msg, strlen(msg) + 1);
+	if (ret < 0) {
+		lkl_test_logf("write error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	/* We expect exactly 1 fd to be ready immediately */
+	ret = lkl_sys_epoll_wait(epoll_fd, &read_result, 1, 0);
+	if (ret != 1) {
+		if (ret < 0)
+			lkl_test_logf("epoll_wait error: %s\n",
+				      lkl_strerror(ret));
+		else
+			lkl_test_logf("epoll_wait: bad ev no %ld\n", ret);
+		return TEST_FAILURE;
+	}
+
+	/* Already tested reading from pipe2 so no need to do it
+	 * here
+	 */
+
+	return TEST_SUCCESS;
+}
+
+LKL_TEST_CALL(chdir_proc, lkl_sys_chdir, 0, "proc");
+
+static int dir_fd;
+
+static int lkl_test_open_cwd(void)
+{
+	dir_fd = lkl_sys_open(".", LKL_O_RDONLY | LKL_O_DIRECTORY, 0);
+	if (dir_fd < 0) {
+		lkl_test_logf("failed to open current directory: %s\n",
+			      lkl_strerror(dir_fd));
+		return TEST_FAILURE;
+	}
+
+	return TEST_SUCCESS;
+}
+
+/* column where to insert a line break for the list file tests below. */
+#define COL_LINE_BREAK 70
+
+static int lkl_test_getdents64(void)
+{
+	long ret;
+	char buf[1024], *pos;
+	struct lkl_linux_dirent64 *de;
+	int wr;
+
+	de = (struct lkl_linux_dirent64 *)buf;
+	ret = lkl_sys_getdents64(dir_fd, de, sizeof(buf));
+
+	wr = lkl_test_logf("%d ", dir_fd);
+
+	if (ret < 0)
+		return TEST_FAILURE;
+
+	for (pos = buf; pos - buf < ret; pos += de->d_reclen) {
+		de = (struct lkl_linux_dirent64 *)pos;
+
+		wr += lkl_test_logf("%s ", de->d_name);
+		if (wr >= COL_LINE_BREAK) {
+			lkl_test_logf("\n");
+			wr = 0;
+		}
+	}
+
+	return TEST_SUCCESS;
+}
+
+LKL_TEST_CALL(close_dir_fd, lkl_sys_close, 0, dir_fd);
+LKL_TEST_CALL(chdir_root, lkl_sys_chdir, 0, "/");
+LKL_TEST_CALL(mount_fs_proc, lkl_sys_mount, 0, "none", "/proc", "proc", 0,
+	      NULL);
+LKL_TEST_CALL(umount_fs_proc, lkl_sys_umount, 0, "/proc", 0);
+
+LKL_TEST_CALL(start_kernel, lkl_start_kernel, 0, &lkl_host_ops,
+	     "mem=16M loglevel=8");
+LKL_TEST_CALL(stop_kernel, lkl_sys_halt, 0);
+
+struct lkl_test tests[] = {
+	LKL_TEST(start_kernel),
+	LKL_TEST(getpid),
+	LKL_TEST(syscall_latency),
+	LKL_TEST(umask),
+	LKL_TEST(umask2),
+	LKL_TEST(creat),
+	LKL_TEST(close),
+	LKL_TEST(failopen),
+	LKL_TEST(open),
+	LKL_TEST(write),
+	LKL_TEST(lseek_cur),
+	LKL_TEST(lseek_end),
+	LKL_TEST(lseek_set),
+	LKL_TEST(read),
+	LKL_TEST(fstat),
+	LKL_TEST(mkdir),
+	LKL_TEST(stat),
+#ifndef __MINGW32__
+	LKL_TEST(nanosleep),
+#endif
+	LKL_TEST(pipe2),
+	LKL_TEST(epoll),
+	LKL_TEST(mount_fs_proc),
+	LKL_TEST(chdir_proc),
+	LKL_TEST(open_cwd),
+	LKL_TEST(getdents64),
+	LKL_TEST(close_dir_fd),
+	LKL_TEST(chdir_root),
+	LKL_TEST(umount_fs_proc),
+	LKL_TEST(stop_kernel),
+};
+
+int main(int argc, const char **argv)
+{
+	lkl_host_ops.print = lkl_test_log;
+
+	return lkl_test_run(tests, sizeof(tests)/sizeof(struct lkl_test),
+			    "boot");
+}
diff --git a/tools/um/tests/boot.sh b/tools/um/tests/boot.sh
new file mode 100755
index 000000000000..d985c04b0ac1
--- /dev/null
+++ b/tools/um/tests/boot.sh
@@ -0,0 +1,9 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+source $script_dir/test.sh
+
+lkl_test_plan 1 "boot"
+lkl_test_run 1
+lkl_test_exec $script_dir/boot
diff --git a/tools/um/tests/run.py b/tools/um/tests/run.py
new file mode 100755
index 000000000000..c96ede90b6ad
--- /dev/null
+++ b/tools/um/tests/run.py
@@ -0,0 +1,172 @@
+#!/usr/bin/env python
+# SPDX-License-Identifier: GPL-2.0
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 2 of the License
+#
+# Author: Octavian Purdila <tavi@cs.pub.ro>
+#
+
+from __future__ import print_function
+
+import argparse
+import os
+import subprocess
+import sys
+import tap13
+import xml.etree.ElementTree as ET
+
+from junit_xml import TestSuite, TestCase
+
+
+class Reporter(tap13.Reporter):
+    def start(self, obj):
+        if type(obj) is tap13.Test:
+            if obj.result == "*":
+                end='\r'
+            else:
+                end='\n'
+            print("  TEST       %-8s %.50s" %
+                  (obj.result, obj.description + " " + obj.comment), end=end)
+
+        elif type(obj) is tap13.Suite:
+            if obj.tests_planned == 0:
+                status = "skip"
+            else:
+                status = ""
+            print("  SUITE      %-8s %s" % (status, obj.name))
+
+    def end(self, obj):
+        if type(obj) is tap13.Test:
+            if obj.result != "ok":
+                try:
+                    print(obj.yaml["log"], end='')
+                except:
+                    None
+
+
+mydir=os.path.dirname(os.path.realpath(__file__))
+
+tests = [
+    'boot.sh',
+]
+
+parser = argparse.ArgumentParser(description='LKL test runner')
+parser.add_argument('tests', nargs='?', action='append',
+                    help='tests to run %s' % tests)
+parser.add_argument('--junit-dir',
+                    help='directory where to store the juni suites')
+parser.add_argument('--gdb', action='store_true', default=False,
+                    help='run simple tests under gdb; implies --pass-through')
+parser.add_argument('--pass-through', action='store_true',  default=False,
+                    help='run the test without interpeting the test output')
+parser.add_argument('--valgrind', action='store_true', default=False,
+                    help='run simple tests under valgrind')
+
+args = parser.parse_args()
+if args.tests == [None]:
+    args.tests = tests
+
+if args.gdb:
+    args.pass_through=True
+    os.environ['GDB']="yes"
+
+if args.valgrind:
+    os.environ['VALGRIND']="yes"
+
+tap = tap13.Parser(Reporter())
+
+os.environ['PATH'] += ":" + mydir
+
+exit_code = 0
+
+for t in args.tests:
+    if not t:
+        continue
+    if args.pass_through:
+        print(t)
+        if subprocess.call(t, shell=True) != 0:
+            exit_code = 1
+    else:
+        p = subprocess.Popen(t, shell=True, stdout=subprocess.PIPE)
+        tap.parse(p.stdout)
+
+if args.pass_through:
+    sys.exit(exit_code)
+
+suites_count = 0
+tests_total = 0
+tests_not_ok = 0
+tests_ok = 0
+tests_skip = 0
+val_errs = 0
+val_fails = 0
+val_skips = 0
+
+for s in tap.run.suites:
+
+    junit_tests = []
+    suites_count += 1
+
+    for t in s.tests:
+        try:
+            secs = t.yaml["time_us"] / 1000000.0
+        except:
+            secs = 0
+        try:
+            log = t.yaml['log']
+        except:
+            log = ""
+
+        jt = TestCase(t.description, elapsed_sec=secs, stdout=log)
+        if t.result == 'skip':
+            jt.add_skipped_info(output=log)
+        elif t.result == 'not ok':
+            jt.add_error_info(output=log)
+
+        junit_tests.append(jt)
+
+        tests_total += 1
+        if t.result == "ok":
+            tests_ok += 1
+        elif t.result == "not ok":
+            tests_not_ok += 1
+            exit_code = 1
+        elif t.result == "skip":
+            tests_skip += 1
+
+    if args.junit_dir:
+        js = TestSuite(s.name, junit_tests)
+        with open(os.path.join(args.junit_dir, os.path.basename(s.name) + '.xml'), 'w') as f:
+            js.to_file(f, [js])
+
+        if os.getenv('VALGRIND') is not None:
+            val_xml = 'valgrind-%s.xml' % os.path.basename(s.name).replace(' ','-')
+            # skipped tests don't generate xml file
+            if os.path.exists(val_xml) is False:
+                continue
+
+            cmd = 'mv %s %s' % (val_xml, args.junit_dir)
+            subprocess.call(cmd, shell=True, )
+
+            cmd = mydir + '/valgrind2xunit.py ' + val_xml
+            subprocess.call(cmd, shell=True, cwd=args.junit_dir)
+
+            # count valgrind results
+            doc = ET.parse(os.path.join(args.junit_dir, 'valgrind-%s_xunit.xml' \
+                                        % (os.path.basename(s.name).replace(' ','-'))))
+            ts = doc.getroot()
+            val_errs += int(ts.get('errors'))
+            val_fails += int(ts.get('failures'))
+            val_skips += int(ts.get('skip'))
+
+print("Summary: %d suites run, %d tests, %d ok, %d not ok, %d skipped" %
+      (suites_count, tests_total, tests_ok, tests_not_ok, tests_skip))
+
+if os.getenv('VALGRIND') is not None:
+    print(" valgrind (memcheck): %d failures, %d skipped" % (val_fails, val_skips))
+    if val_errs or val_fails:
+        exit_code = 1
+
+sys.exit(exit_code)
diff --git a/tools/um/tests/tap13.py b/tools/um/tests/tap13.py
new file mode 100644
index 000000000000..65c73cda7ca1
--- /dev/null
+++ b/tools/um/tests/tap13.py
@@ -0,0 +1,209 @@
+#!/usr/bin/env python
+# SPDX-License-Identifier: GPL-2.0
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 2 of the License
+#
+# Author: Octavian Purdila <tavi@cs.pub.ro>
+#
+# Based on TAP13:
+#
+# Copyright 2013, Red Hat, Inc.
+# Author: Josef Skladanka <jskladan@redhat.com>
+#
+from __future__ import print_function
+
+import re
+import sys
+import yamlish
+
+
+class Reporter(object):
+
+    def start(self, obj):
+        None
+
+    def end(self, obj):
+        None
+
+
+class Test(object):
+    def __init__(self, reporter, result, id, description=None, directive=None,
+                 comment=None):
+        self.reporter = reporter
+        self.result = result
+        if directive:
+            self.result = directive.lower()
+        if id:
+            self.id = int(id)
+        else:
+            self.id = None
+        if description:
+            self.description = description
+        else:
+            self.description = ""
+        if comment:
+            self.comment = "# " + comment
+        else:
+            self.comment = ""
+        self.yaml = None
+        self._yaml_buffer = None
+        self.diagnostics = []
+
+        self.reporter.start(self)
+
+    def end(self):
+        if not self.yaml:
+            self.yaml = yamlish.load(self._yaml_buffer)
+            self.reporter.end(self)
+
+
+class Suite(object):
+    def __init__(self, reporter, start, end, explanation):
+        self.reporter = reporter
+        self.tests = []
+        self.name = explanation
+        self.tests_planned = int(end)
+
+        self.__tests_counter = 0
+        self.__tests_base = 0
+
+        self.reporter.start(self)
+
+    def newTest(self, args):
+        try:
+            self.tests[-1].end()
+        except IndexError:
+            None
+
+        if 'id' not in args or not args['id']:
+            args['id'] = self.__tests_counter
+        else:
+            args['id'] = int(args['id']) + self.__tests_base
+
+        if args['id'] < self.__tests_counter:
+            print("error: bad test id %d, fixing it" % (args['id']))
+            args['id'] = self.__tests_counter
+        # according to TAP13 specs, missing tests must be handled as 'not ok'
+        # here we add the missing tests in sequence
+        while args['id'] > (self.__tests_counter + 1):
+            comment = 'test %d not present' % self.__tests_counter
+            self.tests.append(Test(self.reporter, 'not ok',
+                                   self.__tests_counter, comment=comment))
+            self.__tests_counter += 1
+
+        if args['id'] == self.__tests_counter:
+            if args['directive']:
+                self.test().result = args['directive'].lower()
+            else:
+                self.test().result = args['result']
+            self.reporter.start(self.test())
+        else:
+            self.tests.append(Test(self.reporter, **args))
+            self.__tests_counter += 1
+
+    def test(self):
+        return self.tests[-1]
+
+    def end(self, name, planned):
+        if name == self.name:
+            self.tests_planned += int(planned)
+            self.__tests_base = self.__tests_counter
+            return False
+        try:
+            self.test().end()
+        except IndexError:
+            None
+        if len(self.tests) != self.tests_planned:
+            for i in range(len(self.tests), self.tests_planned):
+                self.tests.append(Test(self.reporter, 'not ok', i+1,
+                                       comment='test not present'))
+        return True
+
+
+class Run(object):
+
+    def __init__(self, reporter):
+        self.reporter = reporter
+        self.suites = []
+
+    def suite(self):
+        return self.suites[-1]
+
+    def test(self):
+        return self.suites[-1].tests[-1]
+
+    def newSuite(self, args):
+        new = False
+        try:
+            if self.suite().end(args['explanation'], args['end']):
+                new = True
+        except IndexError:
+            new = True
+        if new:
+            self.suites.append(Suite(self.reporter, **args))
+
+    def newTest(self, args):
+        self.suite().newTest(args)
+
+
+class Parser(object):
+    RE_PLAN = re.compile(r"^\s*(?P<start>\d+)\.\.(?P<end>\d+)\s*(#\s*(?P<explanation>.*))?\s*$")
+    RE_TEST_LINE = re.compile(r"^\s*(?P<result>(not\s+)?ok|[*]+)\s*(?P<id>\d+)?\s*(?P<description>[^#]+)?\s*(#\s*(?P<directive>TODO|SKIP)?\s*(?P<comment>.+)?)?\s*$",  re.IGNORECASE)
+    RE_EXPLANATION = re.compile(r"^\s*#\s*(?P<explanation>.+)?\s*$")
+    RE_YAMLISH_START = re.compile(r"^\s*---.*$")
+    RE_YAMLISH_END = re.compile(r"^\s*\.\.\.\s*$")
+
+    def __init__(self, reporter):
+        self.seek_test = False
+        self.in_test = False
+        self.in_yaml = False
+        self.run = Run(reporter)
+
+    def parse(self, source):
+        # to avoid input buffering
+        while True:
+            line = source.readline()
+            if not line:
+                break
+
+            if self.in_yaml:
+                if Parser.RE_YAMLISH_END.match(line):
+                    self.run.test()._yaml_buffer.append(line.strip())
+                    self.in_yaml = False
+                else:
+                    self.run.test()._yaml_buffer.append(line.rstrip())
+                continue
+
+            line = line.strip()
+
+            if self.in_test:
+                if Parser.RE_EXPLANATION.match(line):
+                    self.run.test().diagnostics.append(line)
+                    continue
+                if Parser.RE_YAMLISH_START.match(line):
+                    self.run.test()._yaml_buffer = [line.strip()]
+                    self.in_yaml = True
+                    continue
+
+            m = Parser.RE_PLAN.match(line)
+            if m:
+                self.seek_test = True
+                args = m.groupdict()
+                self.run.newSuite(args)
+                continue
+
+            if self.seek_test:
+                m = Parser.RE_TEST_LINE.match(line)
+                if m:
+                    args = m.groupdict()
+                    self.run.newTest(args)
+                    self.in_test = True
+                    continue
+
+            print(line)
+        try:
+            self.run.suite().end(None, 0)
+        except IndexError:
+            None
diff --git a/tools/um/tests/test.c b/tools/um/tests/test.c
new file mode 100644
index 000000000000..37ea21901f99
--- /dev/null
+++ b/tools/um/tests/test.c
@@ -0,0 +1,126 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <stdarg.h>
+#include <time.h>
+
+#include "test.h"
+
+/* circular log buffer */
+
+static char log_buf[0x10000];
+static char *head = log_buf, *tail = log_buf;
+
+static inline void advance(char **ptr)
+{
+	if ((unsigned int)(*ptr - log_buf) >= sizeof(log_buf))
+		*ptr = log_buf;
+	else
+		*ptr = *ptr + 1;
+}
+
+static void log_char(char c)
+{
+	*tail = c;
+	advance(&tail);
+	if (tail == head)
+		advance(&head);
+}
+
+static void print_log(void)
+{
+	char last;
+
+	printf(" log: |\n");
+	last = '\n';
+	while (head != tail) {
+		if (last == '\n')
+			printf("  ");
+		last = *head;
+		putchar(last);
+		advance(&head);
+	}
+	if (last != '\n')
+		putchar('\n');
+}
+
+int lkl_test_run(const struct lkl_test *tests, int nr, const char *fmt, ...)
+{
+	int i, ret, status = TEST_SUCCESS;
+	clock_t start, stop;
+	char name[1024];
+	va_list args;
+
+	va_start(args, fmt);
+	vsnprintf(name, sizeof(name), fmt, args);
+	va_end(args);
+
+	printf("1..%d # %s\n", nr, name);
+	for (i = 1; i <= nr; i++) {
+		const struct lkl_test *t = &tests[i-1];
+		unsigned long delta_us;
+
+		printf("* %d %s\n", i, t->name);
+		fflush(stdout);
+
+		start = clock();
+
+		ret = t->fn(t->arg1, t->arg2, t->arg3);
+
+		stop = clock();
+
+		switch (ret) {
+		case TEST_SUCCESS:
+			printf("ok %d %s\n", i, t->name);
+			break;
+		case TEST_SKIP:
+			printf("ok %d %s # SKIP\n", i, t->name);
+			break;
+		case TEST_BAILOUT:
+			status = TEST_BAILOUT;
+			/* fall through; */
+		case TEST_FAILURE:
+		default:
+			if (status != TEST_BAILOUT)
+				status = TEST_FAILURE;
+			printf("not ok %d %s\n", i, t->name);
+		}
+
+		printf(" ---\n");
+		delta_us = (stop - start) * 1000000 / CLOCKS_PER_SEC;
+		printf(" time_us: %ld\n", delta_us);
+		print_log();
+		printf(" ...\n");
+
+		if (status == TEST_BAILOUT) {
+			printf("Bail out!\n");
+			return TEST_FAILURE;
+		}
+
+		fflush(stdout);
+	}
+
+	return status;
+}
+
+
+void lkl_test_log(const char *str, int len)
+{
+	while (len--)
+		log_char(*(str++));
+}
+
+int lkl_test_logf(const char *fmt, ...)
+{
+	char tmp[1024], *c;
+	va_list args;
+	unsigned int n;
+
+	va_start(args, fmt);
+	n = vsnprintf(tmp, sizeof(tmp), fmt, args);
+	va_end(args);
+
+	for (c = tmp; *c != 0; c++)
+		log_char(*c);
+
+	return n > sizeof(tmp) ? sizeof(tmp) : n;
+}
diff --git a/tools/um/tests/test.h b/tools/um/tests/test.h
new file mode 100644
index 000000000000..f63ad6d419cb
--- /dev/null
+++ b/tools/um/tests/test.h
@@ -0,0 +1,72 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_TEST_H
+#define _LKL_TEST_H
+
+#define TEST_SUCCESS	0
+#define TEST_FAILURE	1
+#define TEST_SKIP	2
+#define TEST_TODO	3
+#define TEST_BAILOUT	4
+
+struct lkl_test {
+	const char *name;
+	int (*fn)();
+	void *arg1, *arg2, *arg3;
+};
+
+/**
+ * Simple wrapper to initialize a test entry.
+ * @name - test name, it assume test function is named test_@name
+ * @vargs - arguments to be passed to the function
+ */
+#define LKL_TEST(name, ...) { #name, lkl_test_##name, __VA_ARGS__ }
+
+/**
+ * lkl_test_run - run a test suite
+ *
+ * @tests - the list of tests to run
+ * @nr - number of tests
+ * @fmt - format string to be used for suite name
+ */
+int lkl_test_run(const struct lkl_test *tests, int nr, const char *fmt, ...);
+
+/**
+ * lkl_test_log - store a string in the test log buffer
+ * @str - the string to log (can be non-NULL terminated)
+ * @len - the string length
+ */
+void lkl_test_log(const char *str, int len);
+
+/**
+ * lkl_test_logf - printf like function to store into the test log buffer
+ * @fmt - printf format string
+ * @vargs - arguments to the format string
+ */
+int lkl_test_logf(const char *fmt, ...) __attribute__((format(printf, 1, 2)));
+
+/**
+ * LKL_TEST_CALL - create a test function as for a LKL call
+ *
+ * The test function will be named lkl_test_@name and will return
+ * TEST_SUCCESS if the called functions returns @expect. Otherwise
+ * will return TEST_FAILUIRE.
+ *
+ * @name - test name; must be unique because it is part of the the
+ * test function; the test function will be named
+ * @call - function to call
+ * @expect - expected return value for success
+ * @args - arguments to pass to the LKL call
+ */
+#define LKL_TEST_CALL(name, call, expect, ...)				\
+	static int lkl_test_##name(void)				\
+	{								\
+		long ret;						\
+									\
+		ret = call(__VA_ARGS__);				\
+		lkl_test_logf("%s(%s) = %ld %s\n", #call, #__VA_ARGS__, \
+			ret, ret < 0 ? lkl_strerror(ret) : "");		\
+		return (ret == expect) ? TEST_SUCCESS : TEST_FAILURE;	\
+	}
+
+
+#endif /* _LKL_TEST_H */
diff --git a/tools/um/tests/test.sh b/tools/um/tests/test.sh
new file mode 100644
index 000000000000..31b7de5c06f7
--- /dev/null
+++ b/tools/um/tests/test.sh
@@ -0,0 +1,181 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+basedir=$(cd $script_dir/..; pwd)
+base_objdir=$(cd ${OUTPUT}/; pwd)
+
+TEST_SUCCESS=0
+TEST_FAILURE=1
+TEST_SKIP=113
+TEST_TODO=114
+TEST_BAILOUT=115
+
+print_log()
+{
+    echo " log: |"
+    while read line; do
+        echo "  $line"
+    done < $1
+}
+
+export_vars()
+{
+    if [ -z "$var_file" ]; then
+        return
+    fi
+
+    for i in $@; do
+        echo "$i=${!i}" >> $var_file
+    done
+}
+
+lkl_test_run()
+{
+    log_file=$(mktemp)
+    export var_file=$(mktemp)
+
+    tid=$1 && shift && tname=$@
+
+    echo "* $tid $tname"
+
+    start=$(date '+%s%9N')
+    # run in a separate shell to avoid -e terminating us
+    $@ 2>&1 | strings >$log_file
+    exit=${PIPESTATUS[0]}
+    stop=$(date '+%s%9N')
+
+    case $exit in
+    $TEST_SUCCESS)
+        echo "ok $tid $tname"
+        ;;
+    $TEST_SKIP)
+        echo "ok $tid $tname # SKIP"
+        ;;
+    $TEST_BAILOUT)
+        echo "not ok $tid $tname"
+        echo "Bail out!"
+        ;;
+    $TEST_FAILURE|*)
+        echo "not ok $tid $tname"
+        ;;
+    esac
+
+    delta=$(((stop-start)/1000))
+
+    echo " ---"
+    echo " time_us: $delta"
+    print_log $log_file
+    echo -e " ..."
+
+    rm $log_file
+    . $var_file
+    rm $var_file
+
+    return $exit
+}
+
+lkl_test_plan()
+{
+    echo "1..$1 # $2"
+    export suite_name="${2// /\-}"
+}
+
+lkl_test_exec()
+{
+    local SUDO=""
+    local WRAPPER=""
+
+    if [ "$1" = "sudo" ]; then
+        SUDO=sudo
+        shift
+    fi
+
+    local file=$1
+    shift
+
+    if [ -n "$LKL_HOST_CONFIG_NT" ]; then
+        file=$file.exe
+    fi
+
+    file=${OUTPUT}/tests/$(basename $file)
+
+    if file $file | grep ARM; then
+        WRAPPER="qemu-arm-static"
+    elif file $file | grep "FreeBSD" ; then
+        ssh_copy "$file" $BSD_WDIR
+        if [ -n "$SUDO" ]; then
+            SUDO=""
+        fi
+        WRAPPER="$MYSSH $SU"
+        # ssh will mess up with pipes ('|') so, escape the pipe char.
+        args="${@//\|/\\\|}"
+        set - $BSD_WDIR/$(basename $file) $args
+        file=""
+    elif [ -n "$GDB" ]; then
+        WRAPPER="gdb"
+        args="$@"
+        set - -ex "run $args" -ex quit $file
+        file=""
+    elif [ -n "$VALGRIND" ]; then
+        WRAPPER="valgrind --suppressions=$script_dir/valgrind.supp \
+                  --leak-check=full --show-leak-kinds=all --xml=yes \
+                  --xml-file=valgrind-$suite_name.xml"
+    fi
+
+    $SUDO $WRAPPER $file "$@"
+}
+
+lkl_test_cmd()
+{
+    local WRAPPER=""
+
+    if [ -z "$QUIET" ]; then
+        SHOPTS="-x"
+    fi
+
+    if [ -n "$LKL_HOST_CONFIG_BSD" ]; then
+        WRAPPER="$MYSSH $SU"
+    fi
+
+    echo "$@" | $WRAPPER sh $SHOPTS
+}
+
+# XXX: $MYSSH and $MYSCP are defined in a circleci docker image.
+# see the definitions in lkl/lkl-docker:circleci/freebsd11/Dockerfile
+ssh_push()
+{
+    while [ -n "$1" ]; do
+        if [[ "$1" = *.sh ]]; then
+            type="script"
+        else
+            type="file"
+        fi
+
+        dir=$(dirname $1)
+        $MYSSH mkdir -p $BSD_WDIR/$dir
+
+        $MYSCP -P 7722 -r $basedir/$1 root@localhost:$BSD_WDIR/$dir
+        if [ "$type" = "script" ]; then
+            $MYSSH chmod a+x $BSD_WDIR/$1
+        fi
+
+        shift
+    done
+}
+
+ssh_copy()
+{
+    $MYSCP -P 7722 -r $1 root@localhost:$2
+}
+
+lkl_test_bsd_cleanup()
+{
+    $MYSSH rm -rf $BSD_WDIR
+}
+
+if [ -n "$LKL_HOST_CONFIG_BSD" ]; then
+    trap lkl_test_bsd_cleanup EXIT
+    export BSD_WDIR=/root/lkl
+    $MYSSH mkdir -p $BSD_WDIR
+fi
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v5 20/21] um: host: add test programs
@ 2020-07-02 14:07       ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-07-02 14:07 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, linux-kernel-library, linux-arch,
	Hajime Tazaki, Akira Moroo

Add a simple LKL test application (boot) that starts the kernel and
performs simple tests that minimally exercise the LKL API.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/um/Makefile       |   5 +-
 tools/um/Targets        |   6 +
 tools/um/tests/Build    |   1 +
 tools/um/tests/boot.c   | 393 ++++++++++++++++++++++++++++++++++++++++
 tools/um/tests/boot.sh  |   9 +
 tools/um/tests/run.py   | 172 ++++++++++++++++++
 tools/um/tests/tap13.py | 209 +++++++++++++++++++++
 tools/um/tests/test.c   | 126 +++++++++++++
 tools/um/tests/test.h   |  72 ++++++++
 tools/um/tests/test.sh  | 181 ++++++++++++++++++
 10 files changed, 1173 insertions(+), 1 deletion(-)
 create mode 100644 tools/um/tests/Build
 create mode 100644 tools/um/tests/boot.c
 create mode 100755 tools/um/tests/boot.sh
 create mode 100755 tools/um/tests/run.py
 create mode 100644 tools/um/tests/tap13.py
 create mode 100644 tools/um/tests/test.c
 create mode 100644 tools/um/tests/test.h
 create mode 100644 tools/um/tests/test.sh

diff --git a/tools/um/Makefile b/tools/um/Makefile
index a93e060b1f89..20de2e4c162f 100644
--- a/tools/um/Makefile
+++ b/tools/um/Makefile
@@ -73,7 +73,10 @@ clean:
 	$(call QUIET_CLEAN, liblinux.a)$(RM) $(OUTPUT)/liblinux.a
 	$(call QUIET_CLEAN, $(TARGETS))$(RM) $(TARGETS)
 
+run-tests:
+	./tests/run.py $(tests)
+
 FORCE: ;
-.PHONY: all clean FORCE
+.PHONY: all clean FORCE run-tests
 .IGNORE: clean
 .NOTPARALLEL : lib/linux.o
diff --git a/tools/um/Targets b/tools/um/Targets
index 4e1f0f4d81a3..2bb90381843c 100644
--- a/tools/um/Targets
+++ b/tools/um/Targets
@@ -1,4 +1,10 @@
+ifeq ($(UMMODE),library)
+progs-y += tests/boot
+else
 progs-y += uml/linux
+endif
+
+LDFLAGS_boot-y := -pie
 
 LDLIBS := -lrt -lpthread -lutil
 LDFLAGS_linux-y := -no-pie -Wl,--wrap,malloc -Wl,--wrap,free -Wl,--wrap,calloc
diff --git a/tools/um/tests/Build b/tools/um/tests/Build
new file mode 100644
index 000000000000..26a607f90dc8
--- /dev/null
+++ b/tools/um/tests/Build
@@ -0,0 +1 @@
+boot-y += boot.o test.o
diff --git a/tools/um/tests/boot.c b/tools/um/tests/boot.c
new file mode 100644
index 000000000000..45b9a7276de7
--- /dev/null
+++ b/tools/um/tests/boot.c
@@ -0,0 +1,393 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <unistd.h>
+#include <string.h>
+#include <time.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <lkl.h>
+#include <lkl_host.h>
+
+#include <sys/stat.h>
+#include <fcntl.h>
+#if defined(__FreeBSD__)
+#include <net/if.h>
+#include <sys/ioctl.h>
+#elif __linux
+#include <sys/epoll.h>
+#include <sys/ioctl.h>
+#elif __MINGW32__
+#include <windows.h>
+#endif
+
+#include "test.h"
+
+#ifndef __MINGW32__
+#define sleep_ns 87654321
+int lkl_test_nanosleep(void)
+{
+	struct lkl_timespec ts = {
+		.tv_sec = 0,
+		.tv_nsec = sleep_ns,
+	};
+	struct timespec start, stop;
+	long delta;
+	long ret;
+
+	clock_gettime(CLOCK_MONOTONIC, &start);
+	ret = lkl_sys_nanosleep((struct __lkl__kernel_timespec *)&ts, NULL);
+	clock_gettime(CLOCK_MONOTONIC, &stop);
+
+	delta = 1e9*(stop.tv_sec - start.tv_sec) +
+		(stop.tv_nsec - start.tv_nsec);
+
+	lkl_test_logf("sleep %ld, expected sleep %d\n", delta, sleep_ns);
+
+	if (ret == 0 && delta > sleep_ns * 0.9)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+#endif
+
+LKL_TEST_CALL(getpid, lkl_sys_getpid, 1)
+
+void check_latency(long (*f)(void), long *min, long *max, long *avg)
+{
+	int i;
+	struct timespec start, stop;
+	unsigned long long sum = 0;
+	static const int count = 20;
+	long delta;
+
+	*min = 1000000000;
+	*max = -1;
+
+	for (i = 0; i < count; i++) {
+		clock_gettime(CLOCK_MONOTONIC, &start);
+		f();
+		clock_gettime(CLOCK_MONOTONIC, &stop);
+
+		delta = 1e9*(stop.tv_sec - start.tv_sec) +
+			(stop.tv_nsec - start.tv_nsec);
+
+		if (*min > delta)
+			*min = delta;
+		if (*max < delta)
+			*max = delta;
+		sum += delta;
+	}
+	*avg = sum / count;
+}
+
+static long native_getpid(void)
+{
+#ifdef __MINGW32__
+	GetCurrentProcessId();
+#else
+	getpid();
+#endif
+	return 0;
+}
+
+int lkl_test_syscall_latency(void)
+{
+	long min, max, avg;
+
+	lkl_test_logf("avg/min/max: ");
+
+	check_latency(lkl_sys_getpid, &min, &max, &avg);
+
+	lkl_test_logf("lkl:%ld/%ld/%ld ", avg, min, max);
+
+	check_latency(native_getpid, &min, &max, &avg);
+
+	lkl_test_logf("native:%ld/%ld/%ld\n", avg, min, max);
+
+	return TEST_SUCCESS;
+}
+
+#define access_rights 0721
+
+LKL_TEST_CALL(creat, lkl_sys_creat, 3, "/file", access_rights)
+LKL_TEST_CALL(close, lkl_sys_close, 0, 0);
+LKL_TEST_CALL(failopen, lkl_sys_open, -LKL_ENOENT, "/file2", 0, 0);
+LKL_TEST_CALL(umask, lkl_sys_umask, 022,  0777);
+LKL_TEST_CALL(umask2, lkl_sys_umask, 0777, 0);
+LKL_TEST_CALL(open, lkl_sys_open, 0, "/file", LKL_O_RDWR, 0);
+static const char wrbuf[] = "test";
+LKL_TEST_CALL(write, lkl_sys_write, sizeof(wrbuf), 0, wrbuf, sizeof(wrbuf));
+LKL_TEST_CALL(lseek_cur, lkl_sys_lseek, sizeof(wrbuf), 0, 0, LKL_SEEK_CUR);
+LKL_TEST_CALL(lseek_end, lkl_sys_lseek, sizeof(wrbuf), 0, 0, LKL_SEEK_END);
+LKL_TEST_CALL(lseek_set, lkl_sys_lseek, 0, 0, 0, LKL_SEEK_SET);
+
+int lkl_test_read(void)
+{
+	char buf[10] = { 0, };
+	long ret;
+
+	ret = lkl_sys_read(0, buf, sizeof(buf));
+
+	lkl_test_logf("lkl_sys_read=%ld buf=%s\n", ret, buf);
+
+	if (ret == sizeof(wrbuf) && !strcmp(wrbuf, buf))
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+int lkl_test_fstat(void)
+{
+	struct lkl_stat stat;
+	long ret;
+
+	ret = lkl_sys_fstat(0, (void *)&stat);
+
+	lkl_test_logf("lkl_sys_fstat=%ld mode=%o size=%ld\n", ret, stat.st_mode,
+		      stat.st_size);
+
+	if (ret == 0 && stat.st_size == sizeof(wrbuf) &&
+	    stat.st_mode == (access_rights | LKL_S_IFREG))
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+LKL_TEST_CALL(mkdir, lkl_sys_mkdir, 0, "/proc", access_rights)
+
+int lkl_test_stat(void)
+{
+	struct lkl_stat stat;
+	long ret;
+
+	ret = lkl_sys_stat("/proc", (void *)&stat);
+
+	lkl_test_logf("lkl_sys_stat(\"/proc\")=%ld mode=%o\n", ret,
+		      stat.st_mode);
+
+	if (ret == 0 && stat.st_mode == (access_rights | LKL_S_IFDIR))
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+static int lkl_test_pipe2(void)
+{
+	int pipe_fds[2];
+	int READ_IDX = 0, WRITE_IDX = 1;
+	static const char msg[] = "Hello world!";
+	char str[20];
+	int msg_len_bytes = strlen(msg) + 1;
+	int cmp_res;
+	long ret;
+
+	ret = lkl_sys_pipe2(pipe_fds, LKL_O_NONBLOCK);
+	if (ret) {
+		lkl_test_logf("pipe2: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_write(pipe_fds[WRITE_IDX], msg, msg_len_bytes);
+	if (ret != msg_len_bytes) {
+		if (ret < 0)
+			lkl_test_logf("write error: %s\n", lkl_strerror(ret));
+		else
+			lkl_test_logf("short write: %ld\n", ret);
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_read(pipe_fds[READ_IDX], str, msg_len_bytes);
+	if (ret != msg_len_bytes) {
+		if (ret < 0)
+			lkl_test_logf("read error: %s\n", lkl_strerror(ret));
+		else
+			lkl_test_logf("short read: %ld\n", ret);
+		return TEST_FAILURE;
+	}
+
+	cmp_res = memcmp(msg, str, msg_len_bytes);
+	if (cmp_res) {
+		lkl_test_logf("memcmp failed: %d\n", cmp_res);
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_close(pipe_fds[0]);
+	if (ret) {
+		lkl_test_logf("close error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_close(pipe_fds[1]);
+	if (ret) {
+		lkl_test_logf("close error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	return TEST_SUCCESS;
+}
+
+static int lkl_test_epoll(void)
+{
+	int epoll_fd, pipe_fds[2];
+	int READ_IDX = 0, WRITE_IDX = 1;
+	struct lkl_epoll_event wait_on, read_result;
+	static const char msg[] = "Hello world!";
+	long ret;
+
+	memset(&wait_on, 0, sizeof(wait_on));
+	memset(&read_result, 0, sizeof(read_result));
+
+	ret = lkl_sys_pipe2(pipe_fds, LKL_O_NONBLOCK);
+	if (ret) {
+		lkl_test_logf("pipe2 error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	epoll_fd = lkl_sys_epoll_create(1);
+	if (epoll_fd < 0) {
+		lkl_test_logf("epoll_create error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	wait_on.events = LKL_POLLIN | LKL_POLLOUT;
+	wait_on.data = pipe_fds[READ_IDX];
+
+	ret = lkl_sys_epoll_ctl(epoll_fd, LKL_EPOLL_CTL_ADD, pipe_fds[READ_IDX],
+				&wait_on);
+	if (ret < 0) {
+		lkl_test_logf("epoll_ctl error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	/* Shouldn't be ready before we have written something */
+	ret = lkl_sys_epoll_wait(epoll_fd, &read_result, 1, 0);
+	if (ret != 0) {
+		if (ret < 0)
+			lkl_test_logf("epoll_wait error: %s\n",
+				      lkl_strerror(ret));
+		else
+			lkl_test_logf("epoll_wait: bad event: 0x%lx\n", ret);
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_write(pipe_fds[WRITE_IDX], msg, strlen(msg) + 1);
+	if (ret < 0) {
+		lkl_test_logf("write error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	/* We expect exactly 1 fd to be ready immediately */
+	ret = lkl_sys_epoll_wait(epoll_fd, &read_result, 1, 0);
+	if (ret != 1) {
+		if (ret < 0)
+			lkl_test_logf("epoll_wait error: %s\n",
+				      lkl_strerror(ret));
+		else
+			lkl_test_logf("epoll_wait: bad ev no %ld\n", ret);
+		return TEST_FAILURE;
+	}
+
+	/* Already tested reading from pipe2 so no need to do it
+	 * here
+	 */
+
+	return TEST_SUCCESS;
+}
+
+LKL_TEST_CALL(chdir_proc, lkl_sys_chdir, 0, "proc");
+
+static int dir_fd;
+
+static int lkl_test_open_cwd(void)
+{
+	dir_fd = lkl_sys_open(".", LKL_O_RDONLY | LKL_O_DIRECTORY, 0);
+	if (dir_fd < 0) {
+		lkl_test_logf("failed to open current directory: %s\n",
+			      lkl_strerror(dir_fd));
+		return TEST_FAILURE;
+	}
+
+	return TEST_SUCCESS;
+}
+
+/* column where to insert a line break for the list file tests below. */
+#define COL_LINE_BREAK 70
+
+static int lkl_test_getdents64(void)
+{
+	long ret;
+	char buf[1024], *pos;
+	struct lkl_linux_dirent64 *de;
+	int wr;
+
+	de = (struct lkl_linux_dirent64 *)buf;
+	ret = lkl_sys_getdents64(dir_fd, de, sizeof(buf));
+
+	wr = lkl_test_logf("%d ", dir_fd);
+
+	if (ret < 0)
+		return TEST_FAILURE;
+
+	for (pos = buf; pos - buf < ret; pos += de->d_reclen) {
+		de = (struct lkl_linux_dirent64 *)pos;
+
+		wr += lkl_test_logf("%s ", de->d_name);
+		if (wr >= COL_LINE_BREAK) {
+			lkl_test_logf("\n");
+			wr = 0;
+		}
+	}
+
+	return TEST_SUCCESS;
+}
+
+LKL_TEST_CALL(close_dir_fd, lkl_sys_close, 0, dir_fd);
+LKL_TEST_CALL(chdir_root, lkl_sys_chdir, 0, "/");
+LKL_TEST_CALL(mount_fs_proc, lkl_sys_mount, 0, "none", "/proc", "proc", 0,
+	      NULL);
+LKL_TEST_CALL(umount_fs_proc, lkl_sys_umount, 0, "/proc", 0);
+
+LKL_TEST_CALL(start_kernel, lkl_start_kernel, 0, &lkl_host_ops,
+	     "mem=16M loglevel=8");
+LKL_TEST_CALL(stop_kernel, lkl_sys_halt, 0);
+
+struct lkl_test tests[] = {
+	LKL_TEST(start_kernel),
+	LKL_TEST(getpid),
+	LKL_TEST(syscall_latency),
+	LKL_TEST(umask),
+	LKL_TEST(umask2),
+	LKL_TEST(creat),
+	LKL_TEST(close),
+	LKL_TEST(failopen),
+	LKL_TEST(open),
+	LKL_TEST(write),
+	LKL_TEST(lseek_cur),
+	LKL_TEST(lseek_end),
+	LKL_TEST(lseek_set),
+	LKL_TEST(read),
+	LKL_TEST(fstat),
+	LKL_TEST(mkdir),
+	LKL_TEST(stat),
+#ifndef __MINGW32__
+	LKL_TEST(nanosleep),
+#endif
+	LKL_TEST(pipe2),
+	LKL_TEST(epoll),
+	LKL_TEST(mount_fs_proc),
+	LKL_TEST(chdir_proc),
+	LKL_TEST(open_cwd),
+	LKL_TEST(getdents64),
+	LKL_TEST(close_dir_fd),
+	LKL_TEST(chdir_root),
+	LKL_TEST(umount_fs_proc),
+	LKL_TEST(stop_kernel),
+};
+
+int main(int argc, const char **argv)
+{
+	lkl_host_ops.print = lkl_test_log;
+
+	return lkl_test_run(tests, sizeof(tests)/sizeof(struct lkl_test),
+			    "boot");
+}
diff --git a/tools/um/tests/boot.sh b/tools/um/tests/boot.sh
new file mode 100755
index 000000000000..d985c04b0ac1
--- /dev/null
+++ b/tools/um/tests/boot.sh
@@ -0,0 +1,9 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+source $script_dir/test.sh
+
+lkl_test_plan 1 "boot"
+lkl_test_run 1
+lkl_test_exec $script_dir/boot
diff --git a/tools/um/tests/run.py b/tools/um/tests/run.py
new file mode 100755
index 000000000000..c96ede90b6ad
--- /dev/null
+++ b/tools/um/tests/run.py
@@ -0,0 +1,172 @@
+#!/usr/bin/env python
+# SPDX-License-Identifier: GPL-2.0
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 2 of the License
+#
+# Author: Octavian Purdila <tavi@cs.pub.ro>
+#
+
+from __future__ import print_function
+
+import argparse
+import os
+import subprocess
+import sys
+import tap13
+import xml.etree.ElementTree as ET
+
+from junit_xml import TestSuite, TestCase
+
+
+class Reporter(tap13.Reporter):
+    def start(self, obj):
+        if type(obj) is tap13.Test:
+            if obj.result == "*":
+                end='\r'
+            else:
+                end='\n'
+            print("  TEST       %-8s %.50s" %
+                  (obj.result, obj.description + " " + obj.comment), end=end)
+
+        elif type(obj) is tap13.Suite:
+            if obj.tests_planned == 0:
+                status = "skip"
+            else:
+                status = ""
+            print("  SUITE      %-8s %s" % (status, obj.name))
+
+    def end(self, obj):
+        if type(obj) is tap13.Test:
+            if obj.result != "ok":
+                try:
+                    print(obj.yaml["log"], end='')
+                except:
+                    None
+
+
+mydir=os.path.dirname(os.path.realpath(__file__))
+
+tests = [
+    'boot.sh',
+]
+
+parser = argparse.ArgumentParser(description='LKL test runner')
+parser.add_argument('tests', nargs='?', action='append',
+                    help='tests to run %s' % tests)
+parser.add_argument('--junit-dir',
+                    help='directory where to store the juni suites')
+parser.add_argument('--gdb', action='store_true', default=False,
+                    help='run simple tests under gdb; implies --pass-through')
+parser.add_argument('--pass-through', action='store_true',  default=False,
+                    help='run the test without interpeting the test output')
+parser.add_argument('--valgrind', action='store_true', default=False,
+                    help='run simple tests under valgrind')
+
+args = parser.parse_args()
+if args.tests == [None]:
+    args.tests = tests
+
+if args.gdb:
+    args.pass_through=True
+    os.environ['GDB']="yes"
+
+if args.valgrind:
+    os.environ['VALGRIND']="yes"
+
+tap = tap13.Parser(Reporter())
+
+os.environ['PATH'] += ":" + mydir
+
+exit_code = 0
+
+for t in args.tests:
+    if not t:
+        continue
+    if args.pass_through:
+        print(t)
+        if subprocess.call(t, shell=True) != 0:
+            exit_code = 1
+    else:
+        p = subprocess.Popen(t, shell=True, stdout=subprocess.PIPE)
+        tap.parse(p.stdout)
+
+if args.pass_through:
+    sys.exit(exit_code)
+
+suites_count = 0
+tests_total = 0
+tests_not_ok = 0
+tests_ok = 0
+tests_skip = 0
+val_errs = 0
+val_fails = 0
+val_skips = 0
+
+for s in tap.run.suites:
+
+    junit_tests = []
+    suites_count += 1
+
+    for t in s.tests:
+        try:
+            secs = t.yaml["time_us"] / 1000000.0
+        except:
+            secs = 0
+        try:
+            log = t.yaml['log']
+        except:
+            log = ""
+
+        jt = TestCase(t.description, elapsed_sec=secs, stdout=log)
+        if t.result == 'skip':
+            jt.add_skipped_info(output=log)
+        elif t.result == 'not ok':
+            jt.add_error_info(output=log)
+
+        junit_tests.append(jt)
+
+        tests_total += 1
+        if t.result == "ok":
+            tests_ok += 1
+        elif t.result == "not ok":
+            tests_not_ok += 1
+            exit_code = 1
+        elif t.result == "skip":
+            tests_skip += 1
+
+    if args.junit_dir:
+        js = TestSuite(s.name, junit_tests)
+        with open(os.path.join(args.junit_dir, os.path.basename(s.name) + '.xml'), 'w') as f:
+            js.to_file(f, [js])
+
+        if os.getenv('VALGRIND') is not None:
+            val_xml = 'valgrind-%s.xml' % os.path.basename(s.name).replace(' ','-')
+            # skipped tests don't generate xml file
+            if os.path.exists(val_xml) is False:
+                continue
+
+            cmd = 'mv %s %s' % (val_xml, args.junit_dir)
+            subprocess.call(cmd, shell=True, )
+
+            cmd = mydir + '/valgrind2xunit.py ' + val_xml
+            subprocess.call(cmd, shell=True, cwd=args.junit_dir)
+
+            # count valgrind results
+            doc = ET.parse(os.path.join(args.junit_dir, 'valgrind-%s_xunit.xml' \
+                                        % (os.path.basename(s.name).replace(' ','-'))))
+            ts = doc.getroot()
+            val_errs += int(ts.get('errors'))
+            val_fails += int(ts.get('failures'))
+            val_skips += int(ts.get('skip'))
+
+print("Summary: %d suites run, %d tests, %d ok, %d not ok, %d skipped" %
+      (suites_count, tests_total, tests_ok, tests_not_ok, tests_skip))
+
+if os.getenv('VALGRIND') is not None:
+    print(" valgrind (memcheck): %d failures, %d skipped" % (val_fails, val_skips))
+    if val_errs or val_fails:
+        exit_code = 1
+
+sys.exit(exit_code)
diff --git a/tools/um/tests/tap13.py b/tools/um/tests/tap13.py
new file mode 100644
index 000000000000..65c73cda7ca1
--- /dev/null
+++ b/tools/um/tests/tap13.py
@@ -0,0 +1,209 @@
+#!/usr/bin/env python
+# SPDX-License-Identifier: GPL-2.0
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 2 of the License
+#
+# Author: Octavian Purdila <tavi@cs.pub.ro>
+#
+# Based on TAP13:
+#
+# Copyright 2013, Red Hat, Inc.
+# Author: Josef Skladanka <jskladan@redhat.com>
+#
+from __future__ import print_function
+
+import re
+import sys
+import yamlish
+
+
+class Reporter(object):
+
+    def start(self, obj):
+        None
+
+    def end(self, obj):
+        None
+
+
+class Test(object):
+    def __init__(self, reporter, result, id, description=None, directive=None,
+                 comment=None):
+        self.reporter = reporter
+        self.result = result
+        if directive:
+            self.result = directive.lower()
+        if id:
+            self.id = int(id)
+        else:
+            self.id = None
+        if description:
+            self.description = description
+        else:
+            self.description = ""
+        if comment:
+            self.comment = "# " + comment
+        else:
+            self.comment = ""
+        self.yaml = None
+        self._yaml_buffer = None
+        self.diagnostics = []
+
+        self.reporter.start(self)
+
+    def end(self):
+        if not self.yaml:
+            self.yaml = yamlish.load(self._yaml_buffer)
+            self.reporter.end(self)
+
+
+class Suite(object):
+    def __init__(self, reporter, start, end, explanation):
+        self.reporter = reporter
+        self.tests = []
+        self.name = explanation
+        self.tests_planned = int(end)
+
+        self.__tests_counter = 0
+        self.__tests_base = 0
+
+        self.reporter.start(self)
+
+    def newTest(self, args):
+        try:
+            self.tests[-1].end()
+        except IndexError:
+            None
+
+        if 'id' not in args or not args['id']:
+            args['id'] = self.__tests_counter
+        else:
+            args['id'] = int(args['id']) + self.__tests_base
+
+        if args['id'] < self.__tests_counter:
+            print("error: bad test id %d, fixing it" % (args['id']))
+            args['id'] = self.__tests_counter
+        # according to TAP13 specs, missing tests must be handled as 'not ok'
+        # here we add the missing tests in sequence
+        while args['id'] > (self.__tests_counter + 1):
+            comment = 'test %d not present' % self.__tests_counter
+            self.tests.append(Test(self.reporter, 'not ok',
+                                   self.__tests_counter, comment=comment))
+            self.__tests_counter += 1
+
+        if args['id'] == self.__tests_counter:
+            if args['directive']:
+                self.test().result = args['directive'].lower()
+            else:
+                self.test().result = args['result']
+            self.reporter.start(self.test())
+        else:
+            self.tests.append(Test(self.reporter, **args))
+            self.__tests_counter += 1
+
+    def test(self):
+        return self.tests[-1]
+
+    def end(self, name, planned):
+        if name == self.name:
+            self.tests_planned += int(planned)
+            self.__tests_base = self.__tests_counter
+            return False
+        try:
+            self.test().end()
+        except IndexError:
+            None
+        if len(self.tests) != self.tests_planned:
+            for i in range(len(self.tests), self.tests_planned):
+                self.tests.append(Test(self.reporter, 'not ok', i+1,
+                                       comment='test not present'))
+        return True
+
+
+class Run(object):
+
+    def __init__(self, reporter):
+        self.reporter = reporter
+        self.suites = []
+
+    def suite(self):
+        return self.suites[-1]
+
+    def test(self):
+        return self.suites[-1].tests[-1]
+
+    def newSuite(self, args):
+        new = False
+        try:
+            if self.suite().end(args['explanation'], args['end']):
+                new = True
+        except IndexError:
+            new = True
+        if new:
+            self.suites.append(Suite(self.reporter, **args))
+
+    def newTest(self, args):
+        self.suite().newTest(args)
+
+
+class Parser(object):
+    RE_PLAN = re.compile(r"^\s*(?P<start>\d+)\.\.(?P<end>\d+)\s*(#\s*(?P<explanation>.*))?\s*$")
+    RE_TEST_LINE = re.compile(r"^\s*(?P<result>(not\s+)?ok|[*]+)\s*(?P<id>\d+)?\s*(?P<description>[^#]+)?\s*(#\s*(?P<directive>TODO|SKIP)?\s*(?P<comment>.+)?)?\s*$",  re.IGNORECASE)
+    RE_EXPLANATION = re.compile(r"^\s*#\s*(?P<explanation>.+)?\s*$")
+    RE_YAMLISH_START = re.compile(r"^\s*---.*$")
+    RE_YAMLISH_END = re.compile(r"^\s*\.\.\.\s*$")
+
+    def __init__(self, reporter):
+        self.seek_test = False
+        self.in_test = False
+        self.in_yaml = False
+        self.run = Run(reporter)
+
+    def parse(self, source):
+        # to avoid input buffering
+        while True:
+            line = source.readline()
+            if not line:
+                break
+
+            if self.in_yaml:
+                if Parser.RE_YAMLISH_END.match(line):
+                    self.run.test()._yaml_buffer.append(line.strip())
+                    self.in_yaml = False
+                else:
+                    self.run.test()._yaml_buffer.append(line.rstrip())
+                continue
+
+            line = line.strip()
+
+            if self.in_test:
+                if Parser.RE_EXPLANATION.match(line):
+                    self.run.test().diagnostics.append(line)
+                    continue
+                if Parser.RE_YAMLISH_START.match(line):
+                    self.run.test()._yaml_buffer = [line.strip()]
+                    self.in_yaml = True
+                    continue
+
+            m = Parser.RE_PLAN.match(line)
+            if m:
+                self.seek_test = True
+                args = m.groupdict()
+                self.run.newSuite(args)
+                continue
+
+            if self.seek_test:
+                m = Parser.RE_TEST_LINE.match(line)
+                if m:
+                    args = m.groupdict()
+                    self.run.newTest(args)
+                    self.in_test = True
+                    continue
+
+            print(line)
+        try:
+            self.run.suite().end(None, 0)
+        except IndexError:
+            None
diff --git a/tools/um/tests/test.c b/tools/um/tests/test.c
new file mode 100644
index 000000000000..37ea21901f99
--- /dev/null
+++ b/tools/um/tests/test.c
@@ -0,0 +1,126 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <stdarg.h>
+#include <time.h>
+
+#include "test.h"
+
+/* circular log buffer */
+
+static char log_buf[0x10000];
+static char *head = log_buf, *tail = log_buf;
+
+static inline void advance(char **ptr)
+{
+	if ((unsigned int)(*ptr - log_buf) >= sizeof(log_buf))
+		*ptr = log_buf;
+	else
+		*ptr = *ptr + 1;
+}
+
+static void log_char(char c)
+{
+	*tail = c;
+	advance(&tail);
+	if (tail == head)
+		advance(&head);
+}
+
+static void print_log(void)
+{
+	char last;
+
+	printf(" log: |\n");
+	last = '\n';
+	while (head != tail) {
+		if (last == '\n')
+			printf("  ");
+		last = *head;
+		putchar(last);
+		advance(&head);
+	}
+	if (last != '\n')
+		putchar('\n');
+}
+
+int lkl_test_run(const struct lkl_test *tests, int nr, const char *fmt, ...)
+{
+	int i, ret, status = TEST_SUCCESS;
+	clock_t start, stop;
+	char name[1024];
+	va_list args;
+
+	va_start(args, fmt);
+	vsnprintf(name, sizeof(name), fmt, args);
+	va_end(args);
+
+	printf("1..%d # %s\n", nr, name);
+	for (i = 1; i <= nr; i++) {
+		const struct lkl_test *t = &tests[i-1];
+		unsigned long delta_us;
+
+		printf("* %d %s\n", i, t->name);
+		fflush(stdout);
+
+		start = clock();
+
+		ret = t->fn(t->arg1, t->arg2, t->arg3);
+
+		stop = clock();
+
+		switch (ret) {
+		case TEST_SUCCESS:
+			printf("ok %d %s\n", i, t->name);
+			break;
+		case TEST_SKIP:
+			printf("ok %d %s # SKIP\n", i, t->name);
+			break;
+		case TEST_BAILOUT:
+			status = TEST_BAILOUT;
+			/* fall through; */
+		case TEST_FAILURE:
+		default:
+			if (status != TEST_BAILOUT)
+				status = TEST_FAILURE;
+			printf("not ok %d %s\n", i, t->name);
+		}
+
+		printf(" ---\n");
+		delta_us = (stop - start) * 1000000 / CLOCKS_PER_SEC;
+		printf(" time_us: %ld\n", delta_us);
+		print_log();
+		printf(" ...\n");
+
+		if (status == TEST_BAILOUT) {
+			printf("Bail out!\n");
+			return TEST_FAILURE;
+		}
+
+		fflush(stdout);
+	}
+
+	return status;
+}
+
+
+void lkl_test_log(const char *str, int len)
+{
+	while (len--)
+		log_char(*(str++));
+}
+
+int lkl_test_logf(const char *fmt, ...)
+{
+	char tmp[1024], *c;
+	va_list args;
+	unsigned int n;
+
+	va_start(args, fmt);
+	n = vsnprintf(tmp, sizeof(tmp), fmt, args);
+	va_end(args);
+
+	for (c = tmp; *c != 0; c++)
+		log_char(*c);
+
+	return n > sizeof(tmp) ? sizeof(tmp) : n;
+}
diff --git a/tools/um/tests/test.h b/tools/um/tests/test.h
new file mode 100644
index 000000000000..f63ad6d419cb
--- /dev/null
+++ b/tools/um/tests/test.h
@@ -0,0 +1,72 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_TEST_H
+#define _LKL_TEST_H
+
+#define TEST_SUCCESS	0
+#define TEST_FAILURE	1
+#define TEST_SKIP	2
+#define TEST_TODO	3
+#define TEST_BAILOUT	4
+
+struct lkl_test {
+	const char *name;
+	int (*fn)();
+	void *arg1, *arg2, *arg3;
+};
+
+/**
+ * Simple wrapper to initialize a test entry.
+ * @name - test name, it assume test function is named test_@name
+ * @vargs - arguments to be passed to the function
+ */
+#define LKL_TEST(name, ...) { #name, lkl_test_##name, __VA_ARGS__ }
+
+/**
+ * lkl_test_run - run a test suite
+ *
+ * @tests - the list of tests to run
+ * @nr - number of tests
+ * @fmt - format string to be used for suite name
+ */
+int lkl_test_run(const struct lkl_test *tests, int nr, const char *fmt, ...);
+
+/**
+ * lkl_test_log - store a string in the test log buffer
+ * @str - the string to log (can be non-NULL terminated)
+ * @len - the string length
+ */
+void lkl_test_log(const char *str, int len);
+
+/**
+ * lkl_test_logf - printf like function to store into the test log buffer
+ * @fmt - printf format string
+ * @vargs - arguments to the format string
+ */
+int lkl_test_logf(const char *fmt, ...) __attribute__((format(printf, 1, 2)));
+
+/**
+ * LKL_TEST_CALL - create a test function as for a LKL call
+ *
+ * The test function will be named lkl_test_@name and will return
+ * TEST_SUCCESS if the called functions returns @expect. Otherwise
+ * will return TEST_FAILUIRE.
+ *
+ * @name - test name; must be unique because it is part of the the
+ * test function; the test function will be named
+ * @call - function to call
+ * @expect - expected return value for success
+ * @args - arguments to pass to the LKL call
+ */
+#define LKL_TEST_CALL(name, call, expect, ...)				\
+	static int lkl_test_##name(void)				\
+	{								\
+		long ret;						\
+									\
+		ret = call(__VA_ARGS__);				\
+		lkl_test_logf("%s(%s) = %ld %s\n", #call, #__VA_ARGS__, \
+			ret, ret < 0 ? lkl_strerror(ret) : "");		\
+		return (ret == expect) ? TEST_SUCCESS : TEST_FAILURE;	\
+	}
+
+
+#endif /* _LKL_TEST_H */
diff --git a/tools/um/tests/test.sh b/tools/um/tests/test.sh
new file mode 100644
index 000000000000..31b7de5c06f7
--- /dev/null
+++ b/tools/um/tests/test.sh
@@ -0,0 +1,181 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+basedir=$(cd $script_dir/..; pwd)
+base_objdir=$(cd ${OUTPUT}/; pwd)
+
+TEST_SUCCESS=0
+TEST_FAILURE=1
+TEST_SKIP=113
+TEST_TODO=114
+TEST_BAILOUT=115
+
+print_log()
+{
+    echo " log: |"
+    while read line; do
+        echo "  $line"
+    done < $1
+}
+
+export_vars()
+{
+    if [ -z "$var_file" ]; then
+        return
+    fi
+
+    for i in $@; do
+        echo "$i=${!i}" >> $var_file
+    done
+}
+
+lkl_test_run()
+{
+    log_file=$(mktemp)
+    export var_file=$(mktemp)
+
+    tid=$1 && shift && tname=$@
+
+    echo "* $tid $tname"
+
+    start=$(date '+%s%9N')
+    # run in a separate shell to avoid -e terminating us
+    $@ 2>&1 | strings >$log_file
+    exit=${PIPESTATUS[0]}
+    stop=$(date '+%s%9N')
+
+    case $exit in
+    $TEST_SUCCESS)
+        echo "ok $tid $tname"
+        ;;
+    $TEST_SKIP)
+        echo "ok $tid $tname # SKIP"
+        ;;
+    $TEST_BAILOUT)
+        echo "not ok $tid $tname"
+        echo "Bail out!"
+        ;;
+    $TEST_FAILURE|*)
+        echo "not ok $tid $tname"
+        ;;
+    esac
+
+    delta=$(((stop-start)/1000))
+
+    echo " ---"
+    echo " time_us: $delta"
+    print_log $log_file
+    echo -e " ..."
+
+    rm $log_file
+    . $var_file
+    rm $var_file
+
+    return $exit
+}
+
+lkl_test_plan()
+{
+    echo "1..$1 # $2"
+    export suite_name="${2// /\-}"
+}
+
+lkl_test_exec()
+{
+    local SUDO=""
+    local WRAPPER=""
+
+    if [ "$1" = "sudo" ]; then
+        SUDO=sudo
+        shift
+    fi
+
+    local file=$1
+    shift
+
+    if [ -n "$LKL_HOST_CONFIG_NT" ]; then
+        file=$file.exe
+    fi
+
+    file=${OUTPUT}/tests/$(basename $file)
+
+    if file $file | grep ARM; then
+        WRAPPER="qemu-arm-static"
+    elif file $file | grep "FreeBSD" ; then
+        ssh_copy "$file" $BSD_WDIR
+        if [ -n "$SUDO" ]; then
+            SUDO=""
+        fi
+        WRAPPER="$MYSSH $SU"
+        # ssh will mess up with pipes ('|') so, escape the pipe char.
+        args="${@//\|/\\\|}"
+        set - $BSD_WDIR/$(basename $file) $args
+        file=""
+    elif [ -n "$GDB" ]; then
+        WRAPPER="gdb"
+        args="$@"
+        set - -ex "run $args" -ex quit $file
+        file=""
+    elif [ -n "$VALGRIND" ]; then
+        WRAPPER="valgrind --suppressions=$script_dir/valgrind.supp \
+                  --leak-check=full --show-leak-kinds=all --xml=yes \
+                  --xml-file=valgrind-$suite_name.xml"
+    fi
+
+    $SUDO $WRAPPER $file "$@"
+}
+
+lkl_test_cmd()
+{
+    local WRAPPER=""
+
+    if [ -z "$QUIET" ]; then
+        SHOPTS="-x"
+    fi
+
+    if [ -n "$LKL_HOST_CONFIG_BSD" ]; then
+        WRAPPER="$MYSSH $SU"
+    fi
+
+    echo "$@" | $WRAPPER sh $SHOPTS
+}
+
+# XXX: $MYSSH and $MYSCP are defined in a circleci docker image.
+# see the definitions in lkl/lkl-docker:circleci/freebsd11/Dockerfile
+ssh_push()
+{
+    while [ -n "$1" ]; do
+        if [[ "$1" = *.sh ]]; then
+            type="script"
+        else
+            type="file"
+        fi
+
+        dir=$(dirname $1)
+        $MYSSH mkdir -p $BSD_WDIR/$dir
+
+        $MYSCP -P 7722 -r $basedir/$1 root@localhost:$BSD_WDIR/$dir
+        if [ "$type" = "script" ]; then
+            $MYSSH chmod a+x $BSD_WDIR/$1
+        fi
+
+        shift
+    done
+}
+
+ssh_copy()
+{
+    $MYSCP -P 7722 -r $1 root@localhost:$2
+}
+
+lkl_test_bsd_cleanup()
+{
+    $MYSSH rm -rf $BSD_WDIR
+}
+
+if [ -n "$LKL_HOST_CONFIG_BSD" ]; then
+    trap lkl_test_bsd_cleanup EXIT
+    export BSD_WDIR=/root/lkl
+    $MYSSH mkdir -p $BSD_WDIR
+fi
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v5 21/21] um: nommu: add block device support of UML
  2020-07-02 14:06     ` Hajime Tazaki
@ 2020-07-02 14:07       ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-07-02 14:07 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, Akira Moroo, linux-kernel-library, linux-arch,
	Hajime Tazaki

This commit adds block device support for nommu mode, and also added
several host utilities to run a simple test program
(tools/um/test/disk.c) successfully.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/include/asm/xor.h                 |   3 +-
 arch/um/include/shared/as-layout.h        |   1 +
 arch/um/kernel/um_arch.c                  |   5 +
 arch/um/nommu/include/uapi/asm/host_ops.h |   7 +
 arch/um/nommu/include/uapi/asm/unistd.h   |   2 +
 arch/um/nommu/um/Kconfig                  |   8 +
 arch/um/nommu/um/setup.c                  |  12 +
 tools/um/Targets                          |   2 +
 tools/um/include/lkl.h                    | 206 ++++++++++
 tools/um/include/lkl_host.h               |   1 +
 tools/um/lib/Build                        |   1 +
 tools/um/lib/fs.c                         | 461 ++++++++++++++++++++++
 tools/um/lib/posix-host.c                 |   1 +
 tools/um/lib/utils.c                      |   3 +
 tools/um/tests/Build                      |   1 +
 tools/um/tests/cla.c                      | 159 ++++++++
 tools/um/tests/cla.h                      |  33 ++
 tools/um/tests/disk.c                     | 168 ++++++++
 tools/um/tests/disk.sh                    |  70 ++++
 tools/um/tests/run.py                     |   2 +
 20 files changed, 1145 insertions(+), 1 deletion(-)
 create mode 100644 tools/um/lib/fs.c
 create mode 100644 tools/um/tests/cla.c
 create mode 100644 tools/um/tests/cla.h
 create mode 100644 tools/um/tests/disk.c
 create mode 100755 tools/um/tests/disk.sh

diff --git a/arch/um/include/asm/xor.h b/arch/um/include/asm/xor.h
index 36b33d62a35d..a5ea458a1ae9 100644
--- a/arch/um/include/asm/xor.h
+++ b/arch/um/include/asm/xor.h
@@ -4,4 +4,5 @@
 
 /* pick an arbitrary one - measuring isn't possible with inf-cpu */
 #define XOR_SELECT_TEMPLATE(x)	\
-	(time_travel_mode == TT_MODE_INFCPU ? &xor_block_8regs : NULL)
+	(time_travel_mode == TT_MODE_INFCPU || (!IS_ENABLED(CONFIG_MMU)) ? \
+	 &xor_block_8regs : NULL)
diff --git a/arch/um/include/shared/as-layout.h b/arch/um/include/shared/as-layout.h
index 5f286ef2721b..4423437a5ace 100644
--- a/arch/um/include/shared/as-layout.h
+++ b/arch/um/include/shared/as-layout.h
@@ -57,6 +57,7 @@ extern unsigned long host_task_size;
 
 extern int linux_main(int argc, char **argv);
 extern void uml_finishsetup(void);
+extern void uml_set_args(char *args);
 
 struct siginfo;
 extern void (*sig_info[])(int, struct siginfo *si, struct uml_pt_regs *);
diff --git a/arch/um/kernel/um_arch.c b/arch/um/kernel/um_arch.c
index 78d6042fd2e6..ed0149821518 100644
--- a/arch/um/kernel/um_arch.c
+++ b/arch/um/kernel/um_arch.c
@@ -42,6 +42,11 @@ static void __init add_arg(char *arg)
 	strcat(command_line, arg);
 }
 
+void __init uml_set_args(char *args)
+{
+	strcat(command_line, args);
+}
+
 /*
  * These fields are initialized at boot time and not changed.
  * XXX This structure is used only in the non-SMP case.  Maybe this
diff --git a/arch/um/nommu/include/uapi/asm/host_ops.h b/arch/um/nommu/include/uapi/asm/host_ops.h
index 0811494c080c..1b2507502969 100644
--- a/arch/um/nommu/include/uapi/asm/host_ops.h
+++ b/arch/um/nommu/include/uapi/asm/host_ops.h
@@ -15,6 +15,11 @@ struct lkl_jmp_buf {
  *
  * These operations must be provided by a host library or by the application
  * itself.
+ *
+ * @um_devices - string containg the list of UML devices in command line
+ * format. This string is appended to the kernel command line and
+ * is provided here for convenience to be implemented by the host library.
+ *
  * @print - optional operation that receives console messages
  * @panic - called during a kernel panic
  *
@@ -65,6 +70,8 @@ struct lkl_jmp_buf {
  *
  */
 struct lkl_host_operations {
+	const char *um_devices;
+
 	void (*print)(const char *str, int len);
 	void (*panic)(void);
 
diff --git a/arch/um/nommu/include/uapi/asm/unistd.h b/arch/um/nommu/include/uapi/asm/unistd.h
index 1762ebb829f5..320762099a62 100644
--- a/arch/um/nommu/include/uapi/asm/unistd.h
+++ b/arch/um/nommu/include/uapi/asm/unistd.h
@@ -3,6 +3,8 @@
 #define __UM_NOMMU_UAPI_UNISTD_H
 
 #define __ARCH_WANT_NEW_STAT
+#define __ARCH_WANT_SET_GET_RLIMIT
+
 #include <asm/bitsperlong.h>
 
 #if __BITS_PER_LONG == 64
diff --git a/arch/um/nommu/um/Kconfig b/arch/um/nommu/um/Kconfig
index 20b3eaccb6f0..c6a3f472fe75 100644
--- a/arch/um/nommu/um/Kconfig
+++ b/arch/um/nommu/um/Kconfig
@@ -4,6 +4,10 @@ config UML_NOMMU
 	select UACCESS_MEMCPY
 	select ARCH_THREAD_STACK_ALLOCATOR
 	select ARCH_HAS_SYSCALL_WRAPPER
+	select VFAT_FS
+	select NLS_CODEPAGE_437
+	select NLS_ISO8859_1
+	select BTRFS_FS
 
 config 64BIT
 	bool
@@ -35,3 +39,7 @@ config STACKTRACE_SUPPORT
 config PRINTK_TIME
 	bool
 	default y
+
+config RAID6_PQ_BENCHMARK
+	bool
+	default n
diff --git a/arch/um/nommu/um/setup.c b/arch/um/nommu/um/setup.c
index f74baea50bd3..6b49bf7a99fb 100644
--- a/arch/um/nommu/um/setup.c
+++ b/arch/um/nommu/um/setup.c
@@ -45,13 +45,25 @@ static void __init *lkl_run_kernel(void *arg)
 	return NULL;
 }
 
+static char _cmd_line[COMMAND_LINE_SIZE];
 int __init lkl_start_kernel(struct lkl_host_operations *ops,
 			    const char *fmt, ...)
 {
+	va_list ap;
 	int ret;
 
 	lkl_ops = ops;
 
+	va_start(ap, fmt);
+	ret = vsnprintf(_cmd_line, COMMAND_LINE_SIZE, fmt, ap);
+	va_end(ap);
+
+	if (ops->um_devices)
+		strscpy(_cmd_line + ret, ops->um_devices,
+			COMMAND_LINE_SIZE - ret);
+
+	uml_set_args(_cmd_line);
+
 	init_sem = lkl_ops->sem_alloc(0);
 	if (!init_sem)
 		return -ENOMEM;
diff --git a/tools/um/Targets b/tools/um/Targets
index 2bb90381843c..f5f8ec4b9dbb 100644
--- a/tools/um/Targets
+++ b/tools/um/Targets
@@ -1,10 +1,12 @@
 ifeq ($(UMMODE),library)
 progs-y += tests/boot
+progs-y += tests/disk
 else
 progs-y += uml/linux
 endif
 
 LDFLAGS_boot-y := -pie
+LDFLAGS_disk-y := -pie
 
 LDLIBS := -lrt -lpthread -lutil
 LDFLAGS_linux-y := -no-pie -Wl,--wrap,malloc -Wl,--wrap,free -Wl,--wrap,calloc
diff --git a/tools/um/include/lkl.h b/tools/um/include/lkl.h
index 707e01b64a70..4a9d874bfbf0 100644
--- a/tools/um/include/lkl.h
+++ b/tools/um/include/lkl.h
@@ -113,6 +113,16 @@ static inline long lkl_sys_creat(const char *file, int mode)
 }
 #endif
 
+#ifdef __lkl__NR_faccessat
+/**
+ * lkl_sys_access - wrapper for lkl_sys_faccessat
+ */
+static inline long lkl_sys_access(const char *file, int mode)
+{
+	return lkl_sys_faccessat(LKL_AT_FDCWD, file, mode);
+}
+#endif
+
 #ifdef __lkl__NR_mkdirat
 /**
  * lkl_sys_mkdir - wrapper for lkl_sys_mkdirat
@@ -123,6 +133,36 @@ static inline long lkl_sys_mkdir(const char *path, mode_t mode)
 }
 #endif
 
+#ifdef __lkl__NR_unlinkat
+/**
+ * lkl_sys_rmdir - wrapper for lkl_sys_unlinkrat
+ */
+static inline long lkl_sys_rmdir(const char *path)
+{
+	return lkl_sys_unlinkat(LKL_AT_FDCWD, path, LKL_AT_REMOVEDIR);
+}
+#endif
+
+#ifdef __lkl__NR_unlinkat
+/**
+ * lkl_sys_unlink - wrapper for lkl_sys_unlinkat
+ */
+static inline long lkl_sys_unlink(const char *path)
+{
+	return lkl_sys_unlinkat(LKL_AT_FDCWD, path, 0);
+}
+#endif
+
+#ifdef __lkl__NR_mknodat
+/**
+ * lkl_sys_mknod - wrapper for lkl_sys_mknodat
+ */
+static inline long lkl_sys_mknod(const char *path, mode_t mode, dev_t dev)
+{
+	return lkl_sys_mknodat(LKL_AT_FDCWD, path, mode, dev);
+}
+#endif
+
 #ifdef __lkl__NR_epoll_create1
 /**
  * lkl_sys_epoll_create - wrapper for lkl_sys_epoll_create1
@@ -144,6 +184,172 @@ static inline long lkl_sys_epoll_wait(int fd, struct lkl_epoll_event *ev,
 }
 #endif
 
+/**
+ * struct lkl_dev_blk_ops - block device host operations, defined in lkl_host.h.
+ */
+struct lkl_dev_blk_ops;
+
+/**
+ * lkl_disk - host disk handle
+ *
+ * @dev - a pointer to private information for this disk backend
+ * @fd - a POSIX file descriptor that can be used by preadv/pwritev
+ * @handle - an NT file handle that can be used by ReadFile/WriteFile
+ */
+struct lkl_disk {
+	void *dev;
+	union {
+		int fd;
+		void *handle;
+	};
+	struct lkl_dev_blk_ops *ops;
+};
+
+/**
+ * lkl_disk_add - add a new disk
+ *
+ * @disk - the host disk handle
+ * @returns a disk id (0 is valid) or a strictly negative value in case of error
+ */
+int lkl_disk_add(struct lkl_disk *disk);
+
+/**
+ * lkl_disk_remove - remove a disk
+ *
+ * This function makes a cleanup of the @disk's private information
+ * that was initialized by lkl_disk_add before.
+ *
+ * @disk - the host disk handle
+ */
+int lkl_disk_remove(struct lkl_disk disk);
+
+/**
+ * lkl_encode_dev_from_sysfs_blkdev - extract device id from sysfs
+ *
+ * This function returns the device id for the given sysfs dev node.
+ * The content of the node has to be in the form 'MAJOR:MINOR'.
+ * Also, this function expects an absolute path which means that sysfs
+ * already has to be mounted at the given path
+ *
+ * @sysfs_path - absolute path to the sysfs dev node
+ * @pdevid - pointer to memory where dev id will be returned
+ * @returns - 0 on success, a negative value on error
+ */
+int lkl_encode_dev_from_sysfs(const char *sysfs_path, uint32_t *pdevid);
+
+/**
+ * lkl_mount_dev - mount a disk
+ *
+ * This functions creates a device file for the given disk, creates a mount
+ * point and mounts the device over the mount point.
+ *
+ * @disk_id - the disk id identifying the disk to be mounted
+ * @part - disk partition or zero for full disk
+ * @fs_type - filesystem type
+ * @flags - mount flags
+ * @opts - additional filesystem specific mount options
+ * @mnt_str - a string that will be filled by this function with the path where
+ * the filesystem has been mounted
+ * @mnt_str_len - size of mnt_str
+ * @returns - 0 on success, a negative value on error
+ */
+long lkl_mount_dev(unsigned int disk_id, unsigned int part, const char *fs_type,
+		   int flags, const char *opts,
+		   char *mnt_str, unsigned int mnt_str_len);
+
+/**
+ * lkl_umount_dev - umount a disk
+ *
+ * This functions umounts the given disks and removes the device file and the
+ * mount point.
+ *
+ * @disk_id - the disk id identifying the disk to be mounted
+ * @part - disk partition or zero for full disk
+ * @flags - umount flags
+ * @timeout_ms - timeout to wait for the kernel to flush closed files so that
+ * umount can succeed
+ * @returns - 0 on success, a negative value on error
+ */
+long lkl_umount_dev(unsigned int disk_id, unsigned int part, int flags,
+		    long timeout_ms);
+
+/**
+ * lkl_umount_timeout - umount filesystem with timeout
+ *
+ * @path - the path to unmount
+ * @flags - umount flags
+ * @timeout_ms - timeout to wait for the kernel to flush closed files so that
+ * umount can succeed
+ * @returns - 0 on success, a negative value on error
+ */
+long lkl_umount_timeout(char *path, int flags, long timeout_ms);
+
+/**
+ * lkl_opendir - open a directory
+ *
+ * @path - directory path
+ * @err - pointer to store the error in case of failure
+ * @returns - a handle to be used when calling lkl_readdir
+ */
+struct lkl_dir *lkl_opendir(const char *path, int *err);
+
+/**
+ * lkl_fdopendir - open a directory
+ *
+ * @fd - file descriptor
+ * @err - pointer to store the error in case of failure
+ * @returns - a handle to be used when calling lkl_readdir
+ */
+struct lkl_dir *lkl_fdopendir(int fd, int *err);
+
+/**
+ * lkl_rewinddir - reset directory stream
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ */
+void lkl_rewinddir(struct lkl_dir *dir);
+
+/**
+ * lkl_closedir - close the directory
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ */
+int lkl_closedir(struct lkl_dir *dir);
+
+/**
+ * lkl_readdir - get the next available entry of the directory
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ * @returns - a lkl_dirent64 entry or NULL if the end of the directory stream is
+ * reached or if an error occurred; check lkl_errdir() to distinguish between
+ * errors or end of the directory stream
+ */
+struct lkl_linux_dirent64 *lkl_readdir(struct lkl_dir *dir);
+
+/**
+ * lkl_errdir - checks if an error occurred during the last lkl_readdir call
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ * @returns - 0 if no error occurred, or a negative value otherwise
+ */
+int lkl_errdir(struct lkl_dir *dir);
+
+/**
+ * lkl_dirfd - gets the file descriptor associated with the directory handle
+ *
+ * @dir - the directory handle as returned by lkl_opendir
+ * @returns - a positive value,which is the LKL file descriptor associated with
+ * the directory handle, or a negative value otherwise
+ */
+int lkl_dirfd(struct lkl_dir *dir);
+
+/**
+ * lkl_mount_fs - mount a file system type like proc, sys
+ * @fstype - file system type. e.g. proc, sys
+ * @returns - 0 on success. 1 if it's already mounted. negative on failure.
+ */
+int lkl_mount_fs(char *fstype);
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/tools/um/include/lkl_host.h b/tools/um/include/lkl_host.h
index 85e80eb4ad0d..12dd95616e43 100644
--- a/tools/um/include/lkl_host.h
+++ b/tools/um/include/lkl_host.h
@@ -10,6 +10,7 @@ extern "C" {
 #include <lkl.h>
 
 extern struct lkl_host_operations lkl_host_ops;
+extern char lkl_um_devs[4096];
 
 /**
  * lkl_printf - print a message via the host print operation
diff --git a/tools/um/lib/Build b/tools/um/lib/Build
index dddff26a3b4e..29491b40746c 100644
--- a/tools/um/lib/Build
+++ b/tools/um/lib/Build
@@ -4,3 +4,4 @@ CFLAGS_posix-host.o += -D_FILE_OFFSET_BITS=64
 liblinux-$(CONFIG_UMMODE_LIB) += utils.o
 liblinux-$(CONFIG_UMMODE_LIB) += posix-host.o
 liblinux-$(CONFIG_UMMODE_LIB) += jmp_buf.o
+liblinux-$(CONFIG_UMMODE_LIB) += fs.o
diff --git a/tools/um/lib/fs.c b/tools/um/lib/fs.c
new file mode 100644
index 000000000000..948aac9730c2
--- /dev/null
+++ b/tools/um/lib/fs.c
@@ -0,0 +1,461 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdarg.h>
+#include <stdio.h>
+#include <string.h>
+#include <stdlib.h>
+#include <lkl_host.h>
+
+#define MAX_FSTYPE_LEN 50
+
+static struct lkl_disk *lkl_disks[16];
+static int registered_blk_dev_idx;
+
+static int lkl_disk_um_add(struct lkl_disk *disk, const char *blkparams)
+{
+	/* concat strings */
+	snprintf(lkl_um_devs + strlen(lkl_um_devs), sizeof(lkl_um_devs),
+		 " ubd%d=%s", registered_blk_dev_idx, blkparams);
+
+	return registered_blk_dev_idx++;
+}
+
+int lkl_disk_add(struct lkl_disk *disk)
+{
+	int ret = -1;
+
+	ret = lkl_disk_um_add(disk, disk->dev);
+
+	lkl_disks[ret] = disk;
+
+	return ret;
+}
+
+int lkl_disk_remove(struct lkl_disk disk)
+{
+	/* FIXME */
+	return 0;
+}
+
+int lkl_mount_fs(char *fstype)
+{
+	char dir[MAX_FSTYPE_LEN+2] = "/";
+	int flags = 0, ret = 0;
+
+	strncat(dir, fstype, MAX_FSTYPE_LEN);
+
+	/* Create with regular umask */
+	ret = lkl_sys_mkdir(dir, 0xff);
+	if (ret && ret != -LKL_EEXIST) {
+		lkl_perror("mount_fs mkdir", ret);
+		return ret;
+	}
+
+	/* We have no use for nonzero flags right now */
+	ret = lkl_sys_mount("none", dir, fstype, flags, NULL);
+	if (ret && ret != -LKL_EBUSY) {
+		lkl_sys_rmdir(dir);
+		return ret;
+	}
+
+	if (ret == -LKL_EBUSY)
+		return 1;
+	return 0;
+}
+
+static uint32_t new_encode_dev(unsigned int major, unsigned int minor)
+{
+	return (minor & 0xff) | (major << 8) | ((minor & ~0xff) << 12);
+}
+
+static int startswith(const char *str, const char *pre)
+{
+	return strncmp(pre, str, strlen(pre)) == 0;
+}
+
+static int get_node_with_prefix(const char *path, const char *prefix,
+				char *result, unsigned int result_len)
+{
+	struct lkl_dir *dir = NULL;
+	struct lkl_linux_dirent64 *dirent;
+	int ret;
+
+	dir = lkl_opendir(path, &ret);
+	if (!dir)
+		return ret;
+
+	ret = -LKL_ENOENT;
+
+	while ((dirent = lkl_readdir(dir))) {
+		if (startswith(dirent->d_name, prefix)) {
+			if (strlen(dirent->d_name) + 1 > result_len) {
+				ret = -LKL_ENOMEM;
+				break;
+			}
+			memcpy(result, dirent->d_name, strlen(dirent->d_name));
+			result[strlen(dirent->d_name)] = '\0';
+			ret = 0;
+			break;
+		}
+	}
+
+	lkl_closedir(dir);
+
+	return ret;
+}
+
+int lkl_encode_dev_from_sysfs(const char *sysfs_path, uint32_t *pdevid)
+{
+	int ret;
+	long fd;
+	int major, minor;
+	char buf[16] = { 0, };
+	char *bufptr;
+
+	fd = lkl_sys_open(sysfs_path, LKL_O_RDONLY, 0);
+	if (fd < 0)
+		return fd;
+
+	ret = lkl_sys_read(fd, buf, sizeof(buf));
+	if (ret < 0)
+		goto out_close;
+
+	if (ret == sizeof(buf)) {
+		ret = -LKL_ENOBUFS;
+		goto out_close;
+	}
+
+	bufptr = strchr(buf, ':');
+	if (bufptr == NULL) {
+		ret = -LKL_EINVAL;
+		goto out_close;
+	}
+	bufptr[0] = '\0';
+	bufptr++;
+
+	major = atoi(buf);
+	minor = atoi(bufptr);
+
+	*pdevid = new_encode_dev(major, minor);
+	ret = 0;
+
+out_close:
+	lkl_sys_close(fd);
+
+	return ret;
+}
+
+#define SYSFS_DEV_UMBLK_CMDLINE_PATH \
+	"/sysfs/devices/platform/uml-blkdev.%d"
+
+struct abuf {
+	char *mem, *ptr;
+	unsigned int len;
+};
+
+static int snprintf_append(struct abuf *buf, const char *fmt, ...)
+{
+	int ret;
+	va_list args;
+
+	if (!buf->ptr)
+		buf->ptr = buf->mem;
+
+	va_start(args, fmt);
+	ret = vsnprintf(buf->ptr, buf->len - (buf->ptr - buf->mem), fmt, args);
+	va_end(args);
+
+	if (ret < 0 || (ret >= (int)(buf->len - (buf->ptr - buf->mem))))
+		return -LKL_ENOMEM;
+
+	buf->ptr += ret;
+
+	return 0;
+}
+
+static int __lkl_get_blkdev(int disk_id, unsigned int part, uint32_t *pdevid,
+			    const char *sysfs_path_fmt, const char *drv_prefix,
+			    const char *disk_prefix)
+{
+	char sysfs_path[LKL_PATH_MAX];
+	char drv_name[LKL_PATH_MAX];
+	char disk_name[LKL_PATH_MAX];
+	struct abuf sysfs_path_buf = {
+		.mem = sysfs_path,
+		.len = sizeof(sysfs_path),
+	};
+	int ret;
+
+	if (disk_id < 0)
+		return -LKL_EINVAL;
+
+	ret = lkl_mount_fs("sysfs");
+	if (ret < 0)
+		return ret;
+
+	ret = snprintf_append(&sysfs_path_buf, sysfs_path_fmt, disk_id);
+	if (ret)
+		return ret;
+
+	ret = get_node_with_prefix(sysfs_path, drv_prefix, drv_name,
+				   sizeof(drv_name));
+	if (ret)
+		return ret;
+
+	ret = snprintf_append(&sysfs_path_buf, "/%s/block", drv_name);
+	if (ret)
+		return ret;
+
+	ret = get_node_with_prefix(sysfs_path, disk_prefix, disk_name,
+				   sizeof(disk_name));
+	if (ret)
+		return ret;
+
+	if (!part)
+		ret = snprintf_append(&sysfs_path_buf, "/%s/dev", disk_name);
+	else
+		ret = snprintf_append(&sysfs_path_buf, "/%s/%s%d/dev",
+				      disk_name, disk_name, part);
+	if (ret)
+		return ret;
+
+	return lkl_encode_dev_from_sysfs(sysfs_path, pdevid);
+}
+
+int lkl_get_blkdev(int disk_id, unsigned int part, uint32_t *pdevid)
+{
+	char *fmt;
+
+	fmt = SYSFS_DEV_UMBLK_CMDLINE_PATH;
+	return __lkl_get_blkdev(disk_id, part, pdevid, fmt, "", "ubd");
+}
+
+long lkl_mount_dev(unsigned int disk_id, unsigned int part,
+		   const char *fs_type, int flags,
+		   const char *data, char *mnt_str, unsigned int mnt_str_len)
+{
+	char dev_str[] = { "/dev/xxxxxxxx" };
+	unsigned int dev;
+	int err;
+	char _data[4096]; /* FIXME: PAGE_SIZE is not exported by LKL */
+
+	if (mnt_str_len < sizeof(dev_str))
+		return -LKL_ENOMEM;
+
+	err = lkl_get_blkdev(disk_id, part, &dev);
+	if (err < 0)
+		return err;
+
+	snprintf(dev_str, sizeof(dev_str), "/dev/%08x", dev);
+	snprintf(mnt_str, mnt_str_len, "/mnt/%08x", dev);
+
+	err = lkl_sys_access("/dev", LKL_S_IRWXO);
+	if (err < 0) {
+		if (err == -LKL_ENOENT)
+			err = lkl_sys_mkdir("/dev", 0700);
+		if (err < 0)
+			return err;
+	}
+
+	err = lkl_sys_mknod(dev_str, LKL_S_IFBLK | 0600, dev);
+	if (err < 0)
+		return err;
+
+	err = lkl_sys_access("/mnt", LKL_S_IRWXO);
+	if (err < 0) {
+		if (err == -LKL_ENOENT)
+			err = lkl_sys_mkdir("/mnt", 0700);
+		if (err < 0)
+			return err;
+	}
+
+	err = lkl_sys_mkdir(mnt_str, 0700);
+	if (err < 0) {
+		lkl_sys_unlink(dev_str);
+		return err;
+	}
+
+	/* kernel always copies a full page */
+	if (data) {
+		strncpy(_data, data, sizeof(_data));
+		_data[sizeof(_data) - 1] = 0;
+	} else {
+		_data[0] = 0;
+	}
+
+	err = lkl_sys_mount(dev_str, mnt_str, (char *)fs_type, flags, _data);
+	if (err < 0) {
+		lkl_sys_unlink(dev_str);
+		lkl_sys_rmdir(mnt_str);
+		return err;
+	}
+
+	return 0;
+}
+
+long lkl_umount_timeout(char *path, int flags, long timeout_ms)
+{
+	long incr = 10000000; /* 10 ms */
+	struct lkl_timespec ts = {
+		.tv_sec = 0,
+		.tv_nsec = incr,
+	};
+	long err;
+
+	do {
+		err = lkl_sys_umount(path, flags);
+		if (err == -LKL_EBUSY) {
+			lkl_sys_nanosleep((struct __lkl__kernel_timespec *)&ts,
+					  NULL);
+			timeout_ms -= incr / 1000000;
+		}
+	} while (err == -LKL_EBUSY && timeout_ms > 0);
+
+	return err;
+}
+
+long lkl_umount_dev(unsigned int disk_id, unsigned int part, int flags,
+		    long timeout_ms)
+{
+	char dev_str[] = { "/dev/xxxxxxxx" };
+	char mnt_str[] = { "/mnt/xxxxxxxx" };
+	unsigned int dev;
+	int err;
+
+	err = lkl_get_blkdev(disk_id, part, &dev);
+	if (err < 0)
+		return err;
+
+	snprintf(dev_str, sizeof(dev_str), "/dev/%08x", dev);
+	snprintf(mnt_str, sizeof(mnt_str), "/mnt/%08x", dev);
+
+	err = lkl_umount_timeout(mnt_str, flags, timeout_ms);
+	if (err)
+		return err;
+
+	err = lkl_sys_unlink(dev_str);
+	if (err)
+		return err;
+
+	return lkl_sys_rmdir(mnt_str);
+}
+
+struct lkl_dir {
+	int fd;
+	char buf[1024];
+	char *pos;
+	int len;
+};
+
+static struct lkl_dir *lkl_dir_alloc(int *err)
+{
+	struct lkl_dir *dir = lkl_host_ops.mem_alloc(sizeof(struct lkl_dir));
+
+	if (!dir) {
+		*err = -LKL_ENOMEM;
+		return NULL;
+	}
+
+	dir->len = 0;
+	dir->pos = NULL;
+
+	return dir;
+}
+
+struct lkl_dir *lkl_opendir(const char *path, int *err)
+{
+	struct lkl_dir *dir = lkl_dir_alloc(err);
+
+	if (!dir) {
+		*err = -LKL_ENOMEM;
+		return NULL;
+	}
+
+	dir->fd = lkl_sys_open(path, LKL_O_RDONLY | LKL_O_DIRECTORY, 0);
+	if (dir->fd < 0) {
+		*err = dir->fd;
+		lkl_host_ops.mem_free(dir);
+		return NULL;
+	}
+
+	*err = 0;
+
+	return dir;
+}
+
+struct lkl_dir *lkl_fdopendir(int fd, int *err)
+{
+	struct lkl_dir *dir = lkl_dir_alloc(err);
+
+	if (!dir)
+		return NULL;
+
+	dir->fd = fd;
+
+	return dir;
+}
+
+void lkl_rewinddir(struct lkl_dir *dir)
+{
+	lkl_sys_lseek(dir->fd, 0, LKL_SEEK_SET);
+	dir->len = 0;
+	dir->pos = NULL;
+}
+
+int lkl_closedir(struct lkl_dir *dir)
+{
+	int ret;
+
+	ret = lkl_sys_close(dir->fd);
+	lkl_host_ops.mem_free(dir);
+
+	return ret;
+}
+
+struct lkl_linux_dirent64 *lkl_readdir(struct lkl_dir *dir)
+{
+	struct lkl_linux_dirent64 *de;
+
+	if (dir->len < 0)
+		return NULL;
+
+	if (!dir->pos || dir->pos - dir->buf >= dir->len)
+		goto read_buf;
+
+return_de:
+	de = (struct lkl_linux_dirent64 *)dir->pos;
+	dir->pos += de->d_reclen;
+
+	return de;
+
+read_buf:
+	dir->pos = NULL;
+	de = (struct lkl_linux_dirent64 *)dir->buf;
+	dir->len = lkl_sys_getdents64(dir->fd, de, sizeof(dir->buf));
+	if (dir->len <= 0)
+		return NULL;
+
+	dir->pos = dir->buf;
+	goto return_de;
+}
+
+int lkl_errdir(struct lkl_dir *dir)
+{
+	if (dir->len >= 0)
+		return 0;
+
+	return dir->len;
+}
+
+int lkl_dirfd(struct lkl_dir *dir)
+{
+	return dir->fd;
+}
+
+int lkl_set_fd_limit(unsigned int fd_limit)
+{
+	struct lkl_rlimit rlim = {
+		.rlim_cur = fd_limit,
+		.rlim_max = fd_limit,
+	};
+	return lkl_sys_setrlimit(LKL_RLIMIT_NOFILE, &rlim);
+}
diff --git a/tools/um/lib/posix-host.c b/tools/um/lib/posix-host.c
index b6b5b2902254..8fd88031bf2b 100644
--- a/tools/um/lib/posix-host.c
+++ b/tools/um/lib/posix-host.c
@@ -263,6 +263,7 @@ static void *tls_get(struct lkl_tls_key *key)
 }
 
 struct lkl_host_operations lkl_host_ops = {
+	.um_devices = lkl_um_devs,
 	.panic = panic,
 	.print = print,
 	.mem_alloc = (void *)malloc,
diff --git a/tools/um/lib/utils.c b/tools/um/lib/utils.c
index 4930479a8a35..ac65cd744a14 100644
--- a/tools/um/lib/utils.c
+++ b/tools/um/lib/utils.c
@@ -4,6 +4,9 @@
 #include <string.h>
 #include <lkl_host.h>
 
+/* XXX: find a better place */
+char lkl_um_devs[4096];
+
 static const char * const lkl_err_strings[] = {
 	"Success",
 	"Operation not permitted",
diff --git a/tools/um/tests/Build b/tools/um/tests/Build
index 26a607f90dc8..b62919cadc50 100644
--- a/tools/um/tests/Build
+++ b/tools/um/tests/Build
@@ -1 +1,2 @@
 boot-y += boot.o test.o
+disk-y += disk.o test.o cla.o
diff --git a/tools/um/tests/cla.c b/tools/um/tests/cla.c
new file mode 100644
index 000000000000..694e3a3822be
--- /dev/null
+++ b/tools/um/tests/cla.c
@@ -0,0 +1,159 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <string.h>
+#include <errno.h>
+#include <stdlib.h>
+#ifdef __MINGW32__
+#include <winsock2.h>
+#else
+#include <sys/socket.h>
+#include <netinet/in.h>
+#include <arpa/inet.h>
+#endif
+
+#include "cla.h"
+
+static int cl_arg_parse_bool(struct cl_arg *arg, const char *value)
+{
+	*((int *)arg->store) = 1;
+	return 0;
+}
+
+static int cl_arg_parse_str(struct cl_arg *arg, const char *value)
+{
+	*((const char **)arg->store) = value;
+	return 0;
+}
+
+static int cl_arg_parse_int(struct cl_arg *arg, const char *value)
+{
+	errno = 0;
+	*((int *)arg->store) = strtol(value, NULL, 0);
+	return errno == 0;
+}
+
+static int cl_arg_parse_str_set(struct cl_arg *arg, const char *value)
+{
+	const char **set = arg->set;
+	int i;
+
+	for (i = 0; set[i] != NULL; i++) {
+		if (strcmp(set[i], value) == 0) {
+			*((int *)arg->store) = i;
+			return 0;
+		}
+	}
+
+	return (-1);
+}
+
+static int cl_arg_parse_ipv4(struct cl_arg *arg, const char *value)
+{
+	unsigned int addr;
+
+	if (!value)
+		return (-1);
+
+	addr = inet_addr(value);
+	if (addr == INADDR_NONE)
+		return (-1);
+	*((unsigned int *)arg->store) = addr;
+	return 0;
+}
+
+static cl_arg_parser_t parsers[] = {
+	[CL_ARG_BOOL] = cl_arg_parse_bool,
+	[CL_ARG_INT] = cl_arg_parse_int,
+	[CL_ARG_STR] = cl_arg_parse_str,
+	[CL_ARG_STR_SET] = cl_arg_parse_str_set,
+	[CL_ARG_IPV4] = cl_arg_parse_ipv4,
+};
+
+static struct cl_arg *find_short_arg(char name, struct cl_arg *args)
+{
+	struct cl_arg *arg;
+
+	for (arg = args; arg->short_name != 0; arg++) {
+		if (arg->short_name == name)
+			return arg;
+	}
+
+	return NULL;
+}
+
+static struct cl_arg *find_long_arg(const char *name, struct cl_arg *args)
+{
+	struct cl_arg *arg;
+
+	for (arg = args; arg->long_name; arg++) {
+		if (strcmp(arg->long_name, name) == 0)
+			return arg;
+	}
+
+	return NULL;
+}
+
+static void print_help(struct cl_arg *args)
+{
+	struct cl_arg *arg;
+
+	fprintf(stderr, "usage:\n");
+	for (arg = args; arg->long_name; arg++) {
+		fprintf(stderr, "-%c, --%-20s %s", arg->short_name,
+			arg->long_name, arg->help);
+		if (arg->type == CL_ARG_STR_SET) {
+			const char **set = arg->set;
+
+			fprintf(stderr, " [ ");
+			while (*set != NULL)
+				fprintf(stderr, "%s ", *(set++));
+			fprintf(stderr, "]");
+		}
+		fprintf(stderr, "\n");
+	}
+}
+
+int cla_parse_args(int argc, const char **argv, struct cl_arg *args)
+{
+	int i;
+
+	for (i = 1; i < argc; i++) {
+		struct cl_arg *arg = NULL;
+		cl_arg_parser_t parser;
+
+		if (argv[i][0] == '-') {
+			if (argv[i][1] != '-')
+				arg = find_short_arg(argv[i][1], args);
+			else
+				arg = find_long_arg(&argv[i][2], args);
+		}
+
+		if (!arg) {
+			fprintf(stderr, "unknown option '%s'\n", argv[i]);
+			print_help(args);
+			return (-1);
+		}
+
+		if (arg->type == CL_ARG_USER || arg->type >= CL_ARG_END)
+			parser = arg->parser;
+		else
+			parser = parsers[arg->type];
+
+		if (!parser) {
+			fprintf(stderr, "can't parse --'%s'/-'%c'\n",
+				arg->long_name, args->short_name);
+			return (-1);
+		}
+
+		if (parser(arg, argv[i + 1]) < 0) {
+			fprintf(stderr, "can't parse '%s'\n", argv[i]);
+			print_help(args);
+			return (-1);
+		}
+
+		if (arg->has_arg)
+			i++;
+	}
+
+	return 0;
+}
diff --git a/tools/um/tests/cla.h b/tools/um/tests/cla.h
new file mode 100644
index 000000000000..3d879233681f
--- /dev/null
+++ b/tools/um/tests/cla.h
@@ -0,0 +1,33 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef _LKL_TEST_CLA_H
+#define _LKL_TEST_CLA_H
+
+enum cl_arg_type {
+	CL_ARG_USER = 0,
+	CL_ARG_BOOL,
+	CL_ARG_INT,
+	CL_ARG_STR,
+	CL_ARG_STR_SET,
+	CL_ARG_IPV4,
+	CL_ARG_END,
+};
+
+struct cl_arg;
+
+typedef int (*cl_arg_parser_t)(struct cl_arg *arg, const char *value);
+
+struct cl_arg {
+	const char *long_name;
+	char short_name;
+	const char *help;
+	int has_arg;
+	enum cl_arg_type type;
+	void *store;
+	void *set;
+	cl_arg_parser_t parser;
+};
+
+int cla_parse_args(int argc, const char **argv, struct cl_arg *args);
+
+
+#endif /* _LKL_TEST_CLA_H */
diff --git a/tools/um/tests/disk.c b/tools/um/tests/disk.c
new file mode 100644
index 000000000000..325934935e6a
--- /dev/null
+++ b/tools/um/tests/disk.c
@@ -0,0 +1,168 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <unistd.h>
+#include <string.h>
+#include <time.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <lkl.h>
+#include <lkl_host.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <sys/ioctl.h>
+
+#include "test.h"
+#include "cla.h"
+
+static struct {
+	int printk;
+	const char *disk;
+	const char *fstype;
+	int partition;
+} cla;
+
+struct cl_arg args[] = {
+	{"disk", 'd', "disk file to use", 1, CL_ARG_STR, &cla.disk},
+	{"partition", 'P', "partition to mount", 1, CL_ARG_INT, &cla.partition},
+	{"type", 't', "filesystem type", 1, CL_ARG_STR, &cla.fstype},
+	{0},
+};
+
+
+static struct lkl_disk disk;
+static int disk_id = -1;
+
+int lkl_test_disk_add(void)
+{
+	disk.fd = open(cla.disk, O_RDWR);
+	if (disk.fd < 0)
+		goto out_unlink;
+
+	disk.ops = NULL;
+	disk.dev = (char *)cla.disk;
+
+	disk_id = lkl_disk_add(&disk);
+	if (disk_id < 0)
+		goto out_close;
+
+	goto out;
+
+out_close:
+	close(disk.fd);
+
+out_unlink:
+	unlink(cla.disk);
+
+out:
+	lkl_test_logf("disk fd/handle %x disk_id %d", disk.fd, disk_id);
+
+	if (disk_id >= 0)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+int lkl_test_disk_remove(void)
+{
+	int ret;
+
+	ret = lkl_disk_remove(disk);
+
+	close(disk.fd);
+
+	if (ret == 0)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+
+static char mnt_point[32];
+
+LKL_TEST_CALL(mount_dev, lkl_mount_dev, 0, disk_id, cla.partition, cla.fstype,
+	      0, NULL, mnt_point, sizeof(mnt_point))
+
+static int lkl_test_umount_dev(void)
+{
+	long ret, ret2;
+
+	ret = lkl_sys_chdir("/");
+
+	ret2 = lkl_umount_dev(disk_id, cla.partition, 0, 1000);
+
+	lkl_test_logf("%ld %ld", ret, ret2);
+
+	if (!ret && !ret2)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+struct lkl_dir *dir;
+
+static int lkl_test_opendir(void)
+{
+	int err;
+
+	dir = lkl_opendir(mnt_point, &err);
+
+	lkl_test_logf("lkl_opedir(%s) = %d %s\n", mnt_point, err,
+		      lkl_strerror(err));
+
+	if (err == 0)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+static int lkl_test_readdir(void)
+{
+	struct lkl_linux_dirent64 *de = lkl_readdir(dir);
+	int wr = 0;
+
+	while (de) {
+		wr += lkl_test_logf("%s ", de->d_name);
+		if (wr >= 70) {
+			lkl_test_logf("\n");
+			wr = 0;
+			break;
+		}
+		de = lkl_readdir(dir);
+	}
+
+	if (lkl_errdir(dir) == 0)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+LKL_TEST_CALL(closedir, lkl_closedir, 0, dir);
+LKL_TEST_CALL(chdir_mnt_point, lkl_sys_chdir, 0, mnt_point);
+LKL_TEST_CALL(start_kernel, lkl_start_kernel, 0, &lkl_host_ops,
+	     "mem=16M loglevel=8");
+LKL_TEST_CALL(stop_kernel, lkl_sys_halt, 0);
+
+struct lkl_test tests[] = {
+	LKL_TEST(disk_add),
+	LKL_TEST(start_kernel),
+	LKL_TEST(mount_dev),
+	LKL_TEST(chdir_mnt_point),
+	LKL_TEST(opendir),
+	LKL_TEST(readdir),
+	LKL_TEST(closedir),
+	LKL_TEST(umount_dev),
+	LKL_TEST(stop_kernel),
+	LKL_TEST(disk_remove),
+
+};
+
+int main(int argc, const char **argv)
+{
+	if (cla_parse_args(argc, argv, args) < 0)
+		return (-1);
+
+	lkl_host_ops.print = lkl_test_log;
+
+	return lkl_test_run(tests, sizeof(tests)/sizeof(struct lkl_test),
+			    "disk %s", cla.fstype);
+}
diff --git a/tools/um/tests/disk.sh b/tools/um/tests/disk.sh
new file mode 100755
index 000000000000..e2ec6cf69d4b
--- /dev/null
+++ b/tools/um/tests/disk.sh
@@ -0,0 +1,70 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+
+source $script_dir/test.sh
+
+function prepfs()
+{
+    set -e
+
+    file=`mktemp`
+
+    dd if=/dev/zero of=$file bs=1024 count=204800
+
+    yes | mkfs.$1 $file
+
+    if ! [ -z $ANDROID_WDIR ]; then
+        adb shell mkdir -p $ANDROID_WDIR
+        adb push $file $ANDROID_WDIR
+        rm $file
+        file=$ANDROID_WDIR/$(basename $file)
+    fi
+    if ! [ -z $BSD_WDIR ]; then
+        $MYSSH mkdir -p $BSD_WDIR
+        ssh_copy $file $BSD_WDIR
+        rm $file
+        file=$BSD_WDIR/$(basename $file)
+    fi
+
+    export_vars file
+}
+
+function cleanfs()
+{
+    set -e
+
+    if ! [ -z $ANDROID_WDIR ]; then
+        adb shell rm $1
+        adb shell rm $ANDROID_WDIR/disk
+    elif ! [ -z $BSD_WDIR ]; then
+        $MYSSH rm $1
+        $MYSSH rm $BSD_WDIR/disk
+    else
+        rm $1
+    fi
+}
+
+if [ "$1" = "-t" ]; then
+    shift
+    fstype=$1
+    shift
+fi
+
+if [ -z "$fstype" ]; then
+    fstype="ext4"
+fi
+
+if [ -z $(which mkfs.$fstype) ]; then
+    lkl_test_plan 0 "disk $fstype"
+    echo "no mkfs.$fstype command"
+    exit 0
+fi
+
+lkl_test_plan 1 "disk $fstype"
+lkl_test_run 1 prepfs $fstype
+lkl_test_exec $script_dir/disk -d $file -t $fstype $@
+lkl_test_plan 1 "disk $fstype"
+lkl_test_run 1 cleanfs $file
+
diff --git a/tools/um/tests/run.py b/tools/um/tests/run.py
index c96ede90b6ad..97d6dedc217c 100755
--- a/tools/um/tests/run.py
+++ b/tools/um/tests/run.py
@@ -50,6 +50,8 @@ mydir=os.path.dirname(os.path.realpath(__file__))
 
 tests = [
     'boot.sh',
+    'disk.sh -t ext4',
+    'disk.sh -t vfat',
 ]
 
 parser = argparse.ArgumentParser(description='LKL test runner')
-- 
2.21.0 (Apple Git-122.2)

^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v5 21/21] um: nommu: add block device support of UML
@ 2020-07-02 14:07       ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-07-02 14:07 UTC (permalink / raw)
  To: linux-um
  Cc: Octavian Purdila, linux-kernel-library, linux-arch,
	Hajime Tazaki, Akira Moroo

This commit adds block device support for nommu mode, and also added
several host utilities to run a simple test program
(tools/um/test/disk.c) successfully.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/include/asm/xor.h                 |   3 +-
 arch/um/include/shared/as-layout.h        |   1 +
 arch/um/kernel/um_arch.c                  |   5 +
 arch/um/nommu/include/uapi/asm/host_ops.h |   7 +
 arch/um/nommu/include/uapi/asm/unistd.h   |   2 +
 arch/um/nommu/um/Kconfig                  |   8 +
 arch/um/nommu/um/setup.c                  |  12 +
 tools/um/Targets                          |   2 +
 tools/um/include/lkl.h                    | 206 ++++++++++
 tools/um/include/lkl_host.h               |   1 +
 tools/um/lib/Build                        |   1 +
 tools/um/lib/fs.c                         | 461 ++++++++++++++++++++++
 tools/um/lib/posix-host.c                 |   1 +
 tools/um/lib/utils.c                      |   3 +
 tools/um/tests/Build                      |   1 +
 tools/um/tests/cla.c                      | 159 ++++++++
 tools/um/tests/cla.h                      |  33 ++
 tools/um/tests/disk.c                     | 168 ++++++++
 tools/um/tests/disk.sh                    |  70 ++++
 tools/um/tests/run.py                     |   2 +
 20 files changed, 1145 insertions(+), 1 deletion(-)
 create mode 100644 tools/um/lib/fs.c
 create mode 100644 tools/um/tests/cla.c
 create mode 100644 tools/um/tests/cla.h
 create mode 100644 tools/um/tests/disk.c
 create mode 100755 tools/um/tests/disk.sh

diff --git a/arch/um/include/asm/xor.h b/arch/um/include/asm/xor.h
index 36b33d62a35d..a5ea458a1ae9 100644
--- a/arch/um/include/asm/xor.h
+++ b/arch/um/include/asm/xor.h
@@ -4,4 +4,5 @@
 
 /* pick an arbitrary one - measuring isn't possible with inf-cpu */
 #define XOR_SELECT_TEMPLATE(x)	\
-	(time_travel_mode == TT_MODE_INFCPU ? &xor_block_8regs : NULL)
+	(time_travel_mode == TT_MODE_INFCPU || (!IS_ENABLED(CONFIG_MMU)) ? \
+	 &xor_block_8regs : NULL)
diff --git a/arch/um/include/shared/as-layout.h b/arch/um/include/shared/as-layout.h
index 5f286ef2721b..4423437a5ace 100644
--- a/arch/um/include/shared/as-layout.h
+++ b/arch/um/include/shared/as-layout.h
@@ -57,6 +57,7 @@ extern unsigned long host_task_size;
 
 extern int linux_main(int argc, char **argv);
 extern void uml_finishsetup(void);
+extern void uml_set_args(char *args);
 
 struct siginfo;
 extern void (*sig_info[])(int, struct siginfo *si, struct uml_pt_regs *);
diff --git a/arch/um/kernel/um_arch.c b/arch/um/kernel/um_arch.c
index 78d6042fd2e6..ed0149821518 100644
--- a/arch/um/kernel/um_arch.c
+++ b/arch/um/kernel/um_arch.c
@@ -42,6 +42,11 @@ static void __init add_arg(char *arg)
 	strcat(command_line, arg);
 }
 
+void __init uml_set_args(char *args)
+{
+	strcat(command_line, args);
+}
+
 /*
  * These fields are initialized at boot time and not changed.
  * XXX This structure is used only in the non-SMP case.  Maybe this
diff --git a/arch/um/nommu/include/uapi/asm/host_ops.h b/arch/um/nommu/include/uapi/asm/host_ops.h
index 0811494c080c..1b2507502969 100644
--- a/arch/um/nommu/include/uapi/asm/host_ops.h
+++ b/arch/um/nommu/include/uapi/asm/host_ops.h
@@ -15,6 +15,11 @@ struct lkl_jmp_buf {
  *
  * These operations must be provided by a host library or by the application
  * itself.
+ *
+ * @um_devices - string containg the list of UML devices in command line
+ * format. This string is appended to the kernel command line and
+ * is provided here for convenience to be implemented by the host library.
+ *
  * @print - optional operation that receives console messages
  * @panic - called during a kernel panic
  *
@@ -65,6 +70,8 @@ struct lkl_jmp_buf {
  *
  */
 struct lkl_host_operations {
+	const char *um_devices;
+
 	void (*print)(const char *str, int len);
 	void (*panic)(void);
 
diff --git a/arch/um/nommu/include/uapi/asm/unistd.h b/arch/um/nommu/include/uapi/asm/unistd.h
index 1762ebb829f5..320762099a62 100644
--- a/arch/um/nommu/include/uapi/asm/unistd.h
+++ b/arch/um/nommu/include/uapi/asm/unistd.h
@@ -3,6 +3,8 @@
 #define __UM_NOMMU_UAPI_UNISTD_H
 
 #define __ARCH_WANT_NEW_STAT
+#define __ARCH_WANT_SET_GET_RLIMIT
+
 #include <asm/bitsperlong.h>
 
 #if __BITS_PER_LONG == 64
diff --git a/arch/um/nommu/um/Kconfig b/arch/um/nommu/um/Kconfig
index 20b3eaccb6f0..c6a3f472fe75 100644
--- a/arch/um/nommu/um/Kconfig
+++ b/arch/um/nommu/um/Kconfig
@@ -4,6 +4,10 @@ config UML_NOMMU
 	select UACCESS_MEMCPY
 	select ARCH_THREAD_STACK_ALLOCATOR
 	select ARCH_HAS_SYSCALL_WRAPPER
+	select VFAT_FS
+	select NLS_CODEPAGE_437
+	select NLS_ISO8859_1
+	select BTRFS_FS
 
 config 64BIT
 	bool
@@ -35,3 +39,7 @@ config STACKTRACE_SUPPORT
 config PRINTK_TIME
 	bool
 	default y
+
+config RAID6_PQ_BENCHMARK
+	bool
+	default n
diff --git a/arch/um/nommu/um/setup.c b/arch/um/nommu/um/setup.c
index f74baea50bd3..6b49bf7a99fb 100644
--- a/arch/um/nommu/um/setup.c
+++ b/arch/um/nommu/um/setup.c
@@ -45,13 +45,25 @@ static void __init *lkl_run_kernel(void *arg)
 	return NULL;
 }
 
+static char _cmd_line[COMMAND_LINE_SIZE];
 int __init lkl_start_kernel(struct lkl_host_operations *ops,
 			    const char *fmt, ...)
 {
+	va_list ap;
 	int ret;
 
 	lkl_ops = ops;
 
+	va_start(ap, fmt);
+	ret = vsnprintf(_cmd_line, COMMAND_LINE_SIZE, fmt, ap);
+	va_end(ap);
+
+	if (ops->um_devices)
+		strscpy(_cmd_line + ret, ops->um_devices,
+			COMMAND_LINE_SIZE - ret);
+
+	uml_set_args(_cmd_line);
+
 	init_sem = lkl_ops->sem_alloc(0);
 	if (!init_sem)
 		return -ENOMEM;
diff --git a/tools/um/Targets b/tools/um/Targets
index 2bb90381843c..f5f8ec4b9dbb 100644
--- a/tools/um/Targets
+++ b/tools/um/Targets
@@ -1,10 +1,12 @@
 ifeq ($(UMMODE),library)
 progs-y += tests/boot
+progs-y += tests/disk
 else
 progs-y += uml/linux
 endif
 
 LDFLAGS_boot-y := -pie
+LDFLAGS_disk-y := -pie
 
 LDLIBS := -lrt -lpthread -lutil
 LDFLAGS_linux-y := -no-pie -Wl,--wrap,malloc -Wl,--wrap,free -Wl,--wrap,calloc
diff --git a/tools/um/include/lkl.h b/tools/um/include/lkl.h
index 707e01b64a70..4a9d874bfbf0 100644
--- a/tools/um/include/lkl.h
+++ b/tools/um/include/lkl.h
@@ -113,6 +113,16 @@ static inline long lkl_sys_creat(const char *file, int mode)
 }
 #endif
 
+#ifdef __lkl__NR_faccessat
+/**
+ * lkl_sys_access - wrapper for lkl_sys_faccessat
+ */
+static inline long lkl_sys_access(const char *file, int mode)
+{
+	return lkl_sys_faccessat(LKL_AT_FDCWD, file, mode);
+}
+#endif
+
 #ifdef __lkl__NR_mkdirat
 /**
  * lkl_sys_mkdir - wrapper for lkl_sys_mkdirat
@@ -123,6 +133,36 @@ static inline long lkl_sys_mkdir(const char *path, mode_t mode)
 }
 #endif
 
+#ifdef __lkl__NR_unlinkat
+/**
+ * lkl_sys_rmdir - wrapper for lkl_sys_unlinkrat
+ */
+static inline long lkl_sys_rmdir(const char *path)
+{
+	return lkl_sys_unlinkat(LKL_AT_FDCWD, path, LKL_AT_REMOVEDIR);
+}
+#endif
+
+#ifdef __lkl__NR_unlinkat
+/**
+ * lkl_sys_unlink - wrapper for lkl_sys_unlinkat
+ */
+static inline long lkl_sys_unlink(const char *path)
+{
+	return lkl_sys_unlinkat(LKL_AT_FDCWD, path, 0);
+}
+#endif
+
+#ifdef __lkl__NR_mknodat
+/**
+ * lkl_sys_mknod - wrapper for lkl_sys_mknodat
+ */
+static inline long lkl_sys_mknod(const char *path, mode_t mode, dev_t dev)
+{
+	return lkl_sys_mknodat(LKL_AT_FDCWD, path, mode, dev);
+}
+#endif
+
 #ifdef __lkl__NR_epoll_create1
 /**
  * lkl_sys_epoll_create - wrapper for lkl_sys_epoll_create1
@@ -144,6 +184,172 @@ static inline long lkl_sys_epoll_wait(int fd, struct lkl_epoll_event *ev,
 }
 #endif
 
+/**
+ * struct lkl_dev_blk_ops - block device host operations, defined in lkl_host.h.
+ */
+struct lkl_dev_blk_ops;
+
+/**
+ * lkl_disk - host disk handle
+ *
+ * @dev - a pointer to private information for this disk backend
+ * @fd - a POSIX file descriptor that can be used by preadv/pwritev
+ * @handle - an NT file handle that can be used by ReadFile/WriteFile
+ */
+struct lkl_disk {
+	void *dev;
+	union {
+		int fd;
+		void *handle;
+	};
+	struct lkl_dev_blk_ops *ops;
+};
+
+/**
+ * lkl_disk_add - add a new disk
+ *
+ * @disk - the host disk handle
+ * @returns a disk id (0 is valid) or a strictly negative value in case of error
+ */
+int lkl_disk_add(struct lkl_disk *disk);
+
+/**
+ * lkl_disk_remove - remove a disk
+ *
+ * This function makes a cleanup of the @disk's private information
+ * that was initialized by lkl_disk_add before.
+ *
+ * @disk - the host disk handle
+ */
+int lkl_disk_remove(struct lkl_disk disk);
+
+/**
+ * lkl_encode_dev_from_sysfs_blkdev - extract device id from sysfs
+ *
+ * This function returns the device id for the given sysfs dev node.
+ * The content of the node has to be in the form 'MAJOR:MINOR'.
+ * Also, this function expects an absolute path which means that sysfs
+ * already has to be mounted at the given path
+ *
+ * @sysfs_path - absolute path to the sysfs dev node
+ * @pdevid - pointer to memory where dev id will be returned
+ * @returns - 0 on success, a negative value on error
+ */
+int lkl_encode_dev_from_sysfs(const char *sysfs_path, uint32_t *pdevid);
+
+/**
+ * lkl_mount_dev - mount a disk
+ *
+ * This functions creates a device file for the given disk, creates a mount
+ * point and mounts the device over the mount point.
+ *
+ * @disk_id - the disk id identifying the disk to be mounted
+ * @part - disk partition or zero for full disk
+ * @fs_type - filesystem type
+ * @flags - mount flags
+ * @opts - additional filesystem specific mount options
+ * @mnt_str - a string that will be filled by this function with the path where
+ * the filesystem has been mounted
+ * @mnt_str_len - size of mnt_str
+ * @returns - 0 on success, a negative value on error
+ */
+long lkl_mount_dev(unsigned int disk_id, unsigned int part, const char *fs_type,
+		   int flags, const char *opts,
+		   char *mnt_str, unsigned int mnt_str_len);
+
+/**
+ * lkl_umount_dev - umount a disk
+ *
+ * This functions umounts the given disks and removes the device file and the
+ * mount point.
+ *
+ * @disk_id - the disk id identifying the disk to be mounted
+ * @part - disk partition or zero for full disk
+ * @flags - umount flags
+ * @timeout_ms - timeout to wait for the kernel to flush closed files so that
+ * umount can succeed
+ * @returns - 0 on success, a negative value on error
+ */
+long lkl_umount_dev(unsigned int disk_id, unsigned int part, int flags,
+		    long timeout_ms);
+
+/**
+ * lkl_umount_timeout - umount filesystem with timeout
+ *
+ * @path - the path to unmount
+ * @flags - umount flags
+ * @timeout_ms - timeout to wait for the kernel to flush closed files so that
+ * umount can succeed
+ * @returns - 0 on success, a negative value on error
+ */
+long lkl_umount_timeout(char *path, int flags, long timeout_ms);
+
+/**
+ * lkl_opendir - open a directory
+ *
+ * @path - directory path
+ * @err - pointer to store the error in case of failure
+ * @returns - a handle to be used when calling lkl_readdir
+ */
+struct lkl_dir *lkl_opendir(const char *path, int *err);
+
+/**
+ * lkl_fdopendir - open a directory
+ *
+ * @fd - file descriptor
+ * @err - pointer to store the error in case of failure
+ * @returns - a handle to be used when calling lkl_readdir
+ */
+struct lkl_dir *lkl_fdopendir(int fd, int *err);
+
+/**
+ * lkl_rewinddir - reset directory stream
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ */
+void lkl_rewinddir(struct lkl_dir *dir);
+
+/**
+ * lkl_closedir - close the directory
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ */
+int lkl_closedir(struct lkl_dir *dir);
+
+/**
+ * lkl_readdir - get the next available entry of the directory
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ * @returns - a lkl_dirent64 entry or NULL if the end of the directory stream is
+ * reached or if an error occurred; check lkl_errdir() to distinguish between
+ * errors or end of the directory stream
+ */
+struct lkl_linux_dirent64 *lkl_readdir(struct lkl_dir *dir);
+
+/**
+ * lkl_errdir - checks if an error occurred during the last lkl_readdir call
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ * @returns - 0 if no error occurred, or a negative value otherwise
+ */
+int lkl_errdir(struct lkl_dir *dir);
+
+/**
+ * lkl_dirfd - gets the file descriptor associated with the directory handle
+ *
+ * @dir - the directory handle as returned by lkl_opendir
+ * @returns - a positive value,which is the LKL file descriptor associated with
+ * the directory handle, or a negative value otherwise
+ */
+int lkl_dirfd(struct lkl_dir *dir);
+
+/**
+ * lkl_mount_fs - mount a file system type like proc, sys
+ * @fstype - file system type. e.g. proc, sys
+ * @returns - 0 on success. 1 if it's already mounted. negative on failure.
+ */
+int lkl_mount_fs(char *fstype);
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/tools/um/include/lkl_host.h b/tools/um/include/lkl_host.h
index 85e80eb4ad0d..12dd95616e43 100644
--- a/tools/um/include/lkl_host.h
+++ b/tools/um/include/lkl_host.h
@@ -10,6 +10,7 @@ extern "C" {
 #include <lkl.h>
 
 extern struct lkl_host_operations lkl_host_ops;
+extern char lkl_um_devs[4096];
 
 /**
  * lkl_printf - print a message via the host print operation
diff --git a/tools/um/lib/Build b/tools/um/lib/Build
index dddff26a3b4e..29491b40746c 100644
--- a/tools/um/lib/Build
+++ b/tools/um/lib/Build
@@ -4,3 +4,4 @@ CFLAGS_posix-host.o += -D_FILE_OFFSET_BITS=64
 liblinux-$(CONFIG_UMMODE_LIB) += utils.o
 liblinux-$(CONFIG_UMMODE_LIB) += posix-host.o
 liblinux-$(CONFIG_UMMODE_LIB) += jmp_buf.o
+liblinux-$(CONFIG_UMMODE_LIB) += fs.o
diff --git a/tools/um/lib/fs.c b/tools/um/lib/fs.c
new file mode 100644
index 000000000000..948aac9730c2
--- /dev/null
+++ b/tools/um/lib/fs.c
@@ -0,0 +1,461 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdarg.h>
+#include <stdio.h>
+#include <string.h>
+#include <stdlib.h>
+#include <lkl_host.h>
+
+#define MAX_FSTYPE_LEN 50
+
+static struct lkl_disk *lkl_disks[16];
+static int registered_blk_dev_idx;
+
+static int lkl_disk_um_add(struct lkl_disk *disk, const char *blkparams)
+{
+	/* concat strings */
+	snprintf(lkl_um_devs + strlen(lkl_um_devs), sizeof(lkl_um_devs),
+		 " ubd%d=%s", registered_blk_dev_idx, blkparams);
+
+	return registered_blk_dev_idx++;
+}
+
+int lkl_disk_add(struct lkl_disk *disk)
+{
+	int ret = -1;
+
+	ret = lkl_disk_um_add(disk, disk->dev);
+
+	lkl_disks[ret] = disk;
+
+	return ret;
+}
+
+int lkl_disk_remove(struct lkl_disk disk)
+{
+	/* FIXME */
+	return 0;
+}
+
+int lkl_mount_fs(char *fstype)
+{
+	char dir[MAX_FSTYPE_LEN+2] = "/";
+	int flags = 0, ret = 0;
+
+	strncat(dir, fstype, MAX_FSTYPE_LEN);
+
+	/* Create with regular umask */
+	ret = lkl_sys_mkdir(dir, 0xff);
+	if (ret && ret != -LKL_EEXIST) {
+		lkl_perror("mount_fs mkdir", ret);
+		return ret;
+	}
+
+	/* We have no use for nonzero flags right now */
+	ret = lkl_sys_mount("none", dir, fstype, flags, NULL);
+	if (ret && ret != -LKL_EBUSY) {
+		lkl_sys_rmdir(dir);
+		return ret;
+	}
+
+	if (ret == -LKL_EBUSY)
+		return 1;
+	return 0;
+}
+
+static uint32_t new_encode_dev(unsigned int major, unsigned int minor)
+{
+	return (minor & 0xff) | (major << 8) | ((minor & ~0xff) << 12);
+}
+
+static int startswith(const char *str, const char *pre)
+{
+	return strncmp(pre, str, strlen(pre)) == 0;
+}
+
+static int get_node_with_prefix(const char *path, const char *prefix,
+				char *result, unsigned int result_len)
+{
+	struct lkl_dir *dir = NULL;
+	struct lkl_linux_dirent64 *dirent;
+	int ret;
+
+	dir = lkl_opendir(path, &ret);
+	if (!dir)
+		return ret;
+
+	ret = -LKL_ENOENT;
+
+	while ((dirent = lkl_readdir(dir))) {
+		if (startswith(dirent->d_name, prefix)) {
+			if (strlen(dirent->d_name) + 1 > result_len) {
+				ret = -LKL_ENOMEM;
+				break;
+			}
+			memcpy(result, dirent->d_name, strlen(dirent->d_name));
+			result[strlen(dirent->d_name)] = '\0';
+			ret = 0;
+			break;
+		}
+	}
+
+	lkl_closedir(dir);
+
+	return ret;
+}
+
+int lkl_encode_dev_from_sysfs(const char *sysfs_path, uint32_t *pdevid)
+{
+	int ret;
+	long fd;
+	int major, minor;
+	char buf[16] = { 0, };
+	char *bufptr;
+
+	fd = lkl_sys_open(sysfs_path, LKL_O_RDONLY, 0);
+	if (fd < 0)
+		return fd;
+
+	ret = lkl_sys_read(fd, buf, sizeof(buf));
+	if (ret < 0)
+		goto out_close;
+
+	if (ret == sizeof(buf)) {
+		ret = -LKL_ENOBUFS;
+		goto out_close;
+	}
+
+	bufptr = strchr(buf, ':');
+	if (bufptr == NULL) {
+		ret = -LKL_EINVAL;
+		goto out_close;
+	}
+	bufptr[0] = '\0';
+	bufptr++;
+
+	major = atoi(buf);
+	minor = atoi(bufptr);
+
+	*pdevid = new_encode_dev(major, minor);
+	ret = 0;
+
+out_close:
+	lkl_sys_close(fd);
+
+	return ret;
+}
+
+#define SYSFS_DEV_UMBLK_CMDLINE_PATH \
+	"/sysfs/devices/platform/uml-blkdev.%d"
+
+struct abuf {
+	char *mem, *ptr;
+	unsigned int len;
+};
+
+static int snprintf_append(struct abuf *buf, const char *fmt, ...)
+{
+	int ret;
+	va_list args;
+
+	if (!buf->ptr)
+		buf->ptr = buf->mem;
+
+	va_start(args, fmt);
+	ret = vsnprintf(buf->ptr, buf->len - (buf->ptr - buf->mem), fmt, args);
+	va_end(args);
+
+	if (ret < 0 || (ret >= (int)(buf->len - (buf->ptr - buf->mem))))
+		return -LKL_ENOMEM;
+
+	buf->ptr += ret;
+
+	return 0;
+}
+
+static int __lkl_get_blkdev(int disk_id, unsigned int part, uint32_t *pdevid,
+			    const char *sysfs_path_fmt, const char *drv_prefix,
+			    const char *disk_prefix)
+{
+	char sysfs_path[LKL_PATH_MAX];
+	char drv_name[LKL_PATH_MAX];
+	char disk_name[LKL_PATH_MAX];
+	struct abuf sysfs_path_buf = {
+		.mem = sysfs_path,
+		.len = sizeof(sysfs_path),
+	};
+	int ret;
+
+	if (disk_id < 0)
+		return -LKL_EINVAL;
+
+	ret = lkl_mount_fs("sysfs");
+	if (ret < 0)
+		return ret;
+
+	ret = snprintf_append(&sysfs_path_buf, sysfs_path_fmt, disk_id);
+	if (ret)
+		return ret;
+
+	ret = get_node_with_prefix(sysfs_path, drv_prefix, drv_name,
+				   sizeof(drv_name));
+	if (ret)
+		return ret;
+
+	ret = snprintf_append(&sysfs_path_buf, "/%s/block", drv_name);
+	if (ret)
+		return ret;
+
+	ret = get_node_with_prefix(sysfs_path, disk_prefix, disk_name,
+				   sizeof(disk_name));
+	if (ret)
+		return ret;
+
+	if (!part)
+		ret = snprintf_append(&sysfs_path_buf, "/%s/dev", disk_name);
+	else
+		ret = snprintf_append(&sysfs_path_buf, "/%s/%s%d/dev",
+				      disk_name, disk_name, part);
+	if (ret)
+		return ret;
+
+	return lkl_encode_dev_from_sysfs(sysfs_path, pdevid);
+}
+
+int lkl_get_blkdev(int disk_id, unsigned int part, uint32_t *pdevid)
+{
+	char *fmt;
+
+	fmt = SYSFS_DEV_UMBLK_CMDLINE_PATH;
+	return __lkl_get_blkdev(disk_id, part, pdevid, fmt, "", "ubd");
+}
+
+long lkl_mount_dev(unsigned int disk_id, unsigned int part,
+		   const char *fs_type, int flags,
+		   const char *data, char *mnt_str, unsigned int mnt_str_len)
+{
+	char dev_str[] = { "/dev/xxxxxxxx" };
+	unsigned int dev;
+	int err;
+	char _data[4096]; /* FIXME: PAGE_SIZE is not exported by LKL */
+
+	if (mnt_str_len < sizeof(dev_str))
+		return -LKL_ENOMEM;
+
+	err = lkl_get_blkdev(disk_id, part, &dev);
+	if (err < 0)
+		return err;
+
+	snprintf(dev_str, sizeof(dev_str), "/dev/%08x", dev);
+	snprintf(mnt_str, mnt_str_len, "/mnt/%08x", dev);
+
+	err = lkl_sys_access("/dev", LKL_S_IRWXO);
+	if (err < 0) {
+		if (err == -LKL_ENOENT)
+			err = lkl_sys_mkdir("/dev", 0700);
+		if (err < 0)
+			return err;
+	}
+
+	err = lkl_sys_mknod(dev_str, LKL_S_IFBLK | 0600, dev);
+	if (err < 0)
+		return err;
+
+	err = lkl_sys_access("/mnt", LKL_S_IRWXO);
+	if (err < 0) {
+		if (err == -LKL_ENOENT)
+			err = lkl_sys_mkdir("/mnt", 0700);
+		if (err < 0)
+			return err;
+	}
+
+	err = lkl_sys_mkdir(mnt_str, 0700);
+	if (err < 0) {
+		lkl_sys_unlink(dev_str);
+		return err;
+	}
+
+	/* kernel always copies a full page */
+	if (data) {
+		strncpy(_data, data, sizeof(_data));
+		_data[sizeof(_data) - 1] = 0;
+	} else {
+		_data[0] = 0;
+	}
+
+	err = lkl_sys_mount(dev_str, mnt_str, (char *)fs_type, flags, _data);
+	if (err < 0) {
+		lkl_sys_unlink(dev_str);
+		lkl_sys_rmdir(mnt_str);
+		return err;
+	}
+
+	return 0;
+}
+
+long lkl_umount_timeout(char *path, int flags, long timeout_ms)
+{
+	long incr = 10000000; /* 10 ms */
+	struct lkl_timespec ts = {
+		.tv_sec = 0,
+		.tv_nsec = incr,
+	};
+	long err;
+
+	do {
+		err = lkl_sys_umount(path, flags);
+		if (err == -LKL_EBUSY) {
+			lkl_sys_nanosleep((struct __lkl__kernel_timespec *)&ts,
+					  NULL);
+			timeout_ms -= incr / 1000000;
+		}
+	} while (err == -LKL_EBUSY && timeout_ms > 0);
+
+	return err;
+}
+
+long lkl_umount_dev(unsigned int disk_id, unsigned int part, int flags,
+		    long timeout_ms)
+{
+	char dev_str[] = { "/dev/xxxxxxxx" };
+	char mnt_str[] = { "/mnt/xxxxxxxx" };
+	unsigned int dev;
+	int err;
+
+	err = lkl_get_blkdev(disk_id, part, &dev);
+	if (err < 0)
+		return err;
+
+	snprintf(dev_str, sizeof(dev_str), "/dev/%08x", dev);
+	snprintf(mnt_str, sizeof(mnt_str), "/mnt/%08x", dev);
+
+	err = lkl_umount_timeout(mnt_str, flags, timeout_ms);
+	if (err)
+		return err;
+
+	err = lkl_sys_unlink(dev_str);
+	if (err)
+		return err;
+
+	return lkl_sys_rmdir(mnt_str);
+}
+
+struct lkl_dir {
+	int fd;
+	char buf[1024];
+	char *pos;
+	int len;
+};
+
+static struct lkl_dir *lkl_dir_alloc(int *err)
+{
+	struct lkl_dir *dir = lkl_host_ops.mem_alloc(sizeof(struct lkl_dir));
+
+	if (!dir) {
+		*err = -LKL_ENOMEM;
+		return NULL;
+	}
+
+	dir->len = 0;
+	dir->pos = NULL;
+
+	return dir;
+}
+
+struct lkl_dir *lkl_opendir(const char *path, int *err)
+{
+	struct lkl_dir *dir = lkl_dir_alloc(err);
+
+	if (!dir) {
+		*err = -LKL_ENOMEM;
+		return NULL;
+	}
+
+	dir->fd = lkl_sys_open(path, LKL_O_RDONLY | LKL_O_DIRECTORY, 0);
+	if (dir->fd < 0) {
+		*err = dir->fd;
+		lkl_host_ops.mem_free(dir);
+		return NULL;
+	}
+
+	*err = 0;
+
+	return dir;
+}
+
+struct lkl_dir *lkl_fdopendir(int fd, int *err)
+{
+	struct lkl_dir *dir = lkl_dir_alloc(err);
+
+	if (!dir)
+		return NULL;
+
+	dir->fd = fd;
+
+	return dir;
+}
+
+void lkl_rewinddir(struct lkl_dir *dir)
+{
+	lkl_sys_lseek(dir->fd, 0, LKL_SEEK_SET);
+	dir->len = 0;
+	dir->pos = NULL;
+}
+
+int lkl_closedir(struct lkl_dir *dir)
+{
+	int ret;
+
+	ret = lkl_sys_close(dir->fd);
+	lkl_host_ops.mem_free(dir);
+
+	return ret;
+}
+
+struct lkl_linux_dirent64 *lkl_readdir(struct lkl_dir *dir)
+{
+	struct lkl_linux_dirent64 *de;
+
+	if (dir->len < 0)
+		return NULL;
+
+	if (!dir->pos || dir->pos - dir->buf >= dir->len)
+		goto read_buf;
+
+return_de:
+	de = (struct lkl_linux_dirent64 *)dir->pos;
+	dir->pos += de->d_reclen;
+
+	return de;
+
+read_buf:
+	dir->pos = NULL;
+	de = (struct lkl_linux_dirent64 *)dir->buf;
+	dir->len = lkl_sys_getdents64(dir->fd, de, sizeof(dir->buf));
+	if (dir->len <= 0)
+		return NULL;
+
+	dir->pos = dir->buf;
+	goto return_de;
+}
+
+int lkl_errdir(struct lkl_dir *dir)
+{
+	if (dir->len >= 0)
+		return 0;
+
+	return dir->len;
+}
+
+int lkl_dirfd(struct lkl_dir *dir)
+{
+	return dir->fd;
+}
+
+int lkl_set_fd_limit(unsigned int fd_limit)
+{
+	struct lkl_rlimit rlim = {
+		.rlim_cur = fd_limit,
+		.rlim_max = fd_limit,
+	};
+	return lkl_sys_setrlimit(LKL_RLIMIT_NOFILE, &rlim);
+}
diff --git a/tools/um/lib/posix-host.c b/tools/um/lib/posix-host.c
index b6b5b2902254..8fd88031bf2b 100644
--- a/tools/um/lib/posix-host.c
+++ b/tools/um/lib/posix-host.c
@@ -263,6 +263,7 @@ static void *tls_get(struct lkl_tls_key *key)
 }
 
 struct lkl_host_operations lkl_host_ops = {
+	.um_devices = lkl_um_devs,
 	.panic = panic,
 	.print = print,
 	.mem_alloc = (void *)malloc,
diff --git a/tools/um/lib/utils.c b/tools/um/lib/utils.c
index 4930479a8a35..ac65cd744a14 100644
--- a/tools/um/lib/utils.c
+++ b/tools/um/lib/utils.c
@@ -4,6 +4,9 @@
 #include <string.h>
 #include <lkl_host.h>
 
+/* XXX: find a better place */
+char lkl_um_devs[4096];
+
 static const char * const lkl_err_strings[] = {
 	"Success",
 	"Operation not permitted",
diff --git a/tools/um/tests/Build b/tools/um/tests/Build
index 26a607f90dc8..b62919cadc50 100644
--- a/tools/um/tests/Build
+++ b/tools/um/tests/Build
@@ -1 +1,2 @@
 boot-y += boot.o test.o
+disk-y += disk.o test.o cla.o
diff --git a/tools/um/tests/cla.c b/tools/um/tests/cla.c
new file mode 100644
index 000000000000..694e3a3822be
--- /dev/null
+++ b/tools/um/tests/cla.c
@@ -0,0 +1,159 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <string.h>
+#include <errno.h>
+#include <stdlib.h>
+#ifdef __MINGW32__
+#include <winsock2.h>
+#else
+#include <sys/socket.h>
+#include <netinet/in.h>
+#include <arpa/inet.h>
+#endif
+
+#include "cla.h"
+
+static int cl_arg_parse_bool(struct cl_arg *arg, const char *value)
+{
+	*((int *)arg->store) = 1;
+	return 0;
+}
+
+static int cl_arg_parse_str(struct cl_arg *arg, const char *value)
+{
+	*((const char **)arg->store) = value;
+	return 0;
+}
+
+static int cl_arg_parse_int(struct cl_arg *arg, const char *value)
+{
+	errno = 0;
+	*((int *)arg->store) = strtol(value, NULL, 0);
+	return errno == 0;
+}
+
+static int cl_arg_parse_str_set(struct cl_arg *arg, const char *value)
+{
+	const char **set = arg->set;
+	int i;
+
+	for (i = 0; set[i] != NULL; i++) {
+		if (strcmp(set[i], value) == 0) {
+			*((int *)arg->store) = i;
+			return 0;
+		}
+	}
+
+	return (-1);
+}
+
+static int cl_arg_parse_ipv4(struct cl_arg *arg, const char *value)
+{
+	unsigned int addr;
+
+	if (!value)
+		return (-1);
+
+	addr = inet_addr(value);
+	if (addr == INADDR_NONE)
+		return (-1);
+	*((unsigned int *)arg->store) = addr;
+	return 0;
+}
+
+static cl_arg_parser_t parsers[] = {
+	[CL_ARG_BOOL] = cl_arg_parse_bool,
+	[CL_ARG_INT] = cl_arg_parse_int,
+	[CL_ARG_STR] = cl_arg_parse_str,
+	[CL_ARG_STR_SET] = cl_arg_parse_str_set,
+	[CL_ARG_IPV4] = cl_arg_parse_ipv4,
+};
+
+static struct cl_arg *find_short_arg(char name, struct cl_arg *args)
+{
+	struct cl_arg *arg;
+
+	for (arg = args; arg->short_name != 0; arg++) {
+		if (arg->short_name == name)
+			return arg;
+	}
+
+	return NULL;
+}
+
+static struct cl_arg *find_long_arg(const char *name, struct cl_arg *args)
+{
+	struct cl_arg *arg;
+
+	for (arg = args; arg->long_name; arg++) {
+		if (strcmp(arg->long_name, name) == 0)
+			return arg;
+	}
+
+	return NULL;
+}
+
+static void print_help(struct cl_arg *args)
+{
+	struct cl_arg *arg;
+
+	fprintf(stderr, "usage:\n");
+	for (arg = args; arg->long_name; arg++) {
+		fprintf(stderr, "-%c, --%-20s %s", arg->short_name,
+			arg->long_name, arg->help);
+		if (arg->type == CL_ARG_STR_SET) {
+			const char **set = arg->set;
+
+			fprintf(stderr, " [ ");
+			while (*set != NULL)
+				fprintf(stderr, "%s ", *(set++));
+			fprintf(stderr, "]");
+		}
+		fprintf(stderr, "\n");
+	}
+}
+
+int cla_parse_args(int argc, const char **argv, struct cl_arg *args)
+{
+	int i;
+
+	for (i = 1; i < argc; i++) {
+		struct cl_arg *arg = NULL;
+		cl_arg_parser_t parser;
+
+		if (argv[i][0] == '-') {
+			if (argv[i][1] != '-')
+				arg = find_short_arg(argv[i][1], args);
+			else
+				arg = find_long_arg(&argv[i][2], args);
+		}
+
+		if (!arg) {
+			fprintf(stderr, "unknown option '%s'\n", argv[i]);
+			print_help(args);
+			return (-1);
+		}
+
+		if (arg->type == CL_ARG_USER || arg->type >= CL_ARG_END)
+			parser = arg->parser;
+		else
+			parser = parsers[arg->type];
+
+		if (!parser) {
+			fprintf(stderr, "can't parse --'%s'/-'%c'\n",
+				arg->long_name, args->short_name);
+			return (-1);
+		}
+
+		if (parser(arg, argv[i + 1]) < 0) {
+			fprintf(stderr, "can't parse '%s'\n", argv[i]);
+			print_help(args);
+			return (-1);
+		}
+
+		if (arg->has_arg)
+			i++;
+	}
+
+	return 0;
+}
diff --git a/tools/um/tests/cla.h b/tools/um/tests/cla.h
new file mode 100644
index 000000000000..3d879233681f
--- /dev/null
+++ b/tools/um/tests/cla.h
@@ -0,0 +1,33 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef _LKL_TEST_CLA_H
+#define _LKL_TEST_CLA_H
+
+enum cl_arg_type {
+	CL_ARG_USER = 0,
+	CL_ARG_BOOL,
+	CL_ARG_INT,
+	CL_ARG_STR,
+	CL_ARG_STR_SET,
+	CL_ARG_IPV4,
+	CL_ARG_END,
+};
+
+struct cl_arg;
+
+typedef int (*cl_arg_parser_t)(struct cl_arg *arg, const char *value);
+
+struct cl_arg {
+	const char *long_name;
+	char short_name;
+	const char *help;
+	int has_arg;
+	enum cl_arg_type type;
+	void *store;
+	void *set;
+	cl_arg_parser_t parser;
+};
+
+int cla_parse_args(int argc, const char **argv, struct cl_arg *args);
+
+
+#endif /* _LKL_TEST_CLA_H */
diff --git a/tools/um/tests/disk.c b/tools/um/tests/disk.c
new file mode 100644
index 000000000000..325934935e6a
--- /dev/null
+++ b/tools/um/tests/disk.c
@@ -0,0 +1,168 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <unistd.h>
+#include <string.h>
+#include <time.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <lkl.h>
+#include <lkl_host.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <sys/ioctl.h>
+
+#include "test.h"
+#include "cla.h"
+
+static struct {
+	int printk;
+	const char *disk;
+	const char *fstype;
+	int partition;
+} cla;
+
+struct cl_arg args[] = {
+	{"disk", 'd', "disk file to use", 1, CL_ARG_STR, &cla.disk},
+	{"partition", 'P', "partition to mount", 1, CL_ARG_INT, &cla.partition},
+	{"type", 't', "filesystem type", 1, CL_ARG_STR, &cla.fstype},
+	{0},
+};
+
+
+static struct lkl_disk disk;
+static int disk_id = -1;
+
+int lkl_test_disk_add(void)
+{
+	disk.fd = open(cla.disk, O_RDWR);
+	if (disk.fd < 0)
+		goto out_unlink;
+
+	disk.ops = NULL;
+	disk.dev = (char *)cla.disk;
+
+	disk_id = lkl_disk_add(&disk);
+	if (disk_id < 0)
+		goto out_close;
+
+	goto out;
+
+out_close:
+	close(disk.fd);
+
+out_unlink:
+	unlink(cla.disk);
+
+out:
+	lkl_test_logf("disk fd/handle %x disk_id %d", disk.fd, disk_id);
+
+	if (disk_id >= 0)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+int lkl_test_disk_remove(void)
+{
+	int ret;
+
+	ret = lkl_disk_remove(disk);
+
+	close(disk.fd);
+
+	if (ret == 0)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+
+static char mnt_point[32];
+
+LKL_TEST_CALL(mount_dev, lkl_mount_dev, 0, disk_id, cla.partition, cla.fstype,
+	      0, NULL, mnt_point, sizeof(mnt_point))
+
+static int lkl_test_umount_dev(void)
+{
+	long ret, ret2;
+
+	ret = lkl_sys_chdir("/");
+
+	ret2 = lkl_umount_dev(disk_id, cla.partition, 0, 1000);
+
+	lkl_test_logf("%ld %ld", ret, ret2);
+
+	if (!ret && !ret2)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+struct lkl_dir *dir;
+
+static int lkl_test_opendir(void)
+{
+	int err;
+
+	dir = lkl_opendir(mnt_point, &err);
+
+	lkl_test_logf("lkl_opedir(%s) = %d %s\n", mnt_point, err,
+		      lkl_strerror(err));
+
+	if (err == 0)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+static int lkl_test_readdir(void)
+{
+	struct lkl_linux_dirent64 *de = lkl_readdir(dir);
+	int wr = 0;
+
+	while (de) {
+		wr += lkl_test_logf("%s ", de->d_name);
+		if (wr >= 70) {
+			lkl_test_logf("\n");
+			wr = 0;
+			break;
+		}
+		de = lkl_readdir(dir);
+	}
+
+	if (lkl_errdir(dir) == 0)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+LKL_TEST_CALL(closedir, lkl_closedir, 0, dir);
+LKL_TEST_CALL(chdir_mnt_point, lkl_sys_chdir, 0, mnt_point);
+LKL_TEST_CALL(start_kernel, lkl_start_kernel, 0, &lkl_host_ops,
+	     "mem=16M loglevel=8");
+LKL_TEST_CALL(stop_kernel, lkl_sys_halt, 0);
+
+struct lkl_test tests[] = {
+	LKL_TEST(disk_add),
+	LKL_TEST(start_kernel),
+	LKL_TEST(mount_dev),
+	LKL_TEST(chdir_mnt_point),
+	LKL_TEST(opendir),
+	LKL_TEST(readdir),
+	LKL_TEST(closedir),
+	LKL_TEST(umount_dev),
+	LKL_TEST(stop_kernel),
+	LKL_TEST(disk_remove),
+
+};
+
+int main(int argc, const char **argv)
+{
+	if (cla_parse_args(argc, argv, args) < 0)
+		return (-1);
+
+	lkl_host_ops.print = lkl_test_log;
+
+	return lkl_test_run(tests, sizeof(tests)/sizeof(struct lkl_test),
+			    "disk %s", cla.fstype);
+}
diff --git a/tools/um/tests/disk.sh b/tools/um/tests/disk.sh
new file mode 100755
index 000000000000..e2ec6cf69d4b
--- /dev/null
+++ b/tools/um/tests/disk.sh
@@ -0,0 +1,70 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+
+source $script_dir/test.sh
+
+function prepfs()
+{
+    set -e
+
+    file=`mktemp`
+
+    dd if=/dev/zero of=$file bs=1024 count=204800
+
+    yes | mkfs.$1 $file
+
+    if ! [ -z $ANDROID_WDIR ]; then
+        adb shell mkdir -p $ANDROID_WDIR
+        adb push $file $ANDROID_WDIR
+        rm $file
+        file=$ANDROID_WDIR/$(basename $file)
+    fi
+    if ! [ -z $BSD_WDIR ]; then
+        $MYSSH mkdir -p $BSD_WDIR
+        ssh_copy $file $BSD_WDIR
+        rm $file
+        file=$BSD_WDIR/$(basename $file)
+    fi
+
+    export_vars file
+}
+
+function cleanfs()
+{
+    set -e
+
+    if ! [ -z $ANDROID_WDIR ]; then
+        adb shell rm $1
+        adb shell rm $ANDROID_WDIR/disk
+    elif ! [ -z $BSD_WDIR ]; then
+        $MYSSH rm $1
+        $MYSSH rm $BSD_WDIR/disk
+    else
+        rm $1
+    fi
+}
+
+if [ "$1" = "-t" ]; then
+    shift
+    fstype=$1
+    shift
+fi
+
+if [ -z "$fstype" ]; then
+    fstype="ext4"
+fi
+
+if [ -z $(which mkfs.$fstype) ]; then
+    lkl_test_plan 0 "disk $fstype"
+    echo "no mkfs.$fstype command"
+    exit 0
+fi
+
+lkl_test_plan 1 "disk $fstype"
+lkl_test_run 1 prepfs $fstype
+lkl_test_exec $script_dir/disk -d $file -t $fstype $@
+lkl_test_plan 1 "disk $fstype"
+lkl_test_run 1 cleanfs $file
+
diff --git a/tools/um/tests/run.py b/tools/um/tests/run.py
index c96ede90b6ad..97d6dedc217c 100755
--- a/tools/um/tests/run.py
+++ b/tools/um/tests/run.py
@@ -50,6 +50,8 @@ mydir=os.path.dirname(os.path.realpath(__file__))
 
 tests = [
     'boot.sh',
+    'disk.sh -t ext4',
+    'disk.sh -t vfat',
 ]
 
 parser = argparse.ArgumentParser(description='LKL test runner')
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* Re: [RFC v5 01/21] um: split build in kernel and host parts
  2020-07-02 14:06       ` Hajime Tazaki
@ 2020-09-21 16:01         ` Anton Ivanov
  -1 siblings, 0 replies; 476+ messages in thread
From: Anton Ivanov @ 2020-09-21 16:01 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um
  Cc: linux-arch, Octavian Purdila, Octavian Purdila, Akira Moroo,
	linux-kernel-library



On 02/07/2020 15:06, Hajime Tazaki wrote:
> From: Octavian Purdila <tavi@cs.pub.ro>
> 
> This patch splits the UML build in two parts: a kernel part that
> generates a relocatable object (linux.o) and a host build part that
> links the relocatable object into an executable.
> 
> This allows us to simplify the linker script since we can now remove
> the host bits (e.g. .rel sections). It also gives us greater
> flexibility since it will be much easier to support other
> architectures and OSes if we use the standard linker script.
> 
> The host build part has been implemented in tools/um so that we can
> reuse the available host build infrastructure.
> 
> The patch also changes the UML build invocation, if before
> 
>   $ make ARCH=um defconfig
>   $ make ARCH=um
> 
> was generating the executable now this only generates the relocatable
> object.
> 
> To fully build UML use:
> 
>   $ make -C tools/um
> 
> which will generate tools/lkl/linux as the UML executable. To
> statically compiled UML use:
> 
>   $ make -C tools/lkl UML_STATIC=y
> 
> Signed-off-by: Octavian Purdila <tavi@cs.pub.ro>
> Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
> ---
>   arch/um/Kconfig                  |  12 +--
>   arch/um/Makefile                 |  14 ++-
>   arch/um/include/asm/common.lds.S |  99 ------------------
>   arch/um/kernel/dyn.lds.S         | 171 -------------------------------
>   arch/um/kernel/uml.lds.S         | 115 ---------------------
>   arch/um/kernel/vmlinux.lds.S     |  91 ++++++++++++++--
>   scripts/link-vmlinux.sh          |  42 +++-----
>   tools/um/Makefile                |  72 +++++++++++++
>   tools/um/Targets                 |   4 +
>   tools/um/uml/Build               |   0
>   10 files changed, 184 insertions(+), 436 deletions(-)
>   delete mode 100644 arch/um/include/asm/common.lds.S
>   delete mode 100644 arch/um/kernel/dyn.lds.S
>   delete mode 100644 arch/um/kernel/uml.lds.S
>   create mode 100644 tools/um/Makefile
>   create mode 100644 tools/um/Targets
>   create mode 100644 tools/um/uml/Build
> 
> diff --git a/arch/um/Kconfig b/arch/um/Kconfig
> index 96ab7026b037..a0a9fd3ab96c 100644
> --- a/arch/um/Kconfig
> +++ b/arch/um/Kconfig
> @@ -20,6 +20,7 @@ config UML
>   	select GENERIC_CLOCKEVENTS
>   	select HAVE_GCC_PLUGINS
>   	select TTY # Needed for line.c
> +	select MODULE_REL_CRCS if MODVERSIONS
>   
>   config MMU
>   	bool
> @@ -79,17 +80,6 @@ config STATIC_LINK
>   	  NOTE: This option is incompatible with some networking features which
>   	  depend on features that require being dynamically loaded (like NSS).
>   
> -config LD_SCRIPT_STATIC
> -	bool
> -	default y
> -	depends on STATIC_LINK
> -
> -config LD_SCRIPT_DYN
> -	bool
> -	default y
> -	depends on !LD_SCRIPT_STATIC
> -	select MODULE_REL_CRCS if MODVERSIONS
> -
>   config HOSTFS
>   	tristate "Host filesystem"
>   	help
> diff --git a/arch/um/Makefile b/arch/um/Makefile
> index 275f5ffdf6f0..c952ddefcc1c 100644
> --- a/arch/um/Makefile
> +++ b/arch/um/Makefile
> @@ -95,14 +95,20 @@ KBUILD_CPPFLAGS += -I$(srctree)/$(HOST_DIR)/include \
>   KERNEL_DEFINES = $(strip -Derrno=kernel_errno -Dsigprocmask=kernel_sigprocmask \
>   			 -Dmktime=kernel_mktime $(ARCH_KERNEL_DEFINES))
>   KBUILD_CFLAGS += $(KERNEL_DEFINES)
> +LDFLAGS_vmlinux += -r
>   
> -PHONY += linux
> +PHONY += linux.o
>   
> -all: linux
> +all: linux.o
>   
> -linux: vmlinux
> +linux.o: vmlinux
>   	@echo '  LINK $@'
> -	$(Q)ln -f $< $@
> +	$(Q)$(OBJCOPY) -R .eh_frame $< $@
> +
> +install: linux.o
> +	@echo "  INSTALL $(INSTALL_PATH)/lib/$<"
> +	@mkdir -p $(INSTALL_PATH)/lib/
> +	@cp $< $(INSTALL_PATH)/lib/
>   
>   define archhelp
>     echo '* linux		- Binary kernel image (./linux) - for backward'
> diff --git a/arch/um/include/asm/common.lds.S b/arch/um/include/asm/common.lds.S
> deleted file mode 100644
> index eca6c452a41b..000000000000
> --- a/arch/um/include/asm/common.lds.S
> +++ /dev/null
> @@ -1,99 +0,0 @@
> -/* SPDX-License-Identifier: GPL-2.0 */
> -#include <asm-generic/vmlinux.lds.h>
> -
> -  .fini      : { *(.fini)    } =0x9090
> -  _etext = .;
> -  PROVIDE (etext = .);
> -
> -  . = ALIGN(4096);
> -  _sdata = .;
> -  PROVIDE (sdata = .);
> -
> -  RO_DATA(4096)
> -
> -  .unprotected : { *(.unprotected) }
> -  . = ALIGN(4096);
> -  PROVIDE (_unprotected_end = .);
> -
> -  . = ALIGN(4096);
> -  EXCEPTION_TABLE(0)
> -
> -  BUG_TABLE
> -
> -  .uml.setup.init : {
> -	__uml_setup_start = .;
> -	*(.uml.setup.init)
> -	__uml_setup_end = .;
> -  }
> -	
> -  .uml.help.init : {
> -	__uml_help_start = .;
> -	*(.uml.help.init)
> -	__uml_help_end = .;
> -  }
> -	
> -  .uml.postsetup.init : {
> -	__uml_postsetup_start = .;
> -	*(.uml.postsetup.init)
> -	__uml_postsetup_end = .;
> -  }
> -	
> -  .init.setup : {
> -	INIT_SETUP(0)
> -  }
> -
> -  PERCPU_SECTION(32)
> -	
> -  .initcall.init : {
> -	INIT_CALLS
> -  }
> -
> -  .con_initcall.init : {
> -	CON_INITCALL
> -  }
> -
> -  .exitcall : {
> -	__exitcall_begin = .;
> -	*(.exitcall.exit)
> -	__exitcall_end = .;
> -  }
> -
> -  .uml.exitcall : {
> -	__uml_exitcall_begin = .;
> -	*(.uml.exitcall.exit)
> -	__uml_exitcall_end = .;
> -  }
> -
> -  . = ALIGN(4);
> -  .altinstructions : {
> -	__alt_instructions = .;
> -	*(.altinstructions)
> -	__alt_instructions_end = .;
> -  }
> -  .altinstr_replacement : { *(.altinstr_replacement) }
> -  /* .exit.text is discard at runtime, not link time, to deal with references
> -     from .altinstructions and .eh_frame */
> -  .exit.text : { EXIT_TEXT }
> -  .exit.data : { *(.exit.data) }
> -
> -  .preinit_array : {
> -	__preinit_array_start = .;
> -	*(.preinit_array)
> -	__preinit_array_end = .;
> -  }
> -  .init_array : {
> -	__init_array_start = .;
> -	*(.init_array)
> -	__init_array_end = .;
> -  }
> -  .fini_array : {
> -	__fini_array_start = .;
> -	*(.fini_array)
> -	__fini_array_end = .;
> -  }
> -
> -   . = ALIGN(4096);
> -  .init.ramfs : {
> -	INIT_RAM_FS
> -  }
> -
> diff --git a/arch/um/kernel/dyn.lds.S b/arch/um/kernel/dyn.lds.S
> deleted file mode 100644
> index f5001481010c..000000000000
> --- a/arch/um/kernel/dyn.lds.S
> +++ /dev/null
> @@ -1,171 +0,0 @@
> -#include <asm/vmlinux.lds.h>
> -#include <asm/page.h>
> -
> -OUTPUT_FORMAT(ELF_FORMAT)
> -OUTPUT_ARCH(ELF_ARCH)
> -ENTRY(_start)
> -jiffies = jiffies_64;
> -
> -SECTIONS
> -{
> -  PROVIDE (__executable_start = START);
> -  . = START + SIZEOF_HEADERS;
> -  .interp         : { *(.interp) }
> -  __binary_start = .;
> -  . = ALIGN(4096);		/* Init code and data */
> -  _text = .;
> -  INIT_TEXT_SECTION(PAGE_SIZE)
> -
> -  . = ALIGN(PAGE_SIZE);
> -
> -  /* Read-only sections, merged into text segment: */
> -  .hash           : { *(.hash) }
> -  .gnu.hash       : { *(.gnu.hash) }
> -  .dynsym         : { *(.dynsym) }
> -  .dynstr         : { *(.dynstr) }
> -  .gnu.version    : { *(.gnu.version) }
> -  .gnu.version_d  : { *(.gnu.version_d) }
> -  .gnu.version_r  : { *(.gnu.version_r) }
> -  .rel.init       : { *(.rel.init) }
> -  .rela.init      : { *(.rela.init) }
> -  .rel.text       : { *(.rel.text .rel.text.* .rel.gnu.linkonce.t.*) }
> -  .rela.text      : { *(.rela.text .rela.text.* .rela.gnu.linkonce.t.*) }
> -  .rel.fini       : { *(.rel.fini) }
> -  .rela.fini      : { *(.rela.fini) }
> -  .rel.rodata     : { *(.rel.rodata .rel.rodata.* .rel.gnu.linkonce.r.*) }
> -  .rela.rodata    : { *(.rela.rodata .rela.rodata.* .rela.gnu.linkonce.r.*) }
> -  .rel.data       : { *(.rel.data .rel.data.* .rel.gnu.linkonce.d.*) }
> -  .rela.data      : { *(.rela.data .rela.data.* .rela.gnu.linkonce.d.*) }
> -  .rel.tdata	  : { *(.rel.tdata .rel.tdata.* .rel.gnu.linkonce.td.*) }
> -  .rela.tdata	  : { *(.rela.tdata .rela.tdata.* .rela.gnu.linkonce.td.*) }
> -  .rel.tbss	  : { *(.rel.tbss .rel.tbss.* .rel.gnu.linkonce.tb.*) }
> -  .rela.tbss	  : { *(.rela.tbss .rela.tbss.* .rela.gnu.linkonce.tb.*) }
> -  .rel.ctors      : { *(.rel.ctors) }
> -  .rela.ctors     : { *(.rela.ctors) }
> -  .rel.dtors      : { *(.rel.dtors) }
> -  .rela.dtors     : { *(.rela.dtors) }
> -  .rel.got        : { *(.rel.got) }
> -  .rela.got       : { *(.rela.got) }
> -  .rel.bss        : { *(.rel.bss .rel.bss.* .rel.gnu.linkonce.b.*) }
> -  .rela.bss       : { *(.rela.bss .rela.bss.* .rela.gnu.linkonce.b.*) }
> -  .rel.plt : {
> -	*(.rel.plt)
> -	PROVIDE_HIDDEN(__rel_iplt_start = .);
> -	*(.rel.iplt)
> -	PROVIDE_HIDDEN(__rel_iplt_end = .);
> -  }
> -  .rela.plt : {
> -	*(.rela.plt)
> -	PROVIDE_HIDDEN(__rela_iplt_start = .);
> -	*(.rela.iplt)
> -	PROVIDE_HIDDEN(__rela_iplt_end = .);
> -  }
> -  .init           : {
> -    KEEP (*(.init))
> -  } =0x90909090
> -  .plt            : { *(.plt) }
> -  .text           : {
> -    _stext = .;
> -    TEXT_TEXT
> -    SCHED_TEXT
> -    CPUIDLE_TEXT
> -    LOCK_TEXT
> -    IRQENTRY_TEXT
> -    SOFTIRQENTRY_TEXT
> -    *(.fixup)
> -    *(.stub .text.* .gnu.linkonce.t.*)
> -    /* .gnu.warning sections are handled specially by elf32.em.  */
> -    *(.gnu.warning)
> -
> -    . = ALIGN(PAGE_SIZE);
> -  } =0x90909090
> -  . = ALIGN(PAGE_SIZE);
> -  .syscall_stub : {
> -	__syscall_stub_start = .;
> -	*(.__syscall_stub*)
> -	__syscall_stub_end = .;
> -  }
> -  .fini           : {
> -    KEEP (*(.fini))
> -  } =0x90909090
> -
> -  .kstrtab : { *(.kstrtab) }
> -
> -  #include <asm/common.lds.S>
> -
> -  __init_begin = .;
> -  init.data : { INIT_DATA }
> -  __init_end = .;
> -
> -  /* Ensure the __preinit_array_start label is properly aligned.  We
> -     could instead move the label definition inside the section, but
> -     the linker would then create the section even if it turns out to
> -     be empty, which isn't pretty.  */
> -  . = ALIGN(32 / 8);
> -  .preinit_array     : { *(.preinit_array) }
> -  .init_array     : { *(.init_array) }
> -  .fini_array     : { *(.fini_array) }
> -  .data           : {
> -    INIT_TASK_DATA(KERNEL_STACK_SIZE)
> -    . = ALIGN(KERNEL_STACK_SIZE);
> -    *(.data..init_irqstack)
> -    DATA_DATA
> -    *(.data.* .gnu.linkonce.d.*)
> -    SORT(CONSTRUCTORS)
> -  }
> -  .data1          : { *(.data1) }
> -  .tdata	  : { *(.tdata .tdata.* .gnu.linkonce.td.*) }
> -  .tbss		  : { *(.tbss .tbss.* .gnu.linkonce.tb.*) *(.tcommon) }
> -  .eh_frame       : { KEEP (*(.eh_frame)) }
> -  .gcc_except_table   : { *(.gcc_except_table) }
> -  .dynamic        : { *(.dynamic) }
> -  .ctors          : {
> -    /* gcc uses crtbegin.o to find the start of
> -       the constructors, so we make sure it is
> -       first.  Because this is a wildcard, it
> -       doesn't matter if the user does not
> -       actually link against crtbegin.o; the
> -       linker won't look for a file to match a
> -       wildcard.  The wildcard also means that it
> -       doesn't matter which directory crtbegin.o
> -       is in.  */
> -    KEEP (*crtbegin.o(.ctors))
> -    /* We don't want to include the .ctor section from
> -       from the crtend.o file until after the sorted ctors.
> -       The .ctor section from the crtend file contains the
> -       end of ctors marker and it must be last */
> -    KEEP (*(EXCLUDE_FILE (*crtend.o ) .ctors))
> -    KEEP (*(SORT(.ctors.*)))
> -    KEEP (*(.ctors))
> -  }
> -  .dtors          : {
> -    KEEP (*crtbegin.o(.dtors))
> -    KEEP (*(EXCLUDE_FILE (*crtend.o ) .dtors))
> -    KEEP (*(SORT(.dtors.*)))
> -    KEEP (*(.dtors))
> -  }
> -  .jcr            : { KEEP (*(.jcr)) }
> -  .got            : { *(.got.plt) *(.got) }
> -  _edata = .;
> -  PROVIDE (edata = .);
> -  .bss            : {
> -   __bss_start = .;
> -   *(.dynbss)
> -   *(.bss .bss.* .gnu.linkonce.b.*)
> -   *(COMMON)
> -   /* Align here to ensure that the .bss section occupies space up to
> -      _end.  Align after .bss to ensure correct alignment even if the
> -      .bss section disappears because there are no input sections.  */
> -   . = ALIGN(32 / 8);
> -  . = ALIGN(32 / 8);
> -  }
> -   __bss_stop = .;
> -  _end = .;
> -  PROVIDE (end = .);
> -
> -  STABS_DEBUG
> -
> -  DWARF_DEBUG
> -
> -  DISCARDS
> -}
> diff --git a/arch/um/kernel/uml.lds.S b/arch/um/kernel/uml.lds.S
> deleted file mode 100644
> index 3b6dab3d4501..000000000000
> --- a/arch/um/kernel/uml.lds.S
> +++ /dev/null
> @@ -1,115 +0,0 @@
> -/* SPDX-License-Identifier: GPL-2.0 */
> -#include <asm/vmlinux.lds.h>
> -#include <asm/page.h>
> -
> -OUTPUT_FORMAT(ELF_FORMAT)
> -OUTPUT_ARCH(ELF_ARCH)
> -ENTRY(_start)
> -jiffies = jiffies_64;
> -
> -SECTIONS
> -{
> -  /* This must contain the right address - not quite the default ELF one.*/
> -  PROVIDE (__executable_start = START);
> -  /* Static binaries stick stuff here, like the sigreturn trampoline,
> -   * invisibly to objdump.  So, just make __binary_start equal to the very
> -   * beginning of the executable, and if there are unmapped pages after this,
> -   * they are forever unusable.
> -   */
> -  __binary_start = START;
> -
> -  . = START + SIZEOF_HEADERS;
> -  . = ALIGN(PAGE_SIZE);
> -
> -  _text = .;
> -  INIT_TEXT_SECTION(0)
> -
> -  .text      :
> -  {
> -    _stext = .;
> -    TEXT_TEXT
> -    SCHED_TEXT
> -    CPUIDLE_TEXT
> -    LOCK_TEXT
> -    IRQENTRY_TEXT
> -    SOFTIRQENTRY_TEXT
> -    *(.fixup)
> -    /* .gnu.warning sections are handled specially by elf32.em.  */
> -    *(.gnu.warning)
> -    *(.gnu.linkonce.t*)
> -  }
> -
> -  . = ALIGN(PAGE_SIZE);
> -  .syscall_stub : {
> -	__syscall_stub_start = .;
> -	*(.__syscall_stub*)
> -	__syscall_stub_end = .;
> -  }
> -
> -  /*
> -   * These are needed even in a static link, even if they wind up being empty.
> -   * Newer glibc needs these __rel{,a}_iplt_{start,end} symbols.
> -   */
> -  .rel.plt : {
> -	*(.rel.plt)
> -	PROVIDE_HIDDEN(__rel_iplt_start = .);
> -	*(.rel.iplt)
> -	PROVIDE_HIDDEN(__rel_iplt_end = .);
> -  }
> -  .rela.plt : {
> -	*(.rela.plt)
> -	PROVIDE_HIDDEN(__rela_iplt_start = .);
> -	*(.rela.iplt)
> -	PROVIDE_HIDDEN(__rela_iplt_end = .);
> -  }
> -
> -  #include <asm/common.lds.S>
> -
> -  __init_begin = .;
> -  init.data : { INIT_DATA }
> -  __init_end = .;
> -
> -  .data    :
> -  {
> -    INIT_TASK_DATA(KERNEL_STACK_SIZE)
> -    . = ALIGN(KERNEL_STACK_SIZE);
> -    *(.data..init_irqstack)
> -    DATA_DATA
> -    *(.gnu.linkonce.d*)
> -    CONSTRUCTORS
> -  }
> -  .data1   : { *(.data1) }
> -  .ctors         :
> -  {
> -    *(.ctors)
> -  }
> -  .dtors         :
> -  {
> -    *(.dtors)
> -  }
> -
> -  .got           : { *(.got.plt) *(.got) }
> -  .dynamic       : { *(.dynamic) }
> -  .tdata	  : { *(.tdata .tdata.* .gnu.linkonce.td.*) }
> -  .tbss		  : { *(.tbss .tbss.* .gnu.linkonce.tb.*) *(.tcommon) }
> -  /* We want the small data sections together, so single-instruction offsets
> -     can access them all, and initialized data all before uninitialized, so
> -     we can shorten the on-disk segment size.  */
> -  .sdata     : { *(.sdata) }
> -  _edata  =  .;
> -  PROVIDE (edata = .);
> -  . = ALIGN(PAGE_SIZE);
> -  __bss_start = .;
> -  PROVIDE(_bss_start = .);
> -  SBSS(0)
> -  BSS(0)
> -   __bss_stop = .;
> -  _end = .;
> -  PROVIDE (end = .);
> -
> -  STABS_DEBUG
> -
> -  DWARF_DEBUG
> -
> -  DISCARDS
> -}
> diff --git a/arch/um/kernel/vmlinux.lds.S b/arch/um/kernel/vmlinux.lds.S
> index 16e49bfa2b42..aaedd66772fa 100644
> --- a/arch/um/kernel/vmlinux.lds.S
> +++ b/arch/um/kernel/vmlinux.lds.S
> @@ -1,8 +1,87 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#include <asm/vmlinux.lds.h>
> +#include <asm/thread_info.h>
> +#include <asm/page.h>
> +#include <asm/thread_info.h>
> +#include <asm/cache.h>
> +#include <linux/export.h>
>   
> -KERNEL_STACK_SIZE = 4096 * (1 << CONFIG_KERNEL_STACK_ORDER);
> +OUTPUT_FORMAT(ELF_FORMAT)
> +jiffies = jiffies_64;
>   
> -#ifdef CONFIG_LD_SCRIPT_STATIC
> -#include "uml.lds.S"
> -#else
> -#include "dyn.lds.S"
> -#endif
> +
> +SECTIONS
> +{
> +	__init_begin = . ;
> +	HEAD_TEXT_SECTION
> +	INIT_TEXT_SECTION(PAGE_SIZE)
> +	INIT_DATA_SECTION(16)
> +	PERCPU_SECTION(L1_CACHE_BYTES)
> +	__init_end = . ;
> +
> +	_stext = . ;
> +	_text = . ;
> +	.text : ALIGN(THREAD_SIZE)
> +	{
> +		__binary_start = . ;
> +		TEXT_TEXT
> +		SCHED_TEXT
> +		LOCK_TEXT
> +		CPUIDLE_TEXT
> +		IRQENTRY_TEXT
> +		SOFTIRQENTRY_TEXT
> +	}
> +	_etext = . ;
> +
> +	.syscall_stub : ALIGN(PAGE_SIZE)
> +	{
> +		__syscall_stub_start = .;
> +		*(.__syscall_stub*)
> +		__syscall_stub_end = .;
> +	}
> +
> +	_sdata = . ;
> +	RO_DATA(PAGE_SIZE)
> +	RW_DATA(L1_CACHE_BYTES, PAGE_SIZE, THREAD_SIZE)
> +	.data : ALIGN(THREAD_SIZE)  { }
> +
> +	EXCEPTION_TABLE(16)
> +
> +	.uml.init_irqstack : ALIGN(THREAD_SIZE) {
> +		*(.data..init_irqstack)
> +	}
> +
> +	.uml.setup.init : ALIGN(8) {
> +		__uml_setup_start = .;
> +		*(.uml.setup.init)
> +		__uml_setup_end = .;
> +	}
> +
> +	.uml.help.init : ALIGN(8)  {
> +		__uml_help_start = .;
> +		*(.uml.help.init)
> +		__uml_help_end = .;
> +	}
> +
> +	.uml.postsetup.init : ALIGN(8) {
> +		__uml_postsetup_start = .;
> +		*(.uml.postsetup.init)
> +		__uml_postsetup_end = .;
> +	}
> +
> +	.uml.exitcall : ALIGN(8) {
> +		__uml_exitcall_begin = .;
> +		*(.uml.exitcall.exit)
> +		__uml_exitcall_end = .;
> +	}
> +	_edata = . ;
> +
> +	BSS_SECTION(0, 0, 0)
> +	_end = . ;
> +
> +	STABS_DEBUG
> +
> +	DWARF_DEBUG
> +
> +	DISCARDS
> +}
> diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
> index d09ab4afbda4..0bbb626ace81 100755
> --- a/scripts/link-vmlinux.sh
> +++ b/scripts/link-vmlinux.sh
> @@ -75,36 +75,18 @@ vmlinux_link()
>   		strip_debug=-Wl,--strip-debug
>   	fi
>   
> -	if [ "${SRCARCH}" != "um" ]; then
> -		objects="--whole-archive			\
> -			${KBUILD_VMLINUX_OBJS}			\
> -			--no-whole-archive			\
> -			--start-group				\
> -			${KBUILD_VMLINUX_LIBS}			\
> -			--end-group				\
> -			${@}"
> -
> -		${LD} ${KBUILD_LDFLAGS} ${LDFLAGS_vmlinux}	\
> -			${strip_debug#-Wl,}			\
> -			-o ${output}				\
> -			-T ${lds} ${objects}
> -	else
> -		objects="-Wl,--whole-archive			\
> -			${KBUILD_VMLINUX_OBJS}			\
> -			-Wl,--no-whole-archive			\
> -			-Wl,--start-group			\
> -			${KBUILD_VMLINUX_LIBS}			\
> -			-Wl,--end-group				\
> -			${@}"
> -
> -		${CC} ${CFLAGS_vmlinux}				\
> -			${strip_debug}				\
> -			-o ${output}				\
> -			-Wl,-T,${lds}				\
> -			${objects}				\
> -			-lutil -lrt -lpthread
> -		rm -f linux
> -	fi
> +	objects="--whole-archive			\
> +		${KBUILD_VMLINUX_OBJS}			\
> +		--no-whole-archive			\
> +		--start-group				\
> +		${KBUILD_VMLINUX_LIBS}			\
> +		--end-group				\
> +		${@}"
> +
> +	${LD} ${KBUILD_LDFLAGS} ${LDFLAGS_vmlinux}	\
> +		${strip_debug#-Wl,}			\
> +		-o ${output}				\
> +		-T ${lds} ${objects}
>   }
>   
>   # generate .BTF typeinfo from DWARF debuginfo
> diff --git a/tools/um/Makefile b/tools/um/Makefile
> new file mode 100644
> index 000000000000..a63466ef7ef5
> --- /dev/null
> +++ b/tools/um/Makefile
> @@ -0,0 +1,72 @@
> +# Do not use make's built-in rules
> +# (this improves performance and avoids hard-to-debug behaviour);
> +# also do not print "Entering directory..." messages from make
> +.SUFFIXES:
> +MAKEFLAGS += -r --no-print-directory
> +
> +KCONFIG?=defconfig
> +
> +ifneq ($(silent),1)
> +  ifneq ($(V),1)
> +	QUIET_AUTOCONF       = @echo '  AUTOCONF '$@;
> +	Q = @
> +  endif
> +endif
> +
> +PREFIX   := /usr
> +
> +ifeq (,$(srctree))
> +  srctree := $(patsubst %/,%,$(dir $(shell pwd)))
> +  srctree := $(patsubst %/,%,$(dir $(srctree)))
> +endif
> +export srctree
> +
> +-include ../scripts/Makefile.include
> +
> +# OUTPUT fixup should be *after* include ../scripts/Makefile.include
> +ifneq ($(OUTPUT),)
> +  OUTPUT := $(OUTPUT)/tools/um/
> +else
> +  OUTPUT := $(CURDIR)/
> +endif
> +export OUTPUT
> +export objtree := $(OUTPUT)/../..
> +
> +
> +all:
> +
> +export CFLAGS += -I$(OUTPUT)/include -Iinclude -Wall -g -O2 -Wextra -fPIC \
> +	 -Wno-unused-parameter \
> +	 -Wno-missing-field-initializers -fno-strict-aliasing
> +
> +-include Targets
> +
> +TARGETS := $(progs-y:%=$(OUTPUT)%)
> +TARGETS += $(libs-y:%=$(OUTPUT)%)
> +all: $(TARGETS)
> +
> +# rule to build linux.o
> +$(OUTPUT)lib/linux.o:
> +	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) $(KCONFIG)
> +	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) install INSTALL_PATH=$(OUTPUT)
> +
> +$(OUTPUT)liblinux.a: $(OUTPUT)lib/linux.o $(OUTPUT)uml/liblinux-in.o
> +	$(QUIET_AR)$(AR) -rc $@ $^
> +
> +# rule to link programs
> +$(OUTPUT)%: $(OUTPUT)%-in.o $(OUTPUT)liblinux.a
> +	$(QUIET_LINK)$(CC) $(LDFLAGS) $(LDFLAGS_$(notdir $*)-y) -o $@ $^ $(LDLIBS) $(LDLIBS_$(notdir $*)-y)
> +
> +$(OUTPUT)%-in.o: FORCE
> +	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
> +
> +clean:
> +	$(call QUIET_CLEAN, objects)find $(OUTPUT) -name '*.o' -delete -o -name '\.*.cmd'\
> +	 -delete -o -name '\.*.d' -delete
> +	$(call QUIET_CLEAN, liblinux.a)$(RM) $(OUTPUT)/liblinux.a
> +	$(call QUIET_CLEAN, $(TARGETS))$(RM) $(TARGETS)
> +
> +FORCE: ;
> +.PHONY: all clean FORCE
> +.IGNORE: clean
> +.NOTPARALLEL : lib/linux.o
> diff --git a/tools/um/Targets b/tools/um/Targets
> new file mode 100644
> index 000000000000..a4711f1ef422
> --- /dev/null
> +++ b/tools/um/Targets
> @@ -0,0 +1,4 @@
> +progs-y += uml/linux
> +LDLIBS_linux-y := -lrt -lpthread -lutil
> +LDFLAGS_linux-y := -no-pie -Wl,--wrap,malloc -Wl,--wrap,free -Wl,--wrap,calloc
> +LDFLAGS_linux-$(UML_STATIC) += -static
> diff --git a/tools/um/uml/Build b/tools/um/uml/Build
> new file mode 100644
> index 000000000000..e69de29bb2d1
> 

Hi Hajime, hi Octavian,

I am going to have a bit more time to look at UML now. Can you rebase/update the patch versus the latest kernel so I can look at it.

It does not apply at present.

Thanks in advance,

-- 
Anton R. Ivanov
Cambridgegreys Limited. Registered in England. Company Number 10273661
https://www.cambridgegreys.com/

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v5 01/21] um: split build in kernel and host parts
@ 2020-09-21 16:01         ` Anton Ivanov
  0 siblings, 0 replies; 476+ messages in thread
From: Anton Ivanov @ 2020-09-21 16:01 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um
  Cc: linux-arch, Octavian Purdila, linux-kernel-library,
	Octavian Purdila, Akira Moroo



On 02/07/2020 15:06, Hajime Tazaki wrote:
> From: Octavian Purdila <tavi@cs.pub.ro>
> 
> This patch splits the UML build in two parts: a kernel part that
> generates a relocatable object (linux.o) and a host build part that
> links the relocatable object into an executable.
> 
> This allows us to simplify the linker script since we can now remove
> the host bits (e.g. .rel sections). It also gives us greater
> flexibility since it will be much easier to support other
> architectures and OSes if we use the standard linker script.
> 
> The host build part has been implemented in tools/um so that we can
> reuse the available host build infrastructure.
> 
> The patch also changes the UML build invocation, if before
> 
>   $ make ARCH=um defconfig
>   $ make ARCH=um
> 
> was generating the executable now this only generates the relocatable
> object.
> 
> To fully build UML use:
> 
>   $ make -C tools/um
> 
> which will generate tools/lkl/linux as the UML executable. To
> statically compiled UML use:
> 
>   $ make -C tools/lkl UML_STATIC=y
> 
> Signed-off-by: Octavian Purdila <tavi@cs.pub.ro>
> Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
> ---
>   arch/um/Kconfig                  |  12 +--
>   arch/um/Makefile                 |  14 ++-
>   arch/um/include/asm/common.lds.S |  99 ------------------
>   arch/um/kernel/dyn.lds.S         | 171 -------------------------------
>   arch/um/kernel/uml.lds.S         | 115 ---------------------
>   arch/um/kernel/vmlinux.lds.S     |  91 ++++++++++++++--
>   scripts/link-vmlinux.sh          |  42 +++-----
>   tools/um/Makefile                |  72 +++++++++++++
>   tools/um/Targets                 |   4 +
>   tools/um/uml/Build               |   0
>   10 files changed, 184 insertions(+), 436 deletions(-)
>   delete mode 100644 arch/um/include/asm/common.lds.S
>   delete mode 100644 arch/um/kernel/dyn.lds.S
>   delete mode 100644 arch/um/kernel/uml.lds.S
>   create mode 100644 tools/um/Makefile
>   create mode 100644 tools/um/Targets
>   create mode 100644 tools/um/uml/Build
> 
> diff --git a/arch/um/Kconfig b/arch/um/Kconfig
> index 96ab7026b037..a0a9fd3ab96c 100644
> --- a/arch/um/Kconfig
> +++ b/arch/um/Kconfig
> @@ -20,6 +20,7 @@ config UML
>   	select GENERIC_CLOCKEVENTS
>   	select HAVE_GCC_PLUGINS
>   	select TTY # Needed for line.c
> +	select MODULE_REL_CRCS if MODVERSIONS
>   
>   config MMU
>   	bool
> @@ -79,17 +80,6 @@ config STATIC_LINK
>   	  NOTE: This option is incompatible with some networking features which
>   	  depend on features that require being dynamically loaded (like NSS).
>   
> -config LD_SCRIPT_STATIC
> -	bool
> -	default y
> -	depends on STATIC_LINK
> -
> -config LD_SCRIPT_DYN
> -	bool
> -	default y
> -	depends on !LD_SCRIPT_STATIC
> -	select MODULE_REL_CRCS if MODVERSIONS
> -
>   config HOSTFS
>   	tristate "Host filesystem"
>   	help
> diff --git a/arch/um/Makefile b/arch/um/Makefile
> index 275f5ffdf6f0..c952ddefcc1c 100644
> --- a/arch/um/Makefile
> +++ b/arch/um/Makefile
> @@ -95,14 +95,20 @@ KBUILD_CPPFLAGS += -I$(srctree)/$(HOST_DIR)/include \
>   KERNEL_DEFINES = $(strip -Derrno=kernel_errno -Dsigprocmask=kernel_sigprocmask \
>   			 -Dmktime=kernel_mktime $(ARCH_KERNEL_DEFINES))
>   KBUILD_CFLAGS += $(KERNEL_DEFINES)
> +LDFLAGS_vmlinux += -r
>   
> -PHONY += linux
> +PHONY += linux.o
>   
> -all: linux
> +all: linux.o
>   
> -linux: vmlinux
> +linux.o: vmlinux
>   	@echo '  LINK $@'
> -	$(Q)ln -f $< $@
> +	$(Q)$(OBJCOPY) -R .eh_frame $< $@
> +
> +install: linux.o
> +	@echo "  INSTALL $(INSTALL_PATH)/lib/$<"
> +	@mkdir -p $(INSTALL_PATH)/lib/
> +	@cp $< $(INSTALL_PATH)/lib/
>   
>   define archhelp
>     echo '* linux		- Binary kernel image (./linux) - for backward'
> diff --git a/arch/um/include/asm/common.lds.S b/arch/um/include/asm/common.lds.S
> deleted file mode 100644
> index eca6c452a41b..000000000000
> --- a/arch/um/include/asm/common.lds.S
> +++ /dev/null
> @@ -1,99 +0,0 @@
> -/* SPDX-License-Identifier: GPL-2.0 */
> -#include <asm-generic/vmlinux.lds.h>
> -
> -  .fini      : { *(.fini)    } =0x9090
> -  _etext = .;
> -  PROVIDE (etext = .);
> -
> -  . = ALIGN(4096);
> -  _sdata = .;
> -  PROVIDE (sdata = .);
> -
> -  RO_DATA(4096)
> -
> -  .unprotected : { *(.unprotected) }
> -  . = ALIGN(4096);
> -  PROVIDE (_unprotected_end = .);
> -
> -  . = ALIGN(4096);
> -  EXCEPTION_TABLE(0)
> -
> -  BUG_TABLE
> -
> -  .uml.setup.init : {
> -	__uml_setup_start = .;
> -	*(.uml.setup.init)
> -	__uml_setup_end = .;
> -  }
> -	
> -  .uml.help.init : {
> -	__uml_help_start = .;
> -	*(.uml.help.init)
> -	__uml_help_end = .;
> -  }
> -	
> -  .uml.postsetup.init : {
> -	__uml_postsetup_start = .;
> -	*(.uml.postsetup.init)
> -	__uml_postsetup_end = .;
> -  }
> -	
> -  .init.setup : {
> -	INIT_SETUP(0)
> -  }
> -
> -  PERCPU_SECTION(32)
> -	
> -  .initcall.init : {
> -	INIT_CALLS
> -  }
> -
> -  .con_initcall.init : {
> -	CON_INITCALL
> -  }
> -
> -  .exitcall : {
> -	__exitcall_begin = .;
> -	*(.exitcall.exit)
> -	__exitcall_end = .;
> -  }
> -
> -  .uml.exitcall : {
> -	__uml_exitcall_begin = .;
> -	*(.uml.exitcall.exit)
> -	__uml_exitcall_end = .;
> -  }
> -
> -  . = ALIGN(4);
> -  .altinstructions : {
> -	__alt_instructions = .;
> -	*(.altinstructions)
> -	__alt_instructions_end = .;
> -  }
> -  .altinstr_replacement : { *(.altinstr_replacement) }
> -  /* .exit.text is discard at runtime, not link time, to deal with references
> -     from .altinstructions and .eh_frame */
> -  .exit.text : { EXIT_TEXT }
> -  .exit.data : { *(.exit.data) }
> -
> -  .preinit_array : {
> -	__preinit_array_start = .;
> -	*(.preinit_array)
> -	__preinit_array_end = .;
> -  }
> -  .init_array : {
> -	__init_array_start = .;
> -	*(.init_array)
> -	__init_array_end = .;
> -  }
> -  .fini_array : {
> -	__fini_array_start = .;
> -	*(.fini_array)
> -	__fini_array_end = .;
> -  }
> -
> -   . = ALIGN(4096);
> -  .init.ramfs : {
> -	INIT_RAM_FS
> -  }
> -
> diff --git a/arch/um/kernel/dyn.lds.S b/arch/um/kernel/dyn.lds.S
> deleted file mode 100644
> index f5001481010c..000000000000
> --- a/arch/um/kernel/dyn.lds.S
> +++ /dev/null
> @@ -1,171 +0,0 @@
> -#include <asm/vmlinux.lds.h>
> -#include <asm/page.h>
> -
> -OUTPUT_FORMAT(ELF_FORMAT)
> -OUTPUT_ARCH(ELF_ARCH)
> -ENTRY(_start)
> -jiffies = jiffies_64;
> -
> -SECTIONS
> -{
> -  PROVIDE (__executable_start = START);
> -  . = START + SIZEOF_HEADERS;
> -  .interp         : { *(.interp) }
> -  __binary_start = .;
> -  . = ALIGN(4096);		/* Init code and data */
> -  _text = .;
> -  INIT_TEXT_SECTION(PAGE_SIZE)
> -
> -  . = ALIGN(PAGE_SIZE);
> -
> -  /* Read-only sections, merged into text segment: */
> -  .hash           : { *(.hash) }
> -  .gnu.hash       : { *(.gnu.hash) }
> -  .dynsym         : { *(.dynsym) }
> -  .dynstr         : { *(.dynstr) }
> -  .gnu.version    : { *(.gnu.version) }
> -  .gnu.version_d  : { *(.gnu.version_d) }
> -  .gnu.version_r  : { *(.gnu.version_r) }
> -  .rel.init       : { *(.rel.init) }
> -  .rela.init      : { *(.rela.init) }
> -  .rel.text       : { *(.rel.text .rel.text.* .rel.gnu.linkonce.t.*) }
> -  .rela.text      : { *(.rela.text .rela.text.* .rela.gnu.linkonce.t.*) }
> -  .rel.fini       : { *(.rel.fini) }
> -  .rela.fini      : { *(.rela.fini) }
> -  .rel.rodata     : { *(.rel.rodata .rel.rodata.* .rel.gnu.linkonce.r.*) }
> -  .rela.rodata    : { *(.rela.rodata .rela.rodata.* .rela.gnu.linkonce.r.*) }
> -  .rel.data       : { *(.rel.data .rel.data.* .rel.gnu.linkonce.d.*) }
> -  .rela.data      : { *(.rela.data .rela.data.* .rela.gnu.linkonce.d.*) }
> -  .rel.tdata	  : { *(.rel.tdata .rel.tdata.* .rel.gnu.linkonce.td.*) }
> -  .rela.tdata	  : { *(.rela.tdata .rela.tdata.* .rela.gnu.linkonce.td.*) }
> -  .rel.tbss	  : { *(.rel.tbss .rel.tbss.* .rel.gnu.linkonce.tb.*) }
> -  .rela.tbss	  : { *(.rela.tbss .rela.tbss.* .rela.gnu.linkonce.tb.*) }
> -  .rel.ctors      : { *(.rel.ctors) }
> -  .rela.ctors     : { *(.rela.ctors) }
> -  .rel.dtors      : { *(.rel.dtors) }
> -  .rela.dtors     : { *(.rela.dtors) }
> -  .rel.got        : { *(.rel.got) }
> -  .rela.got       : { *(.rela.got) }
> -  .rel.bss        : { *(.rel.bss .rel.bss.* .rel.gnu.linkonce.b.*) }
> -  .rela.bss       : { *(.rela.bss .rela.bss.* .rela.gnu.linkonce.b.*) }
> -  .rel.plt : {
> -	*(.rel.plt)
> -	PROVIDE_HIDDEN(__rel_iplt_start = .);
> -	*(.rel.iplt)
> -	PROVIDE_HIDDEN(__rel_iplt_end = .);
> -  }
> -  .rela.plt : {
> -	*(.rela.plt)
> -	PROVIDE_HIDDEN(__rela_iplt_start = .);
> -	*(.rela.iplt)
> -	PROVIDE_HIDDEN(__rela_iplt_end = .);
> -  }
> -  .init           : {
> -    KEEP (*(.init))
> -  } =0x90909090
> -  .plt            : { *(.plt) }
> -  .text           : {
> -    _stext = .;
> -    TEXT_TEXT
> -    SCHED_TEXT
> -    CPUIDLE_TEXT
> -    LOCK_TEXT
> -    IRQENTRY_TEXT
> -    SOFTIRQENTRY_TEXT
> -    *(.fixup)
> -    *(.stub .text.* .gnu.linkonce.t.*)
> -    /* .gnu.warning sections are handled specially by elf32.em.  */
> -    *(.gnu.warning)
> -
> -    . = ALIGN(PAGE_SIZE);
> -  } =0x90909090
> -  . = ALIGN(PAGE_SIZE);
> -  .syscall_stub : {
> -	__syscall_stub_start = .;
> -	*(.__syscall_stub*)
> -	__syscall_stub_end = .;
> -  }
> -  .fini           : {
> -    KEEP (*(.fini))
> -  } =0x90909090
> -
> -  .kstrtab : { *(.kstrtab) }
> -
> -  #include <asm/common.lds.S>
> -
> -  __init_begin = .;
> -  init.data : { INIT_DATA }
> -  __init_end = .;
> -
> -  /* Ensure the __preinit_array_start label is properly aligned.  We
> -     could instead move the label definition inside the section, but
> -     the linker would then create the section even if it turns out to
> -     be empty, which isn't pretty.  */
> -  . = ALIGN(32 / 8);
> -  .preinit_array     : { *(.preinit_array) }
> -  .init_array     : { *(.init_array) }
> -  .fini_array     : { *(.fini_array) }
> -  .data           : {
> -    INIT_TASK_DATA(KERNEL_STACK_SIZE)
> -    . = ALIGN(KERNEL_STACK_SIZE);
> -    *(.data..init_irqstack)
> -    DATA_DATA
> -    *(.data.* .gnu.linkonce.d.*)
> -    SORT(CONSTRUCTORS)
> -  }
> -  .data1          : { *(.data1) }
> -  .tdata	  : { *(.tdata .tdata.* .gnu.linkonce.td.*) }
> -  .tbss		  : { *(.tbss .tbss.* .gnu.linkonce.tb.*) *(.tcommon) }
> -  .eh_frame       : { KEEP (*(.eh_frame)) }
> -  .gcc_except_table   : { *(.gcc_except_table) }
> -  .dynamic        : { *(.dynamic) }
> -  .ctors          : {
> -    /* gcc uses crtbegin.o to find the start of
> -       the constructors, so we make sure it is
> -       first.  Because this is a wildcard, it
> -       doesn't matter if the user does not
> -       actually link against crtbegin.o; the
> -       linker won't look for a file to match a
> -       wildcard.  The wildcard also means that it
> -       doesn't matter which directory crtbegin.o
> -       is in.  */
> -    KEEP (*crtbegin.o(.ctors))
> -    /* We don't want to include the .ctor section from
> -       from the crtend.o file until after the sorted ctors.
> -       The .ctor section from the crtend file contains the
> -       end of ctors marker and it must be last */
> -    KEEP (*(EXCLUDE_FILE (*crtend.o ) .ctors))
> -    KEEP (*(SORT(.ctors.*)))
> -    KEEP (*(.ctors))
> -  }
> -  .dtors          : {
> -    KEEP (*crtbegin.o(.dtors))
> -    KEEP (*(EXCLUDE_FILE (*crtend.o ) .dtors))
> -    KEEP (*(SORT(.dtors.*)))
> -    KEEP (*(.dtors))
> -  }
> -  .jcr            : { KEEP (*(.jcr)) }
> -  .got            : { *(.got.plt) *(.got) }
> -  _edata = .;
> -  PROVIDE (edata = .);
> -  .bss            : {
> -   __bss_start = .;
> -   *(.dynbss)
> -   *(.bss .bss.* .gnu.linkonce.b.*)
> -   *(COMMON)
> -   /* Align here to ensure that the .bss section occupies space up to
> -      _end.  Align after .bss to ensure correct alignment even if the
> -      .bss section disappears because there are no input sections.  */
> -   . = ALIGN(32 / 8);
> -  . = ALIGN(32 / 8);
> -  }
> -   __bss_stop = .;
> -  _end = .;
> -  PROVIDE (end = .);
> -
> -  STABS_DEBUG
> -
> -  DWARF_DEBUG
> -
> -  DISCARDS
> -}
> diff --git a/arch/um/kernel/uml.lds.S b/arch/um/kernel/uml.lds.S
> deleted file mode 100644
> index 3b6dab3d4501..000000000000
> --- a/arch/um/kernel/uml.lds.S
> +++ /dev/null
> @@ -1,115 +0,0 @@
> -/* SPDX-License-Identifier: GPL-2.0 */
> -#include <asm/vmlinux.lds.h>
> -#include <asm/page.h>
> -
> -OUTPUT_FORMAT(ELF_FORMAT)
> -OUTPUT_ARCH(ELF_ARCH)
> -ENTRY(_start)
> -jiffies = jiffies_64;
> -
> -SECTIONS
> -{
> -  /* This must contain the right address - not quite the default ELF one.*/
> -  PROVIDE (__executable_start = START);
> -  /* Static binaries stick stuff here, like the sigreturn trampoline,
> -   * invisibly to objdump.  So, just make __binary_start equal to the very
> -   * beginning of the executable, and if there are unmapped pages after this,
> -   * they are forever unusable.
> -   */
> -  __binary_start = START;
> -
> -  . = START + SIZEOF_HEADERS;
> -  . = ALIGN(PAGE_SIZE);
> -
> -  _text = .;
> -  INIT_TEXT_SECTION(0)
> -
> -  .text      :
> -  {
> -    _stext = .;
> -    TEXT_TEXT
> -    SCHED_TEXT
> -    CPUIDLE_TEXT
> -    LOCK_TEXT
> -    IRQENTRY_TEXT
> -    SOFTIRQENTRY_TEXT
> -    *(.fixup)
> -    /* .gnu.warning sections are handled specially by elf32.em.  */
> -    *(.gnu.warning)
> -    *(.gnu.linkonce.t*)
> -  }
> -
> -  . = ALIGN(PAGE_SIZE);
> -  .syscall_stub : {
> -	__syscall_stub_start = .;
> -	*(.__syscall_stub*)
> -	__syscall_stub_end = .;
> -  }
> -
> -  /*
> -   * These are needed even in a static link, even if they wind up being empty.
> -   * Newer glibc needs these __rel{,a}_iplt_{start,end} symbols.
> -   */
> -  .rel.plt : {
> -	*(.rel.plt)
> -	PROVIDE_HIDDEN(__rel_iplt_start = .);
> -	*(.rel.iplt)
> -	PROVIDE_HIDDEN(__rel_iplt_end = .);
> -  }
> -  .rela.plt : {
> -	*(.rela.plt)
> -	PROVIDE_HIDDEN(__rela_iplt_start = .);
> -	*(.rela.iplt)
> -	PROVIDE_HIDDEN(__rela_iplt_end = .);
> -  }
> -
> -  #include <asm/common.lds.S>
> -
> -  __init_begin = .;
> -  init.data : { INIT_DATA }
> -  __init_end = .;
> -
> -  .data    :
> -  {
> -    INIT_TASK_DATA(KERNEL_STACK_SIZE)
> -    . = ALIGN(KERNEL_STACK_SIZE);
> -    *(.data..init_irqstack)
> -    DATA_DATA
> -    *(.gnu.linkonce.d*)
> -    CONSTRUCTORS
> -  }
> -  .data1   : { *(.data1) }
> -  .ctors         :
> -  {
> -    *(.ctors)
> -  }
> -  .dtors         :
> -  {
> -    *(.dtors)
> -  }
> -
> -  .got           : { *(.got.plt) *(.got) }
> -  .dynamic       : { *(.dynamic) }
> -  .tdata	  : { *(.tdata .tdata.* .gnu.linkonce.td.*) }
> -  .tbss		  : { *(.tbss .tbss.* .gnu.linkonce.tb.*) *(.tcommon) }
> -  /* We want the small data sections together, so single-instruction offsets
> -     can access them all, and initialized data all before uninitialized, so
> -     we can shorten the on-disk segment size.  */
> -  .sdata     : { *(.sdata) }
> -  _edata  =  .;
> -  PROVIDE (edata = .);
> -  . = ALIGN(PAGE_SIZE);
> -  __bss_start = .;
> -  PROVIDE(_bss_start = .);
> -  SBSS(0)
> -  BSS(0)
> -   __bss_stop = .;
> -  _end = .;
> -  PROVIDE (end = .);
> -
> -  STABS_DEBUG
> -
> -  DWARF_DEBUG
> -
> -  DISCARDS
> -}
> diff --git a/arch/um/kernel/vmlinux.lds.S b/arch/um/kernel/vmlinux.lds.S
> index 16e49bfa2b42..aaedd66772fa 100644
> --- a/arch/um/kernel/vmlinux.lds.S
> +++ b/arch/um/kernel/vmlinux.lds.S
> @@ -1,8 +1,87 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#include <asm/vmlinux.lds.h>
> +#include <asm/thread_info.h>
> +#include <asm/page.h>
> +#include <asm/thread_info.h>
> +#include <asm/cache.h>
> +#include <linux/export.h>
>   
> -KERNEL_STACK_SIZE = 4096 * (1 << CONFIG_KERNEL_STACK_ORDER);
> +OUTPUT_FORMAT(ELF_FORMAT)
> +jiffies = jiffies_64;
>   
> -#ifdef CONFIG_LD_SCRIPT_STATIC
> -#include "uml.lds.S"
> -#else
> -#include "dyn.lds.S"
> -#endif
> +
> +SECTIONS
> +{
> +	__init_begin = . ;
> +	HEAD_TEXT_SECTION
> +	INIT_TEXT_SECTION(PAGE_SIZE)
> +	INIT_DATA_SECTION(16)
> +	PERCPU_SECTION(L1_CACHE_BYTES)
> +	__init_end = . ;
> +
> +	_stext = . ;
> +	_text = . ;
> +	.text : ALIGN(THREAD_SIZE)
> +	{
> +		__binary_start = . ;
> +		TEXT_TEXT
> +		SCHED_TEXT
> +		LOCK_TEXT
> +		CPUIDLE_TEXT
> +		IRQENTRY_TEXT
> +		SOFTIRQENTRY_TEXT
> +	}
> +	_etext = . ;
> +
> +	.syscall_stub : ALIGN(PAGE_SIZE)
> +	{
> +		__syscall_stub_start = .;
> +		*(.__syscall_stub*)
> +		__syscall_stub_end = .;
> +	}
> +
> +	_sdata = . ;
> +	RO_DATA(PAGE_SIZE)
> +	RW_DATA(L1_CACHE_BYTES, PAGE_SIZE, THREAD_SIZE)
> +	.data : ALIGN(THREAD_SIZE)  { }
> +
> +	EXCEPTION_TABLE(16)
> +
> +	.uml.init_irqstack : ALIGN(THREAD_SIZE) {
> +		*(.data..init_irqstack)
> +	}
> +
> +	.uml.setup.init : ALIGN(8) {
> +		__uml_setup_start = .;
> +		*(.uml.setup.init)
> +		__uml_setup_end = .;
> +	}
> +
> +	.uml.help.init : ALIGN(8)  {
> +		__uml_help_start = .;
> +		*(.uml.help.init)
> +		__uml_help_end = .;
> +	}
> +
> +	.uml.postsetup.init : ALIGN(8) {
> +		__uml_postsetup_start = .;
> +		*(.uml.postsetup.init)
> +		__uml_postsetup_end = .;
> +	}
> +
> +	.uml.exitcall : ALIGN(8) {
> +		__uml_exitcall_begin = .;
> +		*(.uml.exitcall.exit)
> +		__uml_exitcall_end = .;
> +	}
> +	_edata = . ;
> +
> +	BSS_SECTION(0, 0, 0)
> +	_end = . ;
> +
> +	STABS_DEBUG
> +
> +	DWARF_DEBUG
> +
> +	DISCARDS
> +}
> diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
> index d09ab4afbda4..0bbb626ace81 100755
> --- a/scripts/link-vmlinux.sh
> +++ b/scripts/link-vmlinux.sh
> @@ -75,36 +75,18 @@ vmlinux_link()
>   		strip_debug=-Wl,--strip-debug
>   	fi
>   
> -	if [ "${SRCARCH}" != "um" ]; then
> -		objects="--whole-archive			\
> -			${KBUILD_VMLINUX_OBJS}			\
> -			--no-whole-archive			\
> -			--start-group				\
> -			${KBUILD_VMLINUX_LIBS}			\
> -			--end-group				\
> -			${@}"
> -
> -		${LD} ${KBUILD_LDFLAGS} ${LDFLAGS_vmlinux}	\
> -			${strip_debug#-Wl,}			\
> -			-o ${output}				\
> -			-T ${lds} ${objects}
> -	else
> -		objects="-Wl,--whole-archive			\
> -			${KBUILD_VMLINUX_OBJS}			\
> -			-Wl,--no-whole-archive			\
> -			-Wl,--start-group			\
> -			${KBUILD_VMLINUX_LIBS}			\
> -			-Wl,--end-group				\
> -			${@}"
> -
> -		${CC} ${CFLAGS_vmlinux}				\
> -			${strip_debug}				\
> -			-o ${output}				\
> -			-Wl,-T,${lds}				\
> -			${objects}				\
> -			-lutil -lrt -lpthread
> -		rm -f linux
> -	fi
> +	objects="--whole-archive			\
> +		${KBUILD_VMLINUX_OBJS}			\
> +		--no-whole-archive			\
> +		--start-group				\
> +		${KBUILD_VMLINUX_LIBS}			\
> +		--end-group				\
> +		${@}"
> +
> +	${LD} ${KBUILD_LDFLAGS} ${LDFLAGS_vmlinux}	\
> +		${strip_debug#-Wl,}			\
> +		-o ${output}				\
> +		-T ${lds} ${objects}
>   }
>   
>   # generate .BTF typeinfo from DWARF debuginfo
> diff --git a/tools/um/Makefile b/tools/um/Makefile
> new file mode 100644
> index 000000000000..a63466ef7ef5
> --- /dev/null
> +++ b/tools/um/Makefile
> @@ -0,0 +1,72 @@
> +# Do not use make's built-in rules
> +# (this improves performance and avoids hard-to-debug behaviour);
> +# also do not print "Entering directory..." messages from make
> +.SUFFIXES:
> +MAKEFLAGS += -r --no-print-directory
> +
> +KCONFIG?=defconfig
> +
> +ifneq ($(silent),1)
> +  ifneq ($(V),1)
> +	QUIET_AUTOCONF       = @echo '  AUTOCONF '$@;
> +	Q = @
> +  endif
> +endif
> +
> +PREFIX   := /usr
> +
> +ifeq (,$(srctree))
> +  srctree := $(patsubst %/,%,$(dir $(shell pwd)))
> +  srctree := $(patsubst %/,%,$(dir $(srctree)))
> +endif
> +export srctree
> +
> +-include ../scripts/Makefile.include
> +
> +# OUTPUT fixup should be *after* include ../scripts/Makefile.include
> +ifneq ($(OUTPUT),)
> +  OUTPUT := $(OUTPUT)/tools/um/
> +else
> +  OUTPUT := $(CURDIR)/
> +endif
> +export OUTPUT
> +export objtree := $(OUTPUT)/../..
> +
> +
> +all:
> +
> +export CFLAGS += -I$(OUTPUT)/include -Iinclude -Wall -g -O2 -Wextra -fPIC \
> +	 -Wno-unused-parameter \
> +	 -Wno-missing-field-initializers -fno-strict-aliasing
> +
> +-include Targets
> +
> +TARGETS := $(progs-y:%=$(OUTPUT)%)
> +TARGETS += $(libs-y:%=$(OUTPUT)%)
> +all: $(TARGETS)
> +
> +# rule to build linux.o
> +$(OUTPUT)lib/linux.o:
> +	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) $(KCONFIG)
> +	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) install INSTALL_PATH=$(OUTPUT)
> +
> +$(OUTPUT)liblinux.a: $(OUTPUT)lib/linux.o $(OUTPUT)uml/liblinux-in.o
> +	$(QUIET_AR)$(AR) -rc $@ $^
> +
> +# rule to link programs
> +$(OUTPUT)%: $(OUTPUT)%-in.o $(OUTPUT)liblinux.a
> +	$(QUIET_LINK)$(CC) $(LDFLAGS) $(LDFLAGS_$(notdir $*)-y) -o $@ $^ $(LDLIBS) $(LDLIBS_$(notdir $*)-y)
> +
> +$(OUTPUT)%-in.o: FORCE
> +	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
> +
> +clean:
> +	$(call QUIET_CLEAN, objects)find $(OUTPUT) -name '*.o' -delete -o -name '\.*.cmd'\
> +	 -delete -o -name '\.*.d' -delete
> +	$(call QUIET_CLEAN, liblinux.a)$(RM) $(OUTPUT)/liblinux.a
> +	$(call QUIET_CLEAN, $(TARGETS))$(RM) $(TARGETS)
> +
> +FORCE: ;
> +.PHONY: all clean FORCE
> +.IGNORE: clean
> +.NOTPARALLEL : lib/linux.o
> diff --git a/tools/um/Targets b/tools/um/Targets
> new file mode 100644
> index 000000000000..a4711f1ef422
> --- /dev/null
> +++ b/tools/um/Targets
> @@ -0,0 +1,4 @@
> +progs-y += uml/linux
> +LDLIBS_linux-y := -lrt -lpthread -lutil
> +LDFLAGS_linux-y := -no-pie -Wl,--wrap,malloc -Wl,--wrap,free -Wl,--wrap,calloc
> +LDFLAGS_linux-$(UML_STATIC) += -static
> diff --git a/tools/um/uml/Build b/tools/um/uml/Build
> new file mode 100644
> index 000000000000..e69de29bb2d1
> 

Hi Hajime, hi Octavian,

I am going to have a bit more time to look at UML now. Can you rebase/update the patch versus the latest kernel so I can look at it.

It does not apply at present.

Thanks in advance,

-- 
Anton R. Ivanov
Cambridgegreys Limited. Registered in England. Company Number 10273661
https://www.cambridgegreys.com/

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v5 01/21] um: split build in kernel and host parts
  2020-09-21 16:01         ` Anton Ivanov
@ 2020-09-21 22:27           ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-21 22:27 UTC (permalink / raw)
  To: anton.ivanov
  Cc: linux-um, linux-arch, tavi.purdila, tavi, retrage01,
	linux-kernel-library


On Tue, 22 Sep 2020 01:01:34 +0900,
Anton Ivanov wrote:

> Hi Hajime, hi Octavian,
> 
> I am going to have a bit more time to look at UML now. Can you rebase/update the patch versus the latest kernel so I can look at it.
> 
> It does not apply at present.

Okay, I'll do it.

Thanks,
-- Hajime

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v5 01/21] um: split build in kernel and host parts
@ 2020-09-21 22:27           ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-21 22:27 UTC (permalink / raw)
  To: anton.ivanov
  Cc: linux-arch, tavi.purdila, tavi, linux-um, retrage01,
	linux-kernel-library


On Tue, 22 Sep 2020 01:01:34 +0900,
Anton Ivanov wrote:

> Hi Hajime, hi Octavian,
> 
> I am going to have a bit more time to look at UML now. Can you rebase/update the patch versus the latest kernel so I can look at it.
> 
> It does not apply at present.

Okay, I'll do it.

Thanks,
-- Hajime

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* [RFC v6 00/21] Unifying LKL into UML
  2020-07-02 14:06     ` Hajime Tazaki
@ 2020-09-24  7:12       ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  7:12 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch, Hajime Tazaki

This is another spin of the unification of LKL into UML.  Based on the
discussion of v4 patchset, we have tried to address issue raised and
rewrote the patchset from scratch.  The summary is listed in the
changelog below.

Although there are still bugs in the patchset, we'd like to ask your
opinions on the design we changed.

The milestone section is also updated: this patchset is for the
milestone 1, though the common init API is still not implemented yet.


Changes in rfc v6:
- rebase with the current linus tree

Changes in rfc v5:
- rewrite whole patchset from scratch
- move arch-dependent code of arch/um and arch/x86/um to tools/um
 - mainly code under os-Linux/ involved
 - introduce 2-stage build (kernel, and host-dependent parts)
 - clean up vmlinux.lds.S
- put LKL-specific implementations as a SUBARCH under arch/um/nommu
- introduce !CONFIG_MMU in arch/um
- use struct arch_thread and arch_switch_to() for subarch-specific
  thread implementation
- integrate with the IRQ infrastructure of UML
- tested with block device drivers (ubd) for the proof

Changes in rfc v4: (https://lwn.net/Articles/816276/)
- Rebase on the current uml/master branch
- Fix IRQ handling (bug fix)
- drop a patch for CONFIG_GENERIC_ATOMIC64 (comment by Peter Zijlstra)
- implement vector net driver for UMMODE_LIB (comment by Anton)
- clean up uapi headers to avoid duplicates
- clean up IRQ handling code (comment by Anton)
- fix error handling in test code (comments by David Disseldorp)

Changes in rfc v3:
- use UML drivers (net, block) from LKL programs
- drop virtio device implementations
- drop mingw32 (Windows) host
- drop android (arm/aarch64) host
- drop FreeBSD (x86_64) host
- drop LD_PRELOAD (hijack) support
- update milestone

rfc v2:
- use UMMODE instead of SUBARCH to switch UML or LKL
- tools/lkl directory is still there. I confirmed we can move under arch/um
  (e.g., arch/um/lkl/hosts).  I will move it IF this is preferable.
- drop several patches involved non-uml directory
- drop several patches which are not required
- refine commit logs
- document updated




LKL (Linux Kernel Library) is aiming to allow reusing the Linux kernel code
as extensively as possible with minimal effort and reduced maintenance
overhead.

Examples of how LKL can be used are: creating userspace applications
(running on Linux and other operating systems) that can read or write Linux
filesystems or can use the Linux networking stack, creating kernel drivers
for other operating systems that can read Linux filesystems, bootloaders
support for reading/writing Linux filesystems, etc.

With LKL, the kernel code is compiled into an object file that can be
directly linked by applications. The API offered by LKL is based on the
Linux system call interface.

LKL is originally implemented as an architecture port in arch/lkl, but this
series of commits tries to integrate this into arch/um as one of the mode
of UML.  This was discussed during RFC email of LKL (*1).

The latest LKL version can be found at https://github.com/lkl/linux

Milestone
=========
This patches is a first step toward upstreaming *library mode* of Linux kernel,
but we think we need to have several steps toward our goal, describing in the
below.


Milestone 1: LKL lib on top of UML
 * Kernel - Host build split
 -  Build UML as a relocatable object using the UML's kernel linker script.
 -  Move the ptrace and other well isolated os code out of arch/um to
    tools/um
 -  Use standard host toolchain to create a static library stripped of
    the ptrace code. Use standard host toolchain to build the main UML
    executable.
 -  Add library init API that creates the UML kernel process and starts
    UML.
 * System calls APIs
 -  Add new system call interface based on UML's irq facility.
 -  Use the LKL scripts to export the required headers to create system
    calls APIs that use the UML system calls infrastructure.
 -  Keep the underlying host and driver operations (threads, irqs, etc.)
    as they are now in UML.
 * Boot test
 -  Port the LKL boot test to verify that we are able to programatically
    issue system calls.

Milestone 2: add virtio disk support
 * Export asm/io.h operations to host/os. Create IO access operations
   and redirect them to weak os_ variants that use the current UML
   implementation.
 * Add the LKL IO access layer including generic virtio handling and the
   virtio block device code.
 * Port LKL disk test and disk apps (lklfuse, fs2tar, cptofs)

Milestone 3: new arch ports
  * Abstract the system call / IRQ mode the move the implementation to host
  * Abstract the thread model and move the implementation to host
  * Add LKL thread model and LKL ports


Building LKL the host library and LKL applications
==================================================

% cd tools/um
% make

will build LKL as a object file, it will install it in tools/um/lib together
with the headers files in tools/um/include then will build the host library,
tests and a few of application examples:

* tests/boot - a simple applications that uses LKL and exercises the basic
  LKL APIs

% make run-tests

should run the above `tests/boot` and `tests/net-test` and report errors if
there are any.

Supported hosts
===============

Currently LKL supports Linux userspace applications. New hosts can be added
relatively easy if the host supports gcc and GNU ld. Previous versions of
LKL supported Windows kernel and Haiku kernel hosts, and we also have WIP
patches with rump-hypercall interface, used in UEFI, as well as macOS
userspace (part of POSIX).

There is also musl-libc port for LKL, which might be interested in for some
folks.


Further readings about LKL
=========================

- Discussion in github LKL issue
https://github.com/lkl/linux/issues/304

- LKL (an article)
https://www.researchgate.net/profile/Nicolae_Tapus2/publication/224164682_LKL_The_Linux_kernel_library/links/02bfe50fd921ab4f7c000000.pdf

*1 RFC email to LKML (back in 2015)
https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1012277.html


Please review the following changes for suitability for inclusion. If you have
any objections or suggestions for improvement, please respond to the patches. If
you agree with the changes, please provide your Acked-by.

The following changes since commit 805c6d3c19210c90c109107d189744e960eae025:

  Merge branch 'fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs (2020-09-22 15:08:41 -0700)

are available in the Git repository at:

  git://github.com/thehajime/linux b7b911105ecec6fd84f59e56d3a2c1789002bcbd
  https://github.com/thehajime/linux/tree/uml-lkl-5.9rc6-v6

Hajime Tazaki (18):
  um: move arch/um/os-Linux dir to tools/um/uml
  um: move arch/x86/um/os-Linux to tools/um/uml/
  scritps: um: suppress warnings if SRCARCH=um
  um: extend arch_switch_to for alternate SUBARCH
  um: add nommu mode for UML library mode
  um: nommu: host interface
  um: nommu: memory handling
  um: nommu: kernel thread support
  um: nommu: system call interface and application API
  um: nommu: basic console support
  um: nommu: initialization and cleanup
  um: nommu: integrate with irq infrastructure of UML
  um: nommu: plug in the build system
  um: host: add nommu build for ARCH=um
  um: host: add utilities functions
  um: host: posix host operations
  um: host: add test programs
  um: nommu: add block device support of UML

Octavian Purdila (3):
  um: split build in kernel and host parts
  um: add os init and exit calls
  um: host: implement os_initcalls and os_exitcalls

 arch/um/Kconfig                               |  29 +-
 arch/um/Makefile                              |  25 +-
 arch/um/drivers/Makefile                      |  10 +-
 .../um/{os-Linux => }/drivers/ethertap_kern.c |   0
 arch/um/{os-Linux => }/drivers/tuntap_kern.c  |   0
 arch/um/include/asm/common.lds.S              |  99 ----
 arch/um/include/asm/host_ops.h                |   9 +
 arch/um/include/asm/mmu.h                     |   3 +
 arch/um/include/asm/mmu_context.h             |   8 +
 arch/um/include/asm/page.h                    |  15 +
 arch/um/include/asm/pgtable.h                 |  27 +
 arch/um/include/asm/thread_info.h             |  24 +
 arch/um/include/asm/uaccess.h                 |   6 +
 arch/um/include/asm/xor.h                     |   3 +-
 arch/um/include/shared/as-layout.h            |   1 +
 .../drivers => include/shared}/etap.h         |   0
 arch/um/include/shared/init.h                 |  19 +-
 arch/um/include/shared/os.h                   |   1 +
 .../drivers => include/shared}/tuntap.h       |   0
 arch/um/include/uapi/asm/Kbuild               |   2 +
 arch/um/kernel/Makefile                       |  12 +-
 arch/um/kernel/dyn.lds.S                      | 171 -------
 arch/um/kernel/irq.c                          |  13 +
 arch/um/kernel/process.c                      |  14 +-
 arch/um/kernel/reboot.c                       |   5 +
 arch/um/kernel/time.c                         |   2 +
 arch/um/kernel/um_arch.c                      |  16 +
 arch/um/kernel/uml.lds.S                      | 115 -----
 arch/um/kernel/vmlinux.lds.S                  |  91 +++-
 arch/um/nommu/Makefile                        |   1 +
 arch/um/nommu/Makefile.um                     |  18 +
 arch/um/nommu/include/asm/Kbuild              |   6 +
 arch/um/nommu/include/asm/archparam.h         |   1 +
 arch/um/nommu/include/asm/atomic.h            |  11 +
 arch/um/nommu/include/asm/atomic64.h          | 114 +++++
 arch/um/nommu/include/asm/bitsperlong.h       |  12 +
 arch/um/nommu/include/asm/byteorder.h         |   7 +
 arch/um/nommu/include/asm/cpu.h               |  16 +
 arch/um/nommu/include/asm/elf.h               |  15 +
 arch/um/nommu/include/asm/mm_context.h        |   8 +
 arch/um/nommu/include/asm/processor.h         |  46 ++
 arch/um/nommu/include/asm/ptrace.h            |  21 +
 arch/um/nommu/include/asm/sched.h             |  23 +
 arch/um/nommu/include/asm/segment.h           |   9 +
 arch/um/nommu/include/asm/syscall_wrapper.h   |  57 +++
 arch/um/nommu/include/asm/syscalls.h          |  15 +
 arch/um/nommu/include/uapi/asm/Kbuild         |   4 +
 arch/um/nommu/include/uapi/asm/bitsperlong.h  |  11 +
 arch/um/nommu/include/uapi/asm/byteorder.h    |  11 +
 arch/um/nommu/include/uapi/asm/host_ops.h     | 122 +++++
 arch/um/nommu/include/uapi/asm/sigcontext.h   |  12 +
 arch/um/nommu/include/uapi/asm/syscalls.h     | 287 +++++++++++
 arch/um/nommu/include/uapi/asm/unistd.h       |  17 +
 arch/um/nommu/um/Kconfig                      |  45 ++
 arch/um/nommu/um/Makefile                     |   4 +
 arch/um/nommu/um/bootmem.c                    |  86 ++++
 arch/um/nommu/um/console.c                    |  42 ++
 arch/um/nommu/um/cpu.c                        | 247 ++++++++++
 arch/um/nommu/um/delay.c                      |  31 ++
 arch/um/nommu/um/setup.c                      | 179 +++++++
 arch/um/nommu/um/shared/sysdep/archsetjmp.h   |  13 +
 arch/um/nommu/um/shared/sysdep/faultinfo.h    |   8 +
 .../nommu/um/shared/sysdep/kernel-offsets.h   |  12 +
 arch/um/nommu/um/shared/sysdep/mcontext.h     |   9 +
 arch/um/nommu/um/shared/sysdep/ptrace.h       |  42 ++
 arch/um/nommu/um/shared/sysdep/ptrace_user.h  |   7 +
 arch/um/nommu/um/syscalls.c                   | 199 ++++++++
 arch/um/nommu/um/threads.c                    | 261 ++++++++++
 arch/um/nommu/um/unimplemented.c              |  70 +++
 arch/um/nommu/um/user_constants.h             |  13 +
 arch/um/os-Linux/Makefile                     |  19 -
 arch/um/os-Linux/drivers/Makefile             |  13 -
 arch/um/scripts/headers_install.py            | 197 ++++++++
 arch/x86/um/Makefile                          |   2 +-
 arch/x86/um/os-Linux/Makefile                 |  13 -
 arch/x86/um/ptrace_32.c                       |   2 +-
 arch/x86/um/syscalls_64.c                     |   2 +-
 scripts/headers_install.sh                    |   4 +
 scripts/link-vmlinux.sh                       |  42 +-
 tools/um/Makefile                             |  82 ++++
 tools/um/Targets                              |  13 +
 tools/um/include/lkl.h                        | 357 ++++++++++++++
 tools/um/include/lkl_host.h                   |  27 +
 tools/um/lib/Build                            |   7 +
 tools/um/lib/fs.c                             | 461 ++++++++++++++++++
 tools/um/lib/jmp_buf.c                        |  14 +
 tools/um/lib/jmp_buf.h                        |   8 +
 tools/um/lib/posix-host.c                     | 293 +++++++++++
 tools/um/lib/utils.c                          | 213 ++++++++
 tools/um/tests/Build                          |   2 +
 tools/um/tests/boot.c                         | 393 +++++++++++++++
 tools/um/tests/boot.sh                        |   9 +
 tools/um/tests/cla.c                          | 159 ++++++
 tools/um/tests/cla.h                          |  33 ++
 tools/um/tests/disk.c                         | 168 +++++++
 tools/um/tests/disk.sh                        |  70 +++
 tools/um/tests/run.py                         | 174 +++++++
 tools/um/tests/tap13.py                       | 209 ++++++++
 tools/um/tests/test.c                         | 126 +++++
 tools/um/tests/test.h                         |  72 +++
 tools/um/tests/test.sh                        | 181 +++++++
 tools/um/uml/Build                            |  57 +++
 tools/um/uml/drivers/Build                    |  10 +
 .../um/uml}/drivers/ethertap_user.c           |   0
 .../um/uml}/drivers/tuntap_user.c             |   0
 {arch/um/os-Linux => tools/um/uml}/elf_aux.c  |   0
 {arch/um/os-Linux => tools/um/uml}/execvp.c   |   4 -
 {arch/um/os-Linux => tools/um/uml}/file.c     |   0
 {arch/um/os-Linux => tools/um/uml}/helper.c   |   0
 {arch/um/os-Linux => tools/um/uml}/irq.c      |   0
 {arch/um/os-Linux => tools/um/uml}/main.c     |   0
 {arch/um/os-Linux => tools/um/uml}/mem.c      |   0
 tools/um/uml/nommu/Build                      |   1 +
 tools/um/uml/nommu/registers.c                |  21 +
 tools/um/uml/nommu/unimplemented.c            |  21 +
 {arch/um/os-Linux => tools/um/uml}/process.c  |   2 +
 .../um/os-Linux => tools/um/uml}/registers.c  |   0
 {arch/um/os-Linux => tools/um/uml}/sigio.c    |   0
 {arch/um/os-Linux => tools/um/uml}/signal.c   |  12 +-
 .../skas/Makefile => tools/um/uml/skas/Build  |   6 +-
 {arch/um/os-Linux => tools/um/uml}/skas/mem.c |   0
 .../os-Linux => tools/um/uml}/skas/process.c  |   3 +-
 {arch/um/os-Linux => tools/um/uml}/start_up.c |   0
 {arch/um/os-Linux => tools/um/uml}/time.c     |   0
 {arch/um/os-Linux => tools/um/uml}/tty.c      |   0
 {arch/um/os-Linux => tools/um/uml}/umid.c     |   0
 .../um/os-Linux => tools/um/uml}/user_syms.c  |   1 +
 {arch/um/os-Linux => tools/um/uml}/util.c     |  26 +
 tools/um/uml/x86/Build                        |  11 +
 .../os-Linux => tools/um/uml/x86}/mcontext.c  |   0
 .../um/os-Linux => tools/um/uml/x86}/prctl.c  |   0
 .../os-Linux => tools/um/uml/x86}/registers.c |   0
 .../os-Linux => tools/um/uml/x86}/task_size.c |   0
 .../um/os-Linux => tools/um/uml/x86}/tls.c    |   0
 134 files changed, 5850 insertions(+), 522 deletions(-)
 rename arch/um/{os-Linux => }/drivers/ethertap_kern.c (100%)
 rename arch/um/{os-Linux => }/drivers/tuntap_kern.c (100%)
 delete mode 100644 arch/um/include/asm/common.lds.S
 create mode 100644 arch/um/include/asm/host_ops.h
 rename arch/um/{os-Linux/drivers => include/shared}/etap.h (100%)
 rename arch/um/{os-Linux/drivers => include/shared}/tuntap.h (100%)
 create mode 100644 arch/um/include/uapi/asm/Kbuild
 delete mode 100644 arch/um/kernel/dyn.lds.S
 delete mode 100644 arch/um/kernel/uml.lds.S
 create mode 100644 arch/um/nommu/Makefile
 create mode 100644 arch/um/nommu/Makefile.um
 create mode 100644 arch/um/nommu/include/asm/Kbuild
 create mode 100644 arch/um/nommu/include/asm/archparam.h
 create mode 100644 arch/um/nommu/include/asm/atomic.h
 create mode 100644 arch/um/nommu/include/asm/atomic64.h
 create mode 100644 arch/um/nommu/include/asm/bitsperlong.h
 create mode 100644 arch/um/nommu/include/asm/byteorder.h
 create mode 100644 arch/um/nommu/include/asm/cpu.h
 create mode 100644 arch/um/nommu/include/asm/elf.h
 create mode 100644 arch/um/nommu/include/asm/mm_context.h
 create mode 100644 arch/um/nommu/include/asm/processor.h
 create mode 100644 arch/um/nommu/include/asm/ptrace.h
 create mode 100644 arch/um/nommu/include/asm/sched.h
 create mode 100644 arch/um/nommu/include/asm/segment.h
 create mode 100644 arch/um/nommu/include/asm/syscall_wrapper.h
 create mode 100644 arch/um/nommu/include/asm/syscalls.h
 create mode 100644 arch/um/nommu/include/uapi/asm/Kbuild
 create mode 100644 arch/um/nommu/include/uapi/asm/bitsperlong.h
 create mode 100644 arch/um/nommu/include/uapi/asm/byteorder.h
 create mode 100644 arch/um/nommu/include/uapi/asm/host_ops.h
 create mode 100644 arch/um/nommu/include/uapi/asm/sigcontext.h
 create mode 100644 arch/um/nommu/include/uapi/asm/syscalls.h
 create mode 100644 arch/um/nommu/include/uapi/asm/unistd.h
 create mode 100644 arch/um/nommu/um/Kconfig
 create mode 100644 arch/um/nommu/um/Makefile
 create mode 100644 arch/um/nommu/um/bootmem.c
 create mode 100644 arch/um/nommu/um/console.c
 create mode 100644 arch/um/nommu/um/cpu.c
 create mode 100644 arch/um/nommu/um/delay.c
 create mode 100644 arch/um/nommu/um/setup.c
 create mode 100644 arch/um/nommu/um/shared/sysdep/archsetjmp.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/faultinfo.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/kernel-offsets.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/mcontext.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/ptrace.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/ptrace_user.h
 create mode 100644 arch/um/nommu/um/syscalls.c
 create mode 100644 arch/um/nommu/um/threads.c
 create mode 100644 arch/um/nommu/um/unimplemented.c
 create mode 100644 arch/um/nommu/um/user_constants.h
 delete mode 100644 arch/um/os-Linux/Makefile
 delete mode 100644 arch/um/os-Linux/drivers/Makefile
 create mode 100755 arch/um/scripts/headers_install.py
 delete mode 100644 arch/x86/um/os-Linux/Makefile
 create mode 100644 tools/um/Makefile
 create mode 100644 tools/um/Targets
 create mode 100644 tools/um/include/lkl.h
 create mode 100644 tools/um/include/lkl_host.h
 create mode 100644 tools/um/lib/Build
 create mode 100644 tools/um/lib/fs.c
 create mode 100644 tools/um/lib/jmp_buf.c
 create mode 100644 tools/um/lib/jmp_buf.h
 create mode 100644 tools/um/lib/posix-host.c
 create mode 100644 tools/um/lib/utils.c
 create mode 100644 tools/um/tests/Build
 create mode 100644 tools/um/tests/boot.c
 create mode 100755 tools/um/tests/boot.sh
 create mode 100644 tools/um/tests/cla.c
 create mode 100644 tools/um/tests/cla.h
 create mode 100644 tools/um/tests/disk.c
 create mode 100755 tools/um/tests/disk.sh
 create mode 100755 tools/um/tests/run.py
 create mode 100644 tools/um/tests/tap13.py
 create mode 100644 tools/um/tests/test.c
 create mode 100644 tools/um/tests/test.h
 create mode 100644 tools/um/tests/test.sh
 create mode 100644 tools/um/uml/Build
 create mode 100644 tools/um/uml/drivers/Build
 rename {arch/um/os-Linux => tools/um/uml}/drivers/ethertap_user.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/drivers/tuntap_user.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/elf_aux.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/execvp.c (98%)
 rename {arch/um/os-Linux => tools/um/uml}/file.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/helper.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/irq.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/main.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/mem.c (100%)
 create mode 100644 tools/um/uml/nommu/Build
 create mode 100644 tools/um/uml/nommu/registers.c
 create mode 100644 tools/um/uml/nommu/unimplemented.c
 rename {arch/um/os-Linux => tools/um/uml}/process.c (99%)
 rename {arch/um/os-Linux => tools/um/uml}/registers.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/sigio.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/signal.c (96%)
 rename arch/um/os-Linux/skas/Makefile => tools/um/uml/skas/Build (56%)
 rename {arch/um/os-Linux => tools/um/uml}/skas/mem.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/skas/process.c (99%)
 rename {arch/um/os-Linux => tools/um/uml}/start_up.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/time.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/tty.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/umid.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/user_syms.c (99%)
 rename {arch/um/os-Linux => tools/um/uml}/util.c (89%)
 create mode 100644 tools/um/uml/x86/Build
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/mcontext.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/prctl.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/registers.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/task_size.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/tls.c (100%)

-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply	[flat|nested] 476+ messages in thread
* [RFC v6 00/21] Unifying LKL into UML
@ 2020-09-24  7:12       ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  7:12 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Hajime Tazaki, retrage01

This is another spin of the unification of LKL into UML.  Based on the
discussion of v4 patchset, we have tried to address issue raised and
rewrote the patchset from scratch.  The summary is listed in the
changelog below.

Although there are still bugs in the patchset, we'd like to ask your
opinions on the design we changed.

The milestone section is also updated: this patchset is for the
milestone 1, though the common init API is still not implemented yet.


Changes in rfc v6:
- rebase with the current linus tree

Changes in rfc v5:
- rewrite whole patchset from scratch
- move arch-dependent code of arch/um and arch/x86/um to tools/um
 - mainly code under os-Linux/ involved
 - introduce 2-stage build (kernel, and host-dependent parts)
 - clean up vmlinux.lds.S
- put LKL-specific implementations as a SUBARCH under arch/um/nommu
- introduce !CONFIG_MMU in arch/um
- use struct arch_thread and arch_switch_to() for subarch-specific
  thread implementation
- integrate with the IRQ infrastructure of UML
- tested with block device drivers (ubd) for the proof

Changes in rfc v4: (https://lwn.net/Articles/816276/)
- Rebase on the current uml/master branch
- Fix IRQ handling (bug fix)
- drop a patch for CONFIG_GENERIC_ATOMIC64 (comment by Peter Zijlstra)
- implement vector net driver for UMMODE_LIB (comment by Anton)
- clean up uapi headers to avoid duplicates
- clean up IRQ handling code (comment by Anton)
- fix error handling in test code (comments by David Disseldorp)

Changes in rfc v3:
- use UML drivers (net, block) from LKL programs
- drop virtio device implementations
- drop mingw32 (Windows) host
- drop android (arm/aarch64) host
- drop FreeBSD (x86_64) host
- drop LD_PRELOAD (hijack) support
- update milestone

rfc v2:
- use UMMODE instead of SUBARCH to switch UML or LKL
- tools/lkl directory is still there. I confirmed we can move under arch/um
  (e.g., arch/um/lkl/hosts).  I will move it IF this is preferable.
- drop several patches involved non-uml directory
- drop several patches which are not required
- refine commit logs
- document updated




LKL (Linux Kernel Library) is aiming to allow reusing the Linux kernel code
as extensively as possible with minimal effort and reduced maintenance
overhead.

Examples of how LKL can be used are: creating userspace applications
(running on Linux and other operating systems) that can read or write Linux
filesystems or can use the Linux networking stack, creating kernel drivers
for other operating systems that can read Linux filesystems, bootloaders
support for reading/writing Linux filesystems, etc.

With LKL, the kernel code is compiled into an object file that can be
directly linked by applications. The API offered by LKL is based on the
Linux system call interface.

LKL is originally implemented as an architecture port in arch/lkl, but this
series of commits tries to integrate this into arch/um as one of the mode
of UML.  This was discussed during RFC email of LKL (*1).

The latest LKL version can be found at https://github.com/lkl/linux

Milestone
=========
This patches is a first step toward upstreaming *library mode* of Linux kernel,
but we think we need to have several steps toward our goal, describing in the
below.


Milestone 1: LKL lib on top of UML
 * Kernel - Host build split
 -  Build UML as a relocatable object using the UML's kernel linker script.
 -  Move the ptrace and other well isolated os code out of arch/um to
    tools/um
 -  Use standard host toolchain to create a static library stripped of
    the ptrace code. Use standard host toolchain to build the main UML
    executable.
 -  Add library init API that creates the UML kernel process and starts
    UML.
 * System calls APIs
 -  Add new system call interface based on UML's irq facility.
 -  Use the LKL scripts to export the required headers to create system
    calls APIs that use the UML system calls infrastructure.
 -  Keep the underlying host and driver operations (threads, irqs, etc.)
    as they are now in UML.
 * Boot test
 -  Port the LKL boot test to verify that we are able to programatically
    issue system calls.

Milestone 2: add virtio disk support
 * Export asm/io.h operations to host/os. Create IO access operations
   and redirect them to weak os_ variants that use the current UML
   implementation.
 * Add the LKL IO access layer including generic virtio handling and the
   virtio block device code.
 * Port LKL disk test and disk apps (lklfuse, fs2tar, cptofs)

Milestone 3: new arch ports
  * Abstract the system call / IRQ mode the move the implementation to host
  * Abstract the thread model and move the implementation to host
  * Add LKL thread model and LKL ports


Building LKL the host library and LKL applications
==================================================

% cd tools/um
% make

will build LKL as a object file, it will install it in tools/um/lib together
with the headers files in tools/um/include then will build the host library,
tests and a few of application examples:

* tests/boot - a simple applications that uses LKL and exercises the basic
  LKL APIs

% make run-tests

should run the above `tests/boot` and `tests/net-test` and report errors if
there are any.

Supported hosts
===============

Currently LKL supports Linux userspace applications. New hosts can be added
relatively easy if the host supports gcc and GNU ld. Previous versions of
LKL supported Windows kernel and Haiku kernel hosts, and we also have WIP
patches with rump-hypercall interface, used in UEFI, as well as macOS
userspace (part of POSIX).

There is also musl-libc port for LKL, which might be interested in for some
folks.


Further readings about LKL
=========================

- Discussion in github LKL issue
https://github.com/lkl/linux/issues/304

- LKL (an article)
https://www.researchgate.net/profile/Nicolae_Tapus2/publication/224164682_LKL_The_Linux_kernel_library/links/02bfe50fd921ab4f7c000000.pdf

*1 RFC email to LKML (back in 2015)
https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1012277.html


Please review the following changes for suitability for inclusion. If you have
any objections or suggestions for improvement, please respond to the patches. If
you agree with the changes, please provide your Acked-by.

The following changes since commit 805c6d3c19210c90c109107d189744e960eae025:

  Merge branch 'fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs (2020-09-22 15:08:41 -0700)

are available in the Git repository at:

  git://github.com/thehajime/linux b7b911105ecec6fd84f59e56d3a2c1789002bcbd
  https://github.com/thehajime/linux/tree/uml-lkl-5.9rc6-v6

Hajime Tazaki (18):
  um: move arch/um/os-Linux dir to tools/um/uml
  um: move arch/x86/um/os-Linux to tools/um/uml/
  scritps: um: suppress warnings if SRCARCH=um
  um: extend arch_switch_to for alternate SUBARCH
  um: add nommu mode for UML library mode
  um: nommu: host interface
  um: nommu: memory handling
  um: nommu: kernel thread support
  um: nommu: system call interface and application API
  um: nommu: basic console support
  um: nommu: initialization and cleanup
  um: nommu: integrate with irq infrastructure of UML
  um: nommu: plug in the build system
  um: host: add nommu build for ARCH=um
  um: host: add utilities functions
  um: host: posix host operations
  um: host: add test programs
  um: nommu: add block device support of UML

Octavian Purdila (3):
  um: split build in kernel and host parts
  um: add os init and exit calls
  um: host: implement os_initcalls and os_exitcalls

 arch/um/Kconfig                               |  29 +-
 arch/um/Makefile                              |  25 +-
 arch/um/drivers/Makefile                      |  10 +-
 .../um/{os-Linux => }/drivers/ethertap_kern.c |   0
 arch/um/{os-Linux => }/drivers/tuntap_kern.c  |   0
 arch/um/include/asm/common.lds.S              |  99 ----
 arch/um/include/asm/host_ops.h                |   9 +
 arch/um/include/asm/mmu.h                     |   3 +
 arch/um/include/asm/mmu_context.h             |   8 +
 arch/um/include/asm/page.h                    |  15 +
 arch/um/include/asm/pgtable.h                 |  27 +
 arch/um/include/asm/thread_info.h             |  24 +
 arch/um/include/asm/uaccess.h                 |   6 +
 arch/um/include/asm/xor.h                     |   3 +-
 arch/um/include/shared/as-layout.h            |   1 +
 .../drivers => include/shared}/etap.h         |   0
 arch/um/include/shared/init.h                 |  19 +-
 arch/um/include/shared/os.h                   |   1 +
 .../drivers => include/shared}/tuntap.h       |   0
 arch/um/include/uapi/asm/Kbuild               |   2 +
 arch/um/kernel/Makefile                       |  12 +-
 arch/um/kernel/dyn.lds.S                      | 171 -------
 arch/um/kernel/irq.c                          |  13 +
 arch/um/kernel/process.c                      |  14 +-
 arch/um/kernel/reboot.c                       |   5 +
 arch/um/kernel/time.c                         |   2 +
 arch/um/kernel/um_arch.c                      |  16 +
 arch/um/kernel/uml.lds.S                      | 115 -----
 arch/um/kernel/vmlinux.lds.S                  |  91 +++-
 arch/um/nommu/Makefile                        |   1 +
 arch/um/nommu/Makefile.um                     |  18 +
 arch/um/nommu/include/asm/Kbuild              |   6 +
 arch/um/nommu/include/asm/archparam.h         |   1 +
 arch/um/nommu/include/asm/atomic.h            |  11 +
 arch/um/nommu/include/asm/atomic64.h          | 114 +++++
 arch/um/nommu/include/asm/bitsperlong.h       |  12 +
 arch/um/nommu/include/asm/byteorder.h         |   7 +
 arch/um/nommu/include/asm/cpu.h               |  16 +
 arch/um/nommu/include/asm/elf.h               |  15 +
 arch/um/nommu/include/asm/mm_context.h        |   8 +
 arch/um/nommu/include/asm/processor.h         |  46 ++
 arch/um/nommu/include/asm/ptrace.h            |  21 +
 arch/um/nommu/include/asm/sched.h             |  23 +
 arch/um/nommu/include/asm/segment.h           |   9 +
 arch/um/nommu/include/asm/syscall_wrapper.h   |  57 +++
 arch/um/nommu/include/asm/syscalls.h          |  15 +
 arch/um/nommu/include/uapi/asm/Kbuild         |   4 +
 arch/um/nommu/include/uapi/asm/bitsperlong.h  |  11 +
 arch/um/nommu/include/uapi/asm/byteorder.h    |  11 +
 arch/um/nommu/include/uapi/asm/host_ops.h     | 122 +++++
 arch/um/nommu/include/uapi/asm/sigcontext.h   |  12 +
 arch/um/nommu/include/uapi/asm/syscalls.h     | 287 +++++++++++
 arch/um/nommu/include/uapi/asm/unistd.h       |  17 +
 arch/um/nommu/um/Kconfig                      |  45 ++
 arch/um/nommu/um/Makefile                     |   4 +
 arch/um/nommu/um/bootmem.c                    |  86 ++++
 arch/um/nommu/um/console.c                    |  42 ++
 arch/um/nommu/um/cpu.c                        | 247 ++++++++++
 arch/um/nommu/um/delay.c                      |  31 ++
 arch/um/nommu/um/setup.c                      | 179 +++++++
 arch/um/nommu/um/shared/sysdep/archsetjmp.h   |  13 +
 arch/um/nommu/um/shared/sysdep/faultinfo.h    |   8 +
 .../nommu/um/shared/sysdep/kernel-offsets.h   |  12 +
 arch/um/nommu/um/shared/sysdep/mcontext.h     |   9 +
 arch/um/nommu/um/shared/sysdep/ptrace.h       |  42 ++
 arch/um/nommu/um/shared/sysdep/ptrace_user.h  |   7 +
 arch/um/nommu/um/syscalls.c                   | 199 ++++++++
 arch/um/nommu/um/threads.c                    | 261 ++++++++++
 arch/um/nommu/um/unimplemented.c              |  70 +++
 arch/um/nommu/um/user_constants.h             |  13 +
 arch/um/os-Linux/Makefile                     |  19 -
 arch/um/os-Linux/drivers/Makefile             |  13 -
 arch/um/scripts/headers_install.py            | 197 ++++++++
 arch/x86/um/Makefile                          |   2 +-
 arch/x86/um/os-Linux/Makefile                 |  13 -
 arch/x86/um/ptrace_32.c                       |   2 +-
 arch/x86/um/syscalls_64.c                     |   2 +-
 scripts/headers_install.sh                    |   4 +
 scripts/link-vmlinux.sh                       |  42 +-
 tools/um/Makefile                             |  82 ++++
 tools/um/Targets                              |  13 +
 tools/um/include/lkl.h                        | 357 ++++++++++++++
 tools/um/include/lkl_host.h                   |  27 +
 tools/um/lib/Build                            |   7 +
 tools/um/lib/fs.c                             | 461 ++++++++++++++++++
 tools/um/lib/jmp_buf.c                        |  14 +
 tools/um/lib/jmp_buf.h                        |   8 +
 tools/um/lib/posix-host.c                     | 293 +++++++++++
 tools/um/lib/utils.c                          | 213 ++++++++
 tools/um/tests/Build                          |   2 +
 tools/um/tests/boot.c                         | 393 +++++++++++++++
 tools/um/tests/boot.sh                        |   9 +
 tools/um/tests/cla.c                          | 159 ++++++
 tools/um/tests/cla.h                          |  33 ++
 tools/um/tests/disk.c                         | 168 +++++++
 tools/um/tests/disk.sh                        |  70 +++
 tools/um/tests/run.py                         | 174 +++++++
 tools/um/tests/tap13.py                       | 209 ++++++++
 tools/um/tests/test.c                         | 126 +++++
 tools/um/tests/test.h                         |  72 +++
 tools/um/tests/test.sh                        | 181 +++++++
 tools/um/uml/Build                            |  57 +++
 tools/um/uml/drivers/Build                    |  10 +
 .../um/uml}/drivers/ethertap_user.c           |   0
 .../um/uml}/drivers/tuntap_user.c             |   0
 {arch/um/os-Linux => tools/um/uml}/elf_aux.c  |   0
 {arch/um/os-Linux => tools/um/uml}/execvp.c   |   4 -
 {arch/um/os-Linux => tools/um/uml}/file.c     |   0
 {arch/um/os-Linux => tools/um/uml}/helper.c   |   0
 {arch/um/os-Linux => tools/um/uml}/irq.c      |   0
 {arch/um/os-Linux => tools/um/uml}/main.c     |   0
 {arch/um/os-Linux => tools/um/uml}/mem.c      |   0
 tools/um/uml/nommu/Build                      |   1 +
 tools/um/uml/nommu/registers.c                |  21 +
 tools/um/uml/nommu/unimplemented.c            |  21 +
 {arch/um/os-Linux => tools/um/uml}/process.c  |   2 +
 .../um/os-Linux => tools/um/uml}/registers.c  |   0
 {arch/um/os-Linux => tools/um/uml}/sigio.c    |   0
 {arch/um/os-Linux => tools/um/uml}/signal.c   |  12 +-
 .../skas/Makefile => tools/um/uml/skas/Build  |   6 +-
 {arch/um/os-Linux => tools/um/uml}/skas/mem.c |   0
 .../os-Linux => tools/um/uml}/skas/process.c  |   3 +-
 {arch/um/os-Linux => tools/um/uml}/start_up.c |   0
 {arch/um/os-Linux => tools/um/uml}/time.c     |   0
 {arch/um/os-Linux => tools/um/uml}/tty.c      |   0
 {arch/um/os-Linux => tools/um/uml}/umid.c     |   0
 .../um/os-Linux => tools/um/uml}/user_syms.c  |   1 +
 {arch/um/os-Linux => tools/um/uml}/util.c     |  26 +
 tools/um/uml/x86/Build                        |  11 +
 .../os-Linux => tools/um/uml/x86}/mcontext.c  |   0
 .../um/os-Linux => tools/um/uml/x86}/prctl.c  |   0
 .../os-Linux => tools/um/uml/x86}/registers.c |   0
 .../os-Linux => tools/um/uml/x86}/task_size.c |   0
 .../um/os-Linux => tools/um/uml/x86}/tls.c    |   0
 134 files changed, 5850 insertions(+), 522 deletions(-)
 rename arch/um/{os-Linux => }/drivers/ethertap_kern.c (100%)
 rename arch/um/{os-Linux => }/drivers/tuntap_kern.c (100%)
 delete mode 100644 arch/um/include/asm/common.lds.S
 create mode 100644 arch/um/include/asm/host_ops.h
 rename arch/um/{os-Linux/drivers => include/shared}/etap.h (100%)
 rename arch/um/{os-Linux/drivers => include/shared}/tuntap.h (100%)
 create mode 100644 arch/um/include/uapi/asm/Kbuild
 delete mode 100644 arch/um/kernel/dyn.lds.S
 delete mode 100644 arch/um/kernel/uml.lds.S
 create mode 100644 arch/um/nommu/Makefile
 create mode 100644 arch/um/nommu/Makefile.um
 create mode 100644 arch/um/nommu/include/asm/Kbuild
 create mode 100644 arch/um/nommu/include/asm/archparam.h
 create mode 100644 arch/um/nommu/include/asm/atomic.h
 create mode 100644 arch/um/nommu/include/asm/atomic64.h
 create mode 100644 arch/um/nommu/include/asm/bitsperlong.h
 create mode 100644 arch/um/nommu/include/asm/byteorder.h
 create mode 100644 arch/um/nommu/include/asm/cpu.h
 create mode 100644 arch/um/nommu/include/asm/elf.h
 create mode 100644 arch/um/nommu/include/asm/mm_context.h
 create mode 100644 arch/um/nommu/include/asm/processor.h
 create mode 100644 arch/um/nommu/include/asm/ptrace.h
 create mode 100644 arch/um/nommu/include/asm/sched.h
 create mode 100644 arch/um/nommu/include/asm/segment.h
 create mode 100644 arch/um/nommu/include/asm/syscall_wrapper.h
 create mode 100644 arch/um/nommu/include/asm/syscalls.h
 create mode 100644 arch/um/nommu/include/uapi/asm/Kbuild
 create mode 100644 arch/um/nommu/include/uapi/asm/bitsperlong.h
 create mode 100644 arch/um/nommu/include/uapi/asm/byteorder.h
 create mode 100644 arch/um/nommu/include/uapi/asm/host_ops.h
 create mode 100644 arch/um/nommu/include/uapi/asm/sigcontext.h
 create mode 100644 arch/um/nommu/include/uapi/asm/syscalls.h
 create mode 100644 arch/um/nommu/include/uapi/asm/unistd.h
 create mode 100644 arch/um/nommu/um/Kconfig
 create mode 100644 arch/um/nommu/um/Makefile
 create mode 100644 arch/um/nommu/um/bootmem.c
 create mode 100644 arch/um/nommu/um/console.c
 create mode 100644 arch/um/nommu/um/cpu.c
 create mode 100644 arch/um/nommu/um/delay.c
 create mode 100644 arch/um/nommu/um/setup.c
 create mode 100644 arch/um/nommu/um/shared/sysdep/archsetjmp.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/faultinfo.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/kernel-offsets.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/mcontext.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/ptrace.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/ptrace_user.h
 create mode 100644 arch/um/nommu/um/syscalls.c
 create mode 100644 arch/um/nommu/um/threads.c
 create mode 100644 arch/um/nommu/um/unimplemented.c
 create mode 100644 arch/um/nommu/um/user_constants.h
 delete mode 100644 arch/um/os-Linux/Makefile
 delete mode 100644 arch/um/os-Linux/drivers/Makefile
 create mode 100755 arch/um/scripts/headers_install.py
 delete mode 100644 arch/x86/um/os-Linux/Makefile
 create mode 100644 tools/um/Makefile
 create mode 100644 tools/um/Targets
 create mode 100644 tools/um/include/lkl.h
 create mode 100644 tools/um/include/lkl_host.h
 create mode 100644 tools/um/lib/Build
 create mode 100644 tools/um/lib/fs.c
 create mode 100644 tools/um/lib/jmp_buf.c
 create mode 100644 tools/um/lib/jmp_buf.h
 create mode 100644 tools/um/lib/posix-host.c
 create mode 100644 tools/um/lib/utils.c
 create mode 100644 tools/um/tests/Build
 create mode 100644 tools/um/tests/boot.c
 create mode 100755 tools/um/tests/boot.sh
 create mode 100644 tools/um/tests/cla.c
 create mode 100644 tools/um/tests/cla.h
 create mode 100644 tools/um/tests/disk.c
 create mode 100755 tools/um/tests/disk.sh
 create mode 100755 tools/um/tests/run.py
 create mode 100644 tools/um/tests/tap13.py
 create mode 100644 tools/um/tests/test.c
 create mode 100644 tools/um/tests/test.h
 create mode 100644 tools/um/tests/test.sh
 create mode 100644 tools/um/uml/Build
 create mode 100644 tools/um/uml/drivers/Build
 rename {arch/um/os-Linux => tools/um/uml}/drivers/ethertap_user.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/drivers/tuntap_user.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/elf_aux.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/execvp.c (98%)
 rename {arch/um/os-Linux => tools/um/uml}/file.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/helper.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/irq.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/main.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/mem.c (100%)
 create mode 100644 tools/um/uml/nommu/Build
 create mode 100644 tools/um/uml/nommu/registers.c
 create mode 100644 tools/um/uml/nommu/unimplemented.c
 rename {arch/um/os-Linux => tools/um/uml}/process.c (99%)
 rename {arch/um/os-Linux => tools/um/uml}/registers.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/sigio.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/signal.c (96%)
 rename arch/um/os-Linux/skas/Makefile => tools/um/uml/skas/Build (56%)
 rename {arch/um/os-Linux => tools/um/uml}/skas/mem.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/skas/process.c (99%)
 rename {arch/um/os-Linux => tools/um/uml}/start_up.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/time.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/tty.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/umid.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/user_syms.c (99%)
 rename {arch/um/os-Linux => tools/um/uml}/util.c (89%)
 create mode 100644 tools/um/uml/x86/Build
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/mcontext.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/prctl.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/registers.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/task_size.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/tls.c (100%)

-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* [RFC v6 01/21] um: split build in kernel and host parts
  2020-09-24  7:12       ` Hajime Tazaki
@ 2020-09-24  7:12         ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  7:12 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch,
	Octavian Purdila, Hajime Tazaki

From: Octavian Purdila <tavi@cs.pub.ro>

This patch splits the UML build in two parts: a kernel part that
generates a relocatable object (linux.o) and a host build part that
links the relocatable object into an executable.

This allows us to simplify the linker script since we can now remove
the host bits (e.g. .rel sections). It also gives us greater
flexibility since it will be much easier to support other
architectures and OSes if we use the standard linker script.

The host build part has been implemented in tools/um so that we can
reuse the available host build infrastructure.

The patch also changes the UML build invocation, if before

 $ make ARCH=um defconfig
 $ make ARCH=um

was generating the executable now this only generates the relocatable
object.

To fully build UML use:

 $ make -C tools/um

which will generate tools/lkl/linux as the UML executable. To
statically compiled UML use:

 $ make -C tools/lkl UML_STATIC=y

Signed-off-by: Octavian Purdila <tavi@cs.pub.ro>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/Kconfig                  |  12 +--
 arch/um/Makefile                 |  14 ++-
 arch/um/include/asm/common.lds.S |  99 ------------------
 arch/um/kernel/dyn.lds.S         | 171 -------------------------------
 arch/um/kernel/uml.lds.S         | 115 ---------------------
 arch/um/kernel/vmlinux.lds.S     |  91 ++++++++++++++--
 scripts/link-vmlinux.sh          |  42 +++-----
 tools/um/Makefile                |  72 +++++++++++++
 tools/um/Targets                 |   4 +
 tools/um/uml/Build               |   0
 10 files changed, 184 insertions(+), 436 deletions(-)
 delete mode 100644 arch/um/include/asm/common.lds.S
 delete mode 100644 arch/um/kernel/dyn.lds.S
 delete mode 100644 arch/um/kernel/uml.lds.S
 create mode 100644 tools/um/Makefile
 create mode 100644 tools/um/Targets
 create mode 100644 tools/um/uml/Build

diff --git a/arch/um/Kconfig b/arch/um/Kconfig
index eb51fec75948..e41d31d0d875 100644
--- a/arch/um/Kconfig
+++ b/arch/um/Kconfig
@@ -20,6 +20,7 @@ config UML
 	select GENERIC_CLOCKEVENTS
 	select HAVE_GCC_PLUGINS
 	select TTY # Needed for line.c
+	select MODULE_REL_CRCS if MODVERSIONS
 
 config MMU
 	bool
@@ -79,17 +80,6 @@ config STATIC_LINK
 	  NOTE: This option is incompatible with some networking features which
 	  depend on features that require being dynamically loaded (like NSS).
 
-config LD_SCRIPT_STATIC
-	bool
-	default y
-	depends on STATIC_LINK
-
-config LD_SCRIPT_DYN
-	bool
-	default y
-	depends on !LD_SCRIPT_STATIC
-	select MODULE_REL_CRCS if MODVERSIONS
-
 config HOSTFS
 	tristate "Host filesystem"
 	help
diff --git a/arch/um/Makefile b/arch/um/Makefile
index 1cea46ff9bb7..fdfaff4633e8 100644
--- a/arch/um/Makefile
+++ b/arch/um/Makefile
@@ -95,14 +95,20 @@ KBUILD_CPPFLAGS += -I$(srctree)/$(HOST_DIR)/include \
 KERNEL_DEFINES = $(strip -Derrno=kernel_errno -Dsigprocmask=kernel_sigprocmask \
 			 -Dmktime=kernel_mktime $(ARCH_KERNEL_DEFINES))
 KBUILD_CFLAGS += $(KERNEL_DEFINES)
+LDFLAGS_vmlinux += -r
 
-PHONY += linux
+PHONY += linux.o
 
-all: linux
+all: linux.o
 
-linux: vmlinux
+linux.o: vmlinux
 	@echo '  LINK $@'
-	$(Q)ln -f $< $@
+	$(Q)$(OBJCOPY) -R .eh_frame $< $@
+
+install: linux.o
+	@echo "  INSTALL $(INSTALL_PATH)/lib/$<"
+	@mkdir -p $(INSTALL_PATH)/lib/
+	@cp $< $(INSTALL_PATH)/lib/
 
 define archhelp
   echo '* linux		- Binary kernel image (./linux) - for backward'
diff --git a/arch/um/include/asm/common.lds.S b/arch/um/include/asm/common.lds.S
deleted file mode 100644
index eca6c452a41b..000000000000
--- a/arch/um/include/asm/common.lds.S
+++ /dev/null
@@ -1,99 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#include <asm-generic/vmlinux.lds.h>
-
-  .fini      : { *(.fini)    } =0x9090
-  _etext = .;
-  PROVIDE (etext = .);
-
-  . = ALIGN(4096);
-  _sdata = .;
-  PROVIDE (sdata = .);
-
-  RO_DATA(4096)
-
-  .unprotected : { *(.unprotected) }
-  . = ALIGN(4096);
-  PROVIDE (_unprotected_end = .);
-
-  . = ALIGN(4096);
-  EXCEPTION_TABLE(0)
-
-  BUG_TABLE
-
-  .uml.setup.init : {
-	__uml_setup_start = .;
-	*(.uml.setup.init)
-	__uml_setup_end = .;
-  }
-	
-  .uml.help.init : {
-	__uml_help_start = .;
-	*(.uml.help.init)
-	__uml_help_end = .;
-  }
-	
-  .uml.postsetup.init : {
-	__uml_postsetup_start = .;
-	*(.uml.postsetup.init)
-	__uml_postsetup_end = .;
-  }
-	
-  .init.setup : {
-	INIT_SETUP(0)
-  }
-
-  PERCPU_SECTION(32)
-	
-  .initcall.init : {
-	INIT_CALLS
-  }
-
-  .con_initcall.init : {
-	CON_INITCALL
-  }
-
-  .exitcall : {
-	__exitcall_begin = .;
-	*(.exitcall.exit)
-	__exitcall_end = .;
-  }
-
-  .uml.exitcall : {
-	__uml_exitcall_begin = .;
-	*(.uml.exitcall.exit)
-	__uml_exitcall_end = .;
-  }
-
-  . = ALIGN(4);
-  .altinstructions : {
-	__alt_instructions = .;
-	*(.altinstructions)
-	__alt_instructions_end = .;
-  }
-  .altinstr_replacement : { *(.altinstr_replacement) }
-  /* .exit.text is discard at runtime, not link time, to deal with references
-     from .altinstructions and .eh_frame */
-  .exit.text : { EXIT_TEXT }
-  .exit.data : { *(.exit.data) }
-
-  .preinit_array : {
-	__preinit_array_start = .;
-	*(.preinit_array)
-	__preinit_array_end = .;
-  }
-  .init_array : {
-	__init_array_start = .;
-	*(.init_array)
-	__init_array_end = .;
-  }
-  .fini_array : {
-	__fini_array_start = .;
-	*(.fini_array)
-	__fini_array_end = .;
-  }
-
-   . = ALIGN(4096);
-  .init.ramfs : {
-	INIT_RAM_FS
-  }
-
diff --git a/arch/um/kernel/dyn.lds.S b/arch/um/kernel/dyn.lds.S
deleted file mode 100644
index f5001481010c..000000000000
--- a/arch/um/kernel/dyn.lds.S
+++ /dev/null
@@ -1,171 +0,0 @@
-#include <asm/vmlinux.lds.h>
-#include <asm/page.h>
-
-OUTPUT_FORMAT(ELF_FORMAT)
-OUTPUT_ARCH(ELF_ARCH)
-ENTRY(_start)
-jiffies = jiffies_64;
-
-SECTIONS
-{
-  PROVIDE (__executable_start = START);
-  . = START + SIZEOF_HEADERS;
-  .interp         : { *(.interp) }
-  __binary_start = .;
-  . = ALIGN(4096);		/* Init code and data */
-  _text = .;
-  INIT_TEXT_SECTION(PAGE_SIZE)
-
-  . = ALIGN(PAGE_SIZE);
-
-  /* Read-only sections, merged into text segment: */
-  .hash           : { *(.hash) }
-  .gnu.hash       : { *(.gnu.hash) }
-  .dynsym         : { *(.dynsym) }
-  .dynstr         : { *(.dynstr) }
-  .gnu.version    : { *(.gnu.version) }
-  .gnu.version_d  : { *(.gnu.version_d) }
-  .gnu.version_r  : { *(.gnu.version_r) }
-  .rel.init       : { *(.rel.init) }
-  .rela.init      : { *(.rela.init) }
-  .rel.text       : { *(.rel.text .rel.text.* .rel.gnu.linkonce.t.*) }
-  .rela.text      : { *(.rela.text .rela.text.* .rela.gnu.linkonce.t.*) }
-  .rel.fini       : { *(.rel.fini) }
-  .rela.fini      : { *(.rela.fini) }
-  .rel.rodata     : { *(.rel.rodata .rel.rodata.* .rel.gnu.linkonce.r.*) }
-  .rela.rodata    : { *(.rela.rodata .rela.rodata.* .rela.gnu.linkonce.r.*) }
-  .rel.data       : { *(.rel.data .rel.data.* .rel.gnu.linkonce.d.*) }
-  .rela.data      : { *(.rela.data .rela.data.* .rela.gnu.linkonce.d.*) }
-  .rel.tdata	  : { *(.rel.tdata .rel.tdata.* .rel.gnu.linkonce.td.*) }
-  .rela.tdata	  : { *(.rela.tdata .rela.tdata.* .rela.gnu.linkonce.td.*) }
-  .rel.tbss	  : { *(.rel.tbss .rel.tbss.* .rel.gnu.linkonce.tb.*) }
-  .rela.tbss	  : { *(.rela.tbss .rela.tbss.* .rela.gnu.linkonce.tb.*) }
-  .rel.ctors      : { *(.rel.ctors) }
-  .rela.ctors     : { *(.rela.ctors) }
-  .rel.dtors      : { *(.rel.dtors) }
-  .rela.dtors     : { *(.rela.dtors) }
-  .rel.got        : { *(.rel.got) }
-  .rela.got       : { *(.rela.got) }
-  .rel.bss        : { *(.rel.bss .rel.bss.* .rel.gnu.linkonce.b.*) }
-  .rela.bss       : { *(.rela.bss .rela.bss.* .rela.gnu.linkonce.b.*) }
-  .rel.plt : {
-	*(.rel.plt)
-	PROVIDE_HIDDEN(__rel_iplt_start = .);
-	*(.rel.iplt)
-	PROVIDE_HIDDEN(__rel_iplt_end = .);
-  }
-  .rela.plt : {
-	*(.rela.plt)
-	PROVIDE_HIDDEN(__rela_iplt_start = .);
-	*(.rela.iplt)
-	PROVIDE_HIDDEN(__rela_iplt_end = .);
-  }
-  .init           : {
-    KEEP (*(.init))
-  } =0x90909090
-  .plt            : { *(.plt) }
-  .text           : {
-    _stext = .;
-    TEXT_TEXT
-    SCHED_TEXT
-    CPUIDLE_TEXT
-    LOCK_TEXT
-    IRQENTRY_TEXT
-    SOFTIRQENTRY_TEXT
-    *(.fixup)
-    *(.stub .text.* .gnu.linkonce.t.*)
-    /* .gnu.warning sections are handled specially by elf32.em.  */
-    *(.gnu.warning)
-
-    . = ALIGN(PAGE_SIZE);
-  } =0x90909090
-  . = ALIGN(PAGE_SIZE);
-  .syscall_stub : {
-	__syscall_stub_start = .;
-	*(.__syscall_stub*)
-	__syscall_stub_end = .;
-  }
-  .fini           : {
-    KEEP (*(.fini))
-  } =0x90909090
-
-  .kstrtab : { *(.kstrtab) }
-
-  #include <asm/common.lds.S>
-
-  __init_begin = .;
-  init.data : { INIT_DATA }
-  __init_end = .;
-
-  /* Ensure the __preinit_array_start label is properly aligned.  We
-     could instead move the label definition inside the section, but
-     the linker would then create the section even if it turns out to
-     be empty, which isn't pretty.  */
-  . = ALIGN(32 / 8);
-  .preinit_array     : { *(.preinit_array) }
-  .init_array     : { *(.init_array) }
-  .fini_array     : { *(.fini_array) }
-  .data           : {
-    INIT_TASK_DATA(KERNEL_STACK_SIZE)
-    . = ALIGN(KERNEL_STACK_SIZE);
-    *(.data..init_irqstack)
-    DATA_DATA
-    *(.data.* .gnu.linkonce.d.*)
-    SORT(CONSTRUCTORS)
-  }
-  .data1          : { *(.data1) }
-  .tdata	  : { *(.tdata .tdata.* .gnu.linkonce.td.*) }
-  .tbss		  : { *(.tbss .tbss.* .gnu.linkonce.tb.*) *(.tcommon) }
-  .eh_frame       : { KEEP (*(.eh_frame)) }
-  .gcc_except_table   : { *(.gcc_except_table) }
-  .dynamic        : { *(.dynamic) }
-  .ctors          : {
-    /* gcc uses crtbegin.o to find the start of
-       the constructors, so we make sure it is
-       first.  Because this is a wildcard, it
-       doesn't matter if the user does not
-       actually link against crtbegin.o; the
-       linker won't look for a file to match a
-       wildcard.  The wildcard also means that it
-       doesn't matter which directory crtbegin.o
-       is in.  */
-    KEEP (*crtbegin.o(.ctors))
-    /* We don't want to include the .ctor section from
-       from the crtend.o file until after the sorted ctors.
-       The .ctor section from the crtend file contains the
-       end of ctors marker and it must be last */
-    KEEP (*(EXCLUDE_FILE (*crtend.o ) .ctors))
-    KEEP (*(SORT(.ctors.*)))
-    KEEP (*(.ctors))
-  }
-  .dtors          : {
-    KEEP (*crtbegin.o(.dtors))
-    KEEP (*(EXCLUDE_FILE (*crtend.o ) .dtors))
-    KEEP (*(SORT(.dtors.*)))
-    KEEP (*(.dtors))
-  }
-  .jcr            : { KEEP (*(.jcr)) }
-  .got            : { *(.got.plt) *(.got) }
-  _edata = .;
-  PROVIDE (edata = .);
-  .bss            : {
-   __bss_start = .;
-   *(.dynbss)
-   *(.bss .bss.* .gnu.linkonce.b.*)
-   *(COMMON)
-   /* Align here to ensure that the .bss section occupies space up to
-      _end.  Align after .bss to ensure correct alignment even if the
-      .bss section disappears because there are no input sections.  */
-   . = ALIGN(32 / 8);
-  . = ALIGN(32 / 8);
-  }
-   __bss_stop = .;
-  _end = .;
-  PROVIDE (end = .);
-
-  STABS_DEBUG
-
-  DWARF_DEBUG
-
-  DISCARDS
-}
diff --git a/arch/um/kernel/uml.lds.S b/arch/um/kernel/uml.lds.S
deleted file mode 100644
index 3b6dab3d4501..000000000000
--- a/arch/um/kernel/uml.lds.S
+++ /dev/null
@@ -1,115 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#include <asm/vmlinux.lds.h>
-#include <asm/page.h>
-
-OUTPUT_FORMAT(ELF_FORMAT)
-OUTPUT_ARCH(ELF_ARCH)
-ENTRY(_start)
-jiffies = jiffies_64;
-
-SECTIONS
-{
-  /* This must contain the right address - not quite the default ELF one.*/
-  PROVIDE (__executable_start = START);
-  /* Static binaries stick stuff here, like the sigreturn trampoline,
-   * invisibly to objdump.  So, just make __binary_start equal to the very
-   * beginning of the executable, and if there are unmapped pages after this,
-   * they are forever unusable.
-   */
-  __binary_start = START;
-
-  . = START + SIZEOF_HEADERS;
-  . = ALIGN(PAGE_SIZE);
-
-  _text = .;
-  INIT_TEXT_SECTION(0)
-
-  .text      :
-  {
-    _stext = .;
-    TEXT_TEXT
-    SCHED_TEXT
-    CPUIDLE_TEXT
-    LOCK_TEXT
-    IRQENTRY_TEXT
-    SOFTIRQENTRY_TEXT
-    *(.fixup)
-    /* .gnu.warning sections are handled specially by elf32.em.  */
-    *(.gnu.warning)
-    *(.gnu.linkonce.t*)
-  }
-
-  . = ALIGN(PAGE_SIZE);
-  .syscall_stub : {
-	__syscall_stub_start = .;
-	*(.__syscall_stub*)
-	__syscall_stub_end = .;
-  }
-
-  /*
-   * These are needed even in a static link, even if they wind up being empty.
-   * Newer glibc needs these __rel{,a}_iplt_{start,end} symbols.
-   */
-  .rel.plt : {
-	*(.rel.plt)
-	PROVIDE_HIDDEN(__rel_iplt_start = .);
-	*(.rel.iplt)
-	PROVIDE_HIDDEN(__rel_iplt_end = .);
-  }
-  .rela.plt : {
-	*(.rela.plt)
-	PROVIDE_HIDDEN(__rela_iplt_start = .);
-	*(.rela.iplt)
-	PROVIDE_HIDDEN(__rela_iplt_end = .);
-  }
-
-  #include <asm/common.lds.S>
-
-  __init_begin = .;
-  init.data : { INIT_DATA }
-  __init_end = .;
-
-  .data    :
-  {
-    INIT_TASK_DATA(KERNEL_STACK_SIZE)
-    . = ALIGN(KERNEL_STACK_SIZE);
-    *(.data..init_irqstack)
-    DATA_DATA
-    *(.gnu.linkonce.d*)
-    CONSTRUCTORS
-  }
-  .data1   : { *(.data1) }
-  .ctors         :
-  {
-    *(.ctors)
-  }
-  .dtors         :
-  {
-    *(.dtors)
-  }
-
-  .got           : { *(.got.plt) *(.got) }
-  .dynamic       : { *(.dynamic) }
-  .tdata	  : { *(.tdata .tdata.* .gnu.linkonce.td.*) }
-  .tbss		  : { *(.tbss .tbss.* .gnu.linkonce.tb.*) *(.tcommon) }
-  /* We want the small data sections together, so single-instruction offsets
-     can access them all, and initialized data all before uninitialized, so
-     we can shorten the on-disk segment size.  */
-  .sdata     : { *(.sdata) }
-  _edata  =  .;
-  PROVIDE (edata = .);
-  . = ALIGN(PAGE_SIZE);
-  __bss_start = .;
-  PROVIDE(_bss_start = .);
-  SBSS(0)
-  BSS(0)
-   __bss_stop = .;
-  _end = .;
-  PROVIDE (end = .);
-
-  STABS_DEBUG
-
-  DWARF_DEBUG
-
-  DISCARDS
-}
diff --git a/arch/um/kernel/vmlinux.lds.S b/arch/um/kernel/vmlinux.lds.S
index 16e49bfa2b42..aaedd66772fa 100644
--- a/arch/um/kernel/vmlinux.lds.S
+++ b/arch/um/kernel/vmlinux.lds.S
@@ -1,8 +1,87 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <asm/vmlinux.lds.h>
+#include <asm/thread_info.h>
+#include <asm/page.h>
+#include <asm/thread_info.h>
+#include <asm/cache.h>
+#include <linux/export.h>
 
-KERNEL_STACK_SIZE = 4096 * (1 << CONFIG_KERNEL_STACK_ORDER);
+OUTPUT_FORMAT(ELF_FORMAT)
+jiffies = jiffies_64;
 
-#ifdef CONFIG_LD_SCRIPT_STATIC
-#include "uml.lds.S"
-#else
-#include "dyn.lds.S"
-#endif
+
+SECTIONS
+{
+	__init_begin = . ;
+	HEAD_TEXT_SECTION
+	INIT_TEXT_SECTION(PAGE_SIZE)
+	INIT_DATA_SECTION(16)
+	PERCPU_SECTION(L1_CACHE_BYTES)
+	__init_end = . ;
+
+	_stext = . ;
+	_text = . ;
+	.text : ALIGN(THREAD_SIZE)
+	{
+		__binary_start = . ;
+		TEXT_TEXT
+		SCHED_TEXT
+		LOCK_TEXT
+		CPUIDLE_TEXT
+		IRQENTRY_TEXT
+		SOFTIRQENTRY_TEXT
+	}
+	_etext = . ;
+
+	.syscall_stub : ALIGN(PAGE_SIZE)
+	{
+		__syscall_stub_start = .;
+		*(.__syscall_stub*)
+		__syscall_stub_end = .;
+	}
+
+	_sdata = . ;
+	RO_DATA(PAGE_SIZE)
+	RW_DATA(L1_CACHE_BYTES, PAGE_SIZE, THREAD_SIZE)
+	.data : ALIGN(THREAD_SIZE)  { }
+
+	EXCEPTION_TABLE(16)
+
+	.uml.init_irqstack : ALIGN(THREAD_SIZE) {
+		*(.data..init_irqstack)
+	}
+
+	.uml.setup.init : ALIGN(8) {
+		__uml_setup_start = .;
+		*(.uml.setup.init)
+		__uml_setup_end = .;
+	}
+
+	.uml.help.init : ALIGN(8)  {
+		__uml_help_start = .;
+		*(.uml.help.init)
+		__uml_help_end = .;
+	}
+
+	.uml.postsetup.init : ALIGN(8) {
+		__uml_postsetup_start = .;
+		*(.uml.postsetup.init)
+		__uml_postsetup_end = .;
+	}
+
+	.uml.exitcall : ALIGN(8) {
+		__uml_exitcall_begin = .;
+		*(.uml.exitcall.exit)
+		__uml_exitcall_end = .;
+	}
+	_edata = . ;
+
+	BSS_SECTION(0, 0, 0)
+	_end = . ;
+
+	STABS_DEBUG
+
+	DWARF_DEBUG
+
+	DISCARDS
+}
diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
index e6e2d9e5ff48..903fc8053d07 100755
--- a/scripts/link-vmlinux.sh
+++ b/scripts/link-vmlinux.sh
@@ -102,36 +102,18 @@ vmlinux_link()
 		strip_debug=-Wl,--strip-debug
 	fi
 
-	if [ "${SRCARCH}" != "um" ]; then
-		objects="--whole-archive			\
-			${KBUILD_VMLINUX_OBJS}			\
-			--no-whole-archive			\
-			--start-group				\
-			${KBUILD_VMLINUX_LIBS}			\
-			--end-group				\
-			${@}"
-
-		${LD} ${KBUILD_LDFLAGS} ${LDFLAGS_vmlinux}	\
-			${strip_debug#-Wl,}			\
-			-o ${output}				\
-			-T ${lds} ${objects}
-	else
-		objects="-Wl,--whole-archive			\
-			${KBUILD_VMLINUX_OBJS}			\
-			-Wl,--no-whole-archive			\
-			-Wl,--start-group			\
-			${KBUILD_VMLINUX_LIBS}			\
-			-Wl,--end-group				\
-			${@}"
-
-		${CC} ${CFLAGS_vmlinux}				\
-			${strip_debug}				\
-			-o ${output}				\
-			-Wl,-T,${lds}				\
-			${objects}				\
-			-lutil -lrt -lpthread
-		rm -f linux
-	fi
+	objects="--whole-archive			\
+		${KBUILD_VMLINUX_OBJS}			\
+		--no-whole-archive			\
+		--start-group				\
+		${KBUILD_VMLINUX_LIBS}			\
+		--end-group				\
+		${@}"
+
+	${LD} ${KBUILD_LDFLAGS} ${LDFLAGS_vmlinux}	\
+		${strip_debug#-Wl,}			\
+		-o ${output}				\
+		-T ${lds} ${objects}
 }
 
 # generate .BTF typeinfo from DWARF debuginfo
diff --git a/tools/um/Makefile b/tools/um/Makefile
new file mode 100644
index 000000000000..a63466ef7ef5
--- /dev/null
+++ b/tools/um/Makefile
@@ -0,0 +1,72 @@
+# Do not use make's built-in rules
+# (this improves performance and avoids hard-to-debug behaviour);
+# also do not print "Entering directory..." messages from make
+.SUFFIXES:
+MAKEFLAGS += -r --no-print-directory
+
+KCONFIG?=defconfig
+
+ifneq ($(silent),1)
+  ifneq ($(V),1)
+	QUIET_AUTOCONF       = @echo '  AUTOCONF '$@;
+	Q = @
+  endif
+endif
+
+PREFIX   := /usr
+
+ifeq (,$(srctree))
+  srctree := $(patsubst %/,%,$(dir $(shell pwd)))
+  srctree := $(patsubst %/,%,$(dir $(srctree)))
+endif
+export srctree
+
+-include ../scripts/Makefile.include
+
+# OUTPUT fixup should be *after* include ../scripts/Makefile.include
+ifneq ($(OUTPUT),)
+  OUTPUT := $(OUTPUT)/tools/um/
+else
+  OUTPUT := $(CURDIR)/
+endif
+export OUTPUT
+export objtree := $(OUTPUT)/../..
+
+
+all:
+
+export CFLAGS += -I$(OUTPUT)/include -Iinclude -Wall -g -O2 -Wextra -fPIC \
+	 -Wno-unused-parameter \
+	 -Wno-missing-field-initializers -fno-strict-aliasing
+
+-include Targets
+
+TARGETS := $(progs-y:%=$(OUTPUT)%)
+TARGETS += $(libs-y:%=$(OUTPUT)%)
+all: $(TARGETS)
+
+# rule to build linux.o
+$(OUTPUT)lib/linux.o:
+	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) $(KCONFIG)
+	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) install INSTALL_PATH=$(OUTPUT)
+
+$(OUTPUT)liblinux.a: $(OUTPUT)lib/linux.o $(OUTPUT)uml/liblinux-in.o
+	$(QUIET_AR)$(AR) -rc $@ $^
+
+# rule to link programs
+$(OUTPUT)%: $(OUTPUT)%-in.o $(OUTPUT)liblinux.a
+	$(QUIET_LINK)$(CC) $(LDFLAGS) $(LDFLAGS_$(notdir $*)-y) -o $@ $^ $(LDLIBS) $(LDLIBS_$(notdir $*)-y)
+
+$(OUTPUT)%-in.o: FORCE
+	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
+
+clean:
+	$(call QUIET_CLEAN, objects)find $(OUTPUT) -name '*.o' -delete -o -name '\.*.cmd'\
+	 -delete -o -name '\.*.d' -delete
+	$(call QUIET_CLEAN, liblinux.a)$(RM) $(OUTPUT)/liblinux.a
+	$(call QUIET_CLEAN, $(TARGETS))$(RM) $(TARGETS)
+
+FORCE: ;
+.PHONY: all clean FORCE
+.IGNORE: clean
+.NOTPARALLEL : lib/linux.o
diff --git a/tools/um/Targets b/tools/um/Targets
new file mode 100644
index 000000000000..a4711f1ef422
--- /dev/null
+++ b/tools/um/Targets
@@ -0,0 +1,4 @@
+progs-y += uml/linux
+LDLIBS_linux-y := -lrt -lpthread -lutil
+LDFLAGS_linux-y := -no-pie -Wl,--wrap,malloc -Wl,--wrap,free -Wl,--wrap,calloc
+LDFLAGS_linux-$(UML_STATIC) += -static
diff --git a/tools/um/uml/Build b/tools/um/uml/Build
new file mode 100644
index 000000000000..e69de29bb2d1
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v6 01/21] um: split build in kernel and host parts
@ 2020-09-24  7:12         ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  7:12 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: linux-arch, tavi.purdila, Octavian Purdila, retrage01,
	linux-kernel-library, Hajime Tazaki

From: Octavian Purdila <tavi@cs.pub.ro>

This patch splits the UML build in two parts: a kernel part that
generates a relocatable object (linux.o) and a host build part that
links the relocatable object into an executable.

This allows us to simplify the linker script since we can now remove
the host bits (e.g. .rel sections). It also gives us greater
flexibility since it will be much easier to support other
architectures and OSes if we use the standard linker script.

The host build part has been implemented in tools/um so that we can
reuse the available host build infrastructure.

The patch also changes the UML build invocation, if before

 $ make ARCH=um defconfig
 $ make ARCH=um

was generating the executable now this only generates the relocatable
object.

To fully build UML use:

 $ make -C tools/um

which will generate tools/lkl/linux as the UML executable. To
statically compiled UML use:

 $ make -C tools/lkl UML_STATIC=y

Signed-off-by: Octavian Purdila <tavi@cs.pub.ro>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/Kconfig                  |  12 +--
 arch/um/Makefile                 |  14 ++-
 arch/um/include/asm/common.lds.S |  99 ------------------
 arch/um/kernel/dyn.lds.S         | 171 -------------------------------
 arch/um/kernel/uml.lds.S         | 115 ---------------------
 arch/um/kernel/vmlinux.lds.S     |  91 ++++++++++++++--
 scripts/link-vmlinux.sh          |  42 +++-----
 tools/um/Makefile                |  72 +++++++++++++
 tools/um/Targets                 |   4 +
 tools/um/uml/Build               |   0
 10 files changed, 184 insertions(+), 436 deletions(-)
 delete mode 100644 arch/um/include/asm/common.lds.S
 delete mode 100644 arch/um/kernel/dyn.lds.S
 delete mode 100644 arch/um/kernel/uml.lds.S
 create mode 100644 tools/um/Makefile
 create mode 100644 tools/um/Targets
 create mode 100644 tools/um/uml/Build

diff --git a/arch/um/Kconfig b/arch/um/Kconfig
index eb51fec75948..e41d31d0d875 100644
--- a/arch/um/Kconfig
+++ b/arch/um/Kconfig
@@ -20,6 +20,7 @@ config UML
 	select GENERIC_CLOCKEVENTS
 	select HAVE_GCC_PLUGINS
 	select TTY # Needed for line.c
+	select MODULE_REL_CRCS if MODVERSIONS
 
 config MMU
 	bool
@@ -79,17 +80,6 @@ config STATIC_LINK
 	  NOTE: This option is incompatible with some networking features which
 	  depend on features that require being dynamically loaded (like NSS).
 
-config LD_SCRIPT_STATIC
-	bool
-	default y
-	depends on STATIC_LINK
-
-config LD_SCRIPT_DYN
-	bool
-	default y
-	depends on !LD_SCRIPT_STATIC
-	select MODULE_REL_CRCS if MODVERSIONS
-
 config HOSTFS
 	tristate "Host filesystem"
 	help
diff --git a/arch/um/Makefile b/arch/um/Makefile
index 1cea46ff9bb7..fdfaff4633e8 100644
--- a/arch/um/Makefile
+++ b/arch/um/Makefile
@@ -95,14 +95,20 @@ KBUILD_CPPFLAGS += -I$(srctree)/$(HOST_DIR)/include \
 KERNEL_DEFINES = $(strip -Derrno=kernel_errno -Dsigprocmask=kernel_sigprocmask \
 			 -Dmktime=kernel_mktime $(ARCH_KERNEL_DEFINES))
 KBUILD_CFLAGS += $(KERNEL_DEFINES)
+LDFLAGS_vmlinux += -r
 
-PHONY += linux
+PHONY += linux.o
 
-all: linux
+all: linux.o
 
-linux: vmlinux
+linux.o: vmlinux
 	@echo '  LINK $@'
-	$(Q)ln -f $< $@
+	$(Q)$(OBJCOPY) -R .eh_frame $< $@
+
+install: linux.o
+	@echo "  INSTALL $(INSTALL_PATH)/lib/$<"
+	@mkdir -p $(INSTALL_PATH)/lib/
+	@cp $< $(INSTALL_PATH)/lib/
 
 define archhelp
   echo '* linux		- Binary kernel image (./linux) - for backward'
diff --git a/arch/um/include/asm/common.lds.S b/arch/um/include/asm/common.lds.S
deleted file mode 100644
index eca6c452a41b..000000000000
--- a/arch/um/include/asm/common.lds.S
+++ /dev/null
@@ -1,99 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#include <asm-generic/vmlinux.lds.h>
-
-  .fini      : { *(.fini)    } =0x9090
-  _etext = .;
-  PROVIDE (etext = .);
-
-  . = ALIGN(4096);
-  _sdata = .;
-  PROVIDE (sdata = .);
-
-  RO_DATA(4096)
-
-  .unprotected : { *(.unprotected) }
-  . = ALIGN(4096);
-  PROVIDE (_unprotected_end = .);
-
-  . = ALIGN(4096);
-  EXCEPTION_TABLE(0)
-
-  BUG_TABLE
-
-  .uml.setup.init : {
-	__uml_setup_start = .;
-	*(.uml.setup.init)
-	__uml_setup_end = .;
-  }
-	
-  .uml.help.init : {
-	__uml_help_start = .;
-	*(.uml.help.init)
-	__uml_help_end = .;
-  }
-	
-  .uml.postsetup.init : {
-	__uml_postsetup_start = .;
-	*(.uml.postsetup.init)
-	__uml_postsetup_end = .;
-  }
-	
-  .init.setup : {
-	INIT_SETUP(0)
-  }
-
-  PERCPU_SECTION(32)
-	
-  .initcall.init : {
-	INIT_CALLS
-  }
-
-  .con_initcall.init : {
-	CON_INITCALL
-  }
-
-  .exitcall : {
-	__exitcall_begin = .;
-	*(.exitcall.exit)
-	__exitcall_end = .;
-  }
-
-  .uml.exitcall : {
-	__uml_exitcall_begin = .;
-	*(.uml.exitcall.exit)
-	__uml_exitcall_end = .;
-  }
-
-  . = ALIGN(4);
-  .altinstructions : {
-	__alt_instructions = .;
-	*(.altinstructions)
-	__alt_instructions_end = .;
-  }
-  .altinstr_replacement : { *(.altinstr_replacement) }
-  /* .exit.text is discard at runtime, not link time, to deal with references
-     from .altinstructions and .eh_frame */
-  .exit.text : { EXIT_TEXT }
-  .exit.data : { *(.exit.data) }
-
-  .preinit_array : {
-	__preinit_array_start = .;
-	*(.preinit_array)
-	__preinit_array_end = .;
-  }
-  .init_array : {
-	__init_array_start = .;
-	*(.init_array)
-	__init_array_end = .;
-  }
-  .fini_array : {
-	__fini_array_start = .;
-	*(.fini_array)
-	__fini_array_end = .;
-  }
-
-   . = ALIGN(4096);
-  .init.ramfs : {
-	INIT_RAM_FS
-  }
-
diff --git a/arch/um/kernel/dyn.lds.S b/arch/um/kernel/dyn.lds.S
deleted file mode 100644
index f5001481010c..000000000000
--- a/arch/um/kernel/dyn.lds.S
+++ /dev/null
@@ -1,171 +0,0 @@
-#include <asm/vmlinux.lds.h>
-#include <asm/page.h>
-
-OUTPUT_FORMAT(ELF_FORMAT)
-OUTPUT_ARCH(ELF_ARCH)
-ENTRY(_start)
-jiffies = jiffies_64;
-
-SECTIONS
-{
-  PROVIDE (__executable_start = START);
-  . = START + SIZEOF_HEADERS;
-  .interp         : { *(.interp) }
-  __binary_start = .;
-  . = ALIGN(4096);		/* Init code and data */
-  _text = .;
-  INIT_TEXT_SECTION(PAGE_SIZE)
-
-  . = ALIGN(PAGE_SIZE);
-
-  /* Read-only sections, merged into text segment: */
-  .hash           : { *(.hash) }
-  .gnu.hash       : { *(.gnu.hash) }
-  .dynsym         : { *(.dynsym) }
-  .dynstr         : { *(.dynstr) }
-  .gnu.version    : { *(.gnu.version) }
-  .gnu.version_d  : { *(.gnu.version_d) }
-  .gnu.version_r  : { *(.gnu.version_r) }
-  .rel.init       : { *(.rel.init) }
-  .rela.init      : { *(.rela.init) }
-  .rel.text       : { *(.rel.text .rel.text.* .rel.gnu.linkonce.t.*) }
-  .rela.text      : { *(.rela.text .rela.text.* .rela.gnu.linkonce.t.*) }
-  .rel.fini       : { *(.rel.fini) }
-  .rela.fini      : { *(.rela.fini) }
-  .rel.rodata     : { *(.rel.rodata .rel.rodata.* .rel.gnu.linkonce.r.*) }
-  .rela.rodata    : { *(.rela.rodata .rela.rodata.* .rela.gnu.linkonce.r.*) }
-  .rel.data       : { *(.rel.data .rel.data.* .rel.gnu.linkonce.d.*) }
-  .rela.data      : { *(.rela.data .rela.data.* .rela.gnu.linkonce.d.*) }
-  .rel.tdata	  : { *(.rel.tdata .rel.tdata.* .rel.gnu.linkonce.td.*) }
-  .rela.tdata	  : { *(.rela.tdata .rela.tdata.* .rela.gnu.linkonce.td.*) }
-  .rel.tbss	  : { *(.rel.tbss .rel.tbss.* .rel.gnu.linkonce.tb.*) }
-  .rela.tbss	  : { *(.rela.tbss .rela.tbss.* .rela.gnu.linkonce.tb.*) }
-  .rel.ctors      : { *(.rel.ctors) }
-  .rela.ctors     : { *(.rela.ctors) }
-  .rel.dtors      : { *(.rel.dtors) }
-  .rela.dtors     : { *(.rela.dtors) }
-  .rel.got        : { *(.rel.got) }
-  .rela.got       : { *(.rela.got) }
-  .rel.bss        : { *(.rel.bss .rel.bss.* .rel.gnu.linkonce.b.*) }
-  .rela.bss       : { *(.rela.bss .rela.bss.* .rela.gnu.linkonce.b.*) }
-  .rel.plt : {
-	*(.rel.plt)
-	PROVIDE_HIDDEN(__rel_iplt_start = .);
-	*(.rel.iplt)
-	PROVIDE_HIDDEN(__rel_iplt_end = .);
-  }
-  .rela.plt : {
-	*(.rela.plt)
-	PROVIDE_HIDDEN(__rela_iplt_start = .);
-	*(.rela.iplt)
-	PROVIDE_HIDDEN(__rela_iplt_end = .);
-  }
-  .init           : {
-    KEEP (*(.init))
-  } =0x90909090
-  .plt            : { *(.plt) }
-  .text           : {
-    _stext = .;
-    TEXT_TEXT
-    SCHED_TEXT
-    CPUIDLE_TEXT
-    LOCK_TEXT
-    IRQENTRY_TEXT
-    SOFTIRQENTRY_TEXT
-    *(.fixup)
-    *(.stub .text.* .gnu.linkonce.t.*)
-    /* .gnu.warning sections are handled specially by elf32.em.  */
-    *(.gnu.warning)
-
-    . = ALIGN(PAGE_SIZE);
-  } =0x90909090
-  . = ALIGN(PAGE_SIZE);
-  .syscall_stub : {
-	__syscall_stub_start = .;
-	*(.__syscall_stub*)
-	__syscall_stub_end = .;
-  }
-  .fini           : {
-    KEEP (*(.fini))
-  } =0x90909090
-
-  .kstrtab : { *(.kstrtab) }
-
-  #include <asm/common.lds.S>
-
-  __init_begin = .;
-  init.data : { INIT_DATA }
-  __init_end = .;
-
-  /* Ensure the __preinit_array_start label is properly aligned.  We
-     could instead move the label definition inside the section, but
-     the linker would then create the section even if it turns out to
-     be empty, which isn't pretty.  */
-  . = ALIGN(32 / 8);
-  .preinit_array     : { *(.preinit_array) }
-  .init_array     : { *(.init_array) }
-  .fini_array     : { *(.fini_array) }
-  .data           : {
-    INIT_TASK_DATA(KERNEL_STACK_SIZE)
-    . = ALIGN(KERNEL_STACK_SIZE);
-    *(.data..init_irqstack)
-    DATA_DATA
-    *(.data.* .gnu.linkonce.d.*)
-    SORT(CONSTRUCTORS)
-  }
-  .data1          : { *(.data1) }
-  .tdata	  : { *(.tdata .tdata.* .gnu.linkonce.td.*) }
-  .tbss		  : { *(.tbss .tbss.* .gnu.linkonce.tb.*) *(.tcommon) }
-  .eh_frame       : { KEEP (*(.eh_frame)) }
-  .gcc_except_table   : { *(.gcc_except_table) }
-  .dynamic        : { *(.dynamic) }
-  .ctors          : {
-    /* gcc uses crtbegin.o to find the start of
-       the constructors, so we make sure it is
-       first.  Because this is a wildcard, it
-       doesn't matter if the user does not
-       actually link against crtbegin.o; the
-       linker won't look for a file to match a
-       wildcard.  The wildcard also means that it
-       doesn't matter which directory crtbegin.o
-       is in.  */
-    KEEP (*crtbegin.o(.ctors))
-    /* We don't want to include the .ctor section from
-       from the crtend.o file until after the sorted ctors.
-       The .ctor section from the crtend file contains the
-       end of ctors marker and it must be last */
-    KEEP (*(EXCLUDE_FILE (*crtend.o ) .ctors))
-    KEEP (*(SORT(.ctors.*)))
-    KEEP (*(.ctors))
-  }
-  .dtors          : {
-    KEEP (*crtbegin.o(.dtors))
-    KEEP (*(EXCLUDE_FILE (*crtend.o ) .dtors))
-    KEEP (*(SORT(.dtors.*)))
-    KEEP (*(.dtors))
-  }
-  .jcr            : { KEEP (*(.jcr)) }
-  .got            : { *(.got.plt) *(.got) }
-  _edata = .;
-  PROVIDE (edata = .);
-  .bss            : {
-   __bss_start = .;
-   *(.dynbss)
-   *(.bss .bss.* .gnu.linkonce.b.*)
-   *(COMMON)
-   /* Align here to ensure that the .bss section occupies space up to
-      _end.  Align after .bss to ensure correct alignment even if the
-      .bss section disappears because there are no input sections.  */
-   . = ALIGN(32 / 8);
-  . = ALIGN(32 / 8);
-  }
-   __bss_stop = .;
-  _end = .;
-  PROVIDE (end = .);
-
-  STABS_DEBUG
-
-  DWARF_DEBUG
-
-  DISCARDS
-}
diff --git a/arch/um/kernel/uml.lds.S b/arch/um/kernel/uml.lds.S
deleted file mode 100644
index 3b6dab3d4501..000000000000
--- a/arch/um/kernel/uml.lds.S
+++ /dev/null
@@ -1,115 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#include <asm/vmlinux.lds.h>
-#include <asm/page.h>
-
-OUTPUT_FORMAT(ELF_FORMAT)
-OUTPUT_ARCH(ELF_ARCH)
-ENTRY(_start)
-jiffies = jiffies_64;
-
-SECTIONS
-{
-  /* This must contain the right address - not quite the default ELF one.*/
-  PROVIDE (__executable_start = START);
-  /* Static binaries stick stuff here, like the sigreturn trampoline,
-   * invisibly to objdump.  So, just make __binary_start equal to the very
-   * beginning of the executable, and if there are unmapped pages after this,
-   * they are forever unusable.
-   */
-  __binary_start = START;
-
-  . = START + SIZEOF_HEADERS;
-  . = ALIGN(PAGE_SIZE);
-
-  _text = .;
-  INIT_TEXT_SECTION(0)
-
-  .text      :
-  {
-    _stext = .;
-    TEXT_TEXT
-    SCHED_TEXT
-    CPUIDLE_TEXT
-    LOCK_TEXT
-    IRQENTRY_TEXT
-    SOFTIRQENTRY_TEXT
-    *(.fixup)
-    /* .gnu.warning sections are handled specially by elf32.em.  */
-    *(.gnu.warning)
-    *(.gnu.linkonce.t*)
-  }
-
-  . = ALIGN(PAGE_SIZE);
-  .syscall_stub : {
-	__syscall_stub_start = .;
-	*(.__syscall_stub*)
-	__syscall_stub_end = .;
-  }
-
-  /*
-   * These are needed even in a static link, even if they wind up being empty.
-   * Newer glibc needs these __rel{,a}_iplt_{start,end} symbols.
-   */
-  .rel.plt : {
-	*(.rel.plt)
-	PROVIDE_HIDDEN(__rel_iplt_start = .);
-	*(.rel.iplt)
-	PROVIDE_HIDDEN(__rel_iplt_end = .);
-  }
-  .rela.plt : {
-	*(.rela.plt)
-	PROVIDE_HIDDEN(__rela_iplt_start = .);
-	*(.rela.iplt)
-	PROVIDE_HIDDEN(__rela_iplt_end = .);
-  }
-
-  #include <asm/common.lds.S>
-
-  __init_begin = .;
-  init.data : { INIT_DATA }
-  __init_end = .;
-
-  .data    :
-  {
-    INIT_TASK_DATA(KERNEL_STACK_SIZE)
-    . = ALIGN(KERNEL_STACK_SIZE);
-    *(.data..init_irqstack)
-    DATA_DATA
-    *(.gnu.linkonce.d*)
-    CONSTRUCTORS
-  }
-  .data1   : { *(.data1) }
-  .ctors         :
-  {
-    *(.ctors)
-  }
-  .dtors         :
-  {
-    *(.dtors)
-  }
-
-  .got           : { *(.got.plt) *(.got) }
-  .dynamic       : { *(.dynamic) }
-  .tdata	  : { *(.tdata .tdata.* .gnu.linkonce.td.*) }
-  .tbss		  : { *(.tbss .tbss.* .gnu.linkonce.tb.*) *(.tcommon) }
-  /* We want the small data sections together, so single-instruction offsets
-     can access them all, and initialized data all before uninitialized, so
-     we can shorten the on-disk segment size.  */
-  .sdata     : { *(.sdata) }
-  _edata  =  .;
-  PROVIDE (edata = .);
-  . = ALIGN(PAGE_SIZE);
-  __bss_start = .;
-  PROVIDE(_bss_start = .);
-  SBSS(0)
-  BSS(0)
-   __bss_stop = .;
-  _end = .;
-  PROVIDE (end = .);
-
-  STABS_DEBUG
-
-  DWARF_DEBUG
-
-  DISCARDS
-}
diff --git a/arch/um/kernel/vmlinux.lds.S b/arch/um/kernel/vmlinux.lds.S
index 16e49bfa2b42..aaedd66772fa 100644
--- a/arch/um/kernel/vmlinux.lds.S
+++ b/arch/um/kernel/vmlinux.lds.S
@@ -1,8 +1,87 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <asm/vmlinux.lds.h>
+#include <asm/thread_info.h>
+#include <asm/page.h>
+#include <asm/thread_info.h>
+#include <asm/cache.h>
+#include <linux/export.h>
 
-KERNEL_STACK_SIZE = 4096 * (1 << CONFIG_KERNEL_STACK_ORDER);
+OUTPUT_FORMAT(ELF_FORMAT)
+jiffies = jiffies_64;
 
-#ifdef CONFIG_LD_SCRIPT_STATIC
-#include "uml.lds.S"
-#else
-#include "dyn.lds.S"
-#endif
+
+SECTIONS
+{
+	__init_begin = . ;
+	HEAD_TEXT_SECTION
+	INIT_TEXT_SECTION(PAGE_SIZE)
+	INIT_DATA_SECTION(16)
+	PERCPU_SECTION(L1_CACHE_BYTES)
+	__init_end = . ;
+
+	_stext = . ;
+	_text = . ;
+	.text : ALIGN(THREAD_SIZE)
+	{
+		__binary_start = . ;
+		TEXT_TEXT
+		SCHED_TEXT
+		LOCK_TEXT
+		CPUIDLE_TEXT
+		IRQENTRY_TEXT
+		SOFTIRQENTRY_TEXT
+	}
+	_etext = . ;
+
+	.syscall_stub : ALIGN(PAGE_SIZE)
+	{
+		__syscall_stub_start = .;
+		*(.__syscall_stub*)
+		__syscall_stub_end = .;
+	}
+
+	_sdata = . ;
+	RO_DATA(PAGE_SIZE)
+	RW_DATA(L1_CACHE_BYTES, PAGE_SIZE, THREAD_SIZE)
+	.data : ALIGN(THREAD_SIZE)  { }
+
+	EXCEPTION_TABLE(16)
+
+	.uml.init_irqstack : ALIGN(THREAD_SIZE) {
+		*(.data..init_irqstack)
+	}
+
+	.uml.setup.init : ALIGN(8) {
+		__uml_setup_start = .;
+		*(.uml.setup.init)
+		__uml_setup_end = .;
+	}
+
+	.uml.help.init : ALIGN(8)  {
+		__uml_help_start = .;
+		*(.uml.help.init)
+		__uml_help_end = .;
+	}
+
+	.uml.postsetup.init : ALIGN(8) {
+		__uml_postsetup_start = .;
+		*(.uml.postsetup.init)
+		__uml_postsetup_end = .;
+	}
+
+	.uml.exitcall : ALIGN(8) {
+		__uml_exitcall_begin = .;
+		*(.uml.exitcall.exit)
+		__uml_exitcall_end = .;
+	}
+	_edata = . ;
+
+	BSS_SECTION(0, 0, 0)
+	_end = . ;
+
+	STABS_DEBUG
+
+	DWARF_DEBUG
+
+	DISCARDS
+}
diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
index e6e2d9e5ff48..903fc8053d07 100755
--- a/scripts/link-vmlinux.sh
+++ b/scripts/link-vmlinux.sh
@@ -102,36 +102,18 @@ vmlinux_link()
 		strip_debug=-Wl,--strip-debug
 	fi
 
-	if [ "${SRCARCH}" != "um" ]; then
-		objects="--whole-archive			\
-			${KBUILD_VMLINUX_OBJS}			\
-			--no-whole-archive			\
-			--start-group				\
-			${KBUILD_VMLINUX_LIBS}			\
-			--end-group				\
-			${@}"
-
-		${LD} ${KBUILD_LDFLAGS} ${LDFLAGS_vmlinux}	\
-			${strip_debug#-Wl,}			\
-			-o ${output}				\
-			-T ${lds} ${objects}
-	else
-		objects="-Wl,--whole-archive			\
-			${KBUILD_VMLINUX_OBJS}			\
-			-Wl,--no-whole-archive			\
-			-Wl,--start-group			\
-			${KBUILD_VMLINUX_LIBS}			\
-			-Wl,--end-group				\
-			${@}"
-
-		${CC} ${CFLAGS_vmlinux}				\
-			${strip_debug}				\
-			-o ${output}				\
-			-Wl,-T,${lds}				\
-			${objects}				\
-			-lutil -lrt -lpthread
-		rm -f linux
-	fi
+	objects="--whole-archive			\
+		${KBUILD_VMLINUX_OBJS}			\
+		--no-whole-archive			\
+		--start-group				\
+		${KBUILD_VMLINUX_LIBS}			\
+		--end-group				\
+		${@}"
+
+	${LD} ${KBUILD_LDFLAGS} ${LDFLAGS_vmlinux}	\
+		${strip_debug#-Wl,}			\
+		-o ${output}				\
+		-T ${lds} ${objects}
 }
 
 # generate .BTF typeinfo from DWARF debuginfo
diff --git a/tools/um/Makefile b/tools/um/Makefile
new file mode 100644
index 000000000000..a63466ef7ef5
--- /dev/null
+++ b/tools/um/Makefile
@@ -0,0 +1,72 @@
+# Do not use make's built-in rules
+# (this improves performance and avoids hard-to-debug behaviour);
+# also do not print "Entering directory..." messages from make
+.SUFFIXES:
+MAKEFLAGS += -r --no-print-directory
+
+KCONFIG?=defconfig
+
+ifneq ($(silent),1)
+  ifneq ($(V),1)
+	QUIET_AUTOCONF       = @echo '  AUTOCONF '$@;
+	Q = @
+  endif
+endif
+
+PREFIX   := /usr
+
+ifeq (,$(srctree))
+  srctree := $(patsubst %/,%,$(dir $(shell pwd)))
+  srctree := $(patsubst %/,%,$(dir $(srctree)))
+endif
+export srctree
+
+-include ../scripts/Makefile.include
+
+# OUTPUT fixup should be *after* include ../scripts/Makefile.include
+ifneq ($(OUTPUT),)
+  OUTPUT := $(OUTPUT)/tools/um/
+else
+  OUTPUT := $(CURDIR)/
+endif
+export OUTPUT
+export objtree := $(OUTPUT)/../..
+
+
+all:
+
+export CFLAGS += -I$(OUTPUT)/include -Iinclude -Wall -g -O2 -Wextra -fPIC \
+	 -Wno-unused-parameter \
+	 -Wno-missing-field-initializers -fno-strict-aliasing
+
+-include Targets
+
+TARGETS := $(progs-y:%=$(OUTPUT)%)
+TARGETS += $(libs-y:%=$(OUTPUT)%)
+all: $(TARGETS)
+
+# rule to build linux.o
+$(OUTPUT)lib/linux.o:
+	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) $(KCONFIG)
+	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) install INSTALL_PATH=$(OUTPUT)
+
+$(OUTPUT)liblinux.a: $(OUTPUT)lib/linux.o $(OUTPUT)uml/liblinux-in.o
+	$(QUIET_AR)$(AR) -rc $@ $^
+
+# rule to link programs
+$(OUTPUT)%: $(OUTPUT)%-in.o $(OUTPUT)liblinux.a
+	$(QUIET_LINK)$(CC) $(LDFLAGS) $(LDFLAGS_$(notdir $*)-y) -o $@ $^ $(LDLIBS) $(LDLIBS_$(notdir $*)-y)
+
+$(OUTPUT)%-in.o: FORCE
+	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
+
+clean:
+	$(call QUIET_CLEAN, objects)find $(OUTPUT) -name '*.o' -delete -o -name '\.*.cmd'\
+	 -delete -o -name '\.*.d' -delete
+	$(call QUIET_CLEAN, liblinux.a)$(RM) $(OUTPUT)/liblinux.a
+	$(call QUIET_CLEAN, $(TARGETS))$(RM) $(TARGETS)
+
+FORCE: ;
+.PHONY: all clean FORCE
+.IGNORE: clean
+.NOTPARALLEL : lib/linux.o
diff --git a/tools/um/Targets b/tools/um/Targets
new file mode 100644
index 000000000000..a4711f1ef422
--- /dev/null
+++ b/tools/um/Targets
@@ -0,0 +1,4 @@
+progs-y += uml/linux
+LDLIBS_linux-y := -lrt -lpthread -lutil
+LDFLAGS_linux-y := -no-pie -Wl,--wrap,malloc -Wl,--wrap,free -Wl,--wrap,calloc
+LDFLAGS_linux-$(UML_STATIC) += -static
diff --git a/tools/um/uml/Build b/tools/um/uml/Build
new file mode 100644
index 000000000000..e69de29bb2d1
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v6 02/21] um: add os init and exit calls
  2020-09-24  7:12       ` Hajime Tazaki
@ 2020-09-24  7:12         ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  7:12 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch,
	Octavian Purdila

From: Octavian Purdila <tavi@cs.pub.ro>

These calls are added in preparation for moving some of the code from
the kernel to the host build.

Signed-off-by: Octavian Purdila <tavi@cs.pub.ro>
---
 arch/um/include/shared/init.h | 14 ++++----------
 arch/um/kernel/reboot.c       |  5 +++++
 arch/um/kernel/um_arch.c      | 11 +++++++++++
 3 files changed, 20 insertions(+), 10 deletions(-)

diff --git a/arch/um/include/shared/init.h b/arch/um/include/shared/init.h
index c66de434a983..d09308330ca5 100644
--- a/arch/um/include/shared/init.h
+++ b/arch/um/include/shared/init.h
@@ -109,19 +109,13 @@ extern struct uml_param __uml_setup_start, __uml_setup_end;
 
 #ifdef __UM_HOST__
 
-#define __define_initcall(level,fn) \
-	static initcall_t __initcall_##fn __used \
-	__attribute__((__section__(".initcall" level ".init"))) = fn
-
-/* Userspace initcalls shouldn't depend on anything in the kernel, so we'll
- * make them run first.
- */
-#define __initcall(fn) __define_initcall("1", fn)
+#undef __uml_exit_call
+#define __uml_exit_call		__used __section(os_exitcalls)
+#define __init_call		__used __section(os_initcalls)
 
+#define __initcall(fn) static initcall_t __initcall_##fn __init_call = fn
 #define __exitcall(fn) static exitcall_t __exitcall_##fn __exit_call = fn
 
-#define __init_call	__used __section(.initcall.init)
-
 #endif
 
 #endif /* _LINUX_UML_INIT_H */
diff --git a/arch/um/kernel/reboot.c b/arch/um/kernel/reboot.c
index 48c0610d506e..5420aec411f4 100644
--- a/arch/um/kernel/reboot.c
+++ b/arch/um/kernel/reboot.c
@@ -35,10 +35,15 @@ static void kill_off_processes(void)
 	read_unlock(&tasklist_lock);
 }
 
+void __weak os_exitcalls(void)
+{
+}
+
 void uml_cleanup(void)
 {
 	kmalloc_ok = 0;
 	do_uml_exitcalls();
+	os_exitcalls();
 	kill_off_processes();
 }
 
diff --git a/arch/um/kernel/um_arch.c b/arch/um/kernel/um_arch.c
index 00141e70de56..e2cb76c03b25 100644
--- a/arch/um/kernel/um_arch.c
+++ b/arch/um/kernel/um_arch.c
@@ -377,3 +377,14 @@ void *text_poke(void *addr, const void *opcode, size_t len)
 void text_poke_sync(void)
 {
 }
+
+int __weak os_initcalls(void)
+{
+	return 0;
+}
+
+int __init run_os_initcalls(void)
+{
+	return os_initcalls();
+}
+early_initcall(run_os_initcalls);
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v6 02/21] um: add os init and exit calls
@ 2020-09-24  7:12         ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  7:12 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Octavian Purdila,
	retrage01

From: Octavian Purdila <tavi@cs.pub.ro>

These calls are added in preparation for moving some of the code from
the kernel to the host build.

Signed-off-by: Octavian Purdila <tavi@cs.pub.ro>
---
 arch/um/include/shared/init.h | 14 ++++----------
 arch/um/kernel/reboot.c       |  5 +++++
 arch/um/kernel/um_arch.c      | 11 +++++++++++
 3 files changed, 20 insertions(+), 10 deletions(-)

diff --git a/arch/um/include/shared/init.h b/arch/um/include/shared/init.h
index c66de434a983..d09308330ca5 100644
--- a/arch/um/include/shared/init.h
+++ b/arch/um/include/shared/init.h
@@ -109,19 +109,13 @@ extern struct uml_param __uml_setup_start, __uml_setup_end;
 
 #ifdef __UM_HOST__
 
-#define __define_initcall(level,fn) \
-	static initcall_t __initcall_##fn __used \
-	__attribute__((__section__(".initcall" level ".init"))) = fn
-
-/* Userspace initcalls shouldn't depend on anything in the kernel, so we'll
- * make them run first.
- */
-#define __initcall(fn) __define_initcall("1", fn)
+#undef __uml_exit_call
+#define __uml_exit_call		__used __section(os_exitcalls)
+#define __init_call		__used __section(os_initcalls)
 
+#define __initcall(fn) static initcall_t __initcall_##fn __init_call = fn
 #define __exitcall(fn) static exitcall_t __exitcall_##fn __exit_call = fn
 
-#define __init_call	__used __section(.initcall.init)
-
 #endif
 
 #endif /* _LINUX_UML_INIT_H */
diff --git a/arch/um/kernel/reboot.c b/arch/um/kernel/reboot.c
index 48c0610d506e..5420aec411f4 100644
--- a/arch/um/kernel/reboot.c
+++ b/arch/um/kernel/reboot.c
@@ -35,10 +35,15 @@ static void kill_off_processes(void)
 	read_unlock(&tasklist_lock);
 }
 
+void __weak os_exitcalls(void)
+{
+}
+
 void uml_cleanup(void)
 {
 	kmalloc_ok = 0;
 	do_uml_exitcalls();
+	os_exitcalls();
 	kill_off_processes();
 }
 
diff --git a/arch/um/kernel/um_arch.c b/arch/um/kernel/um_arch.c
index 00141e70de56..e2cb76c03b25 100644
--- a/arch/um/kernel/um_arch.c
+++ b/arch/um/kernel/um_arch.c
@@ -377,3 +377,14 @@ void *text_poke(void *addr, const void *opcode, size_t len)
 void text_poke_sync(void)
 {
 }
+
+int __weak os_initcalls(void)
+{
+	return 0;
+}
+
+int __init run_os_initcalls(void)
+{
+	return os_initcalls();
+}
+early_initcall(run_os_initcalls);
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v6 03/21] um: move arch/um/os-Linux dir to tools/um/uml
  2020-09-24  7:12       ` Hajime Tazaki
@ 2020-09-24  7:12         ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  7:12 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch, Hajime Tazaki

This patch moves underlying OS dependent part under arch/um to tools/um
directory so that arch/um code does not need to build host build
facilities (e.g., libc).

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/Makefile                              |  3 +-
 arch/um/drivers/Makefile                      |  2 +
 .../um/{os-Linux => }/drivers/ethertap_kern.c |  0
 arch/um/{os-Linux => }/drivers/tuntap_kern.c  |  0
 .../drivers => include/shared}/etap.h         |  0
 arch/um/include/shared/init.h                 |  5 ++
 .../drivers => include/shared}/tuntap.h       |  0
 arch/um/os-Linux/Makefile                     | 19 --------
 arch/um/os-Linux/drivers/Makefile             | 13 -----
 tools/um/Makefile                             |  6 ++-
 tools/um/uml/Build                            | 48 +++++++++++++++++++
 tools/um/uml/drivers/Build                    | 10 ++++
 .../um/uml}/drivers/ethertap_user.c           |  0
 .../um/uml}/drivers/tuntap_user.c             |  0
 {arch/um/os-Linux => tools/um/uml}/elf_aux.c  |  0
 {arch/um/os-Linux => tools/um/uml}/execvp.c   |  4 --
 {arch/um/os-Linux => tools/um/uml}/file.c     |  0
 {arch/um/os-Linux => tools/um/uml}/helper.c   |  0
 {arch/um/os-Linux => tools/um/uml}/irq.c      |  0
 {arch/um/os-Linux => tools/um/uml}/main.c     |  0
 {arch/um/os-Linux => tools/um/uml}/mem.c      |  0
 {arch/um/os-Linux => tools/um/uml}/process.c  |  0
 .../um/os-Linux => tools/um/uml}/registers.c  |  0
 {arch/um/os-Linux => tools/um/uml}/sigio.c    |  0
 {arch/um/os-Linux => tools/um/uml}/signal.c   |  0
 .../skas/Makefile => tools/um/uml/skas/Build  |  6 +--
 {arch/um/os-Linux => tools/um/uml}/skas/mem.c |  0
 .../os-Linux => tools/um/uml}/skas/process.c  |  3 +-
 {arch/um/os-Linux => tools/um/uml}/start_up.c |  0
 {arch/um/os-Linux => tools/um/uml}/time.c     |  0
 {arch/um/os-Linux => tools/um/uml}/tty.c      |  0
 {arch/um/os-Linux => tools/um/uml}/umid.c     |  0
 .../um/os-Linux => tools/um/uml}/user_syms.c  |  1 +
 {arch/um/os-Linux => tools/um/uml}/util.c     |  0
 34 files changed, 74 insertions(+), 46 deletions(-)
 rename arch/um/{os-Linux => }/drivers/ethertap_kern.c (100%)
 rename arch/um/{os-Linux => }/drivers/tuntap_kern.c (100%)
 rename arch/um/{os-Linux/drivers => include/shared}/etap.h (100%)
 rename arch/um/{os-Linux/drivers => include/shared}/tuntap.h (100%)
 delete mode 100644 arch/um/os-Linux/Makefile
 delete mode 100644 arch/um/os-Linux/drivers/Makefile
 create mode 100644 tools/um/uml/drivers/Build
 rename {arch/um/os-Linux => tools/um/uml}/drivers/ethertap_user.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/drivers/tuntap_user.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/elf_aux.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/execvp.c (98%)
 rename {arch/um/os-Linux => tools/um/uml}/file.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/helper.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/irq.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/main.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/mem.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/process.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/registers.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/sigio.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/signal.c (100%)
 rename arch/um/os-Linux/skas/Makefile => tools/um/uml/skas/Build (56%)
 rename {arch/um/os-Linux => tools/um/uml}/skas/mem.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/skas/process.c (99%)
 rename {arch/um/os-Linux => tools/um/uml}/start_up.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/time.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/tty.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/umid.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/user_syms.c (99%)
 rename {arch/um/os-Linux => tools/um/uml}/util.c (100%)

diff --git a/arch/um/Makefile b/arch/um/Makefile
index fdfaff4633e8..1f958a9280fe 100644
--- a/arch/um/Makefile
+++ b/arch/um/Makefile
@@ -24,8 +24,7 @@ OS := $(shell uname -s)
 SHELL := /bin/bash
 
 core-y			+= $(ARCH_DIR)/kernel/		\
-			   $(ARCH_DIR)/drivers/		\
-			   $(ARCH_DIR)/os-$(OS)/
+			   $(ARCH_DIR)/drivers/
 
 MODE_INCLUDE	+= -I$(srctree)/$(ARCH_DIR)/include/shared/skas
 
diff --git a/arch/um/drivers/Makefile b/arch/um/drivers/Makefile
index 2a249f619467..ae96c83e312d 100644
--- a/arch/um/drivers/Makefile
+++ b/arch/um/drivers/Makefile
@@ -48,6 +48,8 @@ obj-$(CONFIG_UML_NET_VECTOR) += vector.o
 obj-$(CONFIG_UML_NET_VDE) += vde.o
 obj-$(CONFIG_UML_NET_MCAST) += umcast.o
 obj-$(CONFIG_UML_NET_PCAP) += pcap.o
+obj-$(CONFIG_UML_NET_ETHERTAP) += ethertap_kern.o
+obj-$(CONFIG_UML_NET_TUNTAP) += tuntap_kern.o
 obj-$(CONFIG_UML_NET) += net.o 
 obj-$(CONFIG_MCONSOLE) += mconsole.o
 obj-$(CONFIG_MMAPPER) += mmapper_kern.o 
diff --git a/arch/um/os-Linux/drivers/ethertap_kern.c b/arch/um/drivers/ethertap_kern.c
similarity index 100%
rename from arch/um/os-Linux/drivers/ethertap_kern.c
rename to arch/um/drivers/ethertap_kern.c
diff --git a/arch/um/os-Linux/drivers/tuntap_kern.c b/arch/um/drivers/tuntap_kern.c
similarity index 100%
rename from arch/um/os-Linux/drivers/tuntap_kern.c
rename to arch/um/drivers/tuntap_kern.c
diff --git a/arch/um/os-Linux/drivers/etap.h b/arch/um/include/shared/etap.h
similarity index 100%
rename from arch/um/os-Linux/drivers/etap.h
rename to arch/um/include/shared/etap.h
diff --git a/arch/um/include/shared/init.h b/arch/um/include/shared/init.h
index d09308330ca5..305789667584 100644
--- a/arch/um/include/shared/init.h
+++ b/arch/um/include/shared/init.h
@@ -41,7 +41,12 @@
 typedef int (*initcall_t)(void);
 typedef void (*exitcall_t)(void);
 
+#ifndef __UM_HOST__
 #include <linux/compiler_types.h>
+#else
+#define __used                          __attribute__((__used__))
+#define __section(S)                    __attribute__((__section__(#S)))
+#endif
 
 /* These are for everybody (although not all archs will actually
    discard it in modules) */
diff --git a/arch/um/os-Linux/drivers/tuntap.h b/arch/um/include/shared/tuntap.h
similarity index 100%
rename from arch/um/os-Linux/drivers/tuntap.h
rename to arch/um/include/shared/tuntap.h
diff --git a/arch/um/os-Linux/Makefile b/arch/um/os-Linux/Makefile
deleted file mode 100644
index 839915b8c31c..000000000000
--- a/arch/um/os-Linux/Makefile
+++ /dev/null
@@ -1,19 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0
-# 
-# Copyright (C) 2000 - 2007 Jeff Dike (jdike@{addtoit,linux.intel}.com)
-#
-
-# Don't instrument UML-specific code
-KCOV_INSTRUMENT                := n
-
-obj-y = execvp.o file.o helper.o irq.o main.o mem.o process.o \
-	registers.o sigio.o signal.o start_up.o time.o tty.o \
-	umid.o user_syms.o util.o drivers/ skas/
-
-obj-$(CONFIG_ARCH_REUSE_HOST_VSYSCALL_AREA) += elf_aux.o
-
-USER_OBJS := $(user-objs-y) elf_aux.o execvp.o file.o helper.o irq.o \
-	main.o mem.o process.o registers.o sigio.o signal.o start_up.o time.o \
-	tty.o umid.o util.o
-
-include arch/um/scripts/Makefile.rules
diff --git a/arch/um/os-Linux/drivers/Makefile b/arch/um/os-Linux/drivers/Makefile
deleted file mode 100644
index d79e75f1b69a..000000000000
--- a/arch/um/os-Linux/drivers/Makefile
+++ /dev/null
@@ -1,13 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0
-# 
-# Copyright (C) 2000, 2002 Jeff Dike (jdike@karaya.com)
-#
-
-ethertap-objs := ethertap_kern.o ethertap_user.o
-tuntap-objs := tuntap_kern.o tuntap_user.o
-
-obj-y = 
-obj-$(CONFIG_UML_NET_ETHERTAP) += ethertap.o
-obj-$(CONFIG_UML_NET_TUNTAP) += tuntap.o
-
-include arch/um/scripts/Makefile.rules
diff --git a/tools/um/Makefile b/tools/um/Makefile
index a63466ef7ef5..07b9b93ed817 100644
--- a/tools/um/Makefile
+++ b/tools/um/Makefile
@@ -50,7 +50,7 @@ $(OUTPUT)lib/linux.o:
 	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) $(KCONFIG)
 	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) install INSTALL_PATH=$(OUTPUT)
 
-$(OUTPUT)liblinux.a: $(OUTPUT)lib/linux.o $(OUTPUT)uml/liblinux-in.o
+$(OUTPUT)liblinux.a: $(OUTPUT)lib/linux.o $(OUTPUT)uml/liblinux-in.o $(OUTPUT)lib/liblinux-in.o
 	$(QUIET_AR)$(AR) -rc $@ $^
 
 # rule to link programs
@@ -60,6 +60,10 @@ $(OUTPUT)%: $(OUTPUT)%-in.o $(OUTPUT)liblinux.a
 $(OUTPUT)%-in.o: FORCE
 	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
 
+# rule to build objects
+$(OUTPUT)%-in.o: $(OUTPUT)lib/linux.o
+	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
+
 clean:
 	$(call QUIET_CLEAN, objects)find $(OUTPUT) -name '*.o' -delete -o -name '\.*.cmd'\
 	 -delete -o -name '\.*.d' -delete
diff --git a/tools/um/uml/Build b/tools/um/uml/Build
index e69de29bb2d1..8e0607ee8b0a 100644
--- a/tools/um/uml/Build
+++ b/tools/um/uml/Build
@@ -0,0 +1,48 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Copyright (C) 2000 - 2007 Jeff Dike (jdike@{addtoit,linux.intel}.com)
+#
+
+# Don't instrument UML-specific code
+KCOV_INSTRUMENT                := n
+
+include $(objtree)/include/config/auto.conf
+
+liblinux-y += execvp.o file.o helper.o irq.o main.o mem.o process.o \
+	registers.o sigio.o signal.o start_up.o time.o tty.o \
+	umid.o user_syms.o util.o drivers/ skas/
+
+liblinux-$(CONFIG_ARCH_REUSE_HOST_VSYSCALL_AREA) += elf_aux.o
+
+export O = $(srctree)
+objtree := $(O)
+# reset CFLAGS
+CFLAGS := -g -O2
+
+# from arch/um/Makefile
+ARCH_DIR := arch/um
+HEADER_ARCH 	:= x86
+HOST_DIR := arch/$(HEADER_ARCH)
+ifdef CONFIG_64BIT
+  KBUILD_CFLAGS += -mcmodel=large
+endif
+KBUILD_CFLAGS += $(CFLAGS) $(CFLAGS-y) -D__arch_um__ \
+	$(ARCH_INCLUDE) $(MODE_INCLUDE) -Dvmap=kernel_vmap	\
+	-Dlongjmp=kernel_longjmp -Dsetjmp=kernel_setjmp \
+	-Din6addr_loopback=kernel_in6addr_loopback \
+	-Din6addr_any=kernel_in6addr_any -Dstrrchr=kernel_strrchr
+SHARED_HEADERS	:= $(ARCH_DIR)/include/shared
+MODE_INCLUDE	+= -I$(srctree)/$(ARCH_DIR)/include/shared/skas
+ARCH_INCLUDE	:= -I$(srctree)/$(SHARED_HEADERS)
+ARCH_INCLUDE	+= -I$(srctree)/$(HOST_DIR)/um/shared
+KBUILD_CPPFLAGS += -I$(srctree)/$(HOST_DIR)/um
+USER_CFLAGS := $(patsubst $(KERNEL_DEFINES),,$(patsubst -I%,,$(KBUILD_CFLAGS))) \
+		$(ARCH_INCLUDE) $(MODE_INCLUDE) $(filter -I%,$(CFLAGS)) \
+		-D_FILE_OFFSET_BITS=64 -idirafter $(srctree)/include/uapi \
+		-idirafter $(objtree)/include -D__KERNEL__ -D__UM_HOST__
+
+# from Makefile-os-Linux
+USER_CFLAGS += -D_GNU_SOURCE -D_LARGEFILE64_SOURCE
+
+# from Makefile.rules
+CFLAGS += $(USER_CFLAGS) -include $(srctree)/tools/include/linux/kern_levels.h -include user.h $(CFLAGS_$(basetarget).o)
diff --git a/tools/um/uml/drivers/Build b/tools/um/uml/drivers/Build
new file mode 100644
index 000000000000..c7a8eaa97d72
--- /dev/null
+++ b/tools/um/uml/drivers/Build
@@ -0,0 +1,10 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Copyright (C) 2000, 2002 Jeff Dike (jdike@karaya.com)
+#
+
+include $(objtree)/include/config/auto.conf
+
+
+liblinux-$(CONFIG_UML_NET_ETHERTAP) += ethertap_user.o
+liblinux-$(CONFIG_UML_NET_TUNTAP) += tuntap_user.o
diff --git a/arch/um/os-Linux/drivers/ethertap_user.c b/tools/um/uml/drivers/ethertap_user.c
similarity index 100%
rename from arch/um/os-Linux/drivers/ethertap_user.c
rename to tools/um/uml/drivers/ethertap_user.c
diff --git a/arch/um/os-Linux/drivers/tuntap_user.c b/tools/um/uml/drivers/tuntap_user.c
similarity index 100%
rename from arch/um/os-Linux/drivers/tuntap_user.c
rename to tools/um/uml/drivers/tuntap_user.c
diff --git a/arch/um/os-Linux/elf_aux.c b/tools/um/uml/elf_aux.c
similarity index 100%
rename from arch/um/os-Linux/elf_aux.c
rename to tools/um/uml/elf_aux.c
diff --git a/arch/um/os-Linux/execvp.c b/tools/um/uml/execvp.c
similarity index 98%
rename from arch/um/os-Linux/execvp.c
rename to tools/um/uml/execvp.c
index 84a0777c2a45..c105e0e7b778 100644
--- a/arch/um/os-Linux/execvp.c
+++ b/tools/um/uml/execvp.c
@@ -26,12 +26,8 @@
 #include <errno.h>
 #include <limits.h>
 
-#ifndef TEST
-#include <um_malloc.h>
-#else
 #include <stdio.h>
 #define um_kmalloc malloc
-#endif
 #include <os.h>
 
 /* Execute FILE, searching in the `PATH' environment variable if it contains
diff --git a/arch/um/os-Linux/file.c b/tools/um/uml/file.c
similarity index 100%
rename from arch/um/os-Linux/file.c
rename to tools/um/uml/file.c
diff --git a/arch/um/os-Linux/helper.c b/tools/um/uml/helper.c
similarity index 100%
rename from arch/um/os-Linux/helper.c
rename to tools/um/uml/helper.c
diff --git a/arch/um/os-Linux/irq.c b/tools/um/uml/irq.c
similarity index 100%
rename from arch/um/os-Linux/irq.c
rename to tools/um/uml/irq.c
diff --git a/arch/um/os-Linux/main.c b/tools/um/uml/main.c
similarity index 100%
rename from arch/um/os-Linux/main.c
rename to tools/um/uml/main.c
diff --git a/arch/um/os-Linux/mem.c b/tools/um/uml/mem.c
similarity index 100%
rename from arch/um/os-Linux/mem.c
rename to tools/um/uml/mem.c
diff --git a/arch/um/os-Linux/process.c b/tools/um/uml/process.c
similarity index 100%
rename from arch/um/os-Linux/process.c
rename to tools/um/uml/process.c
diff --git a/arch/um/os-Linux/registers.c b/tools/um/uml/registers.c
similarity index 100%
rename from arch/um/os-Linux/registers.c
rename to tools/um/uml/registers.c
diff --git a/arch/um/os-Linux/sigio.c b/tools/um/uml/sigio.c
similarity index 100%
rename from arch/um/os-Linux/sigio.c
rename to tools/um/uml/sigio.c
diff --git a/arch/um/os-Linux/signal.c b/tools/um/uml/signal.c
similarity index 100%
rename from arch/um/os-Linux/signal.c
rename to tools/um/uml/signal.c
diff --git a/arch/um/os-Linux/skas/Makefile b/tools/um/uml/skas/Build
similarity index 56%
rename from arch/um/os-Linux/skas/Makefile
rename to tools/um/uml/skas/Build
index c4566e788815..5fc9d62df863 100644
--- a/arch/um/os-Linux/skas/Makefile
+++ b/tools/um/uml/skas/Build
@@ -3,8 +3,4 @@
 # Copyright (C) 2002 - 2007 Jeff Dike (jdike@{linux.intel,addtoit}.com)
 #
 
-obj-y := mem.o process.o
-
-USER_OBJS := $(obj-y)
-
-include arch/um/scripts/Makefile.rules
+liblinux-y += mem.o process.o
diff --git a/arch/um/os-Linux/skas/mem.c b/tools/um/uml/skas/mem.c
similarity index 100%
rename from arch/um/os-Linux/skas/mem.c
rename to tools/um/uml/skas/mem.c
diff --git a/arch/um/os-Linux/skas/process.c b/tools/um/uml/skas/process.c
similarity index 99%
rename from arch/um/os-Linux/skas/process.c
rename to tools/um/uml/skas/process.c
index 4fb877b99dde..fca6a08e81bd 100644
--- a/arch/um/os-Linux/skas/process.c
+++ b/tools/um/uml/skas/process.c
@@ -21,7 +21,6 @@
 #include <registers.h>
 #include <skas.h>
 #include <sysdep/stub.h>
-#include <linux/threads.h>
 
 int is_skas_winch(int pid, int fd, void *data)
 {
@@ -248,7 +247,7 @@ static int userspace_tramp(void *stack)
 	return 0;
 }
 
-int userspace_pid[NR_CPUS];
+int userspace_pid[UM_NR_CPUS];
 
 /**
  * start_userspace() - prepare a new userspace process
diff --git a/arch/um/os-Linux/start_up.c b/tools/um/uml/start_up.c
similarity index 100%
rename from arch/um/os-Linux/start_up.c
rename to tools/um/uml/start_up.c
diff --git a/arch/um/os-Linux/time.c b/tools/um/uml/time.c
similarity index 100%
rename from arch/um/os-Linux/time.c
rename to tools/um/uml/time.c
diff --git a/arch/um/os-Linux/tty.c b/tools/um/uml/tty.c
similarity index 100%
rename from arch/um/os-Linux/tty.c
rename to tools/um/uml/tty.c
diff --git a/arch/um/os-Linux/umid.c b/tools/um/uml/umid.c
similarity index 100%
rename from arch/um/os-Linux/umid.c
rename to tools/um/uml/umid.c
diff --git a/arch/um/os-Linux/user_syms.c b/tools/um/uml/user_syms.c
similarity index 99%
rename from arch/um/os-Linux/user_syms.c
rename to tools/um/uml/user_syms.c
index 715594fe5719..4047fd7bfdb7 100644
--- a/arch/um/os-Linux/user_syms.c
+++ b/tools/um/uml/user_syms.c
@@ -13,6 +13,7 @@
 #undef strstr
 #undef memcpy
 #undef memset
+#define EXPORT_SYMBOL(x)
 
 extern size_t strlen(const char *);
 extern void *memmove(void *, const void *, size_t);
diff --git a/arch/um/os-Linux/util.c b/tools/um/uml/util.c
similarity index 100%
rename from arch/um/os-Linux/util.c
rename to tools/um/uml/util.c
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v6 03/21] um: move arch/um/os-Linux dir to tools/um/uml
@ 2020-09-24  7:12         ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  7:12 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Hajime Tazaki, retrage01

This patch moves underlying OS dependent part under arch/um to tools/um
directory so that arch/um code does not need to build host build
facilities (e.g., libc).

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/Makefile                              |  3 +-
 arch/um/drivers/Makefile                      |  2 +
 .../um/{os-Linux => }/drivers/ethertap_kern.c |  0
 arch/um/{os-Linux => }/drivers/tuntap_kern.c  |  0
 .../drivers => include/shared}/etap.h         |  0
 arch/um/include/shared/init.h                 |  5 ++
 .../drivers => include/shared}/tuntap.h       |  0
 arch/um/os-Linux/Makefile                     | 19 --------
 arch/um/os-Linux/drivers/Makefile             | 13 -----
 tools/um/Makefile                             |  6 ++-
 tools/um/uml/Build                            | 48 +++++++++++++++++++
 tools/um/uml/drivers/Build                    | 10 ++++
 .../um/uml}/drivers/ethertap_user.c           |  0
 .../um/uml}/drivers/tuntap_user.c             |  0
 {arch/um/os-Linux => tools/um/uml}/elf_aux.c  |  0
 {arch/um/os-Linux => tools/um/uml}/execvp.c   |  4 --
 {arch/um/os-Linux => tools/um/uml}/file.c     |  0
 {arch/um/os-Linux => tools/um/uml}/helper.c   |  0
 {arch/um/os-Linux => tools/um/uml}/irq.c      |  0
 {arch/um/os-Linux => tools/um/uml}/main.c     |  0
 {arch/um/os-Linux => tools/um/uml}/mem.c      |  0
 {arch/um/os-Linux => tools/um/uml}/process.c  |  0
 .../um/os-Linux => tools/um/uml}/registers.c  |  0
 {arch/um/os-Linux => tools/um/uml}/sigio.c    |  0
 {arch/um/os-Linux => tools/um/uml}/signal.c   |  0
 .../skas/Makefile => tools/um/uml/skas/Build  |  6 +--
 {arch/um/os-Linux => tools/um/uml}/skas/mem.c |  0
 .../os-Linux => tools/um/uml}/skas/process.c  |  3 +-
 {arch/um/os-Linux => tools/um/uml}/start_up.c |  0
 {arch/um/os-Linux => tools/um/uml}/time.c     |  0
 {arch/um/os-Linux => tools/um/uml}/tty.c      |  0
 {arch/um/os-Linux => tools/um/uml}/umid.c     |  0
 .../um/os-Linux => tools/um/uml}/user_syms.c  |  1 +
 {arch/um/os-Linux => tools/um/uml}/util.c     |  0
 34 files changed, 74 insertions(+), 46 deletions(-)
 rename arch/um/{os-Linux => }/drivers/ethertap_kern.c (100%)
 rename arch/um/{os-Linux => }/drivers/tuntap_kern.c (100%)
 rename arch/um/{os-Linux/drivers => include/shared}/etap.h (100%)
 rename arch/um/{os-Linux/drivers => include/shared}/tuntap.h (100%)
 delete mode 100644 arch/um/os-Linux/Makefile
 delete mode 100644 arch/um/os-Linux/drivers/Makefile
 create mode 100644 tools/um/uml/drivers/Build
 rename {arch/um/os-Linux => tools/um/uml}/drivers/ethertap_user.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/drivers/tuntap_user.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/elf_aux.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/execvp.c (98%)
 rename {arch/um/os-Linux => tools/um/uml}/file.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/helper.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/irq.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/main.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/mem.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/process.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/registers.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/sigio.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/signal.c (100%)
 rename arch/um/os-Linux/skas/Makefile => tools/um/uml/skas/Build (56%)
 rename {arch/um/os-Linux => tools/um/uml}/skas/mem.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/skas/process.c (99%)
 rename {arch/um/os-Linux => tools/um/uml}/start_up.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/time.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/tty.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/umid.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/user_syms.c (99%)
 rename {arch/um/os-Linux => tools/um/uml}/util.c (100%)

diff --git a/arch/um/Makefile b/arch/um/Makefile
index fdfaff4633e8..1f958a9280fe 100644
--- a/arch/um/Makefile
+++ b/arch/um/Makefile
@@ -24,8 +24,7 @@ OS := $(shell uname -s)
 SHELL := /bin/bash
 
 core-y			+= $(ARCH_DIR)/kernel/		\
-			   $(ARCH_DIR)/drivers/		\
-			   $(ARCH_DIR)/os-$(OS)/
+			   $(ARCH_DIR)/drivers/
 
 MODE_INCLUDE	+= -I$(srctree)/$(ARCH_DIR)/include/shared/skas
 
diff --git a/arch/um/drivers/Makefile b/arch/um/drivers/Makefile
index 2a249f619467..ae96c83e312d 100644
--- a/arch/um/drivers/Makefile
+++ b/arch/um/drivers/Makefile
@@ -48,6 +48,8 @@ obj-$(CONFIG_UML_NET_VECTOR) += vector.o
 obj-$(CONFIG_UML_NET_VDE) += vde.o
 obj-$(CONFIG_UML_NET_MCAST) += umcast.o
 obj-$(CONFIG_UML_NET_PCAP) += pcap.o
+obj-$(CONFIG_UML_NET_ETHERTAP) += ethertap_kern.o
+obj-$(CONFIG_UML_NET_TUNTAP) += tuntap_kern.o
 obj-$(CONFIG_UML_NET) += net.o 
 obj-$(CONFIG_MCONSOLE) += mconsole.o
 obj-$(CONFIG_MMAPPER) += mmapper_kern.o 
diff --git a/arch/um/os-Linux/drivers/ethertap_kern.c b/arch/um/drivers/ethertap_kern.c
similarity index 100%
rename from arch/um/os-Linux/drivers/ethertap_kern.c
rename to arch/um/drivers/ethertap_kern.c
diff --git a/arch/um/os-Linux/drivers/tuntap_kern.c b/arch/um/drivers/tuntap_kern.c
similarity index 100%
rename from arch/um/os-Linux/drivers/tuntap_kern.c
rename to arch/um/drivers/tuntap_kern.c
diff --git a/arch/um/os-Linux/drivers/etap.h b/arch/um/include/shared/etap.h
similarity index 100%
rename from arch/um/os-Linux/drivers/etap.h
rename to arch/um/include/shared/etap.h
diff --git a/arch/um/include/shared/init.h b/arch/um/include/shared/init.h
index d09308330ca5..305789667584 100644
--- a/arch/um/include/shared/init.h
+++ b/arch/um/include/shared/init.h
@@ -41,7 +41,12 @@
 typedef int (*initcall_t)(void);
 typedef void (*exitcall_t)(void);
 
+#ifndef __UM_HOST__
 #include <linux/compiler_types.h>
+#else
+#define __used                          __attribute__((__used__))
+#define __section(S)                    __attribute__((__section__(#S)))
+#endif
 
 /* These are for everybody (although not all archs will actually
    discard it in modules) */
diff --git a/arch/um/os-Linux/drivers/tuntap.h b/arch/um/include/shared/tuntap.h
similarity index 100%
rename from arch/um/os-Linux/drivers/tuntap.h
rename to arch/um/include/shared/tuntap.h
diff --git a/arch/um/os-Linux/Makefile b/arch/um/os-Linux/Makefile
deleted file mode 100644
index 839915b8c31c..000000000000
--- a/arch/um/os-Linux/Makefile
+++ /dev/null
@@ -1,19 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0
-# 
-# Copyright (C) 2000 - 2007 Jeff Dike (jdike@{addtoit,linux.intel}.com)
-#
-
-# Don't instrument UML-specific code
-KCOV_INSTRUMENT                := n
-
-obj-y = execvp.o file.o helper.o irq.o main.o mem.o process.o \
-	registers.o sigio.o signal.o start_up.o time.o tty.o \
-	umid.o user_syms.o util.o drivers/ skas/
-
-obj-$(CONFIG_ARCH_REUSE_HOST_VSYSCALL_AREA) += elf_aux.o
-
-USER_OBJS := $(user-objs-y) elf_aux.o execvp.o file.o helper.o irq.o \
-	main.o mem.o process.o registers.o sigio.o signal.o start_up.o time.o \
-	tty.o umid.o util.o
-
-include arch/um/scripts/Makefile.rules
diff --git a/arch/um/os-Linux/drivers/Makefile b/arch/um/os-Linux/drivers/Makefile
deleted file mode 100644
index d79e75f1b69a..000000000000
--- a/arch/um/os-Linux/drivers/Makefile
+++ /dev/null
@@ -1,13 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0
-# 
-# Copyright (C) 2000, 2002 Jeff Dike (jdike@karaya.com)
-#
-
-ethertap-objs := ethertap_kern.o ethertap_user.o
-tuntap-objs := tuntap_kern.o tuntap_user.o
-
-obj-y = 
-obj-$(CONFIG_UML_NET_ETHERTAP) += ethertap.o
-obj-$(CONFIG_UML_NET_TUNTAP) += tuntap.o
-
-include arch/um/scripts/Makefile.rules
diff --git a/tools/um/Makefile b/tools/um/Makefile
index a63466ef7ef5..07b9b93ed817 100644
--- a/tools/um/Makefile
+++ b/tools/um/Makefile
@@ -50,7 +50,7 @@ $(OUTPUT)lib/linux.o:
 	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) $(KCONFIG)
 	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) install INSTALL_PATH=$(OUTPUT)
 
-$(OUTPUT)liblinux.a: $(OUTPUT)lib/linux.o $(OUTPUT)uml/liblinux-in.o
+$(OUTPUT)liblinux.a: $(OUTPUT)lib/linux.o $(OUTPUT)uml/liblinux-in.o $(OUTPUT)lib/liblinux-in.o
 	$(QUIET_AR)$(AR) -rc $@ $^
 
 # rule to link programs
@@ -60,6 +60,10 @@ $(OUTPUT)%: $(OUTPUT)%-in.o $(OUTPUT)liblinux.a
 $(OUTPUT)%-in.o: FORCE
 	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
 
+# rule to build objects
+$(OUTPUT)%-in.o: $(OUTPUT)lib/linux.o
+	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
+
 clean:
 	$(call QUIET_CLEAN, objects)find $(OUTPUT) -name '*.o' -delete -o -name '\.*.cmd'\
 	 -delete -o -name '\.*.d' -delete
diff --git a/tools/um/uml/Build b/tools/um/uml/Build
index e69de29bb2d1..8e0607ee8b0a 100644
--- a/tools/um/uml/Build
+++ b/tools/um/uml/Build
@@ -0,0 +1,48 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Copyright (C) 2000 - 2007 Jeff Dike (jdike@{addtoit,linux.intel}.com)
+#
+
+# Don't instrument UML-specific code
+KCOV_INSTRUMENT                := n
+
+include $(objtree)/include/config/auto.conf
+
+liblinux-y += execvp.o file.o helper.o irq.o main.o mem.o process.o \
+	registers.o sigio.o signal.o start_up.o time.o tty.o \
+	umid.o user_syms.o util.o drivers/ skas/
+
+liblinux-$(CONFIG_ARCH_REUSE_HOST_VSYSCALL_AREA) += elf_aux.o
+
+export O = $(srctree)
+objtree := $(O)
+# reset CFLAGS
+CFLAGS := -g -O2
+
+# from arch/um/Makefile
+ARCH_DIR := arch/um
+HEADER_ARCH 	:= x86
+HOST_DIR := arch/$(HEADER_ARCH)
+ifdef CONFIG_64BIT
+  KBUILD_CFLAGS += -mcmodel=large
+endif
+KBUILD_CFLAGS += $(CFLAGS) $(CFLAGS-y) -D__arch_um__ \
+	$(ARCH_INCLUDE) $(MODE_INCLUDE) -Dvmap=kernel_vmap	\
+	-Dlongjmp=kernel_longjmp -Dsetjmp=kernel_setjmp \
+	-Din6addr_loopback=kernel_in6addr_loopback \
+	-Din6addr_any=kernel_in6addr_any -Dstrrchr=kernel_strrchr
+SHARED_HEADERS	:= $(ARCH_DIR)/include/shared
+MODE_INCLUDE	+= -I$(srctree)/$(ARCH_DIR)/include/shared/skas
+ARCH_INCLUDE	:= -I$(srctree)/$(SHARED_HEADERS)
+ARCH_INCLUDE	+= -I$(srctree)/$(HOST_DIR)/um/shared
+KBUILD_CPPFLAGS += -I$(srctree)/$(HOST_DIR)/um
+USER_CFLAGS := $(patsubst $(KERNEL_DEFINES),,$(patsubst -I%,,$(KBUILD_CFLAGS))) \
+		$(ARCH_INCLUDE) $(MODE_INCLUDE) $(filter -I%,$(CFLAGS)) \
+		-D_FILE_OFFSET_BITS=64 -idirafter $(srctree)/include/uapi \
+		-idirafter $(objtree)/include -D__KERNEL__ -D__UM_HOST__
+
+# from Makefile-os-Linux
+USER_CFLAGS += -D_GNU_SOURCE -D_LARGEFILE64_SOURCE
+
+# from Makefile.rules
+CFLAGS += $(USER_CFLAGS) -include $(srctree)/tools/include/linux/kern_levels.h -include user.h $(CFLAGS_$(basetarget).o)
diff --git a/tools/um/uml/drivers/Build b/tools/um/uml/drivers/Build
new file mode 100644
index 000000000000..c7a8eaa97d72
--- /dev/null
+++ b/tools/um/uml/drivers/Build
@@ -0,0 +1,10 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Copyright (C) 2000, 2002 Jeff Dike (jdike@karaya.com)
+#
+
+include $(objtree)/include/config/auto.conf
+
+
+liblinux-$(CONFIG_UML_NET_ETHERTAP) += ethertap_user.o
+liblinux-$(CONFIG_UML_NET_TUNTAP) += tuntap_user.o
diff --git a/arch/um/os-Linux/drivers/ethertap_user.c b/tools/um/uml/drivers/ethertap_user.c
similarity index 100%
rename from arch/um/os-Linux/drivers/ethertap_user.c
rename to tools/um/uml/drivers/ethertap_user.c
diff --git a/arch/um/os-Linux/drivers/tuntap_user.c b/tools/um/uml/drivers/tuntap_user.c
similarity index 100%
rename from arch/um/os-Linux/drivers/tuntap_user.c
rename to tools/um/uml/drivers/tuntap_user.c
diff --git a/arch/um/os-Linux/elf_aux.c b/tools/um/uml/elf_aux.c
similarity index 100%
rename from arch/um/os-Linux/elf_aux.c
rename to tools/um/uml/elf_aux.c
diff --git a/arch/um/os-Linux/execvp.c b/tools/um/uml/execvp.c
similarity index 98%
rename from arch/um/os-Linux/execvp.c
rename to tools/um/uml/execvp.c
index 84a0777c2a45..c105e0e7b778 100644
--- a/arch/um/os-Linux/execvp.c
+++ b/tools/um/uml/execvp.c
@@ -26,12 +26,8 @@
 #include <errno.h>
 #include <limits.h>
 
-#ifndef TEST
-#include <um_malloc.h>
-#else
 #include <stdio.h>
 #define um_kmalloc malloc
-#endif
 #include <os.h>
 
 /* Execute FILE, searching in the `PATH' environment variable if it contains
diff --git a/arch/um/os-Linux/file.c b/tools/um/uml/file.c
similarity index 100%
rename from arch/um/os-Linux/file.c
rename to tools/um/uml/file.c
diff --git a/arch/um/os-Linux/helper.c b/tools/um/uml/helper.c
similarity index 100%
rename from arch/um/os-Linux/helper.c
rename to tools/um/uml/helper.c
diff --git a/arch/um/os-Linux/irq.c b/tools/um/uml/irq.c
similarity index 100%
rename from arch/um/os-Linux/irq.c
rename to tools/um/uml/irq.c
diff --git a/arch/um/os-Linux/main.c b/tools/um/uml/main.c
similarity index 100%
rename from arch/um/os-Linux/main.c
rename to tools/um/uml/main.c
diff --git a/arch/um/os-Linux/mem.c b/tools/um/uml/mem.c
similarity index 100%
rename from arch/um/os-Linux/mem.c
rename to tools/um/uml/mem.c
diff --git a/arch/um/os-Linux/process.c b/tools/um/uml/process.c
similarity index 100%
rename from arch/um/os-Linux/process.c
rename to tools/um/uml/process.c
diff --git a/arch/um/os-Linux/registers.c b/tools/um/uml/registers.c
similarity index 100%
rename from arch/um/os-Linux/registers.c
rename to tools/um/uml/registers.c
diff --git a/arch/um/os-Linux/sigio.c b/tools/um/uml/sigio.c
similarity index 100%
rename from arch/um/os-Linux/sigio.c
rename to tools/um/uml/sigio.c
diff --git a/arch/um/os-Linux/signal.c b/tools/um/uml/signal.c
similarity index 100%
rename from arch/um/os-Linux/signal.c
rename to tools/um/uml/signal.c
diff --git a/arch/um/os-Linux/skas/Makefile b/tools/um/uml/skas/Build
similarity index 56%
rename from arch/um/os-Linux/skas/Makefile
rename to tools/um/uml/skas/Build
index c4566e788815..5fc9d62df863 100644
--- a/arch/um/os-Linux/skas/Makefile
+++ b/tools/um/uml/skas/Build
@@ -3,8 +3,4 @@
 # Copyright (C) 2002 - 2007 Jeff Dike (jdike@{linux.intel,addtoit}.com)
 #
 
-obj-y := mem.o process.o
-
-USER_OBJS := $(obj-y)
-
-include arch/um/scripts/Makefile.rules
+liblinux-y += mem.o process.o
diff --git a/arch/um/os-Linux/skas/mem.c b/tools/um/uml/skas/mem.c
similarity index 100%
rename from arch/um/os-Linux/skas/mem.c
rename to tools/um/uml/skas/mem.c
diff --git a/arch/um/os-Linux/skas/process.c b/tools/um/uml/skas/process.c
similarity index 99%
rename from arch/um/os-Linux/skas/process.c
rename to tools/um/uml/skas/process.c
index 4fb877b99dde..fca6a08e81bd 100644
--- a/arch/um/os-Linux/skas/process.c
+++ b/tools/um/uml/skas/process.c
@@ -21,7 +21,6 @@
 #include <registers.h>
 #include <skas.h>
 #include <sysdep/stub.h>
-#include <linux/threads.h>
 
 int is_skas_winch(int pid, int fd, void *data)
 {
@@ -248,7 +247,7 @@ static int userspace_tramp(void *stack)
 	return 0;
 }
 
-int userspace_pid[NR_CPUS];
+int userspace_pid[UM_NR_CPUS];
 
 /**
  * start_userspace() - prepare a new userspace process
diff --git a/arch/um/os-Linux/start_up.c b/tools/um/uml/start_up.c
similarity index 100%
rename from arch/um/os-Linux/start_up.c
rename to tools/um/uml/start_up.c
diff --git a/arch/um/os-Linux/time.c b/tools/um/uml/time.c
similarity index 100%
rename from arch/um/os-Linux/time.c
rename to tools/um/uml/time.c
diff --git a/arch/um/os-Linux/tty.c b/tools/um/uml/tty.c
similarity index 100%
rename from arch/um/os-Linux/tty.c
rename to tools/um/uml/tty.c
diff --git a/arch/um/os-Linux/umid.c b/tools/um/uml/umid.c
similarity index 100%
rename from arch/um/os-Linux/umid.c
rename to tools/um/uml/umid.c
diff --git a/arch/um/os-Linux/user_syms.c b/tools/um/uml/user_syms.c
similarity index 99%
rename from arch/um/os-Linux/user_syms.c
rename to tools/um/uml/user_syms.c
index 715594fe5719..4047fd7bfdb7 100644
--- a/arch/um/os-Linux/user_syms.c
+++ b/tools/um/uml/user_syms.c
@@ -13,6 +13,7 @@
 #undef strstr
 #undef memcpy
 #undef memset
+#define EXPORT_SYMBOL(x)
 
 extern size_t strlen(const char *);
 extern void *memmove(void *, const void *, size_t);
diff --git a/arch/um/os-Linux/util.c b/tools/um/uml/util.c
similarity index 100%
rename from arch/um/os-Linux/util.c
rename to tools/um/uml/util.c
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v6 04/21] um: host: implement os_initcalls and os_exitcalls
  2020-09-24  7:12       ` Hajime Tazaki
@ 2020-09-24  7:12         ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  7:12 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch,
	Octavian Purdila

From: Octavian Purdila <tavi@cs.pub.ro>

This patch implements the init and exit calls for host code. It uses
the automatic __start_<section> and __stop_<section> variables that
are defined by gcc / ld when using custom sections.

Note that this patch should be merged with "um: move arch/um/os-Linux
dir to tools/um" but for now it is separate to make the review easier.

Signed-off-by: Octavian Purdila <tavi@cs.pub.ro>
---
 tools/um/uml/util.c | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)

diff --git a/tools/um/uml/util.c b/tools/um/uml/util.c
index ecf2f390fad2..4011b36fee7e 100644
--- a/tools/um/uml/util.c
+++ b/tools/um/uml/util.c
@@ -186,3 +186,29 @@ void os_warn(const char *fmt, ...)
 	vfprintf(stderr, fmt, list);
 	va_end(list);
 }
+
+extern void (*__start_os_exitcalls)(void);
+extern void (*__stop_os_exitcalls)(void);
+
+void os_exitcalls(void)
+{
+	exitcall_t *call;
+
+	call = &__stop_os_exitcalls;
+	while (--call >= &__start_os_exitcalls)
+		(*call)();
+}
+
+extern int (*__start_os_initcalls)(void);
+extern int (*__stop_os_initcalls)(void);
+
+int os_initcalls(void)
+{
+	initcall_t *call;
+
+	call = &__stop_os_initcalls;
+	while (--call >= &__start_os_initcalls)
+		(*call)();
+
+	return 0;
+}
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v6 04/21] um: host: implement os_initcalls and os_exitcalls
@ 2020-09-24  7:12         ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  7:12 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Octavian Purdila,
	retrage01

From: Octavian Purdila <tavi@cs.pub.ro>

This patch implements the init and exit calls for host code. It uses
the automatic __start_<section> and __stop_<section> variables that
are defined by gcc / ld when using custom sections.

Note that this patch should be merged with "um: move arch/um/os-Linux
dir to tools/um" but for now it is separate to make the review easier.

Signed-off-by: Octavian Purdila <tavi@cs.pub.ro>
---
 tools/um/uml/util.c | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)

diff --git a/tools/um/uml/util.c b/tools/um/uml/util.c
index ecf2f390fad2..4011b36fee7e 100644
--- a/tools/um/uml/util.c
+++ b/tools/um/uml/util.c
@@ -186,3 +186,29 @@ void os_warn(const char *fmt, ...)
 	vfprintf(stderr, fmt, list);
 	va_end(list);
 }
+
+extern void (*__start_os_exitcalls)(void);
+extern void (*__stop_os_exitcalls)(void);
+
+void os_exitcalls(void)
+{
+	exitcall_t *call;
+
+	call = &__stop_os_exitcalls;
+	while (--call >= &__start_os_exitcalls)
+		(*call)();
+}
+
+extern int (*__start_os_initcalls)(void);
+extern int (*__stop_os_initcalls)(void);
+
+int os_initcalls(void)
+{
+	initcall_t *call;
+
+	call = &__stop_os_initcalls;
+	while (--call >= &__start_os_initcalls)
+		(*call)();
+
+	return 0;
+}
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v6 05/21] um: move arch/x86/um/os-Linux to tools/um/uml/
  2020-09-24  7:12       ` Hajime Tazaki
@ 2020-09-24  7:12         ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  7:12 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch, Hajime Tazaki

Similar to arch/um/os-Linux code, arch/x86 also contains OS dependent
part as well.  This patch moves underlying OS dependent part to tools/um
directory so that arch/x86 code does not need to build host build
facilities (e.g., libc).

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/x86/um/Makefile                                |  2 +-
 arch/x86/um/os-Linux/Makefile                       | 13 -------------
 tools/um/Makefile                                   |  2 +-
 tools/um/uml/Build                                  |  2 ++
 tools/um/uml/x86/Build                              | 11 +++++++++++
 .../x86/um/os-Linux => tools/um/uml/x86}/mcontext.c |  0
 {arch/x86/um/os-Linux => tools/um/uml/x86}/prctl.c  |  0
 .../um/os-Linux => tools/um/uml/x86}/registers.c    |  0
 .../um/os-Linux => tools/um/uml/x86}/task_size.c    |  0
 {arch/x86/um/os-Linux => tools/um/uml/x86}/tls.c    |  0
 10 files changed, 15 insertions(+), 15 deletions(-)
 delete mode 100644 arch/x86/um/os-Linux/Makefile
 create mode 100644 tools/um/uml/x86/Build
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/mcontext.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/prctl.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/registers.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/task_size.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/tls.c (100%)

diff --git a/arch/x86/um/Makefile b/arch/x86/um/Makefile
index 77f70b969d14..25e9846b36a2 100644
--- a/arch/x86/um/Makefile
+++ b/arch/x86/um/Makefile
@@ -13,7 +13,7 @@ obj-y = bugs_$(BITS).o delay.o fault.o ldt.o \
 	ptrace_$(BITS).o ptrace_user.o setjmp_$(BITS).o signal.o \
 	stub_$(BITS).o stub_segv.o \
 	sys_call_table_$(BITS).o sysrq_$(BITS).o tls_$(BITS).o \
-	mem_$(BITS).o subarch.o os-$(OS)/
+	mem_$(BITS).o subarch.o
 
 ifeq ($(CONFIG_X86_32),y)
 
diff --git a/arch/x86/um/os-Linux/Makefile b/arch/x86/um/os-Linux/Makefile
deleted file mode 100644
index 253bfb8cb702..000000000000
--- a/arch/x86/um/os-Linux/Makefile
+++ /dev/null
@@ -1,13 +0,0 @@
-#
-# Copyright (C) 2000 - 2007 Jeff Dike (jdike@{addtoit,linux.intel}.com)
-# Licensed under the GPL
-#
-
-obj-y = registers.o task_size.o mcontext.o
-
-obj-$(CONFIG_X86_32) += tls.o
-obj-$(CONFIG_64BIT) += prctl.o
-
-USER_OBJS := $(obj-y)
-
-include arch/um/scripts/Makefile.rules
diff --git a/tools/um/Makefile b/tools/um/Makefile
index 07b9b93ed817..552ed5f1edae 100644
--- a/tools/um/Makefile
+++ b/tools/um/Makefile
@@ -61,7 +61,7 @@ $(OUTPUT)%-in.o: FORCE
 	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
 
 # rule to build objects
-$(OUTPUT)%-in.o: $(OUTPUT)lib/linux.o
+$(OUTPUT)%-in.o: $(OUTPUT)lib/linux.o FORCE
 	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
 
 clean:
diff --git a/tools/um/uml/Build b/tools/um/uml/Build
index 8e0607ee8b0a..435a9670d3ff 100644
--- a/tools/um/uml/Build
+++ b/tools/um/uml/Build
@@ -12,6 +12,8 @@ liblinux-y += execvp.o file.o helper.o irq.o main.o mem.o process.o \
 	registers.o sigio.o signal.o start_up.o time.o tty.o \
 	umid.o user_syms.o util.o drivers/ skas/
 
+liblinux-y += x86/
+
 liblinux-$(CONFIG_ARCH_REUSE_HOST_VSYSCALL_AREA) += elf_aux.o
 
 export O = $(srctree)
diff --git a/tools/um/uml/x86/Build b/tools/um/uml/x86/Build
new file mode 100644
index 000000000000..16bbc7eb19d1
--- /dev/null
+++ b/tools/um/uml/x86/Build
@@ -0,0 +1,11 @@
+#
+# Copyright (C) 2000 - 2007 Jeff Dike (jdike@{addtoit,linux.intel}.com)
+# Licensed under the GPL
+#
+
+include $(objtree)/include/config/auto.conf
+
+liblinux-y = registers.o task_size.o mcontext.o
+
+liblinux-$(CONFIG_X86_32) += tls.o
+liblinux-$(CONFIG_64BIT) += prctl.o
diff --git a/arch/x86/um/os-Linux/mcontext.c b/tools/um/uml/x86/mcontext.c
similarity index 100%
rename from arch/x86/um/os-Linux/mcontext.c
rename to tools/um/uml/x86/mcontext.c
diff --git a/arch/x86/um/os-Linux/prctl.c b/tools/um/uml/x86/prctl.c
similarity index 100%
rename from arch/x86/um/os-Linux/prctl.c
rename to tools/um/uml/x86/prctl.c
diff --git a/arch/x86/um/os-Linux/registers.c b/tools/um/uml/x86/registers.c
similarity index 100%
rename from arch/x86/um/os-Linux/registers.c
rename to tools/um/uml/x86/registers.c
diff --git a/arch/x86/um/os-Linux/task_size.c b/tools/um/uml/x86/task_size.c
similarity index 100%
rename from arch/x86/um/os-Linux/task_size.c
rename to tools/um/uml/x86/task_size.c
diff --git a/arch/x86/um/os-Linux/tls.c b/tools/um/uml/x86/tls.c
similarity index 100%
rename from arch/x86/um/os-Linux/tls.c
rename to tools/um/uml/x86/tls.c
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v6 05/21] um: move arch/x86/um/os-Linux to tools/um/uml/
@ 2020-09-24  7:12         ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  7:12 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Hajime Tazaki, retrage01

Similar to arch/um/os-Linux code, arch/x86 also contains OS dependent
part as well.  This patch moves underlying OS dependent part to tools/um
directory so that arch/x86 code does not need to build host build
facilities (e.g., libc).

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/x86/um/Makefile                                |  2 +-
 arch/x86/um/os-Linux/Makefile                       | 13 -------------
 tools/um/Makefile                                   |  2 +-
 tools/um/uml/Build                                  |  2 ++
 tools/um/uml/x86/Build                              | 11 +++++++++++
 .../x86/um/os-Linux => tools/um/uml/x86}/mcontext.c |  0
 {arch/x86/um/os-Linux => tools/um/uml/x86}/prctl.c  |  0
 .../um/os-Linux => tools/um/uml/x86}/registers.c    |  0
 .../um/os-Linux => tools/um/uml/x86}/task_size.c    |  0
 {arch/x86/um/os-Linux => tools/um/uml/x86}/tls.c    |  0
 10 files changed, 15 insertions(+), 15 deletions(-)
 delete mode 100644 arch/x86/um/os-Linux/Makefile
 create mode 100644 tools/um/uml/x86/Build
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/mcontext.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/prctl.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/registers.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/task_size.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/tls.c (100%)

diff --git a/arch/x86/um/Makefile b/arch/x86/um/Makefile
index 77f70b969d14..25e9846b36a2 100644
--- a/arch/x86/um/Makefile
+++ b/arch/x86/um/Makefile
@@ -13,7 +13,7 @@ obj-y = bugs_$(BITS).o delay.o fault.o ldt.o \
 	ptrace_$(BITS).o ptrace_user.o setjmp_$(BITS).o signal.o \
 	stub_$(BITS).o stub_segv.o \
 	sys_call_table_$(BITS).o sysrq_$(BITS).o tls_$(BITS).o \
-	mem_$(BITS).o subarch.o os-$(OS)/
+	mem_$(BITS).o subarch.o
 
 ifeq ($(CONFIG_X86_32),y)
 
diff --git a/arch/x86/um/os-Linux/Makefile b/arch/x86/um/os-Linux/Makefile
deleted file mode 100644
index 253bfb8cb702..000000000000
--- a/arch/x86/um/os-Linux/Makefile
+++ /dev/null
@@ -1,13 +0,0 @@
-#
-# Copyright (C) 2000 - 2007 Jeff Dike (jdike@{addtoit,linux.intel}.com)
-# Licensed under the GPL
-#
-
-obj-y = registers.o task_size.o mcontext.o
-
-obj-$(CONFIG_X86_32) += tls.o
-obj-$(CONFIG_64BIT) += prctl.o
-
-USER_OBJS := $(obj-y)
-
-include arch/um/scripts/Makefile.rules
diff --git a/tools/um/Makefile b/tools/um/Makefile
index 07b9b93ed817..552ed5f1edae 100644
--- a/tools/um/Makefile
+++ b/tools/um/Makefile
@@ -61,7 +61,7 @@ $(OUTPUT)%-in.o: FORCE
 	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
 
 # rule to build objects
-$(OUTPUT)%-in.o: $(OUTPUT)lib/linux.o
+$(OUTPUT)%-in.o: $(OUTPUT)lib/linux.o FORCE
 	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
 
 clean:
diff --git a/tools/um/uml/Build b/tools/um/uml/Build
index 8e0607ee8b0a..435a9670d3ff 100644
--- a/tools/um/uml/Build
+++ b/tools/um/uml/Build
@@ -12,6 +12,8 @@ liblinux-y += execvp.o file.o helper.o irq.o main.o mem.o process.o \
 	registers.o sigio.o signal.o start_up.o time.o tty.o \
 	umid.o user_syms.o util.o drivers/ skas/
 
+liblinux-y += x86/
+
 liblinux-$(CONFIG_ARCH_REUSE_HOST_VSYSCALL_AREA) += elf_aux.o
 
 export O = $(srctree)
diff --git a/tools/um/uml/x86/Build b/tools/um/uml/x86/Build
new file mode 100644
index 000000000000..16bbc7eb19d1
--- /dev/null
+++ b/tools/um/uml/x86/Build
@@ -0,0 +1,11 @@
+#
+# Copyright (C) 2000 - 2007 Jeff Dike (jdike@{addtoit,linux.intel}.com)
+# Licensed under the GPL
+#
+
+include $(objtree)/include/config/auto.conf
+
+liblinux-y = registers.o task_size.o mcontext.o
+
+liblinux-$(CONFIG_X86_32) += tls.o
+liblinux-$(CONFIG_64BIT) += prctl.o
diff --git a/arch/x86/um/os-Linux/mcontext.c b/tools/um/uml/x86/mcontext.c
similarity index 100%
rename from arch/x86/um/os-Linux/mcontext.c
rename to tools/um/uml/x86/mcontext.c
diff --git a/arch/x86/um/os-Linux/prctl.c b/tools/um/uml/x86/prctl.c
similarity index 100%
rename from arch/x86/um/os-Linux/prctl.c
rename to tools/um/uml/x86/prctl.c
diff --git a/arch/x86/um/os-Linux/registers.c b/tools/um/uml/x86/registers.c
similarity index 100%
rename from arch/x86/um/os-Linux/registers.c
rename to tools/um/uml/x86/registers.c
diff --git a/arch/x86/um/os-Linux/task_size.c b/tools/um/uml/x86/task_size.c
similarity index 100%
rename from arch/x86/um/os-Linux/task_size.c
rename to tools/um/uml/x86/task_size.c
diff --git a/arch/x86/um/os-Linux/tls.c b/tools/um/uml/x86/tls.c
similarity index 100%
rename from arch/x86/um/os-Linux/tls.c
rename to tools/um/uml/x86/tls.c
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v6 06/21] scritps: um: suppress warnings if SRCARCH=um
  2020-09-24  7:12       ` Hajime Tazaki
@ 2020-09-24  7:12         ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  7:12 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch, Hajime Tazaki

This commit fixes numbers of warning messages about leaked CONFIG
options.  nommu mode of UML requires copies of kernel headers to offer
syscall-like API for the library users.  Thus, the warnings are to be
avoided to function this exposure of API.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 scripts/headers_install.sh | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/scripts/headers_install.sh b/scripts/headers_install.sh
index dd554bd436cc..8890a0147012 100755
--- a/scripts/headers_install.sh
+++ b/scripts/headers_install.sh
@@ -93,6 +93,10 @@ include/uapi/linux/pktcdvd.h:CONFIG_CDROM_PKTCDVD_WCACHE
 
 for c in $configs
 do
+	if [ "$SRCARCH" = "um" ] ; then
+		break
+	fi
+
 	leak_error=1
 
 	for ignore in $config_leak_ignores
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v6 06/21] scritps: um: suppress warnings if SRCARCH=um
@ 2020-09-24  7:12         ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  7:12 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Hajime Tazaki, retrage01

This commit fixes numbers of warning messages about leaked CONFIG
options.  nommu mode of UML requires copies of kernel headers to offer
syscall-like API for the library users.  Thus, the warnings are to be
avoided to function this exposure of API.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 scripts/headers_install.sh | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/scripts/headers_install.sh b/scripts/headers_install.sh
index dd554bd436cc..8890a0147012 100755
--- a/scripts/headers_install.sh
+++ b/scripts/headers_install.sh
@@ -93,6 +93,10 @@ include/uapi/linux/pktcdvd.h:CONFIG_CDROM_PKTCDVD_WCACHE
 
 for c in $configs
 do
+	if [ "$SRCARCH" = "um" ] ; then
+		break
+	fi
+
 	leak_error=1
 
 	for ignore in $config_leak_ignores
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v6 07/21] um: extend arch_switch_to for alternate SUBARCH
  2020-09-24  7:12       ` Hajime Tazaki
@ 2020-09-24  7:12         ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  7:12 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch, Hajime Tazaki

This commit introduces additional argument of previous task when
context switch happens.  New SUBARCH can use the new information to
switch tasks in a subarch-specific manner.

The patch is particularly required by nommu mode implemented as a
SUBARCH of UML.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/kernel/process.c  | 6 +++---
 arch/x86/um/ptrace_32.c   | 2 +-
 arch/x86/um/syscalls_64.c | 2 +-
 3 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/arch/um/kernel/process.c b/arch/um/kernel/process.c
index 26b5e243d3fc..87a8cfa228ca 100644
--- a/arch/um/kernel/process.c
+++ b/arch/um/kernel/process.c
@@ -80,7 +80,7 @@ static inline void set_current(struct task_struct *task)
 		{ external_pid(), task });
 }
 
-extern void arch_switch_to(struct task_struct *to);
+extern void arch_switch_to(struct task_struct *from, struct task_struct *to);
 
 void *__switch_to(struct task_struct *from, struct task_struct *to)
 {
@@ -88,7 +88,7 @@ void *__switch_to(struct task_struct *from, struct task_struct *to)
 	set_current(to);
 
 	switch_threads(&from->thread.switch_buf, &to->thread.switch_buf);
-	arch_switch_to(current);
+	arch_switch_to(from, to);
 
 	return current->thread.prev_sched;
 }
@@ -145,7 +145,7 @@ void fork_handler(void)
 	 * arch_switch_to isn't needed. We could want to apply this to
 	 * improve performance. -bb
 	 */
-	arch_switch_to(current);
+	arch_switch_to(NULL, current);
 
 	current->thread.prev_sched = NULL;
 
diff --git a/arch/x86/um/ptrace_32.c b/arch/x86/um/ptrace_32.c
index 2497bac56066..0f184710d4ca 100644
--- a/arch/x86/um/ptrace_32.c
+++ b/arch/x86/um/ptrace_32.c
@@ -11,7 +11,7 @@
 
 extern int arch_switch_tls(struct task_struct *to);
 
-void arch_switch_to(struct task_struct *to)
+void arch_switch_to(struct task_struct *from, struct task_struct *to)
 {
 	int err = arch_switch_tls(to);
 	if (!err)
diff --git a/arch/x86/um/syscalls_64.c b/arch/x86/um/syscalls_64.c
index 58f51667e2e4..2ef9474d2bd2 100644
--- a/arch/x86/um/syscalls_64.c
+++ b/arch/x86/um/syscalls_64.c
@@ -80,7 +80,7 @@ SYSCALL_DEFINE2(arch_prctl, int, option, unsigned long, arg2)
 	return arch_prctl(current, option, (unsigned long __user *) arg2);
 }
 
-void arch_switch_to(struct task_struct *to)
+void arch_switch_to(struct task_struct *from, struct task_struct *to)
 {
 	if ((to->thread.arch.fs == 0) || (to->mm == NULL))
 		return;
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v6 07/21] um: extend arch_switch_to for alternate SUBARCH
@ 2020-09-24  7:12         ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  7:12 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Hajime Tazaki, retrage01

This commit introduces additional argument of previous task when
context switch happens.  New SUBARCH can use the new information to
switch tasks in a subarch-specific manner.

The patch is particularly required by nommu mode implemented as a
SUBARCH of UML.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/kernel/process.c  | 6 +++---
 arch/x86/um/ptrace_32.c   | 2 +-
 arch/x86/um/syscalls_64.c | 2 +-
 3 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/arch/um/kernel/process.c b/arch/um/kernel/process.c
index 26b5e243d3fc..87a8cfa228ca 100644
--- a/arch/um/kernel/process.c
+++ b/arch/um/kernel/process.c
@@ -80,7 +80,7 @@ static inline void set_current(struct task_struct *task)
 		{ external_pid(), task });
 }
 
-extern void arch_switch_to(struct task_struct *to);
+extern void arch_switch_to(struct task_struct *from, struct task_struct *to);
 
 void *__switch_to(struct task_struct *from, struct task_struct *to)
 {
@@ -88,7 +88,7 @@ void *__switch_to(struct task_struct *from, struct task_struct *to)
 	set_current(to);
 
 	switch_threads(&from->thread.switch_buf, &to->thread.switch_buf);
-	arch_switch_to(current);
+	arch_switch_to(from, to);
 
 	return current->thread.prev_sched;
 }
@@ -145,7 +145,7 @@ void fork_handler(void)
 	 * arch_switch_to isn't needed. We could want to apply this to
 	 * improve performance. -bb
 	 */
-	arch_switch_to(current);
+	arch_switch_to(NULL, current);
 
 	current->thread.prev_sched = NULL;
 
diff --git a/arch/x86/um/ptrace_32.c b/arch/x86/um/ptrace_32.c
index 2497bac56066..0f184710d4ca 100644
--- a/arch/x86/um/ptrace_32.c
+++ b/arch/x86/um/ptrace_32.c
@@ -11,7 +11,7 @@
 
 extern int arch_switch_tls(struct task_struct *to);
 
-void arch_switch_to(struct task_struct *to)
+void arch_switch_to(struct task_struct *from, struct task_struct *to)
 {
 	int err = arch_switch_tls(to);
 	if (!err)
diff --git a/arch/x86/um/syscalls_64.c b/arch/x86/um/syscalls_64.c
index 58f51667e2e4..2ef9474d2bd2 100644
--- a/arch/x86/um/syscalls_64.c
+++ b/arch/x86/um/syscalls_64.c
@@ -80,7 +80,7 @@ SYSCALL_DEFINE2(arch_prctl, int, option, unsigned long, arg2)
 	return arch_prctl(current, option, (unsigned long __user *) arg2);
 }
 
-void arch_switch_to(struct task_struct *to)
+void arch_switch_to(struct task_struct *from, struct task_struct *to)
 {
 	if ((to->thread.arch.fs == 0) || (to->mm == NULL))
 		return;
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v6 08/21] um: add nommu mode for UML library mode
  2020-09-24  7:12       ` Hajime Tazaki
@ 2020-09-24  7:12         ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  7:12 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch, Hajime Tazaki

This patch introduces the nommu operation with UML code so that host
interface can be shrineked for broader environments support.

The nommu mode is implemneted as SUBARCH of arch/um, which places at
arch/um/nommu. This SUBARCH defines mode-specific code of memory
management as well as thread implementation, along with the uapi headers
to be exported to users.

The headers we introduce in this patch are simple wrappers to the
asm-generic headers or stubs for things we don't support, such as
ptrace, DMA, signals, ELF handling and low level processor operations.

nommu mode shares most of arch/um code (irq, thread_info, process
scheduling) but implements its own facilities, which are memory
management (nommu), thread primitives (struct arch_thread), system call
interface, and console.

The outlook of updated directory structure is as follows:

   arch/um
   |-- configs         (untouched)
   |-- drivers         unuse stdio_console.c for !MMU
   |-- include
   |   |-- asm         updated with !CONFIG_MMU
   |   |-- linux       (untouched)
   |   `-- shareda     updated with new functions
   |       `-- skas    (untouched)
   |   `-- uapi        added for upai header installation
   |-- kernel          updated to integrate with !MMU
   |   `-- skas        (untouched, don't use for !MMU)
   |-- nommu           SUBARCH dir (internally =um/nommu)
   |   |-- include
   |   |   |-- asm     headers for subarch specific info
   |   |   `-- uapi    headers for user-visible APIs
   |   `-- um
   |       `-- shared  headers for subarch specific info
   `-- scripts         added a script for header installation

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/nommu/Makefile                        |   1 +
 arch/um/nommu/Makefile.um                     |  18 +++
 arch/um/nommu/include/asm/Kbuild              |   6 +
 arch/um/nommu/include/asm/archparam.h         |   1 +
 arch/um/nommu/include/asm/atomic.h            |  11 ++
 arch/um/nommu/include/asm/atomic64.h          | 114 ++++++++++++++++++
 arch/um/nommu/include/asm/bitsperlong.h       |  12 ++
 arch/um/nommu/include/asm/byteorder.h         |   7 ++
 arch/um/nommu/include/asm/cpu.h               |  13 ++
 arch/um/nommu/include/asm/elf.h               |  15 +++
 arch/um/nommu/include/asm/mm_context.h        |   8 ++
 arch/um/nommu/include/asm/processor.h         |  46 +++++++
 arch/um/nommu/include/asm/ptrace.h            |  21 ++++
 arch/um/nommu/include/asm/sched.h             |  23 ++++
 arch/um/nommu/include/asm/segment.h           |   9 ++
 arch/um/nommu/include/uapi/asm/Kbuild         |   4 +
 arch/um/nommu/include/uapi/asm/bitsperlong.h  |  11 ++
 arch/um/nommu/include/uapi/asm/byteorder.h    |  11 ++
 arch/um/nommu/include/uapi/asm/sigcontext.h   |  12 ++
 arch/um/nommu/um/delay.c                      |  31 +++++
 arch/um/nommu/um/shared/sysdep/archsetjmp.h   |  13 ++
 arch/um/nommu/um/shared/sysdep/faultinfo.h    |   8 ++
 .../nommu/um/shared/sysdep/kernel-offsets.h   |  12 ++
 arch/um/nommu/um/shared/sysdep/mcontext.h     |   9 ++
 arch/um/nommu/um/shared/sysdep/ptrace.h       |  42 +++++++
 arch/um/nommu/um/shared/sysdep/ptrace_user.h  |   7 ++
 arch/um/nommu/um/unimplemented.c              |  70 +++++++++++
 arch/um/nommu/um/user_constants.h             |  13 ++
 28 files changed, 548 insertions(+)
 create mode 100644 arch/um/nommu/Makefile
 create mode 100644 arch/um/nommu/Makefile.um
 create mode 100644 arch/um/nommu/include/asm/Kbuild
 create mode 100644 arch/um/nommu/include/asm/archparam.h
 create mode 100644 arch/um/nommu/include/asm/atomic.h
 create mode 100644 arch/um/nommu/include/asm/atomic64.h
 create mode 100644 arch/um/nommu/include/asm/bitsperlong.h
 create mode 100644 arch/um/nommu/include/asm/byteorder.h
 create mode 100644 arch/um/nommu/include/asm/cpu.h
 create mode 100644 arch/um/nommu/include/asm/elf.h
 create mode 100644 arch/um/nommu/include/asm/mm_context.h
 create mode 100644 arch/um/nommu/include/asm/processor.h
 create mode 100644 arch/um/nommu/include/asm/ptrace.h
 create mode 100644 arch/um/nommu/include/asm/sched.h
 create mode 100644 arch/um/nommu/include/asm/segment.h
 create mode 100644 arch/um/nommu/include/uapi/asm/Kbuild
 create mode 100644 arch/um/nommu/include/uapi/asm/bitsperlong.h
 create mode 100644 arch/um/nommu/include/uapi/asm/byteorder.h
 create mode 100644 arch/um/nommu/include/uapi/asm/sigcontext.h
 create mode 100644 arch/um/nommu/um/delay.c
 create mode 100644 arch/um/nommu/um/shared/sysdep/archsetjmp.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/faultinfo.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/kernel-offsets.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/mcontext.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/ptrace.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/ptrace_user.h
 create mode 100644 arch/um/nommu/um/unimplemented.c
 create mode 100644 arch/um/nommu/um/user_constants.h

diff --git a/arch/um/nommu/Makefile b/arch/um/nommu/Makefile
new file mode 100644
index 000000000000..792d6005489e
--- /dev/null
+++ b/arch/um/nommu/Makefile
@@ -0,0 +1 @@
+#
diff --git a/arch/um/nommu/Makefile.um b/arch/um/nommu/Makefile.um
new file mode 100644
index 000000000000..6cb0e9494d05
--- /dev/null
+++ b/arch/um/nommu/Makefile.um
@@ -0,0 +1,18 @@
+KBUILD_CFLAGS += -fno-builtin -fPIC
+ELF_FORMAT=$(shell $(LD) -r -print-output-format)
+
+ifeq ($(shell uname -s), Linux)
+NPROC=$(shell nproc)
+else # e.g., FreeBSD
+NPROC=$(shell sysctl -n hw.ncpu)
+endif
+
+um_headers_install: $(objtree)/$(HOST_DIR)/include/generated/uapi/asm/syscall_defs.h headers
+	$(Q)$(srctree)/$(ARCH_DIR)/scripts/headers_install.py \
+		$(subst -j,-j$(NPROC),$(findstring -j,$(MAKEFLAGS))) \
+		$(INSTALL_PATH)/include
+
+$(objtree)/$(HOST_DIR)/include/generated/uapi/asm/syscall_defs.h: vmlinux
+	$(Q)$(OBJCOPY) -j .syscall_defs -O binary --set-section-flags .syscall_defs=alloc $< $@
+	$(Q) export tmpfile=$(shell mktemp); \
+	sed 's/\x0//g' $@ > $$tmpfile; mv $$tmpfile $@ ; rm -f $$tmpfile
diff --git a/arch/um/nommu/include/asm/Kbuild b/arch/um/nommu/include/asm/Kbuild
new file mode 100644
index 000000000000..2532e1a0a0d1
--- /dev/null
+++ b/arch/um/nommu/include/asm/Kbuild
@@ -0,0 +1,6 @@
+generic-y += cmpxchg.h
+generic-y += local64.h
+generic-y += seccomp.h
+generic-y += string.h
+generic-y += syscall.h
+generic-y += user.h
diff --git a/arch/um/nommu/include/asm/archparam.h b/arch/um/nommu/include/asm/archparam.h
new file mode 100644
index 000000000000..ea32a7d3cf1b
--- /dev/null
+++ b/arch/um/nommu/include/asm/archparam.h
@@ -0,0 +1 @@
+/* SPDX-License-Identifier: GPL-2.0 */
diff --git a/arch/um/nommu/include/asm/atomic.h b/arch/um/nommu/include/asm/atomic.h
new file mode 100644
index 000000000000..63e2e16bda92
--- /dev/null
+++ b/arch/um/nommu/include/asm/atomic.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_ATOMIC_H
+#define __UM_NOMMU_ATOMIC_H
+
+#include <asm-generic/atomic.h>
+
+#ifndef CONFIG_GENERIC_ATOMIC64
+#include "atomic64.h"
+#endif /* !CONFIG_GENERIC_ATOMIC64 */
+
+#endif
diff --git a/arch/um/nommu/include/asm/atomic64.h b/arch/um/nommu/include/asm/atomic64.h
new file mode 100644
index 000000000000..949360dea7af
--- /dev/null
+++ b/arch/um/nommu/include/asm/atomic64.h
@@ -0,0 +1,114 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_ATOMIC64_H
+#define __UM_NOMMU_ATOMIC64_H
+
+#include <linux/types.h>
+
+#ifdef CONFIG_SMP
+#error "SMP is not supported on this platform"
+#else
+#define ATOMIC64_OP(op, c_op)					\
+	static inline void atomic64_##op(s64 i, atomic64_t *v)	\
+	{							\
+		unsigned long flags;				\
+								\
+		raw_local_irq_save(flags);			\
+		v->counter = v->counter c_op i;			\
+		raw_local_irq_restore(flags);			\
+	}
+
+#define ATOMIC64_OP_RETURN(op, c_op)					\
+	static inline s64 atomic64_##op##_return(s64 i, atomic64_t *v)	\
+	{								\
+		unsigned long flags;					\
+		s64 ret;						\
+									\
+		raw_local_irq_save(flags);				\
+		ret = (v->counter = v->counter c_op i);			\
+		raw_local_irq_restore(flags);				\
+									\
+		return ret;						\
+	}
+
+#define ATOMIC64_FETCH_OP(op, c_op)					\
+	static inline s64 atomic64_fetch_##op(s64 i, atomic64_t *v)	\
+	{								\
+		unsigned long flags;					\
+		s64 ret;						\
+									\
+		raw_local_irq_save(flags);				\
+		ret = v->counter;					\
+		v->counter = v->counter c_op i;				\
+		raw_local_irq_restore(flags);				\
+									\
+		return ret;						\
+	}
+#endif /* CONFIG_SMP */
+
+#ifndef atomic64_add_return
+ATOMIC64_OP_RETURN(add, +)
+#endif
+
+#ifndef atomic64_sub_return
+	ATOMIC64_OP_RETURN(sub, -)
+#endif
+
+#ifndef atomic64_fetch_add
+	ATOMIC64_FETCH_OP(add, +)
+#endif
+
+#ifndef atomic64_fetch_sub
+	ATOMIC64_FETCH_OP(sub, -)
+#endif
+
+#ifndef atomic64_fetch_and
+	ATOMIC64_FETCH_OP(and, &)
+#endif
+
+#ifndef atomic64_fetch_or
+	ATOMIC64_FETCH_OP(or, |)
+#endif
+
+#ifndef atomic64_fetch_xor
+	ATOMIC64_FETCH_OP(xor, ^)
+#endif
+
+#ifndef atomic64_and
+	ATOMIC64_OP(and, &)
+#endif
+
+#ifndef atomic64_or
+	ATOMIC64_OP(or, |)
+#endif
+
+#ifndef atomic64_xor
+	ATOMIC64_OP(xor, ^)
+#endif
+
+#undef ATOMIC64_FETCH_OP
+#undef ATOMIC64_OP_RETURN
+#undef ATOMIC64_OP
+
+
+#define ATOMIC64_INIT(i)       { (i) }
+
+	static inline void atomic64_add(s64 i, atomic64_t *v)
+{
+	atomic64_add_return(i, v);
+}
+
+static inline void atomic64_sub(s64 i, atomic64_t *v)
+{
+	atomic64_sub_return(i, v);
+}
+
+#ifndef atomic64_read
+#define atomic64_read(v)       READ_ONCE((v)->counter)
+#endif
+
+#define atomic64_set(v, i) WRITE_ONCE(((v)->counter), (i))
+
+#define atomic64_xchg(ptr, v)          (xchg(&(ptr)->counter, (v)))
+#define atomic64_cmpxchg(v, old, new)  (cmpxchg(&((v)->counter), (old), (new)))
+
+#endif /* __LKL_ATOMIC64_H */
diff --git a/arch/um/nommu/include/asm/bitsperlong.h b/arch/um/nommu/include/asm/bitsperlong.h
new file mode 100644
index 000000000000..a150cee41e75
--- /dev/null
+++ b/arch/um/nommu/include/asm/bitsperlong.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_BITSPERLONG_H
+#define __UM_NOMMU_BITSPERLONG_H
+
+#include <uapi/asm/bitsperlong.h>
+
+#define BITS_PER_LONG __BITS_PER_LONG
+
+#define BITS_PER_LONG_LONG 64
+
+#endif
+
diff --git a/arch/um/nommu/include/asm/byteorder.h b/arch/um/nommu/include/asm/byteorder.h
new file mode 100644
index 000000000000..920a5fd26cad
--- /dev/null
+++ b/arch/um/nommu/include/asm/byteorder.h
@@ -0,0 +1,7 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_BYTEORDER_H
+#define __UM_NOMMU_BYTEORDER_H
+
+#include <uapi/asm/byteorder.h>
+
+#endif
diff --git a/arch/um/nommu/include/asm/cpu.h b/arch/um/nommu/include/asm/cpu.h
new file mode 100644
index 000000000000..c101c078ef21
--- /dev/null
+++ b/arch/um/nommu/include/asm/cpu.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_CPU_H
+#define __UM_NOMMU_CPU_H
+
+int lkl_cpu_get(void);
+void lkl_cpu_put(void);
+int lkl_cpu_try_run_irq(int irq);
+int lkl_cpu_init(void);
+void lkl_cpu_wait_shutdown(void);
+void lkl_cpu_change_owner(lkl_thread_t owner);
+void lkl_cpu_set_irqs_pending(void);
+
+#endif
diff --git a/arch/um/nommu/include/asm/elf.h b/arch/um/nommu/include/asm/elf.h
new file mode 100644
index 000000000000..edcf63edeed1
--- /dev/null
+++ b/arch/um/nommu/include/asm/elf.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_ELF_H
+#define __UM_NOMMU_ELF_H
+
+#define elf_check_arch(x) 0
+
+#ifdef CONFIG_64BIT
+#define ELF_CLASS ELFCLASS64
+#else
+#define ELF_CLASS ELFCLASS32
+#endif
+
+#define elf_gregset_t long
+#define elf_fpregset_t double
+#endif
diff --git a/arch/um/nommu/include/asm/mm_context.h b/arch/um/nommu/include/asm/mm_context.h
new file mode 100644
index 000000000000..a2e53984aabd
--- /dev/null
+++ b/arch/um/nommu/include/asm/mm_context.h
@@ -0,0 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_MM_CONTEXT_H
+#define __UM_NOMMU_MM_CONTEXT_H
+
+struct uml_arch_mm_context {
+};
+
+#endif
diff --git a/arch/um/nommu/include/asm/processor.h b/arch/um/nommu/include/asm/processor.h
new file mode 100644
index 000000000000..3e8ba870caaf
--- /dev/null
+++ b/arch/um/nommu/include/asm/processor.h
@@ -0,0 +1,46 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_PROCESSOR_H
+#define __UM_NOMMU_PROCESSOR_H
+
+#include <asm/host_ops.h>
+
+struct arch_thread {
+	struct lkl_sem *sched_sem;
+	bool dead;
+	lkl_thread_t tid;
+	struct lkl_jmp_buf sched_jb;
+	unsigned long stackend;
+};
+
+#include <asm/ptrace-generic.h>
+#include <asm/processor-generic.h>
+
+#define INIT_ARCH_THREAD {}
+#define task_pt_regs(tsk) (struct pt_regs *)(NULL)
+
+static inline void cpu_relax(void)
+{
+	unsigned long flags;
+
+	/* since this is usually called in a tight loop waiting for some
+	 * external condition (e.g. jiffies) lets run interrupts now to allow
+	 * the external condition to propagate
+	 */
+	local_irq_save(flags);
+	local_irq_restore(flags);
+}
+
+#define KSTK_EIP(tsk)	(0)
+#define KSTK_ESP(tsk)	(0)
+
+static inline void trap_init(void)
+{
+}
+
+static inline void arch_copy_thread(struct arch_thread *from,
+				    struct arch_thread *to)
+{
+	panic("unimplemented %s: fork isn't supported yet", __func__);
+}
+
+#endif
diff --git a/arch/um/nommu/include/asm/ptrace.h b/arch/um/nommu/include/asm/ptrace.h
new file mode 100644
index 000000000000..b214410e9825
--- /dev/null
+++ b/arch/um/nommu/include/asm/ptrace.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_PTRACE_H
+#define __UM_NOMMU_PTRACE_H
+
+#include <linux/errno.h>
+
+static int reg_dummy __attribute__((unused));
+
+#define PT_REGS_ORIG_SYSCALL(r) (reg_dummy)
+#define PT_REGS_SYSCALL_RET(r) (reg_dummy)
+#define PT_REGS_SET_SYSCALL_RETURN(r, res) (reg_dummy = (res))
+#define REGS_SP(r) (reg_dummy)
+
+#define user_mode(regs) 0
+#define kernel_mode(regs) 1
+#define profile_pc(regs) 0
+#define user_stack_pointer(regs) 0
+
+extern void new_thread_handler(void);
+
+#endif
diff --git a/arch/um/nommu/include/asm/sched.h b/arch/um/nommu/include/asm/sched.h
new file mode 100644
index 000000000000..a4496f482633
--- /dev/null
+++ b/arch/um/nommu/include/asm/sched.h
@@ -0,0 +1,23 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_SCHED_H
+#define __UM_NOMMU_SCHED_H
+
+#include <linux/sched.h>
+#include <uapi/asm/host_ops.h>
+
+static inline void thread_sched_jb(void)
+{
+	if (test_ti_thread_flag(current_thread_info(), TIF_HOST_THREAD)) {
+		set_ti_thread_flag(current_thread_info(), TIF_SCHED_JB);
+		set_current_state(TASK_UNINTERRUPTIBLE);
+		lkl_ops->jmp_buf_set(&current_thread_info()->task->thread.arch.sched_jb,
+				     schedule);
+	} else {
+		lkl_bug("%s can be used only for host task", __func__);
+	}
+}
+
+void switch_to_host_task(struct task_struct *);
+int host_task_stub(void *unused);
+
+#endif
diff --git a/arch/um/nommu/include/asm/segment.h b/arch/um/nommu/include/asm/segment.h
new file mode 100644
index 000000000000..5608da95cb60
--- /dev/null
+++ b/arch/um/nommu/include/asm/segment.h
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_SEGMENT_H
+#define __UM_NOMMU_SEGMENT_H
+
+typedef struct {
+	unsigned long seg;
+} mm_segment_t;
+
+#endif /* _ASM_LKL_SEGMENT_H */
diff --git a/arch/um/nommu/include/uapi/asm/Kbuild b/arch/um/nommu/include/uapi/asm/Kbuild
new file mode 100644
index 000000000000..4f79e5c4846d
--- /dev/null
+++ b/arch/um/nommu/include/uapi/asm/Kbuild
@@ -0,0 +1,4 @@
+# UAPI Header export list
+
+# no generated-y since we need special user headers handling
+# see arch/um/script/headers_install.py
diff --git a/arch/um/nommu/include/uapi/asm/bitsperlong.h b/arch/um/nommu/include/uapi/asm/bitsperlong.h
new file mode 100644
index 000000000000..852566ac2e52
--- /dev/null
+++ b/arch/um/nommu/include/uapi/asm/bitsperlong.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef __UM_NOMMU_UAPI_BITSPERLONG_H
+#define __UM_NOMMU_UAPI_BITSPERLONG_H
+
+#ifdef CONFIG_64BIT
+#define __BITS_PER_LONG 64
+#else
+#define __BITS_PER_LONG 32
+#endif
+
+#endif
diff --git a/arch/um/nommu/include/uapi/asm/byteorder.h b/arch/um/nommu/include/uapi/asm/byteorder.h
new file mode 100644
index 000000000000..e7ad11a751cf
--- /dev/null
+++ b/arch/um/nommu/include/uapi/asm/byteorder.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef __UM_NOMMU_UAPI_BYTEORDER_H
+#define __UM_NOMMU_UAPI_BYTEORDER_H
+
+#if defined(CONFIG_BIG_ENDIAN)
+#include <linux/byteorder/big_endian.h>
+#else
+#include <linux/byteorder/little_endian.h>
+#endif
+
+#endif
diff --git a/arch/um/nommu/include/uapi/asm/sigcontext.h b/arch/um/nommu/include/uapi/asm/sigcontext.h
new file mode 100644
index 000000000000..b934ae7f5550
--- /dev/null
+++ b/arch/um/nommu/include/uapi/asm/sigcontext.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef __UM_NOMMU_UAPI_SIGCONTEXT_H
+#define __UM_NOMMU_UAPI_SIGCONTEXT_H
+
+#include <asm/ptrace-generic.h>
+
+struct sigcontext {
+	struct pt_regs regs;
+	unsigned long oldmask;
+};
+
+#endif
diff --git a/arch/um/nommu/um/delay.c b/arch/um/nommu/um/delay.c
new file mode 100644
index 000000000000..58a366d9b5f0
--- /dev/null
+++ b/arch/um/nommu/um/delay.c
@@ -0,0 +1,31 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/jiffies.h>
+#include <linux/delay.h>
+#include <os.h>
+
+void __ndelay(unsigned long nsecs)
+{
+	long long start = os_nsecs();
+
+	while (os_nsecs() < start + nsecs)
+		;
+}
+
+void __udelay(unsigned long usecs)
+{
+	__ndelay(usecs * NSEC_PER_USEC);
+}
+
+void __const_udelay(unsigned long xloops)
+{
+	__udelay(xloops / 0x10c7ul);
+}
+
+void __delay(unsigned long loops)
+{
+	__ndelay(loops / 5);
+}
+
+void calibrate_delay(void)
+{
+}
diff --git a/arch/um/nommu/um/shared/sysdep/archsetjmp.h b/arch/um/nommu/um/shared/sysdep/archsetjmp.h
new file mode 100644
index 000000000000..4a5c10d7521b
--- /dev/null
+++ b/arch/um/nommu/um/shared/sysdep/archsetjmp.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ARCH_UM_SETJMP_H
+#define __ARCH_UM_SETJMP_H
+
+struct __jmp_buf {
+	unsigned long __dummy;
+};
+#define JB_IP __dummy
+#define JB_SP __dummy
+
+typedef struct __jmp_buf jmp_buf[1];
+
+#endif /* __ARCH_UM_SETJMP_H */
diff --git a/arch/um/nommu/um/shared/sysdep/faultinfo.h b/arch/um/nommu/um/shared/sysdep/faultinfo.h
new file mode 100644
index 000000000000..49210742212d
--- /dev/null
+++ b/arch/um/nommu/um/shared/sysdep/faultinfo.h
@@ -0,0 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ARCH_UM_FAULTINFO_H
+#define __ARCH_UM_FAULTINFO_H
+
+struct faultinfo {
+};
+
+#endif /* __ARCH_UM_FAULTINFO_H */
diff --git a/arch/um/nommu/um/shared/sysdep/kernel-offsets.h b/arch/um/nommu/um/shared/sysdep/kernel-offsets.h
new file mode 100644
index 000000000000..a004bffb7b8d
--- /dev/null
+++ b/arch/um/nommu/um/shared/sysdep/kernel-offsets.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <linux/stddef.h>
+#include <linux/sched.h>
+#include <linux/elf.h>
+#include <linux/crypto.h>
+#include <linux/kbuild.h>
+#include <asm/mman.h>
+
+void foo(void)
+{
+#include <common-offsets.h>
+}
diff --git a/arch/um/nommu/um/shared/sysdep/mcontext.h b/arch/um/nommu/um/shared/sysdep/mcontext.h
new file mode 100644
index 000000000000..b734012a74da
--- /dev/null
+++ b/arch/um/nommu/um/shared/sysdep/mcontext.h
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ARCH_UM_MCONTEXT_H
+#define __ARCH_UM_MCONTEXT_H
+
+extern void get_regs_from_mc(struct uml_pt_regs *regs, mcontext_t *mc);
+
+#define GET_FAULTINFO_FROM_MC(fi, mc) (fi = fi)
+
+#endif /* __ARCH_UM_MCONTEXT_H */
diff --git a/arch/um/nommu/um/shared/sysdep/ptrace.h b/arch/um/nommu/um/shared/sysdep/ptrace.h
new file mode 100644
index 000000000000..bfdfb520a21d
--- /dev/null
+++ b/arch/um/nommu/um/shared/sysdep/ptrace.h
@@ -0,0 +1,42 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ARCH_UM_PTRACE_H
+#define __ARCH_UM_PTRACE_H
+
+#include <generated/user_constants.h>
+#include <linux/errno.h>
+
+struct task_struct;
+
+#define UPT_SYSCALL_NR(r) ((r)->syscall)
+#define UPT_RESTART_SYSCALL(r) ((r)->syscall--) /* XXX */
+
+#define UPT_SP(r) 0
+#define UPT_IP(r) 0
+#define EMPTY_UML_PT_REGS { }
+
+#define MAX_REG_OFFSET (UM_FRAME_SIZE)
+#define MAX_REG_NR ((MAX_REG_OFFSET) / sizeof(unsigned long))
+
+/* unused */
+struct uml_pt_regs {
+	unsigned long gp[1];
+	unsigned long fp[1];
+	long faultinfo;
+	long syscall;
+	int is_user;
+};
+
+extern void arch_init_registers(int pid);
+
+static inline long arch_ptrace(struct task_struct *child,
+			       long request, unsigned long addr,
+			       unsigned long data)
+{
+	return -EINVAL;
+}
+
+static inline void ptrace_disable(struct task_struct *child) {}
+static inline void user_enable_single_step(struct task_struct *child) {}
+static inline void user_disable_single_step(struct task_struct *child) {}
+
+#endif /* __ARCH_UM_PTRACE_H */
diff --git a/arch/um/nommu/um/shared/sysdep/ptrace_user.h b/arch/um/nommu/um/shared/sysdep/ptrace_user.h
new file mode 100644
index 000000000000..86d5cb20fb9f
--- /dev/null
+++ b/arch/um/nommu/um/shared/sysdep/ptrace_user.h
@@ -0,0 +1,7 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ARCH_UM_PTRACE_USER_H
+#define __ARCH_UM_PTRACE_USER_H
+
+#define FP_SIZE 1
+
+#endif
diff --git a/arch/um/nommu/um/unimplemented.c b/arch/um/nommu/um/unimplemented.c
new file mode 100644
index 000000000000..fe33e02e39e5
--- /dev/null
+++ b/arch/um/nommu/um/unimplemented.c
@@ -0,0 +1,70 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/signal.h>
+#include <sysdep/ptrace.h>
+#include <asm/ptrace.h>
+
+/* physmem.c  */
+unsigned long high_physmem;
+
+/* x86/um/setjmp*.S  */
+void kernel_longjmp(void)
+{}
+void kernel_setjmp(void)
+{}
+
+/* trap.c */
+void relay_signal(int sig, struct siginfo *si, struct uml_pt_regs *regs)
+{}
+void bus_handler(int sig, struct siginfo *si, struct uml_pt_regs *regs)
+{}
+void segv_handler(int sig, struct siginfo *unused_si, struct uml_pt_regs *regs)
+{}
+void winch(int sig, struct siginfo *unused_si, struct uml_pt_regs *regs)
+{}
+
+/* tlb.c */
+void flush_tlb_kernel_vm(void)
+{}
+void force_flush_all(void)
+{}
+
+/* skas/process.c */
+void halt_skas(void)
+{}
+int is_skas_winch(int pid, int fd, void *data)
+{
+	return 0;
+}
+void reboot_skas(void)
+{}
+
+int __init start_uml(void)
+{
+	return 0;
+}
+
+/* exec.c */
+void flush_thread(void)
+{}
+
+/* x86/ptrace_64.c */
+int is_syscall(unsigned long addr)
+{
+	return 0;
+}
+
+
+/* x86/sysrq.c */
+void show_regs(struct pt_regs *regs)
+{}
+
+/* x86/signal.c */
+int setup_signal_stack_si(unsigned long stack_top, struct ksignal *ksig,
+			  struct pt_regs *regs, sigset_t *mask)
+{
+	return 0;
+}
+
+/* x86/bugs.c */
+void arch_check_bugs(void)
+{}
diff --git a/arch/um/nommu/um/user_constants.h b/arch/um/nommu/um/user_constants.h
new file mode 100644
index 000000000000..2245d3d24120
--- /dev/null
+++ b/arch/um/nommu/um/user_constants.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_USER_CONSTANTS_H
+#define __UM_NOMMU_USER_CONSTANTS_H
+
+/* XXX: put dummy values */
+#define UM_FRAME_SIZE 4
+#define HOST_FP_SIZE 1
+#define HOST_IP 1
+#define HOST_SP 2
+#define HOST_BP 3
+#define UM_NR_CPUS 1
+
+#endif
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v6 08/21] um: add nommu mode for UML library mode
@ 2020-09-24  7:12         ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  7:12 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Hajime Tazaki, retrage01

This patch introduces the nommu operation with UML code so that host
interface can be shrineked for broader environments support.

The nommu mode is implemneted as SUBARCH of arch/um, which places at
arch/um/nommu. This SUBARCH defines mode-specific code of memory
management as well as thread implementation, along with the uapi headers
to be exported to users.

The headers we introduce in this patch are simple wrappers to the
asm-generic headers or stubs for things we don't support, such as
ptrace, DMA, signals, ELF handling and low level processor operations.

nommu mode shares most of arch/um code (irq, thread_info, process
scheduling) but implements its own facilities, which are memory
management (nommu), thread primitives (struct arch_thread), system call
interface, and console.

The outlook of updated directory structure is as follows:

   arch/um
   |-- configs         (untouched)
   |-- drivers         unuse stdio_console.c for !MMU
   |-- include
   |   |-- asm         updated with !CONFIG_MMU
   |   |-- linux       (untouched)
   |   `-- shareda     updated with new functions
   |       `-- skas    (untouched)
   |   `-- uapi        added for upai header installation
   |-- kernel          updated to integrate with !MMU
   |   `-- skas        (untouched, don't use for !MMU)
   |-- nommu           SUBARCH dir (internally =um/nommu)
   |   |-- include
   |   |   |-- asm     headers for subarch specific info
   |   |   `-- uapi    headers for user-visible APIs
   |   `-- um
   |       `-- shared  headers for subarch specific info
   `-- scripts         added a script for header installation

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/nommu/Makefile                        |   1 +
 arch/um/nommu/Makefile.um                     |  18 +++
 arch/um/nommu/include/asm/Kbuild              |   6 +
 arch/um/nommu/include/asm/archparam.h         |   1 +
 arch/um/nommu/include/asm/atomic.h            |  11 ++
 arch/um/nommu/include/asm/atomic64.h          | 114 ++++++++++++++++++
 arch/um/nommu/include/asm/bitsperlong.h       |  12 ++
 arch/um/nommu/include/asm/byteorder.h         |   7 ++
 arch/um/nommu/include/asm/cpu.h               |  13 ++
 arch/um/nommu/include/asm/elf.h               |  15 +++
 arch/um/nommu/include/asm/mm_context.h        |   8 ++
 arch/um/nommu/include/asm/processor.h         |  46 +++++++
 arch/um/nommu/include/asm/ptrace.h            |  21 ++++
 arch/um/nommu/include/asm/sched.h             |  23 ++++
 arch/um/nommu/include/asm/segment.h           |   9 ++
 arch/um/nommu/include/uapi/asm/Kbuild         |   4 +
 arch/um/nommu/include/uapi/asm/bitsperlong.h  |  11 ++
 arch/um/nommu/include/uapi/asm/byteorder.h    |  11 ++
 arch/um/nommu/include/uapi/asm/sigcontext.h   |  12 ++
 arch/um/nommu/um/delay.c                      |  31 +++++
 arch/um/nommu/um/shared/sysdep/archsetjmp.h   |  13 ++
 arch/um/nommu/um/shared/sysdep/faultinfo.h    |   8 ++
 .../nommu/um/shared/sysdep/kernel-offsets.h   |  12 ++
 arch/um/nommu/um/shared/sysdep/mcontext.h     |   9 ++
 arch/um/nommu/um/shared/sysdep/ptrace.h       |  42 +++++++
 arch/um/nommu/um/shared/sysdep/ptrace_user.h  |   7 ++
 arch/um/nommu/um/unimplemented.c              |  70 +++++++++++
 arch/um/nommu/um/user_constants.h             |  13 ++
 28 files changed, 548 insertions(+)
 create mode 100644 arch/um/nommu/Makefile
 create mode 100644 arch/um/nommu/Makefile.um
 create mode 100644 arch/um/nommu/include/asm/Kbuild
 create mode 100644 arch/um/nommu/include/asm/archparam.h
 create mode 100644 arch/um/nommu/include/asm/atomic.h
 create mode 100644 arch/um/nommu/include/asm/atomic64.h
 create mode 100644 arch/um/nommu/include/asm/bitsperlong.h
 create mode 100644 arch/um/nommu/include/asm/byteorder.h
 create mode 100644 arch/um/nommu/include/asm/cpu.h
 create mode 100644 arch/um/nommu/include/asm/elf.h
 create mode 100644 arch/um/nommu/include/asm/mm_context.h
 create mode 100644 arch/um/nommu/include/asm/processor.h
 create mode 100644 arch/um/nommu/include/asm/ptrace.h
 create mode 100644 arch/um/nommu/include/asm/sched.h
 create mode 100644 arch/um/nommu/include/asm/segment.h
 create mode 100644 arch/um/nommu/include/uapi/asm/Kbuild
 create mode 100644 arch/um/nommu/include/uapi/asm/bitsperlong.h
 create mode 100644 arch/um/nommu/include/uapi/asm/byteorder.h
 create mode 100644 arch/um/nommu/include/uapi/asm/sigcontext.h
 create mode 100644 arch/um/nommu/um/delay.c
 create mode 100644 arch/um/nommu/um/shared/sysdep/archsetjmp.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/faultinfo.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/kernel-offsets.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/mcontext.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/ptrace.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/ptrace_user.h
 create mode 100644 arch/um/nommu/um/unimplemented.c
 create mode 100644 arch/um/nommu/um/user_constants.h

diff --git a/arch/um/nommu/Makefile b/arch/um/nommu/Makefile
new file mode 100644
index 000000000000..792d6005489e
--- /dev/null
+++ b/arch/um/nommu/Makefile
@@ -0,0 +1 @@
+#
diff --git a/arch/um/nommu/Makefile.um b/arch/um/nommu/Makefile.um
new file mode 100644
index 000000000000..6cb0e9494d05
--- /dev/null
+++ b/arch/um/nommu/Makefile.um
@@ -0,0 +1,18 @@
+KBUILD_CFLAGS += -fno-builtin -fPIC
+ELF_FORMAT=$(shell $(LD) -r -print-output-format)
+
+ifeq ($(shell uname -s), Linux)
+NPROC=$(shell nproc)
+else # e.g., FreeBSD
+NPROC=$(shell sysctl -n hw.ncpu)
+endif
+
+um_headers_install: $(objtree)/$(HOST_DIR)/include/generated/uapi/asm/syscall_defs.h headers
+	$(Q)$(srctree)/$(ARCH_DIR)/scripts/headers_install.py \
+		$(subst -j,-j$(NPROC),$(findstring -j,$(MAKEFLAGS))) \
+		$(INSTALL_PATH)/include
+
+$(objtree)/$(HOST_DIR)/include/generated/uapi/asm/syscall_defs.h: vmlinux
+	$(Q)$(OBJCOPY) -j .syscall_defs -O binary --set-section-flags .syscall_defs=alloc $< $@
+	$(Q) export tmpfile=$(shell mktemp); \
+	sed 's/\x0//g' $@ > $$tmpfile; mv $$tmpfile $@ ; rm -f $$tmpfile
diff --git a/arch/um/nommu/include/asm/Kbuild b/arch/um/nommu/include/asm/Kbuild
new file mode 100644
index 000000000000..2532e1a0a0d1
--- /dev/null
+++ b/arch/um/nommu/include/asm/Kbuild
@@ -0,0 +1,6 @@
+generic-y += cmpxchg.h
+generic-y += local64.h
+generic-y += seccomp.h
+generic-y += string.h
+generic-y += syscall.h
+generic-y += user.h
diff --git a/arch/um/nommu/include/asm/archparam.h b/arch/um/nommu/include/asm/archparam.h
new file mode 100644
index 000000000000..ea32a7d3cf1b
--- /dev/null
+++ b/arch/um/nommu/include/asm/archparam.h
@@ -0,0 +1 @@
+/* SPDX-License-Identifier: GPL-2.0 */
diff --git a/arch/um/nommu/include/asm/atomic.h b/arch/um/nommu/include/asm/atomic.h
new file mode 100644
index 000000000000..63e2e16bda92
--- /dev/null
+++ b/arch/um/nommu/include/asm/atomic.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_ATOMIC_H
+#define __UM_NOMMU_ATOMIC_H
+
+#include <asm-generic/atomic.h>
+
+#ifndef CONFIG_GENERIC_ATOMIC64
+#include "atomic64.h"
+#endif /* !CONFIG_GENERIC_ATOMIC64 */
+
+#endif
diff --git a/arch/um/nommu/include/asm/atomic64.h b/arch/um/nommu/include/asm/atomic64.h
new file mode 100644
index 000000000000..949360dea7af
--- /dev/null
+++ b/arch/um/nommu/include/asm/atomic64.h
@@ -0,0 +1,114 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_ATOMIC64_H
+#define __UM_NOMMU_ATOMIC64_H
+
+#include <linux/types.h>
+
+#ifdef CONFIG_SMP
+#error "SMP is not supported on this platform"
+#else
+#define ATOMIC64_OP(op, c_op)					\
+	static inline void atomic64_##op(s64 i, atomic64_t *v)	\
+	{							\
+		unsigned long flags;				\
+								\
+		raw_local_irq_save(flags);			\
+		v->counter = v->counter c_op i;			\
+		raw_local_irq_restore(flags);			\
+	}
+
+#define ATOMIC64_OP_RETURN(op, c_op)					\
+	static inline s64 atomic64_##op##_return(s64 i, atomic64_t *v)	\
+	{								\
+		unsigned long flags;					\
+		s64 ret;						\
+									\
+		raw_local_irq_save(flags);				\
+		ret = (v->counter = v->counter c_op i);			\
+		raw_local_irq_restore(flags);				\
+									\
+		return ret;						\
+	}
+
+#define ATOMIC64_FETCH_OP(op, c_op)					\
+	static inline s64 atomic64_fetch_##op(s64 i, atomic64_t *v)	\
+	{								\
+		unsigned long flags;					\
+		s64 ret;						\
+									\
+		raw_local_irq_save(flags);				\
+		ret = v->counter;					\
+		v->counter = v->counter c_op i;				\
+		raw_local_irq_restore(flags);				\
+									\
+		return ret;						\
+	}
+#endif /* CONFIG_SMP */
+
+#ifndef atomic64_add_return
+ATOMIC64_OP_RETURN(add, +)
+#endif
+
+#ifndef atomic64_sub_return
+	ATOMIC64_OP_RETURN(sub, -)
+#endif
+
+#ifndef atomic64_fetch_add
+	ATOMIC64_FETCH_OP(add, +)
+#endif
+
+#ifndef atomic64_fetch_sub
+	ATOMIC64_FETCH_OP(sub, -)
+#endif
+
+#ifndef atomic64_fetch_and
+	ATOMIC64_FETCH_OP(and, &)
+#endif
+
+#ifndef atomic64_fetch_or
+	ATOMIC64_FETCH_OP(or, |)
+#endif
+
+#ifndef atomic64_fetch_xor
+	ATOMIC64_FETCH_OP(xor, ^)
+#endif
+
+#ifndef atomic64_and
+	ATOMIC64_OP(and, &)
+#endif
+
+#ifndef atomic64_or
+	ATOMIC64_OP(or, |)
+#endif
+
+#ifndef atomic64_xor
+	ATOMIC64_OP(xor, ^)
+#endif
+
+#undef ATOMIC64_FETCH_OP
+#undef ATOMIC64_OP_RETURN
+#undef ATOMIC64_OP
+
+
+#define ATOMIC64_INIT(i)       { (i) }
+
+	static inline void atomic64_add(s64 i, atomic64_t *v)
+{
+	atomic64_add_return(i, v);
+}
+
+static inline void atomic64_sub(s64 i, atomic64_t *v)
+{
+	atomic64_sub_return(i, v);
+}
+
+#ifndef atomic64_read
+#define atomic64_read(v)       READ_ONCE((v)->counter)
+#endif
+
+#define atomic64_set(v, i) WRITE_ONCE(((v)->counter), (i))
+
+#define atomic64_xchg(ptr, v)          (xchg(&(ptr)->counter, (v)))
+#define atomic64_cmpxchg(v, old, new)  (cmpxchg(&((v)->counter), (old), (new)))
+
+#endif /* __LKL_ATOMIC64_H */
diff --git a/arch/um/nommu/include/asm/bitsperlong.h b/arch/um/nommu/include/asm/bitsperlong.h
new file mode 100644
index 000000000000..a150cee41e75
--- /dev/null
+++ b/arch/um/nommu/include/asm/bitsperlong.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_BITSPERLONG_H
+#define __UM_NOMMU_BITSPERLONG_H
+
+#include <uapi/asm/bitsperlong.h>
+
+#define BITS_PER_LONG __BITS_PER_LONG
+
+#define BITS_PER_LONG_LONG 64
+
+#endif
+
diff --git a/arch/um/nommu/include/asm/byteorder.h b/arch/um/nommu/include/asm/byteorder.h
new file mode 100644
index 000000000000..920a5fd26cad
--- /dev/null
+++ b/arch/um/nommu/include/asm/byteorder.h
@@ -0,0 +1,7 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_BYTEORDER_H
+#define __UM_NOMMU_BYTEORDER_H
+
+#include <uapi/asm/byteorder.h>
+
+#endif
diff --git a/arch/um/nommu/include/asm/cpu.h b/arch/um/nommu/include/asm/cpu.h
new file mode 100644
index 000000000000..c101c078ef21
--- /dev/null
+++ b/arch/um/nommu/include/asm/cpu.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_CPU_H
+#define __UM_NOMMU_CPU_H
+
+int lkl_cpu_get(void);
+void lkl_cpu_put(void);
+int lkl_cpu_try_run_irq(int irq);
+int lkl_cpu_init(void);
+void lkl_cpu_wait_shutdown(void);
+void lkl_cpu_change_owner(lkl_thread_t owner);
+void lkl_cpu_set_irqs_pending(void);
+
+#endif
diff --git a/arch/um/nommu/include/asm/elf.h b/arch/um/nommu/include/asm/elf.h
new file mode 100644
index 000000000000..edcf63edeed1
--- /dev/null
+++ b/arch/um/nommu/include/asm/elf.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_ELF_H
+#define __UM_NOMMU_ELF_H
+
+#define elf_check_arch(x) 0
+
+#ifdef CONFIG_64BIT
+#define ELF_CLASS ELFCLASS64
+#else
+#define ELF_CLASS ELFCLASS32
+#endif
+
+#define elf_gregset_t long
+#define elf_fpregset_t double
+#endif
diff --git a/arch/um/nommu/include/asm/mm_context.h b/arch/um/nommu/include/asm/mm_context.h
new file mode 100644
index 000000000000..a2e53984aabd
--- /dev/null
+++ b/arch/um/nommu/include/asm/mm_context.h
@@ -0,0 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_MM_CONTEXT_H
+#define __UM_NOMMU_MM_CONTEXT_H
+
+struct uml_arch_mm_context {
+};
+
+#endif
diff --git a/arch/um/nommu/include/asm/processor.h b/arch/um/nommu/include/asm/processor.h
new file mode 100644
index 000000000000..3e8ba870caaf
--- /dev/null
+++ b/arch/um/nommu/include/asm/processor.h
@@ -0,0 +1,46 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_PROCESSOR_H
+#define __UM_NOMMU_PROCESSOR_H
+
+#include <asm/host_ops.h>
+
+struct arch_thread {
+	struct lkl_sem *sched_sem;
+	bool dead;
+	lkl_thread_t tid;
+	struct lkl_jmp_buf sched_jb;
+	unsigned long stackend;
+};
+
+#include <asm/ptrace-generic.h>
+#include <asm/processor-generic.h>
+
+#define INIT_ARCH_THREAD {}
+#define task_pt_regs(tsk) (struct pt_regs *)(NULL)
+
+static inline void cpu_relax(void)
+{
+	unsigned long flags;
+
+	/* since this is usually called in a tight loop waiting for some
+	 * external condition (e.g. jiffies) lets run interrupts now to allow
+	 * the external condition to propagate
+	 */
+	local_irq_save(flags);
+	local_irq_restore(flags);
+}
+
+#define KSTK_EIP(tsk)	(0)
+#define KSTK_ESP(tsk)	(0)
+
+static inline void trap_init(void)
+{
+}
+
+static inline void arch_copy_thread(struct arch_thread *from,
+				    struct arch_thread *to)
+{
+	panic("unimplemented %s: fork isn't supported yet", __func__);
+}
+
+#endif
diff --git a/arch/um/nommu/include/asm/ptrace.h b/arch/um/nommu/include/asm/ptrace.h
new file mode 100644
index 000000000000..b214410e9825
--- /dev/null
+++ b/arch/um/nommu/include/asm/ptrace.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_PTRACE_H
+#define __UM_NOMMU_PTRACE_H
+
+#include <linux/errno.h>
+
+static int reg_dummy __attribute__((unused));
+
+#define PT_REGS_ORIG_SYSCALL(r) (reg_dummy)
+#define PT_REGS_SYSCALL_RET(r) (reg_dummy)
+#define PT_REGS_SET_SYSCALL_RETURN(r, res) (reg_dummy = (res))
+#define REGS_SP(r) (reg_dummy)
+
+#define user_mode(regs) 0
+#define kernel_mode(regs) 1
+#define profile_pc(regs) 0
+#define user_stack_pointer(regs) 0
+
+extern void new_thread_handler(void);
+
+#endif
diff --git a/arch/um/nommu/include/asm/sched.h b/arch/um/nommu/include/asm/sched.h
new file mode 100644
index 000000000000..a4496f482633
--- /dev/null
+++ b/arch/um/nommu/include/asm/sched.h
@@ -0,0 +1,23 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_SCHED_H
+#define __UM_NOMMU_SCHED_H
+
+#include <linux/sched.h>
+#include <uapi/asm/host_ops.h>
+
+static inline void thread_sched_jb(void)
+{
+	if (test_ti_thread_flag(current_thread_info(), TIF_HOST_THREAD)) {
+		set_ti_thread_flag(current_thread_info(), TIF_SCHED_JB);
+		set_current_state(TASK_UNINTERRUPTIBLE);
+		lkl_ops->jmp_buf_set(&current_thread_info()->task->thread.arch.sched_jb,
+				     schedule);
+	} else {
+		lkl_bug("%s can be used only for host task", __func__);
+	}
+}
+
+void switch_to_host_task(struct task_struct *);
+int host_task_stub(void *unused);
+
+#endif
diff --git a/arch/um/nommu/include/asm/segment.h b/arch/um/nommu/include/asm/segment.h
new file mode 100644
index 000000000000..5608da95cb60
--- /dev/null
+++ b/arch/um/nommu/include/asm/segment.h
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_SEGMENT_H
+#define __UM_NOMMU_SEGMENT_H
+
+typedef struct {
+	unsigned long seg;
+} mm_segment_t;
+
+#endif /* _ASM_LKL_SEGMENT_H */
diff --git a/arch/um/nommu/include/uapi/asm/Kbuild b/arch/um/nommu/include/uapi/asm/Kbuild
new file mode 100644
index 000000000000..4f79e5c4846d
--- /dev/null
+++ b/arch/um/nommu/include/uapi/asm/Kbuild
@@ -0,0 +1,4 @@
+# UAPI Header export list
+
+# no generated-y since we need special user headers handling
+# see arch/um/script/headers_install.py
diff --git a/arch/um/nommu/include/uapi/asm/bitsperlong.h b/arch/um/nommu/include/uapi/asm/bitsperlong.h
new file mode 100644
index 000000000000..852566ac2e52
--- /dev/null
+++ b/arch/um/nommu/include/uapi/asm/bitsperlong.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef __UM_NOMMU_UAPI_BITSPERLONG_H
+#define __UM_NOMMU_UAPI_BITSPERLONG_H
+
+#ifdef CONFIG_64BIT
+#define __BITS_PER_LONG 64
+#else
+#define __BITS_PER_LONG 32
+#endif
+
+#endif
diff --git a/arch/um/nommu/include/uapi/asm/byteorder.h b/arch/um/nommu/include/uapi/asm/byteorder.h
new file mode 100644
index 000000000000..e7ad11a751cf
--- /dev/null
+++ b/arch/um/nommu/include/uapi/asm/byteorder.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef __UM_NOMMU_UAPI_BYTEORDER_H
+#define __UM_NOMMU_UAPI_BYTEORDER_H
+
+#if defined(CONFIG_BIG_ENDIAN)
+#include <linux/byteorder/big_endian.h>
+#else
+#include <linux/byteorder/little_endian.h>
+#endif
+
+#endif
diff --git a/arch/um/nommu/include/uapi/asm/sigcontext.h b/arch/um/nommu/include/uapi/asm/sigcontext.h
new file mode 100644
index 000000000000..b934ae7f5550
--- /dev/null
+++ b/arch/um/nommu/include/uapi/asm/sigcontext.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef __UM_NOMMU_UAPI_SIGCONTEXT_H
+#define __UM_NOMMU_UAPI_SIGCONTEXT_H
+
+#include <asm/ptrace-generic.h>
+
+struct sigcontext {
+	struct pt_regs regs;
+	unsigned long oldmask;
+};
+
+#endif
diff --git a/arch/um/nommu/um/delay.c b/arch/um/nommu/um/delay.c
new file mode 100644
index 000000000000..58a366d9b5f0
--- /dev/null
+++ b/arch/um/nommu/um/delay.c
@@ -0,0 +1,31 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/jiffies.h>
+#include <linux/delay.h>
+#include <os.h>
+
+void __ndelay(unsigned long nsecs)
+{
+	long long start = os_nsecs();
+
+	while (os_nsecs() < start + nsecs)
+		;
+}
+
+void __udelay(unsigned long usecs)
+{
+	__ndelay(usecs * NSEC_PER_USEC);
+}
+
+void __const_udelay(unsigned long xloops)
+{
+	__udelay(xloops / 0x10c7ul);
+}
+
+void __delay(unsigned long loops)
+{
+	__ndelay(loops / 5);
+}
+
+void calibrate_delay(void)
+{
+}
diff --git a/arch/um/nommu/um/shared/sysdep/archsetjmp.h b/arch/um/nommu/um/shared/sysdep/archsetjmp.h
new file mode 100644
index 000000000000..4a5c10d7521b
--- /dev/null
+++ b/arch/um/nommu/um/shared/sysdep/archsetjmp.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ARCH_UM_SETJMP_H
+#define __ARCH_UM_SETJMP_H
+
+struct __jmp_buf {
+	unsigned long __dummy;
+};
+#define JB_IP __dummy
+#define JB_SP __dummy
+
+typedef struct __jmp_buf jmp_buf[1];
+
+#endif /* __ARCH_UM_SETJMP_H */
diff --git a/arch/um/nommu/um/shared/sysdep/faultinfo.h b/arch/um/nommu/um/shared/sysdep/faultinfo.h
new file mode 100644
index 000000000000..49210742212d
--- /dev/null
+++ b/arch/um/nommu/um/shared/sysdep/faultinfo.h
@@ -0,0 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ARCH_UM_FAULTINFO_H
+#define __ARCH_UM_FAULTINFO_H
+
+struct faultinfo {
+};
+
+#endif /* __ARCH_UM_FAULTINFO_H */
diff --git a/arch/um/nommu/um/shared/sysdep/kernel-offsets.h b/arch/um/nommu/um/shared/sysdep/kernel-offsets.h
new file mode 100644
index 000000000000..a004bffb7b8d
--- /dev/null
+++ b/arch/um/nommu/um/shared/sysdep/kernel-offsets.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <linux/stddef.h>
+#include <linux/sched.h>
+#include <linux/elf.h>
+#include <linux/crypto.h>
+#include <linux/kbuild.h>
+#include <asm/mman.h>
+
+void foo(void)
+{
+#include <common-offsets.h>
+}
diff --git a/arch/um/nommu/um/shared/sysdep/mcontext.h b/arch/um/nommu/um/shared/sysdep/mcontext.h
new file mode 100644
index 000000000000..b734012a74da
--- /dev/null
+++ b/arch/um/nommu/um/shared/sysdep/mcontext.h
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ARCH_UM_MCONTEXT_H
+#define __ARCH_UM_MCONTEXT_H
+
+extern void get_regs_from_mc(struct uml_pt_regs *regs, mcontext_t *mc);
+
+#define GET_FAULTINFO_FROM_MC(fi, mc) (fi = fi)
+
+#endif /* __ARCH_UM_MCONTEXT_H */
diff --git a/arch/um/nommu/um/shared/sysdep/ptrace.h b/arch/um/nommu/um/shared/sysdep/ptrace.h
new file mode 100644
index 000000000000..bfdfb520a21d
--- /dev/null
+++ b/arch/um/nommu/um/shared/sysdep/ptrace.h
@@ -0,0 +1,42 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ARCH_UM_PTRACE_H
+#define __ARCH_UM_PTRACE_H
+
+#include <generated/user_constants.h>
+#include <linux/errno.h>
+
+struct task_struct;
+
+#define UPT_SYSCALL_NR(r) ((r)->syscall)
+#define UPT_RESTART_SYSCALL(r) ((r)->syscall--) /* XXX */
+
+#define UPT_SP(r) 0
+#define UPT_IP(r) 0
+#define EMPTY_UML_PT_REGS { }
+
+#define MAX_REG_OFFSET (UM_FRAME_SIZE)
+#define MAX_REG_NR ((MAX_REG_OFFSET) / sizeof(unsigned long))
+
+/* unused */
+struct uml_pt_regs {
+	unsigned long gp[1];
+	unsigned long fp[1];
+	long faultinfo;
+	long syscall;
+	int is_user;
+};
+
+extern void arch_init_registers(int pid);
+
+static inline long arch_ptrace(struct task_struct *child,
+			       long request, unsigned long addr,
+			       unsigned long data)
+{
+	return -EINVAL;
+}
+
+static inline void ptrace_disable(struct task_struct *child) {}
+static inline void user_enable_single_step(struct task_struct *child) {}
+static inline void user_disable_single_step(struct task_struct *child) {}
+
+#endif /* __ARCH_UM_PTRACE_H */
diff --git a/arch/um/nommu/um/shared/sysdep/ptrace_user.h b/arch/um/nommu/um/shared/sysdep/ptrace_user.h
new file mode 100644
index 000000000000..86d5cb20fb9f
--- /dev/null
+++ b/arch/um/nommu/um/shared/sysdep/ptrace_user.h
@@ -0,0 +1,7 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ARCH_UM_PTRACE_USER_H
+#define __ARCH_UM_PTRACE_USER_H
+
+#define FP_SIZE 1
+
+#endif
diff --git a/arch/um/nommu/um/unimplemented.c b/arch/um/nommu/um/unimplemented.c
new file mode 100644
index 000000000000..fe33e02e39e5
--- /dev/null
+++ b/arch/um/nommu/um/unimplemented.c
@@ -0,0 +1,70 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/signal.h>
+#include <sysdep/ptrace.h>
+#include <asm/ptrace.h>
+
+/* physmem.c  */
+unsigned long high_physmem;
+
+/* x86/um/setjmp*.S  */
+void kernel_longjmp(void)
+{}
+void kernel_setjmp(void)
+{}
+
+/* trap.c */
+void relay_signal(int sig, struct siginfo *si, struct uml_pt_regs *regs)
+{}
+void bus_handler(int sig, struct siginfo *si, struct uml_pt_regs *regs)
+{}
+void segv_handler(int sig, struct siginfo *unused_si, struct uml_pt_regs *regs)
+{}
+void winch(int sig, struct siginfo *unused_si, struct uml_pt_regs *regs)
+{}
+
+/* tlb.c */
+void flush_tlb_kernel_vm(void)
+{}
+void force_flush_all(void)
+{}
+
+/* skas/process.c */
+void halt_skas(void)
+{}
+int is_skas_winch(int pid, int fd, void *data)
+{
+	return 0;
+}
+void reboot_skas(void)
+{}
+
+int __init start_uml(void)
+{
+	return 0;
+}
+
+/* exec.c */
+void flush_thread(void)
+{}
+
+/* x86/ptrace_64.c */
+int is_syscall(unsigned long addr)
+{
+	return 0;
+}
+
+
+/* x86/sysrq.c */
+void show_regs(struct pt_regs *regs)
+{}
+
+/* x86/signal.c */
+int setup_signal_stack_si(unsigned long stack_top, struct ksignal *ksig,
+			  struct pt_regs *regs, sigset_t *mask)
+{
+	return 0;
+}
+
+/* x86/bugs.c */
+void arch_check_bugs(void)
+{}
diff --git a/arch/um/nommu/um/user_constants.h b/arch/um/nommu/um/user_constants.h
new file mode 100644
index 000000000000..2245d3d24120
--- /dev/null
+++ b/arch/um/nommu/um/user_constants.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_USER_CONSTANTS_H
+#define __UM_NOMMU_USER_CONSTANTS_H
+
+/* XXX: put dummy values */
+#define UM_FRAME_SIZE 4
+#define HOST_FP_SIZE 1
+#define HOST_IP 1
+#define HOST_SP 2
+#define HOST_BP 3
+#define UM_NR_CPUS 1
+
+#endif
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v6 09/21] um: nommu: host interface
  2020-09-24  7:12       ` Hajime Tazaki
@ 2020-09-24  7:12         ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  7:12 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch, Hajime Tazaki

This patch introduces the host operations that define the interface
between the LKL and the host. These operations must be provided either
by a host library or by the application itself.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/include/asm/host_ops.h            |  9 ++++++++
 arch/um/nommu/include/uapi/asm/host_ops.h | 25 +++++++++++++++++++++++
 2 files changed, 34 insertions(+)
 create mode 100644 arch/um/include/asm/host_ops.h
 create mode 100644 arch/um/nommu/include/uapi/asm/host_ops.h

diff --git a/arch/um/include/asm/host_ops.h b/arch/um/include/asm/host_ops.h
new file mode 100644
index 000000000000..f52423cc4ced
--- /dev/null
+++ b/arch/um/include/asm/host_ops.h
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_HOST_OPS_H
+#define _ASM_LKL_HOST_OPS_H
+
+#include <uapi/asm/host_ops.h>
+
+extern struct lkl_host_operations *lkl_ops;
+
+#endif
diff --git a/arch/um/nommu/include/uapi/asm/host_ops.h b/arch/um/nommu/include/uapi/asm/host_ops.h
new file mode 100644
index 000000000000..d3dad11b459e
--- /dev/null
+++ b/arch/um/nommu/include/uapi/asm/host_ops.h
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef __UM_NOMMU_UAPI_HOST_OPS_H
+#define __UM_NOMMU_UAPI_HOST_OPS_H
+
+/* Defined in {posix,nt}-host.c */
+struct lkl_mutex;
+struct lkl_sem;
+typedef unsigned long lkl_thread_t;
+struct lkl_jmp_buf {
+	unsigned long buf[128];
+};
+
+/**
+ * lkl_host_operations - host operations used by the Linux kernel
+ *
+ * These operations must be provided by a host library or by the application
+ * itself.
+ *
+ */
+struct lkl_host_operations {
+};
+
+void lkl_bug(const char *fmt, ...);
+
+#endif
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v6 09/21] um: nommu: host interface
@ 2020-09-24  7:12         ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  7:12 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Hajime Tazaki, retrage01

This patch introduces the host operations that define the interface
between the LKL and the host. These operations must be provided either
by a host library or by the application itself.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/include/asm/host_ops.h            |  9 ++++++++
 arch/um/nommu/include/uapi/asm/host_ops.h | 25 +++++++++++++++++++++++
 2 files changed, 34 insertions(+)
 create mode 100644 arch/um/include/asm/host_ops.h
 create mode 100644 arch/um/nommu/include/uapi/asm/host_ops.h

diff --git a/arch/um/include/asm/host_ops.h b/arch/um/include/asm/host_ops.h
new file mode 100644
index 000000000000..f52423cc4ced
--- /dev/null
+++ b/arch/um/include/asm/host_ops.h
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_HOST_OPS_H
+#define _ASM_LKL_HOST_OPS_H
+
+#include <uapi/asm/host_ops.h>
+
+extern struct lkl_host_operations *lkl_ops;
+
+#endif
diff --git a/arch/um/nommu/include/uapi/asm/host_ops.h b/arch/um/nommu/include/uapi/asm/host_ops.h
new file mode 100644
index 000000000000..d3dad11b459e
--- /dev/null
+++ b/arch/um/nommu/include/uapi/asm/host_ops.h
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef __UM_NOMMU_UAPI_HOST_OPS_H
+#define __UM_NOMMU_UAPI_HOST_OPS_H
+
+/* Defined in {posix,nt}-host.c */
+struct lkl_mutex;
+struct lkl_sem;
+typedef unsigned long lkl_thread_t;
+struct lkl_jmp_buf {
+	unsigned long buf[128];
+};
+
+/**
+ * lkl_host_operations - host operations used by the Linux kernel
+ *
+ * These operations must be provided by a host library or by the application
+ * itself.
+ *
+ */
+struct lkl_host_operations {
+};
+
+void lkl_bug(const char *fmt, ...);
+
+#endif
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v6 10/21] um: nommu: memory handling
  2020-09-24  7:12       ` Hajime Tazaki
@ 2020-09-24  7:12         ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  7:12 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch, Hajime Tazaki

nommu mode follows the way !CONFIG_MMU architecture of other linux
archs.  Ther is not much work left to do other than initializing the
boot allocator and providing the page and page table definitions.

The backstore memory is allocated via a host operation and the memory
size to be used is specified when the kernel is started, in the
lkl_start_kernel call.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/include/asm/mmu.h                 |  3 +
 arch/um/include/asm/mmu_context.h         |  8 +++
 arch/um/include/asm/page.h                | 15 ++++
 arch/um/include/asm/pgtable.h             | 27 +++++++
 arch/um/include/asm/uaccess.h             |  6 ++
 arch/um/nommu/include/uapi/asm/host_ops.h |  5 ++
 arch/um/nommu/um/bootmem.c                | 86 +++++++++++++++++++++++
 7 files changed, 150 insertions(+)
 create mode 100644 arch/um/nommu/um/bootmem.c

diff --git a/arch/um/include/asm/mmu.h b/arch/um/include/asm/mmu.h
index 5b072aba5b65..c06d6cb67dd7 100644
--- a/arch/um/include/asm/mmu.h
+++ b/arch/um/include/asm/mmu.h
@@ -13,6 +13,9 @@ typedef struct mm_context {
 	struct mm_id id;
 	struct uml_arch_mm_context arch;
 	struct page *stub_pages[2];
+#ifndef CONFIG_MMU
+	unsigned long		end_brk;
+#endif
 } mm_context_t;
 
 extern void __switch_mm(struct mm_id * mm_idp);
diff --git a/arch/um/include/asm/mmu_context.h b/arch/um/include/asm/mmu_context.h
index 17ddd4edf875..4b06c29ae830 100644
--- a/arch/um/include/asm/mmu_context.h
+++ b/arch/um/include/asm/mmu_context.h
@@ -6,6 +6,7 @@
 #ifndef __UM_MMU_CONTEXT_H
 #define __UM_MMU_CONTEXT_H
 
+#ifdef CONFIG_MMU
 #include <linux/sched.h>
 #include <linux/mm_types.h>
 #include <linux/mmap_lock.h>
@@ -75,4 +76,11 @@ extern int init_new_context(struct task_struct *task, struct mm_struct *mm);
 
 extern void destroy_context(struct mm_struct *mm);
 
+#else
+#include <asm-generic/mmu_context.h>
+
+extern void force_flush_all(void);
+
+#endif /* CONFIG_MMU */
+
 #endif
diff --git a/arch/um/include/asm/page.h b/arch/um/include/asm/page.h
index 95af12e82a32..c9d5e487ac6b 100644
--- a/arch/um/include/asm/page.h
+++ b/arch/um/include/asm/page.h
@@ -7,6 +7,8 @@
 #ifndef __UM_PAGE_H
 #define __UM_PAGE_H
 
+#ifdef CONFIG_MMU
+
 #include <linux/const.h>
 
 /* PAGE_SHIFT determines the page size */
@@ -120,4 +122,17 @@ extern unsigned long uml_physmem;
 #define __HAVE_ARCH_GATE_AREA 1
 #endif
 
+#else  /* CONFIG_MMU */
+#define CONFIG_KERNEL_RAM_BASE_ADDRESS memory_start
+#include <asm-generic/page.h>
+
+#define __va_space (8*1024*1024)
+
+#ifndef __ASSEMBLY__
+#include <mem.h>
+void free_mem(void);
+#endif
+
+#endif /* !CONFIG_MMU  */
+
 #endif	/* __UM_PAGE_H */
diff --git a/arch/um/include/asm/pgtable.h b/arch/um/include/asm/pgtable.h
index def376194dce..7e506d5406e3 100644
--- a/arch/um/include/asm/pgtable.h
+++ b/arch/um/include/asm/pgtable.h
@@ -21,6 +21,8 @@
 #define _PAGE_PROTNONE	0x010	/* if the user mapped it with PROT_NONE;
 				   pte_present gives true */
 
+#ifdef CONFIG_MMU
+
 #ifdef CONFIG_3_LEVEL_PGTABLES
 #include <asm/pgtable-3level.h>
 #else
@@ -323,4 +325,29 @@ do {						\
 	__flush_tlb_one((vaddr));		\
 } while (0)
 
+#else  /* CONFIG_MMU */
+
+#include <asm-generic/pgtable-nopud.h>
+
+#define swapper_pg_dir		((pgd_t *)0)
+#define PAGE_KERNEL             __pgprot(0)
+#define PGDIR_SHIFT		21
+#define PGDIR_SIZE		(1UL << PGDIR_SHIFT)
+#define PGDIR_MASK		(~(PGDIR_SIZE-1))
+
+#define VMALLOC_START	0
+#define VMALLOC_END	0xffffffff
+
+/*
+ * ZERO_PAGE is a global shared page that is always zero: used
+ * for zero-mapped memory areas etc..
+ */
+extern unsigned long *empty_zero_page;
+#define ZERO_PAGE(vaddr)	(virt_to_page(empty_zero_page))
+
+extern unsigned long end_iomem;
+
+#endif /* !CONFIG_MMU */
+
+
 #endif
diff --git a/arch/um/include/asm/uaccess.h b/arch/um/include/asm/uaccess.h
index fe66d659acad..95db8f06d295 100644
--- a/arch/um/include/asm/uaccess.h
+++ b/arch/um/include/asm/uaccess.h
@@ -21,6 +21,7 @@
 #define __addr_range_nowrap(addr, size) \
 	((unsigned long) (addr) <= ((unsigned long) (addr) + (size)))
 
+#ifdef CONFIG_MMU
 extern unsigned long raw_copy_from_user(void *to, const void __user *from, unsigned long n);
 extern unsigned long raw_copy_to_user(void __user *to, const void *from, unsigned long n);
 extern long __strncpy_from_user(char *dst, const char __user *src, long count);
@@ -46,4 +47,9 @@ static inline int __access_ok(unsigned long addr, unsigned long size)
 		uaccess_kernel());
 }
 
+#else
+#include <asm-generic/uaccess.h>
+#endif /* CONFIG_MMU */
+
+
 #endif
diff --git a/arch/um/nommu/include/uapi/asm/host_ops.h b/arch/um/nommu/include/uapi/asm/host_ops.h
index d3dad11b459e..5253c3f8de0e 100644
--- a/arch/um/nommu/include/uapi/asm/host_ops.h
+++ b/arch/um/nommu/include/uapi/asm/host_ops.h
@@ -16,8 +16,13 @@ struct lkl_jmp_buf {
  * These operations must be provided by a host library or by the application
  * itself.
  *
+ * @mem_alloc - allocate memory
+ * @mem_free - free memory
+ *
  */
 struct lkl_host_operations {
+	void *(*mem_alloc)(unsigned long mem);
+	void (*mem_free)(void *mem);
 };
 
 void lkl_bug(const char *fmt, ...);
diff --git a/arch/um/nommu/um/bootmem.c b/arch/um/nommu/um/bootmem.c
new file mode 100644
index 000000000000..6ae4c26e2b74
--- /dev/null
+++ b/arch/um/nommu/um/bootmem.c
@@ -0,0 +1,86 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/memblock.h>
+#include <linux/mm.h>
+#include <linux/swap.h>
+
+unsigned long memory_start, memory_end;
+static unsigned long _memory_start, mem_size;
+
+unsigned long *empty_zero_page;
+
+/* XXX: unused */
+unsigned long long highmem;
+int iomem_size;
+int kmalloc_ok = 1;
+
+void __init setup_physmem(unsigned long start, unsigned long reserve_end,
+			  unsigned long mem_sz, unsigned long long _highmem)
+{
+	mem_size = mem_sz;
+
+	_memory_start = (unsigned long)lkl_ops->mem_alloc(mem_size);
+	memory_start = _memory_start;
+	WARN_ON(!memory_start);
+	memory_end = memory_start + mem_size;
+
+	if (PAGE_ALIGN(memory_start) != memory_start) {
+		mem_size -= PAGE_ALIGN(memory_start) - memory_start;
+		memory_start = PAGE_ALIGN(memory_start);
+		mem_size = (mem_size / PAGE_SIZE) * PAGE_SIZE;
+	}
+	pr_info("memblock address range: 0x%lx - 0x%lx\n", memory_start,
+		memory_start+mem_size);
+	/*
+	 * Give all the memory to the bootmap allocator, tell it to put the
+	 * boot mem_map at the start of memory.
+	 */
+	max_low_pfn = virt_to_pfn(memory_end);
+	min_low_pfn = virt_to_pfn(memory_start);
+	memblock_add(memory_start, mem_size);
+
+	empty_zero_page = memblock_alloc(PAGE_SIZE, PAGE_SIZE);
+	memset((void *)empty_zero_page, 0, PAGE_SIZE);
+
+	{
+		unsigned long zones_size[MAX_NR_ZONES] = {0, };
+
+		zones_size[ZONE_NORMAL] = max_low_pfn;
+		free_area_init(zones_size);
+	}
+}
+
+void __init mem_init(void)
+{
+	max_mapnr = (((unsigned long)high_memory) - PAGE_OFFSET) >> PAGE_SHIFT;
+	/* this will put all memory onto the freelists */
+	totalram_pages_add(memblock_free_all());
+	pr_info("Memory available: %luk/%luk RAM\n",
+		(nr_free_pages() << PAGE_SHIFT) >> 10, mem_size >> 10);
+}
+
+/*
+ * In our case __init memory is not part of the page allocator so there is
+ * nothing to free.
+ */
+void free_initmem(void)
+{
+}
+
+void free_mem(void)
+{
+	lkl_ops->mem_free((void *)_memory_start);
+}
+
+void *uml_kmalloc(int size, int flags)
+{
+	return kmalloc(size, flags);
+}
+
+void __init mem_total_pages(unsigned long physmem, unsigned long iomem,
+		     unsigned long _highmem)
+{
+}
+
+void __init paging_init(void)
+{
+}
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v6 10/21] um: nommu: memory handling
@ 2020-09-24  7:12         ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  7:12 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Hajime Tazaki, retrage01

nommu mode follows the way !CONFIG_MMU architecture of other linux
archs.  Ther is not much work left to do other than initializing the
boot allocator and providing the page and page table definitions.

The backstore memory is allocated via a host operation and the memory
size to be used is specified when the kernel is started, in the
lkl_start_kernel call.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/include/asm/mmu.h                 |  3 +
 arch/um/include/asm/mmu_context.h         |  8 +++
 arch/um/include/asm/page.h                | 15 ++++
 arch/um/include/asm/pgtable.h             | 27 +++++++
 arch/um/include/asm/uaccess.h             |  6 ++
 arch/um/nommu/include/uapi/asm/host_ops.h |  5 ++
 arch/um/nommu/um/bootmem.c                | 86 +++++++++++++++++++++++
 7 files changed, 150 insertions(+)
 create mode 100644 arch/um/nommu/um/bootmem.c

diff --git a/arch/um/include/asm/mmu.h b/arch/um/include/asm/mmu.h
index 5b072aba5b65..c06d6cb67dd7 100644
--- a/arch/um/include/asm/mmu.h
+++ b/arch/um/include/asm/mmu.h
@@ -13,6 +13,9 @@ typedef struct mm_context {
 	struct mm_id id;
 	struct uml_arch_mm_context arch;
 	struct page *stub_pages[2];
+#ifndef CONFIG_MMU
+	unsigned long		end_brk;
+#endif
 } mm_context_t;
 
 extern void __switch_mm(struct mm_id * mm_idp);
diff --git a/arch/um/include/asm/mmu_context.h b/arch/um/include/asm/mmu_context.h
index 17ddd4edf875..4b06c29ae830 100644
--- a/arch/um/include/asm/mmu_context.h
+++ b/arch/um/include/asm/mmu_context.h
@@ -6,6 +6,7 @@
 #ifndef __UM_MMU_CONTEXT_H
 #define __UM_MMU_CONTEXT_H
 
+#ifdef CONFIG_MMU
 #include <linux/sched.h>
 #include <linux/mm_types.h>
 #include <linux/mmap_lock.h>
@@ -75,4 +76,11 @@ extern int init_new_context(struct task_struct *task, struct mm_struct *mm);
 
 extern void destroy_context(struct mm_struct *mm);
 
+#else
+#include <asm-generic/mmu_context.h>
+
+extern void force_flush_all(void);
+
+#endif /* CONFIG_MMU */
+
 #endif
diff --git a/arch/um/include/asm/page.h b/arch/um/include/asm/page.h
index 95af12e82a32..c9d5e487ac6b 100644
--- a/arch/um/include/asm/page.h
+++ b/arch/um/include/asm/page.h
@@ -7,6 +7,8 @@
 #ifndef __UM_PAGE_H
 #define __UM_PAGE_H
 
+#ifdef CONFIG_MMU
+
 #include <linux/const.h>
 
 /* PAGE_SHIFT determines the page size */
@@ -120,4 +122,17 @@ extern unsigned long uml_physmem;
 #define __HAVE_ARCH_GATE_AREA 1
 #endif
 
+#else  /* CONFIG_MMU */
+#define CONFIG_KERNEL_RAM_BASE_ADDRESS memory_start
+#include <asm-generic/page.h>
+
+#define __va_space (8*1024*1024)
+
+#ifndef __ASSEMBLY__
+#include <mem.h>
+void free_mem(void);
+#endif
+
+#endif /* !CONFIG_MMU  */
+
 #endif	/* __UM_PAGE_H */
diff --git a/arch/um/include/asm/pgtable.h b/arch/um/include/asm/pgtable.h
index def376194dce..7e506d5406e3 100644
--- a/arch/um/include/asm/pgtable.h
+++ b/arch/um/include/asm/pgtable.h
@@ -21,6 +21,8 @@
 #define _PAGE_PROTNONE	0x010	/* if the user mapped it with PROT_NONE;
 				   pte_present gives true */
 
+#ifdef CONFIG_MMU
+
 #ifdef CONFIG_3_LEVEL_PGTABLES
 #include <asm/pgtable-3level.h>
 #else
@@ -323,4 +325,29 @@ do {						\
 	__flush_tlb_one((vaddr));		\
 } while (0)
 
+#else  /* CONFIG_MMU */
+
+#include <asm-generic/pgtable-nopud.h>
+
+#define swapper_pg_dir		((pgd_t *)0)
+#define PAGE_KERNEL             __pgprot(0)
+#define PGDIR_SHIFT		21
+#define PGDIR_SIZE		(1UL << PGDIR_SHIFT)
+#define PGDIR_MASK		(~(PGDIR_SIZE-1))
+
+#define VMALLOC_START	0
+#define VMALLOC_END	0xffffffff
+
+/*
+ * ZERO_PAGE is a global shared page that is always zero: used
+ * for zero-mapped memory areas etc..
+ */
+extern unsigned long *empty_zero_page;
+#define ZERO_PAGE(vaddr)	(virt_to_page(empty_zero_page))
+
+extern unsigned long end_iomem;
+
+#endif /* !CONFIG_MMU */
+
+
 #endif
diff --git a/arch/um/include/asm/uaccess.h b/arch/um/include/asm/uaccess.h
index fe66d659acad..95db8f06d295 100644
--- a/arch/um/include/asm/uaccess.h
+++ b/arch/um/include/asm/uaccess.h
@@ -21,6 +21,7 @@
 #define __addr_range_nowrap(addr, size) \
 	((unsigned long) (addr) <= ((unsigned long) (addr) + (size)))
 
+#ifdef CONFIG_MMU
 extern unsigned long raw_copy_from_user(void *to, const void __user *from, unsigned long n);
 extern unsigned long raw_copy_to_user(void __user *to, const void *from, unsigned long n);
 extern long __strncpy_from_user(char *dst, const char __user *src, long count);
@@ -46,4 +47,9 @@ static inline int __access_ok(unsigned long addr, unsigned long size)
 		uaccess_kernel());
 }
 
+#else
+#include <asm-generic/uaccess.h>
+#endif /* CONFIG_MMU */
+
+
 #endif
diff --git a/arch/um/nommu/include/uapi/asm/host_ops.h b/arch/um/nommu/include/uapi/asm/host_ops.h
index d3dad11b459e..5253c3f8de0e 100644
--- a/arch/um/nommu/include/uapi/asm/host_ops.h
+++ b/arch/um/nommu/include/uapi/asm/host_ops.h
@@ -16,8 +16,13 @@ struct lkl_jmp_buf {
  * These operations must be provided by a host library or by the application
  * itself.
  *
+ * @mem_alloc - allocate memory
+ * @mem_free - free memory
+ *
  */
 struct lkl_host_operations {
+	void *(*mem_alloc)(unsigned long mem);
+	void (*mem_free)(void *mem);
 };
 
 void lkl_bug(const char *fmt, ...);
diff --git a/arch/um/nommu/um/bootmem.c b/arch/um/nommu/um/bootmem.c
new file mode 100644
index 000000000000..6ae4c26e2b74
--- /dev/null
+++ b/arch/um/nommu/um/bootmem.c
@@ -0,0 +1,86 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/memblock.h>
+#include <linux/mm.h>
+#include <linux/swap.h>
+
+unsigned long memory_start, memory_end;
+static unsigned long _memory_start, mem_size;
+
+unsigned long *empty_zero_page;
+
+/* XXX: unused */
+unsigned long long highmem;
+int iomem_size;
+int kmalloc_ok = 1;
+
+void __init setup_physmem(unsigned long start, unsigned long reserve_end,
+			  unsigned long mem_sz, unsigned long long _highmem)
+{
+	mem_size = mem_sz;
+
+	_memory_start = (unsigned long)lkl_ops->mem_alloc(mem_size);
+	memory_start = _memory_start;
+	WARN_ON(!memory_start);
+	memory_end = memory_start + mem_size;
+
+	if (PAGE_ALIGN(memory_start) != memory_start) {
+		mem_size -= PAGE_ALIGN(memory_start) - memory_start;
+		memory_start = PAGE_ALIGN(memory_start);
+		mem_size = (mem_size / PAGE_SIZE) * PAGE_SIZE;
+	}
+	pr_info("memblock address range: 0x%lx - 0x%lx\n", memory_start,
+		memory_start+mem_size);
+	/*
+	 * Give all the memory to the bootmap allocator, tell it to put the
+	 * boot mem_map at the start of memory.
+	 */
+	max_low_pfn = virt_to_pfn(memory_end);
+	min_low_pfn = virt_to_pfn(memory_start);
+	memblock_add(memory_start, mem_size);
+
+	empty_zero_page = memblock_alloc(PAGE_SIZE, PAGE_SIZE);
+	memset((void *)empty_zero_page, 0, PAGE_SIZE);
+
+	{
+		unsigned long zones_size[MAX_NR_ZONES] = {0, };
+
+		zones_size[ZONE_NORMAL] = max_low_pfn;
+		free_area_init(zones_size);
+	}
+}
+
+void __init mem_init(void)
+{
+	max_mapnr = (((unsigned long)high_memory) - PAGE_OFFSET) >> PAGE_SHIFT;
+	/* this will put all memory onto the freelists */
+	totalram_pages_add(memblock_free_all());
+	pr_info("Memory available: %luk/%luk RAM\n",
+		(nr_free_pages() << PAGE_SHIFT) >> 10, mem_size >> 10);
+}
+
+/*
+ * In our case __init memory is not part of the page allocator so there is
+ * nothing to free.
+ */
+void free_initmem(void)
+{
+}
+
+void free_mem(void)
+{
+	lkl_ops->mem_free((void *)_memory_start);
+}
+
+void *uml_kmalloc(int size, int flags)
+{
+	return kmalloc(size, flags);
+}
+
+void __init mem_total_pages(unsigned long physmem, unsigned long iomem,
+		     unsigned long _highmem)
+{
+}
+
+void __init paging_init(void)
+{
+}
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v6 11/21] um: nommu: kernel thread support
  2020-09-24  7:12       ` Hajime Tazaki
@ 2020-09-24  7:12         ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  7:12 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch, Hajime Tazaki

nommu mode does not support user processes but it must support kernel
threads as part as the normal kernel work-flow. It uses host operations
to create and terminate host threads that are going to run the kernel
threads. It also uses semaphores to synchronize those threads and to
allow the Linux kernel scheduler to control how the kernel threads
run.

Each kernel thread runs in a host threads and has a host semaphore
associated with it - the thread's scheduling semaphore. The semaphore
counter is initialized to 0. The first thing a kernel thread does
after getting spawned, before running any kernel code, is to perform a
down operation to block the thread.

The kernel controls host threads scheduling by performing up and down
operations on the scheduling semaphore. In __switch_context an up
operation on the next thread is performed to wake up a blocked thread,
and a down operation is performed on the prev thread to block it.

A thread is terminated by marking it in free_thread_info and
performing an up operation on the scheduling semaphore at which point
the marked thread will terminate itself.

UML common part (process.c) is extended to call idle functions of
SUBARCH as well as kernel thread detection with a flag information of
TIF_HOST_THREAD.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/include/asm/thread_info.h         |  24 ++
 arch/um/kernel/process.c                  |   8 +-
 arch/um/nommu/include/uapi/asm/host_ops.h |  54 +++++
 arch/um/nommu/um/cpu.c                    | 236 ++++++++++++++++++++
 arch/um/nommu/um/threads.c                | 258 ++++++++++++++++++++++
 5 files changed, 579 insertions(+), 1 deletion(-)
 create mode 100644 arch/um/nommu/um/cpu.c
 create mode 100644 arch/um/nommu/um/threads.c

diff --git a/arch/um/include/asm/thread_info.h b/arch/um/include/asm/thread_info.h
index 4c19ce4c49f1..433584f6b9d0 100644
--- a/arch/um/include/asm/thread_info.h
+++ b/arch/um/include/asm/thread_info.h
@@ -40,6 +40,7 @@ struct thread_info {
 	.real_thread = NULL,			\
 }
 
+#ifdef CONFIG_MMU
 /* how to get the thread information struct from C */
 static inline struct thread_info *current_thread_info(void)
 {
@@ -52,6 +53,27 @@ static inline struct thread_info *current_thread_info(void)
 	return ti;
 }
 
+#else
+
+#define __HAVE_THREAD_FUNCTIONS
+#define task_thread_info(task)	((struct thread_info *)(task)->stack)
+#define task_stack_page(task)	((task)->stack)
+#define end_of_stack(p) (&task_thread_info(p)->aux_fp_regs[FP_SIZE-1])
+
+void threads_init(void);
+void threads_cleanup(void);
+
+unsigned long *alloc_thread_stack_node(struct task_struct *p, int node);
+void setup_thread_stack(struct task_struct *p, struct task_struct *org);
+void free_thread_stack(struct task_struct *tsk);
+
+extern struct thread_info *_current_thread_info;
+static inline struct thread_info *current_thread_info(void)
+{
+	return _current_thread_info;
+}
+#endif /* CONFIG_MMU */
+
 #endif
 
 #define TIF_SYSCALL_TRACE	0	/* syscall trace active */
@@ -63,6 +85,8 @@ static inline struct thread_info *current_thread_info(void)
 #define TIF_RESTORE_SIGMASK	7
 #define TIF_NOTIFY_RESUME	8
 #define TIF_SECCOMP		9	/* secure computing */
+#define TIF_SCHED_JB		10
+#define TIF_HOST_THREAD		11
 
 #define _TIF_SYSCALL_TRACE	(1 << TIF_SYSCALL_TRACE)
 #define _TIF_SIGPENDING		(1 << TIF_SIGPENDING)
diff --git a/arch/um/kernel/process.c b/arch/um/kernel/process.c
index 87a8cfa228ca..432edcbb5eea 100644
--- a/arch/um/kernel/process.c
+++ b/arch/um/kernel/process.c
@@ -156,7 +156,8 @@ int copy_thread(unsigned long clone_flags, unsigned long sp,
 		unsigned long arg, struct task_struct * p, unsigned long tls)
 {
 	void (*handler)(void);
-	int kthread = current->flags & PF_KTHREAD;
+	int kthread = current->flags & PF_KTHREAD ||
+		test_ti_thread_flag(current_thread_info(), TIF_HOST_THREAD);
 	int ret = 0;
 
 	p->thread = (struct thread_struct) INIT_THREAD;
@@ -213,10 +214,15 @@ static void um_idle_sleep(void)
 	}
 }
 
+void __weak subarch_cpu_idle(void)
+{
+}
+
 void arch_cpu_idle(void)
 {
 	cpu_tasks[current_thread_info()->cpu].pid = os_getpid();
 	um_idle_sleep();
+	subarch_cpu_idle();
 	local_irq_enable();
 }
 
diff --git a/arch/um/nommu/include/uapi/asm/host_ops.h b/arch/um/nommu/include/uapi/asm/host_ops.h
index 5253c3f8de0e..720385fccbdf 100644
--- a/arch/um/nommu/include/uapi/asm/host_ops.h
+++ b/arch/um/nommu/include/uapi/asm/host_ops.h
@@ -16,11 +16,65 @@ struct lkl_jmp_buf {
  * These operations must be provided by a host library or by the application
  * itself.
  *
+ * @sem_alloc - allocate a host semaphore an initialize it to count
+ * @sem_free - free a host semaphore
+ * @sem_up - perform an up operation on the semaphore
+ * @sem_down - perform a down operation on the semaphore
+ *
+ * @mutex_alloc - allocate and initialize a host mutex; the recursive parameter
+ * determines if the mutex is recursive or not
+ * @mutex_free - free a host mutex
+ * @mutex_lock - acquire the mutex
+ * @mutex_unlock - release the mutex
+ *
+ * @thread_create - create a new thread and run f(arg) in its context; returns a
+ * thread handle or 0 if the thread could not be created
+ * @thread_detach - on POSIX systems, free up resources held by
+ * pthreads. Noop on Win32.
+ * @thread_exit - terminates the current thread
+ * @thread_join - wait for the given thread to terminate. Returns 0
+ * for success, -1 otherwise
+ *
+ * @gettid - returns the host thread id of the caller, which need not
+ * be the same as the handle returned by thread_create
+ *
+ * @jmp_buf_set - runs the give function and setups a jump back point by saving
+ * the context in the jump buffer; jmp_buf_longjmp can be called from the give
+ * function or any callee in that function to return back to the jump back
+ * point
+ *
+ * NOTE: we can't return from jmp_buf_set before calling jmp_buf_longjmp or
+ * otherwise the saved context (stack) is not going to be valid, so we must pass
+ * the function that will eventually call longjmp here
+ *
+ * @jmp_buf_longjmp - perform a jump back to the saved jump buffer
+ *
  * @mem_alloc - allocate memory
  * @mem_free - free memory
  *
  */
 struct lkl_host_operations {
+	struct lkl_sem *(*sem_alloc)(int count);
+	void (*sem_free)(struct lkl_sem *sem);
+	void (*sem_up)(struct lkl_sem *sem);
+	void (*sem_down)(struct lkl_sem *sem);
+
+	struct lkl_mutex *(*mutex_alloc)(int recursive);
+	void (*mutex_free)(struct lkl_mutex *mutex);
+	void (*mutex_lock)(struct lkl_mutex *mutex);
+	void (*mutex_unlock)(struct lkl_mutex *mutex);
+
+	lkl_thread_t (*thread_create)(void *(*f)(void *), void *arg);
+	void (*thread_detach)(void);
+	void (*thread_exit)(void);
+	int (*thread_join)(lkl_thread_t tid);
+	lkl_thread_t (*thread_self)(void);
+	int (*thread_equal)(lkl_thread_t a, lkl_thread_t b);
+	long (*gettid)(void);
+
+	void (*jmp_buf_set)(struct lkl_jmp_buf *jmpb, void (*f)(void));
+	void (*jmp_buf_longjmp)(struct lkl_jmp_buf *jmpb, int val);
+
 	void *(*mem_alloc)(unsigned long mem);
 	void (*mem_free)(void *mem);
 };
diff --git a/arch/um/nommu/um/cpu.c b/arch/um/nommu/um/cpu.c
new file mode 100644
index 000000000000..9986a3f8c5dd
--- /dev/null
+++ b/arch/um/nommu/um/cpu.c
@@ -0,0 +1,236 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/kernel.h>
+#include <linux/sched/stat.h>
+#include <asm/host_ops.h>
+#include <asm/cpu.h>
+#include <asm/thread_info.h>
+#include <asm/unistd.h>
+#include <asm/sched.h>
+#include <asm/syscalls.h>
+#include <init.h>
+
+void run_irqs(void)
+{
+	panic("unimplemented %s", __func__);
+}
+
+int set_irq_pending(int irq)
+{
+	panic("unimplemented %s", __func__);
+}
+
+/*
+ * This structure is used to get access to the "LKL CPU" that allows us to run
+ * Linux code. Because we have to deal with various synchronization requirements
+ * between idle thread, system calls, interrupts, "reentrancy", CPU shutdown,
+ * imbalance wake up (i.e. acquire the CPU from one thread and release it from
+ * another), we can't use a simple synchronization mechanism such as (recursive)
+ * mutex or semaphore. Instead, we use a mutex and a bunch of status data plus a
+ * semaphore.
+ */
+static struct lkl_cpu {
+	/* lock that protects the CPU status data */
+	struct lkl_mutex *lock;
+	/*
+	 * Since we must free the cpu lock during shutdown we need a
+	 * synchronization algorithm between lkl_cpu_shutdown() and the CPU
+	 * access functions since lkl_cpu_get() gets called from thread
+	 * destructor callback functions which may be scheduled after
+	 * lkl_cpu_shutdown() has freed the cpu lock.
+	 *
+	 * An atomic counter is used to keep track of the number of running
+	 * CPU access functions and allow the shutdown function to wait for
+	 * them.
+	 *
+	 * The shutdown functions adds MAX_THREADS to this counter which allows
+	 * the CPU access functions to check if the shutdown process has
+	 * started.
+	 *
+	 * This algorithm assumes that we never have more the MAX_THREADS
+	 * requesting CPU access.
+	 */
+	#define MAX_THREADS 1000000
+	unsigned int shutdown_gate;
+	bool irqs_pending;
+	/* no of threads waiting the CPU */
+	unsigned int sleepers;
+	/* no of times the current thread got the CPU */
+	unsigned int count;
+	/* current thread that owns the CPU */
+	lkl_thread_t owner;
+	/* semaphore for threads waiting the CPU */
+	struct lkl_sem *sem;
+	/* semaphore used for shutdown */
+	struct lkl_sem *shutdown_sem;
+} cpu;
+
+static int __cpu_try_get_lock(int n)
+{
+	lkl_thread_t self;
+
+	if (__sync_fetch_and_add(&cpu.shutdown_gate, n) >= MAX_THREADS)
+		return -2;
+
+	lkl_ops->mutex_lock(cpu.lock);
+
+	if (cpu.shutdown_gate >= MAX_THREADS)
+		return -1;
+
+	self = lkl_ops->thread_self();
+
+	if (cpu.owner && !lkl_ops->thread_equal(cpu.owner, self))
+		return 0;
+
+	cpu.owner = self;
+	cpu.count++;
+
+	return 1;
+}
+
+static void __cpu_try_get_unlock(int lock_ret, int n)
+{
+	if (lock_ret >= -1)
+		lkl_ops->mutex_unlock(cpu.lock);
+	__sync_fetch_and_sub(&cpu.shutdown_gate, n);
+}
+
+void lkl_cpu_change_owner(lkl_thread_t owner)
+{
+	lkl_ops->mutex_lock(cpu.lock);
+	if (cpu.count > 1)
+		lkl_bug("bad count while changing owner\n");
+	cpu.owner = owner;
+	lkl_ops->mutex_unlock(cpu.lock);
+}
+
+int lkl_cpu_get(void)
+{
+	int ret;
+
+	ret = __cpu_try_get_lock(1);
+
+	while (ret == 0) {
+		cpu.sleepers++;
+		__cpu_try_get_unlock(ret, 0);
+		lkl_ops->sem_down(cpu.sem);
+		ret = __cpu_try_get_lock(0);
+	}
+
+	__cpu_try_get_unlock(ret, 1);
+
+	return ret;
+}
+
+void lkl_cpu_put(void)
+{
+	lkl_ops->mutex_lock(cpu.lock);
+
+	if (!cpu.count || !cpu.owner ||
+	    !lkl_ops->thread_equal(cpu.owner, lkl_ops->thread_self()))
+		lkl_bug("%s: unbalanced put\n", __func__);
+
+	while (cpu.irqs_pending && !irqs_disabled()) {
+		cpu.irqs_pending = false;
+		lkl_ops->mutex_unlock(cpu.lock);
+		run_irqs();
+		lkl_ops->mutex_lock(cpu.lock);
+	}
+
+	if (test_ti_thread_flag(current_thread_info(), TIF_HOST_THREAD) &&
+	    !single_task_running() && cpu.count == 1) {
+		if (in_interrupt())
+			lkl_bug("%s: in interrupt\n", __func__);
+		lkl_ops->mutex_unlock(cpu.lock);
+		thread_sched_jb();
+		return;
+	}
+
+	if (--cpu.count > 0) {
+		lkl_ops->mutex_unlock(cpu.lock);
+		return;
+	}
+
+	if (cpu.sleepers) {
+		cpu.sleepers--;
+		lkl_ops->sem_up(cpu.sem);
+	}
+
+	cpu.owner = 0;
+
+	lkl_ops->mutex_unlock(cpu.lock);
+}
+
+int lkl_cpu_try_run_irq(int irq)
+{
+	int ret;
+
+	ret = __cpu_try_get_lock(1);
+	if (!ret) {
+		set_irq_pending(irq);
+		cpu.irqs_pending = true;
+	}
+	__cpu_try_get_unlock(ret, 1);
+
+	return ret;
+}
+
+static void lkl_cpu_shutdown(void)
+{
+	__sync_fetch_and_add(&cpu.shutdown_gate, MAX_THREADS);
+}
+__uml_exitcall(lkl_cpu_shutdown);
+
+void lkl_cpu_wait_shutdown(void)
+{
+	lkl_ops->sem_down(cpu.shutdown_sem);
+	lkl_ops->sem_free(cpu.shutdown_sem);
+}
+
+static void lkl_cpu_cleanup(bool shutdown)
+{
+	while (__sync_fetch_and_add(&cpu.shutdown_gate, 0) > MAX_THREADS)
+		;
+
+	if (shutdown)
+		lkl_ops->sem_up(cpu.shutdown_sem);
+	else if (cpu.shutdown_sem)
+		lkl_ops->sem_free(cpu.shutdown_sem);
+	if (cpu.sem)
+		lkl_ops->sem_free(cpu.sem);
+	if (cpu.lock)
+		lkl_ops->mutex_free(cpu.lock);
+}
+
+void subarch_cpu_idle(void)
+{
+	if (cpu.shutdown_gate >= MAX_THREADS) {
+
+		lkl_ops->mutex_lock(cpu.lock);
+		while (cpu.sleepers--)
+			lkl_ops->sem_up(cpu.sem);
+		lkl_ops->mutex_unlock(cpu.lock);
+
+		lkl_cpu_cleanup(true);
+
+		lkl_ops->thread_exit();
+	}
+
+#ifdef doesntwork
+	/* switch to idle_host_task */
+	wakeup_idle_host_task();
+#endif
+}
+
+int lkl_cpu_init(void)
+{
+	cpu.lock = lkl_ops->mutex_alloc(0);
+	cpu.sem = lkl_ops->sem_alloc(0);
+	cpu.shutdown_sem = lkl_ops->sem_alloc(0);
+
+	if (!cpu.lock || !cpu.sem || !cpu.shutdown_sem) {
+		lkl_cpu_cleanup(false);
+		return -ENOMEM;
+	}
+
+	return 0;
+}
diff --git a/arch/um/nommu/um/threads.c b/arch/um/nommu/um/threads.c
new file mode 100644
index 000000000000..3e70eccc191a
--- /dev/null
+++ b/arch/um/nommu/um/threads.c
@@ -0,0 +1,258 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/slab.h>
+#include <linux/sched/task.h>
+#include <linux/sched/signal.h>
+#include <asm/host_ops.h>
+#include <asm/cpu.h>
+#include <asm/sched.h>
+
+#include <os.h>
+
+static int init_arch_thread(struct arch_thread *thread)
+{
+	thread->sched_sem = lkl_ops->sem_alloc(0);
+	if (!thread->sched_sem)
+		return -ENOMEM;
+
+	thread->dead = false;
+	thread->tid = 0;
+
+	return 0;
+}
+
+unsigned long *alloc_thread_stack_node(struct task_struct *task, int node)
+{
+	struct thread_info *ti;
+
+	ti = kmalloc(sizeof(*ti), GFP_KERNEL | __GFP_ZERO);
+	if (!ti)
+		return NULL;
+
+	ti->task = task;
+	return (unsigned long *)ti;
+}
+
+/*
+ * The only new tasks created are kernel threads that have a predefined starting
+ * point thus no stack copy is required.
+ */
+void setup_thread_stack(struct task_struct *p, struct task_struct *org)
+{
+	struct thread_info *ti = task_thread_info(p);
+	struct thread_info *org_ti = task_thread_info(org);
+
+	ti->flags = org_ti->flags;
+	ti->preempt_count = org_ti->preempt_count;
+	ti->addr_limit = org_ti->addr_limit;
+}
+
+static void kill_thread(struct thread_info *ti)
+{
+	if (!test_ti_thread_flag(ti, TIF_HOST_THREAD)) {
+		ti->task->thread.arch.dead = true;
+		lkl_ops->sem_up(ti->task->thread.arch.sched_sem);
+		lkl_ops->thread_join(ti->task->thread.arch.tid);
+	}
+	lkl_ops->sem_free(ti->task->thread.arch.sched_sem);
+}
+
+void free_thread_stack(struct task_struct *tsk)
+{
+	struct thread_info *ti = task_thread_info(tsk);
+
+	kill_thread(ti);
+	kfree(ti);
+}
+
+struct thread_info *_current_thread_info = &init_thread_union.thread_info;
+
+void switch_threads(jmp_buf *me, jmp_buf *you)
+{
+	/* NOP */
+}
+
+/*
+ * schedule() expects the return of this function to be the task that we
+ * switched away from. Returning prev is not going to work because we are
+ * actually going to return the previous taks that was scheduled before the
+ * task we are going to wake up, and not the current task, e.g.:
+ *
+ * swapper -> init: saved prev on swapper stack is swapper
+ * init -> ksoftirqd0: saved prev on init stack is init
+ * ksoftirqd0 -> swapper: returned prev is swapper
+ */
+static struct task_struct *abs_prev = &init_task;
+
+void arch_switch_to(struct task_struct *prev,
+		    struct task_struct *next)
+{
+	struct arch_thread *_prev = &prev->thread.arch;
+	struct arch_thread *_next = &next->thread.arch;
+	unsigned long _prev_flags = task_thread_info(prev)->flags;
+	struct lkl_jmp_buf *_prev_jb;
+
+	_current_thread_info = task_thread_info(next);
+	next->thread.prev_sched = prev;
+	abs_prev = prev;
+
+	WARN_ON(!_next->tid);
+	lkl_cpu_change_owner(_next->tid);
+
+	if (test_bit(TIF_SCHED_JB, &_prev_flags)) {
+		/* Atomic. Must be done before wakeup next */
+		clear_ti_thread_flag(task_thread_info(prev), TIF_SCHED_JB);
+		_prev_jb = &_prev->sched_jb;
+	}
+
+	lkl_ops->sem_up(_next->sched_sem);
+	if (test_bit(TIF_SCHED_JB, &_prev_flags))
+		lkl_ops->jmp_buf_longjmp(_prev_jb, 1);
+	else
+		lkl_ops->sem_down(_prev->sched_sem);
+
+	if (_prev->dead)
+		lkl_ops->thread_exit();
+
+	/* __switch_to (arch/um) returns this value */
+	current->thread.prev_sched = abs_prev;
+}
+
+int host_task_stub(void *unused)
+{
+	return 0;
+}
+
+void switch_to_host_task(struct task_struct *task)
+{
+	if (WARN_ON(!test_tsk_thread_flag(task, TIF_HOST_THREAD)))
+		return;
+
+	task->thread.arch.tid = lkl_ops->thread_self();
+
+	if (current == task)
+		return;
+
+	wake_up_process(task);
+	thread_sched_jb();
+	lkl_ops->sem_down(task->thread.arch.sched_sem);
+	schedule_tail(abs_prev);
+}
+
+struct thread_bootstrap_arg {
+	struct thread_info *ti;
+	int (*f)(void *a);
+	void *arg;
+};
+
+static void *thread_bootstrap(void *_tba)
+{
+	struct thread_bootstrap_arg *tba = (struct thread_bootstrap_arg *)_tba;
+	struct thread_info *ti = tba->ti;
+	int (*f)(void *) = tba->f;
+	void *arg = tba->arg;
+
+	lkl_ops->sem_down(ti->task->thread.arch.sched_sem);
+	kfree(tba);
+	if (ti->task->thread.prev_sched)
+		schedule_tail(ti->task->thread.prev_sched);
+
+	f(arg);
+	do_exit(0);
+
+	return NULL;
+}
+
+void new_thread(void *stack, jmp_buf *buf, void (*handler)(void))
+{
+	struct thread_info *ti = (struct thread_info *)stack;
+	struct task_struct *p = ti->task;
+	struct thread_bootstrap_arg *tba;
+	int ret;
+
+	unsigned long esp = (unsigned long)p->thread.request.u.thread.proc;
+	unsigned long unused = (unsigned long)p->thread.request.u.thread.arg;
+
+	ret = init_arch_thread(&p->thread.arch);
+	if (ret < 0)
+		panic("%s: init_arch_thread", __func__);
+
+	if ((int (*)(void *))esp == host_task_stub) {
+		set_ti_thread_flag(ti, TIF_HOST_THREAD);
+		return;
+	}
+
+	tba = kmalloc(sizeof(*tba), GFP_KERNEL);
+	if (!tba)
+		return;
+
+	tba->f = (int (*)(void *))esp;
+	tba->arg = (void *)unused;
+	tba->ti = ti;
+
+	p->thread.arch.tid = lkl_ops->thread_create(thread_bootstrap, tba);
+	if (!p->thread.arch.tid) {
+		kfree(tba);
+		return;
+	}
+}
+
+void show_stack(struct task_struct *task, unsigned long *esp)
+{
+}
+
+static inline void pr_early(const char *str)
+{
+	if (lkl_ops->print)
+		lkl_ops->print(str, strlen(str));
+}
+
+/**
+ * This is called before the kernel initializes, so no kernel calls (including
+ * printk) can't be made yet.
+ */
+void threads_init(void)
+{
+	int ret;
+	struct thread_info *ti = &init_thread_union.thread_info;
+
+	ti->task->thread = (struct thread_struct) INIT_THREAD;
+	ret = init_arch_thread(&ti->task->thread.arch);
+	if (ret < 0)
+		pr_early("lkl: failed to allocate init schedule semaphore\n");
+
+	ti->task->thread.arch.tid = lkl_ops->thread_self();
+}
+
+void threads_cleanup(void)
+{
+	struct task_struct *p, *t;
+
+	for_each_process_thread(p, t) {
+		struct thread_info *ti = task_thread_info(t);
+
+		if (t->pid != 1 && !test_ti_thread_flag(ti, TIF_HOST_THREAD))
+			WARN(!(t->flags & PF_KTHREAD),
+			     "non kernel thread task %s\n", t->comm);
+		WARN(t->state == TASK_RUNNING,
+		     "thread %s still running while halting\n", t->comm);
+
+		kill_thread(ti);
+	}
+
+	lkl_ops->sem_free(
+		init_thread_union.thread_info.task->thread.arch.sched_sem);
+}
+
+void initial_thread_cb_skas(void (*proc)(void *), void *arg)
+{
+	pr_warn("unimplemented %s", __func__);
+}
+
+int arch_set_tls(struct task_struct *new, unsigned long tls)
+{
+	panic("unimplemented %s", __func__);
+}
+void clear_flushed_tls(struct task_struct *task)
+{
+	panic("unimplemented %s", __func__);
+}
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v6 11/21] um: nommu: kernel thread support
@ 2020-09-24  7:12         ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  7:12 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Hajime Tazaki, retrage01

nommu mode does not support user processes but it must support kernel
threads as part as the normal kernel work-flow. It uses host operations
to create and terminate host threads that are going to run the kernel
threads. It also uses semaphores to synchronize those threads and to
allow the Linux kernel scheduler to control how the kernel threads
run.

Each kernel thread runs in a host threads and has a host semaphore
associated with it - the thread's scheduling semaphore. The semaphore
counter is initialized to 0. The first thing a kernel thread does
after getting spawned, before running any kernel code, is to perform a
down operation to block the thread.

The kernel controls host threads scheduling by performing up and down
operations on the scheduling semaphore. In __switch_context an up
operation on the next thread is performed to wake up a blocked thread,
and a down operation is performed on the prev thread to block it.

A thread is terminated by marking it in free_thread_info and
performing an up operation on the scheduling semaphore at which point
the marked thread will terminate itself.

UML common part (process.c) is extended to call idle functions of
SUBARCH as well as kernel thread detection with a flag information of
TIF_HOST_THREAD.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/include/asm/thread_info.h         |  24 ++
 arch/um/kernel/process.c                  |   8 +-
 arch/um/nommu/include/uapi/asm/host_ops.h |  54 +++++
 arch/um/nommu/um/cpu.c                    | 236 ++++++++++++++++++++
 arch/um/nommu/um/threads.c                | 258 ++++++++++++++++++++++
 5 files changed, 579 insertions(+), 1 deletion(-)
 create mode 100644 arch/um/nommu/um/cpu.c
 create mode 100644 arch/um/nommu/um/threads.c

diff --git a/arch/um/include/asm/thread_info.h b/arch/um/include/asm/thread_info.h
index 4c19ce4c49f1..433584f6b9d0 100644
--- a/arch/um/include/asm/thread_info.h
+++ b/arch/um/include/asm/thread_info.h
@@ -40,6 +40,7 @@ struct thread_info {
 	.real_thread = NULL,			\
 }
 
+#ifdef CONFIG_MMU
 /* how to get the thread information struct from C */
 static inline struct thread_info *current_thread_info(void)
 {
@@ -52,6 +53,27 @@ static inline struct thread_info *current_thread_info(void)
 	return ti;
 }
 
+#else
+
+#define __HAVE_THREAD_FUNCTIONS
+#define task_thread_info(task)	((struct thread_info *)(task)->stack)
+#define task_stack_page(task)	((task)->stack)
+#define end_of_stack(p) (&task_thread_info(p)->aux_fp_regs[FP_SIZE-1])
+
+void threads_init(void);
+void threads_cleanup(void);
+
+unsigned long *alloc_thread_stack_node(struct task_struct *p, int node);
+void setup_thread_stack(struct task_struct *p, struct task_struct *org);
+void free_thread_stack(struct task_struct *tsk);
+
+extern struct thread_info *_current_thread_info;
+static inline struct thread_info *current_thread_info(void)
+{
+	return _current_thread_info;
+}
+#endif /* CONFIG_MMU */
+
 #endif
 
 #define TIF_SYSCALL_TRACE	0	/* syscall trace active */
@@ -63,6 +85,8 @@ static inline struct thread_info *current_thread_info(void)
 #define TIF_RESTORE_SIGMASK	7
 #define TIF_NOTIFY_RESUME	8
 #define TIF_SECCOMP		9	/* secure computing */
+#define TIF_SCHED_JB		10
+#define TIF_HOST_THREAD		11
 
 #define _TIF_SYSCALL_TRACE	(1 << TIF_SYSCALL_TRACE)
 #define _TIF_SIGPENDING		(1 << TIF_SIGPENDING)
diff --git a/arch/um/kernel/process.c b/arch/um/kernel/process.c
index 87a8cfa228ca..432edcbb5eea 100644
--- a/arch/um/kernel/process.c
+++ b/arch/um/kernel/process.c
@@ -156,7 +156,8 @@ int copy_thread(unsigned long clone_flags, unsigned long sp,
 		unsigned long arg, struct task_struct * p, unsigned long tls)
 {
 	void (*handler)(void);
-	int kthread = current->flags & PF_KTHREAD;
+	int kthread = current->flags & PF_KTHREAD ||
+		test_ti_thread_flag(current_thread_info(), TIF_HOST_THREAD);
 	int ret = 0;
 
 	p->thread = (struct thread_struct) INIT_THREAD;
@@ -213,10 +214,15 @@ static void um_idle_sleep(void)
 	}
 }
 
+void __weak subarch_cpu_idle(void)
+{
+}
+
 void arch_cpu_idle(void)
 {
 	cpu_tasks[current_thread_info()->cpu].pid = os_getpid();
 	um_idle_sleep();
+	subarch_cpu_idle();
 	local_irq_enable();
 }
 
diff --git a/arch/um/nommu/include/uapi/asm/host_ops.h b/arch/um/nommu/include/uapi/asm/host_ops.h
index 5253c3f8de0e..720385fccbdf 100644
--- a/arch/um/nommu/include/uapi/asm/host_ops.h
+++ b/arch/um/nommu/include/uapi/asm/host_ops.h
@@ -16,11 +16,65 @@ struct lkl_jmp_buf {
  * These operations must be provided by a host library or by the application
  * itself.
  *
+ * @sem_alloc - allocate a host semaphore an initialize it to count
+ * @sem_free - free a host semaphore
+ * @sem_up - perform an up operation on the semaphore
+ * @sem_down - perform a down operation on the semaphore
+ *
+ * @mutex_alloc - allocate and initialize a host mutex; the recursive parameter
+ * determines if the mutex is recursive or not
+ * @mutex_free - free a host mutex
+ * @mutex_lock - acquire the mutex
+ * @mutex_unlock - release the mutex
+ *
+ * @thread_create - create a new thread and run f(arg) in its context; returns a
+ * thread handle or 0 if the thread could not be created
+ * @thread_detach - on POSIX systems, free up resources held by
+ * pthreads. Noop on Win32.
+ * @thread_exit - terminates the current thread
+ * @thread_join - wait for the given thread to terminate. Returns 0
+ * for success, -1 otherwise
+ *
+ * @gettid - returns the host thread id of the caller, which need not
+ * be the same as the handle returned by thread_create
+ *
+ * @jmp_buf_set - runs the give function and setups a jump back point by saving
+ * the context in the jump buffer; jmp_buf_longjmp can be called from the give
+ * function or any callee in that function to return back to the jump back
+ * point
+ *
+ * NOTE: we can't return from jmp_buf_set before calling jmp_buf_longjmp or
+ * otherwise the saved context (stack) is not going to be valid, so we must pass
+ * the function that will eventually call longjmp here
+ *
+ * @jmp_buf_longjmp - perform a jump back to the saved jump buffer
+ *
  * @mem_alloc - allocate memory
  * @mem_free - free memory
  *
  */
 struct lkl_host_operations {
+	struct lkl_sem *(*sem_alloc)(int count);
+	void (*sem_free)(struct lkl_sem *sem);
+	void (*sem_up)(struct lkl_sem *sem);
+	void (*sem_down)(struct lkl_sem *sem);
+
+	struct lkl_mutex *(*mutex_alloc)(int recursive);
+	void (*mutex_free)(struct lkl_mutex *mutex);
+	void (*mutex_lock)(struct lkl_mutex *mutex);
+	void (*mutex_unlock)(struct lkl_mutex *mutex);
+
+	lkl_thread_t (*thread_create)(void *(*f)(void *), void *arg);
+	void (*thread_detach)(void);
+	void (*thread_exit)(void);
+	int (*thread_join)(lkl_thread_t tid);
+	lkl_thread_t (*thread_self)(void);
+	int (*thread_equal)(lkl_thread_t a, lkl_thread_t b);
+	long (*gettid)(void);
+
+	void (*jmp_buf_set)(struct lkl_jmp_buf *jmpb, void (*f)(void));
+	void (*jmp_buf_longjmp)(struct lkl_jmp_buf *jmpb, int val);
+
 	void *(*mem_alloc)(unsigned long mem);
 	void (*mem_free)(void *mem);
 };
diff --git a/arch/um/nommu/um/cpu.c b/arch/um/nommu/um/cpu.c
new file mode 100644
index 000000000000..9986a3f8c5dd
--- /dev/null
+++ b/arch/um/nommu/um/cpu.c
@@ -0,0 +1,236 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/kernel.h>
+#include <linux/sched/stat.h>
+#include <asm/host_ops.h>
+#include <asm/cpu.h>
+#include <asm/thread_info.h>
+#include <asm/unistd.h>
+#include <asm/sched.h>
+#include <asm/syscalls.h>
+#include <init.h>
+
+void run_irqs(void)
+{
+	panic("unimplemented %s", __func__);
+}
+
+int set_irq_pending(int irq)
+{
+	panic("unimplemented %s", __func__);
+}
+
+/*
+ * This structure is used to get access to the "LKL CPU" that allows us to run
+ * Linux code. Because we have to deal with various synchronization requirements
+ * between idle thread, system calls, interrupts, "reentrancy", CPU shutdown,
+ * imbalance wake up (i.e. acquire the CPU from one thread and release it from
+ * another), we can't use a simple synchronization mechanism such as (recursive)
+ * mutex or semaphore. Instead, we use a mutex and a bunch of status data plus a
+ * semaphore.
+ */
+static struct lkl_cpu {
+	/* lock that protects the CPU status data */
+	struct lkl_mutex *lock;
+	/*
+	 * Since we must free the cpu lock during shutdown we need a
+	 * synchronization algorithm between lkl_cpu_shutdown() and the CPU
+	 * access functions since lkl_cpu_get() gets called from thread
+	 * destructor callback functions which may be scheduled after
+	 * lkl_cpu_shutdown() has freed the cpu lock.
+	 *
+	 * An atomic counter is used to keep track of the number of running
+	 * CPU access functions and allow the shutdown function to wait for
+	 * them.
+	 *
+	 * The shutdown functions adds MAX_THREADS to this counter which allows
+	 * the CPU access functions to check if the shutdown process has
+	 * started.
+	 *
+	 * This algorithm assumes that we never have more the MAX_THREADS
+	 * requesting CPU access.
+	 */
+	#define MAX_THREADS 1000000
+	unsigned int shutdown_gate;
+	bool irqs_pending;
+	/* no of threads waiting the CPU */
+	unsigned int sleepers;
+	/* no of times the current thread got the CPU */
+	unsigned int count;
+	/* current thread that owns the CPU */
+	lkl_thread_t owner;
+	/* semaphore for threads waiting the CPU */
+	struct lkl_sem *sem;
+	/* semaphore used for shutdown */
+	struct lkl_sem *shutdown_sem;
+} cpu;
+
+static int __cpu_try_get_lock(int n)
+{
+	lkl_thread_t self;
+
+	if (__sync_fetch_and_add(&cpu.shutdown_gate, n) >= MAX_THREADS)
+		return -2;
+
+	lkl_ops->mutex_lock(cpu.lock);
+
+	if (cpu.shutdown_gate >= MAX_THREADS)
+		return -1;
+
+	self = lkl_ops->thread_self();
+
+	if (cpu.owner && !lkl_ops->thread_equal(cpu.owner, self))
+		return 0;
+
+	cpu.owner = self;
+	cpu.count++;
+
+	return 1;
+}
+
+static void __cpu_try_get_unlock(int lock_ret, int n)
+{
+	if (lock_ret >= -1)
+		lkl_ops->mutex_unlock(cpu.lock);
+	__sync_fetch_and_sub(&cpu.shutdown_gate, n);
+}
+
+void lkl_cpu_change_owner(lkl_thread_t owner)
+{
+	lkl_ops->mutex_lock(cpu.lock);
+	if (cpu.count > 1)
+		lkl_bug("bad count while changing owner\n");
+	cpu.owner = owner;
+	lkl_ops->mutex_unlock(cpu.lock);
+}
+
+int lkl_cpu_get(void)
+{
+	int ret;
+
+	ret = __cpu_try_get_lock(1);
+
+	while (ret == 0) {
+		cpu.sleepers++;
+		__cpu_try_get_unlock(ret, 0);
+		lkl_ops->sem_down(cpu.sem);
+		ret = __cpu_try_get_lock(0);
+	}
+
+	__cpu_try_get_unlock(ret, 1);
+
+	return ret;
+}
+
+void lkl_cpu_put(void)
+{
+	lkl_ops->mutex_lock(cpu.lock);
+
+	if (!cpu.count || !cpu.owner ||
+	    !lkl_ops->thread_equal(cpu.owner, lkl_ops->thread_self()))
+		lkl_bug("%s: unbalanced put\n", __func__);
+
+	while (cpu.irqs_pending && !irqs_disabled()) {
+		cpu.irqs_pending = false;
+		lkl_ops->mutex_unlock(cpu.lock);
+		run_irqs();
+		lkl_ops->mutex_lock(cpu.lock);
+	}
+
+	if (test_ti_thread_flag(current_thread_info(), TIF_HOST_THREAD) &&
+	    !single_task_running() && cpu.count == 1) {
+		if (in_interrupt())
+			lkl_bug("%s: in interrupt\n", __func__);
+		lkl_ops->mutex_unlock(cpu.lock);
+		thread_sched_jb();
+		return;
+	}
+
+	if (--cpu.count > 0) {
+		lkl_ops->mutex_unlock(cpu.lock);
+		return;
+	}
+
+	if (cpu.sleepers) {
+		cpu.sleepers--;
+		lkl_ops->sem_up(cpu.sem);
+	}
+
+	cpu.owner = 0;
+
+	lkl_ops->mutex_unlock(cpu.lock);
+}
+
+int lkl_cpu_try_run_irq(int irq)
+{
+	int ret;
+
+	ret = __cpu_try_get_lock(1);
+	if (!ret) {
+		set_irq_pending(irq);
+		cpu.irqs_pending = true;
+	}
+	__cpu_try_get_unlock(ret, 1);
+
+	return ret;
+}
+
+static void lkl_cpu_shutdown(void)
+{
+	__sync_fetch_and_add(&cpu.shutdown_gate, MAX_THREADS);
+}
+__uml_exitcall(lkl_cpu_shutdown);
+
+void lkl_cpu_wait_shutdown(void)
+{
+	lkl_ops->sem_down(cpu.shutdown_sem);
+	lkl_ops->sem_free(cpu.shutdown_sem);
+}
+
+static void lkl_cpu_cleanup(bool shutdown)
+{
+	while (__sync_fetch_and_add(&cpu.shutdown_gate, 0) > MAX_THREADS)
+		;
+
+	if (shutdown)
+		lkl_ops->sem_up(cpu.shutdown_sem);
+	else if (cpu.shutdown_sem)
+		lkl_ops->sem_free(cpu.shutdown_sem);
+	if (cpu.sem)
+		lkl_ops->sem_free(cpu.sem);
+	if (cpu.lock)
+		lkl_ops->mutex_free(cpu.lock);
+}
+
+void subarch_cpu_idle(void)
+{
+	if (cpu.shutdown_gate >= MAX_THREADS) {
+
+		lkl_ops->mutex_lock(cpu.lock);
+		while (cpu.sleepers--)
+			lkl_ops->sem_up(cpu.sem);
+		lkl_ops->mutex_unlock(cpu.lock);
+
+		lkl_cpu_cleanup(true);
+
+		lkl_ops->thread_exit();
+	}
+
+#ifdef doesntwork
+	/* switch to idle_host_task */
+	wakeup_idle_host_task();
+#endif
+}
+
+int lkl_cpu_init(void)
+{
+	cpu.lock = lkl_ops->mutex_alloc(0);
+	cpu.sem = lkl_ops->sem_alloc(0);
+	cpu.shutdown_sem = lkl_ops->sem_alloc(0);
+
+	if (!cpu.lock || !cpu.sem || !cpu.shutdown_sem) {
+		lkl_cpu_cleanup(false);
+		return -ENOMEM;
+	}
+
+	return 0;
+}
diff --git a/arch/um/nommu/um/threads.c b/arch/um/nommu/um/threads.c
new file mode 100644
index 000000000000..3e70eccc191a
--- /dev/null
+++ b/arch/um/nommu/um/threads.c
@@ -0,0 +1,258 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/slab.h>
+#include <linux/sched/task.h>
+#include <linux/sched/signal.h>
+#include <asm/host_ops.h>
+#include <asm/cpu.h>
+#include <asm/sched.h>
+
+#include <os.h>
+
+static int init_arch_thread(struct arch_thread *thread)
+{
+	thread->sched_sem = lkl_ops->sem_alloc(0);
+	if (!thread->sched_sem)
+		return -ENOMEM;
+
+	thread->dead = false;
+	thread->tid = 0;
+
+	return 0;
+}
+
+unsigned long *alloc_thread_stack_node(struct task_struct *task, int node)
+{
+	struct thread_info *ti;
+
+	ti = kmalloc(sizeof(*ti), GFP_KERNEL | __GFP_ZERO);
+	if (!ti)
+		return NULL;
+
+	ti->task = task;
+	return (unsigned long *)ti;
+}
+
+/*
+ * The only new tasks created are kernel threads that have a predefined starting
+ * point thus no stack copy is required.
+ */
+void setup_thread_stack(struct task_struct *p, struct task_struct *org)
+{
+	struct thread_info *ti = task_thread_info(p);
+	struct thread_info *org_ti = task_thread_info(org);
+
+	ti->flags = org_ti->flags;
+	ti->preempt_count = org_ti->preempt_count;
+	ti->addr_limit = org_ti->addr_limit;
+}
+
+static void kill_thread(struct thread_info *ti)
+{
+	if (!test_ti_thread_flag(ti, TIF_HOST_THREAD)) {
+		ti->task->thread.arch.dead = true;
+		lkl_ops->sem_up(ti->task->thread.arch.sched_sem);
+		lkl_ops->thread_join(ti->task->thread.arch.tid);
+	}
+	lkl_ops->sem_free(ti->task->thread.arch.sched_sem);
+}
+
+void free_thread_stack(struct task_struct *tsk)
+{
+	struct thread_info *ti = task_thread_info(tsk);
+
+	kill_thread(ti);
+	kfree(ti);
+}
+
+struct thread_info *_current_thread_info = &init_thread_union.thread_info;
+
+void switch_threads(jmp_buf *me, jmp_buf *you)
+{
+	/* NOP */
+}
+
+/*
+ * schedule() expects the return of this function to be the task that we
+ * switched away from. Returning prev is not going to work because we are
+ * actually going to return the previous taks that was scheduled before the
+ * task we are going to wake up, and not the current task, e.g.:
+ *
+ * swapper -> init: saved prev on swapper stack is swapper
+ * init -> ksoftirqd0: saved prev on init stack is init
+ * ksoftirqd0 -> swapper: returned prev is swapper
+ */
+static struct task_struct *abs_prev = &init_task;
+
+void arch_switch_to(struct task_struct *prev,
+		    struct task_struct *next)
+{
+	struct arch_thread *_prev = &prev->thread.arch;
+	struct arch_thread *_next = &next->thread.arch;
+	unsigned long _prev_flags = task_thread_info(prev)->flags;
+	struct lkl_jmp_buf *_prev_jb;
+
+	_current_thread_info = task_thread_info(next);
+	next->thread.prev_sched = prev;
+	abs_prev = prev;
+
+	WARN_ON(!_next->tid);
+	lkl_cpu_change_owner(_next->tid);
+
+	if (test_bit(TIF_SCHED_JB, &_prev_flags)) {
+		/* Atomic. Must be done before wakeup next */
+		clear_ti_thread_flag(task_thread_info(prev), TIF_SCHED_JB);
+		_prev_jb = &_prev->sched_jb;
+	}
+
+	lkl_ops->sem_up(_next->sched_sem);
+	if (test_bit(TIF_SCHED_JB, &_prev_flags))
+		lkl_ops->jmp_buf_longjmp(_prev_jb, 1);
+	else
+		lkl_ops->sem_down(_prev->sched_sem);
+
+	if (_prev->dead)
+		lkl_ops->thread_exit();
+
+	/* __switch_to (arch/um) returns this value */
+	current->thread.prev_sched = abs_prev;
+}
+
+int host_task_stub(void *unused)
+{
+	return 0;
+}
+
+void switch_to_host_task(struct task_struct *task)
+{
+	if (WARN_ON(!test_tsk_thread_flag(task, TIF_HOST_THREAD)))
+		return;
+
+	task->thread.arch.tid = lkl_ops->thread_self();
+
+	if (current == task)
+		return;
+
+	wake_up_process(task);
+	thread_sched_jb();
+	lkl_ops->sem_down(task->thread.arch.sched_sem);
+	schedule_tail(abs_prev);
+}
+
+struct thread_bootstrap_arg {
+	struct thread_info *ti;
+	int (*f)(void *a);
+	void *arg;
+};
+
+static void *thread_bootstrap(void *_tba)
+{
+	struct thread_bootstrap_arg *tba = (struct thread_bootstrap_arg *)_tba;
+	struct thread_info *ti = tba->ti;
+	int (*f)(void *) = tba->f;
+	void *arg = tba->arg;
+
+	lkl_ops->sem_down(ti->task->thread.arch.sched_sem);
+	kfree(tba);
+	if (ti->task->thread.prev_sched)
+		schedule_tail(ti->task->thread.prev_sched);
+
+	f(arg);
+	do_exit(0);
+
+	return NULL;
+}
+
+void new_thread(void *stack, jmp_buf *buf, void (*handler)(void))
+{
+	struct thread_info *ti = (struct thread_info *)stack;
+	struct task_struct *p = ti->task;
+	struct thread_bootstrap_arg *tba;
+	int ret;
+
+	unsigned long esp = (unsigned long)p->thread.request.u.thread.proc;
+	unsigned long unused = (unsigned long)p->thread.request.u.thread.arg;
+
+	ret = init_arch_thread(&p->thread.arch);
+	if (ret < 0)
+		panic("%s: init_arch_thread", __func__);
+
+	if ((int (*)(void *))esp == host_task_stub) {
+		set_ti_thread_flag(ti, TIF_HOST_THREAD);
+		return;
+	}
+
+	tba = kmalloc(sizeof(*tba), GFP_KERNEL);
+	if (!tba)
+		return;
+
+	tba->f = (int (*)(void *))esp;
+	tba->arg = (void *)unused;
+	tba->ti = ti;
+
+	p->thread.arch.tid = lkl_ops->thread_create(thread_bootstrap, tba);
+	if (!p->thread.arch.tid) {
+		kfree(tba);
+		return;
+	}
+}
+
+void show_stack(struct task_struct *task, unsigned long *esp)
+{
+}
+
+static inline void pr_early(const char *str)
+{
+	if (lkl_ops->print)
+		lkl_ops->print(str, strlen(str));
+}
+
+/**
+ * This is called before the kernel initializes, so no kernel calls (including
+ * printk) can't be made yet.
+ */
+void threads_init(void)
+{
+	int ret;
+	struct thread_info *ti = &init_thread_union.thread_info;
+
+	ti->task->thread = (struct thread_struct) INIT_THREAD;
+	ret = init_arch_thread(&ti->task->thread.arch);
+	if (ret < 0)
+		pr_early("lkl: failed to allocate init schedule semaphore\n");
+
+	ti->task->thread.arch.tid = lkl_ops->thread_self();
+}
+
+void threads_cleanup(void)
+{
+	struct task_struct *p, *t;
+
+	for_each_process_thread(p, t) {
+		struct thread_info *ti = task_thread_info(t);
+
+		if (t->pid != 1 && !test_ti_thread_flag(ti, TIF_HOST_THREAD))
+			WARN(!(t->flags & PF_KTHREAD),
+			     "non kernel thread task %s\n", t->comm);
+		WARN(t->state == TASK_RUNNING,
+		     "thread %s still running while halting\n", t->comm);
+
+		kill_thread(ti);
+	}
+
+	lkl_ops->sem_free(
+		init_thread_union.thread_info.task->thread.arch.sched_sem);
+}
+
+void initial_thread_cb_skas(void (*proc)(void *), void *arg)
+{
+	pr_warn("unimplemented %s", __func__);
+}
+
+int arch_set_tls(struct task_struct *new, unsigned long tls)
+{
+	panic("unimplemented %s", __func__);
+}
+void clear_flushed_tls(struct task_struct *task)
+{
+	panic("unimplemented %s", __func__);
+}
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v6 12/21] um: nommu: system call interface and application API
  2020-09-24  7:12       ` Hajime Tazaki
@ 2020-09-24  7:12         ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  7:12 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch, Hajime Tazaki

The application API is based on the kernel system call interface in
order to offer a stable API to applications. Note that we can't
offer the full Linux system call interface due to the limitations of
nommu mode such as lack of virtual memory, signal, user processes, etc.

The host is using dedicated kernel thread, which is switched upon the
entry of lkl_syscall so that numbers of context switches can be reduced
for the faster performance.

To expose the API definitions to applications, this commit uses
syscall_wrapper to define arch-specific information of syscall, and this
is used to generate syscall_defs.h for lkl so that it can be used
as a template of the list of syscall available for LKL apps.

To avoid collisions between the Linux API and the LKL API (e.g.  struct
stat, MKNOD, etc.) we use a python script to modify the user headers
and to prefix all of the global symbols (structures, typedefs,
defines) with LKL, lkl, _LKL, _lkl, __LKL or __lkl.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/include/uapi/asm/Kbuild             |   2 +
 arch/um/nommu/include/asm/syscall_wrapper.h |  57 ++++
 arch/um/nommu/include/asm/syscalls.h        |  15 +
 arch/um/nommu/include/uapi/asm/host_ops.h   |  14 +
 arch/um/nommu/include/uapi/asm/syscalls.h   | 287 ++++++++++++++++++++
 arch/um/nommu/include/uapi/asm/unistd.h     |  15 +
 arch/um/nommu/um/syscalls.c                 | 199 ++++++++++++++
 arch/um/scripts/headers_install.py          | 197 ++++++++++++++
 8 files changed, 786 insertions(+)
 create mode 100644 arch/um/include/uapi/asm/Kbuild
 create mode 100644 arch/um/nommu/include/asm/syscall_wrapper.h
 create mode 100644 arch/um/nommu/include/asm/syscalls.h
 create mode 100644 arch/um/nommu/include/uapi/asm/syscalls.h
 create mode 100644 arch/um/nommu/include/uapi/asm/unistd.h
 create mode 100644 arch/um/nommu/um/syscalls.c
 create mode 100755 arch/um/scripts/headers_install.py

diff --git a/arch/um/include/uapi/asm/Kbuild b/arch/um/include/uapi/asm/Kbuild
new file mode 100644
index 000000000000..a983c7f049bc
--- /dev/null
+++ b/arch/um/include/uapi/asm/Kbuild
@@ -0,0 +1,2 @@
+# no generated-y since we need special user headers handling
+# see arch/um/script/headers_install.py
diff --git a/arch/um/nommu/include/asm/syscall_wrapper.h b/arch/um/nommu/include/asm/syscall_wrapper.h
new file mode 100644
index 000000000000..bb2a7d4274f6
--- /dev/null
+++ b/arch/um/nommu/include/asm/syscall_wrapper.h
@@ -0,0 +1,57 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef __UM_SYSCALL_WRAPPER_H
+#define __UM_SYSCALL_WRAPPER_H
+
+#define __SC_ASCII(t, a) #t "," #a
+
+#define __ASCII_MAP0(m, ...)
+#define __ASCII_MAP1(m, t, a) m(t, a)
+#define __ASCII_MAP2(m, t, a, ...) m(t, a) "," __ASCII_MAP1(m, __VA_ARGS__)
+#define __ASCII_MAP3(m, t, a, ...) m(t, a) "," __ASCII_MAP2(m, __VA_ARGS__)
+#define __ASCII_MAP4(m, t, a, ...) m(t, a) "," __ASCII_MAP3(m, __VA_ARGS__)
+#define __ASCII_MAP5(m, t, a, ...) m(t, a) "," __ASCII_MAP4(m, __VA_ARGS__)
+#define __ASCII_MAP6(m, t, a, ...) m(t, a) "," __ASCII_MAP5(m, __VA_ARGS__)
+#define __ASCII_MAP(n, ...) __ASCII_MAP##n(__VA_ARGS__)
+
+#ifdef __MINGW32__
+#define SECTION_ATTRS "n0"
+#else
+#define SECTION_ATTRS "a"
+#endif
+
+#define __SYSCALL_DEFINE_ARCH(x, name, ...)				\
+	asm(".section .syscall_defs,\"" SECTION_ATTRS "\"\n"		\
+	    ".ascii \"#ifdef __NR" #name "\\n\"\n"			\
+	    ".ascii \"SYSCALL_DEFINE" #x "(" #name ","			\
+	    __ASCII_MAP(x, __SC_ASCII, __VA_ARGS__) ")\\n\"\n"		\
+	    ".ascii \"#endif\\n\"\n"					\
+	    ".section .text\n")
+
+#define SYSCALL_DEFINE0(sname)					\
+	SYSCALL_METADATA(_##sname, 0);				\
+	__SYSCALL_DEFINE_ARCH(0, _##sname);			\
+	asmlinkage long sys_##sname(void);			\
+	ALLOW_ERROR_INJECTION(sys_##sname, ERRNO);		\
+	asmlinkage long sys_##sname(void)
+
+#define __SYSCALL_DEFINEx(x, name, ...)					\
+	__SYSCALL_DEFINE_ARCH(x, name, __VA_ARGS__);			\
+	__diag_push();							\
+	__diag_ignore(GCC, 8, "-Wattribute-alias",			\
+		      "Type aliasing is used to sanitize syscall arguments");\
+	asmlinkage long sys##name(__MAP(x, __SC_DECL, __VA_ARGS__))	\
+		__attribute__((alias(__stringify(__se_sys##name))));	\
+	ALLOW_ERROR_INJECTION(sys##name, ERRNO);			\
+	static inline long __do_sys##name(__MAP(x, __SC_DECL, __VA_ARGS__)); \
+	asmlinkage long __se_sys##name(__MAP(x, __SC_LONG, __VA_ARGS__)); \
+	asmlinkage long __se_sys##name(__MAP(x, __SC_LONG, __VA_ARGS__)) \
+	{								\
+		long ret = __do_sys##name(__MAP(x, __SC_CAST, __VA_ARGS__)); \
+		__MAP(x, __SC_TEST, __VA_ARGS__);			\
+		__PROTECT(x, ret, __MAP(x, __SC_ARGS, __VA_ARGS__));	\
+		return ret;						\
+	}								\
+	__diag_pop();							\
+	static inline long __do_sys##name(__MAP(x, __SC_DECL, __VA_ARGS__))
+
+#endif /* __UM_SYSCALL_WRAPPER_H */
diff --git a/arch/um/nommu/include/asm/syscalls.h b/arch/um/nommu/include/asm/syscalls.h
new file mode 100644
index 000000000000..6061d9415dad
--- /dev/null
+++ b/arch/um/nommu/include/asm/syscalls.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+int syscalls_init(void);
+void syscalls_cleanup(void);
+long lkl_syscall(long no, long *params);
+void wakeup_idle_host_task(void);
+
+#define sys_mmap sys_ni_syscall
+#define sys_rt_sigreturn sys_ni_syscall
+#define sys_arch_prctl sys_ni_syscall
+#define sys_iopl sys_ni_syscall
+#define sys_ioperm sys_ni_syscall
+#define sys_clone sys_ni_syscall
+
+int run_syscalls(void);
diff --git a/arch/um/nommu/include/uapi/asm/host_ops.h b/arch/um/nommu/include/uapi/asm/host_ops.h
index 720385fccbdf..6ee9489fc47e 100644
--- a/arch/um/nommu/include/uapi/asm/host_ops.h
+++ b/arch/um/nommu/include/uapi/asm/host_ops.h
@@ -38,6 +38,15 @@ struct lkl_jmp_buf {
  * @gettid - returns the host thread id of the caller, which need not
  * be the same as the handle returned by thread_create
  *
+ * @tls_alloc - allocate a thread local storage key; returns 0 if successful; if
+ * destructor is not NULL it will be called when a thread terminates with its
+ * argument set to the current thread local storage value
+ * @tls_free - frees a thread local storage key; returns 0 if successful
+ * @tls_set - associate data to the thread local storage key; returns 0 if
+ * successful
+ * @tls_get - return data associated with the thread local storage key or NULL
+ * on error
+ *
  * @jmp_buf_set - runs the give function and setups a jump back point by saving
  * the context in the jump buffer; jmp_buf_longjmp can be called from the give
  * function or any callee in that function to return back to the jump back
@@ -72,6 +81,11 @@ struct lkl_host_operations {
 	int (*thread_equal)(lkl_thread_t a, lkl_thread_t b);
 	long (*gettid)(void);
 
+	struct lkl_tls_key *(*tls_alloc)(void (*destructor)(void *));
+	void (*tls_free)(struct lkl_tls_key *key);
+	int (*tls_set)(struct lkl_tls_key *key, void *data);
+	void *(*tls_get)(struct lkl_tls_key *key);
+
 	void (*jmp_buf_set)(struct lkl_jmp_buf *jmpb, void (*f)(void));
 	void (*jmp_buf_longjmp)(struct lkl_jmp_buf *jmpb, int val);
 
diff --git a/arch/um/nommu/include/uapi/asm/syscalls.h b/arch/um/nommu/include/uapi/asm/syscalls.h
new file mode 100644
index 000000000000..751957b978cd
--- /dev/null
+++ b/arch/um/nommu/include/uapi/asm/syscalls.h
@@ -0,0 +1,287 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef __UM_NOMMU_UAPI_SYSCALLS_H
+#define __UM_NOMMU_UAPI_SYSCALLS_H
+
+#include <autoconf.h>
+#include <linux/types.h>
+
+typedef __kernel_uid32_t	qid_t;
+typedef __kernel_fd_set		fd_set;
+typedef __kernel_mode_t		mode_t;
+typedef unsigned short		umode_t;
+typedef __u32			nlink_t;
+typedef __kernel_off_t		off_t;
+typedef __kernel_pid_t		pid_t;
+typedef __kernel_key_t		key_t;
+typedef __kernel_suseconds_t	suseconds_t;
+typedef __kernel_timer_t	timer_t;
+typedef __kernel_clockid_t	clockid_t;
+typedef __kernel_mqd_t		mqd_t;
+typedef __kernel_uid32_t	uid_t;
+typedef __kernel_gid32_t	gid_t;
+typedef __kernel_uid16_t        uid16_t;
+typedef __kernel_gid16_t        gid16_t;
+typedef unsigned long		uintptr_t;
+#ifdef CONFIG_UID16
+typedef __kernel_old_uid_t	old_uid_t;
+typedef __kernel_old_gid_t	old_gid_t;
+#endif
+typedef __kernel_loff_t		loff_t;
+typedef __kernel_size_t		size_t;
+typedef __kernel_ssize_t	ssize_t;
+typedef __kernel_time_t		time_t;
+typedef __kernel_clock_t	clock_t;
+typedef __u32			u32;
+typedef __s32			s32;
+typedef __u64			u64;
+typedef __s64			s64;
+
+typedef __kernel_long_t	__kernel_old_time_t;
+
+#define __user
+
+#include <asm/unistd.h>
+/* Temporary undefine system calls that don't have data types defined in UAPI
+ * headers
+ */
+#undef __NR_kexec_load
+#undef __NR_getcpu
+#undef __NR_sched_getattr
+#undef __NR_sched_setattr
+#undef __NR_sched_setparam
+#undef __NR_sched_getparam
+#undef __NR_sched_setscheduler
+#undef __NR_name_to_handle_at
+#undef __NR_open_by_handle_at
+
+/* deprecated system calls */
+#undef __NR_access
+#undef __NR_chmod
+#undef __NR_chown
+#undef __NR_lchown
+#undef __NR_open
+#undef __NR_creat
+#undef __NR_readlink
+#undef __NR_pipe
+#undef __NR_mknod
+#undef __NR_mkdir
+#undef __NR_rmdir
+#undef __NR_unlink
+#undef __NR_symlink
+#undef __NR_link
+#undef __NR_rename
+#undef __NR_getdents
+#undef __NR_select
+#undef __NR_poll
+#undef __NR_dup2
+#undef __NR_sysfs
+#undef __NR_ustat
+#undef __NR_inotify_init
+#undef __NR_epoll_create
+#undef __NR_epoll_wait
+#undef __NR_signalfd
+#undef __NR_eventfd
+
+#undef __NR_umount
+#define __NR_umount __NR_umount2
+
+#ifdef CONFIG_64BIT
+#define __NR_newfstat __NR3264_fstat
+#define __NR_newfstatat __NR3264_fstatat
+#endif
+
+#include <linux/time.h>
+#include <linux/times.h>
+#include <linux/timex.h>
+#include <linux/capability.h>
+#define __KERNEL__ /* to pull in S_ definitions */
+#include <linux/stat.h>
+#undef __KERNEL__
+#include <linux/errno.h>
+#include <linux/fcntl.h>
+#include <linux/fs.h>
+#include <asm/statfs.h>
+#include <asm/stat.h>
+#include <linux/bpf.h>
+#include <linux/msg.h>
+#include <linux/resource.h>
+#include <linux/sysinfo.h>
+#include <linux/shm.h>
+#include <linux/aio_abi.h>
+#include <linux/socket.h>
+#include <linux/perf_event.h>
+#include <linux/sem.h>
+#include <linux/futex.h>
+#include <linux/poll.h>
+#include <linux/mqueue.h>
+#include <linux/eventpoll.h>
+#include <linux/uio.h>
+#include <asm/signal.h>
+#include <asm/siginfo.h>
+#include <linux/utime.h>
+#include <asm/socket.h>
+#include <linux/icmp.h>
+#include <linux/ip.h>
+
+/* Define data structures used in system calls that are not defined in UAPI
+ * headers
+ */
+struct sockaddr {
+	unsigned short int sa_family;
+	char sa_data[14];
+};
+
+#define __UAPI_DEF_IF_NET_DEVICE_FLAGS_LOWER_UP_DORMANT_ECHO 1
+#define __UAPI_DEF_IF_IFNAMSIZ	1
+#define __UAPI_DEF_IF_NET_DEVICE_FLAGS 1
+#define __UAPI_DEF_IF_IFREQ	1
+#define __UAPI_DEF_IF_IFMAP	1
+#include <linux/if.h>
+#define __UAPI_DEF_IN_IPPROTO	1
+#define __UAPI_DEF_IN_ADDR	1
+#define __UAPI_DEF_IN6_ADDR	1
+#define __UAPI_DEF_IP_MREQ	1
+#define __UAPI_DEF_IN_PKTINFO	1
+#define __UAPI_DEF_SOCKADDR_IN	1
+#define __UAPI_DEF_IN_CLASS	1
+#include <linux/in.h>
+#include <linux/in6.h>
+#include <linux/sockios.h>
+#include <linux/route.h>
+#include <linux/ipv6_route.h>
+#include <linux/ipv6.h>
+#include <linux/netlink.h>
+#include <linux/neighbour.h>
+#include <linux/rtnetlink.h>
+#include <linux/fib_rules.h>
+
+#include <linux/kdev_t.h>
+#include <linux/virtio_blk.h>
+#include <linux/virtio_net.h>
+#include <linux/virtio_ring.h>
+#include <linux/pkt_sched.h>
+#include <linux/io_uring.h>
+#include <linux/utsname.h>
+
+struct user_msghdr {
+	void		__user *msg_name;
+	int		msg_namelen;
+	struct iovec	__user *msg_iov;
+	__kernel_size_t	msg_iovlen;
+	void		__user *msg_control;
+	__kernel_size_t	msg_controllen;
+	unsigned int	msg_flags;
+};
+
+typedef __u32 key_serial_t;
+
+struct mmsghdr {
+	struct user_msghdr  msg_hdr;
+	unsigned int        msg_len;
+};
+
+struct linux_dirent64 {
+	u64		d_ino;
+	s64		d_off;
+	unsigned short	d_reclen;
+	unsigned char	d_type;
+	char		d_name[0];
+};
+
+struct linux_dirent {
+	unsigned long	d_ino;
+	unsigned long	d_off;
+	unsigned short	d_reclen;
+	char		d_name[1];
+};
+
+struct ustat {
+	__kernel_daddr_t	f_tfree;
+	__kernel_ino_t		f_tinode;
+	char			f_fname[6];
+	char			f_fpack[6];
+};
+
+struct __aio_sigset;
+struct clone_args;
+
+typedef __kernel_rwf_t		rwf_t;
+
+long lkl_syscall(long no, long *params);
+long lkl_sys_halt(void);
+
+#define __MAP0(m, ...)
+#define __MAP1(m, t, a) m(t, a)
+#define __MAP2(m, t, a, ...) m(t, a), __MAP1(m, __VA_ARGS__)
+#define __MAP3(m, t, a, ...) m(t, a), __MAP2(m, __VA_ARGS__)
+#define __MAP4(m, t, a, ...) m(t, a), __MAP3(m, __VA_ARGS__)
+#define __MAP5(m, t, a, ...) m(t, a), __MAP4(m, __VA_ARGS__)
+#define __MAP6(m, t, a, ...) m(t, a), __MAP5(m, __VA_ARGS__)
+#define __MAP(n, ...) __MAP##n(__VA_ARGS__)
+
+#define __SC_LONG(t, a) (long)(a)
+#define __SC_TABLE(t, a) {sizeof(t), (long long)(a)}
+#define __SC_DECL(t, a) t a
+
+#define LKL_SYSCALL0(name)					       \
+	static inline long lkl_sys##name(void)			       \
+	{							       \
+		long params[6];					       \
+		return lkl_syscall(__lkl__NR##name, params);	       \
+	}
+
+#if __BITS_PER_LONG == 32
+#define LKL_SYSCALLx(x, name, ...)					\
+	static inline							\
+	long lkl_sys##name(__MAP(x, __SC_DECL, __VA_ARGS__))		\
+	{								\
+		struct {						\
+			unsigned int size;				\
+			long long value;				\
+		} lkl_params[x] = { __MAP(x, __SC_TABLE, __VA_ARGS__) }; \
+		long sys_params[6], i, k;				\
+		for (i = k = 0; i < x && k < 6; i++, k++) {		\
+			if (lkl_params[i].size > sizeof(long) &&	\
+			    k + 1 < 6) {				\
+				sys_params[k] =				\
+					(long)(lkl_params[i].value & (-1UL)); \
+				k++;					\
+				sys_params[k] =				\
+					(long)(lkl_params[i].value >>	\
+					       __BITS_PER_LONG);	\
+			} else {					\
+				sys_params[k] = (long)(lkl_params[i].value); \
+			}						\
+		}							\
+		return lkl_syscall(__lkl__NR##name, sys_params);	\
+	}
+#else
+#define LKL_SYSCALLx(x, name, ...)					\
+	static inline							\
+	long lkl_sys##name(__MAP(x, __SC_DECL, __VA_ARGS__))		\
+	{								\
+		long lkl_params[6] = { __MAP(x, __SC_LONG, __VA_ARGS__) }; \
+		return lkl_syscall(__lkl__NR##name, lkl_params);	\
+	}
+#endif
+
+#define SYSCALL_DEFINE0(name, ...) LKL_SYSCALL0(name)
+#define SYSCALL_DEFINE1(name, ...) LKL_SYSCALLx(1, name, __VA_ARGS__)
+#define SYSCALL_DEFINE2(name, ...) LKL_SYSCALLx(2, name, __VA_ARGS__)
+#define SYSCALL_DEFINE3(name, ...) LKL_SYSCALLx(3, name, __VA_ARGS__)
+#define SYSCALL_DEFINE4(name, ...) LKL_SYSCALLx(4, name, __VA_ARGS__)
+#define SYSCALL_DEFINE5(name, ...) LKL_SYSCALLx(5, name, __VA_ARGS__)
+#define SYSCALL_DEFINE6(name, ...) LKL_SYSCALLx(6, name, __VA_ARGS__)
+
+#if __BITS_PER_LONG == 32
+#pragma GCC diagnostic push
+#pragma GCC diagnostic ignored "-Wpointer-to-int-cast"
+#endif
+
+#include <asm/syscall_defs.h>
+
+#if __BITS_PER_LONG == 32
+#pragma GCC diagnostic pop
+#endif
+
+#endif
diff --git a/arch/um/nommu/include/uapi/asm/unistd.h b/arch/um/nommu/include/uapi/asm/unistd.h
new file mode 100644
index 000000000000..1762ebb829f5
--- /dev/null
+++ b/arch/um/nommu/include/uapi/asm/unistd.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef __UM_NOMMU_UAPI_UNISTD_H
+#define __UM_NOMMU_UAPI_UNISTD_H
+
+#define __ARCH_WANT_NEW_STAT
+#include <asm/bitsperlong.h>
+
+#if __BITS_PER_LONG == 64
+#define __ARCH_WANT_SYS_NEWFSTATAT
+#endif
+
+#include <asm-generic/unistd.h>
+
+
+#endif
diff --git a/arch/um/nommu/um/syscalls.c b/arch/um/nommu/um/syscalls.c
new file mode 100644
index 000000000000..fd343f855bad
--- /dev/null
+++ b/arch/um/nommu/um/syscalls.c
@@ -0,0 +1,199 @@
+// SPDX-License-Identifier: GPL-2.0
+#ifdef CONFIG_ARCH_HAS_SYSCALL_WRAPPER
+#undef CONFIG_ARCH_HAS_SYSCALL_WRAPPER
+#include <linux/syscalls.h>
+#define CONFIG_ARCH_HAS_SYSCALL_WRAPPER
+#endif
+
+#include <linux/stat.h>
+#include <linux/irq.h>
+#include <linux/sched.h>
+#include <linux/interrupt.h>
+#include <linux/jhash.h>
+#include <linux/slab.h>
+#include <linux/types.h>
+#include <linux/net.h>
+#include <linux/task_work.h>
+
+#include <asm/host_ops.h>
+#include <asm/syscalls.h>
+#include <asm/cpu.h>
+#include <asm/sched.h>
+#include <kern_util.h>
+#include <os.h>
+
+typedef long (*syscall_handler_t)(long arg1, ...);
+
+#undef __SYSCALL
+#define __SYSCALL(nr, sym)[nr] = (syscall_handler_t)sym,
+
+syscall_handler_t syscall_table[__NR_syscalls] = {
+	[0 ... __NR_syscalls - 1] =  (syscall_handler_t)sys_ni_syscall,
+#undef __UM_NOMMU_UAPI_UNISTD_H
+#include <asm/unistd.h>
+
+#if __BITS_PER_LONG == 32
+#include <asm/unistd_32.h>
+#endif
+};
+
+
+static long run_syscall(long no, long *params)
+{
+	long ret;
+
+	if (no < 0 || no >= __NR_syscalls)
+		return -ENOSYS;
+
+	ret = syscall_table[no](params[0], params[1], params[2], params[3],
+				params[4], params[5]);
+
+	task_work_run();
+
+	return ret;
+}
+
+
+#define CLONE_FLAGS (CLONE_VM | CLONE_FS | CLONE_FILES | CLONE_THREAD |	\
+		     CLONE_SIGHAND | SIGCHLD)
+
+static int host_task_id;
+static struct task_struct *host0;
+
+static int new_host_task(struct task_struct **task)
+{
+	pid_t pid;
+
+	switch_to_host_task(host0);
+
+	pid = kernel_thread(host_task_stub, NULL, CLONE_FLAGS);
+	if (pid < 0)
+		return pid;
+
+	rcu_read_lock();
+	*task = find_task_by_pid_ns(pid, &init_pid_ns);
+	rcu_read_unlock();
+
+	host_task_id++;
+
+	snprintf((*task)->comm, sizeof((*task)->comm), "host%d", host_task_id);
+
+	return 0;
+}
+static void exit_task(void)
+{
+	do_exit(0);
+}
+
+static void del_host_task(void *arg)
+{
+	struct task_struct *task = (struct task_struct *)arg;
+	struct thread_info *ti = task_thread_info(task);
+
+	if (lkl_cpu_get() < 0)
+		return;
+
+	switch_to_host_task(task);
+	host_task_id--;
+	set_ti_thread_flag(ti, TIF_SCHED_JB);
+	lkl_ops->jmp_buf_set(&ti->task->thread.arch.sched_jb, exit_task);
+}
+
+static struct lkl_tls_key *task_key;
+
+long lkl_syscall(long no, long *params)
+{
+	struct task_struct *task = host0;
+	long ret;
+
+	ret = lkl_cpu_get();
+	if (ret < 0)
+		return ret;
+
+	if (lkl_ops->tls_get) {
+		task = lkl_ops->tls_get(task_key);
+		if (!task) {
+			ret = new_host_task(&task);
+			if (ret)
+				goto out;
+			lkl_ops->tls_set(task_key, task);
+		}
+	}
+
+	switch_to_host_task(task);
+
+	ret = run_syscall(no, params);
+
+	if (no == __NR_reboot) {
+		thread_sched_jb();
+		return ret;
+	}
+
+out:
+	lkl_cpu_put();
+
+	return ret;
+}
+
+static struct task_struct *idle_host_task;
+
+/* called from idle, don't failed, don't block */
+void wakeup_idle_host_task(void)
+{
+	if (!need_resched() && idle_host_task)
+		wake_up_process(idle_host_task);
+}
+
+static int idle_host_task_loop(void *unused)
+{
+	struct thread_info *ti = task_thread_info(current);
+
+	snprintf(current->comm, sizeof(current->comm), "idle_host_task");
+	set_thread_flag(TIF_HOST_THREAD);
+	idle_host_task = current;
+
+	for (;;) {
+		lkl_cpu_put();
+		lkl_ops->sem_down(ti->task->thread.arch.sched_sem);
+		if (idle_host_task == NULL) {
+			lkl_ops->thread_exit();
+			return 0;
+		}
+		schedule_tail(ti->task->thread.prev_sched);
+	}
+}
+
+int syscalls_init(void)
+{
+	snprintf(current->comm, sizeof(current->comm), "host0");
+	set_thread_flag(TIF_HOST_THREAD);
+	host0 = current;
+
+	if (lkl_ops->tls_alloc) {
+		task_key = lkl_ops->tls_alloc(del_host_task);
+		if (!task_key)
+			return -1;
+	}
+
+	if (kernel_thread(idle_host_task_loop, NULL, CLONE_FLAGS) < 0) {
+		if (lkl_ops->tls_free)
+			lkl_ops->tls_free(task_key);
+		return -1;
+	}
+
+	return 0;
+}
+
+void syscalls_cleanup(void)
+{
+	if (idle_host_task) {
+		struct thread_info *ti = task_thread_info(idle_host_task);
+
+		idle_host_task = NULL;
+		lkl_ops->sem_up(ti->task->thread.arch.sched_sem);
+		lkl_ops->thread_join(ti->task->thread.arch.tid);
+	}
+
+	if (lkl_ops->tls_free)
+		lkl_ops->tls_free(task_key);
+}
diff --git a/arch/um/scripts/headers_install.py b/arch/um/scripts/headers_install.py
new file mode 100755
index 000000000000..ec7356c0dee9
--- /dev/null
+++ b/arch/um/scripts/headers_install.py
@@ -0,0 +1,197 @@
+#!/usr/bin/env python
+# SPDX-License-Identifier: GPL-2.0
+import re, os, sys, argparse, multiprocessing, fnmatch
+
+srctree = os.environ["srctree"]
+objtree = os.environ["objtree"]
+install_hdr_path = os.environ["INSTALL_HDR_PATH"]
+header_paths = [ install_hdr_path + "/include/", "arch/um/nommu/include/uapi/",
+                 "arch/um/nommu/include/generated/uapi/", "include/generated/" ]
+
+headers = set()
+includes = set()
+
+def relpath2abspath(relpath):
+    if "generated" in relpath or install_hdr_path in relpath:
+        return objtree + "/" + relpath
+    else:
+        return srctree + "/" + relpath
+
+def find_headers(path):
+    headers.add(path)
+    f = open(relpath2abspath(path))
+    for l in f.readlines():
+        m = re.search("#\s*include <(.*)>", l)
+        try:
+            i = m.group(1)
+            for p in header_paths:
+                if os.access(relpath2abspath(p + i), os.R_OK):
+                    if p + i not in headers:
+                        includes.add(i)
+                        headers.add(p + i)
+                        find_headers(p + i)
+        except:
+            pass
+    f.close()
+
+def has_lkl_prefix(w):
+  return w.startswith("lkl") or w.startswith("_lkl") or w.startswith("__lkl") \
+         or w.startswith("LKL") or w.startswith("_LKL") or w.startswith("__LKL")
+
+def find_symbols(regexp, store):
+    for h in headers:
+        f = open(h)
+        for l in f.readlines():
+            m = regexp.search(l)
+            if not m:
+                continue
+            for e in reversed(m.groups()):
+                if e:
+                    if not has_lkl_prefix(e):
+                        store.add(e)
+                    break
+        f.close()
+
+def find_ml_symbols(regexp, store):
+    for h in headers:
+        for i in regexp.finditer(open(h).read()):
+            for j in reversed(i.groups()):
+                if j:
+                    if not has_lkl_prefix(j):
+                        store.add(j)
+                    break
+
+def find_enums(block_regexp, symbol_regexp, store):
+    for h in headers:
+        # remove comments
+        content = re.sub(re.compile("(\/\*(\*(?!\/)|[^*])*\*\/)", re.S|re.M), " ", open(h).read())
+        # remove preprocesor lines
+        clean_content = ""
+        for l in content.split("\n"):
+            if re.match("\s*#", l):
+                continue
+            clean_content += l + "\n"
+        for i in block_regexp.finditer(clean_content):
+            for j in reversed(i.groups()):
+                if j:
+                    for k in symbol_regexp.finditer(j):
+                        for l in k.groups():
+                            if l:
+                                if not has_lkl_prefix(l):
+                                    store.add(l)
+                                break
+
+def lkl_prefix(w):
+    r = ""
+
+    if w.startswith("__"):
+        r = "__"
+    elif w.startswith("_"):
+        r = "_"
+
+    if w.isupper():
+        r += "LKL"
+    else:
+        r += "lkl"
+
+    if not w.startswith("_"):
+        r += "_"
+
+    r += w
+
+    return r
+
+def replace(h):
+    content = open(h).read()
+    for i in includes:
+        search_str = "(#[ \t]*include[ \t]*[<\"][ \t]*)" + i + "([ \t]*[>\"])"
+        replace_str = "\\1" + "lkl/" + i + "\\2"
+        content = re.sub(search_str, replace_str, content)
+    tmp = ""
+    for w in re.split("(\W+)", content):
+        if w in defines:
+            w = lkl_prefix(w)
+        tmp += w
+    content = tmp
+    for s in structs:
+        search_str = "(\W?struct\s+)" + s + "(\W)"
+        replace_str = "\\1" + lkl_prefix(s) + "\\2"
+        content = re.sub(search_str, replace_str, content, flags = re.MULTILINE)
+    for s in unions:
+        search_str = "(\W?union\s+)" + s + "(\W)"
+        replace_str = "\\1" + lkl_prefix(s) + "\\2"
+        content = re.sub(search_str, replace_str, content, flags = re.MULTILINE)
+    open(h, 'w').write(content)
+
+parser = argparse.ArgumentParser(description='install lkl headers')
+parser.add_argument('path', help='path to install to', )
+parser.add_argument('-j', '--jobs', help='number of parallel jobs', default=1, type=int)
+args = parser.parse_args()
+
+find_headers("arch/um/nommu/include/uapi/asm/syscalls.h")
+headers.add("arch/um/nommu/include/uapi/asm/host_ops.h")
+
+if 'LKL_INSTALL_ADDITIONAL_HEADERS' in os.environ:
+    with open(os.environ['LKL_INSTALL_ADDITIONAL_HEADERS'], 'rU') as f:
+        for line in f.readlines():
+            line = line.split('#', 1)[0].strip()
+            if line != '':
+                headers.add(line)
+
+new_headers = set()
+
+for h in headers:
+    h = relpath2abspath(h)
+    dir = os.path.dirname(h)
+    out_dir = args.path + "/" + re.sub("(" + srctree + "/arch/um/nommu/include/uapi/|arch/um/nommu/include/generated/uapi/|include/generated/uapi/|include/generated|" + install_hdr_path + "/include/)(.*)", "lkl/\\2", dir)
+    try:
+        os.makedirs(out_dir)
+    except:
+        pass
+    print("  INSTALL\t%s" % (out_dir + "/" + os.path.basename(h)))
+    os.system(srctree+"/scripts/headers_install.sh %s %s" % (os.path.abspath(h),
+                                                       out_dir + "/" + os.path.basename(h)))
+    new_headers.add(out_dir + "/" + os.path.basename(h))
+
+headers = new_headers
+
+defines = set()
+structs = set()
+unions = set()
+
+p = re.compile("#[ \t]*define[ \t]*(\w+)")
+find_symbols(p, defines)
+p = re.compile("typedef.*(\(\*(\w+)\)\(.*\)\s*|\W+(\w+)\s*|\s+(\w+)\(.*\)\s*);")
+find_symbols(p, defines)
+p = re.compile("typedef\s+(struct|union)\s+\w*\s*{[^\\{\}]*}\W*(\w+)\s*;", re.M|re.S)
+find_ml_symbols(p, defines)
+defines.add("siginfo_t")
+defines.add("sigevent_t")
+p = re.compile("struct\s+(\w+)\s*\{")
+find_symbols(p, structs)
+structs.add("iovec")
+p = re.compile("union\s+(\w+)\s*\{")
+find_symbols(p, unions)
+p = re.compile("static\s+__inline__(\s+\w+)+\s+(\w+)\([^)]*\)\s")
+find_symbols(p, defines)
+p = re.compile("static\s+__always_inline(\s+\w+)+\s+(\w+)\([^)]*\)\s")
+find_symbols(p, defines)
+p = re.compile("enum\s+(\w*)\s*{([^}]*)}", re.M|re.S)
+q = re.compile("(\w+)\s*(,|=[^,]*|$)", re.M|re.S)
+find_enums(p, q, defines)
+
+# needed for i386
+defines.add("__NR_stime")
+
+def process_header(h):
+    print("  REPLACE\t%s" % (out_dir + "/" + os.path.basename(h)))
+    replace(h)
+
+p = multiprocessing.Pool(args.jobs)
+try:
+    p.map_async(process_header, headers).wait(999999)
+    p.close()
+except:
+    p.terminate()
+finally:
+    p.join()
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v6 12/21] um: nommu: system call interface and application API
@ 2020-09-24  7:12         ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  7:12 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Hajime Tazaki, retrage01

The application API is based on the kernel system call interface in
order to offer a stable API to applications. Note that we can't
offer the full Linux system call interface due to the limitations of
nommu mode such as lack of virtual memory, signal, user processes, etc.

The host is using dedicated kernel thread, which is switched upon the
entry of lkl_syscall so that numbers of context switches can be reduced
for the faster performance.

To expose the API definitions to applications, this commit uses
syscall_wrapper to define arch-specific information of syscall, and this
is used to generate syscall_defs.h for lkl so that it can be used
as a template of the list of syscall available for LKL apps.

To avoid collisions between the Linux API and the LKL API (e.g.  struct
stat, MKNOD, etc.) we use a python script to modify the user headers
and to prefix all of the global symbols (structures, typedefs,
defines) with LKL, lkl, _LKL, _lkl, __LKL or __lkl.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/include/uapi/asm/Kbuild             |   2 +
 arch/um/nommu/include/asm/syscall_wrapper.h |  57 ++++
 arch/um/nommu/include/asm/syscalls.h        |  15 +
 arch/um/nommu/include/uapi/asm/host_ops.h   |  14 +
 arch/um/nommu/include/uapi/asm/syscalls.h   | 287 ++++++++++++++++++++
 arch/um/nommu/include/uapi/asm/unistd.h     |  15 +
 arch/um/nommu/um/syscalls.c                 | 199 ++++++++++++++
 arch/um/scripts/headers_install.py          | 197 ++++++++++++++
 8 files changed, 786 insertions(+)
 create mode 100644 arch/um/include/uapi/asm/Kbuild
 create mode 100644 arch/um/nommu/include/asm/syscall_wrapper.h
 create mode 100644 arch/um/nommu/include/asm/syscalls.h
 create mode 100644 arch/um/nommu/include/uapi/asm/syscalls.h
 create mode 100644 arch/um/nommu/include/uapi/asm/unistd.h
 create mode 100644 arch/um/nommu/um/syscalls.c
 create mode 100755 arch/um/scripts/headers_install.py

diff --git a/arch/um/include/uapi/asm/Kbuild b/arch/um/include/uapi/asm/Kbuild
new file mode 100644
index 000000000000..a983c7f049bc
--- /dev/null
+++ b/arch/um/include/uapi/asm/Kbuild
@@ -0,0 +1,2 @@
+# no generated-y since we need special user headers handling
+# see arch/um/script/headers_install.py
diff --git a/arch/um/nommu/include/asm/syscall_wrapper.h b/arch/um/nommu/include/asm/syscall_wrapper.h
new file mode 100644
index 000000000000..bb2a7d4274f6
--- /dev/null
+++ b/arch/um/nommu/include/asm/syscall_wrapper.h
@@ -0,0 +1,57 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef __UM_SYSCALL_WRAPPER_H
+#define __UM_SYSCALL_WRAPPER_H
+
+#define __SC_ASCII(t, a) #t "," #a
+
+#define __ASCII_MAP0(m, ...)
+#define __ASCII_MAP1(m, t, a) m(t, a)
+#define __ASCII_MAP2(m, t, a, ...) m(t, a) "," __ASCII_MAP1(m, __VA_ARGS__)
+#define __ASCII_MAP3(m, t, a, ...) m(t, a) "," __ASCII_MAP2(m, __VA_ARGS__)
+#define __ASCII_MAP4(m, t, a, ...) m(t, a) "," __ASCII_MAP3(m, __VA_ARGS__)
+#define __ASCII_MAP5(m, t, a, ...) m(t, a) "," __ASCII_MAP4(m, __VA_ARGS__)
+#define __ASCII_MAP6(m, t, a, ...) m(t, a) "," __ASCII_MAP5(m, __VA_ARGS__)
+#define __ASCII_MAP(n, ...) __ASCII_MAP##n(__VA_ARGS__)
+
+#ifdef __MINGW32__
+#define SECTION_ATTRS "n0"
+#else
+#define SECTION_ATTRS "a"
+#endif
+
+#define __SYSCALL_DEFINE_ARCH(x, name, ...)				\
+	asm(".section .syscall_defs,\"" SECTION_ATTRS "\"\n"		\
+	    ".ascii \"#ifdef __NR" #name "\\n\"\n"			\
+	    ".ascii \"SYSCALL_DEFINE" #x "(" #name ","			\
+	    __ASCII_MAP(x, __SC_ASCII, __VA_ARGS__) ")\\n\"\n"		\
+	    ".ascii \"#endif\\n\"\n"					\
+	    ".section .text\n")
+
+#define SYSCALL_DEFINE0(sname)					\
+	SYSCALL_METADATA(_##sname, 0);				\
+	__SYSCALL_DEFINE_ARCH(0, _##sname);			\
+	asmlinkage long sys_##sname(void);			\
+	ALLOW_ERROR_INJECTION(sys_##sname, ERRNO);		\
+	asmlinkage long sys_##sname(void)
+
+#define __SYSCALL_DEFINEx(x, name, ...)					\
+	__SYSCALL_DEFINE_ARCH(x, name, __VA_ARGS__);			\
+	__diag_push();							\
+	__diag_ignore(GCC, 8, "-Wattribute-alias",			\
+		      "Type aliasing is used to sanitize syscall arguments");\
+	asmlinkage long sys##name(__MAP(x, __SC_DECL, __VA_ARGS__))	\
+		__attribute__((alias(__stringify(__se_sys##name))));	\
+	ALLOW_ERROR_INJECTION(sys##name, ERRNO);			\
+	static inline long __do_sys##name(__MAP(x, __SC_DECL, __VA_ARGS__)); \
+	asmlinkage long __se_sys##name(__MAP(x, __SC_LONG, __VA_ARGS__)); \
+	asmlinkage long __se_sys##name(__MAP(x, __SC_LONG, __VA_ARGS__)) \
+	{								\
+		long ret = __do_sys##name(__MAP(x, __SC_CAST, __VA_ARGS__)); \
+		__MAP(x, __SC_TEST, __VA_ARGS__);			\
+		__PROTECT(x, ret, __MAP(x, __SC_ARGS, __VA_ARGS__));	\
+		return ret;						\
+	}								\
+	__diag_pop();							\
+	static inline long __do_sys##name(__MAP(x, __SC_DECL, __VA_ARGS__))
+
+#endif /* __UM_SYSCALL_WRAPPER_H */
diff --git a/arch/um/nommu/include/asm/syscalls.h b/arch/um/nommu/include/asm/syscalls.h
new file mode 100644
index 000000000000..6061d9415dad
--- /dev/null
+++ b/arch/um/nommu/include/asm/syscalls.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+int syscalls_init(void);
+void syscalls_cleanup(void);
+long lkl_syscall(long no, long *params);
+void wakeup_idle_host_task(void);
+
+#define sys_mmap sys_ni_syscall
+#define sys_rt_sigreturn sys_ni_syscall
+#define sys_arch_prctl sys_ni_syscall
+#define sys_iopl sys_ni_syscall
+#define sys_ioperm sys_ni_syscall
+#define sys_clone sys_ni_syscall
+
+int run_syscalls(void);
diff --git a/arch/um/nommu/include/uapi/asm/host_ops.h b/arch/um/nommu/include/uapi/asm/host_ops.h
index 720385fccbdf..6ee9489fc47e 100644
--- a/arch/um/nommu/include/uapi/asm/host_ops.h
+++ b/arch/um/nommu/include/uapi/asm/host_ops.h
@@ -38,6 +38,15 @@ struct lkl_jmp_buf {
  * @gettid - returns the host thread id of the caller, which need not
  * be the same as the handle returned by thread_create
  *
+ * @tls_alloc - allocate a thread local storage key; returns 0 if successful; if
+ * destructor is not NULL it will be called when a thread terminates with its
+ * argument set to the current thread local storage value
+ * @tls_free - frees a thread local storage key; returns 0 if successful
+ * @tls_set - associate data to the thread local storage key; returns 0 if
+ * successful
+ * @tls_get - return data associated with the thread local storage key or NULL
+ * on error
+ *
  * @jmp_buf_set - runs the give function and setups a jump back point by saving
  * the context in the jump buffer; jmp_buf_longjmp can be called from the give
  * function or any callee in that function to return back to the jump back
@@ -72,6 +81,11 @@ struct lkl_host_operations {
 	int (*thread_equal)(lkl_thread_t a, lkl_thread_t b);
 	long (*gettid)(void);
 
+	struct lkl_tls_key *(*tls_alloc)(void (*destructor)(void *));
+	void (*tls_free)(struct lkl_tls_key *key);
+	int (*tls_set)(struct lkl_tls_key *key, void *data);
+	void *(*tls_get)(struct lkl_tls_key *key);
+
 	void (*jmp_buf_set)(struct lkl_jmp_buf *jmpb, void (*f)(void));
 	void (*jmp_buf_longjmp)(struct lkl_jmp_buf *jmpb, int val);
 
diff --git a/arch/um/nommu/include/uapi/asm/syscalls.h b/arch/um/nommu/include/uapi/asm/syscalls.h
new file mode 100644
index 000000000000..751957b978cd
--- /dev/null
+++ b/arch/um/nommu/include/uapi/asm/syscalls.h
@@ -0,0 +1,287 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef __UM_NOMMU_UAPI_SYSCALLS_H
+#define __UM_NOMMU_UAPI_SYSCALLS_H
+
+#include <autoconf.h>
+#include <linux/types.h>
+
+typedef __kernel_uid32_t	qid_t;
+typedef __kernel_fd_set		fd_set;
+typedef __kernel_mode_t		mode_t;
+typedef unsigned short		umode_t;
+typedef __u32			nlink_t;
+typedef __kernel_off_t		off_t;
+typedef __kernel_pid_t		pid_t;
+typedef __kernel_key_t		key_t;
+typedef __kernel_suseconds_t	suseconds_t;
+typedef __kernel_timer_t	timer_t;
+typedef __kernel_clockid_t	clockid_t;
+typedef __kernel_mqd_t		mqd_t;
+typedef __kernel_uid32_t	uid_t;
+typedef __kernel_gid32_t	gid_t;
+typedef __kernel_uid16_t        uid16_t;
+typedef __kernel_gid16_t        gid16_t;
+typedef unsigned long		uintptr_t;
+#ifdef CONFIG_UID16
+typedef __kernel_old_uid_t	old_uid_t;
+typedef __kernel_old_gid_t	old_gid_t;
+#endif
+typedef __kernel_loff_t		loff_t;
+typedef __kernel_size_t		size_t;
+typedef __kernel_ssize_t	ssize_t;
+typedef __kernel_time_t		time_t;
+typedef __kernel_clock_t	clock_t;
+typedef __u32			u32;
+typedef __s32			s32;
+typedef __u64			u64;
+typedef __s64			s64;
+
+typedef __kernel_long_t	__kernel_old_time_t;
+
+#define __user
+
+#include <asm/unistd.h>
+/* Temporary undefine system calls that don't have data types defined in UAPI
+ * headers
+ */
+#undef __NR_kexec_load
+#undef __NR_getcpu
+#undef __NR_sched_getattr
+#undef __NR_sched_setattr
+#undef __NR_sched_setparam
+#undef __NR_sched_getparam
+#undef __NR_sched_setscheduler
+#undef __NR_name_to_handle_at
+#undef __NR_open_by_handle_at
+
+/* deprecated system calls */
+#undef __NR_access
+#undef __NR_chmod
+#undef __NR_chown
+#undef __NR_lchown
+#undef __NR_open
+#undef __NR_creat
+#undef __NR_readlink
+#undef __NR_pipe
+#undef __NR_mknod
+#undef __NR_mkdir
+#undef __NR_rmdir
+#undef __NR_unlink
+#undef __NR_symlink
+#undef __NR_link
+#undef __NR_rename
+#undef __NR_getdents
+#undef __NR_select
+#undef __NR_poll
+#undef __NR_dup2
+#undef __NR_sysfs
+#undef __NR_ustat
+#undef __NR_inotify_init
+#undef __NR_epoll_create
+#undef __NR_epoll_wait
+#undef __NR_signalfd
+#undef __NR_eventfd
+
+#undef __NR_umount
+#define __NR_umount __NR_umount2
+
+#ifdef CONFIG_64BIT
+#define __NR_newfstat __NR3264_fstat
+#define __NR_newfstatat __NR3264_fstatat
+#endif
+
+#include <linux/time.h>
+#include <linux/times.h>
+#include <linux/timex.h>
+#include <linux/capability.h>
+#define __KERNEL__ /* to pull in S_ definitions */
+#include <linux/stat.h>
+#undef __KERNEL__
+#include <linux/errno.h>
+#include <linux/fcntl.h>
+#include <linux/fs.h>
+#include <asm/statfs.h>
+#include <asm/stat.h>
+#include <linux/bpf.h>
+#include <linux/msg.h>
+#include <linux/resource.h>
+#include <linux/sysinfo.h>
+#include <linux/shm.h>
+#include <linux/aio_abi.h>
+#include <linux/socket.h>
+#include <linux/perf_event.h>
+#include <linux/sem.h>
+#include <linux/futex.h>
+#include <linux/poll.h>
+#include <linux/mqueue.h>
+#include <linux/eventpoll.h>
+#include <linux/uio.h>
+#include <asm/signal.h>
+#include <asm/siginfo.h>
+#include <linux/utime.h>
+#include <asm/socket.h>
+#include <linux/icmp.h>
+#include <linux/ip.h>
+
+/* Define data structures used in system calls that are not defined in UAPI
+ * headers
+ */
+struct sockaddr {
+	unsigned short int sa_family;
+	char sa_data[14];
+};
+
+#define __UAPI_DEF_IF_NET_DEVICE_FLAGS_LOWER_UP_DORMANT_ECHO 1
+#define __UAPI_DEF_IF_IFNAMSIZ	1
+#define __UAPI_DEF_IF_NET_DEVICE_FLAGS 1
+#define __UAPI_DEF_IF_IFREQ	1
+#define __UAPI_DEF_IF_IFMAP	1
+#include <linux/if.h>
+#define __UAPI_DEF_IN_IPPROTO	1
+#define __UAPI_DEF_IN_ADDR	1
+#define __UAPI_DEF_IN6_ADDR	1
+#define __UAPI_DEF_IP_MREQ	1
+#define __UAPI_DEF_IN_PKTINFO	1
+#define __UAPI_DEF_SOCKADDR_IN	1
+#define __UAPI_DEF_IN_CLASS	1
+#include <linux/in.h>
+#include <linux/in6.h>
+#include <linux/sockios.h>
+#include <linux/route.h>
+#include <linux/ipv6_route.h>
+#include <linux/ipv6.h>
+#include <linux/netlink.h>
+#include <linux/neighbour.h>
+#include <linux/rtnetlink.h>
+#include <linux/fib_rules.h>
+
+#include <linux/kdev_t.h>
+#include <linux/virtio_blk.h>
+#include <linux/virtio_net.h>
+#include <linux/virtio_ring.h>
+#include <linux/pkt_sched.h>
+#include <linux/io_uring.h>
+#include <linux/utsname.h>
+
+struct user_msghdr {
+	void		__user *msg_name;
+	int		msg_namelen;
+	struct iovec	__user *msg_iov;
+	__kernel_size_t	msg_iovlen;
+	void		__user *msg_control;
+	__kernel_size_t	msg_controllen;
+	unsigned int	msg_flags;
+};
+
+typedef __u32 key_serial_t;
+
+struct mmsghdr {
+	struct user_msghdr  msg_hdr;
+	unsigned int        msg_len;
+};
+
+struct linux_dirent64 {
+	u64		d_ino;
+	s64		d_off;
+	unsigned short	d_reclen;
+	unsigned char	d_type;
+	char		d_name[0];
+};
+
+struct linux_dirent {
+	unsigned long	d_ino;
+	unsigned long	d_off;
+	unsigned short	d_reclen;
+	char		d_name[1];
+};
+
+struct ustat {
+	__kernel_daddr_t	f_tfree;
+	__kernel_ino_t		f_tinode;
+	char			f_fname[6];
+	char			f_fpack[6];
+};
+
+struct __aio_sigset;
+struct clone_args;
+
+typedef __kernel_rwf_t		rwf_t;
+
+long lkl_syscall(long no, long *params);
+long lkl_sys_halt(void);
+
+#define __MAP0(m, ...)
+#define __MAP1(m, t, a) m(t, a)
+#define __MAP2(m, t, a, ...) m(t, a), __MAP1(m, __VA_ARGS__)
+#define __MAP3(m, t, a, ...) m(t, a), __MAP2(m, __VA_ARGS__)
+#define __MAP4(m, t, a, ...) m(t, a), __MAP3(m, __VA_ARGS__)
+#define __MAP5(m, t, a, ...) m(t, a), __MAP4(m, __VA_ARGS__)
+#define __MAP6(m, t, a, ...) m(t, a), __MAP5(m, __VA_ARGS__)
+#define __MAP(n, ...) __MAP##n(__VA_ARGS__)
+
+#define __SC_LONG(t, a) (long)(a)
+#define __SC_TABLE(t, a) {sizeof(t), (long long)(a)}
+#define __SC_DECL(t, a) t a
+
+#define LKL_SYSCALL0(name)					       \
+	static inline long lkl_sys##name(void)			       \
+	{							       \
+		long params[6];					       \
+		return lkl_syscall(__lkl__NR##name, params);	       \
+	}
+
+#if __BITS_PER_LONG == 32
+#define LKL_SYSCALLx(x, name, ...)					\
+	static inline							\
+	long lkl_sys##name(__MAP(x, __SC_DECL, __VA_ARGS__))		\
+	{								\
+		struct {						\
+			unsigned int size;				\
+			long long value;				\
+		} lkl_params[x] = { __MAP(x, __SC_TABLE, __VA_ARGS__) }; \
+		long sys_params[6], i, k;				\
+		for (i = k = 0; i < x && k < 6; i++, k++) {		\
+			if (lkl_params[i].size > sizeof(long) &&	\
+			    k + 1 < 6) {				\
+				sys_params[k] =				\
+					(long)(lkl_params[i].value & (-1UL)); \
+				k++;					\
+				sys_params[k] =				\
+					(long)(lkl_params[i].value >>	\
+					       __BITS_PER_LONG);	\
+			} else {					\
+				sys_params[k] = (long)(lkl_params[i].value); \
+			}						\
+		}							\
+		return lkl_syscall(__lkl__NR##name, sys_params);	\
+	}
+#else
+#define LKL_SYSCALLx(x, name, ...)					\
+	static inline							\
+	long lkl_sys##name(__MAP(x, __SC_DECL, __VA_ARGS__))		\
+	{								\
+		long lkl_params[6] = { __MAP(x, __SC_LONG, __VA_ARGS__) }; \
+		return lkl_syscall(__lkl__NR##name, lkl_params);	\
+	}
+#endif
+
+#define SYSCALL_DEFINE0(name, ...) LKL_SYSCALL0(name)
+#define SYSCALL_DEFINE1(name, ...) LKL_SYSCALLx(1, name, __VA_ARGS__)
+#define SYSCALL_DEFINE2(name, ...) LKL_SYSCALLx(2, name, __VA_ARGS__)
+#define SYSCALL_DEFINE3(name, ...) LKL_SYSCALLx(3, name, __VA_ARGS__)
+#define SYSCALL_DEFINE4(name, ...) LKL_SYSCALLx(4, name, __VA_ARGS__)
+#define SYSCALL_DEFINE5(name, ...) LKL_SYSCALLx(5, name, __VA_ARGS__)
+#define SYSCALL_DEFINE6(name, ...) LKL_SYSCALLx(6, name, __VA_ARGS__)
+
+#if __BITS_PER_LONG == 32
+#pragma GCC diagnostic push
+#pragma GCC diagnostic ignored "-Wpointer-to-int-cast"
+#endif
+
+#include <asm/syscall_defs.h>
+
+#if __BITS_PER_LONG == 32
+#pragma GCC diagnostic pop
+#endif
+
+#endif
diff --git a/arch/um/nommu/include/uapi/asm/unistd.h b/arch/um/nommu/include/uapi/asm/unistd.h
new file mode 100644
index 000000000000..1762ebb829f5
--- /dev/null
+++ b/arch/um/nommu/include/uapi/asm/unistd.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef __UM_NOMMU_UAPI_UNISTD_H
+#define __UM_NOMMU_UAPI_UNISTD_H
+
+#define __ARCH_WANT_NEW_STAT
+#include <asm/bitsperlong.h>
+
+#if __BITS_PER_LONG == 64
+#define __ARCH_WANT_SYS_NEWFSTATAT
+#endif
+
+#include <asm-generic/unistd.h>
+
+
+#endif
diff --git a/arch/um/nommu/um/syscalls.c b/arch/um/nommu/um/syscalls.c
new file mode 100644
index 000000000000..fd343f855bad
--- /dev/null
+++ b/arch/um/nommu/um/syscalls.c
@@ -0,0 +1,199 @@
+// SPDX-License-Identifier: GPL-2.0
+#ifdef CONFIG_ARCH_HAS_SYSCALL_WRAPPER
+#undef CONFIG_ARCH_HAS_SYSCALL_WRAPPER
+#include <linux/syscalls.h>
+#define CONFIG_ARCH_HAS_SYSCALL_WRAPPER
+#endif
+
+#include <linux/stat.h>
+#include <linux/irq.h>
+#include <linux/sched.h>
+#include <linux/interrupt.h>
+#include <linux/jhash.h>
+#include <linux/slab.h>
+#include <linux/types.h>
+#include <linux/net.h>
+#include <linux/task_work.h>
+
+#include <asm/host_ops.h>
+#include <asm/syscalls.h>
+#include <asm/cpu.h>
+#include <asm/sched.h>
+#include <kern_util.h>
+#include <os.h>
+
+typedef long (*syscall_handler_t)(long arg1, ...);
+
+#undef __SYSCALL
+#define __SYSCALL(nr, sym)[nr] = (syscall_handler_t)sym,
+
+syscall_handler_t syscall_table[__NR_syscalls] = {
+	[0 ... __NR_syscalls - 1] =  (syscall_handler_t)sys_ni_syscall,
+#undef __UM_NOMMU_UAPI_UNISTD_H
+#include <asm/unistd.h>
+
+#if __BITS_PER_LONG == 32
+#include <asm/unistd_32.h>
+#endif
+};
+
+
+static long run_syscall(long no, long *params)
+{
+	long ret;
+
+	if (no < 0 || no >= __NR_syscalls)
+		return -ENOSYS;
+
+	ret = syscall_table[no](params[0], params[1], params[2], params[3],
+				params[4], params[5]);
+
+	task_work_run();
+
+	return ret;
+}
+
+
+#define CLONE_FLAGS (CLONE_VM | CLONE_FS | CLONE_FILES | CLONE_THREAD |	\
+		     CLONE_SIGHAND | SIGCHLD)
+
+static int host_task_id;
+static struct task_struct *host0;
+
+static int new_host_task(struct task_struct **task)
+{
+	pid_t pid;
+
+	switch_to_host_task(host0);
+
+	pid = kernel_thread(host_task_stub, NULL, CLONE_FLAGS);
+	if (pid < 0)
+		return pid;
+
+	rcu_read_lock();
+	*task = find_task_by_pid_ns(pid, &init_pid_ns);
+	rcu_read_unlock();
+
+	host_task_id++;
+
+	snprintf((*task)->comm, sizeof((*task)->comm), "host%d", host_task_id);
+
+	return 0;
+}
+static void exit_task(void)
+{
+	do_exit(0);
+}
+
+static void del_host_task(void *arg)
+{
+	struct task_struct *task = (struct task_struct *)arg;
+	struct thread_info *ti = task_thread_info(task);
+
+	if (lkl_cpu_get() < 0)
+		return;
+
+	switch_to_host_task(task);
+	host_task_id--;
+	set_ti_thread_flag(ti, TIF_SCHED_JB);
+	lkl_ops->jmp_buf_set(&ti->task->thread.arch.sched_jb, exit_task);
+}
+
+static struct lkl_tls_key *task_key;
+
+long lkl_syscall(long no, long *params)
+{
+	struct task_struct *task = host0;
+	long ret;
+
+	ret = lkl_cpu_get();
+	if (ret < 0)
+		return ret;
+
+	if (lkl_ops->tls_get) {
+		task = lkl_ops->tls_get(task_key);
+		if (!task) {
+			ret = new_host_task(&task);
+			if (ret)
+				goto out;
+			lkl_ops->tls_set(task_key, task);
+		}
+	}
+
+	switch_to_host_task(task);
+
+	ret = run_syscall(no, params);
+
+	if (no == __NR_reboot) {
+		thread_sched_jb();
+		return ret;
+	}
+
+out:
+	lkl_cpu_put();
+
+	return ret;
+}
+
+static struct task_struct *idle_host_task;
+
+/* called from idle, don't failed, don't block */
+void wakeup_idle_host_task(void)
+{
+	if (!need_resched() && idle_host_task)
+		wake_up_process(idle_host_task);
+}
+
+static int idle_host_task_loop(void *unused)
+{
+	struct thread_info *ti = task_thread_info(current);
+
+	snprintf(current->comm, sizeof(current->comm), "idle_host_task");
+	set_thread_flag(TIF_HOST_THREAD);
+	idle_host_task = current;
+
+	for (;;) {
+		lkl_cpu_put();
+		lkl_ops->sem_down(ti->task->thread.arch.sched_sem);
+		if (idle_host_task == NULL) {
+			lkl_ops->thread_exit();
+			return 0;
+		}
+		schedule_tail(ti->task->thread.prev_sched);
+	}
+}
+
+int syscalls_init(void)
+{
+	snprintf(current->comm, sizeof(current->comm), "host0");
+	set_thread_flag(TIF_HOST_THREAD);
+	host0 = current;
+
+	if (lkl_ops->tls_alloc) {
+		task_key = lkl_ops->tls_alloc(del_host_task);
+		if (!task_key)
+			return -1;
+	}
+
+	if (kernel_thread(idle_host_task_loop, NULL, CLONE_FLAGS) < 0) {
+		if (lkl_ops->tls_free)
+			lkl_ops->tls_free(task_key);
+		return -1;
+	}
+
+	return 0;
+}
+
+void syscalls_cleanup(void)
+{
+	if (idle_host_task) {
+		struct thread_info *ti = task_thread_info(idle_host_task);
+
+		idle_host_task = NULL;
+		lkl_ops->sem_up(ti->task->thread.arch.sched_sem);
+		lkl_ops->thread_join(ti->task->thread.arch.tid);
+	}
+
+	if (lkl_ops->tls_free)
+		lkl_ops->tls_free(task_key);
+}
diff --git a/arch/um/scripts/headers_install.py b/arch/um/scripts/headers_install.py
new file mode 100755
index 000000000000..ec7356c0dee9
--- /dev/null
+++ b/arch/um/scripts/headers_install.py
@@ -0,0 +1,197 @@
+#!/usr/bin/env python
+# SPDX-License-Identifier: GPL-2.0
+import re, os, sys, argparse, multiprocessing, fnmatch
+
+srctree = os.environ["srctree"]
+objtree = os.environ["objtree"]
+install_hdr_path = os.environ["INSTALL_HDR_PATH"]
+header_paths = [ install_hdr_path + "/include/", "arch/um/nommu/include/uapi/",
+                 "arch/um/nommu/include/generated/uapi/", "include/generated/" ]
+
+headers = set()
+includes = set()
+
+def relpath2abspath(relpath):
+    if "generated" in relpath or install_hdr_path in relpath:
+        return objtree + "/" + relpath
+    else:
+        return srctree + "/" + relpath
+
+def find_headers(path):
+    headers.add(path)
+    f = open(relpath2abspath(path))
+    for l in f.readlines():
+        m = re.search("#\s*include <(.*)>", l)
+        try:
+            i = m.group(1)
+            for p in header_paths:
+                if os.access(relpath2abspath(p + i), os.R_OK):
+                    if p + i not in headers:
+                        includes.add(i)
+                        headers.add(p + i)
+                        find_headers(p + i)
+        except:
+            pass
+    f.close()
+
+def has_lkl_prefix(w):
+  return w.startswith("lkl") or w.startswith("_lkl") or w.startswith("__lkl") \
+         or w.startswith("LKL") or w.startswith("_LKL") or w.startswith("__LKL")
+
+def find_symbols(regexp, store):
+    for h in headers:
+        f = open(h)
+        for l in f.readlines():
+            m = regexp.search(l)
+            if not m:
+                continue
+            for e in reversed(m.groups()):
+                if e:
+                    if not has_lkl_prefix(e):
+                        store.add(e)
+                    break
+        f.close()
+
+def find_ml_symbols(regexp, store):
+    for h in headers:
+        for i in regexp.finditer(open(h).read()):
+            for j in reversed(i.groups()):
+                if j:
+                    if not has_lkl_prefix(j):
+                        store.add(j)
+                    break
+
+def find_enums(block_regexp, symbol_regexp, store):
+    for h in headers:
+        # remove comments
+        content = re.sub(re.compile("(\/\*(\*(?!\/)|[^*])*\*\/)", re.S|re.M), " ", open(h).read())
+        # remove preprocesor lines
+        clean_content = ""
+        for l in content.split("\n"):
+            if re.match("\s*#", l):
+                continue
+            clean_content += l + "\n"
+        for i in block_regexp.finditer(clean_content):
+            for j in reversed(i.groups()):
+                if j:
+                    for k in symbol_regexp.finditer(j):
+                        for l in k.groups():
+                            if l:
+                                if not has_lkl_prefix(l):
+                                    store.add(l)
+                                break
+
+def lkl_prefix(w):
+    r = ""
+
+    if w.startswith("__"):
+        r = "__"
+    elif w.startswith("_"):
+        r = "_"
+
+    if w.isupper():
+        r += "LKL"
+    else:
+        r += "lkl"
+
+    if not w.startswith("_"):
+        r += "_"
+
+    r += w
+
+    return r
+
+def replace(h):
+    content = open(h).read()
+    for i in includes:
+        search_str = "(#[ \t]*include[ \t]*[<\"][ \t]*)" + i + "([ \t]*[>\"])"
+        replace_str = "\\1" + "lkl/" + i + "\\2"
+        content = re.sub(search_str, replace_str, content)
+    tmp = ""
+    for w in re.split("(\W+)", content):
+        if w in defines:
+            w = lkl_prefix(w)
+        tmp += w
+    content = tmp
+    for s in structs:
+        search_str = "(\W?struct\s+)" + s + "(\W)"
+        replace_str = "\\1" + lkl_prefix(s) + "\\2"
+        content = re.sub(search_str, replace_str, content, flags = re.MULTILINE)
+    for s in unions:
+        search_str = "(\W?union\s+)" + s + "(\W)"
+        replace_str = "\\1" + lkl_prefix(s) + "\\2"
+        content = re.sub(search_str, replace_str, content, flags = re.MULTILINE)
+    open(h, 'w').write(content)
+
+parser = argparse.ArgumentParser(description='install lkl headers')
+parser.add_argument('path', help='path to install to', )
+parser.add_argument('-j', '--jobs', help='number of parallel jobs', default=1, type=int)
+args = parser.parse_args()
+
+find_headers("arch/um/nommu/include/uapi/asm/syscalls.h")
+headers.add("arch/um/nommu/include/uapi/asm/host_ops.h")
+
+if 'LKL_INSTALL_ADDITIONAL_HEADERS' in os.environ:
+    with open(os.environ['LKL_INSTALL_ADDITIONAL_HEADERS'], 'rU') as f:
+        for line in f.readlines():
+            line = line.split('#', 1)[0].strip()
+            if line != '':
+                headers.add(line)
+
+new_headers = set()
+
+for h in headers:
+    h = relpath2abspath(h)
+    dir = os.path.dirname(h)
+    out_dir = args.path + "/" + re.sub("(" + srctree + "/arch/um/nommu/include/uapi/|arch/um/nommu/include/generated/uapi/|include/generated/uapi/|include/generated|" + install_hdr_path + "/include/)(.*)", "lkl/\\2", dir)
+    try:
+        os.makedirs(out_dir)
+    except:
+        pass
+    print("  INSTALL\t%s" % (out_dir + "/" + os.path.basename(h)))
+    os.system(srctree+"/scripts/headers_install.sh %s %s" % (os.path.abspath(h),
+                                                       out_dir + "/" + os.path.basename(h)))
+    new_headers.add(out_dir + "/" + os.path.basename(h))
+
+headers = new_headers
+
+defines = set()
+structs = set()
+unions = set()
+
+p = re.compile("#[ \t]*define[ \t]*(\w+)")
+find_symbols(p, defines)
+p = re.compile("typedef.*(\(\*(\w+)\)\(.*\)\s*|\W+(\w+)\s*|\s+(\w+)\(.*\)\s*);")
+find_symbols(p, defines)
+p = re.compile("typedef\s+(struct|union)\s+\w*\s*{[^\\{\}]*}\W*(\w+)\s*;", re.M|re.S)
+find_ml_symbols(p, defines)
+defines.add("siginfo_t")
+defines.add("sigevent_t")
+p = re.compile("struct\s+(\w+)\s*\{")
+find_symbols(p, structs)
+structs.add("iovec")
+p = re.compile("union\s+(\w+)\s*\{")
+find_symbols(p, unions)
+p = re.compile("static\s+__inline__(\s+\w+)+\s+(\w+)\([^)]*\)\s")
+find_symbols(p, defines)
+p = re.compile("static\s+__always_inline(\s+\w+)+\s+(\w+)\([^)]*\)\s")
+find_symbols(p, defines)
+p = re.compile("enum\s+(\w*)\s*{([^}]*)}", re.M|re.S)
+q = re.compile("(\w+)\s*(,|=[^,]*|$)", re.M|re.S)
+find_enums(p, q, defines)
+
+# needed for i386
+defines.add("__NR_stime")
+
+def process_header(h):
+    print("  REPLACE\t%s" % (out_dir + "/" + os.path.basename(h)))
+    replace(h)
+
+p = multiprocessing.Pool(args.jobs)
+try:
+    p.map_async(process_header, headers).wait(999999)
+    p.close()
+except:
+    p.terminate()
+finally:
+    p.join()
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v6 13/21] um: nommu: basic console support
  2020-09-24  7:12       ` Hajime Tazaki
@ 2020-09-24  7:12         ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  7:12 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch, Hajime Tazaki

This commit also disables stdio-console of UM if !CONFIG_MMU to avoid
conflicts between multiple consoles.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/drivers/Makefile                  |  8 ++++-
 arch/um/nommu/include/uapi/asm/host_ops.h |  3 ++
 arch/um/nommu/um/console.c                | 42 +++++++++++++++++++++++
 3 files changed, 52 insertions(+), 1 deletion(-)
 create mode 100644 arch/um/nommu/um/console.c

diff --git a/arch/um/drivers/Makefile b/arch/um/drivers/Makefile
index ae96c83e312d..694da4aa550d 100644
--- a/arch/um/drivers/Makefile
+++ b/arch/um/drivers/Makefile
@@ -37,7 +37,13 @@ $(obj)/vde.o: $(obj)/vde_kern.o $(obj)/vde_user.o
 # When the above is fixed, don't forget to add this too!
 #targets += $(obj)/pcap.o
 
-obj-y := stdio_console.o fd.o chan_kern.o chan_user.o line.o
+ifdef CONFIG_MMU
+obj-y := stdio_console.o
+else
+obj-y :=
+endif
+obj-y += fd.o chan_kern.o chan_user.o line.o
+
 obj-$(CONFIG_SSL) += ssl.o
 obj-$(CONFIG_STDERR_CONSOLE) += stderr_console.o
 
diff --git a/arch/um/nommu/include/uapi/asm/host_ops.h b/arch/um/nommu/include/uapi/asm/host_ops.h
index 6ee9489fc47e..133877c857a1 100644
--- a/arch/um/nommu/include/uapi/asm/host_ops.h
+++ b/arch/um/nommu/include/uapi/asm/host_ops.h
@@ -15,6 +15,7 @@ struct lkl_jmp_buf {
  *
  * These operations must be provided by a host library or by the application
  * itself.
+ * @print - optional operation that receives console messages
  *
  * @sem_alloc - allocate a host semaphore an initialize it to count
  * @sem_free - free a host semaphore
@@ -63,6 +64,8 @@ struct lkl_jmp_buf {
  *
  */
 struct lkl_host_operations {
+	void (*print)(const char *str, int len);
+
 	struct lkl_sem *(*sem_alloc)(int count);
 	void (*sem_free)(struct lkl_sem *sem);
 	void (*sem_up)(struct lkl_sem *sem);
diff --git a/arch/um/nommu/um/console.c b/arch/um/nommu/um/console.c
new file mode 100644
index 000000000000..e3194f7bb0dd
--- /dev/null
+++ b/arch/um/nommu/um/console.c
@@ -0,0 +1,42 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/console.h>
+#include <asm/host_ops.h>
+
+static void console_write(struct console *con, const char *str, unsigned len)
+{
+	if (lkl_ops->print)
+		lkl_ops->print(str, len);
+}
+
+#ifdef CONFIG_LKL_EARLY_CONSOLE
+static struct console lkl_boot_console = {
+	.name	= "lkl_boot_console",
+	.write	= console_write,
+	.flags	= CON_PRINTBUFFER | CON_BOOT,
+	.index	= -1,
+};
+
+int __init lkl_boot_console_init(void)
+{
+	register_console(&lkl_boot_console);
+	return 0;
+}
+early_initcall(lkl_boot_console_init);
+#endif
+
+static struct console lkl_console = {
+	.name	= "lkl_console",
+	.write	= console_write,
+	.flags	= CON_PRINTBUFFER,
+	.index	= -1,
+};
+
+int __init lkl_console_init(void)
+{
+	register_console(&lkl_console);
+	return 0;
+}
+core_initcall(lkl_console_init);
+
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v6 13/21] um: nommu: basic console support
@ 2020-09-24  7:12         ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  7:12 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Hajime Tazaki, retrage01

This commit also disables stdio-console of UM if !CONFIG_MMU to avoid
conflicts between multiple consoles.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/drivers/Makefile                  |  8 ++++-
 arch/um/nommu/include/uapi/asm/host_ops.h |  3 ++
 arch/um/nommu/um/console.c                | 42 +++++++++++++++++++++++
 3 files changed, 52 insertions(+), 1 deletion(-)
 create mode 100644 arch/um/nommu/um/console.c

diff --git a/arch/um/drivers/Makefile b/arch/um/drivers/Makefile
index ae96c83e312d..694da4aa550d 100644
--- a/arch/um/drivers/Makefile
+++ b/arch/um/drivers/Makefile
@@ -37,7 +37,13 @@ $(obj)/vde.o: $(obj)/vde_kern.o $(obj)/vde_user.o
 # When the above is fixed, don't forget to add this too!
 #targets += $(obj)/pcap.o
 
-obj-y := stdio_console.o fd.o chan_kern.o chan_user.o line.o
+ifdef CONFIG_MMU
+obj-y := stdio_console.o
+else
+obj-y :=
+endif
+obj-y += fd.o chan_kern.o chan_user.o line.o
+
 obj-$(CONFIG_SSL) += ssl.o
 obj-$(CONFIG_STDERR_CONSOLE) += stderr_console.o
 
diff --git a/arch/um/nommu/include/uapi/asm/host_ops.h b/arch/um/nommu/include/uapi/asm/host_ops.h
index 6ee9489fc47e..133877c857a1 100644
--- a/arch/um/nommu/include/uapi/asm/host_ops.h
+++ b/arch/um/nommu/include/uapi/asm/host_ops.h
@@ -15,6 +15,7 @@ struct lkl_jmp_buf {
  *
  * These operations must be provided by a host library or by the application
  * itself.
+ * @print - optional operation that receives console messages
  *
  * @sem_alloc - allocate a host semaphore an initialize it to count
  * @sem_free - free a host semaphore
@@ -63,6 +64,8 @@ struct lkl_jmp_buf {
  *
  */
 struct lkl_host_operations {
+	void (*print)(const char *str, int len);
+
 	struct lkl_sem *(*sem_alloc)(int count);
 	void (*sem_free)(struct lkl_sem *sem);
 	void (*sem_up)(struct lkl_sem *sem);
diff --git a/arch/um/nommu/um/console.c b/arch/um/nommu/um/console.c
new file mode 100644
index 000000000000..e3194f7bb0dd
--- /dev/null
+++ b/arch/um/nommu/um/console.c
@@ -0,0 +1,42 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/console.h>
+#include <asm/host_ops.h>
+
+static void console_write(struct console *con, const char *str, unsigned len)
+{
+	if (lkl_ops->print)
+		lkl_ops->print(str, len);
+}
+
+#ifdef CONFIG_LKL_EARLY_CONSOLE
+static struct console lkl_boot_console = {
+	.name	= "lkl_boot_console",
+	.write	= console_write,
+	.flags	= CON_PRINTBUFFER | CON_BOOT,
+	.index	= -1,
+};
+
+int __init lkl_boot_console_init(void)
+{
+	register_console(&lkl_boot_console);
+	return 0;
+}
+early_initcall(lkl_boot_console_init);
+#endif
+
+static struct console lkl_console = {
+	.name	= "lkl_console",
+	.write	= console_write,
+	.flags	= CON_PRINTBUFFER,
+	.index	= -1,
+};
+
+int __init lkl_console_init(void)
+{
+	register_console(&lkl_console);
+	return 0;
+}
+core_initcall(lkl_console_init);
+
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v6 14/21] um: nommu: initialization and cleanup
  2020-09-24  7:12       ` Hajime Tazaki
@ 2020-09-24  7:12         ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  7:12 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch, Hajime Tazaki

Add the lkl_start_kernel and lkl_sys_halt APIs that start and
respectively stops the Linux kernel.

lkl_start_kernel creates a separate threads that will run the initial
and idle kernel thread. It waits for the kernel to complete
initialization before returning, to avoid races with system calls
issues by the host application.

During the setup phase, we create "/init" in initial ramfs root
filesystem to avoid mounting the "real" rootfs since ramfs is good
enough for now.

lkl_sys_halt will shutdown the kernel, terminate all threads and
free all host resources used by the kernel before returning.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/nommu/include/uapi/asm/host_ops.h |  14 ++
 arch/um/nommu/um/setup.c                  | 162 ++++++++++++++++++++++
 tools/um/uml/process.c                    |   2 +
 3 files changed, 178 insertions(+)
 create mode 100644 arch/um/nommu/um/setup.c

diff --git a/arch/um/nommu/include/uapi/asm/host_ops.h b/arch/um/nommu/include/uapi/asm/host_ops.h
index 133877c857a1..0811494c080c 100644
--- a/arch/um/nommu/include/uapi/asm/host_ops.h
+++ b/arch/um/nommu/include/uapi/asm/host_ops.h
@@ -16,6 +16,7 @@ struct lkl_jmp_buf {
  * These operations must be provided by a host library or by the application
  * itself.
  * @print - optional operation that receives console messages
+ * @panic - called during a kernel panic
  *
  * @sem_alloc - allocate a host semaphore an initialize it to count
  * @sem_free - free a host semaphore
@@ -65,6 +66,7 @@ struct lkl_jmp_buf {
  */
 struct lkl_host_operations {
 	void (*print)(const char *str, int len);
+	void (*panic)(void);
 
 	struct lkl_sem *(*sem_alloc)(int count);
 	void (*sem_free)(struct lkl_sem *sem);
@@ -96,6 +98,18 @@ struct lkl_host_operations {
 	void (*mem_free)(void *mem);
 };
 
+/**
+ * lkl_start_kernel - registers the host operations and starts the kernel
+ *
+ * The function returns only after the kernel is shutdown with lkl_sys_halt.
+ *
+ * @lkl_ops - pointer to host operations
+ * @cmd_line - format for command line string that is going to be used to
+ * generate the Linux kernel command line
+ */
+int lkl_start_kernel(struct lkl_host_operations *ops,
+		     const char *fmt, ...);
+
 void lkl_bug(const char *fmt, ...);
 
 #endif
diff --git a/arch/um/nommu/um/setup.c b/arch/um/nommu/um/setup.c
new file mode 100644
index 000000000000..eaa74d771f73
--- /dev/null
+++ b/arch/um/nommu/um/setup.c
@@ -0,0 +1,162 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/binfmts.h>
+#include <linux/init.h>
+#include <linux/init_task.h>
+#include <linux/personality.h>
+#include <linux/reboot.h>
+#include <linux/fs.h>
+#include <linux/start_kernel.h>
+#include <linux/syscalls.h>
+#include <linux/tick.h>
+#include <linux/kernel.h>
+#include <linux/console.h>
+#include <asm/host_ops.h>
+#include <asm/irq.h>
+#include <asm/unistd.h>
+#include <asm/syscalls.h>
+#include <asm/cpu.h>
+#include <os.h>
+#include <as-layout.h>
+
+
+static void *init_sem;
+static int is_running;
+
+
+struct lkl_host_operations *lkl_ops;
+
+long lkl_panic_blink(int state)
+{
+	lkl_ops->panic();
+	return 0;
+}
+
+static void __init *lkl_run_kernel(void *arg)
+{
+
+	panic_blink = lkl_panic_blink;
+
+	threads_init();
+	lkl_cpu_get();
+	start_kernel();
+
+	return NULL;
+}
+
+int __init lkl_start_kernel(struct lkl_host_operations *ops,
+			    const char *fmt, ...)
+{
+	int ret;
+
+	lkl_ops = ops;
+
+	init_sem = lkl_ops->sem_alloc(0);
+	if (!init_sem)
+		return -ENOMEM;
+
+	ret = lkl_cpu_init();
+	if (ret)
+		goto out_free_init_sem;
+
+	ret = lkl_ops->thread_create(lkl_run_kernel, NULL);
+	if (!ret) {
+		ret = -ENOMEM;
+		goto out_free_init_sem;
+	}
+
+	lkl_ops->sem_down(init_sem);
+	lkl_ops->sem_free(init_sem);
+	current_thread_info()->task->thread.arch.tid = lkl_ops->thread_self();
+	lkl_cpu_change_owner(current_thread_info()->task->thread.arch.tid);
+
+	lkl_cpu_put();
+	is_running = 1;
+
+	return 0;
+
+out_free_init_sem:
+	lkl_ops->sem_free(init_sem);
+
+	return ret;
+}
+
+int lkl_is_running(void)
+{
+	return is_running;
+}
+
+
+long lkl_sys_halt(void)
+{
+	long err;
+	long params[6] = {LINUX_REBOOT_MAGIC1,
+		LINUX_REBOOT_MAGIC2, LINUX_REBOOT_CMD_RESTART, };
+
+	err = lkl_syscall(__NR_reboot, params);
+	if (err < 0)
+		return err;
+
+	is_running = false;
+
+	lkl_cpu_wait_shutdown();
+
+	syscalls_cleanup();
+	threads_cleanup();
+	/* Shutdown the clockevents source. */
+	tick_suspend_local();
+	free_mem();
+	lkl_ops->thread_join(current_thread_info()->task->thread.arch.tid);
+
+	return 0;
+}
+
+
+static int lkl_run_init(struct linux_binprm *bprm);
+
+static struct linux_binfmt lkl_run_init_binfmt = {
+	.module		= THIS_MODULE,
+	.load_binary	= lkl_run_init,
+};
+
+static int lkl_run_init(struct linux_binprm *bprm)
+{
+	int ret;
+
+	if (strcmp("/init", bprm->filename) != 0)
+		return -EINVAL;
+
+	ret = begin_new_exec(bprm);
+	if (ret)
+		return ret;
+	set_personality(PER_LINUX);
+	setup_new_exec(bprm);
+
+	set_binfmt(&lkl_run_init_binfmt);
+
+	init_pid_ns.child_reaper = 0;
+
+	syscalls_init();
+
+	lkl_ops->sem_up(init_sem);
+	lkl_ops->thread_exit();
+
+	return 0;
+}
+
+
+/* skip mounting the "real" rootfs. ramfs is good enough. */
+static int __init fs_setup(void)
+{
+	int fd, flags;
+
+	if (force_o_largefile())
+		flags |= O_LARGEFILE;
+	fd = do_sys_open(AT_FDCWD, "/init", O_CREAT | flags, 0700);
+	WARN_ON(fd < 0);
+	ksys_close(fd);
+
+	register_binfmt(&lkl_run_init_binfmt);
+
+	return 0;
+}
+late_initcall(fs_setup);
diff --git a/tools/um/uml/process.c b/tools/um/uml/process.c
index e52dd37ddadc..c39d914e80ac 100644
--- a/tools/um/uml/process.c
+++ b/tools/um/uml/process.c
@@ -114,6 +114,8 @@ void os_kill_process(int pid, int reap_child)
 
 void os_kill_ptraced_process(int pid, int reap_child)
 {
+	if (pid == 0)
+		return;
 	kill(pid, SIGKILL);
 	ptrace(PTRACE_KILL, pid);
 	ptrace(PTRACE_CONT, pid);
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v6 14/21] um: nommu: initialization and cleanup
@ 2020-09-24  7:12         ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  7:12 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Hajime Tazaki, retrage01

Add the lkl_start_kernel and lkl_sys_halt APIs that start and
respectively stops the Linux kernel.

lkl_start_kernel creates a separate threads that will run the initial
and idle kernel thread. It waits for the kernel to complete
initialization before returning, to avoid races with system calls
issues by the host application.

During the setup phase, we create "/init" in initial ramfs root
filesystem to avoid mounting the "real" rootfs since ramfs is good
enough for now.

lkl_sys_halt will shutdown the kernel, terminate all threads and
free all host resources used by the kernel before returning.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/nommu/include/uapi/asm/host_ops.h |  14 ++
 arch/um/nommu/um/setup.c                  | 162 ++++++++++++++++++++++
 tools/um/uml/process.c                    |   2 +
 3 files changed, 178 insertions(+)
 create mode 100644 arch/um/nommu/um/setup.c

diff --git a/arch/um/nommu/include/uapi/asm/host_ops.h b/arch/um/nommu/include/uapi/asm/host_ops.h
index 133877c857a1..0811494c080c 100644
--- a/arch/um/nommu/include/uapi/asm/host_ops.h
+++ b/arch/um/nommu/include/uapi/asm/host_ops.h
@@ -16,6 +16,7 @@ struct lkl_jmp_buf {
  * These operations must be provided by a host library or by the application
  * itself.
  * @print - optional operation that receives console messages
+ * @panic - called during a kernel panic
  *
  * @sem_alloc - allocate a host semaphore an initialize it to count
  * @sem_free - free a host semaphore
@@ -65,6 +66,7 @@ struct lkl_jmp_buf {
  */
 struct lkl_host_operations {
 	void (*print)(const char *str, int len);
+	void (*panic)(void);
 
 	struct lkl_sem *(*sem_alloc)(int count);
 	void (*sem_free)(struct lkl_sem *sem);
@@ -96,6 +98,18 @@ struct lkl_host_operations {
 	void (*mem_free)(void *mem);
 };
 
+/**
+ * lkl_start_kernel - registers the host operations and starts the kernel
+ *
+ * The function returns only after the kernel is shutdown with lkl_sys_halt.
+ *
+ * @lkl_ops - pointer to host operations
+ * @cmd_line - format for command line string that is going to be used to
+ * generate the Linux kernel command line
+ */
+int lkl_start_kernel(struct lkl_host_operations *ops,
+		     const char *fmt, ...);
+
 void lkl_bug(const char *fmt, ...);
 
 #endif
diff --git a/arch/um/nommu/um/setup.c b/arch/um/nommu/um/setup.c
new file mode 100644
index 000000000000..eaa74d771f73
--- /dev/null
+++ b/arch/um/nommu/um/setup.c
@@ -0,0 +1,162 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/binfmts.h>
+#include <linux/init.h>
+#include <linux/init_task.h>
+#include <linux/personality.h>
+#include <linux/reboot.h>
+#include <linux/fs.h>
+#include <linux/start_kernel.h>
+#include <linux/syscalls.h>
+#include <linux/tick.h>
+#include <linux/kernel.h>
+#include <linux/console.h>
+#include <asm/host_ops.h>
+#include <asm/irq.h>
+#include <asm/unistd.h>
+#include <asm/syscalls.h>
+#include <asm/cpu.h>
+#include <os.h>
+#include <as-layout.h>
+
+
+static void *init_sem;
+static int is_running;
+
+
+struct lkl_host_operations *lkl_ops;
+
+long lkl_panic_blink(int state)
+{
+	lkl_ops->panic();
+	return 0;
+}
+
+static void __init *lkl_run_kernel(void *arg)
+{
+
+	panic_blink = lkl_panic_blink;
+
+	threads_init();
+	lkl_cpu_get();
+	start_kernel();
+
+	return NULL;
+}
+
+int __init lkl_start_kernel(struct lkl_host_operations *ops,
+			    const char *fmt, ...)
+{
+	int ret;
+
+	lkl_ops = ops;
+
+	init_sem = lkl_ops->sem_alloc(0);
+	if (!init_sem)
+		return -ENOMEM;
+
+	ret = lkl_cpu_init();
+	if (ret)
+		goto out_free_init_sem;
+
+	ret = lkl_ops->thread_create(lkl_run_kernel, NULL);
+	if (!ret) {
+		ret = -ENOMEM;
+		goto out_free_init_sem;
+	}
+
+	lkl_ops->sem_down(init_sem);
+	lkl_ops->sem_free(init_sem);
+	current_thread_info()->task->thread.arch.tid = lkl_ops->thread_self();
+	lkl_cpu_change_owner(current_thread_info()->task->thread.arch.tid);
+
+	lkl_cpu_put();
+	is_running = 1;
+
+	return 0;
+
+out_free_init_sem:
+	lkl_ops->sem_free(init_sem);
+
+	return ret;
+}
+
+int lkl_is_running(void)
+{
+	return is_running;
+}
+
+
+long lkl_sys_halt(void)
+{
+	long err;
+	long params[6] = {LINUX_REBOOT_MAGIC1,
+		LINUX_REBOOT_MAGIC2, LINUX_REBOOT_CMD_RESTART, };
+
+	err = lkl_syscall(__NR_reboot, params);
+	if (err < 0)
+		return err;
+
+	is_running = false;
+
+	lkl_cpu_wait_shutdown();
+
+	syscalls_cleanup();
+	threads_cleanup();
+	/* Shutdown the clockevents source. */
+	tick_suspend_local();
+	free_mem();
+	lkl_ops->thread_join(current_thread_info()->task->thread.arch.tid);
+
+	return 0;
+}
+
+
+static int lkl_run_init(struct linux_binprm *bprm);
+
+static struct linux_binfmt lkl_run_init_binfmt = {
+	.module		= THIS_MODULE,
+	.load_binary	= lkl_run_init,
+};
+
+static int lkl_run_init(struct linux_binprm *bprm)
+{
+	int ret;
+
+	if (strcmp("/init", bprm->filename) != 0)
+		return -EINVAL;
+
+	ret = begin_new_exec(bprm);
+	if (ret)
+		return ret;
+	set_personality(PER_LINUX);
+	setup_new_exec(bprm);
+
+	set_binfmt(&lkl_run_init_binfmt);
+
+	init_pid_ns.child_reaper = 0;
+
+	syscalls_init();
+
+	lkl_ops->sem_up(init_sem);
+	lkl_ops->thread_exit();
+
+	return 0;
+}
+
+
+/* skip mounting the "real" rootfs. ramfs is good enough. */
+static int __init fs_setup(void)
+{
+	int fd, flags;
+
+	if (force_o_largefile())
+		flags |= O_LARGEFILE;
+	fd = do_sys_open(AT_FDCWD, "/init", O_CREAT | flags, 0700);
+	WARN_ON(fd < 0);
+	ksys_close(fd);
+
+	register_binfmt(&lkl_run_init_binfmt);
+
+	return 0;
+}
+late_initcall(fs_setup);
diff --git a/tools/um/uml/process.c b/tools/um/uml/process.c
index e52dd37ddadc..c39d914e80ac 100644
--- a/tools/um/uml/process.c
+++ b/tools/um/uml/process.c
@@ -114,6 +114,8 @@ void os_kill_process(int pid, int reap_child)
 
 void os_kill_ptraced_process(int pid, int reap_child)
 {
+	if (pid == 0)
+		return;
 	kill(pid, SIGKILL);
 	ptrace(PTRACE_KILL, pid);
 	ptrace(PTRACE_CONT, pid);
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v6 15/21] um: nommu: integrate with irq infrastructure of UML
  2020-09-24  7:12       ` Hajime Tazaki
@ 2020-09-24  7:12         ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  7:12 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch, Hajime Tazaki

In order to cooperate with UML's irq infrastructure and nommu threads
based on host threads, irq handlers shall synchronize the our
scheduler which is controlled by struct lkl_cpu.  To do that, the irq
infra notifies its entry of handlers by obtaining cpu access of thread
scheduler (lkl_cpu_try_run_irq) and its release (lkl_cpu_put).

In additon to that, in order to stick the signal handler's thread to the
one of the idle thread, several required configurations of (thread's)
signal mask are added: otherwise handlers running on arbitrary thread
cannot obtain cpu access and immediately fall into pending interrupt
which may slow down the delivery of signals.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/include/shared/os.h     |  1 +
 arch/um/kernel/irq.c            | 13 +++++++++++++
 arch/um/kernel/time.c           |  2 ++
 arch/um/nommu/include/asm/cpu.h |  3 +++
 arch/um/nommu/um/cpu.c          | 17 ++++++++++++++---
 arch/um/nommu/um/setup.c        |  5 +++++
 arch/um/nommu/um/threads.c      |  3 +++
 tools/um/uml/signal.c           | 12 ++++++++++--
 8 files changed, 51 insertions(+), 5 deletions(-)

diff --git a/arch/um/include/shared/os.h b/arch/um/include/shared/os.h
index f467d28fc0b4..0da3ce22e16f 100644
--- a/arch/um/include/shared/os.h
+++ b/arch/um/include/shared/os.h
@@ -241,6 +241,7 @@ extern int set_signals(int enable);
 extern int set_signals_trace(int enable);
 extern int os_is_signal_stack(void);
 extern void deliver_alarm(void);
+extern void set_pending_signals(int sig);
 
 /* util.c */
 extern void stack_protections(unsigned long address);
diff --git a/arch/um/kernel/irq.c b/arch/um/kernel/irq.c
index 3577118bb4a5..4ce7f5fef7f8 100644
--- a/arch/um/kernel/irq.c
+++ b/arch/um/kernel/irq.c
@@ -19,6 +19,7 @@
 #include <kern_util.h>
 #include <os.h>
 #include <irq_user.h>
+#include <asm/cpu.h>
 
 
 extern void free_irqs(void);
@@ -541,6 +542,11 @@ unsigned long to_irq_stack(unsigned long *mask_out)
 	unsigned long mask, old;
 	int nested;
 
+#ifndef CONFIG_MMU
+	if (!nommu_irq_enter(ffs(*mask_out) - 1))
+		return 1;
+#endif
+
 	mask = xchg(&pending_mask, *mask_out);
 	if (mask != 0) {
 		/*
@@ -557,6 +563,10 @@ unsigned long to_irq_stack(unsigned long *mask_out)
 			old |= mask;
 			mask = xchg(&pending_mask, old);
 		} while (mask != old);
+
+#ifndef CONFIG_MMU
+		nommu_irq_exit();
+#endif
 		return 1;
 	}
 
@@ -594,6 +604,9 @@ unsigned long from_irq_stack(int nested)
 	*to = *ti;
 
 	mask = xchg(&pending_mask, 0);
+#ifndef CONFIG_MMU
+	nommu_irq_exit();
+#endif
 	return mask & ~1;
 }
 
diff --git a/arch/um/kernel/time.c b/arch/um/kernel/time.c
index 25eaa6a0c658..9e141fe8fe27 100644
--- a/arch/um/kernel/time.c
+++ b/arch/um/kernel/time.c
@@ -597,11 +597,13 @@ static struct clock_event_device timer_clockevent = {
 
 static irqreturn_t um_timer(int irq, void *dev)
 {
+#ifdef CONFIG_MMU
 	if (get_current()->mm != NULL)
 	{
         /* userspace - relay signal, results in correct userspace timers */
 		os_alarm_process(get_current()->mm->context.id.u.pid);
 	}
+#endif
 
 	(*timer_clockevent.event_handler)(&timer_clockevent);
 
diff --git a/arch/um/nommu/include/asm/cpu.h b/arch/um/nommu/include/asm/cpu.h
index c101c078ef21..c52a78c77bea 100644
--- a/arch/um/nommu/include/asm/cpu.h
+++ b/arch/um/nommu/include/asm/cpu.h
@@ -10,4 +10,7 @@ void lkl_cpu_wait_shutdown(void);
 void lkl_cpu_change_owner(lkl_thread_t owner);
 void lkl_cpu_set_irqs_pending(void);
 
+void nommu_irq_exit(void);
+int nommu_irq_enter(int sig);
+
 #endif
diff --git a/arch/um/nommu/um/cpu.c b/arch/um/nommu/um/cpu.c
index 9986a3f8c5dd..75d8bdc1847e 100644
--- a/arch/um/nommu/um/cpu.c
+++ b/arch/um/nommu/um/cpu.c
@@ -8,15 +8,16 @@
 #include <asm/sched.h>
 #include <asm/syscalls.h>
 #include <init.h>
+#include <os.h>
 
 void run_irqs(void)
 {
-	panic("unimplemented %s", __func__);
+	unblock_signals();
 }
 
-int set_irq_pending(int irq)
+void set_irq_pending(int sig)
 {
-	panic("unimplemented %s", __func__);
+	set_pending_signals(sig);
 }
 
 /*
@@ -174,6 +175,16 @@ int lkl_cpu_try_run_irq(int irq)
 	return ret;
 }
 
+int nommu_irq_enter(int sig)
+{
+	return lkl_cpu_try_run_irq(sig);
+}
+
+void nommu_irq_exit(void)
+{
+	return lkl_cpu_put();
+}
+
 static void lkl_cpu_shutdown(void)
 {
 	__sync_fetch_and_add(&cpu.shutdown_gate, MAX_THREADS);
diff --git a/arch/um/nommu/um/setup.c b/arch/um/nommu/um/setup.c
index eaa74d771f73..922188690139 100644
--- a/arch/um/nommu/um/setup.c
+++ b/arch/um/nommu/um/setup.c
@@ -36,6 +36,8 @@ static void __init *lkl_run_kernel(void *arg)
 
 	panic_blink = lkl_panic_blink;
 
+	/* signal should be received at this thread (main and idle threads) */
+	init_new_thread_signals();
 	threads_init();
 	lkl_cpu_get();
 	start_kernel();
@@ -58,6 +60,9 @@ int __init lkl_start_kernel(struct lkl_host_operations *ops,
 	if (ret)
 		goto out_free_init_sem;
 
+	change_sig(SIGALRM, 0);
+	change_sig(SIGIO, 0);
+
 	ret = lkl_ops->thread_create(lkl_run_kernel, NULL);
 	if (!ret) {
 		ret = -ENOMEM;
diff --git a/arch/um/nommu/um/threads.c b/arch/um/nommu/um/threads.c
index 3e70eccc191a..df96057100fd 100644
--- a/arch/um/nommu/um/threads.c
+++ b/arch/um/nommu/um/threads.c
@@ -151,6 +151,9 @@ static void *thread_bootstrap(void *_tba)
 	int (*f)(void *) = tba->f;
 	void *arg = tba->arg;
 
+	change_sig(SIGALRM, 0);
+	change_sig(SIGIO, 0);
+
 	lkl_ops->sem_down(ti->task->thread.arch.sched_sem);
 	kfree(tba);
 	if (ti->task->thread.prev_sched)
diff --git a/tools/um/uml/signal.c b/tools/um/uml/signal.c
index b58bc68cbe64..2b5c98de21cd 100644
--- a/tools/um/uml/signal.c
+++ b/tools/um/uml/signal.c
@@ -218,8 +218,8 @@ void set_handler(int sig)
 
 	sigemptyset(&sig_mask);
 	sigaddset(&sig_mask, sig);
-	if (sigprocmask(SIG_UNBLOCK, &sig_mask, NULL) < 0)
-		panic("sigprocmask failed - errno = %d\n", errno);
+	if (pthread_sigmask(SIG_UNBLOCK, &sig_mask, NULL) < 0)
+		panic("pthread_sigmask failed - errno = %d\n", errno);
 }
 
 int change_sig(int signal, int on)
@@ -355,3 +355,11 @@ int os_is_signal_stack(void)
 
 	return ss.ss_flags & SS_ONSTACK;
 }
+
+void set_pending_signals(int sig)
+{
+	if (sig == SIGIO)
+		signals_pending |= SIGIO_MASK;
+	else if (sig == SIGALRM)
+		signals_pending |= SIGALRM_MASK;
+}
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v6 15/21] um: nommu: integrate with irq infrastructure of UML
@ 2020-09-24  7:12         ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  7:12 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Hajime Tazaki, retrage01

In order to cooperate with UML's irq infrastructure and nommu threads
based on host threads, irq handlers shall synchronize the our
scheduler which is controlled by struct lkl_cpu.  To do that, the irq
infra notifies its entry of handlers by obtaining cpu access of thread
scheduler (lkl_cpu_try_run_irq) and its release (lkl_cpu_put).

In additon to that, in order to stick the signal handler's thread to the
one of the idle thread, several required configurations of (thread's)
signal mask are added: otherwise handlers running on arbitrary thread
cannot obtain cpu access and immediately fall into pending interrupt
which may slow down the delivery of signals.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/include/shared/os.h     |  1 +
 arch/um/kernel/irq.c            | 13 +++++++++++++
 arch/um/kernel/time.c           |  2 ++
 arch/um/nommu/include/asm/cpu.h |  3 +++
 arch/um/nommu/um/cpu.c          | 17 ++++++++++++++---
 arch/um/nommu/um/setup.c        |  5 +++++
 arch/um/nommu/um/threads.c      |  3 +++
 tools/um/uml/signal.c           | 12 ++++++++++--
 8 files changed, 51 insertions(+), 5 deletions(-)

diff --git a/arch/um/include/shared/os.h b/arch/um/include/shared/os.h
index f467d28fc0b4..0da3ce22e16f 100644
--- a/arch/um/include/shared/os.h
+++ b/arch/um/include/shared/os.h
@@ -241,6 +241,7 @@ extern int set_signals(int enable);
 extern int set_signals_trace(int enable);
 extern int os_is_signal_stack(void);
 extern void deliver_alarm(void);
+extern void set_pending_signals(int sig);
 
 /* util.c */
 extern void stack_protections(unsigned long address);
diff --git a/arch/um/kernel/irq.c b/arch/um/kernel/irq.c
index 3577118bb4a5..4ce7f5fef7f8 100644
--- a/arch/um/kernel/irq.c
+++ b/arch/um/kernel/irq.c
@@ -19,6 +19,7 @@
 #include <kern_util.h>
 #include <os.h>
 #include <irq_user.h>
+#include <asm/cpu.h>
 
 
 extern void free_irqs(void);
@@ -541,6 +542,11 @@ unsigned long to_irq_stack(unsigned long *mask_out)
 	unsigned long mask, old;
 	int nested;
 
+#ifndef CONFIG_MMU
+	if (!nommu_irq_enter(ffs(*mask_out) - 1))
+		return 1;
+#endif
+
 	mask = xchg(&pending_mask, *mask_out);
 	if (mask != 0) {
 		/*
@@ -557,6 +563,10 @@ unsigned long to_irq_stack(unsigned long *mask_out)
 			old |= mask;
 			mask = xchg(&pending_mask, old);
 		} while (mask != old);
+
+#ifndef CONFIG_MMU
+		nommu_irq_exit();
+#endif
 		return 1;
 	}
 
@@ -594,6 +604,9 @@ unsigned long from_irq_stack(int nested)
 	*to = *ti;
 
 	mask = xchg(&pending_mask, 0);
+#ifndef CONFIG_MMU
+	nommu_irq_exit();
+#endif
 	return mask & ~1;
 }
 
diff --git a/arch/um/kernel/time.c b/arch/um/kernel/time.c
index 25eaa6a0c658..9e141fe8fe27 100644
--- a/arch/um/kernel/time.c
+++ b/arch/um/kernel/time.c
@@ -597,11 +597,13 @@ static struct clock_event_device timer_clockevent = {
 
 static irqreturn_t um_timer(int irq, void *dev)
 {
+#ifdef CONFIG_MMU
 	if (get_current()->mm != NULL)
 	{
         /* userspace - relay signal, results in correct userspace timers */
 		os_alarm_process(get_current()->mm->context.id.u.pid);
 	}
+#endif
 
 	(*timer_clockevent.event_handler)(&timer_clockevent);
 
diff --git a/arch/um/nommu/include/asm/cpu.h b/arch/um/nommu/include/asm/cpu.h
index c101c078ef21..c52a78c77bea 100644
--- a/arch/um/nommu/include/asm/cpu.h
+++ b/arch/um/nommu/include/asm/cpu.h
@@ -10,4 +10,7 @@ void lkl_cpu_wait_shutdown(void);
 void lkl_cpu_change_owner(lkl_thread_t owner);
 void lkl_cpu_set_irqs_pending(void);
 
+void nommu_irq_exit(void);
+int nommu_irq_enter(int sig);
+
 #endif
diff --git a/arch/um/nommu/um/cpu.c b/arch/um/nommu/um/cpu.c
index 9986a3f8c5dd..75d8bdc1847e 100644
--- a/arch/um/nommu/um/cpu.c
+++ b/arch/um/nommu/um/cpu.c
@@ -8,15 +8,16 @@
 #include <asm/sched.h>
 #include <asm/syscalls.h>
 #include <init.h>
+#include <os.h>
 
 void run_irqs(void)
 {
-	panic("unimplemented %s", __func__);
+	unblock_signals();
 }
 
-int set_irq_pending(int irq)
+void set_irq_pending(int sig)
 {
-	panic("unimplemented %s", __func__);
+	set_pending_signals(sig);
 }
 
 /*
@@ -174,6 +175,16 @@ int lkl_cpu_try_run_irq(int irq)
 	return ret;
 }
 
+int nommu_irq_enter(int sig)
+{
+	return lkl_cpu_try_run_irq(sig);
+}
+
+void nommu_irq_exit(void)
+{
+	return lkl_cpu_put();
+}
+
 static void lkl_cpu_shutdown(void)
 {
 	__sync_fetch_and_add(&cpu.shutdown_gate, MAX_THREADS);
diff --git a/arch/um/nommu/um/setup.c b/arch/um/nommu/um/setup.c
index eaa74d771f73..922188690139 100644
--- a/arch/um/nommu/um/setup.c
+++ b/arch/um/nommu/um/setup.c
@@ -36,6 +36,8 @@ static void __init *lkl_run_kernel(void *arg)
 
 	panic_blink = lkl_panic_blink;
 
+	/* signal should be received at this thread (main and idle threads) */
+	init_new_thread_signals();
 	threads_init();
 	lkl_cpu_get();
 	start_kernel();
@@ -58,6 +60,9 @@ int __init lkl_start_kernel(struct lkl_host_operations *ops,
 	if (ret)
 		goto out_free_init_sem;
 
+	change_sig(SIGALRM, 0);
+	change_sig(SIGIO, 0);
+
 	ret = lkl_ops->thread_create(lkl_run_kernel, NULL);
 	if (!ret) {
 		ret = -ENOMEM;
diff --git a/arch/um/nommu/um/threads.c b/arch/um/nommu/um/threads.c
index 3e70eccc191a..df96057100fd 100644
--- a/arch/um/nommu/um/threads.c
+++ b/arch/um/nommu/um/threads.c
@@ -151,6 +151,9 @@ static void *thread_bootstrap(void *_tba)
 	int (*f)(void *) = tba->f;
 	void *arg = tba->arg;
 
+	change_sig(SIGALRM, 0);
+	change_sig(SIGIO, 0);
+
 	lkl_ops->sem_down(ti->task->thread.arch.sched_sem);
 	kfree(tba);
 	if (ti->task->thread.prev_sched)
diff --git a/tools/um/uml/signal.c b/tools/um/uml/signal.c
index b58bc68cbe64..2b5c98de21cd 100644
--- a/tools/um/uml/signal.c
+++ b/tools/um/uml/signal.c
@@ -218,8 +218,8 @@ void set_handler(int sig)
 
 	sigemptyset(&sig_mask);
 	sigaddset(&sig_mask, sig);
-	if (sigprocmask(SIG_UNBLOCK, &sig_mask, NULL) < 0)
-		panic("sigprocmask failed - errno = %d\n", errno);
+	if (pthread_sigmask(SIG_UNBLOCK, &sig_mask, NULL) < 0)
+		panic("pthread_sigmask failed - errno = %d\n", errno);
 }
 
 int change_sig(int signal, int on)
@@ -355,3 +355,11 @@ int os_is_signal_stack(void)
 
 	return ss.ss_flags & SS_ONSTACK;
 }
+
+void set_pending_signals(int sig)
+{
+	if (sig == SIGIO)
+		signals_pending |= SIGIO_MASK;
+	else if (sig == SIGALRM)
+		signals_pending |= SIGALRM_MASK;
+}
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v6 16/21] um: nommu: plug in the build system
  2020-09-24  7:12       ` Hajime Tazaki
@ 2020-09-24  7:12         ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  7:12 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch, Hajime Tazaki

Basic Makefiles for building library of nommu mode. Add a new architecture
specific target for installing the resulting library files and headers.

To make nommu binaries build, UML introduced an additional option, UMMODE
variable, to switch the output file of build: kernel (default), or
library (nommu).  Those modes are not able to be ON at the same time.

To build on library mode, users do the following:

  make defconfig ARCH=um UMMODE=library
  make ARCH=um UMMODE=library

Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/Kconfig           | 17 ++++++++++++++---
 arch/um/Makefile          | 10 +++++++++-
 arch/um/kernel/Makefile   | 12 ++++++++----
 arch/um/nommu/um/Kconfig  | 37 +++++++++++++++++++++++++++++++++++++
 arch/um/nommu/um/Makefile |  4 ++++
 5 files changed, 72 insertions(+), 8 deletions(-)
 create mode 100644 arch/um/nommu/um/Kconfig
 create mode 100644 arch/um/nommu/um/Makefile

diff --git a/arch/um/Kconfig b/arch/um/Kconfig
index e41d31d0d875..df7daceb095c 100644
--- a/arch/um/Kconfig
+++ b/arch/um/Kconfig
@@ -22,9 +22,20 @@ config UML
 	select TTY # Needed for line.c
 	select MODULE_REL_CRCS if MODVERSIONS
 
+config UMMODE_LIB
+	bool "UML mode: library mode"
+	default y if "$(UMMODE)" = "library"
+	help
+	 This mode switches a mode to build a library of UML (Linux
+	 Kernel Library/LKL).  This switch is exclusive to "kernel mode"
+	 of UML, which is traditional mode of UML.
+
+	 For more detail about LKL, see
+	 <file:Documentation/virt/uml/lkl.txt>.
+
 config MMU
 	bool
-	default y
+	default y if !UMMODE_LIB
 
 config NO_IOMEM
 	def_bool y
@@ -45,12 +56,12 @@ config LOCKDEP_SUPPORT
 
 config STACKTRACE_SUPPORT
 	bool
-	default y
+	default y if MMU
 	select STACKTRACE
 
 config GENERIC_CALIBRATE_DELAY
 	bool
-	default y
+	default y if MMU
 
 config HZ
 	int
diff --git a/arch/um/Makefile b/arch/um/Makefile
index 1f958a9280fe..73d3ac061b77 100644
--- a/arch/um/Makefile
+++ b/arch/um/Makefile
@@ -17,6 +17,10 @@ else
         KBUILD_DEFCONFIG := $(SUBARCH)_defconfig
 endif
 
+ifeq ($(UMMODE),library)
+  SUBARCH := um/nommu
+endif
+
 ARCH_DIR := arch/um
 OS := $(shell uname -s)
 # We require bash because the vmlinux link and loader script cpp use bash
@@ -102,9 +106,13 @@ all: linux.o
 
 linux.o: vmlinux
 	@echo '  LINK $@'
-	$(Q)$(OBJCOPY) -R .eh_frame $< $@
+	$(Q)$(OBJCOPY) -R .eh_frame -L sem_init -L sem_post -L sem_wait -L sem_destroy $< $@
 
+ifeq ($(UMMODE),library)
+install: linux.o um_headers_install
+else
 install: linux.o
+endif
 	@echo "  INSTALL $(INSTALL_PATH)/lib/$<"
 	@mkdir -p $(INSTALL_PATH)/lib/
 	@cp $< $(INSTALL_PATH)/lib/
diff --git a/arch/um/kernel/Makefile b/arch/um/kernel/Makefile
index 5aa882011e04..bdc4108bc52e 100644
--- a/arch/um/kernel/Makefile
+++ b/arch/um/kernel/Makefile
@@ -14,10 +14,14 @@ CPPFLAGS_vmlinux.lds := -DSTART=$(LDS_START)		\
 			$(LDS_EXTRA)
 extra-y := vmlinux.lds
 
-obj-y = config.o exec.o exitcode.o irq.o ksyms.o mem.o \
-	physmem.o process.o ptrace.o reboot.o sigio.o \
-	signal.o syscall.o sysrq.o time.o tlb.o trap.o \
-	um_arch.o umid.o maccess.o kmsg_dump.o skas/
+obj-y = config.o exitcode.o irq.o ksyms.o \
+	process.o reboot.o sigio.o \
+	signal.o syscall.o time.o \
+	um_arch.o umid.o maccess.o kmsg_dump.o
+
+ifdef CONFIG_MMU
+obj-y += exec.o mem.o physmem.o ptrace.o sysrq.o tlb.o trap.o skas/
+endif
 
 obj-$(CONFIG_BLK_DEV_INITRD) += initrd.o
 obj-$(CONFIG_GPROF)	+= gprof_syms.o
diff --git a/arch/um/nommu/um/Kconfig b/arch/um/nommu/um/Kconfig
new file mode 100644
index 000000000000..20b3eaccb6f0
--- /dev/null
+++ b/arch/um/nommu/um/Kconfig
@@ -0,0 +1,37 @@
+config UML_NOMMU
+	def_bool y
+	depends on !SMP && !MMU
+	select UACCESS_MEMCPY
+	select ARCH_THREAD_STACK_ALLOCATOR
+	select ARCH_HAS_SYSCALL_WRAPPER
+
+config 64BIT
+	bool
+	default y
+
+config GENERIC_CSUM
+	def_bool y
+
+config GENERIC_ATOMIC64
+	bool
+	default y if !64BIT
+
+config SECCOMP
+	bool
+	default n
+
+config GENERIC_HWEIGHT
+	def_bool y
+
+config GENERIC_CALIBRATE_DELAY
+	bool
+	default n
+
+config STACKTRACE_SUPPORT
+	bool
+	default n
+
+# XXX: need this to work well with tap13.py
+config PRINTK_TIME
+	bool
+	default y
diff --git a/arch/um/nommu/um/Makefile b/arch/um/nommu/um/Makefile
new file mode 100644
index 000000000000..b580e0fe97b5
--- /dev/null
+++ b/arch/um/nommu/um/Makefile
@@ -0,0 +1,4 @@
+include/generated/user_constants.h: $(srctree)/arch/um/nommu/um/user_constants.h
+	$(Q)cp -f $^ $@
+
+obj-y = bootmem.o console.o cpu.o delay.o setup.o syscalls.o threads.o unimplemented.o
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v6 16/21] um: nommu: plug in the build system
@ 2020-09-24  7:12         ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  7:12 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Hajime Tazaki, retrage01

Basic Makefiles for building library of nommu mode. Add a new architecture
specific target for installing the resulting library files and headers.

To make nommu binaries build, UML introduced an additional option, UMMODE
variable, to switch the output file of build: kernel (default), or
library (nommu).  Those modes are not able to be ON at the same time.

To build on library mode, users do the following:

  make defconfig ARCH=um UMMODE=library
  make ARCH=um UMMODE=library

Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/Kconfig           | 17 ++++++++++++++---
 arch/um/Makefile          | 10 +++++++++-
 arch/um/kernel/Makefile   | 12 ++++++++----
 arch/um/nommu/um/Kconfig  | 37 +++++++++++++++++++++++++++++++++++++
 arch/um/nommu/um/Makefile |  4 ++++
 5 files changed, 72 insertions(+), 8 deletions(-)
 create mode 100644 arch/um/nommu/um/Kconfig
 create mode 100644 arch/um/nommu/um/Makefile

diff --git a/arch/um/Kconfig b/arch/um/Kconfig
index e41d31d0d875..df7daceb095c 100644
--- a/arch/um/Kconfig
+++ b/arch/um/Kconfig
@@ -22,9 +22,20 @@ config UML
 	select TTY # Needed for line.c
 	select MODULE_REL_CRCS if MODVERSIONS
 
+config UMMODE_LIB
+	bool "UML mode: library mode"
+	default y if "$(UMMODE)" = "library"
+	help
+	 This mode switches a mode to build a library of UML (Linux
+	 Kernel Library/LKL).  This switch is exclusive to "kernel mode"
+	 of UML, which is traditional mode of UML.
+
+	 For more detail about LKL, see
+	 <file:Documentation/virt/uml/lkl.txt>.
+
 config MMU
 	bool
-	default y
+	default y if !UMMODE_LIB
 
 config NO_IOMEM
 	def_bool y
@@ -45,12 +56,12 @@ config LOCKDEP_SUPPORT
 
 config STACKTRACE_SUPPORT
 	bool
-	default y
+	default y if MMU
 	select STACKTRACE
 
 config GENERIC_CALIBRATE_DELAY
 	bool
-	default y
+	default y if MMU
 
 config HZ
 	int
diff --git a/arch/um/Makefile b/arch/um/Makefile
index 1f958a9280fe..73d3ac061b77 100644
--- a/arch/um/Makefile
+++ b/arch/um/Makefile
@@ -17,6 +17,10 @@ else
         KBUILD_DEFCONFIG := $(SUBARCH)_defconfig
 endif
 
+ifeq ($(UMMODE),library)
+  SUBARCH := um/nommu
+endif
+
 ARCH_DIR := arch/um
 OS := $(shell uname -s)
 # We require bash because the vmlinux link and loader script cpp use bash
@@ -102,9 +106,13 @@ all: linux.o
 
 linux.o: vmlinux
 	@echo '  LINK $@'
-	$(Q)$(OBJCOPY) -R .eh_frame $< $@
+	$(Q)$(OBJCOPY) -R .eh_frame -L sem_init -L sem_post -L sem_wait -L sem_destroy $< $@
 
+ifeq ($(UMMODE),library)
+install: linux.o um_headers_install
+else
 install: linux.o
+endif
 	@echo "  INSTALL $(INSTALL_PATH)/lib/$<"
 	@mkdir -p $(INSTALL_PATH)/lib/
 	@cp $< $(INSTALL_PATH)/lib/
diff --git a/arch/um/kernel/Makefile b/arch/um/kernel/Makefile
index 5aa882011e04..bdc4108bc52e 100644
--- a/arch/um/kernel/Makefile
+++ b/arch/um/kernel/Makefile
@@ -14,10 +14,14 @@ CPPFLAGS_vmlinux.lds := -DSTART=$(LDS_START)		\
 			$(LDS_EXTRA)
 extra-y := vmlinux.lds
 
-obj-y = config.o exec.o exitcode.o irq.o ksyms.o mem.o \
-	physmem.o process.o ptrace.o reboot.o sigio.o \
-	signal.o syscall.o sysrq.o time.o tlb.o trap.o \
-	um_arch.o umid.o maccess.o kmsg_dump.o skas/
+obj-y = config.o exitcode.o irq.o ksyms.o \
+	process.o reboot.o sigio.o \
+	signal.o syscall.o time.o \
+	um_arch.o umid.o maccess.o kmsg_dump.o
+
+ifdef CONFIG_MMU
+obj-y += exec.o mem.o physmem.o ptrace.o sysrq.o tlb.o trap.o skas/
+endif
 
 obj-$(CONFIG_BLK_DEV_INITRD) += initrd.o
 obj-$(CONFIG_GPROF)	+= gprof_syms.o
diff --git a/arch/um/nommu/um/Kconfig b/arch/um/nommu/um/Kconfig
new file mode 100644
index 000000000000..20b3eaccb6f0
--- /dev/null
+++ b/arch/um/nommu/um/Kconfig
@@ -0,0 +1,37 @@
+config UML_NOMMU
+	def_bool y
+	depends on !SMP && !MMU
+	select UACCESS_MEMCPY
+	select ARCH_THREAD_STACK_ALLOCATOR
+	select ARCH_HAS_SYSCALL_WRAPPER
+
+config 64BIT
+	bool
+	default y
+
+config GENERIC_CSUM
+	def_bool y
+
+config GENERIC_ATOMIC64
+	bool
+	default y if !64BIT
+
+config SECCOMP
+	bool
+	default n
+
+config GENERIC_HWEIGHT
+	def_bool y
+
+config GENERIC_CALIBRATE_DELAY
+	bool
+	default n
+
+config STACKTRACE_SUPPORT
+	bool
+	default n
+
+# XXX: need this to work well with tap13.py
+config PRINTK_TIME
+	bool
+	default y
diff --git a/arch/um/nommu/um/Makefile b/arch/um/nommu/um/Makefile
new file mode 100644
index 000000000000..b580e0fe97b5
--- /dev/null
+++ b/arch/um/nommu/um/Makefile
@@ -0,0 +1,4 @@
+include/generated/user_constants.h: $(srctree)/arch/um/nommu/um/user_constants.h
+	$(Q)cp -f $^ $@
+
+obj-y = bootmem.o console.o cpu.o delay.o setup.o syscalls.o threads.o unimplemented.o
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v6 17/21] um: host: add nommu build for ARCH=um
  2020-09-24  7:12       ` Hajime Tazaki
@ 2020-09-24  7:12         ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  7:12 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch, Hajime Tazaki

This patch adds the skeleton for the host library.

The host library is implementing the host operations needed by nommu
mode and is split into host dependent (depends on a specific host, e.g.
POSIX hosts) and host independent parts (will work on all supported
hosts).

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/um/Makefile                  |   7 +-
 tools/um/Targets                   |   3 +-
 tools/um/include/lkl.h             | 151 +++++++++++++++++++++++++++++
 tools/um/include/lkl_host.h        |  26 +++++
 tools/um/uml/Build                 |  15 ++-
 tools/um/uml/nommu/Build           |   1 +
 tools/um/uml/nommu/registers.c     |  21 ++++
 tools/um/uml/nommu/unimplemented.c |  21 ++++
 8 files changed, 238 insertions(+), 7 deletions(-)
 create mode 100644 tools/um/include/lkl.h
 create mode 100644 tools/um/include/lkl_host.h
 create mode 100644 tools/um/uml/nommu/Build
 create mode 100644 tools/um/uml/nommu/registers.c
 create mode 100644 tools/um/uml/nommu/unimplemented.c

diff --git a/tools/um/Makefile b/tools/um/Makefile
index 552ed5f1edae..a93e060b1f89 100644
--- a/tools/um/Makefile
+++ b/tools/um/Makefile
@@ -5,6 +5,7 @@
 MAKEFLAGS += -r --no-print-directory
 
 KCONFIG?=defconfig
+UMMODE?=kernel
 
 ifneq ($(silent),1)
   ifneq ($(V),1)
@@ -47,8 +48,9 @@ all: $(TARGETS)
 
 # rule to build linux.o
 $(OUTPUT)lib/linux.o:
-	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) $(KCONFIG)
-	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) install INSTALL_PATH=$(OUTPUT)
+	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um UMMODE=$(UMMODE) $(KOPT) $(KCONFIG)
+	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um UMMODE=$(UMMODE) $(KOPT) install \
+	INSTALL_PATH=$(OUTPUT)
 
 $(OUTPUT)liblinux.a: $(OUTPUT)lib/linux.o $(OUTPUT)uml/liblinux-in.o $(OUTPUT)lib/liblinux-in.o
 	$(QUIET_AR)$(AR) -rc $@ $^
@@ -67,6 +69,7 @@ $(OUTPUT)%-in.o: $(OUTPUT)lib/linux.o FORCE
 clean:
 	$(call QUIET_CLEAN, objects)find $(OUTPUT) -name '*.o' -delete -o -name '\.*.cmd'\
 	 -delete -o -name '\.*.d' -delete
+	$(call QUIET_CLEAN, headers)$(RM) -r $(OUTPUT)/include/lkl/
 	$(call QUIET_CLEAN, liblinux.a)$(RM) $(OUTPUT)/liblinux.a
 	$(call QUIET_CLEAN, $(TARGETS))$(RM) $(TARGETS)
 
diff --git a/tools/um/Targets b/tools/um/Targets
index a4711f1ef422..4e1f0f4d81a3 100644
--- a/tools/um/Targets
+++ b/tools/um/Targets
@@ -1,4 +1,5 @@
 progs-y += uml/linux
-LDLIBS_linux-y := -lrt -lpthread -lutil
+
+LDLIBS := -lrt -lpthread -lutil
 LDFLAGS_linux-y := -no-pie -Wl,--wrap,malloc -Wl,--wrap,free -Wl,--wrap,calloc
 LDFLAGS_linux-$(UML_STATIC) += -static
diff --git a/tools/um/include/lkl.h b/tools/um/include/lkl.h
new file mode 100644
index 000000000000..707e01b64a70
--- /dev/null
+++ b/tools/um/include/lkl.h
@@ -0,0 +1,151 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_H
+#define _LKL_H
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#define _LKL_LIBC_COMPAT_H
+
+#ifdef __cplusplus
+#define class __lkl__class
+#endif
+
+/*
+ * Avoid collisions between Android which defines __unused and
+ * linux/icmp.h which uses __unused as a structure field.
+ */
+#pragma push_macro("__unused")
+#undef __unused
+
+#include <lkl/asm/syscalls.h>
+
+#pragma pop_macro("__unused")
+
+#ifdef __cplusplus
+#undef class
+#endif
+
+/**
+ * lkl_strerror - returns a string describing the given error code
+ *
+ * @err - error code
+ * @returns - string for the given error code
+ */
+const char *lkl_strerror(int err);
+
+/**
+ * lkl_perror - prints a string describing the given error code
+ *
+ * @msg - prefix for the error message
+ * @err - error code
+ */
+void lkl_perror(char *msg, int err);
+
+#if __LKL__BITS_PER_LONG == 64
+#define lkl_sys_fstatat lkl_sys_newfstatat
+#define lkl_sys_fstat lkl_sys_newfstat
+
+#else
+#define __lkl__NR_fcntl __lkl__NR_fcntl64
+
+#define lkl_stat lkl_stat64
+#define lkl_sys_stat lkl_sys_stat64
+#define lkl_sys_lstat lkl_sys_lstat64
+#define lkl_sys_truncate lkl_sys_truncate64
+#define lkl_sys_ftruncate lkl_sys_ftruncate64
+#define lkl_sys_sendfile lkl_sys_sendfile64
+#define lkl_sys_fstatat lkl_sys_fstatat64
+#define lkl_sys_fstat lkl_sys_fstat64
+#define lkl_sys_fcntl lkl_sys_fcntl64
+
+#define lkl_statfs lkl_statfs64
+
+static inline int lkl_sys_statfs(const char *path, struct lkl_statfs *buf)
+{
+	return lkl_sys_statfs64(path, sizeof(*buf), buf);
+}
+
+static inline int lkl_sys_fstatfs(unsigned int fd, struct lkl_statfs *buf)
+{
+	return lkl_sys_fstatfs64(fd, sizeof(*buf), buf);
+}
+
+#define lkl_sys_nanosleep lkl_sys_nanosleep_time32
+static inline int lkl_sys_nanosleep_time32(struct lkl_timespec *rqtp,
+					   struct lkl_timespec *rmtp)
+{
+	long p[6] = {(long)rqtp, (long)rmtp, 0, 0, 0, 0};
+
+	return lkl_syscall(__lkl__NR_nanosleep, p);
+}
+
+#endif
+
+static inline int lkl_sys_stat(const char *path, struct lkl_stat *buf)
+{
+	return lkl_sys_fstatat(LKL_AT_FDCWD, path, buf, 0);
+}
+
+static inline int lkl_sys_lstat(const char *path, struct lkl_stat *buf)
+{
+	return lkl_sys_fstatat(LKL_AT_FDCWD, path, buf,
+			       LKL_AT_SYMLINK_NOFOLLOW);
+}
+
+#ifdef __lkl__NR_openat
+/**
+ * lkl_sys_open - wrapper for lkl_sys_openat
+ */
+static inline long lkl_sys_open(const char *file, int flags, int mode)
+{
+	return lkl_sys_openat(LKL_AT_FDCWD, file, flags, mode);
+}
+
+/**
+ * lkl_sys_creat - wrapper for lkl_sys_openat
+ */
+static inline long lkl_sys_creat(const char *file, int mode)
+{
+	return lkl_sys_openat(LKL_AT_FDCWD, file,
+			      LKL_O_CREAT|LKL_O_WRONLY|LKL_O_TRUNC, mode);
+}
+#endif
+
+#ifdef __lkl__NR_mkdirat
+/**
+ * lkl_sys_mkdir - wrapper for lkl_sys_mkdirat
+ */
+static inline long lkl_sys_mkdir(const char *path, mode_t mode)
+{
+	return lkl_sys_mkdirat(LKL_AT_FDCWD, path, mode);
+}
+#endif
+
+#ifdef __lkl__NR_epoll_create1
+/**
+ * lkl_sys_epoll_create - wrapper for lkl_sys_epoll_create1
+ */
+static inline long lkl_sys_epoll_create(int size)
+{
+	return lkl_sys_epoll_create1(0);
+}
+#endif
+
+#ifdef __lkl__NR_epoll_pwait
+/**
+ * lkl_sys_epoll_wait - wrapper for lkl_sys_epoll_pwait
+ */
+static inline long lkl_sys_epoll_wait(int fd, struct lkl_epoll_event *ev,
+				      int cnt, int to)
+{
+	return lkl_sys_epoll_pwait(fd, ev, cnt, to, 0, _LKL_NSIG/8);
+}
+#endif
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif
diff --git a/tools/um/include/lkl_host.h b/tools/um/include/lkl_host.h
new file mode 100644
index 000000000000..85e80eb4ad0d
--- /dev/null
+++ b/tools/um/include/lkl_host.h
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_HOST_H
+#define _LKL_HOST_H
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include <lkl/asm/host_ops.h>
+#include <lkl.h>
+
+extern struct lkl_host_operations lkl_host_ops;
+
+/**
+ * lkl_printf - print a message via the host print operation
+ *
+ * @fmt: printf like format string
+ */
+int lkl_printf(const char *fmt, ...);
+
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif
diff --git a/tools/um/uml/Build b/tools/um/uml/Build
index 435a9670d3ff..191c6479ad72 100644
--- a/tools/um/uml/Build
+++ b/tools/um/uml/Build
@@ -8,11 +8,19 @@ KCOV_INSTRUMENT                := n
 
 include $(objtree)/include/config/auto.conf
 
-liblinux-y += execvp.o file.o helper.o irq.o main.o mem.o process.o \
-	registers.o sigio.o signal.o start_up.o time.o tty.o \
-	umid.o user_syms.o util.o drivers/ skas/
+liblinux-y += execvp.o file.o helper.o irq.o mem.o process.o \
+	registers.o sigio.o signal.o time.o tty.o \
+	umid.o user_syms.o util.o drivers/
 
+ifdef CONFIG_MMU
+liblinux-y += main.o start_up.o skas/
+HEADER_ARCH 	:= x86
 liblinux-y += x86/
+else
+HEADER_ARCH 	:= um/nommu
+liblinux-y += nommu/
+endif
+
 
 liblinux-$(CONFIG_ARCH_REUSE_HOST_VSYSCALL_AREA) += elf_aux.o
 
@@ -23,7 +31,6 @@ CFLAGS := -g -O2
 
 # from arch/um/Makefile
 ARCH_DIR := arch/um
-HEADER_ARCH 	:= x86
 HOST_DIR := arch/$(HEADER_ARCH)
 ifdef CONFIG_64BIT
   KBUILD_CFLAGS += -mcmodel=large
diff --git a/tools/um/uml/nommu/Build b/tools/um/uml/nommu/Build
new file mode 100644
index 000000000000..98fd6b86a085
--- /dev/null
+++ b/tools/um/uml/nommu/Build
@@ -0,0 +1 @@
+liblinux-y = registers.o unimplemented.o
diff --git a/tools/um/uml/nommu/registers.c b/tools/um/uml/nommu/registers.c
new file mode 100644
index 000000000000..11573a204720
--- /dev/null
+++ b/tools/um/uml/nommu/registers.c
@@ -0,0 +1,21 @@
+// SPDX-License-Identifier: GPL-2.0
+
+struct uml_pt_regs;
+
+int get_fp_registers(int pid, unsigned long *regs)
+{
+	return 0;
+}
+
+int save_i387_registers(int pid, unsigned long *fp_regs)
+{
+	return 0;
+}
+
+void arch_init_registers(int pid)
+{
+}
+
+void get_regs_from_mc(struct uml_pt_regs *regs, void *mc)
+{
+}
diff --git a/tools/um/uml/nommu/unimplemented.c b/tools/um/uml/nommu/unimplemented.c
new file mode 100644
index 000000000000..9da3e5c8bafb
--- /dev/null
+++ b/tools/um/uml/nommu/unimplemented.c
@@ -0,0 +1,21 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <generated/user_constants.h>
+
+struct uml_pt_regs;
+
+/* os-Linux/skas/process.c */
+int userspace_pid[UM_NR_CPUS];
+void userspace(struct uml_pt_regs *regs, unsigned long *aux_fp_regs)
+{}
+
+
+/* x86/os-Linux/task_size.c */
+unsigned long os_get_top_address(void)
+{
+	return 0;
+}
+
+/* start-up.c */
+void os_early_checks(void)
+{
+}
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v6 17/21] um: host: add nommu build for ARCH=um
@ 2020-09-24  7:12         ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  7:12 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Hajime Tazaki, retrage01

This patch adds the skeleton for the host library.

The host library is implementing the host operations needed by nommu
mode and is split into host dependent (depends on a specific host, e.g.
POSIX hosts) and host independent parts (will work on all supported
hosts).

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/um/Makefile                  |   7 +-
 tools/um/Targets                   |   3 +-
 tools/um/include/lkl.h             | 151 +++++++++++++++++++++++++++++
 tools/um/include/lkl_host.h        |  26 +++++
 tools/um/uml/Build                 |  15 ++-
 tools/um/uml/nommu/Build           |   1 +
 tools/um/uml/nommu/registers.c     |  21 ++++
 tools/um/uml/nommu/unimplemented.c |  21 ++++
 8 files changed, 238 insertions(+), 7 deletions(-)
 create mode 100644 tools/um/include/lkl.h
 create mode 100644 tools/um/include/lkl_host.h
 create mode 100644 tools/um/uml/nommu/Build
 create mode 100644 tools/um/uml/nommu/registers.c
 create mode 100644 tools/um/uml/nommu/unimplemented.c

diff --git a/tools/um/Makefile b/tools/um/Makefile
index 552ed5f1edae..a93e060b1f89 100644
--- a/tools/um/Makefile
+++ b/tools/um/Makefile
@@ -5,6 +5,7 @@
 MAKEFLAGS += -r --no-print-directory
 
 KCONFIG?=defconfig
+UMMODE?=kernel
 
 ifneq ($(silent),1)
   ifneq ($(V),1)
@@ -47,8 +48,9 @@ all: $(TARGETS)
 
 # rule to build linux.o
 $(OUTPUT)lib/linux.o:
-	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) $(KCONFIG)
-	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) install INSTALL_PATH=$(OUTPUT)
+	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um UMMODE=$(UMMODE) $(KOPT) $(KCONFIG)
+	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um UMMODE=$(UMMODE) $(KOPT) install \
+	INSTALL_PATH=$(OUTPUT)
 
 $(OUTPUT)liblinux.a: $(OUTPUT)lib/linux.o $(OUTPUT)uml/liblinux-in.o $(OUTPUT)lib/liblinux-in.o
 	$(QUIET_AR)$(AR) -rc $@ $^
@@ -67,6 +69,7 @@ $(OUTPUT)%-in.o: $(OUTPUT)lib/linux.o FORCE
 clean:
 	$(call QUIET_CLEAN, objects)find $(OUTPUT) -name '*.o' -delete -o -name '\.*.cmd'\
 	 -delete -o -name '\.*.d' -delete
+	$(call QUIET_CLEAN, headers)$(RM) -r $(OUTPUT)/include/lkl/
 	$(call QUIET_CLEAN, liblinux.a)$(RM) $(OUTPUT)/liblinux.a
 	$(call QUIET_CLEAN, $(TARGETS))$(RM) $(TARGETS)
 
diff --git a/tools/um/Targets b/tools/um/Targets
index a4711f1ef422..4e1f0f4d81a3 100644
--- a/tools/um/Targets
+++ b/tools/um/Targets
@@ -1,4 +1,5 @@
 progs-y += uml/linux
-LDLIBS_linux-y := -lrt -lpthread -lutil
+
+LDLIBS := -lrt -lpthread -lutil
 LDFLAGS_linux-y := -no-pie -Wl,--wrap,malloc -Wl,--wrap,free -Wl,--wrap,calloc
 LDFLAGS_linux-$(UML_STATIC) += -static
diff --git a/tools/um/include/lkl.h b/tools/um/include/lkl.h
new file mode 100644
index 000000000000..707e01b64a70
--- /dev/null
+++ b/tools/um/include/lkl.h
@@ -0,0 +1,151 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_H
+#define _LKL_H
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#define _LKL_LIBC_COMPAT_H
+
+#ifdef __cplusplus
+#define class __lkl__class
+#endif
+
+/*
+ * Avoid collisions between Android which defines __unused and
+ * linux/icmp.h which uses __unused as a structure field.
+ */
+#pragma push_macro("__unused")
+#undef __unused
+
+#include <lkl/asm/syscalls.h>
+
+#pragma pop_macro("__unused")
+
+#ifdef __cplusplus
+#undef class
+#endif
+
+/**
+ * lkl_strerror - returns a string describing the given error code
+ *
+ * @err - error code
+ * @returns - string for the given error code
+ */
+const char *lkl_strerror(int err);
+
+/**
+ * lkl_perror - prints a string describing the given error code
+ *
+ * @msg - prefix for the error message
+ * @err - error code
+ */
+void lkl_perror(char *msg, int err);
+
+#if __LKL__BITS_PER_LONG == 64
+#define lkl_sys_fstatat lkl_sys_newfstatat
+#define lkl_sys_fstat lkl_sys_newfstat
+
+#else
+#define __lkl__NR_fcntl __lkl__NR_fcntl64
+
+#define lkl_stat lkl_stat64
+#define lkl_sys_stat lkl_sys_stat64
+#define lkl_sys_lstat lkl_sys_lstat64
+#define lkl_sys_truncate lkl_sys_truncate64
+#define lkl_sys_ftruncate lkl_sys_ftruncate64
+#define lkl_sys_sendfile lkl_sys_sendfile64
+#define lkl_sys_fstatat lkl_sys_fstatat64
+#define lkl_sys_fstat lkl_sys_fstat64
+#define lkl_sys_fcntl lkl_sys_fcntl64
+
+#define lkl_statfs lkl_statfs64
+
+static inline int lkl_sys_statfs(const char *path, struct lkl_statfs *buf)
+{
+	return lkl_sys_statfs64(path, sizeof(*buf), buf);
+}
+
+static inline int lkl_sys_fstatfs(unsigned int fd, struct lkl_statfs *buf)
+{
+	return lkl_sys_fstatfs64(fd, sizeof(*buf), buf);
+}
+
+#define lkl_sys_nanosleep lkl_sys_nanosleep_time32
+static inline int lkl_sys_nanosleep_time32(struct lkl_timespec *rqtp,
+					   struct lkl_timespec *rmtp)
+{
+	long p[6] = {(long)rqtp, (long)rmtp, 0, 0, 0, 0};
+
+	return lkl_syscall(__lkl__NR_nanosleep, p);
+}
+
+#endif
+
+static inline int lkl_sys_stat(const char *path, struct lkl_stat *buf)
+{
+	return lkl_sys_fstatat(LKL_AT_FDCWD, path, buf, 0);
+}
+
+static inline int lkl_sys_lstat(const char *path, struct lkl_stat *buf)
+{
+	return lkl_sys_fstatat(LKL_AT_FDCWD, path, buf,
+			       LKL_AT_SYMLINK_NOFOLLOW);
+}
+
+#ifdef __lkl__NR_openat
+/**
+ * lkl_sys_open - wrapper for lkl_sys_openat
+ */
+static inline long lkl_sys_open(const char *file, int flags, int mode)
+{
+	return lkl_sys_openat(LKL_AT_FDCWD, file, flags, mode);
+}
+
+/**
+ * lkl_sys_creat - wrapper for lkl_sys_openat
+ */
+static inline long lkl_sys_creat(const char *file, int mode)
+{
+	return lkl_sys_openat(LKL_AT_FDCWD, file,
+			      LKL_O_CREAT|LKL_O_WRONLY|LKL_O_TRUNC, mode);
+}
+#endif
+
+#ifdef __lkl__NR_mkdirat
+/**
+ * lkl_sys_mkdir - wrapper for lkl_sys_mkdirat
+ */
+static inline long lkl_sys_mkdir(const char *path, mode_t mode)
+{
+	return lkl_sys_mkdirat(LKL_AT_FDCWD, path, mode);
+}
+#endif
+
+#ifdef __lkl__NR_epoll_create1
+/**
+ * lkl_sys_epoll_create - wrapper for lkl_sys_epoll_create1
+ */
+static inline long lkl_sys_epoll_create(int size)
+{
+	return lkl_sys_epoll_create1(0);
+}
+#endif
+
+#ifdef __lkl__NR_epoll_pwait
+/**
+ * lkl_sys_epoll_wait - wrapper for lkl_sys_epoll_pwait
+ */
+static inline long lkl_sys_epoll_wait(int fd, struct lkl_epoll_event *ev,
+				      int cnt, int to)
+{
+	return lkl_sys_epoll_pwait(fd, ev, cnt, to, 0, _LKL_NSIG/8);
+}
+#endif
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif
diff --git a/tools/um/include/lkl_host.h b/tools/um/include/lkl_host.h
new file mode 100644
index 000000000000..85e80eb4ad0d
--- /dev/null
+++ b/tools/um/include/lkl_host.h
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_HOST_H
+#define _LKL_HOST_H
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include <lkl/asm/host_ops.h>
+#include <lkl.h>
+
+extern struct lkl_host_operations lkl_host_ops;
+
+/**
+ * lkl_printf - print a message via the host print operation
+ *
+ * @fmt: printf like format string
+ */
+int lkl_printf(const char *fmt, ...);
+
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif
diff --git a/tools/um/uml/Build b/tools/um/uml/Build
index 435a9670d3ff..191c6479ad72 100644
--- a/tools/um/uml/Build
+++ b/tools/um/uml/Build
@@ -8,11 +8,19 @@ KCOV_INSTRUMENT                := n
 
 include $(objtree)/include/config/auto.conf
 
-liblinux-y += execvp.o file.o helper.o irq.o main.o mem.o process.o \
-	registers.o sigio.o signal.o start_up.o time.o tty.o \
-	umid.o user_syms.o util.o drivers/ skas/
+liblinux-y += execvp.o file.o helper.o irq.o mem.o process.o \
+	registers.o sigio.o signal.o time.o tty.o \
+	umid.o user_syms.o util.o drivers/
 
+ifdef CONFIG_MMU
+liblinux-y += main.o start_up.o skas/
+HEADER_ARCH 	:= x86
 liblinux-y += x86/
+else
+HEADER_ARCH 	:= um/nommu
+liblinux-y += nommu/
+endif
+
 
 liblinux-$(CONFIG_ARCH_REUSE_HOST_VSYSCALL_AREA) += elf_aux.o
 
@@ -23,7 +31,6 @@ CFLAGS := -g -O2
 
 # from arch/um/Makefile
 ARCH_DIR := arch/um
-HEADER_ARCH 	:= x86
 HOST_DIR := arch/$(HEADER_ARCH)
 ifdef CONFIG_64BIT
   KBUILD_CFLAGS += -mcmodel=large
diff --git a/tools/um/uml/nommu/Build b/tools/um/uml/nommu/Build
new file mode 100644
index 000000000000..98fd6b86a085
--- /dev/null
+++ b/tools/um/uml/nommu/Build
@@ -0,0 +1 @@
+liblinux-y = registers.o unimplemented.o
diff --git a/tools/um/uml/nommu/registers.c b/tools/um/uml/nommu/registers.c
new file mode 100644
index 000000000000..11573a204720
--- /dev/null
+++ b/tools/um/uml/nommu/registers.c
@@ -0,0 +1,21 @@
+// SPDX-License-Identifier: GPL-2.0
+
+struct uml_pt_regs;
+
+int get_fp_registers(int pid, unsigned long *regs)
+{
+	return 0;
+}
+
+int save_i387_registers(int pid, unsigned long *fp_regs)
+{
+	return 0;
+}
+
+void arch_init_registers(int pid)
+{
+}
+
+void get_regs_from_mc(struct uml_pt_regs *regs, void *mc)
+{
+}
diff --git a/tools/um/uml/nommu/unimplemented.c b/tools/um/uml/nommu/unimplemented.c
new file mode 100644
index 000000000000..9da3e5c8bafb
--- /dev/null
+++ b/tools/um/uml/nommu/unimplemented.c
@@ -0,0 +1,21 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <generated/user_constants.h>
+
+struct uml_pt_regs;
+
+/* os-Linux/skas/process.c */
+int userspace_pid[UM_NR_CPUS];
+void userspace(struct uml_pt_regs *regs, unsigned long *aux_fp_regs)
+{}
+
+
+/* x86/os-Linux/task_size.c */
+unsigned long os_get_top_address(void)
+{
+	return 0;
+}
+
+/* start-up.c */
+void os_early_checks(void)
+{
+}
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v6 18/21] um: host: add utilities functions
  2020-09-24  7:12       ` Hajime Tazaki
@ 2020-09-24  7:12         ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  7:12 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch, Hajime Tazaki

Add basic utility functions for getting a string from a kernel error
code and a fprintf like function that uses the host print
operation. The latter is useful for informing the user about errors
that occur in the host library.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 tools/um/lib/Build     |   4 +
 tools/um/lib/jmp_buf.c |  14 +++
 tools/um/lib/jmp_buf.h |   8 ++
 tools/um/lib/utils.c   | 210 +++++++++++++++++++++++++++++++++++++++++
 4 files changed, 236 insertions(+)
 create mode 100644 tools/um/lib/Build
 create mode 100644 tools/um/lib/jmp_buf.c
 create mode 100644 tools/um/lib/jmp_buf.h
 create mode 100644 tools/um/lib/utils.c

diff --git a/tools/um/lib/Build b/tools/um/lib/Build
new file mode 100644
index 000000000000..fc967af4104a
--- /dev/null
+++ b/tools/um/lib/Build
@@ -0,0 +1,4 @@
+include $(objtree)/include/config/auto.conf
+
+liblinux-$(CONFIG_UMMODE_LIB) += utils.o
+liblinux-$(CONFIG_UMMODE_LIB) += jmp_buf.o
diff --git a/tools/um/lib/jmp_buf.c b/tools/um/lib/jmp_buf.c
new file mode 100644
index 000000000000..f6bdd7e4bd83
--- /dev/null
+++ b/tools/um/lib/jmp_buf.c
@@ -0,0 +1,14 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <setjmp.h>
+#include <lkl_host.h>
+
+void jmp_buf_set(struct lkl_jmp_buf *jmpb, void (*f)(void))
+{
+	if (!setjmp(*((jmp_buf *)jmpb->buf)))
+		f();
+}
+
+void jmp_buf_longjmp(struct lkl_jmp_buf *jmpb, int val)
+{
+	longjmp(*((jmp_buf *)jmpb->buf), val);
+}
diff --git a/tools/um/lib/jmp_buf.h b/tools/um/lib/jmp_buf.h
new file mode 100644
index 000000000000..8782cbaaf51f
--- /dev/null
+++ b/tools/um/lib/jmp_buf.h
@@ -0,0 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_LIB_JMP_BUF_H
+#define _LKL_LIB_JMP_BUF_H
+
+void jmp_buf_set(struct lkl_jmp_buf *jmpb, void (*f)(void));
+void jmp_buf_longjmp(struct lkl_jmp_buf *jmpb, int val);
+
+#endif
diff --git a/tools/um/lib/utils.c b/tools/um/lib/utils.c
new file mode 100644
index 000000000000..4930479a8a35
--- /dev/null
+++ b/tools/um/lib/utils.c
@@ -0,0 +1,210 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdarg.h>
+#include <stdio.h>
+#include <string.h>
+#include <lkl_host.h>
+
+static const char * const lkl_err_strings[] = {
+	"Success",
+	"Operation not permitted",
+	"No such file or directory",
+	"No such process",
+	"Interrupted system call",
+	"I/O error",
+	"No such device or address",
+	"Argument list too long",
+	"Exec format error",
+	"Bad file number",
+	"No child processes",
+	"Try again",
+	"Out of memory",
+	"Permission denied",
+	"Bad address",
+	"Block device required",
+	"Device or resource busy",
+	"File exists",
+	"Cross-device link",
+	"No such device",
+	"Not a directory",
+	"Is a directory",
+	"Invalid argument",
+	"File table overflow",
+	"Too many open files",
+	"Not a typewriter",
+	"Text file busy",
+	"File too large",
+	"No space left on device",
+	"Illegal seek",
+	"Read-only file system",
+	"Too many links",
+	"Broken pipe",
+	"Math argument out of domain of func",
+	"Math result not representable",
+	"Resource deadlock would occur",
+	"File name too long",
+	"No record locks available",
+	"Invalid system call number",
+	"Directory not empty",
+	"Too many symbolic links encountered",
+	"Bad error code", /* EWOULDBLOCK is EAGAIN */
+	"No message of desired type",
+	"Identifier removed",
+	"Channel number out of range",
+	"Level 2 not synchronized",
+	"Level 3 halted",
+	"Level 3 reset",
+	"Link number out of range",
+	"Protocol driver not attached",
+	"No CSI structure available",
+	"Level 2 halted",
+	"Invalid exchange",
+	"Invalid request descriptor",
+	"Exchange full",
+	"No anode",
+	"Invalid request code",
+	"Invalid slot",
+	"Bad error code", /* EDEADLOCK is EDEADLK */
+	"Bad font file format",
+	"Device not a stream",
+	"No data available",
+	"Timer expired",
+	"Out of streams resources",
+	"Machine is not on the network",
+	"Package not installed",
+	"Object is remote",
+	"Link has been severed",
+	"Advertise error",
+	"Srmount error",
+	"Communication error on send",
+	"Protocol error",
+	"Multihop attempted",
+	"RFS specific error",
+	"Not a data message",
+	"Value too large for defined data type",
+	"Name not unique on network",
+	"File descriptor in bad state",
+	"Remote address changed",
+	"Can not access a needed shared library",
+	"Accessing a corrupted shared library",
+	".lib section in a.out corrupted",
+	"Attempting to link in too many shared libraries",
+	"Cannot exec a shared library directly",
+	"Illegal byte sequence",
+	"Interrupted system call should be restarted",
+	"Streams pipe error",
+	"Too many users",
+	"Socket operation on non-socket",
+	"Destination address required",
+	"Message too long",
+	"Protocol wrong type for socket",
+	"Protocol not available",
+	"Protocol not supported",
+	"Socket type not supported",
+	"Operation not supported on transport endpoint",
+	"Protocol family not supported",
+	"Address family not supported by protocol",
+	"Address already in use",
+	"Cannot assign requested address",
+	"Network is down",
+	"Network is unreachable",
+	"Network dropped connection because of reset",
+	"Software caused connection abort",
+	"Connection reset by peer",
+	"No buffer space available",
+	"Transport endpoint is already connected",
+	"Transport endpoint is not connected",
+	"Cannot send after transport endpoint shutdown",
+	"Too many references: cannot splice",
+	"Connection timed out",
+	"Connection refused",
+	"Host is down",
+	"No route to host",
+	"Operation already in progress",
+	"Operation now in progress",
+	"Stale file handle",
+	"Structure needs cleaning",
+	"Not a XENIX named type file",
+	"No XENIX semaphores available",
+	"Is a named type file",
+	"Remote I/O error",
+	"Quota exceeded",
+	"No medium found",
+	"Wrong medium type",
+	"Operation Canceled",
+	"Required key not available",
+	"Key has expired",
+	"Key has been revoked",
+	"Key was rejected by service",
+	"Owner died",
+	"State not recoverable",
+	"Operation not possible due to RF-kill",
+	"Memory page has hardware error",
+};
+
+const char *lkl_strerror(int err)
+{
+	if (err < 0)
+		err = -err;
+
+	if ((size_t)err >= sizeof(lkl_err_strings) / sizeof(const char *))
+		return "Bad error code";
+
+	return lkl_err_strings[err];
+}
+
+void lkl_perror(char *msg, int err)
+{
+	const char *err_msg = lkl_strerror(err);
+	/* We need to use 'real' printf because lkl_host_ops.print can
+	 * be turned off when debugging is off.
+	 */
+	lkl_printf("%s: %s\n", msg, err_msg);
+}
+
+static int lkl_vprintf(const char *fmt, va_list args)
+{
+	int n;
+	char *buffer;
+	va_list copy;
+
+	if (!lkl_host_ops.print)
+		return 0;
+
+	va_copy(copy, args);
+	n = vsnprintf(NULL, 0, fmt, copy);
+	va_end(copy);
+
+	buffer = lkl_host_ops.mem_alloc(n + 1);
+	if (!buffer)
+		return (-1);
+
+	vsnprintf(buffer, n + 1, fmt, args);
+
+	lkl_host_ops.print(buffer, n);
+	lkl_host_ops.mem_free(buffer);
+
+	return n;
+}
+
+int lkl_printf(const char *fmt, ...)
+{
+	int n;
+	va_list args;
+
+	va_start(args, fmt);
+	n = lkl_vprintf(fmt, args);
+	va_end(args);
+
+	return n;
+}
+
+void lkl_bug(const char *fmt, ...)
+{
+	va_list args;
+
+	va_start(args, fmt);
+	lkl_vprintf(fmt, args);
+	va_end(args);
+
+	lkl_host_ops.panic();
+}
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v6 18/21] um: host: add utilities functions
@ 2020-09-24  7:12         ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  7:12 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Hajime Tazaki, retrage01

Add basic utility functions for getting a string from a kernel error
code and a fprintf like function that uses the host print
operation. The latter is useful for informing the user about errors
that occur in the host library.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 tools/um/lib/Build     |   4 +
 tools/um/lib/jmp_buf.c |  14 +++
 tools/um/lib/jmp_buf.h |   8 ++
 tools/um/lib/utils.c   | 210 +++++++++++++++++++++++++++++++++++++++++
 4 files changed, 236 insertions(+)
 create mode 100644 tools/um/lib/Build
 create mode 100644 tools/um/lib/jmp_buf.c
 create mode 100644 tools/um/lib/jmp_buf.h
 create mode 100644 tools/um/lib/utils.c

diff --git a/tools/um/lib/Build b/tools/um/lib/Build
new file mode 100644
index 000000000000..fc967af4104a
--- /dev/null
+++ b/tools/um/lib/Build
@@ -0,0 +1,4 @@
+include $(objtree)/include/config/auto.conf
+
+liblinux-$(CONFIG_UMMODE_LIB) += utils.o
+liblinux-$(CONFIG_UMMODE_LIB) += jmp_buf.o
diff --git a/tools/um/lib/jmp_buf.c b/tools/um/lib/jmp_buf.c
new file mode 100644
index 000000000000..f6bdd7e4bd83
--- /dev/null
+++ b/tools/um/lib/jmp_buf.c
@@ -0,0 +1,14 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <setjmp.h>
+#include <lkl_host.h>
+
+void jmp_buf_set(struct lkl_jmp_buf *jmpb, void (*f)(void))
+{
+	if (!setjmp(*((jmp_buf *)jmpb->buf)))
+		f();
+}
+
+void jmp_buf_longjmp(struct lkl_jmp_buf *jmpb, int val)
+{
+	longjmp(*((jmp_buf *)jmpb->buf), val);
+}
diff --git a/tools/um/lib/jmp_buf.h b/tools/um/lib/jmp_buf.h
new file mode 100644
index 000000000000..8782cbaaf51f
--- /dev/null
+++ b/tools/um/lib/jmp_buf.h
@@ -0,0 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_LIB_JMP_BUF_H
+#define _LKL_LIB_JMP_BUF_H
+
+void jmp_buf_set(struct lkl_jmp_buf *jmpb, void (*f)(void));
+void jmp_buf_longjmp(struct lkl_jmp_buf *jmpb, int val);
+
+#endif
diff --git a/tools/um/lib/utils.c b/tools/um/lib/utils.c
new file mode 100644
index 000000000000..4930479a8a35
--- /dev/null
+++ b/tools/um/lib/utils.c
@@ -0,0 +1,210 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdarg.h>
+#include <stdio.h>
+#include <string.h>
+#include <lkl_host.h>
+
+static const char * const lkl_err_strings[] = {
+	"Success",
+	"Operation not permitted",
+	"No such file or directory",
+	"No such process",
+	"Interrupted system call",
+	"I/O error",
+	"No such device or address",
+	"Argument list too long",
+	"Exec format error",
+	"Bad file number",
+	"No child processes",
+	"Try again",
+	"Out of memory",
+	"Permission denied",
+	"Bad address",
+	"Block device required",
+	"Device or resource busy",
+	"File exists",
+	"Cross-device link",
+	"No such device",
+	"Not a directory",
+	"Is a directory",
+	"Invalid argument",
+	"File table overflow",
+	"Too many open files",
+	"Not a typewriter",
+	"Text file busy",
+	"File too large",
+	"No space left on device",
+	"Illegal seek",
+	"Read-only file system",
+	"Too many links",
+	"Broken pipe",
+	"Math argument out of domain of func",
+	"Math result not representable",
+	"Resource deadlock would occur",
+	"File name too long",
+	"No record locks available",
+	"Invalid system call number",
+	"Directory not empty",
+	"Too many symbolic links encountered",
+	"Bad error code", /* EWOULDBLOCK is EAGAIN */
+	"No message of desired type",
+	"Identifier removed",
+	"Channel number out of range",
+	"Level 2 not synchronized",
+	"Level 3 halted",
+	"Level 3 reset",
+	"Link number out of range",
+	"Protocol driver not attached",
+	"No CSI structure available",
+	"Level 2 halted",
+	"Invalid exchange",
+	"Invalid request descriptor",
+	"Exchange full",
+	"No anode",
+	"Invalid request code",
+	"Invalid slot",
+	"Bad error code", /* EDEADLOCK is EDEADLK */
+	"Bad font file format",
+	"Device not a stream",
+	"No data available",
+	"Timer expired",
+	"Out of streams resources",
+	"Machine is not on the network",
+	"Package not installed",
+	"Object is remote",
+	"Link has been severed",
+	"Advertise error",
+	"Srmount error",
+	"Communication error on send",
+	"Protocol error",
+	"Multihop attempted",
+	"RFS specific error",
+	"Not a data message",
+	"Value too large for defined data type",
+	"Name not unique on network",
+	"File descriptor in bad state",
+	"Remote address changed",
+	"Can not access a needed shared library",
+	"Accessing a corrupted shared library",
+	".lib section in a.out corrupted",
+	"Attempting to link in too many shared libraries",
+	"Cannot exec a shared library directly",
+	"Illegal byte sequence",
+	"Interrupted system call should be restarted",
+	"Streams pipe error",
+	"Too many users",
+	"Socket operation on non-socket",
+	"Destination address required",
+	"Message too long",
+	"Protocol wrong type for socket",
+	"Protocol not available",
+	"Protocol not supported",
+	"Socket type not supported",
+	"Operation not supported on transport endpoint",
+	"Protocol family not supported",
+	"Address family not supported by protocol",
+	"Address already in use",
+	"Cannot assign requested address",
+	"Network is down",
+	"Network is unreachable",
+	"Network dropped connection because of reset",
+	"Software caused connection abort",
+	"Connection reset by peer",
+	"No buffer space available",
+	"Transport endpoint is already connected",
+	"Transport endpoint is not connected",
+	"Cannot send after transport endpoint shutdown",
+	"Too many references: cannot splice",
+	"Connection timed out",
+	"Connection refused",
+	"Host is down",
+	"No route to host",
+	"Operation already in progress",
+	"Operation now in progress",
+	"Stale file handle",
+	"Structure needs cleaning",
+	"Not a XENIX named type file",
+	"No XENIX semaphores available",
+	"Is a named type file",
+	"Remote I/O error",
+	"Quota exceeded",
+	"No medium found",
+	"Wrong medium type",
+	"Operation Canceled",
+	"Required key not available",
+	"Key has expired",
+	"Key has been revoked",
+	"Key was rejected by service",
+	"Owner died",
+	"State not recoverable",
+	"Operation not possible due to RF-kill",
+	"Memory page has hardware error",
+};
+
+const char *lkl_strerror(int err)
+{
+	if (err < 0)
+		err = -err;
+
+	if ((size_t)err >= sizeof(lkl_err_strings) / sizeof(const char *))
+		return "Bad error code";
+
+	return lkl_err_strings[err];
+}
+
+void lkl_perror(char *msg, int err)
+{
+	const char *err_msg = lkl_strerror(err);
+	/* We need to use 'real' printf because lkl_host_ops.print can
+	 * be turned off when debugging is off.
+	 */
+	lkl_printf("%s: %s\n", msg, err_msg);
+}
+
+static int lkl_vprintf(const char *fmt, va_list args)
+{
+	int n;
+	char *buffer;
+	va_list copy;
+
+	if (!lkl_host_ops.print)
+		return 0;
+
+	va_copy(copy, args);
+	n = vsnprintf(NULL, 0, fmt, copy);
+	va_end(copy);
+
+	buffer = lkl_host_ops.mem_alloc(n + 1);
+	if (!buffer)
+		return (-1);
+
+	vsnprintf(buffer, n + 1, fmt, args);
+
+	lkl_host_ops.print(buffer, n);
+	lkl_host_ops.mem_free(buffer);
+
+	return n;
+}
+
+int lkl_printf(const char *fmt, ...)
+{
+	int n;
+	va_list args;
+
+	va_start(args, fmt);
+	n = lkl_vprintf(fmt, args);
+	va_end(args);
+
+	return n;
+}
+
+void lkl_bug(const char *fmt, ...)
+{
+	va_list args;
+
+	va_start(args, fmt);
+	lkl_vprintf(fmt, args);
+	va_end(args);
+
+	lkl_host_ops.panic();
+}
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v6 19/21] um: host: posix host operations
  2020-09-24  7:12       ` Hajime Tazaki
@ 2020-09-24  7:12         ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  7:12 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch, Hajime Tazaki

This commit implements LKL host operations for POSIX hosts.  The
operations are implemented to be portable in various POSIX OSs.

Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 tools/um/lib/Build        |   2 +
 tools/um/lib/posix-host.c | 292 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 294 insertions(+)
 create mode 100644 tools/um/lib/posix-host.c

diff --git a/tools/um/lib/Build b/tools/um/lib/Build
index fc967af4104a..dddff26a3b4e 100644
--- a/tools/um/lib/Build
+++ b/tools/um/lib/Build
@@ -1,4 +1,6 @@
 include $(objtree)/include/config/auto.conf
+CFLAGS_posix-host.o += -D_FILE_OFFSET_BITS=64
 
 liblinux-$(CONFIG_UMMODE_LIB) += utils.o
+liblinux-$(CONFIG_UMMODE_LIB) += posix-host.o
 liblinux-$(CONFIG_UMMODE_LIB) += jmp_buf.o
diff --git a/tools/um/lib/posix-host.c b/tools/um/lib/posix-host.c
new file mode 100644
index 000000000000..b6b5b2902254
--- /dev/null
+++ b/tools/um/lib/posix-host.c
@@ -0,0 +1,292 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <pthread.h>
+#include <limits.h>
+#include <errno.h>
+#include <string.h>
+#include <stdlib.h>
+#include <assert.h>
+#include <sys/syscall.h>
+#include <lkl_host.h>
+#include "jmp_buf.h"
+
+/* Let's see if the host has semaphore.h */
+#include <unistd.h>
+
+#ifdef _POSIX_SEMAPHORES
+#include <semaphore.h>
+/* TODO(pscollins): We don't support fork() for now, but maybe one day
+ * we will?
+ */
+#define SHARE_SEM 0
+#endif /* _POSIX_SEMAPHORES */
+
+
+struct lkl_mutex {
+	pthread_mutex_t mutex;
+};
+
+struct lkl_sem {
+#ifdef _POSIX_SEMAPHORES
+	sem_t sem;
+#else
+	pthread_mutex_t lock;
+	int count;
+	pthread_cond_t cond;
+#endif /* _POSIX_SEMAPHORES */
+};
+
+struct lkl_tls_key {
+	pthread_key_t key;
+};
+
+static void print(const char *str, int len)
+{
+	int ret __attribute__((unused));
+
+	ret = write(STDOUT_FILENO, str, len);
+}
+
+static void panic(void)
+{
+	assert(0);
+}
+
+#define WARN_UNLESS(exp) do {						\
+		if (exp < 0)						\
+			lkl_printf("%s: %s\n", #exp, strerror(errno));	\
+	} while (0)
+
+static int _warn_pthread(int ret, char *str_exp)
+{
+	if (ret > 0)
+		lkl_printf("%s: %s\n", str_exp, strerror(ret));
+
+	return ret;
+}
+
+
+/* pthread_* functions use the reverse convention */
+#define WARN_PTHREAD(exp) _warn_pthread(exp, #exp)
+
+static struct lkl_sem *sem_alloc(int count)
+{
+	struct lkl_sem *sem;
+
+	sem = malloc(sizeof(*sem));
+	if (!sem)
+		return NULL;
+
+#ifdef _POSIX_SEMAPHORES
+	if (sem_init(&sem->sem, SHARE_SEM, count) < 0) {
+		lkl_printf("sem_init: %s\n", strerror(errno));
+		free(sem);
+		return NULL;
+	}
+#else
+	pthread_mutex_init(&sem->lock, NULL);
+	sem->count = count;
+	WARN_PTHREAD(pthread_cond_init(&sem->cond, NULL));
+#endif /* _POSIX_SEMAPHORES */
+
+	return sem;
+}
+
+static void sem_free(struct lkl_sem *sem)
+{
+#ifdef _POSIX_SEMAPHORES
+	WARN_UNLESS(sem_destroy(&sem->sem));
+#else
+	WARN_PTHREAD(pthread_cond_destroy(&sem->cond));
+	WARN_PTHREAD(pthread_mutex_destroy(&sem->lock));
+#endif /* _POSIX_SEMAPHORES */
+	free(sem);
+}
+
+static void sem_up(struct lkl_sem *sem)
+{
+#ifdef _POSIX_SEMAPHORES
+	WARN_UNLESS(sem_post(&sem->sem));
+#else
+	WARN_PTHREAD(pthread_mutex_lock(&sem->lock));
+	sem->count++;
+	if (sem->count > 0)
+		WARN_PTHREAD(pthread_cond_signal(&sem->cond));
+	WARN_PTHREAD(pthread_mutex_unlock(&sem->lock));
+#endif /* _POSIX_SEMAPHORES */
+
+}
+
+static void sem_down(struct lkl_sem *sem)
+{
+#ifdef _POSIX_SEMAPHORES
+	int err;
+
+	do {
+		err = sem_wait(&sem->sem);
+	} while (err < 0 && errno == EINTR);
+	if (err < 0 && errno != EINTR)
+		lkl_printf("sem_wait: %s\n", strerror(errno));
+#else
+	WARN_PTHREAD(pthread_mutex_lock(&sem->lock));
+	while (sem->count <= 0)
+		WARN_PTHREAD(pthread_cond_wait(&sem->cond, &sem->lock));
+	sem->count--;
+	WARN_PTHREAD(pthread_mutex_unlock(&sem->lock));
+#endif /* _POSIX_SEMAPHORES */
+}
+
+static struct lkl_mutex *mutex_alloc(int recursive)
+{
+	struct lkl_mutex *_mutex = malloc(sizeof(struct lkl_mutex));
+	pthread_mutex_t *mutex = NULL;
+	pthread_mutexattr_t attr;
+
+	if (!_mutex)
+		return NULL;
+
+	mutex = &_mutex->mutex;
+	WARN_PTHREAD(pthread_mutexattr_init(&attr));
+
+	/* PTHREAD_MUTEX_ERRORCHECK is *very* useful for debugging,
+	 * but has some overhead, so we provide an option to turn it
+	 * off.
+	 */
+#ifdef DEBUG
+	if (!recursive)
+		WARN_PTHREAD(pthread_mutexattr_settype(
+				     &attr, PTHREAD_MUTEX_ERRORCHECK));
+#endif /* DEBUG */
+
+	if (recursive)
+		WARN_PTHREAD(pthread_mutexattr_settype(
+				     &attr, PTHREAD_MUTEX_RECURSIVE));
+
+	WARN_PTHREAD(pthread_mutex_init(mutex, &attr));
+
+	return _mutex;
+}
+
+static void mutex_lock(struct lkl_mutex *mutex)
+{
+	WARN_PTHREAD(pthread_mutex_lock(&mutex->mutex));
+}
+
+static void mutex_unlock(struct lkl_mutex *_mutex)
+{
+	pthread_mutex_t *mutex = &_mutex->mutex;
+
+	WARN_PTHREAD(pthread_mutex_unlock(mutex));
+}
+
+static void mutex_free(struct lkl_mutex *_mutex)
+{
+	pthread_mutex_t *mutex = &_mutex->mutex;
+
+	WARN_PTHREAD(pthread_mutex_destroy(mutex));
+	free(_mutex);
+}
+
+static lkl_thread_t thread_create(void* (*fn)(void *), void *arg)
+{
+	pthread_t thread;
+
+	if (WARN_PTHREAD(pthread_create(&thread, NULL, fn, arg)))
+		return 0;
+	else
+		return (lkl_thread_t) thread;
+}
+
+static void thread_detach(void)
+{
+	WARN_PTHREAD(pthread_detach(pthread_self()));
+}
+
+static void thread_exit(void)
+{
+	pthread_exit(NULL);
+}
+
+static int thread_join(lkl_thread_t tid)
+{
+	if (WARN_PTHREAD(pthread_join((pthread_t)tid, NULL)))
+		return (-1);
+	else
+		return 0;
+}
+
+static lkl_thread_t thread_self(void)
+{
+	return (lkl_thread_t)pthread_self();
+}
+
+static int thread_equal(lkl_thread_t a, lkl_thread_t b)
+{
+	return pthread_equal((pthread_t)a, (pthread_t)b);
+}
+
+static long _gettid(void)
+{
+#ifdef	__FreeBSD__
+	return (long)pthread_self();
+#else
+	return syscall(SYS_gettid);
+#endif
+}
+
+static struct lkl_tls_key *tls_alloc(void (*destructor)(void *))
+{
+	struct lkl_tls_key *ret = malloc(sizeof(struct lkl_tls_key));
+
+	if (WARN_PTHREAD(pthread_key_create(&ret->key, destructor))) {
+		free(ret);
+		return NULL;
+	}
+	return ret;
+}
+
+static void tls_free(struct lkl_tls_key *key)
+{
+	WARN_PTHREAD(pthread_key_delete(key->key));
+	free(key);
+}
+
+static int tls_set(struct lkl_tls_key *key, void *data)
+{
+	if (WARN_PTHREAD(pthread_setspecific(key->key, data)))
+		return (-1);
+	return 0;
+}
+
+static void *tls_get(struct lkl_tls_key *key)
+{
+	return pthread_getspecific(key->key);
+}
+
+struct lkl_host_operations lkl_host_ops = {
+	.panic = panic,
+	.print = print,
+	.mem_alloc = (void *)malloc,
+	.mem_free = free,
+	.thread_create = thread_create,
+	.thread_detach = thread_detach,
+	.thread_exit = thread_exit,
+	.thread_join = thread_join,
+	.thread_self = thread_self,
+	.thread_equal = thread_equal,
+	.sem_alloc = sem_alloc,
+	.sem_free = sem_free,
+	.sem_up = sem_up,
+	.sem_down = sem_down,
+	.mutex_alloc = mutex_alloc,
+	.mutex_free = mutex_free,
+	.mutex_lock = mutex_lock,
+	.mutex_unlock = mutex_unlock,
+	.gettid = _gettid,
+	.tls_alloc = tls_alloc,
+	.tls_free = tls_free,
+	.tls_set = tls_set,
+	.tls_get = tls_get,
+	.jmp_buf_set = jmp_buf_set,
+	.jmp_buf_longjmp = jmp_buf_longjmp,
+};
+
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v6 19/21] um: host: posix host operations
@ 2020-09-24  7:12         ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  7:12 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Hajime Tazaki, retrage01

This commit implements LKL host operations for POSIX hosts.  The
operations are implemented to be portable in various POSIX OSs.

Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 tools/um/lib/Build        |   2 +
 tools/um/lib/posix-host.c | 292 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 294 insertions(+)
 create mode 100644 tools/um/lib/posix-host.c

diff --git a/tools/um/lib/Build b/tools/um/lib/Build
index fc967af4104a..dddff26a3b4e 100644
--- a/tools/um/lib/Build
+++ b/tools/um/lib/Build
@@ -1,4 +1,6 @@
 include $(objtree)/include/config/auto.conf
+CFLAGS_posix-host.o += -D_FILE_OFFSET_BITS=64
 
 liblinux-$(CONFIG_UMMODE_LIB) += utils.o
+liblinux-$(CONFIG_UMMODE_LIB) += posix-host.o
 liblinux-$(CONFIG_UMMODE_LIB) += jmp_buf.o
diff --git a/tools/um/lib/posix-host.c b/tools/um/lib/posix-host.c
new file mode 100644
index 000000000000..b6b5b2902254
--- /dev/null
+++ b/tools/um/lib/posix-host.c
@@ -0,0 +1,292 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <pthread.h>
+#include <limits.h>
+#include <errno.h>
+#include <string.h>
+#include <stdlib.h>
+#include <assert.h>
+#include <sys/syscall.h>
+#include <lkl_host.h>
+#include "jmp_buf.h"
+
+/* Let's see if the host has semaphore.h */
+#include <unistd.h>
+
+#ifdef _POSIX_SEMAPHORES
+#include <semaphore.h>
+/* TODO(pscollins): We don't support fork() for now, but maybe one day
+ * we will?
+ */
+#define SHARE_SEM 0
+#endif /* _POSIX_SEMAPHORES */
+
+
+struct lkl_mutex {
+	pthread_mutex_t mutex;
+};
+
+struct lkl_sem {
+#ifdef _POSIX_SEMAPHORES
+	sem_t sem;
+#else
+	pthread_mutex_t lock;
+	int count;
+	pthread_cond_t cond;
+#endif /* _POSIX_SEMAPHORES */
+};
+
+struct lkl_tls_key {
+	pthread_key_t key;
+};
+
+static void print(const char *str, int len)
+{
+	int ret __attribute__((unused));
+
+	ret = write(STDOUT_FILENO, str, len);
+}
+
+static void panic(void)
+{
+	assert(0);
+}
+
+#define WARN_UNLESS(exp) do {						\
+		if (exp < 0)						\
+			lkl_printf("%s: %s\n", #exp, strerror(errno));	\
+	} while (0)
+
+static int _warn_pthread(int ret, char *str_exp)
+{
+	if (ret > 0)
+		lkl_printf("%s: %s\n", str_exp, strerror(ret));
+
+	return ret;
+}
+
+
+/* pthread_* functions use the reverse convention */
+#define WARN_PTHREAD(exp) _warn_pthread(exp, #exp)
+
+static struct lkl_sem *sem_alloc(int count)
+{
+	struct lkl_sem *sem;
+
+	sem = malloc(sizeof(*sem));
+	if (!sem)
+		return NULL;
+
+#ifdef _POSIX_SEMAPHORES
+	if (sem_init(&sem->sem, SHARE_SEM, count) < 0) {
+		lkl_printf("sem_init: %s\n", strerror(errno));
+		free(sem);
+		return NULL;
+	}
+#else
+	pthread_mutex_init(&sem->lock, NULL);
+	sem->count = count;
+	WARN_PTHREAD(pthread_cond_init(&sem->cond, NULL));
+#endif /* _POSIX_SEMAPHORES */
+
+	return sem;
+}
+
+static void sem_free(struct lkl_sem *sem)
+{
+#ifdef _POSIX_SEMAPHORES
+	WARN_UNLESS(sem_destroy(&sem->sem));
+#else
+	WARN_PTHREAD(pthread_cond_destroy(&sem->cond));
+	WARN_PTHREAD(pthread_mutex_destroy(&sem->lock));
+#endif /* _POSIX_SEMAPHORES */
+	free(sem);
+}
+
+static void sem_up(struct lkl_sem *sem)
+{
+#ifdef _POSIX_SEMAPHORES
+	WARN_UNLESS(sem_post(&sem->sem));
+#else
+	WARN_PTHREAD(pthread_mutex_lock(&sem->lock));
+	sem->count++;
+	if (sem->count > 0)
+		WARN_PTHREAD(pthread_cond_signal(&sem->cond));
+	WARN_PTHREAD(pthread_mutex_unlock(&sem->lock));
+#endif /* _POSIX_SEMAPHORES */
+
+}
+
+static void sem_down(struct lkl_sem *sem)
+{
+#ifdef _POSIX_SEMAPHORES
+	int err;
+
+	do {
+		err = sem_wait(&sem->sem);
+	} while (err < 0 && errno == EINTR);
+	if (err < 0 && errno != EINTR)
+		lkl_printf("sem_wait: %s\n", strerror(errno));
+#else
+	WARN_PTHREAD(pthread_mutex_lock(&sem->lock));
+	while (sem->count <= 0)
+		WARN_PTHREAD(pthread_cond_wait(&sem->cond, &sem->lock));
+	sem->count--;
+	WARN_PTHREAD(pthread_mutex_unlock(&sem->lock));
+#endif /* _POSIX_SEMAPHORES */
+}
+
+static struct lkl_mutex *mutex_alloc(int recursive)
+{
+	struct lkl_mutex *_mutex = malloc(sizeof(struct lkl_mutex));
+	pthread_mutex_t *mutex = NULL;
+	pthread_mutexattr_t attr;
+
+	if (!_mutex)
+		return NULL;
+
+	mutex = &_mutex->mutex;
+	WARN_PTHREAD(pthread_mutexattr_init(&attr));
+
+	/* PTHREAD_MUTEX_ERRORCHECK is *very* useful for debugging,
+	 * but has some overhead, so we provide an option to turn it
+	 * off.
+	 */
+#ifdef DEBUG
+	if (!recursive)
+		WARN_PTHREAD(pthread_mutexattr_settype(
+				     &attr, PTHREAD_MUTEX_ERRORCHECK));
+#endif /* DEBUG */
+
+	if (recursive)
+		WARN_PTHREAD(pthread_mutexattr_settype(
+				     &attr, PTHREAD_MUTEX_RECURSIVE));
+
+	WARN_PTHREAD(pthread_mutex_init(mutex, &attr));
+
+	return _mutex;
+}
+
+static void mutex_lock(struct lkl_mutex *mutex)
+{
+	WARN_PTHREAD(pthread_mutex_lock(&mutex->mutex));
+}
+
+static void mutex_unlock(struct lkl_mutex *_mutex)
+{
+	pthread_mutex_t *mutex = &_mutex->mutex;
+
+	WARN_PTHREAD(pthread_mutex_unlock(mutex));
+}
+
+static void mutex_free(struct lkl_mutex *_mutex)
+{
+	pthread_mutex_t *mutex = &_mutex->mutex;
+
+	WARN_PTHREAD(pthread_mutex_destroy(mutex));
+	free(_mutex);
+}
+
+static lkl_thread_t thread_create(void* (*fn)(void *), void *arg)
+{
+	pthread_t thread;
+
+	if (WARN_PTHREAD(pthread_create(&thread, NULL, fn, arg)))
+		return 0;
+	else
+		return (lkl_thread_t) thread;
+}
+
+static void thread_detach(void)
+{
+	WARN_PTHREAD(pthread_detach(pthread_self()));
+}
+
+static void thread_exit(void)
+{
+	pthread_exit(NULL);
+}
+
+static int thread_join(lkl_thread_t tid)
+{
+	if (WARN_PTHREAD(pthread_join((pthread_t)tid, NULL)))
+		return (-1);
+	else
+		return 0;
+}
+
+static lkl_thread_t thread_self(void)
+{
+	return (lkl_thread_t)pthread_self();
+}
+
+static int thread_equal(lkl_thread_t a, lkl_thread_t b)
+{
+	return pthread_equal((pthread_t)a, (pthread_t)b);
+}
+
+static long _gettid(void)
+{
+#ifdef	__FreeBSD__
+	return (long)pthread_self();
+#else
+	return syscall(SYS_gettid);
+#endif
+}
+
+static struct lkl_tls_key *tls_alloc(void (*destructor)(void *))
+{
+	struct lkl_tls_key *ret = malloc(sizeof(struct lkl_tls_key));
+
+	if (WARN_PTHREAD(pthread_key_create(&ret->key, destructor))) {
+		free(ret);
+		return NULL;
+	}
+	return ret;
+}
+
+static void tls_free(struct lkl_tls_key *key)
+{
+	WARN_PTHREAD(pthread_key_delete(key->key));
+	free(key);
+}
+
+static int tls_set(struct lkl_tls_key *key, void *data)
+{
+	if (WARN_PTHREAD(pthread_setspecific(key->key, data)))
+		return (-1);
+	return 0;
+}
+
+static void *tls_get(struct lkl_tls_key *key)
+{
+	return pthread_getspecific(key->key);
+}
+
+struct lkl_host_operations lkl_host_ops = {
+	.panic = panic,
+	.print = print,
+	.mem_alloc = (void *)malloc,
+	.mem_free = free,
+	.thread_create = thread_create,
+	.thread_detach = thread_detach,
+	.thread_exit = thread_exit,
+	.thread_join = thread_join,
+	.thread_self = thread_self,
+	.thread_equal = thread_equal,
+	.sem_alloc = sem_alloc,
+	.sem_free = sem_free,
+	.sem_up = sem_up,
+	.sem_down = sem_down,
+	.mutex_alloc = mutex_alloc,
+	.mutex_free = mutex_free,
+	.mutex_lock = mutex_lock,
+	.mutex_unlock = mutex_unlock,
+	.gettid = _gettid,
+	.tls_alloc = tls_alloc,
+	.tls_free = tls_free,
+	.tls_set = tls_set,
+	.tls_get = tls_get,
+	.jmp_buf_set = jmp_buf_set,
+	.jmp_buf_longjmp = jmp_buf_longjmp,
+};
+
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v6 20/21] um: host: add test programs
  2020-09-24  7:12       ` Hajime Tazaki
@ 2020-09-24  7:13         ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  7:13 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch, Hajime Tazaki

Add a simple LKL test application (boot) that starts the kernel and
performs simple tests that minimally exercise the LKL API.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/um/Makefile       |   5 +-
 tools/um/Targets        |   6 +
 tools/um/tests/Build    |   1 +
 tools/um/tests/boot.c   | 393 ++++++++++++++++++++++++++++++++++++++++
 tools/um/tests/boot.sh  |   9 +
 tools/um/tests/run.py   | 172 ++++++++++++++++++
 tools/um/tests/tap13.py | 209 +++++++++++++++++++++
 tools/um/tests/test.c   | 126 +++++++++++++
 tools/um/tests/test.h   |  72 ++++++++
 tools/um/tests/test.sh  | 181 ++++++++++++++++++
 10 files changed, 1173 insertions(+), 1 deletion(-)
 create mode 100644 tools/um/tests/Build
 create mode 100644 tools/um/tests/boot.c
 create mode 100755 tools/um/tests/boot.sh
 create mode 100755 tools/um/tests/run.py
 create mode 100644 tools/um/tests/tap13.py
 create mode 100644 tools/um/tests/test.c
 create mode 100644 tools/um/tests/test.h
 create mode 100644 tools/um/tests/test.sh

diff --git a/tools/um/Makefile b/tools/um/Makefile
index a93e060b1f89..20de2e4c162f 100644
--- a/tools/um/Makefile
+++ b/tools/um/Makefile
@@ -73,7 +73,10 @@ clean:
 	$(call QUIET_CLEAN, liblinux.a)$(RM) $(OUTPUT)/liblinux.a
 	$(call QUIET_CLEAN, $(TARGETS))$(RM) $(TARGETS)
 
+run-tests:
+	./tests/run.py $(tests)
+
 FORCE: ;
-.PHONY: all clean FORCE
+.PHONY: all clean FORCE run-tests
 .IGNORE: clean
 .NOTPARALLEL : lib/linux.o
diff --git a/tools/um/Targets b/tools/um/Targets
index 4e1f0f4d81a3..2bb90381843c 100644
--- a/tools/um/Targets
+++ b/tools/um/Targets
@@ -1,4 +1,10 @@
+ifeq ($(UMMODE),library)
+progs-y += tests/boot
+else
 progs-y += uml/linux
+endif
+
+LDFLAGS_boot-y := -pie
 
 LDLIBS := -lrt -lpthread -lutil
 LDFLAGS_linux-y := -no-pie -Wl,--wrap,malloc -Wl,--wrap,free -Wl,--wrap,calloc
diff --git a/tools/um/tests/Build b/tools/um/tests/Build
new file mode 100644
index 000000000000..26a607f90dc8
--- /dev/null
+++ b/tools/um/tests/Build
@@ -0,0 +1 @@
+boot-y += boot.o test.o
diff --git a/tools/um/tests/boot.c b/tools/um/tests/boot.c
new file mode 100644
index 000000000000..45b9a7276de7
--- /dev/null
+++ b/tools/um/tests/boot.c
@@ -0,0 +1,393 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <unistd.h>
+#include <string.h>
+#include <time.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <lkl.h>
+#include <lkl_host.h>
+
+#include <sys/stat.h>
+#include <fcntl.h>
+#if defined(__FreeBSD__)
+#include <net/if.h>
+#include <sys/ioctl.h>
+#elif __linux
+#include <sys/epoll.h>
+#include <sys/ioctl.h>
+#elif __MINGW32__
+#include <windows.h>
+#endif
+
+#include "test.h"
+
+#ifndef __MINGW32__
+#define sleep_ns 87654321
+int lkl_test_nanosleep(void)
+{
+	struct lkl_timespec ts = {
+		.tv_sec = 0,
+		.tv_nsec = sleep_ns,
+	};
+	struct timespec start, stop;
+	long delta;
+	long ret;
+
+	clock_gettime(CLOCK_MONOTONIC, &start);
+	ret = lkl_sys_nanosleep((struct __lkl__kernel_timespec *)&ts, NULL);
+	clock_gettime(CLOCK_MONOTONIC, &stop);
+
+	delta = 1e9*(stop.tv_sec - start.tv_sec) +
+		(stop.tv_nsec - start.tv_nsec);
+
+	lkl_test_logf("sleep %ld, expected sleep %d\n", delta, sleep_ns);
+
+	if (ret == 0 && delta > sleep_ns * 0.9)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+#endif
+
+LKL_TEST_CALL(getpid, lkl_sys_getpid, 1)
+
+void check_latency(long (*f)(void), long *min, long *max, long *avg)
+{
+	int i;
+	struct timespec start, stop;
+	unsigned long long sum = 0;
+	static const int count = 20;
+	long delta;
+
+	*min = 1000000000;
+	*max = -1;
+
+	for (i = 0; i < count; i++) {
+		clock_gettime(CLOCK_MONOTONIC, &start);
+		f();
+		clock_gettime(CLOCK_MONOTONIC, &stop);
+
+		delta = 1e9*(stop.tv_sec - start.tv_sec) +
+			(stop.tv_nsec - start.tv_nsec);
+
+		if (*min > delta)
+			*min = delta;
+		if (*max < delta)
+			*max = delta;
+		sum += delta;
+	}
+	*avg = sum / count;
+}
+
+static long native_getpid(void)
+{
+#ifdef __MINGW32__
+	GetCurrentProcessId();
+#else
+	getpid();
+#endif
+	return 0;
+}
+
+int lkl_test_syscall_latency(void)
+{
+	long min, max, avg;
+
+	lkl_test_logf("avg/min/max: ");
+
+	check_latency(lkl_sys_getpid, &min, &max, &avg);
+
+	lkl_test_logf("lkl:%ld/%ld/%ld ", avg, min, max);
+
+	check_latency(native_getpid, &min, &max, &avg);
+
+	lkl_test_logf("native:%ld/%ld/%ld\n", avg, min, max);
+
+	return TEST_SUCCESS;
+}
+
+#define access_rights 0721
+
+LKL_TEST_CALL(creat, lkl_sys_creat, 3, "/file", access_rights)
+LKL_TEST_CALL(close, lkl_sys_close, 0, 0);
+LKL_TEST_CALL(failopen, lkl_sys_open, -LKL_ENOENT, "/file2", 0, 0);
+LKL_TEST_CALL(umask, lkl_sys_umask, 022,  0777);
+LKL_TEST_CALL(umask2, lkl_sys_umask, 0777, 0);
+LKL_TEST_CALL(open, lkl_sys_open, 0, "/file", LKL_O_RDWR, 0);
+static const char wrbuf[] = "test";
+LKL_TEST_CALL(write, lkl_sys_write, sizeof(wrbuf), 0, wrbuf, sizeof(wrbuf));
+LKL_TEST_CALL(lseek_cur, lkl_sys_lseek, sizeof(wrbuf), 0, 0, LKL_SEEK_CUR);
+LKL_TEST_CALL(lseek_end, lkl_sys_lseek, sizeof(wrbuf), 0, 0, LKL_SEEK_END);
+LKL_TEST_CALL(lseek_set, lkl_sys_lseek, 0, 0, 0, LKL_SEEK_SET);
+
+int lkl_test_read(void)
+{
+	char buf[10] = { 0, };
+	long ret;
+
+	ret = lkl_sys_read(0, buf, sizeof(buf));
+
+	lkl_test_logf("lkl_sys_read=%ld buf=%s\n", ret, buf);
+
+	if (ret == sizeof(wrbuf) && !strcmp(wrbuf, buf))
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+int lkl_test_fstat(void)
+{
+	struct lkl_stat stat;
+	long ret;
+
+	ret = lkl_sys_fstat(0, (void *)&stat);
+
+	lkl_test_logf("lkl_sys_fstat=%ld mode=%o size=%ld\n", ret, stat.st_mode,
+		      stat.st_size);
+
+	if (ret == 0 && stat.st_size == sizeof(wrbuf) &&
+	    stat.st_mode == (access_rights | LKL_S_IFREG))
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+LKL_TEST_CALL(mkdir, lkl_sys_mkdir, 0, "/proc", access_rights)
+
+int lkl_test_stat(void)
+{
+	struct lkl_stat stat;
+	long ret;
+
+	ret = lkl_sys_stat("/proc", (void *)&stat);
+
+	lkl_test_logf("lkl_sys_stat(\"/proc\")=%ld mode=%o\n", ret,
+		      stat.st_mode);
+
+	if (ret == 0 && stat.st_mode == (access_rights | LKL_S_IFDIR))
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+static int lkl_test_pipe2(void)
+{
+	int pipe_fds[2];
+	int READ_IDX = 0, WRITE_IDX = 1;
+	static const char msg[] = "Hello world!";
+	char str[20];
+	int msg_len_bytes = strlen(msg) + 1;
+	int cmp_res;
+	long ret;
+
+	ret = lkl_sys_pipe2(pipe_fds, LKL_O_NONBLOCK);
+	if (ret) {
+		lkl_test_logf("pipe2: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_write(pipe_fds[WRITE_IDX], msg, msg_len_bytes);
+	if (ret != msg_len_bytes) {
+		if (ret < 0)
+			lkl_test_logf("write error: %s\n", lkl_strerror(ret));
+		else
+			lkl_test_logf("short write: %ld\n", ret);
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_read(pipe_fds[READ_IDX], str, msg_len_bytes);
+	if (ret != msg_len_bytes) {
+		if (ret < 0)
+			lkl_test_logf("read error: %s\n", lkl_strerror(ret));
+		else
+			lkl_test_logf("short read: %ld\n", ret);
+		return TEST_FAILURE;
+	}
+
+	cmp_res = memcmp(msg, str, msg_len_bytes);
+	if (cmp_res) {
+		lkl_test_logf("memcmp failed: %d\n", cmp_res);
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_close(pipe_fds[0]);
+	if (ret) {
+		lkl_test_logf("close error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_close(pipe_fds[1]);
+	if (ret) {
+		lkl_test_logf("close error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	return TEST_SUCCESS;
+}
+
+static int lkl_test_epoll(void)
+{
+	int epoll_fd, pipe_fds[2];
+	int READ_IDX = 0, WRITE_IDX = 1;
+	struct lkl_epoll_event wait_on, read_result;
+	static const char msg[] = "Hello world!";
+	long ret;
+
+	memset(&wait_on, 0, sizeof(wait_on));
+	memset(&read_result, 0, sizeof(read_result));
+
+	ret = lkl_sys_pipe2(pipe_fds, LKL_O_NONBLOCK);
+	if (ret) {
+		lkl_test_logf("pipe2 error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	epoll_fd = lkl_sys_epoll_create(1);
+	if (epoll_fd < 0) {
+		lkl_test_logf("epoll_create error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	wait_on.events = LKL_POLLIN | LKL_POLLOUT;
+	wait_on.data = pipe_fds[READ_IDX];
+
+	ret = lkl_sys_epoll_ctl(epoll_fd, LKL_EPOLL_CTL_ADD, pipe_fds[READ_IDX],
+				&wait_on);
+	if (ret < 0) {
+		lkl_test_logf("epoll_ctl error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	/* Shouldn't be ready before we have written something */
+	ret = lkl_sys_epoll_wait(epoll_fd, &read_result, 1, 0);
+	if (ret != 0) {
+		if (ret < 0)
+			lkl_test_logf("epoll_wait error: %s\n",
+				      lkl_strerror(ret));
+		else
+			lkl_test_logf("epoll_wait: bad event: 0x%lx\n", ret);
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_write(pipe_fds[WRITE_IDX], msg, strlen(msg) + 1);
+	if (ret < 0) {
+		lkl_test_logf("write error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	/* We expect exactly 1 fd to be ready immediately */
+	ret = lkl_sys_epoll_wait(epoll_fd, &read_result, 1, 0);
+	if (ret != 1) {
+		if (ret < 0)
+			lkl_test_logf("epoll_wait error: %s\n",
+				      lkl_strerror(ret));
+		else
+			lkl_test_logf("epoll_wait: bad ev no %ld\n", ret);
+		return TEST_FAILURE;
+	}
+
+	/* Already tested reading from pipe2 so no need to do it
+	 * here
+	 */
+
+	return TEST_SUCCESS;
+}
+
+LKL_TEST_CALL(chdir_proc, lkl_sys_chdir, 0, "proc");
+
+static int dir_fd;
+
+static int lkl_test_open_cwd(void)
+{
+	dir_fd = lkl_sys_open(".", LKL_O_RDONLY | LKL_O_DIRECTORY, 0);
+	if (dir_fd < 0) {
+		lkl_test_logf("failed to open current directory: %s\n",
+			      lkl_strerror(dir_fd));
+		return TEST_FAILURE;
+	}
+
+	return TEST_SUCCESS;
+}
+
+/* column where to insert a line break for the list file tests below. */
+#define COL_LINE_BREAK 70
+
+static int lkl_test_getdents64(void)
+{
+	long ret;
+	char buf[1024], *pos;
+	struct lkl_linux_dirent64 *de;
+	int wr;
+
+	de = (struct lkl_linux_dirent64 *)buf;
+	ret = lkl_sys_getdents64(dir_fd, de, sizeof(buf));
+
+	wr = lkl_test_logf("%d ", dir_fd);
+
+	if (ret < 0)
+		return TEST_FAILURE;
+
+	for (pos = buf; pos - buf < ret; pos += de->d_reclen) {
+		de = (struct lkl_linux_dirent64 *)pos;
+
+		wr += lkl_test_logf("%s ", de->d_name);
+		if (wr >= COL_LINE_BREAK) {
+			lkl_test_logf("\n");
+			wr = 0;
+		}
+	}
+
+	return TEST_SUCCESS;
+}
+
+LKL_TEST_CALL(close_dir_fd, lkl_sys_close, 0, dir_fd);
+LKL_TEST_CALL(chdir_root, lkl_sys_chdir, 0, "/");
+LKL_TEST_CALL(mount_fs_proc, lkl_sys_mount, 0, "none", "/proc", "proc", 0,
+	      NULL);
+LKL_TEST_CALL(umount_fs_proc, lkl_sys_umount, 0, "/proc", 0);
+
+LKL_TEST_CALL(start_kernel, lkl_start_kernel, 0, &lkl_host_ops,
+	     "mem=16M loglevel=8");
+LKL_TEST_CALL(stop_kernel, lkl_sys_halt, 0);
+
+struct lkl_test tests[] = {
+	LKL_TEST(start_kernel),
+	LKL_TEST(getpid),
+	LKL_TEST(syscall_latency),
+	LKL_TEST(umask),
+	LKL_TEST(umask2),
+	LKL_TEST(creat),
+	LKL_TEST(close),
+	LKL_TEST(failopen),
+	LKL_TEST(open),
+	LKL_TEST(write),
+	LKL_TEST(lseek_cur),
+	LKL_TEST(lseek_end),
+	LKL_TEST(lseek_set),
+	LKL_TEST(read),
+	LKL_TEST(fstat),
+	LKL_TEST(mkdir),
+	LKL_TEST(stat),
+#ifndef __MINGW32__
+	LKL_TEST(nanosleep),
+#endif
+	LKL_TEST(pipe2),
+	LKL_TEST(epoll),
+	LKL_TEST(mount_fs_proc),
+	LKL_TEST(chdir_proc),
+	LKL_TEST(open_cwd),
+	LKL_TEST(getdents64),
+	LKL_TEST(close_dir_fd),
+	LKL_TEST(chdir_root),
+	LKL_TEST(umount_fs_proc),
+	LKL_TEST(stop_kernel),
+};
+
+int main(int argc, const char **argv)
+{
+	lkl_host_ops.print = lkl_test_log;
+
+	return lkl_test_run(tests, sizeof(tests)/sizeof(struct lkl_test),
+			    "boot");
+}
diff --git a/tools/um/tests/boot.sh b/tools/um/tests/boot.sh
new file mode 100755
index 000000000000..d985c04b0ac1
--- /dev/null
+++ b/tools/um/tests/boot.sh
@@ -0,0 +1,9 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+source $script_dir/test.sh
+
+lkl_test_plan 1 "boot"
+lkl_test_run 1
+lkl_test_exec $script_dir/boot
diff --git a/tools/um/tests/run.py b/tools/um/tests/run.py
new file mode 100755
index 000000000000..c96ede90b6ad
--- /dev/null
+++ b/tools/um/tests/run.py
@@ -0,0 +1,172 @@
+#!/usr/bin/env python
+# SPDX-License-Identifier: GPL-2.0
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 2 of the License
+#
+# Author: Octavian Purdila <tavi@cs.pub.ro>
+#
+
+from __future__ import print_function
+
+import argparse
+import os
+import subprocess
+import sys
+import tap13
+import xml.etree.ElementTree as ET
+
+from junit_xml import TestSuite, TestCase
+
+
+class Reporter(tap13.Reporter):
+    def start(self, obj):
+        if type(obj) is tap13.Test:
+            if obj.result == "*":
+                end='\r'
+            else:
+                end='\n'
+            print("  TEST       %-8s %.50s" %
+                  (obj.result, obj.description + " " + obj.comment), end=end)
+
+        elif type(obj) is tap13.Suite:
+            if obj.tests_planned == 0:
+                status = "skip"
+            else:
+                status = ""
+            print("  SUITE      %-8s %s" % (status, obj.name))
+
+    def end(self, obj):
+        if type(obj) is tap13.Test:
+            if obj.result != "ok":
+                try:
+                    print(obj.yaml["log"], end='')
+                except:
+                    None
+
+
+mydir=os.path.dirname(os.path.realpath(__file__))
+
+tests = [
+    'boot.sh',
+]
+
+parser = argparse.ArgumentParser(description='LKL test runner')
+parser.add_argument('tests', nargs='?', action='append',
+                    help='tests to run %s' % tests)
+parser.add_argument('--junit-dir',
+                    help='directory where to store the juni suites')
+parser.add_argument('--gdb', action='store_true', default=False,
+                    help='run simple tests under gdb; implies --pass-through')
+parser.add_argument('--pass-through', action='store_true',  default=False,
+                    help='run the test without interpeting the test output')
+parser.add_argument('--valgrind', action='store_true', default=False,
+                    help='run simple tests under valgrind')
+
+args = parser.parse_args()
+if args.tests == [None]:
+    args.tests = tests
+
+if args.gdb:
+    args.pass_through=True
+    os.environ['GDB']="yes"
+
+if args.valgrind:
+    os.environ['VALGRIND']="yes"
+
+tap = tap13.Parser(Reporter())
+
+os.environ['PATH'] += ":" + mydir
+
+exit_code = 0
+
+for t in args.tests:
+    if not t:
+        continue
+    if args.pass_through:
+        print(t)
+        if subprocess.call(t, shell=True) != 0:
+            exit_code = 1
+    else:
+        p = subprocess.Popen(t, shell=True, stdout=subprocess.PIPE)
+        tap.parse(p.stdout)
+
+if args.pass_through:
+    sys.exit(exit_code)
+
+suites_count = 0
+tests_total = 0
+tests_not_ok = 0
+tests_ok = 0
+tests_skip = 0
+val_errs = 0
+val_fails = 0
+val_skips = 0
+
+for s in tap.run.suites:
+
+    junit_tests = []
+    suites_count += 1
+
+    for t in s.tests:
+        try:
+            secs = t.yaml["time_us"] / 1000000.0
+        except:
+            secs = 0
+        try:
+            log = t.yaml['log']
+        except:
+            log = ""
+
+        jt = TestCase(t.description, elapsed_sec=secs, stdout=log)
+        if t.result == 'skip':
+            jt.add_skipped_info(output=log)
+        elif t.result == 'not ok':
+            jt.add_error_info(output=log)
+
+        junit_tests.append(jt)
+
+        tests_total += 1
+        if t.result == "ok":
+            tests_ok += 1
+        elif t.result == "not ok":
+            tests_not_ok += 1
+            exit_code = 1
+        elif t.result == "skip":
+            tests_skip += 1
+
+    if args.junit_dir:
+        js = TestSuite(s.name, junit_tests)
+        with open(os.path.join(args.junit_dir, os.path.basename(s.name) + '.xml'), 'w') as f:
+            js.to_file(f, [js])
+
+        if os.getenv('VALGRIND') is not None:
+            val_xml = 'valgrind-%s.xml' % os.path.basename(s.name).replace(' ','-')
+            # skipped tests don't generate xml file
+            if os.path.exists(val_xml) is False:
+                continue
+
+            cmd = 'mv %s %s' % (val_xml, args.junit_dir)
+            subprocess.call(cmd, shell=True, )
+
+            cmd = mydir + '/valgrind2xunit.py ' + val_xml
+            subprocess.call(cmd, shell=True, cwd=args.junit_dir)
+
+            # count valgrind results
+            doc = ET.parse(os.path.join(args.junit_dir, 'valgrind-%s_xunit.xml' \
+                                        % (os.path.basename(s.name).replace(' ','-'))))
+            ts = doc.getroot()
+            val_errs += int(ts.get('errors'))
+            val_fails += int(ts.get('failures'))
+            val_skips += int(ts.get('skip'))
+
+print("Summary: %d suites run, %d tests, %d ok, %d not ok, %d skipped" %
+      (suites_count, tests_total, tests_ok, tests_not_ok, tests_skip))
+
+if os.getenv('VALGRIND') is not None:
+    print(" valgrind (memcheck): %d failures, %d skipped" % (val_fails, val_skips))
+    if val_errs or val_fails:
+        exit_code = 1
+
+sys.exit(exit_code)
diff --git a/tools/um/tests/tap13.py b/tools/um/tests/tap13.py
new file mode 100644
index 000000000000..65c73cda7ca1
--- /dev/null
+++ b/tools/um/tests/tap13.py
@@ -0,0 +1,209 @@
+#!/usr/bin/env python
+# SPDX-License-Identifier: GPL-2.0
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 2 of the License
+#
+# Author: Octavian Purdila <tavi@cs.pub.ro>
+#
+# Based on TAP13:
+#
+# Copyright 2013, Red Hat, Inc.
+# Author: Josef Skladanka <jskladan@redhat.com>
+#
+from __future__ import print_function
+
+import re
+import sys
+import yamlish
+
+
+class Reporter(object):
+
+    def start(self, obj):
+        None
+
+    def end(self, obj):
+        None
+
+
+class Test(object):
+    def __init__(self, reporter, result, id, description=None, directive=None,
+                 comment=None):
+        self.reporter = reporter
+        self.result = result
+        if directive:
+            self.result = directive.lower()
+        if id:
+            self.id = int(id)
+        else:
+            self.id = None
+        if description:
+            self.description = description
+        else:
+            self.description = ""
+        if comment:
+            self.comment = "# " + comment
+        else:
+            self.comment = ""
+        self.yaml = None
+        self._yaml_buffer = None
+        self.diagnostics = []
+
+        self.reporter.start(self)
+
+    def end(self):
+        if not self.yaml:
+            self.yaml = yamlish.load(self._yaml_buffer)
+            self.reporter.end(self)
+
+
+class Suite(object):
+    def __init__(self, reporter, start, end, explanation):
+        self.reporter = reporter
+        self.tests = []
+        self.name = explanation
+        self.tests_planned = int(end)
+
+        self.__tests_counter = 0
+        self.__tests_base = 0
+
+        self.reporter.start(self)
+
+    def newTest(self, args):
+        try:
+            self.tests[-1].end()
+        except IndexError:
+            None
+
+        if 'id' not in args or not args['id']:
+            args['id'] = self.__tests_counter
+        else:
+            args['id'] = int(args['id']) + self.__tests_base
+
+        if args['id'] < self.__tests_counter:
+            print("error: bad test id %d, fixing it" % (args['id']))
+            args['id'] = self.__tests_counter
+        # according to TAP13 specs, missing tests must be handled as 'not ok'
+        # here we add the missing tests in sequence
+        while args['id'] > (self.__tests_counter + 1):
+            comment = 'test %d not present' % self.__tests_counter
+            self.tests.append(Test(self.reporter, 'not ok',
+                                   self.__tests_counter, comment=comment))
+            self.__tests_counter += 1
+
+        if args['id'] == self.__tests_counter:
+            if args['directive']:
+                self.test().result = args['directive'].lower()
+            else:
+                self.test().result = args['result']
+            self.reporter.start(self.test())
+        else:
+            self.tests.append(Test(self.reporter, **args))
+            self.__tests_counter += 1
+
+    def test(self):
+        return self.tests[-1]
+
+    def end(self, name, planned):
+        if name == self.name:
+            self.tests_planned += int(planned)
+            self.__tests_base = self.__tests_counter
+            return False
+        try:
+            self.test().end()
+        except IndexError:
+            None
+        if len(self.tests) != self.tests_planned:
+            for i in range(len(self.tests), self.tests_planned):
+                self.tests.append(Test(self.reporter, 'not ok', i+1,
+                                       comment='test not present'))
+        return True
+
+
+class Run(object):
+
+    def __init__(self, reporter):
+        self.reporter = reporter
+        self.suites = []
+
+    def suite(self):
+        return self.suites[-1]
+
+    def test(self):
+        return self.suites[-1].tests[-1]
+
+    def newSuite(self, args):
+        new = False
+        try:
+            if self.suite().end(args['explanation'], args['end']):
+                new = True
+        except IndexError:
+            new = True
+        if new:
+            self.suites.append(Suite(self.reporter, **args))
+
+    def newTest(self, args):
+        self.suite().newTest(args)
+
+
+class Parser(object):
+    RE_PLAN = re.compile(r"^\s*(?P<start>\d+)\.\.(?P<end>\d+)\s*(#\s*(?P<explanation>.*))?\s*$")
+    RE_TEST_LINE = re.compile(r"^\s*(?P<result>(not\s+)?ok|[*]+)\s*(?P<id>\d+)?\s*(?P<description>[^#]+)?\s*(#\s*(?P<directive>TODO|SKIP)?\s*(?P<comment>.+)?)?\s*$",  re.IGNORECASE)
+    RE_EXPLANATION = re.compile(r"^\s*#\s*(?P<explanation>.+)?\s*$")
+    RE_YAMLISH_START = re.compile(r"^\s*---.*$")
+    RE_YAMLISH_END = re.compile(r"^\s*\.\.\.\s*$")
+
+    def __init__(self, reporter):
+        self.seek_test = False
+        self.in_test = False
+        self.in_yaml = False
+        self.run = Run(reporter)
+
+    def parse(self, source):
+        # to avoid input buffering
+        while True:
+            line = source.readline()
+            if not line:
+                break
+
+            if self.in_yaml:
+                if Parser.RE_YAMLISH_END.match(line):
+                    self.run.test()._yaml_buffer.append(line.strip())
+                    self.in_yaml = False
+                else:
+                    self.run.test()._yaml_buffer.append(line.rstrip())
+                continue
+
+            line = line.strip()
+
+            if self.in_test:
+                if Parser.RE_EXPLANATION.match(line):
+                    self.run.test().diagnostics.append(line)
+                    continue
+                if Parser.RE_YAMLISH_START.match(line):
+                    self.run.test()._yaml_buffer = [line.strip()]
+                    self.in_yaml = True
+                    continue
+
+            m = Parser.RE_PLAN.match(line)
+            if m:
+                self.seek_test = True
+                args = m.groupdict()
+                self.run.newSuite(args)
+                continue
+
+            if self.seek_test:
+                m = Parser.RE_TEST_LINE.match(line)
+                if m:
+                    args = m.groupdict()
+                    self.run.newTest(args)
+                    self.in_test = True
+                    continue
+
+            print(line)
+        try:
+            self.run.suite().end(None, 0)
+        except IndexError:
+            None
diff --git a/tools/um/tests/test.c b/tools/um/tests/test.c
new file mode 100644
index 000000000000..37ea21901f99
--- /dev/null
+++ b/tools/um/tests/test.c
@@ -0,0 +1,126 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <stdarg.h>
+#include <time.h>
+
+#include "test.h"
+
+/* circular log buffer */
+
+static char log_buf[0x10000];
+static char *head = log_buf, *tail = log_buf;
+
+static inline void advance(char **ptr)
+{
+	if ((unsigned int)(*ptr - log_buf) >= sizeof(log_buf))
+		*ptr = log_buf;
+	else
+		*ptr = *ptr + 1;
+}
+
+static void log_char(char c)
+{
+	*tail = c;
+	advance(&tail);
+	if (tail == head)
+		advance(&head);
+}
+
+static void print_log(void)
+{
+	char last;
+
+	printf(" log: |\n");
+	last = '\n';
+	while (head != tail) {
+		if (last == '\n')
+			printf("  ");
+		last = *head;
+		putchar(last);
+		advance(&head);
+	}
+	if (last != '\n')
+		putchar('\n');
+}
+
+int lkl_test_run(const struct lkl_test *tests, int nr, const char *fmt, ...)
+{
+	int i, ret, status = TEST_SUCCESS;
+	clock_t start, stop;
+	char name[1024];
+	va_list args;
+
+	va_start(args, fmt);
+	vsnprintf(name, sizeof(name), fmt, args);
+	va_end(args);
+
+	printf("1..%d # %s\n", nr, name);
+	for (i = 1; i <= nr; i++) {
+		const struct lkl_test *t = &tests[i-1];
+		unsigned long delta_us;
+
+		printf("* %d %s\n", i, t->name);
+		fflush(stdout);
+
+		start = clock();
+
+		ret = t->fn(t->arg1, t->arg2, t->arg3);
+
+		stop = clock();
+
+		switch (ret) {
+		case TEST_SUCCESS:
+			printf("ok %d %s\n", i, t->name);
+			break;
+		case TEST_SKIP:
+			printf("ok %d %s # SKIP\n", i, t->name);
+			break;
+		case TEST_BAILOUT:
+			status = TEST_BAILOUT;
+			/* fall through; */
+		case TEST_FAILURE:
+		default:
+			if (status != TEST_BAILOUT)
+				status = TEST_FAILURE;
+			printf("not ok %d %s\n", i, t->name);
+		}
+
+		printf(" ---\n");
+		delta_us = (stop - start) * 1000000 / CLOCKS_PER_SEC;
+		printf(" time_us: %ld\n", delta_us);
+		print_log();
+		printf(" ...\n");
+
+		if (status == TEST_BAILOUT) {
+			printf("Bail out!\n");
+			return TEST_FAILURE;
+		}
+
+		fflush(stdout);
+	}
+
+	return status;
+}
+
+
+void lkl_test_log(const char *str, int len)
+{
+	while (len--)
+		log_char(*(str++));
+}
+
+int lkl_test_logf(const char *fmt, ...)
+{
+	char tmp[1024], *c;
+	va_list args;
+	unsigned int n;
+
+	va_start(args, fmt);
+	n = vsnprintf(tmp, sizeof(tmp), fmt, args);
+	va_end(args);
+
+	for (c = tmp; *c != 0; c++)
+		log_char(*c);
+
+	return n > sizeof(tmp) ? sizeof(tmp) : n;
+}
diff --git a/tools/um/tests/test.h b/tools/um/tests/test.h
new file mode 100644
index 000000000000..f63ad6d419cb
--- /dev/null
+++ b/tools/um/tests/test.h
@@ -0,0 +1,72 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_TEST_H
+#define _LKL_TEST_H
+
+#define TEST_SUCCESS	0
+#define TEST_FAILURE	1
+#define TEST_SKIP	2
+#define TEST_TODO	3
+#define TEST_BAILOUT	4
+
+struct lkl_test {
+	const char *name;
+	int (*fn)();
+	void *arg1, *arg2, *arg3;
+};
+
+/**
+ * Simple wrapper to initialize a test entry.
+ * @name - test name, it assume test function is named test_@name
+ * @vargs - arguments to be passed to the function
+ */
+#define LKL_TEST(name, ...) { #name, lkl_test_##name, __VA_ARGS__ }
+
+/**
+ * lkl_test_run - run a test suite
+ *
+ * @tests - the list of tests to run
+ * @nr - number of tests
+ * @fmt - format string to be used for suite name
+ */
+int lkl_test_run(const struct lkl_test *tests, int nr, const char *fmt, ...);
+
+/**
+ * lkl_test_log - store a string in the test log buffer
+ * @str - the string to log (can be non-NULL terminated)
+ * @len - the string length
+ */
+void lkl_test_log(const char *str, int len);
+
+/**
+ * lkl_test_logf - printf like function to store into the test log buffer
+ * @fmt - printf format string
+ * @vargs - arguments to the format string
+ */
+int lkl_test_logf(const char *fmt, ...) __attribute__((format(printf, 1, 2)));
+
+/**
+ * LKL_TEST_CALL - create a test function as for a LKL call
+ *
+ * The test function will be named lkl_test_@name and will return
+ * TEST_SUCCESS if the called functions returns @expect. Otherwise
+ * will return TEST_FAILUIRE.
+ *
+ * @name - test name; must be unique because it is part of the the
+ * test function; the test function will be named
+ * @call - function to call
+ * @expect - expected return value for success
+ * @args - arguments to pass to the LKL call
+ */
+#define LKL_TEST_CALL(name, call, expect, ...)				\
+	static int lkl_test_##name(void)				\
+	{								\
+		long ret;						\
+									\
+		ret = call(__VA_ARGS__);				\
+		lkl_test_logf("%s(%s) = %ld %s\n", #call, #__VA_ARGS__, \
+			ret, ret < 0 ? lkl_strerror(ret) : "");		\
+		return (ret == expect) ? TEST_SUCCESS : TEST_FAILURE;	\
+	}
+
+
+#endif /* _LKL_TEST_H */
diff --git a/tools/um/tests/test.sh b/tools/um/tests/test.sh
new file mode 100644
index 000000000000..31b7de5c06f7
--- /dev/null
+++ b/tools/um/tests/test.sh
@@ -0,0 +1,181 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+basedir=$(cd $script_dir/..; pwd)
+base_objdir=$(cd ${OUTPUT}/; pwd)
+
+TEST_SUCCESS=0
+TEST_FAILURE=1
+TEST_SKIP=113
+TEST_TODO=114
+TEST_BAILOUT=115
+
+print_log()
+{
+    echo " log: |"
+    while read line; do
+        echo "  $line"
+    done < $1
+}
+
+export_vars()
+{
+    if [ -z "$var_file" ]; then
+        return
+    fi
+
+    for i in $@; do
+        echo "$i=${!i}" >> $var_file
+    done
+}
+
+lkl_test_run()
+{
+    log_file=$(mktemp)
+    export var_file=$(mktemp)
+
+    tid=$1 && shift && tname=$@
+
+    echo "* $tid $tname"
+
+    start=$(date '+%s%9N')
+    # run in a separate shell to avoid -e terminating us
+    $@ 2>&1 | strings >$log_file
+    exit=${PIPESTATUS[0]}
+    stop=$(date '+%s%9N')
+
+    case $exit in
+    $TEST_SUCCESS)
+        echo "ok $tid $tname"
+        ;;
+    $TEST_SKIP)
+        echo "ok $tid $tname # SKIP"
+        ;;
+    $TEST_BAILOUT)
+        echo "not ok $tid $tname"
+        echo "Bail out!"
+        ;;
+    $TEST_FAILURE|*)
+        echo "not ok $tid $tname"
+        ;;
+    esac
+
+    delta=$(((stop-start)/1000))
+
+    echo " ---"
+    echo " time_us: $delta"
+    print_log $log_file
+    echo -e " ..."
+
+    rm $log_file
+    . $var_file
+    rm $var_file
+
+    return $exit
+}
+
+lkl_test_plan()
+{
+    echo "1..$1 # $2"
+    export suite_name="${2// /\-}"
+}
+
+lkl_test_exec()
+{
+    local SUDO=""
+    local WRAPPER=""
+
+    if [ "$1" = "sudo" ]; then
+        SUDO=sudo
+        shift
+    fi
+
+    local file=$1
+    shift
+
+    if [ -n "$LKL_HOST_CONFIG_NT" ]; then
+        file=$file.exe
+    fi
+
+    file=${OUTPUT}/tests/$(basename $file)
+
+    if file $file | grep ARM; then
+        WRAPPER="qemu-arm-static"
+    elif file $file | grep "FreeBSD" ; then
+        ssh_copy "$file" $BSD_WDIR
+        if [ -n "$SUDO" ]; then
+            SUDO=""
+        fi
+        WRAPPER="$MYSSH $SU"
+        # ssh will mess up with pipes ('|') so, escape the pipe char.
+        args="${@//\|/\\\|}"
+        set - $BSD_WDIR/$(basename $file) $args
+        file=""
+    elif [ -n "$GDB" ]; then
+        WRAPPER="gdb"
+        args="$@"
+        set - -ex "run $args" -ex quit $file
+        file=""
+    elif [ -n "$VALGRIND" ]; then
+        WRAPPER="valgrind --suppressions=$script_dir/valgrind.supp \
+                  --leak-check=full --show-leak-kinds=all --xml=yes \
+                  --xml-file=valgrind-$suite_name.xml"
+    fi
+
+    $SUDO $WRAPPER $file "$@"
+}
+
+lkl_test_cmd()
+{
+    local WRAPPER=""
+
+    if [ -z "$QUIET" ]; then
+        SHOPTS="-x"
+    fi
+
+    if [ -n "$LKL_HOST_CONFIG_BSD" ]; then
+        WRAPPER="$MYSSH $SU"
+    fi
+
+    echo "$@" | $WRAPPER sh $SHOPTS
+}
+
+# XXX: $MYSSH and $MYSCP are defined in a circleci docker image.
+# see the definitions in lkl/lkl-docker:circleci/freebsd11/Dockerfile
+ssh_push()
+{
+    while [ -n "$1" ]; do
+        if [[ "$1" = *.sh ]]; then
+            type="script"
+        else
+            type="file"
+        fi
+
+        dir=$(dirname $1)
+        $MYSSH mkdir -p $BSD_WDIR/$dir
+
+        $MYSCP -P 7722 -r $basedir/$1 root@localhost:$BSD_WDIR/$dir
+        if [ "$type" = "script" ]; then
+            $MYSSH chmod a+x $BSD_WDIR/$1
+        fi
+
+        shift
+    done
+}
+
+ssh_copy()
+{
+    $MYSCP -P 7722 -r $1 root@localhost:$2
+}
+
+lkl_test_bsd_cleanup()
+{
+    $MYSSH rm -rf $BSD_WDIR
+}
+
+if [ -n "$LKL_HOST_CONFIG_BSD" ]; then
+    trap lkl_test_bsd_cleanup EXIT
+    export BSD_WDIR=/root/lkl
+    $MYSSH mkdir -p $BSD_WDIR
+fi
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v6 20/21] um: host: add test programs
@ 2020-09-24  7:13         ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  7:13 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Hajime Tazaki, retrage01

Add a simple LKL test application (boot) that starts the kernel and
performs simple tests that minimally exercise the LKL API.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/um/Makefile       |   5 +-
 tools/um/Targets        |   6 +
 tools/um/tests/Build    |   1 +
 tools/um/tests/boot.c   | 393 ++++++++++++++++++++++++++++++++++++++++
 tools/um/tests/boot.sh  |   9 +
 tools/um/tests/run.py   | 172 ++++++++++++++++++
 tools/um/tests/tap13.py | 209 +++++++++++++++++++++
 tools/um/tests/test.c   | 126 +++++++++++++
 tools/um/tests/test.h   |  72 ++++++++
 tools/um/tests/test.sh  | 181 ++++++++++++++++++
 10 files changed, 1173 insertions(+), 1 deletion(-)
 create mode 100644 tools/um/tests/Build
 create mode 100644 tools/um/tests/boot.c
 create mode 100755 tools/um/tests/boot.sh
 create mode 100755 tools/um/tests/run.py
 create mode 100644 tools/um/tests/tap13.py
 create mode 100644 tools/um/tests/test.c
 create mode 100644 tools/um/tests/test.h
 create mode 100644 tools/um/tests/test.sh

diff --git a/tools/um/Makefile b/tools/um/Makefile
index a93e060b1f89..20de2e4c162f 100644
--- a/tools/um/Makefile
+++ b/tools/um/Makefile
@@ -73,7 +73,10 @@ clean:
 	$(call QUIET_CLEAN, liblinux.a)$(RM) $(OUTPUT)/liblinux.a
 	$(call QUIET_CLEAN, $(TARGETS))$(RM) $(TARGETS)
 
+run-tests:
+	./tests/run.py $(tests)
+
 FORCE: ;
-.PHONY: all clean FORCE
+.PHONY: all clean FORCE run-tests
 .IGNORE: clean
 .NOTPARALLEL : lib/linux.o
diff --git a/tools/um/Targets b/tools/um/Targets
index 4e1f0f4d81a3..2bb90381843c 100644
--- a/tools/um/Targets
+++ b/tools/um/Targets
@@ -1,4 +1,10 @@
+ifeq ($(UMMODE),library)
+progs-y += tests/boot
+else
 progs-y += uml/linux
+endif
+
+LDFLAGS_boot-y := -pie
 
 LDLIBS := -lrt -lpthread -lutil
 LDFLAGS_linux-y := -no-pie -Wl,--wrap,malloc -Wl,--wrap,free -Wl,--wrap,calloc
diff --git a/tools/um/tests/Build b/tools/um/tests/Build
new file mode 100644
index 000000000000..26a607f90dc8
--- /dev/null
+++ b/tools/um/tests/Build
@@ -0,0 +1 @@
+boot-y += boot.o test.o
diff --git a/tools/um/tests/boot.c b/tools/um/tests/boot.c
new file mode 100644
index 000000000000..45b9a7276de7
--- /dev/null
+++ b/tools/um/tests/boot.c
@@ -0,0 +1,393 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <unistd.h>
+#include <string.h>
+#include <time.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <lkl.h>
+#include <lkl_host.h>
+
+#include <sys/stat.h>
+#include <fcntl.h>
+#if defined(__FreeBSD__)
+#include <net/if.h>
+#include <sys/ioctl.h>
+#elif __linux
+#include <sys/epoll.h>
+#include <sys/ioctl.h>
+#elif __MINGW32__
+#include <windows.h>
+#endif
+
+#include "test.h"
+
+#ifndef __MINGW32__
+#define sleep_ns 87654321
+int lkl_test_nanosleep(void)
+{
+	struct lkl_timespec ts = {
+		.tv_sec = 0,
+		.tv_nsec = sleep_ns,
+	};
+	struct timespec start, stop;
+	long delta;
+	long ret;
+
+	clock_gettime(CLOCK_MONOTONIC, &start);
+	ret = lkl_sys_nanosleep((struct __lkl__kernel_timespec *)&ts, NULL);
+	clock_gettime(CLOCK_MONOTONIC, &stop);
+
+	delta = 1e9*(stop.tv_sec - start.tv_sec) +
+		(stop.tv_nsec - start.tv_nsec);
+
+	lkl_test_logf("sleep %ld, expected sleep %d\n", delta, sleep_ns);
+
+	if (ret == 0 && delta > sleep_ns * 0.9)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+#endif
+
+LKL_TEST_CALL(getpid, lkl_sys_getpid, 1)
+
+void check_latency(long (*f)(void), long *min, long *max, long *avg)
+{
+	int i;
+	struct timespec start, stop;
+	unsigned long long sum = 0;
+	static const int count = 20;
+	long delta;
+
+	*min = 1000000000;
+	*max = -1;
+
+	for (i = 0; i < count; i++) {
+		clock_gettime(CLOCK_MONOTONIC, &start);
+		f();
+		clock_gettime(CLOCK_MONOTONIC, &stop);
+
+		delta = 1e9*(stop.tv_sec - start.tv_sec) +
+			(stop.tv_nsec - start.tv_nsec);
+
+		if (*min > delta)
+			*min = delta;
+		if (*max < delta)
+			*max = delta;
+		sum += delta;
+	}
+	*avg = sum / count;
+}
+
+static long native_getpid(void)
+{
+#ifdef __MINGW32__
+	GetCurrentProcessId();
+#else
+	getpid();
+#endif
+	return 0;
+}
+
+int lkl_test_syscall_latency(void)
+{
+	long min, max, avg;
+
+	lkl_test_logf("avg/min/max: ");
+
+	check_latency(lkl_sys_getpid, &min, &max, &avg);
+
+	lkl_test_logf("lkl:%ld/%ld/%ld ", avg, min, max);
+
+	check_latency(native_getpid, &min, &max, &avg);
+
+	lkl_test_logf("native:%ld/%ld/%ld\n", avg, min, max);
+
+	return TEST_SUCCESS;
+}
+
+#define access_rights 0721
+
+LKL_TEST_CALL(creat, lkl_sys_creat, 3, "/file", access_rights)
+LKL_TEST_CALL(close, lkl_sys_close, 0, 0);
+LKL_TEST_CALL(failopen, lkl_sys_open, -LKL_ENOENT, "/file2", 0, 0);
+LKL_TEST_CALL(umask, lkl_sys_umask, 022,  0777);
+LKL_TEST_CALL(umask2, lkl_sys_umask, 0777, 0);
+LKL_TEST_CALL(open, lkl_sys_open, 0, "/file", LKL_O_RDWR, 0);
+static const char wrbuf[] = "test";
+LKL_TEST_CALL(write, lkl_sys_write, sizeof(wrbuf), 0, wrbuf, sizeof(wrbuf));
+LKL_TEST_CALL(lseek_cur, lkl_sys_lseek, sizeof(wrbuf), 0, 0, LKL_SEEK_CUR);
+LKL_TEST_CALL(lseek_end, lkl_sys_lseek, sizeof(wrbuf), 0, 0, LKL_SEEK_END);
+LKL_TEST_CALL(lseek_set, lkl_sys_lseek, 0, 0, 0, LKL_SEEK_SET);
+
+int lkl_test_read(void)
+{
+	char buf[10] = { 0, };
+	long ret;
+
+	ret = lkl_sys_read(0, buf, sizeof(buf));
+
+	lkl_test_logf("lkl_sys_read=%ld buf=%s\n", ret, buf);
+
+	if (ret == sizeof(wrbuf) && !strcmp(wrbuf, buf))
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+int lkl_test_fstat(void)
+{
+	struct lkl_stat stat;
+	long ret;
+
+	ret = lkl_sys_fstat(0, (void *)&stat);
+
+	lkl_test_logf("lkl_sys_fstat=%ld mode=%o size=%ld\n", ret, stat.st_mode,
+		      stat.st_size);
+
+	if (ret == 0 && stat.st_size == sizeof(wrbuf) &&
+	    stat.st_mode == (access_rights | LKL_S_IFREG))
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+LKL_TEST_CALL(mkdir, lkl_sys_mkdir, 0, "/proc", access_rights)
+
+int lkl_test_stat(void)
+{
+	struct lkl_stat stat;
+	long ret;
+
+	ret = lkl_sys_stat("/proc", (void *)&stat);
+
+	lkl_test_logf("lkl_sys_stat(\"/proc\")=%ld mode=%o\n", ret,
+		      stat.st_mode);
+
+	if (ret == 0 && stat.st_mode == (access_rights | LKL_S_IFDIR))
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+static int lkl_test_pipe2(void)
+{
+	int pipe_fds[2];
+	int READ_IDX = 0, WRITE_IDX = 1;
+	static const char msg[] = "Hello world!";
+	char str[20];
+	int msg_len_bytes = strlen(msg) + 1;
+	int cmp_res;
+	long ret;
+
+	ret = lkl_sys_pipe2(pipe_fds, LKL_O_NONBLOCK);
+	if (ret) {
+		lkl_test_logf("pipe2: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_write(pipe_fds[WRITE_IDX], msg, msg_len_bytes);
+	if (ret != msg_len_bytes) {
+		if (ret < 0)
+			lkl_test_logf("write error: %s\n", lkl_strerror(ret));
+		else
+			lkl_test_logf("short write: %ld\n", ret);
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_read(pipe_fds[READ_IDX], str, msg_len_bytes);
+	if (ret != msg_len_bytes) {
+		if (ret < 0)
+			lkl_test_logf("read error: %s\n", lkl_strerror(ret));
+		else
+			lkl_test_logf("short read: %ld\n", ret);
+		return TEST_FAILURE;
+	}
+
+	cmp_res = memcmp(msg, str, msg_len_bytes);
+	if (cmp_res) {
+		lkl_test_logf("memcmp failed: %d\n", cmp_res);
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_close(pipe_fds[0]);
+	if (ret) {
+		lkl_test_logf("close error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_close(pipe_fds[1]);
+	if (ret) {
+		lkl_test_logf("close error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	return TEST_SUCCESS;
+}
+
+static int lkl_test_epoll(void)
+{
+	int epoll_fd, pipe_fds[2];
+	int READ_IDX = 0, WRITE_IDX = 1;
+	struct lkl_epoll_event wait_on, read_result;
+	static const char msg[] = "Hello world!";
+	long ret;
+
+	memset(&wait_on, 0, sizeof(wait_on));
+	memset(&read_result, 0, sizeof(read_result));
+
+	ret = lkl_sys_pipe2(pipe_fds, LKL_O_NONBLOCK);
+	if (ret) {
+		lkl_test_logf("pipe2 error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	epoll_fd = lkl_sys_epoll_create(1);
+	if (epoll_fd < 0) {
+		lkl_test_logf("epoll_create error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	wait_on.events = LKL_POLLIN | LKL_POLLOUT;
+	wait_on.data = pipe_fds[READ_IDX];
+
+	ret = lkl_sys_epoll_ctl(epoll_fd, LKL_EPOLL_CTL_ADD, pipe_fds[READ_IDX],
+				&wait_on);
+	if (ret < 0) {
+		lkl_test_logf("epoll_ctl error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	/* Shouldn't be ready before we have written something */
+	ret = lkl_sys_epoll_wait(epoll_fd, &read_result, 1, 0);
+	if (ret != 0) {
+		if (ret < 0)
+			lkl_test_logf("epoll_wait error: %s\n",
+				      lkl_strerror(ret));
+		else
+			lkl_test_logf("epoll_wait: bad event: 0x%lx\n", ret);
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_write(pipe_fds[WRITE_IDX], msg, strlen(msg) + 1);
+	if (ret < 0) {
+		lkl_test_logf("write error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	/* We expect exactly 1 fd to be ready immediately */
+	ret = lkl_sys_epoll_wait(epoll_fd, &read_result, 1, 0);
+	if (ret != 1) {
+		if (ret < 0)
+			lkl_test_logf("epoll_wait error: %s\n",
+				      lkl_strerror(ret));
+		else
+			lkl_test_logf("epoll_wait: bad ev no %ld\n", ret);
+		return TEST_FAILURE;
+	}
+
+	/* Already tested reading from pipe2 so no need to do it
+	 * here
+	 */
+
+	return TEST_SUCCESS;
+}
+
+LKL_TEST_CALL(chdir_proc, lkl_sys_chdir, 0, "proc");
+
+static int dir_fd;
+
+static int lkl_test_open_cwd(void)
+{
+	dir_fd = lkl_sys_open(".", LKL_O_RDONLY | LKL_O_DIRECTORY, 0);
+	if (dir_fd < 0) {
+		lkl_test_logf("failed to open current directory: %s\n",
+			      lkl_strerror(dir_fd));
+		return TEST_FAILURE;
+	}
+
+	return TEST_SUCCESS;
+}
+
+/* column where to insert a line break for the list file tests below. */
+#define COL_LINE_BREAK 70
+
+static int lkl_test_getdents64(void)
+{
+	long ret;
+	char buf[1024], *pos;
+	struct lkl_linux_dirent64 *de;
+	int wr;
+
+	de = (struct lkl_linux_dirent64 *)buf;
+	ret = lkl_sys_getdents64(dir_fd, de, sizeof(buf));
+
+	wr = lkl_test_logf("%d ", dir_fd);
+
+	if (ret < 0)
+		return TEST_FAILURE;
+
+	for (pos = buf; pos - buf < ret; pos += de->d_reclen) {
+		de = (struct lkl_linux_dirent64 *)pos;
+
+		wr += lkl_test_logf("%s ", de->d_name);
+		if (wr >= COL_LINE_BREAK) {
+			lkl_test_logf("\n");
+			wr = 0;
+		}
+	}
+
+	return TEST_SUCCESS;
+}
+
+LKL_TEST_CALL(close_dir_fd, lkl_sys_close, 0, dir_fd);
+LKL_TEST_CALL(chdir_root, lkl_sys_chdir, 0, "/");
+LKL_TEST_CALL(mount_fs_proc, lkl_sys_mount, 0, "none", "/proc", "proc", 0,
+	      NULL);
+LKL_TEST_CALL(umount_fs_proc, lkl_sys_umount, 0, "/proc", 0);
+
+LKL_TEST_CALL(start_kernel, lkl_start_kernel, 0, &lkl_host_ops,
+	     "mem=16M loglevel=8");
+LKL_TEST_CALL(stop_kernel, lkl_sys_halt, 0);
+
+struct lkl_test tests[] = {
+	LKL_TEST(start_kernel),
+	LKL_TEST(getpid),
+	LKL_TEST(syscall_latency),
+	LKL_TEST(umask),
+	LKL_TEST(umask2),
+	LKL_TEST(creat),
+	LKL_TEST(close),
+	LKL_TEST(failopen),
+	LKL_TEST(open),
+	LKL_TEST(write),
+	LKL_TEST(lseek_cur),
+	LKL_TEST(lseek_end),
+	LKL_TEST(lseek_set),
+	LKL_TEST(read),
+	LKL_TEST(fstat),
+	LKL_TEST(mkdir),
+	LKL_TEST(stat),
+#ifndef __MINGW32__
+	LKL_TEST(nanosleep),
+#endif
+	LKL_TEST(pipe2),
+	LKL_TEST(epoll),
+	LKL_TEST(mount_fs_proc),
+	LKL_TEST(chdir_proc),
+	LKL_TEST(open_cwd),
+	LKL_TEST(getdents64),
+	LKL_TEST(close_dir_fd),
+	LKL_TEST(chdir_root),
+	LKL_TEST(umount_fs_proc),
+	LKL_TEST(stop_kernel),
+};
+
+int main(int argc, const char **argv)
+{
+	lkl_host_ops.print = lkl_test_log;
+
+	return lkl_test_run(tests, sizeof(tests)/sizeof(struct lkl_test),
+			    "boot");
+}
diff --git a/tools/um/tests/boot.sh b/tools/um/tests/boot.sh
new file mode 100755
index 000000000000..d985c04b0ac1
--- /dev/null
+++ b/tools/um/tests/boot.sh
@@ -0,0 +1,9 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+source $script_dir/test.sh
+
+lkl_test_plan 1 "boot"
+lkl_test_run 1
+lkl_test_exec $script_dir/boot
diff --git a/tools/um/tests/run.py b/tools/um/tests/run.py
new file mode 100755
index 000000000000..c96ede90b6ad
--- /dev/null
+++ b/tools/um/tests/run.py
@@ -0,0 +1,172 @@
+#!/usr/bin/env python
+# SPDX-License-Identifier: GPL-2.0
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 2 of the License
+#
+# Author: Octavian Purdila <tavi@cs.pub.ro>
+#
+
+from __future__ import print_function
+
+import argparse
+import os
+import subprocess
+import sys
+import tap13
+import xml.etree.ElementTree as ET
+
+from junit_xml import TestSuite, TestCase
+
+
+class Reporter(tap13.Reporter):
+    def start(self, obj):
+        if type(obj) is tap13.Test:
+            if obj.result == "*":
+                end='\r'
+            else:
+                end='\n'
+            print("  TEST       %-8s %.50s" %
+                  (obj.result, obj.description + " " + obj.comment), end=end)
+
+        elif type(obj) is tap13.Suite:
+            if obj.tests_planned == 0:
+                status = "skip"
+            else:
+                status = ""
+            print("  SUITE      %-8s %s" % (status, obj.name))
+
+    def end(self, obj):
+        if type(obj) is tap13.Test:
+            if obj.result != "ok":
+                try:
+                    print(obj.yaml["log"], end='')
+                except:
+                    None
+
+
+mydir=os.path.dirname(os.path.realpath(__file__))
+
+tests = [
+    'boot.sh',
+]
+
+parser = argparse.ArgumentParser(description='LKL test runner')
+parser.add_argument('tests', nargs='?', action='append',
+                    help='tests to run %s' % tests)
+parser.add_argument('--junit-dir',
+                    help='directory where to store the juni suites')
+parser.add_argument('--gdb', action='store_true', default=False,
+                    help='run simple tests under gdb; implies --pass-through')
+parser.add_argument('--pass-through', action='store_true',  default=False,
+                    help='run the test without interpeting the test output')
+parser.add_argument('--valgrind', action='store_true', default=False,
+                    help='run simple tests under valgrind')
+
+args = parser.parse_args()
+if args.tests == [None]:
+    args.tests = tests
+
+if args.gdb:
+    args.pass_through=True
+    os.environ['GDB']="yes"
+
+if args.valgrind:
+    os.environ['VALGRIND']="yes"
+
+tap = tap13.Parser(Reporter())
+
+os.environ['PATH'] += ":" + mydir
+
+exit_code = 0
+
+for t in args.tests:
+    if not t:
+        continue
+    if args.pass_through:
+        print(t)
+        if subprocess.call(t, shell=True) != 0:
+            exit_code = 1
+    else:
+        p = subprocess.Popen(t, shell=True, stdout=subprocess.PIPE)
+        tap.parse(p.stdout)
+
+if args.pass_through:
+    sys.exit(exit_code)
+
+suites_count = 0
+tests_total = 0
+tests_not_ok = 0
+tests_ok = 0
+tests_skip = 0
+val_errs = 0
+val_fails = 0
+val_skips = 0
+
+for s in tap.run.suites:
+
+    junit_tests = []
+    suites_count += 1
+
+    for t in s.tests:
+        try:
+            secs = t.yaml["time_us"] / 1000000.0
+        except:
+            secs = 0
+        try:
+            log = t.yaml['log']
+        except:
+            log = ""
+
+        jt = TestCase(t.description, elapsed_sec=secs, stdout=log)
+        if t.result == 'skip':
+            jt.add_skipped_info(output=log)
+        elif t.result == 'not ok':
+            jt.add_error_info(output=log)
+
+        junit_tests.append(jt)
+
+        tests_total += 1
+        if t.result == "ok":
+            tests_ok += 1
+        elif t.result == "not ok":
+            tests_not_ok += 1
+            exit_code = 1
+        elif t.result == "skip":
+            tests_skip += 1
+
+    if args.junit_dir:
+        js = TestSuite(s.name, junit_tests)
+        with open(os.path.join(args.junit_dir, os.path.basename(s.name) + '.xml'), 'w') as f:
+            js.to_file(f, [js])
+
+        if os.getenv('VALGRIND') is not None:
+            val_xml = 'valgrind-%s.xml' % os.path.basename(s.name).replace(' ','-')
+            # skipped tests don't generate xml file
+            if os.path.exists(val_xml) is False:
+                continue
+
+            cmd = 'mv %s %s' % (val_xml, args.junit_dir)
+            subprocess.call(cmd, shell=True, )
+
+            cmd = mydir + '/valgrind2xunit.py ' + val_xml
+            subprocess.call(cmd, shell=True, cwd=args.junit_dir)
+
+            # count valgrind results
+            doc = ET.parse(os.path.join(args.junit_dir, 'valgrind-%s_xunit.xml' \
+                                        % (os.path.basename(s.name).replace(' ','-'))))
+            ts = doc.getroot()
+            val_errs += int(ts.get('errors'))
+            val_fails += int(ts.get('failures'))
+            val_skips += int(ts.get('skip'))
+
+print("Summary: %d suites run, %d tests, %d ok, %d not ok, %d skipped" %
+      (suites_count, tests_total, tests_ok, tests_not_ok, tests_skip))
+
+if os.getenv('VALGRIND') is not None:
+    print(" valgrind (memcheck): %d failures, %d skipped" % (val_fails, val_skips))
+    if val_errs or val_fails:
+        exit_code = 1
+
+sys.exit(exit_code)
diff --git a/tools/um/tests/tap13.py b/tools/um/tests/tap13.py
new file mode 100644
index 000000000000..65c73cda7ca1
--- /dev/null
+++ b/tools/um/tests/tap13.py
@@ -0,0 +1,209 @@
+#!/usr/bin/env python
+# SPDX-License-Identifier: GPL-2.0
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 2 of the License
+#
+# Author: Octavian Purdila <tavi@cs.pub.ro>
+#
+# Based on TAP13:
+#
+# Copyright 2013, Red Hat, Inc.
+# Author: Josef Skladanka <jskladan@redhat.com>
+#
+from __future__ import print_function
+
+import re
+import sys
+import yamlish
+
+
+class Reporter(object):
+
+    def start(self, obj):
+        None
+
+    def end(self, obj):
+        None
+
+
+class Test(object):
+    def __init__(self, reporter, result, id, description=None, directive=None,
+                 comment=None):
+        self.reporter = reporter
+        self.result = result
+        if directive:
+            self.result = directive.lower()
+        if id:
+            self.id = int(id)
+        else:
+            self.id = None
+        if description:
+            self.description = description
+        else:
+            self.description = ""
+        if comment:
+            self.comment = "# " + comment
+        else:
+            self.comment = ""
+        self.yaml = None
+        self._yaml_buffer = None
+        self.diagnostics = []
+
+        self.reporter.start(self)
+
+    def end(self):
+        if not self.yaml:
+            self.yaml = yamlish.load(self._yaml_buffer)
+            self.reporter.end(self)
+
+
+class Suite(object):
+    def __init__(self, reporter, start, end, explanation):
+        self.reporter = reporter
+        self.tests = []
+        self.name = explanation
+        self.tests_planned = int(end)
+
+        self.__tests_counter = 0
+        self.__tests_base = 0
+
+        self.reporter.start(self)
+
+    def newTest(self, args):
+        try:
+            self.tests[-1].end()
+        except IndexError:
+            None
+
+        if 'id' not in args or not args['id']:
+            args['id'] = self.__tests_counter
+        else:
+            args['id'] = int(args['id']) + self.__tests_base
+
+        if args['id'] < self.__tests_counter:
+            print("error: bad test id %d, fixing it" % (args['id']))
+            args['id'] = self.__tests_counter
+        # according to TAP13 specs, missing tests must be handled as 'not ok'
+        # here we add the missing tests in sequence
+        while args['id'] > (self.__tests_counter + 1):
+            comment = 'test %d not present' % self.__tests_counter
+            self.tests.append(Test(self.reporter, 'not ok',
+                                   self.__tests_counter, comment=comment))
+            self.__tests_counter += 1
+
+        if args['id'] == self.__tests_counter:
+            if args['directive']:
+                self.test().result = args['directive'].lower()
+            else:
+                self.test().result = args['result']
+            self.reporter.start(self.test())
+        else:
+            self.tests.append(Test(self.reporter, **args))
+            self.__tests_counter += 1
+
+    def test(self):
+        return self.tests[-1]
+
+    def end(self, name, planned):
+        if name == self.name:
+            self.tests_planned += int(planned)
+            self.__tests_base = self.__tests_counter
+            return False
+        try:
+            self.test().end()
+        except IndexError:
+            None
+        if len(self.tests) != self.tests_planned:
+            for i in range(len(self.tests), self.tests_planned):
+                self.tests.append(Test(self.reporter, 'not ok', i+1,
+                                       comment='test not present'))
+        return True
+
+
+class Run(object):
+
+    def __init__(self, reporter):
+        self.reporter = reporter
+        self.suites = []
+
+    def suite(self):
+        return self.suites[-1]
+
+    def test(self):
+        return self.suites[-1].tests[-1]
+
+    def newSuite(self, args):
+        new = False
+        try:
+            if self.suite().end(args['explanation'], args['end']):
+                new = True
+        except IndexError:
+            new = True
+        if new:
+            self.suites.append(Suite(self.reporter, **args))
+
+    def newTest(self, args):
+        self.suite().newTest(args)
+
+
+class Parser(object):
+    RE_PLAN = re.compile(r"^\s*(?P<start>\d+)\.\.(?P<end>\d+)\s*(#\s*(?P<explanation>.*))?\s*$")
+    RE_TEST_LINE = re.compile(r"^\s*(?P<result>(not\s+)?ok|[*]+)\s*(?P<id>\d+)?\s*(?P<description>[^#]+)?\s*(#\s*(?P<directive>TODO|SKIP)?\s*(?P<comment>.+)?)?\s*$",  re.IGNORECASE)
+    RE_EXPLANATION = re.compile(r"^\s*#\s*(?P<explanation>.+)?\s*$")
+    RE_YAMLISH_START = re.compile(r"^\s*---.*$")
+    RE_YAMLISH_END = re.compile(r"^\s*\.\.\.\s*$")
+
+    def __init__(self, reporter):
+        self.seek_test = False
+        self.in_test = False
+        self.in_yaml = False
+        self.run = Run(reporter)
+
+    def parse(self, source):
+        # to avoid input buffering
+        while True:
+            line = source.readline()
+            if not line:
+                break
+
+            if self.in_yaml:
+                if Parser.RE_YAMLISH_END.match(line):
+                    self.run.test()._yaml_buffer.append(line.strip())
+                    self.in_yaml = False
+                else:
+                    self.run.test()._yaml_buffer.append(line.rstrip())
+                continue
+
+            line = line.strip()
+
+            if self.in_test:
+                if Parser.RE_EXPLANATION.match(line):
+                    self.run.test().diagnostics.append(line)
+                    continue
+                if Parser.RE_YAMLISH_START.match(line):
+                    self.run.test()._yaml_buffer = [line.strip()]
+                    self.in_yaml = True
+                    continue
+
+            m = Parser.RE_PLAN.match(line)
+            if m:
+                self.seek_test = True
+                args = m.groupdict()
+                self.run.newSuite(args)
+                continue
+
+            if self.seek_test:
+                m = Parser.RE_TEST_LINE.match(line)
+                if m:
+                    args = m.groupdict()
+                    self.run.newTest(args)
+                    self.in_test = True
+                    continue
+
+            print(line)
+        try:
+            self.run.suite().end(None, 0)
+        except IndexError:
+            None
diff --git a/tools/um/tests/test.c b/tools/um/tests/test.c
new file mode 100644
index 000000000000..37ea21901f99
--- /dev/null
+++ b/tools/um/tests/test.c
@@ -0,0 +1,126 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <stdarg.h>
+#include <time.h>
+
+#include "test.h"
+
+/* circular log buffer */
+
+static char log_buf[0x10000];
+static char *head = log_buf, *tail = log_buf;
+
+static inline void advance(char **ptr)
+{
+	if ((unsigned int)(*ptr - log_buf) >= sizeof(log_buf))
+		*ptr = log_buf;
+	else
+		*ptr = *ptr + 1;
+}
+
+static void log_char(char c)
+{
+	*tail = c;
+	advance(&tail);
+	if (tail == head)
+		advance(&head);
+}
+
+static void print_log(void)
+{
+	char last;
+
+	printf(" log: |\n");
+	last = '\n';
+	while (head != tail) {
+		if (last == '\n')
+			printf("  ");
+		last = *head;
+		putchar(last);
+		advance(&head);
+	}
+	if (last != '\n')
+		putchar('\n');
+}
+
+int lkl_test_run(const struct lkl_test *tests, int nr, const char *fmt, ...)
+{
+	int i, ret, status = TEST_SUCCESS;
+	clock_t start, stop;
+	char name[1024];
+	va_list args;
+
+	va_start(args, fmt);
+	vsnprintf(name, sizeof(name), fmt, args);
+	va_end(args);
+
+	printf("1..%d # %s\n", nr, name);
+	for (i = 1; i <= nr; i++) {
+		const struct lkl_test *t = &tests[i-1];
+		unsigned long delta_us;
+
+		printf("* %d %s\n", i, t->name);
+		fflush(stdout);
+
+		start = clock();
+
+		ret = t->fn(t->arg1, t->arg2, t->arg3);
+
+		stop = clock();
+
+		switch (ret) {
+		case TEST_SUCCESS:
+			printf("ok %d %s\n", i, t->name);
+			break;
+		case TEST_SKIP:
+			printf("ok %d %s # SKIP\n", i, t->name);
+			break;
+		case TEST_BAILOUT:
+			status = TEST_BAILOUT;
+			/* fall through; */
+		case TEST_FAILURE:
+		default:
+			if (status != TEST_BAILOUT)
+				status = TEST_FAILURE;
+			printf("not ok %d %s\n", i, t->name);
+		}
+
+		printf(" ---\n");
+		delta_us = (stop - start) * 1000000 / CLOCKS_PER_SEC;
+		printf(" time_us: %ld\n", delta_us);
+		print_log();
+		printf(" ...\n");
+
+		if (status == TEST_BAILOUT) {
+			printf("Bail out!\n");
+			return TEST_FAILURE;
+		}
+
+		fflush(stdout);
+	}
+
+	return status;
+}
+
+
+void lkl_test_log(const char *str, int len)
+{
+	while (len--)
+		log_char(*(str++));
+}
+
+int lkl_test_logf(const char *fmt, ...)
+{
+	char tmp[1024], *c;
+	va_list args;
+	unsigned int n;
+
+	va_start(args, fmt);
+	n = vsnprintf(tmp, sizeof(tmp), fmt, args);
+	va_end(args);
+
+	for (c = tmp; *c != 0; c++)
+		log_char(*c);
+
+	return n > sizeof(tmp) ? sizeof(tmp) : n;
+}
diff --git a/tools/um/tests/test.h b/tools/um/tests/test.h
new file mode 100644
index 000000000000..f63ad6d419cb
--- /dev/null
+++ b/tools/um/tests/test.h
@@ -0,0 +1,72 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_TEST_H
+#define _LKL_TEST_H
+
+#define TEST_SUCCESS	0
+#define TEST_FAILURE	1
+#define TEST_SKIP	2
+#define TEST_TODO	3
+#define TEST_BAILOUT	4
+
+struct lkl_test {
+	const char *name;
+	int (*fn)();
+	void *arg1, *arg2, *arg3;
+};
+
+/**
+ * Simple wrapper to initialize a test entry.
+ * @name - test name, it assume test function is named test_@name
+ * @vargs - arguments to be passed to the function
+ */
+#define LKL_TEST(name, ...) { #name, lkl_test_##name, __VA_ARGS__ }
+
+/**
+ * lkl_test_run - run a test suite
+ *
+ * @tests - the list of tests to run
+ * @nr - number of tests
+ * @fmt - format string to be used for suite name
+ */
+int lkl_test_run(const struct lkl_test *tests, int nr, const char *fmt, ...);
+
+/**
+ * lkl_test_log - store a string in the test log buffer
+ * @str - the string to log (can be non-NULL terminated)
+ * @len - the string length
+ */
+void lkl_test_log(const char *str, int len);
+
+/**
+ * lkl_test_logf - printf like function to store into the test log buffer
+ * @fmt - printf format string
+ * @vargs - arguments to the format string
+ */
+int lkl_test_logf(const char *fmt, ...) __attribute__((format(printf, 1, 2)));
+
+/**
+ * LKL_TEST_CALL - create a test function as for a LKL call
+ *
+ * The test function will be named lkl_test_@name and will return
+ * TEST_SUCCESS if the called functions returns @expect. Otherwise
+ * will return TEST_FAILUIRE.
+ *
+ * @name - test name; must be unique because it is part of the the
+ * test function; the test function will be named
+ * @call - function to call
+ * @expect - expected return value for success
+ * @args - arguments to pass to the LKL call
+ */
+#define LKL_TEST_CALL(name, call, expect, ...)				\
+	static int lkl_test_##name(void)				\
+	{								\
+		long ret;						\
+									\
+		ret = call(__VA_ARGS__);				\
+		lkl_test_logf("%s(%s) = %ld %s\n", #call, #__VA_ARGS__, \
+			ret, ret < 0 ? lkl_strerror(ret) : "");		\
+		return (ret == expect) ? TEST_SUCCESS : TEST_FAILURE;	\
+	}
+
+
+#endif /* _LKL_TEST_H */
diff --git a/tools/um/tests/test.sh b/tools/um/tests/test.sh
new file mode 100644
index 000000000000..31b7de5c06f7
--- /dev/null
+++ b/tools/um/tests/test.sh
@@ -0,0 +1,181 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+basedir=$(cd $script_dir/..; pwd)
+base_objdir=$(cd ${OUTPUT}/; pwd)
+
+TEST_SUCCESS=0
+TEST_FAILURE=1
+TEST_SKIP=113
+TEST_TODO=114
+TEST_BAILOUT=115
+
+print_log()
+{
+    echo " log: |"
+    while read line; do
+        echo "  $line"
+    done < $1
+}
+
+export_vars()
+{
+    if [ -z "$var_file" ]; then
+        return
+    fi
+
+    for i in $@; do
+        echo "$i=${!i}" >> $var_file
+    done
+}
+
+lkl_test_run()
+{
+    log_file=$(mktemp)
+    export var_file=$(mktemp)
+
+    tid=$1 && shift && tname=$@
+
+    echo "* $tid $tname"
+
+    start=$(date '+%s%9N')
+    # run in a separate shell to avoid -e terminating us
+    $@ 2>&1 | strings >$log_file
+    exit=${PIPESTATUS[0]}
+    stop=$(date '+%s%9N')
+
+    case $exit in
+    $TEST_SUCCESS)
+        echo "ok $tid $tname"
+        ;;
+    $TEST_SKIP)
+        echo "ok $tid $tname # SKIP"
+        ;;
+    $TEST_BAILOUT)
+        echo "not ok $tid $tname"
+        echo "Bail out!"
+        ;;
+    $TEST_FAILURE|*)
+        echo "not ok $tid $tname"
+        ;;
+    esac
+
+    delta=$(((stop-start)/1000))
+
+    echo " ---"
+    echo " time_us: $delta"
+    print_log $log_file
+    echo -e " ..."
+
+    rm $log_file
+    . $var_file
+    rm $var_file
+
+    return $exit
+}
+
+lkl_test_plan()
+{
+    echo "1..$1 # $2"
+    export suite_name="${2// /\-}"
+}
+
+lkl_test_exec()
+{
+    local SUDO=""
+    local WRAPPER=""
+
+    if [ "$1" = "sudo" ]; then
+        SUDO=sudo
+        shift
+    fi
+
+    local file=$1
+    shift
+
+    if [ -n "$LKL_HOST_CONFIG_NT" ]; then
+        file=$file.exe
+    fi
+
+    file=${OUTPUT}/tests/$(basename $file)
+
+    if file $file | grep ARM; then
+        WRAPPER="qemu-arm-static"
+    elif file $file | grep "FreeBSD" ; then
+        ssh_copy "$file" $BSD_WDIR
+        if [ -n "$SUDO" ]; then
+            SUDO=""
+        fi
+        WRAPPER="$MYSSH $SU"
+        # ssh will mess up with pipes ('|') so, escape the pipe char.
+        args="${@//\|/\\\|}"
+        set - $BSD_WDIR/$(basename $file) $args
+        file=""
+    elif [ -n "$GDB" ]; then
+        WRAPPER="gdb"
+        args="$@"
+        set - -ex "run $args" -ex quit $file
+        file=""
+    elif [ -n "$VALGRIND" ]; then
+        WRAPPER="valgrind --suppressions=$script_dir/valgrind.supp \
+                  --leak-check=full --show-leak-kinds=all --xml=yes \
+                  --xml-file=valgrind-$suite_name.xml"
+    fi
+
+    $SUDO $WRAPPER $file "$@"
+}
+
+lkl_test_cmd()
+{
+    local WRAPPER=""
+
+    if [ -z "$QUIET" ]; then
+        SHOPTS="-x"
+    fi
+
+    if [ -n "$LKL_HOST_CONFIG_BSD" ]; then
+        WRAPPER="$MYSSH $SU"
+    fi
+
+    echo "$@" | $WRAPPER sh $SHOPTS
+}
+
+# XXX: $MYSSH and $MYSCP are defined in a circleci docker image.
+# see the definitions in lkl/lkl-docker:circleci/freebsd11/Dockerfile
+ssh_push()
+{
+    while [ -n "$1" ]; do
+        if [[ "$1" = *.sh ]]; then
+            type="script"
+        else
+            type="file"
+        fi
+
+        dir=$(dirname $1)
+        $MYSSH mkdir -p $BSD_WDIR/$dir
+
+        $MYSCP -P 7722 -r $basedir/$1 root@localhost:$BSD_WDIR/$dir
+        if [ "$type" = "script" ]; then
+            $MYSSH chmod a+x $BSD_WDIR/$1
+        fi
+
+        shift
+    done
+}
+
+ssh_copy()
+{
+    $MYSCP -P 7722 -r $1 root@localhost:$2
+}
+
+lkl_test_bsd_cleanup()
+{
+    $MYSSH rm -rf $BSD_WDIR
+}
+
+if [ -n "$LKL_HOST_CONFIG_BSD" ]; then
+    trap lkl_test_bsd_cleanup EXIT
+    export BSD_WDIR=/root/lkl
+    $MYSSH mkdir -p $BSD_WDIR
+fi
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v6 21/21] um: nommu: add block device support of UML
  2020-09-24  7:12       ` Hajime Tazaki
@ 2020-09-24  7:13         ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  7:13 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch, Hajime Tazaki

This commit adds block device support for nommu mode, and also added
several host utilities to run a simple test program
(tools/um/test/disk.c) successfully.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/include/asm/xor.h                 |   3 +-
 arch/um/include/shared/as-layout.h        |   1 +
 arch/um/kernel/um_arch.c                  |   5 +
 arch/um/nommu/include/uapi/asm/host_ops.h |   7 +
 arch/um/nommu/include/uapi/asm/unistd.h   |   2 +
 arch/um/nommu/um/Kconfig                  |   8 +
 arch/um/nommu/um/setup.c                  |  12 +
 tools/um/Targets                          |   2 +
 tools/um/include/lkl.h                    | 206 ++++++++++
 tools/um/include/lkl_host.h               |   1 +
 tools/um/lib/Build                        |   1 +
 tools/um/lib/fs.c                         | 461 ++++++++++++++++++++++
 tools/um/lib/posix-host.c                 |   1 +
 tools/um/lib/utils.c                      |   3 +
 tools/um/tests/Build                      |   1 +
 tools/um/tests/cla.c                      | 159 ++++++++
 tools/um/tests/cla.h                      |  33 ++
 tools/um/tests/disk.c                     | 168 ++++++++
 tools/um/tests/disk.sh                    |  70 ++++
 tools/um/tests/run.py                     |   2 +
 20 files changed, 1145 insertions(+), 1 deletion(-)
 create mode 100644 tools/um/lib/fs.c
 create mode 100644 tools/um/tests/cla.c
 create mode 100644 tools/um/tests/cla.h
 create mode 100644 tools/um/tests/disk.c
 create mode 100755 tools/um/tests/disk.sh

diff --git a/arch/um/include/asm/xor.h b/arch/um/include/asm/xor.h
index 36b33d62a35d..a5ea458a1ae9 100644
--- a/arch/um/include/asm/xor.h
+++ b/arch/um/include/asm/xor.h
@@ -4,4 +4,5 @@
 
 /* pick an arbitrary one - measuring isn't possible with inf-cpu */
 #define XOR_SELECT_TEMPLATE(x)	\
-	(time_travel_mode == TT_MODE_INFCPU ? &xor_block_8regs : NULL)
+	(time_travel_mode == TT_MODE_INFCPU || (!IS_ENABLED(CONFIG_MMU)) ? \
+	 &xor_block_8regs : NULL)
diff --git a/arch/um/include/shared/as-layout.h b/arch/um/include/shared/as-layout.h
index 5f286ef2721b..4423437a5ace 100644
--- a/arch/um/include/shared/as-layout.h
+++ b/arch/um/include/shared/as-layout.h
@@ -57,6 +57,7 @@ extern unsigned long host_task_size;
 
 extern int linux_main(int argc, char **argv);
 extern void uml_finishsetup(void);
+extern void uml_set_args(char *args);
 
 struct siginfo;
 extern void (*sig_info[])(int, struct siginfo *si, struct uml_pt_regs *);
diff --git a/arch/um/kernel/um_arch.c b/arch/um/kernel/um_arch.c
index e2cb76c03b25..da6e06cf2808 100644
--- a/arch/um/kernel/um_arch.c
+++ b/arch/um/kernel/um_arch.c
@@ -41,6 +41,11 @@ static void __init add_arg(char *arg)
 	strcat(command_line, arg);
 }
 
+void __init uml_set_args(char *args)
+{
+	strcat(command_line, args);
+}
+
 /*
  * These fields are initialized at boot time and not changed.
  * XXX This structure is used only in the non-SMP case.  Maybe this
diff --git a/arch/um/nommu/include/uapi/asm/host_ops.h b/arch/um/nommu/include/uapi/asm/host_ops.h
index 0811494c080c..1b2507502969 100644
--- a/arch/um/nommu/include/uapi/asm/host_ops.h
+++ b/arch/um/nommu/include/uapi/asm/host_ops.h
@@ -15,6 +15,11 @@ struct lkl_jmp_buf {
  *
  * These operations must be provided by a host library or by the application
  * itself.
+ *
+ * @um_devices - string containg the list of UML devices in command line
+ * format. This string is appended to the kernel command line and
+ * is provided here for convenience to be implemented by the host library.
+ *
  * @print - optional operation that receives console messages
  * @panic - called during a kernel panic
  *
@@ -65,6 +70,8 @@ struct lkl_jmp_buf {
  *
  */
 struct lkl_host_operations {
+	const char *um_devices;
+
 	void (*print)(const char *str, int len);
 	void (*panic)(void);
 
diff --git a/arch/um/nommu/include/uapi/asm/unistd.h b/arch/um/nommu/include/uapi/asm/unistd.h
index 1762ebb829f5..320762099a62 100644
--- a/arch/um/nommu/include/uapi/asm/unistd.h
+++ b/arch/um/nommu/include/uapi/asm/unistd.h
@@ -3,6 +3,8 @@
 #define __UM_NOMMU_UAPI_UNISTD_H
 
 #define __ARCH_WANT_NEW_STAT
+#define __ARCH_WANT_SET_GET_RLIMIT
+
 #include <asm/bitsperlong.h>
 
 #if __BITS_PER_LONG == 64
diff --git a/arch/um/nommu/um/Kconfig b/arch/um/nommu/um/Kconfig
index 20b3eaccb6f0..c6a3f472fe75 100644
--- a/arch/um/nommu/um/Kconfig
+++ b/arch/um/nommu/um/Kconfig
@@ -4,6 +4,10 @@ config UML_NOMMU
 	select UACCESS_MEMCPY
 	select ARCH_THREAD_STACK_ALLOCATOR
 	select ARCH_HAS_SYSCALL_WRAPPER
+	select VFAT_FS
+	select NLS_CODEPAGE_437
+	select NLS_ISO8859_1
+	select BTRFS_FS
 
 config 64BIT
 	bool
@@ -35,3 +39,7 @@ config STACKTRACE_SUPPORT
 config PRINTK_TIME
 	bool
 	default y
+
+config RAID6_PQ_BENCHMARK
+	bool
+	default n
diff --git a/arch/um/nommu/um/setup.c b/arch/um/nommu/um/setup.c
index 922188690139..d71d61979ad6 100644
--- a/arch/um/nommu/um/setup.c
+++ b/arch/um/nommu/um/setup.c
@@ -45,13 +45,25 @@ static void __init *lkl_run_kernel(void *arg)
 	return NULL;
 }
 
+static char _cmd_line[COMMAND_LINE_SIZE];
 int __init lkl_start_kernel(struct lkl_host_operations *ops,
 			    const char *fmt, ...)
 {
+	va_list ap;
 	int ret;
 
 	lkl_ops = ops;
 
+	va_start(ap, fmt);
+	ret = vsnprintf(_cmd_line, COMMAND_LINE_SIZE, fmt, ap);
+	va_end(ap);
+
+	if (ops->um_devices)
+		strscpy(_cmd_line + ret, ops->um_devices,
+			COMMAND_LINE_SIZE - ret);
+
+	uml_set_args(_cmd_line);
+
 	init_sem = lkl_ops->sem_alloc(0);
 	if (!init_sem)
 		return -ENOMEM;
diff --git a/tools/um/Targets b/tools/um/Targets
index 2bb90381843c..f5f8ec4b9dbb 100644
--- a/tools/um/Targets
+++ b/tools/um/Targets
@@ -1,10 +1,12 @@
 ifeq ($(UMMODE),library)
 progs-y += tests/boot
+progs-y += tests/disk
 else
 progs-y += uml/linux
 endif
 
 LDFLAGS_boot-y := -pie
+LDFLAGS_disk-y := -pie
 
 LDLIBS := -lrt -lpthread -lutil
 LDFLAGS_linux-y := -no-pie -Wl,--wrap,malloc -Wl,--wrap,free -Wl,--wrap,calloc
diff --git a/tools/um/include/lkl.h b/tools/um/include/lkl.h
index 707e01b64a70..4a9d874bfbf0 100644
--- a/tools/um/include/lkl.h
+++ b/tools/um/include/lkl.h
@@ -113,6 +113,16 @@ static inline long lkl_sys_creat(const char *file, int mode)
 }
 #endif
 
+#ifdef __lkl__NR_faccessat
+/**
+ * lkl_sys_access - wrapper for lkl_sys_faccessat
+ */
+static inline long lkl_sys_access(const char *file, int mode)
+{
+	return lkl_sys_faccessat(LKL_AT_FDCWD, file, mode);
+}
+#endif
+
 #ifdef __lkl__NR_mkdirat
 /**
  * lkl_sys_mkdir - wrapper for lkl_sys_mkdirat
@@ -123,6 +133,36 @@ static inline long lkl_sys_mkdir(const char *path, mode_t mode)
 }
 #endif
 
+#ifdef __lkl__NR_unlinkat
+/**
+ * lkl_sys_rmdir - wrapper for lkl_sys_unlinkrat
+ */
+static inline long lkl_sys_rmdir(const char *path)
+{
+	return lkl_sys_unlinkat(LKL_AT_FDCWD, path, LKL_AT_REMOVEDIR);
+}
+#endif
+
+#ifdef __lkl__NR_unlinkat
+/**
+ * lkl_sys_unlink - wrapper for lkl_sys_unlinkat
+ */
+static inline long lkl_sys_unlink(const char *path)
+{
+	return lkl_sys_unlinkat(LKL_AT_FDCWD, path, 0);
+}
+#endif
+
+#ifdef __lkl__NR_mknodat
+/**
+ * lkl_sys_mknod - wrapper for lkl_sys_mknodat
+ */
+static inline long lkl_sys_mknod(const char *path, mode_t mode, dev_t dev)
+{
+	return lkl_sys_mknodat(LKL_AT_FDCWD, path, mode, dev);
+}
+#endif
+
 #ifdef __lkl__NR_epoll_create1
 /**
  * lkl_sys_epoll_create - wrapper for lkl_sys_epoll_create1
@@ -144,6 +184,172 @@ static inline long lkl_sys_epoll_wait(int fd, struct lkl_epoll_event *ev,
 }
 #endif
 
+/**
+ * struct lkl_dev_blk_ops - block device host operations, defined in lkl_host.h.
+ */
+struct lkl_dev_blk_ops;
+
+/**
+ * lkl_disk - host disk handle
+ *
+ * @dev - a pointer to private information for this disk backend
+ * @fd - a POSIX file descriptor that can be used by preadv/pwritev
+ * @handle - an NT file handle that can be used by ReadFile/WriteFile
+ */
+struct lkl_disk {
+	void *dev;
+	union {
+		int fd;
+		void *handle;
+	};
+	struct lkl_dev_blk_ops *ops;
+};
+
+/**
+ * lkl_disk_add - add a new disk
+ *
+ * @disk - the host disk handle
+ * @returns a disk id (0 is valid) or a strictly negative value in case of error
+ */
+int lkl_disk_add(struct lkl_disk *disk);
+
+/**
+ * lkl_disk_remove - remove a disk
+ *
+ * This function makes a cleanup of the @disk's private information
+ * that was initialized by lkl_disk_add before.
+ *
+ * @disk - the host disk handle
+ */
+int lkl_disk_remove(struct lkl_disk disk);
+
+/**
+ * lkl_encode_dev_from_sysfs_blkdev - extract device id from sysfs
+ *
+ * This function returns the device id for the given sysfs dev node.
+ * The content of the node has to be in the form 'MAJOR:MINOR'.
+ * Also, this function expects an absolute path which means that sysfs
+ * already has to be mounted at the given path
+ *
+ * @sysfs_path - absolute path to the sysfs dev node
+ * @pdevid - pointer to memory where dev id will be returned
+ * @returns - 0 on success, a negative value on error
+ */
+int lkl_encode_dev_from_sysfs(const char *sysfs_path, uint32_t *pdevid);
+
+/**
+ * lkl_mount_dev - mount a disk
+ *
+ * This functions creates a device file for the given disk, creates a mount
+ * point and mounts the device over the mount point.
+ *
+ * @disk_id - the disk id identifying the disk to be mounted
+ * @part - disk partition or zero for full disk
+ * @fs_type - filesystem type
+ * @flags - mount flags
+ * @opts - additional filesystem specific mount options
+ * @mnt_str - a string that will be filled by this function with the path where
+ * the filesystem has been mounted
+ * @mnt_str_len - size of mnt_str
+ * @returns - 0 on success, a negative value on error
+ */
+long lkl_mount_dev(unsigned int disk_id, unsigned int part, const char *fs_type,
+		   int flags, const char *opts,
+		   char *mnt_str, unsigned int mnt_str_len);
+
+/**
+ * lkl_umount_dev - umount a disk
+ *
+ * This functions umounts the given disks and removes the device file and the
+ * mount point.
+ *
+ * @disk_id - the disk id identifying the disk to be mounted
+ * @part - disk partition or zero for full disk
+ * @flags - umount flags
+ * @timeout_ms - timeout to wait for the kernel to flush closed files so that
+ * umount can succeed
+ * @returns - 0 on success, a negative value on error
+ */
+long lkl_umount_dev(unsigned int disk_id, unsigned int part, int flags,
+		    long timeout_ms);
+
+/**
+ * lkl_umount_timeout - umount filesystem with timeout
+ *
+ * @path - the path to unmount
+ * @flags - umount flags
+ * @timeout_ms - timeout to wait for the kernel to flush closed files so that
+ * umount can succeed
+ * @returns - 0 on success, a negative value on error
+ */
+long lkl_umount_timeout(char *path, int flags, long timeout_ms);
+
+/**
+ * lkl_opendir - open a directory
+ *
+ * @path - directory path
+ * @err - pointer to store the error in case of failure
+ * @returns - a handle to be used when calling lkl_readdir
+ */
+struct lkl_dir *lkl_opendir(const char *path, int *err);
+
+/**
+ * lkl_fdopendir - open a directory
+ *
+ * @fd - file descriptor
+ * @err - pointer to store the error in case of failure
+ * @returns - a handle to be used when calling lkl_readdir
+ */
+struct lkl_dir *lkl_fdopendir(int fd, int *err);
+
+/**
+ * lkl_rewinddir - reset directory stream
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ */
+void lkl_rewinddir(struct lkl_dir *dir);
+
+/**
+ * lkl_closedir - close the directory
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ */
+int lkl_closedir(struct lkl_dir *dir);
+
+/**
+ * lkl_readdir - get the next available entry of the directory
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ * @returns - a lkl_dirent64 entry or NULL if the end of the directory stream is
+ * reached or if an error occurred; check lkl_errdir() to distinguish between
+ * errors or end of the directory stream
+ */
+struct lkl_linux_dirent64 *lkl_readdir(struct lkl_dir *dir);
+
+/**
+ * lkl_errdir - checks if an error occurred during the last lkl_readdir call
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ * @returns - 0 if no error occurred, or a negative value otherwise
+ */
+int lkl_errdir(struct lkl_dir *dir);
+
+/**
+ * lkl_dirfd - gets the file descriptor associated with the directory handle
+ *
+ * @dir - the directory handle as returned by lkl_opendir
+ * @returns - a positive value,which is the LKL file descriptor associated with
+ * the directory handle, or a negative value otherwise
+ */
+int lkl_dirfd(struct lkl_dir *dir);
+
+/**
+ * lkl_mount_fs - mount a file system type like proc, sys
+ * @fstype - file system type. e.g. proc, sys
+ * @returns - 0 on success. 1 if it's already mounted. negative on failure.
+ */
+int lkl_mount_fs(char *fstype);
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/tools/um/include/lkl_host.h b/tools/um/include/lkl_host.h
index 85e80eb4ad0d..12dd95616e43 100644
--- a/tools/um/include/lkl_host.h
+++ b/tools/um/include/lkl_host.h
@@ -10,6 +10,7 @@ extern "C" {
 #include <lkl.h>
 
 extern struct lkl_host_operations lkl_host_ops;
+extern char lkl_um_devs[4096];
 
 /**
  * lkl_printf - print a message via the host print operation
diff --git a/tools/um/lib/Build b/tools/um/lib/Build
index dddff26a3b4e..29491b40746c 100644
--- a/tools/um/lib/Build
+++ b/tools/um/lib/Build
@@ -4,3 +4,4 @@ CFLAGS_posix-host.o += -D_FILE_OFFSET_BITS=64
 liblinux-$(CONFIG_UMMODE_LIB) += utils.o
 liblinux-$(CONFIG_UMMODE_LIB) += posix-host.o
 liblinux-$(CONFIG_UMMODE_LIB) += jmp_buf.o
+liblinux-$(CONFIG_UMMODE_LIB) += fs.o
diff --git a/tools/um/lib/fs.c b/tools/um/lib/fs.c
new file mode 100644
index 000000000000..948aac9730c2
--- /dev/null
+++ b/tools/um/lib/fs.c
@@ -0,0 +1,461 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdarg.h>
+#include <stdio.h>
+#include <string.h>
+#include <stdlib.h>
+#include <lkl_host.h>
+
+#define MAX_FSTYPE_LEN 50
+
+static struct lkl_disk *lkl_disks[16];
+static int registered_blk_dev_idx;
+
+static int lkl_disk_um_add(struct lkl_disk *disk, const char *blkparams)
+{
+	/* concat strings */
+	snprintf(lkl_um_devs + strlen(lkl_um_devs), sizeof(lkl_um_devs),
+		 " ubd%d=%s", registered_blk_dev_idx, blkparams);
+
+	return registered_blk_dev_idx++;
+}
+
+int lkl_disk_add(struct lkl_disk *disk)
+{
+	int ret = -1;
+
+	ret = lkl_disk_um_add(disk, disk->dev);
+
+	lkl_disks[ret] = disk;
+
+	return ret;
+}
+
+int lkl_disk_remove(struct lkl_disk disk)
+{
+	/* FIXME */
+	return 0;
+}
+
+int lkl_mount_fs(char *fstype)
+{
+	char dir[MAX_FSTYPE_LEN+2] = "/";
+	int flags = 0, ret = 0;
+
+	strncat(dir, fstype, MAX_FSTYPE_LEN);
+
+	/* Create with regular umask */
+	ret = lkl_sys_mkdir(dir, 0xff);
+	if (ret && ret != -LKL_EEXIST) {
+		lkl_perror("mount_fs mkdir", ret);
+		return ret;
+	}
+
+	/* We have no use for nonzero flags right now */
+	ret = lkl_sys_mount("none", dir, fstype, flags, NULL);
+	if (ret && ret != -LKL_EBUSY) {
+		lkl_sys_rmdir(dir);
+		return ret;
+	}
+
+	if (ret == -LKL_EBUSY)
+		return 1;
+	return 0;
+}
+
+static uint32_t new_encode_dev(unsigned int major, unsigned int minor)
+{
+	return (minor & 0xff) | (major << 8) | ((minor & ~0xff) << 12);
+}
+
+static int startswith(const char *str, const char *pre)
+{
+	return strncmp(pre, str, strlen(pre)) == 0;
+}
+
+static int get_node_with_prefix(const char *path, const char *prefix,
+				char *result, unsigned int result_len)
+{
+	struct lkl_dir *dir = NULL;
+	struct lkl_linux_dirent64 *dirent;
+	int ret;
+
+	dir = lkl_opendir(path, &ret);
+	if (!dir)
+		return ret;
+
+	ret = -LKL_ENOENT;
+
+	while ((dirent = lkl_readdir(dir))) {
+		if (startswith(dirent->d_name, prefix)) {
+			if (strlen(dirent->d_name) + 1 > result_len) {
+				ret = -LKL_ENOMEM;
+				break;
+			}
+			memcpy(result, dirent->d_name, strlen(dirent->d_name));
+			result[strlen(dirent->d_name)] = '\0';
+			ret = 0;
+			break;
+		}
+	}
+
+	lkl_closedir(dir);
+
+	return ret;
+}
+
+int lkl_encode_dev_from_sysfs(const char *sysfs_path, uint32_t *pdevid)
+{
+	int ret;
+	long fd;
+	int major, minor;
+	char buf[16] = { 0, };
+	char *bufptr;
+
+	fd = lkl_sys_open(sysfs_path, LKL_O_RDONLY, 0);
+	if (fd < 0)
+		return fd;
+
+	ret = lkl_sys_read(fd, buf, sizeof(buf));
+	if (ret < 0)
+		goto out_close;
+
+	if (ret == sizeof(buf)) {
+		ret = -LKL_ENOBUFS;
+		goto out_close;
+	}
+
+	bufptr = strchr(buf, ':');
+	if (bufptr == NULL) {
+		ret = -LKL_EINVAL;
+		goto out_close;
+	}
+	bufptr[0] = '\0';
+	bufptr++;
+
+	major = atoi(buf);
+	minor = atoi(bufptr);
+
+	*pdevid = new_encode_dev(major, minor);
+	ret = 0;
+
+out_close:
+	lkl_sys_close(fd);
+
+	return ret;
+}
+
+#define SYSFS_DEV_UMBLK_CMDLINE_PATH \
+	"/sysfs/devices/platform/uml-blkdev.%d"
+
+struct abuf {
+	char *mem, *ptr;
+	unsigned int len;
+};
+
+static int snprintf_append(struct abuf *buf, const char *fmt, ...)
+{
+	int ret;
+	va_list args;
+
+	if (!buf->ptr)
+		buf->ptr = buf->mem;
+
+	va_start(args, fmt);
+	ret = vsnprintf(buf->ptr, buf->len - (buf->ptr - buf->mem), fmt, args);
+	va_end(args);
+
+	if (ret < 0 || (ret >= (int)(buf->len - (buf->ptr - buf->mem))))
+		return -LKL_ENOMEM;
+
+	buf->ptr += ret;
+
+	return 0;
+}
+
+static int __lkl_get_blkdev(int disk_id, unsigned int part, uint32_t *pdevid,
+			    const char *sysfs_path_fmt, const char *drv_prefix,
+			    const char *disk_prefix)
+{
+	char sysfs_path[LKL_PATH_MAX];
+	char drv_name[LKL_PATH_MAX];
+	char disk_name[LKL_PATH_MAX];
+	struct abuf sysfs_path_buf = {
+		.mem = sysfs_path,
+		.len = sizeof(sysfs_path),
+	};
+	int ret;
+
+	if (disk_id < 0)
+		return -LKL_EINVAL;
+
+	ret = lkl_mount_fs("sysfs");
+	if (ret < 0)
+		return ret;
+
+	ret = snprintf_append(&sysfs_path_buf, sysfs_path_fmt, disk_id);
+	if (ret)
+		return ret;
+
+	ret = get_node_with_prefix(sysfs_path, drv_prefix, drv_name,
+				   sizeof(drv_name));
+	if (ret)
+		return ret;
+
+	ret = snprintf_append(&sysfs_path_buf, "/%s/block", drv_name);
+	if (ret)
+		return ret;
+
+	ret = get_node_with_prefix(sysfs_path, disk_prefix, disk_name,
+				   sizeof(disk_name));
+	if (ret)
+		return ret;
+
+	if (!part)
+		ret = snprintf_append(&sysfs_path_buf, "/%s/dev", disk_name);
+	else
+		ret = snprintf_append(&sysfs_path_buf, "/%s/%s%d/dev",
+				      disk_name, disk_name, part);
+	if (ret)
+		return ret;
+
+	return lkl_encode_dev_from_sysfs(sysfs_path, pdevid);
+}
+
+int lkl_get_blkdev(int disk_id, unsigned int part, uint32_t *pdevid)
+{
+	char *fmt;
+
+	fmt = SYSFS_DEV_UMBLK_CMDLINE_PATH;
+	return __lkl_get_blkdev(disk_id, part, pdevid, fmt, "", "ubd");
+}
+
+long lkl_mount_dev(unsigned int disk_id, unsigned int part,
+		   const char *fs_type, int flags,
+		   const char *data, char *mnt_str, unsigned int mnt_str_len)
+{
+	char dev_str[] = { "/dev/xxxxxxxx" };
+	unsigned int dev;
+	int err;
+	char _data[4096]; /* FIXME: PAGE_SIZE is not exported by LKL */
+
+	if (mnt_str_len < sizeof(dev_str))
+		return -LKL_ENOMEM;
+
+	err = lkl_get_blkdev(disk_id, part, &dev);
+	if (err < 0)
+		return err;
+
+	snprintf(dev_str, sizeof(dev_str), "/dev/%08x", dev);
+	snprintf(mnt_str, mnt_str_len, "/mnt/%08x", dev);
+
+	err = lkl_sys_access("/dev", LKL_S_IRWXO);
+	if (err < 0) {
+		if (err == -LKL_ENOENT)
+			err = lkl_sys_mkdir("/dev", 0700);
+		if (err < 0)
+			return err;
+	}
+
+	err = lkl_sys_mknod(dev_str, LKL_S_IFBLK | 0600, dev);
+	if (err < 0)
+		return err;
+
+	err = lkl_sys_access("/mnt", LKL_S_IRWXO);
+	if (err < 0) {
+		if (err == -LKL_ENOENT)
+			err = lkl_sys_mkdir("/mnt", 0700);
+		if (err < 0)
+			return err;
+	}
+
+	err = lkl_sys_mkdir(mnt_str, 0700);
+	if (err < 0) {
+		lkl_sys_unlink(dev_str);
+		return err;
+	}
+
+	/* kernel always copies a full page */
+	if (data) {
+		strncpy(_data, data, sizeof(_data));
+		_data[sizeof(_data) - 1] = 0;
+	} else {
+		_data[0] = 0;
+	}
+
+	err = lkl_sys_mount(dev_str, mnt_str, (char *)fs_type, flags, _data);
+	if (err < 0) {
+		lkl_sys_unlink(dev_str);
+		lkl_sys_rmdir(mnt_str);
+		return err;
+	}
+
+	return 0;
+}
+
+long lkl_umount_timeout(char *path, int flags, long timeout_ms)
+{
+	long incr = 10000000; /* 10 ms */
+	struct lkl_timespec ts = {
+		.tv_sec = 0,
+		.tv_nsec = incr,
+	};
+	long err;
+
+	do {
+		err = lkl_sys_umount(path, flags);
+		if (err == -LKL_EBUSY) {
+			lkl_sys_nanosleep((struct __lkl__kernel_timespec *)&ts,
+					  NULL);
+			timeout_ms -= incr / 1000000;
+		}
+	} while (err == -LKL_EBUSY && timeout_ms > 0);
+
+	return err;
+}
+
+long lkl_umount_dev(unsigned int disk_id, unsigned int part, int flags,
+		    long timeout_ms)
+{
+	char dev_str[] = { "/dev/xxxxxxxx" };
+	char mnt_str[] = { "/mnt/xxxxxxxx" };
+	unsigned int dev;
+	int err;
+
+	err = lkl_get_blkdev(disk_id, part, &dev);
+	if (err < 0)
+		return err;
+
+	snprintf(dev_str, sizeof(dev_str), "/dev/%08x", dev);
+	snprintf(mnt_str, sizeof(mnt_str), "/mnt/%08x", dev);
+
+	err = lkl_umount_timeout(mnt_str, flags, timeout_ms);
+	if (err)
+		return err;
+
+	err = lkl_sys_unlink(dev_str);
+	if (err)
+		return err;
+
+	return lkl_sys_rmdir(mnt_str);
+}
+
+struct lkl_dir {
+	int fd;
+	char buf[1024];
+	char *pos;
+	int len;
+};
+
+static struct lkl_dir *lkl_dir_alloc(int *err)
+{
+	struct lkl_dir *dir = lkl_host_ops.mem_alloc(sizeof(struct lkl_dir));
+
+	if (!dir) {
+		*err = -LKL_ENOMEM;
+		return NULL;
+	}
+
+	dir->len = 0;
+	dir->pos = NULL;
+
+	return dir;
+}
+
+struct lkl_dir *lkl_opendir(const char *path, int *err)
+{
+	struct lkl_dir *dir = lkl_dir_alloc(err);
+
+	if (!dir) {
+		*err = -LKL_ENOMEM;
+		return NULL;
+	}
+
+	dir->fd = lkl_sys_open(path, LKL_O_RDONLY | LKL_O_DIRECTORY, 0);
+	if (dir->fd < 0) {
+		*err = dir->fd;
+		lkl_host_ops.mem_free(dir);
+		return NULL;
+	}
+
+	*err = 0;
+
+	return dir;
+}
+
+struct lkl_dir *lkl_fdopendir(int fd, int *err)
+{
+	struct lkl_dir *dir = lkl_dir_alloc(err);
+
+	if (!dir)
+		return NULL;
+
+	dir->fd = fd;
+
+	return dir;
+}
+
+void lkl_rewinddir(struct lkl_dir *dir)
+{
+	lkl_sys_lseek(dir->fd, 0, LKL_SEEK_SET);
+	dir->len = 0;
+	dir->pos = NULL;
+}
+
+int lkl_closedir(struct lkl_dir *dir)
+{
+	int ret;
+
+	ret = lkl_sys_close(dir->fd);
+	lkl_host_ops.mem_free(dir);
+
+	return ret;
+}
+
+struct lkl_linux_dirent64 *lkl_readdir(struct lkl_dir *dir)
+{
+	struct lkl_linux_dirent64 *de;
+
+	if (dir->len < 0)
+		return NULL;
+
+	if (!dir->pos || dir->pos - dir->buf >= dir->len)
+		goto read_buf;
+
+return_de:
+	de = (struct lkl_linux_dirent64 *)dir->pos;
+	dir->pos += de->d_reclen;
+
+	return de;
+
+read_buf:
+	dir->pos = NULL;
+	de = (struct lkl_linux_dirent64 *)dir->buf;
+	dir->len = lkl_sys_getdents64(dir->fd, de, sizeof(dir->buf));
+	if (dir->len <= 0)
+		return NULL;
+
+	dir->pos = dir->buf;
+	goto return_de;
+}
+
+int lkl_errdir(struct lkl_dir *dir)
+{
+	if (dir->len >= 0)
+		return 0;
+
+	return dir->len;
+}
+
+int lkl_dirfd(struct lkl_dir *dir)
+{
+	return dir->fd;
+}
+
+int lkl_set_fd_limit(unsigned int fd_limit)
+{
+	struct lkl_rlimit rlim = {
+		.rlim_cur = fd_limit,
+		.rlim_max = fd_limit,
+	};
+	return lkl_sys_setrlimit(LKL_RLIMIT_NOFILE, &rlim);
+}
diff --git a/tools/um/lib/posix-host.c b/tools/um/lib/posix-host.c
index b6b5b2902254..8fd88031bf2b 100644
--- a/tools/um/lib/posix-host.c
+++ b/tools/um/lib/posix-host.c
@@ -263,6 +263,7 @@ static void *tls_get(struct lkl_tls_key *key)
 }
 
 struct lkl_host_operations lkl_host_ops = {
+	.um_devices = lkl_um_devs,
 	.panic = panic,
 	.print = print,
 	.mem_alloc = (void *)malloc,
diff --git a/tools/um/lib/utils.c b/tools/um/lib/utils.c
index 4930479a8a35..ac65cd744a14 100644
--- a/tools/um/lib/utils.c
+++ b/tools/um/lib/utils.c
@@ -4,6 +4,9 @@
 #include <string.h>
 #include <lkl_host.h>
 
+/* XXX: find a better place */
+char lkl_um_devs[4096];
+
 static const char * const lkl_err_strings[] = {
 	"Success",
 	"Operation not permitted",
diff --git a/tools/um/tests/Build b/tools/um/tests/Build
index 26a607f90dc8..b62919cadc50 100644
--- a/tools/um/tests/Build
+++ b/tools/um/tests/Build
@@ -1 +1,2 @@
 boot-y += boot.o test.o
+disk-y += disk.o test.o cla.o
diff --git a/tools/um/tests/cla.c b/tools/um/tests/cla.c
new file mode 100644
index 000000000000..694e3a3822be
--- /dev/null
+++ b/tools/um/tests/cla.c
@@ -0,0 +1,159 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <string.h>
+#include <errno.h>
+#include <stdlib.h>
+#ifdef __MINGW32__
+#include <winsock2.h>
+#else
+#include <sys/socket.h>
+#include <netinet/in.h>
+#include <arpa/inet.h>
+#endif
+
+#include "cla.h"
+
+static int cl_arg_parse_bool(struct cl_arg *arg, const char *value)
+{
+	*((int *)arg->store) = 1;
+	return 0;
+}
+
+static int cl_arg_parse_str(struct cl_arg *arg, const char *value)
+{
+	*((const char **)arg->store) = value;
+	return 0;
+}
+
+static int cl_arg_parse_int(struct cl_arg *arg, const char *value)
+{
+	errno = 0;
+	*((int *)arg->store) = strtol(value, NULL, 0);
+	return errno == 0;
+}
+
+static int cl_arg_parse_str_set(struct cl_arg *arg, const char *value)
+{
+	const char **set = arg->set;
+	int i;
+
+	for (i = 0; set[i] != NULL; i++) {
+		if (strcmp(set[i], value) == 0) {
+			*((int *)arg->store) = i;
+			return 0;
+		}
+	}
+
+	return (-1);
+}
+
+static int cl_arg_parse_ipv4(struct cl_arg *arg, const char *value)
+{
+	unsigned int addr;
+
+	if (!value)
+		return (-1);
+
+	addr = inet_addr(value);
+	if (addr == INADDR_NONE)
+		return (-1);
+	*((unsigned int *)arg->store) = addr;
+	return 0;
+}
+
+static cl_arg_parser_t parsers[] = {
+	[CL_ARG_BOOL] = cl_arg_parse_bool,
+	[CL_ARG_INT] = cl_arg_parse_int,
+	[CL_ARG_STR] = cl_arg_parse_str,
+	[CL_ARG_STR_SET] = cl_arg_parse_str_set,
+	[CL_ARG_IPV4] = cl_arg_parse_ipv4,
+};
+
+static struct cl_arg *find_short_arg(char name, struct cl_arg *args)
+{
+	struct cl_arg *arg;
+
+	for (arg = args; arg->short_name != 0; arg++) {
+		if (arg->short_name == name)
+			return arg;
+	}
+
+	return NULL;
+}
+
+static struct cl_arg *find_long_arg(const char *name, struct cl_arg *args)
+{
+	struct cl_arg *arg;
+
+	for (arg = args; arg->long_name; arg++) {
+		if (strcmp(arg->long_name, name) == 0)
+			return arg;
+	}
+
+	return NULL;
+}
+
+static void print_help(struct cl_arg *args)
+{
+	struct cl_arg *arg;
+
+	fprintf(stderr, "usage:\n");
+	for (arg = args; arg->long_name; arg++) {
+		fprintf(stderr, "-%c, --%-20s %s", arg->short_name,
+			arg->long_name, arg->help);
+		if (arg->type == CL_ARG_STR_SET) {
+			const char **set = arg->set;
+
+			fprintf(stderr, " [ ");
+			while (*set != NULL)
+				fprintf(stderr, "%s ", *(set++));
+			fprintf(stderr, "]");
+		}
+		fprintf(stderr, "\n");
+	}
+}
+
+int cla_parse_args(int argc, const char **argv, struct cl_arg *args)
+{
+	int i;
+
+	for (i = 1; i < argc; i++) {
+		struct cl_arg *arg = NULL;
+		cl_arg_parser_t parser;
+
+		if (argv[i][0] == '-') {
+			if (argv[i][1] != '-')
+				arg = find_short_arg(argv[i][1], args);
+			else
+				arg = find_long_arg(&argv[i][2], args);
+		}
+
+		if (!arg) {
+			fprintf(stderr, "unknown option '%s'\n", argv[i]);
+			print_help(args);
+			return (-1);
+		}
+
+		if (arg->type == CL_ARG_USER || arg->type >= CL_ARG_END)
+			parser = arg->parser;
+		else
+			parser = parsers[arg->type];
+
+		if (!parser) {
+			fprintf(stderr, "can't parse --'%s'/-'%c'\n",
+				arg->long_name, args->short_name);
+			return (-1);
+		}
+
+		if (parser(arg, argv[i + 1]) < 0) {
+			fprintf(stderr, "can't parse '%s'\n", argv[i]);
+			print_help(args);
+			return (-1);
+		}
+
+		if (arg->has_arg)
+			i++;
+	}
+
+	return 0;
+}
diff --git a/tools/um/tests/cla.h b/tools/um/tests/cla.h
new file mode 100644
index 000000000000..3d879233681f
--- /dev/null
+++ b/tools/um/tests/cla.h
@@ -0,0 +1,33 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef _LKL_TEST_CLA_H
+#define _LKL_TEST_CLA_H
+
+enum cl_arg_type {
+	CL_ARG_USER = 0,
+	CL_ARG_BOOL,
+	CL_ARG_INT,
+	CL_ARG_STR,
+	CL_ARG_STR_SET,
+	CL_ARG_IPV4,
+	CL_ARG_END,
+};
+
+struct cl_arg;
+
+typedef int (*cl_arg_parser_t)(struct cl_arg *arg, const char *value);
+
+struct cl_arg {
+	const char *long_name;
+	char short_name;
+	const char *help;
+	int has_arg;
+	enum cl_arg_type type;
+	void *store;
+	void *set;
+	cl_arg_parser_t parser;
+};
+
+int cla_parse_args(int argc, const char **argv, struct cl_arg *args);
+
+
+#endif /* _LKL_TEST_CLA_H */
diff --git a/tools/um/tests/disk.c b/tools/um/tests/disk.c
new file mode 100644
index 000000000000..325934935e6a
--- /dev/null
+++ b/tools/um/tests/disk.c
@@ -0,0 +1,168 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <unistd.h>
+#include <string.h>
+#include <time.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <lkl.h>
+#include <lkl_host.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <sys/ioctl.h>
+
+#include "test.h"
+#include "cla.h"
+
+static struct {
+	int printk;
+	const char *disk;
+	const char *fstype;
+	int partition;
+} cla;
+
+struct cl_arg args[] = {
+	{"disk", 'd', "disk file to use", 1, CL_ARG_STR, &cla.disk},
+	{"partition", 'P', "partition to mount", 1, CL_ARG_INT, &cla.partition},
+	{"type", 't', "filesystem type", 1, CL_ARG_STR, &cla.fstype},
+	{0},
+};
+
+
+static struct lkl_disk disk;
+static int disk_id = -1;
+
+int lkl_test_disk_add(void)
+{
+	disk.fd = open(cla.disk, O_RDWR);
+	if (disk.fd < 0)
+		goto out_unlink;
+
+	disk.ops = NULL;
+	disk.dev = (char *)cla.disk;
+
+	disk_id = lkl_disk_add(&disk);
+	if (disk_id < 0)
+		goto out_close;
+
+	goto out;
+
+out_close:
+	close(disk.fd);
+
+out_unlink:
+	unlink(cla.disk);
+
+out:
+	lkl_test_logf("disk fd/handle %x disk_id %d", disk.fd, disk_id);
+
+	if (disk_id >= 0)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+int lkl_test_disk_remove(void)
+{
+	int ret;
+
+	ret = lkl_disk_remove(disk);
+
+	close(disk.fd);
+
+	if (ret == 0)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+
+static char mnt_point[32];
+
+LKL_TEST_CALL(mount_dev, lkl_mount_dev, 0, disk_id, cla.partition, cla.fstype,
+	      0, NULL, mnt_point, sizeof(mnt_point))
+
+static int lkl_test_umount_dev(void)
+{
+	long ret, ret2;
+
+	ret = lkl_sys_chdir("/");
+
+	ret2 = lkl_umount_dev(disk_id, cla.partition, 0, 1000);
+
+	lkl_test_logf("%ld %ld", ret, ret2);
+
+	if (!ret && !ret2)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+struct lkl_dir *dir;
+
+static int lkl_test_opendir(void)
+{
+	int err;
+
+	dir = lkl_opendir(mnt_point, &err);
+
+	lkl_test_logf("lkl_opedir(%s) = %d %s\n", mnt_point, err,
+		      lkl_strerror(err));
+
+	if (err == 0)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+static int lkl_test_readdir(void)
+{
+	struct lkl_linux_dirent64 *de = lkl_readdir(dir);
+	int wr = 0;
+
+	while (de) {
+		wr += lkl_test_logf("%s ", de->d_name);
+		if (wr >= 70) {
+			lkl_test_logf("\n");
+			wr = 0;
+			break;
+		}
+		de = lkl_readdir(dir);
+	}
+
+	if (lkl_errdir(dir) == 0)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+LKL_TEST_CALL(closedir, lkl_closedir, 0, dir);
+LKL_TEST_CALL(chdir_mnt_point, lkl_sys_chdir, 0, mnt_point);
+LKL_TEST_CALL(start_kernel, lkl_start_kernel, 0, &lkl_host_ops,
+	     "mem=16M loglevel=8");
+LKL_TEST_CALL(stop_kernel, lkl_sys_halt, 0);
+
+struct lkl_test tests[] = {
+	LKL_TEST(disk_add),
+	LKL_TEST(start_kernel),
+	LKL_TEST(mount_dev),
+	LKL_TEST(chdir_mnt_point),
+	LKL_TEST(opendir),
+	LKL_TEST(readdir),
+	LKL_TEST(closedir),
+	LKL_TEST(umount_dev),
+	LKL_TEST(stop_kernel),
+	LKL_TEST(disk_remove),
+
+};
+
+int main(int argc, const char **argv)
+{
+	if (cla_parse_args(argc, argv, args) < 0)
+		return (-1);
+
+	lkl_host_ops.print = lkl_test_log;
+
+	return lkl_test_run(tests, sizeof(tests)/sizeof(struct lkl_test),
+			    "disk %s", cla.fstype);
+}
diff --git a/tools/um/tests/disk.sh b/tools/um/tests/disk.sh
new file mode 100755
index 000000000000..e2ec6cf69d4b
--- /dev/null
+++ b/tools/um/tests/disk.sh
@@ -0,0 +1,70 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+
+source $script_dir/test.sh
+
+function prepfs()
+{
+    set -e
+
+    file=`mktemp`
+
+    dd if=/dev/zero of=$file bs=1024 count=204800
+
+    yes | mkfs.$1 $file
+
+    if ! [ -z $ANDROID_WDIR ]; then
+        adb shell mkdir -p $ANDROID_WDIR
+        adb push $file $ANDROID_WDIR
+        rm $file
+        file=$ANDROID_WDIR/$(basename $file)
+    fi
+    if ! [ -z $BSD_WDIR ]; then
+        $MYSSH mkdir -p $BSD_WDIR
+        ssh_copy $file $BSD_WDIR
+        rm $file
+        file=$BSD_WDIR/$(basename $file)
+    fi
+
+    export_vars file
+}
+
+function cleanfs()
+{
+    set -e
+
+    if ! [ -z $ANDROID_WDIR ]; then
+        adb shell rm $1
+        adb shell rm $ANDROID_WDIR/disk
+    elif ! [ -z $BSD_WDIR ]; then
+        $MYSSH rm $1
+        $MYSSH rm $BSD_WDIR/disk
+    else
+        rm $1
+    fi
+}
+
+if [ "$1" = "-t" ]; then
+    shift
+    fstype=$1
+    shift
+fi
+
+if [ -z "$fstype" ]; then
+    fstype="ext4"
+fi
+
+if [ -z $(which mkfs.$fstype) ]; then
+    lkl_test_plan 0 "disk $fstype"
+    echo "no mkfs.$fstype command"
+    exit 0
+fi
+
+lkl_test_plan 1 "disk $fstype"
+lkl_test_run 1 prepfs $fstype
+lkl_test_exec $script_dir/disk -d $file -t $fstype $@
+lkl_test_plan 1 "disk $fstype"
+lkl_test_run 1 cleanfs $file
+
diff --git a/tools/um/tests/run.py b/tools/um/tests/run.py
index c96ede90b6ad..97d6dedc217c 100755
--- a/tools/um/tests/run.py
+++ b/tools/um/tests/run.py
@@ -50,6 +50,8 @@ mydir=os.path.dirname(os.path.realpath(__file__))
 
 tests = [
     'boot.sh',
+    'disk.sh -t ext4',
+    'disk.sh -t vfat',
 ]
 
 parser = argparse.ArgumentParser(description='LKL test runner')
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v6 21/21] um: nommu: add block device support of UML
@ 2020-09-24  7:13         ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  7:13 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Hajime Tazaki, retrage01

This commit adds block device support for nommu mode, and also added
several host utilities to run a simple test program
(tools/um/test/disk.c) successfully.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/include/asm/xor.h                 |   3 +-
 arch/um/include/shared/as-layout.h        |   1 +
 arch/um/kernel/um_arch.c                  |   5 +
 arch/um/nommu/include/uapi/asm/host_ops.h |   7 +
 arch/um/nommu/include/uapi/asm/unistd.h   |   2 +
 arch/um/nommu/um/Kconfig                  |   8 +
 arch/um/nommu/um/setup.c                  |  12 +
 tools/um/Targets                          |   2 +
 tools/um/include/lkl.h                    | 206 ++++++++++
 tools/um/include/lkl_host.h               |   1 +
 tools/um/lib/Build                        |   1 +
 tools/um/lib/fs.c                         | 461 ++++++++++++++++++++++
 tools/um/lib/posix-host.c                 |   1 +
 tools/um/lib/utils.c                      |   3 +
 tools/um/tests/Build                      |   1 +
 tools/um/tests/cla.c                      | 159 ++++++++
 tools/um/tests/cla.h                      |  33 ++
 tools/um/tests/disk.c                     | 168 ++++++++
 tools/um/tests/disk.sh                    |  70 ++++
 tools/um/tests/run.py                     |   2 +
 20 files changed, 1145 insertions(+), 1 deletion(-)
 create mode 100644 tools/um/lib/fs.c
 create mode 100644 tools/um/tests/cla.c
 create mode 100644 tools/um/tests/cla.h
 create mode 100644 tools/um/tests/disk.c
 create mode 100755 tools/um/tests/disk.sh

diff --git a/arch/um/include/asm/xor.h b/arch/um/include/asm/xor.h
index 36b33d62a35d..a5ea458a1ae9 100644
--- a/arch/um/include/asm/xor.h
+++ b/arch/um/include/asm/xor.h
@@ -4,4 +4,5 @@
 
 /* pick an arbitrary one - measuring isn't possible with inf-cpu */
 #define XOR_SELECT_TEMPLATE(x)	\
-	(time_travel_mode == TT_MODE_INFCPU ? &xor_block_8regs : NULL)
+	(time_travel_mode == TT_MODE_INFCPU || (!IS_ENABLED(CONFIG_MMU)) ? \
+	 &xor_block_8regs : NULL)
diff --git a/arch/um/include/shared/as-layout.h b/arch/um/include/shared/as-layout.h
index 5f286ef2721b..4423437a5ace 100644
--- a/arch/um/include/shared/as-layout.h
+++ b/arch/um/include/shared/as-layout.h
@@ -57,6 +57,7 @@ extern unsigned long host_task_size;
 
 extern int linux_main(int argc, char **argv);
 extern void uml_finishsetup(void);
+extern void uml_set_args(char *args);
 
 struct siginfo;
 extern void (*sig_info[])(int, struct siginfo *si, struct uml_pt_regs *);
diff --git a/arch/um/kernel/um_arch.c b/arch/um/kernel/um_arch.c
index e2cb76c03b25..da6e06cf2808 100644
--- a/arch/um/kernel/um_arch.c
+++ b/arch/um/kernel/um_arch.c
@@ -41,6 +41,11 @@ static void __init add_arg(char *arg)
 	strcat(command_line, arg);
 }
 
+void __init uml_set_args(char *args)
+{
+	strcat(command_line, args);
+}
+
 /*
  * These fields are initialized at boot time and not changed.
  * XXX This structure is used only in the non-SMP case.  Maybe this
diff --git a/arch/um/nommu/include/uapi/asm/host_ops.h b/arch/um/nommu/include/uapi/asm/host_ops.h
index 0811494c080c..1b2507502969 100644
--- a/arch/um/nommu/include/uapi/asm/host_ops.h
+++ b/arch/um/nommu/include/uapi/asm/host_ops.h
@@ -15,6 +15,11 @@ struct lkl_jmp_buf {
  *
  * These operations must be provided by a host library or by the application
  * itself.
+ *
+ * @um_devices - string containg the list of UML devices in command line
+ * format. This string is appended to the kernel command line and
+ * is provided here for convenience to be implemented by the host library.
+ *
  * @print - optional operation that receives console messages
  * @panic - called during a kernel panic
  *
@@ -65,6 +70,8 @@ struct lkl_jmp_buf {
  *
  */
 struct lkl_host_operations {
+	const char *um_devices;
+
 	void (*print)(const char *str, int len);
 	void (*panic)(void);
 
diff --git a/arch/um/nommu/include/uapi/asm/unistd.h b/arch/um/nommu/include/uapi/asm/unistd.h
index 1762ebb829f5..320762099a62 100644
--- a/arch/um/nommu/include/uapi/asm/unistd.h
+++ b/arch/um/nommu/include/uapi/asm/unistd.h
@@ -3,6 +3,8 @@
 #define __UM_NOMMU_UAPI_UNISTD_H
 
 #define __ARCH_WANT_NEW_STAT
+#define __ARCH_WANT_SET_GET_RLIMIT
+
 #include <asm/bitsperlong.h>
 
 #if __BITS_PER_LONG == 64
diff --git a/arch/um/nommu/um/Kconfig b/arch/um/nommu/um/Kconfig
index 20b3eaccb6f0..c6a3f472fe75 100644
--- a/arch/um/nommu/um/Kconfig
+++ b/arch/um/nommu/um/Kconfig
@@ -4,6 +4,10 @@ config UML_NOMMU
 	select UACCESS_MEMCPY
 	select ARCH_THREAD_STACK_ALLOCATOR
 	select ARCH_HAS_SYSCALL_WRAPPER
+	select VFAT_FS
+	select NLS_CODEPAGE_437
+	select NLS_ISO8859_1
+	select BTRFS_FS
 
 config 64BIT
 	bool
@@ -35,3 +39,7 @@ config STACKTRACE_SUPPORT
 config PRINTK_TIME
 	bool
 	default y
+
+config RAID6_PQ_BENCHMARK
+	bool
+	default n
diff --git a/arch/um/nommu/um/setup.c b/arch/um/nommu/um/setup.c
index 922188690139..d71d61979ad6 100644
--- a/arch/um/nommu/um/setup.c
+++ b/arch/um/nommu/um/setup.c
@@ -45,13 +45,25 @@ static void __init *lkl_run_kernel(void *arg)
 	return NULL;
 }
 
+static char _cmd_line[COMMAND_LINE_SIZE];
 int __init lkl_start_kernel(struct lkl_host_operations *ops,
 			    const char *fmt, ...)
 {
+	va_list ap;
 	int ret;
 
 	lkl_ops = ops;
 
+	va_start(ap, fmt);
+	ret = vsnprintf(_cmd_line, COMMAND_LINE_SIZE, fmt, ap);
+	va_end(ap);
+
+	if (ops->um_devices)
+		strscpy(_cmd_line + ret, ops->um_devices,
+			COMMAND_LINE_SIZE - ret);
+
+	uml_set_args(_cmd_line);
+
 	init_sem = lkl_ops->sem_alloc(0);
 	if (!init_sem)
 		return -ENOMEM;
diff --git a/tools/um/Targets b/tools/um/Targets
index 2bb90381843c..f5f8ec4b9dbb 100644
--- a/tools/um/Targets
+++ b/tools/um/Targets
@@ -1,10 +1,12 @@
 ifeq ($(UMMODE),library)
 progs-y += tests/boot
+progs-y += tests/disk
 else
 progs-y += uml/linux
 endif
 
 LDFLAGS_boot-y := -pie
+LDFLAGS_disk-y := -pie
 
 LDLIBS := -lrt -lpthread -lutil
 LDFLAGS_linux-y := -no-pie -Wl,--wrap,malloc -Wl,--wrap,free -Wl,--wrap,calloc
diff --git a/tools/um/include/lkl.h b/tools/um/include/lkl.h
index 707e01b64a70..4a9d874bfbf0 100644
--- a/tools/um/include/lkl.h
+++ b/tools/um/include/lkl.h
@@ -113,6 +113,16 @@ static inline long lkl_sys_creat(const char *file, int mode)
 }
 #endif
 
+#ifdef __lkl__NR_faccessat
+/**
+ * lkl_sys_access - wrapper for lkl_sys_faccessat
+ */
+static inline long lkl_sys_access(const char *file, int mode)
+{
+	return lkl_sys_faccessat(LKL_AT_FDCWD, file, mode);
+}
+#endif
+
 #ifdef __lkl__NR_mkdirat
 /**
  * lkl_sys_mkdir - wrapper for lkl_sys_mkdirat
@@ -123,6 +133,36 @@ static inline long lkl_sys_mkdir(const char *path, mode_t mode)
 }
 #endif
 
+#ifdef __lkl__NR_unlinkat
+/**
+ * lkl_sys_rmdir - wrapper for lkl_sys_unlinkrat
+ */
+static inline long lkl_sys_rmdir(const char *path)
+{
+	return lkl_sys_unlinkat(LKL_AT_FDCWD, path, LKL_AT_REMOVEDIR);
+}
+#endif
+
+#ifdef __lkl__NR_unlinkat
+/**
+ * lkl_sys_unlink - wrapper for lkl_sys_unlinkat
+ */
+static inline long lkl_sys_unlink(const char *path)
+{
+	return lkl_sys_unlinkat(LKL_AT_FDCWD, path, 0);
+}
+#endif
+
+#ifdef __lkl__NR_mknodat
+/**
+ * lkl_sys_mknod - wrapper for lkl_sys_mknodat
+ */
+static inline long lkl_sys_mknod(const char *path, mode_t mode, dev_t dev)
+{
+	return lkl_sys_mknodat(LKL_AT_FDCWD, path, mode, dev);
+}
+#endif
+
 #ifdef __lkl__NR_epoll_create1
 /**
  * lkl_sys_epoll_create - wrapper for lkl_sys_epoll_create1
@@ -144,6 +184,172 @@ static inline long lkl_sys_epoll_wait(int fd, struct lkl_epoll_event *ev,
 }
 #endif
 
+/**
+ * struct lkl_dev_blk_ops - block device host operations, defined in lkl_host.h.
+ */
+struct lkl_dev_blk_ops;
+
+/**
+ * lkl_disk - host disk handle
+ *
+ * @dev - a pointer to private information for this disk backend
+ * @fd - a POSIX file descriptor that can be used by preadv/pwritev
+ * @handle - an NT file handle that can be used by ReadFile/WriteFile
+ */
+struct lkl_disk {
+	void *dev;
+	union {
+		int fd;
+		void *handle;
+	};
+	struct lkl_dev_blk_ops *ops;
+};
+
+/**
+ * lkl_disk_add - add a new disk
+ *
+ * @disk - the host disk handle
+ * @returns a disk id (0 is valid) or a strictly negative value in case of error
+ */
+int lkl_disk_add(struct lkl_disk *disk);
+
+/**
+ * lkl_disk_remove - remove a disk
+ *
+ * This function makes a cleanup of the @disk's private information
+ * that was initialized by lkl_disk_add before.
+ *
+ * @disk - the host disk handle
+ */
+int lkl_disk_remove(struct lkl_disk disk);
+
+/**
+ * lkl_encode_dev_from_sysfs_blkdev - extract device id from sysfs
+ *
+ * This function returns the device id for the given sysfs dev node.
+ * The content of the node has to be in the form 'MAJOR:MINOR'.
+ * Also, this function expects an absolute path which means that sysfs
+ * already has to be mounted at the given path
+ *
+ * @sysfs_path - absolute path to the sysfs dev node
+ * @pdevid - pointer to memory where dev id will be returned
+ * @returns - 0 on success, a negative value on error
+ */
+int lkl_encode_dev_from_sysfs(const char *sysfs_path, uint32_t *pdevid);
+
+/**
+ * lkl_mount_dev - mount a disk
+ *
+ * This functions creates a device file for the given disk, creates a mount
+ * point and mounts the device over the mount point.
+ *
+ * @disk_id - the disk id identifying the disk to be mounted
+ * @part - disk partition or zero for full disk
+ * @fs_type - filesystem type
+ * @flags - mount flags
+ * @opts - additional filesystem specific mount options
+ * @mnt_str - a string that will be filled by this function with the path where
+ * the filesystem has been mounted
+ * @mnt_str_len - size of mnt_str
+ * @returns - 0 on success, a negative value on error
+ */
+long lkl_mount_dev(unsigned int disk_id, unsigned int part, const char *fs_type,
+		   int flags, const char *opts,
+		   char *mnt_str, unsigned int mnt_str_len);
+
+/**
+ * lkl_umount_dev - umount a disk
+ *
+ * This functions umounts the given disks and removes the device file and the
+ * mount point.
+ *
+ * @disk_id - the disk id identifying the disk to be mounted
+ * @part - disk partition or zero for full disk
+ * @flags - umount flags
+ * @timeout_ms - timeout to wait for the kernel to flush closed files so that
+ * umount can succeed
+ * @returns - 0 on success, a negative value on error
+ */
+long lkl_umount_dev(unsigned int disk_id, unsigned int part, int flags,
+		    long timeout_ms);
+
+/**
+ * lkl_umount_timeout - umount filesystem with timeout
+ *
+ * @path - the path to unmount
+ * @flags - umount flags
+ * @timeout_ms - timeout to wait for the kernel to flush closed files so that
+ * umount can succeed
+ * @returns - 0 on success, a negative value on error
+ */
+long lkl_umount_timeout(char *path, int flags, long timeout_ms);
+
+/**
+ * lkl_opendir - open a directory
+ *
+ * @path - directory path
+ * @err - pointer to store the error in case of failure
+ * @returns - a handle to be used when calling lkl_readdir
+ */
+struct lkl_dir *lkl_opendir(const char *path, int *err);
+
+/**
+ * lkl_fdopendir - open a directory
+ *
+ * @fd - file descriptor
+ * @err - pointer to store the error in case of failure
+ * @returns - a handle to be used when calling lkl_readdir
+ */
+struct lkl_dir *lkl_fdopendir(int fd, int *err);
+
+/**
+ * lkl_rewinddir - reset directory stream
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ */
+void lkl_rewinddir(struct lkl_dir *dir);
+
+/**
+ * lkl_closedir - close the directory
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ */
+int lkl_closedir(struct lkl_dir *dir);
+
+/**
+ * lkl_readdir - get the next available entry of the directory
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ * @returns - a lkl_dirent64 entry or NULL if the end of the directory stream is
+ * reached or if an error occurred; check lkl_errdir() to distinguish between
+ * errors or end of the directory stream
+ */
+struct lkl_linux_dirent64 *lkl_readdir(struct lkl_dir *dir);
+
+/**
+ * lkl_errdir - checks if an error occurred during the last lkl_readdir call
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ * @returns - 0 if no error occurred, or a negative value otherwise
+ */
+int lkl_errdir(struct lkl_dir *dir);
+
+/**
+ * lkl_dirfd - gets the file descriptor associated with the directory handle
+ *
+ * @dir - the directory handle as returned by lkl_opendir
+ * @returns - a positive value,which is the LKL file descriptor associated with
+ * the directory handle, or a negative value otherwise
+ */
+int lkl_dirfd(struct lkl_dir *dir);
+
+/**
+ * lkl_mount_fs - mount a file system type like proc, sys
+ * @fstype - file system type. e.g. proc, sys
+ * @returns - 0 on success. 1 if it's already mounted. negative on failure.
+ */
+int lkl_mount_fs(char *fstype);
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/tools/um/include/lkl_host.h b/tools/um/include/lkl_host.h
index 85e80eb4ad0d..12dd95616e43 100644
--- a/tools/um/include/lkl_host.h
+++ b/tools/um/include/lkl_host.h
@@ -10,6 +10,7 @@ extern "C" {
 #include <lkl.h>
 
 extern struct lkl_host_operations lkl_host_ops;
+extern char lkl_um_devs[4096];
 
 /**
  * lkl_printf - print a message via the host print operation
diff --git a/tools/um/lib/Build b/tools/um/lib/Build
index dddff26a3b4e..29491b40746c 100644
--- a/tools/um/lib/Build
+++ b/tools/um/lib/Build
@@ -4,3 +4,4 @@ CFLAGS_posix-host.o += -D_FILE_OFFSET_BITS=64
 liblinux-$(CONFIG_UMMODE_LIB) += utils.o
 liblinux-$(CONFIG_UMMODE_LIB) += posix-host.o
 liblinux-$(CONFIG_UMMODE_LIB) += jmp_buf.o
+liblinux-$(CONFIG_UMMODE_LIB) += fs.o
diff --git a/tools/um/lib/fs.c b/tools/um/lib/fs.c
new file mode 100644
index 000000000000..948aac9730c2
--- /dev/null
+++ b/tools/um/lib/fs.c
@@ -0,0 +1,461 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdarg.h>
+#include <stdio.h>
+#include <string.h>
+#include <stdlib.h>
+#include <lkl_host.h>
+
+#define MAX_FSTYPE_LEN 50
+
+static struct lkl_disk *lkl_disks[16];
+static int registered_blk_dev_idx;
+
+static int lkl_disk_um_add(struct lkl_disk *disk, const char *blkparams)
+{
+	/* concat strings */
+	snprintf(lkl_um_devs + strlen(lkl_um_devs), sizeof(lkl_um_devs),
+		 " ubd%d=%s", registered_blk_dev_idx, blkparams);
+
+	return registered_blk_dev_idx++;
+}
+
+int lkl_disk_add(struct lkl_disk *disk)
+{
+	int ret = -1;
+
+	ret = lkl_disk_um_add(disk, disk->dev);
+
+	lkl_disks[ret] = disk;
+
+	return ret;
+}
+
+int lkl_disk_remove(struct lkl_disk disk)
+{
+	/* FIXME */
+	return 0;
+}
+
+int lkl_mount_fs(char *fstype)
+{
+	char dir[MAX_FSTYPE_LEN+2] = "/";
+	int flags = 0, ret = 0;
+
+	strncat(dir, fstype, MAX_FSTYPE_LEN);
+
+	/* Create with regular umask */
+	ret = lkl_sys_mkdir(dir, 0xff);
+	if (ret && ret != -LKL_EEXIST) {
+		lkl_perror("mount_fs mkdir", ret);
+		return ret;
+	}
+
+	/* We have no use for nonzero flags right now */
+	ret = lkl_sys_mount("none", dir, fstype, flags, NULL);
+	if (ret && ret != -LKL_EBUSY) {
+		lkl_sys_rmdir(dir);
+		return ret;
+	}
+
+	if (ret == -LKL_EBUSY)
+		return 1;
+	return 0;
+}
+
+static uint32_t new_encode_dev(unsigned int major, unsigned int minor)
+{
+	return (minor & 0xff) | (major << 8) | ((minor & ~0xff) << 12);
+}
+
+static int startswith(const char *str, const char *pre)
+{
+	return strncmp(pre, str, strlen(pre)) == 0;
+}
+
+static int get_node_with_prefix(const char *path, const char *prefix,
+				char *result, unsigned int result_len)
+{
+	struct lkl_dir *dir = NULL;
+	struct lkl_linux_dirent64 *dirent;
+	int ret;
+
+	dir = lkl_opendir(path, &ret);
+	if (!dir)
+		return ret;
+
+	ret = -LKL_ENOENT;
+
+	while ((dirent = lkl_readdir(dir))) {
+		if (startswith(dirent->d_name, prefix)) {
+			if (strlen(dirent->d_name) + 1 > result_len) {
+				ret = -LKL_ENOMEM;
+				break;
+			}
+			memcpy(result, dirent->d_name, strlen(dirent->d_name));
+			result[strlen(dirent->d_name)] = '\0';
+			ret = 0;
+			break;
+		}
+	}
+
+	lkl_closedir(dir);
+
+	return ret;
+}
+
+int lkl_encode_dev_from_sysfs(const char *sysfs_path, uint32_t *pdevid)
+{
+	int ret;
+	long fd;
+	int major, minor;
+	char buf[16] = { 0, };
+	char *bufptr;
+
+	fd = lkl_sys_open(sysfs_path, LKL_O_RDONLY, 0);
+	if (fd < 0)
+		return fd;
+
+	ret = lkl_sys_read(fd, buf, sizeof(buf));
+	if (ret < 0)
+		goto out_close;
+
+	if (ret == sizeof(buf)) {
+		ret = -LKL_ENOBUFS;
+		goto out_close;
+	}
+
+	bufptr = strchr(buf, ':');
+	if (bufptr == NULL) {
+		ret = -LKL_EINVAL;
+		goto out_close;
+	}
+	bufptr[0] = '\0';
+	bufptr++;
+
+	major = atoi(buf);
+	minor = atoi(bufptr);
+
+	*pdevid = new_encode_dev(major, minor);
+	ret = 0;
+
+out_close:
+	lkl_sys_close(fd);
+
+	return ret;
+}
+
+#define SYSFS_DEV_UMBLK_CMDLINE_PATH \
+	"/sysfs/devices/platform/uml-blkdev.%d"
+
+struct abuf {
+	char *mem, *ptr;
+	unsigned int len;
+};
+
+static int snprintf_append(struct abuf *buf, const char *fmt, ...)
+{
+	int ret;
+	va_list args;
+
+	if (!buf->ptr)
+		buf->ptr = buf->mem;
+
+	va_start(args, fmt);
+	ret = vsnprintf(buf->ptr, buf->len - (buf->ptr - buf->mem), fmt, args);
+	va_end(args);
+
+	if (ret < 0 || (ret >= (int)(buf->len - (buf->ptr - buf->mem))))
+		return -LKL_ENOMEM;
+
+	buf->ptr += ret;
+
+	return 0;
+}
+
+static int __lkl_get_blkdev(int disk_id, unsigned int part, uint32_t *pdevid,
+			    const char *sysfs_path_fmt, const char *drv_prefix,
+			    const char *disk_prefix)
+{
+	char sysfs_path[LKL_PATH_MAX];
+	char drv_name[LKL_PATH_MAX];
+	char disk_name[LKL_PATH_MAX];
+	struct abuf sysfs_path_buf = {
+		.mem = sysfs_path,
+		.len = sizeof(sysfs_path),
+	};
+	int ret;
+
+	if (disk_id < 0)
+		return -LKL_EINVAL;
+
+	ret = lkl_mount_fs("sysfs");
+	if (ret < 0)
+		return ret;
+
+	ret = snprintf_append(&sysfs_path_buf, sysfs_path_fmt, disk_id);
+	if (ret)
+		return ret;
+
+	ret = get_node_with_prefix(sysfs_path, drv_prefix, drv_name,
+				   sizeof(drv_name));
+	if (ret)
+		return ret;
+
+	ret = snprintf_append(&sysfs_path_buf, "/%s/block", drv_name);
+	if (ret)
+		return ret;
+
+	ret = get_node_with_prefix(sysfs_path, disk_prefix, disk_name,
+				   sizeof(disk_name));
+	if (ret)
+		return ret;
+
+	if (!part)
+		ret = snprintf_append(&sysfs_path_buf, "/%s/dev", disk_name);
+	else
+		ret = snprintf_append(&sysfs_path_buf, "/%s/%s%d/dev",
+				      disk_name, disk_name, part);
+	if (ret)
+		return ret;
+
+	return lkl_encode_dev_from_sysfs(sysfs_path, pdevid);
+}
+
+int lkl_get_blkdev(int disk_id, unsigned int part, uint32_t *pdevid)
+{
+	char *fmt;
+
+	fmt = SYSFS_DEV_UMBLK_CMDLINE_PATH;
+	return __lkl_get_blkdev(disk_id, part, pdevid, fmt, "", "ubd");
+}
+
+long lkl_mount_dev(unsigned int disk_id, unsigned int part,
+		   const char *fs_type, int flags,
+		   const char *data, char *mnt_str, unsigned int mnt_str_len)
+{
+	char dev_str[] = { "/dev/xxxxxxxx" };
+	unsigned int dev;
+	int err;
+	char _data[4096]; /* FIXME: PAGE_SIZE is not exported by LKL */
+
+	if (mnt_str_len < sizeof(dev_str))
+		return -LKL_ENOMEM;
+
+	err = lkl_get_blkdev(disk_id, part, &dev);
+	if (err < 0)
+		return err;
+
+	snprintf(dev_str, sizeof(dev_str), "/dev/%08x", dev);
+	snprintf(mnt_str, mnt_str_len, "/mnt/%08x", dev);
+
+	err = lkl_sys_access("/dev", LKL_S_IRWXO);
+	if (err < 0) {
+		if (err == -LKL_ENOENT)
+			err = lkl_sys_mkdir("/dev", 0700);
+		if (err < 0)
+			return err;
+	}
+
+	err = lkl_sys_mknod(dev_str, LKL_S_IFBLK | 0600, dev);
+	if (err < 0)
+		return err;
+
+	err = lkl_sys_access("/mnt", LKL_S_IRWXO);
+	if (err < 0) {
+		if (err == -LKL_ENOENT)
+			err = lkl_sys_mkdir("/mnt", 0700);
+		if (err < 0)
+			return err;
+	}
+
+	err = lkl_sys_mkdir(mnt_str, 0700);
+	if (err < 0) {
+		lkl_sys_unlink(dev_str);
+		return err;
+	}
+
+	/* kernel always copies a full page */
+	if (data) {
+		strncpy(_data, data, sizeof(_data));
+		_data[sizeof(_data) - 1] = 0;
+	} else {
+		_data[0] = 0;
+	}
+
+	err = lkl_sys_mount(dev_str, mnt_str, (char *)fs_type, flags, _data);
+	if (err < 0) {
+		lkl_sys_unlink(dev_str);
+		lkl_sys_rmdir(mnt_str);
+		return err;
+	}
+
+	return 0;
+}
+
+long lkl_umount_timeout(char *path, int flags, long timeout_ms)
+{
+	long incr = 10000000; /* 10 ms */
+	struct lkl_timespec ts = {
+		.tv_sec = 0,
+		.tv_nsec = incr,
+	};
+	long err;
+
+	do {
+		err = lkl_sys_umount(path, flags);
+		if (err == -LKL_EBUSY) {
+			lkl_sys_nanosleep((struct __lkl__kernel_timespec *)&ts,
+					  NULL);
+			timeout_ms -= incr / 1000000;
+		}
+	} while (err == -LKL_EBUSY && timeout_ms > 0);
+
+	return err;
+}
+
+long lkl_umount_dev(unsigned int disk_id, unsigned int part, int flags,
+		    long timeout_ms)
+{
+	char dev_str[] = { "/dev/xxxxxxxx" };
+	char mnt_str[] = { "/mnt/xxxxxxxx" };
+	unsigned int dev;
+	int err;
+
+	err = lkl_get_blkdev(disk_id, part, &dev);
+	if (err < 0)
+		return err;
+
+	snprintf(dev_str, sizeof(dev_str), "/dev/%08x", dev);
+	snprintf(mnt_str, sizeof(mnt_str), "/mnt/%08x", dev);
+
+	err = lkl_umount_timeout(mnt_str, flags, timeout_ms);
+	if (err)
+		return err;
+
+	err = lkl_sys_unlink(dev_str);
+	if (err)
+		return err;
+
+	return lkl_sys_rmdir(mnt_str);
+}
+
+struct lkl_dir {
+	int fd;
+	char buf[1024];
+	char *pos;
+	int len;
+};
+
+static struct lkl_dir *lkl_dir_alloc(int *err)
+{
+	struct lkl_dir *dir = lkl_host_ops.mem_alloc(sizeof(struct lkl_dir));
+
+	if (!dir) {
+		*err = -LKL_ENOMEM;
+		return NULL;
+	}
+
+	dir->len = 0;
+	dir->pos = NULL;
+
+	return dir;
+}
+
+struct lkl_dir *lkl_opendir(const char *path, int *err)
+{
+	struct lkl_dir *dir = lkl_dir_alloc(err);
+
+	if (!dir) {
+		*err = -LKL_ENOMEM;
+		return NULL;
+	}
+
+	dir->fd = lkl_sys_open(path, LKL_O_RDONLY | LKL_O_DIRECTORY, 0);
+	if (dir->fd < 0) {
+		*err = dir->fd;
+		lkl_host_ops.mem_free(dir);
+		return NULL;
+	}
+
+	*err = 0;
+
+	return dir;
+}
+
+struct lkl_dir *lkl_fdopendir(int fd, int *err)
+{
+	struct lkl_dir *dir = lkl_dir_alloc(err);
+
+	if (!dir)
+		return NULL;
+
+	dir->fd = fd;
+
+	return dir;
+}
+
+void lkl_rewinddir(struct lkl_dir *dir)
+{
+	lkl_sys_lseek(dir->fd, 0, LKL_SEEK_SET);
+	dir->len = 0;
+	dir->pos = NULL;
+}
+
+int lkl_closedir(struct lkl_dir *dir)
+{
+	int ret;
+
+	ret = lkl_sys_close(dir->fd);
+	lkl_host_ops.mem_free(dir);
+
+	return ret;
+}
+
+struct lkl_linux_dirent64 *lkl_readdir(struct lkl_dir *dir)
+{
+	struct lkl_linux_dirent64 *de;
+
+	if (dir->len < 0)
+		return NULL;
+
+	if (!dir->pos || dir->pos - dir->buf >= dir->len)
+		goto read_buf;
+
+return_de:
+	de = (struct lkl_linux_dirent64 *)dir->pos;
+	dir->pos += de->d_reclen;
+
+	return de;
+
+read_buf:
+	dir->pos = NULL;
+	de = (struct lkl_linux_dirent64 *)dir->buf;
+	dir->len = lkl_sys_getdents64(dir->fd, de, sizeof(dir->buf));
+	if (dir->len <= 0)
+		return NULL;
+
+	dir->pos = dir->buf;
+	goto return_de;
+}
+
+int lkl_errdir(struct lkl_dir *dir)
+{
+	if (dir->len >= 0)
+		return 0;
+
+	return dir->len;
+}
+
+int lkl_dirfd(struct lkl_dir *dir)
+{
+	return dir->fd;
+}
+
+int lkl_set_fd_limit(unsigned int fd_limit)
+{
+	struct lkl_rlimit rlim = {
+		.rlim_cur = fd_limit,
+		.rlim_max = fd_limit,
+	};
+	return lkl_sys_setrlimit(LKL_RLIMIT_NOFILE, &rlim);
+}
diff --git a/tools/um/lib/posix-host.c b/tools/um/lib/posix-host.c
index b6b5b2902254..8fd88031bf2b 100644
--- a/tools/um/lib/posix-host.c
+++ b/tools/um/lib/posix-host.c
@@ -263,6 +263,7 @@ static void *tls_get(struct lkl_tls_key *key)
 }
 
 struct lkl_host_operations lkl_host_ops = {
+	.um_devices = lkl_um_devs,
 	.panic = panic,
 	.print = print,
 	.mem_alloc = (void *)malloc,
diff --git a/tools/um/lib/utils.c b/tools/um/lib/utils.c
index 4930479a8a35..ac65cd744a14 100644
--- a/tools/um/lib/utils.c
+++ b/tools/um/lib/utils.c
@@ -4,6 +4,9 @@
 #include <string.h>
 #include <lkl_host.h>
 
+/* XXX: find a better place */
+char lkl_um_devs[4096];
+
 static const char * const lkl_err_strings[] = {
 	"Success",
 	"Operation not permitted",
diff --git a/tools/um/tests/Build b/tools/um/tests/Build
index 26a607f90dc8..b62919cadc50 100644
--- a/tools/um/tests/Build
+++ b/tools/um/tests/Build
@@ -1 +1,2 @@
 boot-y += boot.o test.o
+disk-y += disk.o test.o cla.o
diff --git a/tools/um/tests/cla.c b/tools/um/tests/cla.c
new file mode 100644
index 000000000000..694e3a3822be
--- /dev/null
+++ b/tools/um/tests/cla.c
@@ -0,0 +1,159 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <string.h>
+#include <errno.h>
+#include <stdlib.h>
+#ifdef __MINGW32__
+#include <winsock2.h>
+#else
+#include <sys/socket.h>
+#include <netinet/in.h>
+#include <arpa/inet.h>
+#endif
+
+#include "cla.h"
+
+static int cl_arg_parse_bool(struct cl_arg *arg, const char *value)
+{
+	*((int *)arg->store) = 1;
+	return 0;
+}
+
+static int cl_arg_parse_str(struct cl_arg *arg, const char *value)
+{
+	*((const char **)arg->store) = value;
+	return 0;
+}
+
+static int cl_arg_parse_int(struct cl_arg *arg, const char *value)
+{
+	errno = 0;
+	*((int *)arg->store) = strtol(value, NULL, 0);
+	return errno == 0;
+}
+
+static int cl_arg_parse_str_set(struct cl_arg *arg, const char *value)
+{
+	const char **set = arg->set;
+	int i;
+
+	for (i = 0; set[i] != NULL; i++) {
+		if (strcmp(set[i], value) == 0) {
+			*((int *)arg->store) = i;
+			return 0;
+		}
+	}
+
+	return (-1);
+}
+
+static int cl_arg_parse_ipv4(struct cl_arg *arg, const char *value)
+{
+	unsigned int addr;
+
+	if (!value)
+		return (-1);
+
+	addr = inet_addr(value);
+	if (addr == INADDR_NONE)
+		return (-1);
+	*((unsigned int *)arg->store) = addr;
+	return 0;
+}
+
+static cl_arg_parser_t parsers[] = {
+	[CL_ARG_BOOL] = cl_arg_parse_bool,
+	[CL_ARG_INT] = cl_arg_parse_int,
+	[CL_ARG_STR] = cl_arg_parse_str,
+	[CL_ARG_STR_SET] = cl_arg_parse_str_set,
+	[CL_ARG_IPV4] = cl_arg_parse_ipv4,
+};
+
+static struct cl_arg *find_short_arg(char name, struct cl_arg *args)
+{
+	struct cl_arg *arg;
+
+	for (arg = args; arg->short_name != 0; arg++) {
+		if (arg->short_name == name)
+			return arg;
+	}
+
+	return NULL;
+}
+
+static struct cl_arg *find_long_arg(const char *name, struct cl_arg *args)
+{
+	struct cl_arg *arg;
+
+	for (arg = args; arg->long_name; arg++) {
+		if (strcmp(arg->long_name, name) == 0)
+			return arg;
+	}
+
+	return NULL;
+}
+
+static void print_help(struct cl_arg *args)
+{
+	struct cl_arg *arg;
+
+	fprintf(stderr, "usage:\n");
+	for (arg = args; arg->long_name; arg++) {
+		fprintf(stderr, "-%c, --%-20s %s", arg->short_name,
+			arg->long_name, arg->help);
+		if (arg->type == CL_ARG_STR_SET) {
+			const char **set = arg->set;
+
+			fprintf(stderr, " [ ");
+			while (*set != NULL)
+				fprintf(stderr, "%s ", *(set++));
+			fprintf(stderr, "]");
+		}
+		fprintf(stderr, "\n");
+	}
+}
+
+int cla_parse_args(int argc, const char **argv, struct cl_arg *args)
+{
+	int i;
+
+	for (i = 1; i < argc; i++) {
+		struct cl_arg *arg = NULL;
+		cl_arg_parser_t parser;
+
+		if (argv[i][0] == '-') {
+			if (argv[i][1] != '-')
+				arg = find_short_arg(argv[i][1], args);
+			else
+				arg = find_long_arg(&argv[i][2], args);
+		}
+
+		if (!arg) {
+			fprintf(stderr, "unknown option '%s'\n", argv[i]);
+			print_help(args);
+			return (-1);
+		}
+
+		if (arg->type == CL_ARG_USER || arg->type >= CL_ARG_END)
+			parser = arg->parser;
+		else
+			parser = parsers[arg->type];
+
+		if (!parser) {
+			fprintf(stderr, "can't parse --'%s'/-'%c'\n",
+				arg->long_name, args->short_name);
+			return (-1);
+		}
+
+		if (parser(arg, argv[i + 1]) < 0) {
+			fprintf(stderr, "can't parse '%s'\n", argv[i]);
+			print_help(args);
+			return (-1);
+		}
+
+		if (arg->has_arg)
+			i++;
+	}
+
+	return 0;
+}
diff --git a/tools/um/tests/cla.h b/tools/um/tests/cla.h
new file mode 100644
index 000000000000..3d879233681f
--- /dev/null
+++ b/tools/um/tests/cla.h
@@ -0,0 +1,33 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef _LKL_TEST_CLA_H
+#define _LKL_TEST_CLA_H
+
+enum cl_arg_type {
+	CL_ARG_USER = 0,
+	CL_ARG_BOOL,
+	CL_ARG_INT,
+	CL_ARG_STR,
+	CL_ARG_STR_SET,
+	CL_ARG_IPV4,
+	CL_ARG_END,
+};
+
+struct cl_arg;
+
+typedef int (*cl_arg_parser_t)(struct cl_arg *arg, const char *value);
+
+struct cl_arg {
+	const char *long_name;
+	char short_name;
+	const char *help;
+	int has_arg;
+	enum cl_arg_type type;
+	void *store;
+	void *set;
+	cl_arg_parser_t parser;
+};
+
+int cla_parse_args(int argc, const char **argv, struct cl_arg *args);
+
+
+#endif /* _LKL_TEST_CLA_H */
diff --git a/tools/um/tests/disk.c b/tools/um/tests/disk.c
new file mode 100644
index 000000000000..325934935e6a
--- /dev/null
+++ b/tools/um/tests/disk.c
@@ -0,0 +1,168 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <unistd.h>
+#include <string.h>
+#include <time.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <lkl.h>
+#include <lkl_host.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <sys/ioctl.h>
+
+#include "test.h"
+#include "cla.h"
+
+static struct {
+	int printk;
+	const char *disk;
+	const char *fstype;
+	int partition;
+} cla;
+
+struct cl_arg args[] = {
+	{"disk", 'd', "disk file to use", 1, CL_ARG_STR, &cla.disk},
+	{"partition", 'P', "partition to mount", 1, CL_ARG_INT, &cla.partition},
+	{"type", 't', "filesystem type", 1, CL_ARG_STR, &cla.fstype},
+	{0},
+};
+
+
+static struct lkl_disk disk;
+static int disk_id = -1;
+
+int lkl_test_disk_add(void)
+{
+	disk.fd = open(cla.disk, O_RDWR);
+	if (disk.fd < 0)
+		goto out_unlink;
+
+	disk.ops = NULL;
+	disk.dev = (char *)cla.disk;
+
+	disk_id = lkl_disk_add(&disk);
+	if (disk_id < 0)
+		goto out_close;
+
+	goto out;
+
+out_close:
+	close(disk.fd);
+
+out_unlink:
+	unlink(cla.disk);
+
+out:
+	lkl_test_logf("disk fd/handle %x disk_id %d", disk.fd, disk_id);
+
+	if (disk_id >= 0)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+int lkl_test_disk_remove(void)
+{
+	int ret;
+
+	ret = lkl_disk_remove(disk);
+
+	close(disk.fd);
+
+	if (ret == 0)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+
+static char mnt_point[32];
+
+LKL_TEST_CALL(mount_dev, lkl_mount_dev, 0, disk_id, cla.partition, cla.fstype,
+	      0, NULL, mnt_point, sizeof(mnt_point))
+
+static int lkl_test_umount_dev(void)
+{
+	long ret, ret2;
+
+	ret = lkl_sys_chdir("/");
+
+	ret2 = lkl_umount_dev(disk_id, cla.partition, 0, 1000);
+
+	lkl_test_logf("%ld %ld", ret, ret2);
+
+	if (!ret && !ret2)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+struct lkl_dir *dir;
+
+static int lkl_test_opendir(void)
+{
+	int err;
+
+	dir = lkl_opendir(mnt_point, &err);
+
+	lkl_test_logf("lkl_opedir(%s) = %d %s\n", mnt_point, err,
+		      lkl_strerror(err));
+
+	if (err == 0)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+static int lkl_test_readdir(void)
+{
+	struct lkl_linux_dirent64 *de = lkl_readdir(dir);
+	int wr = 0;
+
+	while (de) {
+		wr += lkl_test_logf("%s ", de->d_name);
+		if (wr >= 70) {
+			lkl_test_logf("\n");
+			wr = 0;
+			break;
+		}
+		de = lkl_readdir(dir);
+	}
+
+	if (lkl_errdir(dir) == 0)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+LKL_TEST_CALL(closedir, lkl_closedir, 0, dir);
+LKL_TEST_CALL(chdir_mnt_point, lkl_sys_chdir, 0, mnt_point);
+LKL_TEST_CALL(start_kernel, lkl_start_kernel, 0, &lkl_host_ops,
+	     "mem=16M loglevel=8");
+LKL_TEST_CALL(stop_kernel, lkl_sys_halt, 0);
+
+struct lkl_test tests[] = {
+	LKL_TEST(disk_add),
+	LKL_TEST(start_kernel),
+	LKL_TEST(mount_dev),
+	LKL_TEST(chdir_mnt_point),
+	LKL_TEST(opendir),
+	LKL_TEST(readdir),
+	LKL_TEST(closedir),
+	LKL_TEST(umount_dev),
+	LKL_TEST(stop_kernel),
+	LKL_TEST(disk_remove),
+
+};
+
+int main(int argc, const char **argv)
+{
+	if (cla_parse_args(argc, argv, args) < 0)
+		return (-1);
+
+	lkl_host_ops.print = lkl_test_log;
+
+	return lkl_test_run(tests, sizeof(tests)/sizeof(struct lkl_test),
+			    "disk %s", cla.fstype);
+}
diff --git a/tools/um/tests/disk.sh b/tools/um/tests/disk.sh
new file mode 100755
index 000000000000..e2ec6cf69d4b
--- /dev/null
+++ b/tools/um/tests/disk.sh
@@ -0,0 +1,70 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+
+source $script_dir/test.sh
+
+function prepfs()
+{
+    set -e
+
+    file=`mktemp`
+
+    dd if=/dev/zero of=$file bs=1024 count=204800
+
+    yes | mkfs.$1 $file
+
+    if ! [ -z $ANDROID_WDIR ]; then
+        adb shell mkdir -p $ANDROID_WDIR
+        adb push $file $ANDROID_WDIR
+        rm $file
+        file=$ANDROID_WDIR/$(basename $file)
+    fi
+    if ! [ -z $BSD_WDIR ]; then
+        $MYSSH mkdir -p $BSD_WDIR
+        ssh_copy $file $BSD_WDIR
+        rm $file
+        file=$BSD_WDIR/$(basename $file)
+    fi
+
+    export_vars file
+}
+
+function cleanfs()
+{
+    set -e
+
+    if ! [ -z $ANDROID_WDIR ]; then
+        adb shell rm $1
+        adb shell rm $ANDROID_WDIR/disk
+    elif ! [ -z $BSD_WDIR ]; then
+        $MYSSH rm $1
+        $MYSSH rm $BSD_WDIR/disk
+    else
+        rm $1
+    fi
+}
+
+if [ "$1" = "-t" ]; then
+    shift
+    fstype=$1
+    shift
+fi
+
+if [ -z "$fstype" ]; then
+    fstype="ext4"
+fi
+
+if [ -z $(which mkfs.$fstype) ]; then
+    lkl_test_plan 0 "disk $fstype"
+    echo "no mkfs.$fstype command"
+    exit 0
+fi
+
+lkl_test_plan 1 "disk $fstype"
+lkl_test_run 1 prepfs $fstype
+lkl_test_exec $script_dir/disk -d $file -t $fstype $@
+lkl_test_plan 1 "disk $fstype"
+lkl_test_run 1 cleanfs $file
+
diff --git a/tools/um/tests/run.py b/tools/um/tests/run.py
index c96ede90b6ad..97d6dedc217c 100755
--- a/tools/um/tests/run.py
+++ b/tools/um/tests/run.py
@@ -50,6 +50,8 @@ mydir=os.path.dirname(os.path.realpath(__file__))
 
 tests = [
     'boot.sh',
+    'disk.sh -t ext4',
+    'disk.sh -t vfat',
 ]
 
 parser = argparse.ArgumentParser(description='LKL test runner')
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* Re: [RFC v6 01/21] um: split build in kernel and host parts
  2020-09-24  7:12         ` Hajime Tazaki
@ 2020-09-24  7:33           ` Anton Ivanov
  -1 siblings, 0 replies; 476+ messages in thread
From: Anton Ivanov @ 2020-09-24  7:33 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch,
	Octavian Purdila



On 24/09/2020 08:12, Hajime Tazaki wrote:
> From: Octavian Purdila <tavi@cs.pub.ro>
> 
> This patch splits the UML build in two parts: a kernel part that
> generates a relocatable object (linux.o) and a host build part that
> links the relocatable object into an executable.
> 
> This allows us to simplify the linker script since we can now remove
> the host bits (e.g. .rel sections). It also gives us greater
> flexibility since it will be much easier to support other
> architectures and OSes if we use the standard linker script.
> 
> The host build part has been implemented in tools/um so that we can
> reuse the available host build infrastructure.
> 
> The patch also changes the UML build invocation, if before
> 
>   $ make ARCH=um defconfig
>   $ make ARCH=um
> 
> was generating the executable now this only generates the relocatable
> object.

This will break packaging in all distributions. We need to figure out an alternative way which is backward compatible with their builds.

> 
> To fully build UML use:
> 
>   $ make -C tools/um
> 
> which will generate tools/lkl/linux as the UML executable. To
> statically compiled UML use:
> 
>   $ make -C tools/lkl UML_STATIC=y
> 
> Signed-off-by: Octavian Purdila <tavi@cs.pub.ro>
> Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
> ---
>   arch/um/Kconfig                  |  12 +--
>   arch/um/Makefile                 |  14 ++-
>   arch/um/include/asm/common.lds.S |  99 ------------------
>   arch/um/kernel/dyn.lds.S         | 171 -------------------------------
>   arch/um/kernel/uml.lds.S         | 115 ---------------------
>   arch/um/kernel/vmlinux.lds.S     |  91 ++++++++++++++--
>   scripts/link-vmlinux.sh          |  42 +++-----
>   tools/um/Makefile                |  72 +++++++++++++
>   tools/um/Targets                 |   4 +
>   tools/um/uml/Build               |   0
>   10 files changed, 184 insertions(+), 436 deletions(-)
>   delete mode 100644 arch/um/include/asm/common.lds.S
>   delete mode 100644 arch/um/kernel/dyn.lds.S
>   delete mode 100644 arch/um/kernel/uml.lds.S
>   create mode 100644 tools/um/Makefile
>   create mode 100644 tools/um/Targets
>   create mode 100644 tools/um/uml/Build
> 
> diff --git a/arch/um/Kconfig b/arch/um/Kconfig
> index eb51fec75948..e41d31d0d875 100644
> --- a/arch/um/Kconfig
> +++ b/arch/um/Kconfig
> @@ -20,6 +20,7 @@ config UML
>   	select GENERIC_CLOCKEVENTS
>   	select HAVE_GCC_PLUGINS
>   	select TTY # Needed for line.c
> +	select MODULE_REL_CRCS if MODVERSIONS
>   
>   config MMU
>   	bool
> @@ -79,17 +80,6 @@ config STATIC_LINK
>   	  NOTE: This option is incompatible with some networking features which
>   	  depend on features that require being dynamically loaded (like NSS).
>   
> -config LD_SCRIPT_STATIC
> -	bool
> -	default y
> -	depends on STATIC_LINK
> -
> -config LD_SCRIPT_DYN
> -	bool
> -	default y
> -	depends on !LD_SCRIPT_STATIC
> -	select MODULE_REL_CRCS if MODVERSIONS
> -
>   config HOSTFS
>   	tristate "Host filesystem"
>   	help
> diff --git a/arch/um/Makefile b/arch/um/Makefile
> index 1cea46ff9bb7..fdfaff4633e8 100644
> --- a/arch/um/Makefile
> +++ b/arch/um/Makefile
> @@ -95,14 +95,20 @@ KBUILD_CPPFLAGS += -I$(srctree)/$(HOST_DIR)/include \
>   KERNEL_DEFINES = $(strip -Derrno=kernel_errno -Dsigprocmask=kernel_sigprocmask \
>   			 -Dmktime=kernel_mktime $(ARCH_KERNEL_DEFINES))
>   KBUILD_CFLAGS += $(KERNEL_DEFINES)
> +LDFLAGS_vmlinux += -r
>   
> -PHONY += linux
> +PHONY += linux.o
>   
> -all: linux
> +all: linux.o
>   
> -linux: vmlinux
> +linux.o: vmlinux
>   	@echo '  LINK $@'
> -	$(Q)ln -f $< $@
> +	$(Q)$(OBJCOPY) -R .eh_frame $< $@
> +
> +install: linux.o
> +	@echo "  INSTALL $(INSTALL_PATH)/lib/$<"
> +	@mkdir -p $(INSTALL_PATH)/lib/
> +	@cp $< $(INSTALL_PATH)/lib/
>   
>   define archhelp
>     echo '* linux		- Binary kernel image (./linux) - for backward'
> diff --git a/arch/um/include/asm/common.lds.S b/arch/um/include/asm/common.lds.S
> deleted file mode 100644
> index eca6c452a41b..000000000000
> --- a/arch/um/include/asm/common.lds.S
> +++ /dev/null
> @@ -1,99 +0,0 @@
> -/* SPDX-License-Identifier: GPL-2.0 */
> -#include <asm-generic/vmlinux.lds.h>
> -
> -  .fini      : { *(.fini)    } =0x9090
> -  _etext = .;
> -  PROVIDE (etext = .);
> -
> -  . = ALIGN(4096);
> -  _sdata = .;
> -  PROVIDE (sdata = .);
> -
> -  RO_DATA(4096)
> -
> -  .unprotected : { *(.unprotected) }
> -  . = ALIGN(4096);
> -  PROVIDE (_unprotected_end = .);
> -
> -  . = ALIGN(4096);
> -  EXCEPTION_TABLE(0)
> -
> -  BUG_TABLE
> -
> -  .uml.setup.init : {
> -	__uml_setup_start = .;
> -	*(.uml.setup.init)
> -	__uml_setup_end = .;
> -  }
> -	
> -  .uml.help.init : {
> -	__uml_help_start = .;
> -	*(.uml.help.init)
> -	__uml_help_end = .;
> -  }
> -	
> -  .uml.postsetup.init : {
> -	__uml_postsetup_start = .;
> -	*(.uml.postsetup.init)
> -	__uml_postsetup_end = .;
> -  }
> -	
> -  .init.setup : {
> -	INIT_SETUP(0)
> -  }
> -
> -  PERCPU_SECTION(32)
> -	
> -  .initcall.init : {
> -	INIT_CALLS
> -  }
> -
> -  .con_initcall.init : {
> -	CON_INITCALL
> -  }
> -
> -  .exitcall : {
> -	__exitcall_begin = .;
> -	*(.exitcall.exit)
> -	__exitcall_end = .;
> -  }
> -
> -  .uml.exitcall : {
> -	__uml_exitcall_begin = .;
> -	*(.uml.exitcall.exit)
> -	__uml_exitcall_end = .;
> -  }
> -
> -  . = ALIGN(4);
> -  .altinstructions : {
> -	__alt_instructions = .;
> -	*(.altinstructions)
> -	__alt_instructions_end = .;
> -  }
> -  .altinstr_replacement : { *(.altinstr_replacement) }
> -  /* .exit.text is discard at runtime, not link time, to deal with references
> -     from .altinstructions and .eh_frame */
> -  .exit.text : { EXIT_TEXT }
> -  .exit.data : { *(.exit.data) }
> -
> -  .preinit_array : {
> -	__preinit_array_start = .;
> -	*(.preinit_array)
> -	__preinit_array_end = .;
> -  }
> -  .init_array : {
> -	__init_array_start = .;
> -	*(.init_array)
> -	__init_array_end = .;
> -  }
> -  .fini_array : {
> -	__fini_array_start = .;
> -	*(.fini_array)
> -	__fini_array_end = .;
> -  }
> -
> -   . = ALIGN(4096);
> -  .init.ramfs : {
> -	INIT_RAM_FS
> -  }
> -
> diff --git a/arch/um/kernel/dyn.lds.S b/arch/um/kernel/dyn.lds.S
> deleted file mode 100644
> index f5001481010c..000000000000
> --- a/arch/um/kernel/dyn.lds.S
> +++ /dev/null
> @@ -1,171 +0,0 @@
> -#include <asm/vmlinux.lds.h>
> -#include <asm/page.h>
> -
> -OUTPUT_FORMAT(ELF_FORMAT)
> -OUTPUT_ARCH(ELF_ARCH)
> -ENTRY(_start)
> -jiffies = jiffies_64;
> -
> -SECTIONS
> -{
> -  PROVIDE (__executable_start = START);
> -  . = START + SIZEOF_HEADERS;
> -  .interp         : { *(.interp) }
> -  __binary_start = .;
> -  . = ALIGN(4096);		/* Init code and data */
> -  _text = .;
> -  INIT_TEXT_SECTION(PAGE_SIZE)
> -
> -  . = ALIGN(PAGE_SIZE);
> -
> -  /* Read-only sections, merged into text segment: */
> -  .hash           : { *(.hash) }
> -  .gnu.hash       : { *(.gnu.hash) }
> -  .dynsym         : { *(.dynsym) }
> -  .dynstr         : { *(.dynstr) }
> -  .gnu.version    : { *(.gnu.version) }
> -  .gnu.version_d  : { *(.gnu.version_d) }
> -  .gnu.version_r  : { *(.gnu.version_r) }
> -  .rel.init       : { *(.rel.init) }
> -  .rela.init      : { *(.rela.init) }
> -  .rel.text       : { *(.rel.text .rel.text.* .rel.gnu.linkonce.t.*) }
> -  .rela.text      : { *(.rela.text .rela.text.* .rela.gnu.linkonce.t.*) }
> -  .rel.fini       : { *(.rel.fini) }
> -  .rela.fini      : { *(.rela.fini) }
> -  .rel.rodata     : { *(.rel.rodata .rel.rodata.* .rel.gnu.linkonce.r.*) }
> -  .rela.rodata    : { *(.rela.rodata .rela.rodata.* .rela.gnu.linkonce.r.*) }
> -  .rel.data       : { *(.rel.data .rel.data.* .rel.gnu.linkonce.d.*) }
> -  .rela.data      : { *(.rela.data .rela.data.* .rela.gnu.linkonce.d.*) }
> -  .rel.tdata	  : { *(.rel.tdata .rel.tdata.* .rel.gnu.linkonce.td.*) }
> -  .rela.tdata	  : { *(.rela.tdata .rela.tdata.* .rela.gnu.linkonce.td.*) }
> -  .rel.tbss	  : { *(.rel.tbss .rel.tbss.* .rel.gnu.linkonce.tb.*) }
> -  .rela.tbss	  : { *(.rela.tbss .rela.tbss.* .rela.gnu.linkonce.tb.*) }
> -  .rel.ctors      : { *(.rel.ctors) }
> -  .rela.ctors     : { *(.rela.ctors) }
> -  .rel.dtors      : { *(.rel.dtors) }
> -  .rela.dtors     : { *(.rela.dtors) }
> -  .rel.got        : { *(.rel.got) }
> -  .rela.got       : { *(.rela.got) }
> -  .rel.bss        : { *(.rel.bss .rel.bss.* .rel.gnu.linkonce.b.*) }
> -  .rela.bss       : { *(.rela.bss .rela.bss.* .rela.gnu.linkonce.b.*) }
> -  .rel.plt : {
> -	*(.rel.plt)
> -	PROVIDE_HIDDEN(__rel_iplt_start = .);
> -	*(.rel.iplt)
> -	PROVIDE_HIDDEN(__rel_iplt_end = .);
> -  }
> -  .rela.plt : {
> -	*(.rela.plt)
> -	PROVIDE_HIDDEN(__rela_iplt_start = .);
> -	*(.rela.iplt)
> -	PROVIDE_HIDDEN(__rela_iplt_end = .);
> -  }
> -  .init           : {
> -    KEEP (*(.init))
> -  } =0x90909090
> -  .plt            : { *(.plt) }
> -  .text           : {
> -    _stext = .;
> -    TEXT_TEXT
> -    SCHED_TEXT
> -    CPUIDLE_TEXT
> -    LOCK_TEXT
> -    IRQENTRY_TEXT
> -    SOFTIRQENTRY_TEXT
> -    *(.fixup)
> -    *(.stub .text.* .gnu.linkonce.t.*)
> -    /* .gnu.warning sections are handled specially by elf32.em.  */
> -    *(.gnu.warning)
> -
> -    . = ALIGN(PAGE_SIZE);
> -  } =0x90909090
> -  . = ALIGN(PAGE_SIZE);
> -  .syscall_stub : {
> -	__syscall_stub_start = .;
> -	*(.__syscall_stub*)
> -	__syscall_stub_end = .;
> -  }
> -  .fini           : {
> -    KEEP (*(.fini))
> -  } =0x90909090
> -
> -  .kstrtab : { *(.kstrtab) }
> -
> -  #include <asm/common.lds.S>
> -
> -  __init_begin = .;
> -  init.data : { INIT_DATA }
> -  __init_end = .;
> -
> -  /* Ensure the __preinit_array_start label is properly aligned.  We
> -     could instead move the label definition inside the section, but
> -     the linker would then create the section even if it turns out to
> -     be empty, which isn't pretty.  */
> -  . = ALIGN(32 / 8);
> -  .preinit_array     : { *(.preinit_array) }
> -  .init_array     : { *(.init_array) }
> -  .fini_array     : { *(.fini_array) }
> -  .data           : {
> -    INIT_TASK_DATA(KERNEL_STACK_SIZE)
> -    . = ALIGN(KERNEL_STACK_SIZE);
> -    *(.data..init_irqstack)
> -    DATA_DATA
> -    *(.data.* .gnu.linkonce.d.*)
> -    SORT(CONSTRUCTORS)
> -  }
> -  .data1          : { *(.data1) }
> -  .tdata	  : { *(.tdata .tdata.* .gnu.linkonce.td.*) }
> -  .tbss		  : { *(.tbss .tbss.* .gnu.linkonce.tb.*) *(.tcommon) }
> -  .eh_frame       : { KEEP (*(.eh_frame)) }
> -  .gcc_except_table   : { *(.gcc_except_table) }
> -  .dynamic        : { *(.dynamic) }
> -  .ctors          : {
> -    /* gcc uses crtbegin.o to find the start of
> -       the constructors, so we make sure it is
> -       first.  Because this is a wildcard, it
> -       doesn't matter if the user does not
> -       actually link against crtbegin.o; the
> -       linker won't look for a file to match a
> -       wildcard.  The wildcard also means that it
> -       doesn't matter which directory crtbegin.o
> -       is in.  */
> -    KEEP (*crtbegin.o(.ctors))
> -    /* We don't want to include the .ctor section from
> -       from the crtend.o file until after the sorted ctors.
> -       The .ctor section from the crtend file contains the
> -       end of ctors marker and it must be last */
> -    KEEP (*(EXCLUDE_FILE (*crtend.o ) .ctors))
> -    KEEP (*(SORT(.ctors.*)))
> -    KEEP (*(.ctors))
> -  }
> -  .dtors          : {
> -    KEEP (*crtbegin.o(.dtors))
> -    KEEP (*(EXCLUDE_FILE (*crtend.o ) .dtors))
> -    KEEP (*(SORT(.dtors.*)))
> -    KEEP (*(.dtors))
> -  }
> -  .jcr            : { KEEP (*(.jcr)) }
> -  .got            : { *(.got.plt) *(.got) }
> -  _edata = .;
> -  PROVIDE (edata = .);
> -  .bss            : {
> -   __bss_start = .;
> -   *(.dynbss)
> -   *(.bss .bss.* .gnu.linkonce.b.*)
> -   *(COMMON)
> -   /* Align here to ensure that the .bss section occupies space up to
> -      _end.  Align after .bss to ensure correct alignment even if the
> -      .bss section disappears because there are no input sections.  */
> -   . = ALIGN(32 / 8);
> -  . = ALIGN(32 / 8);
> -  }
> -   __bss_stop = .;
> -  _end = .;
> -  PROVIDE (end = .);
> -
> -  STABS_DEBUG
> -
> -  DWARF_DEBUG
> -
> -  DISCARDS
> -}
> diff --git a/arch/um/kernel/uml.lds.S b/arch/um/kernel/uml.lds.S
> deleted file mode 100644
> index 3b6dab3d4501..000000000000
> --- a/arch/um/kernel/uml.lds.S
> +++ /dev/null
> @@ -1,115 +0,0 @@
> -/* SPDX-License-Identifier: GPL-2.0 */
> -#include <asm/vmlinux.lds.h>
> -#include <asm/page.h>
> -
> -OUTPUT_FORMAT(ELF_FORMAT)
> -OUTPUT_ARCH(ELF_ARCH)
> -ENTRY(_start)
> -jiffies = jiffies_64;
> -
> -SECTIONS
> -{
> -  /* This must contain the right address - not quite the default ELF one.*/
> -  PROVIDE (__executable_start = START);
> -  /* Static binaries stick stuff here, like the sigreturn trampoline,
> -   * invisibly to objdump.  So, just make __binary_start equal to the very
> -   * beginning of the executable, and if there are unmapped pages after this,
> -   * they are forever unusable.
> -   */
> -  __binary_start = START;
> -
> -  . = START + SIZEOF_HEADERS;
> -  . = ALIGN(PAGE_SIZE);
> -
> -  _text = .;
> -  INIT_TEXT_SECTION(0)
> -
> -  .text      :
> -  {
> -    _stext = .;
> -    TEXT_TEXT
> -    SCHED_TEXT
> -    CPUIDLE_TEXT
> -    LOCK_TEXT
> -    IRQENTRY_TEXT
> -    SOFTIRQENTRY_TEXT
> -    *(.fixup)
> -    /* .gnu.warning sections are handled specially by elf32.em.  */
> -    *(.gnu.warning)
> -    *(.gnu.linkonce.t*)
> -  }
> -
> -  . = ALIGN(PAGE_SIZE);
> -  .syscall_stub : {
> -	__syscall_stub_start = .;
> -	*(.__syscall_stub*)
> -	__syscall_stub_end = .;
> -  }
> -
> -  /*
> -   * These are needed even in a static link, even if they wind up being empty.
> -   * Newer glibc needs these __rel{,a}_iplt_{start,end} symbols.
> -   */
> -  .rel.plt : {
> -	*(.rel.plt)
> -	PROVIDE_HIDDEN(__rel_iplt_start = .);
> -	*(.rel.iplt)
> -	PROVIDE_HIDDEN(__rel_iplt_end = .);
> -  }
> -  .rela.plt : {
> -	*(.rela.plt)
> -	PROVIDE_HIDDEN(__rela_iplt_start = .);
> -	*(.rela.iplt)
> -	PROVIDE_HIDDEN(__rela_iplt_end = .);
> -  }
> -
> -  #include <asm/common.lds.S>
> -
> -  __init_begin = .;
> -  init.data : { INIT_DATA }
> -  __init_end = .;
> -
> -  .data    :
> -  {
> -    INIT_TASK_DATA(KERNEL_STACK_SIZE)
> -    . = ALIGN(KERNEL_STACK_SIZE);
> -    *(.data..init_irqstack)
> -    DATA_DATA
> -    *(.gnu.linkonce.d*)
> -    CONSTRUCTORS
> -  }
> -  .data1   : { *(.data1) }
> -  .ctors         :
> -  {
> -    *(.ctors)
> -  }
> -  .dtors         :
> -  {
> -    *(.dtors)
> -  }
> -
> -  .got           : { *(.got.plt) *(.got) }
> -  .dynamic       : { *(.dynamic) }
> -  .tdata	  : { *(.tdata .tdata.* .gnu.linkonce.td.*) }
> -  .tbss		  : { *(.tbss .tbss.* .gnu.linkonce.tb.*) *(.tcommon) }
> -  /* We want the small data sections together, so single-instruction offsets
> -     can access them all, and initialized data all before uninitialized, so
> -     we can shorten the on-disk segment size.  */
> -  .sdata     : { *(.sdata) }
> -  _edata  =  .;
> -  PROVIDE (edata = .);
> -  . = ALIGN(PAGE_SIZE);
> -  __bss_start = .;
> -  PROVIDE(_bss_start = .);
> -  SBSS(0)
> -  BSS(0)
> -   __bss_stop = .;
> -  _end = .;
> -  PROVIDE (end = .);
> -
> -  STABS_DEBUG
> -
> -  DWARF_DEBUG
> -
> -  DISCARDS
> -}
> diff --git a/arch/um/kernel/vmlinux.lds.S b/arch/um/kernel/vmlinux.lds.S
> index 16e49bfa2b42..aaedd66772fa 100644
> --- a/arch/um/kernel/vmlinux.lds.S
> +++ b/arch/um/kernel/vmlinux.lds.S
> @@ -1,8 +1,87 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#include <asm/vmlinux.lds.h>
> +#include <asm/thread_info.h>
> +#include <asm/page.h>
> +#include <asm/thread_info.h>
> +#include <asm/cache.h>
> +#include <linux/export.h>
>   
> -KERNEL_STACK_SIZE = 4096 * (1 << CONFIG_KERNEL_STACK_ORDER);
> +OUTPUT_FORMAT(ELF_FORMAT)
> +jiffies = jiffies_64;
>   
> -#ifdef CONFIG_LD_SCRIPT_STATIC
> -#include "uml.lds.S"
> -#else
> -#include "dyn.lds.S"
> -#endif
> +
> +SECTIONS
> +{
> +	__init_begin = . ;
> +	HEAD_TEXT_SECTION
> +	INIT_TEXT_SECTION(PAGE_SIZE)
> +	INIT_DATA_SECTION(16)
> +	PERCPU_SECTION(L1_CACHE_BYTES)
> +	__init_end = . ;
> +
> +	_stext = . ;
> +	_text = . ;
> +	.text : ALIGN(THREAD_SIZE)
> +	{
> +		__binary_start = . ;
> +		TEXT_TEXT
> +		SCHED_TEXT
> +		LOCK_TEXT
> +		CPUIDLE_TEXT
> +		IRQENTRY_TEXT
> +		SOFTIRQENTRY_TEXT
> +	}
> +	_etext = . ;
> +
> +	.syscall_stub : ALIGN(PAGE_SIZE)
> +	{
> +		__syscall_stub_start = .;
> +		*(.__syscall_stub*)
> +		__syscall_stub_end = .;
> +	}
> +
> +	_sdata = . ;
> +	RO_DATA(PAGE_SIZE)
> +	RW_DATA(L1_CACHE_BYTES, PAGE_SIZE, THREAD_SIZE)
> +	.data : ALIGN(THREAD_SIZE)  { }
> +
> +	EXCEPTION_TABLE(16)
> +
> +	.uml.init_irqstack : ALIGN(THREAD_SIZE) {
> +		*(.data..init_irqstack)
> +	}
> +
> +	.uml.setup.init : ALIGN(8) {
> +		__uml_setup_start = .;
> +		*(.uml.setup.init)
> +		__uml_setup_end = .;
> +	}
> +
> +	.uml.help.init : ALIGN(8)  {
> +		__uml_help_start = .;
> +		*(.uml.help.init)
> +		__uml_help_end = .;
> +	}
> +
> +	.uml.postsetup.init : ALIGN(8) {
> +		__uml_postsetup_start = .;
> +		*(.uml.postsetup.init)
> +		__uml_postsetup_end = .;
> +	}
> +
> +	.uml.exitcall : ALIGN(8) {
> +		__uml_exitcall_begin = .;
> +		*(.uml.exitcall.exit)
> +		__uml_exitcall_end = .;
> +	}
> +	_edata = . ;
> +
> +	BSS_SECTION(0, 0, 0)
> +	_end = . ;
> +
> +	STABS_DEBUG
> +
> +	DWARF_DEBUG
> +
> +	DISCARDS
> +}
> diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
> index e6e2d9e5ff48..903fc8053d07 100755
> --- a/scripts/link-vmlinux.sh
> +++ b/scripts/link-vmlinux.sh
> @@ -102,36 +102,18 @@ vmlinux_link()
>   		strip_debug=-Wl,--strip-debug
>   	fi
>   
> -	if [ "${SRCARCH}" != "um" ]; then
> -		objects="--whole-archive			\
> -			${KBUILD_VMLINUX_OBJS}			\
> -			--no-whole-archive			\
> -			--start-group				\
> -			${KBUILD_VMLINUX_LIBS}			\
> -			--end-group				\
> -			${@}"
> -
> -		${LD} ${KBUILD_LDFLAGS} ${LDFLAGS_vmlinux}	\
> -			${strip_debug#-Wl,}			\
> -			-o ${output}				\
> -			-T ${lds} ${objects}
> -	else
> -		objects="-Wl,--whole-archive			\
> -			${KBUILD_VMLINUX_OBJS}			\
> -			-Wl,--no-whole-archive			\
> -			-Wl,--start-group			\
> -			${KBUILD_VMLINUX_LIBS}			\
> -			-Wl,--end-group				\
> -			${@}"
> -
> -		${CC} ${CFLAGS_vmlinux}				\
> -			${strip_debug}				\
> -			-o ${output}				\
> -			-Wl,-T,${lds}				\
> -			${objects}				\
> -			-lutil -lrt -lpthread
> -		rm -f linux
> -	fi
> +	objects="--whole-archive			\
> +		${KBUILD_VMLINUX_OBJS}			\
> +		--no-whole-archive			\
> +		--start-group				\
> +		${KBUILD_VMLINUX_LIBS}			\
> +		--end-group				\
> +		${@}"
> +
> +	${LD} ${KBUILD_LDFLAGS} ${LDFLAGS_vmlinux}	\
> +		${strip_debug#-Wl,}			\
> +		-o ${output}				\
> +		-T ${lds} ${objects}
>   }
>   
>   # generate .BTF typeinfo from DWARF debuginfo
> diff --git a/tools/um/Makefile b/tools/um/Makefile
> new file mode 100644
> index 000000000000..a63466ef7ef5
> --- /dev/null
> +++ b/tools/um/Makefile
> @@ -0,0 +1,72 @@
> +# Do not use make's built-in rules
> +# (this improves performance and avoids hard-to-debug behaviour);
> +# also do not print "Entering directory..." messages from make
> +.SUFFIXES:
> +MAKEFLAGS += -r --no-print-directory
> +
> +KCONFIG?=defconfig
> +
> +ifneq ($(silent),1)
> +  ifneq ($(V),1)
> +	QUIET_AUTOCONF       = @echo '  AUTOCONF '$@;
> +	Q = @
> +  endif
> +endif
> +
> +PREFIX   := /usr
> +
> +ifeq (,$(srctree))
> +  srctree := $(patsubst %/,%,$(dir $(shell pwd)))
> +  srctree := $(patsubst %/,%,$(dir $(srctree)))
> +endif
> +export srctree
> +
> +-include ../scripts/Makefile.include
> +
> +# OUTPUT fixup should be *after* include ../scripts/Makefile.include
> +ifneq ($(OUTPUT),)
> +  OUTPUT := $(OUTPUT)/tools/um/
> +else
> +  OUTPUT := $(CURDIR)/
> +endif
> +export OUTPUT
> +export objtree := $(OUTPUT)/../..
> +
> +
> +all:
> +
> +export CFLAGS += -I$(OUTPUT)/include -Iinclude -Wall -g -O2 -Wextra -fPIC \
> +	 -Wno-unused-parameter \
> +	 -Wno-missing-field-initializers -fno-strict-aliasing
> +
> +-include Targets
> +
> +TARGETS := $(progs-y:%=$(OUTPUT)%)
> +TARGETS += $(libs-y:%=$(OUTPUT)%)
> +all: $(TARGETS)
> +
> +# rule to build linux.o
> +$(OUTPUT)lib/linux.o:
> +	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) $(KCONFIG)
> +	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) install INSTALL_PATH=$(OUTPUT)
> +
> +$(OUTPUT)liblinux.a: $(OUTPUT)lib/linux.o $(OUTPUT)uml/liblinux-in.o
> +	$(QUIET_AR)$(AR) -rc $@ $^
> +
> +# rule to link programs
> +$(OUTPUT)%: $(OUTPUT)%-in.o $(OUTPUT)liblinux.a
> +	$(QUIET_LINK)$(CC) $(LDFLAGS) $(LDFLAGS_$(notdir $*)-y) -o $@ $^ $(LDLIBS) $(LDLIBS_$(notdir $*)-y)
> +
> +$(OUTPUT)%-in.o: FORCE
> +	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
> +
> +clean:
> +	$(call QUIET_CLEAN, objects)find $(OUTPUT) -name '*.o' -delete -o -name '\.*.cmd'\
> +	 -delete -o -name '\.*.d' -delete
> +	$(call QUIET_CLEAN, liblinux.a)$(RM) $(OUTPUT)/liblinux.a
> +	$(call QUIET_CLEAN, $(TARGETS))$(RM) $(TARGETS)
> +
> +FORCE: ;
> +.PHONY: all clean FORCE
> +.IGNORE: clean
> +.NOTPARALLEL : lib/linux.o
> diff --git a/tools/um/Targets b/tools/um/Targets
> new file mode 100644
> index 000000000000..a4711f1ef422
> --- /dev/null
> +++ b/tools/um/Targets
> @@ -0,0 +1,4 @@
> +progs-y += uml/linux
> +LDLIBS_linux-y := -lrt -lpthread -lutil
> +LDFLAGS_linux-y := -no-pie -Wl,--wrap,malloc -Wl,--wrap,free -Wl,--wrap,calloc
> +LDFLAGS_linux-$(UML_STATIC) += -static
> diff --git a/tools/um/uml/Build b/tools/um/uml/Build
> new file mode 100644
> index 000000000000..e69de29bb2d1
> 

-- 
Anton R. Ivanov
Cambridgegreys Limited. Registered in England. Company Number 10273661
https://www.cambridgegreys.com/

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v6 01/21] um: split build in kernel and host parts
@ 2020-09-24  7:33           ` Anton Ivanov
  0 siblings, 0 replies; 476+ messages in thread
From: Anton Ivanov @ 2020-09-24  7:33 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Octavian Purdila,
	retrage01



On 24/09/2020 08:12, Hajime Tazaki wrote:
> From: Octavian Purdila <tavi@cs.pub.ro>
> 
> This patch splits the UML build in two parts: a kernel part that
> generates a relocatable object (linux.o) and a host build part that
> links the relocatable object into an executable.
> 
> This allows us to simplify the linker script since we can now remove
> the host bits (e.g. .rel sections). It also gives us greater
> flexibility since it will be much easier to support other
> architectures and OSes if we use the standard linker script.
> 
> The host build part has been implemented in tools/um so that we can
> reuse the available host build infrastructure.
> 
> The patch also changes the UML build invocation, if before
> 
>   $ make ARCH=um defconfig
>   $ make ARCH=um
> 
> was generating the executable now this only generates the relocatable
> object.

This will break packaging in all distributions. We need to figure out an alternative way which is backward compatible with their builds.

> 
> To fully build UML use:
> 
>   $ make -C tools/um
> 
> which will generate tools/lkl/linux as the UML executable. To
> statically compiled UML use:
> 
>   $ make -C tools/lkl UML_STATIC=y
> 
> Signed-off-by: Octavian Purdila <tavi@cs.pub.ro>
> Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
> ---
>   arch/um/Kconfig                  |  12 +--
>   arch/um/Makefile                 |  14 ++-
>   arch/um/include/asm/common.lds.S |  99 ------------------
>   arch/um/kernel/dyn.lds.S         | 171 -------------------------------
>   arch/um/kernel/uml.lds.S         | 115 ---------------------
>   arch/um/kernel/vmlinux.lds.S     |  91 ++++++++++++++--
>   scripts/link-vmlinux.sh          |  42 +++-----
>   tools/um/Makefile                |  72 +++++++++++++
>   tools/um/Targets                 |   4 +
>   tools/um/uml/Build               |   0
>   10 files changed, 184 insertions(+), 436 deletions(-)
>   delete mode 100644 arch/um/include/asm/common.lds.S
>   delete mode 100644 arch/um/kernel/dyn.lds.S
>   delete mode 100644 arch/um/kernel/uml.lds.S
>   create mode 100644 tools/um/Makefile
>   create mode 100644 tools/um/Targets
>   create mode 100644 tools/um/uml/Build
> 
> diff --git a/arch/um/Kconfig b/arch/um/Kconfig
> index eb51fec75948..e41d31d0d875 100644
> --- a/arch/um/Kconfig
> +++ b/arch/um/Kconfig
> @@ -20,6 +20,7 @@ config UML
>   	select GENERIC_CLOCKEVENTS
>   	select HAVE_GCC_PLUGINS
>   	select TTY # Needed for line.c
> +	select MODULE_REL_CRCS if MODVERSIONS
>   
>   config MMU
>   	bool
> @@ -79,17 +80,6 @@ config STATIC_LINK
>   	  NOTE: This option is incompatible with some networking features which
>   	  depend on features that require being dynamically loaded (like NSS).
>   
> -config LD_SCRIPT_STATIC
> -	bool
> -	default y
> -	depends on STATIC_LINK
> -
> -config LD_SCRIPT_DYN
> -	bool
> -	default y
> -	depends on !LD_SCRIPT_STATIC
> -	select MODULE_REL_CRCS if MODVERSIONS
> -
>   config HOSTFS
>   	tristate "Host filesystem"
>   	help
> diff --git a/arch/um/Makefile b/arch/um/Makefile
> index 1cea46ff9bb7..fdfaff4633e8 100644
> --- a/arch/um/Makefile
> +++ b/arch/um/Makefile
> @@ -95,14 +95,20 @@ KBUILD_CPPFLAGS += -I$(srctree)/$(HOST_DIR)/include \
>   KERNEL_DEFINES = $(strip -Derrno=kernel_errno -Dsigprocmask=kernel_sigprocmask \
>   			 -Dmktime=kernel_mktime $(ARCH_KERNEL_DEFINES))
>   KBUILD_CFLAGS += $(KERNEL_DEFINES)
> +LDFLAGS_vmlinux += -r
>   
> -PHONY += linux
> +PHONY += linux.o
>   
> -all: linux
> +all: linux.o
>   
> -linux: vmlinux
> +linux.o: vmlinux
>   	@echo '  LINK $@'
> -	$(Q)ln -f $< $@
> +	$(Q)$(OBJCOPY) -R .eh_frame $< $@
> +
> +install: linux.o
> +	@echo "  INSTALL $(INSTALL_PATH)/lib/$<"
> +	@mkdir -p $(INSTALL_PATH)/lib/
> +	@cp $< $(INSTALL_PATH)/lib/
>   
>   define archhelp
>     echo '* linux		- Binary kernel image (./linux) - for backward'
> diff --git a/arch/um/include/asm/common.lds.S b/arch/um/include/asm/common.lds.S
> deleted file mode 100644
> index eca6c452a41b..000000000000
> --- a/arch/um/include/asm/common.lds.S
> +++ /dev/null
> @@ -1,99 +0,0 @@
> -/* SPDX-License-Identifier: GPL-2.0 */
> -#include <asm-generic/vmlinux.lds.h>
> -
> -  .fini      : { *(.fini)    } =0x9090
> -  _etext = .;
> -  PROVIDE (etext = .);
> -
> -  . = ALIGN(4096);
> -  _sdata = .;
> -  PROVIDE (sdata = .);
> -
> -  RO_DATA(4096)
> -
> -  .unprotected : { *(.unprotected) }
> -  . = ALIGN(4096);
> -  PROVIDE (_unprotected_end = .);
> -
> -  . = ALIGN(4096);
> -  EXCEPTION_TABLE(0)
> -
> -  BUG_TABLE
> -
> -  .uml.setup.init : {
> -	__uml_setup_start = .;
> -	*(.uml.setup.init)
> -	__uml_setup_end = .;
> -  }
> -	
> -  .uml.help.init : {
> -	__uml_help_start = .;
> -	*(.uml.help.init)
> -	__uml_help_end = .;
> -  }
> -	
> -  .uml.postsetup.init : {
> -	__uml_postsetup_start = .;
> -	*(.uml.postsetup.init)
> -	__uml_postsetup_end = .;
> -  }
> -	
> -  .init.setup : {
> -	INIT_SETUP(0)
> -  }
> -
> -  PERCPU_SECTION(32)
> -	
> -  .initcall.init : {
> -	INIT_CALLS
> -  }
> -
> -  .con_initcall.init : {
> -	CON_INITCALL
> -  }
> -
> -  .exitcall : {
> -	__exitcall_begin = .;
> -	*(.exitcall.exit)
> -	__exitcall_end = .;
> -  }
> -
> -  .uml.exitcall : {
> -	__uml_exitcall_begin = .;
> -	*(.uml.exitcall.exit)
> -	__uml_exitcall_end = .;
> -  }
> -
> -  . = ALIGN(4);
> -  .altinstructions : {
> -	__alt_instructions = .;
> -	*(.altinstructions)
> -	__alt_instructions_end = .;
> -  }
> -  .altinstr_replacement : { *(.altinstr_replacement) }
> -  /* .exit.text is discard at runtime, not link time, to deal with references
> -     from .altinstructions and .eh_frame */
> -  .exit.text : { EXIT_TEXT }
> -  .exit.data : { *(.exit.data) }
> -
> -  .preinit_array : {
> -	__preinit_array_start = .;
> -	*(.preinit_array)
> -	__preinit_array_end = .;
> -  }
> -  .init_array : {
> -	__init_array_start = .;
> -	*(.init_array)
> -	__init_array_end = .;
> -  }
> -  .fini_array : {
> -	__fini_array_start = .;
> -	*(.fini_array)
> -	__fini_array_end = .;
> -  }
> -
> -   . = ALIGN(4096);
> -  .init.ramfs : {
> -	INIT_RAM_FS
> -  }
> -
> diff --git a/arch/um/kernel/dyn.lds.S b/arch/um/kernel/dyn.lds.S
> deleted file mode 100644
> index f5001481010c..000000000000
> --- a/arch/um/kernel/dyn.lds.S
> +++ /dev/null
> @@ -1,171 +0,0 @@
> -#include <asm/vmlinux.lds.h>
> -#include <asm/page.h>
> -
> -OUTPUT_FORMAT(ELF_FORMAT)
> -OUTPUT_ARCH(ELF_ARCH)
> -ENTRY(_start)
> -jiffies = jiffies_64;
> -
> -SECTIONS
> -{
> -  PROVIDE (__executable_start = START);
> -  . = START + SIZEOF_HEADERS;
> -  .interp         : { *(.interp) }
> -  __binary_start = .;
> -  . = ALIGN(4096);		/* Init code and data */
> -  _text = .;
> -  INIT_TEXT_SECTION(PAGE_SIZE)
> -
> -  . = ALIGN(PAGE_SIZE);
> -
> -  /* Read-only sections, merged into text segment: */
> -  .hash           : { *(.hash) }
> -  .gnu.hash       : { *(.gnu.hash) }
> -  .dynsym         : { *(.dynsym) }
> -  .dynstr         : { *(.dynstr) }
> -  .gnu.version    : { *(.gnu.version) }
> -  .gnu.version_d  : { *(.gnu.version_d) }
> -  .gnu.version_r  : { *(.gnu.version_r) }
> -  .rel.init       : { *(.rel.init) }
> -  .rela.init      : { *(.rela.init) }
> -  .rel.text       : { *(.rel.text .rel.text.* .rel.gnu.linkonce.t.*) }
> -  .rela.text      : { *(.rela.text .rela.text.* .rela.gnu.linkonce.t.*) }
> -  .rel.fini       : { *(.rel.fini) }
> -  .rela.fini      : { *(.rela.fini) }
> -  .rel.rodata     : { *(.rel.rodata .rel.rodata.* .rel.gnu.linkonce.r.*) }
> -  .rela.rodata    : { *(.rela.rodata .rela.rodata.* .rela.gnu.linkonce.r.*) }
> -  .rel.data       : { *(.rel.data .rel.data.* .rel.gnu.linkonce.d.*) }
> -  .rela.data      : { *(.rela.data .rela.data.* .rela.gnu.linkonce.d.*) }
> -  .rel.tdata	  : { *(.rel.tdata .rel.tdata.* .rel.gnu.linkonce.td.*) }
> -  .rela.tdata	  : { *(.rela.tdata .rela.tdata.* .rela.gnu.linkonce.td.*) }
> -  .rel.tbss	  : { *(.rel.tbss .rel.tbss.* .rel.gnu.linkonce.tb.*) }
> -  .rela.tbss	  : { *(.rela.tbss .rela.tbss.* .rela.gnu.linkonce.tb.*) }
> -  .rel.ctors      : { *(.rel.ctors) }
> -  .rela.ctors     : { *(.rela.ctors) }
> -  .rel.dtors      : { *(.rel.dtors) }
> -  .rela.dtors     : { *(.rela.dtors) }
> -  .rel.got        : { *(.rel.got) }
> -  .rela.got       : { *(.rela.got) }
> -  .rel.bss        : { *(.rel.bss .rel.bss.* .rel.gnu.linkonce.b.*) }
> -  .rela.bss       : { *(.rela.bss .rela.bss.* .rela.gnu.linkonce.b.*) }
> -  .rel.plt : {
> -	*(.rel.plt)
> -	PROVIDE_HIDDEN(__rel_iplt_start = .);
> -	*(.rel.iplt)
> -	PROVIDE_HIDDEN(__rel_iplt_end = .);
> -  }
> -  .rela.plt : {
> -	*(.rela.plt)
> -	PROVIDE_HIDDEN(__rela_iplt_start = .);
> -	*(.rela.iplt)
> -	PROVIDE_HIDDEN(__rela_iplt_end = .);
> -  }
> -  .init           : {
> -    KEEP (*(.init))
> -  } =0x90909090
> -  .plt            : { *(.plt) }
> -  .text           : {
> -    _stext = .;
> -    TEXT_TEXT
> -    SCHED_TEXT
> -    CPUIDLE_TEXT
> -    LOCK_TEXT
> -    IRQENTRY_TEXT
> -    SOFTIRQENTRY_TEXT
> -    *(.fixup)
> -    *(.stub .text.* .gnu.linkonce.t.*)
> -    /* .gnu.warning sections are handled specially by elf32.em.  */
> -    *(.gnu.warning)
> -
> -    . = ALIGN(PAGE_SIZE);
> -  } =0x90909090
> -  . = ALIGN(PAGE_SIZE);
> -  .syscall_stub : {
> -	__syscall_stub_start = .;
> -	*(.__syscall_stub*)
> -	__syscall_stub_end = .;
> -  }
> -  .fini           : {
> -    KEEP (*(.fini))
> -  } =0x90909090
> -
> -  .kstrtab : { *(.kstrtab) }
> -
> -  #include <asm/common.lds.S>
> -
> -  __init_begin = .;
> -  init.data : { INIT_DATA }
> -  __init_end = .;
> -
> -  /* Ensure the __preinit_array_start label is properly aligned.  We
> -     could instead move the label definition inside the section, but
> -     the linker would then create the section even if it turns out to
> -     be empty, which isn't pretty.  */
> -  . = ALIGN(32 / 8);
> -  .preinit_array     : { *(.preinit_array) }
> -  .init_array     : { *(.init_array) }
> -  .fini_array     : { *(.fini_array) }
> -  .data           : {
> -    INIT_TASK_DATA(KERNEL_STACK_SIZE)
> -    . = ALIGN(KERNEL_STACK_SIZE);
> -    *(.data..init_irqstack)
> -    DATA_DATA
> -    *(.data.* .gnu.linkonce.d.*)
> -    SORT(CONSTRUCTORS)
> -  }
> -  .data1          : { *(.data1) }
> -  .tdata	  : { *(.tdata .tdata.* .gnu.linkonce.td.*) }
> -  .tbss		  : { *(.tbss .tbss.* .gnu.linkonce.tb.*) *(.tcommon) }
> -  .eh_frame       : { KEEP (*(.eh_frame)) }
> -  .gcc_except_table   : { *(.gcc_except_table) }
> -  .dynamic        : { *(.dynamic) }
> -  .ctors          : {
> -    /* gcc uses crtbegin.o to find the start of
> -       the constructors, so we make sure it is
> -       first.  Because this is a wildcard, it
> -       doesn't matter if the user does not
> -       actually link against crtbegin.o; the
> -       linker won't look for a file to match a
> -       wildcard.  The wildcard also means that it
> -       doesn't matter which directory crtbegin.o
> -       is in.  */
> -    KEEP (*crtbegin.o(.ctors))
> -    /* We don't want to include the .ctor section from
> -       from the crtend.o file until after the sorted ctors.
> -       The .ctor section from the crtend file contains the
> -       end of ctors marker and it must be last */
> -    KEEP (*(EXCLUDE_FILE (*crtend.o ) .ctors))
> -    KEEP (*(SORT(.ctors.*)))
> -    KEEP (*(.ctors))
> -  }
> -  .dtors          : {
> -    KEEP (*crtbegin.o(.dtors))
> -    KEEP (*(EXCLUDE_FILE (*crtend.o ) .dtors))
> -    KEEP (*(SORT(.dtors.*)))
> -    KEEP (*(.dtors))
> -  }
> -  .jcr            : { KEEP (*(.jcr)) }
> -  .got            : { *(.got.plt) *(.got) }
> -  _edata = .;
> -  PROVIDE (edata = .);
> -  .bss            : {
> -   __bss_start = .;
> -   *(.dynbss)
> -   *(.bss .bss.* .gnu.linkonce.b.*)
> -   *(COMMON)
> -   /* Align here to ensure that the .bss section occupies space up to
> -      _end.  Align after .bss to ensure correct alignment even if the
> -      .bss section disappears because there are no input sections.  */
> -   . = ALIGN(32 / 8);
> -  . = ALIGN(32 / 8);
> -  }
> -   __bss_stop = .;
> -  _end = .;
> -  PROVIDE (end = .);
> -
> -  STABS_DEBUG
> -
> -  DWARF_DEBUG
> -
> -  DISCARDS
> -}
> diff --git a/arch/um/kernel/uml.lds.S b/arch/um/kernel/uml.lds.S
> deleted file mode 100644
> index 3b6dab3d4501..000000000000
> --- a/arch/um/kernel/uml.lds.S
> +++ /dev/null
> @@ -1,115 +0,0 @@
> -/* SPDX-License-Identifier: GPL-2.0 */
> -#include <asm/vmlinux.lds.h>
> -#include <asm/page.h>
> -
> -OUTPUT_FORMAT(ELF_FORMAT)
> -OUTPUT_ARCH(ELF_ARCH)
> -ENTRY(_start)
> -jiffies = jiffies_64;
> -
> -SECTIONS
> -{
> -  /* This must contain the right address - not quite the default ELF one.*/
> -  PROVIDE (__executable_start = START);
> -  /* Static binaries stick stuff here, like the sigreturn trampoline,
> -   * invisibly to objdump.  So, just make __binary_start equal to the very
> -   * beginning of the executable, and if there are unmapped pages after this,
> -   * they are forever unusable.
> -   */
> -  __binary_start = START;
> -
> -  . = START + SIZEOF_HEADERS;
> -  . = ALIGN(PAGE_SIZE);
> -
> -  _text = .;
> -  INIT_TEXT_SECTION(0)
> -
> -  .text      :
> -  {
> -    _stext = .;
> -    TEXT_TEXT
> -    SCHED_TEXT
> -    CPUIDLE_TEXT
> -    LOCK_TEXT
> -    IRQENTRY_TEXT
> -    SOFTIRQENTRY_TEXT
> -    *(.fixup)
> -    /* .gnu.warning sections are handled specially by elf32.em.  */
> -    *(.gnu.warning)
> -    *(.gnu.linkonce.t*)
> -  }
> -
> -  . = ALIGN(PAGE_SIZE);
> -  .syscall_stub : {
> -	__syscall_stub_start = .;
> -	*(.__syscall_stub*)
> -	__syscall_stub_end = .;
> -  }
> -
> -  /*
> -   * These are needed even in a static link, even if they wind up being empty.
> -   * Newer glibc needs these __rel{,a}_iplt_{start,end} symbols.
> -   */
> -  .rel.plt : {
> -	*(.rel.plt)
> -	PROVIDE_HIDDEN(__rel_iplt_start = .);
> -	*(.rel.iplt)
> -	PROVIDE_HIDDEN(__rel_iplt_end = .);
> -  }
> -  .rela.plt : {
> -	*(.rela.plt)
> -	PROVIDE_HIDDEN(__rela_iplt_start = .);
> -	*(.rela.iplt)
> -	PROVIDE_HIDDEN(__rela_iplt_end = .);
> -  }
> -
> -  #include <asm/common.lds.S>
> -
> -  __init_begin = .;
> -  init.data : { INIT_DATA }
> -  __init_end = .;
> -
> -  .data    :
> -  {
> -    INIT_TASK_DATA(KERNEL_STACK_SIZE)
> -    . = ALIGN(KERNEL_STACK_SIZE);
> -    *(.data..init_irqstack)
> -    DATA_DATA
> -    *(.gnu.linkonce.d*)
> -    CONSTRUCTORS
> -  }
> -  .data1   : { *(.data1) }
> -  .ctors         :
> -  {
> -    *(.ctors)
> -  }
> -  .dtors         :
> -  {
> -    *(.dtors)
> -  }
> -
> -  .got           : { *(.got.plt) *(.got) }
> -  .dynamic       : { *(.dynamic) }
> -  .tdata	  : { *(.tdata .tdata.* .gnu.linkonce.td.*) }
> -  .tbss		  : { *(.tbss .tbss.* .gnu.linkonce.tb.*) *(.tcommon) }
> -  /* We want the small data sections together, so single-instruction offsets
> -     can access them all, and initialized data all before uninitialized, so
> -     we can shorten the on-disk segment size.  */
> -  .sdata     : { *(.sdata) }
> -  _edata  =  .;
> -  PROVIDE (edata = .);
> -  . = ALIGN(PAGE_SIZE);
> -  __bss_start = .;
> -  PROVIDE(_bss_start = .);
> -  SBSS(0)
> -  BSS(0)
> -   __bss_stop = .;
> -  _end = .;
> -  PROVIDE (end = .);
> -
> -  STABS_DEBUG
> -
> -  DWARF_DEBUG
> -
> -  DISCARDS
> -}
> diff --git a/arch/um/kernel/vmlinux.lds.S b/arch/um/kernel/vmlinux.lds.S
> index 16e49bfa2b42..aaedd66772fa 100644
> --- a/arch/um/kernel/vmlinux.lds.S
> +++ b/arch/um/kernel/vmlinux.lds.S
> @@ -1,8 +1,87 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#include <asm/vmlinux.lds.h>
> +#include <asm/thread_info.h>
> +#include <asm/page.h>
> +#include <asm/thread_info.h>
> +#include <asm/cache.h>
> +#include <linux/export.h>
>   
> -KERNEL_STACK_SIZE = 4096 * (1 << CONFIG_KERNEL_STACK_ORDER);
> +OUTPUT_FORMAT(ELF_FORMAT)
> +jiffies = jiffies_64;
>   
> -#ifdef CONFIG_LD_SCRIPT_STATIC
> -#include "uml.lds.S"
> -#else
> -#include "dyn.lds.S"
> -#endif
> +
> +SECTIONS
> +{
> +	__init_begin = . ;
> +	HEAD_TEXT_SECTION
> +	INIT_TEXT_SECTION(PAGE_SIZE)
> +	INIT_DATA_SECTION(16)
> +	PERCPU_SECTION(L1_CACHE_BYTES)
> +	__init_end = . ;
> +
> +	_stext = . ;
> +	_text = . ;
> +	.text : ALIGN(THREAD_SIZE)
> +	{
> +		__binary_start = . ;
> +		TEXT_TEXT
> +		SCHED_TEXT
> +		LOCK_TEXT
> +		CPUIDLE_TEXT
> +		IRQENTRY_TEXT
> +		SOFTIRQENTRY_TEXT
> +	}
> +	_etext = . ;
> +
> +	.syscall_stub : ALIGN(PAGE_SIZE)
> +	{
> +		__syscall_stub_start = .;
> +		*(.__syscall_stub*)
> +		__syscall_stub_end = .;
> +	}
> +
> +	_sdata = . ;
> +	RO_DATA(PAGE_SIZE)
> +	RW_DATA(L1_CACHE_BYTES, PAGE_SIZE, THREAD_SIZE)
> +	.data : ALIGN(THREAD_SIZE)  { }
> +
> +	EXCEPTION_TABLE(16)
> +
> +	.uml.init_irqstack : ALIGN(THREAD_SIZE) {
> +		*(.data..init_irqstack)
> +	}
> +
> +	.uml.setup.init : ALIGN(8) {
> +		__uml_setup_start = .;
> +		*(.uml.setup.init)
> +		__uml_setup_end = .;
> +	}
> +
> +	.uml.help.init : ALIGN(8)  {
> +		__uml_help_start = .;
> +		*(.uml.help.init)
> +		__uml_help_end = .;
> +	}
> +
> +	.uml.postsetup.init : ALIGN(8) {
> +		__uml_postsetup_start = .;
> +		*(.uml.postsetup.init)
> +		__uml_postsetup_end = .;
> +	}
> +
> +	.uml.exitcall : ALIGN(8) {
> +		__uml_exitcall_begin = .;
> +		*(.uml.exitcall.exit)
> +		__uml_exitcall_end = .;
> +	}
> +	_edata = . ;
> +
> +	BSS_SECTION(0, 0, 0)
> +	_end = . ;
> +
> +	STABS_DEBUG
> +
> +	DWARF_DEBUG
> +
> +	DISCARDS
> +}
> diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
> index e6e2d9e5ff48..903fc8053d07 100755
> --- a/scripts/link-vmlinux.sh
> +++ b/scripts/link-vmlinux.sh
> @@ -102,36 +102,18 @@ vmlinux_link()
>   		strip_debug=-Wl,--strip-debug
>   	fi
>   
> -	if [ "${SRCARCH}" != "um" ]; then
> -		objects="--whole-archive			\
> -			${KBUILD_VMLINUX_OBJS}			\
> -			--no-whole-archive			\
> -			--start-group				\
> -			${KBUILD_VMLINUX_LIBS}			\
> -			--end-group				\
> -			${@}"
> -
> -		${LD} ${KBUILD_LDFLAGS} ${LDFLAGS_vmlinux}	\
> -			${strip_debug#-Wl,}			\
> -			-o ${output}				\
> -			-T ${lds} ${objects}
> -	else
> -		objects="-Wl,--whole-archive			\
> -			${KBUILD_VMLINUX_OBJS}			\
> -			-Wl,--no-whole-archive			\
> -			-Wl,--start-group			\
> -			${KBUILD_VMLINUX_LIBS}			\
> -			-Wl,--end-group				\
> -			${@}"
> -
> -		${CC} ${CFLAGS_vmlinux}				\
> -			${strip_debug}				\
> -			-o ${output}				\
> -			-Wl,-T,${lds}				\
> -			${objects}				\
> -			-lutil -lrt -lpthread
> -		rm -f linux
> -	fi
> +	objects="--whole-archive			\
> +		${KBUILD_VMLINUX_OBJS}			\
> +		--no-whole-archive			\
> +		--start-group				\
> +		${KBUILD_VMLINUX_LIBS}			\
> +		--end-group				\
> +		${@}"
> +
> +	${LD} ${KBUILD_LDFLAGS} ${LDFLAGS_vmlinux}	\
> +		${strip_debug#-Wl,}			\
> +		-o ${output}				\
> +		-T ${lds} ${objects}
>   }
>   
>   # generate .BTF typeinfo from DWARF debuginfo
> diff --git a/tools/um/Makefile b/tools/um/Makefile
> new file mode 100644
> index 000000000000..a63466ef7ef5
> --- /dev/null
> +++ b/tools/um/Makefile
> @@ -0,0 +1,72 @@
> +# Do not use make's built-in rules
> +# (this improves performance and avoids hard-to-debug behaviour);
> +# also do not print "Entering directory..." messages from make
> +.SUFFIXES:
> +MAKEFLAGS += -r --no-print-directory
> +
> +KCONFIG?=defconfig
> +
> +ifneq ($(silent),1)
> +  ifneq ($(V),1)
> +	QUIET_AUTOCONF       = @echo '  AUTOCONF '$@;
> +	Q = @
> +  endif
> +endif
> +
> +PREFIX   := /usr
> +
> +ifeq (,$(srctree))
> +  srctree := $(patsubst %/,%,$(dir $(shell pwd)))
> +  srctree := $(patsubst %/,%,$(dir $(srctree)))
> +endif
> +export srctree
> +
> +-include ../scripts/Makefile.include
> +
> +# OUTPUT fixup should be *after* include ../scripts/Makefile.include
> +ifneq ($(OUTPUT),)
> +  OUTPUT := $(OUTPUT)/tools/um/
> +else
> +  OUTPUT := $(CURDIR)/
> +endif
> +export OUTPUT
> +export objtree := $(OUTPUT)/../..
> +
> +
> +all:
> +
> +export CFLAGS += -I$(OUTPUT)/include -Iinclude -Wall -g -O2 -Wextra -fPIC \
> +	 -Wno-unused-parameter \
> +	 -Wno-missing-field-initializers -fno-strict-aliasing
> +
> +-include Targets
> +
> +TARGETS := $(progs-y:%=$(OUTPUT)%)
> +TARGETS += $(libs-y:%=$(OUTPUT)%)
> +all: $(TARGETS)
> +
> +# rule to build linux.o
> +$(OUTPUT)lib/linux.o:
> +	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) $(KCONFIG)
> +	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) install INSTALL_PATH=$(OUTPUT)
> +
> +$(OUTPUT)liblinux.a: $(OUTPUT)lib/linux.o $(OUTPUT)uml/liblinux-in.o
> +	$(QUIET_AR)$(AR) -rc $@ $^
> +
> +# rule to link programs
> +$(OUTPUT)%: $(OUTPUT)%-in.o $(OUTPUT)liblinux.a
> +	$(QUIET_LINK)$(CC) $(LDFLAGS) $(LDFLAGS_$(notdir $*)-y) -o $@ $^ $(LDLIBS) $(LDLIBS_$(notdir $*)-y)
> +
> +$(OUTPUT)%-in.o: FORCE
> +	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
> +
> +clean:
> +	$(call QUIET_CLEAN, objects)find $(OUTPUT) -name '*.o' -delete -o -name '\.*.cmd'\
> +	 -delete -o -name '\.*.d' -delete
> +	$(call QUIET_CLEAN, liblinux.a)$(RM) $(OUTPUT)/liblinux.a
> +	$(call QUIET_CLEAN, $(TARGETS))$(RM) $(TARGETS)
> +
> +FORCE: ;
> +.PHONY: all clean FORCE
> +.IGNORE: clean
> +.NOTPARALLEL : lib/linux.o
> diff --git a/tools/um/Targets b/tools/um/Targets
> new file mode 100644
> index 000000000000..a4711f1ef422
> --- /dev/null
> +++ b/tools/um/Targets
> @@ -0,0 +1,4 @@
> +progs-y += uml/linux
> +LDLIBS_linux-y := -lrt -lpthread -lutil
> +LDFLAGS_linux-y := -no-pie -Wl,--wrap,malloc -Wl,--wrap,free -Wl,--wrap,calloc
> +LDFLAGS_linux-$(UML_STATIC) += -static
> diff --git a/tools/um/uml/Build b/tools/um/uml/Build
> new file mode 100644
> index 000000000000..e69de29bb2d1
> 

-- 
Anton R. Ivanov
Cambridgegreys Limited. Registered in England. Company Number 10273661
https://www.cambridgegreys.com/

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v6 01/21] um: split build in kernel and host parts
  2020-09-24  7:12         ` Hajime Tazaki
@ 2020-09-24  7:36           ` Anton Ivanov
  -1 siblings, 0 replies; 476+ messages in thread
From: Anton Ivanov @ 2020-09-24  7:36 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch,
	Octavian Purdila



On 24/09/2020 08:12, Hajime Tazaki wrote:
> From: Octavian Purdila <tavi@cs.pub.ro>
> 
> This patch splits the UML build in two parts: a kernel part that
> generates a relocatable object (linux.o) and a host build part that
> links the relocatable object into an executable.
> 
> This allows us to simplify the linker script since we can now remove
> the host bits (e.g. .rel sections). It also gives us greater
> flexibility since it will be much easier to support other
> architectures and OSes if we use the standard linker script.
> 
> The host build part has been implemented in tools/um so that we can
> reuse the available host build infrastructure.
> 
> The patch also changes the UML build invocation, if before
> 
>   $ make ARCH=um defconfig
>   $ make ARCH=um
> 
> was generating the executable now this only generates the relocatable
> object.

It also fails now.


ERROR: modpost: "memmove" [drivers/net/slip/slhc.ko] undefined!
ERROR: modpost: "memset" [drivers/net/ppp/ppp_generic.ko] undefined!
ERROR: modpost: "memset" [drivers/net/tun.ko] undefined!
ERROR: modpost: "memmove" [drivers/block/loop.ko] undefined!
ERROR: modpost: "memset" [drivers/block/loop.ko] undefined!
ERROR: modpost: "memset" [fs/autofs/autofs4.ko] undefined!
ERROR: modpost: "memset" [fs/isofs/isofs.ko] undefined!
ERROR: modpost: "memset" [fs/binfmt_misc.ko] undefined!
make[1]: *** [scripts/Makefile.modpost:111: Module.symvers] Error 1
make[1]: *** Deleting file 'Module.symvers'
make: *** [Makefile:1388: modules] Error 2
make: *** Waiting for unfinished jobs....


> 
> To fully build UML use:
> 
>   $ make -C tools/um
> 
> which will generate tools/lkl/linux as the UML executable. To
> statically compiled UML use:
> 
>   $ make -C tools/lkl UML_STATIC=y
> 
> Signed-off-by: Octavian Purdila <tavi@cs.pub.ro>
> Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
> ---
>   arch/um/Kconfig                  |  12 +--
>   arch/um/Makefile                 |  14 ++-
>   arch/um/include/asm/common.lds.S |  99 ------------------
>   arch/um/kernel/dyn.lds.S         | 171 -------------------------------
>   arch/um/kernel/uml.lds.S         | 115 ---------------------
>   arch/um/kernel/vmlinux.lds.S     |  91 ++++++++++++++--
>   scripts/link-vmlinux.sh          |  42 +++-----
>   tools/um/Makefile                |  72 +++++++++++++
>   tools/um/Targets                 |   4 +
>   tools/um/uml/Build               |   0
>   10 files changed, 184 insertions(+), 436 deletions(-)
>   delete mode 100644 arch/um/include/asm/common.lds.S
>   delete mode 100644 arch/um/kernel/dyn.lds.S
>   delete mode 100644 arch/um/kernel/uml.lds.S
>   create mode 100644 tools/um/Makefile
>   create mode 100644 tools/um/Targets
>   create mode 100644 tools/um/uml/Build
> 
> diff --git a/arch/um/Kconfig b/arch/um/Kconfig
> index eb51fec75948..e41d31d0d875 100644
> --- a/arch/um/Kconfig
> +++ b/arch/um/Kconfig
> @@ -20,6 +20,7 @@ config UML
>   	select GENERIC_CLOCKEVENTS
>   	select HAVE_GCC_PLUGINS
>   	select TTY # Needed for line.c
> +	select MODULE_REL_CRCS if MODVERSIONS
>   
>   config MMU
>   	bool
> @@ -79,17 +80,6 @@ config STATIC_LINK
>   	  NOTE: This option is incompatible with some networking features which
>   	  depend on features that require being dynamically loaded (like NSS).
>   
> -config LD_SCRIPT_STATIC
> -	bool
> -	default y
> -	depends on STATIC_LINK
> -
> -config LD_SCRIPT_DYN
> -	bool
> -	default y
> -	depends on !LD_SCRIPT_STATIC
> -	select MODULE_REL_CRCS if MODVERSIONS
> -
>   config HOSTFS
>   	tristate "Host filesystem"
>   	help
> diff --git a/arch/um/Makefile b/arch/um/Makefile
> index 1cea46ff9bb7..fdfaff4633e8 100644
> --- a/arch/um/Makefile
> +++ b/arch/um/Makefile
> @@ -95,14 +95,20 @@ KBUILD_CPPFLAGS += -I$(srctree)/$(HOST_DIR)/include \
>   KERNEL_DEFINES = $(strip -Derrno=kernel_errno -Dsigprocmask=kernel_sigprocmask \
>   			 -Dmktime=kernel_mktime $(ARCH_KERNEL_DEFINES))
>   KBUILD_CFLAGS += $(KERNEL_DEFINES)
> +LDFLAGS_vmlinux += -r
>   
> -PHONY += linux
> +PHONY += linux.o
>   
> -all: linux
> +all: linux.o
>   
> -linux: vmlinux
> +linux.o: vmlinux
>   	@echo '  LINK $@'
> -	$(Q)ln -f $< $@
> +	$(Q)$(OBJCOPY) -R .eh_frame $< $@
> +
> +install: linux.o
> +	@echo "  INSTALL $(INSTALL_PATH)/lib/$<"
> +	@mkdir -p $(INSTALL_PATH)/lib/
> +	@cp $< $(INSTALL_PATH)/lib/
>   
>   define archhelp
>     echo '* linux		- Binary kernel image (./linux) - for backward'
> diff --git a/arch/um/include/asm/common.lds.S b/arch/um/include/asm/common.lds.S
> deleted file mode 100644
> index eca6c452a41b..000000000000
> --- a/arch/um/include/asm/common.lds.S
> +++ /dev/null
> @@ -1,99 +0,0 @@
> -/* SPDX-License-Identifier: GPL-2.0 */
> -#include <asm-generic/vmlinux.lds.h>
> -
> -  .fini      : { *(.fini)    } =0x9090
> -  _etext = .;
> -  PROVIDE (etext = .);
> -
> -  . = ALIGN(4096);
> -  _sdata = .;
> -  PROVIDE (sdata = .);
> -
> -  RO_DATA(4096)
> -
> -  .unprotected : { *(.unprotected) }
> -  . = ALIGN(4096);
> -  PROVIDE (_unprotected_end = .);
> -
> -  . = ALIGN(4096);
> -  EXCEPTION_TABLE(0)
> -
> -  BUG_TABLE
> -
> -  .uml.setup.init : {
> -	__uml_setup_start = .;
> -	*(.uml.setup.init)
> -	__uml_setup_end = .;
> -  }
> -	
> -  .uml.help.init : {
> -	__uml_help_start = .;
> -	*(.uml.help.init)
> -	__uml_help_end = .;
> -  }
> -	
> -  .uml.postsetup.init : {
> -	__uml_postsetup_start = .;
> -	*(.uml.postsetup.init)
> -	__uml_postsetup_end = .;
> -  }
> -	
> -  .init.setup : {
> -	INIT_SETUP(0)
> -  }
> -
> -  PERCPU_SECTION(32)
> -	
> -  .initcall.init : {
> -	INIT_CALLS
> -  }
> -
> -  .con_initcall.init : {
> -	CON_INITCALL
> -  }
> -
> -  .exitcall : {
> -	__exitcall_begin = .;
> -	*(.exitcall.exit)
> -	__exitcall_end = .;
> -  }
> -
> -  .uml.exitcall : {
> -	__uml_exitcall_begin = .;
> -	*(.uml.exitcall.exit)
> -	__uml_exitcall_end = .;
> -  }
> -
> -  . = ALIGN(4);
> -  .altinstructions : {
> -	__alt_instructions = .;
> -	*(.altinstructions)
> -	__alt_instructions_end = .;
> -  }
> -  .altinstr_replacement : { *(.altinstr_replacement) }
> -  /* .exit.text is discard at runtime, not link time, to deal with references
> -     from .altinstructions and .eh_frame */
> -  .exit.text : { EXIT_TEXT }
> -  .exit.data : { *(.exit.data) }
> -
> -  .preinit_array : {
> -	__preinit_array_start = .;
> -	*(.preinit_array)
> -	__preinit_array_end = .;
> -  }
> -  .init_array : {
> -	__init_array_start = .;
> -	*(.init_array)
> -	__init_array_end = .;
> -  }
> -  .fini_array : {
> -	__fini_array_start = .;
> -	*(.fini_array)
> -	__fini_array_end = .;
> -  }
> -
> -   . = ALIGN(4096);
> -  .init.ramfs : {
> -	INIT_RAM_FS
> -  }
> -
> diff --git a/arch/um/kernel/dyn.lds.S b/arch/um/kernel/dyn.lds.S
> deleted file mode 100644
> index f5001481010c..000000000000
> --- a/arch/um/kernel/dyn.lds.S
> +++ /dev/null
> @@ -1,171 +0,0 @@
> -#include <asm/vmlinux.lds.h>
> -#include <asm/page.h>
> -
> -OUTPUT_FORMAT(ELF_FORMAT)
> -OUTPUT_ARCH(ELF_ARCH)
> -ENTRY(_start)
> -jiffies = jiffies_64;
> -
> -SECTIONS
> -{
> -  PROVIDE (__executable_start = START);
> -  . = START + SIZEOF_HEADERS;
> -  .interp         : { *(.interp) }
> -  __binary_start = .;
> -  . = ALIGN(4096);		/* Init code and data */
> -  _text = .;
> -  INIT_TEXT_SECTION(PAGE_SIZE)
> -
> -  . = ALIGN(PAGE_SIZE);
> -
> -  /* Read-only sections, merged into text segment: */
> -  .hash           : { *(.hash) }
> -  .gnu.hash       : { *(.gnu.hash) }
> -  .dynsym         : { *(.dynsym) }
> -  .dynstr         : { *(.dynstr) }
> -  .gnu.version    : { *(.gnu.version) }
> -  .gnu.version_d  : { *(.gnu.version_d) }
> -  .gnu.version_r  : { *(.gnu.version_r) }
> -  .rel.init       : { *(.rel.init) }
> -  .rela.init      : { *(.rela.init) }
> -  .rel.text       : { *(.rel.text .rel.text.* .rel.gnu.linkonce.t.*) }
> -  .rela.text      : { *(.rela.text .rela.text.* .rela.gnu.linkonce.t.*) }
> -  .rel.fini       : { *(.rel.fini) }
> -  .rela.fini      : { *(.rela.fini) }
> -  .rel.rodata     : { *(.rel.rodata .rel.rodata.* .rel.gnu.linkonce.r.*) }
> -  .rela.rodata    : { *(.rela.rodata .rela.rodata.* .rela.gnu.linkonce.r.*) }
> -  .rel.data       : { *(.rel.data .rel.data.* .rel.gnu.linkonce.d.*) }
> -  .rela.data      : { *(.rela.data .rela.data.* .rela.gnu.linkonce.d.*) }
> -  .rel.tdata	  : { *(.rel.tdata .rel.tdata.* .rel.gnu.linkonce.td.*) }
> -  .rela.tdata	  : { *(.rela.tdata .rela.tdata.* .rela.gnu.linkonce.td.*) }
> -  .rel.tbss	  : { *(.rel.tbss .rel.tbss.* .rel.gnu.linkonce.tb.*) }
> -  .rela.tbss	  : { *(.rela.tbss .rela.tbss.* .rela.gnu.linkonce.tb.*) }
> -  .rel.ctors      : { *(.rel.ctors) }
> -  .rela.ctors     : { *(.rela.ctors) }
> -  .rel.dtors      : { *(.rel.dtors) }
> -  .rela.dtors     : { *(.rela.dtors) }
> -  .rel.got        : { *(.rel.got) }
> -  .rela.got       : { *(.rela.got) }
> -  .rel.bss        : { *(.rel.bss .rel.bss.* .rel.gnu.linkonce.b.*) }
> -  .rela.bss       : { *(.rela.bss .rela.bss.* .rela.gnu.linkonce.b.*) }
> -  .rel.plt : {
> -	*(.rel.plt)
> -	PROVIDE_HIDDEN(__rel_iplt_start = .);
> -	*(.rel.iplt)
> -	PROVIDE_HIDDEN(__rel_iplt_end = .);
> -  }
> -  .rela.plt : {
> -	*(.rela.plt)
> -	PROVIDE_HIDDEN(__rela_iplt_start = .);
> -	*(.rela.iplt)
> -	PROVIDE_HIDDEN(__rela_iplt_end = .);
> -  }
> -  .init           : {
> -    KEEP (*(.init))
> -  } =0x90909090
> -  .plt            : { *(.plt) }
> -  .text           : {
> -    _stext = .;
> -    TEXT_TEXT
> -    SCHED_TEXT
> -    CPUIDLE_TEXT
> -    LOCK_TEXT
> -    IRQENTRY_TEXT
> -    SOFTIRQENTRY_TEXT
> -    *(.fixup)
> -    *(.stub .text.* .gnu.linkonce.t.*)
> -    /* .gnu.warning sections are handled specially by elf32.em.  */
> -    *(.gnu.warning)
> -
> -    . = ALIGN(PAGE_SIZE);
> -  } =0x90909090
> -  . = ALIGN(PAGE_SIZE);
> -  .syscall_stub : {
> -	__syscall_stub_start = .;
> -	*(.__syscall_stub*)
> -	__syscall_stub_end = .;
> -  }
> -  .fini           : {
> -    KEEP (*(.fini))
> -  } =0x90909090
> -
> -  .kstrtab : { *(.kstrtab) }
> -
> -  #include <asm/common.lds.S>
> -
> -  __init_begin = .;
> -  init.data : { INIT_DATA }
> -  __init_end = .;
> -
> -  /* Ensure the __preinit_array_start label is properly aligned.  We
> -     could instead move the label definition inside the section, but
> -     the linker would then create the section even if it turns out to
> -     be empty, which isn't pretty.  */
> -  . = ALIGN(32 / 8);
> -  .preinit_array     : { *(.preinit_array) }
> -  .init_array     : { *(.init_array) }
> -  .fini_array     : { *(.fini_array) }
> -  .data           : {
> -    INIT_TASK_DATA(KERNEL_STACK_SIZE)
> -    . = ALIGN(KERNEL_STACK_SIZE);
> -    *(.data..init_irqstack)
> -    DATA_DATA
> -    *(.data.* .gnu.linkonce.d.*)
> -    SORT(CONSTRUCTORS)
> -  }
> -  .data1          : { *(.data1) }
> -  .tdata	  : { *(.tdata .tdata.* .gnu.linkonce.td.*) }
> -  .tbss		  : { *(.tbss .tbss.* .gnu.linkonce.tb.*) *(.tcommon) }
> -  .eh_frame       : { KEEP (*(.eh_frame)) }
> -  .gcc_except_table   : { *(.gcc_except_table) }
> -  .dynamic        : { *(.dynamic) }
> -  .ctors          : {
> -    /* gcc uses crtbegin.o to find the start of
> -       the constructors, so we make sure it is
> -       first.  Because this is a wildcard, it
> -       doesn't matter if the user does not
> -       actually link against crtbegin.o; the
> -       linker won't look for a file to match a
> -       wildcard.  The wildcard also means that it
> -       doesn't matter which directory crtbegin.o
> -       is in.  */
> -    KEEP (*crtbegin.o(.ctors))
> -    /* We don't want to include the .ctor section from
> -       from the crtend.o file until after the sorted ctors.
> -       The .ctor section from the crtend file contains the
> -       end of ctors marker and it must be last */
> -    KEEP (*(EXCLUDE_FILE (*crtend.o ) .ctors))
> -    KEEP (*(SORT(.ctors.*)))
> -    KEEP (*(.ctors))
> -  }
> -  .dtors          : {
> -    KEEP (*crtbegin.o(.dtors))
> -    KEEP (*(EXCLUDE_FILE (*crtend.o ) .dtors))
> -    KEEP (*(SORT(.dtors.*)))
> -    KEEP (*(.dtors))
> -  }
> -  .jcr            : { KEEP (*(.jcr)) }
> -  .got            : { *(.got.plt) *(.got) }
> -  _edata = .;
> -  PROVIDE (edata = .);
> -  .bss            : {
> -   __bss_start = .;
> -   *(.dynbss)
> -   *(.bss .bss.* .gnu.linkonce.b.*)
> -   *(COMMON)
> -   /* Align here to ensure that the .bss section occupies space up to
> -      _end.  Align after .bss to ensure correct alignment even if the
> -      .bss section disappears because there are no input sections.  */
> -   . = ALIGN(32 / 8);
> -  . = ALIGN(32 / 8);
> -  }
> -   __bss_stop = .;
> -  _end = .;
> -  PROVIDE (end = .);
> -
> -  STABS_DEBUG
> -
> -  DWARF_DEBUG
> -
> -  DISCARDS
> -}
> diff --git a/arch/um/kernel/uml.lds.S b/arch/um/kernel/uml.lds.S
> deleted file mode 100644
> index 3b6dab3d4501..000000000000
> --- a/arch/um/kernel/uml.lds.S
> +++ /dev/null
> @@ -1,115 +0,0 @@
> -/* SPDX-License-Identifier: GPL-2.0 */
> -#include <asm/vmlinux.lds.h>
> -#include <asm/page.h>
> -
> -OUTPUT_FORMAT(ELF_FORMAT)
> -OUTPUT_ARCH(ELF_ARCH)
> -ENTRY(_start)
> -jiffies = jiffies_64;
> -
> -SECTIONS
> -{
> -  /* This must contain the right address - not quite the default ELF one.*/
> -  PROVIDE (__executable_start = START);
> -  /* Static binaries stick stuff here, like the sigreturn trampoline,
> -   * invisibly to objdump.  So, just make __binary_start equal to the very
> -   * beginning of the executable, and if there are unmapped pages after this,
> -   * they are forever unusable.
> -   */
> -  __binary_start = START;
> -
> -  . = START + SIZEOF_HEADERS;
> -  . = ALIGN(PAGE_SIZE);
> -
> -  _text = .;
> -  INIT_TEXT_SECTION(0)
> -
> -  .text      :
> -  {
> -    _stext = .;
> -    TEXT_TEXT
> -    SCHED_TEXT
> -    CPUIDLE_TEXT
> -    LOCK_TEXT
> -    IRQENTRY_TEXT
> -    SOFTIRQENTRY_TEXT
> -    *(.fixup)
> -    /* .gnu.warning sections are handled specially by elf32.em.  */
> -    *(.gnu.warning)
> -    *(.gnu.linkonce.t*)
> -  }
> -
> -  . = ALIGN(PAGE_SIZE);
> -  .syscall_stub : {
> -	__syscall_stub_start = .;
> -	*(.__syscall_stub*)
> -	__syscall_stub_end = .;
> -  }
> -
> -  /*
> -   * These are needed even in a static link, even if they wind up being empty.
> -   * Newer glibc needs these __rel{,a}_iplt_{start,end} symbols.
> -   */
> -  .rel.plt : {
> -	*(.rel.plt)
> -	PROVIDE_HIDDEN(__rel_iplt_start = .);
> -	*(.rel.iplt)
> -	PROVIDE_HIDDEN(__rel_iplt_end = .);
> -  }
> -  .rela.plt : {
> -	*(.rela.plt)
> -	PROVIDE_HIDDEN(__rela_iplt_start = .);
> -	*(.rela.iplt)
> -	PROVIDE_HIDDEN(__rela_iplt_end = .);
> -  }
> -
> -  #include <asm/common.lds.S>
> -
> -  __init_begin = .;
> -  init.data : { INIT_DATA }
> -  __init_end = .;
> -
> -  .data    :
> -  {
> -    INIT_TASK_DATA(KERNEL_STACK_SIZE)
> -    . = ALIGN(KERNEL_STACK_SIZE);
> -    *(.data..init_irqstack)
> -    DATA_DATA
> -    *(.gnu.linkonce.d*)
> -    CONSTRUCTORS
> -  }
> -  .data1   : { *(.data1) }
> -  .ctors         :
> -  {
> -    *(.ctors)
> -  }
> -  .dtors         :
> -  {
> -    *(.dtors)
> -  }
> -
> -  .got           : { *(.got.plt) *(.got) }
> -  .dynamic       : { *(.dynamic) }
> -  .tdata	  : { *(.tdata .tdata.* .gnu.linkonce.td.*) }
> -  .tbss		  : { *(.tbss .tbss.* .gnu.linkonce.tb.*) *(.tcommon) }
> -  /* We want the small data sections together, so single-instruction offsets
> -     can access them all, and initialized data all before uninitialized, so
> -     we can shorten the on-disk segment size.  */
> -  .sdata     : { *(.sdata) }
> -  _edata  =  .;
> -  PROVIDE (edata = .);
> -  . = ALIGN(PAGE_SIZE);
> -  __bss_start = .;
> -  PROVIDE(_bss_start = .);
> -  SBSS(0)
> -  BSS(0)
> -   __bss_stop = .;
> -  _end = .;
> -  PROVIDE (end = .);
> -
> -  STABS_DEBUG
> -
> -  DWARF_DEBUG
> -
> -  DISCARDS
> -}
> diff --git a/arch/um/kernel/vmlinux.lds.S b/arch/um/kernel/vmlinux.lds.S
> index 16e49bfa2b42..aaedd66772fa 100644
> --- a/arch/um/kernel/vmlinux.lds.S
> +++ b/arch/um/kernel/vmlinux.lds.S
> @@ -1,8 +1,87 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#include <asm/vmlinux.lds.h>
> +#include <asm/thread_info.h>
> +#include <asm/page.h>
> +#include <asm/thread_info.h>
> +#include <asm/cache.h>
> +#include <linux/export.h>
>   
> -KERNEL_STACK_SIZE = 4096 * (1 << CONFIG_KERNEL_STACK_ORDER);
> +OUTPUT_FORMAT(ELF_FORMAT)
> +jiffies = jiffies_64;
>   
> -#ifdef CONFIG_LD_SCRIPT_STATIC
> -#include "uml.lds.S"
> -#else
> -#include "dyn.lds.S"
> -#endif
> +
> +SECTIONS
> +{
> +	__init_begin = . ;
> +	HEAD_TEXT_SECTION
> +	INIT_TEXT_SECTION(PAGE_SIZE)
> +	INIT_DATA_SECTION(16)
> +	PERCPU_SECTION(L1_CACHE_BYTES)
> +	__init_end = . ;
> +
> +	_stext = . ;
> +	_text = . ;
> +	.text : ALIGN(THREAD_SIZE)
> +	{
> +		__binary_start = . ;
> +		TEXT_TEXT
> +		SCHED_TEXT
> +		LOCK_TEXT
> +		CPUIDLE_TEXT
> +		IRQENTRY_TEXT
> +		SOFTIRQENTRY_TEXT
> +	}
> +	_etext = . ;
> +
> +	.syscall_stub : ALIGN(PAGE_SIZE)
> +	{
> +		__syscall_stub_start = .;
> +		*(.__syscall_stub*)
> +		__syscall_stub_end = .;
> +	}
> +
> +	_sdata = . ;
> +	RO_DATA(PAGE_SIZE)
> +	RW_DATA(L1_CACHE_BYTES, PAGE_SIZE, THREAD_SIZE)
> +	.data : ALIGN(THREAD_SIZE)  { }
> +
> +	EXCEPTION_TABLE(16)
> +
> +	.uml.init_irqstack : ALIGN(THREAD_SIZE) {
> +		*(.data..init_irqstack)
> +	}
> +
> +	.uml.setup.init : ALIGN(8) {
> +		__uml_setup_start = .;
> +		*(.uml.setup.init)
> +		__uml_setup_end = .;
> +	}
> +
> +	.uml.help.init : ALIGN(8)  {
> +		__uml_help_start = .;
> +		*(.uml.help.init)
> +		__uml_help_end = .;
> +	}
> +
> +	.uml.postsetup.init : ALIGN(8) {
> +		__uml_postsetup_start = .;
> +		*(.uml.postsetup.init)
> +		__uml_postsetup_end = .;
> +	}
> +
> +	.uml.exitcall : ALIGN(8) {
> +		__uml_exitcall_begin = .;
> +		*(.uml.exitcall.exit)
> +		__uml_exitcall_end = .;
> +	}
> +	_edata = . ;
> +
> +	BSS_SECTION(0, 0, 0)
> +	_end = . ;
> +
> +	STABS_DEBUG
> +
> +	DWARF_DEBUG
> +
> +	DISCARDS
> +}
> diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
> index e6e2d9e5ff48..903fc8053d07 100755
> --- a/scripts/link-vmlinux.sh
> +++ b/scripts/link-vmlinux.sh
> @@ -102,36 +102,18 @@ vmlinux_link()
>   		strip_debug=-Wl,--strip-debug
>   	fi
>   
> -	if [ "${SRCARCH}" != "um" ]; then
> -		objects="--whole-archive			\
> -			${KBUILD_VMLINUX_OBJS}			\
> -			--no-whole-archive			\
> -			--start-group				\
> -			${KBUILD_VMLINUX_LIBS}			\
> -			--end-group				\
> -			${@}"
> -
> -		${LD} ${KBUILD_LDFLAGS} ${LDFLAGS_vmlinux}	\
> -			${strip_debug#-Wl,}			\
> -			-o ${output}				\
> -			-T ${lds} ${objects}
> -	else
> -		objects="-Wl,--whole-archive			\
> -			${KBUILD_VMLINUX_OBJS}			\
> -			-Wl,--no-whole-archive			\
> -			-Wl,--start-group			\
> -			${KBUILD_VMLINUX_LIBS}			\
> -			-Wl,--end-group				\
> -			${@}"
> -
> -		${CC} ${CFLAGS_vmlinux}				\
> -			${strip_debug}				\
> -			-o ${output}				\
> -			-Wl,-T,${lds}				\
> -			${objects}				\
> -			-lutil -lrt -lpthread
> -		rm -f linux
> -	fi
> +	objects="--whole-archive			\
> +		${KBUILD_VMLINUX_OBJS}			\
> +		--no-whole-archive			\
> +		--start-group				\
> +		${KBUILD_VMLINUX_LIBS}			\
> +		--end-group				\
> +		${@}"
> +
> +	${LD} ${KBUILD_LDFLAGS} ${LDFLAGS_vmlinux}	\
> +		${strip_debug#-Wl,}			\
> +		-o ${output}				\
> +		-T ${lds} ${objects}
>   }
>   
>   # generate .BTF typeinfo from DWARF debuginfo
> diff --git a/tools/um/Makefile b/tools/um/Makefile
> new file mode 100644
> index 000000000000..a63466ef7ef5
> --- /dev/null
> +++ b/tools/um/Makefile
> @@ -0,0 +1,72 @@
> +# Do not use make's built-in rules
> +# (this improves performance and avoids hard-to-debug behaviour);
> +# also do not print "Entering directory..." messages from make
> +.SUFFIXES:
> +MAKEFLAGS += -r --no-print-directory
> +
> +KCONFIG?=defconfig
> +
> +ifneq ($(silent),1)
> +  ifneq ($(V),1)
> +	QUIET_AUTOCONF       = @echo '  AUTOCONF '$@;
> +	Q = @
> +  endif
> +endif
> +
> +PREFIX   := /usr
> +
> +ifeq (,$(srctree))
> +  srctree := $(patsubst %/,%,$(dir $(shell pwd)))
> +  srctree := $(patsubst %/,%,$(dir $(srctree)))
> +endif
> +export srctree
> +
> +-include ../scripts/Makefile.include
> +
> +# OUTPUT fixup should be *after* include ../scripts/Makefile.include
> +ifneq ($(OUTPUT),)
> +  OUTPUT := $(OUTPUT)/tools/um/
> +else
> +  OUTPUT := $(CURDIR)/
> +endif
> +export OUTPUT
> +export objtree := $(OUTPUT)/../..
> +
> +
> +all:
> +
> +export CFLAGS += -I$(OUTPUT)/include -Iinclude -Wall -g -O2 -Wextra -fPIC \
> +	 -Wno-unused-parameter \
> +	 -Wno-missing-field-initializers -fno-strict-aliasing
> +
> +-include Targets
> +
> +TARGETS := $(progs-y:%=$(OUTPUT)%)
> +TARGETS += $(libs-y:%=$(OUTPUT)%)
> +all: $(TARGETS)
> +
> +# rule to build linux.o
> +$(OUTPUT)lib/linux.o:
> +	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) $(KCONFIG)
> +	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) install INSTALL_PATH=$(OUTPUT)
> +
> +$(OUTPUT)liblinux.a: $(OUTPUT)lib/linux.o $(OUTPUT)uml/liblinux-in.o
> +	$(QUIET_AR)$(AR) -rc $@ $^
> +
> +# rule to link programs
> +$(OUTPUT)%: $(OUTPUT)%-in.o $(OUTPUT)liblinux.a
> +	$(QUIET_LINK)$(CC) $(LDFLAGS) $(LDFLAGS_$(notdir $*)-y) -o $@ $^ $(LDLIBS) $(LDLIBS_$(notdir $*)-y)
> +
> +$(OUTPUT)%-in.o: FORCE
> +	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
> +
> +clean:
> +	$(call QUIET_CLEAN, objects)find $(OUTPUT) -name '*.o' -delete -o -name '\.*.cmd'\
> +	 -delete -o -name '\.*.d' -delete
> +	$(call QUIET_CLEAN, liblinux.a)$(RM) $(OUTPUT)/liblinux.a
> +	$(call QUIET_CLEAN, $(TARGETS))$(RM) $(TARGETS)
> +
> +FORCE: ;
> +.PHONY: all clean FORCE
> +.IGNORE: clean
> +.NOTPARALLEL : lib/linux.o
> diff --git a/tools/um/Targets b/tools/um/Targets
> new file mode 100644
> index 000000000000..a4711f1ef422
> --- /dev/null
> +++ b/tools/um/Targets
> @@ -0,0 +1,4 @@
> +progs-y += uml/linux
> +LDLIBS_linux-y := -lrt -lpthread -lutil
> +LDFLAGS_linux-y := -no-pie -Wl,--wrap,malloc -Wl,--wrap,free -Wl,--wrap,calloc
> +LDFLAGS_linux-$(UML_STATIC) += -static
> diff --git a/tools/um/uml/Build b/tools/um/uml/Build
> new file mode 100644
> index 000000000000..e69de29bb2d1
> 

-- 
Anton R. Ivanov
Cambridgegreys Limited. Registered in England. Company Number 10273661
https://www.cambridgegreys.com/

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v6 01/21] um: split build in kernel and host parts
@ 2020-09-24  7:36           ` Anton Ivanov
  0 siblings, 0 replies; 476+ messages in thread
From: Anton Ivanov @ 2020-09-24  7:36 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Octavian Purdila,
	retrage01



On 24/09/2020 08:12, Hajime Tazaki wrote:
> From: Octavian Purdila <tavi@cs.pub.ro>
> 
> This patch splits the UML build in two parts: a kernel part that
> generates a relocatable object (linux.o) and a host build part that
> links the relocatable object into an executable.
> 
> This allows us to simplify the linker script since we can now remove
> the host bits (e.g. .rel sections). It also gives us greater
> flexibility since it will be much easier to support other
> architectures and OSes if we use the standard linker script.
> 
> The host build part has been implemented in tools/um so that we can
> reuse the available host build infrastructure.
> 
> The patch also changes the UML build invocation, if before
> 
>   $ make ARCH=um defconfig
>   $ make ARCH=um
> 
> was generating the executable now this only generates the relocatable
> object.

It also fails now.


ERROR: modpost: "memmove" [drivers/net/slip/slhc.ko] undefined!
ERROR: modpost: "memset" [drivers/net/ppp/ppp_generic.ko] undefined!
ERROR: modpost: "memset" [drivers/net/tun.ko] undefined!
ERROR: modpost: "memmove" [drivers/block/loop.ko] undefined!
ERROR: modpost: "memset" [drivers/block/loop.ko] undefined!
ERROR: modpost: "memset" [fs/autofs/autofs4.ko] undefined!
ERROR: modpost: "memset" [fs/isofs/isofs.ko] undefined!
ERROR: modpost: "memset" [fs/binfmt_misc.ko] undefined!
make[1]: *** [scripts/Makefile.modpost:111: Module.symvers] Error 1
make[1]: *** Deleting file 'Module.symvers'
make: *** [Makefile:1388: modules] Error 2
make: *** Waiting for unfinished jobs....


> 
> To fully build UML use:
> 
>   $ make -C tools/um
> 
> which will generate tools/lkl/linux as the UML executable. To
> statically compiled UML use:
> 
>   $ make -C tools/lkl UML_STATIC=y
> 
> Signed-off-by: Octavian Purdila <tavi@cs.pub.ro>
> Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
> ---
>   arch/um/Kconfig                  |  12 +--
>   arch/um/Makefile                 |  14 ++-
>   arch/um/include/asm/common.lds.S |  99 ------------------
>   arch/um/kernel/dyn.lds.S         | 171 -------------------------------
>   arch/um/kernel/uml.lds.S         | 115 ---------------------
>   arch/um/kernel/vmlinux.lds.S     |  91 ++++++++++++++--
>   scripts/link-vmlinux.sh          |  42 +++-----
>   tools/um/Makefile                |  72 +++++++++++++
>   tools/um/Targets                 |   4 +
>   tools/um/uml/Build               |   0
>   10 files changed, 184 insertions(+), 436 deletions(-)
>   delete mode 100644 arch/um/include/asm/common.lds.S
>   delete mode 100644 arch/um/kernel/dyn.lds.S
>   delete mode 100644 arch/um/kernel/uml.lds.S
>   create mode 100644 tools/um/Makefile
>   create mode 100644 tools/um/Targets
>   create mode 100644 tools/um/uml/Build
> 
> diff --git a/arch/um/Kconfig b/arch/um/Kconfig
> index eb51fec75948..e41d31d0d875 100644
> --- a/arch/um/Kconfig
> +++ b/arch/um/Kconfig
> @@ -20,6 +20,7 @@ config UML
>   	select GENERIC_CLOCKEVENTS
>   	select HAVE_GCC_PLUGINS
>   	select TTY # Needed for line.c
> +	select MODULE_REL_CRCS if MODVERSIONS
>   
>   config MMU
>   	bool
> @@ -79,17 +80,6 @@ config STATIC_LINK
>   	  NOTE: This option is incompatible with some networking features which
>   	  depend on features that require being dynamically loaded (like NSS).
>   
> -config LD_SCRIPT_STATIC
> -	bool
> -	default y
> -	depends on STATIC_LINK
> -
> -config LD_SCRIPT_DYN
> -	bool
> -	default y
> -	depends on !LD_SCRIPT_STATIC
> -	select MODULE_REL_CRCS if MODVERSIONS
> -
>   config HOSTFS
>   	tristate "Host filesystem"
>   	help
> diff --git a/arch/um/Makefile b/arch/um/Makefile
> index 1cea46ff9bb7..fdfaff4633e8 100644
> --- a/arch/um/Makefile
> +++ b/arch/um/Makefile
> @@ -95,14 +95,20 @@ KBUILD_CPPFLAGS += -I$(srctree)/$(HOST_DIR)/include \
>   KERNEL_DEFINES = $(strip -Derrno=kernel_errno -Dsigprocmask=kernel_sigprocmask \
>   			 -Dmktime=kernel_mktime $(ARCH_KERNEL_DEFINES))
>   KBUILD_CFLAGS += $(KERNEL_DEFINES)
> +LDFLAGS_vmlinux += -r
>   
> -PHONY += linux
> +PHONY += linux.o
>   
> -all: linux
> +all: linux.o
>   
> -linux: vmlinux
> +linux.o: vmlinux
>   	@echo '  LINK $@'
> -	$(Q)ln -f $< $@
> +	$(Q)$(OBJCOPY) -R .eh_frame $< $@
> +
> +install: linux.o
> +	@echo "  INSTALL $(INSTALL_PATH)/lib/$<"
> +	@mkdir -p $(INSTALL_PATH)/lib/
> +	@cp $< $(INSTALL_PATH)/lib/
>   
>   define archhelp
>     echo '* linux		- Binary kernel image (./linux) - for backward'
> diff --git a/arch/um/include/asm/common.lds.S b/arch/um/include/asm/common.lds.S
> deleted file mode 100644
> index eca6c452a41b..000000000000
> --- a/arch/um/include/asm/common.lds.S
> +++ /dev/null
> @@ -1,99 +0,0 @@
> -/* SPDX-License-Identifier: GPL-2.0 */
> -#include <asm-generic/vmlinux.lds.h>
> -
> -  .fini      : { *(.fini)    } =0x9090
> -  _etext = .;
> -  PROVIDE (etext = .);
> -
> -  . = ALIGN(4096);
> -  _sdata = .;
> -  PROVIDE (sdata = .);
> -
> -  RO_DATA(4096)
> -
> -  .unprotected : { *(.unprotected) }
> -  . = ALIGN(4096);
> -  PROVIDE (_unprotected_end = .);
> -
> -  . = ALIGN(4096);
> -  EXCEPTION_TABLE(0)
> -
> -  BUG_TABLE
> -
> -  .uml.setup.init : {
> -	__uml_setup_start = .;
> -	*(.uml.setup.init)
> -	__uml_setup_end = .;
> -  }
> -	
> -  .uml.help.init : {
> -	__uml_help_start = .;
> -	*(.uml.help.init)
> -	__uml_help_end = .;
> -  }
> -	
> -  .uml.postsetup.init : {
> -	__uml_postsetup_start = .;
> -	*(.uml.postsetup.init)
> -	__uml_postsetup_end = .;
> -  }
> -	
> -  .init.setup : {
> -	INIT_SETUP(0)
> -  }
> -
> -  PERCPU_SECTION(32)
> -	
> -  .initcall.init : {
> -	INIT_CALLS
> -  }
> -
> -  .con_initcall.init : {
> -	CON_INITCALL
> -  }
> -
> -  .exitcall : {
> -	__exitcall_begin = .;
> -	*(.exitcall.exit)
> -	__exitcall_end = .;
> -  }
> -
> -  .uml.exitcall : {
> -	__uml_exitcall_begin = .;
> -	*(.uml.exitcall.exit)
> -	__uml_exitcall_end = .;
> -  }
> -
> -  . = ALIGN(4);
> -  .altinstructions : {
> -	__alt_instructions = .;
> -	*(.altinstructions)
> -	__alt_instructions_end = .;
> -  }
> -  .altinstr_replacement : { *(.altinstr_replacement) }
> -  /* .exit.text is discard at runtime, not link time, to deal with references
> -     from .altinstructions and .eh_frame */
> -  .exit.text : { EXIT_TEXT }
> -  .exit.data : { *(.exit.data) }
> -
> -  .preinit_array : {
> -	__preinit_array_start = .;
> -	*(.preinit_array)
> -	__preinit_array_end = .;
> -  }
> -  .init_array : {
> -	__init_array_start = .;
> -	*(.init_array)
> -	__init_array_end = .;
> -  }
> -  .fini_array : {
> -	__fini_array_start = .;
> -	*(.fini_array)
> -	__fini_array_end = .;
> -  }
> -
> -   . = ALIGN(4096);
> -  .init.ramfs : {
> -	INIT_RAM_FS
> -  }
> -
> diff --git a/arch/um/kernel/dyn.lds.S b/arch/um/kernel/dyn.lds.S
> deleted file mode 100644
> index f5001481010c..000000000000
> --- a/arch/um/kernel/dyn.lds.S
> +++ /dev/null
> @@ -1,171 +0,0 @@
> -#include <asm/vmlinux.lds.h>
> -#include <asm/page.h>
> -
> -OUTPUT_FORMAT(ELF_FORMAT)
> -OUTPUT_ARCH(ELF_ARCH)
> -ENTRY(_start)
> -jiffies = jiffies_64;
> -
> -SECTIONS
> -{
> -  PROVIDE (__executable_start = START);
> -  . = START + SIZEOF_HEADERS;
> -  .interp         : { *(.interp) }
> -  __binary_start = .;
> -  . = ALIGN(4096);		/* Init code and data */
> -  _text = .;
> -  INIT_TEXT_SECTION(PAGE_SIZE)
> -
> -  . = ALIGN(PAGE_SIZE);
> -
> -  /* Read-only sections, merged into text segment: */
> -  .hash           : { *(.hash) }
> -  .gnu.hash       : { *(.gnu.hash) }
> -  .dynsym         : { *(.dynsym) }
> -  .dynstr         : { *(.dynstr) }
> -  .gnu.version    : { *(.gnu.version) }
> -  .gnu.version_d  : { *(.gnu.version_d) }
> -  .gnu.version_r  : { *(.gnu.version_r) }
> -  .rel.init       : { *(.rel.init) }
> -  .rela.init      : { *(.rela.init) }
> -  .rel.text       : { *(.rel.text .rel.text.* .rel.gnu.linkonce.t.*) }
> -  .rela.text      : { *(.rela.text .rela.text.* .rela.gnu.linkonce.t.*) }
> -  .rel.fini       : { *(.rel.fini) }
> -  .rela.fini      : { *(.rela.fini) }
> -  .rel.rodata     : { *(.rel.rodata .rel.rodata.* .rel.gnu.linkonce.r.*) }
> -  .rela.rodata    : { *(.rela.rodata .rela.rodata.* .rela.gnu.linkonce.r.*) }
> -  .rel.data       : { *(.rel.data .rel.data.* .rel.gnu.linkonce.d.*) }
> -  .rela.data      : { *(.rela.data .rela.data.* .rela.gnu.linkonce.d.*) }
> -  .rel.tdata	  : { *(.rel.tdata .rel.tdata.* .rel.gnu.linkonce.td.*) }
> -  .rela.tdata	  : { *(.rela.tdata .rela.tdata.* .rela.gnu.linkonce.td.*) }
> -  .rel.tbss	  : { *(.rel.tbss .rel.tbss.* .rel.gnu.linkonce.tb.*) }
> -  .rela.tbss	  : { *(.rela.tbss .rela.tbss.* .rela.gnu.linkonce.tb.*) }
> -  .rel.ctors      : { *(.rel.ctors) }
> -  .rela.ctors     : { *(.rela.ctors) }
> -  .rel.dtors      : { *(.rel.dtors) }
> -  .rela.dtors     : { *(.rela.dtors) }
> -  .rel.got        : { *(.rel.got) }
> -  .rela.got       : { *(.rela.got) }
> -  .rel.bss        : { *(.rel.bss .rel.bss.* .rel.gnu.linkonce.b.*) }
> -  .rela.bss       : { *(.rela.bss .rela.bss.* .rela.gnu.linkonce.b.*) }
> -  .rel.plt : {
> -	*(.rel.plt)
> -	PROVIDE_HIDDEN(__rel_iplt_start = .);
> -	*(.rel.iplt)
> -	PROVIDE_HIDDEN(__rel_iplt_end = .);
> -  }
> -  .rela.plt : {
> -	*(.rela.plt)
> -	PROVIDE_HIDDEN(__rela_iplt_start = .);
> -	*(.rela.iplt)
> -	PROVIDE_HIDDEN(__rela_iplt_end = .);
> -  }
> -  .init           : {
> -    KEEP (*(.init))
> -  } =0x90909090
> -  .plt            : { *(.plt) }
> -  .text           : {
> -    _stext = .;
> -    TEXT_TEXT
> -    SCHED_TEXT
> -    CPUIDLE_TEXT
> -    LOCK_TEXT
> -    IRQENTRY_TEXT
> -    SOFTIRQENTRY_TEXT
> -    *(.fixup)
> -    *(.stub .text.* .gnu.linkonce.t.*)
> -    /* .gnu.warning sections are handled specially by elf32.em.  */
> -    *(.gnu.warning)
> -
> -    . = ALIGN(PAGE_SIZE);
> -  } =0x90909090
> -  . = ALIGN(PAGE_SIZE);
> -  .syscall_stub : {
> -	__syscall_stub_start = .;
> -	*(.__syscall_stub*)
> -	__syscall_stub_end = .;
> -  }
> -  .fini           : {
> -    KEEP (*(.fini))
> -  } =0x90909090
> -
> -  .kstrtab : { *(.kstrtab) }
> -
> -  #include <asm/common.lds.S>
> -
> -  __init_begin = .;
> -  init.data : { INIT_DATA }
> -  __init_end = .;
> -
> -  /* Ensure the __preinit_array_start label is properly aligned.  We
> -     could instead move the label definition inside the section, but
> -     the linker would then create the section even if it turns out to
> -     be empty, which isn't pretty.  */
> -  . = ALIGN(32 / 8);
> -  .preinit_array     : { *(.preinit_array) }
> -  .init_array     : { *(.init_array) }
> -  .fini_array     : { *(.fini_array) }
> -  .data           : {
> -    INIT_TASK_DATA(KERNEL_STACK_SIZE)
> -    . = ALIGN(KERNEL_STACK_SIZE);
> -    *(.data..init_irqstack)
> -    DATA_DATA
> -    *(.data.* .gnu.linkonce.d.*)
> -    SORT(CONSTRUCTORS)
> -  }
> -  .data1          : { *(.data1) }
> -  .tdata	  : { *(.tdata .tdata.* .gnu.linkonce.td.*) }
> -  .tbss		  : { *(.tbss .tbss.* .gnu.linkonce.tb.*) *(.tcommon) }
> -  .eh_frame       : { KEEP (*(.eh_frame)) }
> -  .gcc_except_table   : { *(.gcc_except_table) }
> -  .dynamic        : { *(.dynamic) }
> -  .ctors          : {
> -    /* gcc uses crtbegin.o to find the start of
> -       the constructors, so we make sure it is
> -       first.  Because this is a wildcard, it
> -       doesn't matter if the user does not
> -       actually link against crtbegin.o; the
> -       linker won't look for a file to match a
> -       wildcard.  The wildcard also means that it
> -       doesn't matter which directory crtbegin.o
> -       is in.  */
> -    KEEP (*crtbegin.o(.ctors))
> -    /* We don't want to include the .ctor section from
> -       from the crtend.o file until after the sorted ctors.
> -       The .ctor section from the crtend file contains the
> -       end of ctors marker and it must be last */
> -    KEEP (*(EXCLUDE_FILE (*crtend.o ) .ctors))
> -    KEEP (*(SORT(.ctors.*)))
> -    KEEP (*(.ctors))
> -  }
> -  .dtors          : {
> -    KEEP (*crtbegin.o(.dtors))
> -    KEEP (*(EXCLUDE_FILE (*crtend.o ) .dtors))
> -    KEEP (*(SORT(.dtors.*)))
> -    KEEP (*(.dtors))
> -  }
> -  .jcr            : { KEEP (*(.jcr)) }
> -  .got            : { *(.got.plt) *(.got) }
> -  _edata = .;
> -  PROVIDE (edata = .);
> -  .bss            : {
> -   __bss_start = .;
> -   *(.dynbss)
> -   *(.bss .bss.* .gnu.linkonce.b.*)
> -   *(COMMON)
> -   /* Align here to ensure that the .bss section occupies space up to
> -      _end.  Align after .bss to ensure correct alignment even if the
> -      .bss section disappears because there are no input sections.  */
> -   . = ALIGN(32 / 8);
> -  . = ALIGN(32 / 8);
> -  }
> -   __bss_stop = .;
> -  _end = .;
> -  PROVIDE (end = .);
> -
> -  STABS_DEBUG
> -
> -  DWARF_DEBUG
> -
> -  DISCARDS
> -}
> diff --git a/arch/um/kernel/uml.lds.S b/arch/um/kernel/uml.lds.S
> deleted file mode 100644
> index 3b6dab3d4501..000000000000
> --- a/arch/um/kernel/uml.lds.S
> +++ /dev/null
> @@ -1,115 +0,0 @@
> -/* SPDX-License-Identifier: GPL-2.0 */
> -#include <asm/vmlinux.lds.h>
> -#include <asm/page.h>
> -
> -OUTPUT_FORMAT(ELF_FORMAT)
> -OUTPUT_ARCH(ELF_ARCH)
> -ENTRY(_start)
> -jiffies = jiffies_64;
> -
> -SECTIONS
> -{
> -  /* This must contain the right address - not quite the default ELF one.*/
> -  PROVIDE (__executable_start = START);
> -  /* Static binaries stick stuff here, like the sigreturn trampoline,
> -   * invisibly to objdump.  So, just make __binary_start equal to the very
> -   * beginning of the executable, and if there are unmapped pages after this,
> -   * they are forever unusable.
> -   */
> -  __binary_start = START;
> -
> -  . = START + SIZEOF_HEADERS;
> -  . = ALIGN(PAGE_SIZE);
> -
> -  _text = .;
> -  INIT_TEXT_SECTION(0)
> -
> -  .text      :
> -  {
> -    _stext = .;
> -    TEXT_TEXT
> -    SCHED_TEXT
> -    CPUIDLE_TEXT
> -    LOCK_TEXT
> -    IRQENTRY_TEXT
> -    SOFTIRQENTRY_TEXT
> -    *(.fixup)
> -    /* .gnu.warning sections are handled specially by elf32.em.  */
> -    *(.gnu.warning)
> -    *(.gnu.linkonce.t*)
> -  }
> -
> -  . = ALIGN(PAGE_SIZE);
> -  .syscall_stub : {
> -	__syscall_stub_start = .;
> -	*(.__syscall_stub*)
> -	__syscall_stub_end = .;
> -  }
> -
> -  /*
> -   * These are needed even in a static link, even if they wind up being empty.
> -   * Newer glibc needs these __rel{,a}_iplt_{start,end} symbols.
> -   */
> -  .rel.plt : {
> -	*(.rel.plt)
> -	PROVIDE_HIDDEN(__rel_iplt_start = .);
> -	*(.rel.iplt)
> -	PROVIDE_HIDDEN(__rel_iplt_end = .);
> -  }
> -  .rela.plt : {
> -	*(.rela.plt)
> -	PROVIDE_HIDDEN(__rela_iplt_start = .);
> -	*(.rela.iplt)
> -	PROVIDE_HIDDEN(__rela_iplt_end = .);
> -  }
> -
> -  #include <asm/common.lds.S>
> -
> -  __init_begin = .;
> -  init.data : { INIT_DATA }
> -  __init_end = .;
> -
> -  .data    :
> -  {
> -    INIT_TASK_DATA(KERNEL_STACK_SIZE)
> -    . = ALIGN(KERNEL_STACK_SIZE);
> -    *(.data..init_irqstack)
> -    DATA_DATA
> -    *(.gnu.linkonce.d*)
> -    CONSTRUCTORS
> -  }
> -  .data1   : { *(.data1) }
> -  .ctors         :
> -  {
> -    *(.ctors)
> -  }
> -  .dtors         :
> -  {
> -    *(.dtors)
> -  }
> -
> -  .got           : { *(.got.plt) *(.got) }
> -  .dynamic       : { *(.dynamic) }
> -  .tdata	  : { *(.tdata .tdata.* .gnu.linkonce.td.*) }
> -  .tbss		  : { *(.tbss .tbss.* .gnu.linkonce.tb.*) *(.tcommon) }
> -  /* We want the small data sections together, so single-instruction offsets
> -     can access them all, and initialized data all before uninitialized, so
> -     we can shorten the on-disk segment size.  */
> -  .sdata     : { *(.sdata) }
> -  _edata  =  .;
> -  PROVIDE (edata = .);
> -  . = ALIGN(PAGE_SIZE);
> -  __bss_start = .;
> -  PROVIDE(_bss_start = .);
> -  SBSS(0)
> -  BSS(0)
> -   __bss_stop = .;
> -  _end = .;
> -  PROVIDE (end = .);
> -
> -  STABS_DEBUG
> -
> -  DWARF_DEBUG
> -
> -  DISCARDS
> -}
> diff --git a/arch/um/kernel/vmlinux.lds.S b/arch/um/kernel/vmlinux.lds.S
> index 16e49bfa2b42..aaedd66772fa 100644
> --- a/arch/um/kernel/vmlinux.lds.S
> +++ b/arch/um/kernel/vmlinux.lds.S
> @@ -1,8 +1,87 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#include <asm/vmlinux.lds.h>
> +#include <asm/thread_info.h>
> +#include <asm/page.h>
> +#include <asm/thread_info.h>
> +#include <asm/cache.h>
> +#include <linux/export.h>
>   
> -KERNEL_STACK_SIZE = 4096 * (1 << CONFIG_KERNEL_STACK_ORDER);
> +OUTPUT_FORMAT(ELF_FORMAT)
> +jiffies = jiffies_64;
>   
> -#ifdef CONFIG_LD_SCRIPT_STATIC
> -#include "uml.lds.S"
> -#else
> -#include "dyn.lds.S"
> -#endif
> +
> +SECTIONS
> +{
> +	__init_begin = . ;
> +	HEAD_TEXT_SECTION
> +	INIT_TEXT_SECTION(PAGE_SIZE)
> +	INIT_DATA_SECTION(16)
> +	PERCPU_SECTION(L1_CACHE_BYTES)
> +	__init_end = . ;
> +
> +	_stext = . ;
> +	_text = . ;
> +	.text : ALIGN(THREAD_SIZE)
> +	{
> +		__binary_start = . ;
> +		TEXT_TEXT
> +		SCHED_TEXT
> +		LOCK_TEXT
> +		CPUIDLE_TEXT
> +		IRQENTRY_TEXT
> +		SOFTIRQENTRY_TEXT
> +	}
> +	_etext = . ;
> +
> +	.syscall_stub : ALIGN(PAGE_SIZE)
> +	{
> +		__syscall_stub_start = .;
> +		*(.__syscall_stub*)
> +		__syscall_stub_end = .;
> +	}
> +
> +	_sdata = . ;
> +	RO_DATA(PAGE_SIZE)
> +	RW_DATA(L1_CACHE_BYTES, PAGE_SIZE, THREAD_SIZE)
> +	.data : ALIGN(THREAD_SIZE)  { }
> +
> +	EXCEPTION_TABLE(16)
> +
> +	.uml.init_irqstack : ALIGN(THREAD_SIZE) {
> +		*(.data..init_irqstack)
> +	}
> +
> +	.uml.setup.init : ALIGN(8) {
> +		__uml_setup_start = .;
> +		*(.uml.setup.init)
> +		__uml_setup_end = .;
> +	}
> +
> +	.uml.help.init : ALIGN(8)  {
> +		__uml_help_start = .;
> +		*(.uml.help.init)
> +		__uml_help_end = .;
> +	}
> +
> +	.uml.postsetup.init : ALIGN(8) {
> +		__uml_postsetup_start = .;
> +		*(.uml.postsetup.init)
> +		__uml_postsetup_end = .;
> +	}
> +
> +	.uml.exitcall : ALIGN(8) {
> +		__uml_exitcall_begin = .;
> +		*(.uml.exitcall.exit)
> +		__uml_exitcall_end = .;
> +	}
> +	_edata = . ;
> +
> +	BSS_SECTION(0, 0, 0)
> +	_end = . ;
> +
> +	STABS_DEBUG
> +
> +	DWARF_DEBUG
> +
> +	DISCARDS
> +}
> diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
> index e6e2d9e5ff48..903fc8053d07 100755
> --- a/scripts/link-vmlinux.sh
> +++ b/scripts/link-vmlinux.sh
> @@ -102,36 +102,18 @@ vmlinux_link()
>   		strip_debug=-Wl,--strip-debug
>   	fi
>   
> -	if [ "${SRCARCH}" != "um" ]; then
> -		objects="--whole-archive			\
> -			${KBUILD_VMLINUX_OBJS}			\
> -			--no-whole-archive			\
> -			--start-group				\
> -			${KBUILD_VMLINUX_LIBS}			\
> -			--end-group				\
> -			${@}"
> -
> -		${LD} ${KBUILD_LDFLAGS} ${LDFLAGS_vmlinux}	\
> -			${strip_debug#-Wl,}			\
> -			-o ${output}				\
> -			-T ${lds} ${objects}
> -	else
> -		objects="-Wl,--whole-archive			\
> -			${KBUILD_VMLINUX_OBJS}			\
> -			-Wl,--no-whole-archive			\
> -			-Wl,--start-group			\
> -			${KBUILD_VMLINUX_LIBS}			\
> -			-Wl,--end-group				\
> -			${@}"
> -
> -		${CC} ${CFLAGS_vmlinux}				\
> -			${strip_debug}				\
> -			-o ${output}				\
> -			-Wl,-T,${lds}				\
> -			${objects}				\
> -			-lutil -lrt -lpthread
> -		rm -f linux
> -	fi
> +	objects="--whole-archive			\
> +		${KBUILD_VMLINUX_OBJS}			\
> +		--no-whole-archive			\
> +		--start-group				\
> +		${KBUILD_VMLINUX_LIBS}			\
> +		--end-group				\
> +		${@}"
> +
> +	${LD} ${KBUILD_LDFLAGS} ${LDFLAGS_vmlinux}	\
> +		${strip_debug#-Wl,}			\
> +		-o ${output}				\
> +		-T ${lds} ${objects}
>   }
>   
>   # generate .BTF typeinfo from DWARF debuginfo
> diff --git a/tools/um/Makefile b/tools/um/Makefile
> new file mode 100644
> index 000000000000..a63466ef7ef5
> --- /dev/null
> +++ b/tools/um/Makefile
> @@ -0,0 +1,72 @@
> +# Do not use make's built-in rules
> +# (this improves performance and avoids hard-to-debug behaviour);
> +# also do not print "Entering directory..." messages from make
> +.SUFFIXES:
> +MAKEFLAGS += -r --no-print-directory
> +
> +KCONFIG?=defconfig
> +
> +ifneq ($(silent),1)
> +  ifneq ($(V),1)
> +	QUIET_AUTOCONF       = @echo '  AUTOCONF '$@;
> +	Q = @
> +  endif
> +endif
> +
> +PREFIX   := /usr
> +
> +ifeq (,$(srctree))
> +  srctree := $(patsubst %/,%,$(dir $(shell pwd)))
> +  srctree := $(patsubst %/,%,$(dir $(srctree)))
> +endif
> +export srctree
> +
> +-include ../scripts/Makefile.include
> +
> +# OUTPUT fixup should be *after* include ../scripts/Makefile.include
> +ifneq ($(OUTPUT),)
> +  OUTPUT := $(OUTPUT)/tools/um/
> +else
> +  OUTPUT := $(CURDIR)/
> +endif
> +export OUTPUT
> +export objtree := $(OUTPUT)/../..
> +
> +
> +all:
> +
> +export CFLAGS += -I$(OUTPUT)/include -Iinclude -Wall -g -O2 -Wextra -fPIC \
> +	 -Wno-unused-parameter \
> +	 -Wno-missing-field-initializers -fno-strict-aliasing
> +
> +-include Targets
> +
> +TARGETS := $(progs-y:%=$(OUTPUT)%)
> +TARGETS += $(libs-y:%=$(OUTPUT)%)
> +all: $(TARGETS)
> +
> +# rule to build linux.o
> +$(OUTPUT)lib/linux.o:
> +	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) $(KCONFIG)
> +	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) install INSTALL_PATH=$(OUTPUT)
> +
> +$(OUTPUT)liblinux.a: $(OUTPUT)lib/linux.o $(OUTPUT)uml/liblinux-in.o
> +	$(QUIET_AR)$(AR) -rc $@ $^
> +
> +# rule to link programs
> +$(OUTPUT)%: $(OUTPUT)%-in.o $(OUTPUT)liblinux.a
> +	$(QUIET_LINK)$(CC) $(LDFLAGS) $(LDFLAGS_$(notdir $*)-y) -o $@ $^ $(LDLIBS) $(LDLIBS_$(notdir $*)-y)
> +
> +$(OUTPUT)%-in.o: FORCE
> +	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
> +
> +clean:
> +	$(call QUIET_CLEAN, objects)find $(OUTPUT) -name '*.o' -delete -o -name '\.*.cmd'\
> +	 -delete -o -name '\.*.d' -delete
> +	$(call QUIET_CLEAN, liblinux.a)$(RM) $(OUTPUT)/liblinux.a
> +	$(call QUIET_CLEAN, $(TARGETS))$(RM) $(TARGETS)
> +
> +FORCE: ;
> +.PHONY: all clean FORCE
> +.IGNORE: clean
> +.NOTPARALLEL : lib/linux.o
> diff --git a/tools/um/Targets b/tools/um/Targets
> new file mode 100644
> index 000000000000..a4711f1ef422
> --- /dev/null
> +++ b/tools/um/Targets
> @@ -0,0 +1,4 @@
> +progs-y += uml/linux
> +LDLIBS_linux-y := -lrt -lpthread -lutil
> +LDFLAGS_linux-y := -no-pie -Wl,--wrap,malloc -Wl,--wrap,free -Wl,--wrap,calloc
> +LDFLAGS_linux-$(UML_STATIC) += -static
> diff --git a/tools/um/uml/Build b/tools/um/uml/Build
> new file mode 100644
> index 000000000000..e69de29bb2d1
> 

-- 
Anton R. Ivanov
Cambridgegreys Limited. Registered in England. Company Number 10273661
https://www.cambridgegreys.com/

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v6 01/21] um: split build in kernel and host parts
  2020-09-24  7:36           ` Anton Ivanov
@ 2020-09-24  8:13             ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  8:13 UTC (permalink / raw)
  To: anton.ivanov
  Cc: linux-um, jdike, richard, tavi.purdila, retrage01,
	linux-kernel-library, linux-arch, tavi


On Thu, 24 Sep 2020 16:36:59 +0900,
Anton Ivanov wrote:

> > The patch also changes the UML build invocation, if before
> > 
> >   $ make ARCH=um defconfig
> >   $ make ARCH=um
> > 
> > was generating the executable now this only generates the relocatable
> > object.
> 
> It also fails now.
> 
> 
> ERROR: modpost: "memmove" [drivers/net/slip/slhc.ko] undefined!
> ERROR: modpost: "memset" [drivers/net/ppp/ppp_generic.ko] undefined!
> ERROR: modpost: "memset" [drivers/net/tun.ko] undefined!
> ERROR: modpost: "memmove" [drivers/block/loop.ko] undefined!
> ERROR: modpost: "memset" [drivers/block/loop.ko] undefined!
> ERROR: modpost: "memset" [fs/autofs/autofs4.ko] undefined!
> ERROR: modpost: "memset" [fs/isofs/isofs.ko] undefined!
> ERROR: modpost: "memset" [fs/binfmt_misc.ko] undefined!
> make[1]: *** [scripts/Makefile.modpost:111: Module.symvers] Error 1
> make[1]: *** Deleting file 'Module.symvers'
> make: *** [Makefile:1388: modules] Error 2
> make: *** Waiting for unfinished jobs....


Thanks for the review, I found this is a regression of patch 03/21,
364a2fc126e5 um: move arch/um/os-Linux dir to tools/um/uml

I will try to find a fix.

-- Hajime

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v6 01/21] um: split build in kernel and host parts
@ 2020-09-24  8:13             ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  8:13 UTC (permalink / raw)
  To: anton.ivanov
  Cc: linux-arch, tavi.purdila, richard, jdike, linux-um, retrage01,
	tavi, linux-kernel-library


On Thu, 24 Sep 2020 16:36:59 +0900,
Anton Ivanov wrote:

> > The patch also changes the UML build invocation, if before
> > 
> >   $ make ARCH=um defconfig
> >   $ make ARCH=um
> > 
> > was generating the executable now this only generates the relocatable
> > object.
> 
> It also fails now.
> 
> 
> ERROR: modpost: "memmove" [drivers/net/slip/slhc.ko] undefined!
> ERROR: modpost: "memset" [drivers/net/ppp/ppp_generic.ko] undefined!
> ERROR: modpost: "memset" [drivers/net/tun.ko] undefined!
> ERROR: modpost: "memmove" [drivers/block/loop.ko] undefined!
> ERROR: modpost: "memset" [drivers/block/loop.ko] undefined!
> ERROR: modpost: "memset" [fs/autofs/autofs4.ko] undefined!
> ERROR: modpost: "memset" [fs/isofs/isofs.ko] undefined!
> ERROR: modpost: "memset" [fs/binfmt_misc.ko] undefined!
> make[1]: *** [scripts/Makefile.modpost:111: Module.symvers] Error 1
> make[1]: *** Deleting file 'Module.symvers'
> make: *** [Makefile:1388: modules] Error 2
> make: *** Waiting for unfinished jobs....


Thanks for the review, I found this is a regression of patch 03/21,
364a2fc126e5 um: move arch/um/os-Linux dir to tools/um/uml

I will try to find a fix.

-- Hajime

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v6 01/21] um: split build in kernel and host parts
  2020-09-24  7:33           ` Anton Ivanov
@ 2020-09-24  8:26             ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  8:26 UTC (permalink / raw)
  To: anton.ivanov
  Cc: linux-um, jdike, richard, tavi.purdila, retrage01,
	linux-kernel-library, linux-arch, tavi


On Thu, 24 Sep 2020 16:33:49 +0900,
Anton Ivanov wrote:

> > The host build part has been implemented in tools/um so that we can
> > reuse the available host build infrastructure.
> > 
> > The patch also changes the UML build invocation, if before
> > 
> >   $ make ARCH=um defconfig
> >   $ make ARCH=um
> > 
> > was generating the executable now this only generates the relocatable
> > object.
> 
> This will break packaging in all distributions. We need to figure out an alternative way which is backward compatible with their builds.

Hmm, I understand the situation.
We may have to put additional steps after generating the relocatable
object in arch/um/Makefile.

-- Hajime

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v6 01/21] um: split build in kernel and host parts
@ 2020-09-24  8:26             ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-09-24  8:26 UTC (permalink / raw)
  To: anton.ivanov
  Cc: linux-arch, tavi.purdila, richard, jdike, linux-um, retrage01,
	tavi, linux-kernel-library


On Thu, 24 Sep 2020 16:33:49 +0900,
Anton Ivanov wrote:

> > The host build part has been implemented in tools/um so that we can
> > reuse the available host build infrastructure.
> > 
> > The patch also changes the UML build invocation, if before
> > 
> >   $ make ARCH=um defconfig
> >   $ make ARCH=um
> > 
> > was generating the executable now this only generates the relocatable
> > object.
> 
> This will break packaging in all distributions. We need to figure out an alternative way which is backward compatible with their builds.

Hmm, I understand the situation.
We may have to put additional steps after generating the relocatable
object in arch/um/Makefile.

-- Hajime

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v6 01/21] um: split build in kernel and host parts
  2020-09-24  8:26             ` Hajime Tazaki
@ 2020-09-24  8:37               ` Anton Ivanov
  -1 siblings, 0 replies; 476+ messages in thread
From: Anton Ivanov @ 2020-09-24  8:37 UTC (permalink / raw)
  To: Hajime Tazaki
  Cc: linux-arch, tavi.purdila, richard, jdike, linux-um, retrage01,
	tavi, linux-kernel-library



On 24/09/2020 09:26, Hajime Tazaki wrote:
> 
> On Thu, 24 Sep 2020 16:33:49 +0900,
> Anton Ivanov wrote:
> 
>>> The host build part has been implemented in tools/um so that we can
>>> reuse the available host build infrastructure.
>>>
>>> The patch also changes the UML build invocation, if before
>>>
>>>    $ make ARCH=um defconfig
>>>    $ make ARCH=um
>>>
>>> was generating the executable now this only generates the relocatable
>>> object.
>>
>> This will break packaging in all distributions. We need to figure out an alternative way which is backward compatible with their builds.
> 
> Hmm, I understand the situation.
> We may have to put additional steps after generating the relocatable
> object in arch/um/Makefile.

Cool.

I will go through the rest of the patches and if I notice something I will comment on specific ones.

Brgds,

> 
> -- Hajime
> 
> _______________________________________________
> linux-um mailing list
> linux-um@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-um
> 

-- 
Anton R. Ivanov
Cambridgegreys Limited. Registered in England. Company Number 10273661
https://www.cambridgegreys.com/

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v6 01/21] um: split build in kernel and host parts
@ 2020-09-24  8:37               ` Anton Ivanov
  0 siblings, 0 replies; 476+ messages in thread
From: Anton Ivanov @ 2020-09-24  8:37 UTC (permalink / raw)
  To: Hajime Tazaki
  Cc: linux-arch, tavi.purdila, richard, jdike, linux-um, retrage01,
	tavi, linux-kernel-library



On 24/09/2020 09:26, Hajime Tazaki wrote:
> 
> On Thu, 24 Sep 2020 16:33:49 +0900,
> Anton Ivanov wrote:
> 
>>> The host build part has been implemented in tools/um so that we can
>>> reuse the available host build infrastructure.
>>>
>>> The patch also changes the UML build invocation, if before
>>>
>>>    $ make ARCH=um defconfig
>>>    $ make ARCH=um
>>>
>>> was generating the executable now this only generates the relocatable
>>> object.
>>
>> This will break packaging in all distributions. We need to figure out an alternative way which is backward compatible with their builds.
> 
> Hmm, I understand the situation.
> We may have to put additional steps after generating the relocatable
> object in arch/um/Makefile.

Cool.

I will go through the rest of the patches and if I notice something I will comment on specific ones.

Brgds,

> 
> -- Hajime
> 
> _______________________________________________
> linux-um mailing list
> linux-um@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-um
> 

-- 
Anton R. Ivanov
Cambridgegreys Limited. Registered in England. Company Number 10273661
https://www.cambridgegreys.com/

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* [RFC v7 00/21] Unifying LKL into UML
  2020-09-24  7:12       ` Hajime Tazaki
@ 2020-10-06  9:44         ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-06  9:44 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch, Hajime Tazaki

This is another spin of the unification of LKL into UML.  Based on the
discussion of v4 patchset, we have tried to address issue raised and
rewrote the patchset from scratch.  The summary is listed in the
changelog below.

Although there are still bugs in the patchset, we'd like to ask your
opinions on the design we changed.

The milestone section is also updated: this patchset is for the
milestone 1, though the common init API is still not implemented yet.


Changes in rfc v7:
- preserve `make ARCH=um` syntax to build UML
- introduce `make ARCH=um UMMODE=library` to build library mode
- fix undefined symbols issue during modpost
- clean up makefiles (arch/um, tools/um)

Changes in rfc v6:
- rebase with the current linus tree

Changes in rfc v5:
- rewrite whole patchset from scratch
- move arch-dependent code of arch/um and arch/x86/um to tools/um
 - mainly code under os-Linux/ involved
 - introduce 2-stage build (kernel, and host-dependent parts)
 - clean up vmlinux.lds.S
- put LKL-specific implementations as a SUBARCH under arch/um/nommu
- introduce !CONFIG_MMU in arch/um
- use struct arch_thread and arch_switch_to() for subarch-specific
  thread implementation
- integrate with the IRQ infrastructure of UML
- tested with block device drivers (ubd) for the proof

Changes in rfc v4: (https://lwn.net/Articles/816276/)
- Rebase on the current uml/master branch
- Fix IRQ handling (bug fix)
- drop a patch for CONFIG_GENERIC_ATOMIC64 (comment by Peter Zijlstra)
- implement vector net driver for UMMODE_LIB (comment by Anton)
- clean up uapi headers to avoid duplicates
- clean up IRQ handling code (comment by Anton)
- fix error handling in test code (comments by David Disseldorp)

Changes in rfc v3:
- use UML drivers (net, block) from LKL programs
- drop virtio device implementations
- drop mingw32 (Windows) host
- drop android (arm/aarch64) host
- drop FreeBSD (x86_64) host
- drop LD_PRELOAD (hijack) support
- update milestone

rfc v2:
- use UMMODE instead of SUBARCH to switch UML or LKL
- tools/lkl directory is still there. I confirmed we can move under arch/um
  (e.g., arch/um/lkl/hosts).  I will move it IF this is preferable.
- drop several patches involved non-uml directory
- drop several patches which are not required
- refine commit logs
- document updated




LKL (Linux Kernel Library) is aiming to allow reusing the Linux kernel code
as extensively as possible with minimal effort and reduced maintenance
overhead.

Examples of how LKL can be used are: creating userspace applications
(running on Linux and other operating systems) that can read or write Linux
filesystems or can use the Linux networking stack, creating kernel drivers
for other operating systems that can read Linux filesystems, bootloaders
support for reading/writing Linux filesystems, etc.

With LKL, the kernel code is compiled into an object file that can be
directly linked by applications. The API offered by LKL is based on the
Linux system call interface.

LKL is originally implemented as an architecture port in arch/lkl, but this
series of commits tries to integrate this into arch/um as one of the mode
of UML.  This was discussed during RFC email of LKL (*1).

The latest LKL version can be found at https://github.com/lkl/linux

Milestone
=========
This patches is a first step toward upstreaming *library mode* of Linux kernel,
but we think we need to have several steps toward our goal, describing in the
below.


Milestone 1: LKL lib on top of UML
 * Kernel - Host build split
 -  Build UML as a relocatable object using the UML's kernel linker script.
 -  Move the ptrace and other well isolated os code out of arch/um to
    tools/um
 -  Use standard host toolchain to create a static library stripped of
    the ptrace code. Use standard host toolchain to build the main UML
    executable.
 -  Add library init API that creates the UML kernel process and starts
    UML.
 * System calls APIs
 -  Add new system call interface based on UML's irq facility.
 -  Use the LKL scripts to export the required headers to create system
    calls APIs that use the UML system calls infrastructure.
 -  Keep the underlying host and driver operations (threads, irqs, etc.)
    as they are now in UML.
 * Boot test
 -  Port the LKL boot test to verify that we are able to programatically
    issue system calls.

Milestone 2: add virtio disk support
 * Export asm/io.h operations to host/os. Create IO access operations
   and redirect them to weak os_ variants that use the current UML
   implementation.
 * Add the LKL IO access layer including generic virtio handling and the
   virtio block device code.
 * Port LKL disk test and disk apps (lklfuse, fs2tar, cptofs)

Milestone 3: new arch ports
  * Abstract the system call / IRQ mode the move the implementation to host
  * Abstract the thread model and move the implementation to host
  * Add LKL thread model and LKL ports


Building LKL the host library and LKL applications
==================================================

% cd tools/um
% make

will build LKL as a object file, it will install it in tools/um/lib together
with the headers files in tools/um/include then will build the host library,
tests and a few of application examples:

* tests/boot - a simple applications that uses LKL and exercises the basic
  LKL APIs

% make run-tests

should run the above `tests/boot` and `tests/net-test` and report errors if
there are any.

Supported hosts
===============

Currently LKL supports Linux userspace applications. New hosts can be added
relatively easy if the host supports gcc and GNU ld. Previous versions of
LKL supported Windows kernel and Haiku kernel hosts, and we also have WIP
patches with rump-hypercall interface, used in UEFI, as well as macOS
userspace (part of POSIX).

There is also musl-libc port for LKL, which might be interested in for some
folks.


Further readings about LKL
=========================

- Discussion in github LKL issue
https://github.com/lkl/linux/issues/304

- LKL (an article)
https://www.researchgate.net/profile/Nicolae_Tapus2/publication/224164682_LKL_The_Linux_kernel_library/links/02bfe50fd921ab4f7c000000.pdf

*1 RFC email to LKML (back in 2015)
https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1012277.html


Please review the following changes for suitability for inclusion. If you have
any objections or suggestions for improvement, please respond to the patches. If
you agree with the changes, please provide your Acked-by.

The following changes since commit 805c6d3c19210c90c109107d189744e960eae025:

  Merge branch 'fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs (2020-09-22 15:08:41 -0700)

are available in the Git repository at:

  git://github.com/thehajime/linux a8686efcbad3c445d3e5bde054c04a58877915f1
  https://github.com/thehajime/linux/tree/uml-lkl-5.9rc6-v7

Hajime Tazaki (18):
  um: move arch/um/os-Linux dir to tools/um/uml
  um: move arch/x86/um/os-Linux to tools/um/uml/
  scritps: um: suppress warnings if SRCARCH=um
  um: extend arch_switch_to for alternate SUBARCH
  um: add nommu mode for UML library mode
  um: nommu: host interface
  um: nommu: memory handling
  um: nommu: kernel thread support
  um: nommu: system call interface and application API
  um: nommu: basic console support
  um: nommu: initialization and cleanup
  um: nommu: integrate with irq infrastructure of UML
  um: nommu: plug in the build system
  um: host: add nommu build for ARCH=um
  um: host: add utilities functions
  um: host: posix host operations
  um: host: add test programs
  um: nommu: add block device support of UML

Octavian Purdila (3):
  um: split build in kernel and host parts
  um: add os init and exit calls
  um: host: implement os_initcalls and os_exitcalls

 arch/um/Kconfig                               |  29 +-
 arch/um/Makefile                              |  35 +-
 arch/um/drivers/Makefile                      |  10 +-
 .../um/{os-Linux => }/drivers/ethertap_kern.c |   0
 arch/um/{os-Linux => }/drivers/tuntap_kern.c  |   0
 arch/um/include/asm/common.lds.S              |  99 ----
 arch/um/include/asm/host_ops.h                |   9 +
 arch/um/include/asm/mmu.h                     |   3 +
 arch/um/include/asm/mmu_context.h             |   8 +
 arch/um/include/asm/page.h                    |  15 +
 arch/um/include/asm/pgtable.h                 |  27 +
 arch/um/include/asm/thread_info.h             |  24 +
 arch/um/include/asm/uaccess.h                 |   6 +
 arch/um/include/asm/xor.h                     |   3 +-
 arch/um/include/shared/as-layout.h            |   1 +
 .../drivers => include/shared}/etap.h         |   0
 arch/um/include/shared/init.h                 |  19 +-
 arch/um/include/shared/os.h                   |   1 +
 .../drivers => include/shared}/tuntap.h       |   0
 arch/um/include/uapi/asm/Kbuild               |   2 +
 arch/um/kernel/Makefile                       |  13 +-
 arch/um/kernel/dyn.lds.S                      | 171 -------
 arch/um/kernel/irq.c                          |  13 +
 arch/um/kernel/process.c                      |  14 +-
 arch/um/kernel/reboot.c                       |   5 +
 arch/um/kernel/time.c                         |   2 +
 arch/um/kernel/um_arch.c                      |  16 +
 arch/um/kernel/uml.lds.S                      | 115 -----
 arch/um/{os-Linux => kernel}/user_syms.c      |   0
 arch/um/kernel/vmlinux.lds.S                  |  91 +++-
 arch/um/nommu/Makefile                        |   1 +
 arch/um/nommu/Makefile.um                     |  22 +
 arch/um/nommu/include/asm/Kbuild              |   6 +
 arch/um/nommu/include/asm/archparam.h         |   1 +
 arch/um/nommu/include/asm/atomic.h            |  11 +
 arch/um/nommu/include/asm/atomic64.h          | 114 +++++
 arch/um/nommu/include/asm/bitsperlong.h       |  12 +
 arch/um/nommu/include/asm/byteorder.h         |   7 +
 arch/um/nommu/include/asm/cpu.h               |  16 +
 arch/um/nommu/include/asm/elf.h               |  15 +
 arch/um/nommu/include/asm/mm_context.h        |   8 +
 arch/um/nommu/include/asm/processor.h         |  46 ++
 arch/um/nommu/include/asm/ptrace.h            |  21 +
 arch/um/nommu/include/asm/sched.h             |  23 +
 arch/um/nommu/include/asm/segment.h           |   9 +
 arch/um/nommu/include/asm/syscall_wrapper.h   |  57 +++
 arch/um/nommu/include/asm/syscalls.h          |  15 +
 arch/um/nommu/include/uapi/asm/Kbuild         |   6 +
 arch/um/nommu/include/uapi/asm/bitsperlong.h  |  11 +
 arch/um/nommu/include/uapi/asm/byteorder.h    |  11 +
 arch/um/nommu/include/uapi/asm/host_ops.h     | 122 +++++
 arch/um/nommu/include/uapi/asm/sigcontext.h   |  12 +
 arch/um/nommu/include/uapi/asm/syscalls.h     | 287 +++++++++++
 arch/um/nommu/include/uapi/asm/unistd.h       |  17 +
 arch/um/nommu/um/Kconfig                      |  45 ++
 arch/um/nommu/um/Makefile                     |   4 +
 arch/um/nommu/um/bootmem.c                    |  87 ++++
 arch/um/nommu/um/console.c                    |  42 ++
 arch/um/nommu/um/cpu.c                        | 247 ++++++++++
 arch/um/nommu/um/delay.c                      |  31 ++
 arch/um/nommu/um/setup.c                      | 179 +++++++
 arch/um/nommu/um/shared/sysdep/archsetjmp.h   |  13 +
 arch/um/nommu/um/shared/sysdep/faultinfo.h    |   8 +
 .../nommu/um/shared/sysdep/kernel-offsets.h   |  12 +
 arch/um/nommu/um/shared/sysdep/mcontext.h     |   9 +
 arch/um/nommu/um/shared/sysdep/ptrace.h       |  42 ++
 arch/um/nommu/um/shared/sysdep/ptrace_user.h  |   7 +
 arch/um/nommu/um/syscalls.c                   | 199 ++++++++
 arch/um/nommu/um/threads.c                    | 262 ++++++++++
 arch/um/nommu/um/unimplemented.c              |  70 +++
 arch/um/nommu/um/user_constants.h             |  13 +
 arch/um/os-Linux/Makefile                     |  19 -
 arch/um/os-Linux/drivers/Makefile             |  13 -
 arch/um/scripts/headers_install.py            | 197 ++++++++
 arch/x86/um/Makefile                          |   2 +-
 arch/x86/um/os-Linux/Makefile                 |  13 -
 arch/x86/um/ptrace_32.c                       |   2 +-
 arch/x86/um/syscalls_64.c                     |   2 +-
 scripts/headers_install.sh                    |   4 +
 scripts/link-vmlinux.sh                       |  42 +-
 tools/um/.gitignore                           |   1 +
 tools/um/Makefile                             |  81 +++
 tools/um/Targets                              |  13 +
 tools/um/include/lkl.h                        | 357 ++++++++++++++
 tools/um/include/lkl_host.h                   |  27 +
 tools/um/lib/Build                            |   7 +
 tools/um/lib/fs.c                             | 461 ++++++++++++++++++
 tools/um/lib/jmp_buf.c                        |  14 +
 tools/um/lib/jmp_buf.h                        |   8 +
 tools/um/lib/posix-host.c                     | 293 +++++++++++
 tools/um/lib/utils.c                          | 213 ++++++++
 tools/um/tests/Build                          |   4 +
 tools/um/tests/boot.c                         | 393 +++++++++++++++
 tools/um/tests/boot.sh                        |   9 +
 tools/um/tests/cla.c                          | 159 ++++++
 tools/um/tests/cla.h                          |  33 ++
 tools/um/tests/disk.c                         | 168 +++++++
 tools/um/tests/disk.sh                        |  70 +++
 tools/um/tests/run.py                         | 174 +++++++
 tools/um/tests/tap13.py                       | 209 ++++++++
 tools/um/tests/test.c                         | 126 +++++
 tools/um/tests/test.h                         |  72 +++
 tools/um/tests/test.sh                        | 181 +++++++
 tools/um/uml/Build                            |  58 +++
 tools/um/uml/drivers/Build                    |  10 +
 .../um/uml}/drivers/ethertap_user.c           |   0
 .../um/uml}/drivers/tuntap_user.c             |   0
 {arch/um/os-Linux => tools/um/uml}/elf_aux.c  |   0
 {arch/um/os-Linux => tools/um/uml}/execvp.c   |   4 -
 {arch/um/os-Linux => tools/um/uml}/file.c     |   0
 {arch/um/os-Linux => tools/um/uml}/helper.c   |   0
 {arch/um/os-Linux => tools/um/uml}/irq.c      |   0
 {arch/um/os-Linux => tools/um/uml}/main.c     |   0
 {arch/um/os-Linux => tools/um/uml}/mem.c      |   0
 tools/um/uml/nommu/Build                      |   1 +
 tools/um/uml/nommu/registers.c                |  21 +
 tools/um/uml/nommu/unimplemented.c            |  21 +
 {arch/um/os-Linux => tools/um/uml}/process.c  |   2 +
 .../um/os-Linux => tools/um/uml}/registers.c  |   0
 {arch/um/os-Linux => tools/um/uml}/sigio.c    |   0
 {arch/um/os-Linux => tools/um/uml}/signal.c   |  12 +-
 .../skas/Makefile => tools/um/uml/skas/Build  |   6 +-
 {arch/um/os-Linux => tools/um/uml}/skas/mem.c |   0
 .../os-Linux => tools/um/uml}/skas/process.c  |   3 +-
 {arch/um/os-Linux => tools/um/uml}/start_up.c |   0
 {arch/um/os-Linux => tools/um/uml}/time.c     |   0
 {arch/um/os-Linux => tools/um/uml}/tty.c      |   0
 {arch/um/os-Linux => tools/um/uml}/umid.c     |   0
 {arch/um/os-Linux => tools/um/uml}/util.c     |  26 +
 tools/um/uml/x86/Build                        |  11 +
 .../os-Linux => tools/um/uml/x86}/mcontext.c  |   0
 .../um/os-Linux => tools/um/uml/x86}/prctl.c  |   0
 .../os-Linux => tools/um/uml/x86}/registers.c |   0
 .../os-Linux => tools/um/uml/x86}/task_size.c |   0
 .../um/os-Linux => tools/um/uml/x86}/tls.c    |   0
 135 files changed, 5870 insertions(+), 523 deletions(-)
 rename arch/um/{os-Linux => }/drivers/ethertap_kern.c (100%)
 rename arch/um/{os-Linux => }/drivers/tuntap_kern.c (100%)
 delete mode 100644 arch/um/include/asm/common.lds.S
 create mode 100644 arch/um/include/asm/host_ops.h
 rename arch/um/{os-Linux/drivers => include/shared}/etap.h (100%)
 rename arch/um/{os-Linux/drivers => include/shared}/tuntap.h (100%)
 create mode 100644 arch/um/include/uapi/asm/Kbuild
 delete mode 100644 arch/um/kernel/dyn.lds.S
 delete mode 100644 arch/um/kernel/uml.lds.S
 rename arch/um/{os-Linux => kernel}/user_syms.c (100%)
 create mode 100644 arch/um/nommu/Makefile
 create mode 100644 arch/um/nommu/Makefile.um
 create mode 100644 arch/um/nommu/include/asm/Kbuild
 create mode 100644 arch/um/nommu/include/asm/archparam.h
 create mode 100644 arch/um/nommu/include/asm/atomic.h
 create mode 100644 arch/um/nommu/include/asm/atomic64.h
 create mode 100644 arch/um/nommu/include/asm/bitsperlong.h
 create mode 100644 arch/um/nommu/include/asm/byteorder.h
 create mode 100644 arch/um/nommu/include/asm/cpu.h
 create mode 100644 arch/um/nommu/include/asm/elf.h
 create mode 100644 arch/um/nommu/include/asm/mm_context.h
 create mode 100644 arch/um/nommu/include/asm/processor.h
 create mode 100644 arch/um/nommu/include/asm/ptrace.h
 create mode 100644 arch/um/nommu/include/asm/sched.h
 create mode 100644 arch/um/nommu/include/asm/segment.h
 create mode 100644 arch/um/nommu/include/asm/syscall_wrapper.h
 create mode 100644 arch/um/nommu/include/asm/syscalls.h
 create mode 100644 arch/um/nommu/include/uapi/asm/Kbuild
 create mode 100644 arch/um/nommu/include/uapi/asm/bitsperlong.h
 create mode 100644 arch/um/nommu/include/uapi/asm/byteorder.h
 create mode 100644 arch/um/nommu/include/uapi/asm/host_ops.h
 create mode 100644 arch/um/nommu/include/uapi/asm/sigcontext.h
 create mode 100644 arch/um/nommu/include/uapi/asm/syscalls.h
 create mode 100644 arch/um/nommu/include/uapi/asm/unistd.h
 create mode 100644 arch/um/nommu/um/Kconfig
 create mode 100644 arch/um/nommu/um/Makefile
 create mode 100644 arch/um/nommu/um/bootmem.c
 create mode 100644 arch/um/nommu/um/console.c
 create mode 100644 arch/um/nommu/um/cpu.c
 create mode 100644 arch/um/nommu/um/delay.c
 create mode 100644 arch/um/nommu/um/setup.c
 create mode 100644 arch/um/nommu/um/shared/sysdep/archsetjmp.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/faultinfo.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/kernel-offsets.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/mcontext.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/ptrace.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/ptrace_user.h
 create mode 100644 arch/um/nommu/um/syscalls.c
 create mode 100644 arch/um/nommu/um/threads.c
 create mode 100644 arch/um/nommu/um/unimplemented.c
 create mode 100644 arch/um/nommu/um/user_constants.h
 delete mode 100644 arch/um/os-Linux/Makefile
 delete mode 100644 arch/um/os-Linux/drivers/Makefile
 create mode 100755 arch/um/scripts/headers_install.py
 delete mode 100644 arch/x86/um/os-Linux/Makefile
 create mode 100644 tools/um/.gitignore
 create mode 100644 tools/um/Makefile
 create mode 100644 tools/um/Targets
 create mode 100644 tools/um/include/lkl.h
 create mode 100644 tools/um/include/lkl_host.h
 create mode 100644 tools/um/lib/Build
 create mode 100644 tools/um/lib/fs.c
 create mode 100644 tools/um/lib/jmp_buf.c
 create mode 100644 tools/um/lib/jmp_buf.h
 create mode 100644 tools/um/lib/posix-host.c
 create mode 100644 tools/um/lib/utils.c
 create mode 100644 tools/um/tests/Build
 create mode 100644 tools/um/tests/boot.c
 create mode 100755 tools/um/tests/boot.sh
 create mode 100644 tools/um/tests/cla.c
 create mode 100644 tools/um/tests/cla.h
 create mode 100644 tools/um/tests/disk.c
 create mode 100755 tools/um/tests/disk.sh
 create mode 100755 tools/um/tests/run.py
 create mode 100644 tools/um/tests/tap13.py
 create mode 100644 tools/um/tests/test.c
 create mode 100644 tools/um/tests/test.h
 create mode 100644 tools/um/tests/test.sh
 create mode 100644 tools/um/uml/Build
 create mode 100644 tools/um/uml/drivers/Build
 rename {arch/um/os-Linux => tools/um/uml}/drivers/ethertap_user.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/drivers/tuntap_user.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/elf_aux.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/execvp.c (98%)
 rename {arch/um/os-Linux => tools/um/uml}/file.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/helper.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/irq.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/main.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/mem.c (100%)
 create mode 100644 tools/um/uml/nommu/Build
 create mode 100644 tools/um/uml/nommu/registers.c
 create mode 100644 tools/um/uml/nommu/unimplemented.c
 rename {arch/um/os-Linux => tools/um/uml}/process.c (99%)
 rename {arch/um/os-Linux => tools/um/uml}/registers.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/sigio.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/signal.c (96%)
 rename arch/um/os-Linux/skas/Makefile => tools/um/uml/skas/Build (56%)
 rename {arch/um/os-Linux => tools/um/uml}/skas/mem.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/skas/process.c (99%)
 rename {arch/um/os-Linux => tools/um/uml}/start_up.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/time.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/tty.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/umid.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/util.c (89%)
 create mode 100644 tools/um/uml/x86/Build
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/mcontext.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/prctl.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/registers.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/task_size.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/tls.c (100%)

-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply	[flat|nested] 476+ messages in thread
* [RFC v7 00/21] Unifying LKL into UML
@ 2020-10-06  9:44         ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-06  9:44 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Hajime Tazaki, retrage01

This is another spin of the unification of LKL into UML.  Based on the
discussion of v4 patchset, we have tried to address issue raised and
rewrote the patchset from scratch.  The summary is listed in the
changelog below.

Although there are still bugs in the patchset, we'd like to ask your
opinions on the design we changed.

The milestone section is also updated: this patchset is for the
milestone 1, though the common init API is still not implemented yet.


Changes in rfc v7:
- preserve `make ARCH=um` syntax to build UML
- introduce `make ARCH=um UMMODE=library` to build library mode
- fix undefined symbols issue during modpost
- clean up makefiles (arch/um, tools/um)

Changes in rfc v6:
- rebase with the current linus tree

Changes in rfc v5:
- rewrite whole patchset from scratch
- move arch-dependent code of arch/um and arch/x86/um to tools/um
 - mainly code under os-Linux/ involved
 - introduce 2-stage build (kernel, and host-dependent parts)
 - clean up vmlinux.lds.S
- put LKL-specific implementations as a SUBARCH under arch/um/nommu
- introduce !CONFIG_MMU in arch/um
- use struct arch_thread and arch_switch_to() for subarch-specific
  thread implementation
- integrate with the IRQ infrastructure of UML
- tested with block device drivers (ubd) for the proof

Changes in rfc v4: (https://lwn.net/Articles/816276/)
- Rebase on the current uml/master branch
- Fix IRQ handling (bug fix)
- drop a patch for CONFIG_GENERIC_ATOMIC64 (comment by Peter Zijlstra)
- implement vector net driver for UMMODE_LIB (comment by Anton)
- clean up uapi headers to avoid duplicates
- clean up IRQ handling code (comment by Anton)
- fix error handling in test code (comments by David Disseldorp)

Changes in rfc v3:
- use UML drivers (net, block) from LKL programs
- drop virtio device implementations
- drop mingw32 (Windows) host
- drop android (arm/aarch64) host
- drop FreeBSD (x86_64) host
- drop LD_PRELOAD (hijack) support
- update milestone

rfc v2:
- use UMMODE instead of SUBARCH to switch UML or LKL
- tools/lkl directory is still there. I confirmed we can move under arch/um
  (e.g., arch/um/lkl/hosts).  I will move it IF this is preferable.
- drop several patches involved non-uml directory
- drop several patches which are not required
- refine commit logs
- document updated




LKL (Linux Kernel Library) is aiming to allow reusing the Linux kernel code
as extensively as possible with minimal effort and reduced maintenance
overhead.

Examples of how LKL can be used are: creating userspace applications
(running on Linux and other operating systems) that can read or write Linux
filesystems or can use the Linux networking stack, creating kernel drivers
for other operating systems that can read Linux filesystems, bootloaders
support for reading/writing Linux filesystems, etc.

With LKL, the kernel code is compiled into an object file that can be
directly linked by applications. The API offered by LKL is based on the
Linux system call interface.

LKL is originally implemented as an architecture port in arch/lkl, but this
series of commits tries to integrate this into arch/um as one of the mode
of UML.  This was discussed during RFC email of LKL (*1).

The latest LKL version can be found at https://github.com/lkl/linux

Milestone
=========
This patches is a first step toward upstreaming *library mode* of Linux kernel,
but we think we need to have several steps toward our goal, describing in the
below.


Milestone 1: LKL lib on top of UML
 * Kernel - Host build split
 -  Build UML as a relocatable object using the UML's kernel linker script.
 -  Move the ptrace and other well isolated os code out of arch/um to
    tools/um
 -  Use standard host toolchain to create a static library stripped of
    the ptrace code. Use standard host toolchain to build the main UML
    executable.
 -  Add library init API that creates the UML kernel process and starts
    UML.
 * System calls APIs
 -  Add new system call interface based on UML's irq facility.
 -  Use the LKL scripts to export the required headers to create system
    calls APIs that use the UML system calls infrastructure.
 -  Keep the underlying host and driver operations (threads, irqs, etc.)
    as they are now in UML.
 * Boot test
 -  Port the LKL boot test to verify that we are able to programatically
    issue system calls.

Milestone 2: add virtio disk support
 * Export asm/io.h operations to host/os. Create IO access operations
   and redirect them to weak os_ variants that use the current UML
   implementation.
 * Add the LKL IO access layer including generic virtio handling and the
   virtio block device code.
 * Port LKL disk test and disk apps (lklfuse, fs2tar, cptofs)

Milestone 3: new arch ports
  * Abstract the system call / IRQ mode the move the implementation to host
  * Abstract the thread model and move the implementation to host
  * Add LKL thread model and LKL ports


Building LKL the host library and LKL applications
==================================================

% cd tools/um
% make

will build LKL as a object file, it will install it in tools/um/lib together
with the headers files in tools/um/include then will build the host library,
tests and a few of application examples:

* tests/boot - a simple applications that uses LKL and exercises the basic
  LKL APIs

% make run-tests

should run the above `tests/boot` and `tests/net-test` and report errors if
there are any.

Supported hosts
===============

Currently LKL supports Linux userspace applications. New hosts can be added
relatively easy if the host supports gcc and GNU ld. Previous versions of
LKL supported Windows kernel and Haiku kernel hosts, and we also have WIP
patches with rump-hypercall interface, used in UEFI, as well as macOS
userspace (part of POSIX).

There is also musl-libc port for LKL, which might be interested in for some
folks.


Further readings about LKL
=========================

- Discussion in github LKL issue
https://github.com/lkl/linux/issues/304

- LKL (an article)
https://www.researchgate.net/profile/Nicolae_Tapus2/publication/224164682_LKL_The_Linux_kernel_library/links/02bfe50fd921ab4f7c000000.pdf

*1 RFC email to LKML (back in 2015)
https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1012277.html


Please review the following changes for suitability for inclusion. If you have
any objections or suggestions for improvement, please respond to the patches. If
you agree with the changes, please provide your Acked-by.

The following changes since commit 805c6d3c19210c90c109107d189744e960eae025:

  Merge branch 'fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs (2020-09-22 15:08:41 -0700)

are available in the Git repository at:

  git://github.com/thehajime/linux a8686efcbad3c445d3e5bde054c04a58877915f1
  https://github.com/thehajime/linux/tree/uml-lkl-5.9rc6-v7

Hajime Tazaki (18):
  um: move arch/um/os-Linux dir to tools/um/uml
  um: move arch/x86/um/os-Linux to tools/um/uml/
  scritps: um: suppress warnings if SRCARCH=um
  um: extend arch_switch_to for alternate SUBARCH
  um: add nommu mode for UML library mode
  um: nommu: host interface
  um: nommu: memory handling
  um: nommu: kernel thread support
  um: nommu: system call interface and application API
  um: nommu: basic console support
  um: nommu: initialization and cleanup
  um: nommu: integrate with irq infrastructure of UML
  um: nommu: plug in the build system
  um: host: add nommu build for ARCH=um
  um: host: add utilities functions
  um: host: posix host operations
  um: host: add test programs
  um: nommu: add block device support of UML

Octavian Purdila (3):
  um: split build in kernel and host parts
  um: add os init and exit calls
  um: host: implement os_initcalls and os_exitcalls

 arch/um/Kconfig                               |  29 +-
 arch/um/Makefile                              |  35 +-
 arch/um/drivers/Makefile                      |  10 +-
 .../um/{os-Linux => }/drivers/ethertap_kern.c |   0
 arch/um/{os-Linux => }/drivers/tuntap_kern.c  |   0
 arch/um/include/asm/common.lds.S              |  99 ----
 arch/um/include/asm/host_ops.h                |   9 +
 arch/um/include/asm/mmu.h                     |   3 +
 arch/um/include/asm/mmu_context.h             |   8 +
 arch/um/include/asm/page.h                    |  15 +
 arch/um/include/asm/pgtable.h                 |  27 +
 arch/um/include/asm/thread_info.h             |  24 +
 arch/um/include/asm/uaccess.h                 |   6 +
 arch/um/include/asm/xor.h                     |   3 +-
 arch/um/include/shared/as-layout.h            |   1 +
 .../drivers => include/shared}/etap.h         |   0
 arch/um/include/shared/init.h                 |  19 +-
 arch/um/include/shared/os.h                   |   1 +
 .../drivers => include/shared}/tuntap.h       |   0
 arch/um/include/uapi/asm/Kbuild               |   2 +
 arch/um/kernel/Makefile                       |  13 +-
 arch/um/kernel/dyn.lds.S                      | 171 -------
 arch/um/kernel/irq.c                          |  13 +
 arch/um/kernel/process.c                      |  14 +-
 arch/um/kernel/reboot.c                       |   5 +
 arch/um/kernel/time.c                         |   2 +
 arch/um/kernel/um_arch.c                      |  16 +
 arch/um/kernel/uml.lds.S                      | 115 -----
 arch/um/{os-Linux => kernel}/user_syms.c      |   0
 arch/um/kernel/vmlinux.lds.S                  |  91 +++-
 arch/um/nommu/Makefile                        |   1 +
 arch/um/nommu/Makefile.um                     |  22 +
 arch/um/nommu/include/asm/Kbuild              |   6 +
 arch/um/nommu/include/asm/archparam.h         |   1 +
 arch/um/nommu/include/asm/atomic.h            |  11 +
 arch/um/nommu/include/asm/atomic64.h          | 114 +++++
 arch/um/nommu/include/asm/bitsperlong.h       |  12 +
 arch/um/nommu/include/asm/byteorder.h         |   7 +
 arch/um/nommu/include/asm/cpu.h               |  16 +
 arch/um/nommu/include/asm/elf.h               |  15 +
 arch/um/nommu/include/asm/mm_context.h        |   8 +
 arch/um/nommu/include/asm/processor.h         |  46 ++
 arch/um/nommu/include/asm/ptrace.h            |  21 +
 arch/um/nommu/include/asm/sched.h             |  23 +
 arch/um/nommu/include/asm/segment.h           |   9 +
 arch/um/nommu/include/asm/syscall_wrapper.h   |  57 +++
 arch/um/nommu/include/asm/syscalls.h          |  15 +
 arch/um/nommu/include/uapi/asm/Kbuild         |   6 +
 arch/um/nommu/include/uapi/asm/bitsperlong.h  |  11 +
 arch/um/nommu/include/uapi/asm/byteorder.h    |  11 +
 arch/um/nommu/include/uapi/asm/host_ops.h     | 122 +++++
 arch/um/nommu/include/uapi/asm/sigcontext.h   |  12 +
 arch/um/nommu/include/uapi/asm/syscalls.h     | 287 +++++++++++
 arch/um/nommu/include/uapi/asm/unistd.h       |  17 +
 arch/um/nommu/um/Kconfig                      |  45 ++
 arch/um/nommu/um/Makefile                     |   4 +
 arch/um/nommu/um/bootmem.c                    |  87 ++++
 arch/um/nommu/um/console.c                    |  42 ++
 arch/um/nommu/um/cpu.c                        | 247 ++++++++++
 arch/um/nommu/um/delay.c                      |  31 ++
 arch/um/nommu/um/setup.c                      | 179 +++++++
 arch/um/nommu/um/shared/sysdep/archsetjmp.h   |  13 +
 arch/um/nommu/um/shared/sysdep/faultinfo.h    |   8 +
 .../nommu/um/shared/sysdep/kernel-offsets.h   |  12 +
 arch/um/nommu/um/shared/sysdep/mcontext.h     |   9 +
 arch/um/nommu/um/shared/sysdep/ptrace.h       |  42 ++
 arch/um/nommu/um/shared/sysdep/ptrace_user.h  |   7 +
 arch/um/nommu/um/syscalls.c                   | 199 ++++++++
 arch/um/nommu/um/threads.c                    | 262 ++++++++++
 arch/um/nommu/um/unimplemented.c              |  70 +++
 arch/um/nommu/um/user_constants.h             |  13 +
 arch/um/os-Linux/Makefile                     |  19 -
 arch/um/os-Linux/drivers/Makefile             |  13 -
 arch/um/scripts/headers_install.py            | 197 ++++++++
 arch/x86/um/Makefile                          |   2 +-
 arch/x86/um/os-Linux/Makefile                 |  13 -
 arch/x86/um/ptrace_32.c                       |   2 +-
 arch/x86/um/syscalls_64.c                     |   2 +-
 scripts/headers_install.sh                    |   4 +
 scripts/link-vmlinux.sh                       |  42 +-
 tools/um/.gitignore                           |   1 +
 tools/um/Makefile                             |  81 +++
 tools/um/Targets                              |  13 +
 tools/um/include/lkl.h                        | 357 ++++++++++++++
 tools/um/include/lkl_host.h                   |  27 +
 tools/um/lib/Build                            |   7 +
 tools/um/lib/fs.c                             | 461 ++++++++++++++++++
 tools/um/lib/jmp_buf.c                        |  14 +
 tools/um/lib/jmp_buf.h                        |   8 +
 tools/um/lib/posix-host.c                     | 293 +++++++++++
 tools/um/lib/utils.c                          | 213 ++++++++
 tools/um/tests/Build                          |   4 +
 tools/um/tests/boot.c                         | 393 +++++++++++++++
 tools/um/tests/boot.sh                        |   9 +
 tools/um/tests/cla.c                          | 159 ++++++
 tools/um/tests/cla.h                          |  33 ++
 tools/um/tests/disk.c                         | 168 +++++++
 tools/um/tests/disk.sh                        |  70 +++
 tools/um/tests/run.py                         | 174 +++++++
 tools/um/tests/tap13.py                       | 209 ++++++++
 tools/um/tests/test.c                         | 126 +++++
 tools/um/tests/test.h                         |  72 +++
 tools/um/tests/test.sh                        | 181 +++++++
 tools/um/uml/Build                            |  58 +++
 tools/um/uml/drivers/Build                    |  10 +
 .../um/uml}/drivers/ethertap_user.c           |   0
 .../um/uml}/drivers/tuntap_user.c             |   0
 {arch/um/os-Linux => tools/um/uml}/elf_aux.c  |   0
 {arch/um/os-Linux => tools/um/uml}/execvp.c   |   4 -
 {arch/um/os-Linux => tools/um/uml}/file.c     |   0
 {arch/um/os-Linux => tools/um/uml}/helper.c   |   0
 {arch/um/os-Linux => tools/um/uml}/irq.c      |   0
 {arch/um/os-Linux => tools/um/uml}/main.c     |   0
 {arch/um/os-Linux => tools/um/uml}/mem.c      |   0
 tools/um/uml/nommu/Build                      |   1 +
 tools/um/uml/nommu/registers.c                |  21 +
 tools/um/uml/nommu/unimplemented.c            |  21 +
 {arch/um/os-Linux => tools/um/uml}/process.c  |   2 +
 .../um/os-Linux => tools/um/uml}/registers.c  |   0
 {arch/um/os-Linux => tools/um/uml}/sigio.c    |   0
 {arch/um/os-Linux => tools/um/uml}/signal.c   |  12 +-
 .../skas/Makefile => tools/um/uml/skas/Build  |   6 +-
 {arch/um/os-Linux => tools/um/uml}/skas/mem.c |   0
 .../os-Linux => tools/um/uml}/skas/process.c  |   3 +-
 {arch/um/os-Linux => tools/um/uml}/start_up.c |   0
 {arch/um/os-Linux => tools/um/uml}/time.c     |   0
 {arch/um/os-Linux => tools/um/uml}/tty.c      |   0
 {arch/um/os-Linux => tools/um/uml}/umid.c     |   0
 {arch/um/os-Linux => tools/um/uml}/util.c     |  26 +
 tools/um/uml/x86/Build                        |  11 +
 .../os-Linux => tools/um/uml/x86}/mcontext.c  |   0
 .../um/os-Linux => tools/um/uml/x86}/prctl.c  |   0
 .../os-Linux => tools/um/uml/x86}/registers.c |   0
 .../os-Linux => tools/um/uml/x86}/task_size.c |   0
 .../um/os-Linux => tools/um/uml/x86}/tls.c    |   0
 135 files changed, 5870 insertions(+), 523 deletions(-)
 rename arch/um/{os-Linux => }/drivers/ethertap_kern.c (100%)
 rename arch/um/{os-Linux => }/drivers/tuntap_kern.c (100%)
 delete mode 100644 arch/um/include/asm/common.lds.S
 create mode 100644 arch/um/include/asm/host_ops.h
 rename arch/um/{os-Linux/drivers => include/shared}/etap.h (100%)
 rename arch/um/{os-Linux/drivers => include/shared}/tuntap.h (100%)
 create mode 100644 arch/um/include/uapi/asm/Kbuild
 delete mode 100644 arch/um/kernel/dyn.lds.S
 delete mode 100644 arch/um/kernel/uml.lds.S
 rename arch/um/{os-Linux => kernel}/user_syms.c (100%)
 create mode 100644 arch/um/nommu/Makefile
 create mode 100644 arch/um/nommu/Makefile.um
 create mode 100644 arch/um/nommu/include/asm/Kbuild
 create mode 100644 arch/um/nommu/include/asm/archparam.h
 create mode 100644 arch/um/nommu/include/asm/atomic.h
 create mode 100644 arch/um/nommu/include/asm/atomic64.h
 create mode 100644 arch/um/nommu/include/asm/bitsperlong.h
 create mode 100644 arch/um/nommu/include/asm/byteorder.h
 create mode 100644 arch/um/nommu/include/asm/cpu.h
 create mode 100644 arch/um/nommu/include/asm/elf.h
 create mode 100644 arch/um/nommu/include/asm/mm_context.h
 create mode 100644 arch/um/nommu/include/asm/processor.h
 create mode 100644 arch/um/nommu/include/asm/ptrace.h
 create mode 100644 arch/um/nommu/include/asm/sched.h
 create mode 100644 arch/um/nommu/include/asm/segment.h
 create mode 100644 arch/um/nommu/include/asm/syscall_wrapper.h
 create mode 100644 arch/um/nommu/include/asm/syscalls.h
 create mode 100644 arch/um/nommu/include/uapi/asm/Kbuild
 create mode 100644 arch/um/nommu/include/uapi/asm/bitsperlong.h
 create mode 100644 arch/um/nommu/include/uapi/asm/byteorder.h
 create mode 100644 arch/um/nommu/include/uapi/asm/host_ops.h
 create mode 100644 arch/um/nommu/include/uapi/asm/sigcontext.h
 create mode 100644 arch/um/nommu/include/uapi/asm/syscalls.h
 create mode 100644 arch/um/nommu/include/uapi/asm/unistd.h
 create mode 100644 arch/um/nommu/um/Kconfig
 create mode 100644 arch/um/nommu/um/Makefile
 create mode 100644 arch/um/nommu/um/bootmem.c
 create mode 100644 arch/um/nommu/um/console.c
 create mode 100644 arch/um/nommu/um/cpu.c
 create mode 100644 arch/um/nommu/um/delay.c
 create mode 100644 arch/um/nommu/um/setup.c
 create mode 100644 arch/um/nommu/um/shared/sysdep/archsetjmp.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/faultinfo.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/kernel-offsets.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/mcontext.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/ptrace.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/ptrace_user.h
 create mode 100644 arch/um/nommu/um/syscalls.c
 create mode 100644 arch/um/nommu/um/threads.c
 create mode 100644 arch/um/nommu/um/unimplemented.c
 create mode 100644 arch/um/nommu/um/user_constants.h
 delete mode 100644 arch/um/os-Linux/Makefile
 delete mode 100644 arch/um/os-Linux/drivers/Makefile
 create mode 100755 arch/um/scripts/headers_install.py
 delete mode 100644 arch/x86/um/os-Linux/Makefile
 create mode 100644 tools/um/.gitignore
 create mode 100644 tools/um/Makefile
 create mode 100644 tools/um/Targets
 create mode 100644 tools/um/include/lkl.h
 create mode 100644 tools/um/include/lkl_host.h
 create mode 100644 tools/um/lib/Build
 create mode 100644 tools/um/lib/fs.c
 create mode 100644 tools/um/lib/jmp_buf.c
 create mode 100644 tools/um/lib/jmp_buf.h
 create mode 100644 tools/um/lib/posix-host.c
 create mode 100644 tools/um/lib/utils.c
 create mode 100644 tools/um/tests/Build
 create mode 100644 tools/um/tests/boot.c
 create mode 100755 tools/um/tests/boot.sh
 create mode 100644 tools/um/tests/cla.c
 create mode 100644 tools/um/tests/cla.h
 create mode 100644 tools/um/tests/disk.c
 create mode 100755 tools/um/tests/disk.sh
 create mode 100755 tools/um/tests/run.py
 create mode 100644 tools/um/tests/tap13.py
 create mode 100644 tools/um/tests/test.c
 create mode 100644 tools/um/tests/test.h
 create mode 100644 tools/um/tests/test.sh
 create mode 100644 tools/um/uml/Build
 create mode 100644 tools/um/uml/drivers/Build
 rename {arch/um/os-Linux => tools/um/uml}/drivers/ethertap_user.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/drivers/tuntap_user.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/elf_aux.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/execvp.c (98%)
 rename {arch/um/os-Linux => tools/um/uml}/file.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/helper.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/irq.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/main.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/mem.c (100%)
 create mode 100644 tools/um/uml/nommu/Build
 create mode 100644 tools/um/uml/nommu/registers.c
 create mode 100644 tools/um/uml/nommu/unimplemented.c
 rename {arch/um/os-Linux => tools/um/uml}/process.c (99%)
 rename {arch/um/os-Linux => tools/um/uml}/registers.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/sigio.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/signal.c (96%)
 rename arch/um/os-Linux/skas/Makefile => tools/um/uml/skas/Build (56%)
 rename {arch/um/os-Linux => tools/um/uml}/skas/mem.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/skas/process.c (99%)
 rename {arch/um/os-Linux => tools/um/uml}/start_up.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/time.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/tty.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/umid.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/util.c (89%)
 create mode 100644 tools/um/uml/x86/Build
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/mcontext.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/prctl.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/registers.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/task_size.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/tls.c (100%)

-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* [RFC v7 01/21] um: split build in kernel and host parts
  2020-10-06  9:44         ` Hajime Tazaki
@ 2020-10-06  9:44           ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-06  9:44 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch,
	Octavian Purdila, Hajime Tazaki

From: Octavian Purdila <tavi@cs.pub.ro>

This patch splits the UML build in two parts: a kernel part that
generates a relocatable object (linux.o) and a host build part that
links the relocatable object into an executable.

This allows us to simplify the linker script since we can now remove
the host bits (e.g. .rel sections). It also gives us greater
flexibility since it will be much easier to support other
architectures and OSes if we use the standard linker script.

The host build part has been implemented in tools/um so that we can
reuse the available host build infrastructure.

The patch preserves the way to invoke the UML build as follows,

 $ make ARCH=um defconfig
 $ make ARCH=um

but it internally calls tools/um/Makefile in order to complete the full
build to link the relocatable object and the rest of files.

To statically compiled UML use:

 $ make ARCH=um UML_STATIC=y

Signed-off-by: Octavian Purdila <tavi@cs.pub.ro>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/Kconfig                  |  12 +--
 arch/um/Makefile                 |  22 +++-
 arch/um/include/asm/common.lds.S |  99 ------------------
 arch/um/kernel/dyn.lds.S         | 171 -------------------------------
 arch/um/kernel/uml.lds.S         | 115 ---------------------
 arch/um/kernel/vmlinux.lds.S     |  91 ++++++++++++++--
 scripts/link-vmlinux.sh          |  42 +++-----
 tools/um/Makefile                |  72 +++++++++++++
 tools/um/Targets                 |   4 +
 tools/um/uml/Build               |   0
 10 files changed, 191 insertions(+), 437 deletions(-)
 delete mode 100644 arch/um/include/asm/common.lds.S
 delete mode 100644 arch/um/kernel/dyn.lds.S
 delete mode 100644 arch/um/kernel/uml.lds.S
 create mode 100644 tools/um/Makefile
 create mode 100644 tools/um/Targets
 create mode 100644 tools/um/uml/Build

diff --git a/arch/um/Kconfig b/arch/um/Kconfig
index eb51fec75948..e41d31d0d875 100644
--- a/arch/um/Kconfig
+++ b/arch/um/Kconfig
@@ -20,6 +20,7 @@ config UML
 	select GENERIC_CLOCKEVENTS
 	select HAVE_GCC_PLUGINS
 	select TTY # Needed for line.c
+	select MODULE_REL_CRCS if MODVERSIONS
 
 config MMU
 	bool
@@ -79,17 +80,6 @@ config STATIC_LINK
 	  NOTE: This option is incompatible with some networking features which
 	  depend on features that require being dynamically loaded (like NSS).
 
-config LD_SCRIPT_STATIC
-	bool
-	default y
-	depends on STATIC_LINK
-
-config LD_SCRIPT_DYN
-	bool
-	default y
-	depends on !LD_SCRIPT_STATIC
-	select MODULE_REL_CRCS if MODVERSIONS
-
 config HOSTFS
 	tristate "Host filesystem"
 	help
diff --git a/arch/um/Makefile b/arch/um/Makefile
index 1cea46ff9bb7..09745b40c52d 100644
--- a/arch/um/Makefile
+++ b/arch/um/Makefile
@@ -95,14 +95,25 @@ KBUILD_CPPFLAGS += -I$(srctree)/$(HOST_DIR)/include \
 KERNEL_DEFINES = $(strip -Derrno=kernel_errno -Dsigprocmask=kernel_sigprocmask \
 			 -Dmktime=kernel_mktime $(ARCH_KERNEL_DEFINES))
 KBUILD_CFLAGS += $(KERNEL_DEFINES)
+LDFLAGS_vmlinux += -r
 
-PHONY += linux
-
+INSTALL_PATH=$(objtree)/tools/um
 all: linux
+install: linux.o
+	@echo "  INSTALL	$(INSTALL_PATH)/lib/$<"
+	@mkdir -p $(INSTALL_PATH)/lib/
+	@cp $< $(INSTALL_PATH)/lib/
+
+PHONY += linux.o
+
+linux: linux.o
+	$(Q)$(MAKE) -C $(srctree)/tools/um
+	$(Q)cp $(objtree)/tools/um/uml/linux $(objtree)/
 
-linux: vmlinux
-	@echo '  LINK $@'
-	$(Q)ln -f $< $@
+linux.o: vmlinux
+	@echo '  LINK	$@'
+	$(Q)$(OBJCOPY) -R .eh_frame $< $@
+	$(Q)mkdir -p $(objtree)/tools/um/lib
 
 define archhelp
   echo '* linux		- Binary kernel image (./linux) - for backward'
@@ -144,5 +155,6 @@ MRPROPER_FILES += arch/$(SUBARCH)/include/generated
 archclean:
 	@find . \( -name '*.bb' -o -name '*.bbg' -o -name '*.da' \
 		-o -name '*.gcov' \) -type f -print | xargs rm -f
+	$(Q)$(MAKE) -C $(srctree)/tools/um clean
 
 export HEADER_ARCH SUBARCH USER_CFLAGS CFLAGS_NO_HARDENING OS DEV_NULL_PATH
diff --git a/arch/um/include/asm/common.lds.S b/arch/um/include/asm/common.lds.S
deleted file mode 100644
index eca6c452a41b..000000000000
--- a/arch/um/include/asm/common.lds.S
+++ /dev/null
@@ -1,99 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#include <asm-generic/vmlinux.lds.h>
-
-  .fini      : { *(.fini)    } =0x9090
-  _etext = .;
-  PROVIDE (etext = .);
-
-  . = ALIGN(4096);
-  _sdata = .;
-  PROVIDE (sdata = .);
-
-  RO_DATA(4096)
-
-  .unprotected : { *(.unprotected) }
-  . = ALIGN(4096);
-  PROVIDE (_unprotected_end = .);
-
-  . = ALIGN(4096);
-  EXCEPTION_TABLE(0)
-
-  BUG_TABLE
-
-  .uml.setup.init : {
-	__uml_setup_start = .;
-	*(.uml.setup.init)
-	__uml_setup_end = .;
-  }
-	
-  .uml.help.init : {
-	__uml_help_start = .;
-	*(.uml.help.init)
-	__uml_help_end = .;
-  }
-	
-  .uml.postsetup.init : {
-	__uml_postsetup_start = .;
-	*(.uml.postsetup.init)
-	__uml_postsetup_end = .;
-  }
-	
-  .init.setup : {
-	INIT_SETUP(0)
-  }
-
-  PERCPU_SECTION(32)
-	
-  .initcall.init : {
-	INIT_CALLS
-  }
-
-  .con_initcall.init : {
-	CON_INITCALL
-  }
-
-  .exitcall : {
-	__exitcall_begin = .;
-	*(.exitcall.exit)
-	__exitcall_end = .;
-  }
-
-  .uml.exitcall : {
-	__uml_exitcall_begin = .;
-	*(.uml.exitcall.exit)
-	__uml_exitcall_end = .;
-  }
-
-  . = ALIGN(4);
-  .altinstructions : {
-	__alt_instructions = .;
-	*(.altinstructions)
-	__alt_instructions_end = .;
-  }
-  .altinstr_replacement : { *(.altinstr_replacement) }
-  /* .exit.text is discard at runtime, not link time, to deal with references
-     from .altinstructions and .eh_frame */
-  .exit.text : { EXIT_TEXT }
-  .exit.data : { *(.exit.data) }
-
-  .preinit_array : {
-	__preinit_array_start = .;
-	*(.preinit_array)
-	__preinit_array_end = .;
-  }
-  .init_array : {
-	__init_array_start = .;
-	*(.init_array)
-	__init_array_end = .;
-  }
-  .fini_array : {
-	__fini_array_start = .;
-	*(.fini_array)
-	__fini_array_end = .;
-  }
-
-   . = ALIGN(4096);
-  .init.ramfs : {
-	INIT_RAM_FS
-  }
-
diff --git a/arch/um/kernel/dyn.lds.S b/arch/um/kernel/dyn.lds.S
deleted file mode 100644
index f5001481010c..000000000000
--- a/arch/um/kernel/dyn.lds.S
+++ /dev/null
@@ -1,171 +0,0 @@
-#include <asm/vmlinux.lds.h>
-#include <asm/page.h>
-
-OUTPUT_FORMAT(ELF_FORMAT)
-OUTPUT_ARCH(ELF_ARCH)
-ENTRY(_start)
-jiffies = jiffies_64;
-
-SECTIONS
-{
-  PROVIDE (__executable_start = START);
-  . = START + SIZEOF_HEADERS;
-  .interp         : { *(.interp) }
-  __binary_start = .;
-  . = ALIGN(4096);		/* Init code and data */
-  _text = .;
-  INIT_TEXT_SECTION(PAGE_SIZE)
-
-  . = ALIGN(PAGE_SIZE);
-
-  /* Read-only sections, merged into text segment: */
-  .hash           : { *(.hash) }
-  .gnu.hash       : { *(.gnu.hash) }
-  .dynsym         : { *(.dynsym) }
-  .dynstr         : { *(.dynstr) }
-  .gnu.version    : { *(.gnu.version) }
-  .gnu.version_d  : { *(.gnu.version_d) }
-  .gnu.version_r  : { *(.gnu.version_r) }
-  .rel.init       : { *(.rel.init) }
-  .rela.init      : { *(.rela.init) }
-  .rel.text       : { *(.rel.text .rel.text.* .rel.gnu.linkonce.t.*) }
-  .rela.text      : { *(.rela.text .rela.text.* .rela.gnu.linkonce.t.*) }
-  .rel.fini       : { *(.rel.fini) }
-  .rela.fini      : { *(.rela.fini) }
-  .rel.rodata     : { *(.rel.rodata .rel.rodata.* .rel.gnu.linkonce.r.*) }
-  .rela.rodata    : { *(.rela.rodata .rela.rodata.* .rela.gnu.linkonce.r.*) }
-  .rel.data       : { *(.rel.data .rel.data.* .rel.gnu.linkonce.d.*) }
-  .rela.data      : { *(.rela.data .rela.data.* .rela.gnu.linkonce.d.*) }
-  .rel.tdata	  : { *(.rel.tdata .rel.tdata.* .rel.gnu.linkonce.td.*) }
-  .rela.tdata	  : { *(.rela.tdata .rela.tdata.* .rela.gnu.linkonce.td.*) }
-  .rel.tbss	  : { *(.rel.tbss .rel.tbss.* .rel.gnu.linkonce.tb.*) }
-  .rela.tbss	  : { *(.rela.tbss .rela.tbss.* .rela.gnu.linkonce.tb.*) }
-  .rel.ctors      : { *(.rel.ctors) }
-  .rela.ctors     : { *(.rela.ctors) }
-  .rel.dtors      : { *(.rel.dtors) }
-  .rela.dtors     : { *(.rela.dtors) }
-  .rel.got        : { *(.rel.got) }
-  .rela.got       : { *(.rela.got) }
-  .rel.bss        : { *(.rel.bss .rel.bss.* .rel.gnu.linkonce.b.*) }
-  .rela.bss       : { *(.rela.bss .rela.bss.* .rela.gnu.linkonce.b.*) }
-  .rel.plt : {
-	*(.rel.plt)
-	PROVIDE_HIDDEN(__rel_iplt_start = .);
-	*(.rel.iplt)
-	PROVIDE_HIDDEN(__rel_iplt_end = .);
-  }
-  .rela.plt : {
-	*(.rela.plt)
-	PROVIDE_HIDDEN(__rela_iplt_start = .);
-	*(.rela.iplt)
-	PROVIDE_HIDDEN(__rela_iplt_end = .);
-  }
-  .init           : {
-    KEEP (*(.init))
-  } =0x90909090
-  .plt            : { *(.plt) }
-  .text           : {
-    _stext = .;
-    TEXT_TEXT
-    SCHED_TEXT
-    CPUIDLE_TEXT
-    LOCK_TEXT
-    IRQENTRY_TEXT
-    SOFTIRQENTRY_TEXT
-    *(.fixup)
-    *(.stub .text.* .gnu.linkonce.t.*)
-    /* .gnu.warning sections are handled specially by elf32.em.  */
-    *(.gnu.warning)
-
-    . = ALIGN(PAGE_SIZE);
-  } =0x90909090
-  . = ALIGN(PAGE_SIZE);
-  .syscall_stub : {
-	__syscall_stub_start = .;
-	*(.__syscall_stub*)
-	__syscall_stub_end = .;
-  }
-  .fini           : {
-    KEEP (*(.fini))
-  } =0x90909090
-
-  .kstrtab : { *(.kstrtab) }
-
-  #include <asm/common.lds.S>
-
-  __init_begin = .;
-  init.data : { INIT_DATA }
-  __init_end = .;
-
-  /* Ensure the __preinit_array_start label is properly aligned.  We
-     could instead move the label definition inside the section, but
-     the linker would then create the section even if it turns out to
-     be empty, which isn't pretty.  */
-  . = ALIGN(32 / 8);
-  .preinit_array     : { *(.preinit_array) }
-  .init_array     : { *(.init_array) }
-  .fini_array     : { *(.fini_array) }
-  .data           : {
-    INIT_TASK_DATA(KERNEL_STACK_SIZE)
-    . = ALIGN(KERNEL_STACK_SIZE);
-    *(.data..init_irqstack)
-    DATA_DATA
-    *(.data.* .gnu.linkonce.d.*)
-    SORT(CONSTRUCTORS)
-  }
-  .data1          : { *(.data1) }
-  .tdata	  : { *(.tdata .tdata.* .gnu.linkonce.td.*) }
-  .tbss		  : { *(.tbss .tbss.* .gnu.linkonce.tb.*) *(.tcommon) }
-  .eh_frame       : { KEEP (*(.eh_frame)) }
-  .gcc_except_table   : { *(.gcc_except_table) }
-  .dynamic        : { *(.dynamic) }
-  .ctors          : {
-    /* gcc uses crtbegin.o to find the start of
-       the constructors, so we make sure it is
-       first.  Because this is a wildcard, it
-       doesn't matter if the user does not
-       actually link against crtbegin.o; the
-       linker won't look for a file to match a
-       wildcard.  The wildcard also means that it
-       doesn't matter which directory crtbegin.o
-       is in.  */
-    KEEP (*crtbegin.o(.ctors))
-    /* We don't want to include the .ctor section from
-       from the crtend.o file until after the sorted ctors.
-       The .ctor section from the crtend file contains the
-       end of ctors marker and it must be last */
-    KEEP (*(EXCLUDE_FILE (*crtend.o ) .ctors))
-    KEEP (*(SORT(.ctors.*)))
-    KEEP (*(.ctors))
-  }
-  .dtors          : {
-    KEEP (*crtbegin.o(.dtors))
-    KEEP (*(EXCLUDE_FILE (*crtend.o ) .dtors))
-    KEEP (*(SORT(.dtors.*)))
-    KEEP (*(.dtors))
-  }
-  .jcr            : { KEEP (*(.jcr)) }
-  .got            : { *(.got.plt) *(.got) }
-  _edata = .;
-  PROVIDE (edata = .);
-  .bss            : {
-   __bss_start = .;
-   *(.dynbss)
-   *(.bss .bss.* .gnu.linkonce.b.*)
-   *(COMMON)
-   /* Align here to ensure that the .bss section occupies space up to
-      _end.  Align after .bss to ensure correct alignment even if the
-      .bss section disappears because there are no input sections.  */
-   . = ALIGN(32 / 8);
-  . = ALIGN(32 / 8);
-  }
-   __bss_stop = .;
-  _end = .;
-  PROVIDE (end = .);
-
-  STABS_DEBUG
-
-  DWARF_DEBUG
-
-  DISCARDS
-}
diff --git a/arch/um/kernel/uml.lds.S b/arch/um/kernel/uml.lds.S
deleted file mode 100644
index 3b6dab3d4501..000000000000
--- a/arch/um/kernel/uml.lds.S
+++ /dev/null
@@ -1,115 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#include <asm/vmlinux.lds.h>
-#include <asm/page.h>
-
-OUTPUT_FORMAT(ELF_FORMAT)
-OUTPUT_ARCH(ELF_ARCH)
-ENTRY(_start)
-jiffies = jiffies_64;
-
-SECTIONS
-{
-  /* This must contain the right address - not quite the default ELF one.*/
-  PROVIDE (__executable_start = START);
-  /* Static binaries stick stuff here, like the sigreturn trampoline,
-   * invisibly to objdump.  So, just make __binary_start equal to the very
-   * beginning of the executable, and if there are unmapped pages after this,
-   * they are forever unusable.
-   */
-  __binary_start = START;
-
-  . = START + SIZEOF_HEADERS;
-  . = ALIGN(PAGE_SIZE);
-
-  _text = .;
-  INIT_TEXT_SECTION(0)
-
-  .text      :
-  {
-    _stext = .;
-    TEXT_TEXT
-    SCHED_TEXT
-    CPUIDLE_TEXT
-    LOCK_TEXT
-    IRQENTRY_TEXT
-    SOFTIRQENTRY_TEXT
-    *(.fixup)
-    /* .gnu.warning sections are handled specially by elf32.em.  */
-    *(.gnu.warning)
-    *(.gnu.linkonce.t*)
-  }
-
-  . = ALIGN(PAGE_SIZE);
-  .syscall_stub : {
-	__syscall_stub_start = .;
-	*(.__syscall_stub*)
-	__syscall_stub_end = .;
-  }
-
-  /*
-   * These are needed even in a static link, even if they wind up being empty.
-   * Newer glibc needs these __rel{,a}_iplt_{start,end} symbols.
-   */
-  .rel.plt : {
-	*(.rel.plt)
-	PROVIDE_HIDDEN(__rel_iplt_start = .);
-	*(.rel.iplt)
-	PROVIDE_HIDDEN(__rel_iplt_end = .);
-  }
-  .rela.plt : {
-	*(.rela.plt)
-	PROVIDE_HIDDEN(__rela_iplt_start = .);
-	*(.rela.iplt)
-	PROVIDE_HIDDEN(__rela_iplt_end = .);
-  }
-
-  #include <asm/common.lds.S>
-
-  __init_begin = .;
-  init.data : { INIT_DATA }
-  __init_end = .;
-
-  .data    :
-  {
-    INIT_TASK_DATA(KERNEL_STACK_SIZE)
-    . = ALIGN(KERNEL_STACK_SIZE);
-    *(.data..init_irqstack)
-    DATA_DATA
-    *(.gnu.linkonce.d*)
-    CONSTRUCTORS
-  }
-  .data1   : { *(.data1) }
-  .ctors         :
-  {
-    *(.ctors)
-  }
-  .dtors         :
-  {
-    *(.dtors)
-  }
-
-  .got           : { *(.got.plt) *(.got) }
-  .dynamic       : { *(.dynamic) }
-  .tdata	  : { *(.tdata .tdata.* .gnu.linkonce.td.*) }
-  .tbss		  : { *(.tbss .tbss.* .gnu.linkonce.tb.*) *(.tcommon) }
-  /* We want the small data sections together, so single-instruction offsets
-     can access them all, and initialized data all before uninitialized, so
-     we can shorten the on-disk segment size.  */
-  .sdata     : { *(.sdata) }
-  _edata  =  .;
-  PROVIDE (edata = .);
-  . = ALIGN(PAGE_SIZE);
-  __bss_start = .;
-  PROVIDE(_bss_start = .);
-  SBSS(0)
-  BSS(0)
-   __bss_stop = .;
-  _end = .;
-  PROVIDE (end = .);
-
-  STABS_DEBUG
-
-  DWARF_DEBUG
-
-  DISCARDS
-}
diff --git a/arch/um/kernel/vmlinux.lds.S b/arch/um/kernel/vmlinux.lds.S
index 16e49bfa2b42..aaedd66772fa 100644
--- a/arch/um/kernel/vmlinux.lds.S
+++ b/arch/um/kernel/vmlinux.lds.S
@@ -1,8 +1,87 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <asm/vmlinux.lds.h>
+#include <asm/thread_info.h>
+#include <asm/page.h>
+#include <asm/thread_info.h>
+#include <asm/cache.h>
+#include <linux/export.h>
 
-KERNEL_STACK_SIZE = 4096 * (1 << CONFIG_KERNEL_STACK_ORDER);
+OUTPUT_FORMAT(ELF_FORMAT)
+jiffies = jiffies_64;
 
-#ifdef CONFIG_LD_SCRIPT_STATIC
-#include "uml.lds.S"
-#else
-#include "dyn.lds.S"
-#endif
+
+SECTIONS
+{
+	__init_begin = . ;
+	HEAD_TEXT_SECTION
+	INIT_TEXT_SECTION(PAGE_SIZE)
+	INIT_DATA_SECTION(16)
+	PERCPU_SECTION(L1_CACHE_BYTES)
+	__init_end = . ;
+
+	_stext = . ;
+	_text = . ;
+	.text : ALIGN(THREAD_SIZE)
+	{
+		__binary_start = . ;
+		TEXT_TEXT
+		SCHED_TEXT
+		LOCK_TEXT
+		CPUIDLE_TEXT
+		IRQENTRY_TEXT
+		SOFTIRQENTRY_TEXT
+	}
+	_etext = . ;
+
+	.syscall_stub : ALIGN(PAGE_SIZE)
+	{
+		__syscall_stub_start = .;
+		*(.__syscall_stub*)
+		__syscall_stub_end = .;
+	}
+
+	_sdata = . ;
+	RO_DATA(PAGE_SIZE)
+	RW_DATA(L1_CACHE_BYTES, PAGE_SIZE, THREAD_SIZE)
+	.data : ALIGN(THREAD_SIZE)  { }
+
+	EXCEPTION_TABLE(16)
+
+	.uml.init_irqstack : ALIGN(THREAD_SIZE) {
+		*(.data..init_irqstack)
+	}
+
+	.uml.setup.init : ALIGN(8) {
+		__uml_setup_start = .;
+		*(.uml.setup.init)
+		__uml_setup_end = .;
+	}
+
+	.uml.help.init : ALIGN(8)  {
+		__uml_help_start = .;
+		*(.uml.help.init)
+		__uml_help_end = .;
+	}
+
+	.uml.postsetup.init : ALIGN(8) {
+		__uml_postsetup_start = .;
+		*(.uml.postsetup.init)
+		__uml_postsetup_end = .;
+	}
+
+	.uml.exitcall : ALIGN(8) {
+		__uml_exitcall_begin = .;
+		*(.uml.exitcall.exit)
+		__uml_exitcall_end = .;
+	}
+	_edata = . ;
+
+	BSS_SECTION(0, 0, 0)
+	_end = . ;
+
+	STABS_DEBUG
+
+	DWARF_DEBUG
+
+	DISCARDS
+}
diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
index e6e2d9e5ff48..903fc8053d07 100755
--- a/scripts/link-vmlinux.sh
+++ b/scripts/link-vmlinux.sh
@@ -102,36 +102,18 @@ vmlinux_link()
 		strip_debug=-Wl,--strip-debug
 	fi
 
-	if [ "${SRCARCH}" != "um" ]; then
-		objects="--whole-archive			\
-			${KBUILD_VMLINUX_OBJS}			\
-			--no-whole-archive			\
-			--start-group				\
-			${KBUILD_VMLINUX_LIBS}			\
-			--end-group				\
-			${@}"
-
-		${LD} ${KBUILD_LDFLAGS} ${LDFLAGS_vmlinux}	\
-			${strip_debug#-Wl,}			\
-			-o ${output}				\
-			-T ${lds} ${objects}
-	else
-		objects="-Wl,--whole-archive			\
-			${KBUILD_VMLINUX_OBJS}			\
-			-Wl,--no-whole-archive			\
-			-Wl,--start-group			\
-			${KBUILD_VMLINUX_LIBS}			\
-			-Wl,--end-group				\
-			${@}"
-
-		${CC} ${CFLAGS_vmlinux}				\
-			${strip_debug}				\
-			-o ${output}				\
-			-Wl,-T,${lds}				\
-			${objects}				\
-			-lutil -lrt -lpthread
-		rm -f linux
-	fi
+	objects="--whole-archive			\
+		${KBUILD_VMLINUX_OBJS}			\
+		--no-whole-archive			\
+		--start-group				\
+		${KBUILD_VMLINUX_LIBS}			\
+		--end-group				\
+		${@}"
+
+	${LD} ${KBUILD_LDFLAGS} ${LDFLAGS_vmlinux}	\
+		${strip_debug#-Wl,}			\
+		-o ${output}				\
+		-T ${lds} ${objects}
 }
 
 # generate .BTF typeinfo from DWARF debuginfo
diff --git a/tools/um/Makefile b/tools/um/Makefile
new file mode 100644
index 000000000000..a63466ef7ef5
--- /dev/null
+++ b/tools/um/Makefile
@@ -0,0 +1,72 @@
+# Do not use make's built-in rules
+# (this improves performance and avoids hard-to-debug behaviour);
+# also do not print "Entering directory..." messages from make
+.SUFFIXES:
+MAKEFLAGS += -r --no-print-directory
+
+KCONFIG?=defconfig
+
+ifneq ($(silent),1)
+  ifneq ($(V),1)
+	QUIET_AUTOCONF       = @echo '  AUTOCONF '$@;
+	Q = @
+  endif
+endif
+
+PREFIX   := /usr
+
+ifeq (,$(srctree))
+  srctree := $(patsubst %/,%,$(dir $(shell pwd)))
+  srctree := $(patsubst %/,%,$(dir $(srctree)))
+endif
+export srctree
+
+-include ../scripts/Makefile.include
+
+# OUTPUT fixup should be *after* include ../scripts/Makefile.include
+ifneq ($(OUTPUT),)
+  OUTPUT := $(OUTPUT)/tools/um/
+else
+  OUTPUT := $(CURDIR)/
+endif
+export OUTPUT
+export objtree := $(OUTPUT)/../..
+
+
+all:
+
+export CFLAGS += -I$(OUTPUT)/include -Iinclude -Wall -g -O2 -Wextra -fPIC \
+	 -Wno-unused-parameter \
+	 -Wno-missing-field-initializers -fno-strict-aliasing
+
+-include Targets
+
+TARGETS := $(progs-y:%=$(OUTPUT)%)
+TARGETS += $(libs-y:%=$(OUTPUT)%)
+all: $(TARGETS)
+
+# rule to build linux.o
+$(OUTPUT)lib/linux.o:
+	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) $(KCONFIG)
+	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) install INSTALL_PATH=$(OUTPUT)
+
+$(OUTPUT)liblinux.a: $(OUTPUT)lib/linux.o $(OUTPUT)uml/liblinux-in.o
+	$(QUIET_AR)$(AR) -rc $@ $^
+
+# rule to link programs
+$(OUTPUT)%: $(OUTPUT)%-in.o $(OUTPUT)liblinux.a
+	$(QUIET_LINK)$(CC) $(LDFLAGS) $(LDFLAGS_$(notdir $*)-y) -o $@ $^ $(LDLIBS) $(LDLIBS_$(notdir $*)-y)
+
+$(OUTPUT)%-in.o: FORCE
+	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
+
+clean:
+	$(call QUIET_CLEAN, objects)find $(OUTPUT) -name '*.o' -delete -o -name '\.*.cmd'\
+	 -delete -o -name '\.*.d' -delete
+	$(call QUIET_CLEAN, liblinux.a)$(RM) $(OUTPUT)/liblinux.a
+	$(call QUIET_CLEAN, $(TARGETS))$(RM) $(TARGETS)
+
+FORCE: ;
+.PHONY: all clean FORCE
+.IGNORE: clean
+.NOTPARALLEL : lib/linux.o
diff --git a/tools/um/Targets b/tools/um/Targets
new file mode 100644
index 000000000000..a4711f1ef422
--- /dev/null
+++ b/tools/um/Targets
@@ -0,0 +1,4 @@
+progs-y += uml/linux
+LDLIBS_linux-y := -lrt -lpthread -lutil
+LDFLAGS_linux-y := -no-pie -Wl,--wrap,malloc -Wl,--wrap,free -Wl,--wrap,calloc
+LDFLAGS_linux-$(UML_STATIC) += -static
diff --git a/tools/um/uml/Build b/tools/um/uml/Build
new file mode 100644
index 000000000000..e69de29bb2d1
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v7 01/21] um: split build in kernel and host parts
@ 2020-10-06  9:44           ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-06  9:44 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: linux-arch, tavi.purdila, Octavian Purdila, retrage01,
	linux-kernel-library, Hajime Tazaki

From: Octavian Purdila <tavi@cs.pub.ro>

This patch splits the UML build in two parts: a kernel part that
generates a relocatable object (linux.o) and a host build part that
links the relocatable object into an executable.

This allows us to simplify the linker script since we can now remove
the host bits (e.g. .rel sections). It also gives us greater
flexibility since it will be much easier to support other
architectures and OSes if we use the standard linker script.

The host build part has been implemented in tools/um so that we can
reuse the available host build infrastructure.

The patch preserves the way to invoke the UML build as follows,

 $ make ARCH=um defconfig
 $ make ARCH=um

but it internally calls tools/um/Makefile in order to complete the full
build to link the relocatable object and the rest of files.

To statically compiled UML use:

 $ make ARCH=um UML_STATIC=y

Signed-off-by: Octavian Purdila <tavi@cs.pub.ro>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/Kconfig                  |  12 +--
 arch/um/Makefile                 |  22 +++-
 arch/um/include/asm/common.lds.S |  99 ------------------
 arch/um/kernel/dyn.lds.S         | 171 -------------------------------
 arch/um/kernel/uml.lds.S         | 115 ---------------------
 arch/um/kernel/vmlinux.lds.S     |  91 ++++++++++++++--
 scripts/link-vmlinux.sh          |  42 +++-----
 tools/um/Makefile                |  72 +++++++++++++
 tools/um/Targets                 |   4 +
 tools/um/uml/Build               |   0
 10 files changed, 191 insertions(+), 437 deletions(-)
 delete mode 100644 arch/um/include/asm/common.lds.S
 delete mode 100644 arch/um/kernel/dyn.lds.S
 delete mode 100644 arch/um/kernel/uml.lds.S
 create mode 100644 tools/um/Makefile
 create mode 100644 tools/um/Targets
 create mode 100644 tools/um/uml/Build

diff --git a/arch/um/Kconfig b/arch/um/Kconfig
index eb51fec75948..e41d31d0d875 100644
--- a/arch/um/Kconfig
+++ b/arch/um/Kconfig
@@ -20,6 +20,7 @@ config UML
 	select GENERIC_CLOCKEVENTS
 	select HAVE_GCC_PLUGINS
 	select TTY # Needed for line.c
+	select MODULE_REL_CRCS if MODVERSIONS
 
 config MMU
 	bool
@@ -79,17 +80,6 @@ config STATIC_LINK
 	  NOTE: This option is incompatible with some networking features which
 	  depend on features that require being dynamically loaded (like NSS).
 
-config LD_SCRIPT_STATIC
-	bool
-	default y
-	depends on STATIC_LINK
-
-config LD_SCRIPT_DYN
-	bool
-	default y
-	depends on !LD_SCRIPT_STATIC
-	select MODULE_REL_CRCS if MODVERSIONS
-
 config HOSTFS
 	tristate "Host filesystem"
 	help
diff --git a/arch/um/Makefile b/arch/um/Makefile
index 1cea46ff9bb7..09745b40c52d 100644
--- a/arch/um/Makefile
+++ b/arch/um/Makefile
@@ -95,14 +95,25 @@ KBUILD_CPPFLAGS += -I$(srctree)/$(HOST_DIR)/include \
 KERNEL_DEFINES = $(strip -Derrno=kernel_errno -Dsigprocmask=kernel_sigprocmask \
 			 -Dmktime=kernel_mktime $(ARCH_KERNEL_DEFINES))
 KBUILD_CFLAGS += $(KERNEL_DEFINES)
+LDFLAGS_vmlinux += -r
 
-PHONY += linux
-
+INSTALL_PATH=$(objtree)/tools/um
 all: linux
+install: linux.o
+	@echo "  INSTALL	$(INSTALL_PATH)/lib/$<"
+	@mkdir -p $(INSTALL_PATH)/lib/
+	@cp $< $(INSTALL_PATH)/lib/
+
+PHONY += linux.o
+
+linux: linux.o
+	$(Q)$(MAKE) -C $(srctree)/tools/um
+	$(Q)cp $(objtree)/tools/um/uml/linux $(objtree)/
 
-linux: vmlinux
-	@echo '  LINK $@'
-	$(Q)ln -f $< $@
+linux.o: vmlinux
+	@echo '  LINK	$@'
+	$(Q)$(OBJCOPY) -R .eh_frame $< $@
+	$(Q)mkdir -p $(objtree)/tools/um/lib
 
 define archhelp
   echo '* linux		- Binary kernel image (./linux) - for backward'
@@ -144,5 +155,6 @@ MRPROPER_FILES += arch/$(SUBARCH)/include/generated
 archclean:
 	@find . \( -name '*.bb' -o -name '*.bbg' -o -name '*.da' \
 		-o -name '*.gcov' \) -type f -print | xargs rm -f
+	$(Q)$(MAKE) -C $(srctree)/tools/um clean
 
 export HEADER_ARCH SUBARCH USER_CFLAGS CFLAGS_NO_HARDENING OS DEV_NULL_PATH
diff --git a/arch/um/include/asm/common.lds.S b/arch/um/include/asm/common.lds.S
deleted file mode 100644
index eca6c452a41b..000000000000
--- a/arch/um/include/asm/common.lds.S
+++ /dev/null
@@ -1,99 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#include <asm-generic/vmlinux.lds.h>
-
-  .fini      : { *(.fini)    } =0x9090
-  _etext = .;
-  PROVIDE (etext = .);
-
-  . = ALIGN(4096);
-  _sdata = .;
-  PROVIDE (sdata = .);
-
-  RO_DATA(4096)
-
-  .unprotected : { *(.unprotected) }
-  . = ALIGN(4096);
-  PROVIDE (_unprotected_end = .);
-
-  . = ALIGN(4096);
-  EXCEPTION_TABLE(0)
-
-  BUG_TABLE
-
-  .uml.setup.init : {
-	__uml_setup_start = .;
-	*(.uml.setup.init)
-	__uml_setup_end = .;
-  }
-	
-  .uml.help.init : {
-	__uml_help_start = .;
-	*(.uml.help.init)
-	__uml_help_end = .;
-  }
-	
-  .uml.postsetup.init : {
-	__uml_postsetup_start = .;
-	*(.uml.postsetup.init)
-	__uml_postsetup_end = .;
-  }
-	
-  .init.setup : {
-	INIT_SETUP(0)
-  }
-
-  PERCPU_SECTION(32)
-	
-  .initcall.init : {
-	INIT_CALLS
-  }
-
-  .con_initcall.init : {
-	CON_INITCALL
-  }
-
-  .exitcall : {
-	__exitcall_begin = .;
-	*(.exitcall.exit)
-	__exitcall_end = .;
-  }
-
-  .uml.exitcall : {
-	__uml_exitcall_begin = .;
-	*(.uml.exitcall.exit)
-	__uml_exitcall_end = .;
-  }
-
-  . = ALIGN(4);
-  .altinstructions : {
-	__alt_instructions = .;
-	*(.altinstructions)
-	__alt_instructions_end = .;
-  }
-  .altinstr_replacement : { *(.altinstr_replacement) }
-  /* .exit.text is discard at runtime, not link time, to deal with references
-     from .altinstructions and .eh_frame */
-  .exit.text : { EXIT_TEXT }
-  .exit.data : { *(.exit.data) }
-
-  .preinit_array : {
-	__preinit_array_start = .;
-	*(.preinit_array)
-	__preinit_array_end = .;
-  }
-  .init_array : {
-	__init_array_start = .;
-	*(.init_array)
-	__init_array_end = .;
-  }
-  .fini_array : {
-	__fini_array_start = .;
-	*(.fini_array)
-	__fini_array_end = .;
-  }
-
-   . = ALIGN(4096);
-  .init.ramfs : {
-	INIT_RAM_FS
-  }
-
diff --git a/arch/um/kernel/dyn.lds.S b/arch/um/kernel/dyn.lds.S
deleted file mode 100644
index f5001481010c..000000000000
--- a/arch/um/kernel/dyn.lds.S
+++ /dev/null
@@ -1,171 +0,0 @@
-#include <asm/vmlinux.lds.h>
-#include <asm/page.h>
-
-OUTPUT_FORMAT(ELF_FORMAT)
-OUTPUT_ARCH(ELF_ARCH)
-ENTRY(_start)
-jiffies = jiffies_64;
-
-SECTIONS
-{
-  PROVIDE (__executable_start = START);
-  . = START + SIZEOF_HEADERS;
-  .interp         : { *(.interp) }
-  __binary_start = .;
-  . = ALIGN(4096);		/* Init code and data */
-  _text = .;
-  INIT_TEXT_SECTION(PAGE_SIZE)
-
-  . = ALIGN(PAGE_SIZE);
-
-  /* Read-only sections, merged into text segment: */
-  .hash           : { *(.hash) }
-  .gnu.hash       : { *(.gnu.hash) }
-  .dynsym         : { *(.dynsym) }
-  .dynstr         : { *(.dynstr) }
-  .gnu.version    : { *(.gnu.version) }
-  .gnu.version_d  : { *(.gnu.version_d) }
-  .gnu.version_r  : { *(.gnu.version_r) }
-  .rel.init       : { *(.rel.init) }
-  .rela.init      : { *(.rela.init) }
-  .rel.text       : { *(.rel.text .rel.text.* .rel.gnu.linkonce.t.*) }
-  .rela.text      : { *(.rela.text .rela.text.* .rela.gnu.linkonce.t.*) }
-  .rel.fini       : { *(.rel.fini) }
-  .rela.fini      : { *(.rela.fini) }
-  .rel.rodata     : { *(.rel.rodata .rel.rodata.* .rel.gnu.linkonce.r.*) }
-  .rela.rodata    : { *(.rela.rodata .rela.rodata.* .rela.gnu.linkonce.r.*) }
-  .rel.data       : { *(.rel.data .rel.data.* .rel.gnu.linkonce.d.*) }
-  .rela.data      : { *(.rela.data .rela.data.* .rela.gnu.linkonce.d.*) }
-  .rel.tdata	  : { *(.rel.tdata .rel.tdata.* .rel.gnu.linkonce.td.*) }
-  .rela.tdata	  : { *(.rela.tdata .rela.tdata.* .rela.gnu.linkonce.td.*) }
-  .rel.tbss	  : { *(.rel.tbss .rel.tbss.* .rel.gnu.linkonce.tb.*) }
-  .rela.tbss	  : { *(.rela.tbss .rela.tbss.* .rela.gnu.linkonce.tb.*) }
-  .rel.ctors      : { *(.rel.ctors) }
-  .rela.ctors     : { *(.rela.ctors) }
-  .rel.dtors      : { *(.rel.dtors) }
-  .rela.dtors     : { *(.rela.dtors) }
-  .rel.got        : { *(.rel.got) }
-  .rela.got       : { *(.rela.got) }
-  .rel.bss        : { *(.rel.bss .rel.bss.* .rel.gnu.linkonce.b.*) }
-  .rela.bss       : { *(.rela.bss .rela.bss.* .rela.gnu.linkonce.b.*) }
-  .rel.plt : {
-	*(.rel.plt)
-	PROVIDE_HIDDEN(__rel_iplt_start = .);
-	*(.rel.iplt)
-	PROVIDE_HIDDEN(__rel_iplt_end = .);
-  }
-  .rela.plt : {
-	*(.rela.plt)
-	PROVIDE_HIDDEN(__rela_iplt_start = .);
-	*(.rela.iplt)
-	PROVIDE_HIDDEN(__rela_iplt_end = .);
-  }
-  .init           : {
-    KEEP (*(.init))
-  } =0x90909090
-  .plt            : { *(.plt) }
-  .text           : {
-    _stext = .;
-    TEXT_TEXT
-    SCHED_TEXT
-    CPUIDLE_TEXT
-    LOCK_TEXT
-    IRQENTRY_TEXT
-    SOFTIRQENTRY_TEXT
-    *(.fixup)
-    *(.stub .text.* .gnu.linkonce.t.*)
-    /* .gnu.warning sections are handled specially by elf32.em.  */
-    *(.gnu.warning)
-
-    . = ALIGN(PAGE_SIZE);
-  } =0x90909090
-  . = ALIGN(PAGE_SIZE);
-  .syscall_stub : {
-	__syscall_stub_start = .;
-	*(.__syscall_stub*)
-	__syscall_stub_end = .;
-  }
-  .fini           : {
-    KEEP (*(.fini))
-  } =0x90909090
-
-  .kstrtab : { *(.kstrtab) }
-
-  #include <asm/common.lds.S>
-
-  __init_begin = .;
-  init.data : { INIT_DATA }
-  __init_end = .;
-
-  /* Ensure the __preinit_array_start label is properly aligned.  We
-     could instead move the label definition inside the section, but
-     the linker would then create the section even if it turns out to
-     be empty, which isn't pretty.  */
-  . = ALIGN(32 / 8);
-  .preinit_array     : { *(.preinit_array) }
-  .init_array     : { *(.init_array) }
-  .fini_array     : { *(.fini_array) }
-  .data           : {
-    INIT_TASK_DATA(KERNEL_STACK_SIZE)
-    . = ALIGN(KERNEL_STACK_SIZE);
-    *(.data..init_irqstack)
-    DATA_DATA
-    *(.data.* .gnu.linkonce.d.*)
-    SORT(CONSTRUCTORS)
-  }
-  .data1          : { *(.data1) }
-  .tdata	  : { *(.tdata .tdata.* .gnu.linkonce.td.*) }
-  .tbss		  : { *(.tbss .tbss.* .gnu.linkonce.tb.*) *(.tcommon) }
-  .eh_frame       : { KEEP (*(.eh_frame)) }
-  .gcc_except_table   : { *(.gcc_except_table) }
-  .dynamic        : { *(.dynamic) }
-  .ctors          : {
-    /* gcc uses crtbegin.o to find the start of
-       the constructors, so we make sure it is
-       first.  Because this is a wildcard, it
-       doesn't matter if the user does not
-       actually link against crtbegin.o; the
-       linker won't look for a file to match a
-       wildcard.  The wildcard also means that it
-       doesn't matter which directory crtbegin.o
-       is in.  */
-    KEEP (*crtbegin.o(.ctors))
-    /* We don't want to include the .ctor section from
-       from the crtend.o file until after the sorted ctors.
-       The .ctor section from the crtend file contains the
-       end of ctors marker and it must be last */
-    KEEP (*(EXCLUDE_FILE (*crtend.o ) .ctors))
-    KEEP (*(SORT(.ctors.*)))
-    KEEP (*(.ctors))
-  }
-  .dtors          : {
-    KEEP (*crtbegin.o(.dtors))
-    KEEP (*(EXCLUDE_FILE (*crtend.o ) .dtors))
-    KEEP (*(SORT(.dtors.*)))
-    KEEP (*(.dtors))
-  }
-  .jcr            : { KEEP (*(.jcr)) }
-  .got            : { *(.got.plt) *(.got) }
-  _edata = .;
-  PROVIDE (edata = .);
-  .bss            : {
-   __bss_start = .;
-   *(.dynbss)
-   *(.bss .bss.* .gnu.linkonce.b.*)
-   *(COMMON)
-   /* Align here to ensure that the .bss section occupies space up to
-      _end.  Align after .bss to ensure correct alignment even if the
-      .bss section disappears because there are no input sections.  */
-   . = ALIGN(32 / 8);
-  . = ALIGN(32 / 8);
-  }
-   __bss_stop = .;
-  _end = .;
-  PROVIDE (end = .);
-
-  STABS_DEBUG
-
-  DWARF_DEBUG
-
-  DISCARDS
-}
diff --git a/arch/um/kernel/uml.lds.S b/arch/um/kernel/uml.lds.S
deleted file mode 100644
index 3b6dab3d4501..000000000000
--- a/arch/um/kernel/uml.lds.S
+++ /dev/null
@@ -1,115 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#include <asm/vmlinux.lds.h>
-#include <asm/page.h>
-
-OUTPUT_FORMAT(ELF_FORMAT)
-OUTPUT_ARCH(ELF_ARCH)
-ENTRY(_start)
-jiffies = jiffies_64;
-
-SECTIONS
-{
-  /* This must contain the right address - not quite the default ELF one.*/
-  PROVIDE (__executable_start = START);
-  /* Static binaries stick stuff here, like the sigreturn trampoline,
-   * invisibly to objdump.  So, just make __binary_start equal to the very
-   * beginning of the executable, and if there are unmapped pages after this,
-   * they are forever unusable.
-   */
-  __binary_start = START;
-
-  . = START + SIZEOF_HEADERS;
-  . = ALIGN(PAGE_SIZE);
-
-  _text = .;
-  INIT_TEXT_SECTION(0)
-
-  .text      :
-  {
-    _stext = .;
-    TEXT_TEXT
-    SCHED_TEXT
-    CPUIDLE_TEXT
-    LOCK_TEXT
-    IRQENTRY_TEXT
-    SOFTIRQENTRY_TEXT
-    *(.fixup)
-    /* .gnu.warning sections are handled specially by elf32.em.  */
-    *(.gnu.warning)
-    *(.gnu.linkonce.t*)
-  }
-
-  . = ALIGN(PAGE_SIZE);
-  .syscall_stub : {
-	__syscall_stub_start = .;
-	*(.__syscall_stub*)
-	__syscall_stub_end = .;
-  }
-
-  /*
-   * These are needed even in a static link, even if they wind up being empty.
-   * Newer glibc needs these __rel{,a}_iplt_{start,end} symbols.
-   */
-  .rel.plt : {
-	*(.rel.plt)
-	PROVIDE_HIDDEN(__rel_iplt_start = .);
-	*(.rel.iplt)
-	PROVIDE_HIDDEN(__rel_iplt_end = .);
-  }
-  .rela.plt : {
-	*(.rela.plt)
-	PROVIDE_HIDDEN(__rela_iplt_start = .);
-	*(.rela.iplt)
-	PROVIDE_HIDDEN(__rela_iplt_end = .);
-  }
-
-  #include <asm/common.lds.S>
-
-  __init_begin = .;
-  init.data : { INIT_DATA }
-  __init_end = .;
-
-  .data    :
-  {
-    INIT_TASK_DATA(KERNEL_STACK_SIZE)
-    . = ALIGN(KERNEL_STACK_SIZE);
-    *(.data..init_irqstack)
-    DATA_DATA
-    *(.gnu.linkonce.d*)
-    CONSTRUCTORS
-  }
-  .data1   : { *(.data1) }
-  .ctors         :
-  {
-    *(.ctors)
-  }
-  .dtors         :
-  {
-    *(.dtors)
-  }
-
-  .got           : { *(.got.plt) *(.got) }
-  .dynamic       : { *(.dynamic) }
-  .tdata	  : { *(.tdata .tdata.* .gnu.linkonce.td.*) }
-  .tbss		  : { *(.tbss .tbss.* .gnu.linkonce.tb.*) *(.tcommon) }
-  /* We want the small data sections together, so single-instruction offsets
-     can access them all, and initialized data all before uninitialized, so
-     we can shorten the on-disk segment size.  */
-  .sdata     : { *(.sdata) }
-  _edata  =  .;
-  PROVIDE (edata = .);
-  . = ALIGN(PAGE_SIZE);
-  __bss_start = .;
-  PROVIDE(_bss_start = .);
-  SBSS(0)
-  BSS(0)
-   __bss_stop = .;
-  _end = .;
-  PROVIDE (end = .);
-
-  STABS_DEBUG
-
-  DWARF_DEBUG
-
-  DISCARDS
-}
diff --git a/arch/um/kernel/vmlinux.lds.S b/arch/um/kernel/vmlinux.lds.S
index 16e49bfa2b42..aaedd66772fa 100644
--- a/arch/um/kernel/vmlinux.lds.S
+++ b/arch/um/kernel/vmlinux.lds.S
@@ -1,8 +1,87 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <asm/vmlinux.lds.h>
+#include <asm/thread_info.h>
+#include <asm/page.h>
+#include <asm/thread_info.h>
+#include <asm/cache.h>
+#include <linux/export.h>
 
-KERNEL_STACK_SIZE = 4096 * (1 << CONFIG_KERNEL_STACK_ORDER);
+OUTPUT_FORMAT(ELF_FORMAT)
+jiffies = jiffies_64;
 
-#ifdef CONFIG_LD_SCRIPT_STATIC
-#include "uml.lds.S"
-#else
-#include "dyn.lds.S"
-#endif
+
+SECTIONS
+{
+	__init_begin = . ;
+	HEAD_TEXT_SECTION
+	INIT_TEXT_SECTION(PAGE_SIZE)
+	INIT_DATA_SECTION(16)
+	PERCPU_SECTION(L1_CACHE_BYTES)
+	__init_end = . ;
+
+	_stext = . ;
+	_text = . ;
+	.text : ALIGN(THREAD_SIZE)
+	{
+		__binary_start = . ;
+		TEXT_TEXT
+		SCHED_TEXT
+		LOCK_TEXT
+		CPUIDLE_TEXT
+		IRQENTRY_TEXT
+		SOFTIRQENTRY_TEXT
+	}
+	_etext = . ;
+
+	.syscall_stub : ALIGN(PAGE_SIZE)
+	{
+		__syscall_stub_start = .;
+		*(.__syscall_stub*)
+		__syscall_stub_end = .;
+	}
+
+	_sdata = . ;
+	RO_DATA(PAGE_SIZE)
+	RW_DATA(L1_CACHE_BYTES, PAGE_SIZE, THREAD_SIZE)
+	.data : ALIGN(THREAD_SIZE)  { }
+
+	EXCEPTION_TABLE(16)
+
+	.uml.init_irqstack : ALIGN(THREAD_SIZE) {
+		*(.data..init_irqstack)
+	}
+
+	.uml.setup.init : ALIGN(8) {
+		__uml_setup_start = .;
+		*(.uml.setup.init)
+		__uml_setup_end = .;
+	}
+
+	.uml.help.init : ALIGN(8)  {
+		__uml_help_start = .;
+		*(.uml.help.init)
+		__uml_help_end = .;
+	}
+
+	.uml.postsetup.init : ALIGN(8) {
+		__uml_postsetup_start = .;
+		*(.uml.postsetup.init)
+		__uml_postsetup_end = .;
+	}
+
+	.uml.exitcall : ALIGN(8) {
+		__uml_exitcall_begin = .;
+		*(.uml.exitcall.exit)
+		__uml_exitcall_end = .;
+	}
+	_edata = . ;
+
+	BSS_SECTION(0, 0, 0)
+	_end = . ;
+
+	STABS_DEBUG
+
+	DWARF_DEBUG
+
+	DISCARDS
+}
diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
index e6e2d9e5ff48..903fc8053d07 100755
--- a/scripts/link-vmlinux.sh
+++ b/scripts/link-vmlinux.sh
@@ -102,36 +102,18 @@ vmlinux_link()
 		strip_debug=-Wl,--strip-debug
 	fi
 
-	if [ "${SRCARCH}" != "um" ]; then
-		objects="--whole-archive			\
-			${KBUILD_VMLINUX_OBJS}			\
-			--no-whole-archive			\
-			--start-group				\
-			${KBUILD_VMLINUX_LIBS}			\
-			--end-group				\
-			${@}"
-
-		${LD} ${KBUILD_LDFLAGS} ${LDFLAGS_vmlinux}	\
-			${strip_debug#-Wl,}			\
-			-o ${output}				\
-			-T ${lds} ${objects}
-	else
-		objects="-Wl,--whole-archive			\
-			${KBUILD_VMLINUX_OBJS}			\
-			-Wl,--no-whole-archive			\
-			-Wl,--start-group			\
-			${KBUILD_VMLINUX_LIBS}			\
-			-Wl,--end-group				\
-			${@}"
-
-		${CC} ${CFLAGS_vmlinux}				\
-			${strip_debug}				\
-			-o ${output}				\
-			-Wl,-T,${lds}				\
-			${objects}				\
-			-lutil -lrt -lpthread
-		rm -f linux
-	fi
+	objects="--whole-archive			\
+		${KBUILD_VMLINUX_OBJS}			\
+		--no-whole-archive			\
+		--start-group				\
+		${KBUILD_VMLINUX_LIBS}			\
+		--end-group				\
+		${@}"
+
+	${LD} ${KBUILD_LDFLAGS} ${LDFLAGS_vmlinux}	\
+		${strip_debug#-Wl,}			\
+		-o ${output}				\
+		-T ${lds} ${objects}
 }
 
 # generate .BTF typeinfo from DWARF debuginfo
diff --git a/tools/um/Makefile b/tools/um/Makefile
new file mode 100644
index 000000000000..a63466ef7ef5
--- /dev/null
+++ b/tools/um/Makefile
@@ -0,0 +1,72 @@
+# Do not use make's built-in rules
+# (this improves performance and avoids hard-to-debug behaviour);
+# also do not print "Entering directory..." messages from make
+.SUFFIXES:
+MAKEFLAGS += -r --no-print-directory
+
+KCONFIG?=defconfig
+
+ifneq ($(silent),1)
+  ifneq ($(V),1)
+	QUIET_AUTOCONF       = @echo '  AUTOCONF '$@;
+	Q = @
+  endif
+endif
+
+PREFIX   := /usr
+
+ifeq (,$(srctree))
+  srctree := $(patsubst %/,%,$(dir $(shell pwd)))
+  srctree := $(patsubst %/,%,$(dir $(srctree)))
+endif
+export srctree
+
+-include ../scripts/Makefile.include
+
+# OUTPUT fixup should be *after* include ../scripts/Makefile.include
+ifneq ($(OUTPUT),)
+  OUTPUT := $(OUTPUT)/tools/um/
+else
+  OUTPUT := $(CURDIR)/
+endif
+export OUTPUT
+export objtree := $(OUTPUT)/../..
+
+
+all:
+
+export CFLAGS += -I$(OUTPUT)/include -Iinclude -Wall -g -O2 -Wextra -fPIC \
+	 -Wno-unused-parameter \
+	 -Wno-missing-field-initializers -fno-strict-aliasing
+
+-include Targets
+
+TARGETS := $(progs-y:%=$(OUTPUT)%)
+TARGETS += $(libs-y:%=$(OUTPUT)%)
+all: $(TARGETS)
+
+# rule to build linux.o
+$(OUTPUT)lib/linux.o:
+	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) $(KCONFIG)
+	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) install INSTALL_PATH=$(OUTPUT)
+
+$(OUTPUT)liblinux.a: $(OUTPUT)lib/linux.o $(OUTPUT)uml/liblinux-in.o
+	$(QUIET_AR)$(AR) -rc $@ $^
+
+# rule to link programs
+$(OUTPUT)%: $(OUTPUT)%-in.o $(OUTPUT)liblinux.a
+	$(QUIET_LINK)$(CC) $(LDFLAGS) $(LDFLAGS_$(notdir $*)-y) -o $@ $^ $(LDLIBS) $(LDLIBS_$(notdir $*)-y)
+
+$(OUTPUT)%-in.o: FORCE
+	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
+
+clean:
+	$(call QUIET_CLEAN, objects)find $(OUTPUT) -name '*.o' -delete -o -name '\.*.cmd'\
+	 -delete -o -name '\.*.d' -delete
+	$(call QUIET_CLEAN, liblinux.a)$(RM) $(OUTPUT)/liblinux.a
+	$(call QUIET_CLEAN, $(TARGETS))$(RM) $(TARGETS)
+
+FORCE: ;
+.PHONY: all clean FORCE
+.IGNORE: clean
+.NOTPARALLEL : lib/linux.o
diff --git a/tools/um/Targets b/tools/um/Targets
new file mode 100644
index 000000000000..a4711f1ef422
--- /dev/null
+++ b/tools/um/Targets
@@ -0,0 +1,4 @@
+progs-y += uml/linux
+LDLIBS_linux-y := -lrt -lpthread -lutil
+LDFLAGS_linux-y := -no-pie -Wl,--wrap,malloc -Wl,--wrap,free -Wl,--wrap,calloc
+LDFLAGS_linux-$(UML_STATIC) += -static
diff --git a/tools/um/uml/Build b/tools/um/uml/Build
new file mode 100644
index 000000000000..e69de29bb2d1
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v7 02/21] um: add os init and exit calls
  2020-10-06  9:44         ` Hajime Tazaki
@ 2020-10-06  9:44           ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-06  9:44 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch,
	Octavian Purdila

From: Octavian Purdila <tavi@cs.pub.ro>

These calls are added in preparation for moving some of the code from
the kernel to the host build.

Signed-off-by: Octavian Purdila <tavi@cs.pub.ro>
---
 arch/um/include/shared/init.h | 14 ++++----------
 arch/um/kernel/reboot.c       |  5 +++++
 arch/um/kernel/um_arch.c      | 11 +++++++++++
 3 files changed, 20 insertions(+), 10 deletions(-)

diff --git a/arch/um/include/shared/init.h b/arch/um/include/shared/init.h
index c66de434a983..d09308330ca5 100644
--- a/arch/um/include/shared/init.h
+++ b/arch/um/include/shared/init.h
@@ -109,19 +109,13 @@ extern struct uml_param __uml_setup_start, __uml_setup_end;
 
 #ifdef __UM_HOST__
 
-#define __define_initcall(level,fn) \
-	static initcall_t __initcall_##fn __used \
-	__attribute__((__section__(".initcall" level ".init"))) = fn
-
-/* Userspace initcalls shouldn't depend on anything in the kernel, so we'll
- * make them run first.
- */
-#define __initcall(fn) __define_initcall("1", fn)
+#undef __uml_exit_call
+#define __uml_exit_call		__used __section(os_exitcalls)
+#define __init_call		__used __section(os_initcalls)
 
+#define __initcall(fn) static initcall_t __initcall_##fn __init_call = fn
 #define __exitcall(fn) static exitcall_t __exitcall_##fn __exit_call = fn
 
-#define __init_call	__used __section(.initcall.init)
-
 #endif
 
 #endif /* _LINUX_UML_INIT_H */
diff --git a/arch/um/kernel/reboot.c b/arch/um/kernel/reboot.c
index 48c0610d506e..5420aec411f4 100644
--- a/arch/um/kernel/reboot.c
+++ b/arch/um/kernel/reboot.c
@@ -35,10 +35,15 @@ static void kill_off_processes(void)
 	read_unlock(&tasklist_lock);
 }
 
+void __weak os_exitcalls(void)
+{
+}
+
 void uml_cleanup(void)
 {
 	kmalloc_ok = 0;
 	do_uml_exitcalls();
+	os_exitcalls();
 	kill_off_processes();
 }
 
diff --git a/arch/um/kernel/um_arch.c b/arch/um/kernel/um_arch.c
index 00141e70de56..e2cb76c03b25 100644
--- a/arch/um/kernel/um_arch.c
+++ b/arch/um/kernel/um_arch.c
@@ -377,3 +377,14 @@ void *text_poke(void *addr, const void *opcode, size_t len)
 void text_poke_sync(void)
 {
 }
+
+int __weak os_initcalls(void)
+{
+	return 0;
+}
+
+int __init run_os_initcalls(void)
+{
+	return os_initcalls();
+}
+early_initcall(run_os_initcalls);
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v7 02/21] um: add os init and exit calls
@ 2020-10-06  9:44           ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-06  9:44 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Octavian Purdila,
	retrage01

From: Octavian Purdila <tavi@cs.pub.ro>

These calls are added in preparation for moving some of the code from
the kernel to the host build.

Signed-off-by: Octavian Purdila <tavi@cs.pub.ro>
---
 arch/um/include/shared/init.h | 14 ++++----------
 arch/um/kernel/reboot.c       |  5 +++++
 arch/um/kernel/um_arch.c      | 11 +++++++++++
 3 files changed, 20 insertions(+), 10 deletions(-)

diff --git a/arch/um/include/shared/init.h b/arch/um/include/shared/init.h
index c66de434a983..d09308330ca5 100644
--- a/arch/um/include/shared/init.h
+++ b/arch/um/include/shared/init.h
@@ -109,19 +109,13 @@ extern struct uml_param __uml_setup_start, __uml_setup_end;
 
 #ifdef __UM_HOST__
 
-#define __define_initcall(level,fn) \
-	static initcall_t __initcall_##fn __used \
-	__attribute__((__section__(".initcall" level ".init"))) = fn
-
-/* Userspace initcalls shouldn't depend on anything in the kernel, so we'll
- * make them run first.
- */
-#define __initcall(fn) __define_initcall("1", fn)
+#undef __uml_exit_call
+#define __uml_exit_call		__used __section(os_exitcalls)
+#define __init_call		__used __section(os_initcalls)
 
+#define __initcall(fn) static initcall_t __initcall_##fn __init_call = fn
 #define __exitcall(fn) static exitcall_t __exitcall_##fn __exit_call = fn
 
-#define __init_call	__used __section(.initcall.init)
-
 #endif
 
 #endif /* _LINUX_UML_INIT_H */
diff --git a/arch/um/kernel/reboot.c b/arch/um/kernel/reboot.c
index 48c0610d506e..5420aec411f4 100644
--- a/arch/um/kernel/reboot.c
+++ b/arch/um/kernel/reboot.c
@@ -35,10 +35,15 @@ static void kill_off_processes(void)
 	read_unlock(&tasklist_lock);
 }
 
+void __weak os_exitcalls(void)
+{
+}
+
 void uml_cleanup(void)
 {
 	kmalloc_ok = 0;
 	do_uml_exitcalls();
+	os_exitcalls();
 	kill_off_processes();
 }
 
diff --git a/arch/um/kernel/um_arch.c b/arch/um/kernel/um_arch.c
index 00141e70de56..e2cb76c03b25 100644
--- a/arch/um/kernel/um_arch.c
+++ b/arch/um/kernel/um_arch.c
@@ -377,3 +377,14 @@ void *text_poke(void *addr, const void *opcode, size_t len)
 void text_poke_sync(void)
 {
 }
+
+int __weak os_initcalls(void)
+{
+	return 0;
+}
+
+int __init run_os_initcalls(void)
+{
+	return os_initcalls();
+}
+early_initcall(run_os_initcalls);
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v7 03/21] um: move arch/um/os-Linux dir to tools/um/uml
  2020-10-06  9:44         ` Hajime Tazaki
@ 2020-10-06  9:44           ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-06  9:44 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch, Hajime Tazaki

This patch moves underlying OS dependent part under arch/um to tools/um
directory so that arch/um code does not need to build host build
facilities (e.g., libc).

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/Makefile                              |  3 +-
 arch/um/drivers/Makefile                      |  2 +
 .../um/{os-Linux => }/drivers/ethertap_kern.c |  0
 arch/um/{os-Linux => }/drivers/tuntap_kern.c  |  0
 .../drivers => include/shared}/etap.h         |  0
 arch/um/include/shared/init.h                 |  5 ++
 .../drivers => include/shared}/tuntap.h       |  0
 arch/um/kernel/Makefile                       |  1 +
 arch/um/{os-Linux => kernel}/user_syms.c      |  0
 arch/um/os-Linux/Makefile                     | 19 --------
 arch/um/os-Linux/drivers/Makefile             | 13 -----
 tools/um/Makefile                             |  6 ++-
 tools/um/uml/Build                            | 48 +++++++++++++++++++
 tools/um/uml/drivers/Build                    | 10 ++++
 .../um/uml}/drivers/ethertap_user.c           |  0
 .../um/uml}/drivers/tuntap_user.c             |  0
 {arch/um/os-Linux => tools/um/uml}/elf_aux.c  |  0
 {arch/um/os-Linux => tools/um/uml}/execvp.c   |  4 --
 {arch/um/os-Linux => tools/um/uml}/file.c     |  0
 {arch/um/os-Linux => tools/um/uml}/helper.c   |  0
 {arch/um/os-Linux => tools/um/uml}/irq.c      |  0
 {arch/um/os-Linux => tools/um/uml}/main.c     |  0
 {arch/um/os-Linux => tools/um/uml}/mem.c      |  0
 {arch/um/os-Linux => tools/um/uml}/process.c  |  0
 .../um/os-Linux => tools/um/uml}/registers.c  |  0
 {arch/um/os-Linux => tools/um/uml}/sigio.c    |  0
 {arch/um/os-Linux => tools/um/uml}/signal.c   |  0
 .../skas/Makefile => tools/um/uml/skas/Build  |  6 +--
 {arch/um/os-Linux => tools/um/uml}/skas/mem.c |  0
 .../os-Linux => tools/um/uml}/skas/process.c  |  3 +-
 {arch/um/os-Linux => tools/um/uml}/start_up.c |  0
 {arch/um/os-Linux => tools/um/uml}/time.c     |  0
 {arch/um/os-Linux => tools/um/uml}/tty.c      |  0
 {arch/um/os-Linux => tools/um/uml}/umid.c     |  0
 {arch/um/os-Linux => tools/um/uml}/util.c     |  0
 35 files changed, 74 insertions(+), 46 deletions(-)
 rename arch/um/{os-Linux => }/drivers/ethertap_kern.c (100%)
 rename arch/um/{os-Linux => }/drivers/tuntap_kern.c (100%)
 rename arch/um/{os-Linux/drivers => include/shared}/etap.h (100%)
 rename arch/um/{os-Linux/drivers => include/shared}/tuntap.h (100%)
 rename arch/um/{os-Linux => kernel}/user_syms.c (100%)
 delete mode 100644 arch/um/os-Linux/Makefile
 delete mode 100644 arch/um/os-Linux/drivers/Makefile
 create mode 100644 tools/um/uml/drivers/Build
 rename {arch/um/os-Linux => tools/um/uml}/drivers/ethertap_user.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/drivers/tuntap_user.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/elf_aux.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/execvp.c (98%)
 rename {arch/um/os-Linux => tools/um/uml}/file.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/helper.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/irq.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/main.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/mem.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/process.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/registers.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/sigio.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/signal.c (100%)
 rename arch/um/os-Linux/skas/Makefile => tools/um/uml/skas/Build (56%)
 rename {arch/um/os-Linux => tools/um/uml}/skas/mem.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/skas/process.c (99%)
 rename {arch/um/os-Linux => tools/um/uml}/start_up.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/time.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/tty.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/umid.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/util.c (100%)

diff --git a/arch/um/Makefile b/arch/um/Makefile
index 09745b40c52d..8be7bc479442 100644
--- a/arch/um/Makefile
+++ b/arch/um/Makefile
@@ -24,8 +24,7 @@ OS := $(shell uname -s)
 SHELL := /bin/bash
 
 core-y			+= $(ARCH_DIR)/kernel/		\
-			   $(ARCH_DIR)/drivers/		\
-			   $(ARCH_DIR)/os-$(OS)/
+			   $(ARCH_DIR)/drivers/
 
 MODE_INCLUDE	+= -I$(srctree)/$(ARCH_DIR)/include/shared/skas
 
diff --git a/arch/um/drivers/Makefile b/arch/um/drivers/Makefile
index 2a249f619467..ae96c83e312d 100644
--- a/arch/um/drivers/Makefile
+++ b/arch/um/drivers/Makefile
@@ -48,6 +48,8 @@ obj-$(CONFIG_UML_NET_VECTOR) += vector.o
 obj-$(CONFIG_UML_NET_VDE) += vde.o
 obj-$(CONFIG_UML_NET_MCAST) += umcast.o
 obj-$(CONFIG_UML_NET_PCAP) += pcap.o
+obj-$(CONFIG_UML_NET_ETHERTAP) += ethertap_kern.o
+obj-$(CONFIG_UML_NET_TUNTAP) += tuntap_kern.o
 obj-$(CONFIG_UML_NET) += net.o 
 obj-$(CONFIG_MCONSOLE) += mconsole.o
 obj-$(CONFIG_MMAPPER) += mmapper_kern.o 
diff --git a/arch/um/os-Linux/drivers/ethertap_kern.c b/arch/um/drivers/ethertap_kern.c
similarity index 100%
rename from arch/um/os-Linux/drivers/ethertap_kern.c
rename to arch/um/drivers/ethertap_kern.c
diff --git a/arch/um/os-Linux/drivers/tuntap_kern.c b/arch/um/drivers/tuntap_kern.c
similarity index 100%
rename from arch/um/os-Linux/drivers/tuntap_kern.c
rename to arch/um/drivers/tuntap_kern.c
diff --git a/arch/um/os-Linux/drivers/etap.h b/arch/um/include/shared/etap.h
similarity index 100%
rename from arch/um/os-Linux/drivers/etap.h
rename to arch/um/include/shared/etap.h
diff --git a/arch/um/include/shared/init.h b/arch/um/include/shared/init.h
index d09308330ca5..305789667584 100644
--- a/arch/um/include/shared/init.h
+++ b/arch/um/include/shared/init.h
@@ -41,7 +41,12 @@
 typedef int (*initcall_t)(void);
 typedef void (*exitcall_t)(void);
 
+#ifndef __UM_HOST__
 #include <linux/compiler_types.h>
+#else
+#define __used                          __attribute__((__used__))
+#define __section(S)                    __attribute__((__section__(#S)))
+#endif
 
 /* These are for everybody (although not all archs will actually
    discard it in modules) */
diff --git a/arch/um/os-Linux/drivers/tuntap.h b/arch/um/include/shared/tuntap.h
similarity index 100%
rename from arch/um/os-Linux/drivers/tuntap.h
rename to arch/um/include/shared/tuntap.h
diff --git a/arch/um/kernel/Makefile b/arch/um/kernel/Makefile
index 5aa882011e04..9b63831a69e1 100644
--- a/arch/um/kernel/Makefile
+++ b/arch/um/kernel/Makefile
@@ -24,6 +24,7 @@ obj-$(CONFIG_GPROF)	+= gprof_syms.o
 obj-$(CONFIG_GCOV)	+= gmon_syms.o
 obj-$(CONFIG_EARLY_PRINTK) += early_printk.o
 obj-$(CONFIG_STACKTRACE) += stacktrace.o
+obj-y += user_syms.o
 
 USER_OBJS := config.o
 
diff --git a/arch/um/os-Linux/user_syms.c b/arch/um/kernel/user_syms.c
similarity index 100%
rename from arch/um/os-Linux/user_syms.c
rename to arch/um/kernel/user_syms.c
diff --git a/arch/um/os-Linux/Makefile b/arch/um/os-Linux/Makefile
deleted file mode 100644
index 839915b8c31c..000000000000
--- a/arch/um/os-Linux/Makefile
+++ /dev/null
@@ -1,19 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0
-# 
-# Copyright (C) 2000 - 2007 Jeff Dike (jdike@{addtoit,linux.intel}.com)
-#
-
-# Don't instrument UML-specific code
-KCOV_INSTRUMENT                := n
-
-obj-y = execvp.o file.o helper.o irq.o main.o mem.o process.o \
-	registers.o sigio.o signal.o start_up.o time.o tty.o \
-	umid.o user_syms.o util.o drivers/ skas/
-
-obj-$(CONFIG_ARCH_REUSE_HOST_VSYSCALL_AREA) += elf_aux.o
-
-USER_OBJS := $(user-objs-y) elf_aux.o execvp.o file.o helper.o irq.o \
-	main.o mem.o process.o registers.o sigio.o signal.o start_up.o time.o \
-	tty.o umid.o util.o
-
-include arch/um/scripts/Makefile.rules
diff --git a/arch/um/os-Linux/drivers/Makefile b/arch/um/os-Linux/drivers/Makefile
deleted file mode 100644
index d79e75f1b69a..000000000000
--- a/arch/um/os-Linux/drivers/Makefile
+++ /dev/null
@@ -1,13 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0
-# 
-# Copyright (C) 2000, 2002 Jeff Dike (jdike@karaya.com)
-#
-
-ethertap-objs := ethertap_kern.o ethertap_user.o
-tuntap-objs := tuntap_kern.o tuntap_user.o
-
-obj-y = 
-obj-$(CONFIG_UML_NET_ETHERTAP) += ethertap.o
-obj-$(CONFIG_UML_NET_TUNTAP) += tuntap.o
-
-include arch/um/scripts/Makefile.rules
diff --git a/tools/um/Makefile b/tools/um/Makefile
index a63466ef7ef5..07b9b93ed817 100644
--- a/tools/um/Makefile
+++ b/tools/um/Makefile
@@ -50,7 +50,7 @@ $(OUTPUT)lib/linux.o:
 	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) $(KCONFIG)
 	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) install INSTALL_PATH=$(OUTPUT)
 
-$(OUTPUT)liblinux.a: $(OUTPUT)lib/linux.o $(OUTPUT)uml/liblinux-in.o
+$(OUTPUT)liblinux.a: $(OUTPUT)lib/linux.o $(OUTPUT)uml/liblinux-in.o $(OUTPUT)lib/liblinux-in.o
 	$(QUIET_AR)$(AR) -rc $@ $^
 
 # rule to link programs
@@ -60,6 +60,10 @@ $(OUTPUT)%: $(OUTPUT)%-in.o $(OUTPUT)liblinux.a
 $(OUTPUT)%-in.o: FORCE
 	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
 
+# rule to build objects
+$(OUTPUT)%-in.o: $(OUTPUT)lib/linux.o
+	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
+
 clean:
 	$(call QUIET_CLEAN, objects)find $(OUTPUT) -name '*.o' -delete -o -name '\.*.cmd'\
 	 -delete -o -name '\.*.d' -delete
diff --git a/tools/um/uml/Build b/tools/um/uml/Build
index e69de29bb2d1..ad93c6ad51fc 100644
--- a/tools/um/uml/Build
+++ b/tools/um/uml/Build
@@ -0,0 +1,48 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Copyright (C) 2000 - 2007 Jeff Dike (jdike@{addtoit,linux.intel}.com)
+#
+
+# Don't instrument UML-specific code
+KCOV_INSTRUMENT                := n
+
+include $(objtree)/include/config/auto.conf
+
+liblinux-y += execvp.o file.o helper.o irq.o main.o mem.o process.o \
+	registers.o sigio.o signal.o start_up.o time.o tty.o \
+	umid.o util.o drivers/ skas/
+
+liblinux-$(CONFIG_ARCH_REUSE_HOST_VSYSCALL_AREA) += elf_aux.o
+
+export O = $(srctree)
+objtree := $(O)
+# reset CFLAGS
+CFLAGS := -g -O2
+
+# from arch/um/Makefile
+ARCH_DIR := arch/um
+HEADER_ARCH 	:= x86
+HOST_DIR := arch/$(HEADER_ARCH)
+ifdef CONFIG_64BIT
+  KBUILD_CFLAGS += -mcmodel=large
+endif
+KBUILD_CFLAGS += $(CFLAGS) $(CFLAGS-y) -D__arch_um__ \
+	$(ARCH_INCLUDE) $(MODE_INCLUDE) -Dvmap=kernel_vmap	\
+	-Dlongjmp=kernel_longjmp -Dsetjmp=kernel_setjmp \
+	-Din6addr_loopback=kernel_in6addr_loopback \
+	-Din6addr_any=kernel_in6addr_any -Dstrrchr=kernel_strrchr
+SHARED_HEADERS	:= $(ARCH_DIR)/include/shared
+MODE_INCLUDE	+= -I$(srctree)/$(ARCH_DIR)/include/shared/skas
+ARCH_INCLUDE	:= -I$(srctree)/$(SHARED_HEADERS)
+ARCH_INCLUDE	+= -I$(srctree)/$(HOST_DIR)/um/shared
+KBUILD_CPPFLAGS += -I$(srctree)/$(HOST_DIR)/um
+USER_CFLAGS := $(patsubst $(KERNEL_DEFINES),,$(patsubst -I%,,$(KBUILD_CFLAGS))) \
+		$(ARCH_INCLUDE) $(MODE_INCLUDE) $(filter -I%,$(CFLAGS)) \
+		-D_FILE_OFFSET_BITS=64 -idirafter $(srctree)/include/uapi \
+		-idirafter $(objtree)/include -D__KERNEL__ -D__UM_HOST__
+
+# from Makefile-os-Linux
+USER_CFLAGS += -D_GNU_SOURCE -D_LARGEFILE64_SOURCE
+
+# from Makefile.rules
+CFLAGS += $(USER_CFLAGS) -include $(srctree)/tools/include/linux/kern_levels.h -include user.h $(CFLAGS_$(basetarget).o)
diff --git a/tools/um/uml/drivers/Build b/tools/um/uml/drivers/Build
new file mode 100644
index 000000000000..c7a8eaa97d72
--- /dev/null
+++ b/tools/um/uml/drivers/Build
@@ -0,0 +1,10 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Copyright (C) 2000, 2002 Jeff Dike (jdike@karaya.com)
+#
+
+include $(objtree)/include/config/auto.conf
+
+
+liblinux-$(CONFIG_UML_NET_ETHERTAP) += ethertap_user.o
+liblinux-$(CONFIG_UML_NET_TUNTAP) += tuntap_user.o
diff --git a/arch/um/os-Linux/drivers/ethertap_user.c b/tools/um/uml/drivers/ethertap_user.c
similarity index 100%
rename from arch/um/os-Linux/drivers/ethertap_user.c
rename to tools/um/uml/drivers/ethertap_user.c
diff --git a/arch/um/os-Linux/drivers/tuntap_user.c b/tools/um/uml/drivers/tuntap_user.c
similarity index 100%
rename from arch/um/os-Linux/drivers/tuntap_user.c
rename to tools/um/uml/drivers/tuntap_user.c
diff --git a/arch/um/os-Linux/elf_aux.c b/tools/um/uml/elf_aux.c
similarity index 100%
rename from arch/um/os-Linux/elf_aux.c
rename to tools/um/uml/elf_aux.c
diff --git a/arch/um/os-Linux/execvp.c b/tools/um/uml/execvp.c
similarity index 98%
rename from arch/um/os-Linux/execvp.c
rename to tools/um/uml/execvp.c
index 84a0777c2a45..c105e0e7b778 100644
--- a/arch/um/os-Linux/execvp.c
+++ b/tools/um/uml/execvp.c
@@ -26,12 +26,8 @@
 #include <errno.h>
 #include <limits.h>
 
-#ifndef TEST
-#include <um_malloc.h>
-#else
 #include <stdio.h>
 #define um_kmalloc malloc
-#endif
 #include <os.h>
 
 /* Execute FILE, searching in the `PATH' environment variable if it contains
diff --git a/arch/um/os-Linux/file.c b/tools/um/uml/file.c
similarity index 100%
rename from arch/um/os-Linux/file.c
rename to tools/um/uml/file.c
diff --git a/arch/um/os-Linux/helper.c b/tools/um/uml/helper.c
similarity index 100%
rename from arch/um/os-Linux/helper.c
rename to tools/um/uml/helper.c
diff --git a/arch/um/os-Linux/irq.c b/tools/um/uml/irq.c
similarity index 100%
rename from arch/um/os-Linux/irq.c
rename to tools/um/uml/irq.c
diff --git a/arch/um/os-Linux/main.c b/tools/um/uml/main.c
similarity index 100%
rename from arch/um/os-Linux/main.c
rename to tools/um/uml/main.c
diff --git a/arch/um/os-Linux/mem.c b/tools/um/uml/mem.c
similarity index 100%
rename from arch/um/os-Linux/mem.c
rename to tools/um/uml/mem.c
diff --git a/arch/um/os-Linux/process.c b/tools/um/uml/process.c
similarity index 100%
rename from arch/um/os-Linux/process.c
rename to tools/um/uml/process.c
diff --git a/arch/um/os-Linux/registers.c b/tools/um/uml/registers.c
similarity index 100%
rename from arch/um/os-Linux/registers.c
rename to tools/um/uml/registers.c
diff --git a/arch/um/os-Linux/sigio.c b/tools/um/uml/sigio.c
similarity index 100%
rename from arch/um/os-Linux/sigio.c
rename to tools/um/uml/sigio.c
diff --git a/arch/um/os-Linux/signal.c b/tools/um/uml/signal.c
similarity index 100%
rename from arch/um/os-Linux/signal.c
rename to tools/um/uml/signal.c
diff --git a/arch/um/os-Linux/skas/Makefile b/tools/um/uml/skas/Build
similarity index 56%
rename from arch/um/os-Linux/skas/Makefile
rename to tools/um/uml/skas/Build
index c4566e788815..5fc9d62df863 100644
--- a/arch/um/os-Linux/skas/Makefile
+++ b/tools/um/uml/skas/Build
@@ -3,8 +3,4 @@
 # Copyright (C) 2002 - 2007 Jeff Dike (jdike@{linux.intel,addtoit}.com)
 #
 
-obj-y := mem.o process.o
-
-USER_OBJS := $(obj-y)
-
-include arch/um/scripts/Makefile.rules
+liblinux-y += mem.o process.o
diff --git a/arch/um/os-Linux/skas/mem.c b/tools/um/uml/skas/mem.c
similarity index 100%
rename from arch/um/os-Linux/skas/mem.c
rename to tools/um/uml/skas/mem.c
diff --git a/arch/um/os-Linux/skas/process.c b/tools/um/uml/skas/process.c
similarity index 99%
rename from arch/um/os-Linux/skas/process.c
rename to tools/um/uml/skas/process.c
index 4fb877b99dde..fca6a08e81bd 100644
--- a/arch/um/os-Linux/skas/process.c
+++ b/tools/um/uml/skas/process.c
@@ -21,7 +21,6 @@
 #include <registers.h>
 #include <skas.h>
 #include <sysdep/stub.h>
-#include <linux/threads.h>
 
 int is_skas_winch(int pid, int fd, void *data)
 {
@@ -248,7 +247,7 @@ static int userspace_tramp(void *stack)
 	return 0;
 }
 
-int userspace_pid[NR_CPUS];
+int userspace_pid[UM_NR_CPUS];
 
 /**
  * start_userspace() - prepare a new userspace process
diff --git a/arch/um/os-Linux/start_up.c b/tools/um/uml/start_up.c
similarity index 100%
rename from arch/um/os-Linux/start_up.c
rename to tools/um/uml/start_up.c
diff --git a/arch/um/os-Linux/time.c b/tools/um/uml/time.c
similarity index 100%
rename from arch/um/os-Linux/time.c
rename to tools/um/uml/time.c
diff --git a/arch/um/os-Linux/tty.c b/tools/um/uml/tty.c
similarity index 100%
rename from arch/um/os-Linux/tty.c
rename to tools/um/uml/tty.c
diff --git a/arch/um/os-Linux/umid.c b/tools/um/uml/umid.c
similarity index 100%
rename from arch/um/os-Linux/umid.c
rename to tools/um/uml/umid.c
diff --git a/arch/um/os-Linux/util.c b/tools/um/uml/util.c
similarity index 100%
rename from arch/um/os-Linux/util.c
rename to tools/um/uml/util.c
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v7 03/21] um: move arch/um/os-Linux dir to tools/um/uml
@ 2020-10-06  9:44           ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-06  9:44 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Hajime Tazaki, retrage01

This patch moves underlying OS dependent part under arch/um to tools/um
directory so that arch/um code does not need to build host build
facilities (e.g., libc).

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/Makefile                              |  3 +-
 arch/um/drivers/Makefile                      |  2 +
 .../um/{os-Linux => }/drivers/ethertap_kern.c |  0
 arch/um/{os-Linux => }/drivers/tuntap_kern.c  |  0
 .../drivers => include/shared}/etap.h         |  0
 arch/um/include/shared/init.h                 |  5 ++
 .../drivers => include/shared}/tuntap.h       |  0
 arch/um/kernel/Makefile                       |  1 +
 arch/um/{os-Linux => kernel}/user_syms.c      |  0
 arch/um/os-Linux/Makefile                     | 19 --------
 arch/um/os-Linux/drivers/Makefile             | 13 -----
 tools/um/Makefile                             |  6 ++-
 tools/um/uml/Build                            | 48 +++++++++++++++++++
 tools/um/uml/drivers/Build                    | 10 ++++
 .../um/uml}/drivers/ethertap_user.c           |  0
 .../um/uml}/drivers/tuntap_user.c             |  0
 {arch/um/os-Linux => tools/um/uml}/elf_aux.c  |  0
 {arch/um/os-Linux => tools/um/uml}/execvp.c   |  4 --
 {arch/um/os-Linux => tools/um/uml}/file.c     |  0
 {arch/um/os-Linux => tools/um/uml}/helper.c   |  0
 {arch/um/os-Linux => tools/um/uml}/irq.c      |  0
 {arch/um/os-Linux => tools/um/uml}/main.c     |  0
 {arch/um/os-Linux => tools/um/uml}/mem.c      |  0
 {arch/um/os-Linux => tools/um/uml}/process.c  |  0
 .../um/os-Linux => tools/um/uml}/registers.c  |  0
 {arch/um/os-Linux => tools/um/uml}/sigio.c    |  0
 {arch/um/os-Linux => tools/um/uml}/signal.c   |  0
 .../skas/Makefile => tools/um/uml/skas/Build  |  6 +--
 {arch/um/os-Linux => tools/um/uml}/skas/mem.c |  0
 .../os-Linux => tools/um/uml}/skas/process.c  |  3 +-
 {arch/um/os-Linux => tools/um/uml}/start_up.c |  0
 {arch/um/os-Linux => tools/um/uml}/time.c     |  0
 {arch/um/os-Linux => tools/um/uml}/tty.c      |  0
 {arch/um/os-Linux => tools/um/uml}/umid.c     |  0
 {arch/um/os-Linux => tools/um/uml}/util.c     |  0
 35 files changed, 74 insertions(+), 46 deletions(-)
 rename arch/um/{os-Linux => }/drivers/ethertap_kern.c (100%)
 rename arch/um/{os-Linux => }/drivers/tuntap_kern.c (100%)
 rename arch/um/{os-Linux/drivers => include/shared}/etap.h (100%)
 rename arch/um/{os-Linux/drivers => include/shared}/tuntap.h (100%)
 rename arch/um/{os-Linux => kernel}/user_syms.c (100%)
 delete mode 100644 arch/um/os-Linux/Makefile
 delete mode 100644 arch/um/os-Linux/drivers/Makefile
 create mode 100644 tools/um/uml/drivers/Build
 rename {arch/um/os-Linux => tools/um/uml}/drivers/ethertap_user.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/drivers/tuntap_user.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/elf_aux.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/execvp.c (98%)
 rename {arch/um/os-Linux => tools/um/uml}/file.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/helper.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/irq.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/main.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/mem.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/process.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/registers.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/sigio.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/signal.c (100%)
 rename arch/um/os-Linux/skas/Makefile => tools/um/uml/skas/Build (56%)
 rename {arch/um/os-Linux => tools/um/uml}/skas/mem.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/skas/process.c (99%)
 rename {arch/um/os-Linux => tools/um/uml}/start_up.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/time.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/tty.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/umid.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/util.c (100%)

diff --git a/arch/um/Makefile b/arch/um/Makefile
index 09745b40c52d..8be7bc479442 100644
--- a/arch/um/Makefile
+++ b/arch/um/Makefile
@@ -24,8 +24,7 @@ OS := $(shell uname -s)
 SHELL := /bin/bash
 
 core-y			+= $(ARCH_DIR)/kernel/		\
-			   $(ARCH_DIR)/drivers/		\
-			   $(ARCH_DIR)/os-$(OS)/
+			   $(ARCH_DIR)/drivers/
 
 MODE_INCLUDE	+= -I$(srctree)/$(ARCH_DIR)/include/shared/skas
 
diff --git a/arch/um/drivers/Makefile b/arch/um/drivers/Makefile
index 2a249f619467..ae96c83e312d 100644
--- a/arch/um/drivers/Makefile
+++ b/arch/um/drivers/Makefile
@@ -48,6 +48,8 @@ obj-$(CONFIG_UML_NET_VECTOR) += vector.o
 obj-$(CONFIG_UML_NET_VDE) += vde.o
 obj-$(CONFIG_UML_NET_MCAST) += umcast.o
 obj-$(CONFIG_UML_NET_PCAP) += pcap.o
+obj-$(CONFIG_UML_NET_ETHERTAP) += ethertap_kern.o
+obj-$(CONFIG_UML_NET_TUNTAP) += tuntap_kern.o
 obj-$(CONFIG_UML_NET) += net.o 
 obj-$(CONFIG_MCONSOLE) += mconsole.o
 obj-$(CONFIG_MMAPPER) += mmapper_kern.o 
diff --git a/arch/um/os-Linux/drivers/ethertap_kern.c b/arch/um/drivers/ethertap_kern.c
similarity index 100%
rename from arch/um/os-Linux/drivers/ethertap_kern.c
rename to arch/um/drivers/ethertap_kern.c
diff --git a/arch/um/os-Linux/drivers/tuntap_kern.c b/arch/um/drivers/tuntap_kern.c
similarity index 100%
rename from arch/um/os-Linux/drivers/tuntap_kern.c
rename to arch/um/drivers/tuntap_kern.c
diff --git a/arch/um/os-Linux/drivers/etap.h b/arch/um/include/shared/etap.h
similarity index 100%
rename from arch/um/os-Linux/drivers/etap.h
rename to arch/um/include/shared/etap.h
diff --git a/arch/um/include/shared/init.h b/arch/um/include/shared/init.h
index d09308330ca5..305789667584 100644
--- a/arch/um/include/shared/init.h
+++ b/arch/um/include/shared/init.h
@@ -41,7 +41,12 @@
 typedef int (*initcall_t)(void);
 typedef void (*exitcall_t)(void);
 
+#ifndef __UM_HOST__
 #include <linux/compiler_types.h>
+#else
+#define __used                          __attribute__((__used__))
+#define __section(S)                    __attribute__((__section__(#S)))
+#endif
 
 /* These are for everybody (although not all archs will actually
    discard it in modules) */
diff --git a/arch/um/os-Linux/drivers/tuntap.h b/arch/um/include/shared/tuntap.h
similarity index 100%
rename from arch/um/os-Linux/drivers/tuntap.h
rename to arch/um/include/shared/tuntap.h
diff --git a/arch/um/kernel/Makefile b/arch/um/kernel/Makefile
index 5aa882011e04..9b63831a69e1 100644
--- a/arch/um/kernel/Makefile
+++ b/arch/um/kernel/Makefile
@@ -24,6 +24,7 @@ obj-$(CONFIG_GPROF)	+= gprof_syms.o
 obj-$(CONFIG_GCOV)	+= gmon_syms.o
 obj-$(CONFIG_EARLY_PRINTK) += early_printk.o
 obj-$(CONFIG_STACKTRACE) += stacktrace.o
+obj-y += user_syms.o
 
 USER_OBJS := config.o
 
diff --git a/arch/um/os-Linux/user_syms.c b/arch/um/kernel/user_syms.c
similarity index 100%
rename from arch/um/os-Linux/user_syms.c
rename to arch/um/kernel/user_syms.c
diff --git a/arch/um/os-Linux/Makefile b/arch/um/os-Linux/Makefile
deleted file mode 100644
index 839915b8c31c..000000000000
--- a/arch/um/os-Linux/Makefile
+++ /dev/null
@@ -1,19 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0
-# 
-# Copyright (C) 2000 - 2007 Jeff Dike (jdike@{addtoit,linux.intel}.com)
-#
-
-# Don't instrument UML-specific code
-KCOV_INSTRUMENT                := n
-
-obj-y = execvp.o file.o helper.o irq.o main.o mem.o process.o \
-	registers.o sigio.o signal.o start_up.o time.o tty.o \
-	umid.o user_syms.o util.o drivers/ skas/
-
-obj-$(CONFIG_ARCH_REUSE_HOST_VSYSCALL_AREA) += elf_aux.o
-
-USER_OBJS := $(user-objs-y) elf_aux.o execvp.o file.o helper.o irq.o \
-	main.o mem.o process.o registers.o sigio.o signal.o start_up.o time.o \
-	tty.o umid.o util.o
-
-include arch/um/scripts/Makefile.rules
diff --git a/arch/um/os-Linux/drivers/Makefile b/arch/um/os-Linux/drivers/Makefile
deleted file mode 100644
index d79e75f1b69a..000000000000
--- a/arch/um/os-Linux/drivers/Makefile
+++ /dev/null
@@ -1,13 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0
-# 
-# Copyright (C) 2000, 2002 Jeff Dike (jdike@karaya.com)
-#
-
-ethertap-objs := ethertap_kern.o ethertap_user.o
-tuntap-objs := tuntap_kern.o tuntap_user.o
-
-obj-y = 
-obj-$(CONFIG_UML_NET_ETHERTAP) += ethertap.o
-obj-$(CONFIG_UML_NET_TUNTAP) += tuntap.o
-
-include arch/um/scripts/Makefile.rules
diff --git a/tools/um/Makefile b/tools/um/Makefile
index a63466ef7ef5..07b9b93ed817 100644
--- a/tools/um/Makefile
+++ b/tools/um/Makefile
@@ -50,7 +50,7 @@ $(OUTPUT)lib/linux.o:
 	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) $(KCONFIG)
 	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) install INSTALL_PATH=$(OUTPUT)
 
-$(OUTPUT)liblinux.a: $(OUTPUT)lib/linux.o $(OUTPUT)uml/liblinux-in.o
+$(OUTPUT)liblinux.a: $(OUTPUT)lib/linux.o $(OUTPUT)uml/liblinux-in.o $(OUTPUT)lib/liblinux-in.o
 	$(QUIET_AR)$(AR) -rc $@ $^
 
 # rule to link programs
@@ -60,6 +60,10 @@ $(OUTPUT)%: $(OUTPUT)%-in.o $(OUTPUT)liblinux.a
 $(OUTPUT)%-in.o: FORCE
 	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
 
+# rule to build objects
+$(OUTPUT)%-in.o: $(OUTPUT)lib/linux.o
+	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
+
 clean:
 	$(call QUIET_CLEAN, objects)find $(OUTPUT) -name '*.o' -delete -o -name '\.*.cmd'\
 	 -delete -o -name '\.*.d' -delete
diff --git a/tools/um/uml/Build b/tools/um/uml/Build
index e69de29bb2d1..ad93c6ad51fc 100644
--- a/tools/um/uml/Build
+++ b/tools/um/uml/Build
@@ -0,0 +1,48 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Copyright (C) 2000 - 2007 Jeff Dike (jdike@{addtoit,linux.intel}.com)
+#
+
+# Don't instrument UML-specific code
+KCOV_INSTRUMENT                := n
+
+include $(objtree)/include/config/auto.conf
+
+liblinux-y += execvp.o file.o helper.o irq.o main.o mem.o process.o \
+	registers.o sigio.o signal.o start_up.o time.o tty.o \
+	umid.o util.o drivers/ skas/
+
+liblinux-$(CONFIG_ARCH_REUSE_HOST_VSYSCALL_AREA) += elf_aux.o
+
+export O = $(srctree)
+objtree := $(O)
+# reset CFLAGS
+CFLAGS := -g -O2
+
+# from arch/um/Makefile
+ARCH_DIR := arch/um
+HEADER_ARCH 	:= x86
+HOST_DIR := arch/$(HEADER_ARCH)
+ifdef CONFIG_64BIT
+  KBUILD_CFLAGS += -mcmodel=large
+endif
+KBUILD_CFLAGS += $(CFLAGS) $(CFLAGS-y) -D__arch_um__ \
+	$(ARCH_INCLUDE) $(MODE_INCLUDE) -Dvmap=kernel_vmap	\
+	-Dlongjmp=kernel_longjmp -Dsetjmp=kernel_setjmp \
+	-Din6addr_loopback=kernel_in6addr_loopback \
+	-Din6addr_any=kernel_in6addr_any -Dstrrchr=kernel_strrchr
+SHARED_HEADERS	:= $(ARCH_DIR)/include/shared
+MODE_INCLUDE	+= -I$(srctree)/$(ARCH_DIR)/include/shared/skas
+ARCH_INCLUDE	:= -I$(srctree)/$(SHARED_HEADERS)
+ARCH_INCLUDE	+= -I$(srctree)/$(HOST_DIR)/um/shared
+KBUILD_CPPFLAGS += -I$(srctree)/$(HOST_DIR)/um
+USER_CFLAGS := $(patsubst $(KERNEL_DEFINES),,$(patsubst -I%,,$(KBUILD_CFLAGS))) \
+		$(ARCH_INCLUDE) $(MODE_INCLUDE) $(filter -I%,$(CFLAGS)) \
+		-D_FILE_OFFSET_BITS=64 -idirafter $(srctree)/include/uapi \
+		-idirafter $(objtree)/include -D__KERNEL__ -D__UM_HOST__
+
+# from Makefile-os-Linux
+USER_CFLAGS += -D_GNU_SOURCE -D_LARGEFILE64_SOURCE
+
+# from Makefile.rules
+CFLAGS += $(USER_CFLAGS) -include $(srctree)/tools/include/linux/kern_levels.h -include user.h $(CFLAGS_$(basetarget).o)
diff --git a/tools/um/uml/drivers/Build b/tools/um/uml/drivers/Build
new file mode 100644
index 000000000000..c7a8eaa97d72
--- /dev/null
+++ b/tools/um/uml/drivers/Build
@@ -0,0 +1,10 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Copyright (C) 2000, 2002 Jeff Dike (jdike@karaya.com)
+#
+
+include $(objtree)/include/config/auto.conf
+
+
+liblinux-$(CONFIG_UML_NET_ETHERTAP) += ethertap_user.o
+liblinux-$(CONFIG_UML_NET_TUNTAP) += tuntap_user.o
diff --git a/arch/um/os-Linux/drivers/ethertap_user.c b/tools/um/uml/drivers/ethertap_user.c
similarity index 100%
rename from arch/um/os-Linux/drivers/ethertap_user.c
rename to tools/um/uml/drivers/ethertap_user.c
diff --git a/arch/um/os-Linux/drivers/tuntap_user.c b/tools/um/uml/drivers/tuntap_user.c
similarity index 100%
rename from arch/um/os-Linux/drivers/tuntap_user.c
rename to tools/um/uml/drivers/tuntap_user.c
diff --git a/arch/um/os-Linux/elf_aux.c b/tools/um/uml/elf_aux.c
similarity index 100%
rename from arch/um/os-Linux/elf_aux.c
rename to tools/um/uml/elf_aux.c
diff --git a/arch/um/os-Linux/execvp.c b/tools/um/uml/execvp.c
similarity index 98%
rename from arch/um/os-Linux/execvp.c
rename to tools/um/uml/execvp.c
index 84a0777c2a45..c105e0e7b778 100644
--- a/arch/um/os-Linux/execvp.c
+++ b/tools/um/uml/execvp.c
@@ -26,12 +26,8 @@
 #include <errno.h>
 #include <limits.h>
 
-#ifndef TEST
-#include <um_malloc.h>
-#else
 #include <stdio.h>
 #define um_kmalloc malloc
-#endif
 #include <os.h>
 
 /* Execute FILE, searching in the `PATH' environment variable if it contains
diff --git a/arch/um/os-Linux/file.c b/tools/um/uml/file.c
similarity index 100%
rename from arch/um/os-Linux/file.c
rename to tools/um/uml/file.c
diff --git a/arch/um/os-Linux/helper.c b/tools/um/uml/helper.c
similarity index 100%
rename from arch/um/os-Linux/helper.c
rename to tools/um/uml/helper.c
diff --git a/arch/um/os-Linux/irq.c b/tools/um/uml/irq.c
similarity index 100%
rename from arch/um/os-Linux/irq.c
rename to tools/um/uml/irq.c
diff --git a/arch/um/os-Linux/main.c b/tools/um/uml/main.c
similarity index 100%
rename from arch/um/os-Linux/main.c
rename to tools/um/uml/main.c
diff --git a/arch/um/os-Linux/mem.c b/tools/um/uml/mem.c
similarity index 100%
rename from arch/um/os-Linux/mem.c
rename to tools/um/uml/mem.c
diff --git a/arch/um/os-Linux/process.c b/tools/um/uml/process.c
similarity index 100%
rename from arch/um/os-Linux/process.c
rename to tools/um/uml/process.c
diff --git a/arch/um/os-Linux/registers.c b/tools/um/uml/registers.c
similarity index 100%
rename from arch/um/os-Linux/registers.c
rename to tools/um/uml/registers.c
diff --git a/arch/um/os-Linux/sigio.c b/tools/um/uml/sigio.c
similarity index 100%
rename from arch/um/os-Linux/sigio.c
rename to tools/um/uml/sigio.c
diff --git a/arch/um/os-Linux/signal.c b/tools/um/uml/signal.c
similarity index 100%
rename from arch/um/os-Linux/signal.c
rename to tools/um/uml/signal.c
diff --git a/arch/um/os-Linux/skas/Makefile b/tools/um/uml/skas/Build
similarity index 56%
rename from arch/um/os-Linux/skas/Makefile
rename to tools/um/uml/skas/Build
index c4566e788815..5fc9d62df863 100644
--- a/arch/um/os-Linux/skas/Makefile
+++ b/tools/um/uml/skas/Build
@@ -3,8 +3,4 @@
 # Copyright (C) 2002 - 2007 Jeff Dike (jdike@{linux.intel,addtoit}.com)
 #
 
-obj-y := mem.o process.o
-
-USER_OBJS := $(obj-y)
-
-include arch/um/scripts/Makefile.rules
+liblinux-y += mem.o process.o
diff --git a/arch/um/os-Linux/skas/mem.c b/tools/um/uml/skas/mem.c
similarity index 100%
rename from arch/um/os-Linux/skas/mem.c
rename to tools/um/uml/skas/mem.c
diff --git a/arch/um/os-Linux/skas/process.c b/tools/um/uml/skas/process.c
similarity index 99%
rename from arch/um/os-Linux/skas/process.c
rename to tools/um/uml/skas/process.c
index 4fb877b99dde..fca6a08e81bd 100644
--- a/arch/um/os-Linux/skas/process.c
+++ b/tools/um/uml/skas/process.c
@@ -21,7 +21,6 @@
 #include <registers.h>
 #include <skas.h>
 #include <sysdep/stub.h>
-#include <linux/threads.h>
 
 int is_skas_winch(int pid, int fd, void *data)
 {
@@ -248,7 +247,7 @@ static int userspace_tramp(void *stack)
 	return 0;
 }
 
-int userspace_pid[NR_CPUS];
+int userspace_pid[UM_NR_CPUS];
 
 /**
  * start_userspace() - prepare a new userspace process
diff --git a/arch/um/os-Linux/start_up.c b/tools/um/uml/start_up.c
similarity index 100%
rename from arch/um/os-Linux/start_up.c
rename to tools/um/uml/start_up.c
diff --git a/arch/um/os-Linux/time.c b/tools/um/uml/time.c
similarity index 100%
rename from arch/um/os-Linux/time.c
rename to tools/um/uml/time.c
diff --git a/arch/um/os-Linux/tty.c b/tools/um/uml/tty.c
similarity index 100%
rename from arch/um/os-Linux/tty.c
rename to tools/um/uml/tty.c
diff --git a/arch/um/os-Linux/umid.c b/tools/um/uml/umid.c
similarity index 100%
rename from arch/um/os-Linux/umid.c
rename to tools/um/uml/umid.c
diff --git a/arch/um/os-Linux/util.c b/tools/um/uml/util.c
similarity index 100%
rename from arch/um/os-Linux/util.c
rename to tools/um/uml/util.c
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v7 04/21] um: host: implement os_initcalls and os_exitcalls
  2020-10-06  9:44         ` Hajime Tazaki
@ 2020-10-06  9:44           ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-06  9:44 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch,
	Octavian Purdila

From: Octavian Purdila <tavi@cs.pub.ro>

This patch implements the init and exit calls for host code. It uses
the automatic __start_<section> and __stop_<section> variables that
are defined by gcc / ld when using custom sections.

Note that this patch should be merged with "um: move arch/um/os-Linux
dir to tools/um" but for now it is separate to make the review easier.

Signed-off-by: Octavian Purdila <tavi@cs.pub.ro>
---
 tools/um/uml/util.c | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)

diff --git a/tools/um/uml/util.c b/tools/um/uml/util.c
index ecf2f390fad2..4011b36fee7e 100644
--- a/tools/um/uml/util.c
+++ b/tools/um/uml/util.c
@@ -186,3 +186,29 @@ void os_warn(const char *fmt, ...)
 	vfprintf(stderr, fmt, list);
 	va_end(list);
 }
+
+extern void (*__start_os_exitcalls)(void);
+extern void (*__stop_os_exitcalls)(void);
+
+void os_exitcalls(void)
+{
+	exitcall_t *call;
+
+	call = &__stop_os_exitcalls;
+	while (--call >= &__start_os_exitcalls)
+		(*call)();
+}
+
+extern int (*__start_os_initcalls)(void);
+extern int (*__stop_os_initcalls)(void);
+
+int os_initcalls(void)
+{
+	initcall_t *call;
+
+	call = &__stop_os_initcalls;
+	while (--call >= &__start_os_initcalls)
+		(*call)();
+
+	return 0;
+}
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v7 04/21] um: host: implement os_initcalls and os_exitcalls
@ 2020-10-06  9:44           ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-06  9:44 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Octavian Purdila,
	retrage01

From: Octavian Purdila <tavi@cs.pub.ro>

This patch implements the init and exit calls for host code. It uses
the automatic __start_<section> and __stop_<section> variables that
are defined by gcc / ld when using custom sections.

Note that this patch should be merged with "um: move arch/um/os-Linux
dir to tools/um" but for now it is separate to make the review easier.

Signed-off-by: Octavian Purdila <tavi@cs.pub.ro>
---
 tools/um/uml/util.c | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)

diff --git a/tools/um/uml/util.c b/tools/um/uml/util.c
index ecf2f390fad2..4011b36fee7e 100644
--- a/tools/um/uml/util.c
+++ b/tools/um/uml/util.c
@@ -186,3 +186,29 @@ void os_warn(const char *fmt, ...)
 	vfprintf(stderr, fmt, list);
 	va_end(list);
 }
+
+extern void (*__start_os_exitcalls)(void);
+extern void (*__stop_os_exitcalls)(void);
+
+void os_exitcalls(void)
+{
+	exitcall_t *call;
+
+	call = &__stop_os_exitcalls;
+	while (--call >= &__start_os_exitcalls)
+		(*call)();
+}
+
+extern int (*__start_os_initcalls)(void);
+extern int (*__stop_os_initcalls)(void);
+
+int os_initcalls(void)
+{
+	initcall_t *call;
+
+	call = &__stop_os_initcalls;
+	while (--call >= &__start_os_initcalls)
+		(*call)();
+
+	return 0;
+}
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v7 05/21] um: move arch/x86/um/os-Linux to tools/um/uml/
  2020-10-06  9:44         ` Hajime Tazaki
@ 2020-10-06  9:44           ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-06  9:44 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch, Hajime Tazaki

Similar to arch/um/os-Linux code, arch/x86 also contains OS dependent
part as well.  This patch moves underlying OS dependent part to tools/um
directory so that arch/x86 code does not need to build host build
facilities (e.g., libc).

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/x86/um/Makefile                                |  2 +-
 arch/x86/um/os-Linux/Makefile                       | 13 -------------
 tools/um/Makefile                                   |  2 +-
 tools/um/uml/Build                                  |  2 ++
 tools/um/uml/x86/Build                              | 11 +++++++++++
 .../x86/um/os-Linux => tools/um/uml/x86}/mcontext.c |  0
 {arch/x86/um/os-Linux => tools/um/uml/x86}/prctl.c  |  0
 .../um/os-Linux => tools/um/uml/x86}/registers.c    |  0
 .../um/os-Linux => tools/um/uml/x86}/task_size.c    |  0
 {arch/x86/um/os-Linux => tools/um/uml/x86}/tls.c    |  0
 10 files changed, 15 insertions(+), 15 deletions(-)
 delete mode 100644 arch/x86/um/os-Linux/Makefile
 create mode 100644 tools/um/uml/x86/Build
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/mcontext.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/prctl.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/registers.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/task_size.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/tls.c (100%)

diff --git a/arch/x86/um/Makefile b/arch/x86/um/Makefile
index 77f70b969d14..25e9846b36a2 100644
--- a/arch/x86/um/Makefile
+++ b/arch/x86/um/Makefile
@@ -13,7 +13,7 @@ obj-y = bugs_$(BITS).o delay.o fault.o ldt.o \
 	ptrace_$(BITS).o ptrace_user.o setjmp_$(BITS).o signal.o \
 	stub_$(BITS).o stub_segv.o \
 	sys_call_table_$(BITS).o sysrq_$(BITS).o tls_$(BITS).o \
-	mem_$(BITS).o subarch.o os-$(OS)/
+	mem_$(BITS).o subarch.o
 
 ifeq ($(CONFIG_X86_32),y)
 
diff --git a/arch/x86/um/os-Linux/Makefile b/arch/x86/um/os-Linux/Makefile
deleted file mode 100644
index 253bfb8cb702..000000000000
--- a/arch/x86/um/os-Linux/Makefile
+++ /dev/null
@@ -1,13 +0,0 @@
-#
-# Copyright (C) 2000 - 2007 Jeff Dike (jdike@{addtoit,linux.intel}.com)
-# Licensed under the GPL
-#
-
-obj-y = registers.o task_size.o mcontext.o
-
-obj-$(CONFIG_X86_32) += tls.o
-obj-$(CONFIG_64BIT) += prctl.o
-
-USER_OBJS := $(obj-y)
-
-include arch/um/scripts/Makefile.rules
diff --git a/tools/um/Makefile b/tools/um/Makefile
index 07b9b93ed817..552ed5f1edae 100644
--- a/tools/um/Makefile
+++ b/tools/um/Makefile
@@ -61,7 +61,7 @@ $(OUTPUT)%-in.o: FORCE
 	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
 
 # rule to build objects
-$(OUTPUT)%-in.o: $(OUTPUT)lib/linux.o
+$(OUTPUT)%-in.o: $(OUTPUT)lib/linux.o FORCE
 	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
 
 clean:
diff --git a/tools/um/uml/Build b/tools/um/uml/Build
index ad93c6ad51fc..0d87d601bc06 100644
--- a/tools/um/uml/Build
+++ b/tools/um/uml/Build
@@ -12,6 +12,8 @@ liblinux-y += execvp.o file.o helper.o irq.o main.o mem.o process.o \
 	registers.o sigio.o signal.o start_up.o time.o tty.o \
 	umid.o util.o drivers/ skas/
 
+liblinux-y += x86/
+
 liblinux-$(CONFIG_ARCH_REUSE_HOST_VSYSCALL_AREA) += elf_aux.o
 
 export O = $(srctree)
diff --git a/tools/um/uml/x86/Build b/tools/um/uml/x86/Build
new file mode 100644
index 000000000000..16bbc7eb19d1
--- /dev/null
+++ b/tools/um/uml/x86/Build
@@ -0,0 +1,11 @@
+#
+# Copyright (C) 2000 - 2007 Jeff Dike (jdike@{addtoit,linux.intel}.com)
+# Licensed under the GPL
+#
+
+include $(objtree)/include/config/auto.conf
+
+liblinux-y = registers.o task_size.o mcontext.o
+
+liblinux-$(CONFIG_X86_32) += tls.o
+liblinux-$(CONFIG_64BIT) += prctl.o
diff --git a/arch/x86/um/os-Linux/mcontext.c b/tools/um/uml/x86/mcontext.c
similarity index 100%
rename from arch/x86/um/os-Linux/mcontext.c
rename to tools/um/uml/x86/mcontext.c
diff --git a/arch/x86/um/os-Linux/prctl.c b/tools/um/uml/x86/prctl.c
similarity index 100%
rename from arch/x86/um/os-Linux/prctl.c
rename to tools/um/uml/x86/prctl.c
diff --git a/arch/x86/um/os-Linux/registers.c b/tools/um/uml/x86/registers.c
similarity index 100%
rename from arch/x86/um/os-Linux/registers.c
rename to tools/um/uml/x86/registers.c
diff --git a/arch/x86/um/os-Linux/task_size.c b/tools/um/uml/x86/task_size.c
similarity index 100%
rename from arch/x86/um/os-Linux/task_size.c
rename to tools/um/uml/x86/task_size.c
diff --git a/arch/x86/um/os-Linux/tls.c b/tools/um/uml/x86/tls.c
similarity index 100%
rename from arch/x86/um/os-Linux/tls.c
rename to tools/um/uml/x86/tls.c
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v7 05/21] um: move arch/x86/um/os-Linux to tools/um/uml/
@ 2020-10-06  9:44           ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-06  9:44 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Hajime Tazaki, retrage01

Similar to arch/um/os-Linux code, arch/x86 also contains OS dependent
part as well.  This patch moves underlying OS dependent part to tools/um
directory so that arch/x86 code does not need to build host build
facilities (e.g., libc).

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/x86/um/Makefile                                |  2 +-
 arch/x86/um/os-Linux/Makefile                       | 13 -------------
 tools/um/Makefile                                   |  2 +-
 tools/um/uml/Build                                  |  2 ++
 tools/um/uml/x86/Build                              | 11 +++++++++++
 .../x86/um/os-Linux => tools/um/uml/x86}/mcontext.c |  0
 {arch/x86/um/os-Linux => tools/um/uml/x86}/prctl.c  |  0
 .../um/os-Linux => tools/um/uml/x86}/registers.c    |  0
 .../um/os-Linux => tools/um/uml/x86}/task_size.c    |  0
 {arch/x86/um/os-Linux => tools/um/uml/x86}/tls.c    |  0
 10 files changed, 15 insertions(+), 15 deletions(-)
 delete mode 100644 arch/x86/um/os-Linux/Makefile
 create mode 100644 tools/um/uml/x86/Build
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/mcontext.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/prctl.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/registers.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/task_size.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/tls.c (100%)

diff --git a/arch/x86/um/Makefile b/arch/x86/um/Makefile
index 77f70b969d14..25e9846b36a2 100644
--- a/arch/x86/um/Makefile
+++ b/arch/x86/um/Makefile
@@ -13,7 +13,7 @@ obj-y = bugs_$(BITS).o delay.o fault.o ldt.o \
 	ptrace_$(BITS).o ptrace_user.o setjmp_$(BITS).o signal.o \
 	stub_$(BITS).o stub_segv.o \
 	sys_call_table_$(BITS).o sysrq_$(BITS).o tls_$(BITS).o \
-	mem_$(BITS).o subarch.o os-$(OS)/
+	mem_$(BITS).o subarch.o
 
 ifeq ($(CONFIG_X86_32),y)
 
diff --git a/arch/x86/um/os-Linux/Makefile b/arch/x86/um/os-Linux/Makefile
deleted file mode 100644
index 253bfb8cb702..000000000000
--- a/arch/x86/um/os-Linux/Makefile
+++ /dev/null
@@ -1,13 +0,0 @@
-#
-# Copyright (C) 2000 - 2007 Jeff Dike (jdike@{addtoit,linux.intel}.com)
-# Licensed under the GPL
-#
-
-obj-y = registers.o task_size.o mcontext.o
-
-obj-$(CONFIG_X86_32) += tls.o
-obj-$(CONFIG_64BIT) += prctl.o
-
-USER_OBJS := $(obj-y)
-
-include arch/um/scripts/Makefile.rules
diff --git a/tools/um/Makefile b/tools/um/Makefile
index 07b9b93ed817..552ed5f1edae 100644
--- a/tools/um/Makefile
+++ b/tools/um/Makefile
@@ -61,7 +61,7 @@ $(OUTPUT)%-in.o: FORCE
 	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
 
 # rule to build objects
-$(OUTPUT)%-in.o: $(OUTPUT)lib/linux.o
+$(OUTPUT)%-in.o: $(OUTPUT)lib/linux.o FORCE
 	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
 
 clean:
diff --git a/tools/um/uml/Build b/tools/um/uml/Build
index ad93c6ad51fc..0d87d601bc06 100644
--- a/tools/um/uml/Build
+++ b/tools/um/uml/Build
@@ -12,6 +12,8 @@ liblinux-y += execvp.o file.o helper.o irq.o main.o mem.o process.o \
 	registers.o sigio.o signal.o start_up.o time.o tty.o \
 	umid.o util.o drivers/ skas/
 
+liblinux-y += x86/
+
 liblinux-$(CONFIG_ARCH_REUSE_HOST_VSYSCALL_AREA) += elf_aux.o
 
 export O = $(srctree)
diff --git a/tools/um/uml/x86/Build b/tools/um/uml/x86/Build
new file mode 100644
index 000000000000..16bbc7eb19d1
--- /dev/null
+++ b/tools/um/uml/x86/Build
@@ -0,0 +1,11 @@
+#
+# Copyright (C) 2000 - 2007 Jeff Dike (jdike@{addtoit,linux.intel}.com)
+# Licensed under the GPL
+#
+
+include $(objtree)/include/config/auto.conf
+
+liblinux-y = registers.o task_size.o mcontext.o
+
+liblinux-$(CONFIG_X86_32) += tls.o
+liblinux-$(CONFIG_64BIT) += prctl.o
diff --git a/arch/x86/um/os-Linux/mcontext.c b/tools/um/uml/x86/mcontext.c
similarity index 100%
rename from arch/x86/um/os-Linux/mcontext.c
rename to tools/um/uml/x86/mcontext.c
diff --git a/arch/x86/um/os-Linux/prctl.c b/tools/um/uml/x86/prctl.c
similarity index 100%
rename from arch/x86/um/os-Linux/prctl.c
rename to tools/um/uml/x86/prctl.c
diff --git a/arch/x86/um/os-Linux/registers.c b/tools/um/uml/x86/registers.c
similarity index 100%
rename from arch/x86/um/os-Linux/registers.c
rename to tools/um/uml/x86/registers.c
diff --git a/arch/x86/um/os-Linux/task_size.c b/tools/um/uml/x86/task_size.c
similarity index 100%
rename from arch/x86/um/os-Linux/task_size.c
rename to tools/um/uml/x86/task_size.c
diff --git a/arch/x86/um/os-Linux/tls.c b/tools/um/uml/x86/tls.c
similarity index 100%
rename from arch/x86/um/os-Linux/tls.c
rename to tools/um/uml/x86/tls.c
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v7 06/21] scritps: um: suppress warnings if SRCARCH=um
  2020-10-06  9:44         ` Hajime Tazaki
@ 2020-10-06  9:44           ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-06  9:44 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch, Hajime Tazaki

This commit fixes numbers of warning messages about leaked CONFIG
options.  nommu mode of UML requires copies of kernel headers to offer
syscall-like API for the library users.  Thus, the warnings are to be
avoided to function this exposure of API.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 scripts/headers_install.sh | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/scripts/headers_install.sh b/scripts/headers_install.sh
index dd554bd436cc..8890a0147012 100755
--- a/scripts/headers_install.sh
+++ b/scripts/headers_install.sh
@@ -93,6 +93,10 @@ include/uapi/linux/pktcdvd.h:CONFIG_CDROM_PKTCDVD_WCACHE
 
 for c in $configs
 do
+	if [ "$SRCARCH" = "um" ] ; then
+		break
+	fi
+
 	leak_error=1
 
 	for ignore in $config_leak_ignores
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v7 06/21] scritps: um: suppress warnings if SRCARCH=um
@ 2020-10-06  9:44           ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-06  9:44 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Hajime Tazaki, retrage01

This commit fixes numbers of warning messages about leaked CONFIG
options.  nommu mode of UML requires copies of kernel headers to offer
syscall-like API for the library users.  Thus, the warnings are to be
avoided to function this exposure of API.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 scripts/headers_install.sh | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/scripts/headers_install.sh b/scripts/headers_install.sh
index dd554bd436cc..8890a0147012 100755
--- a/scripts/headers_install.sh
+++ b/scripts/headers_install.sh
@@ -93,6 +93,10 @@ include/uapi/linux/pktcdvd.h:CONFIG_CDROM_PKTCDVD_WCACHE
 
 for c in $configs
 do
+	if [ "$SRCARCH" = "um" ] ; then
+		break
+	fi
+
 	leak_error=1
 
 	for ignore in $config_leak_ignores
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v7 07/21] um: extend arch_switch_to for alternate SUBARCH
  2020-10-06  9:44         ` Hajime Tazaki
@ 2020-10-06  9:44           ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-06  9:44 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch, Hajime Tazaki

This commit introduces additional argument of previous task when
context switch happens.  New SUBARCH can use the new information to
switch tasks in a subarch-specific manner.

The patch is particularly required by nommu mode implemented as a
SUBARCH of UML.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/kernel/process.c  | 6 +++---
 arch/x86/um/ptrace_32.c   | 2 +-
 arch/x86/um/syscalls_64.c | 2 +-
 3 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/arch/um/kernel/process.c b/arch/um/kernel/process.c
index 26b5e243d3fc..87a8cfa228ca 100644
--- a/arch/um/kernel/process.c
+++ b/arch/um/kernel/process.c
@@ -80,7 +80,7 @@ static inline void set_current(struct task_struct *task)
 		{ external_pid(), task });
 }
 
-extern void arch_switch_to(struct task_struct *to);
+extern void arch_switch_to(struct task_struct *from, struct task_struct *to);
 
 void *__switch_to(struct task_struct *from, struct task_struct *to)
 {
@@ -88,7 +88,7 @@ void *__switch_to(struct task_struct *from, struct task_struct *to)
 	set_current(to);
 
 	switch_threads(&from->thread.switch_buf, &to->thread.switch_buf);
-	arch_switch_to(current);
+	arch_switch_to(from, to);
 
 	return current->thread.prev_sched;
 }
@@ -145,7 +145,7 @@ void fork_handler(void)
 	 * arch_switch_to isn't needed. We could want to apply this to
 	 * improve performance. -bb
 	 */
-	arch_switch_to(current);
+	arch_switch_to(NULL, current);
 
 	current->thread.prev_sched = NULL;
 
diff --git a/arch/x86/um/ptrace_32.c b/arch/x86/um/ptrace_32.c
index 2497bac56066..0f184710d4ca 100644
--- a/arch/x86/um/ptrace_32.c
+++ b/arch/x86/um/ptrace_32.c
@@ -11,7 +11,7 @@
 
 extern int arch_switch_tls(struct task_struct *to);
 
-void arch_switch_to(struct task_struct *to)
+void arch_switch_to(struct task_struct *from, struct task_struct *to)
 {
 	int err = arch_switch_tls(to);
 	if (!err)
diff --git a/arch/x86/um/syscalls_64.c b/arch/x86/um/syscalls_64.c
index 58f51667e2e4..2ef9474d2bd2 100644
--- a/arch/x86/um/syscalls_64.c
+++ b/arch/x86/um/syscalls_64.c
@@ -80,7 +80,7 @@ SYSCALL_DEFINE2(arch_prctl, int, option, unsigned long, arg2)
 	return arch_prctl(current, option, (unsigned long __user *) arg2);
 }
 
-void arch_switch_to(struct task_struct *to)
+void arch_switch_to(struct task_struct *from, struct task_struct *to)
 {
 	if ((to->thread.arch.fs == 0) || (to->mm == NULL))
 		return;
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v7 07/21] um: extend arch_switch_to for alternate SUBARCH
@ 2020-10-06  9:44           ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-06  9:44 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Hajime Tazaki, retrage01

This commit introduces additional argument of previous task when
context switch happens.  New SUBARCH can use the new information to
switch tasks in a subarch-specific manner.

The patch is particularly required by nommu mode implemented as a
SUBARCH of UML.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/kernel/process.c  | 6 +++---
 arch/x86/um/ptrace_32.c   | 2 +-
 arch/x86/um/syscalls_64.c | 2 +-
 3 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/arch/um/kernel/process.c b/arch/um/kernel/process.c
index 26b5e243d3fc..87a8cfa228ca 100644
--- a/arch/um/kernel/process.c
+++ b/arch/um/kernel/process.c
@@ -80,7 +80,7 @@ static inline void set_current(struct task_struct *task)
 		{ external_pid(), task });
 }
 
-extern void arch_switch_to(struct task_struct *to);
+extern void arch_switch_to(struct task_struct *from, struct task_struct *to);
 
 void *__switch_to(struct task_struct *from, struct task_struct *to)
 {
@@ -88,7 +88,7 @@ void *__switch_to(struct task_struct *from, struct task_struct *to)
 	set_current(to);
 
 	switch_threads(&from->thread.switch_buf, &to->thread.switch_buf);
-	arch_switch_to(current);
+	arch_switch_to(from, to);
 
 	return current->thread.prev_sched;
 }
@@ -145,7 +145,7 @@ void fork_handler(void)
 	 * arch_switch_to isn't needed. We could want to apply this to
 	 * improve performance. -bb
 	 */
-	arch_switch_to(current);
+	arch_switch_to(NULL, current);
 
 	current->thread.prev_sched = NULL;
 
diff --git a/arch/x86/um/ptrace_32.c b/arch/x86/um/ptrace_32.c
index 2497bac56066..0f184710d4ca 100644
--- a/arch/x86/um/ptrace_32.c
+++ b/arch/x86/um/ptrace_32.c
@@ -11,7 +11,7 @@
 
 extern int arch_switch_tls(struct task_struct *to);
 
-void arch_switch_to(struct task_struct *to)
+void arch_switch_to(struct task_struct *from, struct task_struct *to)
 {
 	int err = arch_switch_tls(to);
 	if (!err)
diff --git a/arch/x86/um/syscalls_64.c b/arch/x86/um/syscalls_64.c
index 58f51667e2e4..2ef9474d2bd2 100644
--- a/arch/x86/um/syscalls_64.c
+++ b/arch/x86/um/syscalls_64.c
@@ -80,7 +80,7 @@ SYSCALL_DEFINE2(arch_prctl, int, option, unsigned long, arg2)
 	return arch_prctl(current, option, (unsigned long __user *) arg2);
 }
 
-void arch_switch_to(struct task_struct *to)
+void arch_switch_to(struct task_struct *from, struct task_struct *to)
 {
 	if ((to->thread.arch.fs == 0) || (to->mm == NULL))
 		return;
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v7 08/21] um: add nommu mode for UML library mode
  2020-10-06  9:44         ` Hajime Tazaki
@ 2020-10-06  9:44           ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-06  9:44 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch, Hajime Tazaki

This patch introduces the nommu operation with UML code so that host
interface can be shrineked for broader environments support.

The nommu mode is implemneted as SUBARCH of arch/um, which places at
arch/um/nommu. This SUBARCH defines mode-specific code of memory
management as well as thread implementation, along with the uapi headers
to be exported to users.

The headers we introduce in this patch are simple wrappers to the
asm-generic headers or stubs for things we don't support, such as
ptrace, DMA, signals, ELF handling and low level processor operations.

nommu mode shares most of arch/um code (irq, thread_info, process
scheduling) but implements its own facilities, which are memory
management (nommu), thread primitives (struct arch_thread), system call
interface, and console.

The outlook of updated directory structure is as follows:

   arch/um
   |-- configs         (untouched)
   |-- drivers         unuse stdio_console.c for !MMU
   |-- include
   |   |-- asm         updated with !CONFIG_MMU
   |   |-- linux       (untouched)
   |   `-- shared      updated with new functions
   |       `-- skas    (untouched)
   |   `-- uapi        added for upai header installation
   |-- kernel          updated to integrate with !MMU
   |   `-- skas        (untouched, don't use for !MMU)
   |-- nommu           SUBARCH dir (internally =um/nommu)
   |   |-- include
   |   |   |-- asm     headers for subarch specific info
   |   |   `-- uapi    headers for user-visible APIs
   |   `-- um
   |       `-- shared  headers for subarch specific info
   `-- scripts         added a script for header installation

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/nommu/Makefile                        |   1 +
 arch/um/nommu/Makefile.um                     |  22 ++++
 arch/um/nommu/include/asm/Kbuild              |   6 +
 arch/um/nommu/include/asm/archparam.h         |   1 +
 arch/um/nommu/include/asm/atomic.h            |  11 ++
 arch/um/nommu/include/asm/atomic64.h          | 114 ++++++++++++++++++
 arch/um/nommu/include/asm/bitsperlong.h       |  12 ++
 arch/um/nommu/include/asm/byteorder.h         |   7 ++
 arch/um/nommu/include/asm/cpu.h               |  13 ++
 arch/um/nommu/include/asm/elf.h               |  15 +++
 arch/um/nommu/include/asm/mm_context.h        |   8 ++
 arch/um/nommu/include/asm/processor.h         |  46 +++++++
 arch/um/nommu/include/asm/ptrace.h            |  21 ++++
 arch/um/nommu/include/asm/sched.h             |  23 ++++
 arch/um/nommu/include/asm/segment.h           |   9 ++
 arch/um/nommu/include/uapi/asm/Kbuild         |   6 +
 arch/um/nommu/include/uapi/asm/bitsperlong.h  |  11 ++
 arch/um/nommu/include/uapi/asm/byteorder.h    |  11 ++
 arch/um/nommu/include/uapi/asm/sigcontext.h   |  12 ++
 arch/um/nommu/um/delay.c                      |  31 +++++
 arch/um/nommu/um/shared/sysdep/archsetjmp.h   |  13 ++
 arch/um/nommu/um/shared/sysdep/faultinfo.h    |   8 ++
 .../nommu/um/shared/sysdep/kernel-offsets.h   |  12 ++
 arch/um/nommu/um/shared/sysdep/mcontext.h     |   9 ++
 arch/um/nommu/um/shared/sysdep/ptrace.h       |  42 +++++++
 arch/um/nommu/um/shared/sysdep/ptrace_user.h  |   7 ++
 arch/um/nommu/um/unimplemented.c              |  70 +++++++++++
 arch/um/nommu/um/user_constants.h             |  13 ++
 28 files changed, 554 insertions(+)
 create mode 100644 arch/um/nommu/Makefile
 create mode 100644 arch/um/nommu/Makefile.um
 create mode 100644 arch/um/nommu/include/asm/Kbuild
 create mode 100644 arch/um/nommu/include/asm/archparam.h
 create mode 100644 arch/um/nommu/include/asm/atomic.h
 create mode 100644 arch/um/nommu/include/asm/atomic64.h
 create mode 100644 arch/um/nommu/include/asm/bitsperlong.h
 create mode 100644 arch/um/nommu/include/asm/byteorder.h
 create mode 100644 arch/um/nommu/include/asm/cpu.h
 create mode 100644 arch/um/nommu/include/asm/elf.h
 create mode 100644 arch/um/nommu/include/asm/mm_context.h
 create mode 100644 arch/um/nommu/include/asm/processor.h
 create mode 100644 arch/um/nommu/include/asm/ptrace.h
 create mode 100644 arch/um/nommu/include/asm/sched.h
 create mode 100644 arch/um/nommu/include/asm/segment.h
 create mode 100644 arch/um/nommu/include/uapi/asm/Kbuild
 create mode 100644 arch/um/nommu/include/uapi/asm/bitsperlong.h
 create mode 100644 arch/um/nommu/include/uapi/asm/byteorder.h
 create mode 100644 arch/um/nommu/include/uapi/asm/sigcontext.h
 create mode 100644 arch/um/nommu/um/delay.c
 create mode 100644 arch/um/nommu/um/shared/sysdep/archsetjmp.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/faultinfo.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/kernel-offsets.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/mcontext.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/ptrace.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/ptrace_user.h
 create mode 100644 arch/um/nommu/um/unimplemented.c
 create mode 100644 arch/um/nommu/um/user_constants.h

diff --git a/arch/um/nommu/Makefile b/arch/um/nommu/Makefile
new file mode 100644
index 000000000000..792d6005489e
--- /dev/null
+++ b/arch/um/nommu/Makefile
@@ -0,0 +1 @@
+#
diff --git a/arch/um/nommu/Makefile.um b/arch/um/nommu/Makefile.um
new file mode 100644
index 000000000000..3808462d8283
--- /dev/null
+++ b/arch/um/nommu/Makefile.um
@@ -0,0 +1,22 @@
+KBUILD_CFLAGS += -fno-builtin -fPIC
+ELF_FORMAT=$(shell $(LD) -r -print-output-format)
+
+ifeq ($(shell uname -s), Linux)
+NPROC=$(shell nproc)
+else # e.g., FreeBSD
+NPROC=$(shell sysctl -n hw.ncpu)
+endif
+
+um_headers_install: $(objtree)/$(HOST_DIR)/include/generated/uapi/asm/syscall_defs.h headers
+# if syscall_defs.h is newer than generated headers (autoconf.h)
+	$(Q)if [ ! -r $(objtree)/tools/um/include/lkl/autoconf.h ] || \
+	  [ $< -nt $(objtree)/tools/um/include/lkl/autoconf.h ]; then \
+		$(srctree)/$(ARCH_DIR)/scripts/headers_install.py \
+			$(subst -j,-j$(NPROC),$(findstring -j,$(MAKEFLAGS))) \
+			$(INSTALL_PATH)/include ; \
+	fi
+
+$(objtree)/$(HOST_DIR)/include/generated/uapi/asm/syscall_defs.h: vmlinux
+	$(Q)$(OBJCOPY) -j .syscall_defs -O binary --set-section-flags .syscall_defs=alloc $< $@
+	$(Q) export tmpfile=$(shell mktemp); \
+	sed 's/\x0//g' $@ > $$tmpfile; mv $$tmpfile $@ ; rm -f $$tmpfile
diff --git a/arch/um/nommu/include/asm/Kbuild b/arch/um/nommu/include/asm/Kbuild
new file mode 100644
index 000000000000..2532e1a0a0d1
--- /dev/null
+++ b/arch/um/nommu/include/asm/Kbuild
@@ -0,0 +1,6 @@
+generic-y += cmpxchg.h
+generic-y += local64.h
+generic-y += seccomp.h
+generic-y += string.h
+generic-y += syscall.h
+generic-y += user.h
diff --git a/arch/um/nommu/include/asm/archparam.h b/arch/um/nommu/include/asm/archparam.h
new file mode 100644
index 000000000000..ea32a7d3cf1b
--- /dev/null
+++ b/arch/um/nommu/include/asm/archparam.h
@@ -0,0 +1 @@
+/* SPDX-License-Identifier: GPL-2.0 */
diff --git a/arch/um/nommu/include/asm/atomic.h b/arch/um/nommu/include/asm/atomic.h
new file mode 100644
index 000000000000..63e2e16bda92
--- /dev/null
+++ b/arch/um/nommu/include/asm/atomic.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_ATOMIC_H
+#define __UM_NOMMU_ATOMIC_H
+
+#include <asm-generic/atomic.h>
+
+#ifndef CONFIG_GENERIC_ATOMIC64
+#include "atomic64.h"
+#endif /* !CONFIG_GENERIC_ATOMIC64 */
+
+#endif
diff --git a/arch/um/nommu/include/asm/atomic64.h b/arch/um/nommu/include/asm/atomic64.h
new file mode 100644
index 000000000000..949360dea7af
--- /dev/null
+++ b/arch/um/nommu/include/asm/atomic64.h
@@ -0,0 +1,114 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_ATOMIC64_H
+#define __UM_NOMMU_ATOMIC64_H
+
+#include <linux/types.h>
+
+#ifdef CONFIG_SMP
+#error "SMP is not supported on this platform"
+#else
+#define ATOMIC64_OP(op, c_op)					\
+	static inline void atomic64_##op(s64 i, atomic64_t *v)	\
+	{							\
+		unsigned long flags;				\
+								\
+		raw_local_irq_save(flags);			\
+		v->counter = v->counter c_op i;			\
+		raw_local_irq_restore(flags);			\
+	}
+
+#define ATOMIC64_OP_RETURN(op, c_op)					\
+	static inline s64 atomic64_##op##_return(s64 i, atomic64_t *v)	\
+	{								\
+		unsigned long flags;					\
+		s64 ret;						\
+									\
+		raw_local_irq_save(flags);				\
+		ret = (v->counter = v->counter c_op i);			\
+		raw_local_irq_restore(flags);				\
+									\
+		return ret;						\
+	}
+
+#define ATOMIC64_FETCH_OP(op, c_op)					\
+	static inline s64 atomic64_fetch_##op(s64 i, atomic64_t *v)	\
+	{								\
+		unsigned long flags;					\
+		s64 ret;						\
+									\
+		raw_local_irq_save(flags);				\
+		ret = v->counter;					\
+		v->counter = v->counter c_op i;				\
+		raw_local_irq_restore(flags);				\
+									\
+		return ret;						\
+	}
+#endif /* CONFIG_SMP */
+
+#ifndef atomic64_add_return
+ATOMIC64_OP_RETURN(add, +)
+#endif
+
+#ifndef atomic64_sub_return
+	ATOMIC64_OP_RETURN(sub, -)
+#endif
+
+#ifndef atomic64_fetch_add
+	ATOMIC64_FETCH_OP(add, +)
+#endif
+
+#ifndef atomic64_fetch_sub
+	ATOMIC64_FETCH_OP(sub, -)
+#endif
+
+#ifndef atomic64_fetch_and
+	ATOMIC64_FETCH_OP(and, &)
+#endif
+
+#ifndef atomic64_fetch_or
+	ATOMIC64_FETCH_OP(or, |)
+#endif
+
+#ifndef atomic64_fetch_xor
+	ATOMIC64_FETCH_OP(xor, ^)
+#endif
+
+#ifndef atomic64_and
+	ATOMIC64_OP(and, &)
+#endif
+
+#ifndef atomic64_or
+	ATOMIC64_OP(or, |)
+#endif
+
+#ifndef atomic64_xor
+	ATOMIC64_OP(xor, ^)
+#endif
+
+#undef ATOMIC64_FETCH_OP
+#undef ATOMIC64_OP_RETURN
+#undef ATOMIC64_OP
+
+
+#define ATOMIC64_INIT(i)       { (i) }
+
+	static inline void atomic64_add(s64 i, atomic64_t *v)
+{
+	atomic64_add_return(i, v);
+}
+
+static inline void atomic64_sub(s64 i, atomic64_t *v)
+{
+	atomic64_sub_return(i, v);
+}
+
+#ifndef atomic64_read
+#define atomic64_read(v)       READ_ONCE((v)->counter)
+#endif
+
+#define atomic64_set(v, i) WRITE_ONCE(((v)->counter), (i))
+
+#define atomic64_xchg(ptr, v)          (xchg(&(ptr)->counter, (v)))
+#define atomic64_cmpxchg(v, old, new)  (cmpxchg(&((v)->counter), (old), (new)))
+
+#endif /* __LKL_ATOMIC64_H */
diff --git a/arch/um/nommu/include/asm/bitsperlong.h b/arch/um/nommu/include/asm/bitsperlong.h
new file mode 100644
index 000000000000..a150cee41e75
--- /dev/null
+++ b/arch/um/nommu/include/asm/bitsperlong.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_BITSPERLONG_H
+#define __UM_NOMMU_BITSPERLONG_H
+
+#include <uapi/asm/bitsperlong.h>
+
+#define BITS_PER_LONG __BITS_PER_LONG
+
+#define BITS_PER_LONG_LONG 64
+
+#endif
+
diff --git a/arch/um/nommu/include/asm/byteorder.h b/arch/um/nommu/include/asm/byteorder.h
new file mode 100644
index 000000000000..920a5fd26cad
--- /dev/null
+++ b/arch/um/nommu/include/asm/byteorder.h
@@ -0,0 +1,7 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_BYTEORDER_H
+#define __UM_NOMMU_BYTEORDER_H
+
+#include <uapi/asm/byteorder.h>
+
+#endif
diff --git a/arch/um/nommu/include/asm/cpu.h b/arch/um/nommu/include/asm/cpu.h
new file mode 100644
index 000000000000..c101c078ef21
--- /dev/null
+++ b/arch/um/nommu/include/asm/cpu.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_CPU_H
+#define __UM_NOMMU_CPU_H
+
+int lkl_cpu_get(void);
+void lkl_cpu_put(void);
+int lkl_cpu_try_run_irq(int irq);
+int lkl_cpu_init(void);
+void lkl_cpu_wait_shutdown(void);
+void lkl_cpu_change_owner(lkl_thread_t owner);
+void lkl_cpu_set_irqs_pending(void);
+
+#endif
diff --git a/arch/um/nommu/include/asm/elf.h b/arch/um/nommu/include/asm/elf.h
new file mode 100644
index 000000000000..edcf63edeed1
--- /dev/null
+++ b/arch/um/nommu/include/asm/elf.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_ELF_H
+#define __UM_NOMMU_ELF_H
+
+#define elf_check_arch(x) 0
+
+#ifdef CONFIG_64BIT
+#define ELF_CLASS ELFCLASS64
+#else
+#define ELF_CLASS ELFCLASS32
+#endif
+
+#define elf_gregset_t long
+#define elf_fpregset_t double
+#endif
diff --git a/arch/um/nommu/include/asm/mm_context.h b/arch/um/nommu/include/asm/mm_context.h
new file mode 100644
index 000000000000..a2e53984aabd
--- /dev/null
+++ b/arch/um/nommu/include/asm/mm_context.h
@@ -0,0 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_MM_CONTEXT_H
+#define __UM_NOMMU_MM_CONTEXT_H
+
+struct uml_arch_mm_context {
+};
+
+#endif
diff --git a/arch/um/nommu/include/asm/processor.h b/arch/um/nommu/include/asm/processor.h
new file mode 100644
index 000000000000..3e8ba870caaf
--- /dev/null
+++ b/arch/um/nommu/include/asm/processor.h
@@ -0,0 +1,46 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_PROCESSOR_H
+#define __UM_NOMMU_PROCESSOR_H
+
+#include <asm/host_ops.h>
+
+struct arch_thread {
+	struct lkl_sem *sched_sem;
+	bool dead;
+	lkl_thread_t tid;
+	struct lkl_jmp_buf sched_jb;
+	unsigned long stackend;
+};
+
+#include <asm/ptrace-generic.h>
+#include <asm/processor-generic.h>
+
+#define INIT_ARCH_THREAD {}
+#define task_pt_regs(tsk) (struct pt_regs *)(NULL)
+
+static inline void cpu_relax(void)
+{
+	unsigned long flags;
+
+	/* since this is usually called in a tight loop waiting for some
+	 * external condition (e.g. jiffies) lets run interrupts now to allow
+	 * the external condition to propagate
+	 */
+	local_irq_save(flags);
+	local_irq_restore(flags);
+}
+
+#define KSTK_EIP(tsk)	(0)
+#define KSTK_ESP(tsk)	(0)
+
+static inline void trap_init(void)
+{
+}
+
+static inline void arch_copy_thread(struct arch_thread *from,
+				    struct arch_thread *to)
+{
+	panic("unimplemented %s: fork isn't supported yet", __func__);
+}
+
+#endif
diff --git a/arch/um/nommu/include/asm/ptrace.h b/arch/um/nommu/include/asm/ptrace.h
new file mode 100644
index 000000000000..b214410e9825
--- /dev/null
+++ b/arch/um/nommu/include/asm/ptrace.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_PTRACE_H
+#define __UM_NOMMU_PTRACE_H
+
+#include <linux/errno.h>
+
+static int reg_dummy __attribute__((unused));
+
+#define PT_REGS_ORIG_SYSCALL(r) (reg_dummy)
+#define PT_REGS_SYSCALL_RET(r) (reg_dummy)
+#define PT_REGS_SET_SYSCALL_RETURN(r, res) (reg_dummy = (res))
+#define REGS_SP(r) (reg_dummy)
+
+#define user_mode(regs) 0
+#define kernel_mode(regs) 1
+#define profile_pc(regs) 0
+#define user_stack_pointer(regs) 0
+
+extern void new_thread_handler(void);
+
+#endif
diff --git a/arch/um/nommu/include/asm/sched.h b/arch/um/nommu/include/asm/sched.h
new file mode 100644
index 000000000000..a4496f482633
--- /dev/null
+++ b/arch/um/nommu/include/asm/sched.h
@@ -0,0 +1,23 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_SCHED_H
+#define __UM_NOMMU_SCHED_H
+
+#include <linux/sched.h>
+#include <uapi/asm/host_ops.h>
+
+static inline void thread_sched_jb(void)
+{
+	if (test_ti_thread_flag(current_thread_info(), TIF_HOST_THREAD)) {
+		set_ti_thread_flag(current_thread_info(), TIF_SCHED_JB);
+		set_current_state(TASK_UNINTERRUPTIBLE);
+		lkl_ops->jmp_buf_set(&current_thread_info()->task->thread.arch.sched_jb,
+				     schedule);
+	} else {
+		lkl_bug("%s can be used only for host task", __func__);
+	}
+}
+
+void switch_to_host_task(struct task_struct *);
+int host_task_stub(void *unused);
+
+#endif
diff --git a/arch/um/nommu/include/asm/segment.h b/arch/um/nommu/include/asm/segment.h
new file mode 100644
index 000000000000..5608da95cb60
--- /dev/null
+++ b/arch/um/nommu/include/asm/segment.h
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_SEGMENT_H
+#define __UM_NOMMU_SEGMENT_H
+
+typedef struct {
+	unsigned long seg;
+} mm_segment_t;
+
+#endif /* _ASM_LKL_SEGMENT_H */
diff --git a/arch/um/nommu/include/uapi/asm/Kbuild b/arch/um/nommu/include/uapi/asm/Kbuild
new file mode 100644
index 000000000000..0d55c408dac2
--- /dev/null
+++ b/arch/um/nommu/include/uapi/asm/Kbuild
@@ -0,0 +1,6 @@
+# UAPI Header export list
+
+generated-y += syscall_defs.h
+
+# no generated-y since we need special user headers handling
+# see arch/um/script/headers_install.py
diff --git a/arch/um/nommu/include/uapi/asm/bitsperlong.h b/arch/um/nommu/include/uapi/asm/bitsperlong.h
new file mode 100644
index 000000000000..852566ac2e52
--- /dev/null
+++ b/arch/um/nommu/include/uapi/asm/bitsperlong.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef __UM_NOMMU_UAPI_BITSPERLONG_H
+#define __UM_NOMMU_UAPI_BITSPERLONG_H
+
+#ifdef CONFIG_64BIT
+#define __BITS_PER_LONG 64
+#else
+#define __BITS_PER_LONG 32
+#endif
+
+#endif
diff --git a/arch/um/nommu/include/uapi/asm/byteorder.h b/arch/um/nommu/include/uapi/asm/byteorder.h
new file mode 100644
index 000000000000..e7ad11a751cf
--- /dev/null
+++ b/arch/um/nommu/include/uapi/asm/byteorder.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef __UM_NOMMU_UAPI_BYTEORDER_H
+#define __UM_NOMMU_UAPI_BYTEORDER_H
+
+#if defined(CONFIG_BIG_ENDIAN)
+#include <linux/byteorder/big_endian.h>
+#else
+#include <linux/byteorder/little_endian.h>
+#endif
+
+#endif
diff --git a/arch/um/nommu/include/uapi/asm/sigcontext.h b/arch/um/nommu/include/uapi/asm/sigcontext.h
new file mode 100644
index 000000000000..b934ae7f5550
--- /dev/null
+++ b/arch/um/nommu/include/uapi/asm/sigcontext.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef __UM_NOMMU_UAPI_SIGCONTEXT_H
+#define __UM_NOMMU_UAPI_SIGCONTEXT_H
+
+#include <asm/ptrace-generic.h>
+
+struct sigcontext {
+	struct pt_regs regs;
+	unsigned long oldmask;
+};
+
+#endif
diff --git a/arch/um/nommu/um/delay.c b/arch/um/nommu/um/delay.c
new file mode 100644
index 000000000000..58a366d9b5f0
--- /dev/null
+++ b/arch/um/nommu/um/delay.c
@@ -0,0 +1,31 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/jiffies.h>
+#include <linux/delay.h>
+#include <os.h>
+
+void __ndelay(unsigned long nsecs)
+{
+	long long start = os_nsecs();
+
+	while (os_nsecs() < start + nsecs)
+		;
+}
+
+void __udelay(unsigned long usecs)
+{
+	__ndelay(usecs * NSEC_PER_USEC);
+}
+
+void __const_udelay(unsigned long xloops)
+{
+	__udelay(xloops / 0x10c7ul);
+}
+
+void __delay(unsigned long loops)
+{
+	__ndelay(loops / 5);
+}
+
+void calibrate_delay(void)
+{
+}
diff --git a/arch/um/nommu/um/shared/sysdep/archsetjmp.h b/arch/um/nommu/um/shared/sysdep/archsetjmp.h
new file mode 100644
index 000000000000..4a5c10d7521b
--- /dev/null
+++ b/arch/um/nommu/um/shared/sysdep/archsetjmp.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ARCH_UM_SETJMP_H
+#define __ARCH_UM_SETJMP_H
+
+struct __jmp_buf {
+	unsigned long __dummy;
+};
+#define JB_IP __dummy
+#define JB_SP __dummy
+
+typedef struct __jmp_buf jmp_buf[1];
+
+#endif /* __ARCH_UM_SETJMP_H */
diff --git a/arch/um/nommu/um/shared/sysdep/faultinfo.h b/arch/um/nommu/um/shared/sysdep/faultinfo.h
new file mode 100644
index 000000000000..49210742212d
--- /dev/null
+++ b/arch/um/nommu/um/shared/sysdep/faultinfo.h
@@ -0,0 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ARCH_UM_FAULTINFO_H
+#define __ARCH_UM_FAULTINFO_H
+
+struct faultinfo {
+};
+
+#endif /* __ARCH_UM_FAULTINFO_H */
diff --git a/arch/um/nommu/um/shared/sysdep/kernel-offsets.h b/arch/um/nommu/um/shared/sysdep/kernel-offsets.h
new file mode 100644
index 000000000000..a004bffb7b8d
--- /dev/null
+++ b/arch/um/nommu/um/shared/sysdep/kernel-offsets.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <linux/stddef.h>
+#include <linux/sched.h>
+#include <linux/elf.h>
+#include <linux/crypto.h>
+#include <linux/kbuild.h>
+#include <asm/mman.h>
+
+void foo(void)
+{
+#include <common-offsets.h>
+}
diff --git a/arch/um/nommu/um/shared/sysdep/mcontext.h b/arch/um/nommu/um/shared/sysdep/mcontext.h
new file mode 100644
index 000000000000..b734012a74da
--- /dev/null
+++ b/arch/um/nommu/um/shared/sysdep/mcontext.h
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ARCH_UM_MCONTEXT_H
+#define __ARCH_UM_MCONTEXT_H
+
+extern void get_regs_from_mc(struct uml_pt_regs *regs, mcontext_t *mc);
+
+#define GET_FAULTINFO_FROM_MC(fi, mc) (fi = fi)
+
+#endif /* __ARCH_UM_MCONTEXT_H */
diff --git a/arch/um/nommu/um/shared/sysdep/ptrace.h b/arch/um/nommu/um/shared/sysdep/ptrace.h
new file mode 100644
index 000000000000..bfdfb520a21d
--- /dev/null
+++ b/arch/um/nommu/um/shared/sysdep/ptrace.h
@@ -0,0 +1,42 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ARCH_UM_PTRACE_H
+#define __ARCH_UM_PTRACE_H
+
+#include <generated/user_constants.h>
+#include <linux/errno.h>
+
+struct task_struct;
+
+#define UPT_SYSCALL_NR(r) ((r)->syscall)
+#define UPT_RESTART_SYSCALL(r) ((r)->syscall--) /* XXX */
+
+#define UPT_SP(r) 0
+#define UPT_IP(r) 0
+#define EMPTY_UML_PT_REGS { }
+
+#define MAX_REG_OFFSET (UM_FRAME_SIZE)
+#define MAX_REG_NR ((MAX_REG_OFFSET) / sizeof(unsigned long))
+
+/* unused */
+struct uml_pt_regs {
+	unsigned long gp[1];
+	unsigned long fp[1];
+	long faultinfo;
+	long syscall;
+	int is_user;
+};
+
+extern void arch_init_registers(int pid);
+
+static inline long arch_ptrace(struct task_struct *child,
+			       long request, unsigned long addr,
+			       unsigned long data)
+{
+	return -EINVAL;
+}
+
+static inline void ptrace_disable(struct task_struct *child) {}
+static inline void user_enable_single_step(struct task_struct *child) {}
+static inline void user_disable_single_step(struct task_struct *child) {}
+
+#endif /* __ARCH_UM_PTRACE_H */
diff --git a/arch/um/nommu/um/shared/sysdep/ptrace_user.h b/arch/um/nommu/um/shared/sysdep/ptrace_user.h
new file mode 100644
index 000000000000..86d5cb20fb9f
--- /dev/null
+++ b/arch/um/nommu/um/shared/sysdep/ptrace_user.h
@@ -0,0 +1,7 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ARCH_UM_PTRACE_USER_H
+#define __ARCH_UM_PTRACE_USER_H
+
+#define FP_SIZE 1
+
+#endif
diff --git a/arch/um/nommu/um/unimplemented.c b/arch/um/nommu/um/unimplemented.c
new file mode 100644
index 000000000000..fe33e02e39e5
--- /dev/null
+++ b/arch/um/nommu/um/unimplemented.c
@@ -0,0 +1,70 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/signal.h>
+#include <sysdep/ptrace.h>
+#include <asm/ptrace.h>
+
+/* physmem.c  */
+unsigned long high_physmem;
+
+/* x86/um/setjmp*.S  */
+void kernel_longjmp(void)
+{}
+void kernel_setjmp(void)
+{}
+
+/* trap.c */
+void relay_signal(int sig, struct siginfo *si, struct uml_pt_regs *regs)
+{}
+void bus_handler(int sig, struct siginfo *si, struct uml_pt_regs *regs)
+{}
+void segv_handler(int sig, struct siginfo *unused_si, struct uml_pt_regs *regs)
+{}
+void winch(int sig, struct siginfo *unused_si, struct uml_pt_regs *regs)
+{}
+
+/* tlb.c */
+void flush_tlb_kernel_vm(void)
+{}
+void force_flush_all(void)
+{}
+
+/* skas/process.c */
+void halt_skas(void)
+{}
+int is_skas_winch(int pid, int fd, void *data)
+{
+	return 0;
+}
+void reboot_skas(void)
+{}
+
+int __init start_uml(void)
+{
+	return 0;
+}
+
+/* exec.c */
+void flush_thread(void)
+{}
+
+/* x86/ptrace_64.c */
+int is_syscall(unsigned long addr)
+{
+	return 0;
+}
+
+
+/* x86/sysrq.c */
+void show_regs(struct pt_regs *regs)
+{}
+
+/* x86/signal.c */
+int setup_signal_stack_si(unsigned long stack_top, struct ksignal *ksig,
+			  struct pt_regs *regs, sigset_t *mask)
+{
+	return 0;
+}
+
+/* x86/bugs.c */
+void arch_check_bugs(void)
+{}
diff --git a/arch/um/nommu/um/user_constants.h b/arch/um/nommu/um/user_constants.h
new file mode 100644
index 000000000000..2245d3d24120
--- /dev/null
+++ b/arch/um/nommu/um/user_constants.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_USER_CONSTANTS_H
+#define __UM_NOMMU_USER_CONSTANTS_H
+
+/* XXX: put dummy values */
+#define UM_FRAME_SIZE 4
+#define HOST_FP_SIZE 1
+#define HOST_IP 1
+#define HOST_SP 2
+#define HOST_BP 3
+#define UM_NR_CPUS 1
+
+#endif
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v7 08/21] um: add nommu mode for UML library mode
@ 2020-10-06  9:44           ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-06  9:44 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Hajime Tazaki, retrage01

This patch introduces the nommu operation with UML code so that host
interface can be shrineked for broader environments support.

The nommu mode is implemneted as SUBARCH of arch/um, which places at
arch/um/nommu. This SUBARCH defines mode-specific code of memory
management as well as thread implementation, along with the uapi headers
to be exported to users.

The headers we introduce in this patch are simple wrappers to the
asm-generic headers or stubs for things we don't support, such as
ptrace, DMA, signals, ELF handling and low level processor operations.

nommu mode shares most of arch/um code (irq, thread_info, process
scheduling) but implements its own facilities, which are memory
management (nommu), thread primitives (struct arch_thread), system call
interface, and console.

The outlook of updated directory structure is as follows:

   arch/um
   |-- configs         (untouched)
   |-- drivers         unuse stdio_console.c for !MMU
   |-- include
   |   |-- asm         updated with !CONFIG_MMU
   |   |-- linux       (untouched)
   |   `-- shared      updated with new functions
   |       `-- skas    (untouched)
   |   `-- uapi        added for upai header installation
   |-- kernel          updated to integrate with !MMU
   |   `-- skas        (untouched, don't use for !MMU)
   |-- nommu           SUBARCH dir (internally =um/nommu)
   |   |-- include
   |   |   |-- asm     headers for subarch specific info
   |   |   `-- uapi    headers for user-visible APIs
   |   `-- um
   |       `-- shared  headers for subarch specific info
   `-- scripts         added a script for header installation

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/nommu/Makefile                        |   1 +
 arch/um/nommu/Makefile.um                     |  22 ++++
 arch/um/nommu/include/asm/Kbuild              |   6 +
 arch/um/nommu/include/asm/archparam.h         |   1 +
 arch/um/nommu/include/asm/atomic.h            |  11 ++
 arch/um/nommu/include/asm/atomic64.h          | 114 ++++++++++++++++++
 arch/um/nommu/include/asm/bitsperlong.h       |  12 ++
 arch/um/nommu/include/asm/byteorder.h         |   7 ++
 arch/um/nommu/include/asm/cpu.h               |  13 ++
 arch/um/nommu/include/asm/elf.h               |  15 +++
 arch/um/nommu/include/asm/mm_context.h        |   8 ++
 arch/um/nommu/include/asm/processor.h         |  46 +++++++
 arch/um/nommu/include/asm/ptrace.h            |  21 ++++
 arch/um/nommu/include/asm/sched.h             |  23 ++++
 arch/um/nommu/include/asm/segment.h           |   9 ++
 arch/um/nommu/include/uapi/asm/Kbuild         |   6 +
 arch/um/nommu/include/uapi/asm/bitsperlong.h  |  11 ++
 arch/um/nommu/include/uapi/asm/byteorder.h    |  11 ++
 arch/um/nommu/include/uapi/asm/sigcontext.h   |  12 ++
 arch/um/nommu/um/delay.c                      |  31 +++++
 arch/um/nommu/um/shared/sysdep/archsetjmp.h   |  13 ++
 arch/um/nommu/um/shared/sysdep/faultinfo.h    |   8 ++
 .../nommu/um/shared/sysdep/kernel-offsets.h   |  12 ++
 arch/um/nommu/um/shared/sysdep/mcontext.h     |   9 ++
 arch/um/nommu/um/shared/sysdep/ptrace.h       |  42 +++++++
 arch/um/nommu/um/shared/sysdep/ptrace_user.h  |   7 ++
 arch/um/nommu/um/unimplemented.c              |  70 +++++++++++
 arch/um/nommu/um/user_constants.h             |  13 ++
 28 files changed, 554 insertions(+)
 create mode 100644 arch/um/nommu/Makefile
 create mode 100644 arch/um/nommu/Makefile.um
 create mode 100644 arch/um/nommu/include/asm/Kbuild
 create mode 100644 arch/um/nommu/include/asm/archparam.h
 create mode 100644 arch/um/nommu/include/asm/atomic.h
 create mode 100644 arch/um/nommu/include/asm/atomic64.h
 create mode 100644 arch/um/nommu/include/asm/bitsperlong.h
 create mode 100644 arch/um/nommu/include/asm/byteorder.h
 create mode 100644 arch/um/nommu/include/asm/cpu.h
 create mode 100644 arch/um/nommu/include/asm/elf.h
 create mode 100644 arch/um/nommu/include/asm/mm_context.h
 create mode 100644 arch/um/nommu/include/asm/processor.h
 create mode 100644 arch/um/nommu/include/asm/ptrace.h
 create mode 100644 arch/um/nommu/include/asm/sched.h
 create mode 100644 arch/um/nommu/include/asm/segment.h
 create mode 100644 arch/um/nommu/include/uapi/asm/Kbuild
 create mode 100644 arch/um/nommu/include/uapi/asm/bitsperlong.h
 create mode 100644 arch/um/nommu/include/uapi/asm/byteorder.h
 create mode 100644 arch/um/nommu/include/uapi/asm/sigcontext.h
 create mode 100644 arch/um/nommu/um/delay.c
 create mode 100644 arch/um/nommu/um/shared/sysdep/archsetjmp.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/faultinfo.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/kernel-offsets.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/mcontext.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/ptrace.h
 create mode 100644 arch/um/nommu/um/shared/sysdep/ptrace_user.h
 create mode 100644 arch/um/nommu/um/unimplemented.c
 create mode 100644 arch/um/nommu/um/user_constants.h

diff --git a/arch/um/nommu/Makefile b/arch/um/nommu/Makefile
new file mode 100644
index 000000000000..792d6005489e
--- /dev/null
+++ b/arch/um/nommu/Makefile
@@ -0,0 +1 @@
+#
diff --git a/arch/um/nommu/Makefile.um b/arch/um/nommu/Makefile.um
new file mode 100644
index 000000000000..3808462d8283
--- /dev/null
+++ b/arch/um/nommu/Makefile.um
@@ -0,0 +1,22 @@
+KBUILD_CFLAGS += -fno-builtin -fPIC
+ELF_FORMAT=$(shell $(LD) -r -print-output-format)
+
+ifeq ($(shell uname -s), Linux)
+NPROC=$(shell nproc)
+else # e.g., FreeBSD
+NPROC=$(shell sysctl -n hw.ncpu)
+endif
+
+um_headers_install: $(objtree)/$(HOST_DIR)/include/generated/uapi/asm/syscall_defs.h headers
+# if syscall_defs.h is newer than generated headers (autoconf.h)
+	$(Q)if [ ! -r $(objtree)/tools/um/include/lkl/autoconf.h ] || \
+	  [ $< -nt $(objtree)/tools/um/include/lkl/autoconf.h ]; then \
+		$(srctree)/$(ARCH_DIR)/scripts/headers_install.py \
+			$(subst -j,-j$(NPROC),$(findstring -j,$(MAKEFLAGS))) \
+			$(INSTALL_PATH)/include ; \
+	fi
+
+$(objtree)/$(HOST_DIR)/include/generated/uapi/asm/syscall_defs.h: vmlinux
+	$(Q)$(OBJCOPY) -j .syscall_defs -O binary --set-section-flags .syscall_defs=alloc $< $@
+	$(Q) export tmpfile=$(shell mktemp); \
+	sed 's/\x0//g' $@ > $$tmpfile; mv $$tmpfile $@ ; rm -f $$tmpfile
diff --git a/arch/um/nommu/include/asm/Kbuild b/arch/um/nommu/include/asm/Kbuild
new file mode 100644
index 000000000000..2532e1a0a0d1
--- /dev/null
+++ b/arch/um/nommu/include/asm/Kbuild
@@ -0,0 +1,6 @@
+generic-y += cmpxchg.h
+generic-y += local64.h
+generic-y += seccomp.h
+generic-y += string.h
+generic-y += syscall.h
+generic-y += user.h
diff --git a/arch/um/nommu/include/asm/archparam.h b/arch/um/nommu/include/asm/archparam.h
new file mode 100644
index 000000000000..ea32a7d3cf1b
--- /dev/null
+++ b/arch/um/nommu/include/asm/archparam.h
@@ -0,0 +1 @@
+/* SPDX-License-Identifier: GPL-2.0 */
diff --git a/arch/um/nommu/include/asm/atomic.h b/arch/um/nommu/include/asm/atomic.h
new file mode 100644
index 000000000000..63e2e16bda92
--- /dev/null
+++ b/arch/um/nommu/include/asm/atomic.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_ATOMIC_H
+#define __UM_NOMMU_ATOMIC_H
+
+#include <asm-generic/atomic.h>
+
+#ifndef CONFIG_GENERIC_ATOMIC64
+#include "atomic64.h"
+#endif /* !CONFIG_GENERIC_ATOMIC64 */
+
+#endif
diff --git a/arch/um/nommu/include/asm/atomic64.h b/arch/um/nommu/include/asm/atomic64.h
new file mode 100644
index 000000000000..949360dea7af
--- /dev/null
+++ b/arch/um/nommu/include/asm/atomic64.h
@@ -0,0 +1,114 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_ATOMIC64_H
+#define __UM_NOMMU_ATOMIC64_H
+
+#include <linux/types.h>
+
+#ifdef CONFIG_SMP
+#error "SMP is not supported on this platform"
+#else
+#define ATOMIC64_OP(op, c_op)					\
+	static inline void atomic64_##op(s64 i, atomic64_t *v)	\
+	{							\
+		unsigned long flags;				\
+								\
+		raw_local_irq_save(flags);			\
+		v->counter = v->counter c_op i;			\
+		raw_local_irq_restore(flags);			\
+	}
+
+#define ATOMIC64_OP_RETURN(op, c_op)					\
+	static inline s64 atomic64_##op##_return(s64 i, atomic64_t *v)	\
+	{								\
+		unsigned long flags;					\
+		s64 ret;						\
+									\
+		raw_local_irq_save(flags);				\
+		ret = (v->counter = v->counter c_op i);			\
+		raw_local_irq_restore(flags);				\
+									\
+		return ret;						\
+	}
+
+#define ATOMIC64_FETCH_OP(op, c_op)					\
+	static inline s64 atomic64_fetch_##op(s64 i, atomic64_t *v)	\
+	{								\
+		unsigned long flags;					\
+		s64 ret;						\
+									\
+		raw_local_irq_save(flags);				\
+		ret = v->counter;					\
+		v->counter = v->counter c_op i;				\
+		raw_local_irq_restore(flags);				\
+									\
+		return ret;						\
+	}
+#endif /* CONFIG_SMP */
+
+#ifndef atomic64_add_return
+ATOMIC64_OP_RETURN(add, +)
+#endif
+
+#ifndef atomic64_sub_return
+	ATOMIC64_OP_RETURN(sub, -)
+#endif
+
+#ifndef atomic64_fetch_add
+	ATOMIC64_FETCH_OP(add, +)
+#endif
+
+#ifndef atomic64_fetch_sub
+	ATOMIC64_FETCH_OP(sub, -)
+#endif
+
+#ifndef atomic64_fetch_and
+	ATOMIC64_FETCH_OP(and, &)
+#endif
+
+#ifndef atomic64_fetch_or
+	ATOMIC64_FETCH_OP(or, |)
+#endif
+
+#ifndef atomic64_fetch_xor
+	ATOMIC64_FETCH_OP(xor, ^)
+#endif
+
+#ifndef atomic64_and
+	ATOMIC64_OP(and, &)
+#endif
+
+#ifndef atomic64_or
+	ATOMIC64_OP(or, |)
+#endif
+
+#ifndef atomic64_xor
+	ATOMIC64_OP(xor, ^)
+#endif
+
+#undef ATOMIC64_FETCH_OP
+#undef ATOMIC64_OP_RETURN
+#undef ATOMIC64_OP
+
+
+#define ATOMIC64_INIT(i)       { (i) }
+
+	static inline void atomic64_add(s64 i, atomic64_t *v)
+{
+	atomic64_add_return(i, v);
+}
+
+static inline void atomic64_sub(s64 i, atomic64_t *v)
+{
+	atomic64_sub_return(i, v);
+}
+
+#ifndef atomic64_read
+#define atomic64_read(v)       READ_ONCE((v)->counter)
+#endif
+
+#define atomic64_set(v, i) WRITE_ONCE(((v)->counter), (i))
+
+#define atomic64_xchg(ptr, v)          (xchg(&(ptr)->counter, (v)))
+#define atomic64_cmpxchg(v, old, new)  (cmpxchg(&((v)->counter), (old), (new)))
+
+#endif /* __LKL_ATOMIC64_H */
diff --git a/arch/um/nommu/include/asm/bitsperlong.h b/arch/um/nommu/include/asm/bitsperlong.h
new file mode 100644
index 000000000000..a150cee41e75
--- /dev/null
+++ b/arch/um/nommu/include/asm/bitsperlong.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_BITSPERLONG_H
+#define __UM_NOMMU_BITSPERLONG_H
+
+#include <uapi/asm/bitsperlong.h>
+
+#define BITS_PER_LONG __BITS_PER_LONG
+
+#define BITS_PER_LONG_LONG 64
+
+#endif
+
diff --git a/arch/um/nommu/include/asm/byteorder.h b/arch/um/nommu/include/asm/byteorder.h
new file mode 100644
index 000000000000..920a5fd26cad
--- /dev/null
+++ b/arch/um/nommu/include/asm/byteorder.h
@@ -0,0 +1,7 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_BYTEORDER_H
+#define __UM_NOMMU_BYTEORDER_H
+
+#include <uapi/asm/byteorder.h>
+
+#endif
diff --git a/arch/um/nommu/include/asm/cpu.h b/arch/um/nommu/include/asm/cpu.h
new file mode 100644
index 000000000000..c101c078ef21
--- /dev/null
+++ b/arch/um/nommu/include/asm/cpu.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_CPU_H
+#define __UM_NOMMU_CPU_H
+
+int lkl_cpu_get(void);
+void lkl_cpu_put(void);
+int lkl_cpu_try_run_irq(int irq);
+int lkl_cpu_init(void);
+void lkl_cpu_wait_shutdown(void);
+void lkl_cpu_change_owner(lkl_thread_t owner);
+void lkl_cpu_set_irqs_pending(void);
+
+#endif
diff --git a/arch/um/nommu/include/asm/elf.h b/arch/um/nommu/include/asm/elf.h
new file mode 100644
index 000000000000..edcf63edeed1
--- /dev/null
+++ b/arch/um/nommu/include/asm/elf.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_ELF_H
+#define __UM_NOMMU_ELF_H
+
+#define elf_check_arch(x) 0
+
+#ifdef CONFIG_64BIT
+#define ELF_CLASS ELFCLASS64
+#else
+#define ELF_CLASS ELFCLASS32
+#endif
+
+#define elf_gregset_t long
+#define elf_fpregset_t double
+#endif
diff --git a/arch/um/nommu/include/asm/mm_context.h b/arch/um/nommu/include/asm/mm_context.h
new file mode 100644
index 000000000000..a2e53984aabd
--- /dev/null
+++ b/arch/um/nommu/include/asm/mm_context.h
@@ -0,0 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_MM_CONTEXT_H
+#define __UM_NOMMU_MM_CONTEXT_H
+
+struct uml_arch_mm_context {
+};
+
+#endif
diff --git a/arch/um/nommu/include/asm/processor.h b/arch/um/nommu/include/asm/processor.h
new file mode 100644
index 000000000000..3e8ba870caaf
--- /dev/null
+++ b/arch/um/nommu/include/asm/processor.h
@@ -0,0 +1,46 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_PROCESSOR_H
+#define __UM_NOMMU_PROCESSOR_H
+
+#include <asm/host_ops.h>
+
+struct arch_thread {
+	struct lkl_sem *sched_sem;
+	bool dead;
+	lkl_thread_t tid;
+	struct lkl_jmp_buf sched_jb;
+	unsigned long stackend;
+};
+
+#include <asm/ptrace-generic.h>
+#include <asm/processor-generic.h>
+
+#define INIT_ARCH_THREAD {}
+#define task_pt_regs(tsk) (struct pt_regs *)(NULL)
+
+static inline void cpu_relax(void)
+{
+	unsigned long flags;
+
+	/* since this is usually called in a tight loop waiting for some
+	 * external condition (e.g. jiffies) lets run interrupts now to allow
+	 * the external condition to propagate
+	 */
+	local_irq_save(flags);
+	local_irq_restore(flags);
+}
+
+#define KSTK_EIP(tsk)	(0)
+#define KSTK_ESP(tsk)	(0)
+
+static inline void trap_init(void)
+{
+}
+
+static inline void arch_copy_thread(struct arch_thread *from,
+				    struct arch_thread *to)
+{
+	panic("unimplemented %s: fork isn't supported yet", __func__);
+}
+
+#endif
diff --git a/arch/um/nommu/include/asm/ptrace.h b/arch/um/nommu/include/asm/ptrace.h
new file mode 100644
index 000000000000..b214410e9825
--- /dev/null
+++ b/arch/um/nommu/include/asm/ptrace.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_PTRACE_H
+#define __UM_NOMMU_PTRACE_H
+
+#include <linux/errno.h>
+
+static int reg_dummy __attribute__((unused));
+
+#define PT_REGS_ORIG_SYSCALL(r) (reg_dummy)
+#define PT_REGS_SYSCALL_RET(r) (reg_dummy)
+#define PT_REGS_SET_SYSCALL_RETURN(r, res) (reg_dummy = (res))
+#define REGS_SP(r) (reg_dummy)
+
+#define user_mode(regs) 0
+#define kernel_mode(regs) 1
+#define profile_pc(regs) 0
+#define user_stack_pointer(regs) 0
+
+extern void new_thread_handler(void);
+
+#endif
diff --git a/arch/um/nommu/include/asm/sched.h b/arch/um/nommu/include/asm/sched.h
new file mode 100644
index 000000000000..a4496f482633
--- /dev/null
+++ b/arch/um/nommu/include/asm/sched.h
@@ -0,0 +1,23 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_SCHED_H
+#define __UM_NOMMU_SCHED_H
+
+#include <linux/sched.h>
+#include <uapi/asm/host_ops.h>
+
+static inline void thread_sched_jb(void)
+{
+	if (test_ti_thread_flag(current_thread_info(), TIF_HOST_THREAD)) {
+		set_ti_thread_flag(current_thread_info(), TIF_SCHED_JB);
+		set_current_state(TASK_UNINTERRUPTIBLE);
+		lkl_ops->jmp_buf_set(&current_thread_info()->task->thread.arch.sched_jb,
+				     schedule);
+	} else {
+		lkl_bug("%s can be used only for host task", __func__);
+	}
+}
+
+void switch_to_host_task(struct task_struct *);
+int host_task_stub(void *unused);
+
+#endif
diff --git a/arch/um/nommu/include/asm/segment.h b/arch/um/nommu/include/asm/segment.h
new file mode 100644
index 000000000000..5608da95cb60
--- /dev/null
+++ b/arch/um/nommu/include/asm/segment.h
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_SEGMENT_H
+#define __UM_NOMMU_SEGMENT_H
+
+typedef struct {
+	unsigned long seg;
+} mm_segment_t;
+
+#endif /* _ASM_LKL_SEGMENT_H */
diff --git a/arch/um/nommu/include/uapi/asm/Kbuild b/arch/um/nommu/include/uapi/asm/Kbuild
new file mode 100644
index 000000000000..0d55c408dac2
--- /dev/null
+++ b/arch/um/nommu/include/uapi/asm/Kbuild
@@ -0,0 +1,6 @@
+# UAPI Header export list
+
+generated-y += syscall_defs.h
+
+# no generated-y since we need special user headers handling
+# see arch/um/script/headers_install.py
diff --git a/arch/um/nommu/include/uapi/asm/bitsperlong.h b/arch/um/nommu/include/uapi/asm/bitsperlong.h
new file mode 100644
index 000000000000..852566ac2e52
--- /dev/null
+++ b/arch/um/nommu/include/uapi/asm/bitsperlong.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef __UM_NOMMU_UAPI_BITSPERLONG_H
+#define __UM_NOMMU_UAPI_BITSPERLONG_H
+
+#ifdef CONFIG_64BIT
+#define __BITS_PER_LONG 64
+#else
+#define __BITS_PER_LONG 32
+#endif
+
+#endif
diff --git a/arch/um/nommu/include/uapi/asm/byteorder.h b/arch/um/nommu/include/uapi/asm/byteorder.h
new file mode 100644
index 000000000000..e7ad11a751cf
--- /dev/null
+++ b/arch/um/nommu/include/uapi/asm/byteorder.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef __UM_NOMMU_UAPI_BYTEORDER_H
+#define __UM_NOMMU_UAPI_BYTEORDER_H
+
+#if defined(CONFIG_BIG_ENDIAN)
+#include <linux/byteorder/big_endian.h>
+#else
+#include <linux/byteorder/little_endian.h>
+#endif
+
+#endif
diff --git a/arch/um/nommu/include/uapi/asm/sigcontext.h b/arch/um/nommu/include/uapi/asm/sigcontext.h
new file mode 100644
index 000000000000..b934ae7f5550
--- /dev/null
+++ b/arch/um/nommu/include/uapi/asm/sigcontext.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef __UM_NOMMU_UAPI_SIGCONTEXT_H
+#define __UM_NOMMU_UAPI_SIGCONTEXT_H
+
+#include <asm/ptrace-generic.h>
+
+struct sigcontext {
+	struct pt_regs regs;
+	unsigned long oldmask;
+};
+
+#endif
diff --git a/arch/um/nommu/um/delay.c b/arch/um/nommu/um/delay.c
new file mode 100644
index 000000000000..58a366d9b5f0
--- /dev/null
+++ b/arch/um/nommu/um/delay.c
@@ -0,0 +1,31 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/jiffies.h>
+#include <linux/delay.h>
+#include <os.h>
+
+void __ndelay(unsigned long nsecs)
+{
+	long long start = os_nsecs();
+
+	while (os_nsecs() < start + nsecs)
+		;
+}
+
+void __udelay(unsigned long usecs)
+{
+	__ndelay(usecs * NSEC_PER_USEC);
+}
+
+void __const_udelay(unsigned long xloops)
+{
+	__udelay(xloops / 0x10c7ul);
+}
+
+void __delay(unsigned long loops)
+{
+	__ndelay(loops / 5);
+}
+
+void calibrate_delay(void)
+{
+}
diff --git a/arch/um/nommu/um/shared/sysdep/archsetjmp.h b/arch/um/nommu/um/shared/sysdep/archsetjmp.h
new file mode 100644
index 000000000000..4a5c10d7521b
--- /dev/null
+++ b/arch/um/nommu/um/shared/sysdep/archsetjmp.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ARCH_UM_SETJMP_H
+#define __ARCH_UM_SETJMP_H
+
+struct __jmp_buf {
+	unsigned long __dummy;
+};
+#define JB_IP __dummy
+#define JB_SP __dummy
+
+typedef struct __jmp_buf jmp_buf[1];
+
+#endif /* __ARCH_UM_SETJMP_H */
diff --git a/arch/um/nommu/um/shared/sysdep/faultinfo.h b/arch/um/nommu/um/shared/sysdep/faultinfo.h
new file mode 100644
index 000000000000..49210742212d
--- /dev/null
+++ b/arch/um/nommu/um/shared/sysdep/faultinfo.h
@@ -0,0 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ARCH_UM_FAULTINFO_H
+#define __ARCH_UM_FAULTINFO_H
+
+struct faultinfo {
+};
+
+#endif /* __ARCH_UM_FAULTINFO_H */
diff --git a/arch/um/nommu/um/shared/sysdep/kernel-offsets.h b/arch/um/nommu/um/shared/sysdep/kernel-offsets.h
new file mode 100644
index 000000000000..a004bffb7b8d
--- /dev/null
+++ b/arch/um/nommu/um/shared/sysdep/kernel-offsets.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <linux/stddef.h>
+#include <linux/sched.h>
+#include <linux/elf.h>
+#include <linux/crypto.h>
+#include <linux/kbuild.h>
+#include <asm/mman.h>
+
+void foo(void)
+{
+#include <common-offsets.h>
+}
diff --git a/arch/um/nommu/um/shared/sysdep/mcontext.h b/arch/um/nommu/um/shared/sysdep/mcontext.h
new file mode 100644
index 000000000000..b734012a74da
--- /dev/null
+++ b/arch/um/nommu/um/shared/sysdep/mcontext.h
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ARCH_UM_MCONTEXT_H
+#define __ARCH_UM_MCONTEXT_H
+
+extern void get_regs_from_mc(struct uml_pt_regs *regs, mcontext_t *mc);
+
+#define GET_FAULTINFO_FROM_MC(fi, mc) (fi = fi)
+
+#endif /* __ARCH_UM_MCONTEXT_H */
diff --git a/arch/um/nommu/um/shared/sysdep/ptrace.h b/arch/um/nommu/um/shared/sysdep/ptrace.h
new file mode 100644
index 000000000000..bfdfb520a21d
--- /dev/null
+++ b/arch/um/nommu/um/shared/sysdep/ptrace.h
@@ -0,0 +1,42 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ARCH_UM_PTRACE_H
+#define __ARCH_UM_PTRACE_H
+
+#include <generated/user_constants.h>
+#include <linux/errno.h>
+
+struct task_struct;
+
+#define UPT_SYSCALL_NR(r) ((r)->syscall)
+#define UPT_RESTART_SYSCALL(r) ((r)->syscall--) /* XXX */
+
+#define UPT_SP(r) 0
+#define UPT_IP(r) 0
+#define EMPTY_UML_PT_REGS { }
+
+#define MAX_REG_OFFSET (UM_FRAME_SIZE)
+#define MAX_REG_NR ((MAX_REG_OFFSET) / sizeof(unsigned long))
+
+/* unused */
+struct uml_pt_regs {
+	unsigned long gp[1];
+	unsigned long fp[1];
+	long faultinfo;
+	long syscall;
+	int is_user;
+};
+
+extern void arch_init_registers(int pid);
+
+static inline long arch_ptrace(struct task_struct *child,
+			       long request, unsigned long addr,
+			       unsigned long data)
+{
+	return -EINVAL;
+}
+
+static inline void ptrace_disable(struct task_struct *child) {}
+static inline void user_enable_single_step(struct task_struct *child) {}
+static inline void user_disable_single_step(struct task_struct *child) {}
+
+#endif /* __ARCH_UM_PTRACE_H */
diff --git a/arch/um/nommu/um/shared/sysdep/ptrace_user.h b/arch/um/nommu/um/shared/sysdep/ptrace_user.h
new file mode 100644
index 000000000000..86d5cb20fb9f
--- /dev/null
+++ b/arch/um/nommu/um/shared/sysdep/ptrace_user.h
@@ -0,0 +1,7 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ARCH_UM_PTRACE_USER_H
+#define __ARCH_UM_PTRACE_USER_H
+
+#define FP_SIZE 1
+
+#endif
diff --git a/arch/um/nommu/um/unimplemented.c b/arch/um/nommu/um/unimplemented.c
new file mode 100644
index 000000000000..fe33e02e39e5
--- /dev/null
+++ b/arch/um/nommu/um/unimplemented.c
@@ -0,0 +1,70 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/signal.h>
+#include <sysdep/ptrace.h>
+#include <asm/ptrace.h>
+
+/* physmem.c  */
+unsigned long high_physmem;
+
+/* x86/um/setjmp*.S  */
+void kernel_longjmp(void)
+{}
+void kernel_setjmp(void)
+{}
+
+/* trap.c */
+void relay_signal(int sig, struct siginfo *si, struct uml_pt_regs *regs)
+{}
+void bus_handler(int sig, struct siginfo *si, struct uml_pt_regs *regs)
+{}
+void segv_handler(int sig, struct siginfo *unused_si, struct uml_pt_regs *regs)
+{}
+void winch(int sig, struct siginfo *unused_si, struct uml_pt_regs *regs)
+{}
+
+/* tlb.c */
+void flush_tlb_kernel_vm(void)
+{}
+void force_flush_all(void)
+{}
+
+/* skas/process.c */
+void halt_skas(void)
+{}
+int is_skas_winch(int pid, int fd, void *data)
+{
+	return 0;
+}
+void reboot_skas(void)
+{}
+
+int __init start_uml(void)
+{
+	return 0;
+}
+
+/* exec.c */
+void flush_thread(void)
+{}
+
+/* x86/ptrace_64.c */
+int is_syscall(unsigned long addr)
+{
+	return 0;
+}
+
+
+/* x86/sysrq.c */
+void show_regs(struct pt_regs *regs)
+{}
+
+/* x86/signal.c */
+int setup_signal_stack_si(unsigned long stack_top, struct ksignal *ksig,
+			  struct pt_regs *regs, sigset_t *mask)
+{
+	return 0;
+}
+
+/* x86/bugs.c */
+void arch_check_bugs(void)
+{}
diff --git a/arch/um/nommu/um/user_constants.h b/arch/um/nommu/um/user_constants.h
new file mode 100644
index 000000000000..2245d3d24120
--- /dev/null
+++ b/arch/um/nommu/um/user_constants.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_NOMMU_USER_CONSTANTS_H
+#define __UM_NOMMU_USER_CONSTANTS_H
+
+/* XXX: put dummy values */
+#define UM_FRAME_SIZE 4
+#define HOST_FP_SIZE 1
+#define HOST_IP 1
+#define HOST_SP 2
+#define HOST_BP 3
+#define UM_NR_CPUS 1
+
+#endif
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v7 09/21] um: nommu: host interface
  2020-10-06  9:44         ` Hajime Tazaki
@ 2020-10-06  9:44           ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-06  9:44 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch, Hajime Tazaki

This patch introduces the host operations that define the interface
between the LKL and the host. These operations must be provided either
by a host library or by the application itself.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/include/asm/host_ops.h            |  9 ++++++++
 arch/um/nommu/include/uapi/asm/host_ops.h | 25 +++++++++++++++++++++++
 2 files changed, 34 insertions(+)
 create mode 100644 arch/um/include/asm/host_ops.h
 create mode 100644 arch/um/nommu/include/uapi/asm/host_ops.h

diff --git a/arch/um/include/asm/host_ops.h b/arch/um/include/asm/host_ops.h
new file mode 100644
index 000000000000..f52423cc4ced
--- /dev/null
+++ b/arch/um/include/asm/host_ops.h
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_HOST_OPS_H
+#define _ASM_LKL_HOST_OPS_H
+
+#include <uapi/asm/host_ops.h>
+
+extern struct lkl_host_operations *lkl_ops;
+
+#endif
diff --git a/arch/um/nommu/include/uapi/asm/host_ops.h b/arch/um/nommu/include/uapi/asm/host_ops.h
new file mode 100644
index 000000000000..d3dad11b459e
--- /dev/null
+++ b/arch/um/nommu/include/uapi/asm/host_ops.h
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef __UM_NOMMU_UAPI_HOST_OPS_H
+#define __UM_NOMMU_UAPI_HOST_OPS_H
+
+/* Defined in {posix,nt}-host.c */
+struct lkl_mutex;
+struct lkl_sem;
+typedef unsigned long lkl_thread_t;
+struct lkl_jmp_buf {
+	unsigned long buf[128];
+};
+
+/**
+ * lkl_host_operations - host operations used by the Linux kernel
+ *
+ * These operations must be provided by a host library or by the application
+ * itself.
+ *
+ */
+struct lkl_host_operations {
+};
+
+void lkl_bug(const char *fmt, ...);
+
+#endif
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v7 09/21] um: nommu: host interface
@ 2020-10-06  9:44           ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-06  9:44 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Hajime Tazaki, retrage01

This patch introduces the host operations that define the interface
between the LKL and the host. These operations must be provided either
by a host library or by the application itself.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/include/asm/host_ops.h            |  9 ++++++++
 arch/um/nommu/include/uapi/asm/host_ops.h | 25 +++++++++++++++++++++++
 2 files changed, 34 insertions(+)
 create mode 100644 arch/um/include/asm/host_ops.h
 create mode 100644 arch/um/nommu/include/uapi/asm/host_ops.h

diff --git a/arch/um/include/asm/host_ops.h b/arch/um/include/asm/host_ops.h
new file mode 100644
index 000000000000..f52423cc4ced
--- /dev/null
+++ b/arch/um/include/asm/host_ops.h
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_HOST_OPS_H
+#define _ASM_LKL_HOST_OPS_H
+
+#include <uapi/asm/host_ops.h>
+
+extern struct lkl_host_operations *lkl_ops;
+
+#endif
diff --git a/arch/um/nommu/include/uapi/asm/host_ops.h b/arch/um/nommu/include/uapi/asm/host_ops.h
new file mode 100644
index 000000000000..d3dad11b459e
--- /dev/null
+++ b/arch/um/nommu/include/uapi/asm/host_ops.h
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef __UM_NOMMU_UAPI_HOST_OPS_H
+#define __UM_NOMMU_UAPI_HOST_OPS_H
+
+/* Defined in {posix,nt}-host.c */
+struct lkl_mutex;
+struct lkl_sem;
+typedef unsigned long lkl_thread_t;
+struct lkl_jmp_buf {
+	unsigned long buf[128];
+};
+
+/**
+ * lkl_host_operations - host operations used by the Linux kernel
+ *
+ * These operations must be provided by a host library or by the application
+ * itself.
+ *
+ */
+struct lkl_host_operations {
+};
+
+void lkl_bug(const char *fmt, ...);
+
+#endif
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v7 10/21] um: nommu: memory handling
  2020-10-06  9:44         ` Hajime Tazaki
@ 2020-10-06  9:44           ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-06  9:44 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch, Hajime Tazaki

nommu mode follows the way !CONFIG_MMU architecture of other linux
archs.  Ther is not much work left to do other than initializing the
boot allocator and providing the page and page table definitions.

The backstore memory is allocated via a host operation and the memory
size to be used is specified when the kernel is started, in the
lkl_start_kernel call.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/include/asm/mmu.h                 |  3 +
 arch/um/include/asm/mmu_context.h         |  8 +++
 arch/um/include/asm/page.h                | 15 ++++
 arch/um/include/asm/pgtable.h             | 27 +++++++
 arch/um/include/asm/uaccess.h             |  6 ++
 arch/um/nommu/include/uapi/asm/host_ops.h |  5 ++
 arch/um/nommu/um/bootmem.c                | 87 +++++++++++++++++++++++
 7 files changed, 151 insertions(+)
 create mode 100644 arch/um/nommu/um/bootmem.c

diff --git a/arch/um/include/asm/mmu.h b/arch/um/include/asm/mmu.h
index 5b072aba5b65..c06d6cb67dd7 100644
--- a/arch/um/include/asm/mmu.h
+++ b/arch/um/include/asm/mmu.h
@@ -13,6 +13,9 @@ typedef struct mm_context {
 	struct mm_id id;
 	struct uml_arch_mm_context arch;
 	struct page *stub_pages[2];
+#ifndef CONFIG_MMU
+	unsigned long		end_brk;
+#endif
 } mm_context_t;
 
 extern void __switch_mm(struct mm_id * mm_idp);
diff --git a/arch/um/include/asm/mmu_context.h b/arch/um/include/asm/mmu_context.h
index 17ddd4edf875..4b06c29ae830 100644
--- a/arch/um/include/asm/mmu_context.h
+++ b/arch/um/include/asm/mmu_context.h
@@ -6,6 +6,7 @@
 #ifndef __UM_MMU_CONTEXT_H
 #define __UM_MMU_CONTEXT_H
 
+#ifdef CONFIG_MMU
 #include <linux/sched.h>
 #include <linux/mm_types.h>
 #include <linux/mmap_lock.h>
@@ -75,4 +76,11 @@ extern int init_new_context(struct task_struct *task, struct mm_struct *mm);
 
 extern void destroy_context(struct mm_struct *mm);
 
+#else
+#include <asm-generic/mmu_context.h>
+
+extern void force_flush_all(void);
+
+#endif /* CONFIG_MMU */
+
 #endif
diff --git a/arch/um/include/asm/page.h b/arch/um/include/asm/page.h
index 95af12e82a32..c9d5e487ac6b 100644
--- a/arch/um/include/asm/page.h
+++ b/arch/um/include/asm/page.h
@@ -7,6 +7,8 @@
 #ifndef __UM_PAGE_H
 #define __UM_PAGE_H
 
+#ifdef CONFIG_MMU
+
 #include <linux/const.h>
 
 /* PAGE_SHIFT determines the page size */
@@ -120,4 +122,17 @@ extern unsigned long uml_physmem;
 #define __HAVE_ARCH_GATE_AREA 1
 #endif
 
+#else  /* CONFIG_MMU */
+#define CONFIG_KERNEL_RAM_BASE_ADDRESS memory_start
+#include <asm-generic/page.h>
+
+#define __va_space (8*1024*1024)
+
+#ifndef __ASSEMBLY__
+#include <mem.h>
+void free_mem(void);
+#endif
+
+#endif /* !CONFIG_MMU  */
+
 #endif	/* __UM_PAGE_H */
diff --git a/arch/um/include/asm/pgtable.h b/arch/um/include/asm/pgtable.h
index def376194dce..7e506d5406e3 100644
--- a/arch/um/include/asm/pgtable.h
+++ b/arch/um/include/asm/pgtable.h
@@ -21,6 +21,8 @@
 #define _PAGE_PROTNONE	0x010	/* if the user mapped it with PROT_NONE;
 				   pte_present gives true */
 
+#ifdef CONFIG_MMU
+
 #ifdef CONFIG_3_LEVEL_PGTABLES
 #include <asm/pgtable-3level.h>
 #else
@@ -323,4 +325,29 @@ do {						\
 	__flush_tlb_one((vaddr));		\
 } while (0)
 
+#else  /* CONFIG_MMU */
+
+#include <asm-generic/pgtable-nopud.h>
+
+#define swapper_pg_dir		((pgd_t *)0)
+#define PAGE_KERNEL             __pgprot(0)
+#define PGDIR_SHIFT		21
+#define PGDIR_SIZE		(1UL << PGDIR_SHIFT)
+#define PGDIR_MASK		(~(PGDIR_SIZE-1))
+
+#define VMALLOC_START	0
+#define VMALLOC_END	0xffffffff
+
+/*
+ * ZERO_PAGE is a global shared page that is always zero: used
+ * for zero-mapped memory areas etc..
+ */
+extern unsigned long *empty_zero_page;
+#define ZERO_PAGE(vaddr)	(virt_to_page(empty_zero_page))
+
+extern unsigned long end_iomem;
+
+#endif /* !CONFIG_MMU */
+
+
 #endif
diff --git a/arch/um/include/asm/uaccess.h b/arch/um/include/asm/uaccess.h
index fe66d659acad..95db8f06d295 100644
--- a/arch/um/include/asm/uaccess.h
+++ b/arch/um/include/asm/uaccess.h
@@ -21,6 +21,7 @@
 #define __addr_range_nowrap(addr, size) \
 	((unsigned long) (addr) <= ((unsigned long) (addr) + (size)))
 
+#ifdef CONFIG_MMU
 extern unsigned long raw_copy_from_user(void *to, const void __user *from, unsigned long n);
 extern unsigned long raw_copy_to_user(void __user *to, const void *from, unsigned long n);
 extern long __strncpy_from_user(char *dst, const char __user *src, long count);
@@ -46,4 +47,9 @@ static inline int __access_ok(unsigned long addr, unsigned long size)
 		uaccess_kernel());
 }
 
+#else
+#include <asm-generic/uaccess.h>
+#endif /* CONFIG_MMU */
+
+
 #endif
diff --git a/arch/um/nommu/include/uapi/asm/host_ops.h b/arch/um/nommu/include/uapi/asm/host_ops.h
index d3dad11b459e..5253c3f8de0e 100644
--- a/arch/um/nommu/include/uapi/asm/host_ops.h
+++ b/arch/um/nommu/include/uapi/asm/host_ops.h
@@ -16,8 +16,13 @@ struct lkl_jmp_buf {
  * These operations must be provided by a host library or by the application
  * itself.
  *
+ * @mem_alloc - allocate memory
+ * @mem_free - free memory
+ *
  */
 struct lkl_host_operations {
+	void *(*mem_alloc)(unsigned long mem);
+	void (*mem_free)(void *mem);
 };
 
 void lkl_bug(const char *fmt, ...);
diff --git a/arch/um/nommu/um/bootmem.c b/arch/um/nommu/um/bootmem.c
new file mode 100644
index 000000000000..7398a48c05d4
--- /dev/null
+++ b/arch/um/nommu/um/bootmem.c
@@ -0,0 +1,87 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/memblock.h>
+#include <linux/mm.h>
+#include <linux/swap.h>
+
+unsigned long memory_start, memory_end;
+EXPORT_SYMBOL(memory_start);
+static unsigned long _memory_start, mem_size;
+
+unsigned long *empty_zero_page;
+
+/* XXX: unused */
+unsigned long long highmem;
+int iomem_size;
+int kmalloc_ok = 1;
+
+void __init setup_physmem(unsigned long start, unsigned long reserve_end,
+			  unsigned long mem_sz, unsigned long long _highmem)
+{
+	mem_size = mem_sz;
+
+	_memory_start = (unsigned long)lkl_ops->mem_alloc(mem_size);
+	memory_start = _memory_start;
+	WARN_ON(!memory_start);
+	memory_end = memory_start + mem_size;
+
+	if (PAGE_ALIGN(memory_start) != memory_start) {
+		mem_size -= PAGE_ALIGN(memory_start) - memory_start;
+		memory_start = PAGE_ALIGN(memory_start);
+		mem_size = (mem_size / PAGE_SIZE) * PAGE_SIZE;
+	}
+	pr_info("memblock address range: 0x%lx - 0x%lx\n", memory_start,
+		memory_start+mem_size);
+	/*
+	 * Give all the memory to the bootmap allocator, tell it to put the
+	 * boot mem_map at the start of memory.
+	 */
+	max_low_pfn = virt_to_pfn(memory_end);
+	min_low_pfn = virt_to_pfn(memory_start);
+	memblock_add(memory_start, mem_size);
+
+	empty_zero_page = memblock_alloc(PAGE_SIZE, PAGE_SIZE);
+	memset((void *)empty_zero_page, 0, PAGE_SIZE);
+
+	{
+		unsigned long zones_size[MAX_NR_ZONES] = {0, };
+
+		zones_size[ZONE_NORMAL] = max_low_pfn;
+		free_area_init(zones_size);
+	}
+}
+
+void __init mem_init(void)
+{
+	max_mapnr = (((unsigned long)high_memory) - PAGE_OFFSET) >> PAGE_SHIFT;
+	/* this will put all memory onto the freelists */
+	totalram_pages_add(memblock_free_all());
+	pr_info("Memory available: %luk/%luk RAM\n",
+		(nr_free_pages() << PAGE_SHIFT) >> 10, mem_size >> 10);
+}
+
+/*
+ * In our case __init memory is not part of the page allocator so there is
+ * nothing to free.
+ */
+void free_initmem(void)
+{
+}
+
+void free_mem(void)
+{
+	lkl_ops->mem_free((void *)_memory_start);
+}
+
+void *uml_kmalloc(int size, int flags)
+{
+	return kmalloc(size, flags);
+}
+
+void __init mem_total_pages(unsigned long physmem, unsigned long iomem,
+		     unsigned long _highmem)
+{
+}
+
+void __init paging_init(void)
+{
+}
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v7 10/21] um: nommu: memory handling
@ 2020-10-06  9:44           ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-06  9:44 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Hajime Tazaki, retrage01

nommu mode follows the way !CONFIG_MMU architecture of other linux
archs.  Ther is not much work left to do other than initializing the
boot allocator and providing the page and page table definitions.

The backstore memory is allocated via a host operation and the memory
size to be used is specified when the kernel is started, in the
lkl_start_kernel call.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/include/asm/mmu.h                 |  3 +
 arch/um/include/asm/mmu_context.h         |  8 +++
 arch/um/include/asm/page.h                | 15 ++++
 arch/um/include/asm/pgtable.h             | 27 +++++++
 arch/um/include/asm/uaccess.h             |  6 ++
 arch/um/nommu/include/uapi/asm/host_ops.h |  5 ++
 arch/um/nommu/um/bootmem.c                | 87 +++++++++++++++++++++++
 7 files changed, 151 insertions(+)
 create mode 100644 arch/um/nommu/um/bootmem.c

diff --git a/arch/um/include/asm/mmu.h b/arch/um/include/asm/mmu.h
index 5b072aba5b65..c06d6cb67dd7 100644
--- a/arch/um/include/asm/mmu.h
+++ b/arch/um/include/asm/mmu.h
@@ -13,6 +13,9 @@ typedef struct mm_context {
 	struct mm_id id;
 	struct uml_arch_mm_context arch;
 	struct page *stub_pages[2];
+#ifndef CONFIG_MMU
+	unsigned long		end_brk;
+#endif
 } mm_context_t;
 
 extern void __switch_mm(struct mm_id * mm_idp);
diff --git a/arch/um/include/asm/mmu_context.h b/arch/um/include/asm/mmu_context.h
index 17ddd4edf875..4b06c29ae830 100644
--- a/arch/um/include/asm/mmu_context.h
+++ b/arch/um/include/asm/mmu_context.h
@@ -6,6 +6,7 @@
 #ifndef __UM_MMU_CONTEXT_H
 #define __UM_MMU_CONTEXT_H
 
+#ifdef CONFIG_MMU
 #include <linux/sched.h>
 #include <linux/mm_types.h>
 #include <linux/mmap_lock.h>
@@ -75,4 +76,11 @@ extern int init_new_context(struct task_struct *task, struct mm_struct *mm);
 
 extern void destroy_context(struct mm_struct *mm);
 
+#else
+#include <asm-generic/mmu_context.h>
+
+extern void force_flush_all(void);
+
+#endif /* CONFIG_MMU */
+
 #endif
diff --git a/arch/um/include/asm/page.h b/arch/um/include/asm/page.h
index 95af12e82a32..c9d5e487ac6b 100644
--- a/arch/um/include/asm/page.h
+++ b/arch/um/include/asm/page.h
@@ -7,6 +7,8 @@
 #ifndef __UM_PAGE_H
 #define __UM_PAGE_H
 
+#ifdef CONFIG_MMU
+
 #include <linux/const.h>
 
 /* PAGE_SHIFT determines the page size */
@@ -120,4 +122,17 @@ extern unsigned long uml_physmem;
 #define __HAVE_ARCH_GATE_AREA 1
 #endif
 
+#else  /* CONFIG_MMU */
+#define CONFIG_KERNEL_RAM_BASE_ADDRESS memory_start
+#include <asm-generic/page.h>
+
+#define __va_space (8*1024*1024)
+
+#ifndef __ASSEMBLY__
+#include <mem.h>
+void free_mem(void);
+#endif
+
+#endif /* !CONFIG_MMU  */
+
 #endif	/* __UM_PAGE_H */
diff --git a/arch/um/include/asm/pgtable.h b/arch/um/include/asm/pgtable.h
index def376194dce..7e506d5406e3 100644
--- a/arch/um/include/asm/pgtable.h
+++ b/arch/um/include/asm/pgtable.h
@@ -21,6 +21,8 @@
 #define _PAGE_PROTNONE	0x010	/* if the user mapped it with PROT_NONE;
 				   pte_present gives true */
 
+#ifdef CONFIG_MMU
+
 #ifdef CONFIG_3_LEVEL_PGTABLES
 #include <asm/pgtable-3level.h>
 #else
@@ -323,4 +325,29 @@ do {						\
 	__flush_tlb_one((vaddr));		\
 } while (0)
 
+#else  /* CONFIG_MMU */
+
+#include <asm-generic/pgtable-nopud.h>
+
+#define swapper_pg_dir		((pgd_t *)0)
+#define PAGE_KERNEL             __pgprot(0)
+#define PGDIR_SHIFT		21
+#define PGDIR_SIZE		(1UL << PGDIR_SHIFT)
+#define PGDIR_MASK		(~(PGDIR_SIZE-1))
+
+#define VMALLOC_START	0
+#define VMALLOC_END	0xffffffff
+
+/*
+ * ZERO_PAGE is a global shared page that is always zero: used
+ * for zero-mapped memory areas etc..
+ */
+extern unsigned long *empty_zero_page;
+#define ZERO_PAGE(vaddr)	(virt_to_page(empty_zero_page))
+
+extern unsigned long end_iomem;
+
+#endif /* !CONFIG_MMU */
+
+
 #endif
diff --git a/arch/um/include/asm/uaccess.h b/arch/um/include/asm/uaccess.h
index fe66d659acad..95db8f06d295 100644
--- a/arch/um/include/asm/uaccess.h
+++ b/arch/um/include/asm/uaccess.h
@@ -21,6 +21,7 @@
 #define __addr_range_nowrap(addr, size) \
 	((unsigned long) (addr) <= ((unsigned long) (addr) + (size)))
 
+#ifdef CONFIG_MMU
 extern unsigned long raw_copy_from_user(void *to, const void __user *from, unsigned long n);
 extern unsigned long raw_copy_to_user(void __user *to, const void *from, unsigned long n);
 extern long __strncpy_from_user(char *dst, const char __user *src, long count);
@@ -46,4 +47,9 @@ static inline int __access_ok(unsigned long addr, unsigned long size)
 		uaccess_kernel());
 }
 
+#else
+#include <asm-generic/uaccess.h>
+#endif /* CONFIG_MMU */
+
+
 #endif
diff --git a/arch/um/nommu/include/uapi/asm/host_ops.h b/arch/um/nommu/include/uapi/asm/host_ops.h
index d3dad11b459e..5253c3f8de0e 100644
--- a/arch/um/nommu/include/uapi/asm/host_ops.h
+++ b/arch/um/nommu/include/uapi/asm/host_ops.h
@@ -16,8 +16,13 @@ struct lkl_jmp_buf {
  * These operations must be provided by a host library or by the application
  * itself.
  *
+ * @mem_alloc - allocate memory
+ * @mem_free - free memory
+ *
  */
 struct lkl_host_operations {
+	void *(*mem_alloc)(unsigned long mem);
+	void (*mem_free)(void *mem);
 };
 
 void lkl_bug(const char *fmt, ...);
diff --git a/arch/um/nommu/um/bootmem.c b/arch/um/nommu/um/bootmem.c
new file mode 100644
index 000000000000..7398a48c05d4
--- /dev/null
+++ b/arch/um/nommu/um/bootmem.c
@@ -0,0 +1,87 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/memblock.h>
+#include <linux/mm.h>
+#include <linux/swap.h>
+
+unsigned long memory_start, memory_end;
+EXPORT_SYMBOL(memory_start);
+static unsigned long _memory_start, mem_size;
+
+unsigned long *empty_zero_page;
+
+/* XXX: unused */
+unsigned long long highmem;
+int iomem_size;
+int kmalloc_ok = 1;
+
+void __init setup_physmem(unsigned long start, unsigned long reserve_end,
+			  unsigned long mem_sz, unsigned long long _highmem)
+{
+	mem_size = mem_sz;
+
+	_memory_start = (unsigned long)lkl_ops->mem_alloc(mem_size);
+	memory_start = _memory_start;
+	WARN_ON(!memory_start);
+	memory_end = memory_start + mem_size;
+
+	if (PAGE_ALIGN(memory_start) != memory_start) {
+		mem_size -= PAGE_ALIGN(memory_start) - memory_start;
+		memory_start = PAGE_ALIGN(memory_start);
+		mem_size = (mem_size / PAGE_SIZE) * PAGE_SIZE;
+	}
+	pr_info("memblock address range: 0x%lx - 0x%lx\n", memory_start,
+		memory_start+mem_size);
+	/*
+	 * Give all the memory to the bootmap allocator, tell it to put the
+	 * boot mem_map at the start of memory.
+	 */
+	max_low_pfn = virt_to_pfn(memory_end);
+	min_low_pfn = virt_to_pfn(memory_start);
+	memblock_add(memory_start, mem_size);
+
+	empty_zero_page = memblock_alloc(PAGE_SIZE, PAGE_SIZE);
+	memset((void *)empty_zero_page, 0, PAGE_SIZE);
+
+	{
+		unsigned long zones_size[MAX_NR_ZONES] = {0, };
+
+		zones_size[ZONE_NORMAL] = max_low_pfn;
+		free_area_init(zones_size);
+	}
+}
+
+void __init mem_init(void)
+{
+	max_mapnr = (((unsigned long)high_memory) - PAGE_OFFSET) >> PAGE_SHIFT;
+	/* this will put all memory onto the freelists */
+	totalram_pages_add(memblock_free_all());
+	pr_info("Memory available: %luk/%luk RAM\n",
+		(nr_free_pages() << PAGE_SHIFT) >> 10, mem_size >> 10);
+}
+
+/*
+ * In our case __init memory is not part of the page allocator so there is
+ * nothing to free.
+ */
+void free_initmem(void)
+{
+}
+
+void free_mem(void)
+{
+	lkl_ops->mem_free((void *)_memory_start);
+}
+
+void *uml_kmalloc(int size, int flags)
+{
+	return kmalloc(size, flags);
+}
+
+void __init mem_total_pages(unsigned long physmem, unsigned long iomem,
+		     unsigned long _highmem)
+{
+}
+
+void __init paging_init(void)
+{
+}
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v7 11/21] um: nommu: kernel thread support
  2020-10-06  9:44         ` Hajime Tazaki
@ 2020-10-06  9:44           ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-06  9:44 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch, Hajime Tazaki

nommu mode does not support user processes but it must support kernel
threads as part as the normal kernel work-flow. It uses host operations
to create and terminate host threads that are going to run the kernel
threads. It also uses semaphores to synchronize those threads and to
allow the Linux kernel scheduler to control how the kernel threads
run.

Each kernel thread runs in a host threads and has a host semaphore
associated with it - the thread's scheduling semaphore. The semaphore
counter is initialized to 0. The first thing a kernel thread does
after getting spawned, before running any kernel code, is to perform a
down operation to block the thread.

The kernel controls host threads scheduling by performing up and down
operations on the scheduling semaphore. In __switch_context an up
operation on the next thread is performed to wake up a blocked thread,
and a down operation is performed on the prev thread to block it.

A thread is terminated by marking it in free_thread_info and
performing an up operation on the scheduling semaphore at which point
the marked thread will terminate itself.

UML common part (process.c) is extended to call idle functions of
SUBARCH as well as kernel thread detection with a flag information of
TIF_HOST_THREAD.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/include/asm/thread_info.h         |  24 ++
 arch/um/kernel/process.c                  |   8 +-
 arch/um/nommu/include/uapi/asm/host_ops.h |  54 +++++
 arch/um/nommu/um/cpu.c                    | 236 ++++++++++++++++++++
 arch/um/nommu/um/threads.c                | 259 ++++++++++++++++++++++
 5 files changed, 580 insertions(+), 1 deletion(-)
 create mode 100644 arch/um/nommu/um/cpu.c
 create mode 100644 arch/um/nommu/um/threads.c

diff --git a/arch/um/include/asm/thread_info.h b/arch/um/include/asm/thread_info.h
index 4c19ce4c49f1..433584f6b9d0 100644
--- a/arch/um/include/asm/thread_info.h
+++ b/arch/um/include/asm/thread_info.h
@@ -40,6 +40,7 @@ struct thread_info {
 	.real_thread = NULL,			\
 }
 
+#ifdef CONFIG_MMU
 /* how to get the thread information struct from C */
 static inline struct thread_info *current_thread_info(void)
 {
@@ -52,6 +53,27 @@ static inline struct thread_info *current_thread_info(void)
 	return ti;
 }
 
+#else
+
+#define __HAVE_THREAD_FUNCTIONS
+#define task_thread_info(task)	((struct thread_info *)(task)->stack)
+#define task_stack_page(task)	((task)->stack)
+#define end_of_stack(p) (&task_thread_info(p)->aux_fp_regs[FP_SIZE-1])
+
+void threads_init(void);
+void threads_cleanup(void);
+
+unsigned long *alloc_thread_stack_node(struct task_struct *p, int node);
+void setup_thread_stack(struct task_struct *p, struct task_struct *org);
+void free_thread_stack(struct task_struct *tsk);
+
+extern struct thread_info *_current_thread_info;
+static inline struct thread_info *current_thread_info(void)
+{
+	return _current_thread_info;
+}
+#endif /* CONFIG_MMU */
+
 #endif
 
 #define TIF_SYSCALL_TRACE	0	/* syscall trace active */
@@ -63,6 +85,8 @@ static inline struct thread_info *current_thread_info(void)
 #define TIF_RESTORE_SIGMASK	7
 #define TIF_NOTIFY_RESUME	8
 #define TIF_SECCOMP		9	/* secure computing */
+#define TIF_SCHED_JB		10
+#define TIF_HOST_THREAD		11
 
 #define _TIF_SYSCALL_TRACE	(1 << TIF_SYSCALL_TRACE)
 #define _TIF_SIGPENDING		(1 << TIF_SIGPENDING)
diff --git a/arch/um/kernel/process.c b/arch/um/kernel/process.c
index 87a8cfa228ca..432edcbb5eea 100644
--- a/arch/um/kernel/process.c
+++ b/arch/um/kernel/process.c
@@ -156,7 +156,8 @@ int copy_thread(unsigned long clone_flags, unsigned long sp,
 		unsigned long arg, struct task_struct * p, unsigned long tls)
 {
 	void (*handler)(void);
-	int kthread = current->flags & PF_KTHREAD;
+	int kthread = current->flags & PF_KTHREAD ||
+		test_ti_thread_flag(current_thread_info(), TIF_HOST_THREAD);
 	int ret = 0;
 
 	p->thread = (struct thread_struct) INIT_THREAD;
@@ -213,10 +214,15 @@ static void um_idle_sleep(void)
 	}
 }
 
+void __weak subarch_cpu_idle(void)
+{
+}
+
 void arch_cpu_idle(void)
 {
 	cpu_tasks[current_thread_info()->cpu].pid = os_getpid();
 	um_idle_sleep();
+	subarch_cpu_idle();
 	local_irq_enable();
 }
 
diff --git a/arch/um/nommu/include/uapi/asm/host_ops.h b/arch/um/nommu/include/uapi/asm/host_ops.h
index 5253c3f8de0e..720385fccbdf 100644
--- a/arch/um/nommu/include/uapi/asm/host_ops.h
+++ b/arch/um/nommu/include/uapi/asm/host_ops.h
@@ -16,11 +16,65 @@ struct lkl_jmp_buf {
  * These operations must be provided by a host library or by the application
  * itself.
  *
+ * @sem_alloc - allocate a host semaphore an initialize it to count
+ * @sem_free - free a host semaphore
+ * @sem_up - perform an up operation on the semaphore
+ * @sem_down - perform a down operation on the semaphore
+ *
+ * @mutex_alloc - allocate and initialize a host mutex; the recursive parameter
+ * determines if the mutex is recursive or not
+ * @mutex_free - free a host mutex
+ * @mutex_lock - acquire the mutex
+ * @mutex_unlock - release the mutex
+ *
+ * @thread_create - create a new thread and run f(arg) in its context; returns a
+ * thread handle or 0 if the thread could not be created
+ * @thread_detach - on POSIX systems, free up resources held by
+ * pthreads. Noop on Win32.
+ * @thread_exit - terminates the current thread
+ * @thread_join - wait for the given thread to terminate. Returns 0
+ * for success, -1 otherwise
+ *
+ * @gettid - returns the host thread id of the caller, which need not
+ * be the same as the handle returned by thread_create
+ *
+ * @jmp_buf_set - runs the give function and setups a jump back point by saving
+ * the context in the jump buffer; jmp_buf_longjmp can be called from the give
+ * function or any callee in that function to return back to the jump back
+ * point
+ *
+ * NOTE: we can't return from jmp_buf_set before calling jmp_buf_longjmp or
+ * otherwise the saved context (stack) is not going to be valid, so we must pass
+ * the function that will eventually call longjmp here
+ *
+ * @jmp_buf_longjmp - perform a jump back to the saved jump buffer
+ *
  * @mem_alloc - allocate memory
  * @mem_free - free memory
  *
  */
 struct lkl_host_operations {
+	struct lkl_sem *(*sem_alloc)(int count);
+	void (*sem_free)(struct lkl_sem *sem);
+	void (*sem_up)(struct lkl_sem *sem);
+	void (*sem_down)(struct lkl_sem *sem);
+
+	struct lkl_mutex *(*mutex_alloc)(int recursive);
+	void (*mutex_free)(struct lkl_mutex *mutex);
+	void (*mutex_lock)(struct lkl_mutex *mutex);
+	void (*mutex_unlock)(struct lkl_mutex *mutex);
+
+	lkl_thread_t (*thread_create)(void *(*f)(void *), void *arg);
+	void (*thread_detach)(void);
+	void (*thread_exit)(void);
+	int (*thread_join)(lkl_thread_t tid);
+	lkl_thread_t (*thread_self)(void);
+	int (*thread_equal)(lkl_thread_t a, lkl_thread_t b);
+	long (*gettid)(void);
+
+	void (*jmp_buf_set)(struct lkl_jmp_buf *jmpb, void (*f)(void));
+	void (*jmp_buf_longjmp)(struct lkl_jmp_buf *jmpb, int val);
+
 	void *(*mem_alloc)(unsigned long mem);
 	void (*mem_free)(void *mem);
 };
diff --git a/arch/um/nommu/um/cpu.c b/arch/um/nommu/um/cpu.c
new file mode 100644
index 000000000000..9986a3f8c5dd
--- /dev/null
+++ b/arch/um/nommu/um/cpu.c
@@ -0,0 +1,236 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/kernel.h>
+#include <linux/sched/stat.h>
+#include <asm/host_ops.h>
+#include <asm/cpu.h>
+#include <asm/thread_info.h>
+#include <asm/unistd.h>
+#include <asm/sched.h>
+#include <asm/syscalls.h>
+#include <init.h>
+
+void run_irqs(void)
+{
+	panic("unimplemented %s", __func__);
+}
+
+int set_irq_pending(int irq)
+{
+	panic("unimplemented %s", __func__);
+}
+
+/*
+ * This structure is used to get access to the "LKL CPU" that allows us to run
+ * Linux code. Because we have to deal with various synchronization requirements
+ * between idle thread, system calls, interrupts, "reentrancy", CPU shutdown,
+ * imbalance wake up (i.e. acquire the CPU from one thread and release it from
+ * another), we can't use a simple synchronization mechanism such as (recursive)
+ * mutex or semaphore. Instead, we use a mutex and a bunch of status data plus a
+ * semaphore.
+ */
+static struct lkl_cpu {
+	/* lock that protects the CPU status data */
+	struct lkl_mutex *lock;
+	/*
+	 * Since we must free the cpu lock during shutdown we need a
+	 * synchronization algorithm between lkl_cpu_shutdown() and the CPU
+	 * access functions since lkl_cpu_get() gets called from thread
+	 * destructor callback functions which may be scheduled after
+	 * lkl_cpu_shutdown() has freed the cpu lock.
+	 *
+	 * An atomic counter is used to keep track of the number of running
+	 * CPU access functions and allow the shutdown function to wait for
+	 * them.
+	 *
+	 * The shutdown functions adds MAX_THREADS to this counter which allows
+	 * the CPU access functions to check if the shutdown process has
+	 * started.
+	 *
+	 * This algorithm assumes that we never have more the MAX_THREADS
+	 * requesting CPU access.
+	 */
+	#define MAX_THREADS 1000000
+	unsigned int shutdown_gate;
+	bool irqs_pending;
+	/* no of threads waiting the CPU */
+	unsigned int sleepers;
+	/* no of times the current thread got the CPU */
+	unsigned int count;
+	/* current thread that owns the CPU */
+	lkl_thread_t owner;
+	/* semaphore for threads waiting the CPU */
+	struct lkl_sem *sem;
+	/* semaphore used for shutdown */
+	struct lkl_sem *shutdown_sem;
+} cpu;
+
+static int __cpu_try_get_lock(int n)
+{
+	lkl_thread_t self;
+
+	if (__sync_fetch_and_add(&cpu.shutdown_gate, n) >= MAX_THREADS)
+		return -2;
+
+	lkl_ops->mutex_lock(cpu.lock);
+
+	if (cpu.shutdown_gate >= MAX_THREADS)
+		return -1;
+
+	self = lkl_ops->thread_self();
+
+	if (cpu.owner && !lkl_ops->thread_equal(cpu.owner, self))
+		return 0;
+
+	cpu.owner = self;
+	cpu.count++;
+
+	return 1;
+}
+
+static void __cpu_try_get_unlock(int lock_ret, int n)
+{
+	if (lock_ret >= -1)
+		lkl_ops->mutex_unlock(cpu.lock);
+	__sync_fetch_and_sub(&cpu.shutdown_gate, n);
+}
+
+void lkl_cpu_change_owner(lkl_thread_t owner)
+{
+	lkl_ops->mutex_lock(cpu.lock);
+	if (cpu.count > 1)
+		lkl_bug("bad count while changing owner\n");
+	cpu.owner = owner;
+	lkl_ops->mutex_unlock(cpu.lock);
+}
+
+int lkl_cpu_get(void)
+{
+	int ret;
+
+	ret = __cpu_try_get_lock(1);
+
+	while (ret == 0) {
+		cpu.sleepers++;
+		__cpu_try_get_unlock(ret, 0);
+		lkl_ops->sem_down(cpu.sem);
+		ret = __cpu_try_get_lock(0);
+	}
+
+	__cpu_try_get_unlock(ret, 1);
+
+	return ret;
+}
+
+void lkl_cpu_put(void)
+{
+	lkl_ops->mutex_lock(cpu.lock);
+
+	if (!cpu.count || !cpu.owner ||
+	    !lkl_ops->thread_equal(cpu.owner, lkl_ops->thread_self()))
+		lkl_bug("%s: unbalanced put\n", __func__);
+
+	while (cpu.irqs_pending && !irqs_disabled()) {
+		cpu.irqs_pending = false;
+		lkl_ops->mutex_unlock(cpu.lock);
+		run_irqs();
+		lkl_ops->mutex_lock(cpu.lock);
+	}
+
+	if (test_ti_thread_flag(current_thread_info(), TIF_HOST_THREAD) &&
+	    !single_task_running() && cpu.count == 1) {
+		if (in_interrupt())
+			lkl_bug("%s: in interrupt\n", __func__);
+		lkl_ops->mutex_unlock(cpu.lock);
+		thread_sched_jb();
+		return;
+	}
+
+	if (--cpu.count > 0) {
+		lkl_ops->mutex_unlock(cpu.lock);
+		return;
+	}
+
+	if (cpu.sleepers) {
+		cpu.sleepers--;
+		lkl_ops->sem_up(cpu.sem);
+	}
+
+	cpu.owner = 0;
+
+	lkl_ops->mutex_unlock(cpu.lock);
+}
+
+int lkl_cpu_try_run_irq(int irq)
+{
+	int ret;
+
+	ret = __cpu_try_get_lock(1);
+	if (!ret) {
+		set_irq_pending(irq);
+		cpu.irqs_pending = true;
+	}
+	__cpu_try_get_unlock(ret, 1);
+
+	return ret;
+}
+
+static void lkl_cpu_shutdown(void)
+{
+	__sync_fetch_and_add(&cpu.shutdown_gate, MAX_THREADS);
+}
+__uml_exitcall(lkl_cpu_shutdown);
+
+void lkl_cpu_wait_shutdown(void)
+{
+	lkl_ops->sem_down(cpu.shutdown_sem);
+	lkl_ops->sem_free(cpu.shutdown_sem);
+}
+
+static void lkl_cpu_cleanup(bool shutdown)
+{
+	while (__sync_fetch_and_add(&cpu.shutdown_gate, 0) > MAX_THREADS)
+		;
+
+	if (shutdown)
+		lkl_ops->sem_up(cpu.shutdown_sem);
+	else if (cpu.shutdown_sem)
+		lkl_ops->sem_free(cpu.shutdown_sem);
+	if (cpu.sem)
+		lkl_ops->sem_free(cpu.sem);
+	if (cpu.lock)
+		lkl_ops->mutex_free(cpu.lock);
+}
+
+void subarch_cpu_idle(void)
+{
+	if (cpu.shutdown_gate >= MAX_THREADS) {
+
+		lkl_ops->mutex_lock(cpu.lock);
+		while (cpu.sleepers--)
+			lkl_ops->sem_up(cpu.sem);
+		lkl_ops->mutex_unlock(cpu.lock);
+
+		lkl_cpu_cleanup(true);
+
+		lkl_ops->thread_exit();
+	}
+
+#ifdef doesntwork
+	/* switch to idle_host_task */
+	wakeup_idle_host_task();
+#endif
+}
+
+int lkl_cpu_init(void)
+{
+	cpu.lock = lkl_ops->mutex_alloc(0);
+	cpu.sem = lkl_ops->sem_alloc(0);
+	cpu.shutdown_sem = lkl_ops->sem_alloc(0);
+
+	if (!cpu.lock || !cpu.sem || !cpu.shutdown_sem) {
+		lkl_cpu_cleanup(false);
+		return -ENOMEM;
+	}
+
+	return 0;
+}
diff --git a/arch/um/nommu/um/threads.c b/arch/um/nommu/um/threads.c
new file mode 100644
index 000000000000..e9a49100d40e
--- /dev/null
+++ b/arch/um/nommu/um/threads.c
@@ -0,0 +1,259 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/slab.h>
+#include <linux/sched/task.h>
+#include <linux/sched/signal.h>
+#include <asm/host_ops.h>
+#include <asm/cpu.h>
+#include <asm/sched.h>
+
+#include <os.h>
+
+static int init_arch_thread(struct arch_thread *thread)
+{
+	thread->sched_sem = lkl_ops->sem_alloc(0);
+	if (!thread->sched_sem)
+		return -ENOMEM;
+
+	thread->dead = false;
+	thread->tid = 0;
+
+	return 0;
+}
+
+unsigned long *alloc_thread_stack_node(struct task_struct *task, int node)
+{
+	struct thread_info *ti;
+
+	ti = kmalloc(sizeof(*ti), GFP_KERNEL | __GFP_ZERO);
+	if (!ti)
+		return NULL;
+
+	ti->task = task;
+	return (unsigned long *)ti;
+}
+
+/*
+ * The only new tasks created are kernel threads that have a predefined starting
+ * point thus no stack copy is required.
+ */
+void setup_thread_stack(struct task_struct *p, struct task_struct *org)
+{
+	struct thread_info *ti = task_thread_info(p);
+	struct thread_info *org_ti = task_thread_info(org);
+
+	ti->flags = org_ti->flags;
+	ti->preempt_count = org_ti->preempt_count;
+	ti->addr_limit = org_ti->addr_limit;
+}
+
+static void kill_thread(struct thread_info *ti)
+{
+	if (!test_ti_thread_flag(ti, TIF_HOST_THREAD)) {
+		ti->task->thread.arch.dead = true;
+		lkl_ops->sem_up(ti->task->thread.arch.sched_sem);
+		lkl_ops->thread_join(ti->task->thread.arch.tid);
+	}
+	lkl_ops->sem_free(ti->task->thread.arch.sched_sem);
+}
+
+void free_thread_stack(struct task_struct *tsk)
+{
+	struct thread_info *ti = task_thread_info(tsk);
+
+	kill_thread(ti);
+	kfree(ti);
+}
+
+struct thread_info *_current_thread_info = &init_thread_union.thread_info;
+EXPORT_SYMBOL(_current_thread_info);
+
+void switch_threads(jmp_buf *me, jmp_buf *you)
+{
+	/* NOP */
+}
+
+/*
+ * schedule() expects the return of this function to be the task that we
+ * switched away from. Returning prev is not going to work because we are
+ * actually going to return the previous taks that was scheduled before the
+ * task we are going to wake up, and not the current task, e.g.:
+ *
+ * swapper -> init: saved prev on swapper stack is swapper
+ * init -> ksoftirqd0: saved prev on init stack is init
+ * ksoftirqd0 -> swapper: returned prev is swapper
+ */
+static struct task_struct *abs_prev = &init_task;
+
+void arch_switch_to(struct task_struct *prev,
+		    struct task_struct *next)
+{
+	struct arch_thread *_prev = &prev->thread.arch;
+	struct arch_thread *_next = &next->thread.arch;
+	unsigned long _prev_flags = task_thread_info(prev)->flags;
+	struct lkl_jmp_buf *_prev_jb;
+
+	_current_thread_info = task_thread_info(next);
+	next->thread.prev_sched = prev;
+	abs_prev = prev;
+
+	WARN_ON(!_next->tid);
+	lkl_cpu_change_owner(_next->tid);
+
+	if (test_bit(TIF_SCHED_JB, &_prev_flags)) {
+		/* Atomic. Must be done before wakeup next */
+		clear_ti_thread_flag(task_thread_info(prev), TIF_SCHED_JB);
+		_prev_jb = &_prev->sched_jb;
+	}
+
+	lkl_ops->sem_up(_next->sched_sem);
+	if (test_bit(TIF_SCHED_JB, &_prev_flags))
+		lkl_ops->jmp_buf_longjmp(_prev_jb, 1);
+	else
+		lkl_ops->sem_down(_prev->sched_sem);
+
+	if (_prev->dead)
+		lkl_ops->thread_exit();
+
+	/* __switch_to (arch/um) returns this value */
+	current->thread.prev_sched = abs_prev;
+}
+
+int host_task_stub(void *unused)
+{
+	return 0;
+}
+
+void switch_to_host_task(struct task_struct *task)
+{
+	if (WARN_ON(!test_tsk_thread_flag(task, TIF_HOST_THREAD)))
+		return;
+
+	task->thread.arch.tid = lkl_ops->thread_self();
+
+	if (current == task)
+		return;
+
+	wake_up_process(task);
+	thread_sched_jb();
+	lkl_ops->sem_down(task->thread.arch.sched_sem);
+	schedule_tail(abs_prev);
+}
+
+struct thread_bootstrap_arg {
+	struct thread_info *ti;
+	int (*f)(void *a);
+	void *arg;
+};
+
+static void *thread_bootstrap(void *_tba)
+{
+	struct thread_bootstrap_arg *tba = (struct thread_bootstrap_arg *)_tba;
+	struct thread_info *ti = tba->ti;
+	int (*f)(void *) = tba->f;
+	void *arg = tba->arg;
+
+	lkl_ops->sem_down(ti->task->thread.arch.sched_sem);
+	kfree(tba);
+	if (ti->task->thread.prev_sched)
+		schedule_tail(ti->task->thread.prev_sched);
+
+	f(arg);
+	do_exit(0);
+
+	return NULL;
+}
+
+void new_thread(void *stack, jmp_buf *buf, void (*handler)(void))
+{
+	struct thread_info *ti = (struct thread_info *)stack;
+	struct task_struct *p = ti->task;
+	struct thread_bootstrap_arg *tba;
+	int ret;
+
+	unsigned long esp = (unsigned long)p->thread.request.u.thread.proc;
+	unsigned long unused = (unsigned long)p->thread.request.u.thread.arg;
+
+	ret = init_arch_thread(&p->thread.arch);
+	if (ret < 0)
+		panic("%s: init_arch_thread", __func__);
+
+	if ((int (*)(void *))esp == host_task_stub) {
+		set_ti_thread_flag(ti, TIF_HOST_THREAD);
+		return;
+	}
+
+	tba = kmalloc(sizeof(*tba), GFP_KERNEL);
+	if (!tba)
+		return;
+
+	tba->f = (int (*)(void *))esp;
+	tba->arg = (void *)unused;
+	tba->ti = ti;
+
+	p->thread.arch.tid = lkl_ops->thread_create(thread_bootstrap, tba);
+	if (!p->thread.arch.tid) {
+		kfree(tba);
+		return;
+	}
+}
+
+void show_stack(struct task_struct *task, unsigned long *esp)
+{
+}
+
+static inline void pr_early(const char *str)
+{
+	if (lkl_ops->print)
+		lkl_ops->print(str, strlen(str));
+}
+
+/**
+ * This is called before the kernel initializes, so no kernel calls (including
+ * printk) can't be made yet.
+ */
+void threads_init(void)
+{
+	int ret;
+	struct thread_info *ti = &init_thread_union.thread_info;
+
+	ti->task->thread = (struct thread_struct) INIT_THREAD;
+	ret = init_arch_thread(&ti->task->thread.arch);
+	if (ret < 0)
+		pr_early("lkl: failed to allocate init schedule semaphore\n");
+
+	ti->task->thread.arch.tid = lkl_ops->thread_self();
+}
+
+void threads_cleanup(void)
+{
+	struct task_struct *p, *t;
+
+	for_each_process_thread(p, t) {
+		struct thread_info *ti = task_thread_info(t);
+
+		if (t->pid != 1 && !test_ti_thread_flag(ti, TIF_HOST_THREAD))
+			WARN(!(t->flags & PF_KTHREAD),
+			     "non kernel thread task %s\n", t->comm);
+		WARN(t->state == TASK_RUNNING,
+		     "thread %s still running while halting\n", t->comm);
+
+		kill_thread(ti);
+	}
+
+	lkl_ops->sem_free(
+		init_thread_union.thread_info.task->thread.arch.sched_sem);
+}
+
+void initial_thread_cb_skas(void (*proc)(void *), void *arg)
+{
+	pr_warn("unimplemented %s", __func__);
+}
+
+int arch_set_tls(struct task_struct *new, unsigned long tls)
+{
+	panic("unimplemented %s", __func__);
+}
+void clear_flushed_tls(struct task_struct *task)
+{
+	panic("unimplemented %s", __func__);
+}
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v7 11/21] um: nommu: kernel thread support
@ 2020-10-06  9:44           ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-06  9:44 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Hajime Tazaki, retrage01

nommu mode does not support user processes but it must support kernel
threads as part as the normal kernel work-flow. It uses host operations
to create and terminate host threads that are going to run the kernel
threads. It also uses semaphores to synchronize those threads and to
allow the Linux kernel scheduler to control how the kernel threads
run.

Each kernel thread runs in a host threads and has a host semaphore
associated with it - the thread's scheduling semaphore. The semaphore
counter is initialized to 0. The first thing a kernel thread does
after getting spawned, before running any kernel code, is to perform a
down operation to block the thread.

The kernel controls host threads scheduling by performing up and down
operations on the scheduling semaphore. In __switch_context an up
operation on the next thread is performed to wake up a blocked thread,
and a down operation is performed on the prev thread to block it.

A thread is terminated by marking it in free_thread_info and
performing an up operation on the scheduling semaphore at which point
the marked thread will terminate itself.

UML common part (process.c) is extended to call idle functions of
SUBARCH as well as kernel thread detection with a flag information of
TIF_HOST_THREAD.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/include/asm/thread_info.h         |  24 ++
 arch/um/kernel/process.c                  |   8 +-
 arch/um/nommu/include/uapi/asm/host_ops.h |  54 +++++
 arch/um/nommu/um/cpu.c                    | 236 ++++++++++++++++++++
 arch/um/nommu/um/threads.c                | 259 ++++++++++++++++++++++
 5 files changed, 580 insertions(+), 1 deletion(-)
 create mode 100644 arch/um/nommu/um/cpu.c
 create mode 100644 arch/um/nommu/um/threads.c

diff --git a/arch/um/include/asm/thread_info.h b/arch/um/include/asm/thread_info.h
index 4c19ce4c49f1..433584f6b9d0 100644
--- a/arch/um/include/asm/thread_info.h
+++ b/arch/um/include/asm/thread_info.h
@@ -40,6 +40,7 @@ struct thread_info {
 	.real_thread = NULL,			\
 }
 
+#ifdef CONFIG_MMU
 /* how to get the thread information struct from C */
 static inline struct thread_info *current_thread_info(void)
 {
@@ -52,6 +53,27 @@ static inline struct thread_info *current_thread_info(void)
 	return ti;
 }
 
+#else
+
+#define __HAVE_THREAD_FUNCTIONS
+#define task_thread_info(task)	((struct thread_info *)(task)->stack)
+#define task_stack_page(task)	((task)->stack)
+#define end_of_stack(p) (&task_thread_info(p)->aux_fp_regs[FP_SIZE-1])
+
+void threads_init(void);
+void threads_cleanup(void);
+
+unsigned long *alloc_thread_stack_node(struct task_struct *p, int node);
+void setup_thread_stack(struct task_struct *p, struct task_struct *org);
+void free_thread_stack(struct task_struct *tsk);
+
+extern struct thread_info *_current_thread_info;
+static inline struct thread_info *current_thread_info(void)
+{
+	return _current_thread_info;
+}
+#endif /* CONFIG_MMU */
+
 #endif
 
 #define TIF_SYSCALL_TRACE	0	/* syscall trace active */
@@ -63,6 +85,8 @@ static inline struct thread_info *current_thread_info(void)
 #define TIF_RESTORE_SIGMASK	7
 #define TIF_NOTIFY_RESUME	8
 #define TIF_SECCOMP		9	/* secure computing */
+#define TIF_SCHED_JB		10
+#define TIF_HOST_THREAD		11
 
 #define _TIF_SYSCALL_TRACE	(1 << TIF_SYSCALL_TRACE)
 #define _TIF_SIGPENDING		(1 << TIF_SIGPENDING)
diff --git a/arch/um/kernel/process.c b/arch/um/kernel/process.c
index 87a8cfa228ca..432edcbb5eea 100644
--- a/arch/um/kernel/process.c
+++ b/arch/um/kernel/process.c
@@ -156,7 +156,8 @@ int copy_thread(unsigned long clone_flags, unsigned long sp,
 		unsigned long arg, struct task_struct * p, unsigned long tls)
 {
 	void (*handler)(void);
-	int kthread = current->flags & PF_KTHREAD;
+	int kthread = current->flags & PF_KTHREAD ||
+		test_ti_thread_flag(current_thread_info(), TIF_HOST_THREAD);
 	int ret = 0;
 
 	p->thread = (struct thread_struct) INIT_THREAD;
@@ -213,10 +214,15 @@ static void um_idle_sleep(void)
 	}
 }
 
+void __weak subarch_cpu_idle(void)
+{
+}
+
 void arch_cpu_idle(void)
 {
 	cpu_tasks[current_thread_info()->cpu].pid = os_getpid();
 	um_idle_sleep();
+	subarch_cpu_idle();
 	local_irq_enable();
 }
 
diff --git a/arch/um/nommu/include/uapi/asm/host_ops.h b/arch/um/nommu/include/uapi/asm/host_ops.h
index 5253c3f8de0e..720385fccbdf 100644
--- a/arch/um/nommu/include/uapi/asm/host_ops.h
+++ b/arch/um/nommu/include/uapi/asm/host_ops.h
@@ -16,11 +16,65 @@ struct lkl_jmp_buf {
  * These operations must be provided by a host library or by the application
  * itself.
  *
+ * @sem_alloc - allocate a host semaphore an initialize it to count
+ * @sem_free - free a host semaphore
+ * @sem_up - perform an up operation on the semaphore
+ * @sem_down - perform a down operation on the semaphore
+ *
+ * @mutex_alloc - allocate and initialize a host mutex; the recursive parameter
+ * determines if the mutex is recursive or not
+ * @mutex_free - free a host mutex
+ * @mutex_lock - acquire the mutex
+ * @mutex_unlock - release the mutex
+ *
+ * @thread_create - create a new thread and run f(arg) in its context; returns a
+ * thread handle or 0 if the thread could not be created
+ * @thread_detach - on POSIX systems, free up resources held by
+ * pthreads. Noop on Win32.
+ * @thread_exit - terminates the current thread
+ * @thread_join - wait for the given thread to terminate. Returns 0
+ * for success, -1 otherwise
+ *
+ * @gettid - returns the host thread id of the caller, which need not
+ * be the same as the handle returned by thread_create
+ *
+ * @jmp_buf_set - runs the give function and setups a jump back point by saving
+ * the context in the jump buffer; jmp_buf_longjmp can be called from the give
+ * function or any callee in that function to return back to the jump back
+ * point
+ *
+ * NOTE: we can't return from jmp_buf_set before calling jmp_buf_longjmp or
+ * otherwise the saved context (stack) is not going to be valid, so we must pass
+ * the function that will eventually call longjmp here
+ *
+ * @jmp_buf_longjmp - perform a jump back to the saved jump buffer
+ *
  * @mem_alloc - allocate memory
  * @mem_free - free memory
  *
  */
 struct lkl_host_operations {
+	struct lkl_sem *(*sem_alloc)(int count);
+	void (*sem_free)(struct lkl_sem *sem);
+	void (*sem_up)(struct lkl_sem *sem);
+	void (*sem_down)(struct lkl_sem *sem);
+
+	struct lkl_mutex *(*mutex_alloc)(int recursive);
+	void (*mutex_free)(struct lkl_mutex *mutex);
+	void (*mutex_lock)(struct lkl_mutex *mutex);
+	void (*mutex_unlock)(struct lkl_mutex *mutex);
+
+	lkl_thread_t (*thread_create)(void *(*f)(void *), void *arg);
+	void (*thread_detach)(void);
+	void (*thread_exit)(void);
+	int (*thread_join)(lkl_thread_t tid);
+	lkl_thread_t (*thread_self)(void);
+	int (*thread_equal)(lkl_thread_t a, lkl_thread_t b);
+	long (*gettid)(void);
+
+	void (*jmp_buf_set)(struct lkl_jmp_buf *jmpb, void (*f)(void));
+	void (*jmp_buf_longjmp)(struct lkl_jmp_buf *jmpb, int val);
+
 	void *(*mem_alloc)(unsigned long mem);
 	void (*mem_free)(void *mem);
 };
diff --git a/arch/um/nommu/um/cpu.c b/arch/um/nommu/um/cpu.c
new file mode 100644
index 000000000000..9986a3f8c5dd
--- /dev/null
+++ b/arch/um/nommu/um/cpu.c
@@ -0,0 +1,236 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/kernel.h>
+#include <linux/sched/stat.h>
+#include <asm/host_ops.h>
+#include <asm/cpu.h>
+#include <asm/thread_info.h>
+#include <asm/unistd.h>
+#include <asm/sched.h>
+#include <asm/syscalls.h>
+#include <init.h>
+
+void run_irqs(void)
+{
+	panic("unimplemented %s", __func__);
+}
+
+int set_irq_pending(int irq)
+{
+	panic("unimplemented %s", __func__);
+}
+
+/*
+ * This structure is used to get access to the "LKL CPU" that allows us to run
+ * Linux code. Because we have to deal with various synchronization requirements
+ * between idle thread, system calls, interrupts, "reentrancy", CPU shutdown,
+ * imbalance wake up (i.e. acquire the CPU from one thread and release it from
+ * another), we can't use a simple synchronization mechanism such as (recursive)
+ * mutex or semaphore. Instead, we use a mutex and a bunch of status data plus a
+ * semaphore.
+ */
+static struct lkl_cpu {
+	/* lock that protects the CPU status data */
+	struct lkl_mutex *lock;
+	/*
+	 * Since we must free the cpu lock during shutdown we need a
+	 * synchronization algorithm between lkl_cpu_shutdown() and the CPU
+	 * access functions since lkl_cpu_get() gets called from thread
+	 * destructor callback functions which may be scheduled after
+	 * lkl_cpu_shutdown() has freed the cpu lock.
+	 *
+	 * An atomic counter is used to keep track of the number of running
+	 * CPU access functions and allow the shutdown function to wait for
+	 * them.
+	 *
+	 * The shutdown functions adds MAX_THREADS to this counter which allows
+	 * the CPU access functions to check if the shutdown process has
+	 * started.
+	 *
+	 * This algorithm assumes that we never have more the MAX_THREADS
+	 * requesting CPU access.
+	 */
+	#define MAX_THREADS 1000000
+	unsigned int shutdown_gate;
+	bool irqs_pending;
+	/* no of threads waiting the CPU */
+	unsigned int sleepers;
+	/* no of times the current thread got the CPU */
+	unsigned int count;
+	/* current thread that owns the CPU */
+	lkl_thread_t owner;
+	/* semaphore for threads waiting the CPU */
+	struct lkl_sem *sem;
+	/* semaphore used for shutdown */
+	struct lkl_sem *shutdown_sem;
+} cpu;
+
+static int __cpu_try_get_lock(int n)
+{
+	lkl_thread_t self;
+
+	if (__sync_fetch_and_add(&cpu.shutdown_gate, n) >= MAX_THREADS)
+		return -2;
+
+	lkl_ops->mutex_lock(cpu.lock);
+
+	if (cpu.shutdown_gate >= MAX_THREADS)
+		return -1;
+
+	self = lkl_ops->thread_self();
+
+	if (cpu.owner && !lkl_ops->thread_equal(cpu.owner, self))
+		return 0;
+
+	cpu.owner = self;
+	cpu.count++;
+
+	return 1;
+}
+
+static void __cpu_try_get_unlock(int lock_ret, int n)
+{
+	if (lock_ret >= -1)
+		lkl_ops->mutex_unlock(cpu.lock);
+	__sync_fetch_and_sub(&cpu.shutdown_gate, n);
+}
+
+void lkl_cpu_change_owner(lkl_thread_t owner)
+{
+	lkl_ops->mutex_lock(cpu.lock);
+	if (cpu.count > 1)
+		lkl_bug("bad count while changing owner\n");
+	cpu.owner = owner;
+	lkl_ops->mutex_unlock(cpu.lock);
+}
+
+int lkl_cpu_get(void)
+{
+	int ret;
+
+	ret = __cpu_try_get_lock(1);
+
+	while (ret == 0) {
+		cpu.sleepers++;
+		__cpu_try_get_unlock(ret, 0);
+		lkl_ops->sem_down(cpu.sem);
+		ret = __cpu_try_get_lock(0);
+	}
+
+	__cpu_try_get_unlock(ret, 1);
+
+	return ret;
+}
+
+void lkl_cpu_put(void)
+{
+	lkl_ops->mutex_lock(cpu.lock);
+
+	if (!cpu.count || !cpu.owner ||
+	    !lkl_ops->thread_equal(cpu.owner, lkl_ops->thread_self()))
+		lkl_bug("%s: unbalanced put\n", __func__);
+
+	while (cpu.irqs_pending && !irqs_disabled()) {
+		cpu.irqs_pending = false;
+		lkl_ops->mutex_unlock(cpu.lock);
+		run_irqs();
+		lkl_ops->mutex_lock(cpu.lock);
+	}
+
+	if (test_ti_thread_flag(current_thread_info(), TIF_HOST_THREAD) &&
+	    !single_task_running() && cpu.count == 1) {
+		if (in_interrupt())
+			lkl_bug("%s: in interrupt\n", __func__);
+		lkl_ops->mutex_unlock(cpu.lock);
+		thread_sched_jb();
+		return;
+	}
+
+	if (--cpu.count > 0) {
+		lkl_ops->mutex_unlock(cpu.lock);
+		return;
+	}
+
+	if (cpu.sleepers) {
+		cpu.sleepers--;
+		lkl_ops->sem_up(cpu.sem);
+	}
+
+	cpu.owner = 0;
+
+	lkl_ops->mutex_unlock(cpu.lock);
+}
+
+int lkl_cpu_try_run_irq(int irq)
+{
+	int ret;
+
+	ret = __cpu_try_get_lock(1);
+	if (!ret) {
+		set_irq_pending(irq);
+		cpu.irqs_pending = true;
+	}
+	__cpu_try_get_unlock(ret, 1);
+
+	return ret;
+}
+
+static void lkl_cpu_shutdown(void)
+{
+	__sync_fetch_and_add(&cpu.shutdown_gate, MAX_THREADS);
+}
+__uml_exitcall(lkl_cpu_shutdown);
+
+void lkl_cpu_wait_shutdown(void)
+{
+	lkl_ops->sem_down(cpu.shutdown_sem);
+	lkl_ops->sem_free(cpu.shutdown_sem);
+}
+
+static void lkl_cpu_cleanup(bool shutdown)
+{
+	while (__sync_fetch_and_add(&cpu.shutdown_gate, 0) > MAX_THREADS)
+		;
+
+	if (shutdown)
+		lkl_ops->sem_up(cpu.shutdown_sem);
+	else if (cpu.shutdown_sem)
+		lkl_ops->sem_free(cpu.shutdown_sem);
+	if (cpu.sem)
+		lkl_ops->sem_free(cpu.sem);
+	if (cpu.lock)
+		lkl_ops->mutex_free(cpu.lock);
+}
+
+void subarch_cpu_idle(void)
+{
+	if (cpu.shutdown_gate >= MAX_THREADS) {
+
+		lkl_ops->mutex_lock(cpu.lock);
+		while (cpu.sleepers--)
+			lkl_ops->sem_up(cpu.sem);
+		lkl_ops->mutex_unlock(cpu.lock);
+
+		lkl_cpu_cleanup(true);
+
+		lkl_ops->thread_exit();
+	}
+
+#ifdef doesntwork
+	/* switch to idle_host_task */
+	wakeup_idle_host_task();
+#endif
+}
+
+int lkl_cpu_init(void)
+{
+	cpu.lock = lkl_ops->mutex_alloc(0);
+	cpu.sem = lkl_ops->sem_alloc(0);
+	cpu.shutdown_sem = lkl_ops->sem_alloc(0);
+
+	if (!cpu.lock || !cpu.sem || !cpu.shutdown_sem) {
+		lkl_cpu_cleanup(false);
+		return -ENOMEM;
+	}
+
+	return 0;
+}
diff --git a/arch/um/nommu/um/threads.c b/arch/um/nommu/um/threads.c
new file mode 100644
index 000000000000..e9a49100d40e
--- /dev/null
+++ b/arch/um/nommu/um/threads.c
@@ -0,0 +1,259 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/slab.h>
+#include <linux/sched/task.h>
+#include <linux/sched/signal.h>
+#include <asm/host_ops.h>
+#include <asm/cpu.h>
+#include <asm/sched.h>
+
+#include <os.h>
+
+static int init_arch_thread(struct arch_thread *thread)
+{
+	thread->sched_sem = lkl_ops->sem_alloc(0);
+	if (!thread->sched_sem)
+		return -ENOMEM;
+
+	thread->dead = false;
+	thread->tid = 0;
+
+	return 0;
+}
+
+unsigned long *alloc_thread_stack_node(struct task_struct *task, int node)
+{
+	struct thread_info *ti;
+
+	ti = kmalloc(sizeof(*ti), GFP_KERNEL | __GFP_ZERO);
+	if (!ti)
+		return NULL;
+
+	ti->task = task;
+	return (unsigned long *)ti;
+}
+
+/*
+ * The only new tasks created are kernel threads that have a predefined starting
+ * point thus no stack copy is required.
+ */
+void setup_thread_stack(struct task_struct *p, struct task_struct *org)
+{
+	struct thread_info *ti = task_thread_info(p);
+	struct thread_info *org_ti = task_thread_info(org);
+
+	ti->flags = org_ti->flags;
+	ti->preempt_count = org_ti->preempt_count;
+	ti->addr_limit = org_ti->addr_limit;
+}
+
+static void kill_thread(struct thread_info *ti)
+{
+	if (!test_ti_thread_flag(ti, TIF_HOST_THREAD)) {
+		ti->task->thread.arch.dead = true;
+		lkl_ops->sem_up(ti->task->thread.arch.sched_sem);
+		lkl_ops->thread_join(ti->task->thread.arch.tid);
+	}
+	lkl_ops->sem_free(ti->task->thread.arch.sched_sem);
+}
+
+void free_thread_stack(struct task_struct *tsk)
+{
+	struct thread_info *ti = task_thread_info(tsk);
+
+	kill_thread(ti);
+	kfree(ti);
+}
+
+struct thread_info *_current_thread_info = &init_thread_union.thread_info;
+EXPORT_SYMBOL(_current_thread_info);
+
+void switch_threads(jmp_buf *me, jmp_buf *you)
+{
+	/* NOP */
+}
+
+/*
+ * schedule() expects the return of this function to be the task that we
+ * switched away from. Returning prev is not going to work because we are
+ * actually going to return the previous taks that was scheduled before the
+ * task we are going to wake up, and not the current task, e.g.:
+ *
+ * swapper -> init: saved prev on swapper stack is swapper
+ * init -> ksoftirqd0: saved prev on init stack is init
+ * ksoftirqd0 -> swapper: returned prev is swapper
+ */
+static struct task_struct *abs_prev = &init_task;
+
+void arch_switch_to(struct task_struct *prev,
+		    struct task_struct *next)
+{
+	struct arch_thread *_prev = &prev->thread.arch;
+	struct arch_thread *_next = &next->thread.arch;
+	unsigned long _prev_flags = task_thread_info(prev)->flags;
+	struct lkl_jmp_buf *_prev_jb;
+
+	_current_thread_info = task_thread_info(next);
+	next->thread.prev_sched = prev;
+	abs_prev = prev;
+
+	WARN_ON(!_next->tid);
+	lkl_cpu_change_owner(_next->tid);
+
+	if (test_bit(TIF_SCHED_JB, &_prev_flags)) {
+		/* Atomic. Must be done before wakeup next */
+		clear_ti_thread_flag(task_thread_info(prev), TIF_SCHED_JB);
+		_prev_jb = &_prev->sched_jb;
+	}
+
+	lkl_ops->sem_up(_next->sched_sem);
+	if (test_bit(TIF_SCHED_JB, &_prev_flags))
+		lkl_ops->jmp_buf_longjmp(_prev_jb, 1);
+	else
+		lkl_ops->sem_down(_prev->sched_sem);
+
+	if (_prev->dead)
+		lkl_ops->thread_exit();
+
+	/* __switch_to (arch/um) returns this value */
+	current->thread.prev_sched = abs_prev;
+}
+
+int host_task_stub(void *unused)
+{
+	return 0;
+}
+
+void switch_to_host_task(struct task_struct *task)
+{
+	if (WARN_ON(!test_tsk_thread_flag(task, TIF_HOST_THREAD)))
+		return;
+
+	task->thread.arch.tid = lkl_ops->thread_self();
+
+	if (current == task)
+		return;
+
+	wake_up_process(task);
+	thread_sched_jb();
+	lkl_ops->sem_down(task->thread.arch.sched_sem);
+	schedule_tail(abs_prev);
+}
+
+struct thread_bootstrap_arg {
+	struct thread_info *ti;
+	int (*f)(void *a);
+	void *arg;
+};
+
+static void *thread_bootstrap(void *_tba)
+{
+	struct thread_bootstrap_arg *tba = (struct thread_bootstrap_arg *)_tba;
+	struct thread_info *ti = tba->ti;
+	int (*f)(void *) = tba->f;
+	void *arg = tba->arg;
+
+	lkl_ops->sem_down(ti->task->thread.arch.sched_sem);
+	kfree(tba);
+	if (ti->task->thread.prev_sched)
+		schedule_tail(ti->task->thread.prev_sched);
+
+	f(arg);
+	do_exit(0);
+
+	return NULL;
+}
+
+void new_thread(void *stack, jmp_buf *buf, void (*handler)(void))
+{
+	struct thread_info *ti = (struct thread_info *)stack;
+	struct task_struct *p = ti->task;
+	struct thread_bootstrap_arg *tba;
+	int ret;
+
+	unsigned long esp = (unsigned long)p->thread.request.u.thread.proc;
+	unsigned long unused = (unsigned long)p->thread.request.u.thread.arg;
+
+	ret = init_arch_thread(&p->thread.arch);
+	if (ret < 0)
+		panic("%s: init_arch_thread", __func__);
+
+	if ((int (*)(void *))esp == host_task_stub) {
+		set_ti_thread_flag(ti, TIF_HOST_THREAD);
+		return;
+	}
+
+	tba = kmalloc(sizeof(*tba), GFP_KERNEL);
+	if (!tba)
+		return;
+
+	tba->f = (int (*)(void *))esp;
+	tba->arg = (void *)unused;
+	tba->ti = ti;
+
+	p->thread.arch.tid = lkl_ops->thread_create(thread_bootstrap, tba);
+	if (!p->thread.arch.tid) {
+		kfree(tba);
+		return;
+	}
+}
+
+void show_stack(struct task_struct *task, unsigned long *esp)
+{
+}
+
+static inline void pr_early(const char *str)
+{
+	if (lkl_ops->print)
+		lkl_ops->print(str, strlen(str));
+}
+
+/**
+ * This is called before the kernel initializes, so no kernel calls (including
+ * printk) can't be made yet.
+ */
+void threads_init(void)
+{
+	int ret;
+	struct thread_info *ti = &init_thread_union.thread_info;
+
+	ti->task->thread = (struct thread_struct) INIT_THREAD;
+	ret = init_arch_thread(&ti->task->thread.arch);
+	if (ret < 0)
+		pr_early("lkl: failed to allocate init schedule semaphore\n");
+
+	ti->task->thread.arch.tid = lkl_ops->thread_self();
+}
+
+void threads_cleanup(void)
+{
+	struct task_struct *p, *t;
+
+	for_each_process_thread(p, t) {
+		struct thread_info *ti = task_thread_info(t);
+
+		if (t->pid != 1 && !test_ti_thread_flag(ti, TIF_HOST_THREAD))
+			WARN(!(t->flags & PF_KTHREAD),
+			     "non kernel thread task %s\n", t->comm);
+		WARN(t->state == TASK_RUNNING,
+		     "thread %s still running while halting\n", t->comm);
+
+		kill_thread(ti);
+	}
+
+	lkl_ops->sem_free(
+		init_thread_union.thread_info.task->thread.arch.sched_sem);
+}
+
+void initial_thread_cb_skas(void (*proc)(void *), void *arg)
+{
+	pr_warn("unimplemented %s", __func__);
+}
+
+int arch_set_tls(struct task_struct *new, unsigned long tls)
+{
+	panic("unimplemented %s", __func__);
+}
+void clear_flushed_tls(struct task_struct *task)
+{
+	panic("unimplemented %s", __func__);
+}
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v7 12/21] um: nommu: system call interface and application API
  2020-10-06  9:44         ` Hajime Tazaki
@ 2020-10-06  9:44           ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-06  9:44 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch, Hajime Tazaki

The application API is based on the kernel system call interface in
order to offer a stable API to applications. Note that we can't
offer the full Linux system call interface due to the limitations of
nommu mode such as lack of virtual memory, signal, user processes, etc.

The host is using dedicated kernel thread, which is switched upon the
entry of lkl_syscall so that numbers of context switches can be reduced
for the faster performance.

To expose the API definitions to applications, this commit uses
syscall_wrapper to define arch-specific information of syscall, and this
is used to generate syscall_defs.h for lkl so that it can be used
as a template of the list of syscall available for LKL apps.

To avoid collisions between the Linux API and the LKL API (e.g.  struct
stat, MKNOD, etc.) we use a python script to modify the user headers
and to prefix all of the global symbols (structures, typedefs,
defines) with LKL, lkl, _LKL, _lkl, __LKL or __lkl.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/include/uapi/asm/Kbuild             |   2 +
 arch/um/nommu/include/asm/syscall_wrapper.h |  57 ++++
 arch/um/nommu/include/asm/syscalls.h        |  15 +
 arch/um/nommu/include/uapi/asm/host_ops.h   |  14 +
 arch/um/nommu/include/uapi/asm/syscalls.h   | 287 ++++++++++++++++++++
 arch/um/nommu/include/uapi/asm/unistd.h     |  15 +
 arch/um/nommu/um/syscalls.c                 | 199 ++++++++++++++
 arch/um/scripts/headers_install.py          | 197 ++++++++++++++
 8 files changed, 786 insertions(+)
 create mode 100644 arch/um/include/uapi/asm/Kbuild
 create mode 100644 arch/um/nommu/include/asm/syscall_wrapper.h
 create mode 100644 arch/um/nommu/include/asm/syscalls.h
 create mode 100644 arch/um/nommu/include/uapi/asm/syscalls.h
 create mode 100644 arch/um/nommu/include/uapi/asm/unistd.h
 create mode 100644 arch/um/nommu/um/syscalls.c
 create mode 100755 arch/um/scripts/headers_install.py

diff --git a/arch/um/include/uapi/asm/Kbuild b/arch/um/include/uapi/asm/Kbuild
new file mode 100644
index 000000000000..a983c7f049bc
--- /dev/null
+++ b/arch/um/include/uapi/asm/Kbuild
@@ -0,0 +1,2 @@
+# no generated-y since we need special user headers handling
+# see arch/um/script/headers_install.py
diff --git a/arch/um/nommu/include/asm/syscall_wrapper.h b/arch/um/nommu/include/asm/syscall_wrapper.h
new file mode 100644
index 000000000000..bb2a7d4274f6
--- /dev/null
+++ b/arch/um/nommu/include/asm/syscall_wrapper.h
@@ -0,0 +1,57 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef __UM_SYSCALL_WRAPPER_H
+#define __UM_SYSCALL_WRAPPER_H
+
+#define __SC_ASCII(t, a) #t "," #a
+
+#define __ASCII_MAP0(m, ...)
+#define __ASCII_MAP1(m, t, a) m(t, a)
+#define __ASCII_MAP2(m, t, a, ...) m(t, a) "," __ASCII_MAP1(m, __VA_ARGS__)
+#define __ASCII_MAP3(m, t, a, ...) m(t, a) "," __ASCII_MAP2(m, __VA_ARGS__)
+#define __ASCII_MAP4(m, t, a, ...) m(t, a) "," __ASCII_MAP3(m, __VA_ARGS__)
+#define __ASCII_MAP5(m, t, a, ...) m(t, a) "," __ASCII_MAP4(m, __VA_ARGS__)
+#define __ASCII_MAP6(m, t, a, ...) m(t, a) "," __ASCII_MAP5(m, __VA_ARGS__)
+#define __ASCII_MAP(n, ...) __ASCII_MAP##n(__VA_ARGS__)
+
+#ifdef __MINGW32__
+#define SECTION_ATTRS "n0"
+#else
+#define SECTION_ATTRS "a"
+#endif
+
+#define __SYSCALL_DEFINE_ARCH(x, name, ...)				\
+	asm(".section .syscall_defs,\"" SECTION_ATTRS "\"\n"		\
+	    ".ascii \"#ifdef __NR" #name "\\n\"\n"			\
+	    ".ascii \"SYSCALL_DEFINE" #x "(" #name ","			\
+	    __ASCII_MAP(x, __SC_ASCII, __VA_ARGS__) ")\\n\"\n"		\
+	    ".ascii \"#endif\\n\"\n"					\
+	    ".section .text\n")
+
+#define SYSCALL_DEFINE0(sname)					\
+	SYSCALL_METADATA(_##sname, 0);				\
+	__SYSCALL_DEFINE_ARCH(0, _##sname);			\
+	asmlinkage long sys_##sname(void);			\
+	ALLOW_ERROR_INJECTION(sys_##sname, ERRNO);		\
+	asmlinkage long sys_##sname(void)
+
+#define __SYSCALL_DEFINEx(x, name, ...)					\
+	__SYSCALL_DEFINE_ARCH(x, name, __VA_ARGS__);			\
+	__diag_push();							\
+	__diag_ignore(GCC, 8, "-Wattribute-alias",			\
+		      "Type aliasing is used to sanitize syscall arguments");\
+	asmlinkage long sys##name(__MAP(x, __SC_DECL, __VA_ARGS__))	\
+		__attribute__((alias(__stringify(__se_sys##name))));	\
+	ALLOW_ERROR_INJECTION(sys##name, ERRNO);			\
+	static inline long __do_sys##name(__MAP(x, __SC_DECL, __VA_ARGS__)); \
+	asmlinkage long __se_sys##name(__MAP(x, __SC_LONG, __VA_ARGS__)); \
+	asmlinkage long __se_sys##name(__MAP(x, __SC_LONG, __VA_ARGS__)) \
+	{								\
+		long ret = __do_sys##name(__MAP(x, __SC_CAST, __VA_ARGS__)); \
+		__MAP(x, __SC_TEST, __VA_ARGS__);			\
+		__PROTECT(x, ret, __MAP(x, __SC_ARGS, __VA_ARGS__));	\
+		return ret;						\
+	}								\
+	__diag_pop();							\
+	static inline long __do_sys##name(__MAP(x, __SC_DECL, __VA_ARGS__))
+
+#endif /* __UM_SYSCALL_WRAPPER_H */
diff --git a/arch/um/nommu/include/asm/syscalls.h b/arch/um/nommu/include/asm/syscalls.h
new file mode 100644
index 000000000000..6061d9415dad
--- /dev/null
+++ b/arch/um/nommu/include/asm/syscalls.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+int syscalls_init(void);
+void syscalls_cleanup(void);
+long lkl_syscall(long no, long *params);
+void wakeup_idle_host_task(void);
+
+#define sys_mmap sys_ni_syscall
+#define sys_rt_sigreturn sys_ni_syscall
+#define sys_arch_prctl sys_ni_syscall
+#define sys_iopl sys_ni_syscall
+#define sys_ioperm sys_ni_syscall
+#define sys_clone sys_ni_syscall
+
+int run_syscalls(void);
diff --git a/arch/um/nommu/include/uapi/asm/host_ops.h b/arch/um/nommu/include/uapi/asm/host_ops.h
index 720385fccbdf..6ee9489fc47e 100644
--- a/arch/um/nommu/include/uapi/asm/host_ops.h
+++ b/arch/um/nommu/include/uapi/asm/host_ops.h
@@ -38,6 +38,15 @@ struct lkl_jmp_buf {
  * @gettid - returns the host thread id of the caller, which need not
  * be the same as the handle returned by thread_create
  *
+ * @tls_alloc - allocate a thread local storage key; returns 0 if successful; if
+ * destructor is not NULL it will be called when a thread terminates with its
+ * argument set to the current thread local storage value
+ * @tls_free - frees a thread local storage key; returns 0 if successful
+ * @tls_set - associate data to the thread local storage key; returns 0 if
+ * successful
+ * @tls_get - return data associated with the thread local storage key or NULL
+ * on error
+ *
  * @jmp_buf_set - runs the give function and setups a jump back point by saving
  * the context in the jump buffer; jmp_buf_longjmp can be called from the give
  * function or any callee in that function to return back to the jump back
@@ -72,6 +81,11 @@ struct lkl_host_operations {
 	int (*thread_equal)(lkl_thread_t a, lkl_thread_t b);
 	long (*gettid)(void);
 
+	struct lkl_tls_key *(*tls_alloc)(void (*destructor)(void *));
+	void (*tls_free)(struct lkl_tls_key *key);
+	int (*tls_set)(struct lkl_tls_key *key, void *data);
+	void *(*tls_get)(struct lkl_tls_key *key);
+
 	void (*jmp_buf_set)(struct lkl_jmp_buf *jmpb, void (*f)(void));
 	void (*jmp_buf_longjmp)(struct lkl_jmp_buf *jmpb, int val);
 
diff --git a/arch/um/nommu/include/uapi/asm/syscalls.h b/arch/um/nommu/include/uapi/asm/syscalls.h
new file mode 100644
index 000000000000..751957b978cd
--- /dev/null
+++ b/arch/um/nommu/include/uapi/asm/syscalls.h
@@ -0,0 +1,287 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef __UM_NOMMU_UAPI_SYSCALLS_H
+#define __UM_NOMMU_UAPI_SYSCALLS_H
+
+#include <autoconf.h>
+#include <linux/types.h>
+
+typedef __kernel_uid32_t	qid_t;
+typedef __kernel_fd_set		fd_set;
+typedef __kernel_mode_t		mode_t;
+typedef unsigned short		umode_t;
+typedef __u32			nlink_t;
+typedef __kernel_off_t		off_t;
+typedef __kernel_pid_t		pid_t;
+typedef __kernel_key_t		key_t;
+typedef __kernel_suseconds_t	suseconds_t;
+typedef __kernel_timer_t	timer_t;
+typedef __kernel_clockid_t	clockid_t;
+typedef __kernel_mqd_t		mqd_t;
+typedef __kernel_uid32_t	uid_t;
+typedef __kernel_gid32_t	gid_t;
+typedef __kernel_uid16_t        uid16_t;
+typedef __kernel_gid16_t        gid16_t;
+typedef unsigned long		uintptr_t;
+#ifdef CONFIG_UID16
+typedef __kernel_old_uid_t	old_uid_t;
+typedef __kernel_old_gid_t	old_gid_t;
+#endif
+typedef __kernel_loff_t		loff_t;
+typedef __kernel_size_t		size_t;
+typedef __kernel_ssize_t	ssize_t;
+typedef __kernel_time_t		time_t;
+typedef __kernel_clock_t	clock_t;
+typedef __u32			u32;
+typedef __s32			s32;
+typedef __u64			u64;
+typedef __s64			s64;
+
+typedef __kernel_long_t	__kernel_old_time_t;
+
+#define __user
+
+#include <asm/unistd.h>
+/* Temporary undefine system calls that don't have data types defined in UAPI
+ * headers
+ */
+#undef __NR_kexec_load
+#undef __NR_getcpu
+#undef __NR_sched_getattr
+#undef __NR_sched_setattr
+#undef __NR_sched_setparam
+#undef __NR_sched_getparam
+#undef __NR_sched_setscheduler
+#undef __NR_name_to_handle_at
+#undef __NR_open_by_handle_at
+
+/* deprecated system calls */
+#undef __NR_access
+#undef __NR_chmod
+#undef __NR_chown
+#undef __NR_lchown
+#undef __NR_open
+#undef __NR_creat
+#undef __NR_readlink
+#undef __NR_pipe
+#undef __NR_mknod
+#undef __NR_mkdir
+#undef __NR_rmdir
+#undef __NR_unlink
+#undef __NR_symlink
+#undef __NR_link
+#undef __NR_rename
+#undef __NR_getdents
+#undef __NR_select
+#undef __NR_poll
+#undef __NR_dup2
+#undef __NR_sysfs
+#undef __NR_ustat
+#undef __NR_inotify_init
+#undef __NR_epoll_create
+#undef __NR_epoll_wait
+#undef __NR_signalfd
+#undef __NR_eventfd
+
+#undef __NR_umount
+#define __NR_umount __NR_umount2
+
+#ifdef CONFIG_64BIT
+#define __NR_newfstat __NR3264_fstat
+#define __NR_newfstatat __NR3264_fstatat
+#endif
+
+#include <linux/time.h>
+#include <linux/times.h>
+#include <linux/timex.h>
+#include <linux/capability.h>
+#define __KERNEL__ /* to pull in S_ definitions */
+#include <linux/stat.h>
+#undef __KERNEL__
+#include <linux/errno.h>
+#include <linux/fcntl.h>
+#include <linux/fs.h>
+#include <asm/statfs.h>
+#include <asm/stat.h>
+#include <linux/bpf.h>
+#include <linux/msg.h>
+#include <linux/resource.h>
+#include <linux/sysinfo.h>
+#include <linux/shm.h>
+#include <linux/aio_abi.h>
+#include <linux/socket.h>
+#include <linux/perf_event.h>
+#include <linux/sem.h>
+#include <linux/futex.h>
+#include <linux/poll.h>
+#include <linux/mqueue.h>
+#include <linux/eventpoll.h>
+#include <linux/uio.h>
+#include <asm/signal.h>
+#include <asm/siginfo.h>
+#include <linux/utime.h>
+#include <asm/socket.h>
+#include <linux/icmp.h>
+#include <linux/ip.h>
+
+/* Define data structures used in system calls that are not defined in UAPI
+ * headers
+ */
+struct sockaddr {
+	unsigned short int sa_family;
+	char sa_data[14];
+};
+
+#define __UAPI_DEF_IF_NET_DEVICE_FLAGS_LOWER_UP_DORMANT_ECHO 1
+#define __UAPI_DEF_IF_IFNAMSIZ	1
+#define __UAPI_DEF_IF_NET_DEVICE_FLAGS 1
+#define __UAPI_DEF_IF_IFREQ	1
+#define __UAPI_DEF_IF_IFMAP	1
+#include <linux/if.h>
+#define __UAPI_DEF_IN_IPPROTO	1
+#define __UAPI_DEF_IN_ADDR	1
+#define __UAPI_DEF_IN6_ADDR	1
+#define __UAPI_DEF_IP_MREQ	1
+#define __UAPI_DEF_IN_PKTINFO	1
+#define __UAPI_DEF_SOCKADDR_IN	1
+#define __UAPI_DEF_IN_CLASS	1
+#include <linux/in.h>
+#include <linux/in6.h>
+#include <linux/sockios.h>
+#include <linux/route.h>
+#include <linux/ipv6_route.h>
+#include <linux/ipv6.h>
+#include <linux/netlink.h>
+#include <linux/neighbour.h>
+#include <linux/rtnetlink.h>
+#include <linux/fib_rules.h>
+
+#include <linux/kdev_t.h>
+#include <linux/virtio_blk.h>
+#include <linux/virtio_net.h>
+#include <linux/virtio_ring.h>
+#include <linux/pkt_sched.h>
+#include <linux/io_uring.h>
+#include <linux/utsname.h>
+
+struct user_msghdr {
+	void		__user *msg_name;
+	int		msg_namelen;
+	struct iovec	__user *msg_iov;
+	__kernel_size_t	msg_iovlen;
+	void		__user *msg_control;
+	__kernel_size_t	msg_controllen;
+	unsigned int	msg_flags;
+};
+
+typedef __u32 key_serial_t;
+
+struct mmsghdr {
+	struct user_msghdr  msg_hdr;
+	unsigned int        msg_len;
+};
+
+struct linux_dirent64 {
+	u64		d_ino;
+	s64		d_off;
+	unsigned short	d_reclen;
+	unsigned char	d_type;
+	char		d_name[0];
+};
+
+struct linux_dirent {
+	unsigned long	d_ino;
+	unsigned long	d_off;
+	unsigned short	d_reclen;
+	char		d_name[1];
+};
+
+struct ustat {
+	__kernel_daddr_t	f_tfree;
+	__kernel_ino_t		f_tinode;
+	char			f_fname[6];
+	char			f_fpack[6];
+};
+
+struct __aio_sigset;
+struct clone_args;
+
+typedef __kernel_rwf_t		rwf_t;
+
+long lkl_syscall(long no, long *params);
+long lkl_sys_halt(void);
+
+#define __MAP0(m, ...)
+#define __MAP1(m, t, a) m(t, a)
+#define __MAP2(m, t, a, ...) m(t, a), __MAP1(m, __VA_ARGS__)
+#define __MAP3(m, t, a, ...) m(t, a), __MAP2(m, __VA_ARGS__)
+#define __MAP4(m, t, a, ...) m(t, a), __MAP3(m, __VA_ARGS__)
+#define __MAP5(m, t, a, ...) m(t, a), __MAP4(m, __VA_ARGS__)
+#define __MAP6(m, t, a, ...) m(t, a), __MAP5(m, __VA_ARGS__)
+#define __MAP(n, ...) __MAP##n(__VA_ARGS__)
+
+#define __SC_LONG(t, a) (long)(a)
+#define __SC_TABLE(t, a) {sizeof(t), (long long)(a)}
+#define __SC_DECL(t, a) t a
+
+#define LKL_SYSCALL0(name)					       \
+	static inline long lkl_sys##name(void)			       \
+	{							       \
+		long params[6];					       \
+		return lkl_syscall(__lkl__NR##name, params);	       \
+	}
+
+#if __BITS_PER_LONG == 32
+#define LKL_SYSCALLx(x, name, ...)					\
+	static inline							\
+	long lkl_sys##name(__MAP(x, __SC_DECL, __VA_ARGS__))		\
+	{								\
+		struct {						\
+			unsigned int size;				\
+			long long value;				\
+		} lkl_params[x] = { __MAP(x, __SC_TABLE, __VA_ARGS__) }; \
+		long sys_params[6], i, k;				\
+		for (i = k = 0; i < x && k < 6; i++, k++) {		\
+			if (lkl_params[i].size > sizeof(long) &&	\
+			    k + 1 < 6) {				\
+				sys_params[k] =				\
+					(long)(lkl_params[i].value & (-1UL)); \
+				k++;					\
+				sys_params[k] =				\
+					(long)(lkl_params[i].value >>	\
+					       __BITS_PER_LONG);	\
+			} else {					\
+				sys_params[k] = (long)(lkl_params[i].value); \
+			}						\
+		}							\
+		return lkl_syscall(__lkl__NR##name, sys_params);	\
+	}
+#else
+#define LKL_SYSCALLx(x, name, ...)					\
+	static inline							\
+	long lkl_sys##name(__MAP(x, __SC_DECL, __VA_ARGS__))		\
+	{								\
+		long lkl_params[6] = { __MAP(x, __SC_LONG, __VA_ARGS__) }; \
+		return lkl_syscall(__lkl__NR##name, lkl_params);	\
+	}
+#endif
+
+#define SYSCALL_DEFINE0(name, ...) LKL_SYSCALL0(name)
+#define SYSCALL_DEFINE1(name, ...) LKL_SYSCALLx(1, name, __VA_ARGS__)
+#define SYSCALL_DEFINE2(name, ...) LKL_SYSCALLx(2, name, __VA_ARGS__)
+#define SYSCALL_DEFINE3(name, ...) LKL_SYSCALLx(3, name, __VA_ARGS__)
+#define SYSCALL_DEFINE4(name, ...) LKL_SYSCALLx(4, name, __VA_ARGS__)
+#define SYSCALL_DEFINE5(name, ...) LKL_SYSCALLx(5, name, __VA_ARGS__)
+#define SYSCALL_DEFINE6(name, ...) LKL_SYSCALLx(6, name, __VA_ARGS__)
+
+#if __BITS_PER_LONG == 32
+#pragma GCC diagnostic push
+#pragma GCC diagnostic ignored "-Wpointer-to-int-cast"
+#endif
+
+#include <asm/syscall_defs.h>
+
+#if __BITS_PER_LONG == 32
+#pragma GCC diagnostic pop
+#endif
+
+#endif
diff --git a/arch/um/nommu/include/uapi/asm/unistd.h b/arch/um/nommu/include/uapi/asm/unistd.h
new file mode 100644
index 000000000000..1762ebb829f5
--- /dev/null
+++ b/arch/um/nommu/include/uapi/asm/unistd.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef __UM_NOMMU_UAPI_UNISTD_H
+#define __UM_NOMMU_UAPI_UNISTD_H
+
+#define __ARCH_WANT_NEW_STAT
+#include <asm/bitsperlong.h>
+
+#if __BITS_PER_LONG == 64
+#define __ARCH_WANT_SYS_NEWFSTATAT
+#endif
+
+#include <asm-generic/unistd.h>
+
+
+#endif
diff --git a/arch/um/nommu/um/syscalls.c b/arch/um/nommu/um/syscalls.c
new file mode 100644
index 000000000000..fd343f855bad
--- /dev/null
+++ b/arch/um/nommu/um/syscalls.c
@@ -0,0 +1,199 @@
+// SPDX-License-Identifier: GPL-2.0
+#ifdef CONFIG_ARCH_HAS_SYSCALL_WRAPPER
+#undef CONFIG_ARCH_HAS_SYSCALL_WRAPPER
+#include <linux/syscalls.h>
+#define CONFIG_ARCH_HAS_SYSCALL_WRAPPER
+#endif
+
+#include <linux/stat.h>
+#include <linux/irq.h>
+#include <linux/sched.h>
+#include <linux/interrupt.h>
+#include <linux/jhash.h>
+#include <linux/slab.h>
+#include <linux/types.h>
+#include <linux/net.h>
+#include <linux/task_work.h>
+
+#include <asm/host_ops.h>
+#include <asm/syscalls.h>
+#include <asm/cpu.h>
+#include <asm/sched.h>
+#include <kern_util.h>
+#include <os.h>
+
+typedef long (*syscall_handler_t)(long arg1, ...);
+
+#undef __SYSCALL
+#define __SYSCALL(nr, sym)[nr] = (syscall_handler_t)sym,
+
+syscall_handler_t syscall_table[__NR_syscalls] = {
+	[0 ... __NR_syscalls - 1] =  (syscall_handler_t)sys_ni_syscall,
+#undef __UM_NOMMU_UAPI_UNISTD_H
+#include <asm/unistd.h>
+
+#if __BITS_PER_LONG == 32
+#include <asm/unistd_32.h>
+#endif
+};
+
+
+static long run_syscall(long no, long *params)
+{
+	long ret;
+
+	if (no < 0 || no >= __NR_syscalls)
+		return -ENOSYS;
+
+	ret = syscall_table[no](params[0], params[1], params[2], params[3],
+				params[4], params[5]);
+
+	task_work_run();
+
+	return ret;
+}
+
+
+#define CLONE_FLAGS (CLONE_VM | CLONE_FS | CLONE_FILES | CLONE_THREAD |	\
+		     CLONE_SIGHAND | SIGCHLD)
+
+static int host_task_id;
+static struct task_struct *host0;
+
+static int new_host_task(struct task_struct **task)
+{
+	pid_t pid;
+
+	switch_to_host_task(host0);
+
+	pid = kernel_thread(host_task_stub, NULL, CLONE_FLAGS);
+	if (pid < 0)
+		return pid;
+
+	rcu_read_lock();
+	*task = find_task_by_pid_ns(pid, &init_pid_ns);
+	rcu_read_unlock();
+
+	host_task_id++;
+
+	snprintf((*task)->comm, sizeof((*task)->comm), "host%d", host_task_id);
+
+	return 0;
+}
+static void exit_task(void)
+{
+	do_exit(0);
+}
+
+static void del_host_task(void *arg)
+{
+	struct task_struct *task = (struct task_struct *)arg;
+	struct thread_info *ti = task_thread_info(task);
+
+	if (lkl_cpu_get() < 0)
+		return;
+
+	switch_to_host_task(task);
+	host_task_id--;
+	set_ti_thread_flag(ti, TIF_SCHED_JB);
+	lkl_ops->jmp_buf_set(&ti->task->thread.arch.sched_jb, exit_task);
+}
+
+static struct lkl_tls_key *task_key;
+
+long lkl_syscall(long no, long *params)
+{
+	struct task_struct *task = host0;
+	long ret;
+
+	ret = lkl_cpu_get();
+	if (ret < 0)
+		return ret;
+
+	if (lkl_ops->tls_get) {
+		task = lkl_ops->tls_get(task_key);
+		if (!task) {
+			ret = new_host_task(&task);
+			if (ret)
+				goto out;
+			lkl_ops->tls_set(task_key, task);
+		}
+	}
+
+	switch_to_host_task(task);
+
+	ret = run_syscall(no, params);
+
+	if (no == __NR_reboot) {
+		thread_sched_jb();
+		return ret;
+	}
+
+out:
+	lkl_cpu_put();
+
+	return ret;
+}
+
+static struct task_struct *idle_host_task;
+
+/* called from idle, don't failed, don't block */
+void wakeup_idle_host_task(void)
+{
+	if (!need_resched() && idle_host_task)
+		wake_up_process(idle_host_task);
+}
+
+static int idle_host_task_loop(void *unused)
+{
+	struct thread_info *ti = task_thread_info(current);
+
+	snprintf(current->comm, sizeof(current->comm), "idle_host_task");
+	set_thread_flag(TIF_HOST_THREAD);
+	idle_host_task = current;
+
+	for (;;) {
+		lkl_cpu_put();
+		lkl_ops->sem_down(ti->task->thread.arch.sched_sem);
+		if (idle_host_task == NULL) {
+			lkl_ops->thread_exit();
+			return 0;
+		}
+		schedule_tail(ti->task->thread.prev_sched);
+	}
+}
+
+int syscalls_init(void)
+{
+	snprintf(current->comm, sizeof(current->comm), "host0");
+	set_thread_flag(TIF_HOST_THREAD);
+	host0 = current;
+
+	if (lkl_ops->tls_alloc) {
+		task_key = lkl_ops->tls_alloc(del_host_task);
+		if (!task_key)
+			return -1;
+	}
+
+	if (kernel_thread(idle_host_task_loop, NULL, CLONE_FLAGS) < 0) {
+		if (lkl_ops->tls_free)
+			lkl_ops->tls_free(task_key);
+		return -1;
+	}
+
+	return 0;
+}
+
+void syscalls_cleanup(void)
+{
+	if (idle_host_task) {
+		struct thread_info *ti = task_thread_info(idle_host_task);
+
+		idle_host_task = NULL;
+		lkl_ops->sem_up(ti->task->thread.arch.sched_sem);
+		lkl_ops->thread_join(ti->task->thread.arch.tid);
+	}
+
+	if (lkl_ops->tls_free)
+		lkl_ops->tls_free(task_key);
+}
diff --git a/arch/um/scripts/headers_install.py b/arch/um/scripts/headers_install.py
new file mode 100755
index 000000000000..ec7356c0dee9
--- /dev/null
+++ b/arch/um/scripts/headers_install.py
@@ -0,0 +1,197 @@
+#!/usr/bin/env python
+# SPDX-License-Identifier: GPL-2.0
+import re, os, sys, argparse, multiprocessing, fnmatch
+
+srctree = os.environ["srctree"]
+objtree = os.environ["objtree"]
+install_hdr_path = os.environ["INSTALL_HDR_PATH"]
+header_paths = [ install_hdr_path + "/include/", "arch/um/nommu/include/uapi/",
+                 "arch/um/nommu/include/generated/uapi/", "include/generated/" ]
+
+headers = set()
+includes = set()
+
+def relpath2abspath(relpath):
+    if "generated" in relpath or install_hdr_path in relpath:
+        return objtree + "/" + relpath
+    else:
+        return srctree + "/" + relpath
+
+def find_headers(path):
+    headers.add(path)
+    f = open(relpath2abspath(path))
+    for l in f.readlines():
+        m = re.search("#\s*include <(.*)>", l)
+        try:
+            i = m.group(1)
+            for p in header_paths:
+                if os.access(relpath2abspath(p + i), os.R_OK):
+                    if p + i not in headers:
+                        includes.add(i)
+                        headers.add(p + i)
+                        find_headers(p + i)
+        except:
+            pass
+    f.close()
+
+def has_lkl_prefix(w):
+  return w.startswith("lkl") or w.startswith("_lkl") or w.startswith("__lkl") \
+         or w.startswith("LKL") or w.startswith("_LKL") or w.startswith("__LKL")
+
+def find_symbols(regexp, store):
+    for h in headers:
+        f = open(h)
+        for l in f.readlines():
+            m = regexp.search(l)
+            if not m:
+                continue
+            for e in reversed(m.groups()):
+                if e:
+                    if not has_lkl_prefix(e):
+                        store.add(e)
+                    break
+        f.close()
+
+def find_ml_symbols(regexp, store):
+    for h in headers:
+        for i in regexp.finditer(open(h).read()):
+            for j in reversed(i.groups()):
+                if j:
+                    if not has_lkl_prefix(j):
+                        store.add(j)
+                    break
+
+def find_enums(block_regexp, symbol_regexp, store):
+    for h in headers:
+        # remove comments
+        content = re.sub(re.compile("(\/\*(\*(?!\/)|[^*])*\*\/)", re.S|re.M), " ", open(h).read())
+        # remove preprocesor lines
+        clean_content = ""
+        for l in content.split("\n"):
+            if re.match("\s*#", l):
+                continue
+            clean_content += l + "\n"
+        for i in block_regexp.finditer(clean_content):
+            for j in reversed(i.groups()):
+                if j:
+                    for k in symbol_regexp.finditer(j):
+                        for l in k.groups():
+                            if l:
+                                if not has_lkl_prefix(l):
+                                    store.add(l)
+                                break
+
+def lkl_prefix(w):
+    r = ""
+
+    if w.startswith("__"):
+        r = "__"
+    elif w.startswith("_"):
+        r = "_"
+
+    if w.isupper():
+        r += "LKL"
+    else:
+        r += "lkl"
+
+    if not w.startswith("_"):
+        r += "_"
+
+    r += w
+
+    return r
+
+def replace(h):
+    content = open(h).read()
+    for i in includes:
+        search_str = "(#[ \t]*include[ \t]*[<\"][ \t]*)" + i + "([ \t]*[>\"])"
+        replace_str = "\\1" + "lkl/" + i + "\\2"
+        content = re.sub(search_str, replace_str, content)
+    tmp = ""
+    for w in re.split("(\W+)", content):
+        if w in defines:
+            w = lkl_prefix(w)
+        tmp += w
+    content = tmp
+    for s in structs:
+        search_str = "(\W?struct\s+)" + s + "(\W)"
+        replace_str = "\\1" + lkl_prefix(s) + "\\2"
+        content = re.sub(search_str, replace_str, content, flags = re.MULTILINE)
+    for s in unions:
+        search_str = "(\W?union\s+)" + s + "(\W)"
+        replace_str = "\\1" + lkl_prefix(s) + "\\2"
+        content = re.sub(search_str, replace_str, content, flags = re.MULTILINE)
+    open(h, 'w').write(content)
+
+parser = argparse.ArgumentParser(description='install lkl headers')
+parser.add_argument('path', help='path to install to', )
+parser.add_argument('-j', '--jobs', help='number of parallel jobs', default=1, type=int)
+args = parser.parse_args()
+
+find_headers("arch/um/nommu/include/uapi/asm/syscalls.h")
+headers.add("arch/um/nommu/include/uapi/asm/host_ops.h")
+
+if 'LKL_INSTALL_ADDITIONAL_HEADERS' in os.environ:
+    with open(os.environ['LKL_INSTALL_ADDITIONAL_HEADERS'], 'rU') as f:
+        for line in f.readlines():
+            line = line.split('#', 1)[0].strip()
+            if line != '':
+                headers.add(line)
+
+new_headers = set()
+
+for h in headers:
+    h = relpath2abspath(h)
+    dir = os.path.dirname(h)
+    out_dir = args.path + "/" + re.sub("(" + srctree + "/arch/um/nommu/include/uapi/|arch/um/nommu/include/generated/uapi/|include/generated/uapi/|include/generated|" + install_hdr_path + "/include/)(.*)", "lkl/\\2", dir)
+    try:
+        os.makedirs(out_dir)
+    except:
+        pass
+    print("  INSTALL\t%s" % (out_dir + "/" + os.path.basename(h)))
+    os.system(srctree+"/scripts/headers_install.sh %s %s" % (os.path.abspath(h),
+                                                       out_dir + "/" + os.path.basename(h)))
+    new_headers.add(out_dir + "/" + os.path.basename(h))
+
+headers = new_headers
+
+defines = set()
+structs = set()
+unions = set()
+
+p = re.compile("#[ \t]*define[ \t]*(\w+)")
+find_symbols(p, defines)
+p = re.compile("typedef.*(\(\*(\w+)\)\(.*\)\s*|\W+(\w+)\s*|\s+(\w+)\(.*\)\s*);")
+find_symbols(p, defines)
+p = re.compile("typedef\s+(struct|union)\s+\w*\s*{[^\\{\}]*}\W*(\w+)\s*;", re.M|re.S)
+find_ml_symbols(p, defines)
+defines.add("siginfo_t")
+defines.add("sigevent_t")
+p = re.compile("struct\s+(\w+)\s*\{")
+find_symbols(p, structs)
+structs.add("iovec")
+p = re.compile("union\s+(\w+)\s*\{")
+find_symbols(p, unions)
+p = re.compile("static\s+__inline__(\s+\w+)+\s+(\w+)\([^)]*\)\s")
+find_symbols(p, defines)
+p = re.compile("static\s+__always_inline(\s+\w+)+\s+(\w+)\([^)]*\)\s")
+find_symbols(p, defines)
+p = re.compile("enum\s+(\w*)\s*{([^}]*)}", re.M|re.S)
+q = re.compile("(\w+)\s*(,|=[^,]*|$)", re.M|re.S)
+find_enums(p, q, defines)
+
+# needed for i386
+defines.add("__NR_stime")
+
+def process_header(h):
+    print("  REPLACE\t%s" % (out_dir + "/" + os.path.basename(h)))
+    replace(h)
+
+p = multiprocessing.Pool(args.jobs)
+try:
+    p.map_async(process_header, headers).wait(999999)
+    p.close()
+except:
+    p.terminate()
+finally:
+    p.join()
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v7 12/21] um: nommu: system call interface and application API
@ 2020-10-06  9:44           ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-06  9:44 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Hajime Tazaki, retrage01

The application API is based on the kernel system call interface in
order to offer a stable API to applications. Note that we can't
offer the full Linux system call interface due to the limitations of
nommu mode such as lack of virtual memory, signal, user processes, etc.

The host is using dedicated kernel thread, which is switched upon the
entry of lkl_syscall so that numbers of context switches can be reduced
for the faster performance.

To expose the API definitions to applications, this commit uses
syscall_wrapper to define arch-specific information of syscall, and this
is used to generate syscall_defs.h for lkl so that it can be used
as a template of the list of syscall available for LKL apps.

To avoid collisions between the Linux API and the LKL API (e.g.  struct
stat, MKNOD, etc.) we use a python script to modify the user headers
and to prefix all of the global symbols (structures, typedefs,
defines) with LKL, lkl, _LKL, _lkl, __LKL or __lkl.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/include/uapi/asm/Kbuild             |   2 +
 arch/um/nommu/include/asm/syscall_wrapper.h |  57 ++++
 arch/um/nommu/include/asm/syscalls.h        |  15 +
 arch/um/nommu/include/uapi/asm/host_ops.h   |  14 +
 arch/um/nommu/include/uapi/asm/syscalls.h   | 287 ++++++++++++++++++++
 arch/um/nommu/include/uapi/asm/unistd.h     |  15 +
 arch/um/nommu/um/syscalls.c                 | 199 ++++++++++++++
 arch/um/scripts/headers_install.py          | 197 ++++++++++++++
 8 files changed, 786 insertions(+)
 create mode 100644 arch/um/include/uapi/asm/Kbuild
 create mode 100644 arch/um/nommu/include/asm/syscall_wrapper.h
 create mode 100644 arch/um/nommu/include/asm/syscalls.h
 create mode 100644 arch/um/nommu/include/uapi/asm/syscalls.h
 create mode 100644 arch/um/nommu/include/uapi/asm/unistd.h
 create mode 100644 arch/um/nommu/um/syscalls.c
 create mode 100755 arch/um/scripts/headers_install.py

diff --git a/arch/um/include/uapi/asm/Kbuild b/arch/um/include/uapi/asm/Kbuild
new file mode 100644
index 000000000000..a983c7f049bc
--- /dev/null
+++ b/arch/um/include/uapi/asm/Kbuild
@@ -0,0 +1,2 @@
+# no generated-y since we need special user headers handling
+# see arch/um/script/headers_install.py
diff --git a/arch/um/nommu/include/asm/syscall_wrapper.h b/arch/um/nommu/include/asm/syscall_wrapper.h
new file mode 100644
index 000000000000..bb2a7d4274f6
--- /dev/null
+++ b/arch/um/nommu/include/asm/syscall_wrapper.h
@@ -0,0 +1,57 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef __UM_SYSCALL_WRAPPER_H
+#define __UM_SYSCALL_WRAPPER_H
+
+#define __SC_ASCII(t, a) #t "," #a
+
+#define __ASCII_MAP0(m, ...)
+#define __ASCII_MAP1(m, t, a) m(t, a)
+#define __ASCII_MAP2(m, t, a, ...) m(t, a) "," __ASCII_MAP1(m, __VA_ARGS__)
+#define __ASCII_MAP3(m, t, a, ...) m(t, a) "," __ASCII_MAP2(m, __VA_ARGS__)
+#define __ASCII_MAP4(m, t, a, ...) m(t, a) "," __ASCII_MAP3(m, __VA_ARGS__)
+#define __ASCII_MAP5(m, t, a, ...) m(t, a) "," __ASCII_MAP4(m, __VA_ARGS__)
+#define __ASCII_MAP6(m, t, a, ...) m(t, a) "," __ASCII_MAP5(m, __VA_ARGS__)
+#define __ASCII_MAP(n, ...) __ASCII_MAP##n(__VA_ARGS__)
+
+#ifdef __MINGW32__
+#define SECTION_ATTRS "n0"
+#else
+#define SECTION_ATTRS "a"
+#endif
+
+#define __SYSCALL_DEFINE_ARCH(x, name, ...)				\
+	asm(".section .syscall_defs,\"" SECTION_ATTRS "\"\n"		\
+	    ".ascii \"#ifdef __NR" #name "\\n\"\n"			\
+	    ".ascii \"SYSCALL_DEFINE" #x "(" #name ","			\
+	    __ASCII_MAP(x, __SC_ASCII, __VA_ARGS__) ")\\n\"\n"		\
+	    ".ascii \"#endif\\n\"\n"					\
+	    ".section .text\n")
+
+#define SYSCALL_DEFINE0(sname)					\
+	SYSCALL_METADATA(_##sname, 0);				\
+	__SYSCALL_DEFINE_ARCH(0, _##sname);			\
+	asmlinkage long sys_##sname(void);			\
+	ALLOW_ERROR_INJECTION(sys_##sname, ERRNO);		\
+	asmlinkage long sys_##sname(void)
+
+#define __SYSCALL_DEFINEx(x, name, ...)					\
+	__SYSCALL_DEFINE_ARCH(x, name, __VA_ARGS__);			\
+	__diag_push();							\
+	__diag_ignore(GCC, 8, "-Wattribute-alias",			\
+		      "Type aliasing is used to sanitize syscall arguments");\
+	asmlinkage long sys##name(__MAP(x, __SC_DECL, __VA_ARGS__))	\
+		__attribute__((alias(__stringify(__se_sys##name))));	\
+	ALLOW_ERROR_INJECTION(sys##name, ERRNO);			\
+	static inline long __do_sys##name(__MAP(x, __SC_DECL, __VA_ARGS__)); \
+	asmlinkage long __se_sys##name(__MAP(x, __SC_LONG, __VA_ARGS__)); \
+	asmlinkage long __se_sys##name(__MAP(x, __SC_LONG, __VA_ARGS__)) \
+	{								\
+		long ret = __do_sys##name(__MAP(x, __SC_CAST, __VA_ARGS__)); \
+		__MAP(x, __SC_TEST, __VA_ARGS__);			\
+		__PROTECT(x, ret, __MAP(x, __SC_ARGS, __VA_ARGS__));	\
+		return ret;						\
+	}								\
+	__diag_pop();							\
+	static inline long __do_sys##name(__MAP(x, __SC_DECL, __VA_ARGS__))
+
+#endif /* __UM_SYSCALL_WRAPPER_H */
diff --git a/arch/um/nommu/include/asm/syscalls.h b/arch/um/nommu/include/asm/syscalls.h
new file mode 100644
index 000000000000..6061d9415dad
--- /dev/null
+++ b/arch/um/nommu/include/asm/syscalls.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+int syscalls_init(void);
+void syscalls_cleanup(void);
+long lkl_syscall(long no, long *params);
+void wakeup_idle_host_task(void);
+
+#define sys_mmap sys_ni_syscall
+#define sys_rt_sigreturn sys_ni_syscall
+#define sys_arch_prctl sys_ni_syscall
+#define sys_iopl sys_ni_syscall
+#define sys_ioperm sys_ni_syscall
+#define sys_clone sys_ni_syscall
+
+int run_syscalls(void);
diff --git a/arch/um/nommu/include/uapi/asm/host_ops.h b/arch/um/nommu/include/uapi/asm/host_ops.h
index 720385fccbdf..6ee9489fc47e 100644
--- a/arch/um/nommu/include/uapi/asm/host_ops.h
+++ b/arch/um/nommu/include/uapi/asm/host_ops.h
@@ -38,6 +38,15 @@ struct lkl_jmp_buf {
  * @gettid - returns the host thread id of the caller, which need not
  * be the same as the handle returned by thread_create
  *
+ * @tls_alloc - allocate a thread local storage key; returns 0 if successful; if
+ * destructor is not NULL it will be called when a thread terminates with its
+ * argument set to the current thread local storage value
+ * @tls_free - frees a thread local storage key; returns 0 if successful
+ * @tls_set - associate data to the thread local storage key; returns 0 if
+ * successful
+ * @tls_get - return data associated with the thread local storage key or NULL
+ * on error
+ *
  * @jmp_buf_set - runs the give function and setups a jump back point by saving
  * the context in the jump buffer; jmp_buf_longjmp can be called from the give
  * function or any callee in that function to return back to the jump back
@@ -72,6 +81,11 @@ struct lkl_host_operations {
 	int (*thread_equal)(lkl_thread_t a, lkl_thread_t b);
 	long (*gettid)(void);
 
+	struct lkl_tls_key *(*tls_alloc)(void (*destructor)(void *));
+	void (*tls_free)(struct lkl_tls_key *key);
+	int (*tls_set)(struct lkl_tls_key *key, void *data);
+	void *(*tls_get)(struct lkl_tls_key *key);
+
 	void (*jmp_buf_set)(struct lkl_jmp_buf *jmpb, void (*f)(void));
 	void (*jmp_buf_longjmp)(struct lkl_jmp_buf *jmpb, int val);
 
diff --git a/arch/um/nommu/include/uapi/asm/syscalls.h b/arch/um/nommu/include/uapi/asm/syscalls.h
new file mode 100644
index 000000000000..751957b978cd
--- /dev/null
+++ b/arch/um/nommu/include/uapi/asm/syscalls.h
@@ -0,0 +1,287 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef __UM_NOMMU_UAPI_SYSCALLS_H
+#define __UM_NOMMU_UAPI_SYSCALLS_H
+
+#include <autoconf.h>
+#include <linux/types.h>
+
+typedef __kernel_uid32_t	qid_t;
+typedef __kernel_fd_set		fd_set;
+typedef __kernel_mode_t		mode_t;
+typedef unsigned short		umode_t;
+typedef __u32			nlink_t;
+typedef __kernel_off_t		off_t;
+typedef __kernel_pid_t		pid_t;
+typedef __kernel_key_t		key_t;
+typedef __kernel_suseconds_t	suseconds_t;
+typedef __kernel_timer_t	timer_t;
+typedef __kernel_clockid_t	clockid_t;
+typedef __kernel_mqd_t		mqd_t;
+typedef __kernel_uid32_t	uid_t;
+typedef __kernel_gid32_t	gid_t;
+typedef __kernel_uid16_t        uid16_t;
+typedef __kernel_gid16_t        gid16_t;
+typedef unsigned long		uintptr_t;
+#ifdef CONFIG_UID16
+typedef __kernel_old_uid_t	old_uid_t;
+typedef __kernel_old_gid_t	old_gid_t;
+#endif
+typedef __kernel_loff_t		loff_t;
+typedef __kernel_size_t		size_t;
+typedef __kernel_ssize_t	ssize_t;
+typedef __kernel_time_t		time_t;
+typedef __kernel_clock_t	clock_t;
+typedef __u32			u32;
+typedef __s32			s32;
+typedef __u64			u64;
+typedef __s64			s64;
+
+typedef __kernel_long_t	__kernel_old_time_t;
+
+#define __user
+
+#include <asm/unistd.h>
+/* Temporary undefine system calls that don't have data types defined in UAPI
+ * headers
+ */
+#undef __NR_kexec_load
+#undef __NR_getcpu
+#undef __NR_sched_getattr
+#undef __NR_sched_setattr
+#undef __NR_sched_setparam
+#undef __NR_sched_getparam
+#undef __NR_sched_setscheduler
+#undef __NR_name_to_handle_at
+#undef __NR_open_by_handle_at
+
+/* deprecated system calls */
+#undef __NR_access
+#undef __NR_chmod
+#undef __NR_chown
+#undef __NR_lchown
+#undef __NR_open
+#undef __NR_creat
+#undef __NR_readlink
+#undef __NR_pipe
+#undef __NR_mknod
+#undef __NR_mkdir
+#undef __NR_rmdir
+#undef __NR_unlink
+#undef __NR_symlink
+#undef __NR_link
+#undef __NR_rename
+#undef __NR_getdents
+#undef __NR_select
+#undef __NR_poll
+#undef __NR_dup2
+#undef __NR_sysfs
+#undef __NR_ustat
+#undef __NR_inotify_init
+#undef __NR_epoll_create
+#undef __NR_epoll_wait
+#undef __NR_signalfd
+#undef __NR_eventfd
+
+#undef __NR_umount
+#define __NR_umount __NR_umount2
+
+#ifdef CONFIG_64BIT
+#define __NR_newfstat __NR3264_fstat
+#define __NR_newfstatat __NR3264_fstatat
+#endif
+
+#include <linux/time.h>
+#include <linux/times.h>
+#include <linux/timex.h>
+#include <linux/capability.h>
+#define __KERNEL__ /* to pull in S_ definitions */
+#include <linux/stat.h>
+#undef __KERNEL__
+#include <linux/errno.h>
+#include <linux/fcntl.h>
+#include <linux/fs.h>
+#include <asm/statfs.h>
+#include <asm/stat.h>
+#include <linux/bpf.h>
+#include <linux/msg.h>
+#include <linux/resource.h>
+#include <linux/sysinfo.h>
+#include <linux/shm.h>
+#include <linux/aio_abi.h>
+#include <linux/socket.h>
+#include <linux/perf_event.h>
+#include <linux/sem.h>
+#include <linux/futex.h>
+#include <linux/poll.h>
+#include <linux/mqueue.h>
+#include <linux/eventpoll.h>
+#include <linux/uio.h>
+#include <asm/signal.h>
+#include <asm/siginfo.h>
+#include <linux/utime.h>
+#include <asm/socket.h>
+#include <linux/icmp.h>
+#include <linux/ip.h>
+
+/* Define data structures used in system calls that are not defined in UAPI
+ * headers
+ */
+struct sockaddr {
+	unsigned short int sa_family;
+	char sa_data[14];
+};
+
+#define __UAPI_DEF_IF_NET_DEVICE_FLAGS_LOWER_UP_DORMANT_ECHO 1
+#define __UAPI_DEF_IF_IFNAMSIZ	1
+#define __UAPI_DEF_IF_NET_DEVICE_FLAGS 1
+#define __UAPI_DEF_IF_IFREQ	1
+#define __UAPI_DEF_IF_IFMAP	1
+#include <linux/if.h>
+#define __UAPI_DEF_IN_IPPROTO	1
+#define __UAPI_DEF_IN_ADDR	1
+#define __UAPI_DEF_IN6_ADDR	1
+#define __UAPI_DEF_IP_MREQ	1
+#define __UAPI_DEF_IN_PKTINFO	1
+#define __UAPI_DEF_SOCKADDR_IN	1
+#define __UAPI_DEF_IN_CLASS	1
+#include <linux/in.h>
+#include <linux/in6.h>
+#include <linux/sockios.h>
+#include <linux/route.h>
+#include <linux/ipv6_route.h>
+#include <linux/ipv6.h>
+#include <linux/netlink.h>
+#include <linux/neighbour.h>
+#include <linux/rtnetlink.h>
+#include <linux/fib_rules.h>
+
+#include <linux/kdev_t.h>
+#include <linux/virtio_blk.h>
+#include <linux/virtio_net.h>
+#include <linux/virtio_ring.h>
+#include <linux/pkt_sched.h>
+#include <linux/io_uring.h>
+#include <linux/utsname.h>
+
+struct user_msghdr {
+	void		__user *msg_name;
+	int		msg_namelen;
+	struct iovec	__user *msg_iov;
+	__kernel_size_t	msg_iovlen;
+	void		__user *msg_control;
+	__kernel_size_t	msg_controllen;
+	unsigned int	msg_flags;
+};
+
+typedef __u32 key_serial_t;
+
+struct mmsghdr {
+	struct user_msghdr  msg_hdr;
+	unsigned int        msg_len;
+};
+
+struct linux_dirent64 {
+	u64		d_ino;
+	s64		d_off;
+	unsigned short	d_reclen;
+	unsigned char	d_type;
+	char		d_name[0];
+};
+
+struct linux_dirent {
+	unsigned long	d_ino;
+	unsigned long	d_off;
+	unsigned short	d_reclen;
+	char		d_name[1];
+};
+
+struct ustat {
+	__kernel_daddr_t	f_tfree;
+	__kernel_ino_t		f_tinode;
+	char			f_fname[6];
+	char			f_fpack[6];
+};
+
+struct __aio_sigset;
+struct clone_args;
+
+typedef __kernel_rwf_t		rwf_t;
+
+long lkl_syscall(long no, long *params);
+long lkl_sys_halt(void);
+
+#define __MAP0(m, ...)
+#define __MAP1(m, t, a) m(t, a)
+#define __MAP2(m, t, a, ...) m(t, a), __MAP1(m, __VA_ARGS__)
+#define __MAP3(m, t, a, ...) m(t, a), __MAP2(m, __VA_ARGS__)
+#define __MAP4(m, t, a, ...) m(t, a), __MAP3(m, __VA_ARGS__)
+#define __MAP5(m, t, a, ...) m(t, a), __MAP4(m, __VA_ARGS__)
+#define __MAP6(m, t, a, ...) m(t, a), __MAP5(m, __VA_ARGS__)
+#define __MAP(n, ...) __MAP##n(__VA_ARGS__)
+
+#define __SC_LONG(t, a) (long)(a)
+#define __SC_TABLE(t, a) {sizeof(t), (long long)(a)}
+#define __SC_DECL(t, a) t a
+
+#define LKL_SYSCALL0(name)					       \
+	static inline long lkl_sys##name(void)			       \
+	{							       \
+		long params[6];					       \
+		return lkl_syscall(__lkl__NR##name, params);	       \
+	}
+
+#if __BITS_PER_LONG == 32
+#define LKL_SYSCALLx(x, name, ...)					\
+	static inline							\
+	long lkl_sys##name(__MAP(x, __SC_DECL, __VA_ARGS__))		\
+	{								\
+		struct {						\
+			unsigned int size;				\
+			long long value;				\
+		} lkl_params[x] = { __MAP(x, __SC_TABLE, __VA_ARGS__) }; \
+		long sys_params[6], i, k;				\
+		for (i = k = 0; i < x && k < 6; i++, k++) {		\
+			if (lkl_params[i].size > sizeof(long) &&	\
+			    k + 1 < 6) {				\
+				sys_params[k] =				\
+					(long)(lkl_params[i].value & (-1UL)); \
+				k++;					\
+				sys_params[k] =				\
+					(long)(lkl_params[i].value >>	\
+					       __BITS_PER_LONG);	\
+			} else {					\
+				sys_params[k] = (long)(lkl_params[i].value); \
+			}						\
+		}							\
+		return lkl_syscall(__lkl__NR##name, sys_params);	\
+	}
+#else
+#define LKL_SYSCALLx(x, name, ...)					\
+	static inline							\
+	long lkl_sys##name(__MAP(x, __SC_DECL, __VA_ARGS__))		\
+	{								\
+		long lkl_params[6] = { __MAP(x, __SC_LONG, __VA_ARGS__) }; \
+		return lkl_syscall(__lkl__NR##name, lkl_params);	\
+	}
+#endif
+
+#define SYSCALL_DEFINE0(name, ...) LKL_SYSCALL0(name)
+#define SYSCALL_DEFINE1(name, ...) LKL_SYSCALLx(1, name, __VA_ARGS__)
+#define SYSCALL_DEFINE2(name, ...) LKL_SYSCALLx(2, name, __VA_ARGS__)
+#define SYSCALL_DEFINE3(name, ...) LKL_SYSCALLx(3, name, __VA_ARGS__)
+#define SYSCALL_DEFINE4(name, ...) LKL_SYSCALLx(4, name, __VA_ARGS__)
+#define SYSCALL_DEFINE5(name, ...) LKL_SYSCALLx(5, name, __VA_ARGS__)
+#define SYSCALL_DEFINE6(name, ...) LKL_SYSCALLx(6, name, __VA_ARGS__)
+
+#if __BITS_PER_LONG == 32
+#pragma GCC diagnostic push
+#pragma GCC diagnostic ignored "-Wpointer-to-int-cast"
+#endif
+
+#include <asm/syscall_defs.h>
+
+#if __BITS_PER_LONG == 32
+#pragma GCC diagnostic pop
+#endif
+
+#endif
diff --git a/arch/um/nommu/include/uapi/asm/unistd.h b/arch/um/nommu/include/uapi/asm/unistd.h
new file mode 100644
index 000000000000..1762ebb829f5
--- /dev/null
+++ b/arch/um/nommu/include/uapi/asm/unistd.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef __UM_NOMMU_UAPI_UNISTD_H
+#define __UM_NOMMU_UAPI_UNISTD_H
+
+#define __ARCH_WANT_NEW_STAT
+#include <asm/bitsperlong.h>
+
+#if __BITS_PER_LONG == 64
+#define __ARCH_WANT_SYS_NEWFSTATAT
+#endif
+
+#include <asm-generic/unistd.h>
+
+
+#endif
diff --git a/arch/um/nommu/um/syscalls.c b/arch/um/nommu/um/syscalls.c
new file mode 100644
index 000000000000..fd343f855bad
--- /dev/null
+++ b/arch/um/nommu/um/syscalls.c
@@ -0,0 +1,199 @@
+// SPDX-License-Identifier: GPL-2.0
+#ifdef CONFIG_ARCH_HAS_SYSCALL_WRAPPER
+#undef CONFIG_ARCH_HAS_SYSCALL_WRAPPER
+#include <linux/syscalls.h>
+#define CONFIG_ARCH_HAS_SYSCALL_WRAPPER
+#endif
+
+#include <linux/stat.h>
+#include <linux/irq.h>
+#include <linux/sched.h>
+#include <linux/interrupt.h>
+#include <linux/jhash.h>
+#include <linux/slab.h>
+#include <linux/types.h>
+#include <linux/net.h>
+#include <linux/task_work.h>
+
+#include <asm/host_ops.h>
+#include <asm/syscalls.h>
+#include <asm/cpu.h>
+#include <asm/sched.h>
+#include <kern_util.h>
+#include <os.h>
+
+typedef long (*syscall_handler_t)(long arg1, ...);
+
+#undef __SYSCALL
+#define __SYSCALL(nr, sym)[nr] = (syscall_handler_t)sym,
+
+syscall_handler_t syscall_table[__NR_syscalls] = {
+	[0 ... __NR_syscalls - 1] =  (syscall_handler_t)sys_ni_syscall,
+#undef __UM_NOMMU_UAPI_UNISTD_H
+#include <asm/unistd.h>
+
+#if __BITS_PER_LONG == 32
+#include <asm/unistd_32.h>
+#endif
+};
+
+
+static long run_syscall(long no, long *params)
+{
+	long ret;
+
+	if (no < 0 || no >= __NR_syscalls)
+		return -ENOSYS;
+
+	ret = syscall_table[no](params[0], params[1], params[2], params[3],
+				params[4], params[5]);
+
+	task_work_run();
+
+	return ret;
+}
+
+
+#define CLONE_FLAGS (CLONE_VM | CLONE_FS | CLONE_FILES | CLONE_THREAD |	\
+		     CLONE_SIGHAND | SIGCHLD)
+
+static int host_task_id;
+static struct task_struct *host0;
+
+static int new_host_task(struct task_struct **task)
+{
+	pid_t pid;
+
+	switch_to_host_task(host0);
+
+	pid = kernel_thread(host_task_stub, NULL, CLONE_FLAGS);
+	if (pid < 0)
+		return pid;
+
+	rcu_read_lock();
+	*task = find_task_by_pid_ns(pid, &init_pid_ns);
+	rcu_read_unlock();
+
+	host_task_id++;
+
+	snprintf((*task)->comm, sizeof((*task)->comm), "host%d", host_task_id);
+
+	return 0;
+}
+static void exit_task(void)
+{
+	do_exit(0);
+}
+
+static void del_host_task(void *arg)
+{
+	struct task_struct *task = (struct task_struct *)arg;
+	struct thread_info *ti = task_thread_info(task);
+
+	if (lkl_cpu_get() < 0)
+		return;
+
+	switch_to_host_task(task);
+	host_task_id--;
+	set_ti_thread_flag(ti, TIF_SCHED_JB);
+	lkl_ops->jmp_buf_set(&ti->task->thread.arch.sched_jb, exit_task);
+}
+
+static struct lkl_tls_key *task_key;
+
+long lkl_syscall(long no, long *params)
+{
+	struct task_struct *task = host0;
+	long ret;
+
+	ret = lkl_cpu_get();
+	if (ret < 0)
+		return ret;
+
+	if (lkl_ops->tls_get) {
+		task = lkl_ops->tls_get(task_key);
+		if (!task) {
+			ret = new_host_task(&task);
+			if (ret)
+				goto out;
+			lkl_ops->tls_set(task_key, task);
+		}
+	}
+
+	switch_to_host_task(task);
+
+	ret = run_syscall(no, params);
+
+	if (no == __NR_reboot) {
+		thread_sched_jb();
+		return ret;
+	}
+
+out:
+	lkl_cpu_put();
+
+	return ret;
+}
+
+static struct task_struct *idle_host_task;
+
+/* called from idle, don't failed, don't block */
+void wakeup_idle_host_task(void)
+{
+	if (!need_resched() && idle_host_task)
+		wake_up_process(idle_host_task);
+}
+
+static int idle_host_task_loop(void *unused)
+{
+	struct thread_info *ti = task_thread_info(current);
+
+	snprintf(current->comm, sizeof(current->comm), "idle_host_task");
+	set_thread_flag(TIF_HOST_THREAD);
+	idle_host_task = current;
+
+	for (;;) {
+		lkl_cpu_put();
+		lkl_ops->sem_down(ti->task->thread.arch.sched_sem);
+		if (idle_host_task == NULL) {
+			lkl_ops->thread_exit();
+			return 0;
+		}
+		schedule_tail(ti->task->thread.prev_sched);
+	}
+}
+
+int syscalls_init(void)
+{
+	snprintf(current->comm, sizeof(current->comm), "host0");
+	set_thread_flag(TIF_HOST_THREAD);
+	host0 = current;
+
+	if (lkl_ops->tls_alloc) {
+		task_key = lkl_ops->tls_alloc(del_host_task);
+		if (!task_key)
+			return -1;
+	}
+
+	if (kernel_thread(idle_host_task_loop, NULL, CLONE_FLAGS) < 0) {
+		if (lkl_ops->tls_free)
+			lkl_ops->tls_free(task_key);
+		return -1;
+	}
+
+	return 0;
+}
+
+void syscalls_cleanup(void)
+{
+	if (idle_host_task) {
+		struct thread_info *ti = task_thread_info(idle_host_task);
+
+		idle_host_task = NULL;
+		lkl_ops->sem_up(ti->task->thread.arch.sched_sem);
+		lkl_ops->thread_join(ti->task->thread.arch.tid);
+	}
+
+	if (lkl_ops->tls_free)
+		lkl_ops->tls_free(task_key);
+}
diff --git a/arch/um/scripts/headers_install.py b/arch/um/scripts/headers_install.py
new file mode 100755
index 000000000000..ec7356c0dee9
--- /dev/null
+++ b/arch/um/scripts/headers_install.py
@@ -0,0 +1,197 @@
+#!/usr/bin/env python
+# SPDX-License-Identifier: GPL-2.0
+import re, os, sys, argparse, multiprocessing, fnmatch
+
+srctree = os.environ["srctree"]
+objtree = os.environ["objtree"]
+install_hdr_path = os.environ["INSTALL_HDR_PATH"]
+header_paths = [ install_hdr_path + "/include/", "arch/um/nommu/include/uapi/",
+                 "arch/um/nommu/include/generated/uapi/", "include/generated/" ]
+
+headers = set()
+includes = set()
+
+def relpath2abspath(relpath):
+    if "generated" in relpath or install_hdr_path in relpath:
+        return objtree + "/" + relpath
+    else:
+        return srctree + "/" + relpath
+
+def find_headers(path):
+    headers.add(path)
+    f = open(relpath2abspath(path))
+    for l in f.readlines():
+        m = re.search("#\s*include <(.*)>", l)
+        try:
+            i = m.group(1)
+            for p in header_paths:
+                if os.access(relpath2abspath(p + i), os.R_OK):
+                    if p + i not in headers:
+                        includes.add(i)
+                        headers.add(p + i)
+                        find_headers(p + i)
+        except:
+            pass
+    f.close()
+
+def has_lkl_prefix(w):
+  return w.startswith("lkl") or w.startswith("_lkl") or w.startswith("__lkl") \
+         or w.startswith("LKL") or w.startswith("_LKL") or w.startswith("__LKL")
+
+def find_symbols(regexp, store):
+    for h in headers:
+        f = open(h)
+        for l in f.readlines():
+            m = regexp.search(l)
+            if not m:
+                continue
+            for e in reversed(m.groups()):
+                if e:
+                    if not has_lkl_prefix(e):
+                        store.add(e)
+                    break
+        f.close()
+
+def find_ml_symbols(regexp, store):
+    for h in headers:
+        for i in regexp.finditer(open(h).read()):
+            for j in reversed(i.groups()):
+                if j:
+                    if not has_lkl_prefix(j):
+                        store.add(j)
+                    break
+
+def find_enums(block_regexp, symbol_regexp, store):
+    for h in headers:
+        # remove comments
+        content = re.sub(re.compile("(\/\*(\*(?!\/)|[^*])*\*\/)", re.S|re.M), " ", open(h).read())
+        # remove preprocesor lines
+        clean_content = ""
+        for l in content.split("\n"):
+            if re.match("\s*#", l):
+                continue
+            clean_content += l + "\n"
+        for i in block_regexp.finditer(clean_content):
+            for j in reversed(i.groups()):
+                if j:
+                    for k in symbol_regexp.finditer(j):
+                        for l in k.groups():
+                            if l:
+                                if not has_lkl_prefix(l):
+                                    store.add(l)
+                                break
+
+def lkl_prefix(w):
+    r = ""
+
+    if w.startswith("__"):
+        r = "__"
+    elif w.startswith("_"):
+        r = "_"
+
+    if w.isupper():
+        r += "LKL"
+    else:
+        r += "lkl"
+
+    if not w.startswith("_"):
+        r += "_"
+
+    r += w
+
+    return r
+
+def replace(h):
+    content = open(h).read()
+    for i in includes:
+        search_str = "(#[ \t]*include[ \t]*[<\"][ \t]*)" + i + "([ \t]*[>\"])"
+        replace_str = "\\1" + "lkl/" + i + "\\2"
+        content = re.sub(search_str, replace_str, content)
+    tmp = ""
+    for w in re.split("(\W+)", content):
+        if w in defines:
+            w = lkl_prefix(w)
+        tmp += w
+    content = tmp
+    for s in structs:
+        search_str = "(\W?struct\s+)" + s + "(\W)"
+        replace_str = "\\1" + lkl_prefix(s) + "\\2"
+        content = re.sub(search_str, replace_str, content, flags = re.MULTILINE)
+    for s in unions:
+        search_str = "(\W?union\s+)" + s + "(\W)"
+        replace_str = "\\1" + lkl_prefix(s) + "\\2"
+        content = re.sub(search_str, replace_str, content, flags = re.MULTILINE)
+    open(h, 'w').write(content)
+
+parser = argparse.ArgumentParser(description='install lkl headers')
+parser.add_argument('path', help='path to install to', )
+parser.add_argument('-j', '--jobs', help='number of parallel jobs', default=1, type=int)
+args = parser.parse_args()
+
+find_headers("arch/um/nommu/include/uapi/asm/syscalls.h")
+headers.add("arch/um/nommu/include/uapi/asm/host_ops.h")
+
+if 'LKL_INSTALL_ADDITIONAL_HEADERS' in os.environ:
+    with open(os.environ['LKL_INSTALL_ADDITIONAL_HEADERS'], 'rU') as f:
+        for line in f.readlines():
+            line = line.split('#', 1)[0].strip()
+            if line != '':
+                headers.add(line)
+
+new_headers = set()
+
+for h in headers:
+    h = relpath2abspath(h)
+    dir = os.path.dirname(h)
+    out_dir = args.path + "/" + re.sub("(" + srctree + "/arch/um/nommu/include/uapi/|arch/um/nommu/include/generated/uapi/|include/generated/uapi/|include/generated|" + install_hdr_path + "/include/)(.*)", "lkl/\\2", dir)
+    try:
+        os.makedirs(out_dir)
+    except:
+        pass
+    print("  INSTALL\t%s" % (out_dir + "/" + os.path.basename(h)))
+    os.system(srctree+"/scripts/headers_install.sh %s %s" % (os.path.abspath(h),
+                                                       out_dir + "/" + os.path.basename(h)))
+    new_headers.add(out_dir + "/" + os.path.basename(h))
+
+headers = new_headers
+
+defines = set()
+structs = set()
+unions = set()
+
+p = re.compile("#[ \t]*define[ \t]*(\w+)")
+find_symbols(p, defines)
+p = re.compile("typedef.*(\(\*(\w+)\)\(.*\)\s*|\W+(\w+)\s*|\s+(\w+)\(.*\)\s*);")
+find_symbols(p, defines)
+p = re.compile("typedef\s+(struct|union)\s+\w*\s*{[^\\{\}]*}\W*(\w+)\s*;", re.M|re.S)
+find_ml_symbols(p, defines)
+defines.add("siginfo_t")
+defines.add("sigevent_t")
+p = re.compile("struct\s+(\w+)\s*\{")
+find_symbols(p, structs)
+structs.add("iovec")
+p = re.compile("union\s+(\w+)\s*\{")
+find_symbols(p, unions)
+p = re.compile("static\s+__inline__(\s+\w+)+\s+(\w+)\([^)]*\)\s")
+find_symbols(p, defines)
+p = re.compile("static\s+__always_inline(\s+\w+)+\s+(\w+)\([^)]*\)\s")
+find_symbols(p, defines)
+p = re.compile("enum\s+(\w*)\s*{([^}]*)}", re.M|re.S)
+q = re.compile("(\w+)\s*(,|=[^,]*|$)", re.M|re.S)
+find_enums(p, q, defines)
+
+# needed for i386
+defines.add("__NR_stime")
+
+def process_header(h):
+    print("  REPLACE\t%s" % (out_dir + "/" + os.path.basename(h)))
+    replace(h)
+
+p = multiprocessing.Pool(args.jobs)
+try:
+    p.map_async(process_header, headers).wait(999999)
+    p.close()
+except:
+    p.terminate()
+finally:
+    p.join()
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v7 13/21] um: nommu: basic console support
  2020-10-06  9:44         ` Hajime Tazaki
@ 2020-10-06  9:44           ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-06  9:44 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch, Hajime Tazaki

This commit also disables stdio-console of UM if !CONFIG_MMU to avoid
conflicts between multiple consoles.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/drivers/Makefile                  |  8 ++++-
 arch/um/nommu/include/uapi/asm/host_ops.h |  3 ++
 arch/um/nommu/um/console.c                | 42 +++++++++++++++++++++++
 3 files changed, 52 insertions(+), 1 deletion(-)
 create mode 100644 arch/um/nommu/um/console.c

diff --git a/arch/um/drivers/Makefile b/arch/um/drivers/Makefile
index ae96c83e312d..694da4aa550d 100644
--- a/arch/um/drivers/Makefile
+++ b/arch/um/drivers/Makefile
@@ -37,7 +37,13 @@ $(obj)/vde.o: $(obj)/vde_kern.o $(obj)/vde_user.o
 # When the above is fixed, don't forget to add this too!
 #targets += $(obj)/pcap.o
 
-obj-y := stdio_console.o fd.o chan_kern.o chan_user.o line.o
+ifdef CONFIG_MMU
+obj-y := stdio_console.o
+else
+obj-y :=
+endif
+obj-y += fd.o chan_kern.o chan_user.o line.o
+
 obj-$(CONFIG_SSL) += ssl.o
 obj-$(CONFIG_STDERR_CONSOLE) += stderr_console.o
 
diff --git a/arch/um/nommu/include/uapi/asm/host_ops.h b/arch/um/nommu/include/uapi/asm/host_ops.h
index 6ee9489fc47e..133877c857a1 100644
--- a/arch/um/nommu/include/uapi/asm/host_ops.h
+++ b/arch/um/nommu/include/uapi/asm/host_ops.h
@@ -15,6 +15,7 @@ struct lkl_jmp_buf {
  *
  * These operations must be provided by a host library or by the application
  * itself.
+ * @print - optional operation that receives console messages
  *
  * @sem_alloc - allocate a host semaphore an initialize it to count
  * @sem_free - free a host semaphore
@@ -63,6 +64,8 @@ struct lkl_jmp_buf {
  *
  */
 struct lkl_host_operations {
+	void (*print)(const char *str, int len);
+
 	struct lkl_sem *(*sem_alloc)(int count);
 	void (*sem_free)(struct lkl_sem *sem);
 	void (*sem_up)(struct lkl_sem *sem);
diff --git a/arch/um/nommu/um/console.c b/arch/um/nommu/um/console.c
new file mode 100644
index 000000000000..e3194f7bb0dd
--- /dev/null
+++ b/arch/um/nommu/um/console.c
@@ -0,0 +1,42 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/console.h>
+#include <asm/host_ops.h>
+
+static void console_write(struct console *con, const char *str, unsigned len)
+{
+	if (lkl_ops->print)
+		lkl_ops->print(str, len);
+}
+
+#ifdef CONFIG_LKL_EARLY_CONSOLE
+static struct console lkl_boot_console = {
+	.name	= "lkl_boot_console",
+	.write	= console_write,
+	.flags	= CON_PRINTBUFFER | CON_BOOT,
+	.index	= -1,
+};
+
+int __init lkl_boot_console_init(void)
+{
+	register_console(&lkl_boot_console);
+	return 0;
+}
+early_initcall(lkl_boot_console_init);
+#endif
+
+static struct console lkl_console = {
+	.name	= "lkl_console",
+	.write	= console_write,
+	.flags	= CON_PRINTBUFFER,
+	.index	= -1,
+};
+
+int __init lkl_console_init(void)
+{
+	register_console(&lkl_console);
+	return 0;
+}
+core_initcall(lkl_console_init);
+
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v7 13/21] um: nommu: basic console support
@ 2020-10-06  9:44           ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-06  9:44 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Hajime Tazaki, retrage01

This commit also disables stdio-console of UM if !CONFIG_MMU to avoid
conflicts between multiple consoles.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/drivers/Makefile                  |  8 ++++-
 arch/um/nommu/include/uapi/asm/host_ops.h |  3 ++
 arch/um/nommu/um/console.c                | 42 +++++++++++++++++++++++
 3 files changed, 52 insertions(+), 1 deletion(-)
 create mode 100644 arch/um/nommu/um/console.c

diff --git a/arch/um/drivers/Makefile b/arch/um/drivers/Makefile
index ae96c83e312d..694da4aa550d 100644
--- a/arch/um/drivers/Makefile
+++ b/arch/um/drivers/Makefile
@@ -37,7 +37,13 @@ $(obj)/vde.o: $(obj)/vde_kern.o $(obj)/vde_user.o
 # When the above is fixed, don't forget to add this too!
 #targets += $(obj)/pcap.o
 
-obj-y := stdio_console.o fd.o chan_kern.o chan_user.o line.o
+ifdef CONFIG_MMU
+obj-y := stdio_console.o
+else
+obj-y :=
+endif
+obj-y += fd.o chan_kern.o chan_user.o line.o
+
 obj-$(CONFIG_SSL) += ssl.o
 obj-$(CONFIG_STDERR_CONSOLE) += stderr_console.o
 
diff --git a/arch/um/nommu/include/uapi/asm/host_ops.h b/arch/um/nommu/include/uapi/asm/host_ops.h
index 6ee9489fc47e..133877c857a1 100644
--- a/arch/um/nommu/include/uapi/asm/host_ops.h
+++ b/arch/um/nommu/include/uapi/asm/host_ops.h
@@ -15,6 +15,7 @@ struct lkl_jmp_buf {
  *
  * These operations must be provided by a host library or by the application
  * itself.
+ * @print - optional operation that receives console messages
  *
  * @sem_alloc - allocate a host semaphore an initialize it to count
  * @sem_free - free a host semaphore
@@ -63,6 +64,8 @@ struct lkl_jmp_buf {
  *
  */
 struct lkl_host_operations {
+	void (*print)(const char *str, int len);
+
 	struct lkl_sem *(*sem_alloc)(int count);
 	void (*sem_free)(struct lkl_sem *sem);
 	void (*sem_up)(struct lkl_sem *sem);
diff --git a/arch/um/nommu/um/console.c b/arch/um/nommu/um/console.c
new file mode 100644
index 000000000000..e3194f7bb0dd
--- /dev/null
+++ b/arch/um/nommu/um/console.c
@@ -0,0 +1,42 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/console.h>
+#include <asm/host_ops.h>
+
+static void console_write(struct console *con, const char *str, unsigned len)
+{
+	if (lkl_ops->print)
+		lkl_ops->print(str, len);
+}
+
+#ifdef CONFIG_LKL_EARLY_CONSOLE
+static struct console lkl_boot_console = {
+	.name	= "lkl_boot_console",
+	.write	= console_write,
+	.flags	= CON_PRINTBUFFER | CON_BOOT,
+	.index	= -1,
+};
+
+int __init lkl_boot_console_init(void)
+{
+	register_console(&lkl_boot_console);
+	return 0;
+}
+early_initcall(lkl_boot_console_init);
+#endif
+
+static struct console lkl_console = {
+	.name	= "lkl_console",
+	.write	= console_write,
+	.flags	= CON_PRINTBUFFER,
+	.index	= -1,
+};
+
+int __init lkl_console_init(void)
+{
+	register_console(&lkl_console);
+	return 0;
+}
+core_initcall(lkl_console_init);
+
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v7 14/21] um: nommu: initialization and cleanup
  2020-10-06  9:44         ` Hajime Tazaki
@ 2020-10-06  9:44           ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-06  9:44 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch, Hajime Tazaki

Add the lkl_start_kernel and lkl_sys_halt APIs that start and
respectively stops the Linux kernel.

lkl_start_kernel creates a separate threads that will run the initial
and idle kernel thread. It waits for the kernel to complete
initialization before returning, to avoid races with system calls
issues by the host application.

During the setup phase, we create "/init" in initial ramfs root
filesystem to avoid mounting the "real" rootfs since ramfs is good
enough for now.

lkl_sys_halt will shutdown the kernel, terminate all threads and
free all host resources used by the kernel before returning.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/nommu/include/uapi/asm/host_ops.h |  14 ++
 arch/um/nommu/um/setup.c                  | 162 ++++++++++++++++++++++
 tools/um/uml/process.c                    |   2 +
 3 files changed, 178 insertions(+)
 create mode 100644 arch/um/nommu/um/setup.c

diff --git a/arch/um/nommu/include/uapi/asm/host_ops.h b/arch/um/nommu/include/uapi/asm/host_ops.h
index 133877c857a1..0811494c080c 100644
--- a/arch/um/nommu/include/uapi/asm/host_ops.h
+++ b/arch/um/nommu/include/uapi/asm/host_ops.h
@@ -16,6 +16,7 @@ struct lkl_jmp_buf {
  * These operations must be provided by a host library or by the application
  * itself.
  * @print - optional operation that receives console messages
+ * @panic - called during a kernel panic
  *
  * @sem_alloc - allocate a host semaphore an initialize it to count
  * @sem_free - free a host semaphore
@@ -65,6 +66,7 @@ struct lkl_jmp_buf {
  */
 struct lkl_host_operations {
 	void (*print)(const char *str, int len);
+	void (*panic)(void);
 
 	struct lkl_sem *(*sem_alloc)(int count);
 	void (*sem_free)(struct lkl_sem *sem);
@@ -96,6 +98,18 @@ struct lkl_host_operations {
 	void (*mem_free)(void *mem);
 };
 
+/**
+ * lkl_start_kernel - registers the host operations and starts the kernel
+ *
+ * The function returns only after the kernel is shutdown with lkl_sys_halt.
+ *
+ * @lkl_ops - pointer to host operations
+ * @cmd_line - format for command line string that is going to be used to
+ * generate the Linux kernel command line
+ */
+int lkl_start_kernel(struct lkl_host_operations *ops,
+		     const char *fmt, ...);
+
 void lkl_bug(const char *fmt, ...);
 
 #endif
diff --git a/arch/um/nommu/um/setup.c b/arch/um/nommu/um/setup.c
new file mode 100644
index 000000000000..eaa74d771f73
--- /dev/null
+++ b/arch/um/nommu/um/setup.c
@@ -0,0 +1,162 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/binfmts.h>
+#include <linux/init.h>
+#include <linux/init_task.h>
+#include <linux/personality.h>
+#include <linux/reboot.h>
+#include <linux/fs.h>
+#include <linux/start_kernel.h>
+#include <linux/syscalls.h>
+#include <linux/tick.h>
+#include <linux/kernel.h>
+#include <linux/console.h>
+#include <asm/host_ops.h>
+#include <asm/irq.h>
+#include <asm/unistd.h>
+#include <asm/syscalls.h>
+#include <asm/cpu.h>
+#include <os.h>
+#include <as-layout.h>
+
+
+static void *init_sem;
+static int is_running;
+
+
+struct lkl_host_operations *lkl_ops;
+
+long lkl_panic_blink(int state)
+{
+	lkl_ops->panic();
+	return 0;
+}
+
+static void __init *lkl_run_kernel(void *arg)
+{
+
+	panic_blink = lkl_panic_blink;
+
+	threads_init();
+	lkl_cpu_get();
+	start_kernel();
+
+	return NULL;
+}
+
+int __init lkl_start_kernel(struct lkl_host_operations *ops,
+			    const char *fmt, ...)
+{
+	int ret;
+
+	lkl_ops = ops;
+
+	init_sem = lkl_ops->sem_alloc(0);
+	if (!init_sem)
+		return -ENOMEM;
+
+	ret = lkl_cpu_init();
+	if (ret)
+		goto out_free_init_sem;
+
+	ret = lkl_ops->thread_create(lkl_run_kernel, NULL);
+	if (!ret) {
+		ret = -ENOMEM;
+		goto out_free_init_sem;
+	}
+
+	lkl_ops->sem_down(init_sem);
+	lkl_ops->sem_free(init_sem);
+	current_thread_info()->task->thread.arch.tid = lkl_ops->thread_self();
+	lkl_cpu_change_owner(current_thread_info()->task->thread.arch.tid);
+
+	lkl_cpu_put();
+	is_running = 1;
+
+	return 0;
+
+out_free_init_sem:
+	lkl_ops->sem_free(init_sem);
+
+	return ret;
+}
+
+int lkl_is_running(void)
+{
+	return is_running;
+}
+
+
+long lkl_sys_halt(void)
+{
+	long err;
+	long params[6] = {LINUX_REBOOT_MAGIC1,
+		LINUX_REBOOT_MAGIC2, LINUX_REBOOT_CMD_RESTART, };
+
+	err = lkl_syscall(__NR_reboot, params);
+	if (err < 0)
+		return err;
+
+	is_running = false;
+
+	lkl_cpu_wait_shutdown();
+
+	syscalls_cleanup();
+	threads_cleanup();
+	/* Shutdown the clockevents source. */
+	tick_suspend_local();
+	free_mem();
+	lkl_ops->thread_join(current_thread_info()->task->thread.arch.tid);
+
+	return 0;
+}
+
+
+static int lkl_run_init(struct linux_binprm *bprm);
+
+static struct linux_binfmt lkl_run_init_binfmt = {
+	.module		= THIS_MODULE,
+	.load_binary	= lkl_run_init,
+};
+
+static int lkl_run_init(struct linux_binprm *bprm)
+{
+	int ret;
+
+	if (strcmp("/init", bprm->filename) != 0)
+		return -EINVAL;
+
+	ret = begin_new_exec(bprm);
+	if (ret)
+		return ret;
+	set_personality(PER_LINUX);
+	setup_new_exec(bprm);
+
+	set_binfmt(&lkl_run_init_binfmt);
+
+	init_pid_ns.child_reaper = 0;
+
+	syscalls_init();
+
+	lkl_ops->sem_up(init_sem);
+	lkl_ops->thread_exit();
+
+	return 0;
+}
+
+
+/* skip mounting the "real" rootfs. ramfs is good enough. */
+static int __init fs_setup(void)
+{
+	int fd, flags;
+
+	if (force_o_largefile())
+		flags |= O_LARGEFILE;
+	fd = do_sys_open(AT_FDCWD, "/init", O_CREAT | flags, 0700);
+	WARN_ON(fd < 0);
+	ksys_close(fd);
+
+	register_binfmt(&lkl_run_init_binfmt);
+
+	return 0;
+}
+late_initcall(fs_setup);
diff --git a/tools/um/uml/process.c b/tools/um/uml/process.c
index e52dd37ddadc..c39d914e80ac 100644
--- a/tools/um/uml/process.c
+++ b/tools/um/uml/process.c
@@ -114,6 +114,8 @@ void os_kill_process(int pid, int reap_child)
 
 void os_kill_ptraced_process(int pid, int reap_child)
 {
+	if (pid == 0)
+		return;
 	kill(pid, SIGKILL);
 	ptrace(PTRACE_KILL, pid);
 	ptrace(PTRACE_CONT, pid);
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v7 14/21] um: nommu: initialization and cleanup
@ 2020-10-06  9:44           ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-06  9:44 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Hajime Tazaki, retrage01

Add the lkl_start_kernel and lkl_sys_halt APIs that start and
respectively stops the Linux kernel.

lkl_start_kernel creates a separate threads that will run the initial
and idle kernel thread. It waits for the kernel to complete
initialization before returning, to avoid races with system calls
issues by the host application.

During the setup phase, we create "/init" in initial ramfs root
filesystem to avoid mounting the "real" rootfs since ramfs is good
enough for now.

lkl_sys_halt will shutdown the kernel, terminate all threads and
free all host resources used by the kernel before returning.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/nommu/include/uapi/asm/host_ops.h |  14 ++
 arch/um/nommu/um/setup.c                  | 162 ++++++++++++++++++++++
 tools/um/uml/process.c                    |   2 +
 3 files changed, 178 insertions(+)
 create mode 100644 arch/um/nommu/um/setup.c

diff --git a/arch/um/nommu/include/uapi/asm/host_ops.h b/arch/um/nommu/include/uapi/asm/host_ops.h
index 133877c857a1..0811494c080c 100644
--- a/arch/um/nommu/include/uapi/asm/host_ops.h
+++ b/arch/um/nommu/include/uapi/asm/host_ops.h
@@ -16,6 +16,7 @@ struct lkl_jmp_buf {
  * These operations must be provided by a host library or by the application
  * itself.
  * @print - optional operation that receives console messages
+ * @panic - called during a kernel panic
  *
  * @sem_alloc - allocate a host semaphore an initialize it to count
  * @sem_free - free a host semaphore
@@ -65,6 +66,7 @@ struct lkl_jmp_buf {
  */
 struct lkl_host_operations {
 	void (*print)(const char *str, int len);
+	void (*panic)(void);
 
 	struct lkl_sem *(*sem_alloc)(int count);
 	void (*sem_free)(struct lkl_sem *sem);
@@ -96,6 +98,18 @@ struct lkl_host_operations {
 	void (*mem_free)(void *mem);
 };
 
+/**
+ * lkl_start_kernel - registers the host operations and starts the kernel
+ *
+ * The function returns only after the kernel is shutdown with lkl_sys_halt.
+ *
+ * @lkl_ops - pointer to host operations
+ * @cmd_line - format for command line string that is going to be used to
+ * generate the Linux kernel command line
+ */
+int lkl_start_kernel(struct lkl_host_operations *ops,
+		     const char *fmt, ...);
+
 void lkl_bug(const char *fmt, ...);
 
 #endif
diff --git a/arch/um/nommu/um/setup.c b/arch/um/nommu/um/setup.c
new file mode 100644
index 000000000000..eaa74d771f73
--- /dev/null
+++ b/arch/um/nommu/um/setup.c
@@ -0,0 +1,162 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/binfmts.h>
+#include <linux/init.h>
+#include <linux/init_task.h>
+#include <linux/personality.h>
+#include <linux/reboot.h>
+#include <linux/fs.h>
+#include <linux/start_kernel.h>
+#include <linux/syscalls.h>
+#include <linux/tick.h>
+#include <linux/kernel.h>
+#include <linux/console.h>
+#include <asm/host_ops.h>
+#include <asm/irq.h>
+#include <asm/unistd.h>
+#include <asm/syscalls.h>
+#include <asm/cpu.h>
+#include <os.h>
+#include <as-layout.h>
+
+
+static void *init_sem;
+static int is_running;
+
+
+struct lkl_host_operations *lkl_ops;
+
+long lkl_panic_blink(int state)
+{
+	lkl_ops->panic();
+	return 0;
+}
+
+static void __init *lkl_run_kernel(void *arg)
+{
+
+	panic_blink = lkl_panic_blink;
+
+	threads_init();
+	lkl_cpu_get();
+	start_kernel();
+
+	return NULL;
+}
+
+int __init lkl_start_kernel(struct lkl_host_operations *ops,
+			    const char *fmt, ...)
+{
+	int ret;
+
+	lkl_ops = ops;
+
+	init_sem = lkl_ops->sem_alloc(0);
+	if (!init_sem)
+		return -ENOMEM;
+
+	ret = lkl_cpu_init();
+	if (ret)
+		goto out_free_init_sem;
+
+	ret = lkl_ops->thread_create(lkl_run_kernel, NULL);
+	if (!ret) {
+		ret = -ENOMEM;
+		goto out_free_init_sem;
+	}
+
+	lkl_ops->sem_down(init_sem);
+	lkl_ops->sem_free(init_sem);
+	current_thread_info()->task->thread.arch.tid = lkl_ops->thread_self();
+	lkl_cpu_change_owner(current_thread_info()->task->thread.arch.tid);
+
+	lkl_cpu_put();
+	is_running = 1;
+
+	return 0;
+
+out_free_init_sem:
+	lkl_ops->sem_free(init_sem);
+
+	return ret;
+}
+
+int lkl_is_running(void)
+{
+	return is_running;
+}
+
+
+long lkl_sys_halt(void)
+{
+	long err;
+	long params[6] = {LINUX_REBOOT_MAGIC1,
+		LINUX_REBOOT_MAGIC2, LINUX_REBOOT_CMD_RESTART, };
+
+	err = lkl_syscall(__NR_reboot, params);
+	if (err < 0)
+		return err;
+
+	is_running = false;
+
+	lkl_cpu_wait_shutdown();
+
+	syscalls_cleanup();
+	threads_cleanup();
+	/* Shutdown the clockevents source. */
+	tick_suspend_local();
+	free_mem();
+	lkl_ops->thread_join(current_thread_info()->task->thread.arch.tid);
+
+	return 0;
+}
+
+
+static int lkl_run_init(struct linux_binprm *bprm);
+
+static struct linux_binfmt lkl_run_init_binfmt = {
+	.module		= THIS_MODULE,
+	.load_binary	= lkl_run_init,
+};
+
+static int lkl_run_init(struct linux_binprm *bprm)
+{
+	int ret;
+
+	if (strcmp("/init", bprm->filename) != 0)
+		return -EINVAL;
+
+	ret = begin_new_exec(bprm);
+	if (ret)
+		return ret;
+	set_personality(PER_LINUX);
+	setup_new_exec(bprm);
+
+	set_binfmt(&lkl_run_init_binfmt);
+
+	init_pid_ns.child_reaper = 0;
+
+	syscalls_init();
+
+	lkl_ops->sem_up(init_sem);
+	lkl_ops->thread_exit();
+
+	return 0;
+}
+
+
+/* skip mounting the "real" rootfs. ramfs is good enough. */
+static int __init fs_setup(void)
+{
+	int fd, flags;
+
+	if (force_o_largefile())
+		flags |= O_LARGEFILE;
+	fd = do_sys_open(AT_FDCWD, "/init", O_CREAT | flags, 0700);
+	WARN_ON(fd < 0);
+	ksys_close(fd);
+
+	register_binfmt(&lkl_run_init_binfmt);
+
+	return 0;
+}
+late_initcall(fs_setup);
diff --git a/tools/um/uml/process.c b/tools/um/uml/process.c
index e52dd37ddadc..c39d914e80ac 100644
--- a/tools/um/uml/process.c
+++ b/tools/um/uml/process.c
@@ -114,6 +114,8 @@ void os_kill_process(int pid, int reap_child)
 
 void os_kill_ptraced_process(int pid, int reap_child)
 {
+	if (pid == 0)
+		return;
 	kill(pid, SIGKILL);
 	ptrace(PTRACE_KILL, pid);
 	ptrace(PTRACE_CONT, pid);
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v7 15/21] um: nommu: integrate with irq infrastructure of UML
  2020-10-06  9:44         ` Hajime Tazaki
@ 2020-10-06  9:44           ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-06  9:44 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch, Hajime Tazaki

In order to cooperate with UML's irq infrastructure and nommu threads
based on host threads, irq handlers shall synchronize the our
scheduler which is controlled by struct lkl_cpu.  To do that, the irq
infra notifies its entry of handlers by obtaining cpu access of thread
scheduler (lkl_cpu_try_run_irq) and its release (lkl_cpu_put).

In additon to that, in order to stick the signal handler's thread to the
one of the idle thread, several required configurations of (thread's)
signal mask are added: otherwise handlers running on arbitrary thread
cannot obtain cpu access and immediately fall into pending interrupt
which may slow down the delivery of signals.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/include/shared/os.h     |  1 +
 arch/um/kernel/irq.c            | 13 +++++++++++++
 arch/um/kernel/time.c           |  2 ++
 arch/um/nommu/include/asm/cpu.h |  3 +++
 arch/um/nommu/um/cpu.c          | 17 ++++++++++++++---
 arch/um/nommu/um/setup.c        |  5 +++++
 arch/um/nommu/um/threads.c      |  3 +++
 tools/um/uml/signal.c           | 12 ++++++++++--
 8 files changed, 51 insertions(+), 5 deletions(-)

diff --git a/arch/um/include/shared/os.h b/arch/um/include/shared/os.h
index f467d28fc0b4..0da3ce22e16f 100644
--- a/arch/um/include/shared/os.h
+++ b/arch/um/include/shared/os.h
@@ -241,6 +241,7 @@ extern int set_signals(int enable);
 extern int set_signals_trace(int enable);
 extern int os_is_signal_stack(void);
 extern void deliver_alarm(void);
+extern void set_pending_signals(int sig);
 
 /* util.c */
 extern void stack_protections(unsigned long address);
diff --git a/arch/um/kernel/irq.c b/arch/um/kernel/irq.c
index 3577118bb4a5..4ce7f5fef7f8 100644
--- a/arch/um/kernel/irq.c
+++ b/arch/um/kernel/irq.c
@@ -19,6 +19,7 @@
 #include <kern_util.h>
 #include <os.h>
 #include <irq_user.h>
+#include <asm/cpu.h>
 
 
 extern void free_irqs(void);
@@ -541,6 +542,11 @@ unsigned long to_irq_stack(unsigned long *mask_out)
 	unsigned long mask, old;
 	int nested;
 
+#ifndef CONFIG_MMU
+	if (!nommu_irq_enter(ffs(*mask_out) - 1))
+		return 1;
+#endif
+
 	mask = xchg(&pending_mask, *mask_out);
 	if (mask != 0) {
 		/*
@@ -557,6 +563,10 @@ unsigned long to_irq_stack(unsigned long *mask_out)
 			old |= mask;
 			mask = xchg(&pending_mask, old);
 		} while (mask != old);
+
+#ifndef CONFIG_MMU
+		nommu_irq_exit();
+#endif
 		return 1;
 	}
 
@@ -594,6 +604,9 @@ unsigned long from_irq_stack(int nested)
 	*to = *ti;
 
 	mask = xchg(&pending_mask, 0);
+#ifndef CONFIG_MMU
+	nommu_irq_exit();
+#endif
 	return mask & ~1;
 }
 
diff --git a/arch/um/kernel/time.c b/arch/um/kernel/time.c
index 25eaa6a0c658..9e141fe8fe27 100644
--- a/arch/um/kernel/time.c
+++ b/arch/um/kernel/time.c
@@ -597,11 +597,13 @@ static struct clock_event_device timer_clockevent = {
 
 static irqreturn_t um_timer(int irq, void *dev)
 {
+#ifdef CONFIG_MMU
 	if (get_current()->mm != NULL)
 	{
         /* userspace - relay signal, results in correct userspace timers */
 		os_alarm_process(get_current()->mm->context.id.u.pid);
 	}
+#endif
 
 	(*timer_clockevent.event_handler)(&timer_clockevent);
 
diff --git a/arch/um/nommu/include/asm/cpu.h b/arch/um/nommu/include/asm/cpu.h
index c101c078ef21..c52a78c77bea 100644
--- a/arch/um/nommu/include/asm/cpu.h
+++ b/arch/um/nommu/include/asm/cpu.h
@@ -10,4 +10,7 @@ void lkl_cpu_wait_shutdown(void);
 void lkl_cpu_change_owner(lkl_thread_t owner);
 void lkl_cpu_set_irqs_pending(void);
 
+void nommu_irq_exit(void);
+int nommu_irq_enter(int sig);
+
 #endif
diff --git a/arch/um/nommu/um/cpu.c b/arch/um/nommu/um/cpu.c
index 9986a3f8c5dd..75d8bdc1847e 100644
--- a/arch/um/nommu/um/cpu.c
+++ b/arch/um/nommu/um/cpu.c
@@ -8,15 +8,16 @@
 #include <asm/sched.h>
 #include <asm/syscalls.h>
 #include <init.h>
+#include <os.h>
 
 void run_irqs(void)
 {
-	panic("unimplemented %s", __func__);
+	unblock_signals();
 }
 
-int set_irq_pending(int irq)
+void set_irq_pending(int sig)
 {
-	panic("unimplemented %s", __func__);
+	set_pending_signals(sig);
 }
 
 /*
@@ -174,6 +175,16 @@ int lkl_cpu_try_run_irq(int irq)
 	return ret;
 }
 
+int nommu_irq_enter(int sig)
+{
+	return lkl_cpu_try_run_irq(sig);
+}
+
+void nommu_irq_exit(void)
+{
+	return lkl_cpu_put();
+}
+
 static void lkl_cpu_shutdown(void)
 {
 	__sync_fetch_and_add(&cpu.shutdown_gate, MAX_THREADS);
diff --git a/arch/um/nommu/um/setup.c b/arch/um/nommu/um/setup.c
index eaa74d771f73..922188690139 100644
--- a/arch/um/nommu/um/setup.c
+++ b/arch/um/nommu/um/setup.c
@@ -36,6 +36,8 @@ static void __init *lkl_run_kernel(void *arg)
 
 	panic_blink = lkl_panic_blink;
 
+	/* signal should be received at this thread (main and idle threads) */
+	init_new_thread_signals();
 	threads_init();
 	lkl_cpu_get();
 	start_kernel();
@@ -58,6 +60,9 @@ int __init lkl_start_kernel(struct lkl_host_operations *ops,
 	if (ret)
 		goto out_free_init_sem;
 
+	change_sig(SIGALRM, 0);
+	change_sig(SIGIO, 0);
+
 	ret = lkl_ops->thread_create(lkl_run_kernel, NULL);
 	if (!ret) {
 		ret = -ENOMEM;
diff --git a/arch/um/nommu/um/threads.c b/arch/um/nommu/um/threads.c
index e9a49100d40e..fe2fa3e688fd 100644
--- a/arch/um/nommu/um/threads.c
+++ b/arch/um/nommu/um/threads.c
@@ -152,6 +152,9 @@ static void *thread_bootstrap(void *_tba)
 	int (*f)(void *) = tba->f;
 	void *arg = tba->arg;
 
+	change_sig(SIGALRM, 0);
+	change_sig(SIGIO, 0);
+
 	lkl_ops->sem_down(ti->task->thread.arch.sched_sem);
 	kfree(tba);
 	if (ti->task->thread.prev_sched)
diff --git a/tools/um/uml/signal.c b/tools/um/uml/signal.c
index b58bc68cbe64..2b5c98de21cd 100644
--- a/tools/um/uml/signal.c
+++ b/tools/um/uml/signal.c
@@ -218,8 +218,8 @@ void set_handler(int sig)
 
 	sigemptyset(&sig_mask);
 	sigaddset(&sig_mask, sig);
-	if (sigprocmask(SIG_UNBLOCK, &sig_mask, NULL) < 0)
-		panic("sigprocmask failed - errno = %d\n", errno);
+	if (pthread_sigmask(SIG_UNBLOCK, &sig_mask, NULL) < 0)
+		panic("pthread_sigmask failed - errno = %d\n", errno);
 }
 
 int change_sig(int signal, int on)
@@ -355,3 +355,11 @@ int os_is_signal_stack(void)
 
 	return ss.ss_flags & SS_ONSTACK;
 }
+
+void set_pending_signals(int sig)
+{
+	if (sig == SIGIO)
+		signals_pending |= SIGIO_MASK;
+	else if (sig == SIGALRM)
+		signals_pending |= SIGALRM_MASK;
+}
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v7 15/21] um: nommu: integrate with irq infrastructure of UML
@ 2020-10-06  9:44           ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-06  9:44 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Hajime Tazaki, retrage01

In order to cooperate with UML's irq infrastructure and nommu threads
based on host threads, irq handlers shall synchronize the our
scheduler which is controlled by struct lkl_cpu.  To do that, the irq
infra notifies its entry of handlers by obtaining cpu access of thread
scheduler (lkl_cpu_try_run_irq) and its release (lkl_cpu_put).

In additon to that, in order to stick the signal handler's thread to the
one of the idle thread, several required configurations of (thread's)
signal mask are added: otherwise handlers running on arbitrary thread
cannot obtain cpu access and immediately fall into pending interrupt
which may slow down the delivery of signals.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/include/shared/os.h     |  1 +
 arch/um/kernel/irq.c            | 13 +++++++++++++
 arch/um/kernel/time.c           |  2 ++
 arch/um/nommu/include/asm/cpu.h |  3 +++
 arch/um/nommu/um/cpu.c          | 17 ++++++++++++++---
 arch/um/nommu/um/setup.c        |  5 +++++
 arch/um/nommu/um/threads.c      |  3 +++
 tools/um/uml/signal.c           | 12 ++++++++++--
 8 files changed, 51 insertions(+), 5 deletions(-)

diff --git a/arch/um/include/shared/os.h b/arch/um/include/shared/os.h
index f467d28fc0b4..0da3ce22e16f 100644
--- a/arch/um/include/shared/os.h
+++ b/arch/um/include/shared/os.h
@@ -241,6 +241,7 @@ extern int set_signals(int enable);
 extern int set_signals_trace(int enable);
 extern int os_is_signal_stack(void);
 extern void deliver_alarm(void);
+extern void set_pending_signals(int sig);
 
 /* util.c */
 extern void stack_protections(unsigned long address);
diff --git a/arch/um/kernel/irq.c b/arch/um/kernel/irq.c
index 3577118bb4a5..4ce7f5fef7f8 100644
--- a/arch/um/kernel/irq.c
+++ b/arch/um/kernel/irq.c
@@ -19,6 +19,7 @@
 #include <kern_util.h>
 #include <os.h>
 #include <irq_user.h>
+#include <asm/cpu.h>
 
 
 extern void free_irqs(void);
@@ -541,6 +542,11 @@ unsigned long to_irq_stack(unsigned long *mask_out)
 	unsigned long mask, old;
 	int nested;
 
+#ifndef CONFIG_MMU
+	if (!nommu_irq_enter(ffs(*mask_out) - 1))
+		return 1;
+#endif
+
 	mask = xchg(&pending_mask, *mask_out);
 	if (mask != 0) {
 		/*
@@ -557,6 +563,10 @@ unsigned long to_irq_stack(unsigned long *mask_out)
 			old |= mask;
 			mask = xchg(&pending_mask, old);
 		} while (mask != old);
+
+#ifndef CONFIG_MMU
+		nommu_irq_exit();
+#endif
 		return 1;
 	}
 
@@ -594,6 +604,9 @@ unsigned long from_irq_stack(int nested)
 	*to = *ti;
 
 	mask = xchg(&pending_mask, 0);
+#ifndef CONFIG_MMU
+	nommu_irq_exit();
+#endif
 	return mask & ~1;
 }
 
diff --git a/arch/um/kernel/time.c b/arch/um/kernel/time.c
index 25eaa6a0c658..9e141fe8fe27 100644
--- a/arch/um/kernel/time.c
+++ b/arch/um/kernel/time.c
@@ -597,11 +597,13 @@ static struct clock_event_device timer_clockevent = {
 
 static irqreturn_t um_timer(int irq, void *dev)
 {
+#ifdef CONFIG_MMU
 	if (get_current()->mm != NULL)
 	{
         /* userspace - relay signal, results in correct userspace timers */
 		os_alarm_process(get_current()->mm->context.id.u.pid);
 	}
+#endif
 
 	(*timer_clockevent.event_handler)(&timer_clockevent);
 
diff --git a/arch/um/nommu/include/asm/cpu.h b/arch/um/nommu/include/asm/cpu.h
index c101c078ef21..c52a78c77bea 100644
--- a/arch/um/nommu/include/asm/cpu.h
+++ b/arch/um/nommu/include/asm/cpu.h
@@ -10,4 +10,7 @@ void lkl_cpu_wait_shutdown(void);
 void lkl_cpu_change_owner(lkl_thread_t owner);
 void lkl_cpu_set_irqs_pending(void);
 
+void nommu_irq_exit(void);
+int nommu_irq_enter(int sig);
+
 #endif
diff --git a/arch/um/nommu/um/cpu.c b/arch/um/nommu/um/cpu.c
index 9986a3f8c5dd..75d8bdc1847e 100644
--- a/arch/um/nommu/um/cpu.c
+++ b/arch/um/nommu/um/cpu.c
@@ -8,15 +8,16 @@
 #include <asm/sched.h>
 #include <asm/syscalls.h>
 #include <init.h>
+#include <os.h>
 
 void run_irqs(void)
 {
-	panic("unimplemented %s", __func__);
+	unblock_signals();
 }
 
-int set_irq_pending(int irq)
+void set_irq_pending(int sig)
 {
-	panic("unimplemented %s", __func__);
+	set_pending_signals(sig);
 }
 
 /*
@@ -174,6 +175,16 @@ int lkl_cpu_try_run_irq(int irq)
 	return ret;
 }
 
+int nommu_irq_enter(int sig)
+{
+	return lkl_cpu_try_run_irq(sig);
+}
+
+void nommu_irq_exit(void)
+{
+	return lkl_cpu_put();
+}
+
 static void lkl_cpu_shutdown(void)
 {
 	__sync_fetch_and_add(&cpu.shutdown_gate, MAX_THREADS);
diff --git a/arch/um/nommu/um/setup.c b/arch/um/nommu/um/setup.c
index eaa74d771f73..922188690139 100644
--- a/arch/um/nommu/um/setup.c
+++ b/arch/um/nommu/um/setup.c
@@ -36,6 +36,8 @@ static void __init *lkl_run_kernel(void *arg)
 
 	panic_blink = lkl_panic_blink;
 
+	/* signal should be received at this thread (main and idle threads) */
+	init_new_thread_signals();
 	threads_init();
 	lkl_cpu_get();
 	start_kernel();
@@ -58,6 +60,9 @@ int __init lkl_start_kernel(struct lkl_host_operations *ops,
 	if (ret)
 		goto out_free_init_sem;
 
+	change_sig(SIGALRM, 0);
+	change_sig(SIGIO, 0);
+
 	ret = lkl_ops->thread_create(lkl_run_kernel, NULL);
 	if (!ret) {
 		ret = -ENOMEM;
diff --git a/arch/um/nommu/um/threads.c b/arch/um/nommu/um/threads.c
index e9a49100d40e..fe2fa3e688fd 100644
--- a/arch/um/nommu/um/threads.c
+++ b/arch/um/nommu/um/threads.c
@@ -152,6 +152,9 @@ static void *thread_bootstrap(void *_tba)
 	int (*f)(void *) = tba->f;
 	void *arg = tba->arg;
 
+	change_sig(SIGALRM, 0);
+	change_sig(SIGIO, 0);
+
 	lkl_ops->sem_down(ti->task->thread.arch.sched_sem);
 	kfree(tba);
 	if (ti->task->thread.prev_sched)
diff --git a/tools/um/uml/signal.c b/tools/um/uml/signal.c
index b58bc68cbe64..2b5c98de21cd 100644
--- a/tools/um/uml/signal.c
+++ b/tools/um/uml/signal.c
@@ -218,8 +218,8 @@ void set_handler(int sig)
 
 	sigemptyset(&sig_mask);
 	sigaddset(&sig_mask, sig);
-	if (sigprocmask(SIG_UNBLOCK, &sig_mask, NULL) < 0)
-		panic("sigprocmask failed - errno = %d\n", errno);
+	if (pthread_sigmask(SIG_UNBLOCK, &sig_mask, NULL) < 0)
+		panic("pthread_sigmask failed - errno = %d\n", errno);
 }
 
 int change_sig(int signal, int on)
@@ -355,3 +355,11 @@ int os_is_signal_stack(void)
 
 	return ss.ss_flags & SS_ONSTACK;
 }
+
+void set_pending_signals(int sig)
+{
+	if (sig == SIGIO)
+		signals_pending |= SIGIO_MASK;
+	else if (sig == SIGALRM)
+		signals_pending |= SIGALRM_MASK;
+}
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v7 16/21] um: nommu: plug in the build system
  2020-10-06  9:44         ` Hajime Tazaki
@ 2020-10-06  9:44           ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-06  9:44 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch, Hajime Tazaki

Basic Makefiles for building library of nommu mode. Add a new architecture
specific target for installing the resulting library files and headers.

To make nommu binaries build, UML introduced an additional option, UMMODE
variable, to switch the output file of build: kernel (default), or
library (nommu).  Those modes are not able to be ON at the same time.

To build on library mode, users do the following:

  make defconfig ARCH=um UMMODE=library
  make ARCH=um UMMODE=library

Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/Kconfig           | 17 ++++++++++++++---
 arch/um/Makefile          | 12 +++++++++++-
 arch/um/kernel/Makefile   | 14 +++++++++-----
 arch/um/nommu/um/Kconfig  | 37 +++++++++++++++++++++++++++++++++++++
 arch/um/nommu/um/Makefile |  4 ++++
 5 files changed, 75 insertions(+), 9 deletions(-)
 create mode 100644 arch/um/nommu/um/Kconfig
 create mode 100644 arch/um/nommu/um/Makefile

diff --git a/arch/um/Kconfig b/arch/um/Kconfig
index e41d31d0d875..df7daceb095c 100644
--- a/arch/um/Kconfig
+++ b/arch/um/Kconfig
@@ -22,9 +22,20 @@ config UML
 	select TTY # Needed for line.c
 	select MODULE_REL_CRCS if MODVERSIONS
 
+config UMMODE_LIB
+	bool "UML mode: library mode"
+	default y if "$(UMMODE)" = "library"
+	help
+	 This mode switches a mode to build a library of UML (Linux
+	 Kernel Library/LKL).  This switch is exclusive to "kernel mode"
+	 of UML, which is traditional mode of UML.
+
+	 For more detail about LKL, see
+	 <file:Documentation/virt/uml/lkl.txt>.
+
 config MMU
 	bool
-	default y
+	default y if !UMMODE_LIB
 
 config NO_IOMEM
 	def_bool y
@@ -45,12 +56,12 @@ config LOCKDEP_SUPPORT
 
 config STACKTRACE_SUPPORT
 	bool
-	default y
+	default y if MMU
 	select STACKTRACE
 
 config GENERIC_CALIBRATE_DELAY
 	bool
-	default y
+	default y if MMU
 
 config HZ
 	int
diff --git a/arch/um/Makefile b/arch/um/Makefile
index 8be7bc479442..ee35bd030bf8 100644
--- a/arch/um/Makefile
+++ b/arch/um/Makefile
@@ -17,6 +17,10 @@ else
         KBUILD_DEFCONFIG := $(SUBARCH)_defconfig
 endif
 
+ifeq ($(UMMODE),library)
+  SUBARCH := um/nommu
+endif
+
 ARCH_DIR := arch/um
 OS := $(shell uname -s)
 # We require bash because the vmlinux link and loader script cpp use bash
@@ -97,8 +101,14 @@ KBUILD_CFLAGS += $(KERNEL_DEFINES)
 LDFLAGS_vmlinux += -r
 
 INSTALL_PATH=$(objtree)/tools/um
+ifeq ($(UMMODE),library)
+all: install
+	$(Q)$(MAKE) -C $(srctree)/tools/um
+install: linux.o um_headers_install
+else
 all: linux
 install: linux.o
+endif
 	@echo "  INSTALL	$(INSTALL_PATH)/lib/$<"
 	@mkdir -p $(INSTALL_PATH)/lib/
 	@cp $< $(INSTALL_PATH)/lib/
@@ -111,7 +121,7 @@ linux: linux.o
 
 linux.o: vmlinux
 	@echo '  LINK	$@'
-	$(Q)$(OBJCOPY) -R .eh_frame $< $@
+	$(Q)$(OBJCOPY) -R .eh_frame -L sem_init -L sem_post -L sem_wait -L sem_destroy $< $@
 	$(Q)mkdir -p $(objtree)/tools/um/lib
 
 define archhelp
diff --git a/arch/um/kernel/Makefile b/arch/um/kernel/Makefile
index 9b63831a69e1..01605ed439cb 100644
--- a/arch/um/kernel/Makefile
+++ b/arch/um/kernel/Makefile
@@ -14,17 +14,21 @@ CPPFLAGS_vmlinux.lds := -DSTART=$(LDS_START)		\
 			$(LDS_EXTRA)
 extra-y := vmlinux.lds
 
-obj-y = config.o exec.o exitcode.o irq.o ksyms.o mem.o \
-	physmem.o process.o ptrace.o reboot.o sigio.o \
-	signal.o syscall.o sysrq.o time.o tlb.o trap.o \
-	um_arch.o umid.o maccess.o kmsg_dump.o skas/
+obj-y = config.o exitcode.o irq.o ksyms.o \
+	process.o reboot.o sigio.o \
+	signal.o syscall.o time.o \
+	um_arch.o umid.o maccess.o kmsg_dump.o
+
+ifdef CONFIG_MMU
+obj-y += exec.o mem.o physmem.o ptrace.o sysrq.o tlb.o trap.o skas/
+endif
 
 obj-$(CONFIG_BLK_DEV_INITRD) += initrd.o
 obj-$(CONFIG_GPROF)	+= gprof_syms.o
 obj-$(CONFIG_GCOV)	+= gmon_syms.o
 obj-$(CONFIG_EARLY_PRINTK) += early_printk.o
 obj-$(CONFIG_STACKTRACE) += stacktrace.o
-obj-y += user_syms.o
+obj-$(CONFIG_MMU) += user_syms.o
 
 USER_OBJS := config.o
 
diff --git a/arch/um/nommu/um/Kconfig b/arch/um/nommu/um/Kconfig
new file mode 100644
index 000000000000..20b3eaccb6f0
--- /dev/null
+++ b/arch/um/nommu/um/Kconfig
@@ -0,0 +1,37 @@
+config UML_NOMMU
+	def_bool y
+	depends on !SMP && !MMU
+	select UACCESS_MEMCPY
+	select ARCH_THREAD_STACK_ALLOCATOR
+	select ARCH_HAS_SYSCALL_WRAPPER
+
+config 64BIT
+	bool
+	default y
+
+config GENERIC_CSUM
+	def_bool y
+
+config GENERIC_ATOMIC64
+	bool
+	default y if !64BIT
+
+config SECCOMP
+	bool
+	default n
+
+config GENERIC_HWEIGHT
+	def_bool y
+
+config GENERIC_CALIBRATE_DELAY
+	bool
+	default n
+
+config STACKTRACE_SUPPORT
+	bool
+	default n
+
+# XXX: need this to work well with tap13.py
+config PRINTK_TIME
+	bool
+	default y
diff --git a/arch/um/nommu/um/Makefile b/arch/um/nommu/um/Makefile
new file mode 100644
index 000000000000..b580e0fe97b5
--- /dev/null
+++ b/arch/um/nommu/um/Makefile
@@ -0,0 +1,4 @@
+include/generated/user_constants.h: $(srctree)/arch/um/nommu/um/user_constants.h
+	$(Q)cp -f $^ $@
+
+obj-y = bootmem.o console.o cpu.o delay.o setup.o syscalls.o threads.o unimplemented.o
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v7 16/21] um: nommu: plug in the build system
@ 2020-10-06  9:44           ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-06  9:44 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Hajime Tazaki, retrage01

Basic Makefiles for building library of nommu mode. Add a new architecture
specific target for installing the resulting library files and headers.

To make nommu binaries build, UML introduced an additional option, UMMODE
variable, to switch the output file of build: kernel (default), or
library (nommu).  Those modes are not able to be ON at the same time.

To build on library mode, users do the following:

  make defconfig ARCH=um UMMODE=library
  make ARCH=um UMMODE=library

Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/Kconfig           | 17 ++++++++++++++---
 arch/um/Makefile          | 12 +++++++++++-
 arch/um/kernel/Makefile   | 14 +++++++++-----
 arch/um/nommu/um/Kconfig  | 37 +++++++++++++++++++++++++++++++++++++
 arch/um/nommu/um/Makefile |  4 ++++
 5 files changed, 75 insertions(+), 9 deletions(-)
 create mode 100644 arch/um/nommu/um/Kconfig
 create mode 100644 arch/um/nommu/um/Makefile

diff --git a/arch/um/Kconfig b/arch/um/Kconfig
index e41d31d0d875..df7daceb095c 100644
--- a/arch/um/Kconfig
+++ b/arch/um/Kconfig
@@ -22,9 +22,20 @@ config UML
 	select TTY # Needed for line.c
 	select MODULE_REL_CRCS if MODVERSIONS
 
+config UMMODE_LIB
+	bool "UML mode: library mode"
+	default y if "$(UMMODE)" = "library"
+	help
+	 This mode switches a mode to build a library of UML (Linux
+	 Kernel Library/LKL).  This switch is exclusive to "kernel mode"
+	 of UML, which is traditional mode of UML.
+
+	 For more detail about LKL, see
+	 <file:Documentation/virt/uml/lkl.txt>.
+
 config MMU
 	bool
-	default y
+	default y if !UMMODE_LIB
 
 config NO_IOMEM
 	def_bool y
@@ -45,12 +56,12 @@ config LOCKDEP_SUPPORT
 
 config STACKTRACE_SUPPORT
 	bool
-	default y
+	default y if MMU
 	select STACKTRACE
 
 config GENERIC_CALIBRATE_DELAY
 	bool
-	default y
+	default y if MMU
 
 config HZ
 	int
diff --git a/arch/um/Makefile b/arch/um/Makefile
index 8be7bc479442..ee35bd030bf8 100644
--- a/arch/um/Makefile
+++ b/arch/um/Makefile
@@ -17,6 +17,10 @@ else
         KBUILD_DEFCONFIG := $(SUBARCH)_defconfig
 endif
 
+ifeq ($(UMMODE),library)
+  SUBARCH := um/nommu
+endif
+
 ARCH_DIR := arch/um
 OS := $(shell uname -s)
 # We require bash because the vmlinux link and loader script cpp use bash
@@ -97,8 +101,14 @@ KBUILD_CFLAGS += $(KERNEL_DEFINES)
 LDFLAGS_vmlinux += -r
 
 INSTALL_PATH=$(objtree)/tools/um
+ifeq ($(UMMODE),library)
+all: install
+	$(Q)$(MAKE) -C $(srctree)/tools/um
+install: linux.o um_headers_install
+else
 all: linux
 install: linux.o
+endif
 	@echo "  INSTALL	$(INSTALL_PATH)/lib/$<"
 	@mkdir -p $(INSTALL_PATH)/lib/
 	@cp $< $(INSTALL_PATH)/lib/
@@ -111,7 +121,7 @@ linux: linux.o
 
 linux.o: vmlinux
 	@echo '  LINK	$@'
-	$(Q)$(OBJCOPY) -R .eh_frame $< $@
+	$(Q)$(OBJCOPY) -R .eh_frame -L sem_init -L sem_post -L sem_wait -L sem_destroy $< $@
 	$(Q)mkdir -p $(objtree)/tools/um/lib
 
 define archhelp
diff --git a/arch/um/kernel/Makefile b/arch/um/kernel/Makefile
index 9b63831a69e1..01605ed439cb 100644
--- a/arch/um/kernel/Makefile
+++ b/arch/um/kernel/Makefile
@@ -14,17 +14,21 @@ CPPFLAGS_vmlinux.lds := -DSTART=$(LDS_START)		\
 			$(LDS_EXTRA)
 extra-y := vmlinux.lds
 
-obj-y = config.o exec.o exitcode.o irq.o ksyms.o mem.o \
-	physmem.o process.o ptrace.o reboot.o sigio.o \
-	signal.o syscall.o sysrq.o time.o tlb.o trap.o \
-	um_arch.o umid.o maccess.o kmsg_dump.o skas/
+obj-y = config.o exitcode.o irq.o ksyms.o \
+	process.o reboot.o sigio.o \
+	signal.o syscall.o time.o \
+	um_arch.o umid.o maccess.o kmsg_dump.o
+
+ifdef CONFIG_MMU
+obj-y += exec.o mem.o physmem.o ptrace.o sysrq.o tlb.o trap.o skas/
+endif
 
 obj-$(CONFIG_BLK_DEV_INITRD) += initrd.o
 obj-$(CONFIG_GPROF)	+= gprof_syms.o
 obj-$(CONFIG_GCOV)	+= gmon_syms.o
 obj-$(CONFIG_EARLY_PRINTK) += early_printk.o
 obj-$(CONFIG_STACKTRACE) += stacktrace.o
-obj-y += user_syms.o
+obj-$(CONFIG_MMU) += user_syms.o
 
 USER_OBJS := config.o
 
diff --git a/arch/um/nommu/um/Kconfig b/arch/um/nommu/um/Kconfig
new file mode 100644
index 000000000000..20b3eaccb6f0
--- /dev/null
+++ b/arch/um/nommu/um/Kconfig
@@ -0,0 +1,37 @@
+config UML_NOMMU
+	def_bool y
+	depends on !SMP && !MMU
+	select UACCESS_MEMCPY
+	select ARCH_THREAD_STACK_ALLOCATOR
+	select ARCH_HAS_SYSCALL_WRAPPER
+
+config 64BIT
+	bool
+	default y
+
+config GENERIC_CSUM
+	def_bool y
+
+config GENERIC_ATOMIC64
+	bool
+	default y if !64BIT
+
+config SECCOMP
+	bool
+	default n
+
+config GENERIC_HWEIGHT
+	def_bool y
+
+config GENERIC_CALIBRATE_DELAY
+	bool
+	default n
+
+config STACKTRACE_SUPPORT
+	bool
+	default n
+
+# XXX: need this to work well with tap13.py
+config PRINTK_TIME
+	bool
+	default y
diff --git a/arch/um/nommu/um/Makefile b/arch/um/nommu/um/Makefile
new file mode 100644
index 000000000000..b580e0fe97b5
--- /dev/null
+++ b/arch/um/nommu/um/Makefile
@@ -0,0 +1,4 @@
+include/generated/user_constants.h: $(srctree)/arch/um/nommu/um/user_constants.h
+	$(Q)cp -f $^ $@
+
+obj-y = bootmem.o console.o cpu.o delay.o setup.o syscalls.o threads.o unimplemented.o
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v7 17/21] um: host: add nommu build for ARCH=um
  2020-10-06  9:44         ` Hajime Tazaki
@ 2020-10-06  9:44           ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-06  9:44 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch, Hajime Tazaki

This patch adds the skeleton for the host library.

The host library is implementing the host operations needed by nommu
mode and is split into host dependent (depends on a specific host, e.g.
POSIX hosts) and host independent parts (will work on all supported
hosts).

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/um/.gitignore                |   1 +
 tools/um/Makefile                  |  32 +++---
 tools/um/Targets                   |   3 +-
 tools/um/include/lkl.h             | 151 +++++++++++++++++++++++++++++
 tools/um/include/lkl_host.h        |  26 +++++
 tools/um/uml/Build                 |  16 ++-
 tools/um/uml/nommu/Build           |   1 +
 tools/um/uml/nommu/registers.c     |  21 ++++
 tools/um/uml/nommu/unimplemented.c |  21 ++++
 9 files changed, 252 insertions(+), 20 deletions(-)
 create mode 100644 tools/um/.gitignore
 create mode 100644 tools/um/include/lkl.h
 create mode 100644 tools/um/include/lkl_host.h
 create mode 100644 tools/um/uml/nommu/Build
 create mode 100644 tools/um/uml/nommu/registers.c
 create mode 100644 tools/um/uml/nommu/unimplemented.c

diff --git a/tools/um/.gitignore b/tools/um/.gitignore
new file mode 100644
index 000000000000..0d20b6487c61
--- /dev/null
+++ b/tools/um/.gitignore
@@ -0,0 +1 @@
+*.pyc
diff --git a/tools/um/Makefile b/tools/um/Makefile
index 552ed5f1edae..bbd5e0dcadbc 100644
--- a/tools/um/Makefile
+++ b/tools/um/Makefile
@@ -5,6 +5,7 @@
 MAKEFLAGS += -r --no-print-directory
 
 KCONFIG?=defconfig
+UMMODE?=kernel
 
 ifneq ($(silent),1)
   ifneq ($(V),1)
@@ -15,10 +16,8 @@ endif
 
 PREFIX   := /usr
 
-ifeq (,$(srctree))
-  srctree := $(patsubst %/,%,$(dir $(shell pwd)))
-  srctree := $(patsubst %/,%,$(dir $(srctree)))
-endif
+srctree := $(patsubst %/,%,$(dir $(shell pwd)))
+srctree := $(patsubst %/,%,$(dir $(srctree)))
 export srctree
 
 -include ../scripts/Makefile.include
@@ -32,9 +31,6 @@ endif
 export OUTPUT
 export objtree := $(OUTPUT)/../..
 
-
-all:
-
 export CFLAGS += -I$(OUTPUT)/include -Iinclude -Wall -g -O2 -Wextra -fPIC \
 	 -Wno-unused-parameter \
 	 -Wno-missing-field-initializers -fno-strict-aliasing
@@ -45,28 +41,34 @@ TARGETS := $(progs-y:%=$(OUTPUT)%)
 TARGETS += $(libs-y:%=$(OUTPUT)%)
 all: $(TARGETS)
 
-# rule to build linux.o
-$(OUTPUT)lib/linux.o:
-	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) $(KCONFIG)
-	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) install INSTALL_PATH=$(OUTPUT)
+$(objtree)/linux.o:
+	$(Q)echo ""
+	$(Q)echo ""
+	$(Q)echo "==> $@ isn't found; please make ARCH=um"
+	$(Q)echo ""
+	$(Q)echo ""
+	$(Q)exit 1
 
-$(OUTPUT)liblinux.a: $(OUTPUT)lib/linux.o $(OUTPUT)uml/liblinux-in.o $(OUTPUT)lib/liblinux-in.o
+$(OUTPUT)lib/linux.o: $(objtree)/linux.o
+	$(Q)cp $(objtree)/linux.o $(OUTPUT)lib/linux.o
+
+# rule to build linux.o
+$(OUTPUT)liblinux.a: $(OUTPUT)uml/liblinux-in.o $(OUTPUT)lib/liblinux-in.o $(OUTPUT)lib/linux.o
 	$(QUIET_AR)$(AR) -rc $@ $^
 
 # rule to link programs
 $(OUTPUT)%: $(OUTPUT)%-in.o $(OUTPUT)liblinux.a
 	$(QUIET_LINK)$(CC) $(LDFLAGS) $(LDFLAGS_$(notdir $*)-y) -o $@ $^ $(LDLIBS) $(LDLIBS_$(notdir $*)-y)
 
-$(OUTPUT)%-in.o: FORCE
-	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
-
 # rule to build objects
 $(OUTPUT)%-in.o: $(OUTPUT)lib/linux.o FORCE
 	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
 
+RM := rm -f
 clean:
 	$(call QUIET_CLEAN, objects)find $(OUTPUT) -name '*.o' -delete -o -name '\.*.cmd'\
 	 -delete -o -name '\.*.d' -delete
+	$(call QUIET_CLEAN, headers)$(RM) -r $(OUTPUT)/include/lkl/
 	$(call QUIET_CLEAN, liblinux.a)$(RM) $(OUTPUT)/liblinux.a
 	$(call QUIET_CLEAN, $(TARGETS))$(RM) $(TARGETS)
 
diff --git a/tools/um/Targets b/tools/um/Targets
index a4711f1ef422..4e1f0f4d81a3 100644
--- a/tools/um/Targets
+++ b/tools/um/Targets
@@ -1,4 +1,5 @@
 progs-y += uml/linux
-LDLIBS_linux-y := -lrt -lpthread -lutil
+
+LDLIBS := -lrt -lpthread -lutil
 LDFLAGS_linux-y := -no-pie -Wl,--wrap,malloc -Wl,--wrap,free -Wl,--wrap,calloc
 LDFLAGS_linux-$(UML_STATIC) += -static
diff --git a/tools/um/include/lkl.h b/tools/um/include/lkl.h
new file mode 100644
index 000000000000..707e01b64a70
--- /dev/null
+++ b/tools/um/include/lkl.h
@@ -0,0 +1,151 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_H
+#define _LKL_H
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#define _LKL_LIBC_COMPAT_H
+
+#ifdef __cplusplus
+#define class __lkl__class
+#endif
+
+/*
+ * Avoid collisions between Android which defines __unused and
+ * linux/icmp.h which uses __unused as a structure field.
+ */
+#pragma push_macro("__unused")
+#undef __unused
+
+#include <lkl/asm/syscalls.h>
+
+#pragma pop_macro("__unused")
+
+#ifdef __cplusplus
+#undef class
+#endif
+
+/**
+ * lkl_strerror - returns a string describing the given error code
+ *
+ * @err - error code
+ * @returns - string for the given error code
+ */
+const char *lkl_strerror(int err);
+
+/**
+ * lkl_perror - prints a string describing the given error code
+ *
+ * @msg - prefix for the error message
+ * @err - error code
+ */
+void lkl_perror(char *msg, int err);
+
+#if __LKL__BITS_PER_LONG == 64
+#define lkl_sys_fstatat lkl_sys_newfstatat
+#define lkl_sys_fstat lkl_sys_newfstat
+
+#else
+#define __lkl__NR_fcntl __lkl__NR_fcntl64
+
+#define lkl_stat lkl_stat64
+#define lkl_sys_stat lkl_sys_stat64
+#define lkl_sys_lstat lkl_sys_lstat64
+#define lkl_sys_truncate lkl_sys_truncate64
+#define lkl_sys_ftruncate lkl_sys_ftruncate64
+#define lkl_sys_sendfile lkl_sys_sendfile64
+#define lkl_sys_fstatat lkl_sys_fstatat64
+#define lkl_sys_fstat lkl_sys_fstat64
+#define lkl_sys_fcntl lkl_sys_fcntl64
+
+#define lkl_statfs lkl_statfs64
+
+static inline int lkl_sys_statfs(const char *path, struct lkl_statfs *buf)
+{
+	return lkl_sys_statfs64(path, sizeof(*buf), buf);
+}
+
+static inline int lkl_sys_fstatfs(unsigned int fd, struct lkl_statfs *buf)
+{
+	return lkl_sys_fstatfs64(fd, sizeof(*buf), buf);
+}
+
+#define lkl_sys_nanosleep lkl_sys_nanosleep_time32
+static inline int lkl_sys_nanosleep_time32(struct lkl_timespec *rqtp,
+					   struct lkl_timespec *rmtp)
+{
+	long p[6] = {(long)rqtp, (long)rmtp, 0, 0, 0, 0};
+
+	return lkl_syscall(__lkl__NR_nanosleep, p);
+}
+
+#endif
+
+static inline int lkl_sys_stat(const char *path, struct lkl_stat *buf)
+{
+	return lkl_sys_fstatat(LKL_AT_FDCWD, path, buf, 0);
+}
+
+static inline int lkl_sys_lstat(const char *path, struct lkl_stat *buf)
+{
+	return lkl_sys_fstatat(LKL_AT_FDCWD, path, buf,
+			       LKL_AT_SYMLINK_NOFOLLOW);
+}
+
+#ifdef __lkl__NR_openat
+/**
+ * lkl_sys_open - wrapper for lkl_sys_openat
+ */
+static inline long lkl_sys_open(const char *file, int flags, int mode)
+{
+	return lkl_sys_openat(LKL_AT_FDCWD, file, flags, mode);
+}
+
+/**
+ * lkl_sys_creat - wrapper for lkl_sys_openat
+ */
+static inline long lkl_sys_creat(const char *file, int mode)
+{
+	return lkl_sys_openat(LKL_AT_FDCWD, file,
+			      LKL_O_CREAT|LKL_O_WRONLY|LKL_O_TRUNC, mode);
+}
+#endif
+
+#ifdef __lkl__NR_mkdirat
+/**
+ * lkl_sys_mkdir - wrapper for lkl_sys_mkdirat
+ */
+static inline long lkl_sys_mkdir(const char *path, mode_t mode)
+{
+	return lkl_sys_mkdirat(LKL_AT_FDCWD, path, mode);
+}
+#endif
+
+#ifdef __lkl__NR_epoll_create1
+/**
+ * lkl_sys_epoll_create - wrapper for lkl_sys_epoll_create1
+ */
+static inline long lkl_sys_epoll_create(int size)
+{
+	return lkl_sys_epoll_create1(0);
+}
+#endif
+
+#ifdef __lkl__NR_epoll_pwait
+/**
+ * lkl_sys_epoll_wait - wrapper for lkl_sys_epoll_pwait
+ */
+static inline long lkl_sys_epoll_wait(int fd, struct lkl_epoll_event *ev,
+				      int cnt, int to)
+{
+	return lkl_sys_epoll_pwait(fd, ev, cnt, to, 0, _LKL_NSIG/8);
+}
+#endif
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif
diff --git a/tools/um/include/lkl_host.h b/tools/um/include/lkl_host.h
new file mode 100644
index 000000000000..85e80eb4ad0d
--- /dev/null
+++ b/tools/um/include/lkl_host.h
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_HOST_H
+#define _LKL_HOST_H
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include <lkl/asm/host_ops.h>
+#include <lkl.h>
+
+extern struct lkl_host_operations lkl_host_ops;
+
+/**
+ * lkl_printf - print a message via the host print operation
+ *
+ * @fmt: printf like format string
+ */
+int lkl_printf(const char *fmt, ...);
+
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif
diff --git a/tools/um/uml/Build b/tools/um/uml/Build
index 0d87d601bc06..a55c5cc3ebc0 100644
--- a/tools/um/uml/Build
+++ b/tools/um/uml/Build
@@ -8,11 +8,19 @@ KCOV_INSTRUMENT                := n
 
 include $(objtree)/include/config/auto.conf
 
-liblinux-y += execvp.o file.o helper.o irq.o main.o mem.o process.o \
-	registers.o sigio.o signal.o start_up.o time.o tty.o \
-	umid.o util.o drivers/ skas/
+liblinux-y += execvp.o file.o helper.o irq.o mem.o process.o \
+	registers.o sigio.o signal.o time.o tty.o \
+	umid.o util.o drivers/
 
+ifdef CONFIG_MMU
+liblinux-y += main.o start_up.o skas/
+HEADER_ARCH 	:= x86
 liblinux-y += x86/
+else
+HEADER_ARCH 	:= um/nommu
+liblinux-y += nommu/
+endif
+
 
 liblinux-$(CONFIG_ARCH_REUSE_HOST_VSYSCALL_AREA) += elf_aux.o
 
@@ -23,7 +31,6 @@ CFLAGS := -g -O2
 
 # from arch/um/Makefile
 ARCH_DIR := arch/um
-HEADER_ARCH 	:= x86
 HOST_DIR := arch/$(HEADER_ARCH)
 ifdef CONFIG_64BIT
   KBUILD_CFLAGS += -mcmodel=large
@@ -33,6 +40,7 @@ KBUILD_CFLAGS += $(CFLAGS) $(CFLAGS-y) -D__arch_um__ \
 	-Dlongjmp=kernel_longjmp -Dsetjmp=kernel_setjmp \
 	-Din6addr_loopback=kernel_in6addr_loopback \
 	-Din6addr_any=kernel_in6addr_any -Dstrrchr=kernel_strrchr
+KBUILD_CFLAGS := $(filter-out -Dsigprocmask=kernel_sigprocmask,$(KBUILD_CFLAGS))
 SHARED_HEADERS	:= $(ARCH_DIR)/include/shared
 MODE_INCLUDE	+= -I$(srctree)/$(ARCH_DIR)/include/shared/skas
 ARCH_INCLUDE	:= -I$(srctree)/$(SHARED_HEADERS)
diff --git a/tools/um/uml/nommu/Build b/tools/um/uml/nommu/Build
new file mode 100644
index 000000000000..98fd6b86a085
--- /dev/null
+++ b/tools/um/uml/nommu/Build
@@ -0,0 +1 @@
+liblinux-y = registers.o unimplemented.o
diff --git a/tools/um/uml/nommu/registers.c b/tools/um/uml/nommu/registers.c
new file mode 100644
index 000000000000..11573a204720
--- /dev/null
+++ b/tools/um/uml/nommu/registers.c
@@ -0,0 +1,21 @@
+// SPDX-License-Identifier: GPL-2.0
+
+struct uml_pt_regs;
+
+int get_fp_registers(int pid, unsigned long *regs)
+{
+	return 0;
+}
+
+int save_i387_registers(int pid, unsigned long *fp_regs)
+{
+	return 0;
+}
+
+void arch_init_registers(int pid)
+{
+}
+
+void get_regs_from_mc(struct uml_pt_regs *regs, void *mc)
+{
+}
diff --git a/tools/um/uml/nommu/unimplemented.c b/tools/um/uml/nommu/unimplemented.c
new file mode 100644
index 000000000000..9da3e5c8bafb
--- /dev/null
+++ b/tools/um/uml/nommu/unimplemented.c
@@ -0,0 +1,21 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <generated/user_constants.h>
+
+struct uml_pt_regs;
+
+/* os-Linux/skas/process.c */
+int userspace_pid[UM_NR_CPUS];
+void userspace(struct uml_pt_regs *regs, unsigned long *aux_fp_regs)
+{}
+
+
+/* x86/os-Linux/task_size.c */
+unsigned long os_get_top_address(void)
+{
+	return 0;
+}
+
+/* start-up.c */
+void os_early_checks(void)
+{
+}
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v7 17/21] um: host: add nommu build for ARCH=um
@ 2020-10-06  9:44           ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-06  9:44 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Hajime Tazaki, retrage01

This patch adds the skeleton for the host library.

The host library is implementing the host operations needed by nommu
mode and is split into host dependent (depends on a specific host, e.g.
POSIX hosts) and host independent parts (will work on all supported
hosts).

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/um/.gitignore                |   1 +
 tools/um/Makefile                  |  32 +++---
 tools/um/Targets                   |   3 +-
 tools/um/include/lkl.h             | 151 +++++++++++++++++++++++++++++
 tools/um/include/lkl_host.h        |  26 +++++
 tools/um/uml/Build                 |  16 ++-
 tools/um/uml/nommu/Build           |   1 +
 tools/um/uml/nommu/registers.c     |  21 ++++
 tools/um/uml/nommu/unimplemented.c |  21 ++++
 9 files changed, 252 insertions(+), 20 deletions(-)
 create mode 100644 tools/um/.gitignore
 create mode 100644 tools/um/include/lkl.h
 create mode 100644 tools/um/include/lkl_host.h
 create mode 100644 tools/um/uml/nommu/Build
 create mode 100644 tools/um/uml/nommu/registers.c
 create mode 100644 tools/um/uml/nommu/unimplemented.c

diff --git a/tools/um/.gitignore b/tools/um/.gitignore
new file mode 100644
index 000000000000..0d20b6487c61
--- /dev/null
+++ b/tools/um/.gitignore
@@ -0,0 +1 @@
+*.pyc
diff --git a/tools/um/Makefile b/tools/um/Makefile
index 552ed5f1edae..bbd5e0dcadbc 100644
--- a/tools/um/Makefile
+++ b/tools/um/Makefile
@@ -5,6 +5,7 @@
 MAKEFLAGS += -r --no-print-directory
 
 KCONFIG?=defconfig
+UMMODE?=kernel
 
 ifneq ($(silent),1)
   ifneq ($(V),1)
@@ -15,10 +16,8 @@ endif
 
 PREFIX   := /usr
 
-ifeq (,$(srctree))
-  srctree := $(patsubst %/,%,$(dir $(shell pwd)))
-  srctree := $(patsubst %/,%,$(dir $(srctree)))
-endif
+srctree := $(patsubst %/,%,$(dir $(shell pwd)))
+srctree := $(patsubst %/,%,$(dir $(srctree)))
 export srctree
 
 -include ../scripts/Makefile.include
@@ -32,9 +31,6 @@ endif
 export OUTPUT
 export objtree := $(OUTPUT)/../..
 
-
-all:
-
 export CFLAGS += -I$(OUTPUT)/include -Iinclude -Wall -g -O2 -Wextra -fPIC \
 	 -Wno-unused-parameter \
 	 -Wno-missing-field-initializers -fno-strict-aliasing
@@ -45,28 +41,34 @@ TARGETS := $(progs-y:%=$(OUTPUT)%)
 TARGETS += $(libs-y:%=$(OUTPUT)%)
 all: $(TARGETS)
 
-# rule to build linux.o
-$(OUTPUT)lib/linux.o:
-	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) $(KCONFIG)
-	$(Q)CFLAGS= $(MAKE) -C ../.. ARCH=um $(KOPT) install INSTALL_PATH=$(OUTPUT)
+$(objtree)/linux.o:
+	$(Q)echo ""
+	$(Q)echo ""
+	$(Q)echo "==> $@ isn't found; please make ARCH=um"
+	$(Q)echo ""
+	$(Q)echo ""
+	$(Q)exit 1
 
-$(OUTPUT)liblinux.a: $(OUTPUT)lib/linux.o $(OUTPUT)uml/liblinux-in.o $(OUTPUT)lib/liblinux-in.o
+$(OUTPUT)lib/linux.o: $(objtree)/linux.o
+	$(Q)cp $(objtree)/linux.o $(OUTPUT)lib/linux.o
+
+# rule to build linux.o
+$(OUTPUT)liblinux.a: $(OUTPUT)uml/liblinux-in.o $(OUTPUT)lib/liblinux-in.o $(OUTPUT)lib/linux.o
 	$(QUIET_AR)$(AR) -rc $@ $^
 
 # rule to link programs
 $(OUTPUT)%: $(OUTPUT)%-in.o $(OUTPUT)liblinux.a
 	$(QUIET_LINK)$(CC) $(LDFLAGS) $(LDFLAGS_$(notdir $*)-y) -o $@ $^ $(LDLIBS) $(LDLIBS_$(notdir $*)-y)
 
-$(OUTPUT)%-in.o: FORCE
-	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
-
 # rule to build objects
 $(OUTPUT)%-in.o: $(OUTPUT)lib/linux.o FORCE
 	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
 
+RM := rm -f
 clean:
 	$(call QUIET_CLEAN, objects)find $(OUTPUT) -name '*.o' -delete -o -name '\.*.cmd'\
 	 -delete -o -name '\.*.d' -delete
+	$(call QUIET_CLEAN, headers)$(RM) -r $(OUTPUT)/include/lkl/
 	$(call QUIET_CLEAN, liblinux.a)$(RM) $(OUTPUT)/liblinux.a
 	$(call QUIET_CLEAN, $(TARGETS))$(RM) $(TARGETS)
 
diff --git a/tools/um/Targets b/tools/um/Targets
index a4711f1ef422..4e1f0f4d81a3 100644
--- a/tools/um/Targets
+++ b/tools/um/Targets
@@ -1,4 +1,5 @@
 progs-y += uml/linux
-LDLIBS_linux-y := -lrt -lpthread -lutil
+
+LDLIBS := -lrt -lpthread -lutil
 LDFLAGS_linux-y := -no-pie -Wl,--wrap,malloc -Wl,--wrap,free -Wl,--wrap,calloc
 LDFLAGS_linux-$(UML_STATIC) += -static
diff --git a/tools/um/include/lkl.h b/tools/um/include/lkl.h
new file mode 100644
index 000000000000..707e01b64a70
--- /dev/null
+++ b/tools/um/include/lkl.h
@@ -0,0 +1,151 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_H
+#define _LKL_H
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#define _LKL_LIBC_COMPAT_H
+
+#ifdef __cplusplus
+#define class __lkl__class
+#endif
+
+/*
+ * Avoid collisions between Android which defines __unused and
+ * linux/icmp.h which uses __unused as a structure field.
+ */
+#pragma push_macro("__unused")
+#undef __unused
+
+#include <lkl/asm/syscalls.h>
+
+#pragma pop_macro("__unused")
+
+#ifdef __cplusplus
+#undef class
+#endif
+
+/**
+ * lkl_strerror - returns a string describing the given error code
+ *
+ * @err - error code
+ * @returns - string for the given error code
+ */
+const char *lkl_strerror(int err);
+
+/**
+ * lkl_perror - prints a string describing the given error code
+ *
+ * @msg - prefix for the error message
+ * @err - error code
+ */
+void lkl_perror(char *msg, int err);
+
+#if __LKL__BITS_PER_LONG == 64
+#define lkl_sys_fstatat lkl_sys_newfstatat
+#define lkl_sys_fstat lkl_sys_newfstat
+
+#else
+#define __lkl__NR_fcntl __lkl__NR_fcntl64
+
+#define lkl_stat lkl_stat64
+#define lkl_sys_stat lkl_sys_stat64
+#define lkl_sys_lstat lkl_sys_lstat64
+#define lkl_sys_truncate lkl_sys_truncate64
+#define lkl_sys_ftruncate lkl_sys_ftruncate64
+#define lkl_sys_sendfile lkl_sys_sendfile64
+#define lkl_sys_fstatat lkl_sys_fstatat64
+#define lkl_sys_fstat lkl_sys_fstat64
+#define lkl_sys_fcntl lkl_sys_fcntl64
+
+#define lkl_statfs lkl_statfs64
+
+static inline int lkl_sys_statfs(const char *path, struct lkl_statfs *buf)
+{
+	return lkl_sys_statfs64(path, sizeof(*buf), buf);
+}
+
+static inline int lkl_sys_fstatfs(unsigned int fd, struct lkl_statfs *buf)
+{
+	return lkl_sys_fstatfs64(fd, sizeof(*buf), buf);
+}
+
+#define lkl_sys_nanosleep lkl_sys_nanosleep_time32
+static inline int lkl_sys_nanosleep_time32(struct lkl_timespec *rqtp,
+					   struct lkl_timespec *rmtp)
+{
+	long p[6] = {(long)rqtp, (long)rmtp, 0, 0, 0, 0};
+
+	return lkl_syscall(__lkl__NR_nanosleep, p);
+}
+
+#endif
+
+static inline int lkl_sys_stat(const char *path, struct lkl_stat *buf)
+{
+	return lkl_sys_fstatat(LKL_AT_FDCWD, path, buf, 0);
+}
+
+static inline int lkl_sys_lstat(const char *path, struct lkl_stat *buf)
+{
+	return lkl_sys_fstatat(LKL_AT_FDCWD, path, buf,
+			       LKL_AT_SYMLINK_NOFOLLOW);
+}
+
+#ifdef __lkl__NR_openat
+/**
+ * lkl_sys_open - wrapper for lkl_sys_openat
+ */
+static inline long lkl_sys_open(const char *file, int flags, int mode)
+{
+	return lkl_sys_openat(LKL_AT_FDCWD, file, flags, mode);
+}
+
+/**
+ * lkl_sys_creat - wrapper for lkl_sys_openat
+ */
+static inline long lkl_sys_creat(const char *file, int mode)
+{
+	return lkl_sys_openat(LKL_AT_FDCWD, file,
+			      LKL_O_CREAT|LKL_O_WRONLY|LKL_O_TRUNC, mode);
+}
+#endif
+
+#ifdef __lkl__NR_mkdirat
+/**
+ * lkl_sys_mkdir - wrapper for lkl_sys_mkdirat
+ */
+static inline long lkl_sys_mkdir(const char *path, mode_t mode)
+{
+	return lkl_sys_mkdirat(LKL_AT_FDCWD, path, mode);
+}
+#endif
+
+#ifdef __lkl__NR_epoll_create1
+/**
+ * lkl_sys_epoll_create - wrapper for lkl_sys_epoll_create1
+ */
+static inline long lkl_sys_epoll_create(int size)
+{
+	return lkl_sys_epoll_create1(0);
+}
+#endif
+
+#ifdef __lkl__NR_epoll_pwait
+/**
+ * lkl_sys_epoll_wait - wrapper for lkl_sys_epoll_pwait
+ */
+static inline long lkl_sys_epoll_wait(int fd, struct lkl_epoll_event *ev,
+				      int cnt, int to)
+{
+	return lkl_sys_epoll_pwait(fd, ev, cnt, to, 0, _LKL_NSIG/8);
+}
+#endif
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif
diff --git a/tools/um/include/lkl_host.h b/tools/um/include/lkl_host.h
new file mode 100644
index 000000000000..85e80eb4ad0d
--- /dev/null
+++ b/tools/um/include/lkl_host.h
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_HOST_H
+#define _LKL_HOST_H
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include <lkl/asm/host_ops.h>
+#include <lkl.h>
+
+extern struct lkl_host_operations lkl_host_ops;
+
+/**
+ * lkl_printf - print a message via the host print operation
+ *
+ * @fmt: printf like format string
+ */
+int lkl_printf(const char *fmt, ...);
+
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif
diff --git a/tools/um/uml/Build b/tools/um/uml/Build
index 0d87d601bc06..a55c5cc3ebc0 100644
--- a/tools/um/uml/Build
+++ b/tools/um/uml/Build
@@ -8,11 +8,19 @@ KCOV_INSTRUMENT                := n
 
 include $(objtree)/include/config/auto.conf
 
-liblinux-y += execvp.o file.o helper.o irq.o main.o mem.o process.o \
-	registers.o sigio.o signal.o start_up.o time.o tty.o \
-	umid.o util.o drivers/ skas/
+liblinux-y += execvp.o file.o helper.o irq.o mem.o process.o \
+	registers.o sigio.o signal.o time.o tty.o \
+	umid.o util.o drivers/
 
+ifdef CONFIG_MMU
+liblinux-y += main.o start_up.o skas/
+HEADER_ARCH 	:= x86
 liblinux-y += x86/
+else
+HEADER_ARCH 	:= um/nommu
+liblinux-y += nommu/
+endif
+
 
 liblinux-$(CONFIG_ARCH_REUSE_HOST_VSYSCALL_AREA) += elf_aux.o
 
@@ -23,7 +31,6 @@ CFLAGS := -g -O2
 
 # from arch/um/Makefile
 ARCH_DIR := arch/um
-HEADER_ARCH 	:= x86
 HOST_DIR := arch/$(HEADER_ARCH)
 ifdef CONFIG_64BIT
   KBUILD_CFLAGS += -mcmodel=large
@@ -33,6 +40,7 @@ KBUILD_CFLAGS += $(CFLAGS) $(CFLAGS-y) -D__arch_um__ \
 	-Dlongjmp=kernel_longjmp -Dsetjmp=kernel_setjmp \
 	-Din6addr_loopback=kernel_in6addr_loopback \
 	-Din6addr_any=kernel_in6addr_any -Dstrrchr=kernel_strrchr
+KBUILD_CFLAGS := $(filter-out -Dsigprocmask=kernel_sigprocmask,$(KBUILD_CFLAGS))
 SHARED_HEADERS	:= $(ARCH_DIR)/include/shared
 MODE_INCLUDE	+= -I$(srctree)/$(ARCH_DIR)/include/shared/skas
 ARCH_INCLUDE	:= -I$(srctree)/$(SHARED_HEADERS)
diff --git a/tools/um/uml/nommu/Build b/tools/um/uml/nommu/Build
new file mode 100644
index 000000000000..98fd6b86a085
--- /dev/null
+++ b/tools/um/uml/nommu/Build
@@ -0,0 +1 @@
+liblinux-y = registers.o unimplemented.o
diff --git a/tools/um/uml/nommu/registers.c b/tools/um/uml/nommu/registers.c
new file mode 100644
index 000000000000..11573a204720
--- /dev/null
+++ b/tools/um/uml/nommu/registers.c
@@ -0,0 +1,21 @@
+// SPDX-License-Identifier: GPL-2.0
+
+struct uml_pt_regs;
+
+int get_fp_registers(int pid, unsigned long *regs)
+{
+	return 0;
+}
+
+int save_i387_registers(int pid, unsigned long *fp_regs)
+{
+	return 0;
+}
+
+void arch_init_registers(int pid)
+{
+}
+
+void get_regs_from_mc(struct uml_pt_regs *regs, void *mc)
+{
+}
diff --git a/tools/um/uml/nommu/unimplemented.c b/tools/um/uml/nommu/unimplemented.c
new file mode 100644
index 000000000000..9da3e5c8bafb
--- /dev/null
+++ b/tools/um/uml/nommu/unimplemented.c
@@ -0,0 +1,21 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <generated/user_constants.h>
+
+struct uml_pt_regs;
+
+/* os-Linux/skas/process.c */
+int userspace_pid[UM_NR_CPUS];
+void userspace(struct uml_pt_regs *regs, unsigned long *aux_fp_regs)
+{}
+
+
+/* x86/os-Linux/task_size.c */
+unsigned long os_get_top_address(void)
+{
+	return 0;
+}
+
+/* start-up.c */
+void os_early_checks(void)
+{
+}
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v7 18/21] um: host: add utilities functions
  2020-10-06  9:44         ` Hajime Tazaki
@ 2020-10-06  9:44           ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-06  9:44 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch, Hajime Tazaki

Add basic utility functions for getting a string from a kernel error
code and a fprintf like function that uses the host print
operation. The latter is useful for informing the user about errors
that occur in the host library.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 tools/um/lib/Build     |   4 +
 tools/um/lib/jmp_buf.c |  14 +++
 tools/um/lib/jmp_buf.h |   8 ++
 tools/um/lib/utils.c   | 210 +++++++++++++++++++++++++++++++++++++++++
 4 files changed, 236 insertions(+)
 create mode 100644 tools/um/lib/Build
 create mode 100644 tools/um/lib/jmp_buf.c
 create mode 100644 tools/um/lib/jmp_buf.h
 create mode 100644 tools/um/lib/utils.c

diff --git a/tools/um/lib/Build b/tools/um/lib/Build
new file mode 100644
index 000000000000..fc967af4104a
--- /dev/null
+++ b/tools/um/lib/Build
@@ -0,0 +1,4 @@
+include $(objtree)/include/config/auto.conf
+
+liblinux-$(CONFIG_UMMODE_LIB) += utils.o
+liblinux-$(CONFIG_UMMODE_LIB) += jmp_buf.o
diff --git a/tools/um/lib/jmp_buf.c b/tools/um/lib/jmp_buf.c
new file mode 100644
index 000000000000..f6bdd7e4bd83
--- /dev/null
+++ b/tools/um/lib/jmp_buf.c
@@ -0,0 +1,14 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <setjmp.h>
+#include <lkl_host.h>
+
+void jmp_buf_set(struct lkl_jmp_buf *jmpb, void (*f)(void))
+{
+	if (!setjmp(*((jmp_buf *)jmpb->buf)))
+		f();
+}
+
+void jmp_buf_longjmp(struct lkl_jmp_buf *jmpb, int val)
+{
+	longjmp(*((jmp_buf *)jmpb->buf), val);
+}
diff --git a/tools/um/lib/jmp_buf.h b/tools/um/lib/jmp_buf.h
new file mode 100644
index 000000000000..8782cbaaf51f
--- /dev/null
+++ b/tools/um/lib/jmp_buf.h
@@ -0,0 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_LIB_JMP_BUF_H
+#define _LKL_LIB_JMP_BUF_H
+
+void jmp_buf_set(struct lkl_jmp_buf *jmpb, void (*f)(void));
+void jmp_buf_longjmp(struct lkl_jmp_buf *jmpb, int val);
+
+#endif
diff --git a/tools/um/lib/utils.c b/tools/um/lib/utils.c
new file mode 100644
index 000000000000..4930479a8a35
--- /dev/null
+++ b/tools/um/lib/utils.c
@@ -0,0 +1,210 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdarg.h>
+#include <stdio.h>
+#include <string.h>
+#include <lkl_host.h>
+
+static const char * const lkl_err_strings[] = {
+	"Success",
+	"Operation not permitted",
+	"No such file or directory",
+	"No such process",
+	"Interrupted system call",
+	"I/O error",
+	"No such device or address",
+	"Argument list too long",
+	"Exec format error",
+	"Bad file number",
+	"No child processes",
+	"Try again",
+	"Out of memory",
+	"Permission denied",
+	"Bad address",
+	"Block device required",
+	"Device or resource busy",
+	"File exists",
+	"Cross-device link",
+	"No such device",
+	"Not a directory",
+	"Is a directory",
+	"Invalid argument",
+	"File table overflow",
+	"Too many open files",
+	"Not a typewriter",
+	"Text file busy",
+	"File too large",
+	"No space left on device",
+	"Illegal seek",
+	"Read-only file system",
+	"Too many links",
+	"Broken pipe",
+	"Math argument out of domain of func",
+	"Math result not representable",
+	"Resource deadlock would occur",
+	"File name too long",
+	"No record locks available",
+	"Invalid system call number",
+	"Directory not empty",
+	"Too many symbolic links encountered",
+	"Bad error code", /* EWOULDBLOCK is EAGAIN */
+	"No message of desired type",
+	"Identifier removed",
+	"Channel number out of range",
+	"Level 2 not synchronized",
+	"Level 3 halted",
+	"Level 3 reset",
+	"Link number out of range",
+	"Protocol driver not attached",
+	"No CSI structure available",
+	"Level 2 halted",
+	"Invalid exchange",
+	"Invalid request descriptor",
+	"Exchange full",
+	"No anode",
+	"Invalid request code",
+	"Invalid slot",
+	"Bad error code", /* EDEADLOCK is EDEADLK */
+	"Bad font file format",
+	"Device not a stream",
+	"No data available",
+	"Timer expired",
+	"Out of streams resources",
+	"Machine is not on the network",
+	"Package not installed",
+	"Object is remote",
+	"Link has been severed",
+	"Advertise error",
+	"Srmount error",
+	"Communication error on send",
+	"Protocol error",
+	"Multihop attempted",
+	"RFS specific error",
+	"Not a data message",
+	"Value too large for defined data type",
+	"Name not unique on network",
+	"File descriptor in bad state",
+	"Remote address changed",
+	"Can not access a needed shared library",
+	"Accessing a corrupted shared library",
+	".lib section in a.out corrupted",
+	"Attempting to link in too many shared libraries",
+	"Cannot exec a shared library directly",
+	"Illegal byte sequence",
+	"Interrupted system call should be restarted",
+	"Streams pipe error",
+	"Too many users",
+	"Socket operation on non-socket",
+	"Destination address required",
+	"Message too long",
+	"Protocol wrong type for socket",
+	"Protocol not available",
+	"Protocol not supported",
+	"Socket type not supported",
+	"Operation not supported on transport endpoint",
+	"Protocol family not supported",
+	"Address family not supported by protocol",
+	"Address already in use",
+	"Cannot assign requested address",
+	"Network is down",
+	"Network is unreachable",
+	"Network dropped connection because of reset",
+	"Software caused connection abort",
+	"Connection reset by peer",
+	"No buffer space available",
+	"Transport endpoint is already connected",
+	"Transport endpoint is not connected",
+	"Cannot send after transport endpoint shutdown",
+	"Too many references: cannot splice",
+	"Connection timed out",
+	"Connection refused",
+	"Host is down",
+	"No route to host",
+	"Operation already in progress",
+	"Operation now in progress",
+	"Stale file handle",
+	"Structure needs cleaning",
+	"Not a XENIX named type file",
+	"No XENIX semaphores available",
+	"Is a named type file",
+	"Remote I/O error",
+	"Quota exceeded",
+	"No medium found",
+	"Wrong medium type",
+	"Operation Canceled",
+	"Required key not available",
+	"Key has expired",
+	"Key has been revoked",
+	"Key was rejected by service",
+	"Owner died",
+	"State not recoverable",
+	"Operation not possible due to RF-kill",
+	"Memory page has hardware error",
+};
+
+const char *lkl_strerror(int err)
+{
+	if (err < 0)
+		err = -err;
+
+	if ((size_t)err >= sizeof(lkl_err_strings) / sizeof(const char *))
+		return "Bad error code";
+
+	return lkl_err_strings[err];
+}
+
+void lkl_perror(char *msg, int err)
+{
+	const char *err_msg = lkl_strerror(err);
+	/* We need to use 'real' printf because lkl_host_ops.print can
+	 * be turned off when debugging is off.
+	 */
+	lkl_printf("%s: %s\n", msg, err_msg);
+}
+
+static int lkl_vprintf(const char *fmt, va_list args)
+{
+	int n;
+	char *buffer;
+	va_list copy;
+
+	if (!lkl_host_ops.print)
+		return 0;
+
+	va_copy(copy, args);
+	n = vsnprintf(NULL, 0, fmt, copy);
+	va_end(copy);
+
+	buffer = lkl_host_ops.mem_alloc(n + 1);
+	if (!buffer)
+		return (-1);
+
+	vsnprintf(buffer, n + 1, fmt, args);
+
+	lkl_host_ops.print(buffer, n);
+	lkl_host_ops.mem_free(buffer);
+
+	return n;
+}
+
+int lkl_printf(const char *fmt, ...)
+{
+	int n;
+	va_list args;
+
+	va_start(args, fmt);
+	n = lkl_vprintf(fmt, args);
+	va_end(args);
+
+	return n;
+}
+
+void lkl_bug(const char *fmt, ...)
+{
+	va_list args;
+
+	va_start(args, fmt);
+	lkl_vprintf(fmt, args);
+	va_end(args);
+
+	lkl_host_ops.panic();
+}
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v7 18/21] um: host: add utilities functions
@ 2020-10-06  9:44           ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-06  9:44 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Hajime Tazaki, retrage01

Add basic utility functions for getting a string from a kernel error
code and a fprintf like function that uses the host print
operation. The latter is useful for informing the user about errors
that occur in the host library.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 tools/um/lib/Build     |   4 +
 tools/um/lib/jmp_buf.c |  14 +++
 tools/um/lib/jmp_buf.h |   8 ++
 tools/um/lib/utils.c   | 210 +++++++++++++++++++++++++++++++++++++++++
 4 files changed, 236 insertions(+)
 create mode 100644 tools/um/lib/Build
 create mode 100644 tools/um/lib/jmp_buf.c
 create mode 100644 tools/um/lib/jmp_buf.h
 create mode 100644 tools/um/lib/utils.c

diff --git a/tools/um/lib/Build b/tools/um/lib/Build
new file mode 100644
index 000000000000..fc967af4104a
--- /dev/null
+++ b/tools/um/lib/Build
@@ -0,0 +1,4 @@
+include $(objtree)/include/config/auto.conf
+
+liblinux-$(CONFIG_UMMODE_LIB) += utils.o
+liblinux-$(CONFIG_UMMODE_LIB) += jmp_buf.o
diff --git a/tools/um/lib/jmp_buf.c b/tools/um/lib/jmp_buf.c
new file mode 100644
index 000000000000..f6bdd7e4bd83
--- /dev/null
+++ b/tools/um/lib/jmp_buf.c
@@ -0,0 +1,14 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <setjmp.h>
+#include <lkl_host.h>
+
+void jmp_buf_set(struct lkl_jmp_buf *jmpb, void (*f)(void))
+{
+	if (!setjmp(*((jmp_buf *)jmpb->buf)))
+		f();
+}
+
+void jmp_buf_longjmp(struct lkl_jmp_buf *jmpb, int val)
+{
+	longjmp(*((jmp_buf *)jmpb->buf), val);
+}
diff --git a/tools/um/lib/jmp_buf.h b/tools/um/lib/jmp_buf.h
new file mode 100644
index 000000000000..8782cbaaf51f
--- /dev/null
+++ b/tools/um/lib/jmp_buf.h
@@ -0,0 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_LIB_JMP_BUF_H
+#define _LKL_LIB_JMP_BUF_H
+
+void jmp_buf_set(struct lkl_jmp_buf *jmpb, void (*f)(void));
+void jmp_buf_longjmp(struct lkl_jmp_buf *jmpb, int val);
+
+#endif
diff --git a/tools/um/lib/utils.c b/tools/um/lib/utils.c
new file mode 100644
index 000000000000..4930479a8a35
--- /dev/null
+++ b/tools/um/lib/utils.c
@@ -0,0 +1,210 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdarg.h>
+#include <stdio.h>
+#include <string.h>
+#include <lkl_host.h>
+
+static const char * const lkl_err_strings[] = {
+	"Success",
+	"Operation not permitted",
+	"No such file or directory",
+	"No such process",
+	"Interrupted system call",
+	"I/O error",
+	"No such device or address",
+	"Argument list too long",
+	"Exec format error",
+	"Bad file number",
+	"No child processes",
+	"Try again",
+	"Out of memory",
+	"Permission denied",
+	"Bad address",
+	"Block device required",
+	"Device or resource busy",
+	"File exists",
+	"Cross-device link",
+	"No such device",
+	"Not a directory",
+	"Is a directory",
+	"Invalid argument",
+	"File table overflow",
+	"Too many open files",
+	"Not a typewriter",
+	"Text file busy",
+	"File too large",
+	"No space left on device",
+	"Illegal seek",
+	"Read-only file system",
+	"Too many links",
+	"Broken pipe",
+	"Math argument out of domain of func",
+	"Math result not representable",
+	"Resource deadlock would occur",
+	"File name too long",
+	"No record locks available",
+	"Invalid system call number",
+	"Directory not empty",
+	"Too many symbolic links encountered",
+	"Bad error code", /* EWOULDBLOCK is EAGAIN */
+	"No message of desired type",
+	"Identifier removed",
+	"Channel number out of range",
+	"Level 2 not synchronized",
+	"Level 3 halted",
+	"Level 3 reset",
+	"Link number out of range",
+	"Protocol driver not attached",
+	"No CSI structure available",
+	"Level 2 halted",
+	"Invalid exchange",
+	"Invalid request descriptor",
+	"Exchange full",
+	"No anode",
+	"Invalid request code",
+	"Invalid slot",
+	"Bad error code", /* EDEADLOCK is EDEADLK */
+	"Bad font file format",
+	"Device not a stream",
+	"No data available",
+	"Timer expired",
+	"Out of streams resources",
+	"Machine is not on the network",
+	"Package not installed",
+	"Object is remote",
+	"Link has been severed",
+	"Advertise error",
+	"Srmount error",
+	"Communication error on send",
+	"Protocol error",
+	"Multihop attempted",
+	"RFS specific error",
+	"Not a data message",
+	"Value too large for defined data type",
+	"Name not unique on network",
+	"File descriptor in bad state",
+	"Remote address changed",
+	"Can not access a needed shared library",
+	"Accessing a corrupted shared library",
+	".lib section in a.out corrupted",
+	"Attempting to link in too many shared libraries",
+	"Cannot exec a shared library directly",
+	"Illegal byte sequence",
+	"Interrupted system call should be restarted",
+	"Streams pipe error",
+	"Too many users",
+	"Socket operation on non-socket",
+	"Destination address required",
+	"Message too long",
+	"Protocol wrong type for socket",
+	"Protocol not available",
+	"Protocol not supported",
+	"Socket type not supported",
+	"Operation not supported on transport endpoint",
+	"Protocol family not supported",
+	"Address family not supported by protocol",
+	"Address already in use",
+	"Cannot assign requested address",
+	"Network is down",
+	"Network is unreachable",
+	"Network dropped connection because of reset",
+	"Software caused connection abort",
+	"Connection reset by peer",
+	"No buffer space available",
+	"Transport endpoint is already connected",
+	"Transport endpoint is not connected",
+	"Cannot send after transport endpoint shutdown",
+	"Too many references: cannot splice",
+	"Connection timed out",
+	"Connection refused",
+	"Host is down",
+	"No route to host",
+	"Operation already in progress",
+	"Operation now in progress",
+	"Stale file handle",
+	"Structure needs cleaning",
+	"Not a XENIX named type file",
+	"No XENIX semaphores available",
+	"Is a named type file",
+	"Remote I/O error",
+	"Quota exceeded",
+	"No medium found",
+	"Wrong medium type",
+	"Operation Canceled",
+	"Required key not available",
+	"Key has expired",
+	"Key has been revoked",
+	"Key was rejected by service",
+	"Owner died",
+	"State not recoverable",
+	"Operation not possible due to RF-kill",
+	"Memory page has hardware error",
+};
+
+const char *lkl_strerror(int err)
+{
+	if (err < 0)
+		err = -err;
+
+	if ((size_t)err >= sizeof(lkl_err_strings) / sizeof(const char *))
+		return "Bad error code";
+
+	return lkl_err_strings[err];
+}
+
+void lkl_perror(char *msg, int err)
+{
+	const char *err_msg = lkl_strerror(err);
+	/* We need to use 'real' printf because lkl_host_ops.print can
+	 * be turned off when debugging is off.
+	 */
+	lkl_printf("%s: %s\n", msg, err_msg);
+}
+
+static int lkl_vprintf(const char *fmt, va_list args)
+{
+	int n;
+	char *buffer;
+	va_list copy;
+
+	if (!lkl_host_ops.print)
+		return 0;
+
+	va_copy(copy, args);
+	n = vsnprintf(NULL, 0, fmt, copy);
+	va_end(copy);
+
+	buffer = lkl_host_ops.mem_alloc(n + 1);
+	if (!buffer)
+		return (-1);
+
+	vsnprintf(buffer, n + 1, fmt, args);
+
+	lkl_host_ops.print(buffer, n);
+	lkl_host_ops.mem_free(buffer);
+
+	return n;
+}
+
+int lkl_printf(const char *fmt, ...)
+{
+	int n;
+	va_list args;
+
+	va_start(args, fmt);
+	n = lkl_vprintf(fmt, args);
+	va_end(args);
+
+	return n;
+}
+
+void lkl_bug(const char *fmt, ...)
+{
+	va_list args;
+
+	va_start(args, fmt);
+	lkl_vprintf(fmt, args);
+	va_end(args);
+
+	lkl_host_ops.panic();
+}
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v7 19/21] um: host: posix host operations
  2020-10-06  9:44         ` Hajime Tazaki
@ 2020-10-06  9:44           ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-06  9:44 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch, Hajime Tazaki

This commit implements LKL host operations for POSIX hosts.  The
operations are implemented to be portable in various POSIX OSs.

Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 tools/um/lib/Build        |   2 +
 tools/um/lib/posix-host.c | 292 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 294 insertions(+)
 create mode 100644 tools/um/lib/posix-host.c

diff --git a/tools/um/lib/Build b/tools/um/lib/Build
index fc967af4104a..dddff26a3b4e 100644
--- a/tools/um/lib/Build
+++ b/tools/um/lib/Build
@@ -1,4 +1,6 @@
 include $(objtree)/include/config/auto.conf
+CFLAGS_posix-host.o += -D_FILE_OFFSET_BITS=64
 
 liblinux-$(CONFIG_UMMODE_LIB) += utils.o
+liblinux-$(CONFIG_UMMODE_LIB) += posix-host.o
 liblinux-$(CONFIG_UMMODE_LIB) += jmp_buf.o
diff --git a/tools/um/lib/posix-host.c b/tools/um/lib/posix-host.c
new file mode 100644
index 000000000000..b6b5b2902254
--- /dev/null
+++ b/tools/um/lib/posix-host.c
@@ -0,0 +1,292 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <pthread.h>
+#include <limits.h>
+#include <errno.h>
+#include <string.h>
+#include <stdlib.h>
+#include <assert.h>
+#include <sys/syscall.h>
+#include <lkl_host.h>
+#include "jmp_buf.h"
+
+/* Let's see if the host has semaphore.h */
+#include <unistd.h>
+
+#ifdef _POSIX_SEMAPHORES
+#include <semaphore.h>
+/* TODO(pscollins): We don't support fork() for now, but maybe one day
+ * we will?
+ */
+#define SHARE_SEM 0
+#endif /* _POSIX_SEMAPHORES */
+
+
+struct lkl_mutex {
+	pthread_mutex_t mutex;
+};
+
+struct lkl_sem {
+#ifdef _POSIX_SEMAPHORES
+	sem_t sem;
+#else
+	pthread_mutex_t lock;
+	int count;
+	pthread_cond_t cond;
+#endif /* _POSIX_SEMAPHORES */
+};
+
+struct lkl_tls_key {
+	pthread_key_t key;
+};
+
+static void print(const char *str, int len)
+{
+	int ret __attribute__((unused));
+
+	ret = write(STDOUT_FILENO, str, len);
+}
+
+static void panic(void)
+{
+	assert(0);
+}
+
+#define WARN_UNLESS(exp) do {						\
+		if (exp < 0)						\
+			lkl_printf("%s: %s\n", #exp, strerror(errno));	\
+	} while (0)
+
+static int _warn_pthread(int ret, char *str_exp)
+{
+	if (ret > 0)
+		lkl_printf("%s: %s\n", str_exp, strerror(ret));
+
+	return ret;
+}
+
+
+/* pthread_* functions use the reverse convention */
+#define WARN_PTHREAD(exp) _warn_pthread(exp, #exp)
+
+static struct lkl_sem *sem_alloc(int count)
+{
+	struct lkl_sem *sem;
+
+	sem = malloc(sizeof(*sem));
+	if (!sem)
+		return NULL;
+
+#ifdef _POSIX_SEMAPHORES
+	if (sem_init(&sem->sem, SHARE_SEM, count) < 0) {
+		lkl_printf("sem_init: %s\n", strerror(errno));
+		free(sem);
+		return NULL;
+	}
+#else
+	pthread_mutex_init(&sem->lock, NULL);
+	sem->count = count;
+	WARN_PTHREAD(pthread_cond_init(&sem->cond, NULL));
+#endif /* _POSIX_SEMAPHORES */
+
+	return sem;
+}
+
+static void sem_free(struct lkl_sem *sem)
+{
+#ifdef _POSIX_SEMAPHORES
+	WARN_UNLESS(sem_destroy(&sem->sem));
+#else
+	WARN_PTHREAD(pthread_cond_destroy(&sem->cond));
+	WARN_PTHREAD(pthread_mutex_destroy(&sem->lock));
+#endif /* _POSIX_SEMAPHORES */
+	free(sem);
+}
+
+static void sem_up(struct lkl_sem *sem)
+{
+#ifdef _POSIX_SEMAPHORES
+	WARN_UNLESS(sem_post(&sem->sem));
+#else
+	WARN_PTHREAD(pthread_mutex_lock(&sem->lock));
+	sem->count++;
+	if (sem->count > 0)
+		WARN_PTHREAD(pthread_cond_signal(&sem->cond));
+	WARN_PTHREAD(pthread_mutex_unlock(&sem->lock));
+#endif /* _POSIX_SEMAPHORES */
+
+}
+
+static void sem_down(struct lkl_sem *sem)
+{
+#ifdef _POSIX_SEMAPHORES
+	int err;
+
+	do {
+		err = sem_wait(&sem->sem);
+	} while (err < 0 && errno == EINTR);
+	if (err < 0 && errno != EINTR)
+		lkl_printf("sem_wait: %s\n", strerror(errno));
+#else
+	WARN_PTHREAD(pthread_mutex_lock(&sem->lock));
+	while (sem->count <= 0)
+		WARN_PTHREAD(pthread_cond_wait(&sem->cond, &sem->lock));
+	sem->count--;
+	WARN_PTHREAD(pthread_mutex_unlock(&sem->lock));
+#endif /* _POSIX_SEMAPHORES */
+}
+
+static struct lkl_mutex *mutex_alloc(int recursive)
+{
+	struct lkl_mutex *_mutex = malloc(sizeof(struct lkl_mutex));
+	pthread_mutex_t *mutex = NULL;
+	pthread_mutexattr_t attr;
+
+	if (!_mutex)
+		return NULL;
+
+	mutex = &_mutex->mutex;
+	WARN_PTHREAD(pthread_mutexattr_init(&attr));
+
+	/* PTHREAD_MUTEX_ERRORCHECK is *very* useful for debugging,
+	 * but has some overhead, so we provide an option to turn it
+	 * off.
+	 */
+#ifdef DEBUG
+	if (!recursive)
+		WARN_PTHREAD(pthread_mutexattr_settype(
+				     &attr, PTHREAD_MUTEX_ERRORCHECK));
+#endif /* DEBUG */
+
+	if (recursive)
+		WARN_PTHREAD(pthread_mutexattr_settype(
+				     &attr, PTHREAD_MUTEX_RECURSIVE));
+
+	WARN_PTHREAD(pthread_mutex_init(mutex, &attr));
+
+	return _mutex;
+}
+
+static void mutex_lock(struct lkl_mutex *mutex)
+{
+	WARN_PTHREAD(pthread_mutex_lock(&mutex->mutex));
+}
+
+static void mutex_unlock(struct lkl_mutex *_mutex)
+{
+	pthread_mutex_t *mutex = &_mutex->mutex;
+
+	WARN_PTHREAD(pthread_mutex_unlock(mutex));
+}
+
+static void mutex_free(struct lkl_mutex *_mutex)
+{
+	pthread_mutex_t *mutex = &_mutex->mutex;
+
+	WARN_PTHREAD(pthread_mutex_destroy(mutex));
+	free(_mutex);
+}
+
+static lkl_thread_t thread_create(void* (*fn)(void *), void *arg)
+{
+	pthread_t thread;
+
+	if (WARN_PTHREAD(pthread_create(&thread, NULL, fn, arg)))
+		return 0;
+	else
+		return (lkl_thread_t) thread;
+}
+
+static void thread_detach(void)
+{
+	WARN_PTHREAD(pthread_detach(pthread_self()));
+}
+
+static void thread_exit(void)
+{
+	pthread_exit(NULL);
+}
+
+static int thread_join(lkl_thread_t tid)
+{
+	if (WARN_PTHREAD(pthread_join((pthread_t)tid, NULL)))
+		return (-1);
+	else
+		return 0;
+}
+
+static lkl_thread_t thread_self(void)
+{
+	return (lkl_thread_t)pthread_self();
+}
+
+static int thread_equal(lkl_thread_t a, lkl_thread_t b)
+{
+	return pthread_equal((pthread_t)a, (pthread_t)b);
+}
+
+static long _gettid(void)
+{
+#ifdef	__FreeBSD__
+	return (long)pthread_self();
+#else
+	return syscall(SYS_gettid);
+#endif
+}
+
+static struct lkl_tls_key *tls_alloc(void (*destructor)(void *))
+{
+	struct lkl_tls_key *ret = malloc(sizeof(struct lkl_tls_key));
+
+	if (WARN_PTHREAD(pthread_key_create(&ret->key, destructor))) {
+		free(ret);
+		return NULL;
+	}
+	return ret;
+}
+
+static void tls_free(struct lkl_tls_key *key)
+{
+	WARN_PTHREAD(pthread_key_delete(key->key));
+	free(key);
+}
+
+static int tls_set(struct lkl_tls_key *key, void *data)
+{
+	if (WARN_PTHREAD(pthread_setspecific(key->key, data)))
+		return (-1);
+	return 0;
+}
+
+static void *tls_get(struct lkl_tls_key *key)
+{
+	return pthread_getspecific(key->key);
+}
+
+struct lkl_host_operations lkl_host_ops = {
+	.panic = panic,
+	.print = print,
+	.mem_alloc = (void *)malloc,
+	.mem_free = free,
+	.thread_create = thread_create,
+	.thread_detach = thread_detach,
+	.thread_exit = thread_exit,
+	.thread_join = thread_join,
+	.thread_self = thread_self,
+	.thread_equal = thread_equal,
+	.sem_alloc = sem_alloc,
+	.sem_free = sem_free,
+	.sem_up = sem_up,
+	.sem_down = sem_down,
+	.mutex_alloc = mutex_alloc,
+	.mutex_free = mutex_free,
+	.mutex_lock = mutex_lock,
+	.mutex_unlock = mutex_unlock,
+	.gettid = _gettid,
+	.tls_alloc = tls_alloc,
+	.tls_free = tls_free,
+	.tls_set = tls_set,
+	.tls_get = tls_get,
+	.jmp_buf_set = jmp_buf_set,
+	.jmp_buf_longjmp = jmp_buf_longjmp,
+};
+
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v7 19/21] um: host: posix host operations
@ 2020-10-06  9:44           ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-06  9:44 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Hajime Tazaki, retrage01

This commit implements LKL host operations for POSIX hosts.  The
operations are implemented to be portable in various POSIX OSs.

Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 tools/um/lib/Build        |   2 +
 tools/um/lib/posix-host.c | 292 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 294 insertions(+)
 create mode 100644 tools/um/lib/posix-host.c

diff --git a/tools/um/lib/Build b/tools/um/lib/Build
index fc967af4104a..dddff26a3b4e 100644
--- a/tools/um/lib/Build
+++ b/tools/um/lib/Build
@@ -1,4 +1,6 @@
 include $(objtree)/include/config/auto.conf
+CFLAGS_posix-host.o += -D_FILE_OFFSET_BITS=64
 
 liblinux-$(CONFIG_UMMODE_LIB) += utils.o
+liblinux-$(CONFIG_UMMODE_LIB) += posix-host.o
 liblinux-$(CONFIG_UMMODE_LIB) += jmp_buf.o
diff --git a/tools/um/lib/posix-host.c b/tools/um/lib/posix-host.c
new file mode 100644
index 000000000000..b6b5b2902254
--- /dev/null
+++ b/tools/um/lib/posix-host.c
@@ -0,0 +1,292 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <pthread.h>
+#include <limits.h>
+#include <errno.h>
+#include <string.h>
+#include <stdlib.h>
+#include <assert.h>
+#include <sys/syscall.h>
+#include <lkl_host.h>
+#include "jmp_buf.h"
+
+/* Let's see if the host has semaphore.h */
+#include <unistd.h>
+
+#ifdef _POSIX_SEMAPHORES
+#include <semaphore.h>
+/* TODO(pscollins): We don't support fork() for now, but maybe one day
+ * we will?
+ */
+#define SHARE_SEM 0
+#endif /* _POSIX_SEMAPHORES */
+
+
+struct lkl_mutex {
+	pthread_mutex_t mutex;
+};
+
+struct lkl_sem {
+#ifdef _POSIX_SEMAPHORES
+	sem_t sem;
+#else
+	pthread_mutex_t lock;
+	int count;
+	pthread_cond_t cond;
+#endif /* _POSIX_SEMAPHORES */
+};
+
+struct lkl_tls_key {
+	pthread_key_t key;
+};
+
+static void print(const char *str, int len)
+{
+	int ret __attribute__((unused));
+
+	ret = write(STDOUT_FILENO, str, len);
+}
+
+static void panic(void)
+{
+	assert(0);
+}
+
+#define WARN_UNLESS(exp) do {						\
+		if (exp < 0)						\
+			lkl_printf("%s: %s\n", #exp, strerror(errno));	\
+	} while (0)
+
+static int _warn_pthread(int ret, char *str_exp)
+{
+	if (ret > 0)
+		lkl_printf("%s: %s\n", str_exp, strerror(ret));
+
+	return ret;
+}
+
+
+/* pthread_* functions use the reverse convention */
+#define WARN_PTHREAD(exp) _warn_pthread(exp, #exp)
+
+static struct lkl_sem *sem_alloc(int count)
+{
+	struct lkl_sem *sem;
+
+	sem = malloc(sizeof(*sem));
+	if (!sem)
+		return NULL;
+
+#ifdef _POSIX_SEMAPHORES
+	if (sem_init(&sem->sem, SHARE_SEM, count) < 0) {
+		lkl_printf("sem_init: %s\n", strerror(errno));
+		free(sem);
+		return NULL;
+	}
+#else
+	pthread_mutex_init(&sem->lock, NULL);
+	sem->count = count;
+	WARN_PTHREAD(pthread_cond_init(&sem->cond, NULL));
+#endif /* _POSIX_SEMAPHORES */
+
+	return sem;
+}
+
+static void sem_free(struct lkl_sem *sem)
+{
+#ifdef _POSIX_SEMAPHORES
+	WARN_UNLESS(sem_destroy(&sem->sem));
+#else
+	WARN_PTHREAD(pthread_cond_destroy(&sem->cond));
+	WARN_PTHREAD(pthread_mutex_destroy(&sem->lock));
+#endif /* _POSIX_SEMAPHORES */
+	free(sem);
+}
+
+static void sem_up(struct lkl_sem *sem)
+{
+#ifdef _POSIX_SEMAPHORES
+	WARN_UNLESS(sem_post(&sem->sem));
+#else
+	WARN_PTHREAD(pthread_mutex_lock(&sem->lock));
+	sem->count++;
+	if (sem->count > 0)
+		WARN_PTHREAD(pthread_cond_signal(&sem->cond));
+	WARN_PTHREAD(pthread_mutex_unlock(&sem->lock));
+#endif /* _POSIX_SEMAPHORES */
+
+}
+
+static void sem_down(struct lkl_sem *sem)
+{
+#ifdef _POSIX_SEMAPHORES
+	int err;
+
+	do {
+		err = sem_wait(&sem->sem);
+	} while (err < 0 && errno == EINTR);
+	if (err < 0 && errno != EINTR)
+		lkl_printf("sem_wait: %s\n", strerror(errno));
+#else
+	WARN_PTHREAD(pthread_mutex_lock(&sem->lock));
+	while (sem->count <= 0)
+		WARN_PTHREAD(pthread_cond_wait(&sem->cond, &sem->lock));
+	sem->count--;
+	WARN_PTHREAD(pthread_mutex_unlock(&sem->lock));
+#endif /* _POSIX_SEMAPHORES */
+}
+
+static struct lkl_mutex *mutex_alloc(int recursive)
+{
+	struct lkl_mutex *_mutex = malloc(sizeof(struct lkl_mutex));
+	pthread_mutex_t *mutex = NULL;
+	pthread_mutexattr_t attr;
+
+	if (!_mutex)
+		return NULL;
+
+	mutex = &_mutex->mutex;
+	WARN_PTHREAD(pthread_mutexattr_init(&attr));
+
+	/* PTHREAD_MUTEX_ERRORCHECK is *very* useful for debugging,
+	 * but has some overhead, so we provide an option to turn it
+	 * off.
+	 */
+#ifdef DEBUG
+	if (!recursive)
+		WARN_PTHREAD(pthread_mutexattr_settype(
+				     &attr, PTHREAD_MUTEX_ERRORCHECK));
+#endif /* DEBUG */
+
+	if (recursive)
+		WARN_PTHREAD(pthread_mutexattr_settype(
+				     &attr, PTHREAD_MUTEX_RECURSIVE));
+
+	WARN_PTHREAD(pthread_mutex_init(mutex, &attr));
+
+	return _mutex;
+}
+
+static void mutex_lock(struct lkl_mutex *mutex)
+{
+	WARN_PTHREAD(pthread_mutex_lock(&mutex->mutex));
+}
+
+static void mutex_unlock(struct lkl_mutex *_mutex)
+{
+	pthread_mutex_t *mutex = &_mutex->mutex;
+
+	WARN_PTHREAD(pthread_mutex_unlock(mutex));
+}
+
+static void mutex_free(struct lkl_mutex *_mutex)
+{
+	pthread_mutex_t *mutex = &_mutex->mutex;
+
+	WARN_PTHREAD(pthread_mutex_destroy(mutex));
+	free(_mutex);
+}
+
+static lkl_thread_t thread_create(void* (*fn)(void *), void *arg)
+{
+	pthread_t thread;
+
+	if (WARN_PTHREAD(pthread_create(&thread, NULL, fn, arg)))
+		return 0;
+	else
+		return (lkl_thread_t) thread;
+}
+
+static void thread_detach(void)
+{
+	WARN_PTHREAD(pthread_detach(pthread_self()));
+}
+
+static void thread_exit(void)
+{
+	pthread_exit(NULL);
+}
+
+static int thread_join(lkl_thread_t tid)
+{
+	if (WARN_PTHREAD(pthread_join((pthread_t)tid, NULL)))
+		return (-1);
+	else
+		return 0;
+}
+
+static lkl_thread_t thread_self(void)
+{
+	return (lkl_thread_t)pthread_self();
+}
+
+static int thread_equal(lkl_thread_t a, lkl_thread_t b)
+{
+	return pthread_equal((pthread_t)a, (pthread_t)b);
+}
+
+static long _gettid(void)
+{
+#ifdef	__FreeBSD__
+	return (long)pthread_self();
+#else
+	return syscall(SYS_gettid);
+#endif
+}
+
+static struct lkl_tls_key *tls_alloc(void (*destructor)(void *))
+{
+	struct lkl_tls_key *ret = malloc(sizeof(struct lkl_tls_key));
+
+	if (WARN_PTHREAD(pthread_key_create(&ret->key, destructor))) {
+		free(ret);
+		return NULL;
+	}
+	return ret;
+}
+
+static void tls_free(struct lkl_tls_key *key)
+{
+	WARN_PTHREAD(pthread_key_delete(key->key));
+	free(key);
+}
+
+static int tls_set(struct lkl_tls_key *key, void *data)
+{
+	if (WARN_PTHREAD(pthread_setspecific(key->key, data)))
+		return (-1);
+	return 0;
+}
+
+static void *tls_get(struct lkl_tls_key *key)
+{
+	return pthread_getspecific(key->key);
+}
+
+struct lkl_host_operations lkl_host_ops = {
+	.panic = panic,
+	.print = print,
+	.mem_alloc = (void *)malloc,
+	.mem_free = free,
+	.thread_create = thread_create,
+	.thread_detach = thread_detach,
+	.thread_exit = thread_exit,
+	.thread_join = thread_join,
+	.thread_self = thread_self,
+	.thread_equal = thread_equal,
+	.sem_alloc = sem_alloc,
+	.sem_free = sem_free,
+	.sem_up = sem_up,
+	.sem_down = sem_down,
+	.mutex_alloc = mutex_alloc,
+	.mutex_free = mutex_free,
+	.mutex_lock = mutex_lock,
+	.mutex_unlock = mutex_unlock,
+	.gettid = _gettid,
+	.tls_alloc = tls_alloc,
+	.tls_free = tls_free,
+	.tls_set = tls_set,
+	.tls_get = tls_get,
+	.jmp_buf_set = jmp_buf_set,
+	.jmp_buf_longjmp = jmp_buf_longjmp,
+};
+
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v7 20/21] um: host: add test programs
  2020-10-06  9:44         ` Hajime Tazaki
@ 2020-10-06  9:44           ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-06  9:44 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch, Hajime Tazaki

Add a simple LKL test application (boot) that starts the kernel and
performs simple tests that minimally exercise the LKL API.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/um/Makefile       |   5 +-
 tools/um/Targets        |   6 +
 tools/um/tests/Build    |   3 +
 tools/um/tests/boot.c   | 393 ++++++++++++++++++++++++++++++++++++++++
 tools/um/tests/boot.sh  |   9 +
 tools/um/tests/run.py   | 172 ++++++++++++++++++
 tools/um/tests/tap13.py | 209 +++++++++++++++++++++
 tools/um/tests/test.c   | 126 +++++++++++++
 tools/um/tests/test.h   |  72 ++++++++
 tools/um/tests/test.sh  | 181 ++++++++++++++++++
 10 files changed, 1175 insertions(+), 1 deletion(-)
 create mode 100644 tools/um/tests/Build
 create mode 100644 tools/um/tests/boot.c
 create mode 100755 tools/um/tests/boot.sh
 create mode 100755 tools/um/tests/run.py
 create mode 100644 tools/um/tests/tap13.py
 create mode 100644 tools/um/tests/test.c
 create mode 100644 tools/um/tests/test.h
 create mode 100644 tools/um/tests/test.sh

diff --git a/tools/um/Makefile b/tools/um/Makefile
index bbd5e0dcadbc..46d71b809bee 100644
--- a/tools/um/Makefile
+++ b/tools/um/Makefile
@@ -72,7 +72,10 @@ clean:
 	$(call QUIET_CLEAN, liblinux.a)$(RM) $(OUTPUT)/liblinux.a
 	$(call QUIET_CLEAN, $(TARGETS))$(RM) $(TARGETS)
 
+run-tests:
+	./tests/run.py $(tests)
+
 FORCE: ;
-.PHONY: all clean FORCE
+.PHONY: all clean FORCE run-tests
 .IGNORE: clean
 .NOTPARALLEL : lib/linux.o
diff --git a/tools/um/Targets b/tools/um/Targets
index 4e1f0f4d81a3..2bb90381843c 100644
--- a/tools/um/Targets
+++ b/tools/um/Targets
@@ -1,4 +1,10 @@
+ifeq ($(UMMODE),library)
+progs-y += tests/boot
+else
 progs-y += uml/linux
+endif
+
+LDFLAGS_boot-y := -pie
 
 LDLIBS := -lrt -lpthread -lutil
 LDFLAGS_linux-y := -no-pie -Wl,--wrap,malloc -Wl,--wrap,free -Wl,--wrap,calloc
diff --git a/tools/um/tests/Build b/tools/um/tests/Build
new file mode 100644
index 000000000000..564560486f98
--- /dev/null
+++ b/tools/um/tests/Build
@@ -0,0 +1,3 @@
+boot-y += boot.o test.o
+
+CFLAGS_test.o += -Wno-implicit-fallthrough
diff --git a/tools/um/tests/boot.c b/tools/um/tests/boot.c
new file mode 100644
index 000000000000..45b9a7276de7
--- /dev/null
+++ b/tools/um/tests/boot.c
@@ -0,0 +1,393 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <unistd.h>
+#include <string.h>
+#include <time.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <lkl.h>
+#include <lkl_host.h>
+
+#include <sys/stat.h>
+#include <fcntl.h>
+#if defined(__FreeBSD__)
+#include <net/if.h>
+#include <sys/ioctl.h>
+#elif __linux
+#include <sys/epoll.h>
+#include <sys/ioctl.h>
+#elif __MINGW32__
+#include <windows.h>
+#endif
+
+#include "test.h"
+
+#ifndef __MINGW32__
+#define sleep_ns 87654321
+int lkl_test_nanosleep(void)
+{
+	struct lkl_timespec ts = {
+		.tv_sec = 0,
+		.tv_nsec = sleep_ns,
+	};
+	struct timespec start, stop;
+	long delta;
+	long ret;
+
+	clock_gettime(CLOCK_MONOTONIC, &start);
+	ret = lkl_sys_nanosleep((struct __lkl__kernel_timespec *)&ts, NULL);
+	clock_gettime(CLOCK_MONOTONIC, &stop);
+
+	delta = 1e9*(stop.tv_sec - start.tv_sec) +
+		(stop.tv_nsec - start.tv_nsec);
+
+	lkl_test_logf("sleep %ld, expected sleep %d\n", delta, sleep_ns);
+
+	if (ret == 0 && delta > sleep_ns * 0.9)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+#endif
+
+LKL_TEST_CALL(getpid, lkl_sys_getpid, 1)
+
+void check_latency(long (*f)(void), long *min, long *max, long *avg)
+{
+	int i;
+	struct timespec start, stop;
+	unsigned long long sum = 0;
+	static const int count = 20;
+	long delta;
+
+	*min = 1000000000;
+	*max = -1;
+
+	for (i = 0; i < count; i++) {
+		clock_gettime(CLOCK_MONOTONIC, &start);
+		f();
+		clock_gettime(CLOCK_MONOTONIC, &stop);
+
+		delta = 1e9*(stop.tv_sec - start.tv_sec) +
+			(stop.tv_nsec - start.tv_nsec);
+
+		if (*min > delta)
+			*min = delta;
+		if (*max < delta)
+			*max = delta;
+		sum += delta;
+	}
+	*avg = sum / count;
+}
+
+static long native_getpid(void)
+{
+#ifdef __MINGW32__
+	GetCurrentProcessId();
+#else
+	getpid();
+#endif
+	return 0;
+}
+
+int lkl_test_syscall_latency(void)
+{
+	long min, max, avg;
+
+	lkl_test_logf("avg/min/max: ");
+
+	check_latency(lkl_sys_getpid, &min, &max, &avg);
+
+	lkl_test_logf("lkl:%ld/%ld/%ld ", avg, min, max);
+
+	check_latency(native_getpid, &min, &max, &avg);
+
+	lkl_test_logf("native:%ld/%ld/%ld\n", avg, min, max);
+
+	return TEST_SUCCESS;
+}
+
+#define access_rights 0721
+
+LKL_TEST_CALL(creat, lkl_sys_creat, 3, "/file", access_rights)
+LKL_TEST_CALL(close, lkl_sys_close, 0, 0);
+LKL_TEST_CALL(failopen, lkl_sys_open, -LKL_ENOENT, "/file2", 0, 0);
+LKL_TEST_CALL(umask, lkl_sys_umask, 022,  0777);
+LKL_TEST_CALL(umask2, lkl_sys_umask, 0777, 0);
+LKL_TEST_CALL(open, lkl_sys_open, 0, "/file", LKL_O_RDWR, 0);
+static const char wrbuf[] = "test";
+LKL_TEST_CALL(write, lkl_sys_write, sizeof(wrbuf), 0, wrbuf, sizeof(wrbuf));
+LKL_TEST_CALL(lseek_cur, lkl_sys_lseek, sizeof(wrbuf), 0, 0, LKL_SEEK_CUR);
+LKL_TEST_CALL(lseek_end, lkl_sys_lseek, sizeof(wrbuf), 0, 0, LKL_SEEK_END);
+LKL_TEST_CALL(lseek_set, lkl_sys_lseek, 0, 0, 0, LKL_SEEK_SET);
+
+int lkl_test_read(void)
+{
+	char buf[10] = { 0, };
+	long ret;
+
+	ret = lkl_sys_read(0, buf, sizeof(buf));
+
+	lkl_test_logf("lkl_sys_read=%ld buf=%s\n", ret, buf);
+
+	if (ret == sizeof(wrbuf) && !strcmp(wrbuf, buf))
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+int lkl_test_fstat(void)
+{
+	struct lkl_stat stat;
+	long ret;
+
+	ret = lkl_sys_fstat(0, (void *)&stat);
+
+	lkl_test_logf("lkl_sys_fstat=%ld mode=%o size=%ld\n", ret, stat.st_mode,
+		      stat.st_size);
+
+	if (ret == 0 && stat.st_size == sizeof(wrbuf) &&
+	    stat.st_mode == (access_rights | LKL_S_IFREG))
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+LKL_TEST_CALL(mkdir, lkl_sys_mkdir, 0, "/proc", access_rights)
+
+int lkl_test_stat(void)
+{
+	struct lkl_stat stat;
+	long ret;
+
+	ret = lkl_sys_stat("/proc", (void *)&stat);
+
+	lkl_test_logf("lkl_sys_stat(\"/proc\")=%ld mode=%o\n", ret,
+		      stat.st_mode);
+
+	if (ret == 0 && stat.st_mode == (access_rights | LKL_S_IFDIR))
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+static int lkl_test_pipe2(void)
+{
+	int pipe_fds[2];
+	int READ_IDX = 0, WRITE_IDX = 1;
+	static const char msg[] = "Hello world!";
+	char str[20];
+	int msg_len_bytes = strlen(msg) + 1;
+	int cmp_res;
+	long ret;
+
+	ret = lkl_sys_pipe2(pipe_fds, LKL_O_NONBLOCK);
+	if (ret) {
+		lkl_test_logf("pipe2: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_write(pipe_fds[WRITE_IDX], msg, msg_len_bytes);
+	if (ret != msg_len_bytes) {
+		if (ret < 0)
+			lkl_test_logf("write error: %s\n", lkl_strerror(ret));
+		else
+			lkl_test_logf("short write: %ld\n", ret);
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_read(pipe_fds[READ_IDX], str, msg_len_bytes);
+	if (ret != msg_len_bytes) {
+		if (ret < 0)
+			lkl_test_logf("read error: %s\n", lkl_strerror(ret));
+		else
+			lkl_test_logf("short read: %ld\n", ret);
+		return TEST_FAILURE;
+	}
+
+	cmp_res = memcmp(msg, str, msg_len_bytes);
+	if (cmp_res) {
+		lkl_test_logf("memcmp failed: %d\n", cmp_res);
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_close(pipe_fds[0]);
+	if (ret) {
+		lkl_test_logf("close error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_close(pipe_fds[1]);
+	if (ret) {
+		lkl_test_logf("close error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	return TEST_SUCCESS;
+}
+
+static int lkl_test_epoll(void)
+{
+	int epoll_fd, pipe_fds[2];
+	int READ_IDX = 0, WRITE_IDX = 1;
+	struct lkl_epoll_event wait_on, read_result;
+	static const char msg[] = "Hello world!";
+	long ret;
+
+	memset(&wait_on, 0, sizeof(wait_on));
+	memset(&read_result, 0, sizeof(read_result));
+
+	ret = lkl_sys_pipe2(pipe_fds, LKL_O_NONBLOCK);
+	if (ret) {
+		lkl_test_logf("pipe2 error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	epoll_fd = lkl_sys_epoll_create(1);
+	if (epoll_fd < 0) {
+		lkl_test_logf("epoll_create error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	wait_on.events = LKL_POLLIN | LKL_POLLOUT;
+	wait_on.data = pipe_fds[READ_IDX];
+
+	ret = lkl_sys_epoll_ctl(epoll_fd, LKL_EPOLL_CTL_ADD, pipe_fds[READ_IDX],
+				&wait_on);
+	if (ret < 0) {
+		lkl_test_logf("epoll_ctl error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	/* Shouldn't be ready before we have written something */
+	ret = lkl_sys_epoll_wait(epoll_fd, &read_result, 1, 0);
+	if (ret != 0) {
+		if (ret < 0)
+			lkl_test_logf("epoll_wait error: %s\n",
+				      lkl_strerror(ret));
+		else
+			lkl_test_logf("epoll_wait: bad event: 0x%lx\n", ret);
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_write(pipe_fds[WRITE_IDX], msg, strlen(msg) + 1);
+	if (ret < 0) {
+		lkl_test_logf("write error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	/* We expect exactly 1 fd to be ready immediately */
+	ret = lkl_sys_epoll_wait(epoll_fd, &read_result, 1, 0);
+	if (ret != 1) {
+		if (ret < 0)
+			lkl_test_logf("epoll_wait error: %s\n",
+				      lkl_strerror(ret));
+		else
+			lkl_test_logf("epoll_wait: bad ev no %ld\n", ret);
+		return TEST_FAILURE;
+	}
+
+	/* Already tested reading from pipe2 so no need to do it
+	 * here
+	 */
+
+	return TEST_SUCCESS;
+}
+
+LKL_TEST_CALL(chdir_proc, lkl_sys_chdir, 0, "proc");
+
+static int dir_fd;
+
+static int lkl_test_open_cwd(void)
+{
+	dir_fd = lkl_sys_open(".", LKL_O_RDONLY | LKL_O_DIRECTORY, 0);
+	if (dir_fd < 0) {
+		lkl_test_logf("failed to open current directory: %s\n",
+			      lkl_strerror(dir_fd));
+		return TEST_FAILURE;
+	}
+
+	return TEST_SUCCESS;
+}
+
+/* column where to insert a line break for the list file tests below. */
+#define COL_LINE_BREAK 70
+
+static int lkl_test_getdents64(void)
+{
+	long ret;
+	char buf[1024], *pos;
+	struct lkl_linux_dirent64 *de;
+	int wr;
+
+	de = (struct lkl_linux_dirent64 *)buf;
+	ret = lkl_sys_getdents64(dir_fd, de, sizeof(buf));
+
+	wr = lkl_test_logf("%d ", dir_fd);
+
+	if (ret < 0)
+		return TEST_FAILURE;
+
+	for (pos = buf; pos - buf < ret; pos += de->d_reclen) {
+		de = (struct lkl_linux_dirent64 *)pos;
+
+		wr += lkl_test_logf("%s ", de->d_name);
+		if (wr >= COL_LINE_BREAK) {
+			lkl_test_logf("\n");
+			wr = 0;
+		}
+	}
+
+	return TEST_SUCCESS;
+}
+
+LKL_TEST_CALL(close_dir_fd, lkl_sys_close, 0, dir_fd);
+LKL_TEST_CALL(chdir_root, lkl_sys_chdir, 0, "/");
+LKL_TEST_CALL(mount_fs_proc, lkl_sys_mount, 0, "none", "/proc", "proc", 0,
+	      NULL);
+LKL_TEST_CALL(umount_fs_proc, lkl_sys_umount, 0, "/proc", 0);
+
+LKL_TEST_CALL(start_kernel, lkl_start_kernel, 0, &lkl_host_ops,
+	     "mem=16M loglevel=8");
+LKL_TEST_CALL(stop_kernel, lkl_sys_halt, 0);
+
+struct lkl_test tests[] = {
+	LKL_TEST(start_kernel),
+	LKL_TEST(getpid),
+	LKL_TEST(syscall_latency),
+	LKL_TEST(umask),
+	LKL_TEST(umask2),
+	LKL_TEST(creat),
+	LKL_TEST(close),
+	LKL_TEST(failopen),
+	LKL_TEST(open),
+	LKL_TEST(write),
+	LKL_TEST(lseek_cur),
+	LKL_TEST(lseek_end),
+	LKL_TEST(lseek_set),
+	LKL_TEST(read),
+	LKL_TEST(fstat),
+	LKL_TEST(mkdir),
+	LKL_TEST(stat),
+#ifndef __MINGW32__
+	LKL_TEST(nanosleep),
+#endif
+	LKL_TEST(pipe2),
+	LKL_TEST(epoll),
+	LKL_TEST(mount_fs_proc),
+	LKL_TEST(chdir_proc),
+	LKL_TEST(open_cwd),
+	LKL_TEST(getdents64),
+	LKL_TEST(close_dir_fd),
+	LKL_TEST(chdir_root),
+	LKL_TEST(umount_fs_proc),
+	LKL_TEST(stop_kernel),
+};
+
+int main(int argc, const char **argv)
+{
+	lkl_host_ops.print = lkl_test_log;
+
+	return lkl_test_run(tests, sizeof(tests)/sizeof(struct lkl_test),
+			    "boot");
+}
diff --git a/tools/um/tests/boot.sh b/tools/um/tests/boot.sh
new file mode 100755
index 000000000000..d985c04b0ac1
--- /dev/null
+++ b/tools/um/tests/boot.sh
@@ -0,0 +1,9 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+source $script_dir/test.sh
+
+lkl_test_plan 1 "boot"
+lkl_test_run 1
+lkl_test_exec $script_dir/boot
diff --git a/tools/um/tests/run.py b/tools/um/tests/run.py
new file mode 100755
index 000000000000..c96ede90b6ad
--- /dev/null
+++ b/tools/um/tests/run.py
@@ -0,0 +1,172 @@
+#!/usr/bin/env python
+# SPDX-License-Identifier: GPL-2.0
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 2 of the License
+#
+# Author: Octavian Purdila <tavi@cs.pub.ro>
+#
+
+from __future__ import print_function
+
+import argparse
+import os
+import subprocess
+import sys
+import tap13
+import xml.etree.ElementTree as ET
+
+from junit_xml import TestSuite, TestCase
+
+
+class Reporter(tap13.Reporter):
+    def start(self, obj):
+        if type(obj) is tap13.Test:
+            if obj.result == "*":
+                end='\r'
+            else:
+                end='\n'
+            print("  TEST       %-8s %.50s" %
+                  (obj.result, obj.description + " " + obj.comment), end=end)
+
+        elif type(obj) is tap13.Suite:
+            if obj.tests_planned == 0:
+                status = "skip"
+            else:
+                status = ""
+            print("  SUITE      %-8s %s" % (status, obj.name))
+
+    def end(self, obj):
+        if type(obj) is tap13.Test:
+            if obj.result != "ok":
+                try:
+                    print(obj.yaml["log"], end='')
+                except:
+                    None
+
+
+mydir=os.path.dirname(os.path.realpath(__file__))
+
+tests = [
+    'boot.sh',
+]
+
+parser = argparse.ArgumentParser(description='LKL test runner')
+parser.add_argument('tests', nargs='?', action='append',
+                    help='tests to run %s' % tests)
+parser.add_argument('--junit-dir',
+                    help='directory where to store the juni suites')
+parser.add_argument('--gdb', action='store_true', default=False,
+                    help='run simple tests under gdb; implies --pass-through')
+parser.add_argument('--pass-through', action='store_true',  default=False,
+                    help='run the test without interpeting the test output')
+parser.add_argument('--valgrind', action='store_true', default=False,
+                    help='run simple tests under valgrind')
+
+args = parser.parse_args()
+if args.tests == [None]:
+    args.tests = tests
+
+if args.gdb:
+    args.pass_through=True
+    os.environ['GDB']="yes"
+
+if args.valgrind:
+    os.environ['VALGRIND']="yes"
+
+tap = tap13.Parser(Reporter())
+
+os.environ['PATH'] += ":" + mydir
+
+exit_code = 0
+
+for t in args.tests:
+    if not t:
+        continue
+    if args.pass_through:
+        print(t)
+        if subprocess.call(t, shell=True) != 0:
+            exit_code = 1
+    else:
+        p = subprocess.Popen(t, shell=True, stdout=subprocess.PIPE)
+        tap.parse(p.stdout)
+
+if args.pass_through:
+    sys.exit(exit_code)
+
+suites_count = 0
+tests_total = 0
+tests_not_ok = 0
+tests_ok = 0
+tests_skip = 0
+val_errs = 0
+val_fails = 0
+val_skips = 0
+
+for s in tap.run.suites:
+
+    junit_tests = []
+    suites_count += 1
+
+    for t in s.tests:
+        try:
+            secs = t.yaml["time_us"] / 1000000.0
+        except:
+            secs = 0
+        try:
+            log = t.yaml['log']
+        except:
+            log = ""
+
+        jt = TestCase(t.description, elapsed_sec=secs, stdout=log)
+        if t.result == 'skip':
+            jt.add_skipped_info(output=log)
+        elif t.result == 'not ok':
+            jt.add_error_info(output=log)
+
+        junit_tests.append(jt)
+
+        tests_total += 1
+        if t.result == "ok":
+            tests_ok += 1
+        elif t.result == "not ok":
+            tests_not_ok += 1
+            exit_code = 1
+        elif t.result == "skip":
+            tests_skip += 1
+
+    if args.junit_dir:
+        js = TestSuite(s.name, junit_tests)
+        with open(os.path.join(args.junit_dir, os.path.basename(s.name) + '.xml'), 'w') as f:
+            js.to_file(f, [js])
+
+        if os.getenv('VALGRIND') is not None:
+            val_xml = 'valgrind-%s.xml' % os.path.basename(s.name).replace(' ','-')
+            # skipped tests don't generate xml file
+            if os.path.exists(val_xml) is False:
+                continue
+
+            cmd = 'mv %s %s' % (val_xml, args.junit_dir)
+            subprocess.call(cmd, shell=True, )
+
+            cmd = mydir + '/valgrind2xunit.py ' + val_xml
+            subprocess.call(cmd, shell=True, cwd=args.junit_dir)
+
+            # count valgrind results
+            doc = ET.parse(os.path.join(args.junit_dir, 'valgrind-%s_xunit.xml' \
+                                        % (os.path.basename(s.name).replace(' ','-'))))
+            ts = doc.getroot()
+            val_errs += int(ts.get('errors'))
+            val_fails += int(ts.get('failures'))
+            val_skips += int(ts.get('skip'))
+
+print("Summary: %d suites run, %d tests, %d ok, %d not ok, %d skipped" %
+      (suites_count, tests_total, tests_ok, tests_not_ok, tests_skip))
+
+if os.getenv('VALGRIND') is not None:
+    print(" valgrind (memcheck): %d failures, %d skipped" % (val_fails, val_skips))
+    if val_errs or val_fails:
+        exit_code = 1
+
+sys.exit(exit_code)
diff --git a/tools/um/tests/tap13.py b/tools/um/tests/tap13.py
new file mode 100644
index 000000000000..65c73cda7ca1
--- /dev/null
+++ b/tools/um/tests/tap13.py
@@ -0,0 +1,209 @@
+#!/usr/bin/env python
+# SPDX-License-Identifier: GPL-2.0
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 2 of the License
+#
+# Author: Octavian Purdila <tavi@cs.pub.ro>
+#
+# Based on TAP13:
+#
+# Copyright 2013, Red Hat, Inc.
+# Author: Josef Skladanka <jskladan@redhat.com>
+#
+from __future__ import print_function
+
+import re
+import sys
+import yamlish
+
+
+class Reporter(object):
+
+    def start(self, obj):
+        None
+
+    def end(self, obj):
+        None
+
+
+class Test(object):
+    def __init__(self, reporter, result, id, description=None, directive=None,
+                 comment=None):
+        self.reporter = reporter
+        self.result = result
+        if directive:
+            self.result = directive.lower()
+        if id:
+            self.id = int(id)
+        else:
+            self.id = None
+        if description:
+            self.description = description
+        else:
+            self.description = ""
+        if comment:
+            self.comment = "# " + comment
+        else:
+            self.comment = ""
+        self.yaml = None
+        self._yaml_buffer = None
+        self.diagnostics = []
+
+        self.reporter.start(self)
+
+    def end(self):
+        if not self.yaml:
+            self.yaml = yamlish.load(self._yaml_buffer)
+            self.reporter.end(self)
+
+
+class Suite(object):
+    def __init__(self, reporter, start, end, explanation):
+        self.reporter = reporter
+        self.tests = []
+        self.name = explanation
+        self.tests_planned = int(end)
+
+        self.__tests_counter = 0
+        self.__tests_base = 0
+
+        self.reporter.start(self)
+
+    def newTest(self, args):
+        try:
+            self.tests[-1].end()
+        except IndexError:
+            None
+
+        if 'id' not in args or not args['id']:
+            args['id'] = self.__tests_counter
+        else:
+            args['id'] = int(args['id']) + self.__tests_base
+
+        if args['id'] < self.__tests_counter:
+            print("error: bad test id %d, fixing it" % (args['id']))
+            args['id'] = self.__tests_counter
+        # according to TAP13 specs, missing tests must be handled as 'not ok'
+        # here we add the missing tests in sequence
+        while args['id'] > (self.__tests_counter + 1):
+            comment = 'test %d not present' % self.__tests_counter
+            self.tests.append(Test(self.reporter, 'not ok',
+                                   self.__tests_counter, comment=comment))
+            self.__tests_counter += 1
+
+        if args['id'] == self.__tests_counter:
+            if args['directive']:
+                self.test().result = args['directive'].lower()
+            else:
+                self.test().result = args['result']
+            self.reporter.start(self.test())
+        else:
+            self.tests.append(Test(self.reporter, **args))
+            self.__tests_counter += 1
+
+    def test(self):
+        return self.tests[-1]
+
+    def end(self, name, planned):
+        if name == self.name:
+            self.tests_planned += int(planned)
+            self.__tests_base = self.__tests_counter
+            return False
+        try:
+            self.test().end()
+        except IndexError:
+            None
+        if len(self.tests) != self.tests_planned:
+            for i in range(len(self.tests), self.tests_planned):
+                self.tests.append(Test(self.reporter, 'not ok', i+1,
+                                       comment='test not present'))
+        return True
+
+
+class Run(object):
+
+    def __init__(self, reporter):
+        self.reporter = reporter
+        self.suites = []
+
+    def suite(self):
+        return self.suites[-1]
+
+    def test(self):
+        return self.suites[-1].tests[-1]
+
+    def newSuite(self, args):
+        new = False
+        try:
+            if self.suite().end(args['explanation'], args['end']):
+                new = True
+        except IndexError:
+            new = True
+        if new:
+            self.suites.append(Suite(self.reporter, **args))
+
+    def newTest(self, args):
+        self.suite().newTest(args)
+
+
+class Parser(object):
+    RE_PLAN = re.compile(r"^\s*(?P<start>\d+)\.\.(?P<end>\d+)\s*(#\s*(?P<explanation>.*))?\s*$")
+    RE_TEST_LINE = re.compile(r"^\s*(?P<result>(not\s+)?ok|[*]+)\s*(?P<id>\d+)?\s*(?P<description>[^#]+)?\s*(#\s*(?P<directive>TODO|SKIP)?\s*(?P<comment>.+)?)?\s*$",  re.IGNORECASE)
+    RE_EXPLANATION = re.compile(r"^\s*#\s*(?P<explanation>.+)?\s*$")
+    RE_YAMLISH_START = re.compile(r"^\s*---.*$")
+    RE_YAMLISH_END = re.compile(r"^\s*\.\.\.\s*$")
+
+    def __init__(self, reporter):
+        self.seek_test = False
+        self.in_test = False
+        self.in_yaml = False
+        self.run = Run(reporter)
+
+    def parse(self, source):
+        # to avoid input buffering
+        while True:
+            line = source.readline()
+            if not line:
+                break
+
+            if self.in_yaml:
+                if Parser.RE_YAMLISH_END.match(line):
+                    self.run.test()._yaml_buffer.append(line.strip())
+                    self.in_yaml = False
+                else:
+                    self.run.test()._yaml_buffer.append(line.rstrip())
+                continue
+
+            line = line.strip()
+
+            if self.in_test:
+                if Parser.RE_EXPLANATION.match(line):
+                    self.run.test().diagnostics.append(line)
+                    continue
+                if Parser.RE_YAMLISH_START.match(line):
+                    self.run.test()._yaml_buffer = [line.strip()]
+                    self.in_yaml = True
+                    continue
+
+            m = Parser.RE_PLAN.match(line)
+            if m:
+                self.seek_test = True
+                args = m.groupdict()
+                self.run.newSuite(args)
+                continue
+
+            if self.seek_test:
+                m = Parser.RE_TEST_LINE.match(line)
+                if m:
+                    args = m.groupdict()
+                    self.run.newTest(args)
+                    self.in_test = True
+                    continue
+
+            print(line)
+        try:
+            self.run.suite().end(None, 0)
+        except IndexError:
+            None
diff --git a/tools/um/tests/test.c b/tools/um/tests/test.c
new file mode 100644
index 000000000000..37ea21901f99
--- /dev/null
+++ b/tools/um/tests/test.c
@@ -0,0 +1,126 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <stdarg.h>
+#include <time.h>
+
+#include "test.h"
+
+/* circular log buffer */
+
+static char log_buf[0x10000];
+static char *head = log_buf, *tail = log_buf;
+
+static inline void advance(char **ptr)
+{
+	if ((unsigned int)(*ptr - log_buf) >= sizeof(log_buf))
+		*ptr = log_buf;
+	else
+		*ptr = *ptr + 1;
+}
+
+static void log_char(char c)
+{
+	*tail = c;
+	advance(&tail);
+	if (tail == head)
+		advance(&head);
+}
+
+static void print_log(void)
+{
+	char last;
+
+	printf(" log: |\n");
+	last = '\n';
+	while (head != tail) {
+		if (last == '\n')
+			printf("  ");
+		last = *head;
+		putchar(last);
+		advance(&head);
+	}
+	if (last != '\n')
+		putchar('\n');
+}
+
+int lkl_test_run(const struct lkl_test *tests, int nr, const char *fmt, ...)
+{
+	int i, ret, status = TEST_SUCCESS;
+	clock_t start, stop;
+	char name[1024];
+	va_list args;
+
+	va_start(args, fmt);
+	vsnprintf(name, sizeof(name), fmt, args);
+	va_end(args);
+
+	printf("1..%d # %s\n", nr, name);
+	for (i = 1; i <= nr; i++) {
+		const struct lkl_test *t = &tests[i-1];
+		unsigned long delta_us;
+
+		printf("* %d %s\n", i, t->name);
+		fflush(stdout);
+
+		start = clock();
+
+		ret = t->fn(t->arg1, t->arg2, t->arg3);
+
+		stop = clock();
+
+		switch (ret) {
+		case TEST_SUCCESS:
+			printf("ok %d %s\n", i, t->name);
+			break;
+		case TEST_SKIP:
+			printf("ok %d %s # SKIP\n", i, t->name);
+			break;
+		case TEST_BAILOUT:
+			status = TEST_BAILOUT;
+			/* fall through; */
+		case TEST_FAILURE:
+		default:
+			if (status != TEST_BAILOUT)
+				status = TEST_FAILURE;
+			printf("not ok %d %s\n", i, t->name);
+		}
+
+		printf(" ---\n");
+		delta_us = (stop - start) * 1000000 / CLOCKS_PER_SEC;
+		printf(" time_us: %ld\n", delta_us);
+		print_log();
+		printf(" ...\n");
+
+		if (status == TEST_BAILOUT) {
+			printf("Bail out!\n");
+			return TEST_FAILURE;
+		}
+
+		fflush(stdout);
+	}
+
+	return status;
+}
+
+
+void lkl_test_log(const char *str, int len)
+{
+	while (len--)
+		log_char(*(str++));
+}
+
+int lkl_test_logf(const char *fmt, ...)
+{
+	char tmp[1024], *c;
+	va_list args;
+	unsigned int n;
+
+	va_start(args, fmt);
+	n = vsnprintf(tmp, sizeof(tmp), fmt, args);
+	va_end(args);
+
+	for (c = tmp; *c != 0; c++)
+		log_char(*c);
+
+	return n > sizeof(tmp) ? sizeof(tmp) : n;
+}
diff --git a/tools/um/tests/test.h b/tools/um/tests/test.h
new file mode 100644
index 000000000000..f19bbc487a98
--- /dev/null
+++ b/tools/um/tests/test.h
@@ -0,0 +1,72 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_TEST_H
+#define _LKL_TEST_H
+
+#define TEST_SUCCESS	0
+#define TEST_FAILURE	1
+#define TEST_SKIP	2
+#define TEST_TODO	3
+#define TEST_BAILOUT	4
+
+struct lkl_test {
+	const char *name;
+	int (*fn)();
+	void *arg1, *arg2, *arg3;
+};
+
+/**
+ * Simple wrapper to initialize a test entry.
+ * @name - test name, it assume test function is named test_@name
+ * @vargs - arguments to be passed to the function
+ */
+#define LKL_TEST(name, ...) { #name, lkl_test_##name, __VA_ARGS__ }
+
+/**
+ * lkl_test_run - run a test suite
+ *
+ * @tests - the list of tests to run
+ * @nr - number of tests
+ * @fmt - format string to be used for suite name
+ */
+int lkl_test_run(const struct lkl_test *tests, int nr, const char *fmt, ...);
+
+/**
+ * lkl_test_log - store a string in the test log buffer
+ * @str - the string to log (can be non-NULL terminated)
+ * @len - the string length
+ */
+void lkl_test_log(const char *str, int len);
+
+/**
+ * lkl_test_logf - printf like function to store into the test log buffer
+ * @fmt - printf format string
+ * @vargs - arguments to the format string
+ */
+int lkl_test_logf(const char *fmt, ...) __attribute__((format(printf, 1, 2)));
+
+/**
+ * LKL_TEST_CALL - create a test function as for a LKL call
+ *
+ * The test function will be named lkl_test_@name and will return
+ * TEST_SUCCESS if the called functions returns @expect. Otherwise
+ * will return TEST_FAILUIRE.
+ *
+ * @name - test name; must be unique because it is part of the
+ * test function; the test function will be named
+ * @call - function to call
+ * @expect - expected return value for success
+ * @args - arguments to pass to the LKL call
+ */
+#define LKL_TEST_CALL(name, call, expect, ...)				\
+	static int lkl_test_##name(void)				\
+	{								\
+		long ret;						\
+									\
+		ret = call(__VA_ARGS__);				\
+		lkl_test_logf("%s(%s) = %ld %s\n", #call, #__VA_ARGS__, \
+			ret, ret < 0 ? lkl_strerror(ret) : "");		\
+		return (ret == expect) ? TEST_SUCCESS : TEST_FAILURE;	\
+	}
+
+
+#endif /* _LKL_TEST_H */
diff --git a/tools/um/tests/test.sh b/tools/um/tests/test.sh
new file mode 100644
index 000000000000..31b7de5c06f7
--- /dev/null
+++ b/tools/um/tests/test.sh
@@ -0,0 +1,181 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+basedir=$(cd $script_dir/..; pwd)
+base_objdir=$(cd ${OUTPUT}/; pwd)
+
+TEST_SUCCESS=0
+TEST_FAILURE=1
+TEST_SKIP=113
+TEST_TODO=114
+TEST_BAILOUT=115
+
+print_log()
+{
+    echo " log: |"
+    while read line; do
+        echo "  $line"
+    done < $1
+}
+
+export_vars()
+{
+    if [ -z "$var_file" ]; then
+        return
+    fi
+
+    for i in $@; do
+        echo "$i=${!i}" >> $var_file
+    done
+}
+
+lkl_test_run()
+{
+    log_file=$(mktemp)
+    export var_file=$(mktemp)
+
+    tid=$1 && shift && tname=$@
+
+    echo "* $tid $tname"
+
+    start=$(date '+%s%9N')
+    # run in a separate shell to avoid -e terminating us
+    $@ 2>&1 | strings >$log_file
+    exit=${PIPESTATUS[0]}
+    stop=$(date '+%s%9N')
+
+    case $exit in
+    $TEST_SUCCESS)
+        echo "ok $tid $tname"
+        ;;
+    $TEST_SKIP)
+        echo "ok $tid $tname # SKIP"
+        ;;
+    $TEST_BAILOUT)
+        echo "not ok $tid $tname"
+        echo "Bail out!"
+        ;;
+    $TEST_FAILURE|*)
+        echo "not ok $tid $tname"
+        ;;
+    esac
+
+    delta=$(((stop-start)/1000))
+
+    echo " ---"
+    echo " time_us: $delta"
+    print_log $log_file
+    echo -e " ..."
+
+    rm $log_file
+    . $var_file
+    rm $var_file
+
+    return $exit
+}
+
+lkl_test_plan()
+{
+    echo "1..$1 # $2"
+    export suite_name="${2// /\-}"
+}
+
+lkl_test_exec()
+{
+    local SUDO=""
+    local WRAPPER=""
+
+    if [ "$1" = "sudo" ]; then
+        SUDO=sudo
+        shift
+    fi
+
+    local file=$1
+    shift
+
+    if [ -n "$LKL_HOST_CONFIG_NT" ]; then
+        file=$file.exe
+    fi
+
+    file=${OUTPUT}/tests/$(basename $file)
+
+    if file $file | grep ARM; then
+        WRAPPER="qemu-arm-static"
+    elif file $file | grep "FreeBSD" ; then
+        ssh_copy "$file" $BSD_WDIR
+        if [ -n "$SUDO" ]; then
+            SUDO=""
+        fi
+        WRAPPER="$MYSSH $SU"
+        # ssh will mess up with pipes ('|') so, escape the pipe char.
+        args="${@//\|/\\\|}"
+        set - $BSD_WDIR/$(basename $file) $args
+        file=""
+    elif [ -n "$GDB" ]; then
+        WRAPPER="gdb"
+        args="$@"
+        set - -ex "run $args" -ex quit $file
+        file=""
+    elif [ -n "$VALGRIND" ]; then
+        WRAPPER="valgrind --suppressions=$script_dir/valgrind.supp \
+                  --leak-check=full --show-leak-kinds=all --xml=yes \
+                  --xml-file=valgrind-$suite_name.xml"
+    fi
+
+    $SUDO $WRAPPER $file "$@"
+}
+
+lkl_test_cmd()
+{
+    local WRAPPER=""
+
+    if [ -z "$QUIET" ]; then
+        SHOPTS="-x"
+    fi
+
+    if [ -n "$LKL_HOST_CONFIG_BSD" ]; then
+        WRAPPER="$MYSSH $SU"
+    fi
+
+    echo "$@" | $WRAPPER sh $SHOPTS
+}
+
+# XXX: $MYSSH and $MYSCP are defined in a circleci docker image.
+# see the definitions in lkl/lkl-docker:circleci/freebsd11/Dockerfile
+ssh_push()
+{
+    while [ -n "$1" ]; do
+        if [[ "$1" = *.sh ]]; then
+            type="script"
+        else
+            type="file"
+        fi
+
+        dir=$(dirname $1)
+        $MYSSH mkdir -p $BSD_WDIR/$dir
+
+        $MYSCP -P 7722 -r $basedir/$1 root@localhost:$BSD_WDIR/$dir
+        if [ "$type" = "script" ]; then
+            $MYSSH chmod a+x $BSD_WDIR/$1
+        fi
+
+        shift
+    done
+}
+
+ssh_copy()
+{
+    $MYSCP -P 7722 -r $1 root@localhost:$2
+}
+
+lkl_test_bsd_cleanup()
+{
+    $MYSSH rm -rf $BSD_WDIR
+}
+
+if [ -n "$LKL_HOST_CONFIG_BSD" ]; then
+    trap lkl_test_bsd_cleanup EXIT
+    export BSD_WDIR=/root/lkl
+    $MYSSH mkdir -p $BSD_WDIR
+fi
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v7 20/21] um: host: add test programs
@ 2020-10-06  9:44           ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-06  9:44 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Hajime Tazaki, retrage01

Add a simple LKL test application (boot) that starts the kernel and
performs simple tests that minimally exercise the LKL API.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/um/Makefile       |   5 +-
 tools/um/Targets        |   6 +
 tools/um/tests/Build    |   3 +
 tools/um/tests/boot.c   | 393 ++++++++++++++++++++++++++++++++++++++++
 tools/um/tests/boot.sh  |   9 +
 tools/um/tests/run.py   | 172 ++++++++++++++++++
 tools/um/tests/tap13.py | 209 +++++++++++++++++++++
 tools/um/tests/test.c   | 126 +++++++++++++
 tools/um/tests/test.h   |  72 ++++++++
 tools/um/tests/test.sh  | 181 ++++++++++++++++++
 10 files changed, 1175 insertions(+), 1 deletion(-)
 create mode 100644 tools/um/tests/Build
 create mode 100644 tools/um/tests/boot.c
 create mode 100755 tools/um/tests/boot.sh
 create mode 100755 tools/um/tests/run.py
 create mode 100644 tools/um/tests/tap13.py
 create mode 100644 tools/um/tests/test.c
 create mode 100644 tools/um/tests/test.h
 create mode 100644 tools/um/tests/test.sh

diff --git a/tools/um/Makefile b/tools/um/Makefile
index bbd5e0dcadbc..46d71b809bee 100644
--- a/tools/um/Makefile
+++ b/tools/um/Makefile
@@ -72,7 +72,10 @@ clean:
 	$(call QUIET_CLEAN, liblinux.a)$(RM) $(OUTPUT)/liblinux.a
 	$(call QUIET_CLEAN, $(TARGETS))$(RM) $(TARGETS)
 
+run-tests:
+	./tests/run.py $(tests)
+
 FORCE: ;
-.PHONY: all clean FORCE
+.PHONY: all clean FORCE run-tests
 .IGNORE: clean
 .NOTPARALLEL : lib/linux.o
diff --git a/tools/um/Targets b/tools/um/Targets
index 4e1f0f4d81a3..2bb90381843c 100644
--- a/tools/um/Targets
+++ b/tools/um/Targets
@@ -1,4 +1,10 @@
+ifeq ($(UMMODE),library)
+progs-y += tests/boot
+else
 progs-y += uml/linux
+endif
+
+LDFLAGS_boot-y := -pie
 
 LDLIBS := -lrt -lpthread -lutil
 LDFLAGS_linux-y := -no-pie -Wl,--wrap,malloc -Wl,--wrap,free -Wl,--wrap,calloc
diff --git a/tools/um/tests/Build b/tools/um/tests/Build
new file mode 100644
index 000000000000..564560486f98
--- /dev/null
+++ b/tools/um/tests/Build
@@ -0,0 +1,3 @@
+boot-y += boot.o test.o
+
+CFLAGS_test.o += -Wno-implicit-fallthrough
diff --git a/tools/um/tests/boot.c b/tools/um/tests/boot.c
new file mode 100644
index 000000000000..45b9a7276de7
--- /dev/null
+++ b/tools/um/tests/boot.c
@@ -0,0 +1,393 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <unistd.h>
+#include <string.h>
+#include <time.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <lkl.h>
+#include <lkl_host.h>
+
+#include <sys/stat.h>
+#include <fcntl.h>
+#if defined(__FreeBSD__)
+#include <net/if.h>
+#include <sys/ioctl.h>
+#elif __linux
+#include <sys/epoll.h>
+#include <sys/ioctl.h>
+#elif __MINGW32__
+#include <windows.h>
+#endif
+
+#include "test.h"
+
+#ifndef __MINGW32__
+#define sleep_ns 87654321
+int lkl_test_nanosleep(void)
+{
+	struct lkl_timespec ts = {
+		.tv_sec = 0,
+		.tv_nsec = sleep_ns,
+	};
+	struct timespec start, stop;
+	long delta;
+	long ret;
+
+	clock_gettime(CLOCK_MONOTONIC, &start);
+	ret = lkl_sys_nanosleep((struct __lkl__kernel_timespec *)&ts, NULL);
+	clock_gettime(CLOCK_MONOTONIC, &stop);
+
+	delta = 1e9*(stop.tv_sec - start.tv_sec) +
+		(stop.tv_nsec - start.tv_nsec);
+
+	lkl_test_logf("sleep %ld, expected sleep %d\n", delta, sleep_ns);
+
+	if (ret == 0 && delta > sleep_ns * 0.9)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+#endif
+
+LKL_TEST_CALL(getpid, lkl_sys_getpid, 1)
+
+void check_latency(long (*f)(void), long *min, long *max, long *avg)
+{
+	int i;
+	struct timespec start, stop;
+	unsigned long long sum = 0;
+	static const int count = 20;
+	long delta;
+
+	*min = 1000000000;
+	*max = -1;
+
+	for (i = 0; i < count; i++) {
+		clock_gettime(CLOCK_MONOTONIC, &start);
+		f();
+		clock_gettime(CLOCK_MONOTONIC, &stop);
+
+		delta = 1e9*(stop.tv_sec - start.tv_sec) +
+			(stop.tv_nsec - start.tv_nsec);
+
+		if (*min > delta)
+			*min = delta;
+		if (*max < delta)
+			*max = delta;
+		sum += delta;
+	}
+	*avg = sum / count;
+}
+
+static long native_getpid(void)
+{
+#ifdef __MINGW32__
+	GetCurrentProcessId();
+#else
+	getpid();
+#endif
+	return 0;
+}
+
+int lkl_test_syscall_latency(void)
+{
+	long min, max, avg;
+
+	lkl_test_logf("avg/min/max: ");
+
+	check_latency(lkl_sys_getpid, &min, &max, &avg);
+
+	lkl_test_logf("lkl:%ld/%ld/%ld ", avg, min, max);
+
+	check_latency(native_getpid, &min, &max, &avg);
+
+	lkl_test_logf("native:%ld/%ld/%ld\n", avg, min, max);
+
+	return TEST_SUCCESS;
+}
+
+#define access_rights 0721
+
+LKL_TEST_CALL(creat, lkl_sys_creat, 3, "/file", access_rights)
+LKL_TEST_CALL(close, lkl_sys_close, 0, 0);
+LKL_TEST_CALL(failopen, lkl_sys_open, -LKL_ENOENT, "/file2", 0, 0);
+LKL_TEST_CALL(umask, lkl_sys_umask, 022,  0777);
+LKL_TEST_CALL(umask2, lkl_sys_umask, 0777, 0);
+LKL_TEST_CALL(open, lkl_sys_open, 0, "/file", LKL_O_RDWR, 0);
+static const char wrbuf[] = "test";
+LKL_TEST_CALL(write, lkl_sys_write, sizeof(wrbuf), 0, wrbuf, sizeof(wrbuf));
+LKL_TEST_CALL(lseek_cur, lkl_sys_lseek, sizeof(wrbuf), 0, 0, LKL_SEEK_CUR);
+LKL_TEST_CALL(lseek_end, lkl_sys_lseek, sizeof(wrbuf), 0, 0, LKL_SEEK_END);
+LKL_TEST_CALL(lseek_set, lkl_sys_lseek, 0, 0, 0, LKL_SEEK_SET);
+
+int lkl_test_read(void)
+{
+	char buf[10] = { 0, };
+	long ret;
+
+	ret = lkl_sys_read(0, buf, sizeof(buf));
+
+	lkl_test_logf("lkl_sys_read=%ld buf=%s\n", ret, buf);
+
+	if (ret == sizeof(wrbuf) && !strcmp(wrbuf, buf))
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+int lkl_test_fstat(void)
+{
+	struct lkl_stat stat;
+	long ret;
+
+	ret = lkl_sys_fstat(0, (void *)&stat);
+
+	lkl_test_logf("lkl_sys_fstat=%ld mode=%o size=%ld\n", ret, stat.st_mode,
+		      stat.st_size);
+
+	if (ret == 0 && stat.st_size == sizeof(wrbuf) &&
+	    stat.st_mode == (access_rights | LKL_S_IFREG))
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+LKL_TEST_CALL(mkdir, lkl_sys_mkdir, 0, "/proc", access_rights)
+
+int lkl_test_stat(void)
+{
+	struct lkl_stat stat;
+	long ret;
+
+	ret = lkl_sys_stat("/proc", (void *)&stat);
+
+	lkl_test_logf("lkl_sys_stat(\"/proc\")=%ld mode=%o\n", ret,
+		      stat.st_mode);
+
+	if (ret == 0 && stat.st_mode == (access_rights | LKL_S_IFDIR))
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+static int lkl_test_pipe2(void)
+{
+	int pipe_fds[2];
+	int READ_IDX = 0, WRITE_IDX = 1;
+	static const char msg[] = "Hello world!";
+	char str[20];
+	int msg_len_bytes = strlen(msg) + 1;
+	int cmp_res;
+	long ret;
+
+	ret = lkl_sys_pipe2(pipe_fds, LKL_O_NONBLOCK);
+	if (ret) {
+		lkl_test_logf("pipe2: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_write(pipe_fds[WRITE_IDX], msg, msg_len_bytes);
+	if (ret != msg_len_bytes) {
+		if (ret < 0)
+			lkl_test_logf("write error: %s\n", lkl_strerror(ret));
+		else
+			lkl_test_logf("short write: %ld\n", ret);
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_read(pipe_fds[READ_IDX], str, msg_len_bytes);
+	if (ret != msg_len_bytes) {
+		if (ret < 0)
+			lkl_test_logf("read error: %s\n", lkl_strerror(ret));
+		else
+			lkl_test_logf("short read: %ld\n", ret);
+		return TEST_FAILURE;
+	}
+
+	cmp_res = memcmp(msg, str, msg_len_bytes);
+	if (cmp_res) {
+		lkl_test_logf("memcmp failed: %d\n", cmp_res);
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_close(pipe_fds[0]);
+	if (ret) {
+		lkl_test_logf("close error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_close(pipe_fds[1]);
+	if (ret) {
+		lkl_test_logf("close error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	return TEST_SUCCESS;
+}
+
+static int lkl_test_epoll(void)
+{
+	int epoll_fd, pipe_fds[2];
+	int READ_IDX = 0, WRITE_IDX = 1;
+	struct lkl_epoll_event wait_on, read_result;
+	static const char msg[] = "Hello world!";
+	long ret;
+
+	memset(&wait_on, 0, sizeof(wait_on));
+	memset(&read_result, 0, sizeof(read_result));
+
+	ret = lkl_sys_pipe2(pipe_fds, LKL_O_NONBLOCK);
+	if (ret) {
+		lkl_test_logf("pipe2 error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	epoll_fd = lkl_sys_epoll_create(1);
+	if (epoll_fd < 0) {
+		lkl_test_logf("epoll_create error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	wait_on.events = LKL_POLLIN | LKL_POLLOUT;
+	wait_on.data = pipe_fds[READ_IDX];
+
+	ret = lkl_sys_epoll_ctl(epoll_fd, LKL_EPOLL_CTL_ADD, pipe_fds[READ_IDX],
+				&wait_on);
+	if (ret < 0) {
+		lkl_test_logf("epoll_ctl error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	/* Shouldn't be ready before we have written something */
+	ret = lkl_sys_epoll_wait(epoll_fd, &read_result, 1, 0);
+	if (ret != 0) {
+		if (ret < 0)
+			lkl_test_logf("epoll_wait error: %s\n",
+				      lkl_strerror(ret));
+		else
+			lkl_test_logf("epoll_wait: bad event: 0x%lx\n", ret);
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_write(pipe_fds[WRITE_IDX], msg, strlen(msg) + 1);
+	if (ret < 0) {
+		lkl_test_logf("write error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	/* We expect exactly 1 fd to be ready immediately */
+	ret = lkl_sys_epoll_wait(epoll_fd, &read_result, 1, 0);
+	if (ret != 1) {
+		if (ret < 0)
+			lkl_test_logf("epoll_wait error: %s\n",
+				      lkl_strerror(ret));
+		else
+			lkl_test_logf("epoll_wait: bad ev no %ld\n", ret);
+		return TEST_FAILURE;
+	}
+
+	/* Already tested reading from pipe2 so no need to do it
+	 * here
+	 */
+
+	return TEST_SUCCESS;
+}
+
+LKL_TEST_CALL(chdir_proc, lkl_sys_chdir, 0, "proc");
+
+static int dir_fd;
+
+static int lkl_test_open_cwd(void)
+{
+	dir_fd = lkl_sys_open(".", LKL_O_RDONLY | LKL_O_DIRECTORY, 0);
+	if (dir_fd < 0) {
+		lkl_test_logf("failed to open current directory: %s\n",
+			      lkl_strerror(dir_fd));
+		return TEST_FAILURE;
+	}
+
+	return TEST_SUCCESS;
+}
+
+/* column where to insert a line break for the list file tests below. */
+#define COL_LINE_BREAK 70
+
+static int lkl_test_getdents64(void)
+{
+	long ret;
+	char buf[1024], *pos;
+	struct lkl_linux_dirent64 *de;
+	int wr;
+
+	de = (struct lkl_linux_dirent64 *)buf;
+	ret = lkl_sys_getdents64(dir_fd, de, sizeof(buf));
+
+	wr = lkl_test_logf("%d ", dir_fd);
+
+	if (ret < 0)
+		return TEST_FAILURE;
+
+	for (pos = buf; pos - buf < ret; pos += de->d_reclen) {
+		de = (struct lkl_linux_dirent64 *)pos;
+
+		wr += lkl_test_logf("%s ", de->d_name);
+		if (wr >= COL_LINE_BREAK) {
+			lkl_test_logf("\n");
+			wr = 0;
+		}
+	}
+
+	return TEST_SUCCESS;
+}
+
+LKL_TEST_CALL(close_dir_fd, lkl_sys_close, 0, dir_fd);
+LKL_TEST_CALL(chdir_root, lkl_sys_chdir, 0, "/");
+LKL_TEST_CALL(mount_fs_proc, lkl_sys_mount, 0, "none", "/proc", "proc", 0,
+	      NULL);
+LKL_TEST_CALL(umount_fs_proc, lkl_sys_umount, 0, "/proc", 0);
+
+LKL_TEST_CALL(start_kernel, lkl_start_kernel, 0, &lkl_host_ops,
+	     "mem=16M loglevel=8");
+LKL_TEST_CALL(stop_kernel, lkl_sys_halt, 0);
+
+struct lkl_test tests[] = {
+	LKL_TEST(start_kernel),
+	LKL_TEST(getpid),
+	LKL_TEST(syscall_latency),
+	LKL_TEST(umask),
+	LKL_TEST(umask2),
+	LKL_TEST(creat),
+	LKL_TEST(close),
+	LKL_TEST(failopen),
+	LKL_TEST(open),
+	LKL_TEST(write),
+	LKL_TEST(lseek_cur),
+	LKL_TEST(lseek_end),
+	LKL_TEST(lseek_set),
+	LKL_TEST(read),
+	LKL_TEST(fstat),
+	LKL_TEST(mkdir),
+	LKL_TEST(stat),
+#ifndef __MINGW32__
+	LKL_TEST(nanosleep),
+#endif
+	LKL_TEST(pipe2),
+	LKL_TEST(epoll),
+	LKL_TEST(mount_fs_proc),
+	LKL_TEST(chdir_proc),
+	LKL_TEST(open_cwd),
+	LKL_TEST(getdents64),
+	LKL_TEST(close_dir_fd),
+	LKL_TEST(chdir_root),
+	LKL_TEST(umount_fs_proc),
+	LKL_TEST(stop_kernel),
+};
+
+int main(int argc, const char **argv)
+{
+	lkl_host_ops.print = lkl_test_log;
+
+	return lkl_test_run(tests, sizeof(tests)/sizeof(struct lkl_test),
+			    "boot");
+}
diff --git a/tools/um/tests/boot.sh b/tools/um/tests/boot.sh
new file mode 100755
index 000000000000..d985c04b0ac1
--- /dev/null
+++ b/tools/um/tests/boot.sh
@@ -0,0 +1,9 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+source $script_dir/test.sh
+
+lkl_test_plan 1 "boot"
+lkl_test_run 1
+lkl_test_exec $script_dir/boot
diff --git a/tools/um/tests/run.py b/tools/um/tests/run.py
new file mode 100755
index 000000000000..c96ede90b6ad
--- /dev/null
+++ b/tools/um/tests/run.py
@@ -0,0 +1,172 @@
+#!/usr/bin/env python
+# SPDX-License-Identifier: GPL-2.0
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 2 of the License
+#
+# Author: Octavian Purdila <tavi@cs.pub.ro>
+#
+
+from __future__ import print_function
+
+import argparse
+import os
+import subprocess
+import sys
+import tap13
+import xml.etree.ElementTree as ET
+
+from junit_xml import TestSuite, TestCase
+
+
+class Reporter(tap13.Reporter):
+    def start(self, obj):
+        if type(obj) is tap13.Test:
+            if obj.result == "*":
+                end='\r'
+            else:
+                end='\n'
+            print("  TEST       %-8s %.50s" %
+                  (obj.result, obj.description + " " + obj.comment), end=end)
+
+        elif type(obj) is tap13.Suite:
+            if obj.tests_planned == 0:
+                status = "skip"
+            else:
+                status = ""
+            print("  SUITE      %-8s %s" % (status, obj.name))
+
+    def end(self, obj):
+        if type(obj) is tap13.Test:
+            if obj.result != "ok":
+                try:
+                    print(obj.yaml["log"], end='')
+                except:
+                    None
+
+
+mydir=os.path.dirname(os.path.realpath(__file__))
+
+tests = [
+    'boot.sh',
+]
+
+parser = argparse.ArgumentParser(description='LKL test runner')
+parser.add_argument('tests', nargs='?', action='append',
+                    help='tests to run %s' % tests)
+parser.add_argument('--junit-dir',
+                    help='directory where to store the juni suites')
+parser.add_argument('--gdb', action='store_true', default=False,
+                    help='run simple tests under gdb; implies --pass-through')
+parser.add_argument('--pass-through', action='store_true',  default=False,
+                    help='run the test without interpeting the test output')
+parser.add_argument('--valgrind', action='store_true', default=False,
+                    help='run simple tests under valgrind')
+
+args = parser.parse_args()
+if args.tests == [None]:
+    args.tests = tests
+
+if args.gdb:
+    args.pass_through=True
+    os.environ['GDB']="yes"
+
+if args.valgrind:
+    os.environ['VALGRIND']="yes"
+
+tap = tap13.Parser(Reporter())
+
+os.environ['PATH'] += ":" + mydir
+
+exit_code = 0
+
+for t in args.tests:
+    if not t:
+        continue
+    if args.pass_through:
+        print(t)
+        if subprocess.call(t, shell=True) != 0:
+            exit_code = 1
+    else:
+        p = subprocess.Popen(t, shell=True, stdout=subprocess.PIPE)
+        tap.parse(p.stdout)
+
+if args.pass_through:
+    sys.exit(exit_code)
+
+suites_count = 0
+tests_total = 0
+tests_not_ok = 0
+tests_ok = 0
+tests_skip = 0
+val_errs = 0
+val_fails = 0
+val_skips = 0
+
+for s in tap.run.suites:
+
+    junit_tests = []
+    suites_count += 1
+
+    for t in s.tests:
+        try:
+            secs = t.yaml["time_us"] / 1000000.0
+        except:
+            secs = 0
+        try:
+            log = t.yaml['log']
+        except:
+            log = ""
+
+        jt = TestCase(t.description, elapsed_sec=secs, stdout=log)
+        if t.result == 'skip':
+            jt.add_skipped_info(output=log)
+        elif t.result == 'not ok':
+            jt.add_error_info(output=log)
+
+        junit_tests.append(jt)
+
+        tests_total += 1
+        if t.result == "ok":
+            tests_ok += 1
+        elif t.result == "not ok":
+            tests_not_ok += 1
+            exit_code = 1
+        elif t.result == "skip":
+            tests_skip += 1
+
+    if args.junit_dir:
+        js = TestSuite(s.name, junit_tests)
+        with open(os.path.join(args.junit_dir, os.path.basename(s.name) + '.xml'), 'w') as f:
+            js.to_file(f, [js])
+
+        if os.getenv('VALGRIND') is not None:
+            val_xml = 'valgrind-%s.xml' % os.path.basename(s.name).replace(' ','-')
+            # skipped tests don't generate xml file
+            if os.path.exists(val_xml) is False:
+                continue
+
+            cmd = 'mv %s %s' % (val_xml, args.junit_dir)
+            subprocess.call(cmd, shell=True, )
+
+            cmd = mydir + '/valgrind2xunit.py ' + val_xml
+            subprocess.call(cmd, shell=True, cwd=args.junit_dir)
+
+            # count valgrind results
+            doc = ET.parse(os.path.join(args.junit_dir, 'valgrind-%s_xunit.xml' \
+                                        % (os.path.basename(s.name).replace(' ','-'))))
+            ts = doc.getroot()
+            val_errs += int(ts.get('errors'))
+            val_fails += int(ts.get('failures'))
+            val_skips += int(ts.get('skip'))
+
+print("Summary: %d suites run, %d tests, %d ok, %d not ok, %d skipped" %
+      (suites_count, tests_total, tests_ok, tests_not_ok, tests_skip))
+
+if os.getenv('VALGRIND') is not None:
+    print(" valgrind (memcheck): %d failures, %d skipped" % (val_fails, val_skips))
+    if val_errs or val_fails:
+        exit_code = 1
+
+sys.exit(exit_code)
diff --git a/tools/um/tests/tap13.py b/tools/um/tests/tap13.py
new file mode 100644
index 000000000000..65c73cda7ca1
--- /dev/null
+++ b/tools/um/tests/tap13.py
@@ -0,0 +1,209 @@
+#!/usr/bin/env python
+# SPDX-License-Identifier: GPL-2.0
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 2 of the License
+#
+# Author: Octavian Purdila <tavi@cs.pub.ro>
+#
+# Based on TAP13:
+#
+# Copyright 2013, Red Hat, Inc.
+# Author: Josef Skladanka <jskladan@redhat.com>
+#
+from __future__ import print_function
+
+import re
+import sys
+import yamlish
+
+
+class Reporter(object):
+
+    def start(self, obj):
+        None
+
+    def end(self, obj):
+        None
+
+
+class Test(object):
+    def __init__(self, reporter, result, id, description=None, directive=None,
+                 comment=None):
+        self.reporter = reporter
+        self.result = result
+        if directive:
+            self.result = directive.lower()
+        if id:
+            self.id = int(id)
+        else:
+            self.id = None
+        if description:
+            self.description = description
+        else:
+            self.description = ""
+        if comment:
+            self.comment = "# " + comment
+        else:
+            self.comment = ""
+        self.yaml = None
+        self._yaml_buffer = None
+        self.diagnostics = []
+
+        self.reporter.start(self)
+
+    def end(self):
+        if not self.yaml:
+            self.yaml = yamlish.load(self._yaml_buffer)
+            self.reporter.end(self)
+
+
+class Suite(object):
+    def __init__(self, reporter, start, end, explanation):
+        self.reporter = reporter
+        self.tests = []
+        self.name = explanation
+        self.tests_planned = int(end)
+
+        self.__tests_counter = 0
+        self.__tests_base = 0
+
+        self.reporter.start(self)
+
+    def newTest(self, args):
+        try:
+            self.tests[-1].end()
+        except IndexError:
+            None
+
+        if 'id' not in args or not args['id']:
+            args['id'] = self.__tests_counter
+        else:
+            args['id'] = int(args['id']) + self.__tests_base
+
+        if args['id'] < self.__tests_counter:
+            print("error: bad test id %d, fixing it" % (args['id']))
+            args['id'] = self.__tests_counter
+        # according to TAP13 specs, missing tests must be handled as 'not ok'
+        # here we add the missing tests in sequence
+        while args['id'] > (self.__tests_counter + 1):
+            comment = 'test %d not present' % self.__tests_counter
+            self.tests.append(Test(self.reporter, 'not ok',
+                                   self.__tests_counter, comment=comment))
+            self.__tests_counter += 1
+
+        if args['id'] == self.__tests_counter:
+            if args['directive']:
+                self.test().result = args['directive'].lower()
+            else:
+                self.test().result = args['result']
+            self.reporter.start(self.test())
+        else:
+            self.tests.append(Test(self.reporter, **args))
+            self.__tests_counter += 1
+
+    def test(self):
+        return self.tests[-1]
+
+    def end(self, name, planned):
+        if name == self.name:
+            self.tests_planned += int(planned)
+            self.__tests_base = self.__tests_counter
+            return False
+        try:
+            self.test().end()
+        except IndexError:
+            None
+        if len(self.tests) != self.tests_planned:
+            for i in range(len(self.tests), self.tests_planned):
+                self.tests.append(Test(self.reporter, 'not ok', i+1,
+                                       comment='test not present'))
+        return True
+
+
+class Run(object):
+
+    def __init__(self, reporter):
+        self.reporter = reporter
+        self.suites = []
+
+    def suite(self):
+        return self.suites[-1]
+
+    def test(self):
+        return self.suites[-1].tests[-1]
+
+    def newSuite(self, args):
+        new = False
+        try:
+            if self.suite().end(args['explanation'], args['end']):
+                new = True
+        except IndexError:
+            new = True
+        if new:
+            self.suites.append(Suite(self.reporter, **args))
+
+    def newTest(self, args):
+        self.suite().newTest(args)
+
+
+class Parser(object):
+    RE_PLAN = re.compile(r"^\s*(?P<start>\d+)\.\.(?P<end>\d+)\s*(#\s*(?P<explanation>.*))?\s*$")
+    RE_TEST_LINE = re.compile(r"^\s*(?P<result>(not\s+)?ok|[*]+)\s*(?P<id>\d+)?\s*(?P<description>[^#]+)?\s*(#\s*(?P<directive>TODO|SKIP)?\s*(?P<comment>.+)?)?\s*$",  re.IGNORECASE)
+    RE_EXPLANATION = re.compile(r"^\s*#\s*(?P<explanation>.+)?\s*$")
+    RE_YAMLISH_START = re.compile(r"^\s*---.*$")
+    RE_YAMLISH_END = re.compile(r"^\s*\.\.\.\s*$")
+
+    def __init__(self, reporter):
+        self.seek_test = False
+        self.in_test = False
+        self.in_yaml = False
+        self.run = Run(reporter)
+
+    def parse(self, source):
+        # to avoid input buffering
+        while True:
+            line = source.readline()
+            if not line:
+                break
+
+            if self.in_yaml:
+                if Parser.RE_YAMLISH_END.match(line):
+                    self.run.test()._yaml_buffer.append(line.strip())
+                    self.in_yaml = False
+                else:
+                    self.run.test()._yaml_buffer.append(line.rstrip())
+                continue
+
+            line = line.strip()
+
+            if self.in_test:
+                if Parser.RE_EXPLANATION.match(line):
+                    self.run.test().diagnostics.append(line)
+                    continue
+                if Parser.RE_YAMLISH_START.match(line):
+                    self.run.test()._yaml_buffer = [line.strip()]
+                    self.in_yaml = True
+                    continue
+
+            m = Parser.RE_PLAN.match(line)
+            if m:
+                self.seek_test = True
+                args = m.groupdict()
+                self.run.newSuite(args)
+                continue
+
+            if self.seek_test:
+                m = Parser.RE_TEST_LINE.match(line)
+                if m:
+                    args = m.groupdict()
+                    self.run.newTest(args)
+                    self.in_test = True
+                    continue
+
+            print(line)
+        try:
+            self.run.suite().end(None, 0)
+        except IndexError:
+            None
diff --git a/tools/um/tests/test.c b/tools/um/tests/test.c
new file mode 100644
index 000000000000..37ea21901f99
--- /dev/null
+++ b/tools/um/tests/test.c
@@ -0,0 +1,126 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <stdarg.h>
+#include <time.h>
+
+#include "test.h"
+
+/* circular log buffer */
+
+static char log_buf[0x10000];
+static char *head = log_buf, *tail = log_buf;
+
+static inline void advance(char **ptr)
+{
+	if ((unsigned int)(*ptr - log_buf) >= sizeof(log_buf))
+		*ptr = log_buf;
+	else
+		*ptr = *ptr + 1;
+}
+
+static void log_char(char c)
+{
+	*tail = c;
+	advance(&tail);
+	if (tail == head)
+		advance(&head);
+}
+
+static void print_log(void)
+{
+	char last;
+
+	printf(" log: |\n");
+	last = '\n';
+	while (head != tail) {
+		if (last == '\n')
+			printf("  ");
+		last = *head;
+		putchar(last);
+		advance(&head);
+	}
+	if (last != '\n')
+		putchar('\n');
+}
+
+int lkl_test_run(const struct lkl_test *tests, int nr, const char *fmt, ...)
+{
+	int i, ret, status = TEST_SUCCESS;
+	clock_t start, stop;
+	char name[1024];
+	va_list args;
+
+	va_start(args, fmt);
+	vsnprintf(name, sizeof(name), fmt, args);
+	va_end(args);
+
+	printf("1..%d # %s\n", nr, name);
+	for (i = 1; i <= nr; i++) {
+		const struct lkl_test *t = &tests[i-1];
+		unsigned long delta_us;
+
+		printf("* %d %s\n", i, t->name);
+		fflush(stdout);
+
+		start = clock();
+
+		ret = t->fn(t->arg1, t->arg2, t->arg3);
+
+		stop = clock();
+
+		switch (ret) {
+		case TEST_SUCCESS:
+			printf("ok %d %s\n", i, t->name);
+			break;
+		case TEST_SKIP:
+			printf("ok %d %s # SKIP\n", i, t->name);
+			break;
+		case TEST_BAILOUT:
+			status = TEST_BAILOUT;
+			/* fall through; */
+		case TEST_FAILURE:
+		default:
+			if (status != TEST_BAILOUT)
+				status = TEST_FAILURE;
+			printf("not ok %d %s\n", i, t->name);
+		}
+
+		printf(" ---\n");
+		delta_us = (stop - start) * 1000000 / CLOCKS_PER_SEC;
+		printf(" time_us: %ld\n", delta_us);
+		print_log();
+		printf(" ...\n");
+
+		if (status == TEST_BAILOUT) {
+			printf("Bail out!\n");
+			return TEST_FAILURE;
+		}
+
+		fflush(stdout);
+	}
+
+	return status;
+}
+
+
+void lkl_test_log(const char *str, int len)
+{
+	while (len--)
+		log_char(*(str++));
+}
+
+int lkl_test_logf(const char *fmt, ...)
+{
+	char tmp[1024], *c;
+	va_list args;
+	unsigned int n;
+
+	va_start(args, fmt);
+	n = vsnprintf(tmp, sizeof(tmp), fmt, args);
+	va_end(args);
+
+	for (c = tmp; *c != 0; c++)
+		log_char(*c);
+
+	return n > sizeof(tmp) ? sizeof(tmp) : n;
+}
diff --git a/tools/um/tests/test.h b/tools/um/tests/test.h
new file mode 100644
index 000000000000..f19bbc487a98
--- /dev/null
+++ b/tools/um/tests/test.h
@@ -0,0 +1,72 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_TEST_H
+#define _LKL_TEST_H
+
+#define TEST_SUCCESS	0
+#define TEST_FAILURE	1
+#define TEST_SKIP	2
+#define TEST_TODO	3
+#define TEST_BAILOUT	4
+
+struct lkl_test {
+	const char *name;
+	int (*fn)();
+	void *arg1, *arg2, *arg3;
+};
+
+/**
+ * Simple wrapper to initialize a test entry.
+ * @name - test name, it assume test function is named test_@name
+ * @vargs - arguments to be passed to the function
+ */
+#define LKL_TEST(name, ...) { #name, lkl_test_##name, __VA_ARGS__ }
+
+/**
+ * lkl_test_run - run a test suite
+ *
+ * @tests - the list of tests to run
+ * @nr - number of tests
+ * @fmt - format string to be used for suite name
+ */
+int lkl_test_run(const struct lkl_test *tests, int nr, const char *fmt, ...);
+
+/**
+ * lkl_test_log - store a string in the test log buffer
+ * @str - the string to log (can be non-NULL terminated)
+ * @len - the string length
+ */
+void lkl_test_log(const char *str, int len);
+
+/**
+ * lkl_test_logf - printf like function to store into the test log buffer
+ * @fmt - printf format string
+ * @vargs - arguments to the format string
+ */
+int lkl_test_logf(const char *fmt, ...) __attribute__((format(printf, 1, 2)));
+
+/**
+ * LKL_TEST_CALL - create a test function as for a LKL call
+ *
+ * The test function will be named lkl_test_@name and will return
+ * TEST_SUCCESS if the called functions returns @expect. Otherwise
+ * will return TEST_FAILUIRE.
+ *
+ * @name - test name; must be unique because it is part of the
+ * test function; the test function will be named
+ * @call - function to call
+ * @expect - expected return value for success
+ * @args - arguments to pass to the LKL call
+ */
+#define LKL_TEST_CALL(name, call, expect, ...)				\
+	static int lkl_test_##name(void)				\
+	{								\
+		long ret;						\
+									\
+		ret = call(__VA_ARGS__);				\
+		lkl_test_logf("%s(%s) = %ld %s\n", #call, #__VA_ARGS__, \
+			ret, ret < 0 ? lkl_strerror(ret) : "");		\
+		return (ret == expect) ? TEST_SUCCESS : TEST_FAILURE;	\
+	}
+
+
+#endif /* _LKL_TEST_H */
diff --git a/tools/um/tests/test.sh b/tools/um/tests/test.sh
new file mode 100644
index 000000000000..31b7de5c06f7
--- /dev/null
+++ b/tools/um/tests/test.sh
@@ -0,0 +1,181 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+basedir=$(cd $script_dir/..; pwd)
+base_objdir=$(cd ${OUTPUT}/; pwd)
+
+TEST_SUCCESS=0
+TEST_FAILURE=1
+TEST_SKIP=113
+TEST_TODO=114
+TEST_BAILOUT=115
+
+print_log()
+{
+    echo " log: |"
+    while read line; do
+        echo "  $line"
+    done < $1
+}
+
+export_vars()
+{
+    if [ -z "$var_file" ]; then
+        return
+    fi
+
+    for i in $@; do
+        echo "$i=${!i}" >> $var_file
+    done
+}
+
+lkl_test_run()
+{
+    log_file=$(mktemp)
+    export var_file=$(mktemp)
+
+    tid=$1 && shift && tname=$@
+
+    echo "* $tid $tname"
+
+    start=$(date '+%s%9N')
+    # run in a separate shell to avoid -e terminating us
+    $@ 2>&1 | strings >$log_file
+    exit=${PIPESTATUS[0]}
+    stop=$(date '+%s%9N')
+
+    case $exit in
+    $TEST_SUCCESS)
+        echo "ok $tid $tname"
+        ;;
+    $TEST_SKIP)
+        echo "ok $tid $tname # SKIP"
+        ;;
+    $TEST_BAILOUT)
+        echo "not ok $tid $tname"
+        echo "Bail out!"
+        ;;
+    $TEST_FAILURE|*)
+        echo "not ok $tid $tname"
+        ;;
+    esac
+
+    delta=$(((stop-start)/1000))
+
+    echo " ---"
+    echo " time_us: $delta"
+    print_log $log_file
+    echo -e " ..."
+
+    rm $log_file
+    . $var_file
+    rm $var_file
+
+    return $exit
+}
+
+lkl_test_plan()
+{
+    echo "1..$1 # $2"
+    export suite_name="${2// /\-}"
+}
+
+lkl_test_exec()
+{
+    local SUDO=""
+    local WRAPPER=""
+
+    if [ "$1" = "sudo" ]; then
+        SUDO=sudo
+        shift
+    fi
+
+    local file=$1
+    shift
+
+    if [ -n "$LKL_HOST_CONFIG_NT" ]; then
+        file=$file.exe
+    fi
+
+    file=${OUTPUT}/tests/$(basename $file)
+
+    if file $file | grep ARM; then
+        WRAPPER="qemu-arm-static"
+    elif file $file | grep "FreeBSD" ; then
+        ssh_copy "$file" $BSD_WDIR
+        if [ -n "$SUDO" ]; then
+            SUDO=""
+        fi
+        WRAPPER="$MYSSH $SU"
+        # ssh will mess up with pipes ('|') so, escape the pipe char.
+        args="${@//\|/\\\|}"
+        set - $BSD_WDIR/$(basename $file) $args
+        file=""
+    elif [ -n "$GDB" ]; then
+        WRAPPER="gdb"
+        args="$@"
+        set - -ex "run $args" -ex quit $file
+        file=""
+    elif [ -n "$VALGRIND" ]; then
+        WRAPPER="valgrind --suppressions=$script_dir/valgrind.supp \
+                  --leak-check=full --show-leak-kinds=all --xml=yes \
+                  --xml-file=valgrind-$suite_name.xml"
+    fi
+
+    $SUDO $WRAPPER $file "$@"
+}
+
+lkl_test_cmd()
+{
+    local WRAPPER=""
+
+    if [ -z "$QUIET" ]; then
+        SHOPTS="-x"
+    fi
+
+    if [ -n "$LKL_HOST_CONFIG_BSD" ]; then
+        WRAPPER="$MYSSH $SU"
+    fi
+
+    echo "$@" | $WRAPPER sh $SHOPTS
+}
+
+# XXX: $MYSSH and $MYSCP are defined in a circleci docker image.
+# see the definitions in lkl/lkl-docker:circleci/freebsd11/Dockerfile
+ssh_push()
+{
+    while [ -n "$1" ]; do
+        if [[ "$1" = *.sh ]]; then
+            type="script"
+        else
+            type="file"
+        fi
+
+        dir=$(dirname $1)
+        $MYSSH mkdir -p $BSD_WDIR/$dir
+
+        $MYSCP -P 7722 -r $basedir/$1 root@localhost:$BSD_WDIR/$dir
+        if [ "$type" = "script" ]; then
+            $MYSSH chmod a+x $BSD_WDIR/$1
+        fi
+
+        shift
+    done
+}
+
+ssh_copy()
+{
+    $MYSCP -P 7722 -r $1 root@localhost:$2
+}
+
+lkl_test_bsd_cleanup()
+{
+    $MYSSH rm -rf $BSD_WDIR
+}
+
+if [ -n "$LKL_HOST_CONFIG_BSD" ]; then
+    trap lkl_test_bsd_cleanup EXIT
+    export BSD_WDIR=/root/lkl
+    $MYSSH mkdir -p $BSD_WDIR
+fi
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v7 21/21] um: nommu: add block device support of UML
  2020-10-06  9:44         ` Hajime Tazaki
@ 2020-10-06  9:44           ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-06  9:44 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, retrage01, linux-kernel-library, linux-arch, Hajime Tazaki

This commit adds block device support for nommu mode, and also added
several host utilities to run a simple test program
(tools/um/test/disk.c) successfully.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/include/asm/xor.h                 |   3 +-
 arch/um/include/shared/as-layout.h        |   1 +
 arch/um/kernel/um_arch.c                  |   5 +
 arch/um/nommu/include/uapi/asm/host_ops.h |   7 +
 arch/um/nommu/include/uapi/asm/unistd.h   |   2 +
 arch/um/nommu/um/Kconfig                  |   8 +
 arch/um/nommu/um/setup.c                  |  12 +
 tools/um/Targets                          |   2 +
 tools/um/include/lkl.h                    | 206 ++++++++++
 tools/um/include/lkl_host.h               |   1 +
 tools/um/lib/Build                        |   1 +
 tools/um/lib/fs.c                         | 461 ++++++++++++++++++++++
 tools/um/lib/posix-host.c                 |   1 +
 tools/um/lib/utils.c                      |   3 +
 tools/um/tests/Build                      |   1 +
 tools/um/tests/cla.c                      | 159 ++++++++
 tools/um/tests/cla.h                      |  33 ++
 tools/um/tests/disk.c                     | 168 ++++++++
 tools/um/tests/disk.sh                    |  70 ++++
 tools/um/tests/run.py                     |   2 +
 20 files changed, 1145 insertions(+), 1 deletion(-)
 create mode 100644 tools/um/lib/fs.c
 create mode 100644 tools/um/tests/cla.c
 create mode 100644 tools/um/tests/cla.h
 create mode 100644 tools/um/tests/disk.c
 create mode 100755 tools/um/tests/disk.sh

diff --git a/arch/um/include/asm/xor.h b/arch/um/include/asm/xor.h
index 36b33d62a35d..a5ea458a1ae9 100644
--- a/arch/um/include/asm/xor.h
+++ b/arch/um/include/asm/xor.h
@@ -4,4 +4,5 @@
 
 /* pick an arbitrary one - measuring isn't possible with inf-cpu */
 #define XOR_SELECT_TEMPLATE(x)	\
-	(time_travel_mode == TT_MODE_INFCPU ? &xor_block_8regs : NULL)
+	(time_travel_mode == TT_MODE_INFCPU || (!IS_ENABLED(CONFIG_MMU)) ? \
+	 &xor_block_8regs : NULL)
diff --git a/arch/um/include/shared/as-layout.h b/arch/um/include/shared/as-layout.h
index 5f286ef2721b..4423437a5ace 100644
--- a/arch/um/include/shared/as-layout.h
+++ b/arch/um/include/shared/as-layout.h
@@ -57,6 +57,7 @@ extern unsigned long host_task_size;
 
 extern int linux_main(int argc, char **argv);
 extern void uml_finishsetup(void);
+extern void uml_set_args(char *args);
 
 struct siginfo;
 extern void (*sig_info[])(int, struct siginfo *si, struct uml_pt_regs *);
diff --git a/arch/um/kernel/um_arch.c b/arch/um/kernel/um_arch.c
index e2cb76c03b25..da6e06cf2808 100644
--- a/arch/um/kernel/um_arch.c
+++ b/arch/um/kernel/um_arch.c
@@ -41,6 +41,11 @@ static void __init add_arg(char *arg)
 	strcat(command_line, arg);
 }
 
+void __init uml_set_args(char *args)
+{
+	strcat(command_line, args);
+}
+
 /*
  * These fields are initialized at boot time and not changed.
  * XXX This structure is used only in the non-SMP case.  Maybe this
diff --git a/arch/um/nommu/include/uapi/asm/host_ops.h b/arch/um/nommu/include/uapi/asm/host_ops.h
index 0811494c080c..1b2507502969 100644
--- a/arch/um/nommu/include/uapi/asm/host_ops.h
+++ b/arch/um/nommu/include/uapi/asm/host_ops.h
@@ -15,6 +15,11 @@ struct lkl_jmp_buf {
  *
  * These operations must be provided by a host library or by the application
  * itself.
+ *
+ * @um_devices - string containg the list of UML devices in command line
+ * format. This string is appended to the kernel command line and
+ * is provided here for convenience to be implemented by the host library.
+ *
  * @print - optional operation that receives console messages
  * @panic - called during a kernel panic
  *
@@ -65,6 +70,8 @@ struct lkl_jmp_buf {
  *
  */
 struct lkl_host_operations {
+	const char *um_devices;
+
 	void (*print)(const char *str, int len);
 	void (*panic)(void);
 
diff --git a/arch/um/nommu/include/uapi/asm/unistd.h b/arch/um/nommu/include/uapi/asm/unistd.h
index 1762ebb829f5..320762099a62 100644
--- a/arch/um/nommu/include/uapi/asm/unistd.h
+++ b/arch/um/nommu/include/uapi/asm/unistd.h
@@ -3,6 +3,8 @@
 #define __UM_NOMMU_UAPI_UNISTD_H
 
 #define __ARCH_WANT_NEW_STAT
+#define __ARCH_WANT_SET_GET_RLIMIT
+
 #include <asm/bitsperlong.h>
 
 #if __BITS_PER_LONG == 64
diff --git a/arch/um/nommu/um/Kconfig b/arch/um/nommu/um/Kconfig
index 20b3eaccb6f0..c6a3f472fe75 100644
--- a/arch/um/nommu/um/Kconfig
+++ b/arch/um/nommu/um/Kconfig
@@ -4,6 +4,10 @@ config UML_NOMMU
 	select UACCESS_MEMCPY
 	select ARCH_THREAD_STACK_ALLOCATOR
 	select ARCH_HAS_SYSCALL_WRAPPER
+	select VFAT_FS
+	select NLS_CODEPAGE_437
+	select NLS_ISO8859_1
+	select BTRFS_FS
 
 config 64BIT
 	bool
@@ -35,3 +39,7 @@ config STACKTRACE_SUPPORT
 config PRINTK_TIME
 	bool
 	default y
+
+config RAID6_PQ_BENCHMARK
+	bool
+	default n
diff --git a/arch/um/nommu/um/setup.c b/arch/um/nommu/um/setup.c
index 922188690139..d71d61979ad6 100644
--- a/arch/um/nommu/um/setup.c
+++ b/arch/um/nommu/um/setup.c
@@ -45,13 +45,25 @@ static void __init *lkl_run_kernel(void *arg)
 	return NULL;
 }
 
+static char _cmd_line[COMMAND_LINE_SIZE];
 int __init lkl_start_kernel(struct lkl_host_operations *ops,
 			    const char *fmt, ...)
 {
+	va_list ap;
 	int ret;
 
 	lkl_ops = ops;
 
+	va_start(ap, fmt);
+	ret = vsnprintf(_cmd_line, COMMAND_LINE_SIZE, fmt, ap);
+	va_end(ap);
+
+	if (ops->um_devices)
+		strscpy(_cmd_line + ret, ops->um_devices,
+			COMMAND_LINE_SIZE - ret);
+
+	uml_set_args(_cmd_line);
+
 	init_sem = lkl_ops->sem_alloc(0);
 	if (!init_sem)
 		return -ENOMEM;
diff --git a/tools/um/Targets b/tools/um/Targets
index 2bb90381843c..f5f8ec4b9dbb 100644
--- a/tools/um/Targets
+++ b/tools/um/Targets
@@ -1,10 +1,12 @@
 ifeq ($(UMMODE),library)
 progs-y += tests/boot
+progs-y += tests/disk
 else
 progs-y += uml/linux
 endif
 
 LDFLAGS_boot-y := -pie
+LDFLAGS_disk-y := -pie
 
 LDLIBS := -lrt -lpthread -lutil
 LDFLAGS_linux-y := -no-pie -Wl,--wrap,malloc -Wl,--wrap,free -Wl,--wrap,calloc
diff --git a/tools/um/include/lkl.h b/tools/um/include/lkl.h
index 707e01b64a70..4a9d874bfbf0 100644
--- a/tools/um/include/lkl.h
+++ b/tools/um/include/lkl.h
@@ -113,6 +113,16 @@ static inline long lkl_sys_creat(const char *file, int mode)
 }
 #endif
 
+#ifdef __lkl__NR_faccessat
+/**
+ * lkl_sys_access - wrapper for lkl_sys_faccessat
+ */
+static inline long lkl_sys_access(const char *file, int mode)
+{
+	return lkl_sys_faccessat(LKL_AT_FDCWD, file, mode);
+}
+#endif
+
 #ifdef __lkl__NR_mkdirat
 /**
  * lkl_sys_mkdir - wrapper for lkl_sys_mkdirat
@@ -123,6 +133,36 @@ static inline long lkl_sys_mkdir(const char *path, mode_t mode)
 }
 #endif
 
+#ifdef __lkl__NR_unlinkat
+/**
+ * lkl_sys_rmdir - wrapper for lkl_sys_unlinkrat
+ */
+static inline long lkl_sys_rmdir(const char *path)
+{
+	return lkl_sys_unlinkat(LKL_AT_FDCWD, path, LKL_AT_REMOVEDIR);
+}
+#endif
+
+#ifdef __lkl__NR_unlinkat
+/**
+ * lkl_sys_unlink - wrapper for lkl_sys_unlinkat
+ */
+static inline long lkl_sys_unlink(const char *path)
+{
+	return lkl_sys_unlinkat(LKL_AT_FDCWD, path, 0);
+}
+#endif
+
+#ifdef __lkl__NR_mknodat
+/**
+ * lkl_sys_mknod - wrapper for lkl_sys_mknodat
+ */
+static inline long lkl_sys_mknod(const char *path, mode_t mode, dev_t dev)
+{
+	return lkl_sys_mknodat(LKL_AT_FDCWD, path, mode, dev);
+}
+#endif
+
 #ifdef __lkl__NR_epoll_create1
 /**
  * lkl_sys_epoll_create - wrapper for lkl_sys_epoll_create1
@@ -144,6 +184,172 @@ static inline long lkl_sys_epoll_wait(int fd, struct lkl_epoll_event *ev,
 }
 #endif
 
+/**
+ * struct lkl_dev_blk_ops - block device host operations, defined in lkl_host.h.
+ */
+struct lkl_dev_blk_ops;
+
+/**
+ * lkl_disk - host disk handle
+ *
+ * @dev - a pointer to private information for this disk backend
+ * @fd - a POSIX file descriptor that can be used by preadv/pwritev
+ * @handle - an NT file handle that can be used by ReadFile/WriteFile
+ */
+struct lkl_disk {
+	void *dev;
+	union {
+		int fd;
+		void *handle;
+	};
+	struct lkl_dev_blk_ops *ops;
+};
+
+/**
+ * lkl_disk_add - add a new disk
+ *
+ * @disk - the host disk handle
+ * @returns a disk id (0 is valid) or a strictly negative value in case of error
+ */
+int lkl_disk_add(struct lkl_disk *disk);
+
+/**
+ * lkl_disk_remove - remove a disk
+ *
+ * This function makes a cleanup of the @disk's private information
+ * that was initialized by lkl_disk_add before.
+ *
+ * @disk - the host disk handle
+ */
+int lkl_disk_remove(struct lkl_disk disk);
+
+/**
+ * lkl_encode_dev_from_sysfs_blkdev - extract device id from sysfs
+ *
+ * This function returns the device id for the given sysfs dev node.
+ * The content of the node has to be in the form 'MAJOR:MINOR'.
+ * Also, this function expects an absolute path which means that sysfs
+ * already has to be mounted at the given path
+ *
+ * @sysfs_path - absolute path to the sysfs dev node
+ * @pdevid - pointer to memory where dev id will be returned
+ * @returns - 0 on success, a negative value on error
+ */
+int lkl_encode_dev_from_sysfs(const char *sysfs_path, uint32_t *pdevid);
+
+/**
+ * lkl_mount_dev - mount a disk
+ *
+ * This functions creates a device file for the given disk, creates a mount
+ * point and mounts the device over the mount point.
+ *
+ * @disk_id - the disk id identifying the disk to be mounted
+ * @part - disk partition or zero for full disk
+ * @fs_type - filesystem type
+ * @flags - mount flags
+ * @opts - additional filesystem specific mount options
+ * @mnt_str - a string that will be filled by this function with the path where
+ * the filesystem has been mounted
+ * @mnt_str_len - size of mnt_str
+ * @returns - 0 on success, a negative value on error
+ */
+long lkl_mount_dev(unsigned int disk_id, unsigned int part, const char *fs_type,
+		   int flags, const char *opts,
+		   char *mnt_str, unsigned int mnt_str_len);
+
+/**
+ * lkl_umount_dev - umount a disk
+ *
+ * This functions umounts the given disks and removes the device file and the
+ * mount point.
+ *
+ * @disk_id - the disk id identifying the disk to be mounted
+ * @part - disk partition or zero for full disk
+ * @flags - umount flags
+ * @timeout_ms - timeout to wait for the kernel to flush closed files so that
+ * umount can succeed
+ * @returns - 0 on success, a negative value on error
+ */
+long lkl_umount_dev(unsigned int disk_id, unsigned int part, int flags,
+		    long timeout_ms);
+
+/**
+ * lkl_umount_timeout - umount filesystem with timeout
+ *
+ * @path - the path to unmount
+ * @flags - umount flags
+ * @timeout_ms - timeout to wait for the kernel to flush closed files so that
+ * umount can succeed
+ * @returns - 0 on success, a negative value on error
+ */
+long lkl_umount_timeout(char *path, int flags, long timeout_ms);
+
+/**
+ * lkl_opendir - open a directory
+ *
+ * @path - directory path
+ * @err - pointer to store the error in case of failure
+ * @returns - a handle to be used when calling lkl_readdir
+ */
+struct lkl_dir *lkl_opendir(const char *path, int *err);
+
+/**
+ * lkl_fdopendir - open a directory
+ *
+ * @fd - file descriptor
+ * @err - pointer to store the error in case of failure
+ * @returns - a handle to be used when calling lkl_readdir
+ */
+struct lkl_dir *lkl_fdopendir(int fd, int *err);
+
+/**
+ * lkl_rewinddir - reset directory stream
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ */
+void lkl_rewinddir(struct lkl_dir *dir);
+
+/**
+ * lkl_closedir - close the directory
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ */
+int lkl_closedir(struct lkl_dir *dir);
+
+/**
+ * lkl_readdir - get the next available entry of the directory
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ * @returns - a lkl_dirent64 entry or NULL if the end of the directory stream is
+ * reached or if an error occurred; check lkl_errdir() to distinguish between
+ * errors or end of the directory stream
+ */
+struct lkl_linux_dirent64 *lkl_readdir(struct lkl_dir *dir);
+
+/**
+ * lkl_errdir - checks if an error occurred during the last lkl_readdir call
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ * @returns - 0 if no error occurred, or a negative value otherwise
+ */
+int lkl_errdir(struct lkl_dir *dir);
+
+/**
+ * lkl_dirfd - gets the file descriptor associated with the directory handle
+ *
+ * @dir - the directory handle as returned by lkl_opendir
+ * @returns - a positive value,which is the LKL file descriptor associated with
+ * the directory handle, or a negative value otherwise
+ */
+int lkl_dirfd(struct lkl_dir *dir);
+
+/**
+ * lkl_mount_fs - mount a file system type like proc, sys
+ * @fstype - file system type. e.g. proc, sys
+ * @returns - 0 on success. 1 if it's already mounted. negative on failure.
+ */
+int lkl_mount_fs(char *fstype);
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/tools/um/include/lkl_host.h b/tools/um/include/lkl_host.h
index 85e80eb4ad0d..12dd95616e43 100644
--- a/tools/um/include/lkl_host.h
+++ b/tools/um/include/lkl_host.h
@@ -10,6 +10,7 @@ extern "C" {
 #include <lkl.h>
 
 extern struct lkl_host_operations lkl_host_ops;
+extern char lkl_um_devs[4096];
 
 /**
  * lkl_printf - print a message via the host print operation
diff --git a/tools/um/lib/Build b/tools/um/lib/Build
index dddff26a3b4e..29491b40746c 100644
--- a/tools/um/lib/Build
+++ b/tools/um/lib/Build
@@ -4,3 +4,4 @@ CFLAGS_posix-host.o += -D_FILE_OFFSET_BITS=64
 liblinux-$(CONFIG_UMMODE_LIB) += utils.o
 liblinux-$(CONFIG_UMMODE_LIB) += posix-host.o
 liblinux-$(CONFIG_UMMODE_LIB) += jmp_buf.o
+liblinux-$(CONFIG_UMMODE_LIB) += fs.o
diff --git a/tools/um/lib/fs.c b/tools/um/lib/fs.c
new file mode 100644
index 000000000000..948aac9730c2
--- /dev/null
+++ b/tools/um/lib/fs.c
@@ -0,0 +1,461 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdarg.h>
+#include <stdio.h>
+#include <string.h>
+#include <stdlib.h>
+#include <lkl_host.h>
+
+#define MAX_FSTYPE_LEN 50
+
+static struct lkl_disk *lkl_disks[16];
+static int registered_blk_dev_idx;
+
+static int lkl_disk_um_add(struct lkl_disk *disk, const char *blkparams)
+{
+	/* concat strings */
+	snprintf(lkl_um_devs + strlen(lkl_um_devs), sizeof(lkl_um_devs),
+		 " ubd%d=%s", registered_blk_dev_idx, blkparams);
+
+	return registered_blk_dev_idx++;
+}
+
+int lkl_disk_add(struct lkl_disk *disk)
+{
+	int ret = -1;
+
+	ret = lkl_disk_um_add(disk, disk->dev);
+
+	lkl_disks[ret] = disk;
+
+	return ret;
+}
+
+int lkl_disk_remove(struct lkl_disk disk)
+{
+	/* FIXME */
+	return 0;
+}
+
+int lkl_mount_fs(char *fstype)
+{
+	char dir[MAX_FSTYPE_LEN+2] = "/";
+	int flags = 0, ret = 0;
+
+	strncat(dir, fstype, MAX_FSTYPE_LEN);
+
+	/* Create with regular umask */
+	ret = lkl_sys_mkdir(dir, 0xff);
+	if (ret && ret != -LKL_EEXIST) {
+		lkl_perror("mount_fs mkdir", ret);
+		return ret;
+	}
+
+	/* We have no use for nonzero flags right now */
+	ret = lkl_sys_mount("none", dir, fstype, flags, NULL);
+	if (ret && ret != -LKL_EBUSY) {
+		lkl_sys_rmdir(dir);
+		return ret;
+	}
+
+	if (ret == -LKL_EBUSY)
+		return 1;
+	return 0;
+}
+
+static uint32_t new_encode_dev(unsigned int major, unsigned int minor)
+{
+	return (minor & 0xff) | (major << 8) | ((minor & ~0xff) << 12);
+}
+
+static int startswith(const char *str, const char *pre)
+{
+	return strncmp(pre, str, strlen(pre)) == 0;
+}
+
+static int get_node_with_prefix(const char *path, const char *prefix,
+				char *result, unsigned int result_len)
+{
+	struct lkl_dir *dir = NULL;
+	struct lkl_linux_dirent64 *dirent;
+	int ret;
+
+	dir = lkl_opendir(path, &ret);
+	if (!dir)
+		return ret;
+
+	ret = -LKL_ENOENT;
+
+	while ((dirent = lkl_readdir(dir))) {
+		if (startswith(dirent->d_name, prefix)) {
+			if (strlen(dirent->d_name) + 1 > result_len) {
+				ret = -LKL_ENOMEM;
+				break;
+			}
+			memcpy(result, dirent->d_name, strlen(dirent->d_name));
+			result[strlen(dirent->d_name)] = '\0';
+			ret = 0;
+			break;
+		}
+	}
+
+	lkl_closedir(dir);
+
+	return ret;
+}
+
+int lkl_encode_dev_from_sysfs(const char *sysfs_path, uint32_t *pdevid)
+{
+	int ret;
+	long fd;
+	int major, minor;
+	char buf[16] = { 0, };
+	char *bufptr;
+
+	fd = lkl_sys_open(sysfs_path, LKL_O_RDONLY, 0);
+	if (fd < 0)
+		return fd;
+
+	ret = lkl_sys_read(fd, buf, sizeof(buf));
+	if (ret < 0)
+		goto out_close;
+
+	if (ret == sizeof(buf)) {
+		ret = -LKL_ENOBUFS;
+		goto out_close;
+	}
+
+	bufptr = strchr(buf, ':');
+	if (bufptr == NULL) {
+		ret = -LKL_EINVAL;
+		goto out_close;
+	}
+	bufptr[0] = '\0';
+	bufptr++;
+
+	major = atoi(buf);
+	minor = atoi(bufptr);
+
+	*pdevid = new_encode_dev(major, minor);
+	ret = 0;
+
+out_close:
+	lkl_sys_close(fd);
+
+	return ret;
+}
+
+#define SYSFS_DEV_UMBLK_CMDLINE_PATH \
+	"/sysfs/devices/platform/uml-blkdev.%d"
+
+struct abuf {
+	char *mem, *ptr;
+	unsigned int len;
+};
+
+static int snprintf_append(struct abuf *buf, const char *fmt, ...)
+{
+	int ret;
+	va_list args;
+
+	if (!buf->ptr)
+		buf->ptr = buf->mem;
+
+	va_start(args, fmt);
+	ret = vsnprintf(buf->ptr, buf->len - (buf->ptr - buf->mem), fmt, args);
+	va_end(args);
+
+	if (ret < 0 || (ret >= (int)(buf->len - (buf->ptr - buf->mem))))
+		return -LKL_ENOMEM;
+
+	buf->ptr += ret;
+
+	return 0;
+}
+
+static int __lkl_get_blkdev(int disk_id, unsigned int part, uint32_t *pdevid,
+			    const char *sysfs_path_fmt, const char *drv_prefix,
+			    const char *disk_prefix)
+{
+	char sysfs_path[LKL_PATH_MAX];
+	char drv_name[LKL_PATH_MAX];
+	char disk_name[LKL_PATH_MAX];
+	struct abuf sysfs_path_buf = {
+		.mem = sysfs_path,
+		.len = sizeof(sysfs_path),
+	};
+	int ret;
+
+	if (disk_id < 0)
+		return -LKL_EINVAL;
+
+	ret = lkl_mount_fs("sysfs");
+	if (ret < 0)
+		return ret;
+
+	ret = snprintf_append(&sysfs_path_buf, sysfs_path_fmt, disk_id);
+	if (ret)
+		return ret;
+
+	ret = get_node_with_prefix(sysfs_path, drv_prefix, drv_name,
+				   sizeof(drv_name));
+	if (ret)
+		return ret;
+
+	ret = snprintf_append(&sysfs_path_buf, "/%s/block", drv_name);
+	if (ret)
+		return ret;
+
+	ret = get_node_with_prefix(sysfs_path, disk_prefix, disk_name,
+				   sizeof(disk_name));
+	if (ret)
+		return ret;
+
+	if (!part)
+		ret = snprintf_append(&sysfs_path_buf, "/%s/dev", disk_name);
+	else
+		ret = snprintf_append(&sysfs_path_buf, "/%s/%s%d/dev",
+				      disk_name, disk_name, part);
+	if (ret)
+		return ret;
+
+	return lkl_encode_dev_from_sysfs(sysfs_path, pdevid);
+}
+
+int lkl_get_blkdev(int disk_id, unsigned int part, uint32_t *pdevid)
+{
+	char *fmt;
+
+	fmt = SYSFS_DEV_UMBLK_CMDLINE_PATH;
+	return __lkl_get_blkdev(disk_id, part, pdevid, fmt, "", "ubd");
+}
+
+long lkl_mount_dev(unsigned int disk_id, unsigned int part,
+		   const char *fs_type, int flags,
+		   const char *data, char *mnt_str, unsigned int mnt_str_len)
+{
+	char dev_str[] = { "/dev/xxxxxxxx" };
+	unsigned int dev;
+	int err;
+	char _data[4096]; /* FIXME: PAGE_SIZE is not exported by LKL */
+
+	if (mnt_str_len < sizeof(dev_str))
+		return -LKL_ENOMEM;
+
+	err = lkl_get_blkdev(disk_id, part, &dev);
+	if (err < 0)
+		return err;
+
+	snprintf(dev_str, sizeof(dev_str), "/dev/%08x", dev);
+	snprintf(mnt_str, mnt_str_len, "/mnt/%08x", dev);
+
+	err = lkl_sys_access("/dev", LKL_S_IRWXO);
+	if (err < 0) {
+		if (err == -LKL_ENOENT)
+			err = lkl_sys_mkdir("/dev", 0700);
+		if (err < 0)
+			return err;
+	}
+
+	err = lkl_sys_mknod(dev_str, LKL_S_IFBLK | 0600, dev);
+	if (err < 0)
+		return err;
+
+	err = lkl_sys_access("/mnt", LKL_S_IRWXO);
+	if (err < 0) {
+		if (err == -LKL_ENOENT)
+			err = lkl_sys_mkdir("/mnt", 0700);
+		if (err < 0)
+			return err;
+	}
+
+	err = lkl_sys_mkdir(mnt_str, 0700);
+	if (err < 0) {
+		lkl_sys_unlink(dev_str);
+		return err;
+	}
+
+	/* kernel always copies a full page */
+	if (data) {
+		strncpy(_data, data, sizeof(_data));
+		_data[sizeof(_data) - 1] = 0;
+	} else {
+		_data[0] = 0;
+	}
+
+	err = lkl_sys_mount(dev_str, mnt_str, (char *)fs_type, flags, _data);
+	if (err < 0) {
+		lkl_sys_unlink(dev_str);
+		lkl_sys_rmdir(mnt_str);
+		return err;
+	}
+
+	return 0;
+}
+
+long lkl_umount_timeout(char *path, int flags, long timeout_ms)
+{
+	long incr = 10000000; /* 10 ms */
+	struct lkl_timespec ts = {
+		.tv_sec = 0,
+		.tv_nsec = incr,
+	};
+	long err;
+
+	do {
+		err = lkl_sys_umount(path, flags);
+		if (err == -LKL_EBUSY) {
+			lkl_sys_nanosleep((struct __lkl__kernel_timespec *)&ts,
+					  NULL);
+			timeout_ms -= incr / 1000000;
+		}
+	} while (err == -LKL_EBUSY && timeout_ms > 0);
+
+	return err;
+}
+
+long lkl_umount_dev(unsigned int disk_id, unsigned int part, int flags,
+		    long timeout_ms)
+{
+	char dev_str[] = { "/dev/xxxxxxxx" };
+	char mnt_str[] = { "/mnt/xxxxxxxx" };
+	unsigned int dev;
+	int err;
+
+	err = lkl_get_blkdev(disk_id, part, &dev);
+	if (err < 0)
+		return err;
+
+	snprintf(dev_str, sizeof(dev_str), "/dev/%08x", dev);
+	snprintf(mnt_str, sizeof(mnt_str), "/mnt/%08x", dev);
+
+	err = lkl_umount_timeout(mnt_str, flags, timeout_ms);
+	if (err)
+		return err;
+
+	err = lkl_sys_unlink(dev_str);
+	if (err)
+		return err;
+
+	return lkl_sys_rmdir(mnt_str);
+}
+
+struct lkl_dir {
+	int fd;
+	char buf[1024];
+	char *pos;
+	int len;
+};
+
+static struct lkl_dir *lkl_dir_alloc(int *err)
+{
+	struct lkl_dir *dir = lkl_host_ops.mem_alloc(sizeof(struct lkl_dir));
+
+	if (!dir) {
+		*err = -LKL_ENOMEM;
+		return NULL;
+	}
+
+	dir->len = 0;
+	dir->pos = NULL;
+
+	return dir;
+}
+
+struct lkl_dir *lkl_opendir(const char *path, int *err)
+{
+	struct lkl_dir *dir = lkl_dir_alloc(err);
+
+	if (!dir) {
+		*err = -LKL_ENOMEM;
+		return NULL;
+	}
+
+	dir->fd = lkl_sys_open(path, LKL_O_RDONLY | LKL_O_DIRECTORY, 0);
+	if (dir->fd < 0) {
+		*err = dir->fd;
+		lkl_host_ops.mem_free(dir);
+		return NULL;
+	}
+
+	*err = 0;
+
+	return dir;
+}
+
+struct lkl_dir *lkl_fdopendir(int fd, int *err)
+{
+	struct lkl_dir *dir = lkl_dir_alloc(err);
+
+	if (!dir)
+		return NULL;
+
+	dir->fd = fd;
+
+	return dir;
+}
+
+void lkl_rewinddir(struct lkl_dir *dir)
+{
+	lkl_sys_lseek(dir->fd, 0, LKL_SEEK_SET);
+	dir->len = 0;
+	dir->pos = NULL;
+}
+
+int lkl_closedir(struct lkl_dir *dir)
+{
+	int ret;
+
+	ret = lkl_sys_close(dir->fd);
+	lkl_host_ops.mem_free(dir);
+
+	return ret;
+}
+
+struct lkl_linux_dirent64 *lkl_readdir(struct lkl_dir *dir)
+{
+	struct lkl_linux_dirent64 *de;
+
+	if (dir->len < 0)
+		return NULL;
+
+	if (!dir->pos || dir->pos - dir->buf >= dir->len)
+		goto read_buf;
+
+return_de:
+	de = (struct lkl_linux_dirent64 *)dir->pos;
+	dir->pos += de->d_reclen;
+
+	return de;
+
+read_buf:
+	dir->pos = NULL;
+	de = (struct lkl_linux_dirent64 *)dir->buf;
+	dir->len = lkl_sys_getdents64(dir->fd, de, sizeof(dir->buf));
+	if (dir->len <= 0)
+		return NULL;
+
+	dir->pos = dir->buf;
+	goto return_de;
+}
+
+int lkl_errdir(struct lkl_dir *dir)
+{
+	if (dir->len >= 0)
+		return 0;
+
+	return dir->len;
+}
+
+int lkl_dirfd(struct lkl_dir *dir)
+{
+	return dir->fd;
+}
+
+int lkl_set_fd_limit(unsigned int fd_limit)
+{
+	struct lkl_rlimit rlim = {
+		.rlim_cur = fd_limit,
+		.rlim_max = fd_limit,
+	};
+	return lkl_sys_setrlimit(LKL_RLIMIT_NOFILE, &rlim);
+}
diff --git a/tools/um/lib/posix-host.c b/tools/um/lib/posix-host.c
index b6b5b2902254..8fd88031bf2b 100644
--- a/tools/um/lib/posix-host.c
+++ b/tools/um/lib/posix-host.c
@@ -263,6 +263,7 @@ static void *tls_get(struct lkl_tls_key *key)
 }
 
 struct lkl_host_operations lkl_host_ops = {
+	.um_devices = lkl_um_devs,
 	.panic = panic,
 	.print = print,
 	.mem_alloc = (void *)malloc,
diff --git a/tools/um/lib/utils.c b/tools/um/lib/utils.c
index 4930479a8a35..ac65cd744a14 100644
--- a/tools/um/lib/utils.c
+++ b/tools/um/lib/utils.c
@@ -4,6 +4,9 @@
 #include <string.h>
 #include <lkl_host.h>
 
+/* XXX: find a better place */
+char lkl_um_devs[4096];
+
 static const char * const lkl_err_strings[] = {
 	"Success",
 	"Operation not permitted",
diff --git a/tools/um/tests/Build b/tools/um/tests/Build
index 564560486f98..1aa78f9ed7ef 100644
--- a/tools/um/tests/Build
+++ b/tools/um/tests/Build
@@ -1,3 +1,4 @@
 boot-y += boot.o test.o
+disk-y += disk.o test.o cla.o
 
 CFLAGS_test.o += -Wno-implicit-fallthrough
diff --git a/tools/um/tests/cla.c b/tools/um/tests/cla.c
new file mode 100644
index 000000000000..694e3a3822be
--- /dev/null
+++ b/tools/um/tests/cla.c
@@ -0,0 +1,159 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <string.h>
+#include <errno.h>
+#include <stdlib.h>
+#ifdef __MINGW32__
+#include <winsock2.h>
+#else
+#include <sys/socket.h>
+#include <netinet/in.h>
+#include <arpa/inet.h>
+#endif
+
+#include "cla.h"
+
+static int cl_arg_parse_bool(struct cl_arg *arg, const char *value)
+{
+	*((int *)arg->store) = 1;
+	return 0;
+}
+
+static int cl_arg_parse_str(struct cl_arg *arg, const char *value)
+{
+	*((const char **)arg->store) = value;
+	return 0;
+}
+
+static int cl_arg_parse_int(struct cl_arg *arg, const char *value)
+{
+	errno = 0;
+	*((int *)arg->store) = strtol(value, NULL, 0);
+	return errno == 0;
+}
+
+static int cl_arg_parse_str_set(struct cl_arg *arg, const char *value)
+{
+	const char **set = arg->set;
+	int i;
+
+	for (i = 0; set[i] != NULL; i++) {
+		if (strcmp(set[i], value) == 0) {
+			*((int *)arg->store) = i;
+			return 0;
+		}
+	}
+
+	return (-1);
+}
+
+static int cl_arg_parse_ipv4(struct cl_arg *arg, const char *value)
+{
+	unsigned int addr;
+
+	if (!value)
+		return (-1);
+
+	addr = inet_addr(value);
+	if (addr == INADDR_NONE)
+		return (-1);
+	*((unsigned int *)arg->store) = addr;
+	return 0;
+}
+
+static cl_arg_parser_t parsers[] = {
+	[CL_ARG_BOOL] = cl_arg_parse_bool,
+	[CL_ARG_INT] = cl_arg_parse_int,
+	[CL_ARG_STR] = cl_arg_parse_str,
+	[CL_ARG_STR_SET] = cl_arg_parse_str_set,
+	[CL_ARG_IPV4] = cl_arg_parse_ipv4,
+};
+
+static struct cl_arg *find_short_arg(char name, struct cl_arg *args)
+{
+	struct cl_arg *arg;
+
+	for (arg = args; arg->short_name != 0; arg++) {
+		if (arg->short_name == name)
+			return arg;
+	}
+
+	return NULL;
+}
+
+static struct cl_arg *find_long_arg(const char *name, struct cl_arg *args)
+{
+	struct cl_arg *arg;
+
+	for (arg = args; arg->long_name; arg++) {
+		if (strcmp(arg->long_name, name) == 0)
+			return arg;
+	}
+
+	return NULL;
+}
+
+static void print_help(struct cl_arg *args)
+{
+	struct cl_arg *arg;
+
+	fprintf(stderr, "usage:\n");
+	for (arg = args; arg->long_name; arg++) {
+		fprintf(stderr, "-%c, --%-20s %s", arg->short_name,
+			arg->long_name, arg->help);
+		if (arg->type == CL_ARG_STR_SET) {
+			const char **set = arg->set;
+
+			fprintf(stderr, " [ ");
+			while (*set != NULL)
+				fprintf(stderr, "%s ", *(set++));
+			fprintf(stderr, "]");
+		}
+		fprintf(stderr, "\n");
+	}
+}
+
+int cla_parse_args(int argc, const char **argv, struct cl_arg *args)
+{
+	int i;
+
+	for (i = 1; i < argc; i++) {
+		struct cl_arg *arg = NULL;
+		cl_arg_parser_t parser;
+
+		if (argv[i][0] == '-') {
+			if (argv[i][1] != '-')
+				arg = find_short_arg(argv[i][1], args);
+			else
+				arg = find_long_arg(&argv[i][2], args);
+		}
+
+		if (!arg) {
+			fprintf(stderr, "unknown option '%s'\n", argv[i]);
+			print_help(args);
+			return (-1);
+		}
+
+		if (arg->type == CL_ARG_USER || arg->type >= CL_ARG_END)
+			parser = arg->parser;
+		else
+			parser = parsers[arg->type];
+
+		if (!parser) {
+			fprintf(stderr, "can't parse --'%s'/-'%c'\n",
+				arg->long_name, args->short_name);
+			return (-1);
+		}
+
+		if (parser(arg, argv[i + 1]) < 0) {
+			fprintf(stderr, "can't parse '%s'\n", argv[i]);
+			print_help(args);
+			return (-1);
+		}
+
+		if (arg->has_arg)
+			i++;
+	}
+
+	return 0;
+}
diff --git a/tools/um/tests/cla.h b/tools/um/tests/cla.h
new file mode 100644
index 000000000000..3d879233681f
--- /dev/null
+++ b/tools/um/tests/cla.h
@@ -0,0 +1,33 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef _LKL_TEST_CLA_H
+#define _LKL_TEST_CLA_H
+
+enum cl_arg_type {
+	CL_ARG_USER = 0,
+	CL_ARG_BOOL,
+	CL_ARG_INT,
+	CL_ARG_STR,
+	CL_ARG_STR_SET,
+	CL_ARG_IPV4,
+	CL_ARG_END,
+};
+
+struct cl_arg;
+
+typedef int (*cl_arg_parser_t)(struct cl_arg *arg, const char *value);
+
+struct cl_arg {
+	const char *long_name;
+	char short_name;
+	const char *help;
+	int has_arg;
+	enum cl_arg_type type;
+	void *store;
+	void *set;
+	cl_arg_parser_t parser;
+};
+
+int cla_parse_args(int argc, const char **argv, struct cl_arg *args);
+
+
+#endif /* _LKL_TEST_CLA_H */
diff --git a/tools/um/tests/disk.c b/tools/um/tests/disk.c
new file mode 100644
index 000000000000..325934935e6a
--- /dev/null
+++ b/tools/um/tests/disk.c
@@ -0,0 +1,168 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <unistd.h>
+#include <string.h>
+#include <time.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <lkl.h>
+#include <lkl_host.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <sys/ioctl.h>
+
+#include "test.h"
+#include "cla.h"
+
+static struct {
+	int printk;
+	const char *disk;
+	const char *fstype;
+	int partition;
+} cla;
+
+struct cl_arg args[] = {
+	{"disk", 'd', "disk file to use", 1, CL_ARG_STR, &cla.disk},
+	{"partition", 'P', "partition to mount", 1, CL_ARG_INT, &cla.partition},
+	{"type", 't', "filesystem type", 1, CL_ARG_STR, &cla.fstype},
+	{0},
+};
+
+
+static struct lkl_disk disk;
+static int disk_id = -1;
+
+int lkl_test_disk_add(void)
+{
+	disk.fd = open(cla.disk, O_RDWR);
+	if (disk.fd < 0)
+		goto out_unlink;
+
+	disk.ops = NULL;
+	disk.dev = (char *)cla.disk;
+
+	disk_id = lkl_disk_add(&disk);
+	if (disk_id < 0)
+		goto out_close;
+
+	goto out;
+
+out_close:
+	close(disk.fd);
+
+out_unlink:
+	unlink(cla.disk);
+
+out:
+	lkl_test_logf("disk fd/handle %x disk_id %d", disk.fd, disk_id);
+
+	if (disk_id >= 0)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+int lkl_test_disk_remove(void)
+{
+	int ret;
+
+	ret = lkl_disk_remove(disk);
+
+	close(disk.fd);
+
+	if (ret == 0)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+
+static char mnt_point[32];
+
+LKL_TEST_CALL(mount_dev, lkl_mount_dev, 0, disk_id, cla.partition, cla.fstype,
+	      0, NULL, mnt_point, sizeof(mnt_point))
+
+static int lkl_test_umount_dev(void)
+{
+	long ret, ret2;
+
+	ret = lkl_sys_chdir("/");
+
+	ret2 = lkl_umount_dev(disk_id, cla.partition, 0, 1000);
+
+	lkl_test_logf("%ld %ld", ret, ret2);
+
+	if (!ret && !ret2)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+struct lkl_dir *dir;
+
+static int lkl_test_opendir(void)
+{
+	int err;
+
+	dir = lkl_opendir(mnt_point, &err);
+
+	lkl_test_logf("lkl_opedir(%s) = %d %s\n", mnt_point, err,
+		      lkl_strerror(err));
+
+	if (err == 0)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+static int lkl_test_readdir(void)
+{
+	struct lkl_linux_dirent64 *de = lkl_readdir(dir);
+	int wr = 0;
+
+	while (de) {
+		wr += lkl_test_logf("%s ", de->d_name);
+		if (wr >= 70) {
+			lkl_test_logf("\n");
+			wr = 0;
+			break;
+		}
+		de = lkl_readdir(dir);
+	}
+
+	if (lkl_errdir(dir) == 0)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+LKL_TEST_CALL(closedir, lkl_closedir, 0, dir);
+LKL_TEST_CALL(chdir_mnt_point, lkl_sys_chdir, 0, mnt_point);
+LKL_TEST_CALL(start_kernel, lkl_start_kernel, 0, &lkl_host_ops,
+	     "mem=16M loglevel=8");
+LKL_TEST_CALL(stop_kernel, lkl_sys_halt, 0);
+
+struct lkl_test tests[] = {
+	LKL_TEST(disk_add),
+	LKL_TEST(start_kernel),
+	LKL_TEST(mount_dev),
+	LKL_TEST(chdir_mnt_point),
+	LKL_TEST(opendir),
+	LKL_TEST(readdir),
+	LKL_TEST(closedir),
+	LKL_TEST(umount_dev),
+	LKL_TEST(stop_kernel),
+	LKL_TEST(disk_remove),
+
+};
+
+int main(int argc, const char **argv)
+{
+	if (cla_parse_args(argc, argv, args) < 0)
+		return (-1);
+
+	lkl_host_ops.print = lkl_test_log;
+
+	return lkl_test_run(tests, sizeof(tests)/sizeof(struct lkl_test),
+			    "disk %s", cla.fstype);
+}
diff --git a/tools/um/tests/disk.sh b/tools/um/tests/disk.sh
new file mode 100755
index 000000000000..e2ec6cf69d4b
--- /dev/null
+++ b/tools/um/tests/disk.sh
@@ -0,0 +1,70 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+
+source $script_dir/test.sh
+
+function prepfs()
+{
+    set -e
+
+    file=`mktemp`
+
+    dd if=/dev/zero of=$file bs=1024 count=204800
+
+    yes | mkfs.$1 $file
+
+    if ! [ -z $ANDROID_WDIR ]; then
+        adb shell mkdir -p $ANDROID_WDIR
+        adb push $file $ANDROID_WDIR
+        rm $file
+        file=$ANDROID_WDIR/$(basename $file)
+    fi
+    if ! [ -z $BSD_WDIR ]; then
+        $MYSSH mkdir -p $BSD_WDIR
+        ssh_copy $file $BSD_WDIR
+        rm $file
+        file=$BSD_WDIR/$(basename $file)
+    fi
+
+    export_vars file
+}
+
+function cleanfs()
+{
+    set -e
+
+    if ! [ -z $ANDROID_WDIR ]; then
+        adb shell rm $1
+        adb shell rm $ANDROID_WDIR/disk
+    elif ! [ -z $BSD_WDIR ]; then
+        $MYSSH rm $1
+        $MYSSH rm $BSD_WDIR/disk
+    else
+        rm $1
+    fi
+}
+
+if [ "$1" = "-t" ]; then
+    shift
+    fstype=$1
+    shift
+fi
+
+if [ -z "$fstype" ]; then
+    fstype="ext4"
+fi
+
+if [ -z $(which mkfs.$fstype) ]; then
+    lkl_test_plan 0 "disk $fstype"
+    echo "no mkfs.$fstype command"
+    exit 0
+fi
+
+lkl_test_plan 1 "disk $fstype"
+lkl_test_run 1 prepfs $fstype
+lkl_test_exec $script_dir/disk -d $file -t $fstype $@
+lkl_test_plan 1 "disk $fstype"
+lkl_test_run 1 cleanfs $file
+
diff --git a/tools/um/tests/run.py b/tools/um/tests/run.py
index c96ede90b6ad..97d6dedc217c 100755
--- a/tools/um/tests/run.py
+++ b/tools/um/tests/run.py
@@ -50,6 +50,8 @@ mydir=os.path.dirname(os.path.realpath(__file__))
 
 tests = [
     'boot.sh',
+    'disk.sh -t ext4',
+    'disk.sh -t vfat',
 ]
 
 parser = argparse.ArgumentParser(description='LKL test runner')
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v7 21/21] um: nommu: add block device support of UML
@ 2020-10-06  9:44           ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-06  9:44 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Hajime Tazaki, retrage01

This commit adds block device support for nommu mode, and also added
several host utilities to run a simple test program
(tools/um/test/disk.c) successfully.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/include/asm/xor.h                 |   3 +-
 arch/um/include/shared/as-layout.h        |   1 +
 arch/um/kernel/um_arch.c                  |   5 +
 arch/um/nommu/include/uapi/asm/host_ops.h |   7 +
 arch/um/nommu/include/uapi/asm/unistd.h   |   2 +
 arch/um/nommu/um/Kconfig                  |   8 +
 arch/um/nommu/um/setup.c                  |  12 +
 tools/um/Targets                          |   2 +
 tools/um/include/lkl.h                    | 206 ++++++++++
 tools/um/include/lkl_host.h               |   1 +
 tools/um/lib/Build                        |   1 +
 tools/um/lib/fs.c                         | 461 ++++++++++++++++++++++
 tools/um/lib/posix-host.c                 |   1 +
 tools/um/lib/utils.c                      |   3 +
 tools/um/tests/Build                      |   1 +
 tools/um/tests/cla.c                      | 159 ++++++++
 tools/um/tests/cla.h                      |  33 ++
 tools/um/tests/disk.c                     | 168 ++++++++
 tools/um/tests/disk.sh                    |  70 ++++
 tools/um/tests/run.py                     |   2 +
 20 files changed, 1145 insertions(+), 1 deletion(-)
 create mode 100644 tools/um/lib/fs.c
 create mode 100644 tools/um/tests/cla.c
 create mode 100644 tools/um/tests/cla.h
 create mode 100644 tools/um/tests/disk.c
 create mode 100755 tools/um/tests/disk.sh

diff --git a/arch/um/include/asm/xor.h b/arch/um/include/asm/xor.h
index 36b33d62a35d..a5ea458a1ae9 100644
--- a/arch/um/include/asm/xor.h
+++ b/arch/um/include/asm/xor.h
@@ -4,4 +4,5 @@
 
 /* pick an arbitrary one - measuring isn't possible with inf-cpu */
 #define XOR_SELECT_TEMPLATE(x)	\
-	(time_travel_mode == TT_MODE_INFCPU ? &xor_block_8regs : NULL)
+	(time_travel_mode == TT_MODE_INFCPU || (!IS_ENABLED(CONFIG_MMU)) ? \
+	 &xor_block_8regs : NULL)
diff --git a/arch/um/include/shared/as-layout.h b/arch/um/include/shared/as-layout.h
index 5f286ef2721b..4423437a5ace 100644
--- a/arch/um/include/shared/as-layout.h
+++ b/arch/um/include/shared/as-layout.h
@@ -57,6 +57,7 @@ extern unsigned long host_task_size;
 
 extern int linux_main(int argc, char **argv);
 extern void uml_finishsetup(void);
+extern void uml_set_args(char *args);
 
 struct siginfo;
 extern void (*sig_info[])(int, struct siginfo *si, struct uml_pt_regs *);
diff --git a/arch/um/kernel/um_arch.c b/arch/um/kernel/um_arch.c
index e2cb76c03b25..da6e06cf2808 100644
--- a/arch/um/kernel/um_arch.c
+++ b/arch/um/kernel/um_arch.c
@@ -41,6 +41,11 @@ static void __init add_arg(char *arg)
 	strcat(command_line, arg);
 }
 
+void __init uml_set_args(char *args)
+{
+	strcat(command_line, args);
+}
+
 /*
  * These fields are initialized at boot time and not changed.
  * XXX This structure is used only in the non-SMP case.  Maybe this
diff --git a/arch/um/nommu/include/uapi/asm/host_ops.h b/arch/um/nommu/include/uapi/asm/host_ops.h
index 0811494c080c..1b2507502969 100644
--- a/arch/um/nommu/include/uapi/asm/host_ops.h
+++ b/arch/um/nommu/include/uapi/asm/host_ops.h
@@ -15,6 +15,11 @@ struct lkl_jmp_buf {
  *
  * These operations must be provided by a host library or by the application
  * itself.
+ *
+ * @um_devices - string containg the list of UML devices in command line
+ * format. This string is appended to the kernel command line and
+ * is provided here for convenience to be implemented by the host library.
+ *
  * @print - optional operation that receives console messages
  * @panic - called during a kernel panic
  *
@@ -65,6 +70,8 @@ struct lkl_jmp_buf {
  *
  */
 struct lkl_host_operations {
+	const char *um_devices;
+
 	void (*print)(const char *str, int len);
 	void (*panic)(void);
 
diff --git a/arch/um/nommu/include/uapi/asm/unistd.h b/arch/um/nommu/include/uapi/asm/unistd.h
index 1762ebb829f5..320762099a62 100644
--- a/arch/um/nommu/include/uapi/asm/unistd.h
+++ b/arch/um/nommu/include/uapi/asm/unistd.h
@@ -3,6 +3,8 @@
 #define __UM_NOMMU_UAPI_UNISTD_H
 
 #define __ARCH_WANT_NEW_STAT
+#define __ARCH_WANT_SET_GET_RLIMIT
+
 #include <asm/bitsperlong.h>
 
 #if __BITS_PER_LONG == 64
diff --git a/arch/um/nommu/um/Kconfig b/arch/um/nommu/um/Kconfig
index 20b3eaccb6f0..c6a3f472fe75 100644
--- a/arch/um/nommu/um/Kconfig
+++ b/arch/um/nommu/um/Kconfig
@@ -4,6 +4,10 @@ config UML_NOMMU
 	select UACCESS_MEMCPY
 	select ARCH_THREAD_STACK_ALLOCATOR
 	select ARCH_HAS_SYSCALL_WRAPPER
+	select VFAT_FS
+	select NLS_CODEPAGE_437
+	select NLS_ISO8859_1
+	select BTRFS_FS
 
 config 64BIT
 	bool
@@ -35,3 +39,7 @@ config STACKTRACE_SUPPORT
 config PRINTK_TIME
 	bool
 	default y
+
+config RAID6_PQ_BENCHMARK
+	bool
+	default n
diff --git a/arch/um/nommu/um/setup.c b/arch/um/nommu/um/setup.c
index 922188690139..d71d61979ad6 100644
--- a/arch/um/nommu/um/setup.c
+++ b/arch/um/nommu/um/setup.c
@@ -45,13 +45,25 @@ static void __init *lkl_run_kernel(void *arg)
 	return NULL;
 }
 
+static char _cmd_line[COMMAND_LINE_SIZE];
 int __init lkl_start_kernel(struct lkl_host_operations *ops,
 			    const char *fmt, ...)
 {
+	va_list ap;
 	int ret;
 
 	lkl_ops = ops;
 
+	va_start(ap, fmt);
+	ret = vsnprintf(_cmd_line, COMMAND_LINE_SIZE, fmt, ap);
+	va_end(ap);
+
+	if (ops->um_devices)
+		strscpy(_cmd_line + ret, ops->um_devices,
+			COMMAND_LINE_SIZE - ret);
+
+	uml_set_args(_cmd_line);
+
 	init_sem = lkl_ops->sem_alloc(0);
 	if (!init_sem)
 		return -ENOMEM;
diff --git a/tools/um/Targets b/tools/um/Targets
index 2bb90381843c..f5f8ec4b9dbb 100644
--- a/tools/um/Targets
+++ b/tools/um/Targets
@@ -1,10 +1,12 @@
 ifeq ($(UMMODE),library)
 progs-y += tests/boot
+progs-y += tests/disk
 else
 progs-y += uml/linux
 endif
 
 LDFLAGS_boot-y := -pie
+LDFLAGS_disk-y := -pie
 
 LDLIBS := -lrt -lpthread -lutil
 LDFLAGS_linux-y := -no-pie -Wl,--wrap,malloc -Wl,--wrap,free -Wl,--wrap,calloc
diff --git a/tools/um/include/lkl.h b/tools/um/include/lkl.h
index 707e01b64a70..4a9d874bfbf0 100644
--- a/tools/um/include/lkl.h
+++ b/tools/um/include/lkl.h
@@ -113,6 +113,16 @@ static inline long lkl_sys_creat(const char *file, int mode)
 }
 #endif
 
+#ifdef __lkl__NR_faccessat
+/**
+ * lkl_sys_access - wrapper for lkl_sys_faccessat
+ */
+static inline long lkl_sys_access(const char *file, int mode)
+{
+	return lkl_sys_faccessat(LKL_AT_FDCWD, file, mode);
+}
+#endif
+
 #ifdef __lkl__NR_mkdirat
 /**
  * lkl_sys_mkdir - wrapper for lkl_sys_mkdirat
@@ -123,6 +133,36 @@ static inline long lkl_sys_mkdir(const char *path, mode_t mode)
 }
 #endif
 
+#ifdef __lkl__NR_unlinkat
+/**
+ * lkl_sys_rmdir - wrapper for lkl_sys_unlinkrat
+ */
+static inline long lkl_sys_rmdir(const char *path)
+{
+	return lkl_sys_unlinkat(LKL_AT_FDCWD, path, LKL_AT_REMOVEDIR);
+}
+#endif
+
+#ifdef __lkl__NR_unlinkat
+/**
+ * lkl_sys_unlink - wrapper for lkl_sys_unlinkat
+ */
+static inline long lkl_sys_unlink(const char *path)
+{
+	return lkl_sys_unlinkat(LKL_AT_FDCWD, path, 0);
+}
+#endif
+
+#ifdef __lkl__NR_mknodat
+/**
+ * lkl_sys_mknod - wrapper for lkl_sys_mknodat
+ */
+static inline long lkl_sys_mknod(const char *path, mode_t mode, dev_t dev)
+{
+	return lkl_sys_mknodat(LKL_AT_FDCWD, path, mode, dev);
+}
+#endif
+
 #ifdef __lkl__NR_epoll_create1
 /**
  * lkl_sys_epoll_create - wrapper for lkl_sys_epoll_create1
@@ -144,6 +184,172 @@ static inline long lkl_sys_epoll_wait(int fd, struct lkl_epoll_event *ev,
 }
 #endif
 
+/**
+ * struct lkl_dev_blk_ops - block device host operations, defined in lkl_host.h.
+ */
+struct lkl_dev_blk_ops;
+
+/**
+ * lkl_disk - host disk handle
+ *
+ * @dev - a pointer to private information for this disk backend
+ * @fd - a POSIX file descriptor that can be used by preadv/pwritev
+ * @handle - an NT file handle that can be used by ReadFile/WriteFile
+ */
+struct lkl_disk {
+	void *dev;
+	union {
+		int fd;
+		void *handle;
+	};
+	struct lkl_dev_blk_ops *ops;
+};
+
+/**
+ * lkl_disk_add - add a new disk
+ *
+ * @disk - the host disk handle
+ * @returns a disk id (0 is valid) or a strictly negative value in case of error
+ */
+int lkl_disk_add(struct lkl_disk *disk);
+
+/**
+ * lkl_disk_remove - remove a disk
+ *
+ * This function makes a cleanup of the @disk's private information
+ * that was initialized by lkl_disk_add before.
+ *
+ * @disk - the host disk handle
+ */
+int lkl_disk_remove(struct lkl_disk disk);
+
+/**
+ * lkl_encode_dev_from_sysfs_blkdev - extract device id from sysfs
+ *
+ * This function returns the device id for the given sysfs dev node.
+ * The content of the node has to be in the form 'MAJOR:MINOR'.
+ * Also, this function expects an absolute path which means that sysfs
+ * already has to be mounted at the given path
+ *
+ * @sysfs_path - absolute path to the sysfs dev node
+ * @pdevid - pointer to memory where dev id will be returned
+ * @returns - 0 on success, a negative value on error
+ */
+int lkl_encode_dev_from_sysfs(const char *sysfs_path, uint32_t *pdevid);
+
+/**
+ * lkl_mount_dev - mount a disk
+ *
+ * This functions creates a device file for the given disk, creates a mount
+ * point and mounts the device over the mount point.
+ *
+ * @disk_id - the disk id identifying the disk to be mounted
+ * @part - disk partition or zero for full disk
+ * @fs_type - filesystem type
+ * @flags - mount flags
+ * @opts - additional filesystem specific mount options
+ * @mnt_str - a string that will be filled by this function with the path where
+ * the filesystem has been mounted
+ * @mnt_str_len - size of mnt_str
+ * @returns - 0 on success, a negative value on error
+ */
+long lkl_mount_dev(unsigned int disk_id, unsigned int part, const char *fs_type,
+		   int flags, const char *opts,
+		   char *mnt_str, unsigned int mnt_str_len);
+
+/**
+ * lkl_umount_dev - umount a disk
+ *
+ * This functions umounts the given disks and removes the device file and the
+ * mount point.
+ *
+ * @disk_id - the disk id identifying the disk to be mounted
+ * @part - disk partition or zero for full disk
+ * @flags - umount flags
+ * @timeout_ms - timeout to wait for the kernel to flush closed files so that
+ * umount can succeed
+ * @returns - 0 on success, a negative value on error
+ */
+long lkl_umount_dev(unsigned int disk_id, unsigned int part, int flags,
+		    long timeout_ms);
+
+/**
+ * lkl_umount_timeout - umount filesystem with timeout
+ *
+ * @path - the path to unmount
+ * @flags - umount flags
+ * @timeout_ms - timeout to wait for the kernel to flush closed files so that
+ * umount can succeed
+ * @returns - 0 on success, a negative value on error
+ */
+long lkl_umount_timeout(char *path, int flags, long timeout_ms);
+
+/**
+ * lkl_opendir - open a directory
+ *
+ * @path - directory path
+ * @err - pointer to store the error in case of failure
+ * @returns - a handle to be used when calling lkl_readdir
+ */
+struct lkl_dir *lkl_opendir(const char *path, int *err);
+
+/**
+ * lkl_fdopendir - open a directory
+ *
+ * @fd - file descriptor
+ * @err - pointer to store the error in case of failure
+ * @returns - a handle to be used when calling lkl_readdir
+ */
+struct lkl_dir *lkl_fdopendir(int fd, int *err);
+
+/**
+ * lkl_rewinddir - reset directory stream
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ */
+void lkl_rewinddir(struct lkl_dir *dir);
+
+/**
+ * lkl_closedir - close the directory
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ */
+int lkl_closedir(struct lkl_dir *dir);
+
+/**
+ * lkl_readdir - get the next available entry of the directory
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ * @returns - a lkl_dirent64 entry or NULL if the end of the directory stream is
+ * reached or if an error occurred; check lkl_errdir() to distinguish between
+ * errors or end of the directory stream
+ */
+struct lkl_linux_dirent64 *lkl_readdir(struct lkl_dir *dir);
+
+/**
+ * lkl_errdir - checks if an error occurred during the last lkl_readdir call
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ * @returns - 0 if no error occurred, or a negative value otherwise
+ */
+int lkl_errdir(struct lkl_dir *dir);
+
+/**
+ * lkl_dirfd - gets the file descriptor associated with the directory handle
+ *
+ * @dir - the directory handle as returned by lkl_opendir
+ * @returns - a positive value,which is the LKL file descriptor associated with
+ * the directory handle, or a negative value otherwise
+ */
+int lkl_dirfd(struct lkl_dir *dir);
+
+/**
+ * lkl_mount_fs - mount a file system type like proc, sys
+ * @fstype - file system type. e.g. proc, sys
+ * @returns - 0 on success. 1 if it's already mounted. negative on failure.
+ */
+int lkl_mount_fs(char *fstype);
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/tools/um/include/lkl_host.h b/tools/um/include/lkl_host.h
index 85e80eb4ad0d..12dd95616e43 100644
--- a/tools/um/include/lkl_host.h
+++ b/tools/um/include/lkl_host.h
@@ -10,6 +10,7 @@ extern "C" {
 #include <lkl.h>
 
 extern struct lkl_host_operations lkl_host_ops;
+extern char lkl_um_devs[4096];
 
 /**
  * lkl_printf - print a message via the host print operation
diff --git a/tools/um/lib/Build b/tools/um/lib/Build
index dddff26a3b4e..29491b40746c 100644
--- a/tools/um/lib/Build
+++ b/tools/um/lib/Build
@@ -4,3 +4,4 @@ CFLAGS_posix-host.o += -D_FILE_OFFSET_BITS=64
 liblinux-$(CONFIG_UMMODE_LIB) += utils.o
 liblinux-$(CONFIG_UMMODE_LIB) += posix-host.o
 liblinux-$(CONFIG_UMMODE_LIB) += jmp_buf.o
+liblinux-$(CONFIG_UMMODE_LIB) += fs.o
diff --git a/tools/um/lib/fs.c b/tools/um/lib/fs.c
new file mode 100644
index 000000000000..948aac9730c2
--- /dev/null
+++ b/tools/um/lib/fs.c
@@ -0,0 +1,461 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdarg.h>
+#include <stdio.h>
+#include <string.h>
+#include <stdlib.h>
+#include <lkl_host.h>
+
+#define MAX_FSTYPE_LEN 50
+
+static struct lkl_disk *lkl_disks[16];
+static int registered_blk_dev_idx;
+
+static int lkl_disk_um_add(struct lkl_disk *disk, const char *blkparams)
+{
+	/* concat strings */
+	snprintf(lkl_um_devs + strlen(lkl_um_devs), sizeof(lkl_um_devs),
+		 " ubd%d=%s", registered_blk_dev_idx, blkparams);
+
+	return registered_blk_dev_idx++;
+}
+
+int lkl_disk_add(struct lkl_disk *disk)
+{
+	int ret = -1;
+
+	ret = lkl_disk_um_add(disk, disk->dev);
+
+	lkl_disks[ret] = disk;
+
+	return ret;
+}
+
+int lkl_disk_remove(struct lkl_disk disk)
+{
+	/* FIXME */
+	return 0;
+}
+
+int lkl_mount_fs(char *fstype)
+{
+	char dir[MAX_FSTYPE_LEN+2] = "/";
+	int flags = 0, ret = 0;
+
+	strncat(dir, fstype, MAX_FSTYPE_LEN);
+
+	/* Create with regular umask */
+	ret = lkl_sys_mkdir(dir, 0xff);
+	if (ret && ret != -LKL_EEXIST) {
+		lkl_perror("mount_fs mkdir", ret);
+		return ret;
+	}
+
+	/* We have no use for nonzero flags right now */
+	ret = lkl_sys_mount("none", dir, fstype, flags, NULL);
+	if (ret && ret != -LKL_EBUSY) {
+		lkl_sys_rmdir(dir);
+		return ret;
+	}
+
+	if (ret == -LKL_EBUSY)
+		return 1;
+	return 0;
+}
+
+static uint32_t new_encode_dev(unsigned int major, unsigned int minor)
+{
+	return (minor & 0xff) | (major << 8) | ((minor & ~0xff) << 12);
+}
+
+static int startswith(const char *str, const char *pre)
+{
+	return strncmp(pre, str, strlen(pre)) == 0;
+}
+
+static int get_node_with_prefix(const char *path, const char *prefix,
+				char *result, unsigned int result_len)
+{
+	struct lkl_dir *dir = NULL;
+	struct lkl_linux_dirent64 *dirent;
+	int ret;
+
+	dir = lkl_opendir(path, &ret);
+	if (!dir)
+		return ret;
+
+	ret = -LKL_ENOENT;
+
+	while ((dirent = lkl_readdir(dir))) {
+		if (startswith(dirent->d_name, prefix)) {
+			if (strlen(dirent->d_name) + 1 > result_len) {
+				ret = -LKL_ENOMEM;
+				break;
+			}
+			memcpy(result, dirent->d_name, strlen(dirent->d_name));
+			result[strlen(dirent->d_name)] = '\0';
+			ret = 0;
+			break;
+		}
+	}
+
+	lkl_closedir(dir);
+
+	return ret;
+}
+
+int lkl_encode_dev_from_sysfs(const char *sysfs_path, uint32_t *pdevid)
+{
+	int ret;
+	long fd;
+	int major, minor;
+	char buf[16] = { 0, };
+	char *bufptr;
+
+	fd = lkl_sys_open(sysfs_path, LKL_O_RDONLY, 0);
+	if (fd < 0)
+		return fd;
+
+	ret = lkl_sys_read(fd, buf, sizeof(buf));
+	if (ret < 0)
+		goto out_close;
+
+	if (ret == sizeof(buf)) {
+		ret = -LKL_ENOBUFS;
+		goto out_close;
+	}
+
+	bufptr = strchr(buf, ':');
+	if (bufptr == NULL) {
+		ret = -LKL_EINVAL;
+		goto out_close;
+	}
+	bufptr[0] = '\0';
+	bufptr++;
+
+	major = atoi(buf);
+	minor = atoi(bufptr);
+
+	*pdevid = new_encode_dev(major, minor);
+	ret = 0;
+
+out_close:
+	lkl_sys_close(fd);
+
+	return ret;
+}
+
+#define SYSFS_DEV_UMBLK_CMDLINE_PATH \
+	"/sysfs/devices/platform/uml-blkdev.%d"
+
+struct abuf {
+	char *mem, *ptr;
+	unsigned int len;
+};
+
+static int snprintf_append(struct abuf *buf, const char *fmt, ...)
+{
+	int ret;
+	va_list args;
+
+	if (!buf->ptr)
+		buf->ptr = buf->mem;
+
+	va_start(args, fmt);
+	ret = vsnprintf(buf->ptr, buf->len - (buf->ptr - buf->mem), fmt, args);
+	va_end(args);
+
+	if (ret < 0 || (ret >= (int)(buf->len - (buf->ptr - buf->mem))))
+		return -LKL_ENOMEM;
+
+	buf->ptr += ret;
+
+	return 0;
+}
+
+static int __lkl_get_blkdev(int disk_id, unsigned int part, uint32_t *pdevid,
+			    const char *sysfs_path_fmt, const char *drv_prefix,
+			    const char *disk_prefix)
+{
+	char sysfs_path[LKL_PATH_MAX];
+	char drv_name[LKL_PATH_MAX];
+	char disk_name[LKL_PATH_MAX];
+	struct abuf sysfs_path_buf = {
+		.mem = sysfs_path,
+		.len = sizeof(sysfs_path),
+	};
+	int ret;
+
+	if (disk_id < 0)
+		return -LKL_EINVAL;
+
+	ret = lkl_mount_fs("sysfs");
+	if (ret < 0)
+		return ret;
+
+	ret = snprintf_append(&sysfs_path_buf, sysfs_path_fmt, disk_id);
+	if (ret)
+		return ret;
+
+	ret = get_node_with_prefix(sysfs_path, drv_prefix, drv_name,
+				   sizeof(drv_name));
+	if (ret)
+		return ret;
+
+	ret = snprintf_append(&sysfs_path_buf, "/%s/block", drv_name);
+	if (ret)
+		return ret;
+
+	ret = get_node_with_prefix(sysfs_path, disk_prefix, disk_name,
+				   sizeof(disk_name));
+	if (ret)
+		return ret;
+
+	if (!part)
+		ret = snprintf_append(&sysfs_path_buf, "/%s/dev", disk_name);
+	else
+		ret = snprintf_append(&sysfs_path_buf, "/%s/%s%d/dev",
+				      disk_name, disk_name, part);
+	if (ret)
+		return ret;
+
+	return lkl_encode_dev_from_sysfs(sysfs_path, pdevid);
+}
+
+int lkl_get_blkdev(int disk_id, unsigned int part, uint32_t *pdevid)
+{
+	char *fmt;
+
+	fmt = SYSFS_DEV_UMBLK_CMDLINE_PATH;
+	return __lkl_get_blkdev(disk_id, part, pdevid, fmt, "", "ubd");
+}
+
+long lkl_mount_dev(unsigned int disk_id, unsigned int part,
+		   const char *fs_type, int flags,
+		   const char *data, char *mnt_str, unsigned int mnt_str_len)
+{
+	char dev_str[] = { "/dev/xxxxxxxx" };
+	unsigned int dev;
+	int err;
+	char _data[4096]; /* FIXME: PAGE_SIZE is not exported by LKL */
+
+	if (mnt_str_len < sizeof(dev_str))
+		return -LKL_ENOMEM;
+
+	err = lkl_get_blkdev(disk_id, part, &dev);
+	if (err < 0)
+		return err;
+
+	snprintf(dev_str, sizeof(dev_str), "/dev/%08x", dev);
+	snprintf(mnt_str, mnt_str_len, "/mnt/%08x", dev);
+
+	err = lkl_sys_access("/dev", LKL_S_IRWXO);
+	if (err < 0) {
+		if (err == -LKL_ENOENT)
+			err = lkl_sys_mkdir("/dev", 0700);
+		if (err < 0)
+			return err;
+	}
+
+	err = lkl_sys_mknod(dev_str, LKL_S_IFBLK | 0600, dev);
+	if (err < 0)
+		return err;
+
+	err = lkl_sys_access("/mnt", LKL_S_IRWXO);
+	if (err < 0) {
+		if (err == -LKL_ENOENT)
+			err = lkl_sys_mkdir("/mnt", 0700);
+		if (err < 0)
+			return err;
+	}
+
+	err = lkl_sys_mkdir(mnt_str, 0700);
+	if (err < 0) {
+		lkl_sys_unlink(dev_str);
+		return err;
+	}
+
+	/* kernel always copies a full page */
+	if (data) {
+		strncpy(_data, data, sizeof(_data));
+		_data[sizeof(_data) - 1] = 0;
+	} else {
+		_data[0] = 0;
+	}
+
+	err = lkl_sys_mount(dev_str, mnt_str, (char *)fs_type, flags, _data);
+	if (err < 0) {
+		lkl_sys_unlink(dev_str);
+		lkl_sys_rmdir(mnt_str);
+		return err;
+	}
+
+	return 0;
+}
+
+long lkl_umount_timeout(char *path, int flags, long timeout_ms)
+{
+	long incr = 10000000; /* 10 ms */
+	struct lkl_timespec ts = {
+		.tv_sec = 0,
+		.tv_nsec = incr,
+	};
+	long err;
+
+	do {
+		err = lkl_sys_umount(path, flags);
+		if (err == -LKL_EBUSY) {
+			lkl_sys_nanosleep((struct __lkl__kernel_timespec *)&ts,
+					  NULL);
+			timeout_ms -= incr / 1000000;
+		}
+	} while (err == -LKL_EBUSY && timeout_ms > 0);
+
+	return err;
+}
+
+long lkl_umount_dev(unsigned int disk_id, unsigned int part, int flags,
+		    long timeout_ms)
+{
+	char dev_str[] = { "/dev/xxxxxxxx" };
+	char mnt_str[] = { "/mnt/xxxxxxxx" };
+	unsigned int dev;
+	int err;
+
+	err = lkl_get_blkdev(disk_id, part, &dev);
+	if (err < 0)
+		return err;
+
+	snprintf(dev_str, sizeof(dev_str), "/dev/%08x", dev);
+	snprintf(mnt_str, sizeof(mnt_str), "/mnt/%08x", dev);
+
+	err = lkl_umount_timeout(mnt_str, flags, timeout_ms);
+	if (err)
+		return err;
+
+	err = lkl_sys_unlink(dev_str);
+	if (err)
+		return err;
+
+	return lkl_sys_rmdir(mnt_str);
+}
+
+struct lkl_dir {
+	int fd;
+	char buf[1024];
+	char *pos;
+	int len;
+};
+
+static struct lkl_dir *lkl_dir_alloc(int *err)
+{
+	struct lkl_dir *dir = lkl_host_ops.mem_alloc(sizeof(struct lkl_dir));
+
+	if (!dir) {
+		*err = -LKL_ENOMEM;
+		return NULL;
+	}
+
+	dir->len = 0;
+	dir->pos = NULL;
+
+	return dir;
+}
+
+struct lkl_dir *lkl_opendir(const char *path, int *err)
+{
+	struct lkl_dir *dir = lkl_dir_alloc(err);
+
+	if (!dir) {
+		*err = -LKL_ENOMEM;
+		return NULL;
+	}
+
+	dir->fd = lkl_sys_open(path, LKL_O_RDONLY | LKL_O_DIRECTORY, 0);
+	if (dir->fd < 0) {
+		*err = dir->fd;
+		lkl_host_ops.mem_free(dir);
+		return NULL;
+	}
+
+	*err = 0;
+
+	return dir;
+}
+
+struct lkl_dir *lkl_fdopendir(int fd, int *err)
+{
+	struct lkl_dir *dir = lkl_dir_alloc(err);
+
+	if (!dir)
+		return NULL;
+
+	dir->fd = fd;
+
+	return dir;
+}
+
+void lkl_rewinddir(struct lkl_dir *dir)
+{
+	lkl_sys_lseek(dir->fd, 0, LKL_SEEK_SET);
+	dir->len = 0;
+	dir->pos = NULL;
+}
+
+int lkl_closedir(struct lkl_dir *dir)
+{
+	int ret;
+
+	ret = lkl_sys_close(dir->fd);
+	lkl_host_ops.mem_free(dir);
+
+	return ret;
+}
+
+struct lkl_linux_dirent64 *lkl_readdir(struct lkl_dir *dir)
+{
+	struct lkl_linux_dirent64 *de;
+
+	if (dir->len < 0)
+		return NULL;
+
+	if (!dir->pos || dir->pos - dir->buf >= dir->len)
+		goto read_buf;
+
+return_de:
+	de = (struct lkl_linux_dirent64 *)dir->pos;
+	dir->pos += de->d_reclen;
+
+	return de;
+
+read_buf:
+	dir->pos = NULL;
+	de = (struct lkl_linux_dirent64 *)dir->buf;
+	dir->len = lkl_sys_getdents64(dir->fd, de, sizeof(dir->buf));
+	if (dir->len <= 0)
+		return NULL;
+
+	dir->pos = dir->buf;
+	goto return_de;
+}
+
+int lkl_errdir(struct lkl_dir *dir)
+{
+	if (dir->len >= 0)
+		return 0;
+
+	return dir->len;
+}
+
+int lkl_dirfd(struct lkl_dir *dir)
+{
+	return dir->fd;
+}
+
+int lkl_set_fd_limit(unsigned int fd_limit)
+{
+	struct lkl_rlimit rlim = {
+		.rlim_cur = fd_limit,
+		.rlim_max = fd_limit,
+	};
+	return lkl_sys_setrlimit(LKL_RLIMIT_NOFILE, &rlim);
+}
diff --git a/tools/um/lib/posix-host.c b/tools/um/lib/posix-host.c
index b6b5b2902254..8fd88031bf2b 100644
--- a/tools/um/lib/posix-host.c
+++ b/tools/um/lib/posix-host.c
@@ -263,6 +263,7 @@ static void *tls_get(struct lkl_tls_key *key)
 }
 
 struct lkl_host_operations lkl_host_ops = {
+	.um_devices = lkl_um_devs,
 	.panic = panic,
 	.print = print,
 	.mem_alloc = (void *)malloc,
diff --git a/tools/um/lib/utils.c b/tools/um/lib/utils.c
index 4930479a8a35..ac65cd744a14 100644
--- a/tools/um/lib/utils.c
+++ b/tools/um/lib/utils.c
@@ -4,6 +4,9 @@
 #include <string.h>
 #include <lkl_host.h>
 
+/* XXX: find a better place */
+char lkl_um_devs[4096];
+
 static const char * const lkl_err_strings[] = {
 	"Success",
 	"Operation not permitted",
diff --git a/tools/um/tests/Build b/tools/um/tests/Build
index 564560486f98..1aa78f9ed7ef 100644
--- a/tools/um/tests/Build
+++ b/tools/um/tests/Build
@@ -1,3 +1,4 @@
 boot-y += boot.o test.o
+disk-y += disk.o test.o cla.o
 
 CFLAGS_test.o += -Wno-implicit-fallthrough
diff --git a/tools/um/tests/cla.c b/tools/um/tests/cla.c
new file mode 100644
index 000000000000..694e3a3822be
--- /dev/null
+++ b/tools/um/tests/cla.c
@@ -0,0 +1,159 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <string.h>
+#include <errno.h>
+#include <stdlib.h>
+#ifdef __MINGW32__
+#include <winsock2.h>
+#else
+#include <sys/socket.h>
+#include <netinet/in.h>
+#include <arpa/inet.h>
+#endif
+
+#include "cla.h"
+
+static int cl_arg_parse_bool(struct cl_arg *arg, const char *value)
+{
+	*((int *)arg->store) = 1;
+	return 0;
+}
+
+static int cl_arg_parse_str(struct cl_arg *arg, const char *value)
+{
+	*((const char **)arg->store) = value;
+	return 0;
+}
+
+static int cl_arg_parse_int(struct cl_arg *arg, const char *value)
+{
+	errno = 0;
+	*((int *)arg->store) = strtol(value, NULL, 0);
+	return errno == 0;
+}
+
+static int cl_arg_parse_str_set(struct cl_arg *arg, const char *value)
+{
+	const char **set = arg->set;
+	int i;
+
+	for (i = 0; set[i] != NULL; i++) {
+		if (strcmp(set[i], value) == 0) {
+			*((int *)arg->store) = i;
+			return 0;
+		}
+	}
+
+	return (-1);
+}
+
+static int cl_arg_parse_ipv4(struct cl_arg *arg, const char *value)
+{
+	unsigned int addr;
+
+	if (!value)
+		return (-1);
+
+	addr = inet_addr(value);
+	if (addr == INADDR_NONE)
+		return (-1);
+	*((unsigned int *)arg->store) = addr;
+	return 0;
+}
+
+static cl_arg_parser_t parsers[] = {
+	[CL_ARG_BOOL] = cl_arg_parse_bool,
+	[CL_ARG_INT] = cl_arg_parse_int,
+	[CL_ARG_STR] = cl_arg_parse_str,
+	[CL_ARG_STR_SET] = cl_arg_parse_str_set,
+	[CL_ARG_IPV4] = cl_arg_parse_ipv4,
+};
+
+static struct cl_arg *find_short_arg(char name, struct cl_arg *args)
+{
+	struct cl_arg *arg;
+
+	for (arg = args; arg->short_name != 0; arg++) {
+		if (arg->short_name == name)
+			return arg;
+	}
+
+	return NULL;
+}
+
+static struct cl_arg *find_long_arg(const char *name, struct cl_arg *args)
+{
+	struct cl_arg *arg;
+
+	for (arg = args; arg->long_name; arg++) {
+		if (strcmp(arg->long_name, name) == 0)
+			return arg;
+	}
+
+	return NULL;
+}
+
+static void print_help(struct cl_arg *args)
+{
+	struct cl_arg *arg;
+
+	fprintf(stderr, "usage:\n");
+	for (arg = args; arg->long_name; arg++) {
+		fprintf(stderr, "-%c, --%-20s %s", arg->short_name,
+			arg->long_name, arg->help);
+		if (arg->type == CL_ARG_STR_SET) {
+			const char **set = arg->set;
+
+			fprintf(stderr, " [ ");
+			while (*set != NULL)
+				fprintf(stderr, "%s ", *(set++));
+			fprintf(stderr, "]");
+		}
+		fprintf(stderr, "\n");
+	}
+}
+
+int cla_parse_args(int argc, const char **argv, struct cl_arg *args)
+{
+	int i;
+
+	for (i = 1; i < argc; i++) {
+		struct cl_arg *arg = NULL;
+		cl_arg_parser_t parser;
+
+		if (argv[i][0] == '-') {
+			if (argv[i][1] != '-')
+				arg = find_short_arg(argv[i][1], args);
+			else
+				arg = find_long_arg(&argv[i][2], args);
+		}
+
+		if (!arg) {
+			fprintf(stderr, "unknown option '%s'\n", argv[i]);
+			print_help(args);
+			return (-1);
+		}
+
+		if (arg->type == CL_ARG_USER || arg->type >= CL_ARG_END)
+			parser = arg->parser;
+		else
+			parser = parsers[arg->type];
+
+		if (!parser) {
+			fprintf(stderr, "can't parse --'%s'/-'%c'\n",
+				arg->long_name, args->short_name);
+			return (-1);
+		}
+
+		if (parser(arg, argv[i + 1]) < 0) {
+			fprintf(stderr, "can't parse '%s'\n", argv[i]);
+			print_help(args);
+			return (-1);
+		}
+
+		if (arg->has_arg)
+			i++;
+	}
+
+	return 0;
+}
diff --git a/tools/um/tests/cla.h b/tools/um/tests/cla.h
new file mode 100644
index 000000000000..3d879233681f
--- /dev/null
+++ b/tools/um/tests/cla.h
@@ -0,0 +1,33 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef _LKL_TEST_CLA_H
+#define _LKL_TEST_CLA_H
+
+enum cl_arg_type {
+	CL_ARG_USER = 0,
+	CL_ARG_BOOL,
+	CL_ARG_INT,
+	CL_ARG_STR,
+	CL_ARG_STR_SET,
+	CL_ARG_IPV4,
+	CL_ARG_END,
+};
+
+struct cl_arg;
+
+typedef int (*cl_arg_parser_t)(struct cl_arg *arg, const char *value);
+
+struct cl_arg {
+	const char *long_name;
+	char short_name;
+	const char *help;
+	int has_arg;
+	enum cl_arg_type type;
+	void *store;
+	void *set;
+	cl_arg_parser_t parser;
+};
+
+int cla_parse_args(int argc, const char **argv, struct cl_arg *args);
+
+
+#endif /* _LKL_TEST_CLA_H */
diff --git a/tools/um/tests/disk.c b/tools/um/tests/disk.c
new file mode 100644
index 000000000000..325934935e6a
--- /dev/null
+++ b/tools/um/tests/disk.c
@@ -0,0 +1,168 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <unistd.h>
+#include <string.h>
+#include <time.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <lkl.h>
+#include <lkl_host.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <sys/ioctl.h>
+
+#include "test.h"
+#include "cla.h"
+
+static struct {
+	int printk;
+	const char *disk;
+	const char *fstype;
+	int partition;
+} cla;
+
+struct cl_arg args[] = {
+	{"disk", 'd', "disk file to use", 1, CL_ARG_STR, &cla.disk},
+	{"partition", 'P', "partition to mount", 1, CL_ARG_INT, &cla.partition},
+	{"type", 't', "filesystem type", 1, CL_ARG_STR, &cla.fstype},
+	{0},
+};
+
+
+static struct lkl_disk disk;
+static int disk_id = -1;
+
+int lkl_test_disk_add(void)
+{
+	disk.fd = open(cla.disk, O_RDWR);
+	if (disk.fd < 0)
+		goto out_unlink;
+
+	disk.ops = NULL;
+	disk.dev = (char *)cla.disk;
+
+	disk_id = lkl_disk_add(&disk);
+	if (disk_id < 0)
+		goto out_close;
+
+	goto out;
+
+out_close:
+	close(disk.fd);
+
+out_unlink:
+	unlink(cla.disk);
+
+out:
+	lkl_test_logf("disk fd/handle %x disk_id %d", disk.fd, disk_id);
+
+	if (disk_id >= 0)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+int lkl_test_disk_remove(void)
+{
+	int ret;
+
+	ret = lkl_disk_remove(disk);
+
+	close(disk.fd);
+
+	if (ret == 0)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+
+static char mnt_point[32];
+
+LKL_TEST_CALL(mount_dev, lkl_mount_dev, 0, disk_id, cla.partition, cla.fstype,
+	      0, NULL, mnt_point, sizeof(mnt_point))
+
+static int lkl_test_umount_dev(void)
+{
+	long ret, ret2;
+
+	ret = lkl_sys_chdir("/");
+
+	ret2 = lkl_umount_dev(disk_id, cla.partition, 0, 1000);
+
+	lkl_test_logf("%ld %ld", ret, ret2);
+
+	if (!ret && !ret2)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+struct lkl_dir *dir;
+
+static int lkl_test_opendir(void)
+{
+	int err;
+
+	dir = lkl_opendir(mnt_point, &err);
+
+	lkl_test_logf("lkl_opedir(%s) = %d %s\n", mnt_point, err,
+		      lkl_strerror(err));
+
+	if (err == 0)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+static int lkl_test_readdir(void)
+{
+	struct lkl_linux_dirent64 *de = lkl_readdir(dir);
+	int wr = 0;
+
+	while (de) {
+		wr += lkl_test_logf("%s ", de->d_name);
+		if (wr >= 70) {
+			lkl_test_logf("\n");
+			wr = 0;
+			break;
+		}
+		de = lkl_readdir(dir);
+	}
+
+	if (lkl_errdir(dir) == 0)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+LKL_TEST_CALL(closedir, lkl_closedir, 0, dir);
+LKL_TEST_CALL(chdir_mnt_point, lkl_sys_chdir, 0, mnt_point);
+LKL_TEST_CALL(start_kernel, lkl_start_kernel, 0, &lkl_host_ops,
+	     "mem=16M loglevel=8");
+LKL_TEST_CALL(stop_kernel, lkl_sys_halt, 0);
+
+struct lkl_test tests[] = {
+	LKL_TEST(disk_add),
+	LKL_TEST(start_kernel),
+	LKL_TEST(mount_dev),
+	LKL_TEST(chdir_mnt_point),
+	LKL_TEST(opendir),
+	LKL_TEST(readdir),
+	LKL_TEST(closedir),
+	LKL_TEST(umount_dev),
+	LKL_TEST(stop_kernel),
+	LKL_TEST(disk_remove),
+
+};
+
+int main(int argc, const char **argv)
+{
+	if (cla_parse_args(argc, argv, args) < 0)
+		return (-1);
+
+	lkl_host_ops.print = lkl_test_log;
+
+	return lkl_test_run(tests, sizeof(tests)/sizeof(struct lkl_test),
+			    "disk %s", cla.fstype);
+}
diff --git a/tools/um/tests/disk.sh b/tools/um/tests/disk.sh
new file mode 100755
index 000000000000..e2ec6cf69d4b
--- /dev/null
+++ b/tools/um/tests/disk.sh
@@ -0,0 +1,70 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+
+source $script_dir/test.sh
+
+function prepfs()
+{
+    set -e
+
+    file=`mktemp`
+
+    dd if=/dev/zero of=$file bs=1024 count=204800
+
+    yes | mkfs.$1 $file
+
+    if ! [ -z $ANDROID_WDIR ]; then
+        adb shell mkdir -p $ANDROID_WDIR
+        adb push $file $ANDROID_WDIR
+        rm $file
+        file=$ANDROID_WDIR/$(basename $file)
+    fi
+    if ! [ -z $BSD_WDIR ]; then
+        $MYSSH mkdir -p $BSD_WDIR
+        ssh_copy $file $BSD_WDIR
+        rm $file
+        file=$BSD_WDIR/$(basename $file)
+    fi
+
+    export_vars file
+}
+
+function cleanfs()
+{
+    set -e
+
+    if ! [ -z $ANDROID_WDIR ]; then
+        adb shell rm $1
+        adb shell rm $ANDROID_WDIR/disk
+    elif ! [ -z $BSD_WDIR ]; then
+        $MYSSH rm $1
+        $MYSSH rm $BSD_WDIR/disk
+    else
+        rm $1
+    fi
+}
+
+if [ "$1" = "-t" ]; then
+    shift
+    fstype=$1
+    shift
+fi
+
+if [ -z "$fstype" ]; then
+    fstype="ext4"
+fi
+
+if [ -z $(which mkfs.$fstype) ]; then
+    lkl_test_plan 0 "disk $fstype"
+    echo "no mkfs.$fstype command"
+    exit 0
+fi
+
+lkl_test_plan 1 "disk $fstype"
+lkl_test_run 1 prepfs $fstype
+lkl_test_exec $script_dir/disk -d $file -t $fstype $@
+lkl_test_plan 1 "disk $fstype"
+lkl_test_run 1 cleanfs $file
+
diff --git a/tools/um/tests/run.py b/tools/um/tests/run.py
index c96ede90b6ad..97d6dedc217c 100755
--- a/tools/um/tests/run.py
+++ b/tools/um/tests/run.py
@@ -50,6 +50,8 @@ mydir=os.path.dirname(os.path.realpath(__file__))
 
 tests = [
     'boot.sh',
+    'disk.sh -t ext4',
+    'disk.sh -t vfat',
 ]
 
 parser = argparse.ArgumentParser(description='LKL test runner')
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* Re: [RFC v7 00/21] Unifying LKL into UML
  2020-10-06  9:44         ` Hajime Tazaki
@ 2020-10-07 13:30           ` Anton Ivanov
  -1 siblings, 0 replies; 476+ messages in thread
From: Anton Ivanov @ 2020-10-07 13:30 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard
  Cc: tavi.purdila, linux-kernel-library, linux-arch, retrage01


On 06/10/2020 10:44, Hajime Tazaki wrote:
> This is another spin of the unification of LKL into UML.  Based on the
> discussion of v4 patchset, we have tried to address issue raised and
> rewrote the patchset from scratch.  The summary is listed in the
> changelog below.
>
> Although there are still bugs in the patchset, we'd like to ask your
> opinions on the design we changed.
>
> The milestone section is also updated: this patchset is for the
> milestone 1, though the common init API is still not implemented yet.
>
>
> Changes in rfc v7:
> - preserve `make ARCH=um` syntax to build UML
> - introduce `make ARCH=um UMMODE=library` to build library mode
> - fix undefined symbols issue during modpost
> - clean up makefiles (arch/um, tools/um)

Hi Hajime, hi Tavi,

Our starting point should be that it does not break the existing build. It still does.

If I build a "stock configuration" UML after applying the patchset the resulting vmlinux is not executable.

On the positive side, it builds cleanly now. I will try to go through the rest of the patchset later today and see if there is anything else that needs fixing before we do the next version.

Brgds,

A.

>
> Changes in rfc v6:
> - rebase with the current linus tree
>
> Changes in rfc v5:
> - rewrite whole patchset from scratch
> - move arch-dependent code of arch/um and arch/x86/um to tools/um
>   - mainly code under os-Linux/ involved
>   - introduce 2-stage build (kernel, and host-dependent parts)
>   - clean up vmlinux.lds.S
> - put LKL-specific implementations as a SUBARCH under arch/um/nommu
> - introduce !CONFIG_MMU in arch/um
> - use struct arch_thread and arch_switch_to() for subarch-specific
>    thread implementation
> - integrate with the IRQ infrastructure of UML
> - tested with block device drivers (ubd) for the proof
>
> Changes in rfc v4: (https://lwn.net/Articles/816276/)
> - Rebase on the current uml/master branch
> - Fix IRQ handling (bug fix)
> - drop a patch for CONFIG_GENERIC_ATOMIC64 (comment by Peter Zijlstra)
> - implement vector net driver for UMMODE_LIB (comment by Anton)
> - clean up uapi headers to avoid duplicates
> - clean up IRQ handling code (comment by Anton)
> - fix error handling in test code (comments by David Disseldorp)
>
> Changes in rfc v3:
> - use UML drivers (net, block) from LKL programs
> - drop virtio device implementations
> - drop mingw32 (Windows) host
> - drop android (arm/aarch64) host
> - drop FreeBSD (x86_64) host
> - drop LD_PRELOAD (hijack) support
> - update milestone
>
> rfc v2:
> - use UMMODE instead of SUBARCH to switch UML or LKL
> - tools/lkl directory is still there. I confirmed we can move under arch/um
>    (e.g., arch/um/lkl/hosts).  I will move it IF this is preferable.
> - drop several patches involved non-uml directory
> - drop several patches which are not required
> - refine commit logs
> - document updated
>
>
>
>
> LKL (Linux Kernel Library) is aiming to allow reusing the Linux kernel code
> as extensively as possible with minimal effort and reduced maintenance
> overhead.
>
> Examples of how LKL can be used are: creating userspace applications
> (running on Linux and other operating systems) that can read or write Linux
> filesystems or can use the Linux networking stack, creating kernel drivers
> for other operating systems that can read Linux filesystems, bootloaders
> support for reading/writing Linux filesystems, etc.
>
> With LKL, the kernel code is compiled into an object file that can be
> directly linked by applications. The API offered by LKL is based on the
> Linux system call interface.
>
> LKL is originally implemented as an architecture port in arch/lkl, but this
> series of commits tries to integrate this into arch/um as one of the mode
> of UML.  This was discussed during RFC email of LKL (*1).
>
> The latest LKL version can be found at https://github.com/lkl/linux
>
> Milestone
> =========
> This patches is a first step toward upstreaming *library mode* of Linux kernel,
> but we think we need to have several steps toward our goal, describing in the
> below.
>
>
> Milestone 1: LKL lib on top of UML
>   * Kernel - Host build split
>   -  Build UML as a relocatable object using the UML's kernel linker script.
>   -  Move the ptrace and other well isolated os code out of arch/um to
>      tools/um
>   -  Use standard host toolchain to create a static library stripped of
>      the ptrace code. Use standard host toolchain to build the main UML
>      executable.
>   -  Add library init API that creates the UML kernel process and starts
>      UML.
>   * System calls APIs
>   -  Add new system call interface based on UML's irq facility.
>   -  Use the LKL scripts to export the required headers to create system
>      calls APIs that use the UML system calls infrastructure.
>   -  Keep the underlying host and driver operations (threads, irqs, etc.)
>      as they are now in UML.
>   * Boot test
>   -  Port the LKL boot test to verify that we are able to programatically
>      issue system calls.
>
> Milestone 2: add virtio disk support
>   * Export asm/io.h operations to host/os. Create IO access operations
>     and redirect them to weak os_ variants that use the current UML
>     implementation.
>   * Add the LKL IO access layer including generic virtio handling and the
>     virtio block device code.
>   * Port LKL disk test and disk apps (lklfuse, fs2tar, cptofs)
>
> Milestone 3: new arch ports
>    * Abstract the system call / IRQ mode the move the implementation to host
>    * Abstract the thread model and move the implementation to host
>    * Add LKL thread model and LKL ports
>
>
> Building LKL the host library and LKL applications
> ==================================================
>
> % cd tools/um
> % make
>
> will build LKL as a object file, it will install it in tools/um/lib together
> with the headers files in tools/um/include then will build the host library,
> tests and a few of application examples:
>
> * tests/boot - a simple applications that uses LKL and exercises the basic
>    LKL APIs
>
> % make run-tests
>
> should run the above `tests/boot` and `tests/net-test` and report errors if
> there are any.
>
> Supported hosts
> ===============
>
> Currently LKL supports Linux userspace applications. New hosts can be added
> relatively easy if the host supports gcc and GNU ld. Previous versions of
> LKL supported Windows kernel and Haiku kernel hosts, and we also have WIP
> patches with rump-hypercall interface, used in UEFI, as well as macOS
> userspace (part of POSIX).
>
> There is also musl-libc port for LKL, which might be interested in for some
> folks.
>
>
> Further readings about LKL
> =========================
>
> - Discussion in github LKL issue
> https://github.com/lkl/linux/issues/304
>
> - LKL (an article)
> https://www.researchgate.net/profile/Nicolae_Tapus2/publication/224164682_LKL_The_Linux_kernel_library/links/02bfe50fd921ab4f7c000000.pdf
>
> *1 RFC email to LKML (back in 2015)
> https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1012277.html
>
>
> Please review the following changes for suitability for inclusion. If you have
> any objections or suggestions for improvement, please respond to the patches. If
> you agree with the changes, please provide your Acked-by.
>
> The following changes since commit 805c6d3c19210c90c109107d189744e960eae025:
>
>    Merge branch 'fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs (2020-09-22 15:08:41 -0700)
>
> are available in the Git repository at:
>
>    git://github.com/thehajime/linux a8686efcbad3c445d3e5bde054c04a58877915f1
>    https://github.com/thehajime/linux/tree/uml-lkl-5.9rc6-v7
>
> Hajime Tazaki (18):
>    um: move arch/um/os-Linux dir to tools/um/uml
>    um: move arch/x86/um/os-Linux to tools/um/uml/
>    scritps: um: suppress warnings if SRCARCH=um
>    um: extend arch_switch_to for alternate SUBARCH
>    um: add nommu mode for UML library mode
>    um: nommu: host interface
>    um: nommu: memory handling
>    um: nommu: kernel thread support
>    um: nommu: system call interface and application API
>    um: nommu: basic console support
>    um: nommu: initialization and cleanup
>    um: nommu: integrate with irq infrastructure of UML
>    um: nommu: plug in the build system
>    um: host: add nommu build for ARCH=um
>    um: host: add utilities functions
>    um: host: posix host operations
>    um: host: add test programs
>    um: nommu: add block device support of UML
>
> Octavian Purdila (3):
>    um: split build in kernel and host parts
>    um: add os init and exit calls
>    um: host: implement os_initcalls and os_exitcalls
>
>   arch/um/Kconfig                               |  29 +-
>   arch/um/Makefile                              |  35 +-
>   arch/um/drivers/Makefile                      |  10 +-
>   .../um/{os-Linux => }/drivers/ethertap_kern.c |   0
>   arch/um/{os-Linux => }/drivers/tuntap_kern.c  |   0
>   arch/um/include/asm/common.lds.S              |  99 ----
>   arch/um/include/asm/host_ops.h                |   9 +
>   arch/um/include/asm/mmu.h                     |   3 +
>   arch/um/include/asm/mmu_context.h             |   8 +
>   arch/um/include/asm/page.h                    |  15 +
>   arch/um/include/asm/pgtable.h                 |  27 +
>   arch/um/include/asm/thread_info.h             |  24 +
>   arch/um/include/asm/uaccess.h                 |   6 +
>   arch/um/include/asm/xor.h                     |   3 +-
>   arch/um/include/shared/as-layout.h            |   1 +
>   .../drivers => include/shared}/etap.h         |   0
>   arch/um/include/shared/init.h                 |  19 +-
>   arch/um/include/shared/os.h                   |   1 +
>   .../drivers => include/shared}/tuntap.h       |   0
>   arch/um/include/uapi/asm/Kbuild               |   2 +
>   arch/um/kernel/Makefile                       |  13 +-
>   arch/um/kernel/dyn.lds.S                      | 171 -------
>   arch/um/kernel/irq.c                          |  13 +
>   arch/um/kernel/process.c                      |  14 +-
>   arch/um/kernel/reboot.c                       |   5 +
>   arch/um/kernel/time.c                         |   2 +
>   arch/um/kernel/um_arch.c                      |  16 +
>   arch/um/kernel/uml.lds.S                      | 115 -----
>   arch/um/{os-Linux => kernel}/user_syms.c      |   0
>   arch/um/kernel/vmlinux.lds.S                  |  91 +++-
>   arch/um/nommu/Makefile                        |   1 +
>   arch/um/nommu/Makefile.um                     |  22 +
>   arch/um/nommu/include/asm/Kbuild              |   6 +
>   arch/um/nommu/include/asm/archparam.h         |   1 +
>   arch/um/nommu/include/asm/atomic.h            |  11 +
>   arch/um/nommu/include/asm/atomic64.h          | 114 +++++
>   arch/um/nommu/include/asm/bitsperlong.h       |  12 +
>   arch/um/nommu/include/asm/byteorder.h         |   7 +
>   arch/um/nommu/include/asm/cpu.h               |  16 +
>   arch/um/nommu/include/asm/elf.h               |  15 +
>   arch/um/nommu/include/asm/mm_context.h        |   8 +
>   arch/um/nommu/include/asm/processor.h         |  46 ++
>   arch/um/nommu/include/asm/ptrace.h            |  21 +
>   arch/um/nommu/include/asm/sched.h             |  23 +
>   arch/um/nommu/include/asm/segment.h           |   9 +
>   arch/um/nommu/include/asm/syscall_wrapper.h   |  57 +++
>   arch/um/nommu/include/asm/syscalls.h          |  15 +
>   arch/um/nommu/include/uapi/asm/Kbuild         |   6 +
>   arch/um/nommu/include/uapi/asm/bitsperlong.h  |  11 +
>   arch/um/nommu/include/uapi/asm/byteorder.h    |  11 +
>   arch/um/nommu/include/uapi/asm/host_ops.h     | 122 +++++
>   arch/um/nommu/include/uapi/asm/sigcontext.h   |  12 +
>   arch/um/nommu/include/uapi/asm/syscalls.h     | 287 +++++++++++
>   arch/um/nommu/include/uapi/asm/unistd.h       |  17 +
>   arch/um/nommu/um/Kconfig                      |  45 ++
>   arch/um/nommu/um/Makefile                     |   4 +
>   arch/um/nommu/um/bootmem.c                    |  87 ++++
>   arch/um/nommu/um/console.c                    |  42 ++
>   arch/um/nommu/um/cpu.c                        | 247 ++++++++++
>   arch/um/nommu/um/delay.c                      |  31 ++
>   arch/um/nommu/um/setup.c                      | 179 +++++++
>   arch/um/nommu/um/shared/sysdep/archsetjmp.h   |  13 +
>   arch/um/nommu/um/shared/sysdep/faultinfo.h    |   8 +
>   .../nommu/um/shared/sysdep/kernel-offsets.h   |  12 +
>   arch/um/nommu/um/shared/sysdep/mcontext.h     |   9 +
>   arch/um/nommu/um/shared/sysdep/ptrace.h       |  42 ++
>   arch/um/nommu/um/shared/sysdep/ptrace_user.h  |   7 +
>   arch/um/nommu/um/syscalls.c                   | 199 ++++++++
>   arch/um/nommu/um/threads.c                    | 262 ++++++++++
>   arch/um/nommu/um/unimplemented.c              |  70 +++
>   arch/um/nommu/um/user_constants.h             |  13 +
>   arch/um/os-Linux/Makefile                     |  19 -
>   arch/um/os-Linux/drivers/Makefile             |  13 -
>   arch/um/scripts/headers_install.py            | 197 ++++++++
>   arch/x86/um/Makefile                          |   2 +-
>   arch/x86/um/os-Linux/Makefile                 |  13 -
>   arch/x86/um/ptrace_32.c                       |   2 +-
>   arch/x86/um/syscalls_64.c                     |   2 +-
>   scripts/headers_install.sh                    |   4 +
>   scripts/link-vmlinux.sh                       |  42 +-
>   tools/um/.gitignore                           |   1 +
>   tools/um/Makefile                             |  81 +++
>   tools/um/Targets                              |  13 +
>   tools/um/include/lkl.h                        | 357 ++++++++++++++
>   tools/um/include/lkl_host.h                   |  27 +
>   tools/um/lib/Build                            |   7 +
>   tools/um/lib/fs.c                             | 461 ++++++++++++++++++
>   tools/um/lib/jmp_buf.c                        |  14 +
>   tools/um/lib/jmp_buf.h                        |   8 +
>   tools/um/lib/posix-host.c                     | 293 +++++++++++
>   tools/um/lib/utils.c                          | 213 ++++++++
>   tools/um/tests/Build                          |   4 +
>   tools/um/tests/boot.c                         | 393 +++++++++++++++
>   tools/um/tests/boot.sh                        |   9 +
>   tools/um/tests/cla.c                          | 159 ++++++
>   tools/um/tests/cla.h                          |  33 ++
>   tools/um/tests/disk.c                         | 168 +++++++
>   tools/um/tests/disk.sh                        |  70 +++
>   tools/um/tests/run.py                         | 174 +++++++
>   tools/um/tests/tap13.py                       | 209 ++++++++
>   tools/um/tests/test.c                         | 126 +++++
>   tools/um/tests/test.h                         |  72 +++
>   tools/um/tests/test.sh                        | 181 +++++++
>   tools/um/uml/Build                            |  58 +++
>   tools/um/uml/drivers/Build                    |  10 +
>   .../um/uml}/drivers/ethertap_user.c           |   0
>   .../um/uml}/drivers/tuntap_user.c             |   0
>   {arch/um/os-Linux => tools/um/uml}/elf_aux.c  |   0
>   {arch/um/os-Linux => tools/um/uml}/execvp.c   |   4 -
>   {arch/um/os-Linux => tools/um/uml}/file.c     |   0
>   {arch/um/os-Linux => tools/um/uml}/helper.c   |   0
>   {arch/um/os-Linux => tools/um/uml}/irq.c      |   0
>   {arch/um/os-Linux => tools/um/uml}/main.c     |   0
>   {arch/um/os-Linux => tools/um/uml}/mem.c      |   0
>   tools/um/uml/nommu/Build                      |   1 +
>   tools/um/uml/nommu/registers.c                |  21 +
>   tools/um/uml/nommu/unimplemented.c            |  21 +
>   {arch/um/os-Linux => tools/um/uml}/process.c  |   2 +
>   .../um/os-Linux => tools/um/uml}/registers.c  |   0
>   {arch/um/os-Linux => tools/um/uml}/sigio.c    |   0
>   {arch/um/os-Linux => tools/um/uml}/signal.c   |  12 +-
>   .../skas/Makefile => tools/um/uml/skas/Build  |   6 +-
>   {arch/um/os-Linux => tools/um/uml}/skas/mem.c |   0
>   .../os-Linux => tools/um/uml}/skas/process.c  |   3 +-
>   {arch/um/os-Linux => tools/um/uml}/start_up.c |   0
>   {arch/um/os-Linux => tools/um/uml}/time.c     |   0
>   {arch/um/os-Linux => tools/um/uml}/tty.c      |   0
>   {arch/um/os-Linux => tools/um/uml}/umid.c     |   0
>   {arch/um/os-Linux => tools/um/uml}/util.c     |  26 +
>   tools/um/uml/x86/Build                        |  11 +
>   .../os-Linux => tools/um/uml/x86}/mcontext.c  |   0
>   .../um/os-Linux => tools/um/uml/x86}/prctl.c  |   0
>   .../os-Linux => tools/um/uml/x86}/registers.c |   0
>   .../os-Linux => tools/um/uml/x86}/task_size.c |   0
>   .../um/os-Linux => tools/um/uml/x86}/tls.c    |   0
>   135 files changed, 5870 insertions(+), 523 deletions(-)
>   rename arch/um/{os-Linux => }/drivers/ethertap_kern.c (100%)
>   rename arch/um/{os-Linux => }/drivers/tuntap_kern.c (100%)
>   delete mode 100644 arch/um/include/asm/common.lds.S
>   create mode 100644 arch/um/include/asm/host_ops.h
>   rename arch/um/{os-Linux/drivers => include/shared}/etap.h (100%)
>   rename arch/um/{os-Linux/drivers => include/shared}/tuntap.h (100%)
>   create mode 100644 arch/um/include/uapi/asm/Kbuild
>   delete mode 100644 arch/um/kernel/dyn.lds.S
>   delete mode 100644 arch/um/kernel/uml.lds.S
>   rename arch/um/{os-Linux => kernel}/user_syms.c (100%)
>   create mode 100644 arch/um/nommu/Makefile
>   create mode 100644 arch/um/nommu/Makefile.um
>   create mode 100644 arch/um/nommu/include/asm/Kbuild
>   create mode 100644 arch/um/nommu/include/asm/archparam.h
>   create mode 100644 arch/um/nommu/include/asm/atomic.h
>   create mode 100644 arch/um/nommu/include/asm/atomic64.h
>   create mode 100644 arch/um/nommu/include/asm/bitsperlong.h
>   create mode 100644 arch/um/nommu/include/asm/byteorder.h
>   create mode 100644 arch/um/nommu/include/asm/cpu.h
>   create mode 100644 arch/um/nommu/include/asm/elf.h
>   create mode 100644 arch/um/nommu/include/asm/mm_context.h
>   create mode 100644 arch/um/nommu/include/asm/processor.h
>   create mode 100644 arch/um/nommu/include/asm/ptrace.h
>   create mode 100644 arch/um/nommu/include/asm/sched.h
>   create mode 100644 arch/um/nommu/include/asm/segment.h
>   create mode 100644 arch/um/nommu/include/asm/syscall_wrapper.h
>   create mode 100644 arch/um/nommu/include/asm/syscalls.h
>   create mode 100644 arch/um/nommu/include/uapi/asm/Kbuild
>   create mode 100644 arch/um/nommu/include/uapi/asm/bitsperlong.h
>   create mode 100644 arch/um/nommu/include/uapi/asm/byteorder.h
>   create mode 100644 arch/um/nommu/include/uapi/asm/host_ops.h
>   create mode 100644 arch/um/nommu/include/uapi/asm/sigcontext.h
>   create mode 100644 arch/um/nommu/include/uapi/asm/syscalls.h
>   create mode 100644 arch/um/nommu/include/uapi/asm/unistd.h
>   create mode 100644 arch/um/nommu/um/Kconfig
>   create mode 100644 arch/um/nommu/um/Makefile
>   create mode 100644 arch/um/nommu/um/bootmem.c
>   create mode 100644 arch/um/nommu/um/console.c
>   create mode 100644 arch/um/nommu/um/cpu.c
>   create mode 100644 arch/um/nommu/um/delay.c
>   create mode 100644 arch/um/nommu/um/setup.c
>   create mode 100644 arch/um/nommu/um/shared/sysdep/archsetjmp.h
>   create mode 100644 arch/um/nommu/um/shared/sysdep/faultinfo.h
>   create mode 100644 arch/um/nommu/um/shared/sysdep/kernel-offsets.h
>   create mode 100644 arch/um/nommu/um/shared/sysdep/mcontext.h
>   create mode 100644 arch/um/nommu/um/shared/sysdep/ptrace.h
>   create mode 100644 arch/um/nommu/um/shared/sysdep/ptrace_user.h
>   create mode 100644 arch/um/nommu/um/syscalls.c
>   create mode 100644 arch/um/nommu/um/threads.c
>   create mode 100644 arch/um/nommu/um/unimplemented.c
>   create mode 100644 arch/um/nommu/um/user_constants.h
>   delete mode 100644 arch/um/os-Linux/Makefile
>   delete mode 100644 arch/um/os-Linux/drivers/Makefile
>   create mode 100755 arch/um/scripts/headers_install.py
>   delete mode 100644 arch/x86/um/os-Linux/Makefile
>   create mode 100644 tools/um/.gitignore
>   create mode 100644 tools/um/Makefile
>   create mode 100644 tools/um/Targets
>   create mode 100644 tools/um/include/lkl.h
>   create mode 100644 tools/um/include/lkl_host.h
>   create mode 100644 tools/um/lib/Build
>   create mode 100644 tools/um/lib/fs.c
>   create mode 100644 tools/um/lib/jmp_buf.c
>   create mode 100644 tools/um/lib/jmp_buf.h
>   create mode 100644 tools/um/lib/posix-host.c
>   create mode 100644 tools/um/lib/utils.c
>   create mode 100644 tools/um/tests/Build
>   create mode 100644 tools/um/tests/boot.c
>   create mode 100755 tools/um/tests/boot.sh
>   create mode 100644 tools/um/tests/cla.c
>   create mode 100644 tools/um/tests/cla.h
>   create mode 100644 tools/um/tests/disk.c
>   create mode 100755 tools/um/tests/disk.sh
>   create mode 100755 tools/um/tests/run.py
>   create mode 100644 tools/um/tests/tap13.py
>   create mode 100644 tools/um/tests/test.c
>   create mode 100644 tools/um/tests/test.h
>   create mode 100644 tools/um/tests/test.sh
>   create mode 100644 tools/um/uml/Build
>   create mode 100644 tools/um/uml/drivers/Build
>   rename {arch/um/os-Linux => tools/um/uml}/drivers/ethertap_user.c (100%)
>   rename {arch/um/os-Linux => tools/um/uml}/drivers/tuntap_user.c (100%)
>   rename {arch/um/os-Linux => tools/um/uml}/elf_aux.c (100%)
>   rename {arch/um/os-Linux => tools/um/uml}/execvp.c (98%)
>   rename {arch/um/os-Linux => tools/um/uml}/file.c (100%)
>   rename {arch/um/os-Linux => tools/um/uml}/helper.c (100%)
>   rename {arch/um/os-Linux => tools/um/uml}/irq.c (100%)
>   rename {arch/um/os-Linux => tools/um/uml}/main.c (100%)
>   rename {arch/um/os-Linux => tools/um/uml}/mem.c (100%)
>   create mode 100644 tools/um/uml/nommu/Build
>   create mode 100644 tools/um/uml/nommu/registers.c
>   create mode 100644 tools/um/uml/nommu/unimplemented.c
>   rename {arch/um/os-Linux => tools/um/uml}/process.c (99%)
>   rename {arch/um/os-Linux => tools/um/uml}/registers.c (100%)
>   rename {arch/um/os-Linux => tools/um/uml}/sigio.c (100%)
>   rename {arch/um/os-Linux => tools/um/uml}/signal.c (96%)
>   rename arch/um/os-Linux/skas/Makefile => tools/um/uml/skas/Build (56%)
>   rename {arch/um/os-Linux => tools/um/uml}/skas/mem.c (100%)
>   rename {arch/um/os-Linux => tools/um/uml}/skas/process.c (99%)
>   rename {arch/um/os-Linux => tools/um/uml}/start_up.c (100%)
>   rename {arch/um/os-Linux => tools/um/uml}/time.c (100%)
>   rename {arch/um/os-Linux => tools/um/uml}/tty.c (100%)
>   rename {arch/um/os-Linux => tools/um/uml}/umid.c (100%)
>   rename {arch/um/os-Linux => tools/um/uml}/util.c (89%)
>   create mode 100644 tools/um/uml/x86/Build
>   rename {arch/x86/um/os-Linux => tools/um/uml/x86}/mcontext.c (100%)
>   rename {arch/x86/um/os-Linux => tools/um/uml/x86}/prctl.c (100%)
>   rename {arch/x86/um/os-Linux => tools/um/uml/x86}/registers.c (100%)
>   rename {arch/x86/um/os-Linux => tools/um/uml/x86}/task_size.c (100%)
>   rename {arch/x86/um/os-Linux => tools/um/uml/x86}/tls.c (100%)
>
-- 
Anton R. Ivanov
Cambridgegreys Limited. Registered in England. Company Number 10273661
https://www.cambridgegreys.com/


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 00/21] Unifying LKL into UML
@ 2020-10-07 13:30           ` Anton Ivanov
  0 siblings, 0 replies; 476+ messages in thread
From: Anton Ivanov @ 2020-10-07 13:30 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard
  Cc: tavi.purdila, linux-kernel-library, retrage01, linux-arch


On 06/10/2020 10:44, Hajime Tazaki wrote:
> This is another spin of the unification of LKL into UML.  Based on the
> discussion of v4 patchset, we have tried to address issue raised and
> rewrote the patchset from scratch.  The summary is listed in the
> changelog below.
>
> Although there are still bugs in the patchset, we'd like to ask your
> opinions on the design we changed.
>
> The milestone section is also updated: this patchset is for the
> milestone 1, though the common init API is still not implemented yet.
>
>
> Changes in rfc v7:
> - preserve `make ARCH=um` syntax to build UML
> - introduce `make ARCH=um UMMODE=library` to build library mode
> - fix undefined symbols issue during modpost
> - clean up makefiles (arch/um, tools/um)

Hi Hajime, hi Tavi,

Our starting point should be that it does not break the existing build. It still does.

If I build a "stock configuration" UML after applying the patchset the resulting vmlinux is not executable.

On the positive side, it builds cleanly now. I will try to go through the rest of the patchset later today and see if there is anything else that needs fixing before we do the next version.

Brgds,

A.

>
> Changes in rfc v6:
> - rebase with the current linus tree
>
> Changes in rfc v5:
> - rewrite whole patchset from scratch
> - move arch-dependent code of arch/um and arch/x86/um to tools/um
>   - mainly code under os-Linux/ involved
>   - introduce 2-stage build (kernel, and host-dependent parts)
>   - clean up vmlinux.lds.S
> - put LKL-specific implementations as a SUBARCH under arch/um/nommu
> - introduce !CONFIG_MMU in arch/um
> - use struct arch_thread and arch_switch_to() for subarch-specific
>    thread implementation
> - integrate with the IRQ infrastructure of UML
> - tested with block device drivers (ubd) for the proof
>
> Changes in rfc v4: (https://lwn.net/Articles/816276/)
> - Rebase on the current uml/master branch
> - Fix IRQ handling (bug fix)
> - drop a patch for CONFIG_GENERIC_ATOMIC64 (comment by Peter Zijlstra)
> - implement vector net driver for UMMODE_LIB (comment by Anton)
> - clean up uapi headers to avoid duplicates
> - clean up IRQ handling code (comment by Anton)
> - fix error handling in test code (comments by David Disseldorp)
>
> Changes in rfc v3:
> - use UML drivers (net, block) from LKL programs
> - drop virtio device implementations
> - drop mingw32 (Windows) host
> - drop android (arm/aarch64) host
> - drop FreeBSD (x86_64) host
> - drop LD_PRELOAD (hijack) support
> - update milestone
>
> rfc v2:
> - use UMMODE instead of SUBARCH to switch UML or LKL
> - tools/lkl directory is still there. I confirmed we can move under arch/um
>    (e.g., arch/um/lkl/hosts).  I will move it IF this is preferable.
> - drop several patches involved non-uml directory
> - drop several patches which are not required
> - refine commit logs
> - document updated
>
>
>
>
> LKL (Linux Kernel Library) is aiming to allow reusing the Linux kernel code
> as extensively as possible with minimal effort and reduced maintenance
> overhead.
>
> Examples of how LKL can be used are: creating userspace applications
> (running on Linux and other operating systems) that can read or write Linux
> filesystems or can use the Linux networking stack, creating kernel drivers
> for other operating systems that can read Linux filesystems, bootloaders
> support for reading/writing Linux filesystems, etc.
>
> With LKL, the kernel code is compiled into an object file that can be
> directly linked by applications. The API offered by LKL is based on the
> Linux system call interface.
>
> LKL is originally implemented as an architecture port in arch/lkl, but this
> series of commits tries to integrate this into arch/um as one of the mode
> of UML.  This was discussed during RFC email of LKL (*1).
>
> The latest LKL version can be found at https://github.com/lkl/linux
>
> Milestone
> =========
> This patches is a first step toward upstreaming *library mode* of Linux kernel,
> but we think we need to have several steps toward our goal, describing in the
> below.
>
>
> Milestone 1: LKL lib on top of UML
>   * Kernel - Host build split
>   -  Build UML as a relocatable object using the UML's kernel linker script.
>   -  Move the ptrace and other well isolated os code out of arch/um to
>      tools/um
>   -  Use standard host toolchain to create a static library stripped of
>      the ptrace code. Use standard host toolchain to build the main UML
>      executable.
>   -  Add library init API that creates the UML kernel process and starts
>      UML.
>   * System calls APIs
>   -  Add new system call interface based on UML's irq facility.
>   -  Use the LKL scripts to export the required headers to create system
>      calls APIs that use the UML system calls infrastructure.
>   -  Keep the underlying host and driver operations (threads, irqs, etc.)
>      as they are now in UML.
>   * Boot test
>   -  Port the LKL boot test to verify that we are able to programatically
>      issue system calls.
>
> Milestone 2: add virtio disk support
>   * Export asm/io.h operations to host/os. Create IO access operations
>     and redirect them to weak os_ variants that use the current UML
>     implementation.
>   * Add the LKL IO access layer including generic virtio handling and the
>     virtio block device code.
>   * Port LKL disk test and disk apps (lklfuse, fs2tar, cptofs)
>
> Milestone 3: new arch ports
>    * Abstract the system call / IRQ mode the move the implementation to host
>    * Abstract the thread model and move the implementation to host
>    * Add LKL thread model and LKL ports
>
>
> Building LKL the host library and LKL applications
> ==================================================
>
> % cd tools/um
> % make
>
> will build LKL as a object file, it will install it in tools/um/lib together
> with the headers files in tools/um/include then will build the host library,
> tests and a few of application examples:
>
> * tests/boot - a simple applications that uses LKL and exercises the basic
>    LKL APIs
>
> % make run-tests
>
> should run the above `tests/boot` and `tests/net-test` and report errors if
> there are any.
>
> Supported hosts
> ===============
>
> Currently LKL supports Linux userspace applications. New hosts can be added
> relatively easy if the host supports gcc and GNU ld. Previous versions of
> LKL supported Windows kernel and Haiku kernel hosts, and we also have WIP
> patches with rump-hypercall interface, used in UEFI, as well as macOS
> userspace (part of POSIX).
>
> There is also musl-libc port for LKL, which might be interested in for some
> folks.
>
>
> Further readings about LKL
> =========================
>
> - Discussion in github LKL issue
> https://github.com/lkl/linux/issues/304
>
> - LKL (an article)
> https://www.researchgate.net/profile/Nicolae_Tapus2/publication/224164682_LKL_The_Linux_kernel_library/links/02bfe50fd921ab4f7c000000.pdf
>
> *1 RFC email to LKML (back in 2015)
> https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1012277.html
>
>
> Please review the following changes for suitability for inclusion. If you have
> any objections or suggestions for improvement, please respond to the patches. If
> you agree with the changes, please provide your Acked-by.
>
> The following changes since commit 805c6d3c19210c90c109107d189744e960eae025:
>
>    Merge branch 'fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs (2020-09-22 15:08:41 -0700)
>
> are available in the Git repository at:
>
>    git://github.com/thehajime/linux a8686efcbad3c445d3e5bde054c04a58877915f1
>    https://github.com/thehajime/linux/tree/uml-lkl-5.9rc6-v7
>
> Hajime Tazaki (18):
>    um: move arch/um/os-Linux dir to tools/um/uml
>    um: move arch/x86/um/os-Linux to tools/um/uml/
>    scritps: um: suppress warnings if SRCARCH=um
>    um: extend arch_switch_to for alternate SUBARCH
>    um: add nommu mode for UML library mode
>    um: nommu: host interface
>    um: nommu: memory handling
>    um: nommu: kernel thread support
>    um: nommu: system call interface and application API
>    um: nommu: basic console support
>    um: nommu: initialization and cleanup
>    um: nommu: integrate with irq infrastructure of UML
>    um: nommu: plug in the build system
>    um: host: add nommu build for ARCH=um
>    um: host: add utilities functions
>    um: host: posix host operations
>    um: host: add test programs
>    um: nommu: add block device support of UML
>
> Octavian Purdila (3):
>    um: split build in kernel and host parts
>    um: add os init and exit calls
>    um: host: implement os_initcalls and os_exitcalls
>
>   arch/um/Kconfig                               |  29 +-
>   arch/um/Makefile                              |  35 +-
>   arch/um/drivers/Makefile                      |  10 +-
>   .../um/{os-Linux => }/drivers/ethertap_kern.c |   0
>   arch/um/{os-Linux => }/drivers/tuntap_kern.c  |   0
>   arch/um/include/asm/common.lds.S              |  99 ----
>   arch/um/include/asm/host_ops.h                |   9 +
>   arch/um/include/asm/mmu.h                     |   3 +
>   arch/um/include/asm/mmu_context.h             |   8 +
>   arch/um/include/asm/page.h                    |  15 +
>   arch/um/include/asm/pgtable.h                 |  27 +
>   arch/um/include/asm/thread_info.h             |  24 +
>   arch/um/include/asm/uaccess.h                 |   6 +
>   arch/um/include/asm/xor.h                     |   3 +-
>   arch/um/include/shared/as-layout.h            |   1 +
>   .../drivers => include/shared}/etap.h         |   0
>   arch/um/include/shared/init.h                 |  19 +-
>   arch/um/include/shared/os.h                   |   1 +
>   .../drivers => include/shared}/tuntap.h       |   0
>   arch/um/include/uapi/asm/Kbuild               |   2 +
>   arch/um/kernel/Makefile                       |  13 +-
>   arch/um/kernel/dyn.lds.S                      | 171 -------
>   arch/um/kernel/irq.c                          |  13 +
>   arch/um/kernel/process.c                      |  14 +-
>   arch/um/kernel/reboot.c                       |   5 +
>   arch/um/kernel/time.c                         |   2 +
>   arch/um/kernel/um_arch.c                      |  16 +
>   arch/um/kernel/uml.lds.S                      | 115 -----
>   arch/um/{os-Linux => kernel}/user_syms.c      |   0
>   arch/um/kernel/vmlinux.lds.S                  |  91 +++-
>   arch/um/nommu/Makefile                        |   1 +
>   arch/um/nommu/Makefile.um                     |  22 +
>   arch/um/nommu/include/asm/Kbuild              |   6 +
>   arch/um/nommu/include/asm/archparam.h         |   1 +
>   arch/um/nommu/include/asm/atomic.h            |  11 +
>   arch/um/nommu/include/asm/atomic64.h          | 114 +++++
>   arch/um/nommu/include/asm/bitsperlong.h       |  12 +
>   arch/um/nommu/include/asm/byteorder.h         |   7 +
>   arch/um/nommu/include/asm/cpu.h               |  16 +
>   arch/um/nommu/include/asm/elf.h               |  15 +
>   arch/um/nommu/include/asm/mm_context.h        |   8 +
>   arch/um/nommu/include/asm/processor.h         |  46 ++
>   arch/um/nommu/include/asm/ptrace.h            |  21 +
>   arch/um/nommu/include/asm/sched.h             |  23 +
>   arch/um/nommu/include/asm/segment.h           |   9 +
>   arch/um/nommu/include/asm/syscall_wrapper.h   |  57 +++
>   arch/um/nommu/include/asm/syscalls.h          |  15 +
>   arch/um/nommu/include/uapi/asm/Kbuild         |   6 +
>   arch/um/nommu/include/uapi/asm/bitsperlong.h  |  11 +
>   arch/um/nommu/include/uapi/asm/byteorder.h    |  11 +
>   arch/um/nommu/include/uapi/asm/host_ops.h     | 122 +++++
>   arch/um/nommu/include/uapi/asm/sigcontext.h   |  12 +
>   arch/um/nommu/include/uapi/asm/syscalls.h     | 287 +++++++++++
>   arch/um/nommu/include/uapi/asm/unistd.h       |  17 +
>   arch/um/nommu/um/Kconfig                      |  45 ++
>   arch/um/nommu/um/Makefile                     |   4 +
>   arch/um/nommu/um/bootmem.c                    |  87 ++++
>   arch/um/nommu/um/console.c                    |  42 ++
>   arch/um/nommu/um/cpu.c                        | 247 ++++++++++
>   arch/um/nommu/um/delay.c                      |  31 ++
>   arch/um/nommu/um/setup.c                      | 179 +++++++
>   arch/um/nommu/um/shared/sysdep/archsetjmp.h   |  13 +
>   arch/um/nommu/um/shared/sysdep/faultinfo.h    |   8 +
>   .../nommu/um/shared/sysdep/kernel-offsets.h   |  12 +
>   arch/um/nommu/um/shared/sysdep/mcontext.h     |   9 +
>   arch/um/nommu/um/shared/sysdep/ptrace.h       |  42 ++
>   arch/um/nommu/um/shared/sysdep/ptrace_user.h  |   7 +
>   arch/um/nommu/um/syscalls.c                   | 199 ++++++++
>   arch/um/nommu/um/threads.c                    | 262 ++++++++++
>   arch/um/nommu/um/unimplemented.c              |  70 +++
>   arch/um/nommu/um/user_constants.h             |  13 +
>   arch/um/os-Linux/Makefile                     |  19 -
>   arch/um/os-Linux/drivers/Makefile             |  13 -
>   arch/um/scripts/headers_install.py            | 197 ++++++++
>   arch/x86/um/Makefile                          |   2 +-
>   arch/x86/um/os-Linux/Makefile                 |  13 -
>   arch/x86/um/ptrace_32.c                       |   2 +-
>   arch/x86/um/syscalls_64.c                     |   2 +-
>   scripts/headers_install.sh                    |   4 +
>   scripts/link-vmlinux.sh                       |  42 +-
>   tools/um/.gitignore                           |   1 +
>   tools/um/Makefile                             |  81 +++
>   tools/um/Targets                              |  13 +
>   tools/um/include/lkl.h                        | 357 ++++++++++++++
>   tools/um/include/lkl_host.h                   |  27 +
>   tools/um/lib/Build                            |   7 +
>   tools/um/lib/fs.c                             | 461 ++++++++++++++++++
>   tools/um/lib/jmp_buf.c                        |  14 +
>   tools/um/lib/jmp_buf.h                        |   8 +
>   tools/um/lib/posix-host.c                     | 293 +++++++++++
>   tools/um/lib/utils.c                          | 213 ++++++++
>   tools/um/tests/Build                          |   4 +
>   tools/um/tests/boot.c                         | 393 +++++++++++++++
>   tools/um/tests/boot.sh                        |   9 +
>   tools/um/tests/cla.c                          | 159 ++++++
>   tools/um/tests/cla.h                          |  33 ++
>   tools/um/tests/disk.c                         | 168 +++++++
>   tools/um/tests/disk.sh                        |  70 +++
>   tools/um/tests/run.py                         | 174 +++++++
>   tools/um/tests/tap13.py                       | 209 ++++++++
>   tools/um/tests/test.c                         | 126 +++++
>   tools/um/tests/test.h                         |  72 +++
>   tools/um/tests/test.sh                        | 181 +++++++
>   tools/um/uml/Build                            |  58 +++
>   tools/um/uml/drivers/Build                    |  10 +
>   .../um/uml}/drivers/ethertap_user.c           |   0
>   .../um/uml}/drivers/tuntap_user.c             |   0
>   {arch/um/os-Linux => tools/um/uml}/elf_aux.c  |   0
>   {arch/um/os-Linux => tools/um/uml}/execvp.c   |   4 -
>   {arch/um/os-Linux => tools/um/uml}/file.c     |   0
>   {arch/um/os-Linux => tools/um/uml}/helper.c   |   0
>   {arch/um/os-Linux => tools/um/uml}/irq.c      |   0
>   {arch/um/os-Linux => tools/um/uml}/main.c     |   0
>   {arch/um/os-Linux => tools/um/uml}/mem.c      |   0
>   tools/um/uml/nommu/Build                      |   1 +
>   tools/um/uml/nommu/registers.c                |  21 +
>   tools/um/uml/nommu/unimplemented.c            |  21 +
>   {arch/um/os-Linux => tools/um/uml}/process.c  |   2 +
>   .../um/os-Linux => tools/um/uml}/registers.c  |   0
>   {arch/um/os-Linux => tools/um/uml}/sigio.c    |   0
>   {arch/um/os-Linux => tools/um/uml}/signal.c   |  12 +-
>   .../skas/Makefile => tools/um/uml/skas/Build  |   6 +-
>   {arch/um/os-Linux => tools/um/uml}/skas/mem.c |   0
>   .../os-Linux => tools/um/uml}/skas/process.c  |   3 +-
>   {arch/um/os-Linux => tools/um/uml}/start_up.c |   0
>   {arch/um/os-Linux => tools/um/uml}/time.c     |   0
>   {arch/um/os-Linux => tools/um/uml}/tty.c      |   0
>   {arch/um/os-Linux => tools/um/uml}/umid.c     |   0
>   {arch/um/os-Linux => tools/um/uml}/util.c     |  26 +
>   tools/um/uml/x86/Build                        |  11 +
>   .../os-Linux => tools/um/uml/x86}/mcontext.c  |   0
>   .../um/os-Linux => tools/um/uml/x86}/prctl.c  |   0
>   .../os-Linux => tools/um/uml/x86}/registers.c |   0
>   .../os-Linux => tools/um/uml/x86}/task_size.c |   0
>   .../um/os-Linux => tools/um/uml/x86}/tls.c    |   0
>   135 files changed, 5870 insertions(+), 523 deletions(-)
>   rename arch/um/{os-Linux => }/drivers/ethertap_kern.c (100%)
>   rename arch/um/{os-Linux => }/drivers/tuntap_kern.c (100%)
>   delete mode 100644 arch/um/include/asm/common.lds.S
>   create mode 100644 arch/um/include/asm/host_ops.h
>   rename arch/um/{os-Linux/drivers => include/shared}/etap.h (100%)
>   rename arch/um/{os-Linux/drivers => include/shared}/tuntap.h (100%)
>   create mode 100644 arch/um/include/uapi/asm/Kbuild
>   delete mode 100644 arch/um/kernel/dyn.lds.S
>   delete mode 100644 arch/um/kernel/uml.lds.S
>   rename arch/um/{os-Linux => kernel}/user_syms.c (100%)
>   create mode 100644 arch/um/nommu/Makefile
>   create mode 100644 arch/um/nommu/Makefile.um
>   create mode 100644 arch/um/nommu/include/asm/Kbuild
>   create mode 100644 arch/um/nommu/include/asm/archparam.h
>   create mode 100644 arch/um/nommu/include/asm/atomic.h
>   create mode 100644 arch/um/nommu/include/asm/atomic64.h
>   create mode 100644 arch/um/nommu/include/asm/bitsperlong.h
>   create mode 100644 arch/um/nommu/include/asm/byteorder.h
>   create mode 100644 arch/um/nommu/include/asm/cpu.h
>   create mode 100644 arch/um/nommu/include/asm/elf.h
>   create mode 100644 arch/um/nommu/include/asm/mm_context.h
>   create mode 100644 arch/um/nommu/include/asm/processor.h
>   create mode 100644 arch/um/nommu/include/asm/ptrace.h
>   create mode 100644 arch/um/nommu/include/asm/sched.h
>   create mode 100644 arch/um/nommu/include/asm/segment.h
>   create mode 100644 arch/um/nommu/include/asm/syscall_wrapper.h
>   create mode 100644 arch/um/nommu/include/asm/syscalls.h
>   create mode 100644 arch/um/nommu/include/uapi/asm/Kbuild
>   create mode 100644 arch/um/nommu/include/uapi/asm/bitsperlong.h
>   create mode 100644 arch/um/nommu/include/uapi/asm/byteorder.h
>   create mode 100644 arch/um/nommu/include/uapi/asm/host_ops.h
>   create mode 100644 arch/um/nommu/include/uapi/asm/sigcontext.h
>   create mode 100644 arch/um/nommu/include/uapi/asm/syscalls.h
>   create mode 100644 arch/um/nommu/include/uapi/asm/unistd.h
>   create mode 100644 arch/um/nommu/um/Kconfig
>   create mode 100644 arch/um/nommu/um/Makefile
>   create mode 100644 arch/um/nommu/um/bootmem.c
>   create mode 100644 arch/um/nommu/um/console.c
>   create mode 100644 arch/um/nommu/um/cpu.c
>   create mode 100644 arch/um/nommu/um/delay.c
>   create mode 100644 arch/um/nommu/um/setup.c
>   create mode 100644 arch/um/nommu/um/shared/sysdep/archsetjmp.h
>   create mode 100644 arch/um/nommu/um/shared/sysdep/faultinfo.h
>   create mode 100644 arch/um/nommu/um/shared/sysdep/kernel-offsets.h
>   create mode 100644 arch/um/nommu/um/shared/sysdep/mcontext.h
>   create mode 100644 arch/um/nommu/um/shared/sysdep/ptrace.h
>   create mode 100644 arch/um/nommu/um/shared/sysdep/ptrace_user.h
>   create mode 100644 arch/um/nommu/um/syscalls.c
>   create mode 100644 arch/um/nommu/um/threads.c
>   create mode 100644 arch/um/nommu/um/unimplemented.c
>   create mode 100644 arch/um/nommu/um/user_constants.h
>   delete mode 100644 arch/um/os-Linux/Makefile
>   delete mode 100644 arch/um/os-Linux/drivers/Makefile
>   create mode 100755 arch/um/scripts/headers_install.py
>   delete mode 100644 arch/x86/um/os-Linux/Makefile
>   create mode 100644 tools/um/.gitignore
>   create mode 100644 tools/um/Makefile
>   create mode 100644 tools/um/Targets
>   create mode 100644 tools/um/include/lkl.h
>   create mode 100644 tools/um/include/lkl_host.h
>   create mode 100644 tools/um/lib/Build
>   create mode 100644 tools/um/lib/fs.c
>   create mode 100644 tools/um/lib/jmp_buf.c
>   create mode 100644 tools/um/lib/jmp_buf.h
>   create mode 100644 tools/um/lib/posix-host.c
>   create mode 100644 tools/um/lib/utils.c
>   create mode 100644 tools/um/tests/Build
>   create mode 100644 tools/um/tests/boot.c
>   create mode 100755 tools/um/tests/boot.sh
>   create mode 100644 tools/um/tests/cla.c
>   create mode 100644 tools/um/tests/cla.h
>   create mode 100644 tools/um/tests/disk.c
>   create mode 100755 tools/um/tests/disk.sh
>   create mode 100755 tools/um/tests/run.py
>   create mode 100644 tools/um/tests/tap13.py
>   create mode 100644 tools/um/tests/test.c
>   create mode 100644 tools/um/tests/test.h
>   create mode 100644 tools/um/tests/test.sh
>   create mode 100644 tools/um/uml/Build
>   create mode 100644 tools/um/uml/drivers/Build
>   rename {arch/um/os-Linux => tools/um/uml}/drivers/ethertap_user.c (100%)
>   rename {arch/um/os-Linux => tools/um/uml}/drivers/tuntap_user.c (100%)
>   rename {arch/um/os-Linux => tools/um/uml}/elf_aux.c (100%)
>   rename {arch/um/os-Linux => tools/um/uml}/execvp.c (98%)
>   rename {arch/um/os-Linux => tools/um/uml}/file.c (100%)
>   rename {arch/um/os-Linux => tools/um/uml}/helper.c (100%)
>   rename {arch/um/os-Linux => tools/um/uml}/irq.c (100%)
>   rename {arch/um/os-Linux => tools/um/uml}/main.c (100%)
>   rename {arch/um/os-Linux => tools/um/uml}/mem.c (100%)
>   create mode 100644 tools/um/uml/nommu/Build
>   create mode 100644 tools/um/uml/nommu/registers.c
>   create mode 100644 tools/um/uml/nommu/unimplemented.c
>   rename {arch/um/os-Linux => tools/um/uml}/process.c (99%)
>   rename {arch/um/os-Linux => tools/um/uml}/registers.c (100%)
>   rename {arch/um/os-Linux => tools/um/uml}/sigio.c (100%)
>   rename {arch/um/os-Linux => tools/um/uml}/signal.c (96%)
>   rename arch/um/os-Linux/skas/Makefile => tools/um/uml/skas/Build (56%)
>   rename {arch/um/os-Linux => tools/um/uml}/skas/mem.c (100%)
>   rename {arch/um/os-Linux => tools/um/uml}/skas/process.c (99%)
>   rename {arch/um/os-Linux => tools/um/uml}/start_up.c (100%)
>   rename {arch/um/os-Linux => tools/um/uml}/time.c (100%)
>   rename {arch/um/os-Linux => tools/um/uml}/tty.c (100%)
>   rename {arch/um/os-Linux => tools/um/uml}/umid.c (100%)
>   rename {arch/um/os-Linux => tools/um/uml}/util.c (89%)
>   create mode 100644 tools/um/uml/x86/Build
>   rename {arch/x86/um/os-Linux => tools/um/uml/x86}/mcontext.c (100%)
>   rename {arch/x86/um/os-Linux => tools/um/uml/x86}/prctl.c (100%)
>   rename {arch/x86/um/os-Linux => tools/um/uml/x86}/registers.c (100%)
>   rename {arch/x86/um/os-Linux => tools/um/uml/x86}/task_size.c (100%)
>   rename {arch/x86/um/os-Linux => tools/um/uml/x86}/tls.c (100%)
>
-- 
Anton R. Ivanov
Cambridgegreys Limited. Registered in England. Company Number 10273661
https://www.cambridgegreys.com/


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 21/21] um: nommu: add block device support of UML
  2020-10-06  9:44           ` Hajime Tazaki
@ 2020-10-07 14:17             ` Anton Ivanov
  -1 siblings, 0 replies; 476+ messages in thread
From: Anton Ivanov @ 2020-10-07 14:17 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard
  Cc: tavi.purdila, linux-kernel-library, linux-arch, retrage01


On 06/10/2020 10:44, Hajime Tazaki wrote:
> This commit adds block device support for nommu mode, and also added
> several host utilities to run a simple test program
> (tools/um/test/disk.c) successfully.
>
> Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
> ---
>   arch/um/include/asm/xor.h                 |   3 +-
>   arch/um/include/shared/as-layout.h        |   1 +
>   arch/um/kernel/um_arch.c                  |   5 +
>   arch/um/nommu/include/uapi/asm/host_ops.h |   7 +
>   arch/um/nommu/include/uapi/asm/unistd.h   |   2 +
>   arch/um/nommu/um/Kconfig                  |   8 +
>   arch/um/nommu/um/setup.c                  |  12 +
>   tools/um/Targets                          |   2 +
>   tools/um/include/lkl.h                    | 206 ++++++++++
>   tools/um/include/lkl_host.h               |   1 +
>   tools/um/lib/Build                        |   1 +
>   tools/um/lib/fs.c                         | 461 ++++++++++++++++++++++
>   tools/um/lib/posix-host.c                 |   1 +
>   tools/um/lib/utils.c                      |   3 +
>   tools/um/tests/Build                      |   1 +
>   tools/um/tests/cla.c                      | 159 ++++++++
>   tools/um/tests/cla.h                      |  33 ++
>   tools/um/tests/disk.c                     | 168 ++++++++
>   tools/um/tests/disk.sh                    |  70 ++++
>   tools/um/tests/run.py                     |   2 +
>   20 files changed, 1145 insertions(+), 1 deletion(-)
>   create mode 100644 tools/um/lib/fs.c
>   create mode 100644 tools/um/tests/cla.c
>   create mode 100644 tools/um/tests/cla.h
>   create mode 100644 tools/um/tests/disk.c
>   create mode 100755 tools/um/tests/disk.sh
>
> diff --git a/arch/um/include/asm/xor.h b/arch/um/include/asm/xor.h
> index 36b33d62a35d..a5ea458a1ae9 100644
> --- a/arch/um/include/asm/xor.h
> +++ b/arch/um/include/asm/xor.h
> @@ -4,4 +4,5 @@
>   
>   /* pick an arbitrary one - measuring isn't possible with inf-cpu */
>   #define XOR_SELECT_TEMPLATE(x)	\
> -	(time_travel_mode == TT_MODE_INFCPU ? &xor_block_8regs : NULL)
> +	(time_travel_mode == TT_MODE_INFCPU || (!IS_ENABLED(CONFIG_MMU)) ? \
> +	 &xor_block_8regs : NULL)
> diff --git a/arch/um/include/shared/as-layout.h b/arch/um/include/shared/as-layout.h
> index 5f286ef2721b..4423437a5ace 100644
> --- a/arch/um/include/shared/as-layout.h
> +++ b/arch/um/include/shared/as-layout.h
> @@ -57,6 +57,7 @@ extern unsigned long host_task_size;
>   
>   extern int linux_main(int argc, char **argv);
>   extern void uml_finishsetup(void);
> +extern void uml_set_args(char *args);
>   
>   struct siginfo;
>   extern void (*sig_info[])(int, struct siginfo *si, struct uml_pt_regs *);
> diff --git a/arch/um/kernel/um_arch.c b/arch/um/kernel/um_arch.c
> index e2cb76c03b25..da6e06cf2808 100644
> --- a/arch/um/kernel/um_arch.c
> +++ b/arch/um/kernel/um_arch.c
> @@ -41,6 +41,11 @@ static void __init add_arg(char *arg)
>   	strcat(command_line, arg);
>   }
>   
> +void __init uml_set_args(char *args)
> +{
> +	strcat(command_line, args);
> +}
> +
>   /*
>    * These fields are initialized at boot time and not changed.
>    * XXX This structure is used only in the non-SMP case.  Maybe this
> diff --git a/arch/um/nommu/include/uapi/asm/host_ops.h b/arch/um/nommu/include/uapi/asm/host_ops.h
> index 0811494c080c..1b2507502969 100644
> --- a/arch/um/nommu/include/uapi/asm/host_ops.h
> +++ b/arch/um/nommu/include/uapi/asm/host_ops.h
> @@ -15,6 +15,11 @@ struct lkl_jmp_buf {
>    *
>    * These operations must be provided by a host library or by the application
>    * itself.
> + *
> + * @um_devices - string containg the list of UML devices in command line
> + * format. This string is appended to the kernel command line and
> + * is provided here for convenience to be implemented by the host library.
> + *
>    * @print - optional operation that receives console messages
>    * @panic - called during a kernel panic
>    *
> @@ -65,6 +70,8 @@ struct lkl_jmp_buf {
>    *
>    */
>   struct lkl_host_operations {
> +	const char *um_devices;
> +
>   	void (*print)(const char *str, int len);
>   	void (*panic)(void);
>   
> diff --git a/arch/um/nommu/include/uapi/asm/unistd.h b/arch/um/nommu/include/uapi/asm/unistd.h
> index 1762ebb829f5..320762099a62 100644
> --- a/arch/um/nommu/include/uapi/asm/unistd.h
> +++ b/arch/um/nommu/include/uapi/asm/unistd.h
> @@ -3,6 +3,8 @@
>   #define __UM_NOMMU_UAPI_UNISTD_H
>   
>   #define __ARCH_WANT_NEW_STAT
> +#define __ARCH_WANT_SET_GET_RLIMIT
> +
>   #include <asm/bitsperlong.h>
>   
>   #if __BITS_PER_LONG == 64
> diff --git a/arch/um/nommu/um/Kconfig b/arch/um/nommu/um/Kconfig
> index 20b3eaccb6f0..c6a3f472fe75 100644
> --- a/arch/um/nommu/um/Kconfig
> +++ b/arch/um/nommu/um/Kconfig
> @@ -4,6 +4,10 @@ config UML_NOMMU
>   	select UACCESS_MEMCPY
>   	select ARCH_THREAD_STACK_ALLOCATOR
>   	select ARCH_HAS_SYSCALL_WRAPPER
> +	select VFAT_FS
> +	select NLS_CODEPAGE_437
> +	select NLS_ISO8859_1
> +	select BTRFS_FS
>   
>   config 64BIT
>   	bool
> @@ -35,3 +39,7 @@ config STACKTRACE_SUPPORT
>   config PRINTK_TIME
>   	bool
>   	default y
> +
> +config RAID6_PQ_BENCHMARK
> +	bool
> +	default n

Why are we touching this? I thought this is already defined in lib/Kconfig?

Brgds,

A.

> diff --git a/arch/um/nommu/um/setup.c b/arch/um/nommu/um/setup.c
> index 922188690139..d71d61979ad6 100644
> --- a/arch/um/nommu/um/setup.c
> +++ b/arch/um/nommu/um/setup.c
> @@ -45,13 +45,25 @@ static void __init *lkl_run_kernel(void *arg)
>   	return NULL;
>   }
>   
> +static char _cmd_line[COMMAND_LINE_SIZE];
>   int __init lkl_start_kernel(struct lkl_host_operations *ops,
>   			    const char *fmt, ...)
>   {
> +	va_list ap;
>   	int ret;
>   
>   	lkl_ops = ops;
>   
> +	va_start(ap, fmt);
> +	ret = vsnprintf(_cmd_line, COMMAND_LINE_SIZE, fmt, ap);
> +	va_end(ap);
> +
> +	if (ops->um_devices)
> +		strscpy(_cmd_line + ret, ops->um_devices,
> +			COMMAND_LINE_SIZE - ret);
> +
> +	uml_set_args(_cmd_line);
> +
>   	init_sem = lkl_ops->sem_alloc(0);
>   	if (!init_sem)
>   		return -ENOMEM;
> diff --git a/tools/um/Targets b/tools/um/Targets
> index 2bb90381843c..f5f8ec4b9dbb 100644
> --- a/tools/um/Targets
> +++ b/tools/um/Targets
> @@ -1,10 +1,12 @@
>   ifeq ($(UMMODE),library)
>   progs-y += tests/boot
> +progs-y += tests/disk
>   else
>   progs-y += uml/linux
>   endif
>   
>   LDFLAGS_boot-y := -pie
> +LDFLAGS_disk-y := -pie
>   
>   LDLIBS := -lrt -lpthread -lutil
>   LDFLAGS_linux-y := -no-pie -Wl,--wrap,malloc -Wl,--wrap,free -Wl,--wrap,calloc
> diff --git a/tools/um/include/lkl.h b/tools/um/include/lkl.h
> index 707e01b64a70..4a9d874bfbf0 100644
> --- a/tools/um/include/lkl.h
> +++ b/tools/um/include/lkl.h
> @@ -113,6 +113,16 @@ static inline long lkl_sys_creat(const char *file, int mode)
>   }
>   #endif
>   
> +#ifdef __lkl__NR_faccessat
> +/**
> + * lkl_sys_access - wrapper for lkl_sys_faccessat
> + */
> +static inline long lkl_sys_access(const char *file, int mode)
> +{
> +	return lkl_sys_faccessat(LKL_AT_FDCWD, file, mode);
> +}
> +#endif
> +
>   #ifdef __lkl__NR_mkdirat
>   /**
>    * lkl_sys_mkdir - wrapper for lkl_sys_mkdirat
> @@ -123,6 +133,36 @@ static inline long lkl_sys_mkdir(const char *path, mode_t mode)
>   }
>   #endif
>   
> +#ifdef __lkl__NR_unlinkat
> +/**
> + * lkl_sys_rmdir - wrapper for lkl_sys_unlinkrat
> + */
> +static inline long lkl_sys_rmdir(const char *path)
> +{
> +	return lkl_sys_unlinkat(LKL_AT_FDCWD, path, LKL_AT_REMOVEDIR);
> +}
> +#endif
> +
> +#ifdef __lkl__NR_unlinkat
> +/**
> + * lkl_sys_unlink - wrapper for lkl_sys_unlinkat
> + */
> +static inline long lkl_sys_unlink(const char *path)
> +{
> +	return lkl_sys_unlinkat(LKL_AT_FDCWD, path, 0);
> +}
> +#endif
> +
> +#ifdef __lkl__NR_mknodat
> +/**
> + * lkl_sys_mknod - wrapper for lkl_sys_mknodat
> + */
> +static inline long lkl_sys_mknod(const char *path, mode_t mode, dev_t dev)
> +{
> +	return lkl_sys_mknodat(LKL_AT_FDCWD, path, mode, dev);
> +}
> +#endif
> +
>   #ifdef __lkl__NR_epoll_create1
>   /**
>    * lkl_sys_epoll_create - wrapper for lkl_sys_epoll_create1
> @@ -144,6 +184,172 @@ static inline long lkl_sys_epoll_wait(int fd, struct lkl_epoll_event *ev,
>   }
>   #endif
>   
> +/**
> + * struct lkl_dev_blk_ops - block device host operations, defined in lkl_host.h.
> + */
> +struct lkl_dev_blk_ops;
> +
> +/**
> + * lkl_disk - host disk handle
> + *
> + * @dev - a pointer to private information for this disk backend
> + * @fd - a POSIX file descriptor that can be used by preadv/pwritev
> + * @handle - an NT file handle that can be used by ReadFile/WriteFile
> + */
> +struct lkl_disk {
> +	void *dev;
> +	union {
> +		int fd;
> +		void *handle;
> +	};
> +	struct lkl_dev_blk_ops *ops;
> +};
> +
> +/**
> + * lkl_disk_add - add a new disk
> + *
> + * @disk - the host disk handle
> + * @returns a disk id (0 is valid) or a strictly negative value in case of error
> + */
> +int lkl_disk_add(struct lkl_disk *disk);
> +
> +/**
> + * lkl_disk_remove - remove a disk
> + *
> + * This function makes a cleanup of the @disk's private information
> + * that was initialized by lkl_disk_add before.
> + *
> + * @disk - the host disk handle
> + */
> +int lkl_disk_remove(struct lkl_disk disk);
> +
> +/**
> + * lkl_encode_dev_from_sysfs_blkdev - extract device id from sysfs
> + *
> + * This function returns the device id for the given sysfs dev node.
> + * The content of the node has to be in the form 'MAJOR:MINOR'.
> + * Also, this function expects an absolute path which means that sysfs
> + * already has to be mounted at the given path
> + *
> + * @sysfs_path - absolute path to the sysfs dev node
> + * @pdevid - pointer to memory where dev id will be returned
> + * @returns - 0 on success, a negative value on error
> + */
> +int lkl_encode_dev_from_sysfs(const char *sysfs_path, uint32_t *pdevid);
> +
> +/**
> + * lkl_mount_dev - mount a disk
> + *
> + * This functions creates a device file for the given disk, creates a mount
> + * point and mounts the device over the mount point.
> + *
> + * @disk_id - the disk id identifying the disk to be mounted
> + * @part - disk partition or zero for full disk
> + * @fs_type - filesystem type
> + * @flags - mount flags
> + * @opts - additional filesystem specific mount options
> + * @mnt_str - a string that will be filled by this function with the path where
> + * the filesystem has been mounted
> + * @mnt_str_len - size of mnt_str
> + * @returns - 0 on success, a negative value on error
> + */
> +long lkl_mount_dev(unsigned int disk_id, unsigned int part, const char *fs_type,
> +		   int flags, const char *opts,
> +		   char *mnt_str, unsigned int mnt_str_len);
> +
> +/**
> + * lkl_umount_dev - umount a disk
> + *
> + * This functions umounts the given disks and removes the device file and the
> + * mount point.
> + *
> + * @disk_id - the disk id identifying the disk to be mounted
> + * @part - disk partition or zero for full disk
> + * @flags - umount flags
> + * @timeout_ms - timeout to wait for the kernel to flush closed files so that
> + * umount can succeed
> + * @returns - 0 on success, a negative value on error
> + */
> +long lkl_umount_dev(unsigned int disk_id, unsigned int part, int flags,
> +		    long timeout_ms);
> +
> +/**
> + * lkl_umount_timeout - umount filesystem with timeout
> + *
> + * @path - the path to unmount
> + * @flags - umount flags
> + * @timeout_ms - timeout to wait for the kernel to flush closed files so that
> + * umount can succeed
> + * @returns - 0 on success, a negative value on error
> + */
> +long lkl_umount_timeout(char *path, int flags, long timeout_ms);
> +
> +/**
> + * lkl_opendir - open a directory
> + *
> + * @path - directory path
> + * @err - pointer to store the error in case of failure
> + * @returns - a handle to be used when calling lkl_readdir
> + */
> +struct lkl_dir *lkl_opendir(const char *path, int *err);
> +
> +/**
> + * lkl_fdopendir - open a directory
> + *
> + * @fd - file descriptor
> + * @err - pointer to store the error in case of failure
> + * @returns - a handle to be used when calling lkl_readdir
> + */
> +struct lkl_dir *lkl_fdopendir(int fd, int *err);
> +
> +/**
> + * lkl_rewinddir - reset directory stream
> + *
> + * @dir - the directory handler as returned by lkl_opendir
> + */
> +void lkl_rewinddir(struct lkl_dir *dir);
> +
> +/**
> + * lkl_closedir - close the directory
> + *
> + * @dir - the directory handler as returned by lkl_opendir
> + */
> +int lkl_closedir(struct lkl_dir *dir);
> +
> +/**
> + * lkl_readdir - get the next available entry of the directory
> + *
> + * @dir - the directory handler as returned by lkl_opendir
> + * @returns - a lkl_dirent64 entry or NULL if the end of the directory stream is
> + * reached or if an error occurred; check lkl_errdir() to distinguish between
> + * errors or end of the directory stream
> + */
> +struct lkl_linux_dirent64 *lkl_readdir(struct lkl_dir *dir);
> +
> +/**
> + * lkl_errdir - checks if an error occurred during the last lkl_readdir call
> + *
> + * @dir - the directory handler as returned by lkl_opendir
> + * @returns - 0 if no error occurred, or a negative value otherwise
> + */
> +int lkl_errdir(struct lkl_dir *dir);
> +
> +/**
> + * lkl_dirfd - gets the file descriptor associated with the directory handle
> + *
> + * @dir - the directory handle as returned by lkl_opendir
> + * @returns - a positive value,which is the LKL file descriptor associated with
> + * the directory handle, or a negative value otherwise
> + */
> +int lkl_dirfd(struct lkl_dir *dir);
> +
> +/**
> + * lkl_mount_fs - mount a file system type like proc, sys
> + * @fstype - file system type. e.g. proc, sys
> + * @returns - 0 on success. 1 if it's already mounted. negative on failure.
> + */
> +int lkl_mount_fs(char *fstype);
> +
>   #ifdef __cplusplus
>   }
>   #endif
> diff --git a/tools/um/include/lkl_host.h b/tools/um/include/lkl_host.h
> index 85e80eb4ad0d..12dd95616e43 100644
> --- a/tools/um/include/lkl_host.h
> +++ b/tools/um/include/lkl_host.h
> @@ -10,6 +10,7 @@ extern "C" {
>   #include <lkl.h>
>   
>   extern struct lkl_host_operations lkl_host_ops;
> +extern char lkl_um_devs[4096];
>   
>   /**
>    * lkl_printf - print a message via the host print operation
> diff --git a/tools/um/lib/Build b/tools/um/lib/Build
> index dddff26a3b4e..29491b40746c 100644
> --- a/tools/um/lib/Build
> +++ b/tools/um/lib/Build
> @@ -4,3 +4,4 @@ CFLAGS_posix-host.o += -D_FILE_OFFSET_BITS=64
>   liblinux-$(CONFIG_UMMODE_LIB) += utils.o
>   liblinux-$(CONFIG_UMMODE_LIB) += posix-host.o
>   liblinux-$(CONFIG_UMMODE_LIB) += jmp_buf.o
> +liblinux-$(CONFIG_UMMODE_LIB) += fs.o
> diff --git a/tools/um/lib/fs.c b/tools/um/lib/fs.c
> new file mode 100644
> index 000000000000..948aac9730c2
> --- /dev/null
> +++ b/tools/um/lib/fs.c
> @@ -0,0 +1,461 @@
> +// SPDX-License-Identifier: GPL-2.0
> +#include <stdarg.h>
> +#include <stdio.h>
> +#include <string.h>
> +#include <stdlib.h>
> +#include <lkl_host.h>
> +
> +#define MAX_FSTYPE_LEN 50
> +
> +static struct lkl_disk *lkl_disks[16];
> +static int registered_blk_dev_idx;
> +
> +static int lkl_disk_um_add(struct lkl_disk *disk, const char *blkparams)
> +{
> +	/* concat strings */
> +	snprintf(lkl_um_devs + strlen(lkl_um_devs), sizeof(lkl_um_devs),
> +		 " ubd%d=%s", registered_blk_dev_idx, blkparams);
> +
> +	return registered_blk_dev_idx++;
> +}
> +
> +int lkl_disk_add(struct lkl_disk *disk)
> +{
> +	int ret = -1;
> +
> +	ret = lkl_disk_um_add(disk, disk->dev);
> +
> +	lkl_disks[ret] = disk;
> +
> +	return ret;
> +}
> +
> +int lkl_disk_remove(struct lkl_disk disk)
> +{
> +	/* FIXME */
> +	return 0;
> +}
> +
> +int lkl_mount_fs(char *fstype)
> +{
> +	char dir[MAX_FSTYPE_LEN+2] = "/";
> +	int flags = 0, ret = 0;
> +
> +	strncat(dir, fstype, MAX_FSTYPE_LEN);
> +
> +	/* Create with regular umask */
> +	ret = lkl_sys_mkdir(dir, 0xff);
> +	if (ret && ret != -LKL_EEXIST) {
> +		lkl_perror("mount_fs mkdir", ret);
> +		return ret;
> +	}
> +
> +	/* We have no use for nonzero flags right now */
> +	ret = lkl_sys_mount("none", dir, fstype, flags, NULL);
> +	if (ret && ret != -LKL_EBUSY) {
> +		lkl_sys_rmdir(dir);
> +		return ret;
> +	}
> +
> +	if (ret == -LKL_EBUSY)
> +		return 1;
> +	return 0;
> +}
> +
> +static uint32_t new_encode_dev(unsigned int major, unsigned int minor)
> +{
> +	return (minor & 0xff) | (major << 8) | ((minor & ~0xff) << 12);
> +}
> +
> +static int startswith(const char *str, const char *pre)
> +{
> +	return strncmp(pre, str, strlen(pre)) == 0;
> +}
> +
> +static int get_node_with_prefix(const char *path, const char *prefix,
> +				char *result, unsigned int result_len)
> +{
> +	struct lkl_dir *dir = NULL;
> +	struct lkl_linux_dirent64 *dirent;
> +	int ret;
> +
> +	dir = lkl_opendir(path, &ret);
> +	if (!dir)
> +		return ret;
> +
> +	ret = -LKL_ENOENT;
> +
> +	while ((dirent = lkl_readdir(dir))) {
> +		if (startswith(dirent->d_name, prefix)) {
> +			if (strlen(dirent->d_name) + 1 > result_len) {
> +				ret = -LKL_ENOMEM;
> +				break;
> +			}
> +			memcpy(result, dirent->d_name, strlen(dirent->d_name));
> +			result[strlen(dirent->d_name)] = '\0';
> +			ret = 0;
> +			break;
> +		}
> +	}
> +
> +	lkl_closedir(dir);
> +
> +	return ret;
> +}
> +
> +int lkl_encode_dev_from_sysfs(const char *sysfs_path, uint32_t *pdevid)
> +{
> +	int ret;
> +	long fd;
> +	int major, minor;
> +	char buf[16] = { 0, };
> +	char *bufptr;
> +
> +	fd = lkl_sys_open(sysfs_path, LKL_O_RDONLY, 0);
> +	if (fd < 0)
> +		return fd;
> +
> +	ret = lkl_sys_read(fd, buf, sizeof(buf));
> +	if (ret < 0)
> +		goto out_close;
> +
> +	if (ret == sizeof(buf)) {
> +		ret = -LKL_ENOBUFS;
> +		goto out_close;
> +	}
> +
> +	bufptr = strchr(buf, ':');
> +	if (bufptr == NULL) {
> +		ret = -LKL_EINVAL;
> +		goto out_close;
> +	}
> +	bufptr[0] = '\0';
> +	bufptr++;
> +
> +	major = atoi(buf);
> +	minor = atoi(bufptr);
> +
> +	*pdevid = new_encode_dev(major, minor);
> +	ret = 0;
> +
> +out_close:
> +	lkl_sys_close(fd);
> +
> +	return ret;
> +}
> +
> +#define SYSFS_DEV_UMBLK_CMDLINE_PATH \
> +	"/sysfs/devices/platform/uml-blkdev.%d"
> +
> +struct abuf {
> +	char *mem, *ptr;
> +	unsigned int len;
> +};
> +
> +static int snprintf_append(struct abuf *buf, const char *fmt, ...)
> +{
> +	int ret;
> +	va_list args;
> +
> +	if (!buf->ptr)
> +		buf->ptr = buf->mem;
> +
> +	va_start(args, fmt);
> +	ret = vsnprintf(buf->ptr, buf->len - (buf->ptr - buf->mem), fmt, args);
> +	va_end(args);
> +
> +	if (ret < 0 || (ret >= (int)(buf->len - (buf->ptr - buf->mem))))
> +		return -LKL_ENOMEM;
> +
> +	buf->ptr += ret;
> +
> +	return 0;
> +}
> +
> +static int __lkl_get_blkdev(int disk_id, unsigned int part, uint32_t *pdevid,
> +			    const char *sysfs_path_fmt, const char *drv_prefix,
> +			    const char *disk_prefix)
> +{
> +	char sysfs_path[LKL_PATH_MAX];
> +	char drv_name[LKL_PATH_MAX];
> +	char disk_name[LKL_PATH_MAX];
> +	struct abuf sysfs_path_buf = {
> +		.mem = sysfs_path,
> +		.len = sizeof(sysfs_path),
> +	};
> +	int ret;
> +
> +	if (disk_id < 0)
> +		return -LKL_EINVAL;
> +
> +	ret = lkl_mount_fs("sysfs");
> +	if (ret < 0)
> +		return ret;
> +
> +	ret = snprintf_append(&sysfs_path_buf, sysfs_path_fmt, disk_id);
> +	if (ret)
> +		return ret;
> +
> +	ret = get_node_with_prefix(sysfs_path, drv_prefix, drv_name,
> +				   sizeof(drv_name));
> +	if (ret)
> +		return ret;
> +
> +	ret = snprintf_append(&sysfs_path_buf, "/%s/block", drv_name);
> +	if (ret)
> +		return ret;
> +
> +	ret = get_node_with_prefix(sysfs_path, disk_prefix, disk_name,
> +				   sizeof(disk_name));
> +	if (ret)
> +		return ret;
> +
> +	if (!part)
> +		ret = snprintf_append(&sysfs_path_buf, "/%s/dev", disk_name);
> +	else
> +		ret = snprintf_append(&sysfs_path_buf, "/%s/%s%d/dev",
> +				      disk_name, disk_name, part);
> +	if (ret)
> +		return ret;
> +
> +	return lkl_encode_dev_from_sysfs(sysfs_path, pdevid);
> +}
> +
> +int lkl_get_blkdev(int disk_id, unsigned int part, uint32_t *pdevid)
> +{
> +	char *fmt;
> +
> +	fmt = SYSFS_DEV_UMBLK_CMDLINE_PATH;
> +	return __lkl_get_blkdev(disk_id, part, pdevid, fmt, "", "ubd");
> +}
> +
> +long lkl_mount_dev(unsigned int disk_id, unsigned int part,
> +		   const char *fs_type, int flags,
> +		   const char *data, char *mnt_str, unsigned int mnt_str_len)
> +{
> +	char dev_str[] = { "/dev/xxxxxxxx" };
> +	unsigned int dev;
> +	int err;
> +	char _data[4096]; /* FIXME: PAGE_SIZE is not exported by LKL */
> +
> +	if (mnt_str_len < sizeof(dev_str))
> +		return -LKL_ENOMEM;
> +
> +	err = lkl_get_blkdev(disk_id, part, &dev);
> +	if (err < 0)
> +		return err;
> +
> +	snprintf(dev_str, sizeof(dev_str), "/dev/%08x", dev);
> +	snprintf(mnt_str, mnt_str_len, "/mnt/%08x", dev);
> +
> +	err = lkl_sys_access("/dev", LKL_S_IRWXO);
> +	if (err < 0) {
> +		if (err == -LKL_ENOENT)
> +			err = lkl_sys_mkdir("/dev", 0700);
> +		if (err < 0)
> +			return err;
> +	}
> +
> +	err = lkl_sys_mknod(dev_str, LKL_S_IFBLK | 0600, dev);
> +	if (err < 0)
> +		return err;
> +
> +	err = lkl_sys_access("/mnt", LKL_S_IRWXO);
> +	if (err < 0) {
> +		if (err == -LKL_ENOENT)
> +			err = lkl_sys_mkdir("/mnt", 0700);
> +		if (err < 0)
> +			return err;
> +	}
> +
> +	err = lkl_sys_mkdir(mnt_str, 0700);
> +	if (err < 0) {
> +		lkl_sys_unlink(dev_str);
> +		return err;
> +	}
> +
> +	/* kernel always copies a full page */
> +	if (data) {
> +		strncpy(_data, data, sizeof(_data));
> +		_data[sizeof(_data) - 1] = 0;
> +	} else {
> +		_data[0] = 0;
> +	}
> +
> +	err = lkl_sys_mount(dev_str, mnt_str, (char *)fs_type, flags, _data);
> +	if (err < 0) {
> +		lkl_sys_unlink(dev_str);
> +		lkl_sys_rmdir(mnt_str);
> +		return err;
> +	}
> +
> +	return 0;
> +}
> +
> +long lkl_umount_timeout(char *path, int flags, long timeout_ms)
> +{
> +	long incr = 10000000; /* 10 ms */
> +	struct lkl_timespec ts = {
> +		.tv_sec = 0,
> +		.tv_nsec = incr,
> +	};
> +	long err;
> +
> +	do {
> +		err = lkl_sys_umount(path, flags);
> +		if (err == -LKL_EBUSY) {
> +			lkl_sys_nanosleep((struct __lkl__kernel_timespec *)&ts,
> +					  NULL);
> +			timeout_ms -= incr / 1000000;
> +		}
> +	} while (err == -LKL_EBUSY && timeout_ms > 0);
> +
> +	return err;
> +}
> +
> +long lkl_umount_dev(unsigned int disk_id, unsigned int part, int flags,
> +		    long timeout_ms)
> +{
> +	char dev_str[] = { "/dev/xxxxxxxx" };
> +	char mnt_str[] = { "/mnt/xxxxxxxx" };
> +	unsigned int dev;
> +	int err;
> +
> +	err = lkl_get_blkdev(disk_id, part, &dev);
> +	if (err < 0)
> +		return err;
> +
> +	snprintf(dev_str, sizeof(dev_str), "/dev/%08x", dev);
> +	snprintf(mnt_str, sizeof(mnt_str), "/mnt/%08x", dev);
> +
> +	err = lkl_umount_timeout(mnt_str, flags, timeout_ms);
> +	if (err)
> +		return err;
> +
> +	err = lkl_sys_unlink(dev_str);
> +	if (err)
> +		return err;
> +
> +	return lkl_sys_rmdir(mnt_str);
> +}
> +
> +struct lkl_dir {
> +	int fd;
> +	char buf[1024];
> +	char *pos;
> +	int len;
> +};
> +
> +static struct lkl_dir *lkl_dir_alloc(int *err)
> +{
> +	struct lkl_dir *dir = lkl_host_ops.mem_alloc(sizeof(struct lkl_dir));
> +
> +	if (!dir) {
> +		*err = -LKL_ENOMEM;
> +		return NULL;
> +	}
> +
> +	dir->len = 0;
> +	dir->pos = NULL;
> +
> +	return dir;
> +}
> +
> +struct lkl_dir *lkl_opendir(const char *path, int *err)
> +{
> +	struct lkl_dir *dir = lkl_dir_alloc(err);
> +
> +	if (!dir) {
> +		*err = -LKL_ENOMEM;
> +		return NULL;
> +	}
> +
> +	dir->fd = lkl_sys_open(path, LKL_O_RDONLY | LKL_O_DIRECTORY, 0);
> +	if (dir->fd < 0) {
> +		*err = dir->fd;
> +		lkl_host_ops.mem_free(dir);
> +		return NULL;
> +	}
> +
> +	*err = 0;
> +
> +	return dir;
> +}
> +
> +struct lkl_dir *lkl_fdopendir(int fd, int *err)
> +{
> +	struct lkl_dir *dir = lkl_dir_alloc(err);
> +
> +	if (!dir)
> +		return NULL;
> +
> +	dir->fd = fd;
> +
> +	return dir;
> +}
> +
> +void lkl_rewinddir(struct lkl_dir *dir)
> +{
> +	lkl_sys_lseek(dir->fd, 0, LKL_SEEK_SET);
> +	dir->len = 0;
> +	dir->pos = NULL;
> +}
> +
> +int lkl_closedir(struct lkl_dir *dir)
> +{
> +	int ret;
> +
> +	ret = lkl_sys_close(dir->fd);
> +	lkl_host_ops.mem_free(dir);
> +
> +	return ret;
> +}
> +
> +struct lkl_linux_dirent64 *lkl_readdir(struct lkl_dir *dir)
> +{
> +	struct lkl_linux_dirent64 *de;
> +
> +	if (dir->len < 0)
> +		return NULL;
> +
> +	if (!dir->pos || dir->pos - dir->buf >= dir->len)
> +		goto read_buf;
> +
> +return_de:
> +	de = (struct lkl_linux_dirent64 *)dir->pos;
> +	dir->pos += de->d_reclen;
> +
> +	return de;
> +
> +read_buf:
> +	dir->pos = NULL;
> +	de = (struct lkl_linux_dirent64 *)dir->buf;
> +	dir->len = lkl_sys_getdents64(dir->fd, de, sizeof(dir->buf));
> +	if (dir->len <= 0)
> +		return NULL;
> +
> +	dir->pos = dir->buf;
> +	goto return_de;
> +}
> +
> +int lkl_errdir(struct lkl_dir *dir)
> +{
> +	if (dir->len >= 0)
> +		return 0;
> +
> +	return dir->len;
> +}
> +
> +int lkl_dirfd(struct lkl_dir *dir)
> +{
> +	return dir->fd;
> +}
> +
> +int lkl_set_fd_limit(unsigned int fd_limit)
> +{
> +	struct lkl_rlimit rlim = {
> +		.rlim_cur = fd_limit,
> +		.rlim_max = fd_limit,
> +	};
> +	return lkl_sys_setrlimit(LKL_RLIMIT_NOFILE, &rlim);
> +}
> diff --git a/tools/um/lib/posix-host.c b/tools/um/lib/posix-host.c
> index b6b5b2902254..8fd88031bf2b 100644
> --- a/tools/um/lib/posix-host.c
> +++ b/tools/um/lib/posix-host.c
> @@ -263,6 +263,7 @@ static void *tls_get(struct lkl_tls_key *key)
>   }
>   
>   struct lkl_host_operations lkl_host_ops = {
> +	.um_devices = lkl_um_devs,
>   	.panic = panic,
>   	.print = print,
>   	.mem_alloc = (void *)malloc,
> diff --git a/tools/um/lib/utils.c b/tools/um/lib/utils.c
> index 4930479a8a35..ac65cd744a14 100644
> --- a/tools/um/lib/utils.c
> +++ b/tools/um/lib/utils.c
> @@ -4,6 +4,9 @@
>   #include <string.h>
>   #include <lkl_host.h>
>   
> +/* XXX: find a better place */
> +char lkl_um_devs[4096];
> +
>   static const char * const lkl_err_strings[] = {
>   	"Success",
>   	"Operation not permitted",
> diff --git a/tools/um/tests/Build b/tools/um/tests/Build
> index 564560486f98..1aa78f9ed7ef 100644
> --- a/tools/um/tests/Build
> +++ b/tools/um/tests/Build
> @@ -1,3 +1,4 @@
>   boot-y += boot.o test.o
> +disk-y += disk.o test.o cla.o
>   
>   CFLAGS_test.o += -Wno-implicit-fallthrough
> diff --git a/tools/um/tests/cla.c b/tools/um/tests/cla.c
> new file mode 100644
> index 000000000000..694e3a3822be
> --- /dev/null
> +++ b/tools/um/tests/cla.c
> @@ -0,0 +1,159 @@
> +// SPDX-License-Identifier: GPL-2.0
> +#include <stdio.h>
> +#include <string.h>
> +#include <errno.h>
> +#include <stdlib.h>
> +#ifdef __MINGW32__
> +#include <winsock2.h>
> +#else
> +#include <sys/socket.h>
> +#include <netinet/in.h>
> +#include <arpa/inet.h>
> +#endif
> +
> +#include "cla.h"
> +
> +static int cl_arg_parse_bool(struct cl_arg *arg, const char *value)
> +{
> +	*((int *)arg->store) = 1;
> +	return 0;
> +}
> +
> +static int cl_arg_parse_str(struct cl_arg *arg, const char *value)
> +{
> +	*((const char **)arg->store) = value;
> +	return 0;
> +}
> +
> +static int cl_arg_parse_int(struct cl_arg *arg, const char *value)
> +{
> +	errno = 0;
> +	*((int *)arg->store) = strtol(value, NULL, 0);
> +	return errno == 0;
> +}
> +
> +static int cl_arg_parse_str_set(struct cl_arg *arg, const char *value)
> +{
> +	const char **set = arg->set;
> +	int i;
> +
> +	for (i = 0; set[i] != NULL; i++) {
> +		if (strcmp(set[i], value) == 0) {
> +			*((int *)arg->store) = i;
> +			return 0;
> +		}
> +	}
> +
> +	return (-1);
> +}
> +
> +static int cl_arg_parse_ipv4(struct cl_arg *arg, const char *value)
> +{
> +	unsigned int addr;
> +
> +	if (!value)
> +		return (-1);
> +
> +	addr = inet_addr(value);
> +	if (addr == INADDR_NONE)
> +		return (-1);
> +	*((unsigned int *)arg->store) = addr;
> +	return 0;
> +}
> +
> +static cl_arg_parser_t parsers[] = {
> +	[CL_ARG_BOOL] = cl_arg_parse_bool,
> +	[CL_ARG_INT] = cl_arg_parse_int,
> +	[CL_ARG_STR] = cl_arg_parse_str,
> +	[CL_ARG_STR_SET] = cl_arg_parse_str_set,
> +	[CL_ARG_IPV4] = cl_arg_parse_ipv4,
> +};
> +
> +static struct cl_arg *find_short_arg(char name, struct cl_arg *args)
> +{
> +	struct cl_arg *arg;
> +
> +	for (arg = args; arg->short_name != 0; arg++) {
> +		if (arg->short_name == name)
> +			return arg;
> +	}
> +
> +	return NULL;
> +}
> +
> +static struct cl_arg *find_long_arg(const char *name, struct cl_arg *args)
> +{
> +	struct cl_arg *arg;
> +
> +	for (arg = args; arg->long_name; arg++) {
> +		if (strcmp(arg->long_name, name) == 0)
> +			return arg;
> +	}
> +
> +	return NULL;
> +}
> +
> +static void print_help(struct cl_arg *args)
> +{
> +	struct cl_arg *arg;
> +
> +	fprintf(stderr, "usage:\n");
> +	for (arg = args; arg->long_name; arg++) {
> +		fprintf(stderr, "-%c, --%-20s %s", arg->short_name,
> +			arg->long_name, arg->help);
> +		if (arg->type == CL_ARG_STR_SET) {
> +			const char **set = arg->set;
> +
> +			fprintf(stderr, " [ ");
> +			while (*set != NULL)
> +				fprintf(stderr, "%s ", *(set++));
> +			fprintf(stderr, "]");
> +		}
> +		fprintf(stderr, "\n");
> +	}
> +}
> +
> +int cla_parse_args(int argc, const char **argv, struct cl_arg *args)
> +{
> +	int i;
> +
> +	for (i = 1; i < argc; i++) {
> +		struct cl_arg *arg = NULL;
> +		cl_arg_parser_t parser;
> +
> +		if (argv[i][0] == '-') {
> +			if (argv[i][1] != '-')
> +				arg = find_short_arg(argv[i][1], args);
> +			else
> +				arg = find_long_arg(&argv[i][2], args);
> +		}
> +
> +		if (!arg) {
> +			fprintf(stderr, "unknown option '%s'\n", argv[i]);
> +			print_help(args);
> +			return (-1);
> +		}
> +
> +		if (arg->type == CL_ARG_USER || arg->type >= CL_ARG_END)
> +			parser = arg->parser;
> +		else
> +			parser = parsers[arg->type];
> +
> +		if (!parser) {
> +			fprintf(stderr, "can't parse --'%s'/-'%c'\n",
> +				arg->long_name, args->short_name);
> +			return (-1);
> +		}
> +
> +		if (parser(arg, argv[i + 1]) < 0) {
> +			fprintf(stderr, "can't parse '%s'\n", argv[i]);
> +			print_help(args);
> +			return (-1);
> +		}
> +
> +		if (arg->has_arg)
> +			i++;
> +	}
> +
> +	return 0;
> +}
> diff --git a/tools/um/tests/cla.h b/tools/um/tests/cla.h
> new file mode 100644
> index 000000000000..3d879233681f
> --- /dev/null
> +++ b/tools/um/tests/cla.h
> @@ -0,0 +1,33 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +#ifndef _LKL_TEST_CLA_H
> +#define _LKL_TEST_CLA_H
> +
> +enum cl_arg_type {
> +	CL_ARG_USER = 0,
> +	CL_ARG_BOOL,
> +	CL_ARG_INT,
> +	CL_ARG_STR,
> +	CL_ARG_STR_SET,
> +	CL_ARG_IPV4,
> +	CL_ARG_END,
> +};
> +
> +struct cl_arg;
> +
> +typedef int (*cl_arg_parser_t)(struct cl_arg *arg, const char *value);
> +
> +struct cl_arg {
> +	const char *long_name;
> +	char short_name;
> +	const char *help;
> +	int has_arg;
> +	enum cl_arg_type type;
> +	void *store;
> +	void *set;
> +	cl_arg_parser_t parser;
> +};
> +
> +int cla_parse_args(int argc, const char **argv, struct cl_arg *args);
> +
> +
> +#endif /* _LKL_TEST_CLA_H */
> diff --git a/tools/um/tests/disk.c b/tools/um/tests/disk.c
> new file mode 100644
> index 000000000000..325934935e6a
> --- /dev/null
> +++ b/tools/um/tests/disk.c
> @@ -0,0 +1,168 @@
> +// SPDX-License-Identifier: GPL-2.0
> +#include <stdio.h>
> +#include <unistd.h>
> +#include <string.h>
> +#include <time.h>
> +#include <stdlib.h>
> +#include <stdint.h>
> +#include <lkl.h>
> +#include <lkl_host.h>
> +#include <sys/stat.h>
> +#include <fcntl.h>
> +#include <sys/ioctl.h>
> +
> +#include "test.h"
> +#include "cla.h"
> +
> +static struct {
> +	int printk;
> +	const char *disk;
> +	const char *fstype;
> +	int partition;
> +} cla;
> +
> +struct cl_arg args[] = {
> +	{"disk", 'd', "disk file to use", 1, CL_ARG_STR, &cla.disk},
> +	{"partition", 'P', "partition to mount", 1, CL_ARG_INT, &cla.partition},
> +	{"type", 't', "filesystem type", 1, CL_ARG_STR, &cla.fstype},
> +	{0},
> +};
> +
> +
> +static struct lkl_disk disk;
> +static int disk_id = -1;
> +
> +int lkl_test_disk_add(void)
> +{
> +	disk.fd = open(cla.disk, O_RDWR);
> +	if (disk.fd < 0)
> +		goto out_unlink;
> +
> +	disk.ops = NULL;
> +	disk.dev = (char *)cla.disk;
> +
> +	disk_id = lkl_disk_add(&disk);
> +	if (disk_id < 0)
> +		goto out_close;
> +
> +	goto out;
> +
> +out_close:
> +	close(disk.fd);
> +
> +out_unlink:
> +	unlink(cla.disk);
> +
> +out:
> +	lkl_test_logf("disk fd/handle %x disk_id %d", disk.fd, disk_id);
> +
> +	if (disk_id >= 0)
> +		return TEST_SUCCESS;
> +
> +	return TEST_FAILURE;
> +}
> +
> +int lkl_test_disk_remove(void)
> +{
> +	int ret;
> +
> +	ret = lkl_disk_remove(disk);
> +
> +	close(disk.fd);
> +
> +	if (ret == 0)
> +		return TEST_SUCCESS;
> +
> +	return TEST_FAILURE;
> +}
> +
> +
> +static char mnt_point[32];
> +
> +LKL_TEST_CALL(mount_dev, lkl_mount_dev, 0, disk_id, cla.partition, cla.fstype,
> +	      0, NULL, mnt_point, sizeof(mnt_point))
> +
> +static int lkl_test_umount_dev(void)
> +{
> +	long ret, ret2;
> +
> +	ret = lkl_sys_chdir("/");
> +
> +	ret2 = lkl_umount_dev(disk_id, cla.partition, 0, 1000);
> +
> +	lkl_test_logf("%ld %ld", ret, ret2);
> +
> +	if (!ret && !ret2)
> +		return TEST_SUCCESS;
> +
> +	return TEST_FAILURE;
> +}
> +
> +struct lkl_dir *dir;
> +
> +static int lkl_test_opendir(void)
> +{
> +	int err;
> +
> +	dir = lkl_opendir(mnt_point, &err);
> +
> +	lkl_test_logf("lkl_opedir(%s) = %d %s\n", mnt_point, err,
> +		      lkl_strerror(err));
> +
> +	if (err == 0)
> +		return TEST_SUCCESS;
> +
> +	return TEST_FAILURE;
> +}
> +
> +static int lkl_test_readdir(void)
> +{
> +	struct lkl_linux_dirent64 *de = lkl_readdir(dir);
> +	int wr = 0;
> +
> +	while (de) {
> +		wr += lkl_test_logf("%s ", de->d_name);
> +		if (wr >= 70) {
> +			lkl_test_logf("\n");
> +			wr = 0;
> +			break;
> +		}
> +		de = lkl_readdir(dir);
> +	}
> +
> +	if (lkl_errdir(dir) == 0)
> +		return TEST_SUCCESS;
> +
> +	return TEST_FAILURE;
> +}
> +
> +LKL_TEST_CALL(closedir, lkl_closedir, 0, dir);
> +LKL_TEST_CALL(chdir_mnt_point, lkl_sys_chdir, 0, mnt_point);
> +LKL_TEST_CALL(start_kernel, lkl_start_kernel, 0, &lkl_host_ops,
> +	     "mem=16M loglevel=8");
> +LKL_TEST_CALL(stop_kernel, lkl_sys_halt, 0);
> +
> +struct lkl_test tests[] = {
> +	LKL_TEST(disk_add),
> +	LKL_TEST(start_kernel),
> +	LKL_TEST(mount_dev),
> +	LKL_TEST(chdir_mnt_point),
> +	LKL_TEST(opendir),
> +	LKL_TEST(readdir),
> +	LKL_TEST(closedir),
> +	LKL_TEST(umount_dev),
> +	LKL_TEST(stop_kernel),
> +	LKL_TEST(disk_remove),
> +
> +};
> +
> +int main(int argc, const char **argv)
> +{
> +	if (cla_parse_args(argc, argv, args) < 0)
> +		return (-1);
> +
> +	lkl_host_ops.print = lkl_test_log;
> +
> +	return lkl_test_run(tests, sizeof(tests)/sizeof(struct lkl_test),
> +			    "disk %s", cla.fstype);
> +}
> diff --git a/tools/um/tests/disk.sh b/tools/um/tests/disk.sh
> new file mode 100755
> index 000000000000..e2ec6cf69d4b
> --- /dev/null
> +++ b/tools/um/tests/disk.sh
> @@ -0,0 +1,70 @@
> +#!/usr/bin/env bash
> +# SPDX-License-Identifier: GPL-2.0
> +
> +script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
> +
> +source $script_dir/test.sh
> +
> +function prepfs()
> +{
> +    set -e
> +
> +    file=`mktemp`
> +
> +    dd if=/dev/zero of=$file bs=1024 count=204800
> +
> +    yes | mkfs.$1 $file
> +
> +    if ! [ -z $ANDROID_WDIR ]; then
> +        adb shell mkdir -p $ANDROID_WDIR
> +        adb push $file $ANDROID_WDIR
> +        rm $file
> +        file=$ANDROID_WDIR/$(basename $file)
> +    fi
> +    if ! [ -z $BSD_WDIR ]; then
> +        $MYSSH mkdir -p $BSD_WDIR
> +        ssh_copy $file $BSD_WDIR
> +        rm $file
> +        file=$BSD_WDIR/$(basename $file)
> +    fi
> +
> +    export_vars file
> +}
> +
> +function cleanfs()
> +{
> +    set -e
> +
> +    if ! [ -z $ANDROID_WDIR ]; then
> +        adb shell rm $1
> +        adb shell rm $ANDROID_WDIR/disk
> +    elif ! [ -z $BSD_WDIR ]; then
> +        $MYSSH rm $1
> +        $MYSSH rm $BSD_WDIR/disk
> +    else
> +        rm $1
> +    fi
> +}
> +
> +if [ "$1" = "-t" ]; then
> +    shift
> +    fstype=$1
> +    shift
> +fi
> +
> +if [ -z "$fstype" ]; then
> +    fstype="ext4"
> +fi
> +
> +if [ -z $(which mkfs.$fstype) ]; then
> +    lkl_test_plan 0 "disk $fstype"
> +    echo "no mkfs.$fstype command"
> +    exit 0
> +fi
> +
> +lkl_test_plan 1 "disk $fstype"
> +lkl_test_run 1 prepfs $fstype
> +lkl_test_exec $script_dir/disk -d $file -t $fstype $@
> +lkl_test_plan 1 "disk $fstype"
> +lkl_test_run 1 cleanfs $file
> +
> diff --git a/tools/um/tests/run.py b/tools/um/tests/run.py
> index c96ede90b6ad..97d6dedc217c 100755
> --- a/tools/um/tests/run.py
> +++ b/tools/um/tests/run.py
> @@ -50,6 +50,8 @@ mydir=os.path.dirname(os.path.realpath(__file__))
>   
>   tests = [
>       'boot.sh',
> +    'disk.sh -t ext4',
> +    'disk.sh -t vfat',
>   ]
>   
>   parser = argparse.ArgumentParser(description='LKL test runner')

-- 
Anton R. Ivanov
Cambridgegreys Limited. Registered in England. Company Number 10273661
https://www.cambridgegreys.com/


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 21/21] um: nommu: add block device support of UML
@ 2020-10-07 14:17             ` Anton Ivanov
  0 siblings, 0 replies; 476+ messages in thread
From: Anton Ivanov @ 2020-10-07 14:17 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard
  Cc: tavi.purdila, linux-kernel-library, retrage01, linux-arch


On 06/10/2020 10:44, Hajime Tazaki wrote:
> This commit adds block device support for nommu mode, and also added
> several host utilities to run a simple test program
> (tools/um/test/disk.c) successfully.
>
> Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
> ---
>   arch/um/include/asm/xor.h                 |   3 +-
>   arch/um/include/shared/as-layout.h        |   1 +
>   arch/um/kernel/um_arch.c                  |   5 +
>   arch/um/nommu/include/uapi/asm/host_ops.h |   7 +
>   arch/um/nommu/include/uapi/asm/unistd.h   |   2 +
>   arch/um/nommu/um/Kconfig                  |   8 +
>   arch/um/nommu/um/setup.c                  |  12 +
>   tools/um/Targets                          |   2 +
>   tools/um/include/lkl.h                    | 206 ++++++++++
>   tools/um/include/lkl_host.h               |   1 +
>   tools/um/lib/Build                        |   1 +
>   tools/um/lib/fs.c                         | 461 ++++++++++++++++++++++
>   tools/um/lib/posix-host.c                 |   1 +
>   tools/um/lib/utils.c                      |   3 +
>   tools/um/tests/Build                      |   1 +
>   tools/um/tests/cla.c                      | 159 ++++++++
>   tools/um/tests/cla.h                      |  33 ++
>   tools/um/tests/disk.c                     | 168 ++++++++
>   tools/um/tests/disk.sh                    |  70 ++++
>   tools/um/tests/run.py                     |   2 +
>   20 files changed, 1145 insertions(+), 1 deletion(-)
>   create mode 100644 tools/um/lib/fs.c
>   create mode 100644 tools/um/tests/cla.c
>   create mode 100644 tools/um/tests/cla.h
>   create mode 100644 tools/um/tests/disk.c
>   create mode 100755 tools/um/tests/disk.sh
>
> diff --git a/arch/um/include/asm/xor.h b/arch/um/include/asm/xor.h
> index 36b33d62a35d..a5ea458a1ae9 100644
> --- a/arch/um/include/asm/xor.h
> +++ b/arch/um/include/asm/xor.h
> @@ -4,4 +4,5 @@
>   
>   /* pick an arbitrary one - measuring isn't possible with inf-cpu */
>   #define XOR_SELECT_TEMPLATE(x)	\
> -	(time_travel_mode == TT_MODE_INFCPU ? &xor_block_8regs : NULL)
> +	(time_travel_mode == TT_MODE_INFCPU || (!IS_ENABLED(CONFIG_MMU)) ? \
> +	 &xor_block_8regs : NULL)
> diff --git a/arch/um/include/shared/as-layout.h b/arch/um/include/shared/as-layout.h
> index 5f286ef2721b..4423437a5ace 100644
> --- a/arch/um/include/shared/as-layout.h
> +++ b/arch/um/include/shared/as-layout.h
> @@ -57,6 +57,7 @@ extern unsigned long host_task_size;
>   
>   extern int linux_main(int argc, char **argv);
>   extern void uml_finishsetup(void);
> +extern void uml_set_args(char *args);
>   
>   struct siginfo;
>   extern void (*sig_info[])(int, struct siginfo *si, struct uml_pt_regs *);
> diff --git a/arch/um/kernel/um_arch.c b/arch/um/kernel/um_arch.c
> index e2cb76c03b25..da6e06cf2808 100644
> --- a/arch/um/kernel/um_arch.c
> +++ b/arch/um/kernel/um_arch.c
> @@ -41,6 +41,11 @@ static void __init add_arg(char *arg)
>   	strcat(command_line, arg);
>   }
>   
> +void __init uml_set_args(char *args)
> +{
> +	strcat(command_line, args);
> +}
> +
>   /*
>    * These fields are initialized at boot time and not changed.
>    * XXX This structure is used only in the non-SMP case.  Maybe this
> diff --git a/arch/um/nommu/include/uapi/asm/host_ops.h b/arch/um/nommu/include/uapi/asm/host_ops.h
> index 0811494c080c..1b2507502969 100644
> --- a/arch/um/nommu/include/uapi/asm/host_ops.h
> +++ b/arch/um/nommu/include/uapi/asm/host_ops.h
> @@ -15,6 +15,11 @@ struct lkl_jmp_buf {
>    *
>    * These operations must be provided by a host library or by the application
>    * itself.
> + *
> + * @um_devices - string containg the list of UML devices in command line
> + * format. This string is appended to the kernel command line and
> + * is provided here for convenience to be implemented by the host library.
> + *
>    * @print - optional operation that receives console messages
>    * @panic - called during a kernel panic
>    *
> @@ -65,6 +70,8 @@ struct lkl_jmp_buf {
>    *
>    */
>   struct lkl_host_operations {
> +	const char *um_devices;
> +
>   	void (*print)(const char *str, int len);
>   	void (*panic)(void);
>   
> diff --git a/arch/um/nommu/include/uapi/asm/unistd.h b/arch/um/nommu/include/uapi/asm/unistd.h
> index 1762ebb829f5..320762099a62 100644
> --- a/arch/um/nommu/include/uapi/asm/unistd.h
> +++ b/arch/um/nommu/include/uapi/asm/unistd.h
> @@ -3,6 +3,8 @@
>   #define __UM_NOMMU_UAPI_UNISTD_H
>   
>   #define __ARCH_WANT_NEW_STAT
> +#define __ARCH_WANT_SET_GET_RLIMIT
> +
>   #include <asm/bitsperlong.h>
>   
>   #if __BITS_PER_LONG == 64
> diff --git a/arch/um/nommu/um/Kconfig b/arch/um/nommu/um/Kconfig
> index 20b3eaccb6f0..c6a3f472fe75 100644
> --- a/arch/um/nommu/um/Kconfig
> +++ b/arch/um/nommu/um/Kconfig
> @@ -4,6 +4,10 @@ config UML_NOMMU
>   	select UACCESS_MEMCPY
>   	select ARCH_THREAD_STACK_ALLOCATOR
>   	select ARCH_HAS_SYSCALL_WRAPPER
> +	select VFAT_FS
> +	select NLS_CODEPAGE_437
> +	select NLS_ISO8859_1
> +	select BTRFS_FS
>   
>   config 64BIT
>   	bool
> @@ -35,3 +39,7 @@ config STACKTRACE_SUPPORT
>   config PRINTK_TIME
>   	bool
>   	default y
> +
> +config RAID6_PQ_BENCHMARK
> +	bool
> +	default n

Why are we touching this? I thought this is already defined in lib/Kconfig?

Brgds,

A.

> diff --git a/arch/um/nommu/um/setup.c b/arch/um/nommu/um/setup.c
> index 922188690139..d71d61979ad6 100644
> --- a/arch/um/nommu/um/setup.c
> +++ b/arch/um/nommu/um/setup.c
> @@ -45,13 +45,25 @@ static void __init *lkl_run_kernel(void *arg)
>   	return NULL;
>   }
>   
> +static char _cmd_line[COMMAND_LINE_SIZE];
>   int __init lkl_start_kernel(struct lkl_host_operations *ops,
>   			    const char *fmt, ...)
>   {
> +	va_list ap;
>   	int ret;
>   
>   	lkl_ops = ops;
>   
> +	va_start(ap, fmt);
> +	ret = vsnprintf(_cmd_line, COMMAND_LINE_SIZE, fmt, ap);
> +	va_end(ap);
> +
> +	if (ops->um_devices)
> +		strscpy(_cmd_line + ret, ops->um_devices,
> +			COMMAND_LINE_SIZE - ret);
> +
> +	uml_set_args(_cmd_line);
> +
>   	init_sem = lkl_ops->sem_alloc(0);
>   	if (!init_sem)
>   		return -ENOMEM;
> diff --git a/tools/um/Targets b/tools/um/Targets
> index 2bb90381843c..f5f8ec4b9dbb 100644
> --- a/tools/um/Targets
> +++ b/tools/um/Targets
> @@ -1,10 +1,12 @@
>   ifeq ($(UMMODE),library)
>   progs-y += tests/boot
> +progs-y += tests/disk
>   else
>   progs-y += uml/linux
>   endif
>   
>   LDFLAGS_boot-y := -pie
> +LDFLAGS_disk-y := -pie
>   
>   LDLIBS := -lrt -lpthread -lutil
>   LDFLAGS_linux-y := -no-pie -Wl,--wrap,malloc -Wl,--wrap,free -Wl,--wrap,calloc
> diff --git a/tools/um/include/lkl.h b/tools/um/include/lkl.h
> index 707e01b64a70..4a9d874bfbf0 100644
> --- a/tools/um/include/lkl.h
> +++ b/tools/um/include/lkl.h
> @@ -113,6 +113,16 @@ static inline long lkl_sys_creat(const char *file, int mode)
>   }
>   #endif
>   
> +#ifdef __lkl__NR_faccessat
> +/**
> + * lkl_sys_access - wrapper for lkl_sys_faccessat
> + */
> +static inline long lkl_sys_access(const char *file, int mode)
> +{
> +	return lkl_sys_faccessat(LKL_AT_FDCWD, file, mode);
> +}
> +#endif
> +
>   #ifdef __lkl__NR_mkdirat
>   /**
>    * lkl_sys_mkdir - wrapper for lkl_sys_mkdirat
> @@ -123,6 +133,36 @@ static inline long lkl_sys_mkdir(const char *path, mode_t mode)
>   }
>   #endif
>   
> +#ifdef __lkl__NR_unlinkat
> +/**
> + * lkl_sys_rmdir - wrapper for lkl_sys_unlinkrat
> + */
> +static inline long lkl_sys_rmdir(const char *path)
> +{
> +	return lkl_sys_unlinkat(LKL_AT_FDCWD, path, LKL_AT_REMOVEDIR);
> +}
> +#endif
> +
> +#ifdef __lkl__NR_unlinkat
> +/**
> + * lkl_sys_unlink - wrapper for lkl_sys_unlinkat
> + */
> +static inline long lkl_sys_unlink(const char *path)
> +{
> +	return lkl_sys_unlinkat(LKL_AT_FDCWD, path, 0);
> +}
> +#endif
> +
> +#ifdef __lkl__NR_mknodat
> +/**
> + * lkl_sys_mknod - wrapper for lkl_sys_mknodat
> + */
> +static inline long lkl_sys_mknod(const char *path, mode_t mode, dev_t dev)
> +{
> +	return lkl_sys_mknodat(LKL_AT_FDCWD, path, mode, dev);
> +}
> +#endif
> +
>   #ifdef __lkl__NR_epoll_create1
>   /**
>    * lkl_sys_epoll_create - wrapper for lkl_sys_epoll_create1
> @@ -144,6 +184,172 @@ static inline long lkl_sys_epoll_wait(int fd, struct lkl_epoll_event *ev,
>   }
>   #endif
>   
> +/**
> + * struct lkl_dev_blk_ops - block device host operations, defined in lkl_host.h.
> + */
> +struct lkl_dev_blk_ops;
> +
> +/**
> + * lkl_disk - host disk handle
> + *
> + * @dev - a pointer to private information for this disk backend
> + * @fd - a POSIX file descriptor that can be used by preadv/pwritev
> + * @handle - an NT file handle that can be used by ReadFile/WriteFile
> + */
> +struct lkl_disk {
> +	void *dev;
> +	union {
> +		int fd;
> +		void *handle;
> +	};
> +	struct lkl_dev_blk_ops *ops;
> +};
> +
> +/**
> + * lkl_disk_add - add a new disk
> + *
> + * @disk - the host disk handle
> + * @returns a disk id (0 is valid) or a strictly negative value in case of error
> + */
> +int lkl_disk_add(struct lkl_disk *disk);
> +
> +/**
> + * lkl_disk_remove - remove a disk
> + *
> + * This function makes a cleanup of the @disk's private information
> + * that was initialized by lkl_disk_add before.
> + *
> + * @disk - the host disk handle
> + */
> +int lkl_disk_remove(struct lkl_disk disk);
> +
> +/**
> + * lkl_encode_dev_from_sysfs_blkdev - extract device id from sysfs
> + *
> + * This function returns the device id for the given sysfs dev node.
> + * The content of the node has to be in the form 'MAJOR:MINOR'.
> + * Also, this function expects an absolute path which means that sysfs
> + * already has to be mounted at the given path
> + *
> + * @sysfs_path - absolute path to the sysfs dev node
> + * @pdevid - pointer to memory where dev id will be returned
> + * @returns - 0 on success, a negative value on error
> + */
> +int lkl_encode_dev_from_sysfs(const char *sysfs_path, uint32_t *pdevid);
> +
> +/**
> + * lkl_mount_dev - mount a disk
> + *
> + * This functions creates a device file for the given disk, creates a mount
> + * point and mounts the device over the mount point.
> + *
> + * @disk_id - the disk id identifying the disk to be mounted
> + * @part - disk partition or zero for full disk
> + * @fs_type - filesystem type
> + * @flags - mount flags
> + * @opts - additional filesystem specific mount options
> + * @mnt_str - a string that will be filled by this function with the path where
> + * the filesystem has been mounted
> + * @mnt_str_len - size of mnt_str
> + * @returns - 0 on success, a negative value on error
> + */
> +long lkl_mount_dev(unsigned int disk_id, unsigned int part, const char *fs_type,
> +		   int flags, const char *opts,
> +		   char *mnt_str, unsigned int mnt_str_len);
> +
> +/**
> + * lkl_umount_dev - umount a disk
> + *
> + * This functions umounts the given disks and removes the device file and the
> + * mount point.
> + *
> + * @disk_id - the disk id identifying the disk to be mounted
> + * @part - disk partition or zero for full disk
> + * @flags - umount flags
> + * @timeout_ms - timeout to wait for the kernel to flush closed files so that
> + * umount can succeed
> + * @returns - 0 on success, a negative value on error
> + */
> +long lkl_umount_dev(unsigned int disk_id, unsigned int part, int flags,
> +		    long timeout_ms);
> +
> +/**
> + * lkl_umount_timeout - umount filesystem with timeout
> + *
> + * @path - the path to unmount
> + * @flags - umount flags
> + * @timeout_ms - timeout to wait for the kernel to flush closed files so that
> + * umount can succeed
> + * @returns - 0 on success, a negative value on error
> + */
> +long lkl_umount_timeout(char *path, int flags, long timeout_ms);
> +
> +/**
> + * lkl_opendir - open a directory
> + *
> + * @path - directory path
> + * @err - pointer to store the error in case of failure
> + * @returns - a handle to be used when calling lkl_readdir
> + */
> +struct lkl_dir *lkl_opendir(const char *path, int *err);
> +
> +/**
> + * lkl_fdopendir - open a directory
> + *
> + * @fd - file descriptor
> + * @err - pointer to store the error in case of failure
> + * @returns - a handle to be used when calling lkl_readdir
> + */
> +struct lkl_dir *lkl_fdopendir(int fd, int *err);
> +
> +/**
> + * lkl_rewinddir - reset directory stream
> + *
> + * @dir - the directory handler as returned by lkl_opendir
> + */
> +void lkl_rewinddir(struct lkl_dir *dir);
> +
> +/**
> + * lkl_closedir - close the directory
> + *
> + * @dir - the directory handler as returned by lkl_opendir
> + */
> +int lkl_closedir(struct lkl_dir *dir);
> +
> +/**
> + * lkl_readdir - get the next available entry of the directory
> + *
> + * @dir - the directory handler as returned by lkl_opendir
> + * @returns - a lkl_dirent64 entry or NULL if the end of the directory stream is
> + * reached or if an error occurred; check lkl_errdir() to distinguish between
> + * errors or end of the directory stream
> + */
> +struct lkl_linux_dirent64 *lkl_readdir(struct lkl_dir *dir);
> +
> +/**
> + * lkl_errdir - checks if an error occurred during the last lkl_readdir call
> + *
> + * @dir - the directory handler as returned by lkl_opendir
> + * @returns - 0 if no error occurred, or a negative value otherwise
> + */
> +int lkl_errdir(struct lkl_dir *dir);
> +
> +/**
> + * lkl_dirfd - gets the file descriptor associated with the directory handle
> + *
> + * @dir - the directory handle as returned by lkl_opendir
> + * @returns - a positive value,which is the LKL file descriptor associated with
> + * the directory handle, or a negative value otherwise
> + */
> +int lkl_dirfd(struct lkl_dir *dir);
> +
> +/**
> + * lkl_mount_fs - mount a file system type like proc, sys
> + * @fstype - file system type. e.g. proc, sys
> + * @returns - 0 on success. 1 if it's already mounted. negative on failure.
> + */
> +int lkl_mount_fs(char *fstype);
> +
>   #ifdef __cplusplus
>   }
>   #endif
> diff --git a/tools/um/include/lkl_host.h b/tools/um/include/lkl_host.h
> index 85e80eb4ad0d..12dd95616e43 100644
> --- a/tools/um/include/lkl_host.h
> +++ b/tools/um/include/lkl_host.h
> @@ -10,6 +10,7 @@ extern "C" {
>   #include <lkl.h>
>   
>   extern struct lkl_host_operations lkl_host_ops;
> +extern char lkl_um_devs[4096];
>   
>   /**
>    * lkl_printf - print a message via the host print operation
> diff --git a/tools/um/lib/Build b/tools/um/lib/Build
> index dddff26a3b4e..29491b40746c 100644
> --- a/tools/um/lib/Build
> +++ b/tools/um/lib/Build
> @@ -4,3 +4,4 @@ CFLAGS_posix-host.o += -D_FILE_OFFSET_BITS=64
>   liblinux-$(CONFIG_UMMODE_LIB) += utils.o
>   liblinux-$(CONFIG_UMMODE_LIB) += posix-host.o
>   liblinux-$(CONFIG_UMMODE_LIB) += jmp_buf.o
> +liblinux-$(CONFIG_UMMODE_LIB) += fs.o
> diff --git a/tools/um/lib/fs.c b/tools/um/lib/fs.c
> new file mode 100644
> index 000000000000..948aac9730c2
> --- /dev/null
> +++ b/tools/um/lib/fs.c
> @@ -0,0 +1,461 @@
> +// SPDX-License-Identifier: GPL-2.0
> +#include <stdarg.h>
> +#include <stdio.h>
> +#include <string.h>
> +#include <stdlib.h>
> +#include <lkl_host.h>
> +
> +#define MAX_FSTYPE_LEN 50
> +
> +static struct lkl_disk *lkl_disks[16];
> +static int registered_blk_dev_idx;
> +
> +static int lkl_disk_um_add(struct lkl_disk *disk, const char *blkparams)
> +{
> +	/* concat strings */
> +	snprintf(lkl_um_devs + strlen(lkl_um_devs), sizeof(lkl_um_devs),
> +		 " ubd%d=%s", registered_blk_dev_idx, blkparams);
> +
> +	return registered_blk_dev_idx++;
> +}
> +
> +int lkl_disk_add(struct lkl_disk *disk)
> +{
> +	int ret = -1;
> +
> +	ret = lkl_disk_um_add(disk, disk->dev);
> +
> +	lkl_disks[ret] = disk;
> +
> +	return ret;
> +}
> +
> +int lkl_disk_remove(struct lkl_disk disk)
> +{
> +	/* FIXME */
> +	return 0;
> +}
> +
> +int lkl_mount_fs(char *fstype)
> +{
> +	char dir[MAX_FSTYPE_LEN+2] = "/";
> +	int flags = 0, ret = 0;
> +
> +	strncat(dir, fstype, MAX_FSTYPE_LEN);
> +
> +	/* Create with regular umask */
> +	ret = lkl_sys_mkdir(dir, 0xff);
> +	if (ret && ret != -LKL_EEXIST) {
> +		lkl_perror("mount_fs mkdir", ret);
> +		return ret;
> +	}
> +
> +	/* We have no use for nonzero flags right now */
> +	ret = lkl_sys_mount("none", dir, fstype, flags, NULL);
> +	if (ret && ret != -LKL_EBUSY) {
> +		lkl_sys_rmdir(dir);
> +		return ret;
> +	}
> +
> +	if (ret == -LKL_EBUSY)
> +		return 1;
> +	return 0;
> +}
> +
> +static uint32_t new_encode_dev(unsigned int major, unsigned int minor)
> +{
> +	return (minor & 0xff) | (major << 8) | ((minor & ~0xff) << 12);
> +}
> +
> +static int startswith(const char *str, const char *pre)
> +{
> +	return strncmp(pre, str, strlen(pre)) == 0;
> +}
> +
> +static int get_node_with_prefix(const char *path, const char *prefix,
> +				char *result, unsigned int result_len)
> +{
> +	struct lkl_dir *dir = NULL;
> +	struct lkl_linux_dirent64 *dirent;
> +	int ret;
> +
> +	dir = lkl_opendir(path, &ret);
> +	if (!dir)
> +		return ret;
> +
> +	ret = -LKL_ENOENT;
> +
> +	while ((dirent = lkl_readdir(dir))) {
> +		if (startswith(dirent->d_name, prefix)) {
> +			if (strlen(dirent->d_name) + 1 > result_len) {
> +				ret = -LKL_ENOMEM;
> +				break;
> +			}
> +			memcpy(result, dirent->d_name, strlen(dirent->d_name));
> +			result[strlen(dirent->d_name)] = '\0';
> +			ret = 0;
> +			break;
> +		}
> +	}
> +
> +	lkl_closedir(dir);
> +
> +	return ret;
> +}
> +
> +int lkl_encode_dev_from_sysfs(const char *sysfs_path, uint32_t *pdevid)
> +{
> +	int ret;
> +	long fd;
> +	int major, minor;
> +	char buf[16] = { 0, };
> +	char *bufptr;
> +
> +	fd = lkl_sys_open(sysfs_path, LKL_O_RDONLY, 0);
> +	if (fd < 0)
> +		return fd;
> +
> +	ret = lkl_sys_read(fd, buf, sizeof(buf));
> +	if (ret < 0)
> +		goto out_close;
> +
> +	if (ret == sizeof(buf)) {
> +		ret = -LKL_ENOBUFS;
> +		goto out_close;
> +	}
> +
> +	bufptr = strchr(buf, ':');
> +	if (bufptr == NULL) {
> +		ret = -LKL_EINVAL;
> +		goto out_close;
> +	}
> +	bufptr[0] = '\0';
> +	bufptr++;
> +
> +	major = atoi(buf);
> +	minor = atoi(bufptr);
> +
> +	*pdevid = new_encode_dev(major, minor);
> +	ret = 0;
> +
> +out_close:
> +	lkl_sys_close(fd);
> +
> +	return ret;
> +}
> +
> +#define SYSFS_DEV_UMBLK_CMDLINE_PATH \
> +	"/sysfs/devices/platform/uml-blkdev.%d"
> +
> +struct abuf {
> +	char *mem, *ptr;
> +	unsigned int len;
> +};
> +
> +static int snprintf_append(struct abuf *buf, const char *fmt, ...)
> +{
> +	int ret;
> +	va_list args;
> +
> +	if (!buf->ptr)
> +		buf->ptr = buf->mem;
> +
> +	va_start(args, fmt);
> +	ret = vsnprintf(buf->ptr, buf->len - (buf->ptr - buf->mem), fmt, args);
> +	va_end(args);
> +
> +	if (ret < 0 || (ret >= (int)(buf->len - (buf->ptr - buf->mem))))
> +		return -LKL_ENOMEM;
> +
> +	buf->ptr += ret;
> +
> +	return 0;
> +}
> +
> +static int __lkl_get_blkdev(int disk_id, unsigned int part, uint32_t *pdevid,
> +			    const char *sysfs_path_fmt, const char *drv_prefix,
> +			    const char *disk_prefix)
> +{
> +	char sysfs_path[LKL_PATH_MAX];
> +	char drv_name[LKL_PATH_MAX];
> +	char disk_name[LKL_PATH_MAX];
> +	struct abuf sysfs_path_buf = {
> +		.mem = sysfs_path,
> +		.len = sizeof(sysfs_path),
> +	};
> +	int ret;
> +
> +	if (disk_id < 0)
> +		return -LKL_EINVAL;
> +
> +	ret = lkl_mount_fs("sysfs");
> +	if (ret < 0)
> +		return ret;
> +
> +	ret = snprintf_append(&sysfs_path_buf, sysfs_path_fmt, disk_id);
> +	if (ret)
> +		return ret;
> +
> +	ret = get_node_with_prefix(sysfs_path, drv_prefix, drv_name,
> +				   sizeof(drv_name));
> +	if (ret)
> +		return ret;
> +
> +	ret = snprintf_append(&sysfs_path_buf, "/%s/block", drv_name);
> +	if (ret)
> +		return ret;
> +
> +	ret = get_node_with_prefix(sysfs_path, disk_prefix, disk_name,
> +				   sizeof(disk_name));
> +	if (ret)
> +		return ret;
> +
> +	if (!part)
> +		ret = snprintf_append(&sysfs_path_buf, "/%s/dev", disk_name);
> +	else
> +		ret = snprintf_append(&sysfs_path_buf, "/%s/%s%d/dev",
> +				      disk_name, disk_name, part);
> +	if (ret)
> +		return ret;
> +
> +	return lkl_encode_dev_from_sysfs(sysfs_path, pdevid);
> +}
> +
> +int lkl_get_blkdev(int disk_id, unsigned int part, uint32_t *pdevid)
> +{
> +	char *fmt;
> +
> +	fmt = SYSFS_DEV_UMBLK_CMDLINE_PATH;
> +	return __lkl_get_blkdev(disk_id, part, pdevid, fmt, "", "ubd");
> +}
> +
> +long lkl_mount_dev(unsigned int disk_id, unsigned int part,
> +		   const char *fs_type, int flags,
> +		   const char *data, char *mnt_str, unsigned int mnt_str_len)
> +{
> +	char dev_str[] = { "/dev/xxxxxxxx" };
> +	unsigned int dev;
> +	int err;
> +	char _data[4096]; /* FIXME: PAGE_SIZE is not exported by LKL */
> +
> +	if (mnt_str_len < sizeof(dev_str))
> +		return -LKL_ENOMEM;
> +
> +	err = lkl_get_blkdev(disk_id, part, &dev);
> +	if (err < 0)
> +		return err;
> +
> +	snprintf(dev_str, sizeof(dev_str), "/dev/%08x", dev);
> +	snprintf(mnt_str, mnt_str_len, "/mnt/%08x", dev);
> +
> +	err = lkl_sys_access("/dev", LKL_S_IRWXO);
> +	if (err < 0) {
> +		if (err == -LKL_ENOENT)
> +			err = lkl_sys_mkdir("/dev", 0700);
> +		if (err < 0)
> +			return err;
> +	}
> +
> +	err = lkl_sys_mknod(dev_str, LKL_S_IFBLK | 0600, dev);
> +	if (err < 0)
> +		return err;
> +
> +	err = lkl_sys_access("/mnt", LKL_S_IRWXO);
> +	if (err < 0) {
> +		if (err == -LKL_ENOENT)
> +			err = lkl_sys_mkdir("/mnt", 0700);
> +		if (err < 0)
> +			return err;
> +	}
> +
> +	err = lkl_sys_mkdir(mnt_str, 0700);
> +	if (err < 0) {
> +		lkl_sys_unlink(dev_str);
> +		return err;
> +	}
> +
> +	/* kernel always copies a full page */
> +	if (data) {
> +		strncpy(_data, data, sizeof(_data));
> +		_data[sizeof(_data) - 1] = 0;
> +	} else {
> +		_data[0] = 0;
> +	}
> +
> +	err = lkl_sys_mount(dev_str, mnt_str, (char *)fs_type, flags, _data);
> +	if (err < 0) {
> +		lkl_sys_unlink(dev_str);
> +		lkl_sys_rmdir(mnt_str);
> +		return err;
> +	}
> +
> +	return 0;
> +}
> +
> +long lkl_umount_timeout(char *path, int flags, long timeout_ms)
> +{
> +	long incr = 10000000; /* 10 ms */
> +	struct lkl_timespec ts = {
> +		.tv_sec = 0,
> +		.tv_nsec = incr,
> +	};
> +	long err;
> +
> +	do {
> +		err = lkl_sys_umount(path, flags);
> +		if (err == -LKL_EBUSY) {
> +			lkl_sys_nanosleep((struct __lkl__kernel_timespec *)&ts,
> +					  NULL);
> +			timeout_ms -= incr / 1000000;
> +		}
> +	} while (err == -LKL_EBUSY && timeout_ms > 0);
> +
> +	return err;
> +}
> +
> +long lkl_umount_dev(unsigned int disk_id, unsigned int part, int flags,
> +		    long timeout_ms)
> +{
> +	char dev_str[] = { "/dev/xxxxxxxx" };
> +	char mnt_str[] = { "/mnt/xxxxxxxx" };
> +	unsigned int dev;
> +	int err;
> +
> +	err = lkl_get_blkdev(disk_id, part, &dev);
> +	if (err < 0)
> +		return err;
> +
> +	snprintf(dev_str, sizeof(dev_str), "/dev/%08x", dev);
> +	snprintf(mnt_str, sizeof(mnt_str), "/mnt/%08x", dev);
> +
> +	err = lkl_umount_timeout(mnt_str, flags, timeout_ms);
> +	if (err)
> +		return err;
> +
> +	err = lkl_sys_unlink(dev_str);
> +	if (err)
> +		return err;
> +
> +	return lkl_sys_rmdir(mnt_str);
> +}
> +
> +struct lkl_dir {
> +	int fd;
> +	char buf[1024];
> +	char *pos;
> +	int len;
> +};
> +
> +static struct lkl_dir *lkl_dir_alloc(int *err)
> +{
> +	struct lkl_dir *dir = lkl_host_ops.mem_alloc(sizeof(struct lkl_dir));
> +
> +	if (!dir) {
> +		*err = -LKL_ENOMEM;
> +		return NULL;
> +	}
> +
> +	dir->len = 0;
> +	dir->pos = NULL;
> +
> +	return dir;
> +}
> +
> +struct lkl_dir *lkl_opendir(const char *path, int *err)
> +{
> +	struct lkl_dir *dir = lkl_dir_alloc(err);
> +
> +	if (!dir) {
> +		*err = -LKL_ENOMEM;
> +		return NULL;
> +	}
> +
> +	dir->fd = lkl_sys_open(path, LKL_O_RDONLY | LKL_O_DIRECTORY, 0);
> +	if (dir->fd < 0) {
> +		*err = dir->fd;
> +		lkl_host_ops.mem_free(dir);
> +		return NULL;
> +	}
> +
> +	*err = 0;
> +
> +	return dir;
> +}
> +
> +struct lkl_dir *lkl_fdopendir(int fd, int *err)
> +{
> +	struct lkl_dir *dir = lkl_dir_alloc(err);
> +
> +	if (!dir)
> +		return NULL;
> +
> +	dir->fd = fd;
> +
> +	return dir;
> +}
> +
> +void lkl_rewinddir(struct lkl_dir *dir)
> +{
> +	lkl_sys_lseek(dir->fd, 0, LKL_SEEK_SET);
> +	dir->len = 0;
> +	dir->pos = NULL;
> +}
> +
> +int lkl_closedir(struct lkl_dir *dir)
> +{
> +	int ret;
> +
> +	ret = lkl_sys_close(dir->fd);
> +	lkl_host_ops.mem_free(dir);
> +
> +	return ret;
> +}
> +
> +struct lkl_linux_dirent64 *lkl_readdir(struct lkl_dir *dir)
> +{
> +	struct lkl_linux_dirent64 *de;
> +
> +	if (dir->len < 0)
> +		return NULL;
> +
> +	if (!dir->pos || dir->pos - dir->buf >= dir->len)
> +		goto read_buf;
> +
> +return_de:
> +	de = (struct lkl_linux_dirent64 *)dir->pos;
> +	dir->pos += de->d_reclen;
> +
> +	return de;
> +
> +read_buf:
> +	dir->pos = NULL;
> +	de = (struct lkl_linux_dirent64 *)dir->buf;
> +	dir->len = lkl_sys_getdents64(dir->fd, de, sizeof(dir->buf));
> +	if (dir->len <= 0)
> +		return NULL;
> +
> +	dir->pos = dir->buf;
> +	goto return_de;
> +}
> +
> +int lkl_errdir(struct lkl_dir *dir)
> +{
> +	if (dir->len >= 0)
> +		return 0;
> +
> +	return dir->len;
> +}
> +
> +int lkl_dirfd(struct lkl_dir *dir)
> +{
> +	return dir->fd;
> +}
> +
> +int lkl_set_fd_limit(unsigned int fd_limit)
> +{
> +	struct lkl_rlimit rlim = {
> +		.rlim_cur = fd_limit,
> +		.rlim_max = fd_limit,
> +	};
> +	return lkl_sys_setrlimit(LKL_RLIMIT_NOFILE, &rlim);
> +}
> diff --git a/tools/um/lib/posix-host.c b/tools/um/lib/posix-host.c
> index b6b5b2902254..8fd88031bf2b 100644
> --- a/tools/um/lib/posix-host.c
> +++ b/tools/um/lib/posix-host.c
> @@ -263,6 +263,7 @@ static void *tls_get(struct lkl_tls_key *key)
>   }
>   
>   struct lkl_host_operations lkl_host_ops = {
> +	.um_devices = lkl_um_devs,
>   	.panic = panic,
>   	.print = print,
>   	.mem_alloc = (void *)malloc,
> diff --git a/tools/um/lib/utils.c b/tools/um/lib/utils.c
> index 4930479a8a35..ac65cd744a14 100644
> --- a/tools/um/lib/utils.c
> +++ b/tools/um/lib/utils.c
> @@ -4,6 +4,9 @@
>   #include <string.h>
>   #include <lkl_host.h>
>   
> +/* XXX: find a better place */
> +char lkl_um_devs[4096];
> +
>   static const char * const lkl_err_strings[] = {
>   	"Success",
>   	"Operation not permitted",
> diff --git a/tools/um/tests/Build b/tools/um/tests/Build
> index 564560486f98..1aa78f9ed7ef 100644
> --- a/tools/um/tests/Build
> +++ b/tools/um/tests/Build
> @@ -1,3 +1,4 @@
>   boot-y += boot.o test.o
> +disk-y += disk.o test.o cla.o
>   
>   CFLAGS_test.o += -Wno-implicit-fallthrough
> diff --git a/tools/um/tests/cla.c b/tools/um/tests/cla.c
> new file mode 100644
> index 000000000000..694e3a3822be
> --- /dev/null
> +++ b/tools/um/tests/cla.c
> @@ -0,0 +1,159 @@
> +// SPDX-License-Identifier: GPL-2.0
> +#include <stdio.h>
> +#include <string.h>
> +#include <errno.h>
> +#include <stdlib.h>
> +#ifdef __MINGW32__
> +#include <winsock2.h>
> +#else
> +#include <sys/socket.h>
> +#include <netinet/in.h>
> +#include <arpa/inet.h>
> +#endif
> +
> +#include "cla.h"
> +
> +static int cl_arg_parse_bool(struct cl_arg *arg, const char *value)
> +{
> +	*((int *)arg->store) = 1;
> +	return 0;
> +}
> +
> +static int cl_arg_parse_str(struct cl_arg *arg, const char *value)
> +{
> +	*((const char **)arg->store) = value;
> +	return 0;
> +}
> +
> +static int cl_arg_parse_int(struct cl_arg *arg, const char *value)
> +{
> +	errno = 0;
> +	*((int *)arg->store) = strtol(value, NULL, 0);
> +	return errno == 0;
> +}
> +
> +static int cl_arg_parse_str_set(struct cl_arg *arg, const char *value)
> +{
> +	const char **set = arg->set;
> +	int i;
> +
> +	for (i = 0; set[i] != NULL; i++) {
> +		if (strcmp(set[i], value) == 0) {
> +			*((int *)arg->store) = i;
> +			return 0;
> +		}
> +	}
> +
> +	return (-1);
> +}
> +
> +static int cl_arg_parse_ipv4(struct cl_arg *arg, const char *value)
> +{
> +	unsigned int addr;
> +
> +	if (!value)
> +		return (-1);
> +
> +	addr = inet_addr(value);
> +	if (addr == INADDR_NONE)
> +		return (-1);
> +	*((unsigned int *)arg->store) = addr;
> +	return 0;
> +}
> +
> +static cl_arg_parser_t parsers[] = {
> +	[CL_ARG_BOOL] = cl_arg_parse_bool,
> +	[CL_ARG_INT] = cl_arg_parse_int,
> +	[CL_ARG_STR] = cl_arg_parse_str,
> +	[CL_ARG_STR_SET] = cl_arg_parse_str_set,
> +	[CL_ARG_IPV4] = cl_arg_parse_ipv4,
> +};
> +
> +static struct cl_arg *find_short_arg(char name, struct cl_arg *args)
> +{
> +	struct cl_arg *arg;
> +
> +	for (arg = args; arg->short_name != 0; arg++) {
> +		if (arg->short_name == name)
> +			return arg;
> +	}
> +
> +	return NULL;
> +}
> +
> +static struct cl_arg *find_long_arg(const char *name, struct cl_arg *args)
> +{
> +	struct cl_arg *arg;
> +
> +	for (arg = args; arg->long_name; arg++) {
> +		if (strcmp(arg->long_name, name) == 0)
> +			return arg;
> +	}
> +
> +	return NULL;
> +}
> +
> +static void print_help(struct cl_arg *args)
> +{
> +	struct cl_arg *arg;
> +
> +	fprintf(stderr, "usage:\n");
> +	for (arg = args; arg->long_name; arg++) {
> +		fprintf(stderr, "-%c, --%-20s %s", arg->short_name,
> +			arg->long_name, arg->help);
> +		if (arg->type == CL_ARG_STR_SET) {
> +			const char **set = arg->set;
> +
> +			fprintf(stderr, " [ ");
> +			while (*set != NULL)
> +				fprintf(stderr, "%s ", *(set++));
> +			fprintf(stderr, "]");
> +		}
> +		fprintf(stderr, "\n");
> +	}
> +}
> +
> +int cla_parse_args(int argc, const char **argv, struct cl_arg *args)
> +{
> +	int i;
> +
> +	for (i = 1; i < argc; i++) {
> +		struct cl_arg *arg = NULL;
> +		cl_arg_parser_t parser;
> +
> +		if (argv[i][0] == '-') {
> +			if (argv[i][1] != '-')
> +				arg = find_short_arg(argv[i][1], args);
> +			else
> +				arg = find_long_arg(&argv[i][2], args);
> +		}
> +
> +		if (!arg) {
> +			fprintf(stderr, "unknown option '%s'\n", argv[i]);
> +			print_help(args);
> +			return (-1);
> +		}
> +
> +		if (arg->type == CL_ARG_USER || arg->type >= CL_ARG_END)
> +			parser = arg->parser;
> +		else
> +			parser = parsers[arg->type];
> +
> +		if (!parser) {
> +			fprintf(stderr, "can't parse --'%s'/-'%c'\n",
> +				arg->long_name, args->short_name);
> +			return (-1);
> +		}
> +
> +		if (parser(arg, argv[i + 1]) < 0) {
> +			fprintf(stderr, "can't parse '%s'\n", argv[i]);
> +			print_help(args);
> +			return (-1);
> +		}
> +
> +		if (arg->has_arg)
> +			i++;
> +	}
> +
> +	return 0;
> +}
> diff --git a/tools/um/tests/cla.h b/tools/um/tests/cla.h
> new file mode 100644
> index 000000000000..3d879233681f
> --- /dev/null
> +++ b/tools/um/tests/cla.h
> @@ -0,0 +1,33 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +#ifndef _LKL_TEST_CLA_H
> +#define _LKL_TEST_CLA_H
> +
> +enum cl_arg_type {
> +	CL_ARG_USER = 0,
> +	CL_ARG_BOOL,
> +	CL_ARG_INT,
> +	CL_ARG_STR,
> +	CL_ARG_STR_SET,
> +	CL_ARG_IPV4,
> +	CL_ARG_END,
> +};
> +
> +struct cl_arg;
> +
> +typedef int (*cl_arg_parser_t)(struct cl_arg *arg, const char *value);
> +
> +struct cl_arg {
> +	const char *long_name;
> +	char short_name;
> +	const char *help;
> +	int has_arg;
> +	enum cl_arg_type type;
> +	void *store;
> +	void *set;
> +	cl_arg_parser_t parser;
> +};
> +
> +int cla_parse_args(int argc, const char **argv, struct cl_arg *args);
> +
> +
> +#endif /* _LKL_TEST_CLA_H */
> diff --git a/tools/um/tests/disk.c b/tools/um/tests/disk.c
> new file mode 100644
> index 000000000000..325934935e6a
> --- /dev/null
> +++ b/tools/um/tests/disk.c
> @@ -0,0 +1,168 @@
> +// SPDX-License-Identifier: GPL-2.0
> +#include <stdio.h>
> +#include <unistd.h>
> +#include <string.h>
> +#include <time.h>
> +#include <stdlib.h>
> +#include <stdint.h>
> +#include <lkl.h>
> +#include <lkl_host.h>
> +#include <sys/stat.h>
> +#include <fcntl.h>
> +#include <sys/ioctl.h>
> +
> +#include "test.h"
> +#include "cla.h"
> +
> +static struct {
> +	int printk;
> +	const char *disk;
> +	const char *fstype;
> +	int partition;
> +} cla;
> +
> +struct cl_arg args[] = {
> +	{"disk", 'd', "disk file to use", 1, CL_ARG_STR, &cla.disk},
> +	{"partition", 'P', "partition to mount", 1, CL_ARG_INT, &cla.partition},
> +	{"type", 't', "filesystem type", 1, CL_ARG_STR, &cla.fstype},
> +	{0},
> +};
> +
> +
> +static struct lkl_disk disk;
> +static int disk_id = -1;
> +
> +int lkl_test_disk_add(void)
> +{
> +	disk.fd = open(cla.disk, O_RDWR);
> +	if (disk.fd < 0)
> +		goto out_unlink;
> +
> +	disk.ops = NULL;
> +	disk.dev = (char *)cla.disk;
> +
> +	disk_id = lkl_disk_add(&disk);
> +	if (disk_id < 0)
> +		goto out_close;
> +
> +	goto out;
> +
> +out_close:
> +	close(disk.fd);
> +
> +out_unlink:
> +	unlink(cla.disk);
> +
> +out:
> +	lkl_test_logf("disk fd/handle %x disk_id %d", disk.fd, disk_id);
> +
> +	if (disk_id >= 0)
> +		return TEST_SUCCESS;
> +
> +	return TEST_FAILURE;
> +}
> +
> +int lkl_test_disk_remove(void)
> +{
> +	int ret;
> +
> +	ret = lkl_disk_remove(disk);
> +
> +	close(disk.fd);
> +
> +	if (ret == 0)
> +		return TEST_SUCCESS;
> +
> +	return TEST_FAILURE;
> +}
> +
> +
> +static char mnt_point[32];
> +
> +LKL_TEST_CALL(mount_dev, lkl_mount_dev, 0, disk_id, cla.partition, cla.fstype,
> +	      0, NULL, mnt_point, sizeof(mnt_point))
> +
> +static int lkl_test_umount_dev(void)
> +{
> +	long ret, ret2;
> +
> +	ret = lkl_sys_chdir("/");
> +
> +	ret2 = lkl_umount_dev(disk_id, cla.partition, 0, 1000);
> +
> +	lkl_test_logf("%ld %ld", ret, ret2);
> +
> +	if (!ret && !ret2)
> +		return TEST_SUCCESS;
> +
> +	return TEST_FAILURE;
> +}
> +
> +struct lkl_dir *dir;
> +
> +static int lkl_test_opendir(void)
> +{
> +	int err;
> +
> +	dir = lkl_opendir(mnt_point, &err);
> +
> +	lkl_test_logf("lkl_opedir(%s) = %d %s\n", mnt_point, err,
> +		      lkl_strerror(err));
> +
> +	if (err == 0)
> +		return TEST_SUCCESS;
> +
> +	return TEST_FAILURE;
> +}
> +
> +static int lkl_test_readdir(void)
> +{
> +	struct lkl_linux_dirent64 *de = lkl_readdir(dir);
> +	int wr = 0;
> +
> +	while (de) {
> +		wr += lkl_test_logf("%s ", de->d_name);
> +		if (wr >= 70) {
> +			lkl_test_logf("\n");
> +			wr = 0;
> +			break;
> +		}
> +		de = lkl_readdir(dir);
> +	}
> +
> +	if (lkl_errdir(dir) == 0)
> +		return TEST_SUCCESS;
> +
> +	return TEST_FAILURE;
> +}
> +
> +LKL_TEST_CALL(closedir, lkl_closedir, 0, dir);
> +LKL_TEST_CALL(chdir_mnt_point, lkl_sys_chdir, 0, mnt_point);
> +LKL_TEST_CALL(start_kernel, lkl_start_kernel, 0, &lkl_host_ops,
> +	     "mem=16M loglevel=8");
> +LKL_TEST_CALL(stop_kernel, lkl_sys_halt, 0);
> +
> +struct lkl_test tests[] = {
> +	LKL_TEST(disk_add),
> +	LKL_TEST(start_kernel),
> +	LKL_TEST(mount_dev),
> +	LKL_TEST(chdir_mnt_point),
> +	LKL_TEST(opendir),
> +	LKL_TEST(readdir),
> +	LKL_TEST(closedir),
> +	LKL_TEST(umount_dev),
> +	LKL_TEST(stop_kernel),
> +	LKL_TEST(disk_remove),
> +
> +};
> +
> +int main(int argc, const char **argv)
> +{
> +	if (cla_parse_args(argc, argv, args) < 0)
> +		return (-1);
> +
> +	lkl_host_ops.print = lkl_test_log;
> +
> +	return lkl_test_run(tests, sizeof(tests)/sizeof(struct lkl_test),
> +			    "disk %s", cla.fstype);
> +}
> diff --git a/tools/um/tests/disk.sh b/tools/um/tests/disk.sh
> new file mode 100755
> index 000000000000..e2ec6cf69d4b
> --- /dev/null
> +++ b/tools/um/tests/disk.sh
> @@ -0,0 +1,70 @@
> +#!/usr/bin/env bash
> +# SPDX-License-Identifier: GPL-2.0
> +
> +script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
> +
> +source $script_dir/test.sh
> +
> +function prepfs()
> +{
> +    set -e
> +
> +    file=`mktemp`
> +
> +    dd if=/dev/zero of=$file bs=1024 count=204800
> +
> +    yes | mkfs.$1 $file
> +
> +    if ! [ -z $ANDROID_WDIR ]; then
> +        adb shell mkdir -p $ANDROID_WDIR
> +        adb push $file $ANDROID_WDIR
> +        rm $file
> +        file=$ANDROID_WDIR/$(basename $file)
> +    fi
> +    if ! [ -z $BSD_WDIR ]; then
> +        $MYSSH mkdir -p $BSD_WDIR
> +        ssh_copy $file $BSD_WDIR
> +        rm $file
> +        file=$BSD_WDIR/$(basename $file)
> +    fi
> +
> +    export_vars file
> +}
> +
> +function cleanfs()
> +{
> +    set -e
> +
> +    if ! [ -z $ANDROID_WDIR ]; then
> +        adb shell rm $1
> +        adb shell rm $ANDROID_WDIR/disk
> +    elif ! [ -z $BSD_WDIR ]; then
> +        $MYSSH rm $1
> +        $MYSSH rm $BSD_WDIR/disk
> +    else
> +        rm $1
> +    fi
> +}
> +
> +if [ "$1" = "-t" ]; then
> +    shift
> +    fstype=$1
> +    shift
> +fi
> +
> +if [ -z "$fstype" ]; then
> +    fstype="ext4"
> +fi
> +
> +if [ -z $(which mkfs.$fstype) ]; then
> +    lkl_test_plan 0 "disk $fstype"
> +    echo "no mkfs.$fstype command"
> +    exit 0
> +fi
> +
> +lkl_test_plan 1 "disk $fstype"
> +lkl_test_run 1 prepfs $fstype
> +lkl_test_exec $script_dir/disk -d $file -t $fstype $@
> +lkl_test_plan 1 "disk $fstype"
> +lkl_test_run 1 cleanfs $file
> +
> diff --git a/tools/um/tests/run.py b/tools/um/tests/run.py
> index c96ede90b6ad..97d6dedc217c 100755
> --- a/tools/um/tests/run.py
> +++ b/tools/um/tests/run.py
> @@ -50,6 +50,8 @@ mydir=os.path.dirname(os.path.realpath(__file__))
>   
>   tests = [
>       'boot.sh',
> +    'disk.sh -t ext4',
> +    'disk.sh -t vfat',
>   ]
>   
>   parser = argparse.ArgumentParser(description='LKL test runner')

-- 
Anton R. Ivanov
Cambridgegreys Limited. Registered in England. Company Number 10273661
https://www.cambridgegreys.com/


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 18/21] um: host: add utilities functions
  2020-10-06  9:44           ` Hajime Tazaki
@ 2020-10-07 14:53             ` Anton Ivanov
  -1 siblings, 0 replies; 476+ messages in thread
From: Anton Ivanov @ 2020-10-07 14:53 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard
  Cc: tavi.purdila, linux-kernel-library, linux-arch, retrage01



On 06/10/2020 10:44, Hajime Tazaki wrote:
> Add basic utility functions for getting a string from a kernel error
> code and a fprintf like function that uses the host print
> operation. The latter is useful for informing the user about errors
> that occur in the host library.
> 
> Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
> ---
>   tools/um/lib/Build     |   4 +
>   tools/um/lib/jmp_buf.c |  14 +++
>   tools/um/lib/jmp_buf.h |   8 ++
>   tools/um/lib/utils.c   | 210 +++++++++++++++++++++++++++++++++++++++++
>   4 files changed, 236 insertions(+)
>   create mode 100644 tools/um/lib/Build
>   create mode 100644 tools/um/lib/jmp_buf.c
>   create mode 100644 tools/um/lib/jmp_buf.h
>   create mode 100644 tools/um/lib/utils.c
> 
> diff --git a/tools/um/lib/Build b/tools/um/lib/Build
> new file mode 100644
> index 000000000000..fc967af4104a
> --- /dev/null
> +++ b/tools/um/lib/Build
> @@ -0,0 +1,4 @@
> +include $(objtree)/include/config/auto.conf
> +
> +liblinux-$(CONFIG_UMMODE_LIB) += utils.o
> +liblinux-$(CONFIG_UMMODE_LIB) += jmp_buf.o
> diff --git a/tools/um/lib/jmp_buf.c b/tools/um/lib/jmp_buf.c
> new file mode 100644
> index 000000000000..f6bdd7e4bd83
> --- /dev/null
> +++ b/tools/um/lib/jmp_buf.c
> @@ -0,0 +1,14 @@
> +// SPDX-License-Identifier: GPL-2.0
> +#include <setjmp.h>
> +#include <lkl_host.h>
> +
> +void jmp_buf_set(struct lkl_jmp_buf *jmpb, void (*f)(void))
> +{
> +	if (!setjmp(*((jmp_buf *)jmpb->buf)))
> +		f();
> +}
> +
> +void jmp_buf_longjmp(struct lkl_jmp_buf *jmpb, int val)
> +{
> +	longjmp(*((jmp_buf *)jmpb->buf), val);
> +}
> diff --git a/tools/um/lib/jmp_buf.h b/tools/um/lib/jmp_buf.h
> new file mode 100644
> index 000000000000..8782cbaaf51f
> --- /dev/null
> +++ b/tools/um/lib/jmp_buf.h
> @@ -0,0 +1,8 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef _LKL_LIB_JMP_BUF_H
> +#define _LKL_LIB_JMP_BUF_H
> +
> +void jmp_buf_set(struct lkl_jmp_buf *jmpb, void (*f)(void));
> +void jmp_buf_longjmp(struct lkl_jmp_buf *jmpb, int val);
> +
> +#endif
> diff --git a/tools/um/lib/utils.c b/tools/um/lib/utils.c
> new file mode 100644
> index 000000000000..4930479a8a35
> --- /dev/null
> +++ b/tools/um/lib/utils.c
> @@ -0,0 +1,210 @@
> +// SPDX-License-Identifier: GPL-2.0
> +#include <stdarg.h>
> +#include <stdio.h>
> +#include <string.h>
> +#include <lkl_host.h>

These are actually different on different architectures. These look like the x86 values.

IMHO a kernel strerror() would be the right way of dealing with this in the long term (i understand that we cannot call the platform one, because it may be different from the internal Linux errors). It will be useful in a lot of other places.

If we leave it as is, we need to make this arch specific at some point.

> +
> +static const char * const lkl_err_strings[] = {
> +	"Success",
> +	"Operation not permitted",
> +	"No such file or directory",
> +	"No such process",
> +	"Interrupted system call",
> +	"I/O error",
> +	"No such device or address",
> +	"Argument list too long",
> +	"Exec format error",
> +	"Bad file number",
> +	"No child processes",
> +	"Try again",
> +	"Out of memory",
> +	"Permission denied",
> +	"Bad address",
> +	"Block device required",
> +	"Device or resource busy",
> +	"File exists",
> +	"Cross-device link",
> +	"No such device",
> +	"Not a directory",
> +	"Is a directory",
> +	"Invalid argument",
> +	"File table overflow",
> +	"Too many open files",
> +	"Not a typewriter",
> +	"Text file busy",
> +	"File too large",
> +	"No space left on device",
> +	"Illegal seek",
> +	"Read-only file system",
> +	"Too many links",
> +	"Broken pipe",
> +	"Math argument out of domain of func",
> +	"Math result not representable",
> +	"Resource deadlock would occur",
> +	"File name too long",
> +	"No record locks available",
> +	"Invalid system call number",
> +	"Directory not empty",
> +	"Too many symbolic links encountered",
> +	"Bad error code", /* EWOULDBLOCK is EAGAIN */
> +	"No message of desired type",
> +	"Identifier removed",
> +	"Channel number out of range",
> +	"Level 2 not synchronized",
> +	"Level 3 halted",
> +	"Level 3 reset",
> +	"Link number out of range",
> +	"Protocol driver not attached",
> +	"No CSI structure available",
> +	"Level 2 halted",
> +	"Invalid exchange",
> +	"Invalid request descriptor",
> +	"Exchange full",
> +	"No anode",
> +	"Invalid request code",
> +	"Invalid slot",
> +	"Bad error code", /* EDEADLOCK is EDEADLK */
> +	"Bad font file format",
> +	"Device not a stream",
> +	"No data available",
> +	"Timer expired",
> +	"Out of streams resources",
> +	"Machine is not on the network",
> +	"Package not installed",
> +	"Object is remote",
> +	"Link has been severed",
> +	"Advertise error",
> +	"Srmount error",
> +	"Communication error on send",
> +	"Protocol error",
> +	"Multihop attempted",
> +	"RFS specific error",
> +	"Not a data message",
> +	"Value too large for defined data type",
> +	"Name not unique on network",
> +	"File descriptor in bad state",
> +	"Remote address changed",
> +	"Can not access a needed shared library",
> +	"Accessing a corrupted shared library",
> +	".lib section in a.out corrupted",
> +	"Attempting to link in too many shared libraries",
> +	"Cannot exec a shared library directly",
> +	"Illegal byte sequence",
> +	"Interrupted system call should be restarted",
> +	"Streams pipe error",
> +	"Too many users",
> +	"Socket operation on non-socket",
> +	"Destination address required",
> +	"Message too long",
> +	"Protocol wrong type for socket",
> +	"Protocol not available",
> +	"Protocol not supported",
> +	"Socket type not supported",
> +	"Operation not supported on transport endpoint",
> +	"Protocol family not supported",
> +	"Address family not supported by protocol",
> +	"Address already in use",
> +	"Cannot assign requested address",
> +	"Network is down",
> +	"Network is unreachable",
> +	"Network dropped connection because of reset",
> +	"Software caused connection abort",
> +	"Connection reset by peer",
> +	"No buffer space available",
> +	"Transport endpoint is already connected",
> +	"Transport endpoint is not connected",
> +	"Cannot send after transport endpoint shutdown",
> +	"Too many references: cannot splice",
> +	"Connection timed out",
> +	"Connection refused",
> +	"Host is down",
> +	"No route to host",
> +	"Operation already in progress",
> +	"Operation now in progress",
> +	"Stale file handle",
> +	"Structure needs cleaning",
> +	"Not a XENIX named type file",
> +	"No XENIX semaphores available",
> +	"Is a named type file",
> +	"Remote I/O error",
> +	"Quota exceeded",
> +	"No medium found",
> +	"Wrong medium type",
> +	"Operation Canceled",
> +	"Required key not available",
> +	"Key has expired",
> +	"Key has been revoked",
> +	"Key was rejected by service",
> +	"Owner died",
> +	"State not recoverable",
> +	"Operation not possible due to RF-kill",
> +	"Memory page has hardware error",
> +};
> +
> +const char *lkl_strerror(int err)
> +{
> +	if (err < 0)
> +		err = -err;
> +
> +	if ((size_t)err >= sizeof(lkl_err_strings) / sizeof(const char *))
> +		return "Bad error code";
> +
> +	return lkl_err_strings[err];
> +}
> +
> +void lkl_perror(char *msg, int err)
> +{
> +	const char *err_msg = lkl_strerror(err);
> +	/* We need to use 'real' printf because lkl_host_ops.print can
> +	 * be turned off when debugging is off.
> +	 */
> +	lkl_printf("%s: %s\n", msg, err_msg);
> +}
> +
> +static int lkl_vprintf(const char *fmt, va_list args)
> +{
> +	int n;
> +	char *buffer;
> +	va_list copy;
> +
> +	if (!lkl_host_ops.print)
> +		return 0;
> +
> +	va_copy(copy, args);
> +	n = vsnprintf(NULL, 0, fmt, copy);
> +	va_end(copy);
> +
> +	buffer = lkl_host_ops.mem_alloc(n + 1);
> +	if (!buffer)
> +		return (-1);
> +
> +	vsnprintf(buffer, n + 1, fmt, args);
> +
> +	lkl_host_ops.print(buffer, n);
> +	lkl_host_ops.mem_free(buffer);
> +
> +	return n;
> +}
> +
> +int lkl_printf(const char *fmt, ...)
> +{
> +	int n;
> +	va_list args;
> +
> +	va_start(args, fmt);
> +	n = lkl_vprintf(fmt, args);
> +	va_end(args);
> +
> +	return n;
> +}
> +
> +void lkl_bug(const char *fmt, ...)
> +{
> +	va_list args;
> +
> +	va_start(args, fmt);
> +	lkl_vprintf(fmt, args);
> +	va_end(args);
> +
> +	lkl_host_ops.panic();
> +}
> 

-- 
Anton R. Ivanov
Cambridgegreys Limited. Registered in England. Company Number 10273661
https://www.cambridgegreys.com/

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 18/21] um: host: add utilities functions
@ 2020-10-07 14:53             ` Anton Ivanov
  0 siblings, 0 replies; 476+ messages in thread
From: Anton Ivanov @ 2020-10-07 14:53 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard
  Cc: tavi.purdila, linux-kernel-library, retrage01, linux-arch



On 06/10/2020 10:44, Hajime Tazaki wrote:
> Add basic utility functions for getting a string from a kernel error
> code and a fprintf like function that uses the host print
> operation. The latter is useful for informing the user about errors
> that occur in the host library.
> 
> Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
> ---
>   tools/um/lib/Build     |   4 +
>   tools/um/lib/jmp_buf.c |  14 +++
>   tools/um/lib/jmp_buf.h |   8 ++
>   tools/um/lib/utils.c   | 210 +++++++++++++++++++++++++++++++++++++++++
>   4 files changed, 236 insertions(+)
>   create mode 100644 tools/um/lib/Build
>   create mode 100644 tools/um/lib/jmp_buf.c
>   create mode 100644 tools/um/lib/jmp_buf.h
>   create mode 100644 tools/um/lib/utils.c
> 
> diff --git a/tools/um/lib/Build b/tools/um/lib/Build
> new file mode 100644
> index 000000000000..fc967af4104a
> --- /dev/null
> +++ b/tools/um/lib/Build
> @@ -0,0 +1,4 @@
> +include $(objtree)/include/config/auto.conf
> +
> +liblinux-$(CONFIG_UMMODE_LIB) += utils.o
> +liblinux-$(CONFIG_UMMODE_LIB) += jmp_buf.o
> diff --git a/tools/um/lib/jmp_buf.c b/tools/um/lib/jmp_buf.c
> new file mode 100644
> index 000000000000..f6bdd7e4bd83
> --- /dev/null
> +++ b/tools/um/lib/jmp_buf.c
> @@ -0,0 +1,14 @@
> +// SPDX-License-Identifier: GPL-2.0
> +#include <setjmp.h>
> +#include <lkl_host.h>
> +
> +void jmp_buf_set(struct lkl_jmp_buf *jmpb, void (*f)(void))
> +{
> +	if (!setjmp(*((jmp_buf *)jmpb->buf)))
> +		f();
> +}
> +
> +void jmp_buf_longjmp(struct lkl_jmp_buf *jmpb, int val)
> +{
> +	longjmp(*((jmp_buf *)jmpb->buf), val);
> +}
> diff --git a/tools/um/lib/jmp_buf.h b/tools/um/lib/jmp_buf.h
> new file mode 100644
> index 000000000000..8782cbaaf51f
> --- /dev/null
> +++ b/tools/um/lib/jmp_buf.h
> @@ -0,0 +1,8 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef _LKL_LIB_JMP_BUF_H
> +#define _LKL_LIB_JMP_BUF_H
> +
> +void jmp_buf_set(struct lkl_jmp_buf *jmpb, void (*f)(void));
> +void jmp_buf_longjmp(struct lkl_jmp_buf *jmpb, int val);
> +
> +#endif
> diff --git a/tools/um/lib/utils.c b/tools/um/lib/utils.c
> new file mode 100644
> index 000000000000..4930479a8a35
> --- /dev/null
> +++ b/tools/um/lib/utils.c
> @@ -0,0 +1,210 @@
> +// SPDX-License-Identifier: GPL-2.0
> +#include <stdarg.h>
> +#include <stdio.h>
> +#include <string.h>
> +#include <lkl_host.h>

These are actually different on different architectures. These look like the x86 values.

IMHO a kernel strerror() would be the right way of dealing with this in the long term (i understand that we cannot call the platform one, because it may be different from the internal Linux errors). It will be useful in a lot of other places.

If we leave it as is, we need to make this arch specific at some point.

> +
> +static const char * const lkl_err_strings[] = {
> +	"Success",
> +	"Operation not permitted",
> +	"No such file or directory",
> +	"No such process",
> +	"Interrupted system call",
> +	"I/O error",
> +	"No such device or address",
> +	"Argument list too long",
> +	"Exec format error",
> +	"Bad file number",
> +	"No child processes",
> +	"Try again",
> +	"Out of memory",
> +	"Permission denied",
> +	"Bad address",
> +	"Block device required",
> +	"Device or resource busy",
> +	"File exists",
> +	"Cross-device link",
> +	"No such device",
> +	"Not a directory",
> +	"Is a directory",
> +	"Invalid argument",
> +	"File table overflow",
> +	"Too many open files",
> +	"Not a typewriter",
> +	"Text file busy",
> +	"File too large",
> +	"No space left on device",
> +	"Illegal seek",
> +	"Read-only file system",
> +	"Too many links",
> +	"Broken pipe",
> +	"Math argument out of domain of func",
> +	"Math result not representable",
> +	"Resource deadlock would occur",
> +	"File name too long",
> +	"No record locks available",
> +	"Invalid system call number",
> +	"Directory not empty",
> +	"Too many symbolic links encountered",
> +	"Bad error code", /* EWOULDBLOCK is EAGAIN */
> +	"No message of desired type",
> +	"Identifier removed",
> +	"Channel number out of range",
> +	"Level 2 not synchronized",
> +	"Level 3 halted",
> +	"Level 3 reset",
> +	"Link number out of range",
> +	"Protocol driver not attached",
> +	"No CSI structure available",
> +	"Level 2 halted",
> +	"Invalid exchange",
> +	"Invalid request descriptor",
> +	"Exchange full",
> +	"No anode",
> +	"Invalid request code",
> +	"Invalid slot",
> +	"Bad error code", /* EDEADLOCK is EDEADLK */
> +	"Bad font file format",
> +	"Device not a stream",
> +	"No data available",
> +	"Timer expired",
> +	"Out of streams resources",
> +	"Machine is not on the network",
> +	"Package not installed",
> +	"Object is remote",
> +	"Link has been severed",
> +	"Advertise error",
> +	"Srmount error",
> +	"Communication error on send",
> +	"Protocol error",
> +	"Multihop attempted",
> +	"RFS specific error",
> +	"Not a data message",
> +	"Value too large for defined data type",
> +	"Name not unique on network",
> +	"File descriptor in bad state",
> +	"Remote address changed",
> +	"Can not access a needed shared library",
> +	"Accessing a corrupted shared library",
> +	".lib section in a.out corrupted",
> +	"Attempting to link in too many shared libraries",
> +	"Cannot exec a shared library directly",
> +	"Illegal byte sequence",
> +	"Interrupted system call should be restarted",
> +	"Streams pipe error",
> +	"Too many users",
> +	"Socket operation on non-socket",
> +	"Destination address required",
> +	"Message too long",
> +	"Protocol wrong type for socket",
> +	"Protocol not available",
> +	"Protocol not supported",
> +	"Socket type not supported",
> +	"Operation not supported on transport endpoint",
> +	"Protocol family not supported",
> +	"Address family not supported by protocol",
> +	"Address already in use",
> +	"Cannot assign requested address",
> +	"Network is down",
> +	"Network is unreachable",
> +	"Network dropped connection because of reset",
> +	"Software caused connection abort",
> +	"Connection reset by peer",
> +	"No buffer space available",
> +	"Transport endpoint is already connected",
> +	"Transport endpoint is not connected",
> +	"Cannot send after transport endpoint shutdown",
> +	"Too many references: cannot splice",
> +	"Connection timed out",
> +	"Connection refused",
> +	"Host is down",
> +	"No route to host",
> +	"Operation already in progress",
> +	"Operation now in progress",
> +	"Stale file handle",
> +	"Structure needs cleaning",
> +	"Not a XENIX named type file",
> +	"No XENIX semaphores available",
> +	"Is a named type file",
> +	"Remote I/O error",
> +	"Quota exceeded",
> +	"No medium found",
> +	"Wrong medium type",
> +	"Operation Canceled",
> +	"Required key not available",
> +	"Key has expired",
> +	"Key has been revoked",
> +	"Key was rejected by service",
> +	"Owner died",
> +	"State not recoverable",
> +	"Operation not possible due to RF-kill",
> +	"Memory page has hardware error",
> +};
> +
> +const char *lkl_strerror(int err)
> +{
> +	if (err < 0)
> +		err = -err;
> +
> +	if ((size_t)err >= sizeof(lkl_err_strings) / sizeof(const char *))
> +		return "Bad error code";
> +
> +	return lkl_err_strings[err];
> +}
> +
> +void lkl_perror(char *msg, int err)
> +{
> +	const char *err_msg = lkl_strerror(err);
> +	/* We need to use 'real' printf because lkl_host_ops.print can
> +	 * be turned off when debugging is off.
> +	 */
> +	lkl_printf("%s: %s\n", msg, err_msg);
> +}
> +
> +static int lkl_vprintf(const char *fmt, va_list args)
> +{
> +	int n;
> +	char *buffer;
> +	va_list copy;
> +
> +	if (!lkl_host_ops.print)
> +		return 0;
> +
> +	va_copy(copy, args);
> +	n = vsnprintf(NULL, 0, fmt, copy);
> +	va_end(copy);
> +
> +	buffer = lkl_host_ops.mem_alloc(n + 1);
> +	if (!buffer)
> +		return (-1);
> +
> +	vsnprintf(buffer, n + 1, fmt, args);
> +
> +	lkl_host_ops.print(buffer, n);
> +	lkl_host_ops.mem_free(buffer);
> +
> +	return n;
> +}
> +
> +int lkl_printf(const char *fmt, ...)
> +{
> +	int n;
> +	va_list args;
> +
> +	va_start(args, fmt);
> +	n = lkl_vprintf(fmt, args);
> +	va_end(args);
> +
> +	return n;
> +}
> +
> +void lkl_bug(const char *fmt, ...)
> +{
> +	va_list args;
> +
> +	va_start(args, fmt);
> +	lkl_vprintf(fmt, args);
> +	va_end(args);
> +
> +	lkl_host_ops.panic();
> +}
> 

-- 
Anton R. Ivanov
Cambridgegreys Limited. Registered in England. Company Number 10273661
https://www.cambridgegreys.com/

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 18/21] um: host: add utilities functions
  2020-10-07 14:53             ` Anton Ivanov
@ 2020-10-07 15:02               ` Johannes Berg
  -1 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-07 15:02 UTC (permalink / raw)
  To: Anton Ivanov, Hajime Tazaki, linux-um, jdike, richard
  Cc: tavi.purdila, linux-kernel-library, retrage01, linux-arch

On Wed, 2020-10-07 at 15:53 +0100, Anton Ivanov wrote:
> 
> These are actually different on different architectures. These look
> like the x86 values.
> 
> IMHO a kernel strerror() would be the right way of dealing with this
> in the long term (i understand that we cannot call the platform one,
> because it may be different from the internal Linux errors). It will
> be useful in a lot of other places.
> 
> If we leave it as is, we need to make this arch specific at some
> point.
> 
> > +
> > +static const char * const lkl_err_strings[] = {
> > +	"Success",
> > +	"Operation not permitted",

Might be possible to more or less address this (except for arch-specific 
errors that don't always exist) but using C99 initializers?

[0] = "Success",
[EPERM] = "Operation not permitted",
..

johannes


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 18/21] um: host: add utilities functions
@ 2020-10-07 15:02               ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-07 15:02 UTC (permalink / raw)
  To: Anton Ivanov, Hajime Tazaki, linux-um, jdike, richard
  Cc: tavi.purdila, linux-kernel-library, linux-arch, retrage01

On Wed, 2020-10-07 at 15:53 +0100, Anton Ivanov wrote:
> 
> These are actually different on different architectures. These look
> like the x86 values.
> 
> IMHO a kernel strerror() would be the right way of dealing with this
> in the long term (i understand that we cannot call the platform one,
> because it may be different from the internal Linux errors). It will
> be useful in a lot of other places.
> 
> If we leave it as is, we need to make this arch specific at some
> point.
> 
> > +
> > +static const char * const lkl_err_strings[] = {
> > +	"Success",
> > +	"Operation not permitted",

Might be possible to more or less address this (except for arch-specific 
errors that don't always exist) but using C99 initializers?

[0] = "Success",
[EPERM] = "Operation not permitted",
..

johannes


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 18/21] um: host: add utilities functions
  2020-10-07 15:02               ` Johannes Berg
@ 2020-10-07 15:03                 ` Johannes Berg
  -1 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-07 15:03 UTC (permalink / raw)
  To: Anton Ivanov, Hajime Tazaki, linux-um, jdike, richard
  Cc: tavi.purdila, linux-kernel-library, linux-arch, retrage01

On Wed, 2020-10-07 at 17:02 +0200, Johannes Berg wrote:
> On Wed, 2020-10-07 at 15:53 +0100, Anton Ivanov wrote:
> > These are actually different on different architectures. These look
> > like the x86 values.
> > 
> > IMHO a kernel strerror() would be the right way of dealing with this
> > in the long term (i understand that we cannot call the platform one,
> > because it may be different from the internal Linux errors). It will
> > be useful in a lot of other places.
> > 
> > If we leave it as is, we need to make this arch specific at some
> > point.
> > 
> > > +
> > > +static const char * const lkl_err_strings[] = {
> > > +	"Success",
> > > +	"Operation not permitted",
> 
> Might be possible to more or less address this (except for arch-specific 
> errors that don't always exist) but using C99 initializers?
> 
> [0] = "Success",
> [EPERM] = "Operation not permitted",
> ..

But, on the other hand, is it needed at all? I don't think the kernel
ever prints out the actual string ...

johannes


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 18/21] um: host: add utilities functions
@ 2020-10-07 15:03                 ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-07 15:03 UTC (permalink / raw)
  To: Anton Ivanov, Hajime Tazaki, linux-um, jdike, richard
  Cc: tavi.purdila, linux-kernel-library, retrage01, linux-arch

On Wed, 2020-10-07 at 17:02 +0200, Johannes Berg wrote:
> On Wed, 2020-10-07 at 15:53 +0100, Anton Ivanov wrote:
> > These are actually different on different architectures. These look
> > like the x86 values.
> > 
> > IMHO a kernel strerror() would be the right way of dealing with this
> > in the long term (i understand that we cannot call the platform one,
> > because it may be different from the internal Linux errors). It will
> > be useful in a lot of other places.
> > 
> > If we leave it as is, we need to make this arch specific at some
> > point.
> > 
> > > +
> > > +static const char * const lkl_err_strings[] = {
> > > +	"Success",
> > > +	"Operation not permitted",
> 
> Might be possible to more or less address this (except for arch-specific 
> errors that don't always exist) but using C99 initializers?
> 
> [0] = "Success",
> [EPERM] = "Operation not permitted",
> ..

But, on the other hand, is it needed at all? I don't think the kernel
ever prints out the actual string ...

johannes


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 18/21] um: host: add utilities functions
  2020-10-07 15:03                 ` Johannes Berg
@ 2020-10-07 15:10                   ` Anton Ivanov
  -1 siblings, 0 replies; 476+ messages in thread
From: Anton Ivanov @ 2020-10-07 15:10 UTC (permalink / raw)
  To: Johannes Berg, Hajime Tazaki, linux-um, jdike, richard
  Cc: tavi.purdila, linux-kernel-library, retrage01, linux-arch


On 07/10/2020 16:03, Johannes Berg wrote:
> On Wed, 2020-10-07 at 17:02 +0200, Johannes Berg wrote:
>> On Wed, 2020-10-07 at 15:53 +0100, Anton Ivanov wrote:
>>> These are actually different on different architectures. These look
>>> like the x86 values.
>>>
>>> IMHO a kernel strerror() would be the right way of dealing with this
>>> in the long term (i understand that we cannot call the platform one,
>>> because it may be different from the internal Linux errors). It will
>>> be useful in a lot of other places.
>>>
>>> If we leave it as is, we need to make this arch specific at some
>>> point.
>>>
>>>> +
>>>> +static const char * const lkl_err_strings[] = {
>>>> +	"Success",
>>>> +	"Operation not permitted",
>> Might be possible to more or less address this (except for arch-specific
>> errors that don't always exist) but using C99 initializers?
>>
>> [0] = "Success",
>> [EPERM] = "Operation not permitted",
>> ..
> But, on the other hand, is it needed at all? I don't think the kernel
> ever prints out the actual string ...

I can see the use case for a library in a multi-arch environment (which IMHO is the intended use case). It saves the user the effort of digging into the build and figuring out what does this error mean today :)

It is nice to have :)

If we will have it, however, it should be done as you suggested - C99 or some other way where it maps correctly to actual underlying error codes as they may end up being different depending on build config.

> johannes
>
>
> _______________________________________________
> linux-um mailing list
> linux-um@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-um
>
-- 
Anton R. Ivanov
Cambridgegreys Limited. Registered in England. Company Number 10273661
https://www.cambridgegreys.com/


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 18/21] um: host: add utilities functions
@ 2020-10-07 15:10                   ` Anton Ivanov
  0 siblings, 0 replies; 476+ messages in thread
From: Anton Ivanov @ 2020-10-07 15:10 UTC (permalink / raw)
  To: Johannes Berg, Hajime Tazaki, linux-um, jdike, richard
  Cc: tavi.purdila, linux-kernel-library, linux-arch, retrage01


On 07/10/2020 16:03, Johannes Berg wrote:
> On Wed, 2020-10-07 at 17:02 +0200, Johannes Berg wrote:
>> On Wed, 2020-10-07 at 15:53 +0100, Anton Ivanov wrote:
>>> These are actually different on different architectures. These look
>>> like the x86 values.
>>>
>>> IMHO a kernel strerror() would be the right way of dealing with this
>>> in the long term (i understand that we cannot call the platform one,
>>> because it may be different from the internal Linux errors). It will
>>> be useful in a lot of other places.
>>>
>>> If we leave it as is, we need to make this arch specific at some
>>> point.
>>>
>>>> +
>>>> +static const char * const lkl_err_strings[] = {
>>>> +	"Success",
>>>> +	"Operation not permitted",
>> Might be possible to more or less address this (except for arch-specific
>> errors that don't always exist) but using C99 initializers?
>>
>> [0] = "Success",
>> [EPERM] = "Operation not permitted",
>> ..
> But, on the other hand, is it needed at all? I don't think the kernel
> ever prints out the actual string ...

I can see the use case for a library in a multi-arch environment (which IMHO is the intended use case). It saves the user the effort of digging into the build and figuring out what does this error mean today :)

It is nice to have :)

If we will have it, however, it should be done as you suggested - C99 or some other way where it maps correctly to actual underlying error codes as they may end up being different depending on build config.

> johannes
>
>
> _______________________________________________
> linux-um mailing list
> linux-um@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-um
>
-- 
Anton R. Ivanov
Cambridgegreys Limited. Registered in England. Company Number 10273661
https://www.cambridgegreys.com/


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 02/21] um: add os init and exit calls
  2020-10-06  9:44           ` Hajime Tazaki
@ 2020-10-07 15:13             ` Johannes Berg
  -1 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-07 15:13 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Octavian Purdila,
	retrage01

On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> 
> -#define __define_initcall(level,fn) \
> -	static initcall_t __initcall_##fn __used \
> -	__attribute__((__section__(".initcall" level ".init"))) = fn
> -
> -/* Userspace initcalls shouldn't depend on anything in the kernel, so we'll
> - * make them run first.
> - */
> -#define __initcall(fn) __define_initcall("1", fn)
> +#undef __uml_exit_call
> +#define __uml_exit_call		__used __section(os_exitcalls)

Doesn't that break calling of sigio_cleanup and remove_umid_dir?

After all,

> +void __weak os_exitcalls(void)
> +{
> +}

This does nothing so far.

Also, why the __weak?

johannes


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 02/21] um: add os init and exit calls
@ 2020-10-07 15:13             ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-07 15:13 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, retrage01, Octavian Purdila,
	linux-arch

On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> 
> -#define __define_initcall(level,fn) \
> -	static initcall_t __initcall_##fn __used \
> -	__attribute__((__section__(".initcall" level ".init"))) = fn
> -
> -/* Userspace initcalls shouldn't depend on anything in the kernel, so we'll
> - * make them run first.
> - */
> -#define __initcall(fn) __define_initcall("1", fn)
> +#undef __uml_exit_call
> +#define __uml_exit_call		__used __section(os_exitcalls)

Doesn't that break calling of sigio_cleanup and remove_umid_dir?

After all,

> +void __weak os_exitcalls(void)
> +{
> +}

This does nothing so far.

Also, why the __weak?

johannes


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 03/21] um: move arch/um/os-Linux dir to tools/um/uml
  2020-10-06  9:44           ` Hajime Tazaki
@ 2020-10-07 15:20             ` Johannes Berg
  -1 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-07 15:20 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, retrage01

On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> This patch moves underlying OS dependent part under arch/um to tools/um
> directory so that arch/um code does not need to build host build
> facilities (e.g., libc).

Hmm. On the one hand, that build separation seems sensible, but on the
other hand this stuff *does* fundamentally belong to arch/um/, and it's
not a "tool" like basically everything else there that is a pure
userspace application to run *inside* the kernel, not *part of* it.

For that reason, I don't really like this much.


>  tools/um/uml/Build                            | 48 +++++++++++++++++++
>  tools/um/uml/drivers/Build                    | 10 ++++

Also, what's with the names here? What's wrong with "Makefile"?


I'm also not sure I see how this is built at all from the top level
Makefile? Oh. I think Anton said it doesn't ... that alone would be a
reason not to do it I guess.


So why do you think it must be under tools/? Even if you consider "lkl"
a "tool", that doesn't mean it must be there. I also consider a UML
binary ("linux") a "tool" in my simulation environment, etc.


And even LKL, which is the eventual goal here - why would you consider
that a "tool"? I don't think we should.

johannes


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 03/21] um: move arch/um/os-Linux dir to tools/um/uml
@ 2020-10-07 15:20             ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-07 15:20 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, retrage01, linux-arch

On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> This patch moves underlying OS dependent part under arch/um to tools/um
> directory so that arch/um code does not need to build host build
> facilities (e.g., libc).

Hmm. On the one hand, that build separation seems sensible, but on the
other hand this stuff *does* fundamentally belong to arch/um/, and it's
not a "tool" like basically everything else there that is a pure
userspace application to run *inside* the kernel, not *part of* it.

For that reason, I don't really like this much.


>  tools/um/uml/Build                            | 48 +++++++++++++++++++
>  tools/um/uml/drivers/Build                    | 10 ++++

Also, what's with the names here? What's wrong with "Makefile"?


I'm also not sure I see how this is built at all from the top level
Makefile? Oh. I think Anton said it doesn't ... that alone would be a
reason not to do it I guess.


So why do you think it must be under tools/? Even if you consider "lkl"
a "tool", that doesn't mean it must be there. I also consider a UML
binary ("linux") a "tool" in my simulation environment, etc.


And even LKL, which is the eventual goal here - why would you consider
that a "tool"? I don't think we should.

johannes


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 04/21] um: host: implement os_initcalls and os_exitcalls
  2020-10-06  9:44           ` Hajime Tazaki
@ 2020-10-07 15:22             ` Johannes Berg
  -1 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-07 15:22 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, Octavian Purdila,
	retrage01

On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:

> Note that this patch should be merged with "um: move arch/um/os-Linux
> dir to tools/um" but for now it is separate to make the review easier.

Not a fan of that, I must say ...

> +extern void (*__start_os_exitcalls)(void);
> +extern void (*__stop_os_exitcalls)(void);
> +
> +void os_exitcalls(void)
> +{
> +	exitcall_t *call;
> +
> +	call = &__stop_os_exitcalls;
> +	while (--call >= &__start_os_exitcalls)
> +		(*call)();

You should check for and skip NULL pointers, there always are alignment
issues with automatic section filling like this, more so with clang than
gcc.

> +}
> +
> +extern int (*__start_os_initcalls)(void);
> +extern int (*__stop_os_initcalls)(void);
> +
> +int os_initcalls(void)
> +{
> +	initcall_t *call;
> +
> +	call = &__stop_os_initcalls;
> +	while (--call >= &__start_os_initcalls)
> +		(*call)();

Same here.

johannes


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 04/21] um: host: implement os_initcalls and os_exitcalls
@ 2020-10-07 15:22             ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-07 15:22 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, retrage01, Octavian Purdila,
	linux-arch

On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:

> Note that this patch should be merged with "um: move arch/um/os-Linux
> dir to tools/um" but for now it is separate to make the review easier.

Not a fan of that, I must say ...

> +extern void (*__start_os_exitcalls)(void);
> +extern void (*__stop_os_exitcalls)(void);
> +
> +void os_exitcalls(void)
> +{
> +	exitcall_t *call;
> +
> +	call = &__stop_os_exitcalls;
> +	while (--call >= &__start_os_exitcalls)
> +		(*call)();

You should check for and skip NULL pointers, there always are alignment
issues with automatic section filling like this, more so with clang than
gcc.

> +}
> +
> +extern int (*__start_os_initcalls)(void);
> +extern int (*__stop_os_initcalls)(void);
> +
> +int os_initcalls(void)
> +{
> +	initcall_t *call;
> +
> +	call = &__stop_os_initcalls;
> +	while (--call >= &__start_os_initcalls)
> +		(*call)();

Same here.

johannes


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 05/21] um: move arch/x86/um/os-Linux to tools/um/uml/
  2020-10-06  9:44           ` Hajime Tazaki
@ 2020-10-07 15:23             ` Johannes Berg
  -1 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-07 15:23 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, retrage01

On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> Similar to arch/um/os-Linux code, arch/x86 also contains OS dependent
> part as well.  This patch moves underlying OS dependent part to tools/um
> directory so that arch/x86 code does not need to build host build
> facilities (e.g., libc).

Again, I don't agree with the rationale. This isn't a tool in the kernel
sense, and separating only by "what does it link against" doesn't make
much sense either.

johannes



^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 05/21] um: move arch/x86/um/os-Linux to tools/um/uml/
@ 2020-10-07 15:23             ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-07 15:23 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, retrage01, linux-arch

On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> Similar to arch/um/os-Linux code, arch/x86 also contains OS dependent
> part as well.  This patch moves underlying OS dependent part to tools/um
> directory so that arch/x86 code does not need to build host build
> facilities (e.g., libc).

Again, I don't agree with the rationale. This isn't a tool in the kernel
sense, and separating only by "what does it link against" doesn't make
much sense either.

johannes



_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 06/21] scritps: um: suppress warnings if SRCARCH=um
  2020-10-06  9:44           ` Hajime Tazaki
@ 2020-10-07 15:24             ` Johannes Berg
  -1 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-07 15:24 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, retrage01

On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> This commit fixes numbers of warning messages about leaked CONFIG
> options.

No it doesn't ... it suppresses them, not a fan at all.

> nommu mode of UML requires copies of kernel headers to offer
> syscall-like API for the library users.  Thus, the warnings are to be
> avoided to function this exposure of API.

I don't see that here, so maybe make that part of the patch where it
actually happens, with a big comment, and maybe only when actually
building for NOMMU/LKL?

johannes



^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 06/21] scritps: um: suppress warnings if SRCARCH=um
@ 2020-10-07 15:24             ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-07 15:24 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, retrage01, linux-arch

On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> This commit fixes numbers of warning messages about leaked CONFIG
> options.

No it doesn't ... it suppresses them, not a fan at all.

> nommu mode of UML requires copies of kernel headers to offer
> syscall-like API for the library users.  Thus, the warnings are to be
> avoided to function this exposure of API.

I don't see that here, so maybe make that part of the patch where it
actually happens, with a big comment, and maybe only when actually
building for NOMMU/LKL?

johannes



_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 07/21] um: extend arch_switch_to for alternate SUBARCH
  2020-10-06  9:44           ` Hajime Tazaki
@ 2020-10-07 15:25             ` Johannes Berg
  -1 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-07 15:25 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, retrage01

On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> This commit introduces additional argument of previous task when
> context switch happens.  New SUBARCH can use the new information to
> switch tasks in a subarch-specific manner.
> 
> The patch is particularly required by nommu mode implemented as a
> SUBARCH of UML.

Would probably be good to already say why here?

johannes


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 07/21] um: extend arch_switch_to for alternate SUBARCH
@ 2020-10-07 15:25             ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-07 15:25 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, retrage01, linux-arch

On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> This commit introduces additional argument of previous task when
> context switch happens.  New SUBARCH can use the new information to
> switch tasks in a subarch-specific manner.
> 
> The patch is particularly required by nommu mode implemented as a
> SUBARCH of UML.

Would probably be good to already say why here?

johannes


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 08/21] um: add nommu mode for UML library mode
  2020-10-06  9:44           ` Hajime Tazaki
@ 2020-10-07 15:44             ` Johannes Berg
  -1 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-07 15:44 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, retrage01

On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> This patch introduces the nommu operation with UML code so that host
> interface can be shrineked for broader environments support.

shrunk

> The nommu mode is implemneted as SUBARCH of arch/um, which places at

implemented

> arch/um/nommu. This SUBARCH defines mode-specific code of memory
> management as well as thread implementation, along with the uapi headers
> to be exported to users.
> 
> The headers we introduce in this patch are simple wrappers to the
> asm-generic headers or stubs for things we don't support, such as
> ptrace, DMA, signals, ELF handling and low level processor operations.
> 
> nommu mode shares most of arch/um code (irq, thread_info, process
> scheduling) but implements its own facilities, which are memory
> management (nommu), thread primitives (struct arch_thread), system call
> interface, and console.
> 
> The outlook of updated directory structure is as follows:
> 
>    arch/um
>    |-- configs         (untouched)
>    |-- drivers         unuse stdio_console.c for !MMU
>    |-- include
>    |   |-- asm         updated with !CONFIG_MMU
>    |   |-- linux       (untouched)
>    |   `-- shared      updated with new functions
>    |       `-- skas    (untouched)
>    |   `-- uapi        added for upai header installation
>    |-- kernel          updated to integrate with !MMU
>    |   `-- skas        (untouched, don't use for !MMU)
>    |-- nommu           SUBARCH dir (internally =um/nommu)
>    |   |-- include
>    |   |   |-- asm     headers for subarch specific info
>    |   |   `-- uapi    headers for user-visible APIs
>    |   `-- um
>    |       `-- shared  headers for subarch specific info
>    `-- scripts         added a script for header installation

That seems awkward. Might be nice now for "as little changes as
possible", but eventually it seems it would be better to separate that
out into arch/um/{common,nommu,mmu}/ or something like that?

Or perhaps something like

	arch/um/{common,lkl,full}

or something? Not sure I like 'full' though. 'vm'? Hmm.

(also, scripts/ doesn't exist in this patch?)

> +++ b/arch/um/nommu/Makefile
> @@ -0,0 +1 @@
> +#

hmm? maybe add a comment why an empty makefile is needed.

> diff --git a/arch/um/nommu/Makefile.um b/arch/um/nommu/Makefile.um
> new file mode 100644
> index 000000000000..3808462d8283
> --- /dev/null
> +++ b/arch/um/nommu/Makefile.um

[...]

> +ifeq ($(shell uname -s), Linux)
> +NPROC=$(shell nproc)
> +else # e.g., FreeBSD
> +NPROC=$(shell sysctl -n hw.ncpu)
> +endif

That seems very inappropriate here.

> +
> +um_headers_install: $(objtree)/$(HOST_DIR)/include/generated/uapi/asm/syscall_defs.h headers
> +# if syscall_defs.h is newer than generated headers (autoconf.h)
> +	$(Q)if [ ! -r $(objtree)/tools/um/include/lkl/autoconf.h ] || \
> +	  [ $< -nt $(objtree)/tools/um/include/lkl/autoconf.h ]; then \
> +		$(srctree)/$(ARCH_DIR)/scripts/headers_install.py \
> +			$(subst -j,-j$(NPROC),$(findstring -j,$(MAKEFLAGS))) \

Yeah ... just don't.

> +++ b/arch/um/nommu/include/asm/atomic.h
> @@ -0,0 +1,11 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef __UM_NOMMU_ATOMIC_H
> +#define __UM_NOMMU_ATOMIC_H
> +
> +#include <asm-generic/atomic.h>
> +
> +#ifndef CONFIG_GENERIC_ATOMIC64
> +#include "atomic64.h"
> +#endif /* !CONFIG_GENERIC_ATOMIC64 */
> +
> +#endif
> diff --git a/arch/um/nommu/include/asm/atomic64.h b/arch/um/nommu/include/asm/atomic64.h
> new file mode 100644
> index 000000000000..949360dea7af
> --- /dev/null
> +++ b/arch/um/nommu/include/asm/atomic64.h

That doesn't make sense to me, you can control CONFIG_GENERIC_ATOMIC64
to be on, and don't need the ifdef and this file?

> diff --git a/arch/um/nommu/include/asm/bitsperlong.h b/arch/um/nommu/include/asm/bitsperlong.h
> new file mode 100644
> index 000000000000..a150cee41e75
> --- /dev/null
> +++ b/arch/um/nommu/include/asm/bitsperlong.h
> @@ -0,0 +1,12 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef __UM_NOMMU_BITSPERLONG_H
> +#define __UM_NOMMU_BITSPERLONG_H
> +
> +#include <uapi/asm/bitsperlong.h>
> +
> +#define BITS_PER_LONG __BITS_PER_LONG
> +
> +#define BITS_PER_LONG_LONG 64
> +
> +#endif

That's very similar to the contents of include/asm-
generic/bitsperlong.h, add comments why it's needed?

> diff --git a/arch/um/nommu/include/asm/byteorder.h b/arch/um/nommu/include/asm/byteorder.h
> new file mode 100644
> index 000000000000..920a5fd26cad
> --- /dev/null
> +++ b/arch/um/nommu/include/asm/byteorder.h
> @@ -0,0 +1,7 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef __UM_NOMMU_BYTEORDER_H
> +#define __UM_NOMMU_BYTEORDER_H
> +
> +#include <uapi/asm/byteorder.h>
> +
> +#endif

Not sure why this file even exists, it doesn't for any other arch
(except for h8300 which seems odd).

> diff --git a/arch/um/nommu/include/asm/elf.h b/arch/um/nommu/include/asm/elf.h
> new file mode 100644
> index 000000000000..edcf63edeed1
> --- /dev/null
> +++ b/arch/um/nommu/include/asm/elf.h
> @@ -0,0 +1,15 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef __UM_NOMMU_ELF_H
> +#define __UM_NOMMU_ELF_H
> +
> +#define elf_check_arch(x) 0

Please add a comment ...

> +#ifdef CONFIG_64BIT
> +#define ELF_CLASS ELFCLASS64
> +#else
> +#define ELF_CLASS ELFCLASS32
> +#endif
> +
> +#define elf_gregset_t long
> +#define elf_fpregset_t double

Also here.

> +#ifndef __UM_NOMMU_SCHED_H
> +#define __UM_NOMMU_SCHED_H
> +
> +#include <linux/sched.h>
> +#include <uapi/asm/host_ops.h>
> +
> +static inline void thread_sched_jb(void)
> +{
> +	if (test_ti_thread_flag(current_thread_info(), TIF_HOST_THREAD)) {
> +		set_ti_thread_flag(current_thread_info(), TIF_SCHED_JB);
> +		set_current_state(TASK_UNINTERRUPTIBLE);
> +		lkl_ops->jmp_buf_set(&current_thread_info()->task->thread.arch.sched_jb,
> +				     schedule);
> +	} else {
> +		lkl_bug("%s can be used only for host task", __func__);
> +	}
> +}

What's "thread_sched_jb"? Doesn't really seem to exist? Neither does
"TIF_SCHED_JB", at least in my kernel?

Looks like that's added by a later patch, should probably then add this
file also only later so it's actually consistent and one can review all
the pieces together.

> +++ b/arch/um/nommu/include/uapi/asm/Kbuild
> @@ -0,0 +1,6 @@
> +# UAPI Header export list
> +
> +generated-y += syscall_defs.h
> +
> +# no generated-y since we need special user headers handling
> +# see arch/um/script/headers_install.py

Also doesn't exist yet.

> +++ b/arch/um/nommu/um/delay.c

why the double um/ path?

> +/* skas/process.c */
> +void halt_skas(void)
> +{}

nit: missing blank line after this

> +int is_skas_winch(int pid, int fd, void *data)
> +{
> +	return 0;
> +}
> +void reboot_skas(void)
> +{}
> +
> +int __init start_uml(void)
> +{
> +	return 0;
> +}

That seems odd?

Might be better to have an own os.h, or split that into os-uml.h and
os-lkl.h or so?

johannes


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 08/21] um: add nommu mode for UML library mode
@ 2020-10-07 15:44             ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-07 15:44 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, retrage01, linux-arch

On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> This patch introduces the nommu operation with UML code so that host
> interface can be shrineked for broader environments support.

shrunk

> The nommu mode is implemneted as SUBARCH of arch/um, which places at

implemented

> arch/um/nommu. This SUBARCH defines mode-specific code of memory
> management as well as thread implementation, along with the uapi headers
> to be exported to users.
> 
> The headers we introduce in this patch are simple wrappers to the
> asm-generic headers or stubs for things we don't support, such as
> ptrace, DMA, signals, ELF handling and low level processor operations.
> 
> nommu mode shares most of arch/um code (irq, thread_info, process
> scheduling) but implements its own facilities, which are memory
> management (nommu), thread primitives (struct arch_thread), system call
> interface, and console.
> 
> The outlook of updated directory structure is as follows:
> 
>    arch/um
>    |-- configs         (untouched)
>    |-- drivers         unuse stdio_console.c for !MMU
>    |-- include
>    |   |-- asm         updated with !CONFIG_MMU
>    |   |-- linux       (untouched)
>    |   `-- shared      updated with new functions
>    |       `-- skas    (untouched)
>    |   `-- uapi        added for upai header installation
>    |-- kernel          updated to integrate with !MMU
>    |   `-- skas        (untouched, don't use for !MMU)
>    |-- nommu           SUBARCH dir (internally =um/nommu)
>    |   |-- include
>    |   |   |-- asm     headers for subarch specific info
>    |   |   `-- uapi    headers for user-visible APIs
>    |   `-- um
>    |       `-- shared  headers for subarch specific info
>    `-- scripts         added a script for header installation

That seems awkward. Might be nice now for "as little changes as
possible", but eventually it seems it would be better to separate that
out into arch/um/{common,nommu,mmu}/ or something like that?

Or perhaps something like

	arch/um/{common,lkl,full}

or something? Not sure I like 'full' though. 'vm'? Hmm.

(also, scripts/ doesn't exist in this patch?)

> +++ b/arch/um/nommu/Makefile
> @@ -0,0 +1 @@
> +#

hmm? maybe add a comment why an empty makefile is needed.

> diff --git a/arch/um/nommu/Makefile.um b/arch/um/nommu/Makefile.um
> new file mode 100644
> index 000000000000..3808462d8283
> --- /dev/null
> +++ b/arch/um/nommu/Makefile.um

[...]

> +ifeq ($(shell uname -s), Linux)
> +NPROC=$(shell nproc)
> +else # e.g., FreeBSD
> +NPROC=$(shell sysctl -n hw.ncpu)
> +endif

That seems very inappropriate here.

> +
> +um_headers_install: $(objtree)/$(HOST_DIR)/include/generated/uapi/asm/syscall_defs.h headers
> +# if syscall_defs.h is newer than generated headers (autoconf.h)
> +	$(Q)if [ ! -r $(objtree)/tools/um/include/lkl/autoconf.h ] || \
> +	  [ $< -nt $(objtree)/tools/um/include/lkl/autoconf.h ]; then \
> +		$(srctree)/$(ARCH_DIR)/scripts/headers_install.py \
> +			$(subst -j,-j$(NPROC),$(findstring -j,$(MAKEFLAGS))) \

Yeah ... just don't.

> +++ b/arch/um/nommu/include/asm/atomic.h
> @@ -0,0 +1,11 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef __UM_NOMMU_ATOMIC_H
> +#define __UM_NOMMU_ATOMIC_H
> +
> +#include <asm-generic/atomic.h>
> +
> +#ifndef CONFIG_GENERIC_ATOMIC64
> +#include "atomic64.h"
> +#endif /* !CONFIG_GENERIC_ATOMIC64 */
> +
> +#endif
> diff --git a/arch/um/nommu/include/asm/atomic64.h b/arch/um/nommu/include/asm/atomic64.h
> new file mode 100644
> index 000000000000..949360dea7af
> --- /dev/null
> +++ b/arch/um/nommu/include/asm/atomic64.h

That doesn't make sense to me, you can control CONFIG_GENERIC_ATOMIC64
to be on, and don't need the ifdef and this file?

> diff --git a/arch/um/nommu/include/asm/bitsperlong.h b/arch/um/nommu/include/asm/bitsperlong.h
> new file mode 100644
> index 000000000000..a150cee41e75
> --- /dev/null
> +++ b/arch/um/nommu/include/asm/bitsperlong.h
> @@ -0,0 +1,12 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef __UM_NOMMU_BITSPERLONG_H
> +#define __UM_NOMMU_BITSPERLONG_H
> +
> +#include <uapi/asm/bitsperlong.h>
> +
> +#define BITS_PER_LONG __BITS_PER_LONG
> +
> +#define BITS_PER_LONG_LONG 64
> +
> +#endif

That's very similar to the contents of include/asm-
generic/bitsperlong.h, add comments why it's needed?

> diff --git a/arch/um/nommu/include/asm/byteorder.h b/arch/um/nommu/include/asm/byteorder.h
> new file mode 100644
> index 000000000000..920a5fd26cad
> --- /dev/null
> +++ b/arch/um/nommu/include/asm/byteorder.h
> @@ -0,0 +1,7 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef __UM_NOMMU_BYTEORDER_H
> +#define __UM_NOMMU_BYTEORDER_H
> +
> +#include <uapi/asm/byteorder.h>
> +
> +#endif

Not sure why this file even exists, it doesn't for any other arch
(except for h8300 which seems odd).

> diff --git a/arch/um/nommu/include/asm/elf.h b/arch/um/nommu/include/asm/elf.h
> new file mode 100644
> index 000000000000..edcf63edeed1
> --- /dev/null
> +++ b/arch/um/nommu/include/asm/elf.h
> @@ -0,0 +1,15 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef __UM_NOMMU_ELF_H
> +#define __UM_NOMMU_ELF_H
> +
> +#define elf_check_arch(x) 0

Please add a comment ...

> +#ifdef CONFIG_64BIT
> +#define ELF_CLASS ELFCLASS64
> +#else
> +#define ELF_CLASS ELFCLASS32
> +#endif
> +
> +#define elf_gregset_t long
> +#define elf_fpregset_t double

Also here.

> +#ifndef __UM_NOMMU_SCHED_H
> +#define __UM_NOMMU_SCHED_H
> +
> +#include <linux/sched.h>
> +#include <uapi/asm/host_ops.h>
> +
> +static inline void thread_sched_jb(void)
> +{
> +	if (test_ti_thread_flag(current_thread_info(), TIF_HOST_THREAD)) {
> +		set_ti_thread_flag(current_thread_info(), TIF_SCHED_JB);
> +		set_current_state(TASK_UNINTERRUPTIBLE);
> +		lkl_ops->jmp_buf_set(&current_thread_info()->task->thread.arch.sched_jb,
> +				     schedule);
> +	} else {
> +		lkl_bug("%s can be used only for host task", __func__);
> +	}
> +}

What's "thread_sched_jb"? Doesn't really seem to exist? Neither does
"TIF_SCHED_JB", at least in my kernel?

Looks like that's added by a later patch, should probably then add this
file also only later so it's actually consistent and one can review all
the pieces together.

> +++ b/arch/um/nommu/include/uapi/asm/Kbuild
> @@ -0,0 +1,6 @@
> +# UAPI Header export list
> +
> +generated-y += syscall_defs.h
> +
> +# no generated-y since we need special user headers handling
> +# see arch/um/script/headers_install.py

Also doesn't exist yet.

> +++ b/arch/um/nommu/um/delay.c

why the double um/ path?

> +/* skas/process.c */
> +void halt_skas(void)
> +{}

nit: missing blank line after this

> +int is_skas_winch(int pid, int fd, void *data)
> +{
> +	return 0;
> +}
> +void reboot_skas(void)
> +{}
> +
> +int __init start_uml(void)
> +{
> +	return 0;
> +}

That seems odd?

Might be better to have an own os.h, or split that into os-uml.h and
os-lkl.h or so?

johannes


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 09/21] um: nommu: host interface
  2020-10-06  9:44           ` Hajime Tazaki
@ 2020-10-07 15:45             ` Johannes Berg
  -1 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-07 15:45 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, retrage01

On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> This patch introduces the host operations that define the interface
> between the LKL and the host. These operations must be provided either
> by a host library or by the application itself.
> 

I kinda find patches like this very pointless ... you see nothing, and
can't really say anything. Maybe add the infra with the first user?

> +/* Defined in {posix,nt}-host.c */

That doesn't exist either yet ...

johannes


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 09/21] um: nommu: host interface
@ 2020-10-07 15:45             ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-07 15:45 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, retrage01, linux-arch

On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> This patch introduces the host operations that define the interface
> between the LKL and the host. These operations must be provided either
> by a host library or by the application itself.
> 

I kinda find patches like this very pointless ... you see nothing, and
can't really say anything. Maybe add the infra with the first user?

> +/* Defined in {posix,nt}-host.c */

That doesn't exist either yet ...

johannes


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 10/21] um: nommu: memory handling
  2020-10-06  9:44           ` Hajime Tazaki
@ 2020-10-07 15:47             ` Johannes Berg
  -1 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-07 15:47 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, retrage01

On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> 
>   * These operations must be provided by a host library or by the application
>   * itself.
>   *
> + * @mem_alloc - allocate memory
> + * @mem_free - free memory
> + *

Actual kernel-doc would be nicer.

> +	empty_zero_page = memblock_alloc(PAGE_SIZE, PAGE_SIZE);
> +	memset((void *)empty_zero_page, 0, PAGE_SIZE);
> +
> +	{
> +		unsigned long zones_size[MAX_NR_ZONES] = {0, };

Hmm, what's with the extra scope?

johannes


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 10/21] um: nommu: memory handling
@ 2020-10-07 15:47             ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-07 15:47 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, retrage01, linux-arch

On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> 
>   * These operations must be provided by a host library or by the application
>   * itself.
>   *
> + * @mem_alloc - allocate memory
> + * @mem_free - free memory
> + *

Actual kernel-doc would be nicer.

> +	empty_zero_page = memblock_alloc(PAGE_SIZE, PAGE_SIZE);
> +	memset((void *)empty_zero_page, 0, PAGE_SIZE);
> +
> +	{
> +		unsigned long zones_size[MAX_NR_ZONES] = {0, };

Hmm, what's with the extra scope?

johannes


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 11/21] um: nommu: kernel thread support
  2020-10-06  9:44           ` Hajime Tazaki
@ 2020-10-07 18:57             ` Johannes Berg
  -1 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-07 18:57 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, retrage01

On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> nommu mode does not support user processes

I find this really confusing. I'm not sure why you ended up calling this
"nommu mode", but there *are* (still) (other) nommu arches, and they
*do* support userspace processes.

Isn't this really just "LKL mode" or something like that?

>  #define TIF_SYSCALL_TRACE	0	/* syscall trace active */
> @@ -63,6 +85,8 @@ static inline struct thread_info *current_thread_info(void)
>  #define TIF_RESTORE_SIGMASK	7
>  #define TIF_NOTIFY_RESUME	8
>  #define TIF_SECCOMP		9	/* secure computing */
> +#define TIF_SCHED_JB		10
> +#define TIF_HOST_THREAD		11

It'd be nice to document what those mean, and even what "JB" means ... I
saw something about "jump buffer" somewhere, but I have no idea why that
should be a thread flag.

> @@ -16,11 +16,65 @@ struct lkl_jmp_buf {
>   * These operations must be provided by a host library or by the application
>   * itself.
>   *
> + * @sem_alloc - allocate a host semaphore an initialize it to count
> + * @sem_free - free a host semaphore
> + * @sem_up - perform an up operation on the semaphore
> + * @sem_down - perform a down operation on the semaphore
> + *
> + * @mutex_alloc - allocate and initialize a host mutex; the recursive parameter
> + * determines if the mutex is recursive or not
> + * @mutex_free - free a host mutex
> + * @mutex_lock - acquire the mutex
> + * @mutex_unlock - release the mutex
> + *
> + * @thread_create - create a new thread and run f(arg) in its context; returns a
> + * thread handle or 0 if the thread could not be created
> + * @thread_detach - on POSIX systems, free up resources held by
> + * pthreads. Noop on Win32.
> + * @thread_exit - terminates the current thread
> + * @thread_join - wait for the given thread to terminate. Returns 0
> + * for success, -1 otherwise
> + *
> + * @gettid - returns the host thread id of the caller, which need not
> + * be the same as the handle returned by thread_create
> + *
> + * @jmp_buf_set - runs the give function and setups a jump back point by saving
> + * the context in the jump buffer; jmp_buf_longjmp can be called from the give
> + * function or any callee in that function to return back to the jump back
> + * point
> + *
> + * NOTE: we can't return from jmp_buf_set before calling jmp_buf_longjmp or
> + * otherwise the saved context (stack) is not going to be valid, so we must pass
> + * the function that will eventually call longjmp here
> + *
> + * @jmp_buf_longjmp - perform a jump back to the saved jump buffer
> + *
>   * @mem_alloc - allocate memory
>   * @mem_free - free memory

again, kernel-doc.

But I'm starting to doubt the value of having this struct at all. Care
you explain? You're doing everything else already with weak functions,
and you can't very well have _two_ hosts compiled anyway, so what's the
point?

IOW, why isn't this just

void lkl_sem_free(struct lkl_sem *sem);
void lkl_sem_up(struct lkl_sem *sem);
...

and then posix-host.c just includes the header file and implements those
functions?

I don't see any reason for this to be allowed to have multiple variants
linked and then picking them at runtime?

> +/*
> + * This structure is used to get access to the "LKL CPU" that allows us to run

Are you trying to implement SMP? This seems ... rather complex?

> + * Linux code. Because we have to deal with various synchronization requirements
> + * between idle thread, system calls, interrupts, "reentrancy", CPU shutdown,
> + * imbalance wake up (i.e. acquire the CPU from one thread and release it from
> + * another), we can't use a simple synchronization mechanism such as (recursive)
> + * mutex or semaphore. Instead, we use a mutex and a bunch of status data plus a
> + * semaphore.
> + */
> +static struct lkl_cpu {
> +	/* lock that protects the CPU status data */
> +	struct lkl_mutex *lock;
> +	/*
> +	 * Since we must free the cpu lock during shutdown we need a
> +	 * synchronization algorithm between lkl_cpu_shutdown() and the CPU
> +	 * access functions since lkl_cpu_get() gets called from thread
> +	 * destructor callback functions which may be scheduled after
> +	 * lkl_cpu_shutdown() has freed the cpu lock.
> +	 *
> +	 * An atomic counter is used to keep track of the number of running
> +	 * CPU access functions and allow the shutdown function to wait for
> +	 * them.
> +	 *
> +	 * The shutdown functions adds MAX_THREADS to this counter which allows
> +	 * the CPU access functions to check if the shutdown process has
> +	 * started.
> +	 *
> +	 * This algorithm assumes that we never have more the MAX_THREADS
> +	 * requesting CPU access.
> +	 */
> +	#define MAX_THREADS 1000000
> +	unsigned int shutdown_gate;
> +	bool irqs_pending;
> +	/* no of threads waiting the CPU */
> +	unsigned int sleepers;
> +	/* no of times the current thread got the CPU */
> +	unsigned int count;
> +	/* current thread that owns the CPU */
> +	lkl_thread_t owner;
> +	/* semaphore for threads waiting the CPU */
> +	struct lkl_sem *sem;
> +	/* semaphore used for shutdown */
> +	struct lkl_sem *shutdown_sem;
> +} cpu;
> +
> +static int __cpu_try_get_lock(int n)
> +{
> +	lkl_thread_t self;
> +
> +	if (__sync_fetch_and_add(&cpu.shutdown_gate, n) >= MAX_THREADS)
> +		return -2;
> +
> +	lkl_ops->mutex_lock(cpu.lock);
> +
> +	if (cpu.shutdown_gate >= MAX_THREADS)
> +		return -1;
> +
> +	self = lkl_ops->thread_self();
> +
> +	if (cpu.owner && !lkl_ops->thread_equal(cpu.owner, self))
> +		return 0;
> +
> +	cpu.owner = self;
> +	cpu.count++;
> +
> +	return 1;
> +}
> +
> +static void __cpu_try_get_unlock(int lock_ret, int n)
> +{
> +	if (lock_ret >= -1)
> +		lkl_ops->mutex_unlock(cpu.lock);
> +	__sync_fetch_and_sub(&cpu.shutdown_gate, n);
> +}
> +
> +void lkl_cpu_change_owner(lkl_thread_t owner)
> +{
> +	lkl_ops->mutex_lock(cpu.lock);
> +	if (cpu.count > 1)
> +		lkl_bug("bad count while changing owner\n");
> +	cpu.owner = owner;
> +	lkl_ops->mutex_unlock(cpu.lock);
> +}
> +
> +int lkl_cpu_get(void)
> +{
> +	int ret;
> +
> +	ret = __cpu_try_get_lock(1);
> +
> +	while (ret == 0) {
> +		cpu.sleepers++;
> +		__cpu_try_get_unlock(ret, 0);
> +		lkl_ops->sem_down(cpu.sem);
> +		ret = __cpu_try_get_lock(0);
> +	}
> +
> +	__cpu_try_get_unlock(ret, 1);
> +
> +	return ret;
> +}
> +
> +void lkl_cpu_put(void)
> +{
> +	lkl_ops->mutex_lock(cpu.lock);
> +
> +	if (!cpu.count || !cpu.owner ||
> +	    !lkl_ops->thread_equal(cpu.owner, lkl_ops->thread_self()))
> +		lkl_bug("%s: unbalanced put\n", __func__);
> +
> +	while (cpu.irqs_pending && !irqs_disabled()) {
> +		cpu.irqs_pending = false;
> +		lkl_ops->mutex_unlock(cpu.lock);
> +		run_irqs();
> +		lkl_ops->mutex_lock(cpu.lock);
> +	}
> +
> +	if (test_ti_thread_flag(current_thread_info(), TIF_HOST_THREAD) &&
> +	    !single_task_running() && cpu.count == 1) {
> +		if (in_interrupt())
> +			lkl_bug("%s: in interrupt\n", __func__);
> +		lkl_ops->mutex_unlock(cpu.lock);
> +		thread_sched_jb();
> +		return;
> +	}
> +
> +	if (--cpu.count > 0) {
> +		lkl_ops->mutex_unlock(cpu.lock);
> +		return;
> +	}
> +
> +	if (cpu.sleepers) {
> +		cpu.sleepers--;
> +		lkl_ops->sem_up(cpu.sem);
> +	}
> +
> +	cpu.owner = 0;
> +
> +	lkl_ops->mutex_unlock(cpu.lock);
> +}
> +
> +int lkl_cpu_try_run_irq(int irq)
> +{
> +	int ret;
> +
> +	ret = __cpu_try_get_lock(1);
> +	if (!ret) {
> +		set_irq_pending(irq);
> +		cpu.irqs_pending = true;
> +	}
> +	__cpu_try_get_unlock(ret, 1);
> +
> +	return ret;
> +}
> +
> +static void lkl_cpu_shutdown(void)
> +{
> +	__sync_fetch_and_add(&cpu.shutdown_gate, MAX_THREADS);
> +}
> +__uml_exitcall(lkl_cpu_shutdown);
> +
> +void lkl_cpu_wait_shutdown(void)
> +{
> +	lkl_ops->sem_down(cpu.shutdown_sem);
> +	lkl_ops->sem_free(cpu.shutdown_sem);
> +}
> +
> +static void lkl_cpu_cleanup(bool shutdown)
> +{
> +	while (__sync_fetch_and_add(&cpu.shutdown_gate, 0) > MAX_THREADS)
> +		;
> +
> +	if (shutdown)
> +		lkl_ops->sem_up(cpu.shutdown_sem);
> +	else if (cpu.shutdown_sem)
> +		lkl_ops->sem_free(cpu.shutdown_sem);
> +	if (cpu.sem)
> +		lkl_ops->sem_free(cpu.sem);
> +	if (cpu.lock)
> +		lkl_ops->mutex_free(cpu.lock);
> +}

Yeah, what? That's an incomprehensible piece of code. At least add
comments, if it _really_ is necessary?

> +#ifdef doesntwork
> +	/* switch to idle_host_task */
> +	wakeup_idle_host_task();
> +#endif

Well ...

> +/**
> + * This is called before the kernel initializes, so no kernel calls (including
> + * printk) can't be made yet.
> + */

not kernel-doc

try to compile with W=1 :)

johannes


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 11/21] um: nommu: kernel thread support
@ 2020-10-07 18:57             ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-07 18:57 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, retrage01, linux-arch

On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> nommu mode does not support user processes

I find this really confusing. I'm not sure why you ended up calling this
"nommu mode", but there *are* (still) (other) nommu arches, and they
*do* support userspace processes.

Isn't this really just "LKL mode" or something like that?

>  #define TIF_SYSCALL_TRACE	0	/* syscall trace active */
> @@ -63,6 +85,8 @@ static inline struct thread_info *current_thread_info(void)
>  #define TIF_RESTORE_SIGMASK	7
>  #define TIF_NOTIFY_RESUME	8
>  #define TIF_SECCOMP		9	/* secure computing */
> +#define TIF_SCHED_JB		10
> +#define TIF_HOST_THREAD		11

It'd be nice to document what those mean, and even what "JB" means ... I
saw something about "jump buffer" somewhere, but I have no idea why that
should be a thread flag.

> @@ -16,11 +16,65 @@ struct lkl_jmp_buf {
>   * These operations must be provided by a host library or by the application
>   * itself.
>   *
> + * @sem_alloc - allocate a host semaphore an initialize it to count
> + * @sem_free - free a host semaphore
> + * @sem_up - perform an up operation on the semaphore
> + * @sem_down - perform a down operation on the semaphore
> + *
> + * @mutex_alloc - allocate and initialize a host mutex; the recursive parameter
> + * determines if the mutex is recursive or not
> + * @mutex_free - free a host mutex
> + * @mutex_lock - acquire the mutex
> + * @mutex_unlock - release the mutex
> + *
> + * @thread_create - create a new thread and run f(arg) in its context; returns a
> + * thread handle or 0 if the thread could not be created
> + * @thread_detach - on POSIX systems, free up resources held by
> + * pthreads. Noop on Win32.
> + * @thread_exit - terminates the current thread
> + * @thread_join - wait for the given thread to terminate. Returns 0
> + * for success, -1 otherwise
> + *
> + * @gettid - returns the host thread id of the caller, which need not
> + * be the same as the handle returned by thread_create
> + *
> + * @jmp_buf_set - runs the give function and setups a jump back point by saving
> + * the context in the jump buffer; jmp_buf_longjmp can be called from the give
> + * function or any callee in that function to return back to the jump back
> + * point
> + *
> + * NOTE: we can't return from jmp_buf_set before calling jmp_buf_longjmp or
> + * otherwise the saved context (stack) is not going to be valid, so we must pass
> + * the function that will eventually call longjmp here
> + *
> + * @jmp_buf_longjmp - perform a jump back to the saved jump buffer
> + *
>   * @mem_alloc - allocate memory
>   * @mem_free - free memory

again, kernel-doc.

But I'm starting to doubt the value of having this struct at all. Care
you explain? You're doing everything else already with weak functions,
and you can't very well have _two_ hosts compiled anyway, so what's the
point?

IOW, why isn't this just

void lkl_sem_free(struct lkl_sem *sem);
void lkl_sem_up(struct lkl_sem *sem);
...

and then posix-host.c just includes the header file and implements those
functions?

I don't see any reason for this to be allowed to have multiple variants
linked and then picking them at runtime?

> +/*
> + * This structure is used to get access to the "LKL CPU" that allows us to run

Are you trying to implement SMP? This seems ... rather complex?

> + * Linux code. Because we have to deal with various synchronization requirements
> + * between idle thread, system calls, interrupts, "reentrancy", CPU shutdown,
> + * imbalance wake up (i.e. acquire the CPU from one thread and release it from
> + * another), we can't use a simple synchronization mechanism such as (recursive)
> + * mutex or semaphore. Instead, we use a mutex and a bunch of status data plus a
> + * semaphore.
> + */
> +static struct lkl_cpu {
> +	/* lock that protects the CPU status data */
> +	struct lkl_mutex *lock;
> +	/*
> +	 * Since we must free the cpu lock during shutdown we need a
> +	 * synchronization algorithm between lkl_cpu_shutdown() and the CPU
> +	 * access functions since lkl_cpu_get() gets called from thread
> +	 * destructor callback functions which may be scheduled after
> +	 * lkl_cpu_shutdown() has freed the cpu lock.
> +	 *
> +	 * An atomic counter is used to keep track of the number of running
> +	 * CPU access functions and allow the shutdown function to wait for
> +	 * them.
> +	 *
> +	 * The shutdown functions adds MAX_THREADS to this counter which allows
> +	 * the CPU access functions to check if the shutdown process has
> +	 * started.
> +	 *
> +	 * This algorithm assumes that we never have more the MAX_THREADS
> +	 * requesting CPU access.
> +	 */
> +	#define MAX_THREADS 1000000
> +	unsigned int shutdown_gate;
> +	bool irqs_pending;
> +	/* no of threads waiting the CPU */
> +	unsigned int sleepers;
> +	/* no of times the current thread got the CPU */
> +	unsigned int count;
> +	/* current thread that owns the CPU */
> +	lkl_thread_t owner;
> +	/* semaphore for threads waiting the CPU */
> +	struct lkl_sem *sem;
> +	/* semaphore used for shutdown */
> +	struct lkl_sem *shutdown_sem;
> +} cpu;
> +
> +static int __cpu_try_get_lock(int n)
> +{
> +	lkl_thread_t self;
> +
> +	if (__sync_fetch_and_add(&cpu.shutdown_gate, n) >= MAX_THREADS)
> +		return -2;
> +
> +	lkl_ops->mutex_lock(cpu.lock);
> +
> +	if (cpu.shutdown_gate >= MAX_THREADS)
> +		return -1;
> +
> +	self = lkl_ops->thread_self();
> +
> +	if (cpu.owner && !lkl_ops->thread_equal(cpu.owner, self))
> +		return 0;
> +
> +	cpu.owner = self;
> +	cpu.count++;
> +
> +	return 1;
> +}
> +
> +static void __cpu_try_get_unlock(int lock_ret, int n)
> +{
> +	if (lock_ret >= -1)
> +		lkl_ops->mutex_unlock(cpu.lock);
> +	__sync_fetch_and_sub(&cpu.shutdown_gate, n);
> +}
> +
> +void lkl_cpu_change_owner(lkl_thread_t owner)
> +{
> +	lkl_ops->mutex_lock(cpu.lock);
> +	if (cpu.count > 1)
> +		lkl_bug("bad count while changing owner\n");
> +	cpu.owner = owner;
> +	lkl_ops->mutex_unlock(cpu.lock);
> +}
> +
> +int lkl_cpu_get(void)
> +{
> +	int ret;
> +
> +	ret = __cpu_try_get_lock(1);
> +
> +	while (ret == 0) {
> +		cpu.sleepers++;
> +		__cpu_try_get_unlock(ret, 0);
> +		lkl_ops->sem_down(cpu.sem);
> +		ret = __cpu_try_get_lock(0);
> +	}
> +
> +	__cpu_try_get_unlock(ret, 1);
> +
> +	return ret;
> +}
> +
> +void lkl_cpu_put(void)
> +{
> +	lkl_ops->mutex_lock(cpu.lock);
> +
> +	if (!cpu.count || !cpu.owner ||
> +	    !lkl_ops->thread_equal(cpu.owner, lkl_ops->thread_self()))
> +		lkl_bug("%s: unbalanced put\n", __func__);
> +
> +	while (cpu.irqs_pending && !irqs_disabled()) {
> +		cpu.irqs_pending = false;
> +		lkl_ops->mutex_unlock(cpu.lock);
> +		run_irqs();
> +		lkl_ops->mutex_lock(cpu.lock);
> +	}
> +
> +	if (test_ti_thread_flag(current_thread_info(), TIF_HOST_THREAD) &&
> +	    !single_task_running() && cpu.count == 1) {
> +		if (in_interrupt())
> +			lkl_bug("%s: in interrupt\n", __func__);
> +		lkl_ops->mutex_unlock(cpu.lock);
> +		thread_sched_jb();
> +		return;
> +	}
> +
> +	if (--cpu.count > 0) {
> +		lkl_ops->mutex_unlock(cpu.lock);
> +		return;
> +	}
> +
> +	if (cpu.sleepers) {
> +		cpu.sleepers--;
> +		lkl_ops->sem_up(cpu.sem);
> +	}
> +
> +	cpu.owner = 0;
> +
> +	lkl_ops->mutex_unlock(cpu.lock);
> +}
> +
> +int lkl_cpu_try_run_irq(int irq)
> +{
> +	int ret;
> +
> +	ret = __cpu_try_get_lock(1);
> +	if (!ret) {
> +		set_irq_pending(irq);
> +		cpu.irqs_pending = true;
> +	}
> +	__cpu_try_get_unlock(ret, 1);
> +
> +	return ret;
> +}
> +
> +static void lkl_cpu_shutdown(void)
> +{
> +	__sync_fetch_and_add(&cpu.shutdown_gate, MAX_THREADS);
> +}
> +__uml_exitcall(lkl_cpu_shutdown);
> +
> +void lkl_cpu_wait_shutdown(void)
> +{
> +	lkl_ops->sem_down(cpu.shutdown_sem);
> +	lkl_ops->sem_free(cpu.shutdown_sem);
> +}
> +
> +static void lkl_cpu_cleanup(bool shutdown)
> +{
> +	while (__sync_fetch_and_add(&cpu.shutdown_gate, 0) > MAX_THREADS)
> +		;
> +
> +	if (shutdown)
> +		lkl_ops->sem_up(cpu.shutdown_sem);
> +	else if (cpu.shutdown_sem)
> +		lkl_ops->sem_free(cpu.shutdown_sem);
> +	if (cpu.sem)
> +		lkl_ops->sem_free(cpu.sem);
> +	if (cpu.lock)
> +		lkl_ops->mutex_free(cpu.lock);
> +}

Yeah, what? That's an incomprehensible piece of code. At least add
comments, if it _really_ is necessary?

> +#ifdef doesntwork
> +	/* switch to idle_host_task */
> +	wakeup_idle_host_task();
> +#endif

Well ...

> +/**
> + * This is called before the kernel initializes, so no kernel calls (including
> + * printk) can't be made yet.
> + */

not kernel-doc

try to compile with W=1 :)

johannes


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 12/21] um: nommu: system call interface and application API
  2020-10-06  9:44           ` Hajime Tazaki
@ 2020-10-07 19:05             ` Johannes Berg
  -1 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-07 19:05 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, retrage01

On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> 
> +++ b/arch/um/nommu/include/uapi/asm/syscalls.h
> @@ -0,0 +1,287 @@
> +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */

That doesn't really make sense - if you use LKL you're linking Linux, so
you can't use this "WITH Linux-syscall-note"?

> +#ifndef __UM_NOMMU_UAPI_SYSCALLS_H
> +#define __UM_NOMMU_UAPI_SYSCALLS_H

[snip]

This file really worries me, it includes half the world and (re)defines
the other half ... How is this ever going to be maintained?

> index 000000000000..ec7356c0dee9
> --- /dev/null
> +++ b/arch/um/scripts/headers_install.py
> @@ -0,0 +1,197 @@
> +#!/usr/bin/env python

might want to make that explicitly 'python3', some newer distros (e.g.
ubuntu 20.04) are now shipping without a 'python' by default.

> +def has_lkl_prefix(w):
> +  return w.startswith("lkl") or w.startswith("_lkl") or w.startswith("__lkl") \
> +         or w.startswith("LKL") or w.startswith("_LKL") or w.startswith("__LKL")


> +        content = re.sub(re.compile("(\/\*(\*(?!\/)|[^*])*\*\/)", re.S|re.M), " ", open(h).read())
> 
> +    dir = os.path.dirname(h)
> +    out_dir = args.path + "/" + re.sub("(" + srctree + "/arch/um/nommu/include/uapi/|arch/um/nommu/include/generated/uapi/|include/generated/uapi/|include/generated|" + install_hdr_path + "/include/)(.*)", "lkl/\\2", dir)


you have some very long lines in places, I'm sure you could fix that
(e.g. the last one by doing something with '|'.join([...]))

johannes


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 12/21] um: nommu: system call interface and application API
@ 2020-10-07 19:05             ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-07 19:05 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, retrage01, linux-arch

On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> 
> +++ b/arch/um/nommu/include/uapi/asm/syscalls.h
> @@ -0,0 +1,287 @@
> +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */

That doesn't really make sense - if you use LKL you're linking Linux, so
you can't use this "WITH Linux-syscall-note"?

> +#ifndef __UM_NOMMU_UAPI_SYSCALLS_H
> +#define __UM_NOMMU_UAPI_SYSCALLS_H

[snip]

This file really worries me, it includes half the world and (re)defines
the other half ... How is this ever going to be maintained?

> index 000000000000..ec7356c0dee9
> --- /dev/null
> +++ b/arch/um/scripts/headers_install.py
> @@ -0,0 +1,197 @@
> +#!/usr/bin/env python

might want to make that explicitly 'python3', some newer distros (e.g.
ubuntu 20.04) are now shipping without a 'python' by default.

> +def has_lkl_prefix(w):
> +  return w.startswith("lkl") or w.startswith("_lkl") or w.startswith("__lkl") \
> +         or w.startswith("LKL") or w.startswith("_LKL") or w.startswith("__LKL")


> +        content = re.sub(re.compile("(\/\*(\*(?!\/)|[^*])*\*\/)", re.S|re.M), " ", open(h).read())
> 
> +    dir = os.path.dirname(h)
> +    out_dir = args.path + "/" + re.sub("(" + srctree + "/arch/um/nommu/include/uapi/|arch/um/nommu/include/generated/uapi/|include/generated/uapi/|include/generated|" + install_hdr_path + "/include/)(.*)", "lkl/\\2", dir)


you have some very long lines in places, I'm sure you could fix that
(e.g. the last one by doing something with '|'.join([...]))

johannes


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 15/21] um: nommu: integrate with irq infrastructure of UML
  2020-10-06  9:44           ` Hajime Tazaki
@ 2020-10-07 19:09             ` Johannes Berg
  -1 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-07 19:09 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, retrage01


> @@ -541,6 +542,11 @@ unsigned long to_irq_stack(unsigned long *mask_out)
>  	unsigned long mask, old;
>  	int nested;
>  
> +#ifndef CONFIG_MMU

Also here and throughout, not sure I like (and it makes sense) to have
the equation of !CONFIG_MMU == LKL.

I mean, sure, that may actually be _true_ (at least right now), but it
doesn't feel like all of this really is a consequence of *NOMMU*, it's
more a consequence of LKL.

At least I think it is?

johannes


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 15/21] um: nommu: integrate with irq infrastructure of UML
@ 2020-10-07 19:09             ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-07 19:09 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, retrage01, linux-arch


> @@ -541,6 +542,11 @@ unsigned long to_irq_stack(unsigned long *mask_out)
>  	unsigned long mask, old;
>  	int nested;
>  
> +#ifndef CONFIG_MMU

Also here and throughout, not sure I like (and it makes sense) to have
the equation of !CONFIG_MMU == LKL.

I mean, sure, that may actually be _true_ (at least right now), but it
doesn't feel like all of this really is a consequence of *NOMMU*, it's
more a consequence of LKL.

At least I think it is?

johannes


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 16/21] um: nommu: plug in the build system
  2020-10-06  9:44           ` Hajime Tazaki
@ 2020-10-07 19:20             ` Johannes Berg
  -1 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-07 19:20 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, retrage01

On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> Basic Makefiles for building library of nommu mode. Add a new architecture
> specific target for installing the resulting library files and headers.
> 
> To make nommu binaries build, UML introduced an additional option, UMMODE
> variable, to switch the output file of build: kernel (default), or
> library (nommu).  Those modes are not able to be ON at the same time.

Here you did that again :)

I think you should really be up-front about it and call it "kernel" and
"library" mode. Then do the ifdefs on library mode etc.

Sure, kernel mode will be CONFIG_MMU=y and library mode will be
!CONFIG_MMU instead, but it really makes more sense to call things
CONFIG_UMMODE_LIB in most cases, I think?

Yes, a lot of stuff *does* depend on CONFIG_MMU, and that still makes
sense. It just feels like a lot of stuff you made depend on CONFIG_MMU
here (or !CONFIG_MMU) really isn't related too much to that, rather than
being related to "library mode".

>   make defconfig ARCH=um UMMODE=library
>   make ARCH=um UMMODE=library

Why not make it a real Kconfig symbol? I find the ARCH=um tricky enough
to remember all the time, so much I usually write a "GNUmakefile" with

export ARCH=um
include Makefile

:-)

> +config UMMODE_LIB
> +	bool "UML mode: library mode"
> +	default y if "$(UMMODE)" = "library"

So wait, you _can_ switch that through Kconfig then, because you made it
a visible option (string after "bool"). But it won't work, because then
you later in the build system etc. still check UMMODE instead of
CONFIG_UMMODE_LIB. Seems like something that ought to be fixed one way
or the other - at the very least hide this symbol if setting it manually
is invalid.

> +	help
> +	 This mode switches a mode to build a library of UML (Linux
> +	 Kernel Library/LKL).  This switch is exclusive to "kernel mode"
> +	 of UML, which is traditional mode of UML.

Not sure if that historically made more sense, but you don't have any
UMMODE_KERNEL option or something like that, so the help text seems
confusing?


> +	 For more detail about LKL, see
> +	 <file:Documentation/virt/uml/lkl.txt>.
> +
>  config MMU
>  	bool
> -	default y
> +	default y if !UMMODE_LIB
>  
>  config NO_IOMEM
>  	def_bool y
> @@ -45,12 +56,12 @@ config LOCKDEP_SUPPORT
>  
>  config STACKTRACE_SUPPORT
>  	bool
> -	default y
> +	default y if MMU

Same as what I said above - there really shouldn't be any inherent
reason why STACKTRACE_SUPPORT depends on CONFIG_MMU, is there? Apart
from not having implemented it in UMMODE_LIB, that is, and then you
should make this if !UMMODE_LIB?

>  config GENERIC_CALIBRATE_DELAY
>  	bool
> -	default y
> +	default y if MMU

Same here. This one _surely_ has no relation to MMU.

> +ifeq ($(UMMODE),library)
> +  SUBARCH := um/nommu
> +endif

>  INSTALL_PATH=$(objtree)/tools/um
> +ifeq ($(UMMODE),library)

Here a few places using UMMODE which must come from the command line.

>  linux.o: vmlinux
>  	@echo '  LINK	$@'
> -	$(Q)$(OBJCOPY) -R .eh_frame $< $@
> +	$(Q)$(OBJCOPY) -R .eh_frame -L sem_init -L sem_post -L sem_wait -L sem_destroy $< $@

Care to explain?

>  	$(Q)mkdir -p $(objtree)/tools/um/lib
>  
>  define archhelp
> diff --git a/arch/um/kernel/Makefile b/arch/um/kernel/Makefile
> index 9b63831a69e1..01605ed439cb 100644
> --- a/arch/um/kernel/Makefile
> +++ b/arch/um/kernel/Makefile
> @@ -14,17 +14,21 @@ CPPFLAGS_vmlinux.lds := -DSTART=$(LDS_START)		\
>  			$(LDS_EXTRA)
>  extra-y := vmlinux.lds
>  
> -obj-y = config.o exec.o exitcode.o irq.o ksyms.o mem.o \
> -	physmem.o process.o ptrace.o reboot.o sigio.o \
> -	signal.o syscall.o sysrq.o time.o tlb.o trap.o \
> -	um_arch.o umid.o maccess.o kmsg_dump.o skas/
> +obj-y = config.o exitcode.o irq.o ksyms.o \
> +	process.o reboot.o sigio.o \
> +	signal.o syscall.o time.o \
> +	um_arch.o umid.o maccess.o kmsg_dump.o
> +
> +ifdef CONFIG_MMU
> +obj-y += exec.o mem.o physmem.o ptrace.o sysrq.o tlb.o trap.o skas/
> +endif
>  
>  obj-$(CONFIG_BLK_DEV_INITRD) += initrd.o
>  obj-$(CONFIG_GPROF)	+= gprof_syms.o
>  obj-$(CONFIG_GCOV)	+= gmon_syms.o
>  obj-$(CONFIG_EARLY_PRINTK) += early_printk.o
>  obj-$(CONFIG_STACKTRACE) += stacktrace.o
> -obj-y += user_syms.o
> +obj-$(CONFIG_MMU) += user_syms.o
>  
>  USER_OBJS := config.o
>  
> diff --git a/arch/um/nommu/um/Kconfig b/arch/um/nommu/um/Kconfig
> new file mode 100644
> index 000000000000..20b3eaccb6f0
> --- /dev/null
> +++ b/arch/um/nommu/um/Kconfig
> @@ -0,0 +1,37 @@
> +config UML_NOMMU
> +	def_bool y
> +	depends on !SMP && !MMU

again, using MMU to mean "LKL" (both UML_NOMMU and !MMU places)

> +	select UACCESS_MEMCPY
> +	select ARCH_THREAD_STACK_ALLOCATOR
> +	select ARCH_HAS_SYSCALL_WRAPPER

You never use this except for the selects, maybe can go elsewhere?

> +config 64BIT
> +	bool
> +	default y
> +
> +config GENERIC_CSUM
> +	def_bool y
> +
> +config GENERIC_ATOMIC64
> +	bool
> +	default y if !64BIT
> +
> +config SECCOMP
> +	bool
> +	default n
> +
> +config GENERIC_HWEIGHT
> +	def_bool y
> +
> +config GENERIC_CALIBRATE_DELAY
> +	bool
> +	default n
> +
> +config STACKTRACE_SUPPORT
> +	bool
> +	default n

You ... were just changing these elsewhere, so one of that isn't needed?

> +# XXX: need this to work well with tap13.py

What's tap13.py??

johannes


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 16/21] um: nommu: plug in the build system
@ 2020-10-07 19:20             ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-07 19:20 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, retrage01, linux-arch

On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> Basic Makefiles for building library of nommu mode. Add a new architecture
> specific target for installing the resulting library files and headers.
> 
> To make nommu binaries build, UML introduced an additional option, UMMODE
> variable, to switch the output file of build: kernel (default), or
> library (nommu).  Those modes are not able to be ON at the same time.

Here you did that again :)

I think you should really be up-front about it and call it "kernel" and
"library" mode. Then do the ifdefs on library mode etc.

Sure, kernel mode will be CONFIG_MMU=y and library mode will be
!CONFIG_MMU instead, but it really makes more sense to call things
CONFIG_UMMODE_LIB in most cases, I think?

Yes, a lot of stuff *does* depend on CONFIG_MMU, and that still makes
sense. It just feels like a lot of stuff you made depend on CONFIG_MMU
here (or !CONFIG_MMU) really isn't related too much to that, rather than
being related to "library mode".

>   make defconfig ARCH=um UMMODE=library
>   make ARCH=um UMMODE=library

Why not make it a real Kconfig symbol? I find the ARCH=um tricky enough
to remember all the time, so much I usually write a "GNUmakefile" with

export ARCH=um
include Makefile

:-)

> +config UMMODE_LIB
> +	bool "UML mode: library mode"
> +	default y if "$(UMMODE)" = "library"

So wait, you _can_ switch that through Kconfig then, because you made it
a visible option (string after "bool"). But it won't work, because then
you later in the build system etc. still check UMMODE instead of
CONFIG_UMMODE_LIB. Seems like something that ought to be fixed one way
or the other - at the very least hide this symbol if setting it manually
is invalid.

> +	help
> +	 This mode switches a mode to build a library of UML (Linux
> +	 Kernel Library/LKL).  This switch is exclusive to "kernel mode"
> +	 of UML, which is traditional mode of UML.

Not sure if that historically made more sense, but you don't have any
UMMODE_KERNEL option or something like that, so the help text seems
confusing?


> +	 For more detail about LKL, see
> +	 <file:Documentation/virt/uml/lkl.txt>.
> +
>  config MMU
>  	bool
> -	default y
> +	default y if !UMMODE_LIB
>  
>  config NO_IOMEM
>  	def_bool y
> @@ -45,12 +56,12 @@ config LOCKDEP_SUPPORT
>  
>  config STACKTRACE_SUPPORT
>  	bool
> -	default y
> +	default y if MMU

Same as what I said above - there really shouldn't be any inherent
reason why STACKTRACE_SUPPORT depends on CONFIG_MMU, is there? Apart
from not having implemented it in UMMODE_LIB, that is, and then you
should make this if !UMMODE_LIB?

>  config GENERIC_CALIBRATE_DELAY
>  	bool
> -	default y
> +	default y if MMU

Same here. This one _surely_ has no relation to MMU.

> +ifeq ($(UMMODE),library)
> +  SUBARCH := um/nommu
> +endif

>  INSTALL_PATH=$(objtree)/tools/um
> +ifeq ($(UMMODE),library)

Here a few places using UMMODE which must come from the command line.

>  linux.o: vmlinux
>  	@echo '  LINK	$@'
> -	$(Q)$(OBJCOPY) -R .eh_frame $< $@
> +	$(Q)$(OBJCOPY) -R .eh_frame -L sem_init -L sem_post -L sem_wait -L sem_destroy $< $@

Care to explain?

>  	$(Q)mkdir -p $(objtree)/tools/um/lib
>  
>  define archhelp
> diff --git a/arch/um/kernel/Makefile b/arch/um/kernel/Makefile
> index 9b63831a69e1..01605ed439cb 100644
> --- a/arch/um/kernel/Makefile
> +++ b/arch/um/kernel/Makefile
> @@ -14,17 +14,21 @@ CPPFLAGS_vmlinux.lds := -DSTART=$(LDS_START)		\
>  			$(LDS_EXTRA)
>  extra-y := vmlinux.lds
>  
> -obj-y = config.o exec.o exitcode.o irq.o ksyms.o mem.o \
> -	physmem.o process.o ptrace.o reboot.o sigio.o \
> -	signal.o syscall.o sysrq.o time.o tlb.o trap.o \
> -	um_arch.o umid.o maccess.o kmsg_dump.o skas/
> +obj-y = config.o exitcode.o irq.o ksyms.o \
> +	process.o reboot.o sigio.o \
> +	signal.o syscall.o time.o \
> +	um_arch.o umid.o maccess.o kmsg_dump.o
> +
> +ifdef CONFIG_MMU
> +obj-y += exec.o mem.o physmem.o ptrace.o sysrq.o tlb.o trap.o skas/
> +endif
>  
>  obj-$(CONFIG_BLK_DEV_INITRD) += initrd.o
>  obj-$(CONFIG_GPROF)	+= gprof_syms.o
>  obj-$(CONFIG_GCOV)	+= gmon_syms.o
>  obj-$(CONFIG_EARLY_PRINTK) += early_printk.o
>  obj-$(CONFIG_STACKTRACE) += stacktrace.o
> -obj-y += user_syms.o
> +obj-$(CONFIG_MMU) += user_syms.o
>  
>  USER_OBJS := config.o
>  
> diff --git a/arch/um/nommu/um/Kconfig b/arch/um/nommu/um/Kconfig
> new file mode 100644
> index 000000000000..20b3eaccb6f0
> --- /dev/null
> +++ b/arch/um/nommu/um/Kconfig
> @@ -0,0 +1,37 @@
> +config UML_NOMMU
> +	def_bool y
> +	depends on !SMP && !MMU

again, using MMU to mean "LKL" (both UML_NOMMU and !MMU places)

> +	select UACCESS_MEMCPY
> +	select ARCH_THREAD_STACK_ALLOCATOR
> +	select ARCH_HAS_SYSCALL_WRAPPER

You never use this except for the selects, maybe can go elsewhere?

> +config 64BIT
> +	bool
> +	default y
> +
> +config GENERIC_CSUM
> +	def_bool y
> +
> +config GENERIC_ATOMIC64
> +	bool
> +	default y if !64BIT
> +
> +config SECCOMP
> +	bool
> +	default n
> +
> +config GENERIC_HWEIGHT
> +	def_bool y
> +
> +config GENERIC_CALIBRATE_DELAY
> +	bool
> +	default n
> +
> +config STACKTRACE_SUPPORT
> +	bool
> +	default n

You ... were just changing these elsewhere, so one of that isn't needed?

> +# XXX: need this to work well with tap13.py

What's tap13.py??

johannes


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 20/21] um: host: add test programs
  2020-10-06  9:44           ` Hajime Tazaki
@ 2020-10-07 19:23             ` Johannes Berg
  -1 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-07 19:23 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, retrage01

On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> 
> +LKL_TEST_CALL(getpid, lkl_sys_getpid, 1)

Could that be unified with KUNIT somehow?

> +++ b/tools/um/tests/run.py
> @@ -0,0 +1,172 @@
> +#!/usr/bin/env python
> +# SPDX-License-Identifier: GPL-2.0
> +#
> +# This program is free software; you can redistribute it and/or modify
> +# it under the terms of the GNU General Public License as published by
> +# the Free Software Foundation; version 2 of the License
> +#
> +# Author: Octavian Purdila <tavi@cs.pub.ro>
> +#
> +
> +from __future__ import print_function
> +
> +import argparse
> +import os
> +import subprocess
> +import sys
> +import tap13

ok, I see now, you're doing something with TAP (test anything
protocol)...

Hmm. I must say I'm not a fan of adding a whole testing framework to the
kernel like that, even if it's pretty simple.

> +import xml.etree.ElementTree as ET
> +
> +from junit_xml import TestSuite, TestCase

yuck

Let's see if you can use KUNIT instead. Anything beyond that doesn't
really need to live in the kernel, IMHO.

johanens


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 20/21] um: host: add test programs
@ 2020-10-07 19:23             ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-07 19:23 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, retrage01, linux-arch

On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> 
> +LKL_TEST_CALL(getpid, lkl_sys_getpid, 1)

Could that be unified with KUNIT somehow?

> +++ b/tools/um/tests/run.py
> @@ -0,0 +1,172 @@
> +#!/usr/bin/env python
> +# SPDX-License-Identifier: GPL-2.0
> +#
> +# This program is free software; you can redistribute it and/or modify
> +# it under the terms of the GNU General Public License as published by
> +# the Free Software Foundation; version 2 of the License
> +#
> +# Author: Octavian Purdila <tavi@cs.pub.ro>
> +#
> +
> +from __future__ import print_function
> +
> +import argparse
> +import os
> +import subprocess
> +import sys
> +import tap13

ok, I see now, you're doing something with TAP (test anything
protocol)...

Hmm. I must say I'm not a fan of adding a whole testing framework to the
kernel like that, even if it's pretty simple.

> +import xml.etree.ElementTree as ET
> +
> +from junit_xml import TestSuite, TestCase

yuck

Let's see if you can use KUNIT instead. Anything beyond that doesn't
really need to live in the kernel, IMHO.

johanens


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 00/21] Unifying LKL into UML
  2020-10-07 13:30           ` Anton Ivanov
  (?)
@ 2020-10-08 12:12           ` Hajime Tazaki
  2020-10-08 12:50               ` Anton Ivanov
  -1 siblings, 1 reply; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-08 12:12 UTC (permalink / raw)
  To: anton.ivanov
  Cc: linux-um, jdike, richard, tavi.purdila, linux-kernel-library,
	linux-arch, retrage01


Hello Anton,

On Wed, 07 Oct 2020 22:30:03 +0900,
Anton Ivanov wrote:
> 
> 
> On 06/10/2020 10:44, Hajime Tazaki wrote:
> > This is another spin of the unification of LKL into UML.  Based on the
> > discussion of v4 patchset, we have tried to address issue raised and
> > rewrote the patchset from scratch.  The summary is listed in the
> > changelog below.
> > 
> > Although there are still bugs in the patchset, we'd like to ask your
> > opinions on the design we changed.
> > 
> > The milestone section is also updated: this patchset is for the
> > milestone 1, though the common init API is still not implemented yet.
> > 
> > 
> > Changes in rfc v7:
> > - preserve `make ARCH=um` syntax to build UML
> > - introduce `make ARCH=um UMMODE=library` to build library mode
> > - fix undefined symbols issue during modpost
> > - clean up makefiles (arch/um, tools/um)
> 
> Hi Hajime, hi Tavi,
> 
> Our starting point should be that it does not break the existing build. It still does.

I agree with the starting point.

> If I build a "stock configuration" UML after applying the patchset
> the resulting vmlinux is not executable.

Ah, I confirmed the issue.
I was only trying to make the `linux` binary compatible, not vmlinux.

Because vmlinux is now build as a relocatable object, this is
something we need to figure out if we wish to keep vmlinux executable.

Do you think we should make vmlinux executable even if we have the
file linux executable ? If yes, we will work on this to fix the issue.

> On the positive side, it builds cleanly now. I will try to go
> through the rest of the patchset later today and see if there is
> anything else that needs fixing before we do the next version.

thanks for your time.

-- Hajime


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 21/21] um: nommu: add block device support of UML
  2020-10-07 14:17             ` Anton Ivanov
  (?)
@ 2020-10-08 12:13             ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-08 12:13 UTC (permalink / raw)
  To: anton.ivanov
  Cc: linux-um, jdike, richard, tavi.purdila, linux-kernel-library,
	linux-arch, retrage01


On Wed, 07 Oct 2020 23:17:31 +0900,
Anton Ivanov wrote:

> > diff --git a/arch/um/nommu/um/Kconfig b/arch/um/nommu/um/Kconfig
> > index 20b3eaccb6f0..c6a3f472fe75 100644
> > --- a/arch/um/nommu/um/Kconfig
> > +++ b/arch/um/nommu/um/Kconfig
> > @@ -4,6 +4,10 @@ config UML_NOMMU
> >   	select UACCESS_MEMCPY
> >   	select ARCH_THREAD_STACK_ALLOCATOR
> >   	select ARCH_HAS_SYSCALL_WRAPPER
> > +	select VFAT_FS
> > +	select NLS_CODEPAGE_437
> > +	select NLS_ISO8859_1
> > +	select BTRFS_FS
> >     config 64BIT
> >   	bool
> > @@ -35,3 +39,7 @@ config STACKTRACE_SUPPORT
> >   config PRINTK_TIME
> >   	bool
> >   	default y
> > +
> > +config RAID6_PQ_BENCHMARK
> > +	bool
> > +	default n
> 
> Why are we touching this? I thought this is already defined in lib/Kconfig?

With the scheduler which LKL implements, it has the same issue with
what time-travel=inf-cpu has when CONFIG_BTRFS_FS is enable.  I tried
to follow the way of the commit d65197a (below), but if I added
"depends on !RAID6_PQ_BENCHMARK" to config UMMODE_LIB, I got "error:
recursive dependency detected!".

https://github.com/thehajime/linux/commit/d65197ad52494bed3b5e64708281b8295f76c391#diff-c170aa964ad412630a2b5addf306ff14

Thus, to avoid this situation, I did the above additional config
RAID6_PQ_BENCHMARK just for UMMODE_LIB.

I plan to rework on Kconfig (UMMODE_LIB and !MMU) and will figure out
more appropriate way.

-- Hajime


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 00/21] Unifying LKL into UML
  2020-10-08 12:12           ` Hajime Tazaki
@ 2020-10-08 12:50               ` Anton Ivanov
  0 siblings, 0 replies; 476+ messages in thread
From: Anton Ivanov @ 2020-10-08 12:50 UTC (permalink / raw)
  To: Hajime Tazaki
  Cc: linux-um, jdike, richard, tavi.purdila, linux-kernel-library,
	linux-arch, retrage01


On 08/10/2020 13:12, Hajime Tazaki wrote:
> Hello Anton,
>
> On Wed, 07 Oct 2020 22:30:03 +0900,
> Anton Ivanov wrote:
>>
>> On 06/10/2020 10:44, Hajime Tazaki wrote:
>>> This is another spin of the unification of LKL into UML.  Based on the
>>> discussion of v4 patchset, we have tried to address issue raised and
>>> rewrote the patchset from scratch.  The summary is listed in the
>>> changelog below.
>>>
>>> Although there are still bugs in the patchset, we'd like to ask your
>>> opinions on the design we changed.
>>>
>>> The milestone section is also updated: this patchset is for the
>>> milestone 1, though the common init API is still not implemented yet.
>>>
>>>
>>> Changes in rfc v7:
>>> - preserve `make ARCH=um` syntax to build UML
>>> - introduce `make ARCH=um UMMODE=library` to build library mode
>>> - fix undefined symbols issue during modpost
>>> - clean up makefiles (arch/um, tools/um)
>> Hi Hajime, hi Tavi,
>>
>> Our starting point should be that it does not break the existing build. It still does.
> I agree with the starting point.
>
>> If I build a "stock configuration" UML after applying the patchset
>> the resulting vmlinux is not executable.
> Ah, I confirmed the issue.
> I was only trying to make the `linux` binary compatible, not vmlinux.
>
> Because vmlinux is now build as a relocatable object, this is
> something we need to figure out if we wish to keep vmlinux executable.
>
> Do you think we should make vmlinux executable even if we have the
> file linux executable ? If yes, we will work on this to fix the issue.

In my opinion, any relocatable objects, etc should be clearly named - either .o, .so, etc depending on what they are. We should not try to reuse any of the existing files for a different purpose.

I also agree with Johannes that we are not using the tools/ directory for its intended purpose.

We are not trying to build a tool. We are trying to build a sub-architecture. IMHO, the build should use a subdirectory under arch/um.

>> On the positive side, it builds cleanly now. I will try to go
>> through the rest of the patchset later today and see if there is
>> anything else that needs fixing before we do the next version.
> thanks for your time.
>
> -- Hajime
>
>
-- 
Anton R. Ivanov
Cambridgegreys Limited. Registered in England. Company Number 10273661
https://www.cambridgegreys.com/


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 00/21] Unifying LKL into UML
@ 2020-10-08 12:50               ` Anton Ivanov
  0 siblings, 0 replies; 476+ messages in thread
From: Anton Ivanov @ 2020-10-08 12:50 UTC (permalink / raw)
  To: Hajime Tazaki
  Cc: linux-arch, tavi.purdila, richard, jdike, linux-um, retrage01,
	linux-kernel-library


On 08/10/2020 13:12, Hajime Tazaki wrote:
> Hello Anton,
>
> On Wed, 07 Oct 2020 22:30:03 +0900,
> Anton Ivanov wrote:
>>
>> On 06/10/2020 10:44, Hajime Tazaki wrote:
>>> This is another spin of the unification of LKL into UML.  Based on the
>>> discussion of v4 patchset, we have tried to address issue raised and
>>> rewrote the patchset from scratch.  The summary is listed in the
>>> changelog below.
>>>
>>> Although there are still bugs in the patchset, we'd like to ask your
>>> opinions on the design we changed.
>>>
>>> The milestone section is also updated: this patchset is for the
>>> milestone 1, though the common init API is still not implemented yet.
>>>
>>>
>>> Changes in rfc v7:
>>> - preserve `make ARCH=um` syntax to build UML
>>> - introduce `make ARCH=um UMMODE=library` to build library mode
>>> - fix undefined symbols issue during modpost
>>> - clean up makefiles (arch/um, tools/um)
>> Hi Hajime, hi Tavi,
>>
>> Our starting point should be that it does not break the existing build. It still does.
> I agree with the starting point.
>
>> If I build a "stock configuration" UML after applying the patchset
>> the resulting vmlinux is not executable.
> Ah, I confirmed the issue.
> I was only trying to make the `linux` binary compatible, not vmlinux.
>
> Because vmlinux is now build as a relocatable object, this is
> something we need to figure out if we wish to keep vmlinux executable.
>
> Do you think we should make vmlinux executable even if we have the
> file linux executable ? If yes, we will work on this to fix the issue.

In my opinion, any relocatable objects, etc should be clearly named - either .o, .so, etc depending on what they are. We should not try to reuse any of the existing files for a different purpose.

I also agree with Johannes that we are not using the tools/ directory for its intended purpose.

We are not trying to build a tool. We are trying to build a sub-architecture. IMHO, the build should use a subdirectory under arch/um.

>> On the positive side, it builds cleanly now. I will try to go
>> through the rest of the patchset later today and see if there is
>> anything else that needs fixing before we do the next version.
> thanks for your time.
>
> -- Hajime
>
>
-- 
Anton R. Ivanov
Cambridgegreys Limited. Registered in England. Company Number 10273661
https://www.cambridgegreys.com/


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 18/21] um: host: add utilities functions
  2020-10-07 15:10                   ` Anton Ivanov
  (?)
@ 2020-10-08 12:52                   ` Hajime Tazaki
  2020-10-08 19:19                       ` Octavian Purdila
  -1 siblings, 1 reply; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-08 12:52 UTC (permalink / raw)
  To: anton.ivanov
  Cc: johannes, linux-um, jdike, richard, tavi.purdila,
	linux-kernel-library, retrage01, linux-arch


On Thu, 08 Oct 2020 00:10:23 +0900,
Anton Ivanov wrote:
> 
> 
> On 07/10/2020 16:03, Johannes Berg wrote:
> > On Wed, 2020-10-07 at 17:02 +0200, Johannes Berg wrote:
> >> On Wed, 2020-10-07 at 15:53 +0100, Anton Ivanov wrote:
> >>> These are actually different on different architectures. These look
> >>> like the x86 values.

Those variables are based on tools/um/include/lkl/asm-generic/errno.h,
which is generated from include/uapi/asm-generic/errno.h, which LKL
kernel eats for its values.

This is the part of patch 12/21.

> >>> IMHO a kernel strerror() would be the right way of dealing with this
> >>> in the long term (i understand that we cannot call the platform one,
> >>> because it may be different from the internal Linux errors). It will
> >>> be useful in a lot of other places.
> >>> 
> >>> If we leave it as is, we need to make this arch specific at some
> >>> point.

So, this particular code does not contain arch specific part I
believe.
# Tavi, correct me if I'm wrong.

> >>>> +
> >>>> +static const char * const lkl_err_strings[] = {
> >>>> +	"Success",
> >>>> +	"Operation not permitted",
> >> Might be possible to more or less address this (except for arch-specific
> >> errors that don't always exist) but using C99 initializers?
> >> 
> >> [0] = "Success",
> >> [EPERM] = "Operation not permitted",
> >> ..
> > But, on the other hand, is it needed at all? I don't think the kernel
> > ever prints out the actual string ...
> 
> I can see the use case for a library in a multi-arch environment (which IMHO is the intended use case). It saves the user the effort of digging into the build and figuring out what does this error mean today :)
> 
> It is nice to have :)
> 
> If we will have it, however, it should be done as you suggested - C99 or some other way where it maps correctly to actual underlying error codes as they may end up being different depending on build config.

So, this is not for the error values which underlying host generates,
but the ones kernel generates.  And the values should be tied with
uapi/asm-generic/errno.h.

One possible failure that I can see is if
uapi/asm-generic/errno{-base}.h are changed.  In that case, yes, the
way Johannes proposed would be the right way to handle.  The outlook
would be with the prefix and errno name (as follows).

 [0] = "Success",
 [LKL_EPERM] = "Operation not permitted",

-- Hajime

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 18/21] um: host: add utilities functions
  2020-10-07 15:10                   ` Anton Ivanov
  (?)
  (?)
@ 2020-10-08 12:53                   ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-08 12:53 UTC (permalink / raw)
  To: anton.ivanov
  Cc: johannes, linux-um, jdike, richard, tavi.purdila,
	linux-kernel-library, retrage01, linux-arch


On Thu, 08 Oct 2020 00:10:23 +0900,
Anton Ivanov wrote:
> 
> 
> On 07/10/2020 16:03, Johannes Berg wrote:
> > On Wed, 2020-10-07 at 17:02 +0200, Johannes Berg wrote:
> >> On Wed, 2020-10-07 at 15:53 +0100, Anton Ivanov wrote:
> >>> These are actually different on different architectures. These look
> >>> like the x86 values.

Those variables are based on tools/um/include/lkl/asm-generic/errno.h,
which is generated from include/uapi/asm-generic/errno.h, which LKL
kernel eats for its values.

This is the part of patch 12/21.

> >>> IMHO a kernel strerror() would be the right way of dealing with this
> >>> in the long term (i understand that we cannot call the platform one,
> >>> because it may be different from the internal Linux errors). It will
> >>> be useful in a lot of other places.
> >>> 
> >>> If we leave it as is, we need to make this arch specific at some
> >>> point.

So, this particular code does not contain arch specific part I
believe.
# Tavi, correct me if I'm wrong.

> >>>> +
> >>>> +static const char * const lkl_err_strings[] = {
> >>>> +	"Success",
> >>>> +	"Operation not permitted",
> >> Might be possible to more or less address this (except for arch-specific
> >> errors that don't always exist) but using C99 initializers?
> >> 
> >> [0] = "Success",
> >> [EPERM] = "Operation not permitted",
> >> ..
> > But, on the other hand, is it needed at all? I don't think the kernel
> > ever prints out the actual string ...
> 
> I can see the use case for a library in a multi-arch environment (which IMHO is the intended use case). It saves the user the effort of digging into the build and figuring out what does this error mean today :)
> 
> It is nice to have :)
> 
> If we will have it, however, it should be done as you suggested - C99 or some other way where it maps correctly to actual underlying error codes as they may end up being different depending on build config.

So, this is not for the error values which underlying host generates,
but the ones kernel generates.  And the values should be tied with
uapi/asm-generic/errno.h.

One possible failure that I can see is if
uapi/asm-generic/errno{-base}.h are changed.  In that case, yes, the
way Johannes proposed would be the right way to handle.  The outlook
would be with the prefix and errno name (as follows).

 [0] = "Success",
 [LKL_EPERM] = "Operation not permitted",

-- Hajime

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 04/21] um: host: implement os_initcalls and os_exitcalls
  2020-10-07 15:22             ` Johannes Berg
  (?)
@ 2020-10-08 13:16             ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-08 13:16 UTC (permalink / raw)
  To: johannes
  Cc: linux-um, jdike, richard, anton.ivanov, tavi.purdila,
	linux-kernel-library, linux-arch, tavi, retrage01


On Thu, 08 Oct 2020 00:22:18 +0900,
Johannes Berg wrote:
> 
> On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> 
> > Note that this patch should be merged with "um: move arch/um/os-Linux
> > dir to tools/um" but for now it is separate to make the review easier.
> 
> Not a fan of that, I must say ...

Now I found that this patch should be together with the patch 02/21
um: add os init and exit calls.  I will do that from next time.

> > +extern void (*__start_os_exitcalls)(void);
> > +extern void (*__stop_os_exitcalls)(void);
> > +
> > +void os_exitcalls(void)
> > +{
> > +	exitcall_t *call;
> > +
> > +	call = &__stop_os_exitcalls;
> > +	while (--call >= &__start_os_exitcalls)
> > +		(*call)();
> 
> You should check for and skip NULL pointers, there always are alignment
> issues with automatic section filling like this, more so with clang than
> gcc.

Understand, I'll fix it.

> > +}
> > +
> > +extern int (*__start_os_initcalls)(void);
> > +extern int (*__stop_os_initcalls)(void);
> > +
> > +int os_initcalls(void)
> > +{
> > +	initcall_t *call;
> > +
> > +	call = &__stop_os_initcalls;
> > +	while (--call >= &__start_os_initcalls)
> > +		(*call)();
> 
> Same here.

Okay.

-- Hajime

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 02/21] um: add os init and exit calls
  2020-10-07 15:13             ` Johannes Berg
  (?)
@ 2020-10-08 13:18             ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-08 13:18 UTC (permalink / raw)
  To: johannes
  Cc: linux-um, jdike, richard, anton.ivanov, tavi.purdila,
	linux-kernel-library, linux-arch, tavi, retrage01


On Thu, 08 Oct 2020 00:13:02 +0900,
Johannes Berg wrote:
> 
> On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> > 
> > -#define __define_initcall(level,fn) \
> > -	static initcall_t __initcall_##fn __used \
> > -	__attribute__((__section__(".initcall" level ".init"))) = fn
> > -
> > -/* Userspace initcalls shouldn't depend on anything in the kernel, so we'll
> > - * make them run first.
> > - */
> > -#define __initcall(fn) __define_initcall("1", fn)
> > +#undef __uml_exit_call
> > +#define __uml_exit_call		__used __section(os_exitcalls)
> 
> Doesn't that break calling of sigio_cleanup and remove_umid_dir?

Without the patch 04/21 um: host: implement os_initcalls and
os_exitcalls, yes you're right.

> After all,
> 
> > +void __weak os_exitcalls(void)
> > +{
> > +}
> 
> This does nothing so far.
> 
> Also, why the __weak?

The intention is to define os_exitcalls() under tools/um so that the
actual exitcalls is located in different ELF sections (we defined
multiple __uml_exit_call for __UM_HOST__ and ! __UM_HOST__).  Thus
uml_cleanup() must see the symbol but give the place where actual
function is defined at tools/um.  Thus, this is __weak symbol-ed.

-- Hajime

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 00/21] Unifying LKL into UML
  2020-10-08 12:50               ` Anton Ivanov
@ 2020-10-08 17:13                 ` Octavian Purdila
  -1 siblings, 0 replies; 476+ messages in thread
From: Octavian Purdila @ 2020-10-08 17:13 UTC (permalink / raw)
  To: Anton Ivanov, Johannes Berg
  Cc: Hajime Tazaki, linux-um, jdike, Richard Weinberger,
	linux-kernel-library, linux-arch, Akira Moroo

On Thu, Oct 8, 2020 at 3:50 PM Anton Ivanov
<anton.ivanov@cambridgegreys.com> wrote:
>
>
> On 08/10/2020 13:12, Hajime Tazaki wrote:
> > Hello Anton,
> >
> > On Wed, 07 Oct 2020 22:30:03 +0900,
> > Anton Ivanov wrote:
> >>
> >> On 06/10/2020 10:44, Hajime Tazaki wrote:
> >>> This is another spin of the unification of LKL into UML.  Based on the
> >>> discussion of v4 patchset, we have tried to address issue raised and
> >>> rewrote the patchset from scratch.  The summary is listed in the
> >>> changelog below.
> >>>
> >>> Although there are still bugs in the patchset, we'd like to ask your
> >>> opinions on the design we changed.
> >>>
> >>> The milestone section is also updated: this patchset is for the
> >>> milestone 1, though the common init API is still not implemented yet.
> >>>
> >>>
> >>> Changes in rfc v7:
> >>> - preserve `make ARCH=um` syntax to build UML
> >>> - introduce `make ARCH=um UMMODE=library` to build library mode
> >>> - fix undefined symbols issue during modpost
> >>> - clean up makefiles (arch/um, tools/um)
> >> Hi Hajime, hi Tavi,
> >>
> >> Our starting point should be that it does not break the existing build. It still does.
> > I agree with the starting point.
> >
> >> If I build a "stock configuration" UML after applying the patchset
> >> the resulting vmlinux is not executable.
> > Ah, I confirmed the issue.
> > I was only trying to make the `linux` binary compatible, not vmlinux.
> >
> > Because vmlinux is now build as a relocatable object, this is
> > something we need to figure out if we wish to keep vmlinux executable.
> >
> > Do you think we should make vmlinux executable even if we have the
> > file linux executable ? If yes, we will work on this to fix the issue.
>
> In my opinion, any relocatable objects, etc should be clearly named - either .o, .so, etc depending on what they are. We should not try to reuse any of the existing files for a different purpose.
>
> I also agree with Johannes that we are not using the tools/ directory for its intended purpose.
>
> We are not trying to build a tool. We are trying to build a sub-architecture. IMHO, the build should use a subdirectory under arch/um.
>

Hi Anton, Johannes,

From strictly the UML point of view, I can see how this would be a
sub-architecture build since we are doing both the kernel build and
the UML binary as a single step.

If we look at arch/um as just the kernel that enables many
applications, just one of which it is uml, I believe it is cleaner to
build specific applications elsewhere. That is why we proposed to do a
two step build process: one that builds the kernel as a relocatable
object and one that uses this object to build programs, shared
libraries, etc. Besides allowing us to create multiple tools /
applications it also makes it much easier to support multiple OSes or
other environments (e.g. bare-metal applications). As you can see,
this allowed us to remove the linker script for UML and instead just
used the target compiler to build the executable.

We have picked tools because it has the infrastructure to build
programs, shared libraries, and it is the single place (AFAIK) where
we don't link the code in there with the kernel binary.

Thanks,
Tavi

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 00/21] Unifying LKL into UML
@ 2020-10-08 17:13                 ` Octavian Purdila
  0 siblings, 0 replies; 476+ messages in thread
From: Octavian Purdila @ 2020-10-08 17:13 UTC (permalink / raw)
  To: Anton Ivanov, Johannes Berg
  Cc: linux-arch, Richard Weinberger, jdike, linux-um, Akira Moroo,
	linux-kernel-library, Hajime Tazaki

On Thu, Oct 8, 2020 at 3:50 PM Anton Ivanov
<anton.ivanov@cambridgegreys.com> wrote:
>
>
> On 08/10/2020 13:12, Hajime Tazaki wrote:
> > Hello Anton,
> >
> > On Wed, 07 Oct 2020 22:30:03 +0900,
> > Anton Ivanov wrote:
> >>
> >> On 06/10/2020 10:44, Hajime Tazaki wrote:
> >>> This is another spin of the unification of LKL into UML.  Based on the
> >>> discussion of v4 patchset, we have tried to address issue raised and
> >>> rewrote the patchset from scratch.  The summary is listed in the
> >>> changelog below.
> >>>
> >>> Although there are still bugs in the patchset, we'd like to ask your
> >>> opinions on the design we changed.
> >>>
> >>> The milestone section is also updated: this patchset is for the
> >>> milestone 1, though the common init API is still not implemented yet.
> >>>
> >>>
> >>> Changes in rfc v7:
> >>> - preserve `make ARCH=um` syntax to build UML
> >>> - introduce `make ARCH=um UMMODE=library` to build library mode
> >>> - fix undefined symbols issue during modpost
> >>> - clean up makefiles (arch/um, tools/um)
> >> Hi Hajime, hi Tavi,
> >>
> >> Our starting point should be that it does not break the existing build. It still does.
> > I agree with the starting point.
> >
> >> If I build a "stock configuration" UML after applying the patchset
> >> the resulting vmlinux is not executable.
> > Ah, I confirmed the issue.
> > I was only trying to make the `linux` binary compatible, not vmlinux.
> >
> > Because vmlinux is now build as a relocatable object, this is
> > something we need to figure out if we wish to keep vmlinux executable.
> >
> > Do you think we should make vmlinux executable even if we have the
> > file linux executable ? If yes, we will work on this to fix the issue.
>
> In my opinion, any relocatable objects, etc should be clearly named - either .o, .so, etc depending on what they are. We should not try to reuse any of the existing files for a different purpose.
>
> I also agree with Johannes that we are not using the tools/ directory for its intended purpose.
>
> We are not trying to build a tool. We are trying to build a sub-architecture. IMHO, the build should use a subdirectory under arch/um.
>

Hi Anton, Johannes,

From strictly the UML point of view, I can see how this would be a
sub-architecture build since we are doing both the kernel build and
the UML binary as a single step.

If we look at arch/um as just the kernel that enables many
applications, just one of which it is uml, I believe it is cleaner to
build specific applications elsewhere. That is why we proposed to do a
two step build process: one that builds the kernel as a relocatable
object and one that uses this object to build programs, shared
libraries, etc. Besides allowing us to create multiple tools /
applications it also makes it much easier to support multiple OSes or
other environments (e.g. bare-metal applications). As you can see,
this allowed us to remove the linker script for UML and instead just
used the target compiler to build the executable.

We have picked tools because it has the infrastructure to build
programs, shared libraries, and it is the single place (AFAIK) where
we don't link the code in there with the kernel binary.

Thanks,
Tavi

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 00/21] Unifying LKL into UML
  2020-10-08 17:13                 ` Octavian Purdila
@ 2020-10-08 17:18                   ` Anton Ivanov
  -1 siblings, 0 replies; 476+ messages in thread
From: Anton Ivanov @ 2020-10-08 17:18 UTC (permalink / raw)
  To: Octavian Purdila, Johannes Berg
  Cc: Hajime Tazaki, linux-um, jdike, Richard Weinberger,
	linux-kernel-library, linux-arch, Akira Moroo

On 08/10/2020 18:13, Octavian Purdila wrote:
> On Thu, Oct 8, 2020 at 3:50 PM Anton Ivanov
> <anton.ivanov@cambridgegreys.com> wrote:
>>
>> On 08/10/2020 13:12, Hajime Tazaki wrote:
>>> Hello Anton,
>>>
>>> On Wed, 07 Oct 2020 22:30:03 +0900,
>>> Anton Ivanov wrote:
>>>> On 06/10/2020 10:44, Hajime Tazaki wrote:
>>>>> This is another spin of the unification of LKL into UML.  Based on the
>>>>> discussion of v4 patchset, we have tried to address issue raised and
>>>>> rewrote the patchset from scratch.  The summary is listed in the
>>>>> changelog below.
>>>>>
>>>>> Although there are still bugs in the patchset, we'd like to ask your
>>>>> opinions on the design we changed.
>>>>>
>>>>> The milestone section is also updated: this patchset is for the
>>>>> milestone 1, though the common init API is still not implemented yet.
>>>>>
>>>>>
>>>>> Changes in rfc v7:
>>>>> - preserve `make ARCH=um` syntax to build UML
>>>>> - introduce `make ARCH=um UMMODE=library` to build library mode
>>>>> - fix undefined symbols issue during modpost
>>>>> - clean up makefiles (arch/um, tools/um)
>>>> Hi Hajime, hi Tavi,
>>>>
>>>> Our starting point should be that it does not break the existing build. It still does.
>>> I agree with the starting point.
>>>
>>>> If I build a "stock configuration" UML after applying the patchset
>>>> the resulting vmlinux is not executable.
>>> Ah, I confirmed the issue.
>>> I was only trying to make the `linux` binary compatible, not vmlinux.
>>>
>>> Because vmlinux is now build as a relocatable object, this is
>>> something we need to figure out if we wish to keep vmlinux executable.
>>>
>>> Do you think we should make vmlinux executable even if we have the
>>> file linux executable ? If yes, we will work on this to fix the issue.
>> In my opinion, any relocatable objects, etc should be clearly named - either .o, .so, etc depending on what they are. We should not try to reuse any of the existing files for a different purpose.
>>
>> I also agree with Johannes that we are not using the tools/ directory for its intended purpose.
>>
>> We are not trying to build a tool. We are trying to build a sub-architecture. IMHO, the build should use a subdirectory under arch/um.
>>
> Hi Anton, Johannes,
>
>  From strictly the UML point of view, I can see how this would be a
> sub-architecture build since we are doing both the kernel build and
> the UML binary as a single step.
>
> If we look at arch/um as just the kernel that enables many
> applications, just one of which it is uml, I believe it is cleaner to
> build specific applications elsewhere. That is why we proposed to do a
> two step build process: one that builds the kernel as a relocatable
> object and one that uses this object to build programs, shared
> libraries, etc. Besides allowing us to create multiple tools /
> applications it also makes it much easier to support multiple OSes or
> other environments (e.g. bare-metal applications). As you can see,
> this allowed us to remove the linker script for UML and instead just
> used the target compiler to build the executable.

I have no objection to the proposal. In fact I like it very much.

My only concern is that we end up with files which are named identically 
to already existing ones.

IMHO, the kernel relocatable object should be vmlinux.a or vmlinux.so or 
vmlinux.o. It should not replace an already existing file which behaves 
differently.

Brgds,

A.

>
> We have picked tools because it has the infrastructure to build
> programs, shared libraries, and it is the single place (AFAIK) where
> we don't link the code in there with the kernel binary.
>
> Thanks,
> Tavi
>

-- 
Anton R. Ivanov
Cambridgegreys Limited. Registered in England. Company Number 10273661
https://www.cambridgegreys.com/


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 00/21] Unifying LKL into UML
@ 2020-10-08 17:18                   ` Anton Ivanov
  0 siblings, 0 replies; 476+ messages in thread
From: Anton Ivanov @ 2020-10-08 17:18 UTC (permalink / raw)
  To: Octavian Purdila, Johannes Berg
  Cc: linux-arch, Richard Weinberger, jdike, linux-um, Akira Moroo,
	linux-kernel-library, Hajime Tazaki

On 08/10/2020 18:13, Octavian Purdila wrote:
> On Thu, Oct 8, 2020 at 3:50 PM Anton Ivanov
> <anton.ivanov@cambridgegreys.com> wrote:
>>
>> On 08/10/2020 13:12, Hajime Tazaki wrote:
>>> Hello Anton,
>>>
>>> On Wed, 07 Oct 2020 22:30:03 +0900,
>>> Anton Ivanov wrote:
>>>> On 06/10/2020 10:44, Hajime Tazaki wrote:
>>>>> This is another spin of the unification of LKL into UML.  Based on the
>>>>> discussion of v4 patchset, we have tried to address issue raised and
>>>>> rewrote the patchset from scratch.  The summary is listed in the
>>>>> changelog below.
>>>>>
>>>>> Although there are still bugs in the patchset, we'd like to ask your
>>>>> opinions on the design we changed.
>>>>>
>>>>> The milestone section is also updated: this patchset is for the
>>>>> milestone 1, though the common init API is still not implemented yet.
>>>>>
>>>>>
>>>>> Changes in rfc v7:
>>>>> - preserve `make ARCH=um` syntax to build UML
>>>>> - introduce `make ARCH=um UMMODE=library` to build library mode
>>>>> - fix undefined symbols issue during modpost
>>>>> - clean up makefiles (arch/um, tools/um)
>>>> Hi Hajime, hi Tavi,
>>>>
>>>> Our starting point should be that it does not break the existing build. It still does.
>>> I agree with the starting point.
>>>
>>>> If I build a "stock configuration" UML after applying the patchset
>>>> the resulting vmlinux is not executable.
>>> Ah, I confirmed the issue.
>>> I was only trying to make the `linux` binary compatible, not vmlinux.
>>>
>>> Because vmlinux is now build as a relocatable object, this is
>>> something we need to figure out if we wish to keep vmlinux executable.
>>>
>>> Do you think we should make vmlinux executable even if we have the
>>> file linux executable ? If yes, we will work on this to fix the issue.
>> In my opinion, any relocatable objects, etc should be clearly named - either .o, .so, etc depending on what they are. We should not try to reuse any of the existing files for a different purpose.
>>
>> I also agree with Johannes that we are not using the tools/ directory for its intended purpose.
>>
>> We are not trying to build a tool. We are trying to build a sub-architecture. IMHO, the build should use a subdirectory under arch/um.
>>
> Hi Anton, Johannes,
>
>  From strictly the UML point of view, I can see how this would be a
> sub-architecture build since we are doing both the kernel build and
> the UML binary as a single step.
>
> If we look at arch/um as just the kernel that enables many
> applications, just one of which it is uml, I believe it is cleaner to
> build specific applications elsewhere. That is why we proposed to do a
> two step build process: one that builds the kernel as a relocatable
> object and one that uses this object to build programs, shared
> libraries, etc. Besides allowing us to create multiple tools /
> applications it also makes it much easier to support multiple OSes or
> other environments (e.g. bare-metal applications). As you can see,
> this allowed us to remove the linker script for UML and instead just
> used the target compiler to build the executable.

I have no objection to the proposal. In fact I like it very much.

My only concern is that we end up with files which are named identically 
to already existing ones.

IMHO, the kernel relocatable object should be vmlinux.a or vmlinux.so or 
vmlinux.o. It should not replace an already existing file which behaves 
differently.

Brgds,

A.

>
> We have picked tools because it has the infrastructure to build
> programs, shared libraries, and it is the single place (AFAIK) where
> we don't link the code in there with the kernel binary.
>
> Thanks,
> Tavi
>

-- 
Anton R. Ivanov
Cambridgegreys Limited. Registered in England. Company Number 10273661
https://www.cambridgegreys.com/


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 00/21] Unifying LKL into UML
  2020-10-08 17:18                   ` Anton Ivanov
@ 2020-10-08 17:24                     ` Octavian Purdila
  -1 siblings, 0 replies; 476+ messages in thread
From: Octavian Purdila @ 2020-10-08 17:24 UTC (permalink / raw)
  To: Anton Ivanov
  Cc: Johannes Berg, Hajime Tazaki, linux-um, jdike,
	Richard Weinberger, linux-kernel-library, linux-arch,
	Akira Moroo

On Thu, Oct 8, 2020 at 8:18 PM Anton Ivanov
<anton.ivanov@cambridgegreys.com> wrote:
>
> On 08/10/2020 18:13, Octavian Purdila wrote:
> > On Thu, Oct 8, 2020 at 3:50 PM Anton Ivanov
> > <anton.ivanov@cambridgegreys.com> wrote:
> >>
> >> On 08/10/2020 13:12, Hajime Tazaki wrote:
> >>> Hello Anton,
> >>>
> >>> On Wed, 07 Oct 2020 22:30:03 +0900,
> >>> Anton Ivanov wrote:
> >>>> On 06/10/2020 10:44, Hajime Tazaki wrote:
> >>>>> This is another spin of the unification of LKL into UML.  Based on the
> >>>>> discussion of v4 patchset, we have tried to address issue raised and
> >>>>> rewrote the patchset from scratch.  The summary is listed in the
> >>>>> changelog below.
> >>>>>
> >>>>> Although there are still bugs in the patchset, we'd like to ask your
> >>>>> opinions on the design we changed.
> >>>>>
> >>>>> The milestone section is also updated: this patchset is for the
> >>>>> milestone 1, though the common init API is still not implemented yet.
> >>>>>
> >>>>>
> >>>>> Changes in rfc v7:
> >>>>> - preserve `make ARCH=um` syntax to build UML
> >>>>> - introduce `make ARCH=um UMMODE=library` to build library mode
> >>>>> - fix undefined symbols issue during modpost
> >>>>> - clean up makefiles (arch/um, tools/um)
> >>>> Hi Hajime, hi Tavi,
> >>>>
> >>>> Our starting point should be that it does not break the existing build. It still does.
> >>> I agree with the starting point.
> >>>
> >>>> If I build a "stock configuration" UML after applying the patchset
> >>>> the resulting vmlinux is not executable.
> >>> Ah, I confirmed the issue.
> >>> I was only trying to make the `linux` binary compatible, not vmlinux.
> >>>
> >>> Because vmlinux is now build as a relocatable object, this is
> >>> something we need to figure out if we wish to keep vmlinux executable.
> >>>
> >>> Do you think we should make vmlinux executable even if we have the
> >>> file linux executable ? If yes, we will work on this to fix the issue.
> >> In my opinion, any relocatable objects, etc should be clearly named - either .o, .so, etc depending on what they are. We should not try to reuse any of the existing files for a different purpose.
> >>
> >> I also agree with Johannes that we are not using the tools/ directory for its intended purpose.
> >>
> >> We are not trying to build a tool. We are trying to build a sub-architecture. IMHO, the build should use a subdirectory under arch/um.
> >>
> > Hi Anton, Johannes,
> >
> >  From strictly the UML point of view, I can see how this would be a
> > sub-architecture build since we are doing both the kernel build and
> > the UML binary as a single step.
> >
> > If we look at arch/um as just the kernel that enables many
> > applications, just one of which it is uml, I believe it is cleaner to
> > build specific applications elsewhere. That is why we proposed to do a
> > two step build process: one that builds the kernel as a relocatable
> > object and one that uses this object to build programs, shared
> > libraries, etc. Besides allowing us to create multiple tools /
> > applications it also makes it much easier to support multiple OSes or
> > other environments (e.g. bare-metal applications). As you can see,
> > this allowed us to remove the linker script for UML and instead just
> > used the target compiler to build the executable.
>
> I have no objection to the proposal. In fact I like it very much.
>
> My only concern is that we end up with files which are named identically
> to already existing ones.
>
> IMHO, the kernel relocatable object should be vmlinux.a or vmlinux.so or
> vmlinux.o. It should not replace an already existing file which behaves
> differently.
>

Got it, we will change that for the next patch series.

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 00/21] Unifying LKL into UML
@ 2020-10-08 17:24                     ` Octavian Purdila
  0 siblings, 0 replies; 476+ messages in thread
From: Octavian Purdila @ 2020-10-08 17:24 UTC (permalink / raw)
  To: Anton Ivanov
  Cc: linux-arch, Richard Weinberger, jdike, linux-um, Akira Moroo,
	linux-kernel-library, Johannes Berg, Hajime Tazaki

On Thu, Oct 8, 2020 at 8:18 PM Anton Ivanov
<anton.ivanov@cambridgegreys.com> wrote:
>
> On 08/10/2020 18:13, Octavian Purdila wrote:
> > On Thu, Oct 8, 2020 at 3:50 PM Anton Ivanov
> > <anton.ivanov@cambridgegreys.com> wrote:
> >>
> >> On 08/10/2020 13:12, Hajime Tazaki wrote:
> >>> Hello Anton,
> >>>
> >>> On Wed, 07 Oct 2020 22:30:03 +0900,
> >>> Anton Ivanov wrote:
> >>>> On 06/10/2020 10:44, Hajime Tazaki wrote:
> >>>>> This is another spin of the unification of LKL into UML.  Based on the
> >>>>> discussion of v4 patchset, we have tried to address issue raised and
> >>>>> rewrote the patchset from scratch.  The summary is listed in the
> >>>>> changelog below.
> >>>>>
> >>>>> Although there are still bugs in the patchset, we'd like to ask your
> >>>>> opinions on the design we changed.
> >>>>>
> >>>>> The milestone section is also updated: this patchset is for the
> >>>>> milestone 1, though the common init API is still not implemented yet.
> >>>>>
> >>>>>
> >>>>> Changes in rfc v7:
> >>>>> - preserve `make ARCH=um` syntax to build UML
> >>>>> - introduce `make ARCH=um UMMODE=library` to build library mode
> >>>>> - fix undefined symbols issue during modpost
> >>>>> - clean up makefiles (arch/um, tools/um)
> >>>> Hi Hajime, hi Tavi,
> >>>>
> >>>> Our starting point should be that it does not break the existing build. It still does.
> >>> I agree with the starting point.
> >>>
> >>>> If I build a "stock configuration" UML after applying the patchset
> >>>> the resulting vmlinux is not executable.
> >>> Ah, I confirmed the issue.
> >>> I was only trying to make the `linux` binary compatible, not vmlinux.
> >>>
> >>> Because vmlinux is now build as a relocatable object, this is
> >>> something we need to figure out if we wish to keep vmlinux executable.
> >>>
> >>> Do you think we should make vmlinux executable even if we have the
> >>> file linux executable ? If yes, we will work on this to fix the issue.
> >> In my opinion, any relocatable objects, etc should be clearly named - either .o, .so, etc depending on what they are. We should not try to reuse any of the existing files for a different purpose.
> >>
> >> I also agree with Johannes that we are not using the tools/ directory for its intended purpose.
> >>
> >> We are not trying to build a tool. We are trying to build a sub-architecture. IMHO, the build should use a subdirectory under arch/um.
> >>
> > Hi Anton, Johannes,
> >
> >  From strictly the UML point of view, I can see how this would be a
> > sub-architecture build since we are doing both the kernel build and
> > the UML binary as a single step.
> >
> > If we look at arch/um as just the kernel that enables many
> > applications, just one of which it is uml, I believe it is cleaner to
> > build specific applications elsewhere. That is why we proposed to do a
> > two step build process: one that builds the kernel as a relocatable
> > object and one that uses this object to build programs, shared
> > libraries, etc. Besides allowing us to create multiple tools /
> > applications it also makes it much easier to support multiple OSes or
> > other environments (e.g. bare-metal applications). As you can see,
> > this allowed us to remove the linker script for UML and instead just
> > used the target compiler to build the executable.
>
> I have no objection to the proposal. In fact I like it very much.
>
> My only concern is that we end up with files which are named identically
> to already existing ones.
>
> IMHO, the kernel relocatable object should be vmlinux.a or vmlinux.so or
> vmlinux.o. It should not replace an already existing file which behaves
> differently.
>

Got it, we will change that for the next patch series.

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 03/21] um: move arch/um/os-Linux dir to tools/um/uml
  2020-10-07 15:20             ` Johannes Berg
@ 2020-10-08 17:48               ` Octavian Purdila
  -1 siblings, 0 replies; 476+ messages in thread
From: Octavian Purdila @ 2020-10-08 17:48 UTC (permalink / raw)
  To: Johannes Berg
  Cc: Hajime Tazaki, linux-um, jdike, Richard Weinberger, Anton Ivanov,
	linux-kernel-library, linux-arch, Akira Moroo

On Wed, Oct 7, 2020 at 6:20 PM Johannes Berg <johannes@sipsolutions.net> wrote:
>

Hi Johannes,

Thank you for the review.

> On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> > This patch moves underlying OS dependent part under arch/um to tools/um
> > directory so that arch/um code does not need to build host build
> > facilities (e.g., libc).
>
> Hmm. On the one hand, that build separation seems sensible, but on the
> other hand this stuff *does* fundamentally belong to arch/um/, and it's
> not a "tool" like basically everything else there that is a pure
> userspace application to run *inside* the kernel, not *part of* it.
>
> For that reason, I don't really like this much.
>

I see the os_*() calls as dependencies that the kernel uses. Sort of
like calls into the hypervisor or firmware.

The current UML build is already partially split. USER_OBJS build with
a different rule set than the rest of the kernel objects. IMHO this
change just makes this more clear and streamlined, especially with
regard to linking.

>
> >  tools/um/uml/Build                            | 48 +++++++++++++++++++
> >  tools/um/uml/drivers/Build                    | 10 ++++
>
> Also, what's with the names here? What's wrong with "Makefile"?
>
>

We are using the same build system as the rest of the stuff in tools.
Since it is building programs and libraries and not part of the kernel
binary build, it is using a slightly different infrastructure, which
is detailed in tools/build/Documentation/Build.txt

> I'm also not sure I see how this is built at all from the top level
> Makefile? Oh. I think Anton said it doesn't ... that alone would be a
> reason not to do it I guess.
>
>
> So why do you think it must be under tools/? Even if you consider "lkl"
> a "tool", that doesn't mean it must be there. I also consider a UML
> binary ("linux") a "tool" in my simulation environment, etc.
>
>
> And even LKL, which is the eventual goal here - why would you consider
> that a "tool"? I don't think we should.
>

The reason for picking tools was that it already has the
infrastructure to build programs and shared libraries and the fact
that it is the only place in the kernel source tree where code is not
built directly into the kernel binary.

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 03/21] um: move arch/um/os-Linux dir to tools/um/uml
@ 2020-10-08 17:48               ` Octavian Purdila
  0 siblings, 0 replies; 476+ messages in thread
From: Octavian Purdila @ 2020-10-08 17:48 UTC (permalink / raw)
  To: Johannes Berg
  Cc: linux-arch, Richard Weinberger, jdike, linux-um, Akira Moroo,
	linux-kernel-library, Hajime Tazaki, Anton Ivanov

On Wed, Oct 7, 2020 at 6:20 PM Johannes Berg <johannes@sipsolutions.net> wrote:
>

Hi Johannes,

Thank you for the review.

> On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> > This patch moves underlying OS dependent part under arch/um to tools/um
> > directory so that arch/um code does not need to build host build
> > facilities (e.g., libc).
>
> Hmm. On the one hand, that build separation seems sensible, but on the
> other hand this stuff *does* fundamentally belong to arch/um/, and it's
> not a "tool" like basically everything else there that is a pure
> userspace application to run *inside* the kernel, not *part of* it.
>
> For that reason, I don't really like this much.
>

I see the os_*() calls as dependencies that the kernel uses. Sort of
like calls into the hypervisor or firmware.

The current UML build is already partially split. USER_OBJS build with
a different rule set than the rest of the kernel objects. IMHO this
change just makes this more clear and streamlined, especially with
regard to linking.

>
> >  tools/um/uml/Build                            | 48 +++++++++++++++++++
> >  tools/um/uml/drivers/Build                    | 10 ++++
>
> Also, what's with the names here? What's wrong with "Makefile"?
>
>

We are using the same build system as the rest of the stuff in tools.
Since it is building programs and libraries and not part of the kernel
binary build, it is using a slightly different infrastructure, which
is detailed in tools/build/Documentation/Build.txt

> I'm also not sure I see how this is built at all from the top level
> Makefile? Oh. I think Anton said it doesn't ... that alone would be a
> reason not to do it I guess.
>
>
> So why do you think it must be under tools/? Even if you consider "lkl"
> a "tool", that doesn't mean it must be there. I also consider a UML
> binary ("linux") a "tool" in my simulation environment, etc.
>
>
> And even LKL, which is the eventual goal here - why would you consider
> that a "tool"? I don't think we should.
>

The reason for picking tools was that it already has the
infrastructure to build programs and shared libraries and the fact
that it is the only place in the kernel source tree where code is not
built directly into the kernel binary.

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 10/21] um: nommu: memory handling
  2020-10-07 15:47             ` Johannes Berg
@ 2020-10-08 18:07               ` Octavian Purdila
  -1 siblings, 0 replies; 476+ messages in thread
From: Octavian Purdila @ 2020-10-08 18:07 UTC (permalink / raw)
  To: Johannes Berg
  Cc: Hajime Tazaki, linux-um, jdike, Richard Weinberger, Anton Ivanov,
	linux-kernel-library, linux-arch, Akira Moroo

On Wed, Oct 7, 2020 at 6:47 PM Johannes Berg <johannes@sipsolutions.net> wrote:
>
> On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> >
> >   * These operations must be provided by a host library or by the application
> >   * itself.
> >   *
> > + * @mem_alloc - allocate memory
> > + * @mem_free - free memory
> > + *
>
> Actual kernel-doc would be nicer.
>

Thank you, we will make sure to use the proper kernel doc throughout
all patches for the next patch series

> > +     empty_zero_page = memblock_alloc(PAGE_SIZE, PAGE_SIZE);
> > +     memset((void *)empty_zero_page, 0, PAGE_SIZE);
> > +
> > +     {
> > +             unsigned long zones_size[MAX_NR_ZONES] = {0, };
>
> Hmm, what's with the extra scope?
>

Will clean it up in the next patch series, thank you.

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 10/21] um: nommu: memory handling
@ 2020-10-08 18:07               ` Octavian Purdila
  0 siblings, 0 replies; 476+ messages in thread
From: Octavian Purdila @ 2020-10-08 18:07 UTC (permalink / raw)
  To: Johannes Berg
  Cc: linux-arch, Richard Weinberger, jdike, linux-um, Akira Moroo,
	linux-kernel-library, Hajime Tazaki, Anton Ivanov

On Wed, Oct 7, 2020 at 6:47 PM Johannes Berg <johannes@sipsolutions.net> wrote:
>
> On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> >
> >   * These operations must be provided by a host library or by the application
> >   * itself.
> >   *
> > + * @mem_alloc - allocate memory
> > + * @mem_free - free memory
> > + *
>
> Actual kernel-doc would be nicer.
>

Thank you, we will make sure to use the proper kernel doc throughout
all patches for the next patch series

> > +     empty_zero_page = memblock_alloc(PAGE_SIZE, PAGE_SIZE);
> > +     memset((void *)empty_zero_page, 0, PAGE_SIZE);
> > +
> > +     {
> > +             unsigned long zones_size[MAX_NR_ZONES] = {0, };
>
> Hmm, what's with the extra scope?
>

Will clean it up in the next patch series, thank you.

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 09/21] um: nommu: host interface
  2020-10-07 15:45             ` Johannes Berg
@ 2020-10-08 18:10               ` Octavian Purdila
  -1 siblings, 0 replies; 476+ messages in thread
From: Octavian Purdila @ 2020-10-08 18:10 UTC (permalink / raw)
  To: Johannes Berg
  Cc: Hajime Tazaki, linux-um, jdike, Richard Weinberger, Anton Ivanov,
	linux-kernel-library, linux-arch, Akira Moroo

On Wed, Oct 7, 2020 at 6:45 PM Johannes Berg <johannes@sipsolutions.net> wrote:
>
> On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> > This patch introduces the host operations that define the interface
> > between the LKL and the host. These operations must be provided either
> > by a host library or by the application itself.
> >
>
> I kinda find patches like this very pointless ... you see nothing, and
> can't really say anything. Maybe add the infra with the first user?
>
> > +/* Defined in {posix,nt}-host.c */
>
> That doesn't exist either yet ...
>

Noted, we will fix it in the next patch series.

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 09/21] um: nommu: host interface
@ 2020-10-08 18:10               ` Octavian Purdila
  0 siblings, 0 replies; 476+ messages in thread
From: Octavian Purdila @ 2020-10-08 18:10 UTC (permalink / raw)
  To: Johannes Berg
  Cc: linux-arch, Richard Weinberger, jdike, linux-um, Akira Moroo,
	linux-kernel-library, Hajime Tazaki, Anton Ivanov

On Wed, Oct 7, 2020 at 6:45 PM Johannes Berg <johannes@sipsolutions.net> wrote:
>
> On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> > This patch introduces the host operations that define the interface
> > between the LKL and the host. These operations must be provided either
> > by a host library or by the application itself.
> >
>
> I kinda find patches like this very pointless ... you see nothing, and
> can't really say anything. Maybe add the infra with the first user?
>
> > +/* Defined in {posix,nt}-host.c */
>
> That doesn't exist either yet ...
>

Noted, we will fix it in the next patch series.

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 11/21] um: nommu: kernel thread support
  2020-10-07 18:57             ` Johannes Berg
@ 2020-10-08 18:54               ` Octavian Purdila
  -1 siblings, 0 replies; 476+ messages in thread
From: Octavian Purdila @ 2020-10-08 18:54 UTC (permalink / raw)
  To: Johannes Berg
  Cc: Hajime Tazaki, linux-um, jdike, Richard Weinberger, Anton Ivanov,
	linux-kernel-library, linux-arch, Akira Moroo

On Wed, Oct 7, 2020 at 9:57 PM Johannes Berg <johannes@sipsolutions.net> wrote:
>
> On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> > nommu mode does not support user processes
>
> I find this really confusing. I'm not sure why you ended up calling this
> "nommu mode", but there *are* (still) (other) nommu arches, and they
> *do* support userspace processes.
>
> Isn't this really just "LKL mode" or something like that?
>

This is a very good point, while some other patches make sense in the
nommu mode, this one does not - it is rather needed because of the
"library mode".

Not sure what we can do other than creating a new "library mode" in
addition to the "nommu mode". Any suggestions?

> >  #define TIF_SYSCALL_TRACE    0       /* syscall trace active */
> > @@ -63,6 +85,8 @@ static inline struct thread_info *current_thread_info(void)
> >  #define TIF_RESTORE_SIGMASK  7
> >  #define TIF_NOTIFY_RESUME    8
> >  #define TIF_SECCOMP          9       /* secure computing */
> > +#define TIF_SCHED_JB         10
> > +#define TIF_HOST_THREAD              11
>
> It'd be nice to document what those mean, and even what "JB" means ... I
> saw something about "jump buffer" somewhere, but I have no idea why that
> should be a thread flag.
>
> > @@ -16,11 +16,65 @@ struct lkl_jmp_buf {
> >   * These operations must be provided by a host library or by the application
> >   * itself.
> >   *
> > + * @sem_alloc - allocate a host semaphore an initialize it to count
> > + * @sem_free - free a host semaphore
> > + * @sem_up - perform an up operation on the semaphore
> > + * @sem_down - perform a down operation on the semaphore
> > + *
> > + * @mutex_alloc - allocate and initialize a host mutex; the recursive parameter
> > + * determines if the mutex is recursive or not
> > + * @mutex_free - free a host mutex
> > + * @mutex_lock - acquire the mutex
> > + * @mutex_unlock - release the mutex
> > + *
> > + * @thread_create - create a new thread and run f(arg) in its context; returns a
> > + * thread handle or 0 if the thread could not be created
> > + * @thread_detach - on POSIX systems, free up resources held by
> > + * pthreads. Noop on Win32.
> > + * @thread_exit - terminates the current thread
> > + * @thread_join - wait for the given thread to terminate. Returns 0
> > + * for success, -1 otherwise
> > + *
> > + * @gettid - returns the host thread id of the caller, which need not
> > + * be the same as the handle returned by thread_create
> > + *
> > + * @jmp_buf_set - runs the give function and setups a jump back point by saving
> > + * the context in the jump buffer; jmp_buf_longjmp can be called from the give
> > + * function or any callee in that function to return back to the jump back
> > + * point
> > + *
> > + * NOTE: we can't return from jmp_buf_set before calling jmp_buf_longjmp or
> > + * otherwise the saved context (stack) is not going to be valid, so we must pass
> > + * the function that will eventually call longjmp here
> > + *
> > + * @jmp_buf_longjmp - perform a jump back to the saved jump buffer
> > + *
> >   * @mem_alloc - allocate memory
> >   * @mem_free - free memory
>
> again, kernel-doc.
>
> But I'm starting to doubt the value of having this struct at all. Care
> you explain? You're doing everything else already with weak functions,
> and you can't very well have _two_ hosts compiled anyway, so what's the
> point?
>
> IOW, why isn't this just
>
> void lkl_sem_free(struct lkl_sem *sem);
> void lkl_sem_up(struct lkl_sem *sem);
> ...
>
> and then posix-host.c just includes the header file and implements those
> functions?
>
> I don't see any reason for this to be allowed to have multiple variants
> linked and then picking them at runtime?
>

We could try that and see how it goes. This was baked liked this long
time ago, when we wanted to support Windows and there was no proper
support for weak functions in mingw for PE/COFF (it still not
supported but at least we do have a few patches that fix that).

> > +/*
> > + * This structure is used to get access to the "LKL CPU" that allows us to run
>
> Are you trying to implement SMP? This seems ... rather complex?
>
> > + * Linux code. Because we have to deal with various synchronization requirements
> > + * between idle thread, system calls, interrupts, "reentrancy", CPU shutdown,
> > + * imbalance wake up (i.e. acquire the CPU from one thread and release it from
> > + * another), we can't use a simple synchronization mechanism such as (recursive)
> > + * mutex or semaphore. Instead, we use a mutex and a bunch of status data plus a
> > + * semaphore.
> > + */
> > +static struct lkl_cpu {
> > +     /* lock that protects the CPU status data */
> > +     struct lkl_mutex *lock;
> > +     /*
> > +      * Since we must free the cpu lock during shutdown we need a
> > +      * synchronization algorithm between lkl_cpu_shutdown() and the CPU
> > +      * access functions since lkl_cpu_get() gets called from thread
> > +      * destructor callback functions which may be scheduled after
> > +      * lkl_cpu_shutdown() has freed the cpu lock.
> > +      *
> > +      * An atomic counter is used to keep track of the number of running
> > +      * CPU access functions and allow the shutdown function to wait for
> > +      * them.
> > +      *
> > +      * The shutdown functions adds MAX_THREADS to this counter which allows
> > +      * the CPU access functions to check if the shutdown process has
> > +      * started.
> > +      *
> > +      * This algorithm assumes that we never have more the MAX_THREADS
> > +      * requesting CPU access.
> > +      */
> > +     #define MAX_THREADS 1000000
> > +     unsigned int shutdown_gate;
> > +     bool irqs_pending;
> > +     /* no of threads waiting the CPU */
> > +     unsigned int sleepers;
> > +     /* no of times the current thread got the CPU */
> > +     unsigned int count;
> > +     /* current thread that owns the CPU */
> > +     lkl_thread_t owner;
> > +     /* semaphore for threads waiting the CPU */
> > +     struct lkl_sem *sem;
> > +     /* semaphore used for shutdown */
> > +     struct lkl_sem *shutdown_sem;
> > +} cpu;
> > +
> > +static int __cpu_try_get_lock(int n)
> > +{
> > +     lkl_thread_t self;
> > +
> > +     if (__sync_fetch_and_add(&cpu.shutdown_gate, n) >= MAX_THREADS)
> > +             return -2;
> > +
> > +     lkl_ops->mutex_lock(cpu.lock);
> > +
> > +     if (cpu.shutdown_gate >= MAX_THREADS)
> > +             return -1;
> > +
> > +     self = lkl_ops->thread_self();
> > +
> > +     if (cpu.owner && !lkl_ops->thread_equal(cpu.owner, self))
> > +             return 0;
> > +
> > +     cpu.owner = self;
> > +     cpu.count++;
> > +
> > +     return 1;
> > +}
> > +
> > +static void __cpu_try_get_unlock(int lock_ret, int n)
> > +{
> > +     if (lock_ret >= -1)
> > +             lkl_ops->mutex_unlock(cpu.lock);
> > +     __sync_fetch_and_sub(&cpu.shutdown_gate, n);
> > +}
> > +
> > +void lkl_cpu_change_owner(lkl_thread_t owner)
> > +{
> > +     lkl_ops->mutex_lock(cpu.lock);
> > +     if (cpu.count > 1)
> > +             lkl_bug("bad count while changing owner\n");
> > +     cpu.owner = owner;
> > +     lkl_ops->mutex_unlock(cpu.lock);
> > +}
> > +
> > +int lkl_cpu_get(void)
> > +{
> > +     int ret;
> > +
> > +     ret = __cpu_try_get_lock(1);
> > +
> > +     while (ret == 0) {
> > +             cpu.sleepers++;
> > +             __cpu_try_get_unlock(ret, 0);
> > +             lkl_ops->sem_down(cpu.sem);
> > +             ret = __cpu_try_get_lock(0);
> > +     }
> > +
> > +     __cpu_try_get_unlock(ret, 1);
> > +
> > +     return ret;
> > +}
> > +
> > +void lkl_cpu_put(void)
> > +{
> > +     lkl_ops->mutex_lock(cpu.lock);
> > +
> > +     if (!cpu.count || !cpu.owner ||
> > +         !lkl_ops->thread_equal(cpu.owner, lkl_ops->thread_self()))
> > +             lkl_bug("%s: unbalanced put\n", __func__);
> > +
> > +     while (cpu.irqs_pending && !irqs_disabled()) {
> > +             cpu.irqs_pending = false;
> > +             lkl_ops->mutex_unlock(cpu.lock);
> > +             run_irqs();
> > +             lkl_ops->mutex_lock(cpu.lock);
> > +     }
> > +
> > +     if (test_ti_thread_flag(current_thread_info(), TIF_HOST_THREAD) &&
> > +         !single_task_running() && cpu.count == 1) {
> > +             if (in_interrupt())
> > +                     lkl_bug("%s: in interrupt\n", __func__);
> > +             lkl_ops->mutex_unlock(cpu.lock);
> > +             thread_sched_jb();
> > +             return;
> > +     }
> > +
> > +     if (--cpu.count > 0) {
> > +             lkl_ops->mutex_unlock(cpu.lock);
> > +             return;
> > +     }
> > +
> > +     if (cpu.sleepers) {
> > +             cpu.sleepers--;
> > +             lkl_ops->sem_up(cpu.sem);
> > +     }
> > +
> > +     cpu.owner = 0;
> > +
> > +     lkl_ops->mutex_unlock(cpu.lock);
> > +}
> > +
> > +int lkl_cpu_try_run_irq(int irq)
> > +{
> > +     int ret;
> > +
> > +     ret = __cpu_try_get_lock(1);
> > +     if (!ret) {
> > +             set_irq_pending(irq);
> > +             cpu.irqs_pending = true;
> > +     }
> > +     __cpu_try_get_unlock(ret, 1);
> > +
> > +     return ret;
> > +}
> > +
> > +static void lkl_cpu_shutdown(void)
> > +{
> > +     __sync_fetch_and_add(&cpu.shutdown_gate, MAX_THREADS);
> > +}
> > +__uml_exitcall(lkl_cpu_shutdown);
> > +
> > +void lkl_cpu_wait_shutdown(void)
> > +{
> > +     lkl_ops->sem_down(cpu.shutdown_sem);
> > +     lkl_ops->sem_free(cpu.shutdown_sem);
> > +}
> > +
> > +static void lkl_cpu_cleanup(bool shutdown)
> > +{
> > +     while (__sync_fetch_and_add(&cpu.shutdown_gate, 0) > MAX_THREADS)
> > +             ;
> > +
> > +     if (shutdown)
> > +             lkl_ops->sem_up(cpu.shutdown_sem);
> > +     else if (cpu.shutdown_sem)
> > +             lkl_ops->sem_free(cpu.shutdown_sem);
> > +     if (cpu.sem)
> > +             lkl_ops->sem_free(cpu.sem);
> > +     if (cpu.lock)
> > +             lkl_ops->mutex_free(cpu.lock);
> > +}
>
> Yeah, what? That's an incomprehensible piece of code. At least add
> comments, if it _really_ is necessary?
>

Yeah, sorry about that. We missed adding a bunch of comments in the
commit message. It got this complicated because of optimizations to
avoid context switching between the native thread running the
application and the kernel thread running the system call or interrupt
handler.

Maybe we should revert to the initial simpler implementation for now
and add it later?

> > +#ifdef doesntwork
> > +     /* switch to idle_host_task */
> > +     wakeup_idle_host_task();
> > +#endif
>
> Well ...
>
> > +/**
> > + * This is called before the kernel initializes, so no kernel calls (including
> > + * printk) can't be made yet.
> > + */
>
> not kernel-doc
>
> try to compile with W=1 :)
>
> johannes
>

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 11/21] um: nommu: kernel thread support
@ 2020-10-08 18:54               ` Octavian Purdila
  0 siblings, 0 replies; 476+ messages in thread
From: Octavian Purdila @ 2020-10-08 18:54 UTC (permalink / raw)
  To: Johannes Berg
  Cc: linux-arch, Richard Weinberger, jdike, linux-um, Akira Moroo,
	linux-kernel-library, Hajime Tazaki, Anton Ivanov

On Wed, Oct 7, 2020 at 9:57 PM Johannes Berg <johannes@sipsolutions.net> wrote:
>
> On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> > nommu mode does not support user processes
>
> I find this really confusing. I'm not sure why you ended up calling this
> "nommu mode", but there *are* (still) (other) nommu arches, and they
> *do* support userspace processes.
>
> Isn't this really just "LKL mode" or something like that?
>

This is a very good point, while some other patches make sense in the
nommu mode, this one does not - it is rather needed because of the
"library mode".

Not sure what we can do other than creating a new "library mode" in
addition to the "nommu mode". Any suggestions?

> >  #define TIF_SYSCALL_TRACE    0       /* syscall trace active */
> > @@ -63,6 +85,8 @@ static inline struct thread_info *current_thread_info(void)
> >  #define TIF_RESTORE_SIGMASK  7
> >  #define TIF_NOTIFY_RESUME    8
> >  #define TIF_SECCOMP          9       /* secure computing */
> > +#define TIF_SCHED_JB         10
> > +#define TIF_HOST_THREAD              11
>
> It'd be nice to document what those mean, and even what "JB" means ... I
> saw something about "jump buffer" somewhere, but I have no idea why that
> should be a thread flag.
>
> > @@ -16,11 +16,65 @@ struct lkl_jmp_buf {
> >   * These operations must be provided by a host library or by the application
> >   * itself.
> >   *
> > + * @sem_alloc - allocate a host semaphore an initialize it to count
> > + * @sem_free - free a host semaphore
> > + * @sem_up - perform an up operation on the semaphore
> > + * @sem_down - perform a down operation on the semaphore
> > + *
> > + * @mutex_alloc - allocate and initialize a host mutex; the recursive parameter
> > + * determines if the mutex is recursive or not
> > + * @mutex_free - free a host mutex
> > + * @mutex_lock - acquire the mutex
> > + * @mutex_unlock - release the mutex
> > + *
> > + * @thread_create - create a new thread and run f(arg) in its context; returns a
> > + * thread handle or 0 if the thread could not be created
> > + * @thread_detach - on POSIX systems, free up resources held by
> > + * pthreads. Noop on Win32.
> > + * @thread_exit - terminates the current thread
> > + * @thread_join - wait for the given thread to terminate. Returns 0
> > + * for success, -1 otherwise
> > + *
> > + * @gettid - returns the host thread id of the caller, which need not
> > + * be the same as the handle returned by thread_create
> > + *
> > + * @jmp_buf_set - runs the give function and setups a jump back point by saving
> > + * the context in the jump buffer; jmp_buf_longjmp can be called from the give
> > + * function or any callee in that function to return back to the jump back
> > + * point
> > + *
> > + * NOTE: we can't return from jmp_buf_set before calling jmp_buf_longjmp or
> > + * otherwise the saved context (stack) is not going to be valid, so we must pass
> > + * the function that will eventually call longjmp here
> > + *
> > + * @jmp_buf_longjmp - perform a jump back to the saved jump buffer
> > + *
> >   * @mem_alloc - allocate memory
> >   * @mem_free - free memory
>
> again, kernel-doc.
>
> But I'm starting to doubt the value of having this struct at all. Care
> you explain? You're doing everything else already with weak functions,
> and you can't very well have _two_ hosts compiled anyway, so what's the
> point?
>
> IOW, why isn't this just
>
> void lkl_sem_free(struct lkl_sem *sem);
> void lkl_sem_up(struct lkl_sem *sem);
> ...
>
> and then posix-host.c just includes the header file and implements those
> functions?
>
> I don't see any reason for this to be allowed to have multiple variants
> linked and then picking them at runtime?
>

We could try that and see how it goes. This was baked liked this long
time ago, when we wanted to support Windows and there was no proper
support for weak functions in mingw for PE/COFF (it still not
supported but at least we do have a few patches that fix that).

> > +/*
> > + * This structure is used to get access to the "LKL CPU" that allows us to run
>
> Are you trying to implement SMP? This seems ... rather complex?
>
> > + * Linux code. Because we have to deal with various synchronization requirements
> > + * between idle thread, system calls, interrupts, "reentrancy", CPU shutdown,
> > + * imbalance wake up (i.e. acquire the CPU from one thread and release it from
> > + * another), we can't use a simple synchronization mechanism such as (recursive)
> > + * mutex or semaphore. Instead, we use a mutex and a bunch of status data plus a
> > + * semaphore.
> > + */
> > +static struct lkl_cpu {
> > +     /* lock that protects the CPU status data */
> > +     struct lkl_mutex *lock;
> > +     /*
> > +      * Since we must free the cpu lock during shutdown we need a
> > +      * synchronization algorithm between lkl_cpu_shutdown() and the CPU
> > +      * access functions since lkl_cpu_get() gets called from thread
> > +      * destructor callback functions which may be scheduled after
> > +      * lkl_cpu_shutdown() has freed the cpu lock.
> > +      *
> > +      * An atomic counter is used to keep track of the number of running
> > +      * CPU access functions and allow the shutdown function to wait for
> > +      * them.
> > +      *
> > +      * The shutdown functions adds MAX_THREADS to this counter which allows
> > +      * the CPU access functions to check if the shutdown process has
> > +      * started.
> > +      *
> > +      * This algorithm assumes that we never have more the MAX_THREADS
> > +      * requesting CPU access.
> > +      */
> > +     #define MAX_THREADS 1000000
> > +     unsigned int shutdown_gate;
> > +     bool irqs_pending;
> > +     /* no of threads waiting the CPU */
> > +     unsigned int sleepers;
> > +     /* no of times the current thread got the CPU */
> > +     unsigned int count;
> > +     /* current thread that owns the CPU */
> > +     lkl_thread_t owner;
> > +     /* semaphore for threads waiting the CPU */
> > +     struct lkl_sem *sem;
> > +     /* semaphore used for shutdown */
> > +     struct lkl_sem *shutdown_sem;
> > +} cpu;
> > +
> > +static int __cpu_try_get_lock(int n)
> > +{
> > +     lkl_thread_t self;
> > +
> > +     if (__sync_fetch_and_add(&cpu.shutdown_gate, n) >= MAX_THREADS)
> > +             return -2;
> > +
> > +     lkl_ops->mutex_lock(cpu.lock);
> > +
> > +     if (cpu.shutdown_gate >= MAX_THREADS)
> > +             return -1;
> > +
> > +     self = lkl_ops->thread_self();
> > +
> > +     if (cpu.owner && !lkl_ops->thread_equal(cpu.owner, self))
> > +             return 0;
> > +
> > +     cpu.owner = self;
> > +     cpu.count++;
> > +
> > +     return 1;
> > +}
> > +
> > +static void __cpu_try_get_unlock(int lock_ret, int n)
> > +{
> > +     if (lock_ret >= -1)
> > +             lkl_ops->mutex_unlock(cpu.lock);
> > +     __sync_fetch_and_sub(&cpu.shutdown_gate, n);
> > +}
> > +
> > +void lkl_cpu_change_owner(lkl_thread_t owner)
> > +{
> > +     lkl_ops->mutex_lock(cpu.lock);
> > +     if (cpu.count > 1)
> > +             lkl_bug("bad count while changing owner\n");
> > +     cpu.owner = owner;
> > +     lkl_ops->mutex_unlock(cpu.lock);
> > +}
> > +
> > +int lkl_cpu_get(void)
> > +{
> > +     int ret;
> > +
> > +     ret = __cpu_try_get_lock(1);
> > +
> > +     while (ret == 0) {
> > +             cpu.sleepers++;
> > +             __cpu_try_get_unlock(ret, 0);
> > +             lkl_ops->sem_down(cpu.sem);
> > +             ret = __cpu_try_get_lock(0);
> > +     }
> > +
> > +     __cpu_try_get_unlock(ret, 1);
> > +
> > +     return ret;
> > +}
> > +
> > +void lkl_cpu_put(void)
> > +{
> > +     lkl_ops->mutex_lock(cpu.lock);
> > +
> > +     if (!cpu.count || !cpu.owner ||
> > +         !lkl_ops->thread_equal(cpu.owner, lkl_ops->thread_self()))
> > +             lkl_bug("%s: unbalanced put\n", __func__);
> > +
> > +     while (cpu.irqs_pending && !irqs_disabled()) {
> > +             cpu.irqs_pending = false;
> > +             lkl_ops->mutex_unlock(cpu.lock);
> > +             run_irqs();
> > +             lkl_ops->mutex_lock(cpu.lock);
> > +     }
> > +
> > +     if (test_ti_thread_flag(current_thread_info(), TIF_HOST_THREAD) &&
> > +         !single_task_running() && cpu.count == 1) {
> > +             if (in_interrupt())
> > +                     lkl_bug("%s: in interrupt\n", __func__);
> > +             lkl_ops->mutex_unlock(cpu.lock);
> > +             thread_sched_jb();
> > +             return;
> > +     }
> > +
> > +     if (--cpu.count > 0) {
> > +             lkl_ops->mutex_unlock(cpu.lock);
> > +             return;
> > +     }
> > +
> > +     if (cpu.sleepers) {
> > +             cpu.sleepers--;
> > +             lkl_ops->sem_up(cpu.sem);
> > +     }
> > +
> > +     cpu.owner = 0;
> > +
> > +     lkl_ops->mutex_unlock(cpu.lock);
> > +}
> > +
> > +int lkl_cpu_try_run_irq(int irq)
> > +{
> > +     int ret;
> > +
> > +     ret = __cpu_try_get_lock(1);
> > +     if (!ret) {
> > +             set_irq_pending(irq);
> > +             cpu.irqs_pending = true;
> > +     }
> > +     __cpu_try_get_unlock(ret, 1);
> > +
> > +     return ret;
> > +}
> > +
> > +static void lkl_cpu_shutdown(void)
> > +{
> > +     __sync_fetch_and_add(&cpu.shutdown_gate, MAX_THREADS);
> > +}
> > +__uml_exitcall(lkl_cpu_shutdown);
> > +
> > +void lkl_cpu_wait_shutdown(void)
> > +{
> > +     lkl_ops->sem_down(cpu.shutdown_sem);
> > +     lkl_ops->sem_free(cpu.shutdown_sem);
> > +}
> > +
> > +static void lkl_cpu_cleanup(bool shutdown)
> > +{
> > +     while (__sync_fetch_and_add(&cpu.shutdown_gate, 0) > MAX_THREADS)
> > +             ;
> > +
> > +     if (shutdown)
> > +             lkl_ops->sem_up(cpu.shutdown_sem);
> > +     else if (cpu.shutdown_sem)
> > +             lkl_ops->sem_free(cpu.shutdown_sem);
> > +     if (cpu.sem)
> > +             lkl_ops->sem_free(cpu.sem);
> > +     if (cpu.lock)
> > +             lkl_ops->mutex_free(cpu.lock);
> > +}
>
> Yeah, what? That's an incomprehensible piece of code. At least add
> comments, if it _really_ is necessary?
>

Yeah, sorry about that. We missed adding a bunch of comments in the
commit message. It got this complicated because of optimizations to
avoid context switching between the native thread running the
application and the kernel thread running the system call or interrupt
handler.

Maybe we should revert to the initial simpler implementation for now
and add it later?

> > +#ifdef doesntwork
> > +     /* switch to idle_host_task */
> > +     wakeup_idle_host_task();
> > +#endif
>
> Well ...
>
> > +/**
> > + * This is called before the kernel initializes, so no kernel calls (including
> > + * printk) can't be made yet.
> > + */
>
> not kernel-doc
>
> try to compile with W=1 :)
>
> johannes
>

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 12/21] um: nommu: system call interface and application API
  2020-10-07 19:05             ` Johannes Berg
@ 2020-10-08 19:03               ` Octavian Purdila
  -1 siblings, 0 replies; 476+ messages in thread
From: Octavian Purdila @ 2020-10-08 19:03 UTC (permalink / raw)
  To: Johannes Berg
  Cc: Hajime Tazaki, linux-um, jdike, Richard Weinberger, Anton Ivanov,
	linux-kernel-library, linux-arch, Akira Moroo

On Wed, Oct 7, 2020 at 10:05 PM Johannes Berg <johannes@sipsolutions.net> wrote:
>
> On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> >
> > +++ b/arch/um/nommu/include/uapi/asm/syscalls.h
> > @@ -0,0 +1,287 @@
> > +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
>
> That doesn't really make sense - if you use LKL you're linking Linux, so
> you can't use this "WITH Linux-syscall-note"?
>
> > +#ifndef __UM_NOMMU_UAPI_SYSCALLS_H
> > +#define __UM_NOMMU_UAPI_SYSCALLS_H
>
> [snip]
>
> This file really worries me, it includes half the world and (re)defines
> the other half ... How is this ever going to be maintained?
>

There are not that many definitions here, just the ones that were
never defined in uapi headers. And, AFAIK, new code that exposes
structures and data types should always go  into uapi headers and not
directly in glibc, etc. So once we fix the old stuff, it should be
fine?

> > index 000000000000..ec7356c0dee9
> > --- /dev/null
> > +++ b/arch/um/scripts/headers_install.py
> > @@ -0,0 +1,197 @@
> > +#!/usr/bin/env python
>
> might want to make that explicitly 'python3', some newer distros (e.g.
> ubuntu 20.04) are now shipping without a 'python' by default.
>

Good point, will fix it in the next patch series.

> > +def has_lkl_prefix(w):
> > +  return w.startswith("lkl") or w.startswith("_lkl") or w.startswith("__lkl") \
> > +         or w.startswith("LKL") or w.startswith("_LKL") or w.startswith("__LKL")
>
>
> > +        content = re.sub(re.compile("(\/\*(\*(?!\/)|[^*])*\*\/)", re.S|re.M), " ", open(h).read())
> >
> > +    dir = os.path.dirname(h)
> > +    out_dir = args.path + "/" + re.sub("(" + srctree + "/arch/um/nommu/include/uapi/|arch/um/nommu/include/generated/uapi/|include/generated/uapi/|include/generated|" + install_hdr_path + "/include/)(.*)", "lkl/\\2", dir)
>
>
> you have some very long lines in places, I'm sure you could fix that
> (e.g. the last one by doing something with '|'.join([...]))
>

Thanks for pointing it out, we will fix it :)

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 12/21] um: nommu: system call interface and application API
@ 2020-10-08 19:03               ` Octavian Purdila
  0 siblings, 0 replies; 476+ messages in thread
From: Octavian Purdila @ 2020-10-08 19:03 UTC (permalink / raw)
  To: Johannes Berg
  Cc: linux-arch, Richard Weinberger, jdike, linux-um, Akira Moroo,
	linux-kernel-library, Hajime Tazaki, Anton Ivanov

On Wed, Oct 7, 2020 at 10:05 PM Johannes Berg <johannes@sipsolutions.net> wrote:
>
> On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> >
> > +++ b/arch/um/nommu/include/uapi/asm/syscalls.h
> > @@ -0,0 +1,287 @@
> > +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
>
> That doesn't really make sense - if you use LKL you're linking Linux, so
> you can't use this "WITH Linux-syscall-note"?
>
> > +#ifndef __UM_NOMMU_UAPI_SYSCALLS_H
> > +#define __UM_NOMMU_UAPI_SYSCALLS_H
>
> [snip]
>
> This file really worries me, it includes half the world and (re)defines
> the other half ... How is this ever going to be maintained?
>

There are not that many definitions here, just the ones that were
never defined in uapi headers. And, AFAIK, new code that exposes
structures and data types should always go  into uapi headers and not
directly in glibc, etc. So once we fix the old stuff, it should be
fine?

> > index 000000000000..ec7356c0dee9
> > --- /dev/null
> > +++ b/arch/um/scripts/headers_install.py
> > @@ -0,0 +1,197 @@
> > +#!/usr/bin/env python
>
> might want to make that explicitly 'python3', some newer distros (e.g.
> ubuntu 20.04) are now shipping without a 'python' by default.
>

Good point, will fix it in the next patch series.

> > +def has_lkl_prefix(w):
> > +  return w.startswith("lkl") or w.startswith("_lkl") or w.startswith("__lkl") \
> > +         or w.startswith("LKL") or w.startswith("_LKL") or w.startswith("__LKL")
>
>
> > +        content = re.sub(re.compile("(\/\*(\*(?!\/)|[^*])*\*\/)", re.S|re.M), " ", open(h).read())
> >
> > +    dir = os.path.dirname(h)
> > +    out_dir = args.path + "/" + re.sub("(" + srctree + "/arch/um/nommu/include/uapi/|arch/um/nommu/include/generated/uapi/|include/generated/uapi/|include/generated|" + install_hdr_path + "/include/)(.*)", "lkl/\\2", dir)
>
>
> you have some very long lines in places, I'm sure you could fix that
> (e.g. the last one by doing something with '|'.join([...]))
>

Thanks for pointing it out, we will fix it :)

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 18/21] um: host: add utilities functions
  2020-10-08 12:52                   ` Hajime Tazaki
@ 2020-10-08 19:19                       ` Octavian Purdila
  0 siblings, 0 replies; 476+ messages in thread
From: Octavian Purdila @ 2020-10-08 19:19 UTC (permalink / raw)
  To: Hajime Tazaki
  Cc: Anton Ivanov, Johannes Berg, linux-um, jdike, Richard Weinberger,
	linux-kernel-library, Akira Moroo, linux-arch

On Thu, Oct 8, 2020 at 3:52 PM Hajime Tazaki <thehajime@gmail.com> wrote:
>
>
> On Thu, 08 Oct 2020 00:10:23 +0900,
> Anton Ivanov wrote:
> >
> >
> > On 07/10/2020 16:03, Johannes Berg wrote:
> > > On Wed, 2020-10-07 at 17:02 +0200, Johannes Berg wrote:
> > >> On Wed, 2020-10-07 at 15:53 +0100, Anton Ivanov wrote:
> > >>> These are actually different on different architectures. These look
> > >>> like the x86 values.
>
> Those variables are based on tools/um/include/lkl/asm-generic/errno.h,
> which is generated from include/uapi/asm-generic/errno.h, which LKL
> kernel eats for its values.
>
> This is the part of patch 12/21.
>
> > >>> IMHO a kernel strerror() would be the right way of dealing with this
> > >>> in the long term (i understand that we cannot call the platform one,
> > >>> because it may be different from the internal Linux errors). It will
> > >>> be useful in a lot of other places.
> > >>>
> > >>> If we leave it as is, we need to make this arch specific at some
> > >>> point.
>
> So, this particular code does not contain arch specific part I
> believe.
> # Tavi, correct me if I'm wrong.
>

I think Anton was saying that we should do it properly with C99 to
avoid potential issues with changes in configuration and I agree that
is the way to go.

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 18/21] um: host: add utilities functions
@ 2020-10-08 19:19                       ` Octavian Purdila
  0 siblings, 0 replies; 476+ messages in thread
From: Octavian Purdila @ 2020-10-08 19:19 UTC (permalink / raw)
  To: Hajime Tazaki
  Cc: linux-arch, Richard Weinberger, jdike, linux-um, Akira Moroo,
	linux-kernel-library, Johannes Berg, Anton Ivanov

On Thu, Oct 8, 2020 at 3:52 PM Hajime Tazaki <thehajime@gmail.com> wrote:
>
>
> On Thu, 08 Oct 2020 00:10:23 +0900,
> Anton Ivanov wrote:
> >
> >
> > On 07/10/2020 16:03, Johannes Berg wrote:
> > > On Wed, 2020-10-07 at 17:02 +0200, Johannes Berg wrote:
> > >> On Wed, 2020-10-07 at 15:53 +0100, Anton Ivanov wrote:
> > >>> These are actually different on different architectures. These look
> > >>> like the x86 values.
>
> Those variables are based on tools/um/include/lkl/asm-generic/errno.h,
> which is generated from include/uapi/asm-generic/errno.h, which LKL
> kernel eats for its values.
>
> This is the part of patch 12/21.
>
> > >>> IMHO a kernel strerror() would be the right way of dealing with this
> > >>> in the long term (i understand that we cannot call the platform one,
> > >>> because it may be different from the internal Linux errors). It will
> > >>> be useful in a lot of other places.
> > >>>
> > >>> If we leave it as is, we need to make this arch specific at some
> > >>> point.
>
> So, this particular code does not contain arch specific part I
> believe.
> # Tavi, correct me if I'm wrong.
>

I think Anton was saying that we should do it properly with C99 to
avoid potential issues with changes in configuration and I agree that
is the way to go.

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 11/21] um: nommu: kernel thread support
  2020-10-08 18:54               ` Octavian Purdila
@ 2020-10-08 19:39                 ` Johannes Berg
  -1 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-08 19:39 UTC (permalink / raw)
  To: Octavian Purdila
  Cc: Hajime Tazaki, linux-um, jdike, Richard Weinberger, Anton Ivanov,
	linux-kernel-library, linux-arch, Akira Moroo

On Thu, 2020-10-08 at 21:54 +0300, Octavian Purdila wrote:
> On Wed, Oct 7, 2020 at 9:57 PM Johannes Berg <johannes@sipsolutions.net> wrote:
> > On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> > > nommu mode does not support user processes
> > 
> > I find this really confusing. I'm not sure why you ended up calling this
> > "nommu mode", but there *are* (still) (other) nommu arches, and they
> > *do* support userspace processes.
> > 
> > Isn't this really just "LKL mode" or something like that?
> > 
> 
> This is a very good point, while some other patches make sense in the
> nommu mode, this one does not - it is rather needed because of the
> "library mode".
> 
> Not sure what we can do other than creating a new "library mode" in
> addition to the "nommu mode". Any suggestions?

Well there's no "nommu mode" in UML other than what you're doing here,
so as I said on some other patch, it sort of makes sense to have "LKL ==
NOMMU", but the equation doesn't make sense everywhere, since it's not
fundamentally NOMMU that drives the need for things (like here no
userspace, elsewhere the ifdefs, etc.), but LKL-mode.

So I don't think it would be *in addition* to "nommu mode" since such a
thing doesn't exist on UML (only on other architectures), but mostly be
a rename of "nommu mode" to "lkl mode" or so?

Don't really have any other suggestions, or maybe I'm not understanding
your question right.

> > IOW, why isn't this just
> > 
> > void lkl_sem_free(struct lkl_sem *sem);
> > void lkl_sem_up(struct lkl_sem *sem);
> > ...
> > 
> > and then posix-host.c just includes the header file and implements those
> > functions?
> > 
> > I don't see any reason for this to be allowed to have multiple variants
> > linked and then picking them at runtime?
> > 
> 
> We could try that and see how it goes. This was baked liked this long
> time ago, when we wanted to support Windows and there was no proper
> support for weak functions in mingw for PE/COFF (it still not
> supported but at least we do have a few patches that fix that).

You've required weak functions elsewhere, but in this case you don't
even need them since you don't need things to link without an
implementation? At least I don't see why you'd want to be able to link a
binary that doesn't have an implementation of the ops required to run?

> > Yeah, what? That's an incomprehensible piece of code. At least add
> > comments, if it _really_ is necessary?
> > 
> 
> Yeah, sorry about that. We missed adding a bunch of comments in the
> commit message. It got this complicated because of optimizations to
> avoid context switching between the native thread running the
> application and the kernel thread running the system call or interrupt
> handler.
> 
> Maybe we should revert to the initial simpler implementation for now
> and add it later?

Perhaps? Not really sure. Could the optimisations be added in steps so
they're something that can be explained/followed? If not, well, perhaps
to ease review for now it'd make sense to start simpler, but I guess
eventually it'd still want some better explanation of what's going on.

johannes


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 11/21] um: nommu: kernel thread support
@ 2020-10-08 19:39                 ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-08 19:39 UTC (permalink / raw)
  To: Octavian Purdila
  Cc: linux-arch, Richard Weinberger, jdike, linux-um, Akira Moroo,
	linux-kernel-library, Hajime Tazaki, Anton Ivanov

On Thu, 2020-10-08 at 21:54 +0300, Octavian Purdila wrote:
> On Wed, Oct 7, 2020 at 9:57 PM Johannes Berg <johannes@sipsolutions.net> wrote:
> > On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> > > nommu mode does not support user processes
> > 
> > I find this really confusing. I'm not sure why you ended up calling this
> > "nommu mode", but there *are* (still) (other) nommu arches, and they
> > *do* support userspace processes.
> > 
> > Isn't this really just "LKL mode" or something like that?
> > 
> 
> This is a very good point, while some other patches make sense in the
> nommu mode, this one does not - it is rather needed because of the
> "library mode".
> 
> Not sure what we can do other than creating a new "library mode" in
> addition to the "nommu mode". Any suggestions?

Well there's no "nommu mode" in UML other than what you're doing here,
so as I said on some other patch, it sort of makes sense to have "LKL ==
NOMMU", but the equation doesn't make sense everywhere, since it's not
fundamentally NOMMU that drives the need for things (like here no
userspace, elsewhere the ifdefs, etc.), but LKL-mode.

So I don't think it would be *in addition* to "nommu mode" since such a
thing doesn't exist on UML (only on other architectures), but mostly be
a rename of "nommu mode" to "lkl mode" or so?

Don't really have any other suggestions, or maybe I'm not understanding
your question right.

> > IOW, why isn't this just
> > 
> > void lkl_sem_free(struct lkl_sem *sem);
> > void lkl_sem_up(struct lkl_sem *sem);
> > ...
> > 
> > and then posix-host.c just includes the header file and implements those
> > functions?
> > 
> > I don't see any reason for this to be allowed to have multiple variants
> > linked and then picking them at runtime?
> > 
> 
> We could try that and see how it goes. This was baked liked this long
> time ago, when we wanted to support Windows and there was no proper
> support for weak functions in mingw for PE/COFF (it still not
> supported but at least we do have a few patches that fix that).

You've required weak functions elsewhere, but in this case you don't
even need them since you don't need things to link without an
implementation? At least I don't see why you'd want to be able to link a
binary that doesn't have an implementation of the ops required to run?

> > Yeah, what? That's an incomprehensible piece of code. At least add
> > comments, if it _really_ is necessary?
> > 
> 
> Yeah, sorry about that. We missed adding a bunch of comments in the
> commit message. It got this complicated because of optimizations to
> avoid context switching between the native thread running the
> application and the kernel thread running the system call or interrupt
> handler.
> 
> Maybe we should revert to the initial simpler implementation for now
> and add it later?

Perhaps? Not really sure. Could the optimisations be added in steps so
they're something that can be explained/followed? If not, well, perhaps
to ease review for now it'd make sense to start simpler, but I guess
eventually it'd still want some better explanation of what's going on.

johannes


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 12/21] um: nommu: system call interface and application API
  2020-10-08 19:03               ` Octavian Purdila
@ 2020-10-08 19:40                 ` Johannes Berg
  -1 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-08 19:40 UTC (permalink / raw)
  To: Octavian Purdila
  Cc: Hajime Tazaki, linux-um, jdike, Richard Weinberger, Anton Ivanov,
	linux-kernel-library, linux-arch, Akira Moroo

On Thu, 2020-10-08 at 22:03 +0300, Octavian Purdila wrote:
> On Wed, Oct 7, 2020 at 10:05 PM Johannes Berg <johannes@sipsolutions.net> wrote:
> > On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> > > +++ b/arch/um/nommu/include/uapi/asm/syscalls.h
> > > @@ -0,0 +1,287 @@
> > > +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
> > 
> > That doesn't really make sense - if you use LKL you're linking Linux, so
> > you can't use this "WITH Linux-syscall-note"?
> > 
> > > +#ifndef __UM_NOMMU_UAPI_SYSCALLS_H
> > > +#define __UM_NOMMU_UAPI_SYSCALLS_H
> > 
> > [snip]
> > 
> > This file really worries me, it includes half the world and (re)defines
> > the other half ... How is this ever going to be maintained?
> > 
> 
> There are not that many definitions here, just the ones that were
> never defined in uapi headers. And, AFAIK, new code that exposes
> structures and data types should always go  into uapi headers and not
> directly in glibc, etc. So once we fix the old stuff, it should be
> fine?

Yeah, not really sure. Not all of this is really old stuff, e.g. keyring
things were here I noticed.

But since it's userspace API I guess the changes of it breaking are
fairly small anyway, so maybe I just shouldn't worry about it :)

johannes


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 12/21] um: nommu: system call interface and application API
@ 2020-10-08 19:40                 ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-08 19:40 UTC (permalink / raw)
  To: Octavian Purdila
  Cc: linux-arch, Richard Weinberger, jdike, linux-um, Akira Moroo,
	linux-kernel-library, Hajime Tazaki, Anton Ivanov

On Thu, 2020-10-08 at 22:03 +0300, Octavian Purdila wrote:
> On Wed, Oct 7, 2020 at 10:05 PM Johannes Berg <johannes@sipsolutions.net> wrote:
> > On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> > > +++ b/arch/um/nommu/include/uapi/asm/syscalls.h
> > > @@ -0,0 +1,287 @@
> > > +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
> > 
> > That doesn't really make sense - if you use LKL you're linking Linux, so
> > you can't use this "WITH Linux-syscall-note"?
> > 
> > > +#ifndef __UM_NOMMU_UAPI_SYSCALLS_H
> > > +#define __UM_NOMMU_UAPI_SYSCALLS_H
> > 
> > [snip]
> > 
> > This file really worries me, it includes half the world and (re)defines
> > the other half ... How is this ever going to be maintained?
> > 
> 
> There are not that many definitions here, just the ones that were
> never defined in uapi headers. And, AFAIK, new code that exposes
> structures and data types should always go  into uapi headers and not
> directly in glibc, etc. So once we fix the old stuff, it should be
> fine?

Yeah, not really sure. Not all of this is really old stuff, e.g. keyring
things were here I noticed.

But since it's userspace API I guess the changes of it breaking are
fairly small anyway, so maybe I just shouldn't worry about it :)

johannes


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 03/21] um: move arch/um/os-Linux dir to tools/um/uml
  2020-10-08 17:48               ` Octavian Purdila
@ 2020-10-08 19:46                 ` Johannes Berg
  -1 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-08 19:46 UTC (permalink / raw)
  To: Octavian Purdila
  Cc: Hajime Tazaki, linux-um, jdike, Richard Weinberger, Anton Ivanov,
	linux-kernel-library, linux-arch, Akira Moroo

On Thu, 2020-10-08 at 20:48 +0300, Octavian Purdila wrote:

> > On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> > > This patch moves underlying OS dependent part under arch/um to tools/um
> > > directory so that arch/um code does not need to build host build
> > > facilities (e.g., libc).
> > 
> > Hmm. On the one hand, that build separation seems sensible, but on the
> > other hand this stuff *does* fundamentally belong to arch/um/, and it's
> > not a "tool" like basically everything else there that is a pure
> > userspace application to run *inside* the kernel, not *part of* it.
> > 
> > For that reason, I don't really like this much.
> > 
> 
> I see the os_*() calls as dependencies that the kernel uses. Sort of
> like calls into the hypervisor or firmware.

Right.

> The current UML build is already partially split. USER_OBJS build with
> a different rule set than the rest of the kernel objects.

Yes, that's true.

> IMHO this
> change just makes this more clear and streamlined, especially with
> regard to linking.

Well, maybe? But I actually tend to see this less as a question of
(technical) convenience but semantics, and the tools are just not
*meant* to be things that build a kernel, they're things to be used
inside that kernel.

I dunno. Maybe the technical convenience should win, but OTOH the
"contortions" that UML build goes through with USER_OBJS don't really
seem bad enough to merit breaking the notion of what tools are?

> We are using the same build system as the rest of the stuff in tools.
> Since it is building programs and libraries and not part of the kernel
> binary build, it is using a slightly different infrastructure, which
> is detailed in tools/build/Documentation/Build.txt

OK, thanks for the pointer, I hadn't seen that before.

> The reason for picking tools was that it already has the
> infrastructure to build programs and shared libraries and the fact
> that it is the only place in the kernel source tree where code is not
> built directly into the kernel binary.

Yeah, but all of UML/LKL _is_ eventually built into the kernel binary
(or library as it may be). Which is in a way exactly my objection.

You're looking at it the other way around I think - it seems that you're
thinking the kernel binary is the vmlinux.a, and that's what the
kernel's build system worries about; then the "vmlinux.so" (library
mode) or "linux" (standalone mode - perhaps that's a good name?) is the
eventual 'tool' that we build, using the previously built vmlinux.a.

But that really isn't how standalone mode works, and IMHO it also
doesn't match what tools are today.

johannes


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 03/21] um: move arch/um/os-Linux dir to tools/um/uml
@ 2020-10-08 19:46                 ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-08 19:46 UTC (permalink / raw)
  To: Octavian Purdila
  Cc: linux-arch, Richard Weinberger, jdike, linux-um, Akira Moroo,
	linux-kernel-library, Hajime Tazaki, Anton Ivanov

On Thu, 2020-10-08 at 20:48 +0300, Octavian Purdila wrote:

> > On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> > > This patch moves underlying OS dependent part under arch/um to tools/um
> > > directory so that arch/um code does not need to build host build
> > > facilities (e.g., libc).
> > 
> > Hmm. On the one hand, that build separation seems sensible, but on the
> > other hand this stuff *does* fundamentally belong to arch/um/, and it's
> > not a "tool" like basically everything else there that is a pure
> > userspace application to run *inside* the kernel, not *part of* it.
> > 
> > For that reason, I don't really like this much.
> > 
> 
> I see the os_*() calls as dependencies that the kernel uses. Sort of
> like calls into the hypervisor or firmware.

Right.

> The current UML build is already partially split. USER_OBJS build with
> a different rule set than the rest of the kernel objects.

Yes, that's true.

> IMHO this
> change just makes this more clear and streamlined, especially with
> regard to linking.

Well, maybe? But I actually tend to see this less as a question of
(technical) convenience but semantics, and the tools are just not
*meant* to be things that build a kernel, they're things to be used
inside that kernel.

I dunno. Maybe the technical convenience should win, but OTOH the
"contortions" that UML build goes through with USER_OBJS don't really
seem bad enough to merit breaking the notion of what tools are?

> We are using the same build system as the rest of the stuff in tools.
> Since it is building programs and libraries and not part of the kernel
> binary build, it is using a slightly different infrastructure, which
> is detailed in tools/build/Documentation/Build.txt

OK, thanks for the pointer, I hadn't seen that before.

> The reason for picking tools was that it already has the
> infrastructure to build programs and shared libraries and the fact
> that it is the only place in the kernel source tree where code is not
> built directly into the kernel binary.

Yeah, but all of UML/LKL _is_ eventually built into the kernel binary
(or library as it may be). Which is in a way exactly my objection.

You're looking at it the other way around I think - it seems that you're
thinking the kernel binary is the vmlinux.a, and that's what the
kernel's build system worries about; then the "vmlinux.so" (library
mode) or "linux" (standalone mode - perhaps that's a good name?) is the
eventual 'tool' that we build, using the previously built vmlinux.a.

But that really isn't how standalone mode works, and IMHO it also
doesn't match what tools are today.

johannes


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 11/21] um: nommu: kernel thread support
  2020-10-08 19:39                 ` Johannes Berg
@ 2020-10-08 20:25                   ` Octavian Purdila
  -1 siblings, 0 replies; 476+ messages in thread
From: Octavian Purdila @ 2020-10-08 20:25 UTC (permalink / raw)
  To: Johannes Berg
  Cc: Hajime Tazaki, linux-um, jdike, Richard Weinberger, Anton Ivanov,
	linux-kernel-library, linux-arch, Akira Moroo

On Thu, Oct 8, 2020 at 10:39 PM Johannes Berg <johannes@sipsolutions.net> wrote:
>
> On Thu, 2020-10-08 at 21:54 +0300, Octavian Purdila wrote:
> > On Wed, Oct 7, 2020 at 9:57 PM Johannes Berg <johannes@sipsolutions.net> wrote:
> > > On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> > > > nommu mode does not support user processes
> > >
> > > I find this really confusing. I'm not sure why you ended up calling this
> > > "nommu mode", but there *are* (still) (other) nommu arches, and they
> > > *do* support userspace processes.
> > >
> > > Isn't this really just "LKL mode" or something like that?
> > >
> >
> > This is a very good point, while some other patches make sense in the
> > nommu mode, this one does not - it is rather needed because of the
> > "library mode".
> >
> > Not sure what we can do other than creating a new "library mode" in
> > addition to the "nommu mode". Any suggestions?
>
> Well there's no "nommu mode" in UML other than what you're doing here,
> so as I said on some other patch, it sort of makes sense to have "LKL ==
> NOMMU", but the equation doesn't make sense everywhere, since it's not
> fundamentally NOMMU that drives the need for things (like here no
> userspace, elsewhere the ifdefs, etc.), but LKL-mode.
>
> So I don't think it would be *in addition* to "nommu mode" since such a
> thing doesn't exist on UML (only on other architectures), but mostly be
> a rename of "nommu mode" to "lkl mode" or so?
>
> Don't really have any other suggestions, or maybe I'm not understanding
> your question right.

OK, I agree, renaming "nommu mode" to "lkl mode" looks like the right
thing to do for now.

>
> > > IOW, why isn't this just
> > >
> > > void lkl_sem_free(struct lkl_sem *sem);
> > > void lkl_sem_up(struct lkl_sem *sem);
> > > ...
> > >
> > > and then posix-host.c just includes the header file and implements those
> > > functions?
> > >
> > > I don't see any reason for this to be allowed to have multiple variants
> > > linked and then picking them at runtime?
> > >
> >
> > We could try that and see how it goes. This was baked liked this long
> > time ago, when we wanted to support Windows and there was no proper
> > support for weak functions in mingw for PE/COFF (it still not
> > supported but at least we do have a few patches that fix that).
>
> You've required weak functions elsewhere, but in this case you don't
> even need them since you don't need things to link without an
> implementation? At least I don't see why you'd want to be able to link a
> binary that doesn't have an implementation of the ops required to run?

Yeah, all good points :) I'll discuss it more with Hajime to make sure
I haven't missed anything and we will try it in the next patch series.


> > > Yeah, what? That's an incomprehensible piece of code. At least add
> > > comments, if it _really_ is necessary?
> > >
> >
> > Yeah, sorry about that. We missed adding a bunch of comments in the
> > commit message. It got this complicated because of optimizations to
> > avoid context switching between the native thread running the
> > application and the kernel thread running the system call or interrupt
> > handler.
> >
> > Maybe we should revert to the initial simpler implementation for now
> > and add it later?
>
> Perhaps? Not really sure. Could the optimisations be added in steps so
> they're something that can be explained/followed? If not, well, perhaps
> to ease review for now it'd make sense to start simpler, but I guess
> eventually it'd still want some better explanation of what's going on.
>

OK, I'll discuss more with Hajime, at this point I think we might want
to focus on getting the basics merged first. In either case we will
make sure to have it properly explained.

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 11/21] um: nommu: kernel thread support
@ 2020-10-08 20:25                   ` Octavian Purdila
  0 siblings, 0 replies; 476+ messages in thread
From: Octavian Purdila @ 2020-10-08 20:25 UTC (permalink / raw)
  To: Johannes Berg
  Cc: linux-arch, Richard Weinberger, jdike, linux-um, Akira Moroo,
	linux-kernel-library, Hajime Tazaki, Anton Ivanov

On Thu, Oct 8, 2020 at 10:39 PM Johannes Berg <johannes@sipsolutions.net> wrote:
>
> On Thu, 2020-10-08 at 21:54 +0300, Octavian Purdila wrote:
> > On Wed, Oct 7, 2020 at 9:57 PM Johannes Berg <johannes@sipsolutions.net> wrote:
> > > On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> > > > nommu mode does not support user processes
> > >
> > > I find this really confusing. I'm not sure why you ended up calling this
> > > "nommu mode", but there *are* (still) (other) nommu arches, and they
> > > *do* support userspace processes.
> > >
> > > Isn't this really just "LKL mode" or something like that?
> > >
> >
> > This is a very good point, while some other patches make sense in the
> > nommu mode, this one does not - it is rather needed because of the
> > "library mode".
> >
> > Not sure what we can do other than creating a new "library mode" in
> > addition to the "nommu mode". Any suggestions?
>
> Well there's no "nommu mode" in UML other than what you're doing here,
> so as I said on some other patch, it sort of makes sense to have "LKL ==
> NOMMU", but the equation doesn't make sense everywhere, since it's not
> fundamentally NOMMU that drives the need for things (like here no
> userspace, elsewhere the ifdefs, etc.), but LKL-mode.
>
> So I don't think it would be *in addition* to "nommu mode" since such a
> thing doesn't exist on UML (only on other architectures), but mostly be
> a rename of "nommu mode" to "lkl mode" or so?
>
> Don't really have any other suggestions, or maybe I'm not understanding
> your question right.

OK, I agree, renaming "nommu mode" to "lkl mode" looks like the right
thing to do for now.

>
> > > IOW, why isn't this just
> > >
> > > void lkl_sem_free(struct lkl_sem *sem);
> > > void lkl_sem_up(struct lkl_sem *sem);
> > > ...
> > >
> > > and then posix-host.c just includes the header file and implements those
> > > functions?
> > >
> > > I don't see any reason for this to be allowed to have multiple variants
> > > linked and then picking them at runtime?
> > >
> >
> > We could try that and see how it goes. This was baked liked this long
> > time ago, when we wanted to support Windows and there was no proper
> > support for weak functions in mingw for PE/COFF (it still not
> > supported but at least we do have a few patches that fix that).
>
> You've required weak functions elsewhere, but in this case you don't
> even need them since you don't need things to link without an
> implementation? At least I don't see why you'd want to be able to link a
> binary that doesn't have an implementation of the ops required to run?

Yeah, all good points :) I'll discuss it more with Hajime to make sure
I haven't missed anything and we will try it in the next patch series.


> > > Yeah, what? That's an incomprehensible piece of code. At least add
> > > comments, if it _really_ is necessary?
> > >
> >
> > Yeah, sorry about that. We missed adding a bunch of comments in the
> > commit message. It got this complicated because of optimizations to
> > avoid context switching between the native thread running the
> > application and the kernel thread running the system call or interrupt
> > handler.
> >
> > Maybe we should revert to the initial simpler implementation for now
> > and add it later?
>
> Perhaps? Not really sure. Could the optimisations be added in steps so
> they're something that can be explained/followed? If not, well, perhaps
> to ease review for now it'd make sense to start simpler, but I guess
> eventually it'd still want some better explanation of what's going on.
>

OK, I'll discuss more with Hajime, at this point I think we might want
to focus on getting the basics merged first. In either case we will
make sure to have it properly explained.

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 03/21] um: move arch/um/os-Linux dir to tools/um/uml
  2020-10-08 19:46                 ` Johannes Berg
@ 2020-10-08 20:53                   ` Octavian Purdila
  -1 siblings, 0 replies; 476+ messages in thread
From: Octavian Purdila @ 2020-10-08 20:53 UTC (permalink / raw)
  To: Johannes Berg
  Cc: Hajime Tazaki, linux-um, jdike, Richard Weinberger, Anton Ivanov,
	linux-kernel-library, linux-arch, Akira Moroo

On Thu, Oct 8, 2020 at 10:46 PM Johannes Berg <johannes@sipsolutions.net> wrote:
>
> On Thu, 2020-10-08 at 20:48 +0300, Octavian Purdila wrote:
>
> > > On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> > > > This patch moves underlying OS dependent part under arch/um to tools/um
> > > > directory so that arch/um code does not need to build host build
> > > > facilities (e.g., libc).
> > >
> > > Hmm. On the one hand, that build separation seems sensible, but on the
> > > other hand this stuff *does* fundamentally belong to arch/um/, and it's
> > > not a "tool" like basically everything else there that is a pure
> > > userspace application to run *inside* the kernel, not *part of* it.
> > >
> > > For that reason, I don't really like this much.
> > >
> >
> > I see the os_*() calls as dependencies that the kernel uses. Sort of
> > like calls into the hypervisor or firmware.
>
> Right.
>
> > The current UML build is already partially split. USER_OBJS build with
> > a different rule set than the rest of the kernel objects.
>
> Yes, that's true.
>
> > IMHO this
> > change just makes this more clear and streamlined, especially with
> > regard to linking.
>
> Well, maybe? But I actually tend to see this less as a question of
> (technical) convenience but semantics, and the tools are just not
> *meant* to be things that build a kernel, they're things to be used
> inside that kernel.
>
> I dunno. Maybe the technical convenience should win, but OTOH the
> "contortions" that UML build goes through with USER_OBJS don't really
> seem bad enough to merit breaking the notion of what tools are?
>

Half-joking, technically uml is not a kernel, it is a tool that
simulates a kernel and it uses part of the Linux kernel to do that :)

> > We are using the same build system as the rest of the stuff in tools.
> > Since it is building programs and libraries and not part of the kernel
> > binary build, it is using a slightly different infrastructure, which
> > is detailed in tools/build/Documentation/Build.txt
>
> OK, thanks for the pointer, I hadn't seen that before.
>
> > The reason for picking tools was that it already has the
> > infrastructure to build programs and shared libraries and the fact
> > that it is the only place in the kernel source tree where code is not
> > built directly into the kernel binary.
>
> Yeah, but all of UML/LKL _is_ eventually built into the kernel binary
> (or library as it may be). Which is in a way exactly my objection.
>

Leaving the library/standalone build itself aside for a while, do you
agree that the various tools we are building with library mode should
be placed in tools? Things like lklfuse - mounting Linux filesystems
with fuse, or lklhijack.so - a preload library that intercept
network/file system calls and routes them through the library mode.

> You're looking at it the other way around I think - it seems that you're
> thinking the kernel binary is the vmlinux.a, and that's what the
> kernel's build system worries about; then the "vmlinux.so" (library
> mode) or "linux" (standalone mode - perhaps that's a good name?) is the
> eventual 'tool' that we build, using the previously built vmlinux.a.
>

Correct, this is my perspective.

> But that really isn't how standalone mode works, and IMHO it also
> doesn't match what tools are today.
>

What if we could use standalone mode for other purposes that would
require creating a new binary in addition to the current linux binary?

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 03/21] um: move arch/um/os-Linux dir to tools/um/uml
@ 2020-10-08 20:53                   ` Octavian Purdila
  0 siblings, 0 replies; 476+ messages in thread
From: Octavian Purdila @ 2020-10-08 20:53 UTC (permalink / raw)
  To: Johannes Berg
  Cc: linux-arch, Richard Weinberger, jdike, linux-um, Akira Moroo,
	linux-kernel-library, Hajime Tazaki, Anton Ivanov

On Thu, Oct 8, 2020 at 10:46 PM Johannes Berg <johannes@sipsolutions.net> wrote:
>
> On Thu, 2020-10-08 at 20:48 +0300, Octavian Purdila wrote:
>
> > > On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> > > > This patch moves underlying OS dependent part under arch/um to tools/um
> > > > directory so that arch/um code does not need to build host build
> > > > facilities (e.g., libc).
> > >
> > > Hmm. On the one hand, that build separation seems sensible, but on the
> > > other hand this stuff *does* fundamentally belong to arch/um/, and it's
> > > not a "tool" like basically everything else there that is a pure
> > > userspace application to run *inside* the kernel, not *part of* it.
> > >
> > > For that reason, I don't really like this much.
> > >
> >
> > I see the os_*() calls as dependencies that the kernel uses. Sort of
> > like calls into the hypervisor or firmware.
>
> Right.
>
> > The current UML build is already partially split. USER_OBJS build with
> > a different rule set than the rest of the kernel objects.
>
> Yes, that's true.
>
> > IMHO this
> > change just makes this more clear and streamlined, especially with
> > regard to linking.
>
> Well, maybe? But I actually tend to see this less as a question of
> (technical) convenience but semantics, and the tools are just not
> *meant* to be things that build a kernel, they're things to be used
> inside that kernel.
>
> I dunno. Maybe the technical convenience should win, but OTOH the
> "contortions" that UML build goes through with USER_OBJS don't really
> seem bad enough to merit breaking the notion of what tools are?
>

Half-joking, technically uml is not a kernel, it is a tool that
simulates a kernel and it uses part of the Linux kernel to do that :)

> > We are using the same build system as the rest of the stuff in tools.
> > Since it is building programs and libraries and not part of the kernel
> > binary build, it is using a slightly different infrastructure, which
> > is detailed in tools/build/Documentation/Build.txt
>
> OK, thanks for the pointer, I hadn't seen that before.
>
> > The reason for picking tools was that it already has the
> > infrastructure to build programs and shared libraries and the fact
> > that it is the only place in the kernel source tree where code is not
> > built directly into the kernel binary.
>
> Yeah, but all of UML/LKL _is_ eventually built into the kernel binary
> (or library as it may be). Which is in a way exactly my objection.
>

Leaving the library/standalone build itself aside for a while, do you
agree that the various tools we are building with library mode should
be placed in tools? Things like lklfuse - mounting Linux filesystems
with fuse, or lklhijack.so - a preload library that intercept
network/file system calls and routes them through the library mode.

> You're looking at it the other way around I think - it seems that you're
> thinking the kernel binary is the vmlinux.a, and that's what the
> kernel's build system worries about; then the "vmlinux.so" (library
> mode) or "linux" (standalone mode - perhaps that's a good name?) is the
> eventual 'tool' that we build, using the previously built vmlinux.a.
>

Correct, this is my perspective.

> But that really isn't how standalone mode works, and IMHO it also
> doesn't match what tools are today.
>

What if we could use standalone mode for other purposes that would
require creating a new binary in addition to the current linux binary?

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 06/21] scritps: um: suppress warnings if SRCARCH=um
  2020-10-07 15:24             ` Johannes Berg
  (?)
@ 2020-10-09  1:13             ` Hajime Tazaki
  2020-10-09 16:00                 ` Johannes Berg
  -1 siblings, 1 reply; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-09  1:13 UTC (permalink / raw)
  To: johannes
  Cc: linux-um, jdike, richard, anton.ivanov, tavi.purdila,
	linux-kernel-library, linux-arch, retrage01


On Thu, 08 Oct 2020 00:24:17 +0900,
Johannes Berg wrote:
> 
> On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> > This commit fixes numbers of warning messages about leaked CONFIG
> > options.
> 
> No it doesn't ... it suppresses them, not a fan at all.

Agree to rewrite the description.

> > nommu mode of UML requires copies of kernel headers to offer
> > syscall-like API for the library users.  Thus, the warnings are to be
> > avoided to function this exposure of API.
> 
> I don't see that here, so maybe make that part of the patch where it
> actually happens, with a big comment,

Although I didn't, this patch should Cc the maintainer of
./scripts/headers_install.sh.  I thought this should be as a separate
one, but I can Cc (in the commit message) to the commit that actual
build happens if that helps to understand.

> and maybe only when actually building for NOMMU/LKL?

The standalone UML doesn't use the headers_install.sh script (AFAIK),
so I think this would be okay.

-- Hajime


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 07/21] um: extend arch_switch_to for alternate SUBARCH
  2020-10-07 15:25             ` Johannes Berg
  (?)
@ 2020-10-09  1:24             ` Hajime Tazaki
  2020-10-09 16:02                 ` Johannes Berg
  -1 siblings, 1 reply; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-09  1:24 UTC (permalink / raw)
  To: johannes
  Cc: linux-um, jdike, richard, anton.ivanov, tavi.purdila,
	linux-kernel-library, linux-arch, retrage01


On Thu, 08 Oct 2020 00:25:06 +0900,
Johannes Berg wrote:
> 
> On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> > This commit introduces additional argument of previous task when
> > context switch happens.  New SUBARCH can use the new information to
> > switch tasks in a subarch-specific manner.
> > 
> > The patch is particularly required by nommu mode implemented as a
> > SUBARCH of UML.
> 
> Would probably be good to already say why here?

Agree.

The patch is particularly required by nommu mode because it uses
previous task information to control the previous thread (e.g., down
semaphore, terminate thread, clean up thread flags).

Something like this ?

-- Hajime


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 08/21] um: add nommu mode for UML library mode
  2020-10-07 15:44             ` Johannes Berg
  (?)
@ 2020-10-09  3:38             ` Hajime Tazaki
  2020-10-09 16:06                 ` Johannes Berg
  -1 siblings, 1 reply; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-09  3:38 UTC (permalink / raw)
  To: johannes
  Cc: linux-um, jdike, richard, anton.ivanov, tavi.purdila,
	linux-kernel-library, linux-arch, retrage01


Hello Johannes,

On Thu, 08 Oct 2020 00:44:12 +0900,
Johannes Berg wrote:
> 
> On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> > This patch introduces the nommu operation with UML code so that host
> > interface can be shrineked for broader environments support.
> 
> shrunk
> 
> > The nommu mode is implemneted as SUBARCH of arch/um, which places at
> 
> implemented

thanks; will fix both in the next round.

> > arch/um/nommu. This SUBARCH defines mode-specific code of memory
> > management as well as thread implementation, along with the uapi headers
> > to be exported to users.
> > 
> > The headers we introduce in this patch are simple wrappers to the
> > asm-generic headers or stubs for things we don't support, such as
> > ptrace, DMA, signals, ELF handling and low level processor operations.
> > 
> > nommu mode shares most of arch/um code (irq, thread_info, process
> > scheduling) but implements its own facilities, which are memory
> > management (nommu), thread primitives (struct arch_thread), system call
> > interface, and console.
> > 
> > The outlook of updated directory structure is as follows:
> > 
> >    arch/um
> >    |-- configs         (untouched)
> >    |-- drivers         unuse stdio_console.c for !MMU
> >    |-- include
> >    |   |-- asm         updated with !CONFIG_MMU
> >    |   |-- linux       (untouched)
> >    |   `-- shared      updated with new functions
> >    |       `-- skas    (untouched)
> >    |   `-- uapi        added for upai header installation
> >    |-- kernel          updated to integrate with !MMU
> >    |   `-- skas        (untouched, don't use for !MMU)
> >    |-- nommu           SUBARCH dir (internally =um/nommu)
> >    |   |-- include
> >    |   |   |-- asm     headers for subarch specific info
> >    |   |   `-- uapi    headers for user-visible APIs
> >    |   `-- um
> >    |       `-- shared  headers for subarch specific info
> >    `-- scripts         added a script for header installation
> 
> That seems awkward. Might be nice now for "as little changes as
> possible", but eventually it seems it would be better to separate that
> out into arch/um/{common,nommu,mmu}/ or something like that?
> 
> Or perhaps something like
> 
> 	arch/um/{common,lkl,full}
> 
> or something? Not sure I like 'full' though. 'vm'? Hmm.

(I will leave this to the other thread if any)

> (also, scripts/ doesn't exist in this patch?)
> 
> > +++ b/arch/um/nommu/Makefile
> > @@ -0,0 +1 @@
> > +#
> 
> hmm? maybe add a comment why an empty makefile is needed.

Thanks, we will.

> > diff --git a/arch/um/nommu/Makefile.um b/arch/um/nommu/Makefile.um
> > new file mode 100644
> > index 000000000000..3808462d8283
> > --- /dev/null
> > +++ b/arch/um/nommu/Makefile.um
> 
> [...]
> 
> > +ifeq ($(shell uname -s), Linux)
> > +NPROC=$(shell nproc)
> > +else # e.g., FreeBSD
> > +NPROC=$(shell sysctl -n hw.ncpu)
> > +endif
> 
> That seems very inappropriate here.

Will cut the FreeBSD part.

> > +
> > +um_headers_install: $(objtree)/$(HOST_DIR)/include/generated/uapi/asm/syscall_defs.h headers
> > +# if syscall_defs.h is newer than generated headers (autoconf.h)
> > +	$(Q)if [ ! -r $(objtree)/tools/um/include/lkl/autoconf.h ] || \
> > +	  [ $< -nt $(objtree)/tools/um/include/lkl/autoconf.h ]; then \
> > +		$(srctree)/$(ARCH_DIR)/scripts/headers_install.py \
> > +			$(subst -j,-j$(NPROC),$(findstring -j,$(MAKEFLAGS))) \
> 
> Yeah ... just don't.

Will clean up this part.

> > +++ b/arch/um/nommu/include/asm/atomic.h
> > @@ -0,0 +1,11 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +#ifndef __UM_NOMMU_ATOMIC_H
> > +#define __UM_NOMMU_ATOMIC_H
> > +
> > +#include <asm-generic/atomic.h>
> > +
> > +#ifndef CONFIG_GENERIC_ATOMIC64
> > +#include "atomic64.h"
> > +#endif /* !CONFIG_GENERIC_ATOMIC64 */
> > +
> > +#endif
> > diff --git a/arch/um/nommu/include/asm/atomic64.h b/arch/um/nommu/include/asm/atomic64.h
> > new file mode 100644
> > index 000000000000..949360dea7af
> > --- /dev/null
> > +++ b/arch/um/nommu/include/asm/atomic64.h
> 
> That doesn't make sense to me, you can control CONFIG_GENERIC_ATOMIC64
> to be on, and don't need the ifdef and this file?

We were suggested to not use GENERIC_ATOMIC64 on 64bit build during
our v3 patchset.  So we actually control CONFIG_GENERIC_ATOMIC64 to be
on only when !64BIT, and thus need atomic64.h for the 64BIT build.

https://lwn.net/ml/linux-arch/20200205093454.GG14879@hirez.programming.kicks-ass.net/

> > diff --git a/arch/um/nommu/include/asm/bitsperlong.h b/arch/um/nommu/include/asm/bitsperlong.h
> > new file mode 100644
> > index 000000000000..a150cee41e75
> > --- /dev/null
> > +++ b/arch/um/nommu/include/asm/bitsperlong.h
> > @@ -0,0 +1,12 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +#ifndef __UM_NOMMU_BITSPERLONG_H
> > +#define __UM_NOMMU_BITSPERLONG_H
> > +
> > +#include <uapi/asm/bitsperlong.h>
> > +
> > +#define BITS_PER_LONG __BITS_PER_LONG
> > +
> > +#define BITS_PER_LONG_LONG 64
> > +
> > +#endif
> 
> That's very similar to the contents of include/asm-
> generic/bitsperlong.h, add comments why it's needed?

thanks, it seems that this is unnecessary now.  will fix them.

> > diff --git a/arch/um/nommu/include/asm/byteorder.h b/arch/um/nommu/include/asm/byteorder.h
> > new file mode 100644
> > index 000000000000..920a5fd26cad
> > --- /dev/null
> > +++ b/arch/um/nommu/include/asm/byteorder.h
> > @@ -0,0 +1,7 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +#ifndef __UM_NOMMU_BYTEORDER_H
> > +#define __UM_NOMMU_BYTEORDER_H
> > +
> > +#include <uapi/asm/byteorder.h>
> > +
> > +#endif
> 
> Not sure why this file even exists, it doesn't for any other arch
> (except for h8300 which seems odd).

You're right.  We will remove this.

> > diff --git a/arch/um/nommu/include/asm/elf.h b/arch/um/nommu/include/asm/elf.h
> > new file mode 100644
> > index 000000000000..edcf63edeed1
> > --- /dev/null
> > +++ b/arch/um/nommu/include/asm/elf.h
> > @@ -0,0 +1,15 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +#ifndef __UM_NOMMU_ELF_H
> > +#define __UM_NOMMU_ELF_H
> > +
> > +#define elf_check_arch(x) 0
> 
> Please add a comment ...
>
> > +#ifdef CONFIG_64BIT
> > +#define ELF_CLASS ELFCLASS64
> > +#else
> > +#define ELF_CLASS ELFCLASS32
> > +#endif
> > +
> > +#define elf_gregset_t long
> > +#define elf_fpregset_t double
> 
> Also here.

thanks, will add the comments.

> > +#ifndef __UM_NOMMU_SCHED_H
> > +#define __UM_NOMMU_SCHED_H
> > +
> > +#include <linux/sched.h>
> > +#include <uapi/asm/host_ops.h>
> > +
> > +static inline void thread_sched_jb(void)
> > +{
> > +	if (test_ti_thread_flag(current_thread_info(), TIF_HOST_THREAD)) {
> > +		set_ti_thread_flag(current_thread_info(), TIF_SCHED_JB);
> > +		set_current_state(TASK_UNINTERRUPTIBLE);
> > +		lkl_ops->jmp_buf_set(&current_thread_info()->task->thread.arch.sched_jb,
> > +				     schedule);
> > +	} else {
> > +		lkl_bug("%s can be used only for host task", __func__);
> > +	}
> > +}
> 
> What's "thread_sched_jb"? Doesn't really seem to exist? Neither does
> "TIF_SCHED_JB", at least in my kernel?
> 
> Looks like that's added by a later patch, should probably then add this
> file also only later so it's actually consistent and one can review all
> the pieces together.

I understand. I will move this file to the later patch (11/21).

> > +++ b/arch/um/nommu/include/uapi/asm/Kbuild
> > @@ -0,0 +1,6 @@
> > +# UAPI Header export list
> > +
> > +generated-y += syscall_defs.h
> > +
> > +# no generated-y since we need special user headers handling
> > +# see arch/um/script/headers_install.py
> 
> Also doesn't exist yet.

The message is incorrect since uapi header infra has changed.
This also applies to the comment in arch/um/nommu/include/asm/Kbuild.

Will fix both in the next round.

> > +++ b/arch/um/nommu/um/delay.c
> 
> why the double um/ path?

This is because now in arch/um/Makefile,

 - SUBARCH := um/nommu
 - HEADER_ARCH := $(SUBARCH)
 - HOST_DIR := arch/$(HEADER_ARCH)
 - core-y += $(HOST_DIR)/um

the last line expands as arch/um/nommu/um.

> > +/* skas/process.c */
> > +void halt_skas(void)
> > +{}
> 
> nit: missing blank line after this

will fix this and other parts of the whole patchset.

> > +int is_skas_winch(int pid, int fd, void *data)
> > +{
> > +	return 0;
> > +}
> > +void reboot_skas(void)
> > +{}
> > +
> > +int __init start_uml(void)
> > +{
> > +	return 0;
> > +}
> 
> That seems odd?
> 
> Might be better to have an own os.h, or split that into os-uml.h and
> os-lkl.h or so?

Okay, we will think about this.

-- Hajime

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 20/21] um: host: add test programs
  2020-10-07 19:23             ` Johannes Berg
  (?)
@ 2020-10-09  6:24             ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-09  6:24 UTC (permalink / raw)
  To: johannes
  Cc: linux-um, jdike, richard, anton.ivanov, tavi.purdila,
	linux-kernel-library, retrage01, linux-arch


On Thu, 08 Oct 2020 04:23:51 +0900,
Johannes Berg wrote:
> 
> On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> > 
> > +LKL_TEST_CALL(getpid, lkl_sys_getpid, 1)
> 
> Could that be unified with KUNIT somehow?
> 
> > +++ b/tools/um/tests/run.py
> > @@ -0,0 +1,172 @@
> > +#!/usr/bin/env python
> > +# SPDX-License-Identifier: GPL-2.0
> > +#
> > +# This program is free software; you can redistribute it and/or modify
> > +# it under the terms of the GNU General Public License as published by
> > +# the Free Software Foundation; version 2 of the License
> > +#
> > +# Author: Octavian Purdila <tavi@cs.pub.ro>
> > +#
> > +
> > +from __future__ import print_function
> > +
> > +import argparse
> > +import os
> > +import subprocess
> > +import sys
> > +import tap13
> 
> ok, I see now, you're doing something with TAP (test anything
> protocol)...
> 
> Hmm. I must say I'm not a fan of adding a whole testing framework to the
> kernel like that, even if it's pretty simple.
> 
> > +import xml.etree.ElementTree as ET
> > +
> > +from junit_xml import TestSuite, TestCase
> 
> yuck
> 
> Let's see if you can use KUNIT instead. Anything beyond that doesn't
> really need to live in the kernel, IMHO.

I see the point.
I'm gonna play with kunit to check if it's doable.

Thanks,
-- Hajime

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 16/21] um: nommu: plug in the build system
  2020-10-07 19:20             ` Johannes Berg
@ 2020-10-09  7:40               ` Hajime TAZAKI
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime TAZAKI @ 2020-10-09  7:40 UTC (permalink / raw)
  To: Johannes Berg
  Cc: linux-um, Jeff Dike, Richard Weinberger, anton.ivanov,
	Octavian Purdila, linux-kernel-library, linux-arch, Akira Moroo

(I removed UMMODE_LIB vs !MMU part, but leave replies to the others)

On Thu, 08 Oct 2020 04:20:00 +0900,
Johannes Berg wrote:

> >   make defconfig ARCH=um UMMODE=library
> >   make ARCH=um UMMODE=library
>
> Why not make it a real Kconfig symbol? I find the ARCH=um tricky enough
> to remember all the time, so much I usually write a "GNUmakefile" with
>
> export ARCH=um
> include Makefile
>
> :-)
>
> > +config UMMODE_LIB
> > + bool "UML mode: library mode"
> > + default y if "$(UMMODE)" = "library"
>
> So wait, you _can_ switch that through Kconfig then, because you made it
> a visible option (string after "bool"). But it won't work, because then
> you later in the build system etc. still check UMMODE instead of
> CONFIG_UMMODE_LIB. Seems like something that ought to be fixed one way
> or the other - at the very least hide this symbol if setting it manually
> is invalid.

I now see what I went wrong..
thanks, will fix this with your suggestion not to use the command line argument.

> > + help
> > + This mode switches a mode to build a library of UML (Linux
> > + Kernel Library/LKL).  This switch is exclusive to "kernel mode"
> > + of UML, which is traditional mode of UML.
>
> Not sure if that historically made more sense, but you don't have any
> UMMODE_KERNEL option or something like that, so the help text seems
> confusing?

thanks, we will clarify this text.

> > +ifeq ($(UMMODE),library)
> > +  SUBARCH := um/nommu
> > +endif
>
> >  INSTALL_PATH=$(objtree)/tools/um
> > +ifeq ($(UMMODE),library)
>
> Here a few places using UMMODE which must come from the command line.

will fix this too.

> >  linux.o: vmlinux
> >  @echo '  LINK $@'
> > - $(Q)$(OBJCOPY) -R .eh_frame $< $@
> > + $(Q)$(OBJCOPY) -R .eh_frame -L sem_init -L sem_post -L sem_wait -L sem_destroy $< $@
>
> Care to explain?

If we will link a userspace binary with liblinux.a (LKL) and
libpthread, the the symbols sem_init, etc will conflict with the one
in the kernel (ipc/sem.c).  objcopy command with -L localizes the
symbol specified to avoid this conflict.

We will add the description as a comment.

> > + select UACCESS_MEMCPY
> > + select ARCH_THREAD_STACK_ALLOCATOR
> > + select ARCH_HAS_SYSCALL_WRAPPER
>
> You never use this except for the selects, maybe can go elsewhere?

This is used at the patch 12/21, so add Kconfig earlier and move this
to 12/21 would be better.  We'll work on.

> > +config 64BIT
> > + bool
> > + default y
> > +
> > +config GENERIC_CSUM
> > + def_bool y
> > +
> > +config GENERIC_ATOMIC64
> > + bool
> > + default y if !64BIT
> > +
> > +config SECCOMP
> > + bool
> > + default n
> > +
> > +config GENERIC_HWEIGHT
> > + def_bool y
> > +
> > +config GENERIC_CALIBRATE_DELAY
> > + bool
> > + default n
> > +
> > +config STACKTRACE_SUPPORT
> > + bool
> > + default n
>
> You ... were just changing these elsewhere, so one of that isn't needed?

you're right too. will fix it.

-- Hajime

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 16/21] um: nommu: plug in the build system
@ 2020-10-09  7:40               ` Hajime TAZAKI
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime TAZAKI @ 2020-10-09  7:40 UTC (permalink / raw)
  To: Johannes Berg
  Cc: linux-arch, Octavian Purdila, Richard Weinberger, Jeff Dike,
	linux-um, Akira Moroo, linux-kernel-library, anton.ivanov

(I removed UMMODE_LIB vs !MMU part, but leave replies to the others)

On Thu, 08 Oct 2020 04:20:00 +0900,
Johannes Berg wrote:

> >   make defconfig ARCH=um UMMODE=library
> >   make ARCH=um UMMODE=library
>
> Why not make it a real Kconfig symbol? I find the ARCH=um tricky enough
> to remember all the time, so much I usually write a "GNUmakefile" with
>
> export ARCH=um
> include Makefile
>
> :-)
>
> > +config UMMODE_LIB
> > + bool "UML mode: library mode"
> > + default y if "$(UMMODE)" = "library"
>
> So wait, you _can_ switch that through Kconfig then, because you made it
> a visible option (string after "bool"). But it won't work, because then
> you later in the build system etc. still check UMMODE instead of
> CONFIG_UMMODE_LIB. Seems like something that ought to be fixed one way
> or the other - at the very least hide this symbol if setting it manually
> is invalid.

I now see what I went wrong..
thanks, will fix this with your suggestion not to use the command line argument.

> > + help
> > + This mode switches a mode to build a library of UML (Linux
> > + Kernel Library/LKL).  This switch is exclusive to "kernel mode"
> > + of UML, which is traditional mode of UML.
>
> Not sure if that historically made more sense, but you don't have any
> UMMODE_KERNEL option or something like that, so the help text seems
> confusing?

thanks, we will clarify this text.

> > +ifeq ($(UMMODE),library)
> > +  SUBARCH := um/nommu
> > +endif
>
> >  INSTALL_PATH=$(objtree)/tools/um
> > +ifeq ($(UMMODE),library)
>
> Here a few places using UMMODE which must come from the command line.

will fix this too.

> >  linux.o: vmlinux
> >  @echo '  LINK $@'
> > - $(Q)$(OBJCOPY) -R .eh_frame $< $@
> > + $(Q)$(OBJCOPY) -R .eh_frame -L sem_init -L sem_post -L sem_wait -L sem_destroy $< $@
>
> Care to explain?

If we will link a userspace binary with liblinux.a (LKL) and
libpthread, the the symbols sem_init, etc will conflict with the one
in the kernel (ipc/sem.c).  objcopy command with -L localizes the
symbol specified to avoid this conflict.

We will add the description as a comment.

> > + select UACCESS_MEMCPY
> > + select ARCH_THREAD_STACK_ALLOCATOR
> > + select ARCH_HAS_SYSCALL_WRAPPER
>
> You never use this except for the selects, maybe can go elsewhere?

This is used at the patch 12/21, so add Kconfig earlier and move this
to 12/21 would be better.  We'll work on.

> > +config 64BIT
> > + bool
> > + default y
> > +
> > +config GENERIC_CSUM
> > + def_bool y
> > +
> > +config GENERIC_ATOMIC64
> > + bool
> > + default y if !64BIT
> > +
> > +config SECCOMP
> > + bool
> > + default n
> > +
> > +config GENERIC_HWEIGHT
> > + def_bool y
> > +
> > +config GENERIC_CALIBRATE_DELAY
> > + bool
> > + default n
> > +
> > +config STACKTRACE_SUPPORT
> > + bool
> > + default n
>
> You ... were just changing these elsewhere, so one of that isn't needed?

you're right too. will fix it.

-- Hajime

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 03/21] um: move arch/um/os-Linux dir to tools/um/uml
  2020-10-08 20:53                   ` Octavian Purdila
@ 2020-10-09 15:59                     ` Johannes Berg
  -1 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-09 15:59 UTC (permalink / raw)
  To: Octavian Purdila
  Cc: Hajime Tazaki, linux-um, jdike, Richard Weinberger, Anton Ivanov,
	linux-kernel-library, linux-arch, Akira Moroo

On Thu, 2020-10-08 at 23:53 +0300, Octavian Purdila wrote:

> Leaving the library/standalone build itself aside for a while, do you
> agree that the various tools we are building with library mode should
> be placed in tools? Things like lklfuse - mounting Linux filesystems
> with fuse, or lklhijack.so - a preload library that intercept
> network/file system calls and routes them through the library mode.

Yeah, I guess that makes sense. I realize the line gets blurry ...

OTOH, should these even live in the kernel tree? Almost seems like if
the base lkl library is there, the other stuff might not really need to
be, it's supposed to use the (stable) syscall API?

Dunno. Not sure "doesn't need to be" really is an argument for
"shouldn't be" anyway.

> > You're looking at it the other way around I think - it seems that you're
> > thinking the kernel binary is the vmlinux.a, and that's what the
> > kernel's build system worries about; then the "vmlinux.so" (library
> > mode) or "linux" (standalone mode - perhaps that's a good name?) is the
> > eventual 'tool' that we build, using the previously built vmlinux.a.
> > 
> 
> Correct, this is my perspective.
> 
> > But that really isn't how standalone mode works, and IMHO it also
> > doesn't match what tools are today.
> > 
> 
> What if we could use standalone mode for other purposes that would
> require creating a new binary in addition to the current linux binary?

Anything specific you have in mind?

But I still think that "UML standalone 'linux' binary" is something that
seems like the kernel build system should be able do? If only to avoid
regressions from what we have now ...

johannes


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 03/21] um: move arch/um/os-Linux dir to tools/um/uml
@ 2020-10-09 15:59                     ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-09 15:59 UTC (permalink / raw)
  To: Octavian Purdila
  Cc: linux-arch, Richard Weinberger, jdike, linux-um, Akira Moroo,
	linux-kernel-library, Hajime Tazaki, Anton Ivanov

On Thu, 2020-10-08 at 23:53 +0300, Octavian Purdila wrote:

> Leaving the library/standalone build itself aside for a while, do you
> agree that the various tools we are building with library mode should
> be placed in tools? Things like lklfuse - mounting Linux filesystems
> with fuse, or lklhijack.so - a preload library that intercept
> network/file system calls and routes them through the library mode.

Yeah, I guess that makes sense. I realize the line gets blurry ...

OTOH, should these even live in the kernel tree? Almost seems like if
the base lkl library is there, the other stuff might not really need to
be, it's supposed to use the (stable) syscall API?

Dunno. Not sure "doesn't need to be" really is an argument for
"shouldn't be" anyway.

> > You're looking at it the other way around I think - it seems that you're
> > thinking the kernel binary is the vmlinux.a, and that's what the
> > kernel's build system worries about; then the "vmlinux.so" (library
> > mode) or "linux" (standalone mode - perhaps that's a good name?) is the
> > eventual 'tool' that we build, using the previously built vmlinux.a.
> > 
> 
> Correct, this is my perspective.
> 
> > But that really isn't how standalone mode works, and IMHO it also
> > doesn't match what tools are today.
> > 
> 
> What if we could use standalone mode for other purposes that would
> require creating a new binary in addition to the current linux binary?

Anything specific you have in mind?

But I still think that "UML standalone 'linux' binary" is something that
seems like the kernel build system should be able do? If only to avoid
regressions from what we have now ...

johannes


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 06/21] scritps: um: suppress warnings if SRCARCH=um
  2020-10-09  1:13             ` Hajime Tazaki
@ 2020-10-09 16:00                 ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-09 16:00 UTC (permalink / raw)
  To: Hajime Tazaki
  Cc: linux-um, jdike, richard, anton.ivanov, tavi.purdila,
	linux-kernel-library, linux-arch, retrage01

On Fri, 2020-10-09 at 10:13 +0900, Hajime Tazaki wrote:
> 
> > and maybe only when actually building for NOMMU/LKL?
> 
> The standalone UML doesn't use the headers_install.sh script (AFAIK),
> so I think this would be okay.

Yes, I was more thinking in terms of documenting the purpose in the
code, similar really to the argument I made elsewhere about "NOMMU" vs
"LKL mode".

johannes


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 06/21] scritps: um: suppress warnings if SRCARCH=um
@ 2020-10-09 16:00                 ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-09 16:00 UTC (permalink / raw)
  To: Hajime Tazaki
  Cc: linux-arch, tavi.purdila, richard, jdike, linux-um, retrage01,
	linux-kernel-library, anton.ivanov

On Fri, 2020-10-09 at 10:13 +0900, Hajime Tazaki wrote:
> 
> > and maybe only when actually building for NOMMU/LKL?
> 
> The standalone UML doesn't use the headers_install.sh script (AFAIK),
> so I think this would be okay.

Yes, I was more thinking in terms of documenting the purpose in the
code, similar really to the argument I made elsewhere about "NOMMU" vs
"LKL mode".

johannes


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 07/21] um: extend arch_switch_to for alternate SUBARCH
  2020-10-09  1:24             ` Hajime Tazaki
@ 2020-10-09 16:02                 ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-09 16:02 UTC (permalink / raw)
  To: Hajime Tazaki
  Cc: linux-um, jdike, richard, anton.ivanov, tavi.purdila,
	linux-kernel-library, linux-arch, retrage01

On Fri, 2020-10-09 at 10:24 +0900, Hajime Tazaki wrote:
> On Thu, 08 Oct 2020 00:25:06 +0900,
> Johannes Berg wrote:
> > On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> > > This commit introduces additional argument of previous task when
> > > context switch happens.  New SUBARCH can use the new information to
> > > switch tasks in a subarch-specific manner.
> > > 
> > > The patch is particularly required by nommu mode implemented as a
> > > SUBARCH of UML.
> > 
> > Would probably be good to already say why here?
> 
> Agree.
> 
> The patch is particularly required by nommu mode because it uses
> previous task information to control the previous thread (e.g., down
> semaphore, terminate thread, clean up thread flags).
> 
> Something like this ?

Well, still not sure I like "nommu" to refer to "library mode", but I'm
also not sure that you can understand that without the context of how
threads actually work in library mode?

Maybe even something vague like

   Having access to the previous thread will be required in library mode
   for it to implement thread switching correctly.

would give some rationale?

johannes


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 07/21] um: extend arch_switch_to for alternate SUBARCH
@ 2020-10-09 16:02                 ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-09 16:02 UTC (permalink / raw)
  To: Hajime Tazaki
  Cc: linux-arch, tavi.purdila, richard, jdike, linux-um, retrage01,
	linux-kernel-library, anton.ivanov

On Fri, 2020-10-09 at 10:24 +0900, Hajime Tazaki wrote:
> On Thu, 08 Oct 2020 00:25:06 +0900,
> Johannes Berg wrote:
> > On Tue, 2020-10-06 at 18:44 +0900, Hajime Tazaki wrote:
> > > This commit introduces additional argument of previous task when
> > > context switch happens.  New SUBARCH can use the new information to
> > > switch tasks in a subarch-specific manner.
> > > 
> > > The patch is particularly required by nommu mode implemented as a
> > > SUBARCH of UML.
> > 
> > Would probably be good to already say why here?
> 
> Agree.
> 
> The patch is particularly required by nommu mode because it uses
> previous task information to control the previous thread (e.g., down
> semaphore, terminate thread, clean up thread flags).
> 
> Something like this ?

Well, still not sure I like "nommu" to refer to "library mode", but I'm
also not sure that you can understand that without the context of how
threads actually work in library mode?

Maybe even something vague like

   Having access to the previous thread will be required in library mode
   for it to implement thread switching correctly.

would give some rationale?

johannes


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 08/21] um: add nommu mode for UML library mode
  2020-10-09  3:38             ` Hajime Tazaki
@ 2020-10-09 16:06                 ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-09 16:06 UTC (permalink / raw)
  To: Hajime Tazaki
  Cc: linux-um, jdike, richard, anton.ivanov, tavi.purdila,
	linux-kernel-library, linux-arch, retrage01

On Fri, 2020-10-09 at 12:38 +0900, Hajime Tazaki wrote:
> 
> > > +++ b/arch/um/nommu/Makefile.um
> > 
> > [...]
> > 
> > > +ifeq ($(shell uname -s), Linux)
> > > +NPROC=$(shell nproc)
> > > +else # e.g., FreeBSD
> > > +NPROC=$(shell sysctl -n hw.ncpu)
> > > +endif
> > 
> > That seems very inappropriate here.
> 
> Will cut the FreeBSD part.

But even the NPROC and -j flag mangling seems ... awful?

> > > --- /dev/null
> > > +++ b/arch/um/nommu/include/asm/atomic64.h
> > 
> > That doesn't make sense to me, you can control CONFIG_GENERIC_ATOMIC64
> > to be on, and don't need the ifdef and this file?
> 
> We were suggested to not use GENERIC_ATOMIC64 on 64bit build during
> our v3 patchset.  So we actually control CONFIG_GENERIC_ATOMIC64 to be
> on only when !64BIT, and thus need atomic64.h for the 64BIT build.
> 
> https://lwn.net/ml/linux-arch/20200205093454.GG14879@hirez.programming.kicks-ass.net/

Well, yeah, but your architecture _is_ a piece of crap in needing
atomic64 emulation? :-)

How about just using the real atomic64 implementation like UML normally
does? I mean, falling back to the actual underlying CPU. Or even gcc
intrinsics?

You've basically re-implemented CONFIG_GENERIC_ATOMIC64, so perhaps
you've hidden it from PeterZ's view now because you're no longer
touching the real generic atomic64 (which is for 32-bit machines), but
that doesn't actually make this any *better*. Arguably *worse* since
you've just copied it ...

> > > +++ b/arch/um/nommu/um/delay.c
> > 
> > why the double um/ path?
> 
> This is because now in arch/um/Makefile,
> 
>  - SUBARCH := um/nommu
>  - HEADER_ARCH := $(SUBARCH)
>  - HOST_DIR := arch/$(HEADER_ARCH)
>  - core-y += $(HOST_DIR)/um
> 
> the last line expands as arch/um/nommu/um.

But is there a reason for it? Couldn't it be changed?

Anyway, we're still debating the whole build system and layout so not
much point in going into the details now ...

johannes


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 08/21] um: add nommu mode for UML library mode
@ 2020-10-09 16:06                 ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2020-10-09 16:06 UTC (permalink / raw)
  To: Hajime Tazaki
  Cc: linux-arch, tavi.purdila, richard, jdike, linux-um, retrage01,
	linux-kernel-library, anton.ivanov

On Fri, 2020-10-09 at 12:38 +0900, Hajime Tazaki wrote:
> 
> > > +++ b/arch/um/nommu/Makefile.um
> > 
> > [...]
> > 
> > > +ifeq ($(shell uname -s), Linux)
> > > +NPROC=$(shell nproc)
> > > +else # e.g., FreeBSD
> > > +NPROC=$(shell sysctl -n hw.ncpu)
> > > +endif
> > 
> > That seems very inappropriate here.
> 
> Will cut the FreeBSD part.

But even the NPROC and -j flag mangling seems ... awful?

> > > --- /dev/null
> > > +++ b/arch/um/nommu/include/asm/atomic64.h
> > 
> > That doesn't make sense to me, you can control CONFIG_GENERIC_ATOMIC64
> > to be on, and don't need the ifdef and this file?
> 
> We were suggested to not use GENERIC_ATOMIC64 on 64bit build during
> our v3 patchset.  So we actually control CONFIG_GENERIC_ATOMIC64 to be
> on only when !64BIT, and thus need atomic64.h for the 64BIT build.
> 
> https://lwn.net/ml/linux-arch/20200205093454.GG14879@hirez.programming.kicks-ass.net/

Well, yeah, but your architecture _is_ a piece of crap in needing
atomic64 emulation? :-)

How about just using the real atomic64 implementation like UML normally
does? I mean, falling back to the actual underlying CPU. Or even gcc
intrinsics?

You've basically re-implemented CONFIG_GENERIC_ATOMIC64, so perhaps
you've hidden it from PeterZ's view now because you're no longer
touching the real generic atomic64 (which is for 32-bit machines), but
that doesn't actually make this any *better*. Arguably *worse* since
you've just copied it ...

> > > +++ b/arch/um/nommu/um/delay.c
> > 
> > why the double um/ path?
> 
> This is because now in arch/um/Makefile,
> 
>  - SUBARCH := um/nommu
>  - HEADER_ARCH := $(SUBARCH)
>  - HOST_DIR := arch/$(HEADER_ARCH)
>  - core-y += $(HOST_DIR)/um
> 
> the last line expands as arch/um/nommu/um.

But is there a reason for it? Couldn't it be changed?

Anyway, we're still debating the whole build system and layout so not
much point in going into the details now ...

johannes


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v7 08/21] um: add nommu mode for UML library mode
  2020-10-09 16:06                 ` Johannes Berg
  (?)
@ 2020-10-20  8:44                 ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2020-10-20  8:44 UTC (permalink / raw)
  To: johannes
  Cc: linux-arch, tavi.purdila, richard, jdike, linux-um, retrage01,
	linux-kernel-library, anton.ivanov


Hello,

sorry for the late response.

On Sat, 10 Oct 2020 01:06:03 +0900,
Johannes Berg wrote:
> 
> On Fri, 2020-10-09 at 12:38 +0900, Hajime Tazaki wrote:
> > 
> > > > +++ b/arch/um/nommu/Makefile.um
> > > 
> > > [...]
> > > 
> > > > +ifeq ($(shell uname -s), Linux)
> > > > +NPROC=$(shell nproc)
> > > > +else # e.g., FreeBSD
> > > > +NPROC=$(shell sysctl -n hw.ncpu)
> > > > +endif
> > > 
> > > That seems very inappropriate here.
> > 
> > Will cut the FreeBSD part.
> 
> But even the NPROC and -j flag mangling seems ... awful?

Okay, we won't use the -j flag at this place.

> > > > --- /dev/null
> > > > +++ b/arch/um/nommu/include/asm/atomic64.h
> > > 
> > > That doesn't make sense to me, you can control CONFIG_GENERIC_ATOMIC64
> > > to be on, and don't need the ifdef and this file?
> > 
> > We were suggested to not use GENERIC_ATOMIC64 on 64bit build during
> > our v3 patchset.  So we actually control CONFIG_GENERIC_ATOMIC64 to be
> > on only when !64BIT, and thus need atomic64.h for the 64BIT build.
> > 
> > https://lwn.net/ml/linux-arch/20200205093454.GG14879@hirez.programming.kicks-ass.net/
> 
> Well, yeah, but your architecture _is_ a piece of crap in needing
> atomic64 emulation? :-)
> 
> How about just using the real atomic64 implementation like UML normally
> does? I mean, falling back to the actual underlying CPU. Or even gcc
> intrinsics?
> 
> You've basically re-implemented CONFIG_GENERIC_ATOMIC64, so perhaps
> you've hidden it from PeterZ's view now because you're no longer
> touching the real generic atomic64 (which is for 32-bit machines), but
> that doesn't actually make this any *better*. Arguably *worse* since
> you've just copied it ...

We're trying to reimplement atomic64 ops with gcc builtins.

We're now trying to address all of comments we got, and will get here
back once we have done.

-- Hajime

^ permalink raw reply	[flat|nested] 476+ messages in thread
* [RFC v8 00/20] Unifying LKL into UML
  2020-10-06  9:44         ` Hajime Tazaki
@ 2021-01-20  2:27           ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-01-20  2:27 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: thehajime, tavi.purdila, retrage01, linux-kernel-library, linux-arch

This is another spin of the unification of LKL into UML.  Updated and
fixed comments from our v7 patches.  The summary is listed in the
changelog below.

Note that the whole patchset requires the patch "um: ubd: fix command
line handling of ubd" to be correctly tested, which I previously
submitted.


Changes in rfc v8:
- stop using the term "nommu" mode, "library" mode instead
- generic atomic64 rewrite to use gcc builtin atomics
- use kernel-doc format for comments
- stop using a collection of function pointer (struct lkl_host_operations),
  but weak functions
- use python3, not python
- use c99 initializers for strerror
- add comments/descriptions about thread implementation
- remove redundant Kconfig entries (RAID6_PQ_BENCHMARK, STACKTRACE_SUPPORT,
  etc)
- refine code comments
- stop using NPROC for the internal -j flag
- remove make argument of "UMMODE=library"
- preserve the vmlinux file as it is
- rework commit history with reasonable files
- rework test framework using kselftest (tools/testing/selftests/um/)
- add documentation, MAINTAINERS


Changes in rfc v7:
- preserve `make ARCH=um` syntax to build UML
- introduce `make ARCH=um UMMODE=library` to build library mode
- fix undefined symbols issue during modpost
- clean up makefiles (arch/um, tools/um)

Changes in rfc v6:
- rebase with the current linus tree

Changes in rfc v5:
- rewrite whole patchset from scratch
- move arch-dependent code of arch/um and arch/x86/um to tools/um
 - mainly code under os-Linux/ involved
 - introduce 2-stage build (kernel, and host-dependent parts)
 - clean up vmlinux.lds.S
- put LKL-specific implementations as a SUBARCH under arch/um/nommu
- introduce !CONFIG_MMU in arch/um
- use struct arch_thread and arch_switch_to() for subarch-specific
  thread implementation
- integrate with the IRQ infrastructure of UML
- tested with block device drivers (ubd) for the proof

Changes in rfc v4: (https://lwn.net/Articles/816276/)
- Rebase on the current uml/master branch
- Fix IRQ handling (bug fix)
- drop a patch for CONFIG_GENERIC_ATOMIC64 (comment by Peter Zijlstra)
- implement vector net driver for UMMODE_LIB (comment by Anton)
- clean up uapi headers to avoid duplicates
- clean up IRQ handling code (comment by Anton)
- fix error handling in test code (comments by David Disseldorp)

Changes in rfc v3:
- use UML drivers (net, block) from LKL programs
- drop virtio device implementations
- drop mingw32 (Windows) host
- drop android (arm/aarch64) host
- drop FreeBSD (x86_64) host
- drop LD_PRELOAD (hijack) support
- update milestone

rfc v2:
- use UMMODE instead of SUBARCH to switch UML or LKL
- tools/lkl directory is still there. I confirmed we can move under arch/um
  (e.g., arch/um/lkl/hosts).  I will move it IF this is preferable.
- drop several patches involved non-uml directory
- drop several patches which are not required
- refine commit logs
- document updated




LKL (Linux Kernel Library) is aiming to allow reusing the Linux kernel code
as extensively as possible with minimal effort and reduced maintenance
overhead.

Examples of how LKL can be used are: creating userspace applications
(running on Linux and other operating systems) that can read or write Linux
filesystems or can use the Linux networking stack, creating kernel drivers
for other operating systems that can read Linux filesystems, bootloaders
support for reading/writing Linux filesystems, etc.

With LKL, the kernel code is compiled into an object file that can be
directly linked by applications. The API offered by LKL is based on the
Linux system call interface.

LKL is originally implemented as an architecture port in arch/lkl, but this
series of commits tries to integrate this into arch/um as one of the mode
of UML.  This was discussed during RFC email of LKL (*1).

The latest LKL version can be found at https://github.com/lkl/linux

Milestone
=========
This patches is a first step toward upstreaming *library mode* of
Linux kernel, but we think we need to have several steps toward our
goal, describing in the below.


Milestone 1: LKL lib on top of UML
 * Kernel - Host build split
 -  Build UML as a relocatable object using the UML's kernel linker script.
 -  Move the ptrace and other well isolated os code out of arch/um to
    tools/um
 -  Use standard host toolchain to create a static library stripped of
    the ptrace code. Use standard host toolchain to build the main UML
    executable.
 -  Add library init API that creates the UML kernel process and starts
    UML.
 * System calls APIs
 -  Add new system call interface based on UML's irq facility.
 -  Use the LKL scripts to export the required headers to create system
    calls APIs that use the UML system calls infrastructure.
 -  Keep the underlying host and driver operations (threads, irqs, etc.)
    as they are now in UML.
 * Boot test
 -  Port the LKL boot test to verify that we are able to programatically
    issue system calls.

Milestone 2: add virtio disk support
 * Export asm/io.h operations to host/os. Create IO access operations
   and redirect them to weak os_ variants that use the current UML
   implementation.
 * Add the LKL IO access layer including generic virtio handling and the
   virtio block device code.
 * Port LKL disk test and disk apps (lklfuse, fs2tar, cptofs)

Milestone 3: new arch ports
  * Abstract the system call / IRQ mode the move the implementation to host
  * Abstract the thread model and move the implementation to host
  * Add LKL thread model and LKL ports


Building LKL the host library and LKL applications
==================================================

% make ARCH=um SUBARCH=lkl defconfig
% make ARCH=um SUBARCH=lkl

will build LKL as a object file, it will install it in
tools/um/libtogether with the headers files in tools/um/include then
will build the host library, tests and a few of application examples:

* tools/testing/selftests/um/boot.c - a simple applications that uses
  LKL and exercises the basic LKL APIs

* tools/testing/selftests/um/disk.c - a simple applications that tests
  LKL and exercises the basic filesystem-related LKL APIs

Those tests can run with the following kselftest command:

    $ make ARCH=um SUBARCH=lkl TARGETS="um" kselftest

Supported hosts
===============

Currently LKL supports Linux userspace applications. New hosts can be added
relatively easy if the host supports gcc and GNU ld. Previous versions of
LKL supported Windows kernel and Haiku kernel hosts, and we also have WIP
patches with rump-hypercall interface, used in UEFI, as well as macOS
userspace (part of POSIX).

There is also musl-libc port for LKL, which might be interested in for some
folks.


Further readings about LKL
=========================

- Discussion in github LKL issue
https://github.com/lkl/linux/issues/304

- LKL (an article)
https://www.researchgate.net/profile/Nicolae_Tapus2/publication/224164682_LKL_The_Linux_kernel_library/links/02bfe50fd921ab4f7c000000.pdf

*1 RFC email to LKML (back in 2015)
https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1012277.html


Please review the following changes for suitability for inclusion. If you have
any objections or suggestions for improvement, please respond to the patches. If
you agree with the changes, please provide your Acked-by.

The following changes since commit 19c329f6808995b142b3966301f217c831e7cf31:

  Linux 5.11-rc4 (2021-01-17 16:37:05 -0800)

are available in the Git repository at:

  git://github.com/thehajime/linux 043677211bb5562397c511911e7861c3e217611b
  https://github.com/thehajime/linux/tree/uml-lkl-5.11rc4-v8

Hajime Tazaki (18):
  um: move arch/um/os-Linux dir to tools/um/uml
  um: move arch/x86/um/os-Linux to tools/um/uml/
  um: extend arch_switch_to for alternate SUBARCH
  um: add UML library mode
  um: lkl: host interface
  um: lkl: memory handling
  um: lkl: kernel thread support
  um: lkl: system call interface and application API
  um: lkl: basic console support
  um: lkl: initialization and cleanup
  um: lkl: integrate with irq infrastructure of UML
  um: lkl: plug in the build system
  um: host: add library mode build for ARCH=um
  um: host: add utilities functions
  um: host: posix host operations
  selftests/um: lkl: add test programs for library mode of UML
  um: lkl: add block device support of UML
  um: lkl: add documentation

Octavian Purdila (2):
  um: split build in kernel and host parts
  um: implement os_initcalls and os_exitcalls

 Documentation/virt/uml/lkl.txt                |  48 ++
 MAINTAINERS                                   |  10 +
 arch/um/Kconfig                               |  37 +-
 arch/um/Makefile                              |  41 +-
 arch/um/configs/lkl_defconfig                 |  73 +++
 arch/um/drivers/Makefile                      |  10 +-
 .../um/{os-Linux => }/drivers/ethertap_kern.c |   0
 arch/um/{os-Linux => }/drivers/tuntap_kern.c  |   0
 arch/um/include/asm/common.lds.S              |  99 ----
 arch/um/include/asm/host_ops.h                |   9 +
 arch/um/include/asm/mmu.h                     |   3 +
 arch/um/include/asm/mmu_context.h             |  10 +
 arch/um/include/asm/page.h                    |  15 +
 arch/um/include/asm/pgtable.h                 |  27 +
 arch/um/include/asm/thread_info.h             |  24 +
 arch/um/include/asm/uaccess.h                 |   6 +
 arch/um/include/asm/xor.h                     |   3 +-
 arch/um/include/shared/as-layout.h            |   1 +
 .../drivers => include/shared}/etap.h         |   0
 arch/um/include/shared/init.h                 |  19 +-
 arch/um/include/shared/os.h                   |   1 +
 .../drivers => include/shared}/tuntap.h       |   0
 arch/um/kernel/Makefile                       |  13 +-
 arch/um/kernel/dyn.lds.S                      | 171 -------
 arch/um/kernel/irq.c                          |  13 +
 arch/um/kernel/process.c                      |  14 +-
 arch/um/kernel/reboot.c                       |   5 +
 arch/um/kernel/time.c                         |   2 +
 arch/um/kernel/um_arch.c                      |  16 +
 arch/um/kernel/uml.lds.S                      | 115 -----
 arch/um/{os-Linux => kernel}/user_syms.c      |   0
 arch/um/kernel/vmlinux.lds.S                  |  92 +++-
 arch/um/lkl/Makefile                          |   2 +
 arch/um/lkl/Makefile.um                       |  18 +
 arch/um/lkl/include/asm/Kbuild                |   7 +
 arch/um/lkl/include/asm/archparam.h           |   1 +
 arch/um/lkl/include/asm/atomic.h              |  11 +
 arch/um/lkl/include/asm/atomic64.h            |  91 ++++
 arch/um/lkl/include/asm/cpu.h                 |  15 +
 arch/um/lkl/include/asm/elf.h                 |  19 +
 arch/um/lkl/include/asm/mm_context.h          |   8 +
 arch/um/lkl/include/asm/processor.h           |  48 ++
 arch/um/lkl/include/asm/ptrace.h              |  21 +
 arch/um/lkl/include/asm/sched.h               |  23 +
 arch/um/lkl/include/asm/segment.h             |   9 +
 arch/um/lkl/include/asm/syscall_wrapper.h     |  57 +++
 arch/um/lkl/include/asm/syscalls.h            |  15 +
 arch/um/lkl/include/uapi/asm/Kbuild           |   6 +
 arch/um/lkl/include/uapi/asm/bitsperlong.h    |  16 +
 arch/um/lkl/include/uapi/asm/byteorder.h      |  13 +
 arch/um/lkl/include/uapi/asm/host_ops.h       | 280 +++++++++++
 arch/um/lkl/include/uapi/asm/sigcontext.h     |  12 +
 arch/um/lkl/include/uapi/asm/syscalls.h       | 301 ++++++++++++
 arch/um/lkl/include/uapi/asm/unistd.h         |  17 +
 arch/um/lkl/um/Kconfig                        |  21 +
 arch/um/lkl/um/Makefile                       |   4 +
 arch/um/lkl/um/bootmem.c                      | 107 ++++
 arch/um/lkl/um/console.c                      |  41 ++
 arch/um/lkl/um/cpu.c                          | 269 ++++++++++
 arch/um/lkl/um/delay.c                        |  31 ++
 arch/um/lkl/um/setup.c                        | 179 +++++++
 arch/um/lkl/um/shared/sysdep/archsetjmp.h     |  13 +
 arch/um/lkl/um/shared/sysdep/faultinfo.h      |   8 +
 arch/um/lkl/um/shared/sysdep/kernel-offsets.h |  12 +
 arch/um/lkl/um/shared/sysdep/mcontext.h       |   9 +
 arch/um/lkl/um/shared/sysdep/ptrace.h         |  42 ++
 arch/um/lkl/um/shared/sysdep/ptrace_user.h    |   7 +
 arch/um/lkl/um/syscalls.c                     | 193 ++++++++
 arch/um/lkl/um/threads.c                      | 261 ++++++++++
 arch/um/lkl/um/unimplemented.c                |  70 +++
 arch/um/lkl/um/user_constants.h               |  13 +
 arch/um/os-Linux/Makefile                     |  21 -
 arch/um/os-Linux/drivers/Makefile             |  13 -
 arch/um/scripts/headers_install.py            | 200 ++++++++
 arch/x86/um/Makefile                          |   2 +-
 arch/x86/um/os-Linux/Makefile                 |  13 -
 arch/x86/um/ptrace_32.c                       |   2 +-
 arch/x86/um/syscalls_64.c                     |   2 +-
 scripts/headers_install.sh                    |   6 +-
 scripts/link-vmlinux.sh                       |  42 +-
 tools/testing/selftests/Makefile              |   3 +
 tools/testing/selftests/um/Makefile           |  16 +
 tools/testing/selftests/um/boot.c             | 376 ++++++++++++++
 tools/testing/selftests/um/cla.c              | 159 ++++++
 tools/testing/selftests/um/cla.h              |  33 ++
 tools/testing/selftests/um/disk-ext4.sh       |   6 +
 tools/testing/selftests/um/disk-vfat.sh       |   6 +
 tools/testing/selftests/um/disk.c             | 166 +++++++
 tools/testing/selftests/um/disk.sh            |  67 +++
 tools/testing/selftests/um/test.c             | 128 +++++
 tools/testing/selftests/um/test.h             |  72 +++
 tools/testing/selftests/um/test.sh            | 181 +++++++
 tools/um/.gitignore                           |   1 +
 tools/um/Makefile                             |  76 +++
 tools/um/Targets                              |   9 +
 tools/um/include/lkl.h                        | 364 ++++++++++++++
 tools/um/include/lkl_host.h                   |  19 +
 tools/um/lib/Build                            |   7 +
 tools/um/lib/fs.c                             | 461 ++++++++++++++++++
 tools/um/lib/jmp_buf.c                        |  14 +
 tools/um/lib/posix-host.c                     | 273 +++++++++++
 tools/um/lib/utils.c                          | 207 ++++++++
 tools/um/uml/Build                            |  59 +++
 tools/um/uml/drivers/Build                    |  10 +
 .../um/uml}/drivers/ethertap_user.c           |   0
 .../um/uml}/drivers/tuntap_user.c             |   0
 {arch/um/os-Linux => tools/um/uml}/elf_aux.c  |   0
 {arch/um/os-Linux => tools/um/uml}/execvp.c   |   4 -
 {arch/um/os-Linux => tools/um/uml}/file.c     |   0
 {arch/um/os-Linux => tools/um/uml}/helper.c   |   0
 {arch/um/os-Linux => tools/um/uml}/irq.c      |   0
 tools/um/uml/lkl/Build                        |   1 +
 tools/um/uml/lkl/registers.c                  |  21 +
 tools/um/uml/lkl/unimplemented.c              |  21 +
 {arch/um/os-Linux => tools/um/uml}/main.c     |   0
 {arch/um/os-Linux => tools/um/uml}/mem.c      |   0
 {arch/um/os-Linux => tools/um/uml}/process.c  |   2 +
 .../um/os-Linux => tools/um/uml}/registers.c  |   0
 {arch/um/os-Linux => tools/um/uml}/sigio.c    |   0
 {arch/um/os-Linux => tools/um/uml}/signal.c   |  12 +-
 .../skas/Makefile => tools/um/uml/skas/Build  |   6 +-
 {arch/um/os-Linux => tools/um/uml}/skas/mem.c |   0
 .../os-Linux => tools/um/uml}/skas/process.c  |   3 +-
 {arch/um/os-Linux => tools/um/uml}/start_up.c |   0
 {arch/um/os-Linux => tools/um/uml}/time.c     |   0
 {arch/um/os-Linux => tools/um/uml}/tty.c      |   0
 {arch/um/os-Linux => tools/um/uml}/umid.c     |   0
 {arch/um/os-Linux => tools/um/uml}/util.c     |  26 +
 tools/um/uml/x86/Build                        |  11 +
 .../os-Linux => tools/um/uml/x86}/mcontext.c  |   0
 .../um/os-Linux => tools/um/uml/x86}/prctl.c  |   0
 .../os-Linux => tools/um/uml/x86}/registers.c |   0
 .../os-Linux => tools/um/uml/x86}/task_size.c |   0
 .../um/os-Linux => tools/um/uml/x86}/tls.c    |   0
 134 files changed, 5743 insertions(+), 525 deletions(-)
 create mode 100644 Documentation/virt/uml/lkl.txt
 create mode 100644 arch/um/configs/lkl_defconfig
 rename arch/um/{os-Linux => }/drivers/ethertap_kern.c (100%)
 rename arch/um/{os-Linux => }/drivers/tuntap_kern.c (100%)
 delete mode 100644 arch/um/include/asm/common.lds.S
 create mode 100644 arch/um/include/asm/host_ops.h
 rename arch/um/{os-Linux/drivers => include/shared}/etap.h (100%)
 rename arch/um/{os-Linux/drivers => include/shared}/tuntap.h (100%)
 delete mode 100644 arch/um/kernel/dyn.lds.S
 delete mode 100644 arch/um/kernel/uml.lds.S
 rename arch/um/{os-Linux => kernel}/user_syms.c (100%)
 create mode 100644 arch/um/lkl/Makefile
 create mode 100644 arch/um/lkl/Makefile.um
 create mode 100644 arch/um/lkl/include/asm/Kbuild
 create mode 100644 arch/um/lkl/include/asm/archparam.h
 create mode 100644 arch/um/lkl/include/asm/atomic.h
 create mode 100644 arch/um/lkl/include/asm/atomic64.h
 create mode 100644 arch/um/lkl/include/asm/cpu.h
 create mode 100644 arch/um/lkl/include/asm/elf.h
 create mode 100644 arch/um/lkl/include/asm/mm_context.h
 create mode 100644 arch/um/lkl/include/asm/processor.h
 create mode 100644 arch/um/lkl/include/asm/ptrace.h
 create mode 100644 arch/um/lkl/include/asm/sched.h
 create mode 100644 arch/um/lkl/include/asm/segment.h
 create mode 100644 arch/um/lkl/include/asm/syscall_wrapper.h
 create mode 100644 arch/um/lkl/include/asm/syscalls.h
 create mode 100644 arch/um/lkl/include/uapi/asm/Kbuild
 create mode 100644 arch/um/lkl/include/uapi/asm/bitsperlong.h
 create mode 100644 arch/um/lkl/include/uapi/asm/byteorder.h
 create mode 100644 arch/um/lkl/include/uapi/asm/host_ops.h
 create mode 100644 arch/um/lkl/include/uapi/asm/sigcontext.h
 create mode 100644 arch/um/lkl/include/uapi/asm/syscalls.h
 create mode 100644 arch/um/lkl/include/uapi/asm/unistd.h
 create mode 100644 arch/um/lkl/um/Kconfig
 create mode 100644 arch/um/lkl/um/Makefile
 create mode 100644 arch/um/lkl/um/bootmem.c
 create mode 100644 arch/um/lkl/um/console.c
 create mode 100644 arch/um/lkl/um/cpu.c
 create mode 100644 arch/um/lkl/um/delay.c
 create mode 100644 arch/um/lkl/um/setup.c
 create mode 100644 arch/um/lkl/um/shared/sysdep/archsetjmp.h
 create mode 100644 arch/um/lkl/um/shared/sysdep/faultinfo.h
 create mode 100644 arch/um/lkl/um/shared/sysdep/kernel-offsets.h
 create mode 100644 arch/um/lkl/um/shared/sysdep/mcontext.h
 create mode 100644 arch/um/lkl/um/shared/sysdep/ptrace.h
 create mode 100644 arch/um/lkl/um/shared/sysdep/ptrace_user.h
 create mode 100644 arch/um/lkl/um/syscalls.c
 create mode 100644 arch/um/lkl/um/threads.c
 create mode 100644 arch/um/lkl/um/unimplemented.c
 create mode 100644 arch/um/lkl/um/user_constants.h
 delete mode 100644 arch/um/os-Linux/Makefile
 delete mode 100644 arch/um/os-Linux/drivers/Makefile
 create mode 100755 arch/um/scripts/headers_install.py
 delete mode 100644 arch/x86/um/os-Linux/Makefile
 create mode 100644 tools/testing/selftests/um/Makefile
 create mode 100644 tools/testing/selftests/um/boot.c
 create mode 100644 tools/testing/selftests/um/cla.c
 create mode 100644 tools/testing/selftests/um/cla.h
 create mode 100755 tools/testing/selftests/um/disk-ext4.sh
 create mode 100755 tools/testing/selftests/um/disk-vfat.sh
 create mode 100644 tools/testing/selftests/um/disk.c
 create mode 100755 tools/testing/selftests/um/disk.sh
 create mode 100644 tools/testing/selftests/um/test.c
 create mode 100644 tools/testing/selftests/um/test.h
 create mode 100644 tools/testing/selftests/um/test.sh
 create mode 100644 tools/um/.gitignore
 create mode 100644 tools/um/Makefile
 create mode 100644 tools/um/Targets
 create mode 100644 tools/um/include/lkl.h
 create mode 100644 tools/um/include/lkl_host.h
 create mode 100644 tools/um/lib/Build
 create mode 100644 tools/um/lib/fs.c
 create mode 100644 tools/um/lib/jmp_buf.c
 create mode 100644 tools/um/lib/posix-host.c
 create mode 100644 tools/um/lib/utils.c
 create mode 100644 tools/um/uml/Build
 create mode 100644 tools/um/uml/drivers/Build
 rename {arch/um/os-Linux => tools/um/uml}/drivers/ethertap_user.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/drivers/tuntap_user.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/elf_aux.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/execvp.c (98%)
 rename {arch/um/os-Linux => tools/um/uml}/file.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/helper.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/irq.c (100%)
 create mode 100644 tools/um/uml/lkl/Build
 create mode 100644 tools/um/uml/lkl/registers.c
 create mode 100644 tools/um/uml/lkl/unimplemented.c
 rename {arch/um/os-Linux => tools/um/uml}/main.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/mem.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/process.c (99%)
 rename {arch/um/os-Linux => tools/um/uml}/registers.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/sigio.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/signal.c (96%)
 rename arch/um/os-Linux/skas/Makefile => tools/um/uml/skas/Build (56%)
 rename {arch/um/os-Linux => tools/um/uml}/skas/mem.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/skas/process.c (99%)
 rename {arch/um/os-Linux => tools/um/uml}/start_up.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/time.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/tty.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/umid.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/util.c (89%)
 create mode 100644 tools/um/uml/x86/Build
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/mcontext.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/prctl.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/registers.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/task_size.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/tls.c (100%)

-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply	[flat|nested] 476+ messages in thread
* [RFC v8 00/20] Unifying LKL into UML
@ 2021-01-20  2:27           ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-01-20  2:27 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, thehajime, retrage01

This is another spin of the unification of LKL into UML.  Updated and
fixed comments from our v7 patches.  The summary is listed in the
changelog below.

Note that the whole patchset requires the patch "um: ubd: fix command
line handling of ubd" to be correctly tested, which I previously
submitted.


Changes in rfc v8:
- stop using the term "nommu" mode, "library" mode instead
- generic atomic64 rewrite to use gcc builtin atomics
- use kernel-doc format for comments
- stop using a collection of function pointer (struct lkl_host_operations),
  but weak functions
- use python3, not python
- use c99 initializers for strerror
- add comments/descriptions about thread implementation
- remove redundant Kconfig entries (RAID6_PQ_BENCHMARK, STACKTRACE_SUPPORT,
  etc)
- refine code comments
- stop using NPROC for the internal -j flag
- remove make argument of "UMMODE=library"
- preserve the vmlinux file as it is
- rework commit history with reasonable files
- rework test framework using kselftest (tools/testing/selftests/um/)
- add documentation, MAINTAINERS


Changes in rfc v7:
- preserve `make ARCH=um` syntax to build UML
- introduce `make ARCH=um UMMODE=library` to build library mode
- fix undefined symbols issue during modpost
- clean up makefiles (arch/um, tools/um)

Changes in rfc v6:
- rebase with the current linus tree

Changes in rfc v5:
- rewrite whole patchset from scratch
- move arch-dependent code of arch/um and arch/x86/um to tools/um
 - mainly code under os-Linux/ involved
 - introduce 2-stage build (kernel, and host-dependent parts)
 - clean up vmlinux.lds.S
- put LKL-specific implementations as a SUBARCH under arch/um/nommu
- introduce !CONFIG_MMU in arch/um
- use struct arch_thread and arch_switch_to() for subarch-specific
  thread implementation
- integrate with the IRQ infrastructure of UML
- tested with block device drivers (ubd) for the proof

Changes in rfc v4: (https://lwn.net/Articles/816276/)
- Rebase on the current uml/master branch
- Fix IRQ handling (bug fix)
- drop a patch for CONFIG_GENERIC_ATOMIC64 (comment by Peter Zijlstra)
- implement vector net driver for UMMODE_LIB (comment by Anton)
- clean up uapi headers to avoid duplicates
- clean up IRQ handling code (comment by Anton)
- fix error handling in test code (comments by David Disseldorp)

Changes in rfc v3:
- use UML drivers (net, block) from LKL programs
- drop virtio device implementations
- drop mingw32 (Windows) host
- drop android (arm/aarch64) host
- drop FreeBSD (x86_64) host
- drop LD_PRELOAD (hijack) support
- update milestone

rfc v2:
- use UMMODE instead of SUBARCH to switch UML or LKL
- tools/lkl directory is still there. I confirmed we can move under arch/um
  (e.g., arch/um/lkl/hosts).  I will move it IF this is preferable.
- drop several patches involved non-uml directory
- drop several patches which are not required
- refine commit logs
- document updated




LKL (Linux Kernel Library) is aiming to allow reusing the Linux kernel code
as extensively as possible with minimal effort and reduced maintenance
overhead.

Examples of how LKL can be used are: creating userspace applications
(running on Linux and other operating systems) that can read or write Linux
filesystems or can use the Linux networking stack, creating kernel drivers
for other operating systems that can read Linux filesystems, bootloaders
support for reading/writing Linux filesystems, etc.

With LKL, the kernel code is compiled into an object file that can be
directly linked by applications. The API offered by LKL is based on the
Linux system call interface.

LKL is originally implemented as an architecture port in arch/lkl, but this
series of commits tries to integrate this into arch/um as one of the mode
of UML.  This was discussed during RFC email of LKL (*1).

The latest LKL version can be found at https://github.com/lkl/linux

Milestone
=========
This patches is a first step toward upstreaming *library mode* of
Linux kernel, but we think we need to have several steps toward our
goal, describing in the below.


Milestone 1: LKL lib on top of UML
 * Kernel - Host build split
 -  Build UML as a relocatable object using the UML's kernel linker script.
 -  Move the ptrace and other well isolated os code out of arch/um to
    tools/um
 -  Use standard host toolchain to create a static library stripped of
    the ptrace code. Use standard host toolchain to build the main UML
    executable.
 -  Add library init API that creates the UML kernel process and starts
    UML.
 * System calls APIs
 -  Add new system call interface based on UML's irq facility.
 -  Use the LKL scripts to export the required headers to create system
    calls APIs that use the UML system calls infrastructure.
 -  Keep the underlying host and driver operations (threads, irqs, etc.)
    as they are now in UML.
 * Boot test
 -  Port the LKL boot test to verify that we are able to programatically
    issue system calls.

Milestone 2: add virtio disk support
 * Export asm/io.h operations to host/os. Create IO access operations
   and redirect them to weak os_ variants that use the current UML
   implementation.
 * Add the LKL IO access layer including generic virtio handling and the
   virtio block device code.
 * Port LKL disk test and disk apps (lklfuse, fs2tar, cptofs)

Milestone 3: new arch ports
  * Abstract the system call / IRQ mode the move the implementation to host
  * Abstract the thread model and move the implementation to host
  * Add LKL thread model and LKL ports


Building LKL the host library and LKL applications
==================================================

% make ARCH=um SUBARCH=lkl defconfig
% make ARCH=um SUBARCH=lkl

will build LKL as a object file, it will install it in
tools/um/libtogether with the headers files in tools/um/include then
will build the host library, tests and a few of application examples:

* tools/testing/selftests/um/boot.c - a simple applications that uses
  LKL and exercises the basic LKL APIs

* tools/testing/selftests/um/disk.c - a simple applications that tests
  LKL and exercises the basic filesystem-related LKL APIs

Those tests can run with the following kselftest command:

    $ make ARCH=um SUBARCH=lkl TARGETS="um" kselftest

Supported hosts
===============

Currently LKL supports Linux userspace applications. New hosts can be added
relatively easy if the host supports gcc and GNU ld. Previous versions of
LKL supported Windows kernel and Haiku kernel hosts, and we also have WIP
patches with rump-hypercall interface, used in UEFI, as well as macOS
userspace (part of POSIX).

There is also musl-libc port for LKL, which might be interested in for some
folks.


Further readings about LKL
=========================

- Discussion in github LKL issue
https://github.com/lkl/linux/issues/304

- LKL (an article)
https://www.researchgate.net/profile/Nicolae_Tapus2/publication/224164682_LKL_The_Linux_kernel_library/links/02bfe50fd921ab4f7c000000.pdf

*1 RFC email to LKML (back in 2015)
https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1012277.html


Please review the following changes for suitability for inclusion. If you have
any objections or suggestions for improvement, please respond to the patches. If
you agree with the changes, please provide your Acked-by.

The following changes since commit 19c329f6808995b142b3966301f217c831e7cf31:

  Linux 5.11-rc4 (2021-01-17 16:37:05 -0800)

are available in the Git repository at:

  git://github.com/thehajime/linux 043677211bb5562397c511911e7861c3e217611b
  https://github.com/thehajime/linux/tree/uml-lkl-5.11rc4-v8

Hajime Tazaki (18):
  um: move arch/um/os-Linux dir to tools/um/uml
  um: move arch/x86/um/os-Linux to tools/um/uml/
  um: extend arch_switch_to for alternate SUBARCH
  um: add UML library mode
  um: lkl: host interface
  um: lkl: memory handling
  um: lkl: kernel thread support
  um: lkl: system call interface and application API
  um: lkl: basic console support
  um: lkl: initialization and cleanup
  um: lkl: integrate with irq infrastructure of UML
  um: lkl: plug in the build system
  um: host: add library mode build for ARCH=um
  um: host: add utilities functions
  um: host: posix host operations
  selftests/um: lkl: add test programs for library mode of UML
  um: lkl: add block device support of UML
  um: lkl: add documentation

Octavian Purdila (2):
  um: split build in kernel and host parts
  um: implement os_initcalls and os_exitcalls

 Documentation/virt/uml/lkl.txt                |  48 ++
 MAINTAINERS                                   |  10 +
 arch/um/Kconfig                               |  37 +-
 arch/um/Makefile                              |  41 +-
 arch/um/configs/lkl_defconfig                 |  73 +++
 arch/um/drivers/Makefile                      |  10 +-
 .../um/{os-Linux => }/drivers/ethertap_kern.c |   0
 arch/um/{os-Linux => }/drivers/tuntap_kern.c  |   0
 arch/um/include/asm/common.lds.S              |  99 ----
 arch/um/include/asm/host_ops.h                |   9 +
 arch/um/include/asm/mmu.h                     |   3 +
 arch/um/include/asm/mmu_context.h             |  10 +
 arch/um/include/asm/page.h                    |  15 +
 arch/um/include/asm/pgtable.h                 |  27 +
 arch/um/include/asm/thread_info.h             |  24 +
 arch/um/include/asm/uaccess.h                 |   6 +
 arch/um/include/asm/xor.h                     |   3 +-
 arch/um/include/shared/as-layout.h            |   1 +
 .../drivers => include/shared}/etap.h         |   0
 arch/um/include/shared/init.h                 |  19 +-
 arch/um/include/shared/os.h                   |   1 +
 .../drivers => include/shared}/tuntap.h       |   0
 arch/um/kernel/Makefile                       |  13 +-
 arch/um/kernel/dyn.lds.S                      | 171 -------
 arch/um/kernel/irq.c                          |  13 +
 arch/um/kernel/process.c                      |  14 +-
 arch/um/kernel/reboot.c                       |   5 +
 arch/um/kernel/time.c                         |   2 +
 arch/um/kernel/um_arch.c                      |  16 +
 arch/um/kernel/uml.lds.S                      | 115 -----
 arch/um/{os-Linux => kernel}/user_syms.c      |   0
 arch/um/kernel/vmlinux.lds.S                  |  92 +++-
 arch/um/lkl/Makefile                          |   2 +
 arch/um/lkl/Makefile.um                       |  18 +
 arch/um/lkl/include/asm/Kbuild                |   7 +
 arch/um/lkl/include/asm/archparam.h           |   1 +
 arch/um/lkl/include/asm/atomic.h              |  11 +
 arch/um/lkl/include/asm/atomic64.h            |  91 ++++
 arch/um/lkl/include/asm/cpu.h                 |  15 +
 arch/um/lkl/include/asm/elf.h                 |  19 +
 arch/um/lkl/include/asm/mm_context.h          |   8 +
 arch/um/lkl/include/asm/processor.h           |  48 ++
 arch/um/lkl/include/asm/ptrace.h              |  21 +
 arch/um/lkl/include/asm/sched.h               |  23 +
 arch/um/lkl/include/asm/segment.h             |   9 +
 arch/um/lkl/include/asm/syscall_wrapper.h     |  57 +++
 arch/um/lkl/include/asm/syscalls.h            |  15 +
 arch/um/lkl/include/uapi/asm/Kbuild           |   6 +
 arch/um/lkl/include/uapi/asm/bitsperlong.h    |  16 +
 arch/um/lkl/include/uapi/asm/byteorder.h      |  13 +
 arch/um/lkl/include/uapi/asm/host_ops.h       | 280 +++++++++++
 arch/um/lkl/include/uapi/asm/sigcontext.h     |  12 +
 arch/um/lkl/include/uapi/asm/syscalls.h       | 301 ++++++++++++
 arch/um/lkl/include/uapi/asm/unistd.h         |  17 +
 arch/um/lkl/um/Kconfig                        |  21 +
 arch/um/lkl/um/Makefile                       |   4 +
 arch/um/lkl/um/bootmem.c                      | 107 ++++
 arch/um/lkl/um/console.c                      |  41 ++
 arch/um/lkl/um/cpu.c                          | 269 ++++++++++
 arch/um/lkl/um/delay.c                        |  31 ++
 arch/um/lkl/um/setup.c                        | 179 +++++++
 arch/um/lkl/um/shared/sysdep/archsetjmp.h     |  13 +
 arch/um/lkl/um/shared/sysdep/faultinfo.h      |   8 +
 arch/um/lkl/um/shared/sysdep/kernel-offsets.h |  12 +
 arch/um/lkl/um/shared/sysdep/mcontext.h       |   9 +
 arch/um/lkl/um/shared/sysdep/ptrace.h         |  42 ++
 arch/um/lkl/um/shared/sysdep/ptrace_user.h    |   7 +
 arch/um/lkl/um/syscalls.c                     | 193 ++++++++
 arch/um/lkl/um/threads.c                      | 261 ++++++++++
 arch/um/lkl/um/unimplemented.c                |  70 +++
 arch/um/lkl/um/user_constants.h               |  13 +
 arch/um/os-Linux/Makefile                     |  21 -
 arch/um/os-Linux/drivers/Makefile             |  13 -
 arch/um/scripts/headers_install.py            | 200 ++++++++
 arch/x86/um/Makefile                          |   2 +-
 arch/x86/um/os-Linux/Makefile                 |  13 -
 arch/x86/um/ptrace_32.c                       |   2 +-
 arch/x86/um/syscalls_64.c                     |   2 +-
 scripts/headers_install.sh                    |   6 +-
 scripts/link-vmlinux.sh                       |  42 +-
 tools/testing/selftests/Makefile              |   3 +
 tools/testing/selftests/um/Makefile           |  16 +
 tools/testing/selftests/um/boot.c             | 376 ++++++++++++++
 tools/testing/selftests/um/cla.c              | 159 ++++++
 tools/testing/selftests/um/cla.h              |  33 ++
 tools/testing/selftests/um/disk-ext4.sh       |   6 +
 tools/testing/selftests/um/disk-vfat.sh       |   6 +
 tools/testing/selftests/um/disk.c             | 166 +++++++
 tools/testing/selftests/um/disk.sh            |  67 +++
 tools/testing/selftests/um/test.c             | 128 +++++
 tools/testing/selftests/um/test.h             |  72 +++
 tools/testing/selftests/um/test.sh            | 181 +++++++
 tools/um/.gitignore                           |   1 +
 tools/um/Makefile                             |  76 +++
 tools/um/Targets                              |   9 +
 tools/um/include/lkl.h                        | 364 ++++++++++++++
 tools/um/include/lkl_host.h                   |  19 +
 tools/um/lib/Build                            |   7 +
 tools/um/lib/fs.c                             | 461 ++++++++++++++++++
 tools/um/lib/jmp_buf.c                        |  14 +
 tools/um/lib/posix-host.c                     | 273 +++++++++++
 tools/um/lib/utils.c                          | 207 ++++++++
 tools/um/uml/Build                            |  59 +++
 tools/um/uml/drivers/Build                    |  10 +
 .../um/uml}/drivers/ethertap_user.c           |   0
 .../um/uml}/drivers/tuntap_user.c             |   0
 {arch/um/os-Linux => tools/um/uml}/elf_aux.c  |   0
 {arch/um/os-Linux => tools/um/uml}/execvp.c   |   4 -
 {arch/um/os-Linux => tools/um/uml}/file.c     |   0
 {arch/um/os-Linux => tools/um/uml}/helper.c   |   0
 {arch/um/os-Linux => tools/um/uml}/irq.c      |   0
 tools/um/uml/lkl/Build                        |   1 +
 tools/um/uml/lkl/registers.c                  |  21 +
 tools/um/uml/lkl/unimplemented.c              |  21 +
 {arch/um/os-Linux => tools/um/uml}/main.c     |   0
 {arch/um/os-Linux => tools/um/uml}/mem.c      |   0
 {arch/um/os-Linux => tools/um/uml}/process.c  |   2 +
 .../um/os-Linux => tools/um/uml}/registers.c  |   0
 {arch/um/os-Linux => tools/um/uml}/sigio.c    |   0
 {arch/um/os-Linux => tools/um/uml}/signal.c   |  12 +-
 .../skas/Makefile => tools/um/uml/skas/Build  |   6 +-
 {arch/um/os-Linux => tools/um/uml}/skas/mem.c |   0
 .../os-Linux => tools/um/uml}/skas/process.c  |   3 +-
 {arch/um/os-Linux => tools/um/uml}/start_up.c |   0
 {arch/um/os-Linux => tools/um/uml}/time.c     |   0
 {arch/um/os-Linux => tools/um/uml}/tty.c      |   0
 {arch/um/os-Linux => tools/um/uml}/umid.c     |   0
 {arch/um/os-Linux => tools/um/uml}/util.c     |  26 +
 tools/um/uml/x86/Build                        |  11 +
 .../os-Linux => tools/um/uml/x86}/mcontext.c  |   0
 .../um/os-Linux => tools/um/uml/x86}/prctl.c  |   0
 .../os-Linux => tools/um/uml/x86}/registers.c |   0
 .../os-Linux => tools/um/uml/x86}/task_size.c |   0
 .../um/os-Linux => tools/um/uml/x86}/tls.c    |   0
 134 files changed, 5743 insertions(+), 525 deletions(-)
 create mode 100644 Documentation/virt/uml/lkl.txt
 create mode 100644 arch/um/configs/lkl_defconfig
 rename arch/um/{os-Linux => }/drivers/ethertap_kern.c (100%)
 rename arch/um/{os-Linux => }/drivers/tuntap_kern.c (100%)
 delete mode 100644 arch/um/include/asm/common.lds.S
 create mode 100644 arch/um/include/asm/host_ops.h
 rename arch/um/{os-Linux/drivers => include/shared}/etap.h (100%)
 rename arch/um/{os-Linux/drivers => include/shared}/tuntap.h (100%)
 delete mode 100644 arch/um/kernel/dyn.lds.S
 delete mode 100644 arch/um/kernel/uml.lds.S
 rename arch/um/{os-Linux => kernel}/user_syms.c (100%)
 create mode 100644 arch/um/lkl/Makefile
 create mode 100644 arch/um/lkl/Makefile.um
 create mode 100644 arch/um/lkl/include/asm/Kbuild
 create mode 100644 arch/um/lkl/include/asm/archparam.h
 create mode 100644 arch/um/lkl/include/asm/atomic.h
 create mode 100644 arch/um/lkl/include/asm/atomic64.h
 create mode 100644 arch/um/lkl/include/asm/cpu.h
 create mode 100644 arch/um/lkl/include/asm/elf.h
 create mode 100644 arch/um/lkl/include/asm/mm_context.h
 create mode 100644 arch/um/lkl/include/asm/processor.h
 create mode 100644 arch/um/lkl/include/asm/ptrace.h
 create mode 100644 arch/um/lkl/include/asm/sched.h
 create mode 100644 arch/um/lkl/include/asm/segment.h
 create mode 100644 arch/um/lkl/include/asm/syscall_wrapper.h
 create mode 100644 arch/um/lkl/include/asm/syscalls.h
 create mode 100644 arch/um/lkl/include/uapi/asm/Kbuild
 create mode 100644 arch/um/lkl/include/uapi/asm/bitsperlong.h
 create mode 100644 arch/um/lkl/include/uapi/asm/byteorder.h
 create mode 100644 arch/um/lkl/include/uapi/asm/host_ops.h
 create mode 100644 arch/um/lkl/include/uapi/asm/sigcontext.h
 create mode 100644 arch/um/lkl/include/uapi/asm/syscalls.h
 create mode 100644 arch/um/lkl/include/uapi/asm/unistd.h
 create mode 100644 arch/um/lkl/um/Kconfig
 create mode 100644 arch/um/lkl/um/Makefile
 create mode 100644 arch/um/lkl/um/bootmem.c
 create mode 100644 arch/um/lkl/um/console.c
 create mode 100644 arch/um/lkl/um/cpu.c
 create mode 100644 arch/um/lkl/um/delay.c
 create mode 100644 arch/um/lkl/um/setup.c
 create mode 100644 arch/um/lkl/um/shared/sysdep/archsetjmp.h
 create mode 100644 arch/um/lkl/um/shared/sysdep/faultinfo.h
 create mode 100644 arch/um/lkl/um/shared/sysdep/kernel-offsets.h
 create mode 100644 arch/um/lkl/um/shared/sysdep/mcontext.h
 create mode 100644 arch/um/lkl/um/shared/sysdep/ptrace.h
 create mode 100644 arch/um/lkl/um/shared/sysdep/ptrace_user.h
 create mode 100644 arch/um/lkl/um/syscalls.c
 create mode 100644 arch/um/lkl/um/threads.c
 create mode 100644 arch/um/lkl/um/unimplemented.c
 create mode 100644 arch/um/lkl/um/user_constants.h
 delete mode 100644 arch/um/os-Linux/Makefile
 delete mode 100644 arch/um/os-Linux/drivers/Makefile
 create mode 100755 arch/um/scripts/headers_install.py
 delete mode 100644 arch/x86/um/os-Linux/Makefile
 create mode 100644 tools/testing/selftests/um/Makefile
 create mode 100644 tools/testing/selftests/um/boot.c
 create mode 100644 tools/testing/selftests/um/cla.c
 create mode 100644 tools/testing/selftests/um/cla.h
 create mode 100755 tools/testing/selftests/um/disk-ext4.sh
 create mode 100755 tools/testing/selftests/um/disk-vfat.sh
 create mode 100644 tools/testing/selftests/um/disk.c
 create mode 100755 tools/testing/selftests/um/disk.sh
 create mode 100644 tools/testing/selftests/um/test.c
 create mode 100644 tools/testing/selftests/um/test.h
 create mode 100644 tools/testing/selftests/um/test.sh
 create mode 100644 tools/um/.gitignore
 create mode 100644 tools/um/Makefile
 create mode 100644 tools/um/Targets
 create mode 100644 tools/um/include/lkl.h
 create mode 100644 tools/um/include/lkl_host.h
 create mode 100644 tools/um/lib/Build
 create mode 100644 tools/um/lib/fs.c
 create mode 100644 tools/um/lib/jmp_buf.c
 create mode 100644 tools/um/lib/posix-host.c
 create mode 100644 tools/um/lib/utils.c
 create mode 100644 tools/um/uml/Build
 create mode 100644 tools/um/uml/drivers/Build
 rename {arch/um/os-Linux => tools/um/uml}/drivers/ethertap_user.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/drivers/tuntap_user.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/elf_aux.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/execvp.c (98%)
 rename {arch/um/os-Linux => tools/um/uml}/file.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/helper.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/irq.c (100%)
 create mode 100644 tools/um/uml/lkl/Build
 create mode 100644 tools/um/uml/lkl/registers.c
 create mode 100644 tools/um/uml/lkl/unimplemented.c
 rename {arch/um/os-Linux => tools/um/uml}/main.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/mem.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/process.c (99%)
 rename {arch/um/os-Linux => tools/um/uml}/registers.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/sigio.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/signal.c (96%)
 rename arch/um/os-Linux/skas/Makefile => tools/um/uml/skas/Build (56%)
 rename {arch/um/os-Linux => tools/um/uml}/skas/mem.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/skas/process.c (99%)
 rename {arch/um/os-Linux => tools/um/uml}/start_up.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/time.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/tty.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/umid.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/util.c (89%)
 create mode 100644 tools/um/uml/x86/Build
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/mcontext.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/prctl.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/registers.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/task_size.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/tls.c (100%)

-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* [RFC v8 01/20] um: split build in kernel and host parts
  2021-01-20  2:27           ` Hajime Tazaki
@ 2021-01-20  2:27             ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-01-20  2:27 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: thehajime, tavi.purdila, retrage01, linux-kernel-library,
	linux-arch, Octavian Purdila, Masahiro Yamada, Michal Marek,
	linux-kbuild

From: Octavian Purdila <tavi@cs.pub.ro>

This patch splits the UML build in two parts: a kernel part that
generates a relocatable object (linux.o) and a host build part that
links the relocatable object into an executable.

This allows us to simplify the linker script since we can now remove
the host bits (e.g. .rel sections). It also gives us greater
flexibility since it will be much easier to support other
architectures and OSes if we use the standard linker script.

The host build part has been implemented in tools/um so that we can
reuse the available host build infrastructure.

The patch preserves the way to invoke the UML build as follows,

 $ make ARCH=um defconfig
 $ make ARCH=um

but it internally calls tools/um/Makefile in order to complete the full
build to link the relocatable object and the rest of files.

This commit also relaxes the condition of UML in scripts/link-vmlinux.sh
since we can postpone the link process with host libraries (-lpthread,
etc) to the 2nd starge.

Cc: Masahiro Yamada <masahiroy@kernel.org>
Cc: Michal Marek <michal.lkml@markovi.net>
Cc: linux-kbuild@vger.kernel.org
Signed-off-by: Octavian Purdila <tavi@cs.pub.ro>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/Kconfig                  |  12 +--
 arch/um/Makefile                 |  29 ++++--
 arch/um/include/asm/common.lds.S |  99 ------------------
 arch/um/kernel/dyn.lds.S         | 171 -------------------------------
 arch/um/kernel/uml.lds.S         | 115 ---------------------
 arch/um/kernel/vmlinux.lds.S     |  92 +++++++++++++++--
 scripts/link-vmlinux.sh          |  42 +++-----
 tools/um/Makefile                |  74 +++++++++++++
 tools/um/Targets                 |   4 +
 tools/um/uml/Build               |   0
 10 files changed, 200 insertions(+), 438 deletions(-)
 delete mode 100644 arch/um/include/asm/common.lds.S
 delete mode 100644 arch/um/kernel/dyn.lds.S
 delete mode 100644 arch/um/kernel/uml.lds.S
 create mode 100644 tools/um/Makefile
 create mode 100644 tools/um/Targets
 create mode 100644 tools/um/uml/Build

diff --git a/arch/um/Kconfig b/arch/um/Kconfig
index 34d302d1a07f..9e2d9efbcdd1 100644
--- a/arch/um/Kconfig
+++ b/arch/um/Kconfig
@@ -21,6 +21,7 @@ config UML
 	select HAVE_GCC_PLUGINS
 	select SET_FS
 	select TTY # Needed for line.c
+	select MODULE_REL_CRCS if MODVERSIONS
 
 config MMU
 	bool
@@ -80,17 +81,6 @@ config STATIC_LINK
 	  NOTE: This option is incompatible with some networking features which
 	  depend on features that require being dynamically loaded (like NSS).
 
-config LD_SCRIPT_STATIC
-	bool
-	default y
-	depends on STATIC_LINK
-
-config LD_SCRIPT_DYN
-	bool
-	default y
-	depends on !LD_SCRIPT_STATIC
-	select MODULE_REL_CRCS if MODVERSIONS
-
 config HOSTFS
 	tristate "Host filesystem"
 	help
diff --git a/arch/um/Makefile b/arch/um/Makefile
index 1cea46ff9bb7..48fbbc3a0032 100644
--- a/arch/um/Makefile
+++ b/arch/um/Makefile
@@ -95,14 +95,30 @@ KBUILD_CPPFLAGS += -I$(srctree)/$(HOST_DIR)/include \
 KERNEL_DEFINES = $(strip -Derrno=kernel_errno -Dsigprocmask=kernel_sigprocmask \
 			 -Dmktime=kernel_mktime $(ARCH_KERNEL_DEFINES))
 KBUILD_CFLAGS += $(KERNEL_DEFINES)
+LDFLAGS_vmlinux += -r
 
-PHONY += linux
-
+INSTALL_PATH=$(objtree)/tools/um
 all: linux
-
-linux: vmlinux
-	@echo '  LINK $@'
-	$(Q)ln -f $< $@
+pre-2nd: linux.o
+	@echo "  INSTALL $(INSTALL_PATH)/lib/$<"
+	@mkdir -p $(INSTALL_PATH)/lib/
+	@cp $< $(INSTALL_PATH)/lib/
+
+PHONY += linux.o
+
+linux: pre-2nd
+	$(Q)$(MAKE) -C $(srctree)/tools/um
+	$(Q)cp $(objtree)/tools/um/uml/linux $(objtree)/
+	$(Q)cp $(objtree)/linux $(objtree)/vmlinux
+
+linux.o: vmlinux
+#revert vmlinux file from vmlinux.a
+ifneq ("$(wildcard vmlinux.a)","")
+	$(Q)cp vmlinux.a vmlinux
+endif
+	$(Q)cp vmlinux vmlinux.a
+	@echo "  LINK    $@"
+	$(Q)$(OBJCOPY) -R .eh_frame $< $@
 
 define archhelp
   echo '* linux		- Binary kernel image (./linux) - for backward'
@@ -144,5 +160,6 @@ MRPROPER_FILES += arch/$(SUBARCH)/include/generated
 archclean:
 	@find . \( -name '*.bb' -o -name '*.bbg' -o -name '*.da' \
 		-o -name '*.gcov' \) -type f -print | xargs rm -f
+	$(Q)$(MAKE) -C $(srctree)/tools/um clean
 
 export HEADER_ARCH SUBARCH USER_CFLAGS CFLAGS_NO_HARDENING OS DEV_NULL_PATH
diff --git a/arch/um/include/asm/common.lds.S b/arch/um/include/asm/common.lds.S
deleted file mode 100644
index eca6c452a41b..000000000000
--- a/arch/um/include/asm/common.lds.S
+++ /dev/null
@@ -1,99 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#include <asm-generic/vmlinux.lds.h>
-
-  .fini      : { *(.fini)    } =0x9090
-  _etext = .;
-  PROVIDE (etext = .);
-
-  . = ALIGN(4096);
-  _sdata = .;
-  PROVIDE (sdata = .);
-
-  RO_DATA(4096)
-
-  .unprotected : { *(.unprotected) }
-  . = ALIGN(4096);
-  PROVIDE (_unprotected_end = .);
-
-  . = ALIGN(4096);
-  EXCEPTION_TABLE(0)
-
-  BUG_TABLE
-
-  .uml.setup.init : {
-	__uml_setup_start = .;
-	*(.uml.setup.init)
-	__uml_setup_end = .;
-  }
-	
-  .uml.help.init : {
-	__uml_help_start = .;
-	*(.uml.help.init)
-	__uml_help_end = .;
-  }
-	
-  .uml.postsetup.init : {
-	__uml_postsetup_start = .;
-	*(.uml.postsetup.init)
-	__uml_postsetup_end = .;
-  }
-	
-  .init.setup : {
-	INIT_SETUP(0)
-  }
-
-  PERCPU_SECTION(32)
-	
-  .initcall.init : {
-	INIT_CALLS
-  }
-
-  .con_initcall.init : {
-	CON_INITCALL
-  }
-
-  .exitcall : {
-	__exitcall_begin = .;
-	*(.exitcall.exit)
-	__exitcall_end = .;
-  }
-
-  .uml.exitcall : {
-	__uml_exitcall_begin = .;
-	*(.uml.exitcall.exit)
-	__uml_exitcall_end = .;
-  }
-
-  . = ALIGN(4);
-  .altinstructions : {
-	__alt_instructions = .;
-	*(.altinstructions)
-	__alt_instructions_end = .;
-  }
-  .altinstr_replacement : { *(.altinstr_replacement) }
-  /* .exit.text is discard at runtime, not link time, to deal with references
-     from .altinstructions and .eh_frame */
-  .exit.text : { EXIT_TEXT }
-  .exit.data : { *(.exit.data) }
-
-  .preinit_array : {
-	__preinit_array_start = .;
-	*(.preinit_array)
-	__preinit_array_end = .;
-  }
-  .init_array : {
-	__init_array_start = .;
-	*(.init_array)
-	__init_array_end = .;
-  }
-  .fini_array : {
-	__fini_array_start = .;
-	*(.fini_array)
-	__fini_array_end = .;
-  }
-
-   . = ALIGN(4096);
-  .init.ramfs : {
-	INIT_RAM_FS
-  }
-
diff --git a/arch/um/kernel/dyn.lds.S b/arch/um/kernel/dyn.lds.S
deleted file mode 100644
index dacbfabf66d8..000000000000
--- a/arch/um/kernel/dyn.lds.S
+++ /dev/null
@@ -1,171 +0,0 @@
-#include <asm/vmlinux.lds.h>
-#include <asm/page.h>
-
-OUTPUT_FORMAT(ELF_FORMAT)
-OUTPUT_ARCH(ELF_ARCH)
-ENTRY(_start)
-jiffies = jiffies_64;
-
-SECTIONS
-{
-  PROVIDE (__executable_start = START);
-  . = START + SIZEOF_HEADERS;
-  .interp         : { *(.interp) }
-  __binary_start = .;
-  . = ALIGN(4096);		/* Init code and data */
-  _text = .;
-  INIT_TEXT_SECTION(PAGE_SIZE)
-
-  . = ALIGN(PAGE_SIZE);
-
-  /* Read-only sections, merged into text segment: */
-  .hash           : { *(.hash) }
-  .gnu.hash       : { *(.gnu.hash) }
-  .dynsym         : { *(.dynsym) }
-  .dynstr         : { *(.dynstr) }
-  .gnu.version    : { *(.gnu.version) }
-  .gnu.version_d  : { *(.gnu.version_d) }
-  .gnu.version_r  : { *(.gnu.version_r) }
-  .rel.init       : { *(.rel.init) }
-  .rela.init      : { *(.rela.init) }
-  .rel.text       : { *(.rel.text .rel.text.* .rel.gnu.linkonce.t.*) }
-  .rela.text      : { *(.rela.text .rela.text.* .rela.gnu.linkonce.t.*) }
-  .rel.fini       : { *(.rel.fini) }
-  .rela.fini      : { *(.rela.fini) }
-  .rel.rodata     : { *(.rel.rodata .rel.rodata.* .rel.gnu.linkonce.r.*) }
-  .rela.rodata    : { *(.rela.rodata .rela.rodata.* .rela.gnu.linkonce.r.*) }
-  .rel.data       : { *(.rel.data .rel.data.* .rel.gnu.linkonce.d.*) }
-  .rela.data      : { *(.rela.data .rela.data.* .rela.gnu.linkonce.d.*) }
-  .rel.tdata	  : { *(.rel.tdata .rel.tdata.* .rel.gnu.linkonce.td.*) }
-  .rela.tdata	  : { *(.rela.tdata .rela.tdata.* .rela.gnu.linkonce.td.*) }
-  .rel.tbss	  : { *(.rel.tbss .rel.tbss.* .rel.gnu.linkonce.tb.*) }
-  .rela.tbss	  : { *(.rela.tbss .rela.tbss.* .rela.gnu.linkonce.tb.*) }
-  .rel.ctors      : { *(.rel.ctors) }
-  .rela.ctors     : { *(.rela.ctors) }
-  .rel.dtors      : { *(.rel.dtors) }
-  .rela.dtors     : { *(.rela.dtors) }
-  .rel.got        : { *(.rel.got) }
-  .rela.got       : { *(.rela.got) }
-  .rel.bss        : { *(.rel.bss .rel.bss.* .rel.gnu.linkonce.b.*) }
-  .rela.bss       : { *(.rela.bss .rela.bss.* .rela.gnu.linkonce.b.*) }
-  .rel.plt : {
-	*(.rel.plt)
-	PROVIDE_HIDDEN(__rel_iplt_start = .);
-	*(.rel.iplt)
-	PROVIDE_HIDDEN(__rel_iplt_end = .);
-  }
-  .rela.plt : {
-	*(.rela.plt)
-	PROVIDE_HIDDEN(__rela_iplt_start = .);
-	*(.rela.iplt)
-	PROVIDE_HIDDEN(__rela_iplt_end = .);
-  }
-  .init           : {
-    KEEP (*(.init))
-  } =0x90909090
-  .plt            : { *(.plt) }
-  .text           : {
-    _stext = .;
-    TEXT_TEXT
-    SCHED_TEXT
-    CPUIDLE_TEXT
-    LOCK_TEXT
-    IRQENTRY_TEXT
-    SOFTIRQENTRY_TEXT
-    *(.fixup)
-    *(.stub .text.* .gnu.linkonce.t.*)
-    /* .gnu.warning sections are handled specially by elf32.em.  */
-    *(.gnu.warning)
-
-    . = ALIGN(PAGE_SIZE);
-  } =0x90909090
-  . = ALIGN(PAGE_SIZE);
-  .syscall_stub : {
-	__syscall_stub_start = .;
-	*(.__syscall_stub*)
-	__syscall_stub_end = .;
-  }
-  .fini           : {
-    KEEP (*(.fini))
-  } =0x90909090
-
-  .kstrtab : { *(.kstrtab) }
-
-  #include <asm/common.lds.S>
-
-  __init_begin = .;
-  init.data : { INIT_DATA }
-  __init_end = .;
-
-  /* Ensure the __preinit_array_start label is properly aligned.  We
-     could instead move the label definition inside the section, but
-     the linker would then create the section even if it turns out to
-     be empty, which isn't pretty.  */
-  . = ALIGN(32 / 8);
-  .preinit_array     : { *(.preinit_array) }
-  .init_array     : { *(.init_array) }
-  .fini_array     : { *(.fini_array) }
-  .data           : {
-    INIT_TASK_DATA(KERNEL_STACK_SIZE)
-    . = ALIGN(KERNEL_STACK_SIZE);
-    *(.data..init_irqstack)
-    DATA_DATA
-    *(.data.* .gnu.linkonce.d.*)
-    SORT(CONSTRUCTORS)
-  }
-  .data1          : { *(.data1) }
-  .tdata	  : { *(.tdata .tdata.* .gnu.linkonce.td.*) }
-  .tbss		  : { *(.tbss .tbss.* .gnu.linkonce.tb.*) *(.tcommon) }
-  .eh_frame       : { KEEP (*(.eh_frame)) }
-  .gcc_except_table   : { *(.gcc_except_table) }
-  .dynamic        : { *(.dynamic) }
-  .ctors          : {
-    /* gcc uses crtbegin.o to find the start of
-       the constructors, so we make sure it is
-       first.  Because this is a wildcard, it
-       doesn't matter if the user does not
-       actually link against crtbegin.o; the
-       linker won't look for a file to match a
-       wildcard.  The wildcard also means that it
-       doesn't matter which directory crtbegin.o
-       is in.  */
-    KEEP (*crtbegin.o(.ctors))
-    /* We don't want to include the .ctor section from
-       from the crtend.o file until after the sorted ctors.
-       The .ctor section from the crtend file contains the
-       end of ctors marker and it must be last */
-    KEEP (*(EXCLUDE_FILE (*crtend.o ) .ctors))
-    KEEP (*(SORT(.ctors.*)))
-    KEEP (*(.ctors))
-  }
-  .dtors          : {
-    KEEP (*crtbegin.o(.dtors))
-    KEEP (*(EXCLUDE_FILE (*crtend.o ) .dtors))
-    KEEP (*(SORT(.dtors.*)))
-    KEEP (*(.dtors))
-  }
-  .jcr            : { KEEP (*(.jcr)) }
-  .got            : { *(.got.plt) *(.got) }
-  _edata = .;
-  PROVIDE (edata = .);
-  .bss            : {
-   __bss_start = .;
-   *(.dynbss)
-   *(.bss .bss.* .gnu.linkonce.b.*)
-   *(COMMON)
-   /* Align here to ensure that the .bss section occupies space up to
-      _end.  Align after .bss to ensure correct alignment even if the
-      .bss section disappears because there are no input sections.  */
-   . = ALIGN(32 / 8);
-  . = ALIGN(32 / 8);
-  }
-   __bss_stop = .;
-  _end = .;
-  PROVIDE (end = .);
-
-  STABS_DEBUG
-  DWARF_DEBUG
-  ELF_DETAILS
-
-  DISCARDS
-}
diff --git a/arch/um/kernel/uml.lds.S b/arch/um/kernel/uml.lds.S
deleted file mode 100644
index 45d957d7004c..000000000000
--- a/arch/um/kernel/uml.lds.S
+++ /dev/null
@@ -1,115 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#include <asm/vmlinux.lds.h>
-#include <asm/page.h>
-
-OUTPUT_FORMAT(ELF_FORMAT)
-OUTPUT_ARCH(ELF_ARCH)
-ENTRY(_start)
-jiffies = jiffies_64;
-
-SECTIONS
-{
-  /* This must contain the right address - not quite the default ELF one.*/
-  PROVIDE (__executable_start = START);
-  /* Static binaries stick stuff here, like the sigreturn trampoline,
-   * invisibly to objdump.  So, just make __binary_start equal to the very
-   * beginning of the executable, and if there are unmapped pages after this,
-   * they are forever unusable.
-   */
-  __binary_start = START;
-
-  . = START + SIZEOF_HEADERS;
-  . = ALIGN(PAGE_SIZE);
-
-  _text = .;
-  INIT_TEXT_SECTION(0)
-
-  .text      :
-  {
-    _stext = .;
-    TEXT_TEXT
-    SCHED_TEXT
-    CPUIDLE_TEXT
-    LOCK_TEXT
-    IRQENTRY_TEXT
-    SOFTIRQENTRY_TEXT
-    *(.fixup)
-    /* .gnu.warning sections are handled specially by elf32.em.  */
-    *(.gnu.warning)
-    *(.gnu.linkonce.t*)
-  }
-
-  . = ALIGN(PAGE_SIZE);
-  .syscall_stub : {
-	__syscall_stub_start = .;
-	*(.__syscall_stub*)
-	__syscall_stub_end = .;
-  }
-
-  /*
-   * These are needed even in a static link, even if they wind up being empty.
-   * Newer glibc needs these __rel{,a}_iplt_{start,end} symbols.
-   */
-  .rel.plt : {
-	*(.rel.plt)
-	PROVIDE_HIDDEN(__rel_iplt_start = .);
-	*(.rel.iplt)
-	PROVIDE_HIDDEN(__rel_iplt_end = .);
-  }
-  .rela.plt : {
-	*(.rela.plt)
-	PROVIDE_HIDDEN(__rela_iplt_start = .);
-	*(.rela.iplt)
-	PROVIDE_HIDDEN(__rela_iplt_end = .);
-  }
-
-  #include <asm/common.lds.S>
-
-  __init_begin = .;
-  init.data : { INIT_DATA }
-  __init_end = .;
-
-  .data    :
-  {
-    INIT_TASK_DATA(KERNEL_STACK_SIZE)
-    . = ALIGN(KERNEL_STACK_SIZE);
-    *(.data..init_irqstack)
-    DATA_DATA
-    *(.gnu.linkonce.d*)
-    CONSTRUCTORS
-  }
-  .data1   : { *(.data1) }
-  .ctors         :
-  {
-    *(.ctors)
-  }
-  .dtors         :
-  {
-    *(.dtors)
-  }
-
-  .got           : { *(.got.plt) *(.got) }
-  .dynamic       : { *(.dynamic) }
-  .tdata	  : { *(.tdata .tdata.* .gnu.linkonce.td.*) }
-  .tbss		  : { *(.tbss .tbss.* .gnu.linkonce.tb.*) *(.tcommon) }
-  /* We want the small data sections together, so single-instruction offsets
-     can access them all, and initialized data all before uninitialized, so
-     we can shorten the on-disk segment size.  */
-  .sdata     : { *(.sdata) }
-  _edata  =  .;
-  PROVIDE (edata = .);
-  . = ALIGN(PAGE_SIZE);
-  __bss_start = .;
-  PROVIDE(_bss_start = .);
-  SBSS(0)
-  BSS(0)
-   __bss_stop = .;
-  _end = .;
-  PROVIDE (end = .);
-
-  STABS_DEBUG
-  DWARF_DEBUG
-  ELF_DETAILS
-
-  DISCARDS
-}
diff --git a/arch/um/kernel/vmlinux.lds.S b/arch/um/kernel/vmlinux.lds.S
index 16e49bfa2b42..a673f51defda 100644
--- a/arch/um/kernel/vmlinux.lds.S
+++ b/arch/um/kernel/vmlinux.lds.S
@@ -1,8 +1,88 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <asm/vmlinux.lds.h>
+#include <asm/thread_info.h>
+#include <asm/page.h>
+#include <asm/thread_info.h>
+#include <asm/cache.h>
+#include <linux/export.h>
 
-KERNEL_STACK_SIZE = 4096 * (1 << CONFIG_KERNEL_STACK_ORDER);
+OUTPUT_FORMAT(ELF_FORMAT)
+jiffies = jiffies_64;
 
-#ifdef CONFIG_LD_SCRIPT_STATIC
-#include "uml.lds.S"
-#else
-#include "dyn.lds.S"
-#endif
+
+SECTIONS
+{
+	__init_begin = . ;
+	HEAD_TEXT_SECTION
+	INIT_TEXT_SECTION(PAGE_SIZE)
+	INIT_DATA_SECTION(16)
+	PERCPU_SECTION(L1_CACHE_BYTES)
+	__init_end = . ;
+
+	_stext = . ;
+	_text = . ;
+	.text : ALIGN(THREAD_SIZE)
+	{
+		__binary_start = . ;
+		TEXT_TEXT
+		SCHED_TEXT
+		LOCK_TEXT
+		CPUIDLE_TEXT
+		IRQENTRY_TEXT
+		SOFTIRQENTRY_TEXT
+	}
+	_etext = . ;
+
+	.syscall_stub : ALIGN(PAGE_SIZE)
+	{
+		__syscall_stub_start = .;
+		*(.__syscall_stub*)
+		__syscall_stub_end = .;
+	}
+
+	_sdata = . ;
+	RO_DATA(PAGE_SIZE)
+	RW_DATA(L1_CACHE_BYTES, PAGE_SIZE, THREAD_SIZE)
+	.data : ALIGN(THREAD_SIZE)  { }
+
+	EXCEPTION_TABLE(16)
+
+	.uml.init_irqstack : ALIGN(THREAD_SIZE) {
+		*(.data..init_irqstack)
+	}
+
+	.uml.setup.init : ALIGN(8) {
+		__uml_setup_start = .;
+		*(.uml.setup.init)
+		__uml_setup_end = .;
+	}
+
+	.uml.help.init : ALIGN(8)  {
+		__uml_help_start = .;
+		*(.uml.help.init)
+		__uml_help_end = .;
+	}
+
+	.uml.postsetup.init : ALIGN(8) {
+		__uml_postsetup_start = .;
+		*(.uml.postsetup.init)
+		__uml_postsetup_end = .;
+	}
+
+	.uml.exitcall : ALIGN(8) {
+		__uml_exitcall_begin = .;
+		*(.uml.exitcall.exit)
+		__uml_exitcall_end = .;
+	}
+	_edata = . ;
+
+	BSS_SECTION(0, 0, 0)
+	_end = . ;
+
+	STABS_DEBUG
+
+	DWARF_DEBUG
+	ELF_DETAILS
+
+	DISCARDS
+}
diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
index 6eded325c837..9bc0e212657c 100755
--- a/scripts/link-vmlinux.sh
+++ b/scripts/link-vmlinux.sh
@@ -102,36 +102,18 @@ vmlinux_link()
 		strip_debug=-Wl,--strip-debug
 	fi
 
-	if [ "${SRCARCH}" != "um" ]; then
-		objects="--whole-archive			\
-			${KBUILD_VMLINUX_OBJS}			\
-			--no-whole-archive			\
-			--start-group				\
-			${KBUILD_VMLINUX_LIBS}			\
-			--end-group				\
-			${@}"
-
-		${LD} ${KBUILD_LDFLAGS} ${LDFLAGS_vmlinux}	\
-			${strip_debug#-Wl,}			\
-			-o ${output}				\
-			-T ${lds} ${objects}
-	else
-		objects="-Wl,--whole-archive			\
-			${KBUILD_VMLINUX_OBJS}			\
-			-Wl,--no-whole-archive			\
-			-Wl,--start-group			\
-			${KBUILD_VMLINUX_LIBS}			\
-			-Wl,--end-group				\
-			${@}"
-
-		${CC} ${CFLAGS_vmlinux}				\
-			${strip_debug}				\
-			-o ${output}				\
-			-Wl,-T,${lds}				\
-			${objects}				\
-			-lutil -lrt -lpthread
-		rm -f linux
-	fi
+	objects="--whole-archive			\
+		${KBUILD_VMLINUX_OBJS}			\
+		--no-whole-archive			\
+		--start-group				\
+		${KBUILD_VMLINUX_LIBS}			\
+		--end-group				\
+		${@}"
+
+	${LD} ${KBUILD_LDFLAGS} ${LDFLAGS_vmlinux}	\
+		${strip_debug#-Wl,}			\
+		-o ${output}				\
+		-T ${lds} ${objects}
 }
 
 # generate .BTF typeinfo from DWARF debuginfo
diff --git a/tools/um/Makefile b/tools/um/Makefile
new file mode 100644
index 000000000000..2b4e6b63355b
--- /dev/null
+++ b/tools/um/Makefile
@@ -0,0 +1,74 @@
+# Do not use make's built-in rules
+# (this improves performance and avoids hard-to-debug behaviour);
+# also do not print "Entering directory..." messages from make
+.SUFFIXES:
+MAKEFLAGS += -r --no-print-directory
+
+ifneq ($(silent),1)
+  ifneq ($(V),1)
+	QUIET_AUTOCONF       = @echo '  AUTOCONF '$@;
+	Q = @
+  endif
+endif
+
+PREFIX   := /usr
+
+srctree := $(patsubst %/,%,$(dir $(shell pwd)))
+srctree := $(patsubst %/,%,$(dir $(srctree)))
+export srctree
+
+-include ../scripts/Makefile.include
+
+# OUTPUT fixup should be *after* include ../scripts/Makefile.include
+ifneq ($(OUTPUT),)
+  OUTPUT := $(OUTPUT)/tools/um/
+else
+  OUTPUT := $(CURDIR)/
+endif
+export OUTPUT
+export objtree := $(OUTPUT)/../..
+
+export CFLAGS += -I$(OUTPUT)/include -Iinclude -Wall -g -O2 -Wextra -fPIC \
+	 -Wno-unused-parameter \
+	 -Wno-missing-field-initializers -fno-strict-aliasing
+
+-include Targets
+
+TARGETS := $(progs-y:%=$(OUTPUT)%)
+TARGETS += $(libs-y:%=$(OUTPUT)%)
+all: $(TARGETS)
+
+# rule to build linux.o
+$(objtree)/linux.o:
+	$(Q)echo ""
+	$(Q)echo ""
+	$(Q)echo "==> $@ isn't found; please make ARCH=um"
+	$(Q)echo ""
+	$(Q)echo ""
+	$(Q)exit 1
+
+$(OUTPUT)lib/linux.o: $(objtree)/linux.o
+	$(Q)cp $(objtree)/linux.o $(OUTPUT)lib/linux.o
+
+$(OUTPUT)liblinux.a: $(OUTPUT)lib/linux.o $(OUTPUT)uml/liblinux-in.o
+	$(QUIET_AR)$(AR) -rc $@ $^
+
+# rule to link programs
+$(OUTPUT)%: $(OUTPUT)%-in.o $(OUTPUT)liblinux.a
+	$(QUIET_LINK)$(CC) $(LDFLAGS) $(LDFLAGS_$(notdir $*)-y) -o $@ $^ $(LDLIBS) $(LDLIBS_$(notdir $*)-y)
+
+# rule to build objects
+$(OUTPUT)%-in.o: FORCE
+	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
+
+RM := rm -f
+clean:
+	$(call QUIET_CLEAN, objects)find $(OUTPUT) -name '*.o' -delete -o -name '\.*.cmd'\
+	 -delete -o -name '\.*.d' -delete
+	$(call QUIET_CLEAN, liblinux.a)$(RM) $(OUTPUT)/liblinux.a
+	$(call QUIET_CLEAN, $(TARGETS))$(RM) $(TARGETS)
+
+FORCE: ;
+.PHONY: all clean FORCE
+.IGNORE: clean
+.NOTPARALLEL : lib/linux.o
diff --git a/tools/um/Targets b/tools/um/Targets
new file mode 100644
index 000000000000..cfe1d3c3c6ff
--- /dev/null
+++ b/tools/um/Targets
@@ -0,0 +1,4 @@
+progs-y += uml/linux
+LDLIBS_linux-y := -lrt -lpthread -lutil
+LDFLAGS_linux-y := -no-pie -Wl,--wrap,malloc -Wl,--wrap,free -Wl,--wrap,calloc
+LDFLAGS_linux-$(CONFIG_STATIC_LINK) += -static
diff --git a/tools/um/uml/Build b/tools/um/uml/Build
new file mode 100644
index 000000000000..e69de29bb2d1
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v8 01/20] um: split build in kernel and host parts
@ 2021-01-20  2:27             ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-01-20  2:27 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: linux-arch, Michal Marek, linux-kbuild, tavi.purdila,
	Masahiro Yamada, Octavian Purdila, retrage01,
	linux-kernel-library, thehajime

From: Octavian Purdila <tavi@cs.pub.ro>

This patch splits the UML build in two parts: a kernel part that
generates a relocatable object (linux.o) and a host build part that
links the relocatable object into an executable.

This allows us to simplify the linker script since we can now remove
the host bits (e.g. .rel sections). It also gives us greater
flexibility since it will be much easier to support other
architectures and OSes if we use the standard linker script.

The host build part has been implemented in tools/um so that we can
reuse the available host build infrastructure.

The patch preserves the way to invoke the UML build as follows,

 $ make ARCH=um defconfig
 $ make ARCH=um

but it internally calls tools/um/Makefile in order to complete the full
build to link the relocatable object and the rest of files.

This commit also relaxes the condition of UML in scripts/link-vmlinux.sh
since we can postpone the link process with host libraries (-lpthread,
etc) to the 2nd starge.

Cc: Masahiro Yamada <masahiroy@kernel.org>
Cc: Michal Marek <michal.lkml@markovi.net>
Cc: linux-kbuild@vger.kernel.org
Signed-off-by: Octavian Purdila <tavi@cs.pub.ro>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/Kconfig                  |  12 +--
 arch/um/Makefile                 |  29 ++++--
 arch/um/include/asm/common.lds.S |  99 ------------------
 arch/um/kernel/dyn.lds.S         | 171 -------------------------------
 arch/um/kernel/uml.lds.S         | 115 ---------------------
 arch/um/kernel/vmlinux.lds.S     |  92 +++++++++++++++--
 scripts/link-vmlinux.sh          |  42 +++-----
 tools/um/Makefile                |  74 +++++++++++++
 tools/um/Targets                 |   4 +
 tools/um/uml/Build               |   0
 10 files changed, 200 insertions(+), 438 deletions(-)
 delete mode 100644 arch/um/include/asm/common.lds.S
 delete mode 100644 arch/um/kernel/dyn.lds.S
 delete mode 100644 arch/um/kernel/uml.lds.S
 create mode 100644 tools/um/Makefile
 create mode 100644 tools/um/Targets
 create mode 100644 tools/um/uml/Build

diff --git a/arch/um/Kconfig b/arch/um/Kconfig
index 34d302d1a07f..9e2d9efbcdd1 100644
--- a/arch/um/Kconfig
+++ b/arch/um/Kconfig
@@ -21,6 +21,7 @@ config UML
 	select HAVE_GCC_PLUGINS
 	select SET_FS
 	select TTY # Needed for line.c
+	select MODULE_REL_CRCS if MODVERSIONS
 
 config MMU
 	bool
@@ -80,17 +81,6 @@ config STATIC_LINK
 	  NOTE: This option is incompatible with some networking features which
 	  depend on features that require being dynamically loaded (like NSS).
 
-config LD_SCRIPT_STATIC
-	bool
-	default y
-	depends on STATIC_LINK
-
-config LD_SCRIPT_DYN
-	bool
-	default y
-	depends on !LD_SCRIPT_STATIC
-	select MODULE_REL_CRCS if MODVERSIONS
-
 config HOSTFS
 	tristate "Host filesystem"
 	help
diff --git a/arch/um/Makefile b/arch/um/Makefile
index 1cea46ff9bb7..48fbbc3a0032 100644
--- a/arch/um/Makefile
+++ b/arch/um/Makefile
@@ -95,14 +95,30 @@ KBUILD_CPPFLAGS += -I$(srctree)/$(HOST_DIR)/include \
 KERNEL_DEFINES = $(strip -Derrno=kernel_errno -Dsigprocmask=kernel_sigprocmask \
 			 -Dmktime=kernel_mktime $(ARCH_KERNEL_DEFINES))
 KBUILD_CFLAGS += $(KERNEL_DEFINES)
+LDFLAGS_vmlinux += -r
 
-PHONY += linux
-
+INSTALL_PATH=$(objtree)/tools/um
 all: linux
-
-linux: vmlinux
-	@echo '  LINK $@'
-	$(Q)ln -f $< $@
+pre-2nd: linux.o
+	@echo "  INSTALL $(INSTALL_PATH)/lib/$<"
+	@mkdir -p $(INSTALL_PATH)/lib/
+	@cp $< $(INSTALL_PATH)/lib/
+
+PHONY += linux.o
+
+linux: pre-2nd
+	$(Q)$(MAKE) -C $(srctree)/tools/um
+	$(Q)cp $(objtree)/tools/um/uml/linux $(objtree)/
+	$(Q)cp $(objtree)/linux $(objtree)/vmlinux
+
+linux.o: vmlinux
+#revert vmlinux file from vmlinux.a
+ifneq ("$(wildcard vmlinux.a)","")
+	$(Q)cp vmlinux.a vmlinux
+endif
+	$(Q)cp vmlinux vmlinux.a
+	@echo "  LINK    $@"
+	$(Q)$(OBJCOPY) -R .eh_frame $< $@
 
 define archhelp
   echo '* linux		- Binary kernel image (./linux) - for backward'
@@ -144,5 +160,6 @@ MRPROPER_FILES += arch/$(SUBARCH)/include/generated
 archclean:
 	@find . \( -name '*.bb' -o -name '*.bbg' -o -name '*.da' \
 		-o -name '*.gcov' \) -type f -print | xargs rm -f
+	$(Q)$(MAKE) -C $(srctree)/tools/um clean
 
 export HEADER_ARCH SUBARCH USER_CFLAGS CFLAGS_NO_HARDENING OS DEV_NULL_PATH
diff --git a/arch/um/include/asm/common.lds.S b/arch/um/include/asm/common.lds.S
deleted file mode 100644
index eca6c452a41b..000000000000
--- a/arch/um/include/asm/common.lds.S
+++ /dev/null
@@ -1,99 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#include <asm-generic/vmlinux.lds.h>
-
-  .fini      : { *(.fini)    } =0x9090
-  _etext = .;
-  PROVIDE (etext = .);
-
-  . = ALIGN(4096);
-  _sdata = .;
-  PROVIDE (sdata = .);
-
-  RO_DATA(4096)
-
-  .unprotected : { *(.unprotected) }
-  . = ALIGN(4096);
-  PROVIDE (_unprotected_end = .);
-
-  . = ALIGN(4096);
-  EXCEPTION_TABLE(0)
-
-  BUG_TABLE
-
-  .uml.setup.init : {
-	__uml_setup_start = .;
-	*(.uml.setup.init)
-	__uml_setup_end = .;
-  }
-	
-  .uml.help.init : {
-	__uml_help_start = .;
-	*(.uml.help.init)
-	__uml_help_end = .;
-  }
-	
-  .uml.postsetup.init : {
-	__uml_postsetup_start = .;
-	*(.uml.postsetup.init)
-	__uml_postsetup_end = .;
-  }
-	
-  .init.setup : {
-	INIT_SETUP(0)
-  }
-
-  PERCPU_SECTION(32)
-	
-  .initcall.init : {
-	INIT_CALLS
-  }
-
-  .con_initcall.init : {
-	CON_INITCALL
-  }
-
-  .exitcall : {
-	__exitcall_begin = .;
-	*(.exitcall.exit)
-	__exitcall_end = .;
-  }
-
-  .uml.exitcall : {
-	__uml_exitcall_begin = .;
-	*(.uml.exitcall.exit)
-	__uml_exitcall_end = .;
-  }
-
-  . = ALIGN(4);
-  .altinstructions : {
-	__alt_instructions = .;
-	*(.altinstructions)
-	__alt_instructions_end = .;
-  }
-  .altinstr_replacement : { *(.altinstr_replacement) }
-  /* .exit.text is discard at runtime, not link time, to deal with references
-     from .altinstructions and .eh_frame */
-  .exit.text : { EXIT_TEXT }
-  .exit.data : { *(.exit.data) }
-
-  .preinit_array : {
-	__preinit_array_start = .;
-	*(.preinit_array)
-	__preinit_array_end = .;
-  }
-  .init_array : {
-	__init_array_start = .;
-	*(.init_array)
-	__init_array_end = .;
-  }
-  .fini_array : {
-	__fini_array_start = .;
-	*(.fini_array)
-	__fini_array_end = .;
-  }
-
-   . = ALIGN(4096);
-  .init.ramfs : {
-	INIT_RAM_FS
-  }
-
diff --git a/arch/um/kernel/dyn.lds.S b/arch/um/kernel/dyn.lds.S
deleted file mode 100644
index dacbfabf66d8..000000000000
--- a/arch/um/kernel/dyn.lds.S
+++ /dev/null
@@ -1,171 +0,0 @@
-#include <asm/vmlinux.lds.h>
-#include <asm/page.h>
-
-OUTPUT_FORMAT(ELF_FORMAT)
-OUTPUT_ARCH(ELF_ARCH)
-ENTRY(_start)
-jiffies = jiffies_64;
-
-SECTIONS
-{
-  PROVIDE (__executable_start = START);
-  . = START + SIZEOF_HEADERS;
-  .interp         : { *(.interp) }
-  __binary_start = .;
-  . = ALIGN(4096);		/* Init code and data */
-  _text = .;
-  INIT_TEXT_SECTION(PAGE_SIZE)
-
-  . = ALIGN(PAGE_SIZE);
-
-  /* Read-only sections, merged into text segment: */
-  .hash           : { *(.hash) }
-  .gnu.hash       : { *(.gnu.hash) }
-  .dynsym         : { *(.dynsym) }
-  .dynstr         : { *(.dynstr) }
-  .gnu.version    : { *(.gnu.version) }
-  .gnu.version_d  : { *(.gnu.version_d) }
-  .gnu.version_r  : { *(.gnu.version_r) }
-  .rel.init       : { *(.rel.init) }
-  .rela.init      : { *(.rela.init) }
-  .rel.text       : { *(.rel.text .rel.text.* .rel.gnu.linkonce.t.*) }
-  .rela.text      : { *(.rela.text .rela.text.* .rela.gnu.linkonce.t.*) }
-  .rel.fini       : { *(.rel.fini) }
-  .rela.fini      : { *(.rela.fini) }
-  .rel.rodata     : { *(.rel.rodata .rel.rodata.* .rel.gnu.linkonce.r.*) }
-  .rela.rodata    : { *(.rela.rodata .rela.rodata.* .rela.gnu.linkonce.r.*) }
-  .rel.data       : { *(.rel.data .rel.data.* .rel.gnu.linkonce.d.*) }
-  .rela.data      : { *(.rela.data .rela.data.* .rela.gnu.linkonce.d.*) }
-  .rel.tdata	  : { *(.rel.tdata .rel.tdata.* .rel.gnu.linkonce.td.*) }
-  .rela.tdata	  : { *(.rela.tdata .rela.tdata.* .rela.gnu.linkonce.td.*) }
-  .rel.tbss	  : { *(.rel.tbss .rel.tbss.* .rel.gnu.linkonce.tb.*) }
-  .rela.tbss	  : { *(.rela.tbss .rela.tbss.* .rela.gnu.linkonce.tb.*) }
-  .rel.ctors      : { *(.rel.ctors) }
-  .rela.ctors     : { *(.rela.ctors) }
-  .rel.dtors      : { *(.rel.dtors) }
-  .rela.dtors     : { *(.rela.dtors) }
-  .rel.got        : { *(.rel.got) }
-  .rela.got       : { *(.rela.got) }
-  .rel.bss        : { *(.rel.bss .rel.bss.* .rel.gnu.linkonce.b.*) }
-  .rela.bss       : { *(.rela.bss .rela.bss.* .rela.gnu.linkonce.b.*) }
-  .rel.plt : {
-	*(.rel.plt)
-	PROVIDE_HIDDEN(__rel_iplt_start = .);
-	*(.rel.iplt)
-	PROVIDE_HIDDEN(__rel_iplt_end = .);
-  }
-  .rela.plt : {
-	*(.rela.plt)
-	PROVIDE_HIDDEN(__rela_iplt_start = .);
-	*(.rela.iplt)
-	PROVIDE_HIDDEN(__rela_iplt_end = .);
-  }
-  .init           : {
-    KEEP (*(.init))
-  } =0x90909090
-  .plt            : { *(.plt) }
-  .text           : {
-    _stext = .;
-    TEXT_TEXT
-    SCHED_TEXT
-    CPUIDLE_TEXT
-    LOCK_TEXT
-    IRQENTRY_TEXT
-    SOFTIRQENTRY_TEXT
-    *(.fixup)
-    *(.stub .text.* .gnu.linkonce.t.*)
-    /* .gnu.warning sections are handled specially by elf32.em.  */
-    *(.gnu.warning)
-
-    . = ALIGN(PAGE_SIZE);
-  } =0x90909090
-  . = ALIGN(PAGE_SIZE);
-  .syscall_stub : {
-	__syscall_stub_start = .;
-	*(.__syscall_stub*)
-	__syscall_stub_end = .;
-  }
-  .fini           : {
-    KEEP (*(.fini))
-  } =0x90909090
-
-  .kstrtab : { *(.kstrtab) }
-
-  #include <asm/common.lds.S>
-
-  __init_begin = .;
-  init.data : { INIT_DATA }
-  __init_end = .;
-
-  /* Ensure the __preinit_array_start label is properly aligned.  We
-     could instead move the label definition inside the section, but
-     the linker would then create the section even if it turns out to
-     be empty, which isn't pretty.  */
-  . = ALIGN(32 / 8);
-  .preinit_array     : { *(.preinit_array) }
-  .init_array     : { *(.init_array) }
-  .fini_array     : { *(.fini_array) }
-  .data           : {
-    INIT_TASK_DATA(KERNEL_STACK_SIZE)
-    . = ALIGN(KERNEL_STACK_SIZE);
-    *(.data..init_irqstack)
-    DATA_DATA
-    *(.data.* .gnu.linkonce.d.*)
-    SORT(CONSTRUCTORS)
-  }
-  .data1          : { *(.data1) }
-  .tdata	  : { *(.tdata .tdata.* .gnu.linkonce.td.*) }
-  .tbss		  : { *(.tbss .tbss.* .gnu.linkonce.tb.*) *(.tcommon) }
-  .eh_frame       : { KEEP (*(.eh_frame)) }
-  .gcc_except_table   : { *(.gcc_except_table) }
-  .dynamic        : { *(.dynamic) }
-  .ctors          : {
-    /* gcc uses crtbegin.o to find the start of
-       the constructors, so we make sure it is
-       first.  Because this is a wildcard, it
-       doesn't matter if the user does not
-       actually link against crtbegin.o; the
-       linker won't look for a file to match a
-       wildcard.  The wildcard also means that it
-       doesn't matter which directory crtbegin.o
-       is in.  */
-    KEEP (*crtbegin.o(.ctors))
-    /* We don't want to include the .ctor section from
-       from the crtend.o file until after the sorted ctors.
-       The .ctor section from the crtend file contains the
-       end of ctors marker and it must be last */
-    KEEP (*(EXCLUDE_FILE (*crtend.o ) .ctors))
-    KEEP (*(SORT(.ctors.*)))
-    KEEP (*(.ctors))
-  }
-  .dtors          : {
-    KEEP (*crtbegin.o(.dtors))
-    KEEP (*(EXCLUDE_FILE (*crtend.o ) .dtors))
-    KEEP (*(SORT(.dtors.*)))
-    KEEP (*(.dtors))
-  }
-  .jcr            : { KEEP (*(.jcr)) }
-  .got            : { *(.got.plt) *(.got) }
-  _edata = .;
-  PROVIDE (edata = .);
-  .bss            : {
-   __bss_start = .;
-   *(.dynbss)
-   *(.bss .bss.* .gnu.linkonce.b.*)
-   *(COMMON)
-   /* Align here to ensure that the .bss section occupies space up to
-      _end.  Align after .bss to ensure correct alignment even if the
-      .bss section disappears because there are no input sections.  */
-   . = ALIGN(32 / 8);
-  . = ALIGN(32 / 8);
-  }
-   __bss_stop = .;
-  _end = .;
-  PROVIDE (end = .);
-
-  STABS_DEBUG
-  DWARF_DEBUG
-  ELF_DETAILS
-
-  DISCARDS
-}
diff --git a/arch/um/kernel/uml.lds.S b/arch/um/kernel/uml.lds.S
deleted file mode 100644
index 45d957d7004c..000000000000
--- a/arch/um/kernel/uml.lds.S
+++ /dev/null
@@ -1,115 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#include <asm/vmlinux.lds.h>
-#include <asm/page.h>
-
-OUTPUT_FORMAT(ELF_FORMAT)
-OUTPUT_ARCH(ELF_ARCH)
-ENTRY(_start)
-jiffies = jiffies_64;
-
-SECTIONS
-{
-  /* This must contain the right address - not quite the default ELF one.*/
-  PROVIDE (__executable_start = START);
-  /* Static binaries stick stuff here, like the sigreturn trampoline,
-   * invisibly to objdump.  So, just make __binary_start equal to the very
-   * beginning of the executable, and if there are unmapped pages after this,
-   * they are forever unusable.
-   */
-  __binary_start = START;
-
-  . = START + SIZEOF_HEADERS;
-  . = ALIGN(PAGE_SIZE);
-
-  _text = .;
-  INIT_TEXT_SECTION(0)
-
-  .text      :
-  {
-    _stext = .;
-    TEXT_TEXT
-    SCHED_TEXT
-    CPUIDLE_TEXT
-    LOCK_TEXT
-    IRQENTRY_TEXT
-    SOFTIRQENTRY_TEXT
-    *(.fixup)
-    /* .gnu.warning sections are handled specially by elf32.em.  */
-    *(.gnu.warning)
-    *(.gnu.linkonce.t*)
-  }
-
-  . = ALIGN(PAGE_SIZE);
-  .syscall_stub : {
-	__syscall_stub_start = .;
-	*(.__syscall_stub*)
-	__syscall_stub_end = .;
-  }
-
-  /*
-   * These are needed even in a static link, even if they wind up being empty.
-   * Newer glibc needs these __rel{,a}_iplt_{start,end} symbols.
-   */
-  .rel.plt : {
-	*(.rel.plt)
-	PROVIDE_HIDDEN(__rel_iplt_start = .);
-	*(.rel.iplt)
-	PROVIDE_HIDDEN(__rel_iplt_end = .);
-  }
-  .rela.plt : {
-	*(.rela.plt)
-	PROVIDE_HIDDEN(__rela_iplt_start = .);
-	*(.rela.iplt)
-	PROVIDE_HIDDEN(__rela_iplt_end = .);
-  }
-
-  #include <asm/common.lds.S>
-
-  __init_begin = .;
-  init.data : { INIT_DATA }
-  __init_end = .;
-
-  .data    :
-  {
-    INIT_TASK_DATA(KERNEL_STACK_SIZE)
-    . = ALIGN(KERNEL_STACK_SIZE);
-    *(.data..init_irqstack)
-    DATA_DATA
-    *(.gnu.linkonce.d*)
-    CONSTRUCTORS
-  }
-  .data1   : { *(.data1) }
-  .ctors         :
-  {
-    *(.ctors)
-  }
-  .dtors         :
-  {
-    *(.dtors)
-  }
-
-  .got           : { *(.got.plt) *(.got) }
-  .dynamic       : { *(.dynamic) }
-  .tdata	  : { *(.tdata .tdata.* .gnu.linkonce.td.*) }
-  .tbss		  : { *(.tbss .tbss.* .gnu.linkonce.tb.*) *(.tcommon) }
-  /* We want the small data sections together, so single-instruction offsets
-     can access them all, and initialized data all before uninitialized, so
-     we can shorten the on-disk segment size.  */
-  .sdata     : { *(.sdata) }
-  _edata  =  .;
-  PROVIDE (edata = .);
-  . = ALIGN(PAGE_SIZE);
-  __bss_start = .;
-  PROVIDE(_bss_start = .);
-  SBSS(0)
-  BSS(0)
-   __bss_stop = .;
-  _end = .;
-  PROVIDE (end = .);
-
-  STABS_DEBUG
-  DWARF_DEBUG
-  ELF_DETAILS
-
-  DISCARDS
-}
diff --git a/arch/um/kernel/vmlinux.lds.S b/arch/um/kernel/vmlinux.lds.S
index 16e49bfa2b42..a673f51defda 100644
--- a/arch/um/kernel/vmlinux.lds.S
+++ b/arch/um/kernel/vmlinux.lds.S
@@ -1,8 +1,88 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <asm/vmlinux.lds.h>
+#include <asm/thread_info.h>
+#include <asm/page.h>
+#include <asm/thread_info.h>
+#include <asm/cache.h>
+#include <linux/export.h>
 
-KERNEL_STACK_SIZE = 4096 * (1 << CONFIG_KERNEL_STACK_ORDER);
+OUTPUT_FORMAT(ELF_FORMAT)
+jiffies = jiffies_64;
 
-#ifdef CONFIG_LD_SCRIPT_STATIC
-#include "uml.lds.S"
-#else
-#include "dyn.lds.S"
-#endif
+
+SECTIONS
+{
+	__init_begin = . ;
+	HEAD_TEXT_SECTION
+	INIT_TEXT_SECTION(PAGE_SIZE)
+	INIT_DATA_SECTION(16)
+	PERCPU_SECTION(L1_CACHE_BYTES)
+	__init_end = . ;
+
+	_stext = . ;
+	_text = . ;
+	.text : ALIGN(THREAD_SIZE)
+	{
+		__binary_start = . ;
+		TEXT_TEXT
+		SCHED_TEXT
+		LOCK_TEXT
+		CPUIDLE_TEXT
+		IRQENTRY_TEXT
+		SOFTIRQENTRY_TEXT
+	}
+	_etext = . ;
+
+	.syscall_stub : ALIGN(PAGE_SIZE)
+	{
+		__syscall_stub_start = .;
+		*(.__syscall_stub*)
+		__syscall_stub_end = .;
+	}
+
+	_sdata = . ;
+	RO_DATA(PAGE_SIZE)
+	RW_DATA(L1_CACHE_BYTES, PAGE_SIZE, THREAD_SIZE)
+	.data : ALIGN(THREAD_SIZE)  { }
+
+	EXCEPTION_TABLE(16)
+
+	.uml.init_irqstack : ALIGN(THREAD_SIZE) {
+		*(.data..init_irqstack)
+	}
+
+	.uml.setup.init : ALIGN(8) {
+		__uml_setup_start = .;
+		*(.uml.setup.init)
+		__uml_setup_end = .;
+	}
+
+	.uml.help.init : ALIGN(8)  {
+		__uml_help_start = .;
+		*(.uml.help.init)
+		__uml_help_end = .;
+	}
+
+	.uml.postsetup.init : ALIGN(8) {
+		__uml_postsetup_start = .;
+		*(.uml.postsetup.init)
+		__uml_postsetup_end = .;
+	}
+
+	.uml.exitcall : ALIGN(8) {
+		__uml_exitcall_begin = .;
+		*(.uml.exitcall.exit)
+		__uml_exitcall_end = .;
+	}
+	_edata = . ;
+
+	BSS_SECTION(0, 0, 0)
+	_end = . ;
+
+	STABS_DEBUG
+
+	DWARF_DEBUG
+	ELF_DETAILS
+
+	DISCARDS
+}
diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
index 6eded325c837..9bc0e212657c 100755
--- a/scripts/link-vmlinux.sh
+++ b/scripts/link-vmlinux.sh
@@ -102,36 +102,18 @@ vmlinux_link()
 		strip_debug=-Wl,--strip-debug
 	fi
 
-	if [ "${SRCARCH}" != "um" ]; then
-		objects="--whole-archive			\
-			${KBUILD_VMLINUX_OBJS}			\
-			--no-whole-archive			\
-			--start-group				\
-			${KBUILD_VMLINUX_LIBS}			\
-			--end-group				\
-			${@}"
-
-		${LD} ${KBUILD_LDFLAGS} ${LDFLAGS_vmlinux}	\
-			${strip_debug#-Wl,}			\
-			-o ${output}				\
-			-T ${lds} ${objects}
-	else
-		objects="-Wl,--whole-archive			\
-			${KBUILD_VMLINUX_OBJS}			\
-			-Wl,--no-whole-archive			\
-			-Wl,--start-group			\
-			${KBUILD_VMLINUX_LIBS}			\
-			-Wl,--end-group				\
-			${@}"
-
-		${CC} ${CFLAGS_vmlinux}				\
-			${strip_debug}				\
-			-o ${output}				\
-			-Wl,-T,${lds}				\
-			${objects}				\
-			-lutil -lrt -lpthread
-		rm -f linux
-	fi
+	objects="--whole-archive			\
+		${KBUILD_VMLINUX_OBJS}			\
+		--no-whole-archive			\
+		--start-group				\
+		${KBUILD_VMLINUX_LIBS}			\
+		--end-group				\
+		${@}"
+
+	${LD} ${KBUILD_LDFLAGS} ${LDFLAGS_vmlinux}	\
+		${strip_debug#-Wl,}			\
+		-o ${output}				\
+		-T ${lds} ${objects}
 }
 
 # generate .BTF typeinfo from DWARF debuginfo
diff --git a/tools/um/Makefile b/tools/um/Makefile
new file mode 100644
index 000000000000..2b4e6b63355b
--- /dev/null
+++ b/tools/um/Makefile
@@ -0,0 +1,74 @@
+# Do not use make's built-in rules
+# (this improves performance and avoids hard-to-debug behaviour);
+# also do not print "Entering directory..." messages from make
+.SUFFIXES:
+MAKEFLAGS += -r --no-print-directory
+
+ifneq ($(silent),1)
+  ifneq ($(V),1)
+	QUIET_AUTOCONF       = @echo '  AUTOCONF '$@;
+	Q = @
+  endif
+endif
+
+PREFIX   := /usr
+
+srctree := $(patsubst %/,%,$(dir $(shell pwd)))
+srctree := $(patsubst %/,%,$(dir $(srctree)))
+export srctree
+
+-include ../scripts/Makefile.include
+
+# OUTPUT fixup should be *after* include ../scripts/Makefile.include
+ifneq ($(OUTPUT),)
+  OUTPUT := $(OUTPUT)/tools/um/
+else
+  OUTPUT := $(CURDIR)/
+endif
+export OUTPUT
+export objtree := $(OUTPUT)/../..
+
+export CFLAGS += -I$(OUTPUT)/include -Iinclude -Wall -g -O2 -Wextra -fPIC \
+	 -Wno-unused-parameter \
+	 -Wno-missing-field-initializers -fno-strict-aliasing
+
+-include Targets
+
+TARGETS := $(progs-y:%=$(OUTPUT)%)
+TARGETS += $(libs-y:%=$(OUTPUT)%)
+all: $(TARGETS)
+
+# rule to build linux.o
+$(objtree)/linux.o:
+	$(Q)echo ""
+	$(Q)echo ""
+	$(Q)echo "==> $@ isn't found; please make ARCH=um"
+	$(Q)echo ""
+	$(Q)echo ""
+	$(Q)exit 1
+
+$(OUTPUT)lib/linux.o: $(objtree)/linux.o
+	$(Q)cp $(objtree)/linux.o $(OUTPUT)lib/linux.o
+
+$(OUTPUT)liblinux.a: $(OUTPUT)lib/linux.o $(OUTPUT)uml/liblinux-in.o
+	$(QUIET_AR)$(AR) -rc $@ $^
+
+# rule to link programs
+$(OUTPUT)%: $(OUTPUT)%-in.o $(OUTPUT)liblinux.a
+	$(QUIET_LINK)$(CC) $(LDFLAGS) $(LDFLAGS_$(notdir $*)-y) -o $@ $^ $(LDLIBS) $(LDLIBS_$(notdir $*)-y)
+
+# rule to build objects
+$(OUTPUT)%-in.o: FORCE
+	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
+
+RM := rm -f
+clean:
+	$(call QUIET_CLEAN, objects)find $(OUTPUT) -name '*.o' -delete -o -name '\.*.cmd'\
+	 -delete -o -name '\.*.d' -delete
+	$(call QUIET_CLEAN, liblinux.a)$(RM) $(OUTPUT)/liblinux.a
+	$(call QUIET_CLEAN, $(TARGETS))$(RM) $(TARGETS)
+
+FORCE: ;
+.PHONY: all clean FORCE
+.IGNORE: clean
+.NOTPARALLEL : lib/linux.o
diff --git a/tools/um/Targets b/tools/um/Targets
new file mode 100644
index 000000000000..cfe1d3c3c6ff
--- /dev/null
+++ b/tools/um/Targets
@@ -0,0 +1,4 @@
+progs-y += uml/linux
+LDLIBS_linux-y := -lrt -lpthread -lutil
+LDFLAGS_linux-y := -no-pie -Wl,--wrap,malloc -Wl,--wrap,free -Wl,--wrap,calloc
+LDFLAGS_linux-$(CONFIG_STATIC_LINK) += -static
diff --git a/tools/um/uml/Build b/tools/um/uml/Build
new file mode 100644
index 000000000000..e69de29bb2d1
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v8 02/20] um: move arch/um/os-Linux dir to tools/um/uml
  2021-01-20  2:27           ` Hajime Tazaki
@ 2021-01-20  2:27             ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-01-20  2:27 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: thehajime, tavi.purdila, retrage01, linux-kernel-library, linux-arch

This patch moves underlying OS dependent part under arch/um to tools/um
directory so that arch/um code does not need to build host build
facilities (e.g., libc).

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/Makefile                              |  3 +-
 arch/um/drivers/Makefile                      |  2 +
 .../um/{os-Linux => }/drivers/ethertap_kern.c |  0
 arch/um/{os-Linux => }/drivers/tuntap_kern.c  |  0
 .../drivers => include/shared}/etap.h         |  0
 arch/um/include/shared/init.h                 |  5 ++
 .../drivers => include/shared}/tuntap.h       |  0
 arch/um/kernel/Makefile                       |  1 +
 arch/um/{os-Linux => kernel}/user_syms.c      |  0
 arch/um/os-Linux/Makefile                     | 21 --------
 arch/um/os-Linux/drivers/Makefile             | 13 -----
 tools/um/Makefile                             |  4 +-
 tools/um/uml/Build                            | 50 +++++++++++++++++++
 tools/um/uml/drivers/Build                    | 10 ++++
 .../um/uml}/drivers/ethertap_user.c           |  0
 .../um/uml}/drivers/tuntap_user.c             |  0
 {arch/um/os-Linux => tools/um/uml}/elf_aux.c  |  0
 {arch/um/os-Linux => tools/um/uml}/execvp.c   |  4 --
 {arch/um/os-Linux => tools/um/uml}/file.c     |  0
 {arch/um/os-Linux => tools/um/uml}/helper.c   |  0
 {arch/um/os-Linux => tools/um/uml}/irq.c      |  0
 {arch/um/os-Linux => tools/um/uml}/main.c     |  0
 {arch/um/os-Linux => tools/um/uml}/mem.c      |  0
 {arch/um/os-Linux => tools/um/uml}/process.c  |  0
 .../um/os-Linux => tools/um/uml}/registers.c  |  0
 {arch/um/os-Linux => tools/um/uml}/sigio.c    |  0
 {arch/um/os-Linux => tools/um/uml}/signal.c   |  0
 .../skas/Makefile => tools/um/uml/skas/Build  |  6 +--
 {arch/um/os-Linux => tools/um/uml}/skas/mem.c |  0
 .../os-Linux => tools/um/uml}/skas/process.c  |  3 +-
 {arch/um/os-Linux => tools/um/uml}/start_up.c |  0
 {arch/um/os-Linux => tools/um/uml}/time.c     |  0
 {arch/um/os-Linux => tools/um/uml}/tty.c      |  0
 {arch/um/os-Linux => tools/um/uml}/umid.c     |  0
 {arch/um/os-Linux => tools/um/uml}/util.c     |  0
 35 files changed, 73 insertions(+), 49 deletions(-)
 rename arch/um/{os-Linux => }/drivers/ethertap_kern.c (100%)
 rename arch/um/{os-Linux => }/drivers/tuntap_kern.c (100%)
 rename arch/um/{os-Linux/drivers => include/shared}/etap.h (100%)
 rename arch/um/{os-Linux/drivers => include/shared}/tuntap.h (100%)
 rename arch/um/{os-Linux => kernel}/user_syms.c (100%)
 delete mode 100644 arch/um/os-Linux/Makefile
 delete mode 100644 arch/um/os-Linux/drivers/Makefile
 create mode 100644 tools/um/uml/drivers/Build
 rename {arch/um/os-Linux => tools/um/uml}/drivers/ethertap_user.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/drivers/tuntap_user.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/elf_aux.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/execvp.c (98%)
 rename {arch/um/os-Linux => tools/um/uml}/file.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/helper.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/irq.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/main.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/mem.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/process.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/registers.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/sigio.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/signal.c (100%)
 rename arch/um/os-Linux/skas/Makefile => tools/um/uml/skas/Build (56%)
 rename {arch/um/os-Linux => tools/um/uml}/skas/mem.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/skas/process.c (99%)
 rename {arch/um/os-Linux => tools/um/uml}/start_up.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/time.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/tty.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/umid.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/util.c (100%)

diff --git a/arch/um/Makefile b/arch/um/Makefile
index 48fbbc3a0032..bce0cccae085 100644
--- a/arch/um/Makefile
+++ b/arch/um/Makefile
@@ -24,8 +24,7 @@ OS := $(shell uname -s)
 SHELL := /bin/bash
 
 core-y			+= $(ARCH_DIR)/kernel/		\
-			   $(ARCH_DIR)/drivers/		\
-			   $(ARCH_DIR)/os-$(OS)/
+			   $(ARCH_DIR)/drivers/
 
 MODE_INCLUDE	+= -I$(srctree)/$(ARCH_DIR)/include/shared/skas
 
diff --git a/arch/um/drivers/Makefile b/arch/um/drivers/Makefile
index 2a249f619467..ae96c83e312d 100644
--- a/arch/um/drivers/Makefile
+++ b/arch/um/drivers/Makefile
@@ -48,6 +48,8 @@ obj-$(CONFIG_UML_NET_VECTOR) += vector.o
 obj-$(CONFIG_UML_NET_VDE) += vde.o
 obj-$(CONFIG_UML_NET_MCAST) += umcast.o
 obj-$(CONFIG_UML_NET_PCAP) += pcap.o
+obj-$(CONFIG_UML_NET_ETHERTAP) += ethertap_kern.o
+obj-$(CONFIG_UML_NET_TUNTAP) += tuntap_kern.o
 obj-$(CONFIG_UML_NET) += net.o 
 obj-$(CONFIG_MCONSOLE) += mconsole.o
 obj-$(CONFIG_MMAPPER) += mmapper_kern.o 
diff --git a/arch/um/os-Linux/drivers/ethertap_kern.c b/arch/um/drivers/ethertap_kern.c
similarity index 100%
rename from arch/um/os-Linux/drivers/ethertap_kern.c
rename to arch/um/drivers/ethertap_kern.c
diff --git a/arch/um/os-Linux/drivers/tuntap_kern.c b/arch/um/drivers/tuntap_kern.c
similarity index 100%
rename from arch/um/os-Linux/drivers/tuntap_kern.c
rename to arch/um/drivers/tuntap_kern.c
diff --git a/arch/um/os-Linux/drivers/etap.h b/arch/um/include/shared/etap.h
similarity index 100%
rename from arch/um/os-Linux/drivers/etap.h
rename to arch/um/include/shared/etap.h
diff --git a/arch/um/include/shared/init.h b/arch/um/include/shared/init.h
index 1a659e2e8cc3..fa2d3138d497 100644
--- a/arch/um/include/shared/init.h
+++ b/arch/um/include/shared/init.h
@@ -41,7 +41,12 @@
 typedef int (*initcall_t)(void);
 typedef void (*exitcall_t)(void);
 
+#ifndef __UM_HOST__
 #include <linux/compiler_types.h>
+#else
+#define __used                          __attribute__((__used__))
+#define __section(S)                    __attribute__((__section__(#S)))
+#endif
 
 /* These are for everybody (although not all archs will actually
    discard it in modules) */
diff --git a/arch/um/os-Linux/drivers/tuntap.h b/arch/um/include/shared/tuntap.h
similarity index 100%
rename from arch/um/os-Linux/drivers/tuntap.h
rename to arch/um/include/shared/tuntap.h
diff --git a/arch/um/kernel/Makefile b/arch/um/kernel/Makefile
index 5aa882011e04..9b63831a69e1 100644
--- a/arch/um/kernel/Makefile
+++ b/arch/um/kernel/Makefile
@@ -24,6 +24,7 @@ obj-$(CONFIG_GPROF)	+= gprof_syms.o
 obj-$(CONFIG_GCOV)	+= gmon_syms.o
 obj-$(CONFIG_EARLY_PRINTK) += early_printk.o
 obj-$(CONFIG_STACKTRACE) += stacktrace.o
+obj-y += user_syms.o
 
 USER_OBJS := config.o
 
diff --git a/arch/um/os-Linux/user_syms.c b/arch/um/kernel/user_syms.c
similarity index 100%
rename from arch/um/os-Linux/user_syms.c
rename to arch/um/kernel/user_syms.c
diff --git a/arch/um/os-Linux/Makefile b/arch/um/os-Linux/Makefile
deleted file mode 100644
index 77ac50baa3f8..000000000000
--- a/arch/um/os-Linux/Makefile
+++ /dev/null
@@ -1,21 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0
-# 
-# Copyright (C) 2000 - 2007 Jeff Dike (jdike@{addtoit,linux.intel}.com)
-#
-
-# Don't instrument UML-specific code
-KCOV_INSTRUMENT                := n
-
-obj-y = execvp.o file.o helper.o irq.o main.o mem.o process.o \
-	registers.o sigio.o signal.o start_up.o time.o tty.o \
-	umid.o user_syms.o util.o drivers/ skas/
-
-CFLAGS_signal.o += -Wframe-larger-than=4096
-
-obj-$(CONFIG_ARCH_REUSE_HOST_VSYSCALL_AREA) += elf_aux.o
-
-USER_OBJS := $(user-objs-y) elf_aux.o execvp.o file.o helper.o irq.o \
-	main.o mem.o process.o registers.o sigio.o signal.o start_up.o time.o \
-	tty.o umid.o util.o
-
-include arch/um/scripts/Makefile.rules
diff --git a/arch/um/os-Linux/drivers/Makefile b/arch/um/os-Linux/drivers/Makefile
deleted file mode 100644
index d79e75f1b69a..000000000000
--- a/arch/um/os-Linux/drivers/Makefile
+++ /dev/null
@@ -1,13 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0
-# 
-# Copyright (C) 2000, 2002 Jeff Dike (jdike@karaya.com)
-#
-
-ethertap-objs := ethertap_kern.o ethertap_user.o
-tuntap-objs := tuntap_kern.o tuntap_user.o
-
-obj-y = 
-obj-$(CONFIG_UML_NET_ETHERTAP) += ethertap.o
-obj-$(CONFIG_UML_NET_TUNTAP) += tuntap.o
-
-include arch/um/scripts/Makefile.rules
diff --git a/tools/um/Makefile b/tools/um/Makefile
index 2b4e6b63355b..0fdf77261e74 100644
--- a/tools/um/Makefile
+++ b/tools/um/Makefile
@@ -50,7 +50,7 @@ $(objtree)/linux.o:
 $(OUTPUT)lib/linux.o: $(objtree)/linux.o
 	$(Q)cp $(objtree)/linux.o $(OUTPUT)lib/linux.o
 
-$(OUTPUT)liblinux.a: $(OUTPUT)lib/linux.o $(OUTPUT)uml/liblinux-in.o
+$(OUTPUT)liblinux.a: $(OUTPUT)lib/linux.o $(OUTPUT)uml/liblinux-in.o $(OUTPUT)lib/liblinux-in.o
 	$(QUIET_AR)$(AR) -rc $@ $^
 
 # rule to link programs
@@ -58,7 +58,7 @@ $(OUTPUT)%: $(OUTPUT)%-in.o $(OUTPUT)liblinux.a
 	$(QUIET_LINK)$(CC) $(LDFLAGS) $(LDFLAGS_$(notdir $*)-y) -o $@ $^ $(LDLIBS) $(LDLIBS_$(notdir $*)-y)
 
 # rule to build objects
-$(OUTPUT)%-in.o: FORCE
+$(OUTPUT)%-in.o: $(OUTPUT)lib/linux.o FORCE
 	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
 
 RM := rm -f
diff --git a/tools/um/uml/Build b/tools/um/uml/Build
index e69de29bb2d1..0e629c9cb8eb 100644
--- a/tools/um/uml/Build
+++ b/tools/um/uml/Build
@@ -0,0 +1,50 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Copyright (C) 2000 - 2007 Jeff Dike (jdike@{addtoit,linux.intel}.com)
+#
+
+# Don't instrument UML-specific code
+KCOV_INSTRUMENT                := n
+
+include $(objtree)/include/config/auto.conf
+
+liblinux-y += execvp.o file.o helper.o irq.o main.o mem.o process.o \
+	registers.o sigio.o signal.o start_up.o time.o tty.o \
+	umid.o util.o drivers/ skas/
+
+CFLAGS_signal.o += -Wframe-larger-than=4096
+
+liblinux-$(CONFIG_ARCH_REUSE_HOST_VSYSCALL_AREA) += elf_aux.o
+
+export O = $(srctree)
+objtree := $(O)
+# reset CFLAGS
+CFLAGS := -g -O2
+
+# from arch/um/Makefile
+ARCH_DIR := arch/um
+HEADER_ARCH 	:= x86
+HOST_DIR := arch/$(HEADER_ARCH)
+ifdef CONFIG_64BIT
+  KBUILD_CFLAGS += -mcmodel=large
+endif
+KBUILD_CFLAGS += $(CFLAGS) $(CFLAGS-y) -D__arch_um__ \
+	$(ARCH_INCLUDE) $(MODE_INCLUDE) -Dvmap=kernel_vmap	\
+	-Dlongjmp=kernel_longjmp -Dsetjmp=kernel_setjmp \
+	-Din6addr_loopback=kernel_in6addr_loopback \
+	-Din6addr_any=kernel_in6addr_any -Dstrrchr=kernel_strrchr
+SHARED_HEADERS	:= $(ARCH_DIR)/include/shared
+MODE_INCLUDE	+= -I$(srctree)/$(ARCH_DIR)/include/shared/skas
+ARCH_INCLUDE	:= -I$(srctree)/$(SHARED_HEADERS)
+ARCH_INCLUDE	+= -I$(srctree)/$(HOST_DIR)/um/shared
+KBUILD_CPPFLAGS += -I$(srctree)/$(HOST_DIR)/um
+USER_CFLAGS := $(patsubst $(KERNEL_DEFINES),,$(patsubst -I%,,$(KBUILD_CFLAGS))) \
+		$(ARCH_INCLUDE) $(MODE_INCLUDE) $(filter -I%,$(CFLAGS)) \
+		-D_FILE_OFFSET_BITS=64 -idirafter $(srctree)/include/uapi \
+		-idirafter $(objtree)/include -D__KERNEL__ -D__UM_HOST__
+
+# from Makefile-os-Linux
+USER_CFLAGS += -D_GNU_SOURCE -D_LARGEFILE64_SOURCE
+
+# from Makefile.rules
+CFLAGS += $(USER_CFLAGS) -include $(srctree)/tools/include/linux/kern_levels.h -include user.h $(CFLAGS_$(basetarget).o)
diff --git a/tools/um/uml/drivers/Build b/tools/um/uml/drivers/Build
new file mode 100644
index 000000000000..c7a8eaa97d72
--- /dev/null
+++ b/tools/um/uml/drivers/Build
@@ -0,0 +1,10 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Copyright (C) 2000, 2002 Jeff Dike (jdike@karaya.com)
+#
+
+include $(objtree)/include/config/auto.conf
+
+
+liblinux-$(CONFIG_UML_NET_ETHERTAP) += ethertap_user.o
+liblinux-$(CONFIG_UML_NET_TUNTAP) += tuntap_user.o
diff --git a/arch/um/os-Linux/drivers/ethertap_user.c b/tools/um/uml/drivers/ethertap_user.c
similarity index 100%
rename from arch/um/os-Linux/drivers/ethertap_user.c
rename to tools/um/uml/drivers/ethertap_user.c
diff --git a/arch/um/os-Linux/drivers/tuntap_user.c b/tools/um/uml/drivers/tuntap_user.c
similarity index 100%
rename from arch/um/os-Linux/drivers/tuntap_user.c
rename to tools/um/uml/drivers/tuntap_user.c
diff --git a/arch/um/os-Linux/elf_aux.c b/tools/um/uml/elf_aux.c
similarity index 100%
rename from arch/um/os-Linux/elf_aux.c
rename to tools/um/uml/elf_aux.c
diff --git a/arch/um/os-Linux/execvp.c b/tools/um/uml/execvp.c
similarity index 98%
rename from arch/um/os-Linux/execvp.c
rename to tools/um/uml/execvp.c
index 84a0777c2a45..c105e0e7b778 100644
--- a/arch/um/os-Linux/execvp.c
+++ b/tools/um/uml/execvp.c
@@ -26,12 +26,8 @@
 #include <errno.h>
 #include <limits.h>
 
-#ifndef TEST
-#include <um_malloc.h>
-#else
 #include <stdio.h>
 #define um_kmalloc malloc
-#endif
 #include <os.h>
 
 /* Execute FILE, searching in the `PATH' environment variable if it contains
diff --git a/arch/um/os-Linux/file.c b/tools/um/uml/file.c
similarity index 100%
rename from arch/um/os-Linux/file.c
rename to tools/um/uml/file.c
diff --git a/arch/um/os-Linux/helper.c b/tools/um/uml/helper.c
similarity index 100%
rename from arch/um/os-Linux/helper.c
rename to tools/um/uml/helper.c
diff --git a/arch/um/os-Linux/irq.c b/tools/um/uml/irq.c
similarity index 100%
rename from arch/um/os-Linux/irq.c
rename to tools/um/uml/irq.c
diff --git a/arch/um/os-Linux/main.c b/tools/um/uml/main.c
similarity index 100%
rename from arch/um/os-Linux/main.c
rename to tools/um/uml/main.c
diff --git a/arch/um/os-Linux/mem.c b/tools/um/uml/mem.c
similarity index 100%
rename from arch/um/os-Linux/mem.c
rename to tools/um/uml/mem.c
diff --git a/arch/um/os-Linux/process.c b/tools/um/uml/process.c
similarity index 100%
rename from arch/um/os-Linux/process.c
rename to tools/um/uml/process.c
diff --git a/arch/um/os-Linux/registers.c b/tools/um/uml/registers.c
similarity index 100%
rename from arch/um/os-Linux/registers.c
rename to tools/um/uml/registers.c
diff --git a/arch/um/os-Linux/sigio.c b/tools/um/uml/sigio.c
similarity index 100%
rename from arch/um/os-Linux/sigio.c
rename to tools/um/uml/sigio.c
diff --git a/arch/um/os-Linux/signal.c b/tools/um/uml/signal.c
similarity index 100%
rename from arch/um/os-Linux/signal.c
rename to tools/um/uml/signal.c
diff --git a/arch/um/os-Linux/skas/Makefile b/tools/um/uml/skas/Build
similarity index 56%
rename from arch/um/os-Linux/skas/Makefile
rename to tools/um/uml/skas/Build
index c4566e788815..5fc9d62df863 100644
--- a/arch/um/os-Linux/skas/Makefile
+++ b/tools/um/uml/skas/Build
@@ -3,8 +3,4 @@
 # Copyright (C) 2002 - 2007 Jeff Dike (jdike@{linux.intel,addtoit}.com)
 #
 
-obj-y := mem.o process.o
-
-USER_OBJS := $(obj-y)
-
-include arch/um/scripts/Makefile.rules
+liblinux-y += mem.o process.o
diff --git a/arch/um/os-Linux/skas/mem.c b/tools/um/uml/skas/mem.c
similarity index 100%
rename from arch/um/os-Linux/skas/mem.c
rename to tools/um/uml/skas/mem.c
diff --git a/arch/um/os-Linux/skas/process.c b/tools/um/uml/skas/process.c
similarity index 99%
rename from arch/um/os-Linux/skas/process.c
rename to tools/um/uml/skas/process.c
index 0621d521208e..0af44c2857b8 100644
--- a/arch/um/os-Linux/skas/process.c
+++ b/tools/um/uml/skas/process.c
@@ -21,7 +21,6 @@
 #include <registers.h>
 #include <skas.h>
 #include <sysdep/stub.h>
-#include <linux/threads.h>
 
 int is_skas_winch(int pid, int fd, void *data)
 {
@@ -248,7 +247,7 @@ static int userspace_tramp(void *stack)
 	return 0;
 }
 
-int userspace_pid[NR_CPUS];
+int userspace_pid[UM_NR_CPUS];
 
 /**
  * start_userspace() - prepare a new userspace process
diff --git a/arch/um/os-Linux/start_up.c b/tools/um/uml/start_up.c
similarity index 100%
rename from arch/um/os-Linux/start_up.c
rename to tools/um/uml/start_up.c
diff --git a/arch/um/os-Linux/time.c b/tools/um/uml/time.c
similarity index 100%
rename from arch/um/os-Linux/time.c
rename to tools/um/uml/time.c
diff --git a/arch/um/os-Linux/tty.c b/tools/um/uml/tty.c
similarity index 100%
rename from arch/um/os-Linux/tty.c
rename to tools/um/uml/tty.c
diff --git a/arch/um/os-Linux/umid.c b/tools/um/uml/umid.c
similarity index 100%
rename from arch/um/os-Linux/umid.c
rename to tools/um/uml/umid.c
diff --git a/arch/um/os-Linux/util.c b/tools/um/uml/util.c
similarity index 100%
rename from arch/um/os-Linux/util.c
rename to tools/um/uml/util.c
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v8 02/20] um: move arch/um/os-Linux dir to tools/um/uml
@ 2021-01-20  2:27             ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-01-20  2:27 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, thehajime, retrage01

This patch moves underlying OS dependent part under arch/um to tools/um
directory so that arch/um code does not need to build host build
facilities (e.g., libc).

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/Makefile                              |  3 +-
 arch/um/drivers/Makefile                      |  2 +
 .../um/{os-Linux => }/drivers/ethertap_kern.c |  0
 arch/um/{os-Linux => }/drivers/tuntap_kern.c  |  0
 .../drivers => include/shared}/etap.h         |  0
 arch/um/include/shared/init.h                 |  5 ++
 .../drivers => include/shared}/tuntap.h       |  0
 arch/um/kernel/Makefile                       |  1 +
 arch/um/{os-Linux => kernel}/user_syms.c      |  0
 arch/um/os-Linux/Makefile                     | 21 --------
 arch/um/os-Linux/drivers/Makefile             | 13 -----
 tools/um/Makefile                             |  4 +-
 tools/um/uml/Build                            | 50 +++++++++++++++++++
 tools/um/uml/drivers/Build                    | 10 ++++
 .../um/uml}/drivers/ethertap_user.c           |  0
 .../um/uml}/drivers/tuntap_user.c             |  0
 {arch/um/os-Linux => tools/um/uml}/elf_aux.c  |  0
 {arch/um/os-Linux => tools/um/uml}/execvp.c   |  4 --
 {arch/um/os-Linux => tools/um/uml}/file.c     |  0
 {arch/um/os-Linux => tools/um/uml}/helper.c   |  0
 {arch/um/os-Linux => tools/um/uml}/irq.c      |  0
 {arch/um/os-Linux => tools/um/uml}/main.c     |  0
 {arch/um/os-Linux => tools/um/uml}/mem.c      |  0
 {arch/um/os-Linux => tools/um/uml}/process.c  |  0
 .../um/os-Linux => tools/um/uml}/registers.c  |  0
 {arch/um/os-Linux => tools/um/uml}/sigio.c    |  0
 {arch/um/os-Linux => tools/um/uml}/signal.c   |  0
 .../skas/Makefile => tools/um/uml/skas/Build  |  6 +--
 {arch/um/os-Linux => tools/um/uml}/skas/mem.c |  0
 .../os-Linux => tools/um/uml}/skas/process.c  |  3 +-
 {arch/um/os-Linux => tools/um/uml}/start_up.c |  0
 {arch/um/os-Linux => tools/um/uml}/time.c     |  0
 {arch/um/os-Linux => tools/um/uml}/tty.c      |  0
 {arch/um/os-Linux => tools/um/uml}/umid.c     |  0
 {arch/um/os-Linux => tools/um/uml}/util.c     |  0
 35 files changed, 73 insertions(+), 49 deletions(-)
 rename arch/um/{os-Linux => }/drivers/ethertap_kern.c (100%)
 rename arch/um/{os-Linux => }/drivers/tuntap_kern.c (100%)
 rename arch/um/{os-Linux/drivers => include/shared}/etap.h (100%)
 rename arch/um/{os-Linux/drivers => include/shared}/tuntap.h (100%)
 rename arch/um/{os-Linux => kernel}/user_syms.c (100%)
 delete mode 100644 arch/um/os-Linux/Makefile
 delete mode 100644 arch/um/os-Linux/drivers/Makefile
 create mode 100644 tools/um/uml/drivers/Build
 rename {arch/um/os-Linux => tools/um/uml}/drivers/ethertap_user.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/drivers/tuntap_user.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/elf_aux.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/execvp.c (98%)
 rename {arch/um/os-Linux => tools/um/uml}/file.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/helper.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/irq.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/main.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/mem.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/process.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/registers.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/sigio.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/signal.c (100%)
 rename arch/um/os-Linux/skas/Makefile => tools/um/uml/skas/Build (56%)
 rename {arch/um/os-Linux => tools/um/uml}/skas/mem.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/skas/process.c (99%)
 rename {arch/um/os-Linux => tools/um/uml}/start_up.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/time.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/tty.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/umid.c (100%)
 rename {arch/um/os-Linux => tools/um/uml}/util.c (100%)

diff --git a/arch/um/Makefile b/arch/um/Makefile
index 48fbbc3a0032..bce0cccae085 100644
--- a/arch/um/Makefile
+++ b/arch/um/Makefile
@@ -24,8 +24,7 @@ OS := $(shell uname -s)
 SHELL := /bin/bash
 
 core-y			+= $(ARCH_DIR)/kernel/		\
-			   $(ARCH_DIR)/drivers/		\
-			   $(ARCH_DIR)/os-$(OS)/
+			   $(ARCH_DIR)/drivers/
 
 MODE_INCLUDE	+= -I$(srctree)/$(ARCH_DIR)/include/shared/skas
 
diff --git a/arch/um/drivers/Makefile b/arch/um/drivers/Makefile
index 2a249f619467..ae96c83e312d 100644
--- a/arch/um/drivers/Makefile
+++ b/arch/um/drivers/Makefile
@@ -48,6 +48,8 @@ obj-$(CONFIG_UML_NET_VECTOR) += vector.o
 obj-$(CONFIG_UML_NET_VDE) += vde.o
 obj-$(CONFIG_UML_NET_MCAST) += umcast.o
 obj-$(CONFIG_UML_NET_PCAP) += pcap.o
+obj-$(CONFIG_UML_NET_ETHERTAP) += ethertap_kern.o
+obj-$(CONFIG_UML_NET_TUNTAP) += tuntap_kern.o
 obj-$(CONFIG_UML_NET) += net.o 
 obj-$(CONFIG_MCONSOLE) += mconsole.o
 obj-$(CONFIG_MMAPPER) += mmapper_kern.o 
diff --git a/arch/um/os-Linux/drivers/ethertap_kern.c b/arch/um/drivers/ethertap_kern.c
similarity index 100%
rename from arch/um/os-Linux/drivers/ethertap_kern.c
rename to arch/um/drivers/ethertap_kern.c
diff --git a/arch/um/os-Linux/drivers/tuntap_kern.c b/arch/um/drivers/tuntap_kern.c
similarity index 100%
rename from arch/um/os-Linux/drivers/tuntap_kern.c
rename to arch/um/drivers/tuntap_kern.c
diff --git a/arch/um/os-Linux/drivers/etap.h b/arch/um/include/shared/etap.h
similarity index 100%
rename from arch/um/os-Linux/drivers/etap.h
rename to arch/um/include/shared/etap.h
diff --git a/arch/um/include/shared/init.h b/arch/um/include/shared/init.h
index 1a659e2e8cc3..fa2d3138d497 100644
--- a/arch/um/include/shared/init.h
+++ b/arch/um/include/shared/init.h
@@ -41,7 +41,12 @@
 typedef int (*initcall_t)(void);
 typedef void (*exitcall_t)(void);
 
+#ifndef __UM_HOST__
 #include <linux/compiler_types.h>
+#else
+#define __used                          __attribute__((__used__))
+#define __section(S)                    __attribute__((__section__(#S)))
+#endif
 
 /* These are for everybody (although not all archs will actually
    discard it in modules) */
diff --git a/arch/um/os-Linux/drivers/tuntap.h b/arch/um/include/shared/tuntap.h
similarity index 100%
rename from arch/um/os-Linux/drivers/tuntap.h
rename to arch/um/include/shared/tuntap.h
diff --git a/arch/um/kernel/Makefile b/arch/um/kernel/Makefile
index 5aa882011e04..9b63831a69e1 100644
--- a/arch/um/kernel/Makefile
+++ b/arch/um/kernel/Makefile
@@ -24,6 +24,7 @@ obj-$(CONFIG_GPROF)	+= gprof_syms.o
 obj-$(CONFIG_GCOV)	+= gmon_syms.o
 obj-$(CONFIG_EARLY_PRINTK) += early_printk.o
 obj-$(CONFIG_STACKTRACE) += stacktrace.o
+obj-y += user_syms.o
 
 USER_OBJS := config.o
 
diff --git a/arch/um/os-Linux/user_syms.c b/arch/um/kernel/user_syms.c
similarity index 100%
rename from arch/um/os-Linux/user_syms.c
rename to arch/um/kernel/user_syms.c
diff --git a/arch/um/os-Linux/Makefile b/arch/um/os-Linux/Makefile
deleted file mode 100644
index 77ac50baa3f8..000000000000
--- a/arch/um/os-Linux/Makefile
+++ /dev/null
@@ -1,21 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0
-# 
-# Copyright (C) 2000 - 2007 Jeff Dike (jdike@{addtoit,linux.intel}.com)
-#
-
-# Don't instrument UML-specific code
-KCOV_INSTRUMENT                := n
-
-obj-y = execvp.o file.o helper.o irq.o main.o mem.o process.o \
-	registers.o sigio.o signal.o start_up.o time.o tty.o \
-	umid.o user_syms.o util.o drivers/ skas/
-
-CFLAGS_signal.o += -Wframe-larger-than=4096
-
-obj-$(CONFIG_ARCH_REUSE_HOST_VSYSCALL_AREA) += elf_aux.o
-
-USER_OBJS := $(user-objs-y) elf_aux.o execvp.o file.o helper.o irq.o \
-	main.o mem.o process.o registers.o sigio.o signal.o start_up.o time.o \
-	tty.o umid.o util.o
-
-include arch/um/scripts/Makefile.rules
diff --git a/arch/um/os-Linux/drivers/Makefile b/arch/um/os-Linux/drivers/Makefile
deleted file mode 100644
index d79e75f1b69a..000000000000
--- a/arch/um/os-Linux/drivers/Makefile
+++ /dev/null
@@ -1,13 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0
-# 
-# Copyright (C) 2000, 2002 Jeff Dike (jdike@karaya.com)
-#
-
-ethertap-objs := ethertap_kern.o ethertap_user.o
-tuntap-objs := tuntap_kern.o tuntap_user.o
-
-obj-y = 
-obj-$(CONFIG_UML_NET_ETHERTAP) += ethertap.o
-obj-$(CONFIG_UML_NET_TUNTAP) += tuntap.o
-
-include arch/um/scripts/Makefile.rules
diff --git a/tools/um/Makefile b/tools/um/Makefile
index 2b4e6b63355b..0fdf77261e74 100644
--- a/tools/um/Makefile
+++ b/tools/um/Makefile
@@ -50,7 +50,7 @@ $(objtree)/linux.o:
 $(OUTPUT)lib/linux.o: $(objtree)/linux.o
 	$(Q)cp $(objtree)/linux.o $(OUTPUT)lib/linux.o
 
-$(OUTPUT)liblinux.a: $(OUTPUT)lib/linux.o $(OUTPUT)uml/liblinux-in.o
+$(OUTPUT)liblinux.a: $(OUTPUT)lib/linux.o $(OUTPUT)uml/liblinux-in.o $(OUTPUT)lib/liblinux-in.o
 	$(QUIET_AR)$(AR) -rc $@ $^
 
 # rule to link programs
@@ -58,7 +58,7 @@ $(OUTPUT)%: $(OUTPUT)%-in.o $(OUTPUT)liblinux.a
 	$(QUIET_LINK)$(CC) $(LDFLAGS) $(LDFLAGS_$(notdir $*)-y) -o $@ $^ $(LDLIBS) $(LDLIBS_$(notdir $*)-y)
 
 # rule to build objects
-$(OUTPUT)%-in.o: FORCE
+$(OUTPUT)%-in.o: $(OUTPUT)lib/linux.o FORCE
 	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(patsubst %/,%,$(dir $*)) obj=$(notdir $*)
 
 RM := rm -f
diff --git a/tools/um/uml/Build b/tools/um/uml/Build
index e69de29bb2d1..0e629c9cb8eb 100644
--- a/tools/um/uml/Build
+++ b/tools/um/uml/Build
@@ -0,0 +1,50 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Copyright (C) 2000 - 2007 Jeff Dike (jdike@{addtoit,linux.intel}.com)
+#
+
+# Don't instrument UML-specific code
+KCOV_INSTRUMENT                := n
+
+include $(objtree)/include/config/auto.conf
+
+liblinux-y += execvp.o file.o helper.o irq.o main.o mem.o process.o \
+	registers.o sigio.o signal.o start_up.o time.o tty.o \
+	umid.o util.o drivers/ skas/
+
+CFLAGS_signal.o += -Wframe-larger-than=4096
+
+liblinux-$(CONFIG_ARCH_REUSE_HOST_VSYSCALL_AREA) += elf_aux.o
+
+export O = $(srctree)
+objtree := $(O)
+# reset CFLAGS
+CFLAGS := -g -O2
+
+# from arch/um/Makefile
+ARCH_DIR := arch/um
+HEADER_ARCH 	:= x86
+HOST_DIR := arch/$(HEADER_ARCH)
+ifdef CONFIG_64BIT
+  KBUILD_CFLAGS += -mcmodel=large
+endif
+KBUILD_CFLAGS += $(CFLAGS) $(CFLAGS-y) -D__arch_um__ \
+	$(ARCH_INCLUDE) $(MODE_INCLUDE) -Dvmap=kernel_vmap	\
+	-Dlongjmp=kernel_longjmp -Dsetjmp=kernel_setjmp \
+	-Din6addr_loopback=kernel_in6addr_loopback \
+	-Din6addr_any=kernel_in6addr_any -Dstrrchr=kernel_strrchr
+SHARED_HEADERS	:= $(ARCH_DIR)/include/shared
+MODE_INCLUDE	+= -I$(srctree)/$(ARCH_DIR)/include/shared/skas
+ARCH_INCLUDE	:= -I$(srctree)/$(SHARED_HEADERS)
+ARCH_INCLUDE	+= -I$(srctree)/$(HOST_DIR)/um/shared
+KBUILD_CPPFLAGS += -I$(srctree)/$(HOST_DIR)/um
+USER_CFLAGS := $(patsubst $(KERNEL_DEFINES),,$(patsubst -I%,,$(KBUILD_CFLAGS))) \
+		$(ARCH_INCLUDE) $(MODE_INCLUDE) $(filter -I%,$(CFLAGS)) \
+		-D_FILE_OFFSET_BITS=64 -idirafter $(srctree)/include/uapi \
+		-idirafter $(objtree)/include -D__KERNEL__ -D__UM_HOST__
+
+# from Makefile-os-Linux
+USER_CFLAGS += -D_GNU_SOURCE -D_LARGEFILE64_SOURCE
+
+# from Makefile.rules
+CFLAGS += $(USER_CFLAGS) -include $(srctree)/tools/include/linux/kern_levels.h -include user.h $(CFLAGS_$(basetarget).o)
diff --git a/tools/um/uml/drivers/Build b/tools/um/uml/drivers/Build
new file mode 100644
index 000000000000..c7a8eaa97d72
--- /dev/null
+++ b/tools/um/uml/drivers/Build
@@ -0,0 +1,10 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Copyright (C) 2000, 2002 Jeff Dike (jdike@karaya.com)
+#
+
+include $(objtree)/include/config/auto.conf
+
+
+liblinux-$(CONFIG_UML_NET_ETHERTAP) += ethertap_user.o
+liblinux-$(CONFIG_UML_NET_TUNTAP) += tuntap_user.o
diff --git a/arch/um/os-Linux/drivers/ethertap_user.c b/tools/um/uml/drivers/ethertap_user.c
similarity index 100%
rename from arch/um/os-Linux/drivers/ethertap_user.c
rename to tools/um/uml/drivers/ethertap_user.c
diff --git a/arch/um/os-Linux/drivers/tuntap_user.c b/tools/um/uml/drivers/tuntap_user.c
similarity index 100%
rename from arch/um/os-Linux/drivers/tuntap_user.c
rename to tools/um/uml/drivers/tuntap_user.c
diff --git a/arch/um/os-Linux/elf_aux.c b/tools/um/uml/elf_aux.c
similarity index 100%
rename from arch/um/os-Linux/elf_aux.c
rename to tools/um/uml/elf_aux.c
diff --git a/arch/um/os-Linux/execvp.c b/tools/um/uml/execvp.c
similarity index 98%
rename from arch/um/os-Linux/execvp.c
rename to tools/um/uml/execvp.c
index 84a0777c2a45..c105e0e7b778 100644
--- a/arch/um/os-Linux/execvp.c
+++ b/tools/um/uml/execvp.c
@@ -26,12 +26,8 @@
 #include <errno.h>
 #include <limits.h>
 
-#ifndef TEST
-#include <um_malloc.h>
-#else
 #include <stdio.h>
 #define um_kmalloc malloc
-#endif
 #include <os.h>
 
 /* Execute FILE, searching in the `PATH' environment variable if it contains
diff --git a/arch/um/os-Linux/file.c b/tools/um/uml/file.c
similarity index 100%
rename from arch/um/os-Linux/file.c
rename to tools/um/uml/file.c
diff --git a/arch/um/os-Linux/helper.c b/tools/um/uml/helper.c
similarity index 100%
rename from arch/um/os-Linux/helper.c
rename to tools/um/uml/helper.c
diff --git a/arch/um/os-Linux/irq.c b/tools/um/uml/irq.c
similarity index 100%
rename from arch/um/os-Linux/irq.c
rename to tools/um/uml/irq.c
diff --git a/arch/um/os-Linux/main.c b/tools/um/uml/main.c
similarity index 100%
rename from arch/um/os-Linux/main.c
rename to tools/um/uml/main.c
diff --git a/arch/um/os-Linux/mem.c b/tools/um/uml/mem.c
similarity index 100%
rename from arch/um/os-Linux/mem.c
rename to tools/um/uml/mem.c
diff --git a/arch/um/os-Linux/process.c b/tools/um/uml/process.c
similarity index 100%
rename from arch/um/os-Linux/process.c
rename to tools/um/uml/process.c
diff --git a/arch/um/os-Linux/registers.c b/tools/um/uml/registers.c
similarity index 100%
rename from arch/um/os-Linux/registers.c
rename to tools/um/uml/registers.c
diff --git a/arch/um/os-Linux/sigio.c b/tools/um/uml/sigio.c
similarity index 100%
rename from arch/um/os-Linux/sigio.c
rename to tools/um/uml/sigio.c
diff --git a/arch/um/os-Linux/signal.c b/tools/um/uml/signal.c
similarity index 100%
rename from arch/um/os-Linux/signal.c
rename to tools/um/uml/signal.c
diff --git a/arch/um/os-Linux/skas/Makefile b/tools/um/uml/skas/Build
similarity index 56%
rename from arch/um/os-Linux/skas/Makefile
rename to tools/um/uml/skas/Build
index c4566e788815..5fc9d62df863 100644
--- a/arch/um/os-Linux/skas/Makefile
+++ b/tools/um/uml/skas/Build
@@ -3,8 +3,4 @@
 # Copyright (C) 2002 - 2007 Jeff Dike (jdike@{linux.intel,addtoit}.com)
 #
 
-obj-y := mem.o process.o
-
-USER_OBJS := $(obj-y)
-
-include arch/um/scripts/Makefile.rules
+liblinux-y += mem.o process.o
diff --git a/arch/um/os-Linux/skas/mem.c b/tools/um/uml/skas/mem.c
similarity index 100%
rename from arch/um/os-Linux/skas/mem.c
rename to tools/um/uml/skas/mem.c
diff --git a/arch/um/os-Linux/skas/process.c b/tools/um/uml/skas/process.c
similarity index 99%
rename from arch/um/os-Linux/skas/process.c
rename to tools/um/uml/skas/process.c
index 0621d521208e..0af44c2857b8 100644
--- a/arch/um/os-Linux/skas/process.c
+++ b/tools/um/uml/skas/process.c
@@ -21,7 +21,6 @@
 #include <registers.h>
 #include <skas.h>
 #include <sysdep/stub.h>
-#include <linux/threads.h>
 
 int is_skas_winch(int pid, int fd, void *data)
 {
@@ -248,7 +247,7 @@ static int userspace_tramp(void *stack)
 	return 0;
 }
 
-int userspace_pid[NR_CPUS];
+int userspace_pid[UM_NR_CPUS];
 
 /**
  * start_userspace() - prepare a new userspace process
diff --git a/arch/um/os-Linux/start_up.c b/tools/um/uml/start_up.c
similarity index 100%
rename from arch/um/os-Linux/start_up.c
rename to tools/um/uml/start_up.c
diff --git a/arch/um/os-Linux/time.c b/tools/um/uml/time.c
similarity index 100%
rename from arch/um/os-Linux/time.c
rename to tools/um/uml/time.c
diff --git a/arch/um/os-Linux/tty.c b/tools/um/uml/tty.c
similarity index 100%
rename from arch/um/os-Linux/tty.c
rename to tools/um/uml/tty.c
diff --git a/arch/um/os-Linux/umid.c b/tools/um/uml/umid.c
similarity index 100%
rename from arch/um/os-Linux/umid.c
rename to tools/um/uml/umid.c
diff --git a/arch/um/os-Linux/util.c b/tools/um/uml/util.c
similarity index 100%
rename from arch/um/os-Linux/util.c
rename to tools/um/uml/util.c
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v8 03/20] um: move arch/x86/um/os-Linux to tools/um/uml/
  2021-01-20  2:27           ` Hajime Tazaki
@ 2021-01-20  2:27             ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-01-20  2:27 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: thehajime, tavi.purdila, retrage01, linux-kernel-library, linux-arch

Similar to arch/um/os-Linux code, arch/x86 also contains OS dependent
part as well.  This patch moves underlying OS dependent part to tools/um
directory so that arch/x86 code does not need to build host build
facilities (e.g., libc).

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/x86/um/Makefile                                |  2 +-
 arch/x86/um/os-Linux/Makefile                       | 13 -------------
 tools/um/uml/Build                                  |  1 +
 tools/um/uml/x86/Build                              | 11 +++++++++++
 .../x86/um/os-Linux => tools/um/uml/x86}/mcontext.c |  0
 {arch/x86/um/os-Linux => tools/um/uml/x86}/prctl.c  |  0
 .../um/os-Linux => tools/um/uml/x86}/registers.c    |  0
 .../um/os-Linux => tools/um/uml/x86}/task_size.c    |  0
 {arch/x86/um/os-Linux => tools/um/uml/x86}/tls.c    |  0
 9 files changed, 13 insertions(+), 14 deletions(-)
 delete mode 100644 arch/x86/um/os-Linux/Makefile
 create mode 100644 tools/um/uml/x86/Build
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/mcontext.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/prctl.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/registers.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/task_size.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/tls.c (100%)

diff --git a/arch/x86/um/Makefile b/arch/x86/um/Makefile
index 77f70b969d14..25e9846b36a2 100644
--- a/arch/x86/um/Makefile
+++ b/arch/x86/um/Makefile
@@ -13,7 +13,7 @@ obj-y = bugs_$(BITS).o delay.o fault.o ldt.o \
 	ptrace_$(BITS).o ptrace_user.o setjmp_$(BITS).o signal.o \
 	stub_$(BITS).o stub_segv.o \
 	sys_call_table_$(BITS).o sysrq_$(BITS).o tls_$(BITS).o \
-	mem_$(BITS).o subarch.o os-$(OS)/
+	mem_$(BITS).o subarch.o
 
 ifeq ($(CONFIG_X86_32),y)
 
diff --git a/arch/x86/um/os-Linux/Makefile b/arch/x86/um/os-Linux/Makefile
deleted file mode 100644
index 253bfb8cb702..000000000000
--- a/arch/x86/um/os-Linux/Makefile
+++ /dev/null
@@ -1,13 +0,0 @@
-#
-# Copyright (C) 2000 - 2007 Jeff Dike (jdike@{addtoit,linux.intel}.com)
-# Licensed under the GPL
-#
-
-obj-y = registers.o task_size.o mcontext.o
-
-obj-$(CONFIG_X86_32) += tls.o
-obj-$(CONFIG_64BIT) += prctl.o
-
-USER_OBJS := $(obj-y)
-
-include arch/um/scripts/Makefile.rules
diff --git a/tools/um/uml/Build b/tools/um/uml/Build
index 0e629c9cb8eb..810aa99f8409 100644
--- a/tools/um/uml/Build
+++ b/tools/um/uml/Build
@@ -14,6 +14,7 @@ liblinux-y += execvp.o file.o helper.o irq.o main.o mem.o process.o \
 
 CFLAGS_signal.o += -Wframe-larger-than=4096
 
+liblinux-y += x86/
 liblinux-$(CONFIG_ARCH_REUSE_HOST_VSYSCALL_AREA) += elf_aux.o
 
 export O = $(srctree)
diff --git a/tools/um/uml/x86/Build b/tools/um/uml/x86/Build
new file mode 100644
index 000000000000..16bbc7eb19d1
--- /dev/null
+++ b/tools/um/uml/x86/Build
@@ -0,0 +1,11 @@
+#
+# Copyright (C) 2000 - 2007 Jeff Dike (jdike@{addtoit,linux.intel}.com)
+# Licensed under the GPL
+#
+
+include $(objtree)/include/config/auto.conf
+
+liblinux-y = registers.o task_size.o mcontext.o
+
+liblinux-$(CONFIG_X86_32) += tls.o
+liblinux-$(CONFIG_64BIT) += prctl.o
diff --git a/arch/x86/um/os-Linux/mcontext.c b/tools/um/uml/x86/mcontext.c
similarity index 100%
rename from arch/x86/um/os-Linux/mcontext.c
rename to tools/um/uml/x86/mcontext.c
diff --git a/arch/x86/um/os-Linux/prctl.c b/tools/um/uml/x86/prctl.c
similarity index 100%
rename from arch/x86/um/os-Linux/prctl.c
rename to tools/um/uml/x86/prctl.c
diff --git a/arch/x86/um/os-Linux/registers.c b/tools/um/uml/x86/registers.c
similarity index 100%
rename from arch/x86/um/os-Linux/registers.c
rename to tools/um/uml/x86/registers.c
diff --git a/arch/x86/um/os-Linux/task_size.c b/tools/um/uml/x86/task_size.c
similarity index 100%
rename from arch/x86/um/os-Linux/task_size.c
rename to tools/um/uml/x86/task_size.c
diff --git a/arch/x86/um/os-Linux/tls.c b/tools/um/uml/x86/tls.c
similarity index 100%
rename from arch/x86/um/os-Linux/tls.c
rename to tools/um/uml/x86/tls.c
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v8 03/20] um: move arch/x86/um/os-Linux to tools/um/uml/
@ 2021-01-20  2:27             ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-01-20  2:27 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, thehajime, retrage01

Similar to arch/um/os-Linux code, arch/x86 also contains OS dependent
part as well.  This patch moves underlying OS dependent part to tools/um
directory so that arch/x86 code does not need to build host build
facilities (e.g., libc).

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/x86/um/Makefile                                |  2 +-
 arch/x86/um/os-Linux/Makefile                       | 13 -------------
 tools/um/uml/Build                                  |  1 +
 tools/um/uml/x86/Build                              | 11 +++++++++++
 .../x86/um/os-Linux => tools/um/uml/x86}/mcontext.c |  0
 {arch/x86/um/os-Linux => tools/um/uml/x86}/prctl.c  |  0
 .../um/os-Linux => tools/um/uml/x86}/registers.c    |  0
 .../um/os-Linux => tools/um/uml/x86}/task_size.c    |  0
 {arch/x86/um/os-Linux => tools/um/uml/x86}/tls.c    |  0
 9 files changed, 13 insertions(+), 14 deletions(-)
 delete mode 100644 arch/x86/um/os-Linux/Makefile
 create mode 100644 tools/um/uml/x86/Build
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/mcontext.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/prctl.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/registers.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/task_size.c (100%)
 rename {arch/x86/um/os-Linux => tools/um/uml/x86}/tls.c (100%)

diff --git a/arch/x86/um/Makefile b/arch/x86/um/Makefile
index 77f70b969d14..25e9846b36a2 100644
--- a/arch/x86/um/Makefile
+++ b/arch/x86/um/Makefile
@@ -13,7 +13,7 @@ obj-y = bugs_$(BITS).o delay.o fault.o ldt.o \
 	ptrace_$(BITS).o ptrace_user.o setjmp_$(BITS).o signal.o \
 	stub_$(BITS).o stub_segv.o \
 	sys_call_table_$(BITS).o sysrq_$(BITS).o tls_$(BITS).o \
-	mem_$(BITS).o subarch.o os-$(OS)/
+	mem_$(BITS).o subarch.o
 
 ifeq ($(CONFIG_X86_32),y)
 
diff --git a/arch/x86/um/os-Linux/Makefile b/arch/x86/um/os-Linux/Makefile
deleted file mode 100644
index 253bfb8cb702..000000000000
--- a/arch/x86/um/os-Linux/Makefile
+++ /dev/null
@@ -1,13 +0,0 @@
-#
-# Copyright (C) 2000 - 2007 Jeff Dike (jdike@{addtoit,linux.intel}.com)
-# Licensed under the GPL
-#
-
-obj-y = registers.o task_size.o mcontext.o
-
-obj-$(CONFIG_X86_32) += tls.o
-obj-$(CONFIG_64BIT) += prctl.o
-
-USER_OBJS := $(obj-y)
-
-include arch/um/scripts/Makefile.rules
diff --git a/tools/um/uml/Build b/tools/um/uml/Build
index 0e629c9cb8eb..810aa99f8409 100644
--- a/tools/um/uml/Build
+++ b/tools/um/uml/Build
@@ -14,6 +14,7 @@ liblinux-y += execvp.o file.o helper.o irq.o main.o mem.o process.o \
 
 CFLAGS_signal.o += -Wframe-larger-than=4096
 
+liblinux-y += x86/
 liblinux-$(CONFIG_ARCH_REUSE_HOST_VSYSCALL_AREA) += elf_aux.o
 
 export O = $(srctree)
diff --git a/tools/um/uml/x86/Build b/tools/um/uml/x86/Build
new file mode 100644
index 000000000000..16bbc7eb19d1
--- /dev/null
+++ b/tools/um/uml/x86/Build
@@ -0,0 +1,11 @@
+#
+# Copyright (C) 2000 - 2007 Jeff Dike (jdike@{addtoit,linux.intel}.com)
+# Licensed under the GPL
+#
+
+include $(objtree)/include/config/auto.conf
+
+liblinux-y = registers.o task_size.o mcontext.o
+
+liblinux-$(CONFIG_X86_32) += tls.o
+liblinux-$(CONFIG_64BIT) += prctl.o
diff --git a/arch/x86/um/os-Linux/mcontext.c b/tools/um/uml/x86/mcontext.c
similarity index 100%
rename from arch/x86/um/os-Linux/mcontext.c
rename to tools/um/uml/x86/mcontext.c
diff --git a/arch/x86/um/os-Linux/prctl.c b/tools/um/uml/x86/prctl.c
similarity index 100%
rename from arch/x86/um/os-Linux/prctl.c
rename to tools/um/uml/x86/prctl.c
diff --git a/arch/x86/um/os-Linux/registers.c b/tools/um/uml/x86/registers.c
similarity index 100%
rename from arch/x86/um/os-Linux/registers.c
rename to tools/um/uml/x86/registers.c
diff --git a/arch/x86/um/os-Linux/task_size.c b/tools/um/uml/x86/task_size.c
similarity index 100%
rename from arch/x86/um/os-Linux/task_size.c
rename to tools/um/uml/x86/task_size.c
diff --git a/arch/x86/um/os-Linux/tls.c b/tools/um/uml/x86/tls.c
similarity index 100%
rename from arch/x86/um/os-Linux/tls.c
rename to tools/um/uml/x86/tls.c
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v8 04/20] um: implement os_initcalls and os_exitcalls
  2021-01-20  2:27           ` Hajime Tazaki
@ 2021-01-20  2:27             ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-01-20  2:27 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: thehajime, tavi.purdila, retrage01, linux-kernel-library,
	linux-arch, Octavian Purdila

From: Octavian Purdila <tavi@cs.pub.ro>

This patch implements the init and exit calls for host code. It uses
the automatic __start_<section> and __stop_<section> variables that
are defined by gcc / ld when using custom sections.

Note that this patch should be merged with "um: move arch/um/os-Linux
dir to tools/um" but for now it is separate to make the review easier.

Signed-off-by: Octavian Purdila <tavi@cs.pub.ro>
---
 arch/um/include/shared/init.h | 14 ++++----------
 arch/um/kernel/reboot.c       |  5 +++++
 arch/um/kernel/um_arch.c      | 11 +++++++++++
 tools/um/uml/util.c           | 26 ++++++++++++++++++++++++++
 4 files changed, 46 insertions(+), 10 deletions(-)

diff --git a/arch/um/include/shared/init.h b/arch/um/include/shared/init.h
index fa2d3138d497..0d6de61c0f37 100644
--- a/arch/um/include/shared/init.h
+++ b/arch/um/include/shared/init.h
@@ -114,19 +114,13 @@ extern struct uml_param __uml_setup_start, __uml_setup_end;
 
 #ifdef __UM_HOST__
 
-#define __define_initcall(level,fn) \
-	static initcall_t __initcall_##fn __used \
-	__attribute__((__section__(".initcall" level ".init"))) = fn
-
-/* Userspace initcalls shouldn't depend on anything in the kernel, so we'll
- * make them run first.
- */
-#define __initcall(fn) __define_initcall("1", fn)
+#undef __uml_exit_call
+#define __uml_exit_call		__used __section(os_exitcalls)
+#define __init_call		__used __section(os_initcalls)
 
+#define __initcall(fn) static initcall_t __initcall_##fn __init_call = fn
 #define __exitcall(fn) static exitcall_t __exitcall_##fn __exit_call = fn
 
-#define __init_call	__used __section(".initcall.init")
-
 #endif
 
 #endif /* _LINUX_UML_INIT_H */
diff --git a/arch/um/kernel/reboot.c b/arch/um/kernel/reboot.c
index 48c0610d506e..5420aec411f4 100644
--- a/arch/um/kernel/reboot.c
+++ b/arch/um/kernel/reboot.c
@@ -35,10 +35,15 @@ static void kill_off_processes(void)
 	read_unlock(&tasklist_lock);
 }
 
+void __weak os_exitcalls(void)
+{
+}
+
 void uml_cleanup(void)
 {
 	kmalloc_ok = 0;
 	do_uml_exitcalls();
+	os_exitcalls();
 	kill_off_processes();
 }
 
diff --git a/arch/um/kernel/um_arch.c b/arch/um/kernel/um_arch.c
index 31d356b1ffd8..dfc6194b5ac7 100644
--- a/arch/um/kernel/um_arch.c
+++ b/arch/um/kernel/um_arch.c
@@ -444,3 +444,14 @@ static int init_pm_wake_signal(void)
 
 late_initcall(init_pm_wake_signal);
 #endif
+
+int __weak os_initcalls(void)
+{
+	return 0;
+}
+
+int __init run_os_initcalls(void)
+{
+	return os_initcalls();
+}
+early_initcall(run_os_initcalls);
diff --git a/tools/um/uml/util.c b/tools/um/uml/util.c
index 07327425d06e..8a3382c47f98 100644
--- a/tools/um/uml/util.c
+++ b/tools/um/uml/util.c
@@ -186,3 +186,29 @@ void os_warn(const char *fmt, ...)
 	vfprintf(stderr, fmt, list);
 	va_end(list);
 }
+
+extern void (*__start_os_exitcalls)(void);
+extern void (*__stop_os_exitcalls)(void);
+
+void os_exitcalls(void)
+{
+	exitcall_t *call;
+
+	call = &__stop_os_exitcalls;
+	while (--call >= &__start_os_exitcalls && (*call))
+		(*call)();
+}
+
+extern int (*__start_os_initcalls)(void);
+extern int (*__stop_os_initcalls)(void);
+
+int os_initcalls(void)
+{
+	initcall_t *call;
+
+	call = &__stop_os_initcalls;
+	while (--call >= &__start_os_initcalls && (*call))
+		(*call)();
+
+	return 0;
+}
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v8 04/20] um: implement os_initcalls and os_exitcalls
@ 2021-01-20  2:27             ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-01-20  2:27 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: linux-arch, tavi.purdila, Octavian Purdila, retrage01,
	linux-kernel-library, thehajime

From: Octavian Purdila <tavi@cs.pub.ro>

This patch implements the init and exit calls for host code. It uses
the automatic __start_<section> and __stop_<section> variables that
are defined by gcc / ld when using custom sections.

Note that this patch should be merged with "um: move arch/um/os-Linux
dir to tools/um" but for now it is separate to make the review easier.

Signed-off-by: Octavian Purdila <tavi@cs.pub.ro>
---
 arch/um/include/shared/init.h | 14 ++++----------
 arch/um/kernel/reboot.c       |  5 +++++
 arch/um/kernel/um_arch.c      | 11 +++++++++++
 tools/um/uml/util.c           | 26 ++++++++++++++++++++++++++
 4 files changed, 46 insertions(+), 10 deletions(-)

diff --git a/arch/um/include/shared/init.h b/arch/um/include/shared/init.h
index fa2d3138d497..0d6de61c0f37 100644
--- a/arch/um/include/shared/init.h
+++ b/arch/um/include/shared/init.h
@@ -114,19 +114,13 @@ extern struct uml_param __uml_setup_start, __uml_setup_end;
 
 #ifdef __UM_HOST__
 
-#define __define_initcall(level,fn) \
-	static initcall_t __initcall_##fn __used \
-	__attribute__((__section__(".initcall" level ".init"))) = fn
-
-/* Userspace initcalls shouldn't depend on anything in the kernel, so we'll
- * make them run first.
- */
-#define __initcall(fn) __define_initcall("1", fn)
+#undef __uml_exit_call
+#define __uml_exit_call		__used __section(os_exitcalls)
+#define __init_call		__used __section(os_initcalls)
 
+#define __initcall(fn) static initcall_t __initcall_##fn __init_call = fn
 #define __exitcall(fn) static exitcall_t __exitcall_##fn __exit_call = fn
 
-#define __init_call	__used __section(".initcall.init")
-
 #endif
 
 #endif /* _LINUX_UML_INIT_H */
diff --git a/arch/um/kernel/reboot.c b/arch/um/kernel/reboot.c
index 48c0610d506e..5420aec411f4 100644
--- a/arch/um/kernel/reboot.c
+++ b/arch/um/kernel/reboot.c
@@ -35,10 +35,15 @@ static void kill_off_processes(void)
 	read_unlock(&tasklist_lock);
 }
 
+void __weak os_exitcalls(void)
+{
+}
+
 void uml_cleanup(void)
 {
 	kmalloc_ok = 0;
 	do_uml_exitcalls();
+	os_exitcalls();
 	kill_off_processes();
 }
 
diff --git a/arch/um/kernel/um_arch.c b/arch/um/kernel/um_arch.c
index 31d356b1ffd8..dfc6194b5ac7 100644
--- a/arch/um/kernel/um_arch.c
+++ b/arch/um/kernel/um_arch.c
@@ -444,3 +444,14 @@ static int init_pm_wake_signal(void)
 
 late_initcall(init_pm_wake_signal);
 #endif
+
+int __weak os_initcalls(void)
+{
+	return 0;
+}
+
+int __init run_os_initcalls(void)
+{
+	return os_initcalls();
+}
+early_initcall(run_os_initcalls);
diff --git a/tools/um/uml/util.c b/tools/um/uml/util.c
index 07327425d06e..8a3382c47f98 100644
--- a/tools/um/uml/util.c
+++ b/tools/um/uml/util.c
@@ -186,3 +186,29 @@ void os_warn(const char *fmt, ...)
 	vfprintf(stderr, fmt, list);
 	va_end(list);
 }
+
+extern void (*__start_os_exitcalls)(void);
+extern void (*__stop_os_exitcalls)(void);
+
+void os_exitcalls(void)
+{
+	exitcall_t *call;
+
+	call = &__stop_os_exitcalls;
+	while (--call >= &__start_os_exitcalls && (*call))
+		(*call)();
+}
+
+extern int (*__start_os_initcalls)(void);
+extern int (*__stop_os_initcalls)(void);
+
+int os_initcalls(void)
+{
+	initcall_t *call;
+
+	call = &__stop_os_initcalls;
+	while (--call >= &__start_os_initcalls && (*call))
+		(*call)();
+
+	return 0;
+}
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v8 05/20] um: extend arch_switch_to for alternate SUBARCH
  2021-01-20  2:27           ` Hajime Tazaki
@ 2021-01-20  2:27             ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-01-20  2:27 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: thehajime, tavi.purdila, retrage01, linux-kernel-library, linux-arch

This commit introduces additional argument of previous task when
context switch happens.  New SUBARCH can use the new information to
switch tasks in a subarch-specific manner.

The patch is particularly required by library mode, implemented as a
SUBARCH of UML.  Having access to the previous thread will be required
in the library mode for it to implement thread switching correctly.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/kernel/process.c  | 6 +++---
 arch/x86/um/ptrace_32.c   | 2 +-
 arch/x86/um/syscalls_64.c | 2 +-
 3 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/arch/um/kernel/process.c b/arch/um/kernel/process.c
index 2a986ece5478..9b0a36f64339 100644
--- a/arch/um/kernel/process.c
+++ b/arch/um/kernel/process.c
@@ -83,7 +83,7 @@ static inline void set_current(struct task_struct *task)
 		{ external_pid(), task });
 }
 
-extern void arch_switch_to(struct task_struct *to);
+extern void arch_switch_to(struct task_struct *from, struct task_struct *to);
 
 void *__switch_to(struct task_struct *from, struct task_struct *to)
 {
@@ -91,7 +91,7 @@ void *__switch_to(struct task_struct *from, struct task_struct *to)
 	set_current(to);
 
 	switch_threads(&from->thread.switch_buf, &to->thread.switch_buf);
-	arch_switch_to(current);
+	arch_switch_to(from, to);
 
 	return current->thread.prev_sched;
 }
@@ -149,7 +149,7 @@ void fork_handler(void)
 	 * arch_switch_to isn't needed. We could want to apply this to
 	 * improve performance. -bb
 	 */
-	arch_switch_to(current);
+	arch_switch_to(NULL, current);
 
 	current->thread.prev_sched = NULL;
 
diff --git a/arch/x86/um/ptrace_32.c b/arch/x86/um/ptrace_32.c
index 2497bac56066..0f184710d4ca 100644
--- a/arch/x86/um/ptrace_32.c
+++ b/arch/x86/um/ptrace_32.c
@@ -11,7 +11,7 @@
 
 extern int arch_switch_tls(struct task_struct *to);
 
-void arch_switch_to(struct task_struct *to)
+void arch_switch_to(struct task_struct *from, struct task_struct *to)
 {
 	int err = arch_switch_tls(to);
 	if (!err)
diff --git a/arch/x86/um/syscalls_64.c b/arch/x86/um/syscalls_64.c
index 58f51667e2e4..2ef9474d2bd2 100644
--- a/arch/x86/um/syscalls_64.c
+++ b/arch/x86/um/syscalls_64.c
@@ -80,7 +80,7 @@ SYSCALL_DEFINE2(arch_prctl, int, option, unsigned long, arg2)
 	return arch_prctl(current, option, (unsigned long __user *) arg2);
 }
 
-void arch_switch_to(struct task_struct *to)
+void arch_switch_to(struct task_struct *from, struct task_struct *to)
 {
 	if ((to->thread.arch.fs == 0) || (to->mm == NULL))
 		return;
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v8 05/20] um: extend arch_switch_to for alternate SUBARCH
@ 2021-01-20  2:27             ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-01-20  2:27 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, thehajime, retrage01

This commit introduces additional argument of previous task when
context switch happens.  New SUBARCH can use the new information to
switch tasks in a subarch-specific manner.

The patch is particularly required by library mode, implemented as a
SUBARCH of UML.  Having access to the previous thread will be required
in the library mode for it to implement thread switching correctly.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/kernel/process.c  | 6 +++---
 arch/x86/um/ptrace_32.c   | 2 +-
 arch/x86/um/syscalls_64.c | 2 +-
 3 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/arch/um/kernel/process.c b/arch/um/kernel/process.c
index 2a986ece5478..9b0a36f64339 100644
--- a/arch/um/kernel/process.c
+++ b/arch/um/kernel/process.c
@@ -83,7 +83,7 @@ static inline void set_current(struct task_struct *task)
 		{ external_pid(), task });
 }
 
-extern void arch_switch_to(struct task_struct *to);
+extern void arch_switch_to(struct task_struct *from, struct task_struct *to);
 
 void *__switch_to(struct task_struct *from, struct task_struct *to)
 {
@@ -91,7 +91,7 @@ void *__switch_to(struct task_struct *from, struct task_struct *to)
 	set_current(to);
 
 	switch_threads(&from->thread.switch_buf, &to->thread.switch_buf);
-	arch_switch_to(current);
+	arch_switch_to(from, to);
 
 	return current->thread.prev_sched;
 }
@@ -149,7 +149,7 @@ void fork_handler(void)
 	 * arch_switch_to isn't needed. We could want to apply this to
 	 * improve performance. -bb
 	 */
-	arch_switch_to(current);
+	arch_switch_to(NULL, current);
 
 	current->thread.prev_sched = NULL;
 
diff --git a/arch/x86/um/ptrace_32.c b/arch/x86/um/ptrace_32.c
index 2497bac56066..0f184710d4ca 100644
--- a/arch/x86/um/ptrace_32.c
+++ b/arch/x86/um/ptrace_32.c
@@ -11,7 +11,7 @@
 
 extern int arch_switch_tls(struct task_struct *to);
 
-void arch_switch_to(struct task_struct *to)
+void arch_switch_to(struct task_struct *from, struct task_struct *to)
 {
 	int err = arch_switch_tls(to);
 	if (!err)
diff --git a/arch/x86/um/syscalls_64.c b/arch/x86/um/syscalls_64.c
index 58f51667e2e4..2ef9474d2bd2 100644
--- a/arch/x86/um/syscalls_64.c
+++ b/arch/x86/um/syscalls_64.c
@@ -80,7 +80,7 @@ SYSCALL_DEFINE2(arch_prctl, int, option, unsigned long, arg2)
 	return arch_prctl(current, option, (unsigned long __user *) arg2);
 }
 
-void arch_switch_to(struct task_struct *to)
+void arch_switch_to(struct task_struct *from, struct task_struct *to)
 {
 	if ((to->thread.arch.fs == 0) || (to->mm == NULL))
 		return;
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v8 06/20] um: add UML library mode
  2021-01-20  2:27           ` Hajime Tazaki
@ 2021-01-20  2:27             ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-01-20  2:27 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: thehajime, tavi.purdila, retrage01, linux-kernel-library, linux-arch

This patch introduces the library mode with UML code so that
applications can link the Linux kernel code as a reusable library.

The library mode is implemented as SUBARCH of arch/um, which places at
arch/um/lkl. This SUBARCH defines mode-specific code of memory
management as well as thread implementation, along with the uapi headers
to be exported to users.

The headers we introduce in this patch are simple wrappers to the
asm-generic headers or stubs for things we don't support, such as
ptrace, DMA, signals, ELF handling and low level processor operations.

library mode shares most of arch/um code (irq, thread_info, process
scheduling) but implements its own facilities, which are memory
management (nommu), thread primitives (struct arch_thread), system call
interface, and console.

The outlook of updated directory structure is as follows:

   arch/um
   |-- configs         (untouched)
   |-- drivers         unuse stdio_console.c for !MMU
   |-- include
   |   |-- asm         updated with !CONFIG_MMU
   |   |-- linux       (untouched)
   |   `-- shared      updated with new functions
   |       `-- skas    (untouched)
   |   `-- uapi        added for upai header installation
   |-- kernel          updated to integrate with !MMU
   |   `-- skas        (untouched, don't use for !MMU)
   |-- lkl             SUBARCH dir (internally =um/lkl)
   |   |-- include
   |   |   |-- asm     headers for subarch specific info
   |   |   `-- uapi    headers for user-visible APIs
   |   `-- um
   |       `-- shared  headers for subarch specific info
   `-- scripts         added a script for header installation

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/Kconfig                               | 20 +++-
 arch/um/lkl/Makefile                          |  2 +
 arch/um/lkl/Makefile.um                       |  2 +
 arch/um/lkl/include/asm/Kbuild                |  7 ++
 arch/um/lkl/include/asm/archparam.h           |  1 +
 arch/um/lkl/include/asm/atomic.h              | 11 +++
 arch/um/lkl/include/asm/atomic64.h            | 91 +++++++++++++++++++
 arch/um/lkl/include/asm/elf.h                 | 19 ++++
 arch/um/lkl/include/asm/mm_context.h          |  8 ++
 arch/um/lkl/include/asm/processor.h           | 48 ++++++++++
 arch/um/lkl/include/asm/ptrace.h              | 21 +++++
 arch/um/lkl/include/asm/segment.h             |  9 ++
 arch/um/lkl/include/uapi/asm/bitsperlong.h    | 13 +++
 arch/um/lkl/include/uapi/asm/byteorder.h      | 11 +++
 arch/um/lkl/include/uapi/asm/sigcontext.h     | 12 +++
 arch/um/lkl/um/Kconfig                        | 21 +++++
 arch/um/lkl/um/delay.c                        | 31 +++++++
 arch/um/lkl/um/shared/sysdep/archsetjmp.h     | 13 +++
 arch/um/lkl/um/shared/sysdep/faultinfo.h      |  8 ++
 arch/um/lkl/um/shared/sysdep/kernel-offsets.h | 12 +++
 arch/um/lkl/um/shared/sysdep/mcontext.h       |  9 ++
 arch/um/lkl/um/shared/sysdep/ptrace.h         | 42 +++++++++
 arch/um/lkl/um/shared/sysdep/ptrace_user.h    |  7 ++
 arch/um/lkl/um/unimplemented.c                | 70 ++++++++++++++
 arch/um/lkl/um/user_constants.h               | 13 +++
 25 files changed, 498 insertions(+), 3 deletions(-)
 create mode 100644 arch/um/lkl/Makefile
 create mode 100644 arch/um/lkl/Makefile.um
 create mode 100644 arch/um/lkl/include/asm/Kbuild
 create mode 100644 arch/um/lkl/include/asm/archparam.h
 create mode 100644 arch/um/lkl/include/asm/atomic.h
 create mode 100644 arch/um/lkl/include/asm/atomic64.h
 create mode 100644 arch/um/lkl/include/asm/elf.h
 create mode 100644 arch/um/lkl/include/asm/mm_context.h
 create mode 100644 arch/um/lkl/include/asm/processor.h
 create mode 100644 arch/um/lkl/include/asm/ptrace.h
 create mode 100644 arch/um/lkl/include/asm/segment.h
 create mode 100644 arch/um/lkl/include/uapi/asm/bitsperlong.h
 create mode 100644 arch/um/lkl/include/uapi/asm/byteorder.h
 create mode 100644 arch/um/lkl/include/uapi/asm/sigcontext.h
 create mode 100644 arch/um/lkl/um/Kconfig
 create mode 100644 arch/um/lkl/um/delay.c
 create mode 100644 arch/um/lkl/um/shared/sysdep/archsetjmp.h
 create mode 100644 arch/um/lkl/um/shared/sysdep/faultinfo.h
 create mode 100644 arch/um/lkl/um/shared/sysdep/kernel-offsets.h
 create mode 100644 arch/um/lkl/um/shared/sysdep/mcontext.h
 create mode 100644 arch/um/lkl/um/shared/sysdep/ptrace.h
 create mode 100644 arch/um/lkl/um/shared/sysdep/ptrace_user.h
 create mode 100644 arch/um/lkl/um/unimplemented.c
 create mode 100644 arch/um/lkl/um/user_constants.h

diff --git a/arch/um/Kconfig b/arch/um/Kconfig
index 9e2d9efbcdd1..53c7919ec5b7 100644
--- a/arch/um/Kconfig
+++ b/arch/um/Kconfig
@@ -23,9 +23,23 @@ config UML
 	select TTY # Needed for line.c
 	select MODULE_REL_CRCS if MODVERSIONS
 
+config UMMODE_LIB
+	bool "UML mode: library mode"
+	depends on !SMP
+	select UACCESS_MEMCPY
+	select ARCH_THREAD_STACK_ALLOCATOR
+	help
+	 This mode switches a mode to build a library of UML (Linux
+	 Kernel Library/LKL).  If this is Y, the build only generates,
+	 library files so, standalone executable of linux/vmlinux will
+	 not be created.
+
+	 For more detail about LKL, see
+	 <file:Documentation/virt/uml/lkl.txt>.
+
 config MMU
 	bool
-	default y
+	default y if !UMMODE_LIB
 
 config NO_IOMEM
 	def_bool y
@@ -46,12 +60,12 @@ config LOCKDEP_SUPPORT
 
 config STACKTRACE_SUPPORT
 	bool
-	default y
+	default y if !UMMODE_LIB
 	select STACKTRACE
 
 config GENERIC_CALIBRATE_DELAY
 	bool
-	default y
+	default y if !UMMODE_LIB
 
 config HZ
 	int
diff --git a/arch/um/lkl/Makefile b/arch/um/lkl/Makefile
new file mode 100644
index 000000000000..0ea94aaf5c95
--- /dev/null
+++ b/arch/um/lkl/Makefile
@@ -0,0 +1,2 @@
+# empty file to avoid `include arch/$(SRCARCH)/Makefile` error
+# at $(srctree)/Makefile
diff --git a/arch/um/lkl/Makefile.um b/arch/um/lkl/Makefile.um
new file mode 100644
index 000000000000..7d3f590d88a2
--- /dev/null
+++ b/arch/um/lkl/Makefile.um
@@ -0,0 +1,2 @@
+KBUILD_CFLAGS += -fno-builtin -fPIC
+ELF_FORMAT=$(shell $(LD) -r -print-output-format)
diff --git a/arch/um/lkl/include/asm/Kbuild b/arch/um/lkl/include/asm/Kbuild
new file mode 100644
index 000000000000..54ffd5cfa69c
--- /dev/null
+++ b/arch/um/lkl/include/asm/Kbuild
@@ -0,0 +1,7 @@
+generic-y += bitsperlong.h
+generic-y += cmpxchg.h
+generic-y += local64.h
+generic-y += seccomp.h
+generic-y += string.h
+generic-y += syscall.h
+generic-y += user.h
diff --git a/arch/um/lkl/include/asm/archparam.h b/arch/um/lkl/include/asm/archparam.h
new file mode 100644
index 000000000000..ea32a7d3cf1b
--- /dev/null
+++ b/arch/um/lkl/include/asm/archparam.h
@@ -0,0 +1 @@
+/* SPDX-License-Identifier: GPL-2.0 */
diff --git a/arch/um/lkl/include/asm/atomic.h b/arch/um/lkl/include/asm/atomic.h
new file mode 100644
index 000000000000..849148f43ef3
--- /dev/null
+++ b/arch/um/lkl/include/asm/atomic.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_LIBMODE_ATOMIC_H
+#define __UM_LIBMODE_ATOMIC_H
+
+#include <asm-generic/atomic.h>
+
+#ifndef CONFIG_GENERIC_ATOMIC64
+#include "atomic64.h"
+#endif /* !CONFIG_GENERIC_ATOMIC64 */
+
+#endif
diff --git a/arch/um/lkl/include/asm/atomic64.h b/arch/um/lkl/include/asm/atomic64.h
new file mode 100644
index 000000000000..f6157c4a6de6
--- /dev/null
+++ b/arch/um/lkl/include/asm/atomic64.h
@@ -0,0 +1,91 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_LIBMODE_ATOMIC64_H
+#define __UM_LIBMODE_ATOMIC64_H
+
+#include <linux/types.h>
+
+#define ATOMIC64_OP(op, c_op)					\
+	static inline void atomic64_##op(s64 i, atomic64_t *v)	\
+	{							\
+		__atomic_fetch_##op(&v->counter, i, __ATOMIC_RELAXED);	\
+	}
+
+#define ATOMIC64_OP_RETURN(op, c_op)					\
+	static inline s64 atomic64_##op##_return(s64 i, atomic64_t *v)	\
+	{								\
+		return __atomic_##op##_fetch(&v->counter, i, __ATOMIC_RELAXED); \
+	}
+
+#define ATOMIC64_FETCH_OP(op, c_op)					\
+	static inline s64 atomic64_fetch_##op(s64 i, atomic64_t *v)	\
+	{								\
+		return __atomic_fetch_##op(&v->counter, i, __ATOMIC_RELAXED); \
+	}
+
+#ifndef atomic64_add_return
+ATOMIC64_OP_RETURN(add, +)
+#endif
+
+#ifndef atomic64_sub_return
+	ATOMIC64_OP_RETURN(sub, -)
+#endif
+
+#ifndef atomic64_fetch_add
+	ATOMIC64_FETCH_OP(add, +)
+#endif
+
+#ifndef atomic64_fetch_sub
+	ATOMIC64_FETCH_OP(sub, -)
+#endif
+
+#ifndef atomic64_fetch_and
+	ATOMIC64_FETCH_OP(and, &)
+#endif
+
+#ifndef atomic64_fetch_or
+	ATOMIC64_FETCH_OP(or, |)
+#endif
+
+#ifndef atomic64_fetch_xor
+	ATOMIC64_FETCH_OP(xor, ^)
+#endif
+
+#ifndef atomic64_and
+	ATOMIC64_OP(and, &)
+#endif
+
+#ifndef atomic64_or
+	ATOMIC64_OP(or, |)
+#endif
+
+#ifndef atomic64_xor
+	ATOMIC64_OP(xor, ^)
+#endif
+
+#undef ATOMIC64_FETCH_OP
+#undef ATOMIC64_OP_RETURN
+#undef ATOMIC64_OP
+
+
+#define ATOMIC64_INIT(i)       { (i) }
+
+	static inline void atomic64_add(s64 i, atomic64_t *v)
+{
+	atomic64_add_return(i, v);
+}
+
+static inline void atomic64_sub(s64 i, atomic64_t *v)
+{
+	atomic64_sub_return(i, v);
+}
+
+#ifndef atomic64_read
+#define atomic64_read(v)       READ_ONCE((v)->counter)
+#endif
+
+#define atomic64_set(v, i) WRITE_ONCE(((v)->counter), (i))
+
+#define atomic64_xchg(ptr, v)          (xchg(&(ptr)->counter, (v)))
+#define atomic64_cmpxchg(v, old, new)  (cmpxchg(&((v)->counter), (old), (new)))
+
+#endif /* __LKL_ATOMIC64_H */
diff --git a/arch/um/lkl/include/asm/elf.h b/arch/um/lkl/include/asm/elf.h
new file mode 100644
index 000000000000..9716174154c7
--- /dev/null
+++ b/arch/um/lkl/include/asm/elf.h
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_LIBMODE_ELF_H
+#define __UM_LIBMODE_ELF_H
+
+/* no proper elf loader support yet */
+#define elf_check_arch(x) 0
+
+#ifdef CONFIG_64BIT
+#define ELF_CLASS ELFCLASS64
+#else
+#define ELF_CLASS ELFCLASS32
+#endif
+
+/*
+ * ELF register definitions.
+ */
+#define elf_gregset_t long
+#define elf_fpregset_t double
+#endif
diff --git a/arch/um/lkl/include/asm/mm_context.h b/arch/um/lkl/include/asm/mm_context.h
new file mode 100644
index 000000000000..b76e728bacd0
--- /dev/null
+++ b/arch/um/lkl/include/asm/mm_context.h
@@ -0,0 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_LIBMODE_MM_CONTEXT_H
+#define __UM_LIBMODE_MM_CONTEXT_H
+
+struct uml_arch_mm_context {
+};
+
+#endif
diff --git a/arch/um/lkl/include/asm/processor.h b/arch/um/lkl/include/asm/processor.h
new file mode 100644
index 000000000000..e3cab244dd81
--- /dev/null
+++ b/arch/um/lkl/include/asm/processor.h
@@ -0,0 +1,48 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_LIBMODE_PROCESSOR_H
+#define __UM_LIBMODE_PROCESSOR_H
+
+#include <asm/host_ops.h>
+
+struct alt_instr;
+
+struct arch_thread {
+	struct lkl_sem *sched_sem;
+	bool dead;
+	lkl_thread_t tid;
+	struct lkl_jmp_buf sched_jb;
+	unsigned long stackend;
+};
+
+#include <asm/ptrace-generic.h>
+#include <asm/processor-generic.h>
+
+#define INIT_ARCH_THREAD {}
+#define task_pt_regs(tsk) ((struct pt_regs *)(NULL))
+
+static inline void cpu_relax(void)
+{
+	unsigned long flags;
+
+	/* since this is usually called in a tight loop waiting for some
+	 * external condition (e.g. jiffies) lets run interrupts now to allow
+	 * the external condition to propagate
+	 */
+	local_irq_save(flags);
+	local_irq_restore(flags);
+}
+
+#define KSTK_EIP(tsk)	(0)
+#define KSTK_ESP(tsk)	(0)
+
+static inline void trap_init(void)
+{
+}
+
+static inline void arch_copy_thread(struct arch_thread *from,
+				    struct arch_thread *to)
+{
+	panic("unimplemented %s: fork isn't supported yet", __func__);
+}
+
+#endif
diff --git a/arch/um/lkl/include/asm/ptrace.h b/arch/um/lkl/include/asm/ptrace.h
new file mode 100644
index 000000000000..83f4e10fb677
--- /dev/null
+++ b/arch/um/lkl/include/asm/ptrace.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_LIBMODE_PTRACE_H
+#define __UM_LIBMODE_PTRACE_H
+
+#include <linux/errno.h>
+
+static int reg_dummy __attribute__((unused));
+
+#define PT_REGS_ORIG_SYSCALL(r) (reg_dummy)
+#define PT_REGS_SYSCALL_RET(r) (reg_dummy)
+#define PT_REGS_SET_SYSCALL_RETURN(r, res) (reg_dummy = (res))
+#define REGS_SP(r) (reg_dummy)
+
+#define user_mode(regs) 0
+#define kernel_mode(regs) 1
+#define profile_pc(regs) 0
+#define user_stack_pointer(regs) 0
+
+extern void new_thread_handler(void);
+
+#endif
diff --git a/arch/um/lkl/include/asm/segment.h b/arch/um/lkl/include/asm/segment.h
new file mode 100644
index 000000000000..1f6746866b8b
--- /dev/null
+++ b/arch/um/lkl/include/asm/segment.h
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_LIBMODE_SEGMENT_H
+#define __UM_LIBMODE_SEGMENT_H
+
+typedef struct {
+	unsigned long seg;
+} mm_segment_t;
+
+#endif /* _ASM_LKL_SEGMENT_H */
diff --git a/arch/um/lkl/include/uapi/asm/bitsperlong.h b/arch/um/lkl/include/uapi/asm/bitsperlong.h
new file mode 100644
index 000000000000..f6657ba0ff9b
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/bitsperlong.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_LIBMODE_UAPI_BITSPERLONG_H
+#define __UM_LIBMODE_UAPI_BITSPERLONG_H
+
+#ifdef CONFIG_64BIT
+#define __BITS_PER_LONG 64
+#else
+#define __BITS_PER_LONG 32
+#endif
+
+#include <asm-generic/bitsperlong.h>
+
+#endif
diff --git a/arch/um/lkl/include/uapi/asm/byteorder.h b/arch/um/lkl/include/uapi/asm/byteorder.h
new file mode 100644
index 000000000000..a4ae973f440b
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/byteorder.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_LIBMODE_UAPI_BYTEORDER_H
+#define __UM_LIBMODE_UAPI_BYTEORDER_H
+
+#if defined(CONFIG_BIG_ENDIAN)
+#include <linux/byteorder/big_endian.h>
+#else
+#include <linux/byteorder/little_endian.h>
+#endif
+
+#endif
diff --git a/arch/um/lkl/include/uapi/asm/sigcontext.h b/arch/um/lkl/include/uapi/asm/sigcontext.h
new file mode 100644
index 000000000000..9481e6e92203
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/sigcontext.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_LIBMODE_UAPI_SIGCONTEXT_H
+#define __UM_LIBMODE_UAPI_SIGCONTEXT_H
+
+#include <asm/ptrace-generic.h>
+
+struct sigcontext {
+	struct pt_regs regs;
+	unsigned long oldmask;
+};
+
+#endif
diff --git a/arch/um/lkl/um/Kconfig b/arch/um/lkl/um/Kconfig
new file mode 100644
index 000000000000..4e15bc0dd0a3
--- /dev/null
+++ b/arch/um/lkl/um/Kconfig
@@ -0,0 +1,21 @@
+config 64BIT
+	bool
+	default y
+
+config GENERIC_CSUM
+	def_bool y
+
+config GENERIC_ATOMIC64
+	bool
+	default y if !64BIT
+
+config SECCOMP
+	bool
+	default n
+
+config GENERIC_HWEIGHT
+	def_bool y
+
+config PRINTK_TIME
+	bool
+	default y
diff --git a/arch/um/lkl/um/delay.c b/arch/um/lkl/um/delay.c
new file mode 100644
index 000000000000..58a366d9b5f0
--- /dev/null
+++ b/arch/um/lkl/um/delay.c
@@ -0,0 +1,31 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/jiffies.h>
+#include <linux/delay.h>
+#include <os.h>
+
+void __ndelay(unsigned long nsecs)
+{
+	long long start = os_nsecs();
+
+	while (os_nsecs() < start + nsecs)
+		;
+}
+
+void __udelay(unsigned long usecs)
+{
+	__ndelay(usecs * NSEC_PER_USEC);
+}
+
+void __const_udelay(unsigned long xloops)
+{
+	__udelay(xloops / 0x10c7ul);
+}
+
+void __delay(unsigned long loops)
+{
+	__ndelay(loops / 5);
+}
+
+void calibrate_delay(void)
+{
+}
diff --git a/arch/um/lkl/um/shared/sysdep/archsetjmp.h b/arch/um/lkl/um/shared/sysdep/archsetjmp.h
new file mode 100644
index 000000000000..4a5c10d7521b
--- /dev/null
+++ b/arch/um/lkl/um/shared/sysdep/archsetjmp.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ARCH_UM_SETJMP_H
+#define __ARCH_UM_SETJMP_H
+
+struct __jmp_buf {
+	unsigned long __dummy;
+};
+#define JB_IP __dummy
+#define JB_SP __dummy
+
+typedef struct __jmp_buf jmp_buf[1];
+
+#endif /* __ARCH_UM_SETJMP_H */
diff --git a/arch/um/lkl/um/shared/sysdep/faultinfo.h b/arch/um/lkl/um/shared/sysdep/faultinfo.h
new file mode 100644
index 000000000000..49210742212d
--- /dev/null
+++ b/arch/um/lkl/um/shared/sysdep/faultinfo.h
@@ -0,0 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ARCH_UM_FAULTINFO_H
+#define __ARCH_UM_FAULTINFO_H
+
+struct faultinfo {
+};
+
+#endif /* __ARCH_UM_FAULTINFO_H */
diff --git a/arch/um/lkl/um/shared/sysdep/kernel-offsets.h b/arch/um/lkl/um/shared/sysdep/kernel-offsets.h
new file mode 100644
index 000000000000..a004bffb7b8d
--- /dev/null
+++ b/arch/um/lkl/um/shared/sysdep/kernel-offsets.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <linux/stddef.h>
+#include <linux/sched.h>
+#include <linux/elf.h>
+#include <linux/crypto.h>
+#include <linux/kbuild.h>
+#include <asm/mman.h>
+
+void foo(void)
+{
+#include <common-offsets.h>
+}
diff --git a/arch/um/lkl/um/shared/sysdep/mcontext.h b/arch/um/lkl/um/shared/sysdep/mcontext.h
new file mode 100644
index 000000000000..b734012a74da
--- /dev/null
+++ b/arch/um/lkl/um/shared/sysdep/mcontext.h
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ARCH_UM_MCONTEXT_H
+#define __ARCH_UM_MCONTEXT_H
+
+extern void get_regs_from_mc(struct uml_pt_regs *regs, mcontext_t *mc);
+
+#define GET_FAULTINFO_FROM_MC(fi, mc) (fi = fi)
+
+#endif /* __ARCH_UM_MCONTEXT_H */
diff --git a/arch/um/lkl/um/shared/sysdep/ptrace.h b/arch/um/lkl/um/shared/sysdep/ptrace.h
new file mode 100644
index 000000000000..bfdfb520a21d
--- /dev/null
+++ b/arch/um/lkl/um/shared/sysdep/ptrace.h
@@ -0,0 +1,42 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ARCH_UM_PTRACE_H
+#define __ARCH_UM_PTRACE_H
+
+#include <generated/user_constants.h>
+#include <linux/errno.h>
+
+struct task_struct;
+
+#define UPT_SYSCALL_NR(r) ((r)->syscall)
+#define UPT_RESTART_SYSCALL(r) ((r)->syscall--) /* XXX */
+
+#define UPT_SP(r) 0
+#define UPT_IP(r) 0
+#define EMPTY_UML_PT_REGS { }
+
+#define MAX_REG_OFFSET (UM_FRAME_SIZE)
+#define MAX_REG_NR ((MAX_REG_OFFSET) / sizeof(unsigned long))
+
+/* unused */
+struct uml_pt_regs {
+	unsigned long gp[1];
+	unsigned long fp[1];
+	long faultinfo;
+	long syscall;
+	int is_user;
+};
+
+extern void arch_init_registers(int pid);
+
+static inline long arch_ptrace(struct task_struct *child,
+			       long request, unsigned long addr,
+			       unsigned long data)
+{
+	return -EINVAL;
+}
+
+static inline void ptrace_disable(struct task_struct *child) {}
+static inline void user_enable_single_step(struct task_struct *child) {}
+static inline void user_disable_single_step(struct task_struct *child) {}
+
+#endif /* __ARCH_UM_PTRACE_H */
diff --git a/arch/um/lkl/um/shared/sysdep/ptrace_user.h b/arch/um/lkl/um/shared/sysdep/ptrace_user.h
new file mode 100644
index 000000000000..86d5cb20fb9f
--- /dev/null
+++ b/arch/um/lkl/um/shared/sysdep/ptrace_user.h
@@ -0,0 +1,7 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ARCH_UM_PTRACE_USER_H
+#define __ARCH_UM_PTRACE_USER_H
+
+#define FP_SIZE 1
+
+#endif
diff --git a/arch/um/lkl/um/unimplemented.c b/arch/um/lkl/um/unimplemented.c
new file mode 100644
index 000000000000..fe33e02e39e5
--- /dev/null
+++ b/arch/um/lkl/um/unimplemented.c
@@ -0,0 +1,70 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/signal.h>
+#include <sysdep/ptrace.h>
+#include <asm/ptrace.h>
+
+/* physmem.c  */
+unsigned long high_physmem;
+
+/* x86/um/setjmp*.S  */
+void kernel_longjmp(void)
+{}
+void kernel_setjmp(void)
+{}
+
+/* trap.c */
+void relay_signal(int sig, struct siginfo *si, struct uml_pt_regs *regs)
+{}
+void bus_handler(int sig, struct siginfo *si, struct uml_pt_regs *regs)
+{}
+void segv_handler(int sig, struct siginfo *unused_si, struct uml_pt_regs *regs)
+{}
+void winch(int sig, struct siginfo *unused_si, struct uml_pt_regs *regs)
+{}
+
+/* tlb.c */
+void flush_tlb_kernel_vm(void)
+{}
+void force_flush_all(void)
+{}
+
+/* skas/process.c */
+void halt_skas(void)
+{}
+int is_skas_winch(int pid, int fd, void *data)
+{
+	return 0;
+}
+void reboot_skas(void)
+{}
+
+int __init start_uml(void)
+{
+	return 0;
+}
+
+/* exec.c */
+void flush_thread(void)
+{}
+
+/* x86/ptrace_64.c */
+int is_syscall(unsigned long addr)
+{
+	return 0;
+}
+
+
+/* x86/sysrq.c */
+void show_regs(struct pt_regs *regs)
+{}
+
+/* x86/signal.c */
+int setup_signal_stack_si(unsigned long stack_top, struct ksignal *ksig,
+			  struct pt_regs *regs, sigset_t *mask)
+{
+	return 0;
+}
+
+/* x86/bugs.c */
+void arch_check_bugs(void)
+{}
diff --git a/arch/um/lkl/um/user_constants.h b/arch/um/lkl/um/user_constants.h
new file mode 100644
index 000000000000..2eb692a86c9c
--- /dev/null
+++ b/arch/um/lkl/um/user_constants.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_LIBMODE_USER_CONSTANTS_H
+#define __UM_LIBMODE_USER_CONSTANTS_H
+
+/* XXX: put dummy values */
+#define UM_FRAME_SIZE 4
+#define HOST_FP_SIZE 1
+#define HOST_IP 1
+#define HOST_SP 2
+#define HOST_BP 3
+#define UM_NR_CPUS 1
+
+#endif
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v8 06/20] um: add UML library mode
@ 2021-01-20  2:27             ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-01-20  2:27 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, thehajime, retrage01

This patch introduces the library mode with UML code so that
applications can link the Linux kernel code as a reusable library.

The library mode is implemented as SUBARCH of arch/um, which places at
arch/um/lkl. This SUBARCH defines mode-specific code of memory
management as well as thread implementation, along with the uapi headers
to be exported to users.

The headers we introduce in this patch are simple wrappers to the
asm-generic headers or stubs for things we don't support, such as
ptrace, DMA, signals, ELF handling and low level processor operations.

library mode shares most of arch/um code (irq, thread_info, process
scheduling) but implements its own facilities, which are memory
management (nommu), thread primitives (struct arch_thread), system call
interface, and console.

The outlook of updated directory structure is as follows:

   arch/um
   |-- configs         (untouched)
   |-- drivers         unuse stdio_console.c for !MMU
   |-- include
   |   |-- asm         updated with !CONFIG_MMU
   |   |-- linux       (untouched)
   |   `-- shared      updated with new functions
   |       `-- skas    (untouched)
   |   `-- uapi        added for upai header installation
   |-- kernel          updated to integrate with !MMU
   |   `-- skas        (untouched, don't use for !MMU)
   |-- lkl             SUBARCH dir (internally =um/lkl)
   |   |-- include
   |   |   |-- asm     headers for subarch specific info
   |   |   `-- uapi    headers for user-visible APIs
   |   `-- um
   |       `-- shared  headers for subarch specific info
   `-- scripts         added a script for header installation

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/Kconfig                               | 20 +++-
 arch/um/lkl/Makefile                          |  2 +
 arch/um/lkl/Makefile.um                       |  2 +
 arch/um/lkl/include/asm/Kbuild                |  7 ++
 arch/um/lkl/include/asm/archparam.h           |  1 +
 arch/um/lkl/include/asm/atomic.h              | 11 +++
 arch/um/lkl/include/asm/atomic64.h            | 91 +++++++++++++++++++
 arch/um/lkl/include/asm/elf.h                 | 19 ++++
 arch/um/lkl/include/asm/mm_context.h          |  8 ++
 arch/um/lkl/include/asm/processor.h           | 48 ++++++++++
 arch/um/lkl/include/asm/ptrace.h              | 21 +++++
 arch/um/lkl/include/asm/segment.h             |  9 ++
 arch/um/lkl/include/uapi/asm/bitsperlong.h    | 13 +++
 arch/um/lkl/include/uapi/asm/byteorder.h      | 11 +++
 arch/um/lkl/include/uapi/asm/sigcontext.h     | 12 +++
 arch/um/lkl/um/Kconfig                        | 21 +++++
 arch/um/lkl/um/delay.c                        | 31 +++++++
 arch/um/lkl/um/shared/sysdep/archsetjmp.h     | 13 +++
 arch/um/lkl/um/shared/sysdep/faultinfo.h      |  8 ++
 arch/um/lkl/um/shared/sysdep/kernel-offsets.h | 12 +++
 arch/um/lkl/um/shared/sysdep/mcontext.h       |  9 ++
 arch/um/lkl/um/shared/sysdep/ptrace.h         | 42 +++++++++
 arch/um/lkl/um/shared/sysdep/ptrace_user.h    |  7 ++
 arch/um/lkl/um/unimplemented.c                | 70 ++++++++++++++
 arch/um/lkl/um/user_constants.h               | 13 +++
 25 files changed, 498 insertions(+), 3 deletions(-)
 create mode 100644 arch/um/lkl/Makefile
 create mode 100644 arch/um/lkl/Makefile.um
 create mode 100644 arch/um/lkl/include/asm/Kbuild
 create mode 100644 arch/um/lkl/include/asm/archparam.h
 create mode 100644 arch/um/lkl/include/asm/atomic.h
 create mode 100644 arch/um/lkl/include/asm/atomic64.h
 create mode 100644 arch/um/lkl/include/asm/elf.h
 create mode 100644 arch/um/lkl/include/asm/mm_context.h
 create mode 100644 arch/um/lkl/include/asm/processor.h
 create mode 100644 arch/um/lkl/include/asm/ptrace.h
 create mode 100644 arch/um/lkl/include/asm/segment.h
 create mode 100644 arch/um/lkl/include/uapi/asm/bitsperlong.h
 create mode 100644 arch/um/lkl/include/uapi/asm/byteorder.h
 create mode 100644 arch/um/lkl/include/uapi/asm/sigcontext.h
 create mode 100644 arch/um/lkl/um/Kconfig
 create mode 100644 arch/um/lkl/um/delay.c
 create mode 100644 arch/um/lkl/um/shared/sysdep/archsetjmp.h
 create mode 100644 arch/um/lkl/um/shared/sysdep/faultinfo.h
 create mode 100644 arch/um/lkl/um/shared/sysdep/kernel-offsets.h
 create mode 100644 arch/um/lkl/um/shared/sysdep/mcontext.h
 create mode 100644 arch/um/lkl/um/shared/sysdep/ptrace.h
 create mode 100644 arch/um/lkl/um/shared/sysdep/ptrace_user.h
 create mode 100644 arch/um/lkl/um/unimplemented.c
 create mode 100644 arch/um/lkl/um/user_constants.h

diff --git a/arch/um/Kconfig b/arch/um/Kconfig
index 9e2d9efbcdd1..53c7919ec5b7 100644
--- a/arch/um/Kconfig
+++ b/arch/um/Kconfig
@@ -23,9 +23,23 @@ config UML
 	select TTY # Needed for line.c
 	select MODULE_REL_CRCS if MODVERSIONS
 
+config UMMODE_LIB
+	bool "UML mode: library mode"
+	depends on !SMP
+	select UACCESS_MEMCPY
+	select ARCH_THREAD_STACK_ALLOCATOR
+	help
+	 This mode switches a mode to build a library of UML (Linux
+	 Kernel Library/LKL).  If this is Y, the build only generates,
+	 library files so, standalone executable of linux/vmlinux will
+	 not be created.
+
+	 For more detail about LKL, see
+	 <file:Documentation/virt/uml/lkl.txt>.
+
 config MMU
 	bool
-	default y
+	default y if !UMMODE_LIB
 
 config NO_IOMEM
 	def_bool y
@@ -46,12 +60,12 @@ config LOCKDEP_SUPPORT
 
 config STACKTRACE_SUPPORT
 	bool
-	default y
+	default y if !UMMODE_LIB
 	select STACKTRACE
 
 config GENERIC_CALIBRATE_DELAY
 	bool
-	default y
+	default y if !UMMODE_LIB
 
 config HZ
 	int
diff --git a/arch/um/lkl/Makefile b/arch/um/lkl/Makefile
new file mode 100644
index 000000000000..0ea94aaf5c95
--- /dev/null
+++ b/arch/um/lkl/Makefile
@@ -0,0 +1,2 @@
+# empty file to avoid `include arch/$(SRCARCH)/Makefile` error
+# at $(srctree)/Makefile
diff --git a/arch/um/lkl/Makefile.um b/arch/um/lkl/Makefile.um
new file mode 100644
index 000000000000..7d3f590d88a2
--- /dev/null
+++ b/arch/um/lkl/Makefile.um
@@ -0,0 +1,2 @@
+KBUILD_CFLAGS += -fno-builtin -fPIC
+ELF_FORMAT=$(shell $(LD) -r -print-output-format)
diff --git a/arch/um/lkl/include/asm/Kbuild b/arch/um/lkl/include/asm/Kbuild
new file mode 100644
index 000000000000..54ffd5cfa69c
--- /dev/null
+++ b/arch/um/lkl/include/asm/Kbuild
@@ -0,0 +1,7 @@
+generic-y += bitsperlong.h
+generic-y += cmpxchg.h
+generic-y += local64.h
+generic-y += seccomp.h
+generic-y += string.h
+generic-y += syscall.h
+generic-y += user.h
diff --git a/arch/um/lkl/include/asm/archparam.h b/arch/um/lkl/include/asm/archparam.h
new file mode 100644
index 000000000000..ea32a7d3cf1b
--- /dev/null
+++ b/arch/um/lkl/include/asm/archparam.h
@@ -0,0 +1 @@
+/* SPDX-License-Identifier: GPL-2.0 */
diff --git a/arch/um/lkl/include/asm/atomic.h b/arch/um/lkl/include/asm/atomic.h
new file mode 100644
index 000000000000..849148f43ef3
--- /dev/null
+++ b/arch/um/lkl/include/asm/atomic.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_LIBMODE_ATOMIC_H
+#define __UM_LIBMODE_ATOMIC_H
+
+#include <asm-generic/atomic.h>
+
+#ifndef CONFIG_GENERIC_ATOMIC64
+#include "atomic64.h"
+#endif /* !CONFIG_GENERIC_ATOMIC64 */
+
+#endif
diff --git a/arch/um/lkl/include/asm/atomic64.h b/arch/um/lkl/include/asm/atomic64.h
new file mode 100644
index 000000000000..f6157c4a6de6
--- /dev/null
+++ b/arch/um/lkl/include/asm/atomic64.h
@@ -0,0 +1,91 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_LIBMODE_ATOMIC64_H
+#define __UM_LIBMODE_ATOMIC64_H
+
+#include <linux/types.h>
+
+#define ATOMIC64_OP(op, c_op)					\
+	static inline void atomic64_##op(s64 i, atomic64_t *v)	\
+	{							\
+		__atomic_fetch_##op(&v->counter, i, __ATOMIC_RELAXED);	\
+	}
+
+#define ATOMIC64_OP_RETURN(op, c_op)					\
+	static inline s64 atomic64_##op##_return(s64 i, atomic64_t *v)	\
+	{								\
+		return __atomic_##op##_fetch(&v->counter, i, __ATOMIC_RELAXED); \
+	}
+
+#define ATOMIC64_FETCH_OP(op, c_op)					\
+	static inline s64 atomic64_fetch_##op(s64 i, atomic64_t *v)	\
+	{								\
+		return __atomic_fetch_##op(&v->counter, i, __ATOMIC_RELAXED); \
+	}
+
+#ifndef atomic64_add_return
+ATOMIC64_OP_RETURN(add, +)
+#endif
+
+#ifndef atomic64_sub_return
+	ATOMIC64_OP_RETURN(sub, -)
+#endif
+
+#ifndef atomic64_fetch_add
+	ATOMIC64_FETCH_OP(add, +)
+#endif
+
+#ifndef atomic64_fetch_sub
+	ATOMIC64_FETCH_OP(sub, -)
+#endif
+
+#ifndef atomic64_fetch_and
+	ATOMIC64_FETCH_OP(and, &)
+#endif
+
+#ifndef atomic64_fetch_or
+	ATOMIC64_FETCH_OP(or, |)
+#endif
+
+#ifndef atomic64_fetch_xor
+	ATOMIC64_FETCH_OP(xor, ^)
+#endif
+
+#ifndef atomic64_and
+	ATOMIC64_OP(and, &)
+#endif
+
+#ifndef atomic64_or
+	ATOMIC64_OP(or, |)
+#endif
+
+#ifndef atomic64_xor
+	ATOMIC64_OP(xor, ^)
+#endif
+
+#undef ATOMIC64_FETCH_OP
+#undef ATOMIC64_OP_RETURN
+#undef ATOMIC64_OP
+
+
+#define ATOMIC64_INIT(i)       { (i) }
+
+	static inline void atomic64_add(s64 i, atomic64_t *v)
+{
+	atomic64_add_return(i, v);
+}
+
+static inline void atomic64_sub(s64 i, atomic64_t *v)
+{
+	atomic64_sub_return(i, v);
+}
+
+#ifndef atomic64_read
+#define atomic64_read(v)       READ_ONCE((v)->counter)
+#endif
+
+#define atomic64_set(v, i) WRITE_ONCE(((v)->counter), (i))
+
+#define atomic64_xchg(ptr, v)          (xchg(&(ptr)->counter, (v)))
+#define atomic64_cmpxchg(v, old, new)  (cmpxchg(&((v)->counter), (old), (new)))
+
+#endif /* __LKL_ATOMIC64_H */
diff --git a/arch/um/lkl/include/asm/elf.h b/arch/um/lkl/include/asm/elf.h
new file mode 100644
index 000000000000..9716174154c7
--- /dev/null
+++ b/arch/um/lkl/include/asm/elf.h
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_LIBMODE_ELF_H
+#define __UM_LIBMODE_ELF_H
+
+/* no proper elf loader support yet */
+#define elf_check_arch(x) 0
+
+#ifdef CONFIG_64BIT
+#define ELF_CLASS ELFCLASS64
+#else
+#define ELF_CLASS ELFCLASS32
+#endif
+
+/*
+ * ELF register definitions.
+ */
+#define elf_gregset_t long
+#define elf_fpregset_t double
+#endif
diff --git a/arch/um/lkl/include/asm/mm_context.h b/arch/um/lkl/include/asm/mm_context.h
new file mode 100644
index 000000000000..b76e728bacd0
--- /dev/null
+++ b/arch/um/lkl/include/asm/mm_context.h
@@ -0,0 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_LIBMODE_MM_CONTEXT_H
+#define __UM_LIBMODE_MM_CONTEXT_H
+
+struct uml_arch_mm_context {
+};
+
+#endif
diff --git a/arch/um/lkl/include/asm/processor.h b/arch/um/lkl/include/asm/processor.h
new file mode 100644
index 000000000000..e3cab244dd81
--- /dev/null
+++ b/arch/um/lkl/include/asm/processor.h
@@ -0,0 +1,48 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_LIBMODE_PROCESSOR_H
+#define __UM_LIBMODE_PROCESSOR_H
+
+#include <asm/host_ops.h>
+
+struct alt_instr;
+
+struct arch_thread {
+	struct lkl_sem *sched_sem;
+	bool dead;
+	lkl_thread_t tid;
+	struct lkl_jmp_buf sched_jb;
+	unsigned long stackend;
+};
+
+#include <asm/ptrace-generic.h>
+#include <asm/processor-generic.h>
+
+#define INIT_ARCH_THREAD {}
+#define task_pt_regs(tsk) ((struct pt_regs *)(NULL))
+
+static inline void cpu_relax(void)
+{
+	unsigned long flags;
+
+	/* since this is usually called in a tight loop waiting for some
+	 * external condition (e.g. jiffies) lets run interrupts now to allow
+	 * the external condition to propagate
+	 */
+	local_irq_save(flags);
+	local_irq_restore(flags);
+}
+
+#define KSTK_EIP(tsk)	(0)
+#define KSTK_ESP(tsk)	(0)
+
+static inline void trap_init(void)
+{
+}
+
+static inline void arch_copy_thread(struct arch_thread *from,
+				    struct arch_thread *to)
+{
+	panic("unimplemented %s: fork isn't supported yet", __func__);
+}
+
+#endif
diff --git a/arch/um/lkl/include/asm/ptrace.h b/arch/um/lkl/include/asm/ptrace.h
new file mode 100644
index 000000000000..83f4e10fb677
--- /dev/null
+++ b/arch/um/lkl/include/asm/ptrace.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_LIBMODE_PTRACE_H
+#define __UM_LIBMODE_PTRACE_H
+
+#include <linux/errno.h>
+
+static int reg_dummy __attribute__((unused));
+
+#define PT_REGS_ORIG_SYSCALL(r) (reg_dummy)
+#define PT_REGS_SYSCALL_RET(r) (reg_dummy)
+#define PT_REGS_SET_SYSCALL_RETURN(r, res) (reg_dummy = (res))
+#define REGS_SP(r) (reg_dummy)
+
+#define user_mode(regs) 0
+#define kernel_mode(regs) 1
+#define profile_pc(regs) 0
+#define user_stack_pointer(regs) 0
+
+extern void new_thread_handler(void);
+
+#endif
diff --git a/arch/um/lkl/include/asm/segment.h b/arch/um/lkl/include/asm/segment.h
new file mode 100644
index 000000000000..1f6746866b8b
--- /dev/null
+++ b/arch/um/lkl/include/asm/segment.h
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_LIBMODE_SEGMENT_H
+#define __UM_LIBMODE_SEGMENT_H
+
+typedef struct {
+	unsigned long seg;
+} mm_segment_t;
+
+#endif /* _ASM_LKL_SEGMENT_H */
diff --git a/arch/um/lkl/include/uapi/asm/bitsperlong.h b/arch/um/lkl/include/uapi/asm/bitsperlong.h
new file mode 100644
index 000000000000..f6657ba0ff9b
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/bitsperlong.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_LIBMODE_UAPI_BITSPERLONG_H
+#define __UM_LIBMODE_UAPI_BITSPERLONG_H
+
+#ifdef CONFIG_64BIT
+#define __BITS_PER_LONG 64
+#else
+#define __BITS_PER_LONG 32
+#endif
+
+#include <asm-generic/bitsperlong.h>
+
+#endif
diff --git a/arch/um/lkl/include/uapi/asm/byteorder.h b/arch/um/lkl/include/uapi/asm/byteorder.h
new file mode 100644
index 000000000000..a4ae973f440b
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/byteorder.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_LIBMODE_UAPI_BYTEORDER_H
+#define __UM_LIBMODE_UAPI_BYTEORDER_H
+
+#if defined(CONFIG_BIG_ENDIAN)
+#include <linux/byteorder/big_endian.h>
+#else
+#include <linux/byteorder/little_endian.h>
+#endif
+
+#endif
diff --git a/arch/um/lkl/include/uapi/asm/sigcontext.h b/arch/um/lkl/include/uapi/asm/sigcontext.h
new file mode 100644
index 000000000000..9481e6e92203
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/sigcontext.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_LIBMODE_UAPI_SIGCONTEXT_H
+#define __UM_LIBMODE_UAPI_SIGCONTEXT_H
+
+#include <asm/ptrace-generic.h>
+
+struct sigcontext {
+	struct pt_regs regs;
+	unsigned long oldmask;
+};
+
+#endif
diff --git a/arch/um/lkl/um/Kconfig b/arch/um/lkl/um/Kconfig
new file mode 100644
index 000000000000..4e15bc0dd0a3
--- /dev/null
+++ b/arch/um/lkl/um/Kconfig
@@ -0,0 +1,21 @@
+config 64BIT
+	bool
+	default y
+
+config GENERIC_CSUM
+	def_bool y
+
+config GENERIC_ATOMIC64
+	bool
+	default y if !64BIT
+
+config SECCOMP
+	bool
+	default n
+
+config GENERIC_HWEIGHT
+	def_bool y
+
+config PRINTK_TIME
+	bool
+	default y
diff --git a/arch/um/lkl/um/delay.c b/arch/um/lkl/um/delay.c
new file mode 100644
index 000000000000..58a366d9b5f0
--- /dev/null
+++ b/arch/um/lkl/um/delay.c
@@ -0,0 +1,31 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/jiffies.h>
+#include <linux/delay.h>
+#include <os.h>
+
+void __ndelay(unsigned long nsecs)
+{
+	long long start = os_nsecs();
+
+	while (os_nsecs() < start + nsecs)
+		;
+}
+
+void __udelay(unsigned long usecs)
+{
+	__ndelay(usecs * NSEC_PER_USEC);
+}
+
+void __const_udelay(unsigned long xloops)
+{
+	__udelay(xloops / 0x10c7ul);
+}
+
+void __delay(unsigned long loops)
+{
+	__ndelay(loops / 5);
+}
+
+void calibrate_delay(void)
+{
+}
diff --git a/arch/um/lkl/um/shared/sysdep/archsetjmp.h b/arch/um/lkl/um/shared/sysdep/archsetjmp.h
new file mode 100644
index 000000000000..4a5c10d7521b
--- /dev/null
+++ b/arch/um/lkl/um/shared/sysdep/archsetjmp.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ARCH_UM_SETJMP_H
+#define __ARCH_UM_SETJMP_H
+
+struct __jmp_buf {
+	unsigned long __dummy;
+};
+#define JB_IP __dummy
+#define JB_SP __dummy
+
+typedef struct __jmp_buf jmp_buf[1];
+
+#endif /* __ARCH_UM_SETJMP_H */
diff --git a/arch/um/lkl/um/shared/sysdep/faultinfo.h b/arch/um/lkl/um/shared/sysdep/faultinfo.h
new file mode 100644
index 000000000000..49210742212d
--- /dev/null
+++ b/arch/um/lkl/um/shared/sysdep/faultinfo.h
@@ -0,0 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ARCH_UM_FAULTINFO_H
+#define __ARCH_UM_FAULTINFO_H
+
+struct faultinfo {
+};
+
+#endif /* __ARCH_UM_FAULTINFO_H */
diff --git a/arch/um/lkl/um/shared/sysdep/kernel-offsets.h b/arch/um/lkl/um/shared/sysdep/kernel-offsets.h
new file mode 100644
index 000000000000..a004bffb7b8d
--- /dev/null
+++ b/arch/um/lkl/um/shared/sysdep/kernel-offsets.h
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <linux/stddef.h>
+#include <linux/sched.h>
+#include <linux/elf.h>
+#include <linux/crypto.h>
+#include <linux/kbuild.h>
+#include <asm/mman.h>
+
+void foo(void)
+{
+#include <common-offsets.h>
+}
diff --git a/arch/um/lkl/um/shared/sysdep/mcontext.h b/arch/um/lkl/um/shared/sysdep/mcontext.h
new file mode 100644
index 000000000000..b734012a74da
--- /dev/null
+++ b/arch/um/lkl/um/shared/sysdep/mcontext.h
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ARCH_UM_MCONTEXT_H
+#define __ARCH_UM_MCONTEXT_H
+
+extern void get_regs_from_mc(struct uml_pt_regs *regs, mcontext_t *mc);
+
+#define GET_FAULTINFO_FROM_MC(fi, mc) (fi = fi)
+
+#endif /* __ARCH_UM_MCONTEXT_H */
diff --git a/arch/um/lkl/um/shared/sysdep/ptrace.h b/arch/um/lkl/um/shared/sysdep/ptrace.h
new file mode 100644
index 000000000000..bfdfb520a21d
--- /dev/null
+++ b/arch/um/lkl/um/shared/sysdep/ptrace.h
@@ -0,0 +1,42 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ARCH_UM_PTRACE_H
+#define __ARCH_UM_PTRACE_H
+
+#include <generated/user_constants.h>
+#include <linux/errno.h>
+
+struct task_struct;
+
+#define UPT_SYSCALL_NR(r) ((r)->syscall)
+#define UPT_RESTART_SYSCALL(r) ((r)->syscall--) /* XXX */
+
+#define UPT_SP(r) 0
+#define UPT_IP(r) 0
+#define EMPTY_UML_PT_REGS { }
+
+#define MAX_REG_OFFSET (UM_FRAME_SIZE)
+#define MAX_REG_NR ((MAX_REG_OFFSET) / sizeof(unsigned long))
+
+/* unused */
+struct uml_pt_regs {
+	unsigned long gp[1];
+	unsigned long fp[1];
+	long faultinfo;
+	long syscall;
+	int is_user;
+};
+
+extern void arch_init_registers(int pid);
+
+static inline long arch_ptrace(struct task_struct *child,
+			       long request, unsigned long addr,
+			       unsigned long data)
+{
+	return -EINVAL;
+}
+
+static inline void ptrace_disable(struct task_struct *child) {}
+static inline void user_enable_single_step(struct task_struct *child) {}
+static inline void user_disable_single_step(struct task_struct *child) {}
+
+#endif /* __ARCH_UM_PTRACE_H */
diff --git a/arch/um/lkl/um/shared/sysdep/ptrace_user.h b/arch/um/lkl/um/shared/sysdep/ptrace_user.h
new file mode 100644
index 000000000000..86d5cb20fb9f
--- /dev/null
+++ b/arch/um/lkl/um/shared/sysdep/ptrace_user.h
@@ -0,0 +1,7 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ARCH_UM_PTRACE_USER_H
+#define __ARCH_UM_PTRACE_USER_H
+
+#define FP_SIZE 1
+
+#endif
diff --git a/arch/um/lkl/um/unimplemented.c b/arch/um/lkl/um/unimplemented.c
new file mode 100644
index 000000000000..fe33e02e39e5
--- /dev/null
+++ b/arch/um/lkl/um/unimplemented.c
@@ -0,0 +1,70 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/signal.h>
+#include <sysdep/ptrace.h>
+#include <asm/ptrace.h>
+
+/* physmem.c  */
+unsigned long high_physmem;
+
+/* x86/um/setjmp*.S  */
+void kernel_longjmp(void)
+{}
+void kernel_setjmp(void)
+{}
+
+/* trap.c */
+void relay_signal(int sig, struct siginfo *si, struct uml_pt_regs *regs)
+{}
+void bus_handler(int sig, struct siginfo *si, struct uml_pt_regs *regs)
+{}
+void segv_handler(int sig, struct siginfo *unused_si, struct uml_pt_regs *regs)
+{}
+void winch(int sig, struct siginfo *unused_si, struct uml_pt_regs *regs)
+{}
+
+/* tlb.c */
+void flush_tlb_kernel_vm(void)
+{}
+void force_flush_all(void)
+{}
+
+/* skas/process.c */
+void halt_skas(void)
+{}
+int is_skas_winch(int pid, int fd, void *data)
+{
+	return 0;
+}
+void reboot_skas(void)
+{}
+
+int __init start_uml(void)
+{
+	return 0;
+}
+
+/* exec.c */
+void flush_thread(void)
+{}
+
+/* x86/ptrace_64.c */
+int is_syscall(unsigned long addr)
+{
+	return 0;
+}
+
+
+/* x86/sysrq.c */
+void show_regs(struct pt_regs *regs)
+{}
+
+/* x86/signal.c */
+int setup_signal_stack_si(unsigned long stack_top, struct ksignal *ksig,
+			  struct pt_regs *regs, sigset_t *mask)
+{
+	return 0;
+}
+
+/* x86/bugs.c */
+void arch_check_bugs(void)
+{}
diff --git a/arch/um/lkl/um/user_constants.h b/arch/um/lkl/um/user_constants.h
new file mode 100644
index 000000000000..2eb692a86c9c
--- /dev/null
+++ b/arch/um/lkl/um/user_constants.h
@@ -0,0 +1,13 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_LIBMODE_USER_CONSTANTS_H
+#define __UM_LIBMODE_USER_CONSTANTS_H
+
+/* XXX: put dummy values */
+#define UM_FRAME_SIZE 4
+#define HOST_FP_SIZE 1
+#define HOST_IP 1
+#define HOST_SP 2
+#define HOST_BP 3
+#define UM_NR_CPUS 1
+
+#endif
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v8 07/20] um: lkl: host interface
  2021-01-20  2:27           ` Hajime Tazaki
@ 2021-01-20  2:27             ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-01-20  2:27 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: thehajime, tavi.purdila, retrage01, linux-kernel-library, linux-arch

This patch introduces the host operations that define the interface
between the LKL and the host. These operations must be provided either
by a host library or by the application itself.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/include/asm/host_ops.h          |  9 +++++++++
 arch/um/lkl/include/uapi/asm/host_ops.h | 23 +++++++++++++++++++++++
 2 files changed, 32 insertions(+)
 create mode 100644 arch/um/include/asm/host_ops.h
 create mode 100644 arch/um/lkl/include/uapi/asm/host_ops.h

diff --git a/arch/um/include/asm/host_ops.h b/arch/um/include/asm/host_ops.h
new file mode 100644
index 000000000000..f52423cc4ced
--- /dev/null
+++ b/arch/um/include/asm/host_ops.h
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_HOST_OPS_H
+#define _ASM_LKL_HOST_OPS_H
+
+#include <uapi/asm/host_ops.h>
+
+extern struct lkl_host_operations *lkl_ops;
+
+#endif
diff --git a/arch/um/lkl/include/uapi/asm/host_ops.h b/arch/um/lkl/include/uapi/asm/host_ops.h
new file mode 100644
index 000000000000..5d141784541d
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/host_ops.h
@@ -0,0 +1,23 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_LIBMODE_UAPI_HOST_OPS_H
+#define __UM_LIBMODE_UAPI_HOST_OPS_H
+
+/**
+ * struct lkl_host_operations - host operations used by the Linux kernel
+ *
+ * These operations must be provided by a host library or by the application
+ * itself.
+ *
+ */
+struct lkl_host_operations {
+};
+
+/**
+ * lkl_bug - call lkl_panic with a message
+ *
+ * @fmt: message format with parameters
+ *
+ */
+void lkl_bug(const char *fmt, ...);
+
+#endif
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v8 07/20] um: lkl: host interface
@ 2021-01-20  2:27             ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-01-20  2:27 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, thehajime, retrage01

This patch introduces the host operations that define the interface
between the LKL and the host. These operations must be provided either
by a host library or by the application itself.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/include/asm/host_ops.h          |  9 +++++++++
 arch/um/lkl/include/uapi/asm/host_ops.h | 23 +++++++++++++++++++++++
 2 files changed, 32 insertions(+)
 create mode 100644 arch/um/include/asm/host_ops.h
 create mode 100644 arch/um/lkl/include/uapi/asm/host_ops.h

diff --git a/arch/um/include/asm/host_ops.h b/arch/um/include/asm/host_ops.h
new file mode 100644
index 000000000000..f52423cc4ced
--- /dev/null
+++ b/arch/um/include/asm/host_ops.h
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_LKL_HOST_OPS_H
+#define _ASM_LKL_HOST_OPS_H
+
+#include <uapi/asm/host_ops.h>
+
+extern struct lkl_host_operations *lkl_ops;
+
+#endif
diff --git a/arch/um/lkl/include/uapi/asm/host_ops.h b/arch/um/lkl/include/uapi/asm/host_ops.h
new file mode 100644
index 000000000000..5d141784541d
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/host_ops.h
@@ -0,0 +1,23 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_LIBMODE_UAPI_HOST_OPS_H
+#define __UM_LIBMODE_UAPI_HOST_OPS_H
+
+/**
+ * struct lkl_host_operations - host operations used by the Linux kernel
+ *
+ * These operations must be provided by a host library or by the application
+ * itself.
+ *
+ */
+struct lkl_host_operations {
+};
+
+/**
+ * lkl_bug - call lkl_panic with a message
+ *
+ * @fmt: message format with parameters
+ *
+ */
+void lkl_bug(const char *fmt, ...);
+
+#endif
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v8 08/20] um: lkl: memory handling
  2021-01-20  2:27           ` Hajime Tazaki
@ 2021-01-20  2:27             ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-01-20  2:27 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: thehajime, tavi.purdila, retrage01, linux-kernel-library, linux-arch

library mode follows the way !CONFIG_MMU architecture of other linux
archs.  There is not much work left to do other than initializing the
boot allocator and providing the page and page table definitions.

The backstore memory is allocated via host functions and the memory
size to be used is specified when the kernel is started, in the
lkl_start_kernel call.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/include/asm/mmu.h               |   3 +
 arch/um/include/asm/mmu_context.h       |  10 +++
 arch/um/include/asm/page.h              |  15 ++++
 arch/um/include/asm/pgtable.h           |  27 ++++++
 arch/um/include/asm/uaccess.h           |   6 ++
 arch/um/lkl/include/uapi/asm/host_ops.h |  16 ++++
 arch/um/lkl/um/bootmem.c                | 107 ++++++++++++++++++++++++
 7 files changed, 184 insertions(+)
 create mode 100644 arch/um/lkl/um/bootmem.c

diff --git a/arch/um/include/asm/mmu.h b/arch/um/include/asm/mmu.h
index 5b072aba5b65..c06d6cb67dd7 100644
--- a/arch/um/include/asm/mmu.h
+++ b/arch/um/include/asm/mmu.h
@@ -13,6 +13,9 @@ typedef struct mm_context {
 	struct mm_id id;
 	struct uml_arch_mm_context arch;
 	struct page *stub_pages[2];
+#ifndef CONFIG_MMU
+	unsigned long		end_brk;
+#endif
 } mm_context_t;
 
 extern void __switch_mm(struct mm_id * mm_idp);
diff --git a/arch/um/include/asm/mmu_context.h b/arch/um/include/asm/mmu_context.h
index f8a100770691..4bf627feaa0c 100644
--- a/arch/um/include/asm/mmu_context.h
+++ b/arch/um/include/asm/mmu_context.h
@@ -6,6 +6,7 @@
 #ifndef __UM_MMU_CONTEXT_H
 #define __UM_MMU_CONTEXT_H
 
+#ifdef CONFIG_MMU
 #include <linux/sched.h>
 #include <linux/mm_types.h>
 #include <linux/mmap_lock.h>
@@ -73,4 +74,13 @@ extern void destroy_context(struct mm_struct *mm);
 
 #include <asm-generic/mmu_context.h>
 
+#else
+
+#include <asm-generic/nommu_context.h>
+
+extern void force_flush_all(void);
+
+#endif /* CONFIG_MMU */
+
+
 #endif
diff --git a/arch/um/include/asm/page.h b/arch/um/include/asm/page.h
index 95af12e82a32..c9d5e487ac6b 100644
--- a/arch/um/include/asm/page.h
+++ b/arch/um/include/asm/page.h
@@ -7,6 +7,8 @@
 #ifndef __UM_PAGE_H
 #define __UM_PAGE_H
 
+#ifdef CONFIG_MMU
+
 #include <linux/const.h>
 
 /* PAGE_SHIFT determines the page size */
@@ -120,4 +122,17 @@ extern unsigned long uml_physmem;
 #define __HAVE_ARCH_GATE_AREA 1
 #endif
 
+#else  /* CONFIG_MMU */
+#define CONFIG_KERNEL_RAM_BASE_ADDRESS memory_start
+#include <asm-generic/page.h>
+
+#define __va_space (8*1024*1024)
+
+#ifndef __ASSEMBLY__
+#include <mem.h>
+void free_mem(void);
+#endif
+
+#endif /* !CONFIG_MMU  */
+
 #endif	/* __UM_PAGE_H */
diff --git a/arch/um/include/asm/pgtable.h b/arch/um/include/asm/pgtable.h
index 39376bb63abf..58638d888b95 100644
--- a/arch/um/include/asm/pgtable.h
+++ b/arch/um/include/asm/pgtable.h
@@ -21,6 +21,8 @@
 #define _PAGE_PROTNONE	0x010	/* if the user mapped it with PROT_NONE;
 				   pte_present gives true */
 
+#ifdef CONFIG_MMU
+
 #ifdef CONFIG_3_LEVEL_PGTABLES
 #include <asm/pgtable-3level.h>
 #else
@@ -326,4 +328,29 @@ do {						\
 	__flush_tlb_one((vaddr));		\
 } while (0)
 
+#else  /* CONFIG_MMU */
+
+#include <asm-generic/pgtable-nopud.h>
+
+#define swapper_pg_dir		((pgd_t *)0)
+#define PAGE_KERNEL             __pgprot(0)
+#define PGDIR_SHIFT		21
+#define PGDIR_SIZE		(1UL << PGDIR_SHIFT)
+#define PGDIR_MASK		(~(PGDIR_SIZE-1))
+
+#define VMALLOC_START	0
+#define VMALLOC_END	0xffffffff
+
+/*
+ * ZERO_PAGE is a global shared page that is always zero: used
+ * for zero-mapped memory areas etc..
+ */
+extern unsigned long *empty_zero_page;
+#define ZERO_PAGE(vaddr)	(virt_to_page(empty_zero_page))
+
+extern unsigned long end_iomem;
+
+#endif /* !CONFIG_MMU */
+
+
 #endif
diff --git a/arch/um/include/asm/uaccess.h b/arch/um/include/asm/uaccess.h
index fe66d659acad..95db8f06d295 100644
--- a/arch/um/include/asm/uaccess.h
+++ b/arch/um/include/asm/uaccess.h
@@ -21,6 +21,7 @@
 #define __addr_range_nowrap(addr, size) \
 	((unsigned long) (addr) <= ((unsigned long) (addr) + (size)))
 
+#ifdef CONFIG_MMU
 extern unsigned long raw_copy_from_user(void *to, const void __user *from, unsigned long n);
 extern unsigned long raw_copy_to_user(void __user *to, const void *from, unsigned long n);
 extern long __strncpy_from_user(char *dst, const char __user *src, long count);
@@ -46,4 +47,9 @@ static inline int __access_ok(unsigned long addr, unsigned long size)
 		uaccess_kernel());
 }
 
+#else
+#include <asm-generic/uaccess.h>
+#endif /* CONFIG_MMU */
+
+
 #endif
diff --git a/arch/um/lkl/include/uapi/asm/host_ops.h b/arch/um/lkl/include/uapi/asm/host_ops.h
index 5d141784541d..17219802fa1d 100644
--- a/arch/um/lkl/include/uapi/asm/host_ops.h
+++ b/arch/um/lkl/include/uapi/asm/host_ops.h
@@ -20,4 +20,20 @@ struct lkl_host_operations {
  */
 void lkl_bug(const char *fmt, ...);
 
+/**
+ * lkl_mem_alloc - allocate memory
+ *
+ * @mem: the size of memory requested
+ *
+ */
+void *lkl_mem_alloc(unsigned long mem);
+
+/**
+ * lkl_mem_free - free memory
+ *
+ * @mem: the address of memory to be freed
+ *
+ */
+void lkl_mem_free(void *mem);
+
 #endif
diff --git a/arch/um/lkl/um/bootmem.c b/arch/um/lkl/um/bootmem.c
new file mode 100644
index 000000000000..7250924a7d40
--- /dev/null
+++ b/arch/um/lkl/um/bootmem.c
@@ -0,0 +1,107 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/memblock.h>
+#include <linux/mm.h>
+#include <linux/swap.h>
+
+unsigned long memory_end, memory_start;
+EXPORT_SYMBOL(memory_start);
+static unsigned long _memory_start, mem_size;
+
+unsigned long *empty_zero_page;
+
+/* XXX: unused */
+unsigned long long highmem;
+int iomem_size;
+int kmalloc_ok = 1;
+
+void __init setup_physmem(unsigned long start, unsigned long reserve_end,
+			  unsigned long mem_sz, unsigned long long _highmem)
+{
+	unsigned long zones_size[MAX_NR_ZONES] = {0, };
+
+	mem_size = mem_sz;
+
+	_memory_start = (unsigned long)lkl_mem_alloc(mem_size);
+	memory_start = _memory_start;
+	WARN_ON(!memory_start);
+	memory_end = memory_start + mem_size;
+
+	if (PAGE_ALIGN(memory_start) != memory_start) {
+		mem_size -= PAGE_ALIGN(memory_start) - memory_start;
+		memory_start = PAGE_ALIGN(memory_start);
+		mem_size = (mem_size / PAGE_SIZE) * PAGE_SIZE;
+	}
+	pr_info("memblock address range: 0x%lx - 0x%lx\n", memory_start,
+		memory_start+mem_size);
+	/*
+	 * Give all the memory to the bootmap allocator, tell it to put the
+	 * boot mem_map at the start of memory.
+	 */
+	max_low_pfn = virt_to_pfn(memory_end);
+	min_low_pfn = virt_to_pfn(memory_start);
+	memblock_add(memory_start, mem_size);
+
+	empty_zero_page = memblock_alloc(PAGE_SIZE, PAGE_SIZE);
+	memset((void *)empty_zero_page, 0, PAGE_SIZE);
+
+	zones_size[ZONE_NORMAL] = max_low_pfn;
+	free_area_init(zones_size);
+
+}
+
+void __init mem_init(void)
+{
+	max_mapnr = (((unsigned long)high_memory) - PAGE_OFFSET) >> PAGE_SHIFT;
+	/* this will put all memory onto the freelists */
+	totalram_pages_add(memblock_free_all());
+	pr_info("Memory available: %luk/%luk RAM\n",
+		(nr_free_pages() << PAGE_SHIFT) >> 10, mem_size >> 10);
+}
+
+/*
+ * In our case __init memory is not part of the page allocator so there is
+ * nothing to free.
+ */
+void free_initmem(void)
+{
+}
+
+void free_mem(void)
+{
+	lkl_mem_free((void *)_memory_start);
+}
+
+void *uml_kmalloc(int size, int flags)
+{
+	return kmalloc(size, flags);
+}
+
+void __init mem_total_pages(unsigned long physmem, unsigned long iomem,
+		     unsigned long _highmem)
+{
+}
+
+void __init paging_init(void)
+{
+}
+
+
+int set_memory_ro(unsigned long addr, int numpages)
+{
+	return -EOPNOTSUPP;
+}
+
+int set_memory_rw(unsigned long addr, int numpages)
+{
+	return -EOPNOTSUPP;
+}
+
+int set_memory_nx(unsigned long addr, int numpages)
+{
+	return -EOPNOTSUPP;
+}
+
+int set_memory_x(unsigned long addr, int numpages)
+{
+	return -EOPNOTSUPP;
+}
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v8 08/20] um: lkl: memory handling
@ 2021-01-20  2:27             ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-01-20  2:27 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, thehajime, retrage01

library mode follows the way !CONFIG_MMU architecture of other linux
archs.  There is not much work left to do other than initializing the
boot allocator and providing the page and page table definitions.

The backstore memory is allocated via host functions and the memory
size to be used is specified when the kernel is started, in the
lkl_start_kernel call.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/include/asm/mmu.h               |   3 +
 arch/um/include/asm/mmu_context.h       |  10 +++
 arch/um/include/asm/page.h              |  15 ++++
 arch/um/include/asm/pgtable.h           |  27 ++++++
 arch/um/include/asm/uaccess.h           |   6 ++
 arch/um/lkl/include/uapi/asm/host_ops.h |  16 ++++
 arch/um/lkl/um/bootmem.c                | 107 ++++++++++++++++++++++++
 7 files changed, 184 insertions(+)
 create mode 100644 arch/um/lkl/um/bootmem.c

diff --git a/arch/um/include/asm/mmu.h b/arch/um/include/asm/mmu.h
index 5b072aba5b65..c06d6cb67dd7 100644
--- a/arch/um/include/asm/mmu.h
+++ b/arch/um/include/asm/mmu.h
@@ -13,6 +13,9 @@ typedef struct mm_context {
 	struct mm_id id;
 	struct uml_arch_mm_context arch;
 	struct page *stub_pages[2];
+#ifndef CONFIG_MMU
+	unsigned long		end_brk;
+#endif
 } mm_context_t;
 
 extern void __switch_mm(struct mm_id * mm_idp);
diff --git a/arch/um/include/asm/mmu_context.h b/arch/um/include/asm/mmu_context.h
index f8a100770691..4bf627feaa0c 100644
--- a/arch/um/include/asm/mmu_context.h
+++ b/arch/um/include/asm/mmu_context.h
@@ -6,6 +6,7 @@
 #ifndef __UM_MMU_CONTEXT_H
 #define __UM_MMU_CONTEXT_H
 
+#ifdef CONFIG_MMU
 #include <linux/sched.h>
 #include <linux/mm_types.h>
 #include <linux/mmap_lock.h>
@@ -73,4 +74,13 @@ extern void destroy_context(struct mm_struct *mm);
 
 #include <asm-generic/mmu_context.h>
 
+#else
+
+#include <asm-generic/nommu_context.h>
+
+extern void force_flush_all(void);
+
+#endif /* CONFIG_MMU */
+
+
 #endif
diff --git a/arch/um/include/asm/page.h b/arch/um/include/asm/page.h
index 95af12e82a32..c9d5e487ac6b 100644
--- a/arch/um/include/asm/page.h
+++ b/arch/um/include/asm/page.h
@@ -7,6 +7,8 @@
 #ifndef __UM_PAGE_H
 #define __UM_PAGE_H
 
+#ifdef CONFIG_MMU
+
 #include <linux/const.h>
 
 /* PAGE_SHIFT determines the page size */
@@ -120,4 +122,17 @@ extern unsigned long uml_physmem;
 #define __HAVE_ARCH_GATE_AREA 1
 #endif
 
+#else  /* CONFIG_MMU */
+#define CONFIG_KERNEL_RAM_BASE_ADDRESS memory_start
+#include <asm-generic/page.h>
+
+#define __va_space (8*1024*1024)
+
+#ifndef __ASSEMBLY__
+#include <mem.h>
+void free_mem(void);
+#endif
+
+#endif /* !CONFIG_MMU  */
+
 #endif	/* __UM_PAGE_H */
diff --git a/arch/um/include/asm/pgtable.h b/arch/um/include/asm/pgtable.h
index 39376bb63abf..58638d888b95 100644
--- a/arch/um/include/asm/pgtable.h
+++ b/arch/um/include/asm/pgtable.h
@@ -21,6 +21,8 @@
 #define _PAGE_PROTNONE	0x010	/* if the user mapped it with PROT_NONE;
 				   pte_present gives true */
 
+#ifdef CONFIG_MMU
+
 #ifdef CONFIG_3_LEVEL_PGTABLES
 #include <asm/pgtable-3level.h>
 #else
@@ -326,4 +328,29 @@ do {						\
 	__flush_tlb_one((vaddr));		\
 } while (0)
 
+#else  /* CONFIG_MMU */
+
+#include <asm-generic/pgtable-nopud.h>
+
+#define swapper_pg_dir		((pgd_t *)0)
+#define PAGE_KERNEL             __pgprot(0)
+#define PGDIR_SHIFT		21
+#define PGDIR_SIZE		(1UL << PGDIR_SHIFT)
+#define PGDIR_MASK		(~(PGDIR_SIZE-1))
+
+#define VMALLOC_START	0
+#define VMALLOC_END	0xffffffff
+
+/*
+ * ZERO_PAGE is a global shared page that is always zero: used
+ * for zero-mapped memory areas etc..
+ */
+extern unsigned long *empty_zero_page;
+#define ZERO_PAGE(vaddr)	(virt_to_page(empty_zero_page))
+
+extern unsigned long end_iomem;
+
+#endif /* !CONFIG_MMU */
+
+
 #endif
diff --git a/arch/um/include/asm/uaccess.h b/arch/um/include/asm/uaccess.h
index fe66d659acad..95db8f06d295 100644
--- a/arch/um/include/asm/uaccess.h
+++ b/arch/um/include/asm/uaccess.h
@@ -21,6 +21,7 @@
 #define __addr_range_nowrap(addr, size) \
 	((unsigned long) (addr) <= ((unsigned long) (addr) + (size)))
 
+#ifdef CONFIG_MMU
 extern unsigned long raw_copy_from_user(void *to, const void __user *from, unsigned long n);
 extern unsigned long raw_copy_to_user(void __user *to, const void *from, unsigned long n);
 extern long __strncpy_from_user(char *dst, const char __user *src, long count);
@@ -46,4 +47,9 @@ static inline int __access_ok(unsigned long addr, unsigned long size)
 		uaccess_kernel());
 }
 
+#else
+#include <asm-generic/uaccess.h>
+#endif /* CONFIG_MMU */
+
+
 #endif
diff --git a/arch/um/lkl/include/uapi/asm/host_ops.h b/arch/um/lkl/include/uapi/asm/host_ops.h
index 5d141784541d..17219802fa1d 100644
--- a/arch/um/lkl/include/uapi/asm/host_ops.h
+++ b/arch/um/lkl/include/uapi/asm/host_ops.h
@@ -20,4 +20,20 @@ struct lkl_host_operations {
  */
 void lkl_bug(const char *fmt, ...);
 
+/**
+ * lkl_mem_alloc - allocate memory
+ *
+ * @mem: the size of memory requested
+ *
+ */
+void *lkl_mem_alloc(unsigned long mem);
+
+/**
+ * lkl_mem_free - free memory
+ *
+ * @mem: the address of memory to be freed
+ *
+ */
+void lkl_mem_free(void *mem);
+
 #endif
diff --git a/arch/um/lkl/um/bootmem.c b/arch/um/lkl/um/bootmem.c
new file mode 100644
index 000000000000..7250924a7d40
--- /dev/null
+++ b/arch/um/lkl/um/bootmem.c
@@ -0,0 +1,107 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/memblock.h>
+#include <linux/mm.h>
+#include <linux/swap.h>
+
+unsigned long memory_end, memory_start;
+EXPORT_SYMBOL(memory_start);
+static unsigned long _memory_start, mem_size;
+
+unsigned long *empty_zero_page;
+
+/* XXX: unused */
+unsigned long long highmem;
+int iomem_size;
+int kmalloc_ok = 1;
+
+void __init setup_physmem(unsigned long start, unsigned long reserve_end,
+			  unsigned long mem_sz, unsigned long long _highmem)
+{
+	unsigned long zones_size[MAX_NR_ZONES] = {0, };
+
+	mem_size = mem_sz;
+
+	_memory_start = (unsigned long)lkl_mem_alloc(mem_size);
+	memory_start = _memory_start;
+	WARN_ON(!memory_start);
+	memory_end = memory_start + mem_size;
+
+	if (PAGE_ALIGN(memory_start) != memory_start) {
+		mem_size -= PAGE_ALIGN(memory_start) - memory_start;
+		memory_start = PAGE_ALIGN(memory_start);
+		mem_size = (mem_size / PAGE_SIZE) * PAGE_SIZE;
+	}
+	pr_info("memblock address range: 0x%lx - 0x%lx\n", memory_start,
+		memory_start+mem_size);
+	/*
+	 * Give all the memory to the bootmap allocator, tell it to put the
+	 * boot mem_map at the start of memory.
+	 */
+	max_low_pfn = virt_to_pfn(memory_end);
+	min_low_pfn = virt_to_pfn(memory_start);
+	memblock_add(memory_start, mem_size);
+
+	empty_zero_page = memblock_alloc(PAGE_SIZE, PAGE_SIZE);
+	memset((void *)empty_zero_page, 0, PAGE_SIZE);
+
+	zones_size[ZONE_NORMAL] = max_low_pfn;
+	free_area_init(zones_size);
+
+}
+
+void __init mem_init(void)
+{
+	max_mapnr = (((unsigned long)high_memory) - PAGE_OFFSET) >> PAGE_SHIFT;
+	/* this will put all memory onto the freelists */
+	totalram_pages_add(memblock_free_all());
+	pr_info("Memory available: %luk/%luk RAM\n",
+		(nr_free_pages() << PAGE_SHIFT) >> 10, mem_size >> 10);
+}
+
+/*
+ * In our case __init memory is not part of the page allocator so there is
+ * nothing to free.
+ */
+void free_initmem(void)
+{
+}
+
+void free_mem(void)
+{
+	lkl_mem_free((void *)_memory_start);
+}
+
+void *uml_kmalloc(int size, int flags)
+{
+	return kmalloc(size, flags);
+}
+
+void __init mem_total_pages(unsigned long physmem, unsigned long iomem,
+		     unsigned long _highmem)
+{
+}
+
+void __init paging_init(void)
+{
+}
+
+
+int set_memory_ro(unsigned long addr, int numpages)
+{
+	return -EOPNOTSUPP;
+}
+
+int set_memory_rw(unsigned long addr, int numpages)
+{
+	return -EOPNOTSUPP;
+}
+
+int set_memory_nx(unsigned long addr, int numpages)
+{
+	return -EOPNOTSUPP;
+}
+
+int set_memory_x(unsigned long addr, int numpages)
+{
+	return -EOPNOTSUPP;
+}
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v8 09/20] um: lkl: kernel thread support
  2021-01-20  2:27           ` Hajime Tazaki
@ 2021-01-20  2:27             ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-01-20  2:27 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: thehajime, tavi.purdila, retrage01, linux-kernel-library, linux-arch

library mode does not support user processes but it must support kernel
threads as part as the normal kernel work-flow. It uses host functions
to create and terminate host threads that are going to run the kernel
threads. It also uses semaphores to synchronize those threads and to
allow the Linux kernel scheduler to control how the kernel threads
run.

Each kernel thread runs in a host thread and has a host semaphore
associated with it - the thread's scheduling semaphore. The semaphore
counter is initialized to 0. The first thing a kernel thread does
after getting spawned, before running any kernel code, is to perform a
down operation to block the thread.

The kernel controls host threads scheduling by performing up and down
operations on the scheduling semaphore. In __switch_context an up
operation on the next thread is performed to wake up a blocked thread,
and a down operation is performed on the prev thread to block it.

A thread is terminated by marking it in free_thread_info and
performing an up operation on the scheduling semaphore at which point
the marked thread will terminate itself.

Kernel threads are executed with the synchronization to other context
primitivecs of kernel, such as idle thread, system calls, interrupts,
"reentrancy", CPU shutdown, imbalance wake up, which are not
controllable with a simple synchronization mechanism.  Thus we
introduced a logical CPU entity, struct lkl_cpu, to run Linux code with
multiple threads; CPU get/put functions (lkl_cpu_put/lkl_cpu_get) is
used to serialize access between LKL threads and host threads that
issue direct calls.

UML common part (process.c) is extended to call idle functions of
SUBARCH as well as kernel thread detection with a flag information of
TIF_HOST_THREAD.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/include/asm/thread_info.h       |  24 +++
 arch/um/kernel/process.c                |   8 +-
 arch/um/lkl/include/asm/cpu.h           |  11 +
 arch/um/lkl/include/asm/sched.h         |  23 +++
 arch/um/lkl/include/uapi/asm/host_ops.h | 164 +++++++++++++++
 arch/um/lkl/um/cpu.c                    | 223 ++++++++++++++++++++
 arch/um/lkl/um/threads.c                | 258 ++++++++++++++++++++++++
 7 files changed, 710 insertions(+), 1 deletion(-)
 create mode 100644 arch/um/lkl/include/asm/cpu.h
 create mode 100644 arch/um/lkl/include/asm/sched.h
 create mode 100644 arch/um/lkl/um/cpu.c
 create mode 100644 arch/um/lkl/um/threads.c

diff --git a/arch/um/include/asm/thread_info.h b/arch/um/include/asm/thread_info.h
index 3b1cb8b3b186..528e0347a9d8 100644
--- a/arch/um/include/asm/thread_info.h
+++ b/arch/um/include/asm/thread_info.h
@@ -40,6 +40,7 @@ struct thread_info {
 	.real_thread = NULL,			\
 }
 
+#ifndef CONFIG_UMMODE_LIB
 /* how to get the thread information struct from C */
 static inline struct thread_info *current_thread_info(void)
 {
@@ -52,6 +53,27 @@ static inline struct thread_info *current_thread_info(void)
 	return ti;
 }
 
+#else /* CONFIG_UMMODE_LIB */
+
+#define __HAVE_THREAD_FUNCTIONS
+#define task_thread_info(task)	((struct thread_info *)(task)->stack)
+#define task_stack_page(task)	((task)->stack)
+#define end_of_stack(p) (&task_thread_info(p)->aux_fp_regs[FP_SIZE-1])
+
+void threads_init(void);
+void threads_cleanup(void);
+
+unsigned long *alloc_thread_stack_node(struct task_struct *p, int node);
+void setup_thread_stack(struct task_struct *p, struct task_struct *org);
+void free_thread_stack(struct task_struct *tsk);
+
+extern struct thread_info *_current_thread_info;
+static inline struct thread_info *current_thread_info(void)
+{
+	return _current_thread_info;
+}
+#endif /* CONFIG_UMMODE_LIB */
+
 #endif
 
 #define TIF_SYSCALL_TRACE	0	/* syscall trace active */
@@ -64,6 +86,8 @@ static inline struct thread_info *current_thread_info(void)
 #define TIF_RESTORE_SIGMASK	7
 #define TIF_NOTIFY_RESUME	8
 #define TIF_SECCOMP		9	/* secure computing */
+#define TIF_SCHED_JB		10	/* thread is configured with jump buffer */
+#define TIF_HOST_THREAD	11	/* thread is a host task */
 
 #define _TIF_SYSCALL_TRACE	(1 << TIF_SYSCALL_TRACE)
 #define _TIF_SIGPENDING		(1 << TIF_SIGPENDING)
diff --git a/arch/um/kernel/process.c b/arch/um/kernel/process.c
index 9b0a36f64339..535ba738f2f5 100644
--- a/arch/um/kernel/process.c
+++ b/arch/um/kernel/process.c
@@ -160,7 +160,8 @@ int copy_thread(unsigned long clone_flags, unsigned long sp,
 		unsigned long arg, struct task_struct * p, unsigned long tls)
 {
 	void (*handler)(void);
-	int kthread = current->flags & PF_KTHREAD;
+	int kthread = current->flags & PF_KTHREAD ||
+		test_ti_thread_flag(current_thread_info(), TIF_HOST_THREAD);
 	int ret = 0;
 
 	p->thread = (struct thread_struct) INIT_THREAD;
@@ -214,10 +215,15 @@ void um_idle_sleep(void)
 		os_idle_sleep();
 }
 
+void __weak subarch_cpu_idle(void)
+{
+}
+
 void arch_cpu_idle(void)
 {
 	cpu_tasks[current_thread_info()->cpu].pid = os_getpid();
 	um_idle_sleep();
+	subarch_cpu_idle();
 	raw_local_irq_enable();
 }
 
diff --git a/arch/um/lkl/include/asm/cpu.h b/arch/um/lkl/include/asm/cpu.h
new file mode 100644
index 000000000000..c1164187151e
--- /dev/null
+++ b/arch/um/lkl/include/asm/cpu.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_LIBMODE_CPU_H
+#define __UM_LIBMODE_CPU_H
+
+int lkl_cpu_get(void);
+void lkl_cpu_put(void);
+int lkl_cpu_init(void);
+void lkl_cpu_wait_shutdown(void);
+void lkl_cpu_change_owner(lkl_thread_t owner);
+
+#endif
diff --git a/arch/um/lkl/include/asm/sched.h b/arch/um/lkl/include/asm/sched.h
new file mode 100644
index 000000000000..1ac063ed987c
--- /dev/null
+++ b/arch/um/lkl/include/asm/sched.h
@@ -0,0 +1,23 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_LIBMODE_SCHED_H
+#define __UM_LIBMODE_SCHED_H
+
+#include <linux/sched.h>
+#include <uapi/asm/host_ops.h>
+
+static inline void thread_sched_jb(void)
+{
+	if (test_ti_thread_flag(current_thread_info(), TIF_HOST_THREAD)) {
+		set_ti_thread_flag(current_thread_info(), TIF_SCHED_JB);
+		set_current_state(TASK_UNINTERRUPTIBLE);
+		lkl_jmp_buf_set(&current_thread_info()->task->thread.arch.sched_jb,
+				     schedule);
+	} else {
+		lkl_bug("%s can be used only for host task", __func__);
+	}
+}
+
+void switch_to_host_task(struct task_struct *);
+int host_task_stub(void *unused);
+
+#endif
diff --git a/arch/um/lkl/include/uapi/asm/host_ops.h b/arch/um/lkl/include/uapi/asm/host_ops.h
index 17219802fa1d..039a17972d7d 100644
--- a/arch/um/lkl/include/uapi/asm/host_ops.h
+++ b/arch/um/lkl/include/uapi/asm/host_ops.h
@@ -2,6 +2,13 @@
 #ifndef __UM_LIBMODE_UAPI_HOST_OPS_H
 #define __UM_LIBMODE_UAPI_HOST_OPS_H
 
+struct lkl_mutex;
+struct lkl_sem;
+typedef unsigned long lkl_thread_t;
+struct lkl_jmp_buf {
+	unsigned long buf[128];
+};
+
 /**
  * struct lkl_host_operations - host operations used by the Linux kernel
  *
@@ -36,4 +43,161 @@ void *lkl_mem_alloc(unsigned long mem);
  */
 void lkl_mem_free(void *mem);
 
+/**
+ * lkl_sem_alloc - allocate a host semaphore an initialize it to count
+ *
+ * @count: initial counter value
+ *
+ * Return: the pointer of allocated semaphore
+ *
+ */
+struct lkl_sem *lkl_sem_alloc(int count);
+
+/**
+ * lkl_sem_free - free a host semaphore
+ *
+ * @sem: the address of semaphore to be freed
+ *
+ */
+void lkl_sem_free(struct lkl_sem *sem);
+
+/**
+ * lkl_sem_up - perform an up operation on the semaphore
+ *
+ * @sem: semaphore pointer address
+ *
+ */
+void lkl_sem_up(struct lkl_sem *sem);
+
+/**
+ * lkl_sem_down - perform a down operation on the semaphore
+ *
+ * @sem: semaphore pointer address
+ *
+ */
+void lkl_sem_down(struct lkl_sem *sem);
+
+/**
+ * lkl_mutex_alloc - allocate and initialize a host mutex
+ *
+ * @recursive: boolean flag if the mutex is recursive or not
+ *
+ * Return: the pointer of allocated mutex
+ *
+ */
+struct lkl_mutex *lkl_mutex_alloc(int recursive);
+
+/**
+ * lkl_mutex_free - free a host mutex
+ *
+ * @mutex: the mutex pointer to be freed
+ *
+ */
+void lkl_mutex_lock(struct lkl_mutex *mutex);
+
+/**
+ * lkl_mutex_lock - acquire the mutex
+ *
+ * @_mutex: the mutex pointer to be locked
+ *
+ */
+void lkl_mutex_unlock(struct lkl_mutex *_mutex);
+
+/**
+ * lkl_mutex_unlock - release the mutex
+ *
+ * @_mutex: the mutex pointer to be released
+ *
+ */
+void lkl_mutex_free(struct lkl_mutex *_mutex);
+
+/**
+ * lkl_thread_create - create a new thread and run f(arg) in its context
+ *
+ * @fn: a start routine when creating a thread
+ * @arg: an argument for the function @fn
+ *
+ * Return: a thread handle or 0 if the thread could not be created
+ *
+ */
+lkl_thread_t lkl_thread_create(void* (*fn)(void *), void *arg);
+
+/**
+ * lkl_thread_detach - on POSIX systems, free up resources held by
+ * pthreads.
+ *
+ */
+void lkl_thread_detach(void);
+
+/**
+ * lkl_thread_exit - terminates the current thread
+ *
+ */
+void lkl_thread_exit(void);
+
+/**
+ * lkl_thread_join - wait for the given thread to terminate.
+ *
+ * @tid: thread id to wait
+ *
+ * Return: 0 for success, -1 otherwise
+ *
+ */
+int lkl_thread_join(lkl_thread_t tid);
+
+/**
+ * lkl_thread_self - returns the identifier of the current thread
+ *
+ * Return: the identifier of the current thread
+ *
+ */
+lkl_thread_t lkl_thread_self(void);
+
+/**
+ * lkl_thread_equal - compare if thread a and b are identical or not
+ *
+ * @a: a thread to be compared
+ * @b: another thread to be compared with @a
+ *
+ * Return: 1 if @a and @b are same; otherwise 0
+ *
+ */
+int lkl_thread_equal(lkl_thread_t a, lkl_thread_t b);
+
+/**
+ * lkl_gettid - obtain the thread identifier of the current thread
+ *
+ * Return: the host thread id of the caller, which need not be the same
+ * as the handle returned by thread_create
+ *
+ */
+long lkl_gettid(void);
+
+/**
+ * lkl_jmp_buf_set - set the jump buffer with a setup function
+ *
+ * @jmpb: jump buffer to be saved
+ * @f: a function to be called
+ *
+ * This runs the given function and setups a jump back point by saving
+ * the context in the jump buffer; jmp_buf_longjmp can be called from the give
+ * function or any callee in that function to return back to the jump back
+ * point.
+ *
+ * NOTE: we can't return from lkl_jmp_buf_set before calling lkl_jmp_buf_longjmp
+ * or otherwise the saved context (stack) is not going to be valid, so we must
+ * pass the function that will eventually call longjmp here.
+ *
+ */
+void lkl_jmp_buf_set(struct lkl_jmp_buf *jmpb, void (*f)(void));
+
+/**
+ * lkl_jmp_buf_longjmp - perform a jump back to the saved jump buffer
+ *
+ * @jmpb: jump buffer to be restored
+ * @val: returned value of longjmp
+ *
+ */
+void lkl_jmp_buf_longjmp(struct lkl_jmp_buf *jmpb, int val);
+
 #endif
diff --git a/arch/um/lkl/um/cpu.c b/arch/um/lkl/um/cpu.c
new file mode 100644
index 000000000000..75452b28d741
--- /dev/null
+++ b/arch/um/lkl/um/cpu.c
@@ -0,0 +1,223 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/kernel.h>
+#include <linux/sched/stat.h>
+#include <asm/host_ops.h>
+#include <asm/cpu.h>
+#include <asm/thread_info.h>
+#include <asm/unistd.h>
+#include <asm/sched.h>
+#include <asm/syscalls.h>
+#include <init.h>
+
+/*
+ * This structure is used to get access to the "LKL CPU" that allows us to run
+ * Linux code. Because we have to deal with various synchronization requirements
+ * between idle thread, system calls, interrupts, "reentrancy", CPU shutdown,
+ * imbalance wake up (i.e. acquire the CPU from one thread and release it from
+ * another), we can't use a simple synchronization mechanism such as (recursive)
+ * mutex or semaphore. Instead, we use a mutex and a bunch of status data plus a
+ * semaphore.
+ */
+static struct lkl_cpu {
+	/* lock that protects the CPU status data */
+	struct lkl_mutex *lock;
+	/*
+	 * Since we must free the cpu lock during shutdown we need a
+	 * synchronization algorithm between lkl_cpu_shutdown() and the CPU
+	 * access functions since lkl_cpu_get() gets called from thread
+	 * destructor callback functions which may be scheduled after
+	 * lkl_cpu_shutdown() has freed the cpu lock.
+	 *
+	 * An atomic counter is used to keep track of the number of running
+	 * CPU access functions and allow the shutdown function to wait for
+	 * them.
+	 *
+	 * The shutdown functions adds MAX_THREADS to this counter which allows
+	 * the CPU access functions to check if the shutdown process has
+	 * started.
+	 *
+	 * This algorithm assumes that we never have more the MAX_THREADS
+	 * requesting CPU access.
+	 */
+	#define MAX_THREADS 1000000
+	unsigned int shutdown_gate;
+	/* no of threads waiting the CPU */
+	unsigned int sleepers;
+	/* no of times the current thread got the CPU */
+	unsigned int count;
+	/* current thread that owns the CPU */
+	lkl_thread_t owner;
+	/* semaphore for threads waiting the CPU */
+	struct lkl_sem *sem;
+	/* semaphore used for shutdown */
+	struct lkl_sem *shutdown_sem;
+} cpu;
+
+/*
+ * internal routine to acquire LKL CPU's lock
+ */
+static int __cpu_try_get_lock(int n)
+{
+	lkl_thread_t self;
+
+	if (__sync_fetch_and_add(&cpu.shutdown_gate, n) >= MAX_THREADS)
+		return -2;
+
+	lkl_mutex_lock(cpu.lock);
+
+	if (cpu.shutdown_gate >= MAX_THREADS)
+		return -1;
+
+	self = lkl_thread_self();
+
+	/* if someone else is using the cpu, indicate as return 0 */
+	if (cpu.owner && !lkl_thread_equal(cpu.owner, self))
+		return 0;
+
+	/* set the owner of cpu */
+	cpu.owner = self;
+	cpu.count++;
+
+	return 1;
+}
+
+/*
+ * internal routine to release LKL CPU's lock
+ */
+static void __cpu_try_get_unlock(int lock_ret, int n)
+{
+	/* release lock only if __cpu_try_get_lock() holds cpu.lock
+	 * (returns >= -1)
+	 */
+	if (lock_ret >= -1)
+		lkl_mutex_unlock(cpu.lock);
+	__sync_fetch_and_sub(&cpu.shutdown_gate, n);
+}
+
+void lkl_cpu_change_owner(lkl_thread_t owner)
+{
+	lkl_mutex_lock(cpu.lock);
+	if (cpu.count > 1)
+		lkl_bug("bad count while changing owner\n");
+	cpu.owner = owner;
+	lkl_mutex_unlock(cpu.lock);
+}
+
+int lkl_cpu_get(void)
+{
+	int ret;
+
+	ret = __cpu_try_get_lock(1);
+
+	/* when somebody holds a lock, sleep until released,
+	 * with obtaining a semaphore (cpu.sem)
+	 */
+	while (ret == 0) {
+		cpu.sleepers++;
+		__cpu_try_get_unlock(ret, 0);
+		lkl_sem_down(cpu.sem);
+		ret = __cpu_try_get_lock(0);
+	}
+
+	__cpu_try_get_unlock(ret, 1);
+
+	return ret;
+}
+
+void lkl_cpu_put(void)
+{
+	lkl_mutex_lock(cpu.lock);
+
+	if (!cpu.count || !cpu.owner ||
+	    !lkl_thread_equal(cpu.owner, lkl_thread_self()))
+		lkl_bug("%s: unbalanced put\n", __func__);
+
+	/* switch to userspace code if current is host task (TIF_HOST_THREAD),
+	 * AND, there are other running tasks.
+	 */
+	if (test_ti_thread_flag(current_thread_info(), TIF_HOST_THREAD) &&
+	    !single_task_running() && cpu.count == 1) {
+		if (in_interrupt())
+			lkl_bug("%s: in interrupt\n", __func__);
+		lkl_mutex_unlock(cpu.lock);
+		thread_sched_jb();
+		return;
+	}
+
+	/* if there are any other tasks holding cpu lock, return after
+	 * decreasing cpu.count
+	 */
+	if (--cpu.count > 0) {
+		lkl_mutex_unlock(cpu.lock);
+		return;
+	}
+
+	/* release semaphore if slept */
+	if (cpu.sleepers) {
+		cpu.sleepers--;
+		lkl_sem_up(cpu.sem);
+	}
+
+	cpu.owner = 0;
+
+	lkl_mutex_unlock(cpu.lock);
+}
+
+static void lkl_cpu_shutdown(void)
+{
+	__sync_fetch_and_add(&cpu.shutdown_gate, MAX_THREADS);
+}
+__uml_exitcall(lkl_cpu_shutdown);
+
+void lkl_cpu_wait_shutdown(void)
+{
+	lkl_sem_down(cpu.shutdown_sem);
+	lkl_sem_free(cpu.shutdown_sem);
+}
+
+static void lkl_cpu_cleanup(bool shutdown)
+{
+	while (__sync_fetch_and_add(&cpu.shutdown_gate, 0) > MAX_THREADS)
+		;
+
+	/* if caller indicates shutdown, notify the semaphore to release
+	 * the block (lkl_cpu_wait_shutdown()).
+	 */
+	if (shutdown)
+		lkl_sem_up(cpu.shutdown_sem);
+	/* if lkl_cpu_wait_shutdown() is not called, free shutdown_sem here */
+	else if (cpu.shutdown_sem)
+		lkl_sem_free(cpu.shutdown_sem);
+
+	if (cpu.sem)
+		lkl_sem_free(cpu.sem);
+	if (cpu.lock)
+		lkl_mutex_free(cpu.lock);
+}
+
+void subarch_cpu_idle(void)
+{
+	if (cpu.shutdown_gate >= MAX_THREADS) {
+		lkl_mutex_lock(cpu.lock);
+		while (cpu.sleepers--)
+			lkl_sem_up(cpu.sem);
+		lkl_mutex_unlock(cpu.lock);
+
+		lkl_cpu_cleanup(true);
+		lkl_thread_exit();
+	}
+}
+
+int lkl_cpu_init(void)
+{
+	cpu.lock = lkl_mutex_alloc(0);
+	cpu.sem = lkl_sem_alloc(0);
+	cpu.shutdown_sem = lkl_sem_alloc(0);
+
+	if (!cpu.lock || !cpu.sem || !cpu.shutdown_sem) {
+		lkl_cpu_cleanup(false);
+		return -ENOMEM;
+	}
+
+	return 0;
+}
diff --git a/arch/um/lkl/um/threads.c b/arch/um/lkl/um/threads.c
new file mode 100644
index 000000000000..7ef9b9f2a6b7
--- /dev/null
+++ b/arch/um/lkl/um/threads.c
@@ -0,0 +1,258 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/slab.h>
+#include <linux/sched/task.h>
+#include <linux/sched/signal.h>
+#include <asm/host_ops.h>
+#include <asm/cpu.h>
+#include <asm/sched.h>
+
+#include <os.h>
+
+static int init_arch_thread(struct arch_thread *thread)
+{
+	thread->sched_sem = lkl_sem_alloc(0);
+	if (!thread->sched_sem)
+		return -ENOMEM;
+
+	thread->dead = false;
+	thread->tid = 0;
+
+	return 0;
+}
+
+unsigned long *alloc_thread_stack_node(struct task_struct *task, int node)
+{
+	struct thread_info *ti;
+
+	ti = kmalloc(sizeof(*ti), GFP_KERNEL | __GFP_ZERO);
+	if (!ti)
+		return NULL;
+
+	ti->task = task;
+	return (unsigned long *)ti;
+}
+
+/*
+ * The only new tasks created are kernel threads that have a predefined starting
+ * point thus no stack copy is required.
+ */
+void setup_thread_stack(struct task_struct *p, struct task_struct *org)
+{
+	struct thread_info *ti = task_thread_info(p);
+	struct thread_info *org_ti = task_thread_info(org);
+
+	ti->flags = org_ti->flags;
+	ti->preempt_count = org_ti->preempt_count;
+	ti->addr_limit = org_ti->addr_limit;
+}
+
+static void kill_thread(struct thread_info *ti)
+{
+	if (!test_ti_thread_flag(ti, TIF_HOST_THREAD)) {
+		ti->task->thread.arch.dead = true;
+		lkl_sem_up(ti->task->thread.arch.sched_sem);
+		lkl_thread_join(ti->task->thread.arch.tid);
+	}
+	lkl_sem_free(ti->task->thread.arch.sched_sem);
+}
+
+void free_thread_stack(struct task_struct *tsk)
+{
+	struct thread_info *ti = task_thread_info(tsk);
+
+	kill_thread(ti);
+	kfree(ti);
+}
+
+struct thread_info *_current_thread_info = &init_thread_union.thread_info;
+EXPORT_SYMBOL(_current_thread_info);
+
+void switch_threads(jmp_buf *me, jmp_buf *you)
+{
+	/* NOP */
+}
+
+/*
+ * schedule() expects the return of this function to be the task that we
+ * switched away from. Returning prev is not going to work because we are
+ * actually going to return the previous taks that was scheduled before the
+ * task we are going to wake up, and not the current task, e.g.:
+ *
+ * swapper -> init: saved prev on swapper stack is swapper
+ * init -> ksoftirqd0: saved prev on init stack is init
+ * ksoftirqd0 -> swapper: returned prev is swapper
+ */
+static struct task_struct *abs_prev = &init_task;
+
+void arch_switch_to(struct task_struct *prev,
+		    struct task_struct *next)
+{
+	struct arch_thread *_prev = &prev->thread.arch;
+	struct arch_thread *_next = &next->thread.arch;
+	unsigned long _prev_flags = task_thread_info(prev)->flags;
+	struct lkl_jmp_buf *_prev_jb;
+
+	_current_thread_info = task_thread_info(next);
+	next->thread.prev_sched = prev;
+	abs_prev = prev;
+
+	WARN_ON(!_next->tid);
+	lkl_cpu_change_owner(_next->tid);
+
+	if (test_bit(TIF_SCHED_JB, &_prev_flags)) {
+		/* Atomic. Must be done before wakeup next */
+		clear_ti_thread_flag(task_thread_info(prev), TIF_SCHED_JB);
+		_prev_jb = &_prev->sched_jb;
+	}
+
+	lkl_sem_up(_next->sched_sem);
+	if (test_bit(TIF_SCHED_JB, &_prev_flags))
+		lkl_jmp_buf_longjmp(_prev_jb, 1);
+	else
+		lkl_sem_down(_prev->sched_sem);
+
+	if (_prev->dead)
+		lkl_thread_exit();
+
+	/* __switch_to (arch/um) returns this value */
+	current->thread.prev_sched = abs_prev;
+}
+
+int host_task_stub(void *unused)
+{
+	return 0;
+}
+
+void switch_to_host_task(struct task_struct *task)
+{
+	if (WARN_ON(!test_tsk_thread_flag(task, TIF_HOST_THREAD)))
+		return;
+
+	task->thread.arch.tid = lkl_thread_self();
+
+	if (current == task)
+		return;
+
+	wake_up_process(task);
+	thread_sched_jb();
+	lkl_sem_down(task->thread.arch.sched_sem);
+	schedule_tail(abs_prev);
+}
+
+struct thread_bootstrap_arg {
+	struct thread_info *ti;
+	int (*f)(void *a);
+	void *arg;
+};
+
+static void *thread_bootstrap(void *_tba)
+{
+	struct thread_bootstrap_arg *tba = (struct thread_bootstrap_arg *)_tba;
+	struct thread_info *ti = tba->ti;
+	int (*f)(void *) = tba->f;
+	void *arg = tba->arg;
+
+	lkl_sem_down(ti->task->thread.arch.sched_sem);
+	kfree(tba);
+	if (ti->task->thread.prev_sched)
+		schedule_tail(ti->task->thread.prev_sched);
+
+	f(arg);
+	do_exit(0);
+
+	return NULL;
+}
+
+void new_thread(void *stack, jmp_buf *buf, void (*handler)(void))
+{
+	struct thread_info *ti = (struct thread_info *)stack;
+	struct task_struct *p = ti->task;
+	struct thread_bootstrap_arg *tba;
+	int ret;
+
+	unsigned long esp = (unsigned long)p->thread.request.u.thread.proc;
+	unsigned long unused = (unsigned long)p->thread.request.u.thread.arg;
+
+	ret = init_arch_thread(&p->thread.arch);
+	if (ret < 0)
+		panic("%s: init_arch_thread", __func__);
+
+	if ((int (*)(void *))esp == host_task_stub) {
+		set_ti_thread_flag(ti, TIF_HOST_THREAD);
+		return;
+	}
+
+	tba = kmalloc(sizeof(*tba), GFP_KERNEL);
+	if (!tba)
+		return;
+
+	tba->f = (int (*)(void *))esp;
+	tba->arg = (void *)unused;
+	tba->ti = ti;
+
+	p->thread.arch.tid = lkl_thread_create(thread_bootstrap, tba);
+	if (!p->thread.arch.tid) {
+		kfree(tba);
+		return;
+	}
+}
+
+void show_stack(struct task_struct *task, unsigned long *esp)
+{
+}
+
+static inline void pr_early(const char *str)
+{
+	lkl_print(str, strlen(str));
+}
+
+/**
+ * This is called before the kernel initializes, so no kernel calls (including
+ * printk) can't be made yet.
+ */
+void threads_init(void)
+{
+	int ret;
+	struct thread_info *ti = &init_thread_union.thread_info;
+
+	ti->task->thread = (struct thread_struct) INIT_THREAD;
+	ret = init_arch_thread(&ti->task->thread.arch);
+	if (ret < 0)
+		pr_early("lkl: failed to allocate init schedule semaphore\n");
+
+	ti->task->thread.arch.tid = lkl_thread_self();
+}
+
+void threads_cleanup(void)
+{
+	struct task_struct *p, *t;
+
+	for_each_process_thread(p, t) {
+		struct thread_info *ti = task_thread_info(t);
+
+		if (t->pid != 1 && !test_ti_thread_flag(ti, TIF_HOST_THREAD))
+			WARN(!(t->flags & PF_KTHREAD),
+			     "non kernel thread task %s\n", t->comm);
+		WARN(t->state == TASK_RUNNING,
+		     "thread %s still running while halting\n", t->comm);
+
+		kill_thread(ti);
+	}
+
+	lkl_sem_free(
+		init_thread_union.thread_info.task->thread.arch.sched_sem);
+}
+
+void initial_thread_cb_skas(void (*proc)(void *), void *arg)
+{
+	pr_warn("unimplemented %s", __func__);
+}
+
+int arch_set_tls(struct task_struct *new, unsigned long tls)
+{
+	panic("unimplemented %s", __func__);
+}
+void clear_flushed_tls(struct task_struct *task)
+{
+	panic("unimplemented %s", __func__);
+}
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v8 09/20] um: lkl: kernel thread support
@ 2021-01-20  2:27             ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-01-20  2:27 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, thehajime, retrage01

library mode does not support user processes but it must support kernel
threads as part as the normal kernel work-flow. It uses host functions
to create and terminate host threads that are going to run the kernel
threads. It also uses semaphores to synchronize those threads and to
allow the Linux kernel scheduler to control how the kernel threads
run.

Each kernel thread runs in a host thread and has a host semaphore
associated with it - the thread's scheduling semaphore. The semaphore
counter is initialized to 0. The first thing a kernel thread does
after getting spawned, before running any kernel code, is to perform a
down operation to block the thread.

The kernel controls host threads scheduling by performing up and down
operations on the scheduling semaphore. In __switch_context an up
operation on the next thread is performed to wake up a blocked thread,
and a down operation is performed on the prev thread to block it.

A thread is terminated by marking it in free_thread_info and
performing an up operation on the scheduling semaphore at which point
the marked thread will terminate itself.

Kernel threads are executed with the synchronization to other context
primitivecs of kernel, such as idle thread, system calls, interrupts,
"reentrancy", CPU shutdown, imbalance wake up, which are not
controllable with a simple synchronization mechanism.  Thus we
introduced a logical CPU entity, struct lkl_cpu, to run Linux code with
multiple threads; CPU get/put functions (lkl_cpu_put/lkl_cpu_get) is
used to serialize access between LKL threads and host threads that
issue direct calls.

UML common part (process.c) is extended to call idle functions of
SUBARCH as well as kernel thread detection with a flag information of
TIF_HOST_THREAD.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/include/asm/thread_info.h       |  24 +++
 arch/um/kernel/process.c                |   8 +-
 arch/um/lkl/include/asm/cpu.h           |  11 +
 arch/um/lkl/include/asm/sched.h         |  23 +++
 arch/um/lkl/include/uapi/asm/host_ops.h | 164 +++++++++++++++
 arch/um/lkl/um/cpu.c                    | 223 ++++++++++++++++++++
 arch/um/lkl/um/threads.c                | 258 ++++++++++++++++++++++++
 7 files changed, 710 insertions(+), 1 deletion(-)
 create mode 100644 arch/um/lkl/include/asm/cpu.h
 create mode 100644 arch/um/lkl/include/asm/sched.h
 create mode 100644 arch/um/lkl/um/cpu.c
 create mode 100644 arch/um/lkl/um/threads.c

diff --git a/arch/um/include/asm/thread_info.h b/arch/um/include/asm/thread_info.h
index 3b1cb8b3b186..528e0347a9d8 100644
--- a/arch/um/include/asm/thread_info.h
+++ b/arch/um/include/asm/thread_info.h
@@ -40,6 +40,7 @@ struct thread_info {
 	.real_thread = NULL,			\
 }
 
+#ifndef CONFIG_UMMODE_LIB
 /* how to get the thread information struct from C */
 static inline struct thread_info *current_thread_info(void)
 {
@@ -52,6 +53,27 @@ static inline struct thread_info *current_thread_info(void)
 	return ti;
 }
 
+#else /* CONFIG_UMMODE_LIB */
+
+#define __HAVE_THREAD_FUNCTIONS
+#define task_thread_info(task)	((struct thread_info *)(task)->stack)
+#define task_stack_page(task)	((task)->stack)
+#define end_of_stack(p) (&task_thread_info(p)->aux_fp_regs[FP_SIZE-1])
+
+void threads_init(void);
+void threads_cleanup(void);
+
+unsigned long *alloc_thread_stack_node(struct task_struct *p, int node);
+void setup_thread_stack(struct task_struct *p, struct task_struct *org);
+void free_thread_stack(struct task_struct *tsk);
+
+extern struct thread_info *_current_thread_info;
+static inline struct thread_info *current_thread_info(void)
+{
+	return _current_thread_info;
+}
+#endif /* CONFIG_UMMODE_LIB */
+
 #endif
 
 #define TIF_SYSCALL_TRACE	0	/* syscall trace active */
@@ -64,6 +86,8 @@ static inline struct thread_info *current_thread_info(void)
 #define TIF_RESTORE_SIGMASK	7
 #define TIF_NOTIFY_RESUME	8
 #define TIF_SECCOMP		9	/* secure computing */
+#define TIF_SCHED_JB		10	/* thread is configured with jump buffer */
+#define TIF_HOST_THREAD	11	/* thread is a host task */
 
 #define _TIF_SYSCALL_TRACE	(1 << TIF_SYSCALL_TRACE)
 #define _TIF_SIGPENDING		(1 << TIF_SIGPENDING)
diff --git a/arch/um/kernel/process.c b/arch/um/kernel/process.c
index 9b0a36f64339..535ba738f2f5 100644
--- a/arch/um/kernel/process.c
+++ b/arch/um/kernel/process.c
@@ -160,7 +160,8 @@ int copy_thread(unsigned long clone_flags, unsigned long sp,
 		unsigned long arg, struct task_struct * p, unsigned long tls)
 {
 	void (*handler)(void);
-	int kthread = current->flags & PF_KTHREAD;
+	int kthread = current->flags & PF_KTHREAD ||
+		test_ti_thread_flag(current_thread_info(), TIF_HOST_THREAD);
 	int ret = 0;
 
 	p->thread = (struct thread_struct) INIT_THREAD;
@@ -214,10 +215,15 @@ void um_idle_sleep(void)
 		os_idle_sleep();
 }
 
+void __weak subarch_cpu_idle(void)
+{
+}
+
 void arch_cpu_idle(void)
 {
 	cpu_tasks[current_thread_info()->cpu].pid = os_getpid();
 	um_idle_sleep();
+	subarch_cpu_idle();
 	raw_local_irq_enable();
 }
 
diff --git a/arch/um/lkl/include/asm/cpu.h b/arch/um/lkl/include/asm/cpu.h
new file mode 100644
index 000000000000..c1164187151e
--- /dev/null
+++ b/arch/um/lkl/include/asm/cpu.h
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_LIBMODE_CPU_H
+#define __UM_LIBMODE_CPU_H
+
+int lkl_cpu_get(void);
+void lkl_cpu_put(void);
+int lkl_cpu_init(void);
+void lkl_cpu_wait_shutdown(void);
+void lkl_cpu_change_owner(lkl_thread_t owner);
+
+#endif
diff --git a/arch/um/lkl/include/asm/sched.h b/arch/um/lkl/include/asm/sched.h
new file mode 100644
index 000000000000..1ac063ed987c
--- /dev/null
+++ b/arch/um/lkl/include/asm/sched.h
@@ -0,0 +1,23 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_LIBMODE_SCHED_H
+#define __UM_LIBMODE_SCHED_H
+
+#include <linux/sched.h>
+#include <uapi/asm/host_ops.h>
+
+static inline void thread_sched_jb(void)
+{
+	if (test_ti_thread_flag(current_thread_info(), TIF_HOST_THREAD)) {
+		set_ti_thread_flag(current_thread_info(), TIF_SCHED_JB);
+		set_current_state(TASK_UNINTERRUPTIBLE);
+		lkl_jmp_buf_set(&current_thread_info()->task->thread.arch.sched_jb,
+				     schedule);
+	} else {
+		lkl_bug("%s can be used only for host task", __func__);
+	}
+}
+
+void switch_to_host_task(struct task_struct *);
+int host_task_stub(void *unused);
+
+#endif
diff --git a/arch/um/lkl/include/uapi/asm/host_ops.h b/arch/um/lkl/include/uapi/asm/host_ops.h
index 17219802fa1d..039a17972d7d 100644
--- a/arch/um/lkl/include/uapi/asm/host_ops.h
+++ b/arch/um/lkl/include/uapi/asm/host_ops.h
@@ -2,6 +2,13 @@
 #ifndef __UM_LIBMODE_UAPI_HOST_OPS_H
 #define __UM_LIBMODE_UAPI_HOST_OPS_H
 
+struct lkl_mutex;
+struct lkl_sem;
+typedef unsigned long lkl_thread_t;
+struct lkl_jmp_buf {
+	unsigned long buf[128];
+};
+
 /**
  * struct lkl_host_operations - host operations used by the Linux kernel
  *
@@ -36,4 +43,161 @@ void *lkl_mem_alloc(unsigned long mem);
  */
 void lkl_mem_free(void *mem);
 
+/**
+ * lkl_sem_alloc - allocate a host semaphore an initialize it to count
+ *
+ * @count: initial counter value
+ *
+ * Return: the pointer of allocated semaphore
+ *
+ */
+struct lkl_sem *lkl_sem_alloc(int count);
+
+/**
+ * lkl_sem_free - free a host semaphore
+ *
+ * @sem: the address of semaphore to be freed
+ *
+ */
+void lkl_sem_free(struct lkl_sem *sem);
+
+/**
+ * lkl_sem_up - perform an up operation on the semaphore
+ *
+ * @sem: semaphore pointer address
+ *
+ */
+void lkl_sem_up(struct lkl_sem *sem);
+
+/**
+ * lkl_sem_down - perform a down operation on the semaphore
+ *
+ * @sem: semaphore pointer address
+ *
+ */
+void lkl_sem_down(struct lkl_sem *sem);
+
+/**
+ * lkl_mutex_alloc - allocate and initialize a host mutex
+ *
+ * @recursive: boolean flag if the mutex is recursive or not
+ *
+ * Return: the pointer of allocated mutex
+ *
+ */
+struct lkl_mutex *lkl_mutex_alloc(int recursive);
+
+/**
+ * lkl_mutex_free - free a host mutex
+ *
+ * @mutex: the mutex pointer to be freed
+ *
+ */
+void lkl_mutex_lock(struct lkl_mutex *mutex);
+
+/**
+ * lkl_mutex_lock - acquire the mutex
+ *
+ * @_mutex: the mutex pointer to be locked
+ *
+ */
+void lkl_mutex_unlock(struct lkl_mutex *_mutex);
+
+/**
+ * lkl_mutex_unlock - release the mutex
+ *
+ * @_mutex: the mutex pointer to be released
+ *
+ */
+void lkl_mutex_free(struct lkl_mutex *_mutex);
+
+/**
+ * lkl_thread_create - create a new thread and run f(arg) in its context
+ *
+ * @fn: a start routine when creating a thread
+ * @arg: an argument for the function @fn
+ *
+ * Return: a thread handle or 0 if the thread could not be created
+ *
+ */
+lkl_thread_t lkl_thread_create(void* (*fn)(void *), void *arg);
+
+/**
+ * lkl_thread_detach - on POSIX systems, free up resources held by
+ * pthreads.
+ *
+ */
+void lkl_thread_detach(void);
+
+/**
+ * lkl_thread_exit - terminates the current thread
+ *
+ */
+void lkl_thread_exit(void);
+
+/**
+ * lkl_thread_join - wait for the given thread to terminate.
+ *
+ * @tid: thread id to wait
+ *
+ * Return: 0 for success, -1 otherwise
+ *
+ */
+int lkl_thread_join(lkl_thread_t tid);
+
+/**
+ * lkl_thread_self - returns the identifier of the current thread
+ *
+ * Return: the identifier of the current thread
+ *
+ */
+lkl_thread_t lkl_thread_self(void);
+
+/**
+ * lkl_thread_equal - compare if thread a and b are identical or not
+ *
+ * @a: a thread to be compared
+ * @b: another thread to be compared with @a
+ *
+ * Return: 1 if @a and @b are same; otherwise 0
+ *
+ */
+int lkl_thread_equal(lkl_thread_t a, lkl_thread_t b);
+
+/**
+ * lkl_gettid - obtain the thread identifier of the current thread
+ *
+ * Return: the host thread id of the caller, which need not be the same
+ * as the handle returned by thread_create
+ *
+ */
+long lkl_gettid(void);
+
+/**
+ * lkl_jmp_buf_set - set the jump buffer with a setup function
+ *
+ * @jmpb: jump buffer to be saved
+ * @f: a function to be called
+ *
+ * This runs the given function and setups a jump back point by saving
+ * the context in the jump buffer; jmp_buf_longjmp can be called from the give
+ * function or any callee in that function to return back to the jump back
+ * point.
+ *
+ * NOTE: we can't return from lkl_jmp_buf_set before calling lkl_jmp_buf_longjmp
+ * or otherwise the saved context (stack) is not going to be valid, so we must
+ * pass the function that will eventually call longjmp here.
+ *
+ */
+void lkl_jmp_buf_set(struct lkl_jmp_buf *jmpb, void (*f)(void));
+
+/**
+ * lkl_jmp_buf_longjmp - perform a jump back to the saved jump buffer
+ *
+ * @jmpb: jump buffer to be restored
+ * @val: returned value of longjmp
+ *
+ */
+void lkl_jmp_buf_longjmp(struct lkl_jmp_buf *jmpb, int val);
+
 #endif
diff --git a/arch/um/lkl/um/cpu.c b/arch/um/lkl/um/cpu.c
new file mode 100644
index 000000000000..75452b28d741
--- /dev/null
+++ b/arch/um/lkl/um/cpu.c
@@ -0,0 +1,223 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/kernel.h>
+#include <linux/sched/stat.h>
+#include <asm/host_ops.h>
+#include <asm/cpu.h>
+#include <asm/thread_info.h>
+#include <asm/unistd.h>
+#include <asm/sched.h>
+#include <asm/syscalls.h>
+#include <init.h>
+
+/*
+ * This structure is used to get access to the "LKL CPU" that allows us to run
+ * Linux code. Because we have to deal with various synchronization requirements
+ * between idle thread, system calls, interrupts, "reentrancy", CPU shutdown,
+ * imbalance wake up (i.e. acquire the CPU from one thread and release it from
+ * another), we can't use a simple synchronization mechanism such as (recursive)
+ * mutex or semaphore. Instead, we use a mutex and a bunch of status data plus a
+ * semaphore.
+ */
+static struct lkl_cpu {
+	/* lock that protects the CPU status data */
+	struct lkl_mutex *lock;
+	/*
+	 * Since we must free the cpu lock during shutdown we need a
+	 * synchronization algorithm between lkl_cpu_shutdown() and the CPU
+	 * access functions since lkl_cpu_get() gets called from thread
+	 * destructor callback functions which may be scheduled after
+	 * lkl_cpu_shutdown() has freed the cpu lock.
+	 *
+	 * An atomic counter is used to keep track of the number of running
+	 * CPU access functions and allow the shutdown function to wait for
+	 * them.
+	 *
+	 * The shutdown functions adds MAX_THREADS to this counter which allows
+	 * the CPU access functions to check if the shutdown process has
+	 * started.
+	 *
+	 * This algorithm assumes that we never have more the MAX_THREADS
+	 * requesting CPU access.
+	 */
+	#define MAX_THREADS 1000000
+	unsigned int shutdown_gate;
+	/* no of threads waiting the CPU */
+	unsigned int sleepers;
+	/* no of times the current thread got the CPU */
+	unsigned int count;
+	/* current thread that owns the CPU */
+	lkl_thread_t owner;
+	/* semaphore for threads waiting the CPU */
+	struct lkl_sem *sem;
+	/* semaphore used for shutdown */
+	struct lkl_sem *shutdown_sem;
+} cpu;
+
+/*
+ * internal routine to acquire LKL CPU's lock
+ */
+static int __cpu_try_get_lock(int n)
+{
+	lkl_thread_t self;
+
+	if (__sync_fetch_and_add(&cpu.shutdown_gate, n) >= MAX_THREADS)
+		return -2;
+
+	lkl_mutex_lock(cpu.lock);
+
+	if (cpu.shutdown_gate >= MAX_THREADS)
+		return -1;
+
+	self = lkl_thread_self();
+
+	/* if someone else is using the cpu, indicate as return 0 */
+	if (cpu.owner && !lkl_thread_equal(cpu.owner, self))
+		return 0;
+
+	/* set the owner of cpu */
+	cpu.owner = self;
+	cpu.count++;
+
+	return 1;
+}
+
+/*
+ * internal routine to release LKL CPU's lock
+ */
+static void __cpu_try_get_unlock(int lock_ret, int n)
+{
+	/* release lock only if __cpu_try_get_lock() holds cpu.lock
+	 * (returns >= -1)
+	 */
+	if (lock_ret >= -1)
+		lkl_mutex_unlock(cpu.lock);
+	__sync_fetch_and_sub(&cpu.shutdown_gate, n);
+}
+
+void lkl_cpu_change_owner(lkl_thread_t owner)
+{
+	lkl_mutex_lock(cpu.lock);
+	if (cpu.count > 1)
+		lkl_bug("bad count while changing owner\n");
+	cpu.owner = owner;
+	lkl_mutex_unlock(cpu.lock);
+}
+
+int lkl_cpu_get(void)
+{
+	int ret;
+
+	ret = __cpu_try_get_lock(1);
+
+	/* when somebody holds a lock, sleep until released,
+	 * with obtaining a semaphore (cpu.sem)
+	 */
+	while (ret == 0) {
+		cpu.sleepers++;
+		__cpu_try_get_unlock(ret, 0);
+		lkl_sem_down(cpu.sem);
+		ret = __cpu_try_get_lock(0);
+	}
+
+	__cpu_try_get_unlock(ret, 1);
+
+	return ret;
+}
+
+void lkl_cpu_put(void)
+{
+	lkl_mutex_lock(cpu.lock);
+
+	if (!cpu.count || !cpu.owner ||
+	    !lkl_thread_equal(cpu.owner, lkl_thread_self()))
+		lkl_bug("%s: unbalanced put\n", __func__);
+
+	/* switch to userspace code if current is host task (TIF_HOST_THREAD),
+	 * AND, there are other running tasks.
+	 */
+	if (test_ti_thread_flag(current_thread_info(), TIF_HOST_THREAD) &&
+	    !single_task_running() && cpu.count == 1) {
+		if (in_interrupt())
+			lkl_bug("%s: in interrupt\n", __func__);
+		lkl_mutex_unlock(cpu.lock);
+		thread_sched_jb();
+		return;
+	}
+
+	/* if there are any other tasks holding cpu lock, return after
+	 * decreasing cpu.count
+	 */
+	if (--cpu.count > 0) {
+		lkl_mutex_unlock(cpu.lock);
+		return;
+	}
+
+	/* release semaphore if slept */
+	if (cpu.sleepers) {
+		cpu.sleepers--;
+		lkl_sem_up(cpu.sem);
+	}
+
+	cpu.owner = 0;
+
+	lkl_mutex_unlock(cpu.lock);
+}
+
+static void lkl_cpu_shutdown(void)
+{
+	__sync_fetch_and_add(&cpu.shutdown_gate, MAX_THREADS);
+}
+__uml_exitcall(lkl_cpu_shutdown);
+
+void lkl_cpu_wait_shutdown(void)
+{
+	lkl_sem_down(cpu.shutdown_sem);
+	lkl_sem_free(cpu.shutdown_sem);
+}
+
+static void lkl_cpu_cleanup(bool shutdown)
+{
+	while (__sync_fetch_and_add(&cpu.shutdown_gate, 0) > MAX_THREADS)
+		;
+
+	/* if caller indicates shutdown, notify the semaphore to release
+	 * the block (lkl_cpu_wait_shutdown()).
+	 */
+	if (shutdown)
+		lkl_sem_up(cpu.shutdown_sem);
+	/* if lkl_cpu_wait_shutdown() is not called, free shutdown_sem here */
+	else if (cpu.shutdown_sem)
+		lkl_sem_free(cpu.shutdown_sem);
+
+	if (cpu.sem)
+		lkl_sem_free(cpu.sem);
+	if (cpu.lock)
+		lkl_mutex_free(cpu.lock);
+}
+
+void subarch_cpu_idle(void)
+{
+	if (cpu.shutdown_gate >= MAX_THREADS) {
+		lkl_mutex_lock(cpu.lock);
+		while (cpu.sleepers--)
+			lkl_sem_up(cpu.sem);
+		lkl_mutex_unlock(cpu.lock);
+
+		lkl_cpu_cleanup(true);
+		lkl_thread_exit();
+	}
+}
+
+int lkl_cpu_init(void)
+{
+	cpu.lock = lkl_mutex_alloc(0);
+	cpu.sem = lkl_sem_alloc(0);
+	cpu.shutdown_sem = lkl_sem_alloc(0);
+
+	if (!cpu.lock || !cpu.sem || !cpu.shutdown_sem) {
+		lkl_cpu_cleanup(false);
+		return -ENOMEM;
+	}
+
+	return 0;
+}
diff --git a/arch/um/lkl/um/threads.c b/arch/um/lkl/um/threads.c
new file mode 100644
index 000000000000..7ef9b9f2a6b7
--- /dev/null
+++ b/arch/um/lkl/um/threads.c
@@ -0,0 +1,258 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/slab.h>
+#include <linux/sched/task.h>
+#include <linux/sched/signal.h>
+#include <asm/host_ops.h>
+#include <asm/cpu.h>
+#include <asm/sched.h>
+
+#include <os.h>
+
+static int init_arch_thread(struct arch_thread *thread)
+{
+	thread->sched_sem = lkl_sem_alloc(0);
+	if (!thread->sched_sem)
+		return -ENOMEM;
+
+	thread->dead = false;
+	thread->tid = 0;
+
+	return 0;
+}
+
+unsigned long *alloc_thread_stack_node(struct task_struct *task, int node)
+{
+	struct thread_info *ti;
+
+	ti = kmalloc(sizeof(*ti), GFP_KERNEL | __GFP_ZERO);
+	if (!ti)
+		return NULL;
+
+	ti->task = task;
+	return (unsigned long *)ti;
+}
+
+/*
+ * The only new tasks created are kernel threads that have a predefined starting
+ * point thus no stack copy is required.
+ */
+void setup_thread_stack(struct task_struct *p, struct task_struct *org)
+{
+	struct thread_info *ti = task_thread_info(p);
+	struct thread_info *org_ti = task_thread_info(org);
+
+	ti->flags = org_ti->flags;
+	ti->preempt_count = org_ti->preempt_count;
+	ti->addr_limit = org_ti->addr_limit;
+}
+
+static void kill_thread(struct thread_info *ti)
+{
+	if (!test_ti_thread_flag(ti, TIF_HOST_THREAD)) {
+		ti->task->thread.arch.dead = true;
+		lkl_sem_up(ti->task->thread.arch.sched_sem);
+		lkl_thread_join(ti->task->thread.arch.tid);
+	}
+	lkl_sem_free(ti->task->thread.arch.sched_sem);
+}
+
+void free_thread_stack(struct task_struct *tsk)
+{
+	struct thread_info *ti = task_thread_info(tsk);
+
+	kill_thread(ti);
+	kfree(ti);
+}
+
+struct thread_info *_current_thread_info = &init_thread_union.thread_info;
+EXPORT_SYMBOL(_current_thread_info);
+
+void switch_threads(jmp_buf *me, jmp_buf *you)
+{
+	/* NOP */
+}
+
+/*
+ * schedule() expects the return of this function to be the task that we
+ * switched away from. Returning prev is not going to work because we are
+ * actually going to return the previous taks that was scheduled before the
+ * task we are going to wake up, and not the current task, e.g.:
+ *
+ * swapper -> init: saved prev on swapper stack is swapper
+ * init -> ksoftirqd0: saved prev on init stack is init
+ * ksoftirqd0 -> swapper: returned prev is swapper
+ */
+static struct task_struct *abs_prev = &init_task;
+
+void arch_switch_to(struct task_struct *prev,
+		    struct task_struct *next)
+{
+	struct arch_thread *_prev = &prev->thread.arch;
+	struct arch_thread *_next = &next->thread.arch;
+	unsigned long _prev_flags = task_thread_info(prev)->flags;
+	struct lkl_jmp_buf *_prev_jb;
+
+	_current_thread_info = task_thread_info(next);
+	next->thread.prev_sched = prev;
+	abs_prev = prev;
+
+	WARN_ON(!_next->tid);
+	lkl_cpu_change_owner(_next->tid);
+
+	if (test_bit(TIF_SCHED_JB, &_prev_flags)) {
+		/* Atomic. Must be done before wakeup next */
+		clear_ti_thread_flag(task_thread_info(prev), TIF_SCHED_JB);
+		_prev_jb = &_prev->sched_jb;
+	}
+
+	lkl_sem_up(_next->sched_sem);
+	if (test_bit(TIF_SCHED_JB, &_prev_flags))
+		lkl_jmp_buf_longjmp(_prev_jb, 1);
+	else
+		lkl_sem_down(_prev->sched_sem);
+
+	if (_prev->dead)
+		lkl_thread_exit();
+
+	/* __switch_to (arch/um) returns this value */
+	current->thread.prev_sched = abs_prev;
+}
+
+int host_task_stub(void *unused)
+{
+	return 0;
+}
+
+void switch_to_host_task(struct task_struct *task)
+{
+	if (WARN_ON(!test_tsk_thread_flag(task, TIF_HOST_THREAD)))
+		return;
+
+	task->thread.arch.tid = lkl_thread_self();
+
+	if (current == task)
+		return;
+
+	wake_up_process(task);
+	thread_sched_jb();
+	lkl_sem_down(task->thread.arch.sched_sem);
+	schedule_tail(abs_prev);
+}
+
+struct thread_bootstrap_arg {
+	struct thread_info *ti;
+	int (*f)(void *a);
+	void *arg;
+};
+
+static void *thread_bootstrap(void *_tba)
+{
+	struct thread_bootstrap_arg *tba = (struct thread_bootstrap_arg *)_tba;
+	struct thread_info *ti = tba->ti;
+	int (*f)(void *) = tba->f;
+	void *arg = tba->arg;
+
+	lkl_sem_down(ti->task->thread.arch.sched_sem);
+	kfree(tba);
+	if (ti->task->thread.prev_sched)
+		schedule_tail(ti->task->thread.prev_sched);
+
+	f(arg);
+	do_exit(0);
+
+	return NULL;
+}
+
+void new_thread(void *stack, jmp_buf *buf, void (*handler)(void))
+{
+	struct thread_info *ti = (struct thread_info *)stack;
+	struct task_struct *p = ti->task;
+	struct thread_bootstrap_arg *tba;
+	int ret;
+
+	unsigned long esp = (unsigned long)p->thread.request.u.thread.proc;
+	unsigned long unused = (unsigned long)p->thread.request.u.thread.arg;
+
+	ret = init_arch_thread(&p->thread.arch);
+	if (ret < 0)
+		panic("%s: init_arch_thread", __func__);
+
+	if ((int (*)(void *))esp == host_task_stub) {
+		set_ti_thread_flag(ti, TIF_HOST_THREAD);
+		return;
+	}
+
+	tba = kmalloc(sizeof(*tba), GFP_KERNEL);
+	if (!tba)
+		return;
+
+	tba->f = (int (*)(void *))esp;
+	tba->arg = (void *)unused;
+	tba->ti = ti;
+
+	p->thread.arch.tid = lkl_thread_create(thread_bootstrap, tba);
+	if (!p->thread.arch.tid) {
+		kfree(tba);
+		return;
+	}
+}
+
+void show_stack(struct task_struct *task, unsigned long *esp)
+{
+}
+
+static inline void pr_early(const char *str)
+{
+	lkl_print(str, strlen(str));
+}
+
+/**
+ * This is called before the kernel initializes, so no kernel calls (including
+ * printk) can't be made yet.
+ */
+void threads_init(void)
+{
+	int ret;
+	struct thread_info *ti = &init_thread_union.thread_info;
+
+	ti->task->thread = (struct thread_struct) INIT_THREAD;
+	ret = init_arch_thread(&ti->task->thread.arch);
+	if (ret < 0)
+		pr_early("lkl: failed to allocate init schedule semaphore\n");
+
+	ti->task->thread.arch.tid = lkl_thread_self();
+}
+
+void threads_cleanup(void)
+{
+	struct task_struct *p, *t;
+
+	for_each_process_thread(p, t) {
+		struct thread_info *ti = task_thread_info(t);
+
+		if (t->pid != 1 && !test_ti_thread_flag(ti, TIF_HOST_THREAD))
+			WARN(!(t->flags & PF_KTHREAD),
+			     "non kernel thread task %s\n", t->comm);
+		WARN(t->state == TASK_RUNNING,
+		     "thread %s still running while halting\n", t->comm);
+
+		kill_thread(ti);
+	}
+
+	lkl_sem_free(
+		init_thread_union.thread_info.task->thread.arch.sched_sem);
+}
+
+void initial_thread_cb_skas(void (*proc)(void *), void *arg)
+{
+	pr_warn("unimplemented %s", __func__);
+}
+
+int arch_set_tls(struct task_struct *new, unsigned long tls)
+{
+	panic("unimplemented %s", __func__);
+}
+void clear_flushed_tls(struct task_struct *task)
+{
+	panic("unimplemented %s", __func__);
+}
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v8 10/20] um: lkl: system call interface and application API
  2021-01-20  2:27           ` Hajime Tazaki
@ 2021-01-20  2:27             ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-01-20  2:27 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: thehajime, tavi.purdila, retrage01, linux-kernel-library,
	linux-arch, Masahiro Yamada

The application API is based on the kernel system call interface in
order to offer a stable API to applications. Note that we can't
offer the full Linux system call interface due to the limitations of
library mode such as lack of virtual memory, signal, user processes, etc.

The host is using dedicated kernel thread, which is switched upon the
entry of lkl_syscall so that numbers of context switches can be reduced
for the faster performance.

To expose the API definitions to applications, this commit uses
syscall_wrapper to define arch-specific information of syscall, and this
is used to generate syscall_defs.h for lkl so that it can be used
as a template of the list of syscall available for LKL apps.

To avoid collisions between the Linux API and the LKL API (e.g.  struct
stat, MKNOD, etc.) we use a python script to modify the user headers
and to prefix all of the global symbols (structures, typedefs,
defines) with LKL, lkl, _LKL, _lkl, __LKL or __lkl.

This commit also added an exception to SPDX-License-Identifier for the
library mode of UML, since applications may statically link the library,
thus this the GPL exception to the exported headers doesn't apply to.

Cc: Masahiro Yamada <masahiroy@kernel.org>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/Kconfig                            |   1 +
 arch/um/lkl/Makefile.um                    |  16 ++
 arch/um/lkl/include/asm/syscall_wrapper.h  |  57 ++++
 arch/um/lkl/include/asm/syscalls.h         |  15 +
 arch/um/lkl/include/uapi/asm/Kbuild        |   6 +
 arch/um/lkl/include/uapi/asm/bitsperlong.h |   5 +-
 arch/um/lkl/include/uapi/asm/byteorder.h   |   6 +-
 arch/um/lkl/include/uapi/asm/host_ops.h    |  43 +++
 arch/um/lkl/include/uapi/asm/syscalls.h    | 301 +++++++++++++++++++++
 arch/um/lkl/include/uapi/asm/unistd.h      |  15 +
 arch/um/lkl/um/syscalls.c                  | 193 +++++++++++++
 arch/um/scripts/headers_install.py         | 200 ++++++++++++++
 scripts/headers_install.sh                 |   6 +-
 13 files changed, 860 insertions(+), 4 deletions(-)
 create mode 100644 arch/um/lkl/include/asm/syscall_wrapper.h
 create mode 100644 arch/um/lkl/include/asm/syscalls.h
 create mode 100644 arch/um/lkl/include/uapi/asm/Kbuild
 create mode 100644 arch/um/lkl/include/uapi/asm/syscalls.h
 create mode 100644 arch/um/lkl/include/uapi/asm/unistd.h
 create mode 100644 arch/um/lkl/um/syscalls.c
 create mode 100755 arch/um/scripts/headers_install.py

diff --git a/arch/um/Kconfig b/arch/um/Kconfig
index 53c7919ec5b7..24c6596260de 100644
--- a/arch/um/Kconfig
+++ b/arch/um/Kconfig
@@ -28,6 +28,7 @@ config UMMODE_LIB
 	depends on !SMP
 	select UACCESS_MEMCPY
 	select ARCH_THREAD_STACK_ALLOCATOR
+	select ARCH_HAS_SYSCALL_WRAPPER
 	help
 	 This mode switches a mode to build a library of UML (Linux
 	 Kernel Library/LKL).  If this is Y, the build only generates,
diff --git a/arch/um/lkl/Makefile.um b/arch/um/lkl/Makefile.um
index 7d3f590d88a2..8c3f4f1a2032 100644
--- a/arch/um/lkl/Makefile.um
+++ b/arch/um/lkl/Makefile.um
@@ -1,2 +1,18 @@
 KBUILD_CFLAGS += -fno-builtin -fPIC
 ELF_FORMAT=$(shell $(LD) -r -print-output-format)
+
+INSTALL_PATH ?= $(objtree)/tools/um
+SYSCALL_DEFS_H := $(objtree)/$(HOST_DIR)/include/generated/uapi/asm/syscall_defs.h
+REPLACED_SYSCALL_DEFS_H := $(INSTALL_PATH)/include/lkl/asm/syscall_defs.h
+
+um_headers_install: $(REPLACED_SYSCALL_DEFS_H) scripts_unifdef
+
+$(REPLACED_SYSCALL_DEFS_H): $(SYSCALL_DEFS_H) $(srctree)/$(ARCH_DIR)/scripts/headers_install.py
+	$(Q)$(srctree)/$(ARCH_DIR)/scripts/headers_install.py \
+		$(INSTALL_PATH)/include
+
+$(objtree)/$(HOST_DIR)/include/generated/uapi/asm/syscall_defs.h: vmlinux
+	$(Q)$(OBJCOPY) -j .syscall_defs -O binary --set-section-flags \
+		       .syscall_defs=alloc $< $@
+	$(Q) export tmpfile=$(shell mktemp); \
+	sed 's/\x0//g' $@ > $$tmpfile; mv $$tmpfile $@ ; rm -f $$tmpfile
diff --git a/arch/um/lkl/include/asm/syscall_wrapper.h b/arch/um/lkl/include/asm/syscall_wrapper.h
new file mode 100644
index 000000000000..bb2a7d4274f6
--- /dev/null
+++ b/arch/um/lkl/include/asm/syscall_wrapper.h
@@ -0,0 +1,57 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef __UM_SYSCALL_WRAPPER_H
+#define __UM_SYSCALL_WRAPPER_H
+
+#define __SC_ASCII(t, a) #t "," #a
+
+#define __ASCII_MAP0(m, ...)
+#define __ASCII_MAP1(m, t, a) m(t, a)
+#define __ASCII_MAP2(m, t, a, ...) m(t, a) "," __ASCII_MAP1(m, __VA_ARGS__)
+#define __ASCII_MAP3(m, t, a, ...) m(t, a) "," __ASCII_MAP2(m, __VA_ARGS__)
+#define __ASCII_MAP4(m, t, a, ...) m(t, a) "," __ASCII_MAP3(m, __VA_ARGS__)
+#define __ASCII_MAP5(m, t, a, ...) m(t, a) "," __ASCII_MAP4(m, __VA_ARGS__)
+#define __ASCII_MAP6(m, t, a, ...) m(t, a) "," __ASCII_MAP5(m, __VA_ARGS__)
+#define __ASCII_MAP(n, ...) __ASCII_MAP##n(__VA_ARGS__)
+
+#ifdef __MINGW32__
+#define SECTION_ATTRS "n0"
+#else
+#define SECTION_ATTRS "a"
+#endif
+
+#define __SYSCALL_DEFINE_ARCH(x, name, ...)				\
+	asm(".section .syscall_defs,\"" SECTION_ATTRS "\"\n"		\
+	    ".ascii \"#ifdef __NR" #name "\\n\"\n"			\
+	    ".ascii \"SYSCALL_DEFINE" #x "(" #name ","			\
+	    __ASCII_MAP(x, __SC_ASCII, __VA_ARGS__) ")\\n\"\n"		\
+	    ".ascii \"#endif\\n\"\n"					\
+	    ".section .text\n")
+
+#define SYSCALL_DEFINE0(sname)					\
+	SYSCALL_METADATA(_##sname, 0);				\
+	__SYSCALL_DEFINE_ARCH(0, _##sname);			\
+	asmlinkage long sys_##sname(void);			\
+	ALLOW_ERROR_INJECTION(sys_##sname, ERRNO);		\
+	asmlinkage long sys_##sname(void)
+
+#define __SYSCALL_DEFINEx(x, name, ...)					\
+	__SYSCALL_DEFINE_ARCH(x, name, __VA_ARGS__);			\
+	__diag_push();							\
+	__diag_ignore(GCC, 8, "-Wattribute-alias",			\
+		      "Type aliasing is used to sanitize syscall arguments");\
+	asmlinkage long sys##name(__MAP(x, __SC_DECL, __VA_ARGS__))	\
+		__attribute__((alias(__stringify(__se_sys##name))));	\
+	ALLOW_ERROR_INJECTION(sys##name, ERRNO);			\
+	static inline long __do_sys##name(__MAP(x, __SC_DECL, __VA_ARGS__)); \
+	asmlinkage long __se_sys##name(__MAP(x, __SC_LONG, __VA_ARGS__)); \
+	asmlinkage long __se_sys##name(__MAP(x, __SC_LONG, __VA_ARGS__)) \
+	{								\
+		long ret = __do_sys##name(__MAP(x, __SC_CAST, __VA_ARGS__)); \
+		__MAP(x, __SC_TEST, __VA_ARGS__);			\
+		__PROTECT(x, ret, __MAP(x, __SC_ARGS, __VA_ARGS__));	\
+		return ret;						\
+	}								\
+	__diag_pop();							\
+	static inline long __do_sys##name(__MAP(x, __SC_DECL, __VA_ARGS__))
+
+#endif /* __UM_SYSCALL_WRAPPER_H */
diff --git a/arch/um/lkl/include/asm/syscalls.h b/arch/um/lkl/include/asm/syscalls.h
new file mode 100644
index 000000000000..6061d9415dad
--- /dev/null
+++ b/arch/um/lkl/include/asm/syscalls.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+int syscalls_init(void);
+void syscalls_cleanup(void);
+long lkl_syscall(long no, long *params);
+void wakeup_idle_host_task(void);
+
+#define sys_mmap sys_ni_syscall
+#define sys_rt_sigreturn sys_ni_syscall
+#define sys_arch_prctl sys_ni_syscall
+#define sys_iopl sys_ni_syscall
+#define sys_ioperm sys_ni_syscall
+#define sys_clone sys_ni_syscall
+
+int run_syscalls(void);
diff --git a/arch/um/lkl/include/uapi/asm/Kbuild b/arch/um/lkl/include/uapi/asm/Kbuild
new file mode 100644
index 000000000000..1c89a1640263
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/Kbuild
@@ -0,0 +1,6 @@
+# UAPI Header export list
+
+generated-y += syscall_defs.h
+
+# generated-y file is parsed by special user headers handling
+# see arch/um/script/headers_install.py
diff --git a/arch/um/lkl/include/uapi/asm/bitsperlong.h b/arch/um/lkl/include/uapi/asm/bitsperlong.h
index f6657ba0ff9b..b3a5adf0b04f 100644
--- a/arch/um/lkl/include/uapi/asm/bitsperlong.h
+++ b/arch/um/lkl/include/uapi/asm/bitsperlong.h
@@ -2,7 +2,10 @@
 #ifndef __UM_LIBMODE_UAPI_BITSPERLONG_H
 #define __UM_LIBMODE_UAPI_BITSPERLONG_H
 
-#ifdef CONFIG_64BIT
+/* need to add new arch defines here, as we cannot use CONFIG_64BIT here
+ * to avoid CONFIG leaks to userspace
+ */
+#if defined(__x86_64__)
 #define __BITS_PER_LONG 64
 #else
 #define __BITS_PER_LONG 32
diff --git a/arch/um/lkl/include/uapi/asm/byteorder.h b/arch/um/lkl/include/uapi/asm/byteorder.h
index a4ae973f440b..ed19f6a836c6 100644
--- a/arch/um/lkl/include/uapi/asm/byteorder.h
+++ b/arch/um/lkl/include/uapi/asm/byteorder.h
@@ -2,10 +2,12 @@
 #ifndef __UM_LIBMODE_UAPI_BYTEORDER_H
 #define __UM_LIBMODE_UAPI_BYTEORDER_H
 
-#if defined(CONFIG_BIG_ENDIAN)
+#if __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__
 #include <linux/byteorder/big_endian.h>
-#else
+#elif __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
 #include <linux/byteorder/little_endian.h>
+#else
+#error "cannot detect endian of the target"
 #endif
 
 #endif
diff --git a/arch/um/lkl/include/uapi/asm/host_ops.h b/arch/um/lkl/include/uapi/asm/host_ops.h
index 039a17972d7d..b97aa1099a17 100644
--- a/arch/um/lkl/include/uapi/asm/host_ops.h
+++ b/arch/um/lkl/include/uapi/asm/host_ops.h
@@ -200,4 +200,47 @@ void lkl_jmp_buf_set(struct lkl_jmp_buf *jmpb, void (*f)(void));
  */
 void lkl_jmp_buf_longjmp(struct lkl_jmp_buf *jmpb, int val);
 
+/**
+ * lkl_tls_alloc - allocate a thread local storage key
+ *
+ * @destructor: a destructor called upon termination. if this is not NULL it
+ * will be called when a thread terminates with its argument set to the current
+ * thread local storage value
+ *
+ * Return: non-NULL address of allocated struct lkl_tls_key if successful;
+ * otherwise NULL
+ *
+ */
+struct lkl_tls_key *lkl_tls_alloc(void (*destructor)(void *));
+
+/**
+ * lkl_tls_free - frees a thread local storage key
+ *
+ * @key: lkl_tls_key to be freed
+ *
+ */
+void lkl_tls_free(struct lkl_tls_key *key);
+
+/**
+ * lkl_tls_set - associate data to the thread local storage key
+ *
+ * @key: lkl_tls_key to be configured
+ * @data: the value associated with the @key
+ *
+ * Return: 0 if successful
+ *
+ */
+int lkl_tls_set(struct lkl_tls_key *key, void *data);
+
+/**
+ * lkl_tls_get - obtain the value associated with tls
+ *
+ * @key: lkl_tls_key to be queried
+ *
+ * Return: data associated with the thread local storage key or NULL
+ * on error
+ *
+ */
+void *lkl_tls_get(struct lkl_tls_key *key);
+
 #endif
diff --git a/arch/um/lkl/include/uapi/asm/syscalls.h b/arch/um/lkl/include/uapi/asm/syscalls.h
new file mode 100644
index 000000000000..62c343199625
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/syscalls.h
@@ -0,0 +1,301 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_LIBMODE_UAPI_SYSCALLS_H
+#define __UM_LIBMODE_UAPI_SYSCALLS_H
+
+#include <linux/types.h>
+
+typedef __kernel_uid32_t	qid_t;
+typedef __kernel_fd_set		fd_set;
+typedef __kernel_mode_t		mode_t;
+typedef unsigned short		umode_t;
+typedef __u32			nlink_t;
+typedef __kernel_off_t		off_t;
+typedef __kernel_pid_t		pid_t;
+typedef __kernel_key_t		key_t;
+typedef __kernel_suseconds_t	suseconds_t;
+typedef __kernel_timer_t	timer_t;
+typedef __kernel_clockid_t	clockid_t;
+typedef __kernel_mqd_t		mqd_t;
+typedef __kernel_uid32_t	uid_t;
+typedef __kernel_gid32_t	gid_t;
+typedef __kernel_uid16_t        uid16_t;
+typedef __kernel_gid16_t        gid16_t;
+typedef unsigned long		uintptr_t;
+
+typedef __kernel_loff_t		loff_t;
+typedef __kernel_size_t		size_t;
+typedef __kernel_ssize_t	ssize_t;
+typedef __kernel_time_t		time_t;
+typedef __kernel_clock_t	clock_t;
+typedef __u32			u32;
+typedef __s32			s32;
+typedef __u64			u64;
+typedef __s64			s64;
+
+typedef __kernel_long_t	__kernel_old_time_t;
+
+#define __user
+
+#include <asm/unistd.h>
+/* Temporary undefine system calls that don't have data types defined in UAPI
+ * headers
+ */
+#undef __NR_kexec_load
+#undef __NR_getcpu
+#undef __NR_sched_getattr
+#undef __NR_sched_setattr
+#undef __NR_sched_setparam
+#undef __NR_sched_getparam
+#undef __NR_sched_setscheduler
+#undef __NR_name_to_handle_at
+#undef __NR_open_by_handle_at
+
+/* deprecated system calls */
+#undef __NR_access
+#undef __NR_chmod
+#undef __NR_chown
+#undef __NR_lchown
+#undef __NR_open
+#undef __NR_creat
+#undef __NR_readlink
+#undef __NR_pipe
+#undef __NR_mknod
+#undef __NR_mkdir
+#undef __NR_rmdir
+#undef __NR_unlink
+#undef __NR_symlink
+#undef __NR_link
+#undef __NR_rename
+#undef __NR_getdents
+#undef __NR_select
+#undef __NR_poll
+#undef __NR_dup2
+#undef __NR_sysfs
+#undef __NR_ustat
+#undef __NR_inotify_init
+#undef __NR_epoll_create
+#undef __NR_epoll_wait
+#undef __NR_signalfd
+#undef __NR_eventfd
+
+#undef __NR_umount
+#define __NR_umount __NR_umount2
+
+#if defined(__x86_64__)
+#define __NR_newfstat __NR3264_fstat
+#define __NR_newfstatat __NR3264_fstatat
+#endif
+
+#include <linux/time.h>
+#include <linux/times.h>
+#include <linux/timex.h>
+#include <linux/capability.h>
+#define __KERNEL__ /* to pull in S_ definitions */
+#include <linux/stat.h>
+#undef __KERNEL__
+#include <linux/errno.h>
+#include <linux/fcntl.h>
+#include <linux/fs.h>
+#include <asm/statfs.h>
+#include <asm/stat.h>
+#include <linux/bpf.h>
+#include <linux/msg.h>
+#include <linux/resource.h>
+#include <linux/sysinfo.h>
+#include <linux/shm.h>
+#include <linux/aio_abi.h>
+#include <linux/socket.h>
+#include <linux/perf_event.h>
+#include <linux/sem.h>
+#include <linux/futex.h>
+#include <linux/poll.h>
+#include <linux/mqueue.h>
+#include <linux/eventpoll.h>
+#include <linux/uio.h>
+#include <asm/signal.h>
+#include <asm/siginfo.h>
+#include <linux/utime.h>
+#include <asm/socket.h>
+#include <linux/icmp.h>
+#include <linux/ip.h>
+
+/* Define data structures used in system calls that are not defined in UAPI
+ * headers
+ */
+struct sockaddr {
+	unsigned short int sa_family;
+	char sa_data[14];
+};
+
+#define __UAPI_DEF_IF_NET_DEVICE_FLAGS_LOWER_UP_DORMANT_ECHO 1
+#define __UAPI_DEF_IF_IFNAMSIZ	1
+#define __UAPI_DEF_IF_NET_DEVICE_FLAGS 1
+#define __UAPI_DEF_IF_IFREQ	1
+#define __UAPI_DEF_IF_IFMAP	1
+#include <linux/if.h>
+#define __UAPI_DEF_IN_IPPROTO	1
+#define __UAPI_DEF_IN_ADDR	1
+#define __UAPI_DEF_IN6_ADDR	1
+#define __UAPI_DEF_IP_MREQ	1
+#define __UAPI_DEF_IN_PKTINFO	1
+#define __UAPI_DEF_SOCKADDR_IN	1
+#define __UAPI_DEF_IN_CLASS	1
+#include <linux/in.h>
+#include <linux/in6.h>
+#include <linux/sockios.h>
+#include <linux/route.h>
+#include <linux/ipv6_route.h>
+#include <linux/ipv6.h>
+#include <linux/netlink.h>
+#include <linux/neighbour.h>
+#include <linux/rtnetlink.h>
+#include <linux/fib_rules.h>
+
+#include <linux/kdev_t.h>
+#include <linux/virtio_blk.h>
+#include <linux/virtio_net.h>
+#include <linux/virtio_ring.h>
+#include <linux/pkt_sched.h>
+#include <linux/io_uring.h>
+#include <linux/utsname.h>
+
+struct user_msghdr {
+	void		__user *msg_name;
+	int		msg_namelen;
+	struct iovec	__user *msg_iov;
+	__kernel_size_t	msg_iovlen;
+	void		__user *msg_control;
+	__kernel_size_t	msg_controllen;
+	unsigned int	msg_flags;
+};
+
+typedef __u32 key_serial_t;
+
+struct mmsghdr {
+	struct user_msghdr  msg_hdr;
+	unsigned int        msg_len;
+};
+
+struct linux_dirent64 {
+	u64		d_ino;
+	s64		d_off;
+	unsigned short	d_reclen;
+	unsigned char	d_type;
+	char		d_name[0];
+};
+
+struct linux_dirent {
+	unsigned long	d_ino;
+	unsigned long	d_off;
+	unsigned short	d_reclen;
+	char		d_name[1];
+};
+
+struct ustat {
+	__kernel_daddr_t	f_tfree;
+	__kernel_ino_t		f_tinode;
+	char			f_fname[6];
+	char			f_fpack[6];
+};
+
+struct __aio_sigset;
+struct clone_args;
+
+typedef __kernel_rwf_t		rwf_t;
+
+/**
+ * lkl_syscall() - issue LKL system calls
+ *
+ * @no: system call number
+ * @params: parameters of system calls
+ *
+ * Return: 0 if there is no error; otherwise non-zero value returns.
+ *
+ * The function is an entry point of LKL system call.
+ */
+long lkl_syscall(long no, long *params);
+
+/**
+ * lkl_sys_halt() - terminate LKL kernel
+ *
+ * Return: 0 if there is no error; otherwise non-zero value returns.
+ *
+ * The function requests the termination of LKL kernel.
+ */
+long lkl_sys_halt(void);
+
+#define __MAP0(m, ...)
+#define __MAP1(m, t, a) m(t, a)
+#define __MAP2(m, t, a, ...) m(t, a), __MAP1(m, __VA_ARGS__)
+#define __MAP3(m, t, a, ...) m(t, a), __MAP2(m, __VA_ARGS__)
+#define __MAP4(m, t, a, ...) m(t, a), __MAP3(m, __VA_ARGS__)
+#define __MAP5(m, t, a, ...) m(t, a), __MAP4(m, __VA_ARGS__)
+#define __MAP6(m, t, a, ...) m(t, a), __MAP5(m, __VA_ARGS__)
+#define __MAP(n, ...) __MAP##n(__VA_ARGS__)
+
+#define __SC_LONG(t, a) (long)(a)
+#define __SC_TABLE(t, a) {sizeof(t), (long long)(a)}
+#define __SC_DECL(t, a) t a
+
+#define LKL_SYSCALL0(name)					       \
+	static inline long lkl_sys##name(void)			       \
+	{							       \
+		long params[6];					       \
+		return lkl_syscall(__lkl__NR##name, params);	       \
+	}
+
+#if __BITS_PER_LONG == 32
+#define LKL_SYSCALLx(x, name, ...)					\
+	static inline							\
+	long lkl_sys##name(__MAP(x, __SC_DECL, __VA_ARGS__))		\
+	{								\
+		struct {						\
+			unsigned int size;				\
+			long long value;				\
+		} lkl_params[x] = { __MAP(x, __SC_TABLE, __VA_ARGS__) }; \
+		long sys_params[6], i, k;				\
+		for (i = k = 0; i < x && k < 6; i++, k++) {		\
+			if (lkl_params[i].size > sizeof(long) &&	\
+			    k + 1 < 6) {				\
+				sys_params[k] =				\
+					(long)(lkl_params[i].value & (-1UL)); \
+				k++;					\
+				sys_params[k] =				\
+					(long)(lkl_params[i].value >>	\
+					       __BITS_PER_LONG);	\
+			} else {					\
+				sys_params[k] = (long)(lkl_params[i].value); \
+			}						\
+		}							\
+		return lkl_syscall(__lkl__NR##name, sys_params);	\
+	}
+#else
+#define LKL_SYSCALLx(x, name, ...)					\
+	static inline							\
+	long lkl_sys##name(__MAP(x, __SC_DECL, __VA_ARGS__))		\
+	{								\
+		long lkl_params[6] = { __MAP(x, __SC_LONG, __VA_ARGS__) }; \
+		return lkl_syscall(__lkl__NR##name, lkl_params);	\
+	}
+#endif
+
+#define SYSCALL_DEFINE0(name, ...) LKL_SYSCALL0(name)
+#define SYSCALL_DEFINE1(name, ...) LKL_SYSCALLx(1, name, __VA_ARGS__)
+#define SYSCALL_DEFINE2(name, ...) LKL_SYSCALLx(2, name, __VA_ARGS__)
+#define SYSCALL_DEFINE3(name, ...) LKL_SYSCALLx(3, name, __VA_ARGS__)
+#define SYSCALL_DEFINE4(name, ...) LKL_SYSCALLx(4, name, __VA_ARGS__)
+#define SYSCALL_DEFINE5(name, ...) LKL_SYSCALLx(5, name, __VA_ARGS__)
+#define SYSCALL_DEFINE6(name, ...) LKL_SYSCALLx(6, name, __VA_ARGS__)
+
+#if __BITS_PER_LONG == 32
+#pragma GCC diagnostic push
+#pragma GCC diagnostic ignored "-Wpointer-to-int-cast"
+#endif
+
+#include <asm/syscall_defs.h>
+
+#if __BITS_PER_LONG == 32
+#pragma GCC diagnostic pop
+#endif
+
+#endif
diff --git a/arch/um/lkl/include/uapi/asm/unistd.h b/arch/um/lkl/include/uapi/asm/unistd.h
new file mode 100644
index 000000000000..6f5d67b45fbd
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/unistd.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_LIBMODE_UAPI_UNISTD_H
+#define __UM_LIBMODE_UAPI_UNISTD_H
+
+#define __ARCH_WANT_NEW_STAT
+#include <asm/bitsperlong.h>
+
+#if __BITS_PER_LONG == 64
+#define __ARCH_WANT_SYS_NEWFSTATAT
+#endif
+
+#include <asm-generic/unistd.h>
+
+
+#endif
diff --git a/arch/um/lkl/um/syscalls.c b/arch/um/lkl/um/syscalls.c
new file mode 100644
index 000000000000..45cb5c656f69
--- /dev/null
+++ b/arch/um/lkl/um/syscalls.c
@@ -0,0 +1,193 @@
+// SPDX-License-Identifier: GPL-2.0
+#ifdef CONFIG_ARCH_HAS_SYSCALL_WRAPPER
+#undef CONFIG_ARCH_HAS_SYSCALL_WRAPPER
+#include <linux/syscalls.h>
+#define CONFIG_ARCH_HAS_SYSCALL_WRAPPER
+#endif
+
+#include <linux/stat.h>
+#include <linux/irq.h>
+#include <linux/sched.h>
+#include <linux/interrupt.h>
+#include <linux/jhash.h>
+#include <linux/slab.h>
+#include <linux/types.h>
+#include <linux/net.h>
+#include <linux/task_work.h>
+
+#include <asm/host_ops.h>
+#include <asm/syscalls.h>
+#include <asm/cpu.h>
+#include <asm/sched.h>
+#include <kern_util.h>
+#include <os.h>
+
+typedef long (*syscall_handler_t)(long arg1, ...);
+
+#undef __SYSCALL
+#define __SYSCALL(nr, sym)[nr] = (syscall_handler_t)sym,
+
+syscall_handler_t syscall_table[__NR_syscalls] = {
+	[0 ... __NR_syscalls - 1] =  (syscall_handler_t)sys_ni_syscall,
+#undef __UM_LIBMODE_UAPI_UNISTD_H
+#include <asm/unistd.h>
+
+#if __BITS_PER_LONG == 32
+#include <asm/unistd_32.h>
+#endif
+};
+
+
+static long run_syscall(long no, long *params)
+{
+	long ret;
+
+	if (no < 0 || no >= __NR_syscalls)
+		return -ENOSYS;
+
+	ret = syscall_table[no](params[0], params[1], params[2], params[3],
+				params[4], params[5]);
+
+	task_work_run();
+
+	return ret;
+}
+
+
+#define CLONE_FLAGS (CLONE_VM | CLONE_FS | CLONE_FILES | CLONE_THREAD |	\
+		     CLONE_SIGHAND | SIGCHLD)
+
+static int host_task_id;
+static struct task_struct *host0;
+
+static int new_host_task(struct task_struct **task)
+{
+	pid_t pid;
+
+	switch_to_host_task(host0);
+
+	pid = kernel_thread(host_task_stub, NULL, CLONE_FLAGS);
+	if (pid < 0)
+		return pid;
+
+	rcu_read_lock();
+	*task = find_task_by_pid_ns(pid, &init_pid_ns);
+	rcu_read_unlock();
+
+	host_task_id++;
+
+	snprintf((*task)->comm, sizeof((*task)->comm), "host%d", host_task_id);
+
+	return 0;
+}
+static void exit_task(void)
+{
+	do_exit(0);
+}
+
+static void del_host_task(void *arg)
+{
+	struct task_struct *task = (struct task_struct *)arg;
+	struct thread_info *ti = task_thread_info(task);
+
+	if (lkl_cpu_get() < 0)
+		return;
+
+	switch_to_host_task(task);
+	host_task_id--;
+	set_ti_thread_flag(ti, TIF_SCHED_JB);
+	lkl_jmp_buf_set(&ti->task->thread.arch.sched_jb, exit_task);
+}
+
+static struct lkl_tls_key *task_key;
+
+long lkl_syscall(long no, long *params)
+{
+	struct task_struct *task = host0;
+	long ret;
+
+	ret = lkl_cpu_get();
+	if (ret < 0)
+		return ret;
+
+	task = lkl_tls_get(task_key);
+	if (!task) {
+		ret = new_host_task(&task);
+		if (ret)
+			goto out;
+		lkl_tls_set(task_key, task);
+	}
+
+	switch_to_host_task(task);
+
+	ret = run_syscall(no, params);
+
+	if (no == __NR_reboot) {
+		thread_sched_jb();
+		return ret;
+	}
+
+out:
+	lkl_cpu_put();
+
+	return ret;
+}
+
+static struct task_struct *idle_host_task;
+
+/* called from idle, don't failed, don't block */
+void wakeup_idle_host_task(void)
+{
+	if (!need_resched() && idle_host_task)
+		wake_up_process(idle_host_task);
+}
+
+static int idle_host_task_loop(void *unused)
+{
+	struct thread_info *ti = task_thread_info(current);
+
+	snprintf(current->comm, sizeof(current->comm), "idle_host_task");
+	set_thread_flag(TIF_HOST_THREAD);
+	idle_host_task = current;
+
+	for (;;) {
+		lkl_cpu_put();
+		lkl_sem_down(ti->task->thread.arch.sched_sem);
+		if (idle_host_task == NULL) {
+			lkl_thread_exit();
+			return 0;
+		}
+		schedule_tail(ti->task->thread.prev_sched);
+	}
+}
+
+int syscalls_init(void)
+{
+	snprintf(current->comm, sizeof(current->comm), "host0");
+	set_thread_flag(TIF_HOST_THREAD);
+	host0 = current;
+
+	task_key = lkl_tls_alloc(del_host_task);
+	if (!task_key)
+		return -1;
+
+	if (kernel_thread(idle_host_task_loop, NULL, CLONE_FLAGS) < 0) {
+		lkl_tls_free(task_key);
+		return -1;
+	}
+
+	return 0;
+}
+
+void syscalls_cleanup(void)
+{
+	if (idle_host_task) {
+		struct thread_info *ti = task_thread_info(idle_host_task);
+
+		idle_host_task = NULL;
+		lkl_sem_up(ti->task->thread.arch.sched_sem);
+		lkl_thread_join(ti->task->thread.arch.tid);
+	}
+
+	lkl_tls_free(task_key);
+}
diff --git a/arch/um/scripts/headers_install.py b/arch/um/scripts/headers_install.py
new file mode 100755
index 000000000000..0309f3478698
--- /dev/null
+++ b/arch/um/scripts/headers_install.py
@@ -0,0 +1,200 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+import re, os, sys, argparse, multiprocessing, fnmatch
+
+srctree = os.environ["srctree"]
+objtree = os.environ["objtree"]
+header_paths = [ "include/uapi/", "arch/um/lkl/include/uapi/",
+                 "arch/um/lkl/include/generated/uapi/" ]
+
+headers = set()
+includes = set()
+
+def relpath2abspath(relpath):
+    if "generated" in relpath:
+        return objtree + "/" + relpath
+    else:
+        return srctree + "/" + relpath
+
+def find_headers(path):
+    headers.add(path)
+    f = open(relpath2abspath(path))
+    for l in f.readlines():
+        m = re.search("#\s*include <(.*)>", l)
+        try:
+            i = m.group(1)
+            for p in header_paths:
+                if os.access(relpath2abspath(p + i), os.R_OK):
+                    if p + i not in headers:
+                        includes.add(i)
+                        headers.add(p + i)
+                        find_headers(p + i)
+        except:
+            pass
+    f.close()
+
+def has_lkl_prefix(w):
+  return w.startswith("lkl") or w.startswith("_lkl") or w.startswith("__lkl") \
+         or w.startswith("LKL") or w.startswith("_LKL") or w.startswith("__LKL")
+
+def find_symbols(regexp, store):
+    for h in headers:
+        f = open(h)
+        for l in f.readlines():
+            m = regexp.search(l)
+            if not m:
+                continue
+            for e in reversed(m.groups()):
+                if e:
+                    if not has_lkl_prefix(e):
+                        store.add(e)
+                    break
+        f.close()
+
+def find_ml_symbols(regexp, store):
+    for h in headers:
+        for i in regexp.finditer(open(h).read()):
+            for j in reversed(i.groups()):
+                if j:
+                    if not has_lkl_prefix(j):
+                        store.add(j)
+                    break
+
+def find_enums(block_regexp, symbol_regexp, store):
+    for h in headers:
+        # remove comments
+        content = re.sub(re.compile("(\/\*(\*(?!\/)|[^*])*\*\/)", re.S|re.M), " ", open(h).read())
+        # remove preprocesor lines
+        clean_content = ""
+        for l in content.split("\n"):
+            if re.match("\s*#", l):
+                continue
+            clean_content += l + "\n"
+        for i in block_regexp.finditer(clean_content):
+            for j in reversed(i.groups()):
+                if j:
+                    for k in symbol_regexp.finditer(j):
+                        for l in k.groups():
+                            if l:
+                                if not has_lkl_prefix(l):
+                                    store.add(l)
+                                break
+
+def lkl_prefix(w):
+    r = ""
+
+    if w.startswith("__"):
+        r = "__"
+    elif w.startswith("_"):
+        r = "_"
+
+    if w.isupper():
+        r += "LKL"
+    else:
+        r += "lkl"
+
+    if not w.startswith("_"):
+        r += "_"
+
+    r += w
+
+    return r
+
+def replace(h):
+    content = open(h).read()
+    for i in includes:
+        search_str = "(#[ \t]*include[ \t]*[<\"][ \t]*)" + i + "([ \t]*[>\"])"
+        replace_str = "\\1" + "lkl/" + i + "\\2"
+        content = re.sub(search_str, replace_str, content)
+    tmp = ""
+    for w in re.split("(\W+)", content):
+        if w in defines:
+            w = lkl_prefix(w)
+        tmp += w
+    content = tmp
+    for s in structs:
+        search_str = "(\W?struct\s+)" + s + "(\W)"
+        replace_str = "\\1" + lkl_prefix(s) + "\\2"
+        content = re.sub(search_str, replace_str, content, flags = re.MULTILINE)
+    for s in unions:
+        search_str = "(\W?union\s+)" + s + "(\W)"
+        replace_str = "\\1" + lkl_prefix(s) + "\\2"
+        content = re.sub(search_str, replace_str, content, flags = re.MULTILINE)
+    open(h, 'w').write(content)
+
+parser = argparse.ArgumentParser(description='install lkl headers')
+parser.add_argument('path', help='path to install to', )
+args = parser.parse_args()
+
+find_headers("arch/um/lkl/include/uapi/asm/syscalls.h")
+headers.add("arch/um/lkl/include/uapi/asm/host_ops.h")
+
+if 'LKL_INSTALL_ADDITIONAL_HEADERS' in os.environ:
+    with open(os.environ['LKL_INSTALL_ADDITIONAL_HEADERS'], 'rU') as f:
+        for line in f.readlines():
+            line = line.split('#', 1)[0].strip()
+            if line != '':
+                headers.add(line)
+
+new_headers = set()
+
+for h in headers:
+    h = relpath2abspath(h)
+    dir = os.path.dirname(h)
+    out_dir = args.path + "/" + re.sub("(" + srctree + "/arch/um/lkl/include/uapi/|"
+                                       + srctree + "/include/uapi/|"
+                                       + "arch/um/lkl/include/generated/uapi/|"
+                                       + "include/generated/uapi/|"
+                                       + "include/generated)(.*)", "lkl/\\2", dir)
+    try:
+        os.makedirs(out_dir)
+    except:
+        pass
+    print("  INSTALL %s" % (out_dir + "/" + os.path.basename(h)))
+    os.system(srctree+"/scripts/headers_install.sh %s %s" % (os.path.abspath(h),
+                                                       out_dir + "/" + os.path.basename(h)))
+    new_headers.add(out_dir + "/" + os.path.basename(h))
+
+headers = new_headers
+
+defines = set()
+structs = set()
+unions = set()
+
+p = re.compile("#[ \t]*define[ \t]*(\w+)")
+find_symbols(p, defines)
+p = re.compile("typedef.*(\(\*(\w+)\)\(.*\)\s*|\W+(\w+)\s*|\s+(\w+)\(.*\)\s*);")
+find_symbols(p, defines)
+p = re.compile("typedef\s+(struct|union)\s+\w*\s*{[^\\{\}]*}\W*(\w+)\s*;", re.M|re.S)
+find_ml_symbols(p, defines)
+defines.add("siginfo_t")
+defines.add("sigevent_t")
+p = re.compile("struct\s+(\w+)\s*\{")
+find_symbols(p, structs)
+structs.add("iovec")
+p = re.compile("union\s+(\w+)\s*\{")
+find_symbols(p, unions)
+p = re.compile("static\s+__inline__(\s+\w+)+\s+(\w+)\([^)]*\)\s")
+find_symbols(p, defines)
+p = re.compile("static\s+__always_inline(\s+\w+)+\s+(\w+)\([^)]*\)\s")
+find_symbols(p, defines)
+p = re.compile("enum\s+(\w*)\s*{([^}]*)}", re.M|re.S)
+q = re.compile("(\w+)\s*(,|=[^,]*|$)", re.M|re.S)
+find_enums(p, q, defines)
+
+# needed for i386
+defines.add("__NR_stime")
+
+def process_header(h):
+    print("  REPLACE %s" % h)
+    replace(h)
+
+# use num of processors determined by os.cpu_count()
+p = multiprocessing.Pool(None)
+try:
+    p.map_async(process_header, headers).wait(999999)
+    p.close()
+except:
+    p.terminate()
+finally:
+    p.join()
diff --git a/scripts/headers_install.sh b/scripts/headers_install.sh
index dd554bd436cc..d44b92979db4 100755
--- a/scripts/headers_install.sh
+++ b/scripts/headers_install.sh
@@ -24,7 +24,11 @@ TMPFILE=$OUTFILE.tmp
 trap 'rm -f $OUTFILE $TMPFILE' EXIT
 
 # SPDX-License-Identifier with GPL variants must have "WITH Linux-syscall-note"
-if [ -n "$(sed -n -e "/SPDX-License-Identifier:.*GPL-/{/WITH Linux-syscall-note/!p}" $INFILE)" ]; then
+#
+# ARCH=um with library mode exposes uapi headers but applications may be linked
+# statically, thus this GPL exception doesn't apply to.
+if [ -n "$(sed -n -e "/SPDX-License-Identifier:.*GPL-/{/WITH Linux-syscall-note/!p}" $INFILE)" ] &&
+	[ "$SRCARCH" != "um" ]; then
 	echo "error: $INFILE: missing \"WITH Linux-syscall-note\" for SPDX-License-Identifier" >&2
 	exit 1
 fi
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v8 10/20] um: lkl: system call interface and application API
@ 2021-01-20  2:27             ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-01-20  2:27 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: linux-arch, tavi.purdila, Masahiro Yamada, retrage01,
	linux-kernel-library, thehajime

The application API is based on the kernel system call interface in
order to offer a stable API to applications. Note that we can't
offer the full Linux system call interface due to the limitations of
library mode such as lack of virtual memory, signal, user processes, etc.

The host is using dedicated kernel thread, which is switched upon the
entry of lkl_syscall so that numbers of context switches can be reduced
for the faster performance.

To expose the API definitions to applications, this commit uses
syscall_wrapper to define arch-specific information of syscall, and this
is used to generate syscall_defs.h for lkl so that it can be used
as a template of the list of syscall available for LKL apps.

To avoid collisions between the Linux API and the LKL API (e.g.  struct
stat, MKNOD, etc.) we use a python script to modify the user headers
and to prefix all of the global symbols (structures, typedefs,
defines) with LKL, lkl, _LKL, _lkl, __LKL or __lkl.

This commit also added an exception to SPDX-License-Identifier for the
library mode of UML, since applications may statically link the library,
thus this the GPL exception to the exported headers doesn't apply to.

Cc: Masahiro Yamada <masahiroy@kernel.org>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/Kconfig                            |   1 +
 arch/um/lkl/Makefile.um                    |  16 ++
 arch/um/lkl/include/asm/syscall_wrapper.h  |  57 ++++
 arch/um/lkl/include/asm/syscalls.h         |  15 +
 arch/um/lkl/include/uapi/asm/Kbuild        |   6 +
 arch/um/lkl/include/uapi/asm/bitsperlong.h |   5 +-
 arch/um/lkl/include/uapi/asm/byteorder.h   |   6 +-
 arch/um/lkl/include/uapi/asm/host_ops.h    |  43 +++
 arch/um/lkl/include/uapi/asm/syscalls.h    | 301 +++++++++++++++++++++
 arch/um/lkl/include/uapi/asm/unistd.h      |  15 +
 arch/um/lkl/um/syscalls.c                  | 193 +++++++++++++
 arch/um/scripts/headers_install.py         | 200 ++++++++++++++
 scripts/headers_install.sh                 |   6 +-
 13 files changed, 860 insertions(+), 4 deletions(-)
 create mode 100644 arch/um/lkl/include/asm/syscall_wrapper.h
 create mode 100644 arch/um/lkl/include/asm/syscalls.h
 create mode 100644 arch/um/lkl/include/uapi/asm/Kbuild
 create mode 100644 arch/um/lkl/include/uapi/asm/syscalls.h
 create mode 100644 arch/um/lkl/include/uapi/asm/unistd.h
 create mode 100644 arch/um/lkl/um/syscalls.c
 create mode 100755 arch/um/scripts/headers_install.py

diff --git a/arch/um/Kconfig b/arch/um/Kconfig
index 53c7919ec5b7..24c6596260de 100644
--- a/arch/um/Kconfig
+++ b/arch/um/Kconfig
@@ -28,6 +28,7 @@ config UMMODE_LIB
 	depends on !SMP
 	select UACCESS_MEMCPY
 	select ARCH_THREAD_STACK_ALLOCATOR
+	select ARCH_HAS_SYSCALL_WRAPPER
 	help
 	 This mode switches a mode to build a library of UML (Linux
 	 Kernel Library/LKL).  If this is Y, the build only generates,
diff --git a/arch/um/lkl/Makefile.um b/arch/um/lkl/Makefile.um
index 7d3f590d88a2..8c3f4f1a2032 100644
--- a/arch/um/lkl/Makefile.um
+++ b/arch/um/lkl/Makefile.um
@@ -1,2 +1,18 @@
 KBUILD_CFLAGS += -fno-builtin -fPIC
 ELF_FORMAT=$(shell $(LD) -r -print-output-format)
+
+INSTALL_PATH ?= $(objtree)/tools/um
+SYSCALL_DEFS_H := $(objtree)/$(HOST_DIR)/include/generated/uapi/asm/syscall_defs.h
+REPLACED_SYSCALL_DEFS_H := $(INSTALL_PATH)/include/lkl/asm/syscall_defs.h
+
+um_headers_install: $(REPLACED_SYSCALL_DEFS_H) scripts_unifdef
+
+$(REPLACED_SYSCALL_DEFS_H): $(SYSCALL_DEFS_H) $(srctree)/$(ARCH_DIR)/scripts/headers_install.py
+	$(Q)$(srctree)/$(ARCH_DIR)/scripts/headers_install.py \
+		$(INSTALL_PATH)/include
+
+$(objtree)/$(HOST_DIR)/include/generated/uapi/asm/syscall_defs.h: vmlinux
+	$(Q)$(OBJCOPY) -j .syscall_defs -O binary --set-section-flags \
+		       .syscall_defs=alloc $< $@
+	$(Q) export tmpfile=$(shell mktemp); \
+	sed 's/\x0//g' $@ > $$tmpfile; mv $$tmpfile $@ ; rm -f $$tmpfile
diff --git a/arch/um/lkl/include/asm/syscall_wrapper.h b/arch/um/lkl/include/asm/syscall_wrapper.h
new file mode 100644
index 000000000000..bb2a7d4274f6
--- /dev/null
+++ b/arch/um/lkl/include/asm/syscall_wrapper.h
@@ -0,0 +1,57 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef __UM_SYSCALL_WRAPPER_H
+#define __UM_SYSCALL_WRAPPER_H
+
+#define __SC_ASCII(t, a) #t "," #a
+
+#define __ASCII_MAP0(m, ...)
+#define __ASCII_MAP1(m, t, a) m(t, a)
+#define __ASCII_MAP2(m, t, a, ...) m(t, a) "," __ASCII_MAP1(m, __VA_ARGS__)
+#define __ASCII_MAP3(m, t, a, ...) m(t, a) "," __ASCII_MAP2(m, __VA_ARGS__)
+#define __ASCII_MAP4(m, t, a, ...) m(t, a) "," __ASCII_MAP3(m, __VA_ARGS__)
+#define __ASCII_MAP5(m, t, a, ...) m(t, a) "," __ASCII_MAP4(m, __VA_ARGS__)
+#define __ASCII_MAP6(m, t, a, ...) m(t, a) "," __ASCII_MAP5(m, __VA_ARGS__)
+#define __ASCII_MAP(n, ...) __ASCII_MAP##n(__VA_ARGS__)
+
+#ifdef __MINGW32__
+#define SECTION_ATTRS "n0"
+#else
+#define SECTION_ATTRS "a"
+#endif
+
+#define __SYSCALL_DEFINE_ARCH(x, name, ...)				\
+	asm(".section .syscall_defs,\"" SECTION_ATTRS "\"\n"		\
+	    ".ascii \"#ifdef __NR" #name "\\n\"\n"			\
+	    ".ascii \"SYSCALL_DEFINE" #x "(" #name ","			\
+	    __ASCII_MAP(x, __SC_ASCII, __VA_ARGS__) ")\\n\"\n"		\
+	    ".ascii \"#endif\\n\"\n"					\
+	    ".section .text\n")
+
+#define SYSCALL_DEFINE0(sname)					\
+	SYSCALL_METADATA(_##sname, 0);				\
+	__SYSCALL_DEFINE_ARCH(0, _##sname);			\
+	asmlinkage long sys_##sname(void);			\
+	ALLOW_ERROR_INJECTION(sys_##sname, ERRNO);		\
+	asmlinkage long sys_##sname(void)
+
+#define __SYSCALL_DEFINEx(x, name, ...)					\
+	__SYSCALL_DEFINE_ARCH(x, name, __VA_ARGS__);			\
+	__diag_push();							\
+	__diag_ignore(GCC, 8, "-Wattribute-alias",			\
+		      "Type aliasing is used to sanitize syscall arguments");\
+	asmlinkage long sys##name(__MAP(x, __SC_DECL, __VA_ARGS__))	\
+		__attribute__((alias(__stringify(__se_sys##name))));	\
+	ALLOW_ERROR_INJECTION(sys##name, ERRNO);			\
+	static inline long __do_sys##name(__MAP(x, __SC_DECL, __VA_ARGS__)); \
+	asmlinkage long __se_sys##name(__MAP(x, __SC_LONG, __VA_ARGS__)); \
+	asmlinkage long __se_sys##name(__MAP(x, __SC_LONG, __VA_ARGS__)) \
+	{								\
+		long ret = __do_sys##name(__MAP(x, __SC_CAST, __VA_ARGS__)); \
+		__MAP(x, __SC_TEST, __VA_ARGS__);			\
+		__PROTECT(x, ret, __MAP(x, __SC_ARGS, __VA_ARGS__));	\
+		return ret;						\
+	}								\
+	__diag_pop();							\
+	static inline long __do_sys##name(__MAP(x, __SC_DECL, __VA_ARGS__))
+
+#endif /* __UM_SYSCALL_WRAPPER_H */
diff --git a/arch/um/lkl/include/asm/syscalls.h b/arch/um/lkl/include/asm/syscalls.h
new file mode 100644
index 000000000000..6061d9415dad
--- /dev/null
+++ b/arch/um/lkl/include/asm/syscalls.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+int syscalls_init(void);
+void syscalls_cleanup(void);
+long lkl_syscall(long no, long *params);
+void wakeup_idle_host_task(void);
+
+#define sys_mmap sys_ni_syscall
+#define sys_rt_sigreturn sys_ni_syscall
+#define sys_arch_prctl sys_ni_syscall
+#define sys_iopl sys_ni_syscall
+#define sys_ioperm sys_ni_syscall
+#define sys_clone sys_ni_syscall
+
+int run_syscalls(void);
diff --git a/arch/um/lkl/include/uapi/asm/Kbuild b/arch/um/lkl/include/uapi/asm/Kbuild
new file mode 100644
index 000000000000..1c89a1640263
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/Kbuild
@@ -0,0 +1,6 @@
+# UAPI Header export list
+
+generated-y += syscall_defs.h
+
+# generated-y file is parsed by special user headers handling
+# see arch/um/script/headers_install.py
diff --git a/arch/um/lkl/include/uapi/asm/bitsperlong.h b/arch/um/lkl/include/uapi/asm/bitsperlong.h
index f6657ba0ff9b..b3a5adf0b04f 100644
--- a/arch/um/lkl/include/uapi/asm/bitsperlong.h
+++ b/arch/um/lkl/include/uapi/asm/bitsperlong.h
@@ -2,7 +2,10 @@
 #ifndef __UM_LIBMODE_UAPI_BITSPERLONG_H
 #define __UM_LIBMODE_UAPI_BITSPERLONG_H
 
-#ifdef CONFIG_64BIT
+/* need to add new arch defines here, as we cannot use CONFIG_64BIT here
+ * to avoid CONFIG leaks to userspace
+ */
+#if defined(__x86_64__)
 #define __BITS_PER_LONG 64
 #else
 #define __BITS_PER_LONG 32
diff --git a/arch/um/lkl/include/uapi/asm/byteorder.h b/arch/um/lkl/include/uapi/asm/byteorder.h
index a4ae973f440b..ed19f6a836c6 100644
--- a/arch/um/lkl/include/uapi/asm/byteorder.h
+++ b/arch/um/lkl/include/uapi/asm/byteorder.h
@@ -2,10 +2,12 @@
 #ifndef __UM_LIBMODE_UAPI_BYTEORDER_H
 #define __UM_LIBMODE_UAPI_BYTEORDER_H
 
-#if defined(CONFIG_BIG_ENDIAN)
+#if __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__
 #include <linux/byteorder/big_endian.h>
-#else
+#elif __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
 #include <linux/byteorder/little_endian.h>
+#else
+#error "cannot detect endian of the target"
 #endif
 
 #endif
diff --git a/arch/um/lkl/include/uapi/asm/host_ops.h b/arch/um/lkl/include/uapi/asm/host_ops.h
index 039a17972d7d..b97aa1099a17 100644
--- a/arch/um/lkl/include/uapi/asm/host_ops.h
+++ b/arch/um/lkl/include/uapi/asm/host_ops.h
@@ -200,4 +200,47 @@ void lkl_jmp_buf_set(struct lkl_jmp_buf *jmpb, void (*f)(void));
  */
 void lkl_jmp_buf_longjmp(struct lkl_jmp_buf *jmpb, int val);
 
+/**
+ * lkl_tls_alloc - allocate a thread local storage key
+ *
+ * @destructor: a destructor called upon termination. if this is not NULL it
+ * will be called when a thread terminates with its argument set to the current
+ * thread local storage value
+ *
+ * Return: non-NULL address of allocated struct lkl_tls_key if successful;
+ * otherwise NULL
+ *
+ */
+struct lkl_tls_key *lkl_tls_alloc(void (*destructor)(void *));
+
+/**
+ * lkl_tls_free - frees a thread local storage key
+ *
+ * @key: lkl_tls_key to be freed
+ *
+ */
+void lkl_tls_free(struct lkl_tls_key *key);
+
+/**
+ * lkl_tls_set - associate data to the thread local storage key
+ *
+ * @key: lkl_tls_key to be configured
+ * @data: the value associated with the @key
+ *
+ * Return: 0 if successful
+ *
+ */
+int lkl_tls_set(struct lkl_tls_key *key, void *data);
+
+/**
+ * lkl_tls_get - obtain the value associated with tls
+ *
+ * @key: lkl_tls_key to be queried
+ *
+ * Return: data associated with the thread local storage key or NULL
+ * on error
+ *
+ */
+void *lkl_tls_get(struct lkl_tls_key *key);
+
 #endif
diff --git a/arch/um/lkl/include/uapi/asm/syscalls.h b/arch/um/lkl/include/uapi/asm/syscalls.h
new file mode 100644
index 000000000000..62c343199625
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/syscalls.h
@@ -0,0 +1,301 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_LIBMODE_UAPI_SYSCALLS_H
+#define __UM_LIBMODE_UAPI_SYSCALLS_H
+
+#include <linux/types.h>
+
+typedef __kernel_uid32_t	qid_t;
+typedef __kernel_fd_set		fd_set;
+typedef __kernel_mode_t		mode_t;
+typedef unsigned short		umode_t;
+typedef __u32			nlink_t;
+typedef __kernel_off_t		off_t;
+typedef __kernel_pid_t		pid_t;
+typedef __kernel_key_t		key_t;
+typedef __kernel_suseconds_t	suseconds_t;
+typedef __kernel_timer_t	timer_t;
+typedef __kernel_clockid_t	clockid_t;
+typedef __kernel_mqd_t		mqd_t;
+typedef __kernel_uid32_t	uid_t;
+typedef __kernel_gid32_t	gid_t;
+typedef __kernel_uid16_t        uid16_t;
+typedef __kernel_gid16_t        gid16_t;
+typedef unsigned long		uintptr_t;
+
+typedef __kernel_loff_t		loff_t;
+typedef __kernel_size_t		size_t;
+typedef __kernel_ssize_t	ssize_t;
+typedef __kernel_time_t		time_t;
+typedef __kernel_clock_t	clock_t;
+typedef __u32			u32;
+typedef __s32			s32;
+typedef __u64			u64;
+typedef __s64			s64;
+
+typedef __kernel_long_t	__kernel_old_time_t;
+
+#define __user
+
+#include <asm/unistd.h>
+/* Temporary undefine system calls that don't have data types defined in UAPI
+ * headers
+ */
+#undef __NR_kexec_load
+#undef __NR_getcpu
+#undef __NR_sched_getattr
+#undef __NR_sched_setattr
+#undef __NR_sched_setparam
+#undef __NR_sched_getparam
+#undef __NR_sched_setscheduler
+#undef __NR_name_to_handle_at
+#undef __NR_open_by_handle_at
+
+/* deprecated system calls */
+#undef __NR_access
+#undef __NR_chmod
+#undef __NR_chown
+#undef __NR_lchown
+#undef __NR_open
+#undef __NR_creat
+#undef __NR_readlink
+#undef __NR_pipe
+#undef __NR_mknod
+#undef __NR_mkdir
+#undef __NR_rmdir
+#undef __NR_unlink
+#undef __NR_symlink
+#undef __NR_link
+#undef __NR_rename
+#undef __NR_getdents
+#undef __NR_select
+#undef __NR_poll
+#undef __NR_dup2
+#undef __NR_sysfs
+#undef __NR_ustat
+#undef __NR_inotify_init
+#undef __NR_epoll_create
+#undef __NR_epoll_wait
+#undef __NR_signalfd
+#undef __NR_eventfd
+
+#undef __NR_umount
+#define __NR_umount __NR_umount2
+
+#if defined(__x86_64__)
+#define __NR_newfstat __NR3264_fstat
+#define __NR_newfstatat __NR3264_fstatat
+#endif
+
+#include <linux/time.h>
+#include <linux/times.h>
+#include <linux/timex.h>
+#include <linux/capability.h>
+#define __KERNEL__ /* to pull in S_ definitions */
+#include <linux/stat.h>
+#undef __KERNEL__
+#include <linux/errno.h>
+#include <linux/fcntl.h>
+#include <linux/fs.h>
+#include <asm/statfs.h>
+#include <asm/stat.h>
+#include <linux/bpf.h>
+#include <linux/msg.h>
+#include <linux/resource.h>
+#include <linux/sysinfo.h>
+#include <linux/shm.h>
+#include <linux/aio_abi.h>
+#include <linux/socket.h>
+#include <linux/perf_event.h>
+#include <linux/sem.h>
+#include <linux/futex.h>
+#include <linux/poll.h>
+#include <linux/mqueue.h>
+#include <linux/eventpoll.h>
+#include <linux/uio.h>
+#include <asm/signal.h>
+#include <asm/siginfo.h>
+#include <linux/utime.h>
+#include <asm/socket.h>
+#include <linux/icmp.h>
+#include <linux/ip.h>
+
+/* Define data structures used in system calls that are not defined in UAPI
+ * headers
+ */
+struct sockaddr {
+	unsigned short int sa_family;
+	char sa_data[14];
+};
+
+#define __UAPI_DEF_IF_NET_DEVICE_FLAGS_LOWER_UP_DORMANT_ECHO 1
+#define __UAPI_DEF_IF_IFNAMSIZ	1
+#define __UAPI_DEF_IF_NET_DEVICE_FLAGS 1
+#define __UAPI_DEF_IF_IFREQ	1
+#define __UAPI_DEF_IF_IFMAP	1
+#include <linux/if.h>
+#define __UAPI_DEF_IN_IPPROTO	1
+#define __UAPI_DEF_IN_ADDR	1
+#define __UAPI_DEF_IN6_ADDR	1
+#define __UAPI_DEF_IP_MREQ	1
+#define __UAPI_DEF_IN_PKTINFO	1
+#define __UAPI_DEF_SOCKADDR_IN	1
+#define __UAPI_DEF_IN_CLASS	1
+#include <linux/in.h>
+#include <linux/in6.h>
+#include <linux/sockios.h>
+#include <linux/route.h>
+#include <linux/ipv6_route.h>
+#include <linux/ipv6.h>
+#include <linux/netlink.h>
+#include <linux/neighbour.h>
+#include <linux/rtnetlink.h>
+#include <linux/fib_rules.h>
+
+#include <linux/kdev_t.h>
+#include <linux/virtio_blk.h>
+#include <linux/virtio_net.h>
+#include <linux/virtio_ring.h>
+#include <linux/pkt_sched.h>
+#include <linux/io_uring.h>
+#include <linux/utsname.h>
+
+struct user_msghdr {
+	void		__user *msg_name;
+	int		msg_namelen;
+	struct iovec	__user *msg_iov;
+	__kernel_size_t	msg_iovlen;
+	void		__user *msg_control;
+	__kernel_size_t	msg_controllen;
+	unsigned int	msg_flags;
+};
+
+typedef __u32 key_serial_t;
+
+struct mmsghdr {
+	struct user_msghdr  msg_hdr;
+	unsigned int        msg_len;
+};
+
+struct linux_dirent64 {
+	u64		d_ino;
+	s64		d_off;
+	unsigned short	d_reclen;
+	unsigned char	d_type;
+	char		d_name[0];
+};
+
+struct linux_dirent {
+	unsigned long	d_ino;
+	unsigned long	d_off;
+	unsigned short	d_reclen;
+	char		d_name[1];
+};
+
+struct ustat {
+	__kernel_daddr_t	f_tfree;
+	__kernel_ino_t		f_tinode;
+	char			f_fname[6];
+	char			f_fpack[6];
+};
+
+struct __aio_sigset;
+struct clone_args;
+
+typedef __kernel_rwf_t		rwf_t;
+
+/**
+ * lkl_syscall() - issue LKL system calls
+ *
+ * @no: system call number
+ * @params: parameters of system calls
+ *
+ * Return: 0 if there is no error; otherwise non-zero value returns.
+ *
+ * The function is an entry point of LKL system call.
+ */
+long lkl_syscall(long no, long *params);
+
+/**
+ * lkl_sys_halt() - terminate LKL kernel
+ *
+ * Return: 0 if there is no error; otherwise non-zero value returns.
+ *
+ * The function requests the termination of LKL kernel.
+ */
+long lkl_sys_halt(void);
+
+#define __MAP0(m, ...)
+#define __MAP1(m, t, a) m(t, a)
+#define __MAP2(m, t, a, ...) m(t, a), __MAP1(m, __VA_ARGS__)
+#define __MAP3(m, t, a, ...) m(t, a), __MAP2(m, __VA_ARGS__)
+#define __MAP4(m, t, a, ...) m(t, a), __MAP3(m, __VA_ARGS__)
+#define __MAP5(m, t, a, ...) m(t, a), __MAP4(m, __VA_ARGS__)
+#define __MAP6(m, t, a, ...) m(t, a), __MAP5(m, __VA_ARGS__)
+#define __MAP(n, ...) __MAP##n(__VA_ARGS__)
+
+#define __SC_LONG(t, a) (long)(a)
+#define __SC_TABLE(t, a) {sizeof(t), (long long)(a)}
+#define __SC_DECL(t, a) t a
+
+#define LKL_SYSCALL0(name)					       \
+	static inline long lkl_sys##name(void)			       \
+	{							       \
+		long params[6];					       \
+		return lkl_syscall(__lkl__NR##name, params);	       \
+	}
+
+#if __BITS_PER_LONG == 32
+#define LKL_SYSCALLx(x, name, ...)					\
+	static inline							\
+	long lkl_sys##name(__MAP(x, __SC_DECL, __VA_ARGS__))		\
+	{								\
+		struct {						\
+			unsigned int size;				\
+			long long value;				\
+		} lkl_params[x] = { __MAP(x, __SC_TABLE, __VA_ARGS__) }; \
+		long sys_params[6], i, k;				\
+		for (i = k = 0; i < x && k < 6; i++, k++) {		\
+			if (lkl_params[i].size > sizeof(long) &&	\
+			    k + 1 < 6) {				\
+				sys_params[k] =				\
+					(long)(lkl_params[i].value & (-1UL)); \
+				k++;					\
+				sys_params[k] =				\
+					(long)(lkl_params[i].value >>	\
+					       __BITS_PER_LONG);	\
+			} else {					\
+				sys_params[k] = (long)(lkl_params[i].value); \
+			}						\
+		}							\
+		return lkl_syscall(__lkl__NR##name, sys_params);	\
+	}
+#else
+#define LKL_SYSCALLx(x, name, ...)					\
+	static inline							\
+	long lkl_sys##name(__MAP(x, __SC_DECL, __VA_ARGS__))		\
+	{								\
+		long lkl_params[6] = { __MAP(x, __SC_LONG, __VA_ARGS__) }; \
+		return lkl_syscall(__lkl__NR##name, lkl_params);	\
+	}
+#endif
+
+#define SYSCALL_DEFINE0(name, ...) LKL_SYSCALL0(name)
+#define SYSCALL_DEFINE1(name, ...) LKL_SYSCALLx(1, name, __VA_ARGS__)
+#define SYSCALL_DEFINE2(name, ...) LKL_SYSCALLx(2, name, __VA_ARGS__)
+#define SYSCALL_DEFINE3(name, ...) LKL_SYSCALLx(3, name, __VA_ARGS__)
+#define SYSCALL_DEFINE4(name, ...) LKL_SYSCALLx(4, name, __VA_ARGS__)
+#define SYSCALL_DEFINE5(name, ...) LKL_SYSCALLx(5, name, __VA_ARGS__)
+#define SYSCALL_DEFINE6(name, ...) LKL_SYSCALLx(6, name, __VA_ARGS__)
+
+#if __BITS_PER_LONG == 32
+#pragma GCC diagnostic push
+#pragma GCC diagnostic ignored "-Wpointer-to-int-cast"
+#endif
+
+#include <asm/syscall_defs.h>
+
+#if __BITS_PER_LONG == 32
+#pragma GCC diagnostic pop
+#endif
+
+#endif
diff --git a/arch/um/lkl/include/uapi/asm/unistd.h b/arch/um/lkl/include/uapi/asm/unistd.h
new file mode 100644
index 000000000000..6f5d67b45fbd
--- /dev/null
+++ b/arch/um/lkl/include/uapi/asm/unistd.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __UM_LIBMODE_UAPI_UNISTD_H
+#define __UM_LIBMODE_UAPI_UNISTD_H
+
+#define __ARCH_WANT_NEW_STAT
+#include <asm/bitsperlong.h>
+
+#if __BITS_PER_LONG == 64
+#define __ARCH_WANT_SYS_NEWFSTATAT
+#endif
+
+#include <asm-generic/unistd.h>
+
+
+#endif
diff --git a/arch/um/lkl/um/syscalls.c b/arch/um/lkl/um/syscalls.c
new file mode 100644
index 000000000000..45cb5c656f69
--- /dev/null
+++ b/arch/um/lkl/um/syscalls.c
@@ -0,0 +1,193 @@
+// SPDX-License-Identifier: GPL-2.0
+#ifdef CONFIG_ARCH_HAS_SYSCALL_WRAPPER
+#undef CONFIG_ARCH_HAS_SYSCALL_WRAPPER
+#include <linux/syscalls.h>
+#define CONFIG_ARCH_HAS_SYSCALL_WRAPPER
+#endif
+
+#include <linux/stat.h>
+#include <linux/irq.h>
+#include <linux/sched.h>
+#include <linux/interrupt.h>
+#include <linux/jhash.h>
+#include <linux/slab.h>
+#include <linux/types.h>
+#include <linux/net.h>
+#include <linux/task_work.h>
+
+#include <asm/host_ops.h>
+#include <asm/syscalls.h>
+#include <asm/cpu.h>
+#include <asm/sched.h>
+#include <kern_util.h>
+#include <os.h>
+
+typedef long (*syscall_handler_t)(long arg1, ...);
+
+#undef __SYSCALL
+#define __SYSCALL(nr, sym)[nr] = (syscall_handler_t)sym,
+
+syscall_handler_t syscall_table[__NR_syscalls] = {
+	[0 ... __NR_syscalls - 1] =  (syscall_handler_t)sys_ni_syscall,
+#undef __UM_LIBMODE_UAPI_UNISTD_H
+#include <asm/unistd.h>
+
+#if __BITS_PER_LONG == 32
+#include <asm/unistd_32.h>
+#endif
+};
+
+
+static long run_syscall(long no, long *params)
+{
+	long ret;
+
+	if (no < 0 || no >= __NR_syscalls)
+		return -ENOSYS;
+
+	ret = syscall_table[no](params[0], params[1], params[2], params[3],
+				params[4], params[5]);
+
+	task_work_run();
+
+	return ret;
+}
+
+
+#define CLONE_FLAGS (CLONE_VM | CLONE_FS | CLONE_FILES | CLONE_THREAD |	\
+		     CLONE_SIGHAND | SIGCHLD)
+
+static int host_task_id;
+static struct task_struct *host0;
+
+static int new_host_task(struct task_struct **task)
+{
+	pid_t pid;
+
+	switch_to_host_task(host0);
+
+	pid = kernel_thread(host_task_stub, NULL, CLONE_FLAGS);
+	if (pid < 0)
+		return pid;
+
+	rcu_read_lock();
+	*task = find_task_by_pid_ns(pid, &init_pid_ns);
+	rcu_read_unlock();
+
+	host_task_id++;
+
+	snprintf((*task)->comm, sizeof((*task)->comm), "host%d", host_task_id);
+
+	return 0;
+}
+static void exit_task(void)
+{
+	do_exit(0);
+}
+
+static void del_host_task(void *arg)
+{
+	struct task_struct *task = (struct task_struct *)arg;
+	struct thread_info *ti = task_thread_info(task);
+
+	if (lkl_cpu_get() < 0)
+		return;
+
+	switch_to_host_task(task);
+	host_task_id--;
+	set_ti_thread_flag(ti, TIF_SCHED_JB);
+	lkl_jmp_buf_set(&ti->task->thread.arch.sched_jb, exit_task);
+}
+
+static struct lkl_tls_key *task_key;
+
+long lkl_syscall(long no, long *params)
+{
+	struct task_struct *task = host0;
+	long ret;
+
+	ret = lkl_cpu_get();
+	if (ret < 0)
+		return ret;
+
+	task = lkl_tls_get(task_key);
+	if (!task) {
+		ret = new_host_task(&task);
+		if (ret)
+			goto out;
+		lkl_tls_set(task_key, task);
+	}
+
+	switch_to_host_task(task);
+
+	ret = run_syscall(no, params);
+
+	if (no == __NR_reboot) {
+		thread_sched_jb();
+		return ret;
+	}
+
+out:
+	lkl_cpu_put();
+
+	return ret;
+}
+
+static struct task_struct *idle_host_task;
+
+/* called from idle, don't failed, don't block */
+void wakeup_idle_host_task(void)
+{
+	if (!need_resched() && idle_host_task)
+		wake_up_process(idle_host_task);
+}
+
+static int idle_host_task_loop(void *unused)
+{
+	struct thread_info *ti = task_thread_info(current);
+
+	snprintf(current->comm, sizeof(current->comm), "idle_host_task");
+	set_thread_flag(TIF_HOST_THREAD);
+	idle_host_task = current;
+
+	for (;;) {
+		lkl_cpu_put();
+		lkl_sem_down(ti->task->thread.arch.sched_sem);
+		if (idle_host_task == NULL) {
+			lkl_thread_exit();
+			return 0;
+		}
+		schedule_tail(ti->task->thread.prev_sched);
+	}
+}
+
+int syscalls_init(void)
+{
+	snprintf(current->comm, sizeof(current->comm), "host0");
+	set_thread_flag(TIF_HOST_THREAD);
+	host0 = current;
+
+	task_key = lkl_tls_alloc(del_host_task);
+	if (!task_key)
+		return -1;
+
+	if (kernel_thread(idle_host_task_loop, NULL, CLONE_FLAGS) < 0) {
+		lkl_tls_free(task_key);
+		return -1;
+	}
+
+	return 0;
+}
+
+void syscalls_cleanup(void)
+{
+	if (idle_host_task) {
+		struct thread_info *ti = task_thread_info(idle_host_task);
+
+		idle_host_task = NULL;
+		lkl_sem_up(ti->task->thread.arch.sched_sem);
+		lkl_thread_join(ti->task->thread.arch.tid);
+	}
+
+	lkl_tls_free(task_key);
+}
diff --git a/arch/um/scripts/headers_install.py b/arch/um/scripts/headers_install.py
new file mode 100755
index 000000000000..0309f3478698
--- /dev/null
+++ b/arch/um/scripts/headers_install.py
@@ -0,0 +1,200 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+import re, os, sys, argparse, multiprocessing, fnmatch
+
+srctree = os.environ["srctree"]
+objtree = os.environ["objtree"]
+header_paths = [ "include/uapi/", "arch/um/lkl/include/uapi/",
+                 "arch/um/lkl/include/generated/uapi/" ]
+
+headers = set()
+includes = set()
+
+def relpath2abspath(relpath):
+    if "generated" in relpath:
+        return objtree + "/" + relpath
+    else:
+        return srctree + "/" + relpath
+
+def find_headers(path):
+    headers.add(path)
+    f = open(relpath2abspath(path))
+    for l in f.readlines():
+        m = re.search("#\s*include <(.*)>", l)
+        try:
+            i = m.group(1)
+            for p in header_paths:
+                if os.access(relpath2abspath(p + i), os.R_OK):
+                    if p + i not in headers:
+                        includes.add(i)
+                        headers.add(p + i)
+                        find_headers(p + i)
+        except:
+            pass
+    f.close()
+
+def has_lkl_prefix(w):
+  return w.startswith("lkl") or w.startswith("_lkl") or w.startswith("__lkl") \
+         or w.startswith("LKL") or w.startswith("_LKL") or w.startswith("__LKL")
+
+def find_symbols(regexp, store):
+    for h in headers:
+        f = open(h)
+        for l in f.readlines():
+            m = regexp.search(l)
+            if not m:
+                continue
+            for e in reversed(m.groups()):
+                if e:
+                    if not has_lkl_prefix(e):
+                        store.add(e)
+                    break
+        f.close()
+
+def find_ml_symbols(regexp, store):
+    for h in headers:
+        for i in regexp.finditer(open(h).read()):
+            for j in reversed(i.groups()):
+                if j:
+                    if not has_lkl_prefix(j):
+                        store.add(j)
+                    break
+
+def find_enums(block_regexp, symbol_regexp, store):
+    for h in headers:
+        # remove comments
+        content = re.sub(re.compile("(\/\*(\*(?!\/)|[^*])*\*\/)", re.S|re.M), " ", open(h).read())
+        # remove preprocesor lines
+        clean_content = ""
+        for l in content.split("\n"):
+            if re.match("\s*#", l):
+                continue
+            clean_content += l + "\n"
+        for i in block_regexp.finditer(clean_content):
+            for j in reversed(i.groups()):
+                if j:
+                    for k in symbol_regexp.finditer(j):
+                        for l in k.groups():
+                            if l:
+                                if not has_lkl_prefix(l):
+                                    store.add(l)
+                                break
+
+def lkl_prefix(w):
+    r = ""
+
+    if w.startswith("__"):
+        r = "__"
+    elif w.startswith("_"):
+        r = "_"
+
+    if w.isupper():
+        r += "LKL"
+    else:
+        r += "lkl"
+
+    if not w.startswith("_"):
+        r += "_"
+
+    r += w
+
+    return r
+
+def replace(h):
+    content = open(h).read()
+    for i in includes:
+        search_str = "(#[ \t]*include[ \t]*[<\"][ \t]*)" + i + "([ \t]*[>\"])"
+        replace_str = "\\1" + "lkl/" + i + "\\2"
+        content = re.sub(search_str, replace_str, content)
+    tmp = ""
+    for w in re.split("(\W+)", content):
+        if w in defines:
+            w = lkl_prefix(w)
+        tmp += w
+    content = tmp
+    for s in structs:
+        search_str = "(\W?struct\s+)" + s + "(\W)"
+        replace_str = "\\1" + lkl_prefix(s) + "\\2"
+        content = re.sub(search_str, replace_str, content, flags = re.MULTILINE)
+    for s in unions:
+        search_str = "(\W?union\s+)" + s + "(\W)"
+        replace_str = "\\1" + lkl_prefix(s) + "\\2"
+        content = re.sub(search_str, replace_str, content, flags = re.MULTILINE)
+    open(h, 'w').write(content)
+
+parser = argparse.ArgumentParser(description='install lkl headers')
+parser.add_argument('path', help='path to install to', )
+args = parser.parse_args()
+
+find_headers("arch/um/lkl/include/uapi/asm/syscalls.h")
+headers.add("arch/um/lkl/include/uapi/asm/host_ops.h")
+
+if 'LKL_INSTALL_ADDITIONAL_HEADERS' in os.environ:
+    with open(os.environ['LKL_INSTALL_ADDITIONAL_HEADERS'], 'rU') as f:
+        for line in f.readlines():
+            line = line.split('#', 1)[0].strip()
+            if line != '':
+                headers.add(line)
+
+new_headers = set()
+
+for h in headers:
+    h = relpath2abspath(h)
+    dir = os.path.dirname(h)
+    out_dir = args.path + "/" + re.sub("(" + srctree + "/arch/um/lkl/include/uapi/|"
+                                       + srctree + "/include/uapi/|"
+                                       + "arch/um/lkl/include/generated/uapi/|"
+                                       + "include/generated/uapi/|"
+                                       + "include/generated)(.*)", "lkl/\\2", dir)
+    try:
+        os.makedirs(out_dir)
+    except:
+        pass
+    print("  INSTALL %s" % (out_dir + "/" + os.path.basename(h)))
+    os.system(srctree+"/scripts/headers_install.sh %s %s" % (os.path.abspath(h),
+                                                       out_dir + "/" + os.path.basename(h)))
+    new_headers.add(out_dir + "/" + os.path.basename(h))
+
+headers = new_headers
+
+defines = set()
+structs = set()
+unions = set()
+
+p = re.compile("#[ \t]*define[ \t]*(\w+)")
+find_symbols(p, defines)
+p = re.compile("typedef.*(\(\*(\w+)\)\(.*\)\s*|\W+(\w+)\s*|\s+(\w+)\(.*\)\s*);")
+find_symbols(p, defines)
+p = re.compile("typedef\s+(struct|union)\s+\w*\s*{[^\\{\}]*}\W*(\w+)\s*;", re.M|re.S)
+find_ml_symbols(p, defines)
+defines.add("siginfo_t")
+defines.add("sigevent_t")
+p = re.compile("struct\s+(\w+)\s*\{")
+find_symbols(p, structs)
+structs.add("iovec")
+p = re.compile("union\s+(\w+)\s*\{")
+find_symbols(p, unions)
+p = re.compile("static\s+__inline__(\s+\w+)+\s+(\w+)\([^)]*\)\s")
+find_symbols(p, defines)
+p = re.compile("static\s+__always_inline(\s+\w+)+\s+(\w+)\([^)]*\)\s")
+find_symbols(p, defines)
+p = re.compile("enum\s+(\w*)\s*{([^}]*)}", re.M|re.S)
+q = re.compile("(\w+)\s*(,|=[^,]*|$)", re.M|re.S)
+find_enums(p, q, defines)
+
+# needed for i386
+defines.add("__NR_stime")
+
+def process_header(h):
+    print("  REPLACE %s" % h)
+    replace(h)
+
+# use num of processors determined by os.cpu_count()
+p = multiprocessing.Pool(None)
+try:
+    p.map_async(process_header, headers).wait(999999)
+    p.close()
+except:
+    p.terminate()
+finally:
+    p.join()
diff --git a/scripts/headers_install.sh b/scripts/headers_install.sh
index dd554bd436cc..d44b92979db4 100755
--- a/scripts/headers_install.sh
+++ b/scripts/headers_install.sh
@@ -24,7 +24,11 @@ TMPFILE=$OUTFILE.tmp
 trap 'rm -f $OUTFILE $TMPFILE' EXIT
 
 # SPDX-License-Identifier with GPL variants must have "WITH Linux-syscall-note"
-if [ -n "$(sed -n -e "/SPDX-License-Identifier:.*GPL-/{/WITH Linux-syscall-note/!p}" $INFILE)" ]; then
+#
+# ARCH=um with library mode exposes uapi headers but applications may be linked
+# statically, thus this GPL exception doesn't apply to.
+if [ -n "$(sed -n -e "/SPDX-License-Identifier:.*GPL-/{/WITH Linux-syscall-note/!p}" $INFILE)" ] &&
+	[ "$SRCARCH" != "um" ]; then
 	echo "error: $INFILE: missing \"WITH Linux-syscall-note\" for SPDX-License-Identifier" >&2
 	exit 1
 fi
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v8 11/20] um: lkl: basic console support
  2021-01-20  2:27           ` Hajime Tazaki
@ 2021-01-20  2:27             ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-01-20  2:27 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: thehajime, tavi.purdila, retrage01, linux-kernel-library, linux-arch

This patch adds a basic structure of console support in kernel.  Write
operations are deferred to the host print operation.

This commit also disables stdio-console of UML when library mode to avoid
conflicts between multiple consoles.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/drivers/Makefile                |  8 ++++-
 arch/um/lkl/include/uapi/asm/host_ops.h |  9 ++++++
 arch/um/lkl/um/console.c                | 41 +++++++++++++++++++++++++
 3 files changed, 57 insertions(+), 1 deletion(-)
 create mode 100644 arch/um/lkl/um/console.c

diff --git a/arch/um/drivers/Makefile b/arch/um/drivers/Makefile
index ae96c83e312d..dc0d32d62294 100644
--- a/arch/um/drivers/Makefile
+++ b/arch/um/drivers/Makefile
@@ -37,7 +37,13 @@ $(obj)/vde.o: $(obj)/vde_kern.o $(obj)/vde_user.o
 # When the above is fixed, don't forget to add this too!
 #targets += $(obj)/pcap.o
 
-obj-y := stdio_console.o fd.o chan_kern.o chan_user.o line.o
+ifndef CONFIG_UMMODE_LIB
+obj-y := stdio_console.o
+else
+obj-y :=
+endif
+obj-y += fd.o chan_kern.o chan_user.o line.o
+
 obj-$(CONFIG_SSL) += ssl.o
 obj-$(CONFIG_STDERR_CONSOLE) += stderr_console.o
 
diff --git a/arch/um/lkl/include/uapi/asm/host_ops.h b/arch/um/lkl/include/uapi/asm/host_ops.h
index b97aa1099a17..976fc4301b3d 100644
--- a/arch/um/lkl/include/uapi/asm/host_ops.h
+++ b/arch/um/lkl/include/uapi/asm/host_ops.h
@@ -243,4 +243,13 @@ int lkl_tls_set(struct lkl_tls_key *key, void *data);
  */
 void *lkl_tls_get(struct lkl_tls_key *key);
 
+/**
+ * lkl_print - optional operation that receives console messages
+ *
+ * @str: strings to be printed
+ * @len: the length of @str
+ *
+ */
+void lkl_print(const char *str, int len);
+
 #endif
diff --git a/arch/um/lkl/um/console.c b/arch/um/lkl/um/console.c
new file mode 100644
index 000000000000..f2b48dc7ae22
--- /dev/null
+++ b/arch/um/lkl/um/console.c
@@ -0,0 +1,41 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/console.h>
+#include <asm/host_ops.h>
+
+static void console_write(struct console *con, const char *str, unsigned len)
+{
+	lkl_print(str, len);
+}
+
+#ifdef CONFIG_LKL_EARLY_CONSOLE
+static struct console lkl_boot_console = {
+	.name	= "lkl_boot_console",
+	.write	= console_write,
+	.flags	= CON_PRINTBUFFER | CON_BOOT,
+	.index	= -1,
+};
+
+int __init lkl_boot_console_init(void)
+{
+	register_console(&lkl_boot_console);
+	return 0;
+}
+early_initcall(lkl_boot_console_init);
+#endif
+
+static struct console lkl_console = {
+	.name	= "lkl_console",
+	.write	= console_write,
+	.flags	= CON_PRINTBUFFER,
+	.index	= -1,
+};
+
+int __init lkl_console_init(void)
+{
+	register_console(&lkl_console);
+	return 0;
+}
+core_initcall(lkl_console_init);
+
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v8 11/20] um: lkl: basic console support
@ 2021-01-20  2:27             ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-01-20  2:27 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, thehajime, retrage01

This patch adds a basic structure of console support in kernel.  Write
operations are deferred to the host print operation.

This commit also disables stdio-console of UML when library mode to avoid
conflicts between multiple consoles.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/drivers/Makefile                |  8 ++++-
 arch/um/lkl/include/uapi/asm/host_ops.h |  9 ++++++
 arch/um/lkl/um/console.c                | 41 +++++++++++++++++++++++++
 3 files changed, 57 insertions(+), 1 deletion(-)
 create mode 100644 arch/um/lkl/um/console.c

diff --git a/arch/um/drivers/Makefile b/arch/um/drivers/Makefile
index ae96c83e312d..dc0d32d62294 100644
--- a/arch/um/drivers/Makefile
+++ b/arch/um/drivers/Makefile
@@ -37,7 +37,13 @@ $(obj)/vde.o: $(obj)/vde_kern.o $(obj)/vde_user.o
 # When the above is fixed, don't forget to add this too!
 #targets += $(obj)/pcap.o
 
-obj-y := stdio_console.o fd.o chan_kern.o chan_user.o line.o
+ifndef CONFIG_UMMODE_LIB
+obj-y := stdio_console.o
+else
+obj-y :=
+endif
+obj-y += fd.o chan_kern.o chan_user.o line.o
+
 obj-$(CONFIG_SSL) += ssl.o
 obj-$(CONFIG_STDERR_CONSOLE) += stderr_console.o
 
diff --git a/arch/um/lkl/include/uapi/asm/host_ops.h b/arch/um/lkl/include/uapi/asm/host_ops.h
index b97aa1099a17..976fc4301b3d 100644
--- a/arch/um/lkl/include/uapi/asm/host_ops.h
+++ b/arch/um/lkl/include/uapi/asm/host_ops.h
@@ -243,4 +243,13 @@ int lkl_tls_set(struct lkl_tls_key *key, void *data);
  */
 void *lkl_tls_get(struct lkl_tls_key *key);
 
+/**
+ * lkl_print - optional operation that receives console messages
+ *
+ * @str: strings to be printed
+ * @len: the length of @str
+ *
+ */
+void lkl_print(const char *str, int len);
+
 #endif
diff --git a/arch/um/lkl/um/console.c b/arch/um/lkl/um/console.c
new file mode 100644
index 000000000000..f2b48dc7ae22
--- /dev/null
+++ b/arch/um/lkl/um/console.c
@@ -0,0 +1,41 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/console.h>
+#include <asm/host_ops.h>
+
+static void console_write(struct console *con, const char *str, unsigned len)
+{
+	lkl_print(str, len);
+}
+
+#ifdef CONFIG_LKL_EARLY_CONSOLE
+static struct console lkl_boot_console = {
+	.name	= "lkl_boot_console",
+	.write	= console_write,
+	.flags	= CON_PRINTBUFFER | CON_BOOT,
+	.index	= -1,
+};
+
+int __init lkl_boot_console_init(void)
+{
+	register_console(&lkl_boot_console);
+	return 0;
+}
+early_initcall(lkl_boot_console_init);
+#endif
+
+static struct console lkl_console = {
+	.name	= "lkl_console",
+	.write	= console_write,
+	.flags	= CON_PRINTBUFFER,
+	.index	= -1,
+};
+
+int __init lkl_console_init(void)
+{
+	register_console(&lkl_console);
+	return 0;
+}
+core_initcall(lkl_console_init);
+
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v8 12/20] um: lkl: initialization and cleanup
  2021-01-20  2:27           ` Hajime Tazaki
@ 2021-01-20  2:27             ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-01-20  2:27 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: thehajime, tavi.purdila, retrage01, linux-kernel-library, linux-arch

Add the lkl_start_kernel and lkl_sys_halt APIs that start and
respectively stops the Linux kernel.

lkl_start_kernel creates a separate threads that will run the initial
and idle kernel thread. It waits for the kernel to complete
initialization before returning, to avoid races with system calls
issues by the host application.

During the setup phase, we create "/init" in initial ramfs root
filesystem to avoid mounting the "real" rootfs since ramfs is good
enough for now.

lkl_sys_halt will shutdown the kernel, terminate all threads and
free all host resources used by the kernel before returning.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/lkl/include/uapi/asm/host_ops.h |  20 +++
 arch/um/lkl/um/setup.c                  | 162 ++++++++++++++++++++++++
 tools/um/uml/process.c                  |   2 +
 3 files changed, 184 insertions(+)
 create mode 100644 arch/um/lkl/um/setup.c

diff --git a/arch/um/lkl/include/uapi/asm/host_ops.h b/arch/um/lkl/include/uapi/asm/host_ops.h
index 976fc4301b3d..85d7d4790602 100644
--- a/arch/um/lkl/include/uapi/asm/host_ops.h
+++ b/arch/um/lkl/include/uapi/asm/host_ops.h
@@ -252,4 +252,24 @@ void *lkl_tls_get(struct lkl_tls_key *key);
  */
 void lkl_print(const char *str, int len);
 
+/**
+ * lkl_panic - called during a kernel panic
+ *
+ */
+void lkl_panic(void);
+
+/**
+ * lkl_start_kernel() - registers the host operations and starts the kernel
+ *
+ * @ops: pointer to host operations
+ * @fmt: format for command line string that is going to be used to
+ * generate the Linux kernel command line
+ *
+ * Return: 0 if there is no error; otherwise non-zero value returns.
+ *
+ * The function returns only after the kernel is shutdown with lkl_sys_halt.
+ */
+int lkl_start_kernel(struct lkl_host_operations *ops,
+		     const char *fmt, ...);
+
 #endif
diff --git a/arch/um/lkl/um/setup.c b/arch/um/lkl/um/setup.c
new file mode 100644
index 000000000000..ba8338d4fc23
--- /dev/null
+++ b/arch/um/lkl/um/setup.c
@@ -0,0 +1,162 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/binfmts.h>
+#include <linux/init.h>
+#include <linux/init_task.h>
+#include <linux/personality.h>
+#include <linux/reboot.h>
+#include <linux/fs.h>
+#include <linux/start_kernel.h>
+#include <linux/fdtable.h>
+#include <linux/tick.h>
+#include <linux/kernel.h>
+#include <linux/console.h>
+#include <asm/host_ops.h>
+#include <asm/irq.h>
+#include <asm/unistd.h>
+#include <asm/syscalls.h>
+#include <asm/cpu.h>
+#include <os.h>
+#include <as-layout.h>
+
+
+static void *init_sem;
+static int is_running;
+
+
+struct lkl_host_operations *lkl_ops;
+
+long lkl_panic_blink(int state)
+{
+	lkl_panic();
+	return 0;
+}
+
+static void __init *lkl_run_kernel(void *arg)
+{
+
+	panic_blink = lkl_panic_blink;
+
+	threads_init();
+	lkl_cpu_get();
+	start_kernel();
+
+	return NULL;
+}
+
+int __init lkl_start_kernel(struct lkl_host_operations *ops,
+			    const char *fmt, ...)
+{
+	int ret;
+
+	lkl_ops = ops;
+
+	init_sem = lkl_sem_alloc(0);
+	if (!init_sem)
+		return -ENOMEM;
+
+	ret = lkl_cpu_init();
+	if (ret)
+		goto out_free_init_sem;
+
+	ret = lkl_thread_create(lkl_run_kernel, NULL);
+	if (!ret) {
+		ret = -ENOMEM;
+		goto out_free_init_sem;
+	}
+
+	lkl_sem_down(init_sem);
+	lkl_sem_free(init_sem);
+	current_thread_info()->task->thread.arch.tid = lkl_thread_self();
+	lkl_cpu_change_owner(current_thread_info()->task->thread.arch.tid);
+
+	lkl_cpu_put();
+	is_running = 1;
+
+	return 0;
+
+out_free_init_sem:
+	lkl_sem_free(init_sem);
+
+	return ret;
+}
+
+int lkl_is_running(void)
+{
+	return is_running;
+}
+
+
+long lkl_sys_halt(void)
+{
+	long err;
+	long params[6] = {LINUX_REBOOT_MAGIC1,
+		LINUX_REBOOT_MAGIC2, LINUX_REBOOT_CMD_RESTART, };
+
+	err = lkl_syscall(__NR_reboot, params);
+	if (err < 0)
+		return err;
+
+	is_running = false;
+
+	lkl_cpu_wait_shutdown();
+
+	syscalls_cleanup();
+	threads_cleanup();
+	/* Shutdown the clockevents source. */
+	tick_suspend_local();
+	free_mem();
+	lkl_thread_join(current_thread_info()->task->thread.arch.tid);
+
+	return 0;
+}
+
+
+static int lkl_run_init(struct linux_binprm *bprm);
+
+static struct linux_binfmt lkl_run_init_binfmt = {
+	.module		= THIS_MODULE,
+	.load_binary	= lkl_run_init,
+};
+
+static int lkl_run_init(struct linux_binprm *bprm)
+{
+	int ret;
+
+	if (strcmp("/init", bprm->filename) != 0)
+		return -EINVAL;
+
+	ret = begin_new_exec(bprm);
+	if (ret)
+		return ret;
+	set_personality(PER_LINUX);
+	setup_new_exec(bprm);
+
+	set_binfmt(&lkl_run_init_binfmt);
+
+	init_pid_ns.child_reaper = 0;
+
+	syscalls_init();
+
+	lkl_sem_up(init_sem);
+	lkl_thread_exit();
+
+	return 0;
+}
+
+
+/* skip mounting the "real" rootfs. ramfs is good enough. */
+static int __init fs_setup(void)
+{
+	int fd, flags = 0;
+
+	if (force_o_largefile())
+		flags |= O_LARGEFILE;
+	fd = do_sys_open(AT_FDCWD, "/init", O_CREAT | flags, 0700);
+	WARN_ON(fd < 0);
+	close_fd(fd);
+
+	register_binfmt(&lkl_run_init_binfmt);
+
+	return 0;
+}
+late_initcall(fs_setup);
diff --git a/tools/um/uml/process.c b/tools/um/uml/process.c
index e52dd37ddadc..c39d914e80ac 100644
--- a/tools/um/uml/process.c
+++ b/tools/um/uml/process.c
@@ -114,6 +114,8 @@ void os_kill_process(int pid, int reap_child)
 
 void os_kill_ptraced_process(int pid, int reap_child)
 {
+	if (pid == 0)
+		return;
 	kill(pid, SIGKILL);
 	ptrace(PTRACE_KILL, pid);
 	ptrace(PTRACE_CONT, pid);
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v8 12/20] um: lkl: initialization and cleanup
@ 2021-01-20  2:27             ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-01-20  2:27 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, thehajime, retrage01

Add the lkl_start_kernel and lkl_sys_halt APIs that start and
respectively stops the Linux kernel.

lkl_start_kernel creates a separate threads that will run the initial
and idle kernel thread. It waits for the kernel to complete
initialization before returning, to avoid races with system calls
issues by the host application.

During the setup phase, we create "/init" in initial ramfs root
filesystem to avoid mounting the "real" rootfs since ramfs is good
enough for now.

lkl_sys_halt will shutdown the kernel, terminate all threads and
free all host resources used by the kernel before returning.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 arch/um/lkl/include/uapi/asm/host_ops.h |  20 +++
 arch/um/lkl/um/setup.c                  | 162 ++++++++++++++++++++++++
 tools/um/uml/process.c                  |   2 +
 3 files changed, 184 insertions(+)
 create mode 100644 arch/um/lkl/um/setup.c

diff --git a/arch/um/lkl/include/uapi/asm/host_ops.h b/arch/um/lkl/include/uapi/asm/host_ops.h
index 976fc4301b3d..85d7d4790602 100644
--- a/arch/um/lkl/include/uapi/asm/host_ops.h
+++ b/arch/um/lkl/include/uapi/asm/host_ops.h
@@ -252,4 +252,24 @@ void *lkl_tls_get(struct lkl_tls_key *key);
  */
 void lkl_print(const char *str, int len);
 
+/**
+ * lkl_panic - called during a kernel panic
+ *
+ */
+void lkl_panic(void);
+
+/**
+ * lkl_start_kernel() - registers the host operations and starts the kernel
+ *
+ * @ops: pointer to host operations
+ * @fmt: format for command line string that is going to be used to
+ * generate the Linux kernel command line
+ *
+ * Return: 0 if there is no error; otherwise non-zero value returns.
+ *
+ * The function returns only after the kernel is shutdown with lkl_sys_halt.
+ */
+int lkl_start_kernel(struct lkl_host_operations *ops,
+		     const char *fmt, ...);
+
 #endif
diff --git a/arch/um/lkl/um/setup.c b/arch/um/lkl/um/setup.c
new file mode 100644
index 000000000000..ba8338d4fc23
--- /dev/null
+++ b/arch/um/lkl/um/setup.c
@@ -0,0 +1,162 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/binfmts.h>
+#include <linux/init.h>
+#include <linux/init_task.h>
+#include <linux/personality.h>
+#include <linux/reboot.h>
+#include <linux/fs.h>
+#include <linux/start_kernel.h>
+#include <linux/fdtable.h>
+#include <linux/tick.h>
+#include <linux/kernel.h>
+#include <linux/console.h>
+#include <asm/host_ops.h>
+#include <asm/irq.h>
+#include <asm/unistd.h>
+#include <asm/syscalls.h>
+#include <asm/cpu.h>
+#include <os.h>
+#include <as-layout.h>
+
+
+static void *init_sem;
+static int is_running;
+
+
+struct lkl_host_operations *lkl_ops;
+
+long lkl_panic_blink(int state)
+{
+	lkl_panic();
+	return 0;
+}
+
+static void __init *lkl_run_kernel(void *arg)
+{
+
+	panic_blink = lkl_panic_blink;
+
+	threads_init();
+	lkl_cpu_get();
+	start_kernel();
+
+	return NULL;
+}
+
+int __init lkl_start_kernel(struct lkl_host_operations *ops,
+			    const char *fmt, ...)
+{
+	int ret;
+
+	lkl_ops = ops;
+
+	init_sem = lkl_sem_alloc(0);
+	if (!init_sem)
+		return -ENOMEM;
+
+	ret = lkl_cpu_init();
+	if (ret)
+		goto out_free_init_sem;
+
+	ret = lkl_thread_create(lkl_run_kernel, NULL);
+	if (!ret) {
+		ret = -ENOMEM;
+		goto out_free_init_sem;
+	}
+
+	lkl_sem_down(init_sem);
+	lkl_sem_free(init_sem);
+	current_thread_info()->task->thread.arch.tid = lkl_thread_self();
+	lkl_cpu_change_owner(current_thread_info()->task->thread.arch.tid);
+
+	lkl_cpu_put();
+	is_running = 1;
+
+	return 0;
+
+out_free_init_sem:
+	lkl_sem_free(init_sem);
+
+	return ret;
+}
+
+int lkl_is_running(void)
+{
+	return is_running;
+}
+
+
+long lkl_sys_halt(void)
+{
+	long err;
+	long params[6] = {LINUX_REBOOT_MAGIC1,
+		LINUX_REBOOT_MAGIC2, LINUX_REBOOT_CMD_RESTART, };
+
+	err = lkl_syscall(__NR_reboot, params);
+	if (err < 0)
+		return err;
+
+	is_running = false;
+
+	lkl_cpu_wait_shutdown();
+
+	syscalls_cleanup();
+	threads_cleanup();
+	/* Shutdown the clockevents source. */
+	tick_suspend_local();
+	free_mem();
+	lkl_thread_join(current_thread_info()->task->thread.arch.tid);
+
+	return 0;
+}
+
+
+static int lkl_run_init(struct linux_binprm *bprm);
+
+static struct linux_binfmt lkl_run_init_binfmt = {
+	.module		= THIS_MODULE,
+	.load_binary	= lkl_run_init,
+};
+
+static int lkl_run_init(struct linux_binprm *bprm)
+{
+	int ret;
+
+	if (strcmp("/init", bprm->filename) != 0)
+		return -EINVAL;
+
+	ret = begin_new_exec(bprm);
+	if (ret)
+		return ret;
+	set_personality(PER_LINUX);
+	setup_new_exec(bprm);
+
+	set_binfmt(&lkl_run_init_binfmt);
+
+	init_pid_ns.child_reaper = 0;
+
+	syscalls_init();
+
+	lkl_sem_up(init_sem);
+	lkl_thread_exit();
+
+	return 0;
+}
+
+
+/* skip mounting the "real" rootfs. ramfs is good enough. */
+static int __init fs_setup(void)
+{
+	int fd, flags = 0;
+
+	if (force_o_largefile())
+		flags |= O_LARGEFILE;
+	fd = do_sys_open(AT_FDCWD, "/init", O_CREAT | flags, 0700);
+	WARN_ON(fd < 0);
+	close_fd(fd);
+
+	register_binfmt(&lkl_run_init_binfmt);
+
+	return 0;
+}
+late_initcall(fs_setup);
diff --git a/tools/um/uml/process.c b/tools/um/uml/process.c
index e52dd37ddadc..c39d914e80ac 100644
--- a/tools/um/uml/process.c
+++ b/tools/um/uml/process.c
@@ -114,6 +114,8 @@ void os_kill_process(int pid, int reap_child)
 
 void os_kill_ptraced_process(int pid, int reap_child)
 {
+	if (pid == 0)
+		return;
 	kill(pid, SIGKILL);
 	ptrace(PTRACE_KILL, pid);
 	ptrace(PTRACE_CONT, pid);
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v8 13/20] um: lkl: integrate with irq infrastructure of UML
  2021-01-20  2:27           ` Hajime Tazaki
@ 2021-01-20  2:27             ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-01-20  2:27 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: thehajime, tavi.purdila, retrage01, linux-kernel-library, linux-arch

In order to cooperate with UML's irq infrastructure and LKL threads
based on host threads, irq handlers shall synchronize the our
scheduler which is controlled by struct lkl_cpu.  To do that, the irq
infra notifies its entry of handlers by obtaining cpu access of thread
scheduler (lkl_cpu_try_run_irq) and its release (lkl_cpu_put).

In additon to that, in order to stick the signal handler's thread to the
one of the idle thread, several required configurations of (thread's)
signal mask are added: otherwise handlers running on arbitrary thread
cannot obtain cpu access and immediately fall into pending interrupt
which may slow down the delivery of signals.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/include/shared/os.h   |  1 +
 arch/um/kernel/irq.c          | 13 ++++++++++
 arch/um/kernel/time.c         |  2 ++
 arch/um/lkl/include/asm/cpu.h |  4 +++
 arch/um/lkl/um/cpu.c          | 46 +++++++++++++++++++++++++++++++++++
 arch/um/lkl/um/setup.c        |  5 ++++
 arch/um/lkl/um/threads.c      |  3 +++
 tools/um/uml/signal.c         | 12 +++++++--
 8 files changed, 84 insertions(+), 2 deletions(-)

diff --git a/arch/um/include/shared/os.h b/arch/um/include/shared/os.h
index 13d86f94cf0f..cd8583e4df12 100644
--- a/arch/um/include/shared/os.h
+++ b/arch/um/include/shared/os.h
@@ -243,6 +243,7 @@ extern int set_signals_trace(int enable);
 extern int os_is_signal_stack(void);
 extern void deliver_alarm(void);
 extern void register_pm_wake_signal(void);
+extern void set_pending_signals(int sig);
 
 /* util.c */
 extern void stack_protections(unsigned long address);
diff --git a/arch/um/kernel/irq.c b/arch/um/kernel/irq.c
index 3741d2380060..d632156b52ac 100644
--- a/arch/um/kernel/irq.c
+++ b/arch/um/kernel/irq.c
@@ -21,6 +21,7 @@
 #include <irq_user.h>
 #include <irq_kern.h>
 #include <as-layout.h>
+#include <asm/cpu.h>
 
 
 extern void free_irqs(void);
@@ -563,6 +564,11 @@ unsigned long to_irq_stack(unsigned long *mask_out)
 	unsigned long mask, old;
 	int nested;
 
+#ifdef CONFIG_UMMODE_LIB
+	if (!lkl_irq_enter(ffs(*mask_out) - 1))
+		return 1;
+#endif
+
 	mask = xchg(&pending_mask, *mask_out);
 	if (mask != 0) {
 		/*
@@ -579,6 +585,10 @@ unsigned long to_irq_stack(unsigned long *mask_out)
 			old |= mask;
 			mask = xchg(&pending_mask, old);
 		} while (mask != old);
+
+#ifdef CONFIG_UMMODE_LIB
+		lkl_irq_exit();
+#endif
 		return 1;
 	}
 
@@ -616,6 +626,9 @@ unsigned long from_irq_stack(int nested)
 	*to = *ti;
 
 	mask = xchg(&pending_mask, 0);
+#ifdef CONFIG_UMMODE_LIB
+	lkl_irq_exit();
+#endif
 	return mask & ~1;
 }
 
diff --git a/arch/um/kernel/time.c b/arch/um/kernel/time.c
index f4db89b5b5a6..07d705ec5193 100644
--- a/arch/um/kernel/time.c
+++ b/arch/um/kernel/time.c
@@ -659,11 +659,13 @@ static struct clock_event_device timer_clockevent = {
 
 static irqreturn_t um_timer(int irq, void *dev)
 {
+#ifndef CONFIG_UMMODE_LIB
 	if (get_current()->mm != NULL)
 	{
         /* userspace - relay signal, results in correct userspace timers */
 		os_alarm_process(get_current()->mm->context.id.u.pid);
 	}
+#endif
 
 	(*timer_clockevent.event_handler)(&timer_clockevent);
 
diff --git a/arch/um/lkl/include/asm/cpu.h b/arch/um/lkl/include/asm/cpu.h
index c1164187151e..b241b5bbdc53 100644
--- a/arch/um/lkl/include/asm/cpu.h
+++ b/arch/um/lkl/include/asm/cpu.h
@@ -8,4 +8,8 @@ int lkl_cpu_init(void);
 void lkl_cpu_wait_shutdown(void);
 void lkl_cpu_change_owner(lkl_thread_t owner);
 
+int lkl_cpu_try_run_irq(int irq);
+void lkl_irq_exit(void);
+int lkl_irq_enter(int sig);
+
 #endif
diff --git a/arch/um/lkl/um/cpu.c b/arch/um/lkl/um/cpu.c
index 75452b28d741..841c7e9cc881 100644
--- a/arch/um/lkl/um/cpu.c
+++ b/arch/um/lkl/um/cpu.c
@@ -8,6 +8,7 @@
 #include <asm/sched.h>
 #include <asm/syscalls.h>
 #include <init.h>
+#include <os.h>
 
 /*
  * This structure is used to get access to the "LKL CPU" that allows us to run
@@ -41,6 +42,7 @@ static struct lkl_cpu {
 	 */
 	#define MAX_THREADS 1000000
 	unsigned int shutdown_gate;
+	bool irqs_pending;
 	/* no of threads waiting the CPU */
 	unsigned int sleepers;
 	/* no of times the current thread got the CPU */
@@ -53,6 +55,16 @@ static struct lkl_cpu {
 	struct lkl_sem *shutdown_sem;
 } cpu;
 
+static void run_irqs(void)
+{
+	unblock_signals();
+}
+
+static void set_irq_pending(int sig)
+{
+	set_pending_signals(sig);
+}
+
 /*
  * internal routine to acquire LKL CPU's lock
  */
@@ -132,6 +144,16 @@ void lkl_cpu_put(void)
 	    !lkl_thread_equal(cpu.owner, lkl_thread_self()))
 		lkl_bug("%s: unbalanced put\n", __func__);
 
+	/* we're going to trigger irq handlers if there are any pending
+	 * interrupts, and not irq_disabled.
+	 */
+	while (cpu.irqs_pending && !irqs_disabled()) {
+		cpu.irqs_pending = false;
+		lkl_mutex_unlock(cpu.lock);
+		run_irqs();
+		lkl_mutex_lock(cpu.lock);
+	}
+
 	/* switch to userspace code if current is host task (TIF_HOST_THREAD),
 	 * AND, there are other running tasks.
 	 */
@@ -163,6 +185,30 @@ void lkl_cpu_put(void)
 	lkl_mutex_unlock(cpu.lock);
 }
 
+int lkl_cpu_try_run_irq(int irq)
+{
+	int ret;
+
+	ret = __cpu_try_get_lock(1);
+	if (!ret) {
+		set_irq_pending(irq);
+		cpu.irqs_pending = true;
+	}
+	__cpu_try_get_unlock(ret, 1);
+
+	return ret;
+}
+
+int lkl_irq_enter(int sig)
+{
+	return lkl_cpu_try_run_irq(sig);
+}
+
+void lkl_irq_exit(void)
+{
+	return lkl_cpu_put();
+}
+
 static void lkl_cpu_shutdown(void)
 {
 	__sync_fetch_and_add(&cpu.shutdown_gate, MAX_THREADS);
diff --git a/arch/um/lkl/um/setup.c b/arch/um/lkl/um/setup.c
index ba8338d4fc23..12b235826573 100644
--- a/arch/um/lkl/um/setup.c
+++ b/arch/um/lkl/um/setup.c
@@ -36,6 +36,8 @@ static void __init *lkl_run_kernel(void *arg)
 
 	panic_blink = lkl_panic_blink;
 
+	/* signal should be received at this thread (main and idle threads) */
+	init_new_thread_signals();
 	threads_init();
 	lkl_cpu_get();
 	start_kernel();
@@ -58,6 +60,9 @@ int __init lkl_start_kernel(struct lkl_host_operations *ops,
 	if (ret)
 		goto out_free_init_sem;
 
+	change_sig(SIGALRM, 0);
+	change_sig(SIGIO, 0);
+
 	ret = lkl_thread_create(lkl_run_kernel, NULL);
 	if (!ret) {
 		ret = -ENOMEM;
diff --git a/arch/um/lkl/um/threads.c b/arch/um/lkl/um/threads.c
index 7ef9b9f2a6b7..c7ff578b7a91 100644
--- a/arch/um/lkl/um/threads.c
+++ b/arch/um/lkl/um/threads.c
@@ -152,6 +152,9 @@ static void *thread_bootstrap(void *_tba)
 	int (*f)(void *) = tba->f;
 	void *arg = tba->arg;
 
+	change_sig(SIGALRM, 0);
+	change_sig(SIGIO, 0);
+
 	lkl_sem_down(ti->task->thread.arch.sched_sem);
 	kfree(tba);
 	if (ti->task->thread.prev_sched)
diff --git a/tools/um/uml/signal.c b/tools/um/uml/signal.c
index 96f511d1aabe..de04b0dd34bb 100644
--- a/tools/um/uml/signal.c
+++ b/tools/um/uml/signal.c
@@ -230,8 +230,8 @@ void set_handler(int sig)
 
 	sigemptyset(&sig_mask);
 	sigaddset(&sig_mask, sig);
-	if (sigprocmask(SIG_UNBLOCK, &sig_mask, NULL) < 0)
-		panic("sigprocmask failed - errno = %d\n", errno);
+	if (pthread_sigmask(SIG_UNBLOCK, &sig_mask, NULL) < 0)
+		panic("pthread_sigmask failed - errno = %d\n", errno);
 }
 
 void send_sigio_to_self(void)
@@ -375,3 +375,11 @@ int os_is_signal_stack(void)
 
 	return ss.ss_flags & SS_ONSTACK;
 }
+
+void set_pending_signals(int sig)
+{
+	if (sig == SIGIO)
+		signals_pending |= SIGIO_MASK;
+	else if (sig == SIGALRM)
+		signals_pending |= SIGALRM_MASK;
+}
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v8 13/20] um: lkl: integrate with irq infrastructure of UML
@ 2021-01-20  2:27             ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-01-20  2:27 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, thehajime, retrage01

In order to cooperate with UML's irq infrastructure and LKL threads
based on host threads, irq handlers shall synchronize the our
scheduler which is controlled by struct lkl_cpu.  To do that, the irq
infra notifies its entry of handlers by obtaining cpu access of thread
scheduler (lkl_cpu_try_run_irq) and its release (lkl_cpu_put).

In additon to that, in order to stick the signal handler's thread to the
one of the idle thread, several required configurations of (thread's)
signal mask are added: otherwise handlers running on arbitrary thread
cannot obtain cpu access and immediately fall into pending interrupt
which may slow down the delivery of signals.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/include/shared/os.h   |  1 +
 arch/um/kernel/irq.c          | 13 ++++++++++
 arch/um/kernel/time.c         |  2 ++
 arch/um/lkl/include/asm/cpu.h |  4 +++
 arch/um/lkl/um/cpu.c          | 46 +++++++++++++++++++++++++++++++++++
 arch/um/lkl/um/setup.c        |  5 ++++
 arch/um/lkl/um/threads.c      |  3 +++
 tools/um/uml/signal.c         | 12 +++++++--
 8 files changed, 84 insertions(+), 2 deletions(-)

diff --git a/arch/um/include/shared/os.h b/arch/um/include/shared/os.h
index 13d86f94cf0f..cd8583e4df12 100644
--- a/arch/um/include/shared/os.h
+++ b/arch/um/include/shared/os.h
@@ -243,6 +243,7 @@ extern int set_signals_trace(int enable);
 extern int os_is_signal_stack(void);
 extern void deliver_alarm(void);
 extern void register_pm_wake_signal(void);
+extern void set_pending_signals(int sig);
 
 /* util.c */
 extern void stack_protections(unsigned long address);
diff --git a/arch/um/kernel/irq.c b/arch/um/kernel/irq.c
index 3741d2380060..d632156b52ac 100644
--- a/arch/um/kernel/irq.c
+++ b/arch/um/kernel/irq.c
@@ -21,6 +21,7 @@
 #include <irq_user.h>
 #include <irq_kern.h>
 #include <as-layout.h>
+#include <asm/cpu.h>
 
 
 extern void free_irqs(void);
@@ -563,6 +564,11 @@ unsigned long to_irq_stack(unsigned long *mask_out)
 	unsigned long mask, old;
 	int nested;
 
+#ifdef CONFIG_UMMODE_LIB
+	if (!lkl_irq_enter(ffs(*mask_out) - 1))
+		return 1;
+#endif
+
 	mask = xchg(&pending_mask, *mask_out);
 	if (mask != 0) {
 		/*
@@ -579,6 +585,10 @@ unsigned long to_irq_stack(unsigned long *mask_out)
 			old |= mask;
 			mask = xchg(&pending_mask, old);
 		} while (mask != old);
+
+#ifdef CONFIG_UMMODE_LIB
+		lkl_irq_exit();
+#endif
 		return 1;
 	}
 
@@ -616,6 +626,9 @@ unsigned long from_irq_stack(int nested)
 	*to = *ti;
 
 	mask = xchg(&pending_mask, 0);
+#ifdef CONFIG_UMMODE_LIB
+	lkl_irq_exit();
+#endif
 	return mask & ~1;
 }
 
diff --git a/arch/um/kernel/time.c b/arch/um/kernel/time.c
index f4db89b5b5a6..07d705ec5193 100644
--- a/arch/um/kernel/time.c
+++ b/arch/um/kernel/time.c
@@ -659,11 +659,13 @@ static struct clock_event_device timer_clockevent = {
 
 static irqreturn_t um_timer(int irq, void *dev)
 {
+#ifndef CONFIG_UMMODE_LIB
 	if (get_current()->mm != NULL)
 	{
         /* userspace - relay signal, results in correct userspace timers */
 		os_alarm_process(get_current()->mm->context.id.u.pid);
 	}
+#endif
 
 	(*timer_clockevent.event_handler)(&timer_clockevent);
 
diff --git a/arch/um/lkl/include/asm/cpu.h b/arch/um/lkl/include/asm/cpu.h
index c1164187151e..b241b5bbdc53 100644
--- a/arch/um/lkl/include/asm/cpu.h
+++ b/arch/um/lkl/include/asm/cpu.h
@@ -8,4 +8,8 @@ int lkl_cpu_init(void);
 void lkl_cpu_wait_shutdown(void);
 void lkl_cpu_change_owner(lkl_thread_t owner);
 
+int lkl_cpu_try_run_irq(int irq);
+void lkl_irq_exit(void);
+int lkl_irq_enter(int sig);
+
 #endif
diff --git a/arch/um/lkl/um/cpu.c b/arch/um/lkl/um/cpu.c
index 75452b28d741..841c7e9cc881 100644
--- a/arch/um/lkl/um/cpu.c
+++ b/arch/um/lkl/um/cpu.c
@@ -8,6 +8,7 @@
 #include <asm/sched.h>
 #include <asm/syscalls.h>
 #include <init.h>
+#include <os.h>
 
 /*
  * This structure is used to get access to the "LKL CPU" that allows us to run
@@ -41,6 +42,7 @@ static struct lkl_cpu {
 	 */
 	#define MAX_THREADS 1000000
 	unsigned int shutdown_gate;
+	bool irqs_pending;
 	/* no of threads waiting the CPU */
 	unsigned int sleepers;
 	/* no of times the current thread got the CPU */
@@ -53,6 +55,16 @@ static struct lkl_cpu {
 	struct lkl_sem *shutdown_sem;
 } cpu;
 
+static void run_irqs(void)
+{
+	unblock_signals();
+}
+
+static void set_irq_pending(int sig)
+{
+	set_pending_signals(sig);
+}
+
 /*
  * internal routine to acquire LKL CPU's lock
  */
@@ -132,6 +144,16 @@ void lkl_cpu_put(void)
 	    !lkl_thread_equal(cpu.owner, lkl_thread_self()))
 		lkl_bug("%s: unbalanced put\n", __func__);
 
+	/* we're going to trigger irq handlers if there are any pending
+	 * interrupts, and not irq_disabled.
+	 */
+	while (cpu.irqs_pending && !irqs_disabled()) {
+		cpu.irqs_pending = false;
+		lkl_mutex_unlock(cpu.lock);
+		run_irqs();
+		lkl_mutex_lock(cpu.lock);
+	}
+
 	/* switch to userspace code if current is host task (TIF_HOST_THREAD),
 	 * AND, there are other running tasks.
 	 */
@@ -163,6 +185,30 @@ void lkl_cpu_put(void)
 	lkl_mutex_unlock(cpu.lock);
 }
 
+int lkl_cpu_try_run_irq(int irq)
+{
+	int ret;
+
+	ret = __cpu_try_get_lock(1);
+	if (!ret) {
+		set_irq_pending(irq);
+		cpu.irqs_pending = true;
+	}
+	__cpu_try_get_unlock(ret, 1);
+
+	return ret;
+}
+
+int lkl_irq_enter(int sig)
+{
+	return lkl_cpu_try_run_irq(sig);
+}
+
+void lkl_irq_exit(void)
+{
+	return lkl_cpu_put();
+}
+
 static void lkl_cpu_shutdown(void)
 {
 	__sync_fetch_and_add(&cpu.shutdown_gate, MAX_THREADS);
diff --git a/arch/um/lkl/um/setup.c b/arch/um/lkl/um/setup.c
index ba8338d4fc23..12b235826573 100644
--- a/arch/um/lkl/um/setup.c
+++ b/arch/um/lkl/um/setup.c
@@ -36,6 +36,8 @@ static void __init *lkl_run_kernel(void *arg)
 
 	panic_blink = lkl_panic_blink;
 
+	/* signal should be received at this thread (main and idle threads) */
+	init_new_thread_signals();
 	threads_init();
 	lkl_cpu_get();
 	start_kernel();
@@ -58,6 +60,9 @@ int __init lkl_start_kernel(struct lkl_host_operations *ops,
 	if (ret)
 		goto out_free_init_sem;
 
+	change_sig(SIGALRM, 0);
+	change_sig(SIGIO, 0);
+
 	ret = lkl_thread_create(lkl_run_kernel, NULL);
 	if (!ret) {
 		ret = -ENOMEM;
diff --git a/arch/um/lkl/um/threads.c b/arch/um/lkl/um/threads.c
index 7ef9b9f2a6b7..c7ff578b7a91 100644
--- a/arch/um/lkl/um/threads.c
+++ b/arch/um/lkl/um/threads.c
@@ -152,6 +152,9 @@ static void *thread_bootstrap(void *_tba)
 	int (*f)(void *) = tba->f;
 	void *arg = tba->arg;
 
+	change_sig(SIGALRM, 0);
+	change_sig(SIGIO, 0);
+
 	lkl_sem_down(ti->task->thread.arch.sched_sem);
 	kfree(tba);
 	if (ti->task->thread.prev_sched)
diff --git a/tools/um/uml/signal.c b/tools/um/uml/signal.c
index 96f511d1aabe..de04b0dd34bb 100644
--- a/tools/um/uml/signal.c
+++ b/tools/um/uml/signal.c
@@ -230,8 +230,8 @@ void set_handler(int sig)
 
 	sigemptyset(&sig_mask);
 	sigaddset(&sig_mask, sig);
-	if (sigprocmask(SIG_UNBLOCK, &sig_mask, NULL) < 0)
-		panic("sigprocmask failed - errno = %d\n", errno);
+	if (pthread_sigmask(SIG_UNBLOCK, &sig_mask, NULL) < 0)
+		panic("pthread_sigmask failed - errno = %d\n", errno);
 }
 
 void send_sigio_to_self(void)
@@ -375,3 +375,11 @@ int os_is_signal_stack(void)
 
 	return ss.ss_flags & SS_ONSTACK;
 }
+
+void set_pending_signals(int sig)
+{
+	if (sig == SIGIO)
+		signals_pending |= SIGIO_MASK;
+	else if (sig == SIGALRM)
+		signals_pending |= SIGALRM_MASK;
+}
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v8 14/20] um: lkl: plug in the build system
  2021-01-20  2:27           ` Hajime Tazaki
@ 2021-01-20  2:27             ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-01-20  2:27 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: thehajime, tavi.purdila, retrage01, linux-kernel-library, linux-arch

Basic Makefiles for building library of library mode. Add a new
architecture specific target for installing the resulting library files
and headers.

To make LKL binaries build, we can specify the SUBARCH name as "lkl"
to switch the output file of build: kernel (default), or library
(library mode).  Those modes are not able to be ON at the same time.

To build on library mode, users do the following:

  make defconfig ARCH=um SUBARCH=lkl
  make ARCH=um SUBARCH=lkl

Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/Makefile              | 27 +++++++++----
 arch/um/configs/lkl_defconfig | 73 +++++++++++++++++++++++++++++++++++
 arch/um/kernel/Makefile       | 14 ++++---
 arch/um/lkl/um/Makefile       |  4 ++
 tools/um/Makefile             |  3 +-
 tools/um/Targets              |  3 ++
 6 files changed, 111 insertions(+), 13 deletions(-)
 create mode 100644 arch/um/configs/lkl_defconfig
 create mode 100644 arch/um/lkl/um/Makefile

diff --git a/arch/um/Makefile b/arch/um/Makefile
index bce0cccae085..6ddc900604a5 100644
--- a/arch/um/Makefile
+++ b/arch/um/Makefile
@@ -34,6 +34,10 @@ ifneq ($(filter $(SUBARCH),x86 x86_64 i386),)
 	HEADER_ARCH := x86
 endif
 
+ifeq ($(SUBARCH),lkl)
+	HEADER_ARCH := um/lkl
+endif
+
 ifdef CONFIG_64BIT
 	KBUILD_CFLAGS += -mcmodel=large
 endif
@@ -97,19 +101,27 @@ KBUILD_CFLAGS += $(KERNEL_DEFINES)
 LDFLAGS_vmlinux += -r
 
 INSTALL_PATH=$(objtree)/tools/um
-all: linux
-pre-2nd: linux.o
-	@echo "  INSTALL $(INSTALL_PATH)/lib/$<"
-	@mkdir -p $(INSTALL_PATH)/lib/
-	@cp $< $(INSTALL_PATH)/lib/
+ifeq ($(CONFIG_UMMODE_LIB),y)
+all: pre-2nd
+	$(Q)$(MAKE) -C $(srctree)/tools/um
 
-PHONY += linux.o
+pre-2nd: linux.o um_headers_install
+
+else
+all: linux
 
 linux: pre-2nd
 	$(Q)$(MAKE) -C $(srctree)/tools/um
 	$(Q)cp $(objtree)/tools/um/uml/linux $(objtree)/
 	$(Q)cp $(objtree)/linux $(objtree)/vmlinux
 
+pre-2nd: linux.o
+endif
+	@echo "  INSTALL $(INSTALL_PATH)/lib/$<"
+	@mkdir -p $(INSTALL_PATH)/lib/
+	@cp $< $(INSTALL_PATH)/lib/
+
+PHONY += linux.o
 linux.o: vmlinux
 #revert vmlinux file from vmlinux.a
 ifneq ("$(wildcard vmlinux.a)","")
@@ -117,7 +129,8 @@ ifneq ("$(wildcard vmlinux.a)","")
 endif
 	$(Q)cp vmlinux vmlinux.a
 	@echo "  LINK    $@"
-	$(Q)$(OBJCOPY) -R .eh_frame $< $@
+# symbols in ipc/sem.c conflict with sem_init(3) etc, thus localize the kernel one.
+	$(Q)$(OBJCOPY) -R .eh_frame -L sem_init -L sem_post -L sem_wait -L sem_destroy $< $@
 
 define archhelp
   echo '* linux		- Binary kernel image (./linux) - for backward'
diff --git a/arch/um/configs/lkl_defconfig b/arch/um/configs/lkl_defconfig
new file mode 100644
index 000000000000..7bea9e5b1b09
--- /dev/null
+++ b/arch/um/configs/lkl_defconfig
@@ -0,0 +1,73 @@
+CONFIG_SYSVIPC=y
+CONFIG_POSIX_MQUEUE=y
+CONFIG_NO_HZ=y
+CONFIG_HIGH_RES_TIMERS=y
+CONFIG_BSD_PROCESS_ACCT=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_LOG_BUF_SHIFT=14
+CONFIG_CGROUPS=y
+CONFIG_BLK_CGROUP=y
+CONFIG_CGROUP_SCHED=y
+CONFIG_CGROUP_FREEZER=y
+CONFIG_CGROUP_DEVICE=y
+CONFIG_CGROUP_CPUACCT=y
+# CONFIG_PID_NS is not set
+CONFIG_SYSFS_DEPRECATED=y
+CONFIG_CC_OPTIMIZE_FOR_SIZE=y
+CONFIG_SLAB=y
+CONFIG_UMMODE_LIB=y
+CONFIG_HOSTFS=y
+CONFIG_MAGIC_SYSRQ=y
+CONFIG_SSL=y
+CONFIG_NULL_CHAN=y
+CONFIG_PORT_CHAN=y
+CONFIG_PTY_CHAN=y
+CONFIG_TTY_CHAN=y
+CONFIG_XTERM_CHAN=y
+CONFIG_CON_CHAN="pts"
+CONFIG_SSL_CHAN="pts"
+CONFIG_UML_SOUND=m
+CONFIG_UML_NET=y
+CONFIG_UML_NET_ETHERTAP=y
+CONFIG_UML_NET_TUNTAP=y
+CONFIG_UML_NET_SLIP=y
+CONFIG_UML_NET_DAEMON=y
+CONFIG_UML_NET_MCAST=y
+CONFIG_UML_NET_SLIRP=y
+CONFIG_MODULES=y
+CONFIG_MODULE_UNLOAD=y
+# CONFIG_BLK_DEV_BSG is not set
+CONFIG_IOSCHED_BFQ=m
+CONFIG_BINFMT_MISC=m
+CONFIG_NET=y
+CONFIG_PACKET=y
+CONFIG_UNIX=y
+CONFIG_INET=y
+# CONFIG_IPV6 is not set
+CONFIG_DEVTMPFS=y
+CONFIG_DEVTMPFS_MOUNT=y
+CONFIG_BLK_DEV_UBD=y
+CONFIG_BLK_DEV_LOOP=m
+CONFIG_BLK_DEV_NBD=m
+CONFIG_DUMMY=m
+CONFIG_TUN=m
+CONFIG_PPP=m
+CONFIG_SLIP=m
+CONFIG_LEGACY_PTY_COUNT=32
+# CONFIG_HW_RANDOM is not set
+CONFIG_UML_RANDOM=y
+CONFIG_EXT4_FS=y
+CONFIG_REISERFS_FS=y
+CONFIG_BTRFS_FS=y
+CONFIG_QUOTA=y
+CONFIG_AUTOFS4_FS=m
+CONFIG_ISO9660_FS=m
+CONFIG_JOLIET=y
+CONFIG_VFAT_FS=y
+CONFIG_NLS_CODEPAGE_437=y
+CONFIG_NLS_ISO8859_1=y
+# CONFIG_RAID6_PQ_BENCHMARK is not set
+CONFIG_DEBUG_INFO=y
+CONFIG_FRAME_WARN=1024
+CONFIG_DEBUG_KERNEL=y
diff --git a/arch/um/kernel/Makefile b/arch/um/kernel/Makefile
index 9b63831a69e1..781c39b21403 100644
--- a/arch/um/kernel/Makefile
+++ b/arch/um/kernel/Makefile
@@ -14,17 +14,21 @@ CPPFLAGS_vmlinux.lds := -DSTART=$(LDS_START)		\
 			$(LDS_EXTRA)
 extra-y := vmlinux.lds
 
-obj-y = config.o exec.o exitcode.o irq.o ksyms.o mem.o \
-	physmem.o process.o ptrace.o reboot.o sigio.o \
-	signal.o syscall.o sysrq.o time.o tlb.o trap.o \
-	um_arch.o umid.o maccess.o kmsg_dump.o skas/
+obj-y = config.o exitcode.o irq.o ksyms.o \
+	process.o reboot.o sigio.o \
+	signal.o syscall.o time.o \
+	um_arch.o umid.o maccess.o kmsg_dump.o
+
+ifndef CONFIG_UMMODE_LIB
+obj-y += exec.o mem.o physmem.o ptrace.o sysrq.o tlb.o trap.o skas/
+endif
 
 obj-$(CONFIG_BLK_DEV_INITRD) += initrd.o
 obj-$(CONFIG_GPROF)	+= gprof_syms.o
 obj-$(CONFIG_GCOV)	+= gmon_syms.o
 obj-$(CONFIG_EARLY_PRINTK) += early_printk.o
 obj-$(CONFIG_STACKTRACE) += stacktrace.o
-obj-y += user_syms.o
+obj-$(CONFIG_MMU) += user_syms.o
 
 USER_OBJS := config.o
 
diff --git a/arch/um/lkl/um/Makefile b/arch/um/lkl/um/Makefile
new file mode 100644
index 000000000000..b29b598a1bac
--- /dev/null
+++ b/arch/um/lkl/um/Makefile
@@ -0,0 +1,4 @@
+include/generated/user_constants.h: $(srctree)/arch/um/lkl/um/user_constants.h
+	$(Q)cp -f $^ $@
+
+obj-y = bootmem.o console.o cpu.o delay.o setup.o syscalls.o threads.o unimplemented.o
diff --git a/tools/um/Makefile b/tools/um/Makefile
index 0fdf77261e74..877d0bdefa67 100644
--- a/tools/um/Makefile
+++ b/tools/um/Makefile
@@ -32,6 +32,7 @@ export CFLAGS += -I$(OUTPUT)/include -Iinclude -Wall -g -O2 -Wextra -fPIC \
 	 -Wno-unused-parameter \
 	 -Wno-missing-field-initializers -fno-strict-aliasing
 
+-include $(OUTPUT)/../../include/config/auto.conf
 -include Targets
 
 TARGETS := $(progs-y:%=$(OUTPUT)%)
@@ -69,6 +70,6 @@ clean:
 	$(call QUIET_CLEAN, $(TARGETS))$(RM) $(TARGETS)
 
 FORCE: ;
-.PHONY: all clean FORCE
+.PHONY: all clean FORCE $(OUTPUT)/../../include/config/auto.conf
 .IGNORE: clean
 .NOTPARALLEL : lib/linux.o
diff --git a/tools/um/Targets b/tools/um/Targets
index cfe1d3c3c6ff..e8b43c7758fe 100644
--- a/tools/um/Targets
+++ b/tools/um/Targets
@@ -1,4 +1,7 @@
+ifneq ($(CONFIG_UMMODE_LIB),y)
 progs-y += uml/linux
+endif
+
 LDLIBS_linux-y := -lrt -lpthread -lutil
 LDFLAGS_linux-y := -no-pie -Wl,--wrap,malloc -Wl,--wrap,free -Wl,--wrap,calloc
 LDFLAGS_linux-$(CONFIG_STATIC_LINK) += -static
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v8 14/20] um: lkl: plug in the build system
@ 2021-01-20  2:27             ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-01-20  2:27 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, thehajime, retrage01

Basic Makefiles for building library of library mode. Add a new
architecture specific target for installing the resulting library files
and headers.

To make LKL binaries build, we can specify the SUBARCH name as "lkl"
to switch the output file of build: kernel (default), or library
(library mode).  Those modes are not able to be ON at the same time.

To build on library mode, users do the following:

  make defconfig ARCH=um SUBARCH=lkl
  make ARCH=um SUBARCH=lkl

Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/Makefile              | 27 +++++++++----
 arch/um/configs/lkl_defconfig | 73 +++++++++++++++++++++++++++++++++++
 arch/um/kernel/Makefile       | 14 ++++---
 arch/um/lkl/um/Makefile       |  4 ++
 tools/um/Makefile             |  3 +-
 tools/um/Targets              |  3 ++
 6 files changed, 111 insertions(+), 13 deletions(-)
 create mode 100644 arch/um/configs/lkl_defconfig
 create mode 100644 arch/um/lkl/um/Makefile

diff --git a/arch/um/Makefile b/arch/um/Makefile
index bce0cccae085..6ddc900604a5 100644
--- a/arch/um/Makefile
+++ b/arch/um/Makefile
@@ -34,6 +34,10 @@ ifneq ($(filter $(SUBARCH),x86 x86_64 i386),)
 	HEADER_ARCH := x86
 endif
 
+ifeq ($(SUBARCH),lkl)
+	HEADER_ARCH := um/lkl
+endif
+
 ifdef CONFIG_64BIT
 	KBUILD_CFLAGS += -mcmodel=large
 endif
@@ -97,19 +101,27 @@ KBUILD_CFLAGS += $(KERNEL_DEFINES)
 LDFLAGS_vmlinux += -r
 
 INSTALL_PATH=$(objtree)/tools/um
-all: linux
-pre-2nd: linux.o
-	@echo "  INSTALL $(INSTALL_PATH)/lib/$<"
-	@mkdir -p $(INSTALL_PATH)/lib/
-	@cp $< $(INSTALL_PATH)/lib/
+ifeq ($(CONFIG_UMMODE_LIB),y)
+all: pre-2nd
+	$(Q)$(MAKE) -C $(srctree)/tools/um
 
-PHONY += linux.o
+pre-2nd: linux.o um_headers_install
+
+else
+all: linux
 
 linux: pre-2nd
 	$(Q)$(MAKE) -C $(srctree)/tools/um
 	$(Q)cp $(objtree)/tools/um/uml/linux $(objtree)/
 	$(Q)cp $(objtree)/linux $(objtree)/vmlinux
 
+pre-2nd: linux.o
+endif
+	@echo "  INSTALL $(INSTALL_PATH)/lib/$<"
+	@mkdir -p $(INSTALL_PATH)/lib/
+	@cp $< $(INSTALL_PATH)/lib/
+
+PHONY += linux.o
 linux.o: vmlinux
 #revert vmlinux file from vmlinux.a
 ifneq ("$(wildcard vmlinux.a)","")
@@ -117,7 +129,8 @@ ifneq ("$(wildcard vmlinux.a)","")
 endif
 	$(Q)cp vmlinux vmlinux.a
 	@echo "  LINK    $@"
-	$(Q)$(OBJCOPY) -R .eh_frame $< $@
+# symbols in ipc/sem.c conflict with sem_init(3) etc, thus localize the kernel one.
+	$(Q)$(OBJCOPY) -R .eh_frame -L sem_init -L sem_post -L sem_wait -L sem_destroy $< $@
 
 define archhelp
   echo '* linux		- Binary kernel image (./linux) - for backward'
diff --git a/arch/um/configs/lkl_defconfig b/arch/um/configs/lkl_defconfig
new file mode 100644
index 000000000000..7bea9e5b1b09
--- /dev/null
+++ b/arch/um/configs/lkl_defconfig
@@ -0,0 +1,73 @@
+CONFIG_SYSVIPC=y
+CONFIG_POSIX_MQUEUE=y
+CONFIG_NO_HZ=y
+CONFIG_HIGH_RES_TIMERS=y
+CONFIG_BSD_PROCESS_ACCT=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_LOG_BUF_SHIFT=14
+CONFIG_CGROUPS=y
+CONFIG_BLK_CGROUP=y
+CONFIG_CGROUP_SCHED=y
+CONFIG_CGROUP_FREEZER=y
+CONFIG_CGROUP_DEVICE=y
+CONFIG_CGROUP_CPUACCT=y
+# CONFIG_PID_NS is not set
+CONFIG_SYSFS_DEPRECATED=y
+CONFIG_CC_OPTIMIZE_FOR_SIZE=y
+CONFIG_SLAB=y
+CONFIG_UMMODE_LIB=y
+CONFIG_HOSTFS=y
+CONFIG_MAGIC_SYSRQ=y
+CONFIG_SSL=y
+CONFIG_NULL_CHAN=y
+CONFIG_PORT_CHAN=y
+CONFIG_PTY_CHAN=y
+CONFIG_TTY_CHAN=y
+CONFIG_XTERM_CHAN=y
+CONFIG_CON_CHAN="pts"
+CONFIG_SSL_CHAN="pts"
+CONFIG_UML_SOUND=m
+CONFIG_UML_NET=y
+CONFIG_UML_NET_ETHERTAP=y
+CONFIG_UML_NET_TUNTAP=y
+CONFIG_UML_NET_SLIP=y
+CONFIG_UML_NET_DAEMON=y
+CONFIG_UML_NET_MCAST=y
+CONFIG_UML_NET_SLIRP=y
+CONFIG_MODULES=y
+CONFIG_MODULE_UNLOAD=y
+# CONFIG_BLK_DEV_BSG is not set
+CONFIG_IOSCHED_BFQ=m
+CONFIG_BINFMT_MISC=m
+CONFIG_NET=y
+CONFIG_PACKET=y
+CONFIG_UNIX=y
+CONFIG_INET=y
+# CONFIG_IPV6 is not set
+CONFIG_DEVTMPFS=y
+CONFIG_DEVTMPFS_MOUNT=y
+CONFIG_BLK_DEV_UBD=y
+CONFIG_BLK_DEV_LOOP=m
+CONFIG_BLK_DEV_NBD=m
+CONFIG_DUMMY=m
+CONFIG_TUN=m
+CONFIG_PPP=m
+CONFIG_SLIP=m
+CONFIG_LEGACY_PTY_COUNT=32
+# CONFIG_HW_RANDOM is not set
+CONFIG_UML_RANDOM=y
+CONFIG_EXT4_FS=y
+CONFIG_REISERFS_FS=y
+CONFIG_BTRFS_FS=y
+CONFIG_QUOTA=y
+CONFIG_AUTOFS4_FS=m
+CONFIG_ISO9660_FS=m
+CONFIG_JOLIET=y
+CONFIG_VFAT_FS=y
+CONFIG_NLS_CODEPAGE_437=y
+CONFIG_NLS_ISO8859_1=y
+# CONFIG_RAID6_PQ_BENCHMARK is not set
+CONFIG_DEBUG_INFO=y
+CONFIG_FRAME_WARN=1024
+CONFIG_DEBUG_KERNEL=y
diff --git a/arch/um/kernel/Makefile b/arch/um/kernel/Makefile
index 9b63831a69e1..781c39b21403 100644
--- a/arch/um/kernel/Makefile
+++ b/arch/um/kernel/Makefile
@@ -14,17 +14,21 @@ CPPFLAGS_vmlinux.lds := -DSTART=$(LDS_START)		\
 			$(LDS_EXTRA)
 extra-y := vmlinux.lds
 
-obj-y = config.o exec.o exitcode.o irq.o ksyms.o mem.o \
-	physmem.o process.o ptrace.o reboot.o sigio.o \
-	signal.o syscall.o sysrq.o time.o tlb.o trap.o \
-	um_arch.o umid.o maccess.o kmsg_dump.o skas/
+obj-y = config.o exitcode.o irq.o ksyms.o \
+	process.o reboot.o sigio.o \
+	signal.o syscall.o time.o \
+	um_arch.o umid.o maccess.o kmsg_dump.o
+
+ifndef CONFIG_UMMODE_LIB
+obj-y += exec.o mem.o physmem.o ptrace.o sysrq.o tlb.o trap.o skas/
+endif
 
 obj-$(CONFIG_BLK_DEV_INITRD) += initrd.o
 obj-$(CONFIG_GPROF)	+= gprof_syms.o
 obj-$(CONFIG_GCOV)	+= gmon_syms.o
 obj-$(CONFIG_EARLY_PRINTK) += early_printk.o
 obj-$(CONFIG_STACKTRACE) += stacktrace.o
-obj-y += user_syms.o
+obj-$(CONFIG_MMU) += user_syms.o
 
 USER_OBJS := config.o
 
diff --git a/arch/um/lkl/um/Makefile b/arch/um/lkl/um/Makefile
new file mode 100644
index 000000000000..b29b598a1bac
--- /dev/null
+++ b/arch/um/lkl/um/Makefile
@@ -0,0 +1,4 @@
+include/generated/user_constants.h: $(srctree)/arch/um/lkl/um/user_constants.h
+	$(Q)cp -f $^ $@
+
+obj-y = bootmem.o console.o cpu.o delay.o setup.o syscalls.o threads.o unimplemented.o
diff --git a/tools/um/Makefile b/tools/um/Makefile
index 0fdf77261e74..877d0bdefa67 100644
--- a/tools/um/Makefile
+++ b/tools/um/Makefile
@@ -32,6 +32,7 @@ export CFLAGS += -I$(OUTPUT)/include -Iinclude -Wall -g -O2 -Wextra -fPIC \
 	 -Wno-unused-parameter \
 	 -Wno-missing-field-initializers -fno-strict-aliasing
 
+-include $(OUTPUT)/../../include/config/auto.conf
 -include Targets
 
 TARGETS := $(progs-y:%=$(OUTPUT)%)
@@ -69,6 +70,6 @@ clean:
 	$(call QUIET_CLEAN, $(TARGETS))$(RM) $(TARGETS)
 
 FORCE: ;
-.PHONY: all clean FORCE
+.PHONY: all clean FORCE $(OUTPUT)/../../include/config/auto.conf
 .IGNORE: clean
 .NOTPARALLEL : lib/linux.o
diff --git a/tools/um/Targets b/tools/um/Targets
index cfe1d3c3c6ff..e8b43c7758fe 100644
--- a/tools/um/Targets
+++ b/tools/um/Targets
@@ -1,4 +1,7 @@
+ifneq ($(CONFIG_UMMODE_LIB),y)
 progs-y += uml/linux
+endif
+
 LDLIBS_linux-y := -lrt -lpthread -lutil
 LDFLAGS_linux-y := -no-pie -Wl,--wrap,malloc -Wl,--wrap,free -Wl,--wrap,calloc
 LDFLAGS_linux-$(CONFIG_STATIC_LINK) += -static
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v8 15/20] um: host: add library mode build for ARCH=um
  2021-01-20  2:27           ` Hajime Tazaki
@ 2021-01-20  2:27             ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-01-20  2:27 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: thehajime, tavi.purdila, retrage01, linux-kernel-library, linux-arch

This patch adds the skeleton for the host library.

The host library is implementing the host operations needed by library
mode and splits into host dependent (e.g., POSIX hosts) and host
independent parts.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/um/.gitignore              |   1 +
 tools/um/Makefile                |   1 +
 tools/um/Targets                 |   6 +-
 tools/um/include/lkl.h           | 135 +++++++++++++++++++++++++++++++
 tools/um/include/lkl_host.h      |  18 +++++
 tools/um/uml/Build               |  16 +++-
 tools/um/uml/lkl/Build           |   1 +
 tools/um/uml/lkl/registers.c     |  21 +++++
 tools/um/uml/lkl/unimplemented.c |  21 +++++
 9 files changed, 214 insertions(+), 6 deletions(-)
 create mode 100644 tools/um/.gitignore
 create mode 100644 tools/um/include/lkl.h
 create mode 100644 tools/um/include/lkl_host.h
 create mode 100644 tools/um/uml/lkl/Build
 create mode 100644 tools/um/uml/lkl/registers.c
 create mode 100644 tools/um/uml/lkl/unimplemented.c

diff --git a/tools/um/.gitignore b/tools/um/.gitignore
new file mode 100644
index 000000000000..0d20b6487c61
--- /dev/null
+++ b/tools/um/.gitignore
@@ -0,0 +1 @@
+*.pyc
diff --git a/tools/um/Makefile b/tools/um/Makefile
index 877d0bdefa67..c07e91a93372 100644
--- a/tools/um/Makefile
+++ b/tools/um/Makefile
@@ -66,6 +66,7 @@ RM := rm -f
 clean:
 	$(call QUIET_CLEAN, objects)find $(OUTPUT) -name '*.o' -delete -o -name '\.*.cmd'\
 	 -delete -o -name '\.*.d' -delete
+	$(call QUIET_CLEAN, headers)$(RM) -r $(OUTPUT)/include/lkl/
 	$(call QUIET_CLEAN, liblinux.a)$(RM) $(OUTPUT)/liblinux.a
 	$(call QUIET_CLEAN, $(TARGETS))$(RM) $(TARGETS)
 
diff --git a/tools/um/Targets b/tools/um/Targets
index e8b43c7758fe..ae6c8d8b168e 100644
--- a/tools/um/Targets
+++ b/tools/um/Targets
@@ -1,7 +1,9 @@
-ifneq ($(CONFIG_UMMODE_LIB),y)
+ifeq ($(CONFIG_UMMODE_LIB),y)
+libs-y += liblinux.a
+else
 progs-y += uml/linux
 endif
 
-LDLIBS_linux-y := -lrt -lpthread -lutil
+LDLIBS := -lrt -lpthread -lutil
 LDFLAGS_linux-y := -no-pie -Wl,--wrap,malloc -Wl,--wrap,free -Wl,--wrap,calloc
 LDFLAGS_linux-$(CONFIG_STATIC_LINK) += -static
diff --git a/tools/um/include/lkl.h b/tools/um/include/lkl.h
new file mode 100644
index 000000000000..2417ed5ccf71
--- /dev/null
+++ b/tools/um/include/lkl.h
@@ -0,0 +1,135 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_H
+#define _LKL_H
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#define _LKL_LIBC_COMPAT_H
+
+#ifdef __cplusplus
+#define class __lkl__class
+#endif
+
+/*
+ * Avoid collisions between Android which defines __unused and
+ * linux/icmp.h which uses __unused as a structure field.
+ */
+#pragma push_macro("__unused")
+#undef __unused
+
+#include <lkl/asm/syscalls.h>
+
+#pragma pop_macro("__unused")
+
+#ifdef __cplusplus
+#undef class
+#endif
+
+#if __LKL__BITS_PER_LONG == 64
+#define lkl_sys_fstatat lkl_sys_newfstatat
+#define lkl_sys_fstat lkl_sys_newfstat
+
+#else
+#define __lkl__NR_fcntl __lkl__NR_fcntl64
+
+#define lkl_stat lkl_stat64
+#define lkl_sys_stat lkl_sys_stat64
+#define lkl_sys_lstat lkl_sys_lstat64
+#define lkl_sys_truncate lkl_sys_truncate64
+#define lkl_sys_ftruncate lkl_sys_ftruncate64
+#define lkl_sys_sendfile lkl_sys_sendfile64
+#define lkl_sys_fstatat lkl_sys_fstatat64
+#define lkl_sys_fstat lkl_sys_fstat64
+#define lkl_sys_fcntl lkl_sys_fcntl64
+
+#define lkl_statfs lkl_statfs64
+
+static inline int lkl_sys_statfs(const char *path, struct lkl_statfs *buf)
+{
+	return lkl_sys_statfs64(path, sizeof(*buf), buf);
+}
+
+static inline int lkl_sys_fstatfs(unsigned int fd, struct lkl_statfs *buf)
+{
+	return lkl_sys_fstatfs64(fd, sizeof(*buf), buf);
+}
+
+#define lkl_sys_nanosleep lkl_sys_nanosleep_time32
+static inline int lkl_sys_nanosleep_time32(struct lkl_timespec *rqtp,
+					   struct lkl_timespec *rmtp)
+{
+	long p[6] = {(long)rqtp, (long)rmtp, 0, 0, 0, 0};
+
+	return lkl_syscall(__lkl__NR_nanosleep, p);
+}
+
+#endif
+
+static inline int lkl_sys_stat(const char *path, struct lkl_stat *buf)
+{
+	return lkl_sys_fstatat(LKL_AT_FDCWD, path, buf, 0);
+}
+
+static inline int lkl_sys_lstat(const char *path, struct lkl_stat *buf)
+{
+	return lkl_sys_fstatat(LKL_AT_FDCWD, path, buf,
+			       LKL_AT_SYMLINK_NOFOLLOW);
+}
+
+#ifdef __lkl__NR_openat
+/**
+ * lkl_sys_open - wrapper for lkl_sys_openat
+ */
+static inline long lkl_sys_open(const char *file, int flags, int mode)
+{
+	return lkl_sys_openat(LKL_AT_FDCWD, file, flags, mode);
+}
+
+/**
+ * lkl_sys_creat - wrapper for lkl_sys_openat
+ */
+static inline long lkl_sys_creat(const char *file, int mode)
+{
+	return lkl_sys_openat(LKL_AT_FDCWD, file,
+			      LKL_O_CREAT|LKL_O_WRONLY|LKL_O_TRUNC, mode);
+}
+#endif
+
+#ifdef __lkl__NR_mkdirat
+/**
+ * lkl_sys_mkdir - wrapper for lkl_sys_mkdirat
+ */
+static inline long lkl_sys_mkdir(const char *path, mode_t mode)
+{
+	return lkl_sys_mkdirat(LKL_AT_FDCWD, path, mode);
+}
+#endif
+
+#ifdef __lkl__NR_epoll_create1
+/**
+ * lkl_sys_epoll_create - wrapper for lkl_sys_epoll_create1
+ */
+static inline long lkl_sys_epoll_create(int size)
+{
+	return lkl_sys_epoll_create1(0);
+}
+#endif
+
+#ifdef __lkl__NR_epoll_pwait
+/**
+ * lkl_sys_epoll_wait - wrapper for lkl_sys_epoll_pwait
+ */
+static inline long lkl_sys_epoll_wait(int fd, struct lkl_epoll_event *ev,
+				      int cnt, int to)
+{
+	return lkl_sys_epoll_pwait(fd, ev, cnt, to, 0, _LKL_NSIG/8);
+}
+#endif
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif
diff --git a/tools/um/include/lkl_host.h b/tools/um/include/lkl_host.h
new file mode 100644
index 000000000000..5fe7a64dc4dd
--- /dev/null
+++ b/tools/um/include/lkl_host.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_HOST_H
+#define _LKL_HOST_H
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include <lkl/asm/host_ops.h>
+#include <lkl.h>
+
+extern struct lkl_host_operations lkl_host_ops;
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif
diff --git a/tools/um/uml/Build b/tools/um/uml/Build
index 810aa99f8409..b523923afd45 100644
--- a/tools/um/uml/Build
+++ b/tools/um/uml/Build
@@ -8,13 +8,21 @@ KCOV_INSTRUMENT                := n
 
 include $(objtree)/include/config/auto.conf
 
-liblinux-y += execvp.o file.o helper.o irq.o main.o mem.o process.o \
-	registers.o sigio.o signal.o start_up.o time.o tty.o \
-	umid.o util.o drivers/ skas/
+liblinux-y += execvp.o file.o helper.o irq.o mem.o process.o \
+	registers.o sigio.o signal.o time.o tty.o \
+	umid.o util.o drivers/
 
 CFLAGS_signal.o += -Wframe-larger-than=4096
 
+ifndef CONFIG_UMMODE_LIB
+liblinux-y += main.o start_up.o skas/
+HEADER_ARCH 	:= x86
 liblinux-y += x86/
+else
+HEADER_ARCH 	:= um/lkl
+liblinux-y += lkl/
+endif
+
 liblinux-$(CONFIG_ARCH_REUSE_HOST_VSYSCALL_AREA) += elf_aux.o
 
 export O = $(srctree)
@@ -24,7 +32,6 @@ CFLAGS := -g -O2
 
 # from arch/um/Makefile
 ARCH_DIR := arch/um
-HEADER_ARCH 	:= x86
 HOST_DIR := arch/$(HEADER_ARCH)
 ifdef CONFIG_64BIT
   KBUILD_CFLAGS += -mcmodel=large
@@ -34,6 +41,7 @@ KBUILD_CFLAGS += $(CFLAGS) $(CFLAGS-y) -D__arch_um__ \
 	-Dlongjmp=kernel_longjmp -Dsetjmp=kernel_setjmp \
 	-Din6addr_loopback=kernel_in6addr_loopback \
 	-Din6addr_any=kernel_in6addr_any -Dstrrchr=kernel_strrchr
+KBUILD_CFLAGS := $(filter-out -Dsigprocmask=kernel_sigprocmask,$(KBUILD_CFLAGS))
 SHARED_HEADERS	:= $(ARCH_DIR)/include/shared
 MODE_INCLUDE	+= -I$(srctree)/$(ARCH_DIR)/include/shared/skas
 ARCH_INCLUDE	:= -I$(srctree)/$(SHARED_HEADERS)
diff --git a/tools/um/uml/lkl/Build b/tools/um/uml/lkl/Build
new file mode 100644
index 000000000000..98fd6b86a085
--- /dev/null
+++ b/tools/um/uml/lkl/Build
@@ -0,0 +1 @@
+liblinux-y = registers.o unimplemented.o
diff --git a/tools/um/uml/lkl/registers.c b/tools/um/uml/lkl/registers.c
new file mode 100644
index 000000000000..11573a204720
--- /dev/null
+++ b/tools/um/uml/lkl/registers.c
@@ -0,0 +1,21 @@
+// SPDX-License-Identifier: GPL-2.0
+
+struct uml_pt_regs;
+
+int get_fp_registers(int pid, unsigned long *regs)
+{
+	return 0;
+}
+
+int save_i387_registers(int pid, unsigned long *fp_regs)
+{
+	return 0;
+}
+
+void arch_init_registers(int pid)
+{
+}
+
+void get_regs_from_mc(struct uml_pt_regs *regs, void *mc)
+{
+}
diff --git a/tools/um/uml/lkl/unimplemented.c b/tools/um/uml/lkl/unimplemented.c
new file mode 100644
index 000000000000..9da3e5c8bafb
--- /dev/null
+++ b/tools/um/uml/lkl/unimplemented.c
@@ -0,0 +1,21 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <generated/user_constants.h>
+
+struct uml_pt_regs;
+
+/* os-Linux/skas/process.c */
+int userspace_pid[UM_NR_CPUS];
+void userspace(struct uml_pt_regs *regs, unsigned long *aux_fp_regs)
+{}
+
+
+/* x86/os-Linux/task_size.c */
+unsigned long os_get_top_address(void)
+{
+	return 0;
+}
+
+/* start-up.c */
+void os_early_checks(void)
+{
+}
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v8 15/20] um: host: add library mode build for ARCH=um
@ 2021-01-20  2:27             ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-01-20  2:27 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, thehajime, retrage01

This patch adds the skeleton for the host library.

The host library is implementing the host operations needed by library
mode and splits into host dependent (e.g., POSIX hosts) and host
independent parts.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/um/.gitignore              |   1 +
 tools/um/Makefile                |   1 +
 tools/um/Targets                 |   6 +-
 tools/um/include/lkl.h           | 135 +++++++++++++++++++++++++++++++
 tools/um/include/lkl_host.h      |  18 +++++
 tools/um/uml/Build               |  16 +++-
 tools/um/uml/lkl/Build           |   1 +
 tools/um/uml/lkl/registers.c     |  21 +++++
 tools/um/uml/lkl/unimplemented.c |  21 +++++
 9 files changed, 214 insertions(+), 6 deletions(-)
 create mode 100644 tools/um/.gitignore
 create mode 100644 tools/um/include/lkl.h
 create mode 100644 tools/um/include/lkl_host.h
 create mode 100644 tools/um/uml/lkl/Build
 create mode 100644 tools/um/uml/lkl/registers.c
 create mode 100644 tools/um/uml/lkl/unimplemented.c

diff --git a/tools/um/.gitignore b/tools/um/.gitignore
new file mode 100644
index 000000000000..0d20b6487c61
--- /dev/null
+++ b/tools/um/.gitignore
@@ -0,0 +1 @@
+*.pyc
diff --git a/tools/um/Makefile b/tools/um/Makefile
index 877d0bdefa67..c07e91a93372 100644
--- a/tools/um/Makefile
+++ b/tools/um/Makefile
@@ -66,6 +66,7 @@ RM := rm -f
 clean:
 	$(call QUIET_CLEAN, objects)find $(OUTPUT) -name '*.o' -delete -o -name '\.*.cmd'\
 	 -delete -o -name '\.*.d' -delete
+	$(call QUIET_CLEAN, headers)$(RM) -r $(OUTPUT)/include/lkl/
 	$(call QUIET_CLEAN, liblinux.a)$(RM) $(OUTPUT)/liblinux.a
 	$(call QUIET_CLEAN, $(TARGETS))$(RM) $(TARGETS)
 
diff --git a/tools/um/Targets b/tools/um/Targets
index e8b43c7758fe..ae6c8d8b168e 100644
--- a/tools/um/Targets
+++ b/tools/um/Targets
@@ -1,7 +1,9 @@
-ifneq ($(CONFIG_UMMODE_LIB),y)
+ifeq ($(CONFIG_UMMODE_LIB),y)
+libs-y += liblinux.a
+else
 progs-y += uml/linux
 endif
 
-LDLIBS_linux-y := -lrt -lpthread -lutil
+LDLIBS := -lrt -lpthread -lutil
 LDFLAGS_linux-y := -no-pie -Wl,--wrap,malloc -Wl,--wrap,free -Wl,--wrap,calloc
 LDFLAGS_linux-$(CONFIG_STATIC_LINK) += -static
diff --git a/tools/um/include/lkl.h b/tools/um/include/lkl.h
new file mode 100644
index 000000000000..2417ed5ccf71
--- /dev/null
+++ b/tools/um/include/lkl.h
@@ -0,0 +1,135 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_H
+#define _LKL_H
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#define _LKL_LIBC_COMPAT_H
+
+#ifdef __cplusplus
+#define class __lkl__class
+#endif
+
+/*
+ * Avoid collisions between Android which defines __unused and
+ * linux/icmp.h which uses __unused as a structure field.
+ */
+#pragma push_macro("__unused")
+#undef __unused
+
+#include <lkl/asm/syscalls.h>
+
+#pragma pop_macro("__unused")
+
+#ifdef __cplusplus
+#undef class
+#endif
+
+#if __LKL__BITS_PER_LONG == 64
+#define lkl_sys_fstatat lkl_sys_newfstatat
+#define lkl_sys_fstat lkl_sys_newfstat
+
+#else
+#define __lkl__NR_fcntl __lkl__NR_fcntl64
+
+#define lkl_stat lkl_stat64
+#define lkl_sys_stat lkl_sys_stat64
+#define lkl_sys_lstat lkl_sys_lstat64
+#define lkl_sys_truncate lkl_sys_truncate64
+#define lkl_sys_ftruncate lkl_sys_ftruncate64
+#define lkl_sys_sendfile lkl_sys_sendfile64
+#define lkl_sys_fstatat lkl_sys_fstatat64
+#define lkl_sys_fstat lkl_sys_fstat64
+#define lkl_sys_fcntl lkl_sys_fcntl64
+
+#define lkl_statfs lkl_statfs64
+
+static inline int lkl_sys_statfs(const char *path, struct lkl_statfs *buf)
+{
+	return lkl_sys_statfs64(path, sizeof(*buf), buf);
+}
+
+static inline int lkl_sys_fstatfs(unsigned int fd, struct lkl_statfs *buf)
+{
+	return lkl_sys_fstatfs64(fd, sizeof(*buf), buf);
+}
+
+#define lkl_sys_nanosleep lkl_sys_nanosleep_time32
+static inline int lkl_sys_nanosleep_time32(struct lkl_timespec *rqtp,
+					   struct lkl_timespec *rmtp)
+{
+	long p[6] = {(long)rqtp, (long)rmtp, 0, 0, 0, 0};
+
+	return lkl_syscall(__lkl__NR_nanosleep, p);
+}
+
+#endif
+
+static inline int lkl_sys_stat(const char *path, struct lkl_stat *buf)
+{
+	return lkl_sys_fstatat(LKL_AT_FDCWD, path, buf, 0);
+}
+
+static inline int lkl_sys_lstat(const char *path, struct lkl_stat *buf)
+{
+	return lkl_sys_fstatat(LKL_AT_FDCWD, path, buf,
+			       LKL_AT_SYMLINK_NOFOLLOW);
+}
+
+#ifdef __lkl__NR_openat
+/**
+ * lkl_sys_open - wrapper for lkl_sys_openat
+ */
+static inline long lkl_sys_open(const char *file, int flags, int mode)
+{
+	return lkl_sys_openat(LKL_AT_FDCWD, file, flags, mode);
+}
+
+/**
+ * lkl_sys_creat - wrapper for lkl_sys_openat
+ */
+static inline long lkl_sys_creat(const char *file, int mode)
+{
+	return lkl_sys_openat(LKL_AT_FDCWD, file,
+			      LKL_O_CREAT|LKL_O_WRONLY|LKL_O_TRUNC, mode);
+}
+#endif
+
+#ifdef __lkl__NR_mkdirat
+/**
+ * lkl_sys_mkdir - wrapper for lkl_sys_mkdirat
+ */
+static inline long lkl_sys_mkdir(const char *path, mode_t mode)
+{
+	return lkl_sys_mkdirat(LKL_AT_FDCWD, path, mode);
+}
+#endif
+
+#ifdef __lkl__NR_epoll_create1
+/**
+ * lkl_sys_epoll_create - wrapper for lkl_sys_epoll_create1
+ */
+static inline long lkl_sys_epoll_create(int size)
+{
+	return lkl_sys_epoll_create1(0);
+}
+#endif
+
+#ifdef __lkl__NR_epoll_pwait
+/**
+ * lkl_sys_epoll_wait - wrapper for lkl_sys_epoll_pwait
+ */
+static inline long lkl_sys_epoll_wait(int fd, struct lkl_epoll_event *ev,
+				      int cnt, int to)
+{
+	return lkl_sys_epoll_pwait(fd, ev, cnt, to, 0, _LKL_NSIG/8);
+}
+#endif
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif
diff --git a/tools/um/include/lkl_host.h b/tools/um/include/lkl_host.h
new file mode 100644
index 000000000000..5fe7a64dc4dd
--- /dev/null
+++ b/tools/um/include/lkl_host.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_HOST_H
+#define _LKL_HOST_H
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include <lkl/asm/host_ops.h>
+#include <lkl.h>
+
+extern struct lkl_host_operations lkl_host_ops;
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif
diff --git a/tools/um/uml/Build b/tools/um/uml/Build
index 810aa99f8409..b523923afd45 100644
--- a/tools/um/uml/Build
+++ b/tools/um/uml/Build
@@ -8,13 +8,21 @@ KCOV_INSTRUMENT                := n
 
 include $(objtree)/include/config/auto.conf
 
-liblinux-y += execvp.o file.o helper.o irq.o main.o mem.o process.o \
-	registers.o sigio.o signal.o start_up.o time.o tty.o \
-	umid.o util.o drivers/ skas/
+liblinux-y += execvp.o file.o helper.o irq.o mem.o process.o \
+	registers.o sigio.o signal.o time.o tty.o \
+	umid.o util.o drivers/
 
 CFLAGS_signal.o += -Wframe-larger-than=4096
 
+ifndef CONFIG_UMMODE_LIB
+liblinux-y += main.o start_up.o skas/
+HEADER_ARCH 	:= x86
 liblinux-y += x86/
+else
+HEADER_ARCH 	:= um/lkl
+liblinux-y += lkl/
+endif
+
 liblinux-$(CONFIG_ARCH_REUSE_HOST_VSYSCALL_AREA) += elf_aux.o
 
 export O = $(srctree)
@@ -24,7 +32,6 @@ CFLAGS := -g -O2
 
 # from arch/um/Makefile
 ARCH_DIR := arch/um
-HEADER_ARCH 	:= x86
 HOST_DIR := arch/$(HEADER_ARCH)
 ifdef CONFIG_64BIT
   KBUILD_CFLAGS += -mcmodel=large
@@ -34,6 +41,7 @@ KBUILD_CFLAGS += $(CFLAGS) $(CFLAGS-y) -D__arch_um__ \
 	-Dlongjmp=kernel_longjmp -Dsetjmp=kernel_setjmp \
 	-Din6addr_loopback=kernel_in6addr_loopback \
 	-Din6addr_any=kernel_in6addr_any -Dstrrchr=kernel_strrchr
+KBUILD_CFLAGS := $(filter-out -Dsigprocmask=kernel_sigprocmask,$(KBUILD_CFLAGS))
 SHARED_HEADERS	:= $(ARCH_DIR)/include/shared
 MODE_INCLUDE	+= -I$(srctree)/$(ARCH_DIR)/include/shared/skas
 ARCH_INCLUDE	:= -I$(srctree)/$(SHARED_HEADERS)
diff --git a/tools/um/uml/lkl/Build b/tools/um/uml/lkl/Build
new file mode 100644
index 000000000000..98fd6b86a085
--- /dev/null
+++ b/tools/um/uml/lkl/Build
@@ -0,0 +1 @@
+liblinux-y = registers.o unimplemented.o
diff --git a/tools/um/uml/lkl/registers.c b/tools/um/uml/lkl/registers.c
new file mode 100644
index 000000000000..11573a204720
--- /dev/null
+++ b/tools/um/uml/lkl/registers.c
@@ -0,0 +1,21 @@
+// SPDX-License-Identifier: GPL-2.0
+
+struct uml_pt_regs;
+
+int get_fp_registers(int pid, unsigned long *regs)
+{
+	return 0;
+}
+
+int save_i387_registers(int pid, unsigned long *fp_regs)
+{
+	return 0;
+}
+
+void arch_init_registers(int pid)
+{
+}
+
+void get_regs_from_mc(struct uml_pt_regs *regs, void *mc)
+{
+}
diff --git a/tools/um/uml/lkl/unimplemented.c b/tools/um/uml/lkl/unimplemented.c
new file mode 100644
index 000000000000..9da3e5c8bafb
--- /dev/null
+++ b/tools/um/uml/lkl/unimplemented.c
@@ -0,0 +1,21 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <generated/user_constants.h>
+
+struct uml_pt_regs;
+
+/* os-Linux/skas/process.c */
+int userspace_pid[UM_NR_CPUS];
+void userspace(struct uml_pt_regs *regs, unsigned long *aux_fp_regs)
+{}
+
+
+/* x86/os-Linux/task_size.c */
+unsigned long os_get_top_address(void)
+{
+	return 0;
+}
+
+/* start-up.c */
+void os_early_checks(void)
+{
+}
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v8 16/20] um: host: add utilities functions
  2021-01-20  2:27           ` Hajime Tazaki
@ 2021-01-20  2:27             ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-01-20  2:27 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: thehajime, tavi.purdila, retrage01, linux-kernel-library, linux-arch

Add basic utility functions for getting a string from a kernel error
code and a fprintf like function that uses the host print
operation. The latter is useful for informing the user about errors
that occur in the host library.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 tools/um/include/lkl.h |  23 +++++
 tools/um/lib/Build     |   3 +
 tools/um/lib/utils.c   | 205 +++++++++++++++++++++++++++++++++++++++++
 3 files changed, 231 insertions(+)
 create mode 100644 tools/um/lib/Build
 create mode 100644 tools/um/lib/utils.c

diff --git a/tools/um/include/lkl.h b/tools/um/include/lkl.h
index 2417ed5ccf71..97da2d11502f 100644
--- a/tools/um/include/lkl.h
+++ b/tools/um/include/lkl.h
@@ -27,6 +27,29 @@ extern "C" {
 #undef class
 #endif
 
+/**
+ * lkl_printf - print a message via the host print operation
+ *
+ * @fmt: printf like format string
+ */
+int lkl_printf(const char *fmt, ...);
+
+/**
+ * lkl_strerror - returns a string describing the given error code
+ *
+ * @err - error code
+ * @returns - string for the given error code
+ */
+const char *lkl_strerror(int err);
+
+/**
+ * lkl_perror - prints a string describing the given error code
+ *
+ * @msg - prefix for the error message
+ * @err - error code
+ */
+void lkl_perror(char *msg, int err);
+
 #if __LKL__BITS_PER_LONG == 64
 #define lkl_sys_fstatat lkl_sys_newfstatat
 #define lkl_sys_fstat lkl_sys_newfstat
diff --git a/tools/um/lib/Build b/tools/um/lib/Build
new file mode 100644
index 000000000000..77acd6f55e76
--- /dev/null
+++ b/tools/um/lib/Build
@@ -0,0 +1,3 @@
+include $(objtree)/include/config/auto.conf
+
+liblinux-$(CONFIG_UMMODE_LIB) += utils.o
diff --git a/tools/um/lib/utils.c b/tools/um/lib/utils.c
new file mode 100644
index 000000000000..3b38e1a95124
--- /dev/null
+++ b/tools/um/lib/utils.c
@@ -0,0 +1,205 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdarg.h>
+#include <stdio.h>
+#include <string.h>
+#include <lkl_host.h>
+
+static const char * const lkl_err_strings[] = {
+	[0]			= "Success",
+	[LKL_EPERM]		= "Operation not permitted",
+	[LKL_ENOENT]		= "No such file or directory",
+	[LKL_ESRCH]		= "No such process",
+	[LKL_EINTR]		= "Interrupted system call",
+	[LKL_EIO]		= "I/O error",
+	[LKL_ENXIO]		= "No such device or address",
+	[LKL_E2BIG]		= "Argument list too long",
+	[LKL_ENOEXEC]		= "Exec format error",
+	[LKL_EBADF]		= "Bad file number",
+	[LKL_ECHILD]		= "No child processes",
+	[LKL_EAGAIN]		= "Try again",
+	[LKL_ENOMEM]		= "Out of memory",
+	[LKL_EACCES]		= "Permission denied",
+	[LKL_EFAULT]		= "Bad address",
+	[LKL_ENOTBLK]		= "Block device required",
+	[LKL_EBUSY]		= "Device or resource busy",
+	[LKL_EEXIST]		= "File exists",
+	[LKL_EXDEV]		= "Cross-device link",
+	[LKL_ENODEV]		= "No such device",
+	[LKL_ENOTDIR]		= "Not a directory",
+	[LKL_EISDIR]		= "Is a directory",
+	[LKL_EINVAL]		= "Invalid argument",
+	[LKL_ENFILE]		= "File table overflow",
+	[LKL_EMFILE]		= "Too many open files",
+	[LKL_ENOTTY]		= "Not a typewriter",
+	[LKL_ETXTBSY]		= "Text file busy",
+	[LKL_EFBIG]		= "File too large",
+	[LKL_ENOSPC]		= "No space left on device",
+	[LKL_ESPIPE]		= "Illegal seek",
+	[LKL_EROFS]		= "Read-only file system",
+	[LKL_EMLINK]		= "Too many links",
+	[LKL_EPIPE]		= "Broken pipe",
+	[LKL_EDOM]		= "Math argument out of domain of func",
+	[LKL_ERANGE]		= "Math result not representable",
+	[LKL_EDEADLK]		= "Resource deadlock would occur",
+	[LKL_ENAMETOOLONG]	= "File name too long",
+	[LKL_ENOLCK]		= "No record locks available",
+	[LKL_ENOSYS]		= "Invalid system call number",
+	[LKL_ENOTEMPTY]		= "Directory not empty",
+	[LKL_ELOOP]		= "Too many symbolic links encountered",
+	[LKL_ENOMSG]		= "No message of desired type",
+	[LKL_EIDRM]		= "Identifier removed",
+	[LKL_ECHRNG]		= "Channel number out of range",
+	[LKL_EL2NSYNC]		= "Level 2 not synchronized",
+	[LKL_EL3HLT]		= "Level 3 halted",
+	[LKL_EL3RST]		= "Level 3 reset",
+	[LKL_ELNRNG]		= "Link number out of range",
+	[LKL_EUNATCH]		= "Protocol driver not attached",
+	[LKL_ENOCSI]		= "No CSI structure available",
+	[LKL_EL2HLT]		= "Level 2 halted",
+	[LKL_EBADE]		= "Invalid exchange",
+	[LKL_EBADR]		= "Invalid request descriptor",
+	[LKL_EXFULL]		= "Exchange full",
+	[LKL_ENOANO]		= "No anode",
+	[LKL_EBADRQC]		= "Invalid request code",
+	[LKL_EBADSLT]		= "Invalid slot",
+	[LKL_EBFONT]		= "Bad font file format",
+	[LKL_ENOSTR]		= "Device not a stream",
+	[LKL_ENODATA]		= "No data available",
+	[LKL_ETIME]		= "Timer expired",
+	[LKL_ENOSR]		= "Out of streams resources",
+	[LKL_ENONET]		= "Machine is not on the network",
+	[LKL_ENOPKG]		= "Package not installed",
+	[LKL_EREMOTE]		= "Object is remote",
+	[LKL_ENOLINK]		= "Link has been severed",
+	[LKL_EADV]		= "Advertise error",
+	[LKL_ESRMNT]		= "Srmount error",
+	[LKL_ECOMM]		= "Communication error on send",
+	[LKL_EPROTO]		= "Protocol error",
+	[LKL_EMULTIHOP]		= "Multihop attempted",
+	[LKL_EDOTDOT]		= "RFS specific error",
+	[LKL_EBADMSG]		= "Not a data message",
+	[LKL_EOVERFLOW]		= "Value too large for defined data type",
+	[LKL_ENOTUNIQ]		= "Name not unique on network",
+	[LKL_EBADFD]		= "File descriptor in bad state",
+	[LKL_EREMCHG]		= "Remote address changed",
+	[LKL_ELIBACC]		= "Can not access a needed shared library",
+	[LKL_ELIBBAD]		= "Accessing a corrupted shared library",
+	[LKL_ELIBSCN]		= ".lib section in a.out corrupted",
+	[LKL_ELIBMAX]		= "Attempting to link in too many shared libraries",
+	[LKL_ELIBEXEC]		= "Cannot exec a shared library directly",
+	[LKL_EILSEQ]		= "Illegal byte sequence",
+	[LKL_ERESTART]		= "Interrupted system call should be restarted",
+	[LKL_ESTRPIPE]		= "Streams pipe error",
+	[LKL_EUSERS]		= "Too many users",
+	[LKL_ENOTSOCK]		= "Socket operation on non-socket",
+	[LKL_EDESTADDRREQ]	= "Destination address required",
+	[LKL_EMSGSIZE]		= "Message too long",
+	[LKL_EPROTOTYPE]	= "Protocol wrong type for socket",
+	[LKL_ENOPROTOOPT]	= "Protocol not available",
+	[LKL_EPROTONOSUPPORT]	= "Protocol not supported",
+	[LKL_ESOCKTNOSUPPORT]	= "Socket type not supported",
+	[LKL_EOPNOTSUPP]	= "Operation not supported on transport endpoint",
+	[LKL_EPFNOSUPPORT]	= "Protocol family not supported",
+	[LKL_EAFNOSUPPORT]	= "Address family not supported by protocol",
+	[LKL_EADDRINUSE]	= "Address already in use",
+	[LKL_EADDRNOTAVAIL]	= "Cannot assign requested address",
+	[LKL_ENETDOWN]		= "Network is down",
+	[LKL_ENETUNREACH]	= "Network is unreachable",
+	[LKL_ENETRESET]		= "Network dropped connection because of reset",
+	[LKL_ECONNABORTED]	= "Software caused connection abort",
+	[LKL_ECONNRESET]	= "Connection reset by peer",
+	[LKL_ENOBUFS]		= "No buffer space available",
+	[LKL_EISCONN]		= "Transport endpoint is already connected",
+	[LKL_ENOTCONN]		= "Transport endpoint is not connected",
+	[LKL_ESHUTDOWN]		= "Cannot send after transport endpoint shutdown",
+	[LKL_ETOOMANYREFS]	= "Too many references: cannot splice",
+	[LKL_ETIMEDOUT]		= "Connection timed out",
+	[LKL_ECONNREFUSED]	= "Connection refused",
+	[LKL_EHOSTDOWN]		= "Host is down",
+	[LKL_EHOSTUNREACH]	= "No route to host",
+	[LKL_EALREADY]		= "Operation already in progress",
+	[LKL_EINPROGRESS]	= "Operation now in progress",
+	[LKL_ESTALE]		= "Stale file handle",
+	[LKL_EUCLEAN]		= "Structure needs cleaning",
+	[LKL_ENOTNAM]		= "Not a XENIX named type file",
+	[LKL_ENAVAIL]		= "No XENIX semaphores available",
+	[LKL_EISNAM]		= "Is a named type file",
+	[LKL_EREMOTEIO]		= "Remote I/O error",
+	[LKL_EDQUOT]		= "Quota exceeded",
+	[LKL_ENOMEDIUM]		= "No medium found",
+	[LKL_EMEDIUMTYPE]	= "Wrong medium type",
+	[LKL_ECANCELED]		= "Operation Canceled",
+	[LKL_ENOKEY]		= "Required key not available",
+	[LKL_EKEYEXPIRED]	= "Key has expired",
+	[LKL_EKEYREVOKED]	= "Key has been revoked",
+	[LKL_EKEYREJECTED]	= "Key was rejected by service",
+	[LKL_EOWNERDEAD]	= "Owner died",
+	[LKL_ENOTRECOVERABLE]	= "State not recoverable",
+	[LKL_ERFKILL]		= "Operation not possible due to RF-kill",
+	[LKL_EHWPOISON]		= "Memory page has hardware error",
+};
+
+const char *lkl_strerror(int err)
+{
+	if (err < 0)
+		err = -err;
+
+	if ((size_t)err >= sizeof(lkl_err_strings) / sizeof(const char *))
+		return "Bad error code";
+
+	return lkl_err_strings[err];
+}
+
+void lkl_perror(char *msg, int err)
+{
+	const char *err_msg = lkl_strerror(err);
+	/* We need to use 'real' printf because lkl_print can
+	 * be turned off when debugging is off.
+	 */
+	lkl_printf("%s: %s\n", msg, err_msg);
+}
+
+static int lkl_vprintf(const char *fmt, va_list args)
+{
+	int n;
+	char *buffer;
+	va_list copy;
+
+	va_copy(copy, args);
+	n = vsnprintf(NULL, 0, fmt, copy);
+	va_end(copy);
+
+	buffer = lkl_mem_alloc(n + 1);
+	if (!buffer)
+		return (-1);
+
+	vsnprintf(buffer, n + 1, fmt, args);
+
+	lkl_print(buffer, n);
+	lkl_mem_free(buffer);
+
+	return n;
+}
+
+int lkl_printf(const char *fmt, ...)
+{
+	int n;
+	va_list args;
+
+	va_start(args, fmt);
+	n = lkl_vprintf(fmt, args);
+	va_end(args);
+
+	return n;
+}
+
+void lkl_bug(const char *fmt, ...)
+{
+	va_list args;
+
+	va_start(args, fmt);
+	lkl_vprintf(fmt, args);
+	va_end(args);
+
+	lkl_panic();
+}
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v8 16/20] um: host: add utilities functions
@ 2021-01-20  2:27             ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-01-20  2:27 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, thehajime, retrage01

Add basic utility functions for getting a string from a kernel error
code and a fprintf like function that uses the host print
operation. The latter is useful for informing the user about errors
that occur in the host library.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 tools/um/include/lkl.h |  23 +++++
 tools/um/lib/Build     |   3 +
 tools/um/lib/utils.c   | 205 +++++++++++++++++++++++++++++++++++++++++
 3 files changed, 231 insertions(+)
 create mode 100644 tools/um/lib/Build
 create mode 100644 tools/um/lib/utils.c

diff --git a/tools/um/include/lkl.h b/tools/um/include/lkl.h
index 2417ed5ccf71..97da2d11502f 100644
--- a/tools/um/include/lkl.h
+++ b/tools/um/include/lkl.h
@@ -27,6 +27,29 @@ extern "C" {
 #undef class
 #endif
 
+/**
+ * lkl_printf - print a message via the host print operation
+ *
+ * @fmt: printf like format string
+ */
+int lkl_printf(const char *fmt, ...);
+
+/**
+ * lkl_strerror - returns a string describing the given error code
+ *
+ * @err - error code
+ * @returns - string for the given error code
+ */
+const char *lkl_strerror(int err);
+
+/**
+ * lkl_perror - prints a string describing the given error code
+ *
+ * @msg - prefix for the error message
+ * @err - error code
+ */
+void lkl_perror(char *msg, int err);
+
 #if __LKL__BITS_PER_LONG == 64
 #define lkl_sys_fstatat lkl_sys_newfstatat
 #define lkl_sys_fstat lkl_sys_newfstat
diff --git a/tools/um/lib/Build b/tools/um/lib/Build
new file mode 100644
index 000000000000..77acd6f55e76
--- /dev/null
+++ b/tools/um/lib/Build
@@ -0,0 +1,3 @@
+include $(objtree)/include/config/auto.conf
+
+liblinux-$(CONFIG_UMMODE_LIB) += utils.o
diff --git a/tools/um/lib/utils.c b/tools/um/lib/utils.c
new file mode 100644
index 000000000000..3b38e1a95124
--- /dev/null
+++ b/tools/um/lib/utils.c
@@ -0,0 +1,205 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdarg.h>
+#include <stdio.h>
+#include <string.h>
+#include <lkl_host.h>
+
+static const char * const lkl_err_strings[] = {
+	[0]			= "Success",
+	[LKL_EPERM]		= "Operation not permitted",
+	[LKL_ENOENT]		= "No such file or directory",
+	[LKL_ESRCH]		= "No such process",
+	[LKL_EINTR]		= "Interrupted system call",
+	[LKL_EIO]		= "I/O error",
+	[LKL_ENXIO]		= "No such device or address",
+	[LKL_E2BIG]		= "Argument list too long",
+	[LKL_ENOEXEC]		= "Exec format error",
+	[LKL_EBADF]		= "Bad file number",
+	[LKL_ECHILD]		= "No child processes",
+	[LKL_EAGAIN]		= "Try again",
+	[LKL_ENOMEM]		= "Out of memory",
+	[LKL_EACCES]		= "Permission denied",
+	[LKL_EFAULT]		= "Bad address",
+	[LKL_ENOTBLK]		= "Block device required",
+	[LKL_EBUSY]		= "Device or resource busy",
+	[LKL_EEXIST]		= "File exists",
+	[LKL_EXDEV]		= "Cross-device link",
+	[LKL_ENODEV]		= "No such device",
+	[LKL_ENOTDIR]		= "Not a directory",
+	[LKL_EISDIR]		= "Is a directory",
+	[LKL_EINVAL]		= "Invalid argument",
+	[LKL_ENFILE]		= "File table overflow",
+	[LKL_EMFILE]		= "Too many open files",
+	[LKL_ENOTTY]		= "Not a typewriter",
+	[LKL_ETXTBSY]		= "Text file busy",
+	[LKL_EFBIG]		= "File too large",
+	[LKL_ENOSPC]		= "No space left on device",
+	[LKL_ESPIPE]		= "Illegal seek",
+	[LKL_EROFS]		= "Read-only file system",
+	[LKL_EMLINK]		= "Too many links",
+	[LKL_EPIPE]		= "Broken pipe",
+	[LKL_EDOM]		= "Math argument out of domain of func",
+	[LKL_ERANGE]		= "Math result not representable",
+	[LKL_EDEADLK]		= "Resource deadlock would occur",
+	[LKL_ENAMETOOLONG]	= "File name too long",
+	[LKL_ENOLCK]		= "No record locks available",
+	[LKL_ENOSYS]		= "Invalid system call number",
+	[LKL_ENOTEMPTY]		= "Directory not empty",
+	[LKL_ELOOP]		= "Too many symbolic links encountered",
+	[LKL_ENOMSG]		= "No message of desired type",
+	[LKL_EIDRM]		= "Identifier removed",
+	[LKL_ECHRNG]		= "Channel number out of range",
+	[LKL_EL2NSYNC]		= "Level 2 not synchronized",
+	[LKL_EL3HLT]		= "Level 3 halted",
+	[LKL_EL3RST]		= "Level 3 reset",
+	[LKL_ELNRNG]		= "Link number out of range",
+	[LKL_EUNATCH]		= "Protocol driver not attached",
+	[LKL_ENOCSI]		= "No CSI structure available",
+	[LKL_EL2HLT]		= "Level 2 halted",
+	[LKL_EBADE]		= "Invalid exchange",
+	[LKL_EBADR]		= "Invalid request descriptor",
+	[LKL_EXFULL]		= "Exchange full",
+	[LKL_ENOANO]		= "No anode",
+	[LKL_EBADRQC]		= "Invalid request code",
+	[LKL_EBADSLT]		= "Invalid slot",
+	[LKL_EBFONT]		= "Bad font file format",
+	[LKL_ENOSTR]		= "Device not a stream",
+	[LKL_ENODATA]		= "No data available",
+	[LKL_ETIME]		= "Timer expired",
+	[LKL_ENOSR]		= "Out of streams resources",
+	[LKL_ENONET]		= "Machine is not on the network",
+	[LKL_ENOPKG]		= "Package not installed",
+	[LKL_EREMOTE]		= "Object is remote",
+	[LKL_ENOLINK]		= "Link has been severed",
+	[LKL_EADV]		= "Advertise error",
+	[LKL_ESRMNT]		= "Srmount error",
+	[LKL_ECOMM]		= "Communication error on send",
+	[LKL_EPROTO]		= "Protocol error",
+	[LKL_EMULTIHOP]		= "Multihop attempted",
+	[LKL_EDOTDOT]		= "RFS specific error",
+	[LKL_EBADMSG]		= "Not a data message",
+	[LKL_EOVERFLOW]		= "Value too large for defined data type",
+	[LKL_ENOTUNIQ]		= "Name not unique on network",
+	[LKL_EBADFD]		= "File descriptor in bad state",
+	[LKL_EREMCHG]		= "Remote address changed",
+	[LKL_ELIBACC]		= "Can not access a needed shared library",
+	[LKL_ELIBBAD]		= "Accessing a corrupted shared library",
+	[LKL_ELIBSCN]		= ".lib section in a.out corrupted",
+	[LKL_ELIBMAX]		= "Attempting to link in too many shared libraries",
+	[LKL_ELIBEXEC]		= "Cannot exec a shared library directly",
+	[LKL_EILSEQ]		= "Illegal byte sequence",
+	[LKL_ERESTART]		= "Interrupted system call should be restarted",
+	[LKL_ESTRPIPE]		= "Streams pipe error",
+	[LKL_EUSERS]		= "Too many users",
+	[LKL_ENOTSOCK]		= "Socket operation on non-socket",
+	[LKL_EDESTADDRREQ]	= "Destination address required",
+	[LKL_EMSGSIZE]		= "Message too long",
+	[LKL_EPROTOTYPE]	= "Protocol wrong type for socket",
+	[LKL_ENOPROTOOPT]	= "Protocol not available",
+	[LKL_EPROTONOSUPPORT]	= "Protocol not supported",
+	[LKL_ESOCKTNOSUPPORT]	= "Socket type not supported",
+	[LKL_EOPNOTSUPP]	= "Operation not supported on transport endpoint",
+	[LKL_EPFNOSUPPORT]	= "Protocol family not supported",
+	[LKL_EAFNOSUPPORT]	= "Address family not supported by protocol",
+	[LKL_EADDRINUSE]	= "Address already in use",
+	[LKL_EADDRNOTAVAIL]	= "Cannot assign requested address",
+	[LKL_ENETDOWN]		= "Network is down",
+	[LKL_ENETUNREACH]	= "Network is unreachable",
+	[LKL_ENETRESET]		= "Network dropped connection because of reset",
+	[LKL_ECONNABORTED]	= "Software caused connection abort",
+	[LKL_ECONNRESET]	= "Connection reset by peer",
+	[LKL_ENOBUFS]		= "No buffer space available",
+	[LKL_EISCONN]		= "Transport endpoint is already connected",
+	[LKL_ENOTCONN]		= "Transport endpoint is not connected",
+	[LKL_ESHUTDOWN]		= "Cannot send after transport endpoint shutdown",
+	[LKL_ETOOMANYREFS]	= "Too many references: cannot splice",
+	[LKL_ETIMEDOUT]		= "Connection timed out",
+	[LKL_ECONNREFUSED]	= "Connection refused",
+	[LKL_EHOSTDOWN]		= "Host is down",
+	[LKL_EHOSTUNREACH]	= "No route to host",
+	[LKL_EALREADY]		= "Operation already in progress",
+	[LKL_EINPROGRESS]	= "Operation now in progress",
+	[LKL_ESTALE]		= "Stale file handle",
+	[LKL_EUCLEAN]		= "Structure needs cleaning",
+	[LKL_ENOTNAM]		= "Not a XENIX named type file",
+	[LKL_ENAVAIL]		= "No XENIX semaphores available",
+	[LKL_EISNAM]		= "Is a named type file",
+	[LKL_EREMOTEIO]		= "Remote I/O error",
+	[LKL_EDQUOT]		= "Quota exceeded",
+	[LKL_ENOMEDIUM]		= "No medium found",
+	[LKL_EMEDIUMTYPE]	= "Wrong medium type",
+	[LKL_ECANCELED]		= "Operation Canceled",
+	[LKL_ENOKEY]		= "Required key not available",
+	[LKL_EKEYEXPIRED]	= "Key has expired",
+	[LKL_EKEYREVOKED]	= "Key has been revoked",
+	[LKL_EKEYREJECTED]	= "Key was rejected by service",
+	[LKL_EOWNERDEAD]	= "Owner died",
+	[LKL_ENOTRECOVERABLE]	= "State not recoverable",
+	[LKL_ERFKILL]		= "Operation not possible due to RF-kill",
+	[LKL_EHWPOISON]		= "Memory page has hardware error",
+};
+
+const char *lkl_strerror(int err)
+{
+	if (err < 0)
+		err = -err;
+
+	if ((size_t)err >= sizeof(lkl_err_strings) / sizeof(const char *))
+		return "Bad error code";
+
+	return lkl_err_strings[err];
+}
+
+void lkl_perror(char *msg, int err)
+{
+	const char *err_msg = lkl_strerror(err);
+	/* We need to use 'real' printf because lkl_print can
+	 * be turned off when debugging is off.
+	 */
+	lkl_printf("%s: %s\n", msg, err_msg);
+}
+
+static int lkl_vprintf(const char *fmt, va_list args)
+{
+	int n;
+	char *buffer;
+	va_list copy;
+
+	va_copy(copy, args);
+	n = vsnprintf(NULL, 0, fmt, copy);
+	va_end(copy);
+
+	buffer = lkl_mem_alloc(n + 1);
+	if (!buffer)
+		return (-1);
+
+	vsnprintf(buffer, n + 1, fmt, args);
+
+	lkl_print(buffer, n);
+	lkl_mem_free(buffer);
+
+	return n;
+}
+
+int lkl_printf(const char *fmt, ...)
+{
+	int n;
+	va_list args;
+
+	va_start(args, fmt);
+	n = lkl_vprintf(fmt, args);
+	va_end(args);
+
+	return n;
+}
+
+void lkl_bug(const char *fmt, ...)
+{
+	va_list args;
+
+	va_start(args, fmt);
+	lkl_vprintf(fmt, args);
+	va_end(args);
+
+	lkl_panic();
+}
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v8 17/20] um: host: posix host operations
  2021-01-20  2:27           ` Hajime Tazaki
@ 2021-01-20  2:27             ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-01-20  2:27 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: thehajime, tavi.purdila, retrage01, linux-kernel-library, linux-arch

This commit implements LKL host operations for POSIX hosts.  The
operations are implemented to be portable in various POSIX OSs.

Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 tools/um/lib/Build        |   3 +
 tools/um/lib/jmp_buf.c    |  14 ++
 tools/um/lib/posix-host.c | 272 ++++++++++++++++++++++++++++++++++++++
 3 files changed, 289 insertions(+)
 create mode 100644 tools/um/lib/jmp_buf.c
 create mode 100644 tools/um/lib/posix-host.c

diff --git a/tools/um/lib/Build b/tools/um/lib/Build
index 77acd6f55e76..dddff26a3b4e 100644
--- a/tools/um/lib/Build
+++ b/tools/um/lib/Build
@@ -1,3 +1,6 @@
 include $(objtree)/include/config/auto.conf
+CFLAGS_posix-host.o += -D_FILE_OFFSET_BITS=64
 
 liblinux-$(CONFIG_UMMODE_LIB) += utils.o
+liblinux-$(CONFIG_UMMODE_LIB) += posix-host.o
+liblinux-$(CONFIG_UMMODE_LIB) += jmp_buf.o
diff --git a/tools/um/lib/jmp_buf.c b/tools/um/lib/jmp_buf.c
new file mode 100644
index 000000000000..fcb15bbaa38a
--- /dev/null
+++ b/tools/um/lib/jmp_buf.c
@@ -0,0 +1,14 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <setjmp.h>
+#include <lkl_host.h>
+
+void lkl_jmp_buf_set(struct lkl_jmp_buf *jmpb, void (*f)(void))
+{
+	if (!setjmp(*((jmp_buf *)jmpb->buf)))
+		f();
+}
+
+void lkl_jmp_buf_longjmp(struct lkl_jmp_buf *jmpb, int val)
+{
+	longjmp(*((jmp_buf *)jmpb->buf), val);
+}
diff --git a/tools/um/lib/posix-host.c b/tools/um/lib/posix-host.c
new file mode 100644
index 000000000000..f7c170fd3fd9
--- /dev/null
+++ b/tools/um/lib/posix-host.c
@@ -0,0 +1,272 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <pthread.h>
+#include <limits.h>
+#include <errno.h>
+#include <string.h>
+#include <stdlib.h>
+#include <assert.h>
+#include <sys/syscall.h>
+#include <lkl_host.h>
+
+/* Let's see if the host has semaphore.h */
+#include <unistd.h>
+
+#ifdef _POSIX_SEMAPHORES
+#include <semaphore.h>
+/* TODO(pscollins): We don't support fork() for now, but maybe one day
+ * we will?
+ */
+#define SHARE_SEM 0
+#endif /* _POSIX_SEMAPHORES */
+
+
+struct lkl_mutex {
+	pthread_mutex_t mutex;
+};
+
+struct lkl_sem {
+#ifdef _POSIX_SEMAPHORES
+	sem_t sem;
+#else
+	pthread_mutex_t lock;
+	int count;
+	pthread_cond_t cond;
+#endif /* _POSIX_SEMAPHORES */
+};
+
+struct lkl_tls_key {
+	pthread_key_t key;
+};
+
+void *lkl_mem_alloc(unsigned long mem)
+{
+	return malloc(mem);
+}
+
+void lkl_mem_free(void *mem)
+{
+	return free(mem);
+}
+
+void __attribute__((weak)) lkl_print(const char *str, int len)
+{
+	int ret __attribute__((unused));
+
+	ret = write(STDOUT_FILENO, str, len);
+}
+
+void lkl_panic(void)
+{
+	assert(0);
+}
+
+#define WARN_UNLESS(exp) do {						\
+		if (exp < 0)						\
+			lkl_printf("%s: %s\n", #exp, strerror(errno));	\
+	} while (0)
+
+static int _warn_pthread(int ret, char *str_exp)
+{
+	if (ret > 0)
+		lkl_printf("%s: %s\n", str_exp, strerror(ret));
+
+	return ret;
+}
+
+
+/* pthread_* functions use the reverse convention */
+#define WARN_PTHREAD(exp) _warn_pthread(exp, #exp)
+
+struct lkl_sem *lkl_sem_alloc(int count)
+{
+	struct lkl_sem *sem;
+
+	sem = malloc(sizeof(*sem));
+	if (!sem)
+		return NULL;
+
+#ifdef _POSIX_SEMAPHORES
+	if (sem_init(&sem->sem, SHARE_SEM, count) < 0) {
+		lkl_printf("sem_init: %s\n", strerror(errno));
+		free(sem);
+		return NULL;
+	}
+#else
+	pthread_mutex_init(&sem->lock, NULL);
+	sem->count = count;
+	WARN_PTHREAD(pthread_cond_init(&sem->cond, NULL));
+#endif /* _POSIX_SEMAPHORES */
+
+	return sem;
+}
+
+void lkl_sem_free(struct lkl_sem *sem)
+{
+#ifdef _POSIX_SEMAPHORES
+	WARN_UNLESS(sem_destroy(&sem->sem));
+#else
+	WARN_PTHREAD(pthread_cond_destroy(&sem->cond));
+	WARN_PTHREAD(pthread_mutex_destroy(&sem->lock));
+#endif /* _POSIX_SEMAPHORES */
+	free(sem);
+}
+
+void lkl_sem_up(struct lkl_sem *sem)
+{
+#ifdef _POSIX_SEMAPHORES
+	WARN_UNLESS(sem_post(&sem->sem));
+#else
+	WARN_PTHREAD(pthread_mutex_lock(&sem->lock));
+	sem->count++;
+	if (sem->count > 0)
+		WARN_PTHREAD(pthread_cond_signal(&sem->cond));
+	WARN_PTHREAD(pthread_mutex_unlock(&sem->lock));
+#endif /* _POSIX_SEMAPHORES */
+
+}
+
+void lkl_sem_down(struct lkl_sem *sem)
+{
+#ifdef _POSIX_SEMAPHORES
+	int err;
+
+	do {
+		err = sem_wait(&sem->sem);
+	} while (err < 0 && errno == EINTR);
+	if (err < 0 && errno != EINTR)
+		lkl_printf("sem_wait: %s\n", strerror(errno));
+#else
+	WARN_PTHREAD(pthread_mutex_lock(&sem->lock));
+	while (sem->count <= 0)
+		WARN_PTHREAD(pthread_cond_wait(&sem->cond, &sem->lock));
+	sem->count--;
+	WARN_PTHREAD(pthread_mutex_unlock(&sem->lock));
+#endif /* _POSIX_SEMAPHORES */
+}
+
+struct lkl_mutex *lkl_mutex_alloc(int recursive)
+{
+	struct lkl_mutex *_mutex = malloc(sizeof(struct lkl_mutex));
+	pthread_mutex_t *mutex = NULL;
+	pthread_mutexattr_t attr;
+
+	if (!_mutex)
+		return NULL;
+
+	mutex = &_mutex->mutex;
+	WARN_PTHREAD(pthread_mutexattr_init(&attr));
+
+	/* PTHREAD_MUTEX_ERRORCHECK is *very* useful for debugging,
+	 * but has some overhead, so we provide an option to turn it
+	 * off.
+	 */
+#ifdef DEBUG
+	if (!recursive)
+		WARN_PTHREAD(pthread_mutexattr_settype(
+				     &attr, PTHREAD_MUTEX_ERRORCHECK));
+#endif /* DEBUG */
+
+	if (recursive)
+		WARN_PTHREAD(pthread_mutexattr_settype(
+				     &attr, PTHREAD_MUTEX_RECURSIVE));
+
+	WARN_PTHREAD(pthread_mutex_init(mutex, &attr));
+
+	return _mutex;
+}
+
+void lkl_mutex_lock(struct lkl_mutex *mutex)
+{
+	WARN_PTHREAD(pthread_mutex_lock(&mutex->mutex));
+}
+
+void lkl_mutex_unlock(struct lkl_mutex *_mutex)
+{
+	pthread_mutex_t *mutex = &_mutex->mutex;
+
+	WARN_PTHREAD(pthread_mutex_unlock(mutex));
+}
+
+void lkl_mutex_free(struct lkl_mutex *_mutex)
+{
+	pthread_mutex_t *mutex = &_mutex->mutex;
+
+	WARN_PTHREAD(pthread_mutex_destroy(mutex));
+	free(_mutex);
+}
+
+lkl_thread_t lkl_thread_create(void* (*fn)(void *), void *arg)
+{
+	pthread_t thread;
+
+	if (WARN_PTHREAD(pthread_create(&thread, NULL, fn, arg)))
+		return 0;
+	else
+		return (lkl_thread_t) thread;
+}
+
+void lkl_thread_detach(void)
+{
+	WARN_PTHREAD(pthread_detach(pthread_self()));
+}
+
+void lkl_thread_exit(void)
+{
+	pthread_exit(NULL);
+}
+
+int lkl_thread_join(lkl_thread_t tid)
+{
+	if (WARN_PTHREAD(pthread_join((pthread_t)tid, NULL)))
+		return (-1);
+	else
+		return 0;
+}
+
+lkl_thread_t lkl_thread_self(void)
+{
+	return (lkl_thread_t)pthread_self();
+}
+
+int lkl_thread_equal(lkl_thread_t a, lkl_thread_t b)
+{
+	return pthread_equal((pthread_t)a, (pthread_t)b);
+}
+
+long lkl_gettid(void)
+{
+	return syscall(SYS_gettid);
+}
+
+struct lkl_tls_key *lkl_tls_alloc(void (*destructor)(void *))
+{
+	struct lkl_tls_key *ret = malloc(sizeof(struct lkl_tls_key));
+
+	if (WARN_PTHREAD(pthread_key_create(&ret->key, destructor))) {
+		free(ret);
+		return NULL;
+	}
+	return ret;
+}
+
+void lkl_tls_free(struct lkl_tls_key *key)
+{
+	WARN_PTHREAD(pthread_key_delete(key->key));
+	free(key);
+}
+
+int lkl_tls_set(struct lkl_tls_key *key, void *data)
+{
+	if (WARN_PTHREAD(pthread_setspecific(key->key, data)))
+		return (-1);
+	return 0;
+}
+
+void *lkl_tls_get(struct lkl_tls_key *key)
+{
+	return pthread_getspecific(key->key);
+}
+
+struct lkl_host_operations lkl_host_ops = {
+};
+
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v8 17/20] um: host: posix host operations
@ 2021-01-20  2:27             ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-01-20  2:27 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, thehajime, retrage01

This commit implements LKL host operations for POSIX hosts.  The
operations are implemented to be portable in various POSIX OSs.

Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 tools/um/lib/Build        |   3 +
 tools/um/lib/jmp_buf.c    |  14 ++
 tools/um/lib/posix-host.c | 272 ++++++++++++++++++++++++++++++++++++++
 3 files changed, 289 insertions(+)
 create mode 100644 tools/um/lib/jmp_buf.c
 create mode 100644 tools/um/lib/posix-host.c

diff --git a/tools/um/lib/Build b/tools/um/lib/Build
index 77acd6f55e76..dddff26a3b4e 100644
--- a/tools/um/lib/Build
+++ b/tools/um/lib/Build
@@ -1,3 +1,6 @@
 include $(objtree)/include/config/auto.conf
+CFLAGS_posix-host.o += -D_FILE_OFFSET_BITS=64
 
 liblinux-$(CONFIG_UMMODE_LIB) += utils.o
+liblinux-$(CONFIG_UMMODE_LIB) += posix-host.o
+liblinux-$(CONFIG_UMMODE_LIB) += jmp_buf.o
diff --git a/tools/um/lib/jmp_buf.c b/tools/um/lib/jmp_buf.c
new file mode 100644
index 000000000000..fcb15bbaa38a
--- /dev/null
+++ b/tools/um/lib/jmp_buf.c
@@ -0,0 +1,14 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <setjmp.h>
+#include <lkl_host.h>
+
+void lkl_jmp_buf_set(struct lkl_jmp_buf *jmpb, void (*f)(void))
+{
+	if (!setjmp(*((jmp_buf *)jmpb->buf)))
+		f();
+}
+
+void lkl_jmp_buf_longjmp(struct lkl_jmp_buf *jmpb, int val)
+{
+	longjmp(*((jmp_buf *)jmpb->buf), val);
+}
diff --git a/tools/um/lib/posix-host.c b/tools/um/lib/posix-host.c
new file mode 100644
index 000000000000..f7c170fd3fd9
--- /dev/null
+++ b/tools/um/lib/posix-host.c
@@ -0,0 +1,272 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <pthread.h>
+#include <limits.h>
+#include <errno.h>
+#include <string.h>
+#include <stdlib.h>
+#include <assert.h>
+#include <sys/syscall.h>
+#include <lkl_host.h>
+
+/* Let's see if the host has semaphore.h */
+#include <unistd.h>
+
+#ifdef _POSIX_SEMAPHORES
+#include <semaphore.h>
+/* TODO(pscollins): We don't support fork() for now, but maybe one day
+ * we will?
+ */
+#define SHARE_SEM 0
+#endif /* _POSIX_SEMAPHORES */
+
+
+struct lkl_mutex {
+	pthread_mutex_t mutex;
+};
+
+struct lkl_sem {
+#ifdef _POSIX_SEMAPHORES
+	sem_t sem;
+#else
+	pthread_mutex_t lock;
+	int count;
+	pthread_cond_t cond;
+#endif /* _POSIX_SEMAPHORES */
+};
+
+struct lkl_tls_key {
+	pthread_key_t key;
+};
+
+void *lkl_mem_alloc(unsigned long mem)
+{
+	return malloc(mem);
+}
+
+void lkl_mem_free(void *mem)
+{
+	return free(mem);
+}
+
+void __attribute__((weak)) lkl_print(const char *str, int len)
+{
+	int ret __attribute__((unused));
+
+	ret = write(STDOUT_FILENO, str, len);
+}
+
+void lkl_panic(void)
+{
+	assert(0);
+}
+
+#define WARN_UNLESS(exp) do {						\
+		if (exp < 0)						\
+			lkl_printf("%s: %s\n", #exp, strerror(errno));	\
+	} while (0)
+
+static int _warn_pthread(int ret, char *str_exp)
+{
+	if (ret > 0)
+		lkl_printf("%s: %s\n", str_exp, strerror(ret));
+
+	return ret;
+}
+
+
+/* pthread_* functions use the reverse convention */
+#define WARN_PTHREAD(exp) _warn_pthread(exp, #exp)
+
+struct lkl_sem *lkl_sem_alloc(int count)
+{
+	struct lkl_sem *sem;
+
+	sem = malloc(sizeof(*sem));
+	if (!sem)
+		return NULL;
+
+#ifdef _POSIX_SEMAPHORES
+	if (sem_init(&sem->sem, SHARE_SEM, count) < 0) {
+		lkl_printf("sem_init: %s\n", strerror(errno));
+		free(sem);
+		return NULL;
+	}
+#else
+	pthread_mutex_init(&sem->lock, NULL);
+	sem->count = count;
+	WARN_PTHREAD(pthread_cond_init(&sem->cond, NULL));
+#endif /* _POSIX_SEMAPHORES */
+
+	return sem;
+}
+
+void lkl_sem_free(struct lkl_sem *sem)
+{
+#ifdef _POSIX_SEMAPHORES
+	WARN_UNLESS(sem_destroy(&sem->sem));
+#else
+	WARN_PTHREAD(pthread_cond_destroy(&sem->cond));
+	WARN_PTHREAD(pthread_mutex_destroy(&sem->lock));
+#endif /* _POSIX_SEMAPHORES */
+	free(sem);
+}
+
+void lkl_sem_up(struct lkl_sem *sem)
+{
+#ifdef _POSIX_SEMAPHORES
+	WARN_UNLESS(sem_post(&sem->sem));
+#else
+	WARN_PTHREAD(pthread_mutex_lock(&sem->lock));
+	sem->count++;
+	if (sem->count > 0)
+		WARN_PTHREAD(pthread_cond_signal(&sem->cond));
+	WARN_PTHREAD(pthread_mutex_unlock(&sem->lock));
+#endif /* _POSIX_SEMAPHORES */
+
+}
+
+void lkl_sem_down(struct lkl_sem *sem)
+{
+#ifdef _POSIX_SEMAPHORES
+	int err;
+
+	do {
+		err = sem_wait(&sem->sem);
+	} while (err < 0 && errno == EINTR);
+	if (err < 0 && errno != EINTR)
+		lkl_printf("sem_wait: %s\n", strerror(errno));
+#else
+	WARN_PTHREAD(pthread_mutex_lock(&sem->lock));
+	while (sem->count <= 0)
+		WARN_PTHREAD(pthread_cond_wait(&sem->cond, &sem->lock));
+	sem->count--;
+	WARN_PTHREAD(pthread_mutex_unlock(&sem->lock));
+#endif /* _POSIX_SEMAPHORES */
+}
+
+struct lkl_mutex *lkl_mutex_alloc(int recursive)
+{
+	struct lkl_mutex *_mutex = malloc(sizeof(struct lkl_mutex));
+	pthread_mutex_t *mutex = NULL;
+	pthread_mutexattr_t attr;
+
+	if (!_mutex)
+		return NULL;
+
+	mutex = &_mutex->mutex;
+	WARN_PTHREAD(pthread_mutexattr_init(&attr));
+
+	/* PTHREAD_MUTEX_ERRORCHECK is *very* useful for debugging,
+	 * but has some overhead, so we provide an option to turn it
+	 * off.
+	 */
+#ifdef DEBUG
+	if (!recursive)
+		WARN_PTHREAD(pthread_mutexattr_settype(
+				     &attr, PTHREAD_MUTEX_ERRORCHECK));
+#endif /* DEBUG */
+
+	if (recursive)
+		WARN_PTHREAD(pthread_mutexattr_settype(
+				     &attr, PTHREAD_MUTEX_RECURSIVE));
+
+	WARN_PTHREAD(pthread_mutex_init(mutex, &attr));
+
+	return _mutex;
+}
+
+void lkl_mutex_lock(struct lkl_mutex *mutex)
+{
+	WARN_PTHREAD(pthread_mutex_lock(&mutex->mutex));
+}
+
+void lkl_mutex_unlock(struct lkl_mutex *_mutex)
+{
+	pthread_mutex_t *mutex = &_mutex->mutex;
+
+	WARN_PTHREAD(pthread_mutex_unlock(mutex));
+}
+
+void lkl_mutex_free(struct lkl_mutex *_mutex)
+{
+	pthread_mutex_t *mutex = &_mutex->mutex;
+
+	WARN_PTHREAD(pthread_mutex_destroy(mutex));
+	free(_mutex);
+}
+
+lkl_thread_t lkl_thread_create(void* (*fn)(void *), void *arg)
+{
+	pthread_t thread;
+
+	if (WARN_PTHREAD(pthread_create(&thread, NULL, fn, arg)))
+		return 0;
+	else
+		return (lkl_thread_t) thread;
+}
+
+void lkl_thread_detach(void)
+{
+	WARN_PTHREAD(pthread_detach(pthread_self()));
+}
+
+void lkl_thread_exit(void)
+{
+	pthread_exit(NULL);
+}
+
+int lkl_thread_join(lkl_thread_t tid)
+{
+	if (WARN_PTHREAD(pthread_join((pthread_t)tid, NULL)))
+		return (-1);
+	else
+		return 0;
+}
+
+lkl_thread_t lkl_thread_self(void)
+{
+	return (lkl_thread_t)pthread_self();
+}
+
+int lkl_thread_equal(lkl_thread_t a, lkl_thread_t b)
+{
+	return pthread_equal((pthread_t)a, (pthread_t)b);
+}
+
+long lkl_gettid(void)
+{
+	return syscall(SYS_gettid);
+}
+
+struct lkl_tls_key *lkl_tls_alloc(void (*destructor)(void *))
+{
+	struct lkl_tls_key *ret = malloc(sizeof(struct lkl_tls_key));
+
+	if (WARN_PTHREAD(pthread_key_create(&ret->key, destructor))) {
+		free(ret);
+		return NULL;
+	}
+	return ret;
+}
+
+void lkl_tls_free(struct lkl_tls_key *key)
+{
+	WARN_PTHREAD(pthread_key_delete(key->key));
+	free(key);
+}
+
+int lkl_tls_set(struct lkl_tls_key *key, void *data)
+{
+	if (WARN_PTHREAD(pthread_setspecific(key->key, data)))
+		return (-1);
+	return 0;
+}
+
+void *lkl_tls_get(struct lkl_tls_key *key)
+{
+	return pthread_getspecific(key->key);
+}
+
+struct lkl_host_operations lkl_host_ops = {
+};
+
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v8 18/20] selftests/um: lkl: add test programs for library mode of UML
  2021-01-20  2:27           ` Hajime Tazaki
@ 2021-01-20  2:27             ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-01-20  2:27 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: thehajime, tavi.purdila, retrage01, linux-kernel-library,
	linux-arch, Shuah Khan, linux-kselftest

Add a simple LKL test application (boot) that starts the kernel and
performs simple tests that minimally exercise the LKL API.  The tests
are implemented on kselftest framework, which can be invoked with

 make ARCH=um SUBARCH=lkl TARGETS="um" kselftest

Additionally, this commits add a skip for headers_install for ARCH=um
since UML (even with library mode) doesn't have headers install.

```
 cat /tmp/um-test.tap | sed "s/^# selftests/     \# Subtest: selftests/"
|  sed "s/^#/    /" | sed "s/version 13/version 14/" | sed "s/\(ok
[0-9]* \)/\1- /" | ./tools/testing/kunit/kunit.py parse
```

this makes tap14 converter for the kselftest results.

Cc: Shuah Khan <shuah@kernel.org>
Cc: linux-kselftest@vger.kernel.org
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/testing/selftests/Makefile    |   3 +
 tools/testing/selftests/um/Makefile |  13 +
 tools/testing/selftests/um/boot.c   | 376 ++++++++++++++++++++++++++++
 tools/testing/selftests/um/test.c   | 128 ++++++++++
 tools/testing/selftests/um/test.h   |  72 ++++++
 tools/testing/selftests/um/test.sh  | 181 +++++++++++++
 6 files changed, 773 insertions(+)
 create mode 100644 tools/testing/selftests/um/Makefile
 create mode 100644 tools/testing/selftests/um/boot.c
 create mode 100644 tools/testing/selftests/um/test.c
 create mode 100644 tools/testing/selftests/um/test.h
 create mode 100644 tools/testing/selftests/um/test.sh

diff --git a/tools/testing/selftests/Makefile b/tools/testing/selftests/Makefile
index 8a917cb4426a..26b6fc800778 100644
--- a/tools/testing/selftests/Makefile
+++ b/tools/testing/selftests/Makefile
@@ -65,6 +65,7 @@ TARGETS += timers
 endif
 TARGETS += tmpfs
 TARGETS += tpm2
+TARGETS += um
 TARGETS += user
 TARGETS += vDSO
 TARGETS += vm
@@ -154,12 +155,14 @@ endif
 # Local build cases: "make kselftest", "make -C" - headers are installed
 # in the default INSTALL_HDR_PATH usr/include.
 khdr:
+ifneq (um,$(ARCH))
 ifeq (1,$(DEFAULT_INSTALL_HDR_PATH))
 	$(MAKE) --no-builtin-rules ARCH=$(ARCH) -C $(top_srcdir) headers_install
 else
 	$(MAKE) --no-builtin-rules INSTALL_HDR_PATH=$$BUILD/usr \
 		ARCH=$(ARCH) -C $(top_srcdir) headers_install
 endif
+endif
 
 all: khdr
 	@ret=1;							\
diff --git a/tools/testing/selftests/um/Makefile b/tools/testing/selftests/um/Makefile
new file mode 100644
index 000000000000..1b915f4bb751
--- /dev/null
+++ b/tools/testing/selftests/um/Makefile
@@ -0,0 +1,13 @@
+# SPDX-License-Identifier: GPL-2.0-only
+
+top_srcdir = ../../../../
+top_objdir = $(OUTPUT)/../../
+
+CFLAGS := -fPIC -I$(top_objdir)/tools/um/include/ -I$(top_srcdir)/tools/um/include/ -g
+CFLAGS += test.c
+LDFLAGS := -pie
+LDLIBS := -L$(top_objdir)/tools/um -L$(top_srcdir)/tools/um -llinux -lpthread -lrt -lutil
+
+TEST_GEN_PROGS := boot
+
+include ../lib.mk
diff --git a/tools/testing/selftests/um/boot.c b/tools/testing/selftests/um/boot.c
new file mode 100644
index 000000000000..b1330a39a936
--- /dev/null
+++ b/tools/testing/selftests/um/boot.c
@@ -0,0 +1,376 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <unistd.h>
+#include <string.h>
+#include <time.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <lkl.h>
+#include <lkl_host.h>
+
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <sys/epoll.h>
+#include <sys/ioctl.h>
+
+#include "test.h"
+
+#define sleep_ns 87654321
+static int lkl_test_nanosleep(void)
+{
+	struct lkl_timespec ts = {
+		.tv_sec = 0,
+		.tv_nsec = sleep_ns,
+	};
+	struct timespec start, stop;
+	long delta;
+	long ret;
+
+	clock_gettime(CLOCK_MONOTONIC, &start);
+	ret = lkl_sys_nanosleep((struct __lkl__kernel_timespec *)&ts, NULL);
+	clock_gettime(CLOCK_MONOTONIC, &stop);
+
+	delta = 1e9*(stop.tv_sec - start.tv_sec) +
+		(stop.tv_nsec - start.tv_nsec);
+
+	lkl_test_logf("sleep %ld, expected sleep %d\n", delta, sleep_ns);
+
+	if (ret == 0 && delta > sleep_ns * 0.9)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+LKL_TEST_CALL(getpid, lkl_sys_getpid, 1)
+
+void check_latency(long (*f)(void), long *min, long *max, long *avg)
+{
+	int i;
+	struct timespec start, stop;
+	unsigned long long sum = 0;
+	static const int count = 20;
+	long delta;
+
+	*min = 1000000000;
+	*max = -1;
+
+	for (i = 0; i < count; i++) {
+		clock_gettime(CLOCK_MONOTONIC, &start);
+		f();
+		clock_gettime(CLOCK_MONOTONIC, &stop);
+
+		delta = 1e9*(stop.tv_sec - start.tv_sec) +
+			(stop.tv_nsec - start.tv_nsec);
+
+		if (*min > delta)
+			*min = delta;
+		if (*max < delta)
+			*max = delta;
+		sum += delta;
+	}
+	*avg = sum / count;
+}
+
+static long native_getpid(void)
+{
+	getpid();
+	return 0;
+}
+
+int lkl_test_syscall_latency(void)
+{
+	long min, max, avg;
+
+	lkl_test_logf("avg/min/max: ");
+
+	check_latency(lkl_sys_getpid, &min, &max, &avg);
+
+	lkl_test_logf("lkl:%ld/%ld/%ld ", avg, min, max);
+
+	check_latency(native_getpid, &min, &max, &avg);
+
+	lkl_test_logf("native:%ld/%ld/%ld\n", avg, min, max);
+
+	return TEST_SUCCESS;
+}
+
+#define access_rights 0721
+
+LKL_TEST_CALL(creat, lkl_sys_creat, 3, "/file", access_rights)
+LKL_TEST_CALL(close, lkl_sys_close, 0, 0);
+LKL_TEST_CALL(failopen, lkl_sys_open, -LKL_ENOENT, "/file2", 0, 0);
+LKL_TEST_CALL(umask, lkl_sys_umask, 022,  0777);
+LKL_TEST_CALL(umask2, lkl_sys_umask, 0777, 0);
+LKL_TEST_CALL(open, lkl_sys_open, 0, "/file", LKL_O_RDWR, 0);
+static const char wrbuf[] = "test";
+LKL_TEST_CALL(write, lkl_sys_write, sizeof(wrbuf), 0, wrbuf, sizeof(wrbuf));
+LKL_TEST_CALL(lseek_cur, lkl_sys_lseek, sizeof(wrbuf), 0, 0, LKL_SEEK_CUR);
+LKL_TEST_CALL(lseek_end, lkl_sys_lseek, sizeof(wrbuf), 0, 0, LKL_SEEK_END);
+LKL_TEST_CALL(lseek_set, lkl_sys_lseek, 0, 0, 0, LKL_SEEK_SET);
+
+int lkl_test_read(void)
+{
+	char buf[10] = { 0, };
+	long ret;
+
+	ret = lkl_sys_read(0, buf, sizeof(buf));
+
+	lkl_test_logf("lkl_sys_read=%ld buf=%s\n", ret, buf);
+
+	if (ret == sizeof(wrbuf) && !strcmp(wrbuf, buf))
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+int lkl_test_fstat(void)
+{
+	struct lkl_stat stat;
+	long ret;
+
+	ret = lkl_sys_fstat(0, (void *)&stat);
+
+	lkl_test_logf("lkl_sys_fstat=%ld mode=%o size=%ld\n", ret, stat.st_mode,
+		      stat.st_size);
+
+	if (ret == 0 && stat.st_size == sizeof(wrbuf) &&
+	    stat.st_mode == (access_rights | LKL_S_IFREG))
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+LKL_TEST_CALL(mkdir, lkl_sys_mkdir, 0, "/proc", access_rights)
+
+int lkl_test_stat(void)
+{
+	struct lkl_stat stat;
+	long ret;
+
+	ret = lkl_sys_stat("/proc", (void *)&stat);
+
+	lkl_test_logf("lkl_sys_stat(\"/proc\")=%ld mode=%o\n", ret,
+		      stat.st_mode);
+
+	if (ret == 0 && stat.st_mode == (access_rights | LKL_S_IFDIR))
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+static int lkl_test_pipe2(void)
+{
+	int pipe_fds[2];
+	int READ_IDX = 0, WRITE_IDX = 1;
+	static const char msg[] = "Hello world!";
+	char str[20];
+	int msg_len_bytes = strlen(msg) + 1;
+	int cmp_res;
+	long ret;
+
+	ret = lkl_sys_pipe2(pipe_fds, LKL_O_NONBLOCK);
+	if (ret) {
+		lkl_test_logf("pipe2: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_write(pipe_fds[WRITE_IDX], msg, msg_len_bytes);
+	if (ret != msg_len_bytes) {
+		if (ret < 0)
+			lkl_test_logf("write error: %s\n", lkl_strerror(ret));
+		else
+			lkl_test_logf("short write: %ld\n", ret);
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_read(pipe_fds[READ_IDX], str, msg_len_bytes);
+	if (ret != msg_len_bytes) {
+		if (ret < 0)
+			lkl_test_logf("read error: %s\n", lkl_strerror(ret));
+		else
+			lkl_test_logf("short read: %ld\n", ret);
+		return TEST_FAILURE;
+	}
+
+	cmp_res = memcmp(msg, str, msg_len_bytes);
+	if (cmp_res) {
+		lkl_test_logf("memcmp failed: %d\n", cmp_res);
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_close(pipe_fds[0]);
+	if (ret) {
+		lkl_test_logf("close error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_close(pipe_fds[1]);
+	if (ret) {
+		lkl_test_logf("close error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	return TEST_SUCCESS;
+}
+
+static int lkl_test_epoll(void)
+{
+	int epoll_fd, pipe_fds[2];
+	int READ_IDX = 0, WRITE_IDX = 1;
+	struct lkl_epoll_event wait_on, read_result;
+	static const char msg[] = "Hello world!";
+	long ret;
+
+	memset(&wait_on, 0, sizeof(wait_on));
+	memset(&read_result, 0, sizeof(read_result));
+
+	ret = lkl_sys_pipe2(pipe_fds, LKL_O_NONBLOCK);
+	if (ret) {
+		lkl_test_logf("pipe2 error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	epoll_fd = lkl_sys_epoll_create(1);
+	if (epoll_fd < 0) {
+		lkl_test_logf("epoll_create error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	wait_on.events = LKL_POLLIN | LKL_POLLOUT;
+	wait_on.data = pipe_fds[READ_IDX];
+
+	ret = lkl_sys_epoll_ctl(epoll_fd, LKL_EPOLL_CTL_ADD, pipe_fds[READ_IDX],
+				&wait_on);
+	if (ret < 0) {
+		lkl_test_logf("epoll_ctl error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	/* Shouldn't be ready before we have written something */
+	ret = lkl_sys_epoll_wait(epoll_fd, &read_result, 1, 0);
+	if (ret != 0) {
+		if (ret < 0)
+			lkl_test_logf("epoll_wait error: %s\n",
+				      lkl_strerror(ret));
+		else
+			lkl_test_logf("epoll_wait: bad event: 0x%lx\n", ret);
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_write(pipe_fds[WRITE_IDX], msg, strlen(msg) + 1);
+	if (ret < 0) {
+		lkl_test_logf("write error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	/* We expect exactly 1 fd to be ready immediately */
+	ret = lkl_sys_epoll_wait(epoll_fd, &read_result, 1, 0);
+	if (ret != 1) {
+		if (ret < 0)
+			lkl_test_logf("epoll_wait error: %s\n",
+				      lkl_strerror(ret));
+		else
+			lkl_test_logf("epoll_wait: bad ev no %ld\n", ret);
+		return TEST_FAILURE;
+	}
+
+	/* Already tested reading from pipe2 so no need to do it
+	 * here
+	 */
+
+	return TEST_SUCCESS;
+}
+
+LKL_TEST_CALL(chdir_proc, lkl_sys_chdir, 0, "proc");
+
+static int dir_fd;
+
+static int lkl_test_open_cwd(void)
+{
+	dir_fd = lkl_sys_open(".", LKL_O_RDONLY | LKL_O_DIRECTORY, 0);
+	if (dir_fd < 0) {
+		lkl_test_logf("failed to open current directory: %s\n",
+			      lkl_strerror(dir_fd));
+		return TEST_FAILURE;
+	}
+
+	return TEST_SUCCESS;
+}
+
+/* column where to insert a line break for the list file tests below. */
+#define COL_LINE_BREAK 70
+
+static int lkl_test_getdents64(void)
+{
+	long ret;
+	char buf[1024], *pos;
+	struct lkl_linux_dirent64 *de;
+	int wr;
+
+	de = (struct lkl_linux_dirent64 *)buf;
+	ret = lkl_sys_getdents64(dir_fd, de, sizeof(buf));
+
+	wr = lkl_test_logf("%d ", dir_fd);
+
+	if (ret < 0)
+		return TEST_FAILURE;
+
+	for (pos = buf; pos - buf < ret; pos += de->d_reclen) {
+		de = (struct lkl_linux_dirent64 *)pos;
+
+		wr += lkl_test_logf("%s ", de->d_name);
+		if (wr >= COL_LINE_BREAK) {
+			lkl_test_logf("\n");
+			wr = 0;
+		}
+	}
+
+	return TEST_SUCCESS;
+}
+
+LKL_TEST_CALL(close_dir_fd, lkl_sys_close, 0, dir_fd);
+LKL_TEST_CALL(chdir_root, lkl_sys_chdir, 0, "/");
+LKL_TEST_CALL(mount_fs_proc, lkl_sys_mount, 0, "none", "/proc", "proc", 0,
+	      NULL);
+LKL_TEST_CALL(umount_fs_proc, lkl_sys_umount, 0, "/proc", 0);
+
+LKL_TEST_CALL(start_kernel, lkl_start_kernel, 0, &lkl_host_ops,
+	     "mem=16M loglevel=8");
+LKL_TEST_CALL(stop_kernel, lkl_sys_halt, 0);
+
+static struct lkl_test tests[] = {
+	LKL_TEST(start_kernel),
+	LKL_TEST(getpid),
+	LKL_TEST(syscall_latency),
+	LKL_TEST(umask),
+	LKL_TEST(umask2),
+	LKL_TEST(creat),
+	LKL_TEST(close),
+	LKL_TEST(failopen),
+	LKL_TEST(open),
+	LKL_TEST(write),
+	LKL_TEST(lseek_cur),
+	LKL_TEST(lseek_end),
+	LKL_TEST(lseek_set),
+	LKL_TEST(read),
+	LKL_TEST(fstat),
+	LKL_TEST(mkdir),
+	LKL_TEST(stat),
+	LKL_TEST(nanosleep),
+	LKL_TEST(pipe2),
+	LKL_TEST(epoll),
+	LKL_TEST(mount_fs_proc),
+	LKL_TEST(chdir_proc),
+	LKL_TEST(open_cwd),
+	LKL_TEST(getdents64),
+	LKL_TEST(close_dir_fd),
+	LKL_TEST(chdir_root),
+	LKL_TEST(umount_fs_proc),
+	LKL_TEST(stop_kernel),
+};
+
+int main(int argc, const char **argv)
+{
+	return lkl_test_run(tests, sizeof(tests)/sizeof(struct lkl_test),
+			    "boot");
+}
diff --git a/tools/testing/selftests/um/test.c b/tools/testing/selftests/um/test.c
new file mode 100644
index 000000000000..a3e46852d307
--- /dev/null
+++ b/tools/testing/selftests/um/test.c
@@ -0,0 +1,128 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <stdarg.h>
+#include <time.h>
+
+#include "test.h"
+#include "../kselftest.h"
+
+/* circular log buffer */
+
+static char log_buf[0x10000];
+static char *head = log_buf, *tail = log_buf;
+
+static inline void advance(char **ptr)
+{
+	if ((unsigned int)(*ptr - log_buf) >= sizeof(log_buf))
+		*ptr = log_buf;
+	else
+		*ptr = *ptr + 1;
+}
+
+static void log_char(char c)
+{
+	*tail = c;
+	advance(&tail);
+	if (tail == head)
+		advance(&head);
+}
+
+static void print_log(void)
+{
+	char last;
+
+	printf(" log: |\n");
+	last = '\n';
+	while (head != tail) {
+		if (last == '\n')
+			printf("  ");
+		last = *head;
+		putchar(last);
+		advance(&head);
+	}
+	if (last != '\n')
+		putchar('\n');
+}
+
+int lkl_test_run(const struct lkl_test *tests, int nr, const char *fmt, ...)
+{
+	int i, ret, status = TEST_SUCCESS;
+	clock_t start, stop;
+	char name[1024];
+	va_list args;
+
+	va_start(args, fmt);
+	vsnprintf(name, sizeof(name), fmt, args);
+	va_end(args);
+
+	ksft_set_plan(nr);
+
+	for (i = 1; i <= nr; i++) {
+		const struct lkl_test *t = &tests[i-1];
+		unsigned long delta_us;
+
+		printf("* %d %s\n", i, t->name);
+		fflush(stdout);
+
+		start = clock();
+
+		ret = t->fn(t->arg1, t->arg2, t->arg3);
+
+		stop = clock();
+
+		switch (ret) {
+		case TEST_SUCCESS:
+			ksft_test_result_pass("%s\n", t->name);
+			break;
+		case TEST_SKIP:
+			ksft_test_result_skip("%s\n", t->name);
+			break;
+		case TEST_BAILOUT:
+			status = TEST_BAILOUT;
+			/* fall through; */
+		case TEST_FAILURE:
+		default:
+			if (status != TEST_BAILOUT)
+				status = TEST_FAILURE;
+			ksft_test_result_fail("%s\n", t->name);
+		}
+
+		printf(" ---\n");
+		delta_us = (stop - start) * 1000000 / CLOCKS_PER_SEC;
+		printf(" time_us: %ld\n", delta_us);
+		print_log();
+		printf(" ...\n");
+
+		if (status == TEST_BAILOUT) {
+			printf("Bail out!\n");
+			return TEST_FAILURE;
+		}
+
+		fflush(stdout);
+	}
+
+	return status;
+}
+
+
+void lkl_print(const char *str, int len)
+{
+	while (len--)
+		log_char(*(str++));
+}
+
+int lkl_test_logf(const char *fmt, ...)
+{
+	char tmp[1024], *c;
+	va_list args;
+	unsigned int n;
+
+	va_start(args, fmt);
+	n = vsnprintf(tmp, sizeof(tmp), fmt, args);
+	va_end(args);
+
+	for (c = tmp; *c != 0; c++)
+		log_char(*c);
+
+	return n > sizeof(tmp) ? sizeof(tmp) : n;
+}
diff --git a/tools/testing/selftests/um/test.h b/tools/testing/selftests/um/test.h
new file mode 100644
index 000000000000..f19bbc487a98
--- /dev/null
+++ b/tools/testing/selftests/um/test.h
@@ -0,0 +1,72 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_TEST_H
+#define _LKL_TEST_H
+
+#define TEST_SUCCESS	0
+#define TEST_FAILURE	1
+#define TEST_SKIP	2
+#define TEST_TODO	3
+#define TEST_BAILOUT	4
+
+struct lkl_test {
+	const char *name;
+	int (*fn)();
+	void *arg1, *arg2, *arg3;
+};
+
+/**
+ * Simple wrapper to initialize a test entry.
+ * @name - test name, it assume test function is named test_@name
+ * @vargs - arguments to be passed to the function
+ */
+#define LKL_TEST(name, ...) { #name, lkl_test_##name, __VA_ARGS__ }
+
+/**
+ * lkl_test_run - run a test suite
+ *
+ * @tests - the list of tests to run
+ * @nr - number of tests
+ * @fmt - format string to be used for suite name
+ */
+int lkl_test_run(const struct lkl_test *tests, int nr, const char *fmt, ...);
+
+/**
+ * lkl_test_log - store a string in the test log buffer
+ * @str - the string to log (can be non-NULL terminated)
+ * @len - the string length
+ */
+void lkl_test_log(const char *str, int len);
+
+/**
+ * lkl_test_logf - printf like function to store into the test log buffer
+ * @fmt - printf format string
+ * @vargs - arguments to the format string
+ */
+int lkl_test_logf(const char *fmt, ...) __attribute__((format(printf, 1, 2)));
+
+/**
+ * LKL_TEST_CALL - create a test function as for a LKL call
+ *
+ * The test function will be named lkl_test_@name and will return
+ * TEST_SUCCESS if the called functions returns @expect. Otherwise
+ * will return TEST_FAILUIRE.
+ *
+ * @name - test name; must be unique because it is part of the
+ * test function; the test function will be named
+ * @call - function to call
+ * @expect - expected return value for success
+ * @args - arguments to pass to the LKL call
+ */
+#define LKL_TEST_CALL(name, call, expect, ...)				\
+	static int lkl_test_##name(void)				\
+	{								\
+		long ret;						\
+									\
+		ret = call(__VA_ARGS__);				\
+		lkl_test_logf("%s(%s) = %ld %s\n", #call, #__VA_ARGS__, \
+			ret, ret < 0 ? lkl_strerror(ret) : "");		\
+		return (ret == expect) ? TEST_SUCCESS : TEST_FAILURE;	\
+	}
+
+
+#endif /* _LKL_TEST_H */
diff --git a/tools/testing/selftests/um/test.sh b/tools/testing/selftests/um/test.sh
new file mode 100644
index 000000000000..1c708f9ce261
--- /dev/null
+++ b/tools/testing/selftests/um/test.sh
@@ -0,0 +1,181 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+basedir=$(cd $script_dir/..; pwd)
+base_objdir=$(cd ${OUTPUT}/; pwd)
+
+TEST_SUCCESS=0
+TEST_FAILURE=1
+TEST_SKIP=113
+TEST_TODO=114
+TEST_BAILOUT=115
+
+print_log()
+{
+    echo " log: |"
+    while read line; do
+        echo "  $line"
+    done < $1
+}
+
+export_vars()
+{
+    if [ -z "$var_file" ]; then
+        return
+    fi
+
+    for i in $@; do
+        echo "$i=${!i}" >> $var_file
+    done
+}
+
+lkl_test_run()
+{
+    log_file=$(mktemp)
+    export var_file=$(mktemp)
+
+    tid=$1 && shift && tname=$@
+
+    echo "* $tid $tname"
+
+    start=$(date '+%s%9N')
+    # run in a separate shell to avoid -e terminating us
+    $@ 2>&1 | strings >$log_file
+    exit=${PIPESTATUS[0]}
+    stop=$(date '+%s%9N')
+
+    case $exit in
+    $TEST_SUCCESS)
+        echo "ok $tid $tname"
+        ;;
+    $TEST_SKIP)
+        echo "ok $tid $tname # SKIP"
+        ;;
+    $TEST_BAILOUT)
+        echo "not ok $tid $tname"
+        echo "Bail out!"
+        ;;
+    $TEST_FAILURE|*)
+        echo "not ok $tid $tname"
+        ;;
+    esac
+
+    delta=$(((stop-start)/1000))
+
+    echo " ---"
+    echo " time_us: $delta"
+    print_log $log_file
+    echo -e " ..."
+
+    rm $log_file
+    . $var_file
+    rm $var_file
+
+    return $exit
+}
+
+lkl_test_plan()
+{
+    echo "1..$1 # $2"
+    export suite_name="${2// /\-}"
+}
+
+lkl_test_exec()
+{
+    local SUDO=""
+    local WRAPPER=""
+
+    if [ "$1" = "sudo" ]; then
+        SUDO=sudo
+        shift
+    fi
+
+    local file=$1
+    shift
+
+    if [ -n "$LKL_HOST_CONFIG_NT" ]; then
+        file=$file.exe
+    fi
+
+    file=${OUTPUT}/$(basename $file)
+
+    if file $file | grep ARM; then
+        WRAPPER="qemu-arm-static"
+    elif file $file | grep "FreeBSD" ; then
+        ssh_copy "$file" $BSD_WDIR
+        if [ -n "$SUDO" ]; then
+            SUDO=""
+        fi
+        WRAPPER="$MYSSH $SU"
+        # ssh will mess up with pipes ('|') so, escape the pipe char.
+        args="${@//\|/\\\|}"
+        set - $BSD_WDIR/$(basename $file) $args
+        file=""
+    elif [ -n "$GDB" ]; then
+        WRAPPER="gdb"
+        args="$@"
+        set - -ex "run $args" -ex quit $file
+        file=""
+    elif [ -n "$VALGRIND" ]; then
+        WRAPPER="valgrind --suppressions=$script_dir/valgrind.supp \
+                  --leak-check=full --show-leak-kinds=all --xml=yes \
+                  --xml-file=valgrind-$suite_name.xml"
+    fi
+
+    $SUDO $WRAPPER $file "$@"
+}
+
+lkl_test_cmd()
+{
+    local WRAPPER=""
+
+    if [ -z "$QUIET" ]; then
+        SHOPTS="-x"
+    fi
+
+    if [ -n "$LKL_HOST_CONFIG_BSD" ]; then
+        WRAPPER="$MYSSH $SU"
+    fi
+
+    echo "$@" | $WRAPPER sh $SHOPTS
+}
+
+# XXX: $MYSSH and $MYSCP are defined in a circleci docker image.
+# see the definitions in lkl/lkl-docker:circleci/freebsd11/Dockerfile
+ssh_push()
+{
+    while [ -n "$1" ]; do
+        if [[ "$1" = *.sh ]]; then
+            type="script"
+        else
+            type="file"
+        fi
+
+        dir=$(dirname $1)
+        $MYSSH mkdir -p $BSD_WDIR/$dir
+
+        $MYSCP -P 7722 -r $basedir/$1 root@localhost:$BSD_WDIR/$dir
+        if [ "$type" = "script" ]; then
+            $MYSSH chmod a+x $BSD_WDIR/$1
+        fi
+
+        shift
+    done
+}
+
+ssh_copy()
+{
+    $MYSCP -P 7722 -r $1 root@localhost:$2
+}
+
+lkl_test_bsd_cleanup()
+{
+    $MYSSH rm -rf $BSD_WDIR
+}
+
+if [ -n "$LKL_HOST_CONFIG_BSD" ]; then
+    trap lkl_test_bsd_cleanup EXIT
+    export BSD_WDIR=/root/lkl
+    $MYSSH mkdir -p $BSD_WDIR
+fi
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v8 18/20] selftests/um: lkl: add test programs for library mode of UML
@ 2021-01-20  2:27             ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-01-20  2:27 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: linux-arch, tavi.purdila, retrage01, linux-kselftest,
	linux-kernel-library, Shuah Khan, thehajime

Add a simple LKL test application (boot) that starts the kernel and
performs simple tests that minimally exercise the LKL API.  The tests
are implemented on kselftest framework, which can be invoked with

 make ARCH=um SUBARCH=lkl TARGETS="um" kselftest

Additionally, this commits add a skip for headers_install for ARCH=um
since UML (even with library mode) doesn't have headers install.

```
 cat /tmp/um-test.tap | sed "s/^# selftests/     \# Subtest: selftests/"
|  sed "s/^#/    /" | sed "s/version 13/version 14/" | sed "s/\(ok
[0-9]* \)/\1- /" | ./tools/testing/kunit/kunit.py parse
```

this makes tap14 converter for the kselftest results.

Cc: Shuah Khan <shuah@kernel.org>
Cc: linux-kselftest@vger.kernel.org
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
---
 tools/testing/selftests/Makefile    |   3 +
 tools/testing/selftests/um/Makefile |  13 +
 tools/testing/selftests/um/boot.c   | 376 ++++++++++++++++++++++++++++
 tools/testing/selftests/um/test.c   | 128 ++++++++++
 tools/testing/selftests/um/test.h   |  72 ++++++
 tools/testing/selftests/um/test.sh  | 181 +++++++++++++
 6 files changed, 773 insertions(+)
 create mode 100644 tools/testing/selftests/um/Makefile
 create mode 100644 tools/testing/selftests/um/boot.c
 create mode 100644 tools/testing/selftests/um/test.c
 create mode 100644 tools/testing/selftests/um/test.h
 create mode 100644 tools/testing/selftests/um/test.sh

diff --git a/tools/testing/selftests/Makefile b/tools/testing/selftests/Makefile
index 8a917cb4426a..26b6fc800778 100644
--- a/tools/testing/selftests/Makefile
+++ b/tools/testing/selftests/Makefile
@@ -65,6 +65,7 @@ TARGETS += timers
 endif
 TARGETS += tmpfs
 TARGETS += tpm2
+TARGETS += um
 TARGETS += user
 TARGETS += vDSO
 TARGETS += vm
@@ -154,12 +155,14 @@ endif
 # Local build cases: "make kselftest", "make -C" - headers are installed
 # in the default INSTALL_HDR_PATH usr/include.
 khdr:
+ifneq (um,$(ARCH))
 ifeq (1,$(DEFAULT_INSTALL_HDR_PATH))
 	$(MAKE) --no-builtin-rules ARCH=$(ARCH) -C $(top_srcdir) headers_install
 else
 	$(MAKE) --no-builtin-rules INSTALL_HDR_PATH=$$BUILD/usr \
 		ARCH=$(ARCH) -C $(top_srcdir) headers_install
 endif
+endif
 
 all: khdr
 	@ret=1;							\
diff --git a/tools/testing/selftests/um/Makefile b/tools/testing/selftests/um/Makefile
new file mode 100644
index 000000000000..1b915f4bb751
--- /dev/null
+++ b/tools/testing/selftests/um/Makefile
@@ -0,0 +1,13 @@
+# SPDX-License-Identifier: GPL-2.0-only
+
+top_srcdir = ../../../../
+top_objdir = $(OUTPUT)/../../
+
+CFLAGS := -fPIC -I$(top_objdir)/tools/um/include/ -I$(top_srcdir)/tools/um/include/ -g
+CFLAGS += test.c
+LDFLAGS := -pie
+LDLIBS := -L$(top_objdir)/tools/um -L$(top_srcdir)/tools/um -llinux -lpthread -lrt -lutil
+
+TEST_GEN_PROGS := boot
+
+include ../lib.mk
diff --git a/tools/testing/selftests/um/boot.c b/tools/testing/selftests/um/boot.c
new file mode 100644
index 000000000000..b1330a39a936
--- /dev/null
+++ b/tools/testing/selftests/um/boot.c
@@ -0,0 +1,376 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <unistd.h>
+#include <string.h>
+#include <time.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <lkl.h>
+#include <lkl_host.h>
+
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <sys/epoll.h>
+#include <sys/ioctl.h>
+
+#include "test.h"
+
+#define sleep_ns 87654321
+static int lkl_test_nanosleep(void)
+{
+	struct lkl_timespec ts = {
+		.tv_sec = 0,
+		.tv_nsec = sleep_ns,
+	};
+	struct timespec start, stop;
+	long delta;
+	long ret;
+
+	clock_gettime(CLOCK_MONOTONIC, &start);
+	ret = lkl_sys_nanosleep((struct __lkl__kernel_timespec *)&ts, NULL);
+	clock_gettime(CLOCK_MONOTONIC, &stop);
+
+	delta = 1e9*(stop.tv_sec - start.tv_sec) +
+		(stop.tv_nsec - start.tv_nsec);
+
+	lkl_test_logf("sleep %ld, expected sleep %d\n", delta, sleep_ns);
+
+	if (ret == 0 && delta > sleep_ns * 0.9)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+LKL_TEST_CALL(getpid, lkl_sys_getpid, 1)
+
+void check_latency(long (*f)(void), long *min, long *max, long *avg)
+{
+	int i;
+	struct timespec start, stop;
+	unsigned long long sum = 0;
+	static const int count = 20;
+	long delta;
+
+	*min = 1000000000;
+	*max = -1;
+
+	for (i = 0; i < count; i++) {
+		clock_gettime(CLOCK_MONOTONIC, &start);
+		f();
+		clock_gettime(CLOCK_MONOTONIC, &stop);
+
+		delta = 1e9*(stop.tv_sec - start.tv_sec) +
+			(stop.tv_nsec - start.tv_nsec);
+
+		if (*min > delta)
+			*min = delta;
+		if (*max < delta)
+			*max = delta;
+		sum += delta;
+	}
+	*avg = sum / count;
+}
+
+static long native_getpid(void)
+{
+	getpid();
+	return 0;
+}
+
+int lkl_test_syscall_latency(void)
+{
+	long min, max, avg;
+
+	lkl_test_logf("avg/min/max: ");
+
+	check_latency(lkl_sys_getpid, &min, &max, &avg);
+
+	lkl_test_logf("lkl:%ld/%ld/%ld ", avg, min, max);
+
+	check_latency(native_getpid, &min, &max, &avg);
+
+	lkl_test_logf("native:%ld/%ld/%ld\n", avg, min, max);
+
+	return TEST_SUCCESS;
+}
+
+#define access_rights 0721
+
+LKL_TEST_CALL(creat, lkl_sys_creat, 3, "/file", access_rights)
+LKL_TEST_CALL(close, lkl_sys_close, 0, 0);
+LKL_TEST_CALL(failopen, lkl_sys_open, -LKL_ENOENT, "/file2", 0, 0);
+LKL_TEST_CALL(umask, lkl_sys_umask, 022,  0777);
+LKL_TEST_CALL(umask2, lkl_sys_umask, 0777, 0);
+LKL_TEST_CALL(open, lkl_sys_open, 0, "/file", LKL_O_RDWR, 0);
+static const char wrbuf[] = "test";
+LKL_TEST_CALL(write, lkl_sys_write, sizeof(wrbuf), 0, wrbuf, sizeof(wrbuf));
+LKL_TEST_CALL(lseek_cur, lkl_sys_lseek, sizeof(wrbuf), 0, 0, LKL_SEEK_CUR);
+LKL_TEST_CALL(lseek_end, lkl_sys_lseek, sizeof(wrbuf), 0, 0, LKL_SEEK_END);
+LKL_TEST_CALL(lseek_set, lkl_sys_lseek, 0, 0, 0, LKL_SEEK_SET);
+
+int lkl_test_read(void)
+{
+	char buf[10] = { 0, };
+	long ret;
+
+	ret = lkl_sys_read(0, buf, sizeof(buf));
+
+	lkl_test_logf("lkl_sys_read=%ld buf=%s\n", ret, buf);
+
+	if (ret == sizeof(wrbuf) && !strcmp(wrbuf, buf))
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+int lkl_test_fstat(void)
+{
+	struct lkl_stat stat;
+	long ret;
+
+	ret = lkl_sys_fstat(0, (void *)&stat);
+
+	lkl_test_logf("lkl_sys_fstat=%ld mode=%o size=%ld\n", ret, stat.st_mode,
+		      stat.st_size);
+
+	if (ret == 0 && stat.st_size == sizeof(wrbuf) &&
+	    stat.st_mode == (access_rights | LKL_S_IFREG))
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+LKL_TEST_CALL(mkdir, lkl_sys_mkdir, 0, "/proc", access_rights)
+
+int lkl_test_stat(void)
+{
+	struct lkl_stat stat;
+	long ret;
+
+	ret = lkl_sys_stat("/proc", (void *)&stat);
+
+	lkl_test_logf("lkl_sys_stat(\"/proc\")=%ld mode=%o\n", ret,
+		      stat.st_mode);
+
+	if (ret == 0 && stat.st_mode == (access_rights | LKL_S_IFDIR))
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+static int lkl_test_pipe2(void)
+{
+	int pipe_fds[2];
+	int READ_IDX = 0, WRITE_IDX = 1;
+	static const char msg[] = "Hello world!";
+	char str[20];
+	int msg_len_bytes = strlen(msg) + 1;
+	int cmp_res;
+	long ret;
+
+	ret = lkl_sys_pipe2(pipe_fds, LKL_O_NONBLOCK);
+	if (ret) {
+		lkl_test_logf("pipe2: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_write(pipe_fds[WRITE_IDX], msg, msg_len_bytes);
+	if (ret != msg_len_bytes) {
+		if (ret < 0)
+			lkl_test_logf("write error: %s\n", lkl_strerror(ret));
+		else
+			lkl_test_logf("short write: %ld\n", ret);
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_read(pipe_fds[READ_IDX], str, msg_len_bytes);
+	if (ret != msg_len_bytes) {
+		if (ret < 0)
+			lkl_test_logf("read error: %s\n", lkl_strerror(ret));
+		else
+			lkl_test_logf("short read: %ld\n", ret);
+		return TEST_FAILURE;
+	}
+
+	cmp_res = memcmp(msg, str, msg_len_bytes);
+	if (cmp_res) {
+		lkl_test_logf("memcmp failed: %d\n", cmp_res);
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_close(pipe_fds[0]);
+	if (ret) {
+		lkl_test_logf("close error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_close(pipe_fds[1]);
+	if (ret) {
+		lkl_test_logf("close error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	return TEST_SUCCESS;
+}
+
+static int lkl_test_epoll(void)
+{
+	int epoll_fd, pipe_fds[2];
+	int READ_IDX = 0, WRITE_IDX = 1;
+	struct lkl_epoll_event wait_on, read_result;
+	static const char msg[] = "Hello world!";
+	long ret;
+
+	memset(&wait_on, 0, sizeof(wait_on));
+	memset(&read_result, 0, sizeof(read_result));
+
+	ret = lkl_sys_pipe2(pipe_fds, LKL_O_NONBLOCK);
+	if (ret) {
+		lkl_test_logf("pipe2 error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	epoll_fd = lkl_sys_epoll_create(1);
+	if (epoll_fd < 0) {
+		lkl_test_logf("epoll_create error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	wait_on.events = LKL_POLLIN | LKL_POLLOUT;
+	wait_on.data = pipe_fds[READ_IDX];
+
+	ret = lkl_sys_epoll_ctl(epoll_fd, LKL_EPOLL_CTL_ADD, pipe_fds[READ_IDX],
+				&wait_on);
+	if (ret < 0) {
+		lkl_test_logf("epoll_ctl error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	/* Shouldn't be ready before we have written something */
+	ret = lkl_sys_epoll_wait(epoll_fd, &read_result, 1, 0);
+	if (ret != 0) {
+		if (ret < 0)
+			lkl_test_logf("epoll_wait error: %s\n",
+				      lkl_strerror(ret));
+		else
+			lkl_test_logf("epoll_wait: bad event: 0x%lx\n", ret);
+		return TEST_FAILURE;
+	}
+
+	ret = lkl_sys_write(pipe_fds[WRITE_IDX], msg, strlen(msg) + 1);
+	if (ret < 0) {
+		lkl_test_logf("write error: %s\n", lkl_strerror(ret));
+		return TEST_FAILURE;
+	}
+
+	/* We expect exactly 1 fd to be ready immediately */
+	ret = lkl_sys_epoll_wait(epoll_fd, &read_result, 1, 0);
+	if (ret != 1) {
+		if (ret < 0)
+			lkl_test_logf("epoll_wait error: %s\n",
+				      lkl_strerror(ret));
+		else
+			lkl_test_logf("epoll_wait: bad ev no %ld\n", ret);
+		return TEST_FAILURE;
+	}
+
+	/* Already tested reading from pipe2 so no need to do it
+	 * here
+	 */
+
+	return TEST_SUCCESS;
+}
+
+LKL_TEST_CALL(chdir_proc, lkl_sys_chdir, 0, "proc");
+
+static int dir_fd;
+
+static int lkl_test_open_cwd(void)
+{
+	dir_fd = lkl_sys_open(".", LKL_O_RDONLY | LKL_O_DIRECTORY, 0);
+	if (dir_fd < 0) {
+		lkl_test_logf("failed to open current directory: %s\n",
+			      lkl_strerror(dir_fd));
+		return TEST_FAILURE;
+	}
+
+	return TEST_SUCCESS;
+}
+
+/* column where to insert a line break for the list file tests below. */
+#define COL_LINE_BREAK 70
+
+static int lkl_test_getdents64(void)
+{
+	long ret;
+	char buf[1024], *pos;
+	struct lkl_linux_dirent64 *de;
+	int wr;
+
+	de = (struct lkl_linux_dirent64 *)buf;
+	ret = lkl_sys_getdents64(dir_fd, de, sizeof(buf));
+
+	wr = lkl_test_logf("%d ", dir_fd);
+
+	if (ret < 0)
+		return TEST_FAILURE;
+
+	for (pos = buf; pos - buf < ret; pos += de->d_reclen) {
+		de = (struct lkl_linux_dirent64 *)pos;
+
+		wr += lkl_test_logf("%s ", de->d_name);
+		if (wr >= COL_LINE_BREAK) {
+			lkl_test_logf("\n");
+			wr = 0;
+		}
+	}
+
+	return TEST_SUCCESS;
+}
+
+LKL_TEST_CALL(close_dir_fd, lkl_sys_close, 0, dir_fd);
+LKL_TEST_CALL(chdir_root, lkl_sys_chdir, 0, "/");
+LKL_TEST_CALL(mount_fs_proc, lkl_sys_mount, 0, "none", "/proc", "proc", 0,
+	      NULL);
+LKL_TEST_CALL(umount_fs_proc, lkl_sys_umount, 0, "/proc", 0);
+
+LKL_TEST_CALL(start_kernel, lkl_start_kernel, 0, &lkl_host_ops,
+	     "mem=16M loglevel=8");
+LKL_TEST_CALL(stop_kernel, lkl_sys_halt, 0);
+
+static struct lkl_test tests[] = {
+	LKL_TEST(start_kernel),
+	LKL_TEST(getpid),
+	LKL_TEST(syscall_latency),
+	LKL_TEST(umask),
+	LKL_TEST(umask2),
+	LKL_TEST(creat),
+	LKL_TEST(close),
+	LKL_TEST(failopen),
+	LKL_TEST(open),
+	LKL_TEST(write),
+	LKL_TEST(lseek_cur),
+	LKL_TEST(lseek_end),
+	LKL_TEST(lseek_set),
+	LKL_TEST(read),
+	LKL_TEST(fstat),
+	LKL_TEST(mkdir),
+	LKL_TEST(stat),
+	LKL_TEST(nanosleep),
+	LKL_TEST(pipe2),
+	LKL_TEST(epoll),
+	LKL_TEST(mount_fs_proc),
+	LKL_TEST(chdir_proc),
+	LKL_TEST(open_cwd),
+	LKL_TEST(getdents64),
+	LKL_TEST(close_dir_fd),
+	LKL_TEST(chdir_root),
+	LKL_TEST(umount_fs_proc),
+	LKL_TEST(stop_kernel),
+};
+
+int main(int argc, const char **argv)
+{
+	return lkl_test_run(tests, sizeof(tests)/sizeof(struct lkl_test),
+			    "boot");
+}
diff --git a/tools/testing/selftests/um/test.c b/tools/testing/selftests/um/test.c
new file mode 100644
index 000000000000..a3e46852d307
--- /dev/null
+++ b/tools/testing/selftests/um/test.c
@@ -0,0 +1,128 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <stdarg.h>
+#include <time.h>
+
+#include "test.h"
+#include "../kselftest.h"
+
+/* circular log buffer */
+
+static char log_buf[0x10000];
+static char *head = log_buf, *tail = log_buf;
+
+static inline void advance(char **ptr)
+{
+	if ((unsigned int)(*ptr - log_buf) >= sizeof(log_buf))
+		*ptr = log_buf;
+	else
+		*ptr = *ptr + 1;
+}
+
+static void log_char(char c)
+{
+	*tail = c;
+	advance(&tail);
+	if (tail == head)
+		advance(&head);
+}
+
+static void print_log(void)
+{
+	char last;
+
+	printf(" log: |\n");
+	last = '\n';
+	while (head != tail) {
+		if (last == '\n')
+			printf("  ");
+		last = *head;
+		putchar(last);
+		advance(&head);
+	}
+	if (last != '\n')
+		putchar('\n');
+}
+
+int lkl_test_run(const struct lkl_test *tests, int nr, const char *fmt, ...)
+{
+	int i, ret, status = TEST_SUCCESS;
+	clock_t start, stop;
+	char name[1024];
+	va_list args;
+
+	va_start(args, fmt);
+	vsnprintf(name, sizeof(name), fmt, args);
+	va_end(args);
+
+	ksft_set_plan(nr);
+
+	for (i = 1; i <= nr; i++) {
+		const struct lkl_test *t = &tests[i-1];
+		unsigned long delta_us;
+
+		printf("* %d %s\n", i, t->name);
+		fflush(stdout);
+
+		start = clock();
+
+		ret = t->fn(t->arg1, t->arg2, t->arg3);
+
+		stop = clock();
+
+		switch (ret) {
+		case TEST_SUCCESS:
+			ksft_test_result_pass("%s\n", t->name);
+			break;
+		case TEST_SKIP:
+			ksft_test_result_skip("%s\n", t->name);
+			break;
+		case TEST_BAILOUT:
+			status = TEST_BAILOUT;
+			/* fall through; */
+		case TEST_FAILURE:
+		default:
+			if (status != TEST_BAILOUT)
+				status = TEST_FAILURE;
+			ksft_test_result_fail("%s\n", t->name);
+		}
+
+		printf(" ---\n");
+		delta_us = (stop - start) * 1000000 / CLOCKS_PER_SEC;
+		printf(" time_us: %ld\n", delta_us);
+		print_log();
+		printf(" ...\n");
+
+		if (status == TEST_BAILOUT) {
+			printf("Bail out!\n");
+			return TEST_FAILURE;
+		}
+
+		fflush(stdout);
+	}
+
+	return status;
+}
+
+
+void lkl_print(const char *str, int len)
+{
+	while (len--)
+		log_char(*(str++));
+}
+
+int lkl_test_logf(const char *fmt, ...)
+{
+	char tmp[1024], *c;
+	va_list args;
+	unsigned int n;
+
+	va_start(args, fmt);
+	n = vsnprintf(tmp, sizeof(tmp), fmt, args);
+	va_end(args);
+
+	for (c = tmp; *c != 0; c++)
+		log_char(*c);
+
+	return n > sizeof(tmp) ? sizeof(tmp) : n;
+}
diff --git a/tools/testing/selftests/um/test.h b/tools/testing/selftests/um/test.h
new file mode 100644
index 000000000000..f19bbc487a98
--- /dev/null
+++ b/tools/testing/selftests/um/test.h
@@ -0,0 +1,72 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LKL_TEST_H
+#define _LKL_TEST_H
+
+#define TEST_SUCCESS	0
+#define TEST_FAILURE	1
+#define TEST_SKIP	2
+#define TEST_TODO	3
+#define TEST_BAILOUT	4
+
+struct lkl_test {
+	const char *name;
+	int (*fn)();
+	void *arg1, *arg2, *arg3;
+};
+
+/**
+ * Simple wrapper to initialize a test entry.
+ * @name - test name, it assume test function is named test_@name
+ * @vargs - arguments to be passed to the function
+ */
+#define LKL_TEST(name, ...) { #name, lkl_test_##name, __VA_ARGS__ }
+
+/**
+ * lkl_test_run - run a test suite
+ *
+ * @tests - the list of tests to run
+ * @nr - number of tests
+ * @fmt - format string to be used for suite name
+ */
+int lkl_test_run(const struct lkl_test *tests, int nr, const char *fmt, ...);
+
+/**
+ * lkl_test_log - store a string in the test log buffer
+ * @str - the string to log (can be non-NULL terminated)
+ * @len - the string length
+ */
+void lkl_test_log(const char *str, int len);
+
+/**
+ * lkl_test_logf - printf like function to store into the test log buffer
+ * @fmt - printf format string
+ * @vargs - arguments to the format string
+ */
+int lkl_test_logf(const char *fmt, ...) __attribute__((format(printf, 1, 2)));
+
+/**
+ * LKL_TEST_CALL - create a test function as for a LKL call
+ *
+ * The test function will be named lkl_test_@name and will return
+ * TEST_SUCCESS if the called functions returns @expect. Otherwise
+ * will return TEST_FAILUIRE.
+ *
+ * @name - test name; must be unique because it is part of the
+ * test function; the test function will be named
+ * @call - function to call
+ * @expect - expected return value for success
+ * @args - arguments to pass to the LKL call
+ */
+#define LKL_TEST_CALL(name, call, expect, ...)				\
+	static int lkl_test_##name(void)				\
+	{								\
+		long ret;						\
+									\
+		ret = call(__VA_ARGS__);				\
+		lkl_test_logf("%s(%s) = %ld %s\n", #call, #__VA_ARGS__, \
+			ret, ret < 0 ? lkl_strerror(ret) : "");		\
+		return (ret == expect) ? TEST_SUCCESS : TEST_FAILURE;	\
+	}
+
+
+#endif /* _LKL_TEST_H */
diff --git a/tools/testing/selftests/um/test.sh b/tools/testing/selftests/um/test.sh
new file mode 100644
index 000000000000..1c708f9ce261
--- /dev/null
+++ b/tools/testing/selftests/um/test.sh
@@ -0,0 +1,181 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+basedir=$(cd $script_dir/..; pwd)
+base_objdir=$(cd ${OUTPUT}/; pwd)
+
+TEST_SUCCESS=0
+TEST_FAILURE=1
+TEST_SKIP=113
+TEST_TODO=114
+TEST_BAILOUT=115
+
+print_log()
+{
+    echo " log: |"
+    while read line; do
+        echo "  $line"
+    done < $1
+}
+
+export_vars()
+{
+    if [ -z "$var_file" ]; then
+        return
+    fi
+
+    for i in $@; do
+        echo "$i=${!i}" >> $var_file
+    done
+}
+
+lkl_test_run()
+{
+    log_file=$(mktemp)
+    export var_file=$(mktemp)
+
+    tid=$1 && shift && tname=$@
+
+    echo "* $tid $tname"
+
+    start=$(date '+%s%9N')
+    # run in a separate shell to avoid -e terminating us
+    $@ 2>&1 | strings >$log_file
+    exit=${PIPESTATUS[0]}
+    stop=$(date '+%s%9N')
+
+    case $exit in
+    $TEST_SUCCESS)
+        echo "ok $tid $tname"
+        ;;
+    $TEST_SKIP)
+        echo "ok $tid $tname # SKIP"
+        ;;
+    $TEST_BAILOUT)
+        echo "not ok $tid $tname"
+        echo "Bail out!"
+        ;;
+    $TEST_FAILURE|*)
+        echo "not ok $tid $tname"
+        ;;
+    esac
+
+    delta=$(((stop-start)/1000))
+
+    echo " ---"
+    echo " time_us: $delta"
+    print_log $log_file
+    echo -e " ..."
+
+    rm $log_file
+    . $var_file
+    rm $var_file
+
+    return $exit
+}
+
+lkl_test_plan()
+{
+    echo "1..$1 # $2"
+    export suite_name="${2// /\-}"
+}
+
+lkl_test_exec()
+{
+    local SUDO=""
+    local WRAPPER=""
+
+    if [ "$1" = "sudo" ]; then
+        SUDO=sudo
+        shift
+    fi
+
+    local file=$1
+    shift
+
+    if [ -n "$LKL_HOST_CONFIG_NT" ]; then
+        file=$file.exe
+    fi
+
+    file=${OUTPUT}/$(basename $file)
+
+    if file $file | grep ARM; then
+        WRAPPER="qemu-arm-static"
+    elif file $file | grep "FreeBSD" ; then
+        ssh_copy "$file" $BSD_WDIR
+        if [ -n "$SUDO" ]; then
+            SUDO=""
+        fi
+        WRAPPER="$MYSSH $SU"
+        # ssh will mess up with pipes ('|') so, escape the pipe char.
+        args="${@//\|/\\\|}"
+        set - $BSD_WDIR/$(basename $file) $args
+        file=""
+    elif [ -n "$GDB" ]; then
+        WRAPPER="gdb"
+        args="$@"
+        set - -ex "run $args" -ex quit $file
+        file=""
+    elif [ -n "$VALGRIND" ]; then
+        WRAPPER="valgrind --suppressions=$script_dir/valgrind.supp \
+                  --leak-check=full --show-leak-kinds=all --xml=yes \
+                  --xml-file=valgrind-$suite_name.xml"
+    fi
+
+    $SUDO $WRAPPER $file "$@"
+}
+
+lkl_test_cmd()
+{
+    local WRAPPER=""
+
+    if [ -z "$QUIET" ]; then
+        SHOPTS="-x"
+    fi
+
+    if [ -n "$LKL_HOST_CONFIG_BSD" ]; then
+        WRAPPER="$MYSSH $SU"
+    fi
+
+    echo "$@" | $WRAPPER sh $SHOPTS
+}
+
+# XXX: $MYSSH and $MYSCP are defined in a circleci docker image.
+# see the definitions in lkl/lkl-docker:circleci/freebsd11/Dockerfile
+ssh_push()
+{
+    while [ -n "$1" ]; do
+        if [[ "$1" = *.sh ]]; then
+            type="script"
+        else
+            type="file"
+        fi
+
+        dir=$(dirname $1)
+        $MYSSH mkdir -p $BSD_WDIR/$dir
+
+        $MYSCP -P 7722 -r $basedir/$1 root@localhost:$BSD_WDIR/$dir
+        if [ "$type" = "script" ]; then
+            $MYSSH chmod a+x $BSD_WDIR/$1
+        fi
+
+        shift
+    done
+}
+
+ssh_copy()
+{
+    $MYSCP -P 7722 -r $1 root@localhost:$2
+}
+
+lkl_test_bsd_cleanup()
+{
+    $MYSSH rm -rf $BSD_WDIR
+}
+
+if [ -n "$LKL_HOST_CONFIG_BSD" ]; then
+    trap lkl_test_bsd_cleanup EXIT
+    export BSD_WDIR=/root/lkl
+    $MYSSH mkdir -p $BSD_WDIR
+fi
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v8 19/20] um: lkl: add block device support of UML
  2021-01-20  2:27           ` Hajime Tazaki
@ 2021-01-20  2:27             ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-01-20  2:27 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: thehajime, tavi.purdila, retrage01, linux-kernel-library, linux-arch

This commit adds block device support for library mode, and also added
several host utilities to run a simple test program
(tools/um/test/disk.c) successfully.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/Kconfig                         |   4 +
 arch/um/include/asm/xor.h               |   3 +-
 arch/um/include/shared/as-layout.h      |   1 +
 arch/um/kernel/um_arch.c                |   5 +
 arch/um/lkl/include/uapi/asm/host_ops.h |   5 +
 arch/um/lkl/include/uapi/asm/unistd.h   |   2 +
 arch/um/lkl/um/setup.c                  |  12 +
 tools/testing/selftests/um/Makefile     |   5 +-
 tools/testing/selftests/um/cla.c        | 159 ++++++++
 tools/testing/selftests/um/cla.h        |  33 ++
 tools/testing/selftests/um/disk-ext4.sh |   6 +
 tools/testing/selftests/um/disk-vfat.sh |   6 +
 tools/testing/selftests/um/disk.c       | 166 +++++++++
 tools/testing/selftests/um/disk.sh      |  67 ++++
 tools/um/include/lkl.h                  | 206 +++++++++++
 tools/um/include/lkl_host.h             |   1 +
 tools/um/lib/Build                      |   1 +
 tools/um/lib/fs.c                       | 461 ++++++++++++++++++++++++
 tools/um/lib/posix-host.c               |   1 +
 tools/um/lib/utils.c                    |   2 +
 20 files changed, 1144 insertions(+), 2 deletions(-)
 create mode 100644 tools/testing/selftests/um/cla.c
 create mode 100644 tools/testing/selftests/um/cla.h
 create mode 100755 tools/testing/selftests/um/disk-ext4.sh
 create mode 100755 tools/testing/selftests/um/disk-vfat.sh
 create mode 100644 tools/testing/selftests/um/disk.c
 create mode 100755 tools/testing/selftests/um/disk.sh
 create mode 100644 tools/um/lib/fs.c

diff --git a/arch/um/Kconfig b/arch/um/Kconfig
index 24c6596260de..5fb6a852d058 100644
--- a/arch/um/Kconfig
+++ b/arch/um/Kconfig
@@ -29,6 +29,10 @@ config UMMODE_LIB
 	select UACCESS_MEMCPY
 	select ARCH_THREAD_STACK_ALLOCATOR
 	select ARCH_HAS_SYSCALL_WRAPPER
+	select VFAT_FS
+	select NLS_CODEPAGE_437
+	select NLS_ISO8859_1
+	select BTRFS_FS
 	help
 	 This mode switches a mode to build a library of UML (Linux
 	 Kernel Library/LKL).  If this is Y, the build only generates,
diff --git a/arch/um/include/asm/xor.h b/arch/um/include/asm/xor.h
index 36b33d62a35d..934a576ec4b2 100644
--- a/arch/um/include/asm/xor.h
+++ b/arch/um/include/asm/xor.h
@@ -4,4 +4,5 @@
 
 /* pick an arbitrary one - measuring isn't possible with inf-cpu */
 #define XOR_SELECT_TEMPLATE(x)	\
-	(time_travel_mode == TT_MODE_INFCPU ? &xor_block_8regs : NULL)
+	(time_travel_mode == TT_MODE_INFCPU || \
+	 (IS_ENABLED(CONFIG_UMMODE_LIB)) ? &xor_block_8regs : NULL)
diff --git a/arch/um/include/shared/as-layout.h b/arch/um/include/shared/as-layout.h
index 5f286ef2721b..4423437a5ace 100644
--- a/arch/um/include/shared/as-layout.h
+++ b/arch/um/include/shared/as-layout.h
@@ -57,6 +57,7 @@ extern unsigned long host_task_size;
 
 extern int linux_main(int argc, char **argv);
 extern void uml_finishsetup(void);
+extern void uml_set_args(char *args);
 
 struct siginfo;
 extern void (*sig_info[])(int, struct siginfo *si, struct uml_pt_regs *);
diff --git a/arch/um/kernel/um_arch.c b/arch/um/kernel/um_arch.c
index dfc6194b5ac7..3968842346a2 100644
--- a/arch/um/kernel/um_arch.c
+++ b/arch/um/kernel/um_arch.c
@@ -42,6 +42,11 @@ static void __init add_arg(char *arg)
 	strcat(command_line, arg);
 }
 
+void __init uml_set_args(char *args)
+{
+	strcat(command_line, args);
+}
+
 /*
  * These fields are initialized at boot time and not changed.
  * XXX This structure is used only in the non-SMP case.  Maybe this
diff --git a/arch/um/lkl/include/uapi/asm/host_ops.h b/arch/um/lkl/include/uapi/asm/host_ops.h
index 85d7d4790602..c7239df7f807 100644
--- a/arch/um/lkl/include/uapi/asm/host_ops.h
+++ b/arch/um/lkl/include/uapi/asm/host_ops.h
@@ -12,11 +12,16 @@ struct lkl_jmp_buf {
 /**
  * struct lkl_host_operations - host operations used by the Linux kernel
  *
+ * @um_devices: string containg the list of UML devices in command line
+ * format. This string is appended to the kernel command line and
+ * is provided here for convenience to be implemented by the host library.
+ *
  * These operations must be provided by a host library or by the application
  * itself.
  *
  */
 struct lkl_host_operations {
+	const char *um_devices;
 };
 
 /**
diff --git a/arch/um/lkl/include/uapi/asm/unistd.h b/arch/um/lkl/include/uapi/asm/unistd.h
index 6f5d67b45fbd..d777e4b597d2 100644
--- a/arch/um/lkl/include/uapi/asm/unistd.h
+++ b/arch/um/lkl/include/uapi/asm/unistd.h
@@ -3,6 +3,8 @@
 #define __UM_LIBMODE_UAPI_UNISTD_H
 
 #define __ARCH_WANT_NEW_STAT
+#define __ARCH_WANT_SET_GET_RLIMIT
+
 #include <asm/bitsperlong.h>
 
 #if __BITS_PER_LONG == 64
diff --git a/arch/um/lkl/um/setup.c b/arch/um/lkl/um/setup.c
index 12b235826573..f33e9f477b7a 100644
--- a/arch/um/lkl/um/setup.c
+++ b/arch/um/lkl/um/setup.c
@@ -45,13 +45,25 @@ static void __init *lkl_run_kernel(void *arg)
 	return NULL;
 }
 
+static char _cmd_line[COMMAND_LINE_SIZE];
 int __init lkl_start_kernel(struct lkl_host_operations *ops,
 			    const char *fmt, ...)
 {
+	va_list ap;
 	int ret;
 
 	lkl_ops = ops;
 
+	va_start(ap, fmt);
+	ret = vsnprintf(_cmd_line, COMMAND_LINE_SIZE, fmt, ap);
+	va_end(ap);
+
+	if (ops->um_devices)
+		strscpy(_cmd_line + ret, ops->um_devices,
+			COMMAND_LINE_SIZE - ret);
+
+	uml_set_args(_cmd_line);
+
 	init_sem = lkl_sem_alloc(0);
 	if (!init_sem)
 		return -ENOMEM;
diff --git a/tools/testing/selftests/um/Makefile b/tools/testing/selftests/um/Makefile
index 1b915f4bb751..df9561a0b3b4 100644
--- a/tools/testing/selftests/um/Makefile
+++ b/tools/testing/selftests/um/Makefile
@@ -4,10 +4,13 @@ top_srcdir = ../../../../
 top_objdir = $(OUTPUT)/../../
 
 CFLAGS := -fPIC -I$(top_objdir)/tools/um/include/ -I$(top_srcdir)/tools/um/include/ -g
-CFLAGS += test.c
+CFLAGS += test.c cla.c
 LDFLAGS := -pie
 LDLIBS := -L$(top_objdir)/tools/um -L$(top_srcdir)/tools/um -llinux -lpthread -lrt -lutil
 
 TEST_GEN_PROGS := boot
+TEST_PROGS := disk-ext4.sh disk-vfat.sh
+TEST_PROGS_EXTENDED := test.sh disk.sh
+TEST_GEN_FILES := disk
 
 include ../lib.mk
diff --git a/tools/testing/selftests/um/cla.c b/tools/testing/selftests/um/cla.c
new file mode 100644
index 000000000000..694e3a3822be
--- /dev/null
+++ b/tools/testing/selftests/um/cla.c
@@ -0,0 +1,159 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <string.h>
+#include <errno.h>
+#include <stdlib.h>
+#ifdef __MINGW32__
+#include <winsock2.h>
+#else
+#include <sys/socket.h>
+#include <netinet/in.h>
+#include <arpa/inet.h>
+#endif
+
+#include "cla.h"
+
+static int cl_arg_parse_bool(struct cl_arg *arg, const char *value)
+{
+	*((int *)arg->store) = 1;
+	return 0;
+}
+
+static int cl_arg_parse_str(struct cl_arg *arg, const char *value)
+{
+	*((const char **)arg->store) = value;
+	return 0;
+}
+
+static int cl_arg_parse_int(struct cl_arg *arg, const char *value)
+{
+	errno = 0;
+	*((int *)arg->store) = strtol(value, NULL, 0);
+	return errno == 0;
+}
+
+static int cl_arg_parse_str_set(struct cl_arg *arg, const char *value)
+{
+	const char **set = arg->set;
+	int i;
+
+	for (i = 0; set[i] != NULL; i++) {
+		if (strcmp(set[i], value) == 0) {
+			*((int *)arg->store) = i;
+			return 0;
+		}
+	}
+
+	return (-1);
+}
+
+static int cl_arg_parse_ipv4(struct cl_arg *arg, const char *value)
+{
+	unsigned int addr;
+
+	if (!value)
+		return (-1);
+
+	addr = inet_addr(value);
+	if (addr == INADDR_NONE)
+		return (-1);
+	*((unsigned int *)arg->store) = addr;
+	return 0;
+}
+
+static cl_arg_parser_t parsers[] = {
+	[CL_ARG_BOOL] = cl_arg_parse_bool,
+	[CL_ARG_INT] = cl_arg_parse_int,
+	[CL_ARG_STR] = cl_arg_parse_str,
+	[CL_ARG_STR_SET] = cl_arg_parse_str_set,
+	[CL_ARG_IPV4] = cl_arg_parse_ipv4,
+};
+
+static struct cl_arg *find_short_arg(char name, struct cl_arg *args)
+{
+	struct cl_arg *arg;
+
+	for (arg = args; arg->short_name != 0; arg++) {
+		if (arg->short_name == name)
+			return arg;
+	}
+
+	return NULL;
+}
+
+static struct cl_arg *find_long_arg(const char *name, struct cl_arg *args)
+{
+	struct cl_arg *arg;
+
+	for (arg = args; arg->long_name; arg++) {
+		if (strcmp(arg->long_name, name) == 0)
+			return arg;
+	}
+
+	return NULL;
+}
+
+static void print_help(struct cl_arg *args)
+{
+	struct cl_arg *arg;
+
+	fprintf(stderr, "usage:\n");
+	for (arg = args; arg->long_name; arg++) {
+		fprintf(stderr, "-%c, --%-20s %s", arg->short_name,
+			arg->long_name, arg->help);
+		if (arg->type == CL_ARG_STR_SET) {
+			const char **set = arg->set;
+
+			fprintf(stderr, " [ ");
+			while (*set != NULL)
+				fprintf(stderr, "%s ", *(set++));
+			fprintf(stderr, "]");
+		}
+		fprintf(stderr, "\n");
+	}
+}
+
+int cla_parse_args(int argc, const char **argv, struct cl_arg *args)
+{
+	int i;
+
+	for (i = 1; i < argc; i++) {
+		struct cl_arg *arg = NULL;
+		cl_arg_parser_t parser;
+
+		if (argv[i][0] == '-') {
+			if (argv[i][1] != '-')
+				arg = find_short_arg(argv[i][1], args);
+			else
+				arg = find_long_arg(&argv[i][2], args);
+		}
+
+		if (!arg) {
+			fprintf(stderr, "unknown option '%s'\n", argv[i]);
+			print_help(args);
+			return (-1);
+		}
+
+		if (arg->type == CL_ARG_USER || arg->type >= CL_ARG_END)
+			parser = arg->parser;
+		else
+			parser = parsers[arg->type];
+
+		if (!parser) {
+			fprintf(stderr, "can't parse --'%s'/-'%c'\n",
+				arg->long_name, args->short_name);
+			return (-1);
+		}
+
+		if (parser(arg, argv[i + 1]) < 0) {
+			fprintf(stderr, "can't parse '%s'\n", argv[i]);
+			print_help(args);
+			return (-1);
+		}
+
+		if (arg->has_arg)
+			i++;
+	}
+
+	return 0;
+}
diff --git a/tools/testing/selftests/um/cla.h b/tools/testing/selftests/um/cla.h
new file mode 100644
index 000000000000..3d879233681f
--- /dev/null
+++ b/tools/testing/selftests/um/cla.h
@@ -0,0 +1,33 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef _LKL_TEST_CLA_H
+#define _LKL_TEST_CLA_H
+
+enum cl_arg_type {
+	CL_ARG_USER = 0,
+	CL_ARG_BOOL,
+	CL_ARG_INT,
+	CL_ARG_STR,
+	CL_ARG_STR_SET,
+	CL_ARG_IPV4,
+	CL_ARG_END,
+};
+
+struct cl_arg;
+
+typedef int (*cl_arg_parser_t)(struct cl_arg *arg, const char *value);
+
+struct cl_arg {
+	const char *long_name;
+	char short_name;
+	const char *help;
+	int has_arg;
+	enum cl_arg_type type;
+	void *store;
+	void *set;
+	cl_arg_parser_t parser;
+};
+
+int cla_parse_args(int argc, const char **argv, struct cl_arg *args);
+
+
+#endif /* _LKL_TEST_CLA_H */
diff --git a/tools/testing/selftests/um/disk-ext4.sh b/tools/testing/selftests/um/disk-ext4.sh
new file mode 100755
index 000000000000..a021f0efd057
--- /dev/null
+++ b/tools/testing/selftests/um/disk-ext4.sh
@@ -0,0 +1,6 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+
+${script_dir}/disk.sh -t ext4
diff --git a/tools/testing/selftests/um/disk-vfat.sh b/tools/testing/selftests/um/disk-vfat.sh
new file mode 100755
index 000000000000..955ccec40193
--- /dev/null
+++ b/tools/testing/selftests/um/disk-vfat.sh
@@ -0,0 +1,6 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+
+${script_dir}/disk.sh -t vfat
diff --git a/tools/testing/selftests/um/disk.c b/tools/testing/selftests/um/disk.c
new file mode 100644
index 000000000000..d70468e66170
--- /dev/null
+++ b/tools/testing/selftests/um/disk.c
@@ -0,0 +1,166 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <unistd.h>
+#include <string.h>
+#include <time.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <lkl.h>
+#include <lkl_host.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <sys/ioctl.h>
+
+#include "test.h"
+#include "cla.h"
+
+static struct {
+	int printk;
+	const char *disk;
+	const char *fstype;
+	int partition;
+} cla;
+
+struct cl_arg args[] = {
+	{"disk", 'd', "disk file to use", 1, CL_ARG_STR, &cla.disk},
+	{"partition", 'P', "partition to mount", 1, CL_ARG_INT, &cla.partition},
+	{"type", 't', "filesystem type", 1, CL_ARG_STR, &cla.fstype},
+	{0},
+};
+
+
+static struct lkl_disk disk;
+static int disk_id = -1;
+
+int lkl_test_disk_add(void)
+{
+	disk.fd = open(cla.disk, O_RDWR);
+	if (disk.fd < 0)
+		goto out_unlink;
+
+	disk.ops = NULL;
+	disk.dev = (char *)cla.disk;
+
+	disk_id = lkl_disk_add(&disk);
+	if (disk_id < 0)
+		goto out_close;
+
+	goto out;
+
+out_close:
+	close(disk.fd);
+
+out_unlink:
+	unlink(cla.disk);
+
+out:
+	lkl_test_logf("disk fd/handle %x disk_id %d", disk.fd, disk_id);
+
+	if (disk_id >= 0)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+int lkl_test_disk_remove(void)
+{
+	int ret;
+
+	ret = lkl_disk_remove(disk);
+
+	close(disk.fd);
+
+	if (ret == 0)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+
+static char mnt_point[32];
+
+LKL_TEST_CALL(mount_dev, lkl_mount_dev, 0, disk_id, cla.partition, cla.fstype,
+	      0, NULL, mnt_point, sizeof(mnt_point))
+
+static int lkl_test_umount_dev(void)
+{
+	long ret, ret2;
+
+	ret = lkl_sys_chdir("/");
+
+	ret2 = lkl_umount_dev(disk_id, cla.partition, 0, 1000);
+
+	lkl_test_logf("%ld %ld", ret, ret2);
+
+	if (!ret && !ret2)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+struct lkl_dir *dir;
+
+static int lkl_test_opendir(void)
+{
+	int err;
+
+	dir = lkl_opendir(mnt_point, &err);
+
+	lkl_test_logf("lkl_opendir(%s) = %d %s\n", mnt_point, err,
+		      lkl_strerror(err));
+
+	if (err == 0)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+static int lkl_test_readdir(void)
+{
+	struct lkl_linux_dirent64 *de = lkl_readdir(dir);
+	int wr = 0;
+
+	while (de) {
+		wr += lkl_test_logf("%s ", de->d_name);
+		if (wr >= 70) {
+			lkl_test_logf("\n");
+			wr = 0;
+			break;
+		}
+		de = lkl_readdir(dir);
+	}
+
+	if (lkl_errdir(dir) == 0)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+LKL_TEST_CALL(closedir, lkl_closedir, 0, dir);
+LKL_TEST_CALL(chdir_mnt_point, lkl_sys_chdir, 0, mnt_point);
+LKL_TEST_CALL(start_kernel, lkl_start_kernel, 0, &lkl_host_ops,
+	     "mem=16M loglevel=8");
+LKL_TEST_CALL(stop_kernel, lkl_sys_halt, 0);
+
+struct lkl_test tests[] = {
+	LKL_TEST(disk_add),
+	LKL_TEST(start_kernel),
+	LKL_TEST(mount_dev),
+	LKL_TEST(chdir_mnt_point),
+	LKL_TEST(opendir),
+	LKL_TEST(readdir),
+	LKL_TEST(closedir),
+	LKL_TEST(umount_dev),
+	LKL_TEST(stop_kernel),
+	LKL_TEST(disk_remove),
+
+};
+
+int main(int argc, const char **argv)
+{
+	if (cla_parse_args(argc, argv, args) < 0)
+		return (-1);
+
+	return lkl_test_run(tests, sizeof(tests)/sizeof(struct lkl_test),
+			    "disk %s", cla.fstype);
+}
diff --git a/tools/testing/selftests/um/disk.sh b/tools/testing/selftests/um/disk.sh
new file mode 100755
index 000000000000..9cf67a76bbe3
--- /dev/null
+++ b/tools/testing/selftests/um/disk.sh
@@ -0,0 +1,67 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+
+source $script_dir/test.sh
+
+function prepfs()
+{
+    set -e
+
+    file=`mktemp`
+
+    dd if=/dev/zero of=$file bs=1024 count=204800
+
+    yes | mkfs.$1 $file
+
+    if ! [ -z $ANDROID_WDIR ]; then
+        adb shell mkdir -p $ANDROID_WDIR
+        adb push $file $ANDROID_WDIR
+        rm $file
+        file=$ANDROID_WDIR/$(basename $file)
+    fi
+    if ! [ -z $BSD_WDIR ]; then
+        $MYSSH mkdir -p $BSD_WDIR
+        ssh_copy $file $BSD_WDIR
+        rm $file
+        file=$BSD_WDIR/$(basename $file)
+    fi
+
+    export_vars file
+}
+
+function cleanfs()
+{
+    set -e
+
+    if ! [ -z $ANDROID_WDIR ]; then
+        adb shell rm $1
+        adb shell rm $ANDROID_WDIR/disk
+    elif ! [ -z $BSD_WDIR ]; then
+        $MYSSH rm $1
+        $MYSSH rm $BSD_WDIR/disk
+    else
+        rm $1
+    fi
+}
+
+if [ "$1" = "-t" ]; then
+    shift
+    fstype=$1
+    shift
+fi
+
+if [ -z "$fstype" ]; then
+    fstype="ext4"
+fi
+
+if [ -z $(which mkfs.$fstype) ]; then
+    echo "no mkfs.$fstype command"
+    exit 0
+fi
+
+prepfs $fstype
+${OUTPUT}/disk -d $file -t $fstype $@
+cleanfs $file
+
diff --git a/tools/um/include/lkl.h b/tools/um/include/lkl.h
index 97da2d11502f..1cfa228880d4 100644
--- a/tools/um/include/lkl.h
+++ b/tools/um/include/lkl.h
@@ -120,6 +120,16 @@ static inline long lkl_sys_creat(const char *file, int mode)
 }
 #endif
 
+#ifdef __lkl__NR_faccessat
+/**
+ * lkl_sys_access - wrapper for lkl_sys_faccessat
+ */
+static inline long lkl_sys_access(const char *file, int mode)
+{
+	return lkl_sys_faccessat(LKL_AT_FDCWD, file, mode);
+}
+#endif
+
 #ifdef __lkl__NR_mkdirat
 /**
  * lkl_sys_mkdir - wrapper for lkl_sys_mkdirat
@@ -130,6 +140,36 @@ static inline long lkl_sys_mkdir(const char *path, mode_t mode)
 }
 #endif
 
+#ifdef __lkl__NR_unlinkat
+/**
+ * lkl_sys_rmdir - wrapper for lkl_sys_unlinkrat
+ */
+static inline long lkl_sys_rmdir(const char *path)
+{
+	return lkl_sys_unlinkat(LKL_AT_FDCWD, path, LKL_AT_REMOVEDIR);
+}
+#endif
+
+#ifdef __lkl__NR_unlinkat
+/**
+ * lkl_sys_unlink - wrapper for lkl_sys_unlinkat
+ */
+static inline long lkl_sys_unlink(const char *path)
+{
+	return lkl_sys_unlinkat(LKL_AT_FDCWD, path, 0);
+}
+#endif
+
+#ifdef __lkl__NR_mknodat
+/**
+ * lkl_sys_mknod - wrapper for lkl_sys_mknodat
+ */
+static inline long lkl_sys_mknod(const char *path, mode_t mode, dev_t dev)
+{
+	return lkl_sys_mknodat(LKL_AT_FDCWD, path, mode, dev);
+}
+#endif
+
 #ifdef __lkl__NR_epoll_create1
 /**
  * lkl_sys_epoll_create - wrapper for lkl_sys_epoll_create1
@@ -151,6 +191,172 @@ static inline long lkl_sys_epoll_wait(int fd, struct lkl_epoll_event *ev,
 }
 #endif
 
+/**
+ * struct lkl_dev_blk_ops - block device host operations, defined in lkl_host.h.
+ */
+struct lkl_dev_blk_ops;
+
+/**
+ * lkl_disk - host disk handle
+ *
+ * @dev - a pointer to private information for this disk backend
+ * @fd - a POSIX file descriptor that can be used by preadv/pwritev
+ * @handle - an NT file handle that can be used by ReadFile/WriteFile
+ */
+struct lkl_disk {
+	void *dev;
+	union {
+		int fd;
+		void *handle;
+	};
+	struct lkl_dev_blk_ops *ops;
+};
+
+/**
+ * lkl_disk_add - add a new disk
+ *
+ * @disk - the host disk handle
+ * @returns a disk id (0 is valid) or a strictly negative value in case of error
+ */
+int lkl_disk_add(struct lkl_disk *disk);
+
+/**
+ * lkl_disk_remove - remove a disk
+ *
+ * This function makes a cleanup of the @disk's private information
+ * that was initialized by lkl_disk_add before.
+ *
+ * @disk - the host disk handle
+ */
+int lkl_disk_remove(struct lkl_disk disk);
+
+/**
+ * lkl_encode_dev_from_sysfs_blkdev - extract device id from sysfs
+ *
+ * This function returns the device id for the given sysfs dev node.
+ * The content of the node has to be in the form 'MAJOR:MINOR'.
+ * Also, this function expects an absolute path which means that sysfs
+ * already has to be mounted at the given path
+ *
+ * @sysfs_path - absolute path to the sysfs dev node
+ * @pdevid - pointer to memory where dev id will be returned
+ * @returns - 0 on success, a negative value on error
+ */
+int lkl_encode_dev_from_sysfs(const char *sysfs_path, uint32_t *pdevid);
+
+/**
+ * lkl_mount_dev - mount a disk
+ *
+ * This functions creates a device file for the given disk, creates a mount
+ * point and mounts the device over the mount point.
+ *
+ * @disk_id - the disk id identifying the disk to be mounted
+ * @part - disk partition or zero for full disk
+ * @fs_type - filesystem type
+ * @flags - mount flags
+ * @opts - additional filesystem specific mount options
+ * @mnt_str - a string that will be filled by this function with the path where
+ * the filesystem has been mounted
+ * @mnt_str_len - size of mnt_str
+ * @returns - 0 on success, a negative value on error
+ */
+long lkl_mount_dev(unsigned int disk_id, unsigned int part, const char *fs_type,
+		   int flags, const char *opts,
+		   char *mnt_str, unsigned int mnt_str_len);
+
+/**
+ * lkl_umount_dev - umount a disk
+ *
+ * This functions umounts the given disks and removes the device file and the
+ * mount point.
+ *
+ * @disk_id - the disk id identifying the disk to be mounted
+ * @part - disk partition or zero for full disk
+ * @flags - umount flags
+ * @timeout_ms - timeout to wait for the kernel to flush closed files so that
+ * umount can succeed
+ * @returns - 0 on success, a negative value on error
+ */
+long lkl_umount_dev(unsigned int disk_id, unsigned int part, int flags,
+		    long timeout_ms);
+
+/**
+ * lkl_umount_timeout - umount filesystem with timeout
+ *
+ * @path - the path to unmount
+ * @flags - umount flags
+ * @timeout_ms - timeout to wait for the kernel to flush closed files so that
+ * umount can succeed
+ * @returns - 0 on success, a negative value on error
+ */
+long lkl_umount_timeout(char *path, int flags, long timeout_ms);
+
+/**
+ * lkl_opendir - open a directory
+ *
+ * @path - directory path
+ * @err - pointer to store the error in case of failure
+ * @returns - a handle to be used when calling lkl_readdir
+ */
+struct lkl_dir *lkl_opendir(const char *path, int *err);
+
+/**
+ * lkl_fdopendir - open a directory
+ *
+ * @fd - file descriptor
+ * @err - pointer to store the error in case of failure
+ * @returns - a handle to be used when calling lkl_readdir
+ */
+struct lkl_dir *lkl_fdopendir(int fd, int *err);
+
+/**
+ * lkl_rewinddir - reset directory stream
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ */
+void lkl_rewinddir(struct lkl_dir *dir);
+
+/**
+ * lkl_closedir - close the directory
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ */
+int lkl_closedir(struct lkl_dir *dir);
+
+/**
+ * lkl_readdir - get the next available entry of the directory
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ * @returns - a lkl_dirent64 entry or NULL if the end of the directory stream is
+ * reached or if an error occurred; check lkl_errdir() to distinguish between
+ * errors or end of the directory stream
+ */
+struct lkl_linux_dirent64 *lkl_readdir(struct lkl_dir *dir);
+
+/**
+ * lkl_errdir - checks if an error occurred during the last lkl_readdir call
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ * @returns - 0 if no error occurred, or a negative value otherwise
+ */
+int lkl_errdir(struct lkl_dir *dir);
+
+/**
+ * lkl_dirfd - gets the file descriptor associated with the directory handle
+ *
+ * @dir - the directory handle as returned by lkl_opendir
+ * @returns - a positive value,which is the LKL file descriptor associated with
+ * the directory handle, or a negative value otherwise
+ */
+int lkl_dirfd(struct lkl_dir *dir);
+
+/**
+ * lkl_mount_fs - mount a file system type like proc, sys
+ * @fstype - file system type. e.g. proc, sys
+ * @returns - 0 on success. 1 if it's already mounted. negative on failure.
+ */
+int lkl_mount_fs(char *fstype);
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/tools/um/include/lkl_host.h b/tools/um/include/lkl_host.h
index 5fe7a64dc4dd..1a2794ef47a0 100644
--- a/tools/um/include/lkl_host.h
+++ b/tools/um/include/lkl_host.h
@@ -10,6 +10,7 @@ extern "C" {
 #include <lkl.h>
 
 extern struct lkl_host_operations lkl_host_ops;
+extern char lkl_um_devs[4096];
 
 #ifdef __cplusplus
 }
diff --git a/tools/um/lib/Build b/tools/um/lib/Build
index dddff26a3b4e..29491b40746c 100644
--- a/tools/um/lib/Build
+++ b/tools/um/lib/Build
@@ -4,3 +4,4 @@ CFLAGS_posix-host.o += -D_FILE_OFFSET_BITS=64
 liblinux-$(CONFIG_UMMODE_LIB) += utils.o
 liblinux-$(CONFIG_UMMODE_LIB) += posix-host.o
 liblinux-$(CONFIG_UMMODE_LIB) += jmp_buf.o
+liblinux-$(CONFIG_UMMODE_LIB) += fs.o
diff --git a/tools/um/lib/fs.c b/tools/um/lib/fs.c
new file mode 100644
index 000000000000..352b25f0c56f
--- /dev/null
+++ b/tools/um/lib/fs.c
@@ -0,0 +1,461 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdarg.h>
+#include <stdio.h>
+#include <string.h>
+#include <stdlib.h>
+#include <lkl_host.h>
+
+#define MAX_FSTYPE_LEN 50
+
+static struct lkl_disk *lkl_disks[16];
+static int registered_blk_dev_idx;
+
+static int lkl_disk_um_add(struct lkl_disk *disk, const char *blkparams)
+{
+	/* concat strings */
+	snprintf(lkl_um_devs + strlen(lkl_um_devs), sizeof(lkl_um_devs),
+		 " ubd%d=%s", registered_blk_dev_idx, blkparams);
+
+	return registered_blk_dev_idx++;
+}
+
+int lkl_disk_add(struct lkl_disk *disk)
+{
+	int ret = -1;
+
+	ret = lkl_disk_um_add(disk, disk->dev);
+
+	lkl_disks[ret] = disk;
+
+	return ret;
+}
+
+int lkl_disk_remove(struct lkl_disk disk)
+{
+	/* FIXME */
+	return 0;
+}
+
+int lkl_mount_fs(char *fstype)
+{
+	char dir[MAX_FSTYPE_LEN+2] = "/";
+	int flags = 0, ret = 0;
+
+	strncat(dir, fstype, MAX_FSTYPE_LEN);
+
+	/* Create with regular umask */
+	ret = lkl_sys_mkdir(dir, 0xff);
+	if (ret && ret != -LKL_EEXIST) {
+		lkl_perror("mount_fs mkdir", ret);
+		return ret;
+	}
+
+	/* We have no use for nonzero flags right now */
+	ret = lkl_sys_mount("none", dir, fstype, flags, NULL);
+	if (ret && ret != -LKL_EBUSY) {
+		lkl_sys_rmdir(dir);
+		return ret;
+	}
+
+	if (ret == -LKL_EBUSY)
+		return 1;
+	return 0;
+}
+
+static uint32_t new_encode_dev(unsigned int major, unsigned int minor)
+{
+	return (minor & 0xff) | (major << 8) | ((minor & ~0xff) << 12);
+}
+
+static int startswith(const char *str, const char *pre)
+{
+	return strncmp(pre, str, strlen(pre)) == 0;
+}
+
+static int get_node_with_prefix(const char *path, const char *prefix,
+				char *result, unsigned int result_len)
+{
+	struct lkl_dir *dir = NULL;
+	struct lkl_linux_dirent64 *dirent;
+	int ret;
+
+	dir = lkl_opendir(path, &ret);
+	if (!dir)
+		return ret;
+
+	ret = -LKL_ENOENT;
+
+	while ((dirent = lkl_readdir(dir))) {
+		if (startswith(dirent->d_name, prefix)) {
+			if (strlen(dirent->d_name) + 1 > result_len) {
+				ret = -LKL_ENOMEM;
+				break;
+			}
+			memcpy(result, dirent->d_name, strlen(dirent->d_name));
+			result[strlen(dirent->d_name)] = '\0';
+			ret = 0;
+			break;
+		}
+	}
+
+	lkl_closedir(dir);
+
+	return ret;
+}
+
+int lkl_encode_dev_from_sysfs(const char *sysfs_path, uint32_t *pdevid)
+{
+	int ret;
+	long fd;
+	int major, minor;
+	char buf[16] = { 0, };
+	char *bufptr;
+
+	fd = lkl_sys_open(sysfs_path, LKL_O_RDONLY, 0);
+	if (fd < 0)
+		return fd;
+
+	ret = lkl_sys_read(fd, buf, sizeof(buf));
+	if (ret < 0)
+		goto out_close;
+
+	if (ret == sizeof(buf)) {
+		ret = -LKL_ENOBUFS;
+		goto out_close;
+	}
+
+	bufptr = strchr(buf, ':');
+	if (bufptr == NULL) {
+		ret = -LKL_EINVAL;
+		goto out_close;
+	}
+	bufptr[0] = '\0';
+	bufptr++;
+
+	major = atoi(buf);
+	minor = atoi(bufptr);
+
+	*pdevid = new_encode_dev(major, minor);
+	ret = 0;
+
+out_close:
+	lkl_sys_close(fd);
+
+	return ret;
+}
+
+#define SYSFS_DEV_UMBLK_CMDLINE_PATH \
+	"/sysfs/devices/platform/uml-blkdev.%d"
+
+struct abuf {
+	char *mem, *ptr;
+	unsigned int len;
+};
+
+static int snprintf_append(struct abuf *buf, const char *fmt, ...)
+{
+	int ret;
+	va_list args;
+
+	if (!buf->ptr)
+		buf->ptr = buf->mem;
+
+	va_start(args, fmt);
+	ret = vsnprintf(buf->ptr, buf->len - (buf->ptr - buf->mem), fmt, args);
+	va_end(args);
+
+	if (ret < 0 || (ret >= (int)(buf->len - (buf->ptr - buf->mem))))
+		return -LKL_ENOMEM;
+
+	buf->ptr += ret;
+
+	return 0;
+}
+
+static int __lkl_get_blkdev(int disk_id, unsigned int part, uint32_t *pdevid,
+			    const char *sysfs_path_fmt, const char *drv_prefix,
+			    const char *disk_prefix)
+{
+	char sysfs_path[LKL_PATH_MAX];
+	char drv_name[LKL_PATH_MAX];
+	char disk_name[LKL_PATH_MAX];
+	struct abuf sysfs_path_buf = {
+		.mem = sysfs_path,
+		.len = sizeof(sysfs_path),
+	};
+	int ret;
+
+	if (disk_id < 0)
+		return -LKL_EINVAL;
+
+	ret = lkl_mount_fs("sysfs");
+	if (ret < 0)
+		return ret;
+
+	ret = snprintf_append(&sysfs_path_buf, sysfs_path_fmt, disk_id);
+	if (ret)
+		return ret;
+
+	ret = get_node_with_prefix(sysfs_path, drv_prefix, drv_name,
+				   sizeof(drv_name));
+	if (ret)
+		return ret;
+
+	ret = snprintf_append(&sysfs_path_buf, "/%s/block", drv_name);
+	if (ret)
+		return ret;
+
+	ret = get_node_with_prefix(sysfs_path, disk_prefix, disk_name,
+				   sizeof(disk_name));
+	if (ret)
+		return ret;
+
+	if (!part)
+		ret = snprintf_append(&sysfs_path_buf, "/%s/dev", disk_name);
+	else
+		ret = snprintf_append(&sysfs_path_buf, "/%s/%s%d/dev",
+				      disk_name, disk_name, part);
+	if (ret)
+		return ret;
+
+	return lkl_encode_dev_from_sysfs(sysfs_path, pdevid);
+}
+
+int lkl_get_blkdev(int disk_id, unsigned int part, uint32_t *pdevid)
+{
+	char *fmt;
+
+	fmt = SYSFS_DEV_UMBLK_CMDLINE_PATH;
+	return __lkl_get_blkdev(disk_id, part, pdevid, fmt, "", "ubd");
+}
+
+long lkl_mount_dev(unsigned int disk_id, unsigned int part,
+		   const char *fs_type, int flags,
+		   const char *data, char *mnt_str, unsigned int mnt_str_len)
+{
+	char dev_str[] = { "/dev/xxxxxxxx" };
+	unsigned int dev;
+	int err;
+	char _data[4096]; /* FIXME: PAGE_SIZE is not exported by LKL */
+
+	if (mnt_str_len < sizeof(dev_str))
+		return -LKL_ENOMEM;
+
+	err = lkl_get_blkdev(disk_id, part, &dev);
+	if (err < 0)
+		return err;
+
+	snprintf(dev_str, sizeof(dev_str), "/dev/%08x", dev);
+	snprintf(mnt_str, mnt_str_len, "/mnt/%08x", dev);
+
+	err = lkl_sys_access("/dev", LKL_S_IRWXO);
+	if (err < 0) {
+		if (err == -LKL_ENOENT)
+			err = lkl_sys_mkdir("/dev", 0700);
+		if (err < 0)
+			return err;
+	}
+
+	err = lkl_sys_mknod(dev_str, LKL_S_IFBLK | 0600, dev);
+	if (err < 0)
+		return err;
+
+	err = lkl_sys_access("/mnt", LKL_S_IRWXO);
+	if (err < 0) {
+		if (err == -LKL_ENOENT)
+			err = lkl_sys_mkdir("/mnt", 0700);
+		if (err < 0)
+			return err;
+	}
+
+	err = lkl_sys_mkdir(mnt_str, 0700);
+	if (err < 0) {
+		lkl_sys_unlink(dev_str);
+		return err;
+	}
+
+	/* kernel always copies a full page */
+	if (data) {
+		strncpy(_data, data, sizeof(_data));
+		_data[sizeof(_data) - 1] = 0;
+	} else {
+		_data[0] = 0;
+	}
+
+	err = lkl_sys_mount(dev_str, mnt_str, (char *)fs_type, flags, _data);
+	if (err < 0) {
+		lkl_sys_unlink(dev_str);
+		lkl_sys_rmdir(mnt_str);
+		return err;
+	}
+
+	return 0;
+}
+
+long lkl_umount_timeout(char *path, int flags, long timeout_ms)
+{
+	long incr = 10000000; /* 10 ms */
+	struct lkl_timespec ts = {
+		.tv_sec = 0,
+		.tv_nsec = incr,
+	};
+	long err;
+
+	do {
+		err = lkl_sys_umount(path, flags);
+		if (err == -LKL_EBUSY) {
+			lkl_sys_nanosleep((struct __lkl__kernel_timespec *)&ts,
+					  NULL);
+			timeout_ms -= incr / 1000000;
+		}
+	} while (err == -LKL_EBUSY && timeout_ms > 0);
+
+	return err;
+}
+
+long lkl_umount_dev(unsigned int disk_id, unsigned int part, int flags,
+		    long timeout_ms)
+{
+	char dev_str[] = { "/dev/xxxxxxxx" };
+	char mnt_str[] = { "/mnt/xxxxxxxx" };
+	unsigned int dev;
+	int err;
+
+	err = lkl_get_blkdev(disk_id, part, &dev);
+	if (err < 0)
+		return err;
+
+	snprintf(dev_str, sizeof(dev_str), "/dev/%08x", dev);
+	snprintf(mnt_str, sizeof(mnt_str), "/mnt/%08x", dev);
+
+	err = lkl_umount_timeout(mnt_str, flags, timeout_ms);
+	if (err)
+		return err;
+
+	err = lkl_sys_unlink(dev_str);
+	if (err)
+		return err;
+
+	return lkl_sys_rmdir(mnt_str);
+}
+
+struct lkl_dir {
+	int fd;
+	char buf[1024];
+	char *pos;
+	int len;
+};
+
+static struct lkl_dir *lkl_dir_alloc(int *err)
+{
+	struct lkl_dir *dir = lkl_mem_alloc(sizeof(struct lkl_dir));
+
+	if (!dir) {
+		*err = -LKL_ENOMEM;
+		return NULL;
+	}
+
+	dir->len = 0;
+	dir->pos = NULL;
+
+	return dir;
+}
+
+struct lkl_dir *lkl_opendir(const char *path, int *err)
+{
+	struct lkl_dir *dir = lkl_dir_alloc(err);
+
+	if (!dir) {
+		*err = -LKL_ENOMEM;
+		return NULL;
+	}
+
+	dir->fd = lkl_sys_open(path, LKL_O_RDONLY | LKL_O_DIRECTORY, 0);
+	if (dir->fd < 0) {
+		*err = dir->fd;
+		lkl_mem_free(dir);
+		return NULL;
+	}
+
+	*err = 0;
+
+	return dir;
+}
+
+struct lkl_dir *lkl_fdopendir(int fd, int *err)
+{
+	struct lkl_dir *dir = lkl_dir_alloc(err);
+
+	if (!dir)
+		return NULL;
+
+	dir->fd = fd;
+
+	return dir;
+}
+
+void lkl_rewinddir(struct lkl_dir *dir)
+{
+	lkl_sys_lseek(dir->fd, 0, LKL_SEEK_SET);
+	dir->len = 0;
+	dir->pos = NULL;
+}
+
+int lkl_closedir(struct lkl_dir *dir)
+{
+	int ret;
+
+	ret = lkl_sys_close(dir->fd);
+	lkl_mem_free(dir);
+
+	return ret;
+}
+
+struct lkl_linux_dirent64 *lkl_readdir(struct lkl_dir *dir)
+{
+	struct lkl_linux_dirent64 *de;
+
+	if (dir->len < 0)
+		return NULL;
+
+	if (!dir->pos || dir->pos - dir->buf >= dir->len)
+		goto read_buf;
+
+return_de:
+	de = (struct lkl_linux_dirent64 *)dir->pos;
+	dir->pos += de->d_reclen;
+
+	return de;
+
+read_buf:
+	dir->pos = NULL;
+	de = (struct lkl_linux_dirent64 *)dir->buf;
+	dir->len = lkl_sys_getdents64(dir->fd, de, sizeof(dir->buf));
+	if (dir->len <= 0)
+		return NULL;
+
+	dir->pos = dir->buf;
+	goto return_de;
+}
+
+int lkl_errdir(struct lkl_dir *dir)
+{
+	if (dir->len >= 0)
+		return 0;
+
+	return dir->len;
+}
+
+int lkl_dirfd(struct lkl_dir *dir)
+{
+	return dir->fd;
+}
+
+int lkl_set_fd_limit(unsigned int fd_limit)
+{
+	struct lkl_rlimit rlim = {
+		.rlim_cur = fd_limit,
+		.rlim_max = fd_limit,
+	};
+	return lkl_sys_setrlimit(LKL_RLIMIT_NOFILE, &rlim);
+}
diff --git a/tools/um/lib/posix-host.c b/tools/um/lib/posix-host.c
index f7c170fd3fd9..5fc7d6822a7f 100644
--- a/tools/um/lib/posix-host.c
+++ b/tools/um/lib/posix-host.c
@@ -268,5 +268,6 @@ void *lkl_tls_get(struct lkl_tls_key *key)
 }
 
 struct lkl_host_operations lkl_host_ops = {
+	.um_devices = lkl_um_devs,
 };
 
diff --git a/tools/um/lib/utils.c b/tools/um/lib/utils.c
index 3b38e1a95124..03841f70f73e 100644
--- a/tools/um/lib/utils.c
+++ b/tools/um/lib/utils.c
@@ -4,6 +4,8 @@
 #include <string.h>
 #include <lkl_host.h>
 
+char lkl_um_devs[4096];
+
 static const char * const lkl_err_strings[] = {
 	[0]			= "Success",
 	[LKL_EPERM]		= "Operation not permitted",
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v8 19/20] um: lkl: add block device support of UML
@ 2021-01-20  2:27             ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-01-20  2:27 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, thehajime, retrage01

This commit adds block device support for library mode, and also added
several host utilities to run a simple test program
(tools/um/test/disk.c) successfully.

Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 arch/um/Kconfig                         |   4 +
 arch/um/include/asm/xor.h               |   3 +-
 arch/um/include/shared/as-layout.h      |   1 +
 arch/um/kernel/um_arch.c                |   5 +
 arch/um/lkl/include/uapi/asm/host_ops.h |   5 +
 arch/um/lkl/include/uapi/asm/unistd.h   |   2 +
 arch/um/lkl/um/setup.c                  |  12 +
 tools/testing/selftests/um/Makefile     |   5 +-
 tools/testing/selftests/um/cla.c        | 159 ++++++++
 tools/testing/selftests/um/cla.h        |  33 ++
 tools/testing/selftests/um/disk-ext4.sh |   6 +
 tools/testing/selftests/um/disk-vfat.sh |   6 +
 tools/testing/selftests/um/disk.c       | 166 +++++++++
 tools/testing/selftests/um/disk.sh      |  67 ++++
 tools/um/include/lkl.h                  | 206 +++++++++++
 tools/um/include/lkl_host.h             |   1 +
 tools/um/lib/Build                      |   1 +
 tools/um/lib/fs.c                       | 461 ++++++++++++++++++++++++
 tools/um/lib/posix-host.c               |   1 +
 tools/um/lib/utils.c                    |   2 +
 20 files changed, 1144 insertions(+), 2 deletions(-)
 create mode 100644 tools/testing/selftests/um/cla.c
 create mode 100644 tools/testing/selftests/um/cla.h
 create mode 100755 tools/testing/selftests/um/disk-ext4.sh
 create mode 100755 tools/testing/selftests/um/disk-vfat.sh
 create mode 100644 tools/testing/selftests/um/disk.c
 create mode 100755 tools/testing/selftests/um/disk.sh
 create mode 100644 tools/um/lib/fs.c

diff --git a/arch/um/Kconfig b/arch/um/Kconfig
index 24c6596260de..5fb6a852d058 100644
--- a/arch/um/Kconfig
+++ b/arch/um/Kconfig
@@ -29,6 +29,10 @@ config UMMODE_LIB
 	select UACCESS_MEMCPY
 	select ARCH_THREAD_STACK_ALLOCATOR
 	select ARCH_HAS_SYSCALL_WRAPPER
+	select VFAT_FS
+	select NLS_CODEPAGE_437
+	select NLS_ISO8859_1
+	select BTRFS_FS
 	help
 	 This mode switches a mode to build a library of UML (Linux
 	 Kernel Library/LKL).  If this is Y, the build only generates,
diff --git a/arch/um/include/asm/xor.h b/arch/um/include/asm/xor.h
index 36b33d62a35d..934a576ec4b2 100644
--- a/arch/um/include/asm/xor.h
+++ b/arch/um/include/asm/xor.h
@@ -4,4 +4,5 @@
 
 /* pick an arbitrary one - measuring isn't possible with inf-cpu */
 #define XOR_SELECT_TEMPLATE(x)	\
-	(time_travel_mode == TT_MODE_INFCPU ? &xor_block_8regs : NULL)
+	(time_travel_mode == TT_MODE_INFCPU || \
+	 (IS_ENABLED(CONFIG_UMMODE_LIB)) ? &xor_block_8regs : NULL)
diff --git a/arch/um/include/shared/as-layout.h b/arch/um/include/shared/as-layout.h
index 5f286ef2721b..4423437a5ace 100644
--- a/arch/um/include/shared/as-layout.h
+++ b/arch/um/include/shared/as-layout.h
@@ -57,6 +57,7 @@ extern unsigned long host_task_size;
 
 extern int linux_main(int argc, char **argv);
 extern void uml_finishsetup(void);
+extern void uml_set_args(char *args);
 
 struct siginfo;
 extern void (*sig_info[])(int, struct siginfo *si, struct uml_pt_regs *);
diff --git a/arch/um/kernel/um_arch.c b/arch/um/kernel/um_arch.c
index dfc6194b5ac7..3968842346a2 100644
--- a/arch/um/kernel/um_arch.c
+++ b/arch/um/kernel/um_arch.c
@@ -42,6 +42,11 @@ static void __init add_arg(char *arg)
 	strcat(command_line, arg);
 }
 
+void __init uml_set_args(char *args)
+{
+	strcat(command_line, args);
+}
+
 /*
  * These fields are initialized at boot time and not changed.
  * XXX This structure is used only in the non-SMP case.  Maybe this
diff --git a/arch/um/lkl/include/uapi/asm/host_ops.h b/arch/um/lkl/include/uapi/asm/host_ops.h
index 85d7d4790602..c7239df7f807 100644
--- a/arch/um/lkl/include/uapi/asm/host_ops.h
+++ b/arch/um/lkl/include/uapi/asm/host_ops.h
@@ -12,11 +12,16 @@ struct lkl_jmp_buf {
 /**
  * struct lkl_host_operations - host operations used by the Linux kernel
  *
+ * @um_devices: string containg the list of UML devices in command line
+ * format. This string is appended to the kernel command line and
+ * is provided here for convenience to be implemented by the host library.
+ *
  * These operations must be provided by a host library or by the application
  * itself.
  *
  */
 struct lkl_host_operations {
+	const char *um_devices;
 };
 
 /**
diff --git a/arch/um/lkl/include/uapi/asm/unistd.h b/arch/um/lkl/include/uapi/asm/unistd.h
index 6f5d67b45fbd..d777e4b597d2 100644
--- a/arch/um/lkl/include/uapi/asm/unistd.h
+++ b/arch/um/lkl/include/uapi/asm/unistd.h
@@ -3,6 +3,8 @@
 #define __UM_LIBMODE_UAPI_UNISTD_H
 
 #define __ARCH_WANT_NEW_STAT
+#define __ARCH_WANT_SET_GET_RLIMIT
+
 #include <asm/bitsperlong.h>
 
 #if __BITS_PER_LONG == 64
diff --git a/arch/um/lkl/um/setup.c b/arch/um/lkl/um/setup.c
index 12b235826573..f33e9f477b7a 100644
--- a/arch/um/lkl/um/setup.c
+++ b/arch/um/lkl/um/setup.c
@@ -45,13 +45,25 @@ static void __init *lkl_run_kernel(void *arg)
 	return NULL;
 }
 
+static char _cmd_line[COMMAND_LINE_SIZE];
 int __init lkl_start_kernel(struct lkl_host_operations *ops,
 			    const char *fmt, ...)
 {
+	va_list ap;
 	int ret;
 
 	lkl_ops = ops;
 
+	va_start(ap, fmt);
+	ret = vsnprintf(_cmd_line, COMMAND_LINE_SIZE, fmt, ap);
+	va_end(ap);
+
+	if (ops->um_devices)
+		strscpy(_cmd_line + ret, ops->um_devices,
+			COMMAND_LINE_SIZE - ret);
+
+	uml_set_args(_cmd_line);
+
 	init_sem = lkl_sem_alloc(0);
 	if (!init_sem)
 		return -ENOMEM;
diff --git a/tools/testing/selftests/um/Makefile b/tools/testing/selftests/um/Makefile
index 1b915f4bb751..df9561a0b3b4 100644
--- a/tools/testing/selftests/um/Makefile
+++ b/tools/testing/selftests/um/Makefile
@@ -4,10 +4,13 @@ top_srcdir = ../../../../
 top_objdir = $(OUTPUT)/../../
 
 CFLAGS := -fPIC -I$(top_objdir)/tools/um/include/ -I$(top_srcdir)/tools/um/include/ -g
-CFLAGS += test.c
+CFLAGS += test.c cla.c
 LDFLAGS := -pie
 LDLIBS := -L$(top_objdir)/tools/um -L$(top_srcdir)/tools/um -llinux -lpthread -lrt -lutil
 
 TEST_GEN_PROGS := boot
+TEST_PROGS := disk-ext4.sh disk-vfat.sh
+TEST_PROGS_EXTENDED := test.sh disk.sh
+TEST_GEN_FILES := disk
 
 include ../lib.mk
diff --git a/tools/testing/selftests/um/cla.c b/tools/testing/selftests/um/cla.c
new file mode 100644
index 000000000000..694e3a3822be
--- /dev/null
+++ b/tools/testing/selftests/um/cla.c
@@ -0,0 +1,159 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <string.h>
+#include <errno.h>
+#include <stdlib.h>
+#ifdef __MINGW32__
+#include <winsock2.h>
+#else
+#include <sys/socket.h>
+#include <netinet/in.h>
+#include <arpa/inet.h>
+#endif
+
+#include "cla.h"
+
+static int cl_arg_parse_bool(struct cl_arg *arg, const char *value)
+{
+	*((int *)arg->store) = 1;
+	return 0;
+}
+
+static int cl_arg_parse_str(struct cl_arg *arg, const char *value)
+{
+	*((const char **)arg->store) = value;
+	return 0;
+}
+
+static int cl_arg_parse_int(struct cl_arg *arg, const char *value)
+{
+	errno = 0;
+	*((int *)arg->store) = strtol(value, NULL, 0);
+	return errno == 0;
+}
+
+static int cl_arg_parse_str_set(struct cl_arg *arg, const char *value)
+{
+	const char **set = arg->set;
+	int i;
+
+	for (i = 0; set[i] != NULL; i++) {
+		if (strcmp(set[i], value) == 0) {
+			*((int *)arg->store) = i;
+			return 0;
+		}
+	}
+
+	return (-1);
+}
+
+static int cl_arg_parse_ipv4(struct cl_arg *arg, const char *value)
+{
+	unsigned int addr;
+
+	if (!value)
+		return (-1);
+
+	addr = inet_addr(value);
+	if (addr == INADDR_NONE)
+		return (-1);
+	*((unsigned int *)arg->store) = addr;
+	return 0;
+}
+
+static cl_arg_parser_t parsers[] = {
+	[CL_ARG_BOOL] = cl_arg_parse_bool,
+	[CL_ARG_INT] = cl_arg_parse_int,
+	[CL_ARG_STR] = cl_arg_parse_str,
+	[CL_ARG_STR_SET] = cl_arg_parse_str_set,
+	[CL_ARG_IPV4] = cl_arg_parse_ipv4,
+};
+
+static struct cl_arg *find_short_arg(char name, struct cl_arg *args)
+{
+	struct cl_arg *arg;
+
+	for (arg = args; arg->short_name != 0; arg++) {
+		if (arg->short_name == name)
+			return arg;
+	}
+
+	return NULL;
+}
+
+static struct cl_arg *find_long_arg(const char *name, struct cl_arg *args)
+{
+	struct cl_arg *arg;
+
+	for (arg = args; arg->long_name; arg++) {
+		if (strcmp(arg->long_name, name) == 0)
+			return arg;
+	}
+
+	return NULL;
+}
+
+static void print_help(struct cl_arg *args)
+{
+	struct cl_arg *arg;
+
+	fprintf(stderr, "usage:\n");
+	for (arg = args; arg->long_name; arg++) {
+		fprintf(stderr, "-%c, --%-20s %s", arg->short_name,
+			arg->long_name, arg->help);
+		if (arg->type == CL_ARG_STR_SET) {
+			const char **set = arg->set;
+
+			fprintf(stderr, " [ ");
+			while (*set != NULL)
+				fprintf(stderr, "%s ", *(set++));
+			fprintf(stderr, "]");
+		}
+		fprintf(stderr, "\n");
+	}
+}
+
+int cla_parse_args(int argc, const char **argv, struct cl_arg *args)
+{
+	int i;
+
+	for (i = 1; i < argc; i++) {
+		struct cl_arg *arg = NULL;
+		cl_arg_parser_t parser;
+
+		if (argv[i][0] == '-') {
+			if (argv[i][1] != '-')
+				arg = find_short_arg(argv[i][1], args);
+			else
+				arg = find_long_arg(&argv[i][2], args);
+		}
+
+		if (!arg) {
+			fprintf(stderr, "unknown option '%s'\n", argv[i]);
+			print_help(args);
+			return (-1);
+		}
+
+		if (arg->type == CL_ARG_USER || arg->type >= CL_ARG_END)
+			parser = arg->parser;
+		else
+			parser = parsers[arg->type];
+
+		if (!parser) {
+			fprintf(stderr, "can't parse --'%s'/-'%c'\n",
+				arg->long_name, args->short_name);
+			return (-1);
+		}
+
+		if (parser(arg, argv[i + 1]) < 0) {
+			fprintf(stderr, "can't parse '%s'\n", argv[i]);
+			print_help(args);
+			return (-1);
+		}
+
+		if (arg->has_arg)
+			i++;
+	}
+
+	return 0;
+}
diff --git a/tools/testing/selftests/um/cla.h b/tools/testing/selftests/um/cla.h
new file mode 100644
index 000000000000..3d879233681f
--- /dev/null
+++ b/tools/testing/selftests/um/cla.h
@@ -0,0 +1,33 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef _LKL_TEST_CLA_H
+#define _LKL_TEST_CLA_H
+
+enum cl_arg_type {
+	CL_ARG_USER = 0,
+	CL_ARG_BOOL,
+	CL_ARG_INT,
+	CL_ARG_STR,
+	CL_ARG_STR_SET,
+	CL_ARG_IPV4,
+	CL_ARG_END,
+};
+
+struct cl_arg;
+
+typedef int (*cl_arg_parser_t)(struct cl_arg *arg, const char *value);
+
+struct cl_arg {
+	const char *long_name;
+	char short_name;
+	const char *help;
+	int has_arg;
+	enum cl_arg_type type;
+	void *store;
+	void *set;
+	cl_arg_parser_t parser;
+};
+
+int cla_parse_args(int argc, const char **argv, struct cl_arg *args);
+
+
+#endif /* _LKL_TEST_CLA_H */
diff --git a/tools/testing/selftests/um/disk-ext4.sh b/tools/testing/selftests/um/disk-ext4.sh
new file mode 100755
index 000000000000..a021f0efd057
--- /dev/null
+++ b/tools/testing/selftests/um/disk-ext4.sh
@@ -0,0 +1,6 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+
+${script_dir}/disk.sh -t ext4
diff --git a/tools/testing/selftests/um/disk-vfat.sh b/tools/testing/selftests/um/disk-vfat.sh
new file mode 100755
index 000000000000..955ccec40193
--- /dev/null
+++ b/tools/testing/selftests/um/disk-vfat.sh
@@ -0,0 +1,6 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+
+${script_dir}/disk.sh -t vfat
diff --git a/tools/testing/selftests/um/disk.c b/tools/testing/selftests/um/disk.c
new file mode 100644
index 000000000000..d70468e66170
--- /dev/null
+++ b/tools/testing/selftests/um/disk.c
@@ -0,0 +1,166 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdio.h>
+#include <unistd.h>
+#include <string.h>
+#include <time.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <lkl.h>
+#include <lkl_host.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <sys/ioctl.h>
+
+#include "test.h"
+#include "cla.h"
+
+static struct {
+	int printk;
+	const char *disk;
+	const char *fstype;
+	int partition;
+} cla;
+
+struct cl_arg args[] = {
+	{"disk", 'd', "disk file to use", 1, CL_ARG_STR, &cla.disk},
+	{"partition", 'P', "partition to mount", 1, CL_ARG_INT, &cla.partition},
+	{"type", 't', "filesystem type", 1, CL_ARG_STR, &cla.fstype},
+	{0},
+};
+
+
+static struct lkl_disk disk;
+static int disk_id = -1;
+
+int lkl_test_disk_add(void)
+{
+	disk.fd = open(cla.disk, O_RDWR);
+	if (disk.fd < 0)
+		goto out_unlink;
+
+	disk.ops = NULL;
+	disk.dev = (char *)cla.disk;
+
+	disk_id = lkl_disk_add(&disk);
+	if (disk_id < 0)
+		goto out_close;
+
+	goto out;
+
+out_close:
+	close(disk.fd);
+
+out_unlink:
+	unlink(cla.disk);
+
+out:
+	lkl_test_logf("disk fd/handle %x disk_id %d", disk.fd, disk_id);
+
+	if (disk_id >= 0)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+int lkl_test_disk_remove(void)
+{
+	int ret;
+
+	ret = lkl_disk_remove(disk);
+
+	close(disk.fd);
+
+	if (ret == 0)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+
+static char mnt_point[32];
+
+LKL_TEST_CALL(mount_dev, lkl_mount_dev, 0, disk_id, cla.partition, cla.fstype,
+	      0, NULL, mnt_point, sizeof(mnt_point))
+
+static int lkl_test_umount_dev(void)
+{
+	long ret, ret2;
+
+	ret = lkl_sys_chdir("/");
+
+	ret2 = lkl_umount_dev(disk_id, cla.partition, 0, 1000);
+
+	lkl_test_logf("%ld %ld", ret, ret2);
+
+	if (!ret && !ret2)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+struct lkl_dir *dir;
+
+static int lkl_test_opendir(void)
+{
+	int err;
+
+	dir = lkl_opendir(mnt_point, &err);
+
+	lkl_test_logf("lkl_opendir(%s) = %d %s\n", mnt_point, err,
+		      lkl_strerror(err));
+
+	if (err == 0)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+static int lkl_test_readdir(void)
+{
+	struct lkl_linux_dirent64 *de = lkl_readdir(dir);
+	int wr = 0;
+
+	while (de) {
+		wr += lkl_test_logf("%s ", de->d_name);
+		if (wr >= 70) {
+			lkl_test_logf("\n");
+			wr = 0;
+			break;
+		}
+		de = lkl_readdir(dir);
+	}
+
+	if (lkl_errdir(dir) == 0)
+		return TEST_SUCCESS;
+
+	return TEST_FAILURE;
+}
+
+LKL_TEST_CALL(closedir, lkl_closedir, 0, dir);
+LKL_TEST_CALL(chdir_mnt_point, lkl_sys_chdir, 0, mnt_point);
+LKL_TEST_CALL(start_kernel, lkl_start_kernel, 0, &lkl_host_ops,
+	     "mem=16M loglevel=8");
+LKL_TEST_CALL(stop_kernel, lkl_sys_halt, 0);
+
+struct lkl_test tests[] = {
+	LKL_TEST(disk_add),
+	LKL_TEST(start_kernel),
+	LKL_TEST(mount_dev),
+	LKL_TEST(chdir_mnt_point),
+	LKL_TEST(opendir),
+	LKL_TEST(readdir),
+	LKL_TEST(closedir),
+	LKL_TEST(umount_dev),
+	LKL_TEST(stop_kernel),
+	LKL_TEST(disk_remove),
+
+};
+
+int main(int argc, const char **argv)
+{
+	if (cla_parse_args(argc, argv, args) < 0)
+		return (-1);
+
+	return lkl_test_run(tests, sizeof(tests)/sizeof(struct lkl_test),
+			    "disk %s", cla.fstype);
+}
diff --git a/tools/testing/selftests/um/disk.sh b/tools/testing/selftests/um/disk.sh
new file mode 100755
index 000000000000..9cf67a76bbe3
--- /dev/null
+++ b/tools/testing/selftests/um/disk.sh
@@ -0,0 +1,67 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+script_dir=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd)
+
+source $script_dir/test.sh
+
+function prepfs()
+{
+    set -e
+
+    file=`mktemp`
+
+    dd if=/dev/zero of=$file bs=1024 count=204800
+
+    yes | mkfs.$1 $file
+
+    if ! [ -z $ANDROID_WDIR ]; then
+        adb shell mkdir -p $ANDROID_WDIR
+        adb push $file $ANDROID_WDIR
+        rm $file
+        file=$ANDROID_WDIR/$(basename $file)
+    fi
+    if ! [ -z $BSD_WDIR ]; then
+        $MYSSH mkdir -p $BSD_WDIR
+        ssh_copy $file $BSD_WDIR
+        rm $file
+        file=$BSD_WDIR/$(basename $file)
+    fi
+
+    export_vars file
+}
+
+function cleanfs()
+{
+    set -e
+
+    if ! [ -z $ANDROID_WDIR ]; then
+        adb shell rm $1
+        adb shell rm $ANDROID_WDIR/disk
+    elif ! [ -z $BSD_WDIR ]; then
+        $MYSSH rm $1
+        $MYSSH rm $BSD_WDIR/disk
+    else
+        rm $1
+    fi
+}
+
+if [ "$1" = "-t" ]; then
+    shift
+    fstype=$1
+    shift
+fi
+
+if [ -z "$fstype" ]; then
+    fstype="ext4"
+fi
+
+if [ -z $(which mkfs.$fstype) ]; then
+    echo "no mkfs.$fstype command"
+    exit 0
+fi
+
+prepfs $fstype
+${OUTPUT}/disk -d $file -t $fstype $@
+cleanfs $file
+
diff --git a/tools/um/include/lkl.h b/tools/um/include/lkl.h
index 97da2d11502f..1cfa228880d4 100644
--- a/tools/um/include/lkl.h
+++ b/tools/um/include/lkl.h
@@ -120,6 +120,16 @@ static inline long lkl_sys_creat(const char *file, int mode)
 }
 #endif
 
+#ifdef __lkl__NR_faccessat
+/**
+ * lkl_sys_access - wrapper for lkl_sys_faccessat
+ */
+static inline long lkl_sys_access(const char *file, int mode)
+{
+	return lkl_sys_faccessat(LKL_AT_FDCWD, file, mode);
+}
+#endif
+
 #ifdef __lkl__NR_mkdirat
 /**
  * lkl_sys_mkdir - wrapper for lkl_sys_mkdirat
@@ -130,6 +140,36 @@ static inline long lkl_sys_mkdir(const char *path, mode_t mode)
 }
 #endif
 
+#ifdef __lkl__NR_unlinkat
+/**
+ * lkl_sys_rmdir - wrapper for lkl_sys_unlinkrat
+ */
+static inline long lkl_sys_rmdir(const char *path)
+{
+	return lkl_sys_unlinkat(LKL_AT_FDCWD, path, LKL_AT_REMOVEDIR);
+}
+#endif
+
+#ifdef __lkl__NR_unlinkat
+/**
+ * lkl_sys_unlink - wrapper for lkl_sys_unlinkat
+ */
+static inline long lkl_sys_unlink(const char *path)
+{
+	return lkl_sys_unlinkat(LKL_AT_FDCWD, path, 0);
+}
+#endif
+
+#ifdef __lkl__NR_mknodat
+/**
+ * lkl_sys_mknod - wrapper for lkl_sys_mknodat
+ */
+static inline long lkl_sys_mknod(const char *path, mode_t mode, dev_t dev)
+{
+	return lkl_sys_mknodat(LKL_AT_FDCWD, path, mode, dev);
+}
+#endif
+
 #ifdef __lkl__NR_epoll_create1
 /**
  * lkl_sys_epoll_create - wrapper for lkl_sys_epoll_create1
@@ -151,6 +191,172 @@ static inline long lkl_sys_epoll_wait(int fd, struct lkl_epoll_event *ev,
 }
 #endif
 
+/**
+ * struct lkl_dev_blk_ops - block device host operations, defined in lkl_host.h.
+ */
+struct lkl_dev_blk_ops;
+
+/**
+ * lkl_disk - host disk handle
+ *
+ * @dev - a pointer to private information for this disk backend
+ * @fd - a POSIX file descriptor that can be used by preadv/pwritev
+ * @handle - an NT file handle that can be used by ReadFile/WriteFile
+ */
+struct lkl_disk {
+	void *dev;
+	union {
+		int fd;
+		void *handle;
+	};
+	struct lkl_dev_blk_ops *ops;
+};
+
+/**
+ * lkl_disk_add - add a new disk
+ *
+ * @disk - the host disk handle
+ * @returns a disk id (0 is valid) or a strictly negative value in case of error
+ */
+int lkl_disk_add(struct lkl_disk *disk);
+
+/**
+ * lkl_disk_remove - remove a disk
+ *
+ * This function makes a cleanup of the @disk's private information
+ * that was initialized by lkl_disk_add before.
+ *
+ * @disk - the host disk handle
+ */
+int lkl_disk_remove(struct lkl_disk disk);
+
+/**
+ * lkl_encode_dev_from_sysfs_blkdev - extract device id from sysfs
+ *
+ * This function returns the device id for the given sysfs dev node.
+ * The content of the node has to be in the form 'MAJOR:MINOR'.
+ * Also, this function expects an absolute path which means that sysfs
+ * already has to be mounted at the given path
+ *
+ * @sysfs_path - absolute path to the sysfs dev node
+ * @pdevid - pointer to memory where dev id will be returned
+ * @returns - 0 on success, a negative value on error
+ */
+int lkl_encode_dev_from_sysfs(const char *sysfs_path, uint32_t *pdevid);
+
+/**
+ * lkl_mount_dev - mount a disk
+ *
+ * This functions creates a device file for the given disk, creates a mount
+ * point and mounts the device over the mount point.
+ *
+ * @disk_id - the disk id identifying the disk to be mounted
+ * @part - disk partition or zero for full disk
+ * @fs_type - filesystem type
+ * @flags - mount flags
+ * @opts - additional filesystem specific mount options
+ * @mnt_str - a string that will be filled by this function with the path where
+ * the filesystem has been mounted
+ * @mnt_str_len - size of mnt_str
+ * @returns - 0 on success, a negative value on error
+ */
+long lkl_mount_dev(unsigned int disk_id, unsigned int part, const char *fs_type,
+		   int flags, const char *opts,
+		   char *mnt_str, unsigned int mnt_str_len);
+
+/**
+ * lkl_umount_dev - umount a disk
+ *
+ * This functions umounts the given disks and removes the device file and the
+ * mount point.
+ *
+ * @disk_id - the disk id identifying the disk to be mounted
+ * @part - disk partition or zero for full disk
+ * @flags - umount flags
+ * @timeout_ms - timeout to wait for the kernel to flush closed files so that
+ * umount can succeed
+ * @returns - 0 on success, a negative value on error
+ */
+long lkl_umount_dev(unsigned int disk_id, unsigned int part, int flags,
+		    long timeout_ms);
+
+/**
+ * lkl_umount_timeout - umount filesystem with timeout
+ *
+ * @path - the path to unmount
+ * @flags - umount flags
+ * @timeout_ms - timeout to wait for the kernel to flush closed files so that
+ * umount can succeed
+ * @returns - 0 on success, a negative value on error
+ */
+long lkl_umount_timeout(char *path, int flags, long timeout_ms);
+
+/**
+ * lkl_opendir - open a directory
+ *
+ * @path - directory path
+ * @err - pointer to store the error in case of failure
+ * @returns - a handle to be used when calling lkl_readdir
+ */
+struct lkl_dir *lkl_opendir(const char *path, int *err);
+
+/**
+ * lkl_fdopendir - open a directory
+ *
+ * @fd - file descriptor
+ * @err - pointer to store the error in case of failure
+ * @returns - a handle to be used when calling lkl_readdir
+ */
+struct lkl_dir *lkl_fdopendir(int fd, int *err);
+
+/**
+ * lkl_rewinddir - reset directory stream
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ */
+void lkl_rewinddir(struct lkl_dir *dir);
+
+/**
+ * lkl_closedir - close the directory
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ */
+int lkl_closedir(struct lkl_dir *dir);
+
+/**
+ * lkl_readdir - get the next available entry of the directory
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ * @returns - a lkl_dirent64 entry or NULL if the end of the directory stream is
+ * reached or if an error occurred; check lkl_errdir() to distinguish between
+ * errors or end of the directory stream
+ */
+struct lkl_linux_dirent64 *lkl_readdir(struct lkl_dir *dir);
+
+/**
+ * lkl_errdir - checks if an error occurred during the last lkl_readdir call
+ *
+ * @dir - the directory handler as returned by lkl_opendir
+ * @returns - 0 if no error occurred, or a negative value otherwise
+ */
+int lkl_errdir(struct lkl_dir *dir);
+
+/**
+ * lkl_dirfd - gets the file descriptor associated with the directory handle
+ *
+ * @dir - the directory handle as returned by lkl_opendir
+ * @returns - a positive value,which is the LKL file descriptor associated with
+ * the directory handle, or a negative value otherwise
+ */
+int lkl_dirfd(struct lkl_dir *dir);
+
+/**
+ * lkl_mount_fs - mount a file system type like proc, sys
+ * @fstype - file system type. e.g. proc, sys
+ * @returns - 0 on success. 1 if it's already mounted. negative on failure.
+ */
+int lkl_mount_fs(char *fstype);
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/tools/um/include/lkl_host.h b/tools/um/include/lkl_host.h
index 5fe7a64dc4dd..1a2794ef47a0 100644
--- a/tools/um/include/lkl_host.h
+++ b/tools/um/include/lkl_host.h
@@ -10,6 +10,7 @@ extern "C" {
 #include <lkl.h>
 
 extern struct lkl_host_operations lkl_host_ops;
+extern char lkl_um_devs[4096];
 
 #ifdef __cplusplus
 }
diff --git a/tools/um/lib/Build b/tools/um/lib/Build
index dddff26a3b4e..29491b40746c 100644
--- a/tools/um/lib/Build
+++ b/tools/um/lib/Build
@@ -4,3 +4,4 @@ CFLAGS_posix-host.o += -D_FILE_OFFSET_BITS=64
 liblinux-$(CONFIG_UMMODE_LIB) += utils.o
 liblinux-$(CONFIG_UMMODE_LIB) += posix-host.o
 liblinux-$(CONFIG_UMMODE_LIB) += jmp_buf.o
+liblinux-$(CONFIG_UMMODE_LIB) += fs.o
diff --git a/tools/um/lib/fs.c b/tools/um/lib/fs.c
new file mode 100644
index 000000000000..352b25f0c56f
--- /dev/null
+++ b/tools/um/lib/fs.c
@@ -0,0 +1,461 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <stdarg.h>
+#include <stdio.h>
+#include <string.h>
+#include <stdlib.h>
+#include <lkl_host.h>
+
+#define MAX_FSTYPE_LEN 50
+
+static struct lkl_disk *lkl_disks[16];
+static int registered_blk_dev_idx;
+
+static int lkl_disk_um_add(struct lkl_disk *disk, const char *blkparams)
+{
+	/* concat strings */
+	snprintf(lkl_um_devs + strlen(lkl_um_devs), sizeof(lkl_um_devs),
+		 " ubd%d=%s", registered_blk_dev_idx, blkparams);
+
+	return registered_blk_dev_idx++;
+}
+
+int lkl_disk_add(struct lkl_disk *disk)
+{
+	int ret = -1;
+
+	ret = lkl_disk_um_add(disk, disk->dev);
+
+	lkl_disks[ret] = disk;
+
+	return ret;
+}
+
+int lkl_disk_remove(struct lkl_disk disk)
+{
+	/* FIXME */
+	return 0;
+}
+
+int lkl_mount_fs(char *fstype)
+{
+	char dir[MAX_FSTYPE_LEN+2] = "/";
+	int flags = 0, ret = 0;
+
+	strncat(dir, fstype, MAX_FSTYPE_LEN);
+
+	/* Create with regular umask */
+	ret = lkl_sys_mkdir(dir, 0xff);
+	if (ret && ret != -LKL_EEXIST) {
+		lkl_perror("mount_fs mkdir", ret);
+		return ret;
+	}
+
+	/* We have no use for nonzero flags right now */
+	ret = lkl_sys_mount("none", dir, fstype, flags, NULL);
+	if (ret && ret != -LKL_EBUSY) {
+		lkl_sys_rmdir(dir);
+		return ret;
+	}
+
+	if (ret == -LKL_EBUSY)
+		return 1;
+	return 0;
+}
+
+static uint32_t new_encode_dev(unsigned int major, unsigned int minor)
+{
+	return (minor & 0xff) | (major << 8) | ((minor & ~0xff) << 12);
+}
+
+static int startswith(const char *str, const char *pre)
+{
+	return strncmp(pre, str, strlen(pre)) == 0;
+}
+
+static int get_node_with_prefix(const char *path, const char *prefix,
+				char *result, unsigned int result_len)
+{
+	struct lkl_dir *dir = NULL;
+	struct lkl_linux_dirent64 *dirent;
+	int ret;
+
+	dir = lkl_opendir(path, &ret);
+	if (!dir)
+		return ret;
+
+	ret = -LKL_ENOENT;
+
+	while ((dirent = lkl_readdir(dir))) {
+		if (startswith(dirent->d_name, prefix)) {
+			if (strlen(dirent->d_name) + 1 > result_len) {
+				ret = -LKL_ENOMEM;
+				break;
+			}
+			memcpy(result, dirent->d_name, strlen(dirent->d_name));
+			result[strlen(dirent->d_name)] = '\0';
+			ret = 0;
+			break;
+		}
+	}
+
+	lkl_closedir(dir);
+
+	return ret;
+}
+
+int lkl_encode_dev_from_sysfs(const char *sysfs_path, uint32_t *pdevid)
+{
+	int ret;
+	long fd;
+	int major, minor;
+	char buf[16] = { 0, };
+	char *bufptr;
+
+	fd = lkl_sys_open(sysfs_path, LKL_O_RDONLY, 0);
+	if (fd < 0)
+		return fd;
+
+	ret = lkl_sys_read(fd, buf, sizeof(buf));
+	if (ret < 0)
+		goto out_close;
+
+	if (ret == sizeof(buf)) {
+		ret = -LKL_ENOBUFS;
+		goto out_close;
+	}
+
+	bufptr = strchr(buf, ':');
+	if (bufptr == NULL) {
+		ret = -LKL_EINVAL;
+		goto out_close;
+	}
+	bufptr[0] = '\0';
+	bufptr++;
+
+	major = atoi(buf);
+	minor = atoi(bufptr);
+
+	*pdevid = new_encode_dev(major, minor);
+	ret = 0;
+
+out_close:
+	lkl_sys_close(fd);
+
+	return ret;
+}
+
+#define SYSFS_DEV_UMBLK_CMDLINE_PATH \
+	"/sysfs/devices/platform/uml-blkdev.%d"
+
+struct abuf {
+	char *mem, *ptr;
+	unsigned int len;
+};
+
+static int snprintf_append(struct abuf *buf, const char *fmt, ...)
+{
+	int ret;
+	va_list args;
+
+	if (!buf->ptr)
+		buf->ptr = buf->mem;
+
+	va_start(args, fmt);
+	ret = vsnprintf(buf->ptr, buf->len - (buf->ptr - buf->mem), fmt, args);
+	va_end(args);
+
+	if (ret < 0 || (ret >= (int)(buf->len - (buf->ptr - buf->mem))))
+		return -LKL_ENOMEM;
+
+	buf->ptr += ret;
+
+	return 0;
+}
+
+static int __lkl_get_blkdev(int disk_id, unsigned int part, uint32_t *pdevid,
+			    const char *sysfs_path_fmt, const char *drv_prefix,
+			    const char *disk_prefix)
+{
+	char sysfs_path[LKL_PATH_MAX];
+	char drv_name[LKL_PATH_MAX];
+	char disk_name[LKL_PATH_MAX];
+	struct abuf sysfs_path_buf = {
+		.mem = sysfs_path,
+		.len = sizeof(sysfs_path),
+	};
+	int ret;
+
+	if (disk_id < 0)
+		return -LKL_EINVAL;
+
+	ret = lkl_mount_fs("sysfs");
+	if (ret < 0)
+		return ret;
+
+	ret = snprintf_append(&sysfs_path_buf, sysfs_path_fmt, disk_id);
+	if (ret)
+		return ret;
+
+	ret = get_node_with_prefix(sysfs_path, drv_prefix, drv_name,
+				   sizeof(drv_name));
+	if (ret)
+		return ret;
+
+	ret = snprintf_append(&sysfs_path_buf, "/%s/block", drv_name);
+	if (ret)
+		return ret;
+
+	ret = get_node_with_prefix(sysfs_path, disk_prefix, disk_name,
+				   sizeof(disk_name));
+	if (ret)
+		return ret;
+
+	if (!part)
+		ret = snprintf_append(&sysfs_path_buf, "/%s/dev", disk_name);
+	else
+		ret = snprintf_append(&sysfs_path_buf, "/%s/%s%d/dev",
+				      disk_name, disk_name, part);
+	if (ret)
+		return ret;
+
+	return lkl_encode_dev_from_sysfs(sysfs_path, pdevid);
+}
+
+int lkl_get_blkdev(int disk_id, unsigned int part, uint32_t *pdevid)
+{
+	char *fmt;
+
+	fmt = SYSFS_DEV_UMBLK_CMDLINE_PATH;
+	return __lkl_get_blkdev(disk_id, part, pdevid, fmt, "", "ubd");
+}
+
+long lkl_mount_dev(unsigned int disk_id, unsigned int part,
+		   const char *fs_type, int flags,
+		   const char *data, char *mnt_str, unsigned int mnt_str_len)
+{
+	char dev_str[] = { "/dev/xxxxxxxx" };
+	unsigned int dev;
+	int err;
+	char _data[4096]; /* FIXME: PAGE_SIZE is not exported by LKL */
+
+	if (mnt_str_len < sizeof(dev_str))
+		return -LKL_ENOMEM;
+
+	err = lkl_get_blkdev(disk_id, part, &dev);
+	if (err < 0)
+		return err;
+
+	snprintf(dev_str, sizeof(dev_str), "/dev/%08x", dev);
+	snprintf(mnt_str, mnt_str_len, "/mnt/%08x", dev);
+
+	err = lkl_sys_access("/dev", LKL_S_IRWXO);
+	if (err < 0) {
+		if (err == -LKL_ENOENT)
+			err = lkl_sys_mkdir("/dev", 0700);
+		if (err < 0)
+			return err;
+	}
+
+	err = lkl_sys_mknod(dev_str, LKL_S_IFBLK | 0600, dev);
+	if (err < 0)
+		return err;
+
+	err = lkl_sys_access("/mnt", LKL_S_IRWXO);
+	if (err < 0) {
+		if (err == -LKL_ENOENT)
+			err = lkl_sys_mkdir("/mnt", 0700);
+		if (err < 0)
+			return err;
+	}
+
+	err = lkl_sys_mkdir(mnt_str, 0700);
+	if (err < 0) {
+		lkl_sys_unlink(dev_str);
+		return err;
+	}
+
+	/* kernel always copies a full page */
+	if (data) {
+		strncpy(_data, data, sizeof(_data));
+		_data[sizeof(_data) - 1] = 0;
+	} else {
+		_data[0] = 0;
+	}
+
+	err = lkl_sys_mount(dev_str, mnt_str, (char *)fs_type, flags, _data);
+	if (err < 0) {
+		lkl_sys_unlink(dev_str);
+		lkl_sys_rmdir(mnt_str);
+		return err;
+	}
+
+	return 0;
+}
+
+long lkl_umount_timeout(char *path, int flags, long timeout_ms)
+{
+	long incr = 10000000; /* 10 ms */
+	struct lkl_timespec ts = {
+		.tv_sec = 0,
+		.tv_nsec = incr,
+	};
+	long err;
+
+	do {
+		err = lkl_sys_umount(path, flags);
+		if (err == -LKL_EBUSY) {
+			lkl_sys_nanosleep((struct __lkl__kernel_timespec *)&ts,
+					  NULL);
+			timeout_ms -= incr / 1000000;
+		}
+	} while (err == -LKL_EBUSY && timeout_ms > 0);
+
+	return err;
+}
+
+long lkl_umount_dev(unsigned int disk_id, unsigned int part, int flags,
+		    long timeout_ms)
+{
+	char dev_str[] = { "/dev/xxxxxxxx" };
+	char mnt_str[] = { "/mnt/xxxxxxxx" };
+	unsigned int dev;
+	int err;
+
+	err = lkl_get_blkdev(disk_id, part, &dev);
+	if (err < 0)
+		return err;
+
+	snprintf(dev_str, sizeof(dev_str), "/dev/%08x", dev);
+	snprintf(mnt_str, sizeof(mnt_str), "/mnt/%08x", dev);
+
+	err = lkl_umount_timeout(mnt_str, flags, timeout_ms);
+	if (err)
+		return err;
+
+	err = lkl_sys_unlink(dev_str);
+	if (err)
+		return err;
+
+	return lkl_sys_rmdir(mnt_str);
+}
+
+struct lkl_dir {
+	int fd;
+	char buf[1024];
+	char *pos;
+	int len;
+};
+
+static struct lkl_dir *lkl_dir_alloc(int *err)
+{
+	struct lkl_dir *dir = lkl_mem_alloc(sizeof(struct lkl_dir));
+
+	if (!dir) {
+		*err = -LKL_ENOMEM;
+		return NULL;
+	}
+
+	dir->len = 0;
+	dir->pos = NULL;
+
+	return dir;
+}
+
+struct lkl_dir *lkl_opendir(const char *path, int *err)
+{
+	struct lkl_dir *dir = lkl_dir_alloc(err);
+
+	if (!dir) {
+		*err = -LKL_ENOMEM;
+		return NULL;
+	}
+
+	dir->fd = lkl_sys_open(path, LKL_O_RDONLY | LKL_O_DIRECTORY, 0);
+	if (dir->fd < 0) {
+		*err = dir->fd;
+		lkl_mem_free(dir);
+		return NULL;
+	}
+
+	*err = 0;
+
+	return dir;
+}
+
+struct lkl_dir *lkl_fdopendir(int fd, int *err)
+{
+	struct lkl_dir *dir = lkl_dir_alloc(err);
+
+	if (!dir)
+		return NULL;
+
+	dir->fd = fd;
+
+	return dir;
+}
+
+void lkl_rewinddir(struct lkl_dir *dir)
+{
+	lkl_sys_lseek(dir->fd, 0, LKL_SEEK_SET);
+	dir->len = 0;
+	dir->pos = NULL;
+}
+
+int lkl_closedir(struct lkl_dir *dir)
+{
+	int ret;
+
+	ret = lkl_sys_close(dir->fd);
+	lkl_mem_free(dir);
+
+	return ret;
+}
+
+struct lkl_linux_dirent64 *lkl_readdir(struct lkl_dir *dir)
+{
+	struct lkl_linux_dirent64 *de;
+
+	if (dir->len < 0)
+		return NULL;
+
+	if (!dir->pos || dir->pos - dir->buf >= dir->len)
+		goto read_buf;
+
+return_de:
+	de = (struct lkl_linux_dirent64 *)dir->pos;
+	dir->pos += de->d_reclen;
+
+	return de;
+
+read_buf:
+	dir->pos = NULL;
+	de = (struct lkl_linux_dirent64 *)dir->buf;
+	dir->len = lkl_sys_getdents64(dir->fd, de, sizeof(dir->buf));
+	if (dir->len <= 0)
+		return NULL;
+
+	dir->pos = dir->buf;
+	goto return_de;
+}
+
+int lkl_errdir(struct lkl_dir *dir)
+{
+	if (dir->len >= 0)
+		return 0;
+
+	return dir->len;
+}
+
+int lkl_dirfd(struct lkl_dir *dir)
+{
+	return dir->fd;
+}
+
+int lkl_set_fd_limit(unsigned int fd_limit)
+{
+	struct lkl_rlimit rlim = {
+		.rlim_cur = fd_limit,
+		.rlim_max = fd_limit,
+	};
+	return lkl_sys_setrlimit(LKL_RLIMIT_NOFILE, &rlim);
+}
diff --git a/tools/um/lib/posix-host.c b/tools/um/lib/posix-host.c
index f7c170fd3fd9..5fc7d6822a7f 100644
--- a/tools/um/lib/posix-host.c
+++ b/tools/um/lib/posix-host.c
@@ -268,5 +268,6 @@ void *lkl_tls_get(struct lkl_tls_key *key)
 }
 
 struct lkl_host_operations lkl_host_ops = {
+	.um_devices = lkl_um_devs,
 };
 
diff --git a/tools/um/lib/utils.c b/tools/um/lib/utils.c
index 3b38e1a95124..03841f70f73e 100644
--- a/tools/um/lib/utils.c
+++ b/tools/um/lib/utils.c
@@ -4,6 +4,8 @@
 #include <string.h>
 #include <lkl_host.h>
 
+char lkl_um_devs[4096];
+
 static const char * const lkl_err_strings[] = {
 	[0]			= "Success",
 	[LKL_EPERM]		= "Operation not permitted",
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v8 20/20] um: lkl: add documentation
  2021-01-20  2:27           ` Hajime Tazaki
@ 2021-01-20  2:27             ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-01-20  2:27 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: thehajime, tavi.purdila, retrage01, linux-kernel-library, linux-arch

A document describing brief introduction of LKL, why it is needed, and
how it is used is added.  The document is located under uml/ directory.

Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 Documentation/virt/uml/lkl.txt | 48 ++++++++++++++++++++++++++++++++++
 MAINTAINERS                    | 10 +++++++
 2 files changed, 58 insertions(+)
 create mode 100644 Documentation/virt/uml/lkl.txt

diff --git a/Documentation/virt/uml/lkl.txt b/Documentation/virt/uml/lkl.txt
new file mode 100644
index 000000000000..4942b7f093c5
--- /dev/null
+++ b/Documentation/virt/uml/lkl.txt
@@ -0,0 +1,48 @@
+
+Introduction
+============
+
+Library mode of UML, a.k.a. LKL (Linux Kernel Library), is aiming to allow
+reusing the Linux kernel code as extensively as possible with minimal effort and
+reduced maintenance overhead.
+
+Examples of how LKL can be used are: creating userspace applications (running on
+Linux and other operating systems) that can read or write Linux filesystems or
+can use the Linux networking stack, creating kernel drivers for other operating
+systems that can read Linux filesystems, bootloaders support for reading/writing
+Linux filesystems, etc.
+
+With LKL, the kernel code is compiled into an object file that can be directly
+linked by applications. The API offered by LKL is based on the Linux system call
+interface.
+
+LKL is implemented as one of the mode of UML (arch/um), and it internally uses
+sub-architecture (SUBARCH) of UML (SUBARCH=lkl). It uses host operations
+defined by the application or a host library (tools/um).
+
+
+Supported hosts
+===============
+
+The supported host for now is Linux (x86 architecture) userspace applications.
+
+
+Building LKL the host library and LKL applications
+==================================================
+
+    $ make ARCH=um SUBARCH=lkl defconfig
+    $ make ARCH=um SUBARCH=lkl
+
+will build LKL as a object file, it will install it in tools/um together with
+the headers files in tools/um/include then will build the host library, tests
+and a few of application examples:
+
+* tools/testing/selftests/um/boot.c - a simple applications that uses LKL and
+  exercises the basic LKL APIs
+
+* tools/testing/selftests/um/disk.c - a simple applications that tests LKL and
+  exercises the basic filesystem-related LKL APIs
+
+Those tests can run with the following kselftest command:
+
+    $ make ARCH=um SUBARCH=lkl TARGETS="um" kselftest
diff --git a/MAINTAINERS b/MAINTAINERS
index 00836f6452f0..8587a9715020 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -18613,6 +18613,16 @@ F:	arch/um/
 F:	arch/x86/um/
 F:	fs/hostfs/
 
+USER-MODE LINUX (UML) LIBRARY MODE
+M:	Octavian Purdila <tavi.purdila@gmail.com>
+M:	Hajime Tazaki <thehajime@gmail.com>
+L:	linux-um@lists.infradead.org
+S:	Maintained
+T:	https://github.com/lkl/linux
+F:	arch/um/lkl/
+F:	tools/testing/selftests/um/
+F:	tools/um/
+
 USERSPACE COPYIN/COPYOUT (UIOVEC)
 M:	Alexander Viro <viro@zeniv.linux.org.uk>
 S:	Maintained
-- 
2.21.0 (Apple Git-122.2)


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* [RFC v8 20/20] um: lkl: add documentation
@ 2021-01-20  2:27             ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-01-20  2:27 UTC (permalink / raw)
  To: linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, thehajime, retrage01

A document describing brief introduction of LKL, why it is needed, and
how it is used is added.  The document is located under uml/ directory.

Signed-off-by: Octavian Purdila <tavi.purdila@gmail.com>
Signed-off-by: Hajime Tazaki <thehajime@gmail.com>
---
 Documentation/virt/uml/lkl.txt | 48 ++++++++++++++++++++++++++++++++++
 MAINTAINERS                    | 10 +++++++
 2 files changed, 58 insertions(+)
 create mode 100644 Documentation/virt/uml/lkl.txt

diff --git a/Documentation/virt/uml/lkl.txt b/Documentation/virt/uml/lkl.txt
new file mode 100644
index 000000000000..4942b7f093c5
--- /dev/null
+++ b/Documentation/virt/uml/lkl.txt
@@ -0,0 +1,48 @@
+
+Introduction
+============
+
+Library mode of UML, a.k.a. LKL (Linux Kernel Library), is aiming to allow
+reusing the Linux kernel code as extensively as possible with minimal effort and
+reduced maintenance overhead.
+
+Examples of how LKL can be used are: creating userspace applications (running on
+Linux and other operating systems) that can read or write Linux filesystems or
+can use the Linux networking stack, creating kernel drivers for other operating
+systems that can read Linux filesystems, bootloaders support for reading/writing
+Linux filesystems, etc.
+
+With LKL, the kernel code is compiled into an object file that can be directly
+linked by applications. The API offered by LKL is based on the Linux system call
+interface.
+
+LKL is implemented as one of the mode of UML (arch/um), and it internally uses
+sub-architecture (SUBARCH) of UML (SUBARCH=lkl). It uses host operations
+defined by the application or a host library (tools/um).
+
+
+Supported hosts
+===============
+
+The supported host for now is Linux (x86 architecture) userspace applications.
+
+
+Building LKL the host library and LKL applications
+==================================================
+
+    $ make ARCH=um SUBARCH=lkl defconfig
+    $ make ARCH=um SUBARCH=lkl
+
+will build LKL as a object file, it will install it in tools/um together with
+the headers files in tools/um/include then will build the host library, tests
+and a few of application examples:
+
+* tools/testing/selftests/um/boot.c - a simple applications that uses LKL and
+  exercises the basic LKL APIs
+
+* tools/testing/selftests/um/disk.c - a simple applications that tests LKL and
+  exercises the basic filesystem-related LKL APIs
+
+Those tests can run with the following kselftest command:
+
+    $ make ARCH=um SUBARCH=lkl TARGETS="um" kselftest
diff --git a/MAINTAINERS b/MAINTAINERS
index 00836f6452f0..8587a9715020 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -18613,6 +18613,16 @@ F:	arch/um/
 F:	arch/x86/um/
 F:	fs/hostfs/
 
+USER-MODE LINUX (UML) LIBRARY MODE
+M:	Octavian Purdila <tavi.purdila@gmail.com>
+M:	Hajime Tazaki <thehajime@gmail.com>
+L:	linux-um@lists.infradead.org
+S:	Maintained
+T:	https://github.com/lkl/linux
+F:	arch/um/lkl/
+F:	tools/testing/selftests/um/
+F:	tools/um/
+
 USERSPACE COPYIN/COPYOUT (UIOVEC)
 M:	Alexander Viro <viro@zeniv.linux.org.uk>
 S:	Maintained
-- 
2.21.0 (Apple Git-122.2)


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply related	[flat|nested] 476+ messages in thread
* Re: [RFC v8 06/20] um: add UML library mode
  2021-01-20  2:27             ` Hajime Tazaki
@ 2021-03-14 16:49               ` Johannes Berg
  -1 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2021-03-14 16:49 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, retrage01

On Wed, 2021-01-20 at 11:27 +0900, Hajime Tazaki wrote:
> 
> +++ b/arch/um/lkl/include/asm/atomic.h
> @@ -0,0 +1,11 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef __UM_LIBMODE_ATOMIC_H
> +#define __UM_LIBMODE_ATOMIC_H
> +
> +#include <asm-generic/atomic.h>
> +
> +#ifndef CONFIG_GENERIC_ATOMIC64
> +#include "atomic64.h"
> +#endif /* !CONFIG_GENERIC_ATOMIC64 */

That's not actually configurable, is it? When is this on or off?


> +static inline void cpu_relax(void)
> +{
> +	unsigned long flags;
> +
> +	/* since this is usually called in a tight loop waiting for some
> +	 * external condition (e.g. jiffies) lets run interrupts now to allow
> +	 * the external condition to propagate
> +	 */
> +	local_irq_save(flags);
> +	local_irq_restore(flags);

Hmm?

If interrupts are enabled, then you'll get them anyway, and if not, then
this does nothing?

> +static inline void arch_copy_thread(struct arch_thread *from,
> +				    struct arch_thread *to)
> +{
> +	panic("unimplemented %s: fork isn't supported yet", __func__);
> +}

"yet" seems kind of misleading - I mean, it really *won't* be, since
that's basically what UML is, no?

I mean, in principle, nothing actually stops you from building a
linux.so instead of linux binary, and having main() renamed to
linux_main() so the application can start up whenever it needs to, or
something like that?



> +#endif
> diff --git a/arch/um/lkl/include/asm/ptrace.h b/arch/um/lkl/include/asm/ptrace.h
> new file mode 100644
> index 000000000000..83f4e10fb677
> --- /dev/null
> +++ b/arch/um/lkl/include/asm/ptrace.h
> @@ -0,0 +1,21 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef __UM_LIBMODE_PTRACE_H
> +#define __UM_LIBMODE_PTRACE_H
> +
> +#include <linux/errno.h>
> +
> +static int reg_dummy __attribute__((unused));
> +
> +#define PT_REGS_ORIG_SYSCALL(r) (reg_dummy)
> +#define PT_REGS_SYSCALL_RET(r) (reg_dummy)
> +#define PT_REGS_SET_SYSCALL_RETURN(r, res) (reg_dummy = (res))
> +#define REGS_SP(r) (reg_dummy)
> +
> +#define user_mode(regs) 0
> +#define kernel_mode(regs) 1
> +#define profile_pc(regs) 0
> +#define user_stack_pointer(regs) 0
> +
> +extern void new_thread_handler(void);
> +
> +#endif
> diff --git a/arch/um/lkl/include/asm/segment.h b/arch/um/lkl/include/asm/segment.h
> new file mode 100644
> index 000000000000..1f6746866b8b
> --- /dev/null
> +++ b/arch/um/lkl/include/asm/segment.h
> @@ -0,0 +1,9 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef __UM_LIBMODE_SEGMENT_H
> +#define __UM_LIBMODE_SEGMENT_H
> +
> +typedef struct {
> +	unsigned long seg;
> +} mm_segment_t;
> +
> +#endif /* _ASM_LKL_SEGMENT_H */
> diff --git a/arch/um/lkl/include/uapi/asm/bitsperlong.h b/arch/um/lkl/include/uapi/asm/bitsperlong.h
> new file mode 100644
> index 000000000000..f6657ba0ff9b
> --- /dev/null
> +++ b/arch/um/lkl/include/uapi/asm/bitsperlong.h
> @@ -0,0 +1,13 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef __UM_LIBMODE_UAPI_BITSPERLONG_H
> +#define __UM_LIBMODE_UAPI_BITSPERLONG_H
> +
> +#ifdef CONFIG_64BIT
> +#define __BITS_PER_LONG 64
> +#else
> +#define __BITS_PER_LONG 32
> +#endif
> +
> +#include <asm-generic/bitsperlong.h>

First, that include does nothing. Second, the comment there says:

/*
 * There seems to be no way of detecting this automatically from user
 * space, so 64 bit architectures should override this in their
 * bitsperlong.h. In particular, an architecture that supports
 * both 32 and 64 bit user space must not rely on CONFIG_64BIT
 * to decide it, but rather check a compiler provided macro.
 */


So you're doing exactly what it says *not* to?

In fact, that totally makes sense - I mean, this is *uapi* and
applications don't have access to CONFIG_ defines etc.

Do you expose that somehow to LKL users that makes this OK-ish? But it's
still very confusing, and you've not made sure the necessary header is
actually included. IIUC, lkl builds a (shared?) library, so it won't
really work this way?

> +++ b/arch/um/lkl/include/uapi/asm/byteorder.h
> @@ -0,0 +1,11 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef __UM_LIBMODE_UAPI_BYTEORDER_H
> +#define __UM_LIBMODE_UAPI_BYTEORDER_H
> +
> +#if defined(CONFIG_BIG_ENDIAN)
> +#include <linux/byteorder/big_endian.h>

Same here.

> +++ b/arch/um/lkl/um/delay.c
> @@ -0,0 +1,31 @@
> +// SPDX-License-Identifier: GPL-2.0
> +#include <linux/jiffies.h>
> +#include <linux/delay.h>
> +#include <os.h>
> +
> +void __ndelay(unsigned long nsecs)
> +{
> +	long long start = os_nsecs();
> +
> +	while (os_nsecs() < start + nsecs)
> +		;

At least do something like cpu_relax()? Although ... you made that do
nothing, so you probably want an arch-specific thing here?

johannes


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 06/20] um: add UML library mode
@ 2021-03-14 16:49               ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2021-03-14 16:49 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, retrage01

On Wed, 2021-01-20 at 11:27 +0900, Hajime Tazaki wrote:
> 
> +++ b/arch/um/lkl/include/asm/atomic.h
> @@ -0,0 +1,11 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef __UM_LIBMODE_ATOMIC_H
> +#define __UM_LIBMODE_ATOMIC_H
> +
> +#include <asm-generic/atomic.h>
> +
> +#ifndef CONFIG_GENERIC_ATOMIC64
> +#include "atomic64.h"
> +#endif /* !CONFIG_GENERIC_ATOMIC64 */

That's not actually configurable, is it? When is this on or off?


> +static inline void cpu_relax(void)
> +{
> +	unsigned long flags;
> +
> +	/* since this is usually called in a tight loop waiting for some
> +	 * external condition (e.g. jiffies) lets run interrupts now to allow
> +	 * the external condition to propagate
> +	 */
> +	local_irq_save(flags);
> +	local_irq_restore(flags);

Hmm?

If interrupts are enabled, then you'll get them anyway, and if not, then
this does nothing?

> +static inline void arch_copy_thread(struct arch_thread *from,
> +				    struct arch_thread *to)
> +{
> +	panic("unimplemented %s: fork isn't supported yet", __func__);
> +}

"yet" seems kind of misleading - I mean, it really *won't* be, since
that's basically what UML is, no?

I mean, in principle, nothing actually stops you from building a
linux.so instead of linux binary, and having main() renamed to
linux_main() so the application can start up whenever it needs to, or
something like that?



> +#endif
> diff --git a/arch/um/lkl/include/asm/ptrace.h b/arch/um/lkl/include/asm/ptrace.h
> new file mode 100644
> index 000000000000..83f4e10fb677
> --- /dev/null
> +++ b/arch/um/lkl/include/asm/ptrace.h
> @@ -0,0 +1,21 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef __UM_LIBMODE_PTRACE_H
> +#define __UM_LIBMODE_PTRACE_H
> +
> +#include <linux/errno.h>
> +
> +static int reg_dummy __attribute__((unused));
> +
> +#define PT_REGS_ORIG_SYSCALL(r) (reg_dummy)
> +#define PT_REGS_SYSCALL_RET(r) (reg_dummy)
> +#define PT_REGS_SET_SYSCALL_RETURN(r, res) (reg_dummy = (res))
> +#define REGS_SP(r) (reg_dummy)
> +
> +#define user_mode(regs) 0
> +#define kernel_mode(regs) 1
> +#define profile_pc(regs) 0
> +#define user_stack_pointer(regs) 0
> +
> +extern void new_thread_handler(void);
> +
> +#endif
> diff --git a/arch/um/lkl/include/asm/segment.h b/arch/um/lkl/include/asm/segment.h
> new file mode 100644
> index 000000000000..1f6746866b8b
> --- /dev/null
> +++ b/arch/um/lkl/include/asm/segment.h
> @@ -0,0 +1,9 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef __UM_LIBMODE_SEGMENT_H
> +#define __UM_LIBMODE_SEGMENT_H
> +
> +typedef struct {
> +	unsigned long seg;
> +} mm_segment_t;
> +
> +#endif /* _ASM_LKL_SEGMENT_H */
> diff --git a/arch/um/lkl/include/uapi/asm/bitsperlong.h b/arch/um/lkl/include/uapi/asm/bitsperlong.h
> new file mode 100644
> index 000000000000..f6657ba0ff9b
> --- /dev/null
> +++ b/arch/um/lkl/include/uapi/asm/bitsperlong.h
> @@ -0,0 +1,13 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef __UM_LIBMODE_UAPI_BITSPERLONG_H
> +#define __UM_LIBMODE_UAPI_BITSPERLONG_H
> +
> +#ifdef CONFIG_64BIT
> +#define __BITS_PER_LONG 64
> +#else
> +#define __BITS_PER_LONG 32
> +#endif
> +
> +#include <asm-generic/bitsperlong.h>

First, that include does nothing. Second, the comment there says:

/*
 * There seems to be no way of detecting this automatically from user
 * space, so 64 bit architectures should override this in their
 * bitsperlong.h. In particular, an architecture that supports
 * both 32 and 64 bit user space must not rely on CONFIG_64BIT
 * to decide it, but rather check a compiler provided macro.
 */


So you're doing exactly what it says *not* to?

In fact, that totally makes sense - I mean, this is *uapi* and
applications don't have access to CONFIG_ defines etc.

Do you expose that somehow to LKL users that makes this OK-ish? But it's
still very confusing, and you've not made sure the necessary header is
actually included. IIUC, lkl builds a (shared?) library, so it won't
really work this way?

> +++ b/arch/um/lkl/include/uapi/asm/byteorder.h
> @@ -0,0 +1,11 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef __UM_LIBMODE_UAPI_BYTEORDER_H
> +#define __UM_LIBMODE_UAPI_BYTEORDER_H
> +
> +#if defined(CONFIG_BIG_ENDIAN)
> +#include <linux/byteorder/big_endian.h>

Same here.

> +++ b/arch/um/lkl/um/delay.c
> @@ -0,0 +1,31 @@
> +// SPDX-License-Identifier: GPL-2.0
> +#include <linux/jiffies.h>
> +#include <linux/delay.h>
> +#include <os.h>
> +
> +void __ndelay(unsigned long nsecs)
> +{
> +	long long start = os_nsecs();
> +
> +	while (os_nsecs() < start + nsecs)
> +		;

At least do something like cpu_relax()? Although ... you made that do
nothing, so you probably want an arch-specific thing here?

johannes


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 07/20] um: lkl: host interface
  2021-01-20  2:27             ` Hajime Tazaki
@ 2021-03-14 16:50               ` Johannes Berg
  -1 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2021-03-14 16:50 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, retrage01


> +/**
> + * struct lkl_host_operations - host operations used by the Linux kernel
> + *
> + * These operations must be provided by a host library or by the application
> + * itself.
> + *
> + */
> +struct lkl_host_operations {
> +};

IIRC, in previous reviews we discussed this and you said you'd look at
just making those extern functions, instead of function pointers, since
realistically there's no use in being able to switch these at runtime.
What happened to that? Any particular reason not to?

johannes


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 07/20] um: lkl: host interface
@ 2021-03-14 16:50               ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2021-03-14 16:50 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, retrage01


> +/**
> + * struct lkl_host_operations - host operations used by the Linux kernel
> + *
> + * These operations must be provided by a host library or by the application
> + * itself.
> + *
> + */
> +struct lkl_host_operations {
> +};

IIRC, in previous reviews we discussed this and you said you'd look at
just making those extern functions, instead of function pointers, since
realistically there's no use in being able to switch these at runtime.
What happened to that? Any particular reason not to?

johannes


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 08/20] um: lkl: memory handling
  2021-01-20  2:27             ` Hajime Tazaki
@ 2021-03-14 16:53               ` Johannes Berg
  -1 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2021-03-14 16:53 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, retrage01


> 
> +#else
> +
> +#include <asm-generic/nommu_context.h>
> +
> +extern void force_flush_all(void);
> 
nit: no need for "extern"

> +#define CONFIG_KERNEL_RAM_BASE_ADDRESS memory_start
> +#include <asm-generic/page.h>
> +
> +#define __va_space (8*1024*1024)
> +
> +#ifndef __ASSEMBLY__
> +#include <mem.h>

Is that <shared/mem.h>? I think we should start including only with that
prefix.

> +
> +void *uml_kmalloc(int size, int flags)
> +{
> +	return kmalloc(size, flags);
> +}

That could probably still be shared?

> +int set_memory_ro(unsigned long addr, int numpages)
> +{
> +	return -EOPNOTSUPP;
> +}
> +
> +int set_memory_rw(unsigned long addr, int numpages)
> +{
> +	return -EOPNOTSUPP;
> +}
> +
> +int set_memory_nx(unsigned long addr, int numpages)
> +{
> +	return -EOPNOTSUPP;
> +}
> +
> +int set_memory_x(unsigned long addr, int numpages)
> +{
> +	return -EOPNOTSUPP;
> +}

UML for now no longer has these either.

johannes


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 08/20] um: lkl: memory handling
@ 2021-03-14 16:53               ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2021-03-14 16:53 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, retrage01


> 
> +#else
> +
> +#include <asm-generic/nommu_context.h>
> +
> +extern void force_flush_all(void);
> 
nit: no need for "extern"

> +#define CONFIG_KERNEL_RAM_BASE_ADDRESS memory_start
> +#include <asm-generic/page.h>
> +
> +#define __va_space (8*1024*1024)
> +
> +#ifndef __ASSEMBLY__
> +#include <mem.h>

Is that <shared/mem.h>? I think we should start including only with that
prefix.

> +
> +void *uml_kmalloc(int size, int flags)
> +{
> +	return kmalloc(size, flags);
> +}

That could probably still be shared?

> +int set_memory_ro(unsigned long addr, int numpages)
> +{
> +	return -EOPNOTSUPP;
> +}
> +
> +int set_memory_rw(unsigned long addr, int numpages)
> +{
> +	return -EOPNOTSUPP;
> +}
> +
> +int set_memory_nx(unsigned long addr, int numpages)
> +{
> +	return -EOPNOTSUPP;
> +}
> +
> +int set_memory_x(unsigned long addr, int numpages)
> +{
> +	return -EOPNOTSUPP;
> +}

UML for now no longer has these either.

johannes


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 09/20] um: lkl: kernel thread support
  2021-01-20  2:27             ` Hajime Tazaki
@ 2021-03-14 17:01               ` Johannes Berg
  -1 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2021-03-14 17:01 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, retrage01

On Wed, 2021-01-20 at 11:27 +0900, Hajime Tazaki wrote:

> +void __weak subarch_cpu_idle(void)
> +{
> +}
> +
>  void arch_cpu_idle(void)
>  {
>  	cpu_tasks[current_thread_info()->cpu].pid = os_getpid();
>  	um_idle_sleep();
> +	subarch_cpu_idle();


Not sure that belongs into this patch in the first place, but wouldn't
it make some sense to move the um_idle_sleep() into the
subarch_cpu_idle() so LKL (or apps using it) can get full control?

> +/*
> + * This structure is used to get access to the "LKL CPU" that allows us to run
> + * Linux code. Because we have to deal with various synchronization requirements
> + * between idle thread, system calls, interrupts, "reentrancy", CPU shutdown,
> + * imbalance wake up (i.e. acquire the CPU from one thread and release it from
> + * another), we can't use a simple synchronization mechanism such as (recursive)
> + * mutex or semaphore. Instead, we use a mutex and a bunch of status data plus a
> + * semaphore.
> + */

Honestly, some of that documentation, and perhaps even the whole API for
LKL feels like it should come earlier in the series.

E.g. now here I see all those lkl_mutex_lock() (where btw documentation
doesn't always match the function name), so you *don't* have the
function ops pointer struct anymore?

It'd be nice to have some high-level view of what applications *must*
do, and what they *can* do, at the beginning of this series.

> +	 *
> +	 * This algorithm assumes that we never have more the MAX_THREADS
> +	 * requesting CPU access.
> +	 */
> +	#define MAX_THREADS 1000000

What implications does that value have? It seems several orders of
magnitude too large?

> +static int __cpu_try_get_lock(int n)
> +{
> +	lkl_thread_t self;
> +
> +	if (__sync_fetch_and_add(&cpu.shutdown_gate, n) >= MAX_THREADS)
> +		return -2;

Feels like that could be some kind of enum here instead of -2 and -1 and
all that magic.

> +	/* when somebody holds a lock, sleep until released,
> +	 * with obtaining a semaphore (cpu.sem)
> +	 */

nit: /*
      * use this comment style
      */

> +void switch_threads(jmp_buf *me, jmp_buf *you)
> +{
> +	/* NOP */
> +}

Why, actually?

Again, goes back to the high-level design thing I alluded to above, but
it's not clear to me why you need idle (which implies you're running the
scheduler) but not this (which implies you're *not* running the
scheduler)?

johannes


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 09/20] um: lkl: kernel thread support
@ 2021-03-14 17:01               ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2021-03-14 17:01 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, retrage01

On Wed, 2021-01-20 at 11:27 +0900, Hajime Tazaki wrote:

> +void __weak subarch_cpu_idle(void)
> +{
> +}
> +
>  void arch_cpu_idle(void)
>  {
>  	cpu_tasks[current_thread_info()->cpu].pid = os_getpid();
>  	um_idle_sleep();
> +	subarch_cpu_idle();


Not sure that belongs into this patch in the first place, but wouldn't
it make some sense to move the um_idle_sleep() into the
subarch_cpu_idle() so LKL (or apps using it) can get full control?

> +/*
> + * This structure is used to get access to the "LKL CPU" that allows us to run
> + * Linux code. Because we have to deal with various synchronization requirements
> + * between idle thread, system calls, interrupts, "reentrancy", CPU shutdown,
> + * imbalance wake up (i.e. acquire the CPU from one thread and release it from
> + * another), we can't use a simple synchronization mechanism such as (recursive)
> + * mutex or semaphore. Instead, we use a mutex and a bunch of status data plus a
> + * semaphore.
> + */

Honestly, some of that documentation, and perhaps even the whole API for
LKL feels like it should come earlier in the series.

E.g. now here I see all those lkl_mutex_lock() (where btw documentation
doesn't always match the function name), so you *don't* have the
function ops pointer struct anymore?

It'd be nice to have some high-level view of what applications *must*
do, and what they *can* do, at the beginning of this series.

> +	 *
> +	 * This algorithm assumes that we never have more the MAX_THREADS
> +	 * requesting CPU access.
> +	 */
> +	#define MAX_THREADS 1000000

What implications does that value have? It seems several orders of
magnitude too large?

> +static int __cpu_try_get_lock(int n)
> +{
> +	lkl_thread_t self;
> +
> +	if (__sync_fetch_and_add(&cpu.shutdown_gate, n) >= MAX_THREADS)
> +		return -2;

Feels like that could be some kind of enum here instead of -2 and -1 and
all that magic.

> +	/* when somebody holds a lock, sleep until released,
> +	 * with obtaining a semaphore (cpu.sem)
> +	 */

nit: /*
      * use this comment style
      */

> +void switch_threads(jmp_buf *me, jmp_buf *you)
> +{
> +	/* NOP */
> +}

Why, actually?

Again, goes back to the high-level design thing I alluded to above, but
it's not clear to me why you need idle (which implies you're running the
scheduler) but not this (which implies you're *not* running the
scheduler)?

johannes


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 19/20] um: lkl: add block device support of UML
  2021-01-20  2:27             ` Hajime Tazaki
@ 2021-03-14 20:37               ` Johannes Berg
  -1 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2021-03-14 20:37 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, retrage01

On Wed, 2021-01-20 at 11:27 +0900, Hajime Tazaki wrote:
> 
> diff --git a/arch/um/Kconfig b/arch/um/Kconfig
> index 24c6596260de..5fb6a852d058 100644
> --- a/arch/um/Kconfig
> +++ b/arch/um/Kconfig
> @@ -29,6 +29,10 @@ config UMMODE_LIB
>  	select UACCESS_MEMCPY
>  	select ARCH_THREAD_STACK_ALLOCATOR
>  	select ARCH_HAS_SYSCALL_WRAPPER
> +	select VFAT_FS
> +	select NLS_CODEPAGE_437
> +	select NLS_ISO8859_1
> +	select BTRFS_FS

That doesn't really seem to make sense - the sample might need it, but
generally LKL doesn't/shouldn't?

johannes


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 19/20] um: lkl: add block device support of UML
@ 2021-03-14 20:37               ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2021-03-14 20:37 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, retrage01

On Wed, 2021-01-20 at 11:27 +0900, Hajime Tazaki wrote:
> 
> diff --git a/arch/um/Kconfig b/arch/um/Kconfig
> index 24c6596260de..5fb6a852d058 100644
> --- a/arch/um/Kconfig
> +++ b/arch/um/Kconfig
> @@ -29,6 +29,10 @@ config UMMODE_LIB
>  	select UACCESS_MEMCPY
>  	select ARCH_THREAD_STACK_ALLOCATOR
>  	select ARCH_HAS_SYSCALL_WRAPPER
> +	select VFAT_FS
> +	select NLS_CODEPAGE_437
> +	select NLS_ISO8859_1
> +	select BTRFS_FS

That doesn't really seem to make sense - the sample might need it, but
generally LKL doesn't/shouldn't?

johannes


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 12/20] um: lkl: initialization and cleanup
  2021-01-20  2:27             ` Hajime Tazaki
@ 2021-03-14 20:40               ` Johannes Berg
  -1 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2021-03-14 20:40 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, retrage01

On Wed, 2021-01-20 at 11:27 +0900, Hajime Tazaki wrote:
> 
> +	panic_blink = lkl_panic_blink;

Using a panic notifier would seem more appropriate?

> +	init_sem = lkl_sem_alloc(0);

what's the deal with this?

> +	if (!init_sem)
> +		return -ENOMEM;
> +
> +	ret = lkl_cpu_init();
> +	if (ret)
> +		goto out_free_init_sem;
> +
> +	ret = lkl_thread_create(lkl_run_kernel, NULL);
> +	if (!ret) {
> +		ret = -ENOMEM;
> +		goto out_free_init_sem;
> +	}
> +
> +	lkl_sem_down(init_sem);
> +	lkl_sem_free(init_sem);

You free it before the kernel really even started?

johannes


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 12/20] um: lkl: initialization and cleanup
@ 2021-03-14 20:40               ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2021-03-14 20:40 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, retrage01

On Wed, 2021-01-20 at 11:27 +0900, Hajime Tazaki wrote:
> 
> +	panic_blink = lkl_panic_blink;

Using a panic notifier would seem more appropriate?

> +	init_sem = lkl_sem_alloc(0);

what's the deal with this?

> +	if (!init_sem)
> +		return -ENOMEM;
> +
> +	ret = lkl_cpu_init();
> +	if (ret)
> +		goto out_free_init_sem;
> +
> +	ret = lkl_thread_create(lkl_run_kernel, NULL);
> +	if (!ret) {
> +		ret = -ENOMEM;
> +		goto out_free_init_sem;
> +	}
> +
> +	lkl_sem_down(init_sem);
> +	lkl_sem_free(init_sem);

You free it before the kernel really even started?

johannes


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 11/20] um: lkl: basic console support
  2021-01-20  2:27             ` Hajime Tazaki
@ 2021-03-14 20:42               ` Johannes Berg
  -1 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2021-03-14 20:42 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, retrage01


> 
> -obj-y := stdio_console.o fd.o chan_kern.o chan_user.o line.o
> +ifndef CONFIG_UMMODE_LIB
> +obj-y := stdio_console.o
> +else
> +obj-y :=
> +endif
> +obj-y += fd.o chan_kern.o chan_user.o line.o

Might nicer to do via Kconfig, such as

config STDIO_CONSOLE
	def_bool y
	depends on !UMMODE_LIB

and then

obj-$(CONFIG_STDIO_CONSOLE) += stdio_console.o

here. Similar to CONFIG_STDDER_CONSOLE, after all.
> +/**
> + * lkl_print - optional operation that receives console messages

How is it optional? I don't see you having a __weak definition?

johannes


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 11/20] um: lkl: basic console support
@ 2021-03-14 20:42               ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2021-03-14 20:42 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, retrage01


> 
> -obj-y := stdio_console.o fd.o chan_kern.o chan_user.o line.o
> +ifndef CONFIG_UMMODE_LIB
> +obj-y := stdio_console.o
> +else
> +obj-y :=
> +endif
> +obj-y += fd.o chan_kern.o chan_user.o line.o

Might nicer to do via Kconfig, such as

config STDIO_CONSOLE
	def_bool y
	depends on !UMMODE_LIB

and then

obj-$(CONFIG_STDIO_CONSOLE) += stdio_console.o

here. Similar to CONFIG_STDDER_CONSOLE, after all.
> +/**
> + * lkl_print - optional operation that receives console messages

How is it optional? I don't see you having a __weak definition?

johannes


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 13/20] um: lkl: integrate with irq infrastructure of UML
  2021-01-20  2:27             ` Hajime Tazaki
@ 2021-03-14 20:45               ` Johannes Berg
  -1 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2021-03-14 20:45 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, retrage01

On Wed, 2021-01-20 at 11:27 +0900, Hajime Tazaki wrote:
>  static irqreturn_t um_timer(int irq, void *dev)
>  {
> +#ifndef CONFIG_UMMODE_LIB
>  	if (get_current()->mm != NULL)

Why is the ifdef needed - get_current()->mm should always be NULL for
LKL? Surely get_current() must still work?

>  	sigemptyset(&sig_mask);
>  	sigaddset(&sig_mask, sig);
> -	if (sigprocmask(SIG_UNBLOCK, &sig_mask, NULL) < 0)
> -		panic("sigprocmask failed - errno = %d\n", errno);
> +	if (pthread_sigmask(SIG_UNBLOCK, &sig_mask, NULL) < 0)
> +		panic("pthread_sigmask failed - errno = %d\n", errno);

UML doesn't normally link with libpthread, and LKL doesn't actually
appear to require it either (since it has its lkl_thread and all), so
this seems wrong?

johannes


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 13/20] um: lkl: integrate with irq infrastructure of UML
@ 2021-03-14 20:45               ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2021-03-14 20:45 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, retrage01

On Wed, 2021-01-20 at 11:27 +0900, Hajime Tazaki wrote:
>  static irqreturn_t um_timer(int irq, void *dev)
>  {
> +#ifndef CONFIG_UMMODE_LIB
>  	if (get_current()->mm != NULL)

Why is the ifdef needed - get_current()->mm should always be NULL for
LKL? Surely get_current() must still work?

>  	sigemptyset(&sig_mask);
>  	sigaddset(&sig_mask, sig);
> -	if (sigprocmask(SIG_UNBLOCK, &sig_mask, NULL) < 0)
> -		panic("sigprocmask failed - errno = %d\n", errno);
> +	if (pthread_sigmask(SIG_UNBLOCK, &sig_mask, NULL) < 0)
> +		panic("pthread_sigmask failed - errno = %d\n", errno);

UML doesn't normally link with libpthread, and LKL doesn't actually
appear to require it either (since it has its lkl_thread and all), so
this seems wrong?

johannes


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 00/20] Unifying LKL into UML
  2021-01-20  2:27           ` Hajime Tazaki
@ 2021-03-14 21:03             ` Johannes Berg
  -1 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2021-03-14 21:03 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, retrage01

Hi,

So I'm still a bit lost here with this, and what exactly you're doing in
places.

For example, you simulate a single CPU ("depends on !SMP", and anyway
UML only supports that right now), yet on the other hand do a *LOT* of
extra work with lkl_sem, lkl_thread, lkl_mutex, and all that. It's not
clear to me why? Are you trying to model kernel threads as actual
userspace pthreads, but then run only one at a time by way of exclusive
locking?

I think we probably need a bit more architecture introduction here in
the cover letter or the documentation patch. The doc patch basically
just explains what it does, but not how it does anything, or why it was
done in this way.

For example, I'm asking myself:
 * Why NOMMU? UML doesn't really do _much_ with memory protection unless
   you add userspace, which you don't have.
 * Why pthreads and all? You already require jump_buf, so UML's
   switch_threads() ought to be just fine for scheduling? It almost
   seems like you're doing this just so you can serialize against "other
   threads" (application threads), but wouldn't that trivially be
   handled by the application? You could let it hook into switch_to() or
   something, but why should a single "LKL" CPU ever require multiple
   threads? Seems to me that the userspace could be required to
   "lkl_run()" or so (vs. lkl_start()). Heck, you could even exit
   lkl_run() every time you switch tasks in the kernel, and leave
   scheduling the kernel vs. the application entirely up to the
   application? (A trivial application would be simply doing something
   like "while (1) { lkl_run(); pause(); }" mimicking the idle loop of
   UML.

And - kind of the theme behind all these questions - why is this not
making UML actually be a binary that uses LKL? If the design were like
what I'm alluding to above, that should actually be possible? Why should
it not be possible? Why would it not be desirable? (I'm actually
thinking that might be really useful to some of the things I'm doing.)
Yes, if the application actually supports userspace running then it has
som limitations on what it can do (in particular wrt. signals etc.), but
that could be documented and would be OK?

johannes


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 00/20] Unifying LKL into UML
@ 2021-03-14 21:03             ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2021-03-14 21:03 UTC (permalink / raw)
  To: Hajime Tazaki, linux-um, jdike, richard, anton.ivanov
  Cc: tavi.purdila, linux-kernel-library, linux-arch, retrage01

Hi,

So I'm still a bit lost here with this, and what exactly you're doing in
places.

For example, you simulate a single CPU ("depends on !SMP", and anyway
UML only supports that right now), yet on the other hand do a *LOT* of
extra work with lkl_sem, lkl_thread, lkl_mutex, and all that. It's not
clear to me why? Are you trying to model kernel threads as actual
userspace pthreads, but then run only one at a time by way of exclusive
locking?

I think we probably need a bit more architecture introduction here in
the cover letter or the documentation patch. The doc patch basically
just explains what it does, but not how it does anything, or why it was
done in this way.

For example, I'm asking myself:
 * Why NOMMU? UML doesn't really do _much_ with memory protection unless
   you add userspace, which you don't have.
 * Why pthreads and all? You already require jump_buf, so UML's
   switch_threads() ought to be just fine for scheduling? It almost
   seems like you're doing this just so you can serialize against "other
   threads" (application threads), but wouldn't that trivially be
   handled by the application? You could let it hook into switch_to() or
   something, but why should a single "LKL" CPU ever require multiple
   threads? Seems to me that the userspace could be required to
   "lkl_run()" or so (vs. lkl_start()). Heck, you could even exit
   lkl_run() every time you switch tasks in the kernel, and leave
   scheduling the kernel vs. the application entirely up to the
   application? (A trivial application would be simply doing something
   like "while (1) { lkl_run(); pause(); }" mimicking the idle loop of
   UML.

And - kind of the theme behind all these questions - why is this not
making UML actually be a binary that uses LKL? If the design were like
what I'm alluding to above, that should actually be possible? Why should
it not be possible? Why would it not be desirable? (I'm actually
thinking that might be really useful to some of the things I'm doing.)
Yes, if the application actually supports userspace running then it has
som limitations on what it can do (in particular wrt. signals etc.), but
that could be documented and would be OK?

johannes


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 00/20] Unifying LKL into UML
  2021-03-14 21:03             ` Johannes Berg
@ 2021-03-16  1:17               ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-03-16  1:17 UTC (permalink / raw)
  To: johannes
  Cc: linux-um, jdike, richard, anton.ivanov, tavi.purdila,
	linux-kernel-library, linux-arch, retrage01


Hello,

First of all, thanks for all the comments to the patchset which has
been a bit stale.  I'll reply them.

On Mon, 15 Mar 2021 06:03:19 +0900,
Johannes Berg wrote:
> 
> Hi,
> 
> So I'm still a bit lost here with this, and what exactly you're doing in
> places.
> 
> For example, you simulate a single CPU ("depends on !SMP", and anyway
> UML only supports that right now), yet on the other hand do a *LOT* of
> extra work with lkl_sem, lkl_thread, lkl_mutex, and all that. It's not
> clear to me why? Are you trying to model kernel threads as actual
> userspace pthreads, but then run only one at a time by way of exclusive
> locking?
> 
> I think we probably need a bit more architecture introduction here in
> the cover letter or the documentation patch. The doc patch basically
> just explains what it does, but not how it does anything, or why it was
> done in this way.

We didn't write down the details, which are already described in the
LKL's paper (*1).  But I think we can extract/summarize some of
important information from the paper to the document so that the
design is more understandable.

*1 LKL's paper (pointer is also in the cover letter)
https://www.researchgate.net/profile/Nicolae_Tapus2/publication/224164682_LKL_The_Linux_kernel_library/links/02bfe50fd921ab4f7c000000.pdf

> For example, I'm asking myself:
>  * Why NOMMU? UML doesn't really do _much_ with memory protection unless
>    you add userspace, which you don't have.


My interpretation of MMU/NOMMU is like this;

With (emulated) MMU architecture you will have more smooth integration
with other subsystems of kernel tree, because some subsystems/features
are written with "#ifdef CONFIG_MMU".  While NOMMU doesn't, it will
bring a simplified design with better portability.

LKL takes rather to benefit better portability.

>  * Why pthreads and all? You already require jump_buf, so UML's
>    switch_threads() ought to be just fine for scheduling? It almost
>    seems like you're doing this just so you can serialize against "other
>    threads" (application threads), but wouldn't that trivially be
>    handled by the application? You could let it hook into switch_to() or
>    something, but why should a single "LKL" CPU ever require multiple
>    threads? Seems to me that the userspace could be required to
>    "lkl_run()" or so (vs. lkl_start()). Heck, you could even exit
>    lkl_run() every time you switch tasks in the kernel, and leave
>    scheduling the kernel vs. the application entirely up to the
>    application? (A trivial application would be simply doing something
>    like "while (1) { lkl_run(); pause(); }" mimicking the idle loop of
>    UML.

There is a description about this design choice in the LKL paper (*1);

  "implementations based on setjmp - longjmp require usage of a single
  stack space partitioned between all threads. As the Linux kernel
  uses deep stacks (especially in the VFS layer), in an environment
  with small stack sizes (e.g. inside another operating system's
  kernel) this will place a very low limit on the number of possible
  threads."

(from page 2, Section II, 2) Thread Support)

This is a reason of using pthread as a context primitive.

And instead of manually doing lkl_run() to schedule threads and
relying on host scheduler, LKL associates each kernel thread with a
host-provided semaphore so that Linux scheduler has a control of host
scheduler (prepared by pthread).

This is also described (and hasn't changed since then) in the paper *1
(from page 2, Section II, 3) Thread Switching).

> And - kind of the theme behind all these questions - why is this not
> making UML actually be a binary that uses LKL? If the design were like
> what I'm alluding to above, that should actually be possible? Why should
> it not be possible? Why would it not be desirable? (I'm actually
> thinking that might be really useful to some of the things I'm doing.)
> Yes, if the application actually supports userspace running then it has
> som limitations on what it can do (in particular wrt. signals etc.), but
> that could be documented and would be OK?

Let me try to describe how I think why not just generate liblinux.so
from current UML.

Making UML to build a library, which has been a long wanted features,
can be started;


I think there are several functions which the library offers;

- applications can link the library and call functions in the library
- the library will be used as a replacement of libc.a for syscall operations

to design that with UML, what we need to do are;

1) change Makefile to output liblinux.a
we faced linker script issue, which is related with generating
relocatable object in the middle.

2) make the linker-script clean with 2-stage build
we fix the linker issues of (1)

3) expose syscall as a function call
conflicts names (link-time and compile-time conflicts)

4) header rename, object localization
to fix the issue (3)

This is a common set of modifications to a library of UML.

Other parts are a choice of design, I believe.
Because a library is more _reusable_ than an executable (by it means), the
choice of LKL is to be portable, which the current UML doesn't pursue it
extensibly (focus on intel platforms).  Thus, 

5) memory: NOMMU
6) schedule (of irq/thread): pthread-based rather than setjmp/longjmp


Implementing with alternate options 5) and 6) (MMU, jmpbuf) diminishes
the strength of LKL, which we would like to avoid.  But as you
mentioned, nothing prevents us to implement the alternate options 5)
and 6) so, we can share the common part (1-4) if we will start to
implement.

I hope this makes it a bit clear, but let me know if you found
anything unclear.

-- Hajime


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 00/20] Unifying LKL into UML
@ 2021-03-16  1:17               ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-03-16  1:17 UTC (permalink / raw)
  To: johannes
  Cc: linux-um, jdike, richard, anton.ivanov, tavi.purdila,
	linux-kernel-library, linux-arch, retrage01


Hello,

First of all, thanks for all the comments to the patchset which has
been a bit stale.  I'll reply them.

On Mon, 15 Mar 2021 06:03:19 +0900,
Johannes Berg wrote:
> 
> Hi,
> 
> So I'm still a bit lost here with this, and what exactly you're doing in
> places.
> 
> For example, you simulate a single CPU ("depends on !SMP", and anyway
> UML only supports that right now), yet on the other hand do a *LOT* of
> extra work with lkl_sem, lkl_thread, lkl_mutex, and all that. It's not
> clear to me why? Are you trying to model kernel threads as actual
> userspace pthreads, but then run only one at a time by way of exclusive
> locking?
> 
> I think we probably need a bit more architecture introduction here in
> the cover letter or the documentation patch. The doc patch basically
> just explains what it does, but not how it does anything, or why it was
> done in this way.

We didn't write down the details, which are already described in the
LKL's paper (*1).  But I think we can extract/summarize some of
important information from the paper to the document so that the
design is more understandable.

*1 LKL's paper (pointer is also in the cover letter)
https://www.researchgate.net/profile/Nicolae_Tapus2/publication/224164682_LKL_The_Linux_kernel_library/links/02bfe50fd921ab4f7c000000.pdf

> For example, I'm asking myself:
>  * Why NOMMU? UML doesn't really do _much_ with memory protection unless
>    you add userspace, which you don't have.


My interpretation of MMU/NOMMU is like this;

With (emulated) MMU architecture you will have more smooth integration
with other subsystems of kernel tree, because some subsystems/features
are written with "#ifdef CONFIG_MMU".  While NOMMU doesn't, it will
bring a simplified design with better portability.

LKL takes rather to benefit better portability.

>  * Why pthreads and all? You already require jump_buf, so UML's
>    switch_threads() ought to be just fine for scheduling? It almost
>    seems like you're doing this just so you can serialize against "other
>    threads" (application threads), but wouldn't that trivially be
>    handled by the application? You could let it hook into switch_to() or
>    something, but why should a single "LKL" CPU ever require multiple
>    threads? Seems to me that the userspace could be required to
>    "lkl_run()" or so (vs. lkl_start()). Heck, you could even exit
>    lkl_run() every time you switch tasks in the kernel, and leave
>    scheduling the kernel vs. the application entirely up to the
>    application? (A trivial application would be simply doing something
>    like "while (1) { lkl_run(); pause(); }" mimicking the idle loop of
>    UML.

There is a description about this design choice in the LKL paper (*1);

  "implementations based on setjmp - longjmp require usage of a single
  stack space partitioned between all threads. As the Linux kernel
  uses deep stacks (especially in the VFS layer), in an environment
  with small stack sizes (e.g. inside another operating system's
  kernel) this will place a very low limit on the number of possible
  threads."

(from page 2, Section II, 2) Thread Support)

This is a reason of using pthread as a context primitive.

And instead of manually doing lkl_run() to schedule threads and
relying on host scheduler, LKL associates each kernel thread with a
host-provided semaphore so that Linux scheduler has a control of host
scheduler (prepared by pthread).

This is also described (and hasn't changed since then) in the paper *1
(from page 2, Section II, 3) Thread Switching).

> And - kind of the theme behind all these questions - why is this not
> making UML actually be a binary that uses LKL? If the design were like
> what I'm alluding to above, that should actually be possible? Why should
> it not be possible? Why would it not be desirable? (I'm actually
> thinking that might be really useful to some of the things I'm doing.)
> Yes, if the application actually supports userspace running then it has
> som limitations on what it can do (in particular wrt. signals etc.), but
> that could be documented and would be OK?

Let me try to describe how I think why not just generate liblinux.so
from current UML.

Making UML to build a library, which has been a long wanted features,
can be started;


I think there are several functions which the library offers;

- applications can link the library and call functions in the library
- the library will be used as a replacement of libc.a for syscall operations

to design that with UML, what we need to do are;

1) change Makefile to output liblinux.a
we faced linker script issue, which is related with generating
relocatable object in the middle.

2) make the linker-script clean with 2-stage build
we fix the linker issues of (1)

3) expose syscall as a function call
conflicts names (link-time and compile-time conflicts)

4) header rename, object localization
to fix the issue (3)

This is a common set of modifications to a library of UML.

Other parts are a choice of design, I believe.
Because a library is more _reusable_ than an executable (by it means), the
choice of LKL is to be portable, which the current UML doesn't pursue it
extensibly (focus on intel platforms).  Thus, 

5) memory: NOMMU
6) schedule (of irq/thread): pthread-based rather than setjmp/longjmp


Implementing with alternate options 5) and 6) (MMU, jmpbuf) diminishes
the strength of LKL, which we would like to avoid.  But as you
mentioned, nothing prevents us to implement the alternate options 5)
and 6) so, we can share the common part (1-4) if we will start to
implement.

I hope this makes it a bit clear, but let me know if you found
anything unclear.

-- Hajime


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 06/20] um: add UML library mode
  2021-03-14 16:49               ` Johannes Berg
@ 2021-03-16  1:17                 ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-03-16  1:17 UTC (permalink / raw)
  To: johannes
  Cc: linux-um, jdike, richard, anton.ivanov, tavi.purdila,
	linux-kernel-library, linux-arch, retrage01


On Mon, 15 Mar 2021 01:49:19 +0900,
Johannes Berg wrote:
> 
> On Wed, 2021-01-20 at 11:27 +0900, Hajime Tazaki wrote:
> > 
> > +++ b/arch/um/lkl/include/asm/atomic.h
> > @@ -0,0 +1,11 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +#ifndef __UM_LIBMODE_ATOMIC_H
> > +#define __UM_LIBMODE_ATOMIC_H
> > +
> > +#include <asm-generic/atomic.h>
> > +
> > +#ifndef CONFIG_GENERIC_ATOMIC64
> > +#include "atomic64.h"
> > +#endif /* !CONFIG_GENERIC_ATOMIC64 */
> 
> That's not actually configurable, is it? When is this on or off?

This is off when CONFIG_64BIT is off, because in that case (!64BIT) we
use generic atomic64 facility of in-kernel.

> > +static inline void cpu_relax(void)
> > +{
> > +	unsigned long flags;
> > +
> > +	/* since this is usually called in a tight loop waiting for some
> > +	 * external condition (e.g. jiffies) lets run interrupts now to allow
> > +	 * the external condition to propagate
> > +	 */
> > +	local_irq_save(flags);
> > +	local_irq_restore(flags);
> 
> Hmm?
> 
> If interrupts are enabled, then you'll get them anyway, and if not, then
> this does nothing?

You might be right; the original LKL does yield-like operation if this
function is called, but this might not be the case after the
integration with UML.  I will investigate this and fix it.


> > +static inline void arch_copy_thread(struct arch_thread *from,
> > +				    struct arch_thread *to)
> > +{
> > +	panic("unimplemented %s: fork isn't supported yet", __func__);
> > +}
> 
> "yet" seems kind of misleading - I mean, it really *won't* be, since
> that's basically what UML is, no?
> 
> I mean, in principle, nothing actually stops you from building a
> linux.so instead of linux binary, and having main() renamed to
> linux_main() so the application can start up whenever it needs to, or
> something like that?

I commented this in the reply-thread of the [00/20] patch.

> > diff --git a/arch/um/lkl/include/uapi/asm/bitsperlong.h b/arch/um/lkl/include/uapi/asm/bitsperlong.h
> > new file mode 100644
> > index 000000000000..f6657ba0ff9b
> > --- /dev/null
> > +++ b/arch/um/lkl/include/uapi/asm/bitsperlong.h
> > @@ -0,0 +1,13 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +#ifndef __UM_LIBMODE_UAPI_BITSPERLONG_H
> > +#define __UM_LIBMODE_UAPI_BITSPERLONG_H
> > +
> > +#ifdef CONFIG_64BIT
> > +#define __BITS_PER_LONG 64
> > +#else
> > +#define __BITS_PER_LONG 32
> > +#endif
> > +
> > +#include <asm-generic/bitsperlong.h>
> 
> First, that include does nothing.

We followed the way of arch/x86 (and other archs) for this.

This actually includes include/asm-generic/bitsperlong.h, which does
something I think.

> Second, the comment there says:
> 
> /*
>  * There seems to be no way of detecting this automatically from user
>  * space, so 64 bit architectures should override this in their
>  * bitsperlong.h. In particular, an architecture that supports
>  * both 32 and 64 bit user space must not rely on CONFIG_64BIT
>  * to decide it, but rather check a compiler provided macro.
>  */
> 
> 
> So you're doing exactly what it says *not* to?
> 
> In fact, that totally makes sense - I mean, this is *uapi* and
> applications don't have access to CONFIG_ defines etc.

Sorry, this patch is wrong. There is additional diff in [10/20], which
should be in the same patch here, which eliminates not to rely on
CONFIG_*.

I will fix this.
 
> Do you expose that somehow to LKL users that makes this OK-ish? But it's
> still very confusing, and you've not made sure the necessary header is
> actually included. IIUC, lkl builds a (shared?) library, so it won't
> really work this way?
>
> > +++ b/arch/um/lkl/include/uapi/asm/byteorder.h
> > @@ -0,0 +1,11 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +#ifndef __UM_LIBMODE_UAPI_BYTEORDER_H
> > +#define __UM_LIBMODE_UAPI_BYTEORDER_H
> > +
> > +#if defined(CONFIG_BIG_ENDIAN)
> > +#include <linux/byteorder/big_endian.h>
> 
> Same here.

Same problem here; we should not split this in the patch and [10/20].
Will fix this too.

> > +++ b/arch/um/lkl/um/delay.c
> > @@ -0,0 +1,31 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +#include <linux/jiffies.h>
> > +#include <linux/delay.h>
> > +#include <os.h>
> > +
> > +void __ndelay(unsigned long nsecs)
> > +{
> > +	long long start = os_nsecs();
> > +
> > +	while (os_nsecs() < start + nsecs)
> > +		;
> 
> At least do something like cpu_relax()? Although ... you made that do
> nothing, so you probably want an arch-specific thing here?

I think this isn't efficient at all, but portable enough, which is
(sub)arch-specific requirement.

-- Hajime

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 06/20] um: add UML library mode
@ 2021-03-16  1:17                 ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-03-16  1:17 UTC (permalink / raw)
  To: johannes
  Cc: linux-um, jdike, richard, anton.ivanov, tavi.purdila,
	linux-kernel-library, linux-arch, retrage01


On Mon, 15 Mar 2021 01:49:19 +0900,
Johannes Berg wrote:
> 
> On Wed, 2021-01-20 at 11:27 +0900, Hajime Tazaki wrote:
> > 
> > +++ b/arch/um/lkl/include/asm/atomic.h
> > @@ -0,0 +1,11 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +#ifndef __UM_LIBMODE_ATOMIC_H
> > +#define __UM_LIBMODE_ATOMIC_H
> > +
> > +#include <asm-generic/atomic.h>
> > +
> > +#ifndef CONFIG_GENERIC_ATOMIC64
> > +#include "atomic64.h"
> > +#endif /* !CONFIG_GENERIC_ATOMIC64 */
> 
> That's not actually configurable, is it? When is this on or off?

This is off when CONFIG_64BIT is off, because in that case (!64BIT) we
use generic atomic64 facility of in-kernel.

> > +static inline void cpu_relax(void)
> > +{
> > +	unsigned long flags;
> > +
> > +	/* since this is usually called in a tight loop waiting for some
> > +	 * external condition (e.g. jiffies) lets run interrupts now to allow
> > +	 * the external condition to propagate
> > +	 */
> > +	local_irq_save(flags);
> > +	local_irq_restore(flags);
> 
> Hmm?
> 
> If interrupts are enabled, then you'll get them anyway, and if not, then
> this does nothing?

You might be right; the original LKL does yield-like operation if this
function is called, but this might not be the case after the
integration with UML.  I will investigate this and fix it.


> > +static inline void arch_copy_thread(struct arch_thread *from,
> > +				    struct arch_thread *to)
> > +{
> > +	panic("unimplemented %s: fork isn't supported yet", __func__);
> > +}
> 
> "yet" seems kind of misleading - I mean, it really *won't* be, since
> that's basically what UML is, no?
> 
> I mean, in principle, nothing actually stops you from building a
> linux.so instead of linux binary, and having main() renamed to
> linux_main() so the application can start up whenever it needs to, or
> something like that?

I commented this in the reply-thread of the [00/20] patch.

> > diff --git a/arch/um/lkl/include/uapi/asm/bitsperlong.h b/arch/um/lkl/include/uapi/asm/bitsperlong.h
> > new file mode 100644
> > index 000000000000..f6657ba0ff9b
> > --- /dev/null
> > +++ b/arch/um/lkl/include/uapi/asm/bitsperlong.h
> > @@ -0,0 +1,13 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +#ifndef __UM_LIBMODE_UAPI_BITSPERLONG_H
> > +#define __UM_LIBMODE_UAPI_BITSPERLONG_H
> > +
> > +#ifdef CONFIG_64BIT
> > +#define __BITS_PER_LONG 64
> > +#else
> > +#define __BITS_PER_LONG 32
> > +#endif
> > +
> > +#include <asm-generic/bitsperlong.h>
> 
> First, that include does nothing.

We followed the way of arch/x86 (and other archs) for this.

This actually includes include/asm-generic/bitsperlong.h, which does
something I think.

> Second, the comment there says:
> 
> /*
>  * There seems to be no way of detecting this automatically from user
>  * space, so 64 bit architectures should override this in their
>  * bitsperlong.h. In particular, an architecture that supports
>  * both 32 and 64 bit user space must not rely on CONFIG_64BIT
>  * to decide it, but rather check a compiler provided macro.
>  */
> 
> 
> So you're doing exactly what it says *not* to?
> 
> In fact, that totally makes sense - I mean, this is *uapi* and
> applications don't have access to CONFIG_ defines etc.

Sorry, this patch is wrong. There is additional diff in [10/20], which
should be in the same patch here, which eliminates not to rely on
CONFIG_*.

I will fix this.
 
> Do you expose that somehow to LKL users that makes this OK-ish? But it's
> still very confusing, and you've not made sure the necessary header is
> actually included. IIUC, lkl builds a (shared?) library, so it won't
> really work this way?
>
> > +++ b/arch/um/lkl/include/uapi/asm/byteorder.h
> > @@ -0,0 +1,11 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +#ifndef __UM_LIBMODE_UAPI_BYTEORDER_H
> > +#define __UM_LIBMODE_UAPI_BYTEORDER_H
> > +
> > +#if defined(CONFIG_BIG_ENDIAN)
> > +#include <linux/byteorder/big_endian.h>
> 
> Same here.

Same problem here; we should not split this in the patch and [10/20].
Will fix this too.

> > +++ b/arch/um/lkl/um/delay.c
> > @@ -0,0 +1,31 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +#include <linux/jiffies.h>
> > +#include <linux/delay.h>
> > +#include <os.h>
> > +
> > +void __ndelay(unsigned long nsecs)
> > +{
> > +	long long start = os_nsecs();
> > +
> > +	while (os_nsecs() < start + nsecs)
> > +		;
> 
> At least do something like cpu_relax()? Although ... you made that do
> nothing, so you probably want an arch-specific thing here?

I think this isn't efficient at all, but portable enough, which is
(sub)arch-specific requirement.

-- Hajime

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 07/20] um: lkl: host interface
  2021-03-14 16:50               ` Johannes Berg
@ 2021-03-16  1:17                 ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-03-16  1:17 UTC (permalink / raw)
  To: johannes
  Cc: linux-um, jdike, richard, anton.ivanov, tavi.purdila,
	linux-kernel-library, linux-arch, retrage01


On Mon, 15 Mar 2021 01:50:27 +0900,
Johannes Berg wrote:
> 
> 
> > +/**
> > + * struct lkl_host_operations - host operations used by the Linux kernel
> > + *
> > + * These operations must be provided by a host library or by the application
> > + * itself.
> > + *
> > + */
> > +struct lkl_host_operations {
> > +};
> 
> IIRC, in previous reviews we discussed this and you said you'd look at
> just making those extern functions, instead of function pointers, since
> realistically there's no use in being able to switch these at runtime.
> What happened to that? Any particular reason not to?

Yes, I followed your suggestion that we just use functions instead of
function pointers ([17/20]).

Though we won't use pointers, we still need this structure to inform
userspace side (of LKL) to kernel. [19/20], block device
implementation uses this.

-- Hajime


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 07/20] um: lkl: host interface
@ 2021-03-16  1:17                 ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-03-16  1:17 UTC (permalink / raw)
  To: johannes
  Cc: linux-um, jdike, richard, anton.ivanov, tavi.purdila,
	linux-kernel-library, linux-arch, retrage01


On Mon, 15 Mar 2021 01:50:27 +0900,
Johannes Berg wrote:
> 
> 
> > +/**
> > + * struct lkl_host_operations - host operations used by the Linux kernel
> > + *
> > + * These operations must be provided by a host library or by the application
> > + * itself.
> > + *
> > + */
> > +struct lkl_host_operations {
> > +};
> 
> IIRC, in previous reviews we discussed this and you said you'd look at
> just making those extern functions, instead of function pointers, since
> realistically there's no use in being able to switch these at runtime.
> What happened to that? Any particular reason not to?

Yes, I followed your suggestion that we just use functions instead of
function pointers ([17/20]).

Though we won't use pointers, we still need this structure to inform
userspace side (of LKL) to kernel. [19/20], block device
implementation uses this.

-- Hajime


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 08/20] um: lkl: memory handling
  2021-03-14 16:53               ` Johannes Berg
@ 2021-03-16  1:18                 ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-03-16  1:18 UTC (permalink / raw)
  To: johannes
  Cc: linux-um, jdike, richard, anton.ivanov, tavi.purdila,
	linux-kernel-library, linux-arch, retrage01


On Mon, 15 Mar 2021 01:53:01 +0900,
Johannes Berg wrote:
> 
> 
> > 
> > +#else
> > +
> > +#include <asm-generic/nommu_context.h>
> > +
> > +extern void force_flush_all(void);
> > 
> nit: no need for "extern"

thanks, will fix this.

> > +#define CONFIG_KERNEL_RAM_BASE_ADDRESS memory_start
> > +#include <asm-generic/page.h>
> > +
> > +#define __va_space (8*1024*1024)
> > +
> > +#ifndef __ASSEMBLY__
> > +#include <mem.h>
> 
> Is that <shared/mem.h>? I think we should start including only with that
> prefix.

okay, will fix it.

> > +
> > +void *uml_kmalloc(int size, int flags)
> > +{
> > +	return kmalloc(size, flags);
> > +}
> 
> That could probably still be shared?

This function is a stub of arch/um/kernel/mem.c, which LKL doesn't
use for the build.  Thus we defined here.

Or are you suggesting to not stubbing this function, but extracting
the function from mem.c ?

> > +int set_memory_ro(unsigned long addr, int numpages)
> > +{
> > +	return -EOPNOTSUPP;
> > +}
> > +
> > +int set_memory_rw(unsigned long addr, int numpages)
> > +{
> > +	return -EOPNOTSUPP;
> > +}
> > +
> > +int set_memory_nx(unsigned long addr, int numpages)
> > +{
> > +	return -EOPNOTSUPP;
> > +}
> > +
> > +int set_memory_x(unsigned long addr, int numpages)
> > +{
> > +	return -EOPNOTSUPP;
> > +}
> 
> UML for now no longer has these either.

Ah, I see the changes.  Will fix them.

-- Hajime

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 08/20] um: lkl: memory handling
@ 2021-03-16  1:18                 ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-03-16  1:18 UTC (permalink / raw)
  To: johannes
  Cc: linux-um, jdike, richard, anton.ivanov, tavi.purdila,
	linux-kernel-library, linux-arch, retrage01


On Mon, 15 Mar 2021 01:53:01 +0900,
Johannes Berg wrote:
> 
> 
> > 
> > +#else
> > +
> > +#include <asm-generic/nommu_context.h>
> > +
> > +extern void force_flush_all(void);
> > 
> nit: no need for "extern"

thanks, will fix this.

> > +#define CONFIG_KERNEL_RAM_BASE_ADDRESS memory_start
> > +#include <asm-generic/page.h>
> > +
> > +#define __va_space (8*1024*1024)
> > +
> > +#ifndef __ASSEMBLY__
> > +#include <mem.h>
> 
> Is that <shared/mem.h>? I think we should start including only with that
> prefix.

okay, will fix it.

> > +
> > +void *uml_kmalloc(int size, int flags)
> > +{
> > +	return kmalloc(size, flags);
> > +}
> 
> That could probably still be shared?

This function is a stub of arch/um/kernel/mem.c, which LKL doesn't
use for the build.  Thus we defined here.

Or are you suggesting to not stubbing this function, but extracting
the function from mem.c ?

> > +int set_memory_ro(unsigned long addr, int numpages)
> > +{
> > +	return -EOPNOTSUPP;
> > +}
> > +
> > +int set_memory_rw(unsigned long addr, int numpages)
> > +{
> > +	return -EOPNOTSUPP;
> > +}
> > +
> > +int set_memory_nx(unsigned long addr, int numpages)
> > +{
> > +	return -EOPNOTSUPP;
> > +}
> > +
> > +int set_memory_x(unsigned long addr, int numpages)
> > +{
> > +	return -EOPNOTSUPP;
> > +}
> 
> UML for now no longer has these either.

Ah, I see the changes.  Will fix them.

-- Hajime

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 09/20] um: lkl: kernel thread support
  2021-03-14 17:01               ` Johannes Berg
@ 2021-03-16  1:18                 ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-03-16  1:18 UTC (permalink / raw)
  To: johannes
  Cc: linux-um, jdike, richard, anton.ivanov, tavi.purdila,
	linux-kernel-library, linux-arch, retrage01


On Mon, 15 Mar 2021 02:01:20 +0900,
Johannes Berg wrote:
> 
> On Wed, 2021-01-20 at 11:27 +0900, Hajime Tazaki wrote:
> 
> > +void __weak subarch_cpu_idle(void)
> > +{
> > +}
> > +
> >  void arch_cpu_idle(void)
> >  {
> >  	cpu_tasks[current_thread_info()->cpu].pid = os_getpid();
> >  	um_idle_sleep();
> > +	subarch_cpu_idle();
> 
> 
> Not sure that belongs into this patch in the first place, but wouldn't
> it make some sense to move the um_idle_sleep() into the
> subarch_cpu_idle() so LKL (or apps using it) can get full control?

Agree.  I'll move that part to um_idle_sleep.

> > +/*
> > + * This structure is used to get access to the "LKL CPU" that allows us to run
> > + * Linux code. Because we have to deal with various synchronization requirements
> > + * between idle thread, system calls, interrupts, "reentrancy", CPU shutdown,
> > + * imbalance wake up (i.e. acquire the CPU from one thread and release it from
> > + * another), we can't use a simple synchronization mechanism such as (recursive)
> > + * mutex or semaphore. Instead, we use a mutex and a bunch of status data plus a
> > + * semaphore.
> > + */
> 
> Honestly, some of that documentation, and perhaps even the whole API for
> LKL feels like it should come earlier in the series.
> 
> E.g. now here I see all those lkl_mutex_lock() (where btw documentation
> doesn't always match the function name), so you *don't* have the
> function ops pointer struct anymore?

I will check the inconsistency of names.
# and, yes, we don't have function ops anymore.

> It'd be nice to have some high-level view of what applications *must*
> do, and what they *can* do, at the beginning of this series.

agree.

> > +	 *
> > +	 * This algorithm assumes that we never have more the MAX_THREADS
> > +	 * requesting CPU access.
> > +	 */
> > +	#define MAX_THREADS 1000000
> 
> What implications does that value have? It seems several orders of
> magnitude too large?

I guess this is a kind of random number, but will justify (or make it
smaller) after some investigation.

> > +static int __cpu_try_get_lock(int n)
> > +{
> > +	lkl_thread_t self;
> > +
> > +	if (__sync_fetch_and_add(&cpu.shutdown_gate, n) >= MAX_THREADS)
> > +		return -2;
> 
> Feels like that could be some kind of enum here instead of -2 and -1 and
> all that magic.

agree; I will update with an enum.

> > +	/* when somebody holds a lock, sleep until released,
> > +	 * with obtaining a semaphore (cpu.sem)
> > +	 */
> 
> nit: /*
>       * use this comment style
>       */

thanks, it should be. will fix this.

> > +void switch_threads(jmp_buf *me, jmp_buf *you)
> > +{
> > +	/* NOP */
> > +}
> 
> Why, actually?

Our threads doesn't use the UML jmp_buf (use pthread instead) so, this
function has to do nothing.

We can add a comment of this here.

> Again, goes back to the high-level design thing I alluded to above, but
> it's not clear to me why you need idle (which implies you're running the
> scheduler) but not this (which implies you're *not* running the
> scheduler)?

okay, will write a separate document for the high-level description of
what this is.

-- Hajime

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 09/20] um: lkl: kernel thread support
@ 2021-03-16  1:18                 ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-03-16  1:18 UTC (permalink / raw)
  To: johannes
  Cc: linux-um, jdike, richard, anton.ivanov, tavi.purdila,
	linux-kernel-library, linux-arch, retrage01


On Mon, 15 Mar 2021 02:01:20 +0900,
Johannes Berg wrote:
> 
> On Wed, 2021-01-20 at 11:27 +0900, Hajime Tazaki wrote:
> 
> > +void __weak subarch_cpu_idle(void)
> > +{
> > +}
> > +
> >  void arch_cpu_idle(void)
> >  {
> >  	cpu_tasks[current_thread_info()->cpu].pid = os_getpid();
> >  	um_idle_sleep();
> > +	subarch_cpu_idle();
> 
> 
> Not sure that belongs into this patch in the first place, but wouldn't
> it make some sense to move the um_idle_sleep() into the
> subarch_cpu_idle() so LKL (or apps using it) can get full control?

Agree.  I'll move that part to um_idle_sleep.

> > +/*
> > + * This structure is used to get access to the "LKL CPU" that allows us to run
> > + * Linux code. Because we have to deal with various synchronization requirements
> > + * between idle thread, system calls, interrupts, "reentrancy", CPU shutdown,
> > + * imbalance wake up (i.e. acquire the CPU from one thread and release it from
> > + * another), we can't use a simple synchronization mechanism such as (recursive)
> > + * mutex or semaphore. Instead, we use a mutex and a bunch of status data plus a
> > + * semaphore.
> > + */
> 
> Honestly, some of that documentation, and perhaps even the whole API for
> LKL feels like it should come earlier in the series.
> 
> E.g. now here I see all those lkl_mutex_lock() (where btw documentation
> doesn't always match the function name), so you *don't* have the
> function ops pointer struct anymore?

I will check the inconsistency of names.
# and, yes, we don't have function ops anymore.

> It'd be nice to have some high-level view of what applications *must*
> do, and what they *can* do, at the beginning of this series.

agree.

> > +	 *
> > +	 * This algorithm assumes that we never have more the MAX_THREADS
> > +	 * requesting CPU access.
> > +	 */
> > +	#define MAX_THREADS 1000000
> 
> What implications does that value have? It seems several orders of
> magnitude too large?

I guess this is a kind of random number, but will justify (or make it
smaller) after some investigation.

> > +static int __cpu_try_get_lock(int n)
> > +{
> > +	lkl_thread_t self;
> > +
> > +	if (__sync_fetch_and_add(&cpu.shutdown_gate, n) >= MAX_THREADS)
> > +		return -2;
> 
> Feels like that could be some kind of enum here instead of -2 and -1 and
> all that magic.

agree; I will update with an enum.

> > +	/* when somebody holds a lock, sleep until released,
> > +	 * with obtaining a semaphore (cpu.sem)
> > +	 */
> 
> nit: /*
>       * use this comment style
>       */

thanks, it should be. will fix this.

> > +void switch_threads(jmp_buf *me, jmp_buf *you)
> > +{
> > +	/* NOP */
> > +}
> 
> Why, actually?

Our threads doesn't use the UML jmp_buf (use pthread instead) so, this
function has to do nothing.

We can add a comment of this here.

> Again, goes back to the high-level design thing I alluded to above, but
> it's not clear to me why you need idle (which implies you're running the
> scheduler) but not this (which implies you're *not* running the
> scheduler)?

okay, will write a separate document for the high-level description of
what this is.

-- Hajime

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 19/20] um: lkl: add block device support of UML
  2021-03-14 20:37               ` Johannes Berg
@ 2021-03-16  1:19                 ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-03-16  1:19 UTC (permalink / raw)
  To: johannes
  Cc: linux-um, jdike, richard, anton.ivanov, tavi.purdila,
	linux-kernel-library, linux-arch, retrage01


On Mon, 15 Mar 2021 05:37:40 +0900,
Johannes Berg wrote:
> 
> On Wed, 2021-01-20 at 11:27 +0900, Hajime Tazaki wrote:
> > 
> > diff --git a/arch/um/Kconfig b/arch/um/Kconfig
> > index 24c6596260de..5fb6a852d058 100644
> > --- a/arch/um/Kconfig
> > +++ b/arch/um/Kconfig
> > @@ -29,6 +29,10 @@ config UMMODE_LIB
> >  	select UACCESS_MEMCPY
> >  	select ARCH_THREAD_STACK_ALLOCATOR
> >  	select ARCH_HAS_SYSCALL_WRAPPER
> > +	select VFAT_FS
> > +	select NLS_CODEPAGE_437
> > +	select NLS_ISO8859_1
> > +	select BTRFS_FS
> 
> That doesn't really seem to make sense - the sample might need it, but
> generally LKL doesn't/shouldn't?

I'm trying to understand your comment;
Do you mean that enabling those options in Kconfig doesn't make sense ?

and if you mean the sample as sample code, is the added test case
(e.g., tools/testing/selftests/um/disk.c, which is included in the
same patch) for this purpose ?

-- HAjime

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 19/20] um: lkl: add block device support of UML
@ 2021-03-16  1:19                 ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-03-16  1:19 UTC (permalink / raw)
  To: johannes
  Cc: linux-um, jdike, richard, anton.ivanov, tavi.purdila,
	linux-kernel-library, linux-arch, retrage01


On Mon, 15 Mar 2021 05:37:40 +0900,
Johannes Berg wrote:
> 
> On Wed, 2021-01-20 at 11:27 +0900, Hajime Tazaki wrote:
> > 
> > diff --git a/arch/um/Kconfig b/arch/um/Kconfig
> > index 24c6596260de..5fb6a852d058 100644
> > --- a/arch/um/Kconfig
> > +++ b/arch/um/Kconfig
> > @@ -29,6 +29,10 @@ config UMMODE_LIB
> >  	select UACCESS_MEMCPY
> >  	select ARCH_THREAD_STACK_ALLOCATOR
> >  	select ARCH_HAS_SYSCALL_WRAPPER
> > +	select VFAT_FS
> > +	select NLS_CODEPAGE_437
> > +	select NLS_ISO8859_1
> > +	select BTRFS_FS
> 
> That doesn't really seem to make sense - the sample might need it, but
> generally LKL doesn't/shouldn't?

I'm trying to understand your comment;
Do you mean that enabling those options in Kconfig doesn't make sense ?

and if you mean the sample as sample code, is the added test case
(e.g., tools/testing/selftests/um/disk.c, which is included in the
same patch) for this purpose ?

-- HAjime

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 12/20] um: lkl: initialization and cleanup
  2021-03-14 20:40               ` Johannes Berg
@ 2021-03-16  1:19                 ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-03-16  1:19 UTC (permalink / raw)
  To: johannes
  Cc: linux-um, jdike, richard, anton.ivanov, tavi.purdila,
	linux-kernel-library, linux-arch, retrage01


On Mon, 15 Mar 2021 05:40:39 +0900,
Johannes Berg wrote:
> 
> On Wed, 2021-01-20 at 11:27 +0900, Hajime Tazaki wrote:
> > 
> > +	panic_blink = lkl_panic_blink;
> 
> Using a panic notifier would seem more appropriate?

I understand;  will investigate if we can change.

> > +	init_sem = lkl_sem_alloc(0);
> 
> what's the deal with this?
> 
> > +	if (!init_sem)
> > +		return -ENOMEM;
> > +
> > +	ret = lkl_cpu_init();
> > +	if (ret)
> > +		goto out_free_init_sem;
> > +
> > +	ret = lkl_thread_create(lkl_run_kernel, NULL);
> > +	if (!ret) {
> > +		ret = -ENOMEM;
> > +		goto out_free_init_sem;
> > +	}
> > +
> > +	lkl_sem_down(init_sem);
> > +	lkl_sem_free(init_sem);
> 
> You free it before the kernel really even started?

The semaphore (init_sem) is unlocked at lkl_run_init(), so this waits
for finishing the init call. After the initialization ends, it's safe
to free as it's no longer used.

I may add some description here to make it clear.

-- Hajime

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 12/20] um: lkl: initialization and cleanup
@ 2021-03-16  1:19                 ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-03-16  1:19 UTC (permalink / raw)
  To: johannes
  Cc: linux-um, jdike, richard, anton.ivanov, tavi.purdila,
	linux-kernel-library, linux-arch, retrage01


On Mon, 15 Mar 2021 05:40:39 +0900,
Johannes Berg wrote:
> 
> On Wed, 2021-01-20 at 11:27 +0900, Hajime Tazaki wrote:
> > 
> > +	panic_blink = lkl_panic_blink;
> 
> Using a panic notifier would seem more appropriate?

I understand;  will investigate if we can change.

> > +	init_sem = lkl_sem_alloc(0);
> 
> what's the deal with this?
> 
> > +	if (!init_sem)
> > +		return -ENOMEM;
> > +
> > +	ret = lkl_cpu_init();
> > +	if (ret)
> > +		goto out_free_init_sem;
> > +
> > +	ret = lkl_thread_create(lkl_run_kernel, NULL);
> > +	if (!ret) {
> > +		ret = -ENOMEM;
> > +		goto out_free_init_sem;
> > +	}
> > +
> > +	lkl_sem_down(init_sem);
> > +	lkl_sem_free(init_sem);
> 
> You free it before the kernel really even started?

The semaphore (init_sem) is unlocked at lkl_run_init(), so this waits
for finishing the init call. After the initialization ends, it's safe
to free as it's no longer used.

I may add some description here to make it clear.

-- Hajime

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 11/20] um: lkl: basic console support
  2021-03-14 20:42               ` Johannes Berg
@ 2021-03-16  1:19                 ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-03-16  1:19 UTC (permalink / raw)
  To: johannes
  Cc: linux-um, jdike, richard, anton.ivanov, tavi.purdila,
	linux-kernel-library, linux-arch, retrage01


On Mon, 15 Mar 2021 05:42:52 +0900,
Johannes Berg wrote:
> 
> 
> > 
> > -obj-y := stdio_console.o fd.o chan_kern.o chan_user.o line.o
> > +ifndef CONFIG_UMMODE_LIB
> > +obj-y := stdio_console.o
> > +else
> > +obj-y :=
> > +endif
> > +obj-y += fd.o chan_kern.o chan_user.o line.o
> 
> Might nicer to do via Kconfig, such as
> 
> config STDIO_CONSOLE
> 	def_bool y
> 	depends on !UMMODE_LIB
> 
> and then
> 
> obj-$(CONFIG_STDIO_CONSOLE) += stdio_console.o
> 
> here. Similar to CONFIG_STDDER_CONSOLE, after all.

Agree.  I'll fix them.

> > +/**
> > + * lkl_print - optional operation that receives console messages
> 
> How is it optional? I don't see you having a __weak definition?

Optional is misleading...  I will fix the comment.

-- Hajime

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 11/20] um: lkl: basic console support
@ 2021-03-16  1:19                 ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-03-16  1:19 UTC (permalink / raw)
  To: johannes
  Cc: linux-um, jdike, richard, anton.ivanov, tavi.purdila,
	linux-kernel-library, linux-arch, retrage01


On Mon, 15 Mar 2021 05:42:52 +0900,
Johannes Berg wrote:
> 
> 
> > 
> > -obj-y := stdio_console.o fd.o chan_kern.o chan_user.o line.o
> > +ifndef CONFIG_UMMODE_LIB
> > +obj-y := stdio_console.o
> > +else
> > +obj-y :=
> > +endif
> > +obj-y += fd.o chan_kern.o chan_user.o line.o
> 
> Might nicer to do via Kconfig, such as
> 
> config STDIO_CONSOLE
> 	def_bool y
> 	depends on !UMMODE_LIB
> 
> and then
> 
> obj-$(CONFIG_STDIO_CONSOLE) += stdio_console.o
> 
> here. Similar to CONFIG_STDDER_CONSOLE, after all.

Agree.  I'll fix them.

> > +/**
> > + * lkl_print - optional operation that receives console messages
> 
> How is it optional? I don't see you having a __weak definition?

Optional is misleading...  I will fix the comment.

-- Hajime

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 13/20] um: lkl: integrate with irq infrastructure of UML
  2021-03-14 20:45               ` Johannes Berg
@ 2021-03-16  1:20                 ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-03-16  1:20 UTC (permalink / raw)
  To: johannes
  Cc: linux-um, jdike, richard, anton.ivanov, tavi.purdila,
	linux-kernel-library, linux-arch, retrage01



On Mon, 15 Mar 2021 05:45:23 +0900,
Johannes Berg wrote:
> 
> On Wed, 2021-01-20 at 11:27 +0900, Hajime Tazaki wrote:
> >  static irqreturn_t um_timer(int irq, void *dev)
> >  {
> > +#ifndef CONFIG_UMMODE_LIB
> >  	if (get_current()->mm != NULL)
> 
> Why is the ifdef needed - get_current()->mm should always be NULL for
> LKL? Surely get_current() must still work?

What we tried to ifdef is to avoid the following call;

   os_alarm_process(get_current()->mm->context.id.u.pid);

because we didn't use/update get_current()->mm->context.id (struct
mm_id) and calling kill(0, SIGALRM) makes a program puzzled thus,
eliminate it.

> >  	sigemptyset(&sig_mask);
> >  	sigaddset(&sig_mask, sig);
> > -	if (sigprocmask(SIG_UNBLOCK, &sig_mask, NULL) < 0)
> > -		panic("sigprocmask failed - errno = %d\n", errno);
> > +	if (pthread_sigmask(SIG_UNBLOCK, &sig_mask, NULL) < 0)
> > +		panic("pthread_sigmask failed - errno = %d\n", errno);
> 
> UML doesn't normally link with libpthread, and LKL doesn't actually
> appear to require it either (since it has its lkl_thread and all), so
> this seems wrong?

I think both UML/LKL link with libpthread.  See old
scripts/link-vmlinux.sh, or [01/20] patch.

-		${CC} ${CFLAGS_vmlinux}				\
-			${strip_debug}				\
-			-o ${output}				\
-			-Wl,-T,${lds}				\
-			${objects}				\
-			-lutil -lrt -lpthread
-		rm -f linux

-- Hajime

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 13/20] um: lkl: integrate with irq infrastructure of UML
@ 2021-03-16  1:20                 ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-03-16  1:20 UTC (permalink / raw)
  To: johannes
  Cc: linux-um, jdike, richard, anton.ivanov, tavi.purdila,
	linux-kernel-library, linux-arch, retrage01



On Mon, 15 Mar 2021 05:45:23 +0900,
Johannes Berg wrote:
> 
> On Wed, 2021-01-20 at 11:27 +0900, Hajime Tazaki wrote:
> >  static irqreturn_t um_timer(int irq, void *dev)
> >  {
> > +#ifndef CONFIG_UMMODE_LIB
> >  	if (get_current()->mm != NULL)
> 
> Why is the ifdef needed - get_current()->mm should always be NULL for
> LKL? Surely get_current() must still work?

What we tried to ifdef is to avoid the following call;

   os_alarm_process(get_current()->mm->context.id.u.pid);

because we didn't use/update get_current()->mm->context.id (struct
mm_id) and calling kill(0, SIGALRM) makes a program puzzled thus,
eliminate it.

> >  	sigemptyset(&sig_mask);
> >  	sigaddset(&sig_mask, sig);
> > -	if (sigprocmask(SIG_UNBLOCK, &sig_mask, NULL) < 0)
> > -		panic("sigprocmask failed - errno = %d\n", errno);
> > +	if (pthread_sigmask(SIG_UNBLOCK, &sig_mask, NULL) < 0)
> > +		panic("pthread_sigmask failed - errno = %d\n", errno);
> 
> UML doesn't normally link with libpthread, and LKL doesn't actually
> appear to require it either (since it has its lkl_thread and all), so
> this seems wrong?

I think both UML/LKL link with libpthread.  See old
scripts/link-vmlinux.sh, or [01/20] patch.

-		${CC} ${CFLAGS_vmlinux}				\
-			${strip_debug}				\
-			-o ${output}				\
-			-Wl,-T,${lds}				\
-			${objects}				\
-			-lutil -lrt -lpthread
-		rm -f linux

-- Hajime

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 00/20] Unifying LKL into UML
  2021-03-16  1:17               ` Hajime Tazaki
@ 2021-03-16 21:29                 ` Johannes Berg
  -1 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2021-03-16 21:29 UTC (permalink / raw)
  To: Hajime Tazaki
  Cc: linux-um, jdike, richard, anton.ivanov, tavi.purdila,
	linux-kernel-library, linux-arch, retrage01

Hi,

> First of all, thanks for all the comments to the patchset which has
> been a bit stale.  I'll reply them.

Yeah, sorry. I had it marked unread ("to look at") since you posted it.

> We didn't write down the details, which are already described in the
> LKL's paper (*1).  But I think we can extract/summarize some of
> important information from the paper to the document so that the
> design is more understandable.
> 
> *1 LKL's paper (pointer is also in the cover letter)
> https://www.researchgate.net/profile/Nicolae_Tapus2/publication/224164682_LKL_The_Linux_kernel_library/links/02bfe50fd921ab4f7c000000.pdf

OK, I guess I should take a look. Probably I never did, always thinking
that it was more of an overview than technical details and design
decisions.
> 
> My interpretation of MMU/NOMMU is like this;
> 
> With (emulated) MMU architecture you will have more smooth integration
> with other subsystems of kernel tree, because some subsystems/features
> are written with "#ifdef CONFIG_MMU".  While NOMMU doesn't, it will
> bring a simplified design with better portability.
> 
> LKL takes rather to benefit better portability.

I don't think it *matters* so much for portability? I mean, every system
under the sun is going to allow some kind of "mprotect", right? You
don't really want to port LKL to systems that don't have even that?

> >  * Why pthreads and all? You already require jump_buf, so UML's
> >    switch_threads() ought to be just fine for scheduling? It almost
> >    seems like you're doing this just so you can serialize against "other
> >    threads" (application threads), but wouldn't that trivially be
> >    handled by the application? You could let it hook into switch_to() or
> >    something, but why should a single "LKL" CPU ever require multiple
> >    threads? Seems to me that the userspace could be required to
> >    "lkl_run()" or so (vs. lkl_start()). Heck, you could even exit
> >    lkl_run() every time you switch tasks in the kernel, and leave
> >    scheduling the kernel vs. the application entirely up to the
> >    application? (A trivial application would be simply doing something
> >    like "while (1) { lkl_run(); pause(); }" mimicking the idle loop of
> >    UML.
> 
> There is a description about this design choice in the LKL paper (*1);
> 
>   "implementations based on setjmp - longjmp require usage of a single
>   stack space partitioned between all threads. As the Linux kernel
>   uses deep stacks (especially in the VFS layer), in an environment
>   with small stack sizes (e.g. inside another operating system's
>   kernel) this will place a very low limit on the number of possible
>   threads."
> 
> (from page 2, Section II, 2) Thread Support)
> 
> This is a reason of using pthread as a context primitive.

That impliciation (setjmp doesnt do stacks, so must use pthread) really
isn't true, you also have posix contexts or windows fibers. That would
probably be much easier to understands, since real threads imply that
you have actual concurrency, which _shouldn't_ be true in the case of
Linux emulated as being on a single CPU.

Perhaps that just means you chose the wrong abstraction.

In usfstl (something I've been working on) for example, we have an
abstraction called (execution) "contexts", and they can be implemented
using pthreads, fibers, or posix contexts, and you switch between them.

(see https://github.com/linux-test-project/usfstl/blob/main/src/ctx-common.c)

Using real pthreads implies that you have real threading, but then you
need access to real mutexes, etc.

If your abstraction was instead "switch context" then you could still
implement it using pthreads+mutexes, or you could implement it using
fibers on windows, or posix contexts - but you'd have a significantly
reduced API surface, since you'd only expose __switch_to() or similar,
and maybe a new stack allocation etc.

Additionally, I do wonder how UML does this now, it *does* use setjmp,
so are you saying it doesn't properly use the kernel stacks?

> And instead of manually doing lkl_run() to schedule threads and
> relying on host scheduler, LKL associates each kernel thread with a
> host-provided semaphore so that Linux scheduler has a control of host
> scheduler (prepared by pthread).

Right.

That's in line with what I did in my test framework in
https://github.com/linux-test-project/usfstl/blob/main/src/ctx-pthread.c

but like I said above, I think it's the wrong abstraction. Your
abstraction should be "switch context" (or "switch thread"), not dealing
with pthread, mutexes, etc.


> > And - kind of the theme behind all these questions - why is this not
> > making UML actually be a binary that uses LKL? If the design were like
> > what I'm alluding to above, that should actually be possible? Why should
> > it not be possible? Why would it not be desirable? (I'm actually
> > thinking that might be really useful to some of the things I'm doing.)
> > Yes, if the application actually supports userspace running then it has
> > som limitations on what it can do (in particular wrt. signals etc.), but
> > that could be documented and would be OK?
> 
> Let me try to describe how I think why not just generate liblinux.so
> from current UML.
> 
> Making UML to build a library, which has been a long wanted features,
> can be started;
> 
> 
> I think there are several functions which the library offers;
> 
> - applications can link the library and call functions in the library

Right.

> - the library will be used as a replacement of libc.a for syscall operations

Not sure I see this, is that really useful? I mean, most applications
don't live "standalone" in their own world? Dunno. Maybe it's useful.


> to design that with UML, what we need to do are;
> 
> 1) change Makefile to output liblinux.a

or liblinux.so, I guess, dynamic linking should be ok.

> we faced linker script issue, which is related with generating
> relocatable object in the middle.
> 
> 2) make the linker-script clean with 2-stage build
> we fix the linker issues of (1)
> 
> 3) expose syscall as a function call
> conflicts names (link-time and compile-time conflicts)
> 
> 4) header rename, object localization
> to fix the issue (3)
> 
> This is a common set of modifications to a library of UML.

All of this is just _build_ issues. It doesn't mean you couldn't take
some minimal code + liblinux.a and link it to get a "linux" equivalent
to the current UML?

TBH, I started thinking that it might be _really_ nice to be able to
write an application that's *not quite UML* but has all the properties
of UML built into it, i.e. can run userspace etc.

> Other parts are a choice of design, I believe.
> Because a library is more _reusable_ than an executable (by it means), the
> choice of LKL is to be portable, which the current UML doesn't pursue it
> extensibly (focus on intel platforms).
> 

I don't think this really conflicts.

You could have a liblinux.a/liblinux.so and some code that links it all
together to get "linux" (UML). Having userspace running inside the UML
(liblinux) might only be supported on x86 for now, MMU vs. NOMMU might
be something that's configurable at build time, and if you pick NOMMU
you cannot run userspace either, etc.

But conceptually, why wouldn't it be possible to have a liblinux.so that
*does* build with MMU and userspace support, and UML is a wrapper around
it?

> I hope this makes it a bit clear, but let me know if you found
> anything unclear.

See above, I guess :)

Thanks for all the discussion!

johannes


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 00/20] Unifying LKL into UML
@ 2021-03-16 21:29                 ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2021-03-16 21:29 UTC (permalink / raw)
  To: Hajime Tazaki
  Cc: linux-um, jdike, richard, anton.ivanov, tavi.purdila,
	linux-kernel-library, linux-arch, retrage01

Hi,

> First of all, thanks for all the comments to the patchset which has
> been a bit stale.  I'll reply them.

Yeah, sorry. I had it marked unread ("to look at") since you posted it.

> We didn't write down the details, which are already described in the
> LKL's paper (*1).  But I think we can extract/summarize some of
> important information from the paper to the document so that the
> design is more understandable.
> 
> *1 LKL's paper (pointer is also in the cover letter)
> https://www.researchgate.net/profile/Nicolae_Tapus2/publication/224164682_LKL_The_Linux_kernel_library/links/02bfe50fd921ab4f7c000000.pdf

OK, I guess I should take a look. Probably I never did, always thinking
that it was more of an overview than technical details and design
decisions.
> 
> My interpretation of MMU/NOMMU is like this;
> 
> With (emulated) MMU architecture you will have more smooth integration
> with other subsystems of kernel tree, because some subsystems/features
> are written with "#ifdef CONFIG_MMU".  While NOMMU doesn't, it will
> bring a simplified design with better portability.
> 
> LKL takes rather to benefit better portability.

I don't think it *matters* so much for portability? I mean, every system
under the sun is going to allow some kind of "mprotect", right? You
don't really want to port LKL to systems that don't have even that?

> >  * Why pthreads and all? You already require jump_buf, so UML's
> >    switch_threads() ought to be just fine for scheduling? It almost
> >    seems like you're doing this just so you can serialize against "other
> >    threads" (application threads), but wouldn't that trivially be
> >    handled by the application? You could let it hook into switch_to() or
> >    something, but why should a single "LKL" CPU ever require multiple
> >    threads? Seems to me that the userspace could be required to
> >    "lkl_run()" or so (vs. lkl_start()). Heck, you could even exit
> >    lkl_run() every time you switch tasks in the kernel, and leave
> >    scheduling the kernel vs. the application entirely up to the
> >    application? (A trivial application would be simply doing something
> >    like "while (1) { lkl_run(); pause(); }" mimicking the idle loop of
> >    UML.
> 
> There is a description about this design choice in the LKL paper (*1);
> 
>   "implementations based on setjmp - longjmp require usage of a single
>   stack space partitioned between all threads. As the Linux kernel
>   uses deep stacks (especially in the VFS layer), in an environment
>   with small stack sizes (e.g. inside another operating system's
>   kernel) this will place a very low limit on the number of possible
>   threads."
> 
> (from page 2, Section II, 2) Thread Support)
> 
> This is a reason of using pthread as a context primitive.

That impliciation (setjmp doesnt do stacks, so must use pthread) really
isn't true, you also have posix contexts or windows fibers. That would
probably be much easier to understands, since real threads imply that
you have actual concurrency, which _shouldn't_ be true in the case of
Linux emulated as being on a single CPU.

Perhaps that just means you chose the wrong abstraction.

In usfstl (something I've been working on) for example, we have an
abstraction called (execution) "contexts", and they can be implemented
using pthreads, fibers, or posix contexts, and you switch between them.

(see https://github.com/linux-test-project/usfstl/blob/main/src/ctx-common.c)

Using real pthreads implies that you have real threading, but then you
need access to real mutexes, etc.

If your abstraction was instead "switch context" then you could still
implement it using pthreads+mutexes, or you could implement it using
fibers on windows, or posix contexts - but you'd have a significantly
reduced API surface, since you'd only expose __switch_to() or similar,
and maybe a new stack allocation etc.

Additionally, I do wonder how UML does this now, it *does* use setjmp,
so are you saying it doesn't properly use the kernel stacks?

> And instead of manually doing lkl_run() to schedule threads and
> relying on host scheduler, LKL associates each kernel thread with a
> host-provided semaphore so that Linux scheduler has a control of host
> scheduler (prepared by pthread).

Right.

That's in line with what I did in my test framework in
https://github.com/linux-test-project/usfstl/blob/main/src/ctx-pthread.c

but like I said above, I think it's the wrong abstraction. Your
abstraction should be "switch context" (or "switch thread"), not dealing
with pthread, mutexes, etc.


> > And - kind of the theme behind all these questions - why is this not
> > making UML actually be a binary that uses LKL? If the design were like
> > what I'm alluding to above, that should actually be possible? Why should
> > it not be possible? Why would it not be desirable? (I'm actually
> > thinking that might be really useful to some of the things I'm doing.)
> > Yes, if the application actually supports userspace running then it has
> > som limitations on what it can do (in particular wrt. signals etc.), but
> > that could be documented and would be OK?
> 
> Let me try to describe how I think why not just generate liblinux.so
> from current UML.
> 
> Making UML to build a library, which has been a long wanted features,
> can be started;
> 
> 
> I think there are several functions which the library offers;
> 
> - applications can link the library and call functions in the library

Right.

> - the library will be used as a replacement of libc.a for syscall operations

Not sure I see this, is that really useful? I mean, most applications
don't live "standalone" in their own world? Dunno. Maybe it's useful.


> to design that with UML, what we need to do are;
> 
> 1) change Makefile to output liblinux.a

or liblinux.so, I guess, dynamic linking should be ok.

> we faced linker script issue, which is related with generating
> relocatable object in the middle.
> 
> 2) make the linker-script clean with 2-stage build
> we fix the linker issues of (1)
> 
> 3) expose syscall as a function call
> conflicts names (link-time and compile-time conflicts)
> 
> 4) header rename, object localization
> to fix the issue (3)
> 
> This is a common set of modifications to a library of UML.

All of this is just _build_ issues. It doesn't mean you couldn't take
some minimal code + liblinux.a and link it to get a "linux" equivalent
to the current UML?

TBH, I started thinking that it might be _really_ nice to be able to
write an application that's *not quite UML* but has all the properties
of UML built into it, i.e. can run userspace etc.

> Other parts are a choice of design, I believe.
> Because a library is more _reusable_ than an executable (by it means), the
> choice of LKL is to be portable, which the current UML doesn't pursue it
> extensibly (focus on intel platforms).
> 

I don't think this really conflicts.

You could have a liblinux.a/liblinux.so and some code that links it all
together to get "linux" (UML). Having userspace running inside the UML
(liblinux) might only be supported on x86 for now, MMU vs. NOMMU might
be something that's configurable at build time, and if you pick NOMMU
you cannot run userspace either, etc.

But conceptually, why wouldn't it be possible to have a liblinux.so that
*does* build with MMU and userspace support, and UML is a wrapper around
it?

> I hope this makes it a bit clear, but let me know if you found
> anything unclear.

See above, I guess :)

Thanks for all the discussion!

johannes


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 08/20] um: lkl: memory handling
  2021-03-16  1:18                 ` Hajime Tazaki
@ 2021-03-16 21:31                   ` Johannes Berg
  -1 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2021-03-16 21:31 UTC (permalink / raw)
  To: Hajime Tazaki
  Cc: linux-um, jdike, richard, anton.ivanov, tavi.purdila,
	linux-kernel-library, linux-arch, retrage01

On Tue, 2021-03-16 at 10:18 +0900, Hajime Tazaki wrote:
> 
> > > +void *uml_kmalloc(int size, int flags)
> > > +{
> > > +	return kmalloc(size, flags);
> > > +}
> > 
> > That could probably still be shared?
> 
> This function is a stub of arch/um/kernel/mem.c, which LKL doesn't
> use for the build.  Thus we defined here.
> 
> Or are you suggesting to not stubbing this function, but extracting
> the function from mem.c ?

Yes, that's kind of what I was thinking.

OTOH, I guess it's in os-Linux today?

johannes


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 08/20] um: lkl: memory handling
@ 2021-03-16 21:31                   ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2021-03-16 21:31 UTC (permalink / raw)
  To: Hajime Tazaki
  Cc: linux-um, jdike, richard, anton.ivanov, tavi.purdila,
	linux-kernel-library, linux-arch, retrage01

On Tue, 2021-03-16 at 10:18 +0900, Hajime Tazaki wrote:
> 
> > > +void *uml_kmalloc(int size, int flags)
> > > +{
> > > +	return kmalloc(size, flags);
> > > +}
> > 
> > That could probably still be shared?
> 
> This function is a stub of arch/um/kernel/mem.c, which LKL doesn't
> use for the build.  Thus we defined here.
> 
> Or are you suggesting to not stubbing this function, but extracting
> the function from mem.c ?

Yes, that's kind of what I was thinking.

OTOH, I guess it's in os-Linux today?

johannes


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 19/20] um: lkl: add block device support of UML
  2021-03-16  1:19                 ` Hajime Tazaki
@ 2021-03-16 21:32                   ` Johannes Berg
  -1 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2021-03-16 21:32 UTC (permalink / raw)
  To: Hajime Tazaki
  Cc: linux-um, jdike, richard, anton.ivanov, tavi.purdila,
	linux-kernel-library, linux-arch, retrage01

On Tue, 2021-03-16 at 10:19 +0900, Hajime Tazaki wrote:
> 
> > > --- a/arch/um/Kconfig
> > > +++ b/arch/um/Kconfig
> > > @@ -29,6 +29,10 @@ config UMMODE_LIB
> > >  	select UACCESS_MEMCPY
> > >  	select ARCH_THREAD_STACK_ALLOCATOR
> > >  	select ARCH_HAS_SYSCALL_WRAPPER
> > > +	select VFAT_FS
> > > +	select NLS_CODEPAGE_437
> > > +	select NLS_ISO8859_1
> > > +	select BTRFS_FS
> > 
> > That doesn't really seem to make sense - the sample might need it, but
> > generally LKL doesn't/shouldn't?
> 
> I'm trying to understand your comment;
> Do you mean that enabling those options in Kconfig doesn't make sense ?

I mean *always* enabling them doesn't make sense. Why shouldn't somebody
be allowed to build UMMODE_LIB *without* btrfs? If they have no need for
btrfs, why should it select it?

I can understand that your sample code wants btrfs just to show what it
can do, but IMHO it doesn't make sense to *always* enable it.

> and if you mean the sample as sample code, is the added test case
> (e.g., tools/testing/selftests/um/disk.c, which is included in the
> same patch) for this purpose ?

yes, that's what I mean by "sample code"

joahnnes


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 19/20] um: lkl: add block device support of UML
@ 2021-03-16 21:32                   ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2021-03-16 21:32 UTC (permalink / raw)
  To: Hajime Tazaki
  Cc: linux-um, jdike, richard, anton.ivanov, tavi.purdila,
	linux-kernel-library, linux-arch, retrage01

On Tue, 2021-03-16 at 10:19 +0900, Hajime Tazaki wrote:
> 
> > > --- a/arch/um/Kconfig
> > > +++ b/arch/um/Kconfig
> > > @@ -29,6 +29,10 @@ config UMMODE_LIB
> > >  	select UACCESS_MEMCPY
> > >  	select ARCH_THREAD_STACK_ALLOCATOR
> > >  	select ARCH_HAS_SYSCALL_WRAPPER
> > > +	select VFAT_FS
> > > +	select NLS_CODEPAGE_437
> > > +	select NLS_ISO8859_1
> > > +	select BTRFS_FS
> > 
> > That doesn't really seem to make sense - the sample might need it, but
> > generally LKL doesn't/shouldn't?
> 
> I'm trying to understand your comment;
> Do you mean that enabling those options in Kconfig doesn't make sense ?

I mean *always* enabling them doesn't make sense. Why shouldn't somebody
be allowed to build UMMODE_LIB *without* btrfs? If they have no need for
btrfs, why should it select it?

I can understand that your sample code wants btrfs just to show what it
can do, but IMHO it doesn't make sense to *always* enable it.

> and if you mean the sample as sample code, is the added test case
> (e.g., tools/testing/selftests/um/disk.c, which is included in the
> same patch) for this purpose ?

yes, that's what I mean by "sample code"

joahnnes


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 13/20] um: lkl: integrate with irq infrastructure of UML
  2021-03-16  1:20                 ` Hajime Tazaki
@ 2021-03-16 21:36                   ` Johannes Berg
  -1 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2021-03-16 21:36 UTC (permalink / raw)
  To: Hajime Tazaki
  Cc: linux-um, jdike, richard, anton.ivanov, tavi.purdila,
	linux-kernel-library, linux-arch, retrage01

On Tue, 2021-03-16 at 10:20 +0900, Hajime Tazaki wrote:
> 
> > > -	if (sigprocmask(SIG_UNBLOCK, &sig_mask, NULL) < 0)
> > > -		panic("sigprocmask failed - errno = %d\n", errno);
> > > +	if (pthread_sigmask(SIG_UNBLOCK, &sig_mask, NULL) < 0)
> > > +		panic("pthread_sigmask failed - errno = %d\n", errno);
> > 
> > UML doesn't normally link with libpthread, and LKL doesn't actually
> > appear to require it either (since it has its lkl_thread and all), so
> > this seems wrong?
> 
> I think both UML/LKL link with libpthread.  See old
> scripts/link-vmlinux.sh, or [01/20] patch.
> 
> -		${CC} ${CFLAGS_vmlinux}				\
> -			${strip_debug}				\
> -			-o ${output}				\
> -			-Wl,-T,${lds}				\
> -			${objects}				\
> -			-lutil -lrt -lpthread
> -		rm -f linux

Interesting. You're right, I really didn't expect that, nor did I ever
see that before.

How does this interact with all the clone() calls etc. that UML does?

johannes


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 13/20] um: lkl: integrate with irq infrastructure of UML
@ 2021-03-16 21:36                   ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2021-03-16 21:36 UTC (permalink / raw)
  To: Hajime Tazaki
  Cc: linux-um, jdike, richard, anton.ivanov, tavi.purdila,
	linux-kernel-library, linux-arch, retrage01

On Tue, 2021-03-16 at 10:20 +0900, Hajime Tazaki wrote:
> 
> > > -	if (sigprocmask(SIG_UNBLOCK, &sig_mask, NULL) < 0)
> > > -		panic("sigprocmask failed - errno = %d\n", errno);
> > > +	if (pthread_sigmask(SIG_UNBLOCK, &sig_mask, NULL) < 0)
> > > +		panic("pthread_sigmask failed - errno = %d\n", errno);
> > 
> > UML doesn't normally link with libpthread, and LKL doesn't actually
> > appear to require it either (since it has its lkl_thread and all), so
> > this seems wrong?
> 
> I think both UML/LKL link with libpthread.  See old
> scripts/link-vmlinux.sh, or [01/20] patch.
> 
> -		${CC} ${CFLAGS_vmlinux}				\
> -			${strip_debug}				\
> -			-o ${output}				\
> -			-Wl,-T,${lds}				\
> -			${objects}				\
> -			-lutil -lrt -lpthread
> -		rm -f linux

Interesting. You're right, I really didn't expect that, nor did I ever
see that before.

How does this interact with all the clone() calls etc. that UML does?

johannes


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 00/20] Unifying LKL into UML
  2021-03-16 21:29                 ` Johannes Berg
@ 2021-03-17 14:03                   ` Octavian Purdila
  -1 siblings, 0 replies; 476+ messages in thread
From: Octavian Purdila @ 2021-03-17 14:03 UTC (permalink / raw)
  To: Johannes Berg
  Cc: Hajime Tazaki, linux-um, jdike, Richard Weinberger, Anton Ivanov,
	linux-kernel-library, linux-arch, Akira Moroo

On Tue, Mar 16, 2021 at 11:29 PM Johannes Berg
<johannes@sipsolutions.net> wrote:
>
> Hi,

Hi Johannes,

> > My interpretation of MMU/NOMMU is like this;
> >
> > With (emulated) MMU architecture you will have more smooth integration
> > with other subsystems of kernel tree, because some subsystems/features
> > are written with "#ifdef CONFIG_MMU".  While NOMMU doesn't, it will
> > bring a simplified design with better portability.
> >
> > LKL takes rather to benefit better portability.
>
> I don't think it *matters* so much for portability? I mean, every system
> under the sun is going to allow some kind of "mprotect", right? You
> don't really want to port LKL to systems that don't have even that?
>

One use case where this matters are non OS environments such as
bootloaders [1], running on bare-bone hardware or kernel drivers [2,
3]. IMO it would be nice to keep these properties.

[1] https://www.freelists.org/post/linux-kernel-library/UEFI-LKL-port
[2] https://github.com/lkl/lkl-win-fsd
[3] https://www.haiku-os.org/tags/lkl-haiku-fsd/

> > >  * Why pthreads and all? You already require jump_buf, so UML's
> > >    switch_threads() ought to be just fine for scheduling? It almost
> > >    seems like you're doing this just so you can serialize against "other
> > >    threads" (application threads), but wouldn't that trivially be
> > >    handled by the application? You could let it hook into switch_to() or
> > >    something, but why should a single "LKL" CPU ever require multiple
> > >    threads? Seems to me that the userspace could be required to
> > >    "lkl_run()" or so (vs. lkl_start()). Heck, you could even exit
> > >    lkl_run() every time you switch tasks in the kernel, and leave
> > >    scheduling the kernel vs. the application entirely up to the
> > >    application? (A trivial application would be simply doing something
> > >    like "while (1) { lkl_run(); pause(); }" mimicking the idle loop of
> > >    UML.
> >
> > There is a description about this design choice in the LKL paper (*1);
> >
> >   "implementations based on setjmp - longjmp require usage of a single
> >   stack space partitioned between all threads. As the Linux kernel
> >   uses deep stacks (especially in the VFS layer), in an environment
> >   with small stack sizes (e.g. inside another operating system's
> >   kernel) this will place a very low limit on the number of possible
> >   threads."
> >
> > (from page 2, Section II, 2) Thread Support)
> >
> > This is a reason of using pthread as a context primitive.
>
> That impliciation (setjmp doesnt do stacks, so must use pthread) really
> isn't true, you also have posix contexts or windows fibers. That would
> probably be much easier to understands, since real threads imply that
> you have actual concurrency, which _shouldn't_ be true in the case of
> Linux emulated as being on a single CPU.
>
> Perhaps that just means you chose the wrong abstraction.
>
> In usfstl (something I've been working on) for example, we have an
> abstraction called (execution) "contexts", and they can be implemented
> using pthreads, fibers, or posix contexts, and you switch between them.
>
> (see https://github.com/linux-test-project/usfstl/blob/main/src/ctx-common.c)
>
> Using real pthreads implies that you have real threading, but then you
> need access to real mutexes, etc.
>
> If your abstraction was instead "switch context" then you could still
> implement it using pthreads+mutexes, or you could implement it using
> fibers on windows, or posix contexts - but you'd have a significantly
> reduced API surface, since you'd only expose __switch_to() or similar,
> and maybe a new stack allocation etc.
>

You are right. When I started the implementation for ucontext it was
obvious that it would be much simpler to have abstractions closer to
what Linux has (alloc, free and switch threads). But I never got to
finish that and then things went into a different direction.

> Additionally, I do wonder how UML does this now, it *does* use setjmp,
> so are you saying it doesn't properly use the kernel stacks?
>

To clarify a bit the statement in the paper, the context there was
that we should push the thread implementation to the
application/environment we run rather than providing "LKL" threads.
This was particularly important for running LKL in other OSes kernel
drivers. But you are right, we can use the switch abstraction and
implement it with threads and mutexes for those environments where it
helps.

> > to design that with UML, what we need to do are;
> >
> > 1) change Makefile to output liblinux.a
>
> or liblinux.so, I guess, dynamic linking should be ok.
>
> > we faced linker script issue, which is related with generating
> > relocatable object in the middle.
> >
> > 2) make the linker-script clean with 2-stage build
> > we fix the linker issues of (1)
> >
> > 3) expose syscall as a function call
> > conflicts names (link-time and compile-time conflicts)
> >
> > 4) header rename, object localization
> > to fix the issue (3)
> >
> > This is a common set of modifications to a library of UML.
>
> All of this is just _build_ issues. It doesn't mean you couldn't take
> some minimal code + liblinux.a and link it to get a "linux" equivalent
> to the current UML?
>
> TBH, I started thinking that it might be _really_ nice to be able to
> write an application that's *not quite UML* but has all the properties
> of UML built into it, i.e. can run userspace etc.
>
> > Other parts are a choice of design, I believe.
> > Because a library is more _reusable_ than an executable (by it means), the
> > choice of LKL is to be portable, which the current UML doesn't pursue it
> > extensibly (focus on intel platforms).
> >
>
> I don't think this really conflicts.
>
> You could have a liblinux.a/liblinux.so and some code that links it all
> together to get "linux" (UML). Having userspace running inside the UML
> (liblinux) might only be supported on x86 for now, MMU vs. NOMMU might
> be something that's configurable at build time, and if you pick NOMMU
> you cannot run userspace either, etc.
>
> But conceptually, why wouldn't it be possible to have a liblinux.so that
> *does* build with MMU and userspace support, and UML is a wrapper around
> it?
>

This is an interesting idea. Conceptually I think it is possible.
There are lots of details to be figured out before we do this. I think
that having a NOMMU version could be a good step in the right
direction, especially since I think a liblinux.so has more NOMMU
usecases than MMU usecases - but I haven't given too much thought to
the MMU usecases.

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 00/20] Unifying LKL into UML
@ 2021-03-17 14:03                   ` Octavian Purdila
  0 siblings, 0 replies; 476+ messages in thread
From: Octavian Purdila @ 2021-03-17 14:03 UTC (permalink / raw)
  To: Johannes Berg
  Cc: Hajime Tazaki, linux-um, jdike, Richard Weinberger, Anton Ivanov,
	linux-kernel-library, linux-arch, Akira Moroo

On Tue, Mar 16, 2021 at 11:29 PM Johannes Berg
<johannes@sipsolutions.net> wrote:
>
> Hi,

Hi Johannes,

> > My interpretation of MMU/NOMMU is like this;
> >
> > With (emulated) MMU architecture you will have more smooth integration
> > with other subsystems of kernel tree, because some subsystems/features
> > are written with "#ifdef CONFIG_MMU".  While NOMMU doesn't, it will
> > bring a simplified design with better portability.
> >
> > LKL takes rather to benefit better portability.
>
> I don't think it *matters* so much for portability? I mean, every system
> under the sun is going to allow some kind of "mprotect", right? You
> don't really want to port LKL to systems that don't have even that?
>

One use case where this matters are non OS environments such as
bootloaders [1], running on bare-bone hardware or kernel drivers [2,
3]. IMO it would be nice to keep these properties.

[1] https://www.freelists.org/post/linux-kernel-library/UEFI-LKL-port
[2] https://github.com/lkl/lkl-win-fsd
[3] https://www.haiku-os.org/tags/lkl-haiku-fsd/

> > >  * Why pthreads and all? You already require jump_buf, so UML's
> > >    switch_threads() ought to be just fine for scheduling? It almost
> > >    seems like you're doing this just so you can serialize against "other
> > >    threads" (application threads), but wouldn't that trivially be
> > >    handled by the application? You could let it hook into switch_to() or
> > >    something, but why should a single "LKL" CPU ever require multiple
> > >    threads? Seems to me that the userspace could be required to
> > >    "lkl_run()" or so (vs. lkl_start()). Heck, you could even exit
> > >    lkl_run() every time you switch tasks in the kernel, and leave
> > >    scheduling the kernel vs. the application entirely up to the
> > >    application? (A trivial application would be simply doing something
> > >    like "while (1) { lkl_run(); pause(); }" mimicking the idle loop of
> > >    UML.
> >
> > There is a description about this design choice in the LKL paper (*1);
> >
> >   "implementations based on setjmp - longjmp require usage of a single
> >   stack space partitioned between all threads. As the Linux kernel
> >   uses deep stacks (especially in the VFS layer), in an environment
> >   with small stack sizes (e.g. inside another operating system's
> >   kernel) this will place a very low limit on the number of possible
> >   threads."
> >
> > (from page 2, Section II, 2) Thread Support)
> >
> > This is a reason of using pthread as a context primitive.
>
> That impliciation (setjmp doesnt do stacks, so must use pthread) really
> isn't true, you also have posix contexts or windows fibers. That would
> probably be much easier to understands, since real threads imply that
> you have actual concurrency, which _shouldn't_ be true in the case of
> Linux emulated as being on a single CPU.
>
> Perhaps that just means you chose the wrong abstraction.
>
> In usfstl (something I've been working on) for example, we have an
> abstraction called (execution) "contexts", and they can be implemented
> using pthreads, fibers, or posix contexts, and you switch between them.
>
> (see https://github.com/linux-test-project/usfstl/blob/main/src/ctx-common.c)
>
> Using real pthreads implies that you have real threading, but then you
> need access to real mutexes, etc.
>
> If your abstraction was instead "switch context" then you could still
> implement it using pthreads+mutexes, or you could implement it using
> fibers on windows, or posix contexts - but you'd have a significantly
> reduced API surface, since you'd only expose __switch_to() or similar,
> and maybe a new stack allocation etc.
>

You are right. When I started the implementation for ucontext it was
obvious that it would be much simpler to have abstractions closer to
what Linux has (alloc, free and switch threads). But I never got to
finish that and then things went into a different direction.

> Additionally, I do wonder how UML does this now, it *does* use setjmp,
> so are you saying it doesn't properly use the kernel stacks?
>

To clarify a bit the statement in the paper, the context there was
that we should push the thread implementation to the
application/environment we run rather than providing "LKL" threads.
This was particularly important for running LKL in other OSes kernel
drivers. But you are right, we can use the switch abstraction and
implement it with threads and mutexes for those environments where it
helps.

> > to design that with UML, what we need to do are;
> >
> > 1) change Makefile to output liblinux.a
>
> or liblinux.so, I guess, dynamic linking should be ok.
>
> > we faced linker script issue, which is related with generating
> > relocatable object in the middle.
> >
> > 2) make the linker-script clean with 2-stage build
> > we fix the linker issues of (1)
> >
> > 3) expose syscall as a function call
> > conflicts names (link-time and compile-time conflicts)
> >
> > 4) header rename, object localization
> > to fix the issue (3)
> >
> > This is a common set of modifications to a library of UML.
>
> All of this is just _build_ issues. It doesn't mean you couldn't take
> some minimal code + liblinux.a and link it to get a "linux" equivalent
> to the current UML?
>
> TBH, I started thinking that it might be _really_ nice to be able to
> write an application that's *not quite UML* but has all the properties
> of UML built into it, i.e. can run userspace etc.
>
> > Other parts are a choice of design, I believe.
> > Because a library is more _reusable_ than an executable (by it means), the
> > choice of LKL is to be portable, which the current UML doesn't pursue it
> > extensibly (focus on intel platforms).
> >
>
> I don't think this really conflicts.
>
> You could have a liblinux.a/liblinux.so and some code that links it all
> together to get "linux" (UML). Having userspace running inside the UML
> (liblinux) might only be supported on x86 for now, MMU vs. NOMMU might
> be something that's configurable at build time, and if you pick NOMMU
> you cannot run userspace either, etc.
>
> But conceptually, why wouldn't it be possible to have a liblinux.so that
> *does* build with MMU and userspace support, and UML is a wrapper around
> it?
>

This is an interesting idea. Conceptually I think it is possible.
There are lots of details to be figured out before we do this. I think
that having a NOMMU version could be a good step in the right
direction, especially since I think a liblinux.so has more NOMMU
usecases than MMU usecases - but I haven't given too much thought to
the MMU usecases.

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 19/20] um: lkl: add block device support of UML
  2021-03-16 21:32                   ` Johannes Berg
@ 2021-03-17 14:19                     ` Octavian Purdila
  -1 siblings, 0 replies; 476+ messages in thread
From: Octavian Purdila @ 2021-03-17 14:19 UTC (permalink / raw)
  To: Johannes Berg
  Cc: Hajime Tazaki, linux-um, jdike, Richard Weinberger, Anton Ivanov,
	linux-kernel-library, linux-arch, Akira Moroo

On Tue, Mar 16, 2021 at 11:32 PM Johannes Berg
<johannes@sipsolutions.net> wrote:
>
> On Tue, 2021-03-16 at 10:19 +0900, Hajime Tazaki wrote:
> >
> > > > --- a/arch/um/Kconfig
> > > > +++ b/arch/um/Kconfig
> > > > @@ -29,6 +29,10 @@ config UMMODE_LIB
> > > >   select UACCESS_MEMCPY
> > > >   select ARCH_THREAD_STACK_ALLOCATOR
> > > >   select ARCH_HAS_SYSCALL_WRAPPER
> > > > + select VFAT_FS
> > > > + select NLS_CODEPAGE_437
> > > > + select NLS_ISO8859_1
> > > > + select BTRFS_FS
> > >
> > > That doesn't really seem to make sense - the sample might need it, but
> > > generally LKL doesn't/shouldn't?
> >
> > I'm trying to understand your comment;
> > Do you mean that enabling those options in Kconfig doesn't make sense ?
>
> I mean *always* enabling them doesn't make sense. Why shouldn't somebody
> be allowed to build UMMODE_LIB *without* btrfs? If they have no need for
> btrfs, why should it select it?
>
> I can understand that your sample code wants btrfs just to show what it
> can do, but IMHO it doesn't make sense to *always* enable it.
>

I agree. I think these can stay in defconfigs but here is where a
library introduces complications which I don't know how is best to
handle. Should we have the defconfig in arch/um or should we have it
in tools/testing/selftests/um? Or perhaps both places, one being a
generic config that would be used by most applications and one
customized?

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 19/20] um: lkl: add block device support of UML
@ 2021-03-17 14:19                     ` Octavian Purdila
  0 siblings, 0 replies; 476+ messages in thread
From: Octavian Purdila @ 2021-03-17 14:19 UTC (permalink / raw)
  To: Johannes Berg
  Cc: Hajime Tazaki, linux-um, jdike, Richard Weinberger, Anton Ivanov,
	linux-kernel-library, linux-arch, Akira Moroo

On Tue, Mar 16, 2021 at 11:32 PM Johannes Berg
<johannes@sipsolutions.net> wrote:
>
> On Tue, 2021-03-16 at 10:19 +0900, Hajime Tazaki wrote:
> >
> > > > --- a/arch/um/Kconfig
> > > > +++ b/arch/um/Kconfig
> > > > @@ -29,6 +29,10 @@ config UMMODE_LIB
> > > >   select UACCESS_MEMCPY
> > > >   select ARCH_THREAD_STACK_ALLOCATOR
> > > >   select ARCH_HAS_SYSCALL_WRAPPER
> > > > + select VFAT_FS
> > > > + select NLS_CODEPAGE_437
> > > > + select NLS_ISO8859_1
> > > > + select BTRFS_FS
> > >
> > > That doesn't really seem to make sense - the sample might need it, but
> > > generally LKL doesn't/shouldn't?
> >
> > I'm trying to understand your comment;
> > Do you mean that enabling those options in Kconfig doesn't make sense ?
>
> I mean *always* enabling them doesn't make sense. Why shouldn't somebody
> be allowed to build UMMODE_LIB *without* btrfs? If they have no need for
> btrfs, why should it select it?
>
> I can understand that your sample code wants btrfs just to show what it
> can do, but IMHO it doesn't make sense to *always* enable it.
>

I agree. I think these can stay in defconfigs but here is where a
library introduces complications which I don't know how is best to
handle. Should we have the defconfig in arch/um or should we have it
in tools/testing/selftests/um? Or perhaps both places, one being a
generic config that would be used by most applications and one
customized?

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 00/20] Unifying LKL into UML
  2021-03-17 14:03                   ` Octavian Purdila
@ 2021-03-17 14:24                     ` Johannes Berg
  -1 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2021-03-17 14:24 UTC (permalink / raw)
  To: Octavian Purdila
  Cc: Hajime Tazaki, linux-um, jdike, Richard Weinberger, Anton Ivanov,
	linux-kernel-library, linux-arch, Akira Moroo

Hi,

> One use case where this matters are non OS environments such as
> bootloaders [1], running on bare-bone hardware or kernel drivers [2,
> 3]. IMO it would be nice to keep these properties.

OK, that makes sense. Still, it seems it could be a compile-time
decision, and doesn't necessarily mean LKL has to be NOMMU, just that it
could support both?

I'm really trying to see if we can't get UML to be a user of LKL. IMHO
that would be good for the code, and even be good for LKL since then
it's maintained as part of UML as well, not "just" as its own use case.

> > If your abstraction was instead "switch context" then you could still
> > implement it using pthreads+mutexes, or you could implement it using
> > fibers on windows, or posix contexts - but you'd have a significantly
> > reduced API surface, since you'd only expose __switch_to() or similar,
> > and maybe a new stack allocation etc.
> 
> You are right. When I started the implementation for ucontext it was
> obvious that it would be much simpler to have abstractions closer to
> what Linux has (alloc, free and switch threads). But I never got to
> finish that and then things went into a different direction.

OK, sounds like you came to the same conclusion, more or less.

> > Additionally, I do wonder how UML does this now, it *does* use setjmp,
> > so are you saying it doesn't properly use the kernel stacks?
> > 
> 
> To clarify a bit the statement in the paper, the context there was
> that we should push the thread implementation to the
> application/environment we run rather than providing "LKL" threads.
> This was particularly important for running LKL in other OSes kernel
> drivers. But you are right, we can use the switch abstraction and
> implement it with threads and mutexes for those environments where it
> helps.

Right - like I pointed to USFSTL framework, you could have posix
ucontext, fiber and pthread at least, and obviously other things in
other environments (ThreadX anyone? ;-) )

> > But conceptually, why wouldn't it be possible to have a liblinux.so that
> > *does* build with MMU and userspace support, and UML is a wrapper around
> > it?
> > 
> 
> This is an interesting idea. Conceptually I think it is possible.
> There are lots of details to be figured out before we do this. I think
> that having a NOMMU version could be a good step in the right
> direction, especially since I think a liblinux.so has more NOMMU
> usecases than MMU usecases - but I haven't given too much thought to
> the MMU usecases.

Yeah, maybe UML would be the primary use case. I have been thinking that
there would be cases where you could combine kunit and having userspace
though, or unit-style testing but not with kunit which is "inside" the
kernel, but instead having the test code more "outside" the test kernel.
That's all kind of handwaving though and not really that crystallized in
my mind.

That said, I'm not entirely sure NOMMU would be the right path towards
this - if we do want to go this route it'll probably need changes in
both LKL and UML to converge to this point, and at least build it into
the abstractions.

For example the "idle" abstraction discussed elsewhere (is it part of
the app or part of the kernel?), or the thread discussion above (it is
part of the app but how is it implemented?) etc.

johannes



^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 00/20] Unifying LKL into UML
@ 2021-03-17 14:24                     ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2021-03-17 14:24 UTC (permalink / raw)
  To: Octavian Purdila
  Cc: Hajime Tazaki, linux-um, jdike, Richard Weinberger, Anton Ivanov,
	linux-kernel-library, linux-arch, Akira Moroo

Hi,

> One use case where this matters are non OS environments such as
> bootloaders [1], running on bare-bone hardware or kernel drivers [2,
> 3]. IMO it would be nice to keep these properties.

OK, that makes sense. Still, it seems it could be a compile-time
decision, and doesn't necessarily mean LKL has to be NOMMU, just that it
could support both?

I'm really trying to see if we can't get UML to be a user of LKL. IMHO
that would be good for the code, and even be good for LKL since then
it's maintained as part of UML as well, not "just" as its own use case.

> > If your abstraction was instead "switch context" then you could still
> > implement it using pthreads+mutexes, or you could implement it using
> > fibers on windows, or posix contexts - but you'd have a significantly
> > reduced API surface, since you'd only expose __switch_to() or similar,
> > and maybe a new stack allocation etc.
> 
> You are right. When I started the implementation for ucontext it was
> obvious that it would be much simpler to have abstractions closer to
> what Linux has (alloc, free and switch threads). But I never got to
> finish that and then things went into a different direction.

OK, sounds like you came to the same conclusion, more or less.

> > Additionally, I do wonder how UML does this now, it *does* use setjmp,
> > so are you saying it doesn't properly use the kernel stacks?
> > 
> 
> To clarify a bit the statement in the paper, the context there was
> that we should push the thread implementation to the
> application/environment we run rather than providing "LKL" threads.
> This was particularly important for running LKL in other OSes kernel
> drivers. But you are right, we can use the switch abstraction and
> implement it with threads and mutexes for those environments where it
> helps.

Right - like I pointed to USFSTL framework, you could have posix
ucontext, fiber and pthread at least, and obviously other things in
other environments (ThreadX anyone? ;-) )

> > But conceptually, why wouldn't it be possible to have a liblinux.so that
> > *does* build with MMU and userspace support, and UML is a wrapper around
> > it?
> > 
> 
> This is an interesting idea. Conceptually I think it is possible.
> There are lots of details to be figured out before we do this. I think
> that having a NOMMU version could be a good step in the right
> direction, especially since I think a liblinux.so has more NOMMU
> usecases than MMU usecases - but I haven't given too much thought to
> the MMU usecases.

Yeah, maybe UML would be the primary use case. I have been thinking that
there would be cases where you could combine kunit and having userspace
though, or unit-style testing but not with kunit which is "inside" the
kernel, but instead having the test code more "outside" the test kernel.
That's all kind of handwaving though and not really that crystallized in
my mind.

That said, I'm not entirely sure NOMMU would be the right path towards
this - if we do want to go this route it'll probably need changes in
both LKL and UML to converge to this point, and at least build it into
the abstractions.

For example the "idle" abstraction discussed elsewhere (is it part of
the app or part of the kernel?), or the thread discussion above (it is
part of the app but how is it implemented?) etc.

johannes



_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 19/20] um: lkl: add block device support of UML
  2021-03-17 14:19                     ` Octavian Purdila
@ 2021-03-17 14:28                       ` Johannes Berg
  -1 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2021-03-17 14:28 UTC (permalink / raw)
  To: Octavian Purdila
  Cc: Hajime Tazaki, linux-um, jdike, Richard Weinberger, Anton Ivanov,
	linux-kernel-library, linux-arch, Akira Moroo

On Wed, 2021-03-17 at 16:19 +0200, Octavian Purdila wrote:
> > I can understand that your sample code wants btrfs just to show what it
> > can do, but IMHO it doesn't make sense to *always* enable it.

Btw, what I said there also didn't distinguish between "always enable
it" and "always force it enabled".

Right now this patch did the latter, but the former might sort of make
sense, but would take the form of a defconfig rather than a Kconfig code
change.

> 
> I agree. I think these can stay in defconfigs but here is where a
> library introduces complications which I don't know how is best to
> handle. Should we have the defconfig in arch/um or should we have it
> in tools/testing/selftests/um? Or perhaps both places, one being a
> generic config that would be used by most applications and one
> customized?

Yeah, that's a question - and in that sense LKL will never be a general-
purpose "library" since then you'd have to basically build it with
"allyesconfig" instead of other things.

Maybe just put a note there with the tools, and maybe a defconfig, and
then have some kind of detection at example/tool build or even runtime
that the necessary options are selected?

johannes


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 19/20] um: lkl: add block device support of UML
@ 2021-03-17 14:28                       ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2021-03-17 14:28 UTC (permalink / raw)
  To: Octavian Purdila
  Cc: Hajime Tazaki, linux-um, jdike, Richard Weinberger, Anton Ivanov,
	linux-kernel-library, linux-arch, Akira Moroo

On Wed, 2021-03-17 at 16:19 +0200, Octavian Purdila wrote:
> > I can understand that your sample code wants btrfs just to show what it
> > can do, but IMHO it doesn't make sense to *always* enable it.

Btw, what I said there also didn't distinguish between "always enable
it" and "always force it enabled".

Right now this patch did the latter, but the former might sort of make
sense, but would take the form of a defconfig rather than a Kconfig code
change.

> 
> I agree. I think these can stay in defconfigs but here is where a
> library introduces complications which I don't know how is best to
> handle. Should we have the defconfig in arch/um or should we have it
> in tools/testing/selftests/um? Or perhaps both places, one being a
> generic config that would be used by most applications and one
> customized?

Yeah, that's a question - and in that sense LKL will never be a general-
purpose "library" since then you'd have to basically build it with
"allyesconfig" instead of other things.

Maybe just put a note there with the tools, and maybe a defconfig, and
then have some kind of detection at example/tool build or even runtime
that the necessary options are selected?

johannes


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 08/20] um: lkl: memory handling
  2021-03-16 21:31                   ` Johannes Berg
@ 2021-03-18  0:12                     ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-03-18  0:12 UTC (permalink / raw)
  To: johannes
  Cc: linux-um, jdike, richard, anton.ivanov, tavi.purdila,
	linux-kernel-library, linux-arch, retrage01


Hello,

On Wed, 17 Mar 2021 06:31:03 +0900,
Johannes Berg wrote:
> 
> On Tue, 2021-03-16 at 10:18 +0900, Hajime Tazaki wrote:
> > 
> > > > +void *uml_kmalloc(int size, int flags)
> > > > +{
> > > > +	return kmalloc(size, flags);
> > > > +}
> > > 
> > > That could probably still be shared?
> > 
> > This function is a stub of arch/um/kernel/mem.c, which LKL doesn't
> > use for the build.  Thus we defined here.
> > 
> > Or are you suggesting to not stubbing this function, but extracting
> > the function from mem.c ?
> 
> Yes, that's kind of what I was thinking.

Okay, I'll look for a fix with this approach.

> OTOH, I guess it's in os-Linux today?

sorry, I didn't get this line.  uml_kmalloc() is in kernel/mem.c but
what is in os-Linux ?

Thanks,
-- Hajime

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 08/20] um: lkl: memory handling
@ 2021-03-18  0:12                     ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-03-18  0:12 UTC (permalink / raw)
  To: johannes
  Cc: linux-um, jdike, richard, anton.ivanov, tavi.purdila,
	linux-kernel-library, linux-arch, retrage01


Hello,

On Wed, 17 Mar 2021 06:31:03 +0900,
Johannes Berg wrote:
> 
> On Tue, 2021-03-16 at 10:18 +0900, Hajime Tazaki wrote:
> > 
> > > > +void *uml_kmalloc(int size, int flags)
> > > > +{
> > > > +	return kmalloc(size, flags);
> > > > +}
> > > 
> > > That could probably still be shared?
> > 
> > This function is a stub of arch/um/kernel/mem.c, which LKL doesn't
> > use for the build.  Thus we defined here.
> > 
> > Or are you suggesting to not stubbing this function, but extracting
> > the function from mem.c ?
> 
> Yes, that's kind of what I was thinking.

Okay, I'll look for a fix with this approach.

> OTOH, I guess it's in os-Linux today?

sorry, I didn't get this line.  uml_kmalloc() is in kernel/mem.c but
what is in os-Linux ?

Thanks,
-- Hajime

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 19/20] um: lkl: add block device support of UML
  2021-03-17 14:28                       ` Johannes Berg
@ 2021-03-18  0:15                         ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-03-18  0:15 UTC (permalink / raw)
  To: johannes
  Cc: tavi.purdila, linux-um, jdike, richard, anton.ivanov,
	linux-kernel-library, linux-arch, retrage01



On Wed, 17 Mar 2021 23:28:10 +0900,
Johannes Berg wrote:
> 
> On Wed, 2021-03-17 at 16:19 +0200, Octavian Purdila wrote:
> > > I can understand that your sample code wants btrfs just to show what it
> > > can do, but IMHO it doesn't make sense to *always* enable it.
> 
> Btw, what I said there also didn't distinguish between "always enable
> it" and "always force it enabled".
> 
> Right now this patch did the latter, but the former might sort of make
> sense, but would take the form of a defconfig rather than a Kconfig code
> change.

Thanks, I understand the situation.
I agree that using defconfig should be good in this case.  Will fix this.


> > I agree. I think these can stay in defconfigs but here is where a
> > library introduces complications which I don't know how is best to
> > handle. Should we have the defconfig in arch/um or should we have it
> > in tools/testing/selftests/um? Or perhaps both places, one being a
> > generic config that would be used by most applications and one
> > customized?
> 
> Yeah, that's a question - and in that sense LKL will never be a general-
> purpose "library" since then you'd have to basically build it with
> "allyesconfig" instead of other things.
> 
> Maybe just put a note there with the tools, and maybe a defconfig, and
> then have some kind of detection at example/tool build or even runtime
> that the necessary options are selected?

preparing multiple defconfigs (e.g., tiny, normal, allyes) would be
one option for build-time selection.

Other possibilities (rather radical) could be to prepare multiple
sub-libraries (liblinux-core.so liblinux-fs-ext4.so, liblinux-net.so,
etc) so that users can pick only necessary codes when it's build (or
open during runtime).  This needs to handle dependencies as kernel
modules do, thus it's more complicated once we have this ?

-- Hajime

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 19/20] um: lkl: add block device support of UML
@ 2021-03-18  0:15                         ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-03-18  0:15 UTC (permalink / raw)
  To: johannes
  Cc: tavi.purdila, linux-um, jdike, richard, anton.ivanov,
	linux-kernel-library, linux-arch, retrage01



On Wed, 17 Mar 2021 23:28:10 +0900,
Johannes Berg wrote:
> 
> On Wed, 2021-03-17 at 16:19 +0200, Octavian Purdila wrote:
> > > I can understand that your sample code wants btrfs just to show what it
> > > can do, but IMHO it doesn't make sense to *always* enable it.
> 
> Btw, what I said there also didn't distinguish between "always enable
> it" and "always force it enabled".
> 
> Right now this patch did the latter, but the former might sort of make
> sense, but would take the form of a defconfig rather than a Kconfig code
> change.

Thanks, I understand the situation.
I agree that using defconfig should be good in this case.  Will fix this.


> > I agree. I think these can stay in defconfigs but here is where a
> > library introduces complications which I don't know how is best to
> > handle. Should we have the defconfig in arch/um or should we have it
> > in tools/testing/selftests/um? Or perhaps both places, one being a
> > generic config that would be used by most applications and one
> > customized?
> 
> Yeah, that's a question - and in that sense LKL will never be a general-
> purpose "library" since then you'd have to basically build it with
> "allyesconfig" instead of other things.
> 
> Maybe just put a note there with the tools, and maybe a defconfig, and
> then have some kind of detection at example/tool build or even runtime
> that the necessary options are selected?

preparing multiple defconfigs (e.g., tiny, normal, allyes) would be
one option for build-time selection.

Other possibilities (rather radical) could be to prepare multiple
sub-libraries (liblinux-core.so liblinux-fs-ext4.so, liblinux-net.so,
etc) so that users can pick only necessary codes when it's build (or
open during runtime).  This needs to handle dependencies as kernel
modules do, thus it's more complicated once we have this ?

-- Hajime

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 19/20] um: lkl: add block device support of UML
  2021-03-18  0:15                         ` Hajime Tazaki
@ 2021-03-18  0:43                           ` Octavian Purdila
  -1 siblings, 0 replies; 476+ messages in thread
From: Octavian Purdila @ 2021-03-18  0:43 UTC (permalink / raw)
  To: Hajime Tazaki
  Cc: Johannes Berg, linux-um, jdike, Richard Weinberger, Anton Ivanov,
	linux-kernel-library, linux-arch, Akira Moroo

On Thu, Mar 18, 2021 at 2:15 AM Hajime Tazaki <thehajime@gmail.com> wrote:
>
>
>
> On Wed, 17 Mar 2021 23:28:10 +0900,
> Johannes Berg wrote:
> >
> > On Wed, 2021-03-17 at 16:19 +0200, Octavian Purdila wrote:
> > > > I can understand that your sample code wants btrfs just to show what it
> > > > can do, but IMHO it doesn't make sense to *always* enable it.
> >
> > Btw, what I said there also didn't distinguish between "always enable
> > it" and "always force it enabled".
> >
> > Right now this patch did the latter, but the former might sort of make
> > sense, but would take the form of a defconfig rather than a Kconfig code
> > change.
>
> Thanks, I understand the situation.
> I agree that using defconfig should be good in this case.  Will fix this.
>
>
> > > I agree. I think these can stay in defconfigs but here is where a
> > > library introduces complications which I don't know how is best to
> > > handle. Should we have the defconfig in arch/um or should we have it
> > > in tools/testing/selftests/um? Or perhaps both places, one being a
> > > generic config that would be used by most applications and one
> > > customized?
> >
> > Yeah, that's a question - and in that sense LKL will never be a general-
> > purpose "library" since then you'd have to basically build it with
> > "allyesconfig" instead of other things.
> >
> > Maybe just put a note there with the tools, and maybe a defconfig, and
> > then have some kind of detection at example/tool build or even runtime
> > that the necessary options are selected?
>
> preparing multiple defconfigs (e.g., tiny, normal, allyes) would be
> one option for build-time selection.
>
> Other possibilities (rather radical) could be to prepare multiple
> sub-libraries (liblinux-core.so liblinux-fs-ext4.so, liblinux-net.so,
> etc) so that users can pick only necessary codes when it's build (or
> open during runtime).  This needs to handle dependencies as kernel
> modules do, thus it's more complicated once we have this ?
>

Yes, this would be nice as it would allow distributions to build a
package for liblinux and simplify building applications with it. We
may be able to reuse some of the module work, maybe just the
dependency part as relocation should already be handled.

^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 19/20] um: lkl: add block device support of UML
@ 2021-03-18  0:43                           ` Octavian Purdila
  0 siblings, 0 replies; 476+ messages in thread
From: Octavian Purdila @ 2021-03-18  0:43 UTC (permalink / raw)
  To: Hajime Tazaki
  Cc: Johannes Berg, linux-um, jdike, Richard Weinberger, Anton Ivanov,
	linux-kernel-library, linux-arch, Akira Moroo

On Thu, Mar 18, 2021 at 2:15 AM Hajime Tazaki <thehajime@gmail.com> wrote:
>
>
>
> On Wed, 17 Mar 2021 23:28:10 +0900,
> Johannes Berg wrote:
> >
> > On Wed, 2021-03-17 at 16:19 +0200, Octavian Purdila wrote:
> > > > I can understand that your sample code wants btrfs just to show what it
> > > > can do, but IMHO it doesn't make sense to *always* enable it.
> >
> > Btw, what I said there also didn't distinguish between "always enable
> > it" and "always force it enabled".
> >
> > Right now this patch did the latter, but the former might sort of make
> > sense, but would take the form of a defconfig rather than a Kconfig code
> > change.
>
> Thanks, I understand the situation.
> I agree that using defconfig should be good in this case.  Will fix this.
>
>
> > > I agree. I think these can stay in defconfigs but here is where a
> > > library introduces complications which I don't know how is best to
> > > handle. Should we have the defconfig in arch/um or should we have it
> > > in tools/testing/selftests/um? Or perhaps both places, one being a
> > > generic config that would be used by most applications and one
> > > customized?
> >
> > Yeah, that's a question - and in that sense LKL will never be a general-
> > purpose "library" since then you'd have to basically build it with
> > "allyesconfig" instead of other things.
> >
> > Maybe just put a note there with the tools, and maybe a defconfig, and
> > then have some kind of detection at example/tool build or even runtime
> > that the necessary options are selected?
>
> preparing multiple defconfigs (e.g., tiny, normal, allyes) would be
> one option for build-time selection.
>
> Other possibilities (rather radical) could be to prepare multiple
> sub-libraries (liblinux-core.so liblinux-fs-ext4.so, liblinux-net.so,
> etc) so that users can pick only necessary codes when it's build (or
> open during runtime).  This needs to handle dependencies as kernel
> modules do, thus it's more complicated once we have this ?
>

Yes, this would be nice as it would allow distributions to build a
package for liblinux and simplify building applications with it. We
may be able to reuse some of the module work, maybe just the
dependency part as relocation should already be handled.

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 08/20] um: lkl: memory handling
  2021-03-18  0:12                     ` Hajime Tazaki
@ 2021-03-18  8:00                       ` Johannes Berg
  -1 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2021-03-18  8:00 UTC (permalink / raw)
  To: Hajime Tazaki
  Cc: linux-um, jdike, richard, anton.ivanov, tavi.purdila,
	linux-kernel-library, linux-arch, retrage01

On Thu, 2021-03-18 at 09:12 +0900, Hajime Tazaki wrote:
> 
> sorry, I didn't get this line.  uml_kmalloc() is in kernel/mem.c but
> what is in os-Linux ?

Never mind, I was just confused.

johannes


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 08/20] um: lkl: memory handling
@ 2021-03-18  8:00                       ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2021-03-18  8:00 UTC (permalink / raw)
  To: Hajime Tazaki
  Cc: linux-um, jdike, richard, anton.ivanov, tavi.purdila,
	linux-kernel-library, linux-arch, retrage01

On Thu, 2021-03-18 at 09:12 +0900, Hajime Tazaki wrote:
> 
> sorry, I didn't get this line.  uml_kmalloc() is in kernel/mem.c but
> what is in os-Linux ?

Never mind, I was just confused.

johannes


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 00/20] Unifying LKL into UML
  2021-03-17 14:24                     ` Johannes Berg
@ 2021-03-18 14:17                       ` Hajime Tazaki
  -1 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-03-18 14:17 UTC (permalink / raw)
  To: johannes
  Cc: tavi.purdila, linux-um, jdike, richard, anton.ivanov,
	linux-kernel-library, linux-arch, retrage01


Hello,

On Wed, 17 Mar 2021 23:24:14 +0900,
Johannes Berg wrote:
> 
> Hi,
> 
> > One use case where this matters are non OS environments such as
> > bootloaders [1], running on bare-bone hardware or kernel drivers [2,
> > 3]. IMO it would be nice to keep these properties.
> 
> OK, that makes sense. Still, it seems it could be a compile-time
> decision, and doesn't necessarily mean LKL has to be NOMMU, just that it
> could support both?
> 
> I'm really trying to see if we can't get UML to be a user of LKL. IMHO
> that would be good for the code, and even be good for LKL since then
> it's maintained as part of UML as well, not "just" as its own use case.
>
> > > If your abstraction was instead "switch context" then you could still
> > > implement it using pthreads+mutexes, or you could implement it using
> > > fibers on windows, or posix contexts - but you'd have a significantly
> > > reduced API surface, since you'd only expose __switch_to() or similar,
> > > and maybe a new stack allocation etc.
> > 
> > You are right. When I started the implementation for ucontext it was
> > obvious that it would be much simpler to have abstractions closer to
> > what Linux has (alloc, free and switch threads). But I never got to
> > finish that and then things went into a different direction.
> 
> OK, sounds like you came to the same conclusion, more or less.
>
> > > Additionally, I do wonder how UML does this now, it *does* use setjmp,
> > > so are you saying it doesn't properly use the kernel stacks?
> > > 
> > 
> > To clarify a bit the statement in the paper, the context there was
> > that we should push the thread implementation to the
> > application/environment we run rather than providing "LKL" threads.
> > This was particularly important for running LKL in other OSes kernel
> > drivers. But you are right, we can use the switch abstraction and
> > implement it with threads and mutexes for those environments where it
> > helps.
> 
> Right - like I pointed to USFSTL framework, you could have posix
> ucontext, fiber and pthread at least, and obviously other things in
> other environments (ThreadX anyone? ;-) )

I also have an idea for a ThreadX in future, which also implements
actual context in the application/environment/host side (not in kernel
side, as others do).  Though this environment may not provide
mprotect-like features, there is still a value that the application
can run Linux code (e.g., network stack) for instance.

# This story is about our old work of network simulation.
  https://lwn.net/Articles/639333/

> > > But conceptually, why wouldn't it be possible to have a liblinux.so that
> > > *does* build with MMU and userspace support, and UML is a wrapper around
> > > it?
> > > 
> > 
> > This is an interesting idea. Conceptually I think it is possible.
> > There are lots of details to be figured out before we do this. I think
> > that having a NOMMU version could be a good step in the right
> > direction, especially since I think a liblinux.so has more NOMMU
> > usecases than MMU usecases - but I haven't given too much thought to
> > the MMU usecases.
> 
> Yeah, maybe UML would be the primary use case. I have been thinking that
> there would be cases where you could combine kunit and having userspace
> though, or unit-style testing but not with kunit which is "inside" the
> kernel, but instead having the test code more "outside" the test kernel.
> That's all kind of handwaving though and not really that crystallized in
> my mind.
> 
> That said, I'm not entirely sure NOMMU would be the right path towards
> this - if we do want to go this route it'll probably need changes in
> both LKL and UML to converge to this point, and at least build it into
> the abstractions.
> 
> For example the "idle" abstraction discussed elsewhere (is it part of
> the app or part of the kernel?), or the thread discussion above (it is
> part of the app but how is it implemented?) etc.

I agree that LKL (or the library mode) can conceptually offer both
NOMMU/MMU capabilities.

I also think that NOMMU library could be the first step and a minimum
product as MMU implementation may involve a lot of refactoring which
may need more consideration to the current codebase.

We tried with MMU mode library, by sharing build system
(Kconfig/Makefile) and runtime facilities (thread/irq/memory).  But,
we could only do share irq handling for this first step.

When we implement the MMU mode library in future, we may come up with
another abstraction/refactoring into the UML design, which could be a
good outcome.  But I think it is beyond the minimum given (already)
big changes with the current patchset.

-- Hajime


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 00/20] Unifying LKL into UML
@ 2021-03-18 14:17                       ` Hajime Tazaki
  0 siblings, 0 replies; 476+ messages in thread
From: Hajime Tazaki @ 2021-03-18 14:17 UTC (permalink / raw)
  To: johannes
  Cc: tavi.purdila, linux-um, jdike, richard, anton.ivanov,
	linux-kernel-library, linux-arch, retrage01


Hello,

On Wed, 17 Mar 2021 23:24:14 +0900,
Johannes Berg wrote:
> 
> Hi,
> 
> > One use case where this matters are non OS environments such as
> > bootloaders [1], running on bare-bone hardware or kernel drivers [2,
> > 3]. IMO it would be nice to keep these properties.
> 
> OK, that makes sense. Still, it seems it could be a compile-time
> decision, and doesn't necessarily mean LKL has to be NOMMU, just that it
> could support both?
> 
> I'm really trying to see if we can't get UML to be a user of LKL. IMHO
> that would be good for the code, and even be good for LKL since then
> it's maintained as part of UML as well, not "just" as its own use case.
>
> > > If your abstraction was instead "switch context" then you could still
> > > implement it using pthreads+mutexes, or you could implement it using
> > > fibers on windows, or posix contexts - but you'd have a significantly
> > > reduced API surface, since you'd only expose __switch_to() or similar,
> > > and maybe a new stack allocation etc.
> > 
> > You are right. When I started the implementation for ucontext it was
> > obvious that it would be much simpler to have abstractions closer to
> > what Linux has (alloc, free and switch threads). But I never got to
> > finish that and then things went into a different direction.
> 
> OK, sounds like you came to the same conclusion, more or less.
>
> > > Additionally, I do wonder how UML does this now, it *does* use setjmp,
> > > so are you saying it doesn't properly use the kernel stacks?
> > > 
> > 
> > To clarify a bit the statement in the paper, the context there was
> > that we should push the thread implementation to the
> > application/environment we run rather than providing "LKL" threads.
> > This was particularly important for running LKL in other OSes kernel
> > drivers. But you are right, we can use the switch abstraction and
> > implement it with threads and mutexes for those environments where it
> > helps.
> 
> Right - like I pointed to USFSTL framework, you could have posix
> ucontext, fiber and pthread at least, and obviously other things in
> other environments (ThreadX anyone? ;-) )

I also have an idea for a ThreadX in future, which also implements
actual context in the application/environment/host side (not in kernel
side, as others do).  Though this environment may not provide
mprotect-like features, there is still a value that the application
can run Linux code (e.g., network stack) for instance.

# This story is about our old work of network simulation.
  https://lwn.net/Articles/639333/

> > > But conceptually, why wouldn't it be possible to have a liblinux.so that
> > > *does* build with MMU and userspace support, and UML is a wrapper around
> > > it?
> > > 
> > 
> > This is an interesting idea. Conceptually I think it is possible.
> > There are lots of details to be figured out before we do this. I think
> > that having a NOMMU version could be a good step in the right
> > direction, especially since I think a liblinux.so has more NOMMU
> > usecases than MMU usecases - but I haven't given too much thought to
> > the MMU usecases.
> 
> Yeah, maybe UML would be the primary use case. I have been thinking that
> there would be cases where you could combine kunit and having userspace
> though, or unit-style testing but not with kunit which is "inside" the
> kernel, but instead having the test code more "outside" the test kernel.
> That's all kind of handwaving though and not really that crystallized in
> my mind.
> 
> That said, I'm not entirely sure NOMMU would be the right path towards
> this - if we do want to go this route it'll probably need changes in
> both LKL and UML to converge to this point, and at least build it into
> the abstractions.
> 
> For example the "idle" abstraction discussed elsewhere (is it part of
> the app or part of the kernel?), or the thread discussion above (it is
> part of the app but how is it implemented?) etc.

I agree that LKL (or the library mode) can conceptually offer both
NOMMU/MMU capabilities.

I also think that NOMMU library could be the first step and a minimum
product as MMU implementation may involve a lot of refactoring which
may need more consideration to the current codebase.

We tried with MMU mode library, by sharing build system
(Kconfig/Makefile) and runtime facilities (thread/irq/memory).  But,
we could only do share irq handling for this first step.

When we implement the MMU mode library in future, we may come up with
another abstraction/refactoring into the UML design, which could be a
good outcome.  But I think it is beyond the minimum given (already)
big changes with the current patchset.

-- Hajime


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 00/20] Unifying LKL into UML
  2021-03-18 14:17                       ` Hajime Tazaki
@ 2021-03-18 16:28                         ` Johannes Berg
  -1 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2021-03-18 16:28 UTC (permalink / raw)
  To: Hajime Tazaki
  Cc: tavi.purdila, linux-um, jdike, richard, anton.ivanov,
	linux-kernel-library, linux-arch, retrage01

Hi,

> I also have an idea for a ThreadX in future, which also implements
> actual context in the application/environment/host side (not in kernel
> side, as others do).  Though this environment may not provide
> mprotect-like features, there is still a value that the application
> can run Linux code (e.g., network stack) for instance.

Heh. Right.

> I agree that LKL (or the library mode) can conceptually offer both
> NOMMU/MMU capabilities.
> 
> I also think that NOMMU library could be the first step and a minimum
> product as MMU implementation may involve a lot of refactoring which
> may need more consideration to the current codebase.
> 
> We tried with MMU mode library, by sharing build system
> (Kconfig/Makefile) and runtime facilities (thread/irq/memory).  But,
> we could only do share irq handling for this first step.
> 
> When we implement the MMU mode library in future, we may come up with
> another abstraction/refactoring into the UML design, which could be a
> good outcome.  But I think it is beyond the minimum given (already)
> big changes with the current patchset.

Well, arguably that depends on how you look at it.

Understandably, you're looking at this from the POV of getting an "MVP"
(minimum viable product) into mainline as soon as possible.

I can understand why you would do that, and this patchset achieves it:
you get an LKL in mainline that's useful, even if it doesn't achieve the
best possible architecture and code sharing.

But look at it from the opposite side, from mainline's view (at least in
my opinion, others may disagree): getting an LKL (whether as an MVP or
not) isn't really that important! Getting the architecture and code
sharing right are likely the *primary* goals for mainline this
integration.

So from my POV it's *more important* to get the shared facilities,
proper abstraction and refactoring right, likely to the point where UML
is actually "small binary using the library" (in some fashion). Even if
that initially means there actually *won't* be NOMMU mode and a library
that's useful for the LKL use cases.

Yes, that's the longer road into mainline, but it also means that each
step along the way is actually useful to mainline, I'm assuming here
that the necessary code refactoring, abstraction, etc. will by itself
provide some value to UML, but given the messy state it's in, I think
that's almost certainly going to be true.

So a sense "getting LKL into UML" is at odds with "get LKL working
quickly". However, doing it this way may ultimately get it into mainline
faster because it's a much easier incremental route. Say you want to get
all this thread stuff out of the way that we discussed - then if you
need to keep UML working but *using* the abstraction you're adding (in
order to work towards the goal of it using the library) then it becomes
fairly obvious that you cannot use the abstraction that you have with
pthreads, mutexes, and semaphores exposed via APIs, but need to build
the API on "thread switching" primitives instead. I would expect similar
things to be true for other places.


Now, are you/we up for that? I don't know. On the one hand, I know
you're persistent and interested in this, but on the other hand it's
somewhat at odds with your goals. I believe for mainline it'd be better
because the code is no worse off each step along the way.

Taking the thread example again, if we have a thread switching
abstraction and an implementation in UML, worst case (e.g. if you lose
interest) is that it's a somewhat pointless abstraction there, but it
doesn't really make the code significantly worse or more complex.

OTOH, having what we have now with pthreads/mutexes/semaphores *does*
make the code significantly more complex and harder to maintain (IMHO)
because it adds all kinds of special cases, and they're somewhat more
difficult to exercise (yes, there are examples, still).


In any case, I don't think I'm the one making the decisions here, so
take this with a grain of salt.

johannes


^ permalink raw reply	[flat|nested] 476+ messages in thread
* Re: [RFC v8 00/20] Unifying LKL into UML
@ 2021-03-18 16:28                         ` Johannes Berg
  0 siblings, 0 replies; 476+ messages in thread
From: Johannes Berg @ 2021-03-18 16:28 UTC (permalink / raw)
  To: Hajime Tazaki
  Cc: tavi.purdila, linux-um, jdike, richard, anton.ivanov,
	linux-kernel-library, linux-arch, retrage01

Hi,

> I also have an idea for a ThreadX in future, which also implements
> actual context in the application/environment/host side (not in kernel
> side, as others do).  Though this environment may not provide
> mprotect-like features, there is still a value that the application
> can run Linux code (e.g., network stack) for instance.

Heh. Right.

> I agree that LKL (or the library mode) can conceptually offer both
> NOMMU/MMU capabilities.
> 
> I also think that NOMMU library could be the first step and a minimum
> product as MMU implementation may involve a lot of refactoring which
> may need more consideration to the current codebase.
> 
> We tried with MMU mode library, by sharing build system
> (Kconfig/Makefile) and runtime facilities (thread/irq/memory).  But,
> we could only do share irq handling for this first step.
> 
> When we implement the MMU mode library in future, we may come up with
> another abstraction/refactoring into the UML design, which could be a
> good outcome.  But I think it is beyond the minimum given (already)
> big changes with the current patchset.

Well, arguably that depends on how you look at it.

Understandably, you're looking at this from the POV of getting an "MVP"
(minimum viable product) into mainline as soon as possible.

I can understand why you would do that, and this patchset achieves it:
you get an LKL in mainline that's useful, even if it doesn't achieve the
best possible architecture and code sharing.

But look at it from the opposite side, from mainline's view (at least in
my opinion, others may disagree): getting an LKL (whether as an MVP or
not) isn't really that important! Getting the architecture and code
sharing right are likely the *primary* goals for mainline this
integration.

So from my POV it's *more important* to get the shared facilities,
proper abstraction and refactoring right, likely to the point where UML
is actually "small binary using the library" (in some fashion). Even if
that initially means there actually *won't* be NOMMU mode and a library
that's useful for the LKL use cases.

Yes, that's the longer road into mainline, but it also means that each
step along the way is actually useful to mainline, I'm assuming here
that the necessary code refactoring, abstraction, etc. will by itself
provide some value to UML, but given the messy state it's in, I think
that's almost certainly going to be true.

So a sense "getting LKL into UML" is at odds with "get LKL working
quickly". However, doing it this way may ultimately get it into mainline
faster because it's a much easier incremental route. Say you want to get
all this thread stuff out of the way that we discussed - then if you
need to keep UML working but *using* the abstraction you're adding (in
order to work towards the goal of it using the library) then it becomes
fairly obvious that you cannot use the abstraction that you have with
pthreads, mutexes, and semaphores exposed via APIs, but need to build
the API on "thread switching" primitives instead. I would expect similar
things to be true for other places.


Now, are you/we up for that? I don't know. On the one hand, I know
you're persistent and interested in this, but on the other hand it's
somewhat at odds with your goals. I believe for mainline it'd be better
because the code is no worse off each step along the way.

Taking the thread example again, if we have a thread switching
abstraction and an implementation in UML, worst case (e.g. if you lose
interest) is that it's a somewhat pointless abstraction there, but it
doesn't really make the code significantly worse or more complex.

OTOH, having what we have now with pthreads/mutexes/semaphores *does*
make the code significantly more complex and harder to maintain (IMHO)
because it adds all kinds of special cases, and they're somewhat more
difficult to exercise (yes, there are examples, still).


In any case, I don't think I'm the one making the decisions here, so
take this with a grain of salt.

johannes


_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


^ permalink raw reply	[flat|nested] 476+ messages in thread
end of thread, other threads:[~2021-03-18 16:30 UTC | newest]

Thread overview: 476+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-02-05  7:30 [RFC v3 00/26] Unifying LKL into UML Hajime Tazaki
2020-02-05  7:30 ` [RFC v3 01/26] asm-generic: atomic64: allow using generic atomic64 on 64bit platforms Hajime Tazaki
2020-02-05  9:34   ` Peter Zijlstra
2020-02-05 12:24     ` Octavian Purdila
2020-02-05 12:24       ` Octavian Purdila
2020-02-05 12:29       ` Anton Ivanov
2020-02-05 12:29         ` Anton Ivanov
2020-02-05 12:49       ` Peter Zijlstra
2020-02-05 12:49         ` Peter Zijlstra
2020-02-05 14:00         ` Octavian Purdila
2020-02-05 14:00           ` Octavian Purdila
2020-02-05 17:13           ` Peter Zijlstra
2020-02-05 17:13             ` Peter Zijlstra
2020-02-07 12:32             ` Octavian Purdila
2020-02-07 12:32               ` Octavian Purdila
     [not found] ` <cover.1580882335.git.thehajime-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
2020-02-05  7:30   ` [RFC v3 02/26] arch: add __SYSCALL_DEFINE_ARCH Hajime Tazaki
2020-02-05  7:30     ` Hajime Tazaki
2020-02-05  7:30 ` [RFC v3 03/26] um lkl: architecture skeleton for Linux kernel library Hajime Tazaki
2020-02-05  7:30   ` Hajime Tazaki
2020-02-05  7:30 ` [RFC v3 04/26] um lkl: host interface Hajime Tazaki
2020-02-05  7:30   ` Hajime Tazaki
2020-02-05  7:30 ` [RFC v3 05/26] um lkl: memory handling Hajime Tazaki
2020-02-05  7:30   ` Hajime Tazaki
2020-02-05  7:30 ` [RFC v3 06/26] um lkl: kernel threads support Hajime Tazaki
2020-02-05  7:30   ` Hajime Tazaki
2020-02-05  7:30 ` [RFC v3 07/26] um lkl: interrupt support Hajime Tazaki
2020-02-05  7:30   ` Hajime Tazaki
2020-02-05 10:47   ` Anton Ivanov
2020-02-05 10:47     ` Anton Ivanov
2020-02-05 14:46     ` Hajime Tazaki
2020-02-05 14:46       ` Hajime Tazaki
2020-02-05  7:30 ` [RFC v3 08/26] um lkl: system call interface and application API Hajime Tazaki
2020-02-05  7:30   ` Hajime Tazaki
2020-02-05  7:30 ` [RFC v3 09/26] um lkl: timers, time and delay support Hajime Tazaki
2020-02-05  7:30   ` Hajime Tazaki
2020-02-05  7:30 ` [RFC v3 10/26] um lkl: basic kernel console support Hajime Tazaki
2020-02-05  7:30   ` Hajime Tazaki
2020-02-05  7:30 ` [RFC v3 11/26] um lkl: initialization and cleanup Hajime Tazaki
2020-02-05  7:30   ` Hajime Tazaki
2020-02-05  7:30 ` [RFC v3 12/26] um lkl: plug in the build system Hajime Tazaki
2020-02-05  7:30   ` Hajime Tazaki
2020-02-05  7:30 ` [RFC v3 13/26] lkl tools: skeleton for host side library Hajime Tazaki
2020-02-05  7:30   ` Hajime Tazaki
2020-02-05  7:30 ` [RFC v3 14/26] lkl tools: host lib: add utilities functions Hajime Tazaki
2020-02-05  7:30   ` Hajime Tazaki
2020-02-05  7:30 ` [RFC v3 15/26] lkl tools: host lib: filesystem helpers Hajime Tazaki
2020-02-05  7:30   ` Hajime Tazaki
2020-02-05  7:30 ` [RFC v3 16/26] lkl tools: host lib: networking helpers Hajime Tazaki
2020-02-05  7:30 ` [RFC v3 17/26] lkl tools: host lib: posix host operations Hajime Tazaki
2020-02-05  7:30   ` Hajime Tazaki
2020-02-05  7:30 ` [RFC v3 18/26] lkl tools: add test programs Hajime Tazaki
2020-02-05  7:30 ` [RFC v3 19/26] lkl tools: cptofs that reads/writes to/from a filesystem image Hajime Tazaki
2020-02-05  7:30   ` Hajime Tazaki
2020-02-05  7:30 ` [RFC v3 20/26] lkl tools: fs2tar that converts a filesystem image to tar Hajime Tazaki
2020-02-05  7:30   ` Hajime Tazaki
2020-02-05  7:30 ` [RFC v3 21/26] lkl tools: add lklfuse Hajime Tazaki
2020-02-05  7:30 ` [RFC v3 22/26] um lkl: add documentation Hajime Tazaki
2020-02-05  7:30   ` Hajime Tazaki
2020-02-05  7:30 ` [RFC v3 23/26] um lkl: add CI scripts to conduct regression tests Hajime Tazaki
2020-02-05  7:30 ` [RFC v3 24/26] um lkl: add UML network driver for lkl Hajime Tazaki
2020-02-05  7:30 ` [RFC v3 25/26] um lkl: add UML block device driver (ubd) " Hajime Tazaki
2020-02-05  7:30 ` [RFC v3 26/26] um: fix clone flags to be familar with valgrind Hajime Tazaki
2020-03-30 14:45 ` [RFC v4 00/25] Unifying LKL into UML Hajime Tazaki
2020-03-30 14:45   ` Hajime Tazaki
2020-03-30 14:45   ` [RFC v4 01/25] arch: add __SYSCALL_DEFINE_ARCH Hajime Tazaki
2020-03-30 14:45     ` Hajime Tazaki
2020-03-30 14:45   ` [RFC v4 02/25] um lkl: architecture skeleton for Linux kernel library Hajime Tazaki
2020-03-30 14:45     ` Hajime Tazaki
2020-03-30 21:53     ` Johannes Berg
2020-03-30 21:53       ` Johannes Berg
2020-03-30 22:12       ` Richard Weinberger
2020-03-30 22:12         ` Richard Weinberger
2020-03-31  7:08         ` Hajime Tazaki
2020-03-31  7:08           ` Hajime Tazaki
2020-03-31 20:16           ` Johannes Berg
2020-03-31 20:16             ` Johannes Berg
2020-04-02  6:44             ` Hajime Tazaki
2020-04-02  6:44               ` Hajime Tazaki
2020-04-07 19:25               ` Octavian Purdila
2020-04-07 19:25                 ` Octavian Purdila
2020-04-07 19:25                 ` Octavian Purdila
2020-03-30 14:45   ` [RFC v4 03/25] um lkl: host interface Hajime Tazaki
2020-03-30 14:45     ` Hajime Tazaki
2020-03-30 14:45   ` [RFC v4 04/25] um lkl: memory handling Hajime Tazaki
2020-03-30 14:45     ` Hajime Tazaki
2020-03-30 14:45   ` [RFC v4 05/25] um lkl: kernel threads support Hajime Tazaki
2020-03-30 14:45     ` Hajime Tazaki
2020-03-30 14:45   ` [RFC v4 06/25] um lkl: interrupt support Hajime Tazaki
2020-03-30 14:45     ` Hajime Tazaki
2020-03-30 14:45   ` [RFC v4 07/25] um lkl: system call interface and application API Hajime Tazaki
2020-03-30 14:45     ` Hajime Tazaki
2020-03-30 14:45   ` [RFC v4 08/25] um lkl: timers, time and delay support Hajime Tazaki
2020-03-30 14:45     ` Hajime Tazaki
2020-03-30 14:45   ` [RFC v4 09/25] um lkl: basic kernel console support Hajime Tazaki
2020-03-30 14:45     ` Hajime Tazaki
2020-03-30 14:45   ` [RFC v4 10/25] um lkl: initialization and cleanup Hajime Tazaki
2020-03-30 14:45     ` Hajime Tazaki
2020-03-30 14:45   ` [RFC v4 11/25] um lkl: plug in the build system Hajime Tazaki
2020-03-30 14:45     ` Hajime Tazaki
2020-03-30 14:45   ` [RFC v4 12/25] lkl tools: skeleton for host side library Hajime Tazaki
2020-03-30 14:45     ` Hajime Tazaki
2020-03-30 14:45   ` [RFC v4 13/25] lkl tools: host lib: add utilities functions Hajime Tazaki
2020-03-30 14:45     ` Hajime Tazaki
2020-03-30 14:45   ` [RFC v4 14/25] lkl tools: host lib: filesystem helpers Hajime Tazaki
2020-03-30 14:45     ` Hajime Tazaki
2020-03-30 14:45   ` [RFC v4 15/25] lkl tools: host lib: networking helpers Hajime Tazaki
2020-03-30 14:45     ` Hajime Tazaki
2020-03-30 14:45   ` [RFC v4 16/25] lkl tools: host lib: posix host operations Hajime Tazaki
2020-03-30 14:45     ` Hajime Tazaki
2020-03-30 14:45   ` [RFC v4 17/25] lkl tools: add test programs Hajime Tazaki
2020-03-30 14:45     ` Hajime Tazaki
2020-03-30 14:45   ` [RFC v4 18/25] lkl tools: cptofs that reads/writes to/from a filesystem image Hajime Tazaki
2020-03-30 14:45     ` Hajime Tazaki
2020-03-30 14:45   ` [RFC v4 19/25] lkl tools: fs2tar that converts a filesystem image to tar Hajime Tazaki
2020-03-30 14:45     ` Hajime Tazaki
2020-03-30 14:45   ` [RFC v4 20/25] lkl tools: add lklfuse Hajime Tazaki
2020-03-30 14:45     ` Hajime Tazaki
2020-03-30 14:45   ` [RFC v4 21/25] um lkl: add documentation Hajime Tazaki
2020-03-30 14:45     ` Hajime Tazaki
2020-03-30 14:45   ` [RFC v4 22/25] um lkl: add CI scripts to conduct regression tests Hajime Tazaki
2020-03-30 14:45     ` Hajime Tazaki
2020-03-30 14:45   ` [RFC v4 23/25] um lkl: add UML network driver for lkl Hajime Tazaki
2020-03-30 14:45     ` Hajime Tazaki
2020-03-30 21:31     ` Johannes Berg
2020-03-30 21:31       ` Johannes Berg
2020-03-31  2:38       ` Hajime Tazaki
2020-03-31  2:38         ` Hajime Tazaki
2020-03-31 19:52         ` Johannes Berg
2020-03-31 19:52           ` Johannes Berg
2020-03-30 14:45   ` [RFC v4 24/25] um lkl: add UML block device driver (ubd) " Hajime Tazaki
2020-03-30 14:45     ` Hajime Tazaki
2020-03-30 14:45   ` [RFC v4 25/25] um: fix clone flags to be familiar with valgrind Hajime Tazaki
2020-03-30 14:45     ` Hajime Tazaki
2020-07-02 14:06   ` [RFC v5 00/21] Unifying LKL into UML Hajime Tazaki
2020-07-02 14:06     ` Hajime Tazaki
2020-07-02 14:06     ` [RFC v5 01/21] um: split build in kernel and host parts Hajime Tazaki
2020-07-02 14:06       ` Hajime Tazaki
2020-09-21 16:01       ` Anton Ivanov
2020-09-21 16:01         ` Anton Ivanov
2020-09-21 22:27         ` Hajime Tazaki
2020-09-21 22:27           ` Hajime Tazaki
2020-07-02 14:06     ` [RFC v5 02/21] um: add os init and exit calls Hajime Tazaki
2020-07-02 14:06       ` Hajime Tazaki
2020-07-02 14:06     ` [RFC v5 03/21] um: move arch/um/os-Linux dir to tools/um/uml Hajime Tazaki
2020-07-02 14:06       ` Hajime Tazaki
2020-07-02 14:06     ` [RFC v5 04/21] um: host: implement os_initcalls and os_exitcalls Hajime Tazaki
2020-07-02 14:06       ` Hajime Tazaki
2020-07-02 14:06     ` [RFC v5 05/21] um: move arch/x86/um/os-Linux to tools/um/uml/ Hajime Tazaki
2020-07-02 14:06       ` Hajime Tazaki
2020-07-02 14:07     ` [RFC v5 06/21] scritps: um: suppress warnings if SRCARCH=um Hajime Tazaki
2020-07-02 14:07       ` Hajime Tazaki
2020-07-02 14:07     ` [RFC v5 07/21] um: extend arch_switch_to for alternate SUBARCH Hajime Tazaki
2020-07-02 14:07       ` Hajime Tazaki
2020-07-02 14:07     ` [RFC v5 08/21] um: add nommu mode for UML library mode Hajime Tazaki
2020-07-02 14:07       ` Hajime Tazaki
2020-07-02 14:07     ` [RFC v5 09/21] um: nommu: host interface Hajime Tazaki
2020-07-02 14:07       ` Hajime Tazaki
2020-07-02 14:07     ` [RFC v5 10/21] um: nommu: memory handling Hajime Tazaki
2020-07-02 14:07       ` Hajime Tazaki
2020-07-02 14:07     ` [RFC v5 11/21] um: nommu: kernel thread support Hajime Tazaki
2020-07-02 14:07       ` Hajime Tazaki
2020-07-02 14:07     ` [RFC v5 12/21] um: nommu: system call interface and application API Hajime Tazaki
2020-07-02 14:07       ` Hajime Tazaki
2020-07-02 14:07     ` [RFC v5 13/21] um: nommu: basic console support Hajime Tazaki
2020-07-02 14:07       ` Hajime Tazaki
2020-07-02 14:07     ` [RFC v5 14/21] um: nommu: initialization and cleanup Hajime Tazaki
2020-07-02 14:07       ` Hajime Tazaki
2020-07-02 14:07     ` [RFC v5 15/21] um: nommu: integrate with irq infrastructure of UML Hajime Tazaki
2020-07-02 14:07       ` Hajime Tazaki
2020-07-02 14:07     ` [RFC v5 16/21] um: nommu: plug in the build system Hajime Tazaki
2020-07-02 14:07       ` Hajime Tazaki
2020-07-02 14:07     ` [RFC v5 17/21] um: host: add nommu build for ARCH=um Hajime Tazaki
2020-07-02 14:07       ` Hajime Tazaki
2020-07-02 14:07     ` [RFC v5 18/21] um: host: add utilities functions Hajime Tazaki
2020-07-02 14:07       ` Hajime Tazaki
2020-07-02 14:07     ` [RFC v5 19/21] um: host: posix host operations Hajime Tazaki
2020-07-02 14:07       ` Hajime Tazaki
2020-07-02 14:07     ` [RFC v5 20/21] um: host: add test programs Hajime Tazaki
2020-07-02 14:07       ` Hajime Tazaki
2020-07-02 14:07     ` [RFC v5 21/21] um: nommu: add block device support of UML Hajime Tazaki
2020-07-02 14:07       ` Hajime Tazaki
2020-09-24  7:12     ` [RFC v6 00/21] Unifying LKL into UML Hajime Tazaki
2020-09-24  7:12       ` Hajime Tazaki
2020-09-24  7:12       ` [RFC v6 01/21] um: split build in kernel and host parts Hajime Tazaki
2020-09-24  7:12         ` Hajime Tazaki
2020-09-24  7:33         ` Anton Ivanov
2020-09-24  7:33           ` Anton Ivanov
2020-09-24  8:26           ` Hajime Tazaki
2020-09-24  8:26             ` Hajime Tazaki
2020-09-24  8:37             ` Anton Ivanov
2020-09-24  8:37               ` Anton Ivanov
2020-09-24  7:36         ` Anton Ivanov
2020-09-24  7:36           ` Anton Ivanov
2020-09-24  8:13           ` Hajime Tazaki
2020-09-24  8:13             ` Hajime Tazaki
2020-09-24  7:12       ` [RFC v6 02/21] um: add os init and exit calls Hajime Tazaki
2020-09-24  7:12         ` Hajime Tazaki
2020-09-24  7:12       ` [RFC v6 03/21] um: move arch/um/os-Linux dir to tools/um/uml Hajime Tazaki
2020-09-24  7:12         ` Hajime Tazaki
2020-09-24  7:12       ` [RFC v6 04/21] um: host: implement os_initcalls and os_exitcalls Hajime Tazaki
2020-09-24  7:12         ` Hajime Tazaki
2020-09-24  7:12       ` [RFC v6 05/21] um: move arch/x86/um/os-Linux to tools/um/uml/ Hajime Tazaki
2020-09-24  7:12         ` Hajime Tazaki
2020-09-24  7:12       ` [RFC v6 06/21] scritps: um: suppress warnings if SRCARCH=um Hajime Tazaki
2020-09-24  7:12         ` Hajime Tazaki
2020-09-24  7:12       ` [RFC v6 07/21] um: extend arch_switch_to for alternate SUBARCH Hajime Tazaki
2020-09-24  7:12         ` Hajime Tazaki
2020-09-24  7:12       ` [RFC v6 08/21] um: add nommu mode for UML library mode Hajime Tazaki
2020-09-24  7:12         ` Hajime Tazaki
2020-09-24  7:12       ` [RFC v6 09/21] um: nommu: host interface Hajime Tazaki
2020-09-24  7:12         ` Hajime Tazaki
2020-09-24  7:12       ` [RFC v6 10/21] um: nommu: memory handling Hajime Tazaki
2020-09-24  7:12         ` Hajime Tazaki
2020-09-24  7:12       ` [RFC v6 11/21] um: nommu: kernel thread support Hajime Tazaki
2020-09-24  7:12         ` Hajime Tazaki
2020-09-24  7:12       ` [RFC v6 12/21] um: nommu: system call interface and application API Hajime Tazaki
2020-09-24  7:12         ` Hajime Tazaki
2020-09-24  7:12       ` [RFC v6 13/21] um: nommu: basic console support Hajime Tazaki
2020-09-24  7:12         ` Hajime Tazaki
2020-09-24  7:12       ` [RFC v6 14/21] um: nommu: initialization and cleanup Hajime Tazaki
2020-09-24  7:12         ` Hajime Tazaki
2020-09-24  7:12       ` [RFC v6 15/21] um: nommu: integrate with irq infrastructure of UML Hajime Tazaki
2020-09-24  7:12         ` Hajime Tazaki
2020-09-24  7:12       ` [RFC v6 16/21] um: nommu: plug in the build system Hajime Tazaki
2020-09-24  7:12         ` Hajime Tazaki
2020-09-24  7:12       ` [RFC v6 17/21] um: host: add nommu build for ARCH=um Hajime Tazaki
2020-09-24  7:12         ` Hajime Tazaki
2020-09-24  7:12       ` [RFC v6 18/21] um: host: add utilities functions Hajime Tazaki
2020-09-24  7:12         ` Hajime Tazaki
2020-09-24  7:12       ` [RFC v6 19/21] um: host: posix host operations Hajime Tazaki
2020-09-24  7:12         ` Hajime Tazaki
2020-09-24  7:13       ` [RFC v6 20/21] um: host: add test programs Hajime Tazaki
2020-09-24  7:13         ` Hajime Tazaki
2020-09-24  7:13       ` [RFC v6 21/21] um: nommu: add block device support of UML Hajime Tazaki
2020-09-24  7:13         ` Hajime Tazaki
2020-10-06  9:44       ` [RFC v7 00/21] Unifying LKL into UML Hajime Tazaki
2020-10-06  9:44         ` Hajime Tazaki
2020-10-06  9:44         ` [RFC v7 01/21] um: split build in kernel and host parts Hajime Tazaki
2020-10-06  9:44           ` Hajime Tazaki
2020-10-06  9:44         ` [RFC v7 02/21] um: add os init and exit calls Hajime Tazaki
2020-10-06  9:44           ` Hajime Tazaki
2020-10-07 15:13           ` Johannes Berg
2020-10-07 15:13             ` Johannes Berg
2020-10-08 13:18             ` Hajime Tazaki
2020-10-06  9:44         ` [RFC v7 03/21] um: move arch/um/os-Linux dir to tools/um/uml Hajime Tazaki
2020-10-06  9:44           ` Hajime Tazaki
2020-10-07 15:20           ` Johannes Berg
2020-10-07 15:20             ` Johannes Berg
2020-10-08 17:48             ` Octavian Purdila
2020-10-08 17:48               ` Octavian Purdila
2020-10-08 19:46               ` Johannes Berg
2020-10-08 19:46                 ` Johannes Berg
2020-10-08 20:53                 ` Octavian Purdila
2020-10-08 20:53                   ` Octavian Purdila
2020-10-09 15:59                   ` Johannes Berg
2020-10-09 15:59                     ` Johannes Berg
2020-10-06  9:44         ` [RFC v7 04/21] um: host: implement os_initcalls and os_exitcalls Hajime Tazaki
2020-10-06  9:44           ` Hajime Tazaki
2020-10-07 15:22           ` Johannes Berg
2020-10-07 15:22             ` Johannes Berg
2020-10-08 13:16             ` Hajime Tazaki
2020-10-06  9:44         ` [RFC v7 05/21] um: move arch/x86/um/os-Linux to tools/um/uml/ Hajime Tazaki
2020-10-06  9:44           ` Hajime Tazaki
2020-10-07 15:23           ` Johannes Berg
2020-10-07 15:23             ` Johannes Berg
2020-10-06  9:44         ` [RFC v7 06/21] scritps: um: suppress warnings if SRCARCH=um Hajime Tazaki
2020-10-06  9:44           ` Hajime Tazaki
2020-10-07 15:24           ` Johannes Berg
2020-10-07 15:24             ` Johannes Berg
2020-10-09  1:13             ` Hajime Tazaki
2020-10-09 16:00               ` Johannes Berg
2020-10-09 16:00                 ` Johannes Berg
2020-10-06  9:44         ` [RFC v7 07/21] um: extend arch_switch_to for alternate SUBARCH Hajime Tazaki
2020-10-06  9:44           ` Hajime Tazaki
2020-10-07 15:25           ` Johannes Berg
2020-10-07 15:25             ` Johannes Berg
2020-10-09  1:24             ` Hajime Tazaki
2020-10-09 16:02               ` Johannes Berg
2020-10-09 16:02                 ` Johannes Berg
2020-10-06  9:44         ` [RFC v7 08/21] um: add nommu mode for UML library mode Hajime Tazaki
2020-10-06  9:44           ` Hajime Tazaki
2020-10-07 15:44           ` Johannes Berg
2020-10-07 15:44             ` Johannes Berg
2020-10-09  3:38             ` Hajime Tazaki
2020-10-09 16:06               ` Johannes Berg
2020-10-09 16:06                 ` Johannes Berg
2020-10-20  8:44                 ` Hajime Tazaki
2020-10-06  9:44         ` [RFC v7 09/21] um: nommu: host interface Hajime Tazaki
2020-10-06  9:44           ` Hajime Tazaki
2020-10-07 15:45           ` Johannes Berg
2020-10-07 15:45             ` Johannes Berg
2020-10-08 18:10             ` Octavian Purdila
2020-10-08 18:10               ` Octavian Purdila
2020-10-06  9:44         ` [RFC v7 10/21] um: nommu: memory handling Hajime Tazaki
2020-10-06  9:44           ` Hajime Tazaki
2020-10-07 15:47           ` Johannes Berg
2020-10-07 15:47             ` Johannes Berg
2020-10-08 18:07             ` Octavian Purdila
2020-10-08 18:07               ` Octavian Purdila
2020-10-06  9:44         ` [RFC v7 11/21] um: nommu: kernel thread support Hajime Tazaki
2020-10-06  9:44           ` Hajime Tazaki
2020-10-07 18:57           ` Johannes Berg
2020-10-07 18:57             ` Johannes Berg
2020-10-08 18:54             ` Octavian Purdila
2020-10-08 18:54               ` Octavian Purdila
2020-10-08 19:39               ` Johannes Berg
2020-10-08 19:39                 ` Johannes Berg
2020-10-08 20:25                 ` Octavian Purdila
2020-10-08 20:25                   ` Octavian Purdila
2020-10-06  9:44         ` [RFC v7 12/21] um: nommu: system call interface and application API Hajime Tazaki
2020-10-06  9:44           ` Hajime Tazaki
2020-10-07 19:05           ` Johannes Berg
2020-10-07 19:05             ` Johannes Berg
2020-10-08 19:03             ` Octavian Purdila
2020-10-08 19:03               ` Octavian Purdila
2020-10-08 19:40               ` Johannes Berg
2020-10-08 19:40                 ` Johannes Berg
2020-10-06  9:44         ` [RFC v7 13/21] um: nommu: basic console support Hajime Tazaki
2020-10-06  9:44           ` Hajime Tazaki
2020-10-06  9:44         ` [RFC v7 14/21] um: nommu: initialization and cleanup Hajime Tazaki
2020-10-06  9:44           ` Hajime Tazaki
2020-10-06  9:44         ` [RFC v7 15/21] um: nommu: integrate with irq infrastructure of UML Hajime Tazaki
2020-10-06  9:44           ` Hajime Tazaki
2020-10-07 19:09           ` Johannes Berg
2020-10-07 19:09             ` Johannes Berg
2020-10-06  9:44         ` [RFC v7 16/21] um: nommu: plug in the build system Hajime Tazaki
2020-10-06  9:44           ` Hajime Tazaki
2020-10-07 19:20           ` Johannes Berg
2020-10-07 19:20             ` Johannes Berg
2020-10-09  7:40             ` Hajime TAZAKI
2020-10-09  7:40               ` Hajime TAZAKI
2020-10-06  9:44         ` [RFC v7 17/21] um: host: add nommu build for ARCH=um Hajime Tazaki
2020-10-06  9:44           ` Hajime Tazaki
2020-10-06  9:44         ` [RFC v7 18/21] um: host: add utilities functions Hajime Tazaki
2020-10-06  9:44           ` Hajime Tazaki
2020-10-07 14:53           ` Anton Ivanov
2020-10-07 14:53             ` Anton Ivanov
2020-10-07 15:02             ` Johannes Berg
2020-10-07 15:02               ` Johannes Berg
2020-10-07 15:03               ` Johannes Berg
2020-10-07 15:03                 ` Johannes Berg
2020-10-07 15:10                 ` Anton Ivanov
2020-10-07 15:10                   ` Anton Ivanov
2020-10-08 12:52                   ` Hajime Tazaki
2020-10-08 19:19                     ` Octavian Purdila
2020-10-08 19:19                       ` Octavian Purdila
2020-10-08 12:53                   ` Hajime Tazaki
2020-10-06  9:44         ` [RFC v7 19/21] um: host: posix host operations Hajime Tazaki
2020-10-06  9:44           ` Hajime Tazaki
2020-10-06  9:44         ` [RFC v7 20/21] um: host: add test programs Hajime Tazaki
2020-10-06  9:44           ` Hajime Tazaki
2020-10-07 19:23           ` Johannes Berg
2020-10-07 19:23             ` Johannes Berg
2020-10-09  6:24             ` Hajime Tazaki
2020-10-06  9:44         ` [RFC v7 21/21] um: nommu: add block device support of UML Hajime Tazaki
2020-10-06  9:44           ` Hajime Tazaki
2020-10-07 14:17           ` Anton Ivanov
2020-10-07 14:17             ` Anton Ivanov
2020-10-08 12:13             ` Hajime Tazaki
2020-10-07 13:30         ` [RFC v7 00/21] Unifying LKL into UML Anton Ivanov
2020-10-07 13:30           ` Anton Ivanov
2020-10-08 12:12           ` Hajime Tazaki
2020-10-08 12:50             ` Anton Ivanov
2020-10-08 12:50               ` Anton Ivanov
2020-10-08 17:13               ` Octavian Purdila
2020-10-08 17:13                 ` Octavian Purdila
2020-10-08 17:18                 ` Anton Ivanov
2020-10-08 17:18                   ` Anton Ivanov
2020-10-08 17:24                   ` Octavian Purdila
2020-10-08 17:24                     ` Octavian Purdila
2021-01-20  2:27         ` [RFC v8 00/20] " Hajime Tazaki
2021-01-20  2:27           ` Hajime Tazaki
2021-01-20  2:27           ` [RFC v8 01/20] um: split build in kernel and host parts Hajime Tazaki
2021-01-20  2:27             ` Hajime Tazaki
2021-01-20  2:27           ` [RFC v8 02/20] um: move arch/um/os-Linux dir to tools/um/uml Hajime Tazaki
2021-01-20  2:27             ` Hajime Tazaki
2021-01-20  2:27           ` [RFC v8 03/20] um: move arch/x86/um/os-Linux to tools/um/uml/ Hajime Tazaki
2021-01-20  2:27             ` Hajime Tazaki
2021-01-20  2:27           ` [RFC v8 04/20] um: implement os_initcalls and os_exitcalls Hajime Tazaki
2021-01-20  2:27             ` Hajime Tazaki
2021-01-20  2:27           ` [RFC v8 05/20] um: extend arch_switch_to for alternate SUBARCH Hajime Tazaki
2021-01-20  2:27             ` Hajime Tazaki
2021-01-20  2:27           ` [RFC v8 06/20] um: add UML library mode Hajime Tazaki
2021-01-20  2:27             ` Hajime Tazaki
2021-03-14 16:49             ` Johannes Berg
2021-03-14 16:49               ` Johannes Berg
2021-03-16  1:17               ` Hajime Tazaki
2021-03-16  1:17                 ` Hajime Tazaki
2021-01-20  2:27           ` [RFC v8 07/20] um: lkl: host interface Hajime Tazaki
2021-01-20  2:27             ` Hajime Tazaki
2021-03-14 16:50             ` Johannes Berg
2021-03-14 16:50               ` Johannes Berg
2021-03-16  1:17               ` Hajime Tazaki
2021-03-16  1:17                 ` Hajime Tazaki
2021-01-20  2:27           ` [RFC v8 08/20] um: lkl: memory handling Hajime Tazaki
2021-01-20  2:27             ` Hajime Tazaki
2021-03-14 16:53             ` Johannes Berg
2021-03-14 16:53               ` Johannes Berg
2021-03-16  1:18               ` Hajime Tazaki
2021-03-16  1:18                 ` Hajime Tazaki
2021-03-16 21:31                 ` Johannes Berg
2021-03-16 21:31                   ` Johannes Berg
2021-03-18  0:12                   ` Hajime Tazaki
2021-03-18  0:12                     ` Hajime Tazaki
2021-03-18  8:00                     ` Johannes Berg
2021-03-18  8:00                       ` Johannes Berg
2021-01-20  2:27           ` [RFC v8 09/20] um: lkl: kernel thread support Hajime Tazaki
2021-01-20  2:27             ` Hajime Tazaki
2021-03-14 17:01             ` Johannes Berg
2021-03-14 17:01               ` Johannes Berg
2021-03-16  1:18               ` Hajime Tazaki
2021-03-16  1:18                 ` Hajime Tazaki
2021-01-20  2:27           ` [RFC v8 10/20] um: lkl: system call interface and application API Hajime Tazaki
2021-01-20  2:27             ` Hajime Tazaki
2021-01-20  2:27           ` [RFC v8 11/20] um: lkl: basic console support Hajime Tazaki
2021-01-20  2:27             ` Hajime Tazaki
2021-03-14 20:42             ` Johannes Berg
2021-03-14 20:42               ` Johannes Berg
2021-03-16  1:19               ` Hajime Tazaki
2021-03-16  1:19                 ` Hajime Tazaki
2021-01-20  2:27           ` [RFC v8 12/20] um: lkl: initialization and cleanup Hajime Tazaki
2021-01-20  2:27             ` Hajime Tazaki
2021-03-14 20:40             ` Johannes Berg
2021-03-14 20:40               ` Johannes Berg
2021-03-16  1:19               ` Hajime Tazaki
2021-03-16  1:19                 ` Hajime Tazaki
2021-01-20  2:27           ` [RFC v8 13/20] um: lkl: integrate with irq infrastructure of UML Hajime Tazaki
2021-01-20  2:27             ` Hajime Tazaki
2021-03-14 20:45             ` Johannes Berg
2021-03-14 20:45               ` Johannes Berg
2021-03-16  1:20               ` Hajime Tazaki
2021-03-16  1:20                 ` Hajime Tazaki
2021-03-16 21:36                 ` Johannes Berg
2021-03-16 21:36                   ` Johannes Berg
2021-01-20  2:27           ` [RFC v8 14/20] um: lkl: plug in the build system Hajime Tazaki
2021-01-20  2:27             ` Hajime Tazaki
2021-01-20  2:27           ` [RFC v8 15/20] um: host: add library mode build for ARCH=um Hajime Tazaki
2021-01-20  2:27             ` Hajime Tazaki
2021-01-20  2:27           ` [RFC v8 16/20] um: host: add utilities functions Hajime Tazaki
2021-01-20  2:27             ` Hajime Tazaki
2021-01-20  2:27           ` [RFC v8 17/20] um: host: posix host operations Hajime Tazaki
2021-01-20  2:27             ` Hajime Tazaki
2021-01-20  2:27           ` [RFC v8 18/20] selftests/um: lkl: add test programs for library mode of UML Hajime Tazaki
2021-01-20  2:27             ` Hajime Tazaki
2021-01-20  2:27           ` [RFC v8 19/20] um: lkl: add block device support " Hajime Tazaki
2021-01-20  2:27             ` Hajime Tazaki
2021-03-14 20:37             ` Johannes Berg
2021-03-14 20:37               ` Johannes Berg
2021-03-16  1:19               ` Hajime Tazaki
2021-03-16  1:19                 ` Hajime Tazaki
2021-03-16 21:32                 ` Johannes Berg
2021-03-16 21:32                   ` Johannes Berg
2021-03-17 14:19                   ` Octavian Purdila
2021-03-17 14:19                     ` Octavian Purdila
2021-03-17 14:28                     ` Johannes Berg
2021-03-17 14:28                       ` Johannes Berg
2021-03-18  0:15                       ` Hajime Tazaki
2021-03-18  0:15                         ` Hajime Tazaki
2021-03-18  0:43                         ` Octavian Purdila
2021-03-18  0:43                           ` Octavian Purdila
2021-01-20  2:27           ` [RFC v8 20/20] um: lkl: add documentation Hajime Tazaki
2021-01-20  2:27             ` Hajime Tazaki
2021-03-14 21:03           ` [RFC v8 00/20] Unifying LKL into UML Johannes Berg
2021-03-14 21:03             ` Johannes Berg
2021-03-16  1:17             ` Hajime Tazaki
2021-03-16  1:17               ` Hajime Tazaki
2021-03-16 21:29               ` Johannes Berg
2021-03-16 21:29                 ` Johannes Berg
2021-03-17 14:03                 ` Octavian Purdila
2021-03-17 14:03                   ` Octavian Purdila
2021-03-17 14:24                   ` Johannes Berg
2021-03-17 14:24                     ` Johannes Berg
2021-03-18 14:17                     ` Hajime Tazaki
2021-03-18 14:17                       ` Hajime Tazaki
2021-03-18 16:28                       ` Johannes Berg
2021-03-18 16:28                         ` Johannes Berg

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.