From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S935692AbdEVQWd (ORCPT <rfc822;w@1wt.eu>);
        Mon, 22 May 2017 12:22:33 -0400
Received: from mx1.redhat.com ([209.132.183.28]:42243 "EHLO mx1.redhat.com"
        rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP
        id S1751312AbdEVQWa (ORCPT <rfc822;linux-kernel@vger.kernel.org>);
        Mon, 22 May 2017 12:22:30 -0400
DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 776FEC0B39CC
Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com
Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=dhowells@redhat.com
DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 776FEC0B39CC
Subject: [RFC][PATCH 0/9] Make containers kernel objects
From: David Howells <dhowells@redhat.com>
To: trondmy@primarydata.com
Cc: mszeredi@redhat.com, linux-nfs@vger.kernel.org, jlayton@redhat.com,
        linux-kernel@vger.kernel.org, dhowells@redhat.com,
        viro@zeniv.linux.org.uk, linux-fsdevel@vger.kernel.org,
        cgroups@vger.kernel.org, ebiederm@xmission.com
Date: Mon, 22 May 2017 17:22:26 +0100
Message-ID: <149547014649.10599.12025037906646164347.stgit@warthog.procyon.org.uk>
User-Agent: StGit/0.17.1-dirty
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Mon, 22 May 2017 16:22:29 +0000 (UTC)
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


Here are a set of patches to define a container object for the kernel and
to provide some methods to create and manipulate them.

The reason I think this is necessary is that the kernel has no idea how to
direct upcalls to what userspace considers to be a container - current
Linux practice appears to make a "container" just an arbitrarily chosen
junction of namespaces, control groups and files, which may be changed
individually within the "container".

The kernel upcall mechanism then needs to decide which set of namespaces,
etc., it must exec the appropriate upcall program.  Examples of this
include:

 (1) The DNS resolver.  The DNS cache in the kernel should probably be
     per-network namespace, but in userspace the program, its libraries and
     its config data are associated with a mount tree and a user namespace
     and it gets run in a particular pid namespace.

 (2) NFS ID mapper.  The NFS ID mapping cache should also probably be
     per-network namespace.

 (3) nfsdcltrack.  A way for NFSD to access stable storage for tracking
     of persistent state.  Again, network-namespace dependent, but also
     perhaps mount-namespace dependent.

 (4) General request-key upcalls.  Not particularly namespace dependent,
     apart from keyrings being somewhat governed by the user namespace and
     the upcall being configured by the mount namespace.

These patches are built on top of the mount context patchset so that
namespaces can be properly propagated over submounts/automounts.

These patches implement a container object that holds the following things:

 (1) Namespaces.

 (2) A root directory.

 (3) A set of processes, including a designated 'init' process.

 (4) The creator's credentials, including ownership.

 (5) A place to hang security for the container, allowing policies to be
     set per-container.

I also want to add:

 (6) Control groups.

 (7) A per-container keyring that can be added to from outside of the
     container, even once the container is live, for the provision of
     filesystem authentication/encryption keys in advance of the container
     being started.

You can get a list of containers by examining /proc/containers - but I'm
not sure how much value this gets you.  Note that the container in which
you are running is called "<current>" and you can only see other containers
that were started from within yours.  Containers are therefore effectively
hierarchical and an init_container is set up when the system boots.


Some management operations are provided:

 (1) int fd = container_create(const char *name, unsigned int flags);

     Create a container of the given name and return a handle to it as a
     file descriptor.  flags indicates what namespaces should be inherited
     from the caller and what should be replaced new.  It is possible to
     set up a container with a null root filesystem that can be mounted
     later.

 (2) int fsfd = fsopen(const char *fsname, int container_fd,
		       unsigned int flags);

     Prepare a mount context inside the container.  This uses all the
     containers namespaces instead of the caller's.

 (3) fsmount(int fsfd, int dfd, const char *path, unsigned int at_flags,
	     unsigned int flags);

     Mount a prepared superblock.  dfd can be given container_fd to use the
     container to which it refers as the root of the pathwalk.

     If path is "/" and at_flags is AT_FSMOUNT_CONTAINER_ROOT, then this
     will attempt to mount the root of the container and create a mount
     namespace for it.  The container must've been created with
     CONTAINER_NEW_EMPTY_FS_NS.

 (4) pid_t pid = fork_into_container(int container_fd);

     Create the init process in a container.  The process uses that
     container's namespaces instead of the caller's.

 (5) int sfd = container_socket(int container_fd,
				int domain, int type, int protocol);

     Create a socket inside a container.  The socket gets the container's
     namespaces.  This allows netlink operations to be called within that
     container to set it up from outside (at least in theory).

 (6) mkdirat(int dfd, ...);
     mknodat(int dfd, ...);
     openat(int dfd, ...);

     Supplying a container fd as dfd makes the pathwalk happen relative to
     the root of the container.  Note that the path must be *relative*.

And some need to be/could be added:

 (7) Directly set a container's namespaces to allow cross-container
     sharing.

 (8) Adjust the control group membership of a container.

 (9) Add a key inside a container keyring.

(10) Kill/suspend/freeze/reboot container, both from inside and out.

(11) Set container's root dir.

(12) Set the container's security policy.

(13) Allow overlayfs to access filesystems outside of the container in
     which it is being created.


Kernel upcalls are invoked in the root of the container that incurs them
rather than in the init namespace context.  There's still some awkwardness
here if you, say, share a network namespace between containers.  Either the
upcall binaries and configuration must be duplicated between sharing
containers or a container must be elected as the one in which such upcalls
will be done.


Some further thoughts:

 (*) Should there be an AT_IN_CONTAINER flag to provide to syscalls that
     take a container in lieu of AT_FDCWD or a directory fd?  The problem
     is that such as mkdirat() and openat() don't have an at_flags
     argument.

 (*) Should there be a container hierarchy at all?  It seems that this is
     only really necessary for /proc/containers.  Do we want to allow
     containers-within-containers?

 (*) Should each container automatically have its own pid namespace such
     that its 'init' process always appears as pid 1?

 (*) Does this allow kernel upcalls to be accounted against the correct
     control group?

 (*) Should each container have a 'list' of accessible device numbers such
     that certain device files can be made usable within a container?  And
     can devtmpfs/udev be made to show the correct file set for each
     container?


The patches can be found here also:

	http://git.kernel.org/cgit/linux/kernel/git/dhowells/linux-fs.git/log/?h=container

Note that this is dependent on the mount-context branch.

David
---
David Howells (9):
      containers: Rename linux/container.h to linux/container_dev.h
      Implement containers as kernel objects
      Provide /proc/containers
      Allow processes to be forked and upcalled into a container
      Open a socket inside a container
      Allow fs syscall dfd arguments to take a container fd
      Make fsopen() able to initiate mounting into a container
      Honour CONTAINER_NEW_EMPTY_FS_NS
      Sample program for driving container objects


 arch/x86/entry/syscalls/syscall_32.tbl |    3 
 arch/x86/entry/syscalls/syscall_64.tbl |    3 
 drivers/acpi/container.c               |    2 
 drivers/base/container.c               |    2 
 fs/fsopen.c                            |   33 +-
 fs/libfs.c                             |    3 
 fs/namei.c                             |   52 ++-
 fs/namespace.c                         |  108 +++++-
 fs/nfs/namespace.c                     |    2 
 fs/nfs/nfs4namespace.c                 |    4 
 fs/proc/root.c                         |   13 +
 fs/sb_config.c                         |   29 +-
 include/linux/container.h              |   91 ++++-
 include/linux/container_dev.h          |   25 +
 include/linux/cred.h                   |    3 
 include/linux/init_task.h              |    4 
 include/linux/kmod.h                   |    1 
 include/linux/lsm_hooks.h              |   25 +
 include/linux/mount.h                  |    5 
 include/linux/nsproxy.h                |    7 
 include/linux/pid.h                    |    5 
 include/linux/proc_ns.h                |    3 
 include/linux/sb_config.h              |    5 
 include/linux/sched.h                  |    3 
 include/linux/sched/task.h             |    4 
 include/linux/security.h               |   20 +
 include/linux/syscalls.h               |    6 
 include/uapi/linux/container.h         |   28 ++
 include/uapi/linux/fcntl.h             |    2 
 include/uapi/linux/magic.h             |    1 
 init/Kconfig                           |    7 
 init/main.c                            |    4 
 kernel/Makefile                        |    2 
 kernel/container.c                     |  576 ++++++++++++++++++++++++++++++++
 kernel/cred.c                          |   45 ++-
 kernel/exit.c                          |    1 
 kernel/fork.c                          |  117 ++++++-
 kernel/kmod.c                          |   13 +
 kernel/kthread.c                       |    3 
 kernel/namespaces.h                    |   15 +
 kernel/nsproxy.c                       |   34 +-
 kernel/pid.c                           |    4 
 kernel/sys_ni.c                        |    5 
 net/socket.c                           |   37 ++
 samples/containers/test-container.c    |  162 +++++++++
 security/security.c                    |   18 +
 security/selinux/hooks.c               |    5 
 47 files changed, 1408 insertions(+), 132 deletions(-)
 create mode 100644 include/linux/container_dev.h
 create mode 100644 include/uapi/linux/container.h
 create mode 100644 kernel/container.c
 create mode 100644 kernel/namespaces.h
 create mode 100644 samples/containers/test-container.c

From mboxrd@z Thu Jan  1 00:00:00 1970
From: David Howells <dhowells-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
Subject: [RFC][PATCH 0/9] Make containers kernel objects
Date: Mon, 22 May 2017 17:22:26 +0100
Message-ID: <149547014649.10599.12025037906646164347.stgit@warthog.procyon.org.uk>
Mime-Version: 1.0
Content-Transfer-Encoding: 7bit
Return-path: <cgroups-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>
DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 776FEC0B39CC
Sender: cgroups-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
List-ID: <cgroups.vger.kernel.org>
Content-Type: text/plain; charset="us-ascii"
To: trondmy-7I+n7zu2hftEKMMhf/gKZA@public.gmane.org
Cc: mszeredi-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org, linux-nfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, jlayton-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org, linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, dhowells-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org, viro-RmSDqhL/yNMiFSDQTTA3OLVCufUGDwFn@public.gmane.org, linux-fsdevel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, ebiederm-aS9lmoZGLiVWk0Htik3J/w@public.gmane.org


Here are a set of patches to define a container object for the kernel and
to provide some methods to create and manipulate them.

The reason I think this is necessary is that the kernel has no idea how to
direct upcalls to what userspace considers to be a container - current
Linux practice appears to make a "container" just an arbitrarily chosen
junction of namespaces, control groups and files, which may be changed
individually within the "container".

The kernel upcall mechanism then needs to decide which set of namespaces,
etc., it must exec the appropriate upcall program.  Examples of this
include:

 (1) The DNS resolver.  The DNS cache in the kernel should probably be
     per-network namespace, but in userspace the program, its libraries and
     its config data are associated with a mount tree and a user namespace
     and it gets run in a particular pid namespace.

 (2) NFS ID mapper.  The NFS ID mapping cache should also probably be
     per-network namespace.

 (3) nfsdcltrack.  A way for NFSD to access stable storage for tracking
     of persistent state.  Again, network-namespace dependent, but also
     perhaps mount-namespace dependent.

 (4) General request-key upcalls.  Not particularly namespace dependent,
     apart from keyrings being somewhat governed by the user namespace and
     the upcall being configured by the mount namespace.

These patches are built on top of the mount context patchset so that
namespaces can be properly propagated over submounts/automounts.

These patches implement a container object that holds the following things:

 (1) Namespaces.

 (2) A root directory.

 (3) A set of processes, including a designated 'init' process.

 (4) The creator's credentials, including ownership.

 (5) A place to hang security for the container, allowing policies to be
     set per-container.

I also want to add:

 (6) Control groups.

 (7) A per-container keyring that can be added to from outside of the
     container, even once the container is live, for the provision of
     filesystem authentication/encryption keys in advance of the container
     being started.

You can get a list of containers by examining /proc/containers - but I'm
not sure how much value this gets you.  Note that the container in which
you are running is called "<current>" and you can only see other containers
that were started from within yours.  Containers are therefore effectively
hierarchical and an init_container is set up when the system boots.


Some management operations are provided:

 (1) int fd = container_create(const char *name, unsigned int flags);

     Create a container of the given name and return a handle to it as a
     file descriptor.  flags indicates what namespaces should be inherited
     from the caller and what should be replaced new.  It is possible to
     set up a container with a null root filesystem that can be mounted
     later.

 (2) int fsfd = fsopen(const char *fsname, int container_fd,
		       unsigned int flags);

     Prepare a mount context inside the container.  This uses all the
     containers namespaces instead of the caller's.

 (3) fsmount(int fsfd, int dfd, const char *path, unsigned int at_flags,
	     unsigned int flags);

     Mount a prepared superblock.  dfd can be given container_fd to use the
     container to which it refers as the root of the pathwalk.

     If path is "/" and at_flags is AT_FSMOUNT_CONTAINER_ROOT, then this
     will attempt to mount the root of the container and create a mount
     namespace for it.  The container must've been created with
     CONTAINER_NEW_EMPTY_FS_NS.

 (4) pid_t pid = fork_into_container(int container_fd);

     Create the init process in a container.  The process uses that
     container's namespaces instead of the caller's.

 (5) int sfd = container_socket(int container_fd,
				int domain, int type, int protocol);

     Create a socket inside a container.  The socket gets the container's
     namespaces.  This allows netlink operations to be called within that
     container to set it up from outside (at least in theory).

 (6) mkdirat(int dfd, ...);
     mknodat(int dfd, ...);
     openat(int dfd, ...);

     Supplying a container fd as dfd makes the pathwalk happen relative to
     the root of the container.  Note that the path must be *relative*.

And some need to be/could be added:

 (7) Directly set a container's namespaces to allow cross-container
     sharing.

 (8) Adjust the control group membership of a container.

 (9) Add a key inside a container keyring.

(10) Kill/suspend/freeze/reboot container, both from inside and out.

(11) Set container's root dir.

(12) Set the container's security policy.

(13) Allow overlayfs to access filesystems outside of the container in
     which it is being created.


Kernel upcalls are invoked in the root of the container that incurs them
rather than in the init namespace context.  There's still some awkwardness
here if you, say, share a network namespace between containers.  Either the
upcall binaries and configuration must be duplicated between sharing
containers or a container must be elected as the one in which such upcalls
will be done.


Some further thoughts:

 (*) Should there be an AT_IN_CONTAINER flag to provide to syscalls that
     take a container in lieu of AT_FDCWD or a directory fd?  The problem
     is that such as mkdirat() and openat() don't have an at_flags
     argument.

 (*) Should there be a container hierarchy at all?  It seems that this is
     only really necessary for /proc/containers.  Do we want to allow
     containers-within-containers?

 (*) Should each container automatically have its own pid namespace such
     that its 'init' process always appears as pid 1?

 (*) Does this allow kernel upcalls to be accounted against the correct
     control group?

 (*) Should each container have a 'list' of accessible device numbers such
     that certain device files can be made usable within a container?  And
     can devtmpfs/udev be made to show the correct file set for each
     container?


The patches can be found here also:

	http://git.kernel.org/cgit/linux/kernel/git/dhowells/linux-fs.git/log/?h=container

Note that this is dependent on the mount-context branch.

David
---
David Howells (9):
      containers: Rename linux/container.h to linux/container_dev.h
      Implement containers as kernel objects
      Provide /proc/containers
      Allow processes to be forked and upcalled into a container
      Open a socket inside a container
      Allow fs syscall dfd arguments to take a container fd
      Make fsopen() able to initiate mounting into a container
      Honour CONTAINER_NEW_EMPTY_FS_NS
      Sample program for driving container objects


 arch/x86/entry/syscalls/syscall_32.tbl |    3 
 arch/x86/entry/syscalls/syscall_64.tbl |    3 
 drivers/acpi/container.c               |    2 
 drivers/base/container.c               |    2 
 fs/fsopen.c                            |   33 +-
 fs/libfs.c                             |    3 
 fs/namei.c                             |   52 ++-
 fs/namespace.c                         |  108 +++++-
 fs/nfs/namespace.c                     |    2 
 fs/nfs/nfs4namespace.c                 |    4 
 fs/proc/root.c                         |   13 +
 fs/sb_config.c                         |   29 +-
 include/linux/container.h              |   91 ++++-
 include/linux/container_dev.h          |   25 +
 include/linux/cred.h                   |    3 
 include/linux/init_task.h              |    4 
 include/linux/kmod.h                   |    1 
 include/linux/lsm_hooks.h              |   25 +
 include/linux/mount.h                  |    5 
 include/linux/nsproxy.h                |    7 
 include/linux/pid.h                    |    5 
 include/linux/proc_ns.h                |    3 
 include/linux/sb_config.h              |    5 
 include/linux/sched.h                  |    3 
 include/linux/sched/task.h             |    4 
 include/linux/security.h               |   20 +
 include/linux/syscalls.h               |    6 
 include/uapi/linux/container.h         |   28 ++
 include/uapi/linux/fcntl.h             |    2 
 include/uapi/linux/magic.h             |    1 
 init/Kconfig                           |    7 
 init/main.c                            |    4 
 kernel/Makefile                        |    2 
 kernel/container.c                     |  576 ++++++++++++++++++++++++++++++++
 kernel/cred.c                          |   45 ++-
 kernel/exit.c                          |    1 
 kernel/fork.c                          |  117 ++++++-
 kernel/kmod.c                          |   13 +
 kernel/kthread.c                       |    3 
 kernel/namespaces.h                    |   15 +
 kernel/nsproxy.c                       |   34 +-
 kernel/pid.c                           |    4 
 kernel/sys_ni.c                        |    5 
 net/socket.c                           |   37 ++
 samples/containers/test-container.c    |  162 +++++++++
 security/security.c                    |   18 +
 security/selinux/hooks.c               |    5 
 47 files changed, 1408 insertions(+), 132 deletions(-)
 create mode 100644 include/linux/container_dev.h
 create mode 100644 include/uapi/linux/container.h
 create mode 100644 kernel/container.c
 create mode 100644 kernel/namespaces.h
 create mode 100644 samples/containers/test-container.c