Re: [RFC PATCH 02/27] containers: Implement containers as kernel objects

[Trond Myklebust <trondmy@hammerspace.com>]

On Tue, 2019-02-19 at 23:03 +0000, David Howells wrote:
> Trond Myklebust <trondmy@hammerspace.com> wrote:
> 
> > Do we really need a new system call to set up containers? That
> > would
> > force changes to all existing orchestration software.
> 
> No, it wouldn't.  Nothing in my patches forces existing orchestration
> software
> to change, unless it wants to use the new facilities - then it would
> have to
> be changed anyway, right?  I will grant, though, that the extent of
> the change
> might vary.

Right. It depends on what you want to the orchestrator to do. If you
want it to manage authenticated storage for you, then I grant that you
may need to change the existing orchestrator. However if you just want
the containerised software to be able to manage AFS/CIFS/... keys for
its own processes, then it's not obvious to me why you would need a new
orchestrator.

> > Given that the main thing we want to achieve is to direct messages
> > from
> > the kernel to an appropriate handler, why not focus on adding
> > functionality to do just that?
> 
> Because it's *not* just that that is added here.  There are a number
> of things
> this patchset (and one it depends on) provides:
> 
>  (1) The ability to intercept request_key() upcalls that happen
> inside a
>      container, filtered by operative namespace.

The requirement that you need to filter derives from the fact that the
kernel is being forced to run an untrusted executable in user space.
That may be acceptable when running in an uncontainerised environment,
where the executable can be vetted by the sysadmin, but it clearly
isn't in an environment where containers can be set up by untrusted
users.

If we replace the executable with a daemon that is started from inside
the container (presumably by the init process running there), then
there should be no requirement for the orchestrator to filter.

>  (2) The ability to provide a per-container keyring that can hold
> keys that
>      can be used inside the container without any action on behalf of
> the
>      denizens of the container.

Keyrings already define some inheritance semantics for child processes.
Why can't we tweak those semantics to do what is needed?

IOW: instead of adding a container syscall and a new keyring type, why
can't we just define the required keyring type and let it be inherited
through the existing clone() mechanism?

>  (3) The ability to grant permissions to a *container* as a subject,
> allowing
>      it and its denizens to use, but not necessarily read, modify,
> link or
>      invalidate a key.

Again, this sounds like a child process keyring inheritance issue.
Right now, the session keyring does not appear to match the semantics
that you describe, but why couldn't we set up a new keyring type that
can provide them?

>  (4) The ability to create superblocks inside a container with a
> separate
>      mount namespace from outside, such that they can use the
> container keys,
>      thereby allowing the root of a container to be on an
> authenticated
>      filesystem.
> 

I'm not sure that I understand the premise. If the orchestrator is
setting up and managing that authenticated root filesystem, then why do
the containerised processes need to be involved at all?

If, OTOH, the intention is to allow the containerised processes to
manage the filesystems without knowledge of the keyring contents, then
again isn't that really the same issue as (3)?

> > Is there any reason why a syscall to allow an appropriately
> > privileged
> > process to add a keyring-specific message queue to its own
> > user_namespace and obtain a file descriptor to that message queue
> > might
> > not work?
> 
> Yes.  That forces the use of a new user_namespace for every container
> in which
> you want to use any of the above features.  The user_namespace is
> already way
> too big and intrusive a hammer as it is.

No. I would need a user_namespace if I want to allow child processes to
handle request upcalls. Is that unreasonable?

> > With such an implementation, the fallback mechanism could be to
> > walk
> > back up the hierarchy of user_namespaces until a message queue is
> > found, and to invoke the existing request_key mechanism if not.
> 
> That's definitely wrong.  /sbin/request-key should *not* be spawned
> if the key
> to be instantiated is not in all the init namespaces.
> 
> I went with a container object with namespaces for a reason:
> initially, it was
> so that the upcall could take place inside of the container's
> namespaces, but
> now it's do that any request that doesn't match the namespaces on the
> container gets rejected at the boundary - so that some daemon up the
> chain
> doesn't try servicing a request for which it can't access the config
> data or
> would end up talking out of the wrong NIC.
> 
> I can drop the container object part of it for the moment.
> 
> I could instead create 1-3 new namespaces:
> 
>  (1) A namespace with an upcall-interception point.
> 
>  (2) A namespace with a container keyring.
> 
>  (3) A namespace with a subject ID for use in key ACLs.
> 
> I think I should also consider adding:
> 
>  (4) A namespace with keyring names in it.  I'm leaning towards this
> not being
>      part of user_namespace because these probably should not be
> visible
>      between containers.
> 
> David
-- 
Trond Myklebust
CTO, Hammerspace Inc
4300 El Camino Real, Suite 105
Los Altos, CA 94022
www.hammer.space