Re: [PATCH bpf-next v1 00/13] MAC and Audit policy using eBPF (KRSI)

["Mickaël Salaün" <mic@digikod.net>]

On 20/12/2019 16:41, KP Singh wrote:
> From: KP Singh <kpsingh@google.com>
> 
> This patch series is a continuation of the KRSI RFC
> (https://lore.kernel.org/bpf/20190910115527.5235-1-kpsingh@chromium.org/)
> 
> # Motivation
> 
> Google does rich analysis of runtime security data collected from
> internal Linux deployments (corporate devices and servers) to detect and
> thwart threats in real-time. Currently, this is done in custom kernel
> modules but we would like to replace this with something that's upstream
> and useful to others.
> 
> The current kernel infrastructure for providing telemetry (Audit, Perf
> etc.) is disjoint from access enforcement (i.e. LSMs).  Augmenting the
> information provided by audit requires kernel changes to audit, its
> policy language and user-space components. Furthermore, building a MAC
> policy based on the newly added telemetry data requires changes to
> various LSMs and their respective policy languages.
> 
> This patchset proposes a new stackable and privileged LSM which allows
> the LSM hooks to be implemented using eBPF. This facilitates a unified
> and dynamic (not requiring re-compilation of the kernel) audit and MAC
> policy.
> 
> # Why an LSM?
> 
> Linux Security Modules target security behaviours rather than the
> kernel's API. For example, it's easy to miss out a newly added system
> call for executing processes (eg. execve, execveat etc.) but the LSM
> framework ensures that all process executions trigger the relevant hooks
> irrespective of how the process was executed.
> 
> Allowing users to implement LSM hooks at runtime also benefits the LSM
> eco-system by enabling a quick feedback loop from the security community
> about the kind of behaviours that the LSM Framework should be targeting.
> 
> # How does it work?
> 
> The LSM introduces a new eBPF (https://docs.cilium.io/en/v1.6/bpf/)
> program type, BPF_PROG_TYPE_LSM, which can only be attached to a LSM
> hook.  All LSM hooks are exposed as files in securityfs. Attachment
> requires CAP_SYS_ADMIN for loading eBPF programs and CAP_MAC_ADMIN for
> modifying MAC policies.
> 
> The eBPF programs are passed the same arguments as the LSM hooks and
> executed in the body of the hook. If any of the eBPF programs returns an
> error (like ENOPERM), the behaviour represented by the hook is denied.
> 
> Audit logs can be written using a format chosen by the eBPF program to
> the perf events buffer and can be further processed in user-space.
> 
> # Limitations of RFC v1
> 
> In the previous design
> (https://lore.kernel.org/bpf/20190910115527.5235-1-kpsingh@chromium.org/),
> the BPF programs received a context which could be queried to retrieve
> specific pieces of information using specific helpers.
> 
> For example, a program that attaches to the file_mprotect LSM hook and
> queries the VMA region could have had the following context:
> 
> // Special context for the hook.
> struct bpf_mprotect_ctx {
> 	struct vm_area_struct *vma;
> };
> 
> and accessed the fields using a hypothetical helper
> "bpf_mprotect_vma_get_start:
> 
> SEC("lsm/file_mprotect")
> int mprotect_audit(bpf_mprotect_ctx *ctx)
> {
> 	unsigned long vm_start = bpf_mprotect_vma_get_start(ctx);
> 	return 0;
> }
> 
> or directly read them from the context by updating the verifier to allow
> accessing the fields:
> 
> int mprotect_audit(bpf_mprotect_ctx *ctx)
> {
> 	unsigned long vm_start = ctx->vma->vm_start;
> 	return 0;
> }
> 
> As we prototyped policies based on this design, we realized that this
> approach is not general enough. Adding helpers or verifier code for all
> usages would imply a high maintenance cost while severely restricting
> the instrumentation capabilities which is the key value add of our
> eBPF-based LSM.
> 
> Feedback from the BPF maintainers at Linux Plumbers also pushed us
> towards the following, more general, approach.
> 
> # BTF Based Design
> 
> The current design uses BTF
> (https://facebookmicrosites.github.io/bpf/blog/2018/11/14/btf-enhancement.html,
> https://lwn.net/Articles/803258/) which allows verifiable read-only
> structure accesses by field names rather than fixed offsets. This allows
> accessing the hook parameters using a dynamically created context which
> provides a certain degree of ABI stability:
> 
> /* Clang builtin to handle field accesses. */
> #define _(P) (__builtin_preserve_access_index(P))
> 
> // Only declare the structure and fields intended to be used
> // in the program
> struct vm_area_struct {
> 	unsigned long vm_start;
> };
> 
> // Declare the eBPF program mprotect_audit which attaches to
> // to the file_mprotect LSM hook and accepts three arguments.
> BPF_TRACE_3("lsm/file_mprotect", mprotect_audit,
> 	    struct vm_area_struct *, vma,
> 	    unsigned long, reqprot, unsigned long, prot
> {
> 	unsigned long vm_start = _(vma->vm_start);
> 	return 0;
> }
> 
> By relocating field offsets, BTF makes a large portion of kernel data
> structures readily accessible across kernel versions without requiring a
> large corpus of BPF helper functions and requiring recompilation with
> every kernel version. The limitations of BTF compatibility are described
> in BPF Co-Re (http://vger.kernel.org/bpfconf2019_talks/bpf-core.pdf,
> i.e. field renames, #defines and changes to the signature of LSM hooks).
> 
> This design imposes that the MAC policy (eBPF programs) be updated when
> the inspected kernel structures change outside of BTF compatibility
> guarantees. In practice, this is only required when a structure field
> used by a current policy is removed (or renamed) or when the used LSM
> hooks change. We expect the maintenance cost of these changes to be
> acceptable as compared to the previous design
> (https://lore.kernel.org/bpf/20190910115527.5235-1-kpsingh@chromium.org/).
> 
> # Distinction from Landlock
> 
> We believe there exist two distinct use-cases with distinct set of users:
> 
> * Unprivileged processes voluntarily relinquishing privileges with the
>    primary users being software developers.
> 
> * Flexible privileged (CAP_MAC_ADMIN, CAP_SYS_ADMIN) MAC and Audit with
>    the primary users being system policy admins.
> 
> These use-cases imply different APIs and trade-offs:
> 
> * The unprivileged use case requires defining more stable and custom APIs
>    (through opaque contexts and precise helpers).
> 
> * Privileged Audit and MAC requires deeper introspection of the kernel
>    data structures to maximise the flexibility that can be achieved without
>    kernel modification.
> 
> Landlock has demonstrated filesystem sandboxes and now Ptrace access
> control in its patches which are excellent use cases for an unprivileged
> process voluntarily relinquishing privileges.
> 
> However, Landlock has expanded its original goal, "towards unprivileged
> sandboxing", to being a "low-level framework to build
> access-control/audit systems" (https://landlock.io). We feel that the
> design and implementation are still driven by the constraints and
> trade-offs of the former use-case, and do not provide a satisfactory
> solution to the latter.
> 
> We also believe that our approach, direct access to common kernel data
> structures as with BTF, is inappropriate for unprivileged processes and
> probably not a good option for Landlock.
> 
> In conclusion, we feel that the design for a privileged LSM and
> unprivileged LSM are mutually exclusive and that one cannot be built
> "on-top-of" the other. Doing so would limit the capabilities of what can
> be done for an LSM that provides flexible audit and MAC capabilities or
> provide in-appropriate access to kernel internals to an unprivileged
> process.

I don't think that the design for a privileged LSM and an unprivileged 
LSM are necessarily mutually exclusive, however I do agree that the 
design of an *introspection* LSM like this version of KRSI, and an 
unprivileged LSM are mutually exclusive.

> 
> Furthermore, the Landlock design supports its historical use-case only
> when unprivileged eBPF is allowed. This is something that warrants
> discussion before an unprivileged LSM that uses eBPF is upstreamed.

Because of seccomp-bpf, on which the first version of Landlock was based 
on, I then used eBPF as a way to define a security policy which could be 
updated on the fly (thanks to eBPF maps) and evolves over time. The main 
goal of Landlock was and still is to bring sandboxing features to all 
users, which means to have an unprivileged access-control. However, I've 
reach a similar conclusion about eBPF for unprivileged access-control, 
but for slightly different reasons.

eBPF is very powerful and I proved with Landlock that it is possible to 
implement an access-control with it. However, a programmatic 
access-control does not fit well with unprivileged principles (i.e. 
innocuous composability). First, it can be used for side-channel attacks 
against (other) access-controls. Second, composability of eBPF programs 
imply to execute a stack of programs, which does not scale well. Third, 
as shown by multiple attacks, it is quite challenging to safely expose 
eBPF to malicious processes.

I'm working on a version of Landlock without eBPF, but still with the 
initial sought properties: safe unprivileged composability, modularity, 
and dynamic update. I'll send this version soon.

I hope that the work and experience from Landlock to bring eBPF to LSM 
will continue to be used through KRSI. Landlock will now focus on the 
unprivileged sandboxing part, without eBPF. Stay tuned!


> 
> # Why not tracepoints or kprobes?
> 
> In order to do MAC with tracepoints or kprobes, we would need to
> override the return value of the security hook. This is not possible
> with tracepoints or call-site kprobes.
> 
> Attaching to the return boundary (kretprobe) implies that BPF programs
> would always get called after all the other LSM hooks are called and
> clobber the pre-existing LSM semantics.
> 
> Enforcing MAC policy with an actual LSM helps leverage the verified
> semantics of the framework.
> 
> # Usage Examples
> 
> A simple example and some documentation is included in the patchset.
> 
> In order to better illustrate the capabilities of the framework some
> more advanced prototype code has also been published separately:
> 
> * Logging execution events (including environment variables and arguments):
> https://github.com/sinkap/linux-krsi/blob/patch/v1/examples/samples/bpf/lsm_audit_env.c
> * Detecting deletion of running executables:
> https://github.com/sinkap/linux-krsi/blob/patch/v1/examples/samples/bpf/lsm_detect_exec_unlink.c
> * Detection of writes to /proc/<pid>/mem:
> https://github.com/sinkap/linux-krsi/blob/patch/v1/examples/samples/bpf/lsm_audit_env.c
> 
> We have updated Google's internal telemetry infrastructure and have
> started deploying this LSM on our Linux Workstations. This gives us more
> confidence in the real-world applications of such a system.
> 
> KP Singh (13):
>    bpf: Refactor BPF_EVENT context macros to its own header.
>    bpf: lsm: Add a skeleton and config options
>    bpf: lsm: Introduce types for eBPF based LSM
>    bpf: lsm: Allow btf_id based attachment for LSM hooks
>    tools/libbpf: Add support in libbpf for BPF_PROG_TYPE_LSM
>    bpf: lsm: Init Hooks and create files in securityfs
>    bpf: lsm: Implement attach, detach and execution.
>    bpf: lsm: Show attached program names in hook read handler.
>    bpf: lsm: Add a helper function bpf_lsm_event_output
>    bpf: lsm: Handle attachment of the same program
>    tools/libbpf: Add bpf_program__attach_lsm
>    bpf: lsm: Add selftests for BPF_PROG_TYPE_LSM
>    bpf: lsm: Add Documentation
> 
>   Documentation/security/bpf.rst                |  164 +++
>   Documentation/security/index.rst              |    1 +
>   MAINTAINERS                                   |   11 +
>   include/linux/bpf_event.h                     |   78 ++
>   include/linux/bpf_lsm.h                       |   25 +
>   include/linux/bpf_types.h                     |    4 +
>   include/trace/bpf_probe.h                     |   30 +-
>   include/uapi/linux/bpf.h                      |   12 +-
>   kernel/bpf/syscall.c                          |   10 +
>   kernel/bpf/verifier.c                         |   84 +-
>   kernel/trace/bpf_trace.c                      |   24 +-
>   security/Kconfig                              |   11 +-
>   security/Makefile                             |    2 +
>   security/bpf/Kconfig                          |   25 +
>   security/bpf/Makefile                         |    7 +
>   security/bpf/include/bpf_lsm.h                |   63 +
>   security/bpf/include/fs.h                     |   23 +
>   security/bpf/include/hooks.h                  | 1015 +++++++++++++++++
>   security/bpf/lsm.c                            |  160 +++
>   security/bpf/lsm_fs.c                         |  176 +++
>   security/bpf/ops.c                            |  224 ++++
>   tools/include/uapi/linux/bpf.h                |   12 +-
>   tools/lib/bpf/bpf.c                           |    2 +-
>   tools/lib/bpf/bpf.h                           |    6 +
>   tools/lib/bpf/libbpf.c                        |  163 ++-
>   tools/lib/bpf/libbpf.h                        |    4 +
>   tools/lib/bpf/libbpf.map                      |    7 +
>   tools/lib/bpf/libbpf_probes.c                 |    1 +
>   .../bpf/prog_tests/lsm_mprotect_audit.c       |  129 +++
>   .../selftests/bpf/progs/lsm_mprotect_audit.c  |   58 +
>   30 files changed, 2451 insertions(+), 80 deletions(-)
>   create mode 100644 Documentation/security/bpf.rst
>   create mode 100644 include/linux/bpf_event.h
>   create mode 100644 include/linux/bpf_lsm.h
>   create mode 100644 security/bpf/Kconfig
>   create mode 100644 security/bpf/Makefile
>   create mode 100644 security/bpf/include/bpf_lsm.h
>   create mode 100644 security/bpf/include/fs.h
>   create mode 100644 security/bpf/include/hooks.h
>   create mode 100644 security/bpf/lsm.c
>   create mode 100644 security/bpf/lsm_fs.c
>   create mode 100644 security/bpf/ops.c
>   create mode 100644 tools/testing/selftests/bpf/prog_tests/lsm_mprotect_audit.c
>   create mode 100644 tools/testing/selftests/bpf/progs/lsm_mprotect_audit.c
>