Re: Some questions/thoughts on fs-verity

["Colin Walters" <walters@verbum.org>]

On Fri, Nov 8, 2019, at 10:41 PM, Eric Biggers wrote:

> Can you take a step back and consider what security properties you're actually
> trying to achieve? 

I'd like similar security properties to a full dm-verity bootable OS image - if someone gains CAP_SYS_ADMIN and the ability to subvert LSMs etc. and write arbitrary files (and most ideally protection against writes to underlying raw block devices), there should not be a "persistence vector" that allows them to regain CAP_SYS_ADMIN after a reboot (automatically, without re-doing an exploit).

This assumes that the userspace won't go off and execute unverified code from outside the root; see also
https://blog.verbum.org/2017/06/12/on-dm-verity-and-operating-systems/

> In particular, be aware that if you just enable fs-verity on
> a file but never actually compare the hash to anything, it provides integrity
> only (detection of accidental corruption), not any authenticity protection.

Right; as I note in the PR just to start
" Among other things, it does finally add an API that makes files immutable, which will help against some accidental damage."

Content immutable but still unlink() and link()-able being key here.

> In your pull request to OSTree (https://github.com/ostreedev/ostree/pull/1959),
> I see a call to FS_IOC_ENABLE_VERITY.  But there's no corresponding call to
> FS_IOC_MEASURE_VERITY to actually authenticate the file(s), nor is fs-verity's
> built-in signature verification feature being used.  So this is integrity-only;
> there's no protection against malicious modifications to the data on-disk.

Yes, I understand that, but I didn't investigate that too much exactly because of the issues I noted in my original mail; it doesn't help my use case much to measure files if e.g. one can gain code execution by other means.
 
> I.e., fs-verity is really meant to be used as part of a userspace-driven
> authentication policy.  It's not something that magically increases security by
> itself.  That's partly why the scope of fs-verity is limited to file contents:
> userspace can still authenticate other metadata if needed.

If I understand correctly, for the Android/ChromeOS use case, fs-verity will be applied to application ZIP files, and the base OS userspace will verify their signatures before launching.  That seems straightforward.  But that approach effectively requires abandoning a model of writing files directly to the filesystem and instead serializing to a tarball/zipfile.  Being able to support e.g. traditional OCI/Docker-style container images (where all of userspace expects to use the open() syscall rather than e.g. a special resource API[1]) on such a system would require something like a FUSE mount of a fs-verity tarball/zipfile (right?). It's *possible* but FUSE has all sorts of downsides.

It also means (if implemented naively at least) you have similar "write amplification" issues - if just a few files change in an app, you end up having to re-write a full zip/tarball.

Further, for the actual base OS case...hmm, I guess we could run a FUSE mount in the initramfs or so, but...eww.

> As for an attacker exploiting a filesystem bug, yes that is a big problem
> currently in Linux.  However, gaining code execution via such a bug is a *bug*
> and is patchable and mitigable, whereas gaining code execution by modifying
> unauthenticated code stored on-disk is simply working as intended.

Often when I use the term "secure" I mean "As a software vendor, we believe we can fix security issues with this over time, and there won't be too many to be embarrassing".   Canonical examples here being virtualization and container systems like Docker/podman/etc.

It sounds like you're saying you believe that (at least ext4?) could be secure in this sense - it's not clear to me (from previous discussions) that other filesystem developers agree (at least for their filesystems).

Anyways, to restate the goal - having the security properties of a dm-verity base OS, but with a lot more flexiblity (no fixed partition size, can easily have 3 or more images, etc.).

For example, I could imagine some API that allows userspace to also "seal" a directory and symlinks (inside the filesystem, this could end up stored similarly to a fs-verity file).  That'd be a powerful primitive, because one wouldn't need to store everything inside zip files, and it'd be much easier to e.g. have userspace verify the root hash of a sealed directory and have everything else trusted from that.  For example, without having to patch systemd to learn how to measure unit files.

Or really a bottom line here is - I could imagine reworking our userspace to do this FUSE mount of fs-verity tarball model, but if e.g. the kernel filesystems aren't really feasably made safe against malicious code, then...it may not be worth doing.

[1] https://developer.android.com/guide/topics/resources/providing-resources