containers.lists.linux.dev archive mirror
 help / color / mirror / Atom feed
From: YiFei Zhu <zhuyifei1999@gmail.com>
To: containers@lists.linux-foundation.org
Cc: Andrea Arcangeli <aarcange@redhat.com>,
	Giuseppe Scrivano <gscrivan@redhat.com>,
	Kees Cook <keescook@chromium.org>,
	YiFei Zhu <yifeifz2@illinois.edu>,
	Tobin Feldman-Fitzthum <tobin@ibm.com>,
	Dimitrios Skarlatos <dskarlat@cs.cmu.edu>,
	Valentin Rothberg <vrothber@redhat.com>,
	Hubertus Franke <frankeh@us.ibm.com>,
	Jack Chen <jianyan2@illinois.edu>,
	Josep Torrellas <torrella@illinois.edu>,
	bpf@vger.kernel.org, Tianyin Xu <tyxu@illinois.edu>
Subject: [RFC PATCH seccomp 0/2] seccomp: Add bitmap cache of arg-independent filter results that allow syscalls
Date: Mon, 21 Sep 2020 00:35:16 -0500	[thread overview]
Message-ID: <cover.1600661418.git.yifeifz2@illinois.edu> (raw)

From: YiFei Zhu <yifeifz2@illinois.edu>

This series adds a bitmap to cache seccomp filter results if the
result permits a syscall and is indepenent of syscall arguments.
This visibly decreases seccomp overhead for most common seccomp
filters with very little memory footprint.

The overhead of running Seccomp filters has been part of some past
discussions [1][2][3]. Oftentimes, the filters have a large number
of instructions that check syscall numbers one by one and jump based
on that. Some users chain BPF filters which further enlarge the
overhead. A recent work [6] comprehensively measures the Seccomp
overhead and shows that the overhead is non-negligible and has a
non-trivial impact on application performance.

We propose SECCOMP_CACHE, a cache-based solution to minimize the
Seccomp overhead. The basic idea is to cache the result of each
syscall check to save the subsequent overhead of executing the
filters. This is feasible, because the check in Seccomp is stateless.
The checking results of the same syscall ID and argument remains
the same.

We observed some common filters, such as docker's [4] or
systemd's [5], will make most decisions based only on the syscall
numbers, and as past discussions considered, a bitmap where each bit
represents a syscall makes most sense for these filters.

In the past Kees proposed [2] to have an "add this syscall to the
reject bitmask". It is indeed much easier to securely make a reject
accelerator to pre-filter syscalls before passing to the BPF
filters, considering it could only strengthen the security provided
by the filter. However, ultimately, filter rejections are an
exceptional / rare case. Here, instead of accelerating what is
rejected, we accelerate what is allowed. In order not to compromise
the security rules the BPF filters defined, any accept-side
accelerator must complement the BPF filters rather than replacing them.

Statically analyzing BPF bytecode to see if each syscall is going to
always land in allow or reject is more of a rabbit hole, especially
there is no current in-kernel infrastructure to enumerate all the
possible architecture numbers for a given machine. So rather than
doing that, we propose to cache the results after the BPF filters are
run. And since there are filters like docker's who will check
arguments of some syscalls, but not all or none of the syscalls, when
a filter is loaded we analyze it to find whether each syscall is
cacheable (does not access syscall argument or instruction pointer) by
following its control flow graph, and store the result for each filter
in a bitmap. Changes to architecture number or the filter are expected
to be rare and simply cause the cache to be cleared. This solution
shall be fully transparent to userspace.

Ongoing work is to further support arguments with fast hash table
lookups. We are investigating the performance of doing so [6], and how
to best integrate with the existing seccomp infrastructure.

We have done some benchmarks with patch applied against bpf-next
commit 2e80be60c465 ("libbpf: Fix compilation warnings for 64-bit printf args").

Me, in qemu-kvm x86_64 VM, on Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz,
average results:

Without cache, seccomp_benchmark:
  Current BPF sysctl settings:
  net.core.bpf_jit_enable = 1
  net.core.bpf_jit_harden = 0
  Calibrating sample size for 15 seconds worth of syscalls ...
  Benchmarking 23486415 syscalls...
  16.079642020 - 1.013345439 = 15066296581 (15.1s)
  getpid native: 641 ns
  32.080237410 - 16.080763500 = 15999473910 (16.0s)
  getpid RET_ALLOW 1 filter: 681 ns
  48.609461618 - 32.081296173 = 16528165445 (16.5s)
  getpid RET_ALLOW 2 filters: 703 ns
  Estimated total seccomp overhead for 1 filter: 40 ns
  Estimated total seccomp overhead for 2 filters: 62 ns
  Estimated seccomp per-filter overhead: 22 ns
  Estimated seccomp entry overhead: 18 ns

With cache:
  Current BPF sysctl settings:
  net.core.bpf_jit_enable = 1
  net.core.bpf_jit_harden = 0
  Calibrating sample size for 15 seconds worth of syscalls ...
  Benchmarking 23486415 syscalls...
  16.059512499 - 1.014108434 = 15045404065 (15.0s)
  getpid native: 640 ns
  31.651075934 - 16.060637323 = 15590438611 (15.6s)
  getpid RET_ALLOW 1 filter: 663 ns
  47.367316169 - 31.652302661 = 15715013508 (15.7s)
  getpid RET_ALLOW 2 filters: 669 ns
  Estimated total seccomp overhead for 1 filter: 23 ns
  Estimated total seccomp overhead for 2 filters: 29 ns
  Estimated seccomp per-filter overhead: 6 ns
  Estimated seccomp entry overhead: 17 ns

Depending on the run estimated seccomp overhead for 2 filters can be
less than seccomp overhead for 1 filter, resulting in underflow to
estimated seccomp per-filter overhead:
  Estimated total seccomp overhead for 1 filter: 27 ns
  Estimated total seccomp overhead for 2 filters: 21 ns
  Estimated seccomp per-filter overhead: 18446744073709551610 ns
  Estimated seccomp entry overhead: 33 ns

Jack Chen has also run some benchmarks on a bare metal
Intel(R) Xeon(R) CPU E3-1240 v3 @ 3.40GHz, with side channel
mitigations off (spec_store_bypass_disable=off spectre_v2=off mds=off
pti=off l1tf=off), with BPF JIT on and docker default profile,
and reported:

  unixbench syscall mix (https://github.com/kdlucas/byte-unixbench)
  unconfined:      33295685
  docker default:         20661056  60%
  docker default + cache: 25719937  30%

Patch 1 introduces the static analyzer to check for a given filter,
whether the CFG loads the syscall arguments for each syscall number.

Patch 2 implements the bitmap cache.

[1] https://lore.kernel.org/linux-security-module/c22a6c3cefc2412cad00ae14c1371711@huawei.com/T/
[2] https://lore.kernel.org/lkml/202005181120.971232B7B@keescook/T/
[3] https://github.com/seccomp/libseccomp/issues/116
[4] https://github.com/moby/moby/blob/ae0ef82b90356ac613f329a8ef5ee42ca923417d/profiles/seccomp/default.json
[5] https://github.com/systemd/systemd/blob/6743a1caf4037f03dc51a1277855018e4ab61957/src/shared/seccomp-util.c#L270
[6] Draco: Architectural and Operating System Support for System Call Security
    https://tianyin.github.io/pub/draco.pdf, MICRO-53, Oct. 2020

YiFei Zhu (2):
  seccomp/cache: Add "emulator" to check if filter is arg-dependent
  seccomp/cache: Cache filter results that allow syscalls

 arch/x86/Kconfig        |  27 +++
 include/linux/seccomp.h |  22 +++
 kernel/seccomp.c        | 400 +++++++++++++++++++++++++++++++++++++++-
 3 files changed, 446 insertions(+), 3 deletions(-)

--
2.28.0
_______________________________________________
Containers mailing list
Containers@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/containers

             reply	other threads:[~2020-09-21  5:35 UTC|newest]

Thread overview: 151+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-09-21  5:35 YiFei Zhu [this message]
2020-09-21  5:35 ` [RFC PATCH seccomp 1/2] seccomp/cache: Add "emulator" to check if filter is arg-dependent YiFei Zhu
2020-09-21 17:47   ` Jann Horn via Containers
2020-09-21 18:38     ` Jann Horn via Containers
2020-09-21 23:44     ` YiFei Zhu
2020-09-22  0:25       ` Jann Horn via Containers
2020-09-22  0:47         ` YiFei Zhu
2020-09-21  5:35 ` [RFC PATCH seccomp 2/2] seccomp/cache: Cache filter results that allow syscalls YiFei Zhu
2020-09-21 18:08   ` Jann Horn via Containers
2020-09-21 22:50     ` YiFei Zhu
2020-09-21 22:57       ` Jann Horn via Containers
2020-09-21 23:08         ` YiFei Zhu
2020-09-25  0:01   ` [PATCH v2 seccomp 2/6] asm/syscall.h: Add syscall_arches[] array Kees Cook
2020-09-25  0:15     ` Jann Horn via Containers
2020-09-25  0:18       ` Al Viro
2020-09-25  0:24         ` Jann Horn via Containers
2020-09-25  1:27     ` YiFei Zhu
2020-09-25  3:09       ` Kees Cook
2020-09-25  3:28         ` YiFei Zhu
2020-09-25 16:39           ` YiFei Zhu
2020-09-21  5:48 ` [RFC PATCH seccomp 0/2] seccomp: Add bitmap cache of arg-independent filter results that allow syscalls Sargun Dhillon
2020-09-21  7:13   ` YiFei Zhu
2020-09-21  8:30 ` Christian Brauner
2020-09-21  8:44   ` YiFei Zhu
2020-09-21 13:51 ` Tycho Andersen
2020-09-21 15:27   ` YiFei Zhu
2020-09-21 16:39     ` Tycho Andersen
2020-09-21 22:57       ` YiFei Zhu
2020-09-21 19:16 ` Jann Horn via Containers
2020-09-21 19:35   ` Hubertus Franke
2020-09-21 19:45     ` Jann Horn via Containers
2020-09-23 19:26 ` Kees Cook
2020-09-23 22:54   ` YiFei Zhu
2020-09-24  6:52     ` Kees Cook
2020-09-24 12:06 ` [PATCH seccomp 0/6] " YiFei Zhu
2020-09-24 12:06   ` [PATCH seccomp 1/6] seccomp: Move config option SECCOMP to arch/Kconfig YiFei Zhu
2020-09-24 12:06     ` YiFei Zhu
2020-09-24 12:06   ` [PATCH seccomp 2/6] asm/syscall.h: Add syscall_arches[] array YiFei Zhu
2020-09-24 12:06   ` [PATCH seccomp 3/6] seccomp/cache: Add "emulator" to check if filter is arg-dependent YiFei Zhu
2020-09-24 12:06   ` [PATCH seccomp 4/6] seccomp/cache: Lookup syscall allowlist for fast path YiFei Zhu
2020-09-24 12:06   ` [PATCH seccomp 5/6] selftests/seccomp: Compare bitmap vs filter overhead YiFei Zhu
2020-09-24 12:06   ` [PATCH seccomp 6/6] seccomp/cache: Report cache data through /proc/pid/seccomp_cache YiFei Zhu
2020-09-24 12:44   ` [PATCH v2 seccomp 0/6] seccomp: Add bitmap cache of arg-independent filter results that allow syscalls YiFei Zhu
2020-09-24 12:44     ` [PATCH v2 seccomp 1/6] seccomp: Move config option SECCOMP to arch/Kconfig YiFei Zhu
2020-09-24 19:11       ` Kees Cook
2020-10-27  9:52       ` Geert Uytterhoeven
2020-10-27 19:08         ` YiFei Zhu
2020-10-28  0:06         ` Kees Cook
2020-10-28  8:18           ` Geert Uytterhoeven
2020-10-28  9:34             ` Jann Horn via Containers
2020-09-24 12:44     ` [PATCH v2 seccomp 2/6] asm/syscall.h: Add syscall_arches[] array YiFei Zhu
2020-09-24 13:47       ` David Laight
2020-09-24 14:16         ` YiFei Zhu
2020-09-24 14:20           ` David Laight
2020-09-24 14:37             ` YiFei Zhu
2020-09-24 16:02               ` YiFei Zhu
2020-09-24 12:44     ` [PATCH v2 seccomp 3/6] seccomp/cache: Add "emulator" to check if filter is arg-dependent YiFei Zhu
2020-09-24 23:25       ` Kees Cook
2020-09-25  3:04         ` YiFei Zhu
2020-09-25 16:45           ` YiFei Zhu
2020-09-25 19:42             ` Kees Cook
2020-09-25 19:51               ` Andy Lutomirski
2020-09-25 20:37                 ` Kees Cook
2020-09-25 21:07                   ` Andy Lutomirski
2020-09-25 23:49                     ` Kees Cook
2020-09-26  0:34                       ` Andy Lutomirski
2020-09-26  1:23                     ` YiFei Zhu
2020-09-26  2:47                       ` Andy Lutomirski
2020-09-26  4:35                         ` Kees Cook
2020-09-24 12:44     ` [PATCH v2 seccomp 4/6] seccomp/cache: Lookup syscall allowlist for fast path YiFei Zhu
2020-09-24 23:46       ` Kees Cook
2020-09-25  1:55         ` YiFei Zhu
2020-09-24 12:44     ` [PATCH v2 seccomp 5/6] selftests/seccomp: Compare bitmap vs filter overhead YiFei Zhu
2020-09-24 23:47       ` Kees Cook
2020-09-25  1:35         ` YiFei Zhu
2020-09-24 12:44     ` [PATCH v2 seccomp 6/6] seccomp/cache: Report cache data through /proc/pid/seccomp_cache YiFei Zhu
2020-09-24 23:56       ` Kees Cook
2020-09-25  3:11         ` YiFei Zhu
2020-09-25  3:26           ` Kees Cook
2020-09-30 15:19 ` [PATCH v3 seccomp 0/5] seccomp: Add bitmap cache of constant allow filter results YiFei Zhu
2020-09-30 15:19   ` [PATCH v3 seccomp 1/5] x86: Enable seccomp architecture tracking YiFei Zhu
2020-09-30 21:21     ` Kees Cook
2020-09-30 21:33       ` Jann Horn via Containers
2020-09-30 22:53         ` Kees Cook
2020-09-30 23:15           ` Jann Horn via Containers
2020-09-30 15:19   ` [PATCH v3 seccomp 2/5] seccomp/cache: Add "emulator" to check if filter is constant allow YiFei Zhu
2020-09-30 22:24     ` Jann Horn via Containers
2020-09-30 22:49       ` Kees Cook
2020-10-01 11:28       ` YiFei Zhu
2020-10-01 21:08         ` Jann Horn via Containers
2020-09-30 22:40     ` Kees Cook
2020-10-01 11:52       ` YiFei Zhu
2020-10-01 21:05         ` Kees Cook
2020-10-02 11:08           ` YiFei Zhu
2020-10-09  4:47     ` YiFei Zhu
2020-10-09  5:41       ` Kees Cook
2020-09-30 15:19   ` [PATCH v3 seccomp 3/5] seccomp/cache: Lookup syscall allowlist for fast path YiFei Zhu
2020-09-30 21:32     ` Kees Cook
2020-10-09  0:17       ` YiFei Zhu
2020-10-09  5:35         ` Kees Cook
2020-09-30 15:19   ` [PATCH v3 seccomp 4/5] selftests/seccomp: Compare bitmap vs filter overhead YiFei Zhu
2020-09-30 15:19   ` [PATCH v3 seccomp 5/5] seccomp/cache: Report cache data through /proc/pid/seccomp_cache YiFei Zhu
2020-09-30 22:00     ` Jann Horn via Containers
2020-09-30 23:12       ` Kees Cook
2020-10-01 12:06       ` YiFei Zhu
2020-10-01 16:05         ` Jann Horn via Containers
2020-10-01 16:18           ` YiFei Zhu
2020-09-30 22:59     ` Kees Cook
2020-09-30 23:08       ` Jann Horn via Containers
2020-09-30 23:21         ` Kees Cook
2020-10-09 17:14   ` [PATCH v4 seccomp 0/5] seccomp: Add bitmap cache of constant allow filter results YiFei Zhu
2020-10-09 17:14     ` [PATCH v4 seccomp 1/5] seccomp/cache: Lookup syscall allowlist bitmap for fast path YiFei Zhu
2020-10-09 21:30       ` Jann Horn via Containers
2020-10-09 23:18       ` Kees Cook
2020-10-09 17:14     ` [PATCH v4 seccomp 2/5] seccomp/cache: Add "emulator" to check if filter is constant allow YiFei Zhu
2020-10-09 21:30       ` Jann Horn via Containers
2020-10-09 22:47         ` Kees Cook
2020-10-09 17:14     ` [PATCH v4 seccomp 3/5] x86: Enable seccomp architecture tracking YiFei Zhu
2020-10-09 17:25       ` Andy Lutomirski
2020-10-09 18:32         ` YiFei Zhu
2020-10-09 20:59           ` Andy Lutomirski
2020-10-09 17:14     ` [PATCH v4 seccomp 4/5] selftests/seccomp: Compare bitmap vs filter overhead YiFei Zhu
2020-10-09 17:14     ` [PATCH v4 seccomp 5/5] seccomp/cache: Report cache data through /proc/pid/seccomp_cache YiFei Zhu
2020-10-09 21:24       ` kernel test robot
2020-10-09 21:45       ` Jann Horn via Containers
2020-10-09 23:14       ` Kees Cook
2020-10-10 13:26         ` YiFei Zhu
2020-10-12 22:57           ` Kees Cook
2020-10-13  0:31             ` YiFei Zhu
2020-10-22 20:52               ` YiFei Zhu
2020-10-22 22:32                 ` Kees Cook
2020-10-22 23:40                   ` YiFei Zhu
2020-10-24  2:51                     ` Kees Cook
2020-10-30 12:18                       ` YiFei Zhu
2020-11-03 13:00                         ` YiFei Zhu
2020-11-04  0:29                           ` Kees Cook
2020-11-04 11:40                             ` YiFei Zhu
2020-11-04 18:57                               ` Kees Cook
2020-10-11 15:47     ` [PATCH v5 seccomp 0/5]seccomp: Add bitmap cache of constant allow filter results YiFei Zhu
2020-10-11 15:47       ` [PATCH v5 seccomp 1/5] seccomp/cache: Lookup syscall allowlist bitmap for fast path YiFei Zhu
2020-10-12  6:42         ` Jann Horn via Containers
2020-10-11 15:47       ` [PATCH v5 seccomp 2/5] seccomp/cache: Add "emulator" to check if filter is constant allow YiFei Zhu
2020-10-12  6:46         ` Jann Horn via Containers
2020-10-11 15:47       ` [PATCH v5 seccomp 3/5] x86: Enable seccomp architecture tracking YiFei Zhu
2020-10-11 15:47       ` [PATCH v5 seccomp 4/5] selftests/seccomp: Compare bitmap vs filter overhead YiFei Zhu
2020-10-11 15:47       ` [PATCH v5 seccomp 5/5] seccomp/cache: Report cache data through /proc/pid/seccomp_cache YiFei Zhu
2020-10-12  6:49         ` Jann Horn via Containers
2020-12-17 12:14         ` Geert Uytterhoeven
2020-12-17 18:34           ` YiFei Zhu
2020-12-18 12:35             ` Geert Uytterhoeven
2020-10-27 19:14       ` [PATCH v5 seccomp 0/5]seccomp: Add bitmap cache of constant allow filter results Kees Cook

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=cover.1600661418.git.yifeifz2@illinois.edu \
    --to=zhuyifei1999@gmail.com \
    --cc=aarcange@redhat.com \
    --cc=bpf@vger.kernel.org \
    --cc=containers@lists.linux-foundation.org \
    --cc=dskarlat@cs.cmu.edu \
    --cc=frankeh@us.ibm.com \
    --cc=gscrivan@redhat.com \
    --cc=jianyan2@illinois.edu \
    --cc=keescook@chromium.org \
    --cc=tobin@ibm.com \
    --cc=torrella@illinois.edu \
    --cc=tyxu@illinois.edu \
    --cc=vrothber@redhat.com \
    --cc=yifeifz2@illinois.edu \
    --subject='Re: [RFC PATCH seccomp 0/2] seccomp: Add bitmap cache of arg-independent filter results that allow syscalls' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).