Re: [PATCH v8 00/12] Introduce CAP_PERFMON to secure system performance monitoring and observability

[Ravi Bangoria <ravi.bangoria@linux.ibm.com>]

Hi Alexey,

> Currently access to perf_events, i915_perf and other performance
> monitoring and observability subsystems of the kernel is open only for
> a privileged process [1] with CAP_SYS_ADMIN capability enabled in the
> process effective set [2].
> 
> This patch set introduces CAP_PERFMON capability designed to secure
> system performance monitoring and observability operations so that
> CAP_PERFMON would assist CAP_SYS_ADMIN capability in its governing role
> for performance monitoring and observability subsystems of the kernel.

I'm seeing an issue with CAP_PERFMON when I try to record data for a
specific target. I don't know whether this is sort of a regression or
an expected behavior.

Without setting CAP_PERFMON:

   $ getcap ./perf
   $ ./perf stat -a ls
     Error:
     Access to performance monitoring and observability operations is limited.
   $ ./perf stat ls
     Performance counter stats for 'ls':
    
                  2.06 msec task-clock:u              #    0.418 CPUs utilized
                     0      context-switches:u        #    0.000 K/sec
                     0      cpu-migrations:u          #    0.000 K/sec

With CAP_PERFMON:

   $ getcap ./perf
     ./perf = cap_perfmon+ep
   $ ./perf stat -a ls
     Performance counter stats for 'system wide':
    
                142.42 msec cpu-clock                 #   25.062 CPUs utilized
                   182      context-switches          #    0.001 M/sec
                    48      cpu-migrations            #    0.337 K/sec
   $ ./perf stat ls
     Error:
     Access to performance monitoring and observability operations is limited.

Am I missing something silly?

Analysis:
---------
A bit more analysis lead me to below kernel code fs/exec.c:

   begin_new_exec()
   {
         ...
         if (bprm->interp_flags & BINPRM_FLAGS_ENFORCE_NONDUMP ||
             !(uid_eq(current_euid(), current_uid()) &&
               gid_eq(current_egid(), current_gid())))
                 set_dumpable(current->mm, suid_dumpable);
         else
                 set_dumpable(current->mm, SUID_DUMP_USER);

         ...
         commit_creds(bprm->cred);
   }

When I execute './perf stat ls', it's going into else condition and thus sets
dumpable flag as SUID_DUMP_USER. Then in commit_creds():

   int commit_creds(struct cred *new)
   {
         ...
         /* dumpability changes */
         if (...
             !cred_cap_issubset(old, new)) {
                 if (task->mm)
                         set_dumpable(task->mm, suid_dumpable);
   }

!cred_cap_issubset(old, new) fails for perf without any capability and thus
it doesn't execute set_dumpable(). Whereas that condition passes for perf
with CAP_PERFMON and thus it overwrites old value (SUID_DUMP_USER) with
suid_dumpable in mm_flags. On an Ubuntu, suid_dumpable default value is
SUID_DUMP_ROOT. On Fedora, it's SUID_DUMP_DISABLE. (/proc/sys/fs/suid_dumpable).

Now while opening an event:

   perf_event_open()
     ptrace_may_access()
       __ptrace_may_access() {
                 ...
                 if (mm &&
                     ((get_dumpable(mm) != SUID_DUMP_USER) &&
                      !ptrace_has_cap(cred, mm->user_ns, mode)))
                     return -EPERM;
       }

This if condition passes for perf with CAP_PERFMON and thus it returns -EPERM.
But it fails for perf without CAP_PERFMON and thus it goes ahead and returns
success. So opening an event fails when perf has CAP_PREFMON and tries to open
process specific event as normal user.

Workarounds:
------------
Based on above analysis, I found couple of workarounds (examples are on
Ubuntu 18.04.4 powerpc):

Workaround1:
Setting SUID_DUMP_USER as default (in /proc/sys/fs/suid_dumpable) solves the
issue.

   # echo 1 > /proc/sys/fs/suid_dumpable
   $ getcap ./perf
     ./perf = cap_perfmon+ep
   $ ./perf stat ls
     Performance counter stats for 'ls':
    
                  1.47 msec task-clock                #    0.806 CPUs utilized
                     0      context-switches          #    0.000 K/sec
                     0      cpu-migrations            #    0.000 K/sec

Workaround2:
Using CAP_SYS_PTRACE along with CAP_PERFMON solves the issue.

   $ cat /proc/sys/fs/suid_dumpable
     2
   # setcap "cap_perfmon,cap_sys_ptrace=ep" ./perf
   $ ./perf stat ls
     Performance counter stats for 'ls':
    
                  1.41 msec task-clock                #    0.826 CPUs utilized
                     0      context-switches          #    0.000 K/sec
                     0      cpu-migrations            #    0.000 K/sec

Workaround3:
Adding CAP_PERFMON to parent of perf (/bin/bash) also solves the issue.

   $ cat /proc/sys/fs/suid_dumpable
     2
   # setcap "cap_perfmon=ep" /bin/bash
   # setcap "cap_perfmon=ep" ./perf
   $ bash
   $ ./perf stat ls
     Performance counter stats for 'ls':
    
                  1.47 msec task-clock                #    0.806 CPUs utilized
                     0      context-switches          #    0.000 K/sec
                     0      cpu-migrations            #    0.000 K/sec

- Ravi