From: Masami Hiramatsu <mhiramat@kernel.org>
To: Arnaldo Carvalho de Melo <acme@kernel.org>
Cc: Ravi Bangoria <ravi.bangoria@linux.ibm.com>,
Jiri Olsa <jolsa@kernel.org>,
linux-kernel@vger.kernel.org, aneesh.kumar@linux.ibm.com,
Peter Zijlstra <peterz@infradead.org>,
Ingo Molnar <mingo@kernel.org>,
Namhyung Kim <namhyung@kernel.org>,
Ian Rogers <irogers@google.com>
Subject: Re: [PATCH] perf probe: Provide more detail with relocation warning
Date: Wed, 26 May 2021 23:20:20 +0900 [thread overview]
Message-ID: <20210526232020.c1632c2285af811c7531b3cc@kernel.org> (raw)
In-Reply-To: <YK5FfaxFKUNdDBWz@kernel.org>
On Wed, 26 May 2021 09:56:29 -0300
Arnaldo Carvalho de Melo <acme@kernel.org> wrote:
> Em Wed, May 26, 2021 at 03:33:40PM +0900, Masami Hiramatsu escreveu:
> > On Wed, 26 May 2021 10:23:18 +0530 Ravi Bangoria <ravi.bangoria@linux.ibm.com> wrote:
> > > On 5/25/21 6:18 PM, Masami Hiramatsu wrote:
> > > > On Tue, 25 May 2021 10:07:44 +0530 Ravi Bangoria <ravi.bangoria@linux.ibm.com> wrote:
>
> > > >> When run as normal user with default sysctl kernel.kptr_restrict=0
> > > >> and kernel.perf_event_paranoid=2, perf probe fails with:
>
> > > >> $ ./perf probe move_page_tables
> > > >> Relocated base symbol is not found!
>
> > > >> The warning message is not much informative. The reason perf
> > > >> fails is because /proc/kallsyms is restricted by
> > > >> perf_event_paranoid=2 for normal user and thus perf fails to read
> > > >> relocated address of the base symbol.
>
> > > >> Tweaking kptr_restrict and perf_event_paranoid can change the
> > > >> behavior of perf probe. Also, running as root or privileged user
> > > >> works too. Add these details in the warning message.
>
> > > >> Plus, kmap->ref_reloc_sym might not be always set even if
> > > >> host_machine is initialized. Above is the example of the same.
> > > >> Remove that comment.
>
> > > > Yes, those are restricted in some cases. Anyway without priviledged
> > > > (super) user, perf probe can not set the probe in ftrace.
>
> > > > Hmm, I think it should check the effective user-id at first. If it
> > > > is not super user and the action will access tracefs and kallsyms,
> > > > it should warn at that point.
>
> > > If kptr_restrict=2, perf probe fails with same error even for root user.
> > > That's why I thought to just change this warning message.
>
> > Ah, yes. In that case, perf probe must not use the base symbol.
> > (like -D option)
> > OK, then, let's merge this fix.
>
> > Acked-by: Masami Hiramatsu <mhiramat@kernel.org>
>
> Thanks, applied as it improves the current situation.
>
> But as a follow up, to further improve this, we can reuse what 'perf trace' has:
>
> $ perf trace sleep 1
> Error: No permissions to read /sys/kernel/tracing/events/raw_syscalls/sys_(enter|exit)
> Hint: Try 'sudo mount -o remount,mode=755 /sys/kernel/tracing/'
> $ sudo mount -o remount,mode=755 /sys/kernel/tracing/
> $ perf trace sleep 1
> Error: Permission denied.
> Hint: Check /proc/sys/kernel/perf_event_paranoid setting.
> Hint: For your workloads it needs to be <= 1
> Hint: For system wide tracing it needs to be set to -1.
> Hint: Try: 'sudo sh -c "echo -1 > /proc/sys/kernel/perf_event_paranoid"'
> Hint: The current value is 2.
> $
OK, let me check this.
BTW, does perf_event_paranoid affect only perf syscall (and kallsyms),
not the tracefs correct?
> I.e. go the extra step and show what the current value is and what it
> needs to be to achieve what is being attempted.
>
> IOW combine error message with relevant documentation, to save steps.
>
> See what 'perf top' does for an unpriv user:
>
> $ perf top --stdio
> Error:
> Access to performance monitoring and observability operations is limited.
> Enforced MAC policy settings (SELinux) can limit access to performance
> monitoring and observability operations. Inspect system audit records for
> more perf_event access control information and adjusting the policy.
> Consider adjusting /proc/sys/kernel/perf_event_paranoid setting to open
> access to performance monitoring and observability operations for processes
> without CAP_PERFMON, CAP_SYS_PTRACE or CAP_SYS_ADMIN Linux capability.
> More information can be found at 'Perf events and tool security' document:
> https://www.kernel.org/doc/html/latest/admin-guide/perf-security.html
> perf_event_paranoid setting is 2:
> -1: Allow use of (almost) all events by all users
> Ignore mlock limit after perf_event_mlock_kb without CAP_IPC_LOCK
> >= 0: Disallow raw and ftrace function tracepoint access
> >= 1: Disallow CPU event access
> >= 2: Disallow kernel profiling
> To make the adjusted perf_event_paranoid setting permanent preserve it
> in /etc/sysctl.conf (e.g. kernel.perf_event_paranoid = <setting>)
Hmm, I would rather like pointing manpages...
Would we better to have perf-security.7 manpage?
Thank you,
> $
>
> - Arnaldo
>
> >
> > >
> > > Different combinations of privilege, perf_event_paranoid, kptr_restrict:
> > >
> > > Normal/Root user
> > > | perf_event_paranoid
> > > V V kptr_restrict perf probe error
> > > ----------------------------------------------------------------
> > > N -1 0 Failed to open kprobe_events: Permission denied
> > > N 0 0 Failed to open kprobe_events: Permission denied
> > > N 1 0 Failed to open kprobe_events: Permission denied
> > > N 2 0 Relocated base symbol is not found!
> > >
> > > N -1 1 Relocated base symbol is not found!
> > > N 0 1 Relocated base symbol is not found!
> > > N 1 1 Relocated base symbol is not found!
> > > N 2 1 Relocated base symbol is not found!
> > >
> > > N -1 2 Relocated base symbol is not found!
> > > N 0 2 Relocated base symbol is not found!
> > > N 1 2 Relocated base symbol is not found!
> > > N 2 2 Relocated base symbol is not found!
> > >
> > > R -1 0 No error.
> > > R 0 0 No error.
> > > R 1 0 No error.
> > > R 2 0 No error.
> > >
> > > R -1 1 No error.
> > > R 0 1 No error.
> > > R 1 1 No error.
> > > R 2 1 No error.
> > >
> > > R -1 2 Relocated base symbol is not found!
> > > R 0 2 Relocated base symbol is not found!
> > > R 1 2 Relocated base symbol is not found!
> > > R 2 2 Relocated base symbol is not found!
> > >
> > > Ravi
> >
> >
> > --
> > Masami Hiramatsu <mhiramat@kernel.org>
>
> --
>
> - Arnaldo
--
Masami Hiramatsu <mhiramat@kernel.org>
next prev parent reply other threads:[~2021-05-26 14:20 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-05-25 4:37 [PATCH] perf probe: Provide more detail with relocation warning Ravi Bangoria
2021-05-25 12:48 ` Masami Hiramatsu
2021-05-26 4:53 ` Ravi Bangoria
2021-05-26 6:33 ` Masami Hiramatsu
2021-05-26 12:56 ` Arnaldo Carvalho de Melo
2021-05-26 14:20 ` Masami Hiramatsu [this message]
2021-06-02 11:52 ` Arnaldo Carvalho de Melo
2021-06-02 12:12 ` Masami Hiramatsu
2021-06-02 13:15 ` Arnaldo Carvalho de Melo
2021-05-26 9:01 ` [PATCH] tools/perf: doc: Add permission and sysctl notice Masami Hiramatsu
2021-05-26 9:50 ` Ravi Bangoria
2021-05-26 13:16 ` Masami Hiramatsu
2021-05-26 14:51 ` [PATCH v2] " Masami Hiramatsu
2021-06-02 11:53 ` Arnaldo Carvalho de Melo
2021-06-02 13:51 ` Ravi Bangoria
2021-06-04 13:25 ` Arnaldo Carvalho de Melo
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210526232020.c1632c2285af811c7531b3cc@kernel.org \
--to=mhiramat@kernel.org \
--cc=acme@kernel.org \
--cc=aneesh.kumar@linux.ibm.com \
--cc=irogers@google.com \
--cc=jolsa@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@kernel.org \
--cc=namhyung@kernel.org \
--cc=peterz@infradead.org \
--cc=ravi.bangoria@linux.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).