From: Peter Zijlstra <peterz@infradead.org>
To: Jiri Olsa <jolsa@kernel.org>
Cc: Arnaldo Carvalho de Melo <acme@kernel.org>,
lkml <linux-kernel@vger.kernel.org>,
Ingo Molnar <mingo@kernel.org>,
Namhyung Kim <namhyung@kernel.org>,
Alexander Shishkin <alexander.shishkin@linux.intel.com>,
Andi Kleen <andi@firstfloor.org>
Subject: Re: [PATCH] perf: Prevent recursion in ring buffer
Date: Thu, 13 Sep 2018 09:07:40 +0200 [thread overview]
Message-ID: <20180913070740.GT24124@hirez.programming.kicks-ass.net> (raw)
In-Reply-To: <20180912193317.10339-1-jolsa@kernel.org>
On Wed, Sep 12, 2018 at 09:33:17PM +0200, Jiri Olsa wrote:
> Some of the scheduling tracepoints allow the perf_tp_event
> code to write to ring buffer under different cpu than the
> code is running on.
ARGH.. that is indeed borken.
> diff --git a/kernel/events/ring_buffer.c b/kernel/events/ring_buffer.c
> index 4a9937076331..0c976ac414c5 100644
> --- a/kernel/events/ring_buffer.c
> +++ b/kernel/events/ring_buffer.c
> @@ -101,6 +101,7 @@ static void perf_output_put_handle(struct perf_output_handle *handle)
>
> out:
> preempt_enable();
> + atomic_set(&rb->recursion, 0);
> }
>
> static __always_inline bool
> @@ -145,6 +146,12 @@ __perf_output_begin(struct perf_output_handle *handle,
> goto out;
> }
>
> + if (atomic_cmpxchg(&rb->recursion, 0, 1) != 0) {
> + if (rb->nr_pages)
> + local_inc(&rb->lost);
> + goto out;
> + }
> +
> handle->rb = rb;
> handle->event = event;
>
> @@ -286,6 +293,7 @@ ring_buffer_init(struct ring_buffer *rb, long watermark, int flags)
> rb->overwrite = 1;
>
> atomic_set(&rb->refcount, 1);
> + atomic_set(&rb->recursion, 0);
>
> INIT_LIST_HEAD(&rb->event_list);
> spin_lock_init(&rb->event_lock);
That's not a recursion count, that's a test-and-set spinlock, and you
got the memory ordering wrong for that.
Also, we tried very hard to avoid atomic ops in the ring-buffer and you
just wrecked that. Worse, you wrecked previously working interrupt
nesting output.
Let me have a look at this.
next prev parent reply other threads:[~2018-09-13 7:07 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-09-12 19:33 [PATCH] perf: Prevent recursion in ring buffer Jiri Olsa
2018-09-13 7:07 ` Peter Zijlstra [this message]
2018-09-13 7:41 ` Jiri Olsa
2018-09-13 7:46 ` Jiri Olsa
2018-09-13 9:37 ` Peter Zijlstra
2018-09-23 16:13 ` [PATCHv2] perf: Prevent concurent ring buffer access Jiri Olsa
2018-10-02 10:01 ` [tip:perf/core] perf/ring_buffer: " tip-bot for Jiri Olsa
2018-09-13 7:40 ` [PATCH] perf: Prevent recursion in ring buffer Peter Zijlstra
2018-09-13 7:53 ` Jiri Olsa
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180913070740.GT24124@hirez.programming.kicks-ass.net \
--to=peterz@infradead.org \
--cc=acme@kernel.org \
--cc=alexander.shishkin@linux.intel.com \
--cc=andi@firstfloor.org \
--cc=jolsa@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@kernel.org \
--cc=namhyung@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).