linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: David Miller <davem@davemloft.net>
To: jolsa@redhat.com
Cc: acme@kernel.org, linux-kernel@vger.kernel.org,
	namhyung@kernel.org, jolsa@kernel.org
Subject: Re: [PATCH RFC] hist lookups
Date: Tue, 06 Nov 2018 22:13:49 -0800 (PST)	[thread overview]
Message-ID: <20181106.221349.1296959035550004994.davem@davemloft.net> (raw)
In-Reply-To: <20181106204255.GB25020@krava>

From: Jiri Olsa <jolsa@redhat.com>
Date: Tue, 6 Nov 2018 21:42:55 +0100

> I pushed that fix in perf/fixes branch, but I'm still occasionaly
> hitting the namespace crash.. working on it ;-)

Jiri, how can this new scheme work without setting copy_on_queue
for the queued_events we use here?

I don't see copy_on_queue being set and that means the queued event
structures reference the event memory directly in the mmaps, after the
mmap thread has released them back to the queue.

That means new events can come in to the mmap ring and overwrite what
was there previously, maybe even while deliver_event() is in the
middle of parsing the event.

Setting copy_on_queue for data[0] and data[1] makes all of the crashes
go away for me.

I get a lot of "[unknown]" shared objects shortly after perf top
starts up during a full workload.  I've been wondering about one
side effect of how the mmap queues are processed, consider the
following:

	cpu 0			cpu 1

				exec
				create new mmap2 events
				scheduled to cpu 0 for whatever reason
	sample 1
	sample 2

And let's say that perf top is backlogged processing the mmap ring of
events generated for cpu 0, and sees sample 1 and sample 2 before
getting to any of cpu 1's events.

This means the thread and map and symbol objects won't exist and
we'll get those '[Unknown]' histogram entries, and they won't go
away.

When it finally stops looping over the mmap ring for cpu 0's events
it gets to cpu 1's mmap ring and sees the exec and mmap2 events
but at that point it's far too late.

I surmise from what I see with perf top right now that this happens
a lot.

  reply	other threads:[~2018-11-07  6:13 UTC|newest]

Thread overview: 40+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-10-31  5:03 [PATCH RFC] hist lookups David Miller
2018-10-31 12:43 ` Arnaldo Carvalho de Melo
2018-10-31 15:39   ` Jiri Olsa
2018-10-31 16:08     ` David Miller
2018-11-03  6:30       ` David Miller
2018-11-04 20:18         ` Jiri Olsa
2018-11-05  0:50           ` David Miller
2018-11-05 20:34             ` Jiri Olsa
2018-11-05 22:52               ` David Miller
2018-11-06  3:45               ` David Miller
2018-11-06  4:03                 ` David Miller
2018-11-06  4:53                   ` David Miller
2018-11-06 11:54                     ` Jiri Olsa
2018-11-19  5:26                       ` Namhyung Kim
2018-11-19  9:12                         ` Jiri Olsa
2018-11-20  1:13                           ` Namhyung Kim
2018-11-06 20:42                     ` Jiri Olsa
2018-11-07  6:13                       ` David Miller [this message]
2018-11-07  8:32                         ` Jiri Olsa
2018-11-07 19:43                           ` Jiri Olsa
2018-11-07 20:01                             ` David Miller
2018-11-07 20:28                               ` Arnaldo Carvalho de Melo
2018-11-08  6:04                                 ` David Miller
2018-11-08  7:13                               ` Jiri Olsa
2018-11-09  1:07                                 ` David Miller
2018-11-11 19:41                                   ` Jiri Olsa
2018-11-11 19:41                                     ` Jiri Olsa
2018-11-11 22:32                                     ` David Miller
2018-11-11 22:43                                       ` Jiri Olsa
2018-11-11 22:58                                         ` David Miller
2018-11-11 23:08                                     ` David Miller
2018-11-11 23:26                                       ` Jiri Olsa
2018-11-11 23:32                                         ` David Miller
2018-11-13 10:40                                           ` Jiri Olsa
2018-11-19  4:52                                             ` David Miller
2018-11-19  6:28                                               ` Namhyung Kim
2018-11-19  6:33                                                 ` David Miller
2018-11-19  7:16                                                   ` Namhyung Kim
2018-11-19  9:14                                               ` Jiri Olsa
2018-11-06 11:51                 ` Jiri Olsa

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20181106.221349.1296959035550004994.davem@davemloft.net \
    --to=davem@davemloft.net \
    --cc=acme@kernel.org \
    --cc=jolsa@kernel.org \
    --cc=jolsa@redhat.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=namhyung@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).