All of lore.kernel.org
 help / color / mirror / Atom feed
From: Adrian Hunter <adrian.hunter@intel.com>
To: Jiri Olsa <jolsa@redhat.com>
Cc: Jiri Olsa <jolsa@kernel.org>,
	linux-kernel@vger.kernel.org,
	Arnaldo Carvalho de Melo <acme@kernel.org>,
	Corey Ashford <cjashfor@linux.vnet.ibm.com>,
	David Ahern <dsahern@gmail.com>,
	Frederic Weisbecker <fweisbec@gmail.com>,
	Ingo Molnar <mingo@kernel.org>,
	Jean Pihet <jean.pihet@linaro.org>,
	Namhyung Kim <namhyung@kernel.org>,
	Paul Mackerras <paulus@samba.org>,
	Peter Zijlstra <a.p.zijlstra@chello.nl>
Subject: Re: [PATCHv3 00/19] perf tools: Factor ordered samples queue
Date: Mon, 21 Jul 2014 11:47:35 +0300	[thread overview]
Message-ID: <53CCD3A7.5070002@intel.com> (raw)
In-Reply-To: <20140721080219.GA4183@krava.redhat.com>

On 07/21/2014 11:02 AM, Jiri Olsa wrote:
> On Mon, Jul 21, 2014 at 09:43:58AM +0300, Adrian Hunter wrote:
>> On 07/21/2014 12:55 AM, Jiri Olsa wrote:
>>> hi,
>>> this patchset factors session's ordered samples queue,
>>> and allows to limit the size of this queue.
>>>
>>> v3 changes:
>>>   - rebased to latest tip/perf/core
>>>   - add comment for WARN in patch 8 (David)
>>>   - added ordered-events debug variable (David)
>>>   - renamed ordered_events_(get|put) to ordered_events_(new|delete)
>>>   - renamed struct ordered_events_queue to struct ordered_events
>>>
>>> v2 changes:
>>>   - several small changes for review comments (Namhyung)
>>>
>>>
>>> The report command queues events till any of following
>>> conditions is reached:
>>>   - PERF_RECORD_FINISHED_ROUND event is processed
>>>   - end of the file is reached
>>>
>>> Any of above conditions will force the queue to flush some
>>> events while keeping all allocated memory for next events.
>>>
>>> If PERF_RECORD_FINISHED_ROUND is missing the queue will
>>
>> Why is it missing?
> 
> it's stored only for tracepoints now patch 17 fixies that

Wouldn't that make a huge difference all by itself?

I would make that the first patch, and measure the difference
that it makes by itself.

> 
>>
>>> allocate memory for every single event in the perf.data.
>>> This could lead to enormous memory consuption and speed
>>> degradation of report command for huge perf.data files.
>>>
>>> With the quue allocation limit of 100 MB, I've got around
>>> 15% speedup on reporting of ~10GB perf.data file.
>>
>> How do you know the results are still valid?  Wouldn't it
>> be better to wait that extra 15% and know that the data has
>> been processed correctly?
> 
> The HALF flush could cause the out of order message
> (which I get occasionaly anyway). Patch 19 allows

Occasional out-of-order messages would be worth investigating
IMHO.  Either there is a bug or there is some "interesting"
data being recorded.

> out of order events after HALF flush.
> 
> The main reason for me was to control the memory allocation,
> which could get enormous without ROUND events being stored.

But now you are storing them...

> The 100MB queue limit seems to be enough not to hit out of
> order event due to the HALF flush.

...so is the 100MB limit needed at all if you have ROUND
events?


  reply	other threads:[~2014-07-21  8:48 UTC|newest]

Thread overview: 44+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-07-20 21:55 [PATCHv3 00/19] perf tools: Factor ordered samples queue Jiri Olsa
2014-07-20 21:55 ` [PATCH 01/19] perf tools: Fix accounting of " Jiri Olsa
2014-07-28  8:27   ` [tip:perf/core] perf session: " tip-bot for Jiri Olsa
2014-07-20 21:55 ` [PATCH 02/19] perf tools: Rename ordered_samples bool to ordered_events Jiri Olsa
2014-07-20 21:55 ` [PATCH 03/19] perf tools: Rename ordered_samples struct " Jiri Olsa
2014-07-20 21:55 ` [PATCH 04/19] perf tools: Rename ordered_events members Jiri Olsa
2014-07-20 21:55 ` [PATCH 05/19] perf tools: Add ordered_events_(new|delete) interface Jiri Olsa
2014-07-20 21:55 ` [PATCH 06/19] perf tools: Factor ordered_events_flush to be more generic Jiri Olsa
2014-07-20 21:55 ` [PATCH 07/19] perf tools: Limit ordered events queue size Jiri Olsa
2014-07-20 21:55 ` [PATCH 08/19] perf tools: Flush ordered events in case of allocation failure Jiri Olsa
2014-07-20 21:55 ` [PATCH 09/19] perf tools: Make perf_session_deliver_event global Jiri Olsa
2014-07-20 21:55 ` [PATCH 10/19] perf tools: Create ordered-events object Jiri Olsa
2014-07-20 21:55 ` [PATCH 11/19] perf tools: Use list_move in ordered_events_delete function Jiri Olsa
2014-07-20 21:55 ` [PATCH 12/19] perf tools: Add ordered_events_init function Jiri Olsa
2014-07-20 21:55 ` [PATCH 13/19] perf tools: Add ordered_events_free function Jiri Olsa
2014-07-20 21:55 ` [PATCH 14/19] perf tools: Add perf_config_u64 function Jiri Olsa
2014-07-20 21:55 ` [PATCH 15/19] perf tools: Add report.queue-size config file option Jiri Olsa
2014-07-20 21:56 ` [PATCH 16/19] perf tools: Add debug prints for ordered events queue Jiri Olsa
2014-07-20 21:56 ` [PATCH 17/19] perf tools: Always force PERF_RECORD_FINISHED_ROUND event Jiri Olsa
2014-07-24 21:34   ` Arnaldo Carvalho de Melo
2014-07-25 11:34     ` Jiri Olsa
2014-07-25 14:14       ` Arnaldo Carvalho de Melo
2014-07-25 15:45         ` Frederic Weisbecker
2014-07-25 16:12           ` Peter Zijlstra
2014-07-20 21:56 ` [PATCH 18/19] perf tools: Limit the ordered events queue by default to 100MB Jiri Olsa
2014-07-20 21:56 ` [PATCH 19/19] perf tools: Allow out of order messages in forced flush Jiri Olsa
2014-07-21  6:43 ` [PATCHv3 00/19] perf tools: Factor ordered samples queue Adrian Hunter
2014-07-21  8:02   ` Jiri Olsa
2014-07-21  8:47     ` Adrian Hunter [this message]
2014-07-21  9:54       ` Jiri Olsa
2014-07-21 12:09         ` Adrian Hunter
2014-07-21 12:35           ` Jiri Olsa
2014-07-21 12:58             ` Adrian Hunter
2014-07-21 16:31         ` Andi Kleen
2014-07-21 18:23           ` Adrian Hunter
2014-07-21 18:36             ` David Ahern
2014-07-21 18:44               ` Adrian Hunter
2014-07-24 14:19               ` Arnaldo Carvalho de Melo
2014-07-24 14:56                 ` Jiri Olsa
2014-07-24 15:10                   ` Arnaldo Carvalho de Melo
2014-07-24 15:20                     ` Jiri Olsa
2014-07-24 15:51                     ` David Ahern
2014-07-24 18:01                 ` Adrian Hunter
2014-07-21 19:39             ` Andi Kleen

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=53CCD3A7.5070002@intel.com \
    --to=adrian.hunter@intel.com \
    --cc=a.p.zijlstra@chello.nl \
    --cc=acme@kernel.org \
    --cc=cjashfor@linux.vnet.ibm.com \
    --cc=dsahern@gmail.com \
    --cc=fweisbec@gmail.com \
    --cc=jean.pihet@linaro.org \
    --cc=jolsa@kernel.org \
    --cc=jolsa@redhat.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mingo@kernel.org \
    --cc=namhyung@kernel.org \
    --cc=paulus@samba.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.