From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752650AbbCaHdd (ORCPT ); Tue, 31 Mar 2015 03:33:33 -0400 Received: from mail-ig0-f172.google.com ([209.85.213.172]:35706 "EHLO mail-ig0-f172.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750781AbbCaHdb (ORCPT ); Tue, 31 Mar 2015 03:33:31 -0400 MIME-Version: 1.0 In-Reply-To: <1427753974-13380-1-git-send-email-eranian@google.com> References: <1427753974-13380-1-git-send-email-eranian@google.com> From: Brendan Gregg Date: Tue, 31 Mar 2015 00:33:10 -0700 Message-ID: Subject: Re: [PATCH v6 0/4] perf: add support for profiling jitted code To: Stephane Eranian Cc: LKML , acme@redhat.com, Peter Zijlstra , Ingo Molnar , Andi Kleen , Jiri Olsa , Namhyung Kim , Rose Belcher , sukadev@linux.vnet.ibm.com, sonnyrao@chromium.org, johnmccutchan@google.com, David Ahern , adrian.hunter@intel.com, Pawel Moll Content-Type: text/plain; charset=UTF-8 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org G'Day Stephane, On Mon, Mar 30, 2015 at 3:19 PM, Stephane Eranian wrote: [...] > The current support only works when the runtime is monitored from > start to finish: perf record java --agentpath:libpfmjvmti.so my_class. > > Once the run is completed, the jitdump file needs to be injected into > the perf.data file. This is accomplished by using the perf inject command. > This will also generate an ELF image for each jitted function. The > inject MMAP records will point to those ELF images. The reasoning > behind using ELF images is that it makes processing for perf report > and annotate automatic and transparent. It also makes it easier to > package and analyze on a remote machine. [...] This is really impressive work. Do we have an idea of the overhead for running the java agent? Today, I'm using perf-map-agent, loaded dynamically, to dump a /tmp/perf*.map file as needed. My company has tens of thousands of Linux instances running Java, but very few need profiling, and we don't know which beforehand. So a snapshot-on-demand approach is ideal. An always-on approach, well, we'd have to know the overhead (I can build the agent and test...). During a profile, our symbols can change, but it's not been a serious problem. I can take a before and after snapshot of symbols (/tmp/perf.map files), then highlight which have moved and are therefore suspicious (not perfect, but ok). Yes, having a timestamped symbol log would be most accurate, but I wouldn't pay a lot of overhead for it. Is there a (horror) story behind why this was so necessary? Minor comment: I didn't notice an fsync() in the jvmti code. Can symbol updates be buffered and not written out by the time perf is reading them? Again, this is impressive work. Just wanted to share comments from a heavy user of perf with Java. You may already have other, better, ideas for doing on-demand symbol dumps anyway. Brendan