From mboxrd@z Thu Jan 1 00:00:00 1970 From: Brendan Gregg Subject: Re: perf segfault in docker container Date: Tue, 21 Jun 2016 15:43:01 -0700 Message-ID: References: <575A9660.4070907@linux.vnet.ibm.com> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Return-path: Received: from mail-io0-f171.google.com ([209.85.223.171]:35526 "EHLO mail-io0-f171.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752132AbcFUWnc (ORCPT ); Tue, 21 Jun 2016 18:43:32 -0400 Received: by mail-io0-f171.google.com with SMTP id f30so29402008ioj.2 for ; Tue, 21 Jun 2016 15:43:31 -0700 (PDT) In-Reply-To: Sender: linux-perf-users-owner@vger.kernel.org List-ID: To: Aravinda Prasad Cc: "linux-perf-use." , Wang Nan , Hari Bathini , Ananth M , "Naveen N. Rao" On Tue, Jun 21, 2016 at 3:32 PM, Brendan Gregg wrote: > G'Day Aravinda, > [...] >> We would like to know if you are looking for "container-aware tracing" >> and also like to know the scenarios/problems you are trying to debug by >> running perf inside a container. > > Yes, perf needs to be container-aware. > > To start with, we'd like to profile apps running inside Docker > containers, either by running perf in the container, or by running > perf from the host. As in, "perf record -F49 -a -g -- sleep 30". I've > tried both and had both approaches work, with some wrestling of > /tmp/perf-PID.map files and things. > > If perf was container-aware, then running it in the container should > be the easiest way to profile an app, if it's only sampling that > container. > > Also, from within a container, I'd expect to be able to sample kernel > stacks that are running for the container processes (eg, syscalls), > but not asynchronous kernel threads that are running host-wide (eg, > background fsflush). > > More advanced things would involve tracing syscall latency and using > BPF for latency histograms, from within a container. That should be > allowed. > > What about tracepoints? Should a container be able to use the block > I/O tracepoints and see disk I/O latency histograms? Filtering this to > be just the container's block I/O would be tricky. Doing it > system-wide may be allowable, depending on a setting in > perf_event_paranoid. I think in some environments, having a container > trace all tracepoints (disk, tcp, etc) is ok, provided to data is > leaked from another container; whereas in other environments tracing > non-container events would not be ok. Hence setting this in > perf_event_paranoid. > > Brendan An addition for container-aware tracing: perf from the host should be able to find the correct /tmp/perf-PID.map files, even though they are in the container /tmp's. Brendan