From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S966634AbbDYPyC (ORCPT ); Sat, 25 Apr 2015 11:54:02 -0400 Received: from mail-ig0-f171.google.com ([209.85.213.171]:37718 "EHLO mail-ig0-f171.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932949AbbDYPx7 (ORCPT ); Sat, 25 Apr 2015 11:53:59 -0400 Message-ID: <553BB895.20800@gmail.com> Date: Sat, 25 Apr 2015 09:53:57 -0600 From: David Ahern User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: Yunlong Song , a.p.zijlstra@chello.nl, paulus@samba.org, mingo@redhat.com, Arnaldo Carvalho de Melo CC: linux-kernel@vger.kernel.org, wangnan0@huawei.com Subject: Re: [Question] How does perf still record the stack of a specified pid even when that process is interrupted and CPU is scheduled to other process References: <553A45CA.8020808@huawei.com> <553A4C18.3030609@gmail.com> <553B9F30.1040100@huawei.com> In-Reply-To: <553B9F30.1040100@huawei.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 4/25/15 8:05 AM, Yunlong Song wrote: > On 2015/4/24 21:58, David Ahern wrote: >> On 4/24/15 7:31 AM, Yunlong Song wrote: >>> Now we are profiling the performance of ext4 and f2fs on an eMMC card with iozone, >>> we find a case that ext4 is better than f2fs in random write under the test of >>> "iozone -s 262144 -r 64 -i 0 -i 2". We want to analyze the I/O delay of the two >>> file systems. We have got a conclusion that 1% of sys_write takes up 60% time of >>> the overall sys_write (262144/64=4096). We want to find out the call stack during >>> this specific 1% sys_write. Our idea is to record the stack in a certain time period >>> and since the specific 1% case takes up 60% time, the total number of records of its >>> stack should also takes up 60% of the total records, then we can recognize those stacks >>> and figure out what the I/O stack of f2fs is doing in the 1% case. >> >> And to address this specific profiling problem have you tried: >> >> perf trace record -- iozone ... >> perf trace -i perf.data -S >> >> >> >> > > But this only shows the system call like strace, but we want the call stack of kernel functions > in fact. > We haven't added the callchain option yet; on the to-do list. perf trace record -g -- iozone ... perf trace -i perf.data -s --> summary of system calls, max/min/average times perf trace -i perf.data --duration 10.0 -T --> note the timestamp where the write took a "long" time perf script --> search down to *around* the time of interest; you want the syscall entry; timestamp is for exit