From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B4F6EC433ED for ; Wed, 31 Mar 2021 17:56:35 +0000 (UTC) Received: from lists.lttng.org (lists.lttng.org [167.114.26.123]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D982A60FE8 for ; Wed, 31 Mar 2021 17:56:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D982A60FE8 Authentication-Results: mail.kernel.org; dmarc=pass (p=none dis=none) header.from=lists.lttng.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lttng-dev-bounces@lists.lttng.org Received: from lists-lttng01.efficios.com (localhost [IPv6:::1]) by lists.lttng.org (Postfix) with ESMTP id 4F9YsY19w4zSLq; Wed, 31 Mar 2021 13:56:33 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=lists.lttng.org; s=default; t=1617213393; bh=Sk9i6V5dd8IMM/qIvJePrCAznKBxGM5gq81Ga/kl1dM=; h=References:In-Reply-To:Date:To:Cc:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=bk6C+ESBF4vj+Hxp4hyLN9ROBstoUlJ5OQ+H7riP+vsEjDJRZGTxKYQBQpHtO2j3d LNhO5ZSXO7gXYsEFzuOGk3LUBgbjfljSEeSXN9j5A102Df902eBxTJoEPdBZuZCBtz ZhLsi0s1/csLjQ7gHrO0b1SfSC0XZamX4PDSU+0fJouGg+KGr7GN2DqPOp4Siha1tB iDA87EBiIHvegK70g9vpzQmUWxmEPa5kiMGcdLez6GPBR9DStGQrb591sRIE51YECS BjmEKK88OzB4bCjybZD/VYhvqR5Dry0glWxW7n9p1fkC+eEXlEk64nqvhzfybOG8Dy 7WUkVuyh0yDog== Received: from mail-lj1-x22c.google.com (mail-lj1-x22c.google.com [IPv6:2a00:1450:4864:20::22c]) by lists.lttng.org (Postfix) with ESMTPS id 4F9YsW2qyMzRX0 for ; Wed, 31 Mar 2021 13:56:31 -0400 (EDT) Received: by mail-lj1-x22c.google.com with SMTP id u10so24909595lju.7 for ; Wed, 31 Mar 2021 10:56:31 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=3xX2MHnr9GE96CnVFoPEiNXsCpy0qdanYxhKkcqnGiw=; b=p28dqrmHWaO2xKcXMj62UidB+CVBJaHR99TfoB3oLPyAzglF0Gtsir/fI2GCqUR2QK Tj0cJr0RMZ7yWYxKVJMoU/34PaRPuxEyZrsUgdVENI8Q7ZPPT2tdXp8794i2ZnstEZPK 8ieyGdN4SHDgFWf7LygNNeknmjq1ezwvgQ/7MmSAKHjkod8Djof0qUmNTZqgtYv7l6Dr V7msctM3VaNzzbyeXAAHPPe5CJ0hVEICjIyQ+HuFLw3DZ7YKMbSGXxlujItjm4Ucoh2C SfYINOb/FhjtkoSrCIJloUAJnDbjw9exf2mBzWjqR2xac7Hk8D6UK74jXQ9DAwYXo6OA dO7g== X-Gm-Message-State: AOAM532kRiaNthUP+e/60PUYWtPAXhUTGa+1ScTo1NRnUqZ56B6R6z3B FlW+Ex6oe+sKZW9NHNKPo6VbzuFdaSFclFEATMU= X-Google-Smtp-Source: ABdhPJyLU0LQMMZIxbxhCuJjPKzy3aEQEjKbpv0P6vcgSzjkDxY1wBZB/OwnmbrMMVKNmeKG6FbZqVlC5PHKrzOqlNQ= X-Received: by 2002:a2e:b6c6:: with SMTP id m6mr2745438ljo.411.1617213389792; Wed, 31 Mar 2021 10:56:29 -0700 (PDT) MIME-Version: 1.0 References: <20210331174201.GB28307@joraj-alpa> In-Reply-To: <20210331174201.GB28307@joraj-alpa> Date: Wed, 31 Mar 2021 12:55:53 -0500 Message-ID: To: Jonathan Rajotte-Julien Cc: lttng-dev@lists.lttng.org Subject: Re: [lttng-dev] Can't trace function calls X-BeenThere: lttng-dev@lists.lttng.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: LTTng development list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Ramesh Errabolu via lttng-dev Reply-To: Ramesh Errabolu Content-Type: multipart/mixed; boundary="===============2471159736815172805==" Errors-To: lttng-dev-bounces@lists.lttng.org Sender: "lttng-dev" --===============2471159736815172805== Content-Type: multipart/alternative; boundary="0000000000009fead705bed8d52c" --0000000000009fead705bed8d52c Content-Type: text/plain; charset="UTF-8" Thanks for the quick response, appreciate it. At a high level this is what I did. 1. reboot 2. login as sudo 3. set LD_LIBRARY_PATH to point to right locations 4. ldd of executable to show right library versions are picked up 5. execute the workload by specifiy LD_PRELOAD env variable 6. expect to see a trace file in my $HOME/ltt* directory When I say *FAIL*, I mean that there was no trace file in my home directory ~/lttng* I read - https://lttng.org/man/3/lttng-ust-cyg-profile/v2.12/ I didn't read - https://lttng.org/docs/v2.12/#doc-liblttng-ust-cyg-profile I have not run any lttng commands besides what I already gave. If they are needed, I don't know what they are. I tried experimenting with lttng commands create and list. The command to list "traceable" points returned an error as included below. root@RocrLnx23:~/git/compute/out/ubuntu-18.04/18.04/bin# ls ~/ | grep -i ltt root@RocrLnx23:~/git/compute/out/ubuntu-18.04/18.04/bin# lttng create my-kernel-session --output=~/my-kernel-trace Session my-kernel-session created. Traces will be output to /home/user1/git/compute/out/ubuntu-18.04/18.04/bin/~/my-kernel-trace root@RocrLnx23:~/git/compute/out/ubuntu-18.04/18.04/bin# *lttng list --kerne*l *Error: Unable to list kernel events: Kernel tracer not available* root@RocrLnx23:~/git/compute/out/ubuntu-18.04/18.04/bin# A grep for lttng related processes shows me a daemon service running. root@RocrLnx23:~/git/compute/out/ubuntu-18.04/18.04/bin# ps -ef | grep ltt root 1002 1 0 12:16 ? 00:00:00 /usr/bin/lttng-sessiond root 1054 1002 0 12:16 ? 00:00:00 */usr/bin/lttng-sessiond* root 3145 2861 0 12:51 pts/0 00:00:00 grep --color=auto ltt root@RocrLnx23:~/git/compute/out/ubuntu-18.04/18.04/bin# Regards, Ramesh On Wed, Mar 31, 2021 at 12:42 PM Jonathan Rajotte-Julien < jonathan.rajotte-julien@efficios.com> wrote: > Hi, > > On Wed, Mar 31, 2021 at 10:51:03AM -0500, Ramesh Errabolu via lttng-dev > wrote: > > I am trying to capture the list of functions being called by an app. The > > app relies on a couple of shared libraries (libhsa-runtime64.so and > > libhsakmt.so). > > ok. > > > > > My experiments all *FAIL *i.e. there are no log files of the output. > > Not sure what you mean here. But let's continue first. > > > The > > link on lttng-ust-cyg-profile does not tell me where I can get them. > > Could you share the "link" in question? > > > Instrument the build files for the two libraries to allow function > tracing > > by the gcc/clang flags -finstrument-functions and run workload as follows > > > > - Instrument the build files for the two libraries to allow function > > tracing by the gcc/clang flags *-finstrument-functions* and run > workload > > as follows > > Okai. I expect that you have read this: > https://lttng.org/docs/v2.12/#doc-liblttng-ust-cyg-profile > > and this > https://lttng.org/man/3/lttng-ust-cyg-profile/v2.12/ > > > I am using LTTng version Ubuntu: LTTng Stable 2.12 PPA > > > > - apt-get install lttng-tools > > - apt-get install lttng-modules-dkms > > - apt-get install liblttng-ust-dev > > > > Looks good. > > > I noticed no kernel modules with substring "ltt" in it. > > Okai so this is where we need much more information. > > First of all, what are the lttng commands used for your experiments? > (lttng create, lttng enable-channel, lttng enable-event...) > Please provide a sample script of your experiments. > > Second, you are mostly performing userspace tracing here, lttng-modules ( > the > kernel tracer) might not even be needed here unless you also want to do > kernel > tracing. > > What do you mean by "there are no log files of the output."? LTTng produces > traces encoded in CTF (Common Trace Format) that must be read by a reader > (Babeltrace most of the time). > > Cheers > > -- > Jonathan Rajotte-Julien > EfficiOS > --0000000000009fead705bed8d52c Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Thanks for the quick response, appreciate it.

At a high = level this is what I did.
  1. reboot
  2. login as sudo
  3. set = LD_LIBRARY_PATH to point to right locations
  4. ldd of executable to sh= ow right library versions are picked up
  5. execute the workload by spe= cifiy LD_PRELOAD env variable
  6. expect to see a trace file in my $HOM= E/ltt* directory
When I say FAIL, I mean that there was no= trace file in my home directory ~/lttng*

I read -=C2=A0https://lttng.org/man/3/= lttng-ust-cyg-profile/v2.12/
I didn't read -=C2=A0https://lttng.org/doc= s/v2.12/#doc-liblttng-ust-cyg-profile

I have not run any lttng com= mands besides what I already gave. If they are needed, I don't know wha= t they are.

I tried experimenting with lttng commands create and list.= The command to list "traceable" points returned an error as incl= uded below.

root@RocrLnx23:~/git/compute/out/ubuntu-18.04= /18.04/bin# ls ~/ | grep -i ltt
root@RocrLnx= 23:~/git/compute/out/ubuntu-18.04/18.04/bin# lttng create my-kernel-session= --output=3D~/my-kernel-trace
Session my-ker= nel-session created.
Traces will be output t= o /home/user1/git/compute/out/ubuntu-18.04/18.04/bin/~/my-kernel-trace
root@RocrLnx23:~/git/compute/out/ubuntu-18.04/18= .04/bin# lttng list --kernel
Error= : Unable to list kernel events: Kernel tracer not available
root@RocrLnx23:~/git/compute/out/ubuntu-18.04/18.04/bin= #=C2=A0

A grep for lttng related processes shows m= e a daemon service running.

root@RocrLnx23:~/git/compute/= out/ubuntu-18.04/18.04/bin# ps -ef | grep ltt
root =C2=A0 =C2=A0 =C2=A01002 =C2=A0 =C2=A0 1 =C2=A00 12:16 ? =C2=A0 =C2= =A0 =C2=A0 =C2=A000:00:00 /usr/bin/lttng-sessiond
root =C2=A0 =C2=A0 =C2=A01054 =C2=A01002 =C2=A00 12:16 ? =C2=A0 =C2= =A0 =C2=A0 =C2=A000:00:00 /usr/bin/lttng-sessiond
root =C2=A0 =C2=A0 =C2=A03145 =C2=A02861 =C2=A00 12:51 pts/0 = =C2=A0 =C2=A000:00:00 grep --color=3Dauto ltt
root@RocrLnx23:~/git/compute/out/ubuntu-18.04/18.04/bin#=C2=A0

Regards,
Ramesh

=
On Wed= , Mar 31, 2021 at 12:42 PM Jonathan Rajotte-Julien <jonathan.rajotte-julien@efficios.com> wrote:
Hi,=

On Wed, Mar 31, 2021 at 10:51:03AM -0500, Ramesh Errabolu via lttng-dev wro= te:
> I am trying to capture the list of functions being called by an app. T= he
> app relies on a couple of shared libraries (libhsa-runtime64.so and > libhsakmt.so).

ok.

>
> My experiments all *FAIL *i.e. there are no log files of the output.
Not sure what you mean here. But let's continue first.

> The
> link on lttng-ust-cyg-profile does not tell me where I can get them.
Could you share the "link" in question?

> Instrument the build files for the two libraries to allow function tra= cing
> by the gcc/clang flags -finstrument-functions and run workload as foll= ows
>
>=C2=A0 =C2=A0 - Instrument the build files for the two libraries to all= ow function
>=C2=A0 =C2=A0 tracing by the gcc/clang flags *-finstrument-functions* a= nd run workload
>=C2=A0 =C2=A0 as follows

Okai. I expect that you have read this:
https://lttng.org/docs/v2.12/#doc-liblttn= g-ust-cyg-profile

and this
https://lttng.org/man/3/lttng-ust-cyg-profile/v2= .12/

> I am using LTTng version Ubuntu: LTTng Stable 2.12 PPA
>
>=C2=A0 =C2=A0 - apt-get install lttng-tools
>=C2=A0 =C2=A0 - apt-get install lttng-modules-dkms
>=C2=A0 =C2=A0 - apt-get install liblttng-ust-dev
>

Looks good.

> I noticed no kernel modules with substring "ltt" in it.

Okai so this is where we need much more information.

First of all, what are the lttng commands used for your experiments? (lttng= create, lttng enable-channel, lttng enable-event...)
Please provide a sample script of your experiments.

Second, you are mostly performing userspace tracing here, lttng-modules ( t= he
kernel tracer) might not even be needed here unless you also want to do ker= nel
tracing.

What do you mean by "there are no log files of the output."? LTTn= g produces
traces encoded in CTF (Common Trace Format) that must be read by a reader (Babeltrace most of the time).

Cheers

--
Jonathan Rajotte-Julien
EfficiOS
--0000000000009fead705bed8d52c-- --===============2471159736815172805== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ lttng-dev mailing list lttng-dev@lists.lttng.org https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev --===============2471159736815172805==--