Re: 回复:Re: documentation about CTF event payload

Re: 回复:Re: documentation about CTF event payload

[Mathieu Desnoyers]

[-- Attachment #1.1: Type: text/plain, Size: 3597 bytes --]

Hi, 

(adding back lttng-dev in CC for the benefit of others) 

Whenever possible, we try to augment the trace data with such additional 
information at post-processing, because capturing it at run-time repeatedly ends 
up being costly. 

The lttng-analyses project contains state tracker which augment the trace data 
with mapping from file descriptor to corresponding file names (see lttnganalyses/linuxautomaton/io.py). 
I'm not sure if the Trace Compass project models this mapping between file descriptors and their 
associated file, but if not, it would be an *extremely* useful addition. 

lttng-modules already dumps the information needed to create that model: 

- lttng_statedump_file_descriptor dumps all existing file descriptors for all processes, 
- a few system calls (open, dup, dup2, dup3, close, clone(see CLONE_FILES flag), fork, 
fcntl(cmd==F_DUPFD)) allow tracking the file descriptor table state changes during the trace. 

Thanks, 

Mathieu 

----- On Nov 25, 2019, at 7:36 PM, 杨海 <hai.yang@magic-shield.com> wrote: 

> Hi Mathieu

> Thanks for quick response. Here let me give an example. For syscalls open, LTTng
> output filename in entry_open and output fd as ret in exit_open. It would be
> desired to output both filename and fd so we can correlate them.
> I am not sure whether there is a configuration that we can have the richest
> output regarding to syscalls.
> If not, we can modify lttng-modules to output what we need. Or any other
> recommendation?

> Regards
> Hai

> ----------

> 该邮件从移动设备发送

> --------------原始邮件--------------
> 发件人："Mathieu Desnoyers "<mathieu.desnoyers@efficios.com>;
> 发送时间：2019年11月20日(星期三) 晚上10:32
> 收件人："杨海" <hai.yang@magic-shield.com>;
> 抄送："lttng-dev "<lttng-dev@lists.lttng.org>;
> 主题：Re: [lttng-dev] documentation about CTF event payload
> -----------------------------------
> For the system call payload documentation, you might want to refer to the Linux
> system call
> man pages.

> For internal kernel tracepoints like sched_switch, there is no documentation of
> the meaning of
> each field at the moment. This state is the same as the upstream Linux kernel
> trace event. You'll
> have to figure it out on your own. Documenting each field of the ~500-1000 Linux
> kernel tracepoints
> is no small task.

> Thanks,

> Mathieu

> ----- On Nov 19, 2019, at 9:25 PM, 杨海 <hai.yang@magic-shield.com> wrote:

>> To be more specific, I suppose we can refer to
>> instrumentation\syscalls\3.10.0-rc7\x86-64-syscalls-3.10.0-rc7 for the payload
>> format of syscall event. Is it exactly in the CTF syscall event?

>> Regards
>> Hai
>> ------------------ Original ------------------
>> From: "杨海"<hai.yang@magic-shield.com>;
>> Date: Mon, Nov 18, 2019 09:54 AM
>> To: "lttng-dev"<lttng-dev@lists.lttng.org>;
>> Subject: documentation about CTF event payload
>> Hi

>> As LTTng generated CTF and babeltrace parse it, we have the output as attached.
>> We saw events such as sched_switch, but the payload cannot be understood
>> easily. Where we can find the document to explain the LTTng payload and
>> parameters?

>> Regards
>> Hai

>> _______________________________________________
>> lttng-dev mailing list
>> lttng-dev@lists.lttng.org
>> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

> --
> Mathieu Desnoyers
> EfficiOS Inc.
> http://www.efficios.com

-- 
Mathieu Desnoyers 
EfficiOS Inc. 
http://www.efficios.com 

[-- Attachment #1.2: Type: text/html, Size: 5665 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: 回复:Re: documentation about CTF event payload

[杨海]

[-- Attachment #1.1: Type: text/plain, Size: 4534 bytes --]

Thanks Mathieu. If I understand correctly, lttng-modules output necessary information as lttng_statedump_file_descriptor, and it is a part of CTF. But when babeltrace parse CTF, it does not process those information.
lttng-analyses is post-processing python scripts while babeltrace is realtime C parser.
If we decide to extend babeltrace to support the filename-fd mapping like iolog functionality in lttng-analyses, any recommendation?


Regards
Hai
 

------------------ Original ------------------
From: &nbsp;"Mathieu Desnoyers"<mathieu.desnoyers@efficios.com&gt;;
Date: &nbsp;Tue, Nov 26, 2019 10:47 PM
To: &nbsp;"杨海"<hai.yang@magic-shield.com&gt;; 
Cc: &nbsp;"lttng-dev"<lttng-dev@lists.lttng.org&gt;; "Matthew Khouzam"<matthew.khouzam@ericsson.com&gt;; "Geneviève Bastien"<gbastien@versatic.net&gt;; 
Subject: &nbsp;Re: 回复:Re: [lttng-dev] documentation about CTF event payload

&nbsp;

Hi,



(adding back lttng-dev in CC for the benefit of others)



Whenever possible, we try to augment the trace data with such additional

information at post-processing, because capturing it at run-time repeatedly ends
up being costly.



The lttng-analyses project contains state tracker which augment the trace data

with mapping from file descriptor to corresponding file names (see lttnganalyses/linuxautomaton/io.py).
I'm not sure if the Trace Compass project models this mapping between file descriptors and their

associated file, but if not, it would be an *extremely* useful addition.



lttng-modules already dumps the information needed to create that model:



- lttng_statedump_file_descriptor dumps all existing file descriptors for all processes,

- a few system calls (open, dup, dup2, dup3, close, clone(see CLONE_FILES flag), fork,
fcntl(cmd==F_DUPFD)) allow tracking the file descriptor table state changes during the trace.


Thanks,


Mathieu







----- On Nov 25, 2019, at 7:36 PM, 杨海 <hai.yang@magic-shield.com&gt; wrote:

Hi Mathieu

Thanks for quick response. Here let me give an example. For syscalls open, LTTng output filename in entry_open and output fd as ret in exit_open. It would be desired to output both filename and fd so we can correlate them. 
I am not sure whether there is a configuration that we can have the richest output regarding to syscalls. 
If not, we can modify lttng-modules to output what we need. Or any other recommendation?

Regards
Hai 

----------

该邮件从移动设备发送





--------------原始邮件--------------
发件人："Mathieu Desnoyers "<mathieu.desnoyers@efficios.com&gt;;
发送时间：2019年11月20日(星期三) 晚上10:32
收件人："杨海" <hai.yang@magic-shield.com&gt;;
抄送："lttng-dev "<lttng-dev@lists.lttng.org&gt;;
主题：Re: [lttng-dev] documentation about CTF event payload
-----------------------------------

 For the system call payload documentation, you might want to refer to the Linux system call

man pages.


For internal kernel tracepoints like sched_switch, there is no documentation of the meaning of

each field at the moment. This state is the same as the upstream Linux kernel trace event. You'll
have to figure it out on your own. Documenting each field of the ~500-1000 Linux kernel tracepoints
is no small task.


Thanks,


Mathieu


----- On Nov 19, 2019, at 9:25 PM, 杨海 <hai.yang@magic-shield.com&gt; wrote:

To be more specific, I suppose we can refer to instrumentation\syscalls\3.10.0-rc7\x86-64-syscalls-3.10.0-rc7 for the payload format of syscall event. Is it exactly in the CTF syscall event?

Regards
Hai
&nbsp;
------------------ Original ------------------
From: &nbsp;"杨海"<hai.yang@magic-shield.com&gt;;
Date: &nbsp;Mon, Nov 18, 2019 09:54 AM
To: &nbsp;"lttng-dev"<lttng-dev@lists.lttng.org&gt;; 
Subject: &nbsp;documentation about CTF event payload

&nbsp;
Hi

As LTTng generated CTF and babeltrace parse it, we have the output as attached. We saw events such as sched_switch, but the payload cannot be understood easily. Where we can find the document to explain the LTTng payload and parameters?

Regards
Hai




_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev


-- 

Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

 





-- 

Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

[-- Attachment #1.2: Type: text/html, Size: 9657 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Any tips to retrieve the relationship between socket-related processes

[杨海]

[-- Attachment #1.1.1: Type: text/plain, Size: 1682 bytes --]

Hello,


Below briefly describe our question, hope to know how it can be supported by Lttng, babeltrace, analyses etc.


## IPC Trace：
### What we are doing:
We are trying to use lttng to find some IPC trace


### How we did:
Open Lttng first，then ran server(pid = 1812) and client(pid = 1819) and collect trace.
Client would send message to server and server would just echo what it received from client. They talk with each other by file "/tmp/python_unix_sockets_example". 
Scripts can be found here: https://github.com/Aaron0813/LinuxAPTLog/tree/master/socket_AF_UNIX
Trace Link: https://github.com/Aaron0813/LinuxAPTLog/tree/master/socket_AF_UNIX/with_pid_03


###  Trace Analysis(Lttng-analysis IO)
1. sshd transmit data：Since we were using ssh to connect server and client, that's why we can see this trace
2. 1819 client: read data from tty
3. 1819 client: send data by socket
4. 1819 client: send data by tty and ssh
5. 1812 server: send data to tty
6. 1812 server: receive data from client
7. Gsd-color gnome: do some display
8. A new round cycle starts after blue line




### Conclusion
In Lttng-Analysis' log：
1. Could not find any trace that related to /tmp/python_unix_sockets_example(we are using this file to communicate)
2. Could not find any relationship that can help us determine whether several processes are communicating


As for babeltrace log，we can only locate and understand the communication processes between server and client with the help of "/tmp/python_unix_sockets_example". However, we cannot find anything that can let us know that these two processes are communicating.


Regards
Hai

[-- Attachment #1.1.2: Type: text/html, Size: 2146 bytes --]

[-- Attachment #1.2: B01FE1AF@A05EA129.CD00205E.jpg --]
[-- Type: image/jpeg, Size: 188715 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev