lttng-dev.lists.lttng.org archive mirror
 help / color / mirror / Atom feed
* Re: 回复:Re:  documentation about CTF event payload
       [not found] <tencent_5F6B34D931D7AD241AA8EF79@qq.com>
@ 2019-11-26 14:47 ` Mathieu Desnoyers
       [not found] ` <733360507.225.1574779654690.JavaMail.zimbra@efficios.com>
  1 sibling, 0 replies; 4+ messages in thread
From: Mathieu Desnoyers @ 2019-11-26 14:47 UTC (permalink / raw)
  To: 杨海; +Cc: lttng-dev


[-- Attachment #1.1: Type: text/plain, Size: 3597 bytes --]

Hi, 

(adding back lttng-dev in CC for the benefit of others) 

Whenever possible, we try to augment the trace data with such additional 
information at post-processing, because capturing it at run-time repeatedly ends 
up being costly. 

The lttng-analyses project contains state tracker which augment the trace data 
with mapping from file descriptor to corresponding file names (see lttnganalyses/linuxautomaton/io.py). 
I'm not sure if the Trace Compass project models this mapping between file descriptors and their 
associated file, but if not, it would be an *extremely* useful addition. 

lttng-modules already dumps the information needed to create that model: 

- lttng_statedump_file_descriptor dumps all existing file descriptors for all processes, 
- a few system calls (open, dup, dup2, dup3, close, clone(see CLONE_FILES flag), fork, 
fcntl(cmd==F_DUPFD)) allow tracking the file descriptor table state changes during the trace. 

Thanks, 

Mathieu 

----- On Nov 25, 2019, at 7:36 PM, 杨海 <hai.yang@magic-shield.com> wrote: 

> Hi Mathieu

> Thanks for quick response. Here let me give an example. For syscalls open, LTTng
> output filename in entry_open and output fd as ret in exit_open. It would be
> desired to output both filename and fd so we can correlate them.
> I am not sure whether there is a configuration that we can have the richest
> output regarding to syscalls.
> If not, we can modify lttng-modules to output what we need. Or any other
> recommendation?

> Regards
> Hai

> ----------

> 该邮件从移动设备发送

> --------------原始邮件--------------
> 发件人:"Mathieu Desnoyers "<mathieu.desnoyers@efficios.com>;
> 发送时间:2019年11月20日(星期三) 晚上10:32
> 收件人:"杨海" <hai.yang@magic-shield.com>;
> 抄送:"lttng-dev "<lttng-dev@lists.lttng.org>;
> 主题:Re: [lttng-dev] documentation about CTF event payload
> -----------------------------------
> For the system call payload documentation, you might want to refer to the Linux
> system call
> man pages.

> For internal kernel tracepoints like sched_switch, there is no documentation of
> the meaning of
> each field at the moment. This state is the same as the upstream Linux kernel
> trace event. You'll
> have to figure it out on your own. Documenting each field of the ~500-1000 Linux
> kernel tracepoints
> is no small task.

> Thanks,

> Mathieu

> ----- On Nov 19, 2019, at 9:25 PM, 杨海 <hai.yang@magic-shield.com> wrote:

>> To be more specific, I suppose we can refer to
>> instrumentation\syscalls\3.10.0-rc7\x86-64-syscalls-3.10.0-rc7 for the payload
>> format of syscall event. Is it exactly in the CTF syscall event?

>> Regards
>> Hai
>> ------------------ Original ------------------
>> From: "杨海"<hai.yang@magic-shield.com>;
>> Date: Mon, Nov 18, 2019 09:54 AM
>> To: "lttng-dev"<lttng-dev@lists.lttng.org>;
>> Subject: documentation about CTF event payload
>> Hi

>> As LTTng generated CTF and babeltrace parse it, we have the output as attached.
>> We saw events such as sched_switch, but the payload cannot be understood
>> easily. Where we can find the document to explain the LTTng payload and
>> parameters?

>> Regards
>> Hai

>> _______________________________________________
>> lttng-dev mailing list
>> lttng-dev@lists.lttng.org
>> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

> --
> Mathieu Desnoyers
> EfficiOS Inc.
> http://www.efficios.com

-- 
Mathieu Desnoyers 
EfficiOS Inc. 
http://www.efficios.com 

[-- Attachment #1.2: Type: text/html, Size: 5665 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: 回复:Re:  documentation about CTF event payload
       [not found] ` <733360507.225.1574779654690.JavaMail.zimbra@efficios.com>
@ 2019-12-02  2:16   ` 杨海
  2020-01-16  6:21   ` Any tips to retrieve the relationship between socket-related processes 杨海
  1 sibling, 0 replies; 4+ messages in thread
From: 杨海 @ 2019-12-02  2:16 UTC (permalink / raw)
  To: Mathieu Desnoyers; +Cc: lttng-dev


[-- Attachment #1.1: Type: text/plain, Size: 4534 bytes --]

Thanks&nbsp;Mathieu. If I understand correctly, lttng-modules output necessary information as&nbsp;lttng_statedump_file_descriptor, and it is a part of CTF. But when babeltrace parse CTF, it does not process those information.
lttng-analyses is post-processing python scripts while babeltrace is realtime C parser.
If we decide to extend babeltrace to support the filename-fd mapping like iolog functionality in lttng-analyses, any recommendation?


Regards
Hai
&nbsp;

------------------&nbsp;Original&nbsp;------------------
From: &nbsp;"Mathieu Desnoyers"<mathieu.desnoyers@efficios.com&gt;;
Date: &nbsp;Tue, Nov 26, 2019 10:47 PM
To: &nbsp;"杨海"<hai.yang@magic-shield.com&gt;; 
Cc: &nbsp;"lttng-dev"<lttng-dev@lists.lttng.org&gt;; "Matthew Khouzam"<matthew.khouzam@ericsson.com&gt;; "Geneviève Bastien"<gbastien@versatic.net&gt;; 
Subject: &nbsp;Re: 回复:Re: [lttng-dev] documentation about CTF event payload

&nbsp;

Hi,



(adding back lttng-dev in CC for the benefit of others)



Whenever possible, we try to augment the trace data with such additional

information at post-processing, because capturing it at run-time repeatedly ends
up being costly.



The lttng-analyses project contains state tracker which augment the trace data

with mapping from file descriptor to corresponding file names (see lttnganalyses/linuxautomaton/io.py).
I'm not sure if the Trace Compass project models this mapping between file descriptors and their

associated file, but if not, it would be an *extremely* useful addition.



lttng-modules already dumps the information needed to create that model:



- lttng_statedump_file_descriptor dumps all existing file descriptors for all processes,

- a few system calls (open, dup, dup2, dup3, close, clone(see CLONE_FILES flag), fork,
fcntl(cmd==F_DUPFD)) allow tracking the file descriptor table state changes during the trace.


Thanks,


Mathieu







----- On Nov 25, 2019, at 7:36 PM, 杨海 <hai.yang@magic-shield.com&gt; wrote:

Hi Mathieu

Thanks for quick response. Here let me give an example. For syscalls open, LTTng output filename in entry_open and output fd as ret in exit_open. It would be desired to output both filename and fd so we can correlate them. 
I am not sure whether there is a configuration that we can have the richest output regarding to syscalls. 
If not, we can modify lttng-modules to output what we need. Or any other recommendation?

Regards
Hai 

----------

该邮件从移动设备发送





--------------原始邮件--------------
发件人:"Mathieu Desnoyers "<mathieu.desnoyers@efficios.com&gt;;
发送时间:2019年11月20日(星期三) 晚上10:32
收件人:"杨海" <hai.yang@magic-shield.com&gt;;
抄送:"lttng-dev "<lttng-dev@lists.lttng.org&gt;;
主题:Re: [lttng-dev] documentation about CTF event payload
-----------------------------------

 For the system call payload documentation, you might want to refer to the Linux system call

man pages.


For internal kernel tracepoints like sched_switch, there is no documentation of the meaning of

each field at the moment. This state is the same as the upstream Linux kernel trace event. You'll
have to figure it out on your own. Documenting each field of the ~500-1000 Linux kernel tracepoints
is no small task.


Thanks,


Mathieu


----- On Nov 19, 2019, at 9:25 PM, 杨海 <hai.yang@magic-shield.com&gt; wrote:

To be more specific, I suppose we can refer to instrumentation\syscalls\3.10.0-rc7\x86-64-syscalls-3.10.0-rc7 for the payload format of syscall event. Is it exactly in the CTF syscall event?

Regards
Hai
&nbsp;
------------------ Original ------------------
From: &nbsp;"杨海"<hai.yang@magic-shield.com&gt;;
Date: &nbsp;Mon, Nov 18, 2019 09:54 AM
To: &nbsp;"lttng-dev"<lttng-dev@lists.lttng.org&gt;; 
Subject: &nbsp;documentation about CTF event payload

&nbsp;
Hi

As LTTng generated CTF and babeltrace parse it, we have the output as attached. We saw events such as sched_switch, but the payload cannot be understood easily. Where we can find the document to explain the LTTng payload and parameters?

Regards
Hai




_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev


-- 

Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

 





-- 

Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

[-- Attachment #1.2: Type: text/html, Size: 9657 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Any tips to retrieve the relationship between socket-related processes
       [not found] ` <733360507.225.1574779654690.JavaMail.zimbra@efficios.com>
  2019-12-02  2:16   ` 杨海
@ 2020-01-16  6:21   ` 杨海
  1 sibling, 0 replies; 4+ messages in thread
From: 杨海 @ 2020-01-16  6:21 UTC (permalink / raw)
  To: lttng-dev


[-- Attachment #1.1.1: Type: text/plain, Size: 1682 bytes --]

Hello,


Below briefly describe our question, hope to know how it can be supported by Lttng, babeltrace, analyses etc.


## IPC Trace:
### What we are doing:
We are trying to use lttng to find some IPC trace


### How we did:
Open Lttng first,then ran server(pid = 1812) and client(pid = 1819) and collect trace.
Client would send message to server and server would just echo what it received from client. They talk with each other by file "/tmp/python_unix_sockets_example".&nbsp;
Scripts can be found here: https://github.com/Aaron0813/LinuxAPTLog/tree/master/socket_AF_UNIX
Trace Link: https://github.com/Aaron0813/LinuxAPTLog/tree/master/socket_AF_UNIX/with_pid_03


###&nbsp; Trace Analysis(Lttng-analysis IO)
1. sshd transmit data:Since we were using ssh to connect server and client, that's why we can see this trace
2. 1819 client: read data from tty
3. 1819 client: send data by socket
4. 1819 client: send data by tty and ssh
5. 1812 server: send data to tty
6. 1812 server: receive data from client
7. Gsd-color gnome: do some display
8. A new round cycle starts after blue line




### Conclusion
In Lttng-Analysis' log:
1. Could not find any trace that related to /tmp/python_unix_sockets_example(we are using this file to communicate)
2. Could not find any relationship that can help us determine whether several processes are communicating


As for babeltrace log,we can only locate and understand the communication processes between server and client with the help of "/tmp/python_unix_sockets_example". However, we cannot find anything that can let us know that these two processes are communicating.


Regards
Hai

[-- Attachment #1.1.2: Type: text/html, Size: 2146 bytes --]

[-- Attachment #1.2: B01FE1AF@A05EA129.CD00205E.jpg --]
[-- Type: image/jpeg, Size: 188715 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: Any tips to retrieve the relationship between socket-related processes
@ 2020-01-20  2:06 杨海
  0 siblings, 0 replies; 4+ messages in thread
From: 杨海 @ 2020-01-20  2:06 UTC (permalink / raw)
  To: lttng-dev

[-- Attachment #1: Type: text/plain, Size: 2358 bytes --]

Spam detection software, running on the system "ciao.gmane.io",
has identified this incoming email as possible spam.  The original
message has been attached to this so you can view it or label
similar future email.  If you have any questions, see
@@CONTACT_ADDRESS@@ for details.

Content preview:  Hello, I read this article and want to know whether it is
  true that UDP cannot be synchronized between traces and experimental branch
   is needed to get the below-mentioned issue resolved. http://archive.eclip
   [...] 

Content analysis details:   (5.2 points, 5.0 required)

 pts rule name              description
---- ---------------------- --------------------------------------------------
 0.0 URIBL_BLOCKED          ADMINISTRATOR NOTICE: The query to URIBL was
                            blocked.  See
                            http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block
                             for more information.
                            [URIs: eclipse.org]
 0.3 URIBL_RHS_DOB          Contains an URI of a new domain (Day Old Bread)
                            [URIs: magic-shield.com]
 2.4 RCVD_IN_SORBS_SOCKS    RBL: SORBS: sender is open SOCKS proxy
                            server
                            [59.36.132.88 listed in dnsbl.sorbs.net]
 2.5 RCVD_IN_SORBS_HTTP     RBL: SORBS: sender is open HTTP proxy server
 0.6 HK_RANDOM_ENVFROM      Envelope sender username looks random
-0.0 SPF_HELO_PASS          SPF: HELO matches SPF record
-0.0 SPF_PASS               SPF: sender matches SPF record
 0.2 HEADER_FROM_DIFFERENT_DOMAINS From and EnvelopeFrom 2nd level
                            mail domains are different
 0.0 HTML_MESSAGE           BODY: HTML included in message
 0.0 HTML_NONELEMENT_30_40  BODY: 30% to 40% of HTML elements are
                            non-standard
-1.0 MAILING_LIST_MULTI     Multiple indicators imply a widely-seen list
                            manager
 0.1 FROM_EXCESS_BASE64     From: base64 encoded unnecessarily
 0.0 MSGID_FROM_MTA_HEADER  Message-Id was added by a relay

The original message was not completely plain text, and may be unsafe to
open with some email clients; in particular, it may contain a virus,
or confirm that your address can receive spam.  If you wish to view
it, it may be safer to save it to a file and open it with an editor.


[-- Attachment #2: original message before SpamAssassin --]
[-- Type: message/rfc822, Size: 267035 bytes --]

[-- Attachment #2.1.1.1.1: Type: text/plain, Size: 2345 bytes --]

Hello,


I read this article and want to know whether it is true that UDP cannot be synchronized between traces and experimental branch is needed to get the below-mentioned issue resolved.
http://archive.eclipse.org/tracecompass/doc/org.eclipse.tracecompass.doc.user/Trace-synchronization.html#Trace_synchronization&nbsp;


Regards
Hai
&nbsp;
&nbsp;
------------------&nbsp;Original&nbsp;------------------
From: &nbsp;"杨海"<hai.yang@magic-shield.com&gt;;
Date: &nbsp;Thu, Jan 16, 2020 02:21 PM
To: &nbsp;"lttng-dev"<lttng-dev@lists.lttng.org&gt;; 

Subject: &nbsp;Any tips to retrieve the relationship between socket-related processes

&nbsp;

Hello,


Below briefly describe our question, hope to know how it can be supported by Lttng, babeltrace, analyses etc.


## IPC Trace:
### What we are doing:
We are trying to use lttng to find some IPC trace


### How we did:
Open Lttng first,then ran server(pid = 1812) and client(pid = 1819) and collect trace.
Client would send message to server and server would just echo what it received from client. They talk with each other by file "/tmp/python_unix_sockets_example". 
Scripts can be found here: https://github.com/Aaron0813/LinuxAPTLog/tree/master/socket_AF_UNIX
Trace Link: https://github.com/Aaron0813/LinuxAPTLog/tree/master/socket_AF_UNIX/with_pid_03


###&nbsp; Trace Analysis(Lttng-analysis IO)
1. sshd transmit data:Since we were using ssh to connect server and client, that's why we can see this trace
2. 1819 client: read data from tty
3. 1819 client: send data by socket
4. 1819 client: send data by tty and ssh
5. 1812 server: send data to tty
6. 1812 server: receive data from client
7. Gsd-color gnome: do some display
8. A new round cycle starts after blue line




### Conclusion
In Lttng-Analysis' log:
1. Could not find any trace that related to /tmp/python_unix_sockets_example(we are using this file to communicate)
2. Could not find any relationship that can help us determine whether several processes are communicating


As for babeltrace log,we can only locate and understand the communication processes between server and client with the help of "/tmp/python_unix_sockets_example". However, we cannot find anything that can let us know that these two processes are communicating.


Regards
Hai

[-- Attachment #2.1.1.1.2: Type: text/html, Size: 3290 bytes --]

[-- Attachment #2.1.1.2: FF639AB3@028A5D14.1E0B255E --]
[-- Type: application/octet-stream, Size: 188715 bytes --]

[-- Attachment #2.1.2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2020-01-20  2:06 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
     [not found] <tencent_5F6B34D931D7AD241AA8EF79@qq.com>
2019-11-26 14:47 ` 回复:Re: documentation about CTF event payload Mathieu Desnoyers
     [not found] ` <733360507.225.1574779654690.JavaMail.zimbra@efficios.com>
2019-12-02  2:16   ` 杨海
2020-01-16  6:21   ` Any tips to retrieve the relationship between socket-related processes 杨海
2020-01-20  2:06 杨海

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).