Re: LTTng system call tracing on ARM

Re: LTTng system call tracing on ARM

[Pierre-Luc St-Charles]

[-- Attachment #1.1: Type: text/plain, Size: 2107 bytes --]

Hey Jan,

I cannot speak on behalf of everyone here, but during our attempt to port
LTTng to Android, we also noticed that the kernels we were using (3.0.x)
were nowhere near the requirements for syscall tracepoints support. I
believe such support was added on x86/64 way earlier (early 3.x) than on
ARM, which is why it was included in LTTng's modules a while ago. Simply
put, the ARM kernel is late.

There are a few actuals ways to 'enable' syscall tracepoints support on
early ARM kernels, but they all including a bit of kernel hacking/patching.
I could send you some links if you're interested in that.

-PL
On Apr 10, 2013 8:37 AM, "PLSTC" <b0mb00z.it@gmail.com> wrote:

> Hey Jan,
>
> I cannot speak on behalf of everyone here, but during our attempt to port
> LTTng to Android, we also noticed that the kernels we were using (3.0.x)
> were nowhere near the requirements for syscall tracepoints support. I
> believe such support was added on x86/64 way earlier (early 3.x) than on
> ARM, which is why it was included in LTTng's modules a while ago. Simply
> put, the ARM kernel is late.
>
> There are a few actuals ways to 'enable' syscall tracepoints support on
> early ARM kernels, but they all including a bit of kernel hacking/patching.
> I could send you some links if you're interested in that.
>
> -PL
> On Apr 10, 2013 4:40 AM, "Jan Glauber" <jan.glauber@gmail.com> wrote:
>
>> Hi,
>>
>> I want to use LTTng for system call tracing on ARM. Now lttng-modules
>> seems
>> to support system call tracing on ARM already since
>> "8f4f80e LTTng Modules ARM syscall instrumentation".
>>
>> But I wonder how that worked since lttng-syscalls.c is only build under
>> CONFIG_HAVE_SYSCALL_TRACEPOINTS and that was added to ARM only with
>> kernel 3.6
>> (much after than the lttng-modules commit).
>>
>> Am I missing something? Is system call tracing working on ARM with the
>> upstream
>> LTTng version?
>>
>> thanks,
>> Jan
>>
>> _______________________________________________
>> lttng-dev mailing list
>> lttng-dev@lists.lttng.org
>> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>>
>

[-- Attachment #1.2: Type: text/html, Size: 2930 bytes --]

[-- Attachment #2: Type: text/plain, Size: 155 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: LTTng system call tracing on ARM

[Jan Glauber]

On Wed, Apr 10, 2013 at 08:37:18AM -0400, PLSTC wrote:
> Hey Jan,
> 
> I cannot speak on behalf of everyone here, but during our attempt to port
> LTTng to Android, we also noticed that the kernels we were using (3.0.x)
> were nowhere near the requirements for syscall tracepoints support. I
> believe such support was added on x86/64 way earlier (early 3.x) than on
> ARM, which is why it was included in LTTng's modules a while ago. Simply
> put, the ARM kernel is late.

OK, so for ARM kernel version 3.6 is the minimum unless the syscall tracepoint
support is backported.
 
> There are a few actuals ways to 'enable' syscall tracepoints support on
> early ARM kernels, but they all including a bit of kernel hacking/patching.
> I could send you some links if you're interested in that.

Yes, sure!

--Jan

> -PL
> On Apr 10, 2013 4:40 AM, "Jan Glauber" <jan.glauber@gmail.com> wrote:
> 
> > Hi,
> >
> > I want to use LTTng for system call tracing on ARM. Now lttng-modules seems
> > to support system call tracing on ARM already since
> > "8f4f80e LTTng Modules ARM syscall instrumentation".
> >
> > But I wonder how that worked since lttng-syscalls.c is only build under
> > CONFIG_HAVE_SYSCALL_TRACEPOINTS and that was added to ARM only with kernel
> > 3.6
> > (much after than the lttng-modules commit).
> >
> > Am I missing something? Is system call tracing working on ARM with the
> > upstream
> > LTTng version?
> >
> > thanks,
> > Jan
> >
> > _______________________________________________
> > lttng-dev mailing list
> > lttng-dev@lists.lttng.org
> > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
> >

Re: LTTng system call tracing on ARM

[Pierre-Luc St-Charles]

[-- Attachment #1.1: Type: text/plain, Size: 2486 bytes --]

Here are the links I stashed a while back about previous patching attempts
to add syscall tracepoints to the ARM kernel :

Most promising:
http://www.spinics.net/lists/arm-kernel/msg166419.html

Earlier draft(s):
http://lists.infradead.org/pipermail/linux-arm-kernel/2012-February/086974.html
https://lkml.org/lkml/2011/12/1/131
http://comments.gmane.org/gmane.linux.ports.arm.kernel/141933

I believe the first link shows the most refined patch there is out there,
but it might take some minor tinkering to apply it to a different kernel
version. I briefly tried to apply it to the 3.0.31 kernel, but it's a bit
out of my 'tinkering' range, and I never finished it.


Good luck!

-PL



On Wed, Apr 10, 2013 at 9:02 AM, Jan Glauber <jan.glauber@gmail.com> wrote:

> On Wed, Apr 10, 2013 at 08:37:18AM -0400, PLSTC wrote:
> > Hey Jan,
> >
> > I cannot speak on behalf of everyone here, but during our attempt to port
> > LTTng to Android, we also noticed that the kernels we were using (3.0.x)
> > were nowhere near the requirements for syscall tracepoints support. I
> > believe such support was added on x86/64 way earlier (early 3.x) than on
> > ARM, which is why it was included in LTTng's modules a while ago. Simply
> > put, the ARM kernel is late.
>
> OK, so for ARM kernel version 3.6 is the minimum unless the syscall
> tracepoint
> support is backported.
>
> > There are a few actuals ways to 'enable' syscall tracepoints support on
> > early ARM kernels, but they all including a bit of kernel
> hacking/patching.
> > I could send you some links if you're interested in that.
>
> Yes, sure!
>
> --Jan
>
> > -PL
> > On Apr 10, 2013 4:40 AM, "Jan Glauber" <jan.glauber@gmail.com> wrote:
> >
> > > Hi,
> > >
> > > I want to use LTTng for system call tracing on ARM. Now lttng-modules
> seems
> > > to support system call tracing on ARM already since
> > > "8f4f80e LTTng Modules ARM syscall instrumentation".
> > >
> > > But I wonder how that worked since lttng-syscalls.c is only build under
> > > CONFIG_HAVE_SYSCALL_TRACEPOINTS and that was added to ARM only with
> kernel
> > > 3.6
> > > (much after than the lttng-modules commit).
> > >
> > > Am I missing something? Is system call tracing working on ARM with the
> > > upstream
> > > LTTng version?
> > >
> > > thanks,
> > > Jan
> > >
> > > _______________________________________________
> > > lttng-dev mailing list
> > > lttng-dev@lists.lttng.org
> > > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
> > >
>

[-- Attachment #1.2: Type: text/html, Size: 4075 bytes --]

[-- Attachment #2: Type: text/plain, Size: 155 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: LTTng system call tracing on ARM

[Jan Glauber]

On Wed, Apr 10, 2013 at 09:38:33AM -0400, Pierre-Luc St-Charles wrote:
> Here are the links I stashed a while back about previous patching attempts
> to add syscall tracepoints to the ARM kernel :
> 
> Most promising:
> http://www.spinics.net/lists/arm-kernel/msg166419.html
> 
> Earlier draft(s):
> http://lists.infradead.org/pipermail/linux-arm-kernel/2012-February/086974.html
> https://lkml.org/lkml/2011/12/1/131
> http://comments.gmane.org/gmane.linux.ports.arm.kernel/141933
> 
> I believe the first link shows the most refined patch there is out there,
> but it might take some minor tinkering to apply it to a different kernel
> version. I briefly tried to apply it to the 3.0.31 kernel, but it's a bit
> out of my 'tinkering' range, and I never finished it.
> 
> 
> Good luck!

Thanks, fortunately I'm targeting 3.4. It wasn't too hard, got the kernel running
and ftrace already works.

Now I'm facing something hilarious, on loading the lttng-tracer module I get:

Error: could not insert module lttng-tracer.ko: Cannot allocate memory

[15912.259399] alloc_vmap_area: 2 callbacks suppressed
[15912.264953] vmap allocation for size 15998976 failed: use vmalloc=<size> to increase size.
[15912.273895] vmalloc: allocation failure: 15991653 bytes
[15912.279602] insmod: page allocation failure: order:0, mode:0xd0

Never saw vmalloc fail on loading a kernel module before. Hmmm.
Looking at the module reveals:
-rw-rw-r-- 1 jang jang 16036281 Apr 10 17:48 lttng-tracer.ko

Wow. 16 megs for a kernel module ?!? readelf says its all in .rodata.

Doing an objdump on that
Disassembly of section .rodata:

00000000 <sc_table>:
...
    1714:       00f0a788        rscseq  sl, r0, r8, lsl #15
    1718:       00f0a7a0        rscseq  sl, r0, r0, lsr #15
    171c:       00000004        andeq   r0, r0, r4
        ...
  f00054:       00f0a860        rscseq  sl, r0, r0, ror #16
  f00058:       00f0a878        rscseq  sl, r0, r8, ror r8
  f0005c:       00000002        andeq   r0, r0, r2

Now whats that? A big hole in the system call table?

Digging through the various defines and headers of lttng-modules I've found:

instrumentation/syscalls/headers/arm-32-syscalls-2.6.38_integers_override.h:
...
#define OVERRIDE_TABLE_32_sys_set_tls
TRACE_SYSCALL_TABLE(sys_set_tls, sys_set_tls, 0xf0005, 2)

Weird ARM apparently expanded the system call area above 0xf0000. I don't
understand why this override logic is needed but removing that define results
in a kernel module with a much more sane size.

LTTng system call tracing works with that change for me.

Any idea on how the override problem can be solved with the magic system calls?

--Jan

> -PL
> 
> 
> 
> On Wed, Apr 10, 2013 at 9:02 AM, Jan Glauber <jan.glauber@gmail.com> wrote:
> 
> > On Wed, Apr 10, 2013 at 08:37:18AM -0400, PLSTC wrote:
> > > Hey Jan,
> > >
> > > I cannot speak on behalf of everyone here, but during our attempt to port
> > > LTTng to Android, we also noticed that the kernels we were using (3.0.x)
> > > were nowhere near the requirements for syscall tracepoints support. I
> > > believe such support was added on x86/64 way earlier (early 3.x) than on
> > > ARM, which is why it was included in LTTng's modules a while ago. Simply
> > > put, the ARM kernel is late.
> >
> > OK, so for ARM kernel version 3.6 is the minimum unless the syscall
> > tracepoint
> > support is backported.
> >
> > > There are a few actuals ways to 'enable' syscall tracepoints support on
> > > early ARM kernels, but they all including a bit of kernel
> > hacking/patching.
> > > I could send you some links if you're interested in that.
> >
> > Yes, sure!
> >
> > --Jan
> >
> > > -PL
> > > On Apr 10, 2013 4:40 AM, "Jan Glauber" <jan.glauber@gmail.com> wrote:
> > >
> > > > Hi,
> > > >
> > > > I want to use LTTng for system call tracing on ARM. Now lttng-modules
> > seems
> > > > to support system call tracing on ARM already since
> > > > "8f4f80e LTTng Modules ARM syscall instrumentation".
> > > >
> > > > But I wonder how that worked since lttng-syscalls.c is only build under
> > > > CONFIG_HAVE_SYSCALL_TRACEPOINTS and that was added to ARM only with
> > kernel
> > > > 3.6
> > > > (much after than the lttng-modules commit).
> > > >
> > > > Am I missing something? Is system call tracing working on ARM with the
> > > > upstream
> > > > LTTng version?
> > > >
> > > > thanks,
> > > > Jan
> > > >
> > > > _______________________________________________
> > > > lttng-dev mailing list
> > > > lttng-dev@lists.lttng.org
> > > > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
> > > >
> >

Re: LTTng system call tracing on ARM

[Christian Babeux]

Hi Jan,

This seems related to bug #472 [1]. CCing Mathieu.

Christian

[1] - http://bugs.lttng.org/issues/472

On Wed, Apr 10, 2013 at 12:02 PM, Jan Glauber <jan.glauber@gmail.com> wrote:
> On Wed, Apr 10, 2013 at 09:38:33AM -0400, Pierre-Luc St-Charles wrote:
>> Here are the links I stashed a while back about previous patching attempts
>> to add syscall tracepoints to the ARM kernel :
>>
>> Most promising:
>> http://www.spinics.net/lists/arm-kernel/msg166419.html
>>
>> Earlier draft(s):
>> http://lists.infradead.org/pipermail/linux-arm-kernel/2012-February/086974.html
>> https://lkml.org/lkml/2011/12/1/131
>> http://comments.gmane.org/gmane.linux.ports.arm.kernel/141933
>>
>> I believe the first link shows the most refined patch there is out there,
>> but it might take some minor tinkering to apply it to a different kernel
>> version. I briefly tried to apply it to the 3.0.31 kernel, but it's a bit
>> out of my 'tinkering' range, and I never finished it.
>>
>>
>> Good luck!
>
> Thanks, fortunately I'm targeting 3.4. It wasn't too hard, got the kernel running
> and ftrace already works.
>
> Now I'm facing something hilarious, on loading the lttng-tracer module I get:
>
> Error: could not insert module lttng-tracer.ko: Cannot allocate memory
>
> [15912.259399] alloc_vmap_area: 2 callbacks suppressed
> [15912.264953] vmap allocation for size 15998976 failed: use vmalloc=<size> to increase size.
> [15912.273895] vmalloc: allocation failure: 15991653 bytes
> [15912.279602] insmod: page allocation failure: order:0, mode:0xd0
>
> Never saw vmalloc fail on loading a kernel module before. Hmmm.
> Looking at the module reveals:
> -rw-rw-r-- 1 jang jang 16036281 Apr 10 17:48 lttng-tracer.ko
>
> Wow. 16 megs for a kernel module ?!? readelf says its all in .rodata.
>
> Doing an objdump on that
> Disassembly of section .rodata:
>
> 00000000 <sc_table>:
> ...
>     1714:       00f0a788        rscseq  sl, r0, r8, lsl #15
>     1718:       00f0a7a0        rscseq  sl, r0, r0, lsr #15
>     171c:       00000004        andeq   r0, r0, r4
>         ...
>   f00054:       00f0a860        rscseq  sl, r0, r0, ror #16
>   f00058:       00f0a878        rscseq  sl, r0, r8, ror r8
>   f0005c:       00000002        andeq   r0, r0, r2
>
> Now whats that? A big hole in the system call table?
>
> Digging through the various defines and headers of lttng-modules I've found:
>
> instrumentation/syscalls/headers/arm-32-syscalls-2.6.38_integers_override.h:
> ...
> #define OVERRIDE_TABLE_32_sys_set_tls
> TRACE_SYSCALL_TABLE(sys_set_tls, sys_set_tls, 0xf0005, 2)
>
> Weird ARM apparently expanded the system call area above 0xf0000. I don't
> understand why this override logic is needed but removing that define results
> in a kernel module with a much more sane size.
>
> LTTng system call tracing works with that change for me.
>
> Any idea on how the override problem can be solved with the magic system calls?
>
> --Jan
>
>> -PL
>>
>>
>>
>> On Wed, Apr 10, 2013 at 9:02 AM, Jan Glauber <jan.glauber@gmail.com> wrote:
>>
>> > On Wed, Apr 10, 2013 at 08:37:18AM -0400, PLSTC wrote:
>> > > Hey Jan,
>> > >
>> > > I cannot speak on behalf of everyone here, but during our attempt to port
>> > > LTTng to Android, we also noticed that the kernels we were using (3.0.x)
>> > > were nowhere near the requirements for syscall tracepoints support. I
>> > > believe such support was added on x86/64 way earlier (early 3.x) than on
>> > > ARM, which is why it was included in LTTng's modules a while ago. Simply
>> > > put, the ARM kernel is late.
>> >
>> > OK, so for ARM kernel version 3.6 is the minimum unless the syscall
>> > tracepoint
>> > support is backported.
>> >
>> > > There are a few actuals ways to 'enable' syscall tracepoints support on
>> > > early ARM kernels, but they all including a bit of kernel
>> > hacking/patching.
>> > > I could send you some links if you're interested in that.
>> >
>> > Yes, sure!
>> >
>> > --Jan
>> >
>> > > -PL
>> > > On Apr 10, 2013 4:40 AM, "Jan Glauber" <jan.glauber@gmail.com> wrote:
>> > >
>> > > > Hi,
>> > > >
>> > > > I want to use LTTng for system call tracing on ARM. Now lttng-modules
>> > seems
>> > > > to support system call tracing on ARM already since
>> > > > "8f4f80e LTTng Modules ARM syscall instrumentation".
>> > > >
>> > > > But I wonder how that worked since lttng-syscalls.c is only build under
>> > > > CONFIG_HAVE_SYSCALL_TRACEPOINTS and that was added to ARM only with
>> > kernel
>> > > > 3.6
>> > > > (much after than the lttng-modules commit).
>> > > >
>> > > > Am I missing something? Is system call tracing working on ARM with the
>> > > > upstream
>> > > > LTTng version?
>> > > >
>> > > > thanks,
>> > > > Jan
>> > > >
>> > > > _______________________________________________
>> > > > lttng-dev mailing list
>> > > > lttng-dev@lists.lttng.org
>> > > > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>> > > >
>> >
>
> _______________________________________________
> lttng-dev mailing list
> lttng-dev@lists.lttng.org
> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: LTTng system call tracing on ARM

[Mathieu Desnoyers]

* Jan Glauber (jan.glauber@gmail.com) wrote:
> On Wed, Apr 10, 2013 at 09:38:33AM -0400, Pierre-Luc St-Charles wrote:
> > Here are the links I stashed a while back about previous patching attempts
> > to add syscall tracepoints to the ARM kernel :
> > 
> > Most promising:
> > http://www.spinics.net/lists/arm-kernel/msg166419.html
> > 
> > Earlier draft(s):
> > http://lists.infradead.org/pipermail/linux-arm-kernel/2012-February/086974.html
> > https://lkml.org/lkml/2011/12/1/131
> > http://comments.gmane.org/gmane.linux.ports.arm.kernel/141933
> > 
> > I believe the first link shows the most refined patch there is out there,
> > but it might take some minor tinkering to apply it to a different kernel
> > version. I briefly tried to apply it to the 3.0.31 kernel, but it's a bit
> > out of my 'tinkering' range, and I never finished it.
> > 
> > 
> > Good luck!
> 
> Thanks, fortunately I'm targeting 3.4. It wasn't too hard, got the kernel running
> and ftrace already works.
> 
> Now I'm facing something hilarious, on loading the lttng-tracer module I get:
> 
> Error: could not insert module lttng-tracer.ko: Cannot allocate memory
> 
> [15912.259399] alloc_vmap_area: 2 callbacks suppressed
> [15912.264953] vmap allocation for size 15998976 failed: use vmalloc=<size> to increase size.
> [15912.273895] vmalloc: allocation failure: 15991653 bytes
> [15912.279602] insmod: page allocation failure: order:0, mode:0xd0
> 
> Never saw vmalloc fail on loading a kernel module before. Hmmm.
> Looking at the module reveals:
> -rw-rw-r-- 1 jang jang 16036281 Apr 10 17:48 lttng-tracer.ko
> 
> Wow. 16 megs for a kernel module ?!? readelf says its all in .rodata.
> 
> Doing an objdump on that
> Disassembly of section .rodata:
> 
> 00000000 <sc_table>:
> ...
>     1714:       00f0a788        rscseq  sl, r0, r8, lsl #15
>     1718:       00f0a7a0        rscseq  sl, r0, r0, lsr #15
>     171c:       00000004        andeq   r0, r0, r4
>         ...
>   f00054:       00f0a860        rscseq  sl, r0, r0, ror #16
>   f00058:       00f0a878        rscseq  sl, r0, r8, ror r8
>   f0005c:       00000002        andeq   r0, r0, r2
> 
> Now whats that? A big hole in the system call table?
> 
> Digging through the various defines and headers of lttng-modules I've found:
> 
> instrumentation/syscalls/headers/arm-32-syscalls-2.6.38_integers_override.h:
> ...
> #define OVERRIDE_TABLE_32_sys_set_tls
> TRACE_SYSCALL_TABLE(sys_set_tls, sys_set_tls, 0xf0005, 2)
> 
> Weird ARM apparently expanded the system call area above 0xf0000. I don't
> understand why this override logic is needed but removing that define results
> in a kernel module with a much more sane size.
> 
> LTTng system call tracing works with that change for me.
> 
> Any idea on how the override problem can be solved with the magic system calls?

Thanks for the detailed report !!

Please try with this commit:

commit 11fa478f18241fedee86f7dc7820a91c629c9e7e
Author: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Date:   Wed Apr 10 12:14:38 2013 -0400

    Fix: remove ARM set_tls system call override
    
    We'll need to find a better way to instrument ARM-specific system calls
    located at a far offset from the standard systems calls. A 16MB
    lttng-modules kernel module is really not acceptable.
    
    Removing this instrumentation for now. sys_set_tls will appear as
    sys_unknown.
    
    Fixes #472
    
    CC: Ryan Kyser <Ryan.Kyser@jci.com>
    Ref: http://lists.lttng.org/pipermail/lttng-dev/2013-April/019990.html
    Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

diff --git a/instrumentation/syscalls/headers/arm-32-syscalls-2.6.38_integers_override.h b/instrumentation/syscalls/headers/arm-32-syscalls-2.6.38_integers_override.h
index 93b8674..895370f 100644
--- a/instrumentation/syscalls/headers/arm-32-syscalls-2.6.38_integers_override.h
+++ b/instrumentation/syscalls/headers/arm-32-syscalls-2.6.38_integers_override.h
@@ -2,7 +2,6 @@
 
 #define OVERRIDE_TABLE_32_sys_arm_fadvise64_64
 #define OVERRIDE_TABLE_32_sys_sync_file_range2
-#define OVERRIDE_TABLE_32_sys_set_tls
 
 #ifndef CREATE_SYSCALL_TABLE
 
@@ -38,18 +37,6 @@ SC_TRACE_EVENT(sys_sync_file_range2,
 	TP_printk()
 )
 
-SC_TRACE_EVENT(sys_set_tls,
-	TP_PROTO(unsigned int tid, unsigned long tls),
-	TP_ARGS(tid, tls),
-	TP_STRUCT__entry(
-		__field(unsigned int, tid)
-		__field_hex(unsigned int, tls)),
-	TP_fast_assign(
-		tp_assign(tid, tid)
-		tp_assign(tls, tls)),
-	TP_printk()
-)
-
 #else	/* CREATE_SYSCALL_TABLE */
 
 #define OVVERRIDE_TABLE_32_sys_mmap
@@ -59,9 +46,6 @@ TRACE_SYSCALL_TABLE(sys_mmap, sys_mmap, 90, 6)
 TRACE_SYSCALL_TABLE(sys_arm_fadvise64_64, sys_arm_fadvise64_64, 270, 4)
 #define OVERRIDE_TABLE_32_sys_sync_file_range2
 TRACE_SYSCALL_TABLE(sys_sync_file_range2, sys_sync_file_range2, 341, 4)
-#define OVERRIDE_TABLE_32_sys_set_tls
-TRACE_SYSCALL_TABLE(sys_set_tls, sys_set_tls, 0xf0005, 2)
-
 
 #endif /* CREATE_SYSCALL_TABLE */
 


> 
> --Jan
> 
> > -PL
> > 
> > 
> > 
> > On Wed, Apr 10, 2013 at 9:02 AM, Jan Glauber <jan.glauber@gmail.com> wrote:
> > 
> > > On Wed, Apr 10, 2013 at 08:37:18AM -0400, PLSTC wrote:
> > > > Hey Jan,
> > > >
> > > > I cannot speak on behalf of everyone here, but during our attempt to port
> > > > LTTng to Android, we also noticed that the kernels we were using (3.0.x)
> > > > were nowhere near the requirements for syscall tracepoints support. I
> > > > believe such support was added on x86/64 way earlier (early 3.x) than on
> > > > ARM, which is why it was included in LTTng's modules a while ago. Simply
> > > > put, the ARM kernel is late.
> > >
> > > OK, so for ARM kernel version 3.6 is the minimum unless the syscall
> > > tracepoint
> > > support is backported.
> > >
> > > > There are a few actuals ways to 'enable' syscall tracepoints support on
> > > > early ARM kernels, but they all including a bit of kernel
> > > hacking/patching.
> > > > I could send you some links if you're interested in that.
> > >
> > > Yes, sure!
> > >
> > > --Jan
> > >
> > > > -PL
> > > > On Apr 10, 2013 4:40 AM, "Jan Glauber" <jan.glauber@gmail.com> wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I want to use LTTng for system call tracing on ARM. Now lttng-modules
> > > seems
> > > > > to support system call tracing on ARM already since
> > > > > "8f4f80e LTTng Modules ARM syscall instrumentation".
> > > > >
> > > > > But I wonder how that worked since lttng-syscalls.c is only build under
> > > > > CONFIG_HAVE_SYSCALL_TRACEPOINTS and that was added to ARM only with
> > > kernel
> > > > > 3.6
> > > > > (much after than the lttng-modules commit).
> > > > >
> > > > > Am I missing something? Is system call tracing working on ARM with the
> > > > > upstream
> > > > > LTTng version?
> > > > >
> > > > > thanks,
> > > > > Jan
> > > > >
> > > > > _______________________________________________
> > > > > lttng-dev mailing list
> > > > > lttng-dev@lists.lttng.org
> > > > > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
> > > > >
> > >
> 
> _______________________________________________
> lttng-dev mailing list
> lttng-dev@lists.lttng.org
> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

Re: LTTng system call tracing on ARM

[Jan Glauber]

On Wed, Apr 10, 2013 at 12:20:12PM -0400, Mathieu Desnoyers wrote:
> 
> Thanks for the detailed report !!
> 
> Please try with this commit:

Yes, that patch works. But I'm still curious if you can explain what the
override was for ;-

--Jan
 
> commit 11fa478f18241fedee86f7dc7820a91c629c9e7e
> Author: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
> Date:   Wed Apr 10 12:14:38 2013 -0400
> 
>     Fix: remove ARM set_tls system call override
>     
>     We'll need to find a better way to instrument ARM-specific system calls
>     located at a far offset from the standard systems calls. A 16MB
>     lttng-modules kernel module is really not acceptable.
>     
>     Removing this instrumentation for now. sys_set_tls will appear as
>     sys_unknown.
>     
>     Fixes #472
>     
>     CC: Ryan Kyser <Ryan.Kyser@jci.com>
>     Ref: http://lists.lttng.org/pipermail/lttng-dev/2013-April/019990.html
>     Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
> 
> diff --git a/instrumentation/syscalls/headers/arm-32-syscalls-2.6.38_integers_override.h b/instrumentation/syscalls/headers/arm-32-syscalls-2.6.38_integers_override.h
> index 93b8674..895370f 100644
> --- a/instrumentation/syscalls/headers/arm-32-syscalls-2.6.38_integers_override.h
> +++ b/instrumentation/syscalls/headers/arm-32-syscalls-2.6.38_integers_override.h
> @@ -2,7 +2,6 @@
>  
>  #define OVERRIDE_TABLE_32_sys_arm_fadvise64_64
>  #define OVERRIDE_TABLE_32_sys_sync_file_range2
> -#define OVERRIDE_TABLE_32_sys_set_tls
>  
>  #ifndef CREATE_SYSCALL_TABLE
>  
> @@ -38,18 +37,6 @@ SC_TRACE_EVENT(sys_sync_file_range2,
>  	TP_printk()
>  )
>  
> -SC_TRACE_EVENT(sys_set_tls,
> -	TP_PROTO(unsigned int tid, unsigned long tls),
> -	TP_ARGS(tid, tls),
> -	TP_STRUCT__entry(
> -		__field(unsigned int, tid)
> -		__field_hex(unsigned int, tls)),
> -	TP_fast_assign(
> -		tp_assign(tid, tid)
> -		tp_assign(tls, tls)),
> -	TP_printk()
> -)
> -
>  #else	/* CREATE_SYSCALL_TABLE */
>  
>  #define OVVERRIDE_TABLE_32_sys_mmap
> @@ -59,9 +46,6 @@ TRACE_SYSCALL_TABLE(sys_mmap, sys_mmap, 90, 6)
>  TRACE_SYSCALL_TABLE(sys_arm_fadvise64_64, sys_arm_fadvise64_64, 270, 4)
>  #define OVERRIDE_TABLE_32_sys_sync_file_range2
>  TRACE_SYSCALL_TABLE(sys_sync_file_range2, sys_sync_file_range2, 341, 4)
> -#define OVERRIDE_TABLE_32_sys_set_tls
> -TRACE_SYSCALL_TABLE(sys_set_tls, sys_set_tls, 0xf0005, 2)
> -
>  
>  #endif /* CREATE_SYSCALL_TABLE */
>  
> 
> 
> > 
> > --Jan
> > 
> > > -PL
> > > 
> > > 
> > > 
> > > On Wed, Apr 10, 2013 at 9:02 AM, Jan Glauber <jan.glauber@gmail.com> wrote:
> > > 
> > > > On Wed, Apr 10, 2013 at 08:37:18AM -0400, PLSTC wrote:
> > > > > Hey Jan,
> > > > >
> > > > > I cannot speak on behalf of everyone here, but during our attempt to port
> > > > > LTTng to Android, we also noticed that the kernels we were using (3.0.x)
> > > > > were nowhere near the requirements for syscall tracepoints support. I
> > > > > believe such support was added on x86/64 way earlier (early 3.x) than on
> > > > > ARM, which is why it was included in LTTng's modules a while ago. Simply
> > > > > put, the ARM kernel is late.
> > > >
> > > > OK, so for ARM kernel version 3.6 is the minimum unless the syscall
> > > > tracepoint
> > > > support is backported.
> > > >
> > > > > There are a few actuals ways to 'enable' syscall tracepoints support on
> > > > > early ARM kernels, but they all including a bit of kernel
> > > > hacking/patching.
> > > > > I could send you some links if you're interested in that.
> > > >
> > > > Yes, sure!
> > > >
> > > > --Jan
> > > >
> > > > > -PL
> > > > > On Apr 10, 2013 4:40 AM, "Jan Glauber" <jan.glauber@gmail.com> wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I want to use LTTng for system call tracing on ARM. Now lttng-modules
> > > > seems
> > > > > > to support system call tracing on ARM already since
> > > > > > "8f4f80e LTTng Modules ARM syscall instrumentation".
> > > > > >
> > > > > > But I wonder how that worked since lttng-syscalls.c is only build under
> > > > > > CONFIG_HAVE_SYSCALL_TRACEPOINTS and that was added to ARM only with
> > > > kernel
> > > > > > 3.6
> > > > > > (much after than the lttng-modules commit).
> > > > > >
> > > > > > Am I missing something? Is system call tracing working on ARM with the
> > > > > > upstream
> > > > > > LTTng version?
> > > > > >
> > > > > > thanks,
> > > > > > Jan
> > > > > >
> > > > > > _______________________________________________
> > > > > > lttng-dev mailing list
> > > > > > lttng-dev@lists.lttng.org
> > > > > > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
> > > > > >
> > > >
> > 
> > _______________________________________________
> > lttng-dev mailing list
> > lttng-dev@lists.lttng.org
> > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
> 
> -- 
> Mathieu Desnoyers
> EfficiOS Inc.
> http://www.efficios.com

Re: LTTng system call tracing on ARM

[Mathieu Desnoyers]

* Jan Glauber (jan.glauber@gmail.com) wrote:
> On Wed, Apr 10, 2013 at 12:20:12PM -0400, Mathieu Desnoyers wrote:
> > 
> > Thanks for the detailed report !!
> > 
> > Please try with this commit:
> 
> Yes, that patch works. But I'm still curious if you can explain what the
> override was for ;-

The "override" allows defining how to serialize the information for
specific system calls. It can either replace the behavior of the
"semi-automated" description of the system call (brought into
lttng-modules by means of the system call extractor script), or just
provide the serialization description for a system call that was not
available to the system call extractor.

In this case, sys_set_tls was not available for automated extraction, so
the override entry has been added to it would not appear as a
"sys_unknown" system call, but would rather show "sys_set_tls" along
with the well-printed system call arguments.

You may want to have a look at:

lttng-modules/instrumentation/syscalls/README

for details.

If we want to handle the set_tls special-case (and same for other
ARM-specific syscalls), we'll probably want to do that with a
arm-specific table that will contain those system calls.

Thanks,

Mathieu


> 
> --Jan
>  
> > commit 11fa478f18241fedee86f7dc7820a91c629c9e7e
> > Author: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
> > Date:   Wed Apr 10 12:14:38 2013 -0400
> > 
> >     Fix: remove ARM set_tls system call override
> >     
> >     We'll need to find a better way to instrument ARM-specific system calls
> >     located at a far offset from the standard systems calls. A 16MB
> >     lttng-modules kernel module is really not acceptable.
> >     
> >     Removing this instrumentation for now. sys_set_tls will appear as
> >     sys_unknown.
> >     
> >     Fixes #472
> >     
> >     CC: Ryan Kyser <Ryan.Kyser@jci.com>
> >     Ref: http://lists.lttng.org/pipermail/lttng-dev/2013-April/019990.html
> >     Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
> > 
> > diff --git a/instrumentation/syscalls/headers/arm-32-syscalls-2.6.38_integers_override.h b/instrumentation/syscalls/headers/arm-32-syscalls-2.6.38_integers_override.h
> > index 93b8674..895370f 100644
> > --- a/instrumentation/syscalls/headers/arm-32-syscalls-2.6.38_integers_override.h
> > +++ b/instrumentation/syscalls/headers/arm-32-syscalls-2.6.38_integers_override.h
> > @@ -2,7 +2,6 @@
> >  
> >  #define OVERRIDE_TABLE_32_sys_arm_fadvise64_64
> >  #define OVERRIDE_TABLE_32_sys_sync_file_range2
> > -#define OVERRIDE_TABLE_32_sys_set_tls
> >  
> >  #ifndef CREATE_SYSCALL_TABLE
> >  
> > @@ -38,18 +37,6 @@ SC_TRACE_EVENT(sys_sync_file_range2,
> >  	TP_printk()
> >  )
> >  
> > -SC_TRACE_EVENT(sys_set_tls,
> > -	TP_PROTO(unsigned int tid, unsigned long tls),
> > -	TP_ARGS(tid, tls),
> > -	TP_STRUCT__entry(
> > -		__field(unsigned int, tid)
> > -		__field_hex(unsigned int, tls)),
> > -	TP_fast_assign(
> > -		tp_assign(tid, tid)
> > -		tp_assign(tls, tls)),
> > -	TP_printk()
> > -)
> > -
> >  #else	/* CREATE_SYSCALL_TABLE */
> >  
> >  #define OVVERRIDE_TABLE_32_sys_mmap
> > @@ -59,9 +46,6 @@ TRACE_SYSCALL_TABLE(sys_mmap, sys_mmap, 90, 6)
> >  TRACE_SYSCALL_TABLE(sys_arm_fadvise64_64, sys_arm_fadvise64_64, 270, 4)
> >  #define OVERRIDE_TABLE_32_sys_sync_file_range2
> >  TRACE_SYSCALL_TABLE(sys_sync_file_range2, sys_sync_file_range2, 341, 4)
> > -#define OVERRIDE_TABLE_32_sys_set_tls
> > -TRACE_SYSCALL_TABLE(sys_set_tls, sys_set_tls, 0xf0005, 2)
> > -
> >  
> >  #endif /* CREATE_SYSCALL_TABLE */
> >  
> > 
> > 
> > > 
> > > --Jan
> > > 
> > > > -PL
> > > > 
> > > > 
> > > > 
> > > > On Wed, Apr 10, 2013 at 9:02 AM, Jan Glauber <jan.glauber@gmail.com> wrote:
> > > > 
> > > > > On Wed, Apr 10, 2013 at 08:37:18AM -0400, PLSTC wrote:
> > > > > > Hey Jan,
> > > > > >
> > > > > > I cannot speak on behalf of everyone here, but during our attempt to port
> > > > > > LTTng to Android, we also noticed that the kernels we were using (3.0.x)
> > > > > > were nowhere near the requirements for syscall tracepoints support. I
> > > > > > believe such support was added on x86/64 way earlier (early 3.x) than on
> > > > > > ARM, which is why it was included in LTTng's modules a while ago. Simply
> > > > > > put, the ARM kernel is late.
> > > > >
> > > > > OK, so for ARM kernel version 3.6 is the minimum unless the syscall
> > > > > tracepoint
> > > > > support is backported.
> > > > >
> > > > > > There are a few actuals ways to 'enable' syscall tracepoints support on
> > > > > > early ARM kernels, but they all including a bit of kernel
> > > > > hacking/patching.
> > > > > > I could send you some links if you're interested in that.
> > > > >
> > > > > Yes, sure!
> > > > >
> > > > > --Jan
> > > > >
> > > > > > -PL
> > > > > > On Apr 10, 2013 4:40 AM, "Jan Glauber" <jan.glauber@gmail.com> wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > I want to use LTTng for system call tracing on ARM. Now lttng-modules
> > > > > seems
> > > > > > > to support system call tracing on ARM already since
> > > > > > > "8f4f80e LTTng Modules ARM syscall instrumentation".
> > > > > > >
> > > > > > > But I wonder how that worked since lttng-syscalls.c is only build under
> > > > > > > CONFIG_HAVE_SYSCALL_TRACEPOINTS and that was added to ARM only with
> > > > > kernel
> > > > > > > 3.6
> > > > > > > (much after than the lttng-modules commit).
> > > > > > >
> > > > > > > Am I missing something? Is system call tracing working on ARM with the
> > > > > > > upstream
> > > > > > > LTTng version?
> > > > > > >
> > > > > > > thanks,
> > > > > > > Jan
> > > > > > >
> > > > > > > _______________________________________________
> > > > > > > lttng-dev mailing list
> > > > > > > lttng-dev@lists.lttng.org
> > > > > > > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
> > > > > > >
> > > > >
> > > 
> > > _______________________________________________
> > > lttng-dev mailing list
> > > lttng-dev@lists.lttng.org
> > > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
> > 
> > -- 
> > Mathieu Desnoyers
> > EfficiOS Inc.
> > http://www.efficios.com

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

Re: LTTng system call tracing on ARM

[Jan Glauber]

On Wed, Apr 10, 2013 at 12:39:50PM -0400, Mathieu Desnoyers wrote:
> * Jan Glauber (jan.glauber@gmail.com) wrote:
> > On Wed, Apr 10, 2013 at 12:20:12PM -0400, Mathieu Desnoyers wrote:
> > > 
> > > Thanks for the detailed report !!
> > > 
> > > Please try with this commit:
> > 
> > Yes, that patch works. But I'm still curious if you can explain what the
> > override was for ;-
> 
> The "override" allows defining how to serialize the information for
> specific system calls. It can either replace the behavior of the
> "semi-automated" description of the system call (brought into
> lttng-modules by means of the system call extractor script), or just
> provide the serialization description for a system call that was not
> available to the system call extractor.
> 
> In this case, sys_set_tls was not available for automated extraction, so
> the override entry has been added to it would not appear as a
> "sys_unknown" system call, but would rather show "sys_set_tls" along
> with the well-printed system call arguments.

OK, thanks for explaining. So the version in instrumentation/syscalls/headers
doesn't mean much, its just the starting point from where the extractor
script picked up the descriptions.

So we could append recent system calls without generating new headers for
lets say 3.4.
 
> You may want to have a look at:
> 
> lttng-modules/instrumentation/syscalls/README
> 
> for details.
> 
> If we want to handle the set_tls special-case (and same for other
> ARM-specific syscalls), we'll probably want to do that with a
> arm-specific table that will contain those system calls.

Yes, a seperate table would solve the issue. I don't know if ARM is the
only arch doing such tricks to the system call table, so it might be worth
checkingt that.

Thanks,
Jan

> Thanks,
> 
> Mathieu
> 
> 
> > 
> > --Jan
> >  
> > > commit 11fa478f18241fedee86f7dc7820a91c629c9e7e
> > > Author: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
> > > Date:   Wed Apr 10 12:14:38 2013 -0400
> > > 
> > >     Fix: remove ARM set_tls system call override
> > >     
> > >     We'll need to find a better way to instrument ARM-specific system calls
> > >     located at a far offset from the standard systems calls. A 16MB
> > >     lttng-modules kernel module is really not acceptable.
> > >     
> > >     Removing this instrumentation for now. sys_set_tls will appear as
> > >     sys_unknown.
> > >     
> > >     Fixes #472
> > >     
> > >     CC: Ryan Kyser <Ryan.Kyser@jci.com>
> > >     Ref: http://lists.lttng.org/pipermail/lttng-dev/2013-April/019990.html
> > >     Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
> > > 
> > > diff --git a/instrumentation/syscalls/headers/arm-32-syscalls-2.6.38_integers_override.h b/instrumentation/syscalls/headers/arm-32-syscalls-2.6.38_integers_override.h
> > > index 93b8674..895370f 100644
> > > --- a/instrumentation/syscalls/headers/arm-32-syscalls-2.6.38_integers_override.h
> > > +++ b/instrumentation/syscalls/headers/arm-32-syscalls-2.6.38_integers_override.h
> > > @@ -2,7 +2,6 @@
> > >  
> > >  #define OVERRIDE_TABLE_32_sys_arm_fadvise64_64
> > >  #define OVERRIDE_TABLE_32_sys_sync_file_range2
> > > -#define OVERRIDE_TABLE_32_sys_set_tls
> > >  
> > >  #ifndef CREATE_SYSCALL_TABLE
> > >  
> > > @@ -38,18 +37,6 @@ SC_TRACE_EVENT(sys_sync_file_range2,
> > >  	TP_printk()
> > >  )
> > >  
> > > -SC_TRACE_EVENT(sys_set_tls,
> > > -	TP_PROTO(unsigned int tid, unsigned long tls),
> > > -	TP_ARGS(tid, tls),
> > > -	TP_STRUCT__entry(
> > > -		__field(unsigned int, tid)
> > > -		__field_hex(unsigned int, tls)),
> > > -	TP_fast_assign(
> > > -		tp_assign(tid, tid)
> > > -		tp_assign(tls, tls)),
> > > -	TP_printk()
> > > -)
> > > -
> > >  #else	/* CREATE_SYSCALL_TABLE */
> > >  
> > >  #define OVVERRIDE_TABLE_32_sys_mmap
> > > @@ -59,9 +46,6 @@ TRACE_SYSCALL_TABLE(sys_mmap, sys_mmap, 90, 6)
> > >  TRACE_SYSCALL_TABLE(sys_arm_fadvise64_64, sys_arm_fadvise64_64, 270, 4)
> > >  #define OVERRIDE_TABLE_32_sys_sync_file_range2
> > >  TRACE_SYSCALL_TABLE(sys_sync_file_range2, sys_sync_file_range2, 341, 4)
> > > -#define OVERRIDE_TABLE_32_sys_set_tls
> > > -TRACE_SYSCALL_TABLE(sys_set_tls, sys_set_tls, 0xf0005, 2)
> > > -
> > >  
> > >  #endif /* CREATE_SYSCALL_TABLE */
> > >  
> > > 
> > > 
> > > > 
> > > > --Jan
> > > > 
> > > > > -PL
> > > > > 
> > > > > 
> > > > > 
> > > > > On Wed, Apr 10, 2013 at 9:02 AM, Jan Glauber <jan.glauber@gmail.com> wrote:
> > > > > 
> > > > > > On Wed, Apr 10, 2013 at 08:37:18AM -0400, PLSTC wrote:
> > > > > > > Hey Jan,
> > > > > > >
> > > > > > > I cannot speak on behalf of everyone here, but during our attempt to port
> > > > > > > LTTng to Android, we also noticed that the kernels we were using (3.0.x)
> > > > > > > were nowhere near the requirements for syscall tracepoints support. I
> > > > > > > believe such support was added on x86/64 way earlier (early 3.x) than on
> > > > > > > ARM, which is why it was included in LTTng's modules a while ago. Simply
> > > > > > > put, the ARM kernel is late.
> > > > > >
> > > > > > OK, so for ARM kernel version 3.6 is the minimum unless the syscall
> > > > > > tracepoint
> > > > > > support is backported.
> > > > > >
> > > > > > > There are a few actuals ways to 'enable' syscall tracepoints support on
> > > > > > > early ARM kernels, but they all including a bit of kernel
> > > > > > hacking/patching.
> > > > > > > I could send you some links if you're interested in that.
> > > > > >
> > > > > > Yes, sure!
> > > > > >
> > > > > > --Jan
> > > > > >
> > > > > > > -PL
> > > > > > > On Apr 10, 2013 4:40 AM, "Jan Glauber" <jan.glauber@gmail.com> wrote:
> > > > > > >
> > > > > > > > Hi,
> > > > > > > >
> > > > > > > > I want to use LTTng for system call tracing on ARM. Now lttng-modules
> > > > > > seems
> > > > > > > > to support system call tracing on ARM already since
> > > > > > > > "8f4f80e LTTng Modules ARM syscall instrumentation".
> > > > > > > >
> > > > > > > > But I wonder how that worked since lttng-syscalls.c is only build under
> > > > > > > > CONFIG_HAVE_SYSCALL_TRACEPOINTS and that was added to ARM only with
> > > > > > kernel
> > > > > > > > 3.6
> > > > > > > > (much after than the lttng-modules commit).
> > > > > > > >
> > > > > > > > Am I missing something? Is system call tracing working on ARM with the
> > > > > > > > upstream
> > > > > > > > LTTng version?
> > > > > > > >
> > > > > > > > thanks,
> > > > > > > > Jan
> > > > > > > >
> > > > > > > > _______________________________________________
> > > > > > > > lttng-dev mailing list
> > > > > > > > lttng-dev@lists.lttng.org
> > > > > > > > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
> > > > > > > >
> > > > > >
> > > > 
> > > > _______________________________________________
> > > > lttng-dev mailing list
> > > > lttng-dev@lists.lttng.org
> > > > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
> > > 
> > > -- 
> > > Mathieu Desnoyers
> > > EfficiOS Inc.
> > > http://www.efficios.com
> 
> -- 
> Mathieu Desnoyers
> EfficiOS Inc.
> http://www.efficios.com

Re: LTTng system call tracing on ARM

[Mathieu Desnoyers]

* Jan Glauber (jan.glauber@gmail.com) wrote:
> On Wed, Apr 10, 2013 at 12:39:50PM -0400, Mathieu Desnoyers wrote:
> > * Jan Glauber (jan.glauber@gmail.com) wrote:
> > > On Wed, Apr 10, 2013 at 12:20:12PM -0400, Mathieu Desnoyers wrote:
> > > > 
> > > > Thanks for the detailed report !!
> > > > 
> > > > Please try with this commit:
> > > 
> > > Yes, that patch works. But I'm still curious if you can explain what the
> > > override was for ;-
> > 
> > The "override" allows defining how to serialize the information for
> > specific system calls. It can either replace the behavior of the
> > "semi-automated" description of the system call (brought into
> > lttng-modules by means of the system call extractor script), or just
> > provide the serialization description for a system call that was not
> > available to the system call extractor.
> > 
> > In this case, sys_set_tls was not available for automated extraction, so
> > the override entry has been added to it would not appear as a
> > "sys_unknown" system call, but would rather show "sys_set_tls" along
> > with the well-printed system call arguments.
> 
> OK, thanks for explaining. So the version in instrumentation/syscalls/headers
> doesn't mean much, its just the starting point from where the extractor
> script picked up the descriptions.
> 
> So we could append recent system calls without generating new headers for
> lets say 3.4.

Yes, this is one option.

The other option, if we want to upgrade a system call table from e.g.
2.6.38 to 3.4, is to just replace the automatically generated header by
headers generated by extracting system calls from a 3.4 kernel. This
will keep the overrides in places, because they are located into a
separate override file.

Therefore, all work done by hand in addition to the automatically
generated file does not get wasted between updates.

>  
> > You may want to have a look at:
> > 
> > lttng-modules/instrumentation/syscalls/README
> > 
> > for details.
> > 
> > If we want to handle the set_tls special-case (and same for other
> > ARM-specific syscalls), we'll probably want to do that with a
> > arm-specific table that will contain those system calls.
> 
> Yes, a seperate table would solve the issue. I don't know if ARM is the
> only arch doing such tricks to the system call table, so it might be worth
> checkingt that.

Indeed. This would probably be 2.3 material. I'm happy with showing
set_tls as sys_unknown for the moment.

Thanks!

Mathieu

> 
> Thanks,
> Jan
> 
> > Thanks,
> > 
> > Mathieu
> > 
> > 
> > > 
> > > --Jan
> > >  
> > > > commit 11fa478f18241fedee86f7dc7820a91c629c9e7e
> > > > Author: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
> > > > Date:   Wed Apr 10 12:14:38 2013 -0400
> > > > 
> > > >     Fix: remove ARM set_tls system call override
> > > >     
> > > >     We'll need to find a better way to instrument ARM-specific system calls
> > > >     located at a far offset from the standard systems calls. A 16MB
> > > >     lttng-modules kernel module is really not acceptable.
> > > >     
> > > >     Removing this instrumentation for now. sys_set_tls will appear as
> > > >     sys_unknown.
> > > >     
> > > >     Fixes #472
> > > >     
> > > >     CC: Ryan Kyser <Ryan.Kyser@jci.com>
> > > >     Ref: http://lists.lttng.org/pipermail/lttng-dev/2013-April/019990.html
> > > >     Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
> > > > 
> > > > diff --git a/instrumentation/syscalls/headers/arm-32-syscalls-2.6.38_integers_override.h b/instrumentation/syscalls/headers/arm-32-syscalls-2.6.38_integers_override.h
> > > > index 93b8674..895370f 100644
> > > > --- a/instrumentation/syscalls/headers/arm-32-syscalls-2.6.38_integers_override.h
> > > > +++ b/instrumentation/syscalls/headers/arm-32-syscalls-2.6.38_integers_override.h
> > > > @@ -2,7 +2,6 @@
> > > >  
> > > >  #define OVERRIDE_TABLE_32_sys_arm_fadvise64_64
> > > >  #define OVERRIDE_TABLE_32_sys_sync_file_range2
> > > > -#define OVERRIDE_TABLE_32_sys_set_tls
> > > >  
> > > >  #ifndef CREATE_SYSCALL_TABLE
> > > >  
> > > > @@ -38,18 +37,6 @@ SC_TRACE_EVENT(sys_sync_file_range2,
> > > >  	TP_printk()
> > > >  )
> > > >  
> > > > -SC_TRACE_EVENT(sys_set_tls,
> > > > -	TP_PROTO(unsigned int tid, unsigned long tls),
> > > > -	TP_ARGS(tid, tls),
> > > > -	TP_STRUCT__entry(
> > > > -		__field(unsigned int, tid)
> > > > -		__field_hex(unsigned int, tls)),
> > > > -	TP_fast_assign(
> > > > -		tp_assign(tid, tid)
> > > > -		tp_assign(tls, tls)),
> > > > -	TP_printk()
> > > > -)
> > > > -
> > > >  #else	/* CREATE_SYSCALL_TABLE */
> > > >  
> > > >  #define OVVERRIDE_TABLE_32_sys_mmap
> > > > @@ -59,9 +46,6 @@ TRACE_SYSCALL_TABLE(sys_mmap, sys_mmap, 90, 6)
> > > >  TRACE_SYSCALL_TABLE(sys_arm_fadvise64_64, sys_arm_fadvise64_64, 270, 4)
> > > >  #define OVERRIDE_TABLE_32_sys_sync_file_range2
> > > >  TRACE_SYSCALL_TABLE(sys_sync_file_range2, sys_sync_file_range2, 341, 4)
> > > > -#define OVERRIDE_TABLE_32_sys_set_tls
> > > > -TRACE_SYSCALL_TABLE(sys_set_tls, sys_set_tls, 0xf0005, 2)
> > > > -
> > > >  
> > > >  #endif /* CREATE_SYSCALL_TABLE */
> > > >  
> > > > 
> > > > 
> > > > > 
> > > > > --Jan
> > > > > 
> > > > > > -PL
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > On Wed, Apr 10, 2013 at 9:02 AM, Jan Glauber <jan.glauber@gmail.com> wrote:
> > > > > > 
> > > > > > > On Wed, Apr 10, 2013 at 08:37:18AM -0400, PLSTC wrote:
> > > > > > > > Hey Jan,
> > > > > > > >
> > > > > > > > I cannot speak on behalf of everyone here, but during our attempt to port
> > > > > > > > LTTng to Android, we also noticed that the kernels we were using (3.0.x)
> > > > > > > > were nowhere near the requirements for syscall tracepoints support. I
> > > > > > > > believe such support was added on x86/64 way earlier (early 3.x) than on
> > > > > > > > ARM, which is why it was included in LTTng's modules a while ago. Simply
> > > > > > > > put, the ARM kernel is late.
> > > > > > >
> > > > > > > OK, so for ARM kernel version 3.6 is the minimum unless the syscall
> > > > > > > tracepoint
> > > > > > > support is backported.
> > > > > > >
> > > > > > > > There are a few actuals ways to 'enable' syscall tracepoints support on
> > > > > > > > early ARM kernels, but they all including a bit of kernel
> > > > > > > hacking/patching.
> > > > > > > > I could send you some links if you're interested in that.
> > > > > > >
> > > > > > > Yes, sure!
> > > > > > >
> > > > > > > --Jan
> > > > > > >
> > > > > > > > -PL
> > > > > > > > On Apr 10, 2013 4:40 AM, "Jan Glauber" <jan.glauber@gmail.com> wrote:
> > > > > > > >
> > > > > > > > > Hi,
> > > > > > > > >
> > > > > > > > > I want to use LTTng for system call tracing on ARM. Now lttng-modules
> > > > > > > seems
> > > > > > > > > to support system call tracing on ARM already since
> > > > > > > > > "8f4f80e LTTng Modules ARM syscall instrumentation".
> > > > > > > > >
> > > > > > > > > But I wonder how that worked since lttng-syscalls.c is only build under
> > > > > > > > > CONFIG_HAVE_SYSCALL_TRACEPOINTS and that was added to ARM only with
> > > > > > > kernel
> > > > > > > > > 3.6
> > > > > > > > > (much after than the lttng-modules commit).
> > > > > > > > >
> > > > > > > > > Am I missing something? Is system call tracing working on ARM with the
> > > > > > > > > upstream
> > > > > > > > > LTTng version?
> > > > > > > > >
> > > > > > > > > thanks,
> > > > > > > > > Jan
> > > > > > > > >
> > > > > > > > > _______________________________________________
> > > > > > > > > lttng-dev mailing list
> > > > > > > > > lttng-dev@lists.lttng.org
> > > > > > > > > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
> > > > > > > > >
> > > > > > >
> > > > > 
> > > > > _______________________________________________
> > > > > lttng-dev mailing list
> > > > > lttng-dev@lists.lttng.org
> > > > > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
> > > > 
> > > > -- 
> > > > Mathieu Desnoyers
> > > > EfficiOS Inc.
> > > > http://www.efficios.com
> > 
> > -- 
> > Mathieu Desnoyers
> > EfficiOS Inc.
> > http://www.efficios.com

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

LTTng system call tracing on ARM

[Jan Glauber]

Hi,

I want to use LTTng for system call tracing on ARM. Now lttng-modules seems
to support system call tracing on ARM already since
"8f4f80e LTTng Modules ARM syscall instrumentation".

But I wonder how that worked since lttng-syscalls.c is only build under
CONFIG_HAVE_SYSCALL_TRACEPOINTS and that was added to ARM only with kernel 3.6
(much after than the lttng-modules commit).

Am I missing something? Is system call tracing working on ARM with the upstream
LTTng version?

thanks,
Jan