[PATCH v2] tracing/x86: Update syscall trace events to handle new x86 syscall func names

[PATCH v2] tracing/x86: Update syscall trace events to handle new x86 syscall func names

[Steven Rostedt]

From: "Steven Rostedt (VMware)" <rostedt@goodmis.org>

Arnaldo noticed that the latest kernel is missing the syscall event system
directory in x86. I bisected it down to d5a00528b58c ("syscalls/core,
syscalls/x86: Rename struct pt_regs-based sys_*() to __x64_sys_*()").

The system call trace events are special, as there is only one trace event
for all system calls (the raw_syscalls). But a macro that wraps the system
calls creates meta data for them that copies the name to find the system
call that maps to the system call table (the number). At boot up, it does a
kallsyms lookup for this mapping. If it does not find a function, then that
system call is ignored.

Because the x86 system calls had "__x64_" appended to the "sys" for the
names, they do not match the default compare algorithm. As this was a
problem for power pc, the algorithm can be overwritten by the architecture.
The solution is to have x86 have its own algorithm to do the compare and
this brings back the system call trace events.

Reported-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Fixes: d5a00528b58c ("syscalls/core, syscalls/x86: Rename struct pt_regs-based sys_*() to __x64_sys_*()")
Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
---
Changes since v1:

  Added back original compare to not miss 32bit kernel syscalls

 arch/x86/include/asm/ftrace.h | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/arch/x86/include/asm/ftrace.h b/arch/x86/include/asm/ftrace.h
index 09ad88572746..975b8c73bd50 100644
--- a/arch/x86/include/asm/ftrace.h
+++ b/arch/x86/include/asm/ftrace.h
@@ -31,6 +31,16 @@ static inline unsigned long ftrace_call_adjust(unsigned long addr)
 	return addr;
 }
 
+#define ARCH_HAS_SYSCALL_MATCH_SYM_NAME
+static inline bool arch_syscall_match_sym_name(const char *sym, const char *name)
+{
+	/*
+	 * Compare the symbol name with the system call name. Skip the
+	 * "__x64_sys" or simple "sys" prefix.
+	 */
+	return !strcmp(sym + 9, name + 3) || !strcmp(sym + 3, name + 3);
+}
+
 #ifdef CONFIG_DYNAMIC_FTRACE
 
 struct dyn_arch_ftrace {
-- 
2.13.6

Re: [PATCH v2] tracing/x86: Update syscall trace events to handle new x86 syscall func names

[Dominik Brodowski]

>   Added back original compare to not miss 32bit kernel syscalls

s/32bit/32bit and 0-parameter syscalls.

Looks good otherwise.

Thanks,
	Dominik

Re: [PATCH v2] tracing/x86: Update syscall trace events to handle new x86 syscall func names

[Arnaldo Carvalho de Melo]

Em Tue, Apr 17, 2018 at 07:22:36PM +0200, Dominik Brodowski escreveu:
> >   Added back original compare to not miss 32bit kernel syscalls
> 
> s/32bit/32bit and 0-parameter syscalls.

So this should have covered 0-parameter syscalls, ok, I'm double
checking that the last patch is the one I have running... Because
0-parameter syscalls are not working for me, i.e. no
syscalls:sys_enter_getppid, for instance.

- Arnaldo
 
> Looks good otherwise.
> 
> Thanks,
> 	Dominik

Re: [PATCH v2] tracing/x86: Update syscall trace events to handle new x86 syscall func names

[Arnaldo Carvalho de Melo]

Em Tue, Apr 17, 2018 at 03:04:30PM -0300, Arnaldo Carvalho de Melo escreveu:
> Em Tue, Apr 17, 2018 at 07:22:36PM +0200, Dominik Brodowski escreveu:
> > >   Added back original compare to not miss 32bit kernel syscalls
> > 
> > s/32bit/32bit and 0-parameter syscalls.
> 
> So this should have covered 0-parameter syscalls, ok, I'm double
> checking that the last patch is the one I have running... Because
> 0-parameter syscalls are not working for me, i.e. no
> syscalls:sys_enter_getppid, for instance.

Yeah, failing:

[root@jouet ~]# strace -e openat -e file perf test -F -v "mmap interface" |& grep syscalls
openat(AT_FDCWD, "/sys/kernel/debug/tracing/events/syscalls/sys_enter_getsid/format", O_RDONLY) = 3
openat(AT_FDCWD, "/sys/kernel/debug/tracing/events/syscalls/sys_enter_getppid/format", O_RDONLY) = -1 ENOENT (No such file or directory)
[root@jouet ~]# 

[root@jouet ~]# ls -la /sys/kernel/debug/tracing/events/syscalls/sys_enter_getppid/format
ls: cannot access '/sys/kernel/debug/tracing/events/syscalls/sys_enter_getppid/format': No such file or directory
[root@jouet ~]#

With:

+#define ARCH_HAS_SYSCALL_MATCH_SYM_NAME
+static inline bool arch_syscall_match_sym_name(const char *sym, const char *name)
+{
+       /*
+        * Compare the symbol name with the system call name. Skip the
+        * "__x64_sys" prefix.
+        */
+       return !strcmp(sym + 9, name + 3) || !strcmp(sym + 3, name + 3);
+}

in place.


- Arnaldo

Re: [PATCH v2] tracing/x86: Update syscall trace events to handle new x86 syscall func names

[Steven Rostedt]

On Tue, 17 Apr 2018 15:13:04 -0300
Arnaldo Carvalho de Melo <acme@kernel.org> wrote:

> Em Tue, Apr 17, 2018 at 03:04:30PM -0300, Arnaldo Carvalho de Melo escreveu:
> > Em Tue, Apr 17, 2018 at 07:22:36PM +0200, Dominik Brodowski escreveu:  
> > > >   Added back original compare to not miss 32bit kernel syscalls  
> > > 
> > > s/32bit/32bit and 0-parameter syscalls.  
> > 
> > So this should have covered 0-parameter syscalls, ok, I'm double
> > checking that the last patch is the one I have running... Because
> > 0-parameter syscalls are not working for me, i.e. no
> > syscalls:sys_enter_getppid, for instance.  
> 
> Yeah, failing:
> 
> [root@jouet ~]# strace -e openat -e file perf test -F -v "mmap interface" |& grep syscalls
> openat(AT_FDCWD, "/sys/kernel/debug/tracing/events/syscalls/sys_enter_getsid/format", O_RDONLY) = 3
> openat(AT_FDCWD, "/sys/kernel/debug/tracing/events/syscalls/sys_enter_getppid/format", O_RDONLY) = -1 ENOENT (No such file or directory)
> [root@jouet ~]# 
> 
> [root@jouet ~]# ls -la /sys/kernel/debug/tracing/events/syscalls/sys_enter_getppid/format
> ls: cannot access '/sys/kernel/debug/tracing/events/syscalls/sys_enter_getppid/format': No such file or directory
> [root@jouet ~]#
> 
> With:
> 
> +#define ARCH_HAS_SYSCALL_MATCH_SYM_NAME
> +static inline bool arch_syscall_match_sym_name(const char *sym, const char *name)
> +{
> +       /*
> +        * Compare the symbol name with the system call name. Skip the
> +        * "__x64_sys" prefix.
> +        */
> +       return !strcmp(sym + 9, name + 3) || !strcmp(sym + 3, name + 3);
> +}
> 
> in place.
>

Yeah, I did a test too, and found that I'm missing sys_clock_nanosleep.
I'm looking into it.

-- Steve

Re: [PATCH v2] tracing/x86: Update syscall trace events to handle new x86 syscall func names

[Steven Rostedt]

On Tue, 17 Apr 2018 15:13:04 -0300
Arnaldo Carvalho de Melo <acme@kernel.org> wrote:

> Em Tue, Apr 17, 2018 at 03:04:30PM -0300, Arnaldo Carvalho de Melo escreveu:
> > Em Tue, Apr 17, 2018 at 07:22:36PM +0200, Dominik Brodowski escreveu:  
> > > >   Added back original compare to not miss 32bit kernel syscalls  
> > > 
> > > s/32bit/32bit and 0-parameter syscalls.  
> > 
> > So this should have covered 0-parameter syscalls, ok, I'm double
> > checking that the last patch is the one I have running... Because
> > 0-parameter syscalls are not working for me, i.e. no
> > syscalls:sys_enter_getppid, for instance.  
> 
> Yeah, failing:
> 
> [root@jouet ~]# strace -e openat -e file perf test -F -v "mmap interface" |& grep syscalls
> openat(AT_FDCWD, "/sys/kernel/debug/tracing/events/syscalls/sys_enter_getsid/format", O_RDONLY) = 3
> openat(AT_FDCWD, "/sys/kernel/debug/tracing/events/syscalls/sys_enter_getppid/format", O_RDONLY) = -1 ENOENT (No such file or directory)
> [root@jouet ~]# 
> 
> [root@jouet ~]# ls -la /sys/kernel/debug/tracing/events/syscalls/sys_enter_getppid/format
> ls: cannot access '/sys/kernel/debug/tracing/events/syscalls/sys_enter_getppid/format': No such file or directory
> [root@jouet ~]#
> 
> With:
> 
> +#define ARCH_HAS_SYSCALL_MATCH_SYM_NAME
> +static inline bool arch_syscall_match_sym_name(const char *sym, const char *name)
> +{
> +       /*
> +        * Compare the symbol name with the system call name. Skip the
> +        * "__x64_sys" prefix.
> +        */
> +       return !strcmp(sym + 9, name + 3) || !strcmp(sym + 3, name + 3);
> +}
> 
> in place.

It doesn't have to do with the number of parameters, not everything has
"__x64" on it.

Try this patch:

-- Steve

diff --git a/arch/x86/include/asm/ftrace.h b/arch/x86/include/asm/ftrace.h
index 09ad88572746..15582648887f 100644
--- a/arch/x86/include/asm/ftrace.h
+++ b/arch/x86/include/asm/ftrace.h
@@ -31,6 +31,19 @@ static inline unsigned long ftrace_call_adjust(unsigned long addr)
 	return addr;
 }
 
+#define ARCH_HAS_SYSCALL_MATCH_SYM_NAME
+static inline bool arch_syscall_match_sym_name(const char *sym, const char *name)
+{
+	/*
+	 * Compare the symbol name with the system call name. Skip the
+	 * "__x64_sys", "__ia32_sys" or simple "sys" prefix.
+	 */
+	return !strcmp(sym + 3, name + 3) ||
+		(!strncmp(sym, "__x64_", 6) && !strcmp(sym + 9, name + 3)) ||
+		(!strncmp(sym, "__ia32_", 7) && !strcmp(sym + 10, name + 3));
+
+}
+
 #ifdef CONFIG_DYNAMIC_FTRACE
 
 struct dyn_arch_ftrace {

Re: [PATCH v2] tracing/x86: Update syscall trace events to handle new x86 syscall func names

[Arnaldo Carvalho de Melo]

Em Tue, Apr 17, 2018 at 05:41:28PM -0400, Steven Rostedt escreveu:
> On Tue, 17 Apr 2018 15:13:04 -0300 Arnaldo Carvalho de Melo <acme@kernel.org> wrote:
> > Yeah, failing:

> > [root@jouet ~]# strace -e openat -e file perf test -F -v "mmap interface" |& grep syscalls
> > openat(AT_FDCWD, "/sys/kernel/debug/tracing/events/syscalls/sys_enter_getsid/format", O_RDONLY) = 3
> > openat(AT_FDCWD, "/sys/kernel/debug/tracing/events/syscalls/sys_enter_getppid/format", O_RDONLY) = -1 ENOENT (No such file or directory)
 
> It doesn't have to do with the number of parameters, not everything
> has "__x64" on it.
 
> Try this patch:

Trying...
 
> diff --git a/arch/x86/include/asm/ftrace.h b/arch/x86/include/asm/ftrace.h
> index 09ad88572746..15582648887f 100644
> --- a/arch/x86/include/asm/ftrace.h
> +++ b/arch/x86/include/asm/ftrace.h
> @@ -31,6 +31,19 @@ static inline unsigned long ftrace_call_adjust(unsigned long addr)
>  	return addr;
>  }
>  
> +#define ARCH_HAS_SYSCALL_MATCH_SYM_NAME
> +static inline bool arch_syscall_match_sym_name(const char *sym, const char *name)
> +{
> +	/*
> +	 * Compare the symbol name with the system call name. Skip the
> +	 * "__x64_sys", "__ia32_sys" or simple "sys" prefix.
> +	 */
> +	return !strcmp(sym + 3, name + 3) ||
> +		(!strncmp(sym, "__x64_", 6) && !strcmp(sym + 9, name + 3)) ||
> +		(!strncmp(sym, "__ia32_", 7) && !strcmp(sym + 10, name + 3));
> +
> +}
> +
>  #ifdef CONFIG_DYNAMIC_FTRACE
>  
>  struct dyn_arch_ftrace {

Re: [PATCH v2] tracing/x86: Update syscall trace events to handle new x86 syscall func names

[Steven Rostedt]

On Wed, 18 Apr 2018 09:53:22 -0300
Arnaldo Carvalho de Melo <acme@kernel.org> wrote:

> Em Tue, Apr 17, 2018 at 05:41:28PM -0400, Steven Rostedt escreveu:
> > On Tue, 17 Apr 2018 15:13:04 -0300 Arnaldo Carvalho de Melo <acme@kernel.org> wrote:  
> > > Yeah, failing:  
> 
> > > [root@jouet ~]# strace -e openat -e file perf test -F -v "mmap interface" |& grep syscalls
> > > openat(AT_FDCWD, "/sys/kernel/debug/tracing/events/syscalls/sys_enter_getsid/format", O_RDONLY) = 3
> > > openat(AT_FDCWD, "/sys/kernel/debug/tracing/events/syscalls/sys_enter_getppid/format", O_RDONLY) = -1 ENOENT (No such file or directory)  
>  
> > It doesn't have to do with the number of parameters, not everything
> > has "__x64" on it.  
>  
> > Try this patch:  
> 
> Trying...

You're keeping me in suspense!

-- Steve

Re: [PATCH v2] tracing/x86: Update syscall trace events to handle new x86 syscall func names

[Arnaldo Carvalho de Melo]

Em Wed, Apr 18, 2018 at 10:36:06AM -0400, Steven Rostedt escreveu:
> On Wed, 18 Apr 2018 09:53:22 -0300
> Arnaldo Carvalho de Melo <acme@kernel.org> wrote:
> > Em Tue, Apr 17, 2018 at 05:41:28PM -0400, Steven Rostedt escreveu:
> > > On Tue, 17 Apr 2018 15:13:04 -0300 Arnaldo Carvalho de Melo <acme@kernel.org> wrote:  
> > > > Yeah, failing:  

> > > > [root@jouet ~]# strace -e openat -e file perf test -F -v "mmap interface" |& grep syscalls
> > > > openat(AT_FDCWD, "/sys/kernel/debug/tracing/events/syscalls/sys_enter_getsid/format", O_RDONLY) = 3
> > > > openat(AT_FDCWD, "/sys/kernel/debug/tracing/events/syscalls/sys_enter_getppid/format", O_RDONLY) = -1 ENOENT (No such file or directory)  
> >  
> > > It doesn't have to do with the number of parameters, not everything
> > > has "__x64" on it.  

> > > Try this patch:  

> > Trying...
 
> You're keeping me in suspense!

I switched locations, had trouble reconnecting, those tests are ok now,
there is just one case left, related to the syscall routines renames,
but not related to the syscalls:sys_{enter,exit}_NAME tracepoints:

[root@jouet ~]# perf test
 1: vmlinux symtab matches kallsyms                       : Ok
 2: Detect openat syscall event                           : Ok
 3: Detect openat syscall event on all cpus               : Ok
 4: Read samples using the mmap interface                 : Ok
 5: Test data source output                               : Ok
 6: Parse event definition strings                        : Ok
 7: Simple expression parser                              : Ok
 8: PERF_RECORD_* events & perf_sample fields             : Ok
 9: Parse perf pmu format                                 : Ok
10: DSO data read                                         : Ok
11: DSO data cache                                        : Ok
12: DSO data reopen                                       : Ok
13: Roundtrip evsel->name                                 : Ok
14: Parse sched tracepoints fields                        : Ok
15: syscalls:sys_enter_openat event fields                : Ok
16: Setup struct perf_event_attr                          : Ok
17: Match and link multiple hists                         : Ok
18: 'import perf' in python                               : Ok
19: Breakpoint overflow signal handler                    : Ok
20: Breakpoint overflow sampling                          : Ok
21: Breakpoint accounting                                 : Ok
22: Number of exit events of a simple workload            : Ok
23: Software clock events period values                   : Ok
24: Object code reading                                   : Ok
25: Sample parsing                                        : Ok
26: Use a dummy software event to keep tracking           : Ok
27: Parse with no sample_id_all bit set                   : Ok
28: Filter hist entries                                   : Ok
29: Lookup mmap thread                                    : Ok
30: Share thread mg                                       : Ok
31: Sort output of hist entries                           : Ok
32: Cumulate child hist entries                           : Ok
33: Track with sched_switch                               : Ok
34: Filter fds with revents mask in a fdarray             : Ok
35: Add fd to a fdarray, making it autogrow               : Ok
36: kmod_path__parse                                      : Ok
37: Thread map                                            : Ok
38: LLVM search and compile                               :
38.1: Basic BPF llvm compile                              : Ok
38.2: kbuild searching                                    : Ok
38.3: Compile source for BPF prologue generation          : Ok
38.4: Compile source for BPF relocation                   : Ok
39: Session topology                                      : Ok
40: BPF filter                                            :
40.1: Basic BPF filtering                                 : FAILED!
40.2: BPF pinning                                         : Skip
40.3: BPF prologue generation                             : Skip
40.4: BPF relocation checker                              : Skip
41: Synthesize thread map                                 : Ok
42: Remove thread map                                     : Ok
43: Synthesize cpu map                                    : Ok
44: Synthesize stat config                                : Ok
45: Synthesize stat                                       : Ok
46: Synthesize stat round                                 : Ok
47: Synthesize attr update                                : Ok
48: Event times                                           : Ok
49: Read backward ring buffer                             : Ok
50: Print cpu map                                         : Ok
51: Probe SDT events                                      : Ok
52: is_printable_array                                    : Ok
53: Print bitmap                                          : Ok
54: perf hooks                                            : Ok
55: builtin clang support                                 : Skip (not compiled in)
56: unit_number__scnprintf                                : Ok
57: mem2node                                              : Ok
58: x86 rdpmc                                             : Ok
59: Convert perf time to TSC                              : Ok
60: DWARF unwind                                          : Ok
61: x86 instruction decoder - new instructions            : Ok
62: Use vfs_getname probe to get syscall args filenames   : Ok
63: probe libc's inet_pton & backtrace it with ping       : Ok
64: Check open filename arg using perf trace + vfs_getname: Ok
65: probe libc's inet_pton & backtrace it with ping       : Ok
66: Add vfs_getname probe to get syscall args filenames   : Ok
[root@jouet ~]# 

If we use -v for that test we see the problem:

To the point:

  Probe point 'SyS_epoll_pwait' not found.

This is not there anymore, I'll change this test to first figure out
what is the syscall routine for the epoll_pwait syscall so that it works
with pre-syscall-routines-rename and after that.

- Arnaldo

all details:

[root@jouet ~]# perf test -v BPF
40: BPF filter                                            :
40.1: Basic BPF filtering                                 :
--- start ---
test child forked, pid 13225
Kernel build dir is set to /lib/modules/4.17.0-rc1-00024-g7e4e440a91c8-dirty/build
set env: KBUILD_DIR=/lib/modules/4.17.0-rc1-00024-g7e4e440a91c8-dirty/build
unset env: KBUILD_OPTS
include option is set to  -nostdinc -isystem /usr/lib/gcc/x86_64-redhat-linux/7/include -I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated  -I/home/acme/git/linux/include -I./include -I/home/acme/git/linux/arch/x86/include/uapi -I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi -I./include/generated/uapi -include /home/acme/git/linux/include/linux/kconfig.h 
set env: NR_CPUS=4
set env: LINUX_VERSION_CODE=0x41100
set env: CLANG_EXEC=/usr/lib64/ccache/clang
set env: CLANG_OPTIONS=-xc 
set env: KERNEL_INC_OPTIONS= -nostdinc -isystem /usr/lib/gcc/x86_64-redhat-linux/7/include -I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated  -I/home/acme/git/linux/include -I./include -I/home/acme/git/linux/arch/x86/include/uapi -I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi -I./include/generated/uapi -include /home/acme/git/linux/include/linux/kconfig.h 
set env: WORKING_DIR=/lib/modules/4.17.0-rc1-00024-g7e4e440a91c8-dirty/build
set env: CLANG_SOURCE=-
llvm compiling command template: echo '/*
 * bpf-script-example.c
 * Test basic LLVM building
 */
#ifndef LINUX_VERSION_CODE
# error Need LINUX_VERSION_CODE
# error Example: for 4.2 kernel, put 'clang-opt="-DLINUX_VERSION_CODE=0x40200" into llvm section of ~/.perfconfig'
#endif
#define BPF_ANY 0
#define BPF_MAP_TYPE_ARRAY 2
#define BPF_FUNC_map_lookup_elem 1
#define BPF_FUNC_map_update_elem 2

static void *(*bpf_map_lookup_elem)(void *map, void *key) =
	(void *) BPF_FUNC_map_lookup_elem;
static void *(*bpf_map_update_elem)(void *map, void *key, void *value, int flags) =
	(void *) BPF_FUNC_map_update_elem;

struct bpf_map_def {
	unsigned int type;
	unsigned int key_size;
	unsigned int value_size;
	unsigned int max_entries;
};

#define SEC(NAME) __attribute__((section(NAME), used))
struct bpf_map_def SEC("maps") flip_table = {
	.type = BPF_MAP_TYPE_ARRAY,
	.key_size = sizeof(int),
	.value_size = sizeof(int),
	.max_entries = 1,
};

SEC("func=SyS_epoll_pwait")
int bpf_func__SyS_epoll_pwait(void *ctx)
{
	int ind =0;
	int *flag = bpf_map_lookup_elem(&flip_table, &ind);
	int new_flag;
	if (!flag)
		return 0;
	/* flip flag and store back */
	new_flag = !*flag;
	bpf_map_update_elem(&flip_table, &ind, &new_flag, BPF_ANY);
	return new_flag;
}
char _license[] SEC("license") = "GPL";
int _version SEC("version") = LINUX_VERSION_CODE;
' | $CLANG_EXEC -D__KERNEL__ -D__NR_CPUS__=$NR_CPUS -DLINUX_VERSION_CODE=$LINUX_VERSION_CODE $CLANG_OPTIONS $KERNEL_INC_OPTIONS -Wno-unused-value -Wno-pointer-sign -working-directory $WORKING_DIR -c "$CLANG_SOURCE" -target bpf -O2 -o -
llvm compiling command : echo '/*
 * bpf-script-example.c
 * Test basic LLVM building
 */
#ifndef LINUX_VERSION_CODE
# error Need LINUX_VERSION_CODE
# error Example: for 4.2 kernel, put 'clang-opt=-DLINUX_VERSION_CODE=0x40200 into llvm section of ~/.perfconfig'
#endif
#define BPF_ANY 0
#define BPF_MAP_TYPE_ARRAY 2
#define BPF_FUNC_map_lookup_elem 1
#define BPF_FUNC_map_update_elem 2

static void *(*bpf_map_lookup_elem)(void *map, void *key) =
	(void *) BPF_FUNC_map_lookup_elem;
static void *(*bpf_map_update_elem)(void *map, void *key, void *value, int flags) =
	(void *) BPF_FUNC_map_update_elem;

struct bpf_map_def {
	unsigned int type;
	unsigned int key_size;
	unsigned int value_size;
	unsigned int max_entries;
};

#define SEC(NAME) __attribute__((section(NAME), used))
struct bpf_map_def SEC(maps) flip_table = {
	.type = BPF_MAP_TYPE_ARRAY,
	.key_size = sizeof(int),
	.value_size = sizeof(int),
	.max_entries = 1,
};

SEC(func=SyS_epoll_pwait)
int bpf_func__SyS_epoll_pwait(void *ctx)
{
	int ind =0;
	int *flag = bpf_map_lookup_elem(&flip_table, &ind);
	int new_flag;
	if (!flag)
		return 0;
	/* flip flag and store back */
	new_flag = !*flag;
	bpf_map_update_elem(&flip_table, &ind, &new_flag, BPF_ANY);
	return new_flag;
}
char _license[] SEC(license) = GPL;
int _version SEC(version) = LINUX_VERSION_CODE;
' | /usr/lib64/ccache/clang -D__KERNEL__ -D__NR_CPUS__=4 -DLINUX_VERSION_CODE=0x41100 -xc   -nostdinc -isystem /usr/lib/gcc/x86_64-redhat-linux/7/include -I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated  -I/home/acme/git/linux/include -I./include -I/home/acme/git/linux/arch/x86/include/uapi -I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi -I./include/generated/uapi -include /home/acme/git/linux/include/linux/kconfig.h  -Wno-unused-value -Wno-pointer-sign -working-directory /lib/modules/4.17.0-rc1-00024-g7e4e440a91c8-dirty/build -c - -target bpf -O2 -o -
libbpf: loading object '[basic_bpf_test]' from buffer
libbpf: section(1) .strtab, size 122, link 0, flags 0, type=3
libbpf: skip section(1) .strtab
libbpf: section(2) .text, size 0, link 0, flags 6, type=1
libbpf: skip section(2) .text
libbpf: section(3) func=SyS_epoll_pwait, size 192, link 0, flags 6, type=1
libbpf: found program func=SyS_epoll_pwait
libbpf: section(4) .relfunc=SyS_epoll_pwait, size 32, link 8, flags 0, type=9
libbpf: section(5) maps, size 16, link 0, flags 3, type=1
libbpf: section(6) license, size 4, link 0, flags 3, type=1
libbpf: license of [basic_bpf_test] is GPL
libbpf: section(7) version, size 4, link 0, flags 3, type=1
libbpf: kernel version of [basic_bpf_test] is 41100
libbpf: section(8) .symtab, size 168, link 1, flags 0, type=2
libbpf: maps in [basic_bpf_test]: 1 maps in 16 bytes
libbpf: map 0 is "flip_table"
libbpf: collecting relocating info for: 'func=SyS_epoll_pwait'
libbpf: relo for 6 value 0 name 81
libbpf: relocation: insn_idx=4
libbpf: relocation: find map 0 (flip_table) for insn 4
libbpf: relo for 6 value 0 name 81
libbpf: relocation: insn_idx=17
libbpf: relocation: find map 0 (flip_table) for insn 17
bpf: config program 'func=SyS_epoll_pwait'
symbol:SyS_epoll_pwait file:(null) line:0 offset:0 return:0 lazy:(null)
bpf: config 'func=SyS_epoll_pwait' is ok
Looking at the vmlinux_path (8 entries long)
Using /lib/modules/4.17.0-rc1-00024-g7e4e440a91c8-dirty/build/vmlinux for symbols
Open Debuginfo file: /lib/modules/4.17.0-rc1-00024-g7e4e440a91c8-dirty/build/vmlinux
Try to find probe point from debuginfo.
Probe point 'SyS_epoll_pwait' not found.
bpf_probe: failed to convert perf probe events
Failed to add events selected by BPF
test child finished with -1
---- end ----
BPF filter subtest 0: FAILED!
40.2: BPF pinning                                         :
--- force skipped ---
BPF filter subtest 1: Skip
40.3: BPF prologue generation                             :
--- force skipped ---
BPF filter subtest 2: Skip
40.4: BPF relocation checker                              :
--- force skipped ---
BPF filter subtest 3: Skip
[root@jouet ~]# 

Re: [PATCH v2] tracing/x86: Update syscall trace events to handle new x86 syscall func names

[Arnaldo Carvalho de Melo]

Em Wed, Apr 18, 2018 at 12:02:12PM -0300, Arnaldo Carvalho de Melo escreveu:
> Em Wed, Apr 18, 2018 at 10:36:06AM -0400, Steven Rostedt escreveu:
> > On Wed, 18 Apr 2018 09:53:22 -0300
> > Arnaldo Carvalho de Melo <acme@kernel.org> wrote:
> > > Em Tue, Apr 17, 2018 at 05:41:28PM -0400, Steven Rostedt escreveu:
> > > > On Tue, 17 Apr 2018 15:13:04 -0300 Arnaldo Carvalho de Melo <acme@kernel.org> wrote:  
> > > > > Yeah, failing:  
> 
> > > > > [root@jouet ~]# strace -e openat -e file perf test -F -v "mmap interface" |& grep syscalls
> > > > > openat(AT_FDCWD, "/sys/kernel/debug/tracing/events/syscalls/sys_enter_getsid/format", O_RDONLY) = 3
> > > > > openat(AT_FDCWD, "/sys/kernel/debug/tracing/events/syscalls/sys_enter_getppid/format", O_RDONLY) = -1 ENOENT (No such file or directory)  
> > >  
> > > > It doesn't have to do with the number of parameters, not everything
> > > > has "__x64" on it.  
> 
> > > > Try this patch:  
> 
> > > Trying...
>  
> > You're keeping me in suspense!
> 
> I switched locations, had trouble reconnecting, those tests are ok now,
> there is just one case left, related to the syscall routines renames,
> but not related to the syscalls:sys_{enter,exit}_NAME tracepoints:
> 40: BPF filter                                            :
> 40.1: Basic BPF filtering                                 : FAILED!
> 40.2: BPF pinning                                         : Skip
> 40.3: BPF prologue generation                             : Skip
> 40.4: BPF relocation checker                              : Skip
> 
> If we use -v for that test we see the problem:
> 
> To the point:
> 
>   Probe point 'SyS_epoll_pwait' not found.
> 
> This is not there anymore, I'll change this test to first figure out
> what is the syscall routine for the epoll_pwait syscall so that it works
> with pre-syscall-routines-rename and after that.

This does the trick, by not using the main syscall routine, but one
called from it and not renamed, should work with older kernels.

This test should be improved to look if the desired routine is in place,
if not just skip the test and tell about the unavailability of the
wanted function, but that is for later.

- Arnaldo

diff --git a/tools/perf/tests/bpf-script-example.c b/tools/perf/tests/bpf-script-example.c
index e4123c1b0e88..1ca5106df5f1 100644
--- a/tools/perf/tests/bpf-script-example.c
+++ b/tools/perf/tests/bpf-script-example.c
@@ -31,7 +31,7 @@ struct bpf_map_def SEC("maps") flip_table = {
 	.max_entries = 1,
 };
 
-SEC("func=SyS_epoll_pwait")
+SEC("func=do_epoll_wait")
 int bpf_func__SyS_epoll_pwait(void *ctx)
 {
 	int ind =0;

Re: [PATCH v2] tracing/x86: Update syscall trace events to handle new x86 syscall func names

[Steven Rostedt]

On Wed, 18 Apr 2018 12:17:16 -0300
Arnaldo Carvalho de Melo <acme@kernel.org> wrote:

> This does the trick, by not using the main syscall routine, but one
> called from it and not renamed, should work with older kernels.
> 
> This test should be improved to look if the desired routine is in place,
> if not just skip the test and tell about the unavailability of the
> wanted function, but that is for later.

Does this mean you can give me a "Tested-by" for that last patch?

-- Steve

Re: [PATCH v2] tracing/x86: Update syscall trace events to handle new x86 syscall func names

[Arnaldo Carvalho de Melo]

Em Wed, Apr 18, 2018 at 11:20:33AM -0400, Steven Rostedt escreveu:
> On Wed, 18 Apr 2018 12:17:16 -0300
> Arnaldo Carvalho de Melo <acme@kernel.org> wrote:
> 
> > This does the trick, by not using the main syscall routine, but one
> > called from it and not renamed, should work with older kernels.
> > 
> > This test should be improved to look if the desired routine is in place,
> > if not just skip the test and tell about the unavailability of the
> > wanted function, but that is for later.
> 
> Does this mean you can give me a "Tested-by" for that last patch?

Here it is, just written down:

Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>

- Arnaldo

Re: [PATCH v2] tracing/x86: Update syscall trace events to handle new x86 syscall func names

[Steven Rostedt]

On Wed, 18 Apr 2018 12:25:36 -0300
Arnaldo Carvalho de Melo <acme@kernel.org> wrote:

> Em Wed, Apr 18, 2018 at 11:20:33AM -0400, Steven Rostedt escreveu:
> > On Wed, 18 Apr 2018 12:17:16 -0300
> > Arnaldo Carvalho de Melo <acme@kernel.org> wrote:
> >   
> > > This does the trick, by not using the main syscall routine, but one
> > > called from it and not renamed, should work with older kernels.
> > > 
> > > This test should be improved to look if the desired routine is in place,
> > > if not just skip the test and tell about the unavailability of the
> > > wanted function, but that is for later.  
> > 
> > Does this mean you can give me a "Tested-by" for that last patch?  
> 
> Here it is, just written down:
> 
> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
> 

Thanks!

-- Steve

Re: [PATCH v2] tracing/x86: Update syscall trace events to handle new x86 syscall func names

[Steven Rostedt]

On Tue, 17 Apr 2018 19:22:36 +0200
Dominik Brodowski <linux@dominikbrodowski.net> wrote:

> >   Added back original compare to not miss 32bit kernel syscalls  
> 
> s/32bit/32bit and 0-parameter syscalls.
> 
> Looks good otherwise.
> 

You want to send an Acked or Reviewed-by to my final patch (v3).

-- Steve

Re: [PATCH v2] tracing/x86: Update syscall trace events to handle new x86 syscall func names

[Arnaldo Carvalho de Melo]

Em Tue, Apr 17, 2018 at 01:07:02PM -0400, Steven Rostedt escreveu:
> From: "Steven Rostedt (VMware)" <rostedt@goodmis.org>
> 
> Arnaldo noticed that the latest kernel is missing the syscall event system
> directory in x86. I bisected it down to d5a00528b58c ("syscalls/core,
> syscalls/x86: Rename struct pt_regs-based sys_*() to __x64_sys_*()").
> 
> The system call trace events are special, as there is only one trace event
> for all system calls (the raw_syscalls). But a macro that wraps the system
> calls creates meta data for them that copies the name to find the system
> call that maps to the system call table (the number). At boot up, it does a
> kallsyms lookup for this mapping. If it does not find a function, then that
> system call is ignored.
> 
> Because the x86 system calls had "__x64_" appended to the "sys" for the
> names, they do not match the default compare algorithm. As this was a
> problem for power pc, the algorithm can be overwritten by the architecture.
> The solution is to have x86 have its own algorithm to do the compare and
> this brings back the system call trace events.
> 
> Reported-by: Arnaldo Carvalho de Melo <acme@redhat.com>
> Fixes: d5a00528b58c ("syscalls/core, syscalls/x86: Rename struct pt_regs-based sys_*() to __x64_sys_*()")
> Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>

Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>

Again on x86_64, the number of files in
/sys/kernel/debug/tracing/events/syscalls/ is the same, all my perf
related tests pass.

Now to the next seemingly kernel related test failure, unrelated to
this:

 4: Read samples using the mmap interface                 : FAILED!

- Arnaldo

Re: [PATCH v2] tracing/x86: Update syscall trace events to handle new x86 syscall func names

[Arnaldo Carvalho de Melo]

Em Tue, Apr 17, 2018 at 02:29:40PM -0300, Arnaldo Carvalho de Melo escreveu:
> Em Tue, Apr 17, 2018 at 01:07:02PM -0400, Steven Rostedt escreveu:
> > Because the x86 system calls had "__x64_" appended to the "sys" for the
> > names, they do not match the default compare algorithm. As this was a
> > problem for power pc, the algorithm can be overwritten by the architecture.
> > The solution is to have x86 have its own algorithm to do the compare and
> > this brings back the system call trace events.

> > Reported-by: Arnaldo Carvalho de Melo <acme@redhat.com>
> > Fixes: d5a00528b58c ("syscalls/core, syscalls/x86: Rename struct pt_regs-based sys_*() to __x64_sys_*()")
> > Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
 
> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
 
> Again on x86_64, the number of files in
> /sys/kernel/debug/tracing/events/syscalls/ is the same, all my perf
> related tests pass.
> 
> Now to the next seemingly kernel related test failure, unrelated to
> this:
> 
>  4: Read samples using the mmap interface                 : FAILED!

Ok, I take that back, it is related:

[root@jouet perf]# perf test -F -v "mmap interface" |& grep -v plugins
 4: Read samples using the mmap interface                 :
--- start ---
perf_evsel__new(sys_enter_getppid)
---- end ----
Read samples using the mmap interface: FAILED!
[root@jouet perf]#

[root@jouet perf]# strace -e openat -e file perf test -F -v "mmap interface" |& grep syscalls
openat(AT_FDCWD, "/sys/kernel/debug/tracing/events/syscalls/sys_enter_getsid/format", O_RDONLY) = 3
openat(AT_FDCWD, "/sys/kernel/debug/tracing/events/syscalls/sys_enter_getppid/format", O_RDONLY) = -1 ENOENT (No such file or directory)
[root@jouet perf]#

Where did syscalls:sys_enter_getppid go? Zero argument syscall,
investigating...

- Arnaldo