On Fri, Aug 30, 2013 at 6:52 PM, Linus Torvalds wrote: > On Fri, Aug 30, 2013 at 9:37 AM, Sedat Dilek wrote: >> >> Where is this a.out file from or how to generate it? > > Oh, that's just the silly threaded test-binary. I don't know what you > called it. > > As to your config options, yesh, you have some expensive stuff. > DEBUG_OBJECTS and DEBUG_MUTEXES in particular tend to cause lots of > horrible performance issues. I didn't check if there might be other > things.. > I tried w/o DEBUG_OBJECTS and DEBUG_MUTEXES and disabled some unnecessary debug-options, too (see attached diff). This is what I get now... [ TEST-CASE ] $ ~/src/linux-kernel/linux/tools/perf/perf stat --null --repeat 5 ./scripts/t_lockref_from-linus Total loops: 26480075 Total loops: 27002388 Total loops: 25761463 Total loops: 26877615 Total loops: 27047644 Performance counter stats for './scripts/t_lockref_from-linus' (5 runs): 10,008617789 seconds time elapsed ( +- 0,07% ) Looks like this is now 10x faster: ~2.66Mloops (debug) VS. ~26.60Mloops (no-debug). [ PERF-RECORD ] $ sudo ~/src/linux-kernel/linux/tools/perf/perf record -e cycles:pp ./scripts/t_lockref_from-linus Total loops: 26601346 [ perf record: Woken up 25 times to write data ] [ perf record: Captured and wrote 6.100 MB perf.data (~266501 samples) ] [ PERF-REPORT ] $ sudo ~/src/linux-kernel/linux/tools/perf/perf report -f Samples: 159K of event 'cycles:pp', Event count (approx.): 76968896763 12,79% t_lockref_from- [kernel.kallsyms] [k] irq_return 4,36% t_lockref_from- [kernel.kallsyms] [k] __ticket_spin_lock 4,36% t_lockref_from- [kernel.kallsyms] [k] __acct_update_integrals 4,07% t_lockref_from- [kernel.kallsyms] [k] user_exit 3,12% t_lockref_from- [kernel.kallsyms] [k] local_clock 2,83% t_lockref_from- [kernel.kallsyms] [k] lockref_get_or_lock 2,73% t_lockref_from- [kernel.kallsyms] [k] kmem_cache_alloc 2,62% t_lockref_from- [kernel.kallsyms] [k] __d_lookup_rcu 2,53% t_lockref_from- libc-2.15.so [.] __xstat64 2,53% t_lockref_from- [kernel.kallsyms] [k] kmem_cache_free 2,28% t_lockref_from- [kernel.kallsyms] [k] path_init 2,27% t_lockref_from- [kernel.kallsyms] [k] link_path_walk 1,86% t_lockref_from- [kernel.kallsyms] [k] user_enter 1,85% t_lockref_from- [kernel.kallsyms] [k] rcu_eqs_exit_common.isra.43 1,81% t_lockref_from- [kernel.kallsyms] [k] sched_clock_cpu 1,79% t_lockref_from- [kernel.kallsyms] [k] rcu_eqs_enter_common.isra.45 1,78% t_lockref_from- [kernel.kallsyms] [k] path_lookupat 1,67% t_lockref_from- [kernel.kallsyms] [k] native_read_tsc 1,63% t_lockref_from- [kernel.kallsyms] [k] cp_new_stat 1,61% t_lockref_from- [kernel.kallsyms] [k] lockref_put_or_lock 1,53% t_lockref_from- [kernel.kallsyms] [k] account_system_time 1,48% t_lockref_from- [kernel.kallsyms] [k] tracesys 1,47% t_lockref_from- [kernel.kallsyms] [k] copy_user_generic_unrolled 1,46% t_lockref_from- [kernel.kallsyms] [k] syscall_trace_enter 1,39% t_lockref_from- [kernel.kallsyms] [k] jiffies_to_timeval 1,33% t_lockref_from- [kernel.kallsyms] [k] native_sched_clock 1,27% t_lockref_from- [kernel.kallsyms] [k] getname_flags 1,27% t_lockref_from- [kernel.kallsyms] [k] lookup_fast 1,18% t_lockref_from- [kernel.kallsyms] [k] get_vtime_delta 1,05% t_lockref_from- [kernel.kallsyms] [k] syscall_trace_leave 1,03% t_lockref_from- [kernel.kallsyms] [k] generic_fillattr 1,02% t_lockref_from- [kernel.kallsyms] [k] strncpy_from_user 1,00% t_lockref_from- [kernel.kallsyms] [k] user_path_at_empty 0,97% t_lockref_from- [kernel.kallsyms] [k] account_user_time 0,95% t_lockref_from- [kernel.kallsyms] [k] vfs_fstatat 0,95% t_lockref_from- [kernel.kallsyms] [k] system_call_after_swapgs 0,92% t_lockref_from- [kernel.kallsyms] [k] generic_permission 0,91% t_lockref_from- [kernel.kallsyms] [k] filename_lookup 0,80% t_lockref_from- [kernel.kallsyms] [k] vfs_getattr 0,78% t_lockref_from- [kernel.kallsyms] [k] __ticket_spin_unlock 0,74% t_lockref_from- [kernel.kallsyms] [k] complete_walk 0,70% t_lockref_from- [kernel.kallsyms] [k] vtime_account_user 0,68% t_lockref_from- [kernel.kallsyms] [k] d_rcu_to_refcount 0,65% t_lockref_from- [kernel.kallsyms] [k] common_perm 0,62% t_lockref_from- [kernel.kallsyms] [k] rcu_eqs_enter 0,58% t_lockref_from- [kernel.kallsyms] [k] vtime_user_enter 0,57% t_lockref_from- [kernel.kallsyms] [k] __inode_permission 0,55% t_lockref_from- [kernel.kallsyms] [k] dput 0,52% t_lockref_from- [kernel.kallsyms] [k] apparmor_inode_getattr 0,52% t_lockref_from- [kernel.kallsyms] [k] SYSC_newstat 0,52% t_lockref_from- [kernel.kallsyms] [k] mntget 0,49% t_lockref_from- [kernel.kallsyms] [k] cpuacct_account_field 0,48% t_lockref_from- [kernel.kallsyms] [k] __vtime_account_system 0,46% t_lockref_from- t_lockref_from-linus [.] start_routine Thanks for all the explanations and hints. Regards, - Sedat - P.S.: Some words to "perf -f"... $ sudo ~/src/linux-kernel/linux/tools/perf/perf record -f -e cycles:pp ./scripts/t_lockref_from-linus [sudo] password for wearefam: Error: unknown switch `f' usage: perf record [] [] or: perf record [] -- [] -e, --event event selector. use 'perf list' to list available events --filter event filter -p, --pid record events on existing process id -t, --tid record events on existing thread id -r, --realtime collect data with this RT SCHED_FIFO priority -D, --no-delay collect data without buffering -R, --raw-samples collect raw sample records from all opened counters -a, --all-cpus system-wide collection from all CPUs -C, --cpu list of cpus to monitor -c, --count event period to sample -o, --output output file name -i, --no-inherit child tasks do not inherit counters -F, --freq profile at this frequency -m, --mmap-pages number of mmap data pages --group put the counters into a counter group -g, --call-graph do call-graph (stack chain/backtrace) recording: [fp] dwarf -v, --verbose be more verbose (show counter open errors, etc) -q, --quiet don't print any message -s, --stat per thread counts -d, --data Sample addresses -T, --timestamp Sample timestamps -P, --period Sample period -n, --no-samples don't sample -N, --no-buildid-cache do not update the buildid cache -B, --no-buildid do not collect buildids in perf.data -G, --cgroup monitor event in cgroup name only -u, --uid user to profile -b, --branch-any sample any taken branches -j, --branch-filter branch stack filter modes -W, --weight sample by weight (on special events only) - EOT -