linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [RFC PATCH 00/50] tools/perf: Speed up the build system
@ 2013-10-07 11:55 Ingo Molnar
  2013-10-07 11:55 ` [PATCH 01/50] tools/perf: Standardize feature support define names to: HAVE_{FEATURE}_SUPPORT Ingo Molnar
                   ` (51 more replies)
  0 siblings, 52 replies; 84+ messages in thread
From: Ingo Molnar @ 2013-10-07 11:55 UTC (permalink / raw)
  To: linux-kernel
  Cc: Peter Zijlstra, Arnaldo Carvalho de Melo, Namhyung Kim,
	David Ahern, Jiri Olsa

This series improves various aspects of the tools/perf build system.

The biggest change is the rewrite of the feature detection framework and a
conversion of all feature testcases to the new method. The new feature
detection machinery relies on three new methods:

  - split-out, independent testcases that allow "make -j" parallel builds
    [ the old build system built the 20+ testcases serially, greatly
      slowing down the build. ]

  - dependency auto-generation via gcc's -MD feature, which allows the
    'caching' of testcases.

  - a 'test-all' testcase that contains all the other testcases and allows
    a fast check for the typical 'all dependencies are installed' case.

The series also includes a number of fixes and convenience features:

   - auto-detect the number of CPUs on the system and add the -j<cpus>
     option automatically

   - Fix 'make O=/some/dir/ perf.o' type of rules to work in the $(OUTPUT)
     directory

   - Improve documentation builds/install to not require feature-detection

   - speed up linking, eliminate spurious rebuilds

Altogether build speed on my testsystem improved by a factor of 2.1x:

  [BEFORE]

    $ cd tools/perf
    $ time make -j12 install
    ...

      real    0m27.522s
      user    1m59.478s
      sys     0m9.714s

  [AFTER]
    $ cd tools/perf
    $ time make install
    ...

      real    0m12.658s
      user    1m39.507s
      sys     0m7.706s

Re-build speed, which is important to perf developers, increased 5.0x:

  [BEFORE]
    $ touch perf.c
    $ time make
    ...

      real    0m9.368s
      user    0m8.166s
      sys     0m1.015s

  [AFTER]

    $ touch perf.c
    $ time make
    BUILD: Doing 'make -j12' parallel build
    CC perf.o
    LINK perf

      real    0m1.857s
      user    0m1.522s
      sys     0m0.295s

Empty build speed of 'make' in an already built tree - often done by
developers to see whether they are testing the latest binary, increased
more than 50-fold:

  [BEFORE]
    $ time make
    ...

      real    0m9.105s
      user    0m7.975s
      sys     0m0.949s

  [AFTER]

    $ time make

    $ time make
    BUILD: Doing 'make -j12' parallel build

      real    0m0.172s
      user    0m0.119s
      sys     0m0.029s

Thanks,

   Ingo

=============>
Ingo Molnar (50):
  tools/perf: Standardize feature support define names to: HAVE_{FEATURE}_SUPPORT
  tools/perf/build: Add feature check core code
  tools/perf/build: Add 'autodep' functionality, generate feature test dependencies automatically
  tools/perf/build: Split out feature check: 'libnuma'
  tools/perf/build: Split out feature check: 'stackprotector-all'
  tools/perf/build: Split out feature check: 'stackprotector'
  tools/perf/build: Split out feature check: 'volatile-register-var'
  tools/perf/build: Split out feature check: 'fortify-source'
  tools/perf/build: Split out feature check: 'bionic'
  tools/perf/build: Clean up the libelf logic in config/Makefile
  tools/perf/build: Split out feature check: 'libelf'
  tools/perf/build: Split out feature check: 'glibc'
  tools/perf/build: Split out feature check: 'dwarf'
  tools/perf/build: Clean up the mmap logic in config/Makefile
  tools/perf/build: Split out feature check: 'libelf-mmap'
  tools/perf/build: Split out feature check: 'libelf-getphdrnum'
  tools/perf/build: Clean up the libunwind logic in config/Makefile
  tools/perf/build: Split out feature check: 'libunwind'
  tools/perf/build: Split out feature check: 'libaudit'
  tools/perf/build: Split out feature check: 'libslang'
  tools/perf/build: Split out feature check: 'gtk2'
  tools/perf/build: Split out feature check: 'gtk2-infobar'
  tools/perf/build: Split out feature check: 'libperl'
  tools/perf/build: Split out feature check: 'libpython'
  tools/perf/build: Split out feature check: 'libpython-version'
  tools/perf/build: Split out feature check: 'libbfd'
  tools/perf/build: Split out feature check: 'on-exit'
  tools/perf/build: Split out feature check: 'backtrace'
  tools/perf: Clean up util/include/linux/compiler.h
  tools/perf: Turn strlcpy() into a __weak function
  tools/perf/build: Speed up auto-detection of features by adding a 'test-all' target
  tools/perf/build: Speed up git-version test on re-make
  tools/perf/build: Speed up the final link
  tools/perf: Fix double/triple-build of the feature detection logic during 'make install' et al
  tools/perf/build: Invoke feature-checks 'clean' target from the main Makefile
  tools/perf/build: Speed up auto-detection
  tools/perf/build: Improve printout-of auto-detected features
  tools/perf/build: Automatically build in parallel, based on number of CPUs in the system
  tools/perf/build: Flip Makefile.parallel and Makefile.perf
  tools/perf/build: Standardize the various messages output by parallel make
  tools/perf/build: Split out feature checks: 'liberty', 'liberty-z', 'cplus-demangle'
  tools/perf/build: Remove unused config/feature-tests.mak
  tools/perf/build: Clean up various testcases
  tools/perf/build: Collapse the test-all.c testcase
  tools/perf/build: Pass through all targets to Makefile.perf
  tools/perf/build: Make sure autodep feature binaries honor the O= setting
  tools/perf/build: Exclude MAKEFLAGS from nested invocation
  tools/perf/build: Fix non-canonical directory names in O=
  tools/perf/build: Fix O=/some/dir perf.o type of targets
  tools/perf/build: Harmonize the style of the feature testcases

 tools/perf/Documentation/Makefile                             |  19 +-
 tools/perf/Makefile                                           | 840 ++---------------------------
 tools/perf/Makefile.perf                                      | 855 ++++++++++++++++++++++++++++++
 tools/perf/arch/x86/include/perf_regs.h                       |   6 +-
 tools/perf/arch/x86/util/unwind.c                             |   4 +-
 tools/perf/bench/mem-memcpy-arch.h                            |   2 +-
 tools/perf/bench/mem-memcpy.c                                 |   2 +-
 tools/perf/bench/mem-memset-arch.h                            |   2 +-
 tools/perf/bench/mem-memset.c                                 |   2 +-
 tools/perf/builtin-bench.c                                    |   4 +-
 tools/perf/builtin-inject.c                                   |   2 +-
 tools/perf/builtin-probe.c                                    |  14 +-
 tools/perf/builtin-record.c                                   |  12 +-
 tools/perf/config/Makefile                                    | 326 +++++++-----
 tools/perf/config/feature-checks/Makefile                     | 148 ++++++
 tools/perf/config/feature-checks/test-all.c                   | 106 ++++
 tools/perf/config/feature-checks/test-backtrace.c             |  13 +
 tools/perf/config/feature-checks/test-bionic.c                |   6 +
 tools/perf/config/feature-checks/test-cplus-demangle.c        |  14 +
 tools/perf/config/feature-checks/test-dwarf.c                 |  10 +
 tools/perf/config/feature-checks/test-fortify-source.c        |   6 +
 tools/perf/config/feature-checks/test-glibc.c                 |   8 +
 tools/perf/config/feature-checks/test-gtk2-infobar.c          |  11 +
 tools/perf/config/feature-checks/test-gtk2.c                  |  10 +
 tools/perf/config/feature-checks/test-hello.c                 |   6 +
 tools/perf/config/feature-checks/test-libaudit.c              |  10 +
 tools/perf/config/feature-checks/test-libbfd.c                |  15 +
 tools/perf/config/feature-checks/test-libelf-getphdrnum.c     |   8 +
 tools/perf/config/feature-checks/test-libelf-mmap.c           |   8 +
 tools/perf/config/feature-checks/test-libelf.c                |   8 +
 tools/perf/config/feature-checks/test-libnuma.c               |   9 +
 tools/perf/config/feature-checks/test-libperl.c               |   9 +
 tools/perf/config/feature-checks/test-libpython-version.c     |  10 +
 tools/perf/config/feature-checks/test-libpython.c             |   8 +
 tools/perf/config/feature-checks/test-libslang.c              |   6 +
 tools/perf/config/feature-checks/test-libunwind.c             |  27 +
 tools/perf/config/feature-checks/test-on-exit.c               |  15 +
 tools/perf/config/feature-checks/test-stackprotector-all.c    |   6 +
 tools/perf/config/feature-checks/test-stackprotector.c        |   6 +
 tools/perf/config/feature-checks/test-volatile-register-var.c |   6 +
 tools/perf/config/feature-tests.mak                           | 246 ---------
 tools/perf/config/utilities.mak                               |  14 -
 tools/perf/perf.c                                             |   4 +-
 tools/perf/ui/gtk/browser.c                                   |   2 +-
 tools/perf/ui/gtk/gtk.h                                       |   4 +-
 tools/perf/ui/gtk/util.c                                      |   4 +-
 tools/perf/ui/ui.h                                            |   4 +-
 tools/perf/util/annotate.h                                    |   4 +-
 tools/perf/util/cache.h                                       |   3 +-
 tools/perf/util/generate-cmdlist.sh                           |   4 +-
 tools/perf/util/hist.h                                        |   4 +-
 tools/perf/util/include/dwarf-regs.h                          |   2 +-
 tools/perf/util/include/linux/compiler.h                      |  19 +-
 tools/perf/util/map.c                                         |   2 +-
 tools/perf/util/path.c                                        |  10 +-
 tools/perf/util/perf_regs.h                                   |   4 +-
 tools/perf/util/probe-event.c                                 |   4 +-
 tools/perf/util/probe-finder.h                                |   4 +-
 tools/perf/util/symbol-elf.c                                  |   2 +-
 tools/perf/util/symbol.h                                      |   8 +-
 tools/perf/util/unwind.h                                      |   4 +-
 tools/perf/util/util.c                                        |   4 +-
 62 files changed, 1678 insertions(+), 1257 deletions(-)
 create mode 100644 tools/perf/Makefile.perf
 create mode 100644 tools/perf/config/feature-checks/Makefile
 create mode 100644 tools/perf/config/feature-checks/test-all.c
 create mode 100644 tools/perf/config/feature-checks/test-backtrace.c
 create mode 100644 tools/perf/config/feature-checks/test-bionic.c
 create mode 100644 tools/perf/config/feature-checks/test-cplus-demangle.c
 create mode 100644 tools/perf/config/feature-checks/test-dwarf.c
 create mode 100644 tools/perf/config/feature-checks/test-fortify-source.c
 create mode 100644 tools/perf/config/feature-checks/test-glibc.c
 create mode 100644 tools/perf/config/feature-checks/test-gtk2-infobar.c
 create mode 100644 tools/perf/config/feature-checks/test-gtk2.c
 create mode 100644 tools/perf/config/feature-checks/test-hello.c
 create mode 100644 tools/perf/config/feature-checks/test-libaudit.c
 create mode 100644 tools/perf/config/feature-checks/test-libbfd.c
 create mode 100644 tools/perf/config/feature-checks/test-libelf-getphdrnum.c
 create mode 100644 tools/perf/config/feature-checks/test-libelf-mmap.c
 create mode 100644 tools/perf/config/feature-checks/test-libelf.c
 create mode 100644 tools/perf/config/feature-checks/test-libnuma.c
 create mode 100644 tools/perf/config/feature-checks/test-libperl.c
 create mode 100644 tools/perf/config/feature-checks/test-libpython-version.c
 create mode 100644 tools/perf/config/feature-checks/test-libpython.c
 create mode 100644 tools/perf/config/feature-checks/test-libslang.c
 create mode 100644 tools/perf/config/feature-checks/test-libunwind.c
 create mode 100644 tools/perf/config/feature-checks/test-on-exit.c
 create mode 100644 tools/perf/config/feature-checks/test-stackprotector-all.c
 create mode 100644 tools/perf/config/feature-checks/test-stackprotector.c
 create mode 100644 tools/perf/config/feature-checks/test-volatile-register-var.c
 delete mode 100644 tools/perf/config/feature-tests.mak

-- 
1.7.11.7


^ permalink raw reply	[flat|nested] 84+ messages in thread

end of thread, other threads:[~2013-10-15  5:20 UTC | newest]

Thread overview: 84+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-10-07 11:55 [RFC PATCH 00/50] tools/perf: Speed up the build system Ingo Molnar
2013-10-07 11:55 ` [PATCH 01/50] tools/perf: Standardize feature support define names to: HAVE_{FEATURE}_SUPPORT Ingo Molnar
2013-10-07 11:55 ` [PATCH 02/50] tools/perf/build: Add feature check core code Ingo Molnar
2013-10-07 11:55 ` [PATCH 03/50] tools/perf/build: Add 'autodep' functionality, generate feature test dependencies automatically Ingo Molnar
2013-10-07 11:55 ` [PATCH 04/50] tools/perf/build: Split out feature check: 'libnuma' Ingo Molnar
2013-10-07 11:55 ` [PATCH 05/50] tools/perf/build: Split out feature check: 'stackprotector-all' Ingo Molnar
2013-10-07 11:55 ` [PATCH 06/50] tools/perf/build: Split out feature check: 'stackprotector' Ingo Molnar
2013-10-07 11:56 ` [PATCH 07/50] tools/perf/build: Split out feature check: 'volatile-register-var' Ingo Molnar
2013-10-07 11:56 ` [PATCH 08/50] tools/perf/build: Split out feature check: 'fortify-source' Ingo Molnar
2013-10-07 11:56 ` [PATCH 09/50] tools/perf/build: Split out feature check: 'bionic' Ingo Molnar
2013-10-07 11:56 ` [PATCH 10/50] tools/perf/build: Clean up the libelf logic in config/Makefile Ingo Molnar
2013-10-07 11:56 ` [PATCH 11/50] tools/perf/build: Split out feature check: 'libelf' Ingo Molnar
2013-10-07 11:56 ` [PATCH 12/50] tools/perf/build: Split out feature check: 'glibc' Ingo Molnar
2013-10-07 11:56 ` [PATCH 13/50] tools/perf/build: Split out feature check: 'dwarf' Ingo Molnar
2013-10-07 11:56 ` [PATCH 14/50] tools/perf/build: Clean up the mmap logic in config/Makefile Ingo Molnar
2013-10-07 11:56 ` [PATCH 15/50] tools/perf/build: Split out feature check: 'libelf-mmap' Ingo Molnar
2013-10-07 11:56 ` [PATCH 16/50] tools/perf/build: Split out feature check: 'libelf-getphdrnum' Ingo Molnar
2013-10-07 11:56 ` [PATCH 17/50] tools/perf/build: Clean up the libunwind logic in config/Makefile Ingo Molnar
2013-10-07 11:56 ` [PATCH 18/50] tools/perf/build: Split out feature check: 'libunwind' Ingo Molnar
2013-10-07 11:56 ` [PATCH 19/50] tools/perf/build: Split out feature check: 'libaudit' Ingo Molnar
2013-10-07 21:00   ` Jiri Olsa
2013-10-08  8:19     ` Ingo Molnar
2013-10-07 11:56 ` [PATCH 20/50] tools/perf/build: Split out feature check: 'libslang' Ingo Molnar
2013-10-07 14:52   ` David Ahern
2013-10-08  8:06     ` Ingo Molnar
2013-10-07 11:56 ` [PATCH 21/50] tools/perf/build: Split out feature check: 'gtk2' Ingo Molnar
2013-10-07 11:56 ` [PATCH 22/50] tools/perf/build: Split out feature check: 'gtk2-infobar' Ingo Molnar
2013-10-07 11:56 ` [PATCH 23/50] tools/perf/build: Split out feature check: 'libperl' Ingo Molnar
2013-10-07 11:56 ` [PATCH 24/50] tools/perf/build: Split out feature check: 'libpython' Ingo Molnar
2013-10-07 11:56 ` [PATCH 25/50] tools/perf/build: Split out feature check: 'libpython-version' Ingo Molnar
2013-10-07 11:56 ` [PATCH 26/50] tools/perf/build: Split out feature check: 'libbfd' Ingo Molnar
2013-10-07 15:11   ` David Ahern
2013-10-07 21:18     ` Jiri Olsa
2013-10-08  8:15       ` Ingo Molnar
2013-10-07 11:56 ` [PATCH 27/50] tools/perf/build: Split out feature check: 'on-exit' Ingo Molnar
2013-10-07 11:56 ` [PATCH 28/50] tools/perf/build: Split out feature check: 'backtrace' Ingo Molnar
2013-10-07 11:56 ` [PATCH 29/50] tools/perf: Clean up util/include/linux/compiler.h Ingo Molnar
2013-10-07 11:56 ` [PATCH 30/50] tools/perf: Turn strlcpy() into a __weak function Ingo Molnar
2013-10-07 11:56 ` [PATCH 31/50] tools/perf/build: Speed up auto-detection of features by adding a 'test-all' target Ingo Molnar
2013-10-07 21:27   ` Jiri Olsa
2013-10-08  8:23     ` Ingo Molnar
2013-10-07 21:59   ` Jiri Olsa
2013-10-07 11:56 ` [PATCH 32/50] tools/perf/build: Speed up git-version test on re-make Ingo Molnar
2013-10-07 11:56 ` [PATCH 33/50] tools/perf/build: Speed up the final link Ingo Molnar
2013-10-07 11:56 ` [PATCH 34/50] tools/perf: Fix double/triple-build of the feature detection logic during 'make install' et al Ingo Molnar
2013-10-07 11:56 ` [PATCH 35/50] tools/perf/build: Invoke feature-checks 'clean' target from the main Makefile Ingo Molnar
2013-10-07 11:56 ` [PATCH 36/50] tools/perf/build: Speed up auto-detection Ingo Molnar
2013-10-07 21:57   ` Jiri Olsa
2013-10-08  8:36     ` Ingo Molnar
2013-10-07 11:56 ` [PATCH 37/50] tools/perf/build: Improve printout-of auto-detected features Ingo Molnar
2013-10-07 22:12   ` Jiri Olsa
2013-10-08  8:46     ` Ingo Molnar
2013-10-08 15:51       ` Jiri Olsa
2013-10-09  6:42         ` [PATCH 53/52] tools/perf/build: Clean up feature_print_code() Ingo Molnar
2013-10-09  8:38           ` Jiri Olsa
2013-10-15  5:20         ` [tip:perf/core] " tip-bot for Jiri Olsa
2013-10-07 11:56 ` [PATCH 38/50] tools/perf/build: Automatically build in parallel, based on number of CPUs in the system Ingo Molnar
2013-10-07 11:56 ` [PATCH 39/50] tools/perf/build: Flip Makefile.parallel and Makefile.perf Ingo Molnar
2013-10-07 11:56 ` [PATCH 40/50] tools/perf/build: Standardize the various messages output by parallel make Ingo Molnar
2013-10-07 11:56 ` [PATCH 41/50] tools/perf/build: Split out feature checks: 'liberty', 'liberty-z', 'cplus-demangle' Ingo Molnar
2013-10-07 11:56 ` [PATCH 42/50] tools/perf/build: Remove unused config/feature-tests.mak Ingo Molnar
2013-10-07 11:56 ` [PATCH 43/50] tools/perf/build: Clean up various testcases Ingo Molnar
2013-10-07 11:56 ` [PATCH 44/50] tools/perf/build: Collapse the test-all.c testcase Ingo Molnar
2013-10-07 22:25   ` Jiri Olsa
2013-10-08  8:47     ` Ingo Molnar
2013-10-07 11:56 ` [PATCH 45/50] tools/perf/build: Pass through all targets to Makefile.perf Ingo Molnar
2013-10-07 11:56 ` [PATCH 46/50] tools/perf/build: Make sure autodep feature binaries honor the O= setting Ingo Molnar
2013-10-07 11:56 ` [PATCH 47/50] tools/perf/build: Exclude MAKEFLAGS from nested invocation Ingo Molnar
2013-10-07 11:56 ` [PATCH 48/50] tools/perf/build: Fix non-canonical directory names in O= Ingo Molnar
2013-10-07 11:56 ` [PATCH 49/50] tools/perf/build: Fix O=/some/dir perf.o type of targets Ingo Molnar
2013-10-07 22:33   ` Jiri Olsa
2013-10-08  8:49     ` Ingo Molnar
2013-10-07 11:56 ` [PATCH 50/50] tools/perf/build: Harmonize the style of the feature testcases Ingo Molnar
2013-10-07 13:52 ` [RFC PATCH 00/50] tools/perf: Speed up the build system David Ahern
2013-10-07 14:11   ` Ingo Molnar
2013-10-07 15:35     ` David Ahern
2013-10-07 15:51       ` [PATCH] tools/perf/build: Pass through LDFLAGS to feature tests Ingo Molnar
2013-10-07 16:07         ` David Ahern
2013-10-15  5:20         ` [tip:perf/core] " tip-bot for Ingo Molnar
2013-10-07 16:41       ` [RFC PATCH 00/50] tools/perf: Speed up the build system Ingo Molnar
2013-10-08  1:41         ` David Ahern
2013-10-08 12:48           ` Ingo Molnar
2013-10-07 20:15 ` Andi Kleen
2013-10-08  9:02   ` Ingo Molnar

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).