[PATCH 0/6] tools: Add clean targets for tools directory

[PATCH 0/6] tools: Add clean targets for tools directory

[Jiri Olsa]

hi,
sending some tools build fixes,

patches 5 and 6 are RFC and I'd like to have it
tested/confirmed by Wang Nan it suits his needs.

All of them available on top of my perf/fixes branch at:
  git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git

thanks,
jirka


---
Jiri Olsa (6):
      tools: Add clean targets for tools directory
      tools bpf: Add *.cmd files clean up
      tools lockdep: Add *.cmd files clean up
      perf tools: Add missing sources in perf's MANIFEST
      perf build: Add feature-dump target
      perf build: Introduce FEATURES_DUMP make variable

 tools/Makefile             | 11 ++++++++++-
 tools/lib/bpf/Makefile     |  2 +-
 tools/lib/lockdep/Makefile |  2 +-
 tools/perf/MANIFEST        |  2 ++
 tools/perf/Makefile.perf   | 25 ++++++++++++++++++++++++-
 tools/perf/config/Makefile |  4 ++++
 6 files changed, 42 insertions(+), 4 deletions(-)

[PATCH 1/6] tools: Add clean targets for tools directory

[Jiri Olsa]

Adding missing clean targets for following tools directories:
  lib/bpf
  lib/subcmd
  build

This are now cleaned via 'make -C tools clean' command.

Reported-by: Arnaldo Carvalho de Melo <acme@kernel.org>
Link: http://lkml.kernel.org/n/tip-adxn5c48oe0gmjrjmq6we27q@git.kernel.org
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
 tools/Makefile | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/tools/Makefile b/tools/Makefile
index 0ba0df3b516f..4e8e10755e0d 100644
--- a/tools/Makefile
+++ b/tools/Makefile
@@ -127,6 +127,12 @@ liblockdep_clean:
 libapi_clean:
 	$(call descend,lib/api,clean)
 
+libbpf_clean:
+	$(call descend,lib/bpf,clean)
+
+libsubcmd_clean:
+	$(call descend,lib/subcmd,clean)
+
 perf_clean:
 	$(call descend,$(@:_clean=),clean)
 
@@ -142,9 +148,12 @@ tmon_clean:
 freefall_clean:
 	$(call descend,laptop/freefall,clean)
 
+build_clean:
+	$(call descend,build,clean)
+
 clean: acpi_clean cgroup_clean cpupower_clean hv_clean firewire_clean lguest_clean \
 		perf_clean selftests_clean turbostat_clean usb_clean virtio_clean \
 		vm_clean net_clean iio_clean x86_energy_perf_policy_clean tmon_clean \
-		freefall_clean
+		freefall_clean build_clean libbpf_clean libsubcmd_clean liblockdep_clean
 
 .PHONY: FORCE
-- 
2.4.3

[PATCH 2/6] tools bpf: Add *.cmd files clean up

[Jiri Olsa]

Add *.cmd files to be removed within clean target.

Reported-by: Arnaldo Carvalho de Melo <acme@kernel.org>
Link: http://lkml.kernel.org/n/tip-r93edbxrxaz4tnfua055cdeq@git.kernel.org
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
 tools/lib/bpf/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/lib/bpf/Makefile b/tools/lib/bpf/Makefile
index 919b71780710..84e0e986ade4 100644
--- a/tools/lib/bpf/Makefile
+++ b/tools/lib/bpf/Makefile
@@ -192,7 +192,7 @@ config-clean:
 	$(Q)$(MAKE) -C $(srctree)/tools/build/feature/ clean >/dev/null
 
 clean:
-	$(call QUIET_CLEAN, libbpf) $(RM) *.o *~ $(TARGETS) *.a *.so $(VERSION_FILES) .*.d \
+	$(call QUIET_CLEAN, libbpf) $(RM) *.o *~ $(TARGETS) *.a *.so $(VERSION_FILES) .*.d .*.cmd \
 		$(RM) LIBBPF-CFLAGS
 	$(call QUIET_CLEAN, core-gen) $(RM) $(OUTPUT)FEATURE-DUMP.libbpf
 
-- 
2.4.3

[PATCH 3/6] tools lockdep: Add *.cmd files clean up

[Jiri Olsa]

Add *.cmd files to be removed within clean target.

Reported-by: Arnaldo Carvalho de Melo <acme@kernel.org>
Link: http://lkml.kernel.org/n/tip-firafgayyatr3881jhnitiht@git.kernel.org
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
 tools/lib/lockdep/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/lib/lockdep/Makefile b/tools/lib/lockdep/Makefile
index 7e319afac78a..90d2baeb621a 100644
--- a/tools/lib/lockdep/Makefile
+++ b/tools/lib/lockdep/Makefile
@@ -149,7 +149,7 @@ install_lib: all_cmd
 install: install_lib
 
 clean:
-	$(RM) *.o *~ $(TARGETS) *.a *liblockdep*.so* $(VERSION_FILES) .*.d
+	$(RM) *.o *~ $(TARGETS) *.a *liblockdep*.so* $(VERSION_FILES) .*.d .*.cmd
 	$(RM) tags TAGS
 
 PHONY += force
-- 
2.4.3

[PATCH 4/6] perf tools: Add missing sources in perf's MANIFEST

[Jiri Olsa]

From: Jiri Olsa <jolsa@redhat.com>

Adding missing bitmap.[ch] sources to the MINIFEST file.

Link: http://lkml.kernel.org/n/tip-bkwplvnpk6s6a8zi1923dzuj@git.kernel.org
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
 tools/perf/MANIFEST | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/tools/perf/MANIFEST b/tools/perf/MANIFEST
index ddf922f93aa1..2e1fa2357528 100644
--- a/tools/perf/MANIFEST
+++ b/tools/perf/MANIFEST
@@ -28,6 +28,7 @@ tools/lib/string.c
 tools/lib/symbol/kallsyms.c
 tools/lib/symbol/kallsyms.h
 tools/lib/find_bit.c
+tools/lib/bitmap.c
 tools/include/asm/atomic.h
 tools/include/asm/barrier.h
 tools/include/asm/bug.h
@@ -57,6 +58,7 @@ tools/include/linux/rbtree_augmented.h
 tools/include/linux/string.h
 tools/include/linux/types.h
 tools/include/linux/err.h
+tools/include/linux/bitmap.h
 include/asm-generic/bitops/arch_hweight.h
 include/asm-generic/bitops/const_hweight.h
 include/asm-generic/bitops/fls64.h
-- 
2.4.3

[RFC 5/6] perf build: Add feature-dump target

[Jiri Olsa]

To provide FEATURE-DUMP into $(FEATURE_DUMP_COPY)
if defined, with no further action.

Get feature dump of the current build:
  $ make feature-dump
    BUILD:   Doing 'make -j4' parallel build

  Auto-detecting system features:
  ...                         dwarf: [ on  ]

  FEATURE-DUMP file available in FEATURE-DUMP

Get feature dump static build into /tmp/fd file:
  $ make feature-dump FEATURE_DUMP_COPY=/tmp/fd LDFLAGS=-static
    BUILD:   Doing 'make -j4' parallel build

  Auto-detecting system features:
  ...                         dwarf: [ OFF ]

  SNIP

  FEATURE-DUMP file copied into /tmp/fd

Suggested-by: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-xzqhfxw3euqmls3cve0ruuol@git.kernel.org
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
 tools/perf/Makefile.perf | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index 0a22407e1d7d..f758a72df1b3 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -611,6 +611,17 @@ clean: $(LIBTRACEEVENT)-clean $(LIBAPI)-clean $(LIBBPF)-clean $(LIBSUBCMD)-clean
 	$(python-clean)
 
 #
+# To provide FEATURE-DUMP into $(FEATURE_DUMP_COPY)
+# file if defined, with no further action.
+feature-dump:
+ifdef FEATURE_DUMP_COPY
+	@cp $(OUTPUT)FEATURE-DUMP $(FEATURE_DUMP_COPY)
+	@echo "FEATURE-DUMP file copied into $(FEATURE_DUMP_COPY)"
+else
+	@echo "FEATURE-DUMP file available in $(OUTPUT)FEATURE-DUMP"
+endif
+
+#
 # Trick: if ../../.git does not exist - we are building out of tree for example,
 # then force version regeneration:
 #
-- 
2.4.3

[RFC 6/6] perf build: Introduce FEATURES_DUMP make variable

[Jiri Olsa]

Introducing FEATURES_DUMP make variable to provide features
detection dump file and bypass the feature detection.

The intention is to use this during build tests to skip
repeated features detection, like:

Get feature dump static build into /tmp/fd file:
  $ make feature-dump FEATURE_DUMP_COPY=/tmp/fd LDFLAGS=-static
    BUILD:   Doing 'make -j4' parallel build

  Auto-detecting system features:
  ...                         dwarf: [ OFF ]

  SNIP

  FEATURE-DUMP file copied into /tmp/fd

Use /tmp/fd to build perf:
  $ make FEATURES_DUMP=/tmp/fd LDFLAGS=-static

  $ file perf
  perf: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), statically linked, for ...

Suggested-by: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-fhb47m6t18txuwrzu33is2bo@git.kernel.org
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
 tools/perf/Makefile.perf   | 14 +++++++++++++-
 tools/perf/config/Makefile |  4 ++++
 2 files changed, 17 insertions(+), 1 deletion(-)

diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index f758a72df1b3..5d34815c7ccb 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -77,6 +77,9 @@ include config/utilities.mak
 # Define NO_AUXTRACE if you do not want AUX area tracing support
 #
 # Define NO_LIBBPF if you do not want BPF support
+#
+# Define FEATURES_DUMP to provide features detection dump file
+# and bypass the feature detection
 
 # As per kernel Makefile, avoid funny character set dependencies
 unexport LC_ALL
@@ -166,6 +169,15 @@ ifeq ($(config),1)
 include config/Makefile
 endif
 
+# The FEATURE_DUMP_EXPORT holds location of the actual
+# FEATURE_DUMP file to be used to bypass feature detection
+# (for bpf or any other subproject)
+ifeq ($(FEATURES_DUMP),)
+FEATURE_DUMP_EXPORT := $(realpath $(OUTPUT)FEATURE-DUMP)
+else
+FEATURE_DUMP_EXPORT := $(FEATURES_DUMP)
+endif
+
 export prefix bindir sharedir sysconfdir DESTDIR
 
 # sparse is architecture-neutral, which means that we need to tell it
@@ -436,7 +448,7 @@ $(LIBAPI)-clean:
 	$(Q)$(MAKE) -C $(LIB_DIR) O=$(OUTPUT) clean >/dev/null
 
 $(LIBBPF): fixdep FORCE
-	$(Q)$(MAKE) -C $(BPF_DIR) O=$(OUTPUT) $(OUTPUT)libbpf.a FEATURES_DUMP=$(realpath $(OUTPUT)FEATURE-DUMP)
+	$(Q)$(MAKE) -C $(BPF_DIR) O=$(OUTPUT) $(OUTPUT)libbpf.a FEATURES_DUMP=$(FEATURE_DUMP_EXPORT)
 
 $(LIBBPF)-clean:
 	$(call QUIET_CLEAN, libbpf)
diff --git a/tools/perf/config/Makefile b/tools/perf/config/Makefile
index 254d06e39bea..5b2f12d6be3e 100644
--- a/tools/perf/config/Makefile
+++ b/tools/perf/config/Makefile
@@ -181,7 +181,11 @@ LDFLAGS += -Wl,-z,noexecstack
 
 EXTLIBS = -lpthread -lrt -lm -ldl
 
+ifeq ($(FEATURES_DUMP),)
 include $(srctree)/tools/build/Makefile.feature
+else
+include $(FEATURES_DUMP)
+endif
 
 ifeq ($(feature-stackprotector-all), 1)
   CFLAGS += -fstack-protector-all
-- 
2.4.3

Re: [RFC 6/6] perf build: Introduce FEATURES_DUMP make variable

[Wangnan (F)]

On 2016/1/11 18:54, Jiri Olsa wrote:
> Introducing FEATURES_DUMP make variable to provide features
> detection dump file and bypass the feature detection.
>
> The intention is to use this during build tests to skip
> repeated features detection, like:
>
> Get feature dump static build into /tmp/fd file:
>    $ make feature-dump FEATURE_DUMP_COPY=/tmp/fd LDFLAGS=-static
>      BUILD:   Doing 'make -j4' parallel build
>
>    Auto-detecting system features:
>    ...                         dwarf: [ OFF ]
>
>    SNIP
>
>    FEATURE-DUMP file copied into /tmp/fd
>
> Use /tmp/fd to build perf:
>    $ make FEATURES_DUMP=/tmp/fd LDFLAGS=-static
>
>    $ file perf
>    perf: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), statically linked, for ...
>
> Suggested-by: Wang Nan <wangnan0@huawei.com>
> Link: http://lkml.kernel.org/n/tip-fhb47m6t18txuwrzu33is2bo@git.kernel.org
> Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> ---

Thank you for these two patch. However I found a small limitation.
I must skip 'make clean' between two feature-dump, or the result is wrong.

For example:

$ make feature-dump LDFLAGS="-static"
   BUILD:   Doing 'make -j24' parallel build

Auto-detecting system features:
<SNIP>
...                          zlib: [ on  ]
...                          lzma: [ OFF ]     <--- looks good. I don't 
have static lzma library
...                     get_cpuid: [ on  ]
...                           bpf: [ on  ]


$ make feature-dump
   BUILD:   Doing 'make -j24' parallel build

Auto-detecting system features:
<SNIP>
...                          zlib: [ on  ]
...                          lzma: [ on  ]   <--- also good
...                     get_cpuid: [ on  ]
...                           bpf: [ on  ]


$ make feature-dump LDFLAGS="-static"
   BUILD:   Doing 'make -j24' parallel build

Auto-detecting system features:
<SNIP>
...                          zlib: [ on  ]
...                          lzma: [ on  ]     <--- Bad...
...                     get_cpuid: [ on  ]
...                           bpf: [ on  ]

Thank you.

[tip:perf/urgent] tools: Add clean targets for tools directory

[tip-bot for Jiri Olsa]

Commit-ID:  2f5a7f1d13df94b753794b69cc4c920f8bfb2128
Gitweb:     http://git.kernel.org/tip/2f5a7f1d13df94b753794b69cc4c920f8bfb2128
Author:     Jiri Olsa <jolsa@kernel.org>
AuthorDate: Mon, 11 Jan 2016 11:54:48 +0100
Committer:  Arnaldo Carvalho de Melo <acme@redhat.com>
CommitDate: Mon, 11 Jan 2016 12:07:38 -0300

tools: Add clean targets for tools directory

Adding missing clean targets for following tools directories:

  lib/bpf
  lib/subcmd
  build

This are now cleaned via 'make -C tools clean' command.

Reported-and-Tested-by: Arnaldo Carvalho de Melo <acme@kernel.org>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Cc: David Ahern <dsahern@gmail.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/r/1452509693-13452-2-git-send-email-jolsa@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/Makefile | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/tools/Makefile b/tools/Makefile
index 0ba0df3..4e8e107 100644
--- a/tools/Makefile
+++ b/tools/Makefile
@@ -127,6 +127,12 @@ liblockdep_clean:
 libapi_clean:
 	$(call descend,lib/api,clean)
 
+libbpf_clean:
+	$(call descend,lib/bpf,clean)
+
+libsubcmd_clean:
+	$(call descend,lib/subcmd,clean)
+
 perf_clean:
 	$(call descend,$(@:_clean=),clean)
 
@@ -142,9 +148,12 @@ tmon_clean:
 freefall_clean:
 	$(call descend,laptop/freefall,clean)
 
+build_clean:
+	$(call descend,build,clean)
+
 clean: acpi_clean cgroup_clean cpupower_clean hv_clean firewire_clean lguest_clean \
 		perf_clean selftests_clean turbostat_clean usb_clean virtio_clean \
 		vm_clean net_clean iio_clean x86_energy_perf_policy_clean tmon_clean \
-		freefall_clean
+		freefall_clean build_clean libbpf_clean libsubcmd_clean liblockdep_clean
 
 .PHONY: FORCE

[tip:perf/urgent] tools bpf: Add *.cmd files clean up

[tip-bot for Jiri Olsa]

Commit-ID:  22992a320807f8eb04b48407715aacbdedb00941
Gitweb:     http://git.kernel.org/tip/22992a320807f8eb04b48407715aacbdedb00941
Author:     Jiri Olsa <jolsa@kernel.org>
AuthorDate: Mon, 11 Jan 2016 11:54:49 +0100
Committer:  Arnaldo Carvalho de Melo <acme@redhat.com>
CommitDate: Mon, 11 Jan 2016 12:08:30 -0300

tools bpf: Add *.cmd files clean up

Add *.cmd files to be removed within clean target.

Reported-and-Tested-by: Arnaldo Carvalho de Melo <acme@kernel.org>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Cc: David Ahern <dsahern@gmail.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/r/1452509693-13452-3-git-send-email-jolsa@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/lib/bpf/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/lib/bpf/Makefile b/tools/lib/bpf/Makefile
index 919b717..84e0e98 100644
--- a/tools/lib/bpf/Makefile
+++ b/tools/lib/bpf/Makefile
@@ -192,7 +192,7 @@ config-clean:
 	$(Q)$(MAKE) -C $(srctree)/tools/build/feature/ clean >/dev/null
 
 clean:
-	$(call QUIET_CLEAN, libbpf) $(RM) *.o *~ $(TARGETS) *.a *.so $(VERSION_FILES) .*.d \
+	$(call QUIET_CLEAN, libbpf) $(RM) *.o *~ $(TARGETS) *.a *.so $(VERSION_FILES) .*.d .*.cmd \
 		$(RM) LIBBPF-CFLAGS
 	$(call QUIET_CLEAN, core-gen) $(RM) $(OUTPUT)FEATURE-DUMP.libbpf
 

[tip:perf/urgent] tools lockdep: Add *.cmd files clean up

[tip-bot for Jiri Olsa]

Commit-ID:  24ee9b57b95fd4d6d0b89a47c3a8f1b145569d18
Gitweb:     http://git.kernel.org/tip/24ee9b57b95fd4d6d0b89a47c3a8f1b145569d18
Author:     Jiri Olsa <jolsa@kernel.org>
AuthorDate: Mon, 11 Jan 2016 11:54:50 +0100
Committer:  Arnaldo Carvalho de Melo <acme@redhat.com>
CommitDate: Mon, 11 Jan 2016 12:09:05 -0300

tools lockdep: Add *.cmd files clean up

Add *.cmd files to be removed within clean target.

Reported-by: Arnaldo Carvalho de Melo <acme@kernel.org>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Cc: David Ahern <dsahern@gmail.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/r/1452509693-13452-4-git-send-email-jolsa@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/lib/lockdep/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/lib/lockdep/Makefile b/tools/lib/lockdep/Makefile
index 7e319af..90d2bae 100644
--- a/tools/lib/lockdep/Makefile
+++ b/tools/lib/lockdep/Makefile
@@ -149,7 +149,7 @@ install_lib: all_cmd
 install: install_lib
 
 clean:
-	$(RM) *.o *~ $(TARGETS) *.a *liblockdep*.so* $(VERSION_FILES) .*.d
+	$(RM) *.o *~ $(TARGETS) *.a *liblockdep*.so* $(VERSION_FILES) .*.d .*.cmd
 	$(RM) tags TAGS
 
 PHONY += force

[tip:perf/urgent] perf tools: Add missing sources to perf' s MANIFEST

[tip-bot for Jiri Olsa]

Commit-ID:  3238ba01c235e3e9e12fc7bc6ac3f7d513e26a31
Gitweb:     http://git.kernel.org/tip/3238ba01c235e3e9e12fc7bc6ac3f7d513e26a31
Author:     Jiri Olsa <jolsa@redhat.com>
AuthorDate: Mon, 11 Jan 2016 11:54:51 +0100
Committer:  Arnaldo Carvalho de Melo <acme@redhat.com>
CommitDate: Mon, 11 Jan 2016 12:09:25 -0300

perf tools: Add missing sources to perf's MANIFEST

Adding missing bitmap.[ch] sources to the MANIFEST file. Fixes building
'make perf-*-src-pkg' generated tarballs.

Reported-by: Wang Nan <wangnan0@huawei.com>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Cc: David Ahern <dsahern@gmail.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Fixes: 915b0882c310 ("tools lib: Move bitmap.[ch] from tools/perf/ to tools/{lib,include}/")
Link: http://lkml.kernel.org/r/1452509693-13452-5-git-send-email-jolsa@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/MANIFEST | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/tools/perf/MANIFEST b/tools/perf/MANIFEST
index ddf922f..2e1fa23 100644
--- a/tools/perf/MANIFEST
+++ b/tools/perf/MANIFEST
@@ -28,6 +28,7 @@ tools/lib/string.c
 tools/lib/symbol/kallsyms.c
 tools/lib/symbol/kallsyms.h
 tools/lib/find_bit.c
+tools/lib/bitmap.c
 tools/include/asm/atomic.h
 tools/include/asm/barrier.h
 tools/include/asm/bug.h
@@ -57,6 +58,7 @@ tools/include/linux/rbtree_augmented.h
 tools/include/linux/string.h
 tools/include/linux/types.h
 tools/include/linux/err.h
+tools/include/linux/bitmap.h
 include/asm-generic/bitops/arch_hweight.h
 include/asm-generic/bitops/const_hweight.h
 include/asm-generic/bitops/fls64.h

Re: [RFC 6/6] perf build: Introduce FEATURES_DUMP make variable

[Jiri Olsa]

On Tue, Jan 12, 2016 at 05:28:28PM +0800, Wangnan (F) wrote:

SNIP

> For example:
> 
> $ make feature-dump LDFLAGS="-static"
>   BUILD:   Doing 'make -j24' parallel build
> 
> Auto-detecting system features:
> <SNIP>
> ...                          zlib: [ on  ]
> ...                          lzma: [ OFF ]     <--- looks good. I don't have
> static lzma library
> ...                     get_cpuid: [ on  ]
> ...                           bpf: [ on  ]
> 
> 
> $ make feature-dump
>   BUILD:   Doing 'make -j24' parallel build
> 
> Auto-detecting system features:
> <SNIP>
> ...                          zlib: [ on  ]
> ...                          lzma: [ on  ]   <--- also good
> ...                     get_cpuid: [ on  ]
> ...                           bpf: [ on  ]
> 
> 
> $ make feature-dump LDFLAGS="-static"
>   BUILD:   Doing 'make -j24' parallel build
> 
> Auto-detecting system features:
> <SNIP>
> ...                          zlib: [ on  ]
> ...                          lzma: [ on  ]     <--- Bad...
> ...                     get_cpuid: [ on  ]
> ...                           bpf: [ on  ]
> 
> Thank you.

right, it's because feature is not under build framework and
compilation does not care about flags change..

I'll try to check on that.. does this patch help to improve
the current speed or do you need to solve this to get substantial
speedup?

thanks,
jirka

Re: [RFC 6/6] perf build: Introduce FEATURES_DUMP make variable

[Wangnan (F)]

On 2016/1/13 17:19, Jiri Olsa wrote:
> On Tue, Jan 12, 2016 at 05:28:28PM +0800, Wangnan (F) wrote:
>
> SNIP
>
>> For example:
>>
>> $ make feature-dump LDFLAGS="-static"
>>    BUILD:   Doing 'make -j24' parallel build
>>
>> Auto-detecting system features:
>> <SNIP>
>> ...                          zlib: [ on  ]
>> ...                          lzma: [ OFF ]     <--- looks good. I don't have
>> static lzma library
>> ...                     get_cpuid: [ on  ]
>> ...                           bpf: [ on  ]
>>
>>
>> $ make feature-dump
>>    BUILD:   Doing 'make -j24' parallel build
>>
>> Auto-detecting system features:
>> <SNIP>
>> ...                          zlib: [ on  ]
>> ...                          lzma: [ on  ]   <--- also good
>> ...                     get_cpuid: [ on  ]
>> ...                           bpf: [ on  ]
>>
>>
>> $ make feature-dump LDFLAGS="-static"
>>    BUILD:   Doing 'make -j24' parallel build
>>
>> Auto-detecting system features:
>> <SNIP>
>> ...                          zlib: [ on  ]
>> ...                          lzma: [ on  ]     <--- Bad...
>> ...                     get_cpuid: [ on  ]
>> ...                           bpf: [ on  ]
>>
>> Thank you.
> right, it's because feature is not under build framework and
> compilation does not care about flags change..
>
> I'll try to check on that.. does this patch help to improve
> the current speed or do you need to solve this to get substantial
> speedup?

I'm working on it. I have already done it, but still have some
problem. On some machine 'make_static' test target lost
'-liberty -lz' in its linker options. Seems something wrong in
this part:

ifeq ($(feature-libbfd), 1)
   EXTLIBS += -lbfd

   # call all detections now so we get correct
   # status in VF output
   $(call feature_check,liberty)
   $(call feature_check,liberty-z)
   $(call feature_check,cplus-demangle)

   ifeq ($(feature-liberty), 1)
     EXTLIBS += -liberty
   else
     ifeq ($(feature-liberty-z), 1)
       EXTLIBS += -liberty -lz
     endif
   endif
endif

On a 24 core machine the build-test takes about 16 mins. Faster than before:

real    16m7.642s
user    171m47.488s
sys    19m10.692s

Re: [RFC 6/6] perf build: Introduce FEATURES_DUMP make variable

[Jiri Olsa]

On Wed, Jan 13, 2016 at 05:46:13PM +0800, Wangnan (F) wrote:

SNIP

> >
> >I'll try to check on that.. does this patch help to improve
> >the current speed or do you need to solve this to get substantial
> >speedup?
> 
> I'm working on it. I have already done it, but still have some
> problem. On some machine 'make_static' test target lost
> '-liberty -lz' in its linker options. Seems something wrong in
> this part:
> 
> ifeq ($(feature-libbfd), 1)
>   EXTLIBS += -lbfd
> 
>   # call all detections now so we get correct
>   # status in VF output
>   $(call feature_check,liberty)
>   $(call feature_check,liberty-z)
>   $(call feature_check,cplus-demangle)
> 
>   ifeq ($(feature-liberty), 1)
>     EXTLIBS += -liberty
>   else
>     ifeq ($(feature-liberty-z), 1)
>       EXTLIBS += -liberty -lz
>     endif
>   endif
> endif

IIRC we had issues with this before.. different systems
provide those libs in different way, like:
  2cf9040714f3 perf tools: Fix bfd dependency libraries detection

you might want to check feature detection code if it gets
built properly on your system

jirka

Re: [RFC 6/6] perf build: Introduce FEATURES_DUMP make variable

[Wangnan (F)]

On 2016/1/13 18:03, Jiri Olsa wrote:
> On Wed, Jan 13, 2016 at 05:46:13PM +0800, Wangnan (F) wrote:
>
> SNIP
>
>>> I'll try to check on that.. does this patch help to improve
>>> the current speed or do you need to solve this to get substantial
>>> speedup?
>> I'm working on it. I have already done it, but still have some
>> problem. On some machine 'make_static' test target lost
>> '-liberty -lz' in its linker options. Seems something wrong in
>> this part:
>>
>> ifeq ($(feature-libbfd), 1)
>>    EXTLIBS += -lbfd
>>
>>    # call all detections now so we get correct
>>    # status in VF output
>>    $(call feature_check,liberty)
>>    $(call feature_check,liberty-z)
>>    $(call feature_check,cplus-demangle)
>>
>>    ifeq ($(feature-liberty), 1)
>>      EXTLIBS += -liberty
>>    else
>>      ifeq ($(feature-liberty-z), 1)
>>        EXTLIBS += -liberty -lz
>>      endif
>>    endif
>> endif
> IIRC we had issues with this before.. different systems
> provide those libs in different way, like:
>    2cf9040714f3 perf tools: Fix bfd dependency libraries detection
>
> you might want to check feature detection code if it gets
> built properly on your system

No this problem. There's no feature-libiberty and feature-libiberty-z in
output feature-dump file. This is the key reason.

> jirka

Re: [RFC 6/6] perf build: Introduce FEATURES_DUMP make variable

[Wangnan (F)]

On 2016/1/13 18:03, Jiri Olsa wrote:
> On Wed, Jan 13, 2016 at 05:46:13PM +0800, Wangnan (F) wrote:
>
> SNIP
>
>>> I'll try to check on that.. does this patch help to improve
>>> the current speed or do you need to solve this to get substantial
>>> speedup?

Finished.

On a 24 core machine:

Without this feature, a full build-test takes:

real    21m16.579s
user    176m12.989s
sys    20m54.950s

With this feature:

real    16m0.619s
user    172m53.896s
sys    19m22.777s

Speedup: 24.75%.

So this feature is really useful.

Thank you.

Re: [RFC 6/6] perf build: Introduce FEATURES_DUMP make variable

[Jiri Olsa]

On Wed, Jan 13, 2016 at 08:02:24PM +0800, Wangnan (F) wrote:
> 
> 
> On 2016/1/13 18:03, Jiri Olsa wrote:
> >On Wed, Jan 13, 2016 at 05:46:13PM +0800, Wangnan (F) wrote:
> >
> >SNIP
> >
> >>>I'll try to check on that.. does this patch help to improve
> >>>the current speed or do you need to solve this to get substantial
> >>>speedup?
> 
> Finished.
> 
> On a 24 core machine:
> 
> Without this feature, a full build-test takes:
> 
> real    21m16.579s
> user    176m12.989s
> sys    20m54.950s
> 
> With this feature:
> 
> real    16m0.619s
> user    172m53.896s
> sys    19m22.777s
> 
> Speedup: 24.75%.
> 
> So this feature is really useful.

looks great, thanks

jirka