perf tools build broken after v5.1-rc1

perf tools build broken after v5.1-rc1

[Vineet Gupta]

Hi,

When building perf for ARC (v5.1-rc2) I get the following

| In file included from bench/futex-hash.c:26:
| bench/futex.h: In function 'futex_wait':
| bench/futex.h:37:10: error: 'SYS_futex' undeclared (first use in this function);

git bisect led to 1a787fc5ba18ac767e635c58d06a0b46876184e3 ("tools headers uapi:
Sync copy of asm-generic/unistd.h with the kernel sources")

The issue is due to hunk below, rightly brought over from commit c8ce48f06503
("asm-generic: Make time32 syscall numbers optional")

+#if defined(__ARCH_WANT_TIME32_SYSCALLS) || __BITS_PER_LONG != 32
 #define __NR_futex 98
-__SC_COMP(__NR_futex, sys_futex, compat_sys_futex)
+__SC_3264(__NR_futex, sys_futex_time32, sys_futex)
+#endif

While __ARCH_WANT_TIME32_SYSCALLS does exist in
arch/arc/include/uapi/asm/unistd.h, but not inside tools, essentially we lack a
copy such as below which were sync'ed by commit above as well.

   tools/arch/arm64/include/uapi/asm/unistd.h
   tools/arch/x86/include/uapi/asm/unistd.h

A quick fix for ARC will be to create our own version but I presume all existing
arches using generic syscall abi are affected. Thoughts ? In lack of ideas I'll
send out a patch for ARC.

P.S. Why do we need the unistd.h duplication in tools directory, given it could
have used the in-tree unistd headers directly ?

Thx,
-Vineet

perf tools build broken after v5.1-rc1

[Arnaldo Carvalho de Melo]

Em Fri, Apr 19, 2019 at 04:32:58PM -0700, Vineet Gupta escreveu:
> When building perf for ARC (v5.1-rc2) I get the following
 
> | In file included from bench/futex-hash.c:26:
> | bench/futex.h: In function 'futex_wait':
> | bench/futex.h:37:10: error: 'SYS_futex' undeclared (first use in this function);
 
> git bisect led to 1a787fc5ba18ac767e635c58d06a0b46876184e3 ("tools headers uapi:
> Sync copy of asm-generic/unistd.h with the kernel sources")

Humm, I have to check why this:

[perfbuilder at quaco ~]$ podman images | grep ARC
docker.io/acmel/linux-perf-tools-build-fedora                24-x-ARC-uClibc          4c259582a8e6   5 weeks ago      846 MB
[perfbuilder at quaco ~]$

isn't catching this... :-\

FROM docker.io/fedora:24
MAINTAINER Arnaldo Carvalho de Melo <acme at kernel.org>
ENV TOOLCHAIN=arc_gnu_2017.09-rc2_prebuilt_uclibc_le_arc700_linux_install
ENV CROSS=arc-linux-
ENV SOURCEFILE=${TOOLCHAIN}.tar.gz
RUN dnf -y install make flex bison binutils gcc wget tar bzip2 bc findutils xz
RUN wget https://github.com/foss-for-synopsys-dwc-arc-processors/toolchain/releases/download/arc-2017.09-rc2/${SOURCEFILE}
<SNIP>
COPY rx_and_build.sh /
ENV EXTRA_MAKE_ARGS=NO_LIBBPF=1
ENV ARCH=arc
ENV CROSS_COMPILE=/${TOOLCHAIN}/bin/${CROSS}
ENV EXTRA_CFLAGS=-matomic

I have to update this to fedora 29 or 30 and get a newer toolchain...
 
> The issue is due to hunk below, rightly brought over from commit c8ce48f06503
> ("asm-generic: Make time32 syscall numbers optional")
 
> +#if defined(__ARCH_WANT_TIME32_SYSCALLS) || __BITS_PER_LONG != 32
>  #define __NR_futex 98
> -__SC_COMP(__NR_futex, sys_futex, compat_sys_futex)
> +__SC_3264(__NR_futex, sys_futex_time32, sys_futex)
> +#endif
 
> While __ARCH_WANT_TIME32_SYSCALLS does exist in
> arch/arc/include/uapi/asm/unistd.h, but not inside tools, essentially we lack a
> copy such as below which were sync'ed by commit above as well.
 
>    tools/arch/arm64/include/uapi/asm/unistd.h
>    tools/arch/x86/include/uapi/asm/unistd.h

So yeah, we need to grab those copies that define something that is used
conditionally by something we have in tools, I'm trying to cook up a
patch grabbing the missing tools/arch/*/include/uapi/asm/unistd.h files,
will they fire a container test build that should cover lots of the
archs.

thanks for the report, will get back here with a report of the results
or else you'll get the patch flying to Ingo for tip/perf/urgent.
 
> A quick fix for ARC will be to create our own version but I presume all existing
> arches using generic syscall abi are affected. Thoughts ? In lack of ideas I'll
> send out a patch for ARC.
> 
> P.S. Why do we need the unistd.h duplication in tools directory, given it could
> have used the in-tree unistd headers directly ?

I have to write down the explanation and have it in a file, but we can't
use anything in the kernel from outside tools/ to avoid adding a burden
to kernel developers that would then have to make sure that the changes
that they make outside tools/ don't break things living there.

- Arnaldo

perf tools build broken after v5.1-rc1

[Arnaldo Carvalho de Melo]

Em Mon, Apr 22, 2019 at 12:20:27PM -0300, Arnaldo Carvalho de Melo escreveu:
> Em Fri, Apr 19, 2019 at 04:32:58PM -0700, Vineet Gupta escreveu:
> > When building perf for ARC (v5.1-rc2) I get the following
>  
> > | In file included from bench/futex-hash.c:26:
> > | bench/futex.h: In function 'futex_wait':
> > | bench/futex.h:37:10: error: 'SYS_futex' undeclared (first use in this function);
>  
> > git bisect led to 1a787fc5ba18ac767e635c58d06a0b46876184e3 ("tools headers uapi:
> > Sync copy of asm-generic/unistd.h with the kernel sources")
> 
> Humm, I have to check why this:
> 
> [perfbuilder at quaco ~]$ podman images | grep ARC
> docker.io/acmel/linux-perf-tools-build-fedora                24-x-ARC-uClibc          4c259582a8e6   5 weeks ago      846 MB
> [perfbuilder at quaco ~]$
> 
> isn't catching this... :-\
> 
> FROM docker.io/fedora:24
> MAINTAINER Arnaldo Carvalho de Melo <acme at kernel.org>
> ENV TOOLCHAIN=arc_gnu_2017.09-rc2_prebuilt_uclibc_le_arc700_linux_install
> ENV CROSS=arc-linux-
> ENV SOURCEFILE=${TOOLCHAIN}.tar.gz
> RUN dnf -y install make flex bison binutils gcc wget tar bzip2 bc findutils xz
> RUN wget https://github.com/foss-for-synopsys-dwc-arc-processors/toolchain/releases/download/arc-2017.09-rc2/${SOURCEFILE}
> <SNIP>
> COPY rx_and_build.sh /
> ENV EXTRA_MAKE_ARGS=NO_LIBBPF=1
> ENV ARCH=arc
> ENV CROSS_COMPILE=/${TOOLCHAIN}/bin/${CROSS}
> ENV EXTRA_CFLAGS=-matomic

So, now I have a libnuma crossbuilt in this container that allows me to
build a ARC perf binary linked with zlib and numactl-devel, but only
after I applied the fix below.

Can you please provide the feature detection header in the build? I.e.
what I have with my ARC cross build container right now, after applying
the patch below is:

[perfbuilder at 60d5802468f6 perf]$ make $EXTRA_MAKE_ARGS ARCH=$ARCH CROSS_COMPILE=$CROSS_COMPILE EXTRA_CFLAGS="$EXTRA_CFLAGS" -C /git/perf/tools/perf O=/tmp/build/perf
make: Entering directory '/git/perf/tools/perf'
  BUILD:   Doing 'make -j8' parallel build
sh: line 0: command: -c: invalid option
command: usage: command [-pVv] command [arg ...]

Auto-detecting system features:
...                         dwarf: [ OFF ]
...            dwarf_getlocations: [ OFF ]
...                         glibc: [ on  ]
...                          gtk2: [ OFF ]
...                      libaudit: [ OFF ]
...                        libbfd: [ OFF ]
...                        libelf: [ OFF ]
...                       libnuma: [ on  ]
...        numa_num_possible_cpus: [ on  ]
...                       libperl: [ OFF ]
...                     libpython: [ OFF ]
...                      libslang: [ OFF ]
...                     libcrypto: [ OFF ]
...                     libunwind: [ OFF ]
...            libdw-dwarf-unwind: [ OFF ]
...                          zlib: [ OFF ]
...                          lzma: [ OFF ]
...                     get_cpuid: [ OFF ]
...                           bpf: [ on  ]
...                        libaio: [ OFF ]
...        disassembler-four-args: [ OFF ]

Makefile.config:339: No libelf found. Disables 'probe' tool, jvmti and BPF support in 'perf record'. Please install libelf-dev, libelf-devel or elfutils-libelf-devel
Makefile.config:473: No sys/sdt.h found, no SDT events are defined, please install systemtap-sdt-devel or systemtap-sdt-dev
Makefile.config:548: Disabling post unwind, no support found.
Makefile.config:601: No libaudit.h found, disables 'trace' tool, please install audit-libs-devel or libaudit-dev
Makefile.config:613: No libcrypto.h found, disables jitted code injection, please install openssl-devel or libssl-dev
Makefile.config:628: slang not found, disables TUI support. Please install slang-devel, libslang-dev or libslang2-dev
Makefile.config:642: GTK2 not found, disables GTK2 support. Please install gtk2-devel or libgtk2.0-dev
Makefile.config:669: Missing perl devel files. Disabling perl scripting support, please install perl-ExtUtils-Embed/libperl-dev
Makefile.config:696: No python interpreter was found: disables Python support - please install python-devel/python-dev
Makefile.config:785: No liblzma found, disables xz kernel module decompression, please install xz-devel/liblzma-dev
Makefile.config:853: No libbabeltrace found, disables 'perf data' CTF format support, please install libbabeltrace-dev[el]/libbabeltrace-ctf-dev
make: Leaving directory '/git/perf/tools/perf'
[perfbuilder at 60d5802468f6 perf]$ export
declare -x ARCH="arc"
declare -x CROSS="arc-linux-"
declare -x CROSS_COMPILE="/arc_gnu_2019.03-rc1_prebuilt_uclibc_le_archs_linux_install/bin/arc-linux-"
declare -x DISTTAG="f30container"
declare -x EXTRA_CFLAGS="-matomic"
declare -x EXTRA_MAKE_ARGS="NO_LIBBPF=1"
declare -x FBR="f30"
declare -x FGC="f30"
declare -x NUMACTL_SOURCEFILE="github.com-numactl-numactl-archive-master-2019-04-25.zip"
declare -x PATH="/home/perfbuilder/.local/bin:/home/perfbuilder/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
declare -x SOURCEFILE="arc_gnu_2019.03-rc1_prebuilt_uclibc_le_archs_linux_install.tar.gz"
declare -x TOOLCHAIN="arc_gnu_2019.03-rc1_prebuilt_uclibc_le_archs_linux_install"
declare -x container="podman"
[perfbuilder at 60d5802468f6 perf]$ 

commit dd617baf28925cf72fce309232ad9178452e75ab
Author: Arnaldo Carvalho de Melo <acme at redhat.com>
Date:   Thu Apr 25 18:36:51 2019 -0300

    perf bench numa: Add define for RUSAGE_THREAD if not present
    
    While cross building perf to the ARC architecture on a fedora 30 host,
    we were failing with:
    
          CC       /tmp/build/perf/bench/numa.o
      bench/numa.c: In function ?worker_thread?:
      bench/numa.c:1261:12: error: ?RUSAGE_THREAD? undeclared (first use in this function); did you mean ?SIGEV_THREAD??
        getrusage(RUSAGE_THREAD, &rusage);
                  ^~~~~~~~~~~~~
                  SIGEV_THREAD
      bench/numa.c:1261:12: note: each undeclared identifier is reported only once for each function it appears in
    
    [perfbuilder at 60d5802468f6 perf]$ /arc_gnu_2019.03-rc1_prebuilt_uclibc_le_archs_linux_install/bin/arc-linux-gcc --version | head -1
    arc-linux-gcc (ARCv2 ISA Linux uClibc toolchain 2019.03-rc1) 8.3.1 20190225
    [perfbuilder at 60d5802468f6 perf]$
    
    Trying to reproduce a report by Vineet, I noticed that, with just
    cross-built zlib and numactl libraries, I ended up with the above
    failure.
    
    So, since RUSAGE_THREAD is available as a define, check for that and
    numactl libraries, I ended up with the above failure.
    
    So, since RUSAGE_THREAD is available as a define in the system headers,
    check if it is defined in the 'perf bench numa' sources and define it if
    not.
    
    Now it builds and I have to figure out if the problem reported by Vineet
    only takes place if we have libelf or some other library available.
    
    Cc: Arnd Bergmann <arnd at arndb.de>
    Cc: Jiri Olsa <jolsa at kernel.org>
    Cc: linux-snps-arc at lists.infradead.org
    Cc: Namhyung Kim <namhyung at kernel.org>
    Cc: Vineet Gupta <Vineet.Gupta1 at synopsys.com>
    Link: https://lkml.kernel.org/n/tip-2wb4r1gir9xrevbpq7qp0amk at git.kernel.org
    Signed-off-by: Arnaldo Carvalho de Melo <acme at redhat.com>

diff --git a/tools/perf/bench/numa.c b/tools/perf/bench/numa.c
index 98ad783efc69..a7784554a80d 100644
--- a/tools/perf/bench/numa.c
+++ b/tools/perf/bench/numa.c
@@ -39,6 +39,10 @@
 #include <numa.h>
 #include <numaif.h>
 
+#ifndef RUSAGE_THREAD
+# define RUSAGE_THREAD 1
+#endif
+
 /*
  * Regular printout to the terminal, supressed if -q is specified:
  */

perf tools build broken after v5.1-rc1

[Arnaldo Carvalho de Melo]

Em Thu, Apr 25, 2019 at 06:48:00PM -0300, Arnaldo Carvalho de Melo escreveu:
> Em Mon, Apr 22, 2019 at 12:20:27PM -0300, Arnaldo Carvalho de Melo escreveu:
> > Em Fri, Apr 19, 2019 at 04:32:58PM -0700, Vineet Gupta escreveu:
> > > When building perf for ARC (v5.1-rc2) I get the following
> >  
> > > | In file included from bench/futex-hash.c:26:
> > > | bench/futex.h: In function 'futex_wait':
> > > | bench/futex.h:37:10: error: 'SYS_futex' undeclared (first use in this function);
> >  
> > > git bisect led to 1a787fc5ba18ac767e635c58d06a0b46876184e3 ("tools headers uapi:
> > > Sync copy of asm-generic/unistd.h with the kernel sources")
> > 
> > Humm, I have to check why this:
> > 
> > [perfbuilder at quaco ~]$ podman images | grep ARC
> > docker.io/acmel/linux-perf-tools-build-fedora                24-x-ARC-uClibc          4c259582a8e6   5 weeks ago      846 MB
> > [perfbuilder at quaco ~]$
> > 
> > isn't catching this... :-\
> > 
> > FROM docker.io/fedora:24
> > MAINTAINER Arnaldo Carvalho de Melo <acme at kernel.org>
> > ENV TOOLCHAIN=arc_gnu_2017.09-rc2_prebuilt_uclibc_le_arc700_linux_install
> > ENV CROSS=arc-linux-
> > ENV SOURCEFILE=${TOOLCHAIN}.tar.gz
> > RUN dnf -y install make flex bison binutils gcc wget tar bzip2 bc findutils xz
> > RUN wget https://github.com/foss-for-synopsys-dwc-arc-processors/toolchain/releases/download/arc-2017.09-rc2/${SOURCEFILE}
> > <SNIP>
> > COPY rx_and_build.sh /
> > ENV EXTRA_MAKE_ARGS=NO_LIBBPF=1
> > ENV ARCH=arc
> > ENV CROSS_COMPILE=/${TOOLCHAIN}/bin/${CROSS}
> > ENV EXTRA_CFLAGS=-matomic
> 
> So, now I have a libnuma crossbuilt in this container that allows me to
> build a ARC perf binary linked with zlib and numactl-devel, but only
> after I applied the fix below.
> 
> Can you please provide the feature detection header in the build? I.e.
> what I have with my ARC cross build container right now, after applying
> the patch below is:

So, switched from the uCLibc build system to the glibc based one, so
that I could get elfutils building (it needs argp that isn't available
with uCLibc) I have it reproduced, see below, now I'm testing with the
fix, which is to grab the unistd.h files you noticed missing for the
arches that are present in tools/arch/, will post results soon

[perfbuilder at f1798ef8e433 perf]$ make $EXTRA_MAKE_ARGS ARCH=$ARCH CROSS_COMPILE=$CROSS_COMPILE EXTRA_CFLAGS="$EXTRA_CFLAGS" -C /git/perf/tools/perf O=/tmp/build/perf
make: Entering directory '/git/perf/tools/perf'
  BUILD:   Doing 'make -j8' parallel build
  HOSTCC   /tmp/build/perf/fixdep.o
  HOSTLD   /tmp/build/perf/fixdep-in.o
  LINK     /tmp/build/perf/fixdep
sh: line 0: command: -c: invalid option
command: usage: command [-pVv] command [arg ...]

Auto-detecting system features:
...                         dwarf: [ on  ]
...            dwarf_getlocations: [ on  ]
...                         glibc: [ on  ]
...                          gtk2: [ OFF ]
...                      libaudit: [ OFF ]
...                        libbfd: [ OFF ]
...                        libelf: [ on  ]
...                       libnuma: [ on  ]
...        numa_num_possible_cpus: [ on  ]
...                       libperl: [ OFF ]
...                     libpython: [ OFF ]
...                      libslang: [ OFF ]
...                     libcrypto: [ OFF ]
...                     libunwind: [ OFF ]
...            libdw-dwarf-unwind: [ on  ]
...                          zlib: [ on  ]
...                          lzma: [ OFF ]
...                     get_cpuid: [ OFF ]
...                           bpf: [ on  ]
...                        libaio: [ on  ]
...        disassembler-four-args: [ OFF ]

Makefile.config:441: DWARF register mappings have not been defined for architecture arc, DWARF support disabled
Makefile.config:473: No sys/sdt.h found, no SDT events are defined, please install systemtap-sdt-devel or systemtap-sdt-dev
Makefile.config:548: Disabling post unwind, no support found.
Makefile.config:601: No libaudit.h found, disables 'trace' tool, please install audit-libs-devel or libaudit-dev
Makefile.config:613: No libcrypto.h found, disables jitted code injection, please install openssl-devel or libssl-dev
Makefile.config:628: slang not found, disables TUI support. Please install slang-devel, libslang-dev or libslang2-dev
Makefile.config:642: GTK2 not found, disables GTK2 support. Please install gtk2-devel or libgtk2.0-dev
Makefile.config:669: Missing perl devel files. Disabling perl scripting support, please install perl-ExtUtils-Embed/libperl-dev
Makefile.config:696: No python interpreter was found: disables Python support - please install python-devel/python-dev
Makefile.config:754: No bfd.h/libbfd found, please install binutils-dev[el]/zlib-static/libiberty-dev to gain symbol demangling
Makefile.config:785: No liblzma found, disables xz kernel module decompression, please install xz-devel/liblzma-dev
Makefile.config:853: No libbabeltrace found, disables 'perf data' CTF format support, please install libbabeltrace-dev[el]/libbabeltrace-ctf-dev
Makefile.config:897: No openjdk development package found, please install JDK package, e.g. openjdk-8-jdk, java-1.8.0-openjdk-devel
  GEN      /tmp/build/perf/common-cmds.h
  CC       /tmp/build/perf/exec-cmd.o
  CC       /tmp/build/perf/help.o
  MKDIR    /tmp/build/perf/fd/
  CC       /tmp/build/perf/fd/array.o
  MKDIR    /tmp/build/perf/fs/
  CC       /tmp/build/perf/event-parse.o
  CC       /tmp/build/perf/pager.o
  LD       /tmp/build/perf/fd/libapi-in.o
  CC       /tmp/build/perf/cpu.o
  MKDIR    /tmp/build/perf/fs/
  CC       /tmp/build/perf/fs/fs.o
  CC       /tmp/build/perf/debug.o
  CC       /tmp/build/perf/fs/tracing_path.o
  CC       /tmp/build/perf/event-plugin.o
  CC       /tmp/build/perf/parse-options.o
  CC       /tmp/build/perf/str_error_r.o
  CC       /tmp/build/perf/trace-seq.o
  CC       /tmp/build/perf/run-command.o
  LD       /tmp/build/perf/fs/libapi-in.o
  CC       /tmp/build/perf/parse-filter.o
  LD       /tmp/build/perf/libapi-in.o
  AR       /tmp/build/perf/libapi.a
  CC       /tmp/build/perf/sigchain.o
  CC       /tmp/build/perf/parse-utils.o
  CC       /tmp/build/perf/kbuffer-parse.o
  CC       /tmp/build/perf/tep_strerror.o
  CC       /tmp/build/perf/subcmd-config.o
  CC       /tmp/build/perf/event-parse-api.o
  MKDIR    /tmp/build/perf/pmu-events/
  MKDIR    /tmp/build/perf/pmu-events/
  LD       /tmp/build/perf/libsubcmd-in.o
  HOSTCC   /tmp/build/perf/pmu-events/json.o
  HOSTCC   /tmp/build/perf/pmu-events/jsmn.o
  LD       /tmp/build/perf/libtraceevent-in.o
  LINK     /tmp/build/perf/libtraceevent.a
  AR       /tmp/build/perf/libsubcmd.a
  MKDIR    /tmp/build/perf/pmu-events/
  HOSTCC   /tmp/build/perf/pmu-events/jevents.o
  CC       /tmp/build/perf/plugin_kvm.o
  CC       /tmp/build/perf/plugin_kmem.o
  CC       /tmp/build/perf/plugin_jbd2.o
  CC       /tmp/build/perf/plugin_hrtimer.o
  CC       /tmp/build/perf/plugin_mac80211.o
  HOSTLD   /tmp/build/perf/pmu-events/jevents-in.o
  CC       /tmp/build/perf/plugin_sched_switch.o
  LD       /tmp/build/perf/plugin_kmem-in.o
  LD       /tmp/build/perf/plugin_jbd2-in.o
  LD       /tmp/build/perf/plugin_mac80211-in.o
  LD       /tmp/build/perf/plugin_hrtimer-in.o
  LINK     /tmp/build/perf/pmu-events/jevents
  LD       /tmp/build/perf/plugin_kvm-in.o
  CC       /tmp/build/perf/plugin_xen.o
  CC       /tmp/build/perf/plugin_function.o
  CC       /tmp/build/perf/plugin_scsi.o
  PERF_VERSION = 5.1.0-rc6
  CC       /tmp/build/perf/plugin_cfg80211.o
  GEN      perf-archive
  GEN      perf-with-kcore
  LD       /tmp/build/perf/plugin_sched_switch-in.o
  GEN      /tmp/build/perf/pmu-events/pmu-events.c
  LINK     /tmp/build/perf/plugin_jbd2.so
  LINK     /tmp/build/perf/plugin_hrtimer.so
  CC       /tmp/build/perf/pmu-events/pmu-events.o
  LD       /tmp/build/perf/plugin_xen-in.o
  LD       /tmp/build/perf/plugin_cfg80211-in.o
  LD       /tmp/build/perf/plugin_function-in.o
  LD       /tmp/build/perf/plugin_scsi-in.o
  LINK     /tmp/build/perf/plugin_kmem.so
  LINK     /tmp/build/perf/plugin_kvm.so
  LINK     /tmp/build/perf/plugin_mac80211.so
  LINK     /tmp/build/perf/plugin_sched_switch.so
  LINK     /tmp/build/perf/plugin_function.so
  LINK     /tmp/build/perf/plugin_xen.so
  LD       /tmp/build/perf/pmu-events/pmu-events-in.o
  LINK     /tmp/build/perf/plugin_scsi.so
  LINK     /tmp/build/perf/plugin_cfg80211.so
  GEN      /tmp/build/perf/libtraceevent-dynamic-list
  LD       /tmp/build/perf/libperf-in.o
  CC       /tmp/build/perf/builtin-config.o
  CC       /tmp/build/perf/builtin-bench.o
  CC       /tmp/build/perf/builtin-annotate.o
  CC       /tmp/build/perf/builtin-diff.o
  CC       /tmp/build/perf/builtin-ftrace.o
  CC       /tmp/build/perf/builtin-evlist.o
  CC       /tmp/build/perf/builtin-help.o
  CC       /tmp/build/perf/builtin-sched.o
  CC       /tmp/build/perf/builtin-buildid-list.o
  CC       /tmp/build/perf/builtin-buildid-cache.o
  CC       /tmp/build/perf/builtin-kallsyms.o
  CC       /tmp/build/perf/builtin-list.o
  CC       /tmp/build/perf/builtin-record.o
  CC       /tmp/build/perf/builtin-report.o
  CC       /tmp/build/perf/builtin-stat.o
  CC       /tmp/build/perf/builtin-timechart.o
  CC       /tmp/build/perf/builtin-top.o
  CC       /tmp/build/perf/builtin-script.o
  CC       /tmp/build/perf/builtin-kmem.o
  CC       /tmp/build/perf/builtin-lock.o
  CC       /tmp/build/perf/builtin-kvm.o
  CC       /tmp/build/perf/builtin-inject.o
  CC       /tmp/build/perf/builtin-mem.o
  CC       /tmp/build/perf/builtin-data.o
  CC       /tmp/build/perf/builtin-version.o
  CC       /tmp/build/perf/builtin-probe.o
  CC       /tmp/build/perf/builtin-c2c.o
  MKDIR    /tmp/build/perf/bench/
  MKDIR    /tmp/build/perf/tests/
  CC       /tmp/build/perf/bench/sched-messaging.o
  MKDIR    /tmp/build/perf/bench/
  CC       /tmp/build/perf/tests/builtin-test.o
  CC       /tmp/build/perf/bench/sched-pipe.o
  CC       /tmp/build/perf/bench/mem-functions.o
  CC       /tmp/build/perf/bench/futex-hash.o
  CC       /tmp/build/perf/bench/futex-wake.o
In file included from bench/futex-hash.c:26:
bench/futex.h: In function ?futex_wait?:
bench/futex.h:37:10: error: ?SYS_futex? undeclared (first use in this function); did you mean ?SYS_tee??
  syscall(SYS_futex, uaddr, op | opflags, val, timeout, uaddr2, val3)
          ^~~~~~~~~
bench/futex.h:46:9: note: in expansion of macro ?futex?
  return futex(uaddr, FUTEX_WAIT, val, timeout, NULL, 0, opflags);
         ^~~~~
bench/futex.h:37:10: note: each undeclared identifier is reported only once for each function it appears in
  syscall(SYS_futex, uaddr, op | opflags, val, timeout, uaddr2, val3)
          ^~~~~~~~~
bench/futex.h:46:9: note: in expansion of macro ?futex?
  return futex(uaddr, FUTEX_WAIT, val, timeout, NULL, 0, opflags);
         ^~~~~
bench/futex.h: In function ?futex_wake?:
bench/futex.h:37:10: error: ?SYS_futex? undeclared (first use in this function); did you mean ?SYS_tee??
  syscall(SYS_futex, uaddr, op | opflags, val, timeout, uaddr2, val3)
          ^~~~~~~~~
bench/futex.h:56:9: note: in expansion of macro ?futex?
  return futex(uaddr, FUTEX_WAKE, nr_wake, NULL, NULL, 0, opflags);
         ^~~~~
bench/futex.h: In function ?futex_lock_pi?:
bench/futex.h:37:10: error: ?SYS_futex? undeclared (first use in this function); did you mean ?SYS_tee??
  syscall(SYS_futex, uaddr, op | opflags, val, timeout, uaddr2, val3)
          ^~~~~~~~~
bench/futex.h:65:9: note: in expansion of macro ?futex?
  return futex(uaddr, FUTEX_LOCK_PI, 0, timeout, NULL, 0, opflags);
         ^~~~~
bench/futex.h: In function ?futex_unlock_pi?:
bench/futex.h:37:10: error: ?SYS_futex? undeclared (first use in this function); did you mean ?SYS_tee??
  syscall(SYS_futex, uaddr, op | opflags, val, timeout, uaddr2, val3)
          ^~~~~~~~~
bench/futex.h:74:9: note: in expansion of macro ?futex?
  return futex(uaddr, FUTEX_UNLOCK_PI, 0, NULL, NULL, 0, opflags);
         ^~~~~
bench/futex.h: In function ?futex_cmp_requeue?:
bench/futex.h:37:10: error: ?SYS_futex? undeclared (first use in this function); did you mean ?SYS_tee??
  syscall(SYS_futex, uaddr, op | opflags, val, timeout, uaddr2, val3)
          ^~~~~~~~~
bench/futex.h:86:9: note: in expansion of macro ?futex?
  return futex(uaddr, FUTEX_CMP_REQUEUE, nr_wake, nr_requeue, uaddr2,
         ^~~~~
bench/futex.h: In function ?futex_wait?:
bench/futex.h:47:1: error: control reaches end of non-void function [-Werror=return-type]
 }
 ^
cc1: all warnings being treated as errors
mv: cannot stat '/tmp/build/perf/bench/.futex-hash.o.tmp': No such file or directory
make[4]: *** [/git/perf/tools/build/Makefile.build:96: /tmp/build/perf/bench/futex-hash.o] Error 1
make[4]: *** Waiting for unfinished jobs....
  MKDIR    /tmp/build/perf/tests/
  MKDIR    /tmp/build/perf/arch/
  MKDIR    /tmp/build/perf/util/
  CC       /tmp/build/perf/tests/parse-events.o
  CC       /tmp/build/perf/util/annotate.o
  CC       /tmp/build/perf/arch/common.o
In file included from bench/futex-wake.c:24:
bench/futex.h: In function ?futex_wait?:
bench/futex.h:37:10: error: ?SYS_futex? undeclared (first use in this function); did you mean ?SYS_tee??
  syscall(SYS_futex, uaddr, op | opflags, val, timeout, uaddr2, val3)
          ^~~~~~~~~
bench/futex.h:46:9: note: in expansion of macro ?futex?
  return futex(uaddr, FUTEX_WAIT, val, timeout, NULL, 0, opflags);
         ^~~~~
bench/futex.h:37:10: note: each undeclared identifier is reported only once for each function it appears in
  syscall(SYS_futex, uaddr, op | opflags, val, timeout, uaddr2, val3)
          ^~~~~~~~~
bench/futex.h:46:9: note: in expansion of macro ?futex?
  return futex(uaddr, FUTEX_WAIT, val, timeout, NULL, 0, opflags);
         ^~~~~
bench/futex.h: In function ?futex_wake?:
bench/futex.h:37:10: error: ?SYS_futex? undeclared (first use in this function); did you mean ?SYS_tee??
  syscall(SYS_futex, uaddr, op | opflags, val, timeout, uaddr2, val3)
          ^~~~~~~~~
bench/futex.h:56:9: note: in expansion of macro ?futex?
  return futex(uaddr, FUTEX_WAKE, nr_wake, NULL, NULL, 0, opflags);
         ^~~~~
bench/futex.h: In function ?futex_lock_pi?:
bench/futex.h:37:10: error: ?SYS_futex? undeclared (first use in this function); did you mean ?SYS_tee??
  syscall(SYS_futex, uaddr, op | opflags, val, timeout, uaddr2, val3)
          ^~~~~~~~~
bench/futex.h:65:9: note: in expansion of macro ?futex?
  return futex(uaddr, FUTEX_LOCK_PI, 0, timeout, NULL, 0, opflags);
         ^~~~~
bench/futex.h: In function ?futex_unlock_pi?:
bench/futex.h:37:10: error: ?SYS_futex? undeclared (first use in this function); did you mean ?SYS_tee??
  syscall(SYS_futex, uaddr, op | opflags, val, timeout, uaddr2, val3)
          ^~~~~~~~~
bench/futex.h:74:9: note: in expansion of macro ?futex?
  return futex(uaddr, FUTEX_UNLOCK_PI, 0, NULL, NULL, 0, opflags);
         ^~~~~
bench/futex.h: In function ?futex_cmp_requeue?:
bench/futex.h:37:10: error: ?SYS_futex? undeclared (first use in this function); did you mean ?SYS_tee??
  syscall(SYS_futex, uaddr, op | opflags, val, timeout, uaddr2, val3)
          ^~~~~~~~~
bench/futex.h:86:9: note: in expansion of macro ?futex?
  return futex(uaddr, FUTEX_CMP_REQUEUE, nr_wake, nr_requeue, uaddr2,
         ^~~~~
bench/futex.h: In function ?futex_wait?:
bench/futex.h:47:1: error: control reaches end of non-void function [-Werror=return-type]
 }
 ^
bench/futex.h: In function ?futex_wake?:
bench/futex.h:57:1: error: control reaches end of non-void function [-Werror=return-type]
 }
 ^
cc1: all warnings being treated as errors
  MKDIR    /tmp/build/perf/util/
mv: cannot stat '/tmp/build/perf/bench/.futex-wake.o.tmp': No such file or directory
make[4]: *** [/git/perf/tools/build/Makefile.build:96: /tmp/build/perf/bench/futex-wake.o] Error 1
make[3]: *** [/git/perf/tools/build/Makefile.build:139: bench] Error 2
make[3]: *** Waiting for unfinished jobs....
  CC       /tmp/build/perf/util/block-range.o
  CC       /tmp/build/perf/util/build-id.o
  MKDIR    /tmp/build/perf/ui/
  CC       /tmp/build/perf/ui/setup.o
  CC       /tmp/build/perf/util/config.o
  MKDIR    /tmp/build/perf/ui/
  MKDIR    /tmp/build/perf/arch/arc/
  CC       /tmp/build/perf/ui/progress.o
  CC       /tmp/build/perf/ui/helpline.o
  LD       /tmp/build/perf/arch/arc/perf-in.o
  LD       /tmp/build/perf/arch/perf-in.o
  CC       /tmp/build/perf/ui/util.o
  CC       /tmp/build/perf/ui/hist.o
  CC       /tmp/build/perf/tests/dso-data.o
  CC       /tmp/build/perf/util/ctype.o
  MKDIR    /tmp/build/perf/ui/stdio/
  CC       /tmp/build/perf/ui/stdio/hist.o
  CC       /tmp/build/perf/tests/attr.o
  CC       /tmp/build/perf/tests/vmlinux-kallsyms.o
  CC       /tmp/build/perf/util/db-export.o
  CC       /tmp/build/perf/tests/openat-syscall.o
  CC       /tmp/build/perf/tests/openat-syscall-all-cpus.o
  CC       /tmp/build/perf/tests/openat-syscall-tp-fields.o
  CC       /tmp/build/perf/util/env.o
  CC       /tmp/build/perf/util/event.o
  CC       /tmp/build/perf/util/evlist.o
  CC       /tmp/build/perf/tests/mmap-basic.o
  CC       /tmp/build/perf/util/evsel.o
  CC       /tmp/build/perf/util/evsel_fprintf.o
  CC       /tmp/build/perf/util/find_bit.o
  CC       /tmp/build/perf/util/get_current_dir_name.o
  CC       /tmp/build/perf/util/kallsyms.o
  CC       /tmp/build/perf/util/levenshtein.o
  CC       /tmp/build/perf/tests/perf-record.o
  CC       /tmp/build/perf/util/llvm-utils.o
  CC       /tmp/build/perf/tests/evsel-roundtrip-name.o
  CC       /tmp/build/perf/tests/evsel-tp-sched.o
  LD       /tmp/build/perf/ui/perf-in.o
  CC       /tmp/build/perf/tests/fdarray.o
  CC       /tmp/build/perf/util/mmap.o

perf tools build broken after v5.1-rc1

[Arnaldo Carvalho de Melo]

Em Fri, Apr 26, 2019 at 04:28:34PM -0300, Arnaldo Carvalho de Melo escreveu:
> Em Thu, Apr 25, 2019 at 06:48:00PM -0300, Arnaldo Carvalho de Melo escreveu:
> > Em Mon, Apr 22, 2019 at 12:20:27PM -0300, Arnaldo Carvalho de Melo escreveu:
> > > Em Fri, Apr 19, 2019 at 04:32:58PM -0700, Vineet Gupta escreveu:
> > > > When building perf for ARC (v5.1-rc2) I get the following
> > >  
> > > > | In file included from bench/futex-hash.c:26:
> > > > | bench/futex.h: In function 'futex_wait':
> > > > | bench/futex.h:37:10: error: 'SYS_futex' undeclared (first use in this function);
> > >  
> > > > git bisect led to 1a787fc5ba18ac767e635c58d06a0b46876184e3 ("tools headers uapi:
> > > > Sync copy of asm-generic/unistd.h with the kernel sources")
> > > 
> > > Humm, I have to check why this:
> > > 
> > > [perfbuilder at quaco ~]$ podman images | grep ARC
> > > docker.io/acmel/linux-perf-tools-build-fedora                24-x-ARC-uClibc          4c259582a8e6   5 weeks ago      846 MB
> > > [perfbuilder at quaco ~]$
> > > 
> > > isn't catching this... :-\
> > > 
> > > FROM docker.io/fedora:24
> > > MAINTAINER Arnaldo Carvalho de Melo <acme at kernel.org>
> > > ENV TOOLCHAIN=arc_gnu_2017.09-rc2_prebuilt_uclibc_le_arc700_linux_install
> > > ENV CROSS=arc-linux-
> > > ENV SOURCEFILE=${TOOLCHAIN}.tar.gz
> > > RUN dnf -y install make flex bison binutils gcc wget tar bzip2 bc findutils xz
> > > RUN wget https://github.com/foss-for-synopsys-dwc-arc-processors/toolchain/releases/download/arc-2017.09-rc2/${SOURCEFILE}
> > > <SNIP>
> > > COPY rx_and_build.sh /
> > > ENV EXTRA_MAKE_ARGS=NO_LIBBPF=1
> > > ENV ARCH=arc
> > > ENV CROSS_COMPILE=/${TOOLCHAIN}/bin/${CROSS}
> > > ENV EXTRA_CFLAGS=-matomic
> > 
> > So, now I have a libnuma crossbuilt in this container that allows me to
> > build a ARC perf binary linked with zlib and numactl-devel, but only
> > after I applied the fix below.
> > 
> > Can you please provide the feature detection header in the build? I.e.
> > what I have with my ARC cross build container right now, after applying
> > the patch below is:
> 
> So, switched from the uCLibc build system to the glibc based one, so
> that I could get elfutils building (it needs argp that isn't available
> with uCLibc) I have it reproduced, see below, now I'm testing with the
> fix, which is to grab the unistd.h files you noticed missing for the
> arches that are present in tools/arch/, will post results soon

Yep, now it builds:

/tmp/build/perf/perf: ELF 32-bit LSB executable, *unknown arch 0xc3* version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-arc.so.2, for GNU/Linux 3.9.0, with debug_info, not stripped

With this patch:

commit dd423246b321967eace3f3e0fe73d638050b447c
Author: Arnaldo Carvalho de Melo <acme at redhat.com>
Date:   Mon Apr 22 15:21:35 2019 -0300

    WIP
    
    Signed-off-by: Arnaldo Carvalho de Melo <acme at redhat.com>

diff --git a/tools/arch/arc/include/uapi/asm/unistd.h b/tools/arch/arc/include/uapi/asm/unistd.h
new file mode 100644
index 000000000000..5eafa1115162
--- /dev/null
+++ b/tools/arch/arc/include/uapi/asm/unistd.h
@@ -0,0 +1,51 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+/*
+ * Copyright (C) 2004, 2007-2010, 2011-2012 Synopsys, Inc. (www.synopsys.com)
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+/******** no-legacy-syscalls-ABI *******/
+
+/*
+ * Non-typical guard macro to enable inclusion twice in ARCH sys.c
+ * That is how the Generic syscall wrapper generator works
+ */
+#if !defined(_UAPI_ASM_ARC_UNISTD_H) || defined(__SYSCALL)
+#define _UAPI_ASM_ARC_UNISTD_H
+
+#define __ARCH_WANT_RENAMEAT
+#define __ARCH_WANT_STAT64
+#define __ARCH_WANT_SET_GET_RLIMIT
+#define __ARCH_WANT_SYS_EXECVE
+#define __ARCH_WANT_SYS_CLONE
+#define __ARCH_WANT_SYS_VFORK
+#define __ARCH_WANT_SYS_FORK
+#define __ARCH_WANT_TIME32_SYSCALLS
+
+#define sys_mmap2 sys_mmap_pgoff
+
+#include <asm-generic/unistd.h>
+
+#define NR_syscalls	__NR_syscalls
+
+/* Generic syscall (fs/filesystems.c - lost in asm-generic/unistd.h */
+#define __NR_sysfs		(__NR_arch_specific_syscall + 3)
+
+/* ARC specific syscall */
+#define __NR_cacheflush		(__NR_arch_specific_syscall + 0)
+#define __NR_arc_settls		(__NR_arch_specific_syscall + 1)
+#define __NR_arc_gettls		(__NR_arch_specific_syscall + 2)
+#define __NR_arc_usr_cmpxchg	(__NR_arch_specific_syscall + 4)
+
+__SYSCALL(__NR_cacheflush, sys_cacheflush)
+__SYSCALL(__NR_arc_settls, sys_arc_settls)
+__SYSCALL(__NR_arc_gettls, sys_arc_gettls)
+__SYSCALL(__NR_arc_usr_cmpxchg, sys_arc_usr_cmpxchg)
+__SYSCALL(__NR_sysfs, sys_sysfs)
+
+#undef __SYSCALL
+
+#endif
diff --git a/tools/arch/hexagon/include/uapi/asm/unistd.h b/tools/arch/hexagon/include/uapi/asm/unistd.h
new file mode 100644
index 000000000000..432c4db1b623
--- /dev/null
+++ b/tools/arch/hexagon/include/uapi/asm/unistd.h
@@ -0,0 +1,40 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+/*
+ * Syscall support for Hexagon
+ *
+ * Copyright (c) 2010-2011, The Linux Foundation. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 and
+ * only version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
+ * 02110-1301, USA.
+ */
+
+/*
+ *  The kernel pulls this unistd.h in three different ways:
+ *  1.  the "normal" way which gets all the __NR defines
+ *  2.  with __SYSCALL defined to produce function declarations
+ *  3.  with __SYSCALL defined to produce syscall table initialization
+ *  See also:  syscalltab.c
+ */
+
+#define sys_mmap2 sys_mmap_pgoff
+#define __ARCH_WANT_RENAMEAT
+#define __ARCH_WANT_STAT64
+#define __ARCH_WANT_SET_GET_RLIMIT
+#define __ARCH_WANT_SYS_EXECVE
+#define __ARCH_WANT_SYS_CLONE
+#define __ARCH_WANT_SYS_VFORK
+#define __ARCH_WANT_SYS_FORK
+#define __ARCH_WANT_TIME32_SYSCALLS
+
+#include <asm-generic/unistd.h>
diff --git a/tools/arch/riscv/include/uapi/asm/unistd.h b/tools/arch/riscv/include/uapi/asm/unistd.h
new file mode 100644
index 000000000000..0e2eeeb1fd27
--- /dev/null
+++ b/tools/arch/riscv/include/uapi/asm/unistd.h
@@ -0,0 +1,42 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+/*
+ * Copyright (C) 2018 David Abdurachmanov <david.abdurachmanov at gmail.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifdef __LP64__
+#define __ARCH_WANT_NEW_STAT
+#define __ARCH_WANT_SET_GET_RLIMIT
+#endif /* __LP64__ */
+
+#include <asm-generic/unistd.h>
+
+/*
+ * Allows the instruction cache to be flushed from userspace.  Despite RISC-V
+ * having a direct 'fence.i' instruction available to userspace (which we
+ * can't trap!), that's not actually viable when running on Linux because the
+ * kernel might schedule a process on another hart.  There is no way for
+ * userspace to handle this without invoking the kernel (as it doesn't know the
+ * thread->hart mappings), so we've defined a RISC-V specific system call to
+ * flush the instruction cache.
+ *
+ * __NR_riscv_flush_icache is defined to flush the instruction cache over an
+ * address range, with the flush applying to either all threads or just the
+ * caller.  We don't currently do anything with the address range, that's just
+ * in there for forwards compatibility.
+ */
+#ifndef __NR_riscv_flush_icache
+#define __NR_riscv_flush_icache (__NR_arch_specific_syscall + 15)
+#endif
+__SYSCALL(__NR_riscv_flush_icache, sys_riscv_flush_icache)

perf tools build broken after v5.1-rc1

[Vineet Gupta]

On 4/26/19 12:35 PM, Arnaldo Carvalho de Melo wrote:
> Em Fri, Apr 26, 2019 at 04:28:34PM -0300, Arnaldo Carvalho de Melo escreveu:
>> Em Thu, Apr 25, 2019 at 06:48:00PM -0300, Arnaldo Carvalho de Melo escreveu:
>>> Em Mon, Apr 22, 2019 at 12:20:27PM -0300, Arnaldo Carvalho de Melo escreveu:
>>>> Em Fri, Apr 19, 2019 at 04:32:58PM -0700, Vineet Gupta escreveu:
>>>>> When building perf for ARC (v5.1-rc2) I get the following
>>>>  
>>>>> | In file included from bench/futex-hash.c:26:
>>>>> | bench/futex.h: In function 'futex_wait':
>>>>> | bench/futex.h:37:10: error: 'SYS_futex' undeclared (first use in this function);
>>>>  
>>>>> git bisect led to 1a787fc5ba18ac767e635c58d06a0b46876184e3 ("tools headers uapi:
>>>>> Sync copy of asm-generic/unistd.h with the kernel sources")
>>>> Humm, I have to check why this:
>>>>
>>>> [perfbuilder at quaco ~]$ podman images | grep ARC
>>>> docker.io/acmel/linux-perf-tools-build-fedora                24-x-ARC-uClibc          4c259582a8e6   5 weeks ago      846 MB
>>>> [perfbuilder at quaco ~]$
>>>>
>>>> isn't catching this... :-\
>>>>
>>>> FROM docker.io/fedora:24
>>>> MAINTAINER Arnaldo Carvalho de Melo <acme at kernel.org>
>>>> ENV TOOLCHAIN=arc_gnu_2017.09-rc2_prebuilt_uclibc_le_arc700_linux_install
>>>> ENV CROSS=arc-linux-
>>>> ENV SOURCEFILE=${TOOLCHAIN}.tar.gz
>>>> RUN dnf -y install make flex bison binutils gcc wget tar bzip2 bc findutils xz
>>>> RUN wget https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_foss-2Dfor-2Dsynopsys-2Ddwc-2Darc-2Dprocessors_toolchain_releases_download_arc-2D2017.09-2Drc2_-24-257BSOURCEFILE-257D&d=DwIBAg&c=DPL6_X_6JkXFx7AXWqB0tg&r=7FgpX6o3vAhwMrMhLh-4ZJey5kjdNUwOL2CWsFwR4T8&m=obdUU5ZihUrFi8F2O4JfVYScd7CLKNItF83dHmezlkU&s=oOPRKzvGneimee7GFXWyqqoVfgvHu-jwSMKPbSqwP5M&e=
>>>> <SNIP>
>>>> COPY rx_and_build.sh /
>>>> ENV EXTRA_MAKE_ARGS=NO_LIBBPF=1
>>>> ENV ARCH=arc
>>>> ENV CROSS_COMPILE=/${TOOLCHAIN}/bin/${CROSS}
>>>> ENV EXTRA_CFLAGS=-matomic
>>> So, now I have a libnuma crossbuilt in this container that allows me to
>>> build a ARC perf binary linked with zlib and numactl-devel, but only
>>> after I applied the fix below.
>>>
>>> Can you please provide the feature detection header in the build? I.e.
>>> what I have with my ARC cross build container right now, after applying
>>> the patch below is:
>> So, switched from the uCLibc build system to the glibc based one, so
>> that I could get elfutils building (it needs argp that isn't available
>> with uCLibc) I have it reproduced, see below, now I'm testing with the
>> fix, which is to grab the unistd.h files you noticed missing for the
>> arches that are present in tools/arch/, will post results soon
> Yep, now it builds:
>
> /tmp/build/perf/perf: ELF 32-bit LSB executable, *unknown arch 0xc3* version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-arc.so.2, for GNU/Linux 3.9.0, with debug_info, not stripped
>
> With this patch:
>
> commit dd423246b321967eace3f3e0fe73d638050b447c
> Author: Arnaldo Carvalho de Melo <acme at redhat.com>
> Date:   Mon Apr 22 15:21:35 2019 -0300
>
>     WIP
>     
>     Signed-off-by: Arnaldo Carvalho de Melo <acme at redhat.com>
>
> diff --git a/tools/arch/arc/include/uapi/asm/unistd.h b/tools/arch/arc/include/uapi/asm/unistd.h
> new file mode 100644
> index 000000000000..5eafa1115162
> --- /dev/null
> +++ b/tools/arch/arc/include/uapi/asm/unistd.h
> @@ -0,0 +1,51 @@
> +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
> +/*
> + * Copyright (C) 2004, 2007-2010, 2011-2012 Synopsys, Inc. (http://www.synopsys.com)
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + */
> +
> +/******** no-legacy-syscalls-ABI *******/
> +
> +/*
> + * Non-typical guard macro to enable inclusion twice in ARCH sys.c
> + * That is how the Generic syscall wrapper generator works
> + */
> +#if !defined(_UAPI_ASM_ARC_UNISTD_H) || defined(__SYSCALL)

This guard ifdef is not present in existing tools/arch/*/**/unistd.h , nor in the
ones below so I'd suggest just drop it to keep things consistent.
If you still want them, best to call it _TOOLS_UAPI_ASM_ARC_UNISTD_H

> +#define _UAPI_ASM_ARC_UNISTD_H
> +
> +#define __ARCH_WANT_RENAMEAT
> +#define __ARCH_WANT_STAT64
> +#define __ARCH_WANT_SET_GET_RLIMIT
> +#define __ARCH_WANT_SYS_EXECVE
> +#define __ARCH_WANT_SYS_CLONE
> +#define __ARCH_WANT_SYS_VFORK
> +#define __ARCH_WANT_SYS_FORK
> +#define __ARCH_WANT_TIME32_SYSCALLS
> +
> +#define sys_mmap2 sys_mmap_pgoff
> +
> +#include <asm-generic/unistd.h>

> +
> +#define NR_syscalls	__NR_syscalls
> +
> +/* Generic syscall (fs/filesystems.c - lost in asm-generic/unistd.h */
> +#define __NR_sysfs		(__NR_arch_specific_syscall + 3)
> +
> +/* ARC specific syscall */
> +#define __NR_cacheflush		(__NR_arch_specific_syscall + 0)
> +#define __NR_arc_settls		(__NR_arch_specific_syscall + 1)
> +#define __NR_arc_gettls		(__NR_arch_specific_syscall + 2)
> +#define __NR_arc_usr_cmpxchg	(__NR_arch_specific_syscall + 4)
> +
> +__SYSCALL(__NR_cacheflush, sys_cacheflush)
> +__SYSCALL(__NR_arc_settls, sys_arc_settls)
> +__SYSCALL(__NR_arc_gettls, sys_arc_gettls)
> +__SYSCALL(__NR_arc_usr_cmpxchg, sys_arc_usr_cmpxchg)
> +__SYSCALL(__NR_sysfs, sys_sysfs)
> +
> +#undef __SYSCALL
> +
> +#endif
> diff --git a/tools/arch/hexagon/include/uapi/asm/unistd.h b/tools/arch/hexagon/include/uapi/asm/unistd.h
> new file mode 100644
> index 000000000000..432c4db1b623
> --- /dev/null
> +++ b/tools/arch/hexagon/include/uapi/asm/unistd.h
> @@ -0,0 +1,40 @@
> +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
> +/*
> + * Syscall support for Hexagon
> + *
> + * Copyright (c) 2010-2011, The Linux Foundation. All rights reserved.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 and
> + * only version 2 as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program; if not, write to the Free Software
> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
> + * 02110-1301, USA.
> + */
> +
> +/*
> + *  The kernel pulls this unistd.h in three different ways:
> + *  1.  the "normal" way which gets all the __NR defines
> + *  2.  with __SYSCALL defined to produce function declarations
> + *  3.  with __SYSCALL defined to produce syscall table initialization
> + *  See also:  syscalltab.c
> + */
> +
> +#define sys_mmap2 sys_mmap_pgoff
> +#define __ARCH_WANT_RENAMEAT
> +#define __ARCH_WANT_STAT64
> +#define __ARCH_WANT_SET_GET_RLIMIT
> +#define __ARCH_WANT_SYS_EXECVE
> +#define __ARCH_WANT_SYS_CLONE
> +#define __ARCH_WANT_SYS_VFORK
> +#define __ARCH_WANT_SYS_FORK
> +#define __ARCH_WANT_TIME32_SYSCALLS
> +
> +#include <asm-generic/unistd.h>
> diff --git a/tools/arch/riscv/include/uapi/asm/unistd.h b/tools/arch/riscv/include/uapi/asm/unistd.h
> new file mode 100644
> index 000000000000..0e2eeeb1fd27
> --- /dev/null
> +++ b/tools/arch/riscv/include/uapi/asm/unistd.h
> @@ -0,0 +1,42 @@
> +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
> +/*
> + * Copyright (C) 2018 David Abdurachmanov <david.abdurachmanov at gmail.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program.  If not, see <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.gnu.org_licenses_&d=DwIBAg&c=DPL6_X_6JkXFx7AXWqB0tg&r=7FgpX6o3vAhwMrMhLh-4ZJey5kjdNUwOL2CWsFwR4T8&m=obdUU5ZihUrFi8F2O4JfVYScd7CLKNItF83dHmezlkU&s=qw-VNuV5RBu7T5rL77GOTAKBzZB-zvizjJ3Haik7x9g&e=>.
> + */
> +
> +#ifdef __LP64__
> +#define __ARCH_WANT_NEW_STAT
> +#define __ARCH_WANT_SET_GET_RLIMIT
> +#endif /* __LP64__ */
> +
> +#include <asm-generic/unistd.h>
> +
> +/*
> + * Allows the instruction cache to be flushed from userspace.  Despite RISC-V
> + * having a direct 'fence.i' instruction available to userspace (which we
> + * can't trap!), that's not actually viable when running on Linux because the
> + * kernel might schedule a process on another hart.  There is no way for
> + * userspace to handle this without invoking the kernel (as it doesn't know the
> + * thread->hart mappings), so we've defined a RISC-V specific system call to
> + * flush the instruction cache.
> + *
> + * __NR_riscv_flush_icache is defined to flush the instruction cache over an
> + * address range, with the flush applying to either all threads or just the
> + * caller.  We don't currently do anything with the address range, that's just
> + * in there for forwards compatibility.
> + */
> +#ifndef __NR_riscv_flush_icache
> +#define __NR_riscv_flush_icache (__NR_arch_specific_syscall + 15)
> +#endif
> +__SYSCALL(__NR_riscv_flush_icache, sys_riscv_flush_icache)
>

perf tools build broken after v5.1-rc1

[Vineet Gupta]

On 4/25/19 2:48 PM, Arnaldo Carvalho de Melo wrote:
> Em Mon, Apr 22, 2019 at 12:20:27PM -0300, Arnaldo Carvalho de Melo escreveu:
>> Em Fri, Apr 19, 2019 at 04:32:58PM -0700, Vineet Gupta escreveu:
>>> When building perf for ARC (v5.1-rc2) I get the following
>>  
>>> | In file included from bench/futex-hash.c:26:
>>> | bench/futex.h: In function 'futex_wait':
>>> | bench/futex.h:37:10: error: 'SYS_futex' undeclared (first use in this function);
>>  
>>> git bisect led to 1a787fc5ba18ac767e635c58d06a0b46876184e3 ("tools headers uapi:
>>> Sync copy of asm-generic/unistd.h with the kernel sources")
>> Humm, I have to check why this:
>>
>> [perfbuilder at quaco ~]$ podman images | grep ARC
>> docker.io/acmel/linux-perf-tools-build-fedora                24-x-ARC-uClibc          4c259582a8e6   5 weeks ago      846 MB
>> [perfbuilder at quaco ~]$
>>
>> isn't catching this... :-\
>>
>> FROM docker.io/fedora:24
>> MAINTAINER Arnaldo Carvalho de Melo <acme at kernel.org>
>> ENV TOOLCHAIN=arc_gnu_2017.09-rc2_prebuilt_uclibc_le_arc700_linux_install
>> ENV CROSS=arc-linux-
>> ENV SOURCEFILE=${TOOLCHAIN}.tar.gz
>> RUN dnf -y install make flex bison binutils gcc wget tar bzip2 bc findutils xz
>> RUN wget https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_foss-2Dfor-2Dsynopsys-2Ddwc-2Darc-2Dprocessors_toolchain_releases_download_arc-2D2017.09-2Drc2_-24-257BSOURCEFILE-257D&d=DwIDaQ&c=DPL6_X_6JkXFx7AXWqB0tg&r=7FgpX6o3vAhwMrMhLh-4ZJey5kjdNUwOL2CWsFwR4T8&m=HjtufCLozrW47pS5C2YH3safLHQE7eEtmHFZsSWrz1M&s=29g4oKvGuYcLgheCUvZh3wojhhljivpLd8aj7Ur4sKQ&e=
>> <SNIP>
>> COPY rx_and_build.sh /
>> ENV EXTRA_MAKE_ARGS=NO_LIBBPF=1
>> ENV ARCH=arc
>> ENV CROSS_COMPILE=/${TOOLCHAIN}/bin/${CROSS}
>> ENV EXTRA_CFLAGS=-matomic
> So, now I have a libnuma crossbuilt in this container that allows me to
> build a ARC perf binary linked with zlib and numactl-devel, but only
> after I applied the fix below.
>
> Can you please provide the feature detection header in the build? I.e.
> what I have with my ARC cross build container right now, after applying
> the patch below is:
>
> [perfbuilder at 60d5802468f6 perf]$ make $EXTRA_MAKE_ARGS ARCH=$ARCH CROSS_COMPILE=$CROSS_COMPILE EXTRA_CFLAGS="$EXTRA_CFLAGS" -C /git/perf/tools/perf O=/tmp/build/perf
> make: Entering directory '/git/perf/tools/perf'
>   BUILD:   Doing 'make -j8' parallel build
> sh: line 0: command: -c: invalid option
> command: usage: command [-pVv] command [arg ...]
>
> Auto-detecting system features:
> ...                         dwarf: [ OFF ]
> ...            dwarf_getlocations: [ OFF ]
> ...                         glibc: [ on  ]

Not related to current issue, this run uses a uClibc toolchain and yet it is
detecting glibc - doesn't seem right to me.

> ...                          gtk2: [ OFF ]
> ...                      libaudit: [ OFF ]
> ...                        libbfd: [ OFF ]
> ...                        libelf: [ OFF ]
> ...                       libnuma: [ on  ]

Wondering why that is - for me numa is off - even when using a glibc toolchain.

> ...        numa_num_possible_cpus: [ on  ]
> ...                       libperl: [ OFF ]
> ...                     libpython: [ OFF ]
> ...                      libslang: [ OFF ]
> ...                     libcrypto: [ OFF ]
> ...                     libunwind: [ OFF ]
> ...            libdw-dwarf-unwind: [ OFF ]
> ...                          zlib: [ OFF ]
> ...                          lzma: [ OFF ]
> ...                     get_cpuid: [ OFF ]
> ...                           bpf: [ on  ]
> ...                        libaio: [ OFF ]
> ...        disassembler-four-args: [ OFF ]
>
>

For my glibc toolchain, here's the feature detection output

Auto-detecting system features:
...                         dwarf: [ on  ]
...            dwarf_getlocations: [ OFF ]
...                         glibc: [ on  ]
...                          gtk2: [ OFF ]
...                      libaudit: [ OFF ]
...                        libbfd: [ OFF ]
...                        libelf: [ on  ]
...                       libnuma: [ OFF ]
...        numa_num_possible_cpus: [ OFF ]
...                       libperl: [ OFF ]
...                     libpython: [ OFF ]
...                      libslang: [ OFF ]
...                     libcrypto: [ OFF ]
...                     libunwind: [ OFF ]
...            libdw-dwarf-unwind: [ OFF ]
...                          zlib: [ OFF ]
...                          lzma: [ OFF ]
...                     get_cpuid: [ OFF ]
...                           bpf: [ on  ]
...                        libaio: [ on  ]
...        disassembler-four-args: [ OFF ]

perf tools build broken after v5.1-rc1

[Arnaldo Carvalho de Melo]

Em Mon, Apr 29, 2019 at 05:14:54PM +0000, Vineet Gupta escreveu:
> On 4/25/19 2:48 PM, Arnaldo Carvalho de Melo wrote:
> > Em Mon, Apr 22, 2019 at 12:20:27PM -0300, Arnaldo Carvalho de Melo escreveu:
> >> Em Fri, Apr 19, 2019 at 04:32:58PM -0700, Vineet Gupta escreveu:
> >>> When building perf for ARC (v5.1-rc2) I get the following
> >>  
> >>> | In file included from bench/futex-hash.c:26:
> >>> | bench/futex.h: In function 'futex_wait':
> >>> | bench/futex.h:37:10: error: 'SYS_futex' undeclared (first use in this function);
> >>  
> >>> git bisect led to 1a787fc5ba18ac767e635c58d06a0b46876184e3 ("tools headers uapi:
> >>> Sync copy of asm-generic/unistd.h with the kernel sources")
> >> Humm, I have to check why this:
> >>
> >> [perfbuilder at quaco ~]$ podman images | grep ARC
> >> docker.io/acmel/linux-perf-tools-build-fedora                24-x-ARC-uClibc          4c259582a8e6   5 weeks ago      846 MB
> >> [perfbuilder at quaco ~]$
> >>
> >> isn't catching this... :-\
> >>
> >> FROM docker.io/fedora:24
> >> MAINTAINER Arnaldo Carvalho de Melo <acme at kernel.org>
> >> ENV TOOLCHAIN=arc_gnu_2017.09-rc2_prebuilt_uclibc_le_arc700_linux_install
> >> ENV CROSS=arc-linux-
> >> ENV SOURCEFILE=${TOOLCHAIN}.tar.gz
> >> RUN dnf -y install make flex bison binutils gcc wget tar bzip2 bc findutils xz
> >> RUN wget https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_foss-2Dfor-2Dsynopsys-2Ddwc-2Darc-2Dprocessors_toolchain_releases_download_arc-2D2017.09-2Drc2_-24-257BSOURCEFILE-257D&d=DwIDaQ&c=DPL6_X_6JkXFx7AXWqB0tg&r=7FgpX6o3vAhwMrMhLh-4ZJey5kjdNUwOL2CWsFwR4T8&m=HjtufCLozrW47pS5C2YH3safLHQE7eEtmHFZsSWrz1M&s=29g4oKvGuYcLgheCUvZh3wojhhljivpLd8aj7Ur4sKQ&e=
> >> <SNIP>
> >> COPY rx_and_build.sh /
> >> ENV EXTRA_MAKE_ARGS=NO_LIBBPF=1
> >> ENV ARCH=arc
> >> ENV CROSS_COMPILE=/${TOOLCHAIN}/bin/${CROSS}
> >> ENV EXTRA_CFLAGS=-matomic
> > So, now I have a libnuma crossbuilt in this container that allows me to
> > build a ARC perf binary linked with zlib and numactl-devel, but only
> > after I applied the fix below.
> >
> > Can you please provide the feature detection header in the build? I.e.
> > what I have with my ARC cross build container right now, after applying
> > the patch below is:
> >
> > [perfbuilder at 60d5802468f6 perf]$ make $EXTRA_MAKE_ARGS ARCH=$ARCH CROSS_COMPILE=$CROSS_COMPILE EXTRA_CFLAGS="$EXTRA_CFLAGS" -C /git/perf/tools/perf O=/tmp/build/perf
> > make: Entering directory '/git/perf/tools/perf'
> >   BUILD:   Doing 'make -j8' parallel build
> > sh: line 0: command: -c: invalid option
> > command: usage: command [-pVv] command [arg ...]
> >
> > Auto-detecting system features:
> > ...                         dwarf: [ OFF ]
> > ...            dwarf_getlocations: [ OFF ]
> > ...                         glibc: [ on  ]
> 
> Not related to current issue, this run uses a uClibc toolchain and yet it is
> detecting glibc - doesn't seem right to me.

Ok, I'll improve that, I think it just tries to detect a libc, yeah,
see:

[acme at quaco linux]$ cat tools/build/feature/test-glibc.c
// SPDX-License-Identifier: GPL-2.0
#include <stdlib.h>

#if !defined(__UCLIBC__)
#include <gnu/libc-version.h>
#else
#define XSTR(s) STR(s)
#define STR(s) #s
#endif

int main(void)
{
#if !defined(__UCLIBC__)
	const char *version = gnu_get_libc_version();
#else
	const char *version = XSTR(__GLIBC__) "." XSTR(__GLIBC_MINOR__);
#endif

	return (long)version;
}
[acme at quaco linux]$

[perfbuilder at 59ca4b424ded /]$ grep __GLIBC__ /arc_gnu_2017.09-rc2_prebuilt_uclibc_le_arc700_linux_install/arc-snps-linux-uclibc/sysroot/usr/include/*.h
/arc_gnu_2017.09-rc2_prebuilt_uclibc_le_arc700_linux_install/arc-snps-linux-uclibc/sysroot/usr/include/features.h:   The macros `__GNU_LIBRARY__', `__GLIBC__', and `__GLIBC_MINOR__' are
/arc_gnu_2017.09-rc2_prebuilt_uclibc_le_arc700_linux_install/arc-snps-linux-uclibc/sysroot/usr/include/features.h:#define	__GLIBC__	2
/arc_gnu_2017.09-rc2_prebuilt_uclibc_le_arc700_linux_install/arc-snps-linux-uclibc/sysroot/usr/include/features.h:	((__GLIBC__ << 16) + __GLIBC_MINOR__ >= ((maj) << 16) + (min))
[perfbuilder at 59ca4b424ded /]$

Isn't that part of uClibc?
 
> > ...                          gtk2: [ OFF ]
> > ...                      libaudit: [ OFF ]
> > ...                        libbfd: [ OFF ]
> > ...                        libelf: [ OFF ]
> > ...                       libnuma: [ on  ]
> 
> Wondering why that is - for me numa is off - even when using a glibc toolchain.
> 
> > ...        numa_num_possible_cpus: [ on  ]
> > ...                       libperl: [ OFF ]
> > ...                     libpython: [ OFF ]
> > ...                      libslang: [ OFF ]
> > ...                     libcrypto: [ OFF ]
> > ...                     libunwind: [ OFF ]
> > ...            libdw-dwarf-unwind: [ OFF ]
> > ...                          zlib: [ OFF ]
> > ...                          lzma: [ OFF ]
> > ...                     get_cpuid: [ OFF ]
> > ...                           bpf: [ on  ]
> > ...                        libaio: [ OFF ]
> > ...        disassembler-four-args: [ OFF ]
> >
> >
> 
> For my glibc toolchain, here's the feature detection output
> 
> Auto-detecting system features:
> ...                         dwarf: [ on  ]
> ...            dwarf_getlocations: [ OFF ]
> ...                         glibc: [ on  ]
> ...                          gtk2: [ OFF ]
> ...                      libaudit: [ OFF ]
> ...                        libbfd: [ OFF ]
> ...                        libelf: [ on  ]
> ...                       libnuma: [ OFF ]
> ...        numa_num_possible_cpus: [ OFF ]
> ...                       libperl: [ OFF ]
> ...                     libpython: [ OFF ]
> ...                      libslang: [ OFF ]
> ...                     libcrypto: [ OFF ]
> ...                     libunwind: [ OFF ]
> ...            libdw-dwarf-unwind: [ OFF ]
> ...                          zlib: [ OFF ]
> ...                          lzma: [ OFF ]
> ...                     get_cpuid: [ OFF ]
> ...                           bpf: [ on  ]
> ...                        libaio: [ on  ]
> ...        disassembler-four-args: [ OFF ]
> 
> 

-- 

- Arnaldo

Detecting libc in perf (was Re: perf tools build broken after v5.1-rc1)

[Vineet Gupta]

On 4/29/19 6:18 PM, Arnaldo Carvalho de Melo wrote:
>>> Auto-detecting system features:
>>> ...                         dwarf: [ OFF ]
>>> ...            dwarf_getlocations: [ OFF ]
>>> ...                         glibc: [ on  ]
>> Not related to current issue, this run uses a uClibc toolchain and yet it is
>> detecting glibc - doesn't seem right to me.
> Ok, I'll improve that, I think it just tries to detect a libc, yeah,
> see:
>
> [acme at quaco linux]$ cat tools/build/feature/test-glibc.c
> // SPDX-License-Identifier: GPL-2.0
> #include <stdlib.h>
>
> #if !defined(__UCLIBC__)
> #include <gnu/libc-version.h>
> #else
> #define XSTR(s) STR(s)
> #define STR(s) #s
> #endif
>
> int main(void)
> {
> #if !defined(__UCLIBC__)
> 	const char *version = gnu_get_libc_version();
> #else
> 	const char *version = XSTR(__GLIBC__) "." XSTR(__GLIBC_MINOR__);
> #endif
>
> 	return (long)version;
> }
> [acme at quaco linux]$
>
> [perfbuilder at 59ca4b424ded /]$ grep __GLIBC__ /arc_gnu_2017.09-rc2_prebuilt_uclibc_le_arc700_linux_install/arc-snps-linux-uclibc/sysroot/usr/include/*.h
> /arc_gnu_2017.09-rc2_prebuilt_uclibc_le_arc700_linux_install/arc-snps-linux-uclibc/sysroot/usr/include/features.h:   The macros `__GNU_LIBRARY__', `__GLIBC__', and `__GLIBC_MINOR__' are
> /arc_gnu_2017.09-rc2_prebuilt_uclibc_le_arc700_linux_install/arc-snps-linux-uclibc/sysroot/usr/include/features.h:#define	__GLIBC__	2
> /arc_gnu_2017.09-rc2_prebuilt_uclibc_le_arc700_linux_install/arc-snps-linux-uclibc/sysroot/usr/include/features.h:	((__GLIBC__ << 16) + __GLIBC_MINOR__ >= ((maj) << 16) + (min))
> [perfbuilder at 59ca4b424ded /]$
>
> Isn't that part of uClibc?

Right you are. Per the big fat comment right above that code, this gross hack in
uclibc is unavoidable as applications tend to rely on that define.
So a better fix would be to check for various !GLIBC libs explicitly.

#ifdef __UCLIBC__

#elseif defined __MUSL__

...

Not pretty from app usage pov, but that seems to be the only sane way of doing it.

-Vineet

Detecting libc in perf (was Re: perf tools build broken after v5.1-rc1)

[Rich Felker]

On Tue, Apr 30, 2019@03:53:18PM +0000, Vineet Gupta wrote:
> On 4/29/19 6:18 PM, Arnaldo Carvalho de Melo wrote:
> >>> Auto-detecting system features:
> >>> ...                         dwarf: [ OFF ]
> >>> ...            dwarf_getlocations: [ OFF ]
> >>> ...                         glibc: [ on  ]
> >> Not related to current issue, this run uses a uClibc toolchain and yet it is
> >> detecting glibc - doesn't seem right to me.
> > Ok, I'll improve that, I think it just tries to detect a libc, yeah,
> > see:
> >
> > [acme at quaco linux]$ cat tools/build/feature/test-glibc.c
> > // SPDX-License-Identifier: GPL-2.0
> > #include <stdlib.h>
> >
> > #if !defined(__UCLIBC__)
> > #include <gnu/libc-version.h>
> > #else
> > #define XSTR(s) STR(s)
> > #define STR(s) #s
> > #endif
> >
> > int main(void)
> > {
> > #if !defined(__UCLIBC__)
> > 	const char *version = gnu_get_libc_version();
> > #else
> > 	const char *version = XSTR(__GLIBC__) "." XSTR(__GLIBC_MINOR__);
> > #endif
> >
> > 	return (long)version;
> > }
> > [acme at quaco linux]$
> >
> > [perfbuilder at 59ca4b424ded /]$ grep __GLIBC__ /arc_gnu_2017.09-rc2_prebuilt_uclibc_le_arc700_linux_install/arc-snps-linux-uclibc/sysroot/usr/include/*.h
> > /arc_gnu_2017.09-rc2_prebuilt_uclibc_le_arc700_linux_install/arc-snps-linux-uclibc/sysroot/usr/include/features.h:   The macros `__GNU_LIBRARY__', `__GLIBC__', and `__GLIBC_MINOR__' are
> > /arc_gnu_2017.09-rc2_prebuilt_uclibc_le_arc700_linux_install/arc-snps-linux-uclibc/sysroot/usr/include/features.h:#define	__GLIBC__	2
> > /arc_gnu_2017.09-rc2_prebuilt_uclibc_le_arc700_linux_install/arc-snps-linux-uclibc/sysroot/usr/include/features.h:	((__GLIBC__ << 16) + __GLIBC_MINOR__ >= ((maj) << 16) + (min))
> > [perfbuilder at 59ca4b424ded /]$
> >
> > Isn't that part of uClibc?
> 
> Right you are. Per the big fat comment right above that code, this gross hack in
> uclibc is unavoidable as applications tend to rely on that define.
> So a better fix would be to check for various !GLIBC libs explicitly.
> 
> #ifdef __UCLIBC__
> 
> #elseif defined __MUSL__
> 
> ....
> 
> Not pretty from app usage pov, but that seems to be the only sane way of doing it.

What are you trying to achieve? I was just CC'd and I'm missing the
context.

Rich

Detecting libc in perf (was Re: perf tools build broken after v5.1-rc1)

[Vineet Gupta]

On 4/30/19 10:04 AM, Rich Felker wrote:
> On Tue, Apr 30, 2019@03:53:18PM +0000, Vineet Gupta wrote:
>> On 4/29/19 6:18 PM, Arnaldo Carvalho de Melo wrote:
>>>>> Auto-detecting system features:
>>>>> ...                         dwarf: [ OFF ]
>>>>> ...            dwarf_getlocations: [ OFF ]
>>>>> ...                         glibc: [ on  ]
>>>> Not related to current issue, this run uses a uClibc toolchain and yet it is
>>>> detecting glibc - doesn't seem right to me.
>>> Ok, I'll improve that, I think it just tries to detect a libc, yeah,
>>> see:
>>>
>>> [acme at quaco linux]$ cat tools/build/feature/test-glibc.c
>>> // SPDX-License-Identifier: GPL-2.0
>>> #include <stdlib.h>
>>>
>>> #if !defined(__UCLIBC__)
>>> #include <gnu/libc-version.h>
>>> #else
>>> #define XSTR(s) STR(s)
>>> #define STR(s) #s
>>> #endif
>>>
>>> int main(void)
>>> {
>>> #if !defined(__UCLIBC__)
>>> 	const char *version = gnu_get_libc_version();
>>> #else
>>> 	const char *version = XSTR(__GLIBC__) "." XSTR(__GLIBC_MINOR__);
>>> #endif
>>>
>>> 	return (long)version;
>>> }
>>> [acme at quaco linux]$
>>>
>>> [perfbuilder at 59ca4b424ded /]$ grep __GLIBC__ /arc_gnu_2017.09-rc2_prebuilt_uclibc_le_arc700_linux_install/arc-snps-linux-uclibc/sysroot/usr/include/*.h
>>> /arc_gnu_2017.09-rc2_prebuilt_uclibc_le_arc700_linux_install/arc-snps-linux-uclibc/sysroot/usr/include/features.h:   The macros `__GNU_LIBRARY__', `__GLIBC__', and `__GLIBC_MINOR__' are
>>> /arc_gnu_2017.09-rc2_prebuilt_uclibc_le_arc700_linux_install/arc-snps-linux-uclibc/sysroot/usr/include/features.h:#define	__GLIBC__	2
>>> /arc_gnu_2017.09-rc2_prebuilt_uclibc_le_arc700_linux_install/arc-snps-linux-uclibc/sysroot/usr/include/features.h:	((__GLIBC__ << 16) + __GLIBC_MINOR__ >= ((maj) << 16) + (min))
>>> [perfbuilder at 59ca4b424ded /]$
>>>
>>> Isn't that part of uClibc?
>>
>> Right you are. Per the big fat comment right above that code, this gross hack in
>> uclibc is unavoidable as applications tend to rely on that define.
>> So a better fix would be to check for various !GLIBC libs explicitly.
>>
>> #ifdef __UCLIBC__
>>
>> #elseif defined __MUSL__
>>
>> ....
>>
>> Not pretty from app usage pov, but that seems to be the only sane way of doing it.
> 
> What are you trying to achieve? I was just CC'd and I'm missing the
> context.

Sorry I added you as a subject matter expert but didn't provide enough context.

The original issue [1] was perf failing to build on ARC due to perf tools needing
a copy of unistd.h but this thread [2] was a small side issue of auto-detecting
libc variaint in perf tools where despite uClibc tools, glibc is declared to be
detected, due to uClibc's historical hack of defining __GLIBC__. So __GLIBC__ is
not sufficient (and probably not the right interface to begin wtih) to ensure glibc.

[1] http://lists.infradead.org/pipermail/linux-snps-arc/2019-April/005676.html
[2] http://lists.infradead.org/pipermail/linux-snps-arc/2019-April/005684.html

Detecting libc in perf (was Re: perf tools build broken after v5.1-rc1)

[Rich Felker]

On Tue, Apr 30, 2019@10:13:40AM -0700, Vineet Gupta wrote:
> On 4/30/19 10:04 AM, Rich Felker wrote:
> > On Tue, Apr 30, 2019@03:53:18PM +0000, Vineet Gupta wrote:
> >> On 4/29/19 6:18 PM, Arnaldo Carvalho de Melo wrote:
> >>>>> Auto-detecting system features:
> >>>>> ...                         dwarf: [ OFF ]
> >>>>> ...            dwarf_getlocations: [ OFF ]
> >>>>> ...                         glibc: [ on  ]
> >>>> Not related to current issue, this run uses a uClibc toolchain and yet it is
> >>>> detecting glibc - doesn't seem right to me.
> >>> Ok, I'll improve that, I think it just tries to detect a libc, yeah,
> >>> see:
> >>>
> >>> [acme at quaco linux]$ cat tools/build/feature/test-glibc.c
> >>> // SPDX-License-Identifier: GPL-2.0
> >>> #include <stdlib.h>
> >>>
> >>> #if !defined(__UCLIBC__)
> >>> #include <gnu/libc-version.h>
> >>> #else
> >>> #define XSTR(s) STR(s)
> >>> #define STR(s) #s
> >>> #endif
> >>>
> >>> int main(void)
> >>> {
> >>> #if !defined(__UCLIBC__)
> >>> 	const char *version = gnu_get_libc_version();
> >>> #else
> >>> 	const char *version = XSTR(__GLIBC__) "." XSTR(__GLIBC_MINOR__);
> >>> #endif
> >>>
> >>> 	return (long)version;
> >>> }
> >>> [acme at quaco linux]$
> >>>
> >>> [perfbuilder at 59ca4b424ded /]$ grep __GLIBC__ /arc_gnu_2017.09-rc2_prebuilt_uclibc_le_arc700_linux_install/arc-snps-linux-uclibc/sysroot/usr/include/*.h
> >>> /arc_gnu_2017.09-rc2_prebuilt_uclibc_le_arc700_linux_install/arc-snps-linux-uclibc/sysroot/usr/include/features.h:   The macros `__GNU_LIBRARY__', `__GLIBC__', and `__GLIBC_MINOR__' are
> >>> /arc_gnu_2017.09-rc2_prebuilt_uclibc_le_arc700_linux_install/arc-snps-linux-uclibc/sysroot/usr/include/features.h:#define	__GLIBC__	2
> >>> /arc_gnu_2017.09-rc2_prebuilt_uclibc_le_arc700_linux_install/arc-snps-linux-uclibc/sysroot/usr/include/features.h:	((__GLIBC__ << 16) + __GLIBC_MINOR__ >= ((maj) << 16) + (min))
> >>> [perfbuilder at 59ca4b424ded /]$
> >>>
> >>> Isn't that part of uClibc?
> >>
> >> Right you are. Per the big fat comment right above that code, this gross hack in
> >> uclibc is unavoidable as applications tend to rely on that define.
> >> So a better fix would be to check for various !GLIBC libs explicitly.
> >>
> >> #ifdef __UCLIBC__
> >>
> >> #elseif defined __MUSL__
> >>
> >> ....
> >>
> >> Not pretty from app usage pov, but that seems to be the only sane way of doing it.
> > 
> > What are you trying to achieve? I was just CC'd and I'm missing the
> > context.
> 
> Sorry I added you as a subject matter expert but didn't provide enough context.
> 
> The original issue [1] was perf failing to build on ARC due to perf tools needing
> a copy of unistd.h but this thread [2] was a small side issue of auto-detecting
> libc variaint in perf tools where despite uClibc tools, glibc is declared to be
> detected, due to uClibc's historical hack of defining __GLIBC__. So __GLIBC__ is
> not sufficient (and probably not the right interface to begin wtih) to ensure glibc.
> 
> [1] http://lists.infradead.org/pipermail/linux-snps-arc/2019-April/005676.html
> [2] http://lists.infradead.org/pipermail/linux-snps-arc/2019-April/005684.html

I think you misunderstood -- I'm asking what you're trying to achieve
by detecting whether the libc is glibc, rather than whether it has
some particular interface you want to conditionally use. This is a
major smell and is usually something wrong that shouldn't be done.

Rich

Detecting libc in perf (was Re: perf tools build broken after v5.1-rc1)

[Vineet Gupta]

On 4/30/19 8:12 PM, Rich Felker wrote:
>>> What are you trying to achieve? I was just CC'd and I'm missing the
>>> context.
>>
>> Sorry I added you as a subject matter expert but didn't provide enough context.
>>
>> The original issue [1] was perf failing to build on ARC due to perf tools needing
>> a copy of unistd.h but this thread [2] was a small side issue of auto-detecting
>> libc variaint in perf tools where despite uClibc tools, glibc is declared to be
>> detected, due to uClibc's historical hack of defining __GLIBC__. So __GLIBC__ is
>> not sufficient (and probably not the right interface to begin wtih) to ensure glibc.
>>
>> [1] http://lists.infradead.org/pipermail/linux-snps-arc/2019-April/005676.html
>> [2] http://lists.infradead.org/pipermail/linux-snps-arc/2019-April/005684.html
> 
> I think you misunderstood -- 

:-)

> I'm asking what you're trying to achieve
> by detecting whether the libc is glibc, rather than whether it has
> some particular interface you want to conditionally use. This is a
> major smell and is usually something wrong that shouldn't be done.

Good question indeed. Back in 2015 I initially ran into some quirks due to subtle
libc differences.  At the time perf has a fwd ref for strlcpy which exactly
matched glibc but not uClibc.  see commit  a83d869f300bf91 "(perf tools: Elide
strlcpy warning with uclibc)" or 0215d59b154 "(tools lib: Reinstate strlcpy()
header guard with __UCLIBC__)"

But this still used the libc defined symbol __UCLIBC__ or __GLIBC__

Your question however pertains to perf glibc feature check where perf generates an
alternate symbol HAVE_GLIBC_SUPPORT.

This is dubious as first of all it detects glibc even for uClibc builds.

Even of we were to improve it, there seems to be no users of this symbol.

$git grep HAVE_GLIBC_SUPPORT
perf/Makefile.config:  CFLAGS += -DHAVE_GLIBC_SUPPORT
perf/builtin-version.c: STATUS(HAVE_GLIBC_SUPPORT, glibc)

So I'd propose to remove it !

Detecting libc in perf (was Re: perf tools build broken after v5.1-rc1)

[Arnaldo Carvalho de Melo]

Em Thu, May 02, 2019 at 09:55:26AM -0700, Vineet Gupta escreveu:
> On 4/30/19 8:12 PM, Rich Felker wrote:
> >>> What are you trying to achieve? I was just CC'd and I'm missing the
> >>> context.
> >>
> >> Sorry I added you as a subject matter expert but didn't provide enough context.
> >>
> >> The original issue [1] was perf failing to build on ARC due to perf tools needing
> >> a copy of unistd.h but this thread [2] was a small side issue of auto-detecting
> >> libc variaint in perf tools where despite uClibc tools, glibc is declared to be
> >> detected, due to uClibc's historical hack of defining __GLIBC__. So __GLIBC__ is
> >> not sufficient (and probably not the right interface to begin wtih) to ensure glibc.
> >>
> >> [1] http://lists.infradead.org/pipermail/linux-snps-arc/2019-April/005676.html
> >> [2] http://lists.infradead.org/pipermail/linux-snps-arc/2019-April/005684.html
> > 
> > I think you misunderstood -- 
> 
> :-)
> 
> > I'm asking what you're trying to achieve
> > by detecting whether the libc is glibc, rather than whether it has
> > some particular interface you want to conditionally use. This is a
> > major smell and is usually something wrong that shouldn't be done.
> 
> Good question indeed. Back in 2015 I initially ran into some quirks due to subtle
> libc differences.  At the time perf has a fwd ref for strlcpy which exactly
> matched glibc but not uClibc.  see commit  a83d869f300bf91 "(perf tools: Elide
> strlcpy warning with uclibc)" or 0215d59b154 "(tools lib: Reinstate strlcpy()
> header guard with __UCLIBC__)"
> 
> But this still used the libc defined symbol __UCLIBC__ or __GLIBC__
> 
> Your question however pertains to perf glibc feature check where perf generates an
> alternate symbol HAVE_GLIBC_SUPPORT.
> 
> This is dubious as first of all it detects glibc even for uClibc builds.

 
> Even of we were to improve it, there seems to be no users of this symbol.
> 
> $git grep HAVE_GLIBC_SUPPORT
> perf/Makefile.config:  CFLAGS += -DHAVE_GLIBC_SUPPORT
> perf/builtin-version.c: STATUS(HAVE_GLIBC_SUPPORT, glibc)
> 
> So I'd propose to remove it !

This is some remnant of the past, I'll check further but will end up
just ditching it altogether as you suggest :-)

[acme at quaco perf]$ find tools/ -type f | xargs grep HAVE_GLIBC_SUPPORT
tools/perf/builtin-version.c:	STATUS(HAVE_GLIBC_SUPPORT, glibc);
tools/perf/Makefile.config:  CFLAGS += -DHAVE_GLIBC_SUPPORT
[acme at quaco perf]$

Its just this case that ends up using that feature detection program,

[acme at quaco perf]$ vim tools/perf/Makefile.config
[acme at quaco perf]$ find tools/ -type f | xargs grep feature-glibc
tools/perf/Makefile.config:    ifeq ($(feature-glibc), 1)
tools/perf/Makefile.config:ifeq ($(feature-glibc), 1)
[acme at quaco perf]$

BTW the function on it doesn't mean anything, what matters is if the
program builds or not :-)

- Arnaldo

perf tools build broken after v5.1-rc1

[Vineet Gupta]

On 4/29/19 6:18 PM, Arnaldo Carvalho de Melo wrote:
>>>
>>> Auto-detecting system features:
>>> ...                         dwarf: [ OFF ]
>>> ...            dwarf_getlocations: [ OFF ]
>>> ...                         glibc: [ on  ]
>>
>> Not related to current issue, this run uses a uClibc toolchain and yet it is
>> detecting glibc - doesn't seem right to me.
> 
> Ok, I'll improve that, I think it just tries to detect a libc, yeah,
> see:
> 
> [acme at quaco linux]$ cat tools/build/feature/test-glibc.c
> // SPDX-License-Identifier: GPL-2.0
> #include <stdlib.h>
> 
> #if !defined(__UCLIBC__)
> #include <gnu/libc-version.h>
> #else
> #define XSTR(s) STR(s)
> #define STR(s) #s
> #endif
> 
> int main(void)
> {
> #if !defined(__UCLIBC__)
> 	const char *version = gnu_get_libc_version();
> #else
> 	const char *version = XSTR(__GLIBC__) "." XSTR(__GLIBC_MINOR__);
> #endif
> 
> 	return (long)version;
> }

So we return a pointer to a string to declare a glibc build ?
But won't it do so even for other libs, even if those numbers were 0:0. I'm
confused how this feature detction works.

> [acme at quaco linux]$
> 
> [perfbuilder at 59ca4b424ded /]$ grep __GLIBC__ /arc_gnu_2017.09-rc2_prebuilt_uclibc_le_arc700_linux_install/arc-snps-linux-uclibc/sysroot/usr/include/*.h
> /arc_gnu_2017.09-rc2_prebuilt_uclibc_le_arc700_linux_install/arc-snps-linux-uclibc/sysroot/usr/include/features.h:   The macros `__GNU_LIBRARY__', `__GLIBC__', and `__GLIBC_MINOR__' are
> /arc_gnu_2017.09-rc2_prebuilt_uclibc_le_arc700_linux_install/arc-snps-linux-uclibc/sysroot/usr/include/features.h:#define	__GLIBC__	2
> /arc_gnu_2017.09-rc2_prebuilt_uclibc_le_arc700_linux_install/arc-snps-linux-uclibc/sysroot/usr/include/features.h:	((__GLIBC__ << 16) + __GLIBC_MINOR__ >= ((maj) << 16) + (min))
> [perfbuilder at 59ca4b424ded /]$
> 
> Isn't that part of uClibc?

perf tools build broken after v5.1-rc1

[Vineet Gupta]

On 4/22/19 8:31 AM, Arnaldo Carvalho de Melo wrote:
>> A quick fix for ARC will be to create our own version but I presume all existing
>> arches using generic syscall abi are affected. Thoughts ? In lack of ideas I'll
>> send out a patch for ARC.
>>
>> P.S. Why do we need the unistd.h duplication in tools directory, given it could
>> have used the in-tree unistd headers directly ?
> I have to write down the explanation and have it in a file, but we can't
> use anything in the kernel from outside tools/ to avoid adding a burden
> to kernel developers that would then have to make sure that the changes
> that they make outside tools/ don't break things living there.

That is a sound guiding principle in general but I don't agree here. unistd is
backbone of kernel user interface it has to work and can't possibly be broken even
when kernel devs add a new syscall is added or condition-alize existing one. So
adding a copy - and deferring the propagation of in-kernel unistd to usersapce
won't necessarily help with anything and it just adds the burden of keeping them
in sync. Granted we won't necessarily need all the bleeding edge (new syscall
updates) into that header, its still more work.

perf tools build broken after v5.1-rc1

[Arnd Bergmann]

On Mon, Apr 29, 2019@7:17 PM Vineet Gupta <Vineet.Gupta1@synopsys.com> wrote:
>
> On 4/22/19 8:31 AM, Arnaldo Carvalho de Melo wrote:
> >> A quick fix for ARC will be to create our own version but I presume all existing
> >> arches using generic syscall abi are affected. Thoughts ? In lack of ideas I'll
> >> send out a patch for ARC.
> >>
> >> P.S. Why do we need the unistd.h duplication in tools directory, given it could
> >> have used the in-tree unistd headers directly ?
> > I have to write down the explanation and have it in a file, but we can't
> > use anything in the kernel from outside tools/ to avoid adding a burden
> > to kernel developers that would then have to make sure that the changes
> > that they make outside tools/ don't break things living there.
>
> That is a sound guiding principle in general but I don't agree here. unistd is
> backbone of kernel user interface it has to work and can't possibly be broken even
> when kernel devs add a new syscall is added or condition-alize existing one. So
> adding a copy - and deferring the propagation of in-kernel unistd to usersapce
> won't necessarily help with anything and it just adds the burden of keeping them
> in sync. Granted we won't necessarily need all the bleeding edge (new syscall
> updates) into that header, its still more work.

I think more importantly, it seems completely broken to sync a file from
asm-generic but not the arch specific file that includes it.

The 1a787fc5ba18ac7 commit copied over the changes for arm64, but
missed all the other architectures changed in c8ce48f06503 and the
related commits.

      Arnd

perf tools build broken after v5.1-rc1

[Arnaldo Carvalho de Melo]

Em Tue, Apr 30, 2019 at 06:12:35PM +0200, Arnd Bergmann escreveu:
> On Mon, Apr 29, 2019@7:17 PM Vineet Gupta <Vineet.Gupta1@synopsys.com> wrote:
> >
> > On 4/22/19 8:31 AM, Arnaldo Carvalho de Melo wrote:
> > >> A quick fix for ARC will be to create our own version but I presume all existing
> > >> arches using generic syscall abi are affected. Thoughts ? In lack of ideas I'll
> > >> send out a patch for ARC.
> > >>
> > >> P.S. Why do we need the unistd.h duplication in tools directory, given it could
> > >> have used the in-tree unistd headers directly ?
> > > I have to write down the explanation and have it in a file, but we can't
> > > use anything in the kernel from outside tools/ to avoid adding a burden
> > > to kernel developers that would then have to make sure that the changes
> > > that they make outside tools/ don't break things living there.
> >
> > That is a sound guiding principle in general but I don't agree here. unistd is
> > backbone of kernel user interface it has to work and can't possibly be broken even
> > when kernel devs add a new syscall is added or condition-alize existing one. So
> > adding a copy - and deferring the propagation of in-kernel unistd to usersapce
> > won't necessarily help with anything and it just adds the burden of keeping them
> > in sync. Granted we won't necessarily need all the bleeding edge (new syscall
> > updates) into that header, its still more work.
> 
> I think more importantly, it seems completely broken to sync a file from
> asm-generic but not the arch specific file that includes it.
> 
> The 1a787fc5ba18ac7 commit copied over the changes for arm64, but
> missed all the other architectures changed in c8ce48f06503 and the
> related commits.

Right, I have a patch copying the missing headers, and that fixed the
build with the glibc-based toolchain, but then broke the uCLibc one :-\

I'm travelling, so coulnd't get back to this, will try as possible.

- Arnaldo

perf tools build broken after v5.1-rc1

[Vineet Gupta]

On 5/1/19 1:41 PM, Arnaldo Carvalho de Melo wrote:
>> The 1a787fc5ba18ac7 commit copied over the changes for arm64, but
>> missed all the other architectures changed in c8ce48f06503 and the
>> related commits.
> Right, I have a patch copying the missing headers, and that fixed the
> build with the glibc-based toolchain, but then broke the uCLibc one :-\

tools/perf/util/cloexec.c  #includes <sys/syscall.h> which for glibc includes
asm/unistd.h

uClibc <sys/syscall.h> OTOH #include <bits/sysnum.h> containign#define __NR_*
(generated by parsing kernel's unistd). This header does the right thing by
chekcing for redefs, but in the end we still collide with newly added
tools/arc/arc/*/**/unistd.h which doesn't have conditional definitions. I'm sure
this is not an ARC problem, any uClibc build would be affected. Do you have a arm
uclibc toolchain to test ?

All in all this is a mess. The quick band aid I can think of would be to add a
#ifndef __UCLIBC__ in tools/arch/arc/include/uapi/asm/unistd.h which is super
ugly, but in the end the solution is to get rid of this header duplicity.

-Vineet

perf tools build broken after v5.1-rc1

[Arnaldo Carvalho de Melo]

Em Wed, May 01, 2019 at 09:17:52PM +0000, Vineet Gupta escreveu:
> On 5/1/19 1:41 PM, Arnaldo Carvalho de Melo wrote:
> >> The 1a787fc5ba18ac7 commit copied over the changes for arm64, but
> >> missed all the other architectures changed in c8ce48f06503 and the
> >> related commits.
> > Right, I have a patch copying the missing headers, and that fixed the
> > build with the glibc-based toolchain, but then broke the uCLibc one :-\
 
> tools/perf/util/cloexec.c  #includes <sys/syscall.h> which for glibc includes
> asm/unistd.h
 
> uClibc <sys/syscall.h> OTOH #include <bits/sysnum.h> containign#define __NR_*
> (generated by parsing kernel's unistd). This header does the right thing by
> chekcing for redefs, but in the end we still collide with newly added
> tools/arc/arc/*/**/unistd.h which doesn't have conditional definitions. I'm sure
> this is not an ARC problem, any uClibc build would be affected. Do you have a arm
> uclibc toolchain to test ?

This solves it for fedora:29,
arc_gnu_2017.09-rc2_prebuilt_uclibc_le_arc700_linux_install,
arc_gnu_2019.03-rc1_prebuilt_uclibc_le_archs_linux_install and
arc_gnu_2019.03-rc1_prebuilt_glibc_le_archs_linux_install.

Also ok with:

  make -C tools/perf build-test

Now build testing with the full set of containers.

- Arnaldo

commit 1931594a680dba28e98b526192dd065430c850c0
Author: Arnaldo Carvalho de Melo <acme at redhat.com>
Date:   Thu May 2 09:26:23 2019 -0400

    perf tools: Remove needless asm/unistd.h include fixing build in some places
    
    We were including sys/syscall.h and asm/unistd.h, since sys/syscall.h
    includes asm/unistd.h, sometimes this leads to the redefinition of
    defines, breaking the build.
    
    Noticed on ARC with uCLibc.
    
    Cc: Adrian Hunter <adrian.hunter at intel.com>
    Cc: Arnaldo Carvalho de Melo <arnaldo.melo at gmail.com>
    Cc: Arnd Bergmann <arnd at arndb.de>
    Cc: Jiri Olsa <jolsa at kernel.org>
    Cc: Namhyung Kim <namhyung at kernel.org>
    Cc: Rich Felker <dalias at libc.org>
    Cc: Vineet Gupta <Vineet.Gupta1 at synopsys.com>
    Link: https://lkml.kernel.org/n/tip-xjpf80o64i2ko74aj2jih0qg at git.kernel.org
    Signed-off-by: Arnaldo Carvalho de Melo <acme at redhat.com>

diff --git a/tools/perf/util/cloexec.c b/tools/perf/util/cloexec.c
index ca0fff6272be..06f48312c5ed 100644
--- a/tools/perf/util/cloexec.c
+++ b/tools/perf/util/cloexec.c
@@ -7,7 +7,6 @@
 #include "asm/bug.h"
 #include "debug.h"
 #include <unistd.h>
-#include <asm/unistd.h>
 #include <sys/syscall.h>
 
 static unsigned long flag = PERF_FLAG_FD_CLOEXEC;

perf tools build broken after v5.1-rc1

[Arnaldo Carvalho de Melo]

Em Thu, May 02, 2019 at 10:36:18AM -0400, Arnaldo Carvalho de Melo escreveu:
> Em Wed, May 01, 2019 at 09:17:52PM +0000, Vineet Gupta escreveu:
> > On 5/1/19 1:41 PM, Arnaldo Carvalho de Melo wrote:
> > >> The 1a787fc5ba18ac7 commit copied over the changes for arm64, but
> > >> missed all the other architectures changed in c8ce48f06503 and the
> > >> related commits.
> > > Right, I have a patch copying the missing headers, and that fixed the
> > > build with the glibc-based toolchain, but then broke the uCLibc one :-\
>  
> > tools/perf/util/cloexec.c  #includes <sys/syscall.h> which for glibc includes
> > asm/unistd.h
>  
> > uClibc <sys/syscall.h> OTOH #include <bits/sysnum.h> containign#define __NR_*
> > (generated by parsing kernel's unistd). This header does the right thing by
> > chekcing for redefs, but in the end we still collide with newly added
> > tools/arc/arc/*/**/unistd.h which doesn't have conditional definitions. I'm sure
> > this is not an ARC problem, any uClibc build would be affected. Do you have a arm
> > uclibc toolchain to test ?
> 
> This solves it for fedora:29,
> arc_gnu_2017.09-rc2_prebuilt_uclibc_le_arc700_linux_install,
> arc_gnu_2019.03-rc1_prebuilt_uclibc_le_archs_linux_install and
> arc_gnu_2019.03-rc1_prebuilt_glibc_le_archs_linux_install.
> 
> Also ok with:
> 
>   make -C tools/perf build-test
> 
> Now build testing with the full set of containers.

So far, and the alpine ones use musl libc:

[perfbuilder at quaco ~]$ export PERF_TARBALL=http://192.168.124.1/perf/perf-5.1.0-rc6.tar.xz
[perfbuilder at quaco ~]$ time dm
   1 alpine:3.4                    : Ok   gcc (Alpine 5.3.0) 5.3.0
   2 alpine:3.5                    : Ok   gcc (Alpine 6.2.1) 6.2.1 20160822
   3 alpine:3.6                    : Ok   gcc (Alpine 6.3.0) 6.3.0
   4 alpine:3.7                    : Ok   gcc (Alpine 6.4.0) 6.4.0
   5 alpine:3.8                    : Ok   gcc (Alpine 6.4.0) 6.4.0
   6 alpine:3.9                    : Ok   gcc (Alpine 8.3.0) 8.3.0
   7 alpine:edge                   : Ok   gcc (Alpine 8.3.0) 8.3.0
   8 amazonlinux:1                 : Ok   gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-28)
   9 android-ndk:r12b-arm          : Ok   arm-linux-androideabi-gcc (GCC) 4.9.x 20150123 (prerelease)
  10 android-ndk:r15c-arm          : Ok   arm-linux-androideabi-gcc (GCC) 4.9.x 20150123 (prerelease)
  11 centos:5                      : Ok   gcc (GCC) 4.1.2 20080704 (Red Hat 4.1.2-55)
  12 centos:6                      : Ok   gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-23)
  13 centos:7                      : Ok   gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-36)
  14 debian:8                      : Ok   gcc (Debian 4.9.2-10+deb8u2) 4.9.2
  15 debian:9                      : Ok   gcc (Debian 6.3.0-18+deb9u1) 6.3.0 20170516
  16 debian:experimental           : Ok   gcc (Debian 8.3.0-6) 8.3.0
  17 debian:experimental-x-arm64   : Ok   aarch64-linux-gnu-gcc (Debian 8.3.0-4) 8.3.0
  18 debian:experimental-x-mips    : Ok   mips-linux-gnu-gcc (Debian 8.3.0-4) 8.3.0
  19 debian:experimental-x-mips64  : Ok   mips64-linux-gnuabi64-gcc (Debian 8.3.0-2) 8.3.0
  20 debian:experimental-x-mipsel  : Ok   mipsel-linux-gnu-gcc (Debian 8.3.0-4) 8.3.0
  21 fedora:20                     : Ok   gcc (GCC) 4.8.3 20140911 (Red Hat 4.8.3-7)
  22 fedora:22                     : Ok   gcc (GCC) 5.3.1 20160406 (Red Hat 5.3.1-6)
  23 fedora:23                     : Ok   gcc (GCC) 5.3.1 20160406 (Red Hat 5.3.1-6)
  24 fedora:24                     : Ok   gcc (GCC) 6.3.1 20161221 (Red Hat 6.3.1-1)
  25 fedora:24-x-ARC-uClibc        : Ok   arc-linux-gcc (ARCompact ISA Linux uClibc toolchain 2017.09-rc2) 7.1.1 20170710


- Arnaldo

perf tools build broken after v5.1-rc1

[Vineet Gupta]

On 5/2/19 7:36 AM, Arnaldo Carvalho de Melo wrote:
> Em Wed, May 01, 2019 at 09:17:52PM +0000, Vineet Gupta escreveu:
>> On 5/1/19 1:41 PM, Arnaldo Carvalho de Melo wrote:
>>>> The 1a787fc5ba18ac7 commit copied over the changes for arm64, but
>>>> missed all the other architectures changed in c8ce48f06503 and the
>>>> related commits.
>>> Right, I have a patch copying the missing headers, and that fixed the
>>> build with the glibc-based toolchain, but then broke the uCLibc one :-\
>  
>> tools/perf/util/cloexec.c  #includes <sys/syscall.h> which for glibc includes
>> asm/unistd.h
>  
>> uClibc <sys/syscall.h> OTOH #include <bits/sysnum.h> containign#define __NR_*
>> (generated by parsing kernel's unistd). This header does the right thing by
>> chekcing for redefs, but in the end we still collide with newly added
>> tools/arc/arc/*/**/unistd.h which doesn't have conditional definitions. I'm sure
>> this is not an ARC problem, any uClibc build would be affected. Do you have a arm
>> uclibc toolchain to test ?
> This solves it for fedora:29,
> arc_gnu_2017.09-rc2_prebuilt_uclibc_le_arc700_linux_install,
> arc_gnu_2019.03-rc1_prebuilt_uclibc_le_archs_linux_install and
> arc_gnu_2019.03-rc1_prebuilt_glibc_le_archs_linux_install.
>
> Also ok with:
>
>   make -C tools/perf build-test
>
> Now build testing with the full set of containers.
>
> - Arnaldo
>
> commit 1931594a680dba28e98b526192dd065430c850c0
> Author: Arnaldo Carvalho de Melo <acme at redhat.com>
> Date:   Thu May 2 09:26:23 2019 -0400
>
>     perf tools: Remove needless asm/unistd.h include fixing build in some places
>     
>     We were including sys/syscall.h and asm/unistd.h, since sys/syscall.h
>     includes asm/unistd.h, sometimes this leads to the redefinition of
>     defines, breaking the build.
>     
>     Noticed on ARC with uCLibc.

Thx for this Arnaldo.

While this takes care of immediate issues, for the long term, are you open to idea
of removing the header duplicity.
We could use a "less evil" idiom of copying only the minimal bits (since the sync
onus remains one way or the other)
e.g. I spotted below in bpf code and also seen in other ah-hoc multi arch projects

#ifdef __NR_xx
# if defined (__arch_y__)

# elif defined (__arch_z__)

# endif
#endif

perf tools build broken after v5.1-rc1

[Arnaldo Carvalho de Melo]

Em Thu, May 02, 2019 at 04:09:43PM +0000, Vineet Gupta escreveu:
> On 5/2/19 7:36 AM, Arnaldo Carvalho de Melo wrote:
> > Em Wed, May 01, 2019 at 09:17:52PM +0000, Vineet Gupta escreveu:
> >> On 5/1/19 1:41 PM, Arnaldo Carvalho de Melo wrote:
> >>>> The 1a787fc5ba18ac7 commit copied over the changes for arm64, but
> >>>> missed all the other architectures changed in c8ce48f06503 and the
> >>>> related commits.
> >>> Right, I have a patch copying the missing headers, and that fixed the
> >>> build with the glibc-based toolchain, but then broke the uCLibc one :-\
> >  
> >> tools/perf/util/cloexec.c  #includes <sys/syscall.h> which for glibc includes
> >> asm/unistd.h
> >  
> >> uClibc <sys/syscall.h> OTOH #include <bits/sysnum.h> containign#define __NR_*
> >> (generated by parsing kernel's unistd). This header does the right thing by
> >> chekcing for redefs, but in the end we still collide with newly added
> >> tools/arc/arc/*/**/unistd.h which doesn't have conditional definitions. I'm sure
> >> this is not an ARC problem, any uClibc build would be affected. Do you have a arm
> >> uclibc toolchain to test ?
> > This solves it for fedora:29,
> > arc_gnu_2017.09-rc2_prebuilt_uclibc_le_arc700_linux_install,
> > arc_gnu_2019.03-rc1_prebuilt_uclibc_le_archs_linux_install and
> > arc_gnu_2019.03-rc1_prebuilt_glibc_le_archs_linux_install.
> >
> > Also ok with:
> >
> >   make -C tools/perf build-test
> >
> > Now build testing with the full set of containers.
> >
> > - Arnaldo
> >
> > commit 1931594a680dba28e98b526192dd065430c850c0
> > Author: Arnaldo Carvalho de Melo <acme at redhat.com>
> > Date:   Thu May 2 09:26:23 2019 -0400
> >
> >     perf tools: Remove needless asm/unistd.h include fixing build in some places
> >     
> >     We were including sys/syscall.h and asm/unistd.h, since sys/syscall.h
> >     includes asm/unistd.h, sometimes this leads to the redefinition of
> >     defines, breaking the build.
> >     
> >     Noticed on ARC with uCLibc.
> 
> Thx for this Arnaldo.
> 
> While this takes care of immediate issues, for the long term, are you open to idea
> of removing the header duplicity.

In the beginning we used the kernel headers directly, then, acting on
advice/complaints from Linus about tooling breaking when changes were
made in the kernel sources we were using directly, we moved to have
copies and notice when things change so that we could think about what
changed and act accordingly, without putting the burden to the kernel
developers to keep tools/ building, I want to keep it that way.

Now you say, validly, that there are bits that are designed to be used
by userspace, so for those, we should go back to not copying and using
it direcly, elliminating the duplicity you don't like.

I don't know, I'm used to the duplicity and the checks, not breaking
tools even when kernel developers make mistakes in the UAPI headers,
tools/perf is self container wrt the latest and greatest stuff not
present in older environments, and the onus is on perf developers to do
the sync.

This specific issue here happened because I made a mistake, which I
fixed when reported, now I have three containers for cross building for
ARC, two versions for the uCLibc based toolchain, one for the glibc one,
libnuma, elfutils and zlib are cross build there, so should make it less
likely problems like this will happen again.

> We could use a "less evil" idiom of copying only the minimal bits (since the sync
> onus remains one way or the other)
> e.g. I spotted below in bpf code and also seen in other ah-hoc multi arch projects
 
> #ifdef __NR_xx
> # if defined (__arch_y__)
> 
> # elif defined (__arch_z__)
> 
> # endif
> #endif

- Arnaldo

BTW: since the last report:

  25 fedora:24-x-ARC-uClibc        : Ok   arc-linux-gcc (ARCompact ISA Linux uClibc toolchain 2017.09-rc2) 7.1.1 20170710
  26 fedora:25                     : Ok   gcc (GCC) 6.4.1 20170727 (Red Hat 6.4.1-1)
  27 fedora:26                     : Ok   gcc (GCC) 7.3.1 20180130 (Red Hat 7.3.1-2)
  28 fedora:28                     : Ok   gcc (GCC) 8.3.1 20190223 (Red Hat 8.3.1-2)
  29 fedora:29                     : Ok   gcc (GCC) 8.3.1 20190223 (Red Hat 8.3.1-2)
  30 fedora:30                     : Ok   gcc (GCC) 9.0.1 20190312 (Red Hat 9.0.1-0.10)
  31 fedora:30-x-ARC-glibc         : Ok   arc-linux-gcc (ARC HS GNU/Linux glibc toolchain 2019.03-rc1) 8.3.1 20190225
  32 fedora:30-x-ARC-uClibc        : Ok   arc-linux-gcc (ARCv2 ISA Linux uClibc toolchain 2019.03-rc1) 8.3.1 20190225

perf tools build broken after v5.1-rc1

[Vineet Gupta]

On 5/2/19 9:41 AM, Arnaldo Carvalho de Melo wrote:
>> While this takes care of immediate issues, for the long term, are you open to idea
>> of removing the header duplicity.
>
> In the beginning we used the kernel headers directly, then, acting on
> advice/complaints from Linus about tooling breaking when changes were
> made in the kernel sources we were using directly, we moved to have
> copies and notice when things change so that we could think about what
> changed and act accordingly, without putting the burden to the kernel
> developers to keep tools/ building, I want to keep it that way.

Sure, and the reduced duplicity I propose doesn't change that in any way. The onus
is still on perf maintainers to copy over any unistd changes - in the new regime,
it will be just lesser since we only care about a handful of syscalls, not the
entire unistd.


> Now you say, validly, that there are bits that are designed to be used
> by userspace, so for those, we should go back to not copying and using
> it direcly, elliminating the duplicity you don't like.

Indeed.

> I don't know, I'm used to the duplicity and the checks,

:-)

> not breaking
> tools even when kernel developers make mistakes in the UAPI headers,

Not sure how replacing the full header with just a small hunk, out of same header
out-of-line will change anything or cause any more breakage.

> tools/perf is self container wrt the latest and greatest stuff not
> present in older environments, and the onus is on perf developers to do
> the sync.

Sure it is, I'm proposing to make their work less, no more.


> This specific issue here happened because I made a mistake, which I
> fixed when reported,

Exactly, it was a genuien mistake with a super prompt followup - your promptness
is really appreciated and emulation worthy for other maintainers including myself ;-)

> now I have three containers for cross building for
> ARC, two versions for the uCLibc based toolchain, one for the glibc one,
> libnuma, elfutils and zlib are cross build there, so should make it less
> likely problems like this will happen again.

Ok, well lets leave it at that for now then.


>> We could use a "less evil" idiom of copying only the minimal bits (since the sync
>> onus remains one way or the other)
>> e.g. I spotted below in bpf code and also seen in other ah-hoc multi arch projects
>  
>> #ifdef __NR_xx
>> # if defined (__arch_y__)
>>
>> # elif defined (__arch_z__)
>>
>> # endif
>> #endif

Thx,
-Vineet