[BUG] perf report/annotate: consuming too many file descriptors

[Stephane Eranian <eranian@google.com>]

Hi,


I was testing 3.14-rc2 (tip.git) perf and ran into a serious problem
with report/annotate on a collection with lots of shared libraries (500).

Perf ran out of file descriptors (ulimit set to 1024). It did not print
an error message, but simply refused to proceed to objdump.

I ran strace and discovered it was running out of descriptors to
my big surprise! I narrowed it down with strace -etrace=open.
I saw an appalling result.

My .so files were opened at least 4 times each, each
allocated an fd that was kept open, each open incur a read of the ELF
headers. So each dso was consuming 4 fd.

Why?

Because of what's going on in dso__load() when perf iterates
over the binary_type_symtab[]. It tries a bunch of types. For
each match, symsrc_init() is invoked to read the ELF image.
The fd opened there is kept open (for later loading).

The first problem is why is dso__read_binary_type_filename()
blindly returning success on many types. For my files, I got matches
on DEBUGLINK, SYSTEM_DSO_PATH, GUEST_KMODULE,
SYSTEM_PATH_KMODULE.

I have a tendency to believe that the dso__read_binary_type_filename()
meaning has been abused to stash things that do not really
belong there.

Furthermore, I believe many of the type matches do NOT need an ELF
read and certainly not one that keeps a descriptor opened.

This problem is not just about consuming too many fd, it is also about
execution overhead. Why read the ELF headers 4 times?

The current situation makes reporting on large collection impossible for
regular users. I don't quite know how to best address this issue. One way
I found would be for  dso__read_binary_type_filename() to return a value
meaning success but no ELF read. Not sure would be correct, though.

Any ideas on how to fix this?

Thanks.