* Re: [PATCH v2 0/4] perf: enable compression of record mode trace to save storage space
[not found] <b834df0e-26b5-3f8c-7a43-18f675fb7434@linux.intel.com>
@ 2019-01-29 10:45 ` Arnaldo Carvalho de Melo
2019-01-29 10:53 ` Arnaldo Carvalho de Melo
2019-01-29 17:25 ` Andi Kleen
0 siblings, 2 replies; 12+ messages in thread
From: Arnaldo Carvalho de Melo @ 2019-01-29 10:45 UTC (permalink / raw)
To: Alexey Budankov
Cc: Ingo Molnar, Peter Zijlstra, Jiri Olsa, Namhyung Kim,
Alexander Shishkin, Andi Kleen, linux-kernel
Em Mon, Jan 28, 2019 at 09:40:28AM +0300, Alexey Budankov escreveu:
> The patch set implements runtime trace compression for record mode and
> trace file decompression for report mode. Zstandard API [1] is used for
> compression/decompression of data that come from perf_events kernel
Interesting, wasn't aware of this zstd library, I wonder if we can add
it and switch the other compression libraries we link against, so that
we're not adding one more library to the dep list of perf but removing
some instead, do you think this would be possible?
$ ldd ~/bin/perf | wc -l
30
$ ldd ~/bin/perf | grep z
liblzma.so.5 => /lib64/liblzma.so.5 (0x00007f3dcc356000)
libz.so.1 => /lib64/libz.so.1 (0x00007f3dcb2aa000)
libbz2.so.1 => /lib64/libbz2.so.1 (0x00007f3dcb218000)
$
Humm, from the github page it says:
-----
The project is provided as an open-source dual BSD and GPLv2 licensed C
library, and a command line utility producing and decoding .zst, .gz,
.xz and .lz4 files. Should your project require another programming
language, a list of known ports and bindings is provided on Zstandard
homepage.
-----
So it would cover just liblzma and libz, right?
- Arnaldo
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v2 0/4] perf: enable compression of record mode trace to save storage space
2019-01-29 10:45 ` [PATCH v2 0/4] perf: enable compression of record mode trace to save storage space Arnaldo Carvalho de Melo
@ 2019-01-29 10:53 ` Arnaldo Carvalho de Melo
2019-01-29 11:39 ` Alexey Budankov
2019-01-29 17:25 ` Andi Kleen
1 sibling, 1 reply; 12+ messages in thread
From: Arnaldo Carvalho de Melo @ 2019-01-29 10:53 UTC (permalink / raw)
To: Alexey Budankov
Cc: Ingo Molnar, Peter Zijlstra, Jiri Olsa, Namhyung Kim,
Alexander Shishkin, Andi Kleen, linux-kernel
Em Tue, Jan 29, 2019 at 11:45:43AM +0100, Arnaldo Carvalho de Melo escreveu:
> Em Mon, Jan 28, 2019 at 09:40:28AM +0300, Alexey Budankov escreveu:
> > The patch set implements runtime trace compression for record mode and
> > trace file decompression for report mode. Zstandard API [1] is used for
> > compression/decompression of data that come from perf_events kernel
>
> Interesting, wasn't aware of this zstd library, I wonder if we can add
> it and switch the other compression libraries we link against, so that
> we're not adding one more library to the dep list of perf but removing
> some instead, do you think this would be possible?
>
> $ ldd ~/bin/perf | wc -l
> 30
> $ ldd ~/bin/perf | grep z
> liblzma.so.5 => /lib64/liblzma.so.5 (0x00007f3dcc356000)
> libz.so.1 => /lib64/libz.so.1 (0x00007f3dcb2aa000)
> libbz2.so.1 => /lib64/libbz2.so.1 (0x00007f3dcb218000)
> $
>
> Humm, from the github page it says:
>
> -----
> The project is provided as an open-source dual BSD and GPLv2 licensed C
> library, and a command line utility producing and decoding .zst, .gz,
> .xz and .lz4 files. Should your project require another programming
> language, a list of known ports and bindings is provided on Zstandard
> homepage.
> -----
>
> So it would cover just liblzma and libz, right?
Nevermind;
[acme@quaco perf]$ zstdcat ~/git/perf/perf-5.0.0-rc2.tar.xz
zstd: /home/acme/git/perf/perf-5.0.0-rc2.tar.xz: xz/lzma file cannot be uncompressed (zstd compiled without HAVE_LZMA) -- ignored
So it handles those formats, _if_ linked with those libraries, duh.
- Arnaldo
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v2 0/4] perf: enable compression of record mode trace to save storage space
2019-01-29 10:53 ` Arnaldo Carvalho de Melo
@ 2019-01-29 11:39 ` Alexey Budankov
2019-01-29 12:13 ` Arnaldo Carvalho de Melo
0 siblings, 1 reply; 12+ messages in thread
From: Alexey Budankov @ 2019-01-29 11:39 UTC (permalink / raw)
To: Arnaldo Carvalho de Melo
Cc: Ingo Molnar, Peter Zijlstra, Jiri Olsa, Namhyung Kim,
Alexander Shishkin, Andi Kleen, linux-kernel
Hi,
On 29.01.2019 13:53, Arnaldo Carvalho de Melo wrote:
> Em Tue, Jan 29, 2019 at 11:45:43AM +0100, Arnaldo Carvalho de Melo escreveu:
>> Em Mon, Jan 28, 2019 at 09:40:28AM +0300, Alexey Budankov escreveu:
>>> The patch set implements runtime trace compression for record mode and
>>> trace file decompression for report mode. Zstandard API [1] is used for
>>> compression/decompression of data that come from perf_events kernel
>>
>> Interesting, wasn't aware of this zstd library, I wonder if we can add
>> it and switch the other compression libraries we link against, so that
>> we're not adding one more library to the dep list of perf but removing
>> some instead, do you think this would be possible?
Replacing of incorporated compression APIs was not evaluated or tested in
the scope of this patch set work. However according to their numbers in the
docs and the numbers that we have got during testing Zstd API outperforms
the exiting compression libraries as in terms of speed as in terms of
compression ratio (at least libz). Backward compatibility needs to be taken
into account so that old perf files would open by newer perf tool versions.
-Alexey
>>
>> $ ldd ~/bin/perf | wc -l
>> 30
>> $ ldd ~/bin/perf | grep z
>> liblzma.so.5 => /lib64/liblzma.so.5 (0x00007f3dcc356000)
>> libz.so.1 => /lib64/libz.so.1 (0x00007f3dcb2aa000)
>> libbz2.so.1 => /lib64/libbz2.so.1 (0x00007f3dcb218000)
>> $
>>
>> Humm, from the github page it says:
>>
>> -----
>> The project is provided as an open-source dual BSD and GPLv2 licensed C
>> library, and a command line utility producing and decoding .zst, .gz,
>> .xz and .lz4 files. Should your project require another programming
>> language, a list of known ports and bindings is provided on Zstandard
>> homepage.
>> -----
>>
>> So it would cover just liblzma and libz, right?
>
> Nevermind;
>
> [acme@quaco perf]$ zstdcat ~/git/perf/perf-5.0.0-rc2.tar.xz
> zstd: /home/acme/git/perf/perf-5.0.0-rc2.tar.xz: xz/lzma file cannot be uncompressed (zstd compiled without HAVE_LZMA) -- ignored
>
> So it handles those formats, _if_ linked with those libraries, duh.
>
> - Arnaldo
>
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v2 0/4] perf: enable compression of record mode trace to save storage space
2019-01-29 11:39 ` Alexey Budankov
@ 2019-01-29 12:13 ` Arnaldo Carvalho de Melo
2019-01-29 16:39 ` Alexey Budankov
0 siblings, 1 reply; 12+ messages in thread
From: Arnaldo Carvalho de Melo @ 2019-01-29 12:13 UTC (permalink / raw)
To: Alexey Budankov
Cc: Ingo Molnar, Peter Zijlstra, Jiri Olsa, Namhyung Kim,
Alexander Shishkin, Andi Kleen, linux-kernel
Em Tue, Jan 29, 2019 at 02:39:00PM +0300, Alexey Budankov escreveu:
> Hi,
> On 29.01.2019 13:53, Arnaldo Carvalho de Melo wrote:
> > Em Tue, Jan 29, 2019 at 11:45:43AM +0100, Arnaldo Carvalho de Melo escreveu:
> >> Em Mon, Jan 28, 2019 at 09:40:28AM +0300, Alexey Budankov escreveu:
> >>> The patch set implements runtime trace compression for record mode and
> >>> trace file decompression for report mode. Zstandard API [1] is used for
> >>> compression/decompression of data that come from perf_events kernel
> >>
> >> Interesting, wasn't aware of this zstd library, I wonder if we can add
> >> it and switch the other compression libraries we link against, so that
> >> we're not adding one more library to the dep list of perf but removing
> >> some instead, do you think this would be possible?
>
> Replacing of incorporated compression APIs was not evaluated or tested in
> the scope of this patch set work. However according to their numbers in the
> docs and the numbers that we have got during testing Zstd API outperforms
> the exiting compression libraries as in terms of speed as in terms of
> compression ratio (at least libz). Backward compatibility needs to be taken
> into account so that old perf files would open by newer perf tool versions.
Right, I'm not talking in the scope of this patch, its just that while
looking at it, I notice that we're adding yet another compression
library and its description seemed to imply it would support the other
compression formats, which I've learned its not the case, so nevermind.
I'm not talking about using just zstd, as what we mostly do with the
compression libraries is to decompress, not compress, for instance, we
need to uncompress kernel modules to get to its symbols, do annotation
with it, etc.
- Arnaldo
> -Alexey
>
> >>
> >> $ ldd ~/bin/perf | wc -l
> >> 30
> >> $ ldd ~/bin/perf | grep z
> >> liblzma.so.5 => /lib64/liblzma.so.5 (0x00007f3dcc356000)
> >> libz.so.1 => /lib64/libz.so.1 (0x00007f3dcb2aa000)
> >> libbz2.so.1 => /lib64/libbz2.so.1 (0x00007f3dcb218000)
> >> $
> >>
> >> Humm, from the github page it says:
> >>
> >> -----
> >> The project is provided as an open-source dual BSD and GPLv2 licensed C
> >> library, and a command line utility producing and decoding .zst, .gz,
> >> .xz and .lz4 files. Should your project require another programming
> >> language, a list of known ports and bindings is provided on Zstandard
> >> homepage.
> >> -----
> >>
> >> So it would cover just liblzma and libz, right?
> >
> > Nevermind;
> >
> > [acme@quaco perf]$ zstdcat ~/git/perf/perf-5.0.0-rc2.tar.xz
> > zstd: /home/acme/git/perf/perf-5.0.0-rc2.tar.xz: xz/lzma file cannot be uncompressed (zstd compiled without HAVE_LZMA) -- ignored
> >
> > So it handles those formats, _if_ linked with those libraries, duh.
> >
> > - Arnaldo
> >
--
- Arnaldo
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v2 0/4] perf: enable compression of record mode trace to save storage space
2019-01-29 12:13 ` Arnaldo Carvalho de Melo
@ 2019-01-29 16:39 ` Alexey Budankov
0 siblings, 0 replies; 12+ messages in thread
From: Alexey Budankov @ 2019-01-29 16:39 UTC (permalink / raw)
To: Arnaldo Carvalho de Melo
Cc: Ingo Molnar, Peter Zijlstra, Jiri Olsa, Namhyung Kim,
Alexander Shishkin, Andi Kleen, linux-kernel
On 29.01.2019 15:13, Arnaldo Carvalho de Melo wrote:
> Em Tue, Jan 29, 2019 at 02:39:00PM +0300, Alexey Budankov escreveu:
>> Hi,
>> On 29.01.2019 13:53, Arnaldo Carvalho de Melo wrote:
>>> Em Tue, Jan 29, 2019 at 11:45:43AM +0100, Arnaldo Carvalho de Melo escreveu:
>>>> Em Mon, Jan 28, 2019 at 09:40:28AM +0300, Alexey Budankov escreveu:
>>>>> The patch set implements runtime trace compression for record mode and
>>>>> trace file decompression for report mode. Zstandard API [1] is used for
>>>>> compression/decompression of data that come from perf_events kernel
>>>>
>>>> Interesting, wasn't aware of this zstd library, I wonder if we can add
>>>> it and switch the other compression libraries we link against, so that
>>>> we're not adding one more library to the dep list of perf but removing
>>>> some instead, do you think this would be possible?
>>
>> Replacing of incorporated compression APIs was not evaluated or tested in
>> the scope of this patch set work. However according to their numbers in the
>> docs and the numbers that we have got during testing Zstd API outperforms
>> the exiting compression libraries as in terms of speed as in terms of
>> compression ratio (at least libz). Backward compatibility needs to be taken
>> into account so that old perf files would open by newer perf tool versions.
>
> Right, I'm not talking in the scope of this patch, its just that while
> looking at it, I notice that we're adding yet another compression
> library and its description seemed to imply it would support the other
> compression formats, which I've learned its not the case, so nevermind.
>
> I'm not talking about using just zstd, as what we mostly do with the
> compression libraries is to decompress, not compress, for instance, we
> need to uncompress kernel modules to get to its symbols, do annotation
> with it, etc.
Well, yes, having single compression/decompression API implementation that
would handle different compression formats and scenarios sounds reasonable
from support cost perspective.
It looks like Zstandard library could provide such capabilities.
The library is well supported at the moment so fixes and extensions are
released timely enough.
-Alexey
>
> - Arnaldo
>
>> -Alexey
>>
>>>>
>>>> $ ldd ~/bin/perf | wc -l
>>>> 30
>>>> $ ldd ~/bin/perf | grep z
>>>> liblzma.so.5 => /lib64/liblzma.so.5 (0x00007f3dcc356000)
>>>> libz.so.1 => /lib64/libz.so.1 (0x00007f3dcb2aa000)
>>>> libbz2.so.1 => /lib64/libbz2.so.1 (0x00007f3dcb218000)
>>>> $
>>>>
>>>> Humm, from the github page it says:
>>>>
>>>> -----
>>>> The project is provided as an open-source dual BSD and GPLv2 licensed C
>>>> library, and a command line utility producing and decoding .zst, .gz,
>>>> .xz and .lz4 files. Should your project require another programming
>>>> language, a list of known ports and bindings is provided on Zstandard
>>>> homepage.
>>>> -----
>>>>
>>>> So it would cover just liblzma and libz, right?
>>>
>>> Nevermind;
>>>
>>> [acme@quaco perf]$ zstdcat ~/git/perf/perf-5.0.0-rc2.tar.xz
>>> zstd: /home/acme/git/perf/perf-5.0.0-rc2.tar.xz: xz/lzma file cannot be uncompressed (zstd compiled without HAVE_LZMA) -- ignored
>>>
>>> So it handles those formats, _if_ linked with those libraries, duh.
>>>
>>> - Arnaldo
>>>
>
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v2 0/4] perf: enable compression of record mode trace to save storage space
2019-01-29 10:45 ` [PATCH v2 0/4] perf: enable compression of record mode trace to save storage space Arnaldo Carvalho de Melo
2019-01-29 10:53 ` Arnaldo Carvalho de Melo
@ 2019-01-29 17:25 ` Andi Kleen
1 sibling, 0 replies; 12+ messages in thread
From: Andi Kleen @ 2019-01-29 17:25 UTC (permalink / raw)
To: Arnaldo Carvalho de Melo
Cc: Alexey Budankov, Ingo Molnar, Peter Zijlstra, Jiri Olsa,
Namhyung Kim, Alexander Shishkin, linux-kernel
On Tue, Jan 29, 2019 at 11:45:43AM +0100, Arnaldo Carvalho de Melo wrote:
> Em Mon, Jan 28, 2019 at 09:40:28AM +0300, Alexey Budankov escreveu:
> > The patch set implements runtime trace compression for record mode and
> > trace file decompression for report mode. Zstandard API [1] is used for
> > compression/decompression of data that come from perf_events kernel
>
> Interesting, wasn't aware of this zstd library, I wonder if we can add
> it and switch the other compression libraries we link against, so that
> we're not adding one more library to the dep list of perf but removing
> some instead, do you think this would be possible?
Sadly we can't because perf needs the existing libraries to uncompress
modules, and they might be compressed with zlib or bzip2. zstd cannot
uncompress those formats.
-Andi
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH v2 0/4] perf: enable compression of record mode trace to save storage space
@ 2019-02-11 20:17 Alexey Budankov
2019-02-12 12:27 ` Arnaldo Carvalho de Melo
0 siblings, 1 reply; 12+ messages in thread
From: Alexey Budankov @ 2019-02-11 20:17 UTC (permalink / raw)
To: Arnaldo Carvalho de Melo, Ingo Molnar, Peter Zijlstra
Cc: Jiri Olsa, Namhyung Kim, Alexander Shishkin, Andi Kleen, linux-kernel
This is the rebase to the tip of Arnaldo's perf/core repository.
The patch set implements runtime trace compression for record mode and
trace file decompression for report mode. Zstandard API [1] is used for
compression/decompression of data that come from perf_events kernel
data buffers.
Realized -z,--compression_level=n option provides ~3-5x avg. trace file
size reduction on variety of tested workloads what saves user storage
space on larger server systems where trace file size can easily reach
several tens or even hundreds of GiBs, especially when profiling with
dwarf-based stacks and tracing of context-switches.
--mmap-flush option can be used to avoid compressing every single byte
of data and increase compression ratio at the same time lowering tool
runtime overhead.
$ tools/perf/perf record -z 1 -e cycles -- matrix.gcc
$ tools/perf/perf record -z 1 --mmap-flush 1024 -e cycles -- matrix.gcc
$ tools/perf/perf record -z 1 --mmap-flush 1024 --aio -e cycles -- matrix.gcc
The compression functionality can be disabled from the command line
using NO_LIBZSTD define and Zstandard sources can be overridden using
value of LIBZSTD_DIR define:
$ make -C tools/perf NO_LIBZSTD=1 clean all
$ make -C tools/perf LIBZSTD_DIR=/path/to/zstd-1.3.7 clean all
---
Alexey Budankov (4):
feature: realize libzstd check, LIBZSTD_DIR and NO_LIBZSTD defines
perf record: implement -z=<level> and --mmap-flush=<thres> options
perf record: enable runtime trace compression
perf report: support record trace file decompression
tools/build/Makefile.feature | 6 +-
tools/build/feature/Makefile | 6 +-
tools/build/feature/test-all.c | 5 +
tools/build/feature/test-libzstd.c | 12 +
tools/perf/Documentation/perf-record.txt | 9 +
tools/perf/Makefile.config | 20 ++
tools/perf/Makefile.perf | 3 +
tools/perf/builtin-record.c | 167 +++++++++++---
tools/perf/builtin-report.c | 5 +-
tools/perf/perf.h | 2 +
tools/perf/util/env.h | 10 +
tools/perf/util/event.c | 1 +
tools/perf/util/event.h | 7 +
tools/perf/util/evlist.c | 6 +-
tools/perf/util/evlist.h | 3 +-
tools/perf/util/header.c | 45 +++-
tools/perf/util/header.h | 1 +
tools/perf/util/mmap.c | 265 +++++++++++++---------
tools/perf/util/mmap.h | 31 ++-
tools/perf/util/session.c | 271 ++++++++++++++++++++++-
tools/perf/util/session.h | 26 +++
tools/perf/util/tool.h | 2 +
22 files changed, 742 insertions(+), 161 deletions(-)
create mode 100644 tools/build/feature/test-libzstd.c
---
Changes in v2:
- moved compression/decompression code to session layer
- enabled allocation aio data buffers for compression
- enabled trace compression for serial trace streaming
---
[1] https://github.com/facebook/zstd
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v2 0/4] perf: enable compression of record mode trace to save storage space
2019-02-11 20:17 Alexey Budankov
@ 2019-02-12 12:27 ` Arnaldo Carvalho de Melo
2019-02-12 14:06 ` Alexey Budankov
0 siblings, 1 reply; 12+ messages in thread
From: Arnaldo Carvalho de Melo @ 2019-02-12 12:27 UTC (permalink / raw)
To: Alexey Budankov
Cc: Ingo Molnar, Peter Zijlstra, Jiri Olsa, Namhyung Kim,
Alexander Shishkin, Andi Kleen, linux-kernel
Em Mon, Feb 11, 2019 at 11:17:14PM +0300, Alexey Budankov escreveu:
> This is the rebase to the tip of Arnaldo's perf/core repository.
> The patch set implements runtime trace compression for record mode and
> trace file decompression for report mode. Zstandard API [1] is used for
> compression/decompression of data that come from perf_events kernel
> data buffers.
> Realized -z,--compression_level=n option provides ~3-5x avg. trace file
Realized? Perhaps "Using the -z/--compression_level=n options provides
3.5x average size reduction on..."
> size reduction on variety of tested workloads what saves user storage
> space on larger server systems where trace file size can easily reach
> several tens or even hundreds of GiBs, especially when profiling with
> dwarf-based stacks and tracing of context-switches.
> --mmap-flush option can be used to avoid compressing every single byte
What is the default for --mmap-flush when -z is used? Does it vary with
the compression level? Can we expect something that is reasonable or is
using --mmap-flush always needed? If so can we do away with that by
using some sensible default according to the -z level used?
> of data and increase compression ratio at the same time lowering tool
> runtime overhead.
>
> $ tools/perf/perf record -z 1 -e cycles -- matrix.gcc
> $ tools/perf/perf record -z 1 --mmap-flush 1024 -e cycles -- matrix.gcc
> $ tools/perf/perf record -z 1 --mmap-flush 1024 --aio -e cycles -- matrix.gcc
Please avoid having these explanations just on the cover letter since it
will not get recorded in the git history for the project, please add
examples with output from real workloads, preferably providing real
cases so that other people can readily replicate your results when
having access to similar large machines.
> The compression functionality can be disabled from the command line
> using NO_LIBZSTD define and Zstandard sources can be overridden using
> value of LIBZSTD_DIR define:
>
> $ make -C tools/perf NO_LIBZSTD=1 clean all
> $ make -C tools/perf LIBZSTD_DIR=/path/to/zstd-1.3.7 clean all
Showing the relevant output of the these commands before/after
installing the libzstd library is useful as well. From what you use
above is it fair to conclude that there are no readily made packages for
this library and we need to get it from github and build it ourselves?
On my notebook, a Fedora 29 system:
[root@quaco ~]# dnf search zstd
========================================================== Name & Summary Matched: zstd ==========================================================
zstd.x86_64 : Zstd compression library
zstd.x86_64 : Zstd compression library
libzstd.x86_64 : Zstd shared library
libzstd.i686 : Zstd shared library
libzstd.x86_64 : Zstd shared library
libzstd-devel.i686 : Header files for Zstd library
libzstd-devel.x86_64 : Header files for Zstd library
============================================================= Summary Matched: zstd ==============================================================
fedora-repo-zdicts.noarch : Zstd dictionaries for Fedora repository metadata
[root@quaco ~]#
[root@quaco ~]# dnf install libzstd-devel
<SNIP>
libzstd-devel x86_64 1.3.8-1.fc29 updates 39 k
<SNIP>
So, do I need some specific 1.3.7 version? Is that why you used that
LIBZSTD_DIR in your example?
I'm trying these steps, but please consider following these suggestions
in future posts,
And thanks for working on this!
- Arnaldo
> ---
> Alexey Budankov (4):
> feature: realize libzstd check, LIBZSTD_DIR and NO_LIBZSTD defines
> perf record: implement -z=<level> and --mmap-flush=<thres> options
> perf record: enable runtime trace compression
> perf report: support record trace file decompression
>
> tools/build/Makefile.feature | 6 +-
> tools/build/feature/Makefile | 6 +-
> tools/build/feature/test-all.c | 5 +
> tools/build/feature/test-libzstd.c | 12 +
> tools/perf/Documentation/perf-record.txt | 9 +
> tools/perf/Makefile.config | 20 ++
> tools/perf/Makefile.perf | 3 +
> tools/perf/builtin-record.c | 167 +++++++++++---
> tools/perf/builtin-report.c | 5 +-
> tools/perf/perf.h | 2 +
> tools/perf/util/env.h | 10 +
> tools/perf/util/event.c | 1 +
> tools/perf/util/event.h | 7 +
> tools/perf/util/evlist.c | 6 +-
> tools/perf/util/evlist.h | 3 +-
> tools/perf/util/header.c | 45 +++-
> tools/perf/util/header.h | 1 +
> tools/perf/util/mmap.c | 265 +++++++++++++---------
> tools/perf/util/mmap.h | 31 ++-
> tools/perf/util/session.c | 271 ++++++++++++++++++++++-
> tools/perf/util/session.h | 26 +++
> tools/perf/util/tool.h | 2 +
> 22 files changed, 742 insertions(+), 161 deletions(-)
> create mode 100644 tools/build/feature/test-libzstd.c
>
> ---
> Changes in v2:
> - moved compression/decompression code to session layer
> - enabled allocation aio data buffers for compression
> - enabled trace compression for serial trace streaming
>
> ---
> [1] https://github.com/facebook/zstd
--
- Arnaldo
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v2 0/4] perf: enable compression of record mode trace to save storage space
2019-02-12 12:27 ` Arnaldo Carvalho de Melo
@ 2019-02-12 14:06 ` Alexey Budankov
0 siblings, 0 replies; 12+ messages in thread
From: Alexey Budankov @ 2019-02-12 14:06 UTC (permalink / raw)
To: Arnaldo Carvalho de Melo
Cc: Ingo Molnar, Peter Zijlstra, Jiri Olsa, Namhyung Kim,
Alexander Shishkin, Andi Kleen, linux-kernel
On 12.02.2019 15:27, Arnaldo Carvalho de Melo wrote:
> Em Mon, Feb 11, 2019 at 11:17:14PM +0300, Alexey Budankov escreveu:
>> This is the rebase to the tip of Arnaldo's perf/core repository.
>
>> The patch set implements runtime trace compression for record mode and
>> trace file decompression for report mode. Zstandard API [1] is used for
>> compression/decompression of data that come from perf_events kernel
>> data buffers.
>
>> Realized -z,--compression_level=n option provides ~3-5x avg. trace file
>
> Realized? Perhaps "Using the -z/--compression_level=n options provides
> 3.5x average size reduction on..."
Yep, wording could be adjusted. Thanks!
>
>> size reduction on variety of tested workloads what saves user storage
>> space on larger server systems where trace file size can easily reach
>> several tens or even hundreds of GiBs, especially when profiling with
>> dwarf-based stacks and tracing of context-switches.
>
>> --mmap-flush option can be used to avoid compressing every single byte
>
> What is the default for --mmap-flush when -z is used? Does it vary with
> the compression level? Can we expect something that is reasonable or is
> using --mmap-flush always needed? If so can we do away with that by
> using some sensible default according to the -z level used?
Default value is 1 what is equal to the current perf record implementation.
The option is independent from -z setting and doesn't vary with compression
level. Default could be different if -z is specified.
>
>> of data and increase compression ratio at the same time lowering tool
>> runtime overhead.
>>
>> $ tools/perf/perf record -z 1 -e cycles -- matrix.gcc
>> $ tools/perf/perf record -z 1 --mmap-flush 1024 -e cycles -- matrix.gcc
>> $ tools/perf/perf record -z 1 --mmap-flush 1024 --aio -e cycles -- matrix.gcc
>
> Please avoid having these explanations just on the cover letter since it
> will not get recorded in the git history for the project, please add
> examples with output from real workloads, preferably providing real
> cases so that other people can readily replicate your results when
> having access to similar large machines.
Will do. Thanks!
>
>> The compression functionality can be disabled from the command line
>> using NO_LIBZSTD define and Zstandard sources can be overridden using
>> value of LIBZSTD_DIR define:
>>
>> $ make -C tools/perf NO_LIBZSTD=1 clean all
>> $ make -C tools/perf LIBZSTD_DIR=/path/to/zstd-1.3.7 clean all
>
> Showing the relevant output of the these commands before/after
> installing the libzstd library is useful as well. From what you use
> above is it fair to conclude that there are no readily made packages for
> this library and we need to get it from github and build it ourselves?
If your system has some version of the package preinstalled then the build
system finds and uses it during the build. Auto detection feature status is
reported just before compilation starts.
If you still prefer to compile with some newer version that is not
preinstalled, and you don't want upgrading your system with newer version,
you have capability to download, build and then refer the compilation to
the newer just built library version using LIBZSTD_DIR.
>
> On my notebook, a Fedora 29 system:
>
> [root@quaco ~]# dnf search zstd
> ========================================================== Name & Summary Matched: zstd ==========================================================
> zstd.x86_64 : Zstd compression library
> zstd.x86_64 : Zstd compression library
> libzstd.x86_64 : Zstd shared library
> libzstd.i686 : Zstd shared library
> libzstd.x86_64 : Zstd shared library
> libzstd-devel.i686 : Header files for Zstd library
> libzstd-devel.x86_64 : Header files for Zstd library
> ============================================================= Summary Matched: zstd ==============================================================
> fedora-repo-zdicts.noarch : Zstd dictionaries for Fedora repository metadata
> [root@quaco ~]#
> [root@quaco ~]# dnf install libzstd-devel
> <SNIP>
> libzstd-devel x86_64 1.3.8-1.fc29 updates 39 k
> <SNIP>
>
> So, do I need some specific 1.3.7 version? Is that why you used that
> LIBZSTD_DIR in your example?
v1.3.7 is just an example. v1.3.8 can be also provided.
>
> I'm trying these steps, but please consider following these suggestions
> in future posts,
>
> And thanks for working on this!
Thanks for meaningful comments. Will do my best to follow it.
Thanks,
Alexey
>
> - Arnaldo
>
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH v2 0/4] perf: enable compression of record mode trace to save storage space
@ 2019-01-28 7:02 Alexey Budankov
2019-02-11 13:52 ` Jiri Olsa
0 siblings, 1 reply; 12+ messages in thread
From: Alexey Budankov @ 2019-01-28 7:02 UTC (permalink / raw)
To: Arnaldo Carvalho de Melo, Ingo Molnar, Peter Zijlstra
Cc: Jiri Olsa, Namhyung Kim, Alexander Shishkin, Andi Kleen, linux-kernel
The patch set implements runtime trace compression for record mode and
trace file decompression for report mode. Zstandard API [1] is used for
compression/decompression of data that come from perf_events kernel
data buffers.
Realized -z,--compression_level=n option provides ~3-5x avg. trace file
size reduction on variety of tested workloads what saves user storage
space on larger server systems where trace file size can easily reach
several tens or even hundreds of GiBs, especially when profiling with
stacks for later dwarf unwinding and context-switches tracing and etc.
$ tools/perf/perf record -z 1 -e cycles -- matrix.gcc
--mmap-flush option can be used to avoid compressing every single byte
of data and increase compression ratio at the same time lowering tool
runtime overhead.
The compression functionality can be disabled from the command line
using NO_LIBZSTD define and Zstandard sources can be overridden using
value of LIBZSTD_DIR define:
$ make -C tools/perf NO_LIBZSTD=1 clean all
$ make -C tools/perf LIBZSTD_DIR=/path/to/zstd-1.3.7 clean all
The patch set is for Arnaldo's perf/core repository.
---
Alexey Budankov (4):
feature: realize libzstd check, LIBZSTD_DIR and NO_LIBZSTD defines
perf record: implement -z=<level> and --mmap-flush=<thres> options
perf record: enable runtime trace compression
perf report: support record trace file decompression
tools/build/Makefile.feature | 6 +-
tools/build/feature/Makefile | 6 +-
tools/build/feature/test-all.c | 5 +
tools/build/feature/test-libzstd.c | 12 +
tools/perf/Documentation/perf-record.txt | 9 +
tools/perf/Makefile.config | 20 ++
tools/perf/Makefile.perf | 3 +
tools/perf/builtin-record.c | 167 +++++++++++---
tools/perf/builtin-report.c | 5 +-
tools/perf/perf.h | 2 +
tools/perf/util/env.h | 10 +
tools/perf/util/event.c | 1 +
tools/perf/util/event.h | 7 +
tools/perf/util/evlist.c | 6 +-
tools/perf/util/evlist.h | 2 +-
tools/perf/util/header.c | 45 +++-
tools/perf/util/header.h | 1 +
tools/perf/util/mmap.c | 173 ++++++++++-----
tools/perf/util/mmap.h | 31 ++-
tools/perf/util/session.c | 271 ++++++++++++++++++++++-
tools/perf/util/session.h | 26 +++
tools/perf/util/tool.h | 2 +
22 files changed, 695 insertions(+), 115 deletions(-)
create mode 100644 tools/build/feature/test-libzstd.c
---
Changes in v2:
- moved compression/decompression code to session layer
- enabled allocation aio data buffers for compression
- enabled trace compression for serial trace streaming
---
[1] https://github.com/facebook/zstd
---
Examples:
$ make -C tools/perf NO_LIBZSTD=1 clean all
$ make -C tools/perf LIBZSTD_DIR=/path/to/zstd-1.3.7 clean all
$ tools/perf/perf record -z 1 -e cycles -- matrix.gcc
Addr of buf1 = 0x7fc266d52010
Offs of buf1 = 0x7fc266d52180
Addr of buf2 = 0x7fc264d51010
Offs of buf2 = 0x7fc264d511c0
Addr of buf3 = 0x7fc262d50010
Offs of buf3 = 0x7fc262d50100
Addr of buf4 = 0x7fc260d4f010
Offs of buf4 = 0x7fc260d4f140
Threads #: 8 Pthreads
Matrix size: 2048
Using multiply kernel: multiply1
Execution time = 31.471 seconds
[ perf record: Woken up 120 times to write data ]
[ perf record: Compressed 38.118 MB to 7.084 MB, ratio is 5.381 ]
[ perf record: Captured and wrote 7.100 MB perf.data (999192 samples) ]
$ tools/perf/perf report -D --header
# ========
# captured on : Sat Jan 26 11:49:55 2019
# header version : 1
# data offset : 296
# data size : 7444119
# feat offset : 7444415
# hostname : nntvtune39
# os release : 4.19.15-300.fc29.x86_64
# perf version : 4.13.rc5.g3cfa299
# arch : x86_64
# nrcpus online : 8
# nrcpus avail : 8
# cpudesc : Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz
# cpuid : GenuineIntel,6,94,3
# total memory : 16153184 kB
# cmdline : /root/abudanko/kernel/acme/tools/perf/perf record -z 1 -e cycles -- ../../matrix/linux/matrix.gcc
# event : name = cycles, , id = { 2171, 2172, 2173, 2174, 2175, 2176, 2177, 2178 }, size = 112, { sample_period, sample_freq } = 4000, sample_type = IP|TID|TIME|PERIOD, read_format =>
# CPU_TOPOLOGY info available, use -I to display
# NUMA_TOPOLOGY info available, use -I to display
# pmu mappings: intel_pt = 8, software = 1, power = 11, uprobe = 7, uncore_imc = 12, cpu = 4, cstate_core = 18, uncore_cbox_2 = 15, breakpoint = 5, uncore_cbox_0 = 13, tracepoint = 2>
# CACHE info available, use -I to display
# time of first sample : 230574.239204
# time of last sample : 230605.735403
# sample duration : 31496.200 ms
# MEM_TOPOLOGY info available, use -I to display
# compressed : Zstd, level = 1, ratio = 5
# missing features: TRACING_DATA BRANCH_STACK GROUP_DESC AUXTRACE STAT CLOCKID
# ========
#
0x128 [0x20]: event: 79
.
. ... raw event: size 32 bytes
. 0000: 4f 00 00 00 00 00 20 00 1f 00 00 00 00 00 00 00 O..... .........
. 0010: 11 a6 ef 1f 00 00 00 00 e7 16 81 83 f5 ff ff ff ................
0 0x128 [0x20]: PERF_RECORD_TIME_CONV: unhandled!
0x148 [0x50]: event: 1
.
. ... raw event: size 80 bytes
. 0000: 01 00 00 00 01 00 50 00 ff ff ff ff 00 00 00 00 ......P.........
. 0010: 00 00 00 89 ff ff ff ff 00 10 31 37 00 00 00 00 ..........17....
. 0020: 00 00 00 89 ff ff ff ff 5b 6b 65 72 6e 65 6c 2e ........[kernel.
. 0030: 6b 61 6c 6c 73 79 6d 73 5d 5f 74 65 78 74 00 00 kallsyms]_text..
. 0040: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0 0x148 [0x50]: PERF_RECORD_MMAP -1/0: [0xffffffff89000000(0x37311000) @ 0xffffffff89000000]: x [kernel.kallsyms]_text
...
0x6375d [0x8]: event: 68
.
. ... raw event: size 8 bytes
. 0000: 44 00 00 00 00 00 08 00 D.......
0 0x6375d [0x8]: PERF_RECORD_FINISHED_ROUND
0 [0x28]: event: 9
.
. ... raw event: size 40 bytes
. 0000: 09 00 00 00 01 00 28 00 76 78 06 89 ff ff ff ff ......(.vx......
. 0010: d4 1d 00 00 d4 1d 00 00 26 43 9f bf b4 d1 00 00 ........&C......
. 0020: 01 00 00 00 00 00 00 00 ........
230574239204134 0 [0x28]: PERF_RECORD_SAMPLE(IP, 0x1): 7636/7636: 0xffffffff89067876 period: 1 addr: 0
... thread: perf:7636
...... dso: /proc/kcore
0 [0x30]: event: 3
.
. ... raw event: size 48 bytes
. 0000: 03 00 00 00 00 20 30 00 d4 1d 00 00 d4 1d 00 00 ..... 0.........
. 0010: 6d 61 74 72 69 78 2e 67 63 63 00 00 00 00 00 00 matrix.gcc......
. 0020: d4 1d 00 00 d4 1d 00 00 34 4a 9f bf b4 d1 00 00 ........4J......
230574239205940 0 [0x30]: PERF_RECORD_COMM exec: matrix.gcc:7636/7636
0 [0x28]: event: 9
.
. ... raw event: size 40 bytes
. 0000: 09 00 00 00 01 00 28 00 76 78 06 89 ff ff ff ff ......(.vx......
. 0010: d4 1d 00 00 d4 1d 00 00 1f af 9f bf b4 d1 00 00 ................
. 0020: 3f 0c 00 00 00 00 00 00 ?.......
230574239231775 0 [0x28]: PERF_RECORD_SAMPLE(IP, 0x1): 7636/7636: 0xffffffff89067876 period: 3135 addr: 0
... thread: matrix.gcc:7636
...... dso: /proc/kcore
Aggregated stats:
TOTAL events: 1001434
MMAP events: 100
LOST events: 0
COMM events: 2
EXIT events: 9
THROTTLE events: 0
UNTHROTTLE events: 0
FORK events: 8
READ events: 0
SAMPLE events: 999192
MMAP2 events: 7
AUX events: 0
ITRACE_START events: 0
LOST_SAMPLES events: 0
SWITCH events: 0
SWITCH_CPU_WIDE events: 0
NAMESPACES events: 0
KSYMBOL events: 0
BPF_EVENT events: 0
ATTR events: 0
EVENT_TYPE events: 0
TRACING_DATA events: 0
BUILD_ID events: 0
FINISHED_ROUND events: 319
ID_INDEX events: 0
AUXTRACE_INFO events: 0
AUXTRACE events: 0
AUXTRACE_ERROR events: 0
THREAD_MAP events: 1
CPU_MAP events: 1
STAT_CONFIG events: 0
STAT events: 0
STAT_ROUND events: 0
EVENT_UPDATE events: 0
TIME_CONV events: 1
FEATURE events: 0
COMPRESSED events: 1794
cycles stats:
TOTAL events: 999192
MMAP events: 0
LOST events: 0
COMM events: 0
EXIT events: 0
THROTTLE events: 0
UNTHROTTLE events: 0
FORK events: 0
READ events: 0
SAMPLE events: 999192
MMAP2 events: 0
AUX events: 0
ITRACE_START events: 0
LOST_SAMPLES events: 0
SWITCH events: 0
SWITCH_CPU_WIDE events: 0
NAMESPACES events: 0
KSYMBOL events: 0
BPF_EVENT events: 0
ATTR events: 0
EVENT_TYPE events: 0
TRACING_DATA events: 0
BUILD_ID events: 0
FINISHED_ROUND events: 0
ID_INDEX events: 0
AUXTRACE_INFO events: 0
AUXTRACE events: 0
AUXTRACE_ERROR events: 0
THREAD_MAP events: 0
CPU_MAP events: 0
STAT_CONFIG events: 0
STAT events: 0
STAT_ROUND events: 0
EVENT_UPDATE events: 0
TIME_CONV events: 0
FEATURE events: 0
COMPRESSED events: 0
---
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v2 0/4] perf: enable compression of record mode trace to save storage space
2019-01-28 7:02 Alexey Budankov
@ 2019-02-11 13:52 ` Jiri Olsa
2019-02-11 13:59 ` Alexey Budankov
0 siblings, 1 reply; 12+ messages in thread
From: Jiri Olsa @ 2019-02-11 13:52 UTC (permalink / raw)
To: Alexey Budankov
Cc: Arnaldo Carvalho de Melo, Ingo Molnar, Peter Zijlstra, Jiri Olsa,
Namhyung Kim, Alexander Shishkin, Andi Kleen, linux-kernel
On Mon, Jan 28, 2019 at 10:02:49AM +0300, Alexey Budankov wrote:
>
> The patch set implements runtime trace compression for record mode and
> trace file decompression for report mode. Zstandard API [1] is used for
> compression/decompression of data that come from perf_events kernel
> data buffers.
>
> Realized -z,--compression_level=n option provides ~3-5x avg. trace file
> size reduction on variety of tested workloads what saves user storage
> space on larger server systems where trace file size can easily reach
> several tens or even hundreds of GiBs, especially when profiling with
> stacks for later dwarf unwinding and context-switches tracing and etc.
>
> $ tools/perf/perf record -z 1 -e cycles -- matrix.gcc
>
> --mmap-flush option can be used to avoid compressing every single byte
> of data and increase compression ratio at the same time lowering tool
> runtime overhead.
>
> The compression functionality can be disabled from the command line
> using NO_LIBZSTD define and Zstandard sources can be overridden using
> value of LIBZSTD_DIR define:
>
> $ make -C tools/perf NO_LIBZSTD=1 clean all
> $ make -C tools/perf LIBZSTD_DIR=/path/to/zstd-1.3.7 clean all
>
> The patch set is for Arnaldo's perf/core repository.
hi,
got here late and can't apply this anymore.. which revision
(commit id) is this based on? perf/core is moving real fast
these days ;-)
based on errors below I suspect u need to rebase anyway :-\
sry for late response
jirka
>
> ---
> Alexey Budankov (4):
> feature: realize libzstd check, LIBZSTD_DIR and NO_LIBZSTD defines
> perf record: implement -z=<level> and --mmap-flush=<thres> options
got this one failing when applied:
patching file tools/perf/Documentation/perf-record.txt
Hunk #1 succeeded at 459 with fuzz 2 (offset 19 lines).
patching file tools/perf/builtin-record.c
Hunk #1 succeeded at 292 (offset 5 lines).
Hunk #2 succeeded at 336 (offset 5 lines).
Hunk #3 FAILED at 565.
Hunk #4 succeeded at 765 with fuzz 1 (offset 10 lines).
Hunk #5 FAILED at 778.
Hunk #6 succeeded at 806 (offset 11 lines).
Hunk #7 succeeded at 839 (offset 11 lines).
Hunk #8 succeeded at 873 (offset 11 lines).
Hunk #9 succeeded at 1185 (offset 11 lines).
Hunk #10 succeeded at 1218 (offset 11 lines).
Hunk #11 succeeded at 1354 (offset 11 lines).
Hunk #12 succeeded at 1455 (offset 11 lines).
Hunk #13 succeeded at 1866 (offset 26 lines).
Hunk #14 FAILED at 2006.
Hunk #15 succeeded at 2230 with fuzz 2 (offset 32 lines).
3 out of 15 hunks FAILED -- saving rejects to file tools/perf/builtin-record.c.rej
patching file tools/perf/perf.h
Hunk #1 FAILED at 84.
1 out of 1 hunk FAILED -- saving rejects to file tools/perf/perf.h.rej
patching file tools/perf/util/env.h
patching file tools/perf/util/event.c
Hunk #1 succeeded at 68 (offset 2 lines).
patching file tools/perf/util/event.h
Hunk #2 succeeded at 627 (offset 1 line).
Hunk #3 succeeded at 665 (offset 1 line).
patching file tools/perf/util/evlist.c
Hunk #1 FAILED at 1022.
Hunk #2 FAILED at 1032.
Hunk #3 FAILED at 1064.
3 out of 3 hunks FAILED -- saving rejects to file tools/perf/util/evlist.c.rej
patching file tools/perf/util/evlist.h
Hunk #1 FAILED at 165.
1 out of 1 hunk FAILED -- saving rejects to file tools/perf/util/evlist.h.rej
patching file tools/perf/util/header.c
patching file tools/perf/util/header.h
patching file tools/perf/util/mmap.c
Hunk #1 FAILED at 154.
Hunk #2 succeeded at 282 with fuzz 1 (offset 69 lines).
Hunk #3 succeeded at 359 (offset 69 lines).
Hunk #4 succeeded at 434 (offset 97 lines).
Hunk #5 succeeded at 484 (offset 97 lines).
1 out of 5 hunks FAILED -- saving rejects to file tools/perf/util/mmap.c.rej
patching file tools/perf/util/mmap.h
Hunk #1 FAILED at 30.
Hunk #2 FAILED at 69.
2 out of 2 hunks FAILED -- saving rejects to file tools/perf/util/mmap.h.rej
patching file tools/perf/util/session.h
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v2 0/4] perf: enable compression of record mode trace to save storage space
2019-02-11 13:52 ` Jiri Olsa
@ 2019-02-11 13:59 ` Alexey Budankov
0 siblings, 0 replies; 12+ messages in thread
From: Alexey Budankov @ 2019-02-11 13:59 UTC (permalink / raw)
To: Jiri Olsa
Cc: Arnaldo Carvalho de Melo, Ingo Molnar, Peter Zijlstra, Jiri Olsa,
Namhyung Kim, Alexander Shishkin, Andi Kleen, linux-kernel
On 11.02.2019 16:52, Jiri Olsa wrote:
> On Mon, Jan 28, 2019 at 10:02:49AM +0300, Alexey Budankov wrote:
>>
<SNIP>
>> The patch set is for Arnaldo's perf/core repository.
>
> hi,
> got here late and can't apply this anymore.. which revision
> (commit id) is this based on? perf/core is moving real fast
> these days ;-)
>
> based on errors below I suspect u need to rebase anyway :-\
Yes, I will resend shortly.
Thanks,
Alexey
>
> sry for late response
>
> jirka
>
^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2019-02-12 14:06 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <b834df0e-26b5-3f8c-7a43-18f675fb7434@linux.intel.com>
2019-01-29 10:45 ` [PATCH v2 0/4] perf: enable compression of record mode trace to save storage space Arnaldo Carvalho de Melo
2019-01-29 10:53 ` Arnaldo Carvalho de Melo
2019-01-29 11:39 ` Alexey Budankov
2019-01-29 12:13 ` Arnaldo Carvalho de Melo
2019-01-29 16:39 ` Alexey Budankov
2019-01-29 17:25 ` Andi Kleen
2019-02-11 20:17 Alexey Budankov
2019-02-12 12:27 ` Arnaldo Carvalho de Melo
2019-02-12 14:06 ` Alexey Budankov
-- strict thread matches above, loose matches on Subject: below --
2019-01-28 7:02 Alexey Budankov
2019-02-11 13:52 ` Jiri Olsa
2019-02-11 13:59 ` Alexey Budankov
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).