Re: [PATCH v8 00/12] perf: enable compression of record mode trace to save storage space

From: Alexey Budankov <alexey.budankov@linux.intel.com>
To: Jiri Olsa <jolsa@redhat.com>
Cc: Arnaldo Carvalho de Melo <acme@kernel.org>,
	Namhyung Kim <namhyung@kernel.org>,
	Alexander Shishkin <alexander.shishkin@linux.intel.com>,
	Ingo Molnar <mingo@redhat.com>,
	Peter Zijlstra <peterz@infradead.org>,
	Andi Kleen <ak@linux.intel.com>,
	linux-kernel <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v8 00/12] perf: enable compression of record mode trace to save storage space
Date: Fri, 15 Mar 2019 18:18:50 +0300	[thread overview]
Message-ID: <ca99904c-ab22-e7ad-741f-cdda0a8f2e77@linux.intel.com> (raw)
In-Reply-To: <7e54bef0-b438-cf5d-1023-70508cc1bc06@linux.intel.com>

On 15.03.2019 16:43, Alexey Budankov wrote:
> On 15.03.2019 15:28, Jiri Olsa wrote:
>> On Thu, Mar 14, 2019 at 02:26:23PM +0300, Alexey Budankov wrote:
>>>
> <SNIP>
>>> The patch set implements runtime trace compression (-z option) in 
>>> record mode and trace auto decompression in report and inject modes. 
>>> Streaming Zstd API [1] is used for compression and decompression of
>>> data that come from kernel mmaped data buffers.
>>>
> <SNIP>
>>>   $ tools/perf/perf record -z -e cycles -- matrix.gcc
>>>   $ tools/perf/perf record --aio -z -e cycles -- matrix.gcc
>>>   $ tools/perf/perf record -z --mmap-flush 1024 -e cycles -- matrix.gcc
>>>   $ tools/perf/perf record --aio -z --mmap-flush 1K -e cycles -- matrix.gcc
>>
>> hi,
>> I'm getting error with -z:
>>
>> [root@krava perf]# ./perf record -z ./perf bench sched messaging -l 10000
>> # Running 'sched/messaging' benchmark:
>> # 20 sender and receiver processes per group
>> # 10 groups == 400 processes run
>>
>>      Total time: 18.775 [sec]
>> [ perf record: Woken up 57 times to write data ]
>> 0x5228 [0]: failed to process type: 81
>> [ perf record: Captured and wrote 6.453 MB perf.data, compressed (original 21.486 MB, ratio is 3.340) ]
> 
> Reproduced locally. Investigating right now.
> 
> tools/perf/perf record -z tools/perf/perf bench sched messaging -l 10000
> # Running 'sched/messaging' benchmark:
> # 20 sender and receiver processes per group
> # 10 groups == 400 processes run
> 
>      Total time: 8.799 [sec]
> [ perf record: Woken up 35 times to write data ]
> 0x2e48 [0]: failed to process type: 81
> [ perf record: Captured and wrote 3.142 MB perf.data, compressed (original 10.241 MB, ratio is 3.272) ]

Here it is:

diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
index 4c1d4c4f7b33..e69e50db53b0 100644
--- a/tools/perf/builtin-record.c
+++ b/tools/perf/builtin-record.c
@@ -891,7 +891,7 @@ static size_t zstd_compress(struct perf_session *session, void *dst, size_t dst_
                            void *src, size_t src_size)
 {
        size_t compressed;
-       size_t max_record_size = PERF_SAMPLE_MAX_SIZE - sizeof(struct compressed_event);
+       size_t max_record_size = PERF_SAMPLE_MAX_SIZE - sizeof(struct compressed_event) - 1;
 
        compressed = zstd_compress_stream_to_records(&(session->zstd_data),
                                dst, dst_size, src, src_size, max_record_size,

~Alexey

> 
> However it is not observed in my tests on 8 cores Skylake.
> 
> tools/perf/perf record -z ../../matrix/linux/matrix.gcc
> Addr of buf1 = 0x7f2eca1ab010
> Offs of buf1 = 0x7f2eca1ab180
> Addr of buf2 = 0x7f2ec81aa010
> Offs of buf2 = 0x7f2ec81aa1c0
> Addr of buf3 = 0x7f2ec61a9010
> Offs of buf3 = 0x7f2ec61a9100
> Addr of buf4 = 0x7f2ec41a8010
> Offs of buf4 = 0x7f2ec41a8140
> Threads #: 8 Pthreads
> Matrix size: 2048
> Using multiply kernel: multiply1
> Execution time = 30.075 seconds
> [ perf record: Woken up 127 times to write data ]
> [ perf record: Captured and wrote 6.820 MB perf.data (953438 samples), compressed (original 36.372 MB, ratio is 5.344) ]
> 
> Thanks,
> Alexey
> 
>>
>>
>> jirka
>>
> 
