From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_PASS, UNWANTED_LANGUAGE_BODY,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6F951C43381 for ; Sat, 16 Mar 2019 14:13:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 3E4F221900 for ; Sat, 16 Mar 2019 14:13:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727000AbfCPONz (ORCPT ); Sat, 16 Mar 2019 10:13:55 -0400 Received: from mga14.intel.com ([192.55.52.115]:31036 "EHLO mga14.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726064AbfCPONz (ORCPT ); Sat, 16 Mar 2019 10:13:55 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 16 Mar 2019 07:13:55 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.58,486,1544515200"; d="scan'208";a="214756265" Received: from linux.intel.com ([10.54.29.200]) by orsmga001.jf.intel.com with ESMTP; 16 Mar 2019 07:13:54 -0700 Received: from [10.252.10.233] (abudanko-mobl.ccr.corp.intel.com [10.252.10.233]) by linux.intel.com (Postfix) with ESMTP id 3D143580310; Sat, 16 Mar 2019 07:13:52 -0700 (PDT) Subject: [PATCH v9 06/12] perf util: introduce Zstd streaming based compression API From: Alexey Budankov To: Arnaldo Carvalho de Melo Cc: Jiri Olsa , Namhyung Kim , Alexander Shishkin , Ingo Molnar , Peter Zijlstra , Andi Kleen , linux-kernel References: Organization: Intel Corp. Message-ID: <4e4433a6-250e-651a-bb1c-243e3cbbf4af@linux.intel.com> Date: Sat, 16 Mar 2019 17:13:50 +0300 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.5.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Implemented functions are based on Zstd streaming compression API. The functions are used in runtime to compress data that come from mmaped kernel buffer. zstd_init(), zstd_fini() are used for initialization and finalization to allocate and deallocate internal zstd objects. zstd_compress_stream_to_records() is used to convert parts of mmaped kernel buffer into an array of PERF_RECORD_COMPRESSED records. Signed-off-by: Alexey Budankov --- tools/perf/util/Build | 2 ++ tools/perf/util/compress.h | 43 +++++++++++++++++++++++ tools/perf/util/zstd.c | 71 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 116 insertions(+) create mode 100644 tools/perf/util/zstd.c diff --git a/tools/perf/util/Build b/tools/perf/util/Build index 8dd3102301ea..6d5bbc8b589b 100644 --- a/tools/perf/util/Build +++ b/tools/perf/util/Build @@ -145,6 +145,8 @@ perf-y += scripting-engines/ perf-$(CONFIG_ZLIB) += zlib.o perf-$(CONFIG_LZMA) += lzma.o +perf-$(CONFIG_ZSTD) += zstd.o + perf-y += demangle-java.o perf-y += demangle-rust.o diff --git a/tools/perf/util/compress.h b/tools/perf/util/compress.h index 892e92e7e7fc..d00d7cb095aa 100644 --- a/tools/perf/util/compress.h +++ b/tools/perf/util/compress.h @@ -2,6 +2,11 @@ #ifndef PERF_COMPRESS_H #define PERF_COMPRESS_H +#include +#ifdef HAVE_ZSTD_SUPPORT +#include +#endif + #ifdef HAVE_ZLIB_SUPPORT int gzip_decompress_to_file(const char *input, int output_fd); bool gzip_is_compressed(const char *input); @@ -12,4 +17,42 @@ int lzma_decompress_to_file(const char *input, int output_fd); bool lzma_is_compressed(const char *input); #endif +struct zstd_data { +#ifdef HAVE_ZSTD_SUPPORT + ZSTD_CStream *cstream; +#endif +}; + +#ifdef HAVE_ZSTD_SUPPORT + +int zstd_init(struct zstd_data *data, int level); +int zstd_fini(struct zstd_data *data); + +size_t zstd_compress_stream_to_records(struct zstd_data *data, + void *dst, size_t dst_size, void *src, size_t src_size, size_t max_record_size, + size_t process_header(void *record, size_t increment)); + +#else /* !HAVE_ZSTD_SUPPORT */ + +static inline int zstd_init(struct zstd_data *data __maybe_unused, int level __maybe_unused) +{ + return 0; +} + +static inline int zstd_fini(struct zstd_data *data __maybe_unused) +{ + return 0; +} + +static inline size_t zstd_compress_stream_to_records(struct zstd_data *data __maybe_unused, + void *dst __maybe_unused, size_t dst_size __maybe_unused, + void *src __maybe_unused, size_t src_size __maybe_unused, + size_t max_record_size __maybe_unused, + size_t process_header(void *record, size_t increment) __maybe_unused) +{ + return 0; +} + +#endif + #endif /* PERF_COMPRESS_H */ diff --git a/tools/perf/util/zstd.c b/tools/perf/util/zstd.c new file mode 100644 index 000000000000..6d4f69d57567 --- /dev/null +++ b/tools/perf/util/zstd.c @@ -0,0 +1,71 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include + +#include "util/compress.h" +#include "util/debug.h" + +int zstd_init(struct zstd_data *data, int level) +{ + size_t ret; + + data->cstream = ZSTD_createCStream(); + if (data->cstream == NULL) { + pr_err("Couldn't create compression stream.\n"); + return -1; + } + + ret = ZSTD_initCStream(data->cstream, level); + if (ZSTD_isError(ret)) { + pr_err("Failed to initialize compression stream: %s\n", ZSTD_getErrorName(ret)); + return -1; + } + + return 0; +} + +int zstd_fini(struct zstd_data *data) +{ + if (data->cstream) { + ZSTD_freeCStream(data->cstream); + data->cstream = NULL; + } + + return 0; +} + +size_t zstd_compress_stream_to_records(struct zstd_data *data, + void *dst, size_t dst_size, void *src, size_t src_size, size_t max_record_size, + size_t process_header(void *record, size_t increment)) +{ + size_t ret, size, compressed = 0; + ZSTD_inBuffer input = { src, src_size, 0 }; + ZSTD_outBuffer output; + void *record; + + while (input.pos < input.size) { + record = dst; + size = process_header(record, 0); + compressed += size; + dst += size; + dst_size -= size; + output = (ZSTD_outBuffer){ dst, (dst_size > max_record_size) ? + max_record_size : dst_size, 0 }; + ret = ZSTD_compressStream(data->cstream, &output, &input); + ZSTD_flushStream(data->cstream, &output); + if (ZSTD_isError(ret)) { + pr_err("failed to compress %ld bytes: %s\n", + (long)src_size, ZSTD_getErrorName(ret)); + memcpy(dst, src, src_size); + return src_size; + } + size = output.pos; + size = process_header(record, size); + compressed += size; + dst += size; + dst_size -= size; + } + + return compressed; +} + -- 2.20.1