From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_PASS, UNWANTED_LANGUAGE_BODY,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E3CC6C282C3 for ; Tue, 22 Jan 2019 17:49:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id BD1EE20879 for ; Tue, 22 Jan 2019 17:49:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726307AbfAVRs7 (ORCPT ); Tue, 22 Jan 2019 12:48:59 -0500 Received: from mga04.intel.com ([192.55.52.120]:12957 "EHLO mga04.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725950AbfAVRs7 (ORCPT ); Tue, 22 Jan 2019 12:48:59 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 22 Jan 2019 09:48:58 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,507,1539673200"; d="scan'208";a="111792579" Received: from linux.intel.com ([10.54.29.200]) by orsmga008.jf.intel.com with ESMTP; 22 Jan 2019 09:48:58 -0800 Received: from [10.252.24.137] (abudanko-mobl.ccr.corp.intel.com [10.252.24.137]) by linux.intel.com (Postfix) with ESMTP id 2837A580434; Tue, 22 Jan 2019 09:48:55 -0800 (PST) Subject: [PATCH v5 2/4] perf record: bind the AIO user space buffers to nodes From: Alexey Budankov To: Arnaldo Carvalho de Melo , Ingo Molnar , Peter Zijlstra Cc: Jiri Olsa , Namhyung Kim , Alexander Shishkin , Andi Kleen , linux-kernel References: Organization: Intel Corp. Message-ID: <5a5adebc-afe0-4806-81cd-180d49ec043f@linux.intel.com> Date: Tue, 22 Jan 2019 20:48:54 +0300 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Allocate and bind AIO user space buffers to the memory nodes that mmap kernel buffers are bound to. Signed-off-by: Alexey Budankov --- Changes in v4: - fixed compilation issue converting pr_warn() to pr_warning() - implemented stop if mbind() fails Changes in v3: - corrected code style issues - adjusted __aio_alloc,__aio_bind,__aio_free() implementation Changes in v2: - implemented perf_mmap__aio_alloc, perf_mmap__aio_free, perf_mmap__aio_bind and put HAVE_LIBNUMA_SUPPORT #ifdefs in there --- tools/perf/util/mmap.c | 77 +++++++++++++++++++++++++++++++++++++++--- 1 file changed, 73 insertions(+), 4 deletions(-) diff --git a/tools/perf/util/mmap.c b/tools/perf/util/mmap.c index e68ba754a8e2..34be9f900575 100644 --- a/tools/perf/util/mmap.c +++ b/tools/perf/util/mmap.c @@ -10,6 +10,9 @@ #include #include #include +#ifdef HAVE_LIBNUMA_SUPPORT +#include +#endif #include "debug.h" #include "event.h" #include "mmap.h" @@ -154,9 +157,72 @@ void __weak auxtrace_mmap_params__set_idx(struct auxtrace_mmap_params *mp __mayb } #ifdef HAVE_AIO_SUPPORT + +#ifdef HAVE_LIBNUMA_SUPPORT +static int perf_mmap__aio_alloc(struct perf_mmap *map, int index) +{ + map->aio.data[index] = mmap(NULL, perf_mmap__mmap_len(map), PROT_READ|PROT_WRITE, + MAP_PRIVATE|MAP_ANONYMOUS, 0, 0); + if (map->aio.data[index] == MAP_FAILED) { + map->aio.data[index] = NULL; + return -1; + } + + return 0; +} + +static void perf_mmap__aio_free(struct perf_mmap *map, int index) +{ + if (map->aio.data[index]) { + munmap(map->aio.data[index], perf_mmap__mmap_len(map)); + map->aio.data[index] = NULL; + } +} + +static int perf_mmap__aio_bind(struct perf_mmap *map, int index, int cpu, int affinity) +{ + void *data; + size_t mmap_len; + unsigned long node_mask; + + if (affinity != PERF_AFFINITY_SYS && cpu__max_node() > 1) { + data = map->aio.data[index]; + mmap_len = perf_mmap__mmap_len(map); + node_mask = 1UL << cpu__get_node(cpu); + if (mbind(data, mmap_len, MPOL_BIND, &node_mask, 1, 0)) { + pr_err("Failed to bind [%p-%p] AIO buffer to node %d: error %m\n", + data, data + mmap_len, cpu__get_node(cpu)); + return -1; + } + } + + return 0; +} +#else +static int perf_mmap__aio_alloc(struct perf_mmap *map, int index) +{ + map->aio.data[index] = malloc(perf_mmap__mmap_len(map)); + if (map->aio.data[index] == NULL) + return -1; + + return 0; +} + +static void perf_mmap__aio_free(struct perf_mmap *map, int index) +{ + zfree(&(map->aio.data[index])); +} + +static int perf_mmap__aio_bind(struct perf_mmap *map __maybe_unused, int index __maybe_unused, + int cpu __maybe_unused, int affinity __maybe_unused) +{ + return 0; +} +#endif + static int perf_mmap__aio_mmap(struct perf_mmap *map, struct mmap_params *mp) { - int delta_max, i, prio; + int delta_max, i, prio, ret; map->aio.nr_cblocks = mp->nr_cblocks; if (map->aio.nr_cblocks) { @@ -177,11 +243,14 @@ static int perf_mmap__aio_mmap(struct perf_mmap *map, struct mmap_params *mp) } delta_max = sysconf(_SC_AIO_PRIO_DELTA_MAX); for (i = 0; i < map->aio.nr_cblocks; ++i) { - map->aio.data[i] = malloc(perf_mmap__mmap_len(map)); - if (!map->aio.data[i]) { + ret = perf_mmap__aio_alloc(map, i); + if (ret == -1) { pr_debug2("failed to allocate data buffer area, error %m"); return -1; } + ret = perf_mmap__aio_bind(map, i, map->cpu, mp->affinity); + if (ret == -1) + return -1; /* * Use cblock.aio_fildes value different from -1 * to denote started aio write operation on the @@ -210,7 +279,7 @@ static void perf_mmap__aio_munmap(struct perf_mmap *map) int i; for (i = 0; i < map->aio.nr_cblocks; ++i) - zfree(&map->aio.data[i]); + perf_mmap__aio_free(map, i); if (map->aio.data) zfree(&map->aio.data); zfree(&map->aio.cblocks);