From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0FC07C74A5B for ; Fri, 17 Mar 2023 13:11:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230029AbjCQNLm (ORCPT ); Fri, 17 Mar 2023 09:11:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55784 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229590AbjCQNLl (ORCPT ); Fri, 17 Mar 2023 09:11:41 -0400 Received: from frasgout.his.huawei.com (frasgout.his.huawei.com [185.176.79.56]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 577CEE4DAF; Fri, 17 Mar 2023 06:11:36 -0700 (PDT) Received: from lhrpeml500005.china.huawei.com (unknown [172.18.147.200]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4PdPds124yz6J7tB; Fri, 17 Mar 2023 21:11:13 +0800 (CST) Received: from localhost (10.48.148.142) by lhrpeml500005.china.huawei.com (7.191.163.240) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.21; Fri, 17 Mar 2023 13:11:32 +0000 Date: Fri, 17 Mar 2023 13:11:31 +0000 From: Jonathan Cameron To: Jie Zhan CC: , , , , , , , , , , , , , , , , , , , , , , , , Rob Herring Subject: Re: [RFC PATCH v1 0/4] HiSilicon Performance Monitor Control Unit Message-ID: <20230317131131.00005817@Huawei.com> In-Reply-To: <75a1b477-5325-bec9-5750-b09bf0e5dfb0@hisilicon.com> References: <20230206065146.645505-1-zhanjie9@hisilicon.com> <75a1b477-5325-bec9-5750-b09bf0e5dfb0@hisilicon.com> Organization: Huawei Technologies Research and Development (UK) Ltd. X-Mailer: Claws Mail 4.1.0 (GTK 3.24.33; x86_64-w64-mingw32) MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Originating-IP: [10.48.148.142] X-ClientProxiedBy: lhrpeml100001.china.huawei.com (7.191.160.183) To lhrpeml500005.china.huawei.com (7.191.163.240) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, 27 Feb 2023 16:49:46 +0800 Jie Zhan wrote: > Please can anyone have a look at this PMCU patchset and provide some > comments? > > It is much related to the ARM PMU. > > We are looking forward to the feedback. > > Any relevant comments/questions, with respect to software or hardware > design, use cases, coding, are welcome. > > Kind regards, > > Jie > > > On 06/02/2023 14:51, Jie Zhan wrote: > > HiSilicon Performance Monitor Control Unit (PMCU) is a device that offloads > > PMU accesses from CPUs, handling the configuration, event switching, and > > counter reading of core PMUs on Kunpeng SoC. It facilitates fine-grained > > and multi-PMU-event CPU profiling, in which scenario the current 'perf' > > scheme may lose events or drop sampling frequency. With PMCU, users can > > reliably obtain the data of up to 240 PMU events with the sample interval > > of events down to 1ms, while the software overhead of accessing PMUs, as > > well as its impact on target workloads, is reduced. > > > > This patchset contains the documentation, driver, and user perf tool > > support to enable using PMCU with the 'perf_event' framework. > > > > Here are two key questions requested for comments: > > > > - How do we make it compatible with arm_pmu drivers? > > > > Hardware-wise, PMCU uses the existing core PMUs, so PMUs can be accessed > > from CPU and PMCU simultaneously. The current hardware can't guarantee > > mutual exclusive accesses. Hence, scheduling arm_pmu and PMCU events at > > the same time may mess up the operation of PMUs, delivering incorrect > > data for both events, e.g. unexpected events or sample periods. > > Software-wise, we probably need to prevent the two types of events from > > running at the same time, but currently there isn't a clear solution. I've been thinking about this a bit and don't have a good answer yet. So some thoughts that might get some discussion going (some are here mostly to be shot down ;) 1. I suspect adding a hook into the specific pmu driver to reserve a counter is going to be controversial for this usecase. But maybe there is a more generic way... There are lock up detectors that use PMU counters and ensure the counters aren't also used for other purposes and that leads me to wonder if you can use https://elixir.bootlin.com/linux/latest/source/kernel/events/core.c#L12700 perf_event_create_kernel_counter() to do the same as opening a counter from userspace but then not use it. I have no idea if this will work though or if enabling the event would be necessary to prevent it being used elsewhere. 2. It might be possible to reuse any of the infrastructure that exists for userspace PMU counter access or maybe Rob Herring (+CC) has a suggestion based on his work on that feature. 3. It's not nice, but maybe could enforce this constraint just in userspace? We'd have to make sure that both drivers didn't do anything beyond not working correctly if the other driver is messing with the hardware. 4. We can't do the nasty trick of providing a second driver that binds to the PMU hardware to prevent it being used because I think the main arm PMU driver has suppress_bind_attrs = true. Maybe we can make remove work? (original patch for this in 2018 added that line because of a crash on remove - not sure anyone looked at fixing the crash). > > > > - Currently we reply on a sysfs file for users to input event numbers. Is > > there a better way to pass many events? > > > > The perf framework only allows three 64-bit config fields for custom PMU > > configs. Obviously, this can't satisfy our need for passing many events > > at a time. As an event number is 16-bit wide, the config fields can only > > take up to 12 events at a time, or up to 192 events even if we do a > > bitmap of events (and there are more than 192 available event numbers). > > Hence, the current design takes an array of event numbers from a sysfs > > file before starting profiling. However, this may go against the common > > way to schedule perf events through perf commands. > > > > Jie Zhan (4): > > docs: perf: Add documentation for HiSilicon PMCU > > drivers/perf: hisi: Add driver support for HiSilicon PMCU > > perf tool: Add HiSilicon PMCU data recording support > > perf tool: Add HiSilicon PMCU data decoding support > > > > Documentation/admin-guide/perf/hisi-pmcu.rst | 183 +++ > > Documentation/admin-guide/perf/index.rst | 1 + > > drivers/perf/hisilicon/Kconfig | 15 + > > drivers/perf/hisilicon/Makefile | 1 + > > drivers/perf/hisilicon/hisi_pmcu.c | 1096 ++++++++++++++++++ > > tools/perf/arch/arm/util/auxtrace.c | 61 + > > tools/perf/arch/arm64/util/Build | 2 +- > > tools/perf/arch/arm64/util/hisi-pmcu.c | 145 +++ > > tools/perf/util/Build | 1 + > > tools/perf/util/auxtrace.c | 4 + > > tools/perf/util/auxtrace.h | 1 + > > tools/perf/util/hisi-pmcu.c | 305 +++++ > > tools/perf/util/hisi-pmcu.h | 19 + > > 13 files changed, 1833 insertions(+), 1 deletion(-) > > create mode 100644 Documentation/admin-guide/perf/hisi-pmcu.rst > > create mode 100644 drivers/perf/hisilicon/hisi_pmcu.c > > create mode 100644 tools/perf/arch/arm64/util/hisi-pmcu.c > > create mode 100644 tools/perf/util/hisi-pmcu.c > > create mode 100644 tools/perf/util/hisi-pmcu.h > > > > > > base-commit: 830b3c68c1fb1e9176028d02ef86f3cf76aa2476 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id ACD53C74A5B for ; Fri, 17 Mar 2023 13:12:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Subject:CC:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=JRvLz6gnB7PIXgPm/dOYpS50JzI7K70/Joa7eV9uOjQ=; b=abNT0viYPXckCL iaRUeTmLLQNARNYLVn3J/eZY/c+bpmc5joSUbwCA2GfyD4vh0LUDhgr2Ra0obuqPOGXBsZV16lZh2 3WZtMxmOONtBVNe0NTFEo9PwD4fxdFvAnfz1j/y/5Jd7mVGMSDZFKQ45koBvlzSBBTt9L4x3TTqMc CbpgUmW39AMX7itmXm+JDRw+AjpUZbusqQVXPKHy+IPNUU8geujQRVjSx2ByQODJD+5ks0WB73+1V 3OyMvXsNsV779RhqMpkCvst1MLJj50KdyFUxBXW+3qRY+WMK6TOCCtJtnC0bXs+nErd8MeAaipITh 4Ic6bEA5Ds8ZJeKDs3CQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1pd9s5-002JN7-24; Fri, 17 Mar 2023 13:11:49 +0000 Received: from frasgout.his.huawei.com ([185.176.79.56]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1pd9s0-002JLu-1Z for linux-arm-kernel@lists.infradead.org; Fri, 17 Mar 2023 13:11:47 +0000 Received: from lhrpeml500005.china.huawei.com (unknown [172.18.147.200]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4PdPds124yz6J7tB; Fri, 17 Mar 2023 21:11:13 +0800 (CST) Received: from localhost (10.48.148.142) by lhrpeml500005.china.huawei.com (7.191.163.240) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.21; Fri, 17 Mar 2023 13:11:32 +0000 Date: Fri, 17 Mar 2023 13:11:31 +0000 From: Jonathan Cameron To: Jie Zhan CC: , , , , , , , , , , , , , , , , , , , , , , , , Rob Herring Subject: Re: [RFC PATCH v1 0/4] HiSilicon Performance Monitor Control Unit Message-ID: <20230317131131.00005817@Huawei.com> In-Reply-To: <75a1b477-5325-bec9-5750-b09bf0e5dfb0@hisilicon.com> References: <20230206065146.645505-1-zhanjie9@hisilicon.com> <75a1b477-5325-bec9-5750-b09bf0e5dfb0@hisilicon.com> Organization: Huawei Technologies Research and Development (UK) Ltd. X-Mailer: Claws Mail 4.1.0 (GTK 3.24.33; x86_64-w64-mingw32) MIME-Version: 1.0 X-Originating-IP: [10.48.148.142] X-ClientProxiedBy: lhrpeml100001.china.huawei.com (7.191.160.183) To lhrpeml500005.china.huawei.com (7.191.163.240) X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230317_061144_831713_F0A1F2B2 X-CRM114-Status: GOOD ( 46.18 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Mon, 27 Feb 2023 16:49:46 +0800 Jie Zhan wrote: > Please can anyone have a look at this PMCU patchset and provide some > comments? > > It is much related to the ARM PMU. > > We are looking forward to the feedback. > > Any relevant comments/questions, with respect to software or hardware > design, use cases, coding, are welcome. > > Kind regards, > > Jie > > > On 06/02/2023 14:51, Jie Zhan wrote: > > HiSilicon Performance Monitor Control Unit (PMCU) is a device that offloads > > PMU accesses from CPUs, handling the configuration, event switching, and > > counter reading of core PMUs on Kunpeng SoC. It facilitates fine-grained > > and multi-PMU-event CPU profiling, in which scenario the current 'perf' > > scheme may lose events or drop sampling frequency. With PMCU, users can > > reliably obtain the data of up to 240 PMU events with the sample interval > > of events down to 1ms, while the software overhead of accessing PMUs, as > > well as its impact on target workloads, is reduced. > > > > This patchset contains the documentation, driver, and user perf tool > > support to enable using PMCU with the 'perf_event' framework. > > > > Here are two key questions requested for comments: > > > > - How do we make it compatible with arm_pmu drivers? > > > > Hardware-wise, PMCU uses the existing core PMUs, so PMUs can be accessed > > from CPU and PMCU simultaneously. The current hardware can't guarantee > > mutual exclusive accesses. Hence, scheduling arm_pmu and PMCU events at > > the same time may mess up the operation of PMUs, delivering incorrect > > data for both events, e.g. unexpected events or sample periods. > > Software-wise, we probably need to prevent the two types of events from > > running at the same time, but currently there isn't a clear solution. I've been thinking about this a bit and don't have a good answer yet. So some thoughts that might get some discussion going (some are here mostly to be shot down ;) 1. I suspect adding a hook into the specific pmu driver to reserve a counter is going to be controversial for this usecase. But maybe there is a more generic way... There are lock up detectors that use PMU counters and ensure the counters aren't also used for other purposes and that leads me to wonder if you can use https://elixir.bootlin.com/linux/latest/source/kernel/events/core.c#L12700 perf_event_create_kernel_counter() to do the same as opening a counter from userspace but then not use it. I have no idea if this will work though or if enabling the event would be necessary to prevent it being used elsewhere. 2. It might be possible to reuse any of the infrastructure that exists for userspace PMU counter access or maybe Rob Herring (+CC) has a suggestion based on his work on that feature. 3. It's not nice, but maybe could enforce this constraint just in userspace? We'd have to make sure that both drivers didn't do anything beyond not working correctly if the other driver is messing with the hardware. 4. We can't do the nasty trick of providing a second driver that binds to the PMU hardware to prevent it being used because I think the main arm PMU driver has suppress_bind_attrs = true. Maybe we can make remove work? (original patch for this in 2018 added that line because of a crash on remove - not sure anyone looked at fixing the crash). > > > > - Currently we reply on a sysfs file for users to input event numbers. Is > > there a better way to pass many events? > > > > The perf framework only allows three 64-bit config fields for custom PMU > > configs. Obviously, this can't satisfy our need for passing many events > > at a time. As an event number is 16-bit wide, the config fields can only > > take up to 12 events at a time, or up to 192 events even if we do a > > bitmap of events (and there are more than 192 available event numbers). > > Hence, the current design takes an array of event numbers from a sysfs > > file before starting profiling. However, this may go against the common > > way to schedule perf events through perf commands. > > > > Jie Zhan (4): > > docs: perf: Add documentation for HiSilicon PMCU > > drivers/perf: hisi: Add driver support for HiSilicon PMCU > > perf tool: Add HiSilicon PMCU data recording support > > perf tool: Add HiSilicon PMCU data decoding support > > > > Documentation/admin-guide/perf/hisi-pmcu.rst | 183 +++ > > Documentation/admin-guide/perf/index.rst | 1 + > > drivers/perf/hisilicon/Kconfig | 15 + > > drivers/perf/hisilicon/Makefile | 1 + > > drivers/perf/hisilicon/hisi_pmcu.c | 1096 ++++++++++++++++++ > > tools/perf/arch/arm/util/auxtrace.c | 61 + > > tools/perf/arch/arm64/util/Build | 2 +- > > tools/perf/arch/arm64/util/hisi-pmcu.c | 145 +++ > > tools/perf/util/Build | 1 + > > tools/perf/util/auxtrace.c | 4 + > > tools/perf/util/auxtrace.h | 1 + > > tools/perf/util/hisi-pmcu.c | 305 +++++ > > tools/perf/util/hisi-pmcu.h | 19 + > > 13 files changed, 1833 insertions(+), 1 deletion(-) > > create mode 100644 Documentation/admin-guide/perf/hisi-pmcu.rst > > create mode 100644 drivers/perf/hisilicon/hisi_pmcu.c > > create mode 100644 tools/perf/arch/arm64/util/hisi-pmcu.c > > create mode 100644 tools/perf/util/hisi-pmcu.c > > create mode 100644 tools/perf/util/hisi-pmcu.h > > > > > > base-commit: 830b3c68c1fb1e9176028d02ef86f3cf76aa2476 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel