From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4D39EC433F5 for ; Tue, 22 Mar 2022 10:43:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233319AbiCVKpP (ORCPT ); Tue, 22 Mar 2022 06:45:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43106 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233364AbiCVKpK (ORCPT ); Tue, 22 Mar 2022 06:45:10 -0400 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 77F668166C; Tue, 22 Mar 2022 03:43:30 -0700 (PDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D95A1106F; Tue, 22 Mar 2022 03:43:29 -0700 (PDT) Received: from [10.1.197.1] (ewhatever.cambridge.arm.com [10.1.197.1]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 3A4C63F66F; Tue, 22 Mar 2022 03:43:28 -0700 (PDT) Message-ID: Date: Tue, 22 Mar 2022 10:43:26 +0000 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.7.0 Subject: Re: [PATCH 00/10] coresight: Add new API to allocate trace source ID values Content-Language: en-US To: Mike Leach , coresight@lists.linaro.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: mathieu.poirier@linaro.org, peterz@infradead.org, mingo@redhat.com, acme@kernel.org, linux-perf-users@vger.kernel.org, leo.yan@linaro.org, James Clark References: <20220308205000.27646-1-mike.leach@linaro.org> From: Suzuki Kuruppassery Poulose In-Reply-To: <20220308205000.27646-1-mike.leach@linaro.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org + Cc: James Clark Hi Mike, On 08/03/2022 20:49, Mike Leach wrote: > The current method for allocating trace source ID values to sources is > to use a fixed algorithm for CPU based sources of (cpu_num * 2 + 0x10). > The STM is allocated ID 0x1. > > This fixed algorithm is used in both the CoreSight driver code, and by > perf when writing the trace metadata in the AUXTRACE_INFO record. > > The method needs replacing as currently:- > 1. It is inefficient in using available IDs. > 2. Does not scale to larger systems with many cores and the algorithm > has no limits so will generate invalid trace IDs for cpu number > 44. Thanks for addressing this issue. > > Additionally requirements to allocate additional system IDs on some > systems have been seen. > > This patch set introduces an API that allows the allocation of trace IDs > in a dynamic manner. > > Architecturally reserved IDs are never allocated, and the system is > limited to allocating only valid IDs. > > Each of the current trace sources ETM3.x, ETM4.x and STM is updated to use > the new API. > > perf handling is changed so that the ID associated with the CPU is read > from sysfs. The ID allocator is notified when perf events start and stop > so CPU based IDs are kept constant throughout any perf session. > > For the ETMx.x devices IDs are allocated on certain events > a) When using sysfs, an ID will be allocated on hardware enable, and freed > when the sysfs reset is written. > b) When using perf, ID is allocated on hardware enable, and freed on > hardware disable. > > For both cases the ID is allocated when sysfs is read to get the current > trace ID. This ensures that consistent decode metadata can be extracted > from the system where this read occurs before device enable. > > Note: This patchset breaks backward compatibility for perf record. > Because the method for generating the AUXTRACE_INFO meta data has > changed, using an older perf record will result in metadata that > does not match the trace IDs used in the recorded trace data. > This mismatch will cause subsequent decode to fail. Older versions of > perf will still be able to decode data generated by the updated system. I have some concerns over this and the future plans for the dynamic allocation per sink. i.e., we are breaking/modifying the perf now to accommodate the dynamic nature of the trace id of a given CPU/ETM. The proposed approach of exposing this via sysfs may (am not sure if this would be the case) break for the trace-id per sink change, as a sink could assign different trace-id for a CPU depending. So, instead if we make the trace-id available in the perf (say, an new record format, PERF_RECORD_CS_ETM_TRACEID ?) record, we can rely on the new packet for the trace-id, irrespective of how that is allocated and remove the locking/linking of the trace-id with that of the sysfs. This is not something that exists today. (Ideally it would have been nice to have some additional fields in RECORD_AUXINFO, but we don't. Instead of breaking/extending that, we could add a new RECORD). I believe the packet may need to be generated only once for a session and that will also allow the flexibility of moving trace-id allocation around (to a sink in the future). Thoughts ? Kind regards Suzuki > > > Applies to coresight/next [b54f53bc11a5] > Tested on DB410c > > Mike Leach (10): > coresight: trace-id: Add API to dynamically assign trace ID values > coresight: trace-id: Set up source trace ID map for system > coresight: stm: Update STM driver to use Trace ID api > coresight: etm4x: Use trace ID API to dynamically allocate trace ID > coresight: etm3x: Use trace ID API to allocate IDs > coresight: perf: traceid: Add perf notifiers for trace ID > perf: cs-etm: Update event to read trace ID from sysfs > coresight: Remove legacy Trace ID allocation mechanism > coresight: etmX.X: stm: Remove unused legacy source trace ID ops > coresight: trace-id: Add debug & test macros to trace id allocation > > drivers/hwtracing/coresight/Makefile | 2 +- > drivers/hwtracing/coresight/coresight-core.c | 64 ++--- > .../hwtracing/coresight/coresight-etm-perf.c | 16 +- > drivers/hwtracing/coresight/coresight-etm.h | 3 +- > .../coresight/coresight-etm3x-core.c | 93 ++++--- > .../coresight/coresight-etm3x-sysfs.c | 28 +- > .../coresight/coresight-etm4x-core.c | 63 ++++- > .../coresight/coresight-etm4x-sysfs.c | 32 ++- > drivers/hwtracing/coresight/coresight-etm4x.h | 3 + > drivers/hwtracing/coresight/coresight-priv.h | 1 + > drivers/hwtracing/coresight/coresight-stm.c | 49 +--- > .../hwtracing/coresight/coresight-trace-id.c | 255 ++++++++++++++++++ > .../hwtracing/coresight/coresight-trace-id.h | 69 +++++ > include/linux/coresight-pmu.h | 12 - > include/linux/coresight.h | 3 - > tools/perf/arch/arm/util/cs-etm.c | 12 +- > 16 files changed, 530 insertions(+), 175 deletions(-) > create mode 100644 drivers/hwtracing/coresight/coresight-trace-id.c > create mode 100644 drivers/hwtracing/coresight/coresight-trace-id.h > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BC4FFC433F5 for ; Tue, 22 Mar 2022 10:45:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Content-Type: Content-Transfer-Encoding:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:From:References:Cc:To:Subject: MIME-Version:Date:Message-ID:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=JLc8PQsAuW7c3aunylukwSedg/+FwfuMPDnVpDW9nX4=; b=rrNRc4L1WQflp7 rz0GYHLpWFkHbOmwvl98SaqRNcFpc8sfmEpSgXnqVKK1aG4Nm1+jRa2hXQ6Q0CAlqx5/uzYimsq6p sSIiDS9gzJs0Fo7Z17GCskOUKnaDq0f2PnJuswtoOmRuF2zVbQKywgL/oDDbf0rwefFz3woE2jSUE H0u/bbLy9TC7kAd2IBIZbRlOBs8/g4XL8Gk/ficDYK3t9m//TXV/NW+sdOpf228JYWGxSyQKDkYwh akg08TcBwYOZk2myzf/vbjfpP/UBA4z2UcnQNpNL86OFVD4fVdjuqDOZIaWvkuo55VYMLOKpGS6sH Tza8I7wmzTv0qhlAXBbw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nWbzh-00AjdG-D0; Tue, 22 Mar 2022 10:44:05 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nWbzd-00Ajb6-Fa for linux-arm-kernel@lists.infradead.org; Tue, 22 Mar 2022 10:44:03 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D95A1106F; Tue, 22 Mar 2022 03:43:29 -0700 (PDT) Received: from [10.1.197.1] (ewhatever.cambridge.arm.com [10.1.197.1]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 3A4C63F66F; Tue, 22 Mar 2022 03:43:28 -0700 (PDT) Message-ID: Date: Tue, 22 Mar 2022 10:43:26 +0000 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.7.0 Subject: Re: [PATCH 00/10] coresight: Add new API to allocate trace source ID values Content-Language: en-US To: Mike Leach , coresight@lists.linaro.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: mathieu.poirier@linaro.org, peterz@infradead.org, mingo@redhat.com, acme@kernel.org, linux-perf-users@vger.kernel.org, leo.yan@linaro.org, James Clark References: <20220308205000.27646-1-mike.leach@linaro.org> From: Suzuki Kuruppassery Poulose In-Reply-To: <20220308205000.27646-1-mike.leach@linaro.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220322_034401_648660_BA88B0A2 X-CRM114-Status: GOOD ( 33.60 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org + Cc: James Clark Hi Mike, On 08/03/2022 20:49, Mike Leach wrote: > The current method for allocating trace source ID values to sources is > to use a fixed algorithm for CPU based sources of (cpu_num * 2 + 0x10). > The STM is allocated ID 0x1. > > This fixed algorithm is used in both the CoreSight driver code, and by > perf when writing the trace metadata in the AUXTRACE_INFO record. > > The method needs replacing as currently:- > 1. It is inefficient in using available IDs. > 2. Does not scale to larger systems with many cores and the algorithm > has no limits so will generate invalid trace IDs for cpu number > 44. Thanks for addressing this issue. > > Additionally requirements to allocate additional system IDs on some > systems have been seen. > > This patch set introduces an API that allows the allocation of trace IDs > in a dynamic manner. > > Architecturally reserved IDs are never allocated, and the system is > limited to allocating only valid IDs. > > Each of the current trace sources ETM3.x, ETM4.x and STM is updated to use > the new API. > > perf handling is changed so that the ID associated with the CPU is read > from sysfs. The ID allocator is notified when perf events start and stop > so CPU based IDs are kept constant throughout any perf session. > > For the ETMx.x devices IDs are allocated on certain events > a) When using sysfs, an ID will be allocated on hardware enable, and freed > when the sysfs reset is written. > b) When using perf, ID is allocated on hardware enable, and freed on > hardware disable. > > For both cases the ID is allocated when sysfs is read to get the current > trace ID. This ensures that consistent decode metadata can be extracted > from the system where this read occurs before device enable. > > Note: This patchset breaks backward compatibility for perf record. > Because the method for generating the AUXTRACE_INFO meta data has > changed, using an older perf record will result in metadata that > does not match the trace IDs used in the recorded trace data. > This mismatch will cause subsequent decode to fail. Older versions of > perf will still be able to decode data generated by the updated system. I have some concerns over this and the future plans for the dynamic allocation per sink. i.e., we are breaking/modifying the perf now to accommodate the dynamic nature of the trace id of a given CPU/ETM. The proposed approach of exposing this via sysfs may (am not sure if this would be the case) break for the trace-id per sink change, as a sink could assign different trace-id for a CPU depending. So, instead if we make the trace-id available in the perf (say, an new record format, PERF_RECORD_CS_ETM_TRACEID ?) record, we can rely on the new packet for the trace-id, irrespective of how that is allocated and remove the locking/linking of the trace-id with that of the sysfs. This is not something that exists today. (Ideally it would have been nice to have some additional fields in RECORD_AUXINFO, but we don't. Instead of breaking/extending that, we could add a new RECORD). I believe the packet may need to be generated only once for a session and that will also allow the flexibility of moving trace-id allocation around (to a sink in the future). Thoughts ? Kind regards Suzuki > > > Applies to coresight/next [b54f53bc11a5] > Tested on DB410c > > Mike Leach (10): > coresight: trace-id: Add API to dynamically assign trace ID values > coresight: trace-id: Set up source trace ID map for system > coresight: stm: Update STM driver to use Trace ID api > coresight: etm4x: Use trace ID API to dynamically allocate trace ID > coresight: etm3x: Use trace ID API to allocate IDs > coresight: perf: traceid: Add perf notifiers for trace ID > perf: cs-etm: Update event to read trace ID from sysfs > coresight: Remove legacy Trace ID allocation mechanism > coresight: etmX.X: stm: Remove unused legacy source trace ID ops > coresight: trace-id: Add debug & test macros to trace id allocation > > drivers/hwtracing/coresight/Makefile | 2 +- > drivers/hwtracing/coresight/coresight-core.c | 64 ++--- > .../hwtracing/coresight/coresight-etm-perf.c | 16 +- > drivers/hwtracing/coresight/coresight-etm.h | 3 +- > .../coresight/coresight-etm3x-core.c | 93 ++++--- > .../coresight/coresight-etm3x-sysfs.c | 28 +- > .../coresight/coresight-etm4x-core.c | 63 ++++- > .../coresight/coresight-etm4x-sysfs.c | 32 ++- > drivers/hwtracing/coresight/coresight-etm4x.h | 3 + > drivers/hwtracing/coresight/coresight-priv.h | 1 + > drivers/hwtracing/coresight/coresight-stm.c | 49 +--- > .../hwtracing/coresight/coresight-trace-id.c | 255 ++++++++++++++++++ > .../hwtracing/coresight/coresight-trace-id.h | 69 +++++ > include/linux/coresight-pmu.h | 12 - > include/linux/coresight.h | 3 - > tools/perf/arch/arm/util/cs-etm.c | 12 +- > 16 files changed, 530 insertions(+), 175 deletions(-) > create mode 100644 drivers/hwtracing/coresight/coresight-trace-id.c > create mode 100644 drivers/hwtracing/coresight/coresight-trace-id.h > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel