From: Mathieu Poirier <mathieu.poirier@linaro.org>
To: Anshuman Khandual <anshuman.khandual@arm.com>
Cc: linux-arm-kernel@lists.infradead.org, coresight@lists.linaro.org,
suzuki.poulose@arm.com, mike.leach@linaro.org,
lcherian@marvell.com, linux-kernel@vger.kernel.org
Subject: Re: [PATCH V3 11/14] coresight: sink: Add TRBE driver
Date: Fri, 12 Feb 2021 10:02:32 -0700 [thread overview]
Message-ID: <20210212170232.GB2692426@xps15> (raw)
In-Reply-To: <9787ef82-9bd9-b3ec-b899-8e682dfa3971@arm.com>
On Fri, Feb 12, 2021 at 11:13:01AM +0530, Anshuman Khandual wrote:
>
>
> On 2/11/21 12:30 AM, Mathieu Poirier wrote:
> > On Wed, Jan 27, 2021 at 02:25:35PM +0530, Anshuman Khandual wrote:
> >> Trace Buffer Extension (TRBE) implements a trace buffer per CPU which is
> >> accessible via the system registers. The TRBE supports different addressing
> >> modes including CPU virtual address and buffer modes including the circular
> >> buffer mode. The TRBE buffer is addressed by a base pointer (TRBBASER_EL1),
> >> an write pointer (TRBPTR_EL1) and a limit pointer (TRBLIMITR_EL1). But the
> >> access to the trace buffer could be prohibited by a higher exception level
> >> (EL3 or EL2), indicated by TRBIDR_EL1.P. The TRBE can also generate a CPU
> >> private interrupt (PPI) on address translation errors and when the buffer
> >> is full. Overall implementation here is inspired from the Arm SPE driver.
> >>
> >> Cc: Mathieu Poirier <mathieu.poirier@linaro.org>
> >> Cc: Mike Leach <mike.leach@linaro.org>
> >> Cc: Suzuki K Poulose <suzuki.poulose@arm.com>
> >> Signed-off-by: Anshuman Khandual <anshuman.khandual@arm.com>
> >> ---
> >> Changes in V3:
> >>
> >> - Added new DT bindings document TRBE.yaml
> >> - Changed TRBLIMITR_TRIG_MODE_SHIFT from 2 to 3
> >> - Dropped isb() from trbe_reset_local()
> >> - Dropped gap between (void *) and buf->trbe_base
> >> - Changed 'int' to 'unsigned int' in is_trbe_available()
> >> - Dropped unused function set_trbe_running(), set_trbe_virtual_mode(),
> >> set_trbe_enabled() and set_trbe_limit_pointer()
> >> - Changed get_trbe_flag_update(), is_trbe_programmable() and
> >> get_trbe_address_align() to accept TRBIDR value
> >> - Changed is_trbe_running(), is_trbe_abort(), is_trbe_wrap(), is_trbe_trg(),
> >> is_trbe_irq(), get_trbe_bsc() and get_trbe_ec() to accept TRBSR value
> >> - Dropped snapshot mode condition in arm_trbe_alloc_buffer()
> >> - Exit arm_trbe_init() when arm64_kernel_unmapped_at_el0() is enabled
> >> - Compute trbe_limit before trbe_write to get the updated handle
> >> - Added trbe_stop_and_truncate_event()
> >> - Dropped trbe_handle_fatal()
> >>
> >> Documentation/trace/coresight/coresight-trbe.rst | 39 +
> >> arch/arm64/include/asm/sysreg.h | 1 +
> >> drivers/hwtracing/coresight/Kconfig | 11 +
> >> drivers/hwtracing/coresight/Makefile | 1 +
> >> drivers/hwtracing/coresight/coresight-trbe.c | 1023 ++++++++++++++++++++++
> >> drivers/hwtracing/coresight/coresight-trbe.h | 160 ++++
> >> 6 files changed, 1235 insertions(+)
> >> create mode 100644 Documentation/trace/coresight/coresight-trbe.rst
> >> create mode 100644 drivers/hwtracing/coresight/coresight-trbe.c
> >> create mode 100644 drivers/hwtracing/coresight/coresight-trbe.h
> >> +
> >
> > [...]
> >
> >> +static void arm_trbe_probe_coresight_cpu(void *info)
> >> +{
> >> + struct trbe_drvdata *drvdata = info;
> >> + struct coresight_desc desc = { 0 };
> >> + int cpu = smp_processor_id();
> >> + struct trbe_cpudata *cpudata = per_cpu_ptr(drvdata->cpudata, cpu);
> >> + struct coresight_device *trbe_csdev = per_cpu(csdev_sink, cpu);
> >> + u64 trbidr = read_sysreg_s(SYS_TRBIDR_EL1);
> >> + struct device *dev;
> >> +
> >> + if (WARN_ON(!cpudata))
> >> + goto cpu_clear;
> >
> > There is already a check for this in arm_trbe_probe_coresight(), we couldn't be
> > here if there was a problem with the allocation.
>
> Right but just to be extra cautious. Do you really want this to be dropped ?
I don't think it is necessary but there is no harm in keeping it if you are keen
on it.
>
> >
> >> +
> >> + if (trbe_csdev)
> >> + return;
> >
> > Now that's a reason to have a WARN_ON(). If we are probing and a sink is
> > already present in this cpu's slot, something went seriously wrong and we should
> > be clear about it.
>
> Right, will add an WARN_ON().
>
> >
> >> +
> >> + cpudata->cpu = smp_processor_id();
> >> + cpudata->drvdata = drvdata;
> >> + dev = &cpudata->drvdata->pdev->dev;
> >> +
> >> + if (!is_trbe_available()) {
> >> + pr_err("TRBE is not implemented on cpu %d\n", cpudata->cpu);
> >> + goto cpu_clear;
> >> + }
> >> +
> >> + if (!is_trbe_programmable(trbidr)) {
> >> + pr_err("TRBE is owned in higher exception level on cpu %d\n", cpudata->cpu);
> >> + goto cpu_clear;
> >> + }
> >> + desc.name = devm_kasprintf(dev, GFP_KERNEL, "%s%d", DRVNAME, smp_processor_id());
> >
> > We will end up with "arm_trbe0", "arm_trbe1" and so on in sysfs... Is the
> > "arm_" part absolutely needed? I think this should be like what we do for etmv3
> > and etmv4 where only "etmX" shows up in sysfs.
>
> Okay, will drop arm_ here. IIRC this was originally trbeX where X is the cpu number
> but then ended up using DRVNAME as prefix.
>
> >
> >> + if (IS_ERR(desc.name))
> >> + goto cpu_clear;
> >> +
> >> + desc.type = CORESIGHT_DEV_TYPE_SINK;
> >> + desc.subtype.sink_subtype = CORESIGHT_DEV_SUBTYPE_SINK_PERCPU_SYSMEM;
> >> + desc.ops = &arm_trbe_cs_ops;
> >> + desc.pdata = dev_get_platdata(dev);
> >> + desc.groups = arm_trbe_groups;
> >> + desc.dev = dev;
> >> + trbe_csdev = coresight_register(&desc);
> >> + if (IS_ERR(trbe_csdev))
> >> + goto cpu_clear;
> >> +
> >> + dev_set_drvdata(&trbe_csdev->dev, cpudata);
> >> + cpudata->trbe_dbm = get_trbe_flag_update(trbidr);
> >> + cpudata->trbe_align = 1ULL << get_trbe_address_align(trbidr);
> >> + if (cpudata->trbe_align > SZ_2K) {
> >> + pr_err("Unsupported alignment on cpu %d\n", cpudata->cpu);
> >> + goto cpu_clear;
> >
> > Here coresight_unregister() should be called. The other option is to call
> > coresight_register() when everything else is known to be fine, which is the
> > favoured approach.
>
> Okay, will change accordingly.
>
> >
> >> + }
> >> + per_cpu(csdev_sink, cpu) = trbe_csdev;
> >> + trbe_reset_local();
> >> + enable_percpu_irq(drvdata->irq, IRQ_TYPE_NONE);
> >> + return;
> >> +cpu_clear:
> >> + cpumask_clear_cpu(cpudata->cpu, &cpudata->drvdata->supported_cpus);
> >> +}
> >> +
> >> +static void arm_trbe_remove_coresight_cpu(void *info)
> >> +{
> >> + int cpu = smp_processor_id();
> >> + struct trbe_drvdata *drvdata = info;
> >> + struct trbe_cpudata *cpudata = per_cpu_ptr(drvdata->cpudata, cpu);
> >> + struct coresight_device *trbe_csdev = per_cpu(csdev_sink, cpu);
> >> +
> >> + if (trbe_csdev) {
> >
> > In what scenario do you see not having a trbe_csdev and still needing to disable
> > IRQs for the HW? If there is a such a case then a few lines of comment is
> > needed.
> >
> >> + coresight_unregister(trbe_csdev);
> >> + cpudata->drvdata = NULL;
> >> + per_cpu(csdev_sink, cpu) = NULL;
> >> + }
> >> + disable_percpu_irq(drvdata->irq);
> >> + trbe_reset_local();
> >
> > Theoretically this code shouldn't run when the TRBE is enabled, because the CS
> > core will prevent that from happening. As sush disabling interrupts after
> > coresight_unregister() has been called and setting cpudata->drvdata to NULL
> > should be fine. But from an outsider's point of view it will look very bizarre.
> > Either write a comment to explain all that or call the above two before doing
> > the cleanup.
>
> Okay, will move them before the cleanup.
>
> >
> >> +}
> >> +
> >> +static int arm_trbe_probe_coresight(struct trbe_drvdata *drvdata)
> >> +{
> >> + drvdata->cpudata = alloc_percpu(typeof(*drvdata->cpudata));
> >> + if (IS_ERR(drvdata->cpudata))
> >> + return PTR_ERR(drvdata->cpudata);
> >
> > As far as I can tell alloc_percpu() returns NULL on failure and nothing else.
>
> Sure, will change the return code as -ENOMEM when alloc_percpu() returns NULL.
>
> >
> >> +
> >> + arm_trbe_probe_coresight_cpu(drvdata);
> >> + smp_call_function_many(&drvdata->supported_cpus, arm_trbe_probe_coresight_cpu, drvdata, 1);
> >
> > The above two calls look racy to me. The executing process could be moved to
> > another CPU between the call to arm_trbe_probe_coresight_cpu() and
> > smp_call_function_many(), which would prevent the initialisation of the TRBE on
> > the new CPU to be done. I suggest using a for_each_cpu() loop where
> > smp_call_function_single() would be used. That way we are guaranteed all the
> > TRBEs will be initialised.
>
> Okay, will change.
>
> >
> >> + return 0;
> >> +}
> >> +
> >> +static int arm_trbe_remove_coresight(struct trbe_drvdata *drvdata)
> >> +{
> >> + arm_trbe_remove_coresight_cpu(drvdata);
> >> + smp_call_function_many(&drvdata->supported_cpus, arm_trbe_remove_coresight_cpu, drvdata, 1);
> >
> > Same as above.
>
> Okay, will do.
>
> >
> > I'm out of time for today, more to come tomorrow.
>
> Okay.
>
> >
> > Mathieu
>
> - Anshuman
next prev parent reply other threads:[~2021-02-12 17:04 UTC|newest]
Thread overview: 90+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-01-27 8:55 [PATCH V3 00/14] arm64: coresight: Enable ETE and TRBE Anshuman Khandual
2021-01-27 8:55 ` [PATCH V3 01/14] coresight: etm-perf: Allow an event to use different sinks Anshuman Khandual
2021-02-01 23:17 ` Mathieu Poirier
2021-02-02 9:42 ` Suzuki K Poulose
2021-02-02 16:33 ` Mike Leach
2021-02-02 22:41 ` Suzuki K Poulose
2021-02-04 12:27 ` Mike Leach
2021-02-02 16:37 ` Mathieu Poirier
2021-01-27 8:55 ` [PATCH V3 02/14] coresight: Do not scan for graph if none is present Anshuman Khandual
2021-02-01 23:44 ` Mathieu Poirier
2021-02-02 11:10 ` Mike Leach
2021-02-02 14:36 ` Suzuki K Poulose
2021-01-27 8:55 ` [PATCH V3 03/14] coresight: etm4x: Add support for PE OS lock Anshuman Khandual
2021-02-02 17:40 ` Mathieu Poirier
2021-02-02 18:03 ` Mathieu Poirier
2021-02-15 14:08 ` Mike Leach
2021-01-27 8:55 ` [PATCH V3 04/14] coresight: ete: Add support for ETE sysreg access Anshuman Khandual
2021-02-02 17:52 ` Mathieu Poirier
2021-02-03 15:51 ` Suzuki K Poulose
2021-02-15 14:08 ` Mike Leach
2021-01-27 8:55 ` [PATCH V3 05/14] coresight: ete: Add support for ETE tracing Anshuman Khandual
2021-02-02 18:56 ` Mathieu Poirier
2021-02-02 22:50 ` Suzuki K Poulose
2021-02-15 13:21 ` Mike Leach
2021-02-15 14:08 ` Mike Leach
2021-01-27 8:55 ` [PATCH V3 06/14] dts: bindings: Document device tree bindings for ETE Anshuman Khandual
2021-02-09 19:00 ` Rob Herring
2021-02-10 12:33 ` Suzuki K Poulose
2021-02-18 18:33 ` Rob Herring
2021-02-18 22:51 ` Suzuki K Poulose
2021-01-27 8:55 ` [PATCH V3 07/14] coresight: etm-perf: Handle stale output handles Anshuman Khandual
2021-02-03 19:05 ` Mathieu Poirier
2021-02-03 23:36 ` Suzuki K Poulose
2021-02-15 16:27 ` Mike Leach
2021-01-27 8:55 ` [PATCH V3 08/14] coresight: core: Add support for dedicated percpu sinks Anshuman Khandual
2021-01-28 9:16 ` Suzuki K Poulose
2021-02-04 18:34 ` Mathieu Poirier
2021-02-16 10:40 ` Anshuman Khandual
2021-02-16 20:44 ` Mathieu Poirier
2021-02-16 10:21 ` Anshuman Khandual
2021-02-15 16:27 ` Mike Leach
2021-02-15 16:56 ` Mathieu Poirier
2021-02-15 17:58 ` Mike Leach
2021-02-16 20:30 ` Mathieu Poirier
2021-01-27 8:55 ` [PATCH V3 09/14] arm64: Add TRBE definitions Anshuman Khandual
2021-01-28 9:31 ` Suzuki K Poulose
2021-01-28 17:18 ` Catalin Marinas
2021-02-15 18:06 ` Mike Leach
2021-01-27 8:55 ` [PATCH V3 10/14] arm64: nvhe: Allow TRBE access at EL1 Anshuman Khandual
2021-01-27 9:58 ` Marc Zyngier
2021-01-28 9:34 ` Suzuki K Poulose
2021-01-28 9:46 ` Marc Zyngier
2021-01-28 9:48 ` Suzuki K Poulose
2021-01-27 8:55 ` [PATCH V3 11/14] coresight: sink: Add TRBE driver Anshuman Khandual
[not found] ` <12cdc8ca-0a69-bfba-bbcd-fb392d6ca051@arm.com>
2021-02-02 5:55 ` Anshuman Khandual
2021-02-05 17:53 ` Mathieu Poirier
2021-02-08 4:20 ` Anshuman Khandual
2021-02-09 17:39 ` Mathieu Poirier
2021-02-10 4:12 ` Anshuman Khandual
2021-02-10 16:54 ` Mathieu Poirier
2021-02-10 19:00 ` Mathieu Poirier
2021-02-12 5:43 ` Anshuman Khandual
2021-02-12 17:02 ` Mathieu Poirier [this message]
2021-02-11 19:00 ` Mathieu Poirier
2021-02-12 3:31 ` Anshuman Khandual
2021-02-12 16:57 ` Mathieu Poirier
2021-02-15 9:26 ` Anshuman Khandual
2021-02-12 20:26 ` Mathieu Poirier
2021-02-15 9:46 ` Anshuman Khandual
2021-02-16 9:00 ` Mike Leach
2021-02-16 9:44 ` Anshuman Khandual
2021-02-16 12:12 ` Mike Leach
2021-02-18 7:50 ` Suzuki K Poulose
2021-02-18 14:30 ` Mike Leach
2021-02-18 15:14 ` Suzuki K Poulose
2021-02-22 10:42 ` Mike Leach
2021-01-27 8:55 ` [PATCH V3 12/14] dts: bindings: Document device tree bindings for Arm TRBE Anshuman Khandual
2021-02-09 19:04 ` Rob Herring
2021-01-27 8:55 ` [PATCH V3 13/14] perf: aux: Add flags for the buffer format Anshuman Khandual
2021-01-27 12:51 ` Peter Zijlstra
2021-02-16 10:59 ` Mike Leach
2021-01-27 8:55 ` [PATCH V3 14/14] coresight: etm-perf: Add support for trace " Anshuman Khandual
2021-01-27 12:54 ` Peter Zijlstra
2021-01-27 13:00 ` Al Grant
2021-02-18 3:05 ` Anshuman Khandual
2021-01-27 14:12 ` Suzuki K Poulose
2021-02-16 11:01 ` Mike Leach
2021-01-27 18:50 ` [PATCH V3 00/14] arm64: coresight: Enable ETE and TRBE Mathieu Poirier
2021-02-01 18:44 ` Mathieu Poirier
2021-02-18 4:23 ` Anshuman Khandual
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210212170232.GB2692426@xps15 \
--to=mathieu.poirier@linaro.org \
--cc=anshuman.khandual@arm.com \
--cc=coresight@lists.linaro.org \
--cc=lcherian@marvell.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mike.leach@linaro.org \
--cc=suzuki.poulose@arm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).