Re: [PATCH V3 01/14] coresight: etm-perf: Allow an event to use different sinks

[Mathieu Poirier <mathieu.poirier@linaro.org>]

On Tue, Feb 02, 2021 at 09:42:34AM +0000, Suzuki K Poulose wrote:
> On 2/1/21 11:17 PM, Mathieu Poirier wrote:
> > Hi Anshuman,
> > 
> > I have started reviewing this set.  As it is quite voluminous comments will
> > come over serveral days.  I will let you know when I am done.
> > 
> > On Wed, Jan 27, 2021 at 02:25:25PM +0530, Anshuman Khandual wrote:
> > > From: Suzuki K Poulose <suzuki.poulose@arm.com>
> > > 
> > > When there are multiple sinks on the system, in the absence
> > > of a specified sink, it is quite possible that a default sink
> > > for an ETM could be different from that of another ETM. However
> > > we do not support having multiple sinks for an event yet. This
> > > patch allows the event to use the default sinks on the ETMs
> > > where they are scheduled as long as the sinks are of the same
> > > type.
> > > 
> > > e.g, if we have 1x1 topology with per-CPU ETRs, the event can
> > > use the per-CPU ETR for the session. However, if the sinks
> > > are of different type, e.g TMC-ETR on one and a custom sink
> > > on another, the event will only trace on the first detected
> > > sink.
> > > 
> > 
> > I found the above changelog very confusing - I read it several times and still
> > couldn't get all of it.  In the end this patch prevents sinks of different types
> > from being used for session, and this is what the text should reflect.
> 
> Sorry about that. Your inference is correct, but it is only a side effect
> of the primary motive. How about the following :
> 
> "When a sink is not specified by the user, the etm perf driver
> finds a suitable sink automatically based on the first ETM, where
> this event could be scheduled. Then we allocate the sink buffer based
> on the selected sink. This is fine for a CPU bound event as the "sink"
> is always guaranteed to be reachable from the ETM (as this is the only
> ETM where the event is going to be scheduled). However, if we have a task
> bound event, the event could be scheduled on any of the ETMs on the
> system. In this case, currently we automatically select a sink and exclude
> any ETMs that are not reachable from the selected sink. This is
> problematic for 1x1 configurations as we end up in tracing the event
> only on the "first" ETM, as the default sink is local to the first
> ETM and unreachable from the rest.
> However, we could allow the other ETMs to trace if they all have a
> sink that is compatible with the "selected" sink and can use the
> sink buffer. This can be easily done by verifying that they are
> all driven by the same driver and matches the same subtype."
>

Much better, thanks for the rework.
 
> 
> > > Cc: Mathieu Poirier <mathieu.poirier@linaro.org>
> > > Cc: Mike Leach <mike.leach@linaro.org>
> > > Tested-by: Linu Cherian <lcherian@marvell.com>
> > > Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
> > > Signed-off-by: Anshuman Khandual <anshuman.khandual@arm.com>
> > > ---
> > >   drivers/hwtracing/coresight/coresight-etm-perf.c | 48 +++++++++++++++++++-----
> > >   1 file changed, 38 insertions(+), 10 deletions(-)
> > > 
> > > diff --git a/drivers/hwtracing/coresight/coresight-etm-perf.c b/drivers/hwtracing/coresight/coresight-etm-perf.c
> > > index bdc34ca..eb9e7e9 100644
> > > --- a/drivers/hwtracing/coresight/coresight-etm-perf.c
> > > +++ b/drivers/hwtracing/coresight/coresight-etm-perf.c
> > > @@ -204,6 +204,13 @@ static void etm_free_aux(void *data)
> > >   	schedule_work(&event_data->work);
> > >   }
> > > +static bool sinks_match(struct coresight_device *a, struct coresight_device *b)
> > > +{
> > > +	if (!a || !b)
> > > +		return false;
> > > +	return (sink_ops(a) == sink_ops(b));
> > 
> > Yes
> 
> I think we can tighten this by verifying the dev->sub_type matches too.
> 

We could do that but I'm not sure we need to.  I remember spending a few minutes
yesterday thinking about ways to make the test more stringent but in the end I
thought what you had was sufficient, at least for now.  I'll leave that one to
you - proceed as you see fit. 

> > 
> > > +}
> > > +
> > >   static void *etm_setup_aux(struct perf_event *event, void **pages,
> > >   			   int nr_pages, bool overwrite)
> > >   {
> > > @@ -212,6 +219,7 @@ static void *etm_setup_aux(struct perf_event *event, void **pages,
> > >   	cpumask_t *mask;
> > >   	struct coresight_device *sink = NULL;
> > 
> >          struct coresight_device *user_sink = NULL;
> > 
> > >   	struct etm_event_data *event_data = NULL;
> > > +	bool sink_forced = false;
> > >   	event_data = alloc_event_data(cpu);
> > >   	if (!event_data)
> > > @@ -222,6 +230,7 @@ static void *etm_setup_aux(struct perf_event *event, void **pages,
> > >   	if (event->attr.config2) {
> > >   		id = (u32)event->attr.config2;
> > >   		sink = coresight_get_sink_by_id(id);
> > 
> >                  user_sink = coresight_get_sink_by_id(id);
> > 
> > > +		sink_forced = true;
> > >   	}
> > >   	mask = &event_data->mask;
> > > @@ -235,7 +244,7 @@ static void *etm_setup_aux(struct perf_event *event, void **pages,
> > >   	 */
> > >   	for_each_cpu(cpu, mask) {
> > >   		struct list_head *path;
> > > -		struct coresight_device *csdev;
> > 
> >                  struct coresight_device *last_sink = NULL;
> > 
> > > +		struct coresight_device *csdev, *new_sink;
> > >   		csdev = per_cpu(csdev_src, cpu);
> > >   		/*
> > > @@ -249,21 +258,35 @@ static void *etm_setup_aux(struct perf_event *event, void **pages,
> > >   		}
> > >   		/*
> > > -		 * No sink provided - look for a default sink for one of the
> > > -		 * devices. At present we only support topology where all CPUs
> > > -		 * use the same sink [N:1], so only need to find one sink. The
> > > -		 * coresight_build_path later will remove any CPU that does not
> > > -		 * attach to the sink, or if we have not found a sink.
> > > +		 * No sink provided - look for a default sink for all the devices.
> > > +		 * We only support multiple sinks, only if all the default sinks
> > > +		 * are of the same type, so that the sink buffer can be shared
> > > +		 * as the event moves around. We don't trace on a CPU if it can't
> > 
> > s/can't/can't./
> > 
> > > +		 *
> > 
> > Extra line
> > 
> 
> OK
> 
> > >   		 */
> > > -		if (!sink)
> > > -			sink = coresight_find_default_sink(csdev);
> > > +		if (!sink_forced) {
> > > +			new_sink = coresight_find_default_sink(csdev);
> > > +			if (!new_sink) {
> > > +				cpumask_clear_cpu(cpu, mask);
> > > +				continue;
> > > +			}
> > > +			/* Skip checks for the first sink */
> > > +			if (!sink) {
> > > +			       sink = new_sink;
> > > +			} else if (!sinks_match(new_sink, sink)) {
> > > +				cpumask_clear_cpu(cpu, mask);
> > > +				continue;
> > > +			}
> > > +		} else {
> > > +			new_sink = sink;
> > > +		}
> > 
> >                  if (!user_sink) {
> >                          /* find default sink for this CPU */
> >                          sink = coresight_find_default_sink(csdev);
> >                          if (!sink) {
> >                                  cpumask_clear_cpu(cpu, mask);
> >                                  continue;
> >                          }
> > 
> >                          /* Chech new sink with last sink */
> >                          if (last_sink && !sink_match(last_sink, sink)) {
> >                                  cpumask_clear_cpu(cpu, mask);
> >                                  continue;
> >                          }
> > 
> >                          last_sink = sink;
> >                  } else {
> >                          sink = user_sink;
> >                  }
> > 
> 
> Agreed, it is much better readable.
> 
> Suzuki