Re: [PATCH 0/4] coresight: Add ETR-PERF polling.

From: Leo Yan <leo.yan@linaro.org>
To: Denis Nikitin <denik@google.com>
Cc: Mathieu Poirier <mathieu.poirier@linaro.org>,
	Mike Leach <mike.leach@linaro.org>,
	"coresight@lists.linaro.org" <coresight@lists.linaro.org>,
	"linux-arm-kernel@lists.infradead.org"
	<linux-arm-kernel@lists.infradead.org>,
	 Daniel Kiss <Daniel.Kiss@arm.com>
Subject: Re: [PATCH 0/4] coresight: Add ETR-PERF polling.
Date: Tue, 18 May 2021 22:00:40 +0800	[thread overview]
Message-ID: <20210518140040.GC942578@leoy-ThinkPad-X240s> (raw)
In-Reply-To: <CAOYpmdGb8GL1-h+NRF1DCuVfgiT5sEEqQUWzOreG8soUhN8Ceg@mail.gmail.com>

Hi Denis,

On Fri, May 14, 2021 at 02:02:25AM -0700, Denis Nikitin wrote:

[...]

> Hi Mattieu and Leo,
> 
> I did some evaluation of the snapshot mode.

Thanks a lot for the evaluation and share back the result.

> Performance overhead is indeed higher than with ETR polling patch.
> Here are some numbers for comparison (measured on browser
> Speedometer2 benchmark):
> Runtime overhead of ETM tracing with ETR poll period 100ms is less than
> 0.5%. Snapshot mode gives 2.1%.
> With 10ms period I see 4.6% with ETR polling and 22% in snapshot mode.

It's not expected that the snapshot mode causes so big overload.
In my head, two factors might cause the overload:

- The perf interaction between the user space and kernel space;
- The data copying from the ETR's buffer to the AUX ring buffer.

Check one thing: what's the buffer size for ETR polling mode and for
snapshot mode in your experiments?

If I remember correctly, by default the snapshot mode uses 4MB for ETR
buffer, if copying 4MB per 10ms, then it's likely to cause big
overload.  So at the first glance, the overhead difference might be
caused by the by the different buffer size between ETR poll mode and
snapshot mode.

> We could probably utilize the ETM strobing feature and reduce frequency
> of data collection but I see a problem when I'm using both.
> Within a minute of profiling the ETM generates a reasonable profile size
> (with strobing autofdo,preset=9 with period 0x1000 it is up to 20MB).
> But then the size grows unproportionally.
> With a 4 minute run I got a 6.3GB profile.

Just check, as Mathieu has suggested, have you applied the patch [1]
on your local code base for fixing the data copying for snapshot mode?

After applied this patch, one possibility for unproportional issue is
perf tool itself introduces many activities in snapshot mode (especially
for 10ms period), so the perf tool contributes much extra trace data.

Another potential issue is: after setting strobing mode, the snapshot
mode will disable the complete paths for tracers and ETR, so if the
strobing configuration is lost after re-enable tracers, then it might
cause the huge trace data in the later phase.  For this case, we
definitely should fix it.

> I don't see such a problem with the ETR polling patch.
> 
> Leo, could you please take a look at this problem?

Sure.  For easier reproducing the issue, could you share me the
detailed commands (and source code)?

P.s. I saw Mike suggests to continue the ETR polling development, this
is not conflict with snapshot mode.  At my side, I will investigate the
snapshot mode, but don't want to disturb the process for ETR polling
mode, so when the ETR polling patch series is get ready, please go
ahead for upstreaming the patch series.

Thanks,
Leo

[1]. https://lists.linaro.org/pipermail/coresight/2021-April/006254.html

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel