[RFC PATCH] rteval: cyclictest.py: Skip statistics reporting in case of an overflow

[RFC PATCH] rteval: cyclictest.py: Skip statistics reporting in case of an overflow

[Punit Agrawal]

The cyclictest.py module recently gained the capability to parse max
latency values as reported by cyclictest.

When the max latency exceeds the range of the latency histogram (or in
other words, the number of configured buckets), statistics such as
mean and standard deviation can not be calculated correctly due to
lost samples during measurement.

In the case of lost samples, skip statistics generation and report the
max latency warning the user to rerun the measurement.

Signed-off-by: Punit Agrawal <punitagrawal@gmail.com>
---

Hi,

The patch depends on the improvements to cyclictest[0] posted sometime
ago.

In certain environments such as virtualized systems, latencies can be
quite high - such that the samples exceed the range of the histogram
and rteval reports incorrect statistics such as mean and standard
deviation due to missing data.

Now that it is possible to parse the real max latency, it would be
good to use this to detect situations where max latency exceeds
histogram range and take appropriate action.

There are a few different response (or combination thereof) to the
error condition -

* log a message warning the user
* skip statistics reporting in the report
* add an error message in the generated report.

This patch implements the first two points.

I am posting the patch as an RFC to get feedback on the appropriate
action to take in such a situation.

Thoughts?

Thanks,
Punit

[0] https://lore.kernel.org/all/20210917085343.2775300-1-punitagrawal@gmail.com/

 rteval/modules/measurement/cyclictest.py | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/rteval/modules/measurement/cyclictest.py b/rteval/modules/measurement/cyclictest.py
index 50a734c75047..7a619dd1c852 100644
--- a/rteval/modules/measurement/cyclictest.py
+++ b/rteval/modules/measurement/cyclictest.py
@@ -67,6 +67,9 @@ class RunData:
         retval += "mean:       %f\n" % self.__mean
         return retval
 
+    def get_max(self):
+        return self.__max
+
     def update_max(self, value):
         if value > self.__max:
             self.__max = value
@@ -423,6 +426,13 @@ class Cyclictest(rtevalModulePrototype):
         if abrt:
             rep_n.addChild(abrt_n)
 
+        # Let the user know if max latency overshot the number of buckets
+        if self.__cyclicdata["system"].get_max() > self.__buckets:
+            self._log(Log.ERR, "Max latency(%dus) exceeded histogram range(%dus). Skipping statistics" %
+                      (self.__cyclicdata["system"].get_max(), self.__buckets))
+            self._log(Log.ERR, "Increase number of buckets to avoid lost samples")
+            return rep_n
+
         rep_n.addChild(self.__cyclicdata["system"].MakeReport())
         for thr in self.__cpus:
             if str(thr) not in self.__cyclicdata:
-- 
2.32.0