Re: Adding latency tracking to trace-cmd

[Steven Rostedt <rostedt@goodmis.org>]

On Tue, 6 Apr 2021 20:03:46 +0000
<Viktor.Rosendahl@bmw.de> wrote:

> Hi Steve,
> 
> The management was agreeable to it but I haven't had any spare time to work on
> it. 

Awesome.

> 
> To be honest, I could have worked on it during my Easter holidays but I just
> happened to busy myself with other private interests :)

I wouldn't expect you to use your Easter holidays on this anyway ;-)

I was just clearing out my INBOX and noticed that it's been a while, and
decided to ping you so we don't forget.

>> > > 
> > > Would you allow the hackish random algorithm to be used in trace-cmd?
> > > I mean the "--random" option in latency-collector.
> > >   
> > 
> > If you feel it is useful, sure. Although a definition of what it exactly
> > does needs to be a bit more explained. I'm not really sure what use the
> > --random option is for the latency tracer.
> >   
> 
> The usefulness lies in the case where you have latencies that systematically
> occur very close to each other.

[..]

> 
> Without the --random option, I would at this point need to start another test
> campaign, only then would I start seeing the longer latencies from bar().
> 
> On the other hand, if I use the --random option with the latency-collector, then
> what will happen is that sometimes the latency-collector will open the trace
> file immediately and other times it will sleep and wait for up to one second
> before doing so. If it opens the file immediately, we will get the first
> latency. If based on that random toin coss function decides to sleep, then we
> will get the second.
> 
> If a long test camaping is exectuted, and foobar() is called several times
> during that campaign, then there is a good probability that we will capture both
> the latency from foo() and the one from bar().
> Now --random is a bit more complicated because it actually tosses the coin again
> if another latency occurs when it is sleeping before opening the file. The
> probability of that coin toss function are chosen so that if we assume that
> there is a maximum of N closely occuring latencies, we will get each of them
> with probability 1/N. If the latency-collector detects a situation were it
> actually has detected N closely occuring latencies, it will automatically
> increase N to N + 1 and update the probablities of the coin toss accordingly.
> 
> So basically the idea is that by acting randomly, we will not systematically
> lose a particular latency. It will also not make matters worse if we never get
> latencies that occur close to each other; the only drawback is that we sometimes
> wait for one second before opening the trace file and that doesn't really matter
> if there aren't any closely occuring latencies.


Hmm, sounds more like "--various" would be better than "--random". Just
because it appears you wont to try different timings. Having a --random
option just doesn't sound like it's what you expect it to be.

There's already a "-s" option that takes a sleep interval between wakeups,
which sounds similar to what you have. Perhaps we can make "-1" a special
value to do the "random" wakeup thing. "0" is already special to make it
wake up when it detects data in the buffer.

-- Steve