Re: major variations in perfomance figures

Re: major variations in perfomance figures

[Jérémie Galarneau]

On Tue, Mar 31, 2015 at 3:46 PM, Wolfgang Rostek <wolfgang.rostek@gmx.de> wrote:
> Hi all,
>
> I did a first perfomance test extending the demo example
> by tracing 30 simple integers in a loop.
>
> For a similar CPU (i5 quad core 2.8GHz) I saw values
> mentioned around 250ns in the forum.
>
> I've tried the test several time and could come down
> to about 350ns. The machine was almost idle for all
> tests.
>
> What makes me wonder is a large variation with frequent
> runs giving me 750-800ns.
>
> Not the absolute time but the variations isn't clear
> to me. From my understanding the caller path is more
> or less straight to shared memory. Why do different
> runs show more than double the time for the traces?

Good question. Can you share the benchmark's code?
Also, make sure frequency scaling is disabled on your system; it could
explain these kinds of variations.

Jérémie

>
> Wolfgang R.
>
>
>
> _______________________________________________
> lttng-dev mailing list
> lttng-dev@lists.lttng.org
> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev



-- 
Jérémie Galarneau
EfficiOS Inc.
http://www.efficios.com

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: major variations in perfomance figures

[Wolfgang Rostek]

Hi Jérémie,

thanks for your feedback.

I've added only a loop to the end of the demo programm.

Frequence scaling I can't imagine on my dektop machine. 
It is a jump between 350ns and 800ns. In the loop below
all the events of one test run are of equal time distance, 
either slow or fast.

Wolfgang R.


https://github.com/lttng/lttng-ust/blob/master/doc/examples/demo/demo.c
...
tracepoint(ust_tests_demo, starting, 123);
for (i = 0; i < 5; i++) {
netint = htonl(i);
tracepoint(ust_tests_demo2, loop, i, netint, values,
text, strlen(text), dbl, flt);
}
tracepoint(ust_tests_demo, done, 456);
tracepoint(ust_tests_demo3, done, 42);

for (i = 0; i < 300; i++) {
if(i % 10 == 0) {
tracepoint(ust_tests_demo, done, i);         <=========
}
}

fprintf(stderr, " done.\n");
return 0;
}

> Gesendet: Freitag, 03. April 2015 um 16:22 Uhr
> Von: "Jérémie Galarneau" <jeremie.galarneau@efficios.com>
> An: "Wolfgang Rostek" <wolfgang.rostek@gmx.de>
> Cc: "lttng-dev@lists.lttng.org" <lttng-dev@lists.lttng.org>
> Betreff: Re: [lttng-dev] major variations in perfomance figures
>
> On Tue, Mar 31, 2015 at 3:46 PM, Wolfgang Rostek <wolfgang.rostek@gmx.de> wrote:
> > Hi all,
> >
> > I did a first perfomance test extending the demo example
> > by tracing 30 simple integers in a loop.
> >
> > For a similar CPU (i5 quad core 2.8GHz) I saw values
> > mentioned around 250ns in the forum.
> >
> > I've tried the test several time and could come down
> > to about 350ns. The machine was almost idle for all
> > tests.
> >
> > What makes me wonder is a large variation with frequent
> > runs giving me 750-800ns.
> >
> > Not the absolute time but the variations isn't clear
> > to me. From my understanding the caller path is more
> > or less straight to shared memory. Why do different
> > runs show more than double the time for the traces?
> 
> Good question. Can you share the benchmark's code?
> Also, make sure frequency scaling is disabled on your system; it could
> explain these kinds of variations.
> 
> Jérémie
> 
> >
> > Wolfgang R.
> >
> >
> >
> > _______________________________________________
> > lttng-dev mailing list
> > lttng-dev@lists.lttng.org
> > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
> 
> 
> 
> -- 
> Jérémie Galarneau
> EfficiOS Inc.
> http://www.efficios.com
>

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: major variations in perfomance figures

[Jérémie Galarneau]

On Fri, Apr 3, 2015 at 10:47 AM, Wolfgang Rostek <wolfgang.rostek@gmx.de> wrote:
> Hi Jérémie,
>
> thanks for your feedback.
>
> I've added only a loop to the end of the demo programm.
>
> Frequence scaling I can't imagine on my dektop machine.
> It is a jump between 350ns and 800ns. In the loop below
> all the events of one test run are of equal time distance,
> either slow or fast.

Frequency scaling can still definitely play a role, even on a desktop machine.
Also, Core i5 chips support Turbo Boost which should be disabled to
perform benchmarks.

I'm guessing you get your time measurements from the timestamps in the
trace?

Jérémie

>
> Wolfgang R.
>
>
> https://github.com/lttng/lttng-ust/blob/master/doc/examples/demo/demo.c
> ...
> tracepoint(ust_tests_demo, starting, 123);
> for (i = 0; i < 5; i++) {
> netint = htonl(i);
> tracepoint(ust_tests_demo2, loop, i, netint, values,
> text, strlen(text), dbl, flt);
> }
> tracepoint(ust_tests_demo, done, 456);
> tracepoint(ust_tests_demo3, done, 42);
>
> for (i = 0; i < 300; i++) {
> if(i % 10 == 0) {
> tracepoint(ust_tests_demo, done, i);         <=========
> }
> }
>
> fprintf(stderr, " done.\n");
> return 0;
> }
>
>> Gesendet: Freitag, 03. April 2015 um 16:22 Uhr
>> Von: "Jérémie Galarneau" <jeremie.galarneau@efficios.com>
>> An: "Wolfgang Rostek" <wolfgang.rostek@gmx.de>
>> Cc: "lttng-dev@lists.lttng.org" <lttng-dev@lists.lttng.org>
>> Betreff: Re: [lttng-dev] major variations in perfomance figures
>>
>> On Tue, Mar 31, 2015 at 3:46 PM, Wolfgang Rostek <wolfgang.rostek@gmx.de> wrote:
>> > Hi all,
>> >
>> > I did a first perfomance test extending the demo example
>> > by tracing 30 simple integers in a loop.
>> >
>> > For a similar CPU (i5 quad core 2.8GHz) I saw values
>> > mentioned around 250ns in the forum.
>> >
>> > I've tried the test several time and could come down
>> > to about 350ns. The machine was almost idle for all
>> > tests.
>> >
>> > What makes me wonder is a large variation with frequent
>> > runs giving me 750-800ns.
>> >
>> > Not the absolute time but the variations isn't clear
>> > to me. From my understanding the caller path is more
>> > or less straight to shared memory. Why do different
>> > runs show more than double the time for the traces?
>>
>> Good question. Can you share the benchmark's code?
>> Also, make sure frequency scaling is disabled on your system; it could
>> explain these kinds of variations.
>>
>> Jérémie
>>
>> >
>> > Wolfgang R.
>> >
>> >
>> >
>> > _______________________________________________
>> > lttng-dev mailing list
>> > lttng-dev@lists.lttng.org
>> > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>>
>>
>>
>> --
>> Jérémie Galarneau
>> EfficiOS Inc.
>> http://www.efficios.com
>>



-- 
Jérémie Galarneau
EfficiOS Inc.
http://www.efficios.com

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: major variations in perfomance figures

[Wolfgang Rostek]

...
> > Frequence scaling I can't imagine on my dektop machine.
> > It is a jump between 350ns and 800ns. In the loop below
> > all the events of one test run are of equal time distance,
> > either slow or fast.
> 
> Frequency scaling can still definitely play a role, even on a desktop machine.
> Also, Core i5 chips support Turbo Boost which should be disabled to
> perform benchmarks.
> 
> I'm guessing you get your time measurements from the timestamps in the
> trace?
Yes

I'm not hunting for a detailed benchmark. Only such a large variance makes
me thoughtful. For such a test I did expect some 10-30% but not this more 
than 200%.

Wolfgang R.

Re: major variations in perfomance figures

[Jesper Derehag]

[-- Attachment #1: Type: text/html, Size: 1782 bytes --]

[-- Attachment #2: Type: text/plain, Size: 155 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

major variations in perfomance figures

[Wolfgang Rostek]

Hi all,

I did a first perfomance test extending the demo example
by tracing 30 simple integers in a loop.

For a similar CPU (i5 quad core 2.8GHz) I saw values
mentioned around 250ns in the forum.

I've tried the test several time and could come down
to about 350ns. The machine was almost idle for all
tests.

What makes me wonder is a large variation with frequent
runs giving me 750-800ns.

Not the absolute time but the variations isn't clear
to me. From my understanding the caller path is more
or less straight to shared memory. Why do different
runs show more than double the time for the traces?

Wolfgang R.