* [BENCHMARK] Statistical representation of IO load results with contest
@ 2002-09-23 23:26 Con Kolivas
2002-09-23 23:41 ` Ryan Anderson
2002-09-24 5:28 ` Mike Galbraith
0 siblings, 2 replies; 3+ messages in thread
From: Con Kolivas @ 2002-09-23 23:26 UTC (permalink / raw)
To: linux-kernel
Cc: Andrew Morton, Ryan Anderson, Ingo Molnar, Richard B. Johnson,
Oliver Xymoron, Daniel Jacobowitz, Paolo Ciarrocchi,
Robinson Maureira Castillo, gcc
Thank you all who responded with suggestions on how to get useful data out of
the IO load module from contest. These are _new_ results with a
sync,swapoff,swapon before conducting just the IO load. I have digested all your
suggestions and come up with the following:
n=5 for number of samples
Kernel Mean CI(95%)
2.5.38 411 344-477
2.5.39-gcc32 371 224-519
2.5.38-mm2 95 84-105
The mean is a simple average of the results, and the CI(95%) are the 95%
confidence intervals the mean lies between those numbers. These numbers seem to
be the most useful for comparison.
Comparing 2.5.38(gcc2.95.3) with 2.5.38(gcc3.2) there is NO significant
difference (p 0.56)
Comparing 2.5.38 with 2.5.38-mm2 there is a significant diffence (p<0.001)
After playing with all these it appears I should do the following to contest:
Add sync,swapoff,swapon before each load
Perform noload and process_load twice to ensure no abnormal results
Perform mem_load 3 times
Perform IO_fullmem 5 times (and rename it just IO_load)
Drop IO_halfmem (adds no more useful information and just adds time).
Do a statistical analysis like the above when posting information.
Comments?
Con
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [BENCHMARK] Statistical representation of IO load results with contest
2002-09-23 23:26 [BENCHMARK] Statistical representation of IO load results with contest Con Kolivas
@ 2002-09-23 23:41 ` Ryan Anderson
2002-09-24 5:28 ` Mike Galbraith
1 sibling, 0 replies; 3+ messages in thread
From: Ryan Anderson @ 2002-09-23 23:41 UTC (permalink / raw)
To: Con Kolivas
Cc: linux-kernel, Andrew Morton, Ryan Anderson, Ingo Molnar,
Richard B. Johnson, Oliver Xymoron, Daniel Jacobowitz,
Paolo Ciarrocchi, Robinson Maureira Castillo, gcc
On Tue, Sep 24, 2002 at 09:26:38AM +1000, Con Kolivas wrote:
> Do a statistical analysis like the above when posting information.
That's really the most important piece, imo.
--
Ryan Anderson
sometimes Pug Majere
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [BENCHMARK] Statistical representation of IO load results with contest
2002-09-23 23:26 [BENCHMARK] Statistical representation of IO load results with contest Con Kolivas
2002-09-23 23:41 ` Ryan Anderson
@ 2002-09-24 5:28 ` Mike Galbraith
1 sibling, 0 replies; 3+ messages in thread
From: Mike Galbraith @ 2002-09-24 5:28 UTC (permalink / raw)
To: linux-kernel
At 09:26 AM 9/24/2002 +1000, Con Kolivas wrote:
>Thank you all who responded with suggestions on how to get useful data out of
>the IO load module from contest. These are _new_ results with a
>sync,swapoff,swapon before conducting just the IO load. I have digested
>all your
>suggestions and come up with the following:
>
>n=5 for number of samples
>
>Kernel Mean CI(95%)
>2.5.38 411 344-477
>2.5.39-gcc32 371 224-519
>2.5.38-mm2 95 84-105
/me wonders what other benchmarks numbers will look like. (db stuff should
rock?)
-Mike
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2002-09-24 5:26 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2002-09-23 23:26 [BENCHMARK] Statistical representation of IO load results with contest Con Kolivas
2002-09-23 23:41 ` Ryan Anderson
2002-09-24 5:28 ` Mike Galbraith
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).