dd test discussion (was Re: possible regression?)

dd test discussion (was Re: possible regression?)

[Mulyadi Santosa]

On Fri, Jan 21, 2011 at 20:25, Mag Gam <magawake@gmail.com> wrote:
> Typo, in Box 5. Wrong kernel version
>
> Box 5, RHEL 5.4, 2.6.36, dd bs=1024 count=1000000 if=/dev/zero
> bs=4096k of=/dev/null
>
> Anyone had a chance to try this? :-)

$ for a in `seq 1 3`; do \time dd bs=4K count=1M if=/dev/zero of=/dev/null; done

1048576+0 records in
1048576+0 records out
4294967296 bytes (4.3 GB) copied, 1.38893 seconds, 3.1 GB/s
0.21user 1.17system 0:01.39elapsed 99%CPU (0avgtext+0avgdata 2768maxresident)k
0inputs+0outputs (0major+209minor)pagefaults 0swaps

1048576+0 records in
1048576+0 records out
4294967296 bytes (4.3 GB) copied, 1.39358 seconds, 3.1 GB/s
0.23user 1.16system 0:01.39elapsed 99%CPU (0avgtext+0avgdata 2768maxresident)k
0inputs+0outputs (0major+209minor)pagefaults 0swaps


1048576+0 records in
1048576+0 records out
4294967296 bytes (4.3 GB) copied, 1.4527 seconds, 3.0 GB/s
0.17user 1.23system 0:01.45elapsed 96%CPU (0avgtext+0avgdata 2752maxresident)k
0inputs+0outputs (0major+208minor)pagefaults 0swaps


I slightly modified your dd test. Again, this is on 2.6.36.

IMO, you shouldn't worry much about it. Were experiencing something
that related to memory access? It's not a NUMA box, right? Or are you
using PAE? just capping into 896 MB maybe? non standart VM split other
than 3:1?

PS: I rewrite the subject to better match the discussion. With the
hope readers well get better understanding about what the discussion
is all about. Hope you don't mind....

-- 
regards,

Mulyadi Santosa
Freelance Linux trainer and consultant

blog: the-hydra.blogspot.com
training: mulyaditraining.blogspot.com

dd test discussion (was Re: possible regression?)

[Henry Gebhardt]

On Sat, Jan 22, 2011 at 11:49:43AM +0700, Mulyadi Santosa wrote:
> On Fri, Jan 21, 2011 at 20:25, Mag Gam <magawake@gmail.com> wrote:
> > Anyone had a chance to try this? :-)
> 
> $ for a in `seq 1 3`; do \time dd bs=4K count=1M if=/dev/zero of=/dev/null; done

Hm, I also wrote a little script to go through more values of bs.  The
script is attached.  To change what values to use for bs, change the
variables $min, $step, and $max at the beginning of the script, which
are the parameters for "seq".  To get a graph of throughput versus bs,
use it like so:

   $ ./pipespeed.sh > data
   [...]
   $ gnuplot
   plot "data" w l

The highest I get is 8.5 GB/s with bs=22528 on 2.6.37.

How can you get a speed of 30 GB/s?  It seems excessively fast. :)


Greetings,

Henry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pipespeed.sh
Type: application/x-sh
Size: 809 bytes
Desc: not available
Url : http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20110122/4029d980/attachment.sh 

dd test discussion (was Re: possible regression?)

[Mulyadi Santosa]

Hi....

On Sat, Jan 22, 2011 at 19:00, Henry Gebhardt
<hsggebhardt@googlemail.com> wrote:
> How can you get a speed of 30 GB/s? ?It seems excessively fast. :)

I'd like to comment it first :) You read it incorrectly...it's 3.0, not 30 :D

> Greetings,
>
> Henry
>



-- 
regards,

Mulyadi Santosa
Freelance Linux trainer and consultant

blog: the-hydra.blogspot.com
training: mulyaditraining.blogspot.com

dd test discussion (was Re: possible regression?)

[Mulyadi Santosa]

Hi again :)

On Sat, Jan 22, 2011 at 19:00, Henry Gebhardt
<hsggebhardt@googlemail.com> wrote:
> The highest I get is 8.5 GB/s with bs=22528 on 2.6.37.

Thanks for sharing your script. Looking at the gnuplot output (again,
thanks for the gnuplot quick hint :) ), I saw it is capped around 7.26
GiB/s, with block size around 300,000-400,000 (that means 293 - 391
KiB, right?)

It's 2.6.36 btw...

I think major factor that involve here, but not limited to them, are:
- L1/L2/L3 cache size
- slab allocator we use
- current memory fragmentation level
- read ahead size


-- 
regards,

Mulyadi Santosa
Freelance Linux trainer and consultant

blog: the-hydra.blogspot.com
training: mulyaditraining.blogspot.com

dd test discussion (was Re: possible regression?)

[Henry Gebhardt]

On Sat, Jan 22, 2011 at 11:30:52PM +0700, Mulyadi Santosa wrote:
> On Sat, Jan 22, 2011 at 19:00, Henry Gebhardt wrote:
> > How can you get a speed of 30 GB/s? ?It seems excessively fast. :)
> 
> I'd like to comment it first :) You read it incorrectly...it's 3.0, not 30 :D

I was referring to the very first email in the original thread:

On Wed, 19 Jan 2011 22:36:14 -0500, Mag Gam wrote:
> I get around 30Gb/sec
> 
> However, when I do this with 2.6.37 I get close to 5GB/sec

Hm, I just noticed, its a 'b' on the first line, but capitalized, 'B',
later.  Just a typo?


Greetings,

Henry

dd test discussion (was Re: possible regression?)

[Mulyadi Santosa]

On Sun, Jan 23, 2011 at 03:09, Henry Gebhardt
<hsggebhardt@googlemail.com> wrote:
> I was referring to the very first email in the original thread:

Oh that, Mag's...alright now I understand.

> Hm, I just noticed, its a 'b' on the first line, but capitalized, 'B',
> later. ?Just a typo?

Maybe...but AFAIK these days sometimes people confusingly
interchangeably use "b" and "B", "G" and "Gi" without really knowing
what they really mean. Of course, I don't mean Mag must be falling
into the same trap.

But assuming it's really "b", it means it is around 3.75 GiB/s... that
could be lower if he refer to "G", not "Gi"...

-- 
regards,

Mulyadi Santosa
Freelance Linux trainer and consultant

blog: the-hydra.blogspot.com
training: mulyaditraining.blogspot.com