random minor benchmark: Re: Copy 20 tarfiles: ext2 vs (reiser4, unixfile) vs (reiser4,cryptcompress)

random minor benchmark: Re: Copy 20 tarfiles: ext2 vs (reiser4, unixfile) vs (reiser4,cryptcompress)

[Hans Reiser]

Notice how CPU speed (and number of cpus) completely determines
compression performance.

cryptcompress refers to the reiser4 compression plugin, (unix file)
refers to the reiser4 non-compressing plugin.

Edward Shishkin wrote:

> Here are the tests that vs asked for:
> Creation (dd) of 20 tarfiles (the original 200M file is in ramfs)
> Kernel: 2.6.15-mm4 + current git snapshot of reiser4
>
> ------------------------------------------
>
> Laputa workstation
> Uni Intel Pentium 4 (2.26 GHz) 512M RAM
>
> ext2:
> real 2m, 15s
> sys 0m, 14s
>
> reiser4(unix file)
> real 2m, 7s
> sys  0m, 23s
>
> reiser4(cryptcompress, lzo1, 64K)
> real 2m, 13s
> sys 0m, 11s
> ------------------------------------------
>
> Belka workstation
> Dual Intel Xeon (2.4GHz) 1G RAM
>
> ext2:
> real 2m, 16s
> sys 0m, 10s
>
> reiser4(unix file)
> real 2m, 14s
> sys  0m, 17s
>
> reiser4(cryptcompress, lzo1, 64K)
> real 1m, 35s
> sys 0m, 14s
>
>
>
>

Re: random minor benchmark: Re: Copy 20 tarfiles: ext2 vs (reiser4, unixfile) vs (reiser4,cryptcompress)

[Jens Axboe]

On Wed, Jan 25 2006, Hans Reiser wrote:
> Notice how CPU speed (and number of cpus) completely determines
> compression performance.
> 
> cryptcompress refers to the reiser4 compression plugin, (unix file)
> refers to the reiser4 non-compressing plugin.
> 
> Edward Shishkin wrote:
> 
> > Here are the tests that vs asked for:
> > Creation (dd) of 20 tarfiles (the original 200M file is in ramfs)
> > Kernel: 2.6.15-mm4 + current git snapshot of reiser4
> >
> > ------------------------------------------
> >
> > Laputa workstation
> > Uni Intel Pentium 4 (2.26 GHz) 512M RAM
> >
> > ext2:
> > real 2m, 15s
> > sys 0m, 14s
> >
> > reiser4(unix file)
> > real 2m, 7s
> > sys  0m, 23s
> >
> > reiser4(cryptcompress, lzo1, 64K)
> > real 2m, 13s
> > sys 0m, 11s

Just curious - does your crypt plugin reside in user space?

-- 
Jens Axboe

Re: random minor benchmark: Re: Copy 20 tarfiles: ext2 vs (reiser4, unixfile) vs (reiser4,cryptcompress)

[Edward Shishkin]

Jens Axboe wrote:

>On Wed, Jan 25 2006, Hans Reiser wrote:
>  
>
>>Notice how CPU speed (and number of cpus) completely determines
>>compression performance.
>>
>>cryptcompress refers to the reiser4 compression plugin, (unix file)
>>refers to the reiser4 non-compressing plugin.
>>
>>Edward Shishkin wrote:
>>
>>    
>>
>>>Here are the tests that vs asked for:
>>>Creation (dd) of 20 tarfiles (the original 200M file is in ramfs)
>>>Kernel: 2.6.15-mm4 + current git snapshot of reiser4
>>>
>>>------------------------------------------
>>>
>>>Laputa workstation
>>>Uni Intel Pentium 4 (2.26 GHz) 512M RAM
>>>
>>>ext2:
>>>real 2m, 15s
>>>sys 0m, 14s
>>>
>>>reiser4(unix file)
>>>real 2m, 7s
>>>sys  0m, 23s
>>>
>>>reiser4(cryptcompress, lzo1, 64K)
>>>real 2m, 13s
>>>sys 0m, 11s
>>>      
>>>
>
>Just curious - does your crypt plugin reside in user space?
>
>  
>

Nop.
This is just wrappers for linux crypto api, zlib, etc..
so user time is zero and not interesting.

Edward.

Re: random minor benchmark: Re: Copy 20 tarfiles: ext2 vs (reiser4, unixfile) vs (reiser4,cryptcompress)

[Jens Axboe]

On Thu, Jan 26 2006, Edward Shishkin wrote:
> Jens Axboe wrote:
> 
> >On Wed, Jan 25 2006, Hans Reiser wrote:
> > 
> >
> >>Notice how CPU speed (and number of cpus) completely determines
> >>compression performance.
> >>
> >>cryptcompress refers to the reiser4 compression plugin, (unix file)
> >>refers to the reiser4 non-compressing plugin.
> >>
> >>Edward Shishkin wrote:
> >>
> >>   
> >>
> >>>Here are the tests that vs asked for:
> >>>Creation (dd) of 20 tarfiles (the original 200M file is in ramfs)
> >>>Kernel: 2.6.15-mm4 + current git snapshot of reiser4
> >>>
> >>>------------------------------------------
> >>>
> >>>Laputa workstation
> >>>Uni Intel Pentium 4 (2.26 GHz) 512M RAM
> >>>
> >>>ext2:
> >>>real 2m, 15s
> >>>sys 0m, 14s
> >>>
> >>>reiser4(unix file)
> >>>real 2m, 7s
> >>>sys  0m, 23s
> >>>
> >>>reiser4(cryptcompress, lzo1, 64K)
> >>>real 2m, 13s
> >>>sys 0m, 11s
> >>>     
> >>>
> >
> >Just curious - does your crypt plugin reside in user space?
> >
> > 
> >
> 
> Nop.
> This is just wrappers for linux crypto api, zlib, etc..
> so user time is zero and not interesting.

Then why is the sys time lower than the "plain" writes on ext2 and
reiser4? Surely compressing isn't for free, yet the sys time is lower on
the compression write than the others.

-- 
Jens Axboe

Re: random minor benchmark: Re: Copy 20 tarfiles: ext2 vs (reiser4, unixfile) vs (reiser4,cryptcompress)

[Edward Shishkin]

Jens Axboe wrote:

>On Thu, Jan 26 2006, Edward Shishkin wrote:
>  
>
>>Jens Axboe wrote:
>>
>>    
>>
>>>On Wed, Jan 25 2006, Hans Reiser wrote:
>>>
>>>
>>>      
>>>
>>>>Notice how CPU speed (and number of cpus) completely determines
>>>>compression performance.
>>>>
>>>>cryptcompress refers to the reiser4 compression plugin, (unix file)
>>>>refers to the reiser4 non-compressing plugin.
>>>>
>>>>Edward Shishkin wrote:
>>>>
>>>>  
>>>>
>>>>        
>>>>
>>>>>Here are the tests that vs asked for:
>>>>>Creation (dd) of 20 tarfiles (the original 200M file is in ramfs)
>>>>>Kernel: 2.6.15-mm4 + current git snapshot of reiser4
>>>>>
>>>>>------------------------------------------
>>>>>
>>>>>Laputa workstation
>>>>>Uni Intel Pentium 4 (2.26 GHz) 512M RAM
>>>>>
>>>>>ext2:
>>>>>real 2m, 15s
>>>>>sys 0m, 14s
>>>>>
>>>>>reiser4(unix file)
>>>>>real 2m, 7s
>>>>>sys  0m, 23s
>>>>>
>>>>>reiser4(cryptcompress, lzo1, 64K)
>>>>>real 2m, 13s
>>>>>sys 0m, 11s
>>>>>    
>>>>>
>>>>>          
>>>>>
>>>Just curious - does your crypt plugin reside in user space?
>>>
>>>
>>>
>>>      
>>>
>>Nop.
>>This is just wrappers for linux crypto api, zlib, etc..
>>so user time is zero and not interesting.
>>    
>>
>
>Then why is the sys time lower than the "plain" writes on ext2 and
>reiser4? Surely compressing isn't for free, yet the sys time is lower on
>the compression write than the others.
>
>  
>

I guess this is because real compression is going in background
flush, not in sys_write->write_cryptcompress (which just copies
user's data to page cache). So in this case we have something
very similar to ext2. Reiser4 plain write (write_unix_file) is
more complex, and currently we try to reduce its sys time.

Edward.

Re: random minor benchmark: Re: Copy 20 tarfiles: ext2 vs (reiser4, unixfile) vs (reiser4,cryptcompress)

[Hans Reiser]

Edward Shishkin wrote:

>
> I guess this is because real compression is going in background
> flush, not in sys_write->write_cryptcompress (which just copies
> user's data to page cache). So in this case we have something
> very similar to ext2. Reiser4 plain write (write_unix_file) is
> more complex, and currently we try to reduce its sys time.
>
> Edward.
>
>
>
>
Which means that only real time is a meaningful measurement.....

Re: random minor benchmark: Re: Copy 20 tarfiles: ext2 vs (reiser4, unixfile) vs (reiser4,cryptcompress)

[Jens Axboe]

On Thu, Jan 26 2006, Hans Reiser wrote:
> Edward Shishkin wrote:
> 
> >
> > I guess this is because real compression is going in background
> > flush, not in sys_write->write_cryptcompress (which just copies
> > user's data to page cache). So in this case we have something
> > very similar to ext2. Reiser4 plain write (write_unix_file) is
> > more complex, and currently we try to reduce its sys time.
> >
> > Edward.
> >
> >
> >
> >
> Which means that only real time is a meaningful measurement.....

Indeed. I guess the compression stuff cost is hard to quantify, since it
has cache effects on the rest of the system in addition to costing CPU
cycles on its own.

A profile of, say, dbench with and without compression would be
interesting to see. And the actual dbench reults, naturally :-)

-- 
Jens Axboe

Re: random minor benchmark: Re: Copy 20 tarfiles: ext2 vs (reiser4, unixfile) vs (reiser4,cryptcompress)

[Jens Axboe]

On Thu, Jan 26 2006, Edward Shishkin wrote:
> Jens Axboe wrote:
> 
> >On Thu, Jan 26 2006, Edward Shishkin wrote:
> > 
> >
> >>Jens Axboe wrote:
> >>
> >>   
> >>
> >>>On Wed, Jan 25 2006, Hans Reiser wrote:
> >>>
> >>>
> >>>     
> >>>
> >>>>Notice how CPU speed (and number of cpus) completely determines
> >>>>compression performance.
> >>>>
> >>>>cryptcompress refers to the reiser4 compression plugin, (unix file)
> >>>>refers to the reiser4 non-compressing plugin.
> >>>>
> >>>>Edward Shishkin wrote:
> >>>>
> >>>> 
> >>>>
> >>>>       
> >>>>
> >>>>>Here are the tests that vs asked for:
> >>>>>Creation (dd) of 20 tarfiles (the original 200M file is in ramfs)
> >>>>>Kernel: 2.6.15-mm4 + current git snapshot of reiser4
> >>>>>
> >>>>>------------------------------------------
> >>>>>
> >>>>>Laputa workstation
> >>>>>Uni Intel Pentium 4 (2.26 GHz) 512M RAM
> >>>>>
> >>>>>ext2:
> >>>>>real 2m, 15s
> >>>>>sys 0m, 14s
> >>>>>
> >>>>>reiser4(unix file)
> >>>>>real 2m, 7s
> >>>>>sys  0m, 23s
> >>>>>
> >>>>>reiser4(cryptcompress, lzo1, 64K)
> >>>>>real 2m, 13s
> >>>>>sys 0m, 11s
> >>>>>   
> >>>>>
> >>>>>         
> >>>>>
> >>>Just curious - does your crypt plugin reside in user space?
> >>>
> >>>
> >>>
> >>>     
> >>>
> >>Nop.
> >>This is just wrappers for linux crypto api, zlib, etc..
> >>so user time is zero and not interesting.
> >>   
> >>
> >
> >Then why is the sys time lower than the "plain" writes on ext2 and
> >reiser4? Surely compressing isn't for free, yet the sys time is lower on
> >the compression write than the others.
> >
> > 
> >
> 
> I guess this is because real compression is going in background
> flush, not in sys_write->write_cryptcompress (which just copies
> user's data to page cache). So in this case we have something
> very similar to ext2. Reiser4 plain write (write_unix_file) is
> more complex, and currently we try to reduce its sys time.

So the systime quoted above is basically useless, it doesn't reflect the
real time spent in the kernel by far. I think you should note that when
you post these scores, otherwise you're really showing a skewed picture.

-- 
Jens Axboe

Re: random minor benchmark: Re: Copy 20 tarfiles: ext2 vs (reiser4, unixfile) vs (reiser4,cryptcompress)

[Hans Reiser]

Jens Axboe wrote:

> So the systime quoted above is basically useless, it doesn't reflect the
>
>real time spent in the kernel by far. I think you should note that when
>you post these scores, otherwise you're really showing a skewed picture.
>
>  
>
He wasn't expecting me to post the benchmark, and I frankly only looked
at the real time and forgot there was a systime in there that needed
cutting out when I posted it. My error, not his. I must say though, the
real time makes me quite happy, especially considering how CPUs are just
going to keep getting faster.

Best,

Hans

Re: random minor benchmark: Re: Copy 20 tarfiles: ext2 vs (reiser4, unixfile) vs (reiser4,cryptcompress)

[Jens Axboe]

On Fri, Jan 27 2006, Hans Reiser wrote:
> Jens Axboe wrote:
> 
> > So the systime quoted above is basically useless, it doesn't reflect the
> >
> >real time spent in the kernel by far. I think you should note that when
> >you post these scores, otherwise you're really showing a skewed picture.
> >
> >  
> >
> He wasn't expecting me to post the benchmark, and I frankly only looked
> at the real time and forgot there was a systime in there that needed
> cutting out when I posted it. My error, not his. I must say though, the
> real time makes me quite happy, especially considering how CPUs are just
> going to keep getting faster.

Yeah and that's ok, I was just interested in seeing some more
interesting compression benchmarks so I wondered if you had done that.

-- 
Jens Axboe

Re: random minor benchmark: Re: Copy 20 tarfiles: ext2 vs (reiser4, unixfile) vs (reiser4,cryptcompress)

[Hans Reiser]

Jens Axboe wrote:

>
>Yeah and that's ok, I was just interested in seeing some more
>interesting compression benchmarks so I wondered if you had done that.
>
>  
>
I think "random minor benchmark" was an apt description, yes.;-)

First we will debug it fully. Then we will figure out how to change
mongo so that the files do not consist entirely of the letter a as their
contents, and run mongo on it. Probably we will find some way to slice
up a linux kernel tar file into files of random sizes, and assume that
is a fair thing to let it compress during mongo. Then, just so people
won't think mongo is slanted in our favor we will do some cp -r's of
large numbers of linux kernel source trees and time that also.

Hans

Re: random minor benchmark: Re: Copy 20 tarfiles: ext2 vs (reiser4, unixfile) vs (reiser4,cryptcompress)

[Alexander Zarochentsev]

Hello,

On Thursday 26 January 2006 21:56, Jens Axboe wrote:
> On Thu, Jan 26 2006, Edward Shishkin wrote:
> > Jens Axboe wrote:
> > >On Wed, Jan 25 2006, Hans Reiser wrote:
> > >>Notice how CPU speed (and number of cpus) completely determines
> > >>compression performance.
> > >>
> > >>cryptcompress refers to the reiser4 compression plugin, (unix file)
> > >>refers to the reiser4 non-compressing plugin.
> > >>
> > >>Edward Shishkin wrote:
> > >>>Here are the tests that vs asked for:
> > >>>Creation (dd) of 20 tarfiles (the original 200M file is in ramfs)
> > >>>Kernel: 2.6.15-mm4 + current git snapshot of reiser4
> > >>>
> > >>>------------------------------------------
> > >>>
> > >>>Laputa workstation
> > >>>Uni Intel Pentium 4 (2.26 GHz) 512M RAM
> > >>>
> > >>>ext2:
> > >>>real 2m, 15s
> > >>>sys 0m, 14s
> > >>>
> > >>>reiser4(unix file)
> > >>>real 2m, 7s
> > >>>sys  0m, 23s
> > >>>
> > >>>reiser4(cryptcompress, lzo1, 64K)
> > >>>real 2m, 13s
> > >>>sys 0m, 11s
> > >
> > >Just curious - does your crypt plugin reside in user space?
> >
> > Nop.
> > This is just wrappers for linux crypto api, zlib, etc..
> > so user time is zero and not interesting.
>
> Then why is the sys time lower than the "plain" writes on ext2 and
> reiser4? Surely compressing isn't for free, yet the sys time is lower on
> the compression write than the others.

I guess the compression was done by the background writeout daemon.  CPU 
utilization numbers would say more than sys time.

-- 
Alex.

Re: random minor benchmark: Re: Copy 20 tarfiles: ext2 vs (reiser4, unixfile) vs (reiser4,cryptcompress)

[Hans Reiser]

Jens Axboe wrote:

>On Wed, Jan 25 2006, Hans Reiser wrote:
>  
>
>>Notice how CPU speed (and number of cpus) completely determines
>>compression performance.
>>
>>cryptcompress refers to the reiser4 compression plugin, (unix file)
>>refers to the reiser4 non-compressing plugin.
>>
>>Edward Shishkin wrote:
>>
>>    
>>
>>>Here are the tests that vs asked for:
>>>Creation (dd) of 20 tarfiles (the original 200M file is in ramfs)
>>>Kernel: 2.6.15-mm4 + current git snapshot of reiser4
>>>
>>>------------------------------------------
>>>
>>>Laputa workstation
>>>Uni Intel Pentium 4 (2.26 GHz) 512M RAM
>>>
>>>ext2:
>>>real 2m, 15s
>>>sys 0m, 14s
>>>
>>>reiser4(unix file)
>>>real 2m, 7s
>>>sys  0m, 23s
>>>
>>>reiser4(cryptcompress, lzo1, 64K)
>>>real 2m, 13s
>>>sys 0m, 11s
>>>      
>>>
>
>Just curious - does your crypt plugin reside in user space?
>
>  
>
No, kernel.  It would have to  encrypt+compress with every write to be
in user space, we encrypt+compress only at flush time, and that is a key
optimization (encryption is disabled at the moment due to needing a
little API work, but....) for file sets that are cachable.