BTRFS Benchmarking

BTRFS Benchmarking

[Olivier Doucet]

hello everyone,

I made an overall benchmark of BTRFS against EXT4 and XFS. I'm quite
unhappy with BTRFS results, so maybe tuning was not perfect.

http://www.slideshare.net/ezameku/btrfs-benchmark

All data is vectorial, so download the PDF and you can zoom ;)

If you have any feedback on how to improve BTRFS results (and others
fs too !), I would be glad to update my data.

Test protocol
Server : dual CPU Intel L5640 with HT enabled
Operating system : CentOS 6.2 (64bits version) with custom tools/kernels
Kernel : 3.3.0
Btrfs progs: version 0.19
Drive : Seagate 3TB drive (ST33000652SS) SAS attached via an LSI HBA.
Drive was accessed through LVM ;

MKFS options
BTRFS    : none
XFS         : none
EXT4       : none

Mount options
BTRFS          : "noatime,nodiratime"
BTRFS compress : "noatime,nodiratime,compress=lzo"
EXT4           : "noatime,nodiratime"
XFS            : "noatime,nodiratime"

Benchmark is done with Sysbench (fileio test).
Each benchmark was done for 60 seconds, and generated one point on the
graph each second (to see variations).
Right scale is block size.

Data read / written is from /dev/urandom, so cannot be compressed much
(that was expected behaviour).

All second pages has no legend, I'm sorry for that :
- data is 95 percentile aggregate.
- colours are the same.


Overview of results

On sequential read, there is no variations between FS.
On sequential write, BTRFS has lower values than EXT4/XFS. On random
write also.


Olivier

Re: BTRFS Benchmarking

[Josef Bacik]

On Fri, May 04, 2012 at 06:03:50PM +0200, Olivier Doucet wrote:
> hello everyone,
> 
> I made an overall benchmark of BTRFS against EXT4 and XFS. I'm quite
> unhappy with BTRFS results, so maybe tuning was not perfect.
> 
> http://www.slideshare.net/ezameku/btrfs-benchmark
> 
> All data is vectorial, so download the PDF and you can zoom ;)
> 
> If you have any feedback on how to improve BTRFS results (and others
> fs too !), I would be glad to update my data.
> 
> Test protocol
> Server : dual CPU Intel L5640 with HT enabled
> Operating system : CentOS 6.2 (64bits version) with custom tools/kernels
> Kernel : 3.3.0
> Btrfs progs: version 0.19
> Drive : Seagate 3TB drive (ST33000652SS) SAS attached via an LSI HBA.
> Drive was accessed through LVM ;
> 
> MKFS options
> BTRFS    : none
> XFS         : none
> EXT4       : none
> 
> Mount options
> BTRFS          : "noatime,nodiratime"
> BTRFS compress : "noatime,nodiratime,compress=lzo"
> EXT4           : "noatime,nodiratime"
> XFS            : "noatime,nodiratime"
> 
> Benchmark is done with Sysbench (fileio test).
> Each benchmark was done for 60 seconds, and generated one point on the
> graph each second (to see variations).
> Right scale is block size.
> 
> Data read / written is from /dev/urandom, so cannot be compressed much
> (that was expected behaviour).
> 
> All second pages has no legend, I'm sorry for that :
> - data is 95 percentile aggregate.
> - colours are the same.
> 
> 
> Overview of results
> 
> On sequential read, there is no variations between FS.
> On sequential write, BTRFS has lower values than EXT4/XFS. On random
> write also.
> 

Not what I've been seeing at all, but we've been working a lot in this area
recently.  Please retest with btrfs-next.  Thanks,

Josef

Re: BTRFS Benchmarking

[cwillu]

On Fri, May 4, 2012 at 10:07 AM, Josef Bacik <josef@redhat.com> wrote:
> On Fri, May 04, 2012 at 06:03:50PM +0200, Olivier Doucet wrote:
>> hello everyone,
>>
>> I made an overall benchmark of BTRFS against EXT4 and XFS. I'm quite
>> unhappy with BTRFS results, so maybe tuning was not perfect.
>>
>> http://www.slideshare.net/ezameku/btrfs-benchmark
>>
>> All data is vectorial, so download the PDF and you can zoom ;)

In the future, can you post this somewhere that doesn't require a
login?  (I'm happy to host the occasional pdf, for instance).

Re: BTRFS Benchmarking

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 2578 bytes --]

On Fri, May 04, 2012 at 06:03:50PM +0200, Olivier Doucet wrote:
> hello everyone,
> 
> I made an overall benchmark of BTRFS against EXT4 and XFS. I'm quite
> unhappy with BTRFS results, so maybe tuning was not perfect.
> 
> http://www.slideshare.net/ezameku/btrfs-benchmark
> 
> All data is vectorial, so download the PDF and you can zoom ;)
> 
> If you have any feedback on how to improve BTRFS results (and others
> fs too !), I would be glad to update my data.
> 
> Test protocol
> Server : dual CPU Intel L5640 with HT enabled
> Operating system : CentOS 6.2 (64bits version) with custom tools/kernels
> Kernel : 3.3.0
> Btrfs progs: version 0.19
> Drive : Seagate 3TB drive (ST33000652SS) SAS attached via an LSI HBA.
> Drive was accessed through LVM ;
> 
> MKFS options
> BTRFS    : none
> XFS         : none
> EXT4       : none
> 
> Mount options
> BTRFS          : "noatime,nodiratime"
> BTRFS compress : "noatime,nodiratime,compress=lzo"
> EXT4           : "noatime,nodiratime"
> XFS            : "noatime,nodiratime"
> 
> Benchmark is done with Sysbench (fileio test).
> Each benchmark was done for 60 seconds, and generated one point on the
> graph each second (to see variations).
> Right scale is block size.
> 
> Data read / written is from /dev/urandom, so cannot be compressed much
> (that was expected behaviour).
> 
> All second pages has no legend, I'm sorry for that :
> - data is 95 percentile aggregate.
> - colours are the same.

   Can you tell us what the unit of the Y-axis is? Is it MB/s or IOPs
or time for a fixed amount data or... ?

   We've just been discussing this on IRC, and we've come up with two
completely different interpretations of the data, so we're actually
not sure how to interpret this.

   For example, in the seqwr/512 test, is btrfs doing really well at
small numbers of threads, getting steadily worse, or is it doing
really badly, and improves hugely as the number of threads goes up?

   Hugo.

> Overview of results
> 
> On sequential read, there is no variations between FS.
> On sequential write, BTRFS has lower values than EXT4/XFS. On random
> write also.

   I guess what we're after is -- is a lower value better or worse?

   Thanks,
   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
    --- SCSI is usually fixed by remembering that it needs three ---     
        terminations: One at each end of the chain. And the goat.        
                                                                         

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 190 bytes --]

Re: BTRFS Benchmarking

[Olivier Doucet]

Hi,

I uploaded the PDF on Dropbox that does not require login
https://www.dropbox.com/s/5i8l0kmdutxj6pb/sysbench-sas3t-btrfs1.pdf


> =A0 Can you tell us what the unit of the Y-axis is? Is it MB/s or IOP=
s
> or time for a fixed amount data or... ?
Unit is MB/s ;
=46or each test, I gather speed every second.
So for a particular test, there is 60 dots (for example, speed for
BTRFS on 1 thread, blocksize=3D512b, sequential read).
As 60 dots are not easy to read, I created second pages with one
aggregated value as 95 percentile.

Sequential read results seems heavily altered by Linux pagecache. I
wondered what would be the exact methodology to disable this
optimization.
But all FS were run in the exact same behaviour (bash scripted with
umount / mount after each session).


I provided access to the bash script I used and R script to create nice=
 PDF.
=46eel free to give me any feedback on how I could get results more acc=
urate.

https://github.com/odoucet/fs-benchs


> =A0 For example, in the seqwr/512 test, is btrfs doing really well at
> small numbers of threads, getting steadily worse, or is it doing
> really badly, and improves hugely as the number of threads goes up?

It is doing badly but improves when number of threads goes up.


I'll improve my benchmark results by adding self-explainable legends.
I will be happy to discuss my methodology if you think something is
wrong.


Olivier
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: BTRFS Benchmarking

[Olivier Doucet]

> Not what I've been seeing at all, but we've been working a lot in thi=
s area
> recently. =A0Please retest with btrfs-next. =A0Thanks,

Hi,

I tested again with kernel 3.3.4 ; I wondered if latest btrfs code is
present in this release or not.

Results are very similar with 3.3.4 compared to 3.3.0

Olivier
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: BTRFS Benchmarking

[Josef Bacik]

On Mon, May 07, 2012 at 08:42:45PM +0200, Olivier Doucet wrote:
> > Not what I've been seeing at all, but we've been working a lot in t=
his area
> > recently. =A0Please retest with btrfs-next. =A0Thanks,
>=20
> Hi,
>=20
> I tested again with kernel 3.3.4 ; I wondered if latest btrfs code is
> present in this release or not.
>=20
> Results are very similar with 3.3.4 compared to 3.3.0
>=20

It's not, you need to clone the btrfs-next tree and test with that.

Josef
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html