How to make BTRFS crawl

How to make BTRFS crawl

[Swâmi Petaramesh]

Hi,

I was asked about situations "use cases" that would cause BTRFS to slow down 
to a crawl.

And it's exactly what happened to me yesterday when I was trying, on the 
contrary, to speed it up.

So here's the recipe for getting a "slow to the point it is unusable" BTRFS.


1/ Perform a clean, fresh install of a recent distro with a 3.13 kernel (i.e. 
Fedora 20) and a BTRFS root filesystem.

2/ Choose the version with a KDE interface

3/ Configure fstab mountpoints using such options (space_cache will have been 
manually activated once):

/ btrfs   subvol=FEDORA,noatime,compress=lzo,autodefrag

/home btrfs   subvol=HOME,noatime,compress=lzo,autodefrag


4/ Use "chattr +C" to make the following directories NOCOW (move the old 
directory elsewhere, create a new dir, make it nocow, copy files from the old 
one so they are recreated with nocow, check permissions...):

- /home/yourself/.cache
- /home/yourself/.local/share/akonadi

5/ Use IMAP mail in Kmail. Seriously process your email (it will be stored 
using akonadi mysql)

6/ Surf normally the web using Firefox

7/ Install SuSE "snapper" package that will perform a FS snapshot every hour. 
Configure it so it will snapshot both the root FS subvol and the /home subvol

8/ Use the system for 24 hours and you will know that "hardly usable" means... 
Especially every hour-on-the-hour when Kmail or Firefox will try to access 
files that have been recently snapshotted... Your system will be dead with 
saturated HD access for several *minutes*

...Hope this may help hunting this down...

Kind regards.

-- 
Swâmi Petaramesh <swami@petaramesh.org> http://petaramesh.org PGP 9076E32E

Re: How to make BTRFS crawl

[George Mitchell]

Well, Akonadi brought my system to its knees long before I converted to 
btrfs, so somehow I am not surprised.  I have kept akonadi disabled ever 
since.for everything except a portion of Thunderbird and that ONLY with 
sql-lite. Mysql will kill it in no time.  So I am not sure that btrfs is 
the root of the problem here.  Just my two cents, perhaps others have 
different experience with akonadi.


On 04/11/2014 02:42 AM, Swâmi Petaramesh wrote:
> Hi,
>
> I was asked about situations "use cases" that would cause BTRFS to slow down
> to a crawl.
>
> And it's exactly what happened to me yesterday when I was trying, on the
> contrary, to speed it up.
>
> So here's the recipe for getting a "slow to the point it is unusable" BTRFS.
>
>
> 1/ Perform a clean, fresh install of a recent distro with a 3.13 kernel (i.e.
> Fedora 20) and a BTRFS root filesystem.
>
> 2/ Choose the version with a KDE interface
>
> 3/ Configure fstab mountpoints using such options (space_cache will have been
> manually activated once):
>
> / btrfs   subvol=FEDORA,noatime,compress=lzo,autodefrag
>
> /home btrfs   subvol=HOME,noatime,compress=lzo,autodefrag
>
>
> 4/ Use "chattr +C" to make the following directories NOCOW (move the old
> directory elsewhere, create a new dir, make it nocow, copy files from the old
> one so they are recreated with nocow, check permissions...):
>
> - /home/yourself/.cache
> - /home/yourself/.local/share/akonadi
>
> 5/ Use IMAP mail in Kmail. Seriously process your email (it will be stored
> using akonadi mysql)
>
> 6/ Surf normally the web using Firefox
>
> 7/ Install SuSE "snapper" package that will perform a FS snapshot every hour.
> Configure it so it will snapshot both the root FS subvol and the /home subvol
>
> 8/ Use the system for 24 hours and you will know that "hardly usable" means...
> Especially every hour-on-the-hour when Kmail or Firefox will try to access
> files that have been recently snapshotted... Your system will be dead with
> saturated HD access for several *minutes*
>
> ...Hope this may help hunting this down...
>
> Kind regards.
>

Re: How to make BTRFS crawl

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 3426 bytes --]

On Fri, Apr 11, 2014 at 07:11:52AM -0700, George Mitchell wrote:
> Well, Akonadi brought my system to its knees long before I converted to
> btrfs, so somehow I am not surprised.  I have kept akonadi disabled ever
> since.for everything except a portion of Thunderbird and that ONLY with
> sql-lite. Mysql will kill it in no time.  So I am not sure that btrfs is the
> root of the problem here.  Just my two cents, perhaps others have different
> experience with akonadi.

   My opinion (and it is purely opinion, since I haven't used any part
of KDE since the last millennium) is that akonadi probably isn't
massively efficient, *and* that it happens to hit a particular write
pattern that btrfs isn't handling too well. So I don't think it's fair
to point the blame solely at one or the other, but at the interaction
between bad (or awkward) behaviours of the two together.

   I'm surprised that it's showing very poor performance with the SSD,
though -- I'd have thought most of the performance loss would be in
additional seeks from the very fragmented file. Although with lots of
snapshots (e.g. snapper) going on, the benefits of reduced
fragmentation from the nodatacow are largely going to be lost because
each snapshot forces another round of CoWing and fragmentation.

   Hugo.

> On 04/11/2014 02:42 AM, Swâmi Petaramesh wrote:
> >Hi,
> >
> >I was asked about situations "use cases" that would cause BTRFS to slow down
> >to a crawl.
> >
> >And it's exactly what happened to me yesterday when I was trying, on the
> >contrary, to speed it up.
> >
> >So here's the recipe for getting a "slow to the point it is unusable" BTRFS.
> >
> >
> >1/ Perform a clean, fresh install of a recent distro with a 3.13 kernel (i.e.
> >Fedora 20) and a BTRFS root filesystem.
> >
> >2/ Choose the version with a KDE interface
> >
> >3/ Configure fstab mountpoints using such options (space_cache will have been
> >manually activated once):
> >
> >/ btrfs   subvol=FEDORA,noatime,compress=lzo,autodefrag
> >
> >/home btrfs   subvol=HOME,noatime,compress=lzo,autodefrag
> >
> >
> >4/ Use "chattr +C" to make the following directories NOCOW (move the old
> >directory elsewhere, create a new dir, make it nocow, copy files from the old
> >one so they are recreated with nocow, check permissions...):
> >
> >- /home/yourself/.cache
> >- /home/yourself/.local/share/akonadi
> >
> >5/ Use IMAP mail in Kmail. Seriously process your email (it will be stored
> >using akonadi mysql)
> >
> >6/ Surf normally the web using Firefox
> >
> >7/ Install SuSE "snapper" package that will perform a FS snapshot every hour.
> >Configure it so it will snapshot both the root FS subvol and the /home subvol
> >
> >8/ Use the system for 24 hours and you will know that "hardly usable" means...
> >Especially every hour-on-the-hour when Kmail or Firefox will try to access
> >files that have been recently snapshotted... Your system will be dead with
> >saturated HD access for several *minutes*
> >
> >...Hope this may help hunting this down...
> >
> >Kind regards.
> >
> 

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
     --- Geek, n.: Circus sideshow performer specialising in the ---     
                         eating of live animals.                         

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]