All of lore.kernel.org
 help / color / mirror / Atom feed
From: DanglingPointer <danglingpointerexception@gmail.com>
To: linux-btrfs@vger.kernel.org
Cc: danglingpointerexception@gmail.com
Subject: migrating to space_cache=2 and btrfs userspace commands
Date: Wed, 14 Jul 2021 01:38:55 +1000	[thread overview]
Message-ID: <63396688-0dc7-17c5-a830-5893b030a30f@gmail.com> (raw)

We're currently considering switching to "space_cache=v2" with noatime 
mount options for my lab server-workstations running RAID5.

  * One has 13TB of data/metadata in a bunch of 6TB and 2TB disks
    totalling 26TB.
  * Another has about 12TB data/metadata in uniformly sized 6TB disks
    totalling 24TB.
  * Both of the arrays are on individually luks encrypted disks with
    btrfs on top of the luks.
  * Both have "defaults,autodefrag" turned on in fstab.

We're starting to see large pauses during constant backups of millions 
of chunk files (using duplicacy backup) in the 24TB array.

Pauses sometimes take up to 20+ seconds in frequencies after every 
~30secs of the end of the last pause.  "btrfs-transacti" process 
consistently shows up as the blocking process/thread locking up 
filesystem IO.  IO gets into the RAID5 array via nfsd. There are no disk 
or btrfs errors recorded.  scrub last finished yesterday successfully.

After doing some research around the internet, we've come to the 
consideration above as described.  Unfortunately the official 
documentation isn't clear on the following.

Official documentation URL - 
https://btrfs.wiki.kernel.org/index.php/Manpage/btrfs(5)

 1. How to migrate from default space_cache=v1 to space_cache=v2? It
    talks about the reverse, from v2 to v1!
 2. If we use space_cache=v2, is it indeed still the case that the
    "btrfs" command will NOT work with the filesystem?  So will our
    "btrfs scrub start /mount/point/..." cron jobs FAIL?  I'm guessing
    the btrfs command comes from btrfs-progs which is currently v5.4.1-2
    amd64, is that correct?
 3. Any other ideas on how we can get rid of those annoying pauses with
    large backups into the array?

Thanks in advance!

DP


             reply	other threads:[~2021-07-13 15:39 UTC|newest]

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-07-13 15:38 DanglingPointer [this message]
2021-07-14  4:59 ` migrating to space_cache=2 and btrfs userspace commands Qu Wenruo
2021-07-14  5:44   ` Chris Murphy
2021-07-14  6:05     ` Qu Wenruo
2021-07-14  6:54       ` DanglingPointer
2021-07-14  7:07         ` Qu Wenruo
2021-07-14  7:18   ` DanglingPointer
2021-07-14  7:45     ` Qu Wenruo
2021-07-15 16:40       ` DanglingPointer
2021-07-15 22:13         ` Qu Wenruo
2021-07-15 17:51       ` Joshua
2021-07-16 12:42         ` DanglingPointer
2021-07-16 12:59           ` Qu Wenruo
2021-07-16 13:23             ` DanglingPointer
2021-07-16 20:33             ` Joshua Villwock
2021-07-16 23:00               ` Qu Wenruo

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=63396688-0dc7-17c5-a830-5893b030a30f@gmail.com \
    --to=danglingpointerexception@gmail.com \
    --cc=linux-btrfs@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.