From: "Austin S. Hemmelgarn" <ahferroin7@gmail.com>
To: ein <ein.net@gmail.com>, linux-btrfs@vger.kernel.org
Subject: Re: number of subvolumes
Date: Fri, 1 Sep 2017 07:47:05 -0400 [thread overview]
Message-ID: <f0708aac-ac36-20f5-7ea7-d6f515813aed@gmail.com> (raw)
In-Reply-To: <59A9349E.1000302@gmail.com>
On 2017-09-01 06:21, ein wrote:
> Very comprehensive, thank you. I was asking because I'd like to learn
> how really random writes by VM affects BTRFS (vs XFS,Ext4) performance
> and try to develop some workaround to reduce/prevent it while having
> csums, cow (snapshots) and compression.
I've actually done some significant testing of VM's using BTRFS as
backing storage, and would suggest the following:
1. Use raw images in sparse files, and expose them to the VM like SSD's
(so that DISCARD support works). Assuming your guests actually support
DISCARD commands, this will save you just as much space as using
allocate-on-demand formats like qcow2 or VDI, while providing access
patterns that BTRFS is a bit better at dealing with. There are a couple
of specific caveats to doing this however, namely that you can't use the
regular VirtIO block device when using QEMU, and have to use VirtIO SCSI
instead, and you'll need smarter tools to copy images around (ddrescue
is what I would personally suggest, it can handle sparse files
correctly, and gives you a nice progress indicator).
2. Provide separate disks to the VM for any data you need to store in
them. By keeping this separate from the root filesystem's disk, you
reduce what gets fragmented, and also aren't tied to a particular guest
OS to the same degree. As a more concrete example, if you're running a
database server in a VM, give it a separate disk image for storing the
database from the one the root filesystem is on.
3. Run with autodefrag on the host system. Unless you're hitting the
disks hard from inside the VM, this will actually do a reasonably good
job of handling fragmentation.
4. Make sure the filesystem you're storing the disk images on has extra
space. Ideally, I would set things up so that it has enough room to
store a second copy of the largest disk image you expect to use. By
keeping lots of extra space, you give the allocator in BTRFS more
opportunity to arrange things intelligently.
5. Defragmet your disk images regularly (even when using autodefrag),
both inside and outside the VM's. Ideally, run a full defrag inside the
VM, then shut it down and defragment the disk image outside the VM.
Defragmenting inside the VM first will compact free space, which in turn
means that defragmenting outside the VM will do a better job, and that
future fragmentation will be less likely. In general, I'd suggest doing
this at least monthly (possibly a lot more frequently if you're running
databases or similar random access workloads in the VM).
6. Avoid using a CoW filesystem inside the VM's. This sounds odd at
first, but is actually one of the biggest things you can do to reduce
fragmentation. In particular, try to avoid using BTRFS or ZFS on a VM
disk that is stored on BTRFS, both of them will make usual fragmentation
problems look nonexistent in comparison.
next prev parent reply other threads:[~2017-09-01 11:47 UTC|newest]
Thread overview: 44+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-08-22 13:22 netapp-alike snapshots? Ulli Horlacher
2017-08-22 13:44 ` Peter Becker
2017-08-22 14:24 ` Ulli Horlacher
2017-08-22 16:08 ` Peter Becker
2017-08-22 16:48 ` Ulli Horlacher
2017-08-22 16:45 ` Roman Mamedov
2017-08-22 16:57 ` Ulli Horlacher
2017-08-22 17:19 ` A L
2017-08-22 18:01 ` Ulli Horlacher
2017-08-22 18:36 ` Peter Grandi
2017-08-22 20:48 ` Ulli Horlacher
2017-08-23 7:18 ` number of subvolumes Ulli Horlacher
2017-08-23 8:37 ` A L
2017-08-23 16:48 ` Ferry Toth
2017-08-24 17:45 ` Peter Grandi
2017-08-31 6:49 ` Ulli Horlacher
2017-08-31 11:18 ` Austin S. Hemmelgarn
2017-08-31 14:38 ` Michał Sokołowski
2017-08-31 16:18 ` Duncan
2017-09-01 10:21 ` ein
2017-09-01 11:47 ` Austin S. Hemmelgarn [this message]
2017-08-24 19:40 ` Marat Khalili
2017-08-24 21:56 ` Ferry Toth
2017-08-25 5:54 ` Chris Murphy
2017-08-25 11:45 ` Austin S. Hemmelgarn
2017-08-25 12:55 ` Ferry Toth
2017-08-25 19:18 ` Austin S. Hemmelgarn
2017-08-23 12:11 ` Peter Grandi
2017-08-22 21:53 ` user snapshots Ulli Horlacher
2017-08-23 6:28 ` Dmitrii Tcvetkov
2017-08-23 7:16 ` Dmitrii Tcvetkov
2017-08-23 7:20 ` Ulli Horlacher
2017-08-23 11:42 ` Peter Grandi
2017-08-23 21:13 ` Ulli Horlacher
2017-08-25 11:28 ` Austin S. Hemmelgarn
2017-08-22 17:36 ` netapp-alike snapshots? Roman Mamedov
2017-08-22 18:10 ` Ulli Horlacher
2017-09-09 13:26 ` Ulli Horlacher
2017-09-09 13:36 ` Marc MERLIN
2017-09-09 13:44 ` Ulli Horlacher
2017-09-09 19:43 ` Andrei Borzenkov
2017-09-09 19:52 ` Ulli Horlacher
2017-09-10 7:10 ` A L
2017-09-10 14:54 ` Marc MERLIN
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=f0708aac-ac36-20f5-7ea7-d6f515813aed@gmail.com \
--to=ahferroin7@gmail.com \
--cc=ein.net@gmail.com \
--cc=linux-btrfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.