All of lore.kernel.org
 help / color / mirror / Atom feed
From: Christian Theune <ct@flyingcircus.io>
To: "Austin S. Hemmelgarn" <ahferroin7@gmail.com>
Cc: Hugo Mills <hugo@carfax.org.uk>, linux-btrfs@vger.kernel.org
Subject: Re: Shrinking a device - performance?
Date: Mon, 27 Mar 2017 16:49:47 +0200	[thread overview]
Message-ID: <CDED0C65-B382-4EFF-90C6-140F5C8DFC0E@flyingcircus.io> (raw)
In-Reply-To: <bf772a48-5181-d1ea-f7a7-5dffc524e2de@gmail.com>

[-- Attachment #1: Type: text/plain, Size: 1948 bytes --]

Hi,

> On Mar 27, 2017, at 4:17 PM, Austin S. Hemmelgarn <ahferroin7@gmail.com> wrote:
> 
> One other thing that I just thought of:
> For a backup system, assuming some reasonable thinning system is used for the backups, I would personally migrate things slowly over time by putting new backups on the new filesystem, and shrinking the old filesystem as the old backups there get cleaned out.  Unfortunately, most backup software I've seen doesn't handle this well, so it's not all that easy to do, but it does save you from having to migrate data off of the old filesystem, and means you don't have to worry as much about the resize of the old FS taking forever.

Right. This is an option we can do from a software perspective (our own solution - https://bitbucket.org/flyingcircus/backy) but our systems in use can’t hold all the data twice. Even though we’re migrating to a backend implementation that uses less data than before I have to perform an “inplace” migration in some way. This is VM block device backup. So basically we migrate one VM with all its previous data and that works quite fine with a little headroom. However, migrating all VMs to a new “full” backup and then wait for the old to shrink would only work if we had a completely empty backup server in place, which we don’t.

Also: the idea of migrating on btrfs also has its downside - the performance of “mkdir” and “fsync” is abysmal at the moment. I’m waiting for the current shrinking job to finish but this is likely limited to the “find free space” algorithm. We’re talking about a few megabytes converted per second. Sigh.

Cheers,
Christian Theune

--
Christian Theune · ct@flyingcircus.io · +49 345 219401 0
Flying Circus Internet Operations GmbH · http://flyingcircus.io
Forsterstraße 29 · 06112 Halle (Saale) · Deutschland
HR Stendal HRB 21169 · Geschäftsführer: Christian. Theune, Christian. Zagrodnick


[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 496 bytes --]

  reply	other threads:[~2017-03-27 14:50 UTC|newest]

Thread overview: 42+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-03-27 11:17 Shrinking a device - performance? Christian Theune
2017-03-27 13:07 ` Hugo Mills
2017-03-27 13:20   ` Christian Theune
2017-03-27 13:24     ` Hugo Mills
2017-03-27 13:46       ` Austin S. Hemmelgarn
2017-03-27 13:50         ` Christian Theune
2017-03-27 13:54           ` Christian Theune
2017-03-27 14:17             ` Austin S. Hemmelgarn
2017-03-27 14:49               ` Christian Theune [this message]
2017-03-27 15:06                 ` Roman Mamedov
2017-04-01  9:05                   ` Kai Krakow
2017-03-27 14:14           ` Austin S. Hemmelgarn
2017-03-27 14:48     ` Roman Mamedov
2017-03-27 14:53       ` Christian Theune
2017-03-28 14:43         ` Peter Grandi
2017-03-28 14:50           ` Tomasz Kusmierz
2017-03-28 15:06             ` Peter Grandi
2017-03-28 15:35               ` Tomasz Kusmierz
2017-03-28 16:20                 ` Peter Grandi
2017-03-28 14:59           ` Peter Grandi
2017-03-28 15:20             ` Peter Grandi
2017-03-28 15:56           ` Austin S. Hemmelgarn
2017-03-30 15:55             ` Peter Grandi
2017-03-31 12:41               ` Austin S. Hemmelgarn
2017-03-31 17:25                 ` Peter Grandi
2017-03-31 19:38                   ` GWB
2017-03-31 20:27                     ` Peter Grandi
2017-04-01  0:02                       ` GWB
2017-04-01  2:42                         ` Duncan
2017-04-01  4:26                           ` GWB
2017-04-01 11:30                             ` Peter Grandi
2017-03-30 15:00           ` Piotr Pawłow
2017-03-30 16:13             ` Peter Grandi
2017-03-30 22:13               ` Piotr Pawłow
2017-03-31  1:00                 ` GWB
2017-03-31  5:26                   ` Duncan
2017-03-31  5:38                     ` Duncan
2017-03-31 12:37                       ` Peter Grandi
2017-03-31 11:37                   ` Peter Grandi
2017-03-31 10:51                 ` Peter Grandi
2017-03-27 11:51 Christian Theune
2017-03-27 12:55 ` Christian Theune

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CDED0C65-B382-4EFF-90C6-140F5C8DFC0E@flyingcircus.io \
    --to=ct@flyingcircus.io \
    --cc=ahferroin7@gmail.com \
    --cc=hugo@carfax.org.uk \
    --cc=linux-btrfs@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.