Linux-BTRFS Archive on lore.kernel.org
 help / color / Atom feed
From: Filipe Manana <fdmanana@kernel.org>
To: dsterba@suse.cz, Hugo Mills <hugo@carfax.org.uk>,
	Filipe Manana <fdmanana@kernel.org>,
	linux-btrfs <linux-btrfs@vger.kernel.org>
Subject: Re: [PATCH 3/4] Btrfs: check if destination root is read-only for deduplication
Date: Mon, 18 Feb 2019 16:55:10 +0000
Message-ID: <CAL3q7H53D=7nawb2wugg0d4S4KH2Sx=DuZVwUcsfrzrv-+PN6Q@mail.gmail.com> (raw)
In-Reply-To: <20190218153814.GL9874@twin.jikos.cz>

On Mon, Feb 18, 2019 at 3:36 PM David Sterba <dsterba@suse.cz> wrote:
>
> On Thu, Jan 31, 2019 at 04:44:39PM +0000, Hugo Mills wrote:
> > On Thu, Jan 31, 2019 at 04:39:22PM +0000, Filipe Manana wrote:
> > > On Thu, Dec 13, 2018 at 4:08 PM David Sterba <dsterba@suse.cz> wrote:
> > > >
> > > > On Wed, Dec 12, 2018 at 06:05:58PM +0000, fdmanana@kernel.org wrote:
> > > > > From: Filipe Manana <fdmanana@suse.com>
> > > > >
> > > > > Checking if the destination root is read-only was being performed only for
> > > > > clone operations. Make deduplication check it as well, as it does not make
> > > > > sense to not do it, even if it is an operation that does not change the
> > > > > file contents (such as defrag for example, which checks first if the root
> > > > > is read-only).
> > > >
> > > > And this is also change in user-visible behaviour of dedupe, so this
> > > > needs to be verified if it's not breaking existing tools.
> > >
> > > Have you had the chance to do such verification?
> > >
> > > This actually conflicts with send. Send does not expect a root/tree to
> > > change, and with dedupe on read-only roots happening
> > > in parallel with send is going to cause all sorts of unexpected and
> > > undesired problems...
> > >
> > > This is a problem introduced by dedupe ioctl when it landed, since
> > > send existed for a longer time (when nothing else was
> > > allowed to change read-only roots, including defrag).
> > >
> > > I understand it can break some applications, but adding other solution
> > > such as preventing send and dedupe from running in parallel
> > > (erroring out or block and wait for each other, etc) is going to be
> > > really ugly. There's always the workaround for apps to set the
> > > subvolume
> > > to RW mode, do the dedupe, then switch it back to RO mode.
> >
> >    Only if you want your incremental send chain to break on the way
> > past...
> >
> >    I think it's fairly clear by now (particularly from the last thread
> > on this a couple of weeks ago) that making RO subvols RW and then back
> > again is a fast way to broken incremental receives.
>
> So, I think the way it should be fixed is to keep deduplication off the
> read-only subvolumes. The main reason I see is to avoid the random
> problems that arise from send + dedupe interaction. The cost is lower
> deduplication ratio.
>
> The main usecase being a primary subvolume with RO snapshots over time,
> with optional incremental send. That I know is common and widely used.
>
> A problematic usecase that would utilize deduplication over RO snapshots
> does could be something like a set of subvolumes that have very similar
> data, get snapshotted or set RO, followed by a dedupe pass.
>
> To support the latter I'd go only via the RO->RW, dedupe, RW->RO way, as
> this records that there was a change and does not collide with send
> assumptions. That this leads to loss of incremental send must be
> communicated to the user with possible override, eg. --I-know-what-I-am-doing
> option.

I fully agree.
Thanks.

  reply index

Thread overview: 26+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-12-12 18:05 [PATCH 0/4] Btrfs: a few more cleanups and fixes for clone/deduplication fdmanana
2018-12-12 18:05 ` [PATCH 1/4] Btrfs: move duplicated nodatasum check into common reflink/dedupe helper fdmanana
2019-01-11 14:55   ` David Sterba
2018-12-12 18:05 ` [PATCH 2/4] Btrfs: use cross mount point check for cloning and deduplication fdmanana
2018-12-13 16:02   ` David Sterba
2019-01-11 14:38     ` David Sterba
2018-12-12 18:05 ` [PATCH 3/4] Btrfs: check if destination root is read-only for deduplication fdmanana
2018-12-13 16:07   ` David Sterba
2019-01-31 16:39     ` Filipe Manana
2019-01-31 16:44       ` Hugo Mills
2019-02-18 15:38         ` David Sterba
2019-02-18 16:55           ` Filipe Manana [this message]
2019-02-12 17:59       ` Filipe Manana
2019-02-20 16:41       ` Zygo Blaxell
2019-02-20 16:54         ` Filipe Manana
2019-02-20 17:17           ` Zygo Blaxell
2019-02-22 11:13             ` Filipe Manana
2019-02-22 17:25               ` David Sterba
2019-02-21 16:54           ` Zygo Blaxell
2019-02-18 16:01   ` David Sterba
2018-12-12 18:05 ` [PATCH 4/4] Btrfs: remove no longer needed range length checks " fdmanana
2018-12-13 12:20   ` Nikolay Borisov
2019-01-31 16:31   ` Filipe Manana
2019-02-12 17:58     ` Filipe Manana
2019-02-18 15:10     ` David Sterba
2018-12-13 12:19 ` [PATCH 0/4] Btrfs: a few more cleanups and fixes for clone/deduplication Nikolay Borisov

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAL3q7H53D=7nawb2wugg0d4S4KH2Sx=DuZVwUcsfrzrv-+PN6Q@mail.gmail.com' \
    --to=fdmanana@kernel.org \
    --cc=dsterba@suse.cz \
    --cc=hugo@carfax.org.uk \
    --cc=linux-btrfs@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-BTRFS Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-btrfs/0 linux-btrfs/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-btrfs linux-btrfs/ https://lore.kernel.org/linux-btrfs \
		linux-btrfs@vger.kernel.org linux-btrfs@archiver.kernel.org
	public-inbox-index linux-btrfs


Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-btrfs


AGPL code for this site: git clone https://public-inbox.org/ public-inbox