From: David Sterba <email@example.com> To: Hugo Mills <firstname.lastname@example.org>, Filipe Manana <email@example.com>, firstname.lastname@example.org, linux-btrfs <email@example.com> Subject: Re: [PATCH 3/4] Btrfs: check if destination root is read-only for deduplication Date: Mon, 18 Feb 2019 16:38:14 +0100 Message-ID: <20190218153814.GL9874@twin.jikos.cz> (raw) In-Reply-To: <20190131164439.GC4461@carfax.org.uk> On Thu, Jan 31, 2019 at 04:44:39PM +0000, Hugo Mills wrote: > On Thu, Jan 31, 2019 at 04:39:22PM +0000, Filipe Manana wrote: > > On Thu, Dec 13, 2018 at 4:08 PM David Sterba <firstname.lastname@example.org> wrote: > > > > > > On Wed, Dec 12, 2018 at 06:05:58PM +0000, email@example.com wrote: > > > > From: Filipe Manana <firstname.lastname@example.org> > > > > > > > > Checking if the destination root is read-only was being performed only for > > > > clone operations. Make deduplication check it as well, as it does not make > > > > sense to not do it, even if it is an operation that does not change the > > > > file contents (such as defrag for example, which checks first if the root > > > > is read-only). > > > > > > And this is also change in user-visible behaviour of dedupe, so this > > > needs to be verified if it's not breaking existing tools. > > > > Have you had the chance to do such verification? > > > > This actually conflicts with send. Send does not expect a root/tree to > > change, and with dedupe on read-only roots happening > > in parallel with send is going to cause all sorts of unexpected and > > undesired problems... > > > > This is a problem introduced by dedupe ioctl when it landed, since > > send existed for a longer time (when nothing else was > > allowed to change read-only roots, including defrag). > > > > I understand it can break some applications, but adding other solution > > such as preventing send and dedupe from running in parallel > > (erroring out or block and wait for each other, etc) is going to be > > really ugly. There's always the workaround for apps to set the > > subvolume > > to RW mode, do the dedupe, then switch it back to RO mode. > > Only if you want your incremental send chain to break on the way > past... > > I think it's fairly clear by now (particularly from the last thread > on this a couple of weeks ago) that making RO subvols RW and then back > again is a fast way to broken incremental receives. So, I think the way it should be fixed is to keep deduplication off the read-only subvolumes. The main reason I see is to avoid the random problems that arise from send + dedupe interaction. The cost is lower deduplication ratio. The main usecase being a primary subvolume with RO snapshots over time, with optional incremental send. That I know is common and widely used. A problematic usecase that would utilize deduplication over RO snapshots does could be something like a set of subvolumes that have very similar data, get snapshotted or set RO, followed by a dedupe pass. To support the latter I'd go only via the RO->RW, dedupe, RW->RO way, as this records that there was a change and does not collide with send assumptions. That this leads to loss of incremental send must be communicated to the user with possible override, eg. --I-know-what-I-am-doing option.
next prev parent reply index Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-12-12 18:05 [PATCH 0/4] Btrfs: a few more cleanups and fixes for clone/deduplication fdmanana 2018-12-12 18:05 ` [PATCH 1/4] Btrfs: move duplicated nodatasum check into common reflink/dedupe helper fdmanana 2019-01-11 14:55 ` David Sterba 2018-12-12 18:05 ` [PATCH 2/4] Btrfs: use cross mount point check for cloning and deduplication fdmanana 2018-12-13 16:02 ` David Sterba 2019-01-11 14:38 ` David Sterba 2018-12-12 18:05 ` [PATCH 3/4] Btrfs: check if destination root is read-only for deduplication fdmanana 2018-12-13 16:07 ` David Sterba 2019-01-31 16:39 ` Filipe Manana 2019-01-31 16:44 ` Hugo Mills 2019-02-18 15:38 ` David Sterba [this message] 2019-02-18 16:55 ` Filipe Manana 2019-02-12 17:59 ` Filipe Manana 2019-02-20 16:41 ` Zygo Blaxell 2019-02-20 16:54 ` Filipe Manana 2019-02-20 17:17 ` Zygo Blaxell 2019-02-22 11:13 ` Filipe Manana 2019-02-22 17:25 ` David Sterba 2019-02-21 16:54 ` Zygo Blaxell 2019-02-18 16:01 ` David Sterba 2018-12-12 18:05 ` [PATCH 4/4] Btrfs: remove no longer needed range length checks " fdmanana 2018-12-13 12:20 ` Nikolay Borisov 2019-01-31 16:31 ` Filipe Manana 2019-02-12 17:58 ` Filipe Manana 2019-02-18 15:10 ` David Sterba 2018-12-13 12:19 ` [PATCH 0/4] Btrfs: a few more cleanups and fixes for clone/deduplication Nikolay Borisov
Reply instructions: You may reply publically to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20190218153814.GL9874@twin.jikos.cz \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
Linux-BTRFS Archive on lore.kernel.org Archives are clonable: git clone --mirror https://lore.kernel.org/linux-btrfs/0 linux-btrfs/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 linux-btrfs linux-btrfs/ https://lore.kernel.org/linux-btrfs \ email@example.com firstname.lastname@example.org public-inbox-index linux-btrfs Newsgroup available over NNTP: nntp://nntp.lore.kernel.org/org.kernel.vger.linux-btrfs AGPL code for this site: git clone https://public-inbox.org/ public-inbox