Any tips for moving to reflink?

Any tips for moving to reflink?

[Alphazo]

Hello,

xfs never stops impressing me. Most if not all of my storage has been
converted to it, including a 30TB+ unRaid server. I have been
contemplating snapshot and dedup enabled filesystems like btrfs and
ZFS but couldn't justify the jump. The new xfs reflink feature seems a
good addition and compromise to a stable and proven filesystem.
I'm strongly considering using the experimental reflink feature for my
new photo drive. This will be done in a controlled environment with
multiple backups.
Are there any general rules of conduct when working with a dedup
filesystem, like never go above 80% disk usage or never trust df/dh
results?
Also is there any risk of trying to mount a reflink enabled xfs
filesystem on an older kernel that doesn't know about it?

I don't think I need it for the moment but would enabling rmapbt as
well add any risk to the data integrity or impact performance? Can
rmapbt be enabled later on an existing filesystem if required? Will
rmpabt be only used to enhance filesystem recovery?

I'm planning to use this reflink feature for instant local snapshots
and then use my backup software of choice, borg, to keep a long time
history of my work on a remote server. Since borg stores data in a
dedup fashion I can also backup the reflink snapshots and they won't
take additional space. The only drawback is that today borg need to
hash all the files found in a reflink directory in order to find out
about dedup blocks. I asked a question on the borg mailing list
https://github.com/borgbackup/borg/issues/2743 and apparently it won't
be an issue to add a feature to support XFS in order to identify the
physical extents. Is rmapbt required for that?

Alphazo

Re: Any tips for moving to reflink?

[Darrick J. Wong]

On Fri, Jun 30, 2017 at 10:33:03AM +0200, Alphazo wrote:
> Hello,
> 
> xfs never stops impressing me. Most if not all of my storage has been
> converted to it, including a 30TB+ unRaid server. I have been
> contemplating snapshot and dedup enabled filesystems like btrfs and
> ZFS but couldn't justify the jump. The new xfs reflink feature seems a
> good addition and compromise to a stable and proven filesystem.
> I'm strongly considering using the experimental reflink feature for my
> new photo drive.

FWIW, reflink is still an experimental feature and not production ready,
so beware that it could eat your backups...

> This will be done in a controlled environment with multiple backups.

...so good to hear this.  Note that the copy-on-write side of reflink
can fragment files and filesystems more heavily than 

> Are there any general rules of conduct when working with a dedup
> filesystem, like never go above 80% disk usage

That's a pretty good rule of thumb for a filesystem regardless of
whether it supports reflink. :)

That said, we do try to reserve space for future metadata expansion
to avoid running out of space and crashing the fs.

> or never trust df/dh results?

The free counts should be accurate, space reservations notwithstanding.

> Also is there any risk of trying to mount a reflink enabled xfs
> filesystem on an older kernel that doesn't know about it?

No, we set a feature flag bit for reflink (and rmap) so that old
kernels will not try to write to an fs they don't understand.

> I don't think I need it for the moment but would enabling rmapbt as
> well add any risk to the data integrity

It shouldn't.

> or impact performance?

It will, on account of having to update the rmap metadata.

> Can rmapbt be enabled later on an existing filesystem if required?

No.

> Will rmpabt be only used to enhance filesystem recovery?

It may some day be used to implement online shrink, but for now the only
things that /could/ use it are xfs_repair and online fsck.

> I'm planning to use this reflink feature for instant local snapshots
> and then use my backup software of choice, borg, to keep a long time
> history of my work on a remote server. Since borg stores data in a
> dedup fashion I can also backup the reflink snapshots and they won't
> take additional space. The only drawback is that today borg need to
> hash all the files found in a reflink directory in order to find out
> about dedup blocks. I asked a question on the borg mailing list
> https://github.com/borgbackup/borg/issues/2743 and apparently it won't
> be an issue to add a feature to support XFS in order to identify the
> physical extents. Is rmapbt required for that?

borgbackup will probably need to call the GETFSMAP ioctl, which won't
land until 4.12.  On xfs, rmapbt is needed to supply data block
ownership info, which is what borgbackup (and bees, and...) say they
want to be smarter about dedup.

Good luck!

--D

> Alphazo
> --
> To unsubscribe from this list: send the line "unsubscribe linux-xfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Re: Any tips for moving to reflink?

[Marian Beermann]

Hi

> > I'm planning to use this reflink feature for instant local snapshots
> > and then use my backup software of choice, borg, to keep a long time
> > history of my work on a remote server. Since borg stores data in a
> > dedup fashion I can also backup the reflink snapshots and they won't
> > take additional space. The only drawback is that today borg need to
> > hash all the files found in a reflink directory in order to find out
> > about dedup blocks. I asked a question on the borg mailing list
> > https://github.com/borgbackup/borg/issues/2743 and apparently it
> > won't be an issue to add a feature to support XFS in order to
> > identify the physical extents. Is rmapbt required for that?
>
> borgbackup will probably need to call the GETFSMAP ioctl, which won't
> land until 4.12.  On xfs, rmapbt is needed to supply data block
> ownership info, which is what borgbackup (and bees, and...) say they
> want to be smarter about dedup.

My understanding so far was that FIEMAP would be sufficient to query the
extents associated with a file. Shouldn't this be sufficient to know
whether two files on the same file system refer to the same data?

Cheers, Marian

Re: Re: Any tips for moving to reflink?

[Darrick J. Wong]

On Sat, Jul 01, 2017 at 01:41:43PM +0200, Marian Beermann wrote:
> Hi
> 
> > > I'm planning to use this reflink feature for instant local snapshots
> > > and then use my backup software of choice, borg, to keep a long time
> > > history of my work on a remote server. Since borg stores data in a
> > > dedup fashion I can also backup the reflink snapshots and they won't
> > > take additional space. The only drawback is that today borg need to
> > > hash all the files found in a reflink directory in order to find out
> > > about dedup blocks. I asked a question on the borg mailing list
> > > https://github.com/borgbackup/borg/issues/2743 and apparently it
> > > won't be an issue to add a feature to support XFS in order to
> > > identify the physical extents. Is rmapbt required for that?
> >
> > borgbackup will probably need to call the GETFSMAP ioctl, which won't
> > land until 4.12.  On xfs, rmapbt is needed to supply data block
> > ownership info, which is what borgbackup (and bees, and...) say they
> > want to be smarter about dedup.
> 
> My understanding so far was that FIEMAP would be sufficient to query the
> extents associated with a file. Shouldn't this be sufficient to know
> whether two files on the same file system refer to the same data?

Not necessarily -- FIEMAP provides physical offset into a device but
does not actually identify which one, which is a problem on multi-device
filesystems such as btrfs and XFS.  IIRC btrfs creates a virtual
physical offset space consisting of all the devices one after the other,
but then you have to know /that/ mapping too.  GETFSMAP by contrast
tells you which device and where on that device.

--D

> 
> Cheers, Marian
> --
> To unsubscribe from this list: send the line "unsubscribe linux-xfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Any tips for moving to reflink?

[Marian Beermann]

On 01.07.2017 18:38, Darrick J. Wong wrote:
> On Sat, Jul 01, 2017 at 01:41:43PM +0200, Marian Beermann wrote:
>> Hi
>>
>>>> I'm planning to use this reflink feature for instant local snapshots
>>>> and then use my backup software of choice, borg, to keep a long time
>>>> history of my work on a remote server. Since borg stores data in a
>>>> dedup fashion I can also backup the reflink snapshots and they won't
>>>> take additional space. The only drawback is that today borg need to
>>>> hash all the files found in a reflink directory in order to find out
>>>> about dedup blocks. I asked a question on the borg mailing list
>>>> https://github.com/borgbackup/borg/issues/2743 and apparently it
>>>> won't be an issue to add a feature to support XFS in order to
>>>> identify the physical extents. Is rmapbt required for that?
>>>
>>> borgbackup will probably need to call the GETFSMAP ioctl, which won't
>>> land until 4.12.  On xfs, rmapbt is needed to supply data block
>>> ownership info, which is what borgbackup (and bees, and...) say they
>>> want to be smarter about dedup.
>>
>> My understanding so far was that FIEMAP would be sufficient to query the
>> extents associated with a file. Shouldn't this be sufficient to know
>> whether two files on the same file system refer to the same data?
> 
> Not necessarily -- FIEMAP provides physical offset into a device but
> does not actually identify which one, which is a problem on multi-device
> filesystems such as btrfs and XFS.  IIRC btrfs creates a virtual
> physical offset space consisting of all the devices one after the other,
> but then you have to know /that/ mapping too.  GETFSMAP by contrast
> tells you which device and where on that device.
> 

I see. If FIEMAP reports same data, while describing different data,
then it certainly breaks one of the main uses of it (detecting identical
data)?

To clarify the intended;

Borg would essentially hash the output of FIEMAP/GETFSMAP for a given
file and compare this hash with a previous hash.

If the two hashes don't match,
then Borg would re-process the entire file.

It'd be possible to make this more granular, on a per-extent basis

Cheers, Marian

Re: Any tips for moving to reflink?

[Darrick J. Wong]

On Sun, Jul 02, 2017 at 06:40:25PM +0200, Marian Beermann wrote:
> On 01.07.2017 18:38, Darrick J. Wong wrote:
> > On Sat, Jul 01, 2017 at 01:41:43PM +0200, Marian Beermann wrote:
> >> Hi
> >>
> >>>> I'm planning to use this reflink feature for instant local snapshots
> >>>> and then use my backup software of choice, borg, to keep a long time
> >>>> history of my work on a remote server. Since borg stores data in a
> >>>> dedup fashion I can also backup the reflink snapshots and they won't
> >>>> take additional space. The only drawback is that today borg need to
> >>>> hash all the files found in a reflink directory in order to find out
> >>>> about dedup blocks. I asked a question on the borg mailing list
> >>>> https://github.com/borgbackup/borg/issues/2743 and apparently it
> >>>> won't be an issue to add a feature to support XFS in order to
> >>>> identify the physical extents. Is rmapbt required for that?
> >>>
> >>> borgbackup will probably need to call the GETFSMAP ioctl, which won't
> >>> land until 4.12.  On xfs, rmapbt is needed to supply data block
> >>> ownership info, which is what borgbackup (and bees, and...) say they
> >>> want to be smarter about dedup.
> >>
> >> My understanding so far was that FIEMAP would be sufficient to query the
> >> extents associated with a file. Shouldn't this be sufficient to know
> >> whether two files on the same file system refer to the same data?
> > 
> > Not necessarily -- FIEMAP provides physical offset into a device but
> > does not actually identify which one, which is a problem on multi-device
> > filesystems such as btrfs and XFS.  IIRC btrfs creates a virtual
> > physical offset space consisting of all the devices one after the other,
> > but then you have to know /that/ mapping too.  GETFSMAP by contrast
> > tells you which device and where on that device.
> > 
> 
> I see. If FIEMAP reports same data, while describing different data,
> then it certainly breaks one of the main uses of it (detecting identical
> data)?

In theory, given that FIEMAP does not itself return a device identifier,
you're supposed to know which device for which it's returning extent
information a priori.  That's no help at all to the hapless application
writer, of course.

AFAICT, ocfs2 and ext4 have supported FIEMAP since its introduction; for
these two filesystems the device info is easily guessable since they
only support putting files on a single device.

For btrfs you'd have to figure out how it maps devices to its physical
offset address space.  There's probably an ioctl to query the
appropriate btree, but I don't know how.

For XFS you have to query the inode attributes via FS_IOC_FSGETXATTR,
look for the FS_XFLAG_REALTIME flag, and choose between the data device
or the realtime device (which you can find by parsing /proc/mounts).

Not broken, per se, just insufficient for today's needs, hence GETFSMAP.
Though, opening files by inode number isn't straightforward either.

> To clarify the intended;
> 
> Borg would essentially hash the output of FIEMAP/GETFSMAP for a given
> file and compare this hash with a previous hash.
> 
> If the two hashes don't match,
> then Borg would re-process the entire file.
> 
> It'd be possible to make this more granular, on a per-extent basis

Keep in mind that XFS only does CoW if it has to, so file contents don't
necessarily change the extent map.

--D

> 
> Cheers, Marian
> --
> To unsubscribe from this list: send the line "unsubscribe linux-xfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html