how do I know a subvolume is a snapshot?

how do I know a subvolume is a snapshot?

[Ulli Horlacher]

I thought, I can recognize a snapshot when it has a Parent UUID, but this
is not true for snapshots of toplevel subvolumes: 

root@trulla:/# btrfs version
btrfs-progs v4.5.3+20160729

root@trulla:/# btrfs subvolume show /mnt/tmp
/mnt/tmp is toplevel subvolume

root@trulla:/# btrfs subvolume snapshot /mnt/tmp /mnt/tmp/ss
Create a snapshot of '/mnt/tmp' in '/mnt/tmp/ss'

root@trulla:/# btrfs subvolume create /mnt/tmp/xx
Create subvolume '/mnt/tmp/xx'

root@trulla:/# btrfs subvolume show /mnt/tmp/ss
/mnt/tmp/ss
        Name:                   ss
        UUID:                   7732bdde-0485-204e-b41b-833376e791da
        Parent UUID:            -
        Received UUID:          -
        Creation time:          2019-07-17 01:02:48 +0200
        Subvolume ID:           270
        Generation:             60
        Gen at creation:        60
        Parent ID:              5
        Top level ID:           5
        Flags:                  -
        Snapshot(s):

root@trulla:/# btrfs subvolume show /mnt/tmp/xx
/mnt/tmp/xx
        Name:                   xx
        UUID:                   342b2065-1679-8245-bd76-8da598cc33d8
        Parent UUID:            -
        Received UUID:          -
        Creation time:          2019-07-17 01:03:02 +0200
        Subvolume ID:           271
        Generation:             61
        Gen at creation:        61
        Parent ID:              5
        Top level ID:           5
        Flags:                  -
        Snapshot(s):

How do I know that /mnt/tmp/ss is a snapshot?
I cannot see a snapshot identifier.

-- 
Ullrich Horlacher              Server und Virtualisierung
Rechenzentrum TIK         
Universitaet Stuttgart         E-Mail: horlacher@tik.uni-stuttgart.de
Allmandring 30a                Tel:    ++49-711-68565868
70569 Stuttgart (Germany)      WWW:    http://www.tik.uni-stuttgart.de/
REF:<20190716232456.GA26411@tik.uni-stuttgart.de>

Re: how do I know a subvolume is a snapshot?

[Bernhard Kühnel]

Am 17.07.2019 um 01:24 schrieb Ulli Horlacher:
> How do I know that /mnt/tmp/ss is a snapshot?
> I cannot see a snapshot identifier.

From the btrfs-subvolume man page:

>       A snapshot is a subvolume like any other, with given initial
content. By default, snapshots are created
>       read-write. File modifications in a snapshot do not affect the
files in the original subvolume.


I believe the usual practice is to create snapshots with the -r flag and
follow some naming convention (e.g. place them in a common .snapshots
folder named by date), but you're free to switch between read-only and
read-write mode for a snapshot at any time using the btrfs property command.

That allows for some intereresting feats: e.g. there's no guarantee that
a (now) read-only snapshot actually reflects the source's state at
creation time (if someone modified it and re-applied the ro flag). On
the other hand, reverting to a snapshotted state may be as easy as
making the snapshot rw and changing the mount options to use it's
subvolid - no need to copy any files back and forth.


If there is a specific reason why you want to discern snapshots from
other subvolumes (verification/auditing/forensics?), maybe you can get
help here by elaborating further on that.


Bernhard Kühnel

Re: how do I know a subvolume is a snapshot?

[Ulli Horlacher]

On Wed 2019-07-17 (09:45), Bernhard Kühnel wrote:
> Am 17.07.2019 um 01:24 schrieb Ulli Horlacher:
> 
> > How do I know that /mnt/tmp/ss is a snapshot?
> > I cannot see a snapshot identifier.
> 
> From the btrfs-subvolume man page:
> 
> >       A snapshot is a subvolume like any other, with given initial content. By default, snapshots are created
> >       read-write. File modifications in a snapshot do not affect the files in the original subvolume.

I know this, but my question was not "What is a snapshot".


> I believe the usual practice is to create snapshots with the -r flag and
> follow some naming convention (e.g. place them in a common .snapshots
> folder named by date), but you're free to switch between read-only and
> read-write mode for a snapshot at any time using the btrfs property command.

This is true also for my users and co-admins.
Though, I want to know which subvolume is a snapshot of what subvolume.

Non-toplevel subvolume snapshots have a Parent UUID - why not all?!
Or at least some kind of flag "I am a snapshot".


-- 
Ullrich Horlacher              Server und Virtualisierung
Rechenzentrum TIK         
Universitaet Stuttgart         E-Mail: horlacher@tik.uni-stuttgart.de
Allmandring 30a                Tel:    ++49-711-68565868
70569 Stuttgart (Germany)      WWW:    http://www.tik.uni-stuttgart.de/
REF:<1858db2d-8683-0ba9-cc13-9e654a1fa810@allchangeplease.de>

Re: how do I know a subvolume is a snapshot?

[Remi Gauvin]

[-- Attachment #1.1: Type: text/plain, Size: 1054 bytes --]

On 2019-07-17 3:45 a.m., Bernhard Kühnel wrote:

> 
> I believe the usual practice is to create snapshots with the -r flag and
> follow some naming convention (e.g. place them in a common .snapshots
> folder named by date), but you're free to switch between read-only and
> read-write mode for a snapshot at any time using the btrfs property command.
> 
> That allows for some intereresting feats: e.g. there's no guarantee that
> a (now) read-only snapshot actually reflects the source's state at
> creation time (if someone modified it and re-applied the ro flag). On
> the other hand, reverting to a snapshotted state may be as easy as
> making the snapshot rw and changing the mount options to use it's
> subvolid - no need to copy any files back and forth.
>

You should *really* not do this.  Switching subvolumes between rw and ro
will break assumptions that send/receive rely on in interestingly
disastrous, sometimes silent ways.

The proper way to make a ro subvolume snapshot rw is to create a new
snapshot of it.




[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: how do I know a subvolume is a snapshot?

[Hans van Kranenburg]

Hi,

On 7/17/19 1:24 AM, Ulli Horlacher wrote:
> 
> I thought, I can recognize a snapshot when it has a Parent UUID, but this
> is not true for snapshots of toplevel subvolumes: 
> 
> root@trulla:/# btrfs version
> btrfs-progs v4.5.3+20160729
> 
> root@trulla:/# btrfs subvolume show /mnt/tmp
> /mnt/tmp is toplevel subvolume
> 
> root@trulla:/# btrfs subvolume snapshot /mnt/tmp /mnt/tmp/ss
> Create a snapshot of '/mnt/tmp' in '/mnt/tmp/ss'
> 
> root@trulla:/# btrfs subvolume create /mnt/tmp/xx
> Create subvolume '/mnt/tmp/xx'
> 
> root@trulla:/# btrfs subvolume show /mnt/tmp/ss
> /mnt/tmp/ss
>         Name:                   ss
>         UUID:                   7732bdde-0485-204e-b41b-833376e791da
>         Parent UUID:            -
>         Received UUID:          -
>         Creation time:          2019-07-17 01:02:48 +0200
>         Subvolume ID:           270
>         Generation:             60
>         Gen at creation:        60
>         Parent ID:              5
>         Top level ID:           5
>         Flags:                  -
>         Snapshot(s):
> 
> root@trulla:/# btrfs subvolume show /mnt/tmp/xx
> /mnt/tmp/xx
>         Name:                   xx
>         UUID:                   342b2065-1679-8245-bd76-8da598cc33d8
>         Parent UUID:            -
>         Received UUID:          -
>         Creation time:          2019-07-17 01:03:02 +0200
>         Subvolume ID:           271
>         Generation:             61
>         Gen at creation:        61
>         Parent ID:              5
>         Top level ID:           5
>         Flags:                  -
>         Snapshot(s):
> 
> How do I know that /mnt/tmp/ss is a snapshot?
> I cannot see a snapshot identifier.
> 

The technical answer is:

* The uuid of the top-level subvol (id 5) is
00000000-0000-0000-0000-000000000000
* The parent uuid of your snapshot/clone is also 0000... as a result.
* For interpretation of parent uuid field, there is no distinction
between "has no parent uuid" (the field is empty, which technically
means all bits are 0) and the actual value 0000...
* So... the tooling displays it as "-". ¯\_(ツ)_/¯

The practical answer is:

* Don't put anything in the top level subvol that would make you want to
snapshot it.

Note: In btrfs design, there is no technical difference between a
snapshot and a writable clone (as opposed to e.g. netapp snapshots and
clones). This makes the design less complicated, but it results in a
collection of known usability issues, like that if you expect them to be
different, they still aren't.

Hans

RE: how do I know a subvolume is a snapshot?

[misono.tomohiro]

> -----Original Message-----
> From: linux-btrfs-owner@vger.kernel.org [mailto:linux-btrfs-owner@vger.kernel.org] On Behalf Of Hans van Kranenburg
> Sent: Wednesday, July 17, 2019 5:24 PM
> To: linux-btrfs@vger.kernel.org; Ulli Horlacher <framstag@rus.uni-stuttgart.de>
> Subject: Re: how do I know a subvolume is a snapshot?
> 
> Hi,
> 
> On 7/17/19 1:24 AM, Ulli Horlacher wrote:
> >
> > I thought, I can recognize a snapshot when it has a Parent UUID, but
> > this is not true for snapshots of toplevel subvolumes:
> >
> > root@trulla:/# btrfs version
> > btrfs-progs v4.5.3+20160729
> >
> > root@trulla:/# btrfs subvolume show /mnt/tmp /mnt/tmp is toplevel
> > subvolume
> >
> > root@trulla:/# btrfs subvolume snapshot /mnt/tmp /mnt/tmp/ss Create a
> > snapshot of '/mnt/tmp' in '/mnt/tmp/ss'
> >
> > root@trulla:/# btrfs subvolume create /mnt/tmp/xx Create subvolume
> > '/mnt/tmp/xx'
> >
> > root@trulla:/# btrfs subvolume show /mnt/tmp/ss /mnt/tmp/ss
> >         Name:                   ss
> >         UUID:                   7732bdde-0485-204e-b41b-833376e791da
> >         Parent UUID:            -
> >         Received UUID:          -
> >         Creation time:          2019-07-17 01:02:48 +0200
> >         Subvolume ID:           270
> >         Generation:             60
> >         Gen at creation:        60
> >         Parent ID:              5
> >         Top level ID:           5
> >         Flags:                  -
> >         Snapshot(s):
> >
> > root@trulla:/# btrfs subvolume show /mnt/tmp/xx /mnt/tmp/xx
> >         Name:                   xx
> >         UUID:                   342b2065-1679-8245-bd76-8da598cc33d8
> >         Parent UUID:            -
> >         Received UUID:          -
> >         Creation time:          2019-07-17 01:03:02 +0200
> >         Subvolume ID:           271
> >         Generation:             61
> >         Gen at creation:        61
> >         Parent ID:              5
> >         Top level ID:           5
> >         Flags:                  -
> >         Snapshot(s):
> >
> > How do I know that /mnt/tmp/ss is a snapshot?
> > I cannot see a snapshot identifier.
> >
> 
> The technical answer is:
> 
> * The uuid of the top-level subvol (id 5) is
> 00000000-0000-0000-0000-000000000000
> * The parent uuid of your snapshot/clone is also 0000... as a result.
> * For interpretation of parent uuid field, there is no distinction between "has no parent uuid" (the field is empty,
> which technically means all bits are 0) and the actual value 0000...
> * So... the tooling displays it as "-". ¯\_(ツ)_/¯
> 
> The practical answer is:
> 
> * Don't put anything in the top level subvol that would make you want to snapshot it.
> 
> Note: In btrfs design, there is no technical difference between a snapshot and a writable clone (as opposed to e.g.
> netapp snapshots and clones). This makes the design less complicated, but it results in a collection of known usability
> issues, like that if you expect them to be different, they still aren't.
> 
> Hans

Hello,

FYI, this problem should be fixed in mkfs.btrfs >= v4.16 since the top-level
subvolume also gets non-empty UUID at mkfs time.

Thanks

Re: how do I know a subvolume is a snapshot?

[Ulli Horlacher]

On Wed 2019-07-17 (08:57), misono.tomohiro@fujitsu.com wrote:

> FYI, this problem should be fixed in mkfs.btrfs >= v4.16 since the top-level
> subvolume also gets non-empty UUID at mkfs time.

root@fex:~# lsb_release -d
Description:    Ubuntu 18.04.2 LTS

root@fex:~# btrfs version
btrfs-progs v4.15.1

*sigh*

root@fex:~# uname -a
Linux fex 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Can I use/install a newer mkfs.btrfs or do I also have to install a newer
kernel version?


-- 
Ullrich Horlacher              Server und Virtualisierung
Rechenzentrum TIK         
Universitaet Stuttgart         E-Mail: horlacher@tik.uni-stuttgart.de
Allmandring 30a                Tel:    ++49-711-68565868
70569 Stuttgart (Germany)      WWW:    http://www.tik.uni-stuttgart.de/
REF:<TYAPR01MB33604A6035BF09518027AE12E5C90@TYAPR01MB3360.jpnprd01.prod.outlook.com>

RE: how do I know a subvolume is a snapshot?

[misono.tomohiro]

> -----Original Message-----
> From: linux-btrfs-owner@vger.kernel.org [mailto:linux-btrfs-owner@vger.kernel.org] On Behalf Of Ulli Horlacher
> Sent: Wednesday, July 17, 2019 6:07 PM
> To: linux-btrfs@vger.kernel.org
> Subject: Re: how do I know a subvolume is a snapshot?
> 
> On Wed 2019-07-17 (08:57), misono.tomohiro@fujitsu.com wrote:
> 
> > FYI, this problem should be fixed in mkfs.btrfs >= v4.16 since the
> > top-level subvolume also gets non-empty UUID at mkfs time.
> 
> root@fex:~# lsb_release -d
> Description:    Ubuntu 18.04.2 LTS
> 
> root@fex:~# btrfs version
> btrfs-progs v4.15.1
> 
> *sigh*
> 
> root@fex:~# uname -a
> Linux fex 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
> 
> Can I use/install a newer mkfs.btrfs or do I also have to install a newer kernel version?

Only newer mkfs.btrfs should be fine here.

Re: how do I know a subvolume is a snapshot?

[Nikolay Borisov]

On 17.07.19 г. 2:24 ч., Ulli Horlacher wrote:
> I thought, I can recognize a snapshot when it has a Parent UUID, but this
> is not true for snapshots of toplevel subvolumes: 

As you have asked this before - in my testing this is not true. Looking
at the code it also seems snapshots get a parent. Alternatively you have
to parse the root tree - the ROOT_ITEM's offset member should be 0 for
well-known trees/ordinary subvolume or the transaction id when the
snapshot was created.

Re: how do I know a subvolume is a snapshot?

[Ulli Horlacher]

On Wed 2019-07-17 (11:24), Nikolay Borisov wrote:
> 
> 
> On 17.07.19 3. 2:24 G., Ulli Horlacher wrote:
> 
> > I thought, I can recognize a snapshot when it has a Parent UUID, but this
> > is not true for snapshots of toplevel subvolumes: 
> 
> As you have asked this before - in my testing this is not true.

It is true on all my SUSE and Ubuntu systems, for all versions.


> Alternatively you have to parse the root tree - the ROOT_ITEM's offset
> member should be 0 for well-known trees/ordinary subvolume or the
> transaction id when the snapshot was created.

Where can I find this ROOT_ITEM offset member? Which command?


-- 
Ullrich Horlacher              Server und Virtualisierung
Rechenzentrum TIK         
Universitaet Stuttgart         E-Mail: horlacher@tik.uni-stuttgart.de
Allmandring 30a                Tel:    ++49-711-68565868
70569 Stuttgart (Germany)      WWW:    http://www.tik.uni-stuttgart.de/
REF:<eff513b1-a77c-cd5f-5af7-87eae73cff6a@suse.com>

Re: how do I know a subvolume is a snapshot?

[Nikolay Borisov]

On 17.07.19 г. 12:11 ч., Ulli Horlacher wrote:
> On Wed 2019-07-17 (11:24), Nikolay Borisov wrote:
>>
>>
>> On 17.07.19 3. 2:24 G., Ulli Horlacher wrote:
>>
>>> I thought, I can recognize a snapshot when it has a Parent UUID, but this
>>> is not true for snapshots of toplevel subvolumes: 
>>
>> As you have asked this before - in my testing this is not true.
> 
> It is true on all my SUSE and Ubuntu systems, for all versions.

That's strange, as I've shown in the previous thread, using the latest
master doesn't exhibit this behavior.
> 
> 
>> Alternatively you have to parse the root tree - the ROOT_ITEM's offset
>> member should be 0 for well-known trees/ordinary subvolume or the
>> transaction id when the snapshot was created.
> 
> Where can I find this ROOT_ITEM offset member? Which command?

btrfs inspect-internal dump-tree -t 1 /dev/vdb

> 
> 

Re: how do I know a subvolume is a snapshot?

[Andrei Borzenkov]

On Wed, Jul 17, 2019 at 1:14 PM Nikolay Borisov <nborisov@suse.com> wrote:
>
>
>
> On 17.07.19 г. 12:11 ч., Ulli Horlacher wrote:
> > On Wed 2019-07-17 (11:24), Nikolay Borisov wrote:
> >>
> >>
> >> On 17.07.19 3. 2:24 G., Ulli Horlacher wrote:
> >>
> >>> I thought, I can recognize a snapshot when it has a Parent UUID, but this
> >>> is not true for snapshots of toplevel subvolumes:
> >>
> >> As you have asked this before - in my testing this is not true.
> >
> > It is true on all my SUSE and Ubuntu systems, for all versions.
>
> That's strange, as I've shown in the previous thread, using the latest
> master doesn't exhibit this behavior.

I doubt you are not aware that distributions rarely use latest master.

Actually I have here openSUSE Tumbleweed; root top level subvolume
does not have UUID but if I create new filesystem *now* it does. btrfs
tools have been updated since initial installation.

Better question would be - is it possible to fix it for existing
filesystems that had been created using old tools?

Re: how do I know a subvolume is a snapshot?

[Nikolay Borisov]

On 17.07.19 г. 13:29 ч., Andrei Borzenkov wrote:
> On Wed, Jul 17, 2019 at 1:14 PM Nikolay Borisov <nborisov@suse.com> wrote:
>>
>>
>>
>> On 17.07.19 г. 12:11 ч., Ulli Horlacher wrote:
>>> On Wed 2019-07-17 (11:24), Nikolay Borisov wrote:
>>>>
>>>>
>>>> On 17.07.19 3. 2:24 G., Ulli Horlacher wrote:
>>>>
>>>>> I thought, I can recognize a snapshot when it has a Parent UUID, but this
>>>>> is not true for snapshots of toplevel subvolumes:
>>>>
>>>> As you have asked this before - in my testing this is not true.
>>>
>>> It is true on all my SUSE and Ubuntu systems, for all versions.
>>
>> That's strange, as I've shown in the previous thread, using the latest
>> master doesn't exhibit this behavior.
> 
> I doubt you are not aware that distributions rarely use latest master.
> 
> Actually I have here openSUSE Tumbleweed; root top level subvolume
> does not have UUID but if I create new filesystem *now* it does. btrfs
> tools have been updated since initial installation.

I have an ubuntu 18.04 installation lying around and I see : 

sudo btrfs subvolume show btrfs-mount/
/
	Name: 			<FS_TREE>
	UUID: 			-
	Parent UUID: 		-
	Received UUID: 		-
	Creation time: 		-
	Subvolume ID: 		5
	Generation: 		4
	Gen at creation: 	0
	Parent ID: 		0
	Top level ID: 		0
	Flags: 			-
	Snapshot(s):

This is really odd... So this indeed seems to be a userspace problem. 
However, creating a subvolume and then a snapshot I see sane output - parent UUID being there and UUID being there for a kernel-created subvol. : 

nborisov@fisk:~/projects/kernel/source$ sudo btrfs subvolume show btrfs-mount/subvol1-snap1/
subvol1-snap1
	Name: 			subvol1-snap1
	UUID: 			3aebb55e-57bc-9c46-9a34-4ac0220d602e
	Parent UUID: 		fe7e68c6-b9c8-ce4d-8467-7229bd39b0eb
	Received UUID: 		-
	Creation time: 		2019-07-17 13:55:31 +0300
	Subvolume ID: 		258
	Generation: 		9
	Gen at creation: 	9
	Parent ID: 		5
	Top level ID: 		5
	Flags: 			-
	Snapshot(s):

nborisov@fisk:~/projects/kernel/source$ sudo btrfs subvolume show btrfs-mount/subvolume1/
subvolume1
	Name: 			subvolume1
	UUID: 			fe7e68c6-b9c8-ce4d-8467-7229bd39b0eb
	Parent UUID: 		-
	Received UUID: 		-
	Creation time: 		2019-07-17 13:55:14 +0300
	Subvolume ID: 		257
	Generation: 		9
	Gen at creation: 	8
	Parent ID: 		5
	Top level ID: 		5
	Flags: 			-
	Snapshot(s):
				subvol1-snap1


> 
> Better question would be - is it possible to fix it for existing
> filesystems that had been created using old tools?
> 

Re: how do I know a subvolume is a snapshot?

[Andrei Borzenkov]

17.07.2019 14:19, Nikolay Borisov пишет:
> 
> This is really odd... So this indeed seems to be a userspace problem. 


Of course it is user space problem.

commit 0a0a03554aaf56a6e7245e74fa7d8b3c53f1c20f
Author: Misono Tomohiro <misono.tomohiro@jp.fujitsu.com>
Date:   Fri Mar 23 17:16:49 2018 +0900

    btrfs-progs: mkfs: add uuid and otime to ROOT_ITEM of, FS_TREE

    Currently, the top-level subvolume lacks the UUID. As a result, both
    non-snapshot subvolume and snapshot of top-level subvolume do not have
    Parent UUID and cannot be distinguisued. Therefore "fi show" of
    top-level lists all the subvolumes which lacks the UUID in
    "Snapshot(s)" filed.  Also, it lacks the otime information.

    Fix this by adding the UUID and otime at the mkfs time.  As a
    consequence, snapshots of top-level subvolume now have a Parent UUID and
    UUID tree will create an entry for top-level subvolume at mount time.
    This should not cause the problem for current kernel, but user program
    which relies on the empty Parent UUID may be affected by this change.


What about the question - is there tool to fix existing filesystem by
adding this information?

Re: how do I know a subvolume is a snapshot?

[Nikolay Borisov]

On 17.07.19 г. 20:39 ч., Andrei Borzenkov wrote:
> 17.07.2019 14:19, Nikolay Borisov пишет:
>>
>> This is really odd... So this indeed seems to be a userspace problem. 
> 
> 
> Of course it is user space problem.
> 
> commit 0a0a03554aaf56a6e7245e74fa7d8b3c53f1c20f
> Author: Misono Tomohiro <misono.tomohiro@jp.fujitsu.com>
> Date:   Fri Mar 23 17:16:49 2018 +0900
> 
>     btrfs-progs: mkfs: add uuid and otime to ROOT_ITEM of, FS_TREE
> 
>     Currently, the top-level subvolume lacks the UUID. As a result, both
>     non-snapshot subvolume and snapshot of top-level subvolume do not have
>     Parent UUID and cannot be distinguisued. Therefore "fi show" of
>     top-level lists all the subvolumes which lacks the UUID in
>     "Snapshot(s)" filed.  Also, it lacks the otime information.
> 
>     Fix this by adding the UUID and otime at the mkfs time.  As a
>     consequence, snapshots of top-level subvolume now have a Parent UUID and
>     UUID tree will create an entry for top-level subvolume at mount time.
>     This should not cause the problem for current kernel, but user program
>     which relies on the empty Parent UUID may be affected by this change.
> 
> 
> What about the question - is there tool to fix existing filesystem by
> adding this information?

Short answer - no. Long answer - it would be trivial to add something
like that to btrfstune but this will only work for root volumes. Because
for snapshots you'd have to iterate all snapshots for a subvol, ensure
they haven't really changed i.e. are real snapshosts and then populate
their Parent UUID to that of the newly set UUID of their respective parent.

>