* No files in snapshot @ 2019-08-27 9:33 Thomas Schneider 2019-08-27 22:24 ` Chris Murphy 0 siblings, 1 reply; 10+ messages in thread From: Thomas Schneider @ 2019-08-27 9:33 UTC (permalink / raw) To: linux-btrfs Hi, I'm running Debian 10 with btrfs-progs=5.2.1. Creating snapshots with snapper=0.8.2 works w/o errors. root@ld5507:~# uname -r 5.0.18-1-pve root@ld5508:~# btrfs fi show Label: none uuid: 70997a0b-a36f-46cf-9ddc-b88d47eabc9e Total devices 1 FS bytes used 11.17GiB devid 1 size 192.53GiB used 22.02GiB path /dev/sdbq3 root@ld5507:~# btrfs fi df / Data, single: total=18.00GiB, used=10.71GiB System, DUP: total=8.00MiB, used=16.00KiB Metadata, DUP: total=2.00GiB, used=465.84MiB GlobalReserve, single: total=68.03MiB, used=0.00B However, I run into an issue and need to restore various files. I thought that I could simply take the files from a snapshot created before. However, the files required don't exist in any snapshot! Therefore I have created a new snapshot manually to verify if the files will be included, but there's nothing. These files are required: root@ld5507:/usr/bin# ls -l /var/lib/ceph/osd/ceph-4/ insgesamt 56 -rw-r--r-- 1 root root 402 Jun 7 12:18 activate.monmap -rw-r--r-- 1 ceph ceph 3 Jun 7 12:18 active lrwxrwxrwx 1 ceph ceph 58 Jun 7 12:18 block -> /dev/disk/by-partuuid/3bc0c812-2c6b-4544-bbe7-e0444c3448eb -rw-r--r-- 1 ceph ceph 37 Jun 7 12:18 block_uuid -rw-r--r-- 1 ceph ceph 2 Jun 7 12:18 bluefs -rw-r--r-- 1 ceph ceph 37 Jun 7 12:18 ceph_fsid -rw-r--r-- 1 ceph ceph 37 Jun 7 12:18 fsid -rw------- 1 ceph ceph 56 Jun 7 12:18 keyring -rw-r--r-- 1 ceph ceph 8 Jun 7 12:18 kv_backend -rw-r--r-- 1 ceph ceph 21 Jun 7 12:18 magic -rw-r--r-- 1 ceph ceph 4 Jun 7 12:18 mkfs_done -rw-r--r-- 1 ceph ceph 6 Jun 7 12:18 ready -rw-r--r-- 1 ceph ceph 3 Aug 23 09:56 require_osd_release -rw-r--r-- 1 ceph ceph 0 Aug 21 12:41 systemd -rw-r--r-- 1 ceph ceph 10 Jun 7 12:18 type -rw-r--r-- 1 ceph ceph 2 Jun 7 12:18 whoami And there are not files in latest snapshot: root@ld5507:/usr/bin# ls -l /btrfs/@snapshots/158/snapshot/var/lib/ceph/osd/ceph-4/ insgesamt 0 Why is this? THX ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: No files in snapshot 2019-08-27 9:33 No files in snapshot Thomas Schneider @ 2019-08-27 22:24 ` Chris Murphy 2019-08-28 14:45 ` Thomas Schneider 2019-09-04 9:24 ` Thomas Schneider 0 siblings, 2 replies; 10+ messages in thread From: Chris Murphy @ 2019-08-27 22:24 UTC (permalink / raw) To: Thomas Schneider; +Cc: Btrfs BTRFS On Tue, Aug 27, 2019 at 3:33 AM Thomas Schneider <74cmonty@gmail.com> wrote: > > However, I run into an issue and need to restore various files. > > I thought that I could simply take the files from a snapshot created before. > However, the files required don't exist in any snapshot! > > Therefore I have created a new snapshot manually to verify if the files > will be included, but there's nothing. Snapshots are not recursive on Btrfs. The snapshot will not extend into nested subvolumes. Check to see if you are snapshotting the proper subvolume. # btrfs sub list -to /var/lib # btrfs sub list -to /var/ In some sense these are redundant, I'm not sure if your /var/lib is a subvolume or not. Also please include the exact snapshot command you're making. -- Chris Murphy ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: No files in snapshot 2019-08-27 22:24 ` Chris Murphy @ 2019-08-28 14:45 ` Thomas Schneider 2019-09-04 17:36 ` Chris Murphy 2019-09-04 9:24 ` Thomas Schneider 1 sibling, 1 reply; 10+ messages in thread From: Thomas Schneider @ 2019-08-28 14:45 UTC (permalink / raw) To: Chris Murphy; +Cc: Btrfs BTRFS Hi, I was thinking of this, too. But it does not apply. root@ld5507:~# btrfs su list -to /var/lib ID gen top level path -- --- --------- ---- root@ld5507:~# btrfs su list -to /var ID gen top level path -- --- --------- ---- And there are files in other directories: root@ld5507:~# ls -l /.snapshots/158/snapshot/var/lib/ceph/mgr/ceph-ld5507/ insgesamt 4 -rw-r--r-- 1 ceph ceph 61 Mai 28 14:33 keyring root@ld5507:~# ls -l /.snapshots/158/snapshot/var/lib/ceph/mon/ceph-ld5507/ insgesamt 12 -rw------- 1 ceph ceph 77 Mai 28 14:33 keyring -rw-r--r-- 1 ceph ceph 8 Mai 28 14:33 kv_backend -rw-r--r-- 1 ceph ceph 3 Aug 23 09:41 min_mon_release drwxr-xr-x 1 ceph ceph 244 Aug 26 18:37 store.db Only this directories /.snapshots/158/snapshot/var/lib/ceph/osd/ceph-<id>/ are empty: root@ld5507:~# ls -l /.snapshots/158/snapshot/var/lib/ceph/osd/ceph-219/ insgesamt 0 To create a snapshot I run this command: snapper create --type single --description "validate /var/lib/ceph/osd/ceph-<n>" Am 28.08.2019 um 00:24 schrieb Chris Murphy: > On Tue, Aug 27, 2019 at 3:33 AM Thomas Schneider <74cmonty@gmail.com> wrote: >> However, I run into an issue and need to restore various files. >> >> I thought that I could simply take the files from a snapshot created before. >> However, the files required don't exist in any snapshot! >> >> Therefore I have created a new snapshot manually to verify if the files >> will be included, but there's nothing. > Snapshots are not recursive on Btrfs. The snapshot will not extend > into nested subvolumes. Check to see if you are snapshotting the > proper subvolume. > > # btrfs sub list -to /var/lib > # btrfs sub list -to /var/ > > In some sense these are redundant, I'm not sure if your /var/lib is a > subvolume or not. Also please include the exact snapshot command > you're making. > ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: No files in snapshot 2019-08-28 14:45 ` Thomas Schneider @ 2019-09-04 17:36 ` Chris Murphy 2019-09-04 18:02 ` Remi Gauvin 0 siblings, 1 reply; 10+ messages in thread From: Chris Murphy @ 2019-09-04 17:36 UTC (permalink / raw) To: Thomas Schneider; +Cc: Chris Murphy, Btrfs BTRFS On Wed, Aug 28, 2019 at 8:45 AM Thomas Schneider <74cmonty@gmail.com> wrote: > > Hi, > > I was thinking of this, too. But it does not apply. > root@ld5507:~# btrfs su list -to /var/lib > ID gen top level path > -- --- --------- ---- > root@ld5507:~# btrfs su list -to /var > ID gen top level path > -- --- --------- ---- > > And there are files in other directories: > root@ld5507:~# ls -l /.snapshots/158/snapshot/var/lib/ceph/mgr/ceph-ld5507/ > insgesamt 4 > -rw-r--r-- 1 ceph ceph 61 Mai 28 14:33 keyring > > root@ld5507:~# ls -l /.snapshots/158/snapshot/var/lib/ceph/mon/ceph-ld5507/ > insgesamt 12 > -rw------- 1 ceph ceph 77 Mai 28 14:33 keyring > -rw-r--r-- 1 ceph ceph 8 Mai 28 14:33 kv_backend > -rw-r--r-- 1 ceph ceph 3 Aug 23 09:41 min_mon_release > drwxr-xr-x 1 ceph ceph 244 Aug 26 18:37 store.db > > Only this directories > /.snapshots/158/snapshot/var/lib/ceph/osd/ceph-<id>/ are empty: > root@ld5507:~# ls -l /.snapshots/158/snapshot/var/lib/ceph/osd/ceph-219/ > insgesamt 0 > > To create a snapshot I run this command: > snapper create --type single --description "validate > /var/lib/ceph/osd/ceph-<n>" > I don't really know how snapper works. The way 'btrfs subvolume snapshot' works, you must point it to a subvolume. It won't snapshot a regular directory and from what you posted above, there are no subvolumes in /var or /var/lib which means trying to snapshot /var/lib/ceph/osd/ceph-.... would fail. So maybe it's failing but snapper doesn't show the error. I'm not really sure. -- Chris Murphy ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: No files in snapshot 2019-09-04 17:36 ` Chris Murphy @ 2019-09-04 18:02 ` Remi Gauvin 2019-09-04 19:07 ` Chris Murphy 0 siblings, 1 reply; 10+ messages in thread From: Remi Gauvin @ 2019-09-04 18:02 UTC (permalink / raw) To: linux-btrfs [-- Attachment #1.1: Type: text/plain, Size: 756 bytes --] On 2019-09-04 1:36 p.m., Chris Murphy wrote: >> > > I don't really know how snapper works. > > The way 'btrfs subvolume snapshot' works, you must point it to a > subvolume. It won't snapshot a regular directory and from what you > posted above, there are no subvolumes in /var or /var/lib which means > trying to snapshot /var/lib/ceph/osd/ceph-.... would fail. So maybe > it's failing but snapper doesn't show the error. I'm not really sure. > In this case, his snapshots are all of the root. I don't know how Ceph works, but since we already confirmed that there are no subvolumes under /var, the only other explanation is that /var/lib/ceph/osd/ceph-<n> is a submount What is the the result of running: mount | grep /var [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 473 bytes --] ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: No files in snapshot 2019-09-04 18:02 ` Remi Gauvin @ 2019-09-04 19:07 ` Chris Murphy 2019-09-04 20:52 ` Oliver Freyermuth 0 siblings, 1 reply; 10+ messages in thread From: Chris Murphy @ 2019-09-04 19:07 UTC (permalink / raw) To: Remi Gauvin; +Cc: linux-btrfs On Wed, Sep 4, 2019 at 12:24 PM Remi Gauvin <remi@georgianit.com> wrote: > > On 2019-09-04 1:36 p.m., Chris Murphy wrote: > > >> > > > > I don't really know how snapper works. > > > > The way 'btrfs subvolume snapshot' works, you must point it to a > > subvolume. It won't snapshot a regular directory and from what you > > posted above, there are no subvolumes in /var or /var/lib which means > > trying to snapshot /var/lib/ceph/osd/ceph-.... would fail. So maybe > > it's failing but snapper doesn't show the error. I'm not really sure. > > > > In this case, his snapshots are all of the root. > > I don't know how Ceph works, but since we already confirmed that there > are no subvolumes under /var, the only other explanation is that > /var/lib/ceph/osd/ceph-<n> is a submount > > What is the the result of running: > mount | grep /var > Yep. -- Chris Murphy ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: No files in snapshot 2019-09-04 19:07 ` Chris Murphy @ 2019-09-04 20:52 ` Oliver Freyermuth 2019-09-05 8:50 ` Thomas Schneider 0 siblings, 1 reply; 10+ messages in thread From: Oliver Freyermuth @ 2019-09-04 20:52 UTC (permalink / raw) To: Chris Murphy, Remi Gauvin; +Cc: linux-btrfs, Thomas Schneider Am 04.09.19 um 21:07 schrieb Chris Murphy: > On Wed, Sep 4, 2019 at 12:24 PM Remi Gauvin <remi@georgianit.com> wrote: >> >> On 2019-09-04 1:36 p.m., Chris Murphy wrote: >> >>>> >>> >>> I don't really know how snapper works. >>> >>> The way 'btrfs subvolume snapshot' works, you must point it to a >>> subvolume. It won't snapshot a regular directory and from what you >>> posted above, there are no subvolumes in /var or /var/lib which means >>> trying to snapshot /var/lib/ceph/osd/ceph-.... would fail. So maybe >>> it's failing but snapper doesn't show the error. I'm not really sure. >>> >> >> In this case, his snapshots are all of the root. >> >> I don't know how Ceph works, but since we already confirmed that there >> are no subvolumes under /var, the only other explanation is that >> /var/lib/ceph/osd/ceph-<n> is a submount >> >> What is the the result of running: >> mount | grep /var >> > > Yep. > > Looking at Thomas' mail on the Ceph-users list: http://lists.ceph.com/pipermail/ceph-users-ceph.com/2019-August/036679.html I deduce he is using Ceph with the Bluestore backend, which indeed means that /var/lib/ceph/osd/ceph.<n> are tmpfs mounts which are completeley ephemeral and are created from LVM metadata of the actual Ceph OSD disks. That would of course also explain why they are not part of any btrfs snapshot of / (and that also means there is no need to backup anything). ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: No files in snapshot 2019-09-04 20:52 ` Oliver Freyermuth @ 2019-09-05 8:50 ` Thomas Schneider 2019-09-05 18:31 ` Oliver Freyermuth 0 siblings, 1 reply; 10+ messages in thread From: Thomas Schneider @ 2019-09-05 8:50 UTC (permalink / raw) To: Oliver Freyermuth, Chris Murphy, Remi Gauvin; +Cc: linux-btrfs Hi, I would agree with Oliver's conclusion: the relevant directories are all tmpfs mounts. root@ld5505:~# mount | grep /var /dev/sdbq3 on /var/cache type btrfs (rw,noatime,compress=lzo,ssd,space_cache=v2,subvolid=260,subvol=/@cache) /dev/sdbq3 on /var/lib/vz/images type btrfs (rw,noatime,compress=lzo,ssd,space_cache=v2,subvolid=262,subvol=/@images) lxcfs on /var/lib/lxcfs type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other) tmpfs on /var/lib/ceph/osd/ceph-122 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-123 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-105 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-92 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-112 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-77 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-76 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-87 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-119 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-79 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-115 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-98 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-100 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-89 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-108 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-120 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-82 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-118 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-104 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-95 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-1 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-96 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-117 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-116 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-99 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-106 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-110 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-97 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-81 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-121 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-88 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-0 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-94 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-113 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-107 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-101 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-78 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-93 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-85 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-103 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-102 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-109 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-114 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-80 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-111 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-83 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-86 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-91 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-84 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-90 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-8 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-9 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-10 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-11 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-12 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-13 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-14 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-15 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-16 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-17 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-18 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-19 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-20 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-21 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-22 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-23 type tmpfs (rw,relatime) tmpfs on /var/lib/ceph/osd/ceph-24 type tmpfs (rw,relatime) Am 04.09.2019 um 22:52 schrieb Oliver Freyermuth: > Am 04.09.19 um 21:07 schrieb Chris Murphy: >> On Wed, Sep 4, 2019 at 12:24 PM Remi Gauvin <remi@georgianit.com> wrote: >>> On 2019-09-04 1:36 p.m., Chris Murphy wrote: >>> >>>> I don't really know how snapper works. >>>> >>>> The way 'btrfs subvolume snapshot' works, you must point it to a >>>> subvolume. It won't snapshot a regular directory and from what you >>>> posted above, there are no subvolumes in /var or /var/lib which means >>>> trying to snapshot /var/lib/ceph/osd/ceph-.... would fail. So maybe >>>> it's failing but snapper doesn't show the error. I'm not really sure. >>>> >>> In this case, his snapshots are all of the root. >>> >>> I don't know how Ceph works, but since we already confirmed that there >>> are no subvolumes under /var, the only other explanation is that >>> /var/lib/ceph/osd/ceph-<n> is a submount >>> >>> What is the the result of running: >>> mount | grep /var >>> >> Yep. >> >> > Looking at Thomas' mail on the Ceph-users list: > http://lists.ceph.com/pipermail/ceph-users-ceph.com/2019-August/036679.html > I deduce he is using Ceph with the Bluestore backend, which indeed means that /var/lib/ceph/osd/ceph.<n> > are tmpfs mounts which are completeley ephemeral and are created from LVM metadata of the actual Ceph OSD disks. > > That would of course also explain why they are not part of any btrfs snapshot of / (and that also means there is no need to backup anything). ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: No files in snapshot 2019-09-05 8:50 ` Thomas Schneider @ 2019-09-05 18:31 ` Oliver Freyermuth 0 siblings, 0 replies; 10+ messages in thread From: Oliver Freyermuth @ 2019-09-05 18:31 UTC (permalink / raw) To: Thomas Schneider, Chris Murphy, Remi Gauvin; +Cc: linux-btrfs Dear Thomas, Am 05.09.19 um 10:50 schrieb Thomas Schneider: > Hi, > > I would agree with Oliver's conclusion: > the relevant directories are all tmpfs mounts. this of course easily explains it and btrfs can't do anything about these when snapshotting. In case you want to debug the issue in more detail (e.g. how to extract this information stored in tmpfs from LVM for example), we can of course take this to the ceph-users list (or contact me directly). Cheers, Oliver > > root@ld5505:~# mount | grep /var > /dev/sdbq3 on /var/cache type btrfs (rw,noatime,compress=lzo,ssd,space_cache=v2,subvolid=260,subvol=/@cache) > /dev/sdbq3 on /var/lib/vz/images type btrfs (rw,noatime,compress=lzo,ssd,space_cache=v2,subvolid=262,subvol=/@images) > lxcfs on /var/lib/lxcfs type fuse.lxcfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other) > tmpfs on /var/lib/ceph/osd/ceph-122 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-123 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-105 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-92 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-112 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-77 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-76 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-87 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-119 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-79 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-115 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-98 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-100 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-89 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-108 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-120 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-82 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-118 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-104 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-95 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-1 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-96 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-117 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-116 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-99 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-106 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-110 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-97 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-81 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-121 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-88 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-0 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-94 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-113 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-107 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-101 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-78 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-93 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-85 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-103 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-102 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-109 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-114 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-80 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-111 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-83 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-86 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-91 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-84 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-90 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-8 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-9 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-10 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-11 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-12 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-13 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-14 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-15 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-16 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-17 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-18 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-19 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-20 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-21 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-22 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-23 type tmpfs (rw,relatime) > tmpfs on /var/lib/ceph/osd/ceph-24 type tmpfs (rw,relatime) > > > Am 04.09.2019 um 22:52 schrieb Oliver Freyermuth: >> Am 04.09.19 um 21:07 schrieb Chris Murphy: >>> On Wed, Sep 4, 2019 at 12:24 PM Remi Gauvin <remi@georgianit.com> wrote: >>>> On 2019-09-04 1:36 p.m., Chris Murphy wrote: >>>> >>>>> I don't really know how snapper works. >>>>> >>>>> The way 'btrfs subvolume snapshot' works, you must point it to a >>>>> subvolume. It won't snapshot a regular directory and from what you >>>>> posted above, there are no subvolumes in /var or /var/lib which means >>>>> trying to snapshot /var/lib/ceph/osd/ceph-.... would fail. So maybe >>>>> it's failing but snapper doesn't show the error. I'm not really sure. >>>>> >>>> In this case, his snapshots are all of the root. >>>> >>>> I don't know how Ceph works, but since we already confirmed that there >>>> are no subvolumes under /var, the only other explanation is that >>>> /var/lib/ceph/osd/ceph-<n> is a submount >>>> >>>> What is the the result of running: >>>> mount | grep /var >>>> >>> Yep. >>> >>> >> Looking at Thomas' mail on the Ceph-users list: >> http://lists.ceph.com/pipermail/ceph-users-ceph.com/2019-August/036679.html >> I deduce he is using Ceph with the Bluestore backend, which indeed means that /var/lib/ceph/osd/ceph.<n> >> are tmpfs mounts which are completeley ephemeral and are created from LVM metadata of the actual Ceph OSD disks. >> >> That would of course also explain why they are not part of any btrfs snapshot of / (and that also means there is no need to backup anything). > ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: No files in snapshot 2019-08-27 22:24 ` Chris Murphy 2019-08-28 14:45 ` Thomas Schneider @ 2019-09-04 9:24 ` Thomas Schneider 1 sibling, 0 replies; 10+ messages in thread From: Thomas Schneider @ 2019-09-04 9:24 UTC (permalink / raw) To: Chris Murphy; +Cc: Btrfs BTRFS Can you share any update on this issue? ------------------------------- Hi, I was thinking of this, too. But it does not apply. root@ld5507:~# btrfs su list -to /var/lib ID gen top level path -- --- --------- ---- root@ld5507:~# btrfs su list -to /var ID gen top level path -- --- --------- ---- And there are files in other directories: root@ld5507:~# ls -l /.snapshots/158/snapshot/var/lib/ceph/mgr/ceph-ld5507/ insgesamt 4 -rw-r--r-- 1 ceph ceph 61 Mai 28 14:33 keyring root@ld5507:~# ls -l /.snapshots/158/snapshot/var/lib/ceph/mon/ceph-ld5507/ insgesamt 12 -rw------- 1 ceph ceph 77 Mai 28 14:33 keyring -rw-r--r-- 1 ceph ceph 8 Mai 28 14:33 kv_backend -rw-r--r-- 1 ceph ceph 3 Aug 23 09:41 min_mon_release drwxr-xr-x 1 ceph ceph 244 Aug 26 18:37 store.db Only this directories /.snapshots/158/snapshot/var/lib/ceph/osd/ceph-<id>/ are empty: root@ld5507:~# ls -l /.snapshots/158/snapshot/var/lib/ceph/osd/ceph-219/ insgesamt 0 To create a snapshot I run this command: snapper create --type single --description "validate /var/lib/ceph/osd/ceph-<n>" Am 28.08.2019 um 00:24 schrieb Chris Murphy: > On Tue, Aug 27, 2019 at 3:33 AM Thomas Schneider <74cmonty@gmail.com> > wrote: >> However, I run into an issue and need to restore various files. >> >> I thought that I could simply take the files from a snapshot created >> before. >> However, the files required don't exist in any snapshot! >> >> Therefore I have created a new snapshot manually to verify if the files >> will be included, but there's nothing. > Snapshots are not recursive on Btrfs. The snapshot will not extend > into nested subvolumes. Check to see if you are snapshotting the > proper subvolume. > > # btrfs sub list -to /var/lib > # btrfs sub list -to /var/ > > In some sense these are redundant, I'm not sure if your /var/lib is a > subvolume or not. Also please include the exact snapshot command > you're making. > ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2019-09-05 18:31 UTC | newest] Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-08-27 9:33 No files in snapshot Thomas Schneider 2019-08-27 22:24 ` Chris Murphy 2019-08-28 14:45 ` Thomas Schneider 2019-09-04 17:36 ` Chris Murphy 2019-09-04 18:02 ` Remi Gauvin 2019-09-04 19:07 ` Chris Murphy 2019-09-04 20:52 ` Oliver Freyermuth 2019-09-05 8:50 ` Thomas Schneider 2019-09-05 18:31 ` Oliver Freyermuth 2019-09-04 9:24 ` Thomas Schneider
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.