All of lore.kernel.org
 help / color / mirror / Atom feed
* how to rollback / to a snapshot ?
@ 2021-05-26 13:05 Lentes, Bernd
  2021-05-26 14:08 ` Remi Gauvin
  0 siblings, 1 reply; 13+ messages in thread
From: Lentes, Bernd @ 2021-05-26 13:05 UTC (permalink / raw)
  To: Btrfs ML

[-- Attachment #1: Type: text/plain, Size: 3576 bytes --]

Hi,

sorry for asking such a simple question, but ...
i didn't find that in the BTRFS wiki and googleing i found tons of ways, but all different, so now i'm more confused than before.
And it's the first time i have to do that.
I have an Ubuntu box where an upgrade didn't complete. System was 16.04 before, now it "thinks" it's an 18.04, but it still uses the kernel from 16.04 ! 
System is up and running. I don't like to use a live linux for the rollback because i'm currently in quarantine and i can't get physically to the box, just with ssh.

This is my setup:

root@pc65472:/boot# mount |grep btrfs
/dev/mapper/vg1-lv_root on / type btrfs (rw,relatime,ssd,space_cache,subvolid=257,subvol=/@)
/dev/mapper/vg1-lv_root on /home type btrfs (rw,relatime,ssd,space_cache,subvolid=258,subvol=/@home)
/dev/sdb1 on /data type btrfs (rw,relatime,space_cache,subvolid=258,subvol=/@data)
/dev/sdc1 on /local type btrfs (rw,relatime,space_cache,subvolid=5,subvol=/)
/dev/mapper/vg1-lv_root on /var/lib/docker/btrfs type btrfs (rw,relatime,ssd,space_cache,subvolid=257,subvol=/@/var/lib/docker/btrfs)
/dev/mapper/vg1-lv_root on /mnt/snap type btrfs (rw,relatime,ssd,space_cache,subvolid=767,subvol=/@apt-snapshot-release-upgrade-bionic-2021-05-21_10:38:10)
/dev/mapper/vg1-lv_root on /mnt/sub/root-volume type btrfs (rw,relatime,ssd,space_cache,subvolid=5,subvol=/)

/home, /data and /local are not involved, and /mnt/snap is the snapshot i want to rollback.

root@pc65472:/boot# btrfs sub list /
ID 257 gen 44758165 top level 5 path @
ID 258 gen 44758165 top level 5 path @home
 ...
ID 767 gen 44757961 top level 5 path @apt-snapshot-release-upgrade-bionic-2021-05-21_10:38:10
ID 768 gen 44757961 top level 5 path @apt-snapshot-release-upgrade-bionic-2021-05-21_10:38:13
ID 769 gen 44757961 top level 5 path @apt-snapshot-release-upgrade-bionic-2021-05-21_10:38:46

Is it sufficient to change the the default subvolume with btrfs sub set-default to the snapshot and reboot ?
Do i need additionally to modify grub and fstab ?

Or just modify grub and fstab, let it point to the snapshot and reboot ?

What is with mounting the snapshot and rsync it to the current / ?
I think that's not possible in a running system which has / mounted.

In some documents i read to create a snapshot from the snapshot i want rollback to, if something goes wrong.

/boot is on a ext3-partition, it is not involved.


Bernd


-- 

Bernd Lentes 
System Administrator 
Institute for Metabolism and Cell Death (MCD) 
Building 25 - office 122 
HelmholtzZentrum München 
bernd.lentes@helmholtz-muenchen.de 
phone: +49 89 3187 1241 
phone: +49 89 3187 3827 
fax: +49 89 3187 2294 
http://www.helmholtz-muenchen.de/mcd 


Public key: 

30 82 01 0a 02 82 01 01 00 b3 72 3e ce 2c 0a 6f 58 49 2c 92 23 c7 b9 c1 ff 6c 3a 53 be f7 9e e9 24 b7 49 fa 3c e8 de 28 85 2c d3 ed f7 70 03 3f 4d 82 fc cc 96 4f 18 27 1f df 25 b3 13 00 db 4b 1d ec 7f 1b cf f9 cd e8 5b 1f 11 b3 a7 48 f8 c8 37 ed 41 ff 18 9f d7 83 51 a9 bd 86 c2 32 b3 d6 2d 77 ff 32 83 92 67 9e ae ae 9c 99 ce 42 27 6f bf d8 c2 a1 54 fd 2b 6b 12 65 0e 8a 79 56 be 53 89 70 51 02 6a eb 76 b8 92 25 2d 88 aa 57 08 42 ef 57 fb fe 00 71 8e 90 ef b2 e3 22 f3 34 4f 7b f1 c4 b1 7c 2f 1d 6f bd c8 a6 a1 1f 25 f3 e4 4b 6a 23 d3 d2 fa 27 ae 97 80 a3 f0 5a c4 50 4a 45 e3 45 4d 82 9f 8b 87 90 d0 f9 92 2d a7 d2 67 53 e6 ae 1e 72 3e e9 e0 c9 d3 1c 23 e0 75 78 4a 45 60 94 f8 e3 03 0b 09 85 08 d0 6c f3 ff ce fa 50 25 d9 da 81 7b 2a dc 9e 28 8b 83 04 b4 0a 9f 37 b8 ac 58 f1 38 43 0e 72 af 02 03 01 00 01

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 2217 bytes --]

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: how to rollback / to a snapshot ?
  2021-05-26 13:05 how to rollback / to a snapshot ? Lentes, Bernd
@ 2021-05-26 14:08 ` Remi Gauvin
  2021-05-26 14:53   ` Lentes, Bernd
  0 siblings, 1 reply; 13+ messages in thread
From: Remi Gauvin @ 2021-05-26 14:08 UTC (permalink / raw)
  To: Lentes, Bernd, linux-btrfs

I'm going to offer my perspective in the hope that it helps. I would not
be comfortable doing this if I could not get physical access with boot
media in case I make a mistake and need to fix hands on.

On 2021-05-26 9:05 a.m., Lentes, Bernd wrote:

> This is my setup:
> 
> root@pc65472:/boot# mount |grep btrfs
> /dev/mapper/vg1-lv_root on / type btrfs (rw,relatime,ssd,space_cache,subvolid=257,subvol=/@)
> /dev/mapper/vg1-lv_root on /home type btrfs (rw,relatime,ssd,space_cache,subvolid=258,subvol=/@home)
> /dev/sdb1 on /data type btrfs (rw,relatime,space_cache,subvolid=258,subvol=/@data)
> /dev/sdc1 on /local type btrfs (rw,relatime,space_cache,subvolid=5,subvol=/)
> /dev/mapper/vg1-lv_root on /var/lib/docker/btrfs type btrfs (rw,relatime,ssd,space_cache,subvolid=257,subvol=/@/var/lib/docker/btrfs)
> /dev/mapper/vg1-lv_root on /mnt/snap type btrfs (rw,relatime,ssd,space_cache,subvolid=767,subvol=/@apt-snapshot-release-upgrade-bionic-2021-05-21_10:38:10)
> /dev/mapper/vg1-lv_root on /mnt/sub/root-volume type btrfs (rw,relatime,ssd,space_cache,subvolid=5,subvol=/)
> 
> /home, /data and /local are not involved, and /mnt/snap is the snapshot i want to rollback.
> 
> root@pc65472:/boot# btrfs sub list /
> ID 257 gen 44758165 top level 5 path @
> ID 258 gen 44758165 top level 5 path @home
>  ...
> ID 767 gen 44757961 top level 5 path @apt-snapshot-release-upgrade-bionic-2021-05-21_10:38:10
> ID 768 gen 44757961 top level 5 path @apt-snapshot-release-upgrade-bionic-2021-05-21_10:38:13
> ID 769 gen 44757961 top level 5 path @apt-snapshot-release-upgrade-bionic-2021-05-21_10:38:46
> 
> Is it sufficient to change the the default subvolume with btrfs sub set-default to the snapshot and reboot ?

You should probably ignore all the documentation you find on net about
changing your default subvolume.  Those basically come from early
implementations of btrfs before it was common to put root in the '@'
subvolume.

The apt plugin that created those snapshots probably has it's own revert
process that I know nothing about.  So here's how I would do it manually.

Step 1: List the subvolumes that are directly in your @, those will have
to be moved or snapshoted back into the new root.

# btrfs sub list -o /


Hopefully, /@/var/lib/docker/btrfs is the only one, and you can just
move that whole subvolume

# mv /mnt/sub/root-volume/@  /mnt/sub/root-volume/@bad

# btrfs sub snap
/mnt/sub/root-volume/@apt-snapshot-release-upgrade-bionic-2021-05-21_10:38:10
/mnt/sub/root-volume/@

I'm assuming that docker itself has subvolumes, (you truncated the
subvolume list.)  If it does not, you can just snapshot it into the new
@.  If it does have subvolumes, you have to move the whole thing.  Only
do this to a directory/subvolume found with the sub list -o command.


mv /mnt/sub/root-volume/@bad/var/lib/docker/btrfs
/mnt/sub/root-volume/@/var/lib/docker/btrfs

You need to repeat this for any other subvolumes found in the first sub
list command.

Reboot


> Do i need additionally to modify grub and fstab ?
> 

No... once the subvolume you want as your new root is named "@", grub
and fstab will be correct.

> Or just modify grub and fstab, let it point to the snapshot and reboot ?

This path lies madness

> 
> What is with mounting the snapshot and rsync it to the current / ?
> I think that's not possible in a running system which has / mounted.

Seems a very wasteful approach


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: how to rollback / to a snapshot ?
  2021-05-26 14:08 ` Remi Gauvin
@ 2021-05-26 14:53   ` Lentes, Bernd
  2021-05-26 15:05     ` Remi Gauvin
  2021-05-26 15:17     ` Remi Gauvin
  0 siblings, 2 replies; 13+ messages in thread
From: Lentes, Bernd @ 2021-05-26 14:53 UTC (permalink / raw)
  To: Remi Gauvin; +Cc: Btrfs ML

[-- Attachment #1: Type: text/plain, Size: 7228 bytes --]



----- On May 26, 2021, at 4:08 PM, Remi Gauvin remi@georgianit.com wrote:

> On 2021-05-26 9:05 a.m., Lentes, Bernd wrote:
 
> Step 1: List the subvolumes that are directly in your @, those will have
> to be moved or snapshoted back into the new root.
> 
> # btrfs sub list -o /
> 
> 
> Hopefully, /@/var/lib/docker/btrfs is the only one, and you can just
> move that whole subvolume
> 
> # mv /mnt/sub/root-volume/@  /mnt/sub/root-volume/@bad
> 
> # btrfs sub snap
> /mnt/sub/root-volume/@apt-snapshot-release-upgrade-bionic-2021-05-21_10:38:10
> /mnt/sub/root-volume/@
> 
> I'm assuming that docker itself has subvolumes, (you truncated the
> subvolume list.)  If it does not, you can just snapshot it into the new
> @.  If it does have subvolumes, you have to move the whole thing.  Only
> do this to a directory/subvolume found with the sub list -o command.
> 

It has:

root@pc65472:/mnt/sub/root-volume/@apt-snapshot-release-upgrade-bionic-2021-05-21_10:38:10# btrfs sub list -o /
ID 724 gen 44757961 top level 257 path @/var/lib/docker/btrfs/subvolumes/861e224ff669e010d4f88459fcef35a53a13f06a0bae9738c5109442e25d22ef
ID 725 gen 44757961 top level 257 path @/var/lib/docker/btrfs/subvolumes/e29c8a1388d14e5c609ec699ee646bc435d41f1c856c6622c402941d9a462001-init
ID 726 gen 44757961 top level 257 path @/var/lib/docker/btrfs/subvolumes/e29c8a1388d14e5c609ec699ee646bc435d41f1c856c6622c402941d9a462001
ID 727 gen 44757961 top level 257 path @/var/lib/docker/btrfs/subvolumes/0d604657c3d1de047d0e25592701d580c5e25ba80efc7957317d50448e3292fe
ID 728 gen 44757961 top level 257 path @/var/lib/docker/btrfs/subvolumes/14eb3fffe7da6aff12182b49b28bc4632fdf21a8045f532b5819375f8405f5b2
ID 729 gen 44757961 top level 257 path @/var/lib/docker/btrfs/subvolumes/b3620e2dad03b2db17c01e27eb005e054948c2deec480c70a6a399802f06cea9
ID 730 gen 44757961 top level 257 path @/var/lib/docker/btrfs/subvolumes/0e76b1fe113e0801762bb19f444fa911431c309c7f07796866b152be93d7c231
ID 731 gen 44757961 top level 257 path @/var/lib/docker/btrfs/subvolumes/d06aa183056d47960c53beffd7f1e4765bdda7457b9d80eb5253ae24a41fed11
ID 732 gen 44757961 top level 257 path @/var/lib/docker/btrfs/subvolumes/2cc3bd6cc3ba6f7b6d59f24d0a1341094dbe85f6378ca77e989efc0072db574f
ID 733 gen 44757961 top level 257 path @/var/lib/docker/btrfs/subvolumes/5caa7d6a8b6ed674c90eaaa381f9c5f14a16921d5a06a35c3d83ebf330a8e68f
ID 734 gen 44757961 top level 257 path @/var/lib/docker/btrfs/subvolumes/2f953e73e108714c7f9f3b3b95358ef4352fc29df4372cf7dfec91267bc70954
ID 735 gen 44757961 top level 257 path @/var/lib/docker/btrfs/subvolumes/df83a5c69bee2d0b06695bf2645b79b93611beeb8aa1a157dfb5ac4253a30928
ID 736 gen 44757961 top level 257 path @/var/lib/docker/btrfs/subvolumes/e6197fb42e2a624e23a41c0e7cd86d5b5c65b420dfd3d6e1a373c716e79d4311
ID 737 gen 44757961 top level 257 path @/var/lib/docker/btrfs/subvolumes/ededd3b8d9c0b39f2f2dba0102eec282e134a7cef1d3218e3a0c4266f271396e
ID 738 gen 44757961 top level 257 path @/var/lib/docker/btrfs/subvolumes/56d73f1207841c59cefc190f380eb62516983e25e0e9806b8166c790d999592c
ID 739 gen 44757961 top level 257 path @/var/lib/docker/btrfs/subvolumes/f0f3f683697dce3ae18304be732d9cfc3f0aa663d90dca7d351505c15989d9e6
ID 740 gen 44757961 top level 257 path @/var/lib/docker/btrfs/subvolumes/a9f458390614c253b7d06bbf82da48387220ce455693286bb92ea8373d8c2df3
ID 741 gen 44757961 top level 257 path @/var/lib/docker/btrfs/subvolumes/9042f96c1bd673d3ba0ca54b9ed82111fbd979338bc94201e5a78aedf4e3e46b
ID 742 gen 44757961 top level 257 path @/var/lib/docker/btrfs/subvolumes/5e88b78a7a177a1bc902fb0e864d6214777b536c509602552845f3a6e5c70442
ID 743 gen 44757961 top level 257 path @/var/lib/docker/btrfs/subvolumes/3b58a0c99f6345599c3a8295e49cbed58b5c48e5e9c55a0580c1c290db38e7a5
ID 744 gen 44757962 top level 257 path @/var/lib/docker/btrfs/subvolumes/2609ce598882f98706cdfecdcec4111d7efc41bb7499190f75f62409b3854ad2
ID 745 gen 44757962 top level 257 path @/var/lib/docker/btrfs/subvolumes/50f1708e7da7690edb8925e9b35417d052681a22c71b45f4ab11a215e9ddae7e
ID 746 gen 44757962 top level 257 path @/var/lib/docker/btrfs/subvolumes/eb58f0f899885e4b30a2396ab7af6cb0e64a3c28120cd7d08a19bbbc7d6fc244
ID 747 gen 44757962 top level 257 path @/var/lib/docker/btrfs/subvolumes/a12aa99c06a1851d27811df8494c1e8d45f84148a2089296789ae6621b0cfa0f
ID 748 gen 44757962 top level 257 path @/var/lib/docker/btrfs/subvolumes/e1509bdf7aa715f881a651cf3e052081641513c8be4eaad9a9fb30430228ca2d
ID 749 gen 44757962 top level 257 path @/var/lib/docker/btrfs/subvolumes/596b5dce434692b4c2b9c571fcb23c47514ef43bfeb4c8bbde8a0c50fc19aadf
ID 750 gen 44757962 top level 257 path @/var/lib/docker/btrfs/subvolumes/4e92db3bb0240df49da798dc8297d644ddb7d608e10a0f4ee7bae7320a36e5d8
ID 751 gen 44757962 top level 257 path @/var/lib/docker/btrfs/subvolumes/169072a0c2c1b4daa16f9cd4be60c7e63dfc2a5fb4198d0ea71f3a25c91bab8e
ID 752 gen 44757962 top level 257 path @/var/lib/docker/btrfs/subvolumes/f98753db60820b0f45201fb1d76544ccce19cddd5a6acc62e4acdef5a8b60ba2
ID 753 gen 44757962 top level 257 path @/var/lib/docker/btrfs/subvolumes/145e9aa53bf55045ad73b4d230883be59da59471101f94eada05c0c21db81e6c
ID 754 gen 44757962 top level 257 path @/var/lib/docker/btrfs/subvolumes/d349cc5329b14b3cc6550ed43c94ae2445c441c6463cb090fb533860102da0fd
ID 755 gen 44757962 top level 257 path @/var/lib/docker/btrfs/subvolumes/ef7367319f3cea933a873cc94d9a6db0e4108fe5b72ea361e6b42097baff6664
ID 756 gen 44757962 top level 257 path @/var/lib/docker/btrfs/subvolumes/31bcf7fce390941ab6c335ebcc117b40a011c28849ee63c5d1eb9487b2e83120
ID 757 gen 44757962 top level 257 path @/var/lib/docker/btrfs/subvolumes/94bcd2a480c89bd2226af9a3b7dd1a645fe5c1ba5a363d6a9542de63fea3d2fc
ID 758 gen 44757962 top level 257 path @/var/lib/docker/btrfs/subvolumes/cb31c7297ff761a298ecbcefe0ab47314d479b3200ab6af30a4c1b77503e9d37
ID 759 gen 44757962 top level 257 path @/var/lib/docker/btrfs/subvolumes/f73cb52db8e0bc5a3356a5284d7f733b9b7ba2f6494617fdb0a3004691d080b2
ID 760 gen 44757962 top level 257 path @/var/lib/docker/btrfs/subvolumes/65416a53d7da4949897c45d390802d24f2fae704d9a64176fd05d8241ed3aacd
ID 761 gen 44757962 top level 257 path @/var/lib/docker/btrfs/subvolumes/183e345921700ffea88943d1d1fdc8958038245f145cb78dbf0b292059546c3e
ID 762 gen 44757962 top level 257 path @/var/lib/docker/btrfs/subvolumes/22ffacbd4e0c0778d1610da9637f2b6058375fff58f07035a431258f0aec49ab
ID 763 gen 44757962 top level 257 path @/var/lib/docker/btrfs/subvolumes/d4eccdcd6659628ea02834929f6187e0c60fdbcb950a2b498973c4a806ed90f3
ID 764 gen 44757962 top level 257 path @/var/lib/docker/btrfs/subvolumes/b85f5e75207d899700515ebada8dcec8dd8f38f268da9bb382835f3101243d95
ID 765 gen 44757962 top level 257 path @/var/lib/docker/btrfs/subvolumes/5f66e708001a623ae8f219fcbf5a9a9b020bc12403c2cb9ad788880caa79b912-init
ID 766 gen 44757962 top level 257 path @/var/lib/docker/btrfs/subvolumes/5f66e708001a623ae8f219fcbf5a9a9b020bc12403c2cb9ad788880caa79b912


> 
> mv /mnt/sub/root-volume/@bad/var/lib/docker/btrfs
> /mnt/sub/root-volume/@/var/lib/docker/btrfs
> 
> You need to repeat this for any other subvolumes found in the first sub
> list command.

OK. What is their purpose ?
> 
> Reboot

OK. Thanks, i will try that and keep you informed.

Bernd

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 2217 bytes --]

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: how to rollback / to a snapshot ?
  2021-05-26 14:53   ` Lentes, Bernd
@ 2021-05-26 15:05     ` Remi Gauvin
  2021-05-26 15:17     ` Remi Gauvin
  1 sibling, 0 replies; 13+ messages in thread
From: Remi Gauvin @ 2021-05-26 15:05 UTC (permalink / raw)
  To: Lentes, Bernd, linux-btrfs

On 2021-05-26 10:53 a.m., Lentes, Bernd wrote:

>>
>> mv /mnt/sub/root-volume/@bad/var/lib/docker/btrfs
>> /mnt/sub/root-volume/@/var/lib/docker/btrfs
>>
>> You need to repeat this for any other subvolumes found in the first sub
>> list command.
> 
> OK. What is their purpose ?


The snapshot of a subvolume does not contain any other subvolumes.

So if you are replacing root from a snapshot, any of the other
subvolumes that were within root have to be moved from the old to the new.



^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: how to rollback / to a snapshot ?
  2021-05-26 14:53   ` Lentes, Bernd
  2021-05-26 15:05     ` Remi Gauvin
@ 2021-05-26 15:17     ` Remi Gauvin
  2021-05-27 19:37       ` Lentes, Bernd
  1 sibling, 1 reply; 13+ messages in thread
From: Remi Gauvin @ 2021-05-26 15:17 UTC (permalink / raw)
  To: Lentes, Bernd; +Cc: Btrfs ML

On 2021-05-26 10:53 a.m., Lentes, Bernd wrote:

> 
> OK. Thanks, i will try that and keep you informed.
> 
> Bernd
> 

Please note that I made a mistake.


When you snapshot into a new @, it will already have an empty (not
subvolume) @/var/lib/docker/btrfs directory.

You have to get that directory out of the way before you move the
subvolume, otherwise, you'll end up with a  @/var/lib/docker/btrfs/btrfs
and mount will fail.

rmdir /mnt/sub/root-volume/@/var/lib/docker/btrfs

*before* you do:

mv /mnt/sub/root-volume/@bad/var/lib/docker/btrfs
/mnt/sub/root-volume/@/var/lib/docker/btrfs

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: how to rollback / to a snapshot ?
  2021-05-26 15:17     ` Remi Gauvin
@ 2021-05-27 19:37       ` Lentes, Bernd
  2021-05-27 19:43         ` Lentes, Bernd
  2021-05-27 20:41         ` Remi Gauvin
  0 siblings, 2 replies; 13+ messages in thread
From: Lentes, Bernd @ 2021-05-27 19:37 UTC (permalink / raw)
  To: Remi Gauvin, Btrfs ML

[-- Attachment #1: Type: text/plain, Size: 1508 bytes --]


----- On May 26, 2021, at 5:17 PM, Remi Gauvin remi@georgianit.com wrote:

> On 2021-05-26 10:53 a.m., Lentes, Bernd wrote:
> 
>> 
>> OK. Thanks, i will try that and keep you informed.
>> 
>> Bernd
>> 
> 
> Please note that I made a mistake.
> 
> 
> When you snapshot into a new @, it will already have an empty (not
> subvolume) @/var/lib/docker/btrfs directory.
> 
> You have to get that directory out of the way before you move the
> subvolume, otherwise, you'll end up with a  @/var/lib/docker/btrfs/btrfs
> and mount will fail.
> 
> rmdir /mnt/sub/root-volume/@/var/lib/docker/btrfs
> 
> *before* you do:
> 
> mv /mnt/sub/root-volume/@bad/var/lib/docker/btrfs
> /mnt/sub/root-volume/@/var/lib/docker/btrfs

Hi,

i followed your guide and tried a reboot remotely.
PC stuck in BIOS, someone in the office pressed F2, and system booted completely.
But unfortunately in the bad system and without X:

root@pc65472:~# mount|grep btrfs
/dev/mapper/vg1-lv_root on / type btrfs (rw,relatime,ssd,space_cache,subvolid=257,subvol=/@_bad)
/dev/mapper/vg1-lv_root on /home type btrfs (rw,relatime,ssd,space_cache,subvolid=258,subvol=/@home)
/dev/sdb1 on /data type btrfs (rw,relatime,space_cache,subvolid=258,subvol=/@data)
/dev/sdc1 on /local type btrfs (rw,relatime,space_cache,subvolid=5,subvol=/)
/dev/mapper/vg1-lv_root on /var/lib/docker/btrfs type btrfs (rw,relatime,ssd,space_cache,subvolid=257,subvol=/@_bad/var/lib/docker/btrfs)

Any idea ?

I will try tomorrow again, then i'm in front of the PC.

Bernd

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 2217 bytes --]

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: how to rollback / to a snapshot ?
  2021-05-27 19:37       ` Lentes, Bernd
@ 2021-05-27 19:43         ` Lentes, Bernd
  2021-05-27 20:41         ` Remi Gauvin
  1 sibling, 0 replies; 13+ messages in thread
From: Lentes, Bernd @ 2021-05-27 19:43 UTC (permalink / raw)
  To: Remi Gauvin, Btrfs ML

[-- Attachment #1: Type: text/plain, Size: 1210 bytes --]



----- On May 27, 2021, at 9:37 PM, Bernd Lentes bernd.lentes@helmholtz-muenchen.de wrote:

> ----- On May 26, 2021, at 5:17 PM, Remi Gauvin remi@georgianit.com wrote:
> 

> Hi,
> 
> i followed your guide and tried a reboot remotely.
> PC stuck in BIOS, someone in the office pressed F2, and system booted
> completely.
> But unfortunately in the bad system and without X:
> 
> root@pc65472:~# mount|grep btrfs
> /dev/mapper/vg1-lv_root on / type btrfs
> (rw,relatime,ssd,space_cache,subvolid=257,subvol=/@_bad)
> /dev/mapper/vg1-lv_root on /home type btrfs
> (rw,relatime,ssd,space_cache,subvolid=258,subvol=/@home)
> /dev/sdb1 on /data type btrfs
> (rw,relatime,space_cache,subvolid=258,subvol=/@data)
> /dev/sdc1 on /local type btrfs (rw,relatime,space_cache,subvolid=5,subvol=/)
> /dev/mapper/vg1-lv_root on /var/lib/docker/btrfs type btrfs
> (rw,relatime,ssd,space_cache,subvolid=257,subvol=/@_bad/var/lib/docker/btrfs)
> 
> Any idea ?
> 
> I will try tomorrow again, then i'm in front of the PC.
> 
> Bernd

journalctl -xb:

 ...
Mai 27 09:51:38 pc65472 kernel: BTRFS error (device dm-0): could not find root 8
Mai 27 09:51:38 pc65472 kernel: BTRFS error (device dm-0): could not find root 8
 ...

Bernd

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 2217 bytes --]

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: how to rollback / to a snapshot ?
  2021-05-27 19:37       ` Lentes, Bernd
  2021-05-27 19:43         ` Lentes, Bernd
@ 2021-05-27 20:41         ` Remi Gauvin
  2021-05-28 14:33           ` Lentes, Bernd
  1 sibling, 1 reply; 13+ messages in thread
From: Remi Gauvin @ 2021-05-27 20:41 UTC (permalink / raw)
  To: Lentes, Bernd; +Cc: linux-btrfs

On 2021-05-27 3:37 p.m., Lentes, Bernd wrote:

> 
> i followed your guide and tried a reboot remotely.
> PC stuck in BIOS, someone in the office pressed F2, and system booted completely.
> But unfortunately in the bad system and without X:
> 
> root@pc65472:~# mount|grep btrfs
> /dev/mapper/vg1-lv_root on / type btrfs (rw,relatime,ssd,space_cache,subvolid=257,subvol=/@_bad)

This is interesting, because your system is mounting the old subvolume
as root and not the new @.

Check the /etc/fstab, (maybe it would be wise to post it here.).  Maybe
at some point, someone replaced the default 'subvolume=@' with a
subvolid=257?

If that's the case, you should also check the /boot/grub/grub.cfg and
verify that the kernel boot options specify: rootflags=subvol=@

That doesn't really explain why X wouldn't be working.,,, or why
pressing F2 did anything,, (that would indicated the Bios is
experiencing some kind of error?)


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: how to rollback / to a snapshot ?
  2021-05-27 20:41         ` Remi Gauvin
@ 2021-05-28 14:33           ` Lentes, Bernd
  2021-05-28 14:56             ` Remi Gauvin
  0 siblings, 1 reply; 13+ messages in thread
From: Lentes, Bernd @ 2021-05-28 14:33 UTC (permalink / raw)
  To: Remi Gauvin; +Cc: Btrfs ML

[-- Attachment #1: Type: text/plain, Size: 3479 bytes --]



----- On May 27, 2021, at 10:41 PM, Remi Gauvin remi@georgianit.com wrote:

> On 2021-05-27 3:37 p.m., Lentes, Bernd wrote:
> 
>> 
>> i followed your guide and tried a reboot remotely.
>> PC stuck in BIOS, someone in the office pressed F2, and system booted
>> completely.
>> But unfortunately in the bad system and without X:
>> 
>> root@pc65472:~# mount|grep btrfs
>> /dev/mapper/vg1-lv_root on / type btrfs
>> (rw,relatime,ssd,space_cache,subvolid=257,subvol=/@_bad)
> 
> This is interesting, because your system is mounting the old subvolume
> as root and not the new @.
> 
> Check the /etc/fstab, (maybe it would be wise to post it here.).  Maybe
> at some point, someone replaced the default 'subvolume=@' with a
> subvolid=257?

root@pc65472:~# less /etc/fstab
 ...
/dev/mapper/vg1-lv_root /               btrfs   defaults,subvol=@ 0       1
/dev/mapper/vg1-lv_root /home           btrfs   defaults,subvol=@home 0       2
/dev/sdb1               /data           btrfs   defaults,subvol=@data 0       2
UUID=68dc5e68-fc44-4163-bd75-537adf9f1266 /local          btrfs   defaults        0       0

seems to be ok

> 
> If that's the case, you should also check the /boot/grub/grub.cfg and
> verify that the kernel boot options specify: rootflags=subvol=@

root@pc65472:~# grep -i '@_bad' /boot/grub/grub.cfg
    font="/@_bad/usr/share/grub/unicode.pf2"
        linux   /vmlinuz-4.4.0-66-generic root=/dev/mapper/vg1-lv_root ro rootflags=subvol=@_bad  splash=verbose   <=== !!!!
                linux   /vmlinuz-4.4.0-66-generic root=/dev/mapper/vg1-lv_root ro rootflags=subvol=@_bad  splash=verbose   <=== !!!
                linux   /vmlinuz-4.4.0-66-generic root=/dev/mapper/vg1-lv_root ro recovery nomodeset dis_ucode_ldr rootflags=subvol=@_bad

I'm hesitating to manipulate grub.cfg directly, because of:

#
# DO NOT EDIT THIS FILE
#
# It is automatically generated by grub-mkconfig using templates
# from /etc/grub.d and settings from /etc/default/grub
#


interesting:
root@pc65472:/etc/grub.d# grep -i rootflags /etc/grub.d/*
/etc/grub.d/10_linux:       GRUB_CMDLINE_LINUX="rootflags=subvol=${rootsubvol} ${GRUB_CMDLINE_LINUX}"    <=== the culprit ?


/etc/grub.d/10_linux:
 ...
case x"$GRUB_FS" in
    xbtrfs)
        rootsubvol="`make_system_path_relative_to_its_root /`"
        rootsubvol="${rootsubvol#/}"
        if [ "x${rootsubvol}" != x ]; then
            GRUB_CMDLINE_LINUX="rootflags=subvol=${rootsubvol} ${GRUB_CMDLINE_LINUX}"
        fi;;
 ...

grub-mkconfig_lib: (is sourced in /etc/grub.d/10_linux)
make_system_path_relative_to_its_root ()
{
  "${grub_mkrelpath}" "$1"
}

if test "x$grub_mkrelpath" = x; then
  grub_mkrelpath="${bindir}/grub-mkrelpath"
fi

There is a binary "grub-mkrelpath":

root@pc65472:~# which grub-mkrelpath
/usr/bin/grub-mkrelpath
root@pc65472:~# grub-mkrelpath -?
Usage: grub-mkrelpath [OPTION...] PATH
Transform a system filename into GRUB one.

  -?, --help                 give this help list
      --usage                give a short usage message
  -V, --version              print program version

root@pc65472:~#
root@pc65472:~# grub-mkrelpath /
/@_bad  <=== !!!

grub-mkrelpath gives @_bad back, that's the problem. What can i do ?

> That doesn't really explain why X wouldn't be working.,,, or why
> pressing F2 did anything,, (that would indicated the Bios is
> experiencing some kind of error?)

Problem with the BIOS existed already. Problem with X is not severe.
Important is to rollback to the snapshot.

Bernd

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 2217 bytes --]

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: how to rollback / to a snapshot ?
  2021-05-28 14:33           ` Lentes, Bernd
@ 2021-05-28 14:56             ` Remi Gauvin
  2021-05-28 15:36               ` Lentes, Bernd
  0 siblings, 1 reply; 13+ messages in thread
From: Remi Gauvin @ 2021-05-28 14:56 UTC (permalink / raw)
  To: Lentes, Bernd; +Cc: linux-btrfs

On 2021-05-28 10:33 a.m., Lentes, Bernd wrote:

>>
>> If that's the case, you should also check the /boot/grub/grub.cfg and
>> verify that the kernel boot options specify: rootflags=subvol=@
> 
> root@pc65472:~# grep -i '@_bad' /boot/grub/grub.cfg
>     font="/@_bad/usr/share/grub/unicode.pf2"
>         linux   /vmlinuz-4.4.0-66-generic root=/dev/mapper/vg1-lv_root ro rootflags=subvol=@_bad  splash=verbose   <=== !!!!
>                 linux   /vmlinuz-4.4.0-66-generic root=/dev/mapper/vg1-lv_root ro rootflags=subvol=@_bad  splash=verbose   <=== !!!
>                 linux   /vmlinuz-4.4.0-66-generic root=/dev/mapper/vg1-lv_root ro recovery nomodeset dis_ucode_ldr rootflags=subvol=@_bad
> 
> I'm hesitating to manipulate grub.cfg directly, because of:
> 
> #
> # DO NOT EDIT THIS FILE
> #
> # It is automatically generated by grub-mkconfig using templates
> # from /etc/grub.d and settings from /etc/default/grub
> #

It's relatively safe to edit the grub.cfg directly... Just be aware that
next time something runs update-grub, your changes will not persist.

But here, the question I'm left with, is why your grub.cfg changed
between the time you renamed the subvolumes and rebooted... You might
have something that does, in fact, execute an update-grub on shutdown?

You can try editing the grub.cfg and reboot and see if that fixes the
problem... but if it reverts to @_bad, you'll either have to
find/disable the script that is updating grub, or change it at boot time
interactively, (which means, needs boots on the floor)



^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: how to rollback / to a snapshot ?
  2021-05-28 14:56             ` Remi Gauvin
@ 2021-05-28 15:36               ` Lentes, Bernd
  2021-05-28 15:55                 ` Remi Gauvin
  0 siblings, 1 reply; 13+ messages in thread
From: Lentes, Bernd @ 2021-05-28 15:36 UTC (permalink / raw)
  To: Remi Gauvin; +Cc: Btrfs ML

[-- Attachment #1: Type: text/plain, Size: 2032 bytes --]



----- On May 28, 2021, at 4:56 PM, Remi Gauvin remi@georgianit.com wrote:

> On 2021-05-28 10:33 a.m., Lentes, Bernd wrote:
> 
>>>
>>> If that's the case, you should also check the /boot/grub/grub.cfg and
>>> verify that the kernel boot options specify: rootflags=subvol=@
>> 
>> root@pc65472:~# grep -i '@_bad' /boot/grub/grub.cfg
>>     font="/@_bad/usr/share/grub/unicode.pf2"
>>         linux   /vmlinuz-4.4.0-66-generic root=/dev/mapper/vg1-lv_root ro
>>         rootflags=subvol=@_bad  splash=verbose   <=== !!!!
>>                 linux   /vmlinuz-4.4.0-66-generic root=/dev/mapper/vg1-lv_root ro
>>                 rootflags=subvol=@_bad  splash=verbose   <=== !!!
>>                 linux   /vmlinuz-4.4.0-66-generic root=/dev/mapper/vg1-lv_root ro recovery
>>                 nomodeset dis_ucode_ldr rootflags=subvol=@_bad
>> 
>> I'm hesitating to manipulate grub.cfg directly, because of:
>> 
>> #
>> # DO NOT EDIT THIS FILE
>> #
>> # It is automatically generated by grub-mkconfig using templates
>> # from /etc/grub.d and settings from /etc/default/grub
>> #
> 
> It's relatively safe to edit the grub.cfg directly... Just be aware that
> next time something runs update-grub, your changes will not persist.
> 
> But here, the question I'm left with, is why your grub.cfg changed
> between the time you renamed the subvolumes and rebooted... You might
> have something that does, in fact, execute an update-grub on shutdown?
> 

I don't think so ...

> You can try editing the grub.cfg and reboot and see if that fixes the
> problem... but if it reverts to @_bad, you'll either have to
> find/disable the script that is updating grub, or change it at boot time
> interactively, (which means, needs boots on the floor)

I try to follow your guide another time, and just before the reboot i will check
grub-mkrelpath to see if it has changed to @. If yes i will start grub-mkconfig, so that @ is set in the rootflags.
I will also check /boot/grub/grub.cfg.

I will keep you informed.

Thanks for your great help.

Bernd

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 2217 bytes --]

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: how to rollback / to a snapshot ?
  2021-05-28 15:36               ` Lentes, Bernd
@ 2021-05-28 15:55                 ` Remi Gauvin
  2021-05-31 15:22                   ` Lentes, Bernd
  0 siblings, 1 reply; 13+ messages in thread
From: Remi Gauvin @ 2021-05-28 15:55 UTC (permalink / raw)
  To: Lentes, Bernd; +Cc: linux-btrfs

On 2021-05-28 11:36 a.m., Lentes, Bernd wrote:
> 

> I try to follow your guide another time, and just before the reboot i will check
> grub-mkrelpath to see if it has changed to @. If yes i will start grub-mkconfig, so that @ is set in the rootflags.
> I will also check /boot/grub/grub.cfg.
> 

The grub-mkrelpath will be @_bad so long as are still booted in that
subvolume,,, but grub.cfg should not change on reboot.. Ie, it it was
subvol=@ when we started, it should still be the same when rebooting.

Something changed it when you rebooted while following the steps I
outlined. (Or there was some kind of kernel related package updates
running at the time you were doing this.)

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: how to rollback / to a snapshot ?
  2021-05-28 15:55                 ` Remi Gauvin
@ 2021-05-31 15:22                   ` Lentes, Bernd
  0 siblings, 0 replies; 13+ messages in thread
From: Lentes, Bernd @ 2021-05-31 15:22 UTC (permalink / raw)
  To: Remi Gauvin; +Cc: Btrfs ML

[-- Attachment #1: Type: text/plain, Size: 1289 bytes --]


----- On May 28, 2021, at 5:55 PM, Remi Gauvin remi@georgianit.com wrote:

> On 2021-05-28 11:36 a.m., Lentes, Bernd wrote:
>> 
> 
>> I try to follow your guide another time, and just before the reboot i will check
>> grub-mkrelpath to see if it has changed to @. If yes i will start grub-mkconfig,
>> so that @ is set in the rootflags.
>> I will also check /boot/grub/grub.cfg.
>> 
> 
> The grub-mkrelpath will be @_bad so long as are still booted in that
> subvolume,,, but grub.cfg should not change on reboot.. Ie, it it was
> subvol=@ when we started, it should still be the same when rebooting.
> 
> Something changed it when you rebooted while following the steps I
> outlined. (Or there was some kind of kernel related package updates
> running at the time you were doing this.)

Hi Remi,

i finally suceeded. My collegue started accidentally an upgrade from 16.04 to 18.04.
This upgrade didn't finished because the disk got full.
First i had to free up some disk space.
Then i saw that during shutdown the upgrade continued. This is surely the culprit for updating my grub.cfg.
Today i booted with a live cd, changed grub.cfg to '@' instead of '@_bad', and afterwards PC bootet into the right configuration without any problem.

Thank you very much for your help. Great !


Bernd

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 2217 bytes --]

^ permalink raw reply	[flat|nested] 13+ messages in thread

end of thread, other threads:[~2021-05-31 15:32 UTC | newest]

Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-05-26 13:05 how to rollback / to a snapshot ? Lentes, Bernd
2021-05-26 14:08 ` Remi Gauvin
2021-05-26 14:53   ` Lentes, Bernd
2021-05-26 15:05     ` Remi Gauvin
2021-05-26 15:17     ` Remi Gauvin
2021-05-27 19:37       ` Lentes, Bernd
2021-05-27 19:43         ` Lentes, Bernd
2021-05-27 20:41         ` Remi Gauvin
2021-05-28 14:33           ` Lentes, Bernd
2021-05-28 14:56             ` Remi Gauvin
2021-05-28 15:36               ` Lentes, Bernd
2021-05-28 15:55                 ` Remi Gauvin
2021-05-31 15:22                   ` Lentes, Bernd

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.