Btrfs: wipe all the superblock [redhat bugzilla 889888]

Btrfs: wipe all the superblock [redhat bugzilla 889888]

[Goffredo Baroncelli]

Hi all,

Currently wipefs doesn't clear all the superblock of btrfs. Only the first 
one is cleared.

Btrfs has three superblocks. The first one is placed at 64KB, the second 
one at 64MB, the third one at 256GB.

If the first superblock is valid except that the "magic field" is zeroed,
btrfs skips the check of the other superblocks.
If the first superblock is fully invalid, btrfs checks for the other
superblock.

So zeroing the first superblock "magic field" at the beginning seems
that the filesystem is wiped. But when the first superblock is overwritten
(e.g. by another filesystem), then the other two superblocks may be considered
valid, and the filesystem may resurrect.


# make a filesystem, wipe it and check if it disappears

$ sudo mkfs.btrfs -L "Btrfs-test" /dev/loop0
$ sudo btrfs filesystem  show /dev/loop0
Label: 'Btrfs-test'  uuid: 3156cef7-8522-411f-876a-ba8ec32cc781
	Total devices 1 FS bytes used 28.00KB
	devid    1 size 4.00TB used 2.04GB path /dev/loop0

Btrfs Btrfs v0.19
$ sudo wipefs /dev/loop0
offset               type
----------------------------------------------------------------
0x10040              btrfs   [filesystem]
                     LABEL: Btrfs-test
                     UUID:  3156cef7-8522-411f-876a-ba8ec32cc781

ghigo@venice:/tmp$ sudo wipefs /dev/loop0 -a
8 bytes were erased at offset 0x10040 (btrfs)
they were: 5f 42 48 52 66 53 5f 4d
$ sudo btrfs filesystem  show /dev/loop0
Btrfs Btrfs v0.19

# it seems that the filesystem is disappeared 
# now zero all the 1st superblock

$ sudo dd if=/dev/zero of=/dev/loop0 bs=1 count=4k seek=64k
4096+0 records in
4096+0 records out
4096 bytes (4.1 kB) copied, 0.00659795 s, 621 kB/s

# check if the filesystem is resurrected
$ sudo btrfs filesystem  show /dev/loop0
failed to read /dev/sr0
Label: 'Btrfs-test'  uuid: 3156cef7-8522-411f-876a-ba8ec32cc781
	Total devices 1 FS bytes used 28.00KB
	devid    1 size 4.00TB used 2.04GB path /dev/loop0

Btrfs Btrfs v0.19

# it is !!!


# With this patch, wipefs is able to find all the superblock:

$ sudo mkfs.btrfs -L "Btrfs-test" /dev/loop0
$ sudo ~/btrfs/util-linux/wipefs /dev/loop0 
offset               type
----------------------------------------------------------------
0x4000000040         btrfs [bak #2]   [filesystem]
                     LABEL: Btrfs-test
                     UUID:  5ca3239c-363c-4c28-b831-eac03cc5ca62

0x4000040            btrfs [bak #1]   [filesystem]
                     LABEL: Btrfs-test
                     UUID:  5ca3239c-363c-4c28-b831-eac03cc5ca62

0x10040              btrfs   [filesystem]
                     LABEL: Btrfs-test
                     UUID:  5ca3239c-363c-4c28-b831-eac03cc5ca62

ghigo@venice:/tmp$ sudo ~/btrfs/util-linux/wipefs -a /dev/loop0 
/dev/loop0: 8 bytes were erased at offset 0x00010040 (btrfs): 
	5f 42 48 52 66 53 5f 4d
/dev/loop0: 8 bytes were erased at offset 0x04000040 (btrfs [bak #1]):
	5f 42 48 52 66 53 5f 4d
/dev/loop0: 8 bytes were erased at offset 0x4000000040 (btrfs [bak #2]): 
	5f 42 48 52 66 53 5f 4d

# Now even if we zero the 1st superblock the filesystem doesn't resurrect

$ sudo dd if=/dev/zero of=/dev/loop0 bs=1 count=4k seek=64k4096+0 records in
4096+0 records out
4096 bytes (4.1 kB) copied, 0.00643427 s, 637 kB/s
$ sudo btrfs filesystem  show /dev/loop0
Btrfs Btrfs v0.19

Br 
G.Baroncelli

You can pull the change from:
	http://cassiopea.homelinux.net/git/util-linux.git
from the branch
	btrfs-wipefs

Signed-off-by: Goffredo Baroncelli <kreijack@inwind.it>

------
>From 3ae972080495770205ec5da6d7a379a8d4a4d1c9 Mon Sep 17 00:00:00 2001
From: Goffredo Baroncelli <kreijack@inwind.it>
Date: Sun, 6 Jan 2013 18:47:11 +0100
Subject: [PATCH] Add support for wiping the other btrfs superblock.

Btrfs has three superblock. The first one is placed at 64KB, the
second one at 64MB, the third one at 256GB.

If the first superblock is valid except that the "magic field" is zeroed,
btrfs skips the check of the other superblocks.
If the first superblock is fully invalid, btrfs checks for the other
superblock.

So zeroing the first superblock "magic field" at the beginning seems
that the filesystem is wiped. But when the first superblock is overwritten
(e.g. by another filesystem), then the other two superblock may be considered
valid, and the filesystem may resurrect.

This patch allow to find and wipe the other btrfs superblocks signature.
---
 libblkid/src/superblocks/btrfs.c       |   32 ++++++++++++++++++++++++++++++++
 libblkid/src/superblocks/superblocks.c |    2 ++
 libblkid/src/superblocks/superblocks.h |    2 ++
 3 files changed, 36 insertions(+)

diff --git a/libblkid/src/superblocks/btrfs.c b/libblkid/src/superblocks/btrfs.c
index 039be42..d1331e6 100644
--- a/libblkid/src/superblocks/btrfs.c
+++ b/libblkid/src/superblocks/btrfs.c
@@ -91,3 +91,35 @@ const struct blkid_idinfo btrfs_idinfo =
 	}
 };
 
+const struct blkid_idinfo btrfs_idinfo1 =
+{
+	.name		= "btrfs [bak #1]",
+	.usage		= BLKID_USAGE_FILESYSTEM,
+	.probefunc	= probe_btrfs,
+	.minsz		= 64 * 1024 * 1024 + 4 * 1024,
+	.magics		=
+	{
+		{ .magic = "_BHRfS_M", 
+		  .len = 8, 
+		  .kboff = 64 * 1024, 
+		  .sboff = 0x40 },
+		{ NULL }
+	}
+};
+
+const struct blkid_idinfo btrfs_idinfo2 =
+{
+	.name		= "btrfs [bak #2]",
+	.usage		= BLKID_USAGE_FILESYSTEM,
+	.probefunc	= probe_btrfs,
+	.minsz		= 256 * 1024 * 1024 * 1024 + 4 *1024,
+	.magics		=
+	{
+		{ .magic = "_BHRfS_M", 
+		  .len = 8, 
+		  .kboff = 256 * 1024 * 1024, 
+		  .sboff = 0x40 },
+		{ NULL }
+	}
+};
+
diff --git a/libblkid/src/superblocks/superblocks.c b/libblkid/src/superblocks/superblocks.c
index 2929a5f..1e88b63 100644
--- a/libblkid/src/superblocks/superblocks.c
+++ b/libblkid/src/superblocks/superblocks.c
@@ -135,6 +135,8 @@ static const struct blkid_idinfo *idinfos[] =
 	&squashfs_idinfo,
 	&netware_idinfo,
 	&btrfs_idinfo,
+	&btrfs_idinfo1,
+	&btrfs_idinfo2,
 	&ubifs_idinfo,
 	&bfs_idinfo,
 	&vmfs_fs_idinfo,
diff --git a/libblkid/src/superblocks/superblocks.h b/libblkid/src/superblocks/superblocks.h
index 08f1438..974ff8e 100644
--- a/libblkid/src/superblocks/superblocks.h
+++ b/libblkid/src/superblocks/superblocks.h
@@ -59,6 +59,8 @@ extern const struct blkid_idinfo netware_idinfo;
 extern const struct blkid_idinfo sysv_idinfo;
 extern const struct blkid_idinfo xenix_idinfo;
 extern const struct blkid_idinfo btrfs_idinfo;
+extern const struct blkid_idinfo btrfs_idinfo1;
+extern const struct blkid_idinfo btrfs_idinfo2;
 extern const struct blkid_idinfo ubifs_idinfo;
 extern const struct blkid_idinfo zfs_idinfo;
 extern const struct blkid_idinfo bfs_idinfo;
-- 
1.7.10.4


-- 
gpg @keyserver.linux.it: Goffredo Baroncelli (kreijackATinwind.it>
Key fingerprint BBF5 1610 0B64 DAC6 5F7D  17B2 0EDA 9B37 8B82 E0B5

Re: Btrfs: wipe all the superblock [redhat bugzilla 889888]

[David Sterba]

On Sun, Jan 06, 2013 at 07:28:55PM +0100, Goffredo Baroncelli wrote:
> Currently wipefs doesn't clear all the superblock of btrfs. Only the first 
> one is cleared.
> 
> Btrfs has three superblocks. The first one is placed at 64KB, the second 
> one at 64MB, the third one at 256GB.

It can have as much as 4 superblock backup copies:

Superblock offset 0 is 65536 (0x10000, block=16/0x10)
Superblock offset 1 is 67108864 (0x4000000, block=16384/0x4000)
Superblock offset 2 is 274877906944 (0x4000000000, block=67108864/0x4000000)
Superblock offset 3 is 1125899906842624 (0x4000000000000, block=274877906944/0x4000000000)
Superblock offset 4 is 4611686018427387904 (0x4000000000000000, block=1125899906842624/0x4000000000000)

> If the first superblock is valid except that the "magic field" is zeroed,
> btrfs skips the check of the other superblocks.
> If the first superblock is fully invalid, btrfs checks for the other
> superblock.
> 
> So zeroing the first superblock "magic field" at the beginning seems
> that the filesystem is wiped. But when the first superblock is overwritten
> (e.g. by another filesystem), then the other two superblocks may be considered
> valid, and the filesystem may resurrect.

And for that purpose all the superblock copies should be taken into
account, regardless of the tricks that btrfs_mount applies.


david

Re: Btrfs: wipe all the superblock [redhat bugzilla 889888]

[Goffredo Baroncelli]

Hi David,

On 01/07/2013 05:33 PM, David Sterba wrote:
> On Sun, Jan 06, 2013 at 07:28:55PM +0100, Goffredo Baroncelli wrote:
>> Currently wipefs doesn't clear all the superblock of btrfs. Only the first 
>> one is cleared.
>>
>> Btrfs has three superblocks. The first one is placed at 64KB, the second 
>> one at 64MB, the third one at 256GB.
> 
> It can have as much as 4 superblock backup copies:
> 
> Superblock offset 0 is 65536 (0x10000, block=16/0x10)
> Superblock offset 1 is 67108864 (0x4000000, block=16384/0x4000)
> Superblock offset 2 is 274877906944 (0x4000000000, block=67108864/0x4000000)
> Superblock offset 3 is 1125899906842624 (0x4000000000000, block=274877906944/0x4000000000)
> Superblock offset 4 is 4611686018427387904 (0x4000000000000000, block=1125899906842624/0x4000000000000)

Are you sure ?

Regarding the btrfs-progs suite, I looked at the btrfs_read_dev_super():
[..]
        for (i = 0; i < BTRFS_SUPER_MIRROR_MAX; i++) {
                bytenr = btrfs_sb_offset(i);
                ret = pread64(fd, &buf, sizeof(buf), bytenr);

Where BTRFS_SUPER_MIRROR_MAX is 3.

Regarding the kernel code, I looked at several function which call
btrfs_sb_offset(); everywhere there is an upper limit of the superblock
numbero which is BTRFS_SUPER_MIRROR_MAX, which is still 3.

Moreover I performed the following test:

$ ls -lh 7tb-filesystem.img
-rw-r--r-- 1 ghigo ghigo 7.1E Jan  7 18:49 7eb-filesystem.img
$ /sbin/mkfs.btrfs 7eb-filesystem.img

$ cat extract-sign.py
import os

BTRFS_SUPER_MIRROR_SHIFT = 12
BTRFS_SUPER_INFO_OFFSET = (64*1024)

def btrfs_sb_offset(mirror):
    start = 16*1024
    if(mirror):
        return start << (BTRFS_SUPER_MIRROR_SHIFT * mirror)
    return BTRFS_SUPER_INFO_OFFSET

f = open("7eb-filesystem.img","r")
for i in range(5):
    pos = btrfs_sb_offset(i)+64
    f.seek(pos)
    sign = f.read(8)

    print "Superblock #%d - %20d - '%s'"%(i,pos,sign)

$ python extract-sign.py
Superblock #0 -                65600 - '_BHRfS_M'
Superblock #1 -             67108928 - '_BHRfS_M'
Superblock #2 -         274877907008 - '_BHRfS_M'
Superblock #3 -     1125899906842688 - ''
Superblock #4 -  4611686018427387968 - ''


To me it seems that in a 7TB filesystem there is only 3 superblocks.


> 
>> If the first superblock is valid except that the "magic field" is zeroed,
>> btrfs skips the check of the other superblocks.
>> If the first superblock is fully invalid, btrfs checks for the other
>> superblock.
>>
>> So zeroing the first superblock "magic field" at the beginning seems
>> that the filesystem is wiped. But when the first superblock is overwritten
>> (e.g. by another filesystem), then the other two superblocks may be considered
>> valid, and the filesystem may resurrect.
> 
> And for that purpose all the superblock copies should be taken into
> account, regardless of the tricks that btrfs_mount applies.
> 
> 
> david
> 


-- 
gpg @keyserver.linux.it: Goffredo Baroncelli (kreijackATinwind.it>
Key fingerprint BBF5 1610 0B64 DAC6 5F7D  17B2 0EDA 9B37 8B82 E0B5

Re: Btrfs: wipe all the superblock [redhat bugzilla 889888]

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 2876 bytes --]

On Mon, Jan 07, 2013 at 07:20:16PM +0100, Goffredo Baroncelli wrote:
> Hi David,
> 
> On 01/07/2013 05:33 PM, David Sterba wrote:
> > On Sun, Jan 06, 2013 at 07:28:55PM +0100, Goffredo Baroncelli wrote:
> >> Currently wipefs doesn't clear all the superblock of btrfs. Only the first 
> >> one is cleared.
> >>
> >> Btrfs has three superblocks. The first one is placed at 64KB, the second 
> >> one at 64MB, the third one at 256GB.
> > 
> > It can have as much as 4 superblock backup copies:
> > 
> > Superblock offset 0 is 65536 (0x10000, block=16/0x10)
> > Superblock offset 1 is 67108864 (0x4000000, block=16384/0x4000)
> > Superblock offset 2 is 274877906944 (0x4000000000, block=67108864/0x4000000)
> > Superblock offset 3 is 1125899906842624 (0x4000000000000, block=274877906944/0x4000000000)
> > Superblock offset 4 is 4611686018427387904 (0x4000000000000000, block=1125899906842624/0x4000000000000)
> 
> Are you sure ?
> 
> Regarding the btrfs-progs suite, I looked at the btrfs_read_dev_super():
> [..]
>         for (i = 0; i < BTRFS_SUPER_MIRROR_MAX; i++) {
>                 bytenr = btrfs_sb_offset(i);
>                 ret = pread64(fd, &buf, sizeof(buf), bytenr);
> 
> Where BTRFS_SUPER_MIRROR_MAX is 3.
> 
> Regarding the kernel code, I looked at several function which call
> btrfs_sb_offset(); everywhere there is an upper limit of the superblock
> numbero which is BTRFS_SUPER_MIRROR_MAX, which is still 3.
> 
> Moreover I performed the following test:
> 
> $ ls -lh 7tb-filesystem.img
> -rw-r--r-- 1 ghigo ghigo 7.1E Jan  7 18:49 7eb-filesystem.img
> $ /sbin/mkfs.btrfs 7eb-filesystem.img
> 
> $ cat extract-sign.py
> import os
> 
> BTRFS_SUPER_MIRROR_SHIFT = 12
> BTRFS_SUPER_INFO_OFFSET = (64*1024)
> 
> def btrfs_sb_offset(mirror):
>     start = 16*1024
>     if(mirror):
>         return start << (BTRFS_SUPER_MIRROR_SHIFT * mirror)
>     return BTRFS_SUPER_INFO_OFFSET
> 
> f = open("7eb-filesystem.img","r")
> for i in range(5):
>     pos = btrfs_sb_offset(i)+64
>     f.seek(pos)
>     sign = f.read(8)
> 
>     print "Superblock #%d - %20d - '%s'"%(i,pos,sign)
> 
> $ python extract-sign.py
> Superblock #0 -                65600 - '_BHRfS_M'

64 KiB

> Superblock #1 -             67108928 - '_BHRfS_M'

256 MiB

> Superblock #2 -         274877907008 - '_BHRfS_M'

1 TiB

> Superblock #3 -     1125899906842688 - ''

4 PiB

> Superblock #4 -  4611686018427387968 - ''

16 EiB


> To me it seems that in a 7TB filesystem there is only 3 superblocks.

   That would be as expected. How many on a 5 PiB filesystem, though?
Or a 20 EiB one?

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
  --- "He's a nutcase, you know. There's no getting away from it -- ---  
                     he'll end up with a knighthood"                     

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: Btrfs: wipe all the superblock [redhat bugzilla 889888]

[Goffredo Baroncelli]

Hi hugo

On 01/07/2013 07:24 PM, Hugo Mills wrote:
[...]

>>     print "Superblock #%d - %20d - '%s'"%(i,pos,sign)
>>
>> $ python extract-sign.py
>> Superblock #0 -                65600 - '_BHRfS_M'
> 
> 64 KiB

OK, (above)
> 
>> Superblock #1 -             67108928 - '_BHRfS_M'
> 
> 256 MiB

.... above is 64M, below is 256M !
> 
>> Superblock #2 -         274877907008 - '_BHRfS_M'
> 
> 1 TiB
> 
>> Superblock #3 -     1125899906842688 - ''
> 
> 4 PiB
> 
>> Superblock #4 -  4611686018427387968 - ''
> 
> 16 EiB

Not reachable in linux. There is a VFS limits to 8EB.
> 
> 
>> To me it seems that in a 7TB filesystem there is only 3 superblocks.
> 
>    That would be as expected. How many on a 5 PiB filesystem, though?
> Or a 20 EiB one?

I wrote 7TB, but I meant 8EB. I first tried a test with 7TB and I wrote
the email, but then I retested with 8EB and I corrected the email,
forgetting some 7t instead of 8e...

Anyway, if BTRFS would allow more than three super-block,  in the 7TB
case, the super-block at 1TB would appeared
> 
>    Hugo.
> 


-- 
gpg @keyserver.linux.it: Goffredo Baroncelli (kreijackATinwind.it>
Key fingerprint BBF5 1610 0B64 DAC6 5F7D  17B2 0EDA 9B37 8B82 E0B5

Re: Btrfs: wipe all the superblock [redhat bugzilla 889888]

Am 06.01.2013 19:28, schrieb Goffredo Baroncelli:
>
> You can pull the change from:
> 	http://cassiopea.homelinux.net/git/util-linux.git
> from the branch
> 	btrfs-wipefs
>
After applying this patch, findfs and blkid didn't find filesystems 
smaller than 256GB, probably because there is no third superblock.

Günter

Re: Btrfs: wipe all the superblock [redhat bugzilla 889888]

[Günter Gersdorf]

Am 06.01.2013 19:28, schrieb Goffredo Baroncelli:
>
> You can pull the change from:
> 	http://cassiopea.homelinux.net/git/util-linux.git
> from the branch
> 	btrfs-wipefs
>
After applying this patch, findfs and blkid didn't find filesystems 
smaller than 256GB, probably because there is no third superblock.

Günter

Re: Btrfs: wipe all the superblock [redhat bugzilla 889888]

[Karel Zak]

On Sun, Jan 06, 2013 at 07:28:55PM +0100, Goffredo Baroncelli wrote:
> +const struct blkid_idinfo btrfs_idinfo1 =
> +{
> +	.name		= "btrfs [bak #1]",
> +	.usage		= BLKID_USAGE_FILESYSTEM,
> +	.probefunc	= probe_btrfs,
> +	.minsz		= 64 * 1024 * 1024 + 4 * 1024,
> +	.magics		=
> +	{
> +		{ .magic = "_BHRfS_M", 
> +		  .len = 8, 
> +		  .kboff = 64 * 1024, 
> +		  .sboff = 0x40 },
> +		{ NULL }
> +	}
> +};
> +
> +const struct blkid_idinfo btrfs_idinfo2 =
> +{
> +	.name		= "btrfs [bak #2]",
> +	.usage		= BLKID_USAGE_FILESYSTEM,
> +	.probefunc	= probe_btrfs,
> +	.minsz		= 256 * 1024 * 1024 * 1024 + 4 *1024,
> +	.magics		=
> +	{
> +		{ .magic = "_BHRfS_M", 
> +		  .len = 8, 
> +		  .kboff = 256 * 1024 * 1024, 
> +		  .sboff = 0x40 },
> +		{ NULL }
> +	}
> +};

 You can specify more than one magic strings for the same filesystem,
 the .magics = { } is array.

 .magics     = {

    /* backup #1 */
	{ .magic = "_BHRfS_M", 
	  .len = 8, 
	  .kboff = 64 * 1024, 
	  .sboff = 0x40 },
    },

    /* backup #2 */
    {   .magic = "_BHRfS_M",
        .len = 8,
        .kboff = 256 * 1024 * 1024,
        .sboff = 0x40
    }, 
    ...
 }

 see for example libblkid/src/superblocks/reiserfs.c 

 Karel

-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com

Re: Btrfs: wipe all the superblock [redhat bugzilla 889888]

[David Sterba]

On Mon, Jan 07, 2013 at 07:20:16PM +0100, Goffredo Baroncelli wrote:
> On 01/07/2013 05:33 PM, David Sterba wrote:
> > It can have as much as 4 superblock backup copies:
> > 
> > Superblock offset 0 is 65536 (0x10000, block=16/0x10)
> > Superblock offset 1 is 67108864 (0x4000000, block=16384/0x4000)
> > Superblock offset 2 is 274877906944 (0x4000000000, block=67108864/0x4000000)
> > Superblock offset 3 is 1125899906842624 (0x4000000000000, block=274877906944/0x4000000000)
> > Superblock offset 4 is 4611686018427387904 (0x4000000000000000, block=1125899906842624/0x4000000000000)
> 
> Are you sure ?
> 
> Regarding the btrfs-progs suite, I looked at the btrfs_read_dev_super():
> [..]
>         for (i = 0; i < BTRFS_SUPER_MIRROR_MAX; i++) {
>                 bytenr = btrfs_sb_offset(i);
>                 ret = pread64(fd, &buf, sizeof(buf), bytenr);
> 
> Where BTRFS_SUPER_MIRROR_MAX is 3.

My bad, sorry, I was using the values from an old script that computed
the values with a wrong upper limit.

david

Re: Btrfs: wipe all the superblock [redhat bugzilla 889888]

[Karel Zak]

On Sun, Jan 06, 2013 at 07:28:55PM +0100, Goffredo Baroncelli wrote:
> If the first superblock is valid except that the "magic field" is zeroed,
> btrfs skips the check of the other superblocks.
> If the first superblock is fully invalid, btrfs checks for the other
> superblock.

 Hmm... why inconsistent (or missing) superblock is not reported as a
 problem? If I good understand the filesystem is still mountable,
 right?

> So zeroing the first superblock "magic field" at the beginning seems
> that the filesystem is wiped.

 Well, this is exactly the idea behind wipefs(8), just wipe minimal
 number of bytes from the device to make the filesystem invisible for
 libblkid (udev, ...). This concept is relatively safe, if you make a
 mistake than you can restore the magic string, your data should not
 be affected by wipefs(8).

    Karel

-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com

Re: Btrfs: wipe all the superblock [redhat bugzilla 889888]

[Goffredo Baroncelli]

Hi Karel,

On 01/08/2013 07:01 PM, Karel Zak wrote:
> On Sun, Jan 06, 2013 at 07:28:55PM +0100, Goffredo Baroncelli wrote:
>> If the first superblock is valid except that the "magic field" is zeroed,
>> btrfs skips the check of the other superblocks.
>> If the first superblock is fully invalid, btrfs checks for the other
>> superblock.
> 
>  Hmm... why inconsistent (or missing) superblock is not reported as a
>  problem? If I good understand the filesystem is still mountable,
>  right?

It should, however my tests were unsuccessful :-(... Chris ?


> 
>> So zeroing the first superblock "magic field" at the beginning seems
>> that the filesystem is wiped.
> 
>  Well, this is exactly the idea behind wipefs(8), just wipe minimal
>  number of bytes from the device to make the filesystem invisible for
>  libblkid (udev, ...). This concept is relatively safe, if you make a
>  mistake than you can restore the magic string, your data should not
>  be affected by wipefs(8).

I fully agree. However wipefs should zero all three superblock

> 
>     Karel
> 


-- 
gpg @keyserver.linux.it: Goffredo Baroncelli (kreijackATinwind.it>
Key fingerprint BBF5 1610 0B64 DAC6 5F7D  17B2 0EDA 9B37 8B82 E0B5

Re: Btrfs: wipe all the superblock [redhat bugzilla 889888]

[Chris Murphy]

On Jan 8, 2013, at 1:09 PM, Goffredo Baroncelli <kreijack@tiscalinet.it> wrote:

> Hi Karel,
> 
> On 01/08/2013 07:01 PM, Karel Zak wrote:
>> On Sun, Jan 06, 2013 at 07:28:55PM +0100, Goffredo Baroncelli wrote:
>>> If the first superblock is valid except that the "magic field" is zeroed,
>>> btrfs skips the check of the other superblocks.
>>> If the first superblock is fully invalid, btrfs checks for the other
>>> superblock.
>> 
>> Hmm... why inconsistent (or missing) superblock is not reported as a
>> problem? If I good understand the filesystem is still mountable,
>> right?
> 
> It should, however my tests were unsuccessful :-(... Chris ?

I haven't tried mounting after wipefs, only after wipefs and subsequent mkfs.ext4; in that case the file system mounts without error as ext4. If I use mount -t btrfs I get an error. I haven't tried mounting with -t btrfs -o recovery.

Chris Murphy

Re: Btrfs: wipe all the superblock [redhat bugzilla 889888]

[Goffredo Baroncelli]

Hi Günter,

On 01/08/2013 04:48 PM, Günter Gersdorf wrote:
> Am 06.01.2013 19:28, schrieb Goffredo Baroncelli:
>>
>> You can pull the change from:
>>     http://cassiopea.homelinux.net/git/util-linux.git
>> from the branch
>>     btrfs-wipefs
>>
> After applying this patch, findfs and blkid didn't find filesystems
> smaller than 256GB, probably because there is no third superblock.

In my system also the stock "findfs" and "blkid" have the same problem:
ie without my patch these utilities were unable to detect a btrfs
filesystem.

Could you confirm that ?

> 
> Günter





-- 
gpg @keyserver.linux.it: Goffredo Baroncelli (kreijackATinwind.it>
Key fingerprint BBF5 1610 0B64 DAC6 5F7D  17B2 0EDA 9B37 8B82 E0B5

Re: Btrfs: wipe all the superblock [redhat bugzilla 889888]

[Günter Gersdorf]

Am 08.01.2013 21:31, schrieb Goffredo Baroncelli:
> Hi Günter,
>
> On 01/08/2013 04:48 PM, Günter Gersdorf wrote:
>> Am 06.01.2013 19:28, schrieb Goffredo Baroncelli:
>>>
>>> You can pull the change from:
>>>      http://cassiopea.homelinux.net/git/util-linux.git
>>> from the branch
>>>      btrfs-wipefs
>>>
>> After applying this patch, findfs and blkid didn't find filesystems
>> smaller than 256GB, probably because there is no third superblock.
>
> In my system also the stock "findfs" and "blkid" have the same problem:
> ie without my patch these utilities were unable to detect a btrfs
> filesystem.
>
> Could you confirm that ?

Without the patch?
On my systems, after applying the patch blkid and findfs will not find 
any btrfs filesystems, even if bigger than 256GB.

Günter

Re: Btrfs: wipe all the superblock [redhat bugzilla 889888]

[Goffredo Baroncelli]

Hi Karel,

> 
>  You can specify more than one magic strings for the same filesystem,
>  the .magics = { } is array.

thanks for you suggestion. However this seems to me not applicable. I
tried to change the code, and what I got to me seems inconsistently:

Whit this change

1) if I do "wipefs <device>", I got the offset of the first superblock
(good enough)
2) if I do "wipefs -a <device>", I clean-up *all three* superblocks
(very good)
3) if I do "wipefs -o <offset> <device>", I clean-up only the superblock
located at <offset> (very bad)

If the user doesn't know enough btrfs, trying 1) and 3) could think that
the disk is cleaned-up. Instead the 2nd and the 3rd super-blocks still
exist.

>  see for example libblkid/src/superblocks/reiserfs.c 

I think that this is a different case: the reiser superblocks are
*alternative*; instead in the btrfs case, *all the three superblocks*
exist at the same time.

>  Karel
Ciao
Goffredo

-- 
gpg @keyserver.linux.it: Goffredo Baroncelli (kreijackATinwind.it>
Key fingerprint BBF5 1610 0B64 DAC6 5F7D  17B2 0EDA 9B37 8B82 E0B5

Re: Btrfs: wipe all the superblock [redhat bugzilla 889888]

[Karel Zak]

On Wed, Jan 09, 2013 at 06:48:28PM +0100, Goffredo Baroncelli wrote:
> Hi Karel,
> 
> > 
> >  You can specify more than one magic strings for the same filesystem,
> >  the .magics = { } is array.
> 
> thanks for you suggestion. However this seems to me not applicable. I
> tried to change the code, and what I got to me seems inconsistently:
> 
> Whit this change
> 
> 1) if I do "wipefs <device>", I got the offset of the first superblock
> (good enough)
> 2) if I do "wipefs -a <device>", I clean-up *all three* superblocks
> (very good)
> 3) if I do "wipefs -o <offset> <device>", I clean-up only the superblock
> located at <offset> (very bad)

 this is expected behavior described in wipefs man page:

Note  that  some  filesystems  or  some partition tables store more
magic strings on the devices. The wipefs lists the first offset where
a magic string has been detected. The device is not  scanned  for
additional magic strings for the same filesystem. It's possible that
after wipefs -o <offset> will be the same filesystem or partition
table visible by another magic string on another offset.

> If the user doesn't know enough btrfs, trying 1) and 3) could think that
> the disk is cleaned-up. Instead the 2nd and the 3rd super-blocks still
> exist.

 well, users (and installers) usually use wipefs -a or wipefs -t <fsname>

> >  see for example libblkid/src/superblocks/reiserfs.c 
> 
> I think that this is a different case: the reiser superblocks are

 it was example how to specify the magic strings in the code

> *alternative*; instead in the btrfs case, *all the three superblocks*
> exist at the same time.

 this is pretty common to have backup superblock (e.g. GPT) or more
 ways how to detect the filesystem (e.g. FAT).

 Please, send me the patch with the magic strings :-)
 
 I really don't want to add dummy filesystems to the library (like you
 did in the first version of the patch) -- it's very bad idea with
 many side effects.

    Karel

-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com

Re: Btrfs: wipe all the superblock [redhat bugzilla 889888]

[Karel Zak]

gth: 2229

On Wed, Jan 09, 2013 at 06:48:28PM +0100, Goffredo Baroncelli wrote:
> Hi Karel,
> 
> > 
> >  You can specify more than one magic strings for the same filesystem,
> >  the .magics = { } is array.
> 
> thanks for you suggestion. However this seems to me not applicable. I
> tried to change the code, and what I got to me seems inconsistently:
> 
> Whit this change
> 
> 1) if I do "wipefs <device>", I got the offset of the first superblock
> (good enough)
> 2) if I do "wipefs -a <device>", I clean-up *all three* superblocks
> (very good)
> 3) if I do "wipefs -o <offset> <device>", I clean-up only the superblock
> located at <offset> (very bad)

 this is expected behavior described in wipefs man page:

Note  that  some  filesystems  or  some partition tables store more
magic strings on the devices. The wipefs lists the first offset where
a magic string has been detected. The device is not  scanned  for
additional magic strings for the same filesystem. It's possible that
after wipefs -o <offset> will be the same filesystem or partition
table visible by another magic string on another offset.

> If the user doesn't know enough btrfs, trying 1) and 3) could think that
> the disk is cleaned-up. Instead the 2nd and the 3rd super-blocks still
> exist.

 well, users (and installers) usually use wipefs -a or wipefs -t <fsname>

> >  see for example libblkid/src/superblocks/reiserfs.c 
> 
> I think that this is a different case: the reiser superblocks are

 it was example how to specify the magic strings in the code

> *alternative*; instead in the btrfs case, *all the three superblocks*
> exist at the same time.

 this is pretty common to have backup superblock (e.g. GPT) or more
 ways how to detect the filesystem (e.g. FAT).

 Please, send me the patch with the magic strings :-)
 
 I really don't want to add dummy filesystems to the library (like you
 did in the first version of the patch) -- it's very bad idea with
 many side effects.

    Karel

-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo
info.html
Status: O
Content-Length: 1151
Lines: 31

On Wed, Jan 09, 2013 at 02:59:25PM -0200, Renato Zannon wrote:
> From my very limited knowledge, that's all the relevant information I
> have. If there's anything else I can do I would be glad to help.

Thanks for the report.

> [ 3850.907905] kernel BUG at fs/btrfs/inode.c:881!

cow_file_range:

 880         BUG_ON(disk_num_bytes >
 881                btrfs_super_total_bytes(root->fs_info->super_copy));

The point of crash has been seen some time ago, and I was not able to
find any other than my own post

http://permalink.gmane.org/gmane.comp.file-systems.btrfs/16227

It was possible to trigger the crash if the filesystem was created with
a bigger size than the blockdevice (-b check), but as we've talked on
IRC, this was not your case. I'm out of ideas how this could normally
happen. For a single device the filesystem size accounted by superblock
and by sum of all block groups should the same given that no devices are
added/removed.


david