BUG: wipefs dry run

BUG: wipefs dry run

[Ruediger Meier]

Hi,

I wonder why wipefs does not show all MBR and GPT signatures which would
be deleted by using "-fa".

$ truncate -s 100M /tmp/CCC
$ ./sfdisk /tmp/CCC <<EOF
> label: gpt
> label-id: 384A8F14-17FF-4782-A190-A5B858E3FCF0
> device: /tmp/CCC
> unit: sectors
> first-lba: 34
> last-lba: 204766
> EOF

$ ./wipefs  -fa --no-act   /tmp/CCC
/tmp/CCC: 8 bytes were erased at offset 0x00000200 (gpt): 45 46 49 20 50 
41 52 54

$ ./wipefs  -fa    /tmp/CCC
/tmp/CCC: 8 bytes were erased at offset 0x00000200 (gpt): 45 46 49 20 50 
41 52 54
/tmp/CCC: 8 bytes were erased at offset 0x063ffe00 (gpt): 45 46 49 20 50 
41 52 54
/tmp/CCC: 2 bytes were erased at offset 0x000001fe (PMBR): 55 aa


cu,
Rudi

Re: BUG: wipefs dry run

[Karel Zak]

On Wed, May 10, 2017 at 11:54:50AM +0200, Ruediger Meier wrote:
> I wonder why wipefs does not show all MBR and GPT signatures which would
> be deleted by using "-fa".
> 
> $ truncate -s 100M /tmp/CCC
> $ ./sfdisk /tmp/CCC <<EOF
> > label: gpt
> > label-id: 384A8F14-17FF-4782-A190-A5B858E3FCF0
> > device: /tmp/CCC
> > unit: sectors
> > first-lba: 34
> > last-lba: 204766
> > EOF
> 
> $ ./wipefs  -fa --no-act   /tmp/CCC
> /tmp/CCC: 8 bytes were erased at offset 0x00000200 (gpt): 45 46 49 20 50 
> 41 52 54
> 
> $ ./wipefs  -fa    /tmp/CCC
> /tmp/CCC: 8 bytes were erased at offset 0x00000200 (gpt): 45 46 49 20 50 
> 41 52 54
> /tmp/CCC: 8 bytes were erased at offset 0x063ffe00 (gpt): 45 46 49 20 50 
> 41 52 54
> /tmp/CCC: 2 bytes were erased at offset 0x000001fe (PMBR): 55 aa

This is not a bug, man wipefs:

        Note that some filesystems and some partition tables store more magic
        strings on the device.  The wipefs command lists only the  first
        offset  where  a  magic  string  has  been detected.   The  device  is
        not scanned for additional magic strings for the same filesystem.  It
        is possible that after a wipefs -o offset the same filesystem or
        partition table will still be visible because of another magic string
        on another offset.


The "-a" writes zeros, resets the scanner and repeat scan again and
again until nothing is found. I will add note about it to the man
page...

It would be probably possible to extend libblkid to work in some
"ignore previous result" mode to to gather all permutation for the
FS/RAID/etc, but it won't be trivial task...

    Karel

-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com

Re: BUG: wipefs dry run

[Karel Zak]

On Wed, May 10, 2017 at 12:32:02PM +0200, Karel Zak wrote:
> On Wed, May 10, 2017 at 11:54:50AM +0200, Ruediger Meier wrote:
> > I wonder why wipefs does not show all MBR and GPT signatures which would
> > be deleted by using "-fa".
> > 
> > $ truncate -s 100M /tmp/CCC
> > $ ./sfdisk /tmp/CCC <<EOF
> > > label: gpt
> > > label-id: 384A8F14-17FF-4782-A190-A5B858E3FCF0
> > > device: /tmp/CCC
> > > unit: sectors
> > > first-lba: 34
> > > last-lba: 204766
> > > EOF
> > 
> > $ ./wipefs  -fa --no-act   /tmp/CCC
> > /tmp/CCC: 8 bytes were erased at offset 0x00000200 (gpt): 45 46 49 20 50 
> > 41 52 54
> > 
> > $ ./wipefs  -fa    /tmp/CCC
> > /tmp/CCC: 8 bytes were erased at offset 0x00000200 (gpt): 45 46 49 20 50 
> > 41 52 54
> > /tmp/CCC: 8 bytes were erased at offset 0x063ffe00 (gpt): 45 46 49 20 50 
> > 41 52 54
> > /tmp/CCC: 2 bytes were erased at offset 0x000001fe (PMBR): 55 aa
> 
> This is not a bug, man wipefs:
> 
>         Note that some filesystems and some partition tables store more magic
>         strings on the device.  The wipefs command lists only the  first
>         offset  where  a  magic  string  has  been detected.   The  device  is
>         not scanned for additional magic strings for the same filesystem.  It
>         is possible that after a wipefs -o offset the same filesystem or
>         partition table will still be visible because of another magic string
>         on another offset.
> 
> 
> The "-a" writes zeros, resets the scanner and repeat scan again and
> again until nothing is found. I will add note about it to the man
> page...
> 
> It would be probably possible to extend libblkid to work in some
> "ignore previous result" mode to to gather all permutation for the
> FS/RAID/etc, but it won't be trivial task...

I'll think about it for v2.31... there are two ways:

1) rewrite probing functions to gather all possible magic strings
   (not trivial for fat, zfs, etc...)

2) call probing functions more than once (like wipefs -a) and use
   modified buffers where will be removed previously detected magic
   strings. This is the easy way (because library already supports
   "step back" (scan again last detected FS) and buffers management).

    Karel

-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com

Re: BUG: wipefs dry run

[Ruediger Meier]

On Wednesday 10 May 2017, Karel Zak wrote:
> On Wed, May 10, 2017 at 11:54:50AM +0200, Ruediger Meier wrote:
> > I wonder why wipefs does not show all MBR and GPT signatures which
> > would be deleted by using "-fa".
> >
> > $ truncate -s 100M /tmp/CCC
> > $ ./sfdisk /tmp/CCC <<EOF
> >
> > > label: gpt
> > > label-id: 384A8F14-17FF-4782-A190-A5B858E3FCF0
> > > device: /tmp/CCC
> > > unit: sectors
> > > first-lba: 34
> > > last-lba: 204766
> > > EOF
> >
> > $ ./wipefs  -fa --no-act   /tmp/CCC
> > /tmp/CCC: 8 bytes were erased at offset 0x00000200 (gpt): 45 46 49
> > 20 50 41 52 54
> >
> > $ ./wipefs  -fa    /tmp/CCC
> > /tmp/CCC: 8 bytes were erased at offset 0x00000200 (gpt): 45 46 49
> > 20 50 41 52 54
> > /tmp/CCC: 8 bytes were erased at offset 0x063ffe00 (gpt): 45 46 49
> > 20 50 41 52 54
> > /tmp/CCC: 2 bytes were erased at offset 0x000001fe (PMBR): 55 aa
>
> This is not a bug, man wipefs:
>
>         Note that some filesystems and some partition tables store
> more magic strings on the device.  The wipefs command lists only the 
> first offset  where  a  magic  string  has  been detected.   The 
> device  is not scanned for additional magic strings for the same
> filesystem.  It is possible that after a wipefs -o offset the same
> filesystem or partition table will still be visible because of
> another magic string on another offset.


I see.

>
> The "-a" writes zeros, resets the scanner and repeat scan again and
> again until nothing is found. I will add note about it to the man
> page...

Thanks. For me it's a bit hard to guess what wipefs would
actually do.

I have another issue, using wipefs on the same image like above:

$ ./wipefs -fa -t gpt  /tmp/CCC
/tmp/CCC: 8 bytes were erased at offset 0x00000200 (gpt): 45 46 49 20 50 41 52 54
/tmp/CCC: 8 bytes were erased at offset 0x063ffe00 (gpt): 45 46 49 20 50 41 52 54
/tmp/CCC: 8 bytes were erased at offset 0x063ffe00 (gpt): 45 46 49 20 50 41 52 54

There are two duplicated lines in the output and the not-mentioned (P)MBR
is also removed.


> It would be probably possible to extend libblkid to work in some
> "ignore previous result" mode to to gather all permutation for the
> FS/RAID/etc, but it won't be trivial task...

This would be nice.

cu,
RUdi

Re: BUG: wipefs dry run

[Karel Zak]

On Wed, May 10, 2017 at 01:03:10PM +0200, Ruediger Meier wrote:
> > It would be probably possible to extend libblkid to work in some
> > "ignore previous result" mode to to gather all permutation for the
> > FS/RAID/etc, but it won't be trivial task...
> 
> This would be nice.

Implemented (and it was relatively easy to implement :-)

Try "blkid-hide" branch from github, will be in v2.31.

# wipefs -a --no-act /dev/sdc
/dev/sdc: 8 bytes were erased at offset 0x00000200 (gpt): 45 46 49 20 50 41 52 54
/dev/sdc: 8 bytes were erased at offset 0x063ffe00 (gpt): 45 46 49 20 50 41 52 54
/dev/sdc: 2 bytes were erased at offset 0x000001fe (PMBR): 55 aa
/dev/sdc: calling ioctl to re-read partition table: Success

    Karel

-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com

Re: BUG: wipefs dry run

[Ruediger Meier]

On Friday 26 May 2017, Karel Zak wrote:
> On Wed, May 10, 2017 at 01:03:10PM +0200, Ruediger Meier wrote:
> > > It would be probably possible to extend libblkid to work in some
> > > "ignore previous result" mode to to gather all permutation for
> > > the FS/RAID/etc, but it won't be trivial task...
> >
> > This would be nice.
>
> Implemented (and it was relatively easy to implement :-)
>
> Try "blkid-hide" branch from github, will be in v2.31.

> # wipefs -a --no-act /dev/sdc


Thanks, this looks good. There are still a few issues-


$ truncate -s 100M /tmp/test.img
$ echo "label: gpt" | ./sfdisk /tmp/test.img

### this hangs indefinitely
$ wipefs -t gpt /tmp/test.img

### the default list shows too many newlines 
$ ./wipefs  /tmp/test.img
offset               type
----------------------------------------------------------------
0x1fe                PMBR   [partition table]

0x63ffe00            gpt   [partition table]

0x200                gpt   [partition table]


cu,
Rudi

Re: BUG: wipefs dry run

[Karel Zak]

On Thu, Jun 15, 2017 at 04:47:06PM +0200, Ruediger Meier wrote:
> On Friday 26 May 2017, Karel Zak wrote:
> > On Wed, May 10, 2017 at 01:03:10PM +0200, Ruediger Meier wrote:
> > > > It would be probably possible to extend libblkid to work in some
> > > > "ignore previous result" mode to to gather all permutation for
> > > > the FS/RAID/etc, but it won't be trivial task...
> > >
> > > This would be nice.
> >
> > Implemented (and it was relatively easy to implement :-)
> >
> > Try "blkid-hide" branch from github, will be in v2.31.
> 
> > # wipefs -a --no-act /dev/sdc
> 
> 
> Thanks, this looks good. There are still a few issues-

Thanks for testing!

> 
> $ truncate -s 100M /tmp/test.img
> $ echo "label: gpt" | ./sfdisk /tmp/test.img
> 
> ### this hangs indefinitely
> $ wipefs -t gpt /tmp/test.img

Fixed.

> ### the default list shows too many newlines 
> $ ./wipefs  /tmp/test.img
> offset               type
> ----------------------------------------------------------------
> 0x1fe                PMBR   [partition table]
> 
> 0x63ffe00            gpt   [partition table]
> 
> 0x200                gpt   [partition table]

All the output is ugly. I'll probably link it with libsmartcols,
because we support "wipefs /dev/sd*" to list more devices and then the
output is absolutely useless.

    Karel

-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com

Re: BUG: wipefs dry run

[Karel Zak]

On Thu, Jun 15, 2017 at 04:47:06PM +0200, Ruediger Meier wrote:
> ### the default list shows too many newlines 

libsmartcols based now (the library resolves all output issues in this
universe... problem fixed).

    Karel

-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com