Util-Linux Archive on lore.kernel.org
 help / color / Atom feed
* blkid identifying whole disk ntfs as both ntfs and msdos partition table
@ 2019-07-02 10:43 Mike Fleetwood
  2019-07-02 18:14 ` Theodore Ts'o
  2019-07-17 11:16 ` Karel Zak
  0 siblings, 2 replies; 5+ messages in thread
From: Mike Fleetwood @ 2019-07-02 10:43 UTC (permalink / raw)
  To: util-linux

Hi,

Tested with mkntfs from ntfsprogs and blkid and wipefs from util-linux git HEAD.
Test case:

# cat /etc/fedora-release
Fedora release 30 (Thirty)
# dd if=/dev/zero bs=1M of=/dev/sdb
dd: error writing '/dev/sdb': No space left on device
8193+0 records in
8192+0 records out
8589934592 bytes (8.6 GB, 8.0 GiB) copied, 39.6128 s, 217 MB/s
#  mkntfs --version
mkntfs v2017.3.23 (libntfs-3g)
...
# mkntfs -Q -F /dev/sdb
/dev/sdb is entire device, not just one partition.
mkntfs forced anyway.
Cluster size has been automatically set to 4096 bytes.
Creating NTFS volume structures.
mkntfs completed successfully. Have a nice day.
# ./blkid --version
lt-blkid from util-linux 2.34.24-e3bb9  (libblkid 2.34.24, 14-Jun-2019)
# ./blkid /dev/sdb
/dev/sdb: UUID="3B64D28A5CF89146" TYPE="ntfs" PTTYPE="dos"
# ./wipefs /dev/sdb
DEVICE OFFSET TYPE UUID             LABEL
sdb    0x3    ntfs 3B64D28A5CF89146
sdb    0x1fe  dos

Note that the test ntfs image in util-linux doesn't cause this.  Just file
systems created by mkntfs from ntfsprogs.

# dd if=/dev/zero bs=1M of=/dev/sdb
# xzcat tests/ts/blkid/images-fs/ntfs.img.xz > /dev/sdb
# ./blkid /dev/sdb
/dev/sdb: LABEL="M-PM-^]M-PM->M-PM-2M-QM-^KM-PM-9 M-QM-^BM-PM->M-PM-<"
UUID="09CBB6DE30C87310" TYPE="ntfs"
# ./wipefs /dev/sdb
DEVICE OFFSET TYPE UUID             LABEL
sdb    0x3    ntfs 09CBB6DE30C87310 Новый том

Thanks,
Mike

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: blkid identifying whole disk ntfs as both ntfs and msdos partition table
  2019-07-02 10:43 blkid identifying whole disk ntfs as both ntfs and msdos partition table Mike Fleetwood
@ 2019-07-02 18:14 ` Theodore Ts'o
  2019-07-02 19:06   ` Karel Zak
  2019-07-17 11:16 ` Karel Zak
  1 sibling, 1 reply; 5+ messages in thread
From: Theodore Ts'o @ 2019-07-02 18:14 UTC (permalink / raw)
  To: Mike Fleetwood; +Cc: util-linux

On Tue, Jul 02, 2019 at 11:43:34AM +0100, Mike Fleetwood wrote:
> Hi,
> 
> Tested with mkntfs from ntfsprogs and blkid and wipefs from util-linux git HEAD.
> Test case:
> 
> # mkntfs -Q -F /dev/sdb
> /dev/sdb is entire device, not just one partition.
> mkntfs forced anyway.
> Cluster size has been automatically set to 4096 bytes.
> Creating NTFS volume structures.
> mkntfs completed successfully. Have a nice day.
> # ./blkid --version
> lt-blkid from util-linux 2.34.24-e3bb9  (libblkid 2.34.24, 14-Jun-2019)
> # ./blkid /dev/sdb
> /dev/sdb: UUID="3B64D28A5CF89146" TYPE="ntfs" PTTYPE="dos"
> # ./wipefs /dev/sdb
> DEVICE OFFSET TYPE UUID             LABEL
> sdb    0x3    ntfs 3B64D28A5CF89146
> sdb    0x1fe  dos

So what we call the dos partition table, Microsoft calls the "boot
sector", and it's not necessarily a partition table at all.  (Note
also that normally, Microsoft won't let you put an ntfs file system on
the whole disk --- and you had to force this with mkntfs -F.  It's
actually a bad idea, because some versions of Windows will see what it
thinks is an empty disk, and the contents of that whole disk file
system can be very easily overwritten by a confused Windows OS.)

You can see this in include/layout.h in the ntfs-3g sources.

This is a pretty broad and general confusion; fdisk will get confused
and think it is an empty dos partition table, and blkid will as well.
I suspect the right answer here is if the oem_id is "NTFS ", we should
not consider it a dos partition table at all, even if there is an "end
of boot sector" magic value of 0xaa55 at the end of the sector.

   		      	       	      	 - Ted

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: blkid identifying whole disk ntfs as both ntfs and msdos partition table
  2019-07-02 18:14 ` Theodore Ts'o
@ 2019-07-02 19:06   ` Karel Zak
  0 siblings, 0 replies; 5+ messages in thread
From: Karel Zak @ 2019-07-02 19:06 UTC (permalink / raw)
  To: Theodore Ts'o; +Cc: Mike Fleetwood, util-linux

On Tue, Jul 02, 2019 at 02:14:39PM -0400, Theodore Ts'o wrote:
> On Tue, Jul 02, 2019 at 11:43:34AM +0100, Mike Fleetwood wrote:
> > Hi,
> > 
> > Tested with mkntfs from ntfsprogs and blkid and wipefs from util-linux git HEAD.
> > Test case:
> > 
> > # mkntfs -Q -F /dev/sdb
> > /dev/sdb is entire device, not just one partition.
> > mkntfs forced anyway.
> > Cluster size has been automatically set to 4096 bytes.
> > Creating NTFS volume structures.
> > mkntfs completed successfully. Have a nice day.
> > # ./blkid --version
> > lt-blkid from util-linux 2.34.24-e3bb9  (libblkid 2.34.24, 14-Jun-2019)
> > # ./blkid /dev/sdb
> > /dev/sdb: UUID="3B64D28A5CF89146" TYPE="ntfs" PTTYPE="dos"
> > # ./wipefs /dev/sdb
> > DEVICE OFFSET TYPE UUID             LABEL
> > sdb    0x3    ntfs 3B64D28A5CF89146
> > sdb    0x1fe  dos

 Thanks for your report Mike.

> So what we call the dos partition table, Microsoft calls the "boot
> sector", and it's not necessarily a partition table at all.  (Note
> also that normally, Microsoft won't let you put an ntfs file system on
> the whole disk --- and you had to force this with mkntfs -F.  It's
> actually a bad idea, because some versions of Windows will see what it
> thinks is an empty disk, and the contents of that whole disk file
> system can be very easily overwritten by a confused Windows OS.)
> 
> You can see this in include/layout.h in the ntfs-3g sources.
> 
> This is a pretty broad and general confusion; fdisk will get confused
> and think it is an empty dos partition table, and blkid will as well.
> I suspect the right answer here is if the oem_id is "NTFS ", we should
> not consider it a dos partition table at all, even if there is an "end
> of boot sector" magic value of 0xaa55 at the end of the sector.

 Well, our MBR detection is more sophisticated check for 0xaa55 magic
 ;-) but NTFS is not in the game now. I'll fix it. We need extra check
 for NTFS (as we already use for some FATs or GPT+PMBR).

    Karel

-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: blkid identifying whole disk ntfs as both ntfs and msdos partition table
  2019-07-02 10:43 blkid identifying whole disk ntfs as both ntfs and msdos partition table Mike Fleetwood
  2019-07-02 18:14 ` Theodore Ts'o
@ 2019-07-17 11:16 ` Karel Zak
  2019-07-18 20:10   ` Mike Fleetwood
  1 sibling, 1 reply; 5+ messages in thread
From: Karel Zak @ 2019-07-17 11:16 UTC (permalink / raw)
  To: Mike Fleetwood; +Cc: util-linux

On Tue, Jul 02, 2019 at 11:43:34AM +0100, Mike Fleetwood wrote:
> # mkntfs -Q -F /dev/sdb
> /dev/sdb is entire device, not just one partition.
> mkntfs forced anyway.
> Cluster size has been automatically set to 4096 bytes.
> Creating NTFS volume structures.
> mkntfs completed successfully. Have a nice day.
> # ./blkid --version
> lt-blkid from util-linux 2.34.24-e3bb9  (libblkid 2.34.24, 14-Jun-2019)
> # ./blkid /dev/sdb
> /dev/sdb: UUID="3B64D28A5CF89146" TYPE="ntfs" PTTYPE="dos"

Fixed version:

 # ./blkid -p /dev/sdc
 /dev/sdc: UUID="18CBAEE61457A850" TYPE="ntfs" USAGE="filesystem"

> # ./wipefs /dev/sdb
> DEVICE OFFSET TYPE UUID             LABEL
> sdb    0x3    ntfs 3B64D28A5CF89146
> sdb    0x1fe  dos

wipefs has a little bit different semantic, it tries to gather as
much as possible magic strings, so after the bug fix it still reports
"dos" on 0x1fe. It's expected behavior.

wipefs internally masks previously detected magic string (e.g. "ntfs")
to avoid any interaction with the next probing. This is because we
want to see all possibilities. Note wipefs wipes only the magic
strings (and nothing else), so if you wipe "NTFS" than it will be
interpreted as MBR ("dos") as there is no more valid NTFS.

> Note that the test ntfs image in util-linux doesn't cause this.  Just file
> systems created by mkntfs from ntfsprogs.
> 
> # dd if=/dev/zero bs=1M of=/dev/sdb
> # xzcat tests/ts/blkid/images-fs/ntfs.img.xz > /dev/sdb
> # ./blkid /dev/sdb
> /dev/sdb: LABEL="M-PM-^]M-PM->M-PM-2M-QM-^KM-PM-9 M-QM-^BM-PM->M-PM-<"
> UUID="09CBB6DE30C87310" TYPE="ntfs"
> # ./wipefs /dev/sdb
> DEVICE OFFSET TYPE UUID             LABEL
> sdb    0x3    ntfs 09CBB6DE30C87310 Новый том

Sounds like we need another test image :-)


Thanks for your reports!

    Karel

-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: blkid identifying whole disk ntfs as both ntfs and msdos partition table
  2019-07-17 11:16 ` Karel Zak
@ 2019-07-18 20:10   ` Mike Fleetwood
  0 siblings, 0 replies; 5+ messages in thread
From: Mike Fleetwood @ 2019-07-18 20:10 UTC (permalink / raw)
  To: Karel Zak; +Cc: util-linux

On Wed, 17 Jul 2019 at 12:16, Karel Zak <kzak@redhat.com> wrote:
>
> On Tue, Jul 02, 2019 at 11:43:34AM +0100, Mike Fleetwood wrote:
> > # mkntfs -Q -F /dev/sdb
> > /dev/sdb is entire device, not just one partition.
> > mkntfs forced anyway.
> > Cluster size has been automatically set to 4096 bytes.
> > Creating NTFS volume structures.
> > mkntfs completed successfully. Have a nice day.
> > # ./blkid --version
> > lt-blkid from util-linux 2.34.24-e3bb9  (libblkid 2.34.24, 14-Jun-2019)
> > # ./blkid /dev/sdb
> > /dev/sdb: UUID="3B64D28A5CF89146" TYPE="ntfs" PTTYPE="dos"
>
> Fixed version:
>
>  # ./blkid -p /dev/sdc
>  /dev/sdc: UUID="18CBAEE61457A850" TYPE="ntfs" USAGE="filesystem"
>
> > # ./wipefs /dev/sdb
> > DEVICE OFFSET TYPE UUID             LABEL
> > sdb    0x3    ntfs 3B64D28A5CF89146
> > sdb    0x1fe  dos
>
> wipefs has a little bit different semantic, it tries to gather as
> much as possible magic strings, so after the bug fix it still reports
> "dos" on 0x1fe. It's expected behavior.
>
> wipefs internally masks previously detected magic string (e.g. "ntfs")
> to avoid any interaction with the next probing. This is because we
> want to see all possibilities. Note wipefs wipes only the magic
> strings (and nothing else), so if you wipe "NTFS" than it will be
> interpreted as MBR ("dos") as there is no more valid NTFS.
>
> > Note that the test ntfs image in util-linux doesn't cause this.  Just file
> > systems created by mkntfs from ntfsprogs.
> >
> > # dd if=/dev/zero bs=1M of=/dev/sdb
> > # xzcat tests/ts/blkid/images-fs/ntfs.img.xz > /dev/sdb
> > # ./blkid /dev/sdb
> > /dev/sdb: LABEL="M-PM-^]M-PM->M-PM-2M-QM-^KM-PM-9 M-QM-^BM-PM->M-PM-<"
> > UUID="09CBB6DE30C87310" TYPE="ntfs"
> > # ./wipefs /dev/sdb
> > DEVICE OFFSET TYPE UUID             LABEL
> > sdb    0x3    ntfs 09CBB6DE30C87310 Новый том
>
> Sounds like we need another test image :-)
>
>
> Thanks for your reports!
>
>     Karel

Thank you for fixing these bugs so quickly.

Mike

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, back to index

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-07-02 10:43 blkid identifying whole disk ntfs as both ntfs and msdos partition table Mike Fleetwood
2019-07-02 18:14 ` Theodore Ts'o
2019-07-02 19:06   ` Karel Zak
2019-07-17 11:16 ` Karel Zak
2019-07-18 20:10   ` Mike Fleetwood

Util-Linux Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/util-linux/0 util-linux/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 util-linux util-linux/ https://lore.kernel.org/util-linux \
		util-linux@vger.kernel.org util-linux@archiver.kernel.org
	public-inbox-index util-linux


Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.util-linux


AGPL code for this site: git clone https://public-inbox.org/ public-inbox