Re: Atari label false positives

Re: Atari label false positives

[John Paul Adrian Glaubitz]

Hi Phil!

On 05/11/2018 04:18 PM, Phil Susi wrote:
> It seems that the atari label likes to falsely recognize dos labels as
> atari.  Looking at atari_probe, it doesn't seem like this is very hard
> to do.  All that it needs is one word for the size of the disk to be
> less than the physical size of the disk, two bsl words to be zero or
> less than the disk size, the partition start and length words to fit
> within the disk, and 3 bytes to appear to be ascii characters.  Is there
> any way that this can be beefed up at all?

Thanks for digging this up. I wasn't aware of this particular issue before
and I agree, this needs to be addressed. I hope that I didn't cause too
many DOS partitions to be misdetected. Sorry for the inconvenience if I
did. The Atari partition table code has been working great for Debian/m68k so
far though.

> For instance, the name field.. is it really free form ascii or is there
> a specific set of strings that could be limited to?  Also this XGM and
> ICD parts; are they completely optional, so we can't require one of
> those be detected too?

I don't know by heart, but chapter 9 in this Atari filesystem reference
might help us:

> https://people.debian.org/~glaubitz/Atari_HD_File_Sytem_Reference_Guide.pdf

I have also CC'ed the Linux/m68k and Debian/m68k mailing lists in case some of the
guys there knows the answer faster.

Again, sorry for the inconvenience this may have caused. I will look into
how we can improve atari_probe().

Adrian

-- 
  .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz-8fiUuRrzOP0dnm+yROfE0A@public.gmane.org
`. `'   Freie Universitaet Berlin - glaubitz-1Olz3AKvcsuAKZTfuerNgRvVK+yQ3ZXh@public.gmane.org
   `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Re: Atari label false positives

[Phillip Susi]

[-- Attachment #1.1: Type: text/plain, Size: 819 bytes --]

On 5/11/2018 11:29 AM, John Paul Adrian Glaubitz wrote:
> Thanks for digging this up. I wasn't aware of this particular issue before
> and I agree, this needs to be addressed. I hope that I didn't cause too
> many DOS partitions to be misdetected. Sorry for the inconvenience if I
> did. The Atari partition table code has been working great for
> Debian/m68k so
> far though.

I have seen at least one or two reports of Ubuntu users getting false
positives, and then I saw the bug report on the upstream parted bug
tracker the other day where someone claimed to have found a logical
error that caused atari_probe to return true when it shouldn't.  I
finally realized it was not an error ( at least I think not ).

I think I will at least move atari to be probed after all other labels
have failed.



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: Atari label false positives

[John Paul Adrian Glaubitz]

On 05/11/2018 05:56 PM, Phillip Susi wrote:
> On 5/11/2018 11:29 AM, John Paul Adrian Glaubitz wrote:
>> Thanks for digging this up. I wasn't aware of this particular issue before
>> and I agree, this needs to be addressed. I hope that I didn't cause too
>> many DOS partitions to be misdetected. Sorry for the inconvenience if I
>> did. The Atari partition table code has been working great for
>> Debian/m68k so
>> far though.
> 
> I have seen at least one or two reports of Ubuntu users getting false
> positives, and then I saw the bug report on the upstream parted bug
> tracker the other day where someone claimed to have found a logical
> error that caused atari_probe to return true when it shouldn't.  I
> finally realized it was not an error ( at least I think not ).

According to the documentation, p.43 [1], I think offset $01 should always
contain either "GEM", "BGM" or "XGM". They don't seem to be optional.

I guess adding those to atari_probe() should help improve the detection
quite a bit. But I would have to test that first.

> I think I will at least move atari to be probed after all other labels
> have failed.

This sounds like a reasonable thing to do until I have worked out
a patch to improve atari_probe().

Adrian

> [1] https://people.debian.org/~glaubitz/Atari_HD_File_Sytem_Reference_Guide.pdf

-- 
  .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz-8fiUuRrzOP0dnm+yROfE0A@public.gmane.org
`. `'   Freie Universitaet Berlin - glaubitz-1Olz3AKvcsuAKZTfuerNgRvVK+yQ3ZXh@public.gmane.org
   `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Re: [parted-devel] Atari label false positives

[Richard Z]

[-- Attachment #1: Type: text/plain, Size: 1283 bytes --]

On Fri, May 11, 2018 at 06:10:37PM +0200, John Paul Adrian Glaubitz wrote:

> According to the documentation, p.43 [1], I think offset $01 should always
> contain either "GEM", "BGM" or "XGM". They don't seem to be optional.

The Atari partitioning tables are also used on the Q40/Q60 which breaks
many some assumptions.. no GEM here but instead one of two Q40 specific
types such as "QWA" or "QBD" in the first slot.
BTW I have never seen a low-case or camel-case string in this place,
this should be also used.

> I guess adding those to atari_probe() should help improve the detection
> quite a bit. But I would have to test that first.

It would, however I am not sure what happens if a hard disk is 
repurposed from Atari to PC or the other way around partitioning.
There may be some strings or other data left which could be 
misinterpreted.

> >I think I will at least move atari to be probed after all other labels
> >have failed.

in principle both should be probed, prompting user action if both
find something. They could also return a "score" - based on number of 
partitions recognised, their sizes and other info to give a hint which 
one is likely a non-random hit.


Richard

-- 
Name and OpenPGP keys available from pgp key servers


[-- Attachment #2: Type: application/pgp-signature, Size: 811 bytes --]

Re: [parted-devel] Atari label false positives

[Michael Schmitz]

Hi Phillip,

Am 12.05.2018 um 03:56 schrieb Phillip Susi:
> On 5/11/2018 11:29 AM, John Paul Adrian Glaubitz wrote:
>> Thanks for digging this up. I wasn't aware of this particular issue before
>> and I agree, this needs to be addressed. I hope that I didn't cause too
>> many DOS partitions to be misdetected. Sorry for the inconvenience if I
>> did. The Atari partition table code has been working great for
>> Debian/m68k so
>> far though.
> 
> I have seen at least one or two reports of Ubuntu users getting false
> positives, and then I saw the bug report on the upstream parted bug
> tracker the other day where someone claimed to have found a logical
> error that caused atari_probe to return true when it shouldn't.  I
> finally realized it was not an error ( at least I think not ).
> 
> I think I will at least move atari to be probed after all other labels
> have failed.

The link order in the kernel (atari before msdos) has not been chosen at
random. The msdos partition format probe will always succeed on any
valid Atari partition table, while Atari format probe should not succeed
on plain MSDOS partitions.

Do the reporters if these bugs have Atari partition format enabled in
their kernels? I would expect the same false positives to happen there,
if the atari_probe() code is functionally equivalent to the kernel
partition probe code.
Do you have a dump of the root sector (plus additional eventual extended
partition root sector) from these bugs?

Regarding some of your other questions: The presence of extended
partitions is entirely optional (though highly likely). A partition ID
of XGM for one of the primary partitions, and for at least one of the
extended subpartitions (if extended partitions are present) is
distinctive for AHDI extended partition format. Seeing this condition
fulfilled by a MSDOS partition would seem quite unlikely. I run a mix of
AHDI and MSDOS partition format disks on my Atari and have never seen
one misidentified by the kernel.

Presence of the ICD extended partitions block at offset 0x156 of the
device root sector (with only IDs of GEM, BGM, RAW, LNX or SWP expected)
is distinctive for ICD partitions, and even more unlikely to encounter
at random.

Leaves disks without extended partitions prone to misidentifcation.
If you want to make doubly sure there, look for at least one of the
partition IDs commonly used in Atari format partitions (GEM, BGM, RAW,
LNX, SWP), and additionally those identified by Richard for Q40.

If that's still not enough, score both probe results according to the
number of partitions identified, and allow the user to override.

Repurposing disks that had formerly been used on Atari elsewhere should
begin by overwriting the root sector IMO.

Cheers,

	Michael

Re: Atari label false positives

[John Paul Adrian Glaubitz]

On 05/12/2018 01:17 AM, Michael Schmitz wrote:
> The link order in the kernel (atari before msdos) has not been chosen at
> random. The msdos partition format probe will always succeed on any
> valid Atari partition table, while Atari format probe should not succeed
> on plain MSDOS partitions.
> 
> Do the reporters if these bugs have Atari partition format enabled in
> their kernels? I would expect the same false positives to happen there,
> if the atari_probe() code is functionally equivalent to the kernel
> partition probe code.
> Do you have a dump of the root sector (plus additional eventual extended
> partition root sector) from these bugs?

This is a very good point. We should have a look at the probing code in
the kernel because apparently the problem doesn't exist there.

And, yes, it would be interesting to get a dump of these boot sectors as
I haven't run into the problem myself and would be very curios why
the misdetection happens.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz-8fiUuRrzOP0dnm+yROfE0A@public.gmane.org
`. `'   Freie Universitaet Berlin - glaubitz-1Olz3AKvcsuAKZTfuerNgRvVK+yQ3ZXh@public.gmane.org
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Re: [parted-devel] Atari label false positives

[Michael Schmitz]

Hi Adrian,

Am 12.05.2018 um 19:59 schrieb John Paul Adrian Glaubitz:
> On 05/12/2018 01:17 AM, Michael Schmitz wrote:
>> The link order in the kernel (atari before msdos) has not been chosen at
>> random. The msdos partition format probe will always succeed on any
>> valid Atari partition table, while Atari format probe should not succeed
>> on plain MSDOS partitions.

I had that the wrong way - link order is not what matters here, the
checks order has msdos before atari in fact.

>>
>> Do the reporters if these bugs have Atari partition format enabled in
>> their kernels? I would expect the same false positives to happen there,
>> if the atari_probe() code is functionally equivalent to the kernel
>> partition probe code.
>> Do you have a dump of the root sector (plus additional eventual extended
>> partition root sector) from these bugs?
> 
> This is a very good point. We should have a look at the probing code in
> the kernel because apparently the problem doesn't exist there.

It may not exist because the MSDOS partition detection is specific
enough to reject Atari partition tables. But still, compare the kernel
partition parsers to see what may be missing.

Cheers,

	Michael

> And, yes, it would be interesting to get a dump of these boot sectors as
> I haven't run into the problem myself and would be very curios why
> the misdetection happens.
> 
> Adrian
> 

Re: Atari label false positives

[John Paul Adrian Glaubitz]

On 05/12/2018 10:41 AM, Michael Schmitz wrote:
>>> Do you have a dump of the root sector (plus additional eventual extended
>>> partition root sector) from these bugs?
>>
>> This is a very good point. We should have a look at the probing code in
>> the kernel because apparently the problem doesn't exist there.
> 
> It may not exist because the MSDOS partition detection is specific
> enough to reject Atari partition tables. But still, compare the kernel
> partition parsers to see what may be missing.

Which would mean that the kernel probes for DOS partition tables before
Atari partition tables, wouldn't it?

Adrian

-- 
  .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz-8fiUuRrzOP0dnm+yROfE0A@public.gmane.org
`. `'   Freie Universitaet Berlin - glaubitz-1Olz3AKvcsuAKZTfuerNgRvVK+yQ3ZXh@public.gmane.org
   `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Re: [parted-devel] Atari label false positives

[Michael Schmitz]

Hi Adrian,

Am 12.05.2018 um 23:24 schrieb John Paul Adrian Glaubitz:
> On 05/12/2018 10:41 AM, Michael Schmitz wrote:
>>>> Do you have a dump of the root sector (plus additional eventual
>>>> extended
>>>> partition root sector) from these bugs?
>>>
>>> This is a very good point. We should have a look at the probing code in
>>> the kernel because apparently the problem doesn't exist there.
>>
>> It may not exist because the MSDOS partition detection is specific
>> enough to reject Atari partition tables. But still, compare the kernel
>> partition parsers to see what may be missing.
> 
> Which would mean that the kernel probes for DOS partition tables before
> Atari partition tables, wouldn't it?

Yes, it does (as I had stated earlier - block/partitions/checks.c has
only a few rather more specific formats listed before MSDOS. Sorry if
that wasn't clear.).

> 
> Adrian
> 

Re: Atari label false positives

[Phillip Susi]

[-- Attachment #1.1: Type: text/plain, Size: 491 bytes --]

On 5/12/2018 4:41 AM, Michael Schmitz wrote:
> I had that the wrong way - link order is not what matters here, the
> checks order has msdos before atari in fact.

Ok, I was going to say the dos table requires the 55AA signature that is
not expected to be found in an atari table.

One other thing I thought of is the atari table lists the size of the
disk.  Right now it just has to be less than or equal to the actual disk
size.  What if it was required to be strictly equal?



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 473 bytes --]