Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[schmitzmic]

>From 5299e0e64dfb33ac3a1f3137b42178734ce20087 Mon Sep 17 00:00:00 2001

The Amiga RDB partition parser module uses int for partition sector
address and count, which will overflow for disks 2 TB and larger.

Use sector_t as type for sector address and size (as expected by
put_partition) to allow using such disks without danger of data
corruption.

This bug was reported originally in 2012 by Martin Steigerwald
<Martin@lichtvoll.de>, and the fix was created by the RDB author,
Joanne Dow <jdow@earthlink.net>. The patch had been discussed and
reviewed on linux-m68k at that time but never officially submitted.

Following a stern warning by Joanne, a warning is printed if any
partition is found to overflow the old 32 bit calculations, on the
grounds that such a partition would be misparses on legacy 32 bit
systems (other than Linux).

Fixes: https://bugzilla.kernel.org/show_bug.cgi?id=43511
Reported-by: Martin Steigerwald <Martin@lichtvoll.de>
Message-ID: <201206192146.09327.Martin@lichtvoll.de>
Signed-off-by: Michael Schmitz <schmitzmic@gmail.com>
Tested-by: Martin Steigerwald <Martin@lichtvoll.de>
Tested-by: Michael Schmitz <schmitzmic@gmail.com>
---
 block/partitions/amiga.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/block/partitions/amiga.c b/block/partitions/amiga.c
index 5609366..42c3f38 100644
--- a/block/partitions/amiga.c
+++ b/block/partitions/amiga.c
@@ -32,7 +32,8 @@ int amiga_partition(struct parsed_partitions *state)
 	unsigned char *data;
 	struct RigidDiskBlock *rdb;
 	struct PartitionBlock *pb;
-	int start_sect, nr_sects, blk, part, res = 0;
+	sector_t start_sect, nr_sects;
+	int blk, part, res = 0;
 	int blksize = 1;	/* Multiplier for disk block size */
 	int slot = 1;
 	char b[BDEVNAME_SIZE];
@@ -111,6 +112,16 @@ int amiga_partition(struct parsed_partitions *state)
 			     be32_to_cpu(pb->pb_Environment[3]) *
 			     be32_to_cpu(pb->pb_Environment[5]) *
 			     blksize;
+		if (start_sect > INT_MAX || nr_sects > INT_MAX
+			|| (start_sect + nr_sects) > INT_MAX) {
+			pr_err("%s: Warning: RDB partition overflow!\n",
+				bdevname(state->bdev, b));
+			pr_err("%s: start 0x%llX size 0x%llX\n",
+				bdevname(state->bdev, b), start_sect,
+				nr_sects);
+			pr_err("%s: partition incompatible with 32 bit OS\n",
+				bdevname(state->bdev, b));
+		}
 		put_partition(state,slot++,start_sect,nr_sects);
 		{
 			/* Be even more informative to aid mounting */
-- 
1.9.1

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[Martin Steigerwald]

schmitzmic@gmail.com - 27.06.18, 03:24:
> From 5299e0e64dfb33ac3a1f3137b42178734ce20087 Mon Sep 17 00:00:00 2001
>=20
> The Amiga RDB partition parser module uses int for partition sector
> address and count, which will overflow for disks 2 TB and larger.
>=20
> Use sector_t as type for sector address and size (as expected by
> put_partition) to allow using such disks without danger of data
> corruption.
>=20
> This bug was reported originally in 2012 by Martin Steigerwald
> <Martin@lichtvoll.de>, and the fix was created by the RDB author,
> Joanne Dow <jdow@earthlink.net>. The patch had been discussed and
> reviewed on linux-m68k at that time but never officially submitted.
>=20
> Following a stern warning by Joanne, a warning is printed if any
> partition is found to overflow the old 32 bit calculations, on the
> grounds that such a partition would be misparses on legacy 32 bit
> systems (other than Linux).
>=20
> Fixes: https://bugzilla.kernel.org/show_bug.cgi?id=3D43511
> Reported-by: Martin Steigerwald <Martin@lichtvoll.de>
> Message-ID: <201206192146.09327.Martin@lichtvoll.de>
> Signed-off-by: Michael Schmitz <schmitzmic@gmail.com>
> Tested-by: Martin Steigerwald <Martin@lichtvoll.de>
> Tested-by: Michael Schmitz <schmitzmic@gmail.com>
> ---
>  block/partitions/amiga.c | 13 ++++++++++++-
>  1 file changed, 12 insertions(+), 1 deletion(-)
>=20
> diff --git a/block/partitions/amiga.c b/block/partitions/amiga.c
> index 5609366..42c3f38 100644
> --- a/block/partitions/amiga.c
> +++ b/block/partitions/amiga.c
> @@ -32,7 +32,8 @@ int amiga_partition(struct parsed_partitions *state)
> unsigned char *data;
>  	struct RigidDiskBlock *rdb;
>  	struct PartitionBlock *pb;
> -	int start_sect, nr_sects, blk, part, res =3D 0;
> +	sector_t start_sect, nr_sects;
> +	int blk, part, res =3D 0;
>  	int blksize =3D 1;	/* Multiplier for disk block size */
>  	int slot =3D 1;
>  	char b[BDEVNAME_SIZE];
> @@ -111,6 +112,16 @@ int amiga_partition(struct parsed_partitions
> *state) be32_to_cpu(pb->pb_Environment[3]) *
>  			     be32_to_cpu(pb->pb_Environment[5]) *
>  			     blksize;
> +		if (start_sect > INT_MAX || nr_sects > INT_MAX
> +			|| (start_sect + nr_sects) > INT_MAX) {
> +			pr_err("%s: Warning: RDB partition overflow!\n",
> +				bdevname(state->bdev, b));
> +			pr_err("%s: start 0x%llX size 0x%llX\n",
> +				bdevname(state->bdev, b), start_sect,
> +				nr_sects);
> +			pr_err("%s: partition incompatible with 32 bit OS\n",
> +				bdevname(state->bdev, b));
> +		}

I do think the wording of that warning is inaccurate, as outlined in my=20
other mails in thread "Re: moving affs + RDB partition support to=20
staging?" just a few minutes ago (see there for a more complete=20
reasoning). I=B4d word it like this:

partition needs 64 bit disk device support in AmigaOS or AmigaOS like=20
operating systems (NSD64, TD64 or SCSI direct)

I think I would not include any more details, and let Amiga people=20
research what they need and since when it is included officially on=20
their own. As there are at least three variants out there: AmigaOS,=20
MorphOS, AROS.

AmigaOS 4 at least can handle disks of 2 TB size or more. I do not think=20
the wording "RDB overflow" is right either.

http://wiki.amigaos.net/wiki/RDB

=46ilesystem size limits are a different matter.

http://www.amigawiki.de/doku.php?id=3Dde:system:filesystems_limits

>  		put_partition(state,slot++,start_sect,nr_sects);
>  		{
>  			/* Be even more informative to aid mounting */

Thanks,
=2D-=20
Martin

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[Martin Steigerwald]

Thanks a lot again for your patch.

schmitzmic@gmail.com - 27.06.18, 03:24:
> +               if (start_sect > INT_MAX || nr_sects > INT_MAX
> +                       || (start_sect + nr_sects) > INT_MAX) {
> +                       pr_err("%s: Warning: RDB partition
> overflow!\n", +                               bdevname(state->bdev,

I=B4d word this:

Warning: RDB partition 32-bit overflow

AmigaOS developers can do 64 bit math on a 32 bit operating system. Just=20
like Linux can.

> b));
> +                       pr_err("%s: start 0x%llX size 0x%llX\n",
> +                               bdevname(state->bdev, b), start_sect,
> +                               nr_sects);
> +                       pr_err("%s: partition incompatible with 32 bit
> OS\n", +                               bdevname(state->bdev, b));
> +               }

And as stated in my other reply to the patch:

partition needs 64 bit disk device support in AmigaOS or AmigaOS like=20
operating systems (NSD64, TD64 or SCSI direct)

see my other reply to the patch and my other mails in the
"Re: moving affs + RDB partition support to staging?" thread as to why.=20
And for references.

Thanks,
=2D-=20
Martin

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[Geert Uytterhoeven]

Hi Michael,

Thanks for your patch!

On Wed, Jun 27, 2018 at 4:47 AM <schmitzmic@gmail.com> wrote:
> From 5299e0e64dfb33ac3a1f3137b42178734ce20087 Mon Sep 17 00:00:00 2001

??

> The Amiga RDB partition parser module uses int for partition sector
> address and count, which will overflow for disks 2 TB and larger.
>
> Use sector_t as type for sector address and size (as expected by
> put_partition) to allow using such disks without danger of data
> corruption.

Note that sector_t is not guaranteed to be 64-bit:

    #ifdef CONFIG_LBDAF
    typedef u64 sector_t;
    typedef u64 blkcnt_t;
    #else
    typedef unsigned long sector_t;
    typedef unsigned long blkcnt_t;
    #endif

And it seems CONFIG_LBDAF can still be disabled on 32-bit...

> This bug was reported originally in 2012 by Martin Steigerwald
> <Martin@lichtvoll.de>, and the fix was created by the RDB author,
> Joanne Dow <jdow@earthlink.net>. The patch had been discussed and
> reviewed on linux-m68k at that time but never officially submitted.
>
> Following a stern warning by Joanne, a warning is printed if any
> partition is found to overflow the old 32 bit calculations, on the
> grounds that such a partition would be misparses on legacy 32 bit
> systems (other than Linux).
>
> Fixes: https://bugzilla.kernel.org/show_bug.cgi?id=43511
> Reported-by: Martin Steigerwald <Martin@lichtvoll.de>
> Message-ID: <201206192146.09327.Martin@lichtvoll.de>
> Signed-off-by: Michael Schmitz <schmitzmic@gmail.com>
> Tested-by: Martin Steigerwald <Martin@lichtvoll.de>
> Tested-by: Michael Schmitz <schmitzmic@gmail.com>
> ---
>  block/partitions/amiga.c | 13 ++++++++++++-
>  1 file changed, 12 insertions(+), 1 deletion(-)
>
> diff --git a/block/partitions/amiga.c b/block/partitions/amiga.c
> index 5609366..42c3f38 100644
> --- a/block/partitions/amiga.c
> +++ b/block/partitions/amiga.c
> @@ -32,7 +32,8 @@ int amiga_partition(struct parsed_partitions *state)
>         unsigned char *data;
>         struct RigidDiskBlock *rdb;
>         struct PartitionBlock *pb;
> -       int start_sect, nr_sects, blk, part, res = 0;
> +       sector_t start_sect, nr_sects;

As sector_t can still be 32-bit, I think you should use an explicit u64 here.

> +       int blk, part, res = 0;
>         int blksize = 1;        /* Multiplier for disk block size */
>         int slot = 1;
>         char b[BDEVNAME_SIZE];
> @@ -111,6 +112,16 @@ int amiga_partition(struct parsed_partitions *state)
>                              be32_to_cpu(pb->pb_Environment[3]) *
>                              be32_to_cpu(pb->pb_Environment[5]) *
>                              blksize;

Without adding any unsigned long long or ULL stuff to the calculations
for start_sect and nr_sects above, the math will still be done using 32-bit
arithmetic. Or am I missing something?

> +               if (start_sect > INT_MAX || nr_sects > INT_MAX
> +                       || (start_sect + nr_sects) > INT_MAX) {
> +                       pr_err("%s: Warning: RDB partition overflow!\n",
> +                               bdevname(state->bdev, b));
> +                       pr_err("%s: start 0x%llX size 0x%llX\n",
> +                               bdevname(state->bdev, b), start_sect,
> +                               nr_sects);
> +                       pr_err("%s: partition incompatible with 32 bit OS\n",
> +                               bdevname(state->bdev, b));
> +               }

I don't know if the check above is really needed here.
There's also int vs. unsigned int. But see below.

>                 put_partition(state,slot++,start_sect,nr_sects);

Given sector_t may be 32-bit, values may be truncated when calling
put_partition(), so you need to check for that.

Interestingly, even partition parsers that do use u64 (efi, ldm) or loff_t
(ibm) do not have such checks.

Perhaps put_partition() should take u64, and print a warning and ignore the
partition if conversion to sector_t involves truncation?

>                 {
>                         /* Be even more informative to aid mounting */

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[Michael Schmitz]

Hi Martin,

thanks for your comments.

On Wed, Jun 27, 2018 at 8:24 PM, Martin Steigerwald <martin@lichtvoll.de> w=
rote:
> Thanks a lot again for your patch.
>
> schmitzmic@gmail.com - 27.06.18, 03:24:
>> +               if (start_sect > INT_MAX || nr_sects > INT_MAX
>> +                       || (start_sect + nr_sects) > INT_MAX) {
>> +                       pr_err("%s: Warning: RDB partition
>> overflow!\n", +                               bdevname(state->bdev,
>
> I=C2=B4d word this:
>
> Warning: RDB partition 32-bit overflow
>
> AmigaOS developers can do 64 bit math on a 32 bit operating system. Just
> like Linux can.

Yes, I realize that. I hadn't gone back through all the mails on the
subject to find out what the exact requrements are on the AmigaOS
side.

Just trying to be as terse as possible to keep checkpatch happy :-(

>
>> b));
>> +                       pr_err("%s: start 0x%llX size 0x%llX\n",
>> +                               bdevname(state->bdev, b), start_sect,
>> +                               nr_sects);
>> +                       pr_err("%s: partition incompatible with 32 bit
>> OS\n", +                               bdevname(state->bdev, b));
>> +               }
>
> And as stated in my other reply to the patch:
>
> partition needs 64 bit disk device support in AmigaOS or AmigaOS like
> operating systems (NSD64, TD64 or SCSI direct)

I'd probably leave it at 'disk needs 64 bit disk device support on
native OS', and only print that warning once.

Geert has raised another important point about 64 bt device support -
all this is moot when the Linux kernel wasn't built with large block
device support enabled (you'd get the same buggy behaviour as before
the patch there).

> see my other reply to the patch and my other mails in the
> "Re: moving affs + RDB partition support to staging?" thread as to why.
> And for references.

Thanks for collating all the references. Please understand that I
can't read all of that, and as a simple patch mechanic I won't even
try to grasp all the subtleties of RDB (I don't even own an Amiga so I
am quite unlikey to ever use this code path).
But please also understand that for that reason, I take Joanne's
advice about backwards compatibility very serious. My patch (actually
Joanne's originally) changes kernel behaviour from what we consider
broken (allowing 32 bit overflow in partition address calculations) to
what we think is the right thing to do. But there might be someone out
there who used the current behaviour to craft a RDB that aliows two
separate sets of partitions to coexist on the same disk (one set
visible to 32 bit disk drivers, before the 32 bit overflow mark, and a
second set above that mark, visible only to 64 bit drivers. Silently
changing our parser behaviour might cause said user to now trash data
past the overflow mark.). This is a little contrived, and perhaps I am
overcomplicating matters (again), but can't be ruled out.

In the interest of least surprises, we have to fix the 32 bit overflow
(so we can even detect that it would have happened), and give the user
the chance to carefully consider whether to accept the new behaviour.
That means refusing to make available any partition that would have
been affected by such overflow.

The user has then all options available - force old behaviour by using
an older kernel, override the parser to force new behaviour (which we
all assume is correct), or leave the disk well alone.

Cheers,

  Michael

> Thanks,
> --
> Martin
>
>

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[Michael Schmitz]

Hi Geert,

thanks for your feedback!

On Thu, Jun 28, 2018 at 1:30 AM, Geert Uytterhoeven
<geert@linux-m68k.org> wrote:
> Hi Michael,
>
> Thanks for your patch!
>
> On Wed, Jun 27, 2018 at 4:47 AM <schmitzmic@gmail.com> wrote:
>> From 5299e0e64dfb33ac3a1f3137b42178734ce20087 Mon Sep 17 00:00:00 2001
>
> ??

Comes from not using git send-email. Don't ask ...

>> The Amiga RDB partition parser module uses int for partition sector
>> address and count, which will overflow for disks 2 TB and larger.
>>
>> Use sector_t as type for sector address and size (as expected by
>> put_partition) to allow using such disks without danger of data
>> corruption.
>
> Note that sector_t is not guaranteed to be 64-bit:
>
>     #ifdef CONFIG_LBDAF
>     typedef u64 sector_t;
>     typedef u64 blkcnt_t;
>     #else
>     typedef unsigned long sector_t;
>     typedef unsigned long blkcnt_t;
>     #endif

Yes, I had seen that ...

> And it seems CONFIG_LBDAF can still be disabled on 32-bit...

Ouch - missed that bit.

>
>> This bug was reported originally in 2012 by Martin Steigerwald
>> <Martin@lichtvoll.de>, and the fix was created by the RDB author,
>> Joanne Dow <jdow@earthlink.net>. The patch had been discussed and
>> reviewed on linux-m68k at that time but never officially submitted.
>>
>> Following a stern warning by Joanne, a warning is printed if any
>> partition is found to overflow the old 32 bit calculations, on the
>> grounds that such a partition would be misparses on legacy 32 bit
>> systems (other than Linux).
>>
>> Fixes: https://bugzilla.kernel.org/show_bug.cgi?id=43511
>> Reported-by: Martin Steigerwald <Martin@lichtvoll.de>
>> Message-ID: <201206192146.09327.Martin@lichtvoll.de>
>> Signed-off-by: Michael Schmitz <schmitzmic@gmail.com>
>> Tested-by: Martin Steigerwald <Martin@lichtvoll.de>
>> Tested-by: Michael Schmitz <schmitzmic@gmail.com>
>> ---
>>  block/partitions/amiga.c | 13 ++++++++++++-
>>  1 file changed, 12 insertions(+), 1 deletion(-)
>>
>> diff --git a/block/partitions/amiga.c b/block/partitions/amiga.c
>> index 5609366..42c3f38 100644
>> --- a/block/partitions/amiga.c
>> +++ b/block/partitions/amiga.c
>> @@ -32,7 +32,8 @@ int amiga_partition(struct parsed_partitions *state)
>>         unsigned char *data;
>>         struct RigidDiskBlock *rdb;
>>         struct PartitionBlock *pb;
>> -       int start_sect, nr_sects, blk, part, res = 0;
>> +       sector_t start_sect, nr_sects;
>
> As sector_t can still be 32-bit, I think you should use an explicit u64 here.

You're spot on there.

>
>> +       int blk, part, res = 0;
>>         int blksize = 1;        /* Multiplier for disk block size */
>>         int slot = 1;
>>         char b[BDEVNAME_SIZE];
>> @@ -111,6 +112,16 @@ int amiga_partition(struct parsed_partitions *state)
>>                              be32_to_cpu(pb->pb_Environment[3]) *
>>                              be32_to_cpu(pb->pb_Environment[5]) *
>>                              blksize;
>
> Without adding any unsigned long long or ULL stuff to the calculations
> for start_sect and nr_sects above, the math will still be done using 32-bit
> arithmetic. Or am I missing something?

It did appear to do 64 bit arithmetic alright. But I better check what
instrunctions are used there.
I also need to look up the rules on default type promotion - additions
and subtractions would certainly still do 32 bit arithmetics but
multiplication may be different.

>
>> +               if (start_sect > INT_MAX || nr_sects > INT_MAX
>> +                       || (start_sect + nr_sects) > INT_MAX) {
>> +                       pr_err("%s: Warning: RDB partition overflow!\n",
>> +                               bdevname(state->bdev, b));
>> +                       pr_err("%s: start 0x%llX size 0x%llX\n",
>> +                               bdevname(state->bdev, b), start_sect,
>> +                               nr_sects);
>> +                       pr_err("%s: partition incompatible with 32 bit OS\n",
>> +                               bdevname(state->bdev, b));
>> +               }
>
> I don't know if the check above is really needed here.

It will be once I add a jump to rdb_done there ... The third test may
only be needed if no LBD support is present, but we absolutely have to
bail if the calculation of the partition end sector later on in the
kernel truncates.  Maybe I should take a page out of Christoph's book
and put a BUG() there to realy get people's attention?

> There's also int vs. unsigned int. But see below.

start_sect and nr_sects were int, not unsigned int before so I have to
compare to INT_MAX to see whether the old code would have overflowed.
What am I missing? Are you concerned the comparison will always be
false due to data type? I think I would have seen a warning. Anyway,
'seemed to work as intended' with Martin's test case.

>
>>                 put_partition(state,slot++,start_sect,nr_sects);
>
> Given sector_t may be 32-bit, values may be truncated when calling
> put_partition(), so you need to check for that.

put_partition() is inlined so I'm not sure a cast would help there,
And all that happens in put_partition() is that the start address and
size get stuffed into the parsed_partitions struct:

                p->parts[n].from = from;
                p->parts[n].size = size;

That struct has u32 types for from and size if compiled without
CONFIG_LBDAF, so whatever we do with 64 bit arithmetics, this will
lead to truncation here. Only safe option is to bail out on these
kernels.

> Interestingly, even partition parsers that do use u64 (efi, ldm) or loff_t
> (ibm) do not have such checks.

I would expect things to go horribly wrong if these are used without
LBD support. But perhaps the same checks need to be added there
indeed.

> Perhaps put_partition() should take u64, and print a warning and ignore the
> partition if conversion to sector_t involves truncation?

That's something for Jens to ponder :-)

I'll work in ann the suggested changes and then submit this to
linux-block for real.

Thanks again,

  Michael


>
>>                 {
>>                         /* Be even more informative to aid mounting */
>
> Gr{oetje,eeting}s,
>
>                         Geert
>
> --
> Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
>
> In personal conversations with technical people, I call myself a hacker. But
> when I'm talking to journalists I just say "programmer" or something like that.
>                                 -- Linus Torvalds

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[Martin Steigerwald]

Hi Michael.

Michael Schmitz - 27.06.18, 22:13:
> On Wed, Jun 27, 2018 at 8:24 PM, Martin Steigerwald=20
<martin@lichtvoll.de> wrote:
> > Thanks a lot again for your patch.
> >=20
> > schmitzmic@gmail.com - 27.06.18, 03:24:
> >> +               if (start_sect > INT_MAX || nr_sects > INT_MAX
> >> +                       || (start_sect + nr_sects) > INT_MAX) {
> >> +                       pr_err("%s: Warning: RDB partition
> >> overflow!\n", +                               bdevname(state->bdev,
> >=20
> > I=B4d word this:
> >=20
> > Warning: RDB partition 32-bit overflow
> >=20
> > AmigaOS developers can do 64 bit math on a 32 bit operating system.
> > Just like Linux can.
>=20
> Yes, I realize that. I hadn't gone back through all the mails on the
> subject to find out what the exact requrements are on the AmigaOS
> side.
>=20
> Just trying to be as terse as possible to keep checkpatch happy :-(
>=20
> >> b));
> >> +                       pr_err("%s: start 0x%llX size 0x%llX\n",
> >> +                               bdevname(state->bdev, b),
> >> start_sect,
> >> +                               nr_sects);
> >> +                       pr_err("%s: partition incompatible with 32
> >> bit OS\n", +                               bdevname(state->bdev,
> >> b)); +               }
> >=20
> > And as stated in my other reply to the patch:
> >=20
> > partition needs 64 bit disk device support in AmigaOS or AmigaOS
> > like
> > operating systems (NSD64, TD64 or SCSI direct)
>=20
> I'd probably leave it at 'disk needs 64 bit disk device support on
> native OS', and only print that warning once.

This is fine with me.

> Geert has raised another important point about 64 bt device support -
> all this is moot when the Linux kernel wasn't built with large block
> device support enabled (you'd get the same buggy behaviour as before
> the patch there).
>=20
> > see my other reply to the patch and my other mails in the
> > "Re: moving affs + RDB partition support to staging?" thread as to
> > why. And for references.
>=20
> Thanks for collating all the references. Please understand that I
> can't read all of that, and as a simple patch mechanic I won't even
> try to grasp all the subtleties of RDB (I don't even own an Amiga so I
> am quite unlikey to ever use this code path).

I understand that.

> But please also understand that for that reason, I take Joanne's
> advice about backwards compatibility very serious. My patch (actually
> Joanne's originally) changes kernel behaviour from what we consider
> broken (allowing 32 bit overflow in partition address calculations) to
> what we think is the right thing to do. But there might be someone
> out there who used the current behaviour to craft a RDB that aliows
> two separate sets of partitions to coexist on the same disk (one set
> visible to 32 bit disk drivers, before the 32 bit overflow mark, and
> a second set above that mark, visible only to 64 bit drivers.
> Silently changing our parser behaviour might cause said user to now
> trash data past the overflow mark.). This is a little contrived, and
> perhaps I am overcomplicating matters (again), but can't be ruled
> out.
>=20
> In the interest of least surprises, we have to fix the 32 bit overflow
> (so we can even detect that it would have happened), and give the
> user the chance to carefully consider whether to accept the new
> behaviour. That means refusing to make available any partition that
> would have been affected by such overflow.

That is acceptable for me as I told before. Either mount or refuse to=20
mount, but do not overflow and mount nonetheless :)

Mind you, I am not using my Amiga machines either at the moment. And I=20
repurposed the 2 TB disk years ago.=20
=20
> The user has then all options available - force old behaviour by using
> an older kernel, override the parser to force new behaviour (which we
> all assume is correct), or leave the disk well alone.

Sure.

I would not name the kernel option "eat_my_rdb", but use a less=20
dramatizing name.

Maybe just: "allow_64bit_rdb" or something like that.

How does the user come to know about this kernel option? Will you print=20
its name in kernel log?

Thanks,
=2D-=20
Martin

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[jdow]

Three issues exist here in two different places.

As far as a 32 TG disk is concerned RDBs can describe it and mount it safely - 
sort of - modulo the following issues. They are not a problem, I believe, with 
Amiga OSs new enough to understand RDBs. I cannot prove that. They are not 
sufficient, apparently, for Linux, which is why an __int64 equivalent is needed 
rather than a int equivalent.

As far as 4GB is concerned you are limited at 2 GB by int fseek( int ). This is 
not an RDB issue. Take it up with the filesystems. Linux seems to "think" 
differently than AmigaDOS.

As far as going over 4GB disk size you are limited within the OS by anything 
that deals with byte rather than block calculations for position on disk 
calculations. I think everything speaks properly within the OS to handle at 
least up to 8.7 GB partition sizes. For AmigaDOS we're probably OK up to 32 TB 
(with 8k block size). It some Joe somewhere needs 512 byte block size some other 
partition format is needed. RDBs will not handle it without changes which MAY 
cause RDB interpretation issues. (Change block headers and the spirit of RDBs 
can be preserved safely. But - why? GPT exists, is tested, works nicely. One 
must be careful with it to make sure it supports required features. Note that a 
loadable filesystem is nice. It can make a NEARLY impossible to kill home for 
malware. But it does allow patching in newer filesystems with some backwards 
compatibility. Does GPT have a way to support this? Does AmigaDOS support a 
clean way to clean a disk of a corrupted filesystem image?)

The changes to READING the RDBs for Linux are "obvious", __int64 or equivalent. 
That change is sane and clean. (Be careful writing RDBs mkfs-(amigafs). But 
that's a given. We're adults here.) Changes to make Amigas handle 64 bit 
filesystems are "easy". They just won't necessarily be safe past 4 G blocks. I 
suspect if file sizes are kept under 2GiB a compatibility layer can even keep 
the OS thinking it is on a disk it understands and to it safely.

Linux apparently needs the change. It is not an RDB change; it is an RDB parser 
change. Changing RDB, however, needs care. And at least to my perceptions this 
discussion wandered around the Linux RDB parser and actual RDB changes. These 
need separate discussions as to compatibility issues with any AmigaOS that may 
have a chance of even "tasting" the disk let alone "chewing" on the disk.

On the RDB changes issue, I would recommend disappointing an Amigoid (as utterly 
death defying as that may be) by his not being able to mount a disk rather than 
angering that same Amigoid by damaging or losing his data. That's simply life 
preservation on the part of the designers.

{^_^}

On 20180627 01:13, Martin Steigerwald wrote:
> schmitzmic@gmail.com - 27.06.18, 03:24:
>>  From 5299e0e64dfb33ac3a1f3137b42178734ce20087 Mon Sep 17 00:00:00 2001
>>
>> The Amiga RDB partition parser module uses int for partition sector
>> address and count, which will overflow for disks 2 TB and larger.
>>
>> Use sector_t as type for sector address and size (as expected by
>> put_partition) to allow using such disks without danger of data
>> corruption.
>>
>> This bug was reported originally in 2012 by Martin Steigerwald
>> <Martin@lichtvoll.de>, and the fix was created by the RDB author,
>> Joanne Dow <jdow@earthlink.net>. The patch had been discussed and
>> reviewed on linux-m68k at that time but never officially submitted.
>>
>> Following a stern warning by Joanne, a warning is printed if any
>> partition is found to overflow the old 32 bit calculations, on the
>> grounds that such a partition would be misparses on legacy 32 bit
>> systems (other than Linux).
>>
>> Fixes: https://bugzilla.kernel.org/show_bug.cgi?id=43511
>> Reported-by: Martin Steigerwald <Martin@lichtvoll.de>
>> Message-ID: <201206192146.09327.Martin@lichtvoll.de>
>> Signed-off-by: Michael Schmitz <schmitzmic@gmail.com>
>> Tested-by: Martin Steigerwald <Martin@lichtvoll.de>
>> Tested-by: Michael Schmitz <schmitzmic@gmail.com>
>> ---
>>   block/partitions/amiga.c | 13 ++++++++++++-
>>   1 file changed, 12 insertions(+), 1 deletion(-)
>>
>> diff --git a/block/partitions/amiga.c b/block/partitions/amiga.c
>> index 5609366..42c3f38 100644
>> --- a/block/partitions/amiga.c
>> +++ b/block/partitions/amiga.c
>> @@ -32,7 +32,8 @@ int amiga_partition(struct parsed_partitions *state)
>> unsigned char *data;
>>   	struct RigidDiskBlock *rdb;
>>   	struct PartitionBlock *pb;
>> -	int start_sect, nr_sects, blk, part, res = 0;
>> +	sector_t start_sect, nr_sects;
>> +	int blk, part, res = 0;
>>   	int blksize = 1;	/* Multiplier for disk block size */
>>   	int slot = 1;
>>   	char b[BDEVNAME_SIZE];
>> @@ -111,6 +112,16 @@ int amiga_partition(struct parsed_partitions
>> *state) be32_to_cpu(pb->pb_Environment[3]) *
>>   			     be32_to_cpu(pb->pb_Environment[5]) *
>>   			     blksize;
>> +		if (start_sect > INT_MAX || nr_sects > INT_MAX
>> +			|| (start_sect + nr_sects) > INT_MAX) {
>> +			pr_err("%s: Warning: RDB partition overflow!\n",
>> +				bdevname(state->bdev, b));
>> +			pr_err("%s: start 0x%llX size 0x%llX\n",
>> +				bdevname(state->bdev, b), start_sect,
>> +				nr_sects);
>> +			pr_err("%s: partition incompatible with 32 bit OS\n",
>> +				bdevname(state->bdev, b));
>> +		}
> 
> I do think the wording of that warning is inaccurate, as outlined in my
> other mails in thread "Re: moving affs + RDB partition support to
> staging?" just a few minutes ago (see there for a more complete
> reasoning). I´d word it like this:
> 
> partition needs 64 bit disk device support in AmigaOS or AmigaOS like
> operating systems (NSD64, TD64 or SCSI direct)
> 
> I think I would not include any more details, and let Amiga people
> research what they need and since when it is included officially on
> their own. As there are at least three variants out there: AmigaOS,
> MorphOS, AROS.
> 
> AmigaOS 4 at least can handle disks of 2 TB size or more. I do not think
> the wording "RDB overflow" is right either.
> 
> http://wiki.amigaos.net/wiki/RDB
> 
> Filesystem size limits are a different matter.
> 
> http://www.amigawiki.de/doku.php?id=de:system:filesystems_limits
> 
>>   		put_partition(state,slot++,start_sect,nr_sects);
>>   		{
>>   			/* Be even more informative to aid mounting */
> 
> Thanks,
> 

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[jdow]

Oops! It MUST be 64 bit or everything go boom.

{O.O}

On 20180627 06:30, Geert Uytterhoeven wrote:
> Hi Michael,
> 
> Thanks for your patch!
> 
> On Wed, Jun 27, 2018 at 4:47 AM <schmitzmic@gmail.com> wrote:
>>  From 5299e0e64dfb33ac3a1f3137b42178734ce20087 Mon Sep 17 00:00:00 2001
> 
> ??
> 
>> The Amiga RDB partition parser module uses int for partition sector
>> address and count, which will overflow for disks 2 TB and larger.
>>
>> Use sector_t as type for sector address and size (as expected by
>> put_partition) to allow using such disks without danger of data
>> corruption.
> 
> Note that sector_t is not guaranteed to be 64-bit:
> 
>      #ifdef CONFIG_LBDAF
>      typedef u64 sector_t;
>      typedef u64 blkcnt_t;
>      #else
>      typedef unsigned long sector_t;
>      typedef unsigned long blkcnt_t;
>      #endif
> 
> And it seems CONFIG_LBDAF can still be disabled on 32-bit...
> 
>> This bug was reported originally in 2012 by Martin Steigerwald
>> <Martin@lichtvoll.de>, and the fix was created by the RDB author,
>> Joanne Dow <jdow@earthlink.net>. The patch had been discussed and
>> reviewed on linux-m68k at that time but never officially submitted.
>>
>> Following a stern warning by Joanne, a warning is printed if any
>> partition is found to overflow the old 32 bit calculations, on the
>> grounds that such a partition would be misparses on legacy 32 bit
>> systems (other than Linux).
>>
>> Fixes: https://bugzilla.kernel.org/show_bug.cgi?id=43511
>> Reported-by: Martin Steigerwald <Martin@lichtvoll.de>
>> Message-ID: <201206192146.09327.Martin@lichtvoll.de>
>> Signed-off-by: Michael Schmitz <schmitzmic@gmail.com>
>> Tested-by: Martin Steigerwald <Martin@lichtvoll.de>
>> Tested-by: Michael Schmitz <schmitzmic@gmail.com>
>> ---
>>   block/partitions/amiga.c | 13 ++++++++++++-
>>   1 file changed, 12 insertions(+), 1 deletion(-)
>>
>> diff --git a/block/partitions/amiga.c b/block/partitions/amiga.c
>> index 5609366..42c3f38 100644
>> --- a/block/partitions/amiga.c
>> +++ b/block/partitions/amiga.c
>> @@ -32,7 +32,8 @@ int amiga_partition(struct parsed_partitions *state)
>>          unsigned char *data;
>>          struct RigidDiskBlock *rdb;
>>          struct PartitionBlock *pb;
>> -       int start_sect, nr_sects, blk, part, res = 0;
>> +       sector_t start_sect, nr_sects;
> 
> As sector_t can still be 32-bit, I think you should use an explicit u64 here.
> 
>> +       int blk, part, res = 0;
>>          int blksize = 1;        /* Multiplier for disk block size */
>>          int slot = 1;
>>          char b[BDEVNAME_SIZE];
>> @@ -111,6 +112,16 @@ int amiga_partition(struct parsed_partitions *state)
>>                               be32_to_cpu(pb->pb_Environment[3]) *
>>                               be32_to_cpu(pb->pb_Environment[5]) *
>>                               blksize;
> 
> Without adding any unsigned long long or ULL stuff to the calculations
> for start_sect and nr_sects above, the math will still be done using 32-bit
> arithmetic. Or am I missing something?
> 
>> +               if (start_sect > INT_MAX || nr_sects > INT_MAX
>> +                       || (start_sect + nr_sects) > INT_MAX) {
>> +                       pr_err("%s: Warning: RDB partition overflow!\n",
>> +                               bdevname(state->bdev, b));
>> +                       pr_err("%s: start 0x%llX size 0x%llX\n",
>> +                               bdevname(state->bdev, b), start_sect,
>> +                               nr_sects);
>> +                       pr_err("%s: partition incompatible with 32 bit OS\n",
>> +                               bdevname(state->bdev, b));
>> +               }
> 
> I don't know if the check above is really needed here.
> There's also int vs. unsigned int. But see below.
> 
>>                  put_partition(state,slot++,start_sect,nr_sects);
> 
> Given sector_t may be 32-bit, values may be truncated when calling
> put_partition(), so you need to check for that.
> 
> Interestingly, even partition parsers that do use u64 (efi, ldm) or loff_t
> (ibm) do not have such checks.
> 
> Perhaps put_partition() should take u64, and print a warning and ignore the
> partition if conversion to sector_t involves truncation?
> 
>>                  {
>>                          /* Be even more informative to aid mounting */
> 
> Gr{oetje,eeting}s,
> 
>                          Geert
> 

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[jdow]

Um. new 64 bit stuff must be invisible to old 32 bit stuff.
{^_^}

On 20180627 14:20, Martin Steigerwald wrote:
> Hi Michael.
> 
> Michael Schmitz - 27.06.18, 22:13:
>> On Wed, Jun 27, 2018 at 8:24 PM, Martin Steigerwald
> <martin@lichtvoll.de> wrote:
>>> Thanks a lot again for your patch.
>>>
>>> schmitzmic@gmail.com - 27.06.18, 03:24:
>>>> +               if (start_sect > INT_MAX || nr_sects > INT_MAX
>>>> +                       || (start_sect + nr_sects) > INT_MAX) {
>>>> +                       pr_err("%s: Warning: RDB partition
>>>> overflow!\n", +                               bdevname(state->bdev,
>>>
>>> I´d word this:
>>>
>>> Warning: RDB partition 32-bit overflow
>>>
>>> AmigaOS developers can do 64 bit math on a 32 bit operating system.
>>> Just like Linux can.
>>
>> Yes, I realize that. I hadn't gone back through all the mails on the
>> subject to find out what the exact requrements are on the AmigaOS
>> side.
>>
>> Just trying to be as terse as possible to keep checkpatch happy :-(
>>
>>>> b));
>>>> +                       pr_err("%s: start 0x%llX size 0x%llX\n",
>>>> +                               bdevname(state->bdev, b),
>>>> start_sect,
>>>> +                               nr_sects);
>>>> +                       pr_err("%s: partition incompatible with 32
>>>> bit OS\n", +                               bdevname(state->bdev,
>>>> b)); +               }
>>>
>>> And as stated in my other reply to the patch:
>>>
>>> partition needs 64 bit disk device support in AmigaOS or AmigaOS
>>> like
>>> operating systems (NSD64, TD64 or SCSI direct)
>>
>> I'd probably leave it at 'disk needs 64 bit disk device support on
>> native OS', and only print that warning once.
> 
> This is fine with me.
> 
>> Geert has raised another important point about 64 bt device support -
>> all this is moot when the Linux kernel wasn't built with large block
>> device support enabled (you'd get the same buggy behaviour as before
>> the patch there).
>>
>>> see my other reply to the patch and my other mails in the
>>> "Re: moving affs + RDB partition support to staging?" thread as to
>>> why. And for references.
>>
>> Thanks for collating all the references. Please understand that I
>> can't read all of that, and as a simple patch mechanic I won't even
>> try to grasp all the subtleties of RDB (I don't even own an Amiga so I
>> am quite unlikey to ever use this code path).
> 
> I understand that.
> 
>> But please also understand that for that reason, I take Joanne's
>> advice about backwards compatibility very serious. My patch (actually
>> Joanne's originally) changes kernel behaviour from what we consider
>> broken (allowing 32 bit overflow in partition address calculations) to
>> what we think is the right thing to do. But there might be someone
>> out there who used the current behaviour to craft a RDB that aliows
>> two separate sets of partitions to coexist on the same disk (one set
>> visible to 32 bit disk drivers, before the 32 bit overflow mark, and
>> a second set above that mark, visible only to 64 bit drivers.
>> Silently changing our parser behaviour might cause said user to now
>> trash data past the overflow mark.). This is a little contrived, and
>> perhaps I am overcomplicating matters (again), but can't be ruled
>> out.
>>
>> In the interest of least surprises, we have to fix the 32 bit overflow
>> (so we can even detect that it would have happened), and give the
>> user the chance to carefully consider whether to accept the new
>> behaviour. That means refusing to make available any partition that
>> would have been affected by such overflow.
> 
> That is acceptable for me as I told before. Either mount or refuse to
> mount, but do not overflow and mount nonetheless :)
> 
> Mind you, I am not using my Amiga machines either at the moment. And I
> repurposed the 2 TB disk years ago.
>   
>> The user has then all options available - force old behaviour by using
>> an older kernel, override the parser to force new behaviour (which we
>> all assume is correct), or leave the disk well alone.
> 
> Sure.
> 
> I would not name the kernel option "eat_my_rdb", but use a less
> dramatizing name.
> 
> Maybe just: "allow_64bit_rdb" or something like that.
> 
> How does the user come to know about this kernel option? Will you print
> its name in kernel log?
> 
> Thanks,
> 

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[jdow]

Error NNNN: Conventional RDBs cannot define more than 4,294,967,296 blocks.
or
Error NNNN: Conventional RDB block count overflow.

That is a HARD limit. The documentation for error NNNN should suggest larger 
logical block (cluster, whatever) sizes as a way out. Of course, block size 
"could" go up to at least 65536 bytes (if the nonsense size 0 is reinterpreted). 
Then Bob's your uncle up to 281 TB. It should also include a dry notation that 
you are best off with a power of 2 block size larger than or equal to the disk's 
actual sector size, if you can learn that.

{^_^}
(Get everybody)

On 20180627 01:24, Martin Steigerwald wrote:
> Thanks a lot again for your patch.
> 
> schmitzmic@gmail.com - 27.06.18, 03:24:
>> +               if (start_sect > INT_MAX || nr_sects > INT_MAX
>> +                       || (start_sect + nr_sects) > INT_MAX) {
>> +                       pr_err("%s: Warning: RDB partition
>> overflow!\n", +                               bdevname(state->bdev,
> 
> I´d word this:
> 
> Warning: RDB partition 32-bit overflow
> 
> AmigaOS developers can do 64 bit math on a 32 bit operating system. Just
> like Linux can.
> 
>> b));
>> +                       pr_err("%s: start 0x%llX size 0x%llX\n",
>> +                               bdevname(state->bdev, b), start_sect,
>> +                               nr_sects);
>> +                       pr_err("%s: partition incompatible with 32 bit
>> OS\n", +                               bdevname(state->bdev, b));
>> +               }
> 
> And as stated in my other reply to the patch:
> 
> partition needs 64 bit disk device support in AmigaOS or AmigaOS like
> operating systems (NSD64, TD64 or SCSI direct)
> 
> see my other reply to the patch and my other mails in the
> "Re: moving affs + RDB partition support to staging?" thread as to why.
> And for references.
> 
> Thanks,
> 

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[Michael Schmitz]

Hi Martin,

Am 28.06.2018 um 09:20 schrieb Martin Steigerwald:
>>> And as stated in my other reply to the patch:
>>>
>>> partition needs 64 bit disk device support in AmigaOS or AmigaOS
>>> like
>>> operating systems (NSD64, TD64 or SCSI direct)
>>
>> I'd probably leave it at 'disk needs 64 bit disk device support on
>> native OS', and only print that warning once.
>
> This is fine with me.

OK, I'll go with that.

>> In the interest of least surprises, we have to fix the 32 bit overflow
>> (so we can even detect that it would have happened), and give the
>> user the chance to carefully consider whether to accept the new
>> behaviour. That means refusing to make available any partition that
>> would have been affected by such overflow.
>
> That is acceptable for me as I told before. Either mount or refuse to
> mount, but do not overflow and mount nonetheless :)
>
> Mind you, I am not using my Amiga machines either at the moment. And I
> repurposed the 2 TB disk years ago.

That's fine - I understand the 'profile' image was a true binary copy of 
the RDB, and placing that file at offset 0 in an image file is a 
legitimate use?

> I would not name the kernel option "eat_my_rdb", but use a less
> dramatizing name.
>
> Maybe just: "allow_64bit_rdb" or something like that.

I don't expect to get away with that :-)

> How does the user come to know about this kernel option? Will you print
> its name in kernel log?

Depends on how easy we want to make it for users. If I put a BUG() trap 
with the check, the resulting log section will point to a specific line 
in block/partitions/amiga.c, from which the override option will be 
obvious. But that might be a little too opaque for some...

Cheers,

	Michael

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[Geert Uytterhoeven]

Hi Michael,

On Thu, Jun 28, 2018 at 6:59 AM Michael Schmitz <schmitzmic@gmail.com> wrote:
> Am 28.06.2018 um 09:20 schrieb Martin Steigerwald:
> >>> And as stated in my other reply to the patch:
> >>> partition needs 64 bit disk device support in AmigaOS or AmigaOS
> >>> like
> >>> operating systems (NSD64, TD64 or SCSI direct)
> >>
> >> I'd probably leave it at 'disk needs 64 bit disk device support on
> >> native OS', and only print that warning once.
> >
> > This is fine with me.
>
> OK, I'll go with that.

Do we really need the warning?
Once the parsing is fixed doing 64-bit math, it does not matter for Linux
anymore.

Won't it make more sense to have the warning in the tool that created the
partition table in the first place?

> > I would not name the kernel option "eat_my_rdb", but use a less
> > dramatizing name.
> >
> > Maybe just: "allow_64bit_rdb" or something like that.
>
> I don't expect to get away with that :-)

I still fail to see what's the added value of the kernel option...
Either the partition is usable, or not.

> > How does the user come to know about this kernel option? Will you print
> > its name in kernel log?
>
> Depends on how easy we want to make it for users. If I put a BUG() trap
> with the check, the resulting log section will point to a specific line
> in block/partitions/amiga.c, from which the override option will be
> obvious. But that might be a little too opaque for some...

Please don't use BUG(), unless your goal is to attract attention (from
Linus, who dislikes BUG()!).

Using BUG() would be a nice way to DoS someones machine by plugging in
a USB stick with a malformed RDB.

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[Martin Steigerwald]

Hi Geert.

Geert Uytterhoeven - 28.06.18, 08:45:=20
> On Thu, Jun 28, 2018 at 6:59 AM Michael Schmitz <schmitzmic@gmail.com>=20
wrote:
> > Am 28.06.2018 um 09:20 schrieb Martin Steigerwald:
> > >>> And as stated in my other reply to the patch:
> > >>> partition needs 64 bit disk device support in AmigaOS or AmigaOS
> > >>> like
> > >>> operating systems (NSD64, TD64 or SCSI direct)
> > >>=20
> > >> I'd probably leave it at 'disk needs 64 bit disk device support
> > >> on
> > >> native OS', and only print that warning once.
> > >=20
> > > This is fine with me.
> >=20
> > OK, I'll go with that.
>=20
> Do we really need the warning?
> Once the parsing is fixed doing 64-bit math, it does not matter for
> Linux anymore.

Well, irony of this is: In my case the RDB has been created on a machine=20
with a native OS. So Linux warns me about something I already did so on=20
the native OS without any warning. In this case AmigaOS 4.0.
=20
> Won't it make more sense to have the warning in the tool that created
> the partition table in the first place?

Well that would be up to the AmigaOS developers to decide.

And well for amiga-fdisk or parted developers if they ever choose to=20
support this or already do. (I doubt that amiga-fdisk can handle this.)

> > > I would not name the kernel option "eat_my_rdb", but use a less
> > > dramatizing name.
> > >=20
> > > Maybe just: "allow_64bit_rdb" or something like that.
> >=20
> > I don't expect to get away with that :-)
>=20
> I still fail to see what's the added value of the kernel option...
> Either the partition is usable, or not.

Well, I could try to contact some of the current AmigaOS developers=20
about that and ask them whether they would like to give me a statement=20
about this that I am allowed to post here.

I would not know whether they answer and it may take a time. My offer=20
stands, but I would only do this, if you really like to have that=20
official feedback.

Again, I am pretty sure that what I did is safe on AmigaOS 4 at least,=20
but I bet also on AmigaOS <4 with NSD64 or TD64 (except for the=20
filesystem sizes, but AmigaOS < 4 does not have JXFS anyway, and did not=20
have SFS2 as well, maybe that is available now, I don=B4t know).

However Joanne is without doubt an authority on RDBs, but she has not=20
been involved with AmigaOS development for quite some time and, correct=20
me if this is wrong, Joanne, does not know as much about the recent=20
versions, as I or even more so as current AmigaOS developers know.

Thanks,
=2D-=20
Martin

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[Martin Steigerwald]

Hi Michael.

Probably I was right with not submitting a patch myself. I=C2=B4d likely=20
would have been overwhelmed by the discussion and feedback :)

Michael Schmitz - 28.06.18, 06:58:
[=E2=80=A6]
> >> In the interest of least surprises, we have to fix the 32 bit
> >> overflow (so we can even detect that it would have happened), and
> >> give the user the chance to carefully consider whether to accept
> >> the new behaviour. That means refusing to make available any
> >> partition that would have been affected by such overflow.
> >=20
> > That is acceptable for me as I told before. Either mount or refuse
> > to
> > mount, but do not overflow and mount nonetheless :)
> >=20
> > Mind you, I am not using my Amiga machines either at the moment. And
> > I repurposed the 2 TB disk years ago.
>=20
> That's fine - I understand the 'profile' image was a true binary copy
> of the RDB, and placing that file at offset 0 in an image file is a
> legitimate use?

You actually ask me to remember about what that 'profile' image was? :)

Well, in the attachment note on the bug report I wrote: "should be just=20
a binary copy", so I did not know exactly back then either. However the=20
file starts with "RDSK" and then it has "PART" headers and so on. That=20
looks pretty much like a binary copy of an RDB. I am a bit surprised by=20
its small size of 2 KiB. But I see three partitions in there. According=20
to the screenshot I also provided, the disk had three partitions. So=20
probably Media Toolbox has been intelligent enough to just copy the used=20
space of the reserved RDB area. Cause I think the reserved space must=20
have been higher than 2 KiB. However the RDB/disk geometry editing=20
screen does not display it and off hand I do not know where to look=20
inside the RDB to see how much space has been reserved. Interestingly=20
the "Total sectors" value in that Media Toolbox window also overflowed.=20
But from my memory this was just a cosmetic issue in Media Toolbox. The=20
"*.device" device drivers, the filesystems and the RDB handling code in=20
AmigaOS do their own math. That is what NSD64 / TD64 was about.

https://bugzilla.kernel.org/show_bug.cgi?id=3D43511

> > I would not name the kernel option "eat_my_rdb", but use a less
> > dramatizing name.
> >=20
> > Maybe just: "allow_64bit_rdb" or something like that.
>=20
> I don't expect to get away with that :-)

Heh. :)

> > How does the user come to know about this kernel option? Will you
> > print its name in kernel log?
>=20
> Depends on how easy we want to make it for users. If I put a BUG()
> trap with the check, the resulting log section will point to a
> specific line in block/partitions/amiga.c, from which the override
> option will be obvious. But that might be a little too opaque for
> some...

kernel-parameters.txt or mentioning in the warning would also be an=20
option.

Thanks,
=2D-=20
Martin

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[Geert Uytterhoeven]

Hi Martin,

On Thu, Jun 28, 2018 at 9:29 AM Martin Steigerwald <martin@lichtvoll.de> wr=
ote:
> Michael Schmitz - 28.06.18, 06:58:
> [=E2=80=A6]
> > >> In the interest of least surprises, we have to fix the 32 bit
> > >> overflow (so we can even detect that it would have happened), and
> > >> give the user the chance to carefully consider whether to accept
> > >> the new behaviour. That means refusing to make available any
> > >> partition that would have been affected by such overflow.
> > >
> > > That is acceptable for me as I told before. Either mount or refuse
> > > to
> > > mount, but do not overflow and mount nonetheless :)
> > >
> > > Mind you, I am not using my Amiga machines either at the moment. And
> > > I repurposed the 2 TB disk years ago.
> >
> > That's fine - I understand the 'profile' image was a true binary copy
> > of the RDB, and placing that file at offset 0 in an image file is a
> > legitimate use?
>
> You actually ask me to remember about what that 'profile' image was? :)
>
> Well, in the attachment note on the bug report I wrote: "should be just
> a binary copy", so I did not know exactly back then either. However the
> file starts with "RDSK" and then it has "PART" headers and so on. That
> looks pretty much like a binary copy of an RDB. I am a bit surprised by
> its small size of 2 KiB. But I see three partitions in there. According
> to the screenshot I also provided, the disk had three partitions. So
> probably Media Toolbox has been intelligent enough to just copy the used
> space of the reserved RDB area. Cause I think the reserved space must
> have been higher than 2 KiB. However the RDB/disk geometry editing
> screen does not display it and off hand I do not know where to look
> inside the RDB to see how much space has been reserved. Interestingly
> the "Total sectors" value in that Media Toolbox window also overflowed.

The RDB can be anywhere in the first 2 tracks of the disk, and is identifie=
d
by the "RDSK" block (with a correct checksum). The remainder (e.g. "PART"
blocks) is in a linked list. So 2 KiB sounds fine for 3 partitions (1 RDSK =
+
3 * PART =3D 4 blocks =3D 4 * 512 bytes).

Gr{oetje,eeting}s,

                        Geert

--=20
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k=
.org

In personal conversations with technical people, I call myself a hacker. Bu=
t
when I'm talking to journalists I just say "programmer" or something like t=
hat.
                                -- Linus Torvalds

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[jdow]

On 20180627 23:45, Geert Uytterhoeven wrote:
> Hi Michael,
> 
> On Thu, Jun 28, 2018 at 6:59 AM Michael Schmitz <schmitzmic@gmail.com> wrote:
>> Am 28.06.2018 um 09:20 schrieb Martin Steigerwald:
>>>>> And as stated in my other reply to the patch:
>>>>> partition needs 64 bit disk device support in AmigaOS or AmigaOS
>>>>> like
>>>>> operating systems (NSD64, TD64 or SCSI direct)
>>>>
>>>> I'd probably leave it at 'disk needs 64 bit disk device support on
>>>> native OS', and only print that warning once.
>>>
>>> This is fine with me.
>>
>> OK, I'll go with that.
> 
> Do we really need the warning?
> Once the parsing is fixed doing 64-bit math, it does not matter for Linux
> anymore.

Dual booting.
{^_^}

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[Geert Uytterhoeven]

Hi Martin,

On Thu, Jun 28, 2018 at 9:13 AM Martin Steigerwald <martin@lichtvoll.de> wr=
ote:
> Geert Uytterhoeven - 28.06.18, 08:45:
> > On Thu, Jun 28, 2018 at 6:59 AM Michael Schmitz <schmitzmic@gmail.com>
> wrote:
> > > Am 28.06.2018 um 09:20 schrieb Martin Steigerwald:
> > > >>> And as stated in my other reply to the patch:
> > > >>> partition needs 64 bit disk device support in AmigaOS or AmigaOS
> > > >>> like
> > > >>> operating systems (NSD64, TD64 or SCSI direct)
> > > >>
> > > >> I'd probably leave it at 'disk needs 64 bit disk device support
> > > >> on
> > > >> native OS', and only print that warning once.
> > > >
> > > > This is fine with me.
> > >
> > > OK, I'll go with that.
> >
> > Do we really need the warning?
> > Once the parsing is fixed doing 64-bit math, it does not matter for
> > Linux anymore.
>
> Well, irony of this is: In my case the RDB has been created on a machine
> with a native OS. So Linux warns me about something I already did so on
> the native OS without any warning. In this case AmigaOS 4.0.

Exactly.

So moving a disk partitioned under AmigaOS 4.0 to a system running an
older version of AmigaOS can fail miserably. Not a Linux issue.
Linux also doesn't warn about disks with GPT failing to work on old MSDOS.

> > > > I would not name the kernel option "eat_my_rdb", but use a less
> > > > dramatizing name.
> > > >
> > > > Maybe just: "allow_64bit_rdb" or something like that.
> > >
> > > I don't expect to get away with that :-)
> >
> > I still fail to see what's the added value of the kernel option...
> > Either the partition is usable, or not.
>
> Well, I could try to contact some of the current AmigaOS developers
> about that and ask them whether they would like to give me a statement
> about this that I am allowed to post here.
>
> I would not know whether they answer and it may take a time. My offer
> stands, but I would only do this, if you really like to have that
> official feedback.

Let me clarify: what exactly would the kernel option allow? When to use it?

> Again, I am pretty sure that what I did is safe on AmigaOS 4 at least,
> but I bet also on AmigaOS <4 with NSD64 or TD64 (except for the
> filesystem sizes, but AmigaOS < 4 does not have JXFS anyway, and did not
> have SFS2 as well, maybe that is available now, I don=C2=B4t know).
>
> However Joanne is without doubt an authority on RDBs, but she has not

As a former AmigaOS user, I'm fully aware of that (Thanks Joanne! ;-)

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k=
.org

In personal conversations with technical people, I call myself a hacker. Bu=
t
when I'm talking to journalists I just say "programmer" or something like t=
hat.
                                -- Linus Torvalds

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[Geert Uytterhoeven]

Hi Joanne,

On Thu, Jun 28, 2018 at 11:20 AM jdow <jdow@earthlink.net> wrote:
> On 20180627 23:45, Geert Uytterhoeven wrote:
> > On Thu, Jun 28, 2018 at 6:59 AM Michael Schmitz <schmitzmic@gmail.com> wrote:
> >> Am 28.06.2018 um 09:20 schrieb Martin Steigerwald:
> >>>>> And as stated in my other reply to the patch:
> >>>>> partition needs 64 bit disk device support in AmigaOS or AmigaOS
> >>>>> like
> >>>>> operating systems (NSD64, TD64 or SCSI direct)
> >>>>
> >>>> I'd probably leave it at 'disk needs 64 bit disk device support on
> >>>> native OS', and only print that warning once.
> >>>
> >>> This is fine with me.
> >>
> >> OK, I'll go with that.
> >
> > Do we really need the warning?
> > Once the parsing is fixed doing 64-bit math, it does not matter for Linux
> > anymore.
>
> Dual booting.

(Dial/Triple/...) Booting older AmigaOS variants is an issue anyway, with
or without Linux.

It's like _Linux_ printing warnings that you cannot use your disk, created with
Solaris 2, on an old SunOS 4.1 machine ;-)

IMHO (s)he who creates partitions on large disks should make sure (s)he only
uses them on machines that can handle them.  This is not a Linux issue.

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[jdow]

On 20180628 00:39, Geert Uytterhoeven wrote:
> Hi Martin,
> 
> On Thu, Jun 28, 2018 at 9:29 AM Martin Steigerwald <martin@lichtvoll.de> wrote:
>> Michael Schmitz - 28.06.18, 06:58:
>> […]
>>>>> In the interest of least surprises, we have to fix the 32 bit
>>>>> overflow (so we can even detect that it would have happened), and
>>>>> give the user the chance to carefully consider whether to accept
>>>>> the new behaviour. That means refusing to make available any
>>>>> partition that would have been affected by such overflow.
>>>>
>>>> That is acceptable for me as I told before. Either mount or refuse
>>>> to
>>>> mount, but do not overflow and mount nonetheless :)
>>>>
>>>> Mind you, I am not using my Amiga machines either at the moment. And
>>>> I repurposed the 2 TB disk years ago.
>>>
>>> That's fine - I understand the 'profile' image was a true binary copy
>>> of the RDB, and placing that file at offset 0 in an image file is a
>>> legitimate use?
>>
>> You actually ask me to remember about what that 'profile' image was? :)
>>
>> Well, in the attachment note on the bug report I wrote: "should be just
>> a binary copy", so I did not know exactly back then either. However the
>> file starts with "RDSK" and then it has "PART" headers and so on. That
>> looks pretty much like a binary copy of an RDB. I am a bit surprised by
>> its small size of 2 KiB. But I see three partitions in there. According
>> to the screenshot I also provided, the disk had three partitions. So
>> probably Media Toolbox has been intelligent enough to just copy the used
>> space of the reserved RDB area. Cause I think the reserved space must
>> have been higher than 2 KiB. However the RDB/disk geometry editing
>> screen does not display it and off hand I do not know where to look
>> inside the RDB to see how much space has been reserved. Interestingly
>> the "Total sectors" value in that Media Toolbox window also overflowed.
> 
> The RDB can be anywhere in the first 2 tracks of the disk, and is identified
> by the "RDSK" block (with a correct checksum). The remainder (e.g. "PART"
> blocks) is in a linked list. So 2 KiB sounds fine for 3 partitions (1 RDSK +
> 3 * PART = 4 blocks = 4 * 512 bytes).

The RDB can be anywhere in the first 16 blocks on the disk if we are speaking 
officially. That's the entire area that is guaranteed to be inspected for an 
RDSK block. The PART block can, I think, even be located in front of the RDSK if 
you want to be obscene about it. {^_-} That's the kind of thing I checked with 
the HardFrame device driver ROM image. I preferred block 3 to keep it away from 
the space used by other partitioning schemes. It also enabled me to embed it a 
reserved area within the first actual partition just for the halibut. (Pronounce 
it sideways and it makes more sense.) I used that technique fairly often. If you 
think about it that gives you a wee tiny bit more disk space because you can 
tailor the reserved area to precisely fit the filesystem image plus some extra 
in case of updates. I toyed with using a pointer to FDSK blocks in the Dev 
directory but that got too insane. RDBs are insanely flexible, which may not be 
a good thing.

{^_^}

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[Michael Schmitz]

Hi Geert,


Am 28.06.18 um 21:25 schrieb Geert Uytterhoeven:
>
>>> Do we really need the warning?
>>> Once the parsing is fixed doing 64-bit math, it does not matter for
>>> Linux anymore.
>> Well, irony of this is: In my case the RDB has been created on a machine
>> with a native OS. So Linux warns me about something I already did so on
>> the native OS without any warning. In this case AmigaOS 4.0.
> Exactly.
>
> So moving a disk partitioned under AmigaOS 4.0 to a system running an
> older version of AmigaOS can fail miserably. Not a Linux issue.
> Linux also doesn't warn about disks with GPT failing to work on old MSDOS.

Would MSDOS recognize the GPT partition as 'probably FAT', and attempt
to use it?

> Let me clarify: what exactly would the kernel option allow? When to use it?

Whether to use it if safe (on Linux). But whatever Linux does (after
this patch), access will go to the right area of the disk (as specified
by the RDB) so Linux won't any longer stomp on anything that would have
mattered to 32 bit disk drivers. So it really should be safe.

Cheers,

    Michael

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[Geert Uytterhoeven]

Hi Michael,

On Fri, Jun 29, 2018 at 10:43 AM Michael Schmitz <schmitzmic@gmail.com> wrote:
> Am 28.06.18 um 21:25 schrieb Geert Uytterhoeven:
> >>> Do we really need the warning?
> >>> Once the parsing is fixed doing 64-bit math, it does not matter for
> >>> Linux anymore.
> >> Well, irony of this is: In my case the RDB has been created on a machine
> >> with a native OS. So Linux warns me about something I already did so on
> >> the native OS without any warning. In this case AmigaOS 4.0.
> > Exactly.
> >
> > So moving a disk partitioned under AmigaOS 4.0 to a system running an
> > older version of AmigaOS can fail miserably. Not a Linux issue.
> > Linux also doesn't warn about disks with GPT failing to work on old MSDOS.
>
> Would MSDOS recognize the GPT partition as 'probably FAT', and attempt
> to use it?

No idea...

Probably some old Windows or MacOS versions will just suggest to
format your "new" disk ;-)

But it's up to the person (which is not Linux) formatting the disk to
not try to use
it on systems that cannot handle it, and may destroy it.

> > Let me clarify: what exactly would the kernel option allow? When to use it?
>
> Whether to use it if safe (on Linux). But whatever Linux does (after
> this patch), access will go to the right area of the disk (as specified
> by the RDB) so Linux won't any longer stomp on anything that would have
> mattered to 32 bit disk drivers. So it really should be safe.

Personally, I see no reason to depend on a kernel option, if it is safe to use.
Just use it.

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[Michael Schmitz]

Hi Geert,


Am 28.06.18 um 18:45 schrieb Geert Uytterhoeven:
> Hi Michael,
>
> On Thu, Jun 28, 2018 at 6:59 AM Michael Schmitz <schmitzmic@gmail.com> wrote:
>> Am 28.06.2018 um 09:20 schrieb Martin Steigerwald:
>>>>> And as stated in my other reply to the patch:
>>>>> partition needs 64 bit disk device support in AmigaOS or AmigaOS
>>>>> like
>>>>> operating systems (NSD64, TD64 or SCSI direct)
>>>> I'd probably leave it at 'disk needs 64 bit disk device support on
>>>> native OS', and only print that warning once.
>>> This is fine with me.
>> OK, I'll go with that.
> Do we really need the warning?
> Once the parsing is fixed doing 64-bit math, it does not matter for Linux
> anymore.
>
> Won't it make more sense to have the warning in the tool that created the
> partition table in the first place?
Sure, but we've seen one case of this in the wild, and the tool
apparently did not issue a warning.

I agree it's not an issue for Linux. If  you think dropping a warning
about something not actually relevant to Linux in the log would be
confusing, or if convention is to limit warnings strictly to behaviour
relevant to Linux, I can live without the warning. Joanne scared me a
bit about Amigoids angry at data loss, but I suppose there can't be many
around my neck of the woods. 

>
>>> I would not name the kernel option "eat_my_rdb", but use a less
>>> dramatizing name.
>>>
>>> Maybe just: "allow_64bit_rdb" or something like that.
>> I don't expect to get away with that :-)
> I still fail to see what's the added value of the kernel option...
> Either the partition is usable, or not.

The question is - can writes to the disk cause any damage to data on the
disk, as seen by old OS versions? If the answer is no, we won't need the
option after all.

>>> How does the user come to know about this kernel option? Will you print
>>> its name in kernel log?
>> Depends on how easy we want to make it for users. If I put a BUG() trap
>> with the check, the resulting log section will point to a specific line
>> in block/partitions/amiga.c, from which the override option will be
>> obvious. But that might be a little too opaque for some...
> Please don't use BUG(), unless your goal is to attract attention (from
> Linus, who dislikes BUG()!).
I'd rather not abuse his patience. Thanks for the hint.
> Using BUG() would be a nice way to DoS someones machine by plugging in
> a USB stick with a malformed RDB.
I guess I deserved that. But BUG() doesn't panic now, does it? The ones
I saw did allow the kernel to happily carry on.

Cheers,

    Michael

>
> Gr{oetje,eeting}s,
>
>                         Geert
>

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[Michael Schmitz]

Hi Geert,


Am 29.06.18 um 20:51 schrieb Geert Uytterhoeven:
> Hi Michael,
>
>> Would MSDOS recognize the GPT partition as 'probably FAT', and attempt
>> to use it?
> No idea...
>
> Probably some old Windows or MacOS versions will just suggest to
> format your "new" disk ;-)

Yep, that's what I'd expect. Windows even used to trash the LILO boot
code in the MBR if you just happened to boot to it by accident. Did we
complain?

>
> But it's up to the person (which is not Linux) formatting the disk to
> not try to use
> it on systems that cannot handle it, and may destroy it.
>
>>> Let me clarify: what exactly would the kernel option allow? When to use it?
>> Whether to use it if safe (on Linux). But whatever Linux does (after
>> this patch), access will go to the right area of the disk (as specified
>> by the RDB) so Linux won't any longer stomp on anything that would have
>> mattered to 32 bit disk drivers. So it really should be safe.
> Personally, I see no reason to depend on a kernel option, if it is safe to use.
> Just use it.

So to recap - someone partitions a disk on AmigaOS 4.x, taking advantage
of the large block device support there.
Using that disk on AmigaOS 3.1, data loss ensues. Whether or not Linux
(patched) ever touched the disk has no impact on that outcome.

Right?

Cheers,

    Michael

>
> Gr{oetje,eeting}s,
>
>                         Geert
>

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[Geert Uytterhoeven]

Hi Michael,

On Fri, Jun 29, 2018 at 10:58 AM Michael Schmitz <schmitzmic@gmail.com> wrote:
> Am 28.06.18 um 18:45 schrieb Geert Uytterhoeven:
> > On Thu, Jun 28, 2018 at 6:59 AM Michael Schmitz <schmitzmic@gmail.com> wrote:
> >> Am 28.06.2018 um 09:20 schrieb Martin Steigerwald:
> >>>>> And as stated in my other reply to the patch:
> >>>>> partition needs 64 bit disk device support in AmigaOS or AmigaOS
> >>>>> like
> >>>>> operating systems (NSD64, TD64 or SCSI direct)
> >>>> I'd probably leave it at 'disk needs 64 bit disk device support on
> >>>> native OS', and only print that warning once.
> >>> This is fine with me.
> >> OK, I'll go with that.
> > Do we really need the warning?
> > Once the parsing is fixed doing 64-bit math, it does not matter for Linux
> > anymore.
> >
> > Won't it make more sense to have the warning in the tool that created the
> > partition table in the first place?
> Sure, but we've seen one case of this in the wild, and the tool
> apparently did not issue a warning.
>
> I agree it's not an issue for Linux. If  you think dropping a warning
> about something not actually relevant to Linux in the log would be
> confusing, or if convention is to limit warnings strictly to behaviour
> relevant to Linux, I can live without the warning. Joanne scared me a
> bit about Amigoids angry at data loss, but I suppose there can't be many
> around my neck of the woods.
>
> >
> >>> I would not name the kernel option "eat_my_rdb", but use a less
> >>> dramatizing name.
> >>>
> >>> Maybe just: "allow_64bit_rdb" or something like that.
> >> I don't expect to get away with that :-)
> > I still fail to see what's the added value of the kernel option...
> > Either the partition is usable, or not.
>
> The question is - can writes to the disk cause any damage to data on the
> disk, as seen by old OS versions? If the answer is no, we won't need the
> option after all.

You mean someone relying on the parameters of his RDB to overflow using
32-bit calculations, and still have valid offsets on the disk so it's usable?
I think that would need hand-crafting an RDB, if possible at all.
And writing to it on AmigaOS 4.0 or any other OS doing proper 64-bit
calculations would write to the wrong locations, too.
IMHO not something to consider.

> >>> How does the user come to know about this kernel option? Will you print
> >>> its name in kernel log?
> >> Depends on how easy we want to make it for users. If I put a BUG() trap
> >> with the check, the resulting log section will point to a specific line
> >> in block/partitions/amiga.c, from which the override option will be
> >> obvious. But that might be a little too opaque for some...
> > Please don't use BUG(), unless your goal is to attract attention (from
> > Linus, who dislikes BUG()!).
> I'd rather not abuse his patience. Thanks for the hint.
> > Using BUG() would be a nice way to DoS someones machine by plugging in
> > a USB stick with a malformed RDB.
> I guess I deserved that. But BUG() doesn't panic now, does it? The ones
> I saw did allow the kernel to happily carry on.

The one in asm-generic does call panic().
The m68k one calls __builtin_trap(), which may cause a trap (and panic?) or
do nothing, depending on gcc version, I think.

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[Michael Schmitz]

Hi Geert,


Am 28.06.18 um 01:30 schrieb Geert Uytterhoeven:
> Hi Michael,
>
> Thanks for your patch!
>
> On Wed, Jun 27, 2018 at 4:47 AM <schmitzmic@gmail.com> wrote:
>> From 5299e0e64dfb33ac3a1f3137b42178734ce20087 Mon Sep 17 00:00:00 2001
> ??
>
>> The Amiga RDB partition parser module uses int for partition sector
>> address and count, which will overflow for disks 2 TB and larger.
>>
>> Use sector_t as type for sector address and size (as expected by
>> put_partition) to allow using such disks without danger of data
>> corruption.
> Note that sector_t is not guaranteed to be 64-bit:
>
>     #ifdef CONFIG_LBDAF
>     typedef u64 sector_t;
>     typedef u64 blkcnt_t;
>     #else
>     typedef unsigned long sector_t;
>     typedef unsigned long blkcnt_t;
>     #endif
>
> And it seems CONFIG_LBDAF can still be disabled on 32-bit...
>

What are the ramifications of using a 2 TB disk on a kernel without
CONFIG_LBDAF? Are there any safeguards after the partition scan stage
that would prevent the kernel from using partitions on such a disk?

Trying to decide whether we absolutely have to bail if sector_t is 32
bit ...

Cheers,

    Michael

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[Geert Uytterhoeven]

Hi Michael,

On Fri, Jun 29, 2018 at 11:08 AM Michael Schmitz <schmitzmic@gmail.com> wrote:
> Am 29.06.18 um 20:51 schrieb Geert Uytterhoeven:
> >> Would MSDOS recognize the GPT partition as 'probably FAT', and attempt
> >> to use it?
> > No idea...
> >
> > Probably some old Windows or MacOS versions will just suggest to
> > format your "new" disk ;-)
>
> Yep, that's what I'd expect. Windows even used to trash the LILO boot
> code in the MBR if you just happened to boot to it by accident. Did we
> complain?

Those days Linux users just knew not to use Windows ;-)

> > But it's up to the person (which is not Linux) formatting the disk to
> > not try to use
> > it on systems that cannot handle it, and may destroy it.
> >
> >>> Let me clarify: what exactly would the kernel option allow? When to use it?
> >> Whether to use it if safe (on Linux). But whatever Linux does (after
> >> this patch), access will go to the right area of the disk (as specified
> >> by the RDB) so Linux won't any longer stomp on anything that would have
> >> mattered to 32 bit disk drivers. So it really should be safe.
> > Personally, I see no reason to depend on a kernel option, if it is safe to use.
> > Just use it.
>
> So to recap - someone partitions a disk on AmigaOS 4.x, taking advantage
> of the large block device support there.
> Using that disk on AmigaOS 3.1, data loss ensues. Whether or not Linux
> (patched) ever touched the disk has no impact on that outcome.
>
> Right?

That's my understanding, too.

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[Michael Schmitz]

Hi Geert,


Am 29.06.18 um 21:10 schrieb Geert Uytterhoeven:
>
>> The question is - can writes to the disk cause any damage to data on the
>> disk, as seen by old OS versions? If the answer is no, we won't need the
>> option after all.
> You mean someone relying on the parameters of his RDB to overflow using
> 32-bit calculations, and still have valid offsets on the disk so it's usable?
> I think that would need hand-crafting an RDB, if possible at all.
> And writing to it on AmigaOS 4.0 or any other OS doing proper 64-bit
> calculations would write to the wrong locations, too.
> IMHO not something to consider.

Something like that. But I haven't stopped for long enough to work out
if that was even possible.

>
>>>>> How does the user come to know about this kernel option? Will you print
>>>>> its name in kernel log?
>>>> Depends on how easy we want to make it for users. If I put a BUG() trap
>>>> with the check, the resulting log section will point to a specific line
>>>> in block/partitions/amiga.c, from which the override option will be
>>>> obvious. But that might be a little too opaque for some...
>>> Please don't use BUG(), unless your goal is to attract attention (from
>>> Linus, who dislikes BUG()!).
>> I'd rather not abuse his patience. Thanks for the hint.
>>> Using BUG() would be a nice way to DoS someones machine by plugging in
>>> a USB stick with a malformed RDB.
>> I guess I deserved that. But BUG() doesn't panic now, does it? The ones
>> I saw did allow the kernel to happily carry on.
> The one in asm-generic does call panic().

Ouch - that explains Linus's aversion.

> The m68k one calls __builtin_trap(), which may cause a trap (and panic?) or
> do nothing, depending on gcc version, I think.

The trap doesn't cause a panic for me. Might rely on the trap vector
being set up right (which we can rely on at partition scan time), and
the context though. But if m68k is the odd one out in gracefully
handling BUG(), better leave it.

Cheers,

    Michael
>
> Gr{oetje,eeting}s,
>
>                         Geert
>

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[Michael Schmitz]

Hi Geert,

Am 29.06.18 um 21:12 schrieb Geert Uytterhoeven:
> Hi Michael,
>
> On Fri, Jun 29, 2018 at 11:08 AM Michael Schmitz <schmitzmic@gmail.com> wrote:
>> Am 29.06.18 um 20:51 schrieb Geert Uytterhoeven:
>>>> Would MSDOS recognize the GPT partition as 'probably FAT', and attempt
>>>> to use it?
>>> No idea...
>>>
>>> Probably some old Windows or MacOS versions will just suggest to
>>> format your "new" disk ;-)
>> Yep, that's what I'd expect. Windows even used to trash the LILO boot
>> code in the MBR if you just happened to boot to it by accident. Did we
>> complain?
> Those days Linux users just knew not to use Windows ;-)

For good reason. But that happened on a couple of instrument control PCs
I converted from Win2k. Never mount a scratch monkey... 

>>> Personally, I see no reason to depend on a kernel option, if it is safe to use.
>>> Just use it.
>> So to recap - someone partitions a disk on AmigaOS 4.x, taking advantage
>> of the large block device support there.
>> Using that disk on AmigaOS 3.1, data loss ensues. Whether or not Linux
>> (patched) ever touched the disk has no impact on that outcome.
>>
>> Right?
> That's my understanding, too.

OK, that decides it.

Cheers,

    Michael
> Gr{oetje,eeting}s,
>
>                         Geert
>

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[jdow]

On 20180629 01:42, Michael Schmitz wrote:
> Hi Geert,
> 
> 
> Am 28.06.18 um 21:25 schrieb Geert Uytterhoeven:
>>
>>>> Do we really need the warning?
>>>> Once the parsing is fixed doing 64-bit math, it does not matter for
>>>> Linux anymore.
>>> Well, irony of this is: In my case the RDB has been created on a machine
>>> with a native OS. So Linux warns me about something I already did so on
>>> the native OS without any warning. In this case AmigaOS 4.0.
>> Exactly.
>>
>> So moving a disk partitioned under AmigaOS 4.0 to a system running an
>> older version of AmigaOS can fail miserably. Not a Linux issue.
>> Linux also doesn't warn about disks with GPT failing to work on old MSDOS.
> 
> Would MSDOS recognize the GPT partition as 'probably FAT', and attempt
> to use it?

I believe it would fail to mount it and suggest formatting it. The key is to 
fail the mount and not risk damage to the data on it.

>> Let me clarify: what exactly would the kernel option allow? When to use it?
> 
> Whether to use it if safe (on Linux). But whatever Linux does (after
> this patch), access will go to the right area of the disk (as specified
> by the RDB) so Linux won't any longer stomp on anything that would have
> mattered to 32 bit disk drivers. So it really should be safe.

Martin pointed out something privately about RDBs I'd forgotten over the 
archeological eras gone past. The RDB values are all 32 bit unsigned longs. That 
means as long as they are properly promoted AND the file system does not barf 
from a gross overfeeding disks up to about 2^128 bytes can be supported. But you 
need a new uint128_t datatype. So for Amigas or Linux the real limitation is the 
data type and significance that flows between the RDB parser and the file 
system. That is the value that has to be considered. Then if the filesystem 
somewhere inside coughs up a hairball over some internal overflow or other and 
cannot accept say a full uint64_t maximum value of bytes with grace and polish 
it should error out when it sees an incoming disk positions off its internally 
acceptable range.

This means that any errors and failures to mount should lie with the RDB Parser, 
the filesystem, the device driver, and any intervening code between the three. 
The RDBs can express a value far greater than the number of atoms estimated to 
exist in the observable universe. So RDBs won't ever need a change for disk size 
considerations. The parser and filesystems probably and possibly, respectively, 
will.

Error out and don't mount disks or partitions that are outside the size range 
your element of the datapath from disk to user's program can handle at the first 
place this can be detected. It's the polite thing to do to a user's data rather 
than the Maxine Waters thing to do.

{^_^}

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[Andreas Schwab]

On Jun 29 2018, Michael Schmitz <schmitzmic@gmail.com> wrote:

> Would MSDOS recognize the GPT partition as 'probably FAT', and attempt
> to use it?

GPT has the concept of a protective MBR which should prevent such errors.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[Martin Steigerwald]

Hi Michael.

Michael Schmitz - 29.06.18, 10:42:
> Am 28.06.18 um 21:25 schrieb Geert Uytterhoeven:
> >>> Do we really need the warning?
> >>> Once the parsing is fixed doing 64-bit math, it does not matter
> >>> for
> >>> Linux anymore.
> >> 
> >> Well, irony of this is: In my case the RDB has been created on a
> >> machine with a native OS. So Linux warns me about something I
> >> already did so on the native OS without any warning. In this case
> >> AmigaOS 4.0.> 
> > Exactly.
> > 
> > So moving a disk partitioned under AmigaOS 4.0 to a system running
> > an
> > older version of AmigaOS can fail miserably. Not a Linux issue.
> > Linux also doesn't warn about disks with GPT failing to work on old
> > MSDOS.
> Would MSDOS recognize the GPT partition as 'probably FAT', and attempt
> to use it?

As far as I know most GPT partitioning tools create a fake MBR with one 
large partition of a partition type for GPT to tell legacy applications 
to leave the disk alone.

Thanks,
-- 
Martin

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[Martin Steigerwald]

Hi,

Geert Uytterhoeven - 29.06.18, 10:51:
> On Fri, Jun 29, 2018 at 10:43 AM Michael Schmitz=20
<schmitzmic@gmail.com> wrote:
> > Am 28.06.18 um 21:25 schrieb Geert Uytterhoeven:
> > >>> Do we really need the warning?
> > >>> Once the parsing is fixed doing 64-bit math, it does not matter
> > >>> for
> > >>> Linux anymore.
> > >>=20
> > >> Well, irony of this is: In my case the RDB has been created on a
> > >> machine with a native OS. So Linux warns me about something I
> > >> already did so on the native OS without any warning. In this
> > >> case AmigaOS 4.0.> >=20
> > > Exactly.
> > >=20
> > > So moving a disk partitioned under AmigaOS 4.0 to a system running
> > > an
> > > older version of AmigaOS can fail miserably. Not a Linux issue.
> > > Linux also doesn't warn about disks with GPT failing to work on
> > > old MSDOS.>=20
> > Would MSDOS recognize the GPT partition as 'probably FAT', and
> > attempt to use it?
>=20
> No idea...
>=20
> Probably some old Windows or MacOS versions will just suggest to
> format your "new" disk ;-)

Heh=E2=80=A6 MacOS X last I saw it just offers to initialize any disk it do=
es=20
not know about. I had this with a friend who uses Mac quite some years=20
ago with an external harddrive with Linux filesystems. I told him Mac OS=20
X would not understand it, but he did not believe me. Well luckily=20
enough I have been quick enough to unplug the USB cable before he could=20
hit the initialize button. And yes, that is right, Mac OS X at that time=20
did not even tell the user that initializing means *formatting the disk=20
and making all data that was previously on it unavailable*. I did not=20
even believe it that they had no stern warning about initializing in the=20
dialog window. I hope they improved the wording meanwhile.

Thanks,
=2D-=20
Martin

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[Martin Steigerwald]

Hi Michael.

Michael Schmitz - 29.06.18, 11:07:
> > But it's up to the person (which is not Linux) formatting the disk
> > to
> > not try to use
> > it on systems that cannot handle it, and may destroy it.
> >=20
> >>> Let me clarify: what exactly would the kernel option allow? When
> >>> to use it?>>=20
> >> Whether to use it if safe (on Linux). But whatever Linux does
> >> (after
> >> this patch), access will go to the right area of the disk (as
> >> specified by the RDB) so Linux won't any longer stomp on anything
> >> that would have mattered to 32 bit disk drivers. So it really
> >> should be safe.>=20
> > Personally, I see no reason to depend on a kernel option, if it is
> > safe to use. Just use it.
>=20
> So to recap - someone partitions a disk on AmigaOS 4.x, taking
> advantage of the large block device support there.
> Using that disk on AmigaOS 3.1, data loss ensues. Whether or not Linux
> (patched) ever touched the disk has no impact on that outcome.

I am not even completely sure about that. Frankly I have no idea what=20
would happen when using such a disk on AmigaOS 3.1 *without* NSDPatch or=20
TD64 support (I think you could patch AmigaOS 3.1 with 64 Bit support=20
already and some 3rd party harddisk controllers by Phase 5 hat TD64=20
support at that time already). Unless I try it, which I won=C2=B4t at the=20
moment, I=C2=B4d say the behaviour is largely undefined.

But hey, undefined means it may just overwrite start overwriting from=20
the beginning of the disk beyond 32 bit. And I think that is quite=20
likely. It could also crash, but if its an overflow I don=C2=B4t think why =
it=20
would crash. Anyway, I never tried this out.

But in any way: This would happen or not happen no matter whether Linux=20
parsed the RDB or not.

I still think that the native OS warning really does not hurt=E2=80=A6 but =
I=C2=B4d=20
spare myself the kernel option. Having the warning without the kernel=20
option would be a compromise between being cautious and being bold :).

Thanks,
=2D-=20
Martin

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[Martin Steigerwald]

QmV3YXJlOiBFc3NheSBhaGVhZCB3aGljaCBwcm9vZnMgaXQgdG8gdGhlIHBvaW50IHRoYXQgdGhl
cmUgaXMgbm8gCm92ZXJmbG93IGluIFJEQiBiZWZvcmUgOTYgYml0cyBtYXhpbXVtIHZhbHVlIG9m
IHNlY3RvcnM6CgpqZG93IC0gMjkuMDYuMTgsIDExOjMyOgo+IE9uIDIwMTgwNjI5IDAxOjQyLCBN
aWNoYWVsIFNjaG1pdHogd3JvdGU6Cj4gPiBIaSBHZWVydCwKPiA+IAo+ID4gQW0gMjguMDYuMTgg
dW0gMjE6MjUgc2NocmllYiBHZWVydCBVeXR0ZXJob2V2ZW46Cj4gPj4+PiBEbyB3ZSByZWFsbHkg
bmVlZCB0aGUgd2FybmluZz8KPiA+Pj4+IE9uY2UgdGhlIHBhcnNpbmcgaXMgZml4ZWQgZG9pbmcg
NjQtYml0IG1hdGgsIGl0IGRvZXMgbm90IG1hdHRlcgo+ID4+Pj4gZm9yCj4gPj4+PiBMaW51eCBh
bnltb3JlLgo+ID4+PiAKPiA+Pj4gV2VsbCwgaXJvbnkgb2YgdGhpcyBpczogSW4gbXkgY2FzZSB0
aGUgUkRCIGhhcyBiZWVuIGNyZWF0ZWQgb24gYQo+ID4+PiBtYWNoaW5lIHdpdGggYSBuYXRpdmUg
T1MuIFNvIExpbnV4IHdhcm5zIG1lIGFib3V0IHNvbWV0aGluZyBJCj4gPj4+IGFscmVhZHkgZGlk
IHNvIG9uIHRoZSBuYXRpdmUgT1Mgd2l0aG91dCBhbnkgd2FybmluZy4gSW4gdGhpcyBjYXNlCj4g
Pj4+IEFtaWdhT1MgNC4wLj4+IAo+ID4+IEV4YWN0bHkuCj4gPj4gCj4gPj4gU28gbW92aW5nIGEg
ZGlzayBwYXJ0aXRpb25lZCB1bmRlciBBbWlnYU9TIDQuMCB0byBhIHN5c3RlbSBydW5uaW5nCj4g
Pj4gYW4KPiA+PiBvbGRlciB2ZXJzaW9uIG9mIEFtaWdhT1MgY2FuIGZhaWwgbWlzZXJhYmx5LiBO
b3QgYSBMaW51eCBpc3N1ZS4KPiA+PiBMaW51eCBhbHNvIGRvZXNuJ3Qgd2FybiBhYm91dCBkaXNr
cyB3aXRoIEdQVCBmYWlsaW5nIHRvIHdvcmsgb24gb2xkCj4gPj4gTVNET1MuPiAKPiA+IFdvdWxk
IE1TRE9TIHJlY29nbml6ZSB0aGUgR1BUIHBhcnRpdGlvbiBhcyAncHJvYmFibHkgRkFUJywgYW5k
Cj4gPiBhdHRlbXB0IHRvIHVzZSBpdD8KPiAKPiBJIGJlbGlldmUgaXQgd291bGQgZmFpbCB0byBt
b3VudCBpdCBhbmQgc3VnZ2VzdCBmb3JtYXR0aW5nIGl0LiBUaGUga2V5Cj4gaXMgdG8gZmFpbCB0
aGUgbW91bnQgYW5kIG5vdCByaXNrIGRhbWFnZSB0byB0aGUgZGF0YSBvbiBpdC4KPiAKPiA+PiBM
ZXQgbWUgY2xhcmlmeTogd2hhdCBleGFjdGx5IHdvdWxkIHRoZSBrZXJuZWwgb3B0aW9uIGFsbG93
PyBXaGVuIHRvCj4gPj4gdXNlIGl0Pz4gCj4gPiBXaGV0aGVyIHRvIHVzZSBpdCBpZiBzYWZlIChv
biBMaW51eCkuIEJ1dCB3aGF0ZXZlciBMaW51eCBkb2VzIChhZnRlcgo+ID4gdGhpcyBwYXRjaCks
IGFjY2VzcyB3aWxsIGdvIHRvIHRoZSByaWdodCBhcmVhIG9mIHRoZSBkaXNrIChhcwo+ID4gc3Bl
Y2lmaWVkIGJ5IHRoZSBSREIpIHNvIExpbnV4IHdvbid0IGFueSBsb25nZXIgc3RvbXAgb24gYW55
dGhpbmcKPiA+IHRoYXQgd291bGQgaGF2ZSBtYXR0ZXJlZCB0byAzMiBiaXQgZGlzayBkcml2ZXJz
LiBTbyBpdCByZWFsbHkKPiA+IHNob3VsZCBiZSBzYWZlLgo+IE1hcnRpbiBwb2ludGVkIG91dCBz
b21ldGhpbmcgcHJpdmF0ZWx5IGFib3V0IFJEQnMgSSdkIGZvcmdvdHRlbiBvdmVyCj4gdGhlIGFy
Y2hlb2xvZ2ljYWwgZXJhcyBnb25lIHBhc3QuIFRoZSBSREIgdmFsdWVzIGFyZSBhbGwgMzIgYml0
Cj4gdW5zaWduZWQgbG9uZ3MuIFRoYXQgbWVhbnMgYXMgbG9uZyBhcyB0aGV5IGFyZSBwcm9wZXJs
eSBwcm9tb3RlZCBBTkQKPiB0aGUgZmlsZSBzeXN0ZW0gZG9lcyBub3QgYmFyZiBmcm9tIGEgZ3Jv
c3Mgb3ZlcmZlZWRpbmcgZGlza3MgdXAgdG8KPiBhYm91dCAyXjEyOCBieXRlcyBjYW4gYmUgc3Vw
cG9ydGVkLiBCdXQgeW91IG5lZWQgYSBuZXcgdWludDEyOF90Cj4gZGF0YXR5cGUuIFNvIGZvciBB
bWlnYXMgb3IgTGludXggdGhlIHJlYWwgbGltaXRhdGlvbiBpcyB0aGUgZGF0YSB0eXBlCj4gYW5k
IHNpZ25pZmljYW5jZSB0aGF0IGZsb3dzIGJldHdlZW4gdGhlIFJEQiBwYXJzZXIgYW5kIHRoZSBm
aWxlCj4gc3lzdGVtLiBUaGF0IGlzIHRoZSB2YWx1ZSB0aGF0IGhhcyB0byBiZSBjb25zaWRlcmVk
LiBUaGVuIGlmIHRoZQo+IGZpbGVzeXN0ZW0gc29tZXdoZXJlIGluc2lkZSBjb3VnaHMgdXAgYSBo
YWlyYmFsbCBvdmVyIHNvbWUgaW50ZXJuYWwKPiBvdmVyZmxvdyBvciBvdGhlciBhbmQgY2Fubm90
IGFjY2VwdCBzYXkgYSBmdWxsIHVpbnQ2NF90IG1heGltdW0gdmFsdWUKPiBvZiBieXRlcyB3aXRo
IGdyYWNlIGFuZCBwb2xpc2ggaXQgc2hvdWxkIGVycm9yIG91dCB3aGVuIGl0IHNlZXMgYW4KPiBp
bmNvbWluZyBkaXNrIHBvc2l0aW9ucyBvZmYgaXRzIGludGVybmFsbHkgYWNjZXB0YWJsZSByYW5n
ZS4KCkkgdGhpbmsgSSBwb2ludGVkIGl0IG91dCBwdWJsaWNhbGx5IGFzIHdlbGwuCgpJIGRpZCBp
biB0aGUgb3RoZXIgdGhyZWFkICJSZTogQW1pZ2EgUkRCIHBhcnRpdGlvbiBzdXBwb3J0IGZvciBk
aXNrcyA+PSAKMiBUQiIuIEp1c3QgZm9yIHRoZSByZWNvcmQgaW4gYW4gcmVwbHkgdG8geW91ciBw
YXRjaCwgTWljaGFlbCwgSSBjb3B5IGl0IApvdmVyOgoKLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQpCdXQgeWVh
aCwgYXMgbWVudGlvbmVkIEkgcmVzZWFyY2hlZCB0aGUgdG9waWMgYmVmb3JlLiBBbmQgSSB0aGlu
ayB0aGVyZSAKaGFzIG5vdCBldmVuIGJlZW4gYW4gb3ZlcmZsb3cgd2l0aGluIHRoZSBSREI6Cgo+
IFRoZSByYXcsIHRoZW9yZXRpY2FsIGxpbWl0IG9uIHRoZSBtYXhpbXVtIGRldmljZSBjYXBhY2l0
eSBpcyBhYm91dAo+IDJeMTA1IGJ5dGVzOgo+Cj4gMzIgYml0IHJkYl9DeWxpbmRlcnMgKiAzMiBi
aXQgcmRiX0hlYWRzICogMzIgYml0IHJkYl9TZWN0b3JzICogNTEyCj4gYnl0ZXMvc2VjdG9yIGZv
ciB0aGUgSEQgc2l6ZSBpbiBzdHJ1Y3QgUmlnaWREaXNrQmxvY2sKCmh0dHA6Ly93aWtpLmFtaWdh
b3MubmV0L3dpa2kvUkRCXyhBbWlnYV9SaWdpZF9EaXNrX0Jsb2NrKQoKQ29uZmlybWVkIGJ5OgoK
VGhlIC5BREYgKEFtaWdhIERpc2sgRmlsZSkgZm9ybWF0IEZBUToKaHR0cDovL2xjbGV2eS5mcmVl
LmZyL2FkZmxpYi9hZGZfaW5mby5odG1sI3A2CgpCdXQgd2hhdCBkbyBJIHdyaXRlLCB5b3Uga25v
dyB0aGUgUkRCIGZvcm1hdCAKClNvIGp1c3QgZG8gdGhlIGNhbGN1bGF0aW9uIGluIDk2IEJpdCBh
bmQgeW91IGFsbCBhcmUgc2V0IAoKTm93IHRoYXQgaXMgYSByZWFzb24gZm9yIDEyOCBCaXQgQ1BV
cyA6KS4KLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLQoKPiBIZXJlIGFyZSB0aGUgcmVsZXZhbnQgcGllY2VzIG9m
IHRoZSBBREYgZm9ybWF0IEZBUToKPiAgNi4xIFdoYXQgaXMgdGhlIFJpZ2lkIERpc2sgQmxvY2sg
Pwo+IAo+ICogUmlnaWQgRGlzayBibG9jayAoMjU2IGJ5dGVzKSBtdXN0IGV4aXN0IHdpdGhpbiB0
aGUgZmlyc3QgMTYgYmxvY2tzCj4gW+KApl0KPiAKPiAJUGh5c2ljYWwgZHJpdmUgY2FyYWN0ZXJp
c3RpY3MKPiAKPiA2NC80MAl1bG9uZyAJMSAJY3lsaW5kZXJzIAludW1iZXIgb2YgZHJpdmUgY3ls
aW5kZXIKPiA2OC80NCAJdWxvbmcgCTEgCXNlY3RvcnMgCXNlY3RvcnMgcGVyIHRyYWNrCj4gNzIv
NDgJdWxvbmcgCTEgCWhlYWRzIAkJbnVtYmVyIG9mIGRyaXZlIGhlYWRzCj4gW+KApl0KClVMT05H
IGlzIDMyLWJpdCB1bnNpZ25lZC4gU2VlOgoKaHR0cDovL3dpa2kuYW1pZ2Fvcy5uZXQvd2lraS9G
dW5kYW1lbnRhbF9UeXBlcwoKU28gd2UgYmFzaWNhbGx5IGhhdmUgOTYgYml0cyBoZXJlICsgYXQg
bGVhc3QgOSBiaXRzICg1MTIgYnl0ZXMgc2VjdG9yIApzaXplKSBpZiB5b3UgY2FsY3VsYXRlIGJ5
IGJ5dGVzLiBBbmQgYXMgbm90ZWQgYnkgSm9hbm5lOiBJbiBjYXNlIHlvdSAKd291bGQgdXNlIGxh
cmdlIHNlY3RvciBzaXplc+KApiB5b3UgY291bGQgYWRkcmVzcyByZWFsbHkgZXZlbiBtb3JlIAoq
aW5zYW5lKiBhbW91bnRzIG9mIGNhcGFjaXR5LgoKU2VlIHdoeSBJIHN0aWxsIHRoaW5rIFJEQsK0
cyBpcyBxdWl0ZSBicmlsbGlhbnQ/CgpNYXliZSB1c2luZyBVTE9ORyB3YXMganVzdCBieSBhY2Np
ZGVudCwgYnV0IHdlbGwgaXQgbWFkZSBSRELCtHMgZnV0dXJlIApwcm9vZiBmb3IgYWNjZXNzaW5n
IGxhcmdlciBkaXNrcyB0aGFuIGFueXRoaW5nIHRoYXQgaXMgaW4gdGhlIG1hcmtldCAKdG9kYXkg
YW5kIEkgYmV0IGZvciBhIGxvbmcgdGltZSB0byBjb21lIDopCgpBbmQgZm9yIHBhcnRpdGlvbnMg
dGhpcyB3b3JrcyBhcyB3ZWxsOgoKPiAgNi4zIEhvdyBhcmUgcGFydGl0aW9ucyBzdG9yZWQ/Cj4g
Cj4gMTY0L2E0IAl1bG9uZyAJMSAJTG93Q3lsCQlmaXJzdCBjeWxpbmRlciBvZiBhIHBhcnRpdGlv
biAKPiAoaW5jbHVzaXZlKQo+IDE2OC9hOCAJdWxvbmcgCTEgCUhpZ2hDeWwJCWxhc3QgY3lsaW5k
ZXIgb2YgYSBwYXJ0aXRpb24gCj4gKGluY2x1c2l2ZSkACgpTbyB0byBzdW1tYXJpemU6IEkgdGhp
bmsgUkRCwrRzIGNhbiBjb21wZXRlIHdpdGggR1BUIHJlZ2FyZGluZyBtYXhpbXVtIApjYXBhY2l0
eSBuaWNlbHkgZW5vdWdoLiBEZXNwaXRlIEpvYW5uZSBpbnZlbnRlZCBSREIgYXQgbGVhc3QgYSBk
ZWNhZGUgCmJlZm9yZSBzb21lb25lIGludmVudGVkIEdQVC4gQW1pZ2FPUyBoYWQgYW5kIGhhcyBh
IHRvbiBvZiBmbGF3cyBidXQgaXQgCmhhcyBiZWVuIG1hZGUgYnkgZ2VuaXVzwrRlcyBhbmQgaXQg
aXMgYSBsb3Qgb2YgYnJpbGxpYW50IGlkZWFzIGluIGl0LgoKT2theSwgSSB0aGluayBhZnRlciB0
aGlzIGVzc2F5IGl0IHNob3VsZCByZWFsbHkgYmUgY2xlYXIgdGhhdCB0aGUgUkRCIAppdHNlbGYg
Y2FuIGhhbmRsZSBpbnNhbmUgYW1vdW50cyBvZiBjYXBhY2l0eSBpbiBhbnkgY2FzZS4KCj4gVGhp
cyBtZWFucyB0aGF0IGFueSBlcnJvcnMgYW5kIGZhaWx1cmVzIHRvIG1vdW50IHNob3VsZCBsaWUg
d2l0aCB0aGUKPiBSREIgUGFyc2VyLCB0aGUgZmlsZXN5c3RlbSwgdGhlIGRldmljZSBkcml2ZXIs
IGFuZCBhbnkgaW50ZXJ2ZW5pbmcKPiBjb2RlIGJldHdlZW4gdGhlIHRocmVlLiBUaGUgUkRCcyBj
YW4gZXhwcmVzcyBhIHZhbHVlIGZhciBncmVhdGVyIHRoYW4KPiB0aGUgbnVtYmVyIG9mIGF0b21z
IGVzdGltYXRlZCB0byBleGlzdCBpbiB0aGUgb2JzZXJ2YWJsZSB1bml2ZXJzZS4gU28KPiBSREJz
IHdvbid0IGV2ZXIgbmVlZCBhIGNoYW5nZSBmb3IgZGlzayBzaXplIGNvbnNpZGVyYXRpb25zLiBU
aGUKPiBwYXJzZXIgYW5kIGZpbGVzeXN0ZW1zIHByb2JhYmx5IGFuZCBwb3NzaWJseSwgcmVzcGVj
dGl2ZWx5LCB3aWxsLgoKRXhhY3RseSB0aGF0LgoKRmlsZXN5c3RlbXMgaGF2ZSBvdGhlciBsaW1p
dHMuIEFuZCB0aGV5IG5lZWQgdG8gYmFpbCBvdXQgYmVmb3JlIG1vdW50aW5nIAphbnl0aGluZyBi
ZXlvbmQgdGhlaXIgbGltaXRzLgoKU28gdGhlIFJEQiBwYXJzZXIgaXMganVzdCBmaW5lIHdpdGgg
YXQgbGVhc3QgNjQgYml0IGNhbGN1bGF0aW9ucy4gT2YgCmNvdXJzZSBpdCBzaG91bGQgYmFpbCBv
dXQgYmlnIHRpbWUgaWYgaXQgZXZlciBjb21lcyBhY3Jvc3MgYSBSREIgd2hlcmUgCgozMiBiaXQg
cmRiX0N5bGluZGVycyAqIDMyIGJpdCByZGJfSGVhZHMgKiAzMiBiaXQgcmRiX1NlY3RvcnMKCmRv
ZXMgbm90IGZpdCB3aXRoaW4gNjQgQml0LiBVbmxlc3MgaXQgdXNlcyA5NiBvciBldmVuIDEyOCBi
aXQgCmNhbGN1bGF0aW9ucyA6KS4gRXZlbiB0b3VnaCBJIHRoaW5rIGl0IGlzIHVubGlrZWx5IHRo
YXQgdGhlcmUgZXZlciB3b3VsZCAKYmUgc3VjaCBhbiBSREIsIEkgd291bGQgcmF0aGVyIHBsYXkg
aXQgdmVyeSBzYWZlIHdpdGggYW55dGhpbmcgcmVnYXJkaW5nIApwYXJ0aXRpb25pbmcgYW5kIGFk
ZHJlc3NpbmcgY2FwYWNpdHkgb24gZGlza3MuIE5ldmVyIGV2ZXIgb3ZlcmZsb3cgCndpdGhvdXQg
YmFpbGluZyBvdXQuIE9yIHRvIGhhdmUgaXQgd2l0aCBKb2FubmXCtHMgd29yZHM6Cgo+IEVycm9y
IG91dCBhbmQgZG9uJ3QgbW91bnQgZGlza3Mgb3IgcGFydGl0aW9ucyB0aGF0IGFyZSBvdXRzaWRl
IHRoZQo+IHNpemUgcmFuZ2UgeW91ciBlbGVtZW50IG9mIHRoZSBkYXRhcGF0aCBmcm9tIGRpc2sg
dG8gdXNlcidzIHByb2dyYW0KPiBjYW4gaGFuZGxlIGF0IHRoZSBmaXJzdCBwbGFjZSB0aGlzIGNh
biBiZSBkZXRlY3RlZC4gSXQncyB0aGUgcG9saXRlCj4gdGhpbmcgdG8gZG8gdG8gYSB1c2VyJ3Mg
ZGF0YSByYXRoZXIgdGhhbiB0aGUgTWF4aW5lIFdhdGVycyB0aGluZyB0bwo+IGRvLgoKRnVsbHkg
YWdyZWVkLgoKQW1lbiwKLS0gCk1hcnRpbg==

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[Michael Schmitz]

Martin,


Am 30.06.18 um 09:24 schrieb Martin Steigerwald:
> Hi Michael.
>
> Michael Schmitz - 29.06.18, 11:07:
>>> But it's up to the person (which is not Linux) formatting the disk
>>> to
>>> not try to use
>>> it on systems that cannot handle it, and may destroy it.
>>>
>>>>> Let me clarify: what exactly would the kernel option allow? When
>>>>> to use it?>> 
>>>> Whether to use it if safe (on Linux). But whatever Linux does
>>>> (after
>>>> this patch), access will go to the right area of the disk (as
>>>> specified by the RDB) so Linux won't any longer stomp on anything
>>>> that would have mattered to 32 bit disk drivers. So it really
>>>> should be safe.> 
>>> Personally, I see no reason to depend on a kernel option, if it is
>>> safe to use. Just use it.
>> So to recap - someone partitions a disk on AmigaOS 4.x, taking
>> advantage of the large block device support there.
>> Using that disk on AmigaOS 3.1, data loss ensues. Whether or not Linux
>> (patched) ever touched the disk has no impact on that outcome.
> I am not even completely sure about that. Frankly I have no idea what 
> would happen when using such a disk on AmigaOS 3.1 *without* NSDPatch or 
> TD64 support (I think you could patch AmigaOS 3.1 with 64 Bit support 
> already and some 3rd party harddisk controllers by Phase 5 hat TD64 
> support at that time already). Unless I try it, which I won´t at the 
> moment, I´d say the behaviour is largely undefined.
The problem that still remains is with unpatched legacy versions. RDB
does support large enough partitions out of the box, due to C/H/S all
using u32. We all agree there. The question is with file systems and
device drivers - someone, at some point, has to recast the block offset
at filesystem level to an offset expressed in the devices's native block
size (regardless of block size used by RDB and filesystem). There are
MOs that use 2k block sizes natively, but as far as I am aware, disks
are usually 512 byte block size (the last disk system I remember where
you could chose block size at the hardware level, at format time, was an
Emulex disk controller on  MicroVax II).

I've dug through a few specs to get a better handle on this. SCSI-2 used
a maximum of 4 bytes for block addresses. IDE used 6 bytes, so at the
time these systems were designed, there already was a way of requesting
data at offsets beyond what can be expressed in 32 bits. I would hope
that any system supporting IDE would have done offset calculations in 64
bits. Is that too optimistic?
> But hey, undefined means it may just overwrite start overwriting from 
> the beginning of the disk beyond 32 bit. And I think that is quite 
> likely. It could also crash, but if its an overflow I don´t think why it 
> would crash. Anyway, I never tried this out.
If offsets are calculated in 32 bits, excess bits are truncated (I don't
think integer overflows ever trapped?). That wraps the offset to
somewhere within the first 2 TB of the disk (maybe less if signed int
was used, like in our parser).
> But in any way: This would happen or not happen no matter whether Linux 
> parsed the RDB or not.
>
> I still think that the native OS warning really does not hurt… but I´d 
> spare myself the kernel option. Having the warning without the kernel 
> option would be a compromise between being cautious and being bold :).
Yep, I've come around to that conclusion, too. Just need to test it
properly. And I still need to check what will happen if we allow such a
large partition table to parse on a kernel without LBD support (i.e.,
are there any downstream checks in place to refuse partitions larger
than what can be handled by 32 bit).

Cheers,

    Michael


> Thanks,

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[jdow]

On 20180629 14:45, Martin Steigerwald wrote:
 > Beware: Essay ahead which proofs it to the point that there is no
 > overflow in RDB before 96 bits maximum value of sectors:

Time to go into more detail on RDBs. It isn't as simple as it started to appear.

extract from hardblocks.h RDSK block definition
===8<---
     ULONG   rdb_BlockBytes;	/* size of disk blocks */
...
     ULONG   rdb_Cylinders;	/* number of drive cylinders */
     ULONG   rdb_Sectors;	/* sectors per track */
     ULONG   rdb_Heads;		/* number of drive heads */
     ...
     ULONG   rdb_LoCylinder;	/* low cylinder of partitionable disk area */
     ULONG   rdb_HiCylinder;	/* high cylinder of partitionable data area */
     ULONG   rdb_CylBlocks;	/* number of blocks available per cylinder */
===8<---
This has the hard limit embodied within it, unfortunately.

The first four values above give you hope for 2^128 bytes. The next three may 
trash some of it when all three are considered.

Since a cylinder is sectors times heads we have 2^64 blocks capacity embodied in 
rdb_LoCylinder and rdb_HiCylinder. But, our hopes are deftly dashed by the last 
value rdb_CylBlocks which places a hard limit on the product of rdb_Heads and 
rdb_Sectors of 2^32. But, that still allows is a fairly large disk. 2^32-1 
blocks per cylinder times block size, rdb_BlockBytes, of 2^32, although the 
larger block sizes are um er sort of putrid to use. Similar limitations exist 
within dos.h in the InfoData and DosEnvec structure, among other likely places.

Approaches "exist" to allowing large partitions. Some of them are unattractive, 
probably all of them as a matter of fact.
1) For large disks move to GPT and be done with it.
2) "lie" and teach the filesystems to ignore rdb_CylBlocks and similar values 
elsewhere. Then the sky is the limit.
3) Invent a "PA64" 64 bit RDB entry and the other internal structures to make it 
work, InfoData64, DosEnvec64, and so on.

Solution 2 might be the least disruptive way to do it. BUT, a whole host of 
utilities like "info" will have to be tweaked to handle "rdb_CylBlocks" becoming 
meaningless.

So this is what happened with some simple includes mining while I am playing 
hooky from doing some real work.

Good luck, gentlemen.
{^_^}

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[Michael Schmitz]

Joanne,


Am 30.06.18 um 11:24 schrieb jdow:
>
> On 20180629 14:45, Martin Steigerwald wrote:
> > Beware: Essay ahead which proofs it to the point that there is no
> > overflow in RDB before 96 bits maximum value of sectors:
>
> Time to go into more detail on RDBs. It isn't as simple as it started
> to appear.
>
> extract from hardblocks.h RDSK block definition
> ===8<---
>     ULONG   rdb_BlockBytes;    /* size of disk blocks */
> ...
>     ULONG   rdb_Cylinders;    /* number of drive cylinders */
>     ULONG   rdb_Sectors;    /* sectors per track */
>     ULONG   rdb_Heads;        /* number of drive heads */
>     ...
>     ULONG   rdb_LoCylinder;    /* low cylinder of partitionable disk
> area */
>     ULONG   rdb_HiCylinder;    /* high cylinder of partitionable data
> area */
>     ULONG   rdb_CylBlocks;    /* number of blocks available per
> cylinder */
> ===8<---
> This has the hard limit embodied within it, unfortunately.
>
> The first four values above give you hope for 2^128 bytes. The next
> three may trash some of it when all three are considered.
>
> Since a cylinder is sectors times heads we have 2^64 blocks capacity
> embodied in rdb_LoCylinder and rdb_HiCylinder. But, our hopes are
> deftly dashed by the last value rdb_CylBlocks which places a hard
> limit on the product of rdb_Heads and rdb_Sectors of 2^32. But, that
> still allows is a fairly large disk. 2^32-1 blocks per cylinder times
> block size, rdb_BlockBytes, of 2^32, although the larger block sizes
> are um er sort of putrid to use. Similar limitations exist within
> dos.h in the InfoData and DosEnvec structure, among other likely places.
>

As far as Linux is concerned, rdb_CylBlocks is used nowhere, neither in
the RDB parser nor in the AFFS filesystem driver. Only the partition
list is parsed.

Should we use rdb_LoCylinder*rdbCylBlocks and
rdb_HiCylinder*rdbCylBlocks in the RDB parser to verify the detected
partitions are valid according to the RDB's own specified limits? Or can
we absolutely rely on the partitioning tool getting that right?

Any similar surprises in the partition list data structures? The header
I have in Linux is largely non-descriptive there:

struct PartitionBlock {
        __be32  pb_ID;
        __be32  pb_SummedLongs;
        __s32   pb_ChkSum;
        __u32   pb_HostID;
        __be32  pb_Next;
        __u32   pb_Flags;
        __u32   pb_Reserved1[2];
        __u32   pb_DevFlags;
        __u8    pb_DriveName[32];
        __u32   pb_Reserved2[15];
        __be32  pb_Environment[17];
        __u32   pb_EReserved[15];
};

As far as I can guess from the code, pb_Environment[3] (number of heads)
and pb_Environment[5] (number of sectors per cylinder) are abitrarily
chosen so the partition size can be expressed as a difference between
pb_Environment[9] and pb_Environment[10] (low and high cylinder
addresses), which places restrictions on both partition size and
alignment that depend on where on the disk a partition is placed?

Cheers,

    Michael

> Approaches "exist" to allowing large partitions. Some of them are
> unattractive, probably all of them as a matter of fact.
> 1) For large disks move to GPT and be done with it.
> 2) "lie" and teach the filesystems to ignore rdb_CylBlocks and similar
> values elsewhere. Then the sky is the limit.
> 3) Invent a "PA64" 64 bit RDB entry and the other internal structures
> to make it work, InfoData64, DosEnvec64, and so on.
>
> Solution 2 might be the least disruptive way to do it. BUT, a whole
> host of utilities like "info" will have to be tweaked to handle
> "rdb_CylBlocks" becoming meaningless.
>
> So this is what happened with some simple includes mining while I am
> playing hooky from doing some real work.
>
> Good luck, gentlemen.
> {^_^}

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[jdow]

On 20180629 16:24, Michael Schmitz wrote:
 > Martin,
 >
 >
...
 > The problem that still remains is with unpatched legacy versions. RDB
 > does support large enough partitions out of the box, due to C/H/S all
 > using u32. We all agree there. The question is with file systems and

Nope, I bothered to read the source code includes files. As long as anything in 
the OS uses the blocks per cylinder variables in various structures you're SOL. 
rdb_CylinderBlocks is just for starters. The GlobalVec entry also includes a 
CylinderBlocks entry. And it's only uint32_t size.

As I say, good luck gentlemen. Changes, big changes, probably need to be made to 
go beyond 2TB except by using larger block sizes. But, 64k block sizes are sort 
of "putrid" to say the least.

{^_^}

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[jdow]

On 20180629 17:44, Michael Schmitz wrote:

 > struct PartitionBlock {
 >          __be32  pb_ID;
 >          __be32  pb_SummedLongs;
 >          __s32   pb_ChkSum;
 >          __u32   pb_HostID;
 >          __be32  pb_Next;
 >          __u32   pb_Flags;
 >          __u32   pb_Reserved1[2];
 >          __u32   pb_DevFlags;
 >          __u8    pb_DriveName[32];
 >          __u32   pb_Reserved2[15];
 >          __be32  pb_Environment[17];
 >          __u32   pb_EReserved[15];
 > };
  pb_Environment = a struct DosEnvec and it is 20 ULONGs in size. I believe you 
are looking at some old include files. These got added to the end of the 
DosEnvec structure:
     ULONG de_Baud;	     /* Baud rate for serial handler */
     ULONG de_Control;	     /* Control word for handler/filesystem */
     ULONG de_BootBlocks;     /* Number of blocks containing boot code */

 > As far as I can guess from the code, pb_Environment[3] (number of heads)
 > and pb_Environment[5] (number of sectors per cylinder) are abitrarily
 > chosen so the partition size can be expressed as a difference between
 > pb_Environment[9] and pb_Environment[10] (low and high cylinder
 > addresses), which places restrictions on both partition size and
 > alignment that depend on where on the disk a partition is placed?
If you do not teach the OS to ignore Cylinder Blocks type entries and use some 
math on heads and blocks per track the disk size is relatively stuck modulo 
using large blocks.

{^_^}

On 20180629 17:44, Michael Schmitz wrote:
> Joanne,
> 
> 
> Am 30.06.18 um 11:24 schrieb jdow:
>>
>> On 20180629 14:45, Martin Steigerwald wrote:
>>> Beware: Essay ahead which proofs it to the point that there is no
>>> overflow in RDB before 96 bits maximum value of sectors:
>>
>> Time to go into more detail on RDBs. It isn't as simple as it started
>> to appear.
>>
>> extract from hardblocks.h RDSK block definition
>> ===8<---
>>      ULONG   rdb_BlockBytes;    /* size of disk blocks */
>> ...
>>      ULONG   rdb_Cylinders;    /* number of drive cylinders */
>>      ULONG   rdb_Sectors;    /* sectors per track */
>>      ULONG   rdb_Heads;        /* number of drive heads */
>>      ...
>>      ULONG   rdb_LoCylinder;    /* low cylinder of partitionable disk
>> area */
>>      ULONG   rdb_HiCylinder;    /* high cylinder of partitionable data
>> area */
>>      ULONG   rdb_CylBlocks;    /* number of blocks available per
>> cylinder */
>> ===8<---
>> This has the hard limit embodied within it, unfortunately.
>>
>> The first four values above give you hope for 2^128 bytes. The next
>> three may trash some of it when all three are considered.
>>
>> Since a cylinder is sectors times heads we have 2^64 blocks capacity
>> embodied in rdb_LoCylinder and rdb_HiCylinder. But, our hopes are
>> deftly dashed by the last value rdb_CylBlocks which places a hard
>> limit on the product of rdb_Heads and rdb_Sectors of 2^32. But, that
>> still allows is a fairly large disk. 2^32-1 blocks per cylinder times
>> block size, rdb_BlockBytes, of 2^32, although the larger block sizes
>> are um er sort of putrid to use. Similar limitations exist within
>> dos.h in the InfoData and DosEnvec structure, among other likely places.
>>
> 
> As far as Linux is concerned, rdb_CylBlocks is used nowhere, neither in
> the RDB parser nor in the AFFS filesystem driver. Only the partition
> list is parsed.
> 
> Should we use rdb_LoCylinder*rdbCylBlocks and
> rdb_HiCylinder*rdbCylBlocks in the RDB parser to verify the detected
> partitions are valid according to the RDB's own specified limits? Or can
> we absolutely rely on the partitioning tool getting that right?
> 
> Any similar surprises in the partition list data structures? The header
> I have in Linux is largely non-descriptive there:
> 
> struct PartitionBlock {
>          __be32  pb_ID;
>          __be32  pb_SummedLongs;
>          __s32   pb_ChkSum;
>          __u32   pb_HostID;
>          __be32  pb_Next;
>          __u32   pb_Flags;
>          __u32   pb_Reserved1[2];
>          __u32   pb_DevFlags;
>          __u8    pb_DriveName[32];
>          __u32   pb_Reserved2[15];
>          __be32  pb_Environment[17];
>          __u32   pb_EReserved[15];
> };
> 
> As far as I can guess from the code, pb_Environment[3] (number of heads)
> and pb_Environment[5] (number of sectors per cylinder) are abitrarily
> chosen so the partition size can be expressed as a difference between
> pb_Environment[9] and pb_Environment[10] (low and high cylinder
> addresses), which places restrictions on both partition size and
> alignment that depend on where on the disk a partition is placed?
> 
> Cheers,
> 
>      Michael
> 
>> Approaches "exist" to allowing large partitions. Some of them are
>> unattractive, probably all of them as a matter of fact.
>> 1) For large disks move to GPT and be done with it.
>> 2) "lie" and teach the filesystems to ignore rdb_CylBlocks and similar
>> values elsewhere. Then the sky is the limit.
>> 3) Invent a "PA64" 64 bit RDB entry and the other internal structures
>> to make it work, InfoData64, DosEnvec64, and so on.
>>
>> Solution 2 might be the least disruptive way to do it. BUT, a whole
>> host of utilities like "info" will have to be tweaked to handle
>> "rdb_CylBlocks" becoming meaningless.
>>
>> So this is what happened with some simple includes mining while I am
>> playing hooky from doing some real work.
>>
>> Good luck, gentlemen.
>> {^_^}
> 

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[Michael Schmitz]

Joanne,


Am 30.06.18 um 12:57 schrieb jdow:
> On 20180629 17:44, Michael Schmitz wrote:
>
> > struct PartitionBlock {
> >          __be32  pb_ID;
> >          __be32  pb_SummedLongs;
> >          __s32   pb_ChkSum;
> >          __u32   pb_HostID;
> >          __be32  pb_Next;
> >          __u32   pb_Flags;
> >          __u32   pb_Reserved1[2];
> >          __u32   pb_DevFlags;
> >          __u8    pb_DriveName[32];
> >          __u32   pb_Reserved2[15];
> >          __be32  pb_Environment[17];
> >          __u32   pb_EReserved[15];
> > };
>  pb_Environment = a struct DosEnvec and it is 20 ULONGs in size. I
> believe you are looking at some old include files.

Without looking at ancient git history, I'd say between 1993 and 1996.

> These got added to the end of the DosEnvec structure:
>     ULONG de_Baud;         /* Baud rate for serial handler */
>     ULONG de_Control;         /* Control word for handler/filesystem */
>     ULONG de_BootBlocks;     /* Number of blocks containing boot code */
>
> > As far as I can guess from the code, pb_Environment[3] (number of
> heads)
> > and pb_Environment[5] (number of sectors per cylinder) are abitrarily
> > chosen so the partition size can be expressed as a difference between
> > pb_Environment[9] and pb_Environment[10] (low and high cylinder
> > addresses), which places restrictions on both partition size and
> > alignment that depend on where on the disk a partition is placed?
> If you do not teach the OS to ignore Cylinder Blocks type entries and
> use some math on heads and blocks per track the disk size is
> relatively stuck modulo using large blocks.

As long as AmigaOS and Linux agree on how to express start and end
offset for the partitions, that's fine.

But I read your other mail to mean that we're stuck to 2 TB disks for
now. I don't follow that - we can have partitions of 2 TB each by maxing
out rdb_CylBlocks as long as we use 512 bytes per block (since the
product of cylinders and blocks per cylinder is limited to 32 bits) and
using one cylinder per partition (32 bits available there as well)?

But the rdb_CylBlocks limit also means we're safe with 64 bit sector_t
in Linux. Best add a check in the parser to warn us if the product of
head count and sectors per cylinder overflows 32 bit though.

Cheers,

    Michael
>
> {^_^}
>
> On 20180629 17:44, Michael Schmitz wrote:
>> Joanne,
>>
>>
>> Am 30.06.18 um 11:24 schrieb jdow:
>>>
>>> On 20180629 14:45, Martin Steigerwald wrote:
>>>> Beware: Essay ahead which proofs it to the point that there is no
>>>> overflow in RDB before 96 bits maximum value of sectors:
>>>
>>> Time to go into more detail on RDBs. It isn't as simple as it started
>>> to appear.
>>>
>>> extract from hardblocks.h RDSK block definition
>>> ===8<---
>>>      ULONG   rdb_BlockBytes;    /* size of disk blocks */
>>> ...
>>>      ULONG   rdb_Cylinders;    /* number of drive cylinders */
>>>      ULONG   rdb_Sectors;    /* sectors per track */
>>>      ULONG   rdb_Heads;        /* number of drive heads */
>>>      ...
>>>      ULONG   rdb_LoCylinder;    /* low cylinder of partitionable disk
>>> area */
>>>      ULONG   rdb_HiCylinder;    /* high cylinder of partitionable data
>>> area */
>>>      ULONG   rdb_CylBlocks;    /* number of blocks available per
>>> cylinder */
>>> ===8<---
>>> This has the hard limit embodied within it, unfortunately.
>>>
>>> The first four values above give you hope for 2^128 bytes. The next
>>> three may trash some of it when all three are considered.
>>>
>>> Since a cylinder is sectors times heads we have 2^64 blocks capacity
>>> embodied in rdb_LoCylinder and rdb_HiCylinder. But, our hopes are
>>> deftly dashed by the last value rdb_CylBlocks which places a hard
>>> limit on the product of rdb_Heads and rdb_Sectors of 2^32. But, that
>>> still allows is a fairly large disk. 2^32-1 blocks per cylinder times
>>> block size, rdb_BlockBytes, of 2^32, although the larger block sizes
>>> are um er sort of putrid to use. Similar limitations exist within
>>> dos.h in the InfoData and DosEnvec structure, among other likely
>>> places.
>>>
>>
>> As far as Linux is concerned, rdb_CylBlocks is used nowhere, neither in
>> the RDB parser nor in the AFFS filesystem driver. Only the partition
>> list is parsed.
>>
>> Should we use rdb_LoCylinder*rdbCylBlocks and
>> rdb_HiCylinder*rdbCylBlocks in the RDB parser to verify the detected
>> partitions are valid according to the RDB's own specified limits? Or can
>> we absolutely rely on the partitioning tool getting that right?
>>
>> Any similar surprises in the partition list data structures? The header
>> I have in Linux is largely non-descriptive there:
>>
>> struct PartitionBlock {
>>          __be32  pb_ID;
>>          __be32  pb_SummedLongs;
>>          __s32   pb_ChkSum;
>>          __u32   pb_HostID;
>>          __be32  pb_Next;
>>          __u32   pb_Flags;
>>          __u32   pb_Reserved1[2];
>>          __u32   pb_DevFlags;
>>          __u8    pb_DriveName[32];
>>          __u32   pb_Reserved2[15];
>>          __be32  pb_Environment[17];
>>          __u32   pb_EReserved[15];
>> };
>>
>> As far as I can guess from the code, pb_Environment[3] (number of heads)
>> and pb_Environment[5] (number of sectors per cylinder) are abitrarily
>> chosen so the partition size can be expressed as a difference between
>> pb_Environment[9] and pb_Environment[10] (low and high cylinder
>> addresses), which places restrictions on both partition size and
>> alignment that depend on where on the disk a partition is placed?
>>
>> Cheers,
>>
>>      Michael
>>
>>> Approaches "exist" to allowing large partitions. Some of them are
>>> unattractive, probably all of them as a matter of fact.
>>> 1) For large disks move to GPT and be done with it.
>>> 2) "lie" and teach the filesystems to ignore rdb_CylBlocks and similar
>>> values elsewhere. Then the sky is the limit.
>>> 3) Invent a "PA64" 64 bit RDB entry and the other internal structures
>>> to make it work, InfoData64, DosEnvec64, and so on.
>>>
>>> Solution 2 might be the least disruptive way to do it. BUT, a whole
>>> host of utilities like "info" will have to be tweaked to handle
>>> "rdb_CylBlocks" becoming meaningless.
>>>
>>> So this is what happened with some simple includes mining while I am
>>> playing hooky from doing some real work.
>>>
>>> Good luck, gentlemen.
>>> {^_^}
>>

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[jdow]

On 20180629 18:31, Michael Schmitz wrote:> Joanne,
 >
 >
 > Am 30.06.18 um 12:57 schrieb jdow:
 >> On 20180629 17:44, Michael Schmitz wrote:
 >>
 >>> struct PartitionBlock {
 >>>            __be32  pb_ID;
 >>>            __be32  pb_SummedLongs;
 >>>            __s32   pb_ChkSum;
 >>>            __u32   pb_HostID;
 >>>            __be32  pb_Next;
 >>>            __u32   pb_Flags;
 >>>            __u32   pb_Reserved1[2];
 >>>            __u32   pb_DevFlags;
 >>>            __u8    pb_DriveName[32];
 >>>            __u32   pb_Reserved2[15];
 >>>            __be32  pb_Environment[17];
 >>>            __u32   pb_EReserved[15];
 >>> };
 >>   pb_Environment = a struct DosEnvec and it is 20 ULONGs in size. I
 >> believe you are looking at some old include files.
 >
 > Without looking at ancient git history, I'd say between 1993 and 1996.
 >
 >> These got added to the end of the DosEnvec structure:
 >>      ULONG de_Baud;         /* Baud rate for serial handler */
 >>      ULONG de_Control;         /* Control word for handler/filesystem */
 >>      ULONG de_BootBlocks;     /* Number of blocks containing boot code */
 >>
 >>> As far as I can guess from the code, pb_Environment[3] (number of
 >> heads)
 >>> and pb_Environment[5] (number of sectors per cylinder) are abitrarily
 >>> chosen so the partition size can be expressed as a difference between
 >>> pb_Environment[9] and pb_Environment[10] (low and high cylinder
 >>> addresses), which places restrictions on both partition size and
 >>> alignment that depend on where on the disk a partition is placed?
 >> If you do not teach the OS to ignore Cylinder Blocks type entries and
 >> use some math on heads and blocks per track the disk size is
 >> relatively stuck modulo using large blocks.
 >
 > As long as AmigaOS and Linux agree on how to express start and end
 > offset for the partitions, that's fine.
 >
 > But I read your other mail to mean that we're stuck to 2 TB disks for
 > now. I don't follow that - we can have partitions of 2 TB each by maxing
 > out rdb_CylBlocks as long as we use 512 bytes per block (since the
 > product of cylinders and blocks per cylinder is limited to 32 bits) and
 > using one cylinder per partition (32 bits available there as well)?
 >
 > But the rdb_CylBlocks limit also means we're safe with 64 bit sector_t
 > in Linux. Best add a check in the parser to warn us if the product of
 > head count and sectors per cylinder overflows 32 bit though.
 >
 > Cheers,
 >
 >      Michael
How long did it tale s to get to 10 TB disks from 2 TB disks. And a new SD Card 
spec allows for 128 TB disks. Block sizes get sort of ridiculous as you get past 
about 8k bytes or about 32 TB or about two years from now.

Do you want to create disks that will fail on AmigaDOS? AmigaDOS, as far as I 
know, makes heavy use of Cylinder Blocks values. It calculating Cylinder Blocks 
overflows when creating the disk's RDBs the user MUST be informed it is unsafe 
to put on a real Amiga. (I'd also suggest teaching Linux to understand RDSL, 
which would be RDSK++ sort of. Then use that if Cylinder Blocks overflows.) The 
value you will not be able to fill in the DosEnvec structure is:
	ULONG de_HighCyl;	     /* max cylinder. drive specific */

So accessing larger disks once you hit 2 TB means you must increase the logical 
block size. And eventually that will waste HUGE amounts of files when small 
files are being stored.

Any solution will require action on the part of the people developing AmigaDOS 
follow-ons. You might want to get them motivated, somehow, and proceed from 
there with a request to be informed of any RDB changes. I'd suggest to them that 
removing sensitivity to Cylinder Blocks sorts of values from the entire system 
probably would be painful but the simplest solution.

{^_^}

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[Michael Schmitz]

Joanne,


Am 30.06.18 um 15:56 schrieb jdow:
>
> >>> As far as I can guess from the code, pb_Environment[3] (number of
> >> heads)
> >>> and pb_Environment[5] (number of sectors per cylinder) are abitrarily
> >>> chosen so the partition size can be expressed as a difference between
> >>> pb_Environment[9] and pb_Environment[10] (low and high cylinder
> >>> addresses), which places restrictions on both partition size and
> >>> alignment that depend on where on the disk a partition is placed?
> >> If you do not teach the OS to ignore Cylinder Blocks type entries and
> >> use some math on heads and blocks per track the disk size is
> >> relatively stuck modulo using large blocks.
> >
> > As long as AmigaOS and Linux agree on how to express start and end
> > offset for the partitions, that's fine.
> >
> > But I read your other mail to mean that we're stuck to 2 TB disks for
> > now. I don't follow that - we can have partitions of 2 TB each by
> maxing
> > out rdb_CylBlocks as long as we use 512 bytes per block (since the
> > product of cylinders and blocks per cylinder is limited to 32 bits) and
> > using one cylinder per partition (32 bits available there as well)?
> >
> > But the rdb_CylBlocks limit also means we're safe with 64 bit sector_t
> > in Linux. Best add a check in the parser to warn us if the product of
> > head count and sectors per cylinder overflows 32 bit though.
> >
> > Cheers,
> >
> >      Michael
> How long did it tale s to get to 10 TB disks from 2 TB disks. And a
> new SD Card spec allows for 128 TB disks. Block sizes get sort of
> ridiculous as you get past about 8k bytes or about 32 TB or about two
> years from now.

I get that - I just don't get why 32 bits for cylinders plus 32 bits for
blocks per cylinder equals 2 TB (4G of 512 byte blocks). But I don't
know what other limits exist that may restrict the total number of
blocks to 32 bits.

>
> Do you want to create disks that will fail on AmigaDOS? AmigaDOS, as
> far as I know, makes heavy use of Cylinder Blocks values. It
> calculating Cylinder Blocks overflows when creating the disk's RDBs
> the user MUST be informed it is 

I'm not at all planning to create disks for AmigaDOS. I just need to
know what combinations of cylinders, heads and sectors are possible to
encounter on disks that have been created with native tools. Well,
assuming sufficient amounts of braindamage in the corresponding Linux
tools, knowing the absolute outer limits of what these tools could do
would be nice as well, but someone using amiga-fdisk to create a RDSK
block for a 10 TB disk fully deserves any punishment that invites.

(Actually, I lied there. I do plan to create a RDSK block for a 2 TB
disk image where cylinder, head and sector counts all approach the 32
bit limit, just to see that my overflow checks work as intended. But
that's strictly for Linux testing).

> unsafe to put on a real Amiga. (I'd also suggest teaching Linux to
> understand RDSL, which would be RDSK++ sort of. Then use that if
> Cylinder Blocks overflows.) The value you will not be able to fill in
> the DosEnvec structure is:
>     ULONG de_HighCyl;         /* max cylinder. drive specific */

OK, so Cylinder Blocks overflowing is a red flag, and requires to abort
parsing the partition table right away? And HighCyl really means the
max. number of logical blocks, not cylinders (which would have
nr_heads*nr_sects many blocks)? That's probably the cause for my confusion.

>
> So accessing larger disks once you hit 2 TB means you must increase
> the logical block size. And eventually that will waste HUGE amounts of
> files when small files are being stored.
Just like small inodes wastes huge amounts of space for metadata. It's a
tradeoff, and AFFS on a RDSK format disk probably isn't the right choice
for huge disks. Never mind that - if someone _does_ go that way, we need
to make sure we can parse the RDSK information correctly. And if such a
disk causes the 64 bit sector_t in Linux to overflow, I'd like the
parser to spot that, too.

Thanks for your immense patience in explaining all these subtleties to me.

Cheers,

    Michael

>
> Any solution will require action on the part of the people developing
> AmigaDOS follow-ons. You might want to get them motivated, somehow,
> and proceed from there with a request to be informed of any RDB
> changes. I'd suggest to them that removing sensitivity to Cylinder
> Blocks sorts of values from the entire system probably would be
> painful but the simplest solution.
>
> {^_^}
> -- 
> To unsubscribe from this list: send the line "unsubscribe linux-m68k" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[jdow]

Let's get everybody:

On 20180629 22:26, Michael Schmitz wrote:
 > Joanne,
 >
 >
 > Am 30.06.18 um 15:56 schrieb jdow:
 >>
 >>>>> As far as I can guess from the code, pb_Environment[3] (number of
 >>>> heads)
 >>>>> and pb_Environment[5] (number of sectors per cylinder) are abitrarily
 >>>>> chosen so the partition size can be expressed as a difference between
 >>>>> pb_Environment[9] and pb_Environment[10] (low and high cylinder
 >>>>> addresses), which places restrictions on both partition size and
 >>>>> alignment that depend on where on the disk a partition is placed?
 >>>> If you do not teach the OS to ignore Cylinder Blocks type entries and
 >>>> use some math on heads and blocks per track the disk size is
 >>>> relatively stuck modulo using large blocks.
 >>>
 >>> As long as AmigaOS and Linux agree on how to express start and end
 >>> offset for the partitions, that's fine.
 >>>
 >>> But I read your other mail to mean that we're stuck to 2 TB disks for
 >>> now. I don't follow that - we can have partitions of 2 TB each by
 >> maxing
 >>> out rdb_CylBlocks as long as we use 512 bytes per block (since the
 >>> product of cylinders and blocks per cylinder is limited to 32 bits) and
 >>> using one cylinder per partition (32 bits available there as well)?
 >>>
 >>> But the rdb_CylBlocks limit also means we're safe with 64 bit sector_t
 >>> in Linux. Best add a check in the parser to warn us if the product of
 >>> head count and sectors per cylinder overflows 32 bit though.
 >>>
 >>> Cheers,
 >>>
 >>>        Michael
 >> How long did it tale s to get to 10 TB disks from 2 TB disks. And a
 >> new SD Card spec allows for 128 TB disks. Block sizes get sort of
 >> ridiculous as you get past about 8k bytes or about 32 TB or about two
 >> years from now.
 >
 > I get that - I just don't get why 32 bits for cylinders plus 32 bits for
 > blocks per cylinder equals 2 TB (4G of 512 byte blocks). But I don't
 > know what other limits exist that may restrict the total number of
 > blocks to 32 bits.

It overflows uint32_t cylinder blocks aka blocks per cylinder. Linux doesn't 
care. AmigaDOS surely does. If YOU make partitions really large for yourself 
that's OK. If Joe Amigoid does it the potential for an angry red turning to 
purple face is high.

 >> Do you want to create disks that will fail on AmigaDOS? AmigaDOS, as
 >> far as I know, makes heavy use of Cylinder Blocks values. It
 >> calculating Cylinder Blocks overflows when creating the disk's RDBs
 >> the user MUST be informed it is
 >
 > I'm not at all planning to create disks for AmigaDOS. I just need to
 > know what combinations of cylinders, heads and sectors are possible to
 > encounter on disks that have been created with native tools. Well,
 > assuming sufficient amounts of braindamage in the corresponding Linux
 > tools, knowing the absolute outer limits of what these tools could do
 > would be nice as well, but someone using amiga-fdisk to create a RDSK
 > block for a 10 TB disk fully deserves any punishment that invites.

Native AmigaDOS tools SHOULD NOT be able to create something that overflows 
CylinderBlocks values. However, if it can that creates an interesting test case 
to see what various tools, like the AmigaDOS "info" command, do when they they 
are run on such a disk. I don't have OS source to perform searches. And I am not 
setup to feed the system something obscene.

 > (Actually, I lied there. I do plan to create a RDSK block for a 2 TB
 > disk image where cylinder, head and sector counts all approach the 32
 > bit limit, just to see that my overflow checks work as intended. But
 > that's strictly for Linux testing).
 >
 >> unsafe to put on a real Amiga. (I'd also suggest teaching Linux to
 >> understand RDSL, which would be RDSK++ sort of. Then use that if
 >> Cylinder Blocks overflows.) The value you will not be able to fill in
 >> the DosEnvec structure is:
 >>      ULONG de_HighCyl;         /* max cylinder. drive specific */
 >
 > OK, so Cylinder Blocks overflowing is a red flag, and requires to abort
 > parsing the partition table right away? And HighCyl really means the
 > max. number of logical blocks, not cylinders (which would have
 > nr_heads*nr_sects many blocks)? That's probably the cause for my confusion.

I think I picked the wrong value. In RDSK itself this value is what overflows:
     ULONG   rdb_CylBlocks;    /* number of blocks available per cylinder */
And I think that floats around the system in many places with different names. 
As mentioned the "info" command is one item to test. If no crashes are found 
then AmigaDOS may be clean up to obscene sizes. At the moment I do not remember 
what hdwrench.library does with that value other than pass it along as read. Nor 
am I sure what it generates as any suggested values. I don't at this time have a 
disk I can mount as a disk on WinUAE that is more than 2TB. And my Amigas speak 
SCSI so I have no disk for them, either, even if they still boot.

 >
 >>
 >> So accessing larger disks once you hit 2 TB means you must increase
 >> the logical block size. And eventually that will waste HUGE amounts of
 >> files when small files are being stored.
 > Just like small inodes wastes huge amounts of space for metadata. It's a
 > tradeoff, and AFFS on a RDSK format disk probably isn't the right choice
 > for huge disks. Never mind that - if someone _does_ go that way, we need
 > to make sure we can parse the RDSK information correctly. And if such a
 > disk causes the 64 bit sector_t in Linux to overflow, I'd like the
 > parser to spot that, too.
 >
 > Thanks for your immense patience in explaining all these subtleties to me.
 >
 > Cheers,
 >
 >      Michael

And I'm rushing too much so I'm sorry I am making errors. This stuff is 25 years 
in the past since I last looked at it seriously.

{^_^}

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[Martin Steigerwald]

Whoa, my summary essay triggered digging even more accurately into that=20
matter. For some obscure reason I am even enjoying this. :)

jdow - 30.06.18, 05:56:
> On 20180629 18:31, Michael Schmitz wrote:> Joanne,
>=20
>  > Am 30.06.18 um 12:57 schrieb jdow:
>  >> On 20180629 17:44, Michael Schmitz wrote:
>  >>> struct PartitionBlock {
>  >>>=20
>  >>>            __be32  pb_ID;
>  >>>            __be32  pb_SummedLongs;
>  >>>            __s32   pb_ChkSum;
>  >>>            __u32   pb_HostID;
>  >>>            __be32  pb_Next;
>  >>>            __u32   pb_Flags;
>  >>>            __u32   pb_Reserved1[2];
>  >>>            __u32   pb_DevFlags;
>  >>>            __u8    pb_DriveName[32];
>  >>>            __u32   pb_Reserved2[15];
>  >>>            __be32  pb_Environment[17];
>  >>>            __u32   pb_EReserved[15];
>  >>>=20
>  >>> };
>  >>>=20
>  >>   pb_Environment =3D a struct DosEnvec and it is 20 ULONGs in size.
>  >>   I
>  >>=20
>  >> believe you are looking at some old include files.
>  >=20
>  > Without looking at ancient git history, I'd say between 1993 and
>  > 1996. >=20
>  >> These got added to the end of the DosEnvec structure:
>  >>      ULONG de_Baud;         /* Baud rate for serial handler */
>  >>      ULONG de_Control;         /* Control word for
>  >>      handler/filesystem */
>  >>      ULONG de_BootBlocks;     /* Number of blocks containing boot
>  >>      code */
>  >>>=20
>  >>> As far as I can guess from the code, pb_Environment[3] (number of
>  >>=20
>  >> heads)
>  >>=20
>  >>> and pb_Environment[5] (number of sectors per cylinder) are
>  >>> abitrarily
>  >>> chosen so the partition size can be expressed as a difference
>  >>> between
>  >>> pb_Environment[9] and pb_Environment[10] (low and high cylinder
>  >>> addresses), which places restrictions on both partition size and
>  >>> alignment that depend on where on the disk a partition is placed?
>  >>=20
>  >> If you do not teach the OS to ignore Cylinder Blocks type entries
>  >> and
>  >> use some math on heads and blocks per track the disk size is
>  >> relatively stuck modulo using large blocks.
>  >=20
>  > As long as AmigaOS and Linux agree on how to express start and end
>  > offset for the partitions, that's fine.
>  >=20
>  > But I read your other mail to mean that we're stuck to 2 TB disks
>  > for
>  > now. I don't follow that - we can have partitions of 2 TB each by
>  > maxing out rdb_CylBlocks as long as we use 512 bytes per block
>  > (since the product of cylinders and blocks per cylinder is limited
>  > to 32 bits) and using one cylinder per partition (32 bits
>  > available there as well)?
>  >=20
>  > But the rdb_CylBlocks limit also means we're safe with 64 bit
>  > sector_t in Linux. Best add a check in the parser to warn us if
>  > the product of head count and sectors per cylinder overflows 32
>  > bit though.
>  >=20
>  > Cheers,
>  >=20
>  >      Michael
>=20
> How long did it tale s to get to 10 TB disks from 2 TB disks. And a
> new SD Card spec allows for 128 TB disks. Block sizes get sort of
> ridiculous as you get past about 8k bytes or about 32 TB or about two
> years from now.
>=20
> Do you want to create disks that will fail on AmigaDOS? AmigaDOS, as
> far as I know, makes heavy use of Cylinder Blocks values. It
> calculating Cylinder Blocks overflows when creating the disk's RDBs
> the user MUST be informed it is unsafe to put on a real Amiga. (I'd

Joanne, if you are sure on this=E2=80=A6 I=C2=B4d say at least warn if not =
bail out=20
on Cylinder Blocks overflow.

But given what you say here, no partitioning tool on AmigaOS or AmigaOS=20
like operating system would create such an overflow.

Can you verify whether that is the case with the RDB that I attached to=20
the bug report?

Bug 43511 - Partitions: Amiga RDB partition on 2 TB disk way too big,=20
while OK in AmigaOS 4.1=20

https://bugzilla.kernel.org/show_bug.cgi?id=3D43511

https://bugzilla.kernel.org/attachment.cgi?id=3D73771

> also suggest teaching Linux to understand RDSL, which would be RDSK++
> sort of. Then use that if Cylinder Blocks overflows.) The value you
> will not be able to fill in the DosEnvec structure is: ULONG
> de_HighCyl;	     /* max cylinder. drive specific */
>
>=20
> So accessing larger disks once you hit 2 TB means you must increase
> the logical block size. And eventually that will waste HUGE amounts
> of files when small files are being stored.

As far as I am aware, AmigaOS 4.1 still only supports 512 byte sectors.
=20
> Any solution will require action on the part of the people developing
> AmigaDOS follow-ons. You might want to get them motivated, somehow,
> and proceed from there with a request to be informed of any RDB
> changes. I'd suggest to them that removing sensitivity to Cylinder
> Blocks sorts of values from the entire system probably would be
> painful but the simplest solution.

I think for this patch it is important to focus on the *current*=20
situation and make the best out of it.

I am really inclined to point some AmigaOS 4 developers to this=20
discussion and just looked for an archive. Unfortunately there does not=20
appear to be a working one. The one mentioned on

http://www.linux-m68k.org/mail.html

http://aire.ncl.ac.uk/Atari/Mailing-Lists/Linux-680x0-vger-List.index.html

does not send an answer within the HTTP / TCP timeout limit.

I also did not find any archive for linux-block mailing list.

And lore.kernel.org only seems to archive LKML itself which is patch and=20
the discussion we have here is not CC=C2=B4d to.

Any advice?

Thanks,
=2D-=20
Martin

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[Martin Steigerwald]

TWljaGFlbC4gSm9hbm5lLgoKSSBkbyB0aGluayB0aGlzIGRpc2N1c3Npb24gaXMgc2xpZ2h0bHkg
Z2V0dGluZyBvdXQgb2YgaGFuZOKApiBzbyBJIHN1Z2dlc3QgCnRvIGZvY3VzIG9uIHdoYXQgaXRz
IHVwIHRvIHRoZSBrZXJuZWwgdG8gZG8gYW5kIHdoYXQgaXMgbm90LiBBbmQgdG8gCmZvY3VzIG9u
bHkgb24gd2hhdCBpcyB1cCB0byB0aGUgUkRCIHBhcnNlciwgY2F1c2UgdGhlIHBhdGNoIGlzIG9u
IApjaGFuZ2luZyB0aGF0LiBUaGUgUkRCIHBhcnNlciBpcyBub3QgcmVzcG9uc2libGUgZm9yIHdo
YXQgYW55IGZpbGUgCnN5c3RlbSBtYXkgZG8uIFNlY3VyaW5nIEFGRlMgd291bGQgYmUgYSBkaWZm
ZXJlbnQsIGltcG9ydGFudCwgdG9waWMuCgpXaXRoIHRoYXQgbWFpbCBJIGFtIHByb2JhYmx5IG91
dCBhcyB0aGlzIGRpc2N1c3Npb24gdG9vayBhbHJlYWR5IHF1aXRlIGEgCmJpdCBvZiB0aW1lLgoK
QnV0IGZvciBkZXRhaWxzLCByZWFkIG9uOgoKTWljaGFlbCBTY2htaXR6IC0gMzAuMDYuMTgsIDA3
OjI2Ogo+IEFtIDMwLjA2LjE4IHVtIDE1OjU2IHNjaHJpZWIgamRvdzoKPiA+ID4+PiBBcyBmYXIg
YXMgSSBjYW4gZ3Vlc3MgZnJvbSB0aGUgY29kZSwgcGJfRW52aXJvbm1lbnRbM10gKG51bWJlcgo+
ID4gPj4+IG9mCj4gPiA+PiAKPiA+ID4+IGhlYWRzKQo+ID4gPj4gCj4gPiA+Pj4gYW5kIHBiX0Vu
dmlyb25tZW50WzVdIChudW1iZXIgb2Ygc2VjdG9ycyBwZXIgY3lsaW5kZXIpIGFyZQo+ID4gPj4+
IGFiaXRyYXJpbHkKPiA+ID4+PiBjaG9zZW4gc28gdGhlIHBhcnRpdGlvbiBzaXplIGNhbiBiZSBl
eHByZXNzZWQgYXMgYSBkaWZmZXJlbmNlCj4gPiA+Pj4gYmV0d2Vlbgo+ID4gPj4+IHBiX0Vudmly
b25tZW50WzldIGFuZCBwYl9FbnZpcm9ubWVudFsxMF0gKGxvdyBhbmQgaGlnaCBjeWxpbmRlcgo+
ID4gPj4+IGFkZHJlc3NlcyksIHdoaWNoIHBsYWNlcyByZXN0cmljdGlvbnMgb24gYm90aCBwYXJ0
aXRpb24gc2l6ZSBhbmQKPiA+ID4+PiBhbGlnbm1lbnQgdGhhdCBkZXBlbmQgb24gd2hlcmUgb24g
dGhlIGRpc2sgYSBwYXJ0aXRpb24gaXMKPiA+ID4+PiBwbGFjZWQ/Cj4gPiA+PiAKPiA+ID4+IElm
IHlvdSBkbyBub3QgdGVhY2ggdGhlIE9TIHRvIGlnbm9yZSBDeWxpbmRlciBCbG9ja3MgdHlwZSBl
bnRyaWVzCj4gPiA+PiBhbmQKPiA+ID4+IHVzZSBzb21lIG1hdGggb24gaGVhZHMgYW5kIGJsb2Nr
cyBwZXIgdHJhY2sgdGhlIGRpc2sgc2l6ZSBpcwo+ID4gPj4gcmVsYXRpdmVseSBzdHVjayBtb2R1
bG8gdXNpbmcgbGFyZ2UgYmxvY2tzLgo+ID4gPiAKPiA+ID4gQXMgbG9uZyBhcyBBbWlnYU9TIGFu
ZCBMaW51eCBhZ3JlZSBvbiBob3cgdG8gZXhwcmVzcyBzdGFydCBhbmQgZW5kCj4gPiA+IG9mZnNl
dCBmb3IgdGhlIHBhcnRpdGlvbnMsIHRoYXQncyBmaW5lLgo+ID4gPiAKPiA+ID4gQnV0IEkgcmVh
ZCB5b3VyIG90aGVyIG1haWwgdG8gbWVhbiB0aGF0IHdlJ3JlIHN0dWNrIHRvIDIgVEIgZGlza3MK
PiA+ID4gZm9yCj4gPiA+IG5vdy4gSSBkb24ndCBmb2xsb3cgdGhhdCAtIHdlIGNhbiBoYXZlIHBh
cnRpdGlvbnMgb2YgMiBUQiBlYWNoIGJ5Cj4gPiAKPiA+IG1heGluZwo+ID4gCj4gPiA+IG91dCBy
ZGJfQ3lsQmxvY2tzIGFzIGxvbmcgYXMgd2UgdXNlIDUxMiBieXRlcyBwZXIgYmxvY2sgKHNpbmNl
IHRoZQo+ID4gPiBwcm9kdWN0IG9mIGN5bGluZGVycyBhbmQgYmxvY2tzIHBlciBjeWxpbmRlciBp
cyBsaW1pdGVkIHRvIDMyCj4gPiA+IGJpdHMpIGFuZCB1c2luZyBvbmUgY3lsaW5kZXIgcGVyIHBh
cnRpdGlvbiAoMzIgYml0cyBhdmFpbGFibGUKPiA+ID4gdGhlcmUgYXMgd2VsbCk/Cj4gPiA+IAo+
ID4gPiBCdXQgdGhlIHJkYl9DeWxCbG9ja3MgbGltaXQgYWxzbyBtZWFucyB3ZSdyZSBzYWZlIHdp
dGggNjQgYml0Cj4gPiA+IHNlY3Rvcl90IGluIExpbnV4LiBCZXN0IGFkZCBhIGNoZWNrIGluIHRo
ZSBwYXJzZXIgdG8gd2FybiB1cyBpZgo+ID4gPiB0aGUgcHJvZHVjdCBvZiBoZWFkIGNvdW50IGFu
ZCBzZWN0b3JzIHBlciBjeWxpbmRlciBvdmVyZmxvd3MgMzIKPiA+ID4gYml0IHRob3VnaC4KPiA+
ID4gCj4gPiA+IENoZWVycywKPiA+ID4KPiA+ID4gICAgICBNaWNoYWVsCj4gPiAKPiA+IEhvdyBs
b25nIGRpZCBpdCB0YWxlIHMgdG8gZ2V0IHRvIDEwIFRCIGRpc2tzIGZyb20gMiBUQiBkaXNrcy4g
QW5kIGEKPiA+IG5ldyBTRCBDYXJkIHNwZWMgYWxsb3dzIGZvciAxMjggVEIgZGlza3MuIEJsb2Nr
IHNpemVzIGdldCBzb3J0IG9mCj4gPiByaWRpY3Vsb3VzIGFzIHlvdSBnZXQgcGFzdCBhYm91dCA4
ayBieXRlcyBvciBhYm91dCAzMiBUQiBvciBhYm91dAo+ID4gdHdvCj4gPiB5ZWFycyBmcm9tIG5v
dy4KPiAKPiBJIGdldCB0aGF0IC0gSSBqdXN0IGRvbid0IGdldCB3aHkgMzIgYml0cyBmb3IgY3ls
aW5kZXJzIHBsdXMgMzIgYml0cwo+IGZvciBibG9ja3MgcGVyIGN5bGluZGVyIGVxdWFscyAyIFRC
ICg0RyBvZiA1MTIgYnl0ZSBibG9ja3MpLiBCdXQgSQo+IGRvbid0IGtub3cgd2hhdCBvdGhlciBs
aW1pdHMgZXhpc3QgdGhhdCBtYXkgcmVzdHJpY3QgdGhlIHRvdGFsIG51bWJlcgo+IG9mIGJsb2Nr
cyB0byAzMiBiaXRzLgoKSSB0aGluayBmb3IgdGhlIHRvdGFsIGRldmljZSBzaXplOgoKPiBUaGUg
cmF3LCB0aGVvcmV0aWNhbCBsaW1pdCBvbiB0aGUgbWF4aW11bSBkZXZpY2UgY2FwYWNpdHkgaXMg
YWJvdXQKPiAyXjEwNSBieXRlczoKPiAKPiAzMiBiaXQgcmRiX0N5bGluZGVycyAqIDMyIGJpdCBy
ZGJfSGVhZHMgKiAzMiBiaXQgcmRiX1NlY3RvcnMgKiA1MTIKPiBieXRlcy9zZWN0b3IgZm9yIHRo
ZSBIRCBzaXplIGluIHN0cnVjdCBSaWdpZERpc2tCbG9jawAKCmlzIGNvcnJlY3QuIERvIHlvdSBh
Z3JlZSwgSm9hbm5lPwoKaHR0cDovL3dpa2kuYW1pZ2Fvcy5uZXQvd2lraS9SREIKCklmIHNvLCB3
ZSBjYW4gcmVtb3ZlIHRoYXQgbGltaXQgZnJvbSB0aGUgZGlzY3Vzc2lvbiBhbmQgZm9jdXMgb24g
d2hhdCAKcmVtYWlucy4KCgpBcyBmb3IgdGhlIHBhcnRpdGlvbiBzaXplcywgaG93IGFib3V0Cgo+
IFBhcnRpdGlvbiBzaXplCj4gCj4gRm9yIHRoZSBwYXJ0aXRpb25zIHRoZSBtYXhpbXVtIHNpemUg
aXM6AAo+IDMyIGJpdCAoZGVfSGlnaEN5bCArIDEgLSBkZV9Mb3dDeWwpICh0byBnZXQgdGhlIHBh
cnRpdGlvbiBzaXplKSAqIDMyCj4gYml0IGRlX1N1cmZhY2VzICogMzIgYml0IGRlX1NlY3RvcnNQ
ZXJUcmFjayAqIDUxMiBieXRlcy9zZWN0b3IgaW4KPiBzdHJ1Y3QgRG9zRW52ZWMgKD1wYl9FbnZp
cm9ubWVudFtdKSBpbiBzdHJ1Y3QgUGFydGl0aW9uQmxvY2suAAo+Cj4gVGhhdCdzIGZyb20gdGhl
IHBoeXNpY2FsIGRyaXZlIHBhcnQsIHRoZSBhY3R1YWwgZGlzayBzaXplIGxpbWl0IGZvciA+IAo+
IHRoZSBwYXJ0aXRpb25zIG1heSBiZSBtdWNoIHNtYWxsZXIgZGVwZW5kaW5nIG9uIHRoZSBwYXJ0
aXRpb25pbmcKPiBzb2Z0d2FyZSwgaWYgaXQncyBvbmx5IHVzaW5nIHRoZSBsb2dpY2FsIHNpemVz
IGluc3RlYWQsIHdoaWNoIGlzCj4gbGlrZWx5IHRoZSBjYXNlLCBpdCdzIG9ubHkgOCBaaUIgd2l0
aCA1MTIgYnl0ZXMvc2VjdG9yOiAzMiBiaXQKPiByZGJfSGlDeWxpbmRlciAqIDMyIGJpdCByZGJf
Q3lsQmxvY2tzICogNTEyIGJ5dGVzL3NlY3RvciA9IDJeNzMKPiBieXRlcy4gRm9yIHVzaW5nIHRo
ZSBsb2dpY2FsIHNpemVzIHVzaW5nIHNpbXBsZSB1aW50NjQgY2FsY3VsYXRpb25zCj4gKHdpdGgg
c29tZSBvdmVyZmxvdyBjaGVja3MpIHNob3VsZCBiZSBlbm91Z2gsIGZvciBtb3JlIGEgbWF0aCBs
aWJyYXJ5Cj4gd2l0aCBzdXBwb3J0IGZvciBsYXJnZXIgaW50ZWdlcnMgbmVlZHMgdG8gYmUgdXNl
ZCB3aGljaCBwcm9iYWJseSBubwo+IHBhcnRpdGlvbmluZyBzb2Z0d2FyZSBkb2VzLgAKCnRha2Vu
IGZyb20gdGhlIHNhbWUgd2lraSBwYWdlPwoKSSBob3dldmVyIGRvIG5vdCBnZXQgd2hhdCBpdCBt
ZWFucyB3aXRoICJsb2dpY2FsIHNpemVzIi4KCkpvYW5uZSwgd2hhdCBpcyB5b3VyIGZlZWRiYWNr
IG9uIHRoaXM/Cgo+ID4gRG8geW91IHdhbnQgdG8gY3JlYXRlIGRpc2tzIHRoYXQgd2lsbCBmYWls
IG9uIEFtaWdhRE9TPyBBbWlnYURPUywgYXMKPiA+IGZhciBhcyBJIGtub3csIG1ha2VzIGhlYXZ5
IHVzZSBvZiBDeWxpbmRlciBCbG9ja3MgdmFsdWVzLiBJdAo+ID4gY2FsY3VsYXRpbmcgQ3lsaW5k
ZXIgQmxvY2tzIG92ZXJmbG93cyB3aGVuIGNyZWF0aW5nIHRoZSBkaXNrJ3MgUkRCcwo+ID4gdGhl
IHVzZXIgTVVTVCBiZSBpbmZvcm1lZCBpdCBpcwo+IAo+IEknbSBub3QgYXQgYWxsIHBsYW5uaW5n
IHRvIGNyZWF0ZSBkaXNrcyBmb3IgQW1pZ2FET1MuIEkganVzdCBuZWVkIHRvCj4ga25vdyB3aGF0
IGNvbWJpbmF0aW9ucyBvZiBjeWxpbmRlcnMsIGhlYWRzIGFuZCBzZWN0b3JzIGFyZSBwb3NzaWJs
ZSB0bwo+IGVuY291bnRlciBvbiBkaXNrcyB0aGF0IGhhdmUgYmVlbiBjcmVhdGVkIHdpdGggbmF0
aXZlIHRvb2xzLiBXZWxsLAoKRG8gd2UgcmVhbGx5IG5lZWQgdG8gYWRkIGNoZWNrcyB0byB0aG9z
ZSBwb3NzaWJsZSBjb21iaW5hdGlvbnMgdmFsdWVzLCAKSm9hbm5lPyBDYXVzZSwgaWYgd2UgZG8s
IGhvdyBkbyB3ZSBmaW5kIG91dD8KCkFnYWluLCB0aGUgTGludXggUkRCIHBhcnNlciBqdXN0ICpy
ZWFkcyogdGhlIFJEQi4KCkFuZCBhbHJlYWR5IG9uIHRoZSBBbWlnYSBzaWRlIGl0IGNhbiBoYXBw
ZW4gdGhhdCBvbmUgcGFydGl0aW9uaW5nIHRvb2wgCmRvZXMgbm90IHVuZGVyc3RhbmQgdGhlIG90
aGVyLiBUaGF0IFJpZ2lkIERpc2sgQmxvY2sgaXMgbm90IHF1aXRlIGFzIApyaWdpZCBhcyB0aGUg
bmFtZSB3b3VsZCBzdWdnZXN0LCB5b3UgYWxyZWFkeSBzYWlkIGl0cyB2ZXJ5IGZsZXhpYmxlIGFu
ZCAKdGhhdCBpcyBub3QgYWx3YXlzIGEgZ29vZCB0aGluZy4gSSBoYXZlIHNlZW4gYSBQaGFzZSA1
IFJEQiBwYXJ0aXRpb25pbmcgCnRvb2wgKmNydXNoKiBhbiBSREIgY3JlYXRlZCBieSBIRFRvb2xC
b3ggY2F1c2Ugb25lIGNhbGN1bGF0ZWQgZnJvbSAwIGFuZCAKb25lIGZyb20gemVyby4KCkkgZG8g
dGhpbmsgaXMgbmVpdGhlciB0aGUgZHV0eSBub3IgdGhlIHJlc3BvbnNpYmlsaXR5IG9mIHRoZSBM
aW51eCAKa2VybmVsIHRvIGNoZWNrIGZvciBzdWNoIGNyYXAuCgpFc3BlY2lhbGx5IGFzLCBhcyBJ
IHdpbGwgcG9pbnQgb3V0IGZ1cnRoZXIgZG93biBpbiB0aGUgbWFpbCwgaXQgaXMgCmRpZmZpY3Vs
dCB0byBpbXBvc3NpYmxlIHRvIGFjdHVhbGx5IGtub3cgZm9yIHN1cmUgd2hpY2ggY29tYmluYXRp
b25zIGFueSAKb2YgdGhlIHBhcnRpdGlvbmluZyB0b29scyBvbiBuYXRpdmUgb3PCtHMgYWxsb3dl
ZC4KCkV2ZW4gaWYgd2XCtGQgZmluZCBvdXQgdGhlIHBvc3NpYmxlIGNvbWJpbmF0aW9ucyBvZiB0
aGUgb2ZmaWNpYWwgdG9vbHMsIAp0aGVyZSBhcmUgUkRCIHRvb2xzIGxpa2UgUkRCU2FsdiBhbmQg
d2hhdCBub3QuCgpJwrRkIHNheSBpdCBpcyBpbXBvcnRhbnQgdG8gYXZvaWQgb3ZlciBjb21wbGlj
YXRpbmcgdGhpbmdzLiBXZSBjYW5ub3QgCmZ1bGZpbGwgYW5kIGl0IGlzIG5vdCBvdXIgcmVzcG9u
c2liaWxpdHkgdG8gc2F2ZSB0aGUgdXNlciBmcm9tIGFueSBicmFpbiAKZGFtYWdlIGFueSBwYXJ0
aXRpb25pbmcgdG9vbCBvbiBlaXRoZXIgbmF0aXZlIE9TIG9yIExpbnV4IGNyZWF0ZWQgCip3aXRo
aW4qIHRoZSBrZXJuZWwuCgpTbyBJwrRkIHJlY29tbWVuZDogCgotIE1ha2UgdGhlIGNhbGN1bGF0
aW9ucyBiYWlsIG91dCBvbiBvdmVyZmxvd3MgYW5kIHJlZnVzZSB0byBsb2FkIHRoZSBSREIgCmlm
IGFueSBjYWxjdWxhdGlvbiBvdmVyZmxvd3MuCgotIElkZWFsbHkgYWxzbyBzZWN1cmUgQUZGUyB0
aGlzIHdheS4KCi0gSGF2ZSB0aGUgd2FybmluZyBhYm91dCA2NCBiaXQgZGlzayBzdXBwb3J0IG9u
IG5hdGl2ZSBPUwoKYW5kICpiZSBkb25lIHdpdGggaXQqLiBGb3IgdGhlIGtlcm5lbC4KCj4gYXNz
dW1pbmcgc3VmZmljaWVudCBhbW91bnRzIG9mIGJyYWluZGFtYWdlIGluIHRoZSBjb3JyZXNwb25k
aW5nIExpbnV4Cj4gdG9vbHMsIGtub3dpbmcgdGhlIGFic29sdXRlIG91dGVyIGxpbWl0cyBvZiB3
aGF0IHRoZXNlIHRvb2xzIGNvdWxkIGRvCj4gd291bGQgYmUgbmljZSBhcyB3ZWxsLCBidXQgc29t
ZW9uZSB1c2luZyBhbWlnYS1mZGlzayB0byBjcmVhdGUgYSBSRFNLCj4gYmxvY2sgZm9yIGEgMTAg
VEIgZGlzayBmdWxseSBkZXNlcnZlcyBhbnkgcHVuaXNobWVudCB0aGF0IGludml0ZXMuCgpJZiB0
aGVyZSBpcyBhbnkgd2FybmluZyBkdWUgdG8gdXNpbmcgbGltaXRzIHRoYXQgZXhjZWVkIHdoYXQg
SERUb29sQm94IAooQW1pZ2FPUyB1cHRvIDMuMSksIEhEVG9vbGJveCB1c2luZyBoZHdyZW5jaC5s
aWJyYXJ5IChBbWlnYU9TIDMuNSAKb253YXJkcyksIE1lZGlhIFRvb2xib3ggdXNpbmcgU1BfZW5n
aW5lIChBbWlnYU9TIDQgb253YXJkcykgYXMgd2VsbCBhcyAKYWxsIHRoZSB0b29scyBpbiBNb3Jw
aG9zLCBBUk9TIGFuZCBvbiBBbWlOZXQgd291bCB1c2UsIGl0IHdvdWxkIGJlIHVwIHRvIAphbWln
YS1mZGlzayBhbmQgcGFydGVkIHRvIGlzc3VlIHN1Y2ggYSB3YXJuaW5nLgoKQXMgZm9yIHRoZSBS
REIgcGFyc2VyLCBJwrRkIGdvIHdpdGggcm9idXN0bmVzcyBwcmluY2lwbGU6CgpCZSBjb25zZXJ2
YXRpdmUgaW4gd2hhdCB5b3Ugc2VuZCwgYmUgbGliZXJhbCBpbiB3aGF0IHlvdSBhY2NlcHQuCgpo
dHRwczovL2VuLndpa2lwZWRpYS5vcmcvd2lraS9Sb2J1c3RuZXNzX3ByaW5jaXBsZQoKU28gd2hh
dCBldmVyIGNyYXAgb3Igbm9uIGNyYXAgYW55IHBhcnRpdGlvbmluZyB0b29sIGNyZWF0ZXMsIGNo
ZWNrIGZvciAKYW55IG92ZXJmbG93IGluIGNhbGN1bGF0aW9ucyBhbmQgb25seSByZWZ1c2UgdG8g
YWNjZXB0IHRoZSByZXN1bHQgaW4gCmNhc2UgdGhlcmUgaGF2ZSBiZWVuIGFuIG92ZXJmbG93IGlu
IGNhbGN1bGF0aW9ucy4KCklmIGEgdmFsdWUgaXMgYW4gVUxPTkcsIGFjY2VwdCB0aGUgZnVsbCAz
MiBiaXQgdW5zaWduZWQgaW50ZWdlci4gQ2F1c2UgCmlmIHlvdSBkb27CtHQsIGl0IGNvdWxkIG1l
YW4gdGhlIHVzZXIgaXMgbm90IGFibGUgdG8gYWNjZXNzIHRoZSBkaXNrIHdpdGggCnRoYXQgUkRC
LgoKUGFydGl0aW9uaW5nIHRvb2xzIGhvd2V2ZXIgY2FuIGltcG9zZSBhbnkgdGhleSB0aGluayBt
YWtlIHNlbnNlLgoKQXMgZm9yIGxpbWl0cyB3aXRoIGluIG5hdGl2ZSBPUyBwYXJ0aXRpb25pbmcg
dG9vbHMsIGZvciBIRFRvb2xib3ggd2l0aCAKYW5kIHdpdGhvdXQgaGR3cmVuY2gubGlicmFyeSB5
b3UgcHJvYmFibHkgcmVtZW1iZXIsIEpvYW5uZS4gQXMgZm9yIE1lZGlhIApUb29sYm94IHdlwrRk
IGhhdmUgdG8gYXNrIEFtaWdhT1MgZGV2ZWxvcGVycy4gUmVtZW1iZXIgQW1pZ2FPUyBhbmQgCk1v
cnBob3MgZGV2ZWxvcG1lbnQgYXJlIHN0aWxsIGNsb3NlZCBzb3VyY2UuIFNvIHRoZXJlIGlzIG5v
dGhpbmcgZXhjZXB0IAp0aGUgU0RLcyBhbmQgdGhlaXIgRG9jdW1lbnRhdGlvbiB3aGljaCB3ZSBj
YW4gdXNlIHRvIGtub3cgZm9yIHN1cmUuCgpUaGUgbGF0ZXN0IEFtaWdhT1MgNC4xIGRldmVsb3Bt
ZW50IGtpdCBhcHBlYXJzIHRvIGJlIHZlcnNpb24gNTMuMzAsIApmb3VuZCBhdC4gSXQgaXMgZm9y
IEFtaWdhT1MgNC4xIEZpbmFsIEVkaXRpb246CgpodHRwczovL3d3dy5oeXBlcmlvbi1lbnRlcnRh
aW5tZW50LmJpei9pbmRleC5waHAvZG93bmxvYWRzPwp2aWV3PWZpbGVzJnBhcmVudD0zMAoKKHlv
dSBuZWVkIHRvIGNsaWNrIHRoZSBsaW5rIHRvIGFycml2ZSBhbiBhbiBleHRyYSBwYWdlIGFuZCBj
bGljayB0aGUgCmxpbmsgdGhlcmUpCgpiYXNlLmxoYSBpbiBTREtfSW5zdGFsbCBjb250YWlucyB0
aGUgZG9jdW1lbnRhdGlvbi4gSSB3YXMgYWJsZSB0byB1bnBhY2sgCml0IHdpdGggbGhhIHggb24g
TGludXguIEhvd2V2ZXIgSSBoYXZlIG5vIGlkZWEgd2hlcmUgdGhhdCBsaGEgY29tbWFuZCBvbiAK
bXkgRGViaWFuIEdOVS9TaWQgc2V0dXAgY2FtZSBmcm9tOgoKJSBMQU5HPUMgZHBrZyAtUyAvdXNy
L2Jpbi9saGEKZHBrZy1xdWVyeTogbm8gcGF0aCBmb3VuZCBtYXRjaGluZyBwYXR0ZXJuIC91c3Iv
YmluL2xoYQoKSSBiZXQgdGhlcmUgaXMgc29tZSBMaW51eCBsaGEgZG93bmxvYWRhYmxlIHNvbWV3
aGVyZSwgaWYgbmVlZCBiZSBJIGNhbiAKZGlnIGZvciBpdC4KCkhvd2V2ZXIgSSB1c2VkIGdyZXAg
YW5kIGZpbmQgb24gdmFyaW91cyB2YXJpYXRpb25zIG9mIE1lZGlhIFRvb2xib3gsIApTUF9Fbmdp
bmUgYW5kIFJEQiBhcyB3ZWxsIGFzIFJpZ2lkIERpc2sgQmxvY2sgYW5kIGZvdW5kIG5vdGhpbmcg
b2YgCmludGVyZXN0IGZvciBvdXIgdG9waWMuCgpTbyBpdCBhcHBlYXJzIHRoYXQgdGhlIHdvcmtp
bmdzIG9mIFNQX0VuZ2luZSBhcmUgYnVyaWVkIGluIHNvdXJjZSBjb2RlIAp3ZSBkb27CtHQgaGF2
ZSBhY2Nlc3MgdG8uIEFuZCB3ZWxsIEkgd29uwrR0IGJhc2UgYW55IGxpbWl0IGNoZWNraW5nIG9u
IAphc3N1bXB0aW9ucy4gU28gSSB3b27CtHQgc2V0IGFueSBhZGRpdGlvbmFsIGxpbWl0cyB0aGFu
IHRoZSBjaGVjayBmb3IgCm92ZXJmbG93aW5nIGNhbGN1bGF0aW9ucy4KClRoZSBBbWlnYU9TIDMu
OSBOREsgc3RpbGwgYXBwZWFycyB0byBiZSBhdmFpbGFibGUgb24gdGhlIHdlYnNpdGUgb2YgbXkg
CmZvcm1lciBlbXBsb3llZSBIQUFHRSZQQVJUTkVSIEdtYkg6CgpodHRwczovL3d3dy5oYWFnZS1w
YXJ0bmVyLmRlL2Rvd25sb2FkL0FtaWdhT1MvTkRLMzkubGhhCgpUaGlzIG9uZSBpbmNsdWRlZCB0
aGUgaGR3cmVuY2gubGlicmFyeSBhdXRvZG9jcyBhbmQgcmVsZWFzZW5vdGVzLiBJIGRvIApub3Qg
a25vdyB3aGV0aGVyIHRoZXkgaGF2ZSBzb21ldGhpbmcgYWJvdXQgbGltaXRzLiBIb3dldmVyLCBp
dCBkb2VzIG5vdCAKYXBwZWFyIHNvOgoKPiBb4oCmXSBOREtfMy45L0RvY3VtZW50YXRpb24lIGdy
ZXAgbGltaXQgQXV0b2RvY3MvaGR3cmVuY2guZG9jCj4gCj4gICAgIFRoaXMgZnVuY3Rpb24gaXMg
aW50ZW50aW9uYWxseSBsaW1pdGVkIHdyaXRpbmcgb25seSB0byB0aGUgbG93ZXIKPiAgICAgYmxv
Y2sKClRoaXMgaXMgYSBsaW1pdCBhYm91dCB3cml0aW5nIHRoZSBkYXRhIHNlY3Rpb24gb2YgYSBC
b290QmxvY2ssIHNvIAp1bnJlbGF0ZWQgdG8gb3VyIHRvcGljLgoKPiBb4oCmXSBOREtfMy45L0Rv
Y3VtZW50YXRpb24lIGdyZXAgbGltaXQKPiBSZWxlYXNlbm90ZXMvaGR3cmVuY2hfbGliX3JlbG5v
dGVzAAoKTWljaGFlbDoKPiAoQWN0dWFsbHksIEkgbGllZCB0aGVyZS4gSSBkbyBwbGFuIHRvIGNy
ZWF0ZSBhIFJEU0sgYmxvY2sgZm9yIGEgMiBUQgo+IGRpc2sgaW1hZ2Ugd2hlcmUgY3lsaW5kZXIs
IGhlYWQgYW5kIHNlY3RvciBjb3VudHMgYWxsIGFwcHJvYWNoIHRoZSAzMgo+IGJpdCBsaW1pdCwg
anVzdCB0byBzZWUgdGhhdCBteSBvdmVyZmxvdyBjaGVja3Mgd29yayBhcyBpbnRlbmRlZC4gQnV0
Cj4gdGhhdCdzIHN0cmljdGx5IGZvciBMaW51eCB0ZXN0aW5nKS4KCkkgdGhpbmsgdGhhdCBpcyBh
IGdvb2QgdGVzdCBmb3IgdGhlIHBhdGNoLiBJIGFtIGltcHJlc3NlZCB5b3UgYXJlIAp3aWxsaW5n
IHRvIHB1dCB0aGUgZWZmb3J0IGZvciB0aGF0IGludG8gdGhpcyB3b3JrLgoKT2theSwgdGhhdCBp
cyB3aGF0IEkgd2FzIGFibGUgdG8gZGlnIG91dCBvbiB3aGF0IGlzIG9mZmljaWFsbHkgCmF2YWls
YWJsZS4gSXQgYXBwZWFycyB0aGF0IHRoZSBvZmZpY2lhbCBkZXZlbG9wZXIgZG9jdW1lbnRhdGlv
biBmb3IgCkFtaWdhT1MgaXMgcHJldHR5IHNwYXJzZSBvbiBhbnkgb2YgdGhhdC4gRm9yIGFueXRo
aW5nIGVsc2UgScK0ZCBuZWVkIHRvIApjb250YWN0IGN1cnJlbnQgQW1pZ2FPUyBkZXZlbG9wZXJz
LiBJIGFtIG5vIGxvbmdlciBhIG1lbWJlciBvZiB0aGUgCkFtaWdhT1MgdGVhbS4KCk1heWJlIHRo
ZSBBREYgRm9ybWF0IEZBUSBJIG1lbnRpb25lZCBoYXMgYSBiaXQgbW9yZSwgYnV0IEkgZGlkIG5v
dCBmaW5kIAphbnl0aGluZyBvYnZpb3VzIG9uIGEgc2hvcnQgZ2xhbmNlCgpodHRwOi8vbGNsZXZ5
LmZyZWUuZnIvYWRmbGliL2FkZl9pbmZvLmh0bWwKCkFsc28gaXQgaXMgYSB0aGlyZCBwYXJ0eSBz
b3VyY2UuCgo+ID4gdW5zYWZlIHRvIHB1dCBvbiBhIHJlYWwgQW1pZ2EuIChJJ2QgYWxzbyBzdWdn
ZXN0IHRlYWNoaW5nIExpbnV4IHRvCj4gPiB1bmRlcnN0YW5kIFJEU0wsIHdoaWNoIHdvdWxkIGJl
IFJEU0srKyBzb3J0IG9mLiBUaGVuIHVzZSB0aGF0IGlmCj4gPiBDeWxpbmRlciBCbG9ja3Mgb3Zl
cmZsb3dzLikgVGhlIHZhbHVlIHlvdSB3aWxsIG5vdCBiZSBhYmxlIHRvIGZpbGwKPiA+IGluCj4g
PiB0aGUgRG9zRW52ZWMgc3RydWN0dXJlIGlzOgo+ID4gICAgIFVMT05HIGRlX0hpZ2hDeWw7ICAg
ICAgICAgLyogbWF4IGN5bGluZGVyLiBkcml2ZSBzcGVjaWZpYyAqLwo+IAo+IE9LLCBzbyBDeWxp
bmRlciBCbG9ja3Mgb3ZlcmZsb3dpbmcgaXMgYSByZWQgZmxhZywgYW5kIHJlcXVpcmVzIHRvCj4g
YWJvcnQgcGFyc2luZyB0aGUgcGFydGl0aW9uIHRhYmxlIHJpZ2h0IGF3YXk/IEFuZCBIaWdoQ3ls
IHJlYWxseQo+IG1lYW5zIHRoZSBtYXguIG51bWJlciBvZiBsb2dpY2FsIGJsb2Nrcywgbm90IGN5
bGluZGVycyAod2hpY2ggd291bGQKPiBoYXZlIG5yX2hlYWRzKm5yX3NlY3RzIG1hbnkgYmxvY2tz
KT8gVGhhdCdzIHByb2JhYmx5IHRoZSBjYXVzZSBmb3IgbXkKPiBjb25mdXNpb24uCgpXaGVuIGNh
biB0aGV5IGFjdHVhbGx5IG92ZXJmbG93PyBIb3cgYXJlIHRoZXkgY2FsY3VsYXRlZD8KCklmIHRo
ZXJlIGlzIG5vIGNhbGN1bGF0aW9uIHRvIGNhbGN1bGF0ZSBpdCBhbmQgaXQgaXMganVzdCBhIHN0
YXRpYyB2YWx1ZSAKd2l0aGluIHRoZSBSREIsIEnCtGQgYWNjZXB0IHdoYXQgZXZlciB0aGUgdmFs
dWUgaXMgaW4gdGhlIFJEQiwgdW5sZXNzIGEgCmNhbGN1bGF0aW9uIGJhc2VkIG9uIGl0IG92ZXJm
bG93cy4KCkFnYWluLCBrZWVwIGl0IHNpbXBsZS4KCkkgdGhpbmsgaXQgZG9lcyBub3QgbWFrZSBz
ZW5zZSBmb3IgdGhlIExpbnV4IGtlcm5lbCB0byB0cnkgdG8gb3V0c21hcnQgCnRoZSB1c2VyIG9y
IGFueSBvZiB0aGUgcGFydGl0aW9uaW5nIHRvb2xzIG91dCB0aGVyZS4KClRoZSBvbmx5IHF1ZXN0
aW9uIGlzOgoKQ2FuIHRoZSBwYXJ0aXRpb25zIGJlIGNhbGN1bGF0ZWQgaW4gYSB3YXkgdGhhdCBp
dCBpcyBzYWZlIHRvIGFjY2VzcyAKdGhlbSwgaS5lLiB0aGF0IHRoZWlyIGNhbGN1bGF0ZWQgc3Rh
cnQgYW5kIGVuZCBpcyByZWFsbHkgYXQgdGhlIGxvY2F0aW9uIAp0aGUgc3RhcnQgYW5kIGVuZCBp
cyBvbiB0aGUgZGlzaz8gVGhlbiBhY2NlcHQgaXQuCgpGb3IgdGhlIGZpbGUgc3lzdGVtczogQ2Fu
IHRoZSBmaWxlIHN5c3RlbSBzYWZlbHkgYWNjZXNzIHRoaXM/IFRoZW4gCmFjY2VwdC4gSWYgbm90
LCB0aGVuIGRlY2xpbmUuICpJbiB0aGUgZmlsZXN5c3RlbSouIFRoaXMgaXMgbm90aGluZyBmb3Ig
CnRoZSBSREIgcGFyc2VyIHRvIGNoZWNrIHNvIG91dCBvZiBzY29wZSBmb3IgdGhlIGRpc2N1c3Np
b24gb2YgdGhpcyAKcGF0Y2guCgo+ID4gU28gYWNjZXNzaW5nIGxhcmdlciBkaXNrcyBvbmNlIHlv
dSBoaXQgMiBUQiBtZWFucyB5b3UgbXVzdCBpbmNyZWFzZQo+ID4gdGhlIGxvZ2ljYWwgYmxvY2sg
c2l6ZS4gQW5kIGV2ZW50dWFsbHkgdGhhdCB3aWxsIHdhc3RlIEhVR0UgYW1vdW50cwo+ID4gb2Yg
ZmlsZXMgd2hlbiBzbWFsbCBmaWxlcyBhcmUgYmVpbmcgc3RvcmVkLgo+IAo+IEp1c3QgbGlrZSBz
bWFsbCBpbm9kZXMgd2FzdGVzIGh1Z2UgYW1vdW50cyBvZiBzcGFjZSBmb3IgbWV0YWRhdGEuIEl0
J3MKPiBhIHRyYWRlb2ZmLCBhbmQgQUZGUyBvbiBhIFJEU0sgZm9ybWF0IGRpc2sgcHJvYmFibHkg
aXNuJ3QgdGhlIHJpZ2h0Cj4gY2hvaWNlIGZvciBodWdlIGRpc2tzLiBOZXZlciBtaW5kIHRoYXQg
LSBpZiBzb21lb25lIF9kb2VzXyBnbyB0aGF0Cj4gd2F5LCB3ZSBuZWVkIHRvIG1ha2Ugc3VyZSB3
ZSBjYW4gcGFyc2UgdGhlIFJEU0sgaW5mb3JtYXRpb24KPiBjb3JyZWN0bHkuIEFuZCBpZiBzdWNo
IGEgZGlzayBjYXVzZXMgdGhlIDY0IGJpdCBzZWN0b3JfdCBpbiBMaW51eCB0bwo+IG92ZXJmbG93
LCBJJ2QgbGlrZSB0aGUgcGFyc2VyIHRvIHNwb3QgdGhhdCwgdG9vLgo+IAo+IFRoYW5rcyBmb3Ig
eW91ciBpbW1lbnNlIHBhdGllbmNlIGluIGV4cGxhaW5pbmcgYWxsIHRoZXNlIHN1YnRsZXRpZXMg
dG8KPiBtZS4KPgo+IENoZWVycywKPiAKPiAgICAgTWljaGFlbAo+IAo+ID4gQW55IHNvbHV0aW9u
IHdpbGwgcmVxdWlyZSBhY3Rpb24gb24gdGhlIHBhcnQgb2YgdGhlIHBlb3BsZQo+ID4gZGV2ZWxv
cGluZwo+ID4gQW1pZ2FET1MgZm9sbG93LW9ucy4gWW91IG1pZ2h0IHdhbnQgdG8gZ2V0IHRoZW0g
bW90aXZhdGVkLCBzb21laG93LAo+ID4gYW5kIHByb2NlZWQgZnJvbSB0aGVyZSB3aXRoIGEgcmVx
dWVzdCB0byBiZSBpbmZvcm1lZCBvZiBhbnkgUkRCCj4gPiBjaGFuZ2VzLiBJJ2Qgc3VnZ2VzdCB0
byB0aGVtIHRoYXQgcmVtb3Zpbmcgc2Vuc2l0aXZpdHkgdG8gQ3lsaW5kZXIKPiA+IEJsb2NrcyBz
b3J0cyBvZiB2YWx1ZXMgZnJvbSB0aGUgZW50aXJlIHN5c3RlbSBwcm9iYWJseSB3b3VsZCBiZQo+
ID4gcGFpbmZ1bCBidXQgdGhlIHNpbXBsZXN0IHNvbHV0aW9uLgo+ID4gCj4gPiB7Xl9efQpb4oCm
XQoKLS0gCk1hcnRpbg==

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[Martin Steigerwald]

jdow - 30.06.18, 08:47:
> Let's get everybody:
>=20
> On 20180629 22:26, Michael Schmitz wrote:
>  > Joanne,
>  >=20
>  > Am 30.06.18 um 15:56 schrieb jdow:
>  >>>>> As far as I can guess from the code, pb_Environment[3] (number
>  >>>>> of
>  >>>>=20
>  >>>> heads)
>  >>>>=20
>  >>>>> and pb_Environment[5] (number of sectors per cylinder) are
>  >>>>> abitrarily
>  >>>>> chosen so the partition size can be expressed as a difference
>  >>>>> between
>  >>>>> pb_Environment[9] and pb_Environment[10] (low and high cylinder
>  >>>>> addresses), which places restrictions on both partition size
>  >>>>> and
>  >>>>> alignment that depend on where on the disk a partition is
>  >>>>> placed?
>  >>>>=20
>  >>>> If you do not teach the OS to ignore Cylinder Blocks type
>  >>>> entries and
>  >>>> use some math on heads and blocks per track the disk size is
>  >>>> relatively stuck modulo using large blocks.
>  >>>=20
>  >>> As long as AmigaOS and Linux agree on how to express start and
>  >>> end
>  >>> offset for the partitions, that's fine.
>  >>>=20
>  >>> But I read your other mail to mean that we're stuck to 2 TB disks
>  >>> for
>  >>> now. I don't follow that - we can have partitions of 2 TB each by
>  >>=20
>  >> maxing
>  >>=20
>  >>> out rdb_CylBlocks as long as we use 512 bytes per block (since
>  >>> the
>  >>> product of cylinders and blocks per cylinder is limited to 32
>  >>> bits) and using one cylinder per partition (32 bits available
>  >>> there as well)?
>  >>>=20
>  >>> But the rdb_CylBlocks limit also means we're safe with 64 bit
>  >>> sector_t in Linux. Best add a check in the parser to warn us if
>  >>> the product of head count and sectors per cylinder overflows 32
>  >>> bit though.
>  >>>=20
>  >>> Cheers,
>  >>>=20
>  >>>        Michael
>  >>=20
>  >> How long did it tale s to get to 10 TB disks from 2 TB disks. And
>  >> a
>  >> new SD Card spec allows for 128 TB disks. Block sizes get sort of
>  >> ridiculous as you get past about 8k bytes or about 32 TB or about
>  >> two
>  >> years from now.
>  >=20
>  > I get that - I just don't get why 32 bits for cylinders plus 32
>  > bits for blocks per cylinder equals 2 TB (4G of 512 byte blocks).
>  > But I don't know what other limits exist that may restrict the
>  > total number of blocks to 32 bits.
>=20
> It overflows uint32_t cylinder blocks aka blocks per cylinder. Linux
> doesn't care. AmigaDOS surely does. If YOU make partitions really
> large for yourself that's OK. If Joe Amigoid does it the potential
> for an angry red turning to purple face is high.

Ok, let get this straight:

Do you think that is the responsibility of the RDB parser within the=20
Linux kernel to protect the user from anything whatever partitioning=20
tool has created?

If so, how would you make sure the Linux kernel knows about whatever any=20
partitioning tool used by Amiga users can come up with?=20

I=C2=B4d say: Don=C2=B4t bother. It is not the job of the RDB parser to imp=
ose=20
limits on what partitioning tools can create. If native OS tools don=C2=B4t=
=20
create such thing, you don=C2=B4t need to check for it. If someone managed =
to=20
create it with amiga-fdisk or parted, the tool needs to be fixed. *Not*=20
the kernel.

Anyway, that 2 TB disk that started all this *worked* on AmigaOS 4. And=20
I am pretty sure while I cannot proof it, that even a larger disk would=20
work. There is a limit for the boot partition on AmigaOS 4 Classic,=20
which uses AmigaOS 3.1 to bootstrap AmigaOS 4 on Classic Amiga computers=20
like an Amiga 1200 or Amiga 4000 with PowerPC extension card. But=20
according to

Hard drive setup for AmigaOS 4.1 Classic
http://blog.hyperion-entertainment.biz/?p=3D210

AmigaOS classic (i.e. < 4) would crash.

>  >> Do you want to create disks that will fail on AmigaDOS? AmigaDOS,
>  >> as
>  >> far as I know, makes heavy use of Cylinder Blocks values. It
>  >> calculating Cylinder Blocks overflows when creating the disk's
>  >> RDBs
>  >> the user MUST be informed it is
>  >=20
>  > I'm not at all planning to create disks for AmigaDOS. I just need
>  > to
>  > know what combinations of cylinders, heads and sectors are possible
>  > to encounter on disks that have been created with native tools.
>  > Well, assuming sufficient amounts of braindamage in the
>  > corresponding Linux tools, knowing the absolute outer limits of
>  > what these tools could do would be nice as well, but someone using
>  > amiga-fdisk to create a RDSK block for a 10 TB disk fully deserves
>  > any punishment that invites.
> Native AmigaDOS tools SHOULD NOT be able to create something that
> overflows CylinderBlocks values. However, if it can that creates an

There you have it.

Then *why* bother, Joanne?

> interesting test case to see what various tools, like the AmigaDOS
> "info" command, do when they they are run on such a disk. I don't
> have OS source to perform searches. And I am not setup to feed the
> system something obscene.
>=20
>  > (Actually, I lied there. I do plan to create a RDSK block for a 2
>  > TB
>  > disk image where cylinder, head and sector counts all approach the
>  > 32
>  > bit limit, just to see that my overflow checks work as intended.
>  > But
>  > that's strictly for Linux testing).
>  >=20
>  >> unsafe to put on a real Amiga. (I'd also suggest teaching Linux to
>  >> understand RDSL, which would be RDSK++ sort of. Then use that if
>  >> Cylinder Blocks overflows.) The value you will not be able to fill
>  >> in
>  >>=20
>  >> the DosEnvec structure is:
>  >>      ULONG de_HighCyl;         /* max cylinder. drive specific */
>  >=20
>  > OK, so Cylinder Blocks overflowing is a red flag, and requires to
>  > abort parsing the partition table right away? And HighCyl really
>  > means the max. number of logical blocks, not cylinders (which
>  > would have nr_heads*nr_sects many blocks)? That's probably the
>  > cause for my confusion.
> I think I picked the wrong value. In RDSK itself this value is what
> overflows: ULONG   rdb_CylBlocks;    /* number of blocks available
> per cylinder */ And I think that floats around the system in many
> places with different names. As mentioned the "info" command is one
> item to test. If no crashes are found then AmigaDOS may be clean up
> to obscene sizes. At the moment I do not remember what
> hdwrench.library does with that value other than pass it along as
> read. Nor am I sure what it generates as any suggested values. I
> don't at this time have a disk I can mount as a disk on WinUAE that
> is more than 2TB. And my Amigas speak SCSI so I have no disk for
> them, either, even if they still boot.
>  >> So accessing larger disks once you hit 2 TB means you must
>  >> increase
>  >> the logical block size. And eventually that will waste HUGE
>  >> amounts of files when small files are being stored.
>  >=20
>  > Just like small inodes wastes huge amounts of space for metadata.
>  > It's a tradeoff, and AFFS on a RDSK format disk probably isn't the
>  > right choice for huge disks. Never mind that - if someone _does_
>  > go that way, we need to make sure we can parse the RDSK
>  > information correctly. And if such a disk causes the 64 bit
>  > sector_t in Linux to overflow, I'd like the parser to spot that,
>  > too.
>  >=20
>  > Thanks for your immense patience in explaining all these subtleties
>  > to me.
[=E2=80=A6]
>  >      Michael
>=20
> And I'm rushing too much so I'm sorry I am making errors. This stuff
> is 25 years in the past since I last looked at it seriously.

I think its important to focus on what can overflow can happen within=20
calculations the RDB parser (and as a second step the AFFS file system)=20
in the kernel kernel in order to keep this discussion to a manageable=20
size. Be conservative about overflows, but otherwise accept. With a=20
warning if a calculated exceed 32 bit.

As for values in the RDB. If its there, accept it. Some tool has written=20
it there. We don=C2=B4t know whether it did this right or wrong. We don=C2=
=B4t=20
know what the developer of the tool thought when writing it, well except=20
for hdwrench.library I=C2=B4d say as far as you remember. :) And it is not=
=20
our job within the kernel to check that.

There is a ton of more or less legacy software out there on native OS=20
which does something to or with RDBs. I=C2=B4d say it is impossible to say=
=20
what RDB a user may come up with.

Thanks,
=2D-=20
Martin

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[jdow]

For Linux:
1) Make a change to the Linux RDB parser so that the product of the "CHS" values 
goes to at least a 64 bit entity, It may need to go to a 128 bit entity when we 
are encoding data in DNA, crystal lattices, or something else super dense. The 
parser simply feeds data to the OS. No warnings are needed. No errors are 
expected but if they happen do not mount the disk and post an error message the 
user can use to diagnose the problem. "It's broken," is insufficient.
2) if there is an mkfs-(amiga file system of some sort) utility audit it for 
generating values as proper as possible. For the interim a warning of heads 
times sectors overflows uint32_t size storage.

For AmigaDOS:
1) Be aware that there is a potential for problems in the AmigaDOS utilities, 
filesystem, and device driver interfaces. This should be audited for overflow 
issues and to see of the rdb_CylinderBlocks value propagates through the system 
or an equivalent is created as a uint32_t size value. This is a potential 
problem. And fix them.
2) If 4.1 cannot handle block sizes other than 512 bytes it is broken and should 
be fixed. This goes for all tools and utilities as well. Incompatible utilities 
that are found should be flagged to users. This might include old versions of 
AmigaDOS partitioning tools as well as most anything else found to use the "CHS" 
and "block size" values. The size of this audit is unknown. If source code is in 
one place it can be searched for the use of these values as well as 
rdb_CylinderBlocks itself.

Regardless of how you fiddle the values the limit is beyond the size needed to 
have each atom in the visible universe individually addressed by several orders 
of magnitude. The ultimate limit is 2^128. Your value is too small by a few 
orders of magnitude. It's academic, though. This is all predicted on auditing 
the system's use of the "CHS" and block size values and repairing errors found.

Logical block sizes come from using say 16 sectors as one block, or in MS 
parlance "cluster". Call it an space allocation unit.

"Even if we´d find out the possible combinations of the official tools,
there are RDB tools like RDBSalv and what not." "All" means "All." If no 
maintainer exists then flag the tool as broken, and for God's sake don't 
distribute it with the OS.

Re hdwrench.library, I have all the source code. I've wasted enough time that I 
am HIGHLY unmotivated to dig into this morass any further.

On a side note, if HAAGE&PARTNER developed 3.whatever which included 
hdwrench.library as a work for hire for you then that source code can be 
located, including a fairly complete history if its development. Once located it 
can be posted to you if you cannot find it. It's copyrighted to H&P. I did it as 
a consultant for them. If you hired them to do the work then the copyright is 
misattributed and can be rectified.

Regarding hard block writing limitations - THAT must be near the front of super 
large disks. On an 18 gigabyte disk you can even get away with putting the RDBs 
at the end of the disk except for the RDSK block which must be in the first 16 
physical sectors. Everything else is uint32_t block number pointers. {^_-} Use 
your imagination, Luke! {^_-}

Any code that uses the "CHS" values to calculate a position or size of a disk is 
potentially broken. Hopefully there are only a very few. But I do not know and 
am a known paranoid.

At one time I had the impression that filesystems worked based on block numbers 
on the disk, first block and last block of the partition with no particular 
attention being paid to the "CHS" values. Of course the filesystem must think 
properly in terms of cluster size aka logical block size aka N * sector size. 
Erm, usually N is expected to be a power of 2. Non-powers of 2 MIGHT work in 
some cases. The filesystem gets an fssm message. That includes a DosEnvec. 
DosEnvec includes disk information for block size and "CHS" but at the moment I 
don't see a CylinderBlocks value hiding in it. I thought I had seen such at one 
time so in my hurry I may be going past it.

Somebody ****ELSE**** has some work to do now.

{^_^}

On 20180630 01:48, Martin Steigerwald wrote:
> Michael. Joanne.
> 
> I do think this discussion is slightly getting out of hand… so I suggest
> to focus on what its up to the kernel to do and what is not. And to
> focus only on what is up to the RDB parser, cause the patch is on
> changing that. The RDB parser is not responsible for what any file
> system may do. Securing AFFS would be a different, important, topic.
> 
> With that mail I am probably out as this discussion took already quite a
> bit of time.
> 
> But for details, read on:
> 
> Michael Schmitz - 30.06.18, 07:26:
>> Am 30.06.18 um 15:56 schrieb jdow:
>>>>>> As far as I can guess from the code, pb_Environment[3] (number
>>>>>> of
>>>>>
>>>>> heads)
>>>>>
>>>>>> and pb_Environment[5] (number of sectors per cylinder) are
>>>>>> abitrarily
>>>>>> chosen so the partition size can be expressed as a difference
>>>>>> between
>>>>>> pb_Environment[9] and pb_Environment[10] (low and high cylinder
>>>>>> addresses), which places restrictions on both partition size and
>>>>>> alignment that depend on where on the disk a partition is
>>>>>> placed?
>>>>>
>>>>> If you do not teach the OS to ignore Cylinder Blocks type entries
>>>>> and
>>>>> use some math on heads and blocks per track the disk size is
>>>>> relatively stuck modulo using large blocks.
>>>>
>>>> As long as AmigaOS and Linux agree on how to express start and end
>>>> offset for the partitions, that's fine.
>>>>
>>>> But I read your other mail to mean that we're stuck to 2 TB disks
>>>> for
>>>> now. I don't follow that - we can have partitions of 2 TB each by
>>>
>>> maxing
>>>
>>>> out rdb_CylBlocks as long as we use 512 bytes per block (since the
>>>> product of cylinders and blocks per cylinder is limited to 32
>>>> bits) and using one cylinder per partition (32 bits available
>>>> there as well)?
>>>>
>>>> But the rdb_CylBlocks limit also means we're safe with 64 bit
>>>> sector_t in Linux. Best add a check in the parser to warn us if
>>>> the product of head count and sectors per cylinder overflows 32
>>>> bit though.
>>>>
>>>> Cheers,
>>>>
>>>>       Michael
>>>
>>> How long did it tale s to get to 10 TB disks from 2 TB disks. And a
>>> new SD Card spec allows for 128 TB disks. Block sizes get sort of
>>> ridiculous as you get past about 8k bytes or about 32 TB or about
>>> two
>>> years from now.
>>
>> I get that - I just don't get why 32 bits for cylinders plus 32 bits
>> for blocks per cylinder equals 2 TB (4G of 512 byte blocks). But I
>> don't know what other limits exist that may restrict the total number
>> of blocks to 32 bits.
> 
> I think for the total device size:
> 
>> The raw, theoretical limit on the maximum device capacity is about
>> 2^105 bytes:
>>
>> 32 bit rdb_Cylinders * 32 bit rdb_Heads * 32 bit rdb_Sectors * 512
>> bytes/sector for the HD size in struct RigidDiskBlock
> 
> is correct. Do you agree, Joanne?
> 
> http://wiki.amigaos.net/wiki/RDB
> 
> If so, we can remove that limit from the discussion and focus on what
> remains.
> 
> 
> As for the partition sizes, how about
> 
>> Partition size
>>
>> For the partitions the maximum size is:32 bit (de_HighCyl + 1 - de_LowCyl) (to get the partition size) * 32
>> bit de_Surfaces * 32 bit de_SectorsPerTrack * 512 bytes/sector in
>> struct DosEnvec (=pb_Environment[]) in struct PartitionBlock.
>> That's from the physical drive part, the actual disk size limit for >
>> the partitions may be much smaller depending on the partitioning
>> software, if it's only using the logical sizes instead, which is
>> likely the case, it's only 8 ZiB with 512 bytes/sector: 32 bit
>> rdb_HiCylinder * 32 bit rdb_CylBlocks * 512 bytes/sector = 2^73
>> bytes. For using the logical sizes using simple uint64 calculations
>> (with some overflow checks) should be enough, for more a math library
>> with support for larger integers needs to be used which probably no
>> partitioning software does.
> 
> taken from the same wiki page?
> 
> I however do not get what it means with "logical sizes".
> 
> Joanne, what is your feedback on this?
> 
>>> Do you want to create disks that will fail on AmigaDOS? AmigaDOS, as
>>> far as I know, makes heavy use of Cylinder Blocks values. It
>>> calculating Cylinder Blocks overflows when creating the disk's RDBs
>>> the user MUST be informed it is
>>
>> I'm not at all planning to create disks for AmigaDOS. I just need to
>> know what combinations of cylinders, heads and sectors are possible to
>> encounter on disks that have been created with native tools. Well,
> 
> Do we really need to add checks to those possible combinations values,
> Joanne? Cause, if we do, how do we find out?
> 
> Again, the Linux RDB parser just *reads* the RDB.
> 
> And already on the Amiga side it can happen that one partitioning tool
> does not understand the other. That Rigid Disk Block is not quite as
> rigid as the name would suggest, you already said its very flexible and
> that is not always a good thing. I have seen a Phase 5 RDB partitioning
> tool *crush* an RDB created by HDToolBox cause one calculated from 0 and
> one from zero.
> 
> I do think is neither the duty nor the responsibility of the Linux
> kernel to check for such crap.
> 
> Especially as, as I will point out further down in the mail, it is
> difficult to impossible to actually know for sure which combinations any
> of the partitioning tools on native os´s allowed.
> 
> Even if we´d find out the possible combinations of the official tools,
> there are RDB tools like RDBSalv and what not.
> 
> I´d say it is important to avoid over complicating things. We cannot
> fulfill and it is not our responsibility to save the user from any brain
> damage any partitioning tool on either native OS or Linux created
> *within* the kernel.
> 
> So I´d recommend:
> 
> - Make the calculations bail out on overflows and refuse to load the RDB
> if any calculation overflows.
> 
> - Ideally also secure AFFS this way.
> 
> - Have the warning about 64 bit disk support on native OS
> 
> and *be done with it*. For the kernel.
> 
>> assuming sufficient amounts of braindamage in the corresponding Linux
>> tools, knowing the absolute outer limits of what these tools could do
>> would be nice as well, but someone using amiga-fdisk to create a RDSK
>> block for a 10 TB disk fully deserves any punishment that invites.
> 
> If there is any warning due to using limits that exceed what HDToolBox
> (AmigaOS upto 3.1), HDToolbox using hdwrench.library (AmigaOS 3.5
> onwards), Media Toolbox using SP_engine (AmigaOS 4 onwards) as well as
> all the tools in Morphos, AROS and on AmiNet woul use, it would be up to
> amiga-fdisk and parted to issue such a warning.
> 
> As for the RDB parser, I´d go with robustness principle:
> 
> Be conservative in what you send, be liberal in what you accept.
> 
> https://en.wikipedia.org/wiki/Robustness_principle
> 
> So what ever crap or non crap any partitioning tool creates, check for
> any overflow in calculations and only refuse to accept the result in
> case there have been an overflow in calculations.
> 
> If a value is an ULONG, accept the full 32 bit unsigned integer. Cause
> if you don´t, it could mean the user is not able to access the disk with
> that RDB.
> 
> Partitioning tools however can impose any they think make sense.
> 
> As for limits with in native OS partitioning tools, for HDToolbox with
> and without hdwrench.library you probably remember, Joanne. As for Media
> Toolbox we´d have to ask AmigaOS developers. Remember AmigaOS and
> Morphos development are still closed source. So there is nothing except
> the SDKs and their Documentation which we can use to know for sure.
> 
> The latest AmigaOS 4.1 development kit appears to be version 53.30,
> found at. It is for AmigaOS 4.1 Final Edition:
> 
> https://www.hyperion-entertainment.biz/index.php/downloads?
> view=files&parent=30
> 
> (you need to click the link to arrive an an extra page and click the
> link there)
> 
> base.lha in SDK_Install contains the documentation. I was able to unpack
> it with lha x on Linux. However I have no idea where that lha command on
> my Debian GNU/Sid setup came from:
> 
> % LANG=C dpkg -S /usr/bin/lha
> dpkg-query: no path found matching pattern /usr/bin/lha
> 
> I bet there is some Linux lha downloadable somewhere, if need be I can
> dig for it.
> 
> However I used grep and find on various variations of Media Toolbox,
> SP_Engine and RDB as well as Rigid Disk Block and found nothing of
> interest for our topic.
> 
> So it appears that the workings of SP_Engine are buried in source code
> we don´t have access to. And well I won´t base any limit checking on
> assumptions. So I won´t set any additional limits than the check for
> overflowing calculations.
> 
> The AmigaOS 3.9 NDK still appears to be available on the website of my
> former employee HAAGE&PARTNER GmbH:
> 
> https://www.haage-partner.de/download/AmigaOS/NDK39.lha
> 
> This one included the hdwrench.library autodocs and releasenotes. I do
> not know whether they have something about limits. However, it does not
> appear so:
> 
>> […] NDK_3.9/Documentation% grep limit Autodocs/hdwrench.doc
>>
>>      This function is intentionally limited writing only to the lower
>>      block
> 
> This is a limit about writing the data section of a BootBlock, so
> unrelated to our topic.
> 
>> […] NDK_3.9/Documentation% grep limit
>> Releasenotes/hdwrench_lib_relnotes
> 
> Michael:
>> (Actually, I lied there. I do plan to create a RDSK block for a 2 TB
>> disk image where cylinder, head and sector counts all approach the 32
>> bit limit, just to see that my overflow checks work as intended. But
>> that's strictly for Linux testing).
> 
> I think that is a good test for the patch. I am impressed you are
> willing to put the effort for that into this work.
> 
> Okay, that is what I was able to dig out on what is officially
> available. It appears that the official developer documentation for
> AmigaOS is pretty sparse on any of that. For anything else I´d need to
> contact current AmigaOS developers. I am no longer a member of the
> AmigaOS team.
> 
> Maybe the ADF Format FAQ I mentioned has a bit more, but I did not find
> anything obvious on a short glance
> 
> http://lclevy.free.fr/adflib/adf_info.html
> 
> Also it is a third party source.
> 
>>> unsafe to put on a real Amiga. (I'd also suggest teaching Linux to
>>> understand RDSL, which would be RDSK++ sort of. Then use that if
>>> Cylinder Blocks overflows.) The value you will not be able to fill
>>> in
>>> the DosEnvec structure is:
>>>      ULONG de_HighCyl;         /* max cylinder. drive specific */
>>
>> OK, so Cylinder Blocks overflowing is a red flag, and requires to
>> abort parsing the partition table right away? And HighCyl really
>> means the max. number of logical blocks, not cylinders (which would
>> have nr_heads*nr_sects many blocks)? That's probably the cause for my
>> confusion.
> 
> When can they actually overflow? How are they calculated?
> 
> If there is no calculation to calculate it and it is just a static value
> within the RDB, I´d accept what ever the value is in the RDB, unless a
> calculation based on it overflows.
> 
> Again, keep it simple.
> 
> I think it does not make sense for the Linux kernel to try to outsmart
> the user or any of the partitioning tools out there.
> 
> The only question is:
> 
> Can the partitions be calculated in a way that it is safe to access
> them, i.e. that their calculated start and end is really at the location
> the start and end is on the disk? Then accept it.
> 
> For the file systems: Can the file system safely access this? Then
> accept. If not, then decline. *In the filesystem*. This is nothing for
> the RDB parser to check so out of scope for the discussion of this
> patch.
> 
>>> So accessing larger disks once you hit 2 TB means you must increase
>>> the logical block size. And eventually that will waste HUGE amounts
>>> of files when small files are being stored.
>>
>> Just like small inodes wastes huge amounts of space for metadata. It's
>> a tradeoff, and AFFS on a RDSK format disk probably isn't the right
>> choice for huge disks. Never mind that - if someone _does_ go that
>> way, we need to make sure we can parse the RDSK information
>> correctly. And if such a disk causes the 64 bit sector_t in Linux to
>> overflow, I'd like the parser to spot that, too.
>>
>> Thanks for your immense patience in explaining all these subtleties to
>> me.
>>
>> Cheers,
>>
>>      Michael
>>
>>> Any solution will require action on the part of the people
>>> developing
>>> AmigaDOS follow-ons. You might want to get them motivated, somehow,
>>> and proceed from there with a request to be informed of any RDB
>>> changes. I'd suggest to them that removing sensitivity to Cylinder
>>> Blocks sorts of values from the entire system probably would be
>>> painful but the simplest solution.
>>>
>>> {^_^}
> […]
> 

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[jdow]

Get everybody....

On 20180630 00:49, Martin Steigerwald wrote:
 > Whoa, my summary essay triggered digging even more accurately into that
 > matter. For some obscure reason I am even enjoying this. :)
 >
 > jdow - 30.06.18, 05:56:
 >> On 20180629 18:31, Michael Schmitz wrote:> Joanne,
 >>
 >>   > Am 30.06.18 um 12:57 schrieb jdow:
 >>   >> On 20180629 17:44, Michael Schmitz wrote:
 >>   >>> struct PartitionBlock {
 >>   >>>
 >>   >>>            __be32  pb_ID;
 >>   >>>            __be32  pb_SummedLongs;
 >>   >>>            __s32   pb_ChkSum;
 >>   >>>            __u32   pb_HostID;
 >>   >>>            __be32  pb_Next;
 >>   >>>            __u32   pb_Flags;
 >>   >>>            __u32   pb_Reserved1[2];
 >>   >>>            __u32   pb_DevFlags;
 >>   >>>            __u8    pb_DriveName[32];
 >>   >>>            __u32   pb_Reserved2[15];
 >>   >>>            __be32  pb_Environment[17];
 >>   >>>            __u32   pb_EReserved[15];
 >>   >>>
 >>   >>> };
 >>   >>>
 >>   >>   pb_Environment = a struct DosEnvec and it is 20 ULONGs in size.
 >>   >>   I
 >>   >>
 >>   >> believe you are looking at some old include files.
 >>   >
 >>   > Without looking at ancient git history, I'd say between 1993 and
 >>   > 1996. >
 >>   >> These got added to the end of the DosEnvec structure:
 >>   >>      ULONG de_Baud;         /* Baud rate for serial handler */
 >>   >>      ULONG de_Control;         /* Control word for
 >>   >>      handler/filesystem */
 >>   >>      ULONG de_BootBlocks;     /* Number of blocks containing boot
 >>   >>      code */
 >>   >>>
 >>   >>> As far as I can guess from the code, pb_Environment[3] (number of
 >>   >>
 >>   >> heads)
 >>   >>
 >>   >>> and pb_Environment[5] (number of sectors per cylinder) are
 >>   >>> abitrarily
 >>   >>> chosen so the partition size can be expressed as a difference
 >>   >>> between
 >>   >>> pb_Environment[9] and pb_Environment[10] (low and high cylinder
 >>   >>> addresses), which places restrictions on both partition size and
 >>   >>> alignment that depend on where on the disk a partition is placed?
 >>   >>
 >>   >> If you do not teach the OS to ignore Cylinder Blocks type entries
 >>   >> and
 >>   >> use some math on heads and blocks per track the disk size is
 >>   >> relatively stuck modulo using large blocks.
 >>   >
 >>   > As long as AmigaOS and Linux agree on how to express start and end
 >>   > offset for the partitions, that's fine.
 >>   >
 >>   > But I read your other mail to mean that we're stuck to 2 TB disks
 >>   > for
 >>   > now. I don't follow that - we can have partitions of 2 TB each by
 >>   > maxing out rdb_CylBlocks as long as we use 512 bytes per block
 >>   > (since the product of cylinders and blocks per cylinder is limited
 >>   > to 32 bits) and using one cylinder per partition (32 bits
 >>   > available there as well)?
 >>   >
 >>   > But the rdb_CylBlocks limit also means we're safe with 64 bit
 >>   > sector_t in Linux. Best add a check in the parser to warn us if
 >>   > the product of head count and sectors per cylinder overflows 32
 >>   > bit though.
 >>   >
 >>   > Cheers,
 >>   >
 >>   >      Michael
 >>
 >> How long did it tale s to get to 10 TB disks from 2 TB disks. And a
 >> new SD Card spec allows for 128 TB disks. Block sizes get sort of
 >> ridiculous as you get past about 8k bytes or about 32 TB or about two
 >> years from now.
 >>
 >> Do you want to create disks that will fail on AmigaDOS? AmigaDOS, as
 >> far as I know, makes heavy use of Cylinder Blocks values. It
 >> calculating Cylinder Blocks overflows when creating the disk's RDBs
 >> the user MUST be informed it is unsafe to put on a real Amiga. (I'd
 >
 > Joanne, if you are sure on this… I´d say at least warn if not bail out
 > on Cylinder Blocks overflow.

I am not sure how far into the rest of the system rdb_CylinderBlocks propagates. 
It MAY only affect repartitioning the disk. It may affect the "info" command. An 
audit is called for.

 > But given what you say here, no partitioning tool on AmigaOS or AmigaOS
 > like operating system would create such an overflow.
 >
 > Can you verify whether that is the case with the RDB that I attached to
 > the bug report?

No - frankly don't want to take the time to wander through it.

 > Bug 43511 - Partitions: Amiga RDB partition on 2 TB disk way too big,
 > while OK in AmigaOS 4.1

Did ALL the Amiga disk utilities (partitioning tools perhaps excepted) operate 
properly? If they did then the rdb_CylinderBlocks value might not propagate 
through the system.

 > https://bugzilla.kernel.org/show_bug.cgi?id=43511
 >
 > https://bugzilla.kernel.org/attachment.cgi?id=73771
 >
 >> also suggest teaching Linux to understand RDSL, which would be RDSK++
 >> sort of. Then use that if Cylinder Blocks overflows.) The value you
 >> will not be able to fill in the DosEnvec structure is: ULONG
 >> de_HighCyl;         /* max cylinder. drive specific */
 >>
 >>
 >> So accessing larger disks once you hit 2 TB means you must increase
 >> the logical block size. And eventually that will waste HUGE amounts
 >> of files when small files are being stored.
 >
 > As far as I am aware, AmigaOS 4.1 still only supports 512 byte sectors.

If that is the case I had several disks it would have barfed on, some of which 
had no option other than 2k * N block sizes as they were Fujitsu hard sector 
magneto-optical disks. This is something 4.1 MUST fix, IMAO. Is it based on 3.x 
code or is it a complete rewrite?

 >> Any solution will require action on the part of the people developing
 >> AmigaDOS follow-ons. You might want to get them motivated, somehow,
 >> and proceed from there with a request to be informed of any RDB
 >> changes. I'd suggest to them that removing sensitivity to Cylinder
 >> Blocks sorts of values from the entire system probably would be
 >> painful but the simplest solution.
 >
 > I think for this patch it is important to focus on the *current*
 > situation and make the best out of it.
 >
 > I am really inclined to point some AmigaOS 4 developers to this
 > discussion and just looked for an archive. Unfortunately there does not
 > appear to be a working one. The one mentioned on

If nothing else this discussion should prompt them to audit the OS code to see 
just what works and what does not. That will inform the Linux fix. I suggest the 
Linux fix go in and keep an open "potential" bug on the Amiga partitioning tool 
for Linux, presuming there is one. If there isn't, "What, ME worry?"

 > http://www.linux-m68k.org/mail.html
 >
 > http://aire.ncl.ac.uk/Atari/Mailing-Lists/Linux-680x0-vger-List.index.html
 >
 > does not send an answer within the HTTP / TCP timeout limit.
 >
 > I also did not find any archive for linux-block mailing list.
 >
 > And lore.kernel.org only seems to archive LKML itself which is patch and
 > the discussion we have here is not CC´d to.
 >
 > Any advice?

Get the AmigaDOS developers, all of 'em if there is a split with a group trying 
to recreate AmigaDOS open sores.

{^_^}

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[jdow]

As software is discovered to be "broken" at it to the appropriate incompatible list.

Otherwise permanently limit AmigaDOS to 2TB.
{^_^}

On 20180630 02:07, Martin Steigerwald wrote:
> jdow - 30.06.18, 08:47:
>> Let's get everybody:
>>
>> On 20180629 22:26, Michael Schmitz wrote:
>>   > Joanne,
>>   >
>>   > Am 30.06.18 um 15:56 schrieb jdow:
>>   >>>>> As far as I can guess from the code, pb_Environment[3] (number
>>   >>>>> of
>>   >>>>
>>   >>>> heads)
>>   >>>>
>>   >>>>> and pb_Environment[5] (number of sectors per cylinder) are
>>   >>>>> abitrarily
>>   >>>>> chosen so the partition size can be expressed as a difference
>>   >>>>> between
>>   >>>>> pb_Environment[9] and pb_Environment[10] (low and high cylinder
>>   >>>>> addresses), which places restrictions on both partition size
>>   >>>>> and
>>   >>>>> alignment that depend on where on the disk a partition is
>>   >>>>> placed?
>>   >>>>
>>   >>>> If you do not teach the OS to ignore Cylinder Blocks type
>>   >>>> entries and
>>   >>>> use some math on heads and blocks per track the disk size is
>>   >>>> relatively stuck modulo using large blocks.
>>   >>>
>>   >>> As long as AmigaOS and Linux agree on how to express start and
>>   >>> end
>>   >>> offset for the partitions, that's fine.
>>   >>>
>>   >>> But I read your other mail to mean that we're stuck to 2 TB disks
>>   >>> for
>>   >>> now. I don't follow that - we can have partitions of 2 TB each by
>>   >>
>>   >> maxing
>>   >>
>>   >>> out rdb_CylBlocks as long as we use 512 bytes per block (since
>>   >>> the
>>   >>> product of cylinders and blocks per cylinder is limited to 32
>>   >>> bits) and using one cylinder per partition (32 bits available
>>   >>> there as well)?
>>   >>>
>>   >>> But the rdb_CylBlocks limit also means we're safe with 64 bit
>>   >>> sector_t in Linux. Best add a check in the parser to warn us if
>>   >>> the product of head count and sectors per cylinder overflows 32
>>   >>> bit though.
>>   >>>
>>   >>> Cheers,
>>   >>>
>>   >>>        Michael
>>   >>
>>   >> How long did it tale s to get to 10 TB disks from 2 TB disks. And
>>   >> a
>>   >> new SD Card spec allows for 128 TB disks. Block sizes get sort of
>>   >> ridiculous as you get past about 8k bytes or about 32 TB or about
>>   >> two
>>   >> years from now.
>>   >
>>   > I get that - I just don't get why 32 bits for cylinders plus 32
>>   > bits for blocks per cylinder equals 2 TB (4G of 512 byte blocks).
>>   > But I don't know what other limits exist that may restrict the
>>   > total number of blocks to 32 bits.
>>
>> It overflows uint32_t cylinder blocks aka blocks per cylinder. Linux
>> doesn't care. AmigaDOS surely does. If YOU make partitions really
>> large for yourself that's OK. If Joe Amigoid does it the potential
>> for an angry red turning to purple face is high.
> 
> Ok, let get this straight:
> 
> Do you think that is the responsibility of the RDB parser within the
> Linux kernel to protect the user from anything whatever partitioning
> tool has created?
> 
> If so, how would you make sure the Linux kernel knows about whatever any
> partitioning tool used by Amiga users can come up with?
> 
> I´d say: Don´t bother. It is not the job of the RDB parser to impose
> limits on what partitioning tools can create. If native OS tools don´t
> create such thing, you don´t need to check for it. If someone managed to
> create it with amiga-fdisk or parted, the tool needs to be fixed. *Not*
> the kernel.
> 
> Anyway, that 2 TB disk that started all this *worked* on AmigaOS 4. And
> I am pretty sure while I cannot proof it, that even a larger disk would
> work. There is a limit for the boot partition on AmigaOS 4 Classic,
> which uses AmigaOS 3.1 to bootstrap AmigaOS 4 on Classic Amiga computers
> like an Amiga 1200 or Amiga 4000 with PowerPC extension card. But
> according to
> 
> Hard drive setup for AmigaOS 4.1 Classic
> http://blog.hyperion-entertainment.biz/?p=210
> 
> AmigaOS classic (i.e. < 4) would crash.
> 
>>   >> Do you want to create disks that will fail on AmigaDOS? AmigaDOS,
>>   >> as
>>   >> far as I know, makes heavy use of Cylinder Blocks values. It
>>   >> calculating Cylinder Blocks overflows when creating the disk's
>>   >> RDBs
>>   >> the user MUST be informed it is
>>   >
>>   > I'm not at all planning to create disks for AmigaDOS. I just need
>>   > to
>>   > know what combinations of cylinders, heads and sectors are possible
>>   > to encounter on disks that have been created with native tools.
>>   > Well, assuming sufficient amounts of braindamage in the
>>   > corresponding Linux tools, knowing the absolute outer limits of
>>   > what these tools could do would be nice as well, but someone using
>>   > amiga-fdisk to create a RDSK block for a 10 TB disk fully deserves
>>   > any punishment that invites.
>> Native AmigaDOS tools SHOULD NOT be able to create something that
>> overflows CylinderBlocks values. However, if it can that creates an
> 
> There you have it.
> 
> Then *why* bother, Joanne?
> 
>> interesting test case to see what various tools, like the AmigaDOS
>> "info" command, do when they they are run on such a disk. I don't
>> have OS source to perform searches. And I am not setup to feed the
>> system something obscene.
>>
>>   > (Actually, I lied there. I do plan to create a RDSK block for a 2
>>   > TB
>>   > disk image where cylinder, head and sector counts all approach the
>>   > 32
>>   > bit limit, just to see that my overflow checks work as intended.
>>   > But
>>   > that's strictly for Linux testing).
>>   >
>>   >> unsafe to put on a real Amiga. (I'd also suggest teaching Linux to
>>   >> understand RDSL, which would be RDSK++ sort of. Then use that if
>>   >> Cylinder Blocks overflows.) The value you will not be able to fill
>>   >> in
>>   >>
>>   >> the DosEnvec structure is:
>>   >>      ULONG de_HighCyl;         /* max cylinder. drive specific */
>>   >
>>   > OK, so Cylinder Blocks overflowing is a red flag, and requires to
>>   > abort parsing the partition table right away? And HighCyl really
>>   > means the max. number of logical blocks, not cylinders (which
>>   > would have nr_heads*nr_sects many blocks)? That's probably the
>>   > cause for my confusion.
>> I think I picked the wrong value. In RDSK itself this value is what
>> overflows: ULONG   rdb_CylBlocks;    /* number of blocks available
>> per cylinder */ And I think that floats around the system in many
>> places with different names. As mentioned the "info" command is one
>> item to test. If no crashes are found then AmigaDOS may be clean up
>> to obscene sizes. At the moment I do not remember what
>> hdwrench.library does with that value other than pass it along as
>> read. Nor am I sure what it generates as any suggested values. I
>> don't at this time have a disk I can mount as a disk on WinUAE that
>> is more than 2TB. And my Amigas speak SCSI so I have no disk for
>> them, either, even if they still boot.
>>   >> So accessing larger disks once you hit 2 TB means you must
>>   >> increase
>>   >> the logical block size. And eventually that will waste HUGE
>>   >> amounts of files when small files are being stored.
>>   >
>>   > Just like small inodes wastes huge amounts of space for metadata.
>>   > It's a tradeoff, and AFFS on a RDSK format disk probably isn't the
>>   > right choice for huge disks. Never mind that - if someone _does_
>>   > go that way, we need to make sure we can parse the RDSK
>>   > information correctly. And if such a disk causes the 64 bit
>>   > sector_t in Linux to overflow, I'd like the parser to spot that,
>>   > too.
>>   >
>>   > Thanks for your immense patience in explaining all these subtleties
>>   > to me.
> […]
>>   >      Michael
>>
>> And I'm rushing too much so I'm sorry I am making errors. This stuff
>> is 25 years in the past since I last looked at it seriously.
> 
> I think its important to focus on what can overflow can happen within
> calculations the RDB parser (and as a second step the AFFS file system)
> in the kernel kernel in order to keep this discussion to a manageable
> size. Be conservative about overflows, but otherwise accept. With a
> warning if a calculated exceed 32 bit.
> 
> As for values in the RDB. If its there, accept it. Some tool has written
> it there. We don´t know whether it did this right or wrong. We don´t
> know what the developer of the tool thought when writing it, well except
> for hdwrench.library I´d say as far as you remember. :) And it is not
> our job within the kernel to check that.
> 
> There is a ton of more or less legacy software out there on native OS
> which does something to or with RDBs. I´d say it is impossible to say
> what RDB a user may come up with.
> 
> Thanks,
> 

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[Geert Uytterhoeven]

Hi Michael,

On Fri, Jun 29, 2018 at 11:12 AM Michael Schmitz <schmitzmic@gmail.com> wrote:
> Am 28.06.18 um 01:30 schrieb Geert Uytterhoeven:
> > On Wed, Jun 27, 2018 at 4:47 AM <schmitzmic@gmail.com> wrote:
> >> From 5299e0e64dfb33ac3a1f3137b42178734ce20087 Mon Sep 17 00:00:00 2001
> > ??
> >
> >> The Amiga RDB partition parser module uses int for partition sector
> >> address and count, which will overflow for disks 2 TB and larger.
> >>
> >> Use sector_t as type for sector address and size (as expected by
> >> put_partition) to allow using such disks without danger of data
> >> corruption.
> > Note that sector_t is not guaranteed to be 64-bit:
> >
> >     #ifdef CONFIG_LBDAF
> >     typedef u64 sector_t;
> >     typedef u64 blkcnt_t;
> >     #else
> >     typedef unsigned long sector_t;
> >     typedef unsigned long blkcnt_t;
> >     #endif
> >
> > And it seems CONFIG_LBDAF can still be disabled on 32-bit...
> >
>
> What are the ramifications of using a 2 TB disk on a kernel without
> CONFIG_LBDAF? Are there any safeguards after the partition scan stage
> that would prevent the kernel from using partitions on such a disk?
>
> Trying to decide whether we absolutely have to bail if sector_t is 32
> bit ...

Sorry, I don't know.

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[Geert Uytterhoeven]

Hi Andreas,

On Fri, Jun 29, 2018 at 3:26 PM Andreas Schwab <schwab@linux-m68k.org> wrote:
> On Jun 29 2018, Michael Schmitz <schmitzmic@gmail.com> wrote:
> > Would MSDOS recognize the GPT partition as 'probably FAT', and attempt
> > to use it?
>
> GPT has the concept of a protective MBR which should prevent such errors.

Thanks, good to know.

So it looks like GPT and RDSK can coexist on the same disk: an RDSK header
in block 0 (pointing to PART blocks located after the GPT), and GPT in
blocks 1-33.

That means for very large disks, you can have an RDSK describing the
partitions that fit in the first part of the disk that can be accessed by
e.g. AmigaOS 3.0, and a GPT that describes all partitions on the disk,
right?

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[Michael Schmitz]

Hi Geert,

Am 01.07.2018 um 09:10 schrieb Geert Uytterhoeven:
> Hi Michael,
>
> On Fri, Jun 29, 2018 at 11:12 AM Michael Schmitz <schmitzmic@gmail.com> wrote:
>> Am 28.06.18 um 01:30 schrieb Geert Uytterhoeven:
>>> On Wed, Jun 27, 2018 at 4:47 AM <schmitzmic@gmail.com> wrote:
>>>> From 5299e0e64dfb33ac3a1f3137b42178734ce20087 Mon Sep 17 00:00:00 2001
>>> ??
>>>
>>>> The Amiga RDB partition parser module uses int for partition sector
>>>> address and count, which will overflow for disks 2 TB and larger.
>>>>
>>>> Use sector_t as type for sector address and size (as expected by
>>>> put_partition) to allow using such disks without danger of data
>>>> corruption.
>>> Note that sector_t is not guaranteed to be 64-bit:
>>>
>>>     #ifdef CONFIG_LBDAF
>>>     typedef u64 sector_t;
>>>     typedef u64 blkcnt_t;
>>>     #else
>>>     typedef unsigned long sector_t;
>>>     typedef unsigned long blkcnt_t;
>>>     #endif
>>>
>>> And it seems CONFIG_LBDAF can still be disabled on 32-bit...
>>>
>>
>> What are the ramifications of using a 2 TB disk on a kernel without
>> CONFIG_LBDAF? Are there any safeguards after the partition scan stage
>> that would prevent the kernel from using partitions on such a disk?
>>
>> Trying to decide whether we absolutely have to bail if sector_t is 32
>> bit ...
>
> Sorry, I don't know.

Not to worry - I'll find out one way or another. I've added some more 
checks to spot both 32 bit and 64 bit overflows following what we 
discussed in the past days (though the latter really shouldn't happen). 
I'm quite sure that will throw everyone on linux-block in a laughing 
fit, but I'm sure someone will point out a better way of doing this 
during review (or prove it won't be necessary).

Pending final test, I'll submit the result today or tomorrow.

Cheers,

	Michael

>
> Gr{oetje,eeting}s,
>
>                         Geert
>

Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[Michael Schmitz]

Martin,

Am 30.06.18 um 19:49 schrieb Martin Steigerwald:
> I am really inclined to point some AmigaOS 4 developers to this
> discussion and just looked for an archive. Unfortunately there does not 
> appear to be a working one. The one mentioned on
>
> http://www.linux-m68k.org/mail.html
>
> http://aire.ncl.ac.uk/Atari/Mailing-Lists/Linux-680x0-vger-List.index.html
>
> does not send an answer within the HTTP / TCP timeout limit.

Most of the information found online on m68k will be several years out
of date, I'd expect. The page you appear to have found will be several
decades out of date (any mention of the FTP server at Uni Saarbruecken
for list archives is a dead giveaway). The internet doesn't forget, but
that can be a curse rather than a blessing at times.

Anyway, vger sez:

https://marc.info/?l=linux-m68k&r=1&w=2
and
https://www.spinics.net/lists/linux-m68k/

which work for me.
>
> I also did not find any archive for linux-block mailing list

https://marc.info/?l=linux-block works for me.

> .
>
> And lore.kernel.org only seems to archive LKML itself which is patch and 
> the discussion we have here is not CC´d to.

And I'm glad we didn't.

Cheers,

    Michael

>
> Any advice?
>
> Thanks,

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[jdow]

FWIW on the other side this appears to be a good source of Amiga software data.

http://amigadev.elowar.com/
{^_^}

On 20180630 19:43, Michael Schmitz wrote:
> Martin,
> 
> Am 30.06.18 um 19:49 schrieb Martin Steigerwald:
>> I am really inclined to point some AmigaOS 4 developers to this
>> discussion and just looked for an archive. Unfortunately there does not
>> appear to be a working one. The one mentioned on
>>
>> http://www.linux-m68k.org/mail.html
>>
>> http://aire.ncl.ac.uk/Atari/Mailing-Lists/Linux-680x0-vger-List.index.html
>>
>> does not send an answer within the HTTP / TCP timeout limit.
> 
> Most of the information found online on m68k will be several years out
> of date, I'd expect. The page you appear to have found will be several
> decades out of date (any mention of the FTP server at Uni Saarbruecken
> for list archives is a dead giveaway). The internet doesn't forget, but
> that can be a curse rather than a blessing at times.
> 
> Anyway, vger sez:
> 
> https://marc.info/?l=linux-m68k&r=1&w=2
> and
> https://www.spinics.net/lists/linux-m68k/
> 
> which work for me.
>>
>> I also did not find any archive for linux-block mailing list
> 
> https://marc.info/?l=linux-block works for me.
> 
>> .
>>
>> And lore.kernel.org only seems to archive LKML itself which is patch and
>> the discussion we have here is not CC´d to.
> 
> And I'm glad we didn't.
> 
> Cheers,
> 
>      Michael
> 
>>
>> Any advice?
>>
>> Thanks,
> 

Re: Subject: [PATCH RFC] block: fix Amiga RDB partition support for disks >= 2 TB

[Martin Steigerwald]

Michael.

Michael Schmitz - 01.07.18, 04:43:
> Am 30.06.18 um 19:49 schrieb Martin Steigerwald:
> > I am really inclined to point some AmigaOS 4 developers to this
> > discussion and just looked for an archive. Unfortunately there does
> > not appear to be a working one. The one mentioned on
> >=20
> > http://www.linux-m68k.org/mail.html
> >=20
> > http://aire.ncl.ac.uk/Atari/Mailing-Lists/Linux-680x0-vger-List.inde
> > x.html
> >=20
> > does not send an answer within the HTTP / TCP timeout limit.
>=20
> Most of the information found online on m68k will be several years out
> of date, I'd expect. The page you appear to have found will be
> several decades out of date (any mention of the FTP server at Uni
> Saarbruecken for list archives is a dead giveaway). The internet
> doesn't forget, but that can be a curse rather than a blessing at
> times.
>=20
> Anyway, vger sez:
>=20
> https://marc.info/?l=3Dlinux-m68k&r=3D1&w=3D2
> and
> https://www.spinics.net/lists/linux-m68k/
>=20
> which work for me.
>=20
> > I also did not find any archive for linux-block mailing list
>=20
> https://marc.info/?l=3Dlinux-block works for me.

Ah, I thought marc.info was down, but well now I remember that someone=20
took over or something like that.

Did not think of spinics.net.

Well I wrote a mail to some AmigaOS 4 developers. Whether I get a=20
response remains to be seen. However, in any case I=B4d continue with=20
developing the patch. I=B4d I not make the Linux fix dependent on this.

If they share something with me to forward it regarding the Linux patch,=20
I=B4d share that with you.

> > And lore.kernel.org only seems to archive LKML itself which is patch
> > and the discussion we have here is not CC=B4d to.
>=20
> And I'm glad we didn't.

:)

Thanks,
=2D-=20
Martin

[PATCH] block: fix Amiga partition support for disks >= 1 TB

[Michael Schmitz]

The Amiga partition parser module uses signed int for partition sector
address and count, which will overflow for disks larger than 1 TB.

Use u64 as type for sector address and size to allow using disks up to
2 TB without LBD support, and disks larger than 2 TB with LBD. The RBD
format allows to specify disk sizes up to 2^128 bytes (though native
OS limitations reduce this somewhat, to max 2^68 bytes), so check for
u64 overflow carefully to protect against overflowing sector_t.

Bail out if sector addresses overflow 32 bits on kernels without LBD
support.

This bug was reported originally in 2012, and the fix was created by
the RDB author, Joanne Dow <jdow@earthlink.net>. A patch had been
discussed and reviewed on linux-m68k at that time but never officially
submitted. This patch adds additional error checking and warning messages.

Fixes: https://bugzilla.kernel.org/show_bug.cgi?id=43511
Reported-by: Martin Steigerwald <Martin@lichtvoll.de>
Message-ID: <201206192146.09327.Martin@lichtvoll.de>
Signed-off-by: Michael Schmitz <schmitzmic@gmail.com>
Tested-by: Martin Steigerwald <Martin@lichtvoll.de>

Changes from RFC:

- use u64 instead of sector_t, since that may be u32 without LBD support
- check multiplication overflows each step - 3 u32 values may exceed u64!
- warn against use on AmigaDOS if partition data overflow u32 sector count.
- warn if partition CylBlocks larger than what's stored in the RDSK header.
- bail out if we were to overflow sector_t (32 or 64 bit).
---
 block/partitions/amiga.c |   99 +++++++++++++++++++++++++++++++++++++++++++++-
 1 files changed, 98 insertions(+), 1 deletions(-)

diff --git a/block/partitions/amiga.c b/block/partitions/amiga.c
index 5609366..6184d83 100644
--- a/block/partitions/amiga.c
+++ b/block/partitions/amiga.c
@@ -11,6 +11,7 @@
 #define pr_fmt(fmt) fmt
 
 #include <linux/types.h>
+#include <linux/log2.h>
 #include <linux/affs_hardblocks.h>
 
 #include "check.h"
@@ -32,7 +33,9 @@ int amiga_partition(struct parsed_partitions *state)
 	unsigned char *data;
 	struct RigidDiskBlock *rdb;
 	struct PartitionBlock *pb;
-	int start_sect, nr_sects, blk, part, res = 0;
+	u64 start_sect, nr_sects;
+	u64 cylblk, cylblk_res;	/* rdb_CylBlocks = nr_heads*sect_per_track */
+	int blk, part, res = 0, blk_shift = 0, did_warn = 0;
 	int blksize = 1;	/* Multiplier for disk block size */
 	int slot = 1;
 	char b[BDEVNAME_SIZE];
@@ -98,6 +101,79 @@ int amiga_partition(struct parsed_partitions *state)
 		if (checksum_block((__be32 *)pb, be32_to_cpu(pb->pb_SummedLongs) & 0x7F) != 0 )
 			continue;
 
+		/* RDB gives us more than enough rope to hang ourselves with,
+		 * many times over (2^128 bytes if all fields max out).
+		 * Some careful checks are in order.
+		 */
+
+		/* CylBlocks is total number of blocks per cylinder */
+		cylblk = be32_to_cpu(pb->pb_Environment[3]) *
+			 be32_to_cpu(pb->pb_Environment[5]);
+
+		/* check for consistency with RDB defined CylBlocks */
+		if (cylblk > be32_to_cpu(rdb->rdb_CylBlocks)) {
+			pr_err("Dev %s: cylblk 0x%lx > rdb_CylBlocks 0x%x!\n",
+				bdevname(state->bdev, b),
+				(unsigned long) cylblk,
+				be32_to_cpu(rdb->rdb_CylBlocks));
+		}
+
+		/* check for potential overflows - we are going to multiply
+		 * three 32 bit numbers to one 64 bit result later!
+		 * Condition 1: nr_heads * sects_per_track must fit u32!
+		 * NB: This is a HARD limit for AmigaDOS. We don't care much.
+		 */
+
+		if (cylblk > UINT_MAX) {
+			pr_err("Dev %s: hds*sects 0x%lx > UINT_MAX!\n",
+				bdevname(state->bdev, b),
+				(unsigned long) cylblk);
+
+			/* lop off low 32 bits */
+			cylblk_res = cylblk >> 32;
+
+			/* check for further overflow in end result */
+			if (be32_to_cpu(pb->pb_Environment[9]) *
+				cylblk_res * blksize > UINT_MAX) {
+				pr_err("Dev %s: start_sect overflows u64!\n",
+					bdevname(state->bdev, b));
+				res = -1;
+				goto rdb_done;
+			}
+
+			if (be32_to_cpu(pb->pb_Environment[10]) *
+			   cylblk_res * blksize > UINT_MAX) {
+				pr_err("Dev %s: end_sect overflows u64!\n",
+					bdevname(state->bdev, b));
+				res = -1;
+				goto rdb_done;
+			}
+		}
+
+		/* Condition 2: even if CylBlocks did not overflow, the end
+		 * result must still fit u64!
+		 */
+
+		/* how many bits above 32 in cylblk * blksize ? */
+		if (cylblk*blksize > (u64) UINT_MAX)
+			blk_shift = ilog2(cylblk*blksize) - 32;
+
+		if (be32_to_cpu(pb->pb_Environment[9])
+			> (u64) UINT_MAX>>blk_shift) {
+			pr_err("Dev %s: start_sect overflows u64!\n",
+				bdevname(state->bdev, b));
+			res = -1;
+			goto rdb_done;
+		}
+
+		if (be32_to_cpu(pb->pb_Environment[10])
+			> (u64) UINT_MAX>>blk_shift) {
+			pr_err("Dev %s: end_sect overflows u64!\n",
+				bdevname(state->bdev, b));
+			res = -1;
+			goto rdb_done;
+		}
+
 		/* Tell Kernel about it */
 
 		nr_sects = (be32_to_cpu(pb->pb_Environment[10]) + 1 -
@@ -111,6 +187,27 @@ int amiga_partition(struct parsed_partitions *state)
 			     be32_to_cpu(pb->pb_Environment[3]) *
 			     be32_to_cpu(pb->pb_Environment[5]) *
 			     blksize;
+
+		/* Warn user if start_sect or nr_sects overflow u32 */
+		if ((nr_sects > UINT_MAX || start_sect > UINT_MAX ||
+		    (start_sect + nr_sects) > UINT_MAX) && !did_warn) {
+			pr_err("Dev %s: partition 32 bit overflow! ",
+				bdevname(state->bdev, b));
+			pr_cont("start_sect 0x%llX, nr_sects 0x%llx\n",
+				 start_sect, nr_sects);
+			pr_err("Dev %s: partition may  need 64 bit ",
+				bdevname(state->bdev, b));
+			pr_cont("device support on native AmigaOS!\n");
+			/* Without LBD support, better bail out */
+			if (!IS_ENABLED(CONFIG_LBDAF)) {
+				pr_err("Dev %s: no LBD support, aborting!",
+					bdevname(state->bdev, b));
+				res = -1;
+				goto rdb_done;
+			}
+			did_warn++;
+		}
+
 		put_partition(state,slot++,start_sect,nr_sects);
 		{
 			/* Be even more informative to aid mounting */
-- 
1.7.0.4

Re: [PATCH] block: fix Amiga partition support for disks >= 1 TB

[Kars de Jong]

Op ma 2 jul. 2018 om 07:29 schreef Michael Schmitz <schmitzmic@gmail.com>:
> @@ -98,6 +101,79 @@ int amiga_partition(struct parsed_partitions *state)
>                 if (checksum_block((__be32 *)pb, be32_to_cpu(pb->pb_SummedLongs) & 0x7F) != 0 )
>                         continue;
>
> +               /* RDB gives us more than enough rope to hang ourselves with,
> +                * many times over (2^128 bytes if all fields max out).
> +                * Some careful checks are in order.
> +                */
> +
> +               /* CylBlocks is total number of blocks per cylinder */
> +               cylblk = be32_to_cpu(pb->pb_Environment[3]) *
> +                        be32_to_cpu(pb->pb_Environment[5]);
> +

Could you please create #defines for all these magic offsets in pb_Environment?
Below are a few more.
This makes the code much more readable.
Thanks!

> +               /* check for consistency with RDB defined CylBlocks */
> +               if (cylblk > be32_to_cpu(rdb->rdb_CylBlocks)) {
> +                       pr_err("Dev %s: cylblk 0x%lx > rdb_CylBlocks 0x%x!\n",
> +                               bdevname(state->bdev, b),
> +                               (unsigned long) cylblk,
> +                               be32_to_cpu(rdb->rdb_CylBlocks));
> +               }
> +
> +               /* check for potential overflows - we are going to multiply
> +                * three 32 bit numbers to one 64 bit result later!
> +                * Condition 1: nr_heads * sects_per_track must fit u32!
> +                * NB: This is a HARD limit for AmigaDOS. We don't care much.
> +                */
> +
> +               if (cylblk > UINT_MAX) {
> +                       pr_err("Dev %s: hds*sects 0x%lx > UINT_MAX!\n",
> +                               bdevname(state->bdev, b),
> +                               (unsigned long) cylblk);
> +
> +                       /* lop off low 32 bits */
> +                       cylblk_res = cylblk >> 32;
> +
> +                       /* check for further overflow in end result */
> +                       if (be32_to_cpu(pb->pb_Environment[9]) *
> +                               cylblk_res * blksize > UINT_MAX) {
> +                               pr_err("Dev %s: start_sect overflows u64!\n",
> +                                       bdevname(state->bdev, b));
> +                               res = -1;
> +                               goto rdb_done;
> +                       }
> +
> +                       if (be32_to_cpu(pb->pb_Environment[10]) *
> +                          cylblk_res * blksize > UINT_MAX) {
> +                               pr_err("Dev %s: end_sect overflows u64!\n",
> +                                       bdevname(state->bdev, b));
> +                               res = -1;
> +                               goto rdb_done;
> +                       }
> +               }
> +
> +               /* Condition 2: even if CylBlocks did not overflow, the end
> +                * result must still fit u64!
> +                */
> +
> +               /* how many bits above 32 in cylblk * blksize ? */
> +               if (cylblk*blksize > (u64) UINT_MAX)
> +                       blk_shift = ilog2(cylblk*blksize) - 32;
> +
> +               if (be32_to_cpu(pb->pb_Environment[9])
> +                       > (u64) UINT_MAX>>blk_shift) {
> +                       pr_err("Dev %s: start_sect overflows u64!\n",
> +                               bdevname(state->bdev, b));
> +                       res = -1;
> +                       goto rdb_done;
> +               }
> +
> +               if (be32_to_cpu(pb->pb_Environment[10])
> +                       > (u64) UINT_MAX>>blk_shift) {
> +                       pr_err("Dev %s: end_sect overflows u64!\n",
> +                               bdevname(state->bdev, b));
> +                       res = -1;
> +                       goto rdb_done;
> +               }
> +
>                 /* Tell Kernel about it */
>
>                 nr_sects = (be32_to_cpu(pb->pb_Environment[10]) + 1 -

Re: [PATCH] block: fix Amiga partition support for disks >= 1 TB

[Geert Uytterhoeven]

Hi Michael,

On Mon, Jul 2, 2018 at 7:30 AM Michael Schmitz <schmitzmic@gmail.com> wrote:
> The Amiga partition parser module uses signed int for partition sector
> address and count, which will overflow for disks larger than 1 TB.
>
> Use u64 as type for sector address and size to allow using disks up to
> 2 TB without LBD support, and disks larger than 2 TB with LBD. The RBD
> format allows to specify disk sizes up to 2^128 bytes (though native
> OS limitations reduce this somewhat, to max 2^68 bytes), so check for
> u64 overflow carefully to protect against overflowing sector_t.
>
> Bail out if sector addresses overflow 32 bits on kernels without LBD
> support.
>
> This bug was reported originally in 2012, and the fix was created by
> the RDB author, Joanne Dow <jdow@earthlink.net>. A patch had been
> discussed and reviewed on linux-m68k at that time but never officially
> submitted. This patch adds additional error checking and warning messages.
>
> Fixes: https://bugzilla.kernel.org/show_bug.cgi?id=43511
> Reported-by: Martin Steigerwald <Martin@lichtvoll.de>
> Message-ID: <201206192146.09327.Martin@lichtvoll.de>
> Signed-off-by: Michael Schmitz <schmitzmic@gmail.com>
> Tested-by: Martin Steigerwald <Martin@lichtvoll.de>
>
> Changes from RFC:
>
> - use u64 instead of sector_t, since that may be u32 without LBD support
> - check multiplication overflows each step - 3 u32 values may exceed u64!
> - warn against use on AmigaDOS if partition data overflow u32 sector count.
> - warn if partition CylBlocks larger than what's stored in the RDSK header.
> - bail out if we were to overflow sector_t (32 or 64 bit).

Thanks for your patch!

> --- a/block/partitions/amiga.c
> +++ b/block/partitions/amiga.c
> @@ -11,6 +11,7 @@
>  #define pr_fmt(fmt) fmt
>
>  #include <linux/types.h>
> +#include <linux/log2.h>
>  #include <linux/affs_hardblocks.h>
>
>  #include "check.h"
> @@ -32,7 +33,9 @@ int amiga_partition(struct parsed_partitions *state)
>         unsigned char *data;
>         struct RigidDiskBlock *rdb;
>         struct PartitionBlock *pb;
> -       int start_sect, nr_sects, blk, part, res = 0;
> +       u64 start_sect, nr_sects;
> +       u64 cylblk, cylblk_res; /* rdb_CylBlocks = nr_heads*sect_per_track */
> +       int blk, part, res = 0, blk_shift = 0, did_warn = 0;
>         int blksize = 1;        /* Multiplier for disk block size */
>         int slot = 1;
>         char b[BDEVNAME_SIZE];
> @@ -98,6 +101,79 @@ int amiga_partition(struct parsed_partitions *state)
>                 if (checksum_block((__be32 *)pb, be32_to_cpu(pb->pb_SummedLongs) & 0x7F) != 0 )
>                         continue;
>
> +               /* RDB gives us more than enough rope to hang ourselves with,
> +                * many times over (2^128 bytes if all fields max out).
> +                * Some careful checks are in order.
> +                */
> +
> +               /* CylBlocks is total number of blocks per cylinder */
> +               cylblk = be32_to_cpu(pb->pb_Environment[3]) *
> +                        be32_to_cpu(pb->pb_Environment[5]);

Does the above really do a 32 * 32 = 64 bit multiplication?
be32 is unsigned int, and multiplying it will be done in 32-bit arithmetic:

    unsigned int a = 100000;
    unsigned int b = 100000;
    unsigned long long c = a * b;
    unsigned long long d = (unsigned long long)a * b;
    printf("c = %llu\n", c);
    printf("d = %llu\n", d);

prints:

    c = 1410065408
    d = 10000000000

If it does work for you, what am I missing?

> +
> +               /* check for consistency with RDB defined CylBlocks */
> +               if (cylblk > be32_to_cpu(rdb->rdb_CylBlocks)) {
> +                       pr_err("Dev %s: cylblk 0x%lx > rdb_CylBlocks 0x%x!\n",
> +                               bdevname(state->bdev, b),
> +                               (unsigned long) cylblk,

Why the cast? This will truncate the value on 32-bit platforms.
Just use %llu (IMHO decimal is better suited here).

> +                               be32_to_cpu(rdb->rdb_CylBlocks));
> +               }
> +
> +               /* check for potential overflows - we are going to multiply
> +                * three 32 bit numbers to one 64 bit result later!
> +                * Condition 1: nr_heads * sects_per_track must fit u32!
> +                * NB: This is a HARD limit for AmigaDOS. We don't care much.

So, is condition 1 really needed?

> +                */
> +
> +               if (cylblk > UINT_MAX) {
> +                       pr_err("Dev %s: hds*sects 0x%lx > UINT_MAX!\n",
> +                               bdevname(state->bdev, b),
> +                               (unsigned long) cylblk);

Again, why the cast/truncation?

> +
> +                       /* lop off low 32 bits */
> +                       cylblk_res = cylblk >> 32;
> +
> +                       /* check for further overflow in end result */
> +                       if (be32_to_cpu(pb->pb_Environment[9]) *
> +                               cylblk_res * blksize > UINT_MAX) {
> +                               pr_err("Dev %s: start_sect overflows u64!\n",
> +                                       bdevname(state->bdev, b));
> +                               res = -1;
> +                               goto rdb_done;
> +                       }
> +
> +                       if (be32_to_cpu(pb->pb_Environment[10]) *
> +                          cylblk_res * blksize > UINT_MAX) {
> +                               pr_err("Dev %s: end_sect overflows u64!\n",
> +                                       bdevname(state->bdev, b));
> +                               res = -1;
> +                               goto rdb_done;
> +                       }

No need to reinvent the wheel, #include <linux/overflow.h>, and use
check_mul_overflow(), like array3_size() does.

> +               }
> +
> +               /* Condition 2: even if CylBlocks did not overflow, the end
> +                * result must still fit u64!
> +                */
> +
> +               /* how many bits above 32 in cylblk * blksize ? */
> +               if (cylblk*blksize > (u64) UINT_MAX)
> +                       blk_shift = ilog2(cylblk*blksize) - 32;
> +
> +               if (be32_to_cpu(pb->pb_Environment[9])
> +                       > (u64) UINT_MAX>>blk_shift) {
> +                       pr_err("Dev %s: start_sect overflows u64!\n",
> +                               bdevname(state->bdev, b));
> +                       res = -1;
> +                       goto rdb_done;
> +               }
> +
> +               if (be32_to_cpu(pb->pb_Environment[10])
> +                       > (u64) UINT_MAX>>blk_shift) {
> +                       pr_err("Dev %s: end_sect overflows u64!\n",
> +                               bdevname(state->bdev, b));
> +                       res = -1;
> +                       goto rdb_done;

check_mul_overflow()

> +               }
> +
>                 /* Tell Kernel about it */
>
>                 nr_sects = (be32_to_cpu(pb->pb_Environment[10]) + 1 -
> @@ -111,6 +187,27 @@ int amiga_partition(struct parsed_partitions *state)
>                              be32_to_cpu(pb->pb_Environment[3]) *
>                              be32_to_cpu(pb->pb_Environment[5]) *
>                              blksize;

I'm still not convinced the above is done in 64-bit arithmetic...

> +
> +               /* Warn user if start_sect or nr_sects overflow u32 */
> +               if ((nr_sects > UINT_MAX || start_sect > UINT_MAX ||
> +                   (start_sect + nr_sects) > UINT_MAX) && !did_warn) {

I guess "start_sect + nr_sects > UINT_MAX" is sufficient?
I would remove the did_warn check, as multiple partitions may be affected.
Also, RDB doesn't enforce partition ordering, IIRC, so e.g. partitions 1
and 3 could be outside the 2 TiB area, while 2 could be within.

> +                       pr_err("Dev %s: partition 32 bit overflow! ",

pr_warn()

> +                               bdevname(state->bdev, b));
> +                       pr_cont("start_sect 0x%llX, nr_sects 0x%llx\n",
> +                                start_sect, nr_sects);

No need for pr_cont(), just merge the two statements.

> +                       pr_err("Dev %s: partition may  need 64 bit ",

pr_warn()

Drop the "may"?

> +                               bdevname(state->bdev, b));

I would print the partition index, too.

> +                       pr_cont("device support on native AmigaOS!\n");

Just merge the two statements.

I think it can also be just one line printed instead of 2?

    pr_warn("Dev %s: partition %u (%llu-%llu) needs 64-bit device
support\n", ...

> +                       /* Without LBD support, better bail out */
> +                       if (!IS_ENABLED(CONFIG_LBDAF)) {
> +                               pr_err("Dev %s: no LBD support, aborting!",

    pr_err("Dev %s: no LBD support, skipping partition %u\n", ...)?

> +                                       bdevname(state->bdev, b));
> +                               res = -1;
> +                               goto rdb_done;

Aborting here renders any later partitions within the 2 TiB area unaccessible.
So please continue.

> +                       }
> +                       did_warn++;
> +               }
> +
>                 put_partition(state,slot++,start_sect,nr_sects);
>                 {
>                         /* Be even more informative to aid mounting */

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [PATCH] block: fix Amiga partition support for disks >= 1 TB

[Martin Steigerwald]

Hi Michael.

I felt free to Cc my mail address.

Michael Schmitz - 02.07.18, 07:29:
> The Amiga partition parser module uses signed int for partition sector
> address and count, which will overflow for disks larger than 1 TB.
>=20
> Use u64 as type for sector address and size to allow using disks up to
> 2 TB without LBD support, and disks larger than 2 TB with LBD. The RBD
> format allows to specify disk sizes up to 2^128 bytes (though native
> OS limitations reduce this somewhat, to max 2^68 bytes), so check for
> u64 overflow carefully to protect against overflowing sector_t.
>=20
> Bail out if sector addresses overflow 32 bits on kernels without LBD
> support.

=46irst off, I have an reply by AmigaOS developer Olaf Barthel regarding his
recommendations for Linux that I may post here. Quoted part is my question,
unquoted part is his reply.

=2D--------------------------------------------------------------
> Would you, at the current time, given the circumstances you described,=20
> recommend any additional limitations to the Linux RDB parser=20
> implementation? If so, which ones, and why? Note this is just about the=20
> RDB parser for now, not AFFS in Linux and not amiga-fdisk, parted or=20
> what else. I am aware that AFFS in Linux would also need to be checked=20
> for limit handling :).

That's where the restrictions governing RDB and file system diverge. You
can have a perfectly consistent RDB, but the file system may be unable
to make use of it. This is the case, for example, with the default ROM
file system in Kickstart 2.x/3.x which will accept what scsi.device
found in the RDB, but might not work out well at all later because the
ROM file system does not know its own limitations.

Generally, keeping the RDB consistent should be the primary objective.
The question is how you are going to deal with the consequences. If the
AFFS file system can be relied upon to know its limitations then you
might end up with a good pairing with the RDB parser for Linux only. The
problems begin with how an AmigaOS-native file system is expected to
deal with the RDB.

If you want to limit the risk of data loss, then the cautious approach
would be to put limitations into the partitioning software (fdisk), like
we did for HDToolBox/ProdPrep, namely:

1. Do not create new partitions which reach beyond the 4 Gigabyte
   boundary of the storage medium.
2. Do not create new partitions which exceed 2 Gigabytes in size.
3. Do not allow existing partitions to be moved or resized so that
   they reside entirely beyond the 4 Gigabyte boundary, or reach beyond
   that boundary.

These limitations would make it less likely that the AmigaOS 2.x/3.x
default ROM file system would get itself into trouble, and the same goes
for the various mass storage driver RDB parser implementations. Note
that these limitations are just precautions baked into policy and the
user should be able to override and disable them.

I have no idea how this would work within the partitioning framework
used by Linux, though, which I expect is geared towards giving the user
maximum control rather than "training wheels".

Maximum control implies awareness of the consequences of choosing a
partition layout which requires specific file system and mass storage
driver capabilities, so while a warning message "device needs 64 bit
disk device support in native OS" is helpful, it may be better to say
that the layout is not compatible with what the Amiga ROM operating
system version 1.x/2.x/3.x can safely support.
=2D--------------------------------------------------------------

Amiga ROM operating system means the Kickstart ROM. AmigaOS 3.5
and 3.9 patch the Kickstart ROM and replace the default device driver
"scsi.device" while doing so.

Aside from that I=B4d say that this pretty much resembles the result of
our discussion: Be cautious in the partitioning tools, but permissive
in the RDB parser.

I think I ask him about limitations in Amiga Fast Filesystem as well.
Maybe he can give me something to quote about that as well.
=20
> This bug was reported originally in 2012, and the fix was created by
> the RDB author, Joanne Dow <jdow@earthlink.net>. A patch had been
> discussed and reviewed on linux-m68k at that time but never officially
> submitted. This patch adds additional error checking and warning messages.
>=20
> Fixes: https://bugzilla.kernel.org/show_bug.cgi?id=3D43511
> Reported-by: Martin Steigerwald <Martin@lichtvoll.de>
> Message-ID: <201206192146.09327.Martin@lichtvoll.de>
> Signed-off-by: Michael Schmitz <schmitzmic@gmail.com>
> Tested-by: Martin Steigerwald <Martin@lichtvoll.de>
>=20
> Changes from RFC:
>=20
> - use u64 instead of sector_t, since that may be u32 without LBD support
> - check multiplication overflows each step - 3 u32 values may exceed u64!
> - warn against use on AmigaDOS if partition data overflow u32 sector coun=
t.
> - warn if partition CylBlocks larger than what's stored in the RDSK heade=
r.
> - bail out if we were to overflow sector_t (32 or 64 bit).
> ---
>  block/partitions/amiga.c |   99 ++++++++++++++++++++++++++++++++++++++++=
+++++-
>  1 files changed, 98 insertions(+), 1 deletions(-)
>=20
> diff --git a/block/partitions/amiga.c b/block/partitions/amiga.c
> index 5609366..6184d83 100644
> --- a/block/partitions/amiga.c
> +++ b/block/partitions/amiga.c
> @@ -11,6 +11,7 @@
>  #define pr_fmt(fmt) fmt
> =20
>  #include <linux/types.h>
> +#include <linux/log2.h>
>  #include <linux/affs_hardblocks.h>
> =20
>  #include "check.h"
> @@ -32,7 +33,9 @@ int amiga_partition(struct parsed_partitions *state)
>  	unsigned char *data;
>  	struct RigidDiskBlock *rdb;
>  	struct PartitionBlock *pb;
> -	int start_sect, nr_sects, blk, part, res =3D 0;
> +	u64 start_sect, nr_sects;
> +	u64 cylblk, cylblk_res;	/* rdb_CylBlocks =3D nr_heads*sect_per_track */
> +	int blk, part, res =3D 0, blk_shift =3D 0, did_warn =3D 0;
>  	int blksize =3D 1;	/* Multiplier for disk block size */
>  	int slot =3D 1;
>  	char b[BDEVNAME_SIZE];
> @@ -98,6 +101,79 @@ int amiga_partition(struct parsed_partitions *state)
>  		if (checksum_block((__be32 *)pb, be32_to_cpu(pb->pb_SummedLongs) & 0x7=
=46) !=3D 0 )
>  			continue;
> =20
> +		/* RDB gives us more than enough rope to hang ourselves with,
> +		 * many times over (2^128 bytes if all fields max out).
> +		 * Some careful checks are in order.
> +		 */
> +
> +		/* CylBlocks is total number of blocks per cylinder */
> +		cylblk =3D be32_to_cpu(pb->pb_Environment[3]) *
> +			 be32_to_cpu(pb->pb_Environment[5]);

I agree Geert about using constants there.

> +
> +		/* check for consistency with RDB defined CylBlocks */
> +		if (cylblk > be32_to_cpu(rdb->rdb_CylBlocks)) {
> +			pr_err("Dev %s: cylblk 0x%lx > rdb_CylBlocks 0x%x!\n",
> +				bdevname(state->bdev, b),
> +				(unsigned long) cylblk,
> +				be32_to_cpu(rdb->rdb_CylBlocks));
> +		}
> +
> +		/* check for potential overflows - we are going to multiply
> +		 * three 32 bit numbers to one 64 bit result later!
> +		 * Condition 1: nr_heads * sects_per_track must fit u32!
> +		 * NB: This is a HARD limit for AmigaDOS. We don't care much.
> +		 */
> +
> +		if (cylblk > UINT_MAX) {
> +			pr_err("Dev %s: hds*sects 0x%lx > UINT_MAX!\n",
> +				bdevname(state->bdev, b),
> +				(unsigned long) cylblk);
> +
> +			/* lop off low 32 bits */
> +			cylblk_res =3D cylblk >> 32;
> +
> +			/* check for further overflow in end result */
> +			if (be32_to_cpu(pb->pb_Environment[9]) *
> +				cylblk_res * blksize > UINT_MAX) {
> +				pr_err("Dev %s: start_sect overflows u64!\n",
> +					bdevname(state->bdev, b));
> +				res =3D -1;
> +				goto rdb_done;
> +			}
> +
> +			if (be32_to_cpu(pb->pb_Environment[10]) *
> +			   cylblk_res * blksize > UINT_MAX) {
> +				pr_err("Dev %s: end_sect overflows u64!\n",
> +					bdevname(state->bdev, b));
> +				res =3D -1;
> +				goto rdb_done;
> +			}
> +		}
> +
> +		/* Condition 2: even if CylBlocks did not overflow, the end
> +		 * result must still fit u64!
> +		 */
> +
> +		/* how many bits above 32 in cylblk * blksize ? */
> +		if (cylblk*blksize > (u64) UINT_MAX)
> +			blk_shift =3D ilog2(cylblk*blksize) - 32;
> +
> +		if (be32_to_cpu(pb->pb_Environment[9])
> +			> (u64) UINT_MAX>>blk_shift) {
> +			pr_err("Dev %s: start_sect overflows u64!\n",
> +				bdevname(state->bdev, b));
> +			res =3D -1;
> +			goto rdb_done;
> +		}
> +
> +		if (be32_to_cpu(pb->pb_Environment[10])
> +			> (u64) UINT_MAX>>blk_shift) {
> +			pr_err("Dev %s: end_sect overflows u64!\n",
> +				bdevname(state->bdev, b));
> +			res =3D -1;
> +			goto rdb_done;
> +		}
> +
>  		/* Tell Kernel about it */
> =20
>  		nr_sects =3D (be32_to_cpu(pb->pb_Environment[10]) + 1 -
> @@ -111,6 +187,27 @@ int amiga_partition(struct parsed_partitions *state)
>  			     be32_to_cpu(pb->pb_Environment[3]) *
>  			     be32_to_cpu(pb->pb_Environment[5]) *
>  			     blksize;
> +
> +		/* Warn user if start_sect or nr_sects overflow u32 */
> +		if ((nr_sects > UINT_MAX || start_sect > UINT_MAX ||
> +		    (start_sect + nr_sects) > UINT_MAX) && !did_warn) {
> +			pr_err("Dev %s: partition 32 bit overflow! ",
> +				bdevname(state->bdev, b));
> +			pr_cont("start_sect 0x%llX, nr_sects 0x%llx\n",
> +				 start_sect, nr_sects);
> +			pr_err("Dev %s: partition may  need 64 bit ",

two spaces between "may" and "need"

> +				bdevname(state->bdev, b));
> +			pr_cont("device support on native AmigaOS!\n");

Well an option would be to use Olaf=B4s wording, like so:

partition not safely supported on Amiga ROM operating system version 1.x/2.=
x/3.x

I wonder about how to shorten this, well maybe:

partition not safely supported on Amiga ROM OS 1.x/2.x/3.x

Or:

partition not safely support on Amiga Kickstart ROM 1.x/2.x/3.x


Albeit you already decided to go with a more generic warning, which in
my point of view is also fine. Cause it is a bit challenging to word it in a
correct way when mentioning version numbers. Well this is what Olaf
suggested.

> +			/* Without LBD support, better bail out */
> +			if (!IS_ENABLED(CONFIG_LBDAF)) {
> +				pr_err("Dev %s: no LBD support, aborting!",
> +					bdevname(state->bdev, b));
> +				res =3D -1;
> +				goto rdb_done;
> +			}
> +			did_warn++;
> +		}
> +
>  		put_partition(state,slot++,start_sect,nr_sects);
>  		{
>  			/* Be even more informative to aid mounting */
>=20

Thanks,
=2D-=20
Martin

Re: [PATCH] block: fix Amiga partition support for disks >= 1 TB

[Martin Steigerwald]

Martin Steigerwald - 02.07.18, 21:36:
> Hi Michael.
>=20
> I felt free to Cc my mail address.

It was Cc=B4d. KMail truncated the Cc list with an option to expand it.

=2D-=20
Martin

Re: [PATCH] block: fix Amiga partition support for disks >= 1 TB

[Michael Schmitz]

Hi Kars,

thanks for your comments - will do.

Cheers,

  Michael


On Mon, Jul 2, 2018 at 6:38 PM, Kars de Jong <jongk@linux-m68k.org> wrote:
> Op ma 2 jul. 2018 om 07:29 schreef Michael Schmitz <schmitzmic@gmail.com>:
>> @@ -98,6 +101,79 @@ int amiga_partition(struct parsed_partitions *state)
>>                 if (checksum_block((__be32 *)pb, be32_to_cpu(pb->pb_SummedLongs) & 0x7F) != 0 )
>>                         continue;
>>
>> +               /* RDB gives us more than enough rope to hang ourselves with,
>> +                * many times over (2^128 bytes if all fields max out).
>> +                * Some careful checks are in order.
>> +                */
>> +
>> +               /* CylBlocks is total number of blocks per cylinder */
>> +               cylblk = be32_to_cpu(pb->pb_Environment[3]) *
>> +                        be32_to_cpu(pb->pb_Environment[5]);
>> +
>
> Could you please create #defines for all these magic offsets in pb_Environment?
> Below are a few more.
> This makes the code much more readable.
> Thanks!
>
>> +               /* check for consistency with RDB defined CylBlocks */
>> +               if (cylblk > be32_to_cpu(rdb->rdb_CylBlocks)) {
>> +                       pr_err("Dev %s: cylblk 0x%lx > rdb_CylBlocks 0x%x!\n",
>> +                               bdevname(state->bdev, b),
>> +                               (unsigned long) cylblk,
>> +                               be32_to_cpu(rdb->rdb_CylBlocks));
>> +               }
>> +
>> +               /* check for potential overflows - we are going to multiply
>> +                * three 32 bit numbers to one 64 bit result later!
>> +                * Condition 1: nr_heads * sects_per_track must fit u32!
>> +                * NB: This is a HARD limit for AmigaDOS. We don't care much.
>> +                */
>> +
>> +               if (cylblk > UINT_MAX) {
>> +                       pr_err("Dev %s: hds*sects 0x%lx > UINT_MAX!\n",
>> +                               bdevname(state->bdev, b),
>> +                               (unsigned long) cylblk);
>> +
>> +                       /* lop off low 32 bits */
>> +                       cylblk_res = cylblk >> 32;
>> +
>> +                       /* check for further overflow in end result */
>> +                       if (be32_to_cpu(pb->pb_Environment[9]) *
>> +                               cylblk_res * blksize > UINT_MAX) {
>> +                               pr_err("Dev %s: start_sect overflows u64!\n",
>> +                                       bdevname(state->bdev, b));
>> +                               res = -1;
>> +                               goto rdb_done;
>> +                       }
>> +
>> +                       if (be32_to_cpu(pb->pb_Environment[10]) *
>> +                          cylblk_res * blksize > UINT_MAX) {
>> +                               pr_err("Dev %s: end_sect overflows u64!\n",
>> +                                       bdevname(state->bdev, b));
>> +                               res = -1;
>> +                               goto rdb_done;
>> +                       }
>> +               }
>> +
>> +               /* Condition 2: even if CylBlocks did not overflow, the end
>> +                * result must still fit u64!
>> +                */
>> +
>> +               /* how many bits above 32 in cylblk * blksize ? */
>> +               if (cylblk*blksize > (u64) UINT_MAX)
>> +                       blk_shift = ilog2(cylblk*blksize) - 32;
>> +
>> +               if (be32_to_cpu(pb->pb_Environment[9])
>> +                       > (u64) UINT_MAX>>blk_shift) {
>> +                       pr_err("Dev %s: start_sect overflows u64!\n",
>> +                               bdevname(state->bdev, b));
>> +                       res = -1;
>> +                       goto rdb_done;
>> +               }
>> +
>> +               if (be32_to_cpu(pb->pb_Environment[10])
>> +                       > (u64) UINT_MAX>>blk_shift) {
>> +                       pr_err("Dev %s: end_sect overflows u64!\n",
>> +                               bdevname(state->bdev, b));
>> +                       res = -1;
>> +                       goto rdb_done;
>> +               }
>> +
>>                 /* Tell Kernel about it */
>>
>>                 nr_sects = (be32_to_cpu(pb->pb_Environment[10]) + 1 -

Re: [PATCH] block: fix Amiga partition support for disks >= 1 TB

[Michael Schmitz]

Hi Geert,

thanks for your comments!

On Mon, Jul 2, 2018 at 8:29 PM, Geert Uytterhoeven <geert@linux-m68k.org> wrote:
>
>> +               /* CylBlocks is total number of blocks per cylinder */
>> +               cylblk = be32_to_cpu(pb->pb_Environment[3]) *
>> +                        be32_to_cpu(pb->pb_Environment[5]);
>
> Does the above really do a 32 * 32 = 64 bit multiplication?
> be32 is unsigned int, and multiplying it will be done in 32-bit arithmetic:
>
>     unsigned int a = 100000;
>     unsigned int b = 100000;
>     unsigned long long c = a * b;
>     unsigned long long d = (unsigned long long)a * b;
>     printf("c = %llu\n", c);
>     printf("d = %llu\n", d);
>
> prints:
>
>     c = 1410065408
>     d = 10000000000
>
> If it does work for you, what am I missing?

Not sure - I had begun to tease apart the assembly to answer that, but
got sidetracked. You may well be right that I'm still doing 32 bit
muls. The old parser used signed int which is why it overflowed above
1 TB. Haven't had the time to gin up an RDB exceeding 2 TB yet.

>
>> +
>> +               /* check for consistency with RDB defined CylBlocks */
>> +               if (cylblk > be32_to_cpu(rdb->rdb_CylBlocks)) {
>> +                       pr_err("Dev %s: cylblk 0x%lx > rdb_CylBlocks 0x%x!\n",
>> +                               bdevname(state->bdev, b),
>> +                               (unsigned long) cylblk,
>
> Why the cast? This will truncate the value on 32-bit platforms.
> Just use %llu (IMHO decimal is better suited here).

Will do that.

>> +                               be32_to_cpu(rdb->rdb_CylBlocks));
>> +               }
>> +
>> +               /* check for potential overflows - we are going to multiply
>> +                * three 32 bit numbers to one 64 bit result later!
>> +                * Condition 1: nr_heads * sects_per_track must fit u32!
>> +                * NB: This is a HARD limit for AmigaDOS. We don't care much.
>
> So, is condition 1 really needed?

Just an optimization for my overflow calculations ...

>> +                       /* lop off low 32 bits */
>> +                       cylblk_res = cylblk >> 32;
>> +
>> +                       /* check for further overflow in end result */
>> +                       if (be32_to_cpu(pb->pb_Environment[9]) *
>> +                               cylblk_res * blksize > UINT_MAX) {
>> +                               pr_err("Dev %s: start_sect overflows u64!\n",
>> +                                       bdevname(state->bdev, b));
>> +                               res = -1;
>> +                               goto rdb_done;
>> +                       }
>> +
>> +                       if (be32_to_cpu(pb->pb_Environment[10]) *
>> +                          cylblk_res * blksize > UINT_MAX) {
>> +                               pr_err("Dev %s: end_sect overflows u64!\n",
>> +                                       bdevname(state->bdev, b));
>> +                               res = -1;
>> +                               goto rdb_done;
>> +                       }
>
> No need to reinvent the wheel, #include <linux/overflow.h>, and use
> check_mul_overflow(), like array3_size() does.

Thanks, I knew I was missing something there.


>> +               }
>> +
>>                 /* Tell Kernel about it */
>>
>>                 nr_sects = (be32_to_cpu(pb->pb_Environment[10]) + 1 -
>> @@ -111,6 +187,27 @@ int amiga_partition(struct parsed_partitions *state)
>>                              be32_to_cpu(pb->pb_Environment[3]) *
>>                              be32_to_cpu(pb->pb_Environment[5]) *
>>                              blksize;
>
> I'm still not convinced the above is done in 64-bit arithmetic...

I hear you ...

>> +
>> +               /* Warn user if start_sect or nr_sects overflow u32 */
>> +               if ((nr_sects > UINT_MAX || start_sect > UINT_MAX ||
>> +                   (start_sect + nr_sects) > UINT_MAX) && !did_warn) {
>
> I guess "start_sect + nr_sects > UINT_MAX" is sufficient?

No, we need to catch any partition address overflowing. nr_sects >
UINT_MAX may be redundant though.

> I would remove the did_warn check, as multiple partitions may be affected.

Any partition overflowing means danger lurks (in AmigaDOS of
sufficient vintage, that is)

> Also, RDB doesn't enforce partition ordering, IIRC, so e.g. partitions 1
> and 3 could be outside the 2 TiB area, while 2 could be within.

The first partition (partly) outside 2 TB will warn. But the point
about partition ordering later is well taken.
>
>> +                       pr_err("Dev %s: partition 32 bit overflow! ",
>
> pr_warn()

OK.

>
>> +                               bdevname(state->bdev, b));
>> +                       pr_cont("start_sect 0x%llX, nr_sects 0x%llx\n",
>> +                                start_sect, nr_sects);
>
> No need for pr_cont(), just merge the two statements.

Checkpatch catch-22 (thou shalt not exceed 80 cols, thou shalt not
split string consts over multiple lines, and thou shalt not use
pr_cont() without good cause). I'll ignore the 80 cols error then.

>
>> +                       pr_err("Dev %s: partition may  need 64 bit ",
>
> pr_warn()
>
> Drop the "may"?

s/may/will/ ...

>
>> +                               bdevname(state->bdev, b));
>
> I would print the partition index, too.
>
>> +                       pr_cont("device support on native AmigaOS!\n");
>
> Just merge the two statements.
>
> I think it can also be just one line printed instead of 2?

Can do, yes.

>
>     pr_warn("Dev %s: partition %u (%llu-%llu) needs 64-bit device
> support\n", ...
>
>> +                       /* Without LBD support, better bail out */
>> +                       if (!IS_ENABLED(CONFIG_LBDAF)) {
>> +                               pr_err("Dev %s: no LBD support, aborting!",
>
>     pr_err("Dev %s: no LBD support, skipping partition %u\n", ...)?
>
>> +                                       bdevname(state->bdev, b));
>> +                               res = -1;
>> +                               goto rdb_done;
>
> Aborting here renders any later partitions within the 2 TiB area unaccessible.
> So please continue.

Right. I wasn't aware of the funky ordering feature (only seen it on
MacOS before, and I have always held that's as weird as it gets).

I'll try to get a new version out before tomorrow.

Cheers,


>
>> +                       }
>> +                       did_warn++;
>> +               }
>> +
>>                 put_partition(state,slot++,start_sect,nr_sects);
>>                 {
>>                         /* Be even more informative to aid mounting */
>
> Gr{oetje,eeting}s,
>
>                         Geert
>
> --
> Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
>
> In personal conversations with technical people, I call myself a hacker. But
> when I'm talking to journalists I just say "programmer" or something like that.
>                                 -- Linus Torvalds

[PATCH v2] block: fix Amiga partition support for disks >= 1 TB

[Michael Schmitz]

The Amiga partition parser module uses signed int for partition sector
address and count, which will overflow for disks larger than 1 TB.

Use u64 as type for sector address and size to allow using disks up to
2 TB without LBD support, and disks larger than 2 TB with LBD. The RBD
format allows to specify disk sizes up to 2^128 bytes (though native
OS limitations reduce this somewhat, to max 2^68 bytes), so check for
u64 overflow carefully to protect against overflowing sector_t.

Bail out if sector addresses overflow 32 bits on kernels without LBD
support.

This bug was reported originally in 2012, and the fix was created by
the RDB author, Joanne Dow <jdow@earthlink.net>. A patch had been
discussed and reviewed on linux-m68k at that time but never officially
submitted. This patch adds additional error checking and warning messages.

Fixes: https://bugzilla.kernel.org/show_bug.cgi?id=43511
Reported-by: Martin Steigerwald <Martin@lichtvoll.de>
Message-ID: <201206192146.09327.Martin@lichtvoll.de>
Signed-off-by: Michael Schmitz <schmitzmic@gmail.com>
Tested-by: Martin Steigerwald <Martin@lichtvoll.de>

---

Changes from RFC:

- use u64 instead of sector_t, since that may be u32 without LBD support
- check multiplication overflows each step - 3 u32 values may exceed u64!
- warn against use on AmigaDOS if partition data overflow u32 sector count.
- warn if partition CylBlocks larger than what's stored in the RDSK header.
- bail out if we were to overflow sector_t (32 or 64 bit).

Changes from v1:

Kars de Jong:
- use defines for magic offsets in DosEnVec struct

Geert Uytterhoeven:
- use u64 cast for multiplications of u32 numbers
- use array3_size for overflow checks
- change pr_err to pr_warn
- discontinue use of pr_cont
- reword log messages
- drop redundant nr_sects overflow test
- warn against 32 bit overflow for each affected partition
- skip partitions that overflow sector_t size instead of aborting scan
---
 block/partitions/amiga.c |   91 +++++++++++++++++++++++++++++++++++++++++-----
 1 files changed, 81 insertions(+), 10 deletions(-)

diff --git a/block/partitions/amiga.c b/block/partitions/amiga.c
index 5609366..7277ebb 100644
--- a/block/partitions/amiga.c
+++ b/block/partitions/amiga.c
@@ -11,11 +11,20 @@
 #define pr_fmt(fmt) fmt
 
 #include <linux/types.h>
+#include <linux/types.h>
+#include <linux/mm_types.h>
+#include <linux/overflow.h>
 #include <linux/affs_hardblocks.h>
 
 #include "check.h"
 #include "amiga.h"
 
+/* magic offsets in partition DosEnvVec */
+#define NR_HD	3
+#define NR_SECT	5
+#define LO_CYL	9
+#define	HI_CYL	10
+
 static __inline__ u32
 checksum_block(__be32 *m, int size)
 {
@@ -32,7 +41,9 @@ int amiga_partition(struct parsed_partitions *state)
 	unsigned char *data;
 	struct RigidDiskBlock *rdb;
 	struct PartitionBlock *pb;
-	int start_sect, nr_sects, blk, part, res = 0;
+	u64 start_sect, nr_sects;
+	u64 cylblk;		/* rdb_CylBlocks = nr_heads*sect_per_track */
+	int blk, part, res = 0;
 	int blksize = 1;	/* Multiplier for disk block size */
 	int slot = 1;
 	char b[BDEVNAME_SIZE];
@@ -98,19 +109,79 @@ int amiga_partition(struct parsed_partitions *state)
 		if (checksum_block((__be32 *)pb, be32_to_cpu(pb->pb_SummedLongs) & 0x7F) != 0 )
 			continue;
 
+		/* RDB gives us more than enough rope to hang ourselves with,
+		 * many times over (2^128 bytes if all fields max out).
+		 * Some careful checks are in order.
+		 */
+
+		/* CylBlocks is total number of blocks per cylinder */
+		cylblk = be32_to_cpu(pb->pb_Environment[NR_HD]) *
+			 be32_to_cpu(pb->pb_Environment[NR_SECT]);
+
+		/* check for consistency with RDB defined CylBlocks */
+		if (cylblk > be32_to_cpu(rdb->rdb_CylBlocks)) {
+			pr_warn("Dev %s: cylblk %llu > rdb_CylBlocks 0x%x!\n",
+				bdevname(state->bdev, b), cylblk,
+				be32_to_cpu(rdb->rdb_CylBlocks));
+		}
+
+		/* check for potential overflows - we are going to multiply
+		 * three 32 bit numbers to one 64 bit result later!
+		 * Condition 1: nr_heads * sects_per_track must fit u32!
+		 * NB: This is a HARD limit for AmigaDOS. We just warn.
+		 */
+
+		if (cylblk > UINT_MAX) {
+			pr_warn("Dev %s: heads*sects %llu > UINT_MAX!\n",
+				bdevname(state->bdev, b), cylblk);
+		}
+
+		/* Condition 2: even if CylBlocks did not overflow, the end
+		 * result must still fit u64!
+		 */
+
+		if (array3_size(be32_to_cpu(pb->pb_Environment[LO_CYL]),
+			cylblk, blksize) == SIZE_MAX) {
+			pr_err("Dev %s: partition %u start overflows u64, skipping partition!\n",
+				bdevname(state->bdev, b), part);
+			continue;
+		}
+
+		if (array3_size(be32_to_cpu(pb->pb_Environment[HI_CYL]),
+			cylblk, blksize) == SIZE_MAX) {
+			pr_err("Dev %s: partition %u end overflows u64, skipping partition!\n",
+				bdevname(state->bdev, b), part);
+			continue;
+		}
+
 		/* Tell Kernel about it */
 
-		nr_sects = (be32_to_cpu(pb->pb_Environment[10]) + 1 -
-			    be32_to_cpu(pb->pb_Environment[9])) *
-			   be32_to_cpu(pb->pb_Environment[3]) *
-			   be32_to_cpu(pb->pb_Environment[5]) *
-			   blksize;
+		nr_sects = (u64)((be32_to_cpu(pb->pb_Environment[HI_CYL]) + 1 -
+			    be32_to_cpu(pb->pb_Environment[LO_CYL])) *
+			   be32_to_cpu(pb->pb_Environment[NR_HD]) *
+			   be32_to_cpu(pb->pb_Environment[NR_SECT]) *
+			   blksize);
 		if (!nr_sects)
 			continue;
-		start_sect = be32_to_cpu(pb->pb_Environment[9]) *
-			     be32_to_cpu(pb->pb_Environment[3]) *
-			     be32_to_cpu(pb->pb_Environment[5]) *
-			     blksize;
+		start_sect = (u64)(be32_to_cpu(pb->pb_Environment[LO_CYL]) *
+			     be32_to_cpu(pb->pb_Environment[NR_HD]) *
+			     be32_to_cpu(pb->pb_Environment[NR_SECT]) *
+			     blksize);
+
+		/* Warn user if start_sect or nr_sects overflow u32 */
+		if (start_sect > UINT_MAX ||
+		    (start_sect + nr_sects) > UINT_MAX) {
+			pr_warn("Dev %s: partition %u (%llu-%llu) needs 64 bit device support!\n",
+				bdevname(state->bdev, b), part,
+				start_sect, nr_sects);
+			/* Without LBD support, better bail out */
+			if (!IS_ENABLED(CONFIG_LBDAF)) {
+				pr_err("Dev %s: no LBD support, skipping partition %u!",
+					bdevname(state->bdev, b), part);
+				continue;
+			}
+		}
+
 		put_partition(state,slot++,start_sect,nr_sects);
 		{
 			/* Be even more informative to aid mounting */
-- 
1.7.0.4

Re: [PATCH] block: fix Amiga partition support for disks >= 1 TB

[Geert Uytterhoeven]

Hi Michael,

On Tue, Jul 3, 2018 at 1:58 AM Michael Schmitz <schmitzmic@gmail.com> wrote:
> On Mon, Jul 2, 2018 at 8:29 PM, Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> >> +
> >> +               /* Warn user if start_sect or nr_sects overflow u32 */
> >> +               if ((nr_sects > UINT_MAX || start_sect > UINT_MAX ||
> >> +                   (start_sect + nr_sects) > UINT_MAX) && !did_warn) {
> >
> > I guess "start_sect + nr_sects > UINT_MAX" is sufficient?
>
> No, we need to catch any partition address overflowing. nr_sects >
> UINT_MAX may be redundant though.

The three tests may have been needed when both variables were 32-bit,
but when using unsigned 64-bit arithmetic, it shouldn't matter: if any of the
two values doesn't fit in 32-bit, the sum also doesn't.
Or is this also used to catch 64-bit add overflow? In that case, please use
check_add_overflow().

> > I would remove the did_warn check, as multiple partitions may be affected.
>
> Any partition overflowing means danger lurks (in AmigaDOS of
> sufficient vintage, that is)

Yes, that's what I meant: dropping the did_warn check means always printing
the warning, not just for the first "unusable" partition.

> > Also, RDB doesn't enforce partition ordering, IIRC, so e.g. partitions 1
> > and 3 could be outside the 2 TiB area, while 2 could be within.
>
> The first partition (partly) outside 2 TB will warn. But the point
> about partition ordering later is well taken.
> >
> >> +                       pr_err("Dev %s: partition 32 bit overflow! ",
> >
> > pr_warn()
>
> OK.
>
> >
> >> +                               bdevname(state->bdev, b));
> >> +                       pr_cont("start_sect 0x%llX, nr_sects 0x%llx\n",
> >> +                                start_sect, nr_sects);
> >
> > No need for pr_cont(), just merge the two statements.
>
> Checkpatch catch-22 (thou shalt not exceed 80 cols, thou shalt not
> split string consts over multiple lines, and thou shalt not use
> pr_cont() without good cause). I'll ignore the 80 cols error then.

That's the sane thing to do: single pr_warn() statement, and ignoring the 80
columns error if it would mean splitting the string, so people can easily
grep for it when they see the message on their consoles.

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [PATCH] block: fix Amiga partition support for disks >= 1 TB

[Michael Schmitz]

Hi Geert,

Am 03.07.2018 um 19:22 schrieb Geert Uytterhoeven:
> Hi Michael,
>
> On Tue, Jul 3, 2018 at 1:58 AM Michael Schmitz <schmitzmic@gmail.com> wrote:
>> On Mon, Jul 2, 2018 at 8:29 PM, Geert Uytterhoeven <geert@linux-m68k.org> wrote:
>>>> +
>>>> +               /* Warn user if start_sect or nr_sects overflow u32 */
>>>> +               if ((nr_sects > UINT_MAX || start_sect > UINT_MAX ||
>>>> +                   (start_sect + nr_sects) > UINT_MAX) && !did_warn) {
>>>
>>> I guess "start_sect + nr_sects > UINT_MAX" is sufficient?
>>
>> No, we need to catch any partition address overflowing. nr_sects >
>> UINT_MAX may be redundant though.
>
> The three tests may have been needed when both variables were 32-bit,
> but when using unsigned 64-bit arithmetic, it shouldn't matter: if any of the
> two values doesn't fit in 32-bit, the sum also doesn't.

Right, got it now. Too late for v2 of the patch ...

I just see I've messed up some other things (duplicate types.h 
inclusion) - I'll have to respin anyway. I'll hold off on that waiting 
for futher feedback for now.

> Or is this also used to catch 64-bit add overflow? In that case, please use
> check_add_overflow().

No, the start sector is directly calculated from what's stored in the 
DosEnvVec struct (or whatever the precise name was), as product of four 
32 bit numbers. nr_sects is calculated from the 32 bit cylinder address 
difference and the other three 32 bit numbers, no addition overflow there.

>>> I would remove the did_warn check, as multiple partitions may be affected.
>>
>> Any partition overflowing means danger lurks (in AmigaDOS of
>> sufficient vintage, that is)
>
> Yes, that's what I meant: dropping the did_warn check means always printing
> the warning, not just for the first "unusable" partition.

Yes, I've done that now.

>>> No need for pr_cont(), just merge the two statements.
>>
>> Checkpatch catch-22 (thou shalt not exceed 80 cols, thou shalt not
>> split string consts over multiple lines, and thou shalt not use
>> pr_cont() without good cause). I'll ignore the 80 cols error then.
>
> That's the sane thing to do: single pr_warn() statement, and ignoring the 80
> columns error if it would mean splitting the string, so people can easily
> grep for it when they see the message on their consoles.

True - and there isn't a checkpatch error for overlong format strings 
these days (there might have been at some time, which I remembered).

Thanks again!

Cheers,

	Michael

>
> Gr{oetje,eeting}s,
>
>                         Geert
>

Re: [PATCH] block: fix Amiga partition support for disks >= 1 TB

[jdow]

On 20180703 00:22, Geert Uytterhoeven wrote:
> Hi Michael,
> 
> On Tue, Jul 3, 2018 at 1:58 AM Michael Schmitz <schmitzmic@gmail.com> wrote:
>> On Mon, Jul 2, 2018 at 8:29 PM, Geert Uytterhoeven <geert@linux-m68k.org> wrote:
>>>> +
>>>> +               /* Warn user if start_sect or nr_sects overflow u32 */
>>>> +               if ((nr_sects > UINT_MAX || start_sect > UINT_MAX ||
>>>> +                   (start_sect + nr_sects) > UINT_MAX) && !did_warn) {
>>>
>>> I guess "start_sect + nr_sects > UINT_MAX" is sufficient?
>>
>> No, we need to catch any partition address overflowing. nr_sects >
>> UINT_MAX may be redundant though.
> 
> The three tests may have been needed when both variables were 32-bit,
> but when using unsigned 64-bit arithmetic, it shouldn't matter: if any of the
> two values doesn't fit in 32-bit, the sum also doesn't.
> Or is this also used to catch 64-bit add overflow? In that case, please use
> check_add_overflow().

Um, both values can be high 32 bits with the sum being greater than 32 bits. 
Seems to me, though, that the only test needed is whether the sum is greater 
than 32 bits. If either number is greater the sum certainly is. (Of course, 
either a comment to that effect or code that pounds the concept into the code's 
future reader should appear.) The 2TB issue is one that may surprise old hands 
given the appearance of the RDBs being able to support absurdly large, NSA 
pleasing, disks. It's hard to decide where warnings should appear. (And it's 
difficult to justify them in the RDB parser unless it overflows the interface to 
the Linux OS. The mkfs-(RDB reading Amiga filesystem) took should be where the 
warnings are posted.

>>> I would remove the did_warn check, as multiple partitions may be affected.
>>
>> Any partition overflowing means danger lurks (in AmigaDOS of
>> sufficient vintage, that is)
> 
> Yes, that's what I meant: dropping the did_warn check means always printing
> the warning, not just for the first "unusable" partition.

Actually anything greater than 2 gigabytes may be a problem on some versions of 
AmigaDOS that spoke to hard disks of one kind or another. A newcomer to the 
surprisingly active Amiga scene could get distressed by that error. Old timers 
know of the fixes that circulated. Those fixes appear to work with greater than 
2TB disks to some limited degree. There are probably some official "Commodore" 
OS utilities which will blow their little minds at 2TB if not sooner. That's one 
that can be protected for in the partitioning tool. The RDB parser should warn 
if it gets confused or overflows its interface into the OS. Ideally it should 
back out of mounting partitions it does not understand.

Note that the original RDB parser patch I made was part of patch to allow 2k 
physical block size to match some 640 megabyte Fujitsu magneto-optical drives. I 
hope the OS can still cope with that situation. (I don't know if I can recover 
the actual data written to the disk. I can't remember if I limited the LSEG 
data, which Linux should ignore anyway, to 512 bytes or not when I built the low 
level partitioning tools. So LONG ago and a mind with a 7 in front of its age 
gets tad flaky remembering such ancient history.)

>>> Also, RDB doesn't enforce partition ordering, IIRC, so e.g. partitions 1
>>> and 3 could be outside the 2 TiB area, while 2 could be within.
>>
>> The first partition (partly) outside 2 TB will warn. But the point
>> about partition ordering later is well taken.
>>>
>>>> +                       pr_err("Dev %s: partition 32 bit overflow! ",
>>>
>>> pr_warn()
>>
>> OK.
>>
>>>
>>>> +                               bdevname(state->bdev, b));
>>>> +                       pr_cont("start_sect 0x%llX, nr_sects 0x%llx\n",
>>>> +                                start_sect, nr_sects);
>>>
>>> No need for pr_cont(), just merge the two statements.
>>
>> Checkpatch catch-22 (thou shalt not exceed 80 cols, thou shalt not
>> split string consts over multiple lines, and thou shalt not use
>> pr_cont() without good cause). I'll ignore the 80 cols error then.
> 
> That's the sane thing to do: single pr_warn() statement, and ignoring the 80
> columns error if it would mean splitting the string, so people can easily
> grep for it when they see the message on their consoles.
> 
> Gr{oetje,eeting}s,
> 
>                          Geert

And thank you for your work with the Linux community, Geert. I've noticed and 
admired it.

{^_^}   Joanne Dow (Her just cruseing face.)

[PATCH v3] block: fix Amiga partition support for disks >= 1 TB

[Michael Schmitz]

The Amiga partition parser module uses signed int for partition sector
address and count, which will overflow for disks larger than 1 TB.

Use u64 as type for sector address and size to allow using disks up to
2 TB without LBD support, and disks larger than 2 TB with LBD. The RBD
format allows to specify disk sizes up to 2^128 bytes (though native
OS limitations reduce this somewhat, to max 2^68 bytes), so check for
u64 overflow carefully to protect against overflowing sector_t.

Bail out if sector addresses overflow 32 bits on kernels without LBD
support.

This bug was reported originally in 2012, and the fix was created by
the RDB author, Joanne Dow <jdow@earthlink.net>. A patch had been
discussed and reviewed on linux-m68k at that time but never officially
submitted. This patch adds additional error checking and warning messages.

Fixes: https://bugzilla.kernel.org/show_bug.cgi?id=43511
Reported-by: Martin Steigerwald <Martin@lichtvoll.de>
Message-ID: <201206192146.09327.Martin@lichtvoll.de>
Signed-off-by: Michael Schmitz <schmitzmic@gmail.com>
Tested-by: Martin Steigerwald <Martin@lichtvoll.de>

---

Changes from RFC:

- use u64 instead of sector_t, since that may be u32 without LBD support
- check multiplication overflows each step - 3 u32 values may exceed u64!
- warn against use on AmigaDOS if partition data overflow u32 sector count.
- warn if partition CylBlocks larger than what's stored in the RDSK header.
- bail out if we were to overflow sector_t (32 or 64 bit).

Changes from v1:

Kars de Jong:
- use defines for magic offsets in DosEnvec struct

Geert Uytterhoeven:
- use u64 cast for multiplications of u32 numbers
- use array3_size for overflow checks
- change pr_err to pr_warn
- discontinue use of pr_cont
- reword log messages
- drop redundant nr_sects overflow test
- warn against 32 bit overflow for each affected partition
- skip partitions that overflow sector_t size instead of aborting scan

Changes from v2:

- further trim 32 bit overflow test
- correct duplicate types.h inclusion introduced in v2

---
 block/partitions/amiga.c |   89 ++++++++++++++++++++++++++++++++++++++++-----
 1 files changed, 79 insertions(+), 10 deletions(-)

diff --git a/block/partitions/amiga.c b/block/partitions/amiga.c
index 5609366..4e6e478 100644
--- a/block/partitions/amiga.c
+++ b/block/partitions/amiga.c
@@ -11,11 +11,19 @@
 #define pr_fmt(fmt) fmt
 
 #include <linux/types.h>
+#include <linux/mm_types.h>
+#include <linux/overflow.h>
 #include <linux/affs_hardblocks.h>
 
 #include "check.h"
 #include "amiga.h"
 
+/* magic offsets in partition DosEnvVec */
+#define NR_HD	3
+#define NR_SECT	5
+#define LO_CYL	9
+#define	HI_CYL	10
+
 static __inline__ u32
 checksum_block(__be32 *m, int size)
 {
@@ -32,7 +40,9 @@ int amiga_partition(struct parsed_partitions *state)
 	unsigned char *data;
 	struct RigidDiskBlock *rdb;
 	struct PartitionBlock *pb;
-	int start_sect, nr_sects, blk, part, res = 0;
+	u64 start_sect, nr_sects;
+	u64 cylblk;		/* rdb_CylBlocks = nr_heads*sect_per_track */
+	int blk, part, res = 0;
 	int blksize = 1;	/* Multiplier for disk block size */
 	int slot = 1;
 	char b[BDEVNAME_SIZE];
@@ -98,19 +108,78 @@ int amiga_partition(struct parsed_partitions *state)
 		if (checksum_block((__be32 *)pb, be32_to_cpu(pb->pb_SummedLongs) & 0x7F) != 0 )
 			continue;
 
+		/* RDB gives us more than enough rope to hang ourselves with,
+		 * many times over (2^128 bytes if all fields max out).
+		 * Some careful checks are in order.
+		 */
+
+		/* CylBlocks is total number of blocks per cylinder */
+		cylblk = be32_to_cpu(pb->pb_Environment[NR_HD]) *
+			 be32_to_cpu(pb->pb_Environment[NR_SECT]);
+
+		/* check for consistency with RDB defined CylBlocks */
+		if (cylblk > be32_to_cpu(rdb->rdb_CylBlocks)) {
+			pr_warn("Dev %s: cylblk %llu > rdb_CylBlocks 0x%x!\n",
+				bdevname(state->bdev, b), cylblk,
+				be32_to_cpu(rdb->rdb_CylBlocks));
+		}
+
+		/* check for potential overflows - we are going to multiply
+		 * three 32 bit numbers to one 64 bit result later!
+		 * Condition 1: nr_heads * sects_per_track must fit u32!
+		 * NB: This is a HARD limit for AmigaDOS. We just warn.
+		 */
+
+		if (cylblk > UINT_MAX) {
+			pr_warn("Dev %s: heads*sects %llu > UINT_MAX!\n",
+				bdevname(state->bdev, b), cylblk);
+		}
+
+		/* Condition 2: even if CylBlocks did not overflow, the end
+		 * result must still fit u64!
+		 */
+
+		if (array3_size(be32_to_cpu(pb->pb_Environment[LO_CYL]),
+			cylblk, blksize) == SIZE_MAX) {
+			pr_err("Dev %s: partition %u start overflows u64, skipping partition!\n",
+				bdevname(state->bdev, b), part);
+			continue;
+		}
+
+		if (array3_size(be32_to_cpu(pb->pb_Environment[HI_CYL]),
+			cylblk, blksize) == SIZE_MAX) {
+			pr_err("Dev %s: partition %u end overflows u64, skipping partition!\n",
+				bdevname(state->bdev, b), part);
+			continue;
+		}
+
 		/* Tell Kernel about it */
 
-		nr_sects = (be32_to_cpu(pb->pb_Environment[10]) + 1 -
-			    be32_to_cpu(pb->pb_Environment[9])) *
-			   be32_to_cpu(pb->pb_Environment[3]) *
-			   be32_to_cpu(pb->pb_Environment[5]) *
-			   blksize;
+		nr_sects = (u64)((be32_to_cpu(pb->pb_Environment[HI_CYL]) + 1 -
+			    be32_to_cpu(pb->pb_Environment[LO_CYL])) *
+			   be32_to_cpu(pb->pb_Environment[NR_HD]) *
+			   be32_to_cpu(pb->pb_Environment[NR_SECT]) *
+			   blksize);
 		if (!nr_sects)
 			continue;
-		start_sect = be32_to_cpu(pb->pb_Environment[9]) *
-			     be32_to_cpu(pb->pb_Environment[3]) *
-			     be32_to_cpu(pb->pb_Environment[5]) *
-			     blksize;
+		start_sect = (u64)(be32_to_cpu(pb->pb_Environment[LO_CYL]) *
+			     be32_to_cpu(pb->pb_Environment[NR_HD]) *
+			     be32_to_cpu(pb->pb_Environment[NR_SECT]) *
+			     blksize);
+
+		/* Warn user if start_sect or nr_sects overflow u32 */
+		if ((start_sect + nr_sects) > UINT_MAX) {
+			pr_warn("Dev %s: partition %u (%llu-%llu) needs 64 bit device support!\n",
+				bdevname(state->bdev, b), part,
+				start_sect, nr_sects);
+			/* Without LBD support, better bail out */
+			if (!IS_ENABLED(CONFIG_LBDAF)) {
+				pr_err("Dev %s: no LBD support, skipping partition %u!",
+					bdevname(state->bdev, b), part);
+				continue;
+			}
+		}
+
 		put_partition(state,slot++,start_sect,nr_sects);
 		{
 			/* Be even more informative to aid mounting */
-- 
1.7.0.4