Linux-Fsdevel Archive on lore.kernel.org
 help / color / Atom feed
From: Michael Schmitz <schmitzmic@gmail.com>
To: jdow <jdow@earthlink.net>
Cc: Martin Steigerwald <martin@lichtvoll.de>,
	Geert Uytterhoeven <geert@linux-m68k.org>,
	Matthew Wilcox <willy@infradead.org>,
	David Sterba <dsterba@suse.cz>,
	Linux FS Devel <linux-fsdevel@vger.kernel.org>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	Jens Axboe <axboe@kernel.dk>,
	linux-m68k <linux-m68k@lists.linux-m68k.org>
Subject: Re: moving affs + RDB partition support to staging?
Date: Wed, 27 Jun 2018 13:07:42 +1200
Message-ID: <CAOmrzkKLJiWvDwsER0ca09UJbi57Rj6PUmdxizSNPXzZLMErOA@mail.gmail.com> (raw)
In-Reply-To: <8ef4bdc6-4ed0-675e-e26d-0b6e7ab4a00e@earthlink.net>

Joanne,

As far as I have been able to test, the change is backwards compatible
(RDB partitions from an old disk 80 GB disk are still recognized OK).
That"s only been done on an emulator though.

Your advice about the dangers of using RDB disks that would have
failed the current Linux RDB parser on legacy 32 bit systems is well
taken though. Maybe Martin can clarify that for me - was the 2 TB disk
in question ever used on a 32 bit Amiga system?

RDB disk format is meant for legacy use only, so I think we can get
away with printing a big fat warning during boot, advising the user
that the oversize RDB partition(s) scanned are not compatible with
legacy 32 bit AmigaOS. With the proposed fix they will work under both
AmigaOS 4.1 and Linux instead of truncating the first overflowing
partition at disk end and trashing valid partitions that overlap,
which is what Martin was after.

If that still seems too risky, we can make the default behaviour to
bail out once a potential overflow is detected, and allow the user to
override that through a boot parameter. I'd leave that decision up for
the code review on linux-block.

Two more comments: Linux uses 512 byte block sizes for the partition
start and size calculations, so a change of the RDB blocksize to
reduce the block counts stored in the RDB would still result in the
same overflow. And amiga-fdisk is indeed utterly broken and needs
updating (along with probably most legacy m68k partitioners). Adrian
has advertised parted as replacement for the old tools - maybe this
would make a nice test case for parted?

Cheers,

  Michael



On Tue, Jun 26, 2018 at 9:45 PM, jdow <jdow@earthlink.net> wrote:
> If it is not backwards compatible I for one would refuse to use it. And if
> it still mattered that much to me I'd also generate a reasonable
> alternative. Modifying RDBs nay not be even an approximation of a good idea.
> You'd discover that as soon as an RDB uint64_t disk is tasted by a uint32_t
> only system. If it is for your personal use then you're entirely free to
> reject my advice and are probably smart enough to keep it working for
> yourself.
>
> GPT is probably the right way to go. Preserve the ability to read RDBs for
> legacy disks only.
>
> {^_^}
>
>
> On 20180626 01:31, Michael Schmitz wrote:
>>
>> Joanne,
>>
>> I think we all agree that doing 32 bit calculations on 512-byte block
>> addresses that overflow on disks 2 TB and larger is a bug, causing the
>> issues Martin reported. Your patch addresses that by using the correct
>> data type for the calculations (as do other partition parsers that may
>> have to deal with large disks) and fixes Martin's bug, so appears to be
>> the right thing to do.
>>
>> Using 64 bit data types for disks smaller than 2 TB where calculations
>> don't currently overflow is not expected to cause new issues, other than
>> enabling use of disk and partitions larger than 2 TB (which may have
>> ramifications with filesystems on these partitions). So comptibility is
>> preserved.
>>
>> Forcing larger block sizes might be a good strategy to avoid overflow
>> issues in filesystems as well, but I can't see how the block size stored
>> in the RDB would enforce use of the same block size in filesystems.
>> We'll have to rely on the filesystem tools to get that right, too. Linux
>> AFFS does allow block sizes up to 4k (VFS limitation) so this should
>> allow partitions larger than 2 TB to work already (but I suspect Al Viro
>> may have found a few issues when he looked at the AFFS code so I won't
>> say more). Anyway partitioning tools and filesystems are unrelated to
>> the Linux partition parser code which is all we aim to fix in this patch.
>>
>> If you feel strongly about unknown ramifications of any filesystems on
>> partitions larger than 2 TB, say so and I'll have the kernel print a
>> warning about these partitions.
>>
>> I'll get this patch tested on Martin's test case image as well as on a
>> RDB image from a disk known to currently work under Linux (thanks Geert
>> for the losetup hint). Can't do much more without procuring a working
>> Amiga disk image to use with an emulator, sorry. The Amiga I plan to use
>> for tests is a long way away from my home indeed.
>>
>> Cheers,
>>
>>      Michael
>>
>> Am 26.06.18 um 17:17 schrieb jdow:
>>>
>>> As long as it preserves compatibility it should be OK, I suppose.
>>> Personally I'd make any partitioning tool front end gently force the
>>> block size towards 8k as the disk size gets larger. The file systems
>>> may also run into 2TB issues that are not obvious. An unused blocks
>>> list will have to go beyond a uint32_t size, for example. But a block
>>> list (OFS for sure, don't remember for the newer AFS) uses a tad under
>>> 1% of the disk all by itself. A block bitmap is not quite so bad. {^_-}
>>>
>>> Just be sure you are aware of all the ramifications when you make a
>>> change. I remember thinking about this for awhile and then determining
>>> I REALLY did not want to think about it as my brain was getting tied
>>> into a gordian knot.
>>>
>>> {^_^}
>>>
>>> On 20180625 19:23, Michael Schmitz wrote:
>>>>
>>>> Joanne,
>>>>
>>>> Martin's boot log (including your patch) says:
>>>>
>>>> Jun 19 21:19:09 merkaba kernel: [ 7891.843284]  sdb: RDSK (512) sdb1
>>>> (LNX^@)(res 2 spb 1) sdb2 (JXF^D)(res 2 spb 1) sdb3 (DOS^C)(res 2 spb
>>>> 4)
>>>> Jun 19 21:19:09 merkaba kernel: [ 7891.844055] sd 7:0:0:0: [sdb]
>>>> Attached SCSI disk
>>>>
>>>> so it's indeed a case of self inflicted damage (RDSK (512) means 512
>>>> byte blocks) and can be worked around by using a different block size.
>>>>
>>>> Your memory serves right indeed - blocksize is in 512 bytes units.
>>>> I'll still submit a patch to Jens anyway as this may bite others yet.
>>>>
>>>> Cheers,
>>>>
>>>>     Michael
>>>>
>>>>
>>>> On Sun, Jun 24, 2018 at 11:40 PM, jdow <jdow@earthlink.net> wrote:
>>>>>
>>>>> BTW - anybody who uses 512 byte blocks with an Amiga file system is
>>>>> a famn
>>>>> dool.
>>>>>
>>>>> If memory serves the RDBs think in blocks rather than bytes so it
>>>>> should
>>>>> work up to 2 gigablocks whatever your block size is. 512 blocks is
>>>>> 2199023255552 bytes. But that wastes just a WHOLE LOT of disk in
>>>>> block maps.
>>>>> Go up to 4096 or 8192. The latter is 35 TB.
>>>>>
>>>>> {^_^}
>>>>> On 20180624 02:06, Martin Steigerwald wrote:
>>>>>>
>>>>>>
>>>>>> Hi.
>>>>>>
>>>>>> Michael Schmitz - 27.04.18, 04:11:
>>>>>>>
>>>>>>>
>>>>>>> test results at https://bugzilla.kernel.org/show_bug.cgi?id=43511
>>>>>>> indicate the RDB parser bug is fixed by the patch given there, so if
>>>>>>> Martin now submits the patch, all should be well?
>>>>>>
>>>>>>
>>>>>>
>>>>>> Ok, better be honest than having anyone waiting for it:
>>>>>>
>>>>>> I do not care enough about this, in order to motivate myself preparing
>>>>>> the a patch from Joanne Dow´s fix.
>>>>>>
>>>>>> I am not even using my Amiga boxes anymore, not even the Sam440ep
>>>>>> which
>>>>>> I still have in my apartment.
>>>>>>
>>>>>> So RDB support in Linux it remains broken for disks larger 2 TB,
>>>>>> unless
>>>>>> someone else does.
>>>>>>
>>>>>> Thanks.
>>>>>>
>>>>> --
>>>>> To unsubscribe from this list: send the line "unsubscribe
>>>>> linux-m68k" in
>>>>> the body of a message to majordomo@vger.kernel.org
>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>
>>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-m68k" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-m68k" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

  reply index

Thread overview: 73+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-04-25 15:46 Moving unmaintained filesystems to staging Matthew Wilcox
2018-04-25 15:47 ` Christoph Hellwig
2018-04-25 20:30 ` David Sterba
2018-04-26  2:57   ` Matthew Wilcox
2018-04-26 10:28     ` moving affs + RDB partition support to staging? (was: Re: Moving unmaintained filesystems to staging) Martin Steigerwald
2018-04-26 10:45       ` moving affs + RDB partition support to staging? John Paul Adrian Glaubitz
2018-04-26 10:59         ` David Sterba
2018-04-26 11:06           ` John Paul Adrian Glaubitz
2018-05-06  0:59         ` Al Viro
2018-05-06  7:40           ` Al Viro
2018-05-06 20:46             ` Al Viro
2018-05-06 20:49               ` John Paul Adrian Glaubitz
2018-05-06 21:32               ` Al Viro
2018-05-07  2:15                 ` Al Viro
2018-05-07  2:40                   ` Michael Schmitz
2018-05-07  7:08                     ` Martin Steigerwald
2018-05-07 20:50                       ` Michael Schmitz
2018-05-07 20:56                         ` Ingo Jürgensmann
2018-05-07 20:58                           ` John Paul Adrian Glaubitz
2018-05-06  8:40           ` John Paul Adrian Glaubitz
2018-05-06 10:12           ` Martin Steigerwald
2018-04-26 11:00       ` moving affs + RDB partition support to staging? (was: Re: Moving unmaintained filesystems to staging) Christoph Hellwig
2018-04-26 11:08       ` Geert Uytterhoeven
2018-04-26 23:56         ` Finn Thain
2018-04-27  1:43           ` moving affs + RDB partition support to staging? jdow
2018-04-27  1:26         ` jdow
2018-05-06  8:52           ` John Paul Adrian Glaubitz
2018-05-06 10:10             ` Martin Steigerwald
2018-05-07  4:54             ` jdow
2018-04-27  2:11         ` moving affs + RDB partition support to staging? (was: Re: Moving unmaintained filesystems to staging) Michael Schmitz
2018-06-24  9:06           ` Martin Steigerwald
2018-06-24 11:33             ` moving affs + RDB partition support to staging? jdow
2018-06-24 11:40             ` jdow
2018-06-26  2:23               ` Michael Schmitz
2018-06-26  5:17                 ` jdow
2018-06-26  8:12                   ` Martin Steigerwald
2018-06-26  9:46                     ` jdow
2018-06-26  8:31                   ` Michael Schmitz
2018-06-26  9:45                     ` jdow
2018-06-27  1:07                       ` Michael Schmitz [this message]
2018-06-27  6:24                         ` jdow
2018-06-27  8:03                           ` Martin Steigerwald
2018-06-28  2:57                             ` jdow
2018-06-28  7:40                               ` Amiga RDB partition support for disks >= 2 TB (was: Re: moving affs + RDB partition support to staging?) Martin Steigerwald
2018-06-27  9:00                           ` moving affs + RDB partition support to staging? Michael Schmitz
2018-06-28  3:44                             ` jdow
2018-06-28  5:43                               ` Michael Schmitz
2018-06-28  6:39                                 ` jdow
2018-06-28  8:16                                   ` Amiga RDB partition support for disks >= 2 TB (was: Re: moving affs + RDB partition support to staging?) Martin Steigerwald
2018-06-28 10:00                                     ` Amiga RDB partition support for disks >= 2 TB jdow
2018-06-28 11:30                                       ` Martin Steigerwald
2018-06-28 11:38                                         ` Martin Steigerwald
2018-06-28 12:31                                           ` jdow
2018-06-28  8:07                                 ` Amiga RDB partition support for disks >= 2 TB (was: Re: moving affs + RDB partition support to staging?) Martin Steigerwald
2018-06-27  7:57                         ` moving affs + RDB partition support to staging? Martin Steigerwald
2018-06-28  2:56                           ` jdow
2018-06-26  8:02                 ` Martin Steigerwald
2018-06-26  8:40                   ` Michael Schmitz
2018-06-26  9:31                   ` jdow
2018-06-25  7:53             ` moving affs + RDB partition support to staging? (was: Re: Moving unmaintained filesystems to staging) Michael Schmitz
2018-06-25  8:26               ` Martin Steigerwald
2018-06-25  8:40               ` Geert Uytterhoeven
2018-04-27  8:01         ` Martin Steigerwald
2018-04-26  4:58   ` Moving unmaintained filesystems to staging Nikolay Borisov
2018-04-26  5:30     ` Willy Tarreau
2018-04-26  6:11 ` Pavel Machek
2018-04-26 10:36   ` Martin Steigerwald
2018-05-03  9:18     ` Pavel Machek
2018-04-27  1:10   ` Luis R. Rodriguez
2018-04-29 12:07   ` Greg KH
2018-04-29 20:07     ` Ondrej Zary
2018-04-29 23:37       ` Greg KH
2018-05-01 10:14         ` Pavel Machek

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CAOmrzkKLJiWvDwsER0ca09UJbi57Rj6PUmdxizSNPXzZLMErOA@mail.gmail.com \
    --to=schmitzmic@gmail.com \
    --cc=axboe@kernel.dk \
    --cc=dsterba@suse.cz \
    --cc=geert@linux-m68k.org \
    --cc=jdow@earthlink.net \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-m68k@lists.linux-m68k.org \
    --cc=martin@lichtvoll.de \
    --cc=willy@infradead.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-Fsdevel Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-fsdevel/0 linux-fsdevel/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-fsdevel linux-fsdevel/ https://lore.kernel.org/linux-fsdevel \
		linux-fsdevel@vger.kernel.org
	public-inbox-index linux-fsdevel

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-fsdevel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git