raid array with 3T disks and GPT partition

raid array with 3T disks and GPT partition

[Louis-David Mitterrand]

Hi,

I'm trying to create a raid6 array from 10x3T disks. Since disks > 2T
must use the GPT partion table I used parted to created a single
partition on each drive with the correct GPT partion type.

Now how do I make sure that these partitions have the correct "raid
autodetect" (fd) id? Is it even still needed? I didn't find any way to
set that flag in (g)parted.

Thanks,

Re: raid array with 3T disks and GPT partition

[Pim Zandbergen]

On 09/01/2011 05:47 PM, Louis-David Mitterrand wrote:
>
> Now how do I make sure that these partitions have the correct "raid
> autodetect" (fd) id? Is it even still needed? I didn't find any way to
> set that flag in (g)parted.
>
I used gdisk and entered FD00

Re: raid array with 3T disks and GPT partition

[Louis-David Mitterrand]

On Thu, Sep 01, 2011 at 05:53:02PM +0200, Pim Zandbergen wrote:
> On 09/01/2011 05:47 PM, Louis-David Mitterrand wrote:
> >
> >Now how do I make sure that these partitions have the correct "raid
> >autodetect" (fd) id? Is it even still needed? I didn't find any way to
> >set that flag in (g)parted.
> >
> I used gdisk and entered FD00

Didn't know this tool, thanks.

So (g)parted is not appropriate for creating raid-autodetect partitions?

Re: raid array with 3T disks and GPT partition

[Robin Hill]

[-- Attachment #1: Type: text/plain, Size: 1006 bytes --]

On Thu Sep 01, 2011 at 05:47:59PM +0200, Louis-David Mitterrand wrote:

> Hi,
> 
> I'm trying to create a raid6 array from 10x3T disks. Since disks > 2T
> must use the GPT partion table I used parted to created a single
> partition on each drive with the correct GPT partion type.
> 
> Now how do I make sure that these partitions have the correct "raid
> autodetect" (fd) id? Is it even still needed? I didn't find any way to
> set that flag in (g)parted.
> 
It's only needed for kernel auto-assembly (in which case you're also
limited to 0.90 metadata and 2TB drives), so no, there's no need to use
that. 0xDA seems to be the recommended partition type for RAID arrays
nowadays - that should prevent the OS from trying to read them directly.

Cheers,
    Robin
-- 
     ___        
    ( ' }     |       Robin Hill        <robin@robinhill.me.uk> |
   / / )      | Little Jim says ....                            |
  // !!       |      "He fallen in de water !!"                 |

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: raid array with 3T disks and GPT partition

[Pim Zandbergen]

On 09/01/2011 05:57 PM, Louis-David Mitterrand wrote:
> So (g)parted is not appropriate for creating raid-autodetect partitions?

I can't tell. gdisk is the GPT equivalent of fdisk. It's command line
orientated and gives full low-level control. I'm comfortable with that.

Re: raid array with 3T disks and GPT partition

[Louis-David Mitterrand]

On Thu, Sep 01, 2011 at 04:59:13PM +0100, Robin Hill wrote:
> On Thu Sep 01, 2011 at 05:47:59PM +0200, Louis-David Mitterrand wrote:
> > 
> > I'm trying to create a raid6 array from 10x3T disks. Since disks > 2T
> > must use the GPT partion table I used parted to created a single
> > partition on each drive with the correct GPT partion type.
> > 
> > Now how do I make sure that these partitions have the correct "raid
> > autodetect" (fd) id? Is it even still needed? I didn't find any way to
> > set that flag in (g)parted.
> > 
> It's only needed for kernel auto-assembly (in which case you're also
> limited to 0.90 metadata and 2TB drives), so no, there's no need to use
> that. 0xDA seems to be the recommended partition type for RAID arrays
> nowadays - that should prevent the OS from trying to read them directly.

Auto-assembly and metadata are not related: I regularly use 1.2 metadata
on non-boot partitions and they auto-assemble fine.

However lilo won't boot on anything other than 0.9, this I found the
hard way :)

Kernel auto-assembly seems quite useful and desirable to me, especially
when using, say, initrd to unlock dm-crypt partitions (no need to
configure madadm.conf in the initrd).

BTW, what is that 0xDA type?

Cheers,

Re: raid array with 3T disks and GPT partition

[Doug Ledford]

On 09/01/2011 12:33 PM, Louis-David Mitterrand wrote:
> On Thu, Sep 01, 2011 at 04:59:13PM +0100, Robin Hill wrote:
>> On Thu Sep 01, 2011 at 05:47:59PM +0200, Louis-David Mitterrand wrote:
>>>
>>> I'm trying to create a raid6 array from 10x3T disks. Since disks>  2T
>>> must use the GPT partion table I used parted to created a single
>>> partition on each drive with the correct GPT partion type.
>>>
>>> Now how do I make sure that these partitions have the correct "raid
>>> autodetect" (fd) id? Is it even still needed? I didn't find any way to
>>> set that flag in (g)parted.
>>>
>> It's only needed for kernel auto-assembly (in which case you're also
>> limited to 0.90 metadata and 2TB drives), so no, there's no need to use
>> that. 0xDA seems to be the recommended partition type for RAID arrays
>> nowadays - that should prevent the OS from trying to read them directly.
>
> Auto-assembly and metadata are not related: I regularly use 1.2 metadata
> on non-boot partitions and they auto-assemble fine.

They most certainly are related.  There is kernel autoassembly, then 
there is user space assembly that's done by udev.  They are two 
different things.  The kernel will only autoassemble version 0.9 arrays, 
any other arrays are assembled by user space either in the initramfs or 
later on in the boot cycle.  That you don't have to manually run mdadm 
-As doesn't mean that the kernel autoassembly is working on those arrays.

> However lilo won't boot on anything other than 0.9, this I found the
> hard way :)

Lilo should be able to boot on version 1.0 arrays too.  It's just 
version 1.1 and 1.2 arrays that kill lilo.  But, then again, lilo is so 
long since dead that one has to wonder why you are still using it.

> Kernel auto-assembly seems quite useful and desirable to me, especially
> when using, say, initrd to unlock dm-crypt partitions (no need to
> configure madadm.conf in the initrd).
>
> BTW, what is that 0xDA type?

Non-FS Data, which we hijacked for our purposes as it will keep the 
system (any OS in fact) from thinking there should be a filesystem 
there, leaving it free for us to pick up anytime in the boot process.

Re: raid array with 3T disks and GPT partition

[John Robinson]

On 01/09/2011 17:33, Louis-David Mitterrand wrote:
[...]
> Auto-assembly and metadata are not related: I regularly use 1.2 metadata
> on non-boot partitions and they auto-assemble fine.

Not possible. The kernel will not auto-assemble anything other than 0.90 
metadata. You must have mdadm in your initrd.

> However lilo won't boot on anything other than 0.9, this I found the
> hard way :)

I have had lilo boot from 1.0 metadata just fine. It does need to be the 
metadata-at-the-end layout though, hence 1.0 but not 1.1 or 1.2.

> Kernel auto-assembly seems quite useful and desirable to me, especially
> when using, say, initrd to unlock dm-crypt partitions (no need to
> configure madadm.conf in the initrd).

Neil Brown took the decision ages ago not to add any more what ought to 
be userland use-once tools into the kernel, and this is the general 
direction across the whole kernel. If you have an initrd unlocking a 
dm-crypt partition you might as well have mdadm in there as well 
starting your arrays.

> BTW, what is that 0xDA type?

Non-FS data.

Cheers,

John.

Re: raid array with 3T disks and GPT partition

[Robin Hill]

[-- Attachment #1: Type: text/plain, Size: 2174 bytes --]

On Thu Sep 01, 2011 at 06:33:34PM +0200, Louis-David Mitterrand wrote:

> On Thu, Sep 01, 2011 at 04:59:13PM +0100, Robin Hill wrote:
> > On Thu Sep 01, 2011 at 05:47:59PM +0200, Louis-David Mitterrand wrote:
> > > 
> > > I'm trying to create a raid6 array from 10x3T disks. Since disks > 2T
> > > must use the GPT partion table I used parted to created a single
> > > partition on each drive with the correct GPT partion type.
> > > 
> > > Now how do I make sure that these partitions have the correct "raid
> > > autodetect" (fd) id? Is it even still needed? I didn't find any way to
> > > set that flag in (g)parted.
> > > 
> > It's only needed for kernel auto-assembly (in which case you're also
> > limited to 0.90 metadata and 2TB drives), so no, there's no need to use
> > that. 0xDA seems to be the recommended partition type for RAID arrays
> > nowadays - that should prevent the OS from trying to read them directly.
> 
> Auto-assembly and metadata are not related: I regularly use 1.2 metadata
> on non-boot partitions and they auto-assemble fine.
> 
Kernel auto-assembly and metadata are related (see the description for
the --auto-detect option in the mdadm manual page). Most distributions
now use mdadm in an initrd for assembly, which will (of course) work
with any metadata version.

> However lilo won't boot on anything other than 0.9, this I found the
> hard way :)
> 
LILO (and grub) should work with 0.9 or 1.0 metadata, as both of these
write the metadata at the end of the disk, leaving the start looking the
same as a normal disk (for RAID-1 anyway).

> Kernel auto-assembly seems quite useful and desirable to me, especially
> when using, say, initrd to unlock dm-crypt partitions (no need to
> configure madadm.conf in the initrd).
> 
Kernel auto-assembly is not recommended by the main md developer.

> BTW, what is that 0xDA type?
> 
0xDA is "non-FS data".

Cheers,
    Robin
-- 
     ___        
    ( ' }     |       Robin Hill        <robin@robinhill.me.uk> |
   / / )      | Little Jim says ....                            |
  // !!       |      "He fallen in de water !!"                 |

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: raid array with 3T disks and GPT partition

[Rudy Zijlstra]

>> However lilo won't boot on anything other than 0.9, this I found the
>> hard way :)
>
> Lilo should be able to boot on version 1.0 arrays too.  It's just 
> version 1.1 and 1.2 arrays that kill lilo.  But, then again, lilo is 
> so long since dead that one has to wonder why you are still using it.
>
At least lilo has an understandable way of working. grub2 is just.... a 
big pain in the *ass as far as i am concerned.

Re: raid array with 3T disks and GPT partition

[Louis-David Mitterrand]

On Thu, Sep 01, 2011 at 12:55:08PM -0400, Doug Ledford wrote:
> On 09/01/2011 12:33 PM, Louis-David Mitterrand wrote:
> >On Thu, Sep 01, 2011 at 04:59:13PM +0100, Robin Hill wrote:
> >>On Thu Sep 01, 2011 at 05:47:59PM +0200, Louis-David Mitterrand wrote:
> >>>
> >>>I'm trying to create a raid6 array from 10x3T disks. Since disks>  2T
> >>>must use the GPT partion table I used parted to created a single
> >>>partition on each drive with the correct GPT partion type.
> >>>
> >>>Now how do I make sure that these partitions have the correct "raid
> >>>autodetect" (fd) id? Is it even still needed? I didn't find any way to
> >>>set that flag in (g)parted.
> >>>
> >>It's only needed for kernel auto-assembly (in which case you're also
> >>limited to 0.90 metadata and 2TB drives), so no, there's no need to use
> >>that. 0xDA seems to be the recommended partition type for RAID arrays
> >>nowadays - that should prevent the OS from trying to read them directly.
> >
> >Auto-assembly and metadata are not related: I regularly use 1.2 metadata
> >on non-boot partitions and they auto-assemble fine.
> 
> They most certainly are related.  There is kernel autoassembly, then
> there is user space assembly that's done by udev.  They are two
> different things.  The kernel will only autoassemble version 0.9
> arrays, any other arrays are assembled by user space either in the
> initramfs or later on in the boot cycle.  That you don't have to
> manually run mdadm -As doesn't mean that the kernel autoassembly is
> working on those arrays.

Ah, thanks for clearing that up. I thought that when mdadm was not
involved then is must be the kernel. Didn't realize udev was at work
there.

Re: raid array with 3T disks and GPT partition

[Doug Ledford]

On 09/01/2011 03:20 PM, Louis-David Mitterrand wrote:
> On Thu, Sep 01, 2011 at 12:55:08PM -0400, Doug Ledford wrote:
>> On 09/01/2011 12:33 PM, Louis-David Mitterrand wrote:
>>> On Thu, Sep 01, 2011 at 04:59:13PM +0100, Robin Hill wrote:
>>>> On Thu Sep 01, 2011 at 05:47:59PM +0200, Louis-David Mitterrand wrote:
>>>>>
>>>>> I'm trying to create a raid6 array from 10x3T disks. Since disks>   2T
>>>>> must use the GPT partion table I used parted to created a single
>>>>> partition on each drive with the correct GPT partion type.
>>>>>
>>>>> Now how do I make sure that these partitions have the correct "raid
>>>>> autodetect" (fd) id? Is it even still needed? I didn't find any way to
>>>>> set that flag in (g)parted.
>>>>>
>>>> It's only needed for kernel auto-assembly (in which case you're also
>>>> limited to 0.90 metadata and 2TB drives), so no, there's no need to use
>>>> that. 0xDA seems to be the recommended partition type for RAID arrays
>>>> nowadays - that should prevent the OS from trying to read them directly.
>>>
>>> Auto-assembly and metadata are not related: I regularly use 1.2 metadata
>>> on non-boot partitions and they auto-assemble fine.
>>
>> They most certainly are related.  There is kernel autoassembly, then
>> there is user space assembly that's done by udev.  They are two
>> different things.  The kernel will only autoassemble version 0.9
>> arrays, any other arrays are assembled by user space either in the
>> initramfs or later on in the boot cycle.  That you don't have to
>> manually run mdadm -As doesn't mean that the kernel autoassembly is
>> working on those arrays.
>
> Ah, thanks for clearing that up. I thought that when mdadm was not
> involved then is must be the kernel. Didn't realize udev was at work
> there.

Well, udev calls mdadm.  If the kernel doesn't autoassemble it, then 
mdadm is involved at some point.  On older systems, rc.sysinit called 
mdadm to assemble non-boot arrays, on modern systems, udev calls mdadm.

Re: raid array with 3T disks and GPT partition

[Bill Davidsen]

Rudy Zijlstra wrote:
>
>>> However lilo won't boot on anything other than 0.9, this I found the
>>> hard way :)
>>
>> Lilo should be able to boot on version 1.0 arrays too.  It's just 
>> version 1.1 and 1.2 arrays that kill lilo.  But, then again, lilo is 
>> so long since dead that one has to wonder why you are still using it.
>>
> At least lilo has an understandable way of working. grub2 is just.... 
> a big pain in the *ass as far as i am concerned.

I would say that grub seems easier to use than grub2. Haven't used lilo 
since I got off Slackware, and it has an annoying limit of 19 bootable 
kernels.

-- 
Bill Davidsen<davidsen@tmr.com>
   We are not out of the woods yet, but we know the direction and have
taken the first step. The steps are many, but finite in number, and if
we persevere we will reach our destination.  -me, 2010

Re: raid array with 3T disks and GPT partition

[Phillip Susi]

On 9/1/2011 12:55 PM, Doug Ledford wrote:
> Non-FS Data, which we hijacked for our purposes as it will keep the
> system (any OS in fact) from thinking there should be a filesystem
> there, leaving it free for us to pick up anytime in the boot process.

What do you mean?  Linux doesn't care what the partition type code says. 
  I brought that up on lkml a year or so back that it should at least 
respect the hidden flag and not expose that partition, but nobody seemed 
interested.

Re: raid array with 3T disks and GPT partition

[Peter Grandi]

[ ... ]

>> Non-FS Data, which we hijacked for our purposes as it will
>> keep the system (any OS in fact) from thinking there should
>> be a filesystem there, leaving it free for us to pick up
>> anytime in the boot process.

> What do you mean?  Linux doesn't care what the partition type
> code says. I brought that up on lkml a year or so back that
> it should at least respect the hidden flag and not expose that
> partition, but nobody seemed interested.

Probably you were posting to the wrong mailing list, as 'Linux'
(the kernel) indeed does not and should not care about the
partition type code, because all it does is to enumerate
partitions reading them from the label (if any) wrapping them as
block-devices, and the only filetree that 'Linux' mounts is the
'/' one in the block-device that is explicitly indicated (in the
kernel image or at boot).

Automounting, if any, is done by system or user daemons and the
logic (and whether the 'hidden' flag is respected) they follow is
distribution specific. LKML is not where that should be discussed.

The only case where partition types matter to the kernel is for
auto-detection of RAID members, but that is somewhat obsolete as
the man-page says:

http://linux.die.net/man/8/mdadm
 «In-kernel autodetect is not recommended for new installations.
  Using mdadm to detect and assemble arrays - possibly in an
  initrd - is substantially more flexible and should be preferred.»

And even as to the '/' filetree, in general 'Linux' nowadays in
most distros boots only from an 'initrd' and even mounting the '/'
filesystem is done from the 'initrd', and any rules about mounting
which block-devices in the 'initrd' or later in some automount
daemon configuration.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html