Driver for BIOS-based software RAIDs

Driver for BIOS-based software RAIDs

[Thomas Horsten]

Hi,

I've written a driver for the Medley software RAID using the ATARAID framework 
in 2.4 kernels (http://www.infowares.com/linux).

Now, I'm looking into writing a better driver that can also be used with 2.6. 
My thoughts are along the lines of using the MD driver infrastructure and 
adding support for "foreign" RAID superblocks (and possibly personalities).

For the Medley RAID this could be quite an elegant solution, as the Medley 
modes are identical to MD's RAID0 and RAID1 and modes (and the RAID10 mode 
identical to a RAID0 embedded in a RAID1 array).

Now, the problem is that these BIOS based software RAID's all use whole disks 
as opposed to individual partitions like the Linux MD driver.

My idea was to either reserve a new major for "md_wholedisk" mode and split it 
into drive,partition blocks of 16/32 inodes each, like the gendisk driver 
supports, or alternatively split the existing major with, say, the first 64 
minors for normal style MD devs and the upper part split like I suggested 
above (I'm not sure if this would work with gendisk though).

Another way of doing it would be to create a separate native MD RAID for each 
partition on the BIOS RAID, but this has the major drawback that the user 
wouldn't be able to repartition the device.

Brief outline of how this would work:

1) User selects CONFIG_MD_FOREIGN_SUPERBLOCKS and CONFIG_MD_WHOLEDISK_RAID, 
along with one oif the BIOS specific drivers like CONFIG_MD_FOREIGN_MEDLEY. 
Each BIOS driver has its own superblock type.

2a) If compiled into the kernel, a hook in the partition parser will first 
pass each drive to the RAID handler, before attempting to parse the partition 
table. RAID handler will call each FOREIGN_driver in turn to see if there is 
a matching RAID superblock. If there is, it will be added as a partial RAID 
and the partition checker will not try to parse the partition table (which 
will be invalid if the drive is part of a RAID).

2b) If the foreign RAID drivers are compiled as modules, upon loading they 
will walk all genhds (or devices they will support), and detect any custom 
superblocks. The drivers will then use a new hook in the genhd code to zero 
out any partitions on that device.

3) The foreign RAID driver creates mddev_s based on its detection. If 
necessary, the foreign RAID driver can also create its own personalities 
(like the special type of RAID0 that Highpoint devices use, that supports 
having a RAID0 where all disks are not of equal size, without wasting space).

Any comments and ideas so far? Is this approach viable?

Thanks :)

Regards,

Thomas

Re: Driver for BIOS-based software RAIDs

[Kevin P. Fleming]

Thomas Horsten wrote:

> Now, the problem is that these BIOS based software RAID's all use whole disks 
> as opposed to individual partitions like the Linux MD driver.

Would it not make more sense to leverage the device-mapper and just 
create a new personality or two for it? It can already be configured 
to use whole disk devices if the user needs to.

> 
> My idea was to either reserve a new major for "md_wholedisk" mode and split it 
> into drive,partition blocks of 16/32 inodes each, like the gendisk driver 
> supports, or alternatively split the existing major with, say, the first 64 
> minors for normal style MD devs and the upper part split like I suggested 
> above (I'm not sure if this would work with gendisk though).

This sounds very complicated, although what I'm going to suggest is 
not really less complicated :-)

> 
> Another way of doing it would be to create a separate native MD RAID for each 
> partition on the BIOS RAID, but this has the major drawback that the user 
> wouldn't be able to repartition the device.

That's going to be the biggest problem, that you have to support 
partitioning of the RAID array, as opposed to its components. No 
matter what underlying method you choose to implement it, the kernel 
currently does not support partitioning of MD devices (or 
device-mapper devices, for that matter). That means that without a 
significant amount of work, you'd be limited to using userspace 
partition-reading tools that then talk to device-mapper to create 
"logical" devices that are parts of the RAID array. This is what many 
people want the kernel to move towards anyway, so maybe this is a 
reasonable method.

> 3) The foreign RAID driver creates mddev_s based on its detection. If 
> necessary, the foreign RAID driver can also create its own personalities 
> (like the special type of RAID0 that Highpoint devices use, that supports 
> having a RAID0 where all disks are not of equal size, without wasting space).
> 

I think that for 2.6 (well, now 2.7 and then backported to 2.6), you 
should seriously consider using the udev/hotplug/etc. framework that's 
being worked on. Here's a (very basic) overview:

- a hotplug event occurs that signals that a block device has been 
found (raw, whole hard drive at this point)
- /sbin/hotplug (or some script that it invokes) uses a userspace tool 
to look at the device to see if it contains a Medley RAID superblock; 
if so, it records that information somewhere (pending the arrival of 
the other pieces of the array)
- second hotplug event happens (for second raw, whole hard drive)
- again, Medley RAID superblock on device is found, this time a 
complete set of superblocks is known about, so the userspace tool 
passes that information to device-mapper, telling it to create a new 
block device composed of these two devices, using your new (or an 
existing) device-mapper personality
- this causes yet another hotplug event, this time because the RAID 
block device appeared
- device is scanned looking for Medley RAID superblock (in case it's 
RAID0 inside a RAID1)... if none is found, device is scanned for 
understandable partition table, if one is found, device-mapper is used 
to create logical devices out of those partitions

I know this is radically different than what you are proposing, but 
it's the plan for many people to move this direction for 2.7. It 
removes policy problems from the kernel, it supports unlimited layers 
of partitioning/RAID/etc., and the user can control the order that 
things get done (looking for partitions first vs. RAID superblocks 
first, etc.).

Re: Driver for BIOS-based software RAIDs

[Lars Marowsky-Bree]

On 2003-10-13T19:21:45,
   Thomas Horsten <thomas@horsten.com> said:

> Now, the problem is that these BIOS based software RAID's all use whole disks 
> as opposed to individual partitions like the Linux MD driver.

Why is that a problem? You can assemble a MD using whole disks instead
of partition. A block device is a block device is a block device.

> Another way of doing it would be to create a separate native MD RAID for each 
> partition on the BIOS RAID, but this has the major drawback that the user 
> wouldn't be able to repartition the device.

Use a volume manager (LVM1, LVM2, EVMS2 ...) on top of the md. Works
just fine.

> 1) User selects CONFIG_MD_FOREIGN_SUPERBLOCKS and CONFIG_MD_WHOLEDISK_RAID, 
> along with one oif the BIOS specific drivers like CONFIG_MD_FOREIGN_MEDLEY. 
> Each BIOS driver has its own superblock type.

Autodiscovery is a problem, but I'd prefer to do that in the initrd from
within userspace. That's much cleaner.

Sincerely,
    Lars Marowsky-Brée <lmb@suse.de>

-- 
High Availability & Clustering		ever tried. ever failed. no matter.
SuSE Labs				try again. fail again. fail better.
Research & Development, SUSE LINUX AG		-- Samuel Beckett

-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Driver for BIOS-based software RAIDs

[Thomas Horsten]

On Monday 13 October 2003 19:31, Kevin P. Fleming wrote:

> Thomas Horsten wrote:
> > Now, the problem is that these BIOS based software RAID's all use whole
> > disks as opposed to individual partitions like the Linux MD driver.
>
> Would it not make more sense to leverage the device-mapper and just
> create a new personality or two for it? It can already be configured
> to use whole disk devices if the user needs to.

In the light of the desired functionality of this driver, I don't think this 
will work (see below also). The main aims is to make it transparent to the 
user (this is the whole point of BIOS based RAID's). The array should present 
itself as a normal, whole disk style block device inside Linux at boot time 
(if compiled in), so that LILO etc. can be used transparently. Remember the 
user has set up the RAID outside Linux (using the BIOS utility), and is thus 
likely to expect it to be autodetected and autoconfigured. So it should be 
"partitionable" - the whole reason that the BIOS RAID was implemented. The 
RAID should appear as one logical disk to ensure consistency between Linux 
and other OS's that use the array.

> > My idea was to either reserve a new major for "md_wholedisk" mode and
> > split it into drive,partition blocks of 16/32 inodes each, like the
> > gendisk driver supports, or alternatively split the existing major with,
> > say, the first 64 minors for normal style MD devs and the upper part
> > split like I suggested above (I'm not sure if this would work with
> > gendisk though).
>
> This sounds very complicated, although what I'm going to suggest is
> not really less complicated :-)

Well, let's save the simple solutions for the simple problems :-)

> > Another way of doing it would be to create a separate native MD RAID for
> > each partition on the BIOS RAID, but this has the major drawback that the
> > user wouldn't be able to repartition the device.
>
> That's going to be the biggest problem, that you have to support
> partitioning of the RAID array, as opposed to its components. No
> matter what underlying method you choose to implement it, the kernel
> currently does not support partitioning of MD devices (or
> device-mapper devices, for that matter). That means that without a
> significant amount of work, you'd be limited to using userspace
> partition-reading tools that then talk to device-mapper to create
> "logical" devices that are parts of the RAID array. This is what many
> people want the kernel to move towards anyway, so maybe this is a
> reasonable method.

Yes and that's why I propose splitting the MD driver into either two majors, 
where one uses the partitioning support (register_disk in genhd), or using 
the partitioning support for the upper part of the existing minor space. The 
support is there in gendisk, it's just a matter of calling register_disk 
after setting up the right information in the struct. The RAID code would 
need some tweaking but I think it would be workable - I think :-)

> > 3) The foreign RAID driver creates mddev_s based on its detection. If
> > necessary, the foreign RAID driver can also create its own personalities
> > (like the special type of RAID0 that Highpoint devices use, that supports
> > having a RAID0 where all disks are not of equal size, without wasting
> > space).
>
> I think that for 2.6 (well, now 2.7 and then backported to 2.6), you
> should seriously consider using the udev/hotplug/etc. framework that's
> being worked on. Here's a (very basic) overview:

I agree it should be considered but again I don't think it's appropriate - I 
don't want to rely on userspace tools for this type of RAID, since the 
"userspace tools" were already provided by the manufacturer in the form of 
the BIOS configuration utility. This is where the user creates the RAID set, 
and since it's a BIOS drive, again I would prefer the kernel to autodetect it 
completely and let the user see it as he would a "normal" block device.

> I know this is radically different than what you are proposing, but
> it's the plan for many people to move this direction for 2.7. It
> removes policy problems from the kernel, it supports unlimited layers
> of partitioning/RAID/etc., and the user can control the order that
> things get done (looking for partitions first vs. RAID superblocks
> first, etc.).

Using the hotplug interface for the resulting "wholedisk" array would 
certainly be a possibility, and I think this is worth looking into especially 
in the module version of the driver. But I still propose the "partitionable 
gendisk" way for initial detection, since it will make it much simpler to 
install on these types of RAID.

Regards,

Thomas

Re: Driver for BIOS-based software RAIDs

[Thomas Horsten]

On Monday 13 October 2003 20:03, Lars Marowsky-Bree wrote:

> > Now, the problem is that these BIOS based software RAID's all use whole
> > disks as opposed to individual partitions like the Linux MD driver.
>
> Why is that a problem? You can assemble a MD using whole disks instead
> of partition. A block device is a block device is a block device.

The problem is that the block device thus assembled won't have support for 
multiple partitions, for two reasons: the MD driver doesn't align the minor 
to any offset, and it doesn't allocate multiple minors for a RAID so we can 
call register_disk to parse the partition table and create the logical 
partition devices. Both of these are defaults, and it might be possible to 
tweak.

> > Another way of doing it would be to create a separate native MD RAID for
> > each partition on the BIOS RAID, but this has the major drawback that the
> > user wouldn't be able to repartition the device.
>
> Use a volume manager (LVM1, LVM2, EVMS2 ...) on top of the md. Works
> just fine.

If the volume manager can be set up automatically this might just be the 
solution.

> > 1) User selects CONFIG_MD_FOREIGN_SUPERBLOCKS and
> > CONFIG_MD_WHOLEDISK_RAID, along with one oif the BIOS specific drivers
> > like CONFIG_MD_FOREIGN_MEDLEY. Each BIOS driver has its own superblock
> > type.
>
> Autodiscovery is a problem, but I'd prefer to do that in the initrd from
> within userspace. That's much cleaner.

Well due to the nature of BIOS RAID's, the user is likely to expect that it 
will be detected and handled automatically without the need for a separate 
setup (since the machine creates the logical device for them, and this works 
in other OS's that use the BIOS to access the drive).

I agree it would be cleaner to do it in userspace, but autodiscovery is one of 
the major aims of this driver. I would like to support both methods if at all 
possible.

Best regards,

Thomas

Re: Driver for BIOS-based software RAIDs

[Lars Marowsky-Bree]

On 2003-10-13T20:25:30,
   Thomas Horsten <thomas@horsten.com> said:

> The problem is that the block device thus assembled won't have support for 
> multiple partitions, for two reasons: the MD driver doesn't align the minor 
> to any offset, and it doesn't allocate multiple minors for a RAID so we can 
> call register_disk to parse the partition table and create the logical 
> partition devices. Both of these are defaults, and it might be possible to 
> tweak.

You probably could change that easily enough.

> Well due to the nature of BIOS RAID's, the user is likely to expect
> that it will be detected and handled automatically without the need
> for a separate setup (since the machine creates the logical device for
> them, and this works in other OS's that use the BIOS to access the
> drive).

"BIOS RAIDs" is a rather sweet euphemism for "crippled pretense of
hardware RAID, which is in fact emulated by software". I've got no
problem with telling the users that what they have _is_ software RAID
and needs to be treated as such.

Windows doesn't use the BIOS to access the device at all; the
corresponding hardware drivers emulate that.


Sincerely,
    Lars Marowsky-Brée <lmb@suse.de>

-- 
High Availability & Clustering		ever tried. ever failed. no matter.
SuSE Labs				try again. fail again. fail better.
Research & Development, SUSE LINUX AG		-- Samuel Beckett

-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Driver for BIOS-based software RAIDs

[Thomas Horsten]

On Monday 13 October 2003 22:17, Lars Marowsky-Bree wrote:

> "BIOS RAIDs" is a rather sweet euphemism for "crippled pretense of
> hardware RAID, which is in fact emulated by software". I've got no
> problem with telling the users that what they have _is_ software RAID
> and needs to be treated as such.

I completely agree with you, and I will make sure my drivers print "Software 
RAID" all over the screen when it loads :-) Ok, at least on one line then. 
I'm not on a mission to justify the marketing methods of chip manufactureres.

Having said that, BIOS RAID does solve a few problems inherent to software 
RAID's, particularly the boot process and consistency of RAID's between 
operating systems. I want to leverage those advantages and I think that this 
is best done with an autodetecting driver.

> Windows doesn't use the BIOS to access the device at all; the
> corresponding hardware drivers emulate that.

I know, but the BIOS supports its boot up to 32-bit mode. Which means you 
could see the drive from DOS, etc. And putting the RAID in BIOS does have 
some advantages for the average PC motherboard.

Best regards,

Thomas

Re: Driver for BIOS-based software RAIDs

[Lars Marowsky-Bree]

On 2003-10-13T22:38:40,
   Thomas Horsten <thomas@horsten.com> said:

> Having said that, BIOS RAID does solve a few problems inherent to software 
> RAID's, particularly the boot process

The boot process is a real problem. It can be partly solved in the
bootloader - iff the BIOS is smart enough to try all disks until it can
read one copy -, but yes, BIOS RAID does have some advantage here. This
is pre-kernel stage though and so beyond this driver anyway.

After the kernel is up and running, we can handle that just fine.

> and consistency of RAID's between operating systems.

That's fortunately a problem I don't have to care about ;-)

> > Windows doesn't use the BIOS to access the device at all; the
> > corresponding hardware drivers emulate that.
> I know, but the BIOS supports its boot up to 32-bit mode. Which means you 
> could see the drive from DOS, etc.

Well, for the boot loader stage it does have some advantage as I said.

But beyond that it doesn't matter and is bypassed.


Sincerely,
    Lars Marowsky-Brée <lmb@suse.de>

-- 
High Availability & Clustering		ever tried. ever failed. no matter.
SuSE Labs				try again. fail again. fail better.
Research & Development, SUSE LINUX AG		-- Samuel Beckett

-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html