Re: Linux mdadm superblock question.

Re: Linux mdadm superblock question.

[Volker Armin Hemmann]

>0.90 has a very bad problem, which is that it is hard to distinguish
>between a RAID partition at the end of volume and a full RAID device.
>This is because 0.90 doesn't actually tell you the start of the device.
>
>Then, of course, there are a lot of limitations on size, number of
>devices, and so on in 0.90.

but it is the only format supporting autodetection. 

So - when will autodetection be introduced with 1.X? And if not, why not?

All I found was 'autodetection might be troublesome' and nothing else.
 But dealing with initrds is troublesome too. Pure evil even. 

Glück Auf,
Volker
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Linux mdadm superblock question.

[Volker Armin Hemmann]

>0.90 has a very bad problem, which is that it is hard to distinguish
>between a RAID partition at the end of volume and a full RAID device.
>This is because 0.90 doesn't actually tell you the start of the device.
>
>Then, of course, there are a lot of limitations on size, number of
>devices, and so on in 0.90.

but it is the only format supporting autodetection. 

So - when will autodetection be introduced with 1.X? And if not, why not?

All I found was 'autodetection might be troublesome' and nothing else.
 But dealing with initrds is troublesome too. Pure evil even. 

Glück Auf,
Volker

Re: Linux mdadm superblock question.

[Michael Evans]

On Sat, Feb 13, 2010 at 5:51 PM, Volker Armin Hemmann
<volkerarmin@googlemail.com> wrote:
>>0.90 has a very bad problem, which is that it is hard to distinguish
>>between a RAID partition at the end of volume and a full RAID device.
>>This is because 0.90 doesn't actually tell you the start of the device.
>>
>>Then, of course, there are a lot of limitations on size, number of
>>devices, and so on in 0.90.
>
> but it is the only format supporting autodetection.
>
> So - when will autodetection be introduced with 1.X? And if not, why not?
>
> All I found was 'autodetection might be troublesome' and nothing else.
>  But dealing with initrds is troublesome too. Pure evil even.
>
> Glück Auf,
> Volker
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

I remember hearing that 1.x had /no/ plans for kernel level
auto-detection ever.  That can be accomplished in early-userspace
leaving the code in the kernel much less complex, and therefore far
more reliable.

In other words, 'auto-detection' for 1.x format devices is using an
initrd/initramfs.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Linux mdadm superblock question.

[Michael Evans]

On Sat, Feb 13, 2010 at 5:51 PM, Volker Armin Hemmann
<volkerarmin@googlemail.com> wrote:
>>0.90 has a very bad problem, which is that it is hard to distinguish
>>between a RAID partition at the end of volume and a full RAID device.
>>This is because 0.90 doesn't actually tell you the start of the device.
>>
>>Then, of course, there are a lot of limitations on size, number of
>>devices, and so on in 0.90.
>
> but it is the only format supporting autodetection.
>
> So - when will autodetection be introduced with 1.X? And if not, why not?
>
> All I found was 'autodetection might be troublesome' and nothing else.
>  But dealing with initrds is troublesome too. Pure evil even.
>
> Glück Auf,
> Volker
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

I remember hearing that 1.x had /no/ plans for kernel level
auto-detection ever.  That can be accomplished in early-userspace
leaving the code in the kernel much less complex, and therefore far
more reliable.

In other words, 'auto-detection' for 1.x format devices is using an
initrd/initramfs.

Re: Linux mdadm superblock question.

[david]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1542 bytes --]

On Sat, 13 Feb 2010, Michael Evans wrote:

> On Sat, Feb 13, 2010 at 5:51 PM, Volker Armin Hemmann
> <volkerarmin@googlemail.com> wrote:
>>> 0.90 has a very bad problem, which is that it is hard to distinguish
>>> between a RAID partition at the end of volume and a full RAID device.
>>> This is because 0.90 doesn't actually tell you the start of the device.
>>>
>>> Then, of course, there are a lot of limitations on size, number of
>>> devices, and so on in 0.90.
>>
>> but it is the only format supporting autodetection.
>>
>> So - when will autodetection be introduced with 1.X? And if not, why not?
>>
>> All I found was 'autodetection might be troublesome' and nothing else.
>>  But dealing with initrds is troublesome too. Pure evil even.
>>
>> Gl?ck Auf,
>> Volker
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>
> I remember hearing that 1.x had /no/ plans for kernel level
> auto-detection ever.  That can be accomplished in early-userspace
> leaving the code in the kernel much less complex, and therefore far
> more reliable.
>
> In other words, 'auto-detection' for 1.x format devices is using an
> initrd/initramfs.

hmm, I've used 1.x formats without an initrd/initramfs (and without any 
conifg file on the server) and have had no problem with them being 
discovered. I haven't tried to use one for a boot/root device, so that may 
be the difference.

David Lang

Re: Linux mdadm superblock question.

[Michael Evans]

On Sat, Feb 13, 2010 at 11:21 PM,  <david@lang.hm> wrote:
> On Sat, 13 Feb 2010, Michael Evans wrote:
>
>> On Sat, Feb 13, 2010 at 5:51 PM, Volker Armin Hemmann
>> <volkerarmin@googlemail.com> wrote:
>>>>
>>>> 0.90 has a very bad problem, which is that it is hard to distinguish
>>>> between a RAID partition at the end of volume and a full RAID device.
>>>> This is because 0.90 doesn't actually tell you the start of the device.
>>>>
>>>> Then, of course, there are a lot of limitations on size, number of
>>>> devices, and so on in 0.90.
>>>
>>> but it is the only format supporting autodetection.
>>>
>>> So - when will autodetection be introduced with 1.X? And if not, why not?
>>>
>>> All I found was 'autodetection might be troublesome' and nothing else.
>>>  But dealing with initrds is troublesome too. Pure evil even.
>>>
>>> Gl?ck Auf,
>>> Volker
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>
>>
>> I remember hearing that 1.x had /no/ plans for kernel level
>> auto-detection ever.  That can be accomplished in early-userspace
>> leaving the code in the kernel much less complex, and therefore far
>> more reliable.
>>
>> In other words, 'auto-detection' for 1.x format devices is using an
>> initrd/initramfs.
>
> hmm, I've used 1.x formats without an initrd/initramfs (and without any
> conifg file on the server) and have had no problem with them being
> discovered. I haven't tried to use one for a boot/root device, so that may
> be the difference.
>
> David Lang

Yes, that is the difference.  You must have a more traditional simple
block device and filesystem drivers compiled in.  You have no need for
extra drivers or higher level device detection and evaluation (with
user-set policies to determine operation).  Anything past root-fs
mount can happen in normal user-space before logins are enabled.

Re: Linux mdadm superblock question.

[Volker Armin Hemmann]

On Sonntag 14 Februar 2010, you wrote:

> 
> In other words, 'auto-detection' for 1.x format devices is using an
> initrd/initramfs.

which makes 1.x format useless for everybody who does not want to deal with 
initrd/initramfs.

Glück Auf,
Volker

Re: Linux mdadm superblock question.

[John Robinson]

On 14/02/2010 18:40, Volker Armin Hemmann wrote:
> On Sonntag 14 Februar 2010, you wrote:
> 
>> In other words, 'auto-detection' for 1.x format devices is using an
>> initrd/initramfs.
> 
> which makes 1.x format useless for everybody who does not want to deal with 
> initrd/initramfs.

True, but afaik every distro uses an initrd/initramfs and bundles tools 
making it easy to manage and customise them, so what's the problem?

Cheers,

John.

Re: Linux mdadm superblock question.

[Henrique de Moraes Holschuh]

On Sat, 13 Feb 2010, Michael Evans wrote:
> I remember hearing that 1.x had /no/ plans for kernel level
> auto-detection ever.  That can be accomplished in early-userspace
> leaving the code in the kernel much less complex, and therefore far
> more reliable.

Yes, it is far more reliable kernel side, if only because it doesn't do
anything.

But the userspace reliability is _not_ good.  initrds are a source of
problems the moment things start to go wrong, and that's when they are not
the problem themselves.

And the end result is a system that needs manual intervention to get its
root filesystem back.

In my experience, every time we moved critical codepaths to userspace, we
ended up decreasing the *overall* system reliability.

-- 
  "One disk to rule them all, One disk to find them. One disk to bring
  them all and in the darkness grind them. In the Land of Redmond
  where the shadows lie." -- The Silicon Valley Tarot
  Henrique Holschuh

Re: Linux mdadm superblock question.

[Michael Evans]

On Sun, Feb 14, 2010 at 11:34 AM, Henrique de Moraes Holschuh
<hmh@hmh.eng.br> wrote:
> On Sat, 13 Feb 2010, Michael Evans wrote:
>> I remember hearing that 1.x had /no/ plans for kernel level
>> auto-detection ever.  That can be accomplished in early-userspace
>> leaving the code in the kernel much less complex, and therefore far
>> more reliable.
>
> Yes, it is far more reliable kernel side, if only because it doesn't do
> anything.
>
> But the userspace reliability is _not_ good.  initrds are a source of
> problems the moment things start to go wrong, and that's when they are not
> the problem themselves.
>
> And the end result is a system that needs manual intervention to get its
> root filesystem back.
>
> In my experience, every time we moved critical codepaths to userspace, we
> ended up decreasing the *overall* system reliability.
>
> --
>  "One disk to rule them all, One disk to find them. One disk to bring
>  them all and in the darkness grind them. In the Land of Redmond
>  where the shadows lie." -- The Silicon Valley Tarot
>  Henrique Holschuh
>

Maybe you'd like a simple, easy to customize initramfs creator.
That's exactly what I was aiming for when I made AEUIO
https://sourceforge.net/projects/aeuio  There are some things that
could use improvement, but if your system can boot without loading
modules it should be more than sufficient even across kernel versions.

Re: Linux mdadm superblock question.

[Asdo]

Henrique de Moraes Holschuh wrote:
> On Sat, 13 Feb 2010, Michael Evans wrote:
>   
>> I remember hearing that 1.x had /no/ plans for kernel level
>> auto-detection ever.  That can be accomplished in early-userspace
>> leaving the code in the kernel much less complex, and therefore far
>> more reliable.
>>     
>
> Yes, it is far more reliable kernel side, if only because it doesn't do
> anything.
>
> But the userspace reliability is _not_ good.  initrds are a source of
> problems the moment things start to go wrong, and that's when they are not
> the problem themselves.
>
> And the end result is a system that needs manual intervention to get its
> root filesystem back.
>
> In my experience, every time we moved critical codepaths to userspace, we
> ended up decreasing the *overall* system reliability.
>   
I don't see it like this.
You have the same chance to screw up the system by making mistakes in 
the files in /etc, in the networking config, the firewall, the server 
applications...
(note: I speak for Debian/Ubuntu, redhat's initramfs I think is more messy.)
1.x autodetection worked great for me in initramfs. Basically you only 
need /etc/mdadm/mdadm.conf copied to initramfs (via update-initramfs), 
the rest is done by Debian/Ubuntu standard initramfs procedure.
Also consider 1.x allows to choose which arrays are autoassembled 
(hostname written in the array name equal to hostname in the machine or 
specified in mdadm.conf): this is more precise than 0.9 which 
autoassembles all, I think.

Re: Linux mdadm superblock question.

[Henrique de Moraes Holschuh]

On Sun, 14 Feb 2010, Michael Evans wrote:
> Maybe you'd like a simple, easy to customize initramfs creator.

If the distro doesn't use it (as its default initramfs creator, even), it is
a lot more chance for breakage.  Less testing, and all that...

And if I am deploying a specific kernel in a server, you better believe it
is important enough that all due care will be taken so that it won't need an
initrd to mount the root filesystem to begin with ;-)

-- 
  "One disk to rule them all, One disk to find them. One disk to bring
  them all and in the darkness grind them. In the Land of Redmond
  where the shadows lie." -- The Silicon Valley Tarot
  Henrique Holschuh

Re: Linux mdadm superblock question.

[Gabor Gombas]

On Sun, Feb 14, 2010 at 06:53:38PM +0000, John Robinson wrote:

> True, but afaik every distro uses an initrd/initramfs and bundles
> tools making it easy to manage and customise them, so what's the
> problem?

Distro provided initramfs generators have a bad habit assuming you
patch/build your kernel like the distro does. If you want to use a
vanilla kernel with different things built in/built as modules/not built
at all, then you can get nasty surprises, and debugging can be rather
painful.

My current view is if you use a distro kernel, then you should also use
an initramfs (in fact you do not have a choice). But if you want to
build your own kernel, then you should get rid of the initramfs.

Gabor

Re: Linux mdadm superblock question.

[Henrique de Moraes Holschuh]

On Sun, 14 Feb 2010, Asdo wrote:
> >In my experience, every time we moved critical codepaths to userspace, we
> >ended up decreasing the *overall* system reliability.
> I don't see it like this.
> You have the same chance to screw up the system by making mistakes
> in the files in /etc, in the networking config, the firewall, the
> server applications...

Those don't require a reboot test to verify, and are far easier to rollback.

Also, they can (and SHOULD) be done on testbeds.  While the kind of screwup
where an initramfs decides to bite you hard, usually cannot (they tend to
happen when things already went horribly wrong).

> (note: I speak for Debian/Ubuntu, redhat's initramfs I think is more messy.)
> 1.x autodetection worked great for me in initramfs. Basically you
> only need /etc/mdadm/mdadm.conf copied to initramfs (via
> update-initramfs), the rest is done by Debian/Ubuntu standard
> initramfs procedure.

Yeah, cute.  What happens when the initrd is not updated for whatever
reason?  That is a new failure mode that doesn't exist with 0.9 and kernel
autorun.

It boils down to whether failure modes new to 1.x without autorun are more
likely to happen than the failure modes that are specific to 0.9 with
autorun.

IME, the 0.9 ones are less likely to happen, and I have been through quite a
few incidents involving boot problems.  Experience told me that initrds are
far more prone to operator errors than the kernel autorun.  Debian's
*stable* initramfs creators have not screwed up on me yet, but I am well
aware that they could.

> Also consider 1.x allows to choose which arrays are autoassembled
> (hostname written in the array name equal to hostname in the machine
> or specified in mdadm.conf): this is more precise than 0.9 which
> autoassembles all, I think.

That can be either a good or bad thing depending on the situation, so I
would never use it to count for (or against) 1.x or 0.9.

-- 
  "One disk to rule them all, One disk to find them. One disk to bring
  them all and in the darkness grind them. In the Land of Redmond
  where the shadows lie." -- The Silicon Valley Tarot
  Henrique Holschuh

Re: Linux mdadm superblock question.

[Gabor Gombas]

On Sun, Feb 14, 2010 at 09:47:16PM +0100, Asdo wrote:

> 1.x autodetection worked great for me in initramfs. Basically you
> only need /etc/mdadm/mdadm.conf copied to initramfs (via
> update-initramfs),

There is no autodetection with 1.1. Once you have mdadm.conf you have
pretty hard rules about what to look for and how to assemble it - ie.
there is not much left to "auto" detect. Real autodetection would mean
there is _no_ such information available, and you figure out everything
by just looking at the devices you find.

> initramfs procedure.
> Also consider 1.x allows to choose which arrays are autoassembled
> (hostname written in the array name equal to hostname in the machine
> or specified in mdadm.conf): this is more precise than 0.9 which
> autoassembles all, I think.

And also causes much more pain when you install machines on an internal
network where it gets a random name (in fact all new machines get the
same temporary name), then it is moved to its real location and
reconfigured with its real name. And you wonder why your arrays aren't
assembled any more...

Gabor

Re: Linux mdadm superblock question.

[Luca Berra]

On Sun, Feb 14, 2010 at 02:51:59AM +0100, Volker Armin Hemmann wrote:
>>0.90 has a very bad problem, which is that it is hard to distinguish
>>between a RAID partition at the end of volume and a full RAID device.
>>This is because 0.90 doesn't actually tell you the start of the device.
>>
>>Then, of course, there are a lot of limitations on size, number of
>>devices, and so on in 0.90.
>
>but it is the only format supporting autodetection. 
>
>So - when will autodetection be introduced with 1.X? And if not, why not?
>
>All I found was 'autodetection might be troublesome' and nothing else.
Well the discussions that lead to deprecation of in-kernel autodetection
date back to 2005 or earlier, so it's not easy to find.
iirc the main problem was that autodetection used the super minor to
name the device and it got very confused if you moved a drive from a
system to another.

mdadm based autodetection is reliable since you can associate uuid with
device name in mdadm.conf

> But dealing with initrds is troublesome too. Pure evil even. 
not really much more complex than getting kernel to do the right thing.
and it is orders of magnitude easier to debug than kernel code.

L.


-- 
Luca Berra -- bluca@comedia.it
         Communication Media & Services S.r.l.
  /"\
  \ /     ASCII RIBBON CAMPAIGN
   X        AGAINST HTML MAIL
  / \

Re: Linux mdadm superblock question.

[martin f krafft]

[-- Attachment #1: Type: text/plain, Size: 870 bytes --]

also sprach Gabor Gombas <gombasg@digikabel.hu> [2010.02.15.1028 +1300]:
> There is no autodetection with 1.1. Once you have mdadm.conf you have
> pretty hard rules about what to look for and how to assemble it - ie.
> there is not much left to "auto" detect. Real autodetection would mean
> there is _no_ such information available, and you figure out everything
> by just looking at the devices you find.

Which, coincidentally, is where we're heading with incremental
assembly. Check the Debian experimental package if you want to try.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
"we should have a volleyballocracy.
 we elect a six-pack of presidents.
 each one serves until they screw up,
 at which point they rotate."
                                                      -- dennis miller
 
spamtraps: madduck.bogus@madduck.net

[-- Attachment #2: Digital signature (see http://martin-krafft.net/gpg/) --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Linux mdadm superblock question.

[John Robinson]

On 14/02/2010 19:13, Volker Armin Hemmann wrote:
> On Sonntag 14 Februar 2010, you wrote:
>> On 14/02/2010 18:40, Volker Armin Hemmann wrote:
>>> On Sonntag 14 Februar 2010, you wrote:
>>>> In other words, 'auto-detection' for 1.x format devices is using an
>>>> initrd/initramfs.
>>> which makes 1.x format useless for everybody who does not want to deal
>>> with initrd/initramfs.
>> True, but afaik every distro uses an initrd/initramfs and bundles tools
>> making it easy to manage and customise them, so what's the problem?
> 
> and distros do it because of all the drivers they have to ship. But for 
> example I am not bound by such limitations. Why should I deal with that?
> It is hard enough not to forget 'make modules_install'. And now add initrd. 
> Autodetecting just works - but if you use an initrd an it doesn't. Where do 
> you start?
> 
> Initrd's maybe great for distro packagers, but are they really usefull for 
> anybody else?

Not just for distro packagers, they're useful for distro users, which 
are presumably 99% of Linux users these days, including the vast 
majority of enterprise users who like tested, supported systems.

But even for people building their own kernels, initrd/initramfs are 
useful if you're using LVM, or indeed trying to boot off anything that's 
not a simple device.

Cheers,

John.

Re: Linux mdadm superblock question.

[Volker Armin Hemmann]

On Dienstag 16 Februar 2010, John Robinson wrote:
> On 14/02/2010 19:13, Volker Armin Hemmann wrote:
> > On Sonntag 14 Februar 2010, you wrote:
> >> On 14/02/2010 18:40, Volker Armin Hemmann wrote:
> >>> On Sonntag 14 Februar 2010, you wrote:
> >>>> In other words, 'auto-detection' for 1.x format devices is using an
> >>>> initrd/initramfs.
> >>> 
> >>> which makes 1.x format useless for everybody who does not want to deal
> >>> with initrd/initramfs.
> >> 
> >> True, but afaik every distro uses an initrd/initramfs and bundles tools
> >> making it easy to manage and customise them, so what's the problem?
> > 
> > and distros do it because of all the drivers they have to ship. But for
> > example I am not bound by such limitations. Why should I deal with that?
> > It is hard enough not to forget 'make modules_install'. And now add
> > initrd. Autodetecting just works - but if you use an initrd an it
> > doesn't. Where do you start?
> > 
> > Initrd's maybe great for distro packagers, but are they really usefull
> > for anybody else?
> 
> Not just for distro packagers, they're useful for distro users, which
> are presumably 99% of Linux users these days, including the vast
> majority of enterprise users who like tested, supported systems.
> 
> But even for people building their own kernels, initrd/initramfs are
> useful if you're using LVM, or indeed trying to boot off anything that's
> not a simple device.

so assume you have an initrd and metadata 1.x without auto assembling.

You do some changes to the raid and screw up something else. Next boot nothing 
works. Mostly because the mdadm.conf in your initrd is not correct.

You whip out your trusty usb stick with a resuce system - and you are stuck. 
If autoassembling would work, you would have working md devices you could 
mount and edit the files you have to. But you don't and the mdadm.conf in the 
initrd is outdated.

Sounds like 'you are screwed'.

Or you have that famous grub boot line to have root autoassembled but the 
device names changed. 

Yeah, sounds really great.

And that because ...? Is there any good reason not to have autoassmbling in 
the kernel?

Glück Auf
Volker

Re: Linux mdadm superblock question.

[Robin Hill]

[-- Attachment #1: Type: text/plain, Size: 1078 bytes --]

On Tue Feb 16, 2010 at 03:37:08PM +0100, Volker Armin Hemmann wrote:

> so assume you have an initrd and metadata 1.x without auto assembling.
> 
> You do some changes to the raid and screw up something else. Next boot nothing 
> works. Mostly because the mdadm.conf in your initrd is not correct.
> 
> You whip out your trusty usb stick with a resuce system - and you are stuck. 
> If autoassembling would work, you would have working md devices you could 
> mount and edit the files you have to. But you don't and the mdadm.conf in the 
> initrd is outdated.
> 
> Sounds like 'you are screwed'.
> 
Or you drop to the recovery shell in the initrd, edit the mdadm.conf
file (or run mdadm with whatever options you like to find out what's
actually happening), and continue with processing the init script and
booting.

HTH,
    Robin
-- 
     ___        
    ( ' }     |       Robin Hill        <robin@robinhill.me.uk> |
   / / )      | Little Jim says ....                            |
  // !!       |      "He fallen in de water !!"                 |

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: Linux mdadm superblock question.

[Bill Davidsen]

Volker Armin Hemmann wrote:
> On Sonntag 14 Februar 2010, you wrote:
>
>   
>> In other words, 'auto-detection' for 1.x format devices is using an
>> initrd/initramfs.
>>     
>
> which makes 1.x format useless for everybody who does not want to deal with 
> initrd/initramfs.
>   

You make this sound like some major big deal. are you running your own 
distribution? In most cases mkinitrd does the right thing when you "make 
install" the kernel, and if you are doing something in the build so 
complex that it needs options, you really should understand the options 
and be sure you're doing what you want.

Generally this involves preloading a module or two, and if you need it 
every time you probably should have built it in, anyway.

My opinion...

-- 
Bill Davidsen <davidsen@tmr.com>
  "We can't solve today's problems by using the same thinking we
   used in creating them." - Einstein

Re: Linux mdadm superblock question.

[John Robinson]

On 16/02/2010 14:37, Volker Armin Hemmann wrote:
> On Dienstag 16 Februar 2010, John Robinson wrote:
>> On 14/02/2010 19:13, Volker Armin Hemmann wrote:
>>> On Sonntag 14 Februar 2010, you wrote:
>>>> On 14/02/2010 18:40, Volker Armin Hemmann wrote:
>>>>> On Sonntag 14 Februar 2010, you wrote:
>>>>>> In other words, 'auto-detection' for 1.x format devices is using an
>>>>>> initrd/initramfs.
>>>>> which makes 1.x format useless for everybody who does not want to deal
>>>>> with initrd/initramfs.
>>>> True, but afaik every distro uses an initrd/initramfs and bundles tools
>>>> making it easy to manage and customise them, so what's the problem?
>>> and distros do it because of all the drivers they have to ship. But for
>>> example I am not bound by such limitations. Why should I deal with that?
>>> It is hard enough not to forget 'make modules_install'. And now add
>>> initrd. Autodetecting just works - but if you use an initrd an it
>>> doesn't. Where do you start?
>>>
>>> Initrd's maybe great for distro packagers, but are they really usefull
>>> for anybody else?
>> Not just for distro packagers, they're useful for distro users, which
>> are presumably 99% of Linux users these days, including the vast
>> majority of enterprise users who like tested, supported systems.
>>
>> But even for people building their own kernels, initrd/initramfs are
>> useful if you're using LVM, or indeed trying to boot off anything that's
>> not a simple device.
> 
> so assume you have an initrd and metadata 1.x without auto assembling.
> 
> You do some changes to the raid and screw up something else. Next boot nothing 
> works. Mostly because the mdadm.conf in your initrd is not correct.
> 
> You whip out your trusty usb stick with a resuce system - and you are stuck. 
> If autoassembling would work, you would have working md devices you could 
> mount and edit the files you have to. But you don't and the mdadm.conf in the 
> initrd is outdated.
> 
> Sounds like 'you are screwed'.

No; mdadm can assemble arrays without needing a conf file (at least 
arrays which have superblocks).

And if you have otherwise screwed something up with the RAID, no amount 
of in-kernel autoassembly is going to help, in fact it's more likely to 
get it wrong and make things worse; you need a command line and mdadm to 
sort it out.

Cheers,

John.

Re: Linux mdadm superblock question.

[Luca Berra]

On Tue, Feb 16, 2010 at 03:37:08PM +0100, Volker Armin Hemmann wrote:
>so assume you have an initrd and metadata 1.x without auto assembling.
>
>You do some changes to the raid and screw up something else. Next boot nothing 
>works. Mostly because the mdadm.conf in your initrd is not correct.
>
>You whip out your trusty usb stick with a resuce system - and you are stuck. 
>If autoassembling would work, you would have working md devices you could 
>mount and edit the files you have to. But you don't and the mdadm.conf in the 
>initrd is outdated.
>
>Sounds like 'you are screwed'.

did you ever try this?
mdadm is able to activate an array without need of any configuration
file.
The reason a configuration file is copied in initrd is solely to
preserve non-default activation parameters (i.e. array name) and such.

>And that because ...? Is there any good reason not to have autoassmbling in 
>the kernel?

about the same reason why the X server is not in the kernel as well.

L.

-- 
Luca Berra -- bluca@comedia.it
         Communication Media & Services S.r.l.
  /"\
  \ /     ASCII RIBBON CAMPAIGN
   X        AGAINST HTML MAIL
  / \

Re: Linux mdadm superblock question.

[Volker Armin Hemmann]

On Dienstag 16 Februar 2010, Bill Davidsen wrote:
> Volker Armin Hemmann wrote:
> > On Sonntag 14 Februar 2010, you wrote:
> >> In other words, 'auto-detection' for 1.x format devices is using an
> >> initrd/initramfs.
> > 
> > which makes 1.x format useless for everybody who does not want to deal
> > with initrd/initramfs.
> 
> You make this sound like some major big deal. are you running your own
> distribution? In most cases mkinitrd does the right thing when you "make
> install" the kernel, and if you are doing something in the build so
> complex that it needs options, you really should understand the options
> and be sure you're doing what you want.
> 
> Generally this involves preloading a module or two, and if you need it
> every time you probably should have built it in, anyway.
> 
> My opinion...

I am running my own kernels - and of course everything that is needed to boot 
and get the basic system up is built in. Why should I make the disk drivers 
modules? 
That does not make sense.

And the reason is simple: even when the system is completely fucked up, I want 
a kernel that is able to boot until init=/bin/bb takes over.

Glück Auf
Volker

Re: Linux mdadm superblock question.

[Nick Bowler]

On 22:06 Tue 16 Feb     , Volker Armin Hemmann wrote:
> On Dienstag 16 Februar 2010, Bill Davidsen wrote:
> > Volker Armin Hemmann wrote:
> > > On Sonntag 14 Februar 2010, you wrote:
> > >> In other words, 'auto-detection' for 1.x format devices is using an
> > >> initrd/initramfs.
> > > 
> > > which makes 1.x format useless for everybody who does not want to deal
> > > with initrd/initramfs.
> > 
> > You make this sound like some major big deal. are you running your own
> > distribution? In most cases mkinitrd does the right thing when you "make
> > install" the kernel, and if you are doing something in the build so
> > complex that it needs options, you really should understand the options
> > and be sure you're doing what you want.
> > 
> > Generally this involves preloading a module or two, and if you need it
> > every time you probably should have built it in, anyway.
> > 
> > My opinion...
> 
> I am running my own kernels - and of course everything that is needed to boot 
> and get the basic system up is built in. Why should I make the disk drivers 
> modules? 
> That does not make sense.

I agree that it makes little sense to make something a module when you
can't unload it anyway, but...

> And the reason is simple: even when the system is completely fucked up, I want 
> a kernel that is able to boot until init=/bin/bb takes over.

I put a complete set of recovery tools into my initramfses so that when
the system is completely fucked up, I have a kernel that is able to boot
until rdinit=/bin/zsh (or /bin/bb, if you prefer) takes over.

This has the added advantage of working when the root filesystem cannot
be mounted at all: a scenario which does not seem too far-fetched when
the filesystem is located on a raid array.

-- 
Nick Bowler, Elliptic Technologies (http://www.elliptictech.com/)

Re: Linux mdadm superblock question.

[Volker Armin Hemmann]

On Dienstag 16 Februar 2010, Nick Bowler wrote:
> On 22:06 Tue 16 Feb     , Volker Armin Hemmann wrote:
> > On Dienstag 16 Februar 2010, Bill Davidsen wrote:
> > > Volker Armin Hemmann wrote:
> > > > On Sonntag 14 Februar 2010, you wrote:
> > > >> In other words, 'auto-detection' for 1.x format devices is using an
> > > >> initrd/initramfs.
> > > > 
> > > > which makes 1.x format useless for everybody who does not want to
> > > > deal with initrd/initramfs.
> > > 
> > > You make this sound like some major big deal. are you running your own
> > > distribution? In most cases mkinitrd does the right thing when you
> > > "make install" the kernel, and if you are doing something in the build
> > > so complex that it needs options, you really should understand the
> > > options and be sure you're doing what you want.
> > > 
> > > Generally this involves preloading a module or two, and if you need it
> > > every time you probably should have built it in, anyway.
> > > 
> > > My opinion...
> > 
> > I am running my own kernels - and of course everything that is needed to
> > boot and get the basic system up is built in. Why should I make the disk
> > drivers modules?
> > That does not make sense.
> 
> I agree that it makes little sense to make something a module when you
> can't unload it anyway, but...
> 
> > And the reason is simple: even when the system is completely fucked up, I
> > want a kernel that is able to boot until init=/bin/bb takes over.
> 
> I put a complete set of recovery tools into my initramfses so that when
> the system is completely fucked up, I have a kernel that is able to boot
> until rdinit=/bin/zsh (or /bin/bb, if you prefer) takes over.
> 
> This has the added advantage of working when the root filesystem cannot
> be mounted at all: a scenario which does not seem too far-fetched when
> the filesystem is located on a raid array.

and what do you do if you have to boot from a cd/usb stick and need to access 
the raid?

Simple with auto assembling. Not so much without.

Glück Auf,
Volker

Re: Linux mdadm superblock question.

[Mr. James W. Laferriere]

 	Hello Bill ,

On Tue, 16 Feb 2010, Bill Davidsen wrote:
> Volker Armin Hemmann wrote:
>> On Sonntag 14 Februar 2010, you wrote:
>>> In other words, 'auto-detection' for 1.x format devices is using an
>>> initrd/initramfs.
>>
>> which makes 1.x format useless for everybody who does not want to deal with 
>> initrd/initramfs.
>
> You make this sound like some major big deal. are you running your own 
> distribution? In most cases mkinitrd does the right thing when you "make 
> install" the kernel, and if you are doing something in the build so complex 
> that it needs options, you really should understand the options and be sure 
> you're doing what you want.
>
> Generally this involves preloading a module or two, and if you need it every 
> time you probably should have built it in, anyway.
>
> My opinion...
 	My Opinion as well .  That is one of the many reasons why I have my '/' 
autoassemble .  And do to this I am permanently stuck at 0.90 version of the 
raid table .  No big shakes for that .  But at sometime in the past there was a 
discussion to have the 0.90 raid table be removed ,  NOW THAT SCARES THE H?LL 
OUT OF ME .  So far Neil has not done so .

 	I am unaware of any record from Neil or other maintainer(s) of the 
/md/ device tree saying that they will not remove the 0.90 table and the 
autoassembly functions there .  I'd very much like to hear a statement saying 
there will not be a removal of the autoassembly functions for 0.90 raid table 
from the kernel tree .

 		Tia ,  JimL
-- 
+------------------------------------------------------------------+
| James   W.   Laferriere | System    Techniques | Give me VMS     |
| Network&System Engineer | 3237     Holden Road |  Give me Linux  |
| babydr@baby-dragons.com | Fairbanks, AK. 99709 |   only  on  AXP |
+------------------------------------------------------------------+

Re: Linux mdadm superblock question.

[Neil Brown]

On Tue, 16 Feb 2010 16:03:43 -0900 (AKST)
"Mr. James W. Laferriere" <babydr@baby-dragons.com> wrote:

>  	Hello Bill ,
> 
> On Tue, 16 Feb 2010, Bill Davidsen wrote:
> > Volker Armin Hemmann wrote:
> >> On Sonntag 14 Februar 2010, you wrote:
> >>> In other words, 'auto-detection' for 1.x format devices is using an
> >>> initrd/initramfs.
> >>
> >> which makes 1.x format useless for everybody who does not want to deal with 
> >> initrd/initramfs.
> >
> > You make this sound like some major big deal. are you running your own 
> > distribution? In most cases mkinitrd does the right thing when you "make 
> > install" the kernel, and if you are doing something in the build so complex 
> > that it needs options, you really should understand the options and be sure 
> > you're doing what you want.
> >
> > Generally this involves preloading a module or two, and if you need it every 
> > time you probably should have built it in, anyway.
> >
> > My opinion...
>  	My Opinion as well .  That is one of the many reasons why I have my '/' 
> autoassemble .  And do to this I am permanently stuck at 0.90 version of the 
> raid table .  No big shakes for that .  But at sometime in the past there was a 
> discussion to have the 0.90 raid table be removed ,  NOW THAT SCARES THE H?LL 
> OUT OF ME .  So far Neil has not done so .
> 
>  	I am unaware of any record from Neil or other maintainer(s) of the 
> /md/ device tree saying that they will not remove the 0.90 table and the 
> autoassembly functions there .  I'd very much like to hear a statement saying 
> there will not be a removal of the autoassembly functions for 0.90 raid table 
> from the kernel tree .

I will not be removing 0.90 or auto-assemble from the kernel in the
foreseeable future.
None the less, I recommend weaning yourself from your dependence on it.
initramfs is the future, embrace it.

NeilBrown

Re: Linux mdadm superblock question.

[Volker Armin Hemmann]

On Mittwoch 17 Februar 2010, Neil Brown wrote:

> I will not be removing 0.90 or auto-assemble from the kernel in the
> foreseeable future.
> None the less, I recommend weaning yourself from your dependence on it.
> initramfs is the future, embrace it.
> 
> NeilBrown

a future that is worse than the present. For what reason?

Glück Auf,
Volker
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Linux mdadm superblock question.

[Volker Armin Hemmann]

On Mittwoch 17 Februar 2010, Neil Brown wrote:

> I will not be removing 0.90 or auto-assemble from the kernel in the
> foreseeable future.
> None the less, I recommend weaning yourself from your dependence on it.
> initramfs is the future, embrace it.
> 
> NeilBrown

a future that is worse than the present. For what reason?

Glück Auf,
Volker

Re: Linux mdadm superblock question.

[Kyle Moffett]

On Tue, Feb 16, 2010 at 21:01, Neil Brown <neilb@suse.de> wrote:
> On Tue, 16 Feb 2010 16:03:43 -0900 (AKST) "Mr. James W. Laferriere" <babydr@baby-dragons.com> wrote:
>>       I am unaware of any record from Neil or other maintainer(s) of the
>> /md/ device tree saying that they will not remove the 0.90 table and the
>> autoassembly functions there .  I'd very much like to hear a statement saying
>> there will not be a removal of the autoassembly functions for 0.90 raid table
>> from the kernel tree .
>
> I will not be removing 0.90 or auto-assemble from the kernel in the
> foreseeable future.
> None the less, I recommend weaning yourself from your dependence on it.
> initramfs is the future, embrace it.

What are people's reasons for pushback against initramfs?  I've heard
lots of claims that "it's not trustworthy" and "it breaks", but in 7
years of running bootable software RAID boxes on weird architectures
(even running Debian unstable) I have only once or twice had initramfs
problems.

As a software capability, initramfs makes it possible to use
*anything* as a root filesystem, no matter what is necessary to set it
up.  For example, I have seen somebody use DRBD (essentially network
RAID-1) as a root filesystem with a few custom hook scripts added to
the initramfs-tools configs.  Other examples include using Sun ZFS as
a root fs via an initramfs FUSE daemon, a feat which even Solaris
could not accomplish at the time.  Encrypted root filesystems also
require an initramfs to prompt for encryption keys and decrypt the
block device.  Multipath block devices are another example.

You should also take a look at your distro installers.  There is not a
single one made in the last several years which does not use an
initramfs to start networking or access the installation media.  In
fact, of all the distro installers I have had the most consistent
behavior regardless of system hardware from the ones which operate
entirely out of their initramfs.

From a reliability perspective, an initramfs is no more essential
than, say, /sbin/init or /boot/vmlinuz-2.6.33.  Furthermore, all of
the modern initramfs generation tools automatically keep backup copies
exactly the same way that "make install" keeps backup copies of your
kernel images.  The two times I've managed to hose my initramfs I was
able to simply edit my grub config to use a file called something like
"/boot/initramfs-2.6.33.bak" instead.

In fact, I have had several times where an initramfs made my boot
process *more* reliable.  On one of my LVM JBOD systems, I was able to
pull a group of 3 SATA drives whose backplane had failed and drop them
all in USB enclosures to get the system back up and running in a half
an hour.  With just straight partitions on the volumes I would have
been hunting around for 2 hours to figure out where all my partitions
had gone only to have the USB drives spin up in a different order
during the next reboot.

If you're really concerned about boot-process reliability, go ahead
and tell your initramfs tool to include a fully-featured busybox,
coreutils, bash, strace, gdb, and a half-dozen other developer tools.
You may wait an extra 20 seconds for your bootloader to load the damn
thing during boot, but you'll be able to track down that annoying
10-second hang in your /sbin/init program during config-file parsing.

I've built specialized embedded computers with stripped-down chipset
initialization code, a tiny Linux kernel and a special-purpose
initramfs burned into the flash.  By using the fastboot patches and
disabling all the excess drivers, my kernel was fully operational
within the first half-second.  It used the tools on the initramfs to
poke around on the hard disk as a bootloader, then kexec() to load the
operational kernel.

Counting up all the problems I've had with system boot... I've had an
order of magnitude more problems from somebody getting careless with
"rm", "dpkg --purge", etc than with initramfs deficiencies.

Cheers,
Kyle Moffett
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Linux mdadm superblock question.

[Kyle Moffett]

On Tue, Feb 16, 2010 at 21:01, Neil Brown <neilb@suse.de> wrote:
> On Tue, 16 Feb 2010 16:03:43 -0900 (AKST) "Mr. James W. Laferriere" <babydr@baby-dragons.com> wrote:
>>       I am unaware of any record from Neil or other maintainer(s) of the
>> /md/ device tree saying that they will not remove the 0.90 table and the
>> autoassembly functions there .  I'd very much like to hear a statement saying
>> there will not be a removal of the autoassembly functions for 0.90 raid table
>> from the kernel tree .
>
> I will not be removing 0.90 or auto-assemble from the kernel in the
> foreseeable future.
> None the less, I recommend weaning yourself from your dependence on it.
> initramfs is the future, embrace it.

What are people's reasons for pushback against initramfs?  I've heard
lots of claims that "it's not trustworthy" and "it breaks", but in 7
years of running bootable software RAID boxes on weird architectures
(even running Debian unstable) I have only once or twice had initramfs
problems.

As a software capability, initramfs makes it possible to use
*anything* as a root filesystem, no matter what is necessary to set it
up.  For example, I have seen somebody use DRBD (essentially network
RAID-1) as a root filesystem with a few custom hook scripts added to
the initramfs-tools configs.  Other examples include using Sun ZFS as
a root fs via an initramfs FUSE daemon, a feat which even Solaris
could not accomplish at the time.  Encrypted root filesystems also
require an initramfs to prompt for encryption keys and decrypt the
block device.  Multipath block devices are another example.

You should also take a look at your distro installers.  There is not a
single one made in the last several years which does not use an
initramfs to start networking or access the installation media.  In
fact, of all the distro installers I have had the most consistent
behavior regardless of system hardware from the ones which operate
entirely out of their initramfs.

>From a reliability perspective, an initramfs is no more essential
than, say, /sbin/init or /boot/vmlinuz-2.6.33.  Furthermore, all of
the modern initramfs generation tools automatically keep backup copies
exactly the same way that "make install" keeps backup copies of your
kernel images.  The two times I've managed to hose my initramfs I was
able to simply edit my grub config to use a file called something like
"/boot/initramfs-2.6.33.bak" instead.

In fact, I have had several times where an initramfs made my boot
process *more* reliable.  On one of my LVM JBOD systems, I was able to
pull a group of 3 SATA drives whose backplane had failed and drop them
all in USB enclosures to get the system back up and running in a half
an hour.  With just straight partitions on the volumes I would have
been hunting around for 2 hours to figure out where all my partitions
had gone only to have the USB drives spin up in a different order
during the next reboot.

If you're really concerned about boot-process reliability, go ahead
and tell your initramfs tool to include a fully-featured busybox,
coreutils, bash, strace, gdb, and a half-dozen other developer tools.
You may wait an extra 20 seconds for your bootloader to load the damn
thing during boot, but you'll be able to track down that annoying
10-second hang in your /sbin/init program during config-file parsing.

I've built specialized embedded computers with stripped-down chipset
initialization code, a tiny Linux kernel and a special-purpose
initramfs burned into the flash.  By using the fastboot patches and
disabling all the excess drivers, my kernel was fully operational
within the first half-second.  It used the tools on the initramfs to
poke around on the hard disk as a bootloader, then kexec() to load the
operational kernel.

Counting up all the problems I've had with system boot... I've had an
order of magnitude more problems from somebody getting careless with
"rm", "dpkg --purge", etc than with initramfs deficiencies.

Cheers,
Kyle Moffett

Re: Linux mdadm superblock question.

[Rudy Zijlstra]

Kyle Moffett wrote:
> On Tue, Feb 16, 2010 at 21:01, Neil Brown <neilb@suse.de> wrote:
>   
>> On Tue, 16 Feb 2010 16:03:43 -0900 (AKST) "Mr. James W. Laferriere" <babydr@baby-dragons.com> wrote:
>>     
>>>       I am unaware of any record from Neil or other maintainer(s) of the
>>> /md/ device tree saying that they will not remove the 0.90 table and the
>>> autoassembly functions there .  I'd very much like to hear a statement saying
>>> there will not be a removal of the autoassembly functions for 0.90 raid table
>>> from the kernel tree .
>>>       
>> I will not be removing 0.90 or auto-assemble from the kernel in the
>> foreseeable future.
>> None the less, I recommend weaning yourself from your dependence on it.
>> initramfs is the future, embrace it.
>>     
>
> What are people's reasons for pushback against initramfs?  I've heard
> lots of claims that "it's not trustworthy" and "it breaks", but in 7
> years of running bootable software RAID boxes on weird architectures
> (even running Debian unstable) I have only once or twice had initramfs
> problems.
>
> As a software capability, initramfs makes it possible to use
> *anything* as a root filesystem, no matter what is necessary to set it
> up.  For example, I have seen somebody use DRBD (essentially network
> RAID-1) as a root filesystem with a few custom hook scripts added to
> the initramfs-tools configs.  Other examples include using Sun ZFS as
> a root fs via an initramfs FUSE daemon, a feat which even Solaris
> could not accomplish at the time.  Encrypted root filesystems also
> require an initramfs to prompt for encryption keys and decrypt the
> block device.  Multipath block devices are another example.
>
> You should also take a look at your distro installers.  There is not a
> single one made in the last several years which does not use an
> initramfs to start networking or access the installation media.  In
> fact, of all the distro installers I have had the most consistent
> behavior regardless of system hardware from the ones which operate
> entirely out of their initramfs.
>
> From a reliability perspective, an initramfs is no more essential
> than, say, /sbin/init or /boot/vmlinuz-2.6.33.  Furthermore, all of
> the modern initramfs generation tools automatically keep backup copies
> exactly the same way that "make install" keeps backup copies of your
> kernel images.  The two times I've managed to hose my initramfs I was
> able to simply edit my grub config to use a file called something like
> "/boot/initramfs-2.6.33.bak" instead.
>
> In fact, I have had several times where an initramfs made my boot
> process *more* reliable.  On one of my LVM JBOD systems, I was able to
> pull a group of 3 SATA drives whose backplane had failed and drop them
> all in USB enclosures to get the system back up and running in a half
> an hour.  With just straight partitions on the volumes I would have
> been hunting around for 2 hours to figure out where all my partitions
> had gone only to have the USB drives spin up in a different order
> during the next reboot.
>
> If you're really concerned about boot-process reliability, go ahead
> and tell your initramfs tool to include a fully-featured busybox,
> coreutils, bash, strace, gdb, and a half-dozen other developer tools.
> You may wait an extra 20 seconds for your bootloader to load the damn
> thing during boot, but you'll be able to track down that annoying
> 10-second hang in your /sbin/init program during config-file parsing.
>
> I've built specialized embedded computers with stripped-down chipset
> initialization code, a tiny Linux kernel and a special-purpose
> initramfs burned into the flash.  By using the fastboot patches and
> disabling all the excess drivers, my kernel was fully operational
> within the first half-second.  It used the tools on the initramfs to
> poke around on the hard disk as a bootloader, then kexec() to load the
> operational kernel.
>
> Counting up all the problems I've had with system boot... I've had an
> order of magnitude more problems from somebody getting careless with
> "rm", "dpkg --purge", etc than with initramfs deficiencies.
>
>
>   
We are looking at 2 different use-cases i think.

for the power-user system manager, who manages all his servers and has 
knowledgeable backup, initrd may indeed work as above.

I have to keep in mind, that when there is a problem while i am 
travelling (and that happens), there is no sys-admin present. Also, i am 
supporting systems remote where no-one has the knowledge to debug using 
a initrd. In such cases, initrd is an additional step. And each 
additional step is an additional source of mistakes.

1/ distro tools assume that the kernel being build will run on that 
machine. For servers this is often not true. There are very valid 
security reasons to exclude compilation capability from many servers.
2/ For most small shops, there is need for RAID (disks are fallible, 
shop cannot do without server), the RAID should work without being 
visible. If there is a problem with the RAID that causes auto-assemble 
to break, it means i need to travel (>100KM) to trouble shoot. The 
simpler the setup, the more i like it. This is also why i almost always 
use HW raid for the system partitions. The ones i use have userland 
tools in Linux which warn on disk failure, ensure auto rebuild, etc...
Still, for large storage needs it is SW RAID over SATA.
3/ for my home systems, if i need to remote-support to get things 
working again (i am often travelling for my work), the added layer of 
initrd is an added layer of possible mistakes.



Cheers,


Rudy

Re: Linux mdadm superblock question.

[Frans Pop]

Rudy Zijlstra wrote:
> 1/ distro tools assume that the kernel being build will run on that
> machine. For servers this is often not true. There are very valid
> security reasons to exclude compilation capability from many servers.

That's simply not true, at least not for Debian. If you actually use the 
distro tools [1] the only assumptions are made at kernel *installation* 
time, not at kernel build time.

I've been using initramfs-tools generated initrds for years without 
problems, and that includes "root on LVM on LUKS encrypted partition" 
and "root on LVM on RAID" setups.

Cheers,
FJP

[1] I.e. if you build and install the kernel as a .deb package using e.g. 
the deb-pkg target or kernel-package.

Re: Linux mdadm superblock question.

[Nick Bowler]

On 23:18 Tue 16 Feb     , Volker Armin Hemmann wrote:
> On Dienstag 16 Februar 2010, Nick Bowler wrote:
> > I put a complete set of recovery tools into my initramfses so that when
> > the system is completely fucked up, I have a kernel that is able to boot
> > until rdinit=/bin/zsh (or /bin/bb, if you prefer) takes over.
> > 
> > This has the added advantage of working when the root filesystem cannot
> > be mounted at all: a scenario which does not seem too far-fetched when
> > the filesystem is located on a raid array.
> 
> and what do you do if you have to boot from a cd/usb stick and need to access 
> the raid?
> 
> Simple with auto assembling. Not so much without.

The same initramfs can be used on a CD or USB stick.  If you were
referring to using someone else's CD or USB stick, then obviously
mdadm will need to be available.

-- 
Nick Bowler, Elliptic Technologies (http://www.elliptictech.com/)

Re: Linux mdadm superblock question.

[Kyle Moffett]

On Wed, Feb 17, 2010 at 04:38, Rudy Zijlstra
<rudy@grumpydevil.homelinux.org> wrote:
> Kyle Moffett wrote:
>> On Tue, Feb 16, 2010 at 21:01, Neil Brown <neilb@suse.de> wrote:
>>> I will not be removing 0.90 or auto-assemble from the kernel in the
>>> foreseeable future.
>>> None the less, I recommend weaning yourself from your dependence on it.
>>> initramfs is the future, embrace it.
>>>
>>
>> What are people's reasons for pushback against initramfs?  I've heard
>> lots of claims that "it's not trustworthy" and "it breaks", but in 7
>> years of running bootable software RAID boxes on weird architectures
>> (even running Debian unstable) I have only once or twice had initramfs
>> problems.
>>
>> As a software capability, initramfs makes it possible to use
>> *anything* as a root filesystem, no matter what is necessary to set it
>> up.  For example, I have seen somebody use DRBD (essentially network
>> RAID-1) as a root filesystem with a few custom hook scripts added to
>> the initramfs-tools configs.  Other examples include using Sun ZFS as
>> a root fs via an initramfs FUSE daemon, a feat which even Solaris
>> could not accomplish at the time.  Encrypted root filesystems also
>> require an initramfs to prompt for encryption keys and decrypt the
>> block device.  Multipath block devices are another example.
>
> We are looking at 2 different use-cases i think.
>
> for the power-user system manager, who manages all his servers and has
> knowledgeable backup, initrd may indeed work as above.
>
> I have to keep in mind, that when there is a problem while i am travelling
> (and that happens), there is no sys-admin present. Also, i am supporting
> systems remote where no-one has the knowledge to debug using a initrd. In
> such cases, initrd is an additional step. And each additional step is an
> additional source of mistakes.

Actually, a properly built initramfs gives you far more reliable
behavior even without a local sysadmin.  For example, most
graphical-boot tools are designed to be built into an initramfs; I
have seen some prototype initramfs images which provide end-user
accessible GUI boot menus and other tools which function reliably even
when your root filesystem is inaccessible.

> 1/ distro tools assume that the kernel being build will run on that machine.
> For servers this is often not true. There are very valid security reasons to
> exclude compilation capability from many servers.

As Frans Pop states, this is entirely untrue for (at the very least)
Debian.  The "initramfs-tools" package present there works regardless
of where I obtain my kernel.  If I use the "make-kpkg" Debian tool
when building my kernel (regardless of where it is built), then the
resulting package will automatically generate an appropriate initramfs
image when installed.  If for some reason I install a kernel by hand I
can very trivially build the necessary initramfs with this command:

update-initramfs -c -k 2.6.32-mykernel01

In the event that you need to "customize" the initramfs for some
reason, you can simply do so.  When the "update-initramfs" tool is
next run, it will report that the user has customized that image and
avoid modifying it.  If you wish to return to the autogenerated
initramfs you can simply run this command:

update-initramfs -u -t -k 2.6.32-2-amd64

> 2/ For most small shops, there is need for RAID (disks are fallible, shop
> cannot do without server), the RAID should work without being visible. If
> there is a problem with the RAID that causes auto-assemble to break, it
> means i need to travel (>100KM) to trouble shoot. The simpler the setup, the
> more i like it. This is also why i almost always use HW raid for the system
> partitions. The ones i use have userland tools in Linux which warn on disk
> failure, ensure auto rebuild, etc...
> Still, for large storage needs it is SW RAID over SATA.
>
> 3/ for my home systems, if i need to remote-support to get things working
> again (i am often travelling for my work), the added layer of initrd is an
> added layer of possible mistakes.

You are actually just setting yourself up for problems.  RAID
autoassembly has bad corner cases when disks disappear between reboots
(which happens with failing disk head assemblies).  In that case it
will fail to find its root filesystem or wait forever for the last
disk to show up.  I have had even *worse* problems (including
corruption of unrelated logical volumes) with many hardware RAID
controllers, even those from big-name server vendors such as HP and
Dell.

To contrast, an initramfs is configurable to prompt the user,
automatically degrade the array after a small delay, or even play a
kazoo if desired :-D.  One of these days I may get around to building
myself a small GUI wrapper around mdadm on an initramfs which allows a
user to manually recover from RAID problems.

Cheers,
Kyle Moffett
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Linux mdadm superblock question.

[Kyle Moffett]

On Wed, Feb 17, 2010 at 04:38, Rudy Zijlstra
<rudy@grumpydevil.homelinux.org> wrote:
> Kyle Moffett wrote:
>> On Tue, Feb 16, 2010 at 21:01, Neil Brown <neilb@suse.de> wrote:
>>> I will not be removing 0.90 or auto-assemble from the kernel in the
>>> foreseeable future.
>>> None the less, I recommend weaning yourself from your dependence on it.
>>> initramfs is the future, embrace it.
>>>
>>
>> What are people's reasons for pushback against initramfs?  I've heard
>> lots of claims that "it's not trustworthy" and "it breaks", but in 7
>> years of running bootable software RAID boxes on weird architectures
>> (even running Debian unstable) I have only once or twice had initramfs
>> problems.
>>
>> As a software capability, initramfs makes it possible to use
>> *anything* as a root filesystem, no matter what is necessary to set it
>> up.  For example, I have seen somebody use DRBD (essentially network
>> RAID-1) as a root filesystem with a few custom hook scripts added to
>> the initramfs-tools configs.  Other examples include using Sun ZFS as
>> a root fs via an initramfs FUSE daemon, a feat which even Solaris
>> could not accomplish at the time.  Encrypted root filesystems also
>> require an initramfs to prompt for encryption keys and decrypt the
>> block device.  Multipath block devices are another example.
>
> We are looking at 2 different use-cases i think.
>
> for the power-user system manager, who manages all his servers and has
> knowledgeable backup, initrd may indeed work as above.
>
> I have to keep in mind, that when there is a problem while i am travelling
> (and that happens), there is no sys-admin present. Also, i am supporting
> systems remote where no-one has the knowledge to debug using a initrd. In
> such cases, initrd is an additional step. And each additional step is an
> additional source of mistakes.

Actually, a properly built initramfs gives you far more reliable
behavior even without a local sysadmin.  For example, most
graphical-boot tools are designed to be built into an initramfs; I
have seen some prototype initramfs images which provide end-user
accessible GUI boot menus and other tools which function reliably even
when your root filesystem is inaccessible.

> 1/ distro tools assume that the kernel being build will run on that machine.
> For servers this is often not true. There are very valid security reasons to
> exclude compilation capability from many servers.

As Frans Pop states, this is entirely untrue for (at the very least)
Debian.  The "initramfs-tools" package present there works regardless
of where I obtain my kernel.  If I use the "make-kpkg" Debian tool
when building my kernel (regardless of where it is built), then the
resulting package will automatically generate an appropriate initramfs
image when installed.  If for some reason I install a kernel by hand I
can very trivially build the necessary initramfs with this command:

update-initramfs -c -k 2.6.32-mykernel01

In the event that you need to "customize" the initramfs for some
reason, you can simply do so.  When the "update-initramfs" tool is
next run, it will report that the user has customized that image and
avoid modifying it.  If you wish to return to the autogenerated
initramfs you can simply run this command:

update-initramfs -u -t -k 2.6.32-2-amd64

> 2/ For most small shops, there is need for RAID (disks are fallible, shop
> cannot do without server), the RAID should work without being visible. If
> there is a problem with the RAID that causes auto-assemble to break, it
> means i need to travel (>100KM) to trouble shoot. The simpler the setup, the
> more i like it. This is also why i almost always use HW raid for the system
> partitions. The ones i use have userland tools in Linux which warn on disk
> failure, ensure auto rebuild, etc...
> Still, for large storage needs it is SW RAID over SATA.
>
> 3/ for my home systems, if i need to remote-support to get things working
> again (i am often travelling for my work), the added layer of initrd is an
> added layer of possible mistakes.

You are actually just setting yourself up for problems.  RAID
autoassembly has bad corner cases when disks disappear between reboots
(which happens with failing disk head assemblies).  In that case it
will fail to find its root filesystem or wait forever for the last
disk to show up.  I have had even *worse* problems (including
corruption of unrelated logical volumes) with many hardware RAID
controllers, even those from big-name server vendors such as HP and
Dell.

To contrast, an initramfs is configurable to prompt the user,
automatically degrade the array after a small delay, or even play a
kazoo if desired :-D.  One of these days I may get around to building
myself a small GUI wrapper around mdadm on an initramfs which allows a
user to manually recover from RAID problems.

Cheers,
Kyle Moffett

Re: Linux mdadm superblock question.

[david]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1196 bytes --]

On Wed, 17 Feb 2010, Kyle Moffett wrote:

> On Wed, Feb 17, 2010 at 04:38, Rudy Zijlstra
> <rudy@grumpydevil.homelinux.org> wrote:
>> Kyle Moffett wrote:
>>> On Tue, Feb 16, 2010 at 21:01, Neil Brown <neilb@suse.de> wrote:
>>>> I will not be removing 0.90 or auto-assemble from the kernel in the
>>>> foreseeable future.
>>>> None the less, I recommend weaning yourself from your dependence on it.
>>>> initramfs is the future, embrace it.
>>>>
>>>
>>> What are people's reasons for pushback against initramfs?  I've heard
>>> lots of claims that "it's not trustworthy" and "it breaks", but in 7
>>> years of running bootable software RAID boxes on weird architectures
>>> (even running Debian unstable) I have only once or twice had initramfs
>>> problems.

Kyle,
   for a distro that is trying to make one kernel image run on every 
possible type of hardware features like initramfs (and udev, modeules, 
etc) are wonderful.

however for people who run systems that are known ahead of time and 
static (and who build their own kernels instead of just relying on the 
distro default kernel), all of this is unnessesary complication, which 
leaves more room for problems to creep in.

David Lang

Re: Linux mdadm superblock question.

[Nick Bowler]

On 09:41 Wed 17 Feb     , david@lang.hm wrote:
> for a distro that is trying to make one kernel image run on every
> possible type of hardware features like initramfs (and udev, modeules,
> etc) are wonderful.
> 
> however for people who run systems that are known ahead of time and
> static (and who build their own kernels instead of just relying on the
> distro default kernel), all of this is unnessesary complication, which
> leaves more room for problems to creep in.

Such people can easily construct an initramfs containing busybox and
mdadm with a shell script hardcoded to mount their root fs and run
switch_root.  It's a ~10 minute jobbie that only needs to be done once.

-- 
Nick Bowler, Elliptic Technologies (http://www.elliptictech.com/)

Re: Linux mdadm superblock question.

[Volker Armin Hemmann]

On Mittwoch 17 Februar 2010, Nick Bowler wrote:
> On 09:41 Wed 17 Feb     , david@lang.hm wrote:
> > for a distro that is trying to make one kernel image run on every
> > possible type of hardware features like initramfs (and udev, modeules,
> > etc) are wonderful.
> > 
> > however for people who run systems that are known ahead of time and
> > static (and who build their own kernels instead of just relying on the
> > distro default kernel), all of this is unnessesary complication, which
> > leaves more room for problems to creep in.
> 
> Such people can easily construct an initramfs containing busybox and
> mdadm with a shell script hardcoded to mount their root fs and run
> switch_root.  It's a ~10 minute jobbie that only needs to be done once.


and even better when you don't have to do that one time job at all.

btw, what about additional delay? 


Glück Auf,
Volker

Re: Linux mdadm superblock question.

[Nick Bowler]

On 19:27 Wed 17 Feb     , Volker Armin Hemmann wrote:
> On Mittwoch 17 Februar 2010, Nick Bowler wrote:
> > On 09:41 Wed 17 Feb     , david@lang.hm wrote:
> > > for a distro that is trying to make one kernel image run on every
> > > possible type of hardware features like initramfs (and udev, modeules,
> > > etc) are wonderful.
> > > 
> > > however for people who run systems that are known ahead of time and
> > > static (and who build their own kernels instead of just relying on the
> > > distro default kernel), all of this is unnessesary complication, which
> > > leaves more room for problems to creep in.
> > 
> > Such people can easily construct an initramfs containing busybox and
> > mdadm with a shell script hardcoded to mount their root fs and run
> > switch_root.  It's a ~10 minute jobbie that only needs to be done once.
> 
> and even better when you don't have to do that one time job at all.

But people who are building their own kernels are already doing a
(much harder, imo) one time job of configuring their kernels.

> btw, what about additional delay? 

It takes about half a second for mdadm to assemble my root array, is
that what you're referring to?

I assume that kernel auto-assembly is no faster, although I've never
used it.  Regardless, half a second isn't very long to wait.

-- 
Nick Bowler, Elliptic Technologies (http://www.elliptictech.com/)

Re: Linux mdadm superblock question.

[david]

On Wed, 17 Feb 2010, Nick Bowler wrote:

> On 19:27 Wed 17 Feb     , Volker Armin Hemmann wrote:
>> On Mittwoch 17 Februar 2010, Nick Bowler wrote:
>>> On 09:41 Wed 17 Feb     , david@lang.hm wrote:
>>>> for a distro that is trying to make one kernel image run on every
>>>> possible type of hardware features like initramfs (and udev, modeules,
>>>> etc) are wonderful.
>>>>
>>>> however for people who run systems that are known ahead of time and
>>>> static (and who build their own kernels instead of just relying on the
>>>> distro default kernel), all of this is unnessesary complication, which
>>>> leaves more room for problems to creep in.
>>>
>>> Such people can easily construct an initramfs containing busybox and
>>> mdadm with a shell script hardcoded to mount their root fs and run
>>> switch_root.  It's a ~10 minute jobbie that only needs to be done once.
>>
>> and even better when you don't have to do that one time job at all.
>
> But people who are building their own kernels are already doing a
> (much harder, imo) one time job of configuring their kernels.
>
>> btw, what about additional delay?
>
> It takes about half a second for mdadm to assemble my root array, is
> that what you're referring to?
>
> I assume that kernel auto-assembly is no faster, although I've never
> used it.  Regardless, half a second isn't very long to wait.

If you are aiming for a 5-second boot time it's 10% of your total boot 
time. That's a lot for a feature that's not needed.

David Lang

Re: Linux mdadm superblock question.

[Volker Armin Hemmann]

On Mittwoch 17 Februar 2010, Nick Bowler wrote:
> On 19:27 Wed 17 Feb     , Volker Armin Hemmann wrote:
> > On Mittwoch 17 Februar 2010, Nick Bowler wrote:
> > > On 09:41 Wed 17 Feb     , david@lang.hm wrote:
> > > > for a distro that is trying to make one kernel image run on every
> > > > possible type of hardware features like initramfs (and udev,
> > > > modeules, etc) are wonderful.
> > > > 
> > > > however for people who run systems that are known ahead of time and
> > > > static (and who build their own kernels instead of just relying on
> > > > the distro default kernel), all of this is unnessesary complication,
> > > > which leaves more room for problems to creep in.
> > > 
> > > Such people can easily construct an initramfs containing busybox and
> > > mdadm with a shell script hardcoded to mount their root fs and run
> > > switch_root.  It's a ~10 minute jobbie that only needs to be done once.
> > 
> > and even better when you don't have to do that one time job at all.
> 
> But people who are building their own kernels are already doing a
> (much harder, imo) one time job of configuring their kernels.
> 
> > btw, what about additional delay?
> 
> It takes about half a second for mdadm to assemble my root array, is
> that what you're referring to?
> 
> I assume that kernel auto-assembly is no faster, although I've never
> used it.  Regardless, half a second isn't very long to wait.

well at the moment it takes less than two seconds until init takes over.

Adding .5 seconds is a lot. And loading the initrd and changing root isn't 
free either, true?

I remember well all the noise in the past about making linux booting faster. 
So why slow it down with an initrd - especially if you can do without?

Glück Auf,
Volker

Re: Linux mdadm superblock question.

[Nick Bowler]

On 10:41 Wed 17 Feb     , david@lang.hm wrote:
> On Wed, 17 Feb 2010, Nick Bowler wrote:
> > It takes about half a second for mdadm to assemble my root array, is
> > that what you're referring to?
> >
> > I assume that kernel auto-assembly is no faster, although I've never
> > used it.  Regardless, half a second isn't very long to wait.
> 
> If you are aiming for a 5-second boot time it's 10% of your total boot 
> time. That's a lot for a feature that's not needed.

Only if the kernel auto-assembly takes zero time, which it obviously
does not.

-- 
Nick Bowler, Elliptic Technologies (http://www.elliptictech.com/)

Re: Linux mdadm superblock question.

[Gabor Gombas]

On Wed, Feb 17, 2010 at 02:26:46PM +0100, Frans Pop wrote:

> That's simply not true, at least not for Debian. If you actually use the 
> distro tools [1] the only assumptions are made at kernel *installation* 
> time, not at kernel build time.

And that's why network-booted diskless clients and virtual guests have
all sort of useless modules loaded; the HW where the kernel package was
installed in this case is very different from the HW where the kernel
will run. If only there were a switch to prohibit ever looking at the
current machine's configuration when generating the initramfs...

> I've been using initramfs-tools generated initrds for years without 
> problems, and that includes "root on LVM on LUKS encrypted partition" 
> and "root on LVM on RAID" setups.

I've tried a couple of times to use a Debian-built initramfs with a
custom built kernel. The kernel worked fine without an initramfs (it had
everything built in), but it did not boot with the initramfs.

Gabor

Re: Linux mdadm superblock question.

[david]

On Wed, 17 Feb 2010, Nick Bowler wrote:

> On 10:41 Wed 17 Feb     , david@lang.hm wrote:
>> On Wed, 17 Feb 2010, Nick Bowler wrote:
>>> It takes about half a second for mdadm to assemble my root array, is
>>> that what you're referring to?
>>>
>>> I assume that kernel auto-assembly is no faster, although I've never
>>> used it.  Regardless, half a second isn't very long to wait.
>>
>> If you are aiming for a 5-second boot time it's 10% of your total boot
>> time. That's a lot for a feature that's not needed.
>
> Only if the kernel auto-assembly takes zero time, which it obviously
> does not.

the assembly time would probably be the same, but the initramfs being 
proposed did not include that time either.

David Lang

Re: Linux mdadm superblock question.

[Frans Pop]

On Wednesday 17 February 2010, Gabor Gombas wrote:
> On Wed, Feb 17, 2010 at 02:26:46PM +0100, Frans Pop wrote:
> > That's simply not true, at least not for Debian. If you actually use
> > the distro tools [1] the only assumptions are made at kernel
> > *installation* time, not at kernel build time.
>
> And that's why network-booted diskless clients and virtual guests have
> all sort of useless modules loaded; the HW where the kernel package was
> installed in this case is very different from the HW where the kernel
> will run.

Interesting use case. But also a use case for which initramfs-tools 
probably very simply was never intended.

I agree that in this case you probably want to either
- have a very generic initrd that supports anything (Debian default) [1]
or
- provide a list of modules to include in the initrd based on knowing the
  hardware you want to support (e.g. using /etc/initramfs-tools/modules)
  and prevent including anything based on the host system.

I've never really done that so I don't know if initramfs-tools has any 
features to support that.

> If only there were a switch to prohibit ever looking at the 
> current machine's configuration when generating the initramfs...

Did you ever file a wishlist bug report for that?

> > I've been using initramfs-tools generated initrds for years without
> > problems, and that includes "root on LVM on LUKS encrypted partition"
> > and "root on LVM on RAID" setups.
>
> I've tried a couple of times to use a Debian-built initramfs with a
> custom built kernel. The kernel worked fine without an initramfs (it had
> everything built in), but it did not boot with the initramfs.

It's obviously hard to comment on something like this without more details 
(which would be off-topic for this list).


[1] Could still leave you with problems if the clients use something fancy 
for the root fs that uses info copied from /etc.

Re: Linux mdadm superblock question.

[Nick Bowler]

On 13:17 Wed 17 Feb     , david@lang.hm wrote:
> On Wed, 17 Feb 2010, Nick Bowler wrote:
> 
> > On 10:41 Wed 17 Feb     , david@lang.hm wrote:
> >> On Wed, 17 Feb 2010, Nick Bowler wrote:
> >>> It takes about half a second for mdadm to assemble my root array, is
> >>> that what you're referring to?
> >>>
> >>> I assume that kernel auto-assembly is no faster, although I've never
> >>> used it.  Regardless, half a second isn't very long to wait.
> >>
> >> If you are aiming for a 5-second boot time it's 10% of your total boot
> >> time. That's a lot for a feature that's not needed.
> >
> > Only if the kernel auto-assembly takes zero time, which it obviously
> > does not.
> 
> the assembly time would probably be the same, but the initramfs being 
> proposed did not include that time either.

This was the *only* time that was included.  Quoting myself:

> It takes about half a second for mdadm to assemble my root array

I didn't make any claim about any other timings, since I have not made
any measurements (I am not adding instrumentation code to my initramfs
and rebooting the box just to do this, and my watch is not precise
enough to measure the time spent in initramfs).

After the kernel has loaded, but before init on my root fs is run, there
are only three things that cause noticeable delays:

 * probing all the disks.
 * assembling the root array.
 * mounting the root filesystem.

-- 
Nick Bowler, Elliptic Technologies (http://www.elliptictech.com/)

Re: Linux mdadm superblock question.

[david]

On Wed, 17 Feb 2010, Nick Bowler wrote:

> On 13:17 Wed 17 Feb     , david@lang.hm wrote:
>> On Wed, 17 Feb 2010, Nick Bowler wrote:
>>
>>> On 10:41 Wed 17 Feb     , david@lang.hm wrote:
>>>> On Wed, 17 Feb 2010, Nick Bowler wrote:
>>>>> It takes about half a second for mdadm to assemble my root array, is
>>>>> that what you're referring to?
>>>>>
>>>>> I assume that kernel auto-assembly is no faster, although I've never
>>>>> used it.  Regardless, half a second isn't very long to wait.
>>>>
>>>> If you are aiming for a 5-second boot time it's 10% of your total boot
>>>> time. That's a lot for a feature that's not needed.
>>>
>>> Only if the kernel auto-assembly takes zero time, which it obviously
>>> does not.
>>
>> the assembly time would probably be the same, but the initramfs being
>> proposed did not include that time either.
>
> This was the *only* time that was included.  Quoting myself:
>
>> It takes about half a second for mdadm to assemble my root array

sorry, I misunderstood, I thought you were referring to the time added by 
using the initramfs itself.

David Lang

Re: Linux mdadm superblock question.

[H. Peter Anvin]

On 02/17/2010 10:46 AM, Volker Armin Hemmann wrote:
> 
> well at the moment it takes less than two seconds until init takes over.
> 
> Adding .5 seconds is a lot. And loading the initrd and changing root isn't 
> free either, true?
> 
> I remember well all the noise in the past about making linux booting faster. 
> So why slow it down with an initrd - especially if you can do without?
> 

Note that an extremely lightweight initramfs can quite possibly be
faster than doing it in the kernel, just because userspace is so much
less constrained.  I was hoping klibc would catch on for this stuff, but
it hasn't as much as I'd like.

	-hpa

boot times, not mdadm (was: Linux mdadm superblock question.)

[martin f krafft]

[-- Attachment #1: Type: text/plain, Size: 988 bytes --]

also sprach Nick Bowler <nbowler@elliptictech.com> [2010.02.18.1037 +1300]:
> > the assembly time would probably be the same, but the initramfs being 
> > proposed did not include that time either.
> 
> This was the *only* time that was included.  Quoting myself:

If you are discussing boot times rather than mdadm, might I suggest
you change the subject line?

Upstream is keen on finally dropping kernel autoassembly, and
I support that because of the gained flexibility. Boot times are
important for laptops and desktops, which are hardly the primary
target of RAID.

Anyway, this is FLOSS. If you want kernel autoassembly, take over
the code and bring it up to speed.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
"what's your conceptual continuity? --
 well, it should be easy to see:
 the crux of the bisquit is the apopstrophe!"
                                                        -- frank zappa
 
spamtraps: madduck.bogus@madduck.net

[-- Attachment #2: Digital signature (see http://martin-krafft.net/gpg/) --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Linux mdadm superblock question.

[Neil Brown]

On Wed, 17 Feb 2010 03:38:46 +0100
Volker Armin Hemmann <volkerarmin@googlemail.com> wrote:

> On Mittwoch 17 Februar 2010, Neil Brown wrote:
> 
> > I will not be removing 0.90 or auto-assemble from the kernel in the
> > foreseeable future.
> > None the less, I recommend weaning yourself from your dependence on it.
> > initramfs is the future, embrace it.
> > 
> > NeilBrown
> 
> a future that is worse than the present. For what reason?

Reason: some things are easier to implement and maintain in userspace.
Implementing them in the kernel would likely produce a worse products.

Worse than the present:  only if you refuse to embrace it and there-by
contribute to fixing/improving it.

NeilBrown

Re: (boot time consequences of) Linux mdadm superblock question.

[Neil Brown]

On Wed, 17 Feb 2010 10:41:34 -0800 (PST)
david@lang.hm wrote:

> On Wed, 17 Feb 2010, Nick Bowler wrote:
> 
> > On 19:27 Wed 17 Feb     , Volker Armin Hemmann wrote:
> >> On Mittwoch 17 Februar 2010, Nick Bowler wrote:
> >>> On 09:41 Wed 17 Feb     , david@lang.hm wrote:
> >>>> for a distro that is trying to make one kernel image run on every
> >>>> possible type of hardware features like initramfs (and udev, modeules,
> >>>> etc) are wonderful.
> >>>>
> >>>> however for people who run systems that are known ahead of time and
> >>>> static (and who build their own kernels instead of just relying on the
> >>>> distro default kernel), all of this is unnessesary complication, which
> >>>> leaves more room for problems to creep in.
> >>>
> >>> Such people can easily construct an initramfs containing busybox and
> >>> mdadm with a shell script hardcoded to mount their root fs and run
> >>> switch_root.  It's a ~10 minute jobbie that only needs to be done once.
> >>
> >> and even better when you don't have to do that one time job at all.
> >
> > But people who are building their own kernels are already doing a
> > (much harder, imo) one time job of configuring their kernels.
> >
> >> btw, what about additional delay?
> >
> > It takes about half a second for mdadm to assemble my root array, is
> > that what you're referring to?
> >
> > I assume that kernel auto-assembly is no faster, although I've never
> > used it.  Regardless, half a second isn't very long to wait.
> 
> If you are aiming for a 5-second boot time it's 10% of your total boot 
> time. That's a lot for a feature that's not needed.

It is worth noting that the Team that was recently working for v.short boot
times wanted to disable in-kernel autodetect for RAID, and added a
compile-time option to do just that.

The reason is that before the in-kernel autodetection can work reliably you
have to wait for all storage devices to have been discovered.  That wait
can unnecessarily increase the total boot time.

Using user-space autodetection, you can plug "mdadm -I" into udev, and have
arrays assembled as they are found, and filesystems mounted as arrays are
assembled, and then you just have to wait for the root filesystem to appear,
not for "all devices".

Yes, you could make the in-kernel autodetection smarter so it doesn't have to
wait quite so long, but that would make it quite a bit more complex, and it
is harder to maintain the complexity in the kernel.

NeilBrown

Re: (boot time consequences of) Linux mdadm superblock question.

[martin f krafft]

[-- Attachment #1: Type: text/plain, Size: 1366 bytes --]

also sprach Neil Brown <neilb@suse.de> [2010.02.18.1224 +1300]:
> Using user-space autodetection, you can plug "mdadm -I" into udev,
> and have arrays assembled as they are found, and filesystems
> mounted as arrays are assembled, and then you just have to wait
> for the root filesystem to appear, not for "all devices".

The mdadm experimental package offers this via debconf (default off
for now). I would appreciate testers — I literally whacked this up
on a rainy Sunday with an hangover, and while it seems to work fine,
it's probably got warts.

If you don't run Debian or a derivative, you can get the files from
debian/initramfs/* in git://git.debian.org/pkg-mdadm/mdadm.git or

  http://git.debian.org/?p=pkg-mdadm/mdadm.git;a=tree;f=debian/initramfs;hb=HEAD

don't be scared off by the complexity, incremental assembly actually
bypasses most of the (shell) code in both scripts.

> Yes, you could make the in-kernel autodetection smarter so it
> doesn't have to wait quite so long, but that would make it quite
> a bit more complex, and it is harder to maintain the complexity in
> the kernel.

It is definitely a user-space task, if you ask me.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
windows 2000: designed for the internet.
the internet: designed for unix.
 
spamtraps: madduck.bogus@madduck.net

[-- Attachment #2: Digital signature (see http://martin-krafft.net/gpg/) --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: (boot time consequences of) Linux mdadm superblock question.

[Henrique de Moraes Holschuh]

On Thu, 18 Feb 2010, Neil Brown wrote:
> Using user-space autodetection, you can plug "mdadm -I" into udev, and have
> arrays assembled as they are found, and filesystems mounted as arrays are
> assembled, and then you just have to wait for the root filesystem to appear,
> not for "all devices".

Is this ready for testing somewhere?  initramfs+mdadm.conf is operator-error
bait, proper auto-assemble that does away with the requirement of an
up-to-date mdadm.conf inside the initrd would help a great deal, there.

It will need something like LVM has to blacklist/whitelist what device
classes it will scan for superblocks though, or it will eventually cause a
lot of trouble.

-- 
  "One disk to rule them all, One disk to find them. One disk to bring
  them all and in the darkness grind them. In the Land of Redmond
  where the shadows lie." -- The Silicon Valley Tarot
  Henrique Holschuh

Re: (boot time consequences of) Linux mdadm superblock question.

[martin f krafft]

[-- Attachment #1: Type: text/plain, Size: 1291 bytes --]

also sprach Henrique de Moraes Holschuh <hmh@hmh.eng.br> [2010.02.18.1558 +1300]:
> > Using user-space autodetection, you can plug "mdadm -I" into
> > udev, and have arrays assembled as they are found, and
> > filesystems mounted as arrays are assembled, and then you just
> > have to wait for the root filesystem to appear, not for "all
> > devices".
> 
> Is this ready for testing somewhere?  initramfs+mdadm.conf is
> operator-error bait, proper auto-assemble that does away with the
> requirement of an up-to-date mdadm.conf inside the initrd would
> help a great deal, there.

Debian experimental. But so far, I was unable to get rid of
mdadm.conf because it only works without the info in that file if
the homehost is correctly encoded in the metadata. So the challenge
I am facing is http://bugs.debian.org/567468.

> It will need something like LVM has to blacklist/whitelist what
> device classes it will scan for superblocks though, or it will
> eventually cause a lot of trouble.

We rely on linux-base reporting the FS type as linux_raid_member and
mdadm -E finding the metadata if that's the case.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
sex an und für sich ist reine selbstbefriedigung.
 
spamtraps: madduck.bogus@madduck.net

[-- Attachment #2: Digital signature (see http://martin-krafft.net/gpg/) --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Linux mdadm superblock question.

[Goswin von Brederlow]

Nick Bowler <nbowler@elliptictech.com> writes:

> On 09:41 Wed 17 Feb     , david@lang.hm wrote:
>> for a distro that is trying to make one kernel image run on every
>> possible type of hardware features like initramfs (and udev, modeules,
>> etc) are wonderful.
>> 
>> however for people who run systems that are known ahead of time and
>> static (and who build their own kernels instead of just relying on the
>> distro default kernel), all of this is unnessesary complication, which
>> leaves more room for problems to creep in.
>
> Such people can easily construct an initramfs containing busybox and
> mdadm with a shell script hardcoded to mount their root fs and run
> switch_root.  It's a ~10 minute jobbie that only needs to be done once.

Except when mdadm, cryptsetup, lvm change you need to update it.
Esspecially when you set up a new system that might have newer
metadata.

Also at least Debian doesn't (yet) support a common initramfs for their
kernel packaging. You either build a kernel without need for one or you
have a per kernel initramfs that is automatically build and updated
whenever anything in the initrmafs changes. Not often, but still too
often, the initramfs then doesn't work.

Does any other distribution allow building kernel image rpms that will
use a common initramfs for all kernels?

MfG
        Goswin

Re: Linux mdadm superblock question.

[Goswin von Brederlow]

Gabor Gombas <gombasg@digikabel.hu> writes:

> On Wed, Feb 17, 2010 at 02:26:46PM +0100, Frans Pop wrote:
>
>> That's simply not true, at least not for Debian. If you actually use the 
>> distro tools [1] the only assumptions are made at kernel *installation* 
>> time, not at kernel build time.
>
> And that's why network-booted diskless clients and virtual guests have
> all sort of useless modules loaded; the HW where the kernel package was
> installed in this case is very different from the HW where the kernel
> will run. If only there were a switch to prohibit ever looking at the
> current machine's configuration when generating the initramfs...

From my experience you must boot up your client/guest and install the
kernel in there. Then copy the kernel and initrmafs over to the boot
server for use.

Initramfs was never designed to generate an image for another system, or
worse, for a pool of different systems. You can probably make it work on
a case by case basis but that wasn't thought of during the design phase.

Interesting idea though.

>> I've been using initramfs-tools generated initrds for years without 
>> problems, and that includes "root on LVM on LUKS encrypted partition" 
>> and "root on LVM on RAID" setups.
>
> I've tried a couple of times to use a Debian-built initramfs with a
> custom built kernel. The kernel worked fine without an initramfs (it had
> everything built in), but it did not boot with the initramfs.
>
> Gabor

'make-kpkg ... --initrd kernel-image' should build you your custom
kernel with all the magic required to generate a working initramfs.
If not then please do file bugs.

MfG
        Goswin

Re: Linux mdadm superblock question.

[Goswin von Brederlow]

Gabor Gombas <gombasg@digikabel.hu> writes:

> On Wed, Feb 17, 2010 at 02:26:46PM +0100, Frans Pop wrote:
>
>> That's simply not true, at least not for Debian. If you actually use the 
>> distro tools [1] the only assumptions are made at kernel *installation* 
>> time, not at kernel build time.
>
> And that's why network-booted diskless clients and virtual guests have
> all sort of useless modules loaded; the HW where the kernel package was
> installed in this case is very different from the HW where the kernel
> will run. If only there were a switch to prohibit ever looking at the
> current machine's configuration when generating the initramfs...

>From my experience you must boot up your client/guest and install the
kernel in there. Then copy the kernel and initrmafs over to the boot
server for use.

Initramfs was never designed to generate an image for another system, or
worse, for a pool of different systems. You can probably make it work on
a case by case basis but that wasn't thought of during the design phase.

Interesting idea though.

>> I've been using initramfs-tools generated initrds for years without 
>> problems, and that includes "root on LVM on LUKS encrypted partition" 
>> and "root on LVM on RAID" setups.
>
> I've tried a couple of times to use a Debian-built initramfs with a
> custom built kernel. The kernel worked fine without an initramfs (it had
> everything built in), but it did not boot with the initramfs.
>
> Gabor

'make-kpkg ... --initrd kernel-image' should build you your custom
kernel with all the magic required to generate a working initramfs.
If not then please do file bugs.

MfG
        Goswin

Re: (boot time consequences of) Linux mdadm superblock question.

[Daniel Reurich]

> Debian experimental. But so far, I was unable to get rid of
> mdadm.conf because it only works without the info in that file if
> the homehost is correctly encoded in the metadata. So the challenge
> I am facing is http://bugs.debian.org/567468.

Why not use an installation generated uuid like that of the root
filesystem for the homehost identifier.  It's less likely to change then
just about any other system identifier.

Re: (boot time consequences of) Linux mdadm superblock question.

[martin f krafft]

[-- Attachment #1: Type: text/plain, Size: 1203 bytes --]

also sprach Daniel Reurich <daniel@centurion.net.nz> [2010.02.18.1703 +1300]:
> > Debian experimental. But so far, I was unable to get rid of
> > mdadm.conf because it only works without the info in that file if
> > the homehost is correctly encoded in the metadata. So the challenge
> > I am facing is http://bugs.debian.org/567468.
> 
> Why not use an installation generated uuid like that of the root
> filesystem for the homehost identifier.  It's less likely to change then
> just about any other system identifier.

This is a great idea. While I would still have to store that UUID in
the initrd, as you say, it's 99% likely to remain constant, and when
it changes, it's 99% likely that the admin is doing something that
will require more work, so we can resort to 100% hoping that s/he
knows what she's doing.

Neil?

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
"when a woman marries again it is because she detested her first husband.
 when a man marries again it is because he adored his first wife.
 women try their luck; men risk theirs."
                                                        -- oscar wilde
 
spamtraps: madduck.bogus@madduck.net

[-- Attachment #2: Digital signature (see http://martin-krafft.net/gpg/) --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: (boot time consequences of) Linux mdadm superblock question.

[Neil Brown]

On Thu, 18 Feb 2010 17:40:04 +1300
martin f krafft <madduck@madduck.net> wrote:

> also sprach Daniel Reurich <daniel@centurion.net.nz> [2010.02.18.1703 +1300]:
> > > Debian experimental. But so far, I was unable to get rid of
> > > mdadm.conf because it only works without the info in that file if
> > > the homehost is correctly encoded in the metadata. So the challenge
> > > I am facing is http://bugs.debian.org/567468.
> > 
> > Why not use an installation generated uuid like that of the root
> > filesystem for the homehost identifier.  It's less likely to change then
> > just about any other system identifier.
> 
> This is a great idea. While I would still have to store that UUID in
> the initrd, as you say, it's 99% likely to remain constant, and when
> it changes, it's 99% likely that the admin is doing something that
> will require more work, so we can resort to 100% hoping that s/he
> knows what she's doing.
> 
> Neil?
> 

Sure, you can do that.
You would put it in /etc/mdadm.conf
The format would be something like:

  ARRAY /dev/md/root uuid=xxxx:xxxx:xxxx:xxxx

and any array found with that uuid would be assembled as /dev/md/root

NeilBrown

Re: (boot time consequences of) Linux mdadm superblock question.

[Daniel Reurich]

On Thu, 2010-02-18 at 17:40 +1300, martin f krafft wrote:
> also sprach Daniel Reurich <daniel@centurion.net.nz> [2010.02.18.1703 +1300]:
> > > Debian experimental. But so far, I was unable to get rid of
> > > mdadm.conf because it only works without the info in that file if
> > > the homehost is correctly encoded in the metadata. So the challenge
> > > I am facing is http://bugs.debian.org/567468.
> > 
> > Why not use an installation generated uuid like that of the root
> > filesystem for the homehost identifier.  It's less likely to change then
> > just about any other system identifier.
> 
> This is a great idea. While I would still have to store that UUID in
> the initrd, as you say, it's 99% likely to remain constant, and when
> it changes, it's 99% likely that the admin is doing something that
> will require more work, so we can resort to 100% hoping that s/he
> knows what she's doing.

It'd already be there if they used it in /etc/fstab.  (I wish they would
as it would solve a whole lot of other media portability problems as
well, but that's another issue.)

-- 
Daniel Reurich.

Centurion Computer Technology (2005) Ltd
Mobile 021 797 722

Re: (boot time consequences of) Linux mdadm superblock question.

[martin f krafft]

[-- Attachment #1: Type: text/plain, Size: 673 bytes --]

also sprach Neil Brown <neilb@suse.de> [2010.02.18.1810 +1300]:
> You would put it in /etc/mdadm.conf
> The format would be something like:
> 
>   ARRAY /dev/md/root uuid=xxxx:xxxx:xxxx:xxxx
> 
> and any array found with that uuid would be assembled as /dev/md/root

No, we mean using the UUID of the root filesystem (vol_id, not
mdadm's UUID) as the homehost identifier, instead of the hostname.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
"never speak disrespectfully of society.
 only people who can't get into it do that."
                                                        -- oscar wilde
 
spamtraps: madduck.bogus@madduck.net

[-- Attachment #2: Digital signature (see http://martin-krafft.net/gpg/) --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: (boot time consequences of) Linux mdadm superblock question.

[martin f krafft]

[-- Attachment #1: Type: text/plain, Size: 899 bytes --]

also sprach Daniel Reurich <daniel@centurion.net.nz> [2010.02.18.1817 +1300]:
> > This is a great idea. While I would still have to store that UUID in
> > the initrd, as you say, it's 99% likely to remain constant, and when
> > it changes, it's 99% likely that the admin is doing something that
> > will require more work, so we can resort to 100% hoping that s/he
> > knows what she's doing.
> 
> It'd already be there if they used it in /etc/fstab.  (I wish they would
> as it would solve a whole lot of other media portability problems as
> well, but that's another issue.)

/etc/fstab is not copied into the initrd.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
"the duration of passion is proportionate
 with the original resistance of the woman."
                                                   -- honoré de balzac
 
spamtraps: madduck.bogus@madduck.net

[-- Attachment #2: Digital signature (see http://martin-krafft.net/gpg/) --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: (boot time consequences of) Linux mdadm superblock question.

[Neil Brown]

On Thu, 18 Feb 2010 18:21:45 +1300
martin f krafft <madduck@madduck.net> wrote:

> also sprach Neil Brown <neilb@suse.de> [2010.02.18.1810 +1300]:
> > You would put it in /etc/mdadm.conf
> > The format would be something like:
> > 
> >   ARRAY /dev/md/root uuid=xxxx:xxxx:xxxx:xxxx
> > 
> > and any array found with that uuid would be assembled as /dev/md/root
> 
> No, we mean using the UUID of the root filesystem (vol_id, not
> mdadm's UUID) as the homehost identifier, instead of the hostname.
> 

You can set homehost to anything you like, in mdadm.conf or on the command
line..

But it would be rather awkward to store the uuid of the root filesystem in
the metadata for the array that stores the root filesystem (and so is created
before the root filesystem)...

Are you suggesting that when mdadm finds some bits that looks like the form
an array it should test-assemble it, look inside for a filesystem, extra the
uuid of that filesystem and compare against some known uuid before deciding
whether to assemble that array or not?  I hope not.

So I don't think I know what is really being proposed.

??

NeilBrown

Re: Linux mdadm superblock question.

[Luca Berra]

On Thu, Feb 18, 2010 at 04:33:34AM +0100, Goswin von Brederlow wrote:
>Nick Bowler <nbowler@elliptictech.com> writes:
>Also at least Debian doesn't (yet) support a common initramfs for their
>kernel packaging. You either build a kernel without need for one or you
>have a per kernel initramfs that is automatically build and updated
>whenever anything in the initrmafs changes. Not often, but still too
>often, the initramfs then doesn't work.
>
>Does any other distribution allow building kernel image rpms that will
>use a common initramfs for all kernels?

you can do something like this with dracut
actually you need two initrds, one kernel independant that contains
scripts and binaries
the other one, kernel dependant that contains modules (you can skip it
if you don't need modules to mount root)
then use grub commandline to load both

L.

-- 
Luca Berra -- bluca@comedia.it
         Communication Media & Services S.r.l.
  /"\
  \ /     ASCII RIBBON CAMPAIGN
   X        AGAINST HTML MAIL
  / \

Re: Linux mdadm superblock question.

[Ian Dall]

On Tue, 2010-02-16 at 23:18 +0100, Volker Armin Hemmann wrote:
> On Dienstag 16 Februar 2010, Nick Bowler wrote:
> > On 22:06 Tue 16 Feb     , Volker Armin Hemmann wrote:
> > > On Dienstag 16 Februar 2010, Bill Davidsen wrote:
> > > > Volker Armin Hemmann wrote:
> > > > > On Sonntag 14 Februar 2010, you wrote:
> > > > >> In other words, 'auto-detection' for 1.x format devices is using an
> > > > >> initrd/initramfs.
> > > > > 
> > > > > which makes 1.x format useless for everybody who does not want to
> > > > > deal with initrd/initramfs.
> > > > 
> > > > You make this sound like some major big deal. are you running your own
> > > > distribution? In most cases mkinitrd does the right thing when you
> > > > "make install" the kernel, and if you are doing something in the build
> > > > so complex that it needs options, you really should understand the
> > > > options and be sure you're doing what you want.
> > > > 
> > > > Generally this involves preloading a module or two, and if you need it
> > > > every time you probably should have built it in, anyway.
> > > > 
> > > > My opinion...
> > > 
> > > I am running my own kernels - and of course everything that is needed to
> > > boot and get the basic system up is built in. Why should I make the disk
> > > drivers modules?
> > > That does not make sense.
> > 
> > I agree that it makes little sense to make something a module when you
> > can't unload it anyway, but...
> > 
> > > And the reason is simple: even when the system is completely fucked up, I
> > > want a kernel that is able to boot until init=/bin/bb takes over.
> > 
> > I put a complete set of recovery tools into my initramfses so that when
> > the system is completely fucked up, I have a kernel that is able to boot
> > until rdinit=/bin/zsh (or /bin/bb, if you prefer) takes over.
> > 
> > This has the added advantage of working when the root filesystem cannot
> > be mounted at all: a scenario which does not seem too far-fetched when
> > the filesystem is located on a raid array.
> 
> and what do you do if you have to boot from a cd/usb stick and need to access 
> the raid?
> 
> Simple with auto assembling. Not so much without.

I'm not sure what the problem is. I've had to do this (because the disk
with grub on the MBR was the one that failed - now I put grub on them
all).

I booted of the fedora install disk in rescue mode. Told it not to try
and mount any system disks, got into a shell and ran mdadm -As 

I'm not sure what else a kernel auto-assemble would be expected to do
that mdadm -As wouldn't...


-- 
Ian Dall <ian@beware.dropbear.id.au>

Re: Linux mdadm superblock question.

[Nick Bowler]

On 04:33 Thu 18 Feb     , Goswin von Brederlow wrote:
> Nick Bowler <nbowler@elliptictech.com> writes:
> 
> > On 09:41 Wed 17 Feb     , david@lang.hm wrote:
> >> however for people who run systems that are known ahead of time and
> >> static (and who build their own kernels instead of just relying on the
> >> distro default kernel), all of this is unnessesary complication, which
> >> leaves more room for problems to creep in.
> >
> > Such people can easily construct an initramfs containing busybox and
> > mdadm with a shell script hardcoded to mount their root fs and run
> > switch_root.  It's a ~10 minute jobbie that only needs to be done once.
> 
> Except when mdadm, cryptsetup, lvm change you need to update it.
> Esspecially when you set up a new system that might have newer
> metadata.

I meant "once per system".  One typically doesn't _need_ to update the
mdadm in the initramfs, as long as it's capable of assembling the root
array.

> Also at least Debian doesn't (yet) support a common initramfs for their
> kernel packaging. You either build a kernel without need for one or you
> have a per kernel initramfs that is automatically build and updated
> whenever anything in the initrmafs changes. Not often, but still too
> often, the initramfs then doesn't work.

The scenario was when users configure and build their own kernel.  These
users are presumably capable of using grub's "initrd" command or the
CONFIG_INITRAMFS_SOURCE kernel option.

-- 
Nick Bowler, Elliptic Technologies (http://www.elliptictech.com/)

Re: (boot time consequences of) Linux mdadm superblock question.

[martin f krafft]

[-- Attachment #1: Type: text/plain, Size: 1189 bytes --]

also sprach Neil Brown <neilb@suse.de> [2010.02.18.1834 +1300]:
> But it would be rather awkward to store the uuid of the root
> filesystem in the metadata for the array that stores the root
> filesystem (and so is created before the root filesystem)...

True. You'd have to update the superblock UUID right after creation
of the filesystem. That doesn't sound like a robust strategy to
making mdadm.conf optional.

> Are you suggesting that when mdadm finds some bits that looks like
> the form an array it should test-assemble it, look inside for
> a filesystem, extra the uuid of that filesystem and compare
> against some known uuid before deciding whether to assemble that
> array or not?  I hope not.

Hehe, that would be fun, wouldn't it? ;)

> So I don't think I know what is really being proposed.

I really would like to make mdadm.conf optional and still have
things work incrementally from initrd.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
stupidity management for the superuser
is a user space issue in unix systems.
                                                           -- alan cox
 
spamtraps: madduck.bogus@madduck.net

[-- Attachment #2: Digital signature (see http://martin-krafft.net/gpg/) --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: (boot time consequences of) Linux mdadm superblock question.

[Daniel Reurich]

On Fri, 2010-02-19 at 13:42 +1300, martin f krafft wrote:
> also sprach Neil Brown <neilb@suse.de> [2010.02.18.1834 +1300]:
> > But it would be rather awkward to store the uuid of the root
> > filesystem in the metadata for the array that stores the root
> > filesystem (and so is created before the root filesystem)...
> 
> True. You'd have to update the superblock UUID right after creation
> of the filesystem. That doesn't sound like a robust strategy to
> making mdadm.conf optional.
> 
> > Are you suggesting that when mdadm finds some bits that looks like
> > the form an array it should test-assemble it, look inside for
> > a filesystem, extra the uuid of that filesystem and compare
> > against some known uuid before deciding whether to assemble that
> > array or not?  I hope not.
> 
> Hehe, that would be fun, wouldn't it? ;)
> 
> > So I don't think I know what is really being proposed.
> 
> I really would like to make mdadm.conf optional and still have
> things work incrementally from initrd.
> 
But if a generated 'system uuid' value (I just suggested the root fs
UUID because it would be highly unlikely to be unchanged, and nobody
would be likely to fiddle with it) was copied into a file
called /etc/system_uuid and copied into the initrd, then we could add
put into mdadms hook script in initramfs-tools, to verify and update the
homehost variable in the boot time required raid volumes when ever a new
initrd is installed.  (This generally happens on debian whenever a
kernel is installed and mdadm is installed or upgraded.

As an added protection we could include checks in mdadm shutdown script
a check that warns when mdadm.conf doesn't exist and
the /etc/system_uuid doesn't match the homehost value in the boottime
assembled raid volumes.  If we did use the root filesystem UUID for
this, we could compare that as well.

It would be useful to have a tool similar to /bin/hostname that could be
used to create|read|verify|update the system uuid, which would update
all the relevant locations which store and check against this system
uuid.

One could expect LVM would be another prime candidate for system uuid
usage to bind vg's to the host for boot.

But maybe I'm just getting a bit carried away here.  

 
-- 
Daniel Reurich.

Centurion Computer Technology (2005) Ltd
Mobile 021 797 722

Re: Linux mdadm superblock question.

[Michael Evans]

I actually much prefer building my own kernel for servers and
higher-end workstations.  I like having all the core drivers needed to
'get running' in there; even if the more delicate but static logic of
how to track down and assemble/unlock the drives is in the initramfs
image.

The only really annoying issue I've had with my custom initramfs
creator is getting it 'chained' by various distros auto-initramfs
update triggers so that it can grab the version of modules that match
a given kernel.  I have several ways in mind to work around that issue
at various steps, but no known userbase to support besides my self and
thus less motivation to work on that task.

Everything else of course works exactly the same as long as the
configuration hasn't changed on the host system.

Re: Linux mdadm superblock question.

[Michael Evans]

I actually much prefer building my own kernel for servers and
higher-end workstations.  I like having all the core drivers needed to
'get running' in there; even if the more delicate but static logic of
how to track down and assemble/unlock the drives is in the initramfs
image.

The only really annoying issue I've had with my custom initramfs
creator is getting it 'chained' by various distros auto-initramfs
update triggers so that it can grab the version of modules that match
a given kernel.  I have several ways in mind to work around that issue
at various steps, but no known userbase to support besides my self and
thus less motivation to work on that task.

Everything else of course works exactly the same as long as the
configuration hasn't changed on the host system.

Re: (boot time consequences of) Linux mdadm superblock question.

[Piergiorgio Sartor]

Hi,

> True. You'd have to update the superblock UUID right after creation
> of the filesystem. That doesn't sound like a robust strategy to
> making mdadm.conf optional.

it seems that, with "dracut", "mdadm.conf" is optional.

There is the kernel boot paramenter "rd_NO_MDADMCONF",
which forces "dracut" to do not use the "mdadm.conf" in
the initramfs image.

This seems to work also if / is on mdX.

I just tried it, and it was running fine. I did not check
the details, i.e. I've no proof "dracut" was not cheating.

My impression is that it uses the "mdadm -I" functionality.

There are other interesting kernel boot options, like:

rd_MD_UUID=<md uuid>

bye,

-- 

piergiorgio

Re: (boot time consequences of) Linux mdadm superblock question.

[Goswin von Brederlow]

martin f krafft <madduck@madduck.net> writes:

> also sprach Daniel Reurich <daniel@centurion.net.nz> [2010.02.19.0351 +0100]:
>> But if a generated 'system uuid' value (I just suggested the root fs
>> UUID because it would be highly unlikely to be unchanged, and nobody
>> would be likely to fiddle with it) was copied into a file
>> called /etc/system_uuid and copied into the initrd, then we could add
>> put into mdadms hook script in initramfs-tools, to verify and update the
>> homehost variable in the boot time required raid volumes when ever a new
>> initrd is installed.  (This generally happens on debian whenever a
>> kernel is installed and mdadm is installed or upgraded.
>
> Neil's point is that no such value exists. The root filesystem UUID
> is not available when the array is created. And updating the
> homehost in the RAID metadata at boot time would defeat the purpose
> of homehost in the first place.
>
>> As an added protection we could include checks in mdadm shutdown
>> script a check that warns when mdadm.conf doesn't exist and the
>> /etc/system_uuid doesn't match the homehost value in the boottime
>> assembled raid volumes.  If we did use the root filesystem UUID
>> for this, we could compare that as well.
>
> Debian has no policy for this. There is no way to warn a user and
> interrupt the shutdown process.
>
>> It would be useful to have a tool similar to /bin/hostname that
>> could be used to create|read|verify|update the system uuid, which
>> would update all the relevant locations which store and check
>> against this system uuid.
>
> Yes, it would be useful to have a system UUID that could be
> generated by the installer and henceforth written to the newly
> installed system. This is probably something the LSB should push.
> But you could also bring it up for discussion on debian-devel.

How would that work with network boot where the initrd would have to
work for multiple hosts?

MfG
        Goswin

Bug#567468: (boot time consequences of) Linux mdadm superblock question.

[Michael Evans]

On Sun, Feb 21, 2010 at 11:06 PM, Goswin von Brederlow
<goswin-v-b@web.de> wrote:
> martin f krafft <madduck@madduck.net> writes:
>
>> also sprach Daniel Reurich <daniel@centurion.net.nz> [2010.02.19.0351 +0100]:
>>> But if a generated 'system uuid' value (I just suggested the root fs
>>> UUID because it would be highly unlikely to be unchanged, and nobody
>>> would be likely to fiddle with it) was copied into a file
>>> called /etc/system_uuid and copied into the initrd, then we could add
>>> put into mdadms hook script in initramfs-tools, to verify and update the
>>> homehost variable in the boot time required raid volumes when ever a new
>>> initrd is installed.  (This generally happens on debian whenever a
>>> kernel is installed and mdadm is installed or upgraded.
>>
>> Neil's point is that no such value exists. The root filesystem UUID
>> is not available when the array is created. And updating the
>> homehost in the RAID metadata at boot time would defeat the purpose
>> of homehost in the first place.
>>
>>> As an added protection we could include checks in mdadm shutdown
>>> script a check that warns when mdadm.conf doesn't exist and the
>>> /etc/system_uuid doesn't match the homehost value in the boottime
>>> assembled raid volumes.  If we did use the root filesystem UUID
>>> for this, we could compare that as well.
>>
>> Debian has no policy for this. There is no way to warn a user and
>> interrupt the shutdown process.
>>
>>> It would be useful to have a tool similar to /bin/hostname that
>>> could be used to create|read|verify|update the system uuid, which
>>> would update all the relevant locations which store and check
>>> against this system uuid.
>>
>> Yes, it would be useful to have a system UUID that could be
>> generated by the installer and henceforth written to the newly
>> installed system. This is probably something the LSB should push.
>> But you could also bring it up for discussion on debian-devel.
>
> How would that work with network boot where the initrd would have to
> work for multiple hosts?
>
> MfG
>        Goswin
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

I don't know how whatever was mentioned previously would work for
that, but I do have a solution.

Incremental assembly, or examine with all block devices to generate a
new mdadm.conf file.  Then run only devices which are in a complete
state.  The next step would be not mount by uuid, but mount by label.
Presuming you have a consistently labeled rootfs in your deployment
(say mandating that the / filesystem be labeled 'root' or some other
value and that no other FS may share that same label) then it should
work out just fine.

Bug#567468: (boot time consequences of) Linux mdadm superblock question.

[martin f krafft]

[-- Attachment #1: Type: text/plain, Size: 1825 bytes --]

also sprach Goswin von Brederlow <goswin-v-b@web.de> [2010.02.22.0806 +0100]:
> > Yes, it would be useful to have a system UUID that could be
> > generated by the installer and henceforth written to the newly
> > installed system. This is probably something the LSB should push.
> > But you could also bring it up for discussion on debian-devel.
> 
> How would that work with network boot where the initrd would have to
> work for multiple hosts?

Right now, that doesn't work either, except with the traditional
method of simply assembling all arrays found. Neil has implemented
the homehost feature to prevent that. Arguably that's to protect
against a circumstance that is rather rare, and one might not
want/need it. However, if used, then it is true:

  Unless the homehost in the superblock matches the local value, you
  need mdadm.conf to assemble the devices, because the superblock
  information won't be trusted if the homehost doesn't match.

  To be able to determine whether the homehost matches, you need to
  know the value for the system after the initramfs was unpacked.
  Therefore, the homehost value must either be stored in the
  initramfs, which makes it non-portable, or we must use a unique
  identifier of the system that is available from ROM, e.g. the CPU
  ID. I don't think that's standardised.

If auto-assembly of all RAIDs bears dangers and must be regulated,
then we must either have host-specific initramfs's, or be able to
determine the homehost value early during boot otherwise.

-- 
 .''`.   martin f. krafft <madduck@d.o>      Related projects:
: :'  :  proud Debian developer               http://debiansystem.info
`. `'`   http://people.debian.org/~madduck    http://vcs-pkg.org
  `-  Debian - when you have better things to do than fixing systems

[-- Attachment #2: Digital signature (see http://martin-krafft.net/gpg/) --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Bug#567468: (boot time consequences of) Linux mdadm superblock question.

[martin f krafft]

[-- Attachment #1: Type: text/plain, Size: 1365 bytes --]

also sprach Michael Evans <mjevans1983@gmail.com> [2010.02.22.0837 +0100]:
> I don't know how whatever was mentioned previously would work for
> that, but I do have a solution.

[…]

> Incremental assembly, or examine with all block devices to
> generate a new mdadm.conf file.

Please see the thread for reasons why incremental assembly works
only with an mdadm.conf file, or if you can uniquely identify the
system before the root filesystem is mounted.

Please see the thread for reasons why unconditional auto-assembly of
all available arrays may not be desirable.

> Presuming you have a consistently labeled rootfs in your
> deployment (say mandating that the / filesystem be labeled 'root'
> or some other value and that no other FS may share that same
> label) then it should work out just fine.

I fundamentally agree. However, driving this change from mdadm will
be impossible. If that is the way to go, then we must first ensure
that device names become deprecated, and that everyone uses
/dev/disk/by-uuid/*. Only then can we start relying on it.

-- 
 .''`.   martin f. krafft <madduck@d.o>      Related projects:
: :'  :  proud Debian developer               http://debiansystem.info
`. `'`   http://people.debian.org/~madduck    http://vcs-pkg.org
  `-  Debian - when you have better things to do than fixing systems

[-- Attachment #2: Digital signature (see http://martin-krafft.net/gpg/) --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Bug#567468: md homehost (was: Bug#567468: (boot time consequences of) Linux mdadm superblock) question.

[martin f krafft]

[-- Attachment #1: Type: text/plain, Size: 603 bytes --]

also sprach Piergiorgio Sartor <piergiorgio.sartor@nexgo.de> [2010.02.21.2113 +0100]:
> I do not see how the "homehost" plays a role, here.

Neil,

Could you please put forth the argument for why the homehost must
match, and why unconditional auto-assembly is not desirable?
Realistically, what problems are we protecting against?

Thanks,

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
fitter, healthier, more productive
like a pig, in a cage, on antibiotics
                                                          -- radiohead
 
spamtraps: madduck.bogus@madduck.net

[-- Attachment #2: Digital signature (see http://martin-krafft.net/gpg/) --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Bug#567468: (boot time consequences of) Linux mdadm superblock question.

[Daniel Reurich]

On Mon, 2010-02-22 at 10:11 +0100, martin f krafft wrote:
> also sprach Goswin von Brederlow <goswin-v-b@web.de> [2010.02.22.0806 +0100]:
> > > Yes, it would be useful to have a system UUID that could be
> > > generated by the installer and henceforth written to the newly
> > > installed system. This is probably something the LSB should push.
> > > But you could also bring it up for discussion on debian-devel.
> > 
> > How would that work with network boot where the initrd would have to
> > work for multiple hosts?
> 
> Right now, that doesn't work either, except with the traditional
> method of simply assembling all arrays found. Neil has implemented
> the homehost feature to prevent that. Arguably that's to protect
> against a circumstance that is rather rare, and one might not
> want/need it. However, if used, then it is true:
> 
>   Unless the homehost in the superblock matches the local value, you
>   need mdadm.conf to assemble the devices, because the superblock
>   information won't be trusted if the homehost doesn't match.
> 
>   To be able to determine whether the homehost matches, you need to
>   know the value for the system after the initramfs was unpacked.
>   Therefore, the homehost value must either be stored in the
>   initramfs, which makes it non-portable, or we must use a unique
>   identifier of the system that is available from ROM, e.g. the CPU
>   ID. I don't think that's standardised.

Actually, in this case one could use the dhcp assigned hostname or boot
network cards mac address to provide the homehost search string.



-- 
Daniel Reurich.

Centurion Computer Technology (2005) Ltd
Mobile 021 797 722

Bug#567468: md homehost (was: Bug#567468: (boot time consequences of) Linux mdadm superblock) question.

[Daniel Reurich]

> Could you please put forth the argument for why the homehost must
> match, and why unconditional auto-assembly is not desirable?
> Realistically, what problems are we protecting against?

I can think of one or two.  

In the case of network boot, where the kernel and initrd served up via
tftp, but the required filesystems are per host raid volumes served up
ala ATAoverEthernet, iSCSI etc storage network.  This could use the boot
network device MAC or dhcp assigned hostname to as the homehost search
paramater.  This would of course require someway to tell mdadm how to
obtain this homehost parameter.  This could work well where different
groups of hosts using different raid volumes for the root (or other)
filesystems, each with a MAC group (first 3 or 4 MAC fields) is used to
identify that groups homehost search parameter.  

Another scenario, is a dual boot system that has separate raid volumes
for the respective root filesystems, along with a separate initrd image
for each OS.  A system uuid stored in the initrd would work in this case
for the homehost search parameter.


-- 
Daniel Reurich.

Centurion Computer Technology (2005) Ltd
Mobile 021 797 722

Re: md homehost (was: Bug#567468: (boot time consequences of) Linux mdadm superblock) question.

[Neil Brown]

On Mon, 22 Feb 2010 10:16:32 +0100
martin f krafft <madduck@madduck.net> wrote:

> also sprach Piergiorgio Sartor <piergiorgio.sartor@nexgo.de> [2010.02.21.2113 +0100]:
> > I do not see how the "homehost" plays a role, here.
> 
> Neil,
> 
> Could you please put forth the argument for why the homehost must
> match, and why unconditional auto-assembly is not desirable?
> Realistically, what problems are we protecting against?
> 

The problem to protect against is any consequence of rearranging devices
while the host is off, including attaching devices that previously were
attached to a different computer.

mdadm will currently assembly any array that it finds, but will not give a
predictable name to anything that looks like it might be imported from a
different host.
So if you have 'md0' on each of two computers, one computer dies and you move
the devices from that computer to the other, then as long as the bios boots
of the right drive, mdadm will assemble the local array as 'md0' and the
other array as 'something else'.

There are two ways that mdadm determines than array is 'local'.
1/ is the uuid listed against an array in mdadm.conf
2/ is the 'homehost' encoded in the metadata.

If either of those is true, the array is local and gets a predictable name.
If neither, the name gets an _%d suffix.

This is only an issue if you use device name (.e.g /dev/md0 or /dev/md/root)
to mount the root filesystem.
If you use mount-by-uuid then it clearly doesn't matter what name mdadm
assembles the array under.  In that case, the fs UUID (stored on the
initramfs or similar) will assure the necessary uniqueness and mdadm need not
worry about homehost.

But if '/' is mounted by a name in /dev/md/, I want to be sure mdadm puts the
correct array at that name no matter what other arrays might be visible.

Does that clarify things enough?

NeilBrown

Re: md homehost (was: Bug#567468: (boot time consequences of) Linux mdadm superblock) question.

[martin f krafft]

[-- Attachment #1: Type: text/plain, Size: 1328 bytes --]

also sprach Neil Brown <neilb@suse.de> [2010.02.23.0330 +0100]:
> The problem to protect against is any consequence of rearranging
> devices while the host is off, including attaching devices that
> previously were attached to a different computer.

How often does this happen, and how grave/dangerous are the effects?

> But if '/' is mounted by a name in /dev/md/, I want to be sure
> mdadm puts the correct array at that name no matter what other
> arrays might be visible.

Of course it would be nice if this happened, but wouldn't it be
acceptable to assume that if someone swaps drives between machines
that they ought to know how to deal with the consequences, or at
least be ready to tae additional steps to make sure the system still
boots as desired?

Even if the wrong array appeared as /dev/md0 and was mounted as root
device, is there any actual problem, other than inconvenience?
Remember that the person who has previously swapped the drives is
physically in front of (or behind ;)) the machine.

I am unconvinced. I think we should definitely switch to using
filesystem-UUIDs over device names, and that is the only real
solution to the problem, no?

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
Escape Meta Alt Control Shift
 
spamtraps: madduck.bogus@madduck.net

[-- Attachment #2: Digital signature (see http://martin-krafft.net/gpg/) --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: md homehost

[Goswin von Brederlow]

Daniel Reurich <daniel@centurion.net.nz> writes:

>> Could you please put forth the argument for why the homehost must
>> match, and why unconditional auto-assembly is not desirable?
>> Realistically, what problems are we protecting against?
>
> I can think of one or two.  
>
> In the case of network boot, where the kernel and initrd served up via
> tftp, but the required filesystems are per host raid volumes served up
> ala ATAoverEthernet, iSCSI etc storage network.  This could use the boot
> network device MAC or dhcp assigned hostname to as the homehost search
> paramater.  This would of course require someway to tell mdadm how to
> obtain this homehost parameter.  This could work well where different
> groups of hosts using different raid volumes for the root (or other)
> filesystems, each with a MAC group (first 3 or 4 MAC fields) is used to
> identify that groups homehost search parameter.  

To get AoE or iSCSI working you need networking. That means dhcp will
already have run and the hostname be set.

> Another scenario, is a dual boot system that has separate raid volumes
> for the respective root filesystems, along with a separate initrd image
> for each OS.  A system uuid stored in the initrd would work in this case
> for the homehost search parameter.

Or virtualization with raid in the virtual hosts. The host system will
see all the raids of the virtual systems but none of them should be
started.

MfG
        Goswin

Re: md homehost

[Goswin von Brederlow]

martin f krafft <madduck@madduck.net> writes:

> also sprach Neil Brown <neilb@suse.de> [2010.02.23.0330 +0100]:
>> The problem to protect against is any consequence of rearranging
>> devices while the host is off, including attaching devices that
>> previously were attached to a different computer.
>
> How often does this happen, and how grave/dangerous are the effects?
>
>> But if '/' is mounted by a name in /dev/md/, I want to be sure
>> mdadm puts the correct array at that name no matter what other
>> arrays might be visible.
>
> Of course it would be nice if this happened, but wouldn't it be
> acceptable to assume that if someone swaps drives between machines
> that they ought to know how to deal with the consequences, or at
> least be ready to tae additional steps to make sure the system still
> boots as desired?
>
> Even if the wrong array appeared as /dev/md0 and was mounted as root
> device, is there any actual problem, other than inconvenience?
> Remember that the person who has previously swapped the drives is
> physically in front of (or behind ;)) the machine.
>
> I am unconvinced. I think we should definitely switch to using
> filesystem-UUIDs over device names, and that is the only real
> solution to the problem, no?

Both filesystems and LVM have UUIDs. Does dm-crypt / LUKS have one too?

MfG
        Goswin

Re: md homehost

[martin f krafft]

[-- Attachment #1: Type: text/plain, Size: 572 bytes --]

also sprach Goswin von Brederlow <goswin-v-b@web.de> [2010.02.23.0829 +0100]:
> Or virtualization with raid in the virtual hosts. The host system will
> see all the raids of the virtual systems but none of them should be
> started.

I'd say there are better ways to set this up, but this is a fair
point.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
"one should never allow one's mind
 and one's foot to wander at the same time."
                                -- edward perkins (yes, the librarian)
 
spamtraps: madduck.bogus@madduck.net

[-- Attachment #2: Digital signature (see http://martin-krafft.net/gpg/) --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Bug#567468: md homehost

[martin f krafft]

[-- Attachment #1: Type: text/plain, Size: 815 bytes --]

also sprach Goswin von Brederlow <goswin-v-b@web.de> [2010.02.23.0831 +0100]:
> Both filesystems and LVM have UUIDs. Does dm-crypt / LUKS have one
> too?

LVM already identifies PVs using UUIDs, so if you are using
anything-on-LVM-on-md, you need not worry about device names anyway.

dm-crypt currently needs to be told explicitly to use a base device
using /dev/disk/by-uuid if you want it to be flexible.

The only issue homehost protects against, I think, is machines that
use /dev/md0 directly from grub.conf or fstab.

-- 
 .''`.   martin f. krafft <madduck@d.o>      Related projects:
: :'  :  proud Debian developer               http://debiansystem.info
`. `'`   http://people.debian.org/~madduck    http://vcs-pkg.org
  `-  Debian - when you have better things to do than fixing systems

[-- Attachment #2: Digital signature (see http://martin-krafft.net/gpg/) --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: md homehost

[Piergiorgio Sartor]

Hi,

> Both filesystems and LVM have UUIDs. Does dm-crypt / LUKS have one too?

LUKS, of course, has UUID, plain dm-crypt of course not... :-)

bye,

-- 

piergiorgio

Re: md homehost (was: Bug#567468: (boot time consequences of) Linux mdadm superblock) question.

[Neil Brown]

On Tue, 23 Feb 2010 07:27:00 +0100
martin f krafft <madduck@madduck.net> wrote:

> also sprach Neil Brown <neilb@suse.de> [2010.02.23.0330 +0100]:
> > The problem to protect against is any consequence of rearranging
> > devices while the host is off, including attaching devices that
> > previously were attached to a different computer.
> 
> How often does this happen, and how grave/dangerous are the effects?

a/ no idea.
b/ it all depends...
  It is the sort of thing that happens when something has just gone
  drastically wrong and you need to stitch things back together again as
  quickly as you can.  You aren't exactly panicing, but you are probably
  hasty and don't want anything else to go wrong.

  If the array from the 'other' machine with the same name has very different
  content, then things could go wrong in various different ways if we
  depended on that name.
  It is true that the admin would have to by physically present and could
  presumably get a console and 'fix' things.  But it would be best if they
  didn't have too.  They may not even know clearly what to do to 'fix' things
  - because it always worked perfectly before, but this time when in a
    particular hurry, something strange goes wrongs.  I've been there, I
    don't want to inflict it on others.

> 
> > But if '/' is mounted by a name in /dev/md/, I want to be sure
> > mdadm puts the correct array at that name no matter what other
> > arrays might be visible.
> 
> Of course it would be nice if this happened, but wouldn't it be
> acceptable to assume that if someone swaps drives between machines
> that they ought to know how to deal with the consequences, or at
> least be ready to tae additional steps to make sure the system still
> boots as desired?

No.  We cannot assume that an average sys-admin will have a deep knowledge of
md and mdadm.  Many do, many don't.  But in either case the behaviour must be
predictable.
After all, Debian is for "when you have better things to do than fixing
systems"

> 
> Even if the wrong array appeared as /dev/md0 and was mounted as root
> device, is there any actual problem, other than inconvenience?
> Remember that the person who has previously swapped the drives is
> physically in front of (or behind ;)) the machine.
> 
> I am unconvinced. I think we should definitely switch to using
> filesystem-UUIDs over device names, and that is the only real
> solution to the problem, no?
> 

What exactly are you unconvinced of?
I agree completely that mounting filesystems by UUID is the right way to go.
(I also happen to think that assembly md arrays by UUID is the right way to
go too, but while people seem happy to put fs uuids in /etc/fstab, they seem
less happy to put md uuids in /etc/mdadm.conf).

As you say in another email:

> The only issue homehost protects against, I think, is machines that
> use /dev/md0 directly from grub.conf or fstab.

That is exactly correct.  If no code or config file depends on a name like
/dev/mdX or /dev/md/foo, then you don't need to be concerned about the whole
homehost thing.
You can either mount by fs-uuid, or mount e.g.
   /dev/disk/by-id/md-uuid-8fd0af3f:4fbb94ea:12cc2127:f9855db5 


NeilBrown

Re: md homehost (was: Bug#567468: (boot time consequences of) Linux mdadm superblock) question.

[Michael Evans]

On Tue, Feb 23, 2010 at 4:10 PM, Neil Brown <neilb@suse.de> wrote:
> On Tue, 23 Feb 2010 07:27:00 +0100
> martin f krafft <madduck@madduck.net> wrote:
>
>> also sprach Neil Brown <neilb@suse.de> [2010.02.23.0330 +0100]:
>> > The problem to protect against is any consequence of rearranging
>> > devices while the host is off, including attaching devices that
>> > previously were attached to a different computer.
>>
>> How often does this happen, and how grave/dangerous are the effects?
>
> a/ no idea.
> b/ it all depends...
>  It is the sort of thing that happens when something has just gone
>  drastically wrong and you need to stitch things back together again as
>  quickly as you can.  You aren't exactly panicing, but you are probably
>  hasty and don't want anything else to go wrong.
>
>  If the array from the 'other' machine with the same name has very different
>  content, then things could go wrong in various different ways if we
>  depended on that name.
>  It is true that the admin would have to by physically present and could
>  presumably get a console and 'fix' things.  But it would be best if they
>  didn't have too.  They may not even know clearly what to do to 'fix' things
>  - because it always worked perfectly before, but this time when in a
>    particular hurry, something strange goes wrongs.  I've been there, I
>    don't want to inflict it on others.
>
>>
>> > But if '/' is mounted by a name in /dev/md/, I want to be sure
>> > mdadm puts the correct array at that name no matter what other
>> > arrays might be visible.
>>
>> Of course it would be nice if this happened, but wouldn't it be
>> acceptable to assume that if someone swaps drives between machines
>> that they ought to know how to deal with the consequences, or at
>> least be ready to tae additional steps to make sure the system still
>> boots as desired?
>
> No.  We cannot assume that an average sys-admin will have a deep knowledge of
> md and mdadm.  Many do, many don't.  But in either case the behaviour must be
> predictable.
> After all, Debian is for "when you have better things to do than fixing
> systems"
>
>>
>> Even if the wrong array appeared as /dev/md0 and was mounted as root
>> device, is there any actual problem, other than inconvenience?
>> Remember that the person who has previously swapped the drives is
>> physically in front of (or behind ;)) the machine.
>>
>> I am unconvinced. I think we should definitely switch to using
>> filesystem-UUIDs over device names, and that is the only real
>> solution to the problem, no?
>>
>
> What exactly are you unconvinced of?
> I agree completely that mounting filesystems by UUID is the right way to go.
> (I also happen to think that assembly md arrays by UUID is the right way to
> go too, but while people seem happy to put fs uuids in /etc/fstab, they seem
> less happy to put md uuids in /etc/mdadm.conf).
>
> As you say in another email:
>
>> The only issue homehost protects against, I think, is machines that
>> use /dev/md0 directly from grub.conf or fstab.
>
> That is exactly correct.  If no code or config file depends on a name like
> /dev/mdX or /dev/md/foo, then you don't need to be concerned about the whole
> homehost thing.
> You can either mount by fs-uuid, or mount e.g.
>   /dev/disk/by-id/md-uuid-8fd0af3f:4fbb94ea:12cc2127:f9855db5
>
>
> NeilBrown
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Would a permissible behavior be to add a third case:
If an entry is not detected in the mdadm.conf file AND the homehost is
not found to match ask on the standard console what to do with
something like a 30 second timeout; as well as being noisy in the
kernel log so the admin knows why it was slow.

Really there should probably be two questions: 1) Do you want to run
this?  2) What name do you want? (with the defaults being yes, and the
currently chosen alternate name pattern).
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Bug#567468: md homehost

[Goswin von Brederlow]

martin f krafft <madduck@debian.org> writes:

> also sprach Goswin von Brederlow <goswin-v-b@web.de> [2010.02.23.0831 +0100]:
>> Both filesystems and LVM have UUIDs. Does dm-crypt / LUKS have one
>> too?
>
> LVM already identifies PVs using UUIDs, so if you are using
> anything-on-LVM-on-md, you need not worry about device names anyway.
>
> dm-crypt currently needs to be told explicitly to use a base device
> using /dev/disk/by-uuid if you want it to be flexible.
>
> The only issue homehost protects against, I think, is machines that
> use /dev/md0 directly from grub.conf or fstab.

grub.cfg (grub2) uses UUID for grub itself. But the kernel can be bootet
with root=/dev/md0. But in that case where does it get the homehost from
and since when does kernel raid autoconfig have a homehost?

Any other case you have an initramfs that can use LABEL or UUID. Same
for fstab.

MfG
        Goswin

Re: md homehost

[Goswin von Brederlow]

Neil Brown <neilb@suse.de> writes:

> On Tue, 23 Feb 2010 07:27:00 +0100
> martin f krafft <madduck@madduck.net> wrote:
>> The only issue homehost protects against, I think, is machines that
>> use /dev/md0 directly from grub.conf or fstab.
>
> That is exactly correct.  If no code or config file depends on a name like
> /dev/mdX or /dev/md/foo, then you don't need to be concerned about the whole
> homehost thing.
> You can either mount by fs-uuid, or mount e.g.
>    /dev/disk/by-id/md-uuid-8fd0af3f:4fbb94ea:12cc2127:f9855db5 

What if you have two raids (one local, one from the other hosts that
broke down) and both have LVM on it with /dev/vg/root?

Shouldn't it only assemble the local raid (as md0 or whatever) and then
only start the local volume group? If it assembles the remote raid as
/dev/md127 as well then lvm will have problems and the boot will likely
(even randomly) go wrong since only one VG can be activated.

I think it is pretty common for admins to configure LVM to the same
volume group name on different systems. So if you consider raids being
pluged into other systems please keep this in mind.

MfG
        Goswin

Re: Bug#567468: md homehost

[Mario 'BitKoenig' Holbe]

[-- Attachment #1: Type: text/plain, Size: 732 bytes --]

On Wed, Feb 24, 2010 at 02:13:53PM +0100, Goswin von Brederlow wrote:
> grub.cfg (grub2) uses UUID for grub itself. But the kernel can be bootet
> with root=/dev/md0. But in that case where does it get the homehost from
> and since when does kernel raid autoconfig have a homehost?

The homehost attribute does only exist with v1 superblocks. And there is
no in-kernel auto-assembly for v1 superblocks.
v0.9 superblocks (for which in-kernel auto-assembly is deprecated but
still provided) have no homehost.


just my 2 cents
   Mario
-- 
The secret that the NSA could read the Iranian secrets was more
important than any specific Iranian secrets that the NSA could
read.                           -- Bruce Schneier

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 482 bytes --]

Re: Bug#567468: md homehost

[Neil Brown]

On Wed, 24 Feb 2010 18:52:57 +0100
"Mario 'BitKoenig' Holbe" <Mario.Holbe@TU-Ilmenau.DE> wrote:

> On Wed, Feb 24, 2010 at 02:13:53PM +0100, Goswin von Brederlow wrote:
> > grub.cfg (grub2) uses UUID for grub itself. But the kernel can be bootet
> > with root=/dev/md0. But in that case where does it get the homehost from
> > and since when does kernel raid autoconfig have a homehost?
> 
> The homehost attribute does only exist with v1 superblocks. And there is
> no in-kernel auto-assembly for v1 superblocks.
> v0.9 superblocks (for which in-kernel auto-assembly is deprecated but
> still provided) have no homehost.
>

Not entirely correct.  The 'homehost' is encoded in the uuid of v0.90
metadata, so it does affect them too.

in-kernel autodetect does not make use of 'homehost' and so does not protect
you from the potential confusions that homehost tries to protect you from.

NeilBrown

Re: md homehost

[Neil Brown]

On Wed, 24 Feb 2010 14:41:16 +0100
Goswin von Brederlow <goswin-v-b@web.de> wrote:

> Neil Brown <neilb@suse.de> writes:
> 
> > On Tue, 23 Feb 2010 07:27:00 +0100
> > martin f krafft <madduck@madduck.net> wrote:
> >> The only issue homehost protects against, I think, is machines that
> >> use /dev/md0 directly from grub.conf or fstab.
> >
> > That is exactly correct.  If no code or config file depends on a name like
> > /dev/mdX or /dev/md/foo, then you don't need to be concerned about the whole
> > homehost thing.
> > You can either mount by fs-uuid, or mount e.g.
> >    /dev/disk/by-id/md-uuid-8fd0af3f:4fbb94ea:12cc2127:f9855db5 
> 
> What if you have two raids (one local, one from the other hosts that
> broke down) and both have LVM on it with /dev/vg/root?
> 
> Shouldn't it only assemble the local raid (as md0 or whatever) and then
> only start the local volume group? If it assembles the remote raid as
> /dev/md127 as well then lvm will have problems and the boot will likely
> (even randomly) go wrong since only one VG can be activated.
> 
> I think it is pretty common for admins to configure LVM to the same
> volume group name on different systems. So if you consider raids being
> pluged into other systems please keep this in mind.

You are entirely correct.  However lvm problems are not my problems.

It has always been my position that the best way to configure md is to
explicitly list your arrays in mdadm.conf.  But people seem to not like this
and want it to all be 'automatic'.  So I do my best to make it as automatic
as possible but still remove as many of the possible confusion that this can
cause as possible.  But I cannot remove them all.

If you move disks around and boot and lvm gets confused because there are two
things call "/dev/vg/root", then I'm sorry but there is nothing I can do
about that.  If you had an mdadm.conf which listed you md arrays, and had
   auto -all
then you can be sure that mdadm would not be contributing to this problem.

NeilBrown

Re: md homehost

[Goswin von Brederlow]

Neil Brown <neilb@suse.de> writes:

> On Wed, 24 Feb 2010 14:41:16 +0100
> Goswin von Brederlow <goswin-v-b@web.de> wrote:
>
>> Neil Brown <neilb@suse.de> writes:
>> 
>> > On Tue, 23 Feb 2010 07:27:00 +0100
>> > martin f krafft <madduck@madduck.net> wrote:
>> >> The only issue homehost protects against, I think, is machines that
>> >> use /dev/md0 directly from grub.conf or fstab.
>> >
>> > That is exactly correct.  If no code or config file depends on a name like
>> > /dev/mdX or /dev/md/foo, then you don't need to be concerned about the whole
>> > homehost thing.
>> > You can either mount by fs-uuid, or mount e.g.
>> >    /dev/disk/by-id/md-uuid-8fd0af3f:4fbb94ea:12cc2127:f9855db5 
>> 
>> What if you have two raids (one local, one from the other hosts that
>> broke down) and both have LVM on it with /dev/vg/root?
>> 
>> Shouldn't it only assemble the local raid (as md0 or whatever) and then
>> only start the local volume group? If it assembles the remote raid as
>> /dev/md127 as well then lvm will have problems and the boot will likely
>> (even randomly) go wrong since only one VG can be activated.
>> 
>> I think it is pretty common for admins to configure LVM to the same
>> volume group name on different systems. So if you consider raids being
>> pluged into other systems please keep this in mind.
>
> You are entirely correct.  However lvm problems are not my problems.
>
> It has always been my position that the best way to configure md is to
> explicitly list your arrays in mdadm.conf.  But people seem to not like this
> and want it to all be 'automatic'.  So I do my best to make it as automatic
> as possible but still remove as many of the possible confusion that this can
> cause as possible.  But I cannot remove them all.
>
> If you move disks around and boot and lvm gets confused because there are two
> things call "/dev/vg/root", then I'm sorry but there is nothing I can do
> about that.  If you had an mdadm.conf which listed you md arrays, and had
>    auto -all
> then you can be sure that mdadm would not be contributing to this problem.
>
> NeilBrown

Yes you can do something about it: Only start the raid arrays with the
correct homehost.

If the homehost is only used to decide wether the prefered minor in the
metadata is used for the device name then I feel the feature is entirely
useless. It would only help in "stupid" configurations, i.e. when you
use the device name directly.

Another scenario where starting a raid with the wrong homehost would be
bad is when the raid is degraded and you have a global spare. You
probably wouldn't want the global spare of one host to be used to repair
a raid of another host.

MfG
        Goswin

PS: If a raid is not listed in mdadm.conf doesn't it currently start too
but the name can be random?

Re: md homehost

[Neil Brown]

On Thu, 25 Feb 2010 08:16:14 +0100
Goswin von Brederlow <goswin-v-b@web.de> wrote:

> Neil Brown <neilb@suse.de> writes:
> 
> > On Wed, 24 Feb 2010 14:41:16 +0100
> > Goswin von Brederlow <goswin-v-b@web.de> wrote:
> >
> >> Neil Brown <neilb@suse.de> writes:
> >> 
> >> > On Tue, 23 Feb 2010 07:27:00 +0100
> >> > martin f krafft <madduck@madduck.net> wrote:
> >> >> The only issue homehost protects against, I think, is machines that
> >> >> use /dev/md0 directly from grub.conf or fstab.
> >> >
> >> > That is exactly correct.  If no code or config file depends on a name like
> >> > /dev/mdX or /dev/md/foo, then you don't need to be concerned about the whole
> >> > homehost thing.
> >> > You can either mount by fs-uuid, or mount e.g.
> >> >    /dev/disk/by-id/md-uuid-8fd0af3f:4fbb94ea:12cc2127:f9855db5 
> >> 
> >> What if you have two raids (one local, one from the other hosts that
> >> broke down) and both have LVM on it with /dev/vg/root?
> >> 
> >> Shouldn't it only assemble the local raid (as md0 or whatever) and then
> >> only start the local volume group? If it assembles the remote raid as
> >> /dev/md127 as well then lvm will have problems and the boot will likely
> >> (even randomly) go wrong since only one VG can be activated.
> >> 
> >> I think it is pretty common for admins to configure LVM to the same
> >> volume group name on different systems. So if you consider raids being
> >> pluged into other systems please keep this in mind.
> >
> > You are entirely correct.  However lvm problems are not my problems.
> >
> > It has always been my position that the best way to configure md is to
> > explicitly list your arrays in mdadm.conf.  But people seem to not like this
> > and want it to all be 'automatic'.  So I do my best to make it as automatic
> > as possible but still remove as many of the possible confusion that this can
> > cause as possible.  But I cannot remove them all.
> >
> > If you move disks around and boot and lvm gets confused because there are two
> > things call "/dev/vg/root", then I'm sorry but there is nothing I can do
> > about that.  If you had an mdadm.conf which listed you md arrays, and had
> >    auto -all
> > then you can be sure that mdadm would not be contributing to this problem.
> >
> > NeilBrown
> 
> Yes you can do something about it: Only start the raid arrays with the
> correct homehost.

This is what 'homehost' originally did, but I got a lot of push-back on that.
I added the "auto" line in mdadm.conf so that the admin could choose what
happens.
If the particular metadata type is enabled on the auto line, the the array is
assembled with a random name.  If it is disabled, it is not assembled at all
(unless explicitly listed in mdadm.conf).
I'm not sure exactly how 'auto' interacts with 'homehost'.  The documentation
I wrote only talks about arrays listed in mdadm.conf or on the command line,
not arrays with a valid homehost.  I guess I should check..... I think I want
"auto -all" to still assemble arrays with a valid homehost.  I'll confirm
that before I release 3.1.2.

> 
> If the homehost is only used to decide wether the prefered minor in the
> metadata is used for the device name then I feel the feature is entirely
> useless. It would only help in "stupid" configurations, i.e. when you
> use the device name directly.

Yes.

> 
> Another scenario where starting a raid with the wrong homehost would be
> bad is when the raid is degraded and you have a global spare. You
> probably wouldn't want the global spare of one host to be used to repair
> a raid of another host.

I only support global spares that are explicitly listed in mdadm.conf, so
currently this couldn't happen.  One day some one is going to ask for
auto-configure global spares.  Then I'll have to worry about this (or just
say "no").

> 
> MfG
>         Goswin
> 
> PS: If a raid is not listed in mdadm.conf doesn't it currently start too
> but the name can be random?

It depends on the "auto" line in mdadm.conf

Thanks,
NeilBrown

Re: md homehost

[Michael Evans]

On Wed, Feb 24, 2010 at 11:46 PM, Neil Brown <neilb@suse.de> wrote:
> On Thu, 25 Feb 2010 08:16:14 +0100
> Goswin von Brederlow <goswin-v-b@web.de> wrote:
>
>> Neil Brown <neilb@suse.de> writes:
>>
>> > On Wed, 24 Feb 2010 14:41:16 +0100
>> > Goswin von Brederlow <goswin-v-b@web.de> wrote:
>> >
>> >> Neil Brown <neilb@suse.de> writes:
>> >>
>> >> > On Tue, 23 Feb 2010 07:27:00 +0100
>> >> > martin f krafft <madduck@madduck.net> wrote:
>> >> >> The only issue homehost protects against, I think, is machines that
>> >> >> use /dev/md0 directly from grub.conf or fstab.
>> >> >
>> >> > That is exactly correct.  If no code or config file depends on a name like
>> >> > /dev/mdX or /dev/md/foo, then you don't need to be concerned about the whole
>> >> > homehost thing.
>> >> > You can either mount by fs-uuid, or mount e.g.
>> >> >    /dev/disk/by-id/md-uuid-8fd0af3f:4fbb94ea:12cc2127:f9855db5
>> >>
>> >> What if you have two raids (one local, one from the other hosts that
>> >> broke down) and both have LVM on it with /dev/vg/root?
>> >>
>> >> Shouldn't it only assemble the local raid (as md0 or whatever) and then
>> >> only start the local volume group? If it assembles the remote raid as
>> >> /dev/md127 as well then lvm will have problems and the boot will likely
>> >> (even randomly) go wrong since only one VG can be activated.
>> >>
>> >> I think it is pretty common for admins to configure LVM to the same
>> >> volume group name on different systems. So if you consider raids being
>> >> pluged into other systems please keep this in mind.
>> >
>> > You are entirely correct.  However lvm problems are not my problems.
>> >
>> > It has always been my position that the best way to configure md is to
>> > explicitly list your arrays in mdadm.conf.  But people seem to not like this
>> > and want it to all be 'automatic'.  So I do my best to make it as automatic
>> > as possible but still remove as many of the possible confusion that this can
>> > cause as possible.  But I cannot remove them all.
>> >
>> > If you move disks around and boot and lvm gets confused because there are two
>> > things call "/dev/vg/root", then I'm sorry but there is nothing I can do
>> > about that.  If you had an mdadm.conf which listed you md arrays, and had
>> >    auto -all
>> > then you can be sure that mdadm would not be contributing to this problem.
>> >
>> > NeilBrown
>>
>> Yes you can do something about it: Only start the raid arrays with the
>> correct homehost.
>
> This is what 'homehost' originally did, but I got a lot of push-back on that.
> I added the "auto" line in mdadm.conf so that the admin could choose what
> happens.
> If the particular metadata type is enabled on the auto line, the the array is
> assembled with a random name.  If it is disabled, it is not assembled at all
> (unless explicitly listed in mdadm.conf).
> I'm not sure exactly how 'auto' interacts with 'homehost'.  The documentation
> I wrote only talks about arrays listed in mdadm.conf or on the command line,
> not arrays with a valid homehost.  I guess I should check..... I think I want
> "auto -all" to still assemble arrays with a valid homehost.  I'll confirm
> that before I release 3.1.2.
>
>>
>> If the homehost is only used to decide wether the prefered minor in the
>> metadata is used for the device name then I feel the feature is entirely
>> useless. It would only help in "stupid" configurations, i.e. when you
>> use the device name directly.
>
> Yes.
>
>>
>> Another scenario where starting a raid with the wrong homehost would be
>> bad is when the raid is degraded and you have a global spare. You
>> probably wouldn't want the global spare of one host to be used to repair
>> a raid of another host.
>
> I only support global spares that are explicitly listed in mdadm.conf, so
> currently this couldn't happen.  One day some one is going to ask for
> auto-configure global spares.  Then I'll have to worry about this (or just
> say "no").
>
>>
>> MfG
>>         Goswin
>>
>> PS: If a raid is not listed in mdadm.conf doesn't it currently start too
>> but the name can be random?
>
> It depends on the "auto" line in mdadm.conf
>
> Thanks,
> NeilBrown
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

I've always tried to assign volume-groups a host-unique name anyway.
Though I don't currently run enough systems to demand a formal
approach, I imagine a dedicated hostname within the VG name would
work.  You could then use a pattern like sys-${HOSTNAME} or sys-*/ to
obtain the hostname on a nominal basis; though obviously if working on
multiple 'system' volume groups at a time the * method fails...
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: md homehost

[Mario 'BitKoenig' Holbe]

[-- Attachment #1: Type: text/plain, Size: 1339 bytes --]

On Thu, Feb 25, 2010 at 08:16:14AM +0100, Goswin von Brederlow wrote:
> Yes you can do something about it: Only start the raid arrays with the
> correct homehost.

I agree with you that auto-assembling devices with non-matching homehost
(and unknown UUID) is probably not the best idea.

> If the homehost is only used to decide wether the prefered minor in the
> metadata is used for the device name then I feel the feature is entirely
> useless. It would only help in "stupid" configurations, i.e. when you
> use the device name directly.

I don't think this is a "stupid" configuration. md was one of the first
subsystems providing stable device names and I think they are widely
used - alone for historical reasons.
I personally feel way better relying on md device names than on some
UUIDs. The former are just far more stable than some UUID of some
filesystem which changes every time you re-create it. UUIDs of md
devices do change as well if you re-create them, but their name remains.

Yes, I'm aware of filesystem labels, but they rise the same problems as
md devices without homehost attribute.


Mario
-- 
There are trivial truths and the great truths.
The opposite of a trivial truth is plainly false.
The opposite of a great truth is also true.
                                    -- Niels Bohr

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 482 bytes --]