anyone aware of a high availability setup that relies on fully redundant install?

anyone aware of a high availability setup that relies on fully redundant install?

[Robert P. J. Day]

  i figure this is as good a place as any to ask ... is anyone here
aware of anyone using a linux config and install that, for the
purposes of reliability or high availability or whatever you want to
call it, relies on a second, completely independent installation of
linux on the same hard drive?

  i was having a discussion about the reliability of upgrading linux
(any combination of kernel, basic rootfs and/or proprietary apps), and
one approach that was suggested (not by me) was to have two entirely
independent installations on the same hard drive so that, when one
"upgraded" the system, the newer content was installed in the other
partition and the system rebooted to that newer content, the rationale
being that, if the upgraded system didn't work, one could revert back
to the original install.

  the history of this particular approach lies in the fact that the
original S/W running on the system in question was an embedded app
programmed right on bare metal, so the entire app was basically a
monolithic combination of boot loader, kernel and all the apps. in
that case, i can certainly see how one wanted a reliable way to revert
back to a known working system if the new S/W had flaws. but i don't
think that logic holds with a linux installation.

  the major concern seems to be a fear that, if one upgrades the
kernel even a little bit, user space apps might suddenly stop working
because of some new incompatibility introduced by that upgrade. i'm
well aware of gregkh's piece on kernel stable API nonsense:

https://www.kernel.org/doc/Documentation/stable_api_nonsense.txt

so i'm not overly worried, but folks in the discussion coming from the
embedded programming space are using their experiences from that space
to argue that a fully redundant install is the safest approach just in
case something goes wrong with an upgrade.

  not surprisingly, my position is that this is not something they
need to worry about a whole lot but, for the sake of fairness, i
thought i'd look around to see if any linux folks *are* using that
approach.

  anyone? thoughts?

rday

-- 

========================================================================
Robert P. J. Day                                 Ottawa, Ontario, CANADA
                        http://crashcourse.ca

Twitter:                                       http://twitter.com/rpjday
LinkedIn:                               http://ca.linkedin.com/in/rpjday
========================================================================

Res: anyone aware of a high availability setup that relies on fully redundant install?

[Daniel]

?This depends on environment proposes. It,s aways a good thing to have a recovery boot option that can boot a minimum shell to dignoses whats going wrong but a full instalation for that is even useful?? IMHO is not a good idea, since you will have to maintain two instalations per hardware.?
?
You may be looking for something like update rollback. ?I know that yum does this and that smart package manager has this feature two, but I never tested it neither one :)

Other option is rely on filesystem for such thing. btrfs has some nice snapshot features, you may want to take a look.?

For recovery boot option you can have a network boot, loading a complete distinct kernel and rootfs that can be chrooted to in drive instalation for recovery when everything goes wrong. Also, the "remote system" can be maintained independently of host instalation. With RO rootfs you can even boot multiple machines with same kernel/rootfs simultaneously, and maintain one recovery installation for multiple "host machines". The better aproach depends on your enviroment, what you have avaible for recovery and how you want to proceed when something goes wrong.

Regards
?
Enviado?do?meu?smartphone?BlackBerry?10.
? Mensagem original ?
De: Robert P. J. Day
Enviada: domingo, 17 de abril de 2016 11:48
Para: Kernel Newbies
Assunto: anyone aware of a high availability setup that relies on fully redundant install?


i figure this is as good a place as any to ask ... is anyone here
aware of anyone using a linux config and install that, for the
purposes of reliability or high availability or whatever you want to
call it, relies on a second, completely independent installation of
linux on the same hard drive?

i was having a discussion about the reliability of upgrading linux
(any combination of kernel, basic rootfs and/or proprietary apps), and
one approach that was suggested (not by me) was to have two entirely
independent installations on the same hard drive so that, when one
"upgraded" the system, the newer content was installed in the other
partition and the system rebooted to that newer content, the rationale
being that, if the upgraded system didn't work, one could revert back
to the original install.

the history of this particular approach lies in the fact that the
original S/W running on the system in question was an embedded app
programmed right on bare metal, so the entire app was basically a
monolithic combination of boot loader, kernel and all the apps. in
that case, i can certainly see how one wanted a reliable way to revert
back to a known working system if the new S/W had flaws. but i don't
think that logic holds with a linux installation.

the major concern seems to be a fear that, if one upgrades the
kernel even a little bit, user space apps might suddenly stop working
because of some new incompatibility introduced by that upgrade. i'm
well aware of gregkh's piece on kernel stable API nonsense:

https://www.kernel.org/doc/Documentation/stable_api_nonsense.txt

so i'm not overly worried, but folks in the discussion coming from the
embedded programming space are using their experiences from that space
to argue that a fully redundant install is the safest approach just in
case something goes wrong with an upgrade.

not surprisingly, my position is that this is not something they
need to worry about a whole lot but, for the sake of fairness, i
thought i'd look around to see if any linux folks *are* using that
approach.

anyone? thoughts?

rday

-- 

========================================================================
Robert P. J. Day Ottawa, Ontario, CANADA
http://crashcourse.ca

Twitter: http://twitter.com/rpjday
LinkedIn: http://ca.linkedin.com/in/rpjday
========================================================================



_______________________________________________
Kernelnewbies mailing list
Kernelnewbies at kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

anyone aware of a high availability setup that relies on fully redundant install?

[Greg KH]

On Sun, Apr 17, 2016 at 10:47:55AM -0400, Robert P. J. Day wrote:
> 
>   i figure this is as good a place as any to ask ... is anyone here
> aware of anyone using a linux config and install that, for the
> purposes of reliability or high availability or whatever you want to
> call it, relies on a second, completely independent installation of
> linux on the same hard drive?

ChromeOS and CoreOS do this.  There's lots of documentation on the
ChromeOS site for how this works and what is involved.

anyone aware of a high availability setup that relies on fully redundant install?

[Robert P. J. Day]

On Sun, 17 Apr 2016, Greg KH wrote:

> On Sun, Apr 17, 2016 at 10:47:55AM -0400, Robert P. J. Day wrote:
> >
> >   i figure this is as good a place as any to ask ... is anyone here
> > aware of anyone using a linux config and install that, for the
> > purposes of reliability or high availability or whatever you want to
> > call it, relies on a second, completely independent installation of
> > linux on the same hard drive?
>
> ChromeOS and CoreOS do this.  There's lots of documentation on the
> ChromeOS site for how this works and what is involved.

  interesting ... does the CoreOS functionality depend on containers?
what little i know about CoreOS, i would think there's no way to use
it *without* containers.

rday

-- 

========================================================================
Robert P. J. Day                                 Ottawa, Ontario, CANADA
                        http://crashcourse.ca

Twitter:                                       http://twitter.com/rpjday
LinkedIn:                               http://ca.linkedin.com/in/rpjday
========================================================================

anyone aware of a high availability setup that relies on fully redundant install?

[Greg KH]

On Sun, Apr 17, 2016 at 01:16:10PM -0400, Robert P. J. Day wrote:
> On Sun, 17 Apr 2016, Greg KH wrote:
> 
> > On Sun, Apr 17, 2016 at 10:47:55AM -0400, Robert P. J. Day wrote:
> > >
> > >   i figure this is as good a place as any to ask ... is anyone here
> > > aware of anyone using a linux config and install that, for the
> > > purposes of reliability or high availability or whatever you want to
> > > call it, relies on a second, completely independent installation of
> > > linux on the same hard drive?
> >
> > ChromeOS and CoreOS do this.  There's lots of documentation on the
> > ChromeOS site for how this works and what is involved.
> 
>   interesting ... does the CoreOS functionality depend on containers?

Nope, not at all, it's the same design as ChromeOS.

> what little i know about CoreOS, i would think there's no way to use
> it *without* containers.

Not true at all, you can use it without containers, it's just not all
that useful as odds are, the application you want to use isn't included
in it so you need a container for that.

greg k-h

anyone aware of a high availability setup that relies on fully redundant install?

[Valdis.Kletnieks at vt.edu]

On Sun, 17 Apr 2016 10:47:55 -0400, "Robert P. J. Day" said:
>   i figure this is as good a place as any to ask ... is anyone here
> aware of anyone using a linux config and install that, for the
> purposes of reliability or high availability or whatever you want to
> call it, relies on a second, completely independent installation of
> linux on the same hard drive?

IBM's AIX has for a long time had the concept of an 'alternate boot volume',
which can be another logical volume on the same physical hard drive.  But it's
not intended for high-availability, it's for "if this software upgrade goes
pear-shaped I have an easy backout procedure".  And it avoids most of the "you
have to keep two version" issues by providing a tool to copy your *current*
system onto the alternate boot.  I'm sure some Linux distros have stolen the
concept.

Most implementations  of "high availability" would see the phrase "on the same
hard drive" and start pointing and laughing at the single point of failure.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 848 bytes
Desc: not available
Url : http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20160418/4fc0d3b4/attachment.bin 

anyone aware of a high availability setup that relies on fully redundant install?

[Robert P. J. Day]

On Mon, 18 Apr 2016, Valdis.Kletnieks at vt.edu wrote:

> On Sun, 17 Apr 2016 10:47:55 -0400, "Robert P. J. Day" said:
> >   i figure this is as good a place as any to ask ... is anyone here
> > aware of anyone using a linux config and install that, for the
> > purposes of reliability or high availability or whatever you want to
> > call it, relies on a second, completely independent installation of
> > linux on the same hard drive?
>
> IBM's AIX has for a long time had the concept of an 'alternate boot
> volume', which can be another logical volume on the same physical
> hard drive.  But it's not intended for high-availability, it's for
> "if this software upgrade goes pear-shaped I have an easy backout
> procedure".  And it avoids most of the "you have to keep two
> version" issues by providing a tool to copy your *current* system
> onto the alternate boot.  I'm sure some Linux distros have stolen
> the concept.

  that makes sense -- a *minimal* bootable system for recovery and
troubleshooting. but not a fully independent previous install.

> Most implementations of "high availability" would see the phrase "on
> the same hard drive" and start pointing and laughing at the single
> point of failure.

  trust me, i'm aware of that. :-) perhaps i shouldn't have used the
phrase "high availability", this proposal was more for just the
ability to back out of a botched or flawed upgrade.

rday

-- 

========================================================================
Robert P. J. Day                                 Ottawa, Ontario, CANADA
                        http://crashcourse.ca

Twitter:                                       http://twitter.com/rpjday
LinkedIn:                               http://ca.linkedin.com/in/rpjday
========================================================================

anyone aware of a high availability setup that relies on fully redundant install?

[Valdis.Kletnieks at vt.edu]

On Mon, 18 Apr 2016 06:29:21 -0400, "Robert P. J. Day" said:

>   that makes sense -- a *minimal* bootable system for recovery and
> troubleshooting. but not a fully independent previous install.

No, it's a *complete* system - your kernel boot image, /, /usr, /var, and
whatever other file systems you specified to go into 'rootvg'.  The use case is
that before doing maintenance or whatever, you run a shell script that clones
the entire rootvg over to alt_rootvg.  Or you can apply system updates to
the alternate boot, so instead of taking an outage for 2 hours to apply
all your fixes, you apply fixes onto the alternate, and your outage window
is only what it takes to reboot to the updated image - and you still have
the old image to fall back to.

Incredibly useful for that set of systems that you need to minimize downtime,
but the application in question isn't one that you can run multiple instances
behind a load balancer.  We've used it for everything from a Listserv server
to a TSM backup server.

http://sureshaix.blogspot.com/2008/07/alternate-disk-installation.html
http://www.drdobbs.com/aix-alternate-disk-installation/199101222

(Yes, that's an article from 1991.. :)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 848 bytes
Desc: not available
Url : http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20160418/450643d4/attachment.bin 

anyone aware of a high availability setup that relies on fully redundant install?

[Miles Fidelman]

On 4/18/16 12:48 AM, Valdis.Kletnieks at vt.edu wrote:
> On Sun, 17 Apr 2016 10:47:55 -0400, "Robert P. J. Day" said:
>>    i figure this is as good a place as any to ask ... is anyone here
>> aware of anyone using a linux config and install that, for the
>> purposes of reliability or high availability or whatever you want to
>> call it, relies on a second, completely independent installation of
>> linux on the same hard drive?
> <snip>
>
> Most implementations  of "high availability" would see the phrase "on the same
> hard drive" and start pointing and laughing at the single point of failure.
>

Yup.  That was my reaction.  HA starts with either 2 servers and either 
a) mirrored drives (e.g., using DRBD) or b) a HA RAID cluster.

Miles Fidelman

anyone aware of a high availability setup that relies on fully redundant install?

[Andrew Bradford]

Hi Robert,

On 04/17 10:47, Robert P. J. Day wrote:
> 
>   i figure this is as good a place as any to ask ... is anyone here
> aware of anyone using a linux config and install that, for the
> purposes of reliability or high availability or whatever you want to
> call it, relies on a second, completely independent installation of
> linux on the same hard drive?
> 
>   i was having a discussion about the reliability of upgrading linux
> (any combination of kernel, basic rootfs and/or proprietary apps), and
> one approach that was suggested (not by me) was to have two entirely
> independent installations on the same hard drive so that, when one
> "upgraded" the system, the newer content was installed in the other
> partition and the system rebooted to that newer content, the rationale
> being that, if the upgraded system didn't work, one could revert back
> to the original install.
> 
>   the history of this particular approach lies in the fact that the
> original S/W running on the system in question was an embedded app
> programmed right on bare metal, so the entire app was basically a
> monolithic combination of boot loader, kernel and all the apps. in
> that case, i can certainly see how one wanted a reliable way to revert
> back to a known working system if the new S/W had flaws. but i don't
> think that logic holds with a linux installation.
> 
>   the major concern seems to be a fear that, if one upgrades the
> kernel even a little bit, user space apps might suddenly stop working
> because of some new incompatibility introduced by that upgrade. i'm
> well aware of gregkh's piece on kernel stable API nonsense:
> 
> https://www.kernel.org/doc/Documentation/stable_api_nonsense.txt
> 
> so i'm not overly worried, but folks in the discussion coming from the
> embedded programming space are using their experiences from that space
> to argue that a fully redundant install is the safest approach just in
> case something goes wrong with an upgrade.
> 
>   not surprisingly, my position is that this is not something they
> need to worry about a whole lot but, for the sake of fairness, i
> thought i'd look around to see if any linux folks *are* using that
> approach.

Yes, in one product I'm working on now we have the flash partitioned out
with two separate root partitions for this reason.  Doing atomic
upgrades is not easy and this is one simple and effective way to avoid
issues arising out of failed or botched upgrades.  We're more worried
about power going out in the middle of an upgrade moreso than anything
else.

A software upgrade for us wipes the not-booted partition and installs
the entire new system into the not-booted partition, then updates the
bootloader configuration to use that as the root file system.  If the
system fails to boot 3 times (watchdog induced resets) then the
bootloader will fall back to the other partition.

To retain configuration settings across upgrades, there's yet another
partition with config data on it that both rootfs partitions use.  At
boot time the system validates that the config data is in the proper
format and fixes it if it's not.

Thanks,
Andrew