All of lore.kernel.org
 help / color / mirror / Atom feed
* [linux-lvm] Using LVM Mirroring to obtain a usable backup
@ 2009-09-16 13:59 Ambrogio De Lorenzo
  2009-09-16 15:05 ` André Gillibert
  2009-09-16 18:19 ` malahal
  0 siblings, 2 replies; 23+ messages in thread
From: Ambrogio De Lorenzo @ 2009-09-16 13:59 UTC (permalink / raw)
  To: LVM general discussion and development

Hi all.
I'm doing a big upgrade on my computer and I tryed to use lv mirroring
to obtain a copy of my data before to do it.

As I do with LVM on other OS i thought that after mirroring one could
chose to split the mirrored lv.

I can't find how to do it with LVM2 on Linux.

There is a way?
Tnx
 Ambrogio

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [linux-lvm] Using LVM Mirroring to obtain a usable backup
  2009-09-16 13:59 [linux-lvm] Using LVM Mirroring to obtain a usable backup Ambrogio De Lorenzo
@ 2009-09-16 15:05 ` André Gillibert
  2009-09-16 16:47   ` Ambrogio De Lorenzo
  2009-09-16 18:19 ` malahal
  1 sibling, 1 reply; 23+ messages in thread
From: André Gillibert @ 2009-09-16 15:05 UTC (permalink / raw)
  To: linux-lvm

Ambrogio De Lorenzo <ambrogio.de.lorenzo@alice.it> wrote:
> Hi all.
> I'm doing a big upgrade on my computer and I tryed to use lv mirroring
> to obtain a copy of my data before to do it.
> 
> As I do with LVM on other OS i thought that after mirroring one could
> chose to split the mirrored lv.
> 
> I can't find how to do it with LVM2 on Linux.
> 
> There is a way?

Mirroring is software RAID1.
What you want looks like LVM snapshots.
See LVM HOWTO to see how to create snapshots.
A backup may be a good thing to do, too.

-- 
André Gillibert

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [linux-lvm] Using LVM Mirroring to obtain a usable backup
  2009-09-16 15:05 ` André Gillibert
@ 2009-09-16 16:47   ` Ambrogio De Lorenzo
  2009-09-16 16:58     ` Brian J. Murrell
  2009-09-16 20:03     ` André Gillibert
  0 siblings, 2 replies; 23+ messages in thread
From: Ambrogio De Lorenzo @ 2009-09-16 16:47 UTC (permalink / raw)
  To: LVM general discussion and development

Il giorno mer, 16/09/2009 alle 17.05 +0200, André Gillibert ha scritto:

> Mirroring is software RAID1.
> What you want looks like LVM snapshots.
> See LVM HOWTO to see how to create snapshots.
> A backup may be a good thing to do, too.

snapshot is not a good choise for me. I want to have an identical copy
(like dd) on another identical disk, for fast recovery in case of
failure.

This is what I do with raid hardware, and I would like to know if is
possible to do it with lvm mirroring.

I'm looking also on mdadm to mirror a disk (or partition) but I think it
is not usable when disk is already partitioned and used without mdadm.

Backup is good but is not a good choise in this case, because a full
restore is a long process

Tnx
 Ambrogio

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [linux-lvm] Using LVM Mirroring to obtain a usable backup
  2009-09-16 16:47   ` Ambrogio De Lorenzo
@ 2009-09-16 16:58     ` Brian J. Murrell
  2009-09-16 19:15       ` Ambrogio De Lorenzo
  2009-09-16 20:03     ` André Gillibert
  1 sibling, 1 reply; 23+ messages in thread
From: Brian J. Murrell @ 2009-09-16 16:58 UTC (permalink / raw)
  To: LVM general discussion and development

[-- Attachment #1: Type: text/plain, Size: 1428 bytes --]

On Wed, 2009-09-16 at 18:47 +0200, Ambrogio De Lorenzo wrote:
> 
> snapshot is not a good choise for me.

It actually is.  I snapshot all of my filesystems before every upgrade
and let the snapshots hang around until I am happy with the upgrade.

> I want to have an identical copy
> (like dd) on another identical disk, for fast recovery in case of
> failure.

Well, snapshots give you a very space efficient (if not performance
efficient due to the COW nature of snapshots) version of that.  When you
create a snapshot and before your modify any data in the origin (the
volume you are creating a snapshot (a.k.a. copy) of your snapshot is an
identical copy (just as if you'd used dd) of the origin, without using
any additional space even!

You can go on to upgrading the origin and the snapshot will continue to
maintain the "copy" of the origin at the time the snapshot was created.
It's sweet.

> This is what I do with raid hardware, and I would like to know if is
> possible to do it with lvm mirroring.

I've never used LVM mirroring.  Certainly you can do this with Linux's
MD software mirroring, but why bother when snapshots are so much easier?

> I'm looking also on mdadm to mirror a disk (or partition) but I think it
> is not usable when disk is already partitioned and used without mdadm.

Well, you will have that same problem with LVM.  Planning is key in any
of this.

b.


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [linux-lvm] Using LVM Mirroring to obtain a usable backup
  2009-09-16 13:59 [linux-lvm] Using LVM Mirroring to obtain a usable backup Ambrogio De Lorenzo
  2009-09-16 15:05 ` André Gillibert
@ 2009-09-16 18:19 ` malahal
  2009-09-16 19:03   ` Ambrogio De Lorenzo
  1 sibling, 1 reply; 23+ messages in thread
From: malahal @ 2009-09-16 18:19 UTC (permalink / raw)
  To: linux-lvm

Ambrogio De Lorenzo [ambrogio.de.lorenzo@alice.it] wrote:
> Hi all.
> I'm doing a big upgrade on my computer and I tryed to use lv mirroring
> to obtain a copy of my data before to do it.
> 
> As I do with LVM on other OS i thought that after mirroring one could
> chose to split the mirrored lv.
> 
> I can't find how to do it with LVM2 on Linux.

snapshot may be a good fit in this case, but if you insist you can use
lvconvert to convert your linear volume to a mirrored volume. Once the
sync is complete, you can use the same lvconvert to convert the mirror
volume to a linear one.

Thanks, Malahal.

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [linux-lvm] Using LVM Mirroring to obtain a usable backup
  2009-09-16 18:19 ` malahal
@ 2009-09-16 19:03   ` Ambrogio De Lorenzo
  2009-09-16 20:33     ` malahal
  2009-09-17 10:05     ` Bryn M. Reeves
  0 siblings, 2 replies; 23+ messages in thread
From: Ambrogio De Lorenzo @ 2009-09-16 19:03 UTC (permalink / raw)
  To: LVM general discussion and development

Il giorno mer, 16/09/2009 alle 11.19 -0700, malahal@us.ibm.com ha
scritto:
> Ambrogio De Lorenzo [ambrogio.de.lorenzo@alice.it] wrote:
> > Hi all.
> > I'm doing a big upgrade on my computer and I tryed to use lv mirroring
> > to obtain a copy of my data before to do it.
> > 
> > As I do with LVM on other OS i thought that after mirroring one could
> > chose to split the mirrored lv.
> > 
> > I can't find how to do it with LVM2 on Linux.
> 
> snapshot may be a good fit in this case, but if you insist you can use
> lvconvert to convert your linear volume to a mirrored volume. Once the
> sync is complete, you can use the same lvconvert to convert the mirror
> volume to a linear one.
But with this I loose the secondary copy.
It's not a good idea.
Backup and destroy backup :-)

Bye
 Ambrogio

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [linux-lvm] Using LVM Mirroring to obtain a usable backup
  2009-09-16 16:58     ` Brian J. Murrell
@ 2009-09-16 19:15       ` Ambrogio De Lorenzo
  2009-09-16 19:49         ` Brian J. Murrell
  0 siblings, 1 reply; 23+ messages in thread
From: Ambrogio De Lorenzo @ 2009-09-16 19:15 UTC (permalink / raw)
  To: LVM general discussion and development

Il giorno mer, 16/09/2009 alle 12.58 -0400, Brian J. Murrell ha scritto:
> On Wed, 2009-09-16 at 18:47 +0200, Ambrogio De Lorenzo wrote:
> > 
> > snapshot is not a good choise for me.
> 
> It actually is.  I snapshot all of my filesystems before every upgrade
> and let the snapshots hang around until I am happy with the upgrade.
Imagine that upgrade change something in LVM configuration or in lvm
software.
Snapshots are good only if you have the original data and all the
modified data.
This is not a good way to rollback everything.

My idea is:
     1. Insert a new disk via USB
     2. configure mirroring
     3. continue to work and monitor for mirroring progress (performance
        are not a problem)
     4. In a good moment shutdown the system
     5. boot with a live CD with an LVM version compatible
     6. split mirroring of lv (this made available 2 copies with
        different names on 2 different disks)
     7. split VG
     8. reboot without the USB disk (that is a good copy of original VG)
     9. change everything I need (i.e. I can also destroy disk or
        destroy VG) and test it.
    10. If is good I can destroy the USB copy, if it isn't I can boot
        from USB

> Well, snapshots give you a very space efficient (if not performance
> efficient due to the COW nature of snapshots) version of that.  When you
> create a snapshot and before your modify any data in the origin (the
> volume you are creating a snapshot (a.k.a. copy) of your snapshot is an
> identical copy (just as if you'd used dd) of the origin, without using
> any additional space even!
I know very well how snapshot works. And it's for this reason that I
don't think it's a good choise for my scope

> > I'm looking also on mdadm to mirror a disk (or partition) but I think it
> > is not usable when disk is already partitioned and used without mdadm.
> 
> Well, you will have that same problem with LVM.  Planning is key in any
> of this.
Yes, but LVM is already configured for this reason.
Mdadm isn't. I tried to test it but I'm not able to obtain a mirror of
an already partitioned and used disk.

Bye
 Ambrogio

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [linux-lvm] Using LVM Mirroring to obtain a usable backup
  2009-09-16 19:15       ` Ambrogio De Lorenzo
@ 2009-09-16 19:49         ` Brian J. Murrell
  0 siblings, 0 replies; 23+ messages in thread
From: Brian J. Murrell @ 2009-09-16 19:49 UTC (permalink / raw)
  To: LVM general discussion and development

[-- Attachment #1: Type: text/plain, Size: 882 bytes --]

On Wed, 2009-09-16 at 21:15 +0200, Ambrogio De Lorenzo wrote:
> 
> Imagine that upgrade change something in LVM configuration or in lvm
> software.

Well, that is most certainly a corner case.  LVM/device-mapper is very
stable and never caused me a problem of this nature.

> Snapshots are good only if you have the original data and all the
> modified data.

Of course.  You have to have LVM in order for snapshots to work.

> I know very well how snapshot works. And it's for this reason that I
> don't think it's a good choise for my scope

So why not take this opportunity to learn something new?  I love
opportunities like that.

> Yes, but LVM is already configured for this reason.
> Mdadm isn't. I tried to test it but I'm not able to obtain a mirror of
> an already partitioned and used disk.

So (to beat a dead horse) use a snapshot then.

b.


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [linux-lvm] Using LVM Mirroring to obtain a usable backup
  2009-09-16 16:47   ` Ambrogio De Lorenzo
  2009-09-16 16:58     ` Brian J. Murrell
@ 2009-09-16 20:03     ` André Gillibert
  2009-09-16 20:22       ` Ambrogio De Lorenzo
  1 sibling, 1 reply; 23+ messages in thread
From: André Gillibert @ 2009-09-16 20:03 UTC (permalink / raw)
  To: linux-lvm

Ambrogio De Lorenzo <ambrogio.de.lorenzo@alice.it> wrote:
> Il giorno mer, 16/09/2009 alle 17.05 +0200, André Gillibert ha scritto:
> 
> > Mirroring is software RAID1.
> > What you want looks like LVM snapshots.
> > See LVM HOWTO to see how to create snapshots.
> > A backup may be a good thing to do, too.
> 
> snapshot is not a good choise for me. I want to have an identical copy
> (like dd) on another identical disk, for fast recovery in case of
> failure.
> 

Then, create a short-lived snapshot, dd the snapshot to the other disk, and then, remove the snapshot.

-- 
André Gillibert

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [linux-lvm] Using LVM Mirroring to obtain a usable backup
  2009-09-16 20:03     ` André Gillibert
@ 2009-09-16 20:22       ` Ambrogio De Lorenzo
  2009-09-16 22:03         ` Kai Stian Olstad
  0 siblings, 1 reply; 23+ messages in thread
From: Ambrogio De Lorenzo @ 2009-09-16 20:22 UTC (permalink / raw)
  To: LVM general discussion and development

Il giorno mer, 16/09/2009 alle 22.03 +0200, André Gillibert ha scritto:

> > snapshot is not a good choise for me. I want to have an identical copy
> > (like dd) on another identical disk, for fast recovery in case of
> > failure.
> > 
> 
> Then, create a short-lived snapshot, dd the snapshot to the other disk, and then, remove the snapshot.

Hm... 
Another no good approach.
In this way I loose everything appens to disk between the moment in
which I take the snap and the time in which dd end. I think at least 6/7
hours.

The good and most secure way is to mirror disks. In this way all blocks
are replicated, all I/O are replicated and disks are syncronized when I
shutdown the OS.

I will shutdown in a few moments and start an offline dd. Is the last
chance.
I hope it end fast

Bye
 Ambrogio

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [linux-lvm] Using LVM Mirroring to obtain a usable backup
  2009-09-16 19:03   ` Ambrogio De Lorenzo
@ 2009-09-16 20:33     ` malahal
  2009-09-17 10:05     ` Bryn M. Reeves
  1 sibling, 0 replies; 23+ messages in thread
From: malahal @ 2009-09-16 20:33 UTC (permalink / raw)
  To: linux-lvm

Ambrogio De Lorenzo [ambrogio.de.lorenzo@alice.it] wrote:
> > snapshot may be a good fit in this case, but if you insist you can use
> > lvconvert to convert your linear volume to a mirrored volume. Once the
> > sync is complete, you can use the same lvconvert to convert the mirror
> > volume to a linear one.
> But with this I loose the secondary copy.
> It's not a good idea.
> Backup and destroy backup :-)

If that is on a single PV, you can disconnect it to avoid it being
allocated by LVM for other things. I think, taking a snapshot and coping
it somewhere as someone said is a good idea.

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [linux-lvm] Using LVM Mirroring to obtain a usable backup
  2009-09-16 20:22       ` Ambrogio De Lorenzo
@ 2009-09-16 22:03         ` Kai Stian Olstad
  0 siblings, 0 replies; 23+ messages in thread
From: Kai Stian Olstad @ 2009-09-16 22:03 UTC (permalink / raw)
  To: LVM general discussion and development

On Wed, Sep 16, 2009 at 10:22 PM, Ambrogio De Lorenzo
<ambrogio.de.lorenzo@alice.it> wrote:
> The good and most secure way is to mirror disks. In this way all blocks
> are replicated, all I/O are replicated and disks are syncronized when I
> shutdown the OS.

You could use DRBD, then you would have a block to block copy on a
different machine.


-- 
Kai Stian

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [linux-lvm] Using LVM Mirroring to obtain a usable backup
  2009-09-16 19:03   ` Ambrogio De Lorenzo
  2009-09-16 20:33     ` malahal
@ 2009-09-17 10:05     ` Bryn M. Reeves
  2009-09-17 15:06       ` Stuart D. Gathman
  1 sibling, 1 reply; 23+ messages in thread
From: Bryn M. Reeves @ 2009-09-17 10:05 UTC (permalink / raw)
  To: LVM general discussion and development

On Wed, 2009-09-16 at 21:03 +0200, Ambrogio De Lorenzo wrote:
> Il giorno mer, 16/09/2009 alle 11.19 -0700, malahal@us.ibm.com ha
> scritto:
> > Ambrogio De Lorenzo [ambrogio.de.lorenzo@alice.it] wrote:
> > > Hi all.
> > > I'm doing a big upgrade on my computer and I tryed to use lv mirroring
> > > to obtain a copy of my data before to do it.
> > > 
> > > As I do with LVM on other OS i thought that after mirroring one could
> > > chose to split the mirrored lv.
> > > 
> > > I can't find how to do it with LVM2 on Linux.
> > 
> > snapshot may be a good fit in this case, but if you insist you can use
> > lvconvert to convert your linear volume to a mirrored volume. Once the
> > sync is complete, you can use the same lvconvert to convert the mirror
> > volume to a linear one.
> But with this I loose the secondary copy.
> It's not a good idea.
> Backup and destroy backup :-)

Well, the data is still there following the split but you are correct
that there is not currently a direct way to "split" the mirrored LV into
a pair of independent LVs.

Using a snapshot will not actually duplicate the data for you which
seems to be what you want to do.

If you don't want to muck about making manual changes you might be
better off creating a new LV and mirroring the data with dd or some
other block-level tool.

Regards,
Bryn.

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [linux-lvm] Using LVM Mirroring to obtain a usable backup
  2009-09-17 10:05     ` Bryn M. Reeves
@ 2009-09-17 15:06       ` Stuart D. Gathman
  2009-09-17 15:34         ` Brian J. Murrell
  0 siblings, 1 reply; 23+ messages in thread
From: Stuart D. Gathman @ 2009-09-17 15:06 UTC (permalink / raw)
  To: LVM general discussion and development

On Thu, 17 Sep 2009, Bryn M. Reeves wrote:

> If you don't want to muck about making manual changes you might be
> better off creating a new LV and mirroring the data with dd or some
> other block-level tool.

Using dd from a snapshot does not help the OP.  He says the dd will take
5-6 hours, and he needs a backup of the system *just before* the upgrade,
not hours before.  The rollback also needs to be quick.

A snapshot with dd does the "instant" backup part ok, but rollback
requires 5-6 hours to copy the snapsnot to a normal LV.

The OP is correct that creating and breaking a RAID1 mirror does what
he needs.  Apparently, LVM does not currently support this, although
it could be done with dmsetup.

I always build my VGs on md* devices.

My suggestion:

1. Add another disk, create partitions for software RAID1 (md) on the new
disk with missing mirrors, and size <= existing non-md PVs.

2. Make the md device(s) into LVM PVs, add to volume group.  

3. Migrate your LVs to the md PVs.

4. Remove non-md PVs.

5. Turn non-md partitions into md RAID1 partitions, and attach
(hot add) to the md devices created in step 1.

6. Wait for syncronization to complete.

7. Shutdown, remove one of the disks, and do the
upgrade.  (If you don't remove the disk and use mdadm to break the mirror,
you'll have a "hardware clone" which duplicates the VGIDs, and you'll need
to run the importclone script to rename+reUUID the clone.)

Alternate plan B for the paranoid:

Add *2* new disks, set up RAID1 md partitions with up to 3 mirrors, 2 active,
pvcreate, add to VG, and migrate LVs to the RAID1 PVs.  Add 3rd mirror,
synchronize, then physically remove or break mirror and importclone.
This ensures that your data always has@least 2 copies.

Alternate plan C for the brilliant:

Create new lvm command to clone LVs by creating a mirror and then breaking
it.  This is a good complement to a snapnot.  A snapsnot can be taken
"instantly", but take a long time to rollback (by copying to another LV).
A clone takes a long time to synchronize, but is current as of the
when the mirror is broken and can be used immediately as a real (non-snapsnot)
LV for rollback or other purposes.

-- 
	      Stuart D. Gathman <stuart@bmsi.com>
    Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [linux-lvm] Using LVM Mirroring to obtain a usable backup
  2009-09-17 15:06       ` Stuart D. Gathman
@ 2009-09-17 15:34         ` Brian J. Murrell
  2009-09-17 22:49           ` Stuart D. Gathman
  0 siblings, 1 reply; 23+ messages in thread
From: Brian J. Murrell @ 2009-09-17 15:34 UTC (permalink / raw)
  To: LVM general discussion and development

[-- Attachment #1: Type: text/plain, Size: 1970 bytes --]

On Thu, 2009-09-17 at 11:06 -0400, Stuart D. Gathman wrote:
> Using dd from a snapshot does not help the OP.

Right.

> He says the dd will take
> 5-6 hours, and he needs a backup of the system *just before* the upgrade,
> not hours before.

Which a snapshot will achieve.

> The rollback also needs to be quick.

Which a snapshot will achieve.

> A snapshot with dd does the "instant" backup part ok, but rollback
> requires 5-6 hours to copy the snapsnot to a normal LV.

Why would you copy the snapshot anywhere?  If you decided you needed a
roll-back, you boot from the snapshot of the root filesystem and have
the /etc/fstab in that snapshot mount any snapshots of other filesystems
he might have made.  Personally, I keep separate /, /usr, and /var and
so snapshot them all before an upgrade and fix up the /etc/fstab in the
snapshot-of-/ to mount the /usr and /var snapshots (rather than the
origins).

AFAICT, the only issue the OP has with snapshots is that he fears an
upgrade might screw up his LVM configuration.  Personally, I think that
is a low-probability fear, but everyone has to have their own comfort
level I guess.

> The OP is correct that creating and breaking a RAID1 mirror does what
> he needs.

Indeed.

> I always build my VGs on md* devices.

I typically have not.  I typically don't mirror anything except my
backup volume, which fully (as in bare metal restore able) backs up
every filesystem I have in my network.  For that reason I feel that
mirroring the disks in all the machines is overkill.

But granted, building systems on broken mirrors is easy enough and
allows one the flexibility of mirroring at a later date if one wants.

In fact I always thought/felt, that given that the MD superblock is at
the end of a device (or partition), shrinking the filesystem on a
partition and then trying to create an MD mirror with that partition
should be doable.  Never tried though.

b.


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [linux-lvm] Using LVM Mirroring to obtain a usable backup
  2009-09-17 15:34         ` Brian J. Murrell
@ 2009-09-17 22:49           ` Stuart D. Gathman
  2009-09-17 23:26             ` Les Mikesell
  2009-09-17 23:48             ` Brian J. Murrell
  0 siblings, 2 replies; 23+ messages in thread
From: Stuart D. Gathman @ 2009-09-17 22:49 UTC (permalink / raw)
  To: LVM general discussion and development

On Thu, 17 Sep 2009, Brian J. Murrell wrote:

> > The rollback also needs to be quick.
> 
> Which a snapshot will achieve.
> 
> > A snapshot with dd does the "instant" backup part ok, but rollback
> > requires 5-6 hours to copy the snapsnot to a normal LV.
> 
> Why would you copy the snapshot anywhere?  If you decided you needed a
> roll-back, you boot from the snapshot of the root filesystem and have
> the /etc/fstab in that snapshot mount any snapshots of other filesystems
> he might have made.  Personally, I keep separate /, /usr, and /var and
> so snapshot them all before an upgrade and fix up the /etc/fstab in the
> snapshot-of-/ to mount the /usr and /var snapshots (rather than the
> origins).

1) Eventually you still need to copy the snapshot to a normal LV to get 
your performance back - and that will involve 5-6 hours downtime.

2) (minor, but important) Another FAQ is "exactly how big do I need to make my
snapshot so that it is guaranteed never to overflow".

> > I always build my VGs on md* devices.
> 
> I typically have not.  I typically don't mirror anything except my
> backup volume, which fully (as in bare metal restore able) backs up
> every filesystem I have in my network.  For that reason I feel that
> mirroring the disks in all the machines is overkill.

Those with money to burn seem to favor SANs. (And cloning a PV with
a SAN and importclone is an easy solution to the OP problem - if only they had
a SAN.) I'm part of the Po' Fo'k contingent.

-- 
	      Stuart D. Gathman <stuart@bmsi.com>
    Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [linux-lvm] Using LVM Mirroring to obtain a usable backup
  2009-09-17 22:49           ` Stuart D. Gathman
@ 2009-09-17 23:26             ` Les Mikesell
  2009-09-18  0:58               ` Stuart D. Gathman
  2009-09-17 23:48             ` Brian J. Murrell
  1 sibling, 1 reply; 23+ messages in thread
From: Les Mikesell @ 2009-09-17 23:26 UTC (permalink / raw)
  To: LVM general discussion and development

Stuart D. Gathman wrote:
>  
> Those with money to burn seem to favor SANs. (And cloning a PV with
> a SAN and importclone is an easy solution to the OP problem - if only they had
> a SAN.) I'm part of the Po' Fo'k contingent.

Is a SAN something you can emulate with an iscsi target on fairly normal 
equipment?

-- 
   Les Mikesell
    lesmikesell@gmail.com

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [linux-lvm] Using LVM Mirroring to obtain a usable backup
  2009-09-17 22:49           ` Stuart D. Gathman
  2009-09-17 23:26             ` Les Mikesell
@ 2009-09-17 23:48             ` Brian J. Murrell
  2009-09-18  1:55               ` Stuart D. Gathman
  1 sibling, 1 reply; 23+ messages in thread
From: Brian J. Murrell @ 2009-09-17 23:48 UTC (permalink / raw)
  To: LVM general discussion and development

[-- Attachment #1: Type: text/plain, Size: 494 bytes --]

On Thu, 2009-09-17 at 18:49 -0400, Stuart D. Gathman wrote:
> 
> 1) Eventually you still need to copy the snapshot to a normal LV to get 
> your performance back

Will you?  When you are using the snapshot instead of the origin, you
are writing to the COW already, not writing to the origin which requires
a COW copy-out.

> 2) (minor, but important) Another FAQ is "exactly how big do I need to make my
> snapshot so that it is guaranteed never to overflow".

Heh.  Yeah.

b.


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [linux-lvm] Using LVM Mirroring to obtain a usable backup
  2009-09-17 23:26             ` Les Mikesell
@ 2009-09-18  0:58               ` Stuart D. Gathman
  2009-09-18  1:51                 ` Sven Eschenberg
  2009-09-18 16:05                 ` Mark H. Wood
  0 siblings, 2 replies; 23+ messages in thread
From: Stuart D. Gathman @ 2009-09-18  0:58 UTC (permalink / raw)
  To: LVM general discussion and development

On Thu, 17 Sep 2009, Les Mikesell wrote:

> Stuart D. Gathman wrote:
> > 
> > Those with money to burn seem to favor SANs. (And cloning a PV with
> > a SAN and importclone is an easy solution to the OP problem - if only they
> > had
> > a SAN.) I'm part of the Po' Fo'k contingent.
> 
> Is a SAN something you can emulate with an iscsi target on fairly normal
> equipment?

It depends on the software on the SAN server having something like
a snapshot or mirroring facility available.  At its simplest level, a SAN
server can be just a disk accessed by iSCSI or ATAoE.  (And even I can
afford that.)  But high end SAN servers are LVM systems (using something
like ZFS) and clients attach to logical drives that can be cloned, 
snapshotted, etc.  Kind of moves the whole LVM layer to the disk
subsystem (although it is still useful to have an additional LVM layer
locally).  More people people buying high end SAN servers might have
kept Sun from getting bought by Oracle :-)

A high end SAN server means you allocate "disks" from the SAN instead
of buying physical disks.  Adding another "disk" to a server can be as
simple as allocating another "LV" (or whatever the SAN software calls it)
and attaching it.  The OP would simply clone his disk in the SAN
(and needn't worry about the duplicate VGID as long he doesn't attach 
the clone) before doing the upgrade.

I have one client with a SAN system, and it seems to perform well.
It is one client that never needs to install a physical disk on a
server I maintain for them.  (Caveat, booting from iSCSI requires bringing up a
Nic - which is tricky to do for linux in initrd.)

-- 
	      Stuart D. Gathman <stuart@bmsi.com>
    Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [linux-lvm] Using LVM Mirroring to obtain a usable backup
  2009-09-18  0:58               ` Stuart D. Gathman
@ 2009-09-18  1:51                 ` Sven Eschenberg
  2009-09-18 16:05                 ` Mark H. Wood
  1 sibling, 0 replies; 23+ messages in thread
From: Sven Eschenberg @ 2009-09-18  1:51 UTC (permalink / raw)
  To: LVM general discussion and development

Configuring a nic in an initrd with busybox should not be too cumbersome.
For a straight forward approach one could use the kernel command line 
and tell it to get the info via dhcp.
And last but not least one could bootstrap load GPXE via the NICs PXE, 
which in turn passes the pxe discovered IP configuration into the 
kernel, or create some gpxe bootscript including the ip configuration to 
be passed to the kernel, if no dhcp is available.

There are quite some options actually ;-).


Regards

-Sven

Stuart D. Gathman schrieb:
> On Thu, 17 Sep 2009, Les Mikesell wrote:
> 
> 
> I have one client with a SAN system, and it seems to perform well.
> It is one client that never needs to install a physical disk on a
> server I maintain for them.  (Caveat, booting from iSCSI requires bringing up a
> Nic - which is tricky to do for linux in initrd.)
> 

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [linux-lvm] Using LVM Mirroring to obtain a usable backup
  2009-09-17 23:48             ` Brian J. Murrell
@ 2009-09-18  1:55               ` Stuart D. Gathman
  0 siblings, 0 replies; 23+ messages in thread
From: Stuart D. Gathman @ 2009-09-18  1:55 UTC (permalink / raw)
  To: LVM general discussion and development

On Thu, 17 Sep 2009, Brian J. Murrell wrote:

> > 1) Eventually you still need to copy the snapshot to a normal LV to get 
> > your performance back
> 
> Will you?  When you are using the snapshot instead of the origin, you
> are writing to the COW already, not writing to the origin which requires
> a COW copy-out.

Only if you are updating an entire fragment.  Otherwise, it is 
read/modify/write.

Furthermore, even when the snapshot is entirely filled out, the
fragments are in random physical order.  (Could be mitigated by
smart placement in the COW based on LV size.)

Furthermore, even if you don't care about performance, 
you currently *still* have to copy to a real LV to take another snapshot.  

Furthermore, if some future LVM version is capable of recursive snapshots, the
performance issue is intensified with each snapshot layer.

Actually, that last isn't strictly true.  There is the device mapper based
"Zumastor" product that uses a common COW shared among many snapshots
and can take multiple snapshots or snapshots of snapshots with no
(additional) performance degradation.  That could be a standard feature
in some future LVM version.

> > 2) (minor, but important) Another FAQ is "exactly how big do I need to make
> > my snapshot so that it is guaranteed never to overflow".
> 
> Heh.  Yeah.

This is instensified in the Zumastor idea, since overflow can potentially
invalidate hundreds of snapshots, some of which could be essential
(cloned virtual servers for a client).  (Maybe it's best to simply 
revert all the Zumastor LVs to readonly in that case.  Maybe it already does
that.)

-- 
	      Stuart D. Gathman <stuart@bmsi.com>
    Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [linux-lvm] Using LVM Mirroring to obtain a usable backup
  2009-09-18  0:58               ` Stuart D. Gathman
  2009-09-18  1:51                 ` Sven Eschenberg
@ 2009-09-18 16:05                 ` Mark H. Wood
  2009-09-18 19:12                   ` Stuart D. Gathman
  1 sibling, 1 reply; 23+ messages in thread
From: Mark H. Wood @ 2009-09-18 16:05 UTC (permalink / raw)
  To: linux-lvm

[-- Attachment #1: Type: text/plain, Size: 1483 bytes --]

On Thu, Sep 17, 2009 at 08:58:08PM -0400, Stuart D. Gathman wrote:
> A high end SAN server means you allocate "disks" from the SAN instead
> of buying physical disks.  Adding another "disk" to a server can be as
> simple as allocating another "LV" (or whatever the SAN software calls it)
> and attaching it.  The OP would simply clone his disk in the SAN
> (and needn't worry about the duplicate VGID as long he doesn't attach 
> the clone) before doing the upgrade.

Dunno if I'd call our HP MSA1510i "high end" but it works like that.
Just define an initiator, slice off some storage, and assign it.

I usually think of the FC-based stuff as "high end" since a single
fiber switch tends to cost more than we paid for our whole ~16TB iSCSI
SAN.  (Now expanded to more like 40TB and we should be able to take it
up to around 80TB.)

> I have one client with a SAN system, and it seems to perform well.
> It is one client that never needs to install a physical disk on a
> server I maintain for them.  (Caveat, booting from iSCSI requires
> bringing up a Nic - which is tricky to do for linux in initrd.)

A few (more pricey) NICs include iSCSI on the card and would appear to
serve as a normal boot disk so far as the host is concerned.  I
haven't tried that yet.  FC HBAs all seem to have this (and for what
they cost, they should).

-- 
Mark H. Wood, Lead System Programmer   mwood@IUPUI.Edu
Friends don't let friends publish revisable-form documents.

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [linux-lvm] Using LVM Mirroring to obtain a usable backup
  2009-09-18 16:05                 ` Mark H. Wood
@ 2009-09-18 19:12                   ` Stuart D. Gathman
  0 siblings, 0 replies; 23+ messages in thread
From: Stuart D. Gathman @ 2009-09-18 19:12 UTC (permalink / raw)
  To: LVM general discussion and development

On Fri, 18 Sep 2009, Mark H. Wood wrote:

> Dunno if I'd call our HP MSA1510i "high end" but it works like that.

For us Po' Fo'ks, anything more than $1000 is "high end".

-- 
	      Stuart D. Gathman <stuart@bmsi.com>
    Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

^ permalink raw reply	[flat|nested] 23+ messages in thread

end of thread, other threads:[~2009-09-18 19:12 UTC | newest]

Thread overview: 23+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2009-09-16 13:59 [linux-lvm] Using LVM Mirroring to obtain a usable backup Ambrogio De Lorenzo
2009-09-16 15:05 ` André Gillibert
2009-09-16 16:47   ` Ambrogio De Lorenzo
2009-09-16 16:58     ` Brian J. Murrell
2009-09-16 19:15       ` Ambrogio De Lorenzo
2009-09-16 19:49         ` Brian J. Murrell
2009-09-16 20:03     ` André Gillibert
2009-09-16 20:22       ` Ambrogio De Lorenzo
2009-09-16 22:03         ` Kai Stian Olstad
2009-09-16 18:19 ` malahal
2009-09-16 19:03   ` Ambrogio De Lorenzo
2009-09-16 20:33     ` malahal
2009-09-17 10:05     ` Bryn M. Reeves
2009-09-17 15:06       ` Stuart D. Gathman
2009-09-17 15:34         ` Brian J. Murrell
2009-09-17 22:49           ` Stuart D. Gathman
2009-09-17 23:26             ` Les Mikesell
2009-09-18  0:58               ` Stuart D. Gathman
2009-09-18  1:51                 ` Sven Eschenberg
2009-09-18 16:05                 ` Mark H. Wood
2009-09-18 19:12                   ` Stuart D. Gathman
2009-09-17 23:48             ` Brian J. Murrell
2009-09-18  1:55               ` Stuart D. Gathman

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.