[linux-lvm] Mirror between different SAN fabrics

[linux-lvm] Mirror between different SAN fabrics

[mathias.herzog]

[-- Attachment #1: Type: text/plain, Size: 883 bytes --]

Hi,

Using lvm2, I would like to mirror a logical volume between two
different SAN fabrics to guarantee high availability for my data, even
if a whole SAN fabric crashes.
Is there a possibility in lvm2 to define two volume groups, put the
disks from fabric1 into group 1 and the disks from fabric2 into group 2
and then create the mirror over the two groups? I didn't find much
documentation for lvm2 about this, I just know that other Volume
Managers can handle it.

It's essential to group the discs together because I use like 100 discs
for each mirror and I have to be sure that mirroring takes place between
the two different SAN storages.

Mathias



Sicherheitshinweis:
Dieses E-Mail von PostFinance ist signiert. Weitere Informationen finden Sie unter: 
https://www.postfinance.ch/e-signature.
Geben Sie Ihre Sicherheitselemente niemals Dritten bekannt.

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4479 bytes --]

Re: [linux-lvm] Mirror between different SAN fabrics

[Matt P]

[-- Attachment #1: Type: text/plain, Size: 1709 bytes --]

I can't think of any Volume Manager that would let you mirror disks from 2
separate VGs... I know Veritas won't let you, unless they've changed
something in the last version or 2... But other than that I wouldn't see why
you couldn't use 2 different fabrics, they're just disks to LVM... If you're
having trouble managing 200+ PVs, just build the LV first using only disks
from 1 fabric, then make the second fabric available/visible, and add a
mirror using only those disks... I think that would work...

On 12/20/06, mathias.herzog@postfinance.ch <mathias.herzog@postfinance.ch>
wrote:
>
> Hi,
>
> Using lvm2, I would like to mirror a logical volume between two
> different SAN fabrics to guarantee high availability for my data, even
> if a whole SAN fabric crashes.
> Is there a possibility in lvm2 to define two volume groups, put the
> disks from fabric1 into group 1 and the disks from fabric2 into group 2
> and then create the mirror over the two groups? I didn't find much
> documentation for lvm2 about this, I just know that other Volume
> Managers can handle it.
>
> It's essential to group the discs together because I use like 100 discs
> for each mirror and I have to be sure that mirroring takes place between
> the two different SAN storages.
>
> Mathias
>
>
>
> Sicherheitshinweis:
> Dieses E-Mail von PostFinance ist signiert. Weitere Informationen finden
> Sie unter:
> https://www.postfinance.ch/e-signature.
> Geben Sie Ihre Sicherheitselemente niemals Dritten bekannt.
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>
>
>

[-- Attachment #2: Type: text/html, Size: 2329 bytes --]

RE: [linux-lvm] Mirror between different SAN fabrics

[mathias.herzog]

[-- Attachment #1: Type: text/plain, Size: 1258 bytes --]


> From: linux-lvm-bounces@redhat.com 
> [mailto:linux-lvm-bounces@redhat.com] On Behalf Of Matt P
 
> I can't think of any Volume Manager that would let you mirror 
> disks from 2 separate VGs... 
Maybe not from 2 separate VGs but they add an additional level to group
PVs together. With HP-UX you can use "lvcreate -m -s g" to use different
PV-groups for the mirror. With Tru64 as far as I know you can use a Plex
to create a PV-group (should be similar with Veritas)
 
> But other than that I wouldn't see why you couldn't use 2 
> different fabrics, they're just disks to LVM... If you're 
> having trouble managing 200+ PVs, just build the LV first 
> using only disks from 1 fabric, then make the second fabric 
> available/visible, and add a mirror using only those disks... 
> I think that would work... 
That's an idea, but when I'm adding 4 more disks (eg. disk1 and disk2
from fabric1, disk3 and disk4 from fabric2), how can I tell LVM to
mirror Disk1,2 to Disk3,4 and not to mirror Disk1 to Disk2?

Regards Mathias

Sicherheitshinweis:
Dieses E-Mail von PostFinance ist signiert. Weitere Informationen finden Sie unter: 
https://www.postfinance.ch/e-signature.
Geben Sie Ihre Sicherheitselemente niemals Dritten bekannt.

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4479 bytes --]

Re: [linux-lvm] Mirror between different SAN fabrics

[Matt P]

I guess I should have tested the functionality out before I made the
comment. It seems the ability to add a mirror after the fact is still
not available (in lvconvert or otherwise). I'm sure someone will
correct me if I wrong on that.

I did figure out a crazy way you could do it, but it's messy, even
with small number of PVs. It would impact performance and is probably
not feasible in reality.

I think the proper way would be to use mdadm. Setup software RAID1
prior to getting LVM involved...

On 12/27/06, mathias.herzog@postfinance.ch
<mathias.herzog@postfinance.ch> wrote:
>
> > From: linux-lvm-bounces@redhat.com
> > [mailto:linux-lvm-bounces@redhat.com] On Behalf Of Matt P
>
> > I can't think of any Volume Manager that would let you mirror
> > disks from 2 separate VGs...
> Maybe not from 2 separate VGs but they add an additional level to group
> PVs together. With HP-UX you can use "lvcreate -m -s g" to use different
> PV-groups for the mirror. With Tru64 as far as I know you can use a Plex
> to create a PV-group (should be similar with Veritas)
>
> > But other than that I wouldn't see why you couldn't use 2
> > different fabrics, they're just disks to LVM... If you're
> > having trouble managing 200+ PVs, just build the LV first
> > using only disks from 1 fabric, then make the second fabric
> > available/visible, and add a mirror using only those disks...
> > I think that would work...
> That's an idea, but when I'm adding 4 more disks (eg. disk1 and disk2
> from fabric1, disk3 and disk4 from fabric2), how can I tell LVM to
> mirror Disk1,2 to Disk3,4 and not to mirror Disk1 to Disk2?
>
> Regards Mathias
>
> Sicherheitshinweis:
> Dieses E-Mail von PostFinance ist signiert. Weitere Informationen finden Sie unter:
> https://www.postfinance.ch/e-signature.
> Geben Sie Ihre Sicherheitselemente niemals Dritten bekannt.
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>
>
>

RE: [linux-lvm] Mirror between different SAN fabrics

[mathias.herzog]

[-- Attachment #1: Type: text/plain, Size: 569 bytes --]

 
> -----Original Message-----
> From: linux-lvm-bounces@redhat.com 
> [mailto:linux-lvm-bounces@redhat.com] On Behalf Of Matt P
[...]
> I think the proper way would be to use mdadm. Setup software 
> RAID1 prior to getting LVM involved...
That sounds like a good solution. So far I don't know mdadm but I'm
already getting into it

Thanks, Mathias

Sicherheitshinweis:
Dieses E-Mail von PostFinance ist signiert. Weitere Informationen finden Sie unter: 
https://www.postfinance.ch/e-signature.
Geben Sie Ihre Sicherheitselemente niemals Dritten bekannt.

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4479 bytes --]

RE: [linux-lvm] Mirror between different SAN fabrics

[Christian.Rohrmeier]

[-- Attachment #1: Type: text/plain, Size: 4328 bytes --]

Hi,

Here is a nice example from one of my RHEL 4 Oracle servers:

We have three layers:

first the LUNs from the SAN are multipathed to device aliases:

[root@ ~]# multipath -ll
sanb (XXXX60e8003f653000000XXXX000001c7)
[size=101 GB][features="1 queue_if_no_path"][hwhandler="0"]
\_ round-robin 0 [active]
 \_ 0:0:1:1 sdb 8:16 [active][ready]
 \_ 1:0:1:1 sdd 8:48 [active][ready]

sana (XXXX60e80039cbe000000XXXX000006ad)
[size=101 GB][features="1 queue_if_no_path"][hwhandler="0"]
\_ round-robin 0 [active]
 \_ 0:0:0:1 sda 8:0  [active][ready]
 \_ 1:0:0:1 sdc 8:32 [active][ready]

Next these multipath aliases are RAIDed:

[root@ ~]# mdadm --detail /dev/md0
/dev/md0:
        Version : 00.90.01
  Creation Time : Thu Nov  2 13:07:01 2006
     Raid Level : raid1
     Array Size : 106788160 (101.84 GiB 109.35 GB)
    Device Size : 106788160 (101.84 GiB 109.35 GB)
   Raid Devices : 2
  Total Devices : 2
Preferred Minor : 0
    Persistence : Superblock is persistent

    Update Time : Thu Dec 28 09:36:19 2006
          State : clean
 Active Devices : 2
Working Devices : 2
 Failed Devices : 0
  Spare Devices : 0


    Number   Major   Minor   RaidDevice State
       0     253        2        0      active sync   /dev/mapper/sana
       1     253        3        1      active sync   /dev/mapper/sanb
           UUID : b5ac4ae9:99da8114:744a7ebb:aba6f687
         Events : 0.4254576

And finally, the RAID device is used with LVM:

[root@ ~]# vgs -o +devices
  VG   #PV #LV #SN Attr   VSize   VFree Devices
  vg00   2   2   0 wz--n-  31.78G    0  /dev/cciss/c0d0p2(0)
  vg00   2   2   0 wz--n-  31.78G    0  /dev/cciss/c0d0p4(0)
  vg00   2   2   0 wz--n-  31.78G    0  /dev/cciss/c0d0p2(250)
  vg01   1   5   0 wz--n- 101.84G    0  /dev/md0(0)
  vg01   1   5   0 wz--n- 101.84G    0  /dev/md0(5120)
  vg01   1   5   0 wz--n- 101.84G    0  /dev/md0(5376)
  vg01   1   5   0 wz--n- 101.84G    0  /dev/md0(5632)
  vg01   1   5   0 wz--n- 101.84G    0  /dev/md0(8192)

This works very well, as both paths and mirrors are able to break away
without any disruption in disk access.

Cheers,

Christian


                                                                           
             <mathias.herzog@p                                             
             ostfinance.ch>                                                
             Sent by:                                                   To 
             linux-lvm-bounces         <linux-lvm@redhat.com>              
             @redhat.com                                                cc 
                                                                           
                                                                   Subject 
             28.12.2006 09:12          RE: [linux-lvm] Mirror between      
                                       different SAN fabrics               
                                                                           
             Please respond to                                             
                LVM general                                                
              discussion and                                               
                development                                                
             <linux-lvm@redhat                                             
                   .com>                                                   
                                                                           
                                                                           





> -----Original Message-----
> From: linux-lvm-bounces@redhat.com
> [mailto:linux-lvm-bounces@redhat.com] On Behalf Of Matt P
[...]
> I think the proper way would be to use mdadm. Setup software
> RAID1 prior to getting LVM involved...
That sounds like a good solution. So far I don't know mdadm but I'm
already getting into it

Thanks, Mathias

Sicherheitshinweis:
Dieses E-Mail von PostFinance ist signiert. Weitere Informationen finden
Sie unter:
https://www.postfinance.ch/e-signature.
Geben Sie Ihre Sicherheitselemente niemals Dritten bekannt.(See attached
file: smime.p7s)_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7, Size: 4479 bytes --]

RE: [linux-lvm] Mirror between different SAN fabrics

[mathias.herzog]

[-- Attachment #1: Type: text/plain, Size: 3000 bytes --]

Hi

Looks nice your solution. 
But I just found out that unlike lvm2, mdadm is not cluster aware. It
seems not possible to transfer RAID state information from one node to
another.
As we use Red Hat Cluster Suite, we depend on a cluster solution.

Regards Mathias

> -----Original Message-----
> From: linux-lvm-bounces@redhat.com 
> [mailto:linux-lvm-bounces@redhat.com] On Behalf Of 
> Christian.Rohrmeier@SCHERING.DE
> Sent: Donnerstag, 28 Dezember, 2006 09:49
[...] 
> Hi,
> 
> Here is a nice example from one of my RHEL 4 Oracle servers:
> 
> We have three layers:
> 
> first the LUNs from the SAN are multipathed to device aliases:
> 
> [root@ ~]# multipath -ll
> sanb (XXXX60e8003f653000000XXXX000001c7)
> [size=101 GB][features="1 queue_if_no_path"][hwhandler="0"] 
> \_ round-robin 0 [active]  \_ 0:0:1:1 sdb 8:16 
> [active][ready]  \_ 1:0:1:1 sdd 8:48 [active][ready]
> 
> sana (XXXX60e80039cbe000000XXXX000006ad)
> [size=101 GB][features="1 queue_if_no_path"][hwhandler="0"] 
> \_ round-robin 0 [active]  \_ 0:0:0:1 sda 8:0  
> [active][ready]  \_ 1:0:0:1 sdc 8:32 [active][ready]
> 
> Next these multipath aliases are RAIDed:
> 
> [root@ ~]# mdadm --detail /dev/md0
> /dev/md0:
>         Version : 00.90.01
>   Creation Time : Thu Nov  2 13:07:01 2006
>      Raid Level : raid1
>      Array Size : 106788160 (101.84 GiB 109.35 GB)
>     Device Size : 106788160 (101.84 GiB 109.35 GB)
>    Raid Devices : 2
>   Total Devices : 2
> Preferred Minor : 0
>     Persistence : Superblock is persistent
> 
>     Update Time : Thu Dec 28 09:36:19 2006
>           State : clean
>  Active Devices : 2
> Working Devices : 2
>  Failed Devices : 0
>   Spare Devices : 0
> 
> 
>     Number   Major   Minor   RaidDevice State
>        0     253        2        0      active sync   /dev/mapper/sana
>        1     253        3        1      active sync   /dev/mapper/sanb
>            UUID : b5ac4ae9:99da8114:744a7ebb:aba6f687
>          Events : 0.4254576
> 
> And finally, the RAID device is used with LVM:
> 
> [root@ ~]# vgs -o +devices
>   VG   #PV #LV #SN Attr   VSize   VFree Devices
>   vg00   2   2   0 wz--n-  31.78G    0  /dev/cciss/c0d0p2(0)
>   vg00   2   2   0 wz--n-  31.78G    0  /dev/cciss/c0d0p4(0)
>   vg00   2   2   0 wz--n-  31.78G    0  /dev/cciss/c0d0p2(250)
>   vg01   1   5   0 wz--n- 101.84G    0  /dev/md0(0)
>   vg01   1   5   0 wz--n- 101.84G    0  /dev/md0(5120)
>   vg01   1   5   0 wz--n- 101.84G    0  /dev/md0(5376)
>   vg01   1   5   0 wz--n- 101.84G    0  /dev/md0(5632)
>   vg01   1   5   0 wz--n- 101.84G    0  /dev/md0(8192)
> 
> This works very well, as both paths and mirrors are able to 
> break away without any disruption in disk access.
> 
> Cheers,
> 
> Christian

Sicherheitshinweis:
Dieses E-Mail von PostFinance ist signiert. Weitere Informationen finden Sie unter: 
https://www.postfinance.ch/e-signature.
Geben Sie Ihre Sicherheitselemente niemals Dritten bekannt.

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4479 bytes --]

RE: [linux-lvm] Mirror between different SAN fabrics

[Christian.Rohrmeier]

[-- Attachment #1: Type: text/plain, Size: 5364 bytes --]


Hi,

I haven't tried it in a cluster yet. I was planning on using HP's
MC-ServiceGuard to deal with HA clustering. I don't see why the LUNs that
are used on one system with mdadm can't be used another, since the RAID
block is on the disk and is readable even on a system upon which it wasn't
created on. /etc/mdadm.conf will ofcourse need to be copied and kept
current on all cluster nodes, but with the config file and the RAID block
on the disk, an "mdadm --assemble" should work. Importing the LVM
structures should then also not be a problem.

Well, like I said, thats what I assume will work, as I have not yet tested
this setup. I will do so shortly and can report my findings to the list. If
that doesn't work, then I would, as you suggested, also try RH cluster
suite.

Cheers,

Christian



                                                                           
             <mathias.herzog@p                                             
             ostfinance.ch>                                                
             Sent by:                                                   To 
             linux-lvm-bounces         <linux-lvm@redhat.com>              
             @redhat.com                                                cc 
                                                                           
                                                                   Subject 
             28.12.2006 11:13          RE: [linux-lvm] Mirror between      
                                       different SAN fabrics               
                                                                           
             Please respond to                                             
                LVM general                                                
              discussion and                                               
                development                                                
             <linux-lvm@redhat                                             
                   .com>                                                   
                                                                           
                                                                           




Hi

Looks nice your solution.
But I just found out that unlike lvm2, mdadm is not cluster aware. It
seems not possible to transfer RAID state information from one node to
another.
As we use Red Hat Cluster Suite, we depend on a cluster solution.

Regards Mathias

> -----Original Message-----
> From: linux-lvm-bounces@redhat.com
> [mailto:linux-lvm-bounces@redhat.com] On Behalf Of
> Christian.Rohrmeier@SCHERING.DE
> Sent: Donnerstag, 28 Dezember, 2006 09:49
[...]
> Hi,
>
> Here is a nice example from one of my RHEL 4 Oracle servers:
>
> We have three layers:
>
> first the LUNs from the SAN are multipathed to device aliases:
>
> [root@ ~]# multipath -ll
> sanb (XXXX60e8003f653000000XXXX000001c7)
> [size=101 GB][features="1 queue_if_no_path"][hwhandler="0"]
> \_ round-robin 0 [active]  \_ 0:0:1:1 sdb 8:16
> [active][ready]  \_ 1:0:1:1 sdd 8:48 [active][ready]
>
> sana (XXXX60e80039cbe000000XXXX000006ad)
> [size=101 GB][features="1 queue_if_no_path"][hwhandler="0"]
> \_ round-robin 0 [active]  \_ 0:0:0:1 sda 8:0
> [active][ready]  \_ 1:0:0:1 sdc 8:32 [active][ready]
>
> Next these multipath aliases are RAIDed:
>
> [root@ ~]# mdadm --detail /dev/md0
> /dev/md0:
>         Version : 00.90.01
>   Creation Time : Thu Nov  2 13:07:01 2006
>      Raid Level : raid1
>      Array Size : 106788160 (101.84 GiB 109.35 GB)
>     Device Size : 106788160 (101.84 GiB 109.35 GB)
>    Raid Devices : 2
>   Total Devices : 2
> Preferred Minor : 0
>     Persistence : Superblock is persistent
>
>     Update Time : Thu Dec 28 09:36:19 2006
>           State : clean
>  Active Devices : 2
> Working Devices : 2
>  Failed Devices : 0
>   Spare Devices : 0
>
>
>     Number   Major   Minor   RaidDevice State
>        0     253        2        0      active sync   /dev/mapper/sana
>        1     253        3        1      active sync   /dev/mapper/sanb
>            UUID : b5ac4ae9:99da8114:744a7ebb:aba6f687
>          Events : 0.4254576
>
> And finally, the RAID device is used with LVM:
>
> [root@ ~]# vgs -o +devices
>   VG   #PV #LV #SN Attr   VSize   VFree Devices
>   vg00   2   2   0 wz--n-  31.78G    0  /dev/cciss/c0d0p2(0)
>   vg00   2   2   0 wz--n-  31.78G    0  /dev/cciss/c0d0p4(0)
>   vg00   2   2   0 wz--n-  31.78G    0  /dev/cciss/c0d0p2(250)
>   vg01   1   5   0 wz--n- 101.84G    0  /dev/md0(0)
>   vg01   1   5   0 wz--n- 101.84G    0  /dev/md0(5120)
>   vg01   1   5   0 wz--n- 101.84G    0  /dev/md0(5376)
>   vg01   1   5   0 wz--n- 101.84G    0  /dev/md0(5632)
>   vg01   1   5   0 wz--n- 101.84G    0  /dev/md0(8192)
>
> This works very well, as both paths and mirrors are able to
> break away without any disruption in disk access.
>
> Cheers,
>
> Christian

Sicherheitshinweis:
Dieses E-Mail von PostFinance ist signiert. Weitere Informationen finden
Sie unter:
https://www.postfinance.ch/e-signature.
Geben Sie Ihre Sicherheitselemente niemals Dritten bekannt.(See attached
file: smime.p7s)_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7, Size: 4479 bytes --]

RE: [linux-lvm] Mirror between different SAN fabrics

[Graham Wood]

> I haven't tried it in a cluster yet. I was planning on using HP's
> MC-ServiceGuard to deal with HA clustering. I don't see why the LUNs that
> are used on one system with mdadm can't be used another, since the RAID
> block is on the disk and is readable even on a system upon which it wasn't
> created on. /etc/mdadm.conf will ofcourse need to be copied and kept
> current on all cluster nodes, but with the config file and the RAID block
> on the disk, an "mdadm --assemble" should work. Importing the LVM
> structures should then also not be a problem.
Assuming that the underlying devices are "clean" this may indeed  
work... Sometimes.

However, things like the dirty region log are going to be a mess.

Imagine that you've got apache running on node 1 using one GFS volume,  
and mysql on the second using another - both backed onto the same md  
physical volume.  They will each be writing to the same DRL their  
dirty regions, and trampling all over each other's status information.

Then node2 crashes with some IO in progress.  In the time taken for it  
to reboot, the DRL could have been totally over-written by node1 - at  
this point there may be differences between the two underlying devices  
that you don't know about, and you've just caused data corruption.

Even if the devices were clean when the second node came up, the first  
has it open, and the fact that it's not in a clean/shutdown state is  
likely to be recorded too, and node2 is going to be unhappy about that  
too.

All in all, unless md is cluster "aware", it's likely to cause you  
trouble down the line....

Graham

RE: [linux-lvm] Mirror between different SAN fabrics

[Christian.Rohrmeier]

Hi,

ahhh, I see where the difference in view comes from: MC-Service Guard is a
HA clustering suite that does not contain a cluster FS; it simply does
package switching upon failover between two or more nodes. The LUNs of the
(under the multipath/RAID/LVM sandwitch) FS that is mounted on one system
are seen by the other, but the FS is NOT mounted on the second system. On
fail-over, the package is switched from one node to the other, and the
mounting of the FS is also moved from one to the other. Hence, the RAID
block and the LVM structures are consistent (albeit possibly not cleanly
unmounted) when they are taken ahold of by the second node.

Right, if I wanted to have cluster FS I'd have to use Veritas or RH Cluster
Suite. MC-Serviceguard is somehting different. =)

-Christian
_________________
Christian Rohrmeier
Schering AG
Corporate IT - Infrastructure and Services
Computer Systems and Operations
Unix Server Operations
Tel +49 30 468 15794
Fax +49 30 468 95794


                                                                           
             Graham Wood                                                   
             <gwood@dragonhold                                             
             .org>                                                      To 
             Sent by:                  LVM general discussion and          
             linux-lvm-bounces         development <linux-lvm@redhat.com>  
             @redhat.com                                                cc 
                                                                           
                                                                   Subject 
             28.12.2006 12:09          RE: [linux-lvm] Mirror between      
                                       different SAN fabrics               
                                                                           
             Please respond to                                             
                LVM general                                                
              discussion and                                               
                development                                                
             <linux-lvm@redhat                                             
                   .com>                                                   
                                                                           
                                                                           





> I haven't tried it in a cluster yet. I was planning on using HP's
> MC-ServiceGuard to deal with HA clustering. I don't see why the LUNs that
> are used on one system with mdadm can't be used another, since the RAID
> block is on the disk and is readable even on a system upon which it
wasn't
> created on. /etc/mdadm.conf will ofcourse need to be copied and kept
> current on all cluster nodes, but with the config file and the RAID block
> on the disk, an "mdadm --assemble" should work. Importing the LVM
> structures should then also not be a problem.
Assuming that the underlying devices are "clean" this may indeed
work... Sometimes.

However, things like the dirty region log are going to be a mess.

Imagine that you've got apache running on node 1 using one GFS volume,
and mysql on the second using another - both backed onto the same md
physical volume.  They will each be writing to the same DRL their
dirty regions, and trampling all over each other's status information.

Then node2 crashes with some IO in progress.  In the time taken for it
to reboot, the DRL could have been totally over-written by node1 - at
this point there may be differences between the two underlying devices
that you don't know about, and you've just caused data corruption.

Even if the devices were clean when the second node came up, the first
has it open, and the fact that it's not in a clean/shutdown state is
likely to be recorded too, and node2 is going to be unhappy about that
too.

All in all, unless md is cluster "aware", it's likely to cause you
trouble down the line....

Graham


_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

RE: [linux-lvm] Mirror between different SAN fabrics

[Graham Wood]

> ahhh, I see where the difference in view comes from: MC-Service Guard is a
> HA clustering suite that does not contain a cluster FS; it simply does
> package switching upon failover between two or more nodes.
Yeah - if you're looking to create as part of a startup script for the  
"package", and remove it when you stop it being live, that should  
indeed work.

At that stage the md stuff is only ever accessed on a single node, and  
there's no problem.

MC-Serviceguard (last time I used it anyway) is very clean & simple -  
but doesn't provide the same functionality :)

Graham

RE: [linux-lvm] Mirror between different SAN fabrics

[mathias.herzog]

[-- Attachment #1: Type: text/plain, Size: 627 bytes --]


[...] 
> At that stage the md stuff is only ever accessed on a single 
> node, and there's no problem.

I will use GFS Filesystem with all cluster nodes up and running at the
same time, sharing their disks.
So my problem with the lvm2 mirroring feature still exists. Think I have
to use the expensive and not easy configurable Continuous Access
Solution to mirror directly on SAN fabric level...

Mathias

Sicherheitshinweis:
Dieses E-Mail von PostFinance ist signiert. Weitere Informationen finden Sie unter: 
https://www.postfinance.ch/e-signature.
Geben Sie Ihre Sicherheitselemente niemals Dritten bekannt.

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4479 bytes --]

Re: [linux-lvm] Mirror between different SAN fabrics

[Ty! Boyack]

I'm wondering how well a "stacked" lvm approach would work for this.  
Could you take LVM and make a two VGs called "Fabric1VG" and 
"Fabric2VG", where you put all of the "fabric 1 paths" to your PVs into 
Fabric1VG, and all your "fabric 2 paths" to your PVs into Fabric2VG.  
Then form volumes in each of those (it would be your responsibility to 
ensure that you have equal volumes in each fabric group).  So you would 
have Fabric1VG/LVa and Fabric2VG/LVa as "equivilant" devices, going 
across your different fabric paths.  Then you could create a new VG 
called MirrorVG and put both Fabric1VG/LVa and Fabric2VG/LVa into 
MirrorVG as PVs.  Then I think you should be able to create a new 
mirrored LV from MirrorVG which would mirror across your two fabrics.  
This should move all of your management into LVM2, which is cluster 
aware, but it 1) makes management a bit messy (but it is all quite 
scriptable), and 2) adds a wierd LVM 2-layer setup.

Anyone know if this 2-layer LVM approach would kill performance (any 
more than a two layer approach of lvm->mdadm or mdadm->lvm)?

Hmm... Thinking about this further, I've been thinking of this 2-layer 
approach for striping and mirroring, but it looks like it might be more 
problematic for your case, where you really want multipath and 
mirroring.  I'm not sure how to ensure that Fabric1VG/LVa and 
Fabric2VG/LVa are placed on the same PV blocks as the other one.  When 
you create Fabric1VG/LVa it might end up on disks 0,1, and 2, but 
Fabric2VG/LVa might end up on disks 1, 2, and 3 (but by different 
paths).  Anyone know any way to ensure that block placement is 
identical?  Is the block allocation algorithm predictable and repeatable 
so that if you have two VGs with equal PVs in each, and you create LVs 
in the same order in each, they get the same PV mapping?  Or is there a 
built-in randomness?  You would likely be assured of the same failures, 
since a bad block on fabric 1 should also be a bad block on fabric 2, 
and a failed PV would fail in both LVs...  Might work, if the placement 
routines are predictably defined. 

-Ty!




mathias.herzog@postfinance.ch wrote:
> [...] 
>   
>> At that stage the md stuff is only ever accessed on a single 
>> node, and there's no problem.
>>     
>
> I will use GFS Filesystem with all cluster nodes up and running at the
> same time, sharing their disks.
> So my problem with the lvm2 mirroring feature still exists. Think I have
> to use the expensive and not easy configurable Continuous Access
> Solution to mirror directly on SAN fabric level...
>
> Mathias
>
> Sicherheitshinweis:
> Dieses E-Mail von PostFinance ist signiert. Weitere Informationen finden Sie unter: 
> https://www.postfinance.ch/e-signature.
> Geben Sie Ihre Sicherheitselemente niemals Dritten bekannt.
> ------------------------------------------------------------------------
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/


-- 
-===========================-
  Ty! Boyack
  NREL Unix Network Manager
  ty@nrel.colostate.edu
  (970) 491-1186
-===========================-

Re: [linux-lvm] Mirror between different SAN fabrics

[Matt P]

This is basically the "messy" way I mentioned in my reply above. I
found if you pvcreate the  LV device, you end up mangling the lvm data
(this probably comes as little surprise) and it breaks down after
that. So, I ended up using losetup and an "image file", one for/on
each fabric. Then did pvcreate on each loop device, and made a new VG
containing both PVs and created the LV with mirroring.... It
worked.... I did no performance, stability, or failure testing...


On 12/28/06, Ty! Boyack <ty@nrel.colostate.edu> wrote:
> I'm wondering how well a "stacked" lvm approach would work for this.
> Could you take LVM and make a two VGs called "Fabric1VG" and
> "Fabric2VG", where you put all of the "fabric 1 paths" to your PVs into
> Fabric1VG, and all your "fabric 2 paths" to your PVs into Fabric2VG.
> Then form volumes in each of those (it would be your responsibility to
> ensure that you have equal volumes in each fabric group).  So you would
> have Fabric1VG/LVa and Fabric2VG/LVa as "equivilant" devices, going
> across your different fabric paths.  Then you could create a new VG
> called MirrorVG and put both Fabric1VG/LVa and Fabric2VG/LVa into
> MirrorVG as PVs.  Then I think you should be able to create a new
> mirrored LV from MirrorVG which would mirror across your two fabrics.
> This should move all of your management into LVM2, which is cluster
> aware, but it 1) makes management a bit messy (but it is all quite
> scriptable), and 2) adds a wierd LVM 2-layer setup.
>
> Anyone know if this 2-layer LVM approach would kill performance (any
> more than a two layer approach of lvm->mdadm or mdadm->lvm)?
>
> Hmm... Thinking about this further, I've been thinking of this 2-layer
> approach for striping and mirroring, but it looks like it might be more
> problematic for your case, where you really want multipath and
> mirroring.  I'm not sure how to ensure that Fabric1VG/LVa and
> Fabric2VG/LVa are placed on the same PV blocks as the other one.  When
> you create Fabric1VG/LVa it might end up on disks 0,1, and 2, but
> Fabric2VG/LVa might end up on disks 1, 2, and 3 (but by different
> paths).  Anyone know any way to ensure that block placement is
> identical?  Is the block allocation algorithm predictable and repeatable
> so that if you have two VGs with equal PVs in each, and you create LVs
> in the same order in each, they get the same PV mapping?  Or is there a
> built-in randomness?  You would likely be assured of the same failures,
> since a bad block on fabric 1 should also be a bad block on fabric 2,
> and a failed PV would fail in both LVs...  Might work, if the placement
> routines are predictably defined.
>
> -Ty!
>
>
>
>
> mathias.herzog@postfinance.ch wrote:
> > [...]
> >
> >> At that stage the md stuff is only ever accessed on a single
> >> node, and there's no problem.
> >>
> >
> > I will use GFS Filesystem with all cluster nodes up and running at the
> > same time, sharing their disks.
> > So my problem with the lvm2 mirroring feature still exists. Think I have
> > to use the expensive and not easy configurable Continuous Access
> > Solution to mirror directly on SAN fabric level...
> >
> > Mathias
> >
> > Sicherheitshinweis:
> > Dieses E-Mail von PostFinance ist signiert. Weitere Informationen finden Sie unter:
> > https://www.postfinance.ch/e-signature.
> > Geben Sie Ihre Sicherheitselemente niemals Dritten bekannt.
> > ------------------------------------------------------------------------
> >
> > _______________________________________________
> > linux-lvm mailing list
> > linux-lvm@redhat.com
> > https://www.redhat.com/mailman/listinfo/linux-lvm
> > read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>
>
> --
> -===========================-
>   Ty! Boyack
>   NREL Unix Network Manager
>   ty@nrel.colostate.edu
>   (970) 491-1186
> -===========================-
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>

Re: [linux-lvm] Mirror between different SAN fabrics

[Ty! Boyack]

Actually, I'm quite surprised that this approach mangles the lvm data.  
It seems that when you do a pvcreate on a block device, LVM should (and 
I think, does) write the lvm metadata in a region of that device, and 
then never let anything touch that metadata.  This way, if you do a 'dd 
if=/dev/zeros of=<PV-DEVICE>' it blanks out the device, but the metadata 
is intact.

So, if you do a 'pvcreate' on an LV, it should contain a second copy of 
the metadata, unique and independent from the first copy on the original 
block device.  My tests on this has worked fine (although my tests have 
been for building two VGs that have striped volumes across the member 
disks, and then a VG that creates a mirrored LV of the striped volumes, 
so no multipathing is involved).  I'm wondering if we can compare notes 
to see if I'm doing something that makes it look like it's working -- I 
don't want to be quietly destroying my lvm data and not knowing it!!!

I'm doing (roughly, block devices are a bit made-up):

# prepare the physical volumes
pvcreate /dev/sda
pvcreate /dev/sdb
pvcreate /dev/sdc
pvcreate /dev/sdd
pvcreate /dev/sde

# Create volume groups to contain uniquely striped volumes
vgcreate Stripe1VG /dev/sda /dev/sdb
vgcreate Stripe2VG /dev/sdc /dev/sdd

# Create the striped volumes
lvcreate -i 2 -n Stripe1LV -L 1G Stripe1VG
lvcreate -i 2 -n Stripe2LV -L 1G Stripe2VG

# Make the striped volumes into PVs
pvcreate /dev/Stripe1VG/Stripe1LV
pvcreate /dev/Stripe2VG/Stripe2LV

# Create the volume group for mirrored volumes
vgcreate MirrorVG /dev/Stripe1VG/Stripe1LV /dev/Stripe2VG/Stripe2LV /dev/sde
# (Had to use three PVs to have the mirror log in place)

# Create the mirrored volume
lvcreate -m 1 -n Mirror1LV -L 500M MirrorVG

# Filesystem, test, etc.  this will be GFS eventually, but testing with 
ext3 for now.
mke2fs -j -i16384 -v /dev/MirrorVG/Mirror1LV
mkdir /mnt/mirror1lv
mount /dev/MirrorVG/Mirror1LV /mnt/mirror1



Is that about your procedure as well?  When does the lvm data get mangled?

(Sorry if this is going off topic - but if this is solvable it might 
actually answer the original question...)

-Ty!




Matt P wrote:
> This is basically the "messy" way I mentioned in my reply above. I
> found if you pvcreate the  LV device, you end up mangling the lvm data
> (this probably comes as little surprise) and it breaks down after
> that. So, I ended up using losetup and an "image file", one for/on
> each fabric. Then did pvcreate on each loop device, and made a new VG
> containing both PVs and created the LV with mirroring.... It
> worked.... I did no performance, stability, or failure testing...


-- 
-===========================-
  Ty! Boyack
  NREL Unix Network Manager
  ty@nrel.colostate.edu
  (970) 491-1186
-===========================-

Re: [linux-lvm] Mirror between different SAN fabrics

[Matt P]

[-- Attachment #1: Type: text/plain, Size: 3907 bytes --]

That's exactly the steps I used to test it out. The failure I encountered
was at the point of adding the StripeLVs to the MirrorVG. I don't recall the
error I got but I seem to recall it being the same error as if the PV hadn't
been pvcreated. Although when I went through the process again just now,
without any errors... It's quite likely I mistyped or skipped a step during
my first attempt...

I've gone through the whole process 4 times now from raw disk to a mirrored
LV without getting the error again.

It looks as though this could work and isn't nearly as crazy as my initial
configuration. I wonder now if anything wonky will happen if we were to lose
a disk or in the original post if we lost one of the fabrics...

On 12/28/06, Ty! Boyack <ty@nrel.colostate.edu> wrote:
>
> Actually, I'm quite surprised that this approach mangles the lvm data.
> It seems that when you do a pvcreate on a block device, LVM should (and
> I think, does) write the lvm metadata in a region of that device, and
> then never let anything touch that metadata.  This way, if you do a 'dd
> if=/dev/zeros of=<PV-DEVICE>' it blanks out the device, but the metadata
> is intact.
>
> So, if you do a 'pvcreate' on an LV, it should contain a second copy of
> the metadata, unique and independent from the first copy on the original
> block device.  My tests on this has worked fine (although my tests have
> been for building two VGs that have striped volumes across the member
> disks, and then a VG that creates a mirrored LV of the striped volumes,
> so no multipathing is involved).  I'm wondering if we can compare notes
> to see if I'm doing something that makes it look like it's working -- I
> don't want to be quietly destroying my lvm data and not knowing it!!!
>
> I'm doing (roughly, block devices are a bit made-up):
>
> # prepare the physical volumes
> pvcreate /dev/sda
> pvcreate /dev/sdb
> pvcreate /dev/sdc
> pvcreate /dev/sdd
> pvcreate /dev/sde
>
> # Create volume groups to contain uniquely striped volumes
> vgcreate Stripe1VG /dev/sda /dev/sdb
> vgcreate Stripe2VG /dev/sdc /dev/sdd
>
> # Create the striped volumes
> lvcreate -i 2 -n Stripe1LV -L 1G Stripe1VG
> lvcreate -i 2 -n Stripe2LV -L 1G Stripe2VG
>
> # Make the striped volumes into PVs
> pvcreate /dev/Stripe1VG/Stripe1LV
> pvcreate /dev/Stripe2VG/Stripe2LV
>
> # Create the volume group for mirrored volumes
> vgcreate MirrorVG /dev/Stripe1VG/Stripe1LV /dev/Stripe2VG/Stripe2LV
> /dev/sde
> # (Had to use three PVs to have the mirror log in place)
>
> # Create the mirrored volume
> lvcreate -m 1 -n Mirror1LV -L 500M MirrorVG
>
> # Filesystem, test, etc.  this will be GFS eventually, but testing with
> ext3 for now.
> mke2fs -j -i16384 -v /dev/MirrorVG/Mirror1LV
> mkdir /mnt/mirror1lv
> mount /dev/MirrorVG/Mirror1LV /mnt/mirror1
>
>
>
> Is that about your procedure as well?  When does the lvm data get mangled?
>
> (Sorry if this is going off topic - but if this is solvable it might
> actually answer the original question...)
>
> -Ty!
>
>
>
>
> Matt P wrote:
> > This is basically the "messy" way I mentioned in my reply above. I
> > found if you pvcreate the  LV device, you end up mangling the lvm data
> > (this probably comes as little surprise) and it breaks down after
> > that. So, I ended up using losetup and an "image file", one for/on
> > each fabric. Then did pvcreate on each loop device, and made a new VG
> > containing both PVs and created the LV with mirroring.... It
> > worked.... I did no performance, stability, or failure testing...
>
>
> --
> -===========================-
>   Ty! Boyack
>   NREL Unix Network Manager
>   ty@nrel.colostate.edu
>   (970) 491-1186
> -===========================-
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>

[-- Attachment #2: Type: text/html, Size: 4720 bytes --]

RE: [linux-lvm] Mirror between different SAN fabrics

[mathias.herzog]

[-- Attachment #1: Type: text/plain, Size: 5450 bytes --]

Hey... first of all I wish you a happy new year, hope you had a nice
party...
Sorry for my absence from the list, I spent some nice days in the
mountains

I thought about the "stacked" approach. First I didn't even want to
think about a solution like this, but why not give it a try.
I will take quite the same configuration like Ty and try on a Cluster.

If it's stable and fast enough with some basic I/O testing then I will
try with two different fabrics and make tests with some real
applications.
You will definitely hear if I have success (or not)

Thanks for your help

Regards Mathias


> -----Original Message-----
> From: linux-lvm-bounces@redhat.com 
> [mailto:linux-lvm-bounces@redhat.com] On Behalf Of Matt P
> Sent: Freitag, 29 Dezember, 2006 00:25
> To: LVM general discussion and development
> Subject: Re: [linux-lvm] Mirror between different SAN fabrics
> 
> That's exactly the steps I used to test it out. The failure I 
> encountered was at the point of adding the StripeLVs to the 
> MirrorVG. I don't recall the error I got but I seem to recall 
> it being the same error as if the PV hadn't been pvcreated. 
> Although when I went through the process again just now, 
> without any errors... It's quite likely I mistyped or skipped 
> a step during my first attempt... 
> 
> I've gone through the whole process 4 times now from raw disk 
> to a mirrored LV without getting the error again.
> 
> It looks as though this could work and isn't nearly as crazy 
> as my initial configuration. I wonder now if anything wonky 
> will happen if we were to lose a disk or in the original post 
> if we lost one of the fabrics... 
> 
> 
> On 12/28/06, Ty! Boyack <ty@nrel.colostate.edu> wrote:
> 
> 	Actually, I'm quite surprised that this approach 
> mangles the lvm data.
> 	It seems that when you do a pvcreate on a block device, 
> LVM should (and
> 	I think, does) write the lvm metadata in a region of 
> that device, and 
> 	then never let anything touch that metadata.  This way, 
> if you do a 'dd
> 	if=/dev/zeros of=<PV-DEVICE>' it blanks out the device, 
> but the metadata
> 	is intact.
> 	
> 	So, if you do a 'pvcreate' on an LV, it should contain 
> a second copy of 
> 	the metadata, unique and independent from the first 
> copy on the original
> 	block device.  My tests on this has worked fine 
> (although my tests have
> 	been for building two VGs that have striped volumes 
> across the member 
> 	disks, and then a VG that creates a mirrored LV of the 
> striped volumes,
> 	so no multipathing is involved).  I'm wondering if we 
> can compare notes
> 	to see if I'm doing something that makes it look like 
> it's working -- I 
> 	don't want to be quietly destroying my lvm data and not 
> knowing it!!!
> 	
> 	I'm doing (roughly, block devices are a bit made-up):
> 	
> 	# prepare the physical volumes
> 	pvcreate /dev/sda
> 	pvcreate /dev/sdb 
> 	pvcreate /dev/sdc
> 	pvcreate /dev/sdd
> 	pvcreate /dev/sde
> 	
> 	# Create volume groups to contain uniquely striped volumes
> 	vgcreate Stripe1VG /dev/sda /dev/sdb
> 	vgcreate Stripe2VG /dev/sdc /dev/sdd
> 	
> 	# Create the striped volumes 
> 	lvcreate -i 2 -n Stripe1LV -L 1G Stripe1VG
> 	lvcreate -i 2 -n Stripe2LV -L 1G Stripe2VG
> 	
> 	# Make the striped volumes into PVs
> 	pvcreate /dev/Stripe1VG/Stripe1LV
> 	pvcreate /dev/Stripe2VG/Stripe2LV
> 	
> 	# Create the volume group for mirrored volumes 
> 	vgcreate MirrorVG /dev/Stripe1VG/Stripe1LV 
> /dev/Stripe2VG/Stripe2LV /dev/sde
> 	# (Had to use three PVs to have the mirror log in place)
> 	
> 	# Create the mirrored volume
> 	lvcreate -m 1 -n Mirror1LV -L 500M MirrorVG 
> 	
> 	# Filesystem, test, etc.  this will be GFS eventually, 
> but testing with
> 	ext3 for now.
> 	mke2fs -j -i16384 -v /dev/MirrorVG/Mirror1LV
> 	mkdir /mnt/mirror1lv
> 	mount /dev/MirrorVG/Mirror1LV /mnt/mirror1
> 	
> 	
> 	
> 	Is that about your procedure as well?  When does the 
> lvm data get mangled?
> 	
> 	(Sorry if this is going off topic - but if this is 
> solvable it might
> 	actually answer the original question...)
> 	
> 	-Ty!
> 	
> 	
> 	
> 	
> 	Matt P wrote:
> 	> This is basically the "messy" way I mentioned in my 
> reply above. I
> 	> found if you pvcreate the  LV device, you end up 
> mangling the lvm data
> 	> (this probably comes as little surprise) and it 
> breaks down after 
> 	> that. So, I ended up using losetup and an "image 
> file", one for/on
> 	> each fabric. Then did pvcreate on each loop device, 
> and made a new VG
> 	> containing both PVs and created the LV with mirroring.... It 
> 	> worked.... I did no performance, stability, or 
> failure testing...
> 	
> 	
> 	--
> 	-===========================-
> 	  Ty! Boyack
> 	  NREL Unix Network Manager
> 	  ty@nrel.colostate.edu 
> 	  (970) 491-1186
> 	-===========================-
> 	
> 	_______________________________________________
> 	linux-lvm mailing list
> 	linux-lvm@redhat.com
> 	https://www.redhat.com/mailman/listinfo/linux-lvm 
> <https://www.redhat.com/mailman/listinfo/linux-lvm> 
> 	read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
> 	
> 
> 
> 

Sicherheitshinweis:
Dieses E-Mail von PostFinance ist signiert. Weitere Informationen finden Sie unter: 
https://www.postfinance.ch/e-signature.
Geben Sie Ihre Sicherheitselemente niemals Dritten bekannt.

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4479 bytes --]

RE: [linux-lvm] Mirror between different SAN fabrics

[mathias.herzog]

[-- Attachment #1: Type: text/plain, Size: 6034 bytes --]

Hey

I use the same configuration as Ty and it seems that the lvm2 cluster
version doesn't like the stacked approach. When I'm creating the LV in
the last step, a locking error occurs:

---------------
#> lvcreate -m 1 -n MirrorLV -L 500M MirrorVG --corelog

    Logging initialised at Wed Dec  6 10:57:20 2006
    Set umask to 0077
    Loaded external locking library liblvm2clusterlock.so
    Finding volume group "MirrorVG"
    Archiving volume group "MirrorVG" metadata (seqno 1).
    Creating logical volume MirrorLV
    Creating logical volume MirrorLV_mimage_0
    Creating logical volume MirrorLV_mimage_1
    Creating volume group backup "/etc/lvm/backup/MirrorVG" (seqno 2).
  Error locking on node xxx: Internal lvm error, check syslog
  Error locking on node xxx: Internal lvm error, check syslog
  Failed to activate new LV.
    Wiping internal VG cache 
---------------

Syslog says:
lvm[3640]: Volume group for uuid not found: xxx

I can see the LV but since the state is NOT available I cannot access it

#> sudo vgdisplay -v MirrorVG
[...]
LV Name                /dev/MirrorVG/Mirror1LV
LV Status              NOT available
[...]


Mathias


> -----Original Message-----
> From: linux-lvm-bounces@redhat.com 
> [mailto:linux-lvm-bounces@redhat.com] On Behalf Of Matt P
> Sent: Freitag, 29 Dezember, 2006 00:25
> To: LVM general discussion and development
> Subject: Re: [linux-lvm] Mirror between different SAN fabrics
> 
> That's exactly the steps I used to test it out. The failure I 
> encountered was at the point of adding the StripeLVs to the 
> MirrorVG. I don't recall the error I got but I seem to recall 
> it being the same error as if the PV hadn't been pvcreated. 
> Although when I went through the process again just now, 
> without any errors... It's quite likely I mistyped or skipped 
> a step during my first attempt... 
> 
> I've gone through the whole process 4 times now from raw disk 
> to a mirrored LV without getting the error again.
> 
> It looks as though this could work and isn't nearly as crazy 
> as my initial configuration. I wonder now if anything wonky 
> will happen if we were to lose a disk or in the original post 
> if we lost one of the fabrics... 
> 
> 
> On 12/28/06, Ty! Boyack <ty@nrel.colostate.edu> wrote:
> 
> 	Actually, I'm quite surprised that this approach 
> mangles the lvm data.
> 	It seems that when you do a pvcreate on a block device, 
> LVM should (and
> 	I think, does) write the lvm metadata in a region of 
> that device, and 
> 	then never let anything touch that metadata.  This way, 
> if you do a 'dd
> 	if=/dev/zeros of=<PV-DEVICE>' it blanks out the device, 
> but the metadata
> 	is intact.
> 	
> 	So, if you do a 'pvcreate' on an LV, it should contain 
> a second copy of 
> 	the metadata, unique and independent from the first 
> copy on the original
> 	block device.  My tests on this has worked fine 
> (although my tests have
> 	been for building two VGs that have striped volumes 
> across the member 
> 	disks, and then a VG that creates a mirrored LV of the 
> striped volumes,
> 	so no multipathing is involved).  I'm wondering if we 
> can compare notes
> 	to see if I'm doing something that makes it look like 
> it's working -- I 
> 	don't want to be quietly destroying my lvm data and not 
> knowing it!!!
> 	
> 	I'm doing (roughly, block devices are a bit made-up):
> 	
> 	# prepare the physical volumes
> 	pvcreate /dev/sda
> 	pvcreate /dev/sdb 
> 	pvcreate /dev/sdc
> 	pvcreate /dev/sdd
> 	pvcreate /dev/sde
> 	
> 	# Create volume groups to contain uniquely striped volumes
> 	vgcreate Stripe1VG /dev/sda /dev/sdb
> 	vgcreate Stripe2VG /dev/sdc /dev/sdd
> 	
> 	# Create the striped volumes 
> 	lvcreate -i 2 -n Stripe1LV -L 1G Stripe1VG
> 	lvcreate -i 2 -n Stripe2LV -L 1G Stripe2VG
> 	
> 	# Make the striped volumes into PVs
> 	pvcreate /dev/Stripe1VG/Stripe1LV
> 	pvcreate /dev/Stripe2VG/Stripe2LV
> 	
> 	# Create the volume group for mirrored volumes 
> 	vgcreate MirrorVG /dev/Stripe1VG/Stripe1LV 
> /dev/Stripe2VG/Stripe2LV /dev/sde
> 	# (Had to use three PVs to have the mirror log in place)
> 	
> 	# Create the mirrored volume
> 	lvcreate -m 1 -n Mirror1LV -L 500M MirrorVG 
> 	
> 	# Filesystem, test, etc.  this will be GFS eventually, 
> but testing with
> 	ext3 for now.
> 	mke2fs -j -i16384 -v /dev/MirrorVG/Mirror1LV
> 	mkdir /mnt/mirror1lv
> 	mount /dev/MirrorVG/Mirror1LV /mnt/mirror1
> 	
> 	
> 	
> 	Is that about your procedure as well?  When does the 
> lvm data get mangled?
> 	
> 	(Sorry if this is going off topic - but if this is 
> solvable it might
> 	actually answer the original question...)
> 	
> 	-Ty!
> 	
> 	
> 	
> 	
> 	Matt P wrote:
> 	> This is basically the "messy" way I mentioned in my 
> reply above. I
> 	> found if you pvcreate the  LV device, you end up 
> mangling the lvm data
> 	> (this probably comes as little surprise) and it 
> breaks down after 
> 	> that. So, I ended up using losetup and an "image 
> file", one for/on
> 	> each fabric. Then did pvcreate on each loop device, 
> and made a new VG
> 	> containing both PVs and created the LV with mirroring.... It 
> 	> worked.... I did no performance, stability, or 
> failure testing...
> 	
> 	
> 	--
> 	-===========================-
> 	  Ty! Boyack
> 	  NREL Unix Network Manager
> 	  ty@nrel.colostate.edu 
> 	  (970) 491-1186
> 	-===========================-
> 	
> 	_______________________________________________
> 	linux-lvm mailing list
> 	linux-lvm@redhat.com
> 	https://www.redhat.com/mailman/listinfo/linux-lvm 
> <https://www.redhat.com/mailman/listinfo/linux-lvm> 
> 	read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
> 	
> 
> 
> 

Sicherheitshinweis:
Dieses E-Mail von PostFinance ist signiert. Weitere Informationen finden Sie unter: 
https://www.postfinance.ch/e-signature.
Geben Sie Ihre Sicherheitselemente niemals Dritten bekannt.

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4479 bytes --]

[linux-lvm] Filter the Swap Partition

[berthiaume_wayne]

Good morning all...

	I was wondering if anyone knew of a way of filtering out the
swap partition so that lvscan() won't look for it. This is a RHEL 3.0 U8
server with lvm-1.0.8-14.

Thanks and regards,
Wayne.