NIC and HBA based multipathing

NIC and HBA based multipathing

[Scott Moseman]

Device-mapper multipathing in RHEL 4
http://kbase.redhat.com/faq/FAQ_51_7170.shtm

I can (and have) setup a NIC based iSCSI SAN connection.  I can (and
have) setup an HBA based iSCSI SAN connection.  Now I'm trying to get
a machine to connect using multipath with both a NIC and an HBA.  What
are the prerequisites that I should have running before I attempt to
go through the above knowledge base procedure?

I put IPs on both the NIC and HBA on the SAN network.  The SAN is
showing both IPs as trying to access the volume in question.  iscsi-ls
only shows 1 device (/dev/sdb), and it's taking the path through the
NIC to reach the SAN.  I can verify this by watching the SAN volume to
see which of the IP addresses is sending traffic.

Should my HBA show up as another device (/dev/sdc) before I start
attempting to multipath?  Or is multipathd going to link both NIC and
HBA to the /dev/sdb device?

Thanks,
Scott

Re: NIC and HBA based multipathing

[Kevin Foote]

[-- Attachment #1.1: Type: text/plain, Size: 1660 bytes --]

I cant say that I have done that all mine are on an iscsi HBA only but, it
seems like it should work ..
you will need to have multipathd running, and you will have to setup your
/etc/multipath.conf file correctly for your device(s). Also this goes with
out saying each initiator will need the target defined
for their own access.

RHEL:
> rpm -qa | grep multipath
device-mapper-multipath-0.4.5-21.0.1.RHEL4
> chkconfig multipathd on

You will also note that your devices will vary /dev/sdX which is fine they
will
be "joined" if you will under the /dev/mpath/<whatever-you-called-dev> node.

Which is where you will access the device as well.


On 10/30/07, Scott Moseman <scmoseman@gmail.com> wrote:
>
> Device-mapper multipathing in RHEL 4
> http://kbase.redhat.com/faq/FAQ_51_7170.shtm
>
> I can (and have) setup a NIC based iSCSI SAN connection.  I can (and
> have) setup an HBA based iSCSI SAN connection.  Now I'm trying to get
> a machine to connect using multipath with both a NIC and an HBA.  What
> are the prerequisites that I should have running before I attempt to
> go through the above knowledge base procedure?
>
> I put IPs on both the NIC and HBA on the SAN network.  The SAN is
> showing both IPs as trying to access the volume in question.  iscsi-ls
> only shows 1 device (/dev/sdb), and it's taking the path through the
> NIC to reach the SAN.  I can verify this by watching the SAN volume to
> see which of the IP addresses is sending traffic.
>
> Should my HBA show up as another device (/dev/sdc) before I start
> attempting to multipath?  Or is multipathd going to link both NIC and
> HBA to the /dev/sdb device?
>
> Thanks,
> Scott
>

[-- Attachment #1.2: Type: text/html, Size: 2185 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: NIC and HBA based multipathing

[Scott Moseman]

Thanks for the follow-up.  I have each initiator setup with access to
the SAN.  The NIC is accessing /dev/sdb and the HBA is accessing
/dev/sdc.  I can mount either (or both) and access the data on the SAN
volume.

# rpm -qa | grep multipath
device-mapper-multipath-0.4.5-16.1.RHEL4

# /etc/init.d/multipathd status
multipathd (pid 15929) is running...

I'm following the instructions on Red Hat's KB to setup the
multipath.conf file.  The modprobes were successful, the service
started, but...

# multipath -v2
error calling out /sbin/scsi_id -g -u -s /block/sda

...with /dev/sda being my physical hard drive.  dmesg output says:

device-mapper: dm-multipath: error getting device
device-mapper: error adding target to table

# ls -l /dev/mapper/mpath*
ls: /dev/mapper/mpath*: No such file or directory

Can anyone point me in the right direction?  My config file is
currently the exact same as what appears in the Red Hat KB article.
Yes, I followed up with the next KB article and added "sda" to the
blacklist, but I still get the same message running multipath -v2.

Thanks,
Scott



On 10/30/07, Kevin Foote <kevin.foote@gmail.com> wrote:
>
> I cant say that I have done that all mine are on an iscsi HBA only but,
> it seems like it should work .. you will need to have multipathd running,
> and you will have to setup your /etc/multipath.conf file correctly for your
> device(s). Also this goes with out saying each initiator will need the target
> defined for their own access.
>
> RHEL:
> > rpm -qa | grep multipath
> device-mapper-multipath-0.4.5-21.0.1.RHEL4
> > chkconfig multipathd on
>
> You will also note that your devices will vary /dev/sdX which is fine they
> will be "joined" if you will under the /dev/mpath/<whatever-you-called-dev>
> node.  Which is where you will access the device as well.
>
>
> On 10/30/07, Scott Moseman <scmoseman@gmail.com> wrote:
> >
> > Device-mapper multipathing in RHEL 4
> > http://kbase.redhat.com/faq/FAQ_51_7170.shtm
> >
> > I can (and have) setup a NIC based iSCSI SAN connection.  I can (and
> > have) setup an HBA based iSCSI SAN connection.  Now I'm trying to get
> > a machine to connect using multipath with both a NIC and an HBA.  What
> > are the prerequisites that I should have running before I attempt to
> > go through the above knowledge base procedure?
> >
> > I put IPs on both the NIC and HBA on the SAN network.  The SAN is
> > showing both IPs as trying to access the volume in question.  iscsi-ls
> > only shows 1 device (/dev/sdb), and it's taking the path through the
> > NIC to reach the SAN.  I can verify this by watching the SAN volume to
> > see which of the IP addresses is sending traffic.
> >
> > Should my HBA show up as another device (/dev/sdc) before I start
> > attempting to multipath?  Or is multipathd going to link both NIC and
> > HBA to the /dev/sdb device?
> >
> > Thanks,
> > Scott
> >
>

Re: NIC and HBA based multipathing

[Mike Christie]

Scott Moseman wrote:
> Thanks for the follow-up.  I have each initiator setup with access to
> the SAN.  The NIC is accessing /dev/sdb and the HBA is accessing
> /dev/sdc.  I can mount either (or both) and access the data on the SAN
> volume.

Yeah, the iscsi bits looks fine. You will have a /dev/sdX for the nic 
and one of the HBA.

When you start multipath, multipath should then create one multipath 
device from those two scsi ones.

> 
> # rpm -qa | grep multipath
> device-mapper-multipath-0.4.5-16.1.RHEL4
> 
> # /etc/init.d/multipathd status
> multipathd (pid 15929) is running...
> 
> I'm following the instructions on Red Hat's KB to setup the
> multipath.conf file.  The modprobes were successful, the service
> started, but...
> 
> # multipath -v2
> error calling out /sbin/scsi_id -g -u -s /block/sda
> 
> ...with /dev/sda being my physical hard drive.  dmesg output says:
> 
> device-mapper: dm-multipath: error getting device
> device-mapper: error adding target to table

So are you asking about the errors from /dev/sda being accessed by 
multipath, or are you looking for help on adding the iscsi disks to a 
multipath device or both :)

Re: NIC and HBA based multipathing

[Scott Moseman]

On 11/1/07, Mike Christie <michaelc@cs.wisc.edu> wrote:
>
> Yeah, the iscsi bits looks fine. You will have a /dev/sdX for the nic
> and one of the HBA.
>
> When you start multipath, multipath should then create one multipath
> device from those two scsi ones.
>

That makes sense.  Thanks for the confirmation.

> >
> > I'm following the instructions on Red Hat's KB to setup the
> > multipath.conf file.  The modprobes were successful, the service
> > started, but...
> >
> > # multipath -v2
> > error calling out /sbin/scsi_id -g -u -s /block/sda
>
> So are you asking about the errors from /dev/sda being accessed by
> multipath, or are you looking for help on adding the iscsi disks to a
> multipath device or both :)
>

Both, I guess.  I'm following the Red Hat how-to and I'm getting that
error instead of getting a successful output.  If there's an answer for
why I'm getting the error, or an answer for how to fix my multipathd,
I'm open to either one at this point.  :)

I attempted to put sda in my blacklist (per the Red Hat instructions)
but still received the same error message about sda (the local disk).

Thanks,
Scott

Re: NIC and HBA based multipathing

[Mike Anderson]

Scott Moseman <scmoseman@gmail.com> wrote:
> On 11/1/07, Mike Christie <michaelc@cs.wisc.edu> wrote:
> > >
> > > I'm following the instructions on Red Hat's KB to setup the
> > > multipath.conf file.  The modprobes were successful, the service
> > > started, but...
> > >
> > > # multipath -v2
> > > error calling out /sbin/scsi_id -g -u -s /block/sda
> >
> > So are you asking about the errors from /dev/sda being accessed by
> > multipath, or are you looking for help on adding the iscsi disks to a
> > multipath device or both :)
> >
> 
> Both, I guess.  I'm following the Red Hat how-to and I'm getting that
> error instead of getting a successful output.  If there's an answer for
> why I'm getting the error, or an answer for how to fix my multipathd,
> I'm open to either one at this point.  :)
> 
> I attempted to put sda in my blacklist (per the Red Hat instructions)
> but still received the same error message about sda (the local disk).


If you trying to blacklist a device and it is still showing or when you
run multipath -v 4 the device is not listed as blacklisted then there is
most likely a problem in /etc/multipath.conf

Are you using "devnode_blacklist" as indicated in the instructions.

I believe this is incorrect and you should use something like:

blacklist {
        devnode "sda"
}

-andmike
--
Michael Anderson
andmike@linux.vnet.ibm.com

Re: NIC and HBA based multipathing

[Mike Anderson]

Mike Anderson <andmike@linux.vnet.ibm.com> wrote:
> I believe this is incorrect and you should use something like:
> 

Sorry replying to my own mail. I should have not used incorrect. Depending
on the version of tools one runs the keyword devnode_blacklist can
deprecated so one should try blacklist.

-andmike
--
Michael Anderson
andmike@linux.vnet.ibm.com

Re: NIC and HBA based multipathing

[Scott Moseman]

On Nov 1, 2007 3:57 PM, Mike Anderson <andmike@linux.vnet.ibm.com> wrote:
>
> Are you using "devnode_blacklist" as indicated in the instructions.
>
> I believe this is incorrect and you should use something like:
>
> blacklist {
>         devnode "sda"
> }
>

Thanks, Mike, I was just about to give this a try when I came upon
another problem (unrelated to MPIO, though).  The server was down, and
ever since bringing it back online I've been unable to get the HBA to
mount.  I disabled the iSCSI initiator for the NIC so that I'm only
dealing with the HBA, but I get this error:

# mount /dev/sdb1 /sanHBA
mount: /dev/sdb1 already mounted or /sanHBA busy

I did a fresh config using the iscli tool, and /dev/sdb1 was the
resulting partition that it found.

Thanks,
Scott

Re: NIC and HBA based multipathing

[Mike Anderson]

Scott Moseman <scmoseman@gmail.com> wrote:
> Thanks, Mike, I was just about to give this a try when I came upon
> another problem (unrelated to MPIO, though).  The server was down, and
> ever since bringing it back online I've been unable to get the HBA to
> mount.  I disabled the iSCSI initiator for the NIC so that I'm only
> dealing with the HBA, but I get this error:
> 
> # mount /dev/sdb1 /sanHBA
> mount: /dev/sdb1 already mounted or /sanHBA busy
> 
> I did a fresh config using the iscli tool, and /dev/sdb1 was the
> resulting partition that it found.
> 

It appears something else has claimed the device prior to your mount call. 
	- Is it lvm? Run pvs and see if the device was claimed. If that is
	  not what you want you may need to use the lvm.conf filters.

-andmike
--
Michael Anderson
andmike@linux.vnet.ibm.com

Re: NIC and HBA based multipathing

[Scott Moseman]

On Nov 6, 2007 4:20 PM, Mike Anderson <andmike@linux.vnet.ibm.com> wrote:
>
> > # mount /dev/sdb1 /sanHBA
> > mount: /dev/sdb1 already mounted or /sanHBA busy
>
> It appears something else has claimed the device prior to your mount call.
>         - Is it lvm? Run pvs and see if the device was claimed. If that is
>           not what you want you may need to use the lvm.conf filters.
>

Here's what's in dmesg...

scsi1 : qla4xxx
 QLogic iSCSI HBA Driver version: 5.01.00.08
  QLogic ISP4032 @ 0000:14:01.1 hdma+, host#=1, fw=03.00.01.24
  Vendor: EQLOGIC   Model: 100E-00           Rev: 2.3
  Type:   Direct-Access                      ANSI SCSI revision: 05
qla4xxx 0000:14:01.1: scsi(1:0:1:0): Enabled tagged queuing, queue depth 32.
SCSI device sdb: 8417280 512-byte hdwr sectors (4310 MB)
SCSI device sdb: drive cache: write through
 sdb: sdb1
Attached scsi disk sdb at scsi1, channel 0, id 1, lun 0

I can fdisk the /dev/sdb drive and it's the SAN partition alright.

Thanks,
Scott

Re: NIC and HBA based multipathing

[malahal]

Scott Moseman [scmoseman@gmail.com] wrote:
> qla4xxx 0000:14:01.1: scsi(1:0:1:0): Enabled tagged queuing, queue depth 32.
> SCSI device sdb: 8417280 512-byte hdwr sectors (4310 MB)
> SCSI device sdb: drive cache: write through
>  sdb: sdb1
> Attached scsi disk sdb at scsi1, channel 0, id 1, lun 0
> 
> I can fdisk the /dev/sdb drive and it's the SAN partition alright.

As Mike pointed out, quite likely that someone has claimed /dev/sdb1.
Maybe, lvm, evms, dmraid, ...etc. You will have to find out who claimed
it by running commands from lvm/evms/etc... OR if you don't have useful
data on it, simple zero few MB at the beginning and the end of the
partition and reboot. It may work. I am assuming that /sanHBA is not
used by anyone either (like CD'ing into it and then trying to mount).

Helpful commands: fuser -m /dev/sdb1; dd if=/dev/zero of=/dev/sdb1
count=1 bs=1M

Re: NIC and HBA based multipathing

[Mike Anderson]

Scott Moseman <scmoseman@gmail.com> wrote:
> On Nov 6, 2007 4:20 PM, Mike Anderson <andmike@linux.vnet.ibm.com> wrote:
> >
> > > # mount /dev/sdb1 /sanHBA
> > > mount: /dev/sdb1 already mounted or /sanHBA busy
> >
> > It appears something else has claimed the device prior to your mount call.
> >         - Is it lvm? Run pvs and see if the device was claimed. If that is
> >           not what you want you may need to use the lvm.conf filters.
> >
> 
> Here's what's in dmesg...
> 
> scsi1 : qla4xxx
>  QLogic iSCSI HBA Driver version: 5.01.00.08
>   QLogic ISP4032 @ 0000:14:01.1 hdma+, host#=1, fw=03.00.01.24
>   Vendor: EQLOGIC   Model: 100E-00           Rev: 2.3
>   Type:   Direct-Access                      ANSI SCSI revision: 05
> qla4xxx 0000:14:01.1: scsi(1:0:1:0): Enabled tagged queuing, queue depth 32.
> SCSI device sdb: 8417280 512-byte hdwr sectors (4310 MB)
> SCSI device sdb: drive cache: write through
>  sdb: sdb1
> Attached scsi disk sdb at scsi1, channel 0, id 1, lun 0
> 
> I can fdisk the /dev/sdb drive and it's the SAN partition alright.

Are there any dm messages in the dmesg output? 

Are you sure dm multipath did not grab the device(s) on boot?

What is the output of multipath -l?

-andmike
--
Michael Anderson
andmike@linux.vnet.ibm.com

Upgrade issues with v4.8

[Paul Cote]

I just recently upgraded to RHEL V4.5 and to Multipath tools V4.8 but
the paths are not being generated. I thought it may be a new
restrictions that would be managed via the multipath.conf file; but all
tweaks that I've made thus far have not made a difference. Falling back
to V4.7 works just fine but I would prefer to stay current with V4.8.

Any thoughts?

Dmesg does show some errors:

SELinux: initialized (dev 0:10, type nfs), uses genfs_contexts
device-mapper: dm-multipath: error getting device
device-mapper: error adding target to table
device-mapper: dm-multipath: error getting device
device-mapper: error adding target to table
device-mapper: dm-multipath: error getting device
device-mapper: error adding target to table
device-mapper: dm-multipath: error getting device
device-mapper: error adding target to table


With v4.7 ... The path list USE TO look like this;

multipathd> show paths
hcil     dev  dev_t  pri dm_st   chk_st  next_check        
3:0:0:3  sdaa 65:160 0   [active][ready] XXXXXX.... 75/120 
3:0:0:4  sdab 65:176 0   [active][ready] XXXXXX.... 74/120 
3:0:0:5  sdac 65:192 0   [active][ready] XXXXXX.... 72/120 
3:0:0:6  sdad 65:208 0   [active][ready] XXXXXX.... 74/120 
3:0:0:7  sdae 65:224 0   [active][ready] XXXXXX.... 74/120 
3:0:0:8  sdaf 65:240 0   [active][ready] XXXXXX.... 73/120 
3:0:0:9  sdag 66:0   0   [active][ready] XXXXXX.... 73/120 
3:0:0:10 sdah 66:16  0   [active][ready] XXXXXX.... 73/120 
4:0:0:0  sdai 66:32  0   [active][ready] XXXXXXXX.. 101/120
4:0:0:1  sdaj 66:48  0   [active][ready] XXXXXXXX.. 102/120
4:0:0:2  sdak 66:64  0   [active][ready] XXXXXXXX.. 101/120
4:0:0:3  sdal 66:80  0   [active][ready] XXXXXXXX.. 101/120
4:0:0:4  sdam 66:96  0   [active][ready] XXXXXXXX.. 101/120
4:0:0:5  sdan 66:112 0   [active][ready] XXXXXXXX.. 102/120
4:0:0:6  sdao 66:128 0   [active][ready] XXXXXX.... 75/120 
4:0:0:7  sdap 66:144 0   [active][ready] XXXXXX.... 74/120 
4:0:0:8  sdaq 66:160 0   [active][ready] XXXXXX.... 76/120 
4:0:0:9  sdar 66:176 0   [active][ready] XXXXXX.... 76/120 
4:0:0:10 sdas 66:192 0   [active][ready] XXXXXX.... 77/120 
0:0:0:0  sda  8:0    0   [undef] [ready] [orphan]          
1:0:0:0  sdb  8:16   0   [active][ready] XXXXXX.... 75/120 
1:0:0:1  sdc  8:32   0   [active][ready] XXXXXX.... 75/120 
1:0:0:2  sdd  8:48   0   [active][ready] XXXXXX.... 75/120 
1:0:0:3  sde  8:64   0   [active][ready] XXXXXX.... 75/120 
1:0:0:4  sdf  8:80   0   [active][ready] XXXXXX.... 74/120 
1:0:0:5  sdg  8:96   0   [active][ready] XXXXXX.... 72/120 
1:0:0:6  sdh  8:112  0   [active][ready] XXXXXX.... 74/120 
1:0:0:7  sdi  8:128  0   [active][ready] XXXXXX.... 74/120 
1:0:0:8  sdj  8:144  0   [active][ready] XXXXXX.... 73/120 
1:0:0:9  sdk  8:160  0   [active][ready] XXXXXX.... 73/120 
1:0:0:10 sdl  8:176  0   [active][ready] XXXXXX.... 73/120 
2:0:0:0  sdm  8:192  0   [active][ready] XXXXXXXX.. 100/120
2:0:0:1  sdn  8:208  0   [active][ready] XXXXXXXX.. 101/120
2:0:0:2  sdo  8:224  0   [active][ready] XXXXXXXX.. 100/120
2:0:0:3  sdp  8:240  0   [active][ready] XXXXXXXX.. 100/120
2:0:0:4  sdq  65:0   0   [active][ready] XXXXXXXX.. 100/120
2:0:0:5  sdr  65:16  0   [active][ready] XXXXXXXX.. 101/120
2:0:0:6  sds  65:32  0   [active][ready] XXXXXX.... 75/120 
2:0:0:7  sdt  65:48  0   [active][ready] XXXXXX.... 74/120 
2:0:0:8  sdu  65:64  0   [active][ready] XXXXXX.... 76/120 
2:0:0:9  sdv  65:80  0   [active][ready] XXXXXX.... 76/120 
2:0:0:10 sdw  65:96  0   [active][ready] XXXXXX.... 77/120 
3:0:0:0  sdx  65:112 0   [active][ready] XXXXXX.... 75/120 
3:0:0:1  sdy  65:128 0   [active][ready] XXXXXX.... 75/120 
3:0:0:2  sdz  65:144 0   [active][ready] XXXXXX.... 75/120 

But with v4.8; the path list is

multipathd> show paths
hcil    dev dev_t  pri dm_st  chk_st  next_check
#:#:#:# sdz 65:144 1   [undef][undef] [orphan]  
#:#:#:# sdy 65:128 1   [undef][undef] [orphan]  
#:#:#:# sdx 65:112 1   [undef][undef] [orphan]  
#:#:#:# sdw 65:96  1   [undef][undef] [orphan]  
#:#:#:# sdv 65:80  1   [undef][undef] [orphan]  
#:#:#:# sdu 65:64  1   [undef][undef] [orphan]  
#:#:#:# sdt 65:48  1   [undef][undef] [orphan]  
#:#:#:# sds 65:32  1   [undef][undef] [orphan]  
#:#:#:# sdr 65:16  1   [undef][undef] [orphan]  
#:#:#:# sdq 65:0   1   [undef][undef] [orphan]  
#:#:#:# sdp 8:240  1   [undef][undef] [orphan]  
#:#:#:# sdo 8:224  1   [undef][undef] [orphan]  
#:#:#:# sdn 8:208  1   [undef][undef] [orphan]  
#:#:#:# sdm 8:192  1   [undef][undef] [orphan]  
#:#:#:# sdl 8:176  1   [undef][undef] [orphan]  
#:#:#:# sdk 8:160  1   [undef][undef] [orphan]  
#:#:#:# sdj 8:144  1   [undef][undef] [orphan]  
#:#:#:# sdi 8:128  1   [undef][undef] [orphan]  
#:#:#:# sdh 8:112  1   [undef][undef] [orphan]  
#:#:#:# sdg 8:96   1   [undef][undef] [orphan]  
#:#:#:# sdf 8:80   1   [undef][undef] [orphan]  
#:#:#:# sde 8:64   1   [undef][undef] [orphan]  
#:#:#:# sdd 8:48   1   [undef][undef] [orphan]  
#:#:#:# sdc 8:32   1   [undef][undef] [orphan]  
#:#:#:# sdb 8:16   1   [undef][undef] [orphan]  
multipathd> 

Re: NIC and HBA based multipathing

[Scott Moseman]

On Nov 6, 2007 5:01 PM, Mike Anderson <andmike@linux.vnet.ibm.com> wrote:
>
> > > > # mount /dev/sdb1 /sanHBA
> > > > mount: /dev/sdb1 already mounted or /sanHBA busy
>
> Are there any dm messages in the dmesg output?
>
> Are you sure dm multipath did not grab the device(s) on boot?
>
> What is the output of multipath -l?
>

# multipath -l
30690a018f015191a6472441d1500f057
[size=4 GB][features="0"][hwhandler="0"]
\_ round-robin 0 [active]
 \_ 1:0:1:0 sdb 8:16 [active]

I guess that might explain it, huh?  However, I disabled the service...

# service multipathd status
multipathd is stopped

So I was assuming that it wasn't going to be firing up.  Obviously I'm
missing something.

Thanks!
Scott

Re: NIC and HBA based multipathing

[malahal]

On redhat, there is a script (can't recollect but rc.sysinit, I believe)
that creates this without multipathd.  You do need multipathd though for
renaming/creating/acting on multipath events there after.

Thanks, Malahal.

Scott Moseman [scmoseman@gmail.com] wrote:
> On Nov 6, 2007 5:01 PM, Mike Anderson <andmike@linux.vnet.ibm.com> wrote:
> >
> > > > > # mount /dev/sdb1 /sanHBA
> > > > > mount: /dev/sdb1 already mounted or /sanHBA busy
> >
> > Are there any dm messages in the dmesg output?
> >
> > Are you sure dm multipath did not grab the device(s) on boot?
> >
> > What is the output of multipath -l?
> >
> 
> # multipath -l
> 30690a018f015191a6472441d1500f057
> [size=4 GB][features="0"][hwhandler="0"]
> \_ round-robin 0 [active]
>  \_ 1:0:1:0 sdb 8:16 [active]
> 
> I guess that might explain it, huh?  However, I disabled the service...
> 
> # service multipathd status
> multipathd is stopped
> 
> So I was assuming that it wasn't going to be firing up.  Obviously I'm
> missing something.
> 
> Thanks!
> Scott
> 
> --
> dm-devel mailing list
> dm-devel@redhat.com
> https://www.redhat.com/mailman/listinfo/dm-devel

Re: NIC and HBA based multipathing

[Mike Anderson]

malahal@us.ibm.com <malahal@us.ibm.com> wrote:
> On redhat, there is a script (can't recollect but rc.sysinit, I believe)
> that creates this without multipathd.  You do need multipathd though for
> renaming/creating/acting on multipath events there after.
> 

Yes it is rc.sysinit that calls /sbin/multipath.static if nompath was not
specified on the cmdline and /etc/multipath.conf exists. One could return
the /etc/multipath.conf to its original state and blacklist everything
which would be why one would not see this behavior on a fresh no mpath
install.

-andmike
--
Michael Anderson
andmike@linux.vnet.ibm.com

Re: NIC and HBA based multipathing

[Scott Moseman]

[-- Attachment #1: Type: text/plain, Size: 1633 bytes --]

Ok, if multipath is grabbing the HBA device so I'm unable to mount it,
that's fine, I will go with that.  So I re-enabled the iSCSI initiator
so that the NIC device would become active and I can see if multipath
will grab both of them.  No luck, but I'm ahead of where I was
previously, so here's my status.

In dmesg, both of the devices are being found on boot...

HBA-

scsi1 : qla4xxx
 QLogic iSCSI HBA Driver version: 5.01.00.08
  QLogic ISP4032 @ 0000:14:01.1 hdma+, host#=1, fw=03.00.01.24
  Vendor: EQLOGIC   Model: 100E-00           Rev: 2.3
  Type:   Direct-Access                      ANSI SCSI revision: 05
qla4xxx 0000:14:01.1: scsi(1:0:1:0): Enabled tagged queuing, queue depth 32.
SCSI device sdb: 8417280 512-byte hdwr sectors (4310 MB)
SCSI device sdb: drive cache: write through
 sdb: sdb1
Attached scsi disk sdb at scsi1, channel 0, id 1, lun 0

NIC-

scsi3 : SFNet iSCSI driver
  Vendor: EQLOGIC   Model: 100E-00           Rev: 2.3
  Type:   Direct-Access                      ANSI SCSI revision: 05
SCSI device sdc: 8417280 512-byte hdwr sectors (4310 MB)
SCSI device sdc: drive cache: write through
 sdc: sdc1
Attached scsi disk sdc at scsi3, channel 0, id 0, lun 0

# multipath -l
30690a018f015191a6472441d1500f057
[size=4 GB][features="0"][hwhandler="0"]
\_ round-robin 0 [active]
 \_ 1:0:1:0     sdb 8:16 [active][ready]

I'm attaching the output of "multipath -v4" since it's lengthy.
There's some references to sdc (although [faulty]) that I don't
understand, but maybe it will help troubleshoot?

My multipath.conf is setup in accordance with
http://kbase.redhat.com/faq/FAQ_51_7170.shtm .

Thanks,
Scott

[-- Attachment #2: multipath.txt --]
[-- Type: text/plain, Size: 1980 bytes --]


# multipath -v4
load path identifiers cache
#
# all paths in cache :
#
  #:#:#:#  hda 3:0  [faulty] //
30690a018f015191a6472441d1500f057  1:0:1:0 sdb 8:16  [active][ready] EQLOGIC /
  3:0:0:0 sdc 8:32  [faulty] EQLOGIC /100E-00         /2.3 
===== path info hda (mask 0x1f) =====
bus = 2
dev_t = 3:0
size = 8388604
sda blacklisted
===== path info sdb (mask 0x1f) =====
bus = 1
dev_t = 8:16
size = 8417280
vendor = EQLOGIC 
product = 100E-00         
rev = 2.3 
h:b:t:l = 1:0:1:0
tgt_node_name = 
serial = 0690A018F015191A6472441D1500F057
path checker = readsector0 (internal default)
state = 2
getprio = /bin/true (internal default)
prio = 0
uid = 30690a018f015191a6472441d1500f057 (cache)
===== path info sdc (mask 0x1f) =====
bus = 1
dev_t = 8:32
size = 8417280
vendor = EQLOGIC 
product = 100E-00         
rev = 2.3 
h:b:t:l = 3:0:0:0
tgt_node_name = 
#
# all paths :
#
  #:#:#:#  hda 3:0  [faulty] //
30690a018f015191a6472441d1500f057  1:0:1:0 sdb 8:16  [active][ready] EQLOGIC /
  3:0:0:0 sdc 8:32  [faulty] EQLOGIC /100E-00         /2.3 
params = 0 0 1 1 round-robin 0 1 1 8:16 100 
status = 1 0 0 1 1 A 0 1 0 8:16 A 0 
*word = 0, len = 1
*word = 0, len = 1
*word = 1, len = 1
*word = 1, len = 1
*word = round-robin, len = 11
*word = 0, len = 1
*word = 1, len = 1
*word = 1, len = 1
*word = 8:16, len = 4
===== path info sdb (mask 0x8) =====
prio = 0
uid = 30690a018f015191a6472441d1500f057 (cache)
*word = 1, len = 1
*word = 0, len = 1
*word = 1, len = 1
*word = A, len = 1
*word = 1, len = 1
*word = 0, len = 1
*word = A, len = 1
*word = 0, len = 1
pgpolicy = failover (config file default)
selector = round-robin 0 (internal default)
features = 0 (internal default)
hwhandler = 0 (internal default)
rr_weight = 1 (internal default)
no_path_retry = NONE (internal default)
pg_timeout = NONE (internal default)
0 8417280 multipath 0 0 1 1 round-robin 0 1 1 8:16 100
set ACT_NOTHING: map unchanged

[-- Attachment #3: Type: text/plain, Size: 0 bytes --]

Re: NIC and HBA based multipathing

[Scott Moseman]

On Nov 7, 2007 9:07 AM, Scott Moseman <scmoseman@gmail.com> wrote:
>
> Ok, if multipath is grabbing the HBA device so I'm unable to mount it,
> that's fine, I will go with that.  So I re-enabled the iSCSI initiator
> so that the NIC device would become active and I can see if multipath
> will grab both of them.  No luck, but I'm ahead of where I was
> previously, so here's my status.
>
> In dmesg, both of the devices are being found on boot...
>
> HBA-
>
> scsi1 : qla4xxx
>  QLogic iSCSI HBA Driver version: 5.01.00.08
>   QLogic ISP4032 @ 0000:14:01.1 hdma+, host#=1, fw=03.00.01.24
>   Vendor: EQLOGIC   Model: 100E-00           Rev: 2.3
>   Type:   Direct-Access                      ANSI SCSI revision: 05
> qla4xxx 0000:14:01.1: scsi(1:0:1:0): Enabled tagged queuing, queue depth 32.
> SCSI device sdb: 8417280 512-byte hdwr sectors (4310 MB)
> SCSI device sdb: drive cache: write through
>  sdb: sdb1
> Attached scsi disk sdb at scsi1, channel 0, id 1, lun 0
>
> NIC-
>
> scsi3 : SFNet iSCSI driver
>   Vendor: EQLOGIC   Model: 100E-00           Rev: 2.3
>   Type:   Direct-Access                      ANSI SCSI revision: 05
> SCSI device sdc: 8417280 512-byte hdwr sectors (4310 MB)
> SCSI device sdc: drive cache: write through
>  sdc: sdc1
> Attached scsi disk sdc at scsi3, channel 0, id 0, lun 0
>
> # multipath -l
> 30690a018f015191a6472441d1500f057
> [size=4 GB][features="0"][hwhandler="0"]
> \_ round-robin 0 [active]
>  \_ 1:0:1:0     sdb 8:16 [active][ready]
>
> I'm attaching the output of "multipath -v4" since it's lengthy.
> There's some references to sdc (although [faulty]) that I don't
> understand, but maybe it will help troubleshoot?
>
> My multipath.conf is setup in accordance with
> http://kbase.redhat.com/faq/FAQ_51_7170.shtm .
>
> Thanks,
> Scott
>

Maybe no one wants to open my attachment, here's an abbreviated
version of the "multipath -v" output...

# multipath -v4
load path identifiers cache
#
# all paths in cache :
#
  #:#:#:#  hda 3:0  [faulty] //
30690a018f015191a6472441d1500f057  1:0:1:0 sdb 8:16  [active][ready] EQLOGIC /
  3:0:0:0 sdc 8:32  [faulty] EQLOGIC /100E-00         /2.3

Per my previous email, the "sdc" device is the iSCSI connection
through the NIC.  The HBA connection, "sdb", is working fine.

Thanks,
Scott

Re: NIC and HBA based multipathing

[Mike Anderson]

Scott Moseman <scmoseman@gmail.com> wrote:
> Maybe no one wants to open my attachment, here's an abbreviated
> version of the "multipath -v" output...
> 
> # multipath -v4
> load path identifiers cache
> #
> # all paths in cache :
> #
>   #:#:#:#  hda 3:0  [faulty] //
> 30690a018f015191a6472441d1500f057  1:0:1:0 sdb 8:16  [active][ready] EQLOGIC /
>   3:0:0:0 sdc 8:32  [faulty] EQLOGIC /100E-00         /2.3
> 
> Per my previous email, the "sdc" device is the iSCSI connection
> through the NIC.  The HBA connection, "sdb", is working fine.

Based on the previous multipath -v4 output you provided (snippet shown
below) it appears there is a problem accessing the device node of sdc as
it complete sysfs_pathinfo and then appears to fail when it tries to open
the device node.
...
h:b:t:l = 3:0:0:0
tgt_node_name =

Is /dev/sdc ok? Can you do a dd read of the device?

-andmike
--
Michael Anderson
andmike@linux.vnet.ibm.com

Re: NIC and HBA based multipathing

[Scott Moseman]

On Nov 12, 2007 2:18 PM, Mike Anderson <andmike@linux.vnet.ibm.com> wrote:
> >
> > 30690a018f015191a6472441d1500f057  1:0:1:0 sdb 8:16  [active][ready] EQLOGIC /
> >   3:0:0:0 sdc 8:32  [faulty] EQLOGIC /100E-00         /2.3
> >
>
> Based on the previous multipath -v4 output you provided (snippet shown
> below) it appears there is a problem accessing the device node of sdc as
> it complete sysfs_pathinfo and then appears to fail when it tries to open
> the device node.
> ...
> h:b:t:l = 3:0:0:0
> tgt_node_name =
>
> Is /dev/sdc ok? Can you do a dd read of the device?
>

Interestingly enough, I have a usable sdc1, but there's no sdc...

# dd if=/dev/sdc
dd: opening `/dev/sdc': No such file or directory

# ls -l /dev/sd* | grep -v sda
brw-------  1 root root 8, 16 Nov  7 14:43 /dev/sdb
brw-rw----  1 root disk 8, 17 Nov  7 14:43 /dev/sdb1
brw-rw----  1 root disk 8, 33 Nov  7 14:45 /dev/sdc1

I can actually mount /dev/sdc1 and it's the partition on the SAN.  But
I cannot dd or fdisk /dev/sdc itself.

This is a first time I've seen this kind of disk behavior before!

Thanks,
Scott

Re: NIC and HBA based multipathing

[Scott Moseman]

On Nov 12, 2007 2:34 PM, Scott Moseman <scmoseman@gmail.com> wrote:
>
> # ls -l /dev/sd* | grep -v sda
> brw-------  1 root root 8, 16 Nov  7 14:43 /dev/sdb
> brw-rw----  1 root disk 8, 17 Nov  7 14:43 /dev/sdb1
> brw-rw----  1 root disk 8, 33 Nov  7 14:45 /dev/sdc1
>

I can use 'mknod sdc b 8 32' to generate a new /dev/sdc, which I can
fdisk and the data looks good, but once I reboot the /dev/sdc device
is once again removed (or not re-created, whichever the case may be).

Thanks,
Scott

Re: NIC and HBA based multipathing

[Scott Moseman]

On Nov 13, 2007 10:52 AM, Scott Moseman <scmoseman@gmail.com> wrote:
>
> > # ls -l /dev/sd* | grep -v sda
> > brw-------  1 root root 8, 16 Nov  7 14:43 /dev/sdb
> > brw-rw----  1 root disk 8, 17 Nov  7 14:43 /dev/sdb1
> > brw-rw----  1 root disk 8, 33 Nov  7 14:45 /dev/sdc1
>
> I can use 'mknod sdc b 8 32' to generate a new /dev/sdc, which I can
> fdisk and the data looks good, but once I reboot the /dev/sdc device
> is once again removed (or not re-created, whichever the case may be).
>

Does anyone know why the /dev/sdc device might not get created on
boot?  Is this something that the iSCSI initiator should handle?  The
operating system?

Thanks,
Scott

checker msg is "emc_clariion_checker: Path not correctly configured for failover"

[Paul Cote]

Hi

I've recently added an EMC CX3-80 ... And provisioned volumes to my
RHEL4-4 running with multipath tools v4.7.

I've added the recommend settings by EMC for the CX to the
multipath.conf file (below) but I am seeing the following errors;

devices {
## Device attributes requirements for EMC Symmetrix
## are part of the default definitions and do not require separate
## definition.
## Device attributes for EMC CLARiiON
device {
vendor "DGC "
product "*"
path_grouping_policy group_by_prio
getuid_callout "/sbin/scsi_id -g -u -s /block/%n"
prio_callout "/sbin/mpath_prio_emc /dev/%n"
path_checker emc_clariion
path_selector "round-robin 0"
features "1 queue_if_no_path"
no_path_retry 300
hardware_handler "1 emc"
failback immediate
}
}

error calling out /sbin/scsi_id -g -u -s /block/sda
sdd: checker msg is "emc_clariion_checker: Path not correctly configured
for failover"
sde: checker msg is "emc_clariion_checker: Path not correctly configured
for failover"
sdf: checker msg is "emc_clariion_checker: Path not correctly configured
for failover"
sdg: checker msg is "emc_clariion_checker: Path not correctly configured
for failover"
sdh: checker msg is "emc_clariion_checker: Path not correctly configured
for failover"
sdi: checker msg is "emc_clariion_checker: Path not correctly configured
for failover"
sdj: checker msg is "emc_clariion_checker: Path not correctly configured
for failover"
sdk: checker msg is "emc_clariion_checker: Path not correctly configured
for failover"
sdl: checker msg is "emc_clariion_checker: Path not correctly configured
for failover"
sdm: checker msg is "emc_clariion_checker: Path not correctly configured
for failover"
sdn: checker msg is "emc_clariion_checker: Path not correctly configured
for failover"
sdo: checker msg is "emc_clariion_checker: Path not correctly configured
for failover"
sdd: checker msg is "emc_clariion_checker: Path not correctly configured
for failover"
sdg: checker msg is "emc_clariion_checker: Path not correctly configured
for failover"
sdj: checker msg is "emc_clariion_checker: Path not correctly configured
for failover"
sdm: checker msg is "emc_clariion_checker: Path not correctly configured
for failover"
sde: checker msg is "emc_clariion_checker: Path not correctly configured
for failover"
sdh: checker msg is "emc_clariion_checker: Path not correctly configured
for failover"
sdk: checker msg is "emc_clariion_checker: Path not correctly configured
for failover"
sdn: checker msg is "emc_clariion_checker: Path not correctly configured
for failover"
sdf: checker msg is "emc_clariion_checker: Path not correctly configured
for failover"
sdi: checker msg is "emc_clariion_checker: Path not correctly configured
for failover"
sdl: checker msg is "emc_clariion_checker: Path not correctly configured
for failover"
sdo: checker msg is "emc_clariion_checker: Path not correctly configured
for failover"
[root@ldm1066 log]# multipath -F

I'm thinking this is a configuration issue on the CX ... 

Any comments / suggestions?

Thanks,
Paul


Messages show the following;

[root@ldm1066 log]# tail -50 ./messages
Nov 15 11:00:50 localhost last message repeated 2 times
Nov 15 11:05:02 localhost kernel: device-mapper: dm-emc: long trespass
command will be send
Nov 15 11:05:02 localhost kernel: device-mapper: dm-emc: honor
reservation bit will not be set (default)
Nov 15 11:05:02 localhost kernel: device-mapper: dm-emc: emc_pg_init:
sending switch-over command
Nov 15 11:05:02 localhost kernel: Device sdl not ready.
Nov 15 11:05:02 localhost kernel: end_request: I/O error, dev sdl,
sector 72
Nov 15 11:05:02 localhost kernel: device-mapper: dm-multipath: Failing
path 8:176.
Nov 15 11:38:16 localhost sshd(pam_unix)[7939]: session opened for user
root by (uid=0)
Nov 15 11:43:39 localhost kernel: device-mapper: dm-emc: long trespass
command will be send
Nov 15 11:43:39 localhost kernel: device-mapper: dm-emc: honor
reservation bit will not be set (default)
Nov 15 11:43:40 localhost kernel: device-mapper: dm-emc: long trespass
command will be send
Nov 15 11:43:40 localhost kernel: device-mapper: dm-emc: honor
reservation bit will not be set (default)
Nov 15 11:43:40 localhost kernel: device-mapper: dm-emc: long trespass
command will be send
Nov 15 11:43:40 localhost kernel: device-mapper: dm-emc: honor
reservation bit will not be set (default)
Nov 15 11:43:40 localhost kernel: device-mapper: dm-emc: emc_pg_init:
sending switch-over command
Nov 15 11:43:40 localhost kernel: device-mapper: dm-emc: emc_pg_init:
sending switch-over command
Nov 15 11:43:40 localhost kernel: Device sdl not ready.
Nov 15 11:43:40 localhost kernel: end_request: I/O error, dev sdl,
sector 72
Nov 15 11:43:40 localhost kernel: device-mapper: dm-multipath: Failing
path 8:176.
Nov 15 11:43:40 localhost kernel: Device sdj not ready.
Nov 15 11:43:40 localhost kernel: end_request: I/O error, dev sdj,
sector 72
Nov 15 11:43:40 localhost kernel: device-mapper: dm-multipath: Failing
path 8:144.
Nov 15 11:43:41 localhost kernel: device-mapper: dm-emc: emc_pg_init:
sending switch-over command
Nov 15 11:43:41 localhost kernel: Device sdk not ready.
Nov 15 11:43:41 localhost kernel: end_request: I/O error, dev sdk,
sector 72
Nov 15 11:43:41 localhost kernel: device-mapper: dm-multipath: Failing
path 8:160.
Nov 15 12:19:34 localhost kernel: device-mapper: dm-emc: long trespass
command will be send
Nov 15 12:19:34 localhost kernel: device-mapper: dm-emc: honor
reservation bit will not be set (default)
Nov 15 12:19:34 localhost kernel: device-mapper: dm-emc: long trespass
command will be send
Nov 15 12:19:34 localhost kernel: device-mapper: dm-emc: honor
reservation bit will not be set (default)
Nov 15 12:19:34 localhost kernel: device-mapper: dm-emc: long trespass
command will be send
Nov 15 12:19:34 localhost kernel: device-mapper: dm-emc: honor
reservation bit will not be set (default)
Nov 15 12:19:34 localhost kernel: device-mapper: dm-emc: emc_pg_init:
sending switch-over command
Nov 15 12:19:34 localhost kernel: Device sdl not ready.
Nov 15 12:19:34 localhost kernel: end_request: I/O error, dev sdl,
sector 72
Nov 15 12:19:34 localhost kernel: device-mapper: dm-multipath: Failing
path 8:176.
Nov 15 12:19:34 localhost kernel: Device sdl not ready.
Nov 15 12:19:34 localhost kernel: end_request: I/O error, dev sdl,
sector 209715192
Nov 15 12:19:35 localhost kernel: device-mapper: dm-emc: emc_pg_init:
sending switch-over command
Nov 15 12:19:35 localhost kernel: Device sdj not ready.
Nov 15 12:19:35 localhost kernel: end_request: I/O error, dev sdj,
sector 72
Nov 15 12:19:35 localhost kernel: device-mapper: dm-multipath: Failing
path 8:144.
Nov 15 12:19:35 localhost kernel: Device sdj not ready.
Nov 15 12:19:35 localhost kernel: end_request: I/O error, dev sdj,
sector 209715192
Nov 15 12:19:35 localhost kernel: device-mapper: dm-emc: emc_pg_init:
sending switch-over command
Nov 15 12:19:35 localhost kernel: Device sdk not ready.
Nov 15 12:19:35 localhost kernel: end_request: I/O error, dev sdk,
sector 72
Nov 15 12:19:35 localhost kernel: device-mapper: dm-multipath: Failing
path 8:160.
Nov 15 12:19:35 localhost kernel: Device sdk not ready.
Nov 15 12:19:35 localhost kernel: end_request: I/O error, dev sdk,
sector 209715192
[root@ldm1066 log]# time



devices {
## Device attributes requirements for EMC Symmetrix
## are part of the default definitions and do not require separate
## definition.
## Device attributes for EMC CLARiiON
device {
vendor "DGC "
product "*"
path_grouping_policy group_by_prio
getuid_callout "/sbin/scsi_id -g -u -s /block/%n"
prio_callout "/sbin/mpath_prio_emc /dev/%n"
path_checker emc_clariion
path_selector "round-robin 0"
features "1 queue_if_no_path"
no_path_retry 300
hardware_handler "1 emc"
failback immediate
}
}

Re: NIC and HBA based multipathing

[Mike Christie]

Scott Moseman wrote:
> On Nov 13, 2007 10:52 AM, Scott Moseman <scmoseman@gmail.com> wrote:
>>> # ls -l /dev/sd* | grep -v sda
>>> brw-------  1 root root 8, 16 Nov  7 14:43 /dev/sdb
>>> brw-rw----  1 root disk 8, 17 Nov  7 14:43 /dev/sdb1
>>> brw-rw----  1 root disk 8, 33 Nov  7 14:45 /dev/sdc1
>> I can use 'mknod sdc b 8 32' to generate a new /dev/sdc, which I can
>> fdisk and the data looks good, but once I reboot the /dev/sdc device
>> is once again removed (or not re-created, whichever the case may be).
>>
> 
> Does anyone know why the /dev/sdc device might not get created on
> boot?  Is this something that the iSCSI initiator should handle?  The

The iscsi initiator is not responsible for creating device nodes. If you 
see this:

SCSI device sdb: 8417280 512-byte hdwr sectors (4310 MB)
SCSI device sdb: drive cache: write through
  sdb: sdb1
Attached scsi disk sdb at scsi1, channel 0, id 1, lun 0

then the initiator and scsi layer have done everything they can.

> operating system?

It seems like a udev problem, but I do not know for sure.

Re: NIC and HBA based multipathing

[Mike Anderson]

Mike Christie <michaelc@cs.wisc.edu> wrote:
> Scott Moseman wrote:
> >On Nov 13, 2007 10:52 AM, Scott Moseman <scmoseman@gmail.com> wrote:
> >>># ls -l /dev/sd* | grep -v sda
> >>>brw-------  1 root root 8, 16 Nov  7 14:43 /dev/sdb
> >>>brw-rw----  1 root disk 8, 17 Nov  7 14:43 /dev/sdb1
> >>>brw-rw----  1 root disk 8, 33 Nov  7 14:45 /dev/sdc1
> >>I can use 'mknod sdc b 8 32' to generate a new /dev/sdc, which I can
> >>fdisk and the data looks good, but once I reboot the /dev/sdc device
> >>is once again removed (or not re-created, whichever the case may be).
> >>
> >
> >Does anyone know why the /dev/sdc device might not get created on
> >boot?  Is this something that the iSCSI initiator should handle?  The
> 
> The iscsi initiator is not responsible for creating device nodes. If you 
> see this:
> 
> SCSI device sdb: 8417280 512-byte hdwr sectors (4310 MB)
> SCSI device sdb: drive cache: write through
>  sdb: sdb1
> Attached scsi disk sdb at scsi1, channel 0, id 1, lun 0
> 
> then the initiator and scsi layer have done everything they can.
> 
> >operating system?
> 
> It seems like a udev problem, but I do not know for sure.

I would agree that we should look to see what udev is doing when it
receives the events.

Since I believe this is failing on a RHEL4 sytem we cannot use
udevmonitor. If you unload the iSCSI module, edit /etc/udev/udev.conf to
set udev_log="yes", and then load the module again you should get some info
in /var/log/message on what udev is doing.

-andmike
--
Michael Anderson
andmike@linux.vnet.ibm.com

Re: NIC and HBA based multipathing

[Scott Moseman]

On Nov 15, 2007 1:16 PM, Mike Anderson <andmike@linux.vnet.ibm.com> wrote:
>
> > >> I can use 'mknod sdc b 8 32' to generate a new /dev/sdc, which I can
> > >> fdisk and the data looks good, but once I reboot the /dev/sdc device
> > >> is once again removed (or not re-created, whichever the case may be).
>
> I would agree that we should look to see what udev is doing when it
> receives the events.
>
> Since I believe this is failing on a RHEL4 sytem we cannot use
> udevmonitor. If you unload the iSCSI module, edit /etc/udev/udev.conf to
> set udev_log="yes", and then load the module again you should get some info
> in /var/log/message on what udev is doing.
>

Hey Mike,

Here's the guts of what happened...

# cat /var/log/messages | grep udev
Nov 13 15:39:01 ems1 udevsend[1185]: starting udevd daemon
Nov 13 15:39:02 ems1 start_udev: Starting udev:  succeeded
Nov 13 15:39:06 ems1 udevsend[1691]: starting udevd daemon
Nov 15 20:13:58 ems1 udev[30499]: creating device node '/dev/iscsictl'
Nov 15 20:13:58 ems1 udev[30509]: creating device node '/dev/iscsictl'
Nov 15 20:14:03 ems1 udev[30551]: configured rule in
'/etc/udev/rules.d/20-local.rules' at line 2 applied, 'sdc' is ignored
Nov 15 20:14:03 ems1 udev[30552]: configured rule in
'/etc/udev/rules.d/51-by-id.rules' at line 4 applied, added symlink
'disk/by-id/scsi-%c-part%n'
Nov 15 20:14:03 ems1 udev[30552]: creating device node '/dev/sdc1'

# cat /etc/udev/rules.d/20-local.rules
BUS="scsi", SYSFS{vendor}="EQLOGIC", SYSFS{model}="100E-00",
PROGRAM="/sbin/scsi_id -g -u -s /block/%k",
RESULT="30690a018f015191a6472441d1500f057", NAME=""

Maybe I shot myself in the foot.  While I don't necessarily recall
very putting that entry in there, I'm not going to exclude myself as
the cause of the problem (seeing how many steps I jumped through when
playing with all of this HBA, NIC and MPIO stuff).

Should I comment out that line, reboot (or restart iscsi) and see what happens?

Thanks!
Scott

Re: NIC and HBA based multipathing

[Scott Moseman]

On Nov 15, 2007 2:26 PM, Scott Moseman <scmoseman@gmail.com> wrote:
>
> # cat /etc/udev/rules.d/20-local.rules
> BUS="scsi", SYSFS{vendor}="EQLOGIC", SYSFS{model}="100E-00",
> PROGRAM="/sbin/scsi_id -g -u -s /block/%k",
> RESULT="30690a018f015191a6472441d1500f057", NAME=""
>
> Should I comment out that line, reboot (or restart iscsi) and see what happens?
>

It's a lab box, so I went ahead and gave it a try... success!

# multipath -l
30690a018f015191a6472441d1500f057
[size=4 GB][features="0"][hwhandler="0"]
\_ round-robin 0 [enabled]
 \_ 1:0:1:0     sdb 8:16 [active][ready]
\_ round-robin 0 [enabled]
 \_ 7:0:0:0     sdc 8:32 [active][ready]

I guess I can start to play around with it now to see if it works!

Thanks for all your help so far, folks.
Scott