* [NVMeF]: Multipathing setup for NVMeF
@ 2017-04-12 9:28 Ankur Srivastava
2017-04-12 15:00 ` Keith Busch
0 siblings, 1 reply; 3+ messages in thread
From: Ankur Srivastava @ 2017-04-12 9:28 UTC (permalink / raw)
Hi All,
I am working on NVMe over fabric and want to experiment the
Multipathing support for the same.
Setup Info:
RHEL 7.2 with Kernel 4.9.3
[root at localhost ~]# nvme list
Node SN Model
Namespace Usage Format
FW Rev
---------------- --------------------
---------------------------------------- ---------
-------------------------- ---------------- --------
/dev/nvme0n1 30501b622ed15184 Linux
10 268.44 GB / 268.44 GB 512 B + 0 B 4.9.3
/dev/nvme1n1 ef730272d9be107c Linux
10 268.44 GB / 268.44 GB 512 B + 0 B 4.9.3
[root at localhost ~]# ps ax | grep multipath
1272 ? SLl 0:00 /sbin/multipathd
I have connected my Initiator to both the ports of Ethernet
Adapter(Target) to get 2 IO Paths, from the above data "/dev/nvme0n1"
is path 1 and "/dev/nvme1n1" is path 2 for the same namespace.
Note: I am using Null Block device on the Target Side.
But still the multipath is showing an error ie no path to Host for All
the NVMe Drives mapped on the Initiator. Does multipathd supports NVMe
over Fabric ??
Or what I am missing from configuration side ??
Thanks in advance!!
BR~
Ankur
^ permalink raw reply [flat|nested] 3+ messages in thread
* [NVMeF]: Multipathing setup for NVMeF
2017-04-12 9:28 [NVMeF]: Multipathing setup for NVMeF Ankur Srivastava
@ 2017-04-12 15:00 ` Keith Busch
2017-04-18 5:58 ` Ankur Srivastava
0 siblings, 1 reply; 3+ messages in thread
From: Keith Busch @ 2017-04-12 15:00 UTC (permalink / raw)
On Wed, Apr 12, 2017@02:58:05PM +0530, Ankur Srivastava wrote:
> I have connected my Initiator to both the ports of Ethernet
> Adapter(Target) to get 2 IO Paths, from the above data "/dev/nvme0n1"
> is path 1 and "/dev/nvme1n1" is path 2 for the same namespace.
>
> Note: I am using Null Block device on the Target Side.
>
> But still the multipath is showing an error ie no path to Host for All
> the NVMe Drives mapped on the Initiator. Does multipathd supports NVMe
> over Fabric ??
> Or what I am missing from configuration side ??
>
> Thanks in advance!!
I think you need a udev rule to export the wwn like
KERNEL=="nvme*[0-9]n*[0-9]", ENV{DEVTYPE}=="disk", ATTRS{wwid}=="?*", ENV{ID_WWN}="$attr{wwid}"
And multipathd conf needs to use that attribute for uid for NVME,
uid_attribute = "ID_WWN".
These should be there by default if you've very recent versions (within
the last 6 weeks) of multipath-tools and systemd installed.
If your kernel has CONFIG_SCSI_DH set, you'll also need this recent
kernel commit:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit?id=857de6e00778738dc3d61f75acbac35bdc48e533
^ permalink raw reply [flat|nested] 3+ messages in thread
* [NVMeF]: Multipathing setup for NVMeF
2017-04-12 15:00 ` Keith Busch
@ 2017-04-18 5:58 ` Ankur Srivastava
0 siblings, 0 replies; 3+ messages in thread
From: Ankur Srivastava @ 2017-04-18 5:58 UTC (permalink / raw)
Thanks for the useful pointers.
One more query, I have inserted the udev rule for nvme in the file
"/etc/udev/rules.d/10-knem.rules" the rule as "SUBSYSTEM=="nvme",
KERNEL=="nvme*[0-9]n*[0-9]", ENV{DEVTYPE}=="disk", ATTRS{wwid}=="?*",
ENV{ID_WWN}="$attr{10}" here I suspect ID_WWN could be nsid, but not
sure but I am getting a very absurd wwid in the file
"/sys/class/nvme-fabrics/ctl/nvme0/nvme0n1/wwid" the wwid I am getting
is "nvme.0000-6161353331646636333736376632363000-4c696e75780000000000000000000000000000000000000000000000000000000000000000000000-0000000a"
which could be linux generated, So my queries are...
1) Where I can get the correct wwid for nvme over fabrics, is it the
nsid or anything else.
2) Where I could get the below information from NVMeF perspective to
populate the "/etc/multipath.conf" file
devices {
# Enable multipathing for NVMeF Disks.
device {
vendor "????"
product "????"
path_grouping_policy "????"
prio ????
features "????"
no_path_retry ????
path_checker ????
rr_min_io ????
failback ????
fast_io_fail_tmo ????
dev_loss_tmo ????
uid_attribute = "ID_WWN" ????
}
}
Please correct me if I am doing something wrong or missing any step in
configuring multipath feature for NVMeF.
Thanks in advance!
Best Regards
Ankur
On Wed, Apr 12, 2017@8:30 PM, Keith Busch <keith.busch@intel.com> wrote:
> On Wed, Apr 12, 2017@02:58:05PM +0530, Ankur Srivastava wrote:
>> I have connected my Initiator to both the ports of Ethernet
>> Adapter(Target) to get 2 IO Paths, from the above data "/dev/nvme0n1"
>> is path 1 and "/dev/nvme1n1" is path 2 for the same namespace.
>>
>> Note: I am using Null Block device on the Target Side.
>>
>> But still the multipath is showing an error ie no path to Host for All
>> the NVMe Drives mapped on the Initiator. Does multipathd supports NVMe
>> over Fabric ??
>> Or what I am missing from configuration side ??
>>
>> Thanks in advance!!
>
> I think you need a udev rule to export the wwn like
>
> KERNEL=="nvme*[0-9]n*[0-9]", ENV{DEVTYPE}=="disk", ATTRS{wwid}=="?*", ENV{ID_WWN}="$attr{wwid}"
>
> And multipathd conf needs to use that attribute for uid for NVME,
> uid_attribute = "ID_WWN".
>
> These should be there by default if you've very recent versions (within
> the last 6 weeks) of multipath-tools and systemd installed.
>
> If your kernel has CONFIG_SCSI_DH set, you'll also need this recent
> kernel commit:
>
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit?id=857de6e00778738dc3d61f75acbac35bdc48e533
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2017-04-18 5:58 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-04-12 9:28 [NVMeF]: Multipathing setup for NVMeF Ankur Srivastava
2017-04-12 15:00 ` Keith Busch
2017-04-18 5:58 ` Ankur Srivastava
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.