All of lore.kernel.org
 help / color / mirror / Atom feed
* Re: [SPDK] 回复: NVMeF Target Error NVMe controller
@ 2017-02-28 14:38 Ankur Srivastava
  0 siblings, 0 replies; 5+ messages in thread
From: Ankur Srivastava @ 2017-02-28 14:38 UTC (permalink / raw)
  To: spdk

[-- Attachment #1: Type: text/plain, Size: 4775 bytes --]

Thanks Thomas for the work around I have made the subsystem2 as Virtual
Mode, but still the error persists.
Error I am getting is related to the subsystem1.

Modified conf.in
-------------------------------
# Direct controller
[Subsystem1]
  NQN nqn.2016-06.io.spdk:cnode1
  Core 0
  Mode Direct
  Listen RDMA 192.168.25.3:4420
  Host nqn.2016-06.io.spdk:init
  NVMe 0000:06:00.0

# Multiple subsystems are allowed.
# Virtual controller
[Subsystem2]
  NQN nqn.2016-06.io.spdk:cnode2
  Core 0
  Mode Virtual
  Listen RDMA 192.168.25.31:4420
  Host nqn.2016-06.io.spdk:init
  SN SPDK00000000000001
  Namespace Malloc0
  Namespace Malloc1



Error:
--------------------------
[root(a)localhost spdk-master]# app/nvmf_tgt/nvmf_tgt -c etc/spdk/nvmf.conf.in
Starting Intel(R) DPDK initialization ...
[ DPDK EAL parameters: nvmf -c 1 -n 4 -m 2048 --master-lcore=0
--file-prefix=rte0 --proc-type=auto ]
EAL: Detected 12 lcore(s)
EAL: Auto-detected process type: PRIMARY
EAL: Probing VFIO support...
done.
Occupied cpu core mask is 0x1
Occupied cpu socket mask is 0x1
Ioat Copy Engine Offload Enabled
Total cores available: 1
Reactor started on core 0 on socket 0
*** RDMA Transport Init ***
*** RDMA Transport Init ***
allocated subsystem nqn.2014-08.org.nvmexpress.discovery on lcore 0 on
socket 0
allocated subsystem nqn.2016-06.io.spdk:cnode1 on lcore 0 on socket 0
*** NVMf Target Listening on 192.168.25.3 port 4420 ***
conf.c: 565:spdk_nvmf_parse_subsystem: ***ERROR*** Could not find NVMe
controller for Subsystem1
nvmf_tgt.c: 336:spdk_nvmf_startup: ***ERROR*** spdk_nvmf_parse_conf() failed


Can you please tell me more about "domain:bus:func:dev", from where I can
extract this data ?


Thanks and Regards
Ankur


On Tue, Feb 28, 2017 at 7:56 PM, thomas.zhang.8288 <
thomas.zhang.8288(a)aliyun.com> wrote:

> Hi Ankur,
>
> You configuration missing nvme 'domain:bus:func:dev' info in the second
> nvmf subsystem 'subsystem2'. If you choose mode 'Direct' , you have to
> specific 'nvme' bus info in the  nvmf subsystem, otherwise you can choose
> 'virtual mode'.
>
> Best Reagrds
>
> Zhihao
>
>
> ------------------------------------------------------------------
> 发件人:Ankur Srivastava <asrivastava014(a)gmail.com>
> 发送时间:2017年2月28日(星期二) 21:57
> 收件人:spdk <spdk(a)lists.01.org>
> 主 题:[SPDK] NVMeF Target Error NVMe controller
>
> Hi All,
>
>
> Here are some notes that I took on my system when starting the nvmf target and faced an issue with NVMe Controller discovery
>
> 1. Mellanox NICs detected successfully.
>
> 2. Installed neccesary modules
>
>    a. modprobe nvme
>
>    b. modprobe nvmet
>
>    c. modprobe nvme-rdma
>
>    d. modprobe nvmet-rdma
>
> 3. Assign NICs ip addresses
>
>    a. ip addr add 192.168.25.3/24 dev nic0
>    b. ip addr add 192.168.25.31/24 dev nic1
>
>
> 4. My configuration file has the following Subsystems
>
> # Direct controller
> [Subsystem1]
>   NQN nqn.2016-06.io.spdk:cnode1
>   Core 0
>   Mode Direct
>   Listen RDMA 192.168.25.3:4420
>   Host nqn.2016-06.io.spdk:init
>   NVMe 0000:06:00.0
>
> # Multiple subsystems are allowed.
> [Subsystem2]
>   NQN nqn.2016-06.io.spdk:cnode2
>   Core 0
>   Mode Direct
>   Listen RDMA 192.168.25.31:4420
>   Host nqn.2016-06.io.spdk:init
>   SN SPDK00000000000001
>   Namespace Malloc0
>   Namespace Malloc1
>
>
> 5. Start the nvmf target
>
> cd ~/spdk/app/nvmf_tgt
> sudo ./nvmf_tgt -c nvmf.conf.in
>
>
> Getting the following error
>
> [root(a)localhost spdk-master]# app/nvmf_tgt/nvmf_tgt -c etc/spdk/nvmf.conf.in
> Starting Intel(R) DPDK initialization ...
> [ DPDK EAL parameters: nvmf -c 1 -n 4 -m 2048 --master-lcore=0 --file-prefix=rte0 --proc-type=auto ]
> EAL: Detected 12 lcore(s)
> EAL: Auto-detected process type: PRIMARY
> EAL: Probing VFIO support...
> done.
> Occupied cpu core mask is 0x1
> Occupied cpu socket mask is 0x1
> Ioat Copy Engine Offload Enabled
> Total cores available: 1
> Reactor started on core 0 on socket 0
> *** RDMA Transport Init ***
> *** RDMA Transport Init ***
> allocated subsystem nqn.2014-08.org.nvmexpress.discovery on lcore 0 on socket 0
> allocated subsystem nqn.2016-06.io.spdk:cnode1 on lcore 0 on socket 0
> *** NVMf Target Listening on 192.168.25.3 port 4420 ***
> conf.c: 565:spdk_nvmf_parse_subsystem: ***ERROR*** Could not find NVMe controller for Subsystem1
> nvmf_tgt.c: 336:spdk_nvmf_startup: ***ERROR*** spdk_nvmf_parse_conf() failed
>
>
>
>
>
>
> Best Reagrds
>
> Ankur
>
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 8046 bytes --]

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [SPDK] 回复: NVMeF Target Error NVMe controller
@ 2017-02-28 17:31 Wenhua Liu
  0 siblings, 0 replies; 5+ messages in thread
From: Wenhua Liu @ 2017-02-28 17:31 UTC (permalink / raw)
  To: spdk

[-- Attachment #1: Type: text/plain, Size: 10134 bytes --]

Ankur,


As you don't have NVMe drive in your system, you better use virtual mode as you've done for Subsystem2. Here is one of my configurations. I don't have NVMe drive either. I commented out Subsystem1 and had 4 null_blk devices attached to Subystem2.


# Direct controller
#[Subsystem1]
#  NQN nqn.2016-06.io.spdk:cnode1
#  Core 0
#  Mode Direct
#  Listen RDMA 12.12.12.100:4420
#  Host nqn.2016-06.io.spdk:init
#  NVMe 0000:00:00.0

# Multiple subsystems are allowed.
# Virtual controller
[Subsystem2]
  NQN nqn.2016-06.io.spdk:cnode2
  Core 0
  Mode Virtual
  Listen RDMA 12.12.12.100:4420
#  Host nqn.2016-06.io.spdk:init
  SN SPDK00000000000001
  Namespace Malloc0
  Namespace Malloc1
  Namespace Malloc2
  Namespace Malloc3



Thanks,
-Wenhua

________________________________
From: SPDK <spdk-bounces(a)lists.01.org> on behalf of Ankur Srivastava <asrivastava014(a)gmail.com>
Sent: Tuesday, February 28, 2017 8:07 AM
To: thomas.zhang.8288; Storage Performance Development Kit
Subject: Re: [SPDK] 回复: NVMeF Target Error NVMe controller

Hi Thomas,

No I am not having any nvme ssd drive in my system, I was using 'nullb0' device installed through "modprobe null_blk nr_devices=1" for my NVMe over Fabric bench-marking.

Few Question:
1)  Since it is nullb0 device it can't be there in lspci, does this mean this is an issue?
2)  Also

 # Direct controller
[Subsystem1]
  NQN nqn.2016-06.io.spdk:cnode1
  Core 0
  Mode Direct
  Listen RDMA 192.168.25.3:4420<https://urldefense.proofpoint.com/v2/url?u=http-3A__192.168.25.3-3A4420_&d=DwMFaQ&c=uilaK90D4TOVoH58JNXRgQ&r=-xGUcQt5607zrir8TikGRQ&m=sU5wucoK2A7klMDFjRHsVMYEoE3ByzYiqikNuhJ5hhA&s=Yxqg9cF9ZT2wYl_wBcwt8D5tLpAjKLbgfNlwiCIM2r4&e=>
  Host nqn.2016-06.io.spdk:init
  NVMe 0000:06:00.0


In the string "NVMe 0000:06:00.0" above, Do  these numbers('domain:bus:func.dev') belong to the NVMe SSD drive attached via pci ??


Thanks in advance


Regards
Ankur

On Tue, Feb 28, 2017 at 8:30 PM, thomas.zhang.8288 <thomas.zhang.8288(a)aliyun.com<mailto:thomas.zhang.8288(a)aliyun.com>> wrote:
Hi Ankur,

Do you have any NVME SSD drive(s) in the system? You can assign one of nvme ssd bus info in the 'subsystem1',  also you can dump you nvme ssd drive info and extract 'domain:bus:func.dev' info in your system.
    'lspci | grep Non-volatile'

Mode 'direct' means front nvmf directly control backend nvme ssd drive.

Thomas

------------------------------------------------------------------
发件人:Ankur Srivastava <asrivastava014(a)gmail.com<mailto:asrivastava014(a)gmail.com>>
发送时间:2017年2月28日(星期二) 22:38
收件人:Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>; thomas.zhang.8288 <thomas.zhang.8288(a)aliyun.com<mailto:thomas.zhang.8288(a)aliyun.com>>
主 题:Re: [SPDK] 回复: NVMeF Target Error NVMe controller

Thanks Thomas for the work around I have made the subsystem2 as Virtual Mode, but still the error persists.
Error I am getting is related to the subsystem1.

Modified conf.in<https://urldefense.proofpoint.com/v2/url?u=http-3A__conf.in&d=DwMFaQ&c=uilaK90D4TOVoH58JNXRgQ&r=-xGUcQt5607zrir8TikGRQ&m=sU5wucoK2A7klMDFjRHsVMYEoE3ByzYiqikNuhJ5hhA&s=mE_iV6OWqufPt1gVWIDZ47PQPYadYGQbilD3mtZ-ydA&e=>
-------------------------------
# Direct controller
[Subsystem1]
  NQN nqn.2016-06.io.spdk:cnode1
  Core 0
  Mode Direct
  Listen RDMA 192.168.25.3:4420<https://urldefense.proofpoint.com/v2/url?u=http-3A__192.168.25.3-3A4420&d=DwMFaQ&c=uilaK90D4TOVoH58JNXRgQ&r=-xGUcQt5607zrir8TikGRQ&m=sU5wucoK2A7klMDFjRHsVMYEoE3ByzYiqikNuhJ5hhA&s=ETxRygXeng_FLARohCpi81iUwFpX4vmofznwm4sD9mE&e=>
  Host nqn.2016-06.io.spdk:init
  NVMe 0000:06:00.0

# Multiple subsystems are allowed.
# Virtual controller
[Subsystem2]
  NQN nqn.2016-06.io.spdk:cnode2
  Core 0
  Mode Virtual
  Listen RDMA 192.168.25.31:4420<https://urldefense.proofpoint.com/v2/url?u=http-3A__192.168.25.31-3A4420&d=DwMFaQ&c=uilaK90D4TOVoH58JNXRgQ&r=-xGUcQt5607zrir8TikGRQ&m=sU5wucoK2A7klMDFjRHsVMYEoE3ByzYiqikNuhJ5hhA&s=Ib7Uc8QEhlTdDDBU0Bo30ULLQV3N17ueG48qNOsUDjU&e=>
  Host nqn.2016-06.io.spdk:init
  SN SPDK00000000000001
  Namespace Malloc0
  Namespace Malloc1



Error:
--------------------------
[root(a)localhost spdk-master]# app/nvmf_tgt/nvmf_tgt -c etc/spdk/nvmf.conf.in<https://urldefense.proofpoint.com/v2/url?u=http-3A__nvmf.conf.in&d=DwMFaQ&c=uilaK90D4TOVoH58JNXRgQ&r=-xGUcQt5607zrir8TikGRQ&m=sU5wucoK2A7klMDFjRHsVMYEoE3ByzYiqikNuhJ5hhA&s=nfDP9aAx3wB20AJo0AnM3Xkc7FySeVBgkPWmRda9w6g&e=>
Starting Intel(R) DPDK initialization ...
[ DPDK EAL parameters: nvmf -c 1 -n 4 -m 2048 --master-lcore=0 --file-prefix=rte0 --proc-type=auto ]
EAL: Detected 12 lcore(s)
EAL: Auto-detected process type: PRIMARY
EAL: Probing VFIO support...
done.
Occupied cpu core mask is 0x1
Occupied cpu socket mask is 0x1
Ioat Copy Engine Offload Enabled
Total cores available: 1
Reactor started on core 0 on socket 0
*** RDMA Transport Init ***
*** RDMA Transport Init ***
allocated subsystem nqn.2014-08.org.nvmexpress.discovery on lcore 0 on socket 0
allocated subsystem nqn.2016-06.io.spdk:cnode1 on lcore 0 on socket 0
*** NVMf Target Listening on 192.168.25.3 port 4420 ***
conf.c: 565:spdk_nvmf_parse_subsystem: ***ERROR*** Could not find NVMe controller for Subsystem1
nvmf_tgt.c: 336:spdk_nvmf_startup: ***ERROR*** spdk_nvmf_parse_conf() failed


Can you please tell me more about "domain:bus:func:dev", from where I can extract this data ?


Thanks and Regards
Ankur


On Tue, Feb 28, 2017 at 7:56 PM, thomas.zhang.8288 <thomas.zhang.8288(a)aliyun.com<mailto:thomas.zhang.8288(a)aliyun.com>> wrote:
Hi Ankur,

You configuration missing nvme 'domain:bus:func:dev' info in the second nvmf subsystem 'subsystem2'. If you choose mode 'Direct' , you have to specific 'nvme' bus info in the  nvmf subsystem, otherwise you can choose 'virtual mode'.


Best Reagrds

Zhihao

------------------------------------------------------------------
发件人:Ankur Srivastava <asrivastava014(a)gmail.com<mailto:asrivastava014(a)gmail.com>>
发送时间:2017年2月28日(星期二) 21:57
收件人:spdk <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
主 题:[SPDK] NVMeF Target Error NVMe controller


Hi All,


Here are some notes that I took on my system when starting the nvmf target and faced an issue with NVMe Controller discovery


1. Mellanox NICs detected successfully.

2. Installed neccesary modules

   a. modprobe nvme

   b. modprobe nvmet

   c. modprobe nvme-rdma

   d. modprobe nvmet-rdma

3. Assign NICs ip addresses

   a. ip addr add 192.168.25.3/24<https://urldefense.proofpoint.com/v2/url?u=http-3A__192.168.25.3_24&d=DwMFaQ&c=uilaK90D4TOVoH58JNXRgQ&r=-xGUcQt5607zrir8TikGRQ&m=sU5wucoK2A7klMDFjRHsVMYEoE3ByzYiqikNuhJ5hhA&s=9XX8Apg91RE5Zcuc-rztMWhArkTDDQL8ngowR3hzDk8&e=> dev nic0
   b. ip addr add 192.168.25.31/24<https://urldefense.proofpoint.com/v2/url?u=http-3A__192.168.25.31_24&d=DwMFaQ&c=uilaK90D4TOVoH58JNXRgQ&r=-xGUcQt5607zrir8TikGRQ&m=sU5wucoK2A7klMDFjRHsVMYEoE3ByzYiqikNuhJ5hhA&s=9SikGNwgY0KnEvx2cIcyOL29gMfDwbAp1MFE4MTlJBE&e=> dev nic1


4. My configuration file has the following Subsystems

# Direct controller
[Subsystem1]
  NQN nqn.2016-06.io.spdk:cnode1
  Core 0
  Mode Direct
  Listen RDMA 192.168.25.3:4420<https://urldefense.proofpoint.com/v2/url?u=http-3A__192.168.25.3-3A4420&d=DwMFaQ&c=uilaK90D4TOVoH58JNXRgQ&r=-xGUcQt5607zrir8TikGRQ&m=sU5wucoK2A7klMDFjRHsVMYEoE3ByzYiqikNuhJ5hhA&s=ETxRygXeng_FLARohCpi81iUwFpX4vmofznwm4sD9mE&e=>
  Host nqn.2016-06.io.spdk:init
  NVMe 0000:06:00.0

# Multiple subsystems are allowed.
[Subsystem2]
  NQN nqn.2016-06.io.spdk:cnode2
  Core 0
  Mode Direct
  Listen RDMA 192.168.25.31:4420<https://urldefense.proofpoint.com/v2/url?u=http-3A__192.168.25.31-3A4420&d=DwMFaQ&c=uilaK90D4TOVoH58JNXRgQ&r=-xGUcQt5607zrir8TikGRQ&m=sU5wucoK2A7klMDFjRHsVMYEoE3ByzYiqikNuhJ5hhA&s=Ib7Uc8QEhlTdDDBU0Bo30ULLQV3N17ueG48qNOsUDjU&e=>
  Host nqn.2016-06.io.spdk:init
  SN SPDK00000000000001
  Namespace Malloc0
  Namespace Malloc1


5. Start the nvmf target

cd ~/spdk/app/nvmf_tgt
sudo ./nvmf_tgt -c nvmf.conf.in<https://urldefense.proofpoint.com/v2/url?u=http-3A__nvmf.conf.in&d=DwMFaQ&c=uilaK90D4TOVoH58JNXRgQ&r=-xGUcQt5607zrir8TikGRQ&m=sU5wucoK2A7klMDFjRHsVMYEoE3ByzYiqikNuhJ5hhA&s=nfDP9aAx3wB20AJo0AnM3Xkc7FySeVBgkPWmRda9w6g&e=>


Getting the following error

[root(a)localhost spdk-master]# app/nvmf_tgt/nvmf_tgt -c etc/spdk/nvmf.conf.in<https://urldefense.proofpoint.com/v2/url?u=http-3A__nvmf.conf.in&d=DwMFaQ&c=uilaK90D4TOVoH58JNXRgQ&r=-xGUcQt5607zrir8TikGRQ&m=sU5wucoK2A7klMDFjRHsVMYEoE3ByzYiqikNuhJ5hhA&s=nfDP9aAx3wB20AJo0AnM3Xkc7FySeVBgkPWmRda9w6g&e=>
Starting Intel(R) DPDK initialization ...
[ DPDK EAL parameters: nvmf -c 1 -n 4 -m 2048 --master-lcore=0 --file-prefix=rte0 --proc-type=auto ]
EAL: Detected 12 lcore(s)
EAL: Auto-detected process type: PRIMARY
EAL: Probing VFIO support...
done.
Occupied cpu core mask is 0x1
Occupied cpu socket mask is 0x1
Ioat Copy Engine Offload Enabled
Total cores available: 1
Reactor started on core 0 on socket 0
*** RDMA Transport Init ***
*** RDMA Transport Init ***
allocated subsystem nqn.2014-08.org.nvmexpress.discovery on lcore 0 on socket 0
allocated subsystem nqn.2016-06.io.spdk:cnode1 on lcore 0 on socket 0
*** NVMf Target Listening on 192.168.25.3 port 4420 ***
conf.c: 565:spdk_nvmf_parse_subsystem: ***ERROR*** Could not find NVMe controller for Subsystem1
nvmf_tgt.c: 336:spdk_nvmf_startup: ***ERROR*** spdk_nvmf_parse_conf() failed







Best Reagrds

Ankur


_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk<https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.01.org_mailman_listinfo_spdk&d=DwMFaQ&c=uilaK90D4TOVoH58JNXRgQ&r=-xGUcQt5607zrir8TikGRQ&m=sU5wucoK2A7klMDFjRHsVMYEoE3ByzYiqikNuhJ5hhA&s=-xtR83W-WMaJ4Ejz6ZQezu3ijkEK_CVwbZZtcWE7eKM&e=>





[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 18131 bytes --]

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [SPDK] 回复: NVMeF Target Error NVMe controller
@ 2017-02-28 16:48 Walker, Benjamin
  0 siblings, 0 replies; 5+ messages in thread
From: Walker, Benjamin @ 2017-02-28 16:48 UTC (permalink / raw)
  To: spdk

[-- Attachment #1: Type: text/plain, Size: 1547 bytes --]

Direct mode exports a physical NVMe device, specified by
domain:bus:device.function, over NVMe-oF. Virtual mode can export any
type of device, specified using the 'Namespace' directive, over NVME-
oF.


On Tue, 2017-02-28 at 20:08 +0530, Ankur Srivastava wrote:
> Thanks Thomas for the work around I have made the subsystem2 as
> Virtual Mode, but still the error persists.  
> Error I am getting is related to the subsystem1.
> 
> Modified conf.in
> -------------------------------
> # Direct controller
> [Subsystem1]
>   NQN nqn.2016-06.io.spdk:cnode1
>   Core 0
>   Mode Direct
>   Listen RDMA 192.168.25.3:4420
>   Host nqn.2016-06.io.spdk:init
>   NVMe 0000:06:00.0

Here's the error you are getting:

conf.c: 565:spdk_nvmf_parse_subsystem: ***ERROR*** Could not find NVMe
controller for Subsystem1

If you look at Subsystem1, it's in Direct mode and trying to export the
NVMe device at 0000:06:00.0. You explained earlier that you do not have
an NVMe device attached, so the error is telling you that it can't find
the device at that location. Instead, either delete Subsystem1 if you
don't need it, or make Subsystem1 a virtual subsystem and add
namespaces too it like you did for Subsystem2.

> 
> # Multiple subsystems are allowed.
> # Virtual controller
> [Subsystem2]
>   NQN nqn.2016-06.io.spdk:cnode2
>   Core 0
>   Mode Virtual
>   Listen RDMA 192.168.25.31:4420
>   Host nqn.2016-06.io.spdk:init
>   SN SPDK00000000000001
>   Namespace Malloc0
>   Namespace Malloc1
> 

[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 3274 bytes --]

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [SPDK] 回复: NVMeF Target Error NVMe controller
@ 2017-02-28 16:07 Ankur Srivastava
  0 siblings, 0 replies; 5+ messages in thread
From: Ankur Srivastava @ 2017-02-28 16:07 UTC (permalink / raw)
  To: spdk

[-- Attachment #1: Type: text/plain, Size: 6420 bytes --]

Hi Thomas,

No I am not having any nvme ssd drive in my system, I was using 'nullb0'
device installed through "modprobe null_blk nr_devices=1" for my NVMe over
Fabric bench-marking.

Few Question:
1)  Since it is nullb0 device it can't be there in lspci, does this mean
this is an issue?
2)  Also

 # Direct controller
[Subsystem1]
  NQN nqn.2016-06.io.spdk:cnode1
  Core 0
  Mode Direct
  Listen RDMA 192.168.25.3:4420
  Host nqn.2016-06.io.spdk:init
  NVMe 0000:06:00.0


In the string "NVMe 0000:06:00.0" above, Do  these numbers(
'domain:bus:func.dev') belong to the NVMe SSD drive attached via pci ??


Thanks in advance


Regards
Ankur

On Tue, Feb 28, 2017 at 8:30 PM, thomas.zhang.8288 <
thomas.zhang.8288(a)aliyun.com> wrote:

> Hi Ankur,
>
> Do you have any NVME SSD drive(s) in the system? You can assign one of
> nvme ssd bus info in the 'subsystem1',  also you can dump you nvme ssd
> drive info and extract 'domain:bus:func.dev' info in your system.
>     'lspci | grep Non-volatile'
>
> Mode 'direct' means front nvmf directly control backend nvme ssd drive.
>
> Thomas
>
> ------------------------------------------------------------------
> 发件人:Ankur Srivastava <asrivastava014(a)gmail.com>
> 发送时间:2017年2月28日(星期二) 22:38
> 收件人:Storage Performance Development Kit <spdk(a)lists.01.org>;
> thomas.zhang.8288 <thomas.zhang.8288(a)aliyun.com>
> 主 题:Re: [SPDK] 回复: NVMeF Target Error NVMe controller
>
> Thanks Thomas for the work around I have made the subsystem2 as Virtual
> Mode, but still the error persists.
> Error I am getting is related to the subsystem1.
>
> Modified conf.in
> -------------------------------
> # Direct controller
> [Subsystem1]
>   NQN nqn.2016-06.io.spdk:cnode1
>   Core 0
>   Mode Direct
>   Listen RDMA 192.168.25.3:4420
>   Host nqn.2016-06.io.spdk:init
>   NVMe 0000:06:00.0
>
> # Multiple subsystems are allowed.
> # Virtual controller
> [Subsystem2]
>   NQN nqn.2016-06.io.spdk:cnode2
>   Core 0
>   Mode Virtual
>   Listen RDMA 192.168.25.31:4420
>   Host nqn.2016-06.io.spdk:init
>   SN SPDK00000000000001
>   Namespace Malloc0
>   Namespace Malloc1
>
>
>
> Error:
> --------------------------
> [root(a)localhost spdk-master]# app/nvmf_tgt/nvmf_tgt -c etc/spdk/
> nvmf.conf.in
> Starting Intel(R) DPDK initialization ...
> [ DPDK EAL parameters: nvmf -c 1 -n 4 -m 2048 --master-lcore=0
> --file-prefix=rte0 --proc-type=auto ]
> EAL: Detected 12 lcore(s)
> EAL: Auto-detected process type: PRIMARY
> EAL: Probing VFIO support...
> done.
> Occupied cpu core mask is 0x1
> Occupied cpu socket mask is 0x1
> Ioat Copy Engine Offload Enabled
> Total cores available: 1
> Reactor started on core 0 on socket 0
> *** RDMA Transport Init ***
> *** RDMA Transport Init ***
> allocated subsystem nqn.2014-08.org.nvmexpress.discovery on lcore 0 on
> socket 0
> allocated subsystem nqn.2016-06.io.spdk:cnode1 on lcore 0 on socket 0
> *** NVMf Target Listening on 192.168.25.3 port 4420 ***
> conf.c: 565:spdk_nvmf_parse_subsystem: ***ERROR*** Could not find NVMe
> controller for Subsystem1
> nvmf_tgt.c: 336:spdk_nvmf_startup: ***ERROR*** spdk_nvmf_parse_conf()
> failed
>
>
> Can you please tell me more about "domain:bus:func:dev", from where I can
> extract this data ?
>
>
> Thanks and Regards
> Ankur
>
>
> On Tue, Feb 28, 2017 at 7:56 PM, thomas.zhang.8288 <
> thomas.zhang.8288(a)aliyun.com> wrote:
> Hi Ankur,
>
> You configuration missing nvme 'domain:bus:func:dev' info in the second
> nvmf subsystem 'subsystem2'. If you choose mode 'Direct' , you have to
> specific 'nvme' bus info in the  nvmf subsystem, otherwise you can choose
> 'virtual mode'.
>
> Best Reagrds
>
> Zhihao
>
>
> ------------------------------------------------------------------
> 发件人:Ankur Srivastava <asrivastava014(a)gmail.com>
> 发送时间:2017年2月28日(星期二) 21:57
> 收件人:spdk <spdk(a)lists.01.org>
> 主 题:[SPDK] NVMeF Target Error NVMe controller
>
> Hi All,
>
>
> Here are some notes that I took on my system when starting the nvmf target and faced an issue with NVMe Controller discovery
>
> 1. Mellanox NICs detected successfully.
>
> 2. Installed neccesary modules
>
>    a. modprobe nvme
>
>    b. modprobe nvmet
>
>    c. modprobe nvme-rdma
>
>    d. modprobe nvmet-rdma
>
> 3. Assign NICs ip addresses
>
>    a. ip addr add 192.168.25.3/24 dev nic0
>    b. ip addr add 192.168.25.31/24 dev nic1
>
>
> 4. My configuration file has the following Subsystems
>
> # Direct controller
> [Subsystem1]
>   NQN nqn.2016-06.io.spdk:cnode1
>   Core 0
>   Mode Direct
>   Listen RDMA 192.168.25.3:4420
>   Host nqn.2016-06.io.spdk:init
>   NVMe 0000:06:00.0
>
> # Multiple subsystems are allowed.
> [Subsystem2]
>   NQN nqn.2016-06.io.spdk:cnode2
>   Core 0
>   Mode Direct
>   Listen RDMA 192.168.25.31:4420
>   Host nqn.2016-06.io.spdk:init
>   SN SPDK00000000000001
>   Namespace Malloc0
>   Namespace Malloc1
>
>
> 5. Start the nvmf target
>
> cd ~/spdk/app/nvmf_tgt
> sudo ./nvmf_tgt -c nvmf.conf.in
>
>
> Getting the following error
>
> [root(a)localhost spdk-master]# app/nvmf_tgt/nvmf_tgt -c etc/spdk/nvmf.conf.in
> Starting Intel(R) DPDK initialization ...
> [ DPDK EAL parameters: nvmf -c 1 -n 4 -m 2048 --master-lcore=0 --file-prefix=rte0 --proc-type=auto ]
> EAL: Detected 12 lcore(s)
> EAL: Auto-detected process type: PRIMARY
> EAL: Probing VFIO support...
> done.
> Occupied cpu core mask is 0x1
> Occupied cpu socket mask is 0x1
> Ioat Copy Engine Offload Enabled
> Total cores available: 1
> Reactor started on core 0 on socket 0
> *** RDMA Transport Init ***
> *** RDMA Transport Init ***
> allocated subsystem nqn.2014-08.org.nvmexpress.discovery on lcore 0 on socket 0
> allocated subsystem nqn.2016-06.io.spdk:cnode1 on lcore 0 on socket 0
> *** NVMf Target Listening on 192.168.25.3 port 4420 ***
> conf.c: 565:spdk_nvmf_parse_subsystem: ***ERROR*** Could not find NVMe controller for Subsystem1
> nvmf_tgt.c: 336:spdk_nvmf_startup: ***ERROR*** spdk_nvmf_parse_conf() failed
>
>
>
>
>
>
> Best Reagrds
>
> Ankur
>
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
>
>

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 13077 bytes --]

^ permalink raw reply	[flat|nested] 5+ messages in thread

* [SPDK] 回复: NVMeF Target Error NVMe controller
@ 2017-02-28 14:26 thomas.zhang.8288
  0 siblings, 0 replies; 5+ messages in thread
From: thomas.zhang.8288 @ 2017-02-28 14:26 UTC (permalink / raw)
  To: spdk

[-- Attachment #1: Type: text/plain, Size: 2503 bytes --]

Hi Ankur,
You configuration missing nvme 'domain:bus:func:dev' info in the second nvmf subsystem 'subsystem2'. If you choose mode 'Direct' , you have to specific 'nvme' bus info in the  nvmf subsystem, otherwise you can choose 'virtual mode'.
Best ReagrdsZhihao
------------------------------------------------------------------发件人:Ankur Srivastava <asrivastava014(a)gmail.com>发送时间:2017年2月28日(星期二) 21:57收件人:spdk <spdk(a)lists.01.org>主 题:[SPDK] NVMeF Target Error NVMe controller
Hi All,
Here are some notes that I took on my system when starting the nvmf target and faced an issue with NVMe Controller discovery

1. Mellanox NICs detected successfully.2. Installed neccesary modules   a. modprobe nvme   b. modprobe nvmet   c. modprobe nvme-rdma   d. modprobe nvmet-rdma3. Assign NICs ip addresses   a. ip addr add 192.168.25.3/24 dev nic0
   b. ip addr add 192.168.25.31/24 dev nic1


4. My configuration file has the following Subsystems

# Direct controller
[Subsystem1]
  NQN nqn.2016-06.io.spdk:cnode1
  Core 0
  Mode Direct
  Listen RDMA 192.168.25.3:4420
  Host nqn.2016-06.io.spdk:init
  NVMe 0000:06:00.0

# Multiple subsystems are allowed.
[Subsystem2]
  NQN nqn.2016-06.io.spdk:cnode2
  Core 0
  Mode Direct
  Listen RDMA 192.168.25.31:4420
  Host nqn.2016-06.io.spdk:init
  SN SPDK00000000000001
  Namespace Malloc0
  Namespace Malloc1
5. Start the nvmf target
cd ~/spdk/app/nvmf_tgt
sudo ./nvmf_tgt -c nvmf.conf.in
Getting the following error[root(a)localhost spdk-master]# app/nvmf_tgt/nvmf_tgt -c etc/spdk/nvmf.conf.in
Starting Intel(R) DPDK initialization ...
[ DPDK EAL parameters: nvmf -c 1 -n 4 -m 2048 --master-lcore=0 --file-prefix=rte0 --proc-type=auto ]
EAL: Detected 12 lcore(s)
EAL: Auto-detected process type: PRIMARY
EAL: Probing VFIO support...
done.
Occupied cpu core mask is 0x1
Occupied cpu socket mask is 0x1
Ioat Copy Engine Offload Enabled
Total cores available: 1
Reactor started on core 0 on socket 0
*** RDMA Transport Init ***
*** RDMA Transport Init ***
allocated subsystem nqn.2014-08.org.nvmexpress.discovery on lcore 0 on socket 0
allocated subsystem nqn.2016-06.io.spdk:cnode1 on lcore 0 on socket 0
*** NVMf Target Listening on 192.168.25.3 port 4420 ***
conf.c: 565:spdk_nvmf_parse_subsystem: ***ERROR*** Could not find NVMe controller for Subsystem1
nvmf_tgt.c: 336:spdk_nvmf_startup: ***ERROR*** spdk_nvmf_parse_conf() failed





Best ReagrdsAnkur

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 4540 bytes --]

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2017-02-28 17:31 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-02-28 14:38 [SPDK] 回复: NVMeF Target Error NVMe controller Ankur Srivastava
  -- strict thread matches above, loose matches on Subject: below --
2017-02-28 17:31 Wenhua Liu
2017-02-28 16:48 Walker, Benjamin
2017-02-28 16:07 Ankur Srivastava
2017-02-28 14:26 thomas.zhang.8288

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.