* Re: [SPDK] BDEV Registration
@ 2016-10-21 7:25 Kumaraparameshwaran Rathnavel
0 siblings, 0 replies; 12+ messages in thread
From: Kumaraparameshwaran Rathnavel @ 2016-10-21 7:25 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 10352 bytes --]
Hi Everyone ,
Got the issue resolved by increasing the number huge pages.
Thanks for your Support will definitely need in future too.
Thanking You,
Param
> On 21-Oct-2016, at 12:23 PM, Kumaraparameshwaran Rathnavel <krath(a)cloudsimple.com> wrote:
>
> These config lines were commented. I just enabled it. It fails even after that. Is it because of this error No free hugepages reported in hugepages-1048576kB
>
> Thanking You,
> Param
>
> On 21-Oct-2016, at 12:11 PM, Yang, Ziye <ziye.yang(a)intel.com <mailto:ziye.yang(a)intel.com>> wrote:
>>
>> Do you have set malloc module in your configuration file? You need have the malloc configuration such as
>>
>> [Malloc]
>> NumberOfLuns 4
>> LunSizeInMB 64
>>
>> <>
>> From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of Kumaraparameshwaran Rathnavel
>> Sent: Friday, October 21, 2016 2:37 PM
>> To: Storage Performance Development Kit <spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>>
>> Subject: Re: [SPDK] BDEV Registration
>>
>> My platform does not have RNIC but the bdev should get initialised right. Does it have any dependency on this transport type. Irrespective of this bdevs should get added right.
>>
>> Thanking You,
>> Param
>> On 21-Oct-2016, at 11:59 AM, Yang, Ziye <ziye.yang(a)intel.com <mailto:ziye.yang(a)intel.com>> wrote:
>>
>> Virtual controller is supported. From your output, it seems that you did not compile with RDMA, does your platform have RNIC?
>>
>> “conf.c: 420:spdk_nvmf_parse_subsystem: ***ERROR*** Unknown transport type 'RDMA'
>> ”
>>
>> From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of Kumaraparameshwaran Rathnavel
>> Sent: Friday, October 21, 2016 2:27 PM
>> To: Storage Performance Development Kit <spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>>
>> Subject: Re: [SPDK] BDEV Registration
>>
>> Is Virtual controller not supported in the current NVMe/F target implementation. Because in configuration file it is written as only direct mode is currently supported. This is the output generated. and my config file.
>>
>> Starting Intel(R) DPDK initialization ...
>> [ DPDK EAL parameters: nvmf -c 1 -n 4 -m 2048 --master-lcore=0 --file-prefix=rte0 --proc-type=auto ]
>> EAL: Detected 4 lcore(s)
>> EAL: Auto-detected process type: PRIMARY
>> EAL: No free hugepages reported in hugepages-1048576kB
>> EAL: Probing VFIO support...
>> done.
>> Occupied cpu core mask is 0x1
>> Occupied cpu socket mask is 0x1
>> copy_engine_ioat.c: 288:copy_engine_ioat_init: ***ERROR*** Not enough IOAT channels for all cores
>> Total cores available: 1
>> Reactor started on core 0x0
>>
>> *** NVMf Pool Creation ***
>> conf.c: 420:spdk_nvmf_parse_subsystem: ***ERROR*** Unknown transport type 'RDMA'
>> conf.c: 516:spdk_nvmf_parse_subsystem: ***ERROR*** bdev is NULL
>> nvmf_tgt.c: 254:spdk_nvmf_startup: ***ERROR*** spdk_nvmf_parse_conf() failed
>>
>> The subystem part of the config file
>> [Subsystem1]
>> NQN nqn.2016-06.io.spdk:cnode2
>> Core 0
>> Mode Virtual
>> Listen RDMA 192.168.2.21:4420
>> Host nqn.2016-06.io.spdk:init
>> SN SPDK00000000000001
>> Namespace Malloc0
>> Namespace Malloc1
>>
>> Thanking You,
>> Param.
>> On 21-Oct-2016, at 11:43 AM, Yang, Ziye <ziye.yang(a)intel.com <mailto:ziye.yang(a)intel.com>> wrote:
>>
>> #define SPDK_SUBSYSTEM_REGISTER(_name, _init, _fini, _config) \
>> struct spdk_subsystem __spdk_subsystem_ ## _name = { \
>> .name = #_name, \
>> .init = _init, \
>> .fini = _fini, \
>> .config = _config, \
>> }; \
>> __attribute__((constructor)) static void _name ## _register(void) \
>> { \
>> spdk_add_subsystem(&__spdk_subsystem_ ## _name); \
>> }
>>
>> The attribute is constructor, so the function will be directly called before the execution of main. I think that for your case, you just need to change the configuration file.
>>
>> -----Original Message-----
>> From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of Kumaraparameshwaran Rathnavel
>> Sent: Friday, October 21, 2016 2:09 PM
>> To: Storage Performance Development Kit <spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>>
>> Subject: Re: [SPDK] BDEV Registration
>>
>> Thanks for the update.
>>
>> I am still not clear who should call the function SPDK_SUBSYSTEM_REGISTER. I just wanted to add virtual mode subsystem as the NVMe/F target. In configuration the Virtual has two Malloc drives. So even before the main function I should create these devices and add. I find SPDK_SUBSYSTEM_REGISTER in lib/bdev/malloc/blocdev_malloc.c . So who should call this function? Should I call it by writing a separate program.
>>
>> Thanking You,
>> Param
>>
>>
>>
>> Hi Param,
>>
>> In the iscsi_tgt's main:
>>
>> spdk_app_init -> spdk_subsystem_init -> subsystem->init()
>>
>> (bdev is also a subsystem: registered with SPDK_SUBSYSTEM_REGISTER(bdev, spdk_bdev_initialize, spdk_bdev_finish, spdk_bdev_config_text))
>>
>> SPDK_SUBSYSTEM_REGISTER in bdev will be executed before main, and add the bdev subsystem.
>>
>> It will finally call spdk_bdev_initialize -> spdk_bdev_module_initialize -> call module_init.
>> Thanks.
>>
>> Best Regards,
>> Ziye Yang
>>
>> -----Original Message-----
>> From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of
>> Kumaraparameshwaran Rathnavel
>> Sent: Friday, October 21, 2016 12:09 PM
>> To: Storage Performance Development Kit <spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>>
>> Subject: Re: [SPDK] BDEV Registration
>>
>> Thanks Daniel.
>>
>> When will the bdev subsystem get initialised? Which part of the code handles the bdev subsystem initialisation.
>>
>> Thanking You,
>> Param
>>
>>
>> On 21-Oct-2016, at 3:29 AM, Daniel Verkamp <daniel.verkamp(a)intel.com <mailto:daniel.verkamp(a)intel.com>> wrote:
>>
>> On 10/20/2016 01:11 PM, Kumaraparameshwaran Rathnavel wrote:
>>
>>
>> Hi All,
>>
>> I have just started using SPDK. I am in need to implement a NVMe/F target implementation using SPDK. I need to use the Virtual Subsystem. I see that there is support for RAM disks and I am using it as reference to my Virtual implementation .
>>
>> There is a function called SPDK_BDEV_MODULE_REGISTER in the file lib/bdev/malloc/blockdev_malloc.c.
>>
>> How is this function getting called? Who calls this function
>>
>> Regards
>> Param
>>
>> Hi Param,
>>
>> You can find the interface for writing new bdev modules in the header
>> lib/bdev/bdev_module.h - this is where SPDK_BDEV_MODULE_REGISTER() is defined. This macro creates a constructor function that adds your bdev module's function pointer table (defined using a spdk_bdev_module_if) to a global list.
>>
>> Later, when the bdev subsystem is initialized, each bdev module will have its module_init function called, where it should create its bdevs and register them using spdk_bdev_register(). Currently, all of the bdevs in SPDK use our configuration file library to read from a common conf file when determining which bdevs are created and what options to use (e.g. the malloc bdev uses the [Malloc] section of the configuration file).
>>
>> When registering a bdev, you must provide another set of function pointers in fn_table which perform operations on the individual bdevs, like creating a new I/O channel and submitting I/O requests on a channel.
>>
>> To hook up your bdev module in the NVMe over Fabrics target, you will also want to modify mk/spdk.modules.mk to add your new module to the BLOCKDEV_MODULES list. This adds linker options to all of the applications in the SPDK tree that use bdev modules.
>>
>> Please let us know if you have any further questions - the bdev module API is still somewhat in flux and not well documented.
>>
>> Thanks,
>> -- Daniel
>> _______________________________________________
>> SPDK mailing list
>> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
>> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
>>
>> _______________________________________________
>> SPDK mailing list
>> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
>> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
>> _______________________________________________
>> SPDK mailing list
>> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
>> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
>>
>> _______________________________________________
>> SPDK mailing list
>> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
>> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
>> _______________________________________________
>> SPDK mailing list
>> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
>> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
>>
>> _______________________________________________
>> SPDK mailing list
>> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
>> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
>>
>> _______________________________________________
>> SPDK mailing list
>> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
>> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 42255 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [SPDK] BDEV Registration
@ 2016-10-21 6:53 Kumaraparameshwaran Rathnavel
0 siblings, 0 replies; 12+ messages in thread
From: Kumaraparameshwaran Rathnavel @ 2016-10-21 6:53 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 9790 bytes --]
These config lines were commented. I just enabled it. It fails even after that. Is it because of this error No free hugepages reported in hugepages-1048576kB
Thanking You,
Param
On 21-Oct-2016, at 12:11 PM, Yang, Ziye <ziye.yang(a)intel.com> wrote:
>
> Do you have set malloc module in your configuration file? You need have the malloc configuration such as
>
> [Malloc]
> NumberOfLuns 4
> LunSizeInMB 64
>
> <>
> From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Kumaraparameshwaran Rathnavel
> Sent: Friday, October 21, 2016 2:37 PM
> To: Storage Performance Development Kit <spdk(a)lists.01.org>
> Subject: Re: [SPDK] BDEV Registration
>
> My platform does not have RNIC but the bdev should get initialised right. Does it have any dependency on this transport type. Irrespective of this bdevs should get added right.
>
> Thanking You,
> Param
> On 21-Oct-2016, at 11:59 AM, Yang, Ziye <ziye.yang(a)intel.com <mailto:ziye.yang(a)intel.com>> wrote:
>
> Virtual controller is supported. From your output, it seems that you did not compile with RDMA, does your platform have RNIC?
>
> “conf.c: 420:spdk_nvmf_parse_subsystem: ***ERROR*** Unknown transport type 'RDMA'
> ”
>
> From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of Kumaraparameshwaran Rathnavel
> Sent: Friday, October 21, 2016 2:27 PM
> To: Storage Performance Development Kit <spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>>
> Subject: Re: [SPDK] BDEV Registration
>
> Is Virtual controller not supported in the current NVMe/F target implementation. Because in configuration file it is written as only direct mode is currently supported. This is the output generated. and my config file.
>
> Starting Intel(R) DPDK initialization ...
> [ DPDK EAL parameters: nvmf -c 1 -n 4 -m 2048 --master-lcore=0 --file-prefix=rte0 --proc-type=auto ]
> EAL: Detected 4 lcore(s)
> EAL: Auto-detected process type: PRIMARY
> EAL: No free hugepages reported in hugepages-1048576kB
> EAL: Probing VFIO support...
> done.
> Occupied cpu core mask is 0x1
> Occupied cpu socket mask is 0x1
> copy_engine_ioat.c: 288:copy_engine_ioat_init: ***ERROR*** Not enough IOAT channels for all cores
> Total cores available: 1
> Reactor started on core 0x0
>
> *** NVMf Pool Creation ***
> conf.c: 420:spdk_nvmf_parse_subsystem: ***ERROR*** Unknown transport type 'RDMA'
> conf.c: 516:spdk_nvmf_parse_subsystem: ***ERROR*** bdev is NULL
> nvmf_tgt.c: 254:spdk_nvmf_startup: ***ERROR*** spdk_nvmf_parse_conf() failed
>
> The subystem part of the config file
> [Subsystem1]
> NQN nqn.2016-06.io.spdk:cnode2
> Core 0
> Mode Virtual
> Listen RDMA 192.168.2.21:4420
> Host nqn.2016-06.io.spdk:init
> SN SPDK00000000000001
> Namespace Malloc0
> Namespace Malloc1
>
> Thanking You,
> Param.
> On 21-Oct-2016, at 11:43 AM, Yang, Ziye <ziye.yang(a)intel.com <mailto:ziye.yang(a)intel.com>> wrote:
>
> #define SPDK_SUBSYSTEM_REGISTER(_name, _init, _fini, _config) \
> struct spdk_subsystem __spdk_subsystem_ ## _name = { \
> .name = #_name, \
> .init = _init, \
> .fini = _fini, \
> .config = _config, \
> }; \
> __attribute__((constructor)) static void _name ## _register(void) \
> { \
> spdk_add_subsystem(&__spdk_subsystem_ ## _name); \
> }
>
> The attribute is constructor, so the function will be directly called before the execution of main. I think that for your case, you just need to change the configuration file.
>
> -----Original Message-----
> From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of Kumaraparameshwaran Rathnavel
> Sent: Friday, October 21, 2016 2:09 PM
> To: Storage Performance Development Kit <spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>>
> Subject: Re: [SPDK] BDEV Registration
>
> Thanks for the update.
>
> I am still not clear who should call the function SPDK_SUBSYSTEM_REGISTER. I just wanted to add virtual mode subsystem as the NVMe/F target. In configuration the Virtual has two Malloc drives. So even before the main function I should create these devices and add. I find SPDK_SUBSYSTEM_REGISTER in lib/bdev/malloc/blocdev_malloc.c . So who should call this function? Should I call it by writing a separate program.
>
> Thanking You,
> Param
>
>
>
> Hi Param,
>
> In the iscsi_tgt's main:
>
> spdk_app_init -> spdk_subsystem_init -> subsystem->init()
>
> (bdev is also a subsystem: registered with SPDK_SUBSYSTEM_REGISTER(bdev, spdk_bdev_initialize, spdk_bdev_finish, spdk_bdev_config_text))
>
> SPDK_SUBSYSTEM_REGISTER in bdev will be executed before main, and add the bdev subsystem.
>
> It will finally call spdk_bdev_initialize -> spdk_bdev_module_initialize -> call module_init.
> Thanks.
>
> Best Regards,
> Ziye Yang
>
> -----Original Message-----
> From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of
> Kumaraparameshwaran Rathnavel
> Sent: Friday, October 21, 2016 12:09 PM
> To: Storage Performance Development Kit <spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>>
> Subject: Re: [SPDK] BDEV Registration
>
> Thanks Daniel.
>
> When will the bdev subsystem get initialised? Which part of the code handles the bdev subsystem initialisation.
>
> Thanking You,
> Param
>
>
> On 21-Oct-2016, at 3:29 AM, Daniel Verkamp <daniel.verkamp(a)intel.com <mailto:daniel.verkamp(a)intel.com>> wrote:
>
> On 10/20/2016 01:11 PM, Kumaraparameshwaran Rathnavel wrote:
>
>
> Hi All,
>
> I have just started using SPDK. I am in need to implement a NVMe/F target implementation using SPDK. I need to use the Virtual Subsystem. I see that there is support for RAM disks and I am using it as reference to my Virtual implementation .
>
> There is a function called SPDK_BDEV_MODULE_REGISTER in the file lib/bdev/malloc/blockdev_malloc.c.
>
> How is this function getting called? Who calls this function
>
> Regards
> Param
>
> Hi Param,
>
> You can find the interface for writing new bdev modules in the header
> lib/bdev/bdev_module.h - this is where SPDK_BDEV_MODULE_REGISTER() is defined. This macro creates a constructor function that adds your bdev module's function pointer table (defined using a spdk_bdev_module_if) to a global list.
>
> Later, when the bdev subsystem is initialized, each bdev module will have its module_init function called, where it should create its bdevs and register them using spdk_bdev_register(). Currently, all of the bdevs in SPDK use our configuration file library to read from a common conf file when determining which bdevs are created and what options to use (e.g. the malloc bdev uses the [Malloc] section of the configuration file).
>
> When registering a bdev, you must provide another set of function pointers in fn_table which perform operations on the individual bdevs, like creating a new I/O channel and submitting I/O requests on a channel.
>
> To hook up your bdev module in the NVMe over Fabrics target, you will also want to modify mk/spdk.modules.mk to add your new module to the BLOCKDEV_MODULES list. This adds linker options to all of the applications in the SPDK tree that use bdev modules.
>
> Please let us know if you have any further questions - the bdev module API is still somewhat in flux and not well documented.
>
> Thanks,
> -- Daniel
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 41419 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [SPDK] BDEV Registration
@ 2016-10-21 6:41 Yang, Ziye
0 siblings, 0 replies; 12+ messages in thread
From: Yang, Ziye @ 2016-10-21 6:41 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 8533 bytes --]
Do you have set malloc module in your configuration file? You need have the malloc configuration such as
[Malloc]
NumberOfLuns 4
LunSizeInMB 64
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Kumaraparameshwaran Rathnavel
Sent: Friday, October 21, 2016 2:37 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: Re: [SPDK] BDEV Registration
My platform does not have RNIC but the bdev should get initialised right. Does it have any dependency on this transport type. Irrespective of this bdevs should get added right.
Thanking You,
Param
On 21-Oct-2016, at 11:59 AM, Yang, Ziye <ziye.yang(a)intel.com<mailto:ziye.yang(a)intel.com>> wrote:
Virtual controller is supported. From your output, it seems that you did not compile with RDMA, does your platform have RNIC?
“conf.c: 420:spdk_nvmf_parse_subsystem: ***ERROR*** Unknown transport type 'RDMA'
”
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Kumaraparameshwaran Rathnavel
Sent: Friday, October 21, 2016 2:27 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] BDEV Registration
Is Virtual controller not supported in the current NVMe/F target implementation. Because in configuration file it is written as only direct mode is currently supported. This is the output generated. and my config file.
Starting Intel(R) DPDK initialization ...
[ DPDK EAL parameters: nvmf -c 1 -n 4 -m 2048 --master-lcore=0 --file-prefix=rte0 --proc-type=auto ]
EAL: Detected 4 lcore(s)
EAL: Auto-detected process type: PRIMARY
EAL: No free hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
done.
Occupied cpu core mask is 0x1
Occupied cpu socket mask is 0x1
copy_engine_ioat.c: 288:copy_engine_ioat_init: ***ERROR*** Not enough IOAT channels for all cores
Total cores available: 1
Reactor started on core 0x0
*** NVMf Pool Creation ***
conf.c: 420:spdk_nvmf_parse_subsystem: ***ERROR*** Unknown transport type 'RDMA'
conf.c: 516:spdk_nvmf_parse_subsystem: ***ERROR*** bdev is NULL
nvmf_tgt.c: 254:spdk_nvmf_startup: ***ERROR*** spdk_nvmf_parse_conf() failed
The subystem part of the config file
[Subsystem1]
NQN nqn.2016-06.io.spdk:cnode2
Core 0
Mode Virtual
Listen RDMA 192.168.2.21:4420
Host nqn.2016-06.io.spdk:init
SN SPDK00000000000001
Namespace Malloc0
Namespace Malloc1
Thanking You,
Param.
On 21-Oct-2016, at 11:43 AM, Yang, Ziye <ziye.yang(a)intel.com<mailto:ziye.yang(a)intel.com>> wrote:
#define SPDK_SUBSYSTEM_REGISTER(_name, _init, _fini, _config) \
struct spdk_subsystem __spdk_subsystem_ ## _name = { \
.name = #_name, \
.init = _init, \
.fini = _fini, \
.config = _config, \
}; \
__attribute__((constructor)) static void _name ## _register(void) \
{ \
spdk_add_subsystem(&__spdk_subsystem_ ## _name); \
}
The attribute is constructor, so the function will be directly called before the execution of main. I think that for your case, you just need to change the configuration file.
-----Original Message-----
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Kumaraparameshwaran Rathnavel
Sent: Friday, October 21, 2016 2:09 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] BDEV Registration
Thanks for the update.
I am still not clear who should call the function SPDK_SUBSYSTEM_REGISTER. I just wanted to add virtual mode subsystem as the NVMe/F target. In configuration the Virtual has two Malloc drives. So even before the main function I should create these devices and add. I find SPDK_SUBSYSTEM_REGISTER in lib/bdev/malloc/blocdev_malloc.c . So who should call this function? Should I call it by writing a separate program.
Thanking You,
Param
Hi Param,
In the iscsi_tgt's main:
spdk_app_init -> spdk_subsystem_init -> subsystem->init()
(bdev is also a subsystem: registered with SPDK_SUBSYSTEM_REGISTER(bdev, spdk_bdev_initialize, spdk_bdev_finish, spdk_bdev_config_text))
SPDK_SUBSYSTEM_REGISTER in bdev will be executed before main, and add the bdev subsystem.
It will finally call spdk_bdev_initialize -> spdk_bdev_module_initialize -> call module_init.
Thanks.
Best Regards,
Ziye Yang
-----Original Message-----
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of
Kumaraparameshwaran Rathnavel
Sent: Friday, October 21, 2016 12:09 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] BDEV Registration
Thanks Daniel.
When will the bdev subsystem get initialised? Which part of the code handles the bdev subsystem initialisation.
Thanking You,
Param
On 21-Oct-2016, at 3:29 AM, Daniel Verkamp <daniel.verkamp(a)intel.com<mailto:daniel.verkamp(a)intel.com>> wrote:
On 10/20/2016 01:11 PM, Kumaraparameshwaran Rathnavel wrote:
Hi All,
I have just started using SPDK. I am in need to implement a NVMe/F target implementation using SPDK. I need to use the Virtual Subsystem. I see that there is support for RAM disks and I am using it as reference to my Virtual implementation .
There is a function called SPDK_BDEV_MODULE_REGISTER in the file lib/bdev/malloc/blockdev_malloc.c.
How is this function getting called? Who calls this function
Regards
Param
Hi Param,
You can find the interface for writing new bdev modules in the header
lib/bdev/bdev_module.h - this is where SPDK_BDEV_MODULE_REGISTER() is defined. This macro creates a constructor function that adds your bdev module's function pointer table (defined using a spdk_bdev_module_if) to a global list.
Later, when the bdev subsystem is initialized, each bdev module will have its module_init function called, where it should create its bdevs and register them using spdk_bdev_register(). Currently, all of the bdevs in SPDK use our configuration file library to read from a common conf file when determining which bdevs are created and what options to use (e.g. the malloc bdev uses the [Malloc] section of the configuration file).
When registering a bdev, you must provide another set of function pointers in fn_table which perform operations on the individual bdevs, like creating a new I/O channel and submitting I/O requests on a channel.
To hook up your bdev module in the NVMe over Fabrics target, you will also want to modify mk/spdk.modules.mk to add your new module to the BLOCKDEV_MODULES list. This adds linker options to all of the applications in the SPDK tree that use bdev modules.
Please let us know if you have any further questions - the bdev module API is still somewhat in flux and not well documented.
Thanks,
-- Daniel
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 28747 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [SPDK] BDEV Registration
@ 2016-10-21 6:36 Kumaraparameshwaran Rathnavel
0 siblings, 0 replies; 12+ messages in thread
From: Kumaraparameshwaran Rathnavel @ 2016-10-21 6:36 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 8767 bytes --]
My platform does not have RNIC but the bdev should get initialised right. Does it have any dependency on this transport type. Irrespective of this bdevs should get added right.
Thanking You,
Param
> On 21-Oct-2016, at 11:59 AM, Yang, Ziye <ziye.yang(a)intel.com> wrote:
>
> Virtual controller is supported. From your output, it seems that you did not compile with RDMA, does your platform have RNIC?
>
> “conf.c: 420:spdk_nvmf_parse_subsystem: ***ERROR*** Unknown transport type 'RDMA'
> ”
> <>
> From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Kumaraparameshwaran Rathnavel
> Sent: Friday, October 21, 2016 2:27 PM
> To: Storage Performance Development Kit <spdk(a)lists.01.org>
> Subject: Re: [SPDK] BDEV Registration
>
> Is Virtual controller not supported in the current NVMe/F target implementation. Because in configuration file it is written as only direct mode is currently supported. This is the output generated. and my config file.
>
> Starting Intel(R) DPDK initialization ...
> [ DPDK EAL parameters: nvmf -c 1 -n 4 -m 2048 --master-lcore=0 --file-prefix=rte0 --proc-type=auto ]
> EAL: Detected 4 lcore(s)
> EAL: Auto-detected process type: PRIMARY
> EAL: No free hugepages reported in hugepages-1048576kB
> EAL: Probing VFIO support...
> done.
> Occupied cpu core mask is 0x1
> Occupied cpu socket mask is 0x1
> copy_engine_ioat.c: 288:copy_engine_ioat_init: ***ERROR*** Not enough IOAT channels for all cores
> Total cores available: 1
> Reactor started on core 0x0
>
> *** NVMf Pool Creation ***
> conf.c: 420:spdk_nvmf_parse_subsystem: ***ERROR*** Unknown transport type 'RDMA'
> conf.c: 516:spdk_nvmf_parse_subsystem: ***ERROR*** bdev is NULL
> nvmf_tgt.c: 254:spdk_nvmf_startup: ***ERROR*** spdk_nvmf_parse_conf() failed
>
> The subystem part of the config file
> [Subsystem1]
> NQN nqn.2016-06.io.spdk:cnode2
> Core 0
> Mode Virtual
> Listen RDMA 192.168.2.21:4420
> Host nqn.2016-06.io.spdk:init
> SN SPDK00000000000001
> Namespace Malloc0
> Namespace Malloc1
>
> Thanking You,
> Param.
> On 21-Oct-2016, at 11:43 AM, Yang, Ziye <ziye.yang(a)intel.com <mailto:ziye.yang(a)intel.com>> wrote:
>
> #define SPDK_SUBSYSTEM_REGISTER(_name, _init, _fini, _config) \
> struct spdk_subsystem __spdk_subsystem_ ## _name = { \
> .name = #_name, \
> .init = _init, \
> .fini = _fini, \
> .config = _config, \
> }; \
> __attribute__((constructor)) static void _name ## _register(void) \
> { \
> spdk_add_subsystem(&__spdk_subsystem_ ## _name); \
> }
>
> The attribute is constructor, so the function will be directly called before the execution of main. I think that for your case, you just need to change the configuration file.
>
> -----Original Message-----
> From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of Kumaraparameshwaran Rathnavel
> Sent: Friday, October 21, 2016 2:09 PM
> To: Storage Performance Development Kit <spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>>
> Subject: Re: [SPDK] BDEV Registration
>
> Thanks for the update.
>
> I am still not clear who should call the function SPDK_SUBSYSTEM_REGISTER. I just wanted to add virtual mode subsystem as the NVMe/F target. In configuration the Virtual has two Malloc drives. So even before the main function I should create these devices and add. I find SPDK_SUBSYSTEM_REGISTER in lib/bdev/malloc/blocdev_malloc.c . So who should call this function? Should I call it by writing a separate program.
>
> Thanking You,
> Param
>
>
> Hi Param,
>
> In the iscsi_tgt's main:
>
> spdk_app_init -> spdk_subsystem_init -> subsystem->init()
>
> (bdev is also a subsystem: registered with SPDK_SUBSYSTEM_REGISTER(bdev, spdk_bdev_initialize, spdk_bdev_finish, spdk_bdev_config_text))
>
> SPDK_SUBSYSTEM_REGISTER in bdev will be executed before main, and add the bdev subsystem.
>
> It will finally call spdk_bdev_initialize -> spdk_bdev_module_initialize -> call module_init.
> Thanks.
>
> Best Regards,
> Ziye Yang
>
> -----Original Message-----
> From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of
> Kumaraparameshwaran Rathnavel
> Sent: Friday, October 21, 2016 12:09 PM
> To: Storage Performance Development Kit <spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>>
> Subject: Re: [SPDK] BDEV Registration
>
> Thanks Daniel.
>
> When will the bdev subsystem get initialised? Which part of the code handles the bdev subsystem initialisation.
>
> Thanking You,
> Param
>
> On 21-Oct-2016, at 3:29 AM, Daniel Verkamp <daniel.verkamp(a)intel.com <mailto:daniel.verkamp(a)intel.com>> wrote:
>
> On 10/20/2016 01:11 PM, Kumaraparameshwaran Rathnavel wrote:
>
> Hi All,
>
> I have just started using SPDK. I am in need to implement a NVMe/F target implementation using SPDK. I need to use the Virtual Subsystem. I see that there is support for RAM disks and I am using it as reference to my Virtual implementation .
>
> There is a function called SPDK_BDEV_MODULE_REGISTER in the file lib/bdev/malloc/blockdev_malloc.c.
>
> How is this function getting called? Who calls this function
>
> Regards
> Param
>
> Hi Param,
>
> You can find the interface for writing new bdev modules in the header
> lib/bdev/bdev_module.h - this is where SPDK_BDEV_MODULE_REGISTER() is defined. This macro creates a constructor function that adds your bdev module's function pointer table (defined using a spdk_bdev_module_if) to a global list.
>
> Later, when the bdev subsystem is initialized, each bdev module will have its module_init function called, where it should create its bdevs and register them using spdk_bdev_register(). Currently, all of the bdevs in SPDK use our configuration file library to read from a common conf file when determining which bdevs are created and what options to use (e.g. the malloc bdev uses the [Malloc] section of the configuration file).
>
> When registering a bdev, you must provide another set of function pointers in fn_table which perform operations on the individual bdevs, like creating a new I/O channel and submitting I/O requests on a channel.
>
> To hook up your bdev module in the NVMe over Fabrics target, you will also want to modify mk/spdk.modules.mk to add your new module to the BLOCKDEV_MODULES list. This adds linker options to all of the applications in the SPDK tree that use bdev modules.
>
> Please let us know if you have any further questions - the bdev module API is still somewhat in flux and not well documented.
>
> Thanks,
> -- Daniel
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 32772 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [SPDK] BDEV Registration
@ 2016-10-21 6:29 Yang, Ziye
0 siblings, 0 replies; 12+ messages in thread
From: Yang, Ziye @ 2016-10-21 6:29 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 7631 bytes --]
Virtual controller is supported. From your output, it seems that you did not compile with RDMA, does your platform have RNIC?
"conf.c: 420:spdk_nvmf_parse_subsystem: ***ERROR*** Unknown transport type 'RDMA'
"
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Kumaraparameshwaran Rathnavel
Sent: Friday, October 21, 2016 2:27 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: Re: [SPDK] BDEV Registration
Is Virtual controller not supported in the current NVMe/F target implementation. Because in configuration file it is written as only direct mode is currently supported. This is the output generated. and my config file.
Starting Intel(R) DPDK initialization ...
[ DPDK EAL parameters: nvmf -c 1 -n 4 -m 2048 --master-lcore=0 --file-prefix=rte0 --proc-type=auto ]
EAL: Detected 4 lcore(s)
EAL: Auto-detected process type: PRIMARY
EAL: No free hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
done.
Occupied cpu core mask is 0x1
Occupied cpu socket mask is 0x1
copy_engine_ioat.c: 288:copy_engine_ioat_init: ***ERROR*** Not enough IOAT channels for all cores
Total cores available: 1
Reactor started on core 0x0
*** NVMf Pool Creation ***
conf.c: 420:spdk_nvmf_parse_subsystem: ***ERROR*** Unknown transport type 'RDMA'
conf.c: 516:spdk_nvmf_parse_subsystem: ***ERROR*** bdev is NULL
nvmf_tgt.c: 254:spdk_nvmf_startup: ***ERROR*** spdk_nvmf_parse_conf() failed
The subystem part of the config file
[Subsystem1]
NQN nqn.2016-06.io.spdk:cnode2
Core 0
Mode Virtual
Listen RDMA 192.168.2.21:4420
Host nqn.2016-06.io.spdk:init
SN SPDK00000000000001
Namespace Malloc0
Namespace Malloc1
Thanking You,
Param.
On 21-Oct-2016, at 11:43 AM, Yang, Ziye <ziye.yang(a)intel.com<mailto:ziye.yang(a)intel.com>> wrote:
#define SPDK_SUBSYSTEM_REGISTER(_name, _init, _fini, _config) \
struct spdk_subsystem __spdk_subsystem_ ## _name = { \
.name = #_name, \
.init = _init, \
.fini = _fini, \
.config = _config, \
}; \
__attribute__((constructor)) static void _name ## _register(void) \
{ \
spdk_add_subsystem(&__spdk_subsystem_ ## _name); \
}
The attribute is constructor, so the function will be directly called before the execution of main. I think that for your case, you just need to change the configuration file.
-----Original Message-----
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Kumaraparameshwaran Rathnavel
Sent: Friday, October 21, 2016 2:09 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] BDEV Registration
Thanks for the update.
I am still not clear who should call the function SPDK_SUBSYSTEM_REGISTER. I just wanted to add virtual mode subsystem as the NVMe/F target. In configuration the Virtual has two Malloc drives. So even before the main function I should create these devices and add. I find SPDK_SUBSYSTEM_REGISTER in lib/bdev/malloc/blocdev_malloc.c . So who should call this function? Should I call it by writing a separate program.
Thanking You,
Param
Hi Param,
In the iscsi_tgt's main:
spdk_app_init -> spdk_subsystem_init -> subsystem->init()
(bdev is also a subsystem: registered with SPDK_SUBSYSTEM_REGISTER(bdev, spdk_bdev_initialize, spdk_bdev_finish, spdk_bdev_config_text))
SPDK_SUBSYSTEM_REGISTER in bdev will be executed before main, and add the bdev subsystem.
It will finally call spdk_bdev_initialize -> spdk_bdev_module_initialize -> call module_init.
Thanks.
Best Regards,
Ziye Yang
-----Original Message-----
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of
Kumaraparameshwaran Rathnavel
Sent: Friday, October 21, 2016 12:09 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] BDEV Registration
Thanks Daniel.
When will the bdev subsystem get initialised? Which part of the code handles the bdev subsystem initialisation.
Thanking You,
Param
On 21-Oct-2016, at 3:29 AM, Daniel Verkamp <daniel.verkamp(a)intel.com<mailto:daniel.verkamp(a)intel.com>> wrote:
On 10/20/2016 01:11 PM, Kumaraparameshwaran Rathnavel wrote:
Hi All,
I have just started using SPDK. I am in need to implement a NVMe/F target implementation using SPDK. I need to use the Virtual Subsystem. I see that there is support for RAM disks and I am using it as reference to my Virtual implementation .
There is a function called SPDK_BDEV_MODULE_REGISTER in the file lib/bdev/malloc/blockdev_malloc.c.
How is this function getting called? Who calls this function
Regards
Param
Hi Param,
You can find the interface for writing new bdev modules in the header
lib/bdev/bdev_module.h - this is where SPDK_BDEV_MODULE_REGISTER() is defined. This macro creates a constructor function that adds your bdev module's function pointer table (defined using a spdk_bdev_module_if) to a global list.
Later, when the bdev subsystem is initialized, each bdev module will have its module_init function called, where it should create its bdevs and register them using spdk_bdev_register(). Currently, all of the bdevs in SPDK use our configuration file library to read from a common conf file when determining which bdevs are created and what options to use (e.g. the malloc bdev uses the [Malloc] section of the configuration file).
When registering a bdev, you must provide another set of function pointers in fn_table which perform operations on the individual bdevs, like creating a new I/O channel and submitting I/O requests on a channel.
To hook up your bdev module in the NVMe over Fabrics target, you will also want to modify mk/spdk.modules.mk to add your new module to the BLOCKDEV_MODULES list. This adds linker options to all of the applications in the SPDK tree that use bdev modules.
Please let us know if you have any further questions - the bdev module API is still somewhat in flux and not well documented.
Thanks,
-- Daniel
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 21300 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [SPDK] BDEV Registration
@ 2016-10-21 6:27 Kumaraparameshwaran Rathnavel
0 siblings, 0 replies; 12+ messages in thread
From: Kumaraparameshwaran Rathnavel @ 2016-10-21 6:27 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 6525 bytes --]
Is Virtual controller not supported in the current NVMe/F target implementation. Because in configuration file it is written as only direct mode is currently supported. This is the output generated. and my config file.
Starting Intel(R) DPDK initialization ...
[ DPDK EAL parameters: nvmf -c 1 -n 4 -m 2048 --master-lcore=0 --file-prefix=rte0 --proc-type=auto ]
EAL: Detected 4 lcore(s)
EAL: Auto-detected process type: PRIMARY
EAL: No free hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
done.
Occupied cpu core mask is 0x1
Occupied cpu socket mask is 0x1
copy_engine_ioat.c: 288:copy_engine_ioat_init: ***ERROR*** Not enough IOAT channels for all cores
Total cores available: 1
Reactor started on core 0x0
*** NVMf Pool Creation ***
conf.c: 420:spdk_nvmf_parse_subsystem: ***ERROR*** Unknown transport type 'RDMA'
conf.c: 516:spdk_nvmf_parse_subsystem: ***ERROR*** bdev is NULL
nvmf_tgt.c: 254:spdk_nvmf_startup: ***ERROR*** spdk_nvmf_parse_conf() failed
The subystem part of the config file
[Subsystem1]
NQN nqn.2016-06.io.spdk:cnode2
Core 0
Mode Virtual
Listen RDMA 192.168.2.21:4420
Host nqn.2016-06.io.spdk:init
SN SPDK00000000000001
Namespace Malloc0
Namespace Malloc1
Thanking You,
Param.
> On 21-Oct-2016, at 11:43 AM, Yang, Ziye <ziye.yang(a)intel.com> wrote:
>
> #define SPDK_SUBSYSTEM_REGISTER(_name, _init, _fini, _config) \
> struct spdk_subsystem __spdk_subsystem_ ## _name = { \
> .name = #_name, \
> .init = _init, \
> .fini = _fini, \
> .config = _config, \
> }; \
> __attribute__((constructor)) static void _name ## _register(void) \
> { \
> spdk_add_subsystem(&__spdk_subsystem_ ## _name); \
> }
>
> The attribute is constructor, so the function will be directly called before the execution of main. I think that for your case, you just need to change the configuration file.
>
> -----Original Message-----
> From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Kumaraparameshwaran Rathnavel
> Sent: Friday, October 21, 2016 2:09 PM
> To: Storage Performance Development Kit <spdk(a)lists.01.org>
> Subject: Re: [SPDK] BDEV Registration
>
> Thanks for the update.
>
> I am still not clear who should call the function SPDK_SUBSYSTEM_REGISTER. I just wanted to add virtual mode subsystem as the NVMe/F target. In configuration the Virtual has two Malloc drives. So even before the main function I should create these devices and add. I find SPDK_SUBSYSTEM_REGISTER in lib/bdev/malloc/blocdev_malloc.c . So who should call this function? Should I call it by writing a separate program.
>
> Thanking You,
> Param
>>
>> Hi Param,
>>
>> In the iscsi_tgt's main:
>>
>> spdk_app_init -> spdk_subsystem_init -> subsystem->init()
>>
>> (bdev is also a subsystem: registered with SPDK_SUBSYSTEM_REGISTER(bdev, spdk_bdev_initialize, spdk_bdev_finish, spdk_bdev_config_text))
>>
>> SPDK_SUBSYSTEM_REGISTER in bdev will be executed before main, and add the bdev subsystem.
>>
>> It will finally call spdk_bdev_initialize -> spdk_bdev_module_initialize -> call module_init.
>> Thanks.
>>
>> Best Regards,
>> Ziye Yang
>>
>> -----Original Message-----
>> From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of
>> Kumaraparameshwaran Rathnavel
>> Sent: Friday, October 21, 2016 12:09 PM
>> To: Storage Performance Development Kit <spdk(a)lists.01.org>
>> Subject: Re: [SPDK] BDEV Registration
>>
>> Thanks Daniel.
>>
>> When will the bdev subsystem get initialised? Which part of the code handles the bdev subsystem initialisation.
>>
>> Thanking You,
>> Param
>>> On 21-Oct-2016, at 3:29 AM, Daniel Verkamp <daniel.verkamp(a)intel.com> wrote:
>>>
>>> On 10/20/2016 01:11 PM, Kumaraparameshwaran Rathnavel wrote:
>>>> Hi All,
>>>>
>>>> I have just started using SPDK. I am in need to implement a NVMe/F target implementation using SPDK. I need to use the Virtual Subsystem. I see that there is support for RAM disks and I am using it as reference to my Virtual implementation .
>>>>
>>>> There is a function called SPDK_BDEV_MODULE_REGISTER in the file lib/bdev/malloc/blockdev_malloc.c.
>>>>
>>>> How is this function getting called? Who calls this function
>>>>
>>>> Regards
>>>> Param
>>>
>>> Hi Param,
>>>
>>> You can find the interface for writing new bdev modules in the header
>>> lib/bdev/bdev_module.h - this is where SPDK_BDEV_MODULE_REGISTER() is defined. This macro creates a constructor function that adds your bdev module's function pointer table (defined using a spdk_bdev_module_if) to a global list.
>>>
>>> Later, when the bdev subsystem is initialized, each bdev module will have its module_init function called, where it should create its bdevs and register them using spdk_bdev_register(). Currently, all of the bdevs in SPDK use our configuration file library to read from a common conf file when determining which bdevs are created and what options to use (e.g. the malloc bdev uses the [Malloc] section of the configuration file).
>>>
>>> When registering a bdev, you must provide another set of function pointers in fn_table which perform operations on the individual bdevs, like creating a new I/O channel and submitting I/O requests on a channel.
>>>
>>> To hook up your bdev module in the NVMe over Fabrics target, you will also want to modify mk/spdk.modules.mk to add your new module to the BLOCKDEV_MODULES list. This adds linker options to all of the applications in the SPDK tree that use bdev modules.
>>>
>>> Please let us know if you have any further questions - the bdev module API is still somewhat in flux and not well documented.
>>>
>>> Thanks,
>>> -- Daniel
>>> _______________________________________________
>>> SPDK mailing list
>>> SPDK(a)lists.01.org
>>> https://lists.01.org/mailman/listinfo/spdk
>>
>> _______________________________________________
>> SPDK mailing list
>> SPDK(a)lists.01.org
>> https://lists.01.org/mailman/listinfo/spdk
>> _______________________________________________
>> SPDK mailing list
>> SPDK(a)lists.01.org
>> https://lists.01.org/mailman/listinfo/spdk
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 18692 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [SPDK] BDEV Registration
@ 2016-10-21 6:13 Yang, Ziye
0 siblings, 0 replies; 12+ messages in thread
From: Yang, Ziye @ 2016-10-21 6:13 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 4895 bytes --]
#define SPDK_SUBSYSTEM_REGISTER(_name, _init, _fini, _config) \
struct spdk_subsystem __spdk_subsystem_ ## _name = { \
.name = #_name, \
.init = _init, \
.fini = _fini, \
.config = _config, \
}; \
__attribute__((constructor)) static void _name ## _register(void) \
{ \
spdk_add_subsystem(&__spdk_subsystem_ ## _name); \
}
The attribute is constructor, so the function will be directly called before the execution of main. I think that for your case, you just need to change the configuration file.
-----Original Message-----
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Kumaraparameshwaran Rathnavel
Sent: Friday, October 21, 2016 2:09 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: Re: [SPDK] BDEV Registration
Thanks for the update.
I am still not clear who should call the function SPDK_SUBSYSTEM_REGISTER. I just wanted to add virtual mode subsystem as the NVMe/F target. In configuration the Virtual has two Malloc drives. So even before the main function I should create these devices and add. I find SPDK_SUBSYSTEM_REGISTER in lib/bdev/malloc/blocdev_malloc.c . So who should call this function? Should I call it by writing a separate program.
Thanking You,
Param
>
> Hi Param,
>
> In the iscsi_tgt's main:
>
> spdk_app_init -> spdk_subsystem_init -> subsystem->init()
>
> (bdev is also a subsystem: registered with SPDK_SUBSYSTEM_REGISTER(bdev, spdk_bdev_initialize, spdk_bdev_finish, spdk_bdev_config_text))
>
> SPDK_SUBSYSTEM_REGISTER in bdev will be executed before main, and add the bdev subsystem.
>
> It will finally call spdk_bdev_initialize -> spdk_bdev_module_initialize -> call module_init.
> Thanks.
>
> Best Regards,
> Ziye Yang
>
> -----Original Message-----
> From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of
> Kumaraparameshwaran Rathnavel
> Sent: Friday, October 21, 2016 12:09 PM
> To: Storage Performance Development Kit <spdk(a)lists.01.org>
> Subject: Re: [SPDK] BDEV Registration
>
> Thanks Daniel.
>
> When will the bdev subsystem get initialised? Which part of the code handles the bdev subsystem initialisation.
>
> Thanking You,
> Param
>> On 21-Oct-2016, at 3:29 AM, Daniel Verkamp <daniel.verkamp(a)intel.com> wrote:
>>
>> On 10/20/2016 01:11 PM, Kumaraparameshwaran Rathnavel wrote:
>>> Hi All,
>>>
>>> I have just started using SPDK. I am in need to implement a NVMe/F target implementation using SPDK. I need to use the Virtual Subsystem. I see that there is support for RAM disks and I am using it as reference to my Virtual implementation .
>>>
>>> There is a function called SPDK_BDEV_MODULE_REGISTER in the file lib/bdev/malloc/blockdev_malloc.c.
>>>
>>> How is this function getting called? Who calls this function
>>>
>>> Regards
>>> Param
>>
>> Hi Param,
>>
>> You can find the interface for writing new bdev modules in the header
>> lib/bdev/bdev_module.h - this is where SPDK_BDEV_MODULE_REGISTER() is defined. This macro creates a constructor function that adds your bdev module's function pointer table (defined using a spdk_bdev_module_if) to a global list.
>>
>> Later, when the bdev subsystem is initialized, each bdev module will have its module_init function called, where it should create its bdevs and register them using spdk_bdev_register(). Currently, all of the bdevs in SPDK use our configuration file library to read from a common conf file when determining which bdevs are created and what options to use (e.g. the malloc bdev uses the [Malloc] section of the configuration file).
>>
>> When registering a bdev, you must provide another set of function pointers in fn_table which perform operations on the individual bdevs, like creating a new I/O channel and submitting I/O requests on a channel.
>>
>> To hook up your bdev module in the NVMe over Fabrics target, you will also want to modify mk/spdk.modules.mk to add your new module to the BLOCKDEV_MODULES list. This adds linker options to all of the applications in the SPDK tree that use bdev modules.
>>
>> Please let us know if you have any further questions - the bdev module API is still somewhat in flux and not well documented.
>>
>> Thanks,
>> -- Daniel
>> _______________________________________________
>> SPDK mailing list
>> SPDK(a)lists.01.org
>> https://lists.01.org/mailman/listinfo/spdk
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org
https://lists.01.org/mailman/listinfo/spdk
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [SPDK] BDEV Registration
@ 2016-10-21 6:09 Kumaraparameshwaran Rathnavel
0 siblings, 0 replies; 12+ messages in thread
From: Kumaraparameshwaran Rathnavel @ 2016-10-21 6:09 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 3920 bytes --]
Thanks for the update.
I am still not clear who should call the function SPDK_SUBSYSTEM_REGISTER. I just wanted to add virtual mode subsystem as the NVMe/F target. In configuration the Virtual has two Malloc drives. So even before the main function I should create these devices and add. I find SPDK_SUBSYSTEM_REGISTER in lib/bdev/malloc/blocdev_malloc.c . So who should call this function? Should I call it by writing a separate program.
Thanking You,
Param
>
> Hi Param,
>
> In the iscsi_tgt's main:
>
> spdk_app_init -> spdk_subsystem_init -> subsystem->init()
>
> (bdev is also a subsystem: registered with SPDK_SUBSYSTEM_REGISTER(bdev, spdk_bdev_initialize, spdk_bdev_finish, spdk_bdev_config_text))
>
> SPDK_SUBSYSTEM_REGISTER in bdev will be executed before main, and add the bdev subsystem.
>
> It will finally call spdk_bdev_initialize -> spdk_bdev_module_initialize -> call module_init.
> Thanks.
>
> Best Regards,
> Ziye Yang
>
> -----Original Message-----
> From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Kumaraparameshwaran Rathnavel
> Sent: Friday, October 21, 2016 12:09 PM
> To: Storage Performance Development Kit <spdk(a)lists.01.org>
> Subject: Re: [SPDK] BDEV Registration
>
> Thanks Daniel.
>
> When will the bdev subsystem get initialised? Which part of the code handles the bdev subsystem initialisation.
>
> Thanking You,
> Param
>> On 21-Oct-2016, at 3:29 AM, Daniel Verkamp <daniel.verkamp(a)intel.com> wrote:
>>
>> On 10/20/2016 01:11 PM, Kumaraparameshwaran Rathnavel wrote:
>>> Hi All,
>>>
>>> I have just started using SPDK. I am in need to implement a NVMe/F target implementation using SPDK. I need to use the Virtual Subsystem. I see that there is support for RAM disks and I am using it as reference to my Virtual implementation .
>>>
>>> There is a function called SPDK_BDEV_MODULE_REGISTER in the file lib/bdev/malloc/blockdev_malloc.c.
>>>
>>> How is this function getting called? Who calls this function
>>>
>>> Regards
>>> Param
>>
>> Hi Param,
>>
>> You can find the interface for writing new bdev modules in the header
>> lib/bdev/bdev_module.h - this is where SPDK_BDEV_MODULE_REGISTER() is defined. This macro creates a constructor function that adds your bdev module's function pointer table (defined using a spdk_bdev_module_if) to a global list.
>>
>> Later, when the bdev subsystem is initialized, each bdev module will have its module_init function called, where it should create its bdevs and register them using spdk_bdev_register(). Currently, all of the bdevs in SPDK use our configuration file library to read from a common conf file when determining which bdevs are created and what options to use (e.g. the malloc bdev uses the [Malloc] section of the configuration file).
>>
>> When registering a bdev, you must provide another set of function pointers in fn_table which perform operations on the individual bdevs, like creating a new I/O channel and submitting I/O requests on a channel.
>>
>> To hook up your bdev module in the NVMe over Fabrics target, you will also want to modify mk/spdk.modules.mk to add your new module to the BLOCKDEV_MODULES list. This adds linker options to all of the applications in the SPDK tree that use bdev modules.
>>
>> Please let us know if you have any further questions - the bdev module API is still somewhat in flux and not well documented.
>>
>> Thanks,
>> -- Daniel
>> _______________________________________________
>> SPDK mailing list
>> SPDK(a)lists.01.org
>> https://lists.01.org/mailman/listinfo/spdk
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [SPDK] BDEV Registration
@ 2016-10-21 5:31 Yang, Ziye
0 siblings, 0 replies; 12+ messages in thread
From: Yang, Ziye @ 2016-10-21 5:31 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 3206 bytes --]
Hi Param,
In the iscsi_tgt's main:
spdk_app_init -> spdk_subsystem_init -> subsystem->init()
(bdev is also a subsystem: registered with SPDK_SUBSYSTEM_REGISTER(bdev, spdk_bdev_initialize, spdk_bdev_finish, spdk_bdev_config_text))
SPDK_SUBSYSTEM_REGISTER in bdev will be executed before main, and add the bdev subsystem.
It will finally call spdk_bdev_initialize -> spdk_bdev_module_initialize -> call module_init.
Thanks.
Best Regards,
Ziye Yang
-----Original Message-----
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Kumaraparameshwaran Rathnavel
Sent: Friday, October 21, 2016 12:09 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: Re: [SPDK] BDEV Registration
Thanks Daniel.
When will the bdev subsystem get initialised? Which part of the code handles the bdev subsystem initialisation.
Thanking You,
Param
> On 21-Oct-2016, at 3:29 AM, Daniel Verkamp <daniel.verkamp(a)intel.com> wrote:
>
> On 10/20/2016 01:11 PM, Kumaraparameshwaran Rathnavel wrote:
>> Hi All,
>>
>> I have just started using SPDK. I am in need to implement a NVMe/F target implementation using SPDK. I need to use the Virtual Subsystem. I see that there is support for RAM disks and I am using it as reference to my Virtual implementation .
>>
>> There is a function called SPDK_BDEV_MODULE_REGISTER in the file lib/bdev/malloc/blockdev_malloc.c.
>>
>> How is this function getting called? Who calls this function
>>
>> Regards
>> Param
>
> Hi Param,
>
> You can find the interface for writing new bdev modules in the header
> lib/bdev/bdev_module.h - this is where SPDK_BDEV_MODULE_REGISTER() is defined. This macro creates a constructor function that adds your bdev module's function pointer table (defined using a spdk_bdev_module_if) to a global list.
>
> Later, when the bdev subsystem is initialized, each bdev module will have its module_init function called, where it should create its bdevs and register them using spdk_bdev_register(). Currently, all of the bdevs in SPDK use our configuration file library to read from a common conf file when determining which bdevs are created and what options to use (e.g. the malloc bdev uses the [Malloc] section of the configuration file).
>
> When registering a bdev, you must provide another set of function pointers in fn_table which perform operations on the individual bdevs, like creating a new I/O channel and submitting I/O requests on a channel.
>
> To hook up your bdev module in the NVMe over Fabrics target, you will also want to modify mk/spdk.modules.mk to add your new module to the BLOCKDEV_MODULES list. This adds linker options to all of the applications in the SPDK tree that use bdev modules.
>
> Please let us know if you have any further questions - the bdev module API is still somewhat in flux and not well documented.
>
> Thanks,
> -- Daniel
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org
https://lists.01.org/mailman/listinfo/spdk
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [SPDK] BDEV Registration
@ 2016-10-21 4:09 Kumaraparameshwaran Rathnavel
0 siblings, 0 replies; 12+ messages in thread
From: Kumaraparameshwaran Rathnavel @ 2016-10-21 4:09 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 2323 bytes --]
Thanks Daniel.
When will the bdev subsystem get initialised? Which part of the code handles the bdev subsystem initialisation.
Thanking You,
Param
> On 21-Oct-2016, at 3:29 AM, Daniel Verkamp <daniel.verkamp(a)intel.com> wrote:
>
> On 10/20/2016 01:11 PM, Kumaraparameshwaran Rathnavel wrote:
>> Hi All,
>>
>> I have just started using SPDK. I am in need to implement a NVMe/F target implementation using SPDK. I need to use the Virtual Subsystem. I see that there is support for RAM disks and I am using it as reference to my Virtual implementation .
>>
>> There is a function called SPDK_BDEV_MODULE_REGISTER in the file lib/bdev/malloc/blockdev_malloc.c.
>>
>> How is this function getting called? Who calls this function
>>
>> Regards
>> Param
>
> Hi Param,
>
> You can find the interface for writing new bdev modules in the header
> lib/bdev/bdev_module.h - this is where SPDK_BDEV_MODULE_REGISTER() is defined. This macro creates a constructor function that adds your bdev module's function pointer table (defined using a spdk_bdev_module_if) to a global list.
>
> Later, when the bdev subsystem is initialized, each bdev module will have its module_init function called, where it should create its bdevs and register them using spdk_bdev_register(). Currently, all of the bdevs in SPDK use our configuration file library to read from a common conf file when determining which bdevs are created and what options to use (e.g. the malloc bdev uses the [Malloc] section of the configuration file).
>
> When registering a bdev, you must provide another set of function pointers in fn_table which perform operations on the individual bdevs, like creating a new I/O channel and submitting I/O requests on a channel.
>
> To hook up your bdev module in the NVMe over Fabrics target, you will also want to modify mk/spdk.modules.mk to add your new module to the BLOCKDEV_MODULES list. This adds linker options to all of the applications in the SPDK tree that use bdev modules.
>
> Please let us know if you have any further questions - the bdev module API is still somewhat in flux and not well documented.
>
> Thanks,
> -- Daniel
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [SPDK] BDEV Registration
@ 2016-10-20 21:59 Daniel Verkamp
0 siblings, 0 replies; 12+ messages in thread
From: Daniel Verkamp @ 2016-10-20 21:59 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 1921 bytes --]
On 10/20/2016 01:11 PM, Kumaraparameshwaran Rathnavel wrote:
> Hi All,
>
> I have just started using SPDK. I am in need to implement a NVMe/F target implementation using SPDK. I need to use the Virtual Subsystem. I see that there is support for RAM disks and I am using it as reference to my Virtual implementation .
>
> There is a function called SPDK_BDEV_MODULE_REGISTER in the file lib/bdev/malloc/blockdev_malloc.c.
>
> How is this function getting called? Who calls this function
>
> Regards
> Param
Hi Param,
You can find the interface for writing new bdev modules in the header
lib/bdev/bdev_module.h - this is where SPDK_BDEV_MODULE_REGISTER() is
defined. This macro creates a constructor function that adds your bdev
module's function pointer table (defined using a spdk_bdev_module_if) to
a global list.
Later, when the bdev subsystem is initialized, each bdev module will
have its module_init function called, where it should create its bdevs
and register them using spdk_bdev_register(). Currently, all of the
bdevs in SPDK use our configuration file library to read from a common
conf file when determining which bdevs are created and what options to
use (e.g. the malloc bdev uses the [Malloc] section of the configuration
file).
When registering a bdev, you must provide another set of function
pointers in fn_table which perform operations on the individual bdevs,
like creating a new I/O channel and submitting I/O requests on a channel.
To hook up your bdev module in the NVMe over Fabrics target, you will
also want to modify mk/spdk.modules.mk to add your new module to the
BLOCKDEV_MODULES list. This adds linker options to all of the
applications in the SPDK tree that use bdev modules.
Please let us know if you have any further questions - the bdev module
API is still somewhat in flux and not well documented.
Thanks,
-- Daniel
^ permalink raw reply [flat|nested] 12+ messages in thread
* [SPDK] BDEV Registration
@ 2016-10-20 20:11 Kumaraparameshwaran Rathnavel
0 siblings, 0 replies; 12+ messages in thread
From: Kumaraparameshwaran Rathnavel @ 2016-10-20 20:11 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 464 bytes --]
Hi All,
I have just started using SPDK. I am in need to implement a NVMe/F target implementation using SPDK. I need to use the Virtual Subsystem. I see that there is support for RAM disks and I am using it as reference to my Virtual implementation .
There is a function called SPDK_BDEV_MODULE_REGISTER in the file lib/bdev/malloc/blockdev_malloc.c.
How is this function getting called? Who calls this function
Regards
Param
Sent from my iPhone
^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2016-10-21 7:25 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-10-21 7:25 [SPDK] BDEV Registration Kumaraparameshwaran Rathnavel
-- strict thread matches above, loose matches on Subject: below --
2016-10-21 6:53 Kumaraparameshwaran Rathnavel
2016-10-21 6:41 Yang, Ziye
2016-10-21 6:36 Kumaraparameshwaran Rathnavel
2016-10-21 6:29 Yang, Ziye
2016-10-21 6:27 Kumaraparameshwaran Rathnavel
2016-10-21 6:13 Yang, Ziye
2016-10-21 6:09 Kumaraparameshwaran Rathnavel
2016-10-21 5:31 Yang, Ziye
2016-10-21 4:09 Kumaraparameshwaran Rathnavel
2016-10-20 21:59 Daniel Verkamp
2016-10-20 20:11 Kumaraparameshwaran Rathnavel
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.