Re: [SPDK] How to construct thousands lvol-bdevs and export them via iSCSI quickly?

Re: [SPDK] How to construct thousands lvol-bdevs and export them via iSCSI quickly?

[Pelplinski, Piotr]

[-- Attachment #1: Type: text/plain, Size: 4103 bytes --]

You mentioned that creating 3000 lvols took you 6 minutes. 
Did you called rpc.py 3000 times? I think that most of the time took just to initialize python script.

I think that just sending 3000 rpc requests with single python script would reduce the time significantly. 


-- 
Best Regards,
Piotr Pelpliński


> -----Original Message-----
> From: Pelplinski, Piotr
> Sent: Friday, January 11, 2019 11:28 AM
> To: 'Storage Performance Development Kit' <spdk(a)lists.01.org>
> Subject: RE: [SPDK] How to construct thousands lvol-bdevs and export them via
> iSCSI quickly?
> 
> Hi,
> 
> Constructing bdevs or target nodes are considered management operations and
> therefore they might not be optimized for performance.
> I don't see a reason why not just return json array of lvol names. They are going
> to be parsed by a script anyway, right?
> Instead of creating new RPC call, maybe you could add additional parameter to
> construct_lvol_bdev that would create n bdevs?
> Similarly, construct_target_node could add additional parameter to construct
> multiple nodes.
> Would you like to send your code for review on gerrithub?
> 
> Anyway, I think that biggest overhead might not be in sending multiple requests
> instead of one, but the json rpc library itself.
> I did some initial comparisons on creating big rpc responses
> (https://trello.com/c/gg8oXsdo/152-how-to-handle-big-rpc-responses)
> It looks like we are spending most of the time concatenating strings for the
> response.
> Of course, that would probably look a little bit different with rpc calls you
> mention.
> 
> We are looking for couple of different approaches to improve this area.
> If you come up with any suggestions, please share your thoughts.
> 
> 
> --
> Best Regards,
> Piotr Pelpliński
> 
> > -----Original Message-----
> > From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of ???
> > Sent: Friday, January 11, 2019 4:28 AM
> > To: spdk(a)lists.01.org
> > Subject: [SPDK] How to construct thousands lvol-bdevs and export them via
> iSCSI
> > quickly?
> >
> > Hi everyone:
> >
> >
> > I am a novice to use SPDK. Recently, I faced a demand in my project: to
> > construct thousands lvol-bdevs based on a SPDK-bdev as soon as possible,
> then
> > export all of them via iSCSI quickly.
> >
> >
> > At the beginning, I called the `construct_lvol_bdev` JSON-PRC method to
> create
> > a lvol-bdev each time. However, it spent a long time when I need to created
> > about 3000 lvol-bdevs (about 6 minutes). So I choose to add a JSON-PRC
> method
> > by myself, to create thousands lvol-bdevs on one command. Thanks for you
> > guys' great work,  this idea was not difficult to achieve. And I gave this new
> > JSON-PRC method a name called `construct_n_lvol_bdev`.
> >
> >
> > But then I need to export those lvol-bdevs via iSCSI. Normally, when I call the
> > `construct_lvol_bdev` JSON-PRC method, it will construct a lvol-bdev and then
> > return the uuid/name of this one. But when I call my `construct_n_lvol_bdev`
> > method (the method added by me) to create thousands lvol-bdevs at a time, I
> > don't think it's a good user experience to print thousands lvol-bdevs' names in
> > screen after the method return.
> >
> >
> > So, is it a good idea to let `construct_n_lvol_bdev` method records all names
> of
> > the new lvol-bdevs into a text file? And then, use this text file to export those
> > lvol-bdevs by the `construct_target_node` JSON-PRC method (write a Python
> > script to do this).
> > Or, is it a better idea to make a new JSON-RPC method to create and export
> > lvol-bdevs one by one (firstly, create a lvol-bdev A, then export it by iSCSI.
> > Secondly, create another lvol-bdev B, then export this new one by iSCSI ...), or
> > even use multiple threads to this?
> >
> >
> > Thanks for your advice.
> >
> >
> > Jianing Guo
> >
> >
> >
> >
> > _______________________________________________
> > SPDK mailing list
> > SPDK(a)lists.01.org
> > https://lists.01.org/mailman/listinfo/spdk

[SPDK] How to construct thousands lvol-bdevs and export them via iSCSI quickly?

[@ 2019-09-28 2:06 UTC (permalink / raw)]

[-- Attachment #1: Type: text/plain, Size: 1670 bytes --]

Hi everyone:


I am a novice to use SPDK. Recently, I faced a demand in my project: to construct thousands lvol-bdevs based on a SPDK-bdev as soon as possible, then export all of them via iSCSI quickly.


At the beginning, I called the `construct_lvol_bdev` JSON-PRC method to create a lvol-bdev each time. However, it spent a long time when I need to created about 3000 lvol-bdevs (about 6 minutes). So I choose to add a JSON-PRC method by myself, to create thousands lvol-bdevs on one command. Thanks for you guys' great work,  this idea was not difficult to achieve. And I gave this new JSON-PRC method a name called `construct_n_lvol_bdev`.


But then I need to export those lvol-bdevs via iSCSI. Normally, when I call the `construct_lvol_bdev` JSON-PRC method, it will construct a lvol-bdev and then return the uuid/name of this one. But when I call my `construct_n_lvol_bdev` method (the method added by me) to create thousands lvol-bdevs at a time, I don't think it's a good user experience to print thousands lvol-bdevs' names in screen after the method return. 


So, is it a good idea to let `construct_n_lvol_bdev` method records all names of the new lvol-bdevs into a text file? And then, use this text file to export those lvol-bdevs by the `construct_target_node` JSON-PRC method (write a Python script to do this). 
Or, is it a better idea to make a new JSON-RPC method to create and export lvol-bdevs one by one (firstly, create a lvol-bdev A, then export it by iSCSI. Secondly, create another lvol-bdev B, then export this new one by iSCSI ...), or even use multiple threads to this?


Thanks for your advice. 


Jianing Guo

Re: [SPDK] How to construct thousands lvol-bdevs and export them via iSCSI quickly?

[Pelplinski, Piotr]

[-- Attachment #1: Type: text/plain, Size: 3344 bytes --]

Hi,

Constructing bdevs or target nodes are considered management operations and therefore they might not be optimized for performance. 
I don't see a reason why not just return json array of lvol names. They are going to be parsed by a script anyway, right?
Instead of creating new RPC call, maybe you could add additional parameter to construct_lvol_bdev that would create n bdevs?
Similarly, construct_target_node could add additional parameter to construct multiple nodes.
Would you like to send your code for review on gerrithub?

Anyway, I think that biggest overhead might not be in sending multiple requests instead of one, but the json rpc library itself. 
I did some initial comparisons on creating big rpc responses (https://trello.com/c/gg8oXsdo/152-how-to-handle-big-rpc-responses)
It looks like we are spending most of the time concatenating strings for the response.
Of course, that would probably look a little bit different with rpc calls you mention.

We are looking for couple of different approaches to improve this area. 
If you come up with any suggestions, please share your thoughts.


-- 
Best Regards,
Piotr Pelpliński

> -----Original Message-----
> From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of ???
> Sent: Friday, January 11, 2019 4:28 AM
> To: spdk(a)lists.01.org
> Subject: [SPDK] How to construct thousands lvol-bdevs and export them via iSCSI
> quickly?
> 
> Hi everyone:
> 
> 
> I am a novice to use SPDK. Recently, I faced a demand in my project: to
> construct thousands lvol-bdevs based on a SPDK-bdev as soon as possible, then
> export all of them via iSCSI quickly.
> 
> 
> At the beginning, I called the `construct_lvol_bdev` JSON-PRC method to create
> a lvol-bdev each time. However, it spent a long time when I need to created
> about 3000 lvol-bdevs (about 6 minutes). So I choose to add a JSON-PRC method
> by myself, to create thousands lvol-bdevs on one command. Thanks for you
> guys' great work,  this idea was not difficult to achieve. And I gave this new
> JSON-PRC method a name called `construct_n_lvol_bdev`.
> 
> 
> But then I need to export those lvol-bdevs via iSCSI. Normally, when I call the
> `construct_lvol_bdev` JSON-PRC method, it will construct a lvol-bdev and then
> return the uuid/name of this one. But when I call my `construct_n_lvol_bdev`
> method (the method added by me) to create thousands lvol-bdevs at a time, I
> don't think it's a good user experience to print thousands lvol-bdevs' names in
> screen after the method return.
> 
> 
> So, is it a good idea to let `construct_n_lvol_bdev` method records all names of
> the new lvol-bdevs into a text file? And then, use this text file to export those
> lvol-bdevs by the `construct_target_node` JSON-PRC method (write a Python
> script to do this).
> Or, is it a better idea to make a new JSON-RPC method to create and export
> lvol-bdevs one by one (firstly, create a lvol-bdev A, then export it by iSCSI.
> Secondly, create another lvol-bdev B, then export this new one by iSCSI ...), or
> even use multiple threads to this?
> 
> 
> Thanks for your advice.
> 
> 
> Jianing Guo
> 
> 
> 
> 
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk