Re: [SPDK] Add py-spdk client for SPDK

Re: [SPDK] Add py-spdk client for SPDK

[We We]

[-- Attachment #1: Type: text/plain, Size: 11579 bytes --]

Hi Paul,

I checked the mailing list about the meeting, but it was a bit too long.
Could you tell me how I attend the SPDK community meeting this Thu？

Regards,
Helloway


> 在 2018年2月6日，下午12:07，Luse, Paul E <paul.e.luse(a)intel.com> 写道：
> 
> That’s great Helloway.  I would strongly suggest one thing at a time though; first tackle the front end interfaces and then once that’s completed then start seeing where people’s heads are at wrt things like protobuf. That’s pretty common in most open source communities and definitely is here – smaller patches and smaller projects are always better.  Easier to understand, easier to review and easier to land!
>  
> Keep your eyes open on the dist list for the meeting time, look forward to talking to you finally J
>  
> -Paul
>   <>
>  <>From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
> Sent: Monday, February 5, 2018 8:35 PM
> To: Storage Performance Development Kit <spdk(a)lists.01.org>
> Subject: Re: [SPDK] Add py-spdk client for SPDK
>  
> Hi Paul,
>  
> Thank you for your reply and advice. 
>  
> First of all I whole-heartly agree with you that it is a better approach to take the scripts/rpc.py and scripts/rpc components and build those out/refactor them to provide more/better functionality. 
>  
> Secondly, I think you really hit on the target and the very essence of the benefit that a consistent data model protobuf could help bring. We are definitely in line on this one.
>  
> I would like to help to lead the effort which has been discussed thoroughly in this email thread, and work with any contributor in the SPDK community who is interested and willing to participate :)
>  
> Also look forward to my first community meeting this Thursday ! 
>  
> Thanks,
> Helloway
>  
> 在 2018年2月6日，上午3:20，Luse, Paul E <paul.e.luse(a)intel.com <mailto:paul.e.luse(a)intel.com>> 写道：
>  
> Hi WeWe,
>  
> Thanks for the additional insight.  On the py-spdk client yes I see the value that you are describing  however instead of adding on the py-spdk client a better approach might be take the scripts/rpc.py and scripts/rpc components and build those out/refactor them to provide more/better functionality.  There’s enough “likeness” between what you are proposing and what’s already there that I’m afraid a lot of the reluctance to adopt this is based on the increased maintenance of adding this new code weighed against the increased value.  If you improve on what’s there one piece at a time I think you can get a lot accomplished here.
>  
> Also, you should be aware that there’s some pending work to consolidate the SPDK target applications themselves, to have one target application with parameters instead of multiple binaries that do very similar work. I can see a good fit for your goals and that activity to work together to have a single python generic interface as well as a single binary underneath it both architected at the same time.  
>  
> Would you be interested in working with some other community folks on that? I’m not sure who it was that was talking about looking into that but hopefully they’ll see this and chime in J I think there might be a community meeting this Thu, if someone knows please reply – we could talk on that call if that would help.
>  
> Thanks
> Paul
>  
> PS: Or, you could work independently to get a generic python layer usable by applications “built into” the existing rpc/scripts code. A good place to start would be to pick one sample app, like nvmf target, and list out the functions that are currently provided via the existing rpc py code and see how you can add the ones that are in your patch by extending/re-organizing/refactor existing code. I would definitely start with just one application though and get buy-in from a maintainer that this is the right design choice.  Then, the rest of the work will go quite smoothly I think.
>  
>  
> From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of We We
> Sent: Monday, February 5, 2018 2:21 AM
> To: Storage Performance Development Kit <spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>>
> Subject: Re: [SPDK] Add py-spdk client for SPDK
>  
> Hi Paul,
> I am trying to answer your questions from two points.
> Firstly, the py-spdk/nvmf_client.py provides the same abilities that can be done in the /scripts/rpc/nvmf.py, the difference, however, is that they are in the upper and lower layer respectively. Why should we do it? Imagine that there is only one management app which needs to invoke SPDK-base apps, we can write the module alone in your own management app and directly communicate with the directly /scripts/rpc/nvmf.py. However, when there are multiple upper management apps that need to invoke SPDK-base apps, we need to provide the unified software named ‘py-spdk’ and laid between the SPDK-based apps and upper management apps. If we don’t do so, then each management app needs to write the same module to interact with /scripts/rpc/nvmf.py. The py-spdk provides a generic interface for upper management apps. In other words, the upper management apps are not care about the functional realization of the backend, they just work through the interfaces provided by the py-spdk.
>  
> Secondly, why did we use the protobuf as data model? Because the protobuf provides a better standardized description of the API than JSON. From the code point of view, we only need to define a ‘.proto’file that describes the data and compiled it with different languages to the lib file (such as: spdk_pb2.py, spdk.go, etc.). Each ‘message’of the ‘.proto’file will be compiled to a protobuf obj which served as the carrier of the data and returned to the upper management apps. Compared to the JSON, the upper management apps can quickly learn that the returned value which is passed through the lib file rather than the manual document. From the ecological point of view，many mainstream cloud-native apps are using protobuf as their data model (like k8s)，which is also an opportunity for us.    
>  
> I don't know  do I make sense. 
>  
> Regards,
> Helloway
>  
> 在 2018年1月28日，上午5:25，Luse, Paul E <paul.e.luse(a)intel.com <mailto:paul.e.luse(a)intel.com>> 写道：
>  
> Hi Howard,
>  
> I think the main question right now is about the new core functionality in the patch versus what Ben mentioned wrt what is there today in scripts/rpc:
>  
> __init__.py        client.py             lvol.py                nvmf.py
> app.py                iscsi.py nbd.py  pmem.py
> bdev.py              log.py                 net.py                 vhost.py
>  
> This is the change he mentioned that happened since the original py-spdk client was added.  So for example if I look at what is in the patch in py-spdk/nvmf_client.py I don’t see any capabilities that can’t be done in thescripts/rpc/nvmf.py but I could be missing something. I think the protobuf and repo location discussions are secondary to first establishing the value of the core functionality of the patch to the community.
>  
> Can you explain how the SDK Framework layer might be a better approach as compared to working on the files listed above?
>  
> Thanks
> Paul
>  
> PS:  There is an open community conference call every other week, I think the next one is coming up here soon and its usually announced on IRC a few days in advance.   If real time discussion would help we could always look at scheduling a separate open meeting at time more friendly to your time zone.
>  
> From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of Zhipeng Huang
> Sent: Saturday, January 27, 2018 12:11 AM
> To: Storage Performance Development Kit <spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>>
> Subject: Re: [SPDK] Add py-spdk client for SPDK
>  
> Hi Ben,
>  
> After discussing with my team, the general feeling is that keeping protobuf would be a preferred choice. The reason, as i stated earlier, is that proto provides a better standardized description of the API than JSON and also we could create data model rather quickly for bindings/tools written in other languages. We have already generated the go-spdk based upon the proto model for OpenSDS's interaction with SPDK drivers. It provides benefits at least judging from our own practice.
>  
> Frankly given gRPC's wide adoption I don't think this should be a big issue. If you still have doubt about this, I think maybe we could setup a conf call, or I could discuss with Harris when he's in China for SPDK summit.
>  
> For repo, I think it is entirely up to the community's decision on whether maintain it in the main repo or creating a new one.
>  
> On Sat, Jan 27, 2018 at 6:28 AM, Walker, Benjamin <benjamin.walker(a)intel.com <mailto:benjamin.walker(a)intel.com>> wrote:
> On Wed, 2018-01-24 at 07:13 +0800, Zhipeng Huang wrote:
> > Do we have a conclusion on this issue ? If it is ok to have a spdk/sdk repo,
> > then wewe will modify the current patch (get rid of protobuff) and resubmit
> > the patch to the new repo once it is established (meanwhile abandon the
> > current one to spdk/spdk).
> 
> If you remove protobuf, can you describe what is left? Recently scripts/rpc.py
> was refactored to break it up into a set of Python libraries in scripts/rpc,
> plus the command line tool at scripts/rpc.py. What functionality does this new
> code provide over and above what is already present there?
> 
> SPDK is certainly in need of better management tools, so in the most general
> sense the community is very supportive of your effort here. New management tools
> can also go directly into the main spdk repository (a separate repository was
> only suggested when we thought this was a Python binding to the SPDK libraries).
> I'm wondering if an easier way forward would be to continue refining the Python
> packages in scripts/rpc to be more general purpose libraries for sending the
> JSON RPCs. What are your thoughts on that?
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
> 
> 
>  
> -- 
> Zhipeng (Howard) Huang
>  
> Standard Engineer
> IT Standard & Patent/IT Product Line
> Huawei Technologies Co,. Ltd
> Email: huangzhipeng(a)huawei.com <mailto:huangzhipeng(a)huawei.com>
> Office: Huawei Industrial Base, Longgang, Shenzhen
>  
> (Previous)
> Research Assistant
> Mobile Ad-Hoc Network Lab, Calit2
> University of California, Irvine
> Email: zhipengh(a)uci.edu <mailto:zhipengh(a)uci.edu>
> Office: Calit2 Building Room 2402
>  
> OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
>  
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
>  
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 50951 bytes --]

Re: [SPDK] Add py-spdk client for SPDK

[helloway]

[-- Attachment #1: Type: text/plain, Size: 11674 bytes --]

Hi Daniel, Paul,
I have submitted new patch into https://review.gerrithub.io/#/c/405526/, please review it. By contrasting with my previous patch(https://review.gerrithub.io/#/c/403506/), to tell you what I thought originally. I think maybe we  do the same thing to some extent, If I get wrong, please correct me, any thoughts are appreciated. I really hope to receive your opinions.


Thx,
Helloway


On 02/8/2018 21:54，Luse, Paul E<paul.e.luse(a)intel.com> wrote：

It must be next Thu, Piotr?

 

From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Thursday, February 8, 2018 1:53 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: Re: [SPDK] Add py-spdk client for SPDK

 

Hi Paul,

 

I checked the mailing list about the meeting, but it was a bit too long.

Could you tell me how I attend the SPDK community meeting this Thu？

 

Regards,

Helloway

 

 

在 2018年2月6日，下午12:07，Luse, Paul E <paul.e.luse(a)intel.com> 写道：

 

That’s great Helloway.  I would strongly suggest one thing at a time though; first tackle the front end interfaces and then once that’s completed then start seeing where people’s heads are at wrt things like protobuf. That’s pretty common in most open source communities and definitely is here – smaller patches and smaller projects are always better.  Easier to understand, easier to review and easier to land!

 

Keep your eyes open on the dist list for the meeting time, look forward to talking to you finally J

 

-Paul

 

From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Monday, February 5, 2018 8:35 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: Re: [SPDK] Add py-spdk client for SPDK

 

Hi Paul,

 

Thank you for your reply and advice. 

 

First of all I whole-heartly agree with you that it is a better approach to take the scripts/rpc.py and scripts/rpc components and build those out/refactor them to provide more/better functionality. 

 

Secondly, I think you really hit on the target and the very essence of the benefit that a consistent data model protobuf could help bring. We are definitely in line on this one.

 

I would like to help to lead the effort which has been discussed thoroughly in this email thread, and work with any contributor in the SPDK community who is interested and willing to participate :)

 

Also look forward to my first community meeting this Thursday ! 

 

Thanks,

Helloway

 

在 2018年2月6日，上午3:20，Luse, Paul E <paul.e.luse(a)intel.com> 写道：

 

Hi WeWe,

 

Thanks for the additional insight.  On the py-spdk client yes I see the value that you are describing  however instead of adding on the py-spdk client a better approach might be take the scripts/rpc.py and scripts/rpc components and build those out/refactor them to provide more/better functionality.  There’s enough “likeness” between what you are proposing and what’s already there that I’m afraid a lot of the reluctance to adopt this is based on the increased maintenance of adding this new code weighed against the increased value.  If you improve on what’s there one piece at a time I think you can get a lot accomplished here.

 

Also, you should be aware that there’s some pending work to consolidate the SPDK target applications themselves, to have one target application with parameters instead of multiple binaries that do very similar work. I can see a good fit for your goals and that activity to work together to have a single python generic interface as well as a single binary underneath it both architected at the same time.  

 

Would you be interested in working with some other community folks on that? I’m not sure who it was that was talking about looking into that but hopefully they’ll see this and chime in J I think there might be a community meeting this Thu, if someone knows please reply – we could talk on that call if that would help.

 

Thanks

Paul

 

PS: Or, you could work independently to get a generic python layer usable by applications “built into” the existing rpc/scripts code. A good place to start would be to pick one sample app, like nvmf target, and list out the functions that are currently provided via the existing rpc py code and see how you can add the ones that are in your patch by extending/re-organizing/refactor existing code. I would definitely start with just one application though and get buy-in from a maintainer that this is the right design choice.  Then, the rest of the work will go quite smoothly I think.

 

 

From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Monday, February 5, 2018 2:21 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: Re: [SPDK] Add py-spdk client for SPDK

 

Hi Paul,

I am trying to answer your questions from two points.

Firstly, the py-spdk/nvmf_client.py provides the same abilities that can be done in the /scripts/rpc/nvmf.py, the difference, however, is that they are in the upper and lower layer respectively. Why should we do it? Imagine that there is only one management app which needs to invoke SPDK-base apps, we can write the module alone in your own management app and directly communicate with the directly /scripts/rpc/nvmf.py. However, when there are multiple upper management apps that need to invoke SPDK-base apps, we need to provide the unified software named ‘py-spdk’ and laid between the SPDK-based apps and upper management apps. If we don’t do so, then each management app needs to write the same module to interact with /scripts/rpc/nvmf.py. The py-spdk provides a generic interface for upper management apps. In other words, the upper management apps are not care about the functional realization of the backend, they just work through the interfaces provided by the py-spdk.

 

Secondly, why did we use the protobuf as data model? Because the protobuf provides a better standardized description of the API than JSON. From the code point of view, we only need to define a ‘.proto’file that describes the data and compiled it with different languages to the lib file (such as: spdk_pb2.py, spdk.go, etc.). Each ‘message’of the ‘.proto’file will be compiled to a protobuf obj which served as the carrier of the data and returned to the upper management apps. Compared to the JSON, the upper management apps can quickly learn that the returned value which is passed through the lib file rather than the manual document. From the ecological point of view，many mainstream cloud-native apps are using protobuf as their data model (like k8s)，which is also an opportunity for us.    

 

I don't know  do I make sense. 

 

Regards,

Helloway

 

在 2018年1月28日，上午5:25，Luse, Paul E <paul.e.luse(a)intel.com> 写道：

 

Hi Howard,

 

I think the main question right now is about the new core functionality in the patch versus what Ben mentioned wrt what is there today in scripts/rpc:

 

__init__.py        client.py             lvol.py                nvmf.py

app.py                iscsi.py nbd.py  pmem.py

bdev.py              log.py                 net.py                 vhost.py

 

This is the change he mentioned that happened since the original py-spdk client was added.  So for example if I look at what is in the patch in py-spdk/nvmf_client.py I don’t see any capabilities that can’t be done in thescripts/rpc/nvmf.py but I could be missing something. I think the protobuf and repo location discussions are secondary to first establishing the value of the core functionality of the patch to the community.

 

Can you explain how the SDK Framework layer might be a better approach as compared to working on the files listed above?

 

Thanks

Paul

 

PS:  There is an open community conference call every other week, I think the next one is coming up here soon and its usually announced on IRC a few days in advance.   If real time discussion would help we could always look at scheduling a separate open meeting at time more friendly to your time zone.

 

From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Zhipeng Huang
Sent: Saturday, January 27, 2018 12:11 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: Re: [SPDK] Add py-spdk client for SPDK

 

Hi Ben,

 

After discussing with my team, the general feeling is that keeping protobuf would be a preferred choice. The reason, as i stated earlier, is that proto provides a better standardized description of the API than JSON and also we could create data model rather quickly for bindings/tools written in other languages. We have already generated the go-spdk based upon the proto model for OpenSDS's interaction with SPDK drivers. It provides benefits at least judging from our own practice.

 

Frankly given gRPC's wide adoption I don't think this should be a big issue. If you still have doubt about this, I think maybe we could setup a conf call, or I could discuss with Harris when he's in China for SPDK summit.

 

For repo, I think it is entirely up to the community's decision on whether maintain it in the main repo or creating a new one.

 

On Sat, Jan 27, 2018 at 6:28 AM, Walker, Benjamin <benjamin.walker(a)intel.com> wrote:

On Wed, 2018-01-24 at 07:13 +0800, Zhipeng Huang wrote:
> Do we have a conclusion on this issue ? If it is ok to have a spdk/sdk repo,
> then wewe will modify the current patch (get rid of protobuff) and resubmit
> the patch to the new repo once it is established (meanwhile abandon the
> current one to spdk/spdk).

If you remove protobuf, can you describe what is left? Recently scripts/rpc.py
was refactored to break it up into a set of Python libraries in scripts/rpc,
plus the command line tool at scripts/rpc.py. What functionality does this new
code provide over and above what is already present there?

SPDK is certainly in need of better management tools, so in the most general
sense the community is very supportive of your effort here. New management tools
can also go directly into the main spdk repository (a separate repository was
only suggested when we thought this was a Python binding to the SPDK libraries).
I'm wondering if an easier way forward would be to continue refining the Python
packages in scripts/rpc to be more general purpose libraries for sending the
JSON RPCs. What are your thoughts on that?

_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org
https://lists.01.org/mailman/listinfo/spdk





 

-- 

Zhipeng (Howard) Huang

 

Standard Engineer

IT Standard & Patent/IT Product Line

Huawei Technologies Co,. Ltd

Email: huangzhipeng(a)huawei.com

Office: Huawei Industrial Base, Longgang, Shenzhen

 

(Previous)

Research Assistant

Mobile Ad-Hoc Network Lab, Calit2

University of California, Irvine

Email: zhipengh(a)uci.edu

Office: Calit2 Building Room 2402

 

OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado

_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org
https://lists.01.org/mailman/listinfo/spdk

 

_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org
https://lists.01.org/mailman/listinfo/spdk

 

_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org
https://lists.01.org/mailman/listinfo/spdk

 

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 36320 bytes --]

Re: [SPDK] Add py-spdk client for SPDK

[Luse, Paul E]

[-- Attachment #1: Type: text/plain, Size: 11276 bytes --]

It must be next Thu, Piotr?

From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Thursday, February 8, 2018 1:53 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: Re: [SPDK] Add py-spdk client for SPDK

Hi Paul,

I checked the mailing list about the meeting, but it was a bit too long.
Could you tell me how I attend the SPDK community meeting this Thu？

Regards,
Helloway


在 2018年2月6日，下午12:07，Luse, Paul E <paul.e.luse(a)intel.com<mailto:paul.e.luse(a)intel.com>> 写道：

That’s great Helloway.  I would strongly suggest one thing at a time though; first tackle the front end interfaces and then once that’s completed then start seeing where people’s heads are at wrt things like protobuf. That’s pretty common in most open source communities and definitely is here – smaller patches and smaller projects are always better.  Easier to understand, easier to review and easier to land!

Keep your eyes open on the dist list for the meeting time, look forward to talking to you finally ☺

-Paul

From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Monday, February 5, 2018 8:35 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] Add py-spdk client for SPDK

Hi Paul,

Thank you for your reply and advice.

First of all I whole-heartly agree with you that it is a better approach to take the scripts/rpc.py and scripts/rpc components and build those out/refactor them to provide more/better functionality.

Secondly, I think you really hit on the target and the very essence of the benefit that a consistent data model protobuf could help bring. We are definitely in line on this one.

I would like to help to lead the effort which has been discussed thoroughly in this email thread, and work with any contributor in the SPDK community who is interested and willing to participate :)

Also look forward to my first community meeting this Thursday !

Thanks,
Helloway

在 2018年2月6日，上午3:20，Luse, Paul E <paul.e.luse(a)intel.com<mailto:paul.e.luse(a)intel.com>> 写道：

Hi WeWe,

Thanks for the additional insight.  On the py-spdk client yes I see the value that you are describing  however instead of adding on the py-spdk client a better approach might be take the scripts/rpc.py and scripts/rpc components and build those out/refactor them to provide more/better functionality.  There’s enough “likeness” between what you are proposing and what’s already there that I’m afraid a lot of the reluctance to adopt this is based on the increased maintenance of adding this new code weighed against the increased value.  If you improve on what’s there one piece at a time I think you can get a lot accomplished here.

Also, you should be aware that there’s some pending work to consolidate the SPDK target applications themselves, to have one target application with parameters instead of multiple binaries that do very similar work. I can see a good fit for your goals and that activity to work together to have a single python generic interface as well as a single binary underneath it both architected at the same time.

Would you be interested in working with some other community folks on that? I’m not sure who it was that was talking about looking into that but hopefully they’ll see this and chime in ☺ I think there might be a community meeting this Thu, if someone knows please reply – we could talk on that call if that would help.

Thanks
Paul

PS: Or, you could work independently to get a generic python layer usable by applications “built into” the existing rpc/scripts code. A good place to start would be to pick one sample app, like nvmf target, and list out the functions that are currently provided via the existing rpc py code and see how you can add the ones that are in your patch by extending/re-organizing/refactor existing code. I would definitely start with just one application though and get buy-in from a maintainer that this is the right design choice.  Then, the rest of the work will go quite smoothly I think.


From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Monday, February 5, 2018 2:21 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] Add py-spdk client for SPDK

Hi Paul,
I am trying to answer your questions from two points.
Firstly, the py-spdk/nvmf_client.py provides the same abilities that can be done in the /scripts/rpc/nvmf.py, the difference, however, is that they are in the upper and lower layer respectively. Why should we do it? Imagine that there is only one management app which needs to invoke SPDK-base apps, we can write the module alone in your own management app and directly communicate with the directly /scripts/rpc/nvmf.py. However, when there are multiple upper management apps that need to invoke SPDK-base apps, we need to provide the unified software named ‘py-spdk’ and laid between the SPDK-based apps and upper management apps. If we don’t do so, then each management app needs to write the same module to interact with /scripts/rpc/nvmf.py. The py-spdk provides a generic interface for upper management apps. In other words, the upper management apps are not care about the functional realization of the backend, they just work through the interfaces provided by the py-spdk.

Secondly, why did we use the protobuf as data model? Because the protobuf provides a better standardized description of the API than JSON. From the code point of view, we only need to define a ‘.proto’file that describes the data and compiled it with different languages to the lib file (such as: spdk_pb2.py, spdk.go, etc.). Each ‘message’of the ‘.proto’file will be compiled to a protobuf obj which served as the carrier of the data and returned to the upper management apps. Compared to the JSON, the upper management apps can quickly learn that the returned value which is passed through the lib file rather than the manual document. From the ecological point of view，many mainstream cloud-native apps are using protobuf as their data model (like k8s)，which is also an opportunity for us.

I don't know  do I make sense.

Regards,
Helloway

在 2018年1月28日，上午5:25，Luse, Paul E <paul.e.luse(a)intel.com<mailto:paul.e.luse(a)intel.com>> 写道：

Hi Howard,

I think the main question right now is about the new core functionality in the patch versus what Ben mentioned wrt what is there today in scripts/rpc:

__init__.py        client.py             lvol.py                nvmf.py
app.py                iscsi.py nbd.py  pmem.py
bdev.py              log.py                 net.py                 vhost.py

This is the change he mentioned that happened since the original py-spdk client was added.  So for example if I look at what is in the patch in py-spdk/nvmf_client.py I don’t see any capabilities that can’t be done in thescripts/rpc/nvmf.py but I could be missing something. I think the protobuf and repo location discussions are secondary to first establishing the value of the core functionality of the patch to the community.

Can you explain how the SDK Framework layer might be a better approach as compared to working on the files listed above?

Thanks
Paul

PS:  There is an open community conference call every other week, I think the next one is coming up here soon and its usually announced on IRC a few days in advance.   If real time discussion would help we could always look at scheduling a separate open meeting at time more friendly to your time zone.

From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Zhipeng Huang
Sent: Saturday, January 27, 2018 12:11 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] Add py-spdk client for SPDK

Hi Ben,

After discussing with my team, the general feeling is that keeping protobuf would be a preferred choice. The reason, as i stated earlier, is that proto provides a better standardized description of the API than JSON and also we could create data model rather quickly for bindings/tools written in other languages. We have already generated the go-spdk based upon the proto model for OpenSDS's interaction with SPDK drivers. It provides benefits at least judging from our own practice.

Frankly given gRPC's wide adoption I don't think this should be a big issue. If you still have doubt about this, I think maybe we could setup a conf call, or I could discuss with Harris when he's in China for SPDK summit.

For repo, I think it is entirely up to the community's decision on whether maintain it in the main repo or creating a new one.

On Sat, Jan 27, 2018 at 6:28 AM, Walker, Benjamin <benjamin.walker(a)intel.com<mailto:benjamin.walker(a)intel.com>> wrote:
On Wed, 2018-01-24 at 07:13 +0800, Zhipeng Huang wrote:
> Do we have a conclusion on this issue ? If it is ok to have a spdk/sdk repo,
> then wewe will modify the current patch (get rid of protobuff) and resubmit
> the patch to the new repo once it is established (meanwhile abandon the
> current one to spdk/spdk).

If you remove protobuf, can you describe what is left? Recently scripts/rpc.py
was refactored to break it up into a set of Python libraries in scripts/rpc,
plus the command line tool at scripts/rpc.py. What functionality does this new
code provide over and above what is already present there?

SPDK is certainly in need of better management tools, so in the most general
sense the community is very supportive of your effort here. New management tools
can also go directly into the main spdk repository (a separate repository was
only suggested when we thought this was a Python binding to the SPDK libraries).
I'm wondering if an easier way forward would be to continue refining the Python
packages in scripts/rpc to be more general purpose libraries for sending the
JSON RPCs. What are your thoughts on that?
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk



--
Zhipeng (Howard) Huang

Standard Engineer
IT Standard & Patent/IT Product Line
Huawei Technologies Co,. Ltd
Email: huangzhipeng(a)huawei.com<mailto:huangzhipeng(a)huawei.com>
Office: Huawei Industrial Base, Longgang, Shenzhen

(Previous)
Research Assistant
Mobile Ad-Hoc Network Lab, Calit2
University of California, Irvine
Email: zhipengh(a)uci.edu<mailto:zhipengh(a)uci.edu>
Office: Calit2 Building Room 2402

OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk

_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk

_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk


[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 37571 bytes --]

Re: [SPDK] Add py-spdk client for SPDK

[Luse, Paul E]

[-- Attachment #1: Type: text/plain, Size: 10526 bytes --]

That’s great Helloway.  I would strongly suggest one thing at a time though; first tackle the front end interfaces and then once that’s completed then start seeing where people’s heads are at wrt things like protobuf. That’s pretty common in most open source communities and definitely is here – smaller patches and smaller projects are always better.  Easier to understand, easier to review and easier to land!

Keep your eyes open on the dist list for the meeting time, look forward to talking to you finally ☺

-Paul

From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Monday, February 5, 2018 8:35 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: Re: [SPDK] Add py-spdk client for SPDK

Hi Paul,

Thank you for your reply and advice.

First of all I whole-heartly agree with you that it is a better approach to take the scripts/rpc.py and scripts/rpc components and build those out/refactor them to provide more/better functionality.

Secondly, I think you really hit on the target and the very essence of the benefit that a consistent data model protobuf could help bring. We are definitely in line on this one.

I would like to help to lead the effort which has been discussed thoroughly in this email thread, and work with any contributor in the SPDK community who is interested and willing to participate :)

Also look forward to my first community meeting this Thursday !

Thanks,
Helloway

在 2018年2月6日，上午3:20，Luse, Paul E <paul.e.luse(a)intel.com<mailto:paul.e.luse(a)intel.com>> 写道：

Hi WeWe,

Thanks for the additional insight.  On the py-spdk client yes I see the value that you are describing  however instead of adding on the py-spdk client a better approach might be take the scripts/rpc.py and scripts/rpc components and build those out/refactor them to provide more/better functionality.  There’s enough “likeness” between what you are proposing and what’s already there that I’m afraid a lot of the reluctance to adopt this is based on the increased maintenance of adding this new code weighed against the increased value.  If you improve on what’s there one piece at a time I think you can get a lot accomplished here.

Also, you should be aware that there’s some pending work to consolidate the SPDK target applications themselves, to have one target application with parameters instead of multiple binaries that do very similar work. I can see a good fit for your goals and that activity to work together to have a single python generic interface as well as a single binary underneath it both architected at the same time.

Would you be interested in working with some other community folks on that? I’m not sure who it was that was talking about looking into that but hopefully they’ll see this and chime in ☺ I think there might be a community meeting this Thu, if someone knows please reply – we could talk on that call if that would help.

Thanks
Paul

PS: Or, you could work independently to get a generic python layer usable by applications “built into” the existing rpc/scripts code. A good place to start would be to pick one sample app, like nvmf target, and list out the functions that are currently provided via the existing rpc py code and see how you can add the ones that are in your patch by extending/re-organizing/refactor existing code. I would definitely start with just one application though and get buy-in from a maintainer that this is the right design choice.  Then, the rest of the work will go quite smoothly I think.


From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Monday, February 5, 2018 2:21 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] Add py-spdk client for SPDK

Hi Paul,
I am trying to answer your questions from two points.
Firstly, the py-spdk/nvmf_client.py provides the same abilities that can be done in the /scripts/rpc/nvmf.py, the difference, however, is that they are in the upper and lower layer respectively. Why should we do it? Imagine that there is only one management app which needs to invoke SPDK-base apps, we can write the module alone in your own management app and directly communicate with the directly /scripts/rpc/nvmf.py. However, when there are multiple upper management apps that need to invoke SPDK-base apps, we need to provide the unified software named ‘py-spdk’ and laid between the SPDK-based apps and upper management apps. If we don’t do so, then each management app needs to write the same module to interact with /scripts/rpc/nvmf.py. The py-spdk provides a generic interface for upper management apps. In other words, the upper management apps are not care about the functional realization of the backend, they just work through the interfaces provided by the py-spdk.

Secondly, why did we use the protobuf as data model? Because the protobuf provides a better standardized description of the API than JSON. From the code point of view, we only need to define a ‘.proto’file that describes the data and compiled it with different languages to the lib file (such as: spdk_pb2.py, spdk.go, etc.). Each ‘message’of the ‘.proto’file will be compiled to a protobuf obj which served as the carrier of the data and returned to the upper management apps. Compared to the JSON, the upper management apps can quickly learn that the returned value which is passed through the lib file rather than the manual document. From the ecological point of view，many mainstream cloud-native apps are using protobuf as their data model (like k8s)，which is also an opportunity for us.

I don't know  do I make sense.

Regards,
Helloway

在 2018年1月28日，上午5:25，Luse, Paul E <paul.e.luse(a)intel.com<mailto:paul.e.luse(a)intel.com>> 写道：

Hi Howard,

I think the main question right now is about the new core functionality in the patch versus what Ben mentioned wrt what is there today in scripts/rpc:

__init__.py        client.py             lvol.py                nvmf.py
app.py                iscsi.py nbd.py  pmem.py
bdev.py              log.py                 net.py                 vhost.py

This is the change he mentioned that happened since the original py-spdk client was added.  So for example if I look at what is in the patch in py-spdk/nvmf_client.py I don’t see any capabilities that can’t be done in thescripts/rpc/nvmf.py but I could be missing something. I think the protobuf and repo location discussions are secondary to first establishing the value of the core functionality of the patch to the community.

Can you explain how the SDK Framework layer might be a better approach as compared to working on the files listed above?

Thanks
Paul

PS:  There is an open community conference call every other week, I think the next one is coming up here soon and its usually announced on IRC a few days in advance.  If real time discussion would help we could always look at scheduling a separate open meeting at time more friendly to your time zone.

From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Zhipeng Huang
Sent: Saturday, January 27, 2018 12:11 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] Add py-spdk client for SPDK

Hi Ben,

After discussing with my team, the general feeling is that keeping protobuf would be a preferred choice. The reason, as i stated earlier, is that proto provides a better standardized description of the API than JSON and also we could create data model rather quickly for bindings/tools written in other languages. We have already generated the go-spdk based upon the proto model for OpenSDS's interaction with SPDK drivers. It provides benefits at least judging from our own practice.

Frankly given gRPC's wide adoption I don't think this should be a big issue. If you still have doubt about this, I think maybe we could setup a conf call, or I could discuss with Harris when he's in China for SPDK summit.

For repo, I think it is entirely up to the community's decision on whether maintain it in the main repo or creating a new one.

On Sat, Jan 27, 2018 at 6:28 AM, Walker, Benjamin <benjamin.walker(a)intel.com<mailto:benjamin.walker(a)intel.com>> wrote:
On Wed, 2018-01-24 at 07:13 +0800, Zhipeng Huang wrote:
> Do we have a conclusion on this issue ? If it is ok to have a spdk/sdk repo,
> then wewe will modify the current patch (get rid of protobuff) and resubmit
> the patch to the new repo once it is established (meanwhile abandon the
> current one to spdk/spdk).

If you remove protobuf, can you describe what is left? Recently scripts/rpc.py
was refactored to break it up into a set of Python libraries in scripts/rpc,
plus the command line tool at scripts/rpc.py. What functionality does this new
code provide over and above what is already present there?

SPDK is certainly in need of better management tools, so in the most general
sense the community is very supportive of your effort here. New management tools
can also go directly into the main spdk repository (a separate repository was
only suggested when we thought this was a Python binding to the SPDK libraries).
I'm wondering if an easier way forward would be to continue refining the Python
packages in scripts/rpc to be more general purpose libraries for sending the
JSON RPCs. What are your thoughts on that?
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk



--
Zhipeng (Howard) Huang

Standard Engineer
IT Standard & Patent/IT Product Line
Huawei Technologies Co,. Ltd
Email: huangzhipeng(a)huawei.com<mailto:huangzhipeng(a)huawei.com>
Office: Huawei Industrial Base, Longgang, Shenzhen

(Previous)
Research Assistant
Mobile Ad-Hoc Network Lab, Calit2
University of California, Irvine
Email: zhipengh(a)uci.edu<mailto:zhipengh(a)uci.edu>
Office: Calit2 Building Room 2402

OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk

_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk


[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 34311 bytes --]

Re: [SPDK] Add py-spdk client for SPDK

[We We]

[-- Attachment #1: Type: text/plain, Size: 10089 bytes --]

Hi Paul,

Thank you for your reply and advice. 

First of all I whole-heartly agree with you that it is a better approach to take the scripts/rpc.py and scripts/rpc components and build those out/refactor them to provide more/better functionality. 

Secondly, I think you really hit on the target and the very essence of the benefit that a consistent data model protobuf could help bring. We are definitely in line on this one.

I would like to help to lead the effort which has been discussed thoroughly in this email thread, and work with any contributor in the SPDK community who is interested and willing to participate :)

Also look forward to my first community meeting this Thursday ! 

Thanks,
Helloway

> 在 2018年2月6日，上午3:20，Luse, Paul E <paul.e.luse(a)intel.com> 写道：
> 
> Hi WeWe,
>   <>
> Thanks for the additional insight.  On the py-spdk client yes I see the value that you are describing  however instead of adding on the py-spdk client a better approach might be take the scripts/rpc.py and scripts/rpc components and build those out/refactor them to provide more/better functionality.  There’s enough “likeness” between what you are proposing and what’s already there that I’m afraid a lot of the reluctance to adopt this is based on the increased maintenance of adding this new code weighed against the increased value.  If you improve on what’s there one piece at a time I think you can get a lot accomplished here.
>  
> Also, you should be aware that there’s some pending work to consolidate the SPDK target applications themselves, to have one target application with parameters instead of multiple binaries that do very similar work. I can see a good fit for your goals and that activity to work together to have a single python generic interface as well as a single binary underneath it both architected at the same time.  
>  
> Would you be interested in working with some other community folks on that? I’m not sure who it was that was talking about looking into that but hopefully they’ll see this and chime in J I think there might be a community meeting this Thu, if someone knows please reply – we could talk on that call if that would help.
>  
> Thanks
> Paul
>  
> PS: Or, you could work independently to get a generic python layer usable by applications “built into” the existing rpc/scripts code. A good place to start would be to pick one sample app, like nvmf target, and list out the functions that are currently provided via the existing rpc py code and see how you can add the ones that are in your patch by extending/re-organizing/refactor existing code. I would definitely start with just one application though and get buy-in from a maintainer that this is the right design choice.  Then, the rest of the work will go quite smoothly I think.
>  
>  
>  <>From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
> Sent: Monday, February 5, 2018 2:21 AM
> To: Storage Performance Development Kit <spdk(a)lists.01.org>
> Subject: Re: [SPDK] Add py-spdk client for SPDK
>  
> Hi Paul,
> I am trying to answer your questions from two points.
> Firstly, the py-spdk/nvmf_client.py provides the same abilities that can be done in the /scripts/rpc/nvmf.py, the difference, however, is that they are in the upper and lower layer respectively. Why should we do it? Imagine that there is only one management app which needs to invoke SPDK-base apps, we can write the module alone in your own management app and directly communicate with the directly /scripts/rpc/nvmf.py. However, when there are multiple upper management apps that need to invoke SPDK-base apps, we need to provide the unified software named ‘py-spdk’ and laid between the SPDK-based apps and upper management apps. If we don’t do so, then each management app needs to write the same module to interact with /scripts/rpc/nvmf.py. The py-spdk provides a generic interface for upper management apps. In other words, the upper management apps are not care about the functional realization of the backend, they just work through the interfaces provided by the py-spdk.
>  
> Secondly, why did we use the protobuf as data model? Because the protobuf provides a better standardized description of the API than JSON. From the code point of view, we only need to define a ‘.proto’file that describes the data and compiled it with different languages to the lib file (such as: spdk_pb2.py, spdk.go, etc.). Each ‘message’of the ‘.proto’file will be compiled to a protobuf obj which served as the carrier of the data and returned to the upper management apps. Compared to the JSON, the upper management apps can quickly learn that the returned value which is passed through the lib file rather than the manual document. From the ecological point of view，many mainstream cloud-native apps are using protobuf as their data model (like k8s)，which is also an opportunity for us.    
>  
> I don't know  do I make sense. 
>  
> Regards,
> Helloway
>  
> 在 2018年1月28日，上午5:25，Luse, Paul E <paul.e.luse(a)intel.com <mailto:paul.e.luse(a)intel.com>> 写道：
>  
> Hi Howard,
>  
> I think the main question right now is about the new core functionality in the patch versus what Ben mentioned wrt what is there today in scripts/rpc:
>  
> __init__.py        client.py             lvol.py                nvmf.py
> app.py                iscsi.py nbd.py  pmem.py
> bdev.py              log.py                 net.py                 vhost.py
>  
> This is the change he mentioned that happened since the original py-spdk client was added.  So for example if I look at what is in the patch in py-spdk/nvmf_client.py I don’t see any capabilities that can’t be done in thescripts/rpc/nvmf.py but I could be missing something. I think the protobuf and repo location discussions are secondary to first establishing the value of the core functionality of the patch to the community.
>  
> Can you explain how the SDK Framework layer might be a better approach as compared to working on the files listed above?
>  
> Thanks
> Paul
>  
> PS:  There is an open community conference call every other week, I think the next one is coming up here soon and its usually announced on IRC a few days in advance.  If real time discussion would help we could always look at scheduling a separate open meeting at time more friendly to your time zone.
>  
> From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of Zhipeng Huang
> Sent: Saturday, January 27, 2018 12:11 AM
> To: Storage Performance Development Kit <spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>>
> Subject: Re: [SPDK] Add py-spdk client for SPDK
>  
> Hi Ben,
>  
> After discussing with my team, the general feeling is that keeping protobuf would be a preferred choice. The reason, as i stated earlier, is that proto provides a better standardized description of the API than JSON and also we could create data model rather quickly for bindings/tools written in other languages. We have already generated the go-spdk based upon the proto model for OpenSDS's interaction with SPDK drivers. It provides benefits at least judging from our own practice.
>  
> Frankly given gRPC's wide adoption I don't think this should be a big issue. If you still have doubt about this, I think maybe we could setup a conf call, or I could discuss with Harris when he's in China for SPDK summit.
>  
> For repo, I think it is entirely up to the community's decision on whether maintain it in the main repo or creating a new one.
>  
> On Sat, Jan 27, 2018 at 6:28 AM, Walker, Benjamin <benjamin.walker(a)intel.com <mailto:benjamin.walker(a)intel.com>> wrote:
> On Wed, 2018-01-24 at 07:13 +0800, Zhipeng Huang wrote:
> > Do we have a conclusion on this issue ? If it is ok to have a spdk/sdk repo,
> > then wewe will modify the current patch (get rid of protobuff) and resubmit
> > the patch to the new repo once it is established (meanwhile abandon the
> > current one to spdk/spdk).
> 
> If you remove protobuf, can you describe what is left? Recently scripts/rpc.py
> was refactored to break it up into a set of Python libraries in scripts/rpc,
> plus the command line tool at scripts/rpc.py. What functionality does this new
> code provide over and above what is already present there?
> 
> SPDK is certainly in need of better management tools, so in the most general
> sense the community is very supportive of your effort here. New management tools
> can also go directly into the main spdk repository (a separate repository was
> only suggested when we thought this was a Python binding to the SPDK libraries).
> I'm wondering if an easier way forward would be to continue refining the Python
> packages in scripts/rpc to be more general purpose libraries for sending the
> JSON RPCs. What are your thoughts on that?
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
> 
> 
>  
> -- 
> Zhipeng (Howard) Huang
>  
> Standard Engineer
> IT Standard & Patent/IT Product Line
> Huawei Technologies Co,. Ltd
> Email: huangzhipeng(a)huawei.com <mailto:huangzhipeng(a)huawei.com>
> Office: Huawei Industrial Base, Longgang, Shenzhen
>  
> (Previous)
> Research Assistant
> Mobile Ad-Hoc Network Lab, Calit2
> University of California, Irvine
> Email: zhipengh(a)uci.edu <mailto:zhipengh(a)uci.edu>
> Office: Calit2 Building Room 2402
>  
> OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
>  
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk


[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 37330 bytes --]

Re: [SPDK] Add py-spdk client for SPDK

[Luse, Paul E]

[-- Attachment #1: Type: text/plain, Size: 8728 bytes --]

Hi WeWe,

Thanks for the additional insight.  On the py-spdk client yes I see the value that you are describing  however instead of adding on the py-spdk client a better approach might be take the scripts/rpc.py and scripts/rpc components and build those out/refactor them to provide more/better functionality.  There’s enough “likeness” between what you are proposing and what’s already there that I’m afraid a lot of the reluctance to adopt this is based on the increased maintenance of adding this new code weighed against the increased value.  If you improve on what’s there one piece at a time I think you can get a lot accomplished here.

Also, you should be aware that there’s some pending work to consolidate the SPDK target applications themselves, to have one target application with parameters instead of multiple binaries that do very similar work. I can see a good fit for your goals and that activity to work together to have a single python generic interface as well as a single binary underneath it both architected at the same time.

Would you be interested in working with some other community folks on that? I’m not sure who it was that was talking about looking into that but hopefully they’ll see this and chime in :) I think there might be a community meeting this Thu, if someone knows please reply - we could talk on that call if that would help.

Thanks
Paul

PS: Or, you could work independently to get a generic python layer usable by applications “built into” the existing rpc/scripts code. A good place to start would be to pick one sample app, like nvmf target, and list out the functions that are currently provided via the existing rpc py code and see how you can add the ones that are in your patch by extending/re-organizing/refactor existing code. I would definitely start with just one application though and get buy-in from a maintainer that this is the right design choice.  Then, the rest of the work will go quite smoothly I think.


From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Monday, February 5, 2018 2:21 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: Re: [SPDK] Add py-spdk client for SPDK

Hi Paul,
I am trying to answer your questions from two points.
Firstly, the py-spdk/nvmf_client.py provides the same abilities that can be done in the /scripts/rpc/nvmf.py, the difference, however, is that they are in the upper and lower layer respectively. Why should we do it? Imagine that there is only one management app which needs to invoke SPDK-base apps, we can write the module alone in your own management app and directly communicate with the directly /scripts/rpc/nvmf.py. However, when there are multiple upper management apps that need to invoke SPDK-base apps, we need to provide the unified software named ‘py-spdk’ and laid between the SPDK-based apps and upper management apps. If we don’t do so, then each management app needs to write the same module to interact with /scripts/rpc/nvmf.py. The py-spdk provides a generic interface for upper management apps. In other words, the upper management apps are not care about the functional realization of the backend, they just work through the interfaces provided by the py-spdk.

Secondly, why did we use the protobuf as data model? Because the protobuf provides a better standardized description of the API than JSON. From the code point of view, we only need to define a ‘.proto’file that describes the data and compiled it with different languages to the lib file (such as: spdk_pb2.py, spdk.go, etc.). Each ‘message’of the ‘.proto’file will be compiled to a protobuf obj which served as the carrier of the data and returned to the upper management apps. Compared to the JSON, the upper management apps can quickly learn that the returned value which is passed through the lib file rather than the manual document. From the ecological point of view，many mainstream cloud-native apps are using protobuf as their data model (like k8s)，which is also an opportunity for us.

I don't know  do I make sense.

Regards,
Helloway

在 2018年1月28日，上午5:25，Luse, Paul E <paul.e.luse(a)intel.com<mailto:paul.e.luse(a)intel.com>> 写道：

Hi Howard,

I think the main question right now is about the new core functionality in the patch versus what Ben mentioned wrt what is there today in scripts/rpc:

__init__.py        client.py             lvol.py                nvmf.py
app.py                iscsi.py nbd.py  pmem.py
bdev.py              log.py                 net.py                 vhost.py

This is the change he mentioned that happened since the original py-spdk client was added.  So for example if I look at what is in the patch in py-spdk/nvmf_client.py I don’t see any capabilities that can’t be done in thescripts/rpc/nvmf.py but I could be missing something. I think the protobuf and repo location discussions are secondary to first establishing the value of the core functionality of the patch to the community.

Can you explain how the SDK Framework layer might be a better approach as compared to working on the files listed above?

Thanks
Paul

PS:  There is an open community conference call every other week, I think the next one is coming up here soon and its usually announced on IRC a few days in advance.  If real time discussion would help we could always look at scheduling a separate open meeting at time more friendly to your time zone.

From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Zhipeng Huang
Sent: Saturday, January 27, 2018 12:11 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] Add py-spdk client for SPDK

Hi Ben,

After discussing with my team, the general feeling is that keeping protobuf would be a preferred choice. The reason, as i stated earlier, is that proto provides a better standardized description of the API than JSON and also we could create data model rather quickly for bindings/tools written in other languages. We have already generated the go-spdk based upon the proto model for OpenSDS's interaction with SPDK drivers. It provides benefits at least judging from our own practice.

Frankly given gRPC's wide adoption I don't think this should be a big issue. If you still have doubt about this, I think maybe we could setup a conf call, or I could discuss with Harris when he's in China for SPDK summit.

For repo, I think it is entirely up to the community's decision on whether maintain it in the main repo or creating a new one.

On Sat, Jan 27, 2018 at 6:28 AM, Walker, Benjamin <benjamin.walker(a)intel.com<mailto:benjamin.walker(a)intel.com>> wrote:
On Wed, 2018-01-24 at 07:13 +0800, Zhipeng Huang wrote:
> Do we have a conclusion on this issue ? If it is ok to have a spdk/sdk repo,
> then wewe will modify the current patch (get rid of protobuff) and resubmit
> the patch to the new repo once it is established (meanwhile abandon the
> current one to spdk/spdk).

If you remove protobuf, can you describe what is left? Recently scripts/rpc.py
was refactored to break it up into a set of Python libraries in scripts/rpc,
plus the command line tool at scripts/rpc.py. What functionality does this new
code provide over and above what is already present there?

SPDK is certainly in need of better management tools, so in the most general
sense the community is very supportive of your effort here. New management tools
can also go directly into the main spdk repository (a separate repository was
only suggested when we thought this was a Python binding to the SPDK libraries).
I'm wondering if an easier way forward would be to continue refining the Python
packages in scripts/rpc to be more general purpose libraries for sending the
JSON RPCs. What are your thoughts on that?
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk



--
Zhipeng (Howard) Huang

Standard Engineer
IT Standard & Patent/IT Product Line
Huawei Technologies Co,. Ltd
Email: huangzhipeng(a)huawei.com<mailto:huangzhipeng(a)huawei.com>
Office: Huawei Industrial Base, Longgang, Shenzhen

(Previous)
Research Assistant
Mobile Ad-Hoc Network Lab, Calit2
University of California, Irvine
Email: zhipengh(a)uci.edu<mailto:zhipengh(a)uci.edu>
Office: Calit2 Building Room 2402

OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk


[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 25777 bytes --]

Re: [SPDK] Add py-spdk client for SPDK

[We We]

[-- Attachment #1: Type: text/plain, Size: 6653 bytes --]

Hi Paul,

I am trying to answer your questions from two points.

Firstly, the py-spdk/nvmf_client.py provides the same abilities that can be done in the /scripts/rpc/nvmf.py, the difference, however, is that they are in the upper and lower layer respectively. Why should we do it? Imagine that there is only one management app which needs to invoke SPDK-base apps, we can write the module alone in your own management app and directly communicate with the directly /scripts/rpc/nvmf.py. However, when there are multiple upper management apps that need to invoke SPDK-base apps, we need to provide the unified software named ‘py-spdk’ and laid between the SPDK-based apps and upper management apps. If we don’t do so, then each management app needs to write the same module to interact with /scripts/rpc/nvmf.py. The py-spdk provides a generic interface for upper management apps. In other words, the upper management apps are not care about the functional realization of the backend, they just work through the interfaces provided by the py-spdk.

 

Secondly, why did we use the protobuf as data model? Because the protobuf provides a better standardized description of the API than JSON. From the code point of view, we only need to define a ‘.proto’file that describes the data and compiled it with different languages to the lib file (such as: spdk_pb2.py, spdk.go, etc.). Each ‘message’of the ‘.proto’file will be compiled to a protobuf obj which served as the carrier of the data and returned to the upper management apps. Compared to the JSON, the upper management apps can quickly learn that the returned value which is passed through the lib file rather than the manual document. From the ecological point of view，many mainstream cloud-native apps are using protobuf as their data model (like k8s)，which is also an opportunity for us.    

I don't know  do I make sense. 

Regards,
Helloway

> 在 2018年1月28日，上午5:25，Luse, Paul E <paul.e.luse(a)intel.com> 写道：
> 
> Hi Howard,
>   <>
> I think the main question right now is about the new core functionality in the patch versus what Ben mentioned wrt what is there today in scripts/rpc:
>  
> __init__.py        client.py             lvol.py                nvmf.py
> app.py                iscsi.py nbd.py  pmem.py
> bdev.py              log.py                 net.py                 vhost.py
>  
> This is the change he mentioned that happened since the original py-spdk client was added.  So for example if I look at what is in the patch in py-spdk/nvmf_client.py I don’t see any capabilities that can’t be done in thescripts/rpc/nvmf.py but I could be missing something. I think the protobuf and repo location discussions are secondary to first establishing the value of the core functionality of the patch to the community.
>  
> Can you explain how the SDK Framework layer might be a better approach as compared to working on the files listed above?
>  
> Thanks
> Paul
>  
> PS:  There is an open community conference call every other week, I think the next one is coming up here soon and its usually announced on IRC a few days in advance.  If real time discussion would help we could always look at scheduling a separate open meeting at time more friendly to your time zone.
>  
>  <>From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Zhipeng Huang
> Sent: Saturday, January 27, 2018 12:11 AM
> To: Storage Performance Development Kit <spdk(a)lists.01.org>
> Subject: Re: [SPDK] Add py-spdk client for SPDK
>  
> Hi Ben,
>  
> After discussing with my team, the general feeling is that keeping protobuf would be a preferred choice. The reason, as i stated earlier, is that proto provides a better standardized description of the API than JSON and also we could create data model rather quickly for bindings/tools written in other languages. We have already generated the go-spdk based upon the proto model for OpenSDS's interaction with SPDK drivers. It provides benefits at least judging from our own practice.
>  
> Frankly given gRPC's wide adoption I don't think this should be a big issue. If you still have doubt about this, I think maybe we could setup a conf call, or I could discuss with Harris when he's in China for SPDK summit.
>  
> For repo, I think it is entirely up to the community's decision on whether maintain it in the main repo or creating a new one.
>  
> On Sat, Jan 27, 2018 at 6:28 AM, Walker, Benjamin <benjamin.walker(a)intel.com <mailto:benjamin.walker(a)intel.com>> wrote:
> On Wed, 2018-01-24 at 07:13 +0800, Zhipeng Huang wrote:
> > Do we have a conclusion on this issue ? If it is ok to have a spdk/sdk repo,
> > then wewe will modify the current patch (get rid of protobuff) and resubmit
> > the patch to the new repo once it is established (meanwhile abandon the
> > current one to spdk/spdk).
> 
> If you remove protobuf, can you describe what is left? Recently scripts/rpc.py
> was refactored to break it up into a set of Python libraries in scripts/rpc,
> plus the command line tool at scripts/rpc.py. What functionality does this new
> code provide over and above what is already present there?
> 
> SPDK is certainly in need of better management tools, so in the most general
> sense the community is very supportive of your effort here. New management tools
> can also go directly into the main spdk repository (a separate repository was
> only suggested when we thought this was a Python binding to the SPDK libraries).
> I'm wondering if an easier way forward would be to continue refining the Python
> packages in scripts/rpc to be more general purpose libraries for sending the
> JSON RPCs. What are your thoughts on that?
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
> 
> 
>  
> -- 
> Zhipeng (Howard) Huang
>  
> Standard Engineer
> IT Standard & Patent/IT Product Line
> Huawei Technologies Co,. Ltd
> Email: huangzhipeng(a)huawei.com <mailto:huangzhipeng(a)huawei.com>
> Office: Huawei Industrial Base, Longgang, Shenzhen
>  
> (Previous)
> Research Assistant
> Mobile Ad-Hoc Network Lab, Calit2
> University of California, Irvine
> Email: zhipengh(a)uci.edu <mailto:zhipengh(a)uci.edu>
> Office: Calit2 Building Room 2402
>  
> OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk


[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 58342 bytes --]

Re: [SPDK] Add py-spdk client for SPDK

[Luse, Paul E]

[-- Attachment #1: Type: text/plain, Size: 4262 bytes --]

Hi Howard,

I think the main question right now is about the new core functionality in the patch versus what Ben mentioned wrt what is there today in scripts/rpc:

__init__.py        client.py             lvol.py                nvmf.py
app.py                iscsi.py nbd.py  pmem.py
bdev.py              log.py                 net.py                 vhost.py

This is the change he mentioned that happened since the original py-spdk client was added.  So for example if I look at what is in the patch in py-spdk/nvmf_client.py I don’t see any capabilities that can’t be done in the scripts/rpc/nvmf.py but I could be missing something. I think the protobuf and repo location discussions are secondary to first establishing the value of the core functionality of the patch to the community.

Can you explain how the SDK Framework layer might be a better approach as compared to working on the files listed above?

Thanks
Paul

PS:  There is an open community conference call every other week, I think the next one is coming up here soon and its usually announced on IRC a few days in advance.  If real time discussion would help we could always look at scheduling a separate open meeting at time more friendly to your time zone.

From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Zhipeng Huang
Sent: Saturday, January 27, 2018 12:11 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: Re: [SPDK] Add py-spdk client for SPDK

Hi Ben,

After discussing with my team, the general feeling is that keeping protobuf would be a preferred choice. The reason, as i stated earlier, is that proto provides a better standardized description of the API than JSON and also we could create data model rather quickly for bindings/tools written in other languages. We have already generated the go-spdk based upon the proto model for OpenSDS's interaction with SPDK drivers. It provides benefits at least judging from our own practice.

Frankly given gRPC's wide adoption I don't think this should be a big issue. If you still have doubt about this, I think maybe we could setup a conf call, or I could discuss with Harris when he's in China for SPDK summit.

For repo, I think it is entirely up to the community's decision on whether maintain it in the main repo or creating a new one.

On Sat, Jan 27, 2018 at 6:28 AM, Walker, Benjamin <benjamin.walker(a)intel.com<mailto:benjamin.walker(a)intel.com>> wrote:
On Wed, 2018-01-24 at 07:13 +0800, Zhipeng Huang wrote:
> Do we have a conclusion on this issue ? If it is ok to have a spdk/sdk repo,
> then wewe will modify the current patch (get rid of protobuff) and resubmit
> the patch to the new repo once it is established (meanwhile abandon the
> current one to spdk/spdk).

If you remove protobuf, can you describe what is left? Recently scripts/rpc.py
was refactored to break it up into a set of Python libraries in scripts/rpc,
plus the command line tool at scripts/rpc.py. What functionality does this new
code provide over and above what is already present there?

SPDK is certainly in need of better management tools, so in the most general
sense the community is very supportive of your effort here. New management tools
can also go directly into the main spdk repository (a separate repository was
only suggested when we thought this was a Python binding to the SPDK libraries).
I'm wondering if an easier way forward would be to continue refining the Python
packages in scripts/rpc to be more general purpose libraries for sending the
JSON RPCs. What are your thoughts on that?
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk



--
Zhipeng (Howard) Huang

Standard Engineer
IT Standard & Patent/IT Product Line
Huawei Technologies Co,. Ltd
Email: huangzhipeng(a)huawei.com<mailto:huangzhipeng(a)huawei.com>
Office: Huawei Industrial Base, Longgang, Shenzhen

(Previous)
Research Assistant
Mobile Ad-Hoc Network Lab, Calit2
University of California, Irvine
Email: zhipengh(a)uci.edu<mailto:zhipengh(a)uci.edu>
Office: Calit2 Building Room 2402

OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 11081 bytes --]

Re: [SPDK] Add py-spdk client for SPDK

[Zhipeng Huang]

[-- Attachment #1: Type: text/plain, Size: 2747 bytes --]

Hi Ben,

After discussing with my team, the general feeling is that keeping protobuf
would be a preferred choice. The reason, as i stated earlier, is that proto
provides a better standardized description of the API than JSON and also we
could create data model rather quickly for bindings/tools written in other
languages. We have already generated the go-spdk based upon the proto model
for OpenSDS's interaction with SPDK drivers. It provides benefits at least
judging from our own practice.

Frankly given gRPC's wide adoption I don't think this should be a big
issue. If you still have doubt about this, I think maybe we could setup a
conf call, or I could discuss with Harris when he's in China for SPDK
summit.

For repo, I think it is entirely up to the community's decision on whether
maintain it in the main repo or creating a new one.

On Sat, Jan 27, 2018 at 6:28 AM, Walker, Benjamin <benjamin.walker(a)intel.com
> wrote:

> On Wed, 2018-01-24 at 07:13 +0800, Zhipeng Huang wrote:
> > Do we have a conclusion on this issue ? If it is ok to have a spdk/sdk
> repo,
> > then wewe will modify the current patch (get rid of protobuff) and
> resubmit
> > the patch to the new repo once it is established (meanwhile abandon the
> > current one to spdk/spdk).
>
> If you remove protobuf, can you describe what is left? Recently
> scripts/rpc.py
> was refactored to break it up into a set of Python libraries in
> scripts/rpc,
> plus the command line tool at scripts/rpc.py. What functionality does this
> new
> code provide over and above what is already present there?
>
> SPDK is certainly in need of better management tools, so in the most
> general
> sense the community is very supportive of your effort here. New management
> tools
> can also go directly into the main spdk repository (a separate repository
> was
> only suggested when we thought this was a Python binding to the SPDK
> libraries).
> I'm wondering if an easier way forward would be to continue refining the
> Python
> packages in scripts/rpc to be more general purpose libraries for sending
> the
> JSON RPCs. What are your thoughts on that?
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>



-- 
Zhipeng (Howard) Huang

Standard Engineer
IT Standard & Patent/IT Product Line
Huawei Technologies Co,. Ltd
Email: huangzhipeng(a)huawei.com
Office: Huawei Industrial Base, Longgang, Shenzhen

(Previous)
Research Assistant
Mobile Ad-Hoc Network Lab, Calit2
University of California, Irvine
Email: zhipengh(a)uci.edu
Office: Calit2 Building Room 2402

OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 3872 bytes --]

Re: [SPDK] Add py-spdk client for SPDK

[Walker, Benjamin]

[-- Attachment #1: Type: text/plain, Size: 1145 bytes --]

On Wed, 2018-01-24 at 07:13 +0800, Zhipeng Huang wrote:
> Do we have a conclusion on this issue ? If it is ok to have a spdk/sdk repo,
> then wewe will modify the current patch (get rid of protobuff) and resubmit
> the patch to the new repo once it is established (meanwhile abandon the
> current one to spdk/spdk).

If you remove protobuf, can you describe what is left? Recently scripts/rpc.py
was refactored to break it up into a set of Python libraries in scripts/rpc,
plus the command line tool at scripts/rpc.py. What functionality does this new
code provide over and above what is already present there?

SPDK is certainly in need of better management tools, so in the most general
sense the community is very supportive of your effort here. New management tools
can also go directly into the main spdk repository (a separate repository was
only suggested when we thought this was a Python binding to the SPDK libraries).
I'm wondering if an easier way forward would be to continue refining the Python
packages in scripts/rpc to be more general purpose libraries for sending the
JSON RPCs. What are your thoughts on that?

Re: [SPDK] Add py-spdk client for SPDK

[Zhipeng Huang]

[-- Attachment #1: Type: text/plain, Size: 18196 bytes --]

Hi guys,

Do we have a conclusion on this issue ? If it is ok to have a spdk/sdk
repo, then wewe will modify the current patch (get rid of protobuff) and
resubmit the patch to the new repo once it is established (meanwhile
abandon the current one to spdk/spdk).

On Fri, Jan 19, 2018 at 6:57 AM, Zhipeng Huang <zhipengh512(a)gmail.com>
wrote:

> Hi Ben,
>
> You are absolutely right on the first point.
>
> On the second point, the main reason for us is that protobuf could auto
> generate the data model for any interraces. Let's say we want to write a go
> tool, we could just write the API similar to py-spdk, and the protobuf will
> auto generate the msg bodies that will be exchanged over the API.
>
> Protobuf is purely from an easy to use perspective, if you think it adds
> too much complexity then we could just use plain JSON RPC :)
>
> On Jan 19, 2018 12:58 AM, "Walker, Benjamin" <benjamin.walker(a)intel.com>
> wrote:
>
>> I'm still not at all clear on either point, so please allow me to ask a
>> few more quesrions.
>>
>> These python scripts interact with the SPDK applications through JSON
>> RPC, not through direct calls, correct? If that is the case, let's call
>> this a management tool or management library and not use the term binding
>> from here on out. I think that will go a long way toward helping everyone
>> understand the role this code is playing.
>>
>> As far as using protobuf, most languages have facilities to send json
>> rpcs natively. This is part of the standard library in go, for example.
>> Protobufs are great for generating high performance parsers of binary
>> protocols in multiple languages, but for a management interface I think
>> JSON RPC is a better choice. It is plain text and much more flexible. Is
>> there any reason you can't just send the JSON RPCs from your go application
>> directly? Surely this can't be any harder than sending the protobuf
>> messages.
>>
>> I hope I'm understanding.
>>
>> Thanks,
>> Ben
>>
>>
>> -------- Original message --------
>> From: Zhipeng Huang <zhipengh512(a)gmail.com>
>> Date: 1/18/18 8:06 AM (GMT-08:00)
>> To: Storage Performance Development Kit <spdk(a)lists.01.org>
>> Subject: Re: [SPDK] Add py-spdk client for SPDK
>>
>> Hi Ben,
>>
>> I could help hellowway answer a few questions.
>>
>> First of all, for py-spdk we are shooting for a SDK framework for SPDK so
>> that if someone develop a userspace storage app based upon SPDK, he could
>> quickly develop the SDK for his app based upon py-spdk. You are right that
>> typically when we say binding we mean a direct integration, however for
>> py-spdk we also want it to be a client that could remotely interact with
>> the management plane. So in essence py-spdk could be used as a python
>> binding for SPDK as well as a simple python client.
>>
>> Second on the use of protobuf, is that we see possibilities of having
>> different lang bindings for SPDK. Using protobuf is a great way to define a
>> common data model and reuse it in these different lang bindings.
>>
>> For example we have an open source storage mgmt project called OpenSDS
>> that might use SPDK for NVMe Over Fabric JBOFs. OpenSDS is written in
>> golang therefore we could quickly build a go-spdk based upon the protobuf
>> data model.
>>
>> Hope I make things clearer :)
>>
>> On Thu, Jan 18, 2018 at 11:49 PM, Walker, Benjamin <
>> benjamin.walker(a)intel.com> wrote:
>>
>>> Hi Helloway,
>>>
>>> I'll admit up front that I haven't read through your code yet, but I
>>> have read your trello document. I have a couple of very basic questions.
>>>
>>> First, usually when people say "binding" they mean a way to call a
>>> library from a different language. A python binding to SPDK would mean a
>>> wrapper around the C libraries so that they could be called from python
>>> directly, with no interprocess communication. My understanding of what
>>> you're writing is that it is not a binding, but rather a management tool
>>> for the SPDK example target applications. Am I understanding this correctly?
>>>
>>> Second, SPDK exposes JSON RPC interfaces for management of the target
>>> applications. These RPCs can be sent from any language and we already have
>>> some python scripts that do just that.  So why do you need protobufs? What
>>> new value does your code add above what we have in scripts/rpc.py already?
>>> Is it just more fully featured?
>>>
>>> Thanks in advance for helping me understand.
>>>
>>>
>>> -------- Original message --------
>>> From: We We <simple_hlw(a)163.com>
>>> Date: 1/18/18 8:12 AM (GMT-07:00)
>>> To: Storage Performance Development Kit <spdk(a)lists.01.org>
>>> Subject: Re: [SPDK] Add py-spdk client for SPDK
>>>
>>> Hi Maciek,
>>>
>>> Thanks!! I am very happy to receive your kind feedback.
>>> In my opinion,  I indeed want to put the py-spdk into the spdk/py-spdk
>>> repo rathan than spdk/spdk repo. Because I think the py-spdk is built upon
>>> the SPDK which can provide a general python binding for spdk framework. I
>>> don't know whether I keep consistent with you. Please let me know what you
>>> think. And is it convenient for you to tell me what you have done with SPDK
>>> and Cinder?
>>> Any thoughts are appreciated.
>>>
>>> Regards,
>>> Helloway
>>>
>>>
>>>
>>> 在 2018年1月18日，下午10:22，Szwed, Maciej <maciej.szwed(a)intel.com> 写道：
>>>
>>> Hello WeWe,
>>> I’ve looked closer at your patch and it looks good (some minor tweaks
>>> maybe, some error handling missing), but before I comment it on Gerrit we
>>> have make decision where we want to keep it. I would rather see it outside
>>> spdk/spdk repo. There is some work ongoing to connect SPDK with Cinder. I
>>> think that these two could end up in same repository.
>>>
>>> Regards,
>>> Maciek
>>>
>>> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org
>>> <spdk-bounces(a)lists.01.org>] *On Behalf Of *Zhipeng Huang
>>> *Sent:* Thursday, January 18, 2018 3:01 AM
>>> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
>>> *Subject:* Re: [SPDK] Add py-spdk client for SPDK
>>>
>>> Thanks and Wewe is working on to fix that in the next patch :)
>>>
>>> As she stated earlier, we think it is better to have a standalone repo
>>> hosting all the different lang bindings for spdk under
>>> https://github.com/spdk , similar example could be found at openstack
>>> sdk, so that we don't blow up the main spdk repo
>>>
>>> On Thu, Jan 18, 2018 at 9:49 AM, Luse, Paul E <paul.e.luse(a)intel.com>
>>> wrote:
>>>
>>> FYI there are other format issues as well, if you look at some of the
>>> other test machines you’ll see this, for example, on the patch:
>>>
>>> Checking coding style... OK
>>> Checking comment style... OK
>>> Checking blank lines at end of file... Incorrect end-of-file formatting
>>> detected
>>> py-spdk/README.md: Extra trailing newline
>>> Checking for POSIX includes... OK
>>> Checking Python style... OK
>>>
>>>
>>> All very easy to catch & fix locally J
>>>
>>> Thx
>>> Paul
>>>
>>> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org] *On Behalf Of *Zhipeng
>>> Huang
>>> *Sent:* Wednesday, January 17, 2018 6:33 PM
>>>
>>> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
>>> *Subject:* Re: [SPDK] Add py-spdk client for SPDK
>>>
>>> Hi Paul,
>>>
>>> Thanks for getting the CI running. Seems most of the problems caused by
>>> pep8 missing. Is it possible to add python support in the CI system ?
>>> otherwise we will hit many pip pkg missing problems.
>>>
>>> On Thu, Jan 18, 2018 at 8:17 AM, Luse, Paul E <paul.e.luse(a)intel.com>
>>> wrote:
>>>
>>> Hmm yeah I’m not sure why it ran on patch set 2 but not the last couple…
>>> Seth, anyone?
>>>
>>> We might need one of the maintainers to clear it for CI execution, it’s
>>> an extra unfortunate security requirement we have on the test pool,
>>> otherwise its fully automated
>>>
>>> Thx
>>> Paul
>>>
>>> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org] *On Behalf Of *Zhipeng
>>> Huang
>>> *Sent:* Wednesday, January 17, 2018 4:04 PM
>>>
>>> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
>>> *Subject:* Re: [SPDK] Add py-spdk client for SPDK
>>>
>>> Paul, the problem is that Wewe's later patches doesn't trigger CI at
>>> all. Is it how it supposed to work that we should fix the problem before
>>> hand ?
>>> Since in OpenStack or kubernetes CI should be triggered everytime the
>>> patches come in.
>>>
>>> On Thu, Jan 18, 2018 at 5:39 AM, Luse, Paul E <paul.e.luse(a)intel.com>
>>> wrote:
>>>
>>> Hi WeWe
>>>
>>>
>>> So wrt which repo, I didn’t see that but we don’t really have a
>>> precedent for subprojects like that.  I’ll let one of the maintainers
>>> comment on the options but I believe they would be (a) maintain on your own
>>> elsewhere or (b) include in SPDK like any other code.  Option (b) is really
>>> for “core” SPDK elements that make sense for most use cases and a python
>>> binding may or may not fit, it clearly isn’t out of the question in my mind
>>> though as it’s not directly tied to OpenStack.
>>>
>>> On your patch not passing, see the screenshot below where I highlighted
>>> the -1 from the CI system.  If you click on that link it will take you to
>>> the results and you can see the failure I mention.  Have you tried running
>>> the check_format.sh locally?
>>>
>>> Thx
>>> Paul
>>>
>>>
>>> <image001.png>
>>>
>>> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org] *On Behalf Of *We We
>>> *Sent:* Wednesday, January 17, 2018 10:26 AM
>>>
>>> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
>>> *Subject:* Re: [SPDK] Add py-spdk client for SPDK
>>>
>>> Hi, Paul
>>> I am sorry.  Is  the question on my patch you mentioned  the following
>>> Q1?
>>> Q1: is your intention to propose this as part of the SPDK repo or do you
>>> intent to maintain this in your own repo and are simply asking for review
>>> comments?
>>> If so, I see it and response to you on the patch.  Maybe you didn’t see
>>> it.
>>>
>>> 1. About Q1, my idea is to treat the py-spdk as another subrepo in the
>>> SPDK community. For example: SPDK repo -->https://github.com/spdk/spd
>>> k.git, py-spdk repo -->https://github.com/spdk/py-spdk.git. Am I
>>> reasonable？
>>>
>>>
>>> 2. About a copy of rpc.py. Thank you for your careful review, the
>>> py-spdk doesn’t need to modify and copy the files of SPDK, I have deleted
>>> the rpc.py and updated the new patch. Please visit it.
>>>
>>>
>>> 3. About my patch didn’t pass CI.  My patch didn’t show nothing
>>> including the “verified -1”. I didn’t know whether the CI executed, so
>>> I can’t track anything down.  Is there anyone who met this problem?
>>>
>>> Regards,
>>> Helloway
>>>
>>>
>>>
>>>
>>> 在 2018年1月17日，下午10:32，Luse, Paul E <paul.e.luse(a)intel.com> 写道：
>>>
>>> Hi WeWe,
>>>
>>> Thanks for the trello update, did you see my question on your patch? I
>>> went to look specifically at the changes to existing files to start with
>>> and it appears that your patch includes a copy of rpc.py basically
>>> duplicating it within the repo.  Did I miss something or is that what you
>>> intended?
>>>
>>> Wrt  question on why your patch didn’t pass CI, it’s pretty easy to
>>> track these things down most of the time.  In the review, click on the link
>>> below where it says verified -1 and you will see a webpage with the status
>>> of each of the test machines.  Many of the are red and indicate failure.
>>> Click on one and drill down to find the builg.log file and you’ll see some
>>> formatting issues:
>>>
>>> ========== Backtrace start: ==========
>>>
>>> in ./autobuild.sh:23 -> main()
>>>      ...
>>>    18
>>>    19  ./configure $config_params
>>>    20
>>>    21  timing_enter check_format
>>>    22  if [ $SPDK_RUN_CHECK_FORMAT -eq 1 ]; then
>>> => 23          ./scripts/check_format.sh
>>>    24  fi
>>>    25  timing_exit check_format
>>>    26
>>>    27  timing_enter build_kmod
>>>    28  if [ $SPDK_BUILD_IOAT_KMOD -eq 1 ]; then
>>>      ...
>>>
>>> ========== Backtrace end ==========
>>>
>>> You can run the script, ./scripts/check_format.sh, locally and correct
>>> the errors and resubmit.  We always ask that developers run this scripts in
>>> advance to make sure things are formatted.  Depending on whether the
>>> maintainers think the SPDK repo makes sense for this patch you’ll need some
>>> test code also – there is clearly a lot of value in this patch though and
>>> regardless of whether it lands here or is maintained elsewhere!
>>>
>>> Use this list or the patch review if you have more questions on getting
>>> it to pass.
>>>
>>> Thanks!
>>> Paul
>>>
>>>
>>>
>>> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org
>>> <spdk-bounces(a)lists.01.org>] *On Behalf Of *We We
>>> *Sent:* Wednesday, January 17, 2018 5:24 AM
>>> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
>>> *Subject:* Re: [SPDK] Add py-spdk client for SPDK
>>>
>>> Hi, all
>>> I have submitted the introduction of py-spdk on trello
>>> https://trello.com/c/Bn3DIG7F/87-add-py-spdk-client-for-spdk, please
>>> visit it. I will be grateful to your kindness.
>>>
>>> Regards,
>>>
>>> Helloway
>>>
>>>
>>> 在 2018年1月16日，上午11:30，We We <simple_hlw(a)163.com> 写道：
>>>
>>> Hi, all
>>> I have submitted the py-spdk(written in python) for more than six days.
>>> Why the code-review couldn’t pass and show nothing?
>>> Is there anyone who can help me?
>>>
>>> Thanks,
>>> Helloway
>>>
>>>
>>> 在 2018年1月10日，下午9:46，We We <simple_hlw(a)163.com> 写道：
>>>
>>> Hi, all
>>> I have submitted the py-spdk code onhttps://review.gerrithub.io/
>>> #/c/379741/, please take some time to visit it, I will be very grateful
>>> to you.
>>> The py-spdk is client which can help the upper-level app to communicate
>>> with the SPDK-based app (such as: nvmf_tgt, vhost, iscsi_tgt, etc.). Should
>>> I submit it into the other repo I rebuild rather than SPDK repo? Because I
>>> think it is a relatively independent kit upon the SPDK.
>>>
>>> If you have some thoughts about the py-spdk, please share with me.
>>>
>>> Regards,
>>> Helloway
>>> _______________________________________________
>>> SPDK mailing list
>>> SPDK(a)lists.01.org
>>> https://lists.01.org/mailman/listinfo/spdk
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> SPDK mailing list
>>> SPDK(a)lists.01.org
>>> https://lists.01.org/mailman/listinfo/spdk
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> SPDK mailing list
>>> SPDK(a)lists.01.org
>>> https://lists.01.org/mailman/listinfo/spdk
>>>
>>>
>>>
>>>
>>> --
>>> Zhipeng (Howard) Huang
>>>
>>> Standard Engineer
>>> IT Standard & Patent/IT Product Line
>>> Huawei Technologies Co,. Ltd
>>> Email: huangzhipeng(a)huawei.com
>>> Office: Huawei Industrial Base, Longgang, Shenzhen
>>>
>>> (Previous)
>>> Research Assistant
>>> Mobile Ad-Hoc Network Lab, Calit2
>>> University of California, Irvine
>>> Email: zhipengh(a)uci.edu
>>> Office: Calit2 Building Room 2402
>>>
>>> OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado
>>>
>>>
>>> _______________________________________________
>>> SPDK mailing list
>>> SPDK(a)lists.01.org
>>> https://lists.01.org/mailman/listinfo/spdk
>>>
>>>
>>>
>>>
>>> --
>>> Zhipeng (Howard) Huang
>>>
>>> Standard Engineer
>>> IT Standard & Patent/IT Product Line
>>> Huawei Technologies Co,. Ltd
>>> Email: huangzhipeng(a)huawei.com
>>> Office: Huawei Industrial Base, Longgang, Shenzhen
>>>
>>> (Previous)
>>> Research Assistant
>>> Mobile Ad-Hoc Network Lab, Calit2
>>> University of California, Irvine
>>> Email: zhipengh(a)uci.edu
>>> Office: Calit2 Building Room 2402
>>>
>>> OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado
>>>
>>>
>>> _______________________________________________
>>> SPDK mailing list
>>> SPDK(a)lists.01.org
>>> https://lists.01.org/mailman/listinfo/spdk
>>>
>>>
>>>
>>>
>>> --
>>> Zhipeng (Howard) Huang
>>>
>>> Standard Engineer
>>> IT Standard & Patent/IT Product Line
>>> Huawei Technologies Co,. Ltd
>>> Email: huangzhipeng(a)huawei.com
>>> Office: Huawei Industrial Base, Longgang, Shenzhen
>>>
>>> (Previous)
>>> Research Assistant
>>> Mobile Ad-Hoc Network Lab, Calit2
>>> University of California, Irvine
>>> Email: zhipengh(a)uci.edu
>>> Office: Calit2 Building Room 2402
>>>
>>> OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado
>>> _______________________________________________
>>> SPDK mailing list
>>> SPDK(a)lists.01.org
>>> https://lists.01.org/mailman/listinfo/spdk
>>>
>>>
>>>
>>> _______________________________________________
>>> SPDK mailing list
>>> SPDK(a)lists.01.org
>>> https://lists.01.org/mailman/listinfo/spdk
>>>
>>>
>>
>>
>> --
>> Zhipeng (Howard) Huang
>>
>> Standard Engineer
>> IT Standard & Patent/IT Product Line
>> Huawei Technologies Co,. Ltd
>> Email: huangzhipeng(a)huawei.com
>> Office: Huawei Industrial Base, Longgang, Shenzhen
>>
>> (Previous)
>> Research Assistant
>> Mobile Ad-Hoc Network Lab, Calit2
>> University of California, Irvine
>> Email: zhipengh(a)uci.edu
>> Office: Calit2 Building Room 2402
>>
>> OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado
>>
>> _______________________________________________
>> SPDK mailing list
>> SPDK(a)lists.01.org
>> https://lists.01.org/mailman/listinfo/spdk
>>
>>


-- 
Zhipeng (Howard) Huang

Standard Engineer
IT Standard & Patent/IT Product Line
Huawei Technologies Co,. Ltd
Email: huangzhipeng(a)huawei.com
Office: Huawei Industrial Base, Longgang, Shenzhen

(Previous)
Research Assistant
Mobile Ad-Hoc Network Lab, Calit2
University of California, Irvine
Email: zhipengh(a)uci.edu
Office: Calit2 Building Room 2402

OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 78837 bytes --]

Re: [SPDK] Add py-spdk client for SPDK

[Zhipeng Huang]

[-- Attachment #1: Type: text/plain, Size: 16907 bytes --]

Hi Ben,

You are absolutely right on the first point.

On the second point, the main reason for us is that protobuf could auto
generate the data model for any interraces. Let's say we want to write a go
tool, we could just write the API similar to py-spdk, and the protobuf will
auto generate the msg bodies that will be exchanged over the API.

Protobuf is purely from an easy to use perspective, if you think it adds
too much complexity then we could just use plain JSON RPC :)

On Jan 19, 2018 12:58 AM, "Walker, Benjamin" <benjamin.walker(a)intel.com>
wrote:

> I'm still not at all clear on either point, so please allow me to ask a
> few more quesrions.
>
> These python scripts interact with the SPDK applications through JSON RPC,
> not through direct calls, correct? If that is the case, let's call this a
> management tool or management library and not use the term binding from
> here on out. I think that will go a long way toward helping everyone
> understand the role this code is playing.
>
> As far as using protobuf, most languages have facilities to send json rpcs
> natively. This is part of the standard library in go, for example.
> Protobufs are great for generating high performance parsers of binary
> protocols in multiple languages, but for a management interface I think
> JSON RPC is a better choice. It is plain text and much more flexible. Is
> there any reason you can't just send the JSON RPCs from your go application
> directly? Surely this can't be any harder than sending the protobuf
> messages.
>
> I hope I'm understanding.
>
> Thanks,
> Ben
>
>
> -------- Original message --------
> From: Zhipeng Huang <zhipengh512(a)gmail.com>
> Date: 1/18/18 8:06 AM (GMT-08:00)
> To: Storage Performance Development Kit <spdk(a)lists.01.org>
> Subject: Re: [SPDK] Add py-spdk client for SPDK
>
> Hi Ben,
>
> I could help hellowway answer a few questions.
>
> First of all, for py-spdk we are shooting for a SDK framework for SPDK so
> that if someone develop a userspace storage app based upon SPDK, he could
> quickly develop the SDK for his app based upon py-spdk. You are right that
> typically when we say binding we mean a direct integration, however for
> py-spdk we also want it to be a client that could remotely interact with
> the management plane. So in essence py-spdk could be used as a python
> binding for SPDK as well as a simple python client.
>
> Second on the use of protobuf, is that we see possibilities of having
> different lang bindings for SPDK. Using protobuf is a great way to define a
> common data model and reuse it in these different lang bindings.
>
> For example we have an open source storage mgmt project called OpenSDS
> that might use SPDK for NVMe Over Fabric JBOFs. OpenSDS is written in
> golang therefore we could quickly build a go-spdk based upon the protobuf
> data model.
>
> Hope I make things clearer :)
>
> On Thu, Jan 18, 2018 at 11:49 PM, Walker, Benjamin <
> benjamin.walker(a)intel.com> wrote:
>
>> Hi Helloway,
>>
>> I'll admit up front that I haven't read through your code yet, but I have
>> read your trello document. I have a couple of very basic questions.
>>
>> First, usually when people say "binding" they mean a way to call a
>> library from a different language. A python binding to SPDK would mean a
>> wrapper around the C libraries so that they could be called from python
>> directly, with no interprocess communication. My understanding of what
>> you're writing is that it is not a binding, but rather a management tool
>> for the SPDK example target applications. Am I understanding this correctly?
>>
>> Second, SPDK exposes JSON RPC interfaces for management of the target
>> applications. These RPCs can be sent from any language and we already have
>> some python scripts that do just that.  So why do you need protobufs? What
>> new value does your code add above what we have in scripts/rpc.py already?
>> Is it just more fully featured?
>>
>> Thanks in advance for helping me understand.
>>
>>
>> -------- Original message --------
>> From: We We <simple_hlw(a)163.com>
>> Date: 1/18/18 8:12 AM (GMT-07:00)
>> To: Storage Performance Development Kit <spdk(a)lists.01.org>
>> Subject: Re: [SPDK] Add py-spdk client for SPDK
>>
>> Hi Maciek,
>>
>> Thanks!! I am very happy to receive your kind feedback.
>> In my opinion,  I indeed want to put the py-spdk into the spdk/py-spdk
>> repo rathan than spdk/spdk repo. Because I think the py-spdk is built upon
>> the SPDK which can provide a general python binding for spdk framework. I
>> don't know whether I keep consistent with you. Please let me know what you
>> think. And is it convenient for you to tell me what you have done with SPDK
>> and Cinder?
>> Any thoughts are appreciated.
>>
>> Regards,
>> Helloway
>>
>>
>>
>> 在 2018年1月18日，下午10:22，Szwed, Maciej <maciej.szwed(a)intel.com> 写道：
>>
>> Hello WeWe,
>> I’ve looked closer at your patch and it looks good (some minor tweaks
>> maybe, some error handling missing), but before I comment it on Gerrit we
>> have make decision where we want to keep it. I would rather see it outside
>> spdk/spdk repo. There is some work ongoing to connect SPDK with Cinder. I
>> think that these two could end up in same repository.
>>
>> Regards,
>> Maciek
>>
>> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org
>> <spdk-bounces(a)lists.01.org>] *On Behalf Of *Zhipeng Huang
>> *Sent:* Thursday, January 18, 2018 3:01 AM
>> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
>> *Subject:* Re: [SPDK] Add py-spdk client for SPDK
>>
>> Thanks and Wewe is working on to fix that in the next patch :)
>>
>> As she stated earlier, we think it is better to have a standalone repo
>> hosting all the different lang bindings for spdk under
>> https://github.com/spdk , similar example could be found at openstack
>> sdk, so that we don't blow up the main spdk repo
>>
>> On Thu, Jan 18, 2018 at 9:49 AM, Luse, Paul E <paul.e.luse(a)intel.com>
>> wrote:
>>
>> FYI there are other format issues as well, if you look at some of the
>> other test machines you’ll see this, for example, on the patch:
>>
>> Checking coding style... OK
>> Checking comment style... OK
>> Checking blank lines at end of file... Incorrect end-of-file formatting
>> detected
>> py-spdk/README.md: Extra trailing newline
>> Checking for POSIX includes... OK
>> Checking Python style... OK
>>
>>
>> All very easy to catch & fix locally J
>>
>> Thx
>> Paul
>>
>> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org] *On Behalf Of *Zhipeng
>> Huang
>> *Sent:* Wednesday, January 17, 2018 6:33 PM
>>
>> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
>> *Subject:* Re: [SPDK] Add py-spdk client for SPDK
>>
>> Hi Paul,
>>
>> Thanks for getting the CI running. Seems most of the problems caused by
>> pep8 missing. Is it possible to add python support in the CI system ?
>> otherwise we will hit many pip pkg missing problems.
>>
>> On Thu, Jan 18, 2018 at 8:17 AM, Luse, Paul E <paul.e.luse(a)intel.com>
>> wrote:
>>
>> Hmm yeah I’m not sure why it ran on patch set 2 but not the last couple…
>> Seth, anyone?
>>
>> We might need one of the maintainers to clear it for CI execution, it’s
>> an extra unfortunate security requirement we have on the test pool,
>> otherwise its fully automated
>>
>> Thx
>> Paul
>>
>> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org] *On Behalf Of *Zhipeng
>> Huang
>> *Sent:* Wednesday, January 17, 2018 4:04 PM
>>
>> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
>> *Subject:* Re: [SPDK] Add py-spdk client for SPDK
>>
>> Paul, the problem is that Wewe's later patches doesn't trigger CI at all.
>> Is it how it supposed to work that we should fix the problem before hand ?
>> Since in OpenStack or kubernetes CI should be triggered everytime the
>> patches come in.
>>
>> On Thu, Jan 18, 2018 at 5:39 AM, Luse, Paul E <paul.e.luse(a)intel.com>
>> wrote:
>>
>> Hi WeWe
>>
>>
>> So wrt which repo, I didn’t see that but we don’t really have a precedent
>> for subprojects like that.  I’ll let one of the maintainers comment on the
>> options but I believe they would be (a) maintain on your own elsewhere or
>> (b) include in SPDK like any other code.  Option (b) is really for “core”
>> SPDK elements that make sense for most use cases and a python binding may
>> or may not fit, it clearly isn’t out of the question in my mind though as
>> it’s not directly tied to OpenStack.
>>
>> On your patch not passing, see the screenshot below where I highlighted
>> the -1 from the CI system.  If you click on that link it will take you to
>> the results and you can see the failure I mention.  Have you tried running
>> the check_format.sh locally?
>>
>> Thx
>> Paul
>>
>>
>> <image001.png>
>>
>> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org] *On Behalf Of *We We
>> *Sent:* Wednesday, January 17, 2018 10:26 AM
>>
>> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
>> *Subject:* Re: [SPDK] Add py-spdk client for SPDK
>>
>> Hi, Paul
>> I am sorry.  Is  the question on my patch you mentioned  the following Q1?
>> Q1: is your intention to propose this as part of the SPDK repo or do you
>> intent to maintain this in your own repo and are simply asking for review
>> comments?
>> If so, I see it and response to you on the patch.  Maybe you didn’t see
>> it.
>>
>> 1. About Q1, my idea is to treat the py-spdk as another subrepo in the
>> SPDK community. For example: SPDK repo -->https://github.com/spdk/spd
>> k.git, py-spdk repo -->https://github.com/spdk/py-spdk.git. Am I
>> reasonable？
>>
>>
>> 2. About a copy of rpc.py. Thank you for your careful review, the
>> py-spdk doesn’t need to modify and copy the files of SPDK, I have deleted
>> the rpc.py and updated the new patch. Please visit it.
>>
>>
>> 3. About my patch didn’t pass CI.  My patch didn’t show nothing
>> including the “verified -1”. I didn’t know whether the CI executed, so I
>> can’t track anything down.  Is there anyone who met this problem?
>>
>> Regards,
>> Helloway
>>
>>
>>
>>
>> 在 2018年1月17日，下午10:32，Luse, Paul E <paul.e.luse(a)intel.com> 写道：
>>
>> Hi WeWe,
>>
>> Thanks for the trello update, did you see my question on your patch? I
>> went to look specifically at the changes to existing files to start with
>> and it appears that your patch includes a copy of rpc.py basically
>> duplicating it within the repo.  Did I miss something or is that what you
>> intended?
>>
>> Wrt  question on why your patch didn’t pass CI, it’s pretty easy to track
>> these things down most of the time.  In the review, click on the link below
>> where it says verified -1 and you will see a webpage with the status of
>> each of the test machines.  Many of the are red and indicate failure.
>> Click on one and drill down to find the builg.log file and you’ll see some
>> formatting issues:
>>
>> ========== Backtrace start: ==========
>>
>> in ./autobuild.sh:23 -> main()
>>      ...
>>    18
>>    19  ./configure $config_params
>>    20
>>    21  timing_enter check_format
>>    22  if [ $SPDK_RUN_CHECK_FORMAT -eq 1 ]; then
>> => 23          ./scripts/check_format.sh
>>    24  fi
>>    25  timing_exit check_format
>>    26
>>    27  timing_enter build_kmod
>>    28  if [ $SPDK_BUILD_IOAT_KMOD -eq 1 ]; then
>>      ...
>>
>> ========== Backtrace end ==========
>>
>> You can run the script, ./scripts/check_format.sh, locally and correct
>> the errors and resubmit.  We always ask that developers run this scripts in
>> advance to make sure things are formatted.  Depending on whether the
>> maintainers think the SPDK repo makes sense for this patch you’ll need some
>> test code also – there is clearly a lot of value in this patch though and
>> regardless of whether it lands here or is maintained elsewhere!
>>
>> Use this list or the patch review if you have more questions on getting
>> it to pass.
>>
>> Thanks!
>> Paul
>>
>>
>>
>> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org
>> <spdk-bounces(a)lists.01.org>] *On Behalf Of *We We
>> *Sent:* Wednesday, January 17, 2018 5:24 AM
>> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
>> *Subject:* Re: [SPDK] Add py-spdk client for SPDK
>>
>> Hi, all
>> I have submitted the introduction of py-spdk on trello
>> https://trello.com/c/Bn3DIG7F/87-add-py-spdk-client-for-spdk, please
>> visit it. I will be grateful to your kindness.
>>
>> Regards,
>>
>> Helloway
>>
>>
>> 在 2018年1月16日，上午11:30，We We <simple_hlw(a)163.com> 写道：
>>
>> Hi, all
>> I have submitted the py-spdk(written in python) for more than six days.
>> Why the code-review couldn’t pass and show nothing?
>> Is there anyone who can help me?
>>
>> Thanks,
>> Helloway
>>
>>
>> 在 2018年1月10日，下午9:46，We We <simple_hlw(a)163.com> 写道：
>>
>> Hi, all
>> I have submitted the py-spdk code onhttps://review.gerrithub.io/
>> #/c/379741/, please take some time to visit it, I will be very grateful
>> to you.
>> The py-spdk is client which can help the upper-level app to communicate
>> with the SPDK-based app (such as: nvmf_tgt, vhost, iscsi_tgt, etc.). Should
>> I submit it into the other repo I rebuild rather than SPDK repo? Because I
>> think it is a relatively independent kit upon the SPDK.
>>
>> If you have some thoughts about the py-spdk, please share with me.
>>
>> Regards,
>> Helloway
>> _______________________________________________
>> SPDK mailing list
>> SPDK(a)lists.01.org
>> https://lists.01.org/mailman/listinfo/spdk
>>
>>
>>
>>
>> _______________________________________________
>> SPDK mailing list
>> SPDK(a)lists.01.org
>> https://lists.01.org/mailman/listinfo/spdk
>>
>>
>>
>>
>> _______________________________________________
>> SPDK mailing list
>> SPDK(a)lists.01.org
>> https://lists.01.org/mailman/listinfo/spdk
>>
>>
>>
>>
>> --
>> Zhipeng (Howard) Huang
>>
>> Standard Engineer
>> IT Standard & Patent/IT Product Line
>> Huawei Technologies Co,. Ltd
>> Email: huangzhipeng(a)huawei.com
>> Office: Huawei Industrial Base, Longgang, Shenzhen
>>
>> (Previous)
>> Research Assistant
>> Mobile Ad-Hoc Network Lab, Calit2
>> University of California, Irvine
>> Email: zhipengh(a)uci.edu
>> Office: Calit2 Building Room 2402
>>
>> OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado
>>
>>
>> _______________________________________________
>> SPDK mailing list
>> SPDK(a)lists.01.org
>> https://lists.01.org/mailman/listinfo/spdk
>>
>>
>>
>>
>> --
>> Zhipeng (Howard) Huang
>>
>> Standard Engineer
>> IT Standard & Patent/IT Product Line
>> Huawei Technologies Co,. Ltd
>> Email: huangzhipeng(a)huawei.com
>> Office: Huawei Industrial Base, Longgang, Shenzhen
>>
>> (Previous)
>> Research Assistant
>> Mobile Ad-Hoc Network Lab, Calit2
>> University of California, Irvine
>> Email: zhipengh(a)uci.edu
>> Office: Calit2 Building Room 2402
>>
>> OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado
>>
>>
>> _______________________________________________
>> SPDK mailing list
>> SPDK(a)lists.01.org
>> https://lists.01.org/mailman/listinfo/spdk
>>
>>
>>
>>
>> --
>> Zhipeng (Howard) Huang
>>
>> Standard Engineer
>> IT Standard & Patent/IT Product Line
>> Huawei Technologies Co,. Ltd
>> Email: huangzhipeng(a)huawei.com
>> Office: Huawei Industrial Base, Longgang, Shenzhen
>>
>> (Previous)
>> Research Assistant
>> Mobile Ad-Hoc Network Lab, Calit2
>> University of California, Irvine
>> Email: zhipengh(a)uci.edu
>> Office: Calit2 Building Room 2402
>>
>> OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado
>> _______________________________________________
>> SPDK mailing list
>> SPDK(a)lists.01.org
>> https://lists.01.org/mailman/listinfo/spdk
>>
>>
>>
>> _______________________________________________
>> SPDK mailing list
>> SPDK(a)lists.01.org
>> https://lists.01.org/mailman/listinfo/spdk
>>
>>
>
>
> --
> Zhipeng (Howard) Huang
>
> Standard Engineer
> IT Standard & Patent/IT Product Line
> Huawei Technologies Co,. Ltd
> Email: huangzhipeng(a)huawei.com
> Office: Huawei Industrial Base, Longgang, Shenzhen
>
> (Previous)
> Research Assistant
> Mobile Ad-Hoc Network Lab, Calit2
> University of California, Irvine
> Email: zhipengh(a)uci.edu
> Office: Calit2 Building Room 2402
>
> OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 75851 bytes --]

Re: [SPDK] Add py-spdk client for SPDK

[Walker, Benjamin]

[-- Attachment #1: Type: text/plain, Size: 15748 bytes --]

I'm still not at all clear on either point, so please allow me to ask a few more quesrions.

These python scripts interact with the SPDK applications through JSON RPC, not through direct calls, correct? If that is the case, let's call this a management tool or management library and not use the term binding from here on out. I think that will go a long way toward helping everyone understand the role this code is playing.

As far as using protobuf, most languages have facilities to send json rpcs natively. This is part of the standard library in go, for example. Protobufs are great for generating high performance parsers of binary protocols in multiple languages, but for a management interface I think JSON RPC is a better choice. It is plain text and much more flexible. Is there any reason you can't just send the JSON RPCs from your go application directly? Surely this can't be any harder than sending the protobuf messages.

I hope I'm understanding.

Thanks,
Ben


-------- Original message --------
From: Zhipeng Huang <zhipengh512(a)gmail.com>
Date: 1/18/18 8:06 AM (GMT-08:00)
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: Re: [SPDK] Add py-spdk client for SPDK

Hi Ben,

I could help hellowway answer a few questions.

First of all, for py-spdk we are shooting for a SDK framework for SPDK so that if someone develop a userspace storage app based upon SPDK, he could quickly develop the SDK for his app based upon py-spdk. You are right that typically when we say binding we mean a direct integration, however for py-spdk we also want it to be a client that could remotely interact with the management plane. So in essence py-spdk could be used as a python binding for SPDK as well as a simple python client.

Second on the use of protobuf, is that we see possibilities of having different lang bindings for SPDK. Using protobuf is a great way to define a common data model and reuse it in these different lang bindings.

For example we have an open source storage mgmt project called OpenSDS that might use SPDK for NVMe Over Fabric JBOFs. OpenSDS is written in golang therefore we could quickly build a go-spdk based upon the protobuf data model.

Hope I make things clearer :)

On Thu, Jan 18, 2018 at 11:49 PM, Walker, Benjamin <benjamin.walker(a)intel.com<mailto:benjamin.walker(a)intel.com>> wrote:
Hi Helloway,

I'll admit up front that I haven't read through your code yet, but I have read your trello document. I have a couple of very basic questions.

First, usually when people say "binding" they mean a way to call a library from a different language. A python binding to SPDK would mean a wrapper around the C libraries so that they could be called from python directly, with no interprocess communication. My understanding of what you're writing is that it is not a binding, but rather a management tool for the SPDK example target applications. Am I understanding this correctly?

Second, SPDK exposes JSON RPC interfaces for management of the target applications. These RPCs can be sent from any language and we already have some python scripts that do just that.  So why do you need protobufs? What new value does your code add above what we have in scripts/rpc.py already? Is it just more fully featured?

Thanks in advance for helping me understand.


-------- Original message --------
From: We We <simple_hlw(a)163.com<mailto:simple_hlw(a)163.com>>
Date: 1/18/18 8:12 AM (GMT-07:00)
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] Add py-spdk client for SPDK

Hi Maciek,

Thanks!! I am very happy to receive your kind feedback.
In my opinion,  I indeed want to put the py-spdk into the spdk/py-spdk repo rathan than spdk/spdk repo. Because I think the py-spdk is built upon the SPDK which can provide a general python binding for spdk framework. I don't know whether I keep consistent with you. Please let me know what you think. And is it convenient for you to tell me what you have done with SPDK and Cinder?
Any thoughts are appreciated.

Regards,
Helloway



在 2018年1月18日，下午10:22，Szwed, Maciej <maciej.szwed(a)intel.com<mailto:maciej.szwed(a)intel.com>> 写道：

Hello WeWe,
I’ve looked closer at your patch and it looks good (some minor tweaks maybe, some error handling missing), but before I comment it on Gerrit we have make decision where we want to keep it. I would rather see it outside spdk/spdk repo. There is some work ongoing to connect SPDK with Cinder. I think that these two could end up in same repository.

Regards,
Maciek

From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Zhipeng Huang
Sent: Thursday, January 18, 2018 3:01 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] Add py-spdk client for SPDK

Thanks and Wewe is working on to fix that in the next patch :)

As she stated earlier, we think it is better to have a standalone repo hosting all the different lang bindings for spdk under https://github.com/spdk , similar example could be found at openstack sdk, so that we don't blow up the main spdk repo

On Thu, Jan 18, 2018 at 9:49 AM, Luse, Paul E <paul.e.luse(a)intel.com<mailto:paul.e.luse(a)intel.com>> wrote:
FYI there are other format issues as well, if you look at some of the other test machines you’ll see this, for example, on the patch:

Checking coding style... OK
Checking comment style... OK
Checking blank lines at end of file... Incorrect end-of-file formatting detected
py-spdk/README.md: Extra trailing newline
Checking for POSIX includes... OK
Checking Python style... OK


All very easy to catch & fix locally :)

Thx
Paul

From: SPDK [mailto:spdk-bounces(a)lists.01.org<mailto:spdk-bounces(a)lists.01.org>] On Behalf Of Zhipeng Huang
Sent: Wednesday, January 17, 2018 6:33 PM

To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] Add py-spdk client for SPDK

Hi Paul,

Thanks for getting the CI running. Seems most of the problems caused by pep8 missing. Is it possible to add python support in the CI system ? otherwise we will hit many pip pkg missing problems.

On Thu, Jan 18, 2018 at 8:17 AM, Luse, Paul E <paul.e.luse(a)intel.com<mailto:paul.e.luse(a)intel.com>> wrote:
Hmm yeah I’m not sure why it ran on patch set 2 but not the last couple… Seth, anyone?

We might need one of the maintainers to clear it for CI execution, it’s an extra unfortunate security requirement we have on the test pool, otherwise its fully automated

Thx
Paul

From: SPDK [mailto:spdk-bounces(a)lists.01.org<mailto:spdk-bounces(a)lists.01.org>] On Behalf Of Zhipeng Huang
Sent: Wednesday, January 17, 2018 4:04 PM

To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] Add py-spdk client for SPDK

Paul, the problem is that Wewe's later patches doesn't trigger CI at all. Is it how it supposed to work that we should fix the problem before hand ?
Since in OpenStack or kubernetes CI should be triggered everytime the patches come in.

On Thu, Jan 18, 2018 at 5:39 AM, Luse, Paul E <paul.e.luse(a)intel.com<mailto:paul.e.luse(a)intel.com>> wrote:
Hi WeWe


So wrt which repo, I didn’t see that but we don’t really have a precedent for subprojects like that.  I’ll let one of the maintainers comment on the options but I believe they would be (a) maintain on your own elsewhere or (b) include in SPDK like any other code.  Option (b) is really for “core” SPDK elements that make sense for most use cases and a python binding may or may not fit, it clearly isn’t out of the question in my mind though as it’s not directly tied to OpenStack.

On your patch not passing, see the screenshot below where I highlighted the -1 from the CI system.  If you click on that link it will take you to the results and you can see the failure I mention.  Have you tried running the check_format.sh locally?

Thx
Paul


<image001.png>

From: SPDK [mailto:spdk-bounces(a)lists.01.org<mailto:spdk-bounces(a)lists.01.org>] On Behalf Of We We
Sent: Wednesday, January 17, 2018 10:26 AM

To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] Add py-spdk client for SPDK

Hi, Paul
I am sorry.  Is  the question on my patch you mentioned  the following Q1?
Q1: is your intention to propose this as part of the SPDK repo or do you intent to maintain this in your own repo and are simply asking for review comments?
If so, I see it and response to you on the patch.  Maybe you didn’t see it.

1. About Q1, my idea is to treat the py-spdk as another subrepo in the SPDK community. For example: SPDK repo -->https://github.com/spdk/spdk.git, py-spdk repo -->https://github.com/spdk/py-spdk.git. Am I reasonable？

2. About a copy of rpc.py. Thank you for your careful review, the py-spdk doesn’t need to modify and copy the files of SPDK, I have deleted the rpc.py and updated the new patch. Please visit it.

3. About my patch didn’t pass CI.  My patch didn’t show nothing including the “verified -1”. I didn’t know whether the CI executed, so I can’t track anything down.  Is there anyone who met this problem?

Regards,
Helloway


在 2018年1月17日，下午10:32，Luse, Paul E <paul.e.luse(a)intel.com<mailto:paul.e.luse(a)intel.com>> 写道：

Hi WeWe,

Thanks for the trello update, did you see my question on your patch? I went to look specifically at the changes to existing files to start with and it appears that your patch includes a copy of rpc.py basically duplicating it within the repo.  Did I miss something or is that what you intended?

Wrt  question on why your patch didn’t pass CI, it’s pretty easy to track these things down most of the time.  In the review, click on the link below where it says verified -1 and you will see a webpage with the status of each of the test machines.  Many of the are red and indicate failure.  Click on one and drill down to find the builg.log file and you’ll see some formatting issues:

========== Backtrace start: ==========

in ./autobuild.sh:23 -> main()
     ...
   18
   19  ./configure $config_params
   20
   21  timing_enter check_format
   22  if [ $SPDK_RUN_CHECK_FORMAT -eq 1 ]; then
=> 23          ./scripts/check_format.sh
   24  fi
   25  timing_exit check_format
   26
   27  timing_enter build_kmod
   28  if [ $SPDK_BUILD_IOAT_KMOD -eq 1 ]; then
     ...

========== Backtrace end ==========

You can run the script, ./scripts/check_format.sh, locally and correct the errors and resubmit.  We always ask that developers run this scripts in advance to make sure things are formatted.  Depending on whether the maintainers think the SPDK repo makes sense for this patch you’ll need some test code also � there is clearly a lot of value in this patch though and regardless of whether it lands here or is maintained elsewhere!

Use this list or the patch review if you have more questions on getting it to pass.

Thanks!
Paul



From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Wednesday, January 17, 2018 5:24 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] Add py-spdk client for SPDK

Hi, all
I have submitted the introduction of py-spdk on trello https://trello.com/c/Bn3DIG7F/87-add-py-spdk-client-for-spdk, please visit it. I will be grateful to your kindness.

Regards,

Helloway

在 2018年1月16日，上午11:30，We We <simple_hlw(a)163.com<mailto:simple_hlw(a)163.com>> 写道：

Hi, all
I have submitted the py-spdk(written in python) for more than six days. Why the code-review couldn’t pass and show nothing?
Is there anyone who can help me?

Thanks,
Helloway

在 2018年1月10日，下午9:46，We We <simple_hlw(a)163.com<mailto:simple_hlw(a)163.com>> 写道：

Hi, all
I have submitted the py-spdk code onhttps://review.gerrithub.io/#/c/379741/, please take some time to visit it, I will be very grateful to you.
The py-spdk is client which can help the upper-level app to communicate with the SPDK-based app (such as: nvmf_tgt, vhost, iscsi_tgt, etc.). Should I submit it into the other repo I rebuild rather than SPDK repo? Because I think it is a relatively independent kit upon the SPDK.

If you have some thoughts about the py-spdk, please share with me.

Regards,
Helloway
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk


_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk


_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk



--
Zhipeng (Howard) Huang

Standard Engineer
IT Standard & Patent/IT Product Line
Huawei Technologies Co,. Ltd
Email: huangzhipeng(a)huawei.com<mailto:huangzhipeng(a)huawei.com>
Office: Huawei Industrial Base, Longgang, Shenzhen

(Previous)
Research Assistant
Mobile Ad-Hoc Network Lab, Calit2
University of California, Irvine
Email: zhipengh(a)uci.edu<mailto:zhipengh(a)uci.edu>
Office: Calit2 Building Room 2402

OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado

_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk



--
Zhipeng (Howard) Huang

Standard Engineer
IT Standard & Patent/IT Product Line
Huawei Technologies Co,. Ltd
Email: huangzhipeng(a)huawei.com<mailto:huangzhipeng(a)huawei.com>
Office: Huawei Industrial Base, Longgang, Shenzhen

(Previous)
Research Assistant
Mobile Ad-Hoc Network Lab, Calit2
University of California, Irvine
Email: zhipengh(a)uci.edu<mailto:zhipengh(a)uci.edu>
Office: Calit2 Building Room 2402

OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado

_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk



--
Zhipeng (Howard) Huang

Standard Engineer
IT Standard & Patent/IT Product Line
Huawei Technologies Co,. Ltd
Email: huangzhipeng(a)huawei.com<mailto:huangzhipeng(a)huawei.com>
Office: Huawei Industrial Base, Longgang, Shenzhen

(Previous)
Research Assistant
Mobile Ad-Hoc Network Lab, Calit2
University of California, Irvine
Email: zhipengh(a)uci.edu<mailto:zhipengh(a)uci.edu>
Office: Calit2 Building Room 2402

OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk


_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk




--
Zhipeng (Howard) Huang

Standard Engineer
IT Standard & Patent/IT Product Line
Huawei Technologies Co,. Ltd
Email: huangzhipeng(a)huawei.com<mailto:huangzhipeng(a)huawei.com>
Office: Huawei Industrial Base, Longgang, Shenzhen

(Previous)
Research Assistant
Mobile Ad-Hoc Network Lab, Calit2
University of California, Irvine
Email: zhipengh(a)uci.edu<mailto:zhipengh(a)uci.edu>
Office: Calit2 Building Room 2402

OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 72738 bytes --]

Re: [SPDK] Add py-spdk client for SPDK

[Zhipeng Huang]

[-- Attachment #1: Type: text/plain, Size: 14424 bytes --]

Hi Ben,

I could help hellowway answer a few questions.

First of all, for py-spdk we are shooting for a SDK framework for SPDK so
that if someone develop a userspace storage app based upon SPDK, he could
quickly develop the SDK for his app based upon py-spdk. You are right that
typically when we say binding we mean a direct integration, however for
py-spdk we also want it to be a client that could remotely interact with
the management plane. So in essence py-spdk could be used as a python
binding for SPDK as well as a simple python client.

Second on the use of protobuf, is that we see possibilities of having
different lang bindings for SPDK. Using protobuf is a great way to define a
common data model and reuse it in these different lang bindings.

For example we have an open source storage mgmt project called OpenSDS that
might use SPDK for NVMe Over Fabric JBOFs. OpenSDS is written in golang
therefore we could quickly build a go-spdk based upon the protobuf data
model.

Hope I make things clearer :)

On Thu, Jan 18, 2018 at 11:49 PM, Walker, Benjamin <
benjamin.walker(a)intel.com> wrote:

> Hi Helloway,
>
> I'll admit up front that I haven't read through your code yet, but I have
> read your trello document. I have a couple of very basic questions.
>
> First, usually when people say "binding" they mean a way to call a library
> from a different language. A python binding to SPDK would mean a wrapper
> around the C libraries so that they could be called from python directly,
> with no interprocess communication. My understanding of what you're writing
> is that it is not a binding, but rather a management tool for the SPDK
> example target applications. Am I understanding this correctly?
>
> Second, SPDK exposes JSON RPC interfaces for management of the target
> applications. These RPCs can be sent from any language and we already have
> some python scripts that do just that.  So why do you need protobufs? What
> new value does your code add above what we have in scripts/rpc.py already?
> Is it just more fully featured?
>
> Thanks in advance for helping me understand.
>
>
> -------- Original message --------
> From: We We <simple_hlw(a)163.com>
> Date: 1/18/18 8:12 AM (GMT-07:00)
> To: Storage Performance Development Kit <spdk(a)lists.01.org>
> Subject: Re: [SPDK] Add py-spdk client for SPDK
>
> Hi Maciek,
>
> Thanks!! I am very happy to receive your kind feedback.
> In my opinion,  I indeed want to put the py-spdk into the spdk/py-spdk
> repo rathan than spdk/spdk repo. Because I think the py-spdk is built upon
> the SPDK which can provide a general python binding for spdk framework. I
> don't know whether I keep consistent with you. Please let me know what you
> think. And is it convenient for you to tell me what you have done with SPDK
> and Cinder?
> Any thoughts are appreciated.
>
> Regards,
> Helloway
>
>
>
> 在 2018年1月18日，下午10:22，Szwed, Maciej <maciej.szwed(a)intel.com> 写道：
>
> Hello WeWe,
> I’ve looked closer at your patch and it looks good (some minor tweaks
> maybe, some error handling missing), but before I comment it on Gerrit we
> have make decision where we want to keep it. I would rather see it outside
> spdk/spdk repo. There is some work ongoing to connect SPDK with Cinder. I
> think that these two could end up in same repository.
>
> Regards,
> Maciek
>
> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org <spdk-bounces(a)lists.01.org>
> ] *On Behalf Of *Zhipeng Huang
> *Sent:* Thursday, January 18, 2018 3:01 AM
> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
> *Subject:* Re: [SPDK] Add py-spdk client for SPDK
>
> Thanks and Wewe is working on to fix that in the next patch :)
>
> As she stated earlier, we think it is better to have a standalone repo
> hosting all the different lang bindings for spdk under
> https://github.com/spdk , similar example could be found at openstack
> sdk, so that we don't blow up the main spdk repo
>
> On Thu, Jan 18, 2018 at 9:49 AM, Luse, Paul E <paul.e.luse(a)intel.com>
> wrote:
>
> FYI there are other format issues as well, if you look at some of the
> other test machines you’ll see this, for example, on the patch:
>
> Checking coding style... OK
> Checking comment style... OK
> Checking blank lines at end of file... Incorrect end-of-file formatting
> detected
> py-spdk/README.md: Extra trailing newline
> Checking for POSIX includes... OK
> Checking Python style... OK
>
>
> All very easy to catch & fix locally J
>
> Thx
> Paul
>
> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org] *On Behalf Of *Zhipeng
> Huang
> *Sent:* Wednesday, January 17, 2018 6:33 PM
>
> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
> *Subject:* Re: [SPDK] Add py-spdk client for SPDK
>
> Hi Paul,
>
> Thanks for getting the CI running. Seems most of the problems caused by
> pep8 missing. Is it possible to add python support in the CI system ?
> otherwise we will hit many pip pkg missing problems.
>
> On Thu, Jan 18, 2018 at 8:17 AM, Luse, Paul E <paul.e.luse(a)intel.com>
> wrote:
>
> Hmm yeah I’m not sure why it ran on patch set 2 but not the last couple…
> Seth, anyone?
>
> We might need one of the maintainers to clear it for CI execution, it’s an
> extra unfortunate security requirement we have on the test pool, otherwise
> its fully automated
>
> Thx
> Paul
>
> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org] *On Behalf Of *Zhipeng
> Huang
> *Sent:* Wednesday, January 17, 2018 4:04 PM
>
> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
> *Subject:* Re: [SPDK] Add py-spdk client for SPDK
>
> Paul, the problem is that Wewe's later patches doesn't trigger CI at all.
> Is it how it supposed to work that we should fix the problem before hand ?
> Since in OpenStack or kubernetes CI should be triggered everytime the
> patches come in.
>
> On Thu, Jan 18, 2018 at 5:39 AM, Luse, Paul E <paul.e.luse(a)intel.com>
> wrote:
>
> Hi WeWe
>
>
> So wrt which repo, I didn’t see that but we don’t really have a precedent
> for subprojects like that.  I’ll let one of the maintainers comment on the
> options but I believe they would be (a) maintain on your own elsewhere or
> (b) include in SPDK like any other code.  Option (b) is really for “core”
> SPDK elements that make sense for most use cases and a python binding may
> or may not fit, it clearly isn’t out of the question in my mind though as
> it’s not directly tied to OpenStack.
>
> On your patch not passing, see the screenshot below where I highlighted
> the -1 from the CI system.  If you click on that link it will take you to
> the results and you can see the failure I mention.  Have you tried running
> the check_format.sh locally?
>
> Thx
> Paul
>
>
> <image001.png>
>
> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org] *On Behalf Of *We We
> *Sent:* Wednesday, January 17, 2018 10:26 AM
>
> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
> *Subject:* Re: [SPDK] Add py-spdk client for SPDK
>
> Hi, Paul
> I am sorry.  Is  the question on my patch you mentioned  the following Q1?
> Q1: is your intention to propose this as part of the SPDK repo or do you
> intent to maintain this in your own repo and are simply asking for review
> comments?
> If so, I see it and response to you on the patch.  Maybe you didn’t see
> it.
>
> 1. About Q1, my idea is to treat the py-spdk as another subrepo in the
> SPDK community. For example: SPDK repo -->https://github.com/spdk/spdk.git,
> py-spdk repo -->https://github.com/spdk/py-spdk.git. Am I reasonable？
>
>
> 2. About a copy of rpc.py. Thank you for your careful review, the py-spdk
> doesn’t need to modify and copy the files of SPDK, I have deleted the
> rpc.py and updated the new patch. Please visit it.
>
>
> 3. About my patch didn’t pass CI.  My patch didn’t show nothing including
> the “verified -1”. I didn’t know whether the CI executed, so I can’t
> track anything down.  Is there anyone who met this problem?
>
> Regards,
> Helloway
>
>
>
>
> 在 2018年1月17日，下午10:32，Luse, Paul E <paul.e.luse(a)intel.com> 写道：
>
> Hi WeWe,
>
> Thanks for the trello update, did you see my question on your patch? I
> went to look specifically at the changes to existing files to start with
> and it appears that your patch includes a copy of rpc.py basically
> duplicating it within the repo.  Did I miss something or is that what you
> intended?
>
> Wrt  question on why your patch didn’t pass CI, it’s pretty easy to track
> these things down most of the time.  In the review, click on the link below
> where it says verified -1 and you will see a webpage with the status of
> each of the test machines.  Many of the are red and indicate failure.
> Click on one and drill down to find the builg.log file and you’ll see some
> formatting issues:
>
> ========== Backtrace start: ==========
>
> in ./autobuild.sh:23 -> main()
>      ...
>    18
>    19  ./configure $config_params
>    20
>    21  timing_enter check_format
>    22  if [ $SPDK_RUN_CHECK_FORMAT -eq 1 ]; then
> => 23          ./scripts/check_format.sh
>    24  fi
>    25  timing_exit check_format
>    26
>    27  timing_enter build_kmod
>    28  if [ $SPDK_BUILD_IOAT_KMOD -eq 1 ]; then
>      ...
>
> ========== Backtrace end ==========
>
> You can run the script, ./scripts/check_format.sh, locally and correct
> the errors and resubmit.  We always ask that developers run this scripts in
> advance to make sure things are formatted.  Depending on whether the
> maintainers think the SPDK repo makes sense for this patch you’ll need some
> test code also – there is clearly a lot of value in this patch though and
> regardless of whether it lands here or is maintained elsewhere!
>
> Use this list or the patch review if you have more questions on getting it
> to pass.
>
> Thanks!
> Paul
>
>
>
> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org <spdk-bounces(a)lists.01.org>
> ] *On Behalf Of *We We
> *Sent:* Wednesday, January 17, 2018 5:24 AM
> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
> *Subject:* Re: [SPDK] Add py-spdk client for SPDK
>
> Hi, all
> I have submitted the introduction of py-spdk on trello
> https://trello.com/c/Bn3DIG7F/87-add-py-spdk-client-for-spdk, please
> visit it. I will be grateful to your kindness.
>
> Regards,
>
> Helloway
>
>
> 在 2018年1月16日，上午11:30，We We <simple_hlw(a)163.com> 写道：
>
> Hi, all
> I have submitted the py-spdk(written in python) for more than six days.
> Why the code-review couldn’t pass and show nothing?
> Is there anyone who can help me?
>
> Thanks,
> Helloway
>
>
> 在 2018年1月10日，下午9:46，We We <simple_hlw(a)163.com> 写道：
>
> Hi, all
> I have submitted the py-spdk code onhttps://review.gerrithub.io/
> #/c/379741/, please take some time to visit it, I will be very grateful
> to you.
> The py-spdk is client which can help the upper-level app to communicate
> with the SPDK-based app (such as: nvmf_tgt, vhost, iscsi_tgt, etc.). Should
> I submit it into the other repo I rebuild rather than SPDK repo? Because I
> think it is a relatively independent kit upon the SPDK.
>
> If you have some thoughts about the py-spdk, please share with me.
>
> Regards,
> Helloway
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
>
>
> --
> Zhipeng (Howard) Huang
>
> Standard Engineer
> IT Standard & Patent/IT Product Line
> Huawei Technologies Co,. Ltd
> Email: huangzhipeng(a)huawei.com
> Office: Huawei Industrial Base, Longgang, Shenzhen
>
> (Previous)
> Research Assistant
> Mobile Ad-Hoc Network Lab, Calit2
> University of California, Irvine
> Email: zhipengh(a)uci.edu
> Office: Calit2 Building Room 2402
>
> OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
>
>
> --
> Zhipeng (Howard) Huang
>
> Standard Engineer
> IT Standard & Patent/IT Product Line
> Huawei Technologies Co,. Ltd
> Email: huangzhipeng(a)huawei.com
> Office: Huawei Industrial Base, Longgang, Shenzhen
>
> (Previous)
> Research Assistant
> Mobile Ad-Hoc Network Lab, Calit2
> University of California, Irvine
> Email: zhipengh(a)uci.edu
> Office: Calit2 Building Room 2402
>
> OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
>
>
> --
> Zhipeng (Howard) Huang
>
> Standard Engineer
> IT Standard & Patent/IT Product Line
> Huawei Technologies Co,. Ltd
> Email: huangzhipeng(a)huawei.com
> Office: Huawei Industrial Base, Longgang, Shenzhen
>
> (Previous)
> Research Assistant
> Mobile Ad-Hoc Network Lab, Calit2
> University of California, Irvine
> Email: zhipengh(a)uci.edu
> Office: Calit2 Building Room 2402
>
> OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>


-- 
Zhipeng (Howard) Huang

Standard Engineer
IT Standard & Patent/IT Product Line
Huawei Technologies Co,. Ltd
Email: huangzhipeng(a)huawei.com
Office: Huawei Industrial Base, Longgang, Shenzhen

(Previous)
Research Assistant
Mobile Ad-Hoc Network Lab, Calit2
University of California, Irvine
Email: zhipengh(a)uci.edu
Office: Calit2 Building Room 2402

OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 71184 bytes --]

Re: [SPDK] Add py-spdk client for SPDK

[Walker, Benjamin]

[-- Attachment #1: Type: text/plain, Size: 12654 bytes --]

Hi Helloway,

I'll admit up front that I haven't read through your code yet, but I have read your trello document. I have a couple of very basic questions.

First, usually when people say "binding" they mean a way to call a library from a different language. A python binding to SPDK would mean a wrapper around the C libraries so that they could be called from python directly, with no interprocess communication. My understanding of what you're writing is that it is not a binding, but rather a management tool for the SPDK example target applications. Am I understanding this correctly?

Second, SPDK exposes JSON RPC interfaces for management of the target applications. These RPCs can be sent from any language and we already have some python scripts that do just that.  So why do you need protobufs? What new value does your code add above what we have in scripts/rpc.py already? Is it just more fully featured?

Thanks in advance for helping me understand.


-------- Original message --------
From: We We <simple_hlw(a)163.com>
Date: 1/18/18 8:12 AM (GMT-07:00)
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: Re: [SPDK] Add py-spdk client for SPDK

Hi Maciek,

Thanks!! I am very happy to receive your kind feedback.
In my opinion,  I indeed want to put the py-spdk into the spdk/py-spdk repo rathan than spdk/spdk repo. Because I think the py-spdk is built upon the SPDK which can provide a general python binding for spdk framework. I don't know whether I keep consistent with you. Please let me know what you think. And is it convenient for you to tell me what you have done with SPDK and Cinder?
Any thoughts are appreciated.

Regards,
Helloway



在 2018年1月18日，下午10:22，Szwed, Maciej <maciej.szwed(a)intel.com<mailto:maciej.szwed(a)intel.com>> 写道：

Hello WeWe,
I’ve looked closer at your patch and it looks good (some minor tweaks maybe, some error handling missing), but before I comment it on Gerrit we have make decision where we want to keep it. I would rather see it outside spdk/spdk repo. There is some work ongoing to connect SPDK with Cinder. I think that these two could end up in same repository.

Regards,
Maciek

From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Zhipeng Huang
Sent: Thursday, January 18, 2018 3:01 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] Add py-spdk client for SPDK

Thanks and Wewe is working on to fix that in the next patch :)

As she stated earlier, we think it is better to have a standalone repo hosting all the different lang bindings for spdk under https://github.com/spdk , similar example could be found at openstack sdk, so that we don't blow up the main spdk repo

On Thu, Jan 18, 2018 at 9:49 AM, Luse, Paul E <paul.e.luse(a)intel.com<mailto:paul.e.luse(a)intel.com>> wrote:
FYI there are other format issues as well, if you look at some of the other test machines you’ll see this, for example, on the patch:

Checking coding style... OK
Checking comment style... OK
Checking blank lines at end of file... Incorrect end-of-file formatting detected
py-spdk/README.md: Extra trailing newline
Checking for POSIX includes... OK
Checking Python style... OK


All very easy to catch & fix locally :)

Thx
Paul

From: SPDK [mailto:spdk-bounces(a)lists.01.org<mailto:spdk-bounces(a)lists.01.org>] On Behalf Of Zhipeng Huang
Sent: Wednesday, January 17, 2018 6:33 PM

To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] Add py-spdk client for SPDK

Hi Paul,

Thanks for getting the CI running. Seems most of the problems caused by pep8 missing. Is it possible to add python support in the CI system ? otherwise we will hit many pip pkg missing problems.

On Thu, Jan 18, 2018 at 8:17 AM, Luse, Paul E <paul.e.luse(a)intel.com<mailto:paul.e.luse(a)intel.com>> wrote:
Hmm yeah I’m not sure why it ran on patch set 2 but not the last couple… Seth, anyone?

We might need one of the maintainers to clear it for CI execution, it’s an extra unfortunate security requirement we have on the test pool, otherwise its fully automated

Thx
Paul

From: SPDK [mailto:spdk-bounces(a)lists.01.org<mailto:spdk-bounces(a)lists.01.org>] On Behalf Of Zhipeng Huang
Sent: Wednesday, January 17, 2018 4:04 PM

To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] Add py-spdk client for SPDK

Paul, the problem is that Wewe's later patches doesn't trigger CI at all. Is it how it supposed to work that we should fix the problem before hand ?
Since in OpenStack or kubernetes CI should be triggered everytime the patches come in.

On Thu, Jan 18, 2018 at 5:39 AM, Luse, Paul E <paul.e.luse(a)intel.com<mailto:paul.e.luse(a)intel.com>> wrote:
Hi WeWe


So wrt which repo, I didn’t see that but we don’t really have a precedent for subprojects like that.  I’ll let one of the maintainers comment on the options but I believe they would be (a) maintain on your own elsewhere or (b) include in SPDK like any other code.  Option (b) is really for “core” SPDK elements that make sense for most use cases and a python binding may or may not fit, it clearly isn’t out of the question in my mind though as it’s not directly tied to OpenStack.

On your patch not passing, see the screenshot below where I highlighted the -1 from the CI system.  If you click on that link it will take you to the results and you can see the failure I mention.  Have you tried running the check_format.sh locally?

Thx
Paul


<image001.png>

From: SPDK [mailto:spdk-bounces(a)lists.01.org<mailto:spdk-bounces(a)lists.01.org>] On Behalf Of We We
Sent: Wednesday, January 17, 2018 10:26 AM

To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] Add py-spdk client for SPDK

Hi, Paul
I am sorry.  Is  the question on my patch you mentioned  the following Q1?
Q1: is your intention to propose this as part of the SPDK repo or do you intent to maintain this in your own repo and are simply asking for review comments?
If so, I see it and response to you on the patch.  Maybe you didn’t see it.

1. About Q1, my idea is to treat the py-spdk as another subrepo in the SPDK community. For example: SPDK repo -->https://github.com/spdk/spdk.git, py-spdk repo -->https://github.com/spdk/py-spdk.git. Am I reasonable？

2. About a copy of rpc.py. Thank you for your careful review, the py-spdk doesn’t need to modify and copy the files of SPDK, I have deleted the rpc.py and updated the new patch. Please visit it.

3. About my patch didn’t pass CI.  My patch didn’t show nothing including the “verified -1”. I didn’t know whether the CI executed, so I can’t track anything down.  Is there anyone who met this problem?

Regards,
Helloway


在 2018年1月17日，下午10:32，Luse, Paul E <paul.e.luse(a)intel.com<mailto:paul.e.luse(a)intel.com>> 写道：

Hi WeWe,

Thanks for the trello update, did you see my question on your patch? I went to look specifically at the changes to existing files to start with and it appears that your patch includes a copy of rpc.py basically duplicating it within the repo.  Did I miss something or is that what you intended?

Wrt  question on why your patch didn’t pass CI, it’s pretty easy to track these things down most of the time.  In the review, click on the link below where it says verified -1 and you will see a webpage with the status of each of the test machines.  Many of the are red and indicate failure.  Click on one and drill down to find the builg.log file and you’ll see some formatting issues:

========== Backtrace start: ==========

in ./autobuild.sh:23 -> main()
     ...
   18
   19  ./configure $config_params
   20
   21  timing_enter check_format
   22  if [ $SPDK_RUN_CHECK_FORMAT -eq 1 ]; then
=> 23          ./scripts/check_format.sh
   24  fi
   25  timing_exit check_format
   26
   27  timing_enter build_kmod
   28  if [ $SPDK_BUILD_IOAT_KMOD -eq 1 ]; then
     ...

========== Backtrace end ==========

You can run the script, ./scripts/check_format.sh, locally and correct the errors and resubmit.  We always ask that developers run this scripts in advance to make sure things are formatted.  Depending on whether the maintainers think the SPDK repo makes sense for this patch you’ll need some test code also � there is clearly a lot of value in this patch though and regardless of whether it lands here or is maintained elsewhere!

Use this list or the patch review if you have more questions on getting it to pass.

Thanks!
Paul



From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Wednesday, January 17, 2018 5:24 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] Add py-spdk client for SPDK

Hi, all
I have submitted the introduction of py-spdk on trello https://trello.com/c/Bn3DIG7F/87-add-py-spdk-client-for-spdk, please visit it. I will be grateful to your kindness.

Regards,

Helloway

在 2018年1月16日，上午11:30，We We <simple_hlw(a)163.com<mailto:simple_hlw(a)163.com>> 写道：

Hi, all
I have submitted the py-spdk(written in python) for more than six days. Why the code-review couldn’t pass and show nothing?
Is there anyone who can help me?

Thanks,
Helloway

在 2018年1月10日，下午9:46，We We <simple_hlw(a)163.com<mailto:simple_hlw(a)163.com>> 写道：

Hi, all
I have submitted the py-spdk code onhttps://review.gerrithub.io/#/c/379741/, please take some time to visit it, I will be very grateful to you.
The py-spdk is client which can help the upper-level app to communicate with the SPDK-based app (such as: nvmf_tgt, vhost, iscsi_tgt, etc.). Should I submit it into the other repo I rebuild rather than SPDK repo? Because I think it is a relatively independent kit upon the SPDK.

If you have some thoughts about the py-spdk, please share with me.

Regards,
Helloway
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk


_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk


_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk



--
Zhipeng (Howard) Huang

Standard Engineer
IT Standard & Patent/IT Product Line
Huawei Technologies Co,. Ltd
Email: huangzhipeng(a)huawei.com<mailto:huangzhipeng(a)huawei.com>
Office: Huawei Industrial Base, Longgang, Shenzhen

(Previous)
Research Assistant
Mobile Ad-Hoc Network Lab, Calit2
University of California, Irvine
Email: zhipengh(a)uci.edu<mailto:zhipengh(a)uci.edu>
Office: Calit2 Building Room 2402

OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado

_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk



--
Zhipeng (Howard) Huang

Standard Engineer
IT Standard & Patent/IT Product Line
Huawei Technologies Co,. Ltd
Email: huangzhipeng(a)huawei.com<mailto:huangzhipeng(a)huawei.com>
Office: Huawei Industrial Base, Longgang, Shenzhen

(Previous)
Research Assistant
Mobile Ad-Hoc Network Lab, Calit2
University of California, Irvine
Email: zhipengh(a)uci.edu<mailto:zhipengh(a)uci.edu>
Office: Calit2 Building Room 2402

OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado

_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk



--
Zhipeng (Howard) Huang

Standard Engineer
IT Standard & Patent/IT Product Line
Huawei Technologies Co,. Ltd
Email: huangzhipeng(a)huawei.com<mailto:huangzhipeng(a)huawei.com>
Office: Huawei Industrial Base, Longgang, Shenzhen

(Previous)
Research Assistant
Mobile Ad-Hoc Network Lab, Calit2
University of California, Irvine
Email: zhipengh(a)uci.edu<mailto:zhipengh(a)uci.edu>
Office: Calit2 Building Room 2402

OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk


[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 74777 bytes --]

Re: [SPDK] Add py-spdk client for SPDK

[Tushar Gohad]

[-- Attachment #1: Type: text/plain, Size: 12529 bytes --]

Hi Helloway,

We are working with the Cinder / Nvoa community on baseline
NVMe-over-Fabrics support (Linux kernel based NVMe-oF target) at the
moment, which we anticipate should merge by February (it's in code review
upstream at the moment for Openstack Queens release).  We are working, in
parallel, on -- 1) a Cinder NVMe-oF prototype based on SPDK lvol, derived
from the existing Cinder work upstream [1,2], 2) addressing gaps in lvol
feature-set against Cinder requirements for new volume drivers.

You can track progress on Trello [3].  We'll add more detail here in the
next few days, including if/how we could use py-spdk as part of this
prototype.

Thanks,
Tushar


[1] - https://review.openstack.org/#/c/504944/17
[2] - https://review.openstack.org/#/c/504944/17
[3] - https://trello.com/b/g45xPI4D/cinder-integration



On Thu, Jan 18, 2018 at 8:12 AM, We We <simple_hlw(a)163.com> wrote:

> Hi Maciek,
>
> Thanks!! I am very happy to receive your kind feedback.
> In my opinion,  I indeed want to put the py-spdk into the spdk/py-spdk
> repo rathan than spdk/spdk repo. Because I think the py-spdk is built upon
> the SPDK which can provide a general python binding for spdk framework. I
> don't know whether I keep consistent with you. Please let me know what you
> think. And is it convenient for you to tell me what you have done with SPDK
> and Cinder?
> Any thoughts are appreciated.
>
> Regards,
> Helloway
>
>
>
> 在 2018年1月18日，下午10:22，Szwed, Maciej <maciej.szwed(a)intel.com> 写道：
>
> Hello WeWe,
> I’ve looked closer at your patch and it looks good (some minor tweaks
> maybe, some error handling missing), but before I comment it on Gerrit we
> have make decision where we want to keep it. I would rather see it outside
> spdk/spdk repo. There is some work ongoing to connect SPDK with Cinder. I
> think that these two could end up in same repository.
>
> Regards,
> Maciek
>
> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org <spdk-bounces(a)lists.01.org>
> ] *On Behalf Of *Zhipeng Huang
> *Sent:* Thursday, January 18, 2018 3:01 AM
> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
> *Subject:* Re: [SPDK] Add py-spdk client for SPDK
>
> Thanks and Wewe is working on to fix that in the next patch :)
>
> As she stated earlier, we think it is better to have a standalone repo
> hosting all the different lang bindings for spdk under
> https://github.com/spdk , similar example could be found at openstack
> sdk, so that we don't blow up the main spdk repo
>
> On Thu, Jan 18, 2018 at 9:49 AM, Luse, Paul E <paul.e.luse(a)intel.com>
> wrote:
>
> FYI there are other format issues as well, if you look at some of the
> other test machines you’ll see this, for example, on the patch:
>
> Checking coding style... OK
> Checking comment style... OK
> Checking blank lines at end of file... Incorrect end-of-file formatting
> detected
> py-spdk/README.md: Extra trailing newline
> Checking for POSIX includes... OK
> Checking Python style... OK
>
>
> All very easy to catch & fix locally J
>
> Thx
> Paul
>
> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org] *On Behalf Of *Zhipeng
> Huang
> *Sent:* Wednesday, January 17, 2018 6:33 PM
>
> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
> *Subject:* Re: [SPDK] Add py-spdk client for SPDK
>
> Hi Paul,
>
> Thanks for getting the CI running. Seems most of the problems caused by
> pep8 missing. Is it possible to add python support in the CI system ?
> otherwise we will hit many pip pkg missing problems.
>
> On Thu, Jan 18, 2018 at 8:17 AM, Luse, Paul E <paul.e.luse(a)intel.com>
> wrote:
>
> Hmm yeah I’m not sure why it ran on patch set 2 but not the last couple…
> Seth, anyone?
>
> We might need one of the maintainers to clear it for CI execution, it’s an
> extra unfortunate security requirement we have on the test pool, otherwise
> its fully automated
>
> Thx
> Paul
>
> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org] *On Behalf Of *Zhipeng
> Huang
> *Sent:* Wednesday, January 17, 2018 4:04 PM
>
> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
> *Subject:* Re: [SPDK] Add py-spdk client for SPDK
>
> Paul, the problem is that Wewe's later patches doesn't trigger CI at all.
> Is it how it supposed to work that we should fix the problem before hand ?
> Since in OpenStack or kubernetes CI should be triggered everytime the
> patches come in.
>
> On Thu, Jan 18, 2018 at 5:39 AM, Luse, Paul E <paul.e.luse(a)intel.com>
> wrote:
>
> Hi WeWe
>
>
> So wrt which repo, I didn’t see that but we don’t really have a precedent
> for subprojects like that.  I’ll let one of the maintainers comment on the
> options but I believe they would be (a) maintain on your own elsewhere or
> (b) include in SPDK like any other code.  Option (b) is really for “core”
> SPDK elements that make sense for most use cases and a python binding may
> or may not fit, it clearly isn’t out of the question in my mind though as
> it’s not directly tied to OpenStack.
>
> On your patch not passing, see the screenshot below where I highlighted
> the -1 from the CI system.  If you click on that link it will take you to
> the results and you can see the failure I mention.  Have you tried running
> the check_format.sh locally?
>
> Thx
> Paul
>
>
> <image001.png>
>
> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org] *On Behalf Of *We We
> *Sent:* Wednesday, January 17, 2018 10:26 AM
>
> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
> *Subject:* Re: [SPDK] Add py-spdk client for SPDK
>
> Hi, Paul
> I am sorry.  Is  the question on my patch you mentioned  the following Q1?
> Q1: is your intention to propose this as part of the SPDK repo or do you
> intent to maintain this in your own repo and are simply asking for review
> comments?
> If so, I see it and response to you on the patch.  Maybe you didn’t see
> it.
>
> 1. About Q1, my idea is to treat the py-spdk as another subrepo in the
> SPDK community. For example: SPDK repo -->https://github.com/spdk/spdk.git,
> py-spdk repo -->https://github.com/spdk/py-spdk.git. Am I reasonable？
>
>
> 2. About a copy of rpc.py. Thank you for your careful review, the py-spdk
> doesn’t need to modify and copy the files of SPDK, I have deleted the
> rpc.py and updated the new patch. Please visit it.
>
>
> 3. About my patch didn’t pass CI.  My patch didn’t show nothing including
> the “verified -1”. I didn’t know whether the CI executed, so I can’t
> track anything down.  Is there anyone who met this problem?
>
> Regards,
> Helloway
>
>
>
>
> 在 2018年1月17日，下午10:32，Luse, Paul E <paul.e.luse(a)intel.com> 写道：
>
> Hi WeWe,
>
> Thanks for the trello update, did you see my question on your patch? I
> went to look specifically at the changes to existing files to start with
> and it appears that your patch includes a copy of rpc.py basically
> duplicating it within the repo.  Did I miss something or is that what you
> intended?
>
> Wrt  question on why your patch didn’t pass CI, it’s pretty easy to track
> these things down most of the time.  In the review, click on the link below
> where it says verified -1 and you will see a webpage with the status of
> each of the test machines.  Many of the are red and indicate failure.
> Click on one and drill down to find the builg.log file and you’ll see some
> formatting issues:
>
> ========== Backtrace start: ==========
>
> in ./autobuild.sh:23 -> main()
>      ...
>    18
>    19  ./configure $config_params
>    20
>    21  timing_enter check_format
>    22  if [ $SPDK_RUN_CHECK_FORMAT -eq 1 ]; then
> => 23          ./scripts/check_format.sh
>    24  fi
>    25  timing_exit check_format
>    26
>    27  timing_enter build_kmod
>    28  if [ $SPDK_BUILD_IOAT_KMOD -eq 1 ]; then
>      ...
>
> ========== Backtrace end ==========
>
> You can run the script, ./scripts/check_format.sh, locally and correct
> the errors and resubmit.  We always ask that developers run this scripts in
> advance to make sure things are formatted.  Depending on whether the
> maintainers think the SPDK repo makes sense for this patch you’ll need some
> test code also – there is clearly a lot of value in this patch though and
> regardless of whether it lands here or is maintained elsewhere!
>
> Use this list or the patch review if you have more questions on getting it
> to pass.
>
> Thanks!
> Paul
>
>
>
> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org <spdk-bounces(a)lists.01.org>
> ] *On Behalf Of *We We
> *Sent:* Wednesday, January 17, 2018 5:24 AM
> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
> *Subject:* Re: [SPDK] Add py-spdk client for SPDK
>
> Hi, all
> I have submitted the introduction of py-spdk on trello
> https://trello.com/c/Bn3DIG7F/87-add-py-spdk-client-for-spdk, please
> visit it. I will be grateful to your kindness.
>
> Regards,
>
> Helloway
>
>
> 在 2018年1月16日，上午11:30，We We <simple_hlw(a)163.com> 写道：
>
> Hi, all
> I have submitted the py-spdk(written in python) for more than six days.
> Why the code-review couldn’t pass and show nothing?
> Is there anyone who can help me?
>
> Thanks,
> Helloway
>
>
> 在 2018年1月10日，下午9:46，We We <simple_hlw(a)163.com> 写道：
>
> Hi, all
> I have submitted the py-spdk code onhttps://review.gerrithub.io/
> #/c/379741/, please take some time to visit it, I will be very grateful
> to you.
> The py-spdk is client which can help the upper-level app to communicate
> with the SPDK-based app (such as: nvmf_tgt, vhost, iscsi_tgt, etc.). Should
> I submit it into the other repo I rebuild rather than SPDK repo? Because I
> think it is a relatively independent kit upon the SPDK.
>
> If you have some thoughts about the py-spdk, please share with me.
>
> Regards,
> Helloway
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
>
>
> --
> Zhipeng (Howard) Huang
>
> Standard Engineer
> IT Standard & Patent/IT Product Line
> Huawei Technologies Co,. Ltd
> Email: huangzhipeng(a)huawei.com
> Office: Huawei Industrial Base, Longgang, Shenzhen
>
> (Previous)
> Research Assistant
> Mobile Ad-Hoc Network Lab, Calit2
> University of California, Irvine
> Email: zhipengh(a)uci.edu
> Office: Calit2 Building Room 2402
>
> OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
>
>
> --
> Zhipeng (Howard) Huang
>
> Standard Engineer
> IT Standard & Patent/IT Product Line
> Huawei Technologies Co,. Ltd
> Email: huangzhipeng(a)huawei.com
> Office: Huawei Industrial Base, Longgang, Shenzhen
>
> (Previous)
> Research Assistant
> Mobile Ad-Hoc Network Lab, Calit2
> University of California, Irvine
> Email: zhipengh(a)uci.edu
> Office: Calit2 Building Room 2402
>
> OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
>
>
> --
> Zhipeng (Howard) Huang
>
> Standard Engineer
> IT Standard & Patent/IT Product Line
> Huawei Technologies Co,. Ltd
> Email: huangzhipeng(a)huawei.com
> Office: Huawei Industrial Base, Longgang, Shenzhen
>
> (Previous)
> Research Assistant
> Mobile Ad-Hoc Network Lab, Calit2
> University of California, Irvine
> Email: zhipengh(a)uci.edu
> Office: Calit2 Building Room 2402
>
> OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 71659 bytes --]

Re: [SPDK] Add py-spdk client for SPDK

[We We]

[-- Attachment #1: Type: text/plain, Size: 12642 bytes --]

Hi Maciek,

Thanks!! I am very happy to receive your kind feedback. 
In my opinion,  I indeed want to put the py-spdk into the spdk/py-spdk repo rathan than spdk/spdk repo. Because I think the py-spdk is built upon the SPDK which can provide a general python binding for spdk framework. I don't know whether I keep consistent with you. Please let me know what you think. And is it convenient for you to tell me what you have done with SPDK and Cinder?
Any thoughts are appreciated.

Regards,
Helloway

    

> 在 2018年1月18日，下午10:22，Szwed, Maciej <maciej.szwed(a)intel.com <mailto:maciej.szwed(a)intel.com>> 写道：
> 
> Hello WeWe,
> I’ve looked closer at your patch and it looks good (some minor tweaks maybe, some error handling missing), but before I comment it on Gerrit we have make decision where we want to keep it. I would rather see it outside spdk/spdk repo. There is some work ongoing to connect SPDK with Cinder. I think that these two could end up in same repository.
>  
> Regards,
> Maciek
>   <>
>  <>From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of Zhipeng Huang
> Sent: Thursday, January 18, 2018 3:01 AM
> To: Storage Performance Development Kit <spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>>
> Subject: Re: [SPDK] Add py-spdk client for SPDK
>  
> Thanks and Wewe is working on to fix that in the next patch :)
>  
> As she stated earlier, we think it is better to have a standalone repo hosting all the different lang bindings for spdk under https://github.com/spdk <https://github.com/spdk> , similar example could be found at openstack sdk, so that we don't blow up the main spdk repo
>  
> On Thu, Jan 18, 2018 at 9:49 AM, Luse, Paul E <paul.e.luse(a)intel.com <mailto:paul.e.luse(a)intel.com>> wrote:
> FYI there are other format issues as well, if you look at some of the other test machines you’ll see this, for example, on the patch:
>  
> Checking coding style... OK
> Checking comment style... OK
> Checking blank lines at end of file... Incorrect end-of-file formatting detected
> py-spdk/README.md: Extra trailing newline
> Checking for POSIX includes... OK
> Checking Python style... OK
>  
>  
> All very easy to catch & fix locally J
>  
> Thx
> Paul
>   <>
>  <>From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of Zhipeng Huang
> Sent: Wednesday, January 17, 2018 6:33 PM
> 
> To: Storage Performance Development Kit <spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>>
> Subject: Re: [SPDK] Add py-spdk client for SPDK
>  
> Hi Paul,
>  
> Thanks for getting the CI running. Seems most of the problems caused by pep8 missing. Is it possible to add python support in the CI system ? otherwise we will hit many pip pkg missing problems.
>  
> On Thu, Jan 18, 2018 at 8:17 AM, Luse, Paul E <paul.e.luse(a)intel.com <mailto:paul.e.luse(a)intel.com>> wrote:
> Hmm yeah I’m not sure why it ran on patch set 2 but not the last couple… Seth, anyone?
>  
> We might need one of the maintainers to clear it for CI execution, it’s an extra unfortunate security requirement we have on the test pool, otherwise its fully automated
>  
> Thx
> Paul
>  
> From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of Zhipeng Huang
> Sent: Wednesday, January 17, 2018 4:04 PM
> 
> To: Storage Performance Development Kit <spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>>
> Subject: Re: [SPDK] Add py-spdk client for SPDK
>  
> Paul, the problem is that Wewe's later patches doesn't trigger CI at all. Is it how it supposed to work that we should fix the problem before hand ?
> Since in OpenStack or kubernetes CI should be triggered everytime the patches come in.
>  
> On Thu, Jan 18, 2018 at 5:39 AM, Luse, Paul E <paul.e.luse(a)intel.com <mailto:paul.e.luse(a)intel.com>> wrote:
> Hi WeWe
>  
>  
> So wrt which repo, I didn’t see that but we don’t really have a precedent for subprojects like that.  I’ll let one of the maintainers comment on the options but I believe they would be (a) maintain on your own elsewhere or (b) include in SPDK like any other code.  Option (b) is really for “core” SPDK elements that make sense for most use cases and a python binding may or may not fit, it clearly isn’t out of the question in my mind though as it’s not directly tied to OpenStack.
>  
> On your patch not passing, see the screenshot below where I highlighted the -1 from the CI system.  If you click on that link it will take you to the results and you can see the failure I mention.  Have you tried running the check_format.sh locally?
>  
> Thx
> Paul
>  
>  
> <image001.png>
>  
>  <>From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of We We
> Sent: Wednesday, January 17, 2018 10:26 AM
> 
> To: Storage Performance Development Kit <spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>>
> Subject: Re: [SPDK] Add py-spdk client for SPDK
>  
> Hi, Paul
> I am sorry.  Is  the question on my patch you mentioned  the following Q1?
> Q1: is your intention to propose this as part of the SPDK repo or do you intent to maintain this in your own repo and are simply asking for review comments?
> If so, I see it and response to you on the patch.  Maybe you didn’t see it. 
>  
> 1. About Q1, my idea is to treat the py-spdk as another subrepo in the SPDK community. For example: SPDK repo -->https://github.com/spdk/spdk.git <https://github.com/spdk/spdk.git>, py-spdk repo -->https://github.com/spdk/py-spdk.git <https://github.com/spdk/py-spdk.git>. Am I reasonable？
>  
> 
> 2. About a copy of rpc.py. Thank you for your careful review, the py-spdk doesn’t need to modify and copy the files of SPDK, I have deleted the rpc.py and updated the new patch. Please visit it.
>  
> 
> 3. About my patch didn’t pass CI.  My patch didn’t show nothing including the “verified -1”. I didn’t know whether the CI executed, so I can’t track anything down.  Is there anyone who met this problem?
>  
> Regards,
> Helloway
>  
>  
> 
> 在 2018年1月17日，下午10:32，Luse, Paul E <paul.e.luse(a)intel.com <mailto:paul.e.luse(a)intel.com>> 写道：
>  
> Hi WeWe,
>  
> Thanks for the trello update, did you see my question on your patch? I went to look specifically at the changes to existing files to start with and it appears that your patch includes a copy of rpc.py basically duplicating it within the repo.  Did I miss something or is that what you intended?
>  
> Wrt  question on why your patch didn’t pass CI, it’s pretty easy to track these things down most of the time.  In the review, click on the link below where it says verified -1 and you will see a webpage with the status of each of the test machines.  Many of the are red and indicate failure.  Click on one and drill down to find the builg.log file and you’ll see some formatting issues:
>  
> ========== Backtrace start: ==========
>  
> in ./autobuild.sh:23 -> main()
>      ...
>    18 
>    19  ./configure $config_params
>    20 
>    21  timing_enter check_format
>    22  if [ $SPDK_RUN_CHECK_FORMAT -eq 1 ]; then
> => 23          ./scripts/check_format.sh
>    24  fi
>    25  timing_exit check_format
>    26 
>    27  timing_enter build_kmod
>    28  if [ $SPDK_BUILD_IOAT_KMOD -eq 1 ]; then
>      ...
>  
> ========== Backtrace end ==========
>  
> You can run the script, ./scripts/check_format.sh, locally and correct the errors and resubmit.  We always ask that developers run this scripts in advance to make sure things are formatted.  Depending on whether the maintainers think the SPDK repo makes sense for this patch you’ll need some test code also – there is clearly a lot of value in this patch though and regardless of whether it lands here or is maintained elsewhere!
>  
> Use this list or the patch review if you have more questions on getting it to pass.
>  
> Thanks!
> Paul
>  
>  
>  
> From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of We We
> Sent: Wednesday, January 17, 2018 5:24 AM
> To: Storage Performance Development Kit <spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>>
> Subject: Re: [SPDK] Add py-spdk client for SPDK
>  
> Hi, all
> I have submitted the introduction of py-spdk on trello https://trello.com/c/Bn3DIG7F/87-add-py-spdk-client-for-spdk <https://trello.com/c/Bn3DIG7F/87-add-py-spdk-client-for-spdk>, please visit it. I will be grateful to your kindness.
>  
> Regards,
>  
> Helloway
>  
> 在 2018年1月16日，上午11:30，We We <simple_hlw(a)163.com <mailto:simple_hlw(a)163.com>> 写道：
>  
> Hi, all
> I have submitted the py-spdk(written in python) for more than six days. Why the code-review couldn’t pass and show nothing?
> Is there anyone who can help me?
>  
> Thanks,
> Helloway
>  
> 在 2018年1月10日，下午9:46，We We <simple_hlw(a)163.com <mailto:simple_hlw(a)163.com>> 写道：
>  
> Hi, all
> I have submitted the py-spdk code onhttps://review.gerrithub.io/#/c/379741/ <https://review.gerrithub.io/#/c/379741/>, please take some time to visit it, I will be very grateful to you. 
> The py-spdk is client which can help the upper-level app to communicate with the SPDK-based app (such as: nvmf_tgt, vhost, iscsi_tgt, etc.). Should I submit it into the other repo I rebuild rather than SPDK repo? Because I think it is a relatively independent kit upon the SPDK. 
>  
> If you have some thoughts about the py-spdk, please share with me.
>  
> Regards,
> Helloway
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
>  
>  
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
>  
> 
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
> 
> 
>  
> --
> Zhipeng (Howard) Huang
>  
> Standard Engineer
> IT Standard & Patent/IT Product Line
> Huawei Technologies Co,. Ltd
> Email: huangzhipeng(a)huawei.com <mailto:huangzhipeng(a)huawei.com>
> Office: Huawei Industrial Base, Longgang, Shenzhen
>  
> (Previous)
> Research Assistant
> Mobile Ad-Hoc Network Lab, Calit2
> University of California, Irvine
> Email: zhipengh(a)uci.edu <mailto:zhipengh(a)uci.edu>
> Office: Calit2 Building Room 2402
>  
> OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado
> 
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
> 
> 
>  
> --
> Zhipeng (Howard) Huang
>  
> Standard Engineer
> IT Standard & Patent/IT Product Line
> Huawei Technologies Co,. Ltd
> Email: huangzhipeng(a)huawei.com <mailto:huangzhipeng(a)huawei.com>
> Office: Huawei Industrial Base, Longgang, Shenzhen
>  
> (Previous)
> Research Assistant
> Mobile Ad-Hoc Network Lab, Calit2
> University of California, Irvine
> Email: zhipengh(a)uci.edu <mailto:zhipengh(a)uci.edu>
> Office: Calit2 Building Room 2402
>  
> OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado
> 
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
> 
> 
>  
> -- 
> Zhipeng (Howard) Huang
>  
> Standard Engineer
> IT Standard & Patent/IT Product Line
> Huawei Technologies Co,. Ltd
> Email: huangzhipeng(a)huawei.com <mailto:huangzhipeng(a)huawei.com>
> Office: Huawei Industrial Base, Longgang, Shenzhen
>  
> (Previous)
> Research Assistant
> Mobile Ad-Hoc Network Lab, Calit2
> University of California, Irvine
> Email: zhipengh(a)uci.edu <mailto:zhipengh(a)uci.edu>
> Office: Calit2 Building Room 2402
>  
> OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 79679 bytes --]

Re: [SPDK] Add py-spdk client for SPDK

[Szwed, Maciej]

[-- Attachment #1: Type: text/plain, Size: 10652 bytes --]

Hello WeWe,
I’ve looked closer at your patch and it looks good (some minor tweaks maybe, some error handling missing), but before I comment it on Gerrit we have make decision where we want to keep it. I would rather see it outside spdk/spdk repo. There is some work ongoing to connect SPDK with Cinder. I think that these two could end up in same repository.

Regards,
Maciek

From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Zhipeng Huang
Sent: Thursday, January 18, 2018 3:01 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: Re: [SPDK] Add py-spdk client for SPDK

Thanks and Wewe is working on to fix that in the next patch :)

As she stated earlier, we think it is better to have a standalone repo hosting all the different lang bindings for spdk under https://github.com/spdk , similar example could be found at openstack sdk, so that we don't blow up the main spdk repo

On Thu, Jan 18, 2018 at 9:49 AM, Luse, Paul E <paul.e.luse(a)intel.com<mailto:paul.e.luse(a)intel.com>> wrote:
FYI there are other format issues as well, if you look at some of the other test machines you’ll see this, for example, on the patch:

Checking coding style... OK
Checking comment style... OK
Checking blank lines at end of file... Incorrect end-of-file formatting detected
py-spdk/README.md: Extra trailing newline
Checking for POSIX includes... OK
Checking Python style... OK


All very easy to catch & fix locally ☺

Thx
Paul

From: SPDK [mailto:spdk-bounces(a)lists.01.org<mailto:spdk-bounces(a)lists.01.org>] On Behalf Of Zhipeng Huang
Sent: Wednesday, January 17, 2018 6:33 PM

To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] Add py-spdk client for SPDK

Hi Paul,

Thanks for getting the CI running. Seems most of the problems caused by pep8 missing. Is it possible to add python support in the CI system ? otherwise we will hit many pip pkg missing problems.

On Thu, Jan 18, 2018 at 8:17 AM, Luse, Paul E <paul.e.luse(a)intel.com<mailto:paul.e.luse(a)intel.com>> wrote:
Hmm yeah I’m not sure why it ran on patch set 2 but not the last couple… Seth, anyone?

We might need one of the maintainers to clear it for CI execution, it’s an extra unfortunate security requirement we have on the test pool, otherwise its fully automated

Thx
Paul

From: SPDK [mailto:spdk-bounces(a)lists.01.org<mailto:spdk-bounces(a)lists.01.org>] On Behalf Of Zhipeng Huang
Sent: Wednesday, January 17, 2018 4:04 PM

To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] Add py-spdk client for SPDK

Paul, the problem is that Wewe's later patches doesn't trigger CI at all. Is it how it supposed to work that we should fix the problem before hand ?
Since in OpenStack or kubernetes CI should be triggered everytime the patches come in.

On Thu, Jan 18, 2018 at 5:39 AM, Luse, Paul E <paul.e.luse(a)intel.com<mailto:paul.e.luse(a)intel.com>> wrote:
Hi WeWe


So wrt which repo, I didn’t see that but we don’t really have a precedent for subprojects like that.  I’ll let one of the maintainers comment on the options but I believe they would be (a) maintain on your own elsewhere or (b) include in SPDK like any other code.  Option (b) is really for “core” SPDK elements that make sense for most use cases and a python binding may or may not fit, it clearly isn’t out of the question in my mind though as it’s not directly tied to OpenStack.

On your patch not passing, see the screenshot below where I highlighted the -1 from the CI system.  If you click on that link it will take you to the results and you can see the failure I mention.  Have you tried running the check_format.sh locally?

Thx
Paul


[cid:image001.png(a)01D3906F.607B2890]

From: SPDK [mailto:spdk-bounces(a)lists.01.org<mailto:spdk-bounces(a)lists.01.org>] On Behalf Of We We
Sent: Wednesday, January 17, 2018 10:26 AM

To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] Add py-spdk client for SPDK

Hi, Paul
I am sorry.  Is  the question on my patch you mentioned  the following Q1?
Q1: is your intention to propose this as part of the SPDK repo or do you intent to maintain this in your own repo and are simply asking for review comments?
If so, I see it and response to you on the patch.  Maybe you didn’t see it.

1. About Q1, my idea is to treat the py-spdk as another subrepo in the SPDK community. For example: SPDK repo -->https://github.com/spdk/spdk.git, py-spdk repo -->https://github.com/spdk/py-spdk.git. Am I reasonable？

2. About a copy of rpc.py. Thank you for your careful review, the py-spdk doesn’t need to modify and copy the files of SPDK, I have deleted the rpc.py and updated the new patch. Please visit it.

3. About my patch didn’t pass CI.  My patch didn’t show nothing including the “verified -1”. I didn’t know whether the CI executed, so I can’t track anything down.  Is there anyone who met this problem?

Regards,
Helloway


在 2018年1月17日，下午10:32，Luse, Paul E <paul.e.luse(a)intel.com<mailto:paul.e.luse(a)intel.com>> 写道：

Hi WeWe,

Thanks for the trello update, did you see my question on your patch? I went to look specifically at the changes to existing files to start with and it appears that your patch includes a copy of rpc.py basically duplicating it within the repo.  Did I miss something or is that what you intended?

Wrt  question on why your patch didn’t pass CI, it’s pretty easy to track these things down most of the time.  In the review, click on the link below where it says verified -1 and you will see a webpage with the status of each of the test machines.  Many of the are red and indicate failure.  Click on one and drill down to find the builg.log file and you’ll see some formatting issues:

========== Backtrace start: ==========

in ./autobuild.sh:23 -> main()
     ...
   18
   19  ./configure $config_params
   20
   21  timing_enter check_format
   22  if [ $SPDK_RUN_CHECK_FORMAT -eq 1 ]; then
=> 23          ./scripts/check_format.sh
   24  fi
   25  timing_exit check_format
   26
   27  timing_enter build_kmod
   28  if [ $SPDK_BUILD_IOAT_KMOD -eq 1 ]; then
     ...

========== Backtrace end ==========

You can run the script, ./scripts/check_format.sh, locally and correct the errors and resubmit.  We always ask that developers run this scripts in advance to make sure things are formatted.  Depending on whether the maintainers think the SPDK repo makes sense for this patch you’ll need some test code also – there is clearly a lot of value in this patch though and regardless of whether it lands here or is maintained elsewhere!

Use this list or the patch review if you have more questions on getting it to pass.

Thanks!
Paul



From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Wednesday, January 17, 2018 5:24 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] Add py-spdk client for SPDK

Hi, all
I have submitted the introduction of py-spdk on trello https://trello.com/c/Bn3DIG7F/87-add-py-spdk-client-for-spdk, please visit it. I will be grateful to your kindness.

Regards,

Helloway

在 2018年1月16日，上午11:30，We We <simple_hlw(a)163.com<mailto:simple_hlw(a)163.com>> 写道：

Hi, all
I have submitted the py-spdk(written in python) for more than six days. Why the code-review couldn’t pass and show nothing?
Is there anyone who can help me?

Thanks,
Helloway

在 2018年1月10日，下午9:46，We We <simple_hlw(a)163.com<mailto:simple_hlw(a)163.com>> 写道：

Hi, all
I have submitted the py-spdk code onhttps://review.gerrithub.io/#/c/379741/, please take some time to visit it, I will be very grateful to you.
The py-spdk is client which can help the upper-level app to communicate with the SPDK-based app (such as: nvmf_tgt, vhost, iscsi_tgt, etc.). Should I submit it into the other repo I rebuild rather than SPDK repo? Because I think it is a relatively independent kit upon the SPDK.

If you have some thoughts about the py-spdk, please share with me.

Regards,
Helloway
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk


_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk


_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk



--
Zhipeng (Howard) Huang

Standard Engineer
IT Standard & Patent/IT Product Line
Huawei Technologies Co,. Ltd
Email: huangzhipeng(a)huawei.com<mailto:huangzhipeng(a)huawei.com>
Office: Huawei Industrial Base, Longgang, Shenzhen

(Previous)
Research Assistant
Mobile Ad-Hoc Network Lab, Calit2
University of California, Irvine
Email: zhipengh(a)uci.edu<mailto:zhipengh(a)uci.edu>
Office: Calit2 Building Room 2402

OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado

_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk



--
Zhipeng (Howard) Huang

Standard Engineer
IT Standard & Patent/IT Product Line
Huawei Technologies Co,. Ltd
Email: huangzhipeng(a)huawei.com<mailto:huangzhipeng(a)huawei.com>
Office: Huawei Industrial Base, Longgang, Shenzhen

(Previous)
Research Assistant
Mobile Ad-Hoc Network Lab, Calit2
University of California, Irvine
Email: zhipengh(a)uci.edu<mailto:zhipengh(a)uci.edu>
Office: Calit2 Building Room 2402

OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado

_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk



--
Zhipeng (Howard) Huang

Standard Engineer
IT Standard & Patent/IT Product Line
Huawei Technologies Co,. Ltd
Email: huangzhipeng(a)huawei.com<mailto:huangzhipeng(a)huawei.com>
Office: Huawei Industrial Base, Longgang, Shenzhen

(Previous)
Research Assistant
Mobile Ad-Hoc Network Lab, Calit2
University of California, Irvine
Email: zhipengh(a)uci.edu<mailto:zhipengh(a)uci.edu>
Office: Calit2 Building Room 2402

OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 59552 bytes --]

[-- Attachment #3: image001.png --]
[-- Type: image/png, Size: 108863 bytes --]

Re: [SPDK] Add py-spdk client for SPDK

[Zhipeng Huang]

[-- Attachment #1: Type: text/plain, Size: 10371 bytes --]

Thanks and Wewe is working on to fix that in the next patch :)

As she stated earlier, we think it is better to have a standalone repo
hosting all the different lang bindings for spdk under
https://github.com/spdk , similar example could be found at openstack sdk,
so that we don't blow up the main spdk repo

On Thu, Jan 18, 2018 at 9:49 AM, Luse, Paul E <paul.e.luse(a)intel.com> wrote:

> FYI there are other format issues as well, if you look at some of the
> other test machines you’ll see this, for example, on the patch:
>
>
>
> Checking coding style... OK
>
> Checking comment style... OK
>
> Checking blank lines at end of file... Incorrect end-of-file formatting
> detected
>
> py-spdk/README.md: Extra trailing newline
>
> Checking for POSIX includes... OK
>
> Checking Python style... OK
>
>
>
>
>
> All very easy to catch & fix locally J
>
>
>
> Thx
>
> Paul
>
>
>
> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org] *On Behalf Of *Zhipeng
> Huang
> *Sent:* Wednesday, January 17, 2018 6:33 PM
>
> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
> *Subject:* Re: [SPDK] Add py-spdk client for SPDK
>
>
>
> Hi Paul,
>
>
>
> Thanks for getting the CI running. Seems most of the problems caused by
> pep8 missing. Is it possible to add python support in the CI system ?
> otherwise we will hit many pip pkg missing problems.
>
>
>
> On Thu, Jan 18, 2018 at 8:17 AM, Luse, Paul E <paul.e.luse(a)intel.com>
> wrote:
>
> Hmm yeah I’m not sure why it ran on patch set 2 but not the last couple…
> Seth, anyone?
>
>
>
> We might need one of the maintainers to clear it for CI execution, it’s an
> extra unfortunate security requirement we have on the test pool, otherwise
> its fully automated
>
>
>
> Thx
> Paul
>
>
>
> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org] *On Behalf Of *Zhipeng
> Huang
> *Sent:* Wednesday, January 17, 2018 4:04 PM
>
>
> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
> *Subject:* Re: [SPDK] Add py-spdk client for SPDK
>
>
>
> Paul, the problem is that Wewe's later patches doesn't trigger CI at all.
> Is it how it supposed to work that we should fix the problem before hand ?
>
> Since in OpenStack or kubernetes CI should be triggered everytime the
> patches come in.
>
>
>
> On Thu, Jan 18, 2018 at 5:39 AM, Luse, Paul E <paul.e.luse(a)intel.com>
> wrote:
>
> Hi WeWe
>
>
>
>
>
> So wrt which repo, I didn’t see that but we don’t really have a precedent
> for subprojects like that.  I’ll let one of the maintainers comment on the
> options but I believe they would be (a) maintain on your own elsewhere or
> (b) include in SPDK like any other code.  Option (b) is really for “core”
> SPDK elements that make sense for most use cases and a python binding may
> or may not fit, it clearly isn’t out of the question in my mind though as
> it’s not directly tied to OpenStack.
>
>
>
> On your patch not passing, see the screenshot below where I highlighted
> the -1 from the CI system.  If you click on that link it will take you to
> the results and you can see the failure I mention.  Have you tried running
> the check_format.sh locally?
>
>
>
> Thx
>
> Paul
>
>
>
>
>
>
>
> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org] *On Behalf Of *We We
> *Sent:* Wednesday, January 17, 2018 10:26 AM
>
>
> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
> *Subject:* Re: [SPDK] Add py-spdk client for SPDK
>
>
>
> Hi, Paul
>
> I am sorry.  Is  the question on my patch you mentioned  the following Q1?
>
> Q1: is your intention to propose this as part of the SPDK repo or do you
> intent to maintain this in your own repo and are simply asking for review
> comments?
>
> If so, I see it and response to you on the patch.  Maybe you didn’t see
> it.
>
>
>
> 1. About Q1, my idea is to treat the py-spdk as another subrepo in the
> SPDK community. For example: SPDK repo -->https://github.com/spdk/spdk.git,
> py-spdk repo -->https://github.com/spdk/py-spdk.git. Am I reasonable？
>
>
>
> 2. About a copy of rpc.py. Thank you for your careful review, the py-spdk
> doesn’t need to modify and copy the files of SPDK, I have deleted the
> rpc.py and updated the new patch. Please visit it.
>
>
>
> 3. About my patch didn’t pass CI.  My patch didn’t show nothing including
> the “verified -1”. I didn’t know whether the CI executed, so I can’t
> track anything down.  Is there anyone who met this problem?
>
>
>
> Regards,
>
> Helloway
>
>
>
>
>
> 在 2018年1月17日，下午10:32，Luse, Paul E <paul.e.luse(a)intel.com> 写道：
>
>
>
> Hi WeWe,
>
>
>
> Thanks for the trello update, did you see my question on your patch? I
> went to look specifically at the changes to existing files to start with
> and it appears that your patch includes a copy of rpc.py basically
> duplicating it within the repo.  Did I miss something or is that what you
> intended?
>
>
>
> Wrt  question on why your patch didn’t pass CI, it’s pretty easy to track
> these things down most of the time.  In the review, click on the link below
> where it says verified -1 and you will see a webpage with the status of
> each of the test machines.  Many of the are red and indicate failure.
> Click on one and drill down to find the builg.log file and you’ll see some
> formatting issues:
>
>
>
> ========== Backtrace start: ==========
>
>
>
> in ./autobuild.sh:23 -> main()
>
>      ...
>
>    18
>
>    19  ./configure $config_params
>
>    20
>
>    21  timing_enter check_format
>
>    22  if [ $SPDK_RUN_CHECK_FORMAT -eq 1 ]; then
>
> => 23          ./scripts/check_format.sh
>
>    24  fi
>
>    25  timing_exit check_format
>
>    26
>
>    27  timing_enter build_kmod
>
>    28  if [ $SPDK_BUILD_IOAT_KMOD -eq 1 ]; then
>
>      ...
>
>
>
> ========== Backtrace end ==========
>
>
>
> You can run the script, ./scripts/check_format.sh, locally and correct
> the errors and resubmit.  We always ask that developers run this scripts in
> advance to make sure things are formatted.  Depending on whether the
> maintainers think the SPDK repo makes sense for this patch you’ll need some
> test code also – there is clearly a lot of value in this patch though and
> regardless of whether it lands here or is maintained elsewhere!
>
>
>
> Use this list or the patch review if you have more questions on getting it
> to pass.
>
>
>
> Thanks!
>
> Paul
>
>
>
>
>
>
>
> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org <spdk-bounces(a)lists.01.org>
> ] *On Behalf Of *We We
> *Sent:* Wednesday, January 17, 2018 5:24 AM
> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
> *Subject:* Re: [SPDK] Add py-spdk client for SPDK
>
>
>
> Hi, all
>
> I have submitted the introduction of py-spdk on trello
> https://trello.com/c/Bn3DIG7F/87-add-py-spdk-client-for-spdk, please
> visit it. I will be grateful to your kindness.
>
>
>
> Regards,
>
>
>
> Helloway
>
>
>
> 在 2018年1月16日，上午11:30，We We <simple_hlw(a)163.com> 写道：
>
>
>
> Hi, all
>
> I have submitted the py-spdk(written in python) for more than six days.
> Why the code-review couldn’t pass and show nothing?
>
> Is there anyone who can help me?
>
>
>
> Thanks,
>
> Helloway
>
>
>
> 在 2018年1月10日，下午9:46，We We <simple_hlw(a)163.com> 写道：
>
>
>
> Hi, all
>
> I have submitted the py-spdk code onhttps://review.gerrithub.io/
> #/c/379741/, please take some time to visit it, I will be very grateful
> to you.
>
> The py-spdk is client which can help the upper-level app to communicate
> with the SPDK-based app (such as: nvmf_tgt, vhost, iscsi_tgt, etc.). Should
> I submit it into the other repo I rebuild rather than SPDK repo? Because I
> think it is a relatively independent kit upon the SPDK.
>
>
>
> If you have some thoughts about the py-spdk, please share with me.
>
>
>
> Regards,
>
> Helloway
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
>
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
>
>
>
> --
>
> Zhipeng (Howard) Huang
>
>
>
> Standard Engineer
>
> IT Standard & Patent/IT Product Line
>
> Huawei Technologies Co,. Ltd
>
> Email: huangzhipeng(a)huawei.com
>
> Office: Huawei Industrial Base, Longgang, Shenzhen
>
>
>
> (Previous)
>
> Research Assistant
>
> Mobile Ad-Hoc Network Lab, Calit2
>
> University of California, Irvine
>
> Email: zhipengh(a)uci.edu
>
> Office: Calit2 Building Room 2402
>
>
>
> OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
>
>
>
> --
>
> Zhipeng (Howard) Huang
>
>
>
> Standard Engineer
>
> IT Standard & Patent/IT Product Line
>
> Huawei Technologies Co,. Ltd
>
> Email: huangzhipeng(a)huawei.com
>
> Office: Huawei Industrial Base, Longgang, Shenzhen
>
>
>
> (Previous)
>
> Research Assistant
>
> Mobile Ad-Hoc Network Lab, Calit2
>
> University of California, Irvine
>
> Email: zhipengh(a)uci.edu
>
> Office: Calit2 Building Room 2402
>
>
>
> OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>


-- 
Zhipeng (Howard) Huang

Standard Engineer
IT Standard & Patent/IT Product Line
Huawei Technologies Co,. Ltd
Email: huangzhipeng(a)huawei.com
Office: Huawei Industrial Base, Longgang, Shenzhen

(Previous)
Research Assistant
Mobile Ad-Hoc Network Lab, Calit2
University of California, Irvine
Email: zhipengh(a)uci.edu
Office: Calit2 Building Room 2402

OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 36708 bytes --]

[-- Attachment #3: image001.png --]
[-- Type: image/png, Size: 108863 bytes --]

Re: [SPDK] Add py-spdk client for SPDK

[Luse, Paul E]

[-- Attachment #1: Type: text/plain, Size: 8887 bytes --]

FYI there are other format issues as well, if you look at some of the other test machines you’ll see this, for example, on the patch:

Checking coding style... OK
Checking comment style... OK
Checking blank lines at end of file... Incorrect end-of-file formatting detected
py-spdk/README.md: Extra trailing newline
Checking for POSIX includes... OK
Checking Python style... OK


All very easy to catch & fix locally ☺

Thx
Paul

From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Zhipeng Huang
Sent: Wednesday, January 17, 2018 6:33 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: Re: [SPDK] Add py-spdk client for SPDK

Hi Paul,

Thanks for getting the CI running. Seems most of the problems caused by pep8 missing. Is it possible to add python support in the CI system ? otherwise we will hit many pip pkg missing problems.

On Thu, Jan 18, 2018 at 8:17 AM, Luse, Paul E <paul.e.luse(a)intel.com<mailto:paul.e.luse(a)intel.com>> wrote:
Hmm yeah I’m not sure why it ran on patch set 2 but not the last couple… Seth, anyone?

We might need one of the maintainers to clear it for CI execution, it’s an extra unfortunate security requirement we have on the test pool, otherwise its fully automated

Thx
Paul

From: SPDK [mailto:spdk-bounces(a)lists.01.org<mailto:spdk-bounces(a)lists.01.org>] On Behalf Of Zhipeng Huang
Sent: Wednesday, January 17, 2018 4:04 PM

To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] Add py-spdk client for SPDK

Paul, the problem is that Wewe's later patches doesn't trigger CI at all. Is it how it supposed to work that we should fix the problem before hand ?
Since in OpenStack or kubernetes CI should be triggered everytime the patches come in.

On Thu, Jan 18, 2018 at 5:39 AM, Luse, Paul E <paul.e.luse(a)intel.com<mailto:paul.e.luse(a)intel.com>> wrote:
Hi WeWe


So wrt which repo, I didn’t see that but we don’t really have a precedent for subprojects like that.  I’ll let one of the maintainers comment on the options but I believe they would be (a) maintain on your own elsewhere or (b) include in SPDK like any other code.  Option (b) is really for “core” SPDK elements that make sense for most use cases and a python binding may or may not fit, it clearly isn’t out of the question in my mind though as it’s not directly tied to OpenStack.

On your patch not passing, see the screenshot below where I highlighted the -1 from the CI system.  If you click on that link it will take you to the results and you can see the failure I mention.  Have you tried running the check_format.sh locally?

Thx
Paul


[cid:image001.png(a)01D38FC3.B54837C0]

From: SPDK [mailto:spdk-bounces(a)lists.01.org<mailto:spdk-bounces(a)lists.01.org>] On Behalf Of We We
Sent: Wednesday, January 17, 2018 10:26 AM

To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] Add py-spdk client for SPDK

Hi, Paul
I am sorry.  Is  the question on my patch you mentioned  the following Q1?
Q1: is your intention to propose this as part of the SPDK repo or do you intent to maintain this in your own repo and are simply asking for review comments?
If so, I see it and response to you on the patch.  Maybe you didn’t see it.

1. About Q1, my idea is to treat the py-spdk as another subrepo in the SPDK community. For example: SPDK repo -->https://github.com/spdk/spdk.git, py-spdk repo -->https://github.com/spdk/py-spdk.git. Am I reasonable？

2. About a copy of rpc.py. Thank you for your careful review, the py-spdk doesn’t need to modify and copy the files of SPDK, I have deleted the rpc.py and updated the new patch. Please visit it.

3. About my patch didn’t pass CI.  My patch didn’t show nothing including the “verified -1”. I didn’t know whether the CI executed, so I can’t track anything down.  Is there anyone who met this problem?

Regards,
Helloway


在 2018年1月17日，下午10:32，Luse, Paul E <paul.e.luse(a)intel.com<mailto:paul.e.luse(a)intel.com>> 写道：

Hi WeWe,

Thanks for the trello update, did you see my question on your patch? I went to look specifically at the changes to existing files to start with and it appears that your patch includes a copy of rpc.py basically duplicating it within the repo.  Did I miss something or is that what you intended?

Wrt  question on why your patch didn’t pass CI, it’s pretty easy to track these things down most of the time.  In the review, click on the link below where it says verified -1 and you will see a webpage with the status of each of the test machines.  Many of the are red and indicate failure.  Click on one and drill down to find the builg.log file and you’ll see some formatting issues:

========== Backtrace start: ==========

in ./autobuild.sh:23 -> main()
     ...
   18
   19  ./configure $config_params
   20
   21  timing_enter check_format
   22  if [ $SPDK_RUN_CHECK_FORMAT -eq 1 ]; then
=> 23          ./scripts/check_format.sh
   24  fi
   25  timing_exit check_format
   26
   27  timing_enter build_kmod
   28  if [ $SPDK_BUILD_IOAT_KMOD -eq 1 ]; then
     ...

========== Backtrace end ==========

You can run the script, ./scripts/check_format.sh, locally and correct the errors and resubmit.  We always ask that developers run this scripts in advance to make sure things are formatted.  Depending on whether the maintainers think the SPDK repo makes sense for this patch you’ll need some test code also – there is clearly a lot of value in this patch though and regardless of whether it lands here or is maintained elsewhere!

Use this list or the patch review if you have more questions on getting it to pass.

Thanks!
Paul



From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Wednesday, January 17, 2018 5:24 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] Add py-spdk client for SPDK

Hi, all
I have submitted the introduction of py-spdk on trello https://trello.com/c/Bn3DIG7F/87-add-py-spdk-client-for-spdk, please visit it. I will be grateful to your kindness.

Regards,

Helloway

在 2018年1月16日，上午11:30，We We <simple_hlw(a)163.com<mailto:simple_hlw(a)163.com>> 写道：

Hi, all
I have submitted the py-spdk(written in python) for more than six days. Why the code-review couldn’t pass and show nothing?
Is there anyone who can help me?

Thanks,
Helloway

在 2018年1月10日，下午9:46，We We <simple_hlw(a)163.com<mailto:simple_hlw(a)163.com>> 写道：

Hi, all
I have submitted the py-spdk code onhttps://review.gerrithub.io/#/c/379741/, please take some time to visit it, I will be very grateful to you.
The py-spdk is client which can help the upper-level app to communicate with the SPDK-based app (such as: nvmf_tgt, vhost, iscsi_tgt, etc.). Should I submit it into the other repo I rebuild rather than SPDK repo? Because I think it is a relatively independent kit upon the SPDK.

If you have some thoughts about the py-spdk, please share with me.

Regards,
Helloway
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk


_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk


_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk



--
Zhipeng (Howard) Huang

Standard Engineer
IT Standard & Patent/IT Product Line
Huawei Technologies Co,. Ltd
Email: huangzhipeng(a)huawei.com<mailto:huangzhipeng(a)huawei.com>
Office: Huawei Industrial Base, Longgang, Shenzhen

(Previous)
Research Assistant
Mobile Ad-Hoc Network Lab, Calit2
University of California, Irvine
Email: zhipengh(a)uci.edu<mailto:zhipengh(a)uci.edu>
Office: Calit2 Building Room 2402

OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado

_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk



--
Zhipeng (Howard) Huang

Standard Engineer
IT Standard & Patent/IT Product Line
Huawei Technologies Co,. Ltd
Email: huangzhipeng(a)huawei.com<mailto:huangzhipeng(a)huawei.com>
Office: Huawei Industrial Base, Longgang, Shenzhen

(Previous)
Research Assistant
Mobile Ad-Hoc Network Lab, Calit2
University of California, Irvine
Email: zhipengh(a)uci.edu<mailto:zhipengh(a)uci.edu>
Office: Calit2 Building Room 2402

OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 45474 bytes --]

[-- Attachment #3: image001.png --]
[-- Type: image/png, Size: 108863 bytes --]

Re: [SPDK] Add py-spdk client for SPDK

[Luse, Paul E]

[-- Attachment #1: Type: text/plain, Size: 8531 bytes --]

Yeah I think one of the guys can do that.  I’ll mention it on IRC also…

Thx
Paul

From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Zhipeng Huang
Sent: Wednesday, January 17, 2018 6:33 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: Re: [SPDK] Add py-spdk client for SPDK

Hi Paul,

Thanks for getting the CI running. Seems most of the problems caused by pep8 missing. Is it possible to add python support in the CI system ? otherwise we will hit many pip pkg missing problems.

On Thu, Jan 18, 2018 at 8:17 AM, Luse, Paul E <paul.e.luse(a)intel.com<mailto:paul.e.luse(a)intel.com>> wrote:
Hmm yeah I’m not sure why it ran on patch set 2 but not the last couple… Seth, anyone?

We might need one of the maintainers to clear it for CI execution, it’s an extra unfortunate security requirement we have on the test pool, otherwise its fully automated

Thx
Paul

From: SPDK [mailto:spdk-bounces(a)lists.01.org<mailto:spdk-bounces(a)lists.01.org>] On Behalf Of Zhipeng Huang
Sent: Wednesday, January 17, 2018 4:04 PM

To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] Add py-spdk client for SPDK

Paul, the problem is that Wewe's later patches doesn't trigger CI at all. Is it how it supposed to work that we should fix the problem before hand ?
Since in OpenStack or kubernetes CI should be triggered everytime the patches come in.

On Thu, Jan 18, 2018 at 5:39 AM, Luse, Paul E <paul.e.luse(a)intel.com<mailto:paul.e.luse(a)intel.com>> wrote:
Hi WeWe


So wrt which repo, I didn’t see that but we don’t really have a precedent for subprojects like that.  I’ll let one of the maintainers comment on the options but I believe they would be (a) maintain on your own elsewhere or (b) include in SPDK like any other code.  Option (b) is really for “core” SPDK elements that make sense for most use cases and a python binding may or may not fit, it clearly isn’t out of the question in my mind though as it’s not directly tied to OpenStack.

On your patch not passing, see the screenshot below where I highlighted the -1 from the CI system.  If you click on that link it will take you to the results and you can see the failure I mention.  Have you tried running the check_format.sh locally?

Thx
Paul


[cid:image001.png(a)01D38FC2.92234A10]

From: SPDK [mailto:spdk-bounces(a)lists.01.org<mailto:spdk-bounces(a)lists.01.org>] On Behalf Of We We
Sent: Wednesday, January 17, 2018 10:26 AM

To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] Add py-spdk client for SPDK

Hi, Paul
I am sorry.  Is  the question on my patch you mentioned  the following Q1?
Q1: is your intention to propose this as part of the SPDK repo or do you intent to maintain this in your own repo and are simply asking for review comments?
If so, I see it and response to you on the patch.  Maybe you didn’t see it.

1. About Q1, my idea is to treat the py-spdk as another subrepo in the SPDK community. For example: SPDK repo -->https://github.com/spdk/spdk.git, py-spdk repo -->https://github.com/spdk/py-spdk.git. Am I reasonable？

2. About a copy of rpc.py. Thank you for your careful review, the py-spdk doesn’t need to modify and copy the files of SPDK, I have deleted the rpc.py and updated the new patch. Please visit it.

3. About my patch didn’t pass CI.  My patch didn’t show nothing including the “verified -1”. I didn’t know whether the CI executed, so I can’t track anything down.  Is there anyone who met this problem?

Regards,
Helloway


在 2018年1月17日，下午10:32，Luse, Paul E <paul.e.luse(a)intel.com<mailto:paul.e.luse(a)intel.com>> 写道：

Hi WeWe,

Thanks for the trello update, did you see my question on your patch? I went to look specifically at the changes to existing files to start with and it appears that your patch includes a copy of rpc.py basically duplicating it within the repo.  Did I miss something or is that what you intended?

Wrt  question on why your patch didn’t pass CI, it’s pretty easy to track these things down most of the time.  In the review, click on the link below where it says verified -1 and you will see a webpage with the status of each of the test machines.  Many of the are red and indicate failure.  Click on one and drill down to find the builg.log file and you’ll see some formatting issues:

========== Backtrace start: ==========

in ./autobuild.sh:23 -> main()
     ...
   18
   19  ./configure $config_params
   20
   21  timing_enter check_format
   22  if [ $SPDK_RUN_CHECK_FORMAT -eq 1 ]; then
=> 23          ./scripts/check_format.sh
   24  fi
   25  timing_exit check_format
   26
   27  timing_enter build_kmod
   28  if [ $SPDK_BUILD_IOAT_KMOD -eq 1 ]; then
     ...

========== Backtrace end ==========

You can run the script, ./scripts/check_format.sh, locally and correct the errors and resubmit.  We always ask that developers run this scripts in advance to make sure things are formatted.  Depending on whether the maintainers think the SPDK repo makes sense for this patch you’ll need some test code also – there is clearly a lot of value in this patch though and regardless of whether it lands here or is maintained elsewhere!

Use this list or the patch review if you have more questions on getting it to pass.

Thanks!
Paul



From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Wednesday, January 17, 2018 5:24 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] Add py-spdk client for SPDK

Hi, all
I have submitted the introduction of py-spdk on trello https://trello.com/c/Bn3DIG7F/87-add-py-spdk-client-for-spdk, please visit it. I will be grateful to your kindness.

Regards,

Helloway

在 2018年1月16日，上午11:30，We We <simple_hlw(a)163.com<mailto:simple_hlw(a)163.com>> 写道：

Hi, all
I have submitted the py-spdk(written in python) for more than six days. Why the code-review couldn’t pass and show nothing?
Is there anyone who can help me?

Thanks,
Helloway

在 2018年1月10日，下午9:46，We We <simple_hlw(a)163.com<mailto:simple_hlw(a)163.com>> 写道：

Hi, all
I have submitted the py-spdk code onhttps://review.gerrithub.io/#/c/379741/, please take some time to visit it, I will be very grateful to you.
The py-spdk is client which can help the upper-level app to communicate with the SPDK-based app (such as: nvmf_tgt, vhost, iscsi_tgt, etc.). Should I submit it into the other repo I rebuild rather than SPDK repo? Because I think it is a relatively independent kit upon the SPDK.

If you have some thoughts about the py-spdk, please share with me.

Regards,
Helloway
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk


_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk


_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk



--
Zhipeng (Howard) Huang

Standard Engineer
IT Standard & Patent/IT Product Line
Huawei Technologies Co,. Ltd
Email: huangzhipeng(a)huawei.com<mailto:huangzhipeng(a)huawei.com>
Office: Huawei Industrial Base, Longgang, Shenzhen

(Previous)
Research Assistant
Mobile Ad-Hoc Network Lab, Calit2
University of California, Irvine
Email: zhipengh(a)uci.edu<mailto:zhipengh(a)uci.edu>
Office: Calit2 Building Room 2402

OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado

_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk



--
Zhipeng (Howard) Huang

Standard Engineer
IT Standard & Patent/IT Product Line
Huawei Technologies Co,. Ltd
Email: huangzhipeng(a)huawei.com<mailto:huangzhipeng(a)huawei.com>
Office: Huawei Industrial Base, Longgang, Shenzhen

(Previous)
Research Assistant
Mobile Ad-Hoc Network Lab, Calit2
University of California, Irvine
Email: zhipengh(a)uci.edu<mailto:zhipengh(a)uci.edu>
Office: Calit2 Building Room 2402

OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 43152 bytes --]

[-- Attachment #3: image001.png --]
[-- Type: image/png, Size: 108863 bytes --]

Re: [SPDK] Add py-spdk client for SPDK

[Zhipeng Huang]

[-- Attachment #1: Type: text/plain, Size: 8488 bytes --]

Hi Paul,

Thanks for getting the CI running. Seems most of the problems caused by
pep8 missing. Is it possible to add python support in the CI system ?
otherwise we will hit many pip pkg missing problems.

On Thu, Jan 18, 2018 at 8:17 AM, Luse, Paul E <paul.e.luse(a)intel.com> wrote:

> Hmm yeah I’m not sure why it ran on patch set 2 but not the last couple…
> Seth, anyone?
>
>
>
> We might need one of the maintainers to clear it for CI execution, it’s an
> extra unfortunate security requirement we have on the test pool, otherwise
> its fully automated
>
>
>
> Thx
> Paul
>
>
>
> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org] *On Behalf Of *Zhipeng
> Huang
> *Sent:* Wednesday, January 17, 2018 4:04 PM
>
> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
> *Subject:* Re: [SPDK] Add py-spdk client for SPDK
>
>
>
> Paul, the problem is that Wewe's later patches doesn't trigger CI at all.
> Is it how it supposed to work that we should fix the problem before hand ?
>
> Since in OpenStack or kubernetes CI should be triggered everytime the
> patches come in.
>
>
>
> On Thu, Jan 18, 2018 at 5:39 AM, Luse, Paul E <paul.e.luse(a)intel.com>
> wrote:
>
> Hi WeWe
>
>
>
>
>
> So wrt which repo, I didn’t see that but we don’t really have a precedent
> for subprojects like that.  I’ll let one of the maintainers comment on the
> options but I believe they would be (a) maintain on your own elsewhere or
> (b) include in SPDK like any other code.  Option (b) is really for “core”
> SPDK elements that make sense for most use cases and a python binding may
> or may not fit, it clearly isn’t out of the question in my mind though as
> it’s not directly tied to OpenStack.
>
>
>
> On your patch not passing, see the screenshot below where I highlighted
> the -1 from the CI system.  If you click on that link it will take you to
> the results and you can see the failure I mention.  Have you tried running
> the check_format.sh locally?
>
>
>
> Thx
>
> Paul
>
>
>
>
>
>
>
> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org] *On Behalf Of *We We
> *Sent:* Wednesday, January 17, 2018 10:26 AM
>
>
> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
> *Subject:* Re: [SPDK] Add py-spdk client for SPDK
>
>
>
> Hi, Paul
>
> I am sorry.  Is  the question on my patch you mentioned  the following Q1?
>
> Q1: is your intention to propose this as part of the SPDK repo or do you
> intent to maintain this in your own repo and are simply asking for review
> comments?
>
> If so, I see it and response to you on the patch.  Maybe you didn’t see
> it.
>
>
>
> 1. About Q1, my idea is to treat the py-spdk as another subrepo in the
> SPDK community. For example: SPDK repo -->https://github.com/spdk/spdk.git,
> py-spdk repo -->https://github.com/spdk/py-spdk.git. Am I reasonable？
>
>
>
> 2. About a copy of rpc.py. Thank you for your careful review, the py-spdk
> doesn’t need to modify and copy the files of SPDK, I have deleted the
> rpc.py and updated the new patch. Please visit it.
>
>
>
> 3. About my patch didn’t pass CI.  My patch didn’t show nothing including
> the “verified -1”. I didn’t know whether the CI executed, so I can’t
> track anything down.  Is there anyone who met this problem?
>
>
>
> Regards,
>
> Helloway
>
>
>
>
>
> 在 2018年1月17日，下午10:32，Luse, Paul E <paul.e.luse(a)intel.com> 写道：
>
>
>
> Hi WeWe,
>
>
>
> Thanks for the trello update, did you see my question on your patch? I
> went to look specifically at the changes to existing files to start with
> and it appears that your patch includes a copy of rpc.py basically
> duplicating it within the repo.  Did I miss something or is that what you
> intended?
>
>
>
> Wrt  question on why your patch didn’t pass CI, it’s pretty easy to track
> these things down most of the time.  In the review, click on the link below
> where it says verified -1 and you will see a webpage with the status of
> each of the test machines.  Many of the are red and indicate failure.
> Click on one and drill down to find the builg.log file and you’ll see some
> formatting issues:
>
>
>
> ========== Backtrace start: ==========
>
>
>
> in ./autobuild.sh:23 -> main()
>
>      ...
>
>    18
>
>    19  ./configure $config_params
>
>    20
>
>    21  timing_enter check_format
>
>    22  if [ $SPDK_RUN_CHECK_FORMAT -eq 1 ]; then
>
> => 23          ./scripts/check_format.sh
>
>    24  fi
>
>    25  timing_exit check_format
>
>    26
>
>    27  timing_enter build_kmod
>
>    28  if [ $SPDK_BUILD_IOAT_KMOD -eq 1 ]; then
>
>      ...
>
>
>
> ========== Backtrace end ==========
>
>
>
> You can run the script, ./scripts/check_format.sh, locally and correct
> the errors and resubmit.  We always ask that developers run this scripts in
> advance to make sure things are formatted.  Depending on whether the
> maintainers think the SPDK repo makes sense for this patch you’ll need some
> test code also – there is clearly a lot of value in this patch though and
> regardless of whether it lands here or is maintained elsewhere!
>
>
>
> Use this list or the patch review if you have more questions on getting it
> to pass.
>
>
>
> Thanks!
>
> Paul
>
>
>
>
>
>
>
> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org <spdk-bounces(a)lists.01.org>
> ] *On Behalf Of *We We
> *Sent:* Wednesday, January 17, 2018 5:24 AM
> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
> *Subject:* Re: [SPDK] Add py-spdk client for SPDK
>
>
>
> Hi, all
>
> I have submitted the introduction of py-spdk on trello
> https://trello.com/c/Bn3DIG7F/87-add-py-spdk-client-for-spdk, please
> visit it. I will be grateful to your kindness.
>
>
>
> Regards,
>
>
>
> Helloway
>
>
>
> 在 2018年1月16日，上午11:30，We We <simple_hlw(a)163.com> 写道：
>
>
>
> Hi, all
>
> I have submitted the py-spdk(written in python) for more than six days.
> Why the code-review couldn’t pass and show nothing?
>
> Is there anyone who can help me?
>
>
>
> Thanks,
>
> Helloway
>
>
>
> 在 2018年1月10日，下午9:46，We We <simple_hlw(a)163.com> 写道：
>
>
>
> Hi, all
>
> I have submitted the py-spdk code onhttps://review.gerrithub.io/
> #/c/379741/, please take some time to visit it, I will be very grateful
> to you.
>
> The py-spdk is client which can help the upper-level app to communicate
> with the SPDK-based app (such as: nvmf_tgt, vhost, iscsi_tgt, etc.). Should
> I submit it into the other repo I rebuild rather than SPDK repo? Because I
> think it is a relatively independent kit upon the SPDK.
>
>
>
> If you have some thoughts about the py-spdk, please share with me.
>
>
>
> Regards,
>
> Helloway
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
>
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
>
>
>
> --
>
> Zhipeng (Howard) Huang
>
>
>
> Standard Engineer
>
> IT Standard & Patent/IT Product Line
>
> Huawei Technologies Co,. Ltd
>
> Email: huangzhipeng(a)huawei.com
>
> Office: Huawei Industrial Base, Longgang, Shenzhen
>
>
>
> (Previous)
>
> Research Assistant
>
> Mobile Ad-Hoc Network Lab, Calit2
>
> University of California, Irvine
>
> Email: zhipengh(a)uci.edu
>
> Office: Calit2 Building Room 2402
>
>
>
> OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>


-- 
Zhipeng (Howard) Huang

Standard Engineer
IT Standard & Patent/IT Product Line
Huawei Technologies Co,. Ltd
Email: huangzhipeng(a)huawei.com
Office: Huawei Industrial Base, Longgang, Shenzhen

(Previous)
Research Assistant
Mobile Ad-Hoc Network Lab, Calit2
University of California, Irvine
Email: zhipengh(a)uci.edu
Office: Calit2 Building Room 2402

OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 30129 bytes --]

[-- Attachment #3: image001.png --]
[-- Type: image/png, Size: 108863 bytes --]

Re: [SPDK] Add py-spdk client for SPDK

[Luse, Paul E]

[-- Attachment #1: Type: text/plain, Size: 7162 bytes --]

Hmm yeah I’m not sure why it ran on patch set 2 but not the last couple… Seth, anyone?

We might need one of the maintainers to clear it for CI execution, it’s an extra unfortunate security requirement we have on the test pool, otherwise its fully automated

Thx
Paul

From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Zhipeng Huang
Sent: Wednesday, January 17, 2018 4:04 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: Re: [SPDK] Add py-spdk client for SPDK

Paul, the problem is that Wewe's later patches doesn't trigger CI at all. Is it how it supposed to work that we should fix the problem before hand ?
Since in OpenStack or kubernetes CI should be triggered everytime the patches come in.

On Thu, Jan 18, 2018 at 5:39 AM, Luse, Paul E <paul.e.luse(a)intel.com<mailto:paul.e.luse(a)intel.com>> wrote:
Hi WeWe


So wrt which repo, I didn’t see that but we don’t really have a precedent for subprojects like that.  I’ll let one of the maintainers comment on the options but I believe they would be (a) maintain on your own elsewhere or (b) include in SPDK like any other code.  Option (b) is really for “core” SPDK elements that make sense for most use cases and a python binding may or may not fit, it clearly isn’t out of the question in my mind though as it’s not directly tied to OpenStack.

On your patch not passing, see the screenshot below where I highlighted the -1 from the CI system.  If you click on that link it will take you to the results and you can see the failure I mention.  Have you tried running the check_format.sh locally?

Thx
Paul


[cid:image001.png(a)01D38FB6.CE11CFD0]

From: SPDK [mailto:spdk-bounces(a)lists.01.org<mailto:spdk-bounces(a)lists.01.org>] On Behalf Of We We
Sent: Wednesday, January 17, 2018 10:26 AM

To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] Add py-spdk client for SPDK

Hi, Paul
I am sorry.  Is  the question on my patch you mentioned  the following Q1?
Q1: is your intention to propose this as part of the SPDK repo or do you intent to maintain this in your own repo and are simply asking for review comments?
If so, I see it and response to you on the patch.  Maybe you didn’t see it.

1. About Q1, my idea is to treat the py-spdk as another subrepo in the SPDK community. For example: SPDK repo -->https://github.com/spdk/spdk.git, py-spdk repo -->https://github.com/spdk/py-spdk.git. Am I reasonable？

2. About a copy of rpc.py. Thank you for your careful review, the py-spdk doesn’t need to modify and copy the files of SPDK, I have deleted the rpc.py and updated the new patch. Please visit it.

3. About my patch didn’t pass CI.  My patch didn’t show nothing including the “verified -1”. I didn’t know whether the CI executed, so I can’t track anything down.  Is there anyone who met this problem?

Regards,
Helloway


在 2018年1月17日，下午10:32，Luse, Paul E <paul.e.luse(a)intel.com<mailto:paul.e.luse(a)intel.com>> 写道：

Hi WeWe,

Thanks for the trello update, did you see my question on your patch? I went to look specifically at the changes to existing files to start with and it appears that your patch includes a copy of rpc.py basically duplicating it within the repo.  Did I miss something or is that what you intended?

Wrt  question on why your patch didn’t pass CI, it’s pretty easy to track these things down most of the time.  In the review, click on the link below where it says verified -1 and you will see a webpage with the status of each of the test machines.  Many of the are red and indicate failure.  Click on one and drill down to find the builg.log file and you’ll see some formatting issues:

========== Backtrace start: ==========

in ./autobuild.sh:23 -> main()
     ...
   18
   19  ./configure $config_params
   20
   21  timing_enter check_format
   22  if [ $SPDK_RUN_CHECK_FORMAT -eq 1 ]; then
=> 23          ./scripts/check_format.sh
   24  fi
   25  timing_exit check_format
   26
   27  timing_enter build_kmod
   28  if [ $SPDK_BUILD_IOAT_KMOD -eq 1 ]; then
     ...

========== Backtrace end ==========

You can run the script, ./scripts/check_format.sh, locally and correct the errors and resubmit.  We always ask that developers run this scripts in advance to make sure things are formatted.  Depending on whether the maintainers think the SPDK repo makes sense for this patch you’ll need some test code also – there is clearly a lot of value in this patch though and regardless of whether it lands here or is maintained elsewhere!

Use this list or the patch review if you have more questions on getting it to pass.

Thanks!
Paul



From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Wednesday, January 17, 2018 5:24 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] Add py-spdk client for SPDK

Hi, all
I have submitted the introduction of py-spdk on trello https://trello.com/c/Bn3DIG7F/87-add-py-spdk-client-for-spdk, please visit it. I will be grateful to your kindness.

Regards,

Helloway

在 2018年1月16日，上午11:30，We We <simple_hlw(a)163.com<mailto:simple_hlw(a)163.com>> 写道：

Hi, all
I have submitted the py-spdk(written in python) for more than six days. Why the code-review couldn’t pass and show nothing?
Is there anyone who can help me?

Thanks,
Helloway

在 2018年1月10日，下午9:46，We We <simple_hlw(a)163.com<mailto:simple_hlw(a)163.com>> 写道：

Hi, all
I have submitted the py-spdk code onhttps://review.gerrithub.io/#/c/379741/, please take some time to visit it, I will be very grateful to you.
The py-spdk is client which can help the upper-level app to communicate with the SPDK-based app (such as: nvmf_tgt, vhost, iscsi_tgt, etc.). Should I submit it into the other repo I rebuild rather than SPDK repo? Because I think it is a relatively independent kit upon the SPDK.

If you have some thoughts about the py-spdk, please share with me.

Regards,
Helloway
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk


_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk


_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk



--
Zhipeng (Howard) Huang

Standard Engineer
IT Standard & Patent/IT Product Line
Huawei Technologies Co,. Ltd
Email: huangzhipeng(a)huawei.com<mailto:huangzhipeng(a)huawei.com>
Office: Huawei Industrial Base, Longgang, Shenzhen

(Previous)
Research Assistant
Mobile Ad-Hoc Network Lab, Calit2
University of California, Irvine
Email: zhipengh(a)uci.edu<mailto:zhipengh(a)uci.edu>
Office: Calit2 Building Room 2402

OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 36677 bytes --]

[-- Attachment #3: image001.png --]
[-- Type: image/png, Size: 108863 bytes --]

Re: [SPDK] Add py-spdk client for SPDK

[Zhipeng Huang]

[-- Attachment #1: Type: text/plain, Size: 6916 bytes --]

Paul, the problem is that Wewe's later patches doesn't trigger CI at all.
Is it how it supposed to work that we should fix the problem before hand ?
Since in OpenStack or kubernetes CI should be triggered everytime the
patches come in.

On Thu, Jan 18, 2018 at 5:39 AM, Luse, Paul E <paul.e.luse(a)intel.com> wrote:

> Hi WeWe
>
>
>
>
>
> So wrt which repo, I didn’t see that but we don’t really have a precedent
> for subprojects like that.  I’ll let one of the maintainers comment on the
> options but I believe they would be (a) maintain on your own elsewhere or
> (b) include in SPDK like any other code.  Option (b) is really for “core”
> SPDK elements that make sense for most use cases and a python binding may
> or may not fit, it clearly isn’t out of the question in my mind though as
> it’s not directly tied to OpenStack.
>
>
>
> On your patch not passing, see the screenshot below where I highlighted
> the -1 from the CI system.  If you click on that link it will take you to
> the results and you can see the failure I mention.  Have you tried running
> the check_format.sh locally?
>
>
>
> Thx
>
> Paul
>
>
>
>
>
>
>
> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org] *On Behalf Of *We We
> *Sent:* Wednesday, January 17, 2018 10:26 AM
>
> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
> *Subject:* Re: [SPDK] Add py-spdk client for SPDK
>
>
>
> Hi, Paul
>
> I am sorry.  Is  the question on my patch you mentioned  the following Q1?
>
> Q1: is your intention to propose this as part of the SPDK repo or do you
> intent to maintain this in your own repo and are simply asking for review
> comments?
>
> If so, I see it and response to you on the patch.  Maybe you didn’t see
> it.
>
>
>
> 1. About Q1, my idea is to treat the py-spdk as another subrepo in the
> SPDK community. For example: SPDK repo -->https://github.com/spdk/spdk.git,
> py-spdk repo -->https://github.com/spdk/py-spdk.git. Am I reasonable？
>
>
>
> 2. About a copy of rpc.py. Thank you for your careful review, the py-spdk
> doesn’t need to modify and copy the files of SPDK, I have deleted the
> rpc.py and updated the new patch. Please visit it.
>
>
>
> 3. About my patch didn’t pass CI.  My patch didn’t show nothing including
> the “verified -1”. I didn’t know whether the CI executed, so I can’t
> track anything down.  Is there anyone who met this problem?
>
>
>
> Regards,
>
> Helloway
>
>
>
>
>
> 在 2018年1月17日，下午10:32，Luse, Paul E <paul.e.luse(a)intel.com> 写道：
>
>
>
> Hi WeWe,
>
>
>
> Thanks for the trello update, did you see my question on your patch? I
> went to look specifically at the changes to existing files to start with
> and it appears that your patch includes a copy of rpc.py basically
> duplicating it within the repo.  Did I miss something or is that what you
> intended?
>
>
>
> Wrt  question on why your patch didn’t pass CI, it’s pretty easy to track
> these things down most of the time.  In the review, click on the link below
> where it says verified -1 and you will see a webpage with the status of
> each of the test machines.  Many of the are red and indicate failure.
> Click on one and drill down to find the builg.log file and you’ll see some
> formatting issues:
>
>
>
> ========== Backtrace start: ==========
>
>
>
> in ./autobuild.sh:23 -> main()
>
>      ...
>
>    18
>
>    19  ./configure $config_params
>
>    20
>
>    21  timing_enter check_format
>
>    22  if [ $SPDK_RUN_CHECK_FORMAT -eq 1 ]; then
>
> => 23          ./scripts/check_format.sh
>
>    24  fi
>
>    25  timing_exit check_format
>
>    26
>
>    27  timing_enter build_kmod
>
>    28  if [ $SPDK_BUILD_IOAT_KMOD -eq 1 ]; then
>
>      ...
>
>
>
> ========== Backtrace end ==========
>
>
>
> You can run the script, ./scripts/check_format.sh, locally and correct
> the errors and resubmit.  We always ask that developers run this scripts in
> advance to make sure things are formatted.  Depending on whether the
> maintainers think the SPDK repo makes sense for this patch you’ll need some
> test code also – there is clearly a lot of value in this patch though and
> regardless of whether it lands here or is maintained elsewhere!
>
>
>
> Use this list or the patch review if you have more questions on getting it
> to pass.
>
>
>
> Thanks!
>
> Paul
>
>
>
>
>
>
>
> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org <spdk-bounces(a)lists.01.org>
> ] *On Behalf Of *We We
> *Sent:* Wednesday, January 17, 2018 5:24 AM
> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
> *Subject:* Re: [SPDK] Add py-spdk client for SPDK
>
>
>
> Hi, all
>
> I have submitted the introduction of py-spdk on trello
> https://trello.com/c/Bn3DIG7F/87-add-py-spdk-client-for-spdk, please
> visit it. I will be grateful to your kindness.
>
>
>
> Regards,
>
>
>
> Helloway
>
>
>
> 在 2018年1月16日，上午11:30，We We <simple_hlw(a)163.com> 写道：
>
>
>
> Hi, all
>
> I have submitted the py-spdk(written in python) for more than six days.
> Why the code-review couldn’t pass and show nothing?
>
> Is there anyone who can help me?
>
>
>
> Thanks,
>
> Helloway
>
>
>
> 在 2018年1月10日，下午9:46，We We <simple_hlw(a)163.com> 写道：
>
>
>
> Hi, all
>
> I have submitted the py-spdk code onhttps://review.gerrithub.io/
> #/c/379741/, please take some time to visit it, I will be very grateful
> to you.
>
> The py-spdk is client which can help the upper-level app to communicate
> with the SPDK-based app (such as: nvmf_tgt, vhost, iscsi_tgt, etc.). Should
> I submit it into the other repo I rebuild rather than SPDK repo? Because I
> think it is a relatively independent kit upon the SPDK.
>
>
>
> If you have some thoughts about the py-spdk, please share with me.
>
>
>
> Regards,
>
> Helloway
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
>
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>


-- 
Zhipeng (Howard) Huang

Standard Engineer
IT Standard & Patent/IT Product Line
Huawei Technologies Co,. Ltd
Email: huangzhipeng(a)huawei.com
Office: Huawei Industrial Base, Longgang, Shenzhen

(Previous)
Research Assistant
Mobile Ad-Hoc Network Lab, Calit2
University of California, Irvine
Email: zhipengh(a)uci.edu
Office: Calit2 Building Room 2402

OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 27234 bytes --]

[-- Attachment #3: image001.png --]
[-- Type: image/png, Size: 108863 bytes --]

Re: [SPDK] Add py-spdk client for SPDK

[Luse, Paul E]

[-- Attachment #1: Type: text/plain, Size: 5579 bytes --]

Hi WeWe


So wrt which repo, I didn’t see that but we don’t really have a precedent for subprojects like that.  I’ll let one of the maintainers comment on the options but I believe they would be (a) maintain on your own elsewhere or (b) include in SPDK like any other code.  Option (b) is really for “core” SPDK elements that make sense for most use cases and a python binding may or may not fit, it clearly isn’t out of the question in my mind though as it’s not directly tied to OpenStack.

On your patch not passing, see the screenshot below where I highlighted the -1 from the CI system.  If you click on that link it will take you to the results and you can see the failure I mention.  Have you tried running the check_format.sh locally?

Thx
Paul


[cid:image001.png(a)01D38FA0.EC7DC0C0]

From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Wednesday, January 17, 2018 10:26 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: Re: [SPDK] Add py-spdk client for SPDK

Hi, Paul
I am sorry.  Is  the question on my patch you mentioned  the following Q1?
Q1: is your intention to propose this as part of the SPDK repo or do you intent to maintain this in your own repo and are simply asking for review comments?
If so, I see it and response to you on the patch.  Maybe you didn’t see it.

1. About Q1, my idea is to treat the py-spdk as another subrepo in the SPDK community. For example: SPDK repo -->https://github.com/spdk/spdk.git, py-spdk repo -->https://github.com/spdk/py-spdk.git. Am I reasonable？


2. About a copy of rpc.py. Thank you for your careful review, the py-spdk doesn’t need to modify and copy the files of SPDK, I have deleted the rpc.py and updated the new patch. Please visit it.


3. About my patch didn’t pass CI.  My patch didn’t show nothing including the “verified -1”. I didn’t know whether the CI executed, so I can’t track anything down.  Is there anyone who met this problem?

Regards,
Helloway



在 2018年1月17日，下午10:32，Luse, Paul E <paul.e.luse(a)intel.com<mailto:paul.e.luse(a)intel.com>> 写道：

Hi WeWe,

Thanks for the trello update, did you see my question on your patch? I went to look specifically at the changes to existing files to start with and it appears that your patch includes a copy of rpc.py basically duplicating it within the repo.  Did I miss something or is that what you intended?

Wrt  question on why your patch didn’t pass CI, it’s pretty easy to track these things down most of the time.  In the review, click on the link below where it says verified -1 and you will see a webpage with the status of each of the test machines.  Many of the are red and indicate failure.  Click on one and drill down to find the builg.log file and you’ll see some formatting issues:

========== Backtrace start: ==========

in ./autobuild.sh:23 -> main()
     ...
   18
   19  ./configure $config_params
   20
   21  timing_enter check_format
   22  if [ $SPDK_RUN_CHECK_FORMAT -eq 1 ]; then
=> 23          ./scripts/check_format.sh
   24  fi
   25  timing_exit check_format
   26
   27  timing_enter build_kmod
   28  if [ $SPDK_BUILD_IOAT_KMOD -eq 1 ]; then
     ...

========== Backtrace end ==========

You can run the script, ./scripts/check_format.sh, locally and correct the errors and resubmit.  We always ask that developers run this scripts in advance to make sure things are formatted.  Depending on whether the maintainers think the SPDK repo makes sense for this patch you’ll need some test code also – there is clearly a lot of value in this patch though and regardless of whether it lands here or is maintained elsewhere!

Use this list or the patch review if you have more questions on getting it to pass.

Thanks!
Paul



From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Wednesday, January 17, 2018 5:24 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] Add py-spdk client for SPDK

Hi, all
I have submitted the introduction of py-spdk on trello https://trello.com/c/Bn3DIG7F/87-add-py-spdk-client-for-spdk, please visit it. I will be grateful to your kindness.

Regards,

Helloway

在 2018年1月16日，上午11:30，We We <simple_hlw(a)163.com<mailto:simple_hlw(a)163.com>> 写道：

Hi, all
I have submitted the py-spdk(written in python) for more than six days. Why the code-review couldn’t pass and show nothing?
Is there anyone who can help me?

Thanks,
Helloway

在 2018年1月10日，下午9:46，We We <simple_hlw(a)163.com<mailto:simple_hlw(a)163.com>> 写道：

Hi, all
I have submitted the py-spdk code onhttps://review.gerrithub.io/#/c/379741/, please take some time to visit it, I will be very grateful to you.
The py-spdk is client which can help the upper-level app to communicate with the SPDK-based app (such as: nvmf_tgt, vhost, iscsi_tgt, etc.). Should I submit it into the other repo I rebuild rather than SPDK repo? Because I think it is a relatively independent kit upon the SPDK.

If you have some thoughts about the py-spdk, please share with me.

Regards,
Helloway
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk


_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk


[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 27621 bytes --]

[-- Attachment #3: image001.png --]
[-- Type: image/png, Size: 108863 bytes --]

Re: [SPDK] Add py-spdk client for SPDK

[We We]

[-- Attachment #1: Type: text/plain, Size: 4927 bytes --]

Hi, Paul
I am sorry.  Is  the question on my patch you mentioned  the following Q1?
Q1: is your intention to propose this as part of the SPDK repo or do you intent to maintain this in your own repo and are simply asking for review comments?
If so, I see it and response to you on the patch.  Maybe you didn’t see it. 

1. About Q1, my idea is to treat the py-spdk as another subrepo in the SPDK community. For example: SPDK repo -->https://github.com/spdk/spdk.git <https://github.com/spdk/spdk.git>,  py-spdk repo -->https://github.com/spdk/py-spdk.git <https://github.com/spdk/py-spdk.git>. Am I reasonable？

2. About a copy of rpc.py. Thank you for your careful review,  the py-spdk doesn’t need to modify and copy the files of SPDK, I have deleted the rpc.py and updated the new patch. Please visit it.

3. About my patch didn’t pass CI.  My patch didn’t show nothing including the “verified -1”. I didn’t know whether the CI executed, so I can’t track anything down.  Is there anyone who met this problem?

Regards,
Helloway


> 在 2018年1月17日，下午10:32，Luse, Paul E <paul.e.luse(a)intel.com> 写道：
> 
> Hi WeWe,
>  
> Thanks for the trello update, did you see my question on your patch? I went to look specifically at the changes to existing files to start with and it appears that your patch includes a copy of rpc.py basically duplicating it within the repo.  Did I miss something or is that what you intended?
>  
> Wrt  question on why your patch didn’t pass CI, it’s pretty easy to track these things down most of the time.  In the review, click on the link below where it says verified -1 and you will see a webpage with the status of each of the test machines.  Many of the are red and indicate failure.  Click on one and drill down to find the builg.log file and you’ll see some formatting issues:
>  
> ========== Backtrace start: ==========
>  
> in ./autobuild.sh:23 -> main()
>      ...
>    18 
>    19  ./configure $config_params
>    20 
>    21  timing_enter check_format
>    22  if [ $SPDK_RUN_CHECK_FORMAT -eq 1 ]; then
> => 23          ./scripts/check_format.sh
>    24  fi
>    25  timing_exit check_format
>    26 
>    27  timing_enter build_kmod
>    28  if [ $SPDK_BUILD_IOAT_KMOD -eq 1 ]; then
>      ...
>  
> ========== Backtrace end ==========
>  
> You can run the script, ./scripts/check_format.sh, locally and correct the errors and resubmit.  We always ask that developers run this scripts in advance to make sure things are formatted.  Depending on whether the maintainers think the SPDK repo makes sense for this patch you’ll need some test code also – there is clearly a lot of value in this patch though and regardless of whether it lands here or is maintained elsewhere!
>  
> Use this list or the patch review if you have more questions on getting it to pass.
>  
> Thanks!
> Paul
>  
>  
>   <>
>  <>From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
> Sent: Wednesday, January 17, 2018 5:24 AM
> To: Storage Performance Development Kit <spdk(a)lists.01.org>
> Subject: Re: [SPDK] Add py-spdk client for SPDK
>  
> Hi, all
> I have submitted the introduction of py-spdk on trello https://trello.com/c/Bn3DIG7F/87-add-py-spdk-client-for-spdk <https://trello.com/c/Bn3DIG7F/87-add-py-spdk-client-for-spdk>, please visit it. I will be grateful to your kindness.
>  
> Regards,
>  
> Helloway
>  
> 在 2018年1月16日，上午11:30，We We <simple_hlw(a)163.com <mailto:simple_hlw(a)163.com>> 写道：
>  
> Hi, all
> I have submitted the py-spdk(written in python) for more than six days. Why the code-review couldn’t pass and show nothing?
> Is there anyone who can help me?
>  
> Thanks,
> Helloway
>  
> 在 2018年1月10日，下午9:46，We We <simple_hlw(a)163.com <mailto:simple_hlw(a)163.com>> 写道：
>  
> Hi, all
> I have submitted the py-spdk code onhttps://review.gerrithub.io/#/c/379741/ <https://review.gerrithub.io/#/c/379741/>, please take some time to visit it, I will be very grateful to you. 
> The py-spdk is client which can help the upper-level app to communicate with the SPDK-based app (such as: nvmf_tgt, vhost, iscsi_tgt, etc.). Should I submit it into the other repo I rebuild rather than SPDK repo? Because I think it is a relatively independent kit upon the SPDK. 
>  
> If you have some thoughts about the py-spdk, please share with me.
>  
> Regards,
> Helloway
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
>  
>  
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 26124 bytes --]

Re: [SPDK] Add py-spdk client for SPDK

[Luse, Paul E]

[-- Attachment #1: Type: text/plain, Size: 3223 bytes --]

Hi WeWe,

Thanks for the trello update, did you see my question on your patch? I went to look specifically at the changes to existing files to start with and it appears that your patch includes a copy of rpc.py basically duplicating it within the repo.  Did I miss something or is that what you intended?

Wrt  question on why your patch didn’t pass CI, it’s pretty easy to track these things down most of the time.  In the review, click on the link below where it says verified -1 and you will see a webpage with the status of each of the test machines.  Many of the are red and indicate failure.  Click on one and drill down to find the builg.log file and you’ll see some formatting issues:

========== Backtrace start: ==========

in ./autobuild.sh:23 -> main()
     ...
   18
   19  ./configure $config_params
   20
   21  timing_enter check_format
   22  if [ $SPDK_RUN_CHECK_FORMAT -eq 1 ]; then
=> 23          ./scripts/check_format.sh
   24  fi
   25  timing_exit check_format
   26
   27  timing_enter build_kmod
   28  if [ $SPDK_BUILD_IOAT_KMOD -eq 1 ]; then
     ...

========== Backtrace end ==========

You can run the script, ./scripts/check_format.sh, locally and correct the errors and resubmit.  We always ask that developers run this scripts in advance to make sure things are formatted.  Depending on whether the maintainers think the SPDK repo makes sense for this patch you’ll need some test code also - there is clearly a lot of value in this patch though and regardless of whether it lands here or is maintained elsewhere!

Use this list or the patch review if you have more questions on getting it to pass.

Thanks!
Paul



From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Wednesday, January 17, 2018 5:24 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: Re: [SPDK] Add py-spdk client for SPDK

Hi, all
I have submitted the introduction of py-spdk on trello https://trello.com/c/Bn3DIG7F/87-add-py-spdk-client-for-spdk, please visit it. I will be grateful to your kindness.

Regards,

Helloway

在 2018年1月16日，上午11:30，We We <simple_hlw(a)163.com<mailto:simple_hlw(a)163.com>> 写道：

Hi, all
I have submitted the py-spdk(written in python) for more than six days. Why the code-review couldn’t pass and show nothing?
Is there anyone who can help me?

Thanks,
Helloway

在 2018年1月10日，下午9:46，We We <simple_hlw(a)163.com<mailto:simple_hlw(a)163.com>> 写道：

Hi, all
I have submitted the py-spdk code on https://review.gerrithub.io/#/c/379741/, please take some time to visit it, I will be very grateful to you.
The py-spdk is client which can help the upper-level app to communicate with the SPDK-based app (such as: nvmf_tgt, vhost, iscsi_tgt, etc.). Should I submit it into the other repo I rebuild rather than SPDK repo? Because I think it is a relatively independent kit upon the SPDK.

If you have some thoughts about the py-spdk, please share with me.

Regards,
Helloway
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk



[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 14254 bytes --]

Re: [SPDK] Add py-spdk client for SPDK

[We We]

[-- Attachment #1: Type: text/plain, Size: 1459 bytes --]

Hi, all
I have submitted the introduction of py-spdk on trello https://trello.com/c/Bn3DIG7F/87-add-py-spdk-client-for-spdk <https://trello.com/c/Bn3DIG7F/87-add-py-spdk-client-for-spdk>, please visit it. I will be grateful to your kindness.

Regards,

Helloway

> 在 2018年1月16日，上午11:30，We We <simple_hlw(a)163.com> 写道：
> 
> Hi, all
> I have submitted the py-spdk(written in python) for more than six days. Why the code-review couldn’t pass and show nothing?
> Is there anyone who can help me?
> 
> Thanks,
> Helloway
> 
>> 在 2018年1月10日，下午9:46，We We <simple_hlw(a)163.com <mailto:simple_hlw(a)163.com>> 写道：
>> 
>> Hi, all
>> 
>> I have submitted the py-spdk code on https://review.gerrithub.io/#/c/379741/ <https://review.gerrithub.io/#/c/379741/>, please take some time to visit it, I will be very grateful to you. 
>> 
>> The py-spdk is client which can help the upper-level app to communicate with the SPDK-based app (such as: nvmf_tgt, vhost, iscsi_tgt, etc.). Should I submit it into the other repo I rebuild rather than SPDK repo? Because I think it is a relatively independent kit upon the SPDK. 
>> 
>> If you have some thoughts about the py-spdk, please share with me.
>> 
>> Regards,
>> Helloway
>> _______________________________________________
>> SPDK mailing list
>> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
>> https://lists.01.org/mailman/listinfo/spdk
> 


[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 40233 bytes --]

Re: [SPDK] Add py-spdk client for SPDK

[Luse, Paul E]

[-- Attachment #1: Type: text/plain, Size: 1464 bytes --]

Hi WeWe,

This was a 3 day weekend in the US (yesterday was a holiday)… we also have a release coming up soon so please bear with us :) I tried to get to it before the end of the day Fri but didn’t get to it, soon though!!!

Thanks!
Paul


From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Monday, January 15, 2018 8:31 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: Re: [SPDK] Add py-spdk client for SPDK

Hi, all
I have submitted the py-spdk(written in python) for more than six days. Why the code-review couldn’t pass and show nothing?
Is there anyone who can help me?

Thanks,
Helloway

在 2018年1月10日，下午9:46，We We <simple_hlw(a)163.com<mailto:simple_hlw(a)163.com>> 写道：

Hi, all
I have submitted the py-spdk code on https://review.gerrithub.io/#/c/379741/, please take some time to visit it, I will be very grateful to you.
The py-spdk is client which can help the upper-level app to communicate with the SPDK-based app (such as: nvmf_tgt, vhost, iscsi_tgt, etc.). Should I submit it into the other repo I rebuild rather than SPDK repo? Because I think it is a relatively independent kit upon the SPDK.

If you have some thoughts about the py-spdk, please share with me.

Regards,
Helloway
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk


[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 7099 bytes --]

Re: [SPDK] Add py-spdk client for SPDK

[We We]

[-- Attachment #1: Type: text/plain, Size: 1013 bytes --]

Hi, all
I have submitted the py-spdk(written in python) for more than six days. Why the code-review couldn’t pass and show nothing?
Is there anyone who can help me?

Thanks,
Helloway

> 在 2018年1月10日，下午9:46，We We <simple_hlw(a)163.com> 写道：
> 
> Hi, all
> 
> I have submitted the py-spdk code on https://review.gerrithub.io/#/c/379741/ <https://review.gerrithub.io/#/c/379741/>, please take some time to visit it, I will be very grateful to you. 
> 
> The py-spdk is client which can help the upper-level app to communicate with the SPDK-based app (such as: nvmf_tgt, vhost, iscsi_tgt, etc.). Should I submit it into the other repo I rebuild rather than SPDK repo? Because I think it is a relatively independent kit upon the SPDK. 
> 
> If you have some thoughts about the py-spdk, please share with me.
> 
> Regards,
> Helloway
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk


[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 39255 bytes --]

Re: [SPDK] Add py-spdk client for SPDK

[@ 2018-01-11 6:16 UTC (permalink / raw)]

[-- Attachment #1: Type: text/plain, Size: 3734 bytes --]

Hello Howard-san,

Thank you for your helpful feedback.
I’ll read and learn WeWe’s updated patches with keeping your feedback in mind.

Thanks,
Shuhei

From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Zhipeng Huang
Sent: Thursday, January 11, 2018 11:09 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: [!]Re: [SPDK] Add py-spdk client for SPDK

Hi Mastumoto San,

For "dynamically managing", I did not mean to change SPDK on the fly, but perform life cycle management on SPDK based application as a whole.

With that said, I think if cyborg-db could persist SPDK application meta/config data,  might be helpful :)

On Thu, Jan 11, 2018 at 9:56 AM, 松本周平 / MATSUMOTO，SHUUHEI <shuhei.matsumoto.xt(a)hitachi.com<mailto:shuhei.matsumoto.xt(a)hitachi.com>> wrote:

Hi WeWe, Paul,



As long as I understand, currently SPDK read configuration from configuration file and keep configuration only in memory.

SPDK does not have nonvolatile dynamically changeable configuration database yet.



Developing configuration database into SPDK will be cumbersome.

If Cyborg takes that role, we may be able to get one simple storage system.



This is just an idea I have not pursued this yet.



Thanks,

Shuhei

________________________________
差出人: SPDK <spdk-bounces(a)lists.01.org<mailto:spdk-bounces(a)lists.01.org>> が Luse, Paul E <paul.e.luse(a)intel.com<mailto:paul.e.luse(a)intel.com>> の代理で送信
送信日時: 2018年1月11日 10:37
宛先: Storage Performance Development Kit
件名: [!]Re: [SPDK] Add py-spdk client for SPDK


Hi WeWe,



Thanks for continuing to work on this, the progress sounds encouraging! I haven’t looked at it yet but it seems like there are opportunities beyond Cyborg for this and I have to admit I don’t totally see how SPDK, as a SW solution, fits the Cyborg model.  As with my last comment on your patch I think it would be great if you could explain your vision a bit more both with Cyborg and beyond.  I believe this would generate more interest in getting some eyes on this.  Personally I will make a commitment to look at it before the end of the week.



Thanks again!

Paul



From: SPDK [mailto:spdk-bounces(a)lists.01.org<mailto:spdk-bounces(a)lists.01.org>] On Behalf Of We We
Sent: Wednesday, January 10, 2018 6:46 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: [SPDK] Add py-spdk client for SPDK



Hi, all

I have submitted the py-spdk code on https://review.gerrithub.io/#/c/379741/, please take some time to visit it, I will be very grateful to you.

The py-spdk is client which can help the upper-level app to communicate with the SPDK-based app (such as: nvmf_tgt, vhost, iscsi_tgt, etc.). Should I submit it into the other repo I rebuild rather than SPDK repo? Because I think it is a relatively independent kit upon the SPDK.



If you have some thoughts about the py-spdk, please share with me.



Regards,

Helloway

_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk



--
Zhipeng (Howard) Huang

Standard Engineer
IT Standard & Patent/IT Product Line
Huawei Technologies Co,. Ltd
Email: huangzhipeng(a)huawei.com<mailto:huangzhipeng(a)huawei.com>
Office: Huawei Industrial Base, Longgang, Shenzhen

(Previous)
Research Assistant
Mobile Ad-Hoc Network Lab, Calit2
University of California, Irvine
Email: zhipengh(a)uci.edu<mailto:zhipengh(a)uci.edu>
Office: Calit2 Building Room 2402

OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 20520 bytes --]

Re: [SPDK] Add py-spdk client for SPDK

[Zhipeng Huang]

[-- Attachment #1: Type: text/plain, Size: 3237 bytes --]

Hi Mastumoto San,

For "dynamically managing", I did not mean to change SPDK on the fly, but
perform life cycle management on SPDK based application as a whole.

With that said, I think if cyborg-db could persist SPDK application
meta/config data,  might be helpful :)

On Thu, Jan 11, 2018 at 9:56 AM, 松本周平 / MATSUMOTO，SHUUHEI <
shuhei.matsumoto.xt(a)hitachi.com> wrote:

> Hi WeWe, Paul,
>
>
> As long as I understand, currently SPDK read configuration
> from configuration file and keep configuration only in memory.
>
> SPDK does not have nonvolatile dynamically changeable configuration
> database yet.
>
>
> Developing configuration database into SPDK will be cumbersome.
>
> If Cyborg takes that role, we may be able to get one simple storage system.
>
>
> This is just an idea I have not pursued this yet.
>
>
> Thanks,
>
> Shuhei
>
>
> ------------------------------
> *差出人:* SPDK <spdk-bounces(a)lists.01.org> が Luse, Paul E <
> paul.e.luse(a)intel.com> の代理で送信
> *送信日時:* 2018年1月11日 10:37
> *宛先:* Storage Performance Development Kit
> *件名:* [!]Re: [SPDK] Add py-spdk client for SPDK
>
>
> Hi WeWe,
>
>
>
> Thanks for continuing to work on this, the progress sounds encouraging! I
> haven’t looked at it yet but it seems like there are opportunities beyond
> Cyborg for this and I have to admit I don’t totally see how SPDK, as a SW
> solution, fits the Cyborg model.  As with my last comment on your patch I
> think it would be great if you could explain your vision a bit more both
> with Cyborg and beyond.  I believe this would generate more interest in
> getting some eyes on this.  Personally I will make a commitment to look at
> it before the end of the week.
>
>
>
> Thanks again!
>
> Paul
>
>
>
> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org] *On Behalf Of *We We
> *Sent:* Wednesday, January 10, 2018 6:46 AM
> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
> *Subject:* [SPDK] Add py-spdk client for SPDK
>
>
>
> Hi, all
>
> I have submitted the py-spdk code on https://review.gerrithub.io/#/
> c/379741/, please take some time to visit it, I will be very grateful to
> you.
>
> The py-spdk is client which can help the upper-level app to communicate
> with the SPDK-based app (such as: nvmf_tgt, vhost, iscsi_tgt, etc.). Should
> I submit it into the other repo I rebuild rather than SPDK repo? Because I
> think it is a relatively independent kit upon the SPDK.
>
>
>
> If you have some thoughts about the py-spdk, please share with me.
>
>
>
> Regards,
>
> Helloway
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>


-- 
Zhipeng (Howard) Huang

Standard Engineer
IT Standard & Patent/IT Product Line
Huawei Technologies Co,. Ltd
Email: huangzhipeng(a)huawei.com
Office: Huawei Industrial Base, Longgang, Shenzhen

(Previous)
Research Assistant
Mobile Ad-Hoc Network Lab, Calit2
University of California, Irvine
Email: zhipengh(a)uci.edu
Office: Calit2 Building Room 2402

OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 8107 bytes --]

Re: [SPDK] Add py-spdk client for SPDK

[@ 2018-01-11 1:56 UTC (permalink / raw)]

[-- Attachment #1: Type: text/plain, Size: 2108 bytes --]

Hi WeWe, Paul,


As long as I understand, currently SPDK read configuration from configuration file and keep configuration only in memory.

SPDK does not have nonvolatile dynamically changeable configuration database yet.


Developing configuration database into SPDK will be cumbersome.

If Cyborg takes that role, we may be able to get one simple storage system.


This is just an idea I have not pursued this yet.


Thanks,

Shuhei


________________________________
差出人: SPDK <spdk-bounces(a)lists.01.org> が Luse, Paul E <paul.e.luse(a)intel.com> の代理で送信
送信日時: 2018年1月11日 10:37
宛先: Storage Performance Development Kit
件名: [!]Re: [SPDK] Add py-spdk client for SPDK


Hi WeWe,



Thanks for continuing to work on this, the progress sounds encouraging! I haven’t looked at it yet but it seems like there are opportunities beyond Cyborg for this and I have to admit I don’t totally see how SPDK, as a SW solution, fits the Cyborg model.  As with my last comment on your patch I think it would be great if you could explain your vision a bit more both with Cyborg and beyond.  I believe this would generate more interest in getting some eyes on this.  Personally I will make a commitment to look at it before the end of the week.



Thanks again!

Paul



From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Wednesday, January 10, 2018 6:46 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: [SPDK] Add py-spdk client for SPDK



Hi, all

I have submitted the py-spdk code on https://review.gerrithub.io/#/c/379741/, please take some time to visit it, I will be very grateful to you.

The py-spdk is client which can help the upper-level app to communicate with the SPDK-based app (such as: nvmf_tgt, vhost, iscsi_tgt, etc.). Should I submit it into the other repo I rebuild rather than SPDK repo? Because I think it is a relatively independent kit upon the SPDK.



If you have some thoughts about the py-spdk, please share with me.



Regards,

Helloway

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 5861 bytes --]

Re: [SPDK] Add py-spdk client for SPDK

[Zhipeng Huang]

[-- Attachment #1: Type: text/plain, Size: 3036 bytes --]

Hi Paul,

I could chime in and help explaining a bit. From Cyborg's perspective, we
view SPDK as a software acceleration framework and any app that built upon
SPDK could be treated as a software accelerator. We want to enable user to
be able to dynamically launch or disengage a SPDK app in user space for any
acceleration purpose it might need.

From SPDK's perspective, I totally agree with you that py-spdk's work goes
beyond Cyborg. py-spdk provides a general python binding for spdk framework
which means SPDK app developers could quickly develop a SDK based upon
py-spdk.

Furthermore, I don't know if you noticed, py-spdk utilizes the protobuf
data modeling which means if there are people want to develop golang or
java bindings, they could just define the interface and then reuse the data
model we have developed for py-spdk.

Hope this explanation makes it more clearer :)


On Thu, Jan 11, 2018 at 9:37 AM, Luse, Paul E <paul.e.luse(a)intel.com> wrote:

> Hi WeWe,
>
>
>
> Thanks for continuing to work on this, the progress sounds encouraging! I
> haven’t looked at it yet but it seems like there are opportunities beyond
> Cyborg for this and I have to admit I don’t totally see how SPDK, as a SW
> solution, fits the Cyborg model.  As with my last comment on your patch I
> think it would be great if you could explain your vision a bit more both
> with Cyborg and beyond.  I believe this would generate more interest in
> getting some eyes on this.  Personally I will make a commitment to look at
> it before the end of the week.
>
>
>
> Thanks again!
>
> Paul
>
>
>
> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org] *On Behalf Of *We We
> *Sent:* Wednesday, January 10, 2018 6:46 AM
> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
> *Subject:* [SPDK] Add py-spdk client for SPDK
>
>
>
> Hi, all
>
> I have submitted the py-spdk code on https://review.gerrithub.io/#/
> c/379741/, please take some time to visit it, I will be very grateful to
> you.
>
> The py-spdk is client which can help the upper-level app to communicate
> with the SPDK-based app (such as: nvmf_tgt, vhost, iscsi_tgt, etc.). Should
> I submit it into the other repo I rebuild rather than SPDK repo? Because I
> think it is a relatively independent kit upon the SPDK.
>
>
>
> If you have some thoughts about the py-spdk, please share with me.
>
>
>
> Regards,
>
> Helloway
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>


-- 
Zhipeng (Howard) Huang

Standard Engineer
IT Standard & Patent/IT Product Line
Huawei Technologies Co,. Ltd
Email: huangzhipeng(a)huawei.com
Office: Huawei Industrial Base, Longgang, Shenzhen

(Previous)
Research Assistant
Mobile Ad-Hoc Network Lab, Calit2
University of California, Irvine
Email: zhipengh(a)uci.edu
Office: Calit2 Building Room 2402

OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 6661 bytes --]

Re: [SPDK] Add py-spdk client for SPDK

[Luse, Paul E]

[-- Attachment #1: Type: text/plain, Size: 1333 bytes --]

Hi WeWe,

Thanks for continuing to work on this, the progress sounds encouraging! I haven't looked at it yet but it seems like there are opportunities beyond Cyborg for this and I have to admit I don't totally see how SPDK, as a SW solution, fits the Cyborg model.  As with my last comment on your patch I think it would be great if you could explain your vision a bit more both with Cyborg and beyond.  I believe this would generate more interest in getting some eyes on this.  Personally I will make a commitment to look at it before the end of the week.

Thanks again!
Paul

From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Wednesday, January 10, 2018 6:46 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: [SPDK] Add py-spdk client for SPDK

Hi, all
I have submitted the py-spdk code on https://review.gerrithub.io/#/c/379741/, please take some time to visit it, I will be very grateful to you.
The py-spdk is client which can help the upper-level app to communicate with the SPDK-based app (such as: nvmf_tgt, vhost, iscsi_tgt, etc.). Should I submit it into the other repo I rebuild rather than SPDK repo? Because I think it is a relatively independent kit upon the SPDK.

If you have some thoughts about the py-spdk, please share with me.

Regards,
Helloway

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 5723 bytes --]

[SPDK] Add py-spdk client for SPDK

[We We]

[-- Attachment #1: Type: text/plain, Size: 533 bytes --]

Hi, all

I have submitted the py-spdk code on https://review.gerrithub.io/#/c/379741/, please take some time to visit it, I will be very grateful to you. 

The py-spdk is client which can help the upper-level app to communicate with the SPDK-based app (such as: nvmf_tgt, vhost, iscsi_tgt, etc.). Should I submit it into the other repo I rebuild rather than SPDK repo? Because I think it is a relatively independent kit upon the SPDK. 

If you have some thoughts about the py-spdk, please share with me.

Regards,
Helloway

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 38232 bytes --]