Re: [SPDK] SPDK Blobstore support object store?

[We We <simple_hlw at 163.com>]

[-- Attachment #1: Type: text/plain, Size: 7716 bytes --]

Hi Paul,

Thank you for adding me to the Trello board.

About these questions you mentioned, here are my thoughts:

Q1: Have you looked into Swift as a candidate for this activity or were you just using that as an example of a popular project?

A1: The blobkv provides the generic k-v interface module aimed to obj storage systems ( Such as: Swift, S3 ), so Swift and S3 are just used as examples of a popular project.

 
Q2: it’s probably the case that we’d want to target a specific object storage system and start with an investigation over how feasible it would be to bolt in SPDK at a high level.

A2: As the first answer said, the blobkv doesn’t target a specific object storage system. Our current thinking is that the blobkv is a standard k-v interface which located at the back end of S3, Swift, and so on.

 
For the reason above, we want to discuss with community about which is the proper size for the key/value pair in the blobkv? Could we design the k-v size as the same with the blob size so that the conversion from k-v to blob is minimum? I’m wondering if it is reasonable?

 
Regards,

Helloway

> 在 2017年9月1日，下午9:14，Luse, Paul E <paul.e.luse(a)intel.com <mailto:paul.e.luse(a)intel.com>> 写道：
> 
> Hi Helloway,
>  
> I just added you to the Trello board.  Wrt your answer about API, yes that makes sense.  Also, SPDK as it is today would need a tremendous amount of work to support an application level object storage API J  
>  
> So with S3 though, that’s kind of up to Amazon.  For Swift, although it’s an open source project, there are some challenges there as well.  Not that they’re not solvable of course, have you looked into Swift as a candidate for this activity or were you just using that as an example of a popular project?
>  
> Given the first point above, it’s probably the case that we’d want to target a specific object storage system and start with an investigation over how feasible it would be to bolt in SPDK at a high level and, depending on the project, engage with either that community or a company with a significant interest (installation) in making this happen don’t you think?  Don’t get me wrong, I think it’s a great idea I’m just trying to help talk out the right approach.  Either via some light abstraction layer or something we’d definitely want to identify a target system and see how interesting it might be to folks before we start any kind of detailed design discussions I think.
>  
> Thanks!
> Paul
>   <>
> From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of Yang, Ziye
> Sent: Friday, September 1, 2017 12:37 AM
> To: Storage Performance Development Kit <spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>>
> Subject: Re: [SPDK] SPDK Blobstore support object store?
>  
> Hi We We,
>  
> For the membership, could  you add by yourself. If you cannot,  I think that Jim and Daniel can help you.
>  
> From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of We We
> Sent: Friday, September 1, 2017 2:20 PM
> To: Storage Performance Development Kit <spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>>
> Subject: Re: [SPDK] SPDK Blobstore support object store?
>  
> Hi, @Paul @ Ziye
> Thank you for all of your reply.
> About these questions you mentioned, here are my thoughts:
> Q1: Why blobkv doesn't tying into existing applications that support something like S3 or native Swift?
> A1: Swift and S3 are different with blobkv at the hierarchy. Swift and S3 provide the application level k-v. However, the blobkv provides processing of generic k-v as a back-end behind Swift and S3，rather than the application level k-v.  
>  
> This actually leads to another important question I want to discuss with the community, which is the proper size for the key/value pair in the blobkv. Our current thinking is that we could design the k-v size as the same with the blob size so that the conversion from k-v to blob is minimum. I’m wondering if this is a reasonable choice?
>  
> Q2: Could you also put this in SPDK trello: https://trello.com/spdk? <https://trello.com/spdk?>
> A2: I have a trello account (simple_hlw(a)163.com <mailto:simple_hlw(a)163.com>), and I am not a member in SPDK trello. Do I need to be a member before I have permissions to put this in SPDK trello?
>  
> Regards,
> Helloway
>  
>  
>  
> 在 2017年9月1日，上午7:52，Yang, Ziye <ziye.yang(a)intel.com <mailto:ziye.yang(a)intel.com>> 写道：
>  
> Hi We We,
>  
> Could you also put this in SPDK trello: https:// <https://trello.com/spdk>trello.com/spdk <https://trello.com/spdk> ?
>  
> Thanks.
>  
>  
> From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of We We
> Sent: Friday, September 1, 2017 12:04 AM
> To: Storage Performance Development Kit <spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>>
> Subject: Re: [SPDK] SPDK Blobstore support object store?
>  
> Hi all,
> Thank you for your answers.
> I have submitted the proposal about  the blobkv (blobstore object) design ( https://github.com/spdk/spdk/pull/188/files?short_path=420ca87#diff-420ca87f40f0f8170bb68bc5c742b6dc <https://github.com/spdk/spdk/pull/188/files?short_path=420ca87#diff-420ca87f40f0f8170bb68bc5c742b6dc> ), please visit it.
> 
> Regards,
> Helloway
>  
> 在 2017年8月31日，下午8:16，Yang, Ziye <ziye.yang(a)intel.com <mailto:ziye.yang(a)intel.com>> 写道：
>  
> Hi,
>  
> Currently, SPDK can be integrated with Ceph in the following two ways:
>  
> 1       SPDK has  rbd bdev,  thus you can have SPDK iSCSI target which uses the exported rbd device by Ceph.
> 2       SPDK  can be integrated into bluestore in Ceph, the code is NVMEDEVICE.CC/h, located in src/os/bluestore/ folder. However  SPDK is not enabled in Ceph. You need to build with WITH_SPDK=on option. And these days, we are doing some work to make SPDK can be compiled default in Ceph.
>  
> Also in SPDK for object store support,  we do not have a detailed plan now.
>  
> Best Regards,
> Ziye Yang
>  
> From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of We We
> Sent: Monday, August 28, 2017 9:17 PM
> To: spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>
> Subject: [SPDK] SPDK Blobstore support object store?
>  
> Hi，
> In the source code of SPDk (https://github.com/spdk/spdk/tree/master/lib <https://github.com/spdk/spdk/tree/master/lib>), we can find spdk <https://github.com/spdk/spdk>/lib <https://github.com/spdk/spdk/tree/master/lib>/bdev <https://github.com/spdk/spdk/tree/master/lib/bdev>/ module that means  SPDk is able to be in favor of block store and accelerate ceph block store. However, I don not see anything about object store. Does SPDK support object store? Is there any plan to do with object store?
> 
> 
> 
> 
> Could someone can give me the answer?
>  
> Regards,
> Helloway
>  
>  
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
>  
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
>  
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 73311 bytes --]