* Re: [SPDK] SPDK Blobstore support object store?
@ 2017-08-28 13:53 Luse, Paul E
0 siblings, 0 replies; 21+ messages in thread
From: Luse, Paul E @ 2017-08-28 13:53 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 1442 bytes --]
Hi Helloway,
You are correct in that much of the focus on the various SPDK modules is block. If you’d like more general info, the community had summit meeting back in April, you can find that info here: http://www.spdk.io/news/2017/06/21/summit_videos/
Regarding object storage, I’m not aware of anyone out there actively working on anything but some of us have discussed it for sure and I suspect there’ll be more talk on this topic throughout the rest of the year. Is your question just a general one or did you have a specific API and/or use case in mind? Curious if anyone else out there has specific interests in this are as well?
Note that we’re also on IRC, freenode on channel #spdk
Thx
Paul
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Monday, August 28, 2017 6:17 AM
To: spdk(a)lists.01.org
Subject: [SPDK] SPDK Blobstore support object store?
Hi，
In the source code of SPDk (https://github.com/spdk/spdk/tree/master/lib), we can find spdk<https://github.com/spdk/spdk>/lib<https://github.com/spdk/spdk/tree/master/lib>/bdev<https://github.com/spdk/spdk/tree/master/lib/bdev>/ module that means SPDk is able to be in favor of block store and accelerate ceph block store. However, I don not see anything about object store. Does SPDK support object store? Is there any plan to do with object store?
Could someone can give me the answer?
Regards,
Helloway
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 7999 bytes --]
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [SPDK] SPDK Blobstore support object store?
@ 2017-09-07 14:19 Luse, Paul E
0 siblings, 0 replies; 21+ messages in thread
From: Luse, Paul E @ 2017-09-07 14:19 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 14749 bytes --]
That’s great, Helloway. Howard had mentioned Swift as an initial design target, I think this is a good choice given it’s all open, so starting to dig into exactly how this would bolt into a Swift Storage Node would be a good next step. I had also asked a question earlier about the value in the new proposed module. If it’s doing little more than translating API semantics and the new top end does not conform to any existing interface then it doesn’t seem like it makes a lot of sense so I’m guessing you had something in mind there. If we can dig into that a little bit as well we may find that slight modifications to blobstore itself (as opposed to a lightweight shim that translates between two new APIS) might be another approach.
Let us know what you think, thanks for bring up the idea and posting on Trello by the way!!!
-Paul
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Thursday, September 7, 2017 4:56 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: Re: [SPDK] SPDK Blobstore support object store?
Hi Paul,
I had submitted the "Blobkv Design Interface to Add” on https://trello.com/b/P5xBO7UR/things-to-do, please visit it.
Thx,
Helloway
在 2017年9月5日，下午9:46，Zhipeng Huang <zhipengh512(a)gmail.com<mailto:zhipengh512(a)gmail.com>> 写道：
Thanks Paul, I understand that we need a specific use case / scenario to jump start the work so that we could know if our design actually works :) I think we could start with Swift for the front end side :)
So I guess helloway could update the proposal with all the feedback and we continue discussion ?
On Sep 5, 2017 9:59 AM, "Luse, Paul E" <paul.e.luse(a)intel.com<mailto:paul.e.luse(a)intel.com>> wrote:
Hi Howard,
Yup, I understand. My points are talking about the back end, the point at which the storage application interfaces with the actual storage. In some cases, it’s direct, in some cases it’s via a filesystem and in some cases it a database or a combination of all of these things (with or without caching). It’s also not the case that objects are always sliced up, in some cases they are and when they are they may or may not be erasure coded and in the cases they’re not they are likely replicated. There are lots of variations is the point; even the very, very back end isn’t the same between object different storage systems ☺
Anyway, 1MB does sound like a good general number but again without some targeted usage I think you’ll find it challenging to get very many folks to work on it with you.
Thx
Paul
From: SPDK [mailto:spdk-bounces(a)lists.01.org<mailto:spdk-bounces(a)lists.01.org>] On Behalf Of Zhipeng Huang
Sent: Monday, September 4, 2017 5:55 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] SPDK Blobstore support object store?
Hi Paul,
This is Howard from Helloway's team. One of the reason that we don't want to target a specific object front-end (S3、Swift or Ceph RGW) is that (1)it is difficult to harmonize across these solutions and (2) we view spdk more of a back end role for these APIs
For most of the object storage services, they provide the functionalities of slicing up k/V pair they received to fit the back end storage. For example S3 would consume a 4M object but slice it up to a smaller size when it actually stores these objects, if I understand correctly. It will be the same case for Ceph or Swift.
Therefore we envisioned that blobkv provides the smaller sized object api that acts as the back end for S3、Swift or Ceph. I think 1M would be a reasonable kv size for blobkv ：)
On Sep 5, 2017 1:31 AM, "Luse, Paul E" <paul.e.luse(a)intel.com<mailto:paul.e.luse(a)intel.com>> wrote:
Hi Helloway,
OK, it just seems that w/o a target application (obj storage system or DB) some design decisions may get made that will make it less efficient than it could be to “fit” into at least one system really well – there are no standards for back end storage nodes for object storage systems so a “one size fits all” is highly unlikely (arguably impossible). Anyways, it’s certainly not a pre-requisite for you to work on it by any means, however it may impact how the maintainers look at your work as a possible contribution. Are you planning on designing something for contribution to the community? If so, some more discussion in this area is warranted for sure as everyone will want to make sure that whatever lands in the repo is not only robust/complete but readily usable as well.
Anyway wrt size, it seems like a reasonable starting approach however object storage can be a little tricky when it comes to object size as really anything goes. It’s more function of how the object storage system is being used as it is anything else. Most people are familiar with large objects being typical but that’s certainly not the case all of the time. There are many uses of Swift, for example, that have sub MB object sizes and choose an object storage system because of its API (REST, etc). With blobstore the page size is 4K and the default cluster size, which is the minimum size for a blob, is 1MB. The latter is configurable but not the former so if you had in mind objects of size 1MB or larger then, yeah, thinking of an object as a blob in terms of size makes sense.
There are many more considerations as well that I’m sure you’re aware of, curious as to what your thoughts were in some other areas like (just brainstorming here):
- Metadata storage support. Blobstore has a pretty minimal metadata (xattr) capability at least as compared to some of the common obj storage systems out there. A mapping of common capabilities here versus what blobstore does natively now would be a good exercise as closing the gaps could involve a decent amount of work in the new module and/or in blobstore itself. Most systems make some use of FS xattrs in combination with other non-FS mechanisms (KV DB) to store various types of metadata
- Metadata search: same kind of thoughts as above but keeping in mind that the search capability of the storage node could be severity limited if the actual back end storage didn’t have a pretty robust mechanism for caching and/or retrieving object metadata (blobstore does not)
- Support for containers: Containers in terms of how Swift uses them, basically S3 buckets. One could easily consider this out of scope if the idea is that this new thing is only targeted at objects themselves and not any sort of storage system metadata needs. I think that’s probably reasonable but I haven’t thought it all the way through
- Permissions requirements & enforcement
I’m sure we could come up a bunch more too and, again, having a target system to bolt into would help flesh out even more and drive a more robust design I think.
Thanks!!
Paul
From: SPDK [mailto:spdk-bounces(a)lists.01.org<mailto:spdk-bounces(a)lists.01.org>] On Behalf Of We We
Sent: Sunday, September 3, 2017 9:10 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] SPDK Blobstore support object store?
Hi Paul,
Thank you for adding me to the Trello board.
About these questions you mentioned, here are my thoughts:
Q1: Have you looked into Swift as a candidate for this activity or were you just using that as an example of a popular project?
A1: The blobkv provides the generic k-v interface module aimed to obj storage systems ( Such as: Swift, S3 ), so Swift and S3 are just used as examples of a popular project.
Q2: it’s probably the case that we’d want to target a specific object storage system and start with an investigation over how feasible it would be to bolt in SPDK at a high level.
A2: As the first answer said, the blobkv doesn’t target a specific object storage system. Our current thinking is that the blobkv is a standard k-v interface which located at the back end of S3, Swift, and so on.
For the reason above, we want to discuss with community about which is the proper size for the key/value pair in the blobkv? Could we design the k-v size as the same with the blob size so that the conversion from k-v to blob is minimum? I’m wondering if it is reasonable?
Regards,
Helloway
在 2017年9月1日，下午9:14，Luse, Paul E <paul.e.luse(a)intel.com<mailto:paul.e.luse(a)intel.com>> 写道：
Hi Helloway,
I just added you to the Trello board. Wrt your answer about API, yes that makes sense. Also, SPDK as it is today would need a tremendous amount of work to support an application level object storage API ☺
So with S3 though, that’s kind of up to Amazon. For Swift, although it’s an open source project, there are some challenges there as well. Not that they’re not solvable of course, have you looked into Swift as a candidate for this activity or were you just using that as an example of a popular project?
Given the first point above, it’s probably the case that we’d want to target a specific object storage system and start with an investigation over how feasible it would be to bolt in SPDK at a high level and, depending on the project, engage with either that community or a company with a significant interest (installation) in making this happen don’t you think? Don’t get me wrong, I think it’s a great idea I’m just trying to help talk out the right approach. Either via some light abstraction layer or something we’d definitely want to identify a target system and see how interesting it might be to folks before we start any kind of detailed design discussions I think.
Thanks!
Paul
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Yang, Ziye
Sent: Friday, September 1, 2017 12:37 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] SPDK Blobstore support object store?
Hi We We,
For the membership, could you add by yourself. If you cannot, I think that Jim and Daniel can help you.
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Friday, September 1, 2017 2:20 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] SPDK Blobstore support object store?
Hi, @Paul @ Ziye
Thank you for all of your reply.
About these questions you mentioned, here are my thoughts:
Q1: Why blobkv doesn't tying into existing applications that support something like S3 or native Swift?
A1: Swift and S3 are different with blobkv at the hierarchy. Swift and S3 provide the application level k-v. However, the blobkv provides processing of generic k-v as a back-end behind Swift and S3，rather than the application level k-v.
This actually leads to another important question I want to discuss with the community, which is the proper size for the key/value pair in the blobkv. Our current thinking is that we could design the k-v size as the same with the blob size so that the conversion from k-v to blob is minimum. I’m wondering if this is a reasonable choice?
Q2: Could you also put this in SPDK trello: https://trello.com/spdk?
A2: I have a trello account (simple_hlw(a)163.com<mailto:simple_hlw(a)163.com>), and I am not a member in SPDK trello. Do I need to be a member before I have permissions to put this in SPDK trello?
Regards,
Helloway
在 2017年9月1日，上午7:52，Yang, Ziye <ziye.yang(a)intel.com<mailto:ziye.yang(a)intel.com>> 写道：
Hi We We,
Could you also put this in SPDK trello: https://<https://trello.com/spdk>trello.com/spdk<https://trello.com/spdk> ?
Thanks.
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Friday, September 1, 2017 12:04 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] SPDK Blobstore support object store?
Hi all,
Thank you for your answers.
I have submitted the proposal about the blobkv (blobstore object) design ( https://github.com/spdk/spdk/pull/188/files?short_path=420ca87#diff-420ca87f40f0f8170bb68bc5c742b6dc ), please visit it.
Regards,
Helloway
在 2017年8月31日，下午8:16，Yang, Ziye <ziye.yang(a)intel.com<mailto:ziye.yang(a)intel.com>> 写道：
Hi,
Currently, SPDK can be integrated with Ceph in the following two ways:
1 SPDK has rbd bdev, thus you can have SPDK iSCSI target which uses the exported rbd device by Ceph.
2 SPDK can be integrated into bluestore in Ceph, the code is NVMEDEVICE.CC/h<http://nvmedevice.cc/h>, located in src/os/bluestore/ folder. However SPDK is not enabled in Ceph. You need to build with WITH_SPDK=on option. And these days, we are doing some work to make SPDK can be compiled default in Ceph.
Also in SPDK for object store support, we do not have a detailed plan now.
Best Regards,
Ziye Yang
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Monday, August 28, 2017 9:17 PM
To: spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>
Subject: [SPDK] SPDK Blobstore support object store?
Hi，
In the source code of SPDk (https://github.com/spdk/spdk/tree/master/lib), we can find spdk<https://github.com/spdk/spdk>/lib<https://github.com/spdk/spdk/tree/master/lib>/bdev<https://github.com/spdk/spdk/tree/master/lib/bdev>/ module that means SPDk is able to be in favor of block store and accelerate ceph block store. However, I don not see anything about object store. Does SPDK support object store? Is there any plan to do with object store?
Could someone can give me the answer?
Regards,
Helloway
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 57117 bytes --]
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [SPDK] SPDK Blobstore support object store?
@ 2017-09-07 11:55 We We
0 siblings, 0 replies; 21+ messages in thread
From: We We @ 2017-09-07 11:55 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 15247 bytes --]
Hi Paul,
I had submitted the "Blobkv Design Interface to Add” on https://trello.com/b/P5xBO7UR/things-to-do <https://trello.com/b/P5xBO7UR/things-to-do>, please visit it.
Thx,
Helloway
> 在 2017年9月5日，下午9:46，Zhipeng Huang <zhipengh512(a)gmail.com <mailto:zhipengh512(a)gmail.com>> 写道：
>
> Thanks Paul, I understand that we need a specific use case / scenario to jump start the work so that we could know if our design actually works :) I think we could start with Swift for the front end side :)
>
> So I guess helloway could update the proposal with all the feedback and we continue discussion ?
>
> On Sep 5, 2017 9:59 AM, "Luse, Paul E" <paul.e.luse(a)intel.com <mailto:paul.e.luse(a)intel.com>> wrote:
> Hi Howard,
>
>
>
> Yup, I understand. My points are talking about the back end, the point at which the storage application interfaces with the actual storage. In some cases, it’s direct, in some cases it’s via a filesystem and in some cases it a database or a combination of all of these things (with or without caching). It’s also not the case that objects are always sliced up, in some cases they are and when they are they may or may not be erasure coded and in the cases they’re not they are likely replicated. There are lots of variations is the point; even the very, very back end isn’t the same between object different storage systems J
>
> <>
> Anyway, 1MB does sound like a good general number but again without some targeted usage I think you’ll find it challenging to get very many folks to work on it with you.
>
>
>
> Thx
>
> Paul
>
>
>
> From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of Zhipeng Huang
> Sent: Monday, September 4, 2017 5:55 PM
> To: Storage Performance Development Kit <spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>>
> Subject: Re: [SPDK] SPDK Blobstore support object store?
>
>
>
> Hi Paul,
>
>
>
> This is Howard from Helloway's team. One of the reason that we don't want to target a specific object front-end (S3、Swift or Ceph RGW) is that (1)it is difficult to harmonize across these solutions and (2) we view spdk more of a back end role for these APIs
>
>
>
> For most of the object storage services, they provide the functionalities of slicing up k/V pair they received to fit the back end storage. For example S3 would consume a 4M object but slice it up to a smaller size when it actually stores these objects, if I understand correctly. It will be the same case for Ceph or Swift.
>
>
>
> Therefore we envisioned that blobkv provides the smaller sized object api that acts as the back end for S3、Swift or Ceph. I think 1M would be a reasonable kv size for blobkv ：)
>
>
>
> On Sep 5, 2017 1:31 AM, "Luse, Paul E" <paul.e.luse(a)intel.com <mailto:paul.e.luse(a)intel.com>> wrote:
>
> Hi Helloway,
>
>
>
> OK, it just seems that w/o a target application (obj storage system or DB) some design decisions may get made that will make it less efficient than it could be to “fit” into at least one system really well – there are no standards for back end storage nodes for object storage systems so a “one size fits all” is highly unlikely (arguably impossible). Anyways, it’s certainly not a pre-requisite for you to work on it by any means, however it may impact how the maintainers look at your work as a possible contribution. Are you planning on designing something for contribution to the community? If so, some more discussion in this area is warranted for sure as everyone will want to make sure that whatever lands in the repo is not only robust/complete but readily usable as well.
>
>
>
> Anyway wrt size, it seems like a reasonable starting approach however object storage can be a little tricky when it comes to object size as really anything goes. It’s more function of how the object storage system is being used as it is anything else. Most people are familiar with large objects being typical but that’s certainly not the case all of the time. There are many uses of Swift, for example, that have sub MB object sizes and choose an object storage system because of its API (REST, etc). With blobstore the page size is 4K and the default cluster size, which is the minimum size for a blob, is 1MB. The latter is configurable but not the former so if you had in mind objects of size 1MB or larger then, yeah, thinking of an object as a blob in terms of size makes sense.
>
>
>
> There are many more considerations as well that I’m sure you’re aware of, curious as to what your thoughts were in some other areas like (just brainstorming here):
>
>
>
> - Metadata storage support. Blobstore has a pretty minimal metadata (xattr) capability at least as compared to some of the common obj storage systems out there. A mapping of common capabilities here versus what blobstore does natively now would be a good exercise as closing the gaps could involve a decent amount of work in the new module and/or in blobstore itself. Most systems make some use of FS xattrs in combination with other non-FS mechanisms (KV DB) to store various types of metadata
>
> - Metadata search: same kind of thoughts as above but keeping in mind that the search capability of the storage node could be severity limited if the actual back end storage didn’t have a pretty robust mechanism for caching and/or retrieving object metadata (blobstore does not)
>
> - Support for containers: Containers in terms of how Swift uses them, basically S3 buckets. One could easily consider this out of scope if the idea is that this new thing is only targeted at objects themselves and not any sort of storage system metadata needs. I think that’s probably reasonable but I haven’t thought it all the way through
>
> - Permissions requirements & enforcement
>
>
>
> I’m sure we could come up a bunch more too and, again, having a target system to bolt into would help flesh out even more and drive a more robust design I think.
>
>
>
> Thanks!!
>
> Paul
>
> <>
> From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of We We
> Sent: Sunday, September 3, 2017 9:10 PM
> To: Storage Performance Development Kit <spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>>
> Subject: Re: [SPDK] SPDK Blobstore support object store?
>
>
>
> Hi Paul,
>
> Thank you for adding me to the Trello board.
>
> About these questions you mentioned, here are my thoughts:
>
> Q1: Have you looked into Swift as a candidate for this activity or were you just using that as an example of a popular project?
>
> A1: The blobkv provides the generic k-v interface module aimed to obj storage systems ( Such as: Swift, S3 ), so Swift and S3 are just used as examples of a popular project.
>
>
>
> Q2: it’s probably the case that we’d want to target a specific object storage system and start with an investigation over how feasible it would be to bolt in SPDK at a high level.
>
> A2: As the first answer said, the blobkv doesn’t target a specific object storage system. Our current thinking is that the blobkv is a standard k-v interface which located at the back end of S3, Swift, and so on.
>
>
>
> For the reason above, we want to discuss with community about which is the proper size for the key/value pair in the blobkv? Could we design the k-v size as the same with the blob size so that the conversion from k-v to blob is minimum? I’m wondering if it is reasonable?
>
>
>
> Regards,
>
> Helloway
>
> 在 2017年9月1日，下午9:14，Luse, Paul E <paul.e.luse(a)intel.com <mailto:paul.e.luse(a)intel.com>> 写道：
>
>
>
> Hi Helloway,
>
>
>
> I just added you to the Trello board. Wrt your answer about API, yes that makes sense. Also, SPDK as it is today would need a tremendous amount of work to support an application level object storage API J
>
>
>
> So with S3 though, that’s kind of up to Amazon. For Swift, although it’s an open source project, there are some challenges there as well. Not that they’re not solvable of course, have you looked into Swift as a candidate for this activity or were you just using that as an example of a popular project?
>
>
>
> Given the first point above, it’s probably the case that we’d want to target a specific object storage system and start with an investigation over how feasible it would be to bolt in SPDK at a high level and, depending on the project, engage with either that community or a company with a significant interest (installation) in making this happen don’t you think? Don’t get me wrong, I think it’s a great idea I’m just trying to help talk out the right approach. Either via some light abstraction layer or something we’d definitely want to identify a target system and see how interesting it might be to folks before we start any kind of detailed design discussions I think.
>
>
>
> Thanks!
>
> Paul
>
>
>
> From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of Yang, Ziye
> Sent: Friday, September 1, 2017 12:37 AM
> To: Storage Performance Development Kit <spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>>
> Subject: Re: [SPDK] SPDK Blobstore support object store?
>
>
>
> Hi We We,
>
>
>
> For the membership, could you add by yourself. If you cannot, I think that Jim and Daniel can help you.
>
>
>
> From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of We We
> Sent: Friday, September 1, 2017 2:20 PM
> To: Storage Performance Development Kit <spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>>
> Subject: Re: [SPDK] SPDK Blobstore support object store?
>
>
>
> Hi, @Paul @ Ziye
>
> Thank you for all of your reply.
>
> About these questions you mentioned, here are my thoughts:
>
> Q1: Why blobkv doesn't tying into existing applications that support something like S3 or native Swift?
>
> A1: Swift and S3 are different with blobkv at the hierarchy. Swift and S3 provide the application level k-v. However, the blobkv provides processing of generic k-v as a back-end behind Swift and S3，rather than the application level k-v.
>
>
>
> This actually leads to another important question I want to discuss with the community, which is the proper size for the key/value pair in the blobkv. Our current thinking is that we could design the k-v size as the same with the blob size so that the conversion from k-v to blob is minimum. I’m wondering if this is a reasonable choice?
>
>
>
> Q2: Could you also put this in SPDK trello: https://trello.com/spdk? <https://trello.com/spdk?>
> A2: I have a trello account (simple_hlw(a)163.com <mailto:simple_hlw(a)163.com>), and I am not a member in SPDK trello. Do I need to be a member before I have permissions to put this in SPDK trello?
>
>
>
> Regards,
>
> Helloway
>
>
>
>
>
>
>
> 在 2017年9月1日，上午7:52，Yang, Ziye <ziye.yang(a)intel.com <mailto:ziye.yang(a)intel.com>> 写道：
>
>
>
> Hi We We,
>
>
>
> Could you also put this in SPDK trello: https:// <https://trello.com/spdk>trello.com/spdk <https://trello.com/spdk> ?
>
>
>
> Thanks.
>
>
>
>
>
> From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of We We
> Sent: Friday, September 1, 2017 12:04 AM
> To: Storage Performance Development Kit <spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>>
> Subject: Re: [SPDK] SPDK Blobstore support object store?
>
>
>
> Hi all,
>
> Thank you for your answers.
>
> I have submitted the proposal about the blobkv (blobstore object) design ( https://github.com/spdk/spdk/pull/188/files?short_path=420ca87#diff-420ca87f40f0f8170bb68bc5c742b6dc <https://github.com/spdk/spdk/pull/188/files?short_path=420ca87#diff-420ca87f40f0f8170bb68bc5c742b6dc> ), please visit it.
>
>
> Regards,
> Helloway
>
>
>
> 在 2017年8月31日，下午8:16，Yang, Ziye <ziye.yang(a)intel.com <mailto:ziye.yang(a)intel.com>> 写道：
>
>
>
> Hi,
>
>
>
> Currently, SPDK can be integrated with Ceph in the following two ways:
>
>
>
> 1 SPDK has rbd bdev, thus you can have SPDK iSCSI target which uses the exported rbd device by Ceph.
>
> 2 SPDK can be integrated into bluestore in Ceph, the code isNVMEDEVICE.CC/h <http://nvmedevice.cc/h>, located in src/os/bluestore/ folder. However SPDK is not enabled in Ceph. You need to build with WITH_SPDK=on option. And these days, we are doing some work to make SPDK can be compiled default in Ceph.
>
>
>
> Also in SPDK for object store support, we do not have a detailed plan now.
>
>
>
> Best Regards,
>
> Ziye Yang
>
>
>
> From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of We We
> Sent: Monday, August 28, 2017 9:17 PM
> To: spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>
> Subject: [SPDK] SPDK Blobstore support object store?
>
>
>
> Hi，
>
> In the source code of SPDk (https://github.com/spdk/spdk/tree/master/lib <https://github.com/spdk/spdk/tree/master/lib>), we can find spdk <https://github.com/spdk/spdk>/lib <https://github.com/spdk/spdk/tree/master/lib>/bdev <https://github.com/spdk/spdk/tree/master/lib/bdev>/ module that means SPDk is able to be in favor of block store and accelerate ceph block store. However, I don not see anything about object store. Does SPDK support object store? Is there any plan to do with object store?
>
>
>
>
>
> Could someone can give me the answer?
>
>
>
> Regards,
>
> Helloway
>
>
>
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
>
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk
>
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 50290 bytes --]
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [SPDK] SPDK Blobstore support object store?
@ 2017-09-07 11:52 We We
0 siblings, 0 replies; 21+ messages in thread
From: We We @ 2017-09-07 11:52 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 15185 bytes --]
Hi Paul,
I had submitted the "Blobkv Design Interface to Add” on https://trello.com/b/P5xBO7UR/things-to-do <https://trello.com/b/P5xBO7UR/things-to-do>, please visit it.
Thx,
Helloway
> 在 2017年9月5日，下午9:46，Zhipeng Huang <zhipengh512(a)gmail.com> 写道：
>
> Thanks Paul, I understand that we need a specific use case / scenario to jump start the work so that we could know if our design actually works :) I think we could start with Swift for the front end side :)
>
> So I guess helloway could update the proposal with all the feedback and we continue discussion ?
>
> On Sep 5, 2017 9:59 AM, "Luse, Paul E" <paul.e.luse(a)intel.com <mailto:paul.e.luse(a)intel.com>> wrote:
> Hi Howard,
>
>
>
> Yup, I understand. My points are talking about the back end, the point at which the storage application interfaces with the actual storage. In some cases, it’s direct, in some cases it’s via a filesystem and in some cases it a database or a combination of all of these things (with or without caching). It’s also not the case that objects are always sliced up, in some cases they are and when they are they may or may not be erasure coded and in the cases they’re not they are likely replicated. There are lots of variations is the point; even the very, very back end isn’t the same between object different storage systems J
>
> <>
> Anyway, 1MB does sound like a good general number but again without some targeted usage I think you’ll find it challenging to get very many folks to work on it with you.
>
>
>
> Thx
>
> Paul
>
>
>
> From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of Zhipeng Huang
> Sent: Monday, September 4, 2017 5:55 PM
> To: Storage Performance Development Kit <spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>>
> Subject: Re: [SPDK] SPDK Blobstore support object store?
>
>
>
> Hi Paul,
>
>
>
> This is Howard from Helloway's team. One of the reason that we don't want to target a specific object front-end (S3、Swift or Ceph RGW) is that (1)it is difficult to harmonize across these solutions and (2) we view spdk more of a back end role for these APIs
>
>
>
> For most of the object storage services, they provide the functionalities of slicing up k/V pair they received to fit the back end storage. For example S3 would consume a 4M object but slice it up to a smaller size when it actually stores these objects, if I understand correctly. It will be the same case for Ceph or Swift.
>
>
>
> Therefore we envisioned that blobkv provides the smaller sized object api that acts as the back end for S3、Swift or Ceph. I think 1M would be a reasonable kv size for blobkv ：)
>
>
>
> On Sep 5, 2017 1:31 AM, "Luse, Paul E" <paul.e.luse(a)intel.com <mailto:paul.e.luse(a)intel.com>> wrote:
>
> Hi Helloway,
>
>
>
> OK, it just seems that w/o a target application (obj storage system or DB) some design decisions may get made that will make it less efficient than it could be to “fit” into at least one system really well – there are no standards for back end storage nodes for object storage systems so a “one size fits all” is highly unlikely (arguably impossible). Anyways, it’s certainly not a pre-requisite for you to work on it by any means, however it may impact how the maintainers look at your work as a possible contribution. Are you planning on designing something for contribution to the community? If so, some more discussion in this area is warranted for sure as everyone will want to make sure that whatever lands in the repo is not only robust/complete but readily usable as well.
>
>
>
> Anyway wrt size, it seems like a reasonable starting approach however object storage can be a little tricky when it comes to object size as really anything goes. It’s more function of how the object storage system is being used as it is anything else. Most people are familiar with large objects being typical but that’s certainly not the case all of the time. There are many uses of Swift, for example, that have sub MB object sizes and choose an object storage system because of its API (REST, etc). With blobstore the page size is 4K and the default cluster size, which is the minimum size for a blob, is 1MB. The latter is configurable but not the former so if you had in mind objects of size 1MB or larger then, yeah, thinking of an object as a blob in terms of size makes sense.
>
>
>
> There are many more considerations as well that I’m sure you’re aware of, curious as to what your thoughts were in some other areas like (just brainstorming here):
>
>
>
> - Metadata storage support. Blobstore has a pretty minimal metadata (xattr) capability at least as compared to some of the common obj storage systems out there. A mapping of common capabilities here versus what blobstore does natively now would be a good exercise as closing the gaps could involve a decent amount of work in the new module and/or in blobstore itself. Most systems make some use of FS xattrs in combination with other non-FS mechanisms (KV DB) to store various types of metadata
>
> - Metadata search: same kind of thoughts as above but keeping in mind that the search capability of the storage node could be severity limited if the actual back end storage didn’t have a pretty robust mechanism for caching and/or retrieving object metadata (blobstore does not)
>
> - Support for containers: Containers in terms of how Swift uses them, basically S3 buckets. One could easily consider this out of scope if the idea is that this new thing is only targeted at objects themselves and not any sort of storage system metadata needs. I think that’s probably reasonable but I haven’t thought it all the way through
>
> - Permissions requirements & enforcement
>
>
>
> I’m sure we could come up a bunch more too and, again, having a target system to bolt into would help flesh out even more and drive a more robust design I think.
>
>
>
> Thanks!!
>
> Paul
>
> <>
> From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of We We
> Sent: Sunday, September 3, 2017 9:10 PM
> To: Storage Performance Development Kit <spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>>
> Subject: Re: [SPDK] SPDK Blobstore support object store?
>
>
>
> Hi Paul,
>
> Thank you for adding me to the Trello board.
>
> About these questions you mentioned, here are my thoughts:
>
> Q1: Have you looked into Swift as a candidate for this activity or were you just using that as an example of a popular project?
>
> A1: The blobkv provides the generic k-v interface module aimed to obj storage systems ( Such as: Swift, S3 ), so Swift and S3 are just used as examples of a popular project.
>
>
>
> Q2: it’s probably the case that we’d want to target a specific object storage system and start with an investigation over how feasible it would be to bolt in SPDK at a high level.
>
> A2: As the first answer said, the blobkv doesn’t target a specific object storage system. Our current thinking is that the blobkv is a standard k-v interface which located at the back end of S3, Swift, and so on.
>
>
>
> For the reason above, we want to discuss with community about which is the proper size for the key/value pair in the blobkv? Could we design the k-v size as the same with the blob size so that the conversion from k-v to blob is minimum? I’m wondering if it is reasonable?
>
>
>
> Regards,
>
> Helloway
>
> 在 2017年9月1日，下午9:14，Luse, Paul E <paul.e.luse(a)intel.com <mailto:paul.e.luse(a)intel.com>> 写道：
>
>
>
> Hi Helloway,
>
>
>
> I just added you to the Trello board. Wrt your answer about API, yes that makes sense. Also, SPDK as it is today would need a tremendous amount of work to support an application level object storage API J
>
>
>
> So with S3 though, that’s kind of up to Amazon. For Swift, although it’s an open source project, there are some challenges there as well. Not that they’re not solvable of course, have you looked into Swift as a candidate for this activity or were you just using that as an example of a popular project?
>
>
>
> Given the first point above, it’s probably the case that we’d want to target a specific object storage system and start with an investigation over how feasible it would be to bolt in SPDK at a high level and, depending on the project, engage with either that community or a company with a significant interest (installation) in making this happen don’t you think? Don’t get me wrong, I think it’s a great idea I’m just trying to help talk out the right approach. Either via some light abstraction layer or something we’d definitely want to identify a target system and see how interesting it might be to folks before we start any kind of detailed design discussions I think.
>
>
>
> Thanks!
>
> Paul
>
>
>
> From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of Yang, Ziye
> Sent: Friday, September 1, 2017 12:37 AM
> To: Storage Performance Development Kit <spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>>
> Subject: Re: [SPDK] SPDK Blobstore support object store?
>
>
>
> Hi We We,
>
>
>
> For the membership, could you add by yourself. If you cannot, I think that Jim and Daniel can help you.
>
>
>
> From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of We We
> Sent: Friday, September 1, 2017 2:20 PM
> To: Storage Performance Development Kit <spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>>
> Subject: Re: [SPDK] SPDK Blobstore support object store?
>
>
>
> Hi, @Paul @ Ziye
>
> Thank you for all of your reply.
>
> About these questions you mentioned, here are my thoughts:
>
> Q1: Why blobkv doesn't tying into existing applications that support something like S3 or native Swift?
>
> A1: Swift and S3 are different with blobkv at the hierarchy. Swift and S3 provide the application level k-v. However, the blobkv provides processing of generic k-v as a back-end behind Swift and S3，rather than the application level k-v.
>
>
>
> This actually leads to another important question I want to discuss with the community, which is the proper size for the key/value pair in the blobkv. Our current thinking is that we could design the k-v size as the same with the blob size so that the conversion from k-v to blob is minimum. I’m wondering if this is a reasonable choice?
>
>
>
> Q2: Could you also put this in SPDK trello: https://trello.com/spdk? <https://trello.com/spdk?>
> A2: I have a trello account (simple_hlw(a)163.com <mailto:simple_hlw(a)163.com>), and I am not a member in SPDK trello. Do I need to be a member before I have permissions to put this in SPDK trello?
>
>
>
> Regards,
>
> Helloway
>
>
>
>
>
>
>
> 在 2017年9月1日，上午7:52，Yang, Ziye <ziye.yang(a)intel.com <mailto:ziye.yang(a)intel.com>> 写道：
>
>
>
> Hi We We,
>
>
>
> Could you also put this in SPDK trello: https:// <https://trello.com/spdk>trello.com/spdk <https://trello.com/spdk> ?
>
>
>
> Thanks.
>
>
>
>
>
> From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of We We
> Sent: Friday, September 1, 2017 12:04 AM
> To: Storage Performance Development Kit <spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>>
> Subject: Re: [SPDK] SPDK Blobstore support object store?
>
>
>
> Hi all,
>
> Thank you for your answers.
>
> I have submitted the proposal about the blobkv (blobstore object) design ( https://github.com/spdk/spdk/pull/188/files?short_path=420ca87#diff-420ca87f40f0f8170bb68bc5c742b6dc <https://github.com/spdk/spdk/pull/188/files?short_path=420ca87#diff-420ca87f40f0f8170bb68bc5c742b6dc> ), please visit it.
>
>
> Regards,
> Helloway
>
>
>
> 在 2017年8月31日，下午8:16，Yang, Ziye <ziye.yang(a)intel.com <mailto:ziye.yang(a)intel.com>> 写道：
>
>
>
> Hi,
>
>
>
> Currently, SPDK can be integrated with Ceph in the following two ways:
>
>
>
> 1 SPDK has rbd bdev, thus you can have SPDK iSCSI target which uses the exported rbd device by Ceph.
>
> 2 SPDK can be integrated into bluestore in Ceph, the code isNVMEDEVICE.CC/h <http://nvmedevice.cc/h>, located in src/os/bluestore/ folder. However SPDK is not enabled in Ceph. You need to build with WITH_SPDK=on option. And these days, we are doing some work to make SPDK can be compiled default in Ceph.
>
>
>
> Also in SPDK for object store support, we do not have a detailed plan now.
>
>
>
> Best Regards,
>
> Ziye Yang
>
>
>
> From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of We We
> Sent: Monday, August 28, 2017 9:17 PM
> To: spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>
> Subject: [SPDK] SPDK Blobstore support object store?
>
>
>
> Hi，
>
> In the source code of SPDk (https://github.com/spdk/spdk/tree/master/lib <https://github.com/spdk/spdk/tree/master/lib>), we can find spdk <https://github.com/spdk/spdk>/lib <https://github.com/spdk/spdk/tree/master/lib>/bdev <https://github.com/spdk/spdk/tree/master/lib/bdev>/ module that means SPDk is able to be in favor of block store and accelerate ceph block store. However, I don not see anything about object store. Does SPDK support object store? Is there any plan to do with object store?
>
>
>
>
>
> Could someone can give me the answer?
>
>
>
> Regards,
>
> Helloway
>
>
>
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
>
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 50024 bytes --]
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [SPDK] SPDK Blobstore support object store?
@ 2017-09-05 13:46 Zhipeng Huang
0 siblings, 0 replies; 21+ messages in thread
From: Zhipeng Huang @ 2017-09-05 13:46 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 13845 bytes --]
Thanks Paul, I understand that we need a specific use case / scenario to
jump start the work so that we could know if our design actually works :) I
think we could start with Swift for the front end side :)
So I guess helloway could update the proposal with all the feedback and we
continue discussion ?
On Sep 5, 2017 9:59 AM, "Luse, Paul E" <paul.e.luse(a)intel.com> wrote:
> Hi Howard,
>
>
>
> Yup, I understand. My points are talking about the back end, the point at
> which the storage application interfaces with the actual storage. In some
> cases, it’s direct, in some cases it’s via a filesystem and in some cases
> it a database or a combination of all of these things (with or without
> caching). It’s also not the case that objects are always sliced up, in some
> cases they are and when they are they may or may not be erasure coded and
> in the cases they’re not they are likely replicated. There are lots of
> variations is the point; even the very, very back end isn’t the same
> between object different storage systems J
>
>
>
> Anyway, 1MB does sound like a good general number but again without some
> targeted usage I think you’ll find it challenging to get very many folks to
> work on it with you.
>
>
>
> Thx
>
> Paul
>
>
>
> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org] *On Behalf Of *Zhipeng
> Huang
> *Sent:* Monday, September 4, 2017 5:55 PM
> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
> *Subject:* Re: [SPDK] SPDK Blobstore support object store?
>
>
>
> Hi Paul,
>
>
>
> This is Howard from Helloway's team. One of the reason that we don't want
> to target a specific object front-end (S3、Swift or Ceph RGW) is that
> (1)it is difficult to harmonize across these solutions and (2) we view spdk
> more of a back end role for these APIs
>
>
>
> For most of the object storage services, they provide the functionalities
> of slicing up k/V pair they received to fit the back end storage. For
> example S3 would consume a 4M object but slice it up to a smaller size when
> it actually stores these objects, if I understand correctly. It will be the
> same case for Ceph or Swift.
>
>
>
> Therefore we envisioned that blobkv provides the smaller sized object api
> that acts as the back end for S3、Swift or Ceph. I think 1M would be a
> reasonable kv size for blobkv ：)
>
>
>
> On Sep 5, 2017 1:31 AM, "Luse, Paul E" <paul.e.luse(a)intel.com> wrote:
>
> Hi Helloway,
>
>
>
> OK, it just seems that w/o a target application (obj storage system or DB)
> some design decisions may get made that will make it less efficient than it
> could be to “fit” into at least one system really well – there are no
> standards for back end storage nodes for object storage systems so a “one
> size fits all” is highly unlikely (arguably impossible). Anyways, it’s
> certainly not a pre-requisite for you to work on it by any means, however
> it may impact how the maintainers look at your work as a possible
> contribution. *Are you planning on designing something for contribution
> to the community?* If so, some more discussion in this area is warranted
> for sure as everyone will want to make sure that whatever lands in the repo
> is not only robust/complete but readily usable as well.
>
>
>
> Anyway wrt size, it seems like a reasonable starting approach however
> object storage can be a little tricky when it comes to object size as
> really anything goes. It’s more function of how the object storage system
> is being used as it is anything else. Most people are familiar with large
> objects being typical but that’s certainly not the case all of the time.
> There are many uses of Swift, for example, that have sub MB object sizes
> and choose an object storage system because of its API (REST, etc). With
> blobstore the page size is 4K and the default cluster size, which is the
> minimum size for a blob, is 1MB. The latter is configurable but not the
> former so if you had in mind objects of size 1MB or larger then, yeah,
> thinking of an object as a blob in terms of size makes sense.
>
>
>
> There are many more considerations as well that I’m sure you’re aware of,
> curious as to what your thoughts were in some other areas like (just
> brainstorming here):
>
>
>
> - Metadata storage support. Blobstore has a pretty minimal
> metadata (xattr) capability at least as compared to some of the common obj
> storage systems out there. A mapping of common capabilities here versus
> what blobstore does natively now would be a good exercise as closing the
> gaps could involve a decent amount of work in the new module and/or in
> blobstore itself. Most systems make some use of FS xattrs in combination
> with other non-FS mechanisms (KV DB) to store various types of metadata
>
> - Metadata search: same kind of thoughts as above but keeping in
> mind that the search capability of the storage node could be severity
> limited if the actual back end storage didn’t have a pretty robust
> mechanism for caching and/or retrieving object metadata (blobstore does not)
>
> - Support for containers: Containers in terms of how Swift uses
> them, basically S3 buckets. One could easily consider this out of scope if
> the idea is that this new thing is only targeted at objects themselves and
> not any sort of storage system metadata needs. I think that’s probably
> reasonable but I haven’t thought it all the way through
>
> - Permissions requirements & enforcement
>
>
>
> I’m sure we could come up a bunch more too and, again, having a target
> system to bolt into would help flesh out even more and drive a more robust
> design I think.
>
>
>
> Thanks!!
>
> Paul
>
>
>
> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org] *On Behalf Of *We We
> *Sent:* Sunday, September 3, 2017 9:10 PM
> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
> *Subject:* Re: [SPDK] SPDK Blobstore support object store?
>
>
>
> Hi Paul,
>
> Thank you for adding me to the Trello board.
>
> About these questions you mentioned, here are my thoughts:
>
> Q1: Have you looked into Swift as a candidate for this activity or were
> you just using that as an example of a popular project?
>
> A1: The blobkv provides the generic k-v interface module aimed to obj
> storage systems ( Such as: Swift, S3 ), so Swift and S3 are just used as
> examples of a popular project.
>
>
>
> Q2: it’s probably the case that we’d want to target a specific object
> storage system and start with an investigation over how feasible it would
> be to bolt in SPDK at a high level.
>
> A2: As the first answer said, the blobkv doesn’t target a specific object
> storage system. Our current thinking is that the blobkv is a standard k-v
> interface which located at the back end of S3, Swift, and so on.
>
>
>
> For the reason above, we want to discuss with community about which is the
> proper size for the key/value pair in the blobkv? Could we design the k-v
> size as the same with the blob size so that the conversion from k-v to blob
> is minimum? I’m wondering if it is reasonable?
>
>
>
> Regards,
>
> Helloway
>
> 在 2017年9月1日，下午9:14，Luse, Paul E <paul.e.luse(a)intel.com> 写道：
>
>
>
> Hi Helloway,
>
>
>
> I just added you to the Trello board. Wrt your answer about API, yes that
> makes sense. Also, SPDK as it is today would need a tremendous amount of
> work to support an application level object storage API J
>
>
>
> So with S3 though, that’s kind of up to Amazon. For Swift, although it’s
> an open source project, there are some challenges there as well. Not that
> they’re not solvable of course, have you looked into Swift as a candidate
> for this activity or were you just using that as an example of a popular
> project?
>
>
>
> Given the first point above, it’s probably the case that we’d want to
> target a specific object storage system and start with an investigation
> over how feasible it would be to bolt in SPDK at a high level and,
> depending on the project, engage with either that community or a company
> with a significant interest (installation) in making this happen don’t you
> think? Don’t get me wrong, I think it’s a great idea I’m just trying to
> help talk out the right approach. Either via some light abstraction layer
> or something we’d definitely want to identify a target system and see how
> interesting it might be to folks before we start any kind of detailed
> design discussions I think.
>
>
>
> Thanks!
>
> Paul
>
>
>
> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org <spdk-bounces(a)lists.01.org>
> ] *On Behalf Of *Yang, Ziye
> *Sent:* Friday, September 1, 2017 12:37 AM
> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
> *Subject:* Re: [SPDK] SPDK Blobstore support object store?
>
>
>
> Hi We We,
>
>
>
> For the membership, could you add by yourself. If you cannot, I think
> that Jim and Daniel can help you.
>
>
>
> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org <spdk-bounces(a)lists.01.org>
> ] *On Behalf Of *We We
> *Sent:* Friday, September 1, 2017 2:20 PM
> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
> *Subject:* Re: [SPDK] SPDK Blobstore support object store?
>
>
>
> Hi, @Paul @ Ziye
>
> Thank you for all of your reply.
>
> About these questions you mentioned, here are my thoughts:
>
> Q1: Why blobkv doesn't tying into existing applications that support
> something like S3 or native Swift?
>
> A1: Swift and S3 are different with blobkv at the hierarchy. Swift and S3
> provide the application level k-v. However, the blobkv provides processing
> of generic k-v as a back-end behind Swift and S3，rather than the
> application level k-v.
>
>
>
> This actually leads to another important question I want to discuss with
> the community, which is the proper size for the key/value pair in the
> blobkv. Our current thinking is that we could design the k-v size as the
> same with the blob size so that the conversion from k-v to blob is minimum.
> I’m wondering if this is a reasonable choice?
>
>
>
> Q2: Could you also put this in SPDK trello: https://trello.com/spdk?
>
> A2: I have a trello account (simple_hlw(a)163.com), and I am not a member
> in SPDK trello. Do I need to be a member before I have permissions to put
> this in SPDK trello?
>
>
>
> Regards,
>
> Helloway
>
>
>
>
>
>
>
> 在 2017年9月1日，上午7:52，Yang, Ziye <ziye.yang(a)intel.com> 写道：
>
>
>
> Hi We We,
>
>
>
> Could you also put this in SPDK trello: https:// <https://trello.com/spdk>
> trello.com/spdk ?
>
>
>
> Thanks.
>
>
>
>
>
> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org <spdk-bounces(a)lists.01.org>
> ] *On Behalf Of *We We
> *Sent:* Friday, September 1, 2017 12:04 AM
> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
> *Subject:* Re: [SPDK] SPDK Blobstore support object store?
>
>
>
> Hi all,
>
> Thank you for your answers.
>
> I have submitted the proposal about the blobkv (blobstore object) design
> ( https://github.com/spdk/spdk/pull/188/files?short_path=420ca87#diff-
> 420ca87f40f0f8170bb68bc5c742b6dc ), please visit it.
>
>
> Regards,
> Helloway
>
>
>
> 在 2017年8月31日，下午8:16，Yang, Ziye <ziye.yang(a)intel.com> 写道：
>
>
>
> Hi,
>
>
>
> Currently, SPDK can be integrated with Ceph in the following two ways:
>
>
>
> 1 SPDK has rbd bdev, thus you can have SPDK iSCSI target which
> uses the exported rbd device by Ceph.
>
> 2 SPDK can be integrated into bluestore in Ceph, the code is
> NVMEDEVICE.CC/h, located in src/os/bluestore/ folder. However SPDK is
> not enabled in Ceph. You need to build with WITH_SPDK=on option. And these
> days, we are doing some work to make SPDK can be compiled default in Ceph.
>
>
>
> Also in SPDK for object store support, we do not have a detailed plan now.
>
>
>
> Best Regards,
>
> Ziye Yang
>
>
>
> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org <spdk-bounces(a)lists.01.org>
> ] *On Behalf Of *We We
> *Sent:* Monday, August 28, 2017 9:17 PM
> *To:* spdk(a)lists.01.org
> *Subject:* [SPDK] SPDK Blobstore support object store?
>
>
>
> Hi，
>
> In the source code of SPDk (https://github.com/spdk/spdk/tree/master/lib),
> we can find *spdk* <https://github.com/spdk/spdk>/lib
> <https://github.com/spdk/spdk/tree/master/lib>/bdev
> <https://github.com/spdk/spdk/tree/master/lib/bdev>/ module that means SPDk
> is able to be in favor of block store and accelerate ceph block store.
> However, I don not see anything about object store. Does SPDK support
> object store? Is there any plan to do with object store?
>
>
>
>
> Could someone can give me the answer?
>
>
>
> Regards,
>
> Helloway
>
>
>
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 41486 bytes --]
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [SPDK] SPDK Blobstore support object store?
@ 2017-09-05 13:20 Luse, Paul E
0 siblings, 0 replies; 21+ messages in thread
From: Luse, Paul E @ 2017-09-05 13:20 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 13871 bytes --]
Sounds good, thanks! One other thing to think about while updating the proposal – a comparison over what is being proposed vs what blobstore does today would be useful too I think. For example, what value would this new module have over blobstore as opposed to potentially making a few changes to blobstore itself? Just another thought for the discussion…
Thx
Paul
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Zhipeng Huang
Sent: Monday, September 4, 2017 9:16 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: Re: [SPDK] SPDK Blobstore support object store?
Thanks Paul, I understand that we need a specific use case / scenario to jump start the work so that we could know if our design actually works :) I think we could start with Swift for the front end side and iterate on that.
So I think helloway could update her proposal with all the feedbacks so far and we continue discussion ?
On Sep 5, 2017 9:59 AM, "Luse, Paul E" <paul.e.luse(a)intel.com<mailto:paul.e.luse(a)intel.com>> wrote:
Hi Howard,
Yup, I understand. My points are talking about the back end, the point at which the storage application interfaces with the actual storage. In some cases, it’s direct, in some cases it’s via a filesystem and in some cases it a database or a combination of all of these things (with or without caching). It’s also not the case that objects are always sliced up, in some cases they are and when they are they may or may not be erasure coded and in the cases they’re not they are likely replicated. There are lots of variations is the point; even the very, very back end isn’t the same between object different storage systems ☺
Anyway, 1MB does sound like a good general number but again without some targeted usage I think you’ll find it challenging to get very many folks to work on it with you.
Thx
Paul
From: SPDK [mailto:spdk-bounces(a)lists.01.org<mailto:spdk-bounces(a)lists.01.org>] On Behalf Of Zhipeng Huang
Sent: Monday, September 4, 2017 5:55 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] SPDK Blobstore support object store?
Hi Paul,
This is Howard from Helloway's team. One of the reason that we don't want to target a specific object front-end (S3、Swift or Ceph RGW) is that (1)it is difficult to harmonize across these solutions and (2) we view spdk more of a back end role for these APIs
For most of the object storage services, they provide the functionalities of slicing up k/V pair they received to fit the back end storage. For example S3 would consume a 4M object but slice it up to a smaller size when it actually stores these objects, if I understand correctly. It will be the same case for Ceph or Swift.
Therefore we envisioned that blobkv provides the smaller sized object api that acts as the back end for S3、Swift or Ceph. I think 1M would be a reasonable kv size for blobkv ：)
On Sep 5, 2017 1:31 AM, "Luse, Paul E" <paul.e.luse(a)intel.com<mailto:paul.e.luse(a)intel.com>> wrote:
Hi Helloway,
OK, it just seems that w/o a target application (obj storage system or DB) some design decisions may get made that will make it less efficient than it could be to “fit” into at least one system really well – there are no standards for back end storage nodes for object storage systems so a “one size fits all” is highly unlikely (arguably impossible). Anyways, it’s certainly not a pre-requisite for you to work on it by any means, however it may impact how the maintainers look at your work as a possible contribution. Are you planning on designing something for contribution to the community? If so, some more discussion in this area is warranted for sure as everyone will want to make sure that whatever lands in the repo is not only robust/complete but readily usable as well.
Anyway wrt size, it seems like a reasonable starting approach however object storage can be a little tricky when it comes to object size as really anything goes. It’s more function of how the object storage system is being used as it is anything else. Most people are familiar with large objects being typical but that’s certainly not the case all of the time. There are many uses of Swift, for example, that have sub MB object sizes and choose an object storage system because of its API (REST, etc). With blobstore the page size is 4K and the default cluster size, which is the minimum size for a blob, is 1MB. The latter is configurable but not the former so if you had in mind objects of size 1MB or larger then, yeah, thinking of an object as a blob in terms of size makes sense.
There are many more considerations as well that I’m sure you’re aware of, curious as to what your thoughts were in some other areas like (just brainstorming here):
- Metadata storage support. Blobstore has a pretty minimal metadata (xattr) capability at least as compared to some of the common obj storage systems out there. A mapping of common capabilities here versus what blobstore does natively now would be a good exercise as closing the gaps could involve a decent amount of work in the new module and/or in blobstore itself. Most systems make some use of FS xattrs in combination with other non-FS mechanisms (KV DB) to store various types of metadata
- Metadata search: same kind of thoughts as above but keeping in mind that the search capability of the storage node could be severity limited if the actual back end storage didn’t have a pretty robust mechanism for caching and/or retrieving object metadata (blobstore does not)
- Support for containers: Containers in terms of how Swift uses them, basically S3 buckets. One could easily consider this out of scope if the idea is that this new thing is only targeted at objects themselves and not any sort of storage system metadata needs. I think that’s probably reasonable but I haven’t thought it all the way through
- Permissions requirements & enforcement
I’m sure we could come up a bunch more too and, again, having a target system to bolt into would help flesh out even more and drive a more robust design I think.
Thanks!!
Paul
From: SPDK [mailto:spdk-bounces(a)lists.01.org<mailto:spdk-bounces(a)lists.01.org>] On Behalf Of We We
Sent: Sunday, September 3, 2017 9:10 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] SPDK Blobstore support object store?
Hi Paul,
Thank you for adding me to the Trello board.
About these questions you mentioned, here are my thoughts:
Q1: Have you looked into Swift as a candidate for this activity or were you just using that as an example of a popular project?
A1: The blobkv provides the generic k-v interface module aimed to obj storage systems ( Such as: Swift, S3 ), so Swift and S3 are just used as examples of a popular project.
Q2: it’s probably the case that we’d want to target a specific object storage system and start with an investigation over how feasible it would be to bolt in SPDK at a high level.
A2: As the first answer said, the blobkv doesn’t target a specific object storage system. Our current thinking is that the blobkv is a standard k-v interface which located at the back end of S3, Swift, and so on.
For the reason above, we want to discuss with community about which is the proper size for the key/value pair in the blobkv? Could we design the k-v size as the same with the blob size so that the conversion from k-v to blob is minimum? I’m wondering if it is reasonable?
Regards,
Helloway
在 2017年9月1日，下午9:14，Luse, Paul E <paul.e.luse(a)intel.com<mailto:paul.e.luse(a)intel.com>> 写道：
Hi Helloway,
I just added you to the Trello board. Wrt your answer about API, yes that makes sense. Also, SPDK as it is today would need a tremendous amount of work to support an application level object storage API ☺
So with S3 though, that’s kind of up to Amazon. For Swift, although it’s an open source project, there are some challenges there as well. Not that they’re not solvable of course, have you looked into Swift as a candidate for this activity or were you just using that as an example of a popular project?
Given the first point above, it’s probably the case that we’d want to target a specific object storage system and start with an investigation over how feasible it would be to bolt in SPDK at a high level and, depending on the project, engage with either that community or a company with a significant interest (installation) in making this happen don’t you think? Don’t get me wrong, I think it’s a great idea I’m just trying to help talk out the right approach. Either via some light abstraction layer or something we’d definitely want to identify a target system and see how interesting it might be to folks before we start any kind of detailed design discussions I think.
Thanks!
Paul
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Yang, Ziye
Sent: Friday, September 1, 2017 12:37 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] SPDK Blobstore support object store?
Hi We We,
For the membership, could you add by yourself. If you cannot, I think that Jim and Daniel can help you.
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Friday, September 1, 2017 2:20 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] SPDK Blobstore support object store?
Hi, @Paul @ Ziye
Thank you for all of your reply.
About these questions you mentioned, here are my thoughts:
Q1: Why blobkv doesn't tying into existing applications that support something like S3 or native Swift?
A1: Swift and S3 are different with blobkv at the hierarchy. Swift and S3 provide the application level k-v. However, the blobkv provides processing of generic k-v as a back-end behind Swift and S3，rather than the application level k-v.
This actually leads to another important question I want to discuss with the community, which is the proper size for the key/value pair in the blobkv. Our current thinking is that we could design the k-v size as the same with the blob size so that the conversion from k-v to blob is minimum. I’m wondering if this is a reasonable choice?
Q2: Could you also put this in SPDK trello: https://trello.com/spdk?
A2: I have a trello account (simple_hlw(a)163.com<mailto:simple_hlw(a)163.com>), and I am not a member in SPDK trello. Do I need to be a member before I have permissions to put this in SPDK trello?
Regards,
Helloway
在 2017年9月1日，上午7:52，Yang, Ziye <ziye.yang(a)intel.com<mailto:ziye.yang(a)intel.com>> 写道：
Hi We We,
Could you also put this in SPDK trello: https://<https://trello.com/spdk>trello.com/spdk<https://trello.com/spdk> ?
Thanks.
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Friday, September 1, 2017 12:04 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] SPDK Blobstore support object store?
Hi all,
Thank you for your answers.
I have submitted the proposal about the blobkv (blobstore object) design ( https://github.com/spdk/spdk/pull/188/files?short_path=420ca87#diff-420ca87f40f0f8170bb68bc5c742b6dc ), please visit it.
Regards,
Helloway
在 2017年8月31日，下午8:16，Yang, Ziye <ziye.yang(a)intel.com<mailto:ziye.yang(a)intel.com>> 写道：
Hi,
Currently, SPDK can be integrated with Ceph in the following two ways:
1 SPDK has rbd bdev, thus you can have SPDK iSCSI target which uses the exported rbd device by Ceph.
2 SPDK can be integrated into bluestore in Ceph, the code is NVMEDEVICE.CC/h<http://NVMEDEVICE.CC/h>, located in src/os/bluestore/ folder. However SPDK is not enabled in Ceph. You need to build with WITH_SPDK=on option. And these days, we are doing some work to make SPDK can be compiled default in Ceph.
Also in SPDK for object store support, we do not have a detailed plan now.
Best Regards,
Ziye Yang
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Monday, August 28, 2017 9:17 PM
To: spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>
Subject: [SPDK] SPDK Blobstore support object store?
Hi，
In the source code of SPDk (https://github.com/spdk/spdk/tree/master/lib), we can find spdk<https://github.com/spdk/spdk>/lib<https://github.com/spdk/spdk/tree/master/lib>/bdev<https://github.com/spdk/spdk/tree/master/lib/bdev>/ module that means SPDk is able to be in favor of block store and accelerate ceph block store. However, I don not see anything about object store. Does SPDK support object store? Is there any plan to do with object store?
Could someone can give me the answer?
Regards,
Helloway
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 55874 bytes --]
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [SPDK] SPDK Blobstore support object store?
@ 2017-09-05 4:16 Zhipeng Huang
0 siblings, 0 replies; 21+ messages in thread
From: Zhipeng Huang @ 2017-09-05 4:16 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 13871 bytes --]
Thanks Paul, I understand that we need a specific use case / scenario to
jump start the work so that we could know if our design actually works :) I
think we could start with Swift for the front end side and iterate on that.
So I think helloway could update her proposal with all the feedbacks so far
and we continue discussion ?
On Sep 5, 2017 9:59 AM, "Luse, Paul E" <paul.e.luse(a)intel.com> wrote:
> Hi Howard,
>
>
>
> Yup, I understand. My points are talking about the back end, the point at
> which the storage application interfaces with the actual storage. In some
> cases, it’s direct, in some cases it’s via a filesystem and in some cases
> it a database or a combination of all of these things (with or without
> caching). It’s also not the case that objects are always sliced up, in some
> cases they are and when they are they may or may not be erasure coded and
> in the cases they’re not they are likely replicated. There are lots of
> variations is the point; even the very, very back end isn’t the same
> between object different storage systems J
>
>
>
> Anyway, 1MB does sound like a good general number but again without some
> targeted usage I think you’ll find it challenging to get very many folks to
> work on it with you.
>
>
>
> Thx
>
> Paul
>
>
>
> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org] *On Behalf Of *Zhipeng
> Huang
> *Sent:* Monday, September 4, 2017 5:55 PM
> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
> *Subject:* Re: [SPDK] SPDK Blobstore support object store?
>
>
>
> Hi Paul,
>
>
>
> This is Howard from Helloway's team. One of the reason that we don't want
> to target a specific object front-end (S3、Swift or Ceph RGW) is that
> (1)it is difficult to harmonize across these solutions and (2) we view spdk
> more of a back end role for these APIs
>
>
>
> For most of the object storage services, they provide the functionalities
> of slicing up k/V pair they received to fit the back end storage. For
> example S3 would consume a 4M object but slice it up to a smaller size when
> it actually stores these objects, if I understand correctly. It will be the
> same case for Ceph or Swift.
>
>
>
> Therefore we envisioned that blobkv provides the smaller sized object api
> that acts as the back end for S3、Swift or Ceph. I think 1M would be a
> reasonable kv size for blobkv ：)
>
>
>
> On Sep 5, 2017 1:31 AM, "Luse, Paul E" <paul.e.luse(a)intel.com> wrote:
>
> Hi Helloway,
>
>
>
> OK, it just seems that w/o a target application (obj storage system or DB)
> some design decisions may get made that will make it less efficient than it
> could be to “fit” into at least one system really well – there are no
> standards for back end storage nodes for object storage systems so a “one
> size fits all” is highly unlikely (arguably impossible). Anyways, it’s
> certainly not a pre-requisite for you to work on it by any means, however
> it may impact how the maintainers look at your work as a possible
> contribution. *Are you planning on designing something for contribution
> to the community?* If so, some more discussion in this area is warranted
> for sure as everyone will want to make sure that whatever lands in the repo
> is not only robust/complete but readily usable as well.
>
>
>
> Anyway wrt size, it seems like a reasonable starting approach however
> object storage can be a little tricky when it comes to object size as
> really anything goes. It’s more function of how the object storage system
> is being used as it is anything else. Most people are familiar with large
> objects being typical but that’s certainly not the case all of the time.
> There are many uses of Swift, for example, that have sub MB object sizes
> and choose an object storage system because of its API (REST, etc). With
> blobstore the page size is 4K and the default cluster size, which is the
> minimum size for a blob, is 1MB. The latter is configurable but not the
> former so if you had in mind objects of size 1MB or larger then, yeah,
> thinking of an object as a blob in terms of size makes sense.
>
>
>
> There are many more considerations as well that I’m sure you’re aware of,
> curious as to what your thoughts were in some other areas like (just
> brainstorming here):
>
>
>
> - Metadata storage support. Blobstore has a pretty minimal
> metadata (xattr) capability at least as compared to some of the common obj
> storage systems out there. A mapping of common capabilities here versus
> what blobstore does natively now would be a good exercise as closing the
> gaps could involve a decent amount of work in the new module and/or in
> blobstore itself. Most systems make some use of FS xattrs in combination
> with other non-FS mechanisms (KV DB) to store various types of metadata
>
> - Metadata search: same kind of thoughts as above but keeping in
> mind that the search capability of the storage node could be severity
> limited if the actual back end storage didn’t have a pretty robust
> mechanism for caching and/or retrieving object metadata (blobstore does not)
>
> - Support for containers: Containers in terms of how Swift uses
> them, basically S3 buckets. One could easily consider this out of scope if
> the idea is that this new thing is only targeted at objects themselves and
> not any sort of storage system metadata needs. I think that’s probably
> reasonable but I haven’t thought it all the way through
>
> - Permissions requirements & enforcement
>
>
>
> I’m sure we could come up a bunch more too and, again, having a target
> system to bolt into would help flesh out even more and drive a more robust
> design I think.
>
>
>
> Thanks!!
>
> Paul
>
>
>
> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org] *On Behalf Of *We We
> *Sent:* Sunday, September 3, 2017 9:10 PM
> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
> *Subject:* Re: [SPDK] SPDK Blobstore support object store?
>
>
>
> Hi Paul,
>
> Thank you for adding me to the Trello board.
>
> About these questions you mentioned, here are my thoughts:
>
> Q1: Have you looked into Swift as a candidate for this activity or were
> you just using that as an example of a popular project?
>
> A1: The blobkv provides the generic k-v interface module aimed to obj
> storage systems ( Such as: Swift, S3 ), so Swift and S3 are just used as
> examples of a popular project.
>
>
>
> Q2: it’s probably the case that we’d want to target a specific object
> storage system and start with an investigation over how feasible it would
> be to bolt in SPDK at a high level.
>
> A2: As the first answer said, the blobkv doesn’t target a specific object
> storage system. Our current thinking is that the blobkv is a standard k-v
> interface which located at the back end of S3, Swift, and so on.
>
>
>
> For the reason above, we want to discuss with community about which is the
> proper size for the key/value pair in the blobkv? Could we design the k-v
> size as the same with the blob size so that the conversion from k-v to blob
> is minimum? I’m wondering if it is reasonable?
>
>
>
> Regards,
>
> Helloway
>
> 在 2017年9月1日，下午9:14，Luse, Paul E <paul.e.luse(a)intel.com> 写道：
>
>
>
> Hi Helloway,
>
>
>
> I just added you to the Trello board. Wrt your answer about API, yes that
> makes sense. Also, SPDK as it is today would need a tremendous amount of
> work to support an application level object storage API J
>
>
>
> So with S3 though, that’s kind of up to Amazon. For Swift, although it’s
> an open source project, there are some challenges there as well. Not that
> they’re not solvable of course, have you looked into Swift as a candidate
> for this activity or were you just using that as an example of a popular
> project?
>
>
>
> Given the first point above, it’s probably the case that we’d want to
> target a specific object storage system and start with an investigation
> over how feasible it would be to bolt in SPDK at a high level and,
> depending on the project, engage with either that community or a company
> with a significant interest (installation) in making this happen don’t you
> think? Don’t get me wrong, I think it’s a great idea I’m just trying to
> help talk out the right approach. Either via some light abstraction layer
> or something we’d definitely want to identify a target system and see how
> interesting it might be to folks before we start any kind of detailed
> design discussions I think.
>
>
>
> Thanks!
>
> Paul
>
>
>
> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org <spdk-bounces(a)lists.01.org>
> ] *On Behalf Of *Yang, Ziye
> *Sent:* Friday, September 1, 2017 12:37 AM
> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
> *Subject:* Re: [SPDK] SPDK Blobstore support object store?
>
>
>
> Hi We We,
>
>
>
> For the membership, could you add by yourself. If you cannot, I think
> that Jim and Daniel can help you.
>
>
>
> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org <spdk-bounces(a)lists.01.org>
> ] *On Behalf Of *We We
> *Sent:* Friday, September 1, 2017 2:20 PM
> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
> *Subject:* Re: [SPDK] SPDK Blobstore support object store?
>
>
>
> Hi, @Paul @ Ziye
>
> Thank you for all of your reply.
>
> About these questions you mentioned, here are my thoughts:
>
> Q1: Why blobkv doesn't tying into existing applications that support
> something like S3 or native Swift?
>
> A1: Swift and S3 are different with blobkv at the hierarchy. Swift and S3
> provide the application level k-v. However, the blobkv provides processing
> of generic k-v as a back-end behind Swift and S3，rather than the
> application level k-v.
>
>
>
> This actually leads to another important question I want to discuss with
> the community, which is the proper size for the key/value pair in the
> blobkv. Our current thinking is that we could design the k-v size as the
> same with the blob size so that the conversion from k-v to blob is minimum.
> I’m wondering if this is a reasonable choice?
>
>
>
> Q2: Could you also put this in SPDK trello: https://trello.com/spdk?
>
> A2: I have a trello account (simple_hlw(a)163.com), and I am not a member
> in SPDK trello. Do I need to be a member before I have permissions to put
> this in SPDK trello?
>
>
>
> Regards,
>
> Helloway
>
>
>
>
>
>
>
> 在 2017年9月1日，上午7:52，Yang, Ziye <ziye.yang(a)intel.com> 写道：
>
>
>
> Hi We We,
>
>
>
> Could you also put this in SPDK trello: https:// <https://trello.com/spdk>
> trello.com/spdk ?
>
>
>
> Thanks.
>
>
>
>
>
> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org <spdk-bounces(a)lists.01.org>
> ] *On Behalf Of *We We
> *Sent:* Friday, September 1, 2017 12:04 AM
> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
> *Subject:* Re: [SPDK] SPDK Blobstore support object store?
>
>
>
> Hi all,
>
> Thank you for your answers.
>
> I have submitted the proposal about the blobkv (blobstore object) design
> ( https://github.com/spdk/spdk/pull/188/files?short_
> path=420ca87#diff-420ca87f40f0f8170bb68bc5c742b6dc ), please visit it.
>
>
> Regards,
> Helloway
>
>
>
> 在 2017年8月31日，下午8:16，Yang, Ziye <ziye.yang(a)intel.com> 写道：
>
>
>
> Hi,
>
>
>
> Currently, SPDK can be integrated with Ceph in the following two ways:
>
>
>
> 1 SPDK has rbd bdev, thus you can have SPDK iSCSI target which
> uses the exported rbd device by Ceph.
>
> 2 SPDK can be integrated into bluestore in Ceph, the code is
> NVMEDEVICE.CC/h, located in src/os/bluestore/ folder. However SPDK is
> not enabled in Ceph. You need to build with WITH_SPDK=on option. And these
> days, we are doing some work to make SPDK can be compiled default in Ceph.
>
>
>
> Also in SPDK for object store support, we do not have a detailed plan now.
>
>
>
> Best Regards,
>
> Ziye Yang
>
>
>
> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org <spdk-bounces(a)lists.01.org>
> ] *On Behalf Of *We We
> *Sent:* Monday, August 28, 2017 9:17 PM
> *To:* spdk(a)lists.01.org
> *Subject:* [SPDK] SPDK Blobstore support object store?
>
>
>
> Hi，
>
> In the source code of SPDk (https://github.com/spdk/spdk/tree/master/lib),
> we can find *spdk* <https://github.com/spdk/spdk>/lib
> <https://github.com/spdk/spdk/tree/master/lib>/bdev
> <https://github.com/spdk/spdk/tree/master/lib/bdev>/ module that means SPDk
> is able to be in favor of block store and accelerate ceph block store.
> However, I don not see anything about object store. Does SPDK support
> object store? Is there any plan to do with object store?
>
>
>
>
> Could someone can give me the answer?
>
>
>
> Regards,
>
> Helloway
>
>
>
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 42540 bytes --]
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [SPDK] SPDK Blobstore support object store?
@ 2017-09-05 1:59 Luse, Paul E
0 siblings, 0 replies; 21+ messages in thread
From: Luse, Paul E @ 2017-09-05 1:59 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 12589 bytes --]
Hi Howard,
Yup, I understand. My points are talking about the back end, the point at which the storage application interfaces with the actual storage. In some cases, it’s direct, in some cases it’s via a filesystem and in some cases it a database or a combination of all of these things (with or without caching). It’s also not the case that objects are always sliced up, in some cases they are and when they are they may or may not be erasure coded and in the cases they’re not they are likely replicated. There are lots of variations is the point; even the very, very back end isn’t the same between object different storage systems ☺
Anyway, 1MB does sound like a good general number but again without some targeted usage I think you’ll find it challenging to get very many folks to work on it with you.
Thx
Paul
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Zhipeng Huang
Sent: Monday, September 4, 2017 5:55 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: Re: [SPDK] SPDK Blobstore support object store?
Hi Paul,
This is Howard from Helloway's team. One of the reason that we don't want to target a specific object front-end (S3、Swift or Ceph RGW) is that (1)it is difficult to harmonize across these solutions and (2) we view spdk more of a back end role for these APIs
For most of the object storage services, they provide the functionalities of slicing up k/V pair they received to fit the back end storage. For example S3 would consume a 4M object but slice it up to a smaller size when it actually stores these objects, if I understand correctly. It will be the same case for Ceph or Swift.
Therefore we envisioned that blobkv provides the smaller sized object api that acts as the back end for S3、Swift or Ceph. I think 1M would be a reasonable kv size for blobkv ：)
On Sep 5, 2017 1:31 AM, "Luse, Paul E" <paul.e.luse(a)intel.com<mailto:paul.e.luse(a)intel.com>> wrote:
Hi Helloway,
OK, it just seems that w/o a target application (obj storage system or DB) some design decisions may get made that will make it less efficient than it could be to “fit” into at least one system really well – there are no standards for back end storage nodes for object storage systems so a “one size fits all” is highly unlikely (arguably impossible). Anyways, it’s certainly not a pre-requisite for you to work on it by any means, however it may impact how the maintainers look at your work as a possible contribution. Are you planning on designing something for contribution to the community? If so, some more discussion in this area is warranted for sure as everyone will want to make sure that whatever lands in the repo is not only robust/complete but readily usable as well.
Anyway wrt size, it seems like a reasonable starting approach however object storage can be a little tricky when it comes to object size as really anything goes. It’s more function of how the object storage system is being used as it is anything else. Most people are familiar with large objects being typical but that’s certainly not the case all of the time. There are many uses of Swift, for example, that have sub MB object sizes and choose an object storage system because of its API (REST, etc). With blobstore the page size is 4K and the default cluster size, which is the minimum size for a blob, is 1MB. The latter is configurable but not the former so if you had in mind objects of size 1MB or larger then, yeah, thinking of an object as a blob in terms of size makes sense.
There are many more considerations as well that I’m sure you’re aware of, curious as to what your thoughts were in some other areas like (just brainstorming here):
- Metadata storage support. Blobstore has a pretty minimal metadata (xattr) capability at least as compared to some of the common obj storage systems out there. A mapping of common capabilities here versus what blobstore does natively now would be a good exercise as closing the gaps could involve a decent amount of work in the new module and/or in blobstore itself. Most systems make some use of FS xattrs in combination with other non-FS mechanisms (KV DB) to store various types of metadata
- Metadata search: same kind of thoughts as above but keeping in mind that the search capability of the storage node could be severity limited if the actual back end storage didn’t have a pretty robust mechanism for caching and/or retrieving object metadata (blobstore does not)
- Support for containers: Containers in terms of how Swift uses them, basically S3 buckets. One could easily consider this out of scope if the idea is that this new thing is only targeted at objects themselves and not any sort of storage system metadata needs. I think that’s probably reasonable but I haven’t thought it all the way through
- Permissions requirements & enforcement
I’m sure we could come up a bunch more too and, again, having a target system to bolt into would help flesh out even more and drive a more robust design I think.
Thanks!!
Paul
From: SPDK [mailto:spdk-bounces(a)lists.01.org<mailto:spdk-bounces(a)lists.01.org>] On Behalf Of We We
Sent: Sunday, September 3, 2017 9:10 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] SPDK Blobstore support object store?
Hi Paul,
Thank you for adding me to the Trello board.
About these questions you mentioned, here are my thoughts:
Q1: Have you looked into Swift as a candidate for this activity or were you just using that as an example of a popular project?
A1: The blobkv provides the generic k-v interface module aimed to obj storage systems ( Such as: Swift, S3 ), so Swift and S3 are just used as examples of a popular project.
Q2: it’s probably the case that we’d want to target a specific object storage system and start with an investigation over how feasible it would be to bolt in SPDK at a high level.
A2: As the first answer said, the blobkv doesn’t target a specific object storage system. Our current thinking is that the blobkv is a standard k-v interface which located at the back end of S3, Swift, and so on.
For the reason above, we want to discuss with community about which is the proper size for the key/value pair in the blobkv? Could we design the k-v size as the same with the blob size so that the conversion from k-v to blob is minimum? I’m wondering if it is reasonable?
Regards,
Helloway
在 2017年9月1日，下午9:14，Luse, Paul E <paul.e.luse(a)intel.com<mailto:paul.e.luse(a)intel.com>> 写道：
Hi Helloway,
I just added you to the Trello board. Wrt your answer about API, yes that makes sense. Also, SPDK as it is today would need a tremendous amount of work to support an application level object storage API ☺
So with S3 though, that’s kind of up to Amazon. For Swift, although it’s an open source project, there are some challenges there as well. Not that they’re not solvable of course, have you looked into Swift as a candidate for this activity or were you just using that as an example of a popular project?
Given the first point above, it’s probably the case that we’d want to target a specific object storage system and start with an investigation over how feasible it would be to bolt in SPDK at a high level and, depending on the project, engage with either that community or a company with a significant interest (installation) in making this happen don’t you think? Don’t get me wrong, I think it’s a great idea I’m just trying to help talk out the right approach. Either via some light abstraction layer or something we’d definitely want to identify a target system and see how interesting it might be to folks before we start any kind of detailed design discussions I think.
Thanks!
Paul
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Yang, Ziye
Sent: Friday, September 1, 2017 12:37 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] SPDK Blobstore support object store?
Hi We We,
For the membership, could you add by yourself. If you cannot, I think that Jim and Daniel can help you.
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Friday, September 1, 2017 2:20 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] SPDK Blobstore support object store?
Hi, @Paul @ Ziye
Thank you for all of your reply.
About these questions you mentioned, here are my thoughts:
Q1: Why blobkv doesn't tying into existing applications that support something like S3 or native Swift?
A1: Swift and S3 are different with blobkv at the hierarchy. Swift and S3 provide the application level k-v. However, the blobkv provides processing of generic k-v as a back-end behind Swift and S3，rather than the application level k-v.
This actually leads to another important question I want to discuss with the community, which is the proper size for the key/value pair in the blobkv. Our current thinking is that we could design the k-v size as the same with the blob size so that the conversion from k-v to blob is minimum. I’m wondering if this is a reasonable choice?
Q2: Could you also put this in SPDK trello: https://trello.com/spdk?
A2: I have a trello account (simple_hlw(a)163.com<mailto:simple_hlw(a)163.com>), and I am not a member in SPDK trello. Do I need to be a member before I have permissions to put this in SPDK trello?
Regards,
Helloway
在 2017年9月1日，上午7:52，Yang, Ziye <ziye.yang(a)intel.com<mailto:ziye.yang(a)intel.com>> 写道：
Hi We We,
Could you also put this in SPDK trello: https://<https://trello.com/spdk>trello.com/spdk<https://trello.com/spdk> ?
Thanks.
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Friday, September 1, 2017 12:04 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] SPDK Blobstore support object store?
Hi all,
Thank you for your answers.
I have submitted the proposal about the blobkv (blobstore object) design ( https://github.com/spdk/spdk/pull/188/files?short_path=420ca87#diff-420ca87f40f0f8170bb68bc5c742b6dc ), please visit it.
Regards,
Helloway
在 2017年8月31日，下午8:16，Yang, Ziye <ziye.yang(a)intel.com<mailto:ziye.yang(a)intel.com>> 写道：
Hi,
Currently, SPDK can be integrated with Ceph in the following two ways:
1 SPDK has rbd bdev, thus you can have SPDK iSCSI target which uses the exported rbd device by Ceph.
2 SPDK can be integrated into bluestore in Ceph, the code is NVMEDEVICE.CC/h<http://NVMEDEVICE.CC/h>, located in src/os/bluestore/ folder. However SPDK is not enabled in Ceph. You need to build with WITH_SPDK=on option. And these days, we are doing some work to make SPDK can be compiled default in Ceph.
Also in SPDK for object store support, we do not have a detailed plan now.
Best Regards,
Ziye Yang
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Monday, August 28, 2017 9:17 PM
To: spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>
Subject: [SPDK] SPDK Blobstore support object store?
Hi，
In the source code of SPDk (https://github.com/spdk/spdk/tree/master/lib), we can find spdk<https://github.com/spdk/spdk>/lib<https://github.com/spdk/spdk/tree/master/lib>/bdev<https://github.com/spdk/spdk/tree/master/lib/bdev>/ module that means SPDk is able to be in favor of block store and accelerate ceph block store. However, I don not see anything about object store. Does SPDK support object store? Is there any plan to do with object store?
Could someone can give me the answer?
Regards,
Helloway
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 49852 bytes --]
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [SPDK] SPDK Blobstore support object store?
@ 2017-09-05 0:55 Zhipeng Huang
0 siblings, 0 replies; 21+ messages in thread
From: Zhipeng Huang @ 2017-09-05 0:55 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 12073 bytes --]
Hi Paul,
This is Howard from Helloway's team. One of the reason that we don't want
to target a specific object front-end (S3、Swift or Ceph RGW) is that (1)it
is difficult to harmonize across these solutions and (2) we view spdk more
of a back end role for these APIs
For most of the object storage services, they provide the functionalities
of slicing up k/V pair they received to fit the back end storage. For
example S3 would consume a 4M object but slice it up to a smaller size when
it actually stores these objects, if I understand correctly. It will be the
same case for Ceph or Swift.
Therefore we envisioned that blobkv provides the smaller sized object api
that acts as the back end for S3、Swift or Ceph. I think 1M would be a
reasonable kv size for blobkv ：)
On Sep 5, 2017 1:31 AM, "Luse, Paul E" <paul.e.luse(a)intel.com> wrote:
> Hi Helloway,
>
>
>
> OK, it just seems that w/o a target application (obj storage system or DB)
> some design decisions may get made that will make it less efficient than it
> could be to “fit” into at least one system really well – there are no
> standards for back end storage nodes for object storage systems so a “one
> size fits all” is highly unlikely (arguably impossible). Anyways, it’s
> certainly not a pre-requisite for you to work on it by any means, however
> it may impact how the maintainers look at your work as a possible
> contribution. *Are you planning on designing something for contribution
> to the community?* If so, some more discussion in this area is warranted
> for sure as everyone will want to make sure that whatever lands in the repo
> is not only robust/complete but readily usable as well.
>
>
>
> Anyway wrt size, it seems like a reasonable starting approach however
> object storage can be a little tricky when it comes to object size as
> really anything goes. It’s more function of how the object storage system
> is being used as it is anything else. Most people are familiar with large
> objects being typical but that’s certainly not the case all of the time.
> There are many uses of Swift, for example, that have sub MB object sizes
> and choose an object storage system because of its API (REST, etc). With
> blobstore the page size is 4K and the default cluster size, which is the
> minimum size for a blob, is 1MB. The latter is configurable but not the
> former so if you had in mind objects of size 1MB or larger then, yeah,
> thinking of an object as a blob in terms of size makes sense.
>
>
>
> There are many more considerations as well that I’m sure you’re aware of,
> curious as to what your thoughts were in some other areas like (just
> brainstorming here):
>
>
>
> - Metadata storage support. Blobstore has a pretty minimal
> metadata (xattr) capability at least as compared to some of the common obj
> storage systems out there. A mapping of common capabilities here versus
> what blobstore does natively now would be a good exercise as closing the
> gaps could involve a decent amount of work in the new module and/or in
> blobstore itself. Most systems make some use of FS xattrs in combination
> with other non-FS mechanisms (KV DB) to store various types of metadata
>
> - Metadata search: same kind of thoughts as above but keeping in
> mind that the search capability of the storage node could be severity
> limited if the actual back end storage didn’t have a pretty robust
> mechanism for caching and/or retrieving object metadata (blobstore does not)
>
> - Support for containers: Containers in terms of how Swift uses
> them, basically S3 buckets. One could easily consider this out of scope if
> the idea is that this new thing is only targeted at objects themselves and
> not any sort of storage system metadata needs. I think that’s probably
> reasonable but I haven’t thought it all the way through
>
> - Permissions requirements & enforcement
>
>
>
> I’m sure we could come up a bunch more too and, again, having a target
> system to bolt into would help flesh out even more and drive a more robust
> design I think.
>
>
>
> Thanks!!
>
> Paul
>
>
>
> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org] *On Behalf Of *We We
> *Sent:* Sunday, September 3, 2017 9:10 PM
> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
> *Subject:* Re: [SPDK] SPDK Blobstore support object store?
>
>
>
> Hi Paul,
>
> Thank you for adding me to the Trello board.
>
> About these questions you mentioned, here are my thoughts:
>
> Q1: Have you looked into Swift as a candidate for this activity or were
> you just using that as an example of a popular project?
>
> A1: The blobkv provides the generic k-v interface module aimed to obj
> storage systems ( Such as: Swift, S3 ), so Swift and S3 are just used as
> examples of a popular project.
>
>
>
> Q2: it’s probably the case that we’d want to target a specific object
> storage system and start with an investigation over how feasible it would
> be to bolt in SPDK at a high level.
>
> A2: As the first answer said, the blobkv doesn’t target a specific object
> storage system. Our current thinking is that the blobkv is a standard k-v
> interface which located at the back end of S3, Swift, and so on.
>
>
>
> For the reason above, we want to discuss with community about which is the
> proper size for the key/value pair in the blobkv? Could we design the k-v
> size as the same with the blob size so that the conversion from k-v to blob
> is minimum? I’m wondering if it is reasonable?
>
>
>
> Regards,
>
> Helloway
>
> 在 2017年9月1日，下午9:14，Luse, Paul E <paul.e.luse(a)intel.com> 写道：
>
>
>
> Hi Helloway,
>
>
>
> I just added you to the Trello board. Wrt your answer about API, yes that
> makes sense. Also, SPDK as it is today would need a tremendous amount of
> work to support an application level object storage API J
>
>
>
> So with S3 though, that’s kind of up to Amazon. For Swift, although it’s
> an open source project, there are some challenges there as well. Not that
> they’re not solvable of course, have you looked into Swift as a candidate
> for this activity or were you just using that as an example of a popular
> project?
>
>
>
> Given the first point above, it’s probably the case that we’d want to
> target a specific object storage system and start with an investigation
> over how feasible it would be to bolt in SPDK at a high level and,
> depending on the project, engage with either that community or a company
> with a significant interest (installation) in making this happen don’t you
> think? Don’t get me wrong, I think it’s a great idea I’m just trying to
> help talk out the right approach. Either via some light abstraction layer
> or something we’d definitely want to identify a target system and see how
> interesting it might be to folks before we start any kind of detailed
> design discussions I think.
>
>
>
> Thanks!
>
> Paul
>
>
>
> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org <spdk-bounces(a)lists.01.org>
> ] *On Behalf Of *Yang, Ziye
> *Sent:* Friday, September 1, 2017 12:37 AM
> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
> *Subject:* Re: [SPDK] SPDK Blobstore support object store?
>
>
>
> Hi We We,
>
>
>
> For the membership, could you add by yourself. If you cannot, I think
> that Jim and Daniel can help you.
>
>
>
> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org <spdk-bounces(a)lists.01.org>
> ] *On Behalf Of *We We
> *Sent:* Friday, September 1, 2017 2:20 PM
> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
> *Subject:* Re: [SPDK] SPDK Blobstore support object store?
>
>
>
> Hi, @Paul @ Ziye
>
> Thank you for all of your reply.
>
> About these questions you mentioned, here are my thoughts:
>
> Q1: Why blobkv doesn't tying into existing applications that support
> something like S3 or native Swift?
>
> A1: Swift and S3 are different with blobkv at the hierarchy. Swift and S3
> provide the application level k-v. However, the blobkv provides processing
> of generic k-v as a back-end behind Swift and S3，rather than the
> application level k-v.
>
>
>
> This actually leads to another important question I want to discuss with
> the community, which is the proper size for the key/value pair in the
> blobkv. Our current thinking is that we could design the k-v size as the
> same with the blob size so that the conversion from k-v to blob is minimum.
> I’m wondering if this is a reasonable choice?
>
>
>
> Q2: Could you also put this in SPDK trello: https://trello.com/spdk?
>
> A2: I have a trello account (simple_hlw(a)163.com), and I am not a member
> in SPDK trello. Do I need to be a member before I have permissions to put
> this in SPDK trello?
>
>
>
> Regards,
>
> Helloway
>
>
>
>
>
>
>
> 在 2017年9月1日，上午7:52，Yang, Ziye <ziye.yang(a)intel.com> 写道：
>
>
>
> Hi We We,
>
>
>
> Could you also put this in SPDK trello: https:// <https://trello.com/spdk>
> trello.com/spdk ?
>
>
>
> Thanks.
>
>
>
>
>
> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org <spdk-bounces(a)lists.01.org>
> ] *On Behalf Of *We We
> *Sent:* Friday, September 1, 2017 12:04 AM
> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
> *Subject:* Re: [SPDK] SPDK Blobstore support object store?
>
>
>
> Hi all,
>
> Thank you for your answers.
>
> I have submitted the proposal about the blobkv (blobstore object) design
> ( https://github.com/spdk/spdk/pull/188/files?short_path=420ca87#diff-
> 420ca87f40f0f8170bb68bc5c742b6dc ), please visit it.
>
>
> Regards,
> Helloway
>
>
>
> 在 2017年8月31日，下午8:16，Yang, Ziye <ziye.yang(a)intel.com> 写道：
>
>
>
> Hi,
>
>
>
> Currently, SPDK can be integrated with Ceph in the following two ways:
>
>
>
> 1 SPDK has rbd bdev, thus you can have SPDK iSCSI target which
> uses the exported rbd device by Ceph.
>
> 2 SPDK can be integrated into bluestore in Ceph, the code is
> NVMEDEVICE.CC/h, located in src/os/bluestore/ folder. However SPDK is
> not enabled in Ceph. You need to build with WITH_SPDK=on option. And these
> days, we are doing some work to make SPDK can be compiled default in Ceph.
>
>
>
> Also in SPDK for object store support, we do not have a detailed plan now.
>
>
>
> Best Regards,
>
> Ziye Yang
>
>
>
> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org <spdk-bounces(a)lists.01.org>
> ] *On Behalf Of *We We
> *Sent:* Monday, August 28, 2017 9:17 PM
> *To:* spdk(a)lists.01.org
> *Subject:* [SPDK] SPDK Blobstore support object store?
>
>
>
> Hi，
>
> In the source code of SPDk (https://github.com/spdk/spdk/tree/master/lib),
> we can find *spdk* <https://github.com/spdk/spdk>/lib
> <https://github.com/spdk/spdk/tree/master/lib>/bdev
> <https://github.com/spdk/spdk/tree/master/lib/bdev>/ module that means SPDk
> is able to be in favor of block store and accelerate ceph block store.
> However, I don not see anything about object store. Does SPDK support
> object store? Is there any plan to do with object store?
>
>
>
>
>
> Could someone can give me the answer?
>
>
>
> Regards,
>
> Helloway
>
>
>
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 35122 bytes --]
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [SPDK] SPDK Blobstore support object store?
@ 2017-09-04 17:31 Luse, Paul E
0 siblings, 0 replies; 21+ messages in thread
From: Luse, Paul E @ 2017-09-04 17:31 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 10362 bytes --]
Hi Helloway,
OK, it just seems that w/o a target application (obj storage system or DB) some design decisions may get made that will make it less efficient than it could be to “fit” into at least one system really well - there are no standards for back end storage nodes for object storage systems so a “one size fits all” is highly unlikely (arguably impossible). Anyways, it’s certainly not a pre-requisite for you to work on it by any means, however it may impact how the maintainers look at your work as a possible contribution. Are you planning on designing something for contribution to the community? If so, some more discussion in this area is warranted for sure as everyone will want to make sure that whatever lands in the repo is not only robust/complete but readily usable as well.
Anyway wrt size, it seems like a reasonable starting approach however object storage can be a little tricky when it comes to object size as really anything goes. It’s more function of how the object storage system is being used as it is anything else. Most people are familiar with large objects being typical but that’s certainly not the case all of the time. There are many uses of Swift, for example, that have sub MB object sizes and choose an object storage system because of its API (REST, etc). With blobstore the page size is 4K and the default cluster size, which is the minimum size for a blob, is 1MB. The latter is configurable but not the former so if you had in mind objects of size 1MB or larger then, yeah, thinking of an object as a blob in terms of size makes sense.
There are many more considerations as well that I’m sure you’re aware of, curious as to what your thoughts were in some other areas like (just brainstorming here):
- Metadata storage support. Blobstore has a pretty minimal metadata (xattr) capability at least as compared to some of the common obj storage systems out there. A mapping of common capabilities here versus what blobstore does natively now would be a good exercise as closing the gaps could involve a decent amount of work in the new module and/or in blobstore itself. Most systems make some use of FS xattrs in combination with other non-FS mechanisms (KV DB) to store various types of metadata
- Metadata search: same kind of thoughts as above but keeping in mind that the search capability of the storage node could be severity limited if the actual back end storage didn’t have a pretty robust mechanism for caching and/or retrieving object metadata (blobstore does not)
- Support for containers: Containers in terms of how Swift uses them, basically S3 buckets. One could easily consider this out of scope if the idea is that this new thing is only targeted at objects themselves and not any sort of storage system metadata needs. I think that’s probably reasonable but I haven’t thought it all the way through
- Permissions requirements & enforcement
I’m sure we could come up a bunch more too and, again, having a target system to bolt into would help flesh out even more and drive a more robust design I think.
Thanks!!
Paul
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Sunday, September 3, 2017 9:10 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: Re: [SPDK] SPDK Blobstore support object store?
Hi Paul,
Thank you for adding me to the Trello board.
About these questions you mentioned, here are my thoughts:
Q1: Have you looked into Swift as a candidate for this activity or were you just using that as an example of a popular project?
A1: The blobkv provides the generic k-v interface module aimed to obj storage systems ( Such as: Swift, S3 ), so Swift and S3 are just used as examples of a popular project.
Q2: it’s probably the case that we’d want to target a specific object storage system and start with an investigation over how feasible it would be to bolt in SPDK at a high level.
A2: As the first answer said, the blobkv doesn’t target a specific object storage system. Our current thinking is that the blobkv is a standard k-v interface which located at the back end of S3, Swift, and so on.
For the reason above, we want to discuss with community about which is the proper size for the key/value pair in the blobkv? Could we design the k-v size as the same with the blob size so that the conversion from k-v to blob is minimum? I’m wondering if it is reasonable?
Regards,
Helloway
在 2017年9月1日，下午9:14，Luse, Paul E <paul.e.luse(a)intel.com<mailto:paul.e.luse(a)intel.com>> 写道：
Hi Helloway,
I just added you to the Trello board. Wrt your answer about API, yes that makes sense. Also, SPDK as it is today would need a tremendous amount of work to support an application level object storage API :)
So with S3 though, that’s kind of up to Amazon. For Swift, although it’s an open source project, there are some challenges there as well. Not that they’re not solvable of course, have you looked into Swift as a candidate for this activity or were you just using that as an example of a popular project?
Given the first point above, it’s probably the case that we’d want to target a specific object storage system and start with an investigation over how feasible it would be to bolt in SPDK at a high level and, depending on the project, engage with either that community or a company with a significant interest (installation) in making this happen don’t you think? Don’t get me wrong, I think it’s a great idea I’m just trying to help talk out the right approach. Either via some light abstraction layer or something we’d definitely want to identify a target system and see how interesting it might be to folks before we start any kind of detailed design discussions I think.
Thanks!
Paul
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Yang, Ziye
Sent: Friday, September 1, 2017 12:37 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] SPDK Blobstore support object store?
Hi We We,
For the membership, could you add by yourself. If you cannot, I think that Jim and Daniel can help you.
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Friday, September 1, 2017 2:20 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] SPDK Blobstore support object store?
Hi, @Paul @ Ziye
Thank you for all of your reply.
About these questions you mentioned, here are my thoughts:
Q1: Why blobkv doesn't tying into existing applications that support something like S3 or native Swift?
A1: Swift and S3 are different with blobkv at the hierarchy. Swift and S3 provide the application level k-v. However, the blobkv provides processing of generic k-v as a back-end behind Swift and S3，rather than the application level k-v.
This actually leads to another important question I want to discuss with the community, which is the proper size for the key/value pair in the blobkv. Our current thinking is that we could design the k-v size as the same with the blob size so that the conversion from k-v to blob is minimum. I’m wondering if this is a reasonable choice?
Q2: Could you also put this in SPDK trello: https://trello.com/spdk?
A2: I have a trello account (simple_hlw(a)163.com<mailto:simple_hlw(a)163.com>), and I am not a member in SPDK trello. Do I need to be a member before I have permissions to put this in SPDK trello?
Regards,
Helloway
在 2017年9月1日，上午7:52，Yang, Ziye <ziye.yang(a)intel.com<mailto:ziye.yang(a)intel.com>> 写道：
Hi We We,
Could you also put this in SPDK trello: https://<https://trello.com/spdk>trello.com/spdk<https://trello.com/spdk> ?
Thanks.
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Friday, September 1, 2017 12:04 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] SPDK Blobstore support object store?
Hi all,
Thank you for your answers.
I have submitted the proposal about the blobkv (blobstore object) design ( https://github.com/spdk/spdk/pull/188/files?short_path=420ca87#diff-420ca87f40f0f8170bb68bc5c742b6dc ), please visit it.
Regards,
Helloway
在 2017年8月31日，下午8:16，Yang, Ziye <ziye.yang(a)intel.com<mailto:ziye.yang(a)intel.com>> 写道：
Hi,
Currently, SPDK can be integrated with Ceph in the following two ways:
1 SPDK has rbd bdev, thus you can have SPDK iSCSI target which uses the exported rbd device by Ceph.
2 SPDK can be integrated into bluestore in Ceph, the code is NVMEDEVICE.CC/h, located in src/os/bluestore/ folder. However SPDK is not enabled in Ceph. You need to build with WITH_SPDK=on option. And these days, we are doing some work to make SPDK can be compiled default in Ceph.
Also in SPDK for object store support, we do not have a detailed plan now.
Best Regards,
Ziye Yang
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Monday, August 28, 2017 9:17 PM
To: spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>
Subject: [SPDK] SPDK Blobstore support object store?
Hi，
In the source code of SPDk (https://github.com/spdk/spdk/tree/master/lib), we can find spdk<https://github.com/spdk/spdk>/lib<https://github.com/spdk/spdk/tree/master/lib>/bdev<https://github.com/spdk/spdk/tree/master/lib/bdev>/ module that means SPDk is able to be in favor of block store and accelerate ceph block store. However, I don not see anything about object store. Does SPDK support object store? Is there any plan to do with object store?
Could someone can give me the answer?
Regards,
Helloway
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 38119 bytes --]
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [SPDK] SPDK Blobstore support object store?
@ 2017-09-04 4:09 We We
0 siblings, 0 replies; 21+ messages in thread
From: We We @ 2017-09-04 4:09 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 7716 bytes --]
Hi Paul,
Thank you for adding me to the Trello board.
About these questions you mentioned, here are my thoughts:
Q1: Have you looked into Swift as a candidate for this activity or were you just using that as an example of a popular project?
A1: The blobkv provides the generic k-v interface module aimed to obj storage systems ( Such as: Swift, S3 ), so Swift and S3 are just used as examples of a popular project.
Q2: it’s probably the case that we’d want to target a specific object storage system and start with an investigation over how feasible it would be to bolt in SPDK at a high level.
A2: As the first answer said, the blobkv doesn’t target a specific object storage system. Our current thinking is that the blobkv is a standard k-v interface which located at the back end of S3, Swift, and so on.
For the reason above, we want to discuss with community about which is the proper size for the key/value pair in the blobkv? Could we design the k-v size as the same with the blob size so that the conversion from k-v to blob is minimum? I’m wondering if it is reasonable?
Regards,
Helloway
> 在 2017年9月1日，下午9:14，Luse, Paul E <paul.e.luse(a)intel.com <mailto:paul.e.luse(a)intel.com>> 写道：
>
> Hi Helloway,
>
> I just added you to the Trello board. Wrt your answer about API, yes that makes sense. Also, SPDK as it is today would need a tremendous amount of work to support an application level object storage API J
>
> So with S3 though, that’s kind of up to Amazon. For Swift, although it’s an open source project, there are some challenges there as well. Not that they’re not solvable of course, have you looked into Swift as a candidate for this activity or were you just using that as an example of a popular project?
>
> Given the first point above, it’s probably the case that we’d want to target a specific object storage system and start with an investigation over how feasible it would be to bolt in SPDK at a high level and, depending on the project, engage with either that community or a company with a significant interest (installation) in making this happen don’t you think? Don’t get me wrong, I think it’s a great idea I’m just trying to help talk out the right approach. Either via some light abstraction layer or something we’d definitely want to identify a target system and see how interesting it might be to folks before we start any kind of detailed design discussions I think.
>
> Thanks!
> Paul
> <>
> From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of Yang, Ziye
> Sent: Friday, September 1, 2017 12:37 AM
> To: Storage Performance Development Kit <spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>>
> Subject: Re: [SPDK] SPDK Blobstore support object store?
>
> Hi We We,
>
> For the membership, could you add by yourself. If you cannot, I think that Jim and Daniel can help you.
>
> From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of We We
> Sent: Friday, September 1, 2017 2:20 PM
> To: Storage Performance Development Kit <spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>>
> Subject: Re: [SPDK] SPDK Blobstore support object store?
>
> Hi, @Paul @ Ziye
> Thank you for all of your reply.
> About these questions you mentioned, here are my thoughts:
> Q1: Why blobkv doesn't tying into existing applications that support something like S3 or native Swift?
> A1: Swift and S3 are different with blobkv at the hierarchy. Swift and S3 provide the application level k-v. However, the blobkv provides processing of generic k-v as a back-end behind Swift and S3，rather than the application level k-v.
>
> This actually leads to another important question I want to discuss with the community, which is the proper size for the key/value pair in the blobkv. Our current thinking is that we could design the k-v size as the same with the blob size so that the conversion from k-v to blob is minimum. I’m wondering if this is a reasonable choice?
>
> Q2: Could you also put this in SPDK trello: https://trello.com/spdk? <https://trello.com/spdk?>
> A2: I have a trello account (simple_hlw(a)163.com <mailto:simple_hlw(a)163.com>), and I am not a member in SPDK trello. Do I need to be a member before I have permissions to put this in SPDK trello?
>
> Regards,
> Helloway
>
>
>
> 在 2017年9月1日，上午7:52，Yang, Ziye <ziye.yang(a)intel.com <mailto:ziye.yang(a)intel.com>> 写道：
>
> Hi We We,
>
> Could you also put this in SPDK trello: https:// <https://trello.com/spdk>trello.com/spdk <https://trello.com/spdk> ?
>
> Thanks.
>
>
> From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of We We
> Sent: Friday, September 1, 2017 12:04 AM
> To: Storage Performance Development Kit <spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>>
> Subject: Re: [SPDK] SPDK Blobstore support object store?
>
> Hi all,
> Thank you for your answers.
> I have submitted the proposal about the blobkv (blobstore object) design ( https://github.com/spdk/spdk/pull/188/files?short_path=420ca87#diff-420ca87f40f0f8170bb68bc5c742b6dc <https://github.com/spdk/spdk/pull/188/files?short_path=420ca87#diff-420ca87f40f0f8170bb68bc5c742b6dc> ), please visit it.
>
> Regards,
> Helloway
>
> 在 2017年8月31日，下午8:16，Yang, Ziye <ziye.yang(a)intel.com <mailto:ziye.yang(a)intel.com>> 写道：
>
> Hi,
>
> Currently, SPDK can be integrated with Ceph in the following two ways:
>
> 1 SPDK has rbd bdev, thus you can have SPDK iSCSI target which uses the exported rbd device by Ceph.
> 2 SPDK can be integrated into bluestore in Ceph, the code is NVMEDEVICE.CC/h, located in src/os/bluestore/ folder. However SPDK is not enabled in Ceph. You need to build with WITH_SPDK=on option. And these days, we are doing some work to make SPDK can be compiled default in Ceph.
>
> Also in SPDK for object store support, we do not have a detailed plan now.
>
> Best Regards,
> Ziye Yang
>
> From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of We We
> Sent: Monday, August 28, 2017 9:17 PM
> To: spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>
> Subject: [SPDK] SPDK Blobstore support object store?
>
> Hi，
> In the source code of SPDk (https://github.com/spdk/spdk/tree/master/lib <https://github.com/spdk/spdk/tree/master/lib>), we can find spdk <https://github.com/spdk/spdk>/lib <https://github.com/spdk/spdk/tree/master/lib>/bdev <https://github.com/spdk/spdk/tree/master/lib/bdev>/ module that means SPDk is able to be in favor of block store and accelerate ceph block store. However, I don not see anything about object store. Does SPDK support object store? Is there any plan to do with object store?
>
>
>
>
> Could someone can give me the answer?
>
> Regards,
> Helloway
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 73311 bytes --]
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [SPDK] SPDK Blobstore support object store?
@ 2017-09-01 13:14 Luse, Paul E
0 siblings, 0 replies; 21+ messages in thread
From: Luse, Paul E @ 2017-09-01 13:14 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 5498 bytes --]
Hi Helloway,
I just added you to the Trello board. Wrt your answer about API, yes that makes sense. Also, SPDK as it is today would need a tremendous amount of work to support an application level object storage API :)
So with S3 though, that’s kind of up to Amazon. For Swift, although it’s an open source project, there are some challenges there as well. Not that they’re not solvable of course, have you looked into Swift as a candidate for this activity or were you just using that as an example of a popular project?
Given the first point above, it’s probably the case that we’d want to target a specific object storage system and start with an investigation over how feasible it would be to bolt in SPDK at a high level and, depending on the project, engage with either that community or a company with a significant interest (installation) in making this happen don’t you think? Don’t get me wrong, I think it’s a great idea I’m just trying to help talk out the right approach. Either via some light abstraction layer or something we’d definitely want to identify a target system and see how interesting it might be to folks before we start any kind of detailed design discussions I think.
Thanks!
Paul
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Yang, Ziye
Sent: Friday, September 1, 2017 12:37 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: Re: [SPDK] SPDK Blobstore support object store?
Hi We We,
For the membership, could you add by yourself. If you cannot, I think that Jim and Daniel can help you.
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Friday, September 1, 2017 2:20 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: Re: [SPDK] SPDK Blobstore support object store?
Hi, @Paul @ Ziye
Thank you for all of your reply.
About these questions you mentioned, here are my thoughts:
Q1: Why blobkv doesn't tying into existing applications that support something like S3 or native Swift?
A1: Swift and S3 are different with blobkv at the hierarchy. Swift and S3 provide the application level k-v. However, the blobkv provides processing of generic k-v as a back-end behind Swift and S3，rather than the application level k-v.
This actually leads to another important question I want to discuss with the community, which is the proper size for the key/value pair in the blobkv. Our current thinking is that we could design the k-v size as the same with the blob size so that the conversion from k-v to blob is minimum. I’m wondering if this is a reasonable choice?
Q2: Could you also put this in SPDK trello: https://trello.com/spdk?
A2: I have a trello account (simple_hlw(a)163.com<mailto:simple_hlw(a)163.com>), and I am not a member in SPDK trello. Do I need to be a member before I have permissions to put this in SPDK trello?
Regards,
Helloway
在 2017年9月1日，上午7:52，Yang, Ziye <ziye.yang(a)intel.com<mailto:ziye.yang(a)intel.com>> 写道：
Hi We We,
Could you also put this in SPDK trello: https://<https://trello.com/spdk>trello.com/spdk<https://trello.com/spdk> ?
Thanks.
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Friday, September 1, 2017 12:04 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] SPDK Blobstore support object store?
Hi all,
Thank you for your answers.
I have submitted the proposal about the blobkv (blobstore object) design ( https://github.com/spdk/spdk/pull/188/files?short_path=420ca87#diff-420ca87f40f0f8170bb68bc5c742b6dc ), please visit it.
Regards,
Helloway
在 2017年8月31日，下午8:16，Yang, Ziye <ziye.yang(a)intel.com<mailto:ziye.yang(a)intel.com>> 写道：
Hi,
Currently, SPDK can be integrated with Ceph in the following two ways:
1 SPDK has rbd bdev, thus you can have SPDK iSCSI target which uses the exported rbd device by Ceph.
2 SPDK can be integrated into bluestore in Ceph, the code is NVMEDEVICE.CC/h, located in src/os/bluestore/ folder. However SPDK is not enabled in Ceph. You need to build with WITH_SPDK=on option. And these days, we are doing some work to make SPDK can be compiled default in Ceph.
Also in SPDK for object store support, we do not have a detailed plan now.
Best Regards,
Ziye Yang
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Monday, August 28, 2017 9:17 PM
To: spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>
Subject: [SPDK] SPDK Blobstore support object store?
Hi，
In the source code of SPDk (https://github.com/spdk/spdk/tree/master/lib), we can find spdk<https://github.com/spdk/spdk>/lib<https://github.com/spdk/spdk/tree/master/lib>/bdev<https://github.com/spdk/spdk/tree/master/lib/bdev>/ module that means SPDk is able to be in favor of block store and accelerate ceph block store. However, I don not see anything about object store. Does SPDK support object store? Is there any plan to do with object store?
Could someone can give me the answer?
Regards,
Helloway
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 22998 bytes --]
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [SPDK] SPDK Blobstore support object store?
@ 2017-09-01 7:36 Yang, Ziye
0 siblings, 0 replies; 21+ messages in thread
From: Yang, Ziye @ 2017-09-01 7:36 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 4010 bytes --]
Hi We We,
For the membership, could you add by yourself. If you cannot, I think that Jim and Daniel can help you.
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Friday, September 1, 2017 2:20 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: Re: [SPDK] SPDK Blobstore support object store?
Hi, @Paul @ Ziye
Thank you for all of your reply.
About these questions you mentioned, here are my thoughts:
Q1: Why blobkv doesn't tying into existing applications that support something like S3 or native Swift?
A1: Swift and S3 are different with blobkv at the hierarchy. Swift and S3 provide the application level k-v. However, the blobkv provides processing of generic k-v as a back-end behind Swift and S3，rather than the application level k-v.
This actually leads to another important question I want to discuss with the community, which is the proper size for the key/value pair in the blobkv. Our current thinking is that we could design the k-v size as the same with the blob size so that the conversion from k-v to blob is minimum. I’m wondering if this is a reasonable choice?
Q2: Could you also put this in SPDK trello: https://trello.com/spdk?
A2: I have a trello account (simple_hlw(a)163.com<mailto:simple_hlw(a)163.com>), and I am not a member in SPDK trello. Do I need to be a member before I have permissions to put this in SPDK trello?
Regards,
Helloway
在 2017年9月1日，上午7:52，Yang, Ziye <ziye.yang(a)intel.com<mailto:ziye.yang(a)intel.com>> 写道：
Hi We We,
Could you also put this in SPDK trello: https://<https://trello.com/spdk>trello.com/spdk<https://trello.com/spdk> ?
Thanks.
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Friday, September 1, 2017 12:04 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] SPDK Blobstore support object store?
Hi all,
Thank you for your answers.
I have submitted the proposal about the blobkv (blobstore object) design ( https://github.com/spdk/spdk/pull/188/files?short_path=420ca87#diff-420ca87f40f0f8170bb68bc5c742b6dc ), please visit it.
Regards,
Helloway
在 2017年8月31日，下午8:16，Yang, Ziye <ziye.yang(a)intel.com<mailto:ziye.yang(a)intel.com>> 写道：
Hi,
Currently, SPDK can be integrated with Ceph in the following two ways:
1 SPDK has rbd bdev, thus you can have SPDK iSCSI target which uses the exported rbd device by Ceph.
2 SPDK can be integrated into bluestore in Ceph, the code is NVMEDEVICE.CC/h, located in src/os/bluestore/ folder. However SPDK is not enabled in Ceph. You need to build with WITH_SPDK=on option. And these days, we are doing some work to make SPDK can be compiled default in Ceph.
Also in SPDK for object store support, we do not have a detailed plan now.
Best Regards,
Ziye Yang
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Monday, August 28, 2017 9:17 PM
To: spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>
Subject: [SPDK] SPDK Blobstore support object store?
Hi，
In the source code of SPDk (https://github.com/spdk/spdk/tree/master/lib), we can find spdk<https://github.com/spdk/spdk>/lib<https://github.com/spdk/spdk/tree/master/lib>/bdev<https://github.com/spdk/spdk/tree/master/lib/bdev>/ module that means SPDk is able to be in favor of block store and accelerate ceph block store. However, I don not see anything about object store. Does SPDK support object store? Is there any plan to do with object store?
Could someone can give me the answer?
Regards,
Helloway
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 18967 bytes --]
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [SPDK] SPDK Blobstore support object store?
@ 2017-09-01 6:20 We We
0 siblings, 0 replies; 21+ messages in thread
From: We We @ 2017-09-01 6:20 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 3954 bytes --]
Hi, @Paul @ Ziye
Thank you for all of your reply.
About these questions you mentioned, here are my thoughts:
Q1: Why blobkv doesn't tying into existing applications that support something like S3 or native Swift?
A1: Swift and S3 are different with blobkv at the hierarchy. Swift and S3 provide the application level k-v. However, the blobkv provides processing of generic k-v as a back-end behind Swift and S3，rather than the application level k-v.
This actually leads to another important question I want to discuss with the community, which is the proper size for the key/value pair in the blobkv. Our current thinking is that we could design the k-v size as the same with the blob size so that the conversion from k-v to blob is minimum. I’m wondering if this is a reasonable choice?
Q2: Could you also put this in SPDK trello: https://trello.com/spdk?
A2: I have a trello account (simple_hlw(a)163.com), and I am not a member in SPDK trello. Do I need to be a member before I have permissions to put this in SPDK trello?
Regards,
Helloway
>
> 在 2017年9月1日，上午7:52，Yang, Ziye <ziye.yang(a)intel.com> 写道：
>
> Hi We We,
>
> Could you also put this in SPDK trello: https:// <https://trello.com/spdk>trello.com/spdk <https://trello.com/spdk> ?
>
> Thanks.
>
> <>
> From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
> Sent: Friday, September 1, 2017 12:04 AM
> To: Storage Performance Development Kit <spdk(a)lists.01.org>
> Subject: Re: [SPDK] SPDK Blobstore support object store?
>
> Hi all,
> Thank you for your answers.
> I have submitted the proposal about the blobkv (blobstore object) design ( https://github.com/spdk/spdk/pull/188/files?short_path=420ca87#diff-420ca87f40f0f8170bb68bc5c742b6dc <https://github.com/spdk/spdk/pull/188/files?short_path=420ca87#diff-420ca87f40f0f8170bb68bc5c742b6dc> ), please visit it.
>
> Regards,
> Helloway
>
> 在 2017年8月31日，下午8:16，Yang, Ziye <ziye.yang(a)intel.com <mailto:ziye.yang(a)intel.com>> 写道：
>
> Hi,
>
> Currently, SPDK can be integrated with Ceph in the following two ways:
>
> 1 SPDK has rbd bdev, thus you can have SPDK iSCSI target which uses the exported rbd device by Ceph.
> 2 SPDK can be integrated into bluestore in Ceph, the code is NVMEDEVICE.CC/h, located in src/os/bluestore/ folder. However SPDK is not enabled in Ceph. You need to build with WITH_SPDK=on option. And these days, we are doing some work to make SPDK can be compiled default in Ceph.
>
> Also in SPDK for object store support, we do not have a detailed plan now.
>
> Best Regards,
> Ziye Yang
>
> From: SPDK [mailto:spdk-bounces(a)lists.01.org <mailto:spdk-bounces(a)lists.01.org>] On Behalf Of We We
> Sent: Monday, August 28, 2017 9:17 PM
> To: spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>
> Subject: [SPDK] SPDK Blobstore support object store?
>
> Hi，
> In the source code of SPDk (https://github.com/spdk/spdk/tree/master/lib <https://github.com/spdk/spdk/tree/master/lib>), we can find spdk <https://github.com/spdk/spdk>/lib <https://github.com/spdk/spdk/tree/master/lib>/bdev <https://github.com/spdk/spdk/tree/master/lib/bdev>/ module that means SPDk is able to be in favor of block store and accelerate ceph block store. However, I don not see anything about object store. Does SPDK support object store? Is there any plan to do with object store?
>
>
>
> Could someone can give me the answer?
>
> Regards,
> Helloway
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 59775 bytes --]
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [SPDK] SPDK Blobstore support object store?
@ 2017-08-31 23:52 Yang, Ziye
0 siblings, 0 replies; 21+ messages in thread
From: Yang, Ziye @ 2017-08-31 23:52 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 2251 bytes --]
Hi We We,
Could you also put this in SPDK trello: https://<https://trello.com/spdk>trello.com/spdk<https://trello.com/spdk> ?
Thanks.
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Friday, September 1, 2017 12:04 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: Re: [SPDK] SPDK Blobstore support object store?
Hi all,
Thank you for your answers.
I have submitted the proposal about the blobkv (blobstore object) design ( https://github.com/spdk/spdk/pull/188/files?short_path=420ca87#diff-420ca87f40f0f8170bb68bc5c742b6dc ), please visit it.
Regards,
Helloway
在 2017年8月31日，下午8:16，Yang, Ziye <ziye.yang(a)intel.com<mailto:ziye.yang(a)intel.com>> 写道：
Hi,
Currently, SPDK can be integrated with Ceph in the following two ways:
1 SPDK has rbd bdev, thus you can have SPDK iSCSI target which uses the exported rbd device by Ceph.
2 SPDK can be integrated into bluestore in Ceph, the code is NVMEDEVICE.CC/h, located in src/os/bluestore/ folder. However SPDK is not enabled in Ceph. You need to build with WITH_SPDK=on option. And these days, we are doing some work to make SPDK can be compiled default in Ceph.
Also in SPDK for object store support, we do not have a detailed plan now.
Best Regards,
Ziye Yang
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Monday, August 28, 2017 9:17 PM
To: spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>
Subject: [SPDK] SPDK Blobstore support object store?
Hi，
In the source code of SPDk (https://github.com/spdk/spdk/tree/master/lib), we can find spdk<https://github.com/spdk/spdk>/lib<https://github.com/spdk/spdk/tree/master/lib>/bdev<https://github.com/spdk/spdk/tree/master/lib/bdev>/ module that means SPDk is able to be in favor of block store and accelerate ceph block store. However, I don not see anything about object store. Does SPDK support object store? Is there any plan to do with object store?
Could someone can give me the answer?
Regards,
Helloway
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 13136 bytes --]
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [SPDK] SPDK Blobstore support object store?
@ 2017-08-31 17:30 Luse, Paul E
0 siblings, 0 replies; 21+ messages in thread
From: Luse, Paul E @ 2017-08-31 17:30 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 2620 bytes --]
Hi Helloway,
Now that’s a great way to start the discussion!! Yes, I think certainly blobstore has a role in any SPDK solution that presents an object interface. Just to start with a basic question, your write up mentions some popular defacto standard interfaces for obj storage systems but then your high level blobkv if looks to be someone new & different. What are your thoughts on the external interface and tying into existing applications that support something like S3 or native Swift?
Thx
Paul
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Thursday, August 31, 2017 9:04 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: Re: [SPDK] SPDK Blobstore support object store?
Hi all,
Thank you for your answers.
I have submitted the proposal about the blobkv (blobstore object) design ( https://github.com/spdk/spdk/pull/188/files?short_path=420ca87#diff-420ca87f40f0f8170bb68bc5c742b6dc ), please visit it.
Regards,
Helloway
在 2017年8月31日，下午8:16，Yang, Ziye <ziye.yang(a)intel.com<mailto:ziye.yang(a)intel.com>> 写道：
Hi,
Currently, SPDK can be integrated with Ceph in the following two ways:
1 SPDK has rbd bdev, thus you can have SPDK iSCSI target which uses the exported rbd device by Ceph.
2 SPDK can be integrated into bluestore in Ceph, the code is NVMEDEVICE.CC/h, located in src/os/bluestore/ folder. However SPDK is not enabled in Ceph. You need to build with WITH_SPDK=on option. And these days, we are doing some work to make SPDK can be compiled default in Ceph.
Also in SPDK for object store support, we do not have a detailed plan now.
Best Regards,
Ziye Yang
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Monday, August 28, 2017 9:17 PM
To: spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>
Subject: [SPDK] SPDK Blobstore support object store?
Hi，
In the source code of SPDk (https://github.com/spdk/spdk/tree/master/lib), we can find spdk<https://github.com/spdk/spdk>/lib<https://github.com/spdk/spdk/tree/master/lib>/bdev<https://github.com/spdk/spdk/tree/master/lib/bdev>/ module that means SPDk is able to be in favor of block store and accelerate ceph block store. However, I don not see anything about object store. Does SPDK support object store? Is there any plan to do with object store?
Could someone can give me the answer?
Regards,
Helloway
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 13237 bytes --]
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [SPDK] SPDK Blobstore support object store?
@ 2017-08-31 16:03 We We
0 siblings, 0 replies; 21+ messages in thread
From: We We @ 2017-08-31 16:03 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 2089 bytes --]
Hi all,
Thank you for your answers.
I have submitted the proposal about the blobkv (blobstore object) design ( https://github.com/spdk/spdk/pull/188/files?short_path=420ca87#diff-420ca87f40f0f8170bb68bc5c742b6dc <https://github.com/spdk/spdk/pull/188/files?short_path=420ca87#diff-420ca87f40f0f8170bb68bc5c742b6dc> ), please visit it.
Regards,
Helloway
> 在 2017年8月31日，下午8:16，Yang, Ziye <ziye.yang(a)intel.com> 写道：
>
> Hi,
>
> Currently, SPDK can be integrated with Ceph in the following two ways:
>
> 1 SPDK has rbd bdev, thus you can have SPDK iSCSI target which uses the exported rbd device by Ceph.
> 2 SPDK can be integrated into bluestore in Ceph, the code is NVMEDEVICE.CC/h, located in src/os/bluestore/ folder. However SPDK is not enabled in Ceph. You need to build with WITH_SPDK=on option. And these days, we are doing some work to make SPDK can be compiled default in Ceph.
>
> Also in SPDK for object store support, we do not have a detailed plan now.
>
> Best Regards,
> Ziye Yang
> <>
> From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
> Sent: Monday, August 28, 2017 9:17 PM
> To: spdk(a)lists.01.org
> Subject: [SPDK] SPDK Blobstore support object store?
>
> Hi，
> In the source code of SPDk (https://github.com/spdk/spdk/tree/master/lib <https://github.com/spdk/spdk/tree/master/lib>), we can find spdk <https://github.com/spdk/spdk>/lib <https://github.com/spdk/spdk/tree/master/lib>/bdev <https://github.com/spdk/spdk/tree/master/lib/bdev>/ module that means SPDk is able to be in favor of block store and accelerate ceph block store. However, I don not see anything about object store. Does SPDK support object store? Is there any plan to do with object store?
>
>
> Could someone can give me the answer?
>
> Regards,
> Helloway
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 14589 bytes --]
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [SPDK] SPDK Blobstore support object store?
@ 2017-08-31 12:16 Yang, Ziye
0 siblings, 0 replies; 21+ messages in thread
From: Yang, Ziye @ 2017-08-31 12:16 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 1310 bytes --]
Hi,
Currently, SPDK can be integrated with Ceph in the following two ways:
1 SPDK has rbd bdev, thus you can have SPDK iSCSI target which uses the exported rbd device by Ceph.
2 SPDK can be integrated into bluestore in Ceph, the code is NVMEDEVICE.CC/h, located in src/os/bluestore/ folder. However SPDK is not enabled in Ceph. You need to build with WITH_SPDK=on option. And these days, we are doing some work to make SPDK can be compiled default in Ceph.
Also in SPDK for object store support, we do not have a detailed plan now.
Best Regards,
Ziye Yang
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Monday, August 28, 2017 9:17 PM
To: spdk(a)lists.01.org
Subject: [SPDK] SPDK Blobstore support object store?
Hi，
In the source code of SPDk (https://github.com/spdk/spdk/tree/master/lib), we can find spdk<https://github.com/spdk/spdk>/lib<https://github.com/spdk/spdk/tree/master/lib>/bdev<https://github.com/spdk/spdk/tree/master/lib/bdev>/ module that means SPDk is able to be in favor of block store and accelerate ceph block store. However, I don not see anything about object store. Does SPDK support object store? Is there any plan to do with object store?
Could someone can give me the answer?
Regards,
Helloway
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 9726 bytes --]
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [SPDK] SPDK Blobstore support object store?
@ 2017-08-28 17:16 Luse, Paul E
0 siblings, 0 replies; 21+ messages in thread
From: Luse, Paul E @ 2017-08-28 17:16 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 3431 bytes --]
Hi Helloway,
There are several folks in the community that have worked on Ceph enhancements and you can find a lot of that info in the list archives, for example https://lists.01.org/pipermail/spdk/2017-February/000329.html
You can also google SPDK & Ceph and find presentations like this one from Ziye who also monitors this email list https://www.slideshare.net/DanielleWomboldt/ceph-day-beijing-spdk-for-ceph
If you would like to help work on this aspect of SPDK, this is how you join the discussion - you just joined :) Take a look at the materials I just mentioned and use either this list or IRC to see what the current activities are and where you can contribute.
With regards to an SPDK module that exposes an object interface, which is actually what I thought you meant at first, that’s where I was saying that I wasn’t aware of any real activity. If there’s any interest there then this list or IRC, mentioned below, would be the place to try and get some discussion started.
Thx
Paul
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Monday, August 28, 2017 7:38 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: Re: [SPDK] SPDK Blobstore support object store?
Hi Paul,
Thank you for your answer. we know there is also object store in the ceph. spdk/lib/bdev/ can accelerate ceph block store, so we think if we can also be able to use spdk to accelerate ceph object store? If we can, Do you have any directions and guidelines of object store? If we want to contribute for object store, what should we do about it? And who can we join in the discussion with?
Thx
Helloway
在 2017年8月28日，下午9:53，Luse, Paul E <paul.e.luse(a)intel.com<mailto:paul.e.luse(a)intel.com>> 写道：
Hi Helloway,
You are correct in that much of the focus on the various SPDK modules is block. If you’d like more general info, the community had summit meeting back in April, you can find that info here: http://www.spdk.io/news/2017/06/21/summit_videos/
Regarding object storage, I’m not aware of anyone out there actively working on anything but some of us have discussed it for sure and I suspect there’ll be more talk on this topic throughout the rest of the year. Is your question just a general one or did you have a specific API and/or use case in mind? Curious if anyone else out there has specific interests in this are as well?
Note that we’re also on IRC, freenode on channel #spdk
Thx
Paul
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
Sent: Monday, August 28, 2017 6:17 AM
To: spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>
Subject: [SPDK] SPDK Blobstore support object store?
Hi，
In the source code of SPDk (https://github.com/spdk/spdk/tree/master/lib), we can find spdk<https://github.com/spdk/spdk>/lib<https://github.com/spdk/spdk/tree/master/lib>/bdev<https://github.com/spdk/spdk/tree/master/lib/bdev>/ module that means SPDk is able to be in favor of block store and accelerate ceph block store. However, I don not see anything about object store. Does SPDK support object store? Is there any plan to do with object store?
Could someone can give me the answer?
Regards,
Helloway
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 14495 bytes --]
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [SPDK] SPDK Blobstore support object store?
@ 2017-08-28 14:37 We We
0 siblings, 0 replies; 21+ messages in thread
From: We We @ 2017-08-28 14:37 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 2343 bytes --]
Hi Paul,
Thank you for your answer. we know there is also object store in the ceph. spdk/lib/bdev/ can accelerate ceph block store, so we think if we can also be able to use spdk to accelerate ceph object store? If we can, Do you have any directions and guidelines of object store? If we want to contribute for object store, what should we do about it? And who can we join in the discussion with?
Thx
Helloway
> 在 2017年8月28日，下午9:53，Luse, Paul E <paul.e.luse(a)intel.com> 写道：
>
> Hi Helloway,
>
> You are correct in that much of the focus on the various SPDK modules is block. If you’d like more general info, the community had summit meeting back in April, you can find that info here: http://www.spdk.io/news/2017/06/21/summit_videos/ <http://www.spdk.io/news/2017/06/21/summit_videos/>
>
> Regarding object storage, I’m not aware of anyone out there actively working on anything but some of us have discussed it for sure and I suspect there’ll be more talk on this topic throughout the rest of the year. Is your question just a general one or did you have a specific API and/or use case in mind? Curious if anyone else out there has specific interests in this are as well?
>
> Note that we’re also on IRC, freenode on channel #spdk
>
> Thx
> Paul
> <>
> From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We
> Sent: Monday, August 28, 2017 6:17 AM
> To: spdk(a)lists.01.org
> Subject: [SPDK] SPDK Blobstore support object store?
>
> Hi，
> In the source code of SPDk (https://github.com/spdk/spdk/tree/master/lib <https://github.com/spdk/spdk/tree/master/lib>), we can find spdk <https://github.com/spdk/spdk>/lib <https://github.com/spdk/spdk/tree/master/lib>/bdev <https://github.com/spdk/spdk/tree/master/lib/bdev>/ module that means SPDk is able to be in favor of block store and accelerate ceph block store. However, I don not see anything about object store. Does SPDK support object store? Is there any plan to do with object store?
>
>
> Could someone can give me the answer?
>
> Regards,
> Helloway
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org <mailto:SPDK(a)lists.01.org>
> https://lists.01.org/mailman/listinfo/spdk <https://lists.01.org/mailman/listinfo/spdk>
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 14109 bytes --]
^ permalink raw reply [flat|nested] 21+ messages in thread
* [SPDK] SPDK Blobstore support object store?
@ 2017-08-28 13:16 We We
0 siblings, 0 replies; 21+ messages in thread
From: We We @ 2017-08-28 13:16 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 576 bytes --]
Hi，
In the source code of SPDk (https://github.com/spdk/spdk/tree/master/lib <https://github.com/spdk/spdk/tree/master/lib>), we can find spdk <https://github.com/spdk/spdk>/lib <https://github.com/spdk/spdk/tree/master/lib>/bdev <https://github.com/spdk/spdk/tree/master/lib/bdev>/ module that means SPDk is able to be in favor of block store and accelerate ceph block store. However, I don not see anything about object store. Does SPDK support object store? Is there any plan to do with object store?
Could someone can give me the answer?
Regards,
Helloway
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 4908 bytes --]
^ permalink raw reply [flat|nested] 21+ messages in thread
end of thread, other threads:[~2017-09-07 14:19 UTC | newest]
Thread overview: 21+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-08-28 13:53 [SPDK] SPDK Blobstore support object store? Luse, Paul E
-- strict thread matches above, loose matches on Subject: below --
2017-09-07 14:19 Luse, Paul E
2017-09-07 11:55 We We
2017-09-07 11:52 We We
2017-09-05 13:46 Zhipeng Huang
2017-09-05 13:20 Luse, Paul E
2017-09-05 4:16 Zhipeng Huang
2017-09-05 1:59 Luse, Paul E
2017-09-05 0:55 Zhipeng Huang
2017-09-04 17:31 Luse, Paul E
2017-09-04 4:09 We We
2017-09-01 13:14 Luse, Paul E
2017-09-01 7:36 Yang, Ziye
2017-09-01 6:20 We We
2017-08-31 23:52 Yang, Ziye
2017-08-31 17:30 Luse, Paul E
2017-08-31 16:03 We We
2017-08-31 12:16 Yang, Ziye
2017-08-28 17:16 Luse, Paul E
2017-08-28 14:37 We We
2017-08-28 13:16 We We
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.