Hi Howard, Yup, I understand. My points are talking about the back end, the point at which the storage application interfaces with the actual storage. In some cases, it’s direct, in some cases it’s via a filesystem and in some cases it a database or a combination of all of these things (with or without caching). It’s also not the case that objects are always sliced up, in some cases they are and when they are they may or may not be erasure coded and in the cases they’re not they are likely replicated. There are lots of variations is the point; even the very, very back end isn’t the same between object different storage systems ☺ Anyway, 1MB does sound like a good general number but again without some targeted usage I think you’ll find it challenging to get very many folks to work on it with you. Thx Paul From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Zhipeng Huang Sent: Monday, September 4, 2017 5:55 PM To: Storage Performance Development Kit Subject: Re: [SPDK] SPDK Blobstore support object store? Hi Paul, This is Howard from Helloway's team. One of the reason that we don't want to target a specific object front-end (S3、Swift or Ceph RGW) is that (1)it is difficult to harmonize across these solutions and (2) we view spdk more of a back end role for these APIs For most of the object storage services, they provide the functionalities of slicing up k/V pair they received to fit the back end storage. For example S3 would consume a 4M object but slice it up to a smaller size when it actually stores these objects, if I understand correctly. It will be the same case for Ceph or Swift. Therefore we envisioned that blobkv provides the smaller sized object api that acts as the back end for S3、Swift or Ceph. I think 1M would be a reasonable kv size for blobkv ：) On Sep 5, 2017 1:31 AM, "Luse, Paul E" > wrote: Hi Helloway, OK, it just seems that w/o a target application (obj storage system or DB) some design decisions may get made that will make it less efficient than it could be to “fit” into at least one system really well – there are no standards for back end storage nodes for object storage systems so a “one size fits all” is highly unlikely (arguably impossible). Anyways, it’s certainly not a pre-requisite for you to work on it by any means, however it may impact how the maintainers look at your work as a possible contribution. Are you planning on designing something for contribution to the community? If so, some more discussion in this area is warranted for sure as everyone will want to make sure that whatever lands in the repo is not only robust/complete but readily usable as well. Anyway wrt size, it seems like a reasonable starting approach however object storage can be a little tricky when it comes to object size as really anything goes. It’s more function of how the object storage system is being used as it is anything else. Most people are familiar with large objects being typical but that’s certainly not the case all of the time. There are many uses of Swift, for example, that have sub MB object sizes and choose an object storage system because of its API (REST, etc). With blobstore the page size is 4K and the default cluster size, which is the minimum size for a blob, is 1MB. The latter is configurable but not the former so if you had in mind objects of size 1MB or larger then, yeah, thinking of an object as a blob in terms of size makes sense. There are many more considerations as well that I’m sure you’re aware of, curious as to what your thoughts were in some other areas like (just brainstorming here): - Metadata storage support. Blobstore has a pretty minimal metadata (xattr) capability at least as compared to some of the common obj storage systems out there. A mapping of common capabilities here versus what blobstore does natively now would be a good exercise as closing the gaps could involve a decent amount of work in the new module and/or in blobstore itself. Most systems make some use of FS xattrs in combination with other non-FS mechanisms (KV DB) to store various types of metadata - Metadata search: same kind of thoughts as above but keeping in mind that the search capability of the storage node could be severity limited if the actual back end storage didn’t have a pretty robust mechanism for caching and/or retrieving object metadata (blobstore does not) - Support for containers: Containers in terms of how Swift uses them, basically S3 buckets. One could easily consider this out of scope if the idea is that this new thing is only targeted at objects themselves and not any sort of storage system metadata needs. I think that’s probably reasonable but I haven’t thought it all the way through - Permissions requirements & enforcement I’m sure we could come up a bunch more too and, again, having a target system to bolt into would help flesh out even more and drive a more robust design I think. Thanks!! Paul From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We Sent: Sunday, September 3, 2017 9:10 PM To: Storage Performance Development Kit > Subject: Re: [SPDK] SPDK Blobstore support object store? Hi Paul, Thank you for adding me to the Trello board. About these questions you mentioned, here are my thoughts: Q1: Have you looked into Swift as a candidate for this activity or were you just using that as an example of a popular project? A1: The blobkv provides the generic k-v interface module aimed to obj storage systems ( Such as: Swift, S3 ), so Swift and S3 are just used as examples of a popular project. Q2: it’s probably the case that we’d want to target a specific object storage system and start with an investigation over how feasible it would be to bolt in SPDK at a high level. A2: As the first answer said, the blobkv doesn’t target a specific object storage system. Our current thinking is that the blobkv is a standard k-v interface which located at the back end of S3, Swift, and so on. For the reason above, we want to discuss with community about which is the proper size for the key/value pair in the blobkv? Could we design the k-v size as the same with the blob size so that the conversion from k-v to blob is minimum? I’m wondering if it is reasonable? Regards, Helloway 在 2017年9月1日，下午9:14，Luse, Paul E > 写道： Hi Helloway, I just added you to the Trello board. Wrt your answer about API, yes that makes sense. Also, SPDK as it is today would need a tremendous amount of work to support an application level object storage API ☺ So with S3 though, that’s kind of up to Amazon. For Swift, although it’s an open source project, there are some challenges there as well. Not that they’re not solvable of course, have you looked into Swift as a candidate for this activity or were you just using that as an example of a popular project? Given the first point above, it’s probably the case that we’d want to target a specific object storage system and start with an investigation over how feasible it would be to bolt in SPDK at a high level and, depending on the project, engage with either that community or a company with a significant interest (installation) in making this happen don’t you think? Don’t get me wrong, I think it’s a great idea I’m just trying to help talk out the right approach. Either via some light abstraction layer or something we’d definitely want to identify a target system and see how interesting it might be to folks before we start any kind of detailed design discussions I think. Thanks! Paul From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Yang, Ziye Sent: Friday, September 1, 2017 12:37 AM To: Storage Performance Development Kit > Subject: Re: [SPDK] SPDK Blobstore support object store? Hi We We, For the membership, could you add by yourself. If you cannot, I think that Jim and Daniel can help you. From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We Sent: Friday, September 1, 2017 2:20 PM To: Storage Performance Development Kit > Subject: Re: [SPDK] SPDK Blobstore support object store? Hi, @Paul @ Ziye Thank you for all of your reply. About these questions you mentioned, here are my thoughts: Q1: Why blobkv doesn't tying into existing applications that support something like S3 or native Swift? A1: Swift and S3 are different with blobkv at the hierarchy. Swift and S3 provide the application level k-v. However, the blobkv provides processing of generic k-v as a back-end behind Swift and S3，rather than the application level k-v. This actually leads to another important question I want to discuss with the community, which is the proper size for the key/value pair in the blobkv. Our current thinking is that we could design the k-v size as the same with the blob size so that the conversion from k-v to blob is minimum. I’m wondering if this is a reasonable choice? Q2: Could you also put this in SPDK trello: https://trello.com/spdk? A2: I have a trello account (simple_hlw(a)163.com), and I am not a member in SPDK trello. Do I need to be a member before I have permissions to put this in SPDK trello? Regards, Helloway 在 2017年9月1日，上午7:52，Yang, Ziye > 写道： Hi We We, Could you also put this in SPDK trello: https://trello.com/spdk ? Thanks. From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We Sent: Friday, September 1, 2017 12:04 AM To: Storage Performance Development Kit > Subject: Re: [SPDK] SPDK Blobstore support object store? Hi all, Thank you for your answers. I have submitted the proposal about the blobkv (blobstore object) design ( https://github.com/spdk/spdk/pull/188/files?short_path=420ca87#diff-420ca87f40f0f8170bb68bc5c742b6dc ), please visit it. Regards, Helloway 在 2017年8月31日，下午8:16，Yang, Ziye > 写道： Hi, Currently, SPDK can be integrated with Ceph in the following two ways: 1 SPDK has rbd bdev, thus you can have SPDK iSCSI target which uses the exported rbd device by Ceph. 2 SPDK can be integrated into bluestore in Ceph, the code is NVMEDEVICE.CC/h, located in src/os/bluestore/ folder. However SPDK is not enabled in Ceph. You need to build with WITH_SPDK=on option. And these days, we are doing some work to make SPDK can be compiled default in Ceph. Also in SPDK for object store support, we do not have a detailed plan now. Best Regards, Ziye Yang From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of We We Sent: Monday, August 28, 2017 9:17 PM To: spdk(a)lists.01.org Subject: [SPDK] SPDK Blobstore support object store? Hi， In the source code of SPDk (https://github.com/spdk/spdk/tree/master/lib), we can find spdk/lib/bdev/ module that means SPDk is able to be in favor of block store and accelerate ceph block store. However, I don not see anything about object store. Does SPDK support object store? Is there any plan to do with object store? Could someone can give me the answer? Regards, Helloway _______________________________________________ SPDK mailing list SPDK(a)lists.01.org https://lists.01.org/mailman/listinfo/spdk _______________________________________________ SPDK mailing list SPDK(a)lists.01.org https://lists.01.org/mailman/listinfo/spdk _______________________________________________ SPDK mailing list SPDK(a)lists.01.org https://lists.01.org/mailman/listinfo/spdk _______________________________________________ SPDK mailing list SPDK(a)lists.01.org https://lists.01.org/mailman/listinfo/spdk