RGW, future directions

RGW, future directions

[Yehuda Sadeh]

RGW is maturing. Beside looking at performance, which highly ties into
RADOS performance, we'd like to hear whether there are certain pain
points or future directions that you (you as in the ceph community)
would like to see us taking.

There are a few directions that we were thinking about:

1. Extend Object Storage API

Swift and S3 has some features that we don't currently support. We can
certainly extend our functionality, however, is there any demand for
more features? E.g., self destructing objects, web site, user logs,
etc.

2. Better OpenStack interoperability

Keystone support? Other?

3. New features

Some examples:

 - multitenancy: api for domains and user management
 - snapshots
 - computation front end: upload object, then do some data
transformation/calculation.
 - simple key-value api

4. CDMI

Sage brought up the CDMI support question to ceph-devel, and I don't
remember him getting any response. Is there any intereset in CDMI?


5. Native apache/nginx module or embedded web server

We still need to prove that the web server is a bottleneck, or poses
scaling issues. Writing a correct native nginx module will require
turning rgw process model into event driven, which is not going to be
easy.

6. Improve garbage collection

Currently rgw generates intent logs for garabage removal that require
running an external tool later, which is an administrative pain. We
can implement other solutions (OSD side garbage collection,
integrating cleanup process into the gateway, etc.) but we need to
understand the priority.

7. libradosgw

We have had this in mind for some time now. Creating a programming api
for rgw, not too different from librados and librbd. It'll hopefully
make code much cleaner. It will allow users to write different front
ends for the rgw backend, and it will make it easier for users to
write applications that interact with the backend, e.g., do processing
on objects that users uploaded, FUSE for rgw without S3 as an
intermediate, etc.

8. Administration tools improvement

We can always do better there.

9. Other ideas?


Any comments are welcome!

Thanks,
Yehuda
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: RGW, future directions

[Sławomir Skowron]

On Tue, May 22, 2012 at 8:07 PM, Yehuda Sadeh <yehuda@inktank.com> wrote:
> RGW is maturing. Beside looking at performance, which highly ties into
> RADOS performance, we'd like to hear whether there are certain pain
> points or future directions that you (you as in the ceph community)
> would like to see us taking.
>
> There are a few directions that we were thinking about:
>
> 1. Extend Object Storage API
>
> Swift and S3 has some features that we don't currently support. We can
> certainly extend our functionality, however, is there any demand for
> more features? E.g., self destructing objects, web site, user logs,
> etc.

More compatibility with S3 and swift is good.

>
> 2. Better OpenStack interoperability
>
> Keystone support? Other?
>
> 3. New features
>
> Some examples:
>
>  - multitenancy: api for domains and user management
>  - snapshots
>  - computation front end: upload object, then do some data
> transformation/calculation.
>  - simple key-value api
>
> 4. CDMI
>
> Sage brought up the CDMI support question to ceph-devel, and I don't
> remember him getting any response. Is there any intereset in CDMI?
>
>
> 5. Native apache/nginx module or embedded web server
>
> We still need to prove that the web server is a bottleneck, or poses
> scaling issues. Writing a correct native nginx module will require
> turning rgw process model into event driven, which is not going to be
> easy.
>

nginx module is nice thing.

> 6. Improve garbage collection
>
> Currently rgw generates intent logs for garabage removal that require
> running an external tool later, which is an administrative pain. We
> can implement other solutions (OSD side garbage collection,
> integrating cleanup process into the gateway, etc.) but we need to
> understand the priority.

crontab can handle this task for now, but in big workload, better if
it's integrated, like scrub, and tuned via conf

>
> 7. libradosgw
>
> We have had this in mind for some time now. Creating a programming api
> for rgw, not too different from librados and librbd. It'll hopefully
> make code much cleaner. It will allow users to write different front
> ends for the rgw backend, and it will make it easier for users to
> write applications that interact with the backend, e.g., do processing
> on objects that users uploaded, FUSE for rgw without S3 as an
> intermediate, etc.
>
> 8. Administration tools improvement
>
> We can always do better there.
>
> 9. Other ideas?

- I would like to see a feature that can make replication between
clusters. For start 2 clusters. It's a very good feature, when you
have two datacenters, and repliacation goes via hight speed link, but
aplications at top of clusters does not need to handle this tasks, and
data are consistent.

- gracefull upgrade

- reload cluster config without restart daemons - or maybe this exist
right now ??

>
>
> Any comments are welcome!
>
> Thanks,
> Yehuda
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
-----
Pozdrawiam

Sławek "sZiBis" Skowron
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: RGW, future directions

[Yehuda Sadeh]

On Tue, May 22, 2012 at 11:25 AM, Sławomir Skowron <szibis@gmail.com> wrote:
> On Tue, May 22, 2012 at 8:07 PM, Yehuda Sadeh <yehuda@inktank.com> wrote:
>> RGW is maturing. Beside looking at performance, which highly ties into
>> RADOS performance, we'd like to hear whether there are certain pain
>> points or future directions that you (you as in the ceph community)
>> would like to see us taking.
>>
>> There are a few directions that we were thinking about:
>>
>> 1. Extend Object Storage API
>>
>> Swift and S3 has some features that we don't currently support. We can
>> certainly extend our functionality, however, is there any demand for
>> more features? E.g., self destructing objects, web site, user logs,
>> etc.
>
> More compatibility with S3 and swift is good.

Any specific functional interest?

>
>>
>> 2. Better OpenStack interoperability
>>
>> Keystone support? Other?
>>
>> 3. New features
>>
>> Some examples:
>>
>>  - multitenancy: api for domains and user management
>>  - snapshots
>>  - computation front end: upload object, then do some data
>> transformation/calculation.
>>  - simple key-value api
>>
>> 4. CDMI
>>
>> Sage brought up the CDMI support question to ceph-devel, and I don't
>> remember him getting any response. Is there any intereset in CDMI?
>>
>>
>> 5. Native apache/nginx module or embedded web server
>>
>> We still need to prove that the web server is a bottleneck, or poses
>> scaling issues. Writing a correct native nginx module will require
>> turning rgw process model into event driven, which is not going to be
>> easy.
>>
>
> nginx module is nice thing.

It would be nice to have some concrete numbers as to where apache or
nginx with fastcgi holding us back, and how a dedicated module is
going to improve that. As a rule of thumb it is a no brainer, but
still we want to have a better understanding of the situation before
we dive into such a project.

>
>> 6. Improve garbage collection
>>
>> Currently rgw generates intent logs for garabage removal that require
>> running an external tool later, which is an administrative pain. We
>> can implement other solutions (OSD side garbage collection,
>> integrating cleanup process into the gateway, etc.) but we need to
>> understand the priority.
>
> crontab can handle this task for now, but in big workload, better if
> it's integrated, like scrub, and tuned via conf

Yeah. One of the original ideas was to leverage scrubbing for objects
expiration (issue #1994). The discussion never converged, as the devil
is as always in the details. We can revive that discussion.

>
>>
>> 7. libradosgw
>>
>> We have had this in mind for some time now. Creating a programming api
>> for rgw, not too different from librados and librbd. It'll hopefully
>> make code much cleaner. It will allow users to write different front
>> ends for the rgw backend, and it will make it easier for users to
>> write applications that interact with the backend, e.g., do processing
>> on objects that users uploaded, FUSE for rgw without S3 as an
>> intermediate, etc.
>>
>> 8. Administration tools improvement
>>
>> We can always do better there.
>>
>> 9. Other ideas?
>
> - I would like to see a feature that can make replication between
> clusters. For start 2 clusters. It's a very good feature, when you
> have two datacenters, and repliacation goes via hight speed link, but
> aplications at top of clusters does not need to handle this tasks, and
> data are consistent.

This is a more generic ceph issue. We do need, however, to be able to
support multiple clusters in rgw. Opened issue #2460.

>
> - gracefull upgrade

Can you elaborate on that? This is always our intention, we've put
some mechanisms in place to help with that, although it is not always
possible.

>
> - reload cluster config without restart daemons - or maybe this exist
> right now ??
>

I opened issue #2459.

Thanks,
Yehuda
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: RGW, future directions

[Sławomir Skowron]

On Tue, May 22, 2012 at 9:09 PM, Yehuda Sadeh <yehuda@inktank.com> wrote:
> On Tue, May 22, 2012 at 11:25 AM, Sławomir Skowron <szibis@gmail.com> wrote:
>> On Tue, May 22, 2012 at 8:07 PM, Yehuda Sadeh <yehuda@inktank.com> wrote:
>>> RGW is maturing. Beside looking at performance, which highly ties into
>>> RADOS performance, we'd like to hear whether there are certain pain
>>> points or future directions that you (you as in the ceph community)
>>> would like to see us taking.
>>>
>>> There are a few directions that we were thinking about:
>>>
>>> 1. Extend Object Storage API
>>>
>>> Swift and S3 has some features that we don't currently support. We can
>>> certainly extend our functionality, however, is there any demand for
>>> more features? E.g., self destructing objects, web site, user logs,
>>> etc.
>>
>> More compatibility with S3 and swift is good.

Now i have no issues about that, because we use only simples parts of
S3, but there is always, a good if compatibility is better.

>
> Any specific functional interest?
>
>>
>>>
>>> 2. Better OpenStack interoperability
>>>
>>> Keystone support? Other?
>>>
>>> 3. New features
>>>
>>> Some examples:
>>>
>>>  - multitenancy: api for domains and user management

multitenancy - quotas, QoS, and user management it's nice feature for
admin, especially in cloud systems, with many different type of
control, even performance, and nice integration API for external
accounting services in cloud system.

>>>  - snapshots
>>>  - computation front end: upload object, then do some data
>>> transformation/calculation.
>>>  - simple key-value api
>>>
>>> 4. CDMI
>>>
>>> Sage brought up the CDMI support question to ceph-devel, and I don't
>>> remember him getting any response. Is there any intereset in CDMI?
>>>
>>>
>>> 5. Native apache/nginx module or embedded web server
>>>
>>> We still need to prove that the web server is a bottleneck, or poses
>>> scaling issues. Writing a correct native nginx module will require
>>> turning rgw process model into event driven, which is not going to be
>>> easy.
>>>
>>
>> nginx module is nice thing.
>
> It would be nice to have some concrete numbers as to where apache or
> nginx with fastcgi holding us back, and how a dedicated module is
> going to improve that. As a rule of thumb it is a no brainer, but
> still we want to have a better understanding of the situation before
> we dive into such a project.

Now it's hard to say, beacause for now there is now intensive traffic
@ this cluster. I have only synthetic test. Reads from one machine in
cluster are about 500req/s (no nginx cache) via nginx <-> radosgw
fcgi, but machine can handle more (cpu, ram, drives). I need to test
that how it's realy holding this a whole service. With cache enabled
in nginx, reads operation are no issue, but writes (PUTs, DELETEs)
will always be problematic, but now radosgw have improved with this a
lot.

>
>>
>>> 6. Improve garbage collection
>>>
>>> Currently rgw generates intent logs for garabage removal that require
>>> running an external tool later, which is an administrative pain. We
>>> can implement other solutions (OSD side garbage collection,
>>> integrating cleanup process into the gateway, etc.) but we need to
>>> understand the priority.
>>
>> crontab can handle this task for now, but in big workload, better if
>> it's integrated, like scrub, and tuned via conf
>
> Yeah. One of the original ideas was to leverage scrubbing for objects
> expiration (issue #1994). The discussion never converged, as the devil
> is as always in the details. We can revive that discussion.
>
>>
>>>
>>> 7. libradosgw
>>>
>>> We have had this in mind for some time now. Creating a programming api
>>> for rgw, not too different from librados and librbd. It'll hopefully
>>> make code much cleaner. It will allow users to write different front
>>> ends for the rgw backend, and it will make it easier for users to
>>> write applications that interact with the backend, e.g., do processing
>>> on objects that users uploaded, FUSE for rgw without S3 as an
>>> intermediate, etc.
>>>
>>> 8. Administration tools improvement
>>>
>>> We can always do better there.
>>>
>>> 9. Other ideas?
>>
>> - I would like to see a feature that can make replication between
>> clusters. For start 2 clusters. It's a very good feature, when you
>> have two datacenters, and repliacation goes via hight speed link, but
>> aplications at top of clusters does not need to handle this tasks, and
>> data are consistent.
>
> This is a more generic ceph issue. We do need, however, to be able to
> support multiple clusters in rgw. Opened issue #2460.

For rbd is nice feature too :)

>
>>
>> - gracefull upgrade
>
> Can you elaborate on that? This is always our intention, we've put
> some mechanisms in place to help with that, although it is not always
> possible.

How to upgrade cluster without any down ?? How to upgrade cluster with
new features that change logic with 100% uptime ?? Yes i understand
that sometimes, big changes need restart, a whole cluster at once, but
in many situations, info about that before upgrade, or tool that tell
administrator, that this will be needed it's important. Tests and
integration env before upgrade is helpful, but maybe there is a way to
make upgrade more smooth and secure.

>
>>
>> - reload cluster config without restart daemons - or maybe this exist
>> right now ??
>>
>
> I opened issue #2459.


>
> Thanks,
> Yehuda



-- 
-----
Pozdrawiam

Sławek "sZiBis" Skowron
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: RGW, future directions

[Henry C Chang]

2012/5/23 Yehuda Sadeh <yehuda@inktank.com>:
> RGW is maturing. Beside looking at performance, which highly ties into
> RADOS performance, we'd like to hear whether there are certain pain
> points or future directions that you (you as in the ceph community)
> would like to see us taking.
>
> There are a few directions that we were thinking about:
>
> 1. Extend Object Storage API
>
> Swift and S3 has some features that we don't currently support. We can
> certainly extend our functionality, however, is there any demand for
> more features? E.g., self destructing objects, web site, user logs,
> etc.
>

I would like to see radosgw supporting:
- Object versioning
- Bucket policy

> 2. Better OpenStack interoperability
>
> Keystone support? Other?
>
> 3. New features
>
> Some examples:
>
>  - multitenancy: api for domains and user management
>  - snapshots
>  - computation front end: upload object, then do some data
> transformation/calculation.
>  - simple key-value api
>
> 4. CDMI
>
> Sage brought up the CDMI support question to ceph-devel, and I don't
> remember him getting any response. Is there any intereset in CDMI?
>
>
> 5. Native apache/nginx module or embedded web server
>
> We still need to prove that the web server is a bottleneck, or poses
> scaling issues. Writing a correct native nginx module will require
> turning rgw process model into event driven, which is not going to be
> easy.
>
> 6. Improve garbage collection
>
> Currently rgw generates intent logs for garabage removal that require
> running an external tool later, which is an administrative pain. We
> can implement other solutions (OSD side garbage collection,
> integrating cleanup process into the gateway, etc.) but we need to
> understand the priority.
>
> 7. libradosgw
>
> We have had this in mind for some time now. Creating a programming api
> for rgw, not too different from librados and librbd. It'll hopefully
> make code much cleaner. It will allow users to write different front
> ends for the rgw backend, and it will make it easier for users to
> write applications that interact with the backend, e.g., do processing
> on objects that users uploaded, FUSE for rgw without S3 as an
> intermediate, etc.
>

I am also interested in libradosgw. With it, we can support other
protocols easily. (e.g., webdav)


> 8. Administration tools improvement
>
> We can always do better there.
>
> 9. Other ideas?
>
>
> Any comments are welcome!
>
> Thanks,
> Yehuda
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: RGW, future directions

[Kiran Patil]

Hello,

How about this CloudFS management system ?

The CloudFS management system consists of two parts: a very simple
web-based management daemon called cloudfsd, and scripts to perform
various discrete functions.

http://git.fedorahosted.org/git/?p=CloudFS.git;a=summary

http://git.fedorahosted.org/git/?p=CloudFS.git;a=blob;f=scripts/README.ssl;h=3547f6b354d8b3455359ea019f6db9ce74862d2e;hb=cloudfsd

I am not sure, whether it can be reused for ceph.

Thanks,
Kiran.

On Tue, May 22, 2012 at 11:37 PM, Yehuda Sadeh <yehuda@inktank.com> wrote:
>
> RGW is maturing. Beside looking at performance, which highly ties into
> RADOS performance, we'd like to hear whether there are certain pain
> points or future directions that you (you as in the ceph community)
> would like to see us taking.
>
> There are a few directions that we were thinking about:
>
> 1. Extend Object Storage API
>
> Swift and S3 has some features that we don't currently support. We can
> certainly extend our functionality, however, is there any demand for
> more features? E.g., self destructing objects, web site, user logs,
> etc.
>
> 2. Better OpenStack interoperability
>
> Keystone support? Other?
>
> 3. New features
>
> Some examples:
>
>  - multitenancy: api for domains and user management
>  - snapshots
>  - computation front end: upload object, then do some data
> transformation/calculation.
>  - simple key-value api
>
> 4. CDMI
>
> Sage brought up the CDMI support question to ceph-devel, and I don't
> remember him getting any response. Is there any intereset in CDMI?
>
>
> 5. Native apache/nginx module or embedded web server
>
> We still need to prove that the web server is a bottleneck, or poses
> scaling issues. Writing a correct native nginx module will require
> turning rgw process model into event driven, which is not going to be
> easy.
>
> 6. Improve garbage collection
>
> Currently rgw generates intent logs for garabage removal that require
> running an external tool later, which is an administrative pain. We
> can implement other solutions (OSD side garbage collection,
> integrating cleanup process into the gateway, etc.) but we need to
> understand the priority.
>
> 7. libradosgw
>
> We have had this in mind for some time now. Creating a programming api
> for rgw, not too different from librados and librbd. It'll hopefully
> make code much cleaner. It will allow users to write different front
> ends for the rgw backend, and it will make it easier for users to
> write applications that interact with the backend, e.g., do processing
> on objects that users uploaded, FUSE for rgw without S3 as an
> intermediate, etc.
>
> 8. Administration tools improvement
>
> We can always do better there.
>
> 9. Other ideas?
>
>
> Any comments are welcome!
>
> Thanks,
> Yehuda
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: RGW, future directions

[Kiran Patil]

Sorry I missed to send this link.

http://git.fedorahosted.org/git/?p=CloudFS.git;a=blob;f=doc/mgmt_manual.md;h=bfcbbe9769f8726ecd1aefcf19e1159074971110;hb=HEAD

On Wed, May 23, 2012 at 3:47 PM, Kiran Patil <kirantpatil@gmail.com> wrote:
> Hello,
>
> How about this CloudFS management system ?
>
> The CloudFS management system consists of two parts: a very simple
> web-based management daemon called cloudfsd, and scripts to perform
> various discrete functions.
>
> http://git.fedorahosted.org/git/?p=CloudFS.git;a=summary
>
> http://git.fedorahosted.org/git/?p=CloudFS.git;a=blob;f=scripts/README.ssl;h=3547f6b354d8b3455359ea019f6db9ce74862d2e;hb=cloudfsd
>
> I am not sure, whether it can be reused for ceph.
>
> Thanks,
> Kiran.
>
> On Tue, May 22, 2012 at 11:37 PM, Yehuda Sadeh <yehuda@inktank.com> wrote:
>>
>> RGW is maturing. Beside looking at performance, which highly ties into
>> RADOS performance, we'd like to hear whether there are certain pain
>> points or future directions that you (you as in the ceph community)
>> would like to see us taking.
>>
>> There are a few directions that we were thinking about:
>>
>> 1. Extend Object Storage API
>>
>> Swift and S3 has some features that we don't currently support. We can
>> certainly extend our functionality, however, is there any demand for
>> more features? E.g., self destructing objects, web site, user logs,
>> etc.
>>
>> 2. Better OpenStack interoperability
>>
>> Keystone support? Other?
>>
>> 3. New features
>>
>> Some examples:
>>
>>  - multitenancy: api for domains and user management
>>  - snapshots
>>  - computation front end: upload object, then do some data
>> transformation/calculation.
>>  - simple key-value api
>>
>> 4. CDMI
>>
>> Sage brought up the CDMI support question to ceph-devel, and I don't
>> remember him getting any response. Is there any intereset in CDMI?
>>
>>
>> 5. Native apache/nginx module or embedded web server
>>
>> We still need to prove that the web server is a bottleneck, or poses
>> scaling issues. Writing a correct native nginx module will require
>> turning rgw process model into event driven, which is not going to be
>> easy.
>>
>> 6. Improve garbage collection
>>
>> Currently rgw generates intent logs for garabage removal that require
>> running an external tool later, which is an administrative pain. We
>> can implement other solutions (OSD side garbage collection,
>> integrating cleanup process into the gateway, etc.) but we need to
>> understand the priority.
>>
>> 7. libradosgw
>>
>> We have had this in mind for some time now. Creating a programming api
>> for rgw, not too different from librados and librbd. It'll hopefully
>> make code much cleaner. It will allow users to write different front
>> ends for the rgw backend, and it will make it easier for users to
>> write applications that interact with the backend, e.g., do processing
>> on objects that users uploaded, FUSE for rgw without S3 as an
>> intermediate, etc.
>>
>> 8. Administration tools improvement
>>
>> We can always do better there.
>>
>> 9. Other ideas?
>>
>>
>> Any comments are welcome!
>>
>> Thanks,
>> Yehuda
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: RGW, future directions

[Xiaopong Tran]

>
>I would like to see radosgw supporting:
>- Object versioning
>- Bucket policy
>

+1 for these.

>>
>> 7. libradosgw
>>

Yes, please :)

Cheers,

Xiaopong

Re: RGW, future directions

[Xiaopong Tran]

>
>I would like to see radosgw supporting:
>- Object versioning
>- Bucket policy
>

+1 for these.

>>
>> 7. libradosgw
>>

Yes, please :)

Cheers,

Xiaopong

Re: RGW, future directions

[Guilhem LETTRON]

Hi,

2012/5/22 Yehuda Sadeh <yehuda@inktank.com>:
> RGW is maturing. Beside looking at performance, which highly ties into
> RADOS performance, we'd like to hear whether there are certain pain
> points or future directions that you (you as in the ceph community)
> would like to see us taking.
>
> There are a few directions that we were thinking about:

> 3. New features
>
> Some examples:
>
>  - multitenancy: api for domains and user management
>  - snapshots
>  - computation front end: upload object, then do some data
> transformation/calculation.
>  - simple key-value api

+1 any new feature can be a new solution for futur needs.

> 5. Native apache/nginx module or embedded web server
>
> We still need to prove that the web server is a bottleneck, or poses
> scaling issues. Writing a correct native nginx module will require
> turning rgw process model into event driven, which is not going to be
> easy.

It is, for me the #1 feature :
- I only use Nginx for everythings... instead radosgw
- another process / another deamon / another problem
- resolve once for all the "buffering" problem : only apache/fastcgi
can send without buffering. And when you send BIG files, it's not an
option.

> 8. Administration tools improvement
>
> We can always do better there.

Agree, bucket / pool management is quite an ordeal.
It come (of course) with a better documentation.

> Thanks,
> Yehuda

Thanks,
Guilhem Lettron
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: RGW, future directions

[Yehuda Sadeh]

On Wed, May 23, 2012 at 1:59 AM, Henry C Chang <henry.cy.chang@gmail.com> wrote:
> 2012/5/23 Yehuda Sadeh <yehuda@inktank.com>:
>> RGW is maturing. Beside looking at performance, which highly ties into
>> RADOS performance, we'd like to hear whether there are certain pain
>> points or future directions that you (you as in the ceph community)
>> would like to see us taking.
>>
>> There are a few directions that we were thinking about:
>>
>> 1. Extend Object Storage API
>>
>> Swift and S3 has some features that we don't currently support. We can
>> certainly extend our functionality, however, is there any demand for
>> more features? E.g., self destructing objects, web site, user logs,
>> etc.
>>
>
> I would like to see radosgw supporting:
> - Object versioning

Yeah, we had that in mind for while now. Ceph issue #418. We have made
some progress towards that, but we're not there yet.

> - Bucket policy

Anything specific that you'd like to see there? Bucket policy is very
tied into amazon aws, and I don't think we should clone that. We can,
however, look at what additional useful functionality we can have.

>> 7. libradosgw
>>
>> We have had this in mind for some time now. Creating a programming api
>> for rgw, not too different from librados and librbd. It'll hopefully
>> make code much cleaner. It will allow users to write different front
>> ends for the rgw backend, and it will make it easier for users to
>> write applications that interact with the backend, e.g., do processing
>> on objects that users uploaded, FUSE for rgw without S3 as an
>> intermediate, etc.
>>
>
> I am also interested in libradosgw. With it, we can support other
> protocols easily. (e.g., webdav)
>


Thanks,
Yehuda

Re: RGW, future directions

[Yehuda Sadeh]

On Wed, May 23, 2012 at 3:17 AM, Kiran Patil <kirantpatil@gmail.com> wrote:
> Hello,
>
> How about this CloudFS management system ?
>
> The CloudFS management system consists of two parts: a very simple
> web-based management daemon called cloudfsd, and scripts to perform
> various discrete functions.
>
> http://git.fedorahosted.org/git/?p=CloudFS.git;a=summary
>
> http://git.fedorahosted.org/git/?p=CloudFS.git;a=blob;f=scripts/README.ssl;h=3547f6b354d8b3455359ea019f6db9ce74862d2e;hb=cloudfsd
>
> I am not sure, whether it can be reused for ceph.
>

Not sure either. However, the idea of having management API may ease
any such integration.

Thanks,
Yehuda
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: RGW, future directions

[Wido den Hollander]

On 22-05-12 20:07, Yehuda Sadeh wrote:
> RGW is maturing. Beside looking at performance, which highly ties into
> RADOS performance, we'd like to hear whether there are certain pain
> points or future directions that you (you as in the ceph community)
> would like to see us taking.
>
> There are a few directions that we were thinking about:
>
> 1. Extend Object Storage API
>
> Swift and S3 has some features that we don't currently support. We can
> certainly extend our functionality, however, is there any demand for
> more features? E.g., self destructing objects, web site, user logs,
> etc.
>
> 2. Better OpenStack interoperability
>
> Keystone support? Other?
>
> 3. New features
>
> Some examples:
>
>   - multitenancy: api for domains and user management
>   - snapshots
>   - computation front end: upload object, then do some data
> transformation/calculation.
>   - simple key-value api
>
> 4. CDMI
>
> Sage brought up the CDMI support question to ceph-devel, and I don't
> remember him getting any response. Is there any intereset in CDMI?
>
>
> 5. Native apache/nginx module or embedded web server
>
> We still need to prove that the web server is a bottleneck, or poses
> scaling issues. Writing a correct native nginx module will require
> turning rgw process model into event driven, which is not going to be
> easy.
>

I'd not go for a native nginx or Apache module, that would bring extra C 
code into the story which would mean extra dependencies.

My vote would still go to a embedded webserver written in something like 
Python. You could then use Apache/nginx/Varnish as a reverse proxy in 
front and do all kinds of cool stuff.

You could even doing caching in nginx or Varnish and let the RGW notify 
those proxy's when an object has changed so they can purge their cache. 
This would dramatically improve the performance of the gateway.

It would also simplify the code, why try to do caching on your own when 
some great HTTP caches are out there?

> 6. Improve garbage collection
>
> Currently rgw generates intent logs for garabage removal that require
> running an external tool later, which is an administrative pain. We
> can implement other solutions (OSD side garbage collection,
> integrating cleanup process into the gateway, etc.) but we need to
> understand the priority.
>
> 7. libradosgw
>
> We have had this in mind for some time now. Creating a programming api
> for rgw, not too different from librados and librbd. It'll hopefully
> make code much cleaner. It will allow users to write different front
> ends for the rgw backend, and it will make it easier for users to
> write applications that interact with the backend, e.g., do processing
> on objects that users uploaded, FUSE for rgw without S3 as an
> intermediate, etc.
>

Yes, I would really like this. Combine this with the Python 
stand-alone/embedded webserver I proposed and you get a really nice RGW 
I think.

> 8. Administration tools improvement
>
> We can always do better there.
>

When we have libradosgw it wouldn't be that hard to make a nice web 
front-end where you can manage the whole thing.

> 9. Other ideas?
>
>
> Any comments are welcome!
>
> Thanks,
> Yehuda
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: RGW, future directions

[Henry C Chang]

2012/5/24 Yehuda Sadeh <yehuda@inktank.com>:
> On Wed, May 23, 2012 at 1:59 AM, Henry C Chang <henry.cy.chang@gmail.com> wrote:
>> 2012/5/23 Yehuda Sadeh <yehuda@inktank.com>:
>>> RGW is maturing. Beside looking at performance, which highly ties into
>>> RADOS performance, we'd like to hear whether there are certain pain
>>> points or future directions that you (you as in the ceph community)
>>> would like to see us taking.
>>>
>>> There are a few directions that we were thinking about:
>>>
>>> 1. Extend Object Storage API
>>>
>>> Swift and S3 has some features that we don't currently support. We can
>>> certainly extend our functionality, however, is there any demand for
>>> more features? E.g., self destructing objects, web site, user logs,
>>> etc.
>>>
>>
>> I would like to see radosgw supporting:
>> - Object versioning
>
> Yeah, we had that in mind for while now. Ceph issue #418. We have made
> some progress towards that, but we're not there yet.
>
>> - Bucket policy
>
> Anything specific that you'd like to see there? Bucket policy is very
> tied into amazon aws, and I don't think we should clone that. We can,
> however, look at what additional useful functionality we can have.
>

To me, one of the most powerful thing of bucket policy is: it allows
you to set permissions for a large number of objects with one
statement. Below is a simple example. You can specify the "Resource"
with a wildcard (or even <bucket>/<obj_prefix>*).

{
  "Version":"2008-10-17",
  "Statement":[{
	"Sid":"AddPerm",
        "Effect":"Allow",
	  "Principal": {
            "AWS": "*"
         },
      "Action":["s3:GetObject"],
      "Resource":["arn:aws:s3:::bucket/*"
      ]
    }
  ]
}

I agree, we don't have to clone that. Similar functionality would be great!!


>>> 7. libradosgw
>>>
>>> We have had this in mind for some time now. Creating a programming api
>>> for rgw, not too different from librados and librbd. It'll hopefully
>>> make code much cleaner. It will allow users to write different front
>>> ends for the rgw backend, and it will make it easier for users to
>>> write applications that interact with the backend, e.g., do processing
>>> on objects that users uploaded, FUSE for rgw without S3 as an
>>> intermediate, etc.
>>>
>>
>> I am also interested in libradosgw. With it, we can support other
>> protocols easily. (e.g., webdav)
>>
>
>
> Thanks,
> Yehuda

Re: RGW, future directions

[Sławomir Skowron]

On Thu, May 24, 2012 at 7:15 AM, Wido den Hollander <wido@widodh.nl> wrote:
>
>
> On 22-05-12 20:07, Yehuda Sadeh wrote:
>>
>> RGW is maturing. Beside looking at performance, which highly ties into
>> RADOS performance, we'd like to hear whether there are certain pain
>> points or future directions that you (you as in the ceph community)
>> would like to see us taking.
>>
>> There are a few directions that we were thinking about:
>>
>> 1. Extend Object Storage API
>>
>> Swift and S3 has some features that we don't currently support. We can
>> certainly extend our functionality, however, is there any demand for
>> more features? E.g., self destructing objects, web site, user logs,
>> etc.
>>
>> 2. Better OpenStack interoperability
>>
>> Keystone support? Other?
>>
>> 3. New features
>>
>> Some examples:
>>
>>  - multitenancy: api for domains and user management
>>  - snapshots
>>  - computation front end: upload object, then do some data
>> transformation/calculation.
>>  - simple key-value api
>>
>> 4. CDMI
>>
>> Sage brought up the CDMI support question to ceph-devel, and I don't
>> remember him getting any response. Is there any intereset in CDMI?
>>
>>
>> 5. Native apache/nginx module or embedded web server
>>
>> We still need to prove that the web server is a bottleneck, or poses
>> scaling issues. Writing a correct native nginx module will require
>> turning rgw process model into event driven, which is not going to be
>> easy.
>>
>
> I'd not go for a native nginx or Apache module, that would bring extra C
> code into the story which would mean extra dependencies.
>
> My vote would still go to a embedded webserver written in something like
> Python. You could then use Apache/nginx/Varnish as a reverse proxy in front
> and do all kinds of cool stuff.
>
> You could even doing caching in nginx or Varnish and let the RGW notify
> those proxy's when an object has changed so they can purge their cache. This
> would dramatically improve the performance of the gateway.
>
> It would also simplify the code, why try to do caching on your own when some
> great HTTP caches are out there?
>

100% +1

>
>> 6. Improve garbage collection
>>
>> Currently rgw generates intent logs for garabage removal that require
>> running an external tool later, which is an administrative pain. We
>> can implement other solutions (OSD side garbage collection,
>> integrating cleanup process into the gateway, etc.) but we need to
>> understand the priority.
>>
>> 7. libradosgw
>>
>> We have had this in mind for some time now. Creating a programming api
>> for rgw, not too different from librados and librbd. It'll hopefully
>> make code much cleaner. It will allow users to write different front
>> ends for the rgw backend, and it will make it easier for users to
>> write applications that interact with the backend, e.g., do processing
>> on objects that users uploaded, FUSE for rgw without S3 as an
>> intermediate, etc.
>>
>
> Yes, I would really like this. Combine this with the Python
> stand-alone/embedded webserver I proposed and you get a really nice RGW I
> think.
>
>
>> 8. Administration tools improvement
>>
>> We can always do better there.
>>
>
> When we have libradosgw it wouldn't be that hard to make a nice web
> front-end where you can manage the whole thing.
>
>
>> 9. Other ideas?
>>
>>
>> Any comments are welcome!
>>
>> Thanks,
>> Yehuda
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
-----
Pozdrawiam

Sławek "sZiBis" Skowron
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html