Computation in the storage

Computation in the storage

[nebil.benmabrouk]

Hello,

I  am interested in embedding some specific data processing in the storage close to the data.
I read that, we can integrate some code (e.g., java classes) in Ceph, but I find no documentation about it. 
Can you give some information or documents that help me to achieve this objective.

Best regards,
Nebil Ben Mabrouk

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

Re: Computation in the storage

[Joao Eduardo Luis]

On 12/02/2016 11:03 AM, nebil.benmabrouk@orange.com wrote:
> Hello,
>
> I  am interested in embedding some specific data processing in the storage close to the data.
> I read that, we can integrate some code (e.g., java classes) in Ceph, but I find no documentation about it.

What you are looking is actually 'rados object classes' (or ceph 
classes, according to the docs), and you may find a brief remark about 
that here

  http://docs.ceph.com/docs/master/architecture/#extending-ceph

which contains pointers to the code you should look into.

Somehow it feels like there should be some more information about this 
somewhere, but I can't really recall where.

   -Joao

Re: Computation in the storage

[Brad Hubbard]

On Fri, Dec 2, 2016 at 9:40 PM, Joao Eduardo Luis <joao@suse.de> wrote:
> On 12/02/2016 11:03 AM, nebil.benmabrouk@orange.com wrote:
>>
>> Hello,
>>
>> I  am interested in embedding some specific data processing in the storage
>> close to the data.
>> I read that, we can integrate some code (e.g., java classes) in Ceph, but
>> I find no documentation about it.
>
>
> What you are looking is actually 'rados object classes' (or ceph classes,
> according to the docs), and you may find a brief remark about that here
>
>  http://docs.ceph.com/docs/master/architecture/#extending-ceph
>
> which contains pointers to the code you should look into.
>
> Somehow it feels like there should be some more information about this
> somewhere, but I can't really recall where.

Hmmm... that needs updating. Any takers?

Look at the following.

src/objclass/objclass.h
src/test/fooclass.cc
src/test/barclass.cc

All the code below src/cls/, especially "hello".

>
>   -Joao
>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Cheers,
Brad

Re: Computation in the storage

[Kamble, Nitin A]

> On Dec 2, 2016, at 3:49 AM, Brad Hubbard <bhubbard@redhat.com> wrote:
> 
> On Fri, Dec 2, 2016 at 9:40 PM, Joao Eduardo Luis <joao@suse.de> wrote:
>> On 12/02/2016 11:03 AM, nebil.benmabrouk@orange.com wrote:
>>> 
>>> Hello,
>>> 
>>> I  am interested in embedding some specific data processing in the storage
>>> close to the data.
>>> I read that, we can integrate some code (e.g., java classes) in Ceph, but
>>> I find no documentation about it.
>> 
>> 
>> What you are looking is actually 'rados object classes' (or ceph classes,
>> according to the docs), and you may find a brief remark about that here
>> 
>> http://docs.ceph.com/docs/master/architecture/#extending-ceph
>> 

Is anybody using real RADOS object classes on ceph clusters? Any of them on github?
I would like to look at some of the real world examples if it is there.
And more documentation of this feature would definitely help too.

Thanks,
Nitin

>> which contains pointers to the code you should look into.
>> 
>> Somehow it feels like there should be some more information about this
>> somewhere, but I can't really recall where.
> 
> Hmmm... that needs updating. Any takers?
> 
> Look at the following.
> 
> src/objclass/objclass.h
> src/test/fooclass.cc
> src/test/barclass.cc
> 
> All the code below src/cls/, especially "hello".
> 
>> 
>>  -Joao
>> 
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 
> 
> -- 
> Cheers,
> Brad
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Computation in the storage

[Brad Hubbard]

On Sat, Dec 3, 2016 at 9:04 AM, Kamble, Nitin A <Nitin.Kamble@teradata.com> wrote:
>
>> On Dec 2, 2016, at 3:49 AM, Brad Hubbard <bhubbard@redhat.com> wrote:
>>
>> On Fri, Dec 2, 2016 at 9:40 PM, Joao Eduardo Luis <joao@suse.de> wrote:
>>> On 12/02/2016 11:03 AM, nebil.benmabrouk@orange.com wrote:
>>>>
>>>> Hello,
>>>>
>>>> I  am interested in embedding some specific data processing in the storage
>>>> close to the data.
>>>> I read that, we can integrate some code (e.g., java classes) in Ceph, but
>>>> I find no documentation about it.
>>>
>>>
>>> What you are looking is actually 'rados object classes' (or ceph classes,
>>> according to the docs), and you may find a brief remark about that here
>>>
>>> http://docs.ceph.com/docs/master/architecture/#extending-ceph
>>>
>
> Is anybody using real RADOS object classes on ceph clusters? Any of them on github?
> I would like to look at some of the real world examples if it is there.

Most of what's in src/cls/ are real world examples, although not third party
which I know is what you meant but I'll let others comment on those.

If you "ls /usr/lib64/rados-classes/libcls_*" (by default) you'll see the
classes being loaded by the ceph osds when they start.

> And more documentation of this feature would definitely help too.
>
> Thanks,
> Nitin
>
>>> which contains pointers to the code you should look into.
>>>
>>> Somehow it feels like there should be some more information about this
>>> somewhere, but I can't really recall where.
>>
>> Hmmm... that needs updating. Any takers?
>>
>> Look at the following.
>>
>> src/objclass/objclass.h
>> src/test/fooclass.cc
>> src/test/barclass.cc
>>
>> All the code below src/cls/, especially "hello".
>>
>>>
>>>  -Joao
>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>>
>>
>> --
>> Cheers,
>> Brad
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>



-- 
Cheers,
Brad

Re: Computation in the storage

[Noah Watkins]

On Fri, Dec 2, 2016 at 3:04 PM, Kamble, Nitin A
<Nitin.Kamble@teradata.com> wrote:
>
> Is anybody using real RADOS object classes on ceph clusters? Any of them on github?
> I would like to look at some of the real world examples if it is there.
> And more documentation of this feature would definitely help too.

We use object classes [0] for a distributed shared log [1] on Ceph.

[0]: https://github.com/noahdesu/ceph/blob/zlog/jewel/src/cls/zlog/cls_zlog.cc
[1]: https://github.com/noahdesu/zlog

Re: Computation in the storage

[Kamble, Nitin A]

> On Dec 4, 2016, at 3:02 PM, Noah Watkins <noahwatkins@gmail.com> wrote:
> 
> On Fri, Dec 2, 2016 at 3:04 PM, Kamble, Nitin A
> <Nitin.Kamble@teradata.com> wrote:
>> 
>> Is anybody using real RADOS object classes on ceph clusters? Any of them on github?
>> I would like to look at some of the real world examples if it is there.
>> And more documentation of this feature would definitely help too.
> 
> We use object classes [0] for a distributed shared log [1] on Ceph.
> 
> [0]: https://github.com/noahdesu/ceph/blob/zlog/jewel/src/cls/zlog/cls_zlog.cc
> [1]: https://github.com/noahdesu/zlog


Thank you Noah & Brad for your pointers for sample Ceph classes. 

Nitin

Re: Computation in the storage

[Kamble, Nitin A]

Hi Noah,

I looked at your blog [0] about implementing RADOS object classes using Lua scripts. The
link [1] to the source code in ceph repository is missing. Seems like this support has been
removed from upstream ceph, and it is not supported by the ceph releases as of now.
Can you confirm?


[0] http://noahdesu.github.io/2015/12/14/towards-rados-object-class-management.html
[1] https://github.com/ceph/ceph/tree/cls-lua

Thanks,
Nitin


> On Dec 5, 2016, at 3:08 PM, Kamble, Nitin A <Nitin.Kamble@Teradata.com> wrote:
> 
> 
>> On Dec 4, 2016, at 3:02 PM, Noah Watkins <noahwatkins@gmail.com> wrote:
>> 
>> On Fri, Dec 2, 2016 at 3:04 PM, Kamble, Nitin A
>> <Nitin.Kamble@teradata.com> wrote:
>>> 
>>> Is anybody using real RADOS object classes on ceph clusters? Any of them on github?
>>> I would like to look at some of the real world examples if it is there.
>>> And more documentation of this feature would definitely help too.
>> 
>> We use object classes [0] for a distributed shared log [1] on Ceph.
>> 
>> [0]: https://github.com/noahdesu/ceph/blob/zlog/jewel/src/cls/zlog/cls_zlog.cc
>> [1]: https://github.com/noahdesu/zlog

Re: Computation in the storage

[Noah Watkins]

Nitin,

The links on the blog are out of date. Lua support has been merged.
See https://github.com/ceph/ceph/tree/master/src/cls/lua

- Noah

On Fri, Dec 16, 2016 at 10:00 AM, Kamble, Nitin A
<Nitin.Kamble@teradata.com> wrote:
> Hi Noah,
>
> I looked at your blog [0] about implementing RADOS object classes using Lua scripts. The
> link [1] to the source code in ceph repository is missing. Seems like this support has been
> removed from upstream ceph, and it is not supported by the ceph releases as of now.
> Can you confirm?
>
>
> [0] http://noahdesu.github.io/2015/12/14/towards-rados-object-class-management.html
> [1] https://github.com/ceph/ceph/tree/cls-lua
>
> Thanks,
> Nitin
>
>
>> On Dec 5, 2016, at 3:08 PM, Kamble, Nitin A <Nitin.Kamble@Teradata.com> wrote:
>>
>>
>>> On Dec 4, 2016, at 3:02 PM, Noah Watkins <noahwatkins@gmail.com> wrote:
>>>
>>> On Fri, Dec 2, 2016 at 3:04 PM, Kamble, Nitin A
>>> <Nitin.Kamble@teradata.com> wrote:
>>>>
>>>> Is anybody using real RADOS object classes on ceph clusters? Any of them on github?
>>>> I would like to look at some of the real world examples if it is there.
>>>> And more documentation of this feature would definitely help too.
>>>
>>> We use object classes [0] for a distributed shared log [1] on Ceph.
>>>
>>> [0]: https://github.com/noahdesu/ceph/blob/zlog/jewel/src/cls/zlog/cls_zlog.cc
>>> [1]: https://github.com/noahdesu/zlog
>

Re: Computation in the storage

[Kamble, Nitin A]

Hi Noah,
Is this the functionality [2] to use osd-local lua object classes in ceph upstream now?

If yes, which release it went in?

Thanks,
Nitin

[2] http://noahdesu.github.io/2015/12/06/load-lua-rados-plugin-from-fs.html

> On Dec 16, 2016, at 10:00 AM, Kamble, Nitin A <Nitin.Kamble@Teradata.com> wrote:
> 
> Hi Noah,
> 
> I looked at your blog [0] about implementing RADOS object classes using Lua scripts. The
> link [1] to the source code in ceph repository is missing. Seems like this support has been
> removed from upstream ceph, and it is not supported by the ceph releases as of now.
> Can you confirm?
> 
> 
> [0] http://noahdesu.github.io/2015/12/14/towards-rados-object-class-management.html
> [1] https://github.com/ceph/ceph/tree/cls-lua
> 
> Thanks,
> Nitin
> 
> 
>> On Dec 5, 2016, at 3:08 PM, Kamble, Nitin A <Nitin.Kamble@Teradata.com> wrote:
>> 
>> 
>>> On Dec 4, 2016, at 3:02 PM, Noah Watkins <noahwatkins@gmail.com> wrote:
>>> 
>>> On Fri, Dec 2, 2016 at 3:04 PM, Kamble, Nitin A
>>> <Nitin.Kamble@teradata.com> wrote:
>>>> 
>>>> Is anybody using real RADOS object classes on ceph clusters? Any of them on github?
>>>> I would like to look at some of the real world examples if it is there.
>>>> And more documentation of this feature would definitely help too.
>>> 
>>> We use object classes [0] for a distributed shared log [1] on Ceph.
>>> 
>>> [0]: https://github.com/noahdesu/ceph/blob/zlog/jewel/src/cls/zlog/cls_zlog.cc
>>> [1]: https://github.com/noahdesu/zlog
> 

Re: Computation in the storage

[Noah Watkins]

Lua object classes are supported in Kraken [0], which allows one to
send a Lua script to be run in the OSD locally [1]. The link you
referenced is to a feature that is not in Kraken which will load Lua
scripts from disk rather than requiring them to be attached to each
request (e.g. using Lua scripts in /usr/share/ceph etc...)

[0]: http://ceph.com/releases/kraken-11-0-2-released/
[1]: http://noahdesu.github.io/2014/01/22/dynamic-rados-object-interfaces.html

On Fri, Dec 16, 2016 at 2:46 PM, Kamble, Nitin A
<Nitin.Kamble@teradata.com> wrote:
> Hi Noah,
> Is this the functionality [2] to use osd-local lua object classes in ceph upstream now?
>
> If yes, which release it went in?
>
> Thanks,
> Nitin
>
> [2] http://noahdesu.github.io/2015/12/06/load-lua-rados-plugin-from-fs.html
>
>> On Dec 16, 2016, at 10:00 AM, Kamble, Nitin A <Nitin.Kamble@Teradata.com> wrote:
>>
>> Hi Noah,
>>
>> I looked at your blog [0] about implementing RADOS object classes using Lua scripts. The
>> link [1] to the source code in ceph repository is missing. Seems like this support has been
>> removed from upstream ceph, and it is not supported by the ceph releases as of now.
>> Can you confirm?
>>
>>
>> [0] http://noahdesu.github.io/2015/12/14/towards-rados-object-class-management.html
>> [1] https://github.com/ceph/ceph/tree/cls-lua
>>
>> Thanks,
>> Nitin
>>
>>
>>> On Dec 5, 2016, at 3:08 PM, Kamble, Nitin A <Nitin.Kamble@Teradata.com> wrote:
>>>
>>>
>>>> On Dec 4, 2016, at 3:02 PM, Noah Watkins <noahwatkins@gmail.com> wrote:
>>>>
>>>> On Fri, Dec 2, 2016 at 3:04 PM, Kamble, Nitin A
>>>> <Nitin.Kamble@teradata.com> wrote:
>>>>>
>>>>> Is anybody using real RADOS object classes on ceph clusters? Any of them on github?
>>>>> I would like to look at some of the real world examples if it is there.
>>>>> And more documentation of this feature would definitely help too.
>>>>
>>>> We use object classes [0] for a distributed shared log [1] on Ceph.
>>>>
>>>> [0]: https://github.com/noahdesu/ceph/blob/zlog/jewel/src/cls/zlog/cls_zlog.cc
>>>> [1]: https://github.com/noahdesu/zlog
>>
>

Re: Computation in the storage

[Kamble, Nitin A]

Hi Noah,

That’s great to know. Thanks for the prompt response.

Nitin

> On Dec 16, 2016, at 10:22 AM, Noah Watkins <noahwatkins@gmail.com> wrote:
> 
> Nitin,
> 
> The links on the blog are out of date. Lua support has been merged.
> See https://github.com/ceph/ceph/tree/master/src/cls/lua
> 
> - Noah
> 
> On Fri, Dec 16, 2016 at 10:00 AM, Kamble, Nitin A
> <Nitin.Kamble@teradata.com> wrote:
>> Hi Noah,
>> 
>> I looked at your blog [0] about implementing RADOS object classes using Lua scripts. The
>> link [1] to the source code in ceph repository is missing. Seems like this support has been
>> removed from upstream ceph, and it is not supported by the ceph releases as of now.
>> Can you confirm?
>> 
>> 
>> [0] http://noahdesu.github.io/2015/12/14/towards-rados-object-class-management.html
>> [1] https://github.com/ceph/ceph/tree/cls-lua
>> 
>> Thanks,
>> Nitin
>> 
>> 
>>> On Dec 5, 2016, at 3:08 PM, Kamble, Nitin A <Nitin.Kamble@Teradata.com> wrote:
>>> 
>>> 
>>>> On Dec 4, 2016, at 3:02 PM, Noah Watkins <noahwatkins@gmail.com> wrote:
>>>> 
>>>> On Fri, Dec 2, 2016 at 3:04 PM, Kamble, Nitin A
>>>> <Nitin.Kamble@teradata.com> wrote:
>>>>> 
>>>>> Is anybody using real RADOS object classes on ceph clusters? Any of them on github?
>>>>> I would like to look at some of the real world examples if it is there.
>>>>> And more documentation of this feature would definitely help too.
>>>> 
>>>> We use object classes [0] for a distributed shared log [1] on Ceph.
>>>> 
>>>> [0]: https://github.com/noahdesu/ceph/blob/zlog/jewel/src/cls/zlog/cls_zlog.cc
>>>> [1]: https://github.com/noahdesu/zlog
>> 

Re: Computation in the storage

[Kamble, Nitin A]

Hi Noah,

That’s great to know. Thanks for the prompt response.

Nitin

> On Dec 16, 2016, at 10:22 AM, Noah Watkins <noahwatkins@gmail.com> wrote:
> 
> Nitin,
> 
> The links on the blog are out of date. Lua support has been merged.
> See https://github.com/ceph/ceph/tree/master/src/cls/lua
> 
> - Noah
> 
> On Fri, Dec 16, 2016 at 10:00 AM, Kamble, Nitin A
> <Nitin.Kamble@teradata.com> wrote:
>> Hi Noah,
>> 
>> I looked at your blog [0] about implementing RADOS object classes using Lua scripts. The
>> link [1] to the source code in ceph repository is missing. Seems like this support has been
>> removed from upstream ceph, and it is not supported by the ceph releases as of now.
>> Can you confirm?
>> 
>> 
>> [0] http://noahdesu.github.io/2015/12/14/towards-rados-object-class-management.html
>> [1] https://github.com/ceph/ceph/tree/cls-lua
>> 
>> Thanks,
>> Nitin
>> 
>> 
>>> On Dec 5, 2016, at 3:08 PM, Kamble, Nitin A <Nitin.Kamble@Teradata.com> wrote:
>>> 
>>> 
>>>> On Dec 4, 2016, at 3:02 PM, Noah Watkins <noahwatkins@gmail.com> wrote:
>>>> 
>>>> On Fri, Dec 2, 2016 at 3:04 PM, Kamble, Nitin A
>>>> <Nitin.Kamble@teradata.com> wrote:
>>>>> 
>>>>> Is anybody using real RADOS object classes on ceph clusters? Any of them on github?
>>>>> I would like to look at some of the real world examples if it is there.
>>>>> And more documentation of this feature would definitely help too.
>>>> 
>>>> We use object classes [0] for a distributed shared log [1] on Ceph.
>>>> 
>>>> [0]: https://github.com/noahdesu/ceph/blob/zlog/jewel/src/cls/zlog/cls_zlog.cc
>>>> [1]: https://github.com/noahdesu/zlog
>> 

RADOS Object class capabilities

[Kamble, Nitin A]

So far I am impressed to find out what the RADOS object class can do.
Is it possible to override one of the RADOS basic functions or it’s behavior such as object 
read/write for all/other clients, by implementing a RADOS object class?

It will be quiet powerful and dangerous if is supported.

Thanks & Regards,
Nitin

Re: RADOS Object class capabilities

[Wido den Hollander]

> Op 9 januari 2017 om 3:52 schreef "Kamble, Nitin A" <Nitin.Kamble@Teradata.com>:
> 
> 
> So far I am impressed to find out what the RADOS object class can do.
> Is it possible to override one of the RADOS basic functions or it’s behavior such as object 
> read/write for all/other clients, by implementing a RADOS object class?
> 

Afaik not. Using rados_exec() you can execute functions from the object class, but a object class don't override existing RADOS functionality.

Wido

> It will be quiet powerful and dangerous if is supported.
> 
> Thanks & Regards,
> Nitin
> 
> N�����r��y���b�X��ǧv�^�)޺{.n�+���z�]z���{ay�\x1dʇڙ�,j\a��f���h���z�\x1e�w���\f���j:+v���w�j�m����\a����zZ+��ݢj"��!�i

Re: RADOS Object class capabilities

[Kamble, Nitin A]

> On Jan 9, 2017, at 12:58 AM, Wido den Hollander <wido@42on.com> wrote:
> 
> 
>> Op 9 januari 2017 om 3:52 schreef "Kamble, Nitin A" <Nitin.Kamble@Teradata.com>:
>> 
>> 
>> So far I am impressed to find out what the RADOS object class can do.
>> Is it possible to override one of the RADOS basic functions or it’s behavior such as object 
>> read/write for all/other clients, by implementing a RADOS object class?
>> 
> 
> Afaik not. Using rados_exec() you can execute functions from the object class, but a object class don't override existing RADOS functionality.
> 
> Wido

Thanks for the precise answer.

Regards,
Nitin

> 
>> It will be quiet powerful and dangerous if is supported.
>> 
>> Thanks & Regards,
>> Nitin
>> 
>> N�����r��y���b�X��ǧv�^�)޺{.n�+���z�]z���{ay�\x1dʇڙ�,j\a��f���h���z�\x1e�w���\f���j:+v���w�j�m����\a����zZ+��ݢj"��!�i