Mojette Transform implementation

Mojette Transform implementation

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 1277 bytes --]

Hi Pierre,

As discussed today over the phone, I'm very much interested in trying to reuse the Mojette Transform implementation in the context of the work related to the implementation of an erasure coded backend in Ceph. The progress can be tracked from http://tracker.ceph.com/issues/5046 and the sub-tickets and the original discussion from the last Ceph summit is at http://wiki.ceph.com/01Planning/02Blueprints/Dumpling/Erasure_encoding_as_a_storage_backend

I was reading the code of rbs_restore_one_rb_entry at
https://github.com/rozofs/rozofs/blob/master/src/storaged/rbs.c#L104
and assumed the implementation is in
https://github.com/rozofs/rozofs/blob/master/src/storaged/rbs_transform.h
https://github.com/rozofs/rozofs/blob/master/src/storaged/rbs_transform.cc

And also tried to understand http://en.wikipedia.org/wiki/Mojette_Transform and bought "The Mojette Transform: Theory and Applications" http://eu.wiley.com/WileyCDA/WileyTitle/productCd-1848210809.html in the hope to get more sense out of it ;-)

Is there a part of the rozofs code you would advise me to read first in order to understand how it works ?

Cheers

-- 
Loïc Dachary, Artisan Logiciel Libre
All that is necessary for the triumph of evil is that good people do nothing.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: Mojette Transform implementation

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 2349 bytes --]

Hi Pierre,

I took a look at https://github.com/rozofs/rozofs/blob/master/src/rozofsmount/rozofsmount.c#L1557
and it gave me a high level view of the fuse bindings of rozofs, thanks for this :-)

I'm now looking into the functions that could be wrapped into a library implementing Mojette Transform so that it could be used by rozofs and ceph. Which file would you recommend I study for bootstraping this effort ?

Cheers


On 05/29/2013 05:23 PM, Pierre Evenou wrote:
> 2013/5/29 Loic Dachary <loic@dachary.org <mailto:loic@dachary.org>>
> 
>     Hi Pierre,
> 
>     As discussed today over the phone, I'm very much interested in trying to reuse the Mojette Transform implementation in the context of the work related to the implementation of an erasure coded backend in Ceph. The progress can be tracked from http://tracker.ceph.com/issues/5046 and the sub-tickets and the original discussion from the last Ceph summit is at http://wiki.ceph.com/01Planning/02Blueprints/Dumpling/Erasure_encoding_as_a_storage_backend
> 
> 
> Thanks for the links.
>  
> 
>     I was reading the code of rbs_restore_one_rb_entry at
>     https://github.com/rozofs/rozofs/blob/master/src/storaged/rbs.c#L104
>     and assumed the implementation is in
>     https://github.com/rozofs/rozofs/blob/master/src/storaged/rbs_transform.h
>     https://github.com/rozofs/rozofs/blob/master/src/storaged/rbs_transform.cc
> 
>     And also tried to understand http://en.wikipedia.org/wiki/Mojette_Transform and bought "The Mojette Transform: Theory and Applications" http://eu.wiley.com/WileyCDA/WileyTitle/productCd-1848210809.html in the hope to get more sense out of it ;-)
> 
>     Is there a part of the rozofs code you would advise me to read first in order to understand how it works ?
> 
> 
> I would recommend to have a look to rozofsmount.c witch is a key part in RozoFS.
>  
> 
>     Cheers
> 
>     --
>     Loïc Dachary, Artisan Logiciel Libre
>     All that is necessary for the triumph of evil is that good people do nothing.
> 
> 
> 
> 
> -- 
> Pierre Evenou│+33.7.60.09.08.80│pierre.evenou@rozofs.com <mailto:pierre.evenou@rozofs.com>│www.rozofs.com <http://www.rozofs.com/>

-- 
Loïc Dachary, Artisan Logiciel Libre
All that is necessary for the triumph of evil is that good people do nothing.




[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: Mojette Transform implementation

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 3078 bytes --]

Hi Pierre,

Thanks for the pointer to
https://github.com/rozofs/rozofs/blob/master/rozofs/common/transform.c
https://github.com/rozofs/rozofs/blob/master/rozofs/common/transform.h
If I understand correctly these are the core functions implementing the algorithm. Would you advise to expose these three functions so that a third party software such as ceph use it ? Or are there more high level functions that are easier to use to do something like:

 encode(char* buffer_in, char* buffer_out[])
 decode(char* buffer_in[], char* buffer_out) 
 repair(char* buffer_in[], char* buffer_out, int indices_of_damaged_buffers[])

Cheers

On 05/30/2013 05:49 PM, Loic Dachary wrote:
> Hi Pierre,
> 
> I took a look at https://github.com/rozofs/rozofs/blob/master/src/rozofsmount/rozofsmount.c#L1557
> and it gave me a high level view of the fuse bindings of rozofs, thanks for this :-)
> 
> I'm now looking into the functions that could be wrapped into a library implementing Mojette Transform so that it could be used by rozofs and ceph. Which file would you recommend I study for bootstraping this effort ?
> 
> Cheers
> 
> 
> On 05/29/2013 05:23 PM, Pierre Evenou wrote:
>> 2013/5/29 Loic Dachary <loic@dachary.org <mailto:loic@dachary.org>>
>>
>>     Hi Pierre,
>>
>>     As discussed today over the phone, I'm very much interested in trying to reuse the Mojette Transform implementation in the context of the work related to the implementation of an erasure coded backend in Ceph. The progress can be tracked from http://tracker.ceph.com/issues/5046 and the sub-tickets and the original discussion from the last Ceph summit is at http://wiki.ceph.com/01Planning/02Blueprints/Dumpling/Erasure_encoding_as_a_storage_backend
>>
>>
>> Thanks for the links.
>>  
>>
>>     I was reading the code of rbs_restore_one_rb_entry at
>>     https://github.com/rozofs/rozofs/blob/master/src/storaged/rbs.c#L104
>>     and assumed the implementation is in
>>     https://github.com/rozofs/rozofs/blob/master/src/storaged/rbs_transform.h
>>     https://github.com/rozofs/rozofs/blob/master/src/storaged/rbs_transform.cc
>>
>>     And also tried to understand http://en.wikipedia.org/wiki/Mojette_Transform and bought "The Mojette Transform: Theory and Applications" http://eu.wiley.com/WileyCDA/WileyTitle/productCd-1848210809.html in the hope to get more sense out of it ;-)
>>
>>     Is there a part of the rozofs code you would advise me to read first in order to understand how it works ?
>>
>>
>> I would recommend to have a look to rozofsmount.c witch is a key part in RozoFS.
>>  
>>
>>     Cheers
>>
>>     --
>>     Loïc Dachary, Artisan Logiciel Libre
>>     All that is necessary for the triumph of evil is that good people do nothing.
>>
>>
>>
>>
>> -- 
>> Pierre Evenou│+33.7.60.09.08.80│pierre.evenou@rozofs.com <mailto:pierre.evenou@rozofs.com>│www.rozofs.com <http://www.rozofs.com/>
> 

-- 
Loïc Dachary, Artisan Logiciel Libre
All that is necessary for the triumph of evil is that good people do nothing.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 262 bytes --]