Mojette based Erasure Code is more performant than ISA-L or Jerasure

Mojette based Erasure Code is more performant than ISA-L or Jerasure

[Chandan Kumar Singh]

Hi

The authors of  Mojette Erasure Encoding claim that this EC has higher
performance than ISA-L and Jerasure and can be used for hot storage
use cases. Their EC is used in open source RozoFS. What is your
evaluation of it? It will be nice if it is available as another EC
plugin.

Regards
Chandan

Re: Mojette based Erasure Code is more performant than ISA-L or Jerasure

[Loic Dachary]

Hi,

On 20/04/2016 15:52, Chandan Kumar Singh wrote:
> Hi
> 
> The authors of  Mojette Erasure Encoding claim that this EC has higher
> performance than ISA-L and Jerasure and can be used for hot storage
> use cases. Their EC is used in open source RozoFS. What is your
> evaluation of it? It will be nice if it is available as another EC
> plugin.

RozoFS is no longer developped (last commit was july last year). It would be nice to have an alternative implementation of the same algorithm, well maintained. It's sad to see Free Software being frozen. But Free Software is never dead, all it needs is good will and a little work to be resurected ! I've never seen any benchmark regarding performances, that would be the first step.

Cheers
 
-- 
Loïc Dachary, Artisan Logiciel Libre

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Mojette based Erasure Code is more performant than ISA-L or Jerasure

[Chandan Kumar Singh]

Hi

Here is the link for the benchmark study :
http://arxiv.org/pdf/1504.07038.pdf. Initially, the mailing list was
blocking it because of HTML subparts.

Regards
Chandan

On Wed, Apr 20, 2016 at 6:28 PM, Loic Dachary <loic@dachary.org> wrote:
> Hi,
>
> On 20/04/2016 15:52, Chandan Kumar Singh wrote:
>> Hi
>>
>> The authors of  Mojette Erasure Encoding claim that this EC has higher
>> performance than ISA-L and Jerasure and can be used for hot storage
>> use cases. Their EC is used in open source RozoFS. What is your
>> evaluation of it? It will be nice if it is available as another EC
>> plugin.
>
> RozoFS is no longer developped (last commit was july last year). It would be nice to have an alternative implementation of the same algorithm, well maintained. It's sad to see Free Software being frozen. But Free Software is never dead, all it needs is good will and a little work to be resurected ! I've never seen any benchmark regarding performances, that would be the first step.
>
> Cheers
>
> --
> Loïc Dachary, Artisan Logiciel Libre
>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Mojette based Erasure Code is more performant than ISA-L or Jerasure (or not)

[Loic Dachary]

On 20/04/2016 20:10, Chandan Kumar Singh wrote:
> Hi
> 
> Here is the link for the benchmark study :
> http://arxiv.org/pdf/1504.07038.pdf. Initially, the mailing list was
> blocking it because of HTML subparts.

I meant that the first step would be to repeat the benchmark as described in the paper. Unless I missed something the details of how the results were obtained have not been published. It is enough to get just one detail wrong and see significantly different results. 

Here is an example. From an implementor point of view, the performance difference that matters is when reading data that has not been damaged (i.e. what happens 99.99% of the time). Mojette does not have systematic code. What it means is that reading data always requires decoding chunks. With systematic code, which is what jerasure / isa-l offer and what Ceph uses, reading erasure coded data does not require decoding. This is the Achilles' heel of this erasure code method. It is supposed to be measured by the graph in figure 3. For instance the bottom of figure 3 states that ISA-L requires 640 cycles to read 8KB while Mojette requires 1750 when there is no erasure (i.e. no chunk lost). That makes little sense to me as you don't need ISA-L at all to read the original 8KB (that's what systematic codes are good for). This detail would need to be clarified before the experiment can be repeated.

Mojette is definitely interesting and it would be nice to see Free Software minded people working on it. There is a long way to go before it's useable, but exploring new avenues is fascinating :-)

Cheers
 
> Regards
> Chandan
> 
> On Wed, Apr 20, 2016 at 6:28 PM, Loic Dachary <loic@dachary.org> wrote:
>> Hi,
>>
>> On 20/04/2016 15:52, Chandan Kumar Singh wrote:
>>> Hi
>>>
>>> The authors of  Mojette Erasure Encoding claim that this EC has higher
>>> performance than ISA-L and Jerasure and can be used for hot storage
>>> use cases. Their EC is used in open source RozoFS. What is your
>>> evaluation of it? It will be nice if it is available as another EC
>>> plugin.
>>
>> RozoFS is no longer developped (last commit was july last year). It would be nice to have an alternative implementation of the same algorithm, well maintained. It's sad to see Free Software being frozen. But Free Software is never dead, all it needs is good will and a little work to be resurected ! I've never seen any benchmark regarding performances, that would be the first step.
>>
>> Cheers
>>
>> --
>> Loïc Dachary, Artisan Logiciel Libre
>>
> 

-- 
Loïc Dachary, Artisan Logiciel Libre
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Mojette based Erasure Code is more performant than ISA-L or Jerasure (or not)

[Chandan Kumar Singh]

Hi

As expected, Your observation in interesting. I had just glanced
through the paper initially but will go through it again. I started
looking at alternatives after reading about StreamScale Inc's claims
on jerasure. As of now, I suppose ISA is the best bet.

Regards
Chandan


On Wed, Apr 20, 2016 at 10:01 PM, Loic Dachary <loic@dachary.org> wrote:
>
>
> On 20/04/2016 20:10, Chandan Kumar Singh wrote:
>> Hi
>>
>> Here is the link for the benchmark study :
>> http://arxiv.org/pdf/1504.07038.pdf. Initially, the mailing list was
>> blocking it because of HTML subparts.
>
> I meant that the first step would be to repeat the benchmark as described in the paper. Unless I missed something the details of how the results were obtained have not been published. It is enough to get just one detail wrong and see significantly different results.
>
> Here is an example. From an implementor point of view, the performance difference that matters is when reading data that has not been damaged (i.e. what happens 99.99% of the time). Mojette does not have systematic code. What it means is that reading data always requires decoding chunks. With systematic code, which is what jerasure / isa-l offer and what Ceph uses, reading erasure coded data does not require decoding. This is the Achilles' heel of this erasure code method. It is supposed to be measured by the graph in figure 3. For instance the bottom of figure 3 states that ISA-L requires 640 cycles to read 8KB while Mojette requires 1750 when there is no erasure (i.e. no chunk lost). That makes little sense to me as you don't need ISA-L at all to read the original 8KB (that's what systematic codes are good for). This detail would need to be clarified before the experiment can be repeated.
>
> Mojette is definitely interesting and it would be nice to see Free Software minded people working on it. There is a long way to go before it's useable, but exploring new avenues is fascinating :-)
>
> Cheers
>
>> Regards
>> Chandan
>>
>> On Wed, Apr 20, 2016 at 6:28 PM, Loic Dachary <loic@dachary.org> wrote:
>>> Hi,
>>>
>>> On 20/04/2016 15:52, Chandan Kumar Singh wrote:
>>>> Hi
>>>>
>>>> The authors of  Mojette Erasure Encoding claim that this EC has higher
>>>> performance than ISA-L and Jerasure and can be used for hot storage
>>>> use cases. Their EC is used in open source RozoFS. What is your
>>>> evaluation of it? It will be nice if it is available as another EC
>>>> plugin.
>>>
>>> RozoFS is no longer developped (last commit was july last year). It would be nice to have an alternative implementation of the same algorithm, well maintained. It's sad to see Free Software being frozen. But Free Software is never dead, all it needs is good will and a little work to be resurected ! I've never seen any benchmark regarding performances, that would be the first step.
>>>
>>> Cheers
>>>
>>> --
>>> Loïc Dachary, Artisan Logiciel Libre
>>>
>>
>
> --
> Loïc Dachary, Artisan Logiciel Libre
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Mojette based Erasure Code is more performant than ISA-L or Jerasure (or not)

[Loic Dachary]

On 20/04/2016 23:48, Chandan Kumar Singh wrote:
> Hi
> 
> As expected, Your observation in interesting. I had just glanced
> through the paper initially but will go through it again. I started
> looking at alternatives after reading about StreamScale Inc's claims
> on jerasure. As of now, I suppose ISA is the best bet.

Where did you read that StreamScale has claims on jerasure ? Again, the devil is in the details ;-)

> On Wed, Apr 20, 2016 at 10:01 PM, Loic Dachary <loic@dachary.org> wrote:
>>
>>
>> On 20/04/2016 20:10, Chandan Kumar Singh wrote:
>>> Hi
>>>
>>> Here is the link for the benchmark study :
>>> http://arxiv.org/pdf/1504.07038.pdf. Initially, the mailing list was
>>> blocking it because of HTML subparts.
>>
>> I meant that the first step would be to repeat the benchmark as described in the paper. Unless I missed something the details of how the results were obtained have not been published. It is enough to get just one detail wrong and see significantly different results.
>>
>> Here is an example. From an implementor point of view, the performance difference that matters is when reading data that has not been damaged (i.e. what happens 99.99% of the time). Mojette does not have systematic code. What it means is that reading data always requires decoding chunks. With systematic code, which is what jerasure / isa-l offer and what Ceph uses, reading erasure coded data does not require decoding. This is the Achilles' heel of this erasure code method. It is supposed to be measured by the graph in figure 3. For instance the bottom of figure 3 states that ISA-L requires 640 cycles to read 8KB while Mojette requires 1750 when there is no erasure (i.e. no chunk lost). That makes little sense to me as you don't need ISA-L at all to read the original 8KB (that's what systematic codes are good for). This detail would need to be clarified before the experiment can be repeated.
>>
>> Mojette is definitely interesting and it would be nice to see Free Software minded people working on it. There is a long way to go before it's useable, but exploring new avenues is fascinating :-)
>>
>> Cheers
>>
>>> Regards
>>> Chandan
>>>
>>> On Wed, Apr 20, 2016 at 6:28 PM, Loic Dachary <loic@dachary.org> wrote:
>>>> Hi,
>>>>
>>>> On 20/04/2016 15:52, Chandan Kumar Singh wrote:
>>>>> Hi
>>>>>
>>>>> The authors of  Mojette Erasure Encoding claim that this EC has higher
>>>>> performance than ISA-L and Jerasure and can be used for hot storage
>>>>> use cases. Their EC is used in open source RozoFS. What is your
>>>>> evaluation of it? It will be nice if it is available as another EC
>>>>> plugin.
>>>>
>>>> RozoFS is no longer developped (last commit was july last year). It would be nice to have an alternative implementation of the same algorithm, well maintained. It's sad to see Free Software being frozen. But Free Software is never dead, all it needs is good will and a little work to be resurected ! I've never seen any benchmark regarding performances, that would be the first step.
>>>>
>>>> Cheers
>>>>
>>>> --
>>>> Loïc Dachary, Artisan Logiciel Libre
>>>>
>>>
>>
>> --
>> Loïc Dachary, Artisan Logiciel Libre
> 

-- 
Loïc Dachary, Artisan Logiciel Libre
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Mojette based Erasure Code is more performant than ISA-L or Jerasure (or not)

[Chandan Kumar Singh]

There is a warning on Jim Plank's website

http://web.eecs.utk.edu/~plank/plank/www/software.html

and the following article

https://www.techdirt.com/articles/20141115/07113529155/patent-troll-kills-open-source-project-speeding-up-computation-erasure-codes.shtml


On Wed, Apr 20, 2016 at 10:43 PM, Loic Dachary <loic@dachary.org> wrote:
>
>
> On 20/04/2016 23:48, Chandan Kumar Singh wrote:
>> Hi
>>
>> As expected, Your observation in interesting. I had just glanced
>> through the paper initially but will go through it again. I started
>> looking at alternatives after reading about StreamScale Inc's claims
>> on jerasure. As of now, I suppose ISA is the best bet.
>
> Where did you read that StreamScale has claims on jerasure ? Again, the devil is in the details ;-)
>
>> On Wed, Apr 20, 2016 at 10:01 PM, Loic Dachary <loic@dachary.org> wrote:
>>>
>>>
>>> On 20/04/2016 20:10, Chandan Kumar Singh wrote:
>>>> Hi
>>>>
>>>> Here is the link for the benchmark study :
>>>> http://arxiv.org/pdf/1504.07038.pdf. Initially, the mailing list was
>>>> blocking it because of HTML subparts.
>>>
>>> I meant that the first step would be to repeat the benchmark as described in the paper. Unless I missed something the details of how the results were obtained have not been published. It is enough to get just one detail wrong and see significantly different results.
>>>
>>> Here is an example. From an implementor point of view, the performance difference that matters is when reading data that has not been damaged (i.e. what happens 99.99% of the time). Mojette does not have systematic code. What it means is that reading data always requires decoding chunks. With systematic code, which is what jerasure / isa-l offer and what Ceph uses, reading erasure coded data does not require decoding. This is the Achilles' heel of this erasure code method. It is supposed to be measured by the graph in figure 3. For instance the bottom of figure 3 states that ISA-L requires 640 cycles to read 8KB while Mojette requires 1750 when there is no erasure (i.e. no chunk lost). That makes little sense to me as you don't need ISA-L at all to read the original 8KB (that's what systematic codes are good for). This detail would need to be clarified before the experiment can be repeated.
>>>
>>> Mojette is definitely interesting and it would be nice to see Free Software minded people working on it. There is a long way to go before it's useable, but exploring new avenues is fascinating :-)
>>>
>>> Cheers
>>>
>>>> Regards
>>>> Chandan
>>>>
>>>> On Wed, Apr 20, 2016 at 6:28 PM, Loic Dachary <loic@dachary.org> wrote:
>>>>> Hi,
>>>>>
>>>>> On 20/04/2016 15:52, Chandan Kumar Singh wrote:
>>>>>> Hi
>>>>>>
>>>>>> The authors of  Mojette Erasure Encoding claim that this EC has higher
>>>>>> performance than ISA-L and Jerasure and can be used for hot storage
>>>>>> use cases. Their EC is used in open source RozoFS. What is your
>>>>>> evaluation of it? It will be nice if it is available as another EC
>>>>>> plugin.
>>>>>
>>>>> RozoFS is no longer developped (last commit was july last year). It would be nice to have an alternative implementation of the same algorithm, well maintained. It's sad to see Free Software being frozen. But Free Software is never dead, all it needs is good will and a little work to be resurected ! I've never seen any benchmark regarding performances, that would be the first step.
>>>>>
>>>>> Cheers
>>>>>
>>>>> --
>>>>> Loïc Dachary, Artisan Logiciel Libre
>>>>>
>>>>
>>>
>>> --
>>> Loïc Dachary, Artisan Logiciel Libre
>>
>
> --
> Loïc Dachary, Artisan Logiciel Libre
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Mojette based Erasure Code is more performant than ISA-L or Jerasure (or not)

[Chandan Kumar Singh]

I am sure you have gone through the bullet points by Plank on that
site. A interesting hint is in the other article which mentions that
Plank used to be a consultant for StreamScale when he worked on
jerasure.

https://www.techdirt.com/articles/20141115/07113529155/patent-troll-kills-open-source-project-speeding-up-computation-erasure-codes.shtml

On Wed, Apr 20, 2016 at 11:10 PM, Loic Dachary <loic@dachary.org> wrote:
>
>
> On 21/04/2016 00:19, Chandan Kumar Singh wrote:
>> There is a warning on Jim Plank's website
>>
>> http://web.eecs.utk.edu/~plank/plank/www/software.html
>
> Where in this page is it explained why (I quote the page) "GF-Complete and Jerasure version 2.0 have been removed from web sites at the University of Tennessee and bitbucket.org." ?
>
>> and the following article
>>
>> https://www.techdirt.com/articles/20141115/07113529155/patent-troll-kills-open-source-project-speeding-up-computation-erasure-codes.shtml
>>
>>
>> On Wed, Apr 20, 2016 at 10:43 PM, Loic Dachary <loic@dachary.org> wrote:
>>>
>>>
>>> On 20/04/2016 23:48, Chandan Kumar Singh wrote:
>>>> Hi
>>>>
>>>> As expected, Your observation in interesting. I had just glanced
>>>> through the paper initially but will go through it again. I started
>>>> looking at alternatives after reading about StreamScale Inc's claims
>>>> on jerasure. As of now, I suppose ISA is the best bet.
>>>
>>> Where did you read that StreamScale has claims on jerasure ? Again, the devil is in the details ;-)
>>>
>>>> On Wed, Apr 20, 2016 at 10:01 PM, Loic Dachary <loic@dachary.org> wrote:
>>>>>
>>>>>
>>>>> On 20/04/2016 20:10, Chandan Kumar Singh wrote:
>>>>>> Hi
>>>>>>
>>>>>> Here is the link for the benchmark study :
>>>>>> http://arxiv.org/pdf/1504.07038.pdf. Initially, the mailing list was
>>>>>> blocking it because of HTML subparts.
>>>>>
>>>>> I meant that the first step would be to repeat the benchmark as described in the paper. Unless I missed something the details of how the results were obtained have not been published. It is enough to get just one detail wrong and see significantly different results.
>>>>>
>>>>> Here is an example. From an implementor point of view, the performance difference that matters is when reading data that has not been damaged (i.e. what happens 99.99% of the time). Mojette does not have systematic code. What it means is that reading data always requires decoding chunks. With systematic code, which is what jerasure / isa-l offer and what Ceph uses, reading erasure coded data does not require decoding. This is the Achilles' heel of this erasure code method. It is supposed to be measured by the graph in figure 3. For instance the bottom of figure 3 states that ISA-L requires 640 cycles to read 8KB while Mojette requires 1750 when there is no erasure (i.e. no chunk lost). That makes little sense to me as you don't need ISA-L at all to read the original 8KB (that's what systematic codes are good for). This detail would need to be clarified before the experiment can be repeated.
>>>>>
>>>>> Mojette is definitely interesting and it would be nice to see Free Software minded people working on it. There is a long way to go before it's useable, but exploring new avenues is fascinating :-)
>>>>>
>>>>> Cheers
>>>>>
>>>>>> Regards
>>>>>> Chandan
>>>>>>
>>>>>> On Wed, Apr 20, 2016 at 6:28 PM, Loic Dachary <loic@dachary.org> wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> On 20/04/2016 15:52, Chandan Kumar Singh wrote:
>>>>>>>> Hi
>>>>>>>>
>>>>>>>> The authors of  Mojette Erasure Encoding claim that this EC has higher
>>>>>>>> performance than ISA-L and Jerasure and can be used for hot storage
>>>>>>>> use cases. Their EC is used in open source RozoFS. What is your
>>>>>>>> evaluation of it? It will be nice if it is available as another EC
>>>>>>>> plugin.
>>>>>>>
>>>>>>> RozoFS is no longer developped (last commit was july last year). It would be nice to have an alternative implementation of the same algorithm, well maintained. It's sad to see Free Software being frozen. But Free Software is never dead, all it needs is good will and a little work to be resurected ! I've never seen any benchmark regarding performances, that would be the first step.
>>>>>>>
>>>>>>> Cheers
>>>>>>>
>>>>>>> --
>>>>>>> Loïc Dachary, Artisan Logiciel Libre
>>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> Loïc Dachary, Artisan Logiciel Libre
>>>>
>>>
>>> --
>>> Loïc Dachary, Artisan Logiciel Libre
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>
> --
> Loïc Dachary, Artisan Logiciel Libre
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Mojette based Erasure Code is more performant than ISA-L or Jerasure (or not)

[Chandan Kumar Singh]

Well, there is no explicit explanation as to why "GF-Complete and
Jerasure version 2.0 have been removed from web sites at the
University of Tennessee and bitbucket.org." I might not even look for
implicit reasons but what I do notice is that Plank mentions
StreamScale's claims and expects users to be aware of it.

On Wed, Apr 20, 2016 at 11:10 PM, Loic Dachary <loic@dachary.org> wrote:
>
>
> On 21/04/2016 00:19, Chandan Kumar Singh wrote:
>> There is a warning on Jim Plank's website
>>
>> http://web.eecs.utk.edu/~plank/plank/www/software.html
>
> Where in this page is it explained why (I quote the page) "GF-Complete and Jerasure version 2.0 have been removed from web sites at the University of Tennessee and bitbucket.org." ?
>
>> and the following article
>>
>> https://www.techdirt.com/articles/20141115/07113529155/patent-troll-kills-open-source-project-speeding-up-computation-erasure-codes.shtml
>>
>>
>> On Wed, Apr 20, 2016 at 10:43 PM, Loic Dachary <loic@dachary.org> wrote:
>>>
>>>
>>> On 20/04/2016 23:48, Chandan Kumar Singh wrote:
>>>> Hi
>>>>
>>>> As expected, Your observation in interesting. I had just glanced
>>>> through the paper initially but will go through it again. I started
>>>> looking at alternatives after reading about StreamScale Inc's claims
>>>> on jerasure. As of now, I suppose ISA is the best bet.
>>>
>>> Where did you read that StreamScale has claims on jerasure ? Again, the devil is in the details ;-)
>>>
>>>> On Wed, Apr 20, 2016 at 10:01 PM, Loic Dachary <loic@dachary.org> wrote:
>>>>>
>>>>>
>>>>> On 20/04/2016 20:10, Chandan Kumar Singh wrote:
>>>>>> Hi
>>>>>>
>>>>>> Here is the link for the benchmark study :
>>>>>> http://arxiv.org/pdf/1504.07038.pdf. Initially, the mailing list was
>>>>>> blocking it because of HTML subparts.
>>>>>
>>>>> I meant that the first step would be to repeat the benchmark as described in the paper. Unless I missed something the details of how the results were obtained have not been published. It is enough to get just one detail wrong and see significantly different results.
>>>>>
>>>>> Here is an example. From an implementor point of view, the performance difference that matters is when reading data that has not been damaged (i.e. what happens 99.99% of the time). Mojette does not have systematic code. What it means is that reading data always requires decoding chunks. With systematic code, which is what jerasure / isa-l offer and what Ceph uses, reading erasure coded data does not require decoding. This is the Achilles' heel of this erasure code method. It is supposed to be measured by the graph in figure 3. For instance the bottom of figure 3 states that ISA-L requires 640 cycles to read 8KB while Mojette requires 1750 when there is no erasure (i.e. no chunk lost). That makes little sense to me as you don't need ISA-L at all to read the original 8KB (that's what systematic codes are good for). This detail would need to be clarified before the experiment can be repeated.
>>>>>
>>>>> Mojette is definitely interesting and it would be nice to see Free Software minded people working on it. There is a long way to go before it's useable, but exploring new avenues is fascinating :-)
>>>>>
>>>>> Cheers
>>>>>
>>>>>> Regards
>>>>>> Chandan
>>>>>>
>>>>>> On Wed, Apr 20, 2016 at 6:28 PM, Loic Dachary <loic@dachary.org> wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> On 20/04/2016 15:52, Chandan Kumar Singh wrote:
>>>>>>>> Hi
>>>>>>>>
>>>>>>>> The authors of  Mojette Erasure Encoding claim that this EC has higher
>>>>>>>> performance than ISA-L and Jerasure and can be used for hot storage
>>>>>>>> use cases. Their EC is used in open source RozoFS. What is your
>>>>>>>> evaluation of it? It will be nice if it is available as another EC
>>>>>>>> plugin.
>>>>>>>
>>>>>>> RozoFS is no longer developped (last commit was july last year). It would be nice to have an alternative implementation of the same algorithm, well maintained. It's sad to see Free Software being frozen. But Free Software is never dead, all it needs is good will and a little work to be resurected ! I've never seen any benchmark regarding performances, that would be the first step.
>>>>>>>
>>>>>>> Cheers
>>>>>>>
>>>>>>> --
>>>>>>> Loïc Dachary, Artisan Logiciel Libre
>>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> Loïc Dachary, Artisan Logiciel Libre
>>>>
>>>
>>> --
>>> Loïc Dachary, Artisan Logiciel Libre
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>
> --
> Loïc Dachary, Artisan Logiciel Libre
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Mojette based Erasure Code is more performant than ISA-L or Jerasure

[Benoît Parrein]

Hi Chandan,

thanks you for your post

indeed we observe in our experiment that Mojette erasure coding has 
better performance than ISA-L (thanks to the discrete geometry and 
Mojette transform). A systematic/standard version exists. It is a patent 
(as couples of erasure coding). We are few cycles above memcopy 
instructions.
A paper is currently under revision and will appear soon on this.

Moreover RozoFS is alive! see on github https://github.com/rozofs/rozofs 
(last commit 6 days ago).
8 peoples are working daily to improve this fully posix DFS which is 
today the only one on the earth erasure coding based (at our best 
knowledge).
We compared to CephFS (in replication mode) and outperform it. 
Particularly, in random I/Os where the 4KB (or 8KB) blocks structure is 
well suited for Rozo.
Same. A paper will appear soon on this point.

Best regards
bparrein, University of Nantes



Le 20/04/2016 10:52, Chandan Kumar Singh a écrit :
> Hi
>
> The authors of  Mojette Erasure Encoding claim that this EC has higher
> performance than ISA-L and Jerasure and can be used for hot storage
> use cases. Their EC is used in open source RozoFS. What is your
> evaluation of it? It will be nice if it is available as another EC
> plugin.
>
> Regards
> Chandan
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

-- 
----------------
Benoît Parrein
Maître de Conférences (HDR)
Université de Nantes (Polytech Nantes), laboratoire IRCCyN
web: http://www.irccyn.ec-nantes.fr/~parrein/
tel: +33(0)240683050

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Mojette based Erasure Code is more performant than ISA-L or Jerasure

[Alexandre DERUMIER]

Hi Loic,

>>RozoFS is no longer developped (last commit was july last year)

Seem that they have released rozofs 2.0 the 21th april 2016
https://github.com/rozofs/rozofs/releases

and they are activity in the git since then.


I don't think the project is dead, because they are doing a lot of commercial communication recently


http://www.theregister.co.uk/2015/12/01/rozo_aiming_to_fill_scaleout_nas_object_scalability_gap/

with a roadmap up to 2017.

v2.1 Jun 2016 – Fast indexing, versioning, metadata re-use on file deletion*
v2.2 Dec 2016 – Per-directory snapshots (possibly), storage QoS (possible throttling)
V2.3 Jun 2017 – Fast disk encryption, SSD caching


Regards,

Alexandre


----- Mail original -----
De: "Loic Dachary" <loic@dachary.org>
À: "Chandan Kumar Singh" <chandan.kr.singh@gmail.com>, "ceph-devel" <ceph-devel@vger.kernel.org>
Envoyé: Mercredi 20 Avril 2016 14:58:31
Objet: Re: Mojette based Erasure Code is more performant than ISA-L or Jerasure

Hi, 

On 20/04/2016 15:52, Chandan Kumar Singh wrote: 
> Hi 
> 
> The authors of Mojette Erasure Encoding claim that this EC has higher 
> performance than ISA-L and Jerasure and can be used for hot storage 
> use cases. Their EC is used in open source RozoFS. What is your 
> evaluation of it? It will be nice if it is available as another EC 
> plugin. 

RozoFS is no longer developped (last commit was july last year). It would be nice to have an alternative implementation of the same algorithm, well maintained. It's sad to see Free Software being frozen. But Free Software is never dead, all it needs is good will and a little work to be resurected ! I've never seen any benchmark regarding performances, that would be the first step. 

Cheers 

-- 
Loïc Dachary, Artisan Logiciel Libre 

-- 
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in 
the body of a message to majordomo@vger.kernel.org 
More majordomo info at http://vger.kernel.org/majordomo-info.html 
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Mojette based Erasure Code is more performant than ISA-L or Jerasure

[Alexandre DERUMIER]

>>We compared to CephFS (in replication mode) and outperform it. 
>>Particularly, in random I/Os where the 4KB (or 8KB) blocks structure is 
>>well suited for Rozo. 

I have found your paper here

https://departements.telecom-bretagne.eu/data/elec/SITES/PublicElec/BenoitParrein_interGDR.pdf

100k 4k randread, an 80k 4k iops randwrite, not too bad with 4 nodes ! 

What was the bottleneck ? cpu ?




----- Mail original -----
De: "Benoît Parrein" <benoit.parrein@polytech.univ-nantes.fr>
À: "Chandan Kumar Singh" <chandan.kr.singh@gmail.com>, "ceph-devel" <ceph-devel@vger.kernel.org>
Envoyé: Vendredi 3 Juin 2016 15:25:37
Objet: Re: Mojette based Erasure Code is more performant than ISA-L or Jerasure

Hi Chandan, 

thanks you for your post 

indeed we observe in our experiment that Mojette erasure coding has 
better performance than ISA-L (thanks to the discrete geometry and 
Mojette transform). A systematic/standard version exists. It is a patent 
(as couples of erasure coding). We are few cycles above memcopy 
instructions. 
A paper is currently under revision and will appear soon on this. 

Moreover RozoFS is alive! see on github https://github.com/rozofs/rozofs 
(last commit 6 days ago). 
8 peoples are working daily to improve this fully posix DFS which is 
today the only one on the earth erasure coding based (at our best 
knowledge). 
We compared to CephFS (in replication mode) and outperform it. 
Particularly, in random I/Os where the 4KB (or 8KB) blocks structure is 
well suited for Rozo. 
Same. A paper will appear soon on this point. 

Best regards 
bparrein, University of Nantes 



Le 20/04/2016 10:52, Chandan Kumar Singh a écrit : 
> Hi 
> 
> The authors of Mojette Erasure Encoding claim that this EC has higher 
> performance than ISA-L and Jerasure and can be used for hot storage 
> use cases. Their EC is used in open source RozoFS. What is your 
> evaluation of it? It will be nice if it is available as another EC 
> plugin. 
> 
> Regards 
> Chandan 
> -- 
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in 
> the body of a message to majordomo@vger.kernel.org 
> More majordomo info at http://vger.kernel.org/majordomo-info.html 
> 

-- 
---------------- 
Benoît Parrein 
Maître de Conférences (HDR) 
Université de Nantes (Polytech Nantes), laboratoire IRCCyN 
web: http://www.irccyn.ec-nantes.fr/~parrein/ 
tel: +33(0)240683050 

-- 
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in 
the body of a message to majordomo@vger.kernel.org 
More majordomo info at http://vger.kernel.org/majordomo-info.html 
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html