Erasure Code Plugins : PLUGINS_V3 feature

Erasure Code Plugins : PLUGINS_V3 feature

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 1120 bytes --]

Hi Sam,

The SHEC plugin[0] has been running in the rados runs[1] in the past few months. It also has a matching corpus verification which runs on every make check[2] as well as its optimized variants. I believe the flag "experimental" can now be removed. 

In order to do so, we need to use a PLUGINS_V3 feature, in the same way we did back in Giant when the ISA and LRC plugins were introduced[3]. This won't be necessary in the future, when there is a generic plugin mechanism, but right now that's what we need. It would be a commit very similar to the one implementing PLUGINS_V2[4].

Is this agreeable to you ? Or would you rather see another way to resolve this ?

Cheers

[0] https://github.com/ceph/ceph/tree/master/src/erasure-code/shec
[1] https://github.com/ceph/ceph-qa-suite/tree/master/suites/rados/thrash-erasure-code-shec
[2] https://github.com/ceph/ceph-erasure-code-corpus/blob/master/v0.92-988/non-regression.sh#L52
[3] http://tracker.ceph.com/issues/9343
[4] https://github.com/ceph/ceph/commit/9687150ceac9cc7e506bc227f430d4207a6d7489

-- 
Loïc Dachary, Artisan Logiciel Libre


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Erasure Code Plugins : PLUGINS_V3 feature

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 1313 bytes --]

Hi Sam,

How does this proposal sound ? It would be great if that was done before the feature freeze.

Cheers

On 29/07/2015 11:16, Loic Dachary wrote:
> Hi Sam,
> 
> The SHEC plugin[0] has been running in the rados runs[1] in the past few months. It also has a matching corpus verification which runs on every make check[2] as well as its optimized variants. I believe the flag "experimental" can now be removed. 
> 
> In order to do so, we need to use a PLUGINS_V3 feature, in the same way we did back in Giant when the ISA and LRC plugins were introduced[3]. This won't be necessary in the future, when there is a generic plugin mechanism, but right now that's what we need. It would be a commit very similar to the one implementing PLUGINS_V2[4].
> 
> Is this agreeable to you ? Or would you rather see another way to resolve this ?
> 
> Cheers
> 
> [0] https://github.com/ceph/ceph/tree/master/src/erasure-code/shec
> [1] https://github.com/ceph/ceph-qa-suite/tree/master/suites/rados/thrash-erasure-code-shec
> [2] https://github.com/ceph/ceph-erasure-code-corpus/blob/master/v0.92-988/non-regression.sh#L52
> [3] http://tracker.ceph.com/issues/9343
> [4] https://github.com/ceph/ceph/commit/9687150ceac9cc7e506bc227f430d4207a6d7489
> 

-- 
Loïc Dachary, Artisan Logiciel Libre


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Erasure Code Plugins : PLUGINS_V3 feature

[Sage Weil]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1724 bytes --]

On Wed, 5 Aug 2015, Loic Dachary wrote:
> Hi Sam,
> 
> How does this proposal sound ? It would be great if that was done before 
> the feature freeze.

I think it's a good time.

Takeshi, note that what this really means is that the on-disk encoding 
needs to remain fixed.  If we decide to change it down the line, we'll 
have to make a 'shec2' or similar so that the old format is still 
decodable (or ensure that existing data can still be read in some other 
way).

Sound good?

sage


> 
> Cheers
> 
> On 29/07/2015 11:16, Loic Dachary wrote:
> > Hi Sam,
> > 
> > The SHEC plugin[0] has been running in the rados runs[1] in the past few months. It also has a matching corpus verification which runs on every make check[2] as well as its optimized variants. I believe the flag "experimental" can now be removed. 
> > 
> > In order to do so, we need to use a PLUGINS_V3 feature, in the same way we did back in Giant when the ISA and LRC plugins were introduced[3]. This won't be necessary in the future, when there is a generic plugin mechanism, but right now that's what we need. It would be a commit very similar to the one implementing PLUGINS_V2[4].
> > 
> > Is this agreeable to you ? Or would you rather see another way to resolve this ?
> > 
> > Cheers
> > 
> > [0] https://github.com/ceph/ceph/tree/master/src/erasure-code/shec
> > [1] https://github.com/ceph/ceph-qa-suite/tree/master/suites/rados/thrash-erasure-code-shec
> > [2] https://github.com/ceph/ceph-erasure-code-corpus/blob/master/v0.92-988/non-regression.sh#L52
> > [3] http://tracker.ceph.com/issues/9343
> > [4] https://github.com/ceph/ceph/commit/9687150ceac9cc7e506bc227f430d4207a6d7489
> > 
> 
> -- 
> Loïc Dachary, Artisan Logiciel Libre
> 
> 

RE: Erasure Code Plugins : PLUGINS_V3 feature

[Miyamae, Takeshi]

Dear Sage,

> note that what this really means is that the on-disk encoding needs to remain fixed.

Thank you for letting us know the important notice.
We have no plan to change shec's format at this moment, but we will remember the
comment for any future events.

Best Regards,
Takeshi Miyamae

-----Original Message-----
From: Sage Weil [mailto:sweil@redhat.com] 
Sent: Thursday, August 6, 2015 3:45 AM
To: Loic Dachary; Miyamae, Takeshi/宮前 剛
Cc: Samuel Just; Ceph Development
Subject: Re: Erasure Code Plugins : PLUGINS_V3 feature

On Wed, 5 Aug 2015, Loic Dachary wrote:
> Hi Sam,
> 
> How does this proposal sound ? It would be great if that was done 
> before the feature freeze.

I think it's a good time.

Takeshi, note that what this really means is that the on-disk encoding needs to remain fixed.  If we decide to change it down the line, we'll have to make a 'shec2' or similar so that the old format is still decodable (or ensure that existing data can still be read in some other way).

Sound good?

sage


> 
> Cheers
> 
> On 29/07/2015 11:16, Loic Dachary wrote:
> > Hi Sam,
> > 
> > The SHEC plugin[0] has been running in the rados runs[1] in the past few months. It also has a matching corpus verification which runs on every make check[2] as well as its optimized variants. I believe the flag "experimental" can now be removed. 
> > 
> > In order to do so, we need to use a PLUGINS_V3 feature, in the same way we did back in Giant when the ISA and LRC plugins were introduced[3]. This won't be necessary in the future, when there is a generic plugin mechanism, but right now that's what we need. It would be a commit very similar to the one implementing PLUGINS_V2[4].
> > 
> > Is this agreeable to you ? Or would you rather see another way to resolve this ?
> > 
> > Cheers
> > 
> > [0] https://github.com/ceph/ceph/tree/master/src/erasure-code/shec
> > [1] 
> > https://github.com/ceph/ceph-qa-suite/tree/master/suites/rados/thras
> > h-erasure-code-shec [2] 
> > https://github.com/ceph/ceph-erasure-code-corpus/blob/master/v0.92-9
> > 88/non-regression.sh#L52 [3] http://tracker.ceph.com/issues/9343
> > [4] 
> > https://github.com/ceph/ceph/commit/9687150ceac9cc7e506bc227f430d420
> > 7a6d7489
> > 
> 
> --
> Loïc Dachary, Artisan Logiciel Libre
> 
> 

Re: Erasure Code Plugins : PLUGINS_V3 feature

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 2684 bytes --]

Hi Takeshi,

https://github.com/ceph/ceph/pull/5493 is ready for your review. The matching integration tests can be found at https://github.com/ceph/ceph-qa-suite/pull/523

Cheers

On 06/08/2015 02:28, Miyamae, Takeshi wrote:
> Dear Sage,
> 
>> note that what this really means is that the on-disk encoding needs to remain fixed.
> 
> Thank you for letting us know the important notice.
> We have no plan to change shec's format at this moment, but we will remember the
> comment for any future events.
> 
> Best Regards,
> Takeshi Miyamae
> 
> -----Original Message-----
> From: Sage Weil [mailto:sweil@redhat.com] 
> Sent: Thursday, August 6, 2015 3:45 AM
> To: Loic Dachary; Miyamae, Takeshi/宮前 剛
> Cc: Samuel Just; Ceph Development
> Subject: Re: Erasure Code Plugins : PLUGINS_V3 feature
> 
> On Wed, 5 Aug 2015, Loic Dachary wrote:
>> Hi Sam,
>>
>> How does this proposal sound ? It would be great if that was done 
>> before the feature freeze.
> 
> I think it's a good time.
> 
> Takeshi, note that what this really means is that the on-disk encoding needs to remain fixed.  If we decide to change it down the line, we'll have to make a 'shec2' or similar so that the old format is still decodable (or ensure that existing data can still be read in some other way).
> 
> Sound good?
> 
> sage
> 
> 
>>
>> Cheers
>>
>> On 29/07/2015 11:16, Loic Dachary wrote:
>>> Hi Sam,
>>>
>>> The SHEC plugin[0] has been running in the rados runs[1] in the past few months. It also has a matching corpus verification which runs on every make check[2] as well as its optimized variants. I believe the flag "experimental" can now be removed. 
>>>
>>> In order to do so, we need to use a PLUGINS_V3 feature, in the same way we did back in Giant when the ISA and LRC plugins were introduced[3]. This won't be necessary in the future, when there is a generic plugin mechanism, but right now that's what we need. It would be a commit very similar to the one implementing PLUGINS_V2[4].
>>>
>>> Is this agreeable to you ? Or would you rather see another way to resolve this ?
>>>
>>> Cheers
>>>
>>> [0] https://github.com/ceph/ceph/tree/master/src/erasure-code/shec
>>> [1] 
>>> https://github.com/ceph/ceph-qa-suite/tree/master/suites/rados/thras
>>> h-erasure-code-shec [2] 
>>> https://github.com/ceph/ceph-erasure-code-corpus/blob/master/v0.92-9
>>> 88/non-regression.sh#L52 [3] http://tracker.ceph.com/issues/9343
>>> [4] 
>>> https://github.com/ceph/ceph/commit/9687150ceac9cc7e506bc227f430d420
>>> 7a6d7489
>>>
>>
>> --
>> Loïc Dachary, Artisan Logiciel Libre
>>
>>

-- 
Loïc Dachary, Artisan Logiciel Libre


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

RE: Erasure Code Plugins : PLUGINS_V3 feature

[Miyamae, Takeshi]

Hi Loic,

Thank you for arranging PLUGINS_V3 feature.
I had just started to review pull request #5493. Please wait just a moment.

By the way, may I ask what kind of status #5257 (decoding cache:
the last immediate request from SHEC) currently is?
https://github.com/ceph/ceph/pull/5257
Tell us if we need to rebase again.

Best regards,
Takeshi Miyamae

-----Original Message-----
From: Loic Dachary [mailto:loic@dachary.org] 
Sent: Thursday, August 6, 2015 10:58 PM
To: Miyamae, Takeshi/宮前 剛
Cc: Ceph Development; Shiozawa, Kensuke/塩沢 賢輔; Nakao, Takanori/中尾 鷹詔
Subject: Re: Erasure Code Plugins : PLUGINS_V3 feature

Hi Takeshi,

https://github.com/ceph/ceph/pull/5493 is ready for your review. The matching integration tests can be found at https://github.com/ceph/ceph-qa-suite/pull/523

Cheers

On 06/08/2015 02:28, Miyamae, Takeshi wrote:
> Dear Sage,
> 
>> note that what this really means is that the on-disk encoding needs to remain fixed.
> 
> Thank you for letting us know the important notice.
> We have no plan to change shec's format at this moment, but we will 
> remember the comment for any future events.
> 
> Best Regards,
> Takeshi Miyamae
> 
> -----Original Message-----
> From: Sage Weil [mailto:sweil@redhat.com]
> Sent: Thursday, August 6, 2015 3:45 AM
> To: Loic Dachary; Miyamae, Takeshi/宮前 剛
> Cc: Samuel Just; Ceph Development
> Subject: Re: Erasure Code Plugins : PLUGINS_V3 feature
> 
> On Wed, 5 Aug 2015, Loic Dachary wrote:
>> Hi Sam,
>>
>> How does this proposal sound ? It would be great if that was done 
>> before the feature freeze.
> 
> I think it's a good time.
> 
> Takeshi, note that what this really means is that the on-disk encoding needs to remain fixed.  If we decide to change it down the line, we'll have to make a 'shec2' or similar so that the old format is still decodable (or ensure that existing data can still be read in some other way).
> 
> Sound good?
> 
> sage
> 
> 
>>
>> Cheers
>>
>> On 29/07/2015 11:16, Loic Dachary wrote:
>>> Hi Sam,
>>>
>>> The SHEC plugin[0] has been running in the rados runs[1] in the past few months. It also has a matching corpus verification which runs on every make check[2] as well as its optimized variants. I believe the flag "experimental" can now be removed. 
>>>
>>> In order to do so, we need to use a PLUGINS_V3 feature, in the same way we did back in Giant when the ISA and LRC plugins were introduced[3]. This won't be necessary in the future, when there is a generic plugin mechanism, but right now that's what we need. It would be a commit very similar to the one implementing PLUGINS_V2[4].
>>>
>>> Is this agreeable to you ? Or would you rather see another way to resolve this ?
>>>
>>> Cheers
>>>
>>> [0] https://github.com/ceph/ceph/tree/master/src/erasure-code/shec
>>> [1]
>>> https://github.com/ceph/ceph-qa-suite/tree/master/suites/rados/thras
>>> h-erasure-code-shec [2]
>>> https://github.com/ceph/ceph-erasure-code-corpus/blob/master/v0.92-9
>>> 88/non-regression.sh#L52 [3] http://tracker.ceph.com/issues/9343
>>> [4]
>>> https://github.com/ceph/ceph/commit/9687150ceac9cc7e506bc227f430d420
>>> 7a6d7489
>>>
>>
>> --
>> Loïc Dachary, Artisan Logiciel Libre
>>
>>

-- 
Loïc Dachary, Artisan Logiciel Libre