From mboxrd@z Thu Jan 1 00:00:00 1970 From: Loic Dachary Subject: Re: Deadline of Github pull request for Hammer release (question) Date: Tue, 20 Jan 2015 08:22:43 +0100 Message-ID: <54BE0243.60301@dachary.org> References: <870DE8DBB716524BAE51B2D499EC81E40AA3F9AB@g01jpexmbyt24> <54AC048E.9040200@dachary.org> <870DE8DBB716524BAE51B2D499EC81E40AA40849@g01jpexmbyt24> <54B4F2C0.2040805@dachary.org> <870DE8DBB716524BAE51B2D499EC81E40AA40872@g01jpexmbyt24> <54B5137F.7090902@dachary.org> <870DE8DBB716524BAE51B2D499EC81E40AA40983@g01jpexmbyt24> <54B55DE8.6010003@dachary.org> <870DE8DBB716524BAE51B2D499EC81E40AA5B1F2@g01jpexmbyt24> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="tUk6hgMgNuoAaiPvmF2wxiPSuUG8eipN3" Return-path: Received: from mail2.dachary.org ([91.121.57.175]:43630 "EHLO smtp.dmail.dachary.org" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1752039AbbATHWp (ORCPT ); Tue, 20 Jan 2015 02:22:45 -0500 In-Reply-To: <870DE8DBB716524BAE51B2D499EC81E40AA5B1F2@g01jpexmbyt24> Sender: ceph-devel-owner@vger.kernel.org List-ID: To: "Miyamae, Takeshi" Cc: Ceph Development , "Shiozawa, Kensuke" , "Nakao, Takanori" , "Paul Von-Stamwitz (PVonStamwitz@us.fujitsu.com)" This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --tUk6hgMgNuoAaiPvmF2wxiPSuUG8eipN3 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi, Thanks for the update. To speed up things, I could review what's already = published. Did you add the tests already ? Cheers On 20/01/2015 01:48, Miyamae, Takeshi wrote: > Hi Loic, >=20 > Thank you for your advice which suggested providing SHEC as an extra-pa= ckage. > We believe it's generally a good idea, but we are hoping SHEC would be = merged > into Hammer because that would have an apparent advantage from a viewpo= int of > maintenance support. >=20 > As for the thread safety issue, we totally agree with you and we are tr= ying to solve it. > We will complete it in a few days and we'd like to ask you to review it= again so that > it might be in time for Hammer's feature freeze (v0.93). >=20 > Best regards, > Takeshi Miyamae >=20 > -----Original Message----- > From: ceph-devel-owner@vger.kernel.org [mailto:ceph-devel-owner@vger.ke= rnel.org] On Behalf Of Loic Dachary > Sent: Wednesday, January 14, 2015 3:03 AM > To: Miyamae, Takeshi/=E5=AE=AE=E5=89=8D =E5=89=9B > Cc: Ceph Development; Shiozawa, Kensuke/=E5=A1=A9=E6=B2=A2 =E8=B3=A2=E8= =BC=94; Nakao, Takanori/=E4=B8=AD=E5=B0=BE =E9=B7=B9=E8=A9=94; Paul Von-S= tamwitz (PVonStamwitz@us.fujitsu.com) > Subject: Re: Deadline of Github pull request for Hammer release (questi= on) >=20 > Hi, >=20 > On 13/01/2015 18:05, Miyamae, Takeshi wrote: >> Hi Loic, >> >> Thank you for your quick review. >> >>> Could you reference the jerasure files (they are in the jerasure=20 >>> plugin already) instead of including them in your patch ? >> >> We had used the reference of jerasure v2.0 files before, but changed=20 >> to including v1.2 files after the patent issue. >=20 > If you are refering to http://erasure-code-patents.xyz/, it can safely = be ignored. >=20 >> However, we can restore the reference right away if we are needed. >> >>> In ::prepare you reuse the matrix, if possible. If your intention is = >>> to improve performances, you should consider using the same approach = as the isa plugin. >> >> We have found a performance issue caused by redundant initializations = >> of the module, and solved it by caching the initialized variables in m= emory. >> If that is the same approach as isa-plugin you mentioned, we also do=20 >> not have the same issue any more. >=20 > The ISA plugin approach is thread safe, that's the important part. >=20 >>> I'll have more comments once you have unit / functional tests >> >> We do have the those unit/functional tests, but the server is unreacha= ble at this moment. >> Please let us upload them to the repository tomorrow. >=20 > Great. >=20 >>> Regarding your initial question: I think it is too late for the Hamme= r release. >> >> We are so disappointed to hear that, but we must apologize for the lat= e request. >> If there would be any chance that SHEC be pulled into hammer, we are=20 >> very happy to conduct all the necessary tests by ourselves. >> Please let us know the detailed schedule if this is a feasible option.= >=20 > Note that since this is a plugin it will be easy for someone to install= it on a Hammer cluster, simply by copying the plugin files to each OSD /= MON and restarting them. If there is some kind of urgency for you to be = able to install this new plugin, I would advise making a package that con= tains just this file, restart the OSD once installed and recommend that t= he installation is done on all machines of the cluster (because every OSD= / MON need to know about the plugin).=20 >=20 > Cheers >=20 >> Best regards, >> Takeshi Miyamae >> >> -----Original Message----- >> From: ceph-devel-owner@vger.kernel.org=20 >> [mailto:ceph-devel-owner@vger.kernel.org] On Behalf Of Loic Dachary >> Sent: Tuesday, January 13, 2015 9:46 PM >> To: Miyamae, Takeshi/=E5=AE=AE=E5=89=8D =E5=89=9B >> Cc: Ceph Development; Shiozawa, Kensuke/=E5=A1=A9=E6=B2=A2 =E8=B3=A2=E8= =BC=94; Nakao, Takanori/=E4=B8=AD=E5=B0=BE =E9=B7=B9=E8=A9=94;=20 >> Paul Von-Stamwitz (PVonStamwitz@us.fujitsu.com) >> Subject: Re: Deadline of Github pull request for Hammer release=20 >> (question) >> >> Hi, >> >> It's a great start :-) A few comments: >> >> Could you reference the jerasure files (they are in the jerasure plugi= n already) instead of including them in your patch ? >> >> In ::prepare you reuse the matrix, if possible. If your intention is=20 >> to improve performances, you should consider using the same approach=20 >> as the isa plugin. See=20 >> http://workbench.dachary.org/ceph/ceph/blob/giant/src/erasure-code/isa= >> /ErasureCodePluginIsa.cc#L36 which is and independent type=20 >> http://workbench.dachary.org/ceph/ceph/blob/giant/src/erasure-code/isa= >> /ErasureCodeIsaTableCache.h with only one instance and is populated as= =20 >> needed and protected by a lock to make it thread safe :=20 >> http://workbench.dachary.org/ceph/ceph/blob/giant/src/erasure-code/isa= >> /ErasureCodeIsaTableCache.h#L101 >> >> I'll have more comments once you have unit / functional tests ( simila= r to http://workbench.dachary.org/ceph/ceph/blob/giant/src/test/erasure-c= ode/TestErasureCodeJerasure.cc ).=20 >> >> Regarding your initial question: I think it is too late for the Hammer= release. But from what I read it looks like we'll be able to merge in th= e early stages of the next release and that will give us time to properly= test it. >> >> Cheers >> >> On 13/01/2015 11:34, Miyamae, Takeshi wrote: >>> Hi Loic, >>> >>> I'm so sorry. The following is the correct repository. >>> >>> https://github.com/t-miyamae/ceph >>> >>> Best regards, >>> Takeshi Miyamae >>> >>> -----Original Message----- >>> From: Loic Dachary [mailto:loic@dachary.org] >>> Sent: Tuesday, January 13, 2015 7:26 PM >>> To: Miyamae, Takeshi/=E5=AE=AE=E5=89=8D =E5=89=9B >>> Cc: Ceph Development; Shiozawa, Kensuke/=E5=A1=A9=E6=B2=A2 =E8=B3=A2=E8= =BC=94; Nakao, Takanori/=E4=B8=AD=E5=B0=BE =E9=B7=B9=E8=A9=94; >>> Paul Von-Stamwitz (PVonStamwitz@us.fujitsu.com) >>> Subject: Re: Deadline of Github pull request for Hammer release >>> (question) >>> >>> Hi, >>> >>> On 13/01/2015 11:24, Miyamae, Takeshi wrote: >>>> Hi Loic, >>>> >>>>> Although we're late in the Hammer roadmap, it's a good time for an = early preview. It will help show what needs to be changed to accomodate t= he SHEC plugin. >>>> >>>> Thank you for your advices. >>>> We have uploaded our latest codes to the following folk repository f= or an early review. >>>> >>>> https://github.com/miyamae-takeshi/multiple-shec >>> >>> It's 404 ? Is it a private repository maybe ? >>> >>>> >>>> SHEC is located in src/erasure-code/shec directory. >>>> >>>> We are verifying SHEC's advantages on our Ceph cluster. It takes a l= ittle bit more. >>>> Would you please start the review before that? >>>> >>>> Best regards, >>>> Takeshi Miyamae >>>> >>>> -----Original Message----- >>>> From: ceph-devel-owner@vger.kernel.org=20 >>>> [mailto:ceph-devel-owner@vger.kernel.org] On Behalf Of Loic Dachary >>>> Sent: Wednesday, January 7, 2015 12:52 AM >>>> To: Miyamae, Takeshi/=E5=AE=AE=E5=89=8D =E5=89=9B >>>> Cc: Ceph Development; Shiozawa, Kensuke/=E5=A1=A9=E6=B2=A2 =E8=B3=A2= =E8=BC=94; Nakao, Takanori/=E4=B8=AD=E5=B0=BE =E9=B7=B9=E8=A9=94 >>>> Subject: Re: Deadline of Github pull request for Hammer release >>>> (question) >>>> >>>> Hi, >>>> >>>> On 06/01/2015 12:49, Miyamae, Takeshi wrote: >>>>> Dear Loic, >>>>> >>>>> I'm Takeshi Miyamae, one of the authors of SHEC's blueprint. >>>>> >>>>> Shingled Erasure Code (SHEC) >>>>> https://wiki.ceph.com/Planning/Blueprints/Hammer/Shingled_Erasure_C= >>>>> o >>>>> d >>>>> e >>>>> _(SHEC) >>>> >>>> The work you have done is quite impressive :-) >>>> >>>>> We have revised our blueprint shown in the last CDS to extend our=20 >>>>> erasure code layouts and describe the guideline for choosing SHEC a= mong various EC plugins. >>>>> We believe the blueprint now answers all the comments given at the = CDS. >>>> >>>> Great. >>>> >>>>> In addition, we would like to ask for your advice on the schedule=20 >>>>> of our github pull request. More specifically, we would like to=20 >>>>> know its deadline for Hammer release. >>>>> (As we have not really completed our verification of SHEC, we are=20 >>>>> wondering if we should make it open for early preview.) >>>> >>>> Although we're late in the Hammer roadmap, it's a good time for an e= arly preview. It will help show what needs to be changed to accomodate th= e SHEC plugin. >>>> >>>> Cheers >>>> >>>>> Thank you in advance, >>>>> Takeshi Miyamae >>>>> >>>>> -- >>>>> To unsubscribe from this list: send the line "unsubscribe ceph-deve= l"=20 >>>>> in the body of a message to majordomo@vger.kernel.org More=20 >>>>> majordomo info at http://vger.kernel.org/majordomo-info.html >>>>> >>>> >>>> -- >>>> Lo=C3=AFc Dachary, Artisan Logiciel Libre >>>> >>> >>> -- >>> Lo=C3=AFc Dachary, Artisan Logiciel Libre >>> >> >> -- >> Lo=C3=AFc Dachary, Artisan Logiciel Libre >> >=20 > -- > Lo=C3=AFc Dachary, Artisan Logiciel Libre >=20 --=20 Lo=C3=AFc Dachary, Artisan Logiciel Libre --tUk6hgMgNuoAaiPvmF2wxiPSuUG8eipN3 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iEYEARECAAYFAlS+AkMACgkQ8dLMyEl6F20a6wCfaRUnJZml0OJLv6nvkRBdtOMm b7QAnjLuuso5LMEqpYGg946S/yz7A5PE =7nfA -----END PGP SIGNATURE----- --tUk6hgMgNuoAaiPvmF2wxiPSuUG8eipN3--