* Standalone libcrush: call for participation @ 2017-01-20 23:16 Loic Dachary 2017-01-23 16:29 ` Adam C. Emerson 0 siblings, 1 reply; 10+ messages in thread From: Loic Dachary @ 2017-01-20 23:16 UTC (permalink / raw) To: Ceph Development Hi Ceph, The idea of packaging CRUSH [1] as a standalone library that can be used by Ceph and other projects is not new. From the beginning Sage expected/hoped that the algorithm would be useful for lots of different applications. Other storage systems, load balancers, CDNs, or any other distributed thing where a compact description of how to map a large number of things to other things would be useful. This has partly been proven out in that some storage systems "use CRUSH," but they use it in the sense that they reimplemented something based on the underlying algorithm. It's also a place where we haven't seen a need to do anything ceph-specific; the problems we've fixed and changes we've made are general issues with the original algorithm and implementation. The main barrier has been that the code and tools are built using a lot of the generic ceph infrastructure like encode/decode/bufferlists. The beginning of the next ceph release cycle seems like a good time to initiate that effort. If someone is willing to help with the bootstrap, now is a good time. The library will be hosted under the libcrush.org domain name. But that's about the only thing that happened so far :-) Cheers [1] https://github.com/ceph/ceph/tree/master/src/crush -- Loïc Dachary, Artisan Logiciel Libre ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Standalone libcrush: call for participation 2017-01-20 23:16 Standalone libcrush: call for participation Loic Dachary @ 2017-01-23 16:29 ` Adam C. Emerson 2017-01-23 17:00 ` Loic Dachary 0 siblings, 1 reply; 10+ messages in thread From: Adam C. Emerson @ 2017-01-23 16:29 UTC (permalink / raw) To: Loic Dachary; +Cc: Ceph Development [-- Attachment #1: Type: text/plain, Size: 744 bytes --] On 21/01/2017, Loic Dachary wrote: [snip] > The beginning of the next ceph release cycle seems like a good time > to initiate that effort. If someone is willing to help with the > bootstrap, now is a good time. The library will be hosted under the > libcrush.org domain name. But that's about the only thing that > happened so far :-) Would the Ceph git repository continue to have its own version of CRUSH included, or would we pull in libcrush as a submodule? Also, does this affect the relationship between userspace CRUSH and Linux kernel CRUSH? -- Senior Software Engineer Red Hat Storage, Ann Arbor, MI, US IRC: Aemerson@{RedHat, OFTC} 0x80F7544B90EDBFB9 E707 86BA 0C1B 62CC 152C 7C12 80F7 544B 90ED BFB9 [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 603 bytes --] ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Standalone libcrush: call for participation 2017-01-23 16:29 ` Adam C. Emerson @ 2017-01-23 17:00 ` Loic Dachary 2017-01-23 18:02 ` Adam C. Emerson 2017-01-23 18:43 ` Ilya Dryomov 0 siblings, 2 replies; 10+ messages in thread From: Loic Dachary @ 2017-01-23 17:00 UTC (permalink / raw) To: Ceph Development Hi Adam, On 01/23/2017 05:29 PM, Adam C. Emerson wrote: > On 21/01/2017, Loic Dachary wrote: > [snip] >> The beginning of the next ceph release cycle seems like a good time >> to initiate that effort. If someone is willing to help with the >> bootstrap, now is a good time. The library will be hosted under the >> libcrush.org domain name. But that's about the only thing that >> happened so far :-) > > Would the Ceph git repository continue to have its own version of > CRUSH included, or would we pull in libcrush as a submodule? If all goes well and libcrush becomes actively and reliably maintained outside of the Ceph tree, it could either be a submodule or even a packaged dependency. Before that happens things can stay the way they are. I volunteer to manually monitor and cherry-pick whatever changes in the Ceph tree onto the standalone repository ( http://libcrush.org/main/libcrush/ ) so that it does not drift away. > Also, does this affect the relationship between userspace CRUSH and > Linux kernel CRUSH? I don't know :-) Unless I'm mistaken the code in the linux kernel is an unmodified copy of the code from src/crush/*.[ch]. I however have no clue how it is tested to work as it should in both contexts. Other than running integration tests with teuthology. How do you suggest we go about it ? Cheers -- Loïc Dachary, Artisan Logiciel Libre ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Standalone libcrush: call for participation 2017-01-23 17:00 ` Loic Dachary @ 2017-01-23 18:02 ` Adam C. Emerson 2017-01-23 18:43 ` Ilya Dryomov 1 sibling, 0 replies; 10+ messages in thread From: Adam C. Emerson @ 2017-01-23 18:02 UTC (permalink / raw) To: Loic Dachary; +Cc: Ceph Development [-- Attachment #1: Type: text/plain, Size: 902 bytes --] On 23/01/2017, Loic Dachary wrote: > Hi Adam, I don't know :-) Unless I'm mistaken the code in the linux > kernel is an unmodified copy of the code from src/crush/*.[ch]. I > however have no clue how it is tested to work as it should in both > contexts. Other than running integration tests with teuthology. > > How do you suggest we go about it ? I don't actually know. I just wondered, I had the vague impression that the CRUSH from Ceph was copied into the kernel but I didn't know who did it, if it was parallel development or a blanket copy or what. It /does/ mean that any changes to libcrush have to be kept as kernel compatible C code, which isn't bad, it's just good to make sure it's explicit. -- Senior Software Engineer Red Hat Storage, Ann Arbor, MI, US IRC: Aemerson@{RedHat, OFTC} 0x80F7544B90EDBFB9 E707 86BA 0C1B 62CC 152C 7C12 80F7 544B 90ED BFB9 [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 603 bytes --] ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Standalone libcrush: call for participation 2017-01-23 17:00 ` Loic Dachary 2017-01-23 18:02 ` Adam C. Emerson @ 2017-01-23 18:43 ` Ilya Dryomov 2017-01-23 20:08 ` Loic Dachary [not found] ` <5d8fcfa6-3bf7-b29e-22a9-4d2864194afd@dachary.org> 1 sibling, 2 replies; 10+ messages in thread From: Ilya Dryomov @ 2017-01-23 18:43 UTC (permalink / raw) To: Loic Dachary; +Cc: Ceph Development On Mon, Jan 23, 2017 at 6:00 PM, Loic Dachary <loic@dachary.org> wrote: > Hi Adam, > > On 01/23/2017 05:29 PM, Adam C. Emerson wrote: >> On 21/01/2017, Loic Dachary wrote: >> [snip] >>> The beginning of the next ceph release cycle seems like a good time >>> to initiate that effort. If someone is willing to help with the >>> bootstrap, now is a good time. The library will be hosted under the >>> libcrush.org domain name. But that's about the only thing that >>> happened so far :-) >> >> Would the Ceph git repository continue to have its own version of >> CRUSH included, or would we pull in libcrush as a submodule? > > If all goes well and libcrush becomes actively and reliably maintained outside of the Ceph tree, it could either be a submodule or even a packaged dependency. Before that happens things can stay the way they are. I volunteer to manually monitor and cherry-pick whatever changes in the Ceph tree onto the standalone repository ( http://libcrush.org/main/libcrush/ ) so that it does not drift away. > >> Also, does this affect the relationship between userspace CRUSH and >> Linux kernel CRUSH? > > I don't know :-) Unless I'm mistaken the code in the linux kernel is an unmodified copy of the code from src/crush/*.[ch]. Correct. libcrush won't impact the kernel in any way, as long as the C core <-> C++ wrappers boundary stays intact, which it should ;) Some documentation will probably need to be added to cover things like basic kernel compatibility (no floating point, stubs for asserts, etc) and coding style for the C bits. The core algorithm is far from where libcrush initial activities would be centered though, I imagine. > I however have no clue how it is tested to work as it should in both contexts. Other than running integration tests with teuthology. > > How do you suggest we go about it ? I don't think we need anything special here. Thanks, Ilya ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Standalone libcrush: call for participation 2017-01-23 18:43 ` Ilya Dryomov @ 2017-01-23 20:08 ` Loic Dachary 2017-01-23 21:27 ` Ilya Dryomov [not found] ` <5d8fcfa6-3bf7-b29e-22a9-4d2864194afd@dachary.org> 1 sibling, 1 reply; 10+ messages in thread From: Loic Dachary @ 2017-01-23 20:08 UTC (permalink / raw) To: Ilya Dryomov; +Cc: Ceph Development Hi Ilya, On 01/23/2017 07:43 PM, Ilya Dryomov wrote: > On Mon, Jan 23, 2017 at 6:00 PM, Loic Dachary <loic@dachary.org> wrote: >> Hi Adam, >> >> On 01/23/2017 05:29 PM, Adam C. Emerson wrote: >>> On 21/01/2017, Loic Dachary wrote: >>> [snip] >>>> The beginning of the next ceph release cycle seems like a good time >>>> to initiate that effort. If someone is willing to help with the >>>> bootstrap, now is a good time. The library will be hosted under the >>>> libcrush.org domain name. But that's about the only thing that >>>> happened so far :-) >>> >>> Would the Ceph git repository continue to have its own version of >>> CRUSH included, or would we pull in libcrush as a submodule? >> >> If all goes well and libcrush becomes actively and reliably maintained outside of the Ceph tree, it could either be a submodule or even a packaged dependency. Before that happens things can stay the way they are. I volunteer to manually monitor and cherry-pick whatever changes in the Ceph tree onto the standalone repository ( http://libcrush.org/main/libcrush/ ) so that it does not drift away. >> >>> Also, does this affect the relationship between userspace CRUSH and >>> Linux kernel CRUSH? >> >> I don't know :-) Unless I'm mistaken the code in the linux kernel is an unmodified copy of the code from src/crush/*.[ch]. > > Correct. libcrush won't impact the kernel in any way, as long as the > C core <-> C++ wrappers boundary stays intact, which it should ;) Yeah :-) Just rephrasing so I'm sure I understand you correctly: the important thing is that the C API stays as it is (functions & structs & constants) and that the semantic of each function is backward compatible. As long as it holds true, the kernel integration is going to be ok. > Some documentation will probably need to be added to cover things like > basic kernel compatibility (no floating point, stubs for asserts, etc) > and coding style for the C bits. Ack. > The core algorithm is far from where > libcrush initial activities would be centered though, I imagine. Indeed, more documentation & packaging right now. >> I however have no clue how it is tested to work as it should in both contexts. Other than running integration tests with teuthology. >> >> How do you suggest we go about it ? > > I don't think we need anything special here. When something changes in the .[ch] files, how do you ensure it's not breaking something ? Cheers -- Loïc Dachary, Artisan Logiciel Libre ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Standalone libcrush: call for participation 2017-01-23 20:08 ` Loic Dachary @ 2017-01-23 21:27 ` Ilya Dryomov 0 siblings, 0 replies; 10+ messages in thread From: Ilya Dryomov @ 2017-01-23 21:27 UTC (permalink / raw) To: Loic Dachary; +Cc: Ceph Development On Mon, Jan 23, 2017 at 9:08 PM, Loic Dachary <loic@dachary.org> wrote: > Hi Ilya, > > On 01/23/2017 07:43 PM, Ilya Dryomov wrote: >> On Mon, Jan 23, 2017 at 6:00 PM, Loic Dachary <loic@dachary.org> wrote: >>> Hi Adam, >>> >>> On 01/23/2017 05:29 PM, Adam C. Emerson wrote: >>>> On 21/01/2017, Loic Dachary wrote: >>>> [snip] >>>>> The beginning of the next ceph release cycle seems like a good time >>>>> to initiate that effort. If someone is willing to help with the >>>>> bootstrap, now is a good time. The library will be hosted under the >>>>> libcrush.org domain name. But that's about the only thing that >>>>> happened so far :-) >>>> >>>> Would the Ceph git repository continue to have its own version of >>>> CRUSH included, or would we pull in libcrush as a submodule? >>> >>> If all goes well and libcrush becomes actively and reliably maintained outside of the Ceph tree, it could either be a submodule or even a packaged dependency. Before that happens things can stay the way they are. I volunteer to manually monitor and cherry-pick whatever changes in the Ceph tree onto the standalone repository ( http://libcrush.org/main/libcrush/ ) so that it does not drift away. >>> >>>> Also, does this affect the relationship between userspace CRUSH and >>>> Linux kernel CRUSH? >>> >>> I don't know :-) Unless I'm mistaken the code in the linux kernel is an unmodified copy of the code from src/crush/*.[ch]. >> >> Correct. libcrush won't impact the kernel in any way, as long as the >> C core <-> C++ wrappers boundary stays intact, which it should ;) > > Yeah :-) Just rephrasing so I'm sure I understand you correctly: the important thing is that the C API stays as it is (functions & structs & constants) and that the semantic of each function is backward compatible. As long as it holds true, the kernel integration is going to be ok. Well, that's a bit too strict. Ideally, we need to able to cut and-paste these files into the kernel with no or little effort. The API can change -- it just needs to be possible to bring the changes over to the kernel client. > >> Some documentation will probably need to be added to cover things like >> basic kernel compatibility (no floating point, stubs for asserts, etc) >> and coding style for the C bits. > > Ack. > >> The core algorithm is far from where >> libcrush initial activities would be centered though, I imagine. > > Indeed, more documentation & packaging right now. > >>> I however have no clue how it is tested to work as it should in both contexts. Other than running integration tests with teuthology. >>> >>> How do you suggest we go about it ? >> >> I don't think we need anything special here. > > When something changes in the .[ch] files, how do you ensure it's not breaking something ? Run the k* test suites, mostly. For invasive changes (like a new tunable), instrument the kernel to dump the crushtool --show-mappings output to the console and compare it with the .t, but that's a manual process that requires writing some code. Such changes happen around twice a year, so it hasn't been a problem. Thanks, Ilya ^ permalink raw reply [flat|nested] 10+ messages in thread
[parent not found: <5d8fcfa6-3bf7-b29e-22a9-4d2864194afd@dachary.org>]
* Re: Standalone libcrush: call for participation [not found] ` <5d8fcfa6-3bf7-b29e-22a9-4d2864194afd@dachary.org> @ 2017-01-23 21:44 ` Ilya Dryomov 2017-01-23 21:59 ` Loic Dachary 0 siblings, 1 reply; 10+ messages in thread From: Ilya Dryomov @ 2017-01-23 21:44 UTC (permalink / raw) To: Loic Dachary; +Cc: Ceph Development On Mon, Jan 23, 2017 at 9:16 PM, Loic Dachary <loic@dachary.org> wrote: > >> Some documentation will probably need to be added to cover things like >> basic kernel compatibility (no floating point, stubs for asserts, etc) >> and coding style for the C bits. > > I added this > > http://libcrush.org/main/libcrush/blob/wip-2-doxygen/doc/contributing.md#linux-kernel > > Do you have anything else in mind ? No need to try to list all the kernel quirks -- just point to crush_compat.h. These are the files that are shared: - crush.[ch] - crush_ln_table.h - hash.[ch] - mapper.[ch] Everything else is fair game, notably builder.[ch]. Thanks, Ilya ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Standalone libcrush: call for participation 2017-01-23 21:44 ` Ilya Dryomov @ 2017-01-23 21:59 ` Loic Dachary 2017-01-24 10:08 ` Ilya Dryomov 0 siblings, 1 reply; 10+ messages in thread From: Loic Dachary @ 2017-01-23 21:59 UTC (permalink / raw) To: Ilya Dryomov; +Cc: Ceph Development On 01/23/2017 10:44 PM, Ilya Dryomov wrote: > On Mon, Jan 23, 2017 at 9:16 PM, Loic Dachary <loic@dachary.org> wrote: >> >>> Some documentation will probably need to be added to cover things like >>> basic kernel compatibility (no floating point, stubs for asserts, etc) >>> and coding style for the C bits. >> >> I added this >> >> http://libcrush.org/main/libcrush/blob/wip-2-doxygen/doc/contributing.md#linux-kernel >> >> Do you have anything else in mind ? > > No need to try to list all the kernel quirks -- just point to > crush_compat.h. > > These are the files that are shared: > > - crush.[ch] > - crush_ln_table.h > - hash.[ch] > - mapper.[ch] > > Everything else is fair game, notably builder.[ch]. Understood :-) Is this http://libcrush.org/main/libcrush/blob/wip-2-doxygen/doc/mainpage.dox#L47 better ? Cheers -- Loïc Dachary, Artisan Logiciel Libre ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Standalone libcrush: call for participation 2017-01-23 21:59 ` Loic Dachary @ 2017-01-24 10:08 ` Ilya Dryomov 0 siblings, 0 replies; 10+ messages in thread From: Ilya Dryomov @ 2017-01-24 10:08 UTC (permalink / raw) To: Loic Dachary; +Cc: Ceph Development On Mon, Jan 23, 2017 at 10:59 PM, Loic Dachary <loic@dachary.org> wrote: > > > On 01/23/2017 10:44 PM, Ilya Dryomov wrote: >> On Mon, Jan 23, 2017 at 9:16 PM, Loic Dachary <loic@dachary.org> wrote: >>> >>>> Some documentation will probably need to be added to cover things like >>>> basic kernel compatibility (no floating point, stubs for asserts, etc) >>>> and coding style for the C bits. >>> >>> I added this >>> >>> http://libcrush.org/main/libcrush/blob/wip-2-doxygen/doc/contributing.md#linux-kernel >>> >>> Do you have anything else in mind ? >> >> No need to try to list all the kernel quirks -- just point to >> crush_compat.h. >> >> These are the files that are shared: >> >> - crush.[ch] >> - crush_ln_table.h >> - hash.[ch] >> - mapper.[ch] >> >> Everything else is fair game, notably builder.[ch]. > > Understood :-) > > Is this http://libcrush.org/main/libcrush/blob/wip-2-doxygen/doc/mainpage.dox#L47 better ? Looks good to me. Thanks, Ilya ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2017-01-24 10:08 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-01-20 23:16 Standalone libcrush: call for participation Loic Dachary
2017-01-23 16:29 ` Adam C. Emerson
2017-01-23 17:00 ` Loic Dachary
2017-01-23 18:02 ` Adam C. Emerson
2017-01-23 18:43 ` Ilya Dryomov
2017-01-23 20:08 ` Loic Dachary
2017-01-23 21:27 ` Ilya Dryomov
[not found] ` <5d8fcfa6-3bf7-b29e-22a9-4d2864194afd@dachary.org>
2017-01-23 21:44 ` Ilya Dryomov
2017-01-23 21:59 ` Loic Dachary
2017-01-24 10:08 ` Ilya Dryomov
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.