From mboxrd@z Thu Jan 1 00:00:00 1970 From: Matt Benjamin Subject: Re: inline vector container Date: Thu, 4 Feb 2016 12:03:05 -0500 (EST) Message-ID: <1830420093.32258943.1454605385127.JavaMail.zimbra@redhat.com> References: <1382260306.31547015.1454533040094.JavaMail.zimbra@redhat.com> <59448274.31581132.1454534771209.JavaMail.zimbra@redhat.com> <1674294591.31606738.1454537442587.JavaMail.zimbra@redhat.com> <56B33EB6.5030004@digiware.nl> <1606323224.32079941.1454596978667.JavaMail.zimbra@redhat.com> <56B383EB.6020207@digiware.nl> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: QUOTED-PRINTABLE Return-path: Received: from mx6-phx2.redhat.com ([209.132.183.39]:51505 "EHLO mx6-phx2.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750941AbcBDRDK convert rfc822-to-8bit (ORCPT ); Thu, 4 Feb 2016 12:03:10 -0500 In-Reply-To: <56B383EB.6020207@digiware.nl> Sender: ceph-devel-owner@vger.kernel.org List-ID: To: Willem Jan Withagen Cc: Allen Samuels , Casey Bodley , Sage Weil , ceph-devel@vger.kernel.org Agree, we're all trying to avoid that. Matt ----- Original Message ----- > From: "Willem Jan Withagen" > To: "Matt Benjamin" > Cc: "Allen Samuels" , "Casey Bodley" , "Sage Weil" , > ceph-devel@vger.kernel.org > Sent: Thursday, February 4, 2016 12:01:31 PM > Subject: Re: inline vector container >=20 > On 4-2-2016 15:42, Matt Benjamin wrote: > > Hi Willem, > >=20 > > Milosz pasted a file inline. Facebook's upstream repo is > > https://github.com/facebook/folly (Mr. Google), but what I was > > agreeing is, for the present, if we can't realize boost small_vecto= r, > > we might adapt just that type from Folly for the short term (that w= as > > Milosz meaning, in this thread). > >=20 > > (So don't panic ;) >=20 > It'll take a bit more to panic, but it would be a shame if we have to > port something for the intermediate 2-times. And then to only throw i= t > out a bit later. >=20 > Where I still have to look at "trivia" like AIO, and RBD... >=20 > Although my main interest is to get RBD in the freebsd guest DOM's > running, especially bhyve. So I hope that that is going to be a bit l= ess > convoluted than writting a RBD device in the kernel. Especially since > only recently I saw patches to add VirtFS to the bhyve virt-io infra. >=20 > --WjW >=20 > >=20 > > Matt > >=20 > > ----- Original Message ----- > >> From: "Willem Jan Withagen" To: "Matt Benjamin" > >> , "Allen Samuels" > >> Cc: "Casey Bodley" > >> , "Sage Weil" , > >> ceph-devel@vger.kernel.org Sent: Thursday, February 4, 2016 7:06:1= 4 > >> AM Subject: Re: inline vector container > >>=20 > >> On 3-2-2016 23:10, Matt Benjamin wrote: > >>> I'll prioritize this. Using more of folly may be a worthy > >>> project as well, but that can wait for later... :) > >>=20 > >> Hi Matt, > >>=20 > >> Can you give me a URL for the folly stuff. Since I cannot find any > >> reference to it in the FreeBSD ports. > >>=20 > >> I think I'd otherwise prefer Boost, since the ports already > >> contains a fair amount of Boost stuff. > >>=20 > >> Currently I have installed: boost-all-1.55.0 The "meta-port" > >> for boost libraries boost-docs-1.55.0 Documentation for > >> libraries from boost.org boost-jam-1.55.0 Build tool from the > >> boost.org boost-libs-1.55.0_9 Free portable C++ libraries > >> (without Boost.Python) > >>=20 > >> But I see that the new ports have migrated to: > >> /usr/ports/devel/boost_build/Makefile:PORTVERSION=3D 2.0.m12 > >>=20 > >> So that would move it beyond 1.6, I presume. > >>=20 > >> --WjW > >>=20 > >>=20 > >>> tx! > >>>=20 > >>> Matt > >>>=20 > >>> ----- Original Message ----- > >>>> From: "Allen Samuels" To: "Casey > >>>> Bodley" , "Matt Benjamin" > >>>> Cc: "Sage Weil" , > >>>> ceph-devel@vger.kernel.org Sent: Wednesday, February 3, 2016 > >>>> 5:06:57 PM Subject: RE: inline vector container > >>>>=20 > >>>> I would say that just pulling the latest boost into the tree > >>>> and using it for builds is the best path forward. But I'm in no > >>>> way competent to make that kind of change in the build system > >>>> -- especially during the automake cmake changeover period. > >>>>=20 > >>>>=20 > >>>> Allen Samuels Software Architect, Fellow, Systems and Software > >>>> Solutions > >>>>=20 > >>>> 2880 Junction Avenue, San Jose, CA 95134 T: +1 408 801 7030| M: > >>>> +1 408 780 6416 allen.samuels@SanDisk.com > >>>>=20 > >>>>=20 > >>>> -----Original Message----- From: Casey Bodley > >>>> [mailto:cbodley@redhat.com] Sent: Wednesday, February 03, 2016 > >>>> 1:26 PM To: Matt Benjamin Cc: Sage Weil > >>>> ; Allen Samuels ; > >>>> ceph-devel@vger.kernel.org Subject: Re: inline vector > >>>> container > >>>>=20 > >>>> Hi Allen, > >>>>=20 > >>>> I did spend some time a couple months ago experimenting with a > >>>> custom allocator with inline storage for standard containers. > >>>> You can find my branch on github at > >>>> https://github.com/cbodley/ceph/commits/wip-preallocator. > >>>>=20 > >>>> I was aiming for a general solution that worked with > >>>> non-contiguous containers, but couldn't find a good way to > >>>> handle deallocations. So the approach was really only a good > >>>> fit for std::vector. > >>>>=20 > >>>> I also had trouble getting it to do the right thing with the > >>>> copy and move operators. I found that it was trying to use the > >>>> new allocator to release memory that came from the old > >>>> allocator. Presumably there's a way to make this work using > >>>> std::allocator_traits; have you had better luck here? I'd love > >>>> to see your prototype. > >>>>=20 > >>>> I tend to agree that we should go with the boost solution if we > >>>> can. But if a) we can't use boost::small_vector in the > >>>> near-term due to versioning, and b) we can get a custom version > >>>> tested and working, it might be worth pursuing. > >>>>=20 > >>>> Casey > >>>>=20 > >>>> ----- Original Message ----- > >>>>> Hi, > >>>>>=20 > >>>>> I think we should go with small_vector for this role, yes. I > >>>>> found two issues. First, there are critical defects in > >>>>> small_vector prior to boost 1.6.0. Second, though it is > >>>>> header-only, the coupling with related version-specific boost > >>>>> types (container, allocator, respective detail types, and > >>>>> platform-specific selectors...) seems to make pulling it in > >>>>> separately pretty unrealistic as an option. I looked at the > >>>>> cost of pulling in and selectively compiling (libs) an > >>>>> entire boost 1.6, and that went a lot easier (added less than > >>>>> a minute to my build time). > >>>>>=20 > >>>>> Matt > >>>>>=20 > >>>>> ----- Original Message ----- > >>>>>> From: "Sage Weil" To: "Allen Samuels" > >>>>>> Cc: ceph-devel@vger.kernel.org, > >>>>>> mbenjami@redhat.com, cbodley@redhat.com Sent: Wednesday, > >>>>>> February 3, 2016 3:04:57 PM Subject: Re: inline vector > >>>>>> container > >>>>>>=20 > >>>>>> On Wed, 3 Feb 2016, Allen Samuels wrote: > >>>>>>> One thing that the code base needs is a simple vector > >>>>>>> container that is optimized for a small number of > >>>>>>> elements, i.e., for small element counts we avoid the > >>>>>>> malloc/free overhead. But fully supports being resized > >>>>>>> into larger containers that are unbounded. Of course it > >>>>>>> will need to follow all of the proper object copy/move > >>>>>>> semantics so that things like unique_ptr, etc work > >>>>>>> correctly. > >>>>>>>=20 > >>>>>>> I heard rumors that there was one available in the boost > >>>>>>> world, but I can=E2=80=99t seem to find it. Do you know about= one > >>>>>>> being available? > >>>>>>=20 > >>>>>> http://www.boost.org/doc/libs/1_60_0/doc/html/boost/container/= small_ > >>>>>> > >>>>>>=20 > vector.html > >>>>>>=20 > >>>>>>> I=E2=80=99ve been prototyping one myself, just to get re-fami= liar > >>>>>>> with all of the C++11 move sementics, and trivial > >>>>>>> constructors, etc. > >>>>>>>=20 > >>>>>>> The code is pretty complete, but the testing gets rather > >>>>>>> involved. There=E2=80=99s no reason to contain this if it=E2=80= =99s > >>>>>>> available elsewhere. But if it isn=E2=80=99t I plan on contin= uing > >>>>>>> this=E2=80=A6 > >>>>>>=20 > >>>>>> I think Casey had prototyped something similar using a > >>>>>> custom allocator, too, but I didn't actually look at the > >>>>>> result. And Matt talked about just pulling the boost one > >>>>>> in-tree until we get updated boost in the supported > >>>>>> distros, but I forget if he ran into problems there or > >>>>>> not... > >>>>>>=20 > >>>>>> sage > >>>>>=20 > >>>>> -- -- Matt Benjamin Red Hat, Inc. 315 West Huron Street, > >>>>> Suite 140A Ann Arbor, Michigan 48103 > >>>>>=20 > >>>>> http://www.redhat.com/en/technologies/storage > >>>>>=20 > >>>>> tel. 734-707-0660 fax. 734-769-8938 cel. 734-216-5309 -- > >>>>> To unsubscribe from this list: send the line "unsubscribe > >>>>> ceph-devel" in the body of a message to > >>>>> majordomo@vger.kernel.org More majordomo info at > >>>>> http://vger.kernel.org/majordomo-info.html > >>>>>=20 > >>>> PLEASE NOTE: The information contained in this electronic mail > >>>> message is intended only for the use of the designated > >>>> recipient(s) named above. If the reader of this message is not > >>>> the intended recipient, you are hereby notified that you have > >>>> received this message in error and that any review, > >>>> dissemination, distribution, or copying of this message is > >>>> strictly prohibited. If you have received this communication in > >>>> error, please notify the sender by telephone or e-mail (as > >>>> shown above) immediately and destroy any and all copies of this > >>>> message in your possession (whether hard copies or > >>>> electronically stored copies). > >>>>=20 > >>>=20 > >>=20 > >> -- To unsubscribe from this list: send the line "unsubscribe > >> ceph-devel" in the body of a message to majordomo@vger.kernel.org > >> More majordomo info at http://vger.kernel.org/majordomo-info.html > >>=20 > >=20 >=20 >=20 --=20 --=20 Matt Benjamin Red Hat, Inc. 315 West Huron Street, Suite 140A Ann Arbor, Michigan 48103 http://www.redhat.com/en/technologies/storage tel. 734-707-0660 fax. 734-769-8938 cel. 734-216-5309 -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" i= n the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html