All of lore.kernel.org
 help / color / mirror / Atom feed
* building boost statically
@ 2016-04-28 22:47 Sage Weil
  2016-04-28 22:56 ` Robin H. Johnson
                   ` (2 more replies)
  0 siblings, 3 replies; 21+ messages in thread
From: Sage Weil @ 2016-04-28 22:47 UTC (permalink / raw)
  To: ceph-devel; +Cc: kdreyer

Hi all,

We'd really like to use the latest boost, mainly so that we can use 
small_vector in a zillion places and avoid extra memory allocations.  The 
distros, as always, are behind.

The thoguht is to bring it in as yet another submodule, build it 
statically (it's pretty fast), and link statically.  There are a handful 
of places where the distro boost links dynamically.  For example, on 
xenial, it's

libboost-iostreams1.58.0
libboost-program-options1.58.0
libboost-random1.58.0
libboost-system1.58.0
libboost-thread1.58.0

I'm not sure how we should deal with that.  We could simply link it 
statically.  Alternatively, we could ship packages.. but if we do that we 
might as well just conditionally build against the new features and then 
build a full set of backported boost packages for the distros we care 
about.

I'm not sure how to best proceed.  But it would be really great to get 
access to new boost features like small_vector...

sage


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: building boost statically
  2016-04-28 22:47 building boost statically Sage Weil
@ 2016-04-28 22:56 ` Robin H. Johnson
  2016-04-28 23:14   ` Sage Weil
  2016-04-28 23:05 ` Yehuda Sadeh-Weinraub
  2016-04-29  2:31 ` Haomai Wang
  2 siblings, 1 reply; 21+ messages in thread
From: Robin H. Johnson @ 2016-04-28 22:56 UTC (permalink / raw)
  To: ceph-devel

[-- Attachment #1: Type: text/plain, Size: 1151 bytes --]

On Thu, Apr 28, 2016 at 06:47:55PM -0400,  Sage Weil wrote:
> Hi all,
> 
> We'd really like to use the latest boost, mainly so that we can use 
> small_vector in a zillion places and avoid extra memory allocations.  The 
> distros, as always, are behind.
From a distribution perspective, LESS static building is better, so this
proposal is a concern, but I do realize that the Gentoo/ArchLinux/CoreOS
perspective is very different than the Ubuntu LTS perspective.

What's the state of having a more-up-to-date Boost in things like Ubuntu
backports, so that you don't have the overhead of having to maintain
your own boost packages or statically link?

If support for statically linking a newer Boost is brought it, please,
please keep dynamic boost builds as fully supported, for distributions
that can keep up to date. As a bonus, at some point in the future, when
the slower distributions catch up, you might be able to escape the
static again.

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Infrastructure Lead, Foundation Trustee
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 445 bytes --]

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: building boost statically
  2016-04-28 22:47 building boost statically Sage Weil
  2016-04-28 22:56 ` Robin H. Johnson
@ 2016-04-28 23:05 ` Yehuda Sadeh-Weinraub
  2016-04-29  2:31 ` Haomai Wang
  2 siblings, 0 replies; 21+ messages in thread
From: Yehuda Sadeh-Weinraub @ 2016-04-28 23:05 UTC (permalink / raw)
  To: Sage Weil; +Cc: ceph-devel, Dreyer, Ken

On Thu, Apr 28, 2016 at 3:47 PM, Sage Weil <sweil@redhat.com> wrote:
> Hi all,
>
> We'd really like to use the latest boost, mainly so that we can use
> small_vector in a zillion places and avoid extra memory allocations.  The
> distros, as always, are behind.
>
> The thoguht is to bring it in as yet another submodule, build it
> statically (it's pretty fast), and link statically.  There are a handful
> of places where the distro boost links dynamically.  For example, on
> xenial, it's
>
> libboost-iostreams1.58.0
> libboost-program-options1.58.0
> libboost-random1.58.0
> libboost-system1.58.0
> libboost-thread1.58.0
>
> I'm not sure how we should deal with that.  We could simply link it
> statically.  Alternatively, we could ship packages.. but if we do that we
> might as well just conditionally build against the new features and then
> build a full set of backported boost packages for the distros we care
> about.
>
> I'm not sure how to best proceed.  But it would be really great to get
> access to new boost features like small_vector...
>

If we need to have it as a submodule then build it conditionally. Then
make a decision per distro, whether we want to not use new
functionality, or backport packages, or have it linked statically.

Yehuda

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: building boost statically
  2016-04-28 22:56 ` Robin H. Johnson
@ 2016-04-28 23:14   ` Sage Weil
  2016-04-28 23:45     ` Matt Benjamin
  2016-04-29  7:01     ` Piotr Dałek
  0 siblings, 2 replies; 21+ messages in thread
From: Sage Weil @ 2016-04-28 23:14 UTC (permalink / raw)
  To: ceph-devel

On Thu, 28 Apr 2016, Robin H. Johnson wrote:
> On Thu, Apr 28, 2016 at 06:47:55PM -0400,  Sage Weil wrote:
> > Hi all,
> > 
> > We'd really like to use the latest boost, mainly so that we can use 
> > small_vector in a zillion places and avoid extra memory allocations.  The 
> > distros, as always, are behind.
>
> From a distribution perspective, LESS static building is better, so this
> proposal is a concern, but I do realize that the Gentoo/ArchLinux/CoreOS
> perspective is very different than the Ubuntu LTS perspective.
> 
> What's the state of having a more-up-to-date Boost in things like Ubuntu
> backports, so that you don't have the overhead of having to maintain
> your own boost packages or statically link?
>
> If support for statically linking a newer Boost is brought it, please,
> please keep dynamic boost builds as fully supported, for distributions
> that can keep up to date. As a bonus, at some point in the future, when
> the slower distributions catch up, you might be able to escape the
> static again.

Yeah, having the option to either build statically or dynamically against 
an up-to-date distro is probably the right carrot/stick combination to 
incentivize the distros to move to a newer boost.

Being able to conditionally not use the new stuff (e.g., typedef 
small_vector<> back to vector<>) may or may not work well, depending on 
which new thing we're trying to use.

FWIW, we made the static -> dynamic transition with leveldb because the 
distros complained and it was nothing but sadness--so so many hours wasted 
chasing bugs in ancient distro versions of leveldb, and a huge matrix of 
version possibilities that made it difficult to reproduce user issues.  I 
can't tell if there was ever an instance where it actually gained us 
anything (e.g., security update in leveldb), but I kind of doubt it.

sage

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: building boost statically
  2016-04-28 23:14   ` Sage Weil
@ 2016-04-28 23:45     ` Matt Benjamin
  2016-04-29  7:01     ` Piotr Dałek
  1 sibling, 0 replies; 21+ messages in thread
From: Matt Benjamin @ 2016-04-28 23:45 UTC (permalink / raw)
  To: Sage Weil; +Cc: ceph-devel

Hi,

----- Original Message -----
> From: "Sage Weil" <sage@newdream.net>
> To: ceph-devel@vger.kernel.org
> Sent: Thursday, April 28, 2016 7:14:15 PM
> Subject: Re: building boost statically
> 
> On Thu, 28 Apr 2016, Robin H. Johnson wrote:
> > On Thu, Apr 28, 2016 at 06:47:55PM -0400,  Sage Weil wrote:
> > > Hi all,
> > > 
> > > We'd really like to use the latest boost, mainly so that we can use
> > > small_vector in a zillion places and avoid extra memory allocations.  The
> > > distros, as always, are behind.
> >
> > From a distribution perspective, LESS static building is better, so this
> > proposal is a concern, but I do realize that the Gentoo/ArchLinux/CoreOS
> > perspective is very different than the Ubuntu LTS perspective.
> > 
> > What's the state of having a more-up-to-date Boost in things like Ubuntu
> > backports, so that you don't have the overhead of having to maintain
> > your own boost packages or statically link?
> >
> > If support for statically linking a newer Boost is brought it, please,
> > please keep dynamic boost builds as fully supported, for distributions
> > that can keep up to date. As a bonus, at some point in the future, when
> > the slower distributions catch up, you might be able to escape the
> > static again.
> 
> Yeah, having the option to either build statically or dynamically against
> an up-to-date distro is probably the right carrot/stick combination to
> incentivize the distros to move to a newer boost.

++

> 
> Being able to conditionally not use the new stuff (e.g., typedef
> small_vector<> back to vector<>) may or may not work well, depending on
> which new thing we're trying to use.

I wouldn't want to do that.

> 
> FWIW, we made the static -> dynamic transition with leveldb because the
> distros complained and it was nothing but sadness--so so many hours wasted
> chasing bugs in ancient distro versions of leveldb, and a huge matrix of
> version possibilities that made it difficult to reproduce user issues.  I
> can't tell if there was ever an instance where it actually gained us
> anything (e.g., security update in leveldb), but I kind of doubt it.

This reminds me--we seem to still have issues with distro leveldbs, rocksdbs and clang++.

Matt

> 
> sage
> --


-- 
Matt Benjamin
Red Hat, Inc.
315 West Huron Street, Suite 140A
Ann Arbor, Michigan 48103

http://www.redhat.com/en/technologies/storage

tel.  734-707-0660
fax.  734-769-8938
cel.  734-216-5309

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: building boost statically
  2016-04-28 22:47 building boost statically Sage Weil
  2016-04-28 22:56 ` Robin H. Johnson
  2016-04-28 23:05 ` Yehuda Sadeh-Weinraub
@ 2016-04-29  2:31 ` Haomai Wang
  2016-04-29 22:55   ` Ken Dreyer
  2 siblings, 1 reply; 21+ messages in thread
From: Haomai Wang @ 2016-04-29  2:31 UTC (permalink / raw)
  To: Sage Weil; +Cc: ceph-devel, Ken Dreyer

On Thu, Apr 28, 2016 at 5:47 PM, Sage Weil <sweil@redhat.com> wrote:
> Hi all,
>
> We'd really like to use the latest boost, mainly so that we can use
> small_vector in a zillion places and avoid extra memory allocations.  The
> distros, as always, are behind.
>
> The thoguht is to bring it in as yet another submodule, build it
> statically (it's pretty fast), and link statically.  There are a handful
> of places where the distro boost links dynamically.  For example, on
> xenial, it's
>
> libboost-iostreams1.58.0
> libboost-program-options1.58.0
> libboost-random1.58.0
> libboost-system1.58.0
> libboost-thread1.58.0

Yes, I'm always glad to see more deps removed!

And I think we can remove part of boost library deps totally to reduce
adding static lib size? like libboost-random and libboost-thread or
anything else. We may replace with c++11 lib(I'm not a boost fan....).

>
> I'm not sure how we should deal with that.  We could simply link it
> statically.  Alternatively, we could ship packages.. but if we do that we
> might as well just conditionally build against the new features and then
> build a full set of backported boost packages for the distros we care
> about.
>
> I'm not sure how to best proceed.  But it would be really great to get
> access to new boost features like small_vector...
>
> sage
>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: building boost statically
  2016-04-28 23:14   ` Sage Weil
  2016-04-28 23:45     ` Matt Benjamin
@ 2016-04-29  7:01     ` Piotr Dałek
  1 sibling, 0 replies; 21+ messages in thread
From: Piotr Dałek @ 2016-04-29  7:01 UTC (permalink / raw)
  To: ceph-devel

On Thu, Apr 28, 2016 at 07:14:15PM -0400, Sage Weil wrote:
> On Thu, 28 Apr 2016, Robin H. Johnson wrote:
> > If support for statically linking a newer Boost is brought it, please,
> > please keep dynamic boost builds as fully supported, for distributions
> > that can keep up to date. As a bonus, at some point in the future, when
> > the slower distributions catch up, you might be able to escape the
> > static again.
> 
> Yeah, having the option to either build statically or dynamically against 
> an up-to-date distro is probably the right carrot/stick combination to 
> incentivize the distros to move to a newer boost.

Plus, final binary sizes would be smaller if dynamic linking would succeed.
 
> Being able to conditionally not use the new stuff (e.g., typedef 
> small_vector<> back to vector<>) may or may not work well, depending on 
> which new thing we're trying to use.

That's bad idea, IMHO.
Why not check for features we need on configure state and use static or
dynamic accordingly?

-- 
Piotr Dałek
branch@predictor.org.pl
http://blog.predictor.org.pl
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: building boost statically
  2016-04-29  2:31 ` Haomai Wang
@ 2016-04-29 22:55   ` Ken Dreyer
  2016-04-30  1:43     ` Sage Weil
  0 siblings, 1 reply; 21+ messages in thread
From: Ken Dreyer @ 2016-04-29 22:55 UTC (permalink / raw)
  To: Haomai Wang; +Cc: Sage Weil, ceph-devel

On Thu, Apr 28, 2016 at 8:31 PM, Haomai Wang <haomai@xsky.com> wrote:
> Yes, I'm always glad to see more deps removed!

It's not removed, it's just moved. The Ceph project will then be on
the hook for all boost security updates.

I think we should bundle less not more. Build times are already long enough...

- Ken

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: building boost statically
  2016-04-29 22:55   ` Ken Dreyer
@ 2016-04-30  1:43     ` Sage Weil
  2016-04-30  3:24       ` Allen Samuels
  0 siblings, 1 reply; 21+ messages in thread
From: Sage Weil @ 2016-04-30  1:43 UTC (permalink / raw)
  To: Ken Dreyer; +Cc: Haomai Wang, ceph-devel

On Fri, 29 Apr 2016, Ken Dreyer wrote:
> On Thu, Apr 28, 2016 at 8:31 PM, Haomai Wang <haomai@xsky.com> wrote:
> > Yes, I'm always glad to see more deps removed!
> 
> It's not removed, it's just moved. The Ceph project will then be on
> the hook for all boost security updates.
> 
> I think we should bundle less not more.  Build times are already long enough...

If building separate packages and including them in all the repos is in 
fact easier, that seems like the right choice.  Static linkage makes 
development easier, but we can make do.

Whether it's static or dynamic, though, we're on the hook for updating the 
packages for bugs or security update either way.  Unless we just give up 
an wait for the distros to catch up, but I don't think that's a viable 
option...

sage


^ permalink raw reply	[flat|nested] 21+ messages in thread

* RE: building boost statically
  2016-04-30  1:43     ` Sage Weil
@ 2016-04-30  3:24       ` Allen Samuels
  2016-05-01 15:08         ` Jesse Williamson
  0 siblings, 1 reply; 21+ messages in thread
From: Allen Samuels @ 2016-04-30  3:24 UTC (permalink / raw)
  To: Sage Weil, Ken Dreyer; +Cc: Haomai Wang, ceph-devel

Let's not wait. 

As another option. If all we're looking for is "small_vector", then I looked at taking the equivalent out of the Folly library a while ago. It looked like it would be relatively easy to hack up the one header file to make it standalone. Then we could just include it in the Ceph source tree as a common file (it's a header-only thing). Once we're confident that all of the relevant distro's are updated we can easily dispose of the file and do a top level rename. 


Allen Samuels
Software Architect, Fellow, Systems and Software Solutions 

2880 Junction Avenue, San Jose, CA 95134
T: +1 408 801 7030| M: +1 408 780 6416
allen.samuels@SanDisk.com

> -----Original Message-----
> From: ceph-devel-owner@vger.kernel.org [mailto:ceph-devel-
> owner@vger.kernel.org] On Behalf Of Sage Weil
> Sent: Friday, April 29, 2016 6:44 PM
> To: Ken Dreyer <kdreyer@redhat.com>
> Cc: Haomai Wang <haomai@xsky.com>; ceph-devel@vger.kernel.org
> Subject: Re: building boost statically
> 
> On Fri, 29 Apr 2016, Ken Dreyer wrote:
> > On Thu, Apr 28, 2016 at 8:31 PM, Haomai Wang <haomai@xsky.com>
> wrote:
> > > Yes, I'm always glad to see more deps removed!
> >
> > It's not removed, it's just moved. The Ceph project will then be on
> > the hook for all boost security updates.
> >
> > I think we should bundle less not more.  Build times are already long
> enough...
> 
> If building separate packages and including them in all the repos is in fact
> easier, that seems like the right choice.  Static linkage makes development
> easier, but we can make do.
> 
> Whether it's static or dynamic, though, we're on the hook for updating the
> packages for bugs or security update either way.  Unless we just give up an
> wait for the distros to catch up, but I don't think that's a viable option...
> 
> sage
> 
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the
> body of a message to majordomo@vger.kernel.org More majordomo info at
> http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 21+ messages in thread

* RE: building boost statically
  2016-04-30  3:24       ` Allen Samuels
@ 2016-05-01 15:08         ` Jesse Williamson
  2016-05-01 15:14           ` Sage Weil
  0 siblings, 1 reply; 21+ messages in thread
From: Jesse Williamson @ 2016-05-01 15:08 UTC (permalink / raw)
  To: Allen Samuels; +Cc: Sage Weil, Ken Dreyer, Haomai Wang, ceph-devel

On Sat, 30 Apr 2016, Allen Samuels wrote:

> Let's not wait.
>
> As another option. If all we're looking for is "small_vector", then I 
> looked at taking the equivalent out of the Folly library a while ago. It 
> looked like it would be relatively easy to hack up the one header file 
> to make it standalone.

There's a utility in boost, "bcp", that might be useful:

http://www.boost.org/doc/libs/1_60_0/tools/bcp/doc/html/index.html

-Jesse

^ permalink raw reply	[flat|nested] 21+ messages in thread

* RE: building boost statically
  2016-05-01 15:08         ` Jesse Williamson
@ 2016-05-01 15:14           ` Sage Weil
  2016-05-02  3:28             ` Adam C. Emerson
  2016-05-03 16:28             ` Robert LeBlanc
  0 siblings, 2 replies; 21+ messages in thread
From: Sage Weil @ 2016-05-01 15:14 UTC (permalink / raw)
  To: Jesse Williamson; +Cc: Allen Samuels, Ken Dreyer, Haomai Wang, ceph-devel

On Sun, 1 May 2016, Jesse Williamson wrote:
> On Sat, 30 Apr 2016, Allen Samuels wrote:
> 
> > Let's not wait.
> > 
> > As another option. If all we're looking for is "small_vector", then I looked
> > at taking the equivalent out of the Folly library a while ago. It looked
> > like it would be relatively easy to hack up the one header file to make it
> > standalone.
> 
> There's a utility in boost, "bcp", that might be useful:
> 
> http://www.boost.org/doc/libs/1_60_0/tools/bcp/doc/html/index.html

Given that this is the only piece we want immediately (that i know of), 
that there would be significant work to either pull boost inline as a 
submodule *or* to build and ship a proper package, let's go the bcp route 
for now.

Jesse, is this something you can tackle?

Thanks!
sage

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: building boost statically
  2016-05-01 15:14           ` Sage Weil
@ 2016-05-02  3:28             ` Adam C. Emerson
  2016-05-02 16:09               ` Jesse Williamson
  2016-05-03 16:28             ` Robert LeBlanc
  1 sibling, 1 reply; 21+ messages in thread
From: Adam C. Emerson @ 2016-05-02  3:28 UTC (permalink / raw)
  To: Sage Weil
  Cc: Jesse Williamson, Allen Samuels, Ken Dreyer, Haomai Wang, ceph-devel

On 01/05/2016, Sage Weil wrote:
> Given that this is the only piece we want immediately (that i know of), 
> that there would be significant work to either pull boost inline as a 
> submodule *or* to build and ship a proper package, let's go the bcp route 
> for now.
> 
> Jesse, is this something you can tackle?
> 

I'm not sure if I like this idea. There are other places where I've
wanted to use something newer than the boost on Trusty Tahr/RHEL7, like
the newer intrusive_ptr. Some other people have mentioned wishing for
newer boost functionality, too.

If we're going to bundle anything at all, I'd rather we bundle the
whole thing so we get the full advantage of trying to swallow the extra
capacity. BCP sounds like we're signing up for just as much work and
maintenance burden and doesn't give us the full benefit.

-- 
Senior Software Engineer           Red Hat Storage, Ann Arbor, MI, US
IRC: Aemerson@{RedHat, OFTC, Freenode}
0x80F7544B90EDBFB9 E707 86BA 0C1B 62CC 152C  7C12 80F7 544B 90ED BFB9

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: building boost statically
  2016-05-02  3:28             ` Adam C. Emerson
@ 2016-05-02 16:09               ` Jesse Williamson
  2016-05-02 16:14                 ` Adam C. Emerson
  0 siblings, 1 reply; 21+ messages in thread
From: Jesse Williamson @ 2016-05-02 16:09 UTC (permalink / raw)
  To: Adam C. Emerson
  Cc: Sage Weil, Allen Samuels, Ken Dreyer, Haomai Wang, ceph-devel

On Sun, 1 May 2016, Adam C. Emerson wrote:

>> Jesse, is this something you can tackle?

I'll have a look at it this morning and see how it goes.

> I'm not sure if I like this idea. There are other places where I've
> wanted to use something newer than the boost on Trusty Tahr/RHEL7, like
> the newer intrusive_ptr. Some other people have mentioned wishing for
> newer boost functionality, too.

I think that in the long run, rather than bundling or using bcp, it would 
be nice to have cmake/autoconf let you point to the boost version you'd 
like to use.

-Jesse


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: building boost statically
  2016-05-02 16:09               ` Jesse Williamson
@ 2016-05-02 16:14                 ` Adam C. Emerson
  2016-05-02 16:18                   ` Jesse Williamson
  2016-05-02 23:04                   ` Matt Benjamin
  0 siblings, 2 replies; 21+ messages in thread
From: Adam C. Emerson @ 2016-05-02 16:14 UTC (permalink / raw)
  To: Jesse Williamson
  Cc: Sage Weil, Allen Samuels, Ken Dreyer, Haomai Wang, ceph-devel

On 02/05/2016, Jesse Williamson wrote:
> I think that in the long run, rather than bundling or using bcp, it would be
> nice to have cmake/autoconf let you point to the boost version you'd like to
> use.

If we could manage it, that would be my ideal world. Rather than
making it A Submodule, say, if we could have cmake examine the system
boost to see what version it is and, if the version is greater than or
equal to the required version, just use it. Otherwise just fetch the
required version and build against it statically.

-- 
Senior Software Engineer           Red Hat Storage, Ann Arbor, MI, US
IRC: Aemerson@{RedHat, OFTC, Freenode}
0x80F7544B90EDBFB9 E707 86BA 0C1B 62CC 152C  7C12 80F7 544B 90ED BFB9

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: building boost statically
  2016-05-02 16:14                 ` Adam C. Emerson
@ 2016-05-02 16:18                   ` Jesse Williamson
  2016-05-02 16:31                     ` Adam C. Emerson
  2016-05-02 23:04                   ` Matt Benjamin
  1 sibling, 1 reply; 21+ messages in thread
From: Jesse Williamson @ 2016-05-02 16:18 UTC (permalink / raw)
  To: Adam C. Emerson
  Cc: Sage Weil, Allen Samuels, Ken Dreyer, Haomai Wang, ceph-devel

On Mon, 2 May 2016, Adam C. Emerson wrote:

> If we could manage it, that would be my ideal world. Rather than
> making it A Submodule, say, if we could have cmake examine the system
> boost to see what version it is and, if the version is greater than or
> equal to the required version, just use it. Otherwise just fetch the
> required version and build against it statically.

Auto-fetching would be nice. One caution against just going with 
greater-than is that some libraries in the Boost project introduce 
breaking changes now and again, so being able to go with a specific 
version can be important.

-Jesse


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: building boost statically
  2016-05-02 16:18                   ` Jesse Williamson
@ 2016-05-02 16:31                     ` Adam C. Emerson
  0 siblings, 0 replies; 21+ messages in thread
From: Adam C. Emerson @ 2016-05-02 16:31 UTC (permalink / raw)
  To: Jesse Williamson
  Cc: Sage Weil, Allen Samuels, Ken Dreyer, Haomai Wang, ceph-devel

On 02/05/2016, Jesse Williamson wrote:
> Auto-fetching would be nice. One caution against just going with
> greater-than is that some libraries in the Boost project introduce breaking
> changes now and again, so being able to go with a specific version can be
> important.

Specific range maybe?

Or, if we required an exact match, had CMake print out that it is
going to fetch that version rather than using the system Boost, and we
had an option to override and tell it to use the system Boost anyway
that would be good.

That might be a good compromise between a distribution's desire to use
system packages and our desire not to be on the hook for versions
we've never tested against.

-- 
Senior Software Engineer           Red Hat Storage, Ann Arbor, MI, US
IRC: Aemerson@{RedHat, OFTC, Freenode}
0x80F7544B90EDBFB9 E707 86BA 0C1B 62CC 152C  7C12 80F7 544B 90ED BFB9

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: building boost statically
  2016-05-02 16:14                 ` Adam C. Emerson
  2016-05-02 16:18                   ` Jesse Williamson
@ 2016-05-02 23:04                   ` Matt Benjamin
  2016-05-03  1:01                     ` Jesse Williamson
  1 sibling, 1 reply; 21+ messages in thread
From: Matt Benjamin @ 2016-05-02 23:04 UTC (permalink / raw)
  To: Adam C. Emerson
  Cc: Jesse Williamson, Sage Weil, Allen Samuels, Ken Dreyer,
	Haomai Wang, ceph-devel

Just wanted to ++ this.

Matt

----- Original Message -----
> From: "Adam C. Emerson" <aemerson@redhat.com>
> To: "Jesse Williamson" <jwilliamson@suse.com>
> Cc: "Sage Weil" <sage@newdream.net>, "Allen Samuels" <Allen.Samuels@sandisk.com>, "Ken Dreyer" <kdreyer@redhat.com>,
> "Haomai Wang" <haomai@xsky.com>, ceph-devel@vger.kernel.org
> Sent: Monday, May 2, 2016 12:14:50 PM
> Subject: Re: building boost statically
> 
> On 02/05/2016, Jesse Williamson wrote:
> > I think that in the long run, rather than bundling or using bcp, it would
> > be
> > nice to have cmake/autoconf let you point to the boost version you'd like
> > to
> > use.
> 
> If we could manage it, that would be my ideal world. Rather than
> making it A Submodule, say, if we could have cmake examine the system
> boost to see what version it is and, if the version is greater than or
> equal to the required version, just use it. Otherwise just fetch the
> required version and build against it statically.
> 
> --
> Senior Software Engineer           Red Hat Storage, Ann Arbor, MI, US
> IRC: Aemerson@{RedHat, OFTC, Freenode}
> 0x80F7544B90EDBFB9 E707 86BA 0C1B 62CC 152C  7C12 80F7 544B 90ED BFB9
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

-- 
Matt Benjamin
Red Hat, Inc.
315 West Huron Street, Suite 140A
Ann Arbor, Michigan 48103

http://www.redhat.com/en/technologies/storage

tel.  734-707-0660
fax.  734-769-8938
cel.  734-216-5309

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: building boost statically
  2016-05-02 23:04                   ` Matt Benjamin
@ 2016-05-03  1:01                     ` Jesse Williamson
  2016-05-03  1:14                       ` Matt Benjamin
  0 siblings, 1 reply; 21+ messages in thread
From: Jesse Williamson @ 2016-05-03  1:01 UTC (permalink / raw)
  To: Matt Benjamin
  Cc: Adam C. Emerson, Sage Weil, Allen Samuels, Ken Dreyer,
	Haomai Wang, ceph-devel

On Mon, 2 May 2016, Matt Benjamin wrote:

So, just a quick update here: I used bcp to extract boost containers. This 
works nicely, but you end up with 1628 files. It's doing what it's 
/supposed/ to do, since part of the purpose is to provide all the 
dependencies (for example, Boost::MPL is used), but that seems like maybe 
not what we want to do.

Just using the boost/container headers is also possible, at a bit of risk 
that if we try to build on a system where one of the dependencies is 
incompatible in some way, Ceph won't build (perhaps a fairly low risk, but 
I'm throwing it out there). If that risk is acceptable, then we wouldn't 
have a large set of headers, but in effect would still be shipping our own 
little mini-fork.

CMake looks like it has a module that can help us, and the general 
direction during an IRC chat was that we should do this as part of the 
move to cmake, rather than fidding with autoconf.

-Jesse

> Just wanted to ++ this.
>
> Matt
>
> ----- Original Message -----
>> From: "Adam C. Emerson" <aemerson@redhat.com>
>> To: "Jesse Williamson" <jwilliamson@suse.com>
>> Cc: "Sage Weil" <sage@newdream.net>, "Allen Samuels" <Allen.Samuels@sandisk.com>, "Ken Dreyer" <kdreyer@redhat.com>,
>> "Haomai Wang" <haomai@xsky.com>, ceph-devel@vger.kernel.org
>> Sent: Monday, May 2, 2016 12:14:50 PM
>> Subject: Re: building boost statically
>>
>> On 02/05/2016, Jesse Williamson wrote:
>>> I think that in the long run, rather than bundling or using bcp, it would
>>> be
>>> nice to have cmake/autoconf let you point to the boost version you'd like
>>> to
>>> use.
>>
>> If we could manage it, that would be my ideal world. Rather than
>> making it A Submodule, say, if we could have cmake examine the system
>> boost to see what version it is and, if the version is greater than or
>> equal to the required version, just use it. Otherwise just fetch the
>> required version and build against it statically.
>>
>> --
>> Senior Software Engineer           Red Hat Storage, Ann Arbor, MI, US
>> IRC: Aemerson@{RedHat, OFTC, Freenode}
>> 0x80F7544B90EDBFB9 E707 86BA 0C1B 62CC 152C  7C12 80F7 544B 90ED BFB9
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>
> -- 
> Matt Benjamin
> Red Hat, Inc.
> 315 West Huron Street, Suite 140A
> Ann Arbor, Michigan 48103
>
> http://www.redhat.com/en/technologies/storage
>
> tel.  734-707-0660
> fax.  734-769-8938
> cel.  734-216-5309
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: building boost statically
  2016-05-03  1:01                     ` Jesse Williamson
@ 2016-05-03  1:14                       ` Matt Benjamin
  0 siblings, 0 replies; 21+ messages in thread
From: Matt Benjamin @ 2016-05-03  1:14 UTC (permalink / raw)
  To: Jesse Williamson
  Cc: Adam C. Emerson, Sage Weil, Allen Samuels, Ken Dreyer,
	Haomai Wang, ceph-devel

Hi Jesse,

----- Original Message -----
> From: "Jesse Williamson" <jwilliamson@suse.com>
> To: "Matt Benjamin" <mbenjamin@redhat.com>
> Cc: "Adam C. Emerson" <aemerson@redhat.com>, "Sage Weil" <sage@newdream.net>, "Allen Samuels"
> <Allen.Samuels@sandisk.com>, "Ken Dreyer" <kdreyer@redhat.com>, "Haomai Wang" <haomai@xsky.com>,
> ceph-devel@vger.kernel.org
> Sent: Monday, May 2, 2016 9:01:44 PM
> Subject: Re: building boost statically
> 
> On Mon, 2 May 2016, Matt Benjamin wrote:
> 
> So, just a quick update here: I used bcp to extract boost containers. This
> works nicely, but you end up with 1628 files. It's doing what it's
> /supposed/ to do, since part of the purpose is to provide all the
> dependencies (for example, Boost::MPL is used), but that seems like maybe
> not what we want to do.

Yes.  I experimented with this, but it didn't seem at all compact enough to be preferable to just building boost.  The latter, building just what Ceph uses from boost now, took less than a minute on a single core (i7).

> 
> Just using the boost/container headers is also possible, at a bit of risk
> that if we try to build on a system where one of the dependencies is
> incompatible in some way, Ceph won't build (perhaps a fairly low risk, but
> I'm throwing it out there). If that risk is acceptable, then we wouldn't
> have a large set of headers, but in effect would still be shipping our own
> little mini-fork.

I experimented with this also.  What I found was that the number of boost internal (including unexposed) dependencies is much larger than container.  This is a marvel of efficiency for the boost library internally, but isn't helpful for extracting a single like small_vector.  In particular, conflicts with an installed boost are many.

> 
> CMake looks like it has a module that can help us, and the general
> direction during an IRC chat was that we should do this as part of the
> move to cmake, rather than fidding with autoconf.

That seems like a good idea.

Matt

> 
> -Jesse
> 
> > Just wanted to ++ this.
> >
> > Matt
> >
> > ----- Original Message -----
> >> From: "Adam C. Emerson" <aemerson@redhat.com>
> >> To: "Jesse Williamson" <jwilliamson@suse.com>
> >> Cc: "Sage Weil" <sage@newdream.net>, "Allen Samuels"
> >> <Allen.Samuels@sandisk.com>, "Ken Dreyer" <kdreyer@redhat.com>,
> >> "Haomai Wang" <haomai@xsky.com>, ceph-devel@vger.kernel.org
> >> Sent: Monday, May 2, 2016 12:14:50 PM
> >> Subject: Re: building boost statically
> >>
> >> On 02/05/2016, Jesse Williamson wrote:
> >>> I think that in the long run, rather than bundling or using bcp, it would
> >>> be
> >>> nice to have cmake/autoconf let you point to the boost version you'd like
> >>> to
> >>> use.
> >>
> >> If we could manage it, that would be my ideal world. Rather than
> >> making it A Submodule, say, if we could have cmake examine the system
> >> boost to see what version it is and, if the version is greater than or
> >> equal to the required version, just use it. Otherwise just fetch the
> >> required version and build against it statically.
> >>
> >> --
> >> Senior Software Engineer           Red Hat Storage, Ann Arbor, MI, US
> >> IRC: Aemerson@{RedHat, OFTC, Freenode}
> >> 0x80F7544B90EDBFB9 E707 86BA 0C1B 62CC 152C  7C12 80F7 544B 90ED BFB9
> >> --
> >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> >> the body of a message to majordomo@vger.kernel.org
> >> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >>
> >
> > --
> > Matt Benjamin
> > Red Hat, Inc.
> > 315 West Huron Street, Suite 140A
> > Ann Arbor, Michigan 48103
> >
> > http://www.redhat.com/en/technologies/storage
> >
> > tel.  734-707-0660
> > fax.  734-769-8938
> > cel.  734-216-5309
> > --
> > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >
> >
> 

-- 
Matt Benjamin
Red Hat, Inc.
315 West Huron Street, Suite 140A
Ann Arbor, Michigan 48103

http://www.redhat.com/en/technologies/storage

tel.  734-707-0660
fax.  734-769-8938
cel.  734-216-5309

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: building boost statically
  2016-05-01 15:14           ` Sage Weil
  2016-05-02  3:28             ` Adam C. Emerson
@ 2016-05-03 16:28             ` Robert LeBlanc
  1 sibling, 0 replies; 21+ messages in thread
From: Robert LeBlanc @ 2016-05-03 16:28 UTC (permalink / raw)
  To: Sage Weil
  Cc: Jesse Williamson, Allen Samuels, Ken Dreyer, Haomai Wang, ceph-devel

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

I really wanted to use intrusive maps that was in boost 1.59 for the
WPQ. So I think there are multiple places that could leverage the new
boost features if it were available.
-----BEGIN PGP SIGNATURE-----
Version: Mailvelope v1.3.6
Comment: https://www.mailvelope.com

wsFcBAEBCAAQBQJXKNGPCRDmVDuy+mK58QAAr3cP+wbXVtaBFTkvJQSAH+TQ
5C8NOQcmph0Nmf/tL9byox8USML8OS4RiKh2EI6MnbRLa6ZrE65irnkBEXX3
b+OM1JysNbLnyo+myTn7MEtGfjQCG4rsm0JtHX69cs35kLRtLdvRj7YTiWkf
MWjt9nIuiLDWJH9TyhO9aBQp+t52m3u1Lxrl7XL9ie7vFq8eQp4yXvESvj9t
5xawxNp3xaDLZHt48qA2kmuA1S9oqymqNWJU4yqeKqZY1OtGE1FN5bg1cn0d
ApvuI250jyWEV0pjyp0T8Auq+E0x7+Fn52FRfyZzMNO3FeRzAMn+y2z8djT9
XHf6j3Y/PAF9udASVguaUHPg4OnUihM+ffGmMCkSrpoNcc/cWRpB8bRyG/12
UKbwwREYQ+UhENohoblkl3zTqn3E7891/d34mObcmZUQKy5i5i1l+wl33cZO
NtXY/xC567kXAUv1zKEx+2bSUhQTtA4rJh7Kqrgk/lyJrNozRsK3lhQbucx7
a45VA2GW9RqI9zX9GSiDDZiZinOMExcN+E+5DadsdeOPGqswE7lgmDF7BH0L
TIJjnr+4r4vSQ+4/8OEHP05Gh5OZv5p/penzF+LCvkboYDDZGfwwbfbNE04P
96An6l8Yf+L2XbhJfptR0155zjhyBRCvm8FK8M6O/3BckSDEC7IvYVMVgp1F
RB99
=SK2q
-----END PGP SIGNATURE-----
----------------
Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904  C70E E654 3BB2 FA62 B9F1


On Sun, May 1, 2016 at 9:14 AM, Sage Weil <sage@newdream.net> wrote:
> On Sun, 1 May 2016, Jesse Williamson wrote:
>> On Sat, 30 Apr 2016, Allen Samuels wrote:
>>
>> > Let's not wait.
>> >
>> > As another option. If all we're looking for is "small_vector", then I looked
>> > at taking the equivalent out of the Folly library a while ago. It looked
>> > like it would be relatively easy to hack up the one header file to make it
>> > standalone.
>>
>> There's a utility in boost, "bcp", that might be useful:
>>
>> http://www.boost.org/doc/libs/1_60_0/tools/bcp/doc/html/index.html
>
> Given that this is the only piece we want immediately (that i know of),
> that there would be significant work to either pull boost inline as a
> submodule *or* to build and ship a proper package, let's go the bcp route
> for now.
>
> Jesse, is this something you can tackle?
>
> Thanks!
> sage
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 21+ messages in thread

end of thread, other threads:[~2016-05-03 16:28 UTC | newest]

Thread overview: 21+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-04-28 22:47 building boost statically Sage Weil
2016-04-28 22:56 ` Robin H. Johnson
2016-04-28 23:14   ` Sage Weil
2016-04-28 23:45     ` Matt Benjamin
2016-04-29  7:01     ` Piotr Dałek
2016-04-28 23:05 ` Yehuda Sadeh-Weinraub
2016-04-29  2:31 ` Haomai Wang
2016-04-29 22:55   ` Ken Dreyer
2016-04-30  1:43     ` Sage Weil
2016-04-30  3:24       ` Allen Samuels
2016-05-01 15:08         ` Jesse Williamson
2016-05-01 15:14           ` Sage Weil
2016-05-02  3:28             ` Adam C. Emerson
2016-05-02 16:09               ` Jesse Williamson
2016-05-02 16:14                 ` Adam C. Emerson
2016-05-02 16:18                   ` Jesse Williamson
2016-05-02 16:31                     ` Adam C. Emerson
2016-05-02 23:04                   ` Matt Benjamin
2016-05-03  1:01                     ` Jesse Williamson
2016-05-03  1:14                       ` Matt Benjamin
2016-05-03 16:28             ` Robert LeBlanc

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.