From mboxrd@z Thu Jan 1 00:00:00 1970 From: Brad Hubbard Subject: Re: increasingly large packages and longer build times Date: Mon, 28 Aug 2017 08:30:55 +1000 Message-ID: References: <71c7b782-6d7e-1e06-e889-bbbf9004ebfe@suse.cz> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Return-path: Received: from mail-lf0-f48.google.com ([209.85.215.48]:35318 "EHLO mail-lf0-f48.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751352AbdH0Wa6 (ORCPT ); Sun, 27 Aug 2017 18:30:58 -0400 Received: by mail-lf0-f48.google.com with SMTP id k186so15740457lfe.2 for ; Sun, 27 Aug 2017 15:30:57 -0700 (PDT) In-Reply-To: Sender: ceph-devel-owner@vger.kernel.org List-ID: To: Sage Weil Cc: Alfredo Deza , kefu chai , Nathan Cutler , Gregory Farnum , ceph-devel , Ken Dreyer On Thu, Aug 24, 2017 at 11:36 PM, Sage Weil wrote: > On Thu, 24 Aug 2017, Alfredo Deza wrote: >> On Thu, Aug 24, 2017 at 4:41 AM, kefu chai wrote: >> > On Wed, Aug 23, 2017 at 2:58 AM, Alfredo Deza wrote: >> >> On Tue, Aug 22, 2017 at 9:35 AM, kefu chai wrote: >> >>> On Tue, Aug 22, 2017 at 4:27 PM, Nathan Cutler wrote: >> >>>>> my main concern would be the downstream. how shall we accommodate the >> >>>>> packaging of downstream? for example, what if the boost package >> >>>>> maintainers of SuSE/fedora/debian/ubuntu are not ready to package the >> >>>>> boost version we want to use in future? >> >>>>> >> >>>>> but as long as we don't require newer boost to build, we are safe on >> >>>>> debian and ubuntu at this moment. as boost 1.61 is required for >> >>>>> building ceph, and both debian unstable and ubuntu artful package >> >>>>> boost v1.62. >> >>>> >> >>>> >> >>>> The very latest cutting-edge versions of the distros may ship boost >= 1.61 >> >>>> but the stable versions most likely do not. >> >>> >> >>> yeah. but, IIRC, debian stable does not accepts new packages unless >> >>> they contain critical bug fixes. the new packages will go to the >> >>> unstable or experimental distribution first. so, presumably, debian >> >>> will be fine. guess ubuntu is using similar strategy for including >> >>> packages in its LTS distros. >> >> >> >> Why are you concerned with distros and the availability to have a >> >> package at the version that we need? >> >> >> >> We publish our own repos, where we could have whatever boost version >> >> we need. Distro package maintainers have to >> >> decide what they can or can't do. For us it shouldn't matter. >> >> >> > >> > i think it matters. and i believe it'd be desirable if Ceph can be >> > easier for downstream to package. if the downstream finds that Ceph is >> > too difficult to package, and give up, that's surely not the end of >> > the world. but it could decrease the popularity level of ceph, and in >> > long term, it might hurt Ceph. >> >> I fully agree here Kefu, but we don't make it easier by embedding >> libraries! Those need to >> get removed in distros. Most distros will *not* allow embedding >> dependencies like we do. That means that someone (probably >> a package maintainer) will have to remove these, figure out what >> versions are needed, and attempt to make it >> work with whatever that distro version will provide. > > There are several categories here: > > Yes: > - A bunch of stuff we embed can definitely be separated out; we embedded > only because distros didn't have packages yet (e.g., zstd). As soon as > they are packages the submodules can be removed. > > Maybe: > - The big one, though, is boost, which is mostly headers... only a *tiny* > bit of code can be dynamically loaded, so there is minimal benefit to > using the distro package... except that 'git clone' and the build are > slightly faster. We could use a more up to date distro package, but we > won't be able to put it in el7. I worry that cases like this will be > problematic: even if we build the updated package, it may be undesireable > to require users to install a newer version on their system. Should we ship a ceph-boost, etc. package then that does not overwrite distro packages (installs in a different location)? > > No: > - Then there is stuff that's fast-moving and new and unlikely to be > helpful if separated out (spdk, dpdk). The distro packages will never be > up to date. I'm not sure they even have any dynamically-linkable code... > would have to check. > > - And then rocksdb and civetweb are truly embedded. We sometimes have to > carry modifications so it is a bad idea to use a distro package (our > modifications may be incompatible with other users on the system). > > Note that although distros complain about static linking, they have never > actually taken a stand on the issue with Ceph. *shrug* > > sage > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- Cheers, Brad