Re: Understanding some of the Cmake logics

[kefu chai <tchaikov@gmail.com>]

On Tue, Oct 10, 2017 at 11:13 PM, Willem Jan Withagen <wjw@digiware.nl> wrote:
> On 10-10-2017 16:58, kefu chai wrote:
>> On Tue, Oct 10, 2017 at 9:54 PM, Willem Jan Withagen <wjw@digiware.nl> wrote:
>>> On 10-10-2017 15:31, kefu chai wrote:
>>>> On Tue, Oct 10, 2017 at 8:06 PM, Willem Jan Withagen <wjw@digiware.nl> wrote:
>>>>> On 10-10-2017 13:14, kefu chai wrote:
>>>>>> + ceph-devel
>>>>>>
>>>>>> On Tue, Oct 10, 2017 at 4:49 AM, Willem Jan Withagen <wjw@digiware.nl> wrote:
>>>>>>> Hi Kefu,
>>>>>>>
>>>>>>> Sort of wondering what in the current code below is the relation between
>>>>>>> gpertools and tcmalloc?
>>>>>>
>>>>>> could you be more specific?
>>>>>
>>>>> While looking at the code of CMakeLists.txt:286, in the case that no
>>>>> ALLOCATOR is defined.
>>>>> There the current flow of code seems to suggest that gperf-tools is
>>>>> always linked to tcmalloc. And once gperftools is found, the ALLOCATOR
>>>>> is set to tcmalloc. This then breaks rocksdb building.
>>>>>     `set(HAVE_LIBTCMALLOC ${GPERFTOOLS_FOUND})`
>>>>> like:
>>>>> ```
>>>>>  CMake Error at cmake/modules/BuildRocksDB.cmake:63 (message):
>>>>>    Incompatible tcmalloc v2.6.1 and rocksdb v5.8.0, please install
>>>>>  gperf-tools
>>>>>    2.5 (not 2.5.93) or >= 2.6.2, or switch to another allocator using
>>>>> 'cmake -DALLOCATOR=libc'.
>>>>>  Call Stack (most recent call first):
>>>>>    src/CMakeLists.txt:825 (build_rocksdb)
>>>>> ```
>>>>>
>>>>>>> Reason for the question is that once the gperf packages is installed,
>>>>>>> as a bonus the code below declares the allocator to be tcmalloc.
>>>>>>> But gperf under FreeBSD does not have tcmalloc.
>>>>>>
>>>>>> please note gperf here is not GNU perf [1] but google-perftools. and
>>>>>> why do you claim the gperf under FreeBSD does not have tcmalloc? could
>>>>>> you elaborate this a little bit?
>>>>>
>>>>> My bad, I mixed gperf and gperf-tools. We are talking about gperf-tools.
>>>>> And yes, gperftools brings tcmalloc but not in the version mix that is
>>>>> asked for. gperf-tools was at 2.5 one time, but has upgraded to 2.6.1_1
>>>>> on FreeBSD[1]. Getting other versions will require manual work to get it
>>>>> to work.
>>>>>
>>>>> But the question is:
>>>>>     Do I really want my allocator to be tcmalloc when the native
>>>>>     libc allocator is jemalloc?
>>>>
>>>> i think it's up to you.
>>>
>>> Oke, it is what the code now forces...
>>> And I've tried avoiding tcmalloc uptil now, and would like to do so for
>>> the time being..
>>>
>>>>> So is the test in CMakeLists.txt:286-324 what we want on FreeBSD?
>>>>
>>>> see http://tracker.ceph.com/issues/21422. so again, it's up to you.
>>>
>>> The tracker suggests that it is an error in 2.5.93, but
>>> cmake/modules/BuildRocksDB.cmake suggests that it is only fixed in
>>> 2.6.2...??
>>
>> 2.6.2 is not yet released. so if you want to stick with tcmalloc, you
>> need to build from the latest (or recent enough) source. actually, i
>> am not sure what you question is..
>
> The question: Do you know if 2.6.2 is the first gperf-tools-version >
> 2.5.93 where this bug from the tracker is solved?

https://github.com/ceph/ceph/pull/17788/commits/ff8d6a730bb774271b9eea6af338e2d3ce3c48a5

>
>>>
>>>>> The workaround this (for the time being) is to call cmake with
>>>>> -DALLOCATOR=libc as is suggested.
>>>
>>> I'll use this as workaround for the time being, since that is least
>>> invasive. And when that works out, try and make a PR for the allocator
>>> selector in CmakeLists.txt, and/or wait until gperf-tools 2.6.2 is
>>> available on FreeBSD.
>>
>> what is issue you want to address?
>
> The issue I want to address is that I'm not able to build the intree
> rocksdb ATM. Because tcmalloc is not => 2.6.2, and tcmalloc is required
> because gperf-tools is detected.

the error message offered two solutions. i think, what you need is
just to choose one of them.

>
> Hardcoding the allocator during cmake-ing for now solves this catch-22.
>
> --WjW
>
>>
>>>
>>> --WjW
>>>
>>>>> [1]
>>>>> https://svnweb.freebsd.org/ports/head/devel/google-perftools/Makefile?revision=450353&view=markup
>>>>>
>>>>>>
>>>>>> --
>>>>>> [1] http://portsmon.freebsd.org/portoverview.py?category=devel&portname=gperf
>>>>>> [2] http://portsmon.freebsd.org/portoverview.py?category=devel&portname=google-perftools
>>>>>>
>>>>>>>
>>>>>>> So I guess I need to fix the code below the to prevent this implicit
>>>>>>> relation
>>>>>>> under FreeBSD.
>>>>>>>
>>>>>>> And the problem that arises from this all lies in compiling rocksdb,
>>>>>>> because
>>>>>>> under FreeBSD that also needs to be build with libc as allocator.
>>>>>>> The introduction of the new rocksdb building makes this clear.
>>>>>>>
>>>>>>> CMake Error at cmake/modules/BuildRocksDB.cmake:63 (message):
>>>>>>>   Incompatible tcmalloc v2.6.1 and rocksdb v5.8.0, please install
>>>>>>> gperf-tools
>>>>>>>   2.5 (not 2.5.93) or >= 2.6.2, or switch to another allocator using 'cmake
>>>>>>>   -DALLOCATOR=libc'.
>>>>>>> Call Stack (most recent call first):
>>>>>>>   src/CMakeLists.txt:825 (build_rocksdb)
>>>>>>>
>>>>>>>
>>>>>>> --WjW
>>>>>>>
>>>>>>>
>>>>>>> a614c3b9c1e (Ali Maredia         2015-12-30 21:06:08 -0500 286)
>>>>>>> if(ALLOCATOR)
>>>>>>> ca9946d2b5a (Bassam Tabbara      2016-10-10 14:37:58 -0700 287)
>>>>>>> if(${ALLOCATOR} MATCHES "tcmalloc(_minimal)?")
>>>>>>> 6c12edbf9d2 (Kefu Chai           2016-08-11 00:12:05 +0800 288)
>>>>>>> find_package(gperftools REQUIRED)
>>>>>>> 6c12edbf9d2 (Kefu Chai           2016-08-11 00:12:05 +0800 289)
>>>>>>> set(HAVE_LIBTCMALLOC ON)
>>>>>>> a614c3b9c1e (Ali Maredia         2015-12-30 21:06:08 -0500 290)
>>>>>>> elseif(${ALLOCATOR} STREQUAL "jemalloc")
>>>>>>> a614c3b9c1e (Ali Maredia         2015-12-30 21:06:08 -0500 291)
>>>>>>> find_package(JeMalloc REQUIRED)
>>>>>>> 0ef09f32d46 (Ali Maredia         2016-05-02 18:21:32 -0400 292)
>>>>>>> set(HAVE_LIBJEMALLOC ${JEMALLOC_FOUND})
>>>>>>> 83b74f52501 (Matt Benjamin       2016-12-21 23:36:37 -0500 293)
>>>>>>> set(HAVE_JEMALLOC 1)
>>>>>>> a614c3b9c1e (Ali Maredia         2015-12-30 21:06:08 -0500 294)   endif()
>>>>>>> a614c3b9c1e (Ali Maredia         2015-12-30 21:06:08 -0500 295)
>>>>>>> else(ALLOCATOR)
>>>>>>> 6c12edbf9d2 (Kefu Chai           2016-08-11 00:12:05 +0800 296)
>>>>>>> find_package(gperftools)
>>>>>>> 6c12edbf9d2 (Kefu Chai           2016-08-11 00:12:05 +0800 297)
>>>>>>> set(HAVE_LIBTCMALLOC ${GPERFTOOLS_FOUND})
>>>>>>> 6c12edbf9d2 (Kefu Chai           2016-08-11 00:12:05 +0800 298)   if(NOT
>>>>>>> GPERFTOOLS_FOUND)
>>>>>>> 0ef09f32d46 (Ali Maredia         2016-05-02 18:21:32 -0400 299)
>>>>>>> find_package(JeMalloc)
>>>>>>> 0ef09f32d46 (Ali Maredia         2016-05-02 18:21:32 -0400 300)
>>>>>>> set(HAVE_LIBJEMALLOC ${JEMALLOC_FOUND})
>>>>>>> 6c12edbf9d2 (Kefu Chai           2016-08-11 00:12:05 +0800 301)
>>>>>>> endif(NOT GPERFTOOLS_FOUND)
>>>>>>> 6c12edbf9d2 (Kefu Chai           2016-08-11 00:12:05 +0800 302)
>>>>>>> if(GPERFTOOLS_FOUND)
>>>>>>> 48a1f8eff5b (Kefu Chai           2016-06-08 11:34:44 +0800 303)
>>>>>>> set(ALLOCATOR tcmalloc)
>>>>>>> 48a1f8eff5b (Kefu Chai           2016-06-08 11:34:44 +0800 304)
>>>>>>> elseif(JEMALLOC_FOUND)
>>>>>>> 48a1f8eff5b (Kefu Chai           2016-06-08 11:34:44 +0800 305)
>>>>>>> set(ALLOCATOR jemalloc)
>>>>>>> 48a1f8eff5b (Kefu Chai           2016-06-08 11:34:44 +0800 306)   else()
>>>>>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 307)
>>>>>>> if(NOT FREEBSD)
>>>>>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 308)       #
>>>>>>> FreeBSD already has jemalloc as its default allocator
>>>>>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 309)
>>>>>>> message(WARNING "tcmalloc and jemalloc not found, falling back to libc")
>>>>>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 310)     endif()
>>>>>>> a614c3b9c1e (Ali Maredia         2015-12-30 21:06:08 -0500 311)
>>>>>>> set(ALLOCATOR "libc")
>>>>>>> 6c12edbf9d2 (Kefu Chai           2016-08-11 00:12:05 +0800 312)
>>>>>>> endif(GPERFTOOLS_FOUND)
>>>>>>> a614c3b9c1e (Ali Maredia         2015-12-30 21:06:08 -0500 313)
>>>>>>> endif(ALLOCATOR)
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>
>>
>>
>>
>



-- 
Regards
Kefu Chai