* Re: Understanding some of the Cmake logics
[not found] <b06780c7-13ce-1274-735f-806a1f4f8062@digiware.nl>
@ 2017-10-10 11:14 ` kefu chai
2017-10-10 12:06 ` Willem Jan Withagen
0 siblings, 1 reply; 10+ messages in thread
From: kefu chai @ 2017-10-10 11:14 UTC (permalink / raw)
To: Willem Jan Withagen, ceph-devel
+ ceph-devel
On Tue, Oct 10, 2017 at 4:49 AM, Willem Jan Withagen <wjw@digiware.nl> wrote:
> Hi Kefu,
>
> Sort of wondering what in the current code below is the relation between
> gpertools and tcmalloc?
could you be more specific?
>
> Reason for the question is that once the gperf packages is installed,
> as a bonus the code below declares the allocator to be tcmalloc.
> But gperf under FreeBSD does not have tcmalloc.
please note gperf here is not GNU perf [1] but google-perftools. and
why do you claim the gperf under FreeBSD does not have tcmalloc? could
you elaborate this a little bit?
--
[1] http://portsmon.freebsd.org/portoverview.py?category=devel&portname=gperf
[2] http://portsmon.freebsd.org/portoverview.py?category=devel&portname=google-perftools
>
> So I guess I need to fix the code below the to prevent this implicit
> relation
> under FreeBSD.
>
> And the problem that arises from this all lies in compiling rocksdb,
> because
> under FreeBSD that also needs to be build with libc as allocator.
> The introduction of the new rocksdb building makes this clear.
>
> CMake Error at cmake/modules/BuildRocksDB.cmake:63 (message):
> Incompatible tcmalloc v2.6.1 and rocksdb v5.8.0, please install
> gperf-tools
> 2.5 (not 2.5.93) or >= 2.6.2, or switch to another allocator using 'cmake
> -DALLOCATOR=libc'.
> Call Stack (most recent call first):
> src/CMakeLists.txt:825 (build_rocksdb)
>
>
> --WjW
>
>
> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 286)
> if(ALLOCATOR)
> ca9946d2b5a (Bassam Tabbara 2016-10-10 14:37:58 -0700 287)
> if(${ALLOCATOR} MATCHES "tcmalloc(_minimal)?")
> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 288)
> find_package(gperftools REQUIRED)
> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 289)
> set(HAVE_LIBTCMALLOC ON)
> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 290)
> elseif(${ALLOCATOR} STREQUAL "jemalloc")
> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 291)
> find_package(JeMalloc REQUIRED)
> 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 292)
> set(HAVE_LIBJEMALLOC ${JEMALLOC_FOUND})
> 83b74f52501 (Matt Benjamin 2016-12-21 23:36:37 -0500 293)
> set(HAVE_JEMALLOC 1)
> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 294) endif()
> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 295)
> else(ALLOCATOR)
> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 296)
> find_package(gperftools)
> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 297)
> set(HAVE_LIBTCMALLOC ${GPERFTOOLS_FOUND})
> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 298) if(NOT
> GPERFTOOLS_FOUND)
> 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 299)
> find_package(JeMalloc)
> 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 300)
> set(HAVE_LIBJEMALLOC ${JEMALLOC_FOUND})
> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 301)
> endif(NOT GPERFTOOLS_FOUND)
> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 302)
> if(GPERFTOOLS_FOUND)
> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 303)
> set(ALLOCATOR tcmalloc)
> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 304)
> elseif(JEMALLOC_FOUND)
> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 305)
> set(ALLOCATOR jemalloc)
> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 306) else()
> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 307)
> if(NOT FREEBSD)
> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 308) #
> FreeBSD already has jemalloc as its default allocator
> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 309)
> message(WARNING "tcmalloc and jemalloc not found, falling back to libc")
> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 310) endif()
> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 311)
> set(ALLOCATOR "libc")
> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 312)
> endif(GPERFTOOLS_FOUND)
> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 313)
> endif(ALLOCATOR)
>
--
Regards
Kefu Chai
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Understanding some of the Cmake logics
2017-10-10 11:14 ` Understanding some of the Cmake logics kefu chai
@ 2017-10-10 12:06 ` Willem Jan Withagen
2017-10-10 13:31 ` kefu chai
0 siblings, 1 reply; 10+ messages in thread
From: Willem Jan Withagen @ 2017-10-10 12:06 UTC (permalink / raw)
To: kefu chai, ceph-devel
On 10-10-2017 13:14, kefu chai wrote:
> + ceph-devel
>
> On Tue, Oct 10, 2017 at 4:49 AM, Willem Jan Withagen <wjw@digiware.nl> wrote:
>> Hi Kefu,
>>
>> Sort of wondering what in the current code below is the relation between
>> gpertools and tcmalloc?
>
> could you be more specific?
While looking at the code of CMakeLists.txt:286, in the case that no
ALLOCATOR is defined.
There the current flow of code seems to suggest that gperf-tools is
always linked to tcmalloc. And once gperftools is found, the ALLOCATOR
is set to tcmalloc. This then breaks rocksdb building.
`set(HAVE_LIBTCMALLOC ${GPERFTOOLS_FOUND})`
like:
```
CMake Error at cmake/modules/BuildRocksDB.cmake:63 (message):
Incompatible tcmalloc v2.6.1 and rocksdb v5.8.0, please install
gperf-tools
2.5 (not 2.5.93) or >= 2.6.2, or switch to another allocator using
'cmake -DALLOCATOR=libc'.
Call Stack (most recent call first):
src/CMakeLists.txt:825 (build_rocksdb)
```
>> Reason for the question is that once the gperf packages is installed,
>> as a bonus the code below declares the allocator to be tcmalloc.
>> But gperf under FreeBSD does not have tcmalloc.
>
> please note gperf here is not GNU perf [1] but google-perftools. and
> why do you claim the gperf under FreeBSD does not have tcmalloc? could
> you elaborate this a little bit?
My bad, I mixed gperf and gperf-tools. We are talking about gperf-tools.
And yes, gperftools brings tcmalloc but not in the version mix that is
asked for. gperf-tools was at 2.5 one time, but has upgraded to 2.6.1_1
on FreeBSD[1]. Getting other versions will require manual work to get it
to work.
But the question is:
Do I really want my allocator to be tcmalloc when the native
libc allocator is jemalloc?
So is the test in CMakeLists.txt:286-324 what we want on FreeBSD?
The workaround this (for the time being) is to call cmake with
-DALLOCATOR=libc as is suggested.
--WjW
[1]
https://svnweb.freebsd.org/ports/head/devel/google-perftools/Makefile?revision=450353&view=markup
>
> --
> [1] http://portsmon.freebsd.org/portoverview.py?category=devel&portname=gperf
> [2] http://portsmon.freebsd.org/portoverview.py?category=devel&portname=google-perftools
>
>>
>> So I guess I need to fix the code below the to prevent this implicit
>> relation
>> under FreeBSD.
>>
>> And the problem that arises from this all lies in compiling rocksdb,
>> because
>> under FreeBSD that also needs to be build with libc as allocator.
>> The introduction of the new rocksdb building makes this clear.
>>
>> CMake Error at cmake/modules/BuildRocksDB.cmake:63 (message):
>> Incompatible tcmalloc v2.6.1 and rocksdb v5.8.0, please install
>> gperf-tools
>> 2.5 (not 2.5.93) or >= 2.6.2, or switch to another allocator using 'cmake
>> -DALLOCATOR=libc'.
>> Call Stack (most recent call first):
>> src/CMakeLists.txt:825 (build_rocksdb)
>>
>>
>> --WjW
>>
>>
>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 286)
>> if(ALLOCATOR)
>> ca9946d2b5a (Bassam Tabbara 2016-10-10 14:37:58 -0700 287)
>> if(${ALLOCATOR} MATCHES "tcmalloc(_minimal)?")
>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 288)
>> find_package(gperftools REQUIRED)
>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 289)
>> set(HAVE_LIBTCMALLOC ON)
>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 290)
>> elseif(${ALLOCATOR} STREQUAL "jemalloc")
>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 291)
>> find_package(JeMalloc REQUIRED)
>> 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 292)
>> set(HAVE_LIBJEMALLOC ${JEMALLOC_FOUND})
>> 83b74f52501 (Matt Benjamin 2016-12-21 23:36:37 -0500 293)
>> set(HAVE_JEMALLOC 1)
>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 294) endif()
>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 295)
>> else(ALLOCATOR)
>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 296)
>> find_package(gperftools)
>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 297)
>> set(HAVE_LIBTCMALLOC ${GPERFTOOLS_FOUND})
>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 298) if(NOT
>> GPERFTOOLS_FOUND)
>> 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 299)
>> find_package(JeMalloc)
>> 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 300)
>> set(HAVE_LIBJEMALLOC ${JEMALLOC_FOUND})
>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 301)
>> endif(NOT GPERFTOOLS_FOUND)
>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 302)
>> if(GPERFTOOLS_FOUND)
>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 303)
>> set(ALLOCATOR tcmalloc)
>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 304)
>> elseif(JEMALLOC_FOUND)
>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 305)
>> set(ALLOCATOR jemalloc)
>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 306) else()
>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 307)
>> if(NOT FREEBSD)
>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 308) #
>> FreeBSD already has jemalloc as its default allocator
>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 309)
>> message(WARNING "tcmalloc and jemalloc not found, falling back to libc")
>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 310) endif()
>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 311)
>> set(ALLOCATOR "libc")
>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 312)
>> endif(GPERFTOOLS_FOUND)
>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 313)
>> endif(ALLOCATOR)
>>
>
>
>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Understanding some of the Cmake logics
2017-10-10 12:06 ` Willem Jan Withagen
@ 2017-10-10 13:31 ` kefu chai
2017-10-10 13:54 ` Willem Jan Withagen
0 siblings, 1 reply; 10+ messages in thread
From: kefu chai @ 2017-10-10 13:31 UTC (permalink / raw)
To: Willem Jan Withagen; +Cc: ceph-devel
On Tue, Oct 10, 2017 at 8:06 PM, Willem Jan Withagen <wjw@digiware.nl> wrote:
> On 10-10-2017 13:14, kefu chai wrote:
>> + ceph-devel
>>
>> On Tue, Oct 10, 2017 at 4:49 AM, Willem Jan Withagen <wjw@digiware.nl> wrote:
>>> Hi Kefu,
>>>
>>> Sort of wondering what in the current code below is the relation between
>>> gpertools and tcmalloc?
>>
>> could you be more specific?
>
> While looking at the code of CMakeLists.txt:286, in the case that no
> ALLOCATOR is defined.
> There the current flow of code seems to suggest that gperf-tools is
> always linked to tcmalloc. And once gperftools is found, the ALLOCATOR
> is set to tcmalloc. This then breaks rocksdb building.
> `set(HAVE_LIBTCMALLOC ${GPERFTOOLS_FOUND})`
> like:
> ```
> CMake Error at cmake/modules/BuildRocksDB.cmake:63 (message):
> Incompatible tcmalloc v2.6.1 and rocksdb v5.8.0, please install
> gperf-tools
> 2.5 (not 2.5.93) or >= 2.6.2, or switch to another allocator using
> 'cmake -DALLOCATOR=libc'.
> Call Stack (most recent call first):
> src/CMakeLists.txt:825 (build_rocksdb)
> ```
>
>>> Reason for the question is that once the gperf packages is installed,
>>> as a bonus the code below declares the allocator to be tcmalloc.
>>> But gperf under FreeBSD does not have tcmalloc.
>>
>> please note gperf here is not GNU perf [1] but google-perftools. and
>> why do you claim the gperf under FreeBSD does not have tcmalloc? could
>> you elaborate this a little bit?
>
> My bad, I mixed gperf and gperf-tools. We are talking about gperf-tools.
> And yes, gperftools brings tcmalloc but not in the version mix that is
> asked for. gperf-tools was at 2.5 one time, but has upgraded to 2.6.1_1
> on FreeBSD[1]. Getting other versions will require manual work to get it
> to work.
>
> But the question is:
> Do I really want my allocator to be tcmalloc when the native
> libc allocator is jemalloc?
i think it's up to you.
> So is the test in CMakeLists.txt:286-324 what we want on FreeBSD?
see http://tracker.ceph.com/issues/21422. so again, it's up to you.
>
> The workaround this (for the time being) is to call cmake with
> -DALLOCATOR=libc as is suggested.
>
> --WjW
>
> [1]
> https://svnweb.freebsd.org/ports/head/devel/google-perftools/Makefile?revision=450353&view=markup
>
>>
>> --
>> [1] http://portsmon.freebsd.org/portoverview.py?category=devel&portname=gperf
>> [2] http://portsmon.freebsd.org/portoverview.py?category=devel&portname=google-perftools
>>
>>>
>>> So I guess I need to fix the code below the to prevent this implicit
>>> relation
>>> under FreeBSD.
>>>
>>> And the problem that arises from this all lies in compiling rocksdb,
>>> because
>>> under FreeBSD that also needs to be build with libc as allocator.
>>> The introduction of the new rocksdb building makes this clear.
>>>
>>> CMake Error at cmake/modules/BuildRocksDB.cmake:63 (message):
>>> Incompatible tcmalloc v2.6.1 and rocksdb v5.8.0, please install
>>> gperf-tools
>>> 2.5 (not 2.5.93) or >= 2.6.2, or switch to another allocator using 'cmake
>>> -DALLOCATOR=libc'.
>>> Call Stack (most recent call first):
>>> src/CMakeLists.txt:825 (build_rocksdb)
>>>
>>>
>>> --WjW
>>>
>>>
>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 286)
>>> if(ALLOCATOR)
>>> ca9946d2b5a (Bassam Tabbara 2016-10-10 14:37:58 -0700 287)
>>> if(${ALLOCATOR} MATCHES "tcmalloc(_minimal)?")
>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 288)
>>> find_package(gperftools REQUIRED)
>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 289)
>>> set(HAVE_LIBTCMALLOC ON)
>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 290)
>>> elseif(${ALLOCATOR} STREQUAL "jemalloc")
>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 291)
>>> find_package(JeMalloc REQUIRED)
>>> 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 292)
>>> set(HAVE_LIBJEMALLOC ${JEMALLOC_FOUND})
>>> 83b74f52501 (Matt Benjamin 2016-12-21 23:36:37 -0500 293)
>>> set(HAVE_JEMALLOC 1)
>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 294) endif()
>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 295)
>>> else(ALLOCATOR)
>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 296)
>>> find_package(gperftools)
>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 297)
>>> set(HAVE_LIBTCMALLOC ${GPERFTOOLS_FOUND})
>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 298) if(NOT
>>> GPERFTOOLS_FOUND)
>>> 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 299)
>>> find_package(JeMalloc)
>>> 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 300)
>>> set(HAVE_LIBJEMALLOC ${JEMALLOC_FOUND})
>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 301)
>>> endif(NOT GPERFTOOLS_FOUND)
>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 302)
>>> if(GPERFTOOLS_FOUND)
>>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 303)
>>> set(ALLOCATOR tcmalloc)
>>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 304)
>>> elseif(JEMALLOC_FOUND)
>>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 305)
>>> set(ALLOCATOR jemalloc)
>>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 306) else()
>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 307)
>>> if(NOT FREEBSD)
>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 308) #
>>> FreeBSD already has jemalloc as its default allocator
>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 309)
>>> message(WARNING "tcmalloc and jemalloc not found, falling back to libc")
>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 310) endif()
>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 311)
>>> set(ALLOCATOR "libc")
>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 312)
>>> endif(GPERFTOOLS_FOUND)
>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 313)
>>> endif(ALLOCATOR)
>>>
>>
>>
>>
>
--
Regards
Kefu Chai
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Understanding some of the Cmake logics
2017-10-10 13:31 ` kefu chai
@ 2017-10-10 13:54 ` Willem Jan Withagen
2017-10-10 14:58 ` kefu chai
0 siblings, 1 reply; 10+ messages in thread
From: Willem Jan Withagen @ 2017-10-10 13:54 UTC (permalink / raw)
To: kefu chai; +Cc: ceph-devel
On 10-10-2017 15:31, kefu chai wrote:
> On Tue, Oct 10, 2017 at 8:06 PM, Willem Jan Withagen <wjw@digiware.nl> wrote:
>> On 10-10-2017 13:14, kefu chai wrote:
>>> + ceph-devel
>>>
>>> On Tue, Oct 10, 2017 at 4:49 AM, Willem Jan Withagen <wjw@digiware.nl> wrote:
>>>> Hi Kefu,
>>>>
>>>> Sort of wondering what in the current code below is the relation between
>>>> gpertools and tcmalloc?
>>>
>>> could you be more specific?
>>
>> While looking at the code of CMakeLists.txt:286, in the case that no
>> ALLOCATOR is defined.
>> There the current flow of code seems to suggest that gperf-tools is
>> always linked to tcmalloc. And once gperftools is found, the ALLOCATOR
>> is set to tcmalloc. This then breaks rocksdb building.
>> `set(HAVE_LIBTCMALLOC ${GPERFTOOLS_FOUND})`
>> like:
>> ```
>> CMake Error at cmake/modules/BuildRocksDB.cmake:63 (message):
>> Incompatible tcmalloc v2.6.1 and rocksdb v5.8.0, please install
>> gperf-tools
>> 2.5 (not 2.5.93) or >= 2.6.2, or switch to another allocator using
>> 'cmake -DALLOCATOR=libc'.
>> Call Stack (most recent call first):
>> src/CMakeLists.txt:825 (build_rocksdb)
>> ```
>>
>>>> Reason for the question is that once the gperf packages is installed,
>>>> as a bonus the code below declares the allocator to be tcmalloc.
>>>> But gperf under FreeBSD does not have tcmalloc.
>>>
>>> please note gperf here is not GNU perf [1] but google-perftools. and
>>> why do you claim the gperf under FreeBSD does not have tcmalloc? could
>>> you elaborate this a little bit?
>>
>> My bad, I mixed gperf and gperf-tools. We are talking about gperf-tools.
>> And yes, gperftools brings tcmalloc but not in the version mix that is
>> asked for. gperf-tools was at 2.5 one time, but has upgraded to 2.6.1_1
>> on FreeBSD[1]. Getting other versions will require manual work to get it
>> to work.
>>
>> But the question is:
>> Do I really want my allocator to be tcmalloc when the native
>> libc allocator is jemalloc?
>
> i think it's up to you.
Oke, it is what the code now forces...
And I've tried avoiding tcmalloc uptil now, and would like to do so for
the time being..
>> So is the test in CMakeLists.txt:286-324 what we want on FreeBSD?
>
> see http://tracker.ceph.com/issues/21422. so again, it's up to you.
The tracker suggests that it is an error in 2.5.93, but
cmake/modules/BuildRocksDB.cmake suggests that it is only fixed in
2.6.2...??
>> The workaround this (for the time being) is to call cmake with
>> -DALLOCATOR=libc as is suggested.
I'll use this as workaround for the time being, since that is least
invasive. And when that works out, try and make a PR for the allocator
selector in CmakeLists.txt, and/or wait until gperf-tools 2.6.2 is
available on FreeBSD.
--WjW
>> [1]
>> https://svnweb.freebsd.org/ports/head/devel/google-perftools/Makefile?revision=450353&view=markup
>>
>>>
>>> --
>>> [1] http://portsmon.freebsd.org/portoverview.py?category=devel&portname=gperf
>>> [2] http://portsmon.freebsd.org/portoverview.py?category=devel&portname=google-perftools
>>>
>>>>
>>>> So I guess I need to fix the code below the to prevent this implicit
>>>> relation
>>>> under FreeBSD.
>>>>
>>>> And the problem that arises from this all lies in compiling rocksdb,
>>>> because
>>>> under FreeBSD that also needs to be build with libc as allocator.
>>>> The introduction of the new rocksdb building makes this clear.
>>>>
>>>> CMake Error at cmake/modules/BuildRocksDB.cmake:63 (message):
>>>> Incompatible tcmalloc v2.6.1 and rocksdb v5.8.0, please install
>>>> gperf-tools
>>>> 2.5 (not 2.5.93) or >= 2.6.2, or switch to another allocator using 'cmake
>>>> -DALLOCATOR=libc'.
>>>> Call Stack (most recent call first):
>>>> src/CMakeLists.txt:825 (build_rocksdb)
>>>>
>>>>
>>>> --WjW
>>>>
>>>>
>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 286)
>>>> if(ALLOCATOR)
>>>> ca9946d2b5a (Bassam Tabbara 2016-10-10 14:37:58 -0700 287)
>>>> if(${ALLOCATOR} MATCHES "tcmalloc(_minimal)?")
>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 288)
>>>> find_package(gperftools REQUIRED)
>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 289)
>>>> set(HAVE_LIBTCMALLOC ON)
>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 290)
>>>> elseif(${ALLOCATOR} STREQUAL "jemalloc")
>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 291)
>>>> find_package(JeMalloc REQUIRED)
>>>> 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 292)
>>>> set(HAVE_LIBJEMALLOC ${JEMALLOC_FOUND})
>>>> 83b74f52501 (Matt Benjamin 2016-12-21 23:36:37 -0500 293)
>>>> set(HAVE_JEMALLOC 1)
>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 294) endif()
>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 295)
>>>> else(ALLOCATOR)
>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 296)
>>>> find_package(gperftools)
>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 297)
>>>> set(HAVE_LIBTCMALLOC ${GPERFTOOLS_FOUND})
>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 298) if(NOT
>>>> GPERFTOOLS_FOUND)
>>>> 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 299)
>>>> find_package(JeMalloc)
>>>> 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 300)
>>>> set(HAVE_LIBJEMALLOC ${JEMALLOC_FOUND})
>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 301)
>>>> endif(NOT GPERFTOOLS_FOUND)
>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 302)
>>>> if(GPERFTOOLS_FOUND)
>>>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 303)
>>>> set(ALLOCATOR tcmalloc)
>>>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 304)
>>>> elseif(JEMALLOC_FOUND)
>>>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 305)
>>>> set(ALLOCATOR jemalloc)
>>>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 306) else()
>>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 307)
>>>> if(NOT FREEBSD)
>>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 308) #
>>>> FreeBSD already has jemalloc as its default allocator
>>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 309)
>>>> message(WARNING "tcmalloc and jemalloc not found, falling back to libc")
>>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 310) endif()
>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 311)
>>>> set(ALLOCATOR "libc")
>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 312)
>>>> endif(GPERFTOOLS_FOUND)
>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 313)
>>>> endif(ALLOCATOR)
>>>>
>>>
>>>
>>>
>>
>
>
>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Understanding some of the Cmake logics
2017-10-10 13:54 ` Willem Jan Withagen
@ 2017-10-10 14:58 ` kefu chai
2017-10-10 15:13 ` Willem Jan Withagen
0 siblings, 1 reply; 10+ messages in thread
From: kefu chai @ 2017-10-10 14:58 UTC (permalink / raw)
To: Willem Jan Withagen; +Cc: ceph-devel
On Tue, Oct 10, 2017 at 9:54 PM, Willem Jan Withagen <wjw@digiware.nl> wrote:
> On 10-10-2017 15:31, kefu chai wrote:
>> On Tue, Oct 10, 2017 at 8:06 PM, Willem Jan Withagen <wjw@digiware.nl> wrote:
>>> On 10-10-2017 13:14, kefu chai wrote:
>>>> + ceph-devel
>>>>
>>>> On Tue, Oct 10, 2017 at 4:49 AM, Willem Jan Withagen <wjw@digiware.nl> wrote:
>>>>> Hi Kefu,
>>>>>
>>>>> Sort of wondering what in the current code below is the relation between
>>>>> gpertools and tcmalloc?
>>>>
>>>> could you be more specific?
>>>
>>> While looking at the code of CMakeLists.txt:286, in the case that no
>>> ALLOCATOR is defined.
>>> There the current flow of code seems to suggest that gperf-tools is
>>> always linked to tcmalloc. And once gperftools is found, the ALLOCATOR
>>> is set to tcmalloc. This then breaks rocksdb building.
>>> `set(HAVE_LIBTCMALLOC ${GPERFTOOLS_FOUND})`
>>> like:
>>> ```
>>> CMake Error at cmake/modules/BuildRocksDB.cmake:63 (message):
>>> Incompatible tcmalloc v2.6.1 and rocksdb v5.8.0, please install
>>> gperf-tools
>>> 2.5 (not 2.5.93) or >= 2.6.2, or switch to another allocator using
>>> 'cmake -DALLOCATOR=libc'.
>>> Call Stack (most recent call first):
>>> src/CMakeLists.txt:825 (build_rocksdb)
>>> ```
>>>
>>>>> Reason for the question is that once the gperf packages is installed,
>>>>> as a bonus the code below declares the allocator to be tcmalloc.
>>>>> But gperf under FreeBSD does not have tcmalloc.
>>>>
>>>> please note gperf here is not GNU perf [1] but google-perftools. and
>>>> why do you claim the gperf under FreeBSD does not have tcmalloc? could
>>>> you elaborate this a little bit?
>>>
>>> My bad, I mixed gperf and gperf-tools. We are talking about gperf-tools.
>>> And yes, gperftools brings tcmalloc but not in the version mix that is
>>> asked for. gperf-tools was at 2.5 one time, but has upgraded to 2.6.1_1
>>> on FreeBSD[1]. Getting other versions will require manual work to get it
>>> to work.
>>>
>>> But the question is:
>>> Do I really want my allocator to be tcmalloc when the native
>>> libc allocator is jemalloc?
>>
>> i think it's up to you.
>
> Oke, it is what the code now forces...
> And I've tried avoiding tcmalloc uptil now, and would like to do so for
> the time being..
>
>>> So is the test in CMakeLists.txt:286-324 what we want on FreeBSD?
>>
>> see http://tracker.ceph.com/issues/21422. so again, it's up to you.
>
> The tracker suggests that it is an error in 2.5.93, but
> cmake/modules/BuildRocksDB.cmake suggests that it is only fixed in
> 2.6.2...??
2.6.2 is not yet released. so if you want to stick with tcmalloc, you
need to build from the latest (or recent enough) source. actually, i
am not sure what you question is..
>
>>> The workaround this (for the time being) is to call cmake with
>>> -DALLOCATOR=libc as is suggested.
>
> I'll use this as workaround for the time being, since that is least
> invasive. And when that works out, try and make a PR for the allocator
> selector in CmakeLists.txt, and/or wait until gperf-tools 2.6.2 is
> available on FreeBSD.
what is issue you want to address?
>
> --WjW
>
>>> [1]
>>> https://svnweb.freebsd.org/ports/head/devel/google-perftools/Makefile?revision=450353&view=markup
>>>
>>>>
>>>> --
>>>> [1] http://portsmon.freebsd.org/portoverview.py?category=devel&portname=gperf
>>>> [2] http://portsmon.freebsd.org/portoverview.py?category=devel&portname=google-perftools
>>>>
>>>>>
>>>>> So I guess I need to fix the code below the to prevent this implicit
>>>>> relation
>>>>> under FreeBSD.
>>>>>
>>>>> And the problem that arises from this all lies in compiling rocksdb,
>>>>> because
>>>>> under FreeBSD that also needs to be build with libc as allocator.
>>>>> The introduction of the new rocksdb building makes this clear.
>>>>>
>>>>> CMake Error at cmake/modules/BuildRocksDB.cmake:63 (message):
>>>>> Incompatible tcmalloc v2.6.1 and rocksdb v5.8.0, please install
>>>>> gperf-tools
>>>>> 2.5 (not 2.5.93) or >= 2.6.2, or switch to another allocator using 'cmake
>>>>> -DALLOCATOR=libc'.
>>>>> Call Stack (most recent call first):
>>>>> src/CMakeLists.txt:825 (build_rocksdb)
>>>>>
>>>>>
>>>>> --WjW
>>>>>
>>>>>
>>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 286)
>>>>> if(ALLOCATOR)
>>>>> ca9946d2b5a (Bassam Tabbara 2016-10-10 14:37:58 -0700 287)
>>>>> if(${ALLOCATOR} MATCHES "tcmalloc(_minimal)?")
>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 288)
>>>>> find_package(gperftools REQUIRED)
>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 289)
>>>>> set(HAVE_LIBTCMALLOC ON)
>>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 290)
>>>>> elseif(${ALLOCATOR} STREQUAL "jemalloc")
>>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 291)
>>>>> find_package(JeMalloc REQUIRED)
>>>>> 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 292)
>>>>> set(HAVE_LIBJEMALLOC ${JEMALLOC_FOUND})
>>>>> 83b74f52501 (Matt Benjamin 2016-12-21 23:36:37 -0500 293)
>>>>> set(HAVE_JEMALLOC 1)
>>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 294) endif()
>>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 295)
>>>>> else(ALLOCATOR)
>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 296)
>>>>> find_package(gperftools)
>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 297)
>>>>> set(HAVE_LIBTCMALLOC ${GPERFTOOLS_FOUND})
>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 298) if(NOT
>>>>> GPERFTOOLS_FOUND)
>>>>> 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 299)
>>>>> find_package(JeMalloc)
>>>>> 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 300)
>>>>> set(HAVE_LIBJEMALLOC ${JEMALLOC_FOUND})
>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 301)
>>>>> endif(NOT GPERFTOOLS_FOUND)
>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 302)
>>>>> if(GPERFTOOLS_FOUND)
>>>>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 303)
>>>>> set(ALLOCATOR tcmalloc)
>>>>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 304)
>>>>> elseif(JEMALLOC_FOUND)
>>>>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 305)
>>>>> set(ALLOCATOR jemalloc)
>>>>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 306) else()
>>>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 307)
>>>>> if(NOT FREEBSD)
>>>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 308) #
>>>>> FreeBSD already has jemalloc as its default allocator
>>>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 309)
>>>>> message(WARNING "tcmalloc and jemalloc not found, falling back to libc")
>>>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 310) endif()
>>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 311)
>>>>> set(ALLOCATOR "libc")
>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 312)
>>>>> endif(GPERFTOOLS_FOUND)
>>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 313)
>>>>> endif(ALLOCATOR)
>>>>>
>>>>
>>>>
>>>>
>>>
>>
>>
>>
>
--
Regards
Kefu Chai
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Understanding some of the Cmake logics
2017-10-10 14:58 ` kefu chai
@ 2017-10-10 15:13 ` Willem Jan Withagen
2017-10-10 15:19 ` kefu chai
0 siblings, 1 reply; 10+ messages in thread
From: Willem Jan Withagen @ 2017-10-10 15:13 UTC (permalink / raw)
To: kefu chai; +Cc: ceph-devel
On 10-10-2017 16:58, kefu chai wrote:
> On Tue, Oct 10, 2017 at 9:54 PM, Willem Jan Withagen <wjw@digiware.nl> wrote:
>> On 10-10-2017 15:31, kefu chai wrote:
>>> On Tue, Oct 10, 2017 at 8:06 PM, Willem Jan Withagen <wjw@digiware.nl> wrote:
>>>> On 10-10-2017 13:14, kefu chai wrote:
>>>>> + ceph-devel
>>>>>
>>>>> On Tue, Oct 10, 2017 at 4:49 AM, Willem Jan Withagen <wjw@digiware.nl> wrote:
>>>>>> Hi Kefu,
>>>>>>
>>>>>> Sort of wondering what in the current code below is the relation between
>>>>>> gpertools and tcmalloc?
>>>>>
>>>>> could you be more specific?
>>>>
>>>> While looking at the code of CMakeLists.txt:286, in the case that no
>>>> ALLOCATOR is defined.
>>>> There the current flow of code seems to suggest that gperf-tools is
>>>> always linked to tcmalloc. And once gperftools is found, the ALLOCATOR
>>>> is set to tcmalloc. This then breaks rocksdb building.
>>>> `set(HAVE_LIBTCMALLOC ${GPERFTOOLS_FOUND})`
>>>> like:
>>>> ```
>>>> CMake Error at cmake/modules/BuildRocksDB.cmake:63 (message):
>>>> Incompatible tcmalloc v2.6.1 and rocksdb v5.8.0, please install
>>>> gperf-tools
>>>> 2.5 (not 2.5.93) or >= 2.6.2, or switch to another allocator using
>>>> 'cmake -DALLOCATOR=libc'.
>>>> Call Stack (most recent call first):
>>>> src/CMakeLists.txt:825 (build_rocksdb)
>>>> ```
>>>>
>>>>>> Reason for the question is that once the gperf packages is installed,
>>>>>> as a bonus the code below declares the allocator to be tcmalloc.
>>>>>> But gperf under FreeBSD does not have tcmalloc.
>>>>>
>>>>> please note gperf here is not GNU perf [1] but google-perftools. and
>>>>> why do you claim the gperf under FreeBSD does not have tcmalloc? could
>>>>> you elaborate this a little bit?
>>>>
>>>> My bad, I mixed gperf and gperf-tools. We are talking about gperf-tools.
>>>> And yes, gperftools brings tcmalloc but not in the version mix that is
>>>> asked for. gperf-tools was at 2.5 one time, but has upgraded to 2.6.1_1
>>>> on FreeBSD[1]. Getting other versions will require manual work to get it
>>>> to work.
>>>>
>>>> But the question is:
>>>> Do I really want my allocator to be tcmalloc when the native
>>>> libc allocator is jemalloc?
>>>
>>> i think it's up to you.
>>
>> Oke, it is what the code now forces...
>> And I've tried avoiding tcmalloc uptil now, and would like to do so for
>> the time being..
>>
>>>> So is the test in CMakeLists.txt:286-324 what we want on FreeBSD?
>>>
>>> see http://tracker.ceph.com/issues/21422. so again, it's up to you.
>>
>> The tracker suggests that it is an error in 2.5.93, but
>> cmake/modules/BuildRocksDB.cmake suggests that it is only fixed in
>> 2.6.2...??
>
> 2.6.2 is not yet released. so if you want to stick with tcmalloc, you
> need to build from the latest (or recent enough) source. actually, i
> am not sure what you question is..
The question: Do you know if 2.6.2 is the first gperf-tools-version >
2.5.93 where this bug from the tracker is solved?
>>
>>>> The workaround this (for the time being) is to call cmake with
>>>> -DALLOCATOR=libc as is suggested.
>>
>> I'll use this as workaround for the time being, since that is least
>> invasive. And when that works out, try and make a PR for the allocator
>> selector in CmakeLists.txt, and/or wait until gperf-tools 2.6.2 is
>> available on FreeBSD.
>
> what is issue you want to address?
The issue I want to address is that I'm not able to build the intree
rocksdb ATM. Because tcmalloc is not => 2.6.2, and tcmalloc is required
because gperf-tools is detected.
Hardcoding the allocator during cmake-ing for now solves this catch-22.
--WjW
>
>>
>> --WjW
>>
>>>> [1]
>>>> https://svnweb.freebsd.org/ports/head/devel/google-perftools/Makefile?revision=450353&view=markup
>>>>
>>>>>
>>>>> --
>>>>> [1] http://portsmon.freebsd.org/portoverview.py?category=devel&portname=gperf
>>>>> [2] http://portsmon.freebsd.org/portoverview.py?category=devel&portname=google-perftools
>>>>>
>>>>>>
>>>>>> So I guess I need to fix the code below the to prevent this implicit
>>>>>> relation
>>>>>> under FreeBSD.
>>>>>>
>>>>>> And the problem that arises from this all lies in compiling rocksdb,
>>>>>> because
>>>>>> under FreeBSD that also needs to be build with libc as allocator.
>>>>>> The introduction of the new rocksdb building makes this clear.
>>>>>>
>>>>>> CMake Error at cmake/modules/BuildRocksDB.cmake:63 (message):
>>>>>> Incompatible tcmalloc v2.6.1 and rocksdb v5.8.0, please install
>>>>>> gperf-tools
>>>>>> 2.5 (not 2.5.93) or >= 2.6.2, or switch to another allocator using 'cmake
>>>>>> -DALLOCATOR=libc'.
>>>>>> Call Stack (most recent call first):
>>>>>> src/CMakeLists.txt:825 (build_rocksdb)
>>>>>>
>>>>>>
>>>>>> --WjW
>>>>>>
>>>>>>
>>>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 286)
>>>>>> if(ALLOCATOR)
>>>>>> ca9946d2b5a (Bassam Tabbara 2016-10-10 14:37:58 -0700 287)
>>>>>> if(${ALLOCATOR} MATCHES "tcmalloc(_minimal)?")
>>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 288)
>>>>>> find_package(gperftools REQUIRED)
>>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 289)
>>>>>> set(HAVE_LIBTCMALLOC ON)
>>>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 290)
>>>>>> elseif(${ALLOCATOR} STREQUAL "jemalloc")
>>>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 291)
>>>>>> find_package(JeMalloc REQUIRED)
>>>>>> 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 292)
>>>>>> set(HAVE_LIBJEMALLOC ${JEMALLOC_FOUND})
>>>>>> 83b74f52501 (Matt Benjamin 2016-12-21 23:36:37 -0500 293)
>>>>>> set(HAVE_JEMALLOC 1)
>>>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 294) endif()
>>>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 295)
>>>>>> else(ALLOCATOR)
>>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 296)
>>>>>> find_package(gperftools)
>>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 297)
>>>>>> set(HAVE_LIBTCMALLOC ${GPERFTOOLS_FOUND})
>>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 298) if(NOT
>>>>>> GPERFTOOLS_FOUND)
>>>>>> 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 299)
>>>>>> find_package(JeMalloc)
>>>>>> 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 300)
>>>>>> set(HAVE_LIBJEMALLOC ${JEMALLOC_FOUND})
>>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 301)
>>>>>> endif(NOT GPERFTOOLS_FOUND)
>>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 302)
>>>>>> if(GPERFTOOLS_FOUND)
>>>>>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 303)
>>>>>> set(ALLOCATOR tcmalloc)
>>>>>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 304)
>>>>>> elseif(JEMALLOC_FOUND)
>>>>>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 305)
>>>>>> set(ALLOCATOR jemalloc)
>>>>>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 306) else()
>>>>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 307)
>>>>>> if(NOT FREEBSD)
>>>>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 308) #
>>>>>> FreeBSD already has jemalloc as its default allocator
>>>>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 309)
>>>>>> message(WARNING "tcmalloc and jemalloc not found, falling back to libc")
>>>>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 310) endif()
>>>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 311)
>>>>>> set(ALLOCATOR "libc")
>>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 312)
>>>>>> endif(GPERFTOOLS_FOUND)
>>>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 313)
>>>>>> endif(ALLOCATOR)
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>>
>>
>
>
>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Understanding some of the Cmake logics
2017-10-10 15:13 ` Willem Jan Withagen
@ 2017-10-10 15:19 ` kefu chai
2017-10-10 15:27 ` Willem Jan Withagen
0 siblings, 1 reply; 10+ messages in thread
From: kefu chai @ 2017-10-10 15:19 UTC (permalink / raw)
To: Willem Jan Withagen; +Cc: ceph-devel
On Tue, Oct 10, 2017 at 11:13 PM, Willem Jan Withagen <wjw@digiware.nl> wrote:
> On 10-10-2017 16:58, kefu chai wrote:
>> On Tue, Oct 10, 2017 at 9:54 PM, Willem Jan Withagen <wjw@digiware.nl> wrote:
>>> On 10-10-2017 15:31, kefu chai wrote:
>>>> On Tue, Oct 10, 2017 at 8:06 PM, Willem Jan Withagen <wjw@digiware.nl> wrote:
>>>>> On 10-10-2017 13:14, kefu chai wrote:
>>>>>> + ceph-devel
>>>>>>
>>>>>> On Tue, Oct 10, 2017 at 4:49 AM, Willem Jan Withagen <wjw@digiware.nl> wrote:
>>>>>>> Hi Kefu,
>>>>>>>
>>>>>>> Sort of wondering what in the current code below is the relation between
>>>>>>> gpertools and tcmalloc?
>>>>>>
>>>>>> could you be more specific?
>>>>>
>>>>> While looking at the code of CMakeLists.txt:286, in the case that no
>>>>> ALLOCATOR is defined.
>>>>> There the current flow of code seems to suggest that gperf-tools is
>>>>> always linked to tcmalloc. And once gperftools is found, the ALLOCATOR
>>>>> is set to tcmalloc. This then breaks rocksdb building.
>>>>> `set(HAVE_LIBTCMALLOC ${GPERFTOOLS_FOUND})`
>>>>> like:
>>>>> ```
>>>>> CMake Error at cmake/modules/BuildRocksDB.cmake:63 (message):
>>>>> Incompatible tcmalloc v2.6.1 and rocksdb v5.8.0, please install
>>>>> gperf-tools
>>>>> 2.5 (not 2.5.93) or >= 2.6.2, or switch to another allocator using
>>>>> 'cmake -DALLOCATOR=libc'.
>>>>> Call Stack (most recent call first):
>>>>> src/CMakeLists.txt:825 (build_rocksdb)
>>>>> ```
>>>>>
>>>>>>> Reason for the question is that once the gperf packages is installed,
>>>>>>> as a bonus the code below declares the allocator to be tcmalloc.
>>>>>>> But gperf under FreeBSD does not have tcmalloc.
>>>>>>
>>>>>> please note gperf here is not GNU perf [1] but google-perftools. and
>>>>>> why do you claim the gperf under FreeBSD does not have tcmalloc? could
>>>>>> you elaborate this a little bit?
>>>>>
>>>>> My bad, I mixed gperf and gperf-tools. We are talking about gperf-tools.
>>>>> And yes, gperftools brings tcmalloc but not in the version mix that is
>>>>> asked for. gperf-tools was at 2.5 one time, but has upgraded to 2.6.1_1
>>>>> on FreeBSD[1]. Getting other versions will require manual work to get it
>>>>> to work.
>>>>>
>>>>> But the question is:
>>>>> Do I really want my allocator to be tcmalloc when the native
>>>>> libc allocator is jemalloc?
>>>>
>>>> i think it's up to you.
>>>
>>> Oke, it is what the code now forces...
>>> And I've tried avoiding tcmalloc uptil now, and would like to do so for
>>> the time being..
>>>
>>>>> So is the test in CMakeLists.txt:286-324 what we want on FreeBSD?
>>>>
>>>> see http://tracker.ceph.com/issues/21422. so again, it's up to you.
>>>
>>> The tracker suggests that it is an error in 2.5.93, but
>>> cmake/modules/BuildRocksDB.cmake suggests that it is only fixed in
>>> 2.6.2...??
>>
>> 2.6.2 is not yet released. so if you want to stick with tcmalloc, you
>> need to build from the latest (or recent enough) source. actually, i
>> am not sure what you question is..
>
> The question: Do you know if 2.6.2 is the first gperf-tools-version >
> 2.5.93 where this bug from the tracker is solved?
https://github.com/ceph/ceph/pull/17788/commits/ff8d6a730bb774271b9eea6af338e2d3ce3c48a5
>
>>>
>>>>> The workaround this (for the time being) is to call cmake with
>>>>> -DALLOCATOR=libc as is suggested.
>>>
>>> I'll use this as workaround for the time being, since that is least
>>> invasive. And when that works out, try and make a PR for the allocator
>>> selector in CmakeLists.txt, and/or wait until gperf-tools 2.6.2 is
>>> available on FreeBSD.
>>
>> what is issue you want to address?
>
> The issue I want to address is that I'm not able to build the intree
> rocksdb ATM. Because tcmalloc is not => 2.6.2, and tcmalloc is required
> because gperf-tools is detected.
the error message offered two solutions. i think, what you need is
just to choose one of them.
>
> Hardcoding the allocator during cmake-ing for now solves this catch-22.
>
> --WjW
>
>>
>>>
>>> --WjW
>>>
>>>>> [1]
>>>>> https://svnweb.freebsd.org/ports/head/devel/google-perftools/Makefile?revision=450353&view=markup
>>>>>
>>>>>>
>>>>>> --
>>>>>> [1] http://portsmon.freebsd.org/portoverview.py?category=devel&portname=gperf
>>>>>> [2] http://portsmon.freebsd.org/portoverview.py?category=devel&portname=google-perftools
>>>>>>
>>>>>>>
>>>>>>> So I guess I need to fix the code below the to prevent this implicit
>>>>>>> relation
>>>>>>> under FreeBSD.
>>>>>>>
>>>>>>> And the problem that arises from this all lies in compiling rocksdb,
>>>>>>> because
>>>>>>> under FreeBSD that also needs to be build with libc as allocator.
>>>>>>> The introduction of the new rocksdb building makes this clear.
>>>>>>>
>>>>>>> CMake Error at cmake/modules/BuildRocksDB.cmake:63 (message):
>>>>>>> Incompatible tcmalloc v2.6.1 and rocksdb v5.8.0, please install
>>>>>>> gperf-tools
>>>>>>> 2.5 (not 2.5.93) or >= 2.6.2, or switch to another allocator using 'cmake
>>>>>>> -DALLOCATOR=libc'.
>>>>>>> Call Stack (most recent call first):
>>>>>>> src/CMakeLists.txt:825 (build_rocksdb)
>>>>>>>
>>>>>>>
>>>>>>> --WjW
>>>>>>>
>>>>>>>
>>>>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 286)
>>>>>>> if(ALLOCATOR)
>>>>>>> ca9946d2b5a (Bassam Tabbara 2016-10-10 14:37:58 -0700 287)
>>>>>>> if(${ALLOCATOR} MATCHES "tcmalloc(_minimal)?")
>>>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 288)
>>>>>>> find_package(gperftools REQUIRED)
>>>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 289)
>>>>>>> set(HAVE_LIBTCMALLOC ON)
>>>>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 290)
>>>>>>> elseif(${ALLOCATOR} STREQUAL "jemalloc")
>>>>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 291)
>>>>>>> find_package(JeMalloc REQUIRED)
>>>>>>> 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 292)
>>>>>>> set(HAVE_LIBJEMALLOC ${JEMALLOC_FOUND})
>>>>>>> 83b74f52501 (Matt Benjamin 2016-12-21 23:36:37 -0500 293)
>>>>>>> set(HAVE_JEMALLOC 1)
>>>>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 294) endif()
>>>>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 295)
>>>>>>> else(ALLOCATOR)
>>>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 296)
>>>>>>> find_package(gperftools)
>>>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 297)
>>>>>>> set(HAVE_LIBTCMALLOC ${GPERFTOOLS_FOUND})
>>>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 298) if(NOT
>>>>>>> GPERFTOOLS_FOUND)
>>>>>>> 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 299)
>>>>>>> find_package(JeMalloc)
>>>>>>> 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 300)
>>>>>>> set(HAVE_LIBJEMALLOC ${JEMALLOC_FOUND})
>>>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 301)
>>>>>>> endif(NOT GPERFTOOLS_FOUND)
>>>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 302)
>>>>>>> if(GPERFTOOLS_FOUND)
>>>>>>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 303)
>>>>>>> set(ALLOCATOR tcmalloc)
>>>>>>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 304)
>>>>>>> elseif(JEMALLOC_FOUND)
>>>>>>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 305)
>>>>>>> set(ALLOCATOR jemalloc)
>>>>>>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 306) else()
>>>>>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 307)
>>>>>>> if(NOT FREEBSD)
>>>>>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 308) #
>>>>>>> FreeBSD already has jemalloc as its default allocator
>>>>>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 309)
>>>>>>> message(WARNING "tcmalloc and jemalloc not found, falling back to libc")
>>>>>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 310) endif()
>>>>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 311)
>>>>>>> set(ALLOCATOR "libc")
>>>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 312)
>>>>>>> endif(GPERFTOOLS_FOUND)
>>>>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 313)
>>>>>>> endif(ALLOCATOR)
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>
>>
>>
>>
>
--
Regards
Kefu Chai
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Understanding some of the Cmake logics
2017-10-10 15:19 ` kefu chai
@ 2017-10-10 15:27 ` Willem Jan Withagen
2017-10-10 15:35 ` kefu chai
0 siblings, 1 reply; 10+ messages in thread
From: Willem Jan Withagen @ 2017-10-10 15:27 UTC (permalink / raw)
To: kefu chai; +Cc: ceph-devel
On 10-10-2017 17:19, kefu chai wrote:
> On Tue, Oct 10, 2017 at 11:13 PM, Willem Jan Withagen <wjw@digiware.nl> wrote:
>> On 10-10-2017 16:58, kefu chai wrote:
>>> On Tue, Oct 10, 2017 at 9:54 PM, Willem Jan Withagen <wjw@digiware.nl> wrote:
>>>> On 10-10-2017 15:31, kefu chai wrote:
>>>>> On Tue, Oct 10, 2017 at 8:06 PM, Willem Jan Withagen <wjw@digiware.nl> wrote:
>>>>>> On 10-10-2017 13:14, kefu chai wrote:
>>>>>>> + ceph-devel
>>>>>>>
>>>>>>> On Tue, Oct 10, 2017 at 4:49 AM, Willem Jan Withagen <wjw@digiware.nl> wrote:
>>>>>>>> Hi Kefu,
>>>>>>>>
>>>>>>>> Sort of wondering what in the current code below is the relation between
>>>>>>>> gpertools and tcmalloc?
>>>>>>>
>>>>>>> could you be more specific?
>>>>>>
>>>>>> While looking at the code of CMakeLists.txt:286, in the case that no
>>>>>> ALLOCATOR is defined.
>>>>>> There the current flow of code seems to suggest that gperf-tools is
>>>>>> always linked to tcmalloc. And once gperftools is found, the ALLOCATOR
>>>>>> is set to tcmalloc. This then breaks rocksdb building.
>>>>>> `set(HAVE_LIBTCMALLOC ${GPERFTOOLS_FOUND})`
>>>>>> like:
>>>>>> ```
>>>>>> CMake Error at cmake/modules/BuildRocksDB.cmake:63 (message):
>>>>>> Incompatible tcmalloc v2.6.1 and rocksdb v5.8.0, please install
>>>>>> gperf-tools
>>>>>> 2.5 (not 2.5.93) or >= 2.6.2, or switch to another allocator using
>>>>>> 'cmake -DALLOCATOR=libc'.
>>>>>> Call Stack (most recent call first):
>>>>>> src/CMakeLists.txt:825 (build_rocksdb)
>>>>>> ```
>>>>>>
>>>>>>>> Reason for the question is that once the gperf packages is installed,
>>>>>>>> as a bonus the code below declares the allocator to be tcmalloc.
>>>>>>>> But gperf under FreeBSD does not have tcmalloc.
>>>>>>>
>>>>>>> please note gperf here is not GNU perf [1] but google-perftools. and
>>>>>>> why do you claim the gperf under FreeBSD does not have tcmalloc? could
>>>>>>> you elaborate this a little bit?
>>>>>>
>>>>>> My bad, I mixed gperf and gperf-tools. We are talking about gperf-tools.
>>>>>> And yes, gperftools brings tcmalloc but not in the version mix that is
>>>>>> asked for. gperf-tools was at 2.5 one time, but has upgraded to 2.6.1_1
>>>>>> on FreeBSD[1]. Getting other versions will require manual work to get it
>>>>>> to work.
>>>>>>
>>>>>> But the question is:
>>>>>> Do I really want my allocator to be tcmalloc when the native
>>>>>> libc allocator is jemalloc?
>>>>>
>>>>> i think it's up to you.
>>>>
>>>> Oke, it is what the code now forces...
>>>> And I've tried avoiding tcmalloc uptil now, and would like to do so for
>>>> the time being..
>>>>
>>>>>> So is the test in CMakeLists.txt:286-324 what we want on FreeBSD?
>>>>>
>>>>> see http://tracker.ceph.com/issues/21422. so again, it's up to you.
>>>>
>>>> The tracker suggests that it is an error in 2.5.93, but
>>>> cmake/modules/BuildRocksDB.cmake suggests that it is only fixed in
>>>> 2.6.2...??
>>>
>>> 2.6.2 is not yet released. so if you want to stick with tcmalloc, you
>>> need to build from the latest (or recent enough) source. actually, i
>>> am not sure what you question is..
>>
>> The question: Do you know if 2.6.2 is the first gperf-tools-version >
>> 2.5.93 where this bug from the tracker is solved?
>
> https://github.com/ceph/ceph/pull/17788/commits/ff8d6a730bb774271b9eea6af338e2d3ce3c48a5
fascinating reading...
Shows that managing version can be a bitch.
>>>>>> The workaround this (for the time being) is to call cmake with
>>>>>> -DALLOCATOR=libc as is suggested.
>>>>
>>>> I'll use this as workaround for the time being, since that is least
>>>> invasive. And when that works out, try and make a PR for the allocator
>>>> selector in CmakeLists.txt, and/or wait until gperf-tools 2.6.2 is
>>>> available on FreeBSD.
>>>
>>> what is issue you want to address?
>>
>> The issue I want to address is that I'm not able to build the intree
>> rocksdb ATM. Because tcmalloc is not => 2.6.2, and tcmalloc is required
>> because gperf-tools is detected.
>
> the error message offered two solutions. i think, what you need is
> just to choose one of them.
Right, so I'm going with libc for the moment...
Thanx for your patient explanations,
--WjW
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Understanding some of the Cmake logics
2017-10-10 15:27 ` Willem Jan Withagen
@ 2017-10-10 15:35 ` kefu chai
2017-10-10 15:41 ` Willem Jan Withagen
0 siblings, 1 reply; 10+ messages in thread
From: kefu chai @ 2017-10-10 15:35 UTC (permalink / raw)
To: Willem Jan Withagen; +Cc: ceph-devel
On Tue, Oct 10, 2017 at 11:27 PM, Willem Jan Withagen <wjw@digiware.nl> wrote:
> On 10-10-2017 17:19, kefu chai wrote:
>> On Tue, Oct 10, 2017 at 11:13 PM, Willem Jan Withagen <wjw@digiware.nl> wrote:
>>> On 10-10-2017 16:58, kefu chai wrote:
>>>> On Tue, Oct 10, 2017 at 9:54 PM, Willem Jan Withagen <wjw@digiware.nl> wrote:
>>>>> On 10-10-2017 15:31, kefu chai wrote:
>>>>>> On Tue, Oct 10, 2017 at 8:06 PM, Willem Jan Withagen <wjw@digiware.nl> wrote:
>>>>>>> On 10-10-2017 13:14, kefu chai wrote:
>>>>>>>> + ceph-devel
>>>>>>>>
>>>>>>>> On Tue, Oct 10, 2017 at 4:49 AM, Willem Jan Withagen <wjw@digiware.nl> wrote:
>>>>>>>>> Hi Kefu,
>>>>>>>>>
>>>>>>>>> Sort of wondering what in the current code below is the relation between
>>>>>>>>> gpertools and tcmalloc?
>>>>>>>>
>>>>>>>> could you be more specific?
>>>>>>>
>>>>>>> While looking at the code of CMakeLists.txt:286, in the case that no
>>>>>>> ALLOCATOR is defined.
>>>>>>> There the current flow of code seems to suggest that gperf-tools is
>>>>>>> always linked to tcmalloc. And once gperftools is found, the ALLOCATOR
>>>>>>> is set to tcmalloc. This then breaks rocksdb building.
>>>>>>> `set(HAVE_LIBTCMALLOC ${GPERFTOOLS_FOUND})`
>>>>>>> like:
>>>>>>> ```
>>>>>>> CMake Error at cmake/modules/BuildRocksDB.cmake:63 (message):
>>>>>>> Incompatible tcmalloc v2.6.1 and rocksdb v5.8.0, please install
>>>>>>> gperf-tools
>>>>>>> 2.5 (not 2.5.93) or >= 2.6.2, or switch to another allocator using
>>>>>>> 'cmake -DALLOCATOR=libc'.
>>>>>>> Call Stack (most recent call first):
>>>>>>> src/CMakeLists.txt:825 (build_rocksdb)
>>>>>>> ```
>>>>>>>
>>>>>>>>> Reason for the question is that once the gperf packages is installed,
>>>>>>>>> as a bonus the code below declares the allocator to be tcmalloc.
>>>>>>>>> But gperf under FreeBSD does not have tcmalloc.
>>>>>>>>
>>>>>>>> please note gperf here is not GNU perf [1] but google-perftools. and
>>>>>>>> why do you claim the gperf under FreeBSD does not have tcmalloc? could
>>>>>>>> you elaborate this a little bit?
>>>>>>>
>>>>>>> My bad, I mixed gperf and gperf-tools. We are talking about gperf-tools.
>>>>>>> And yes, gperftools brings tcmalloc but not in the version mix that is
>>>>>>> asked for. gperf-tools was at 2.5 one time, but has upgraded to 2.6.1_1
>>>>>>> on FreeBSD[1]. Getting other versions will require manual work to get it
>>>>>>> to work.
>>>>>>>
>>>>>>> But the question is:
>>>>>>> Do I really want my allocator to be tcmalloc when the native
>>>>>>> libc allocator is jemalloc?
>>>>>>
>>>>>> i think it's up to you.
>>>>>
>>>>> Oke, it is what the code now forces...
>>>>> And I've tried avoiding tcmalloc uptil now, and would like to do so for
>>>>> the time being..
>>>>>
>>>>>>> So is the test in CMakeLists.txt:286-324 what we want on FreeBSD?
>>>>>>
>>>>>> see http://tracker.ceph.com/issues/21422. so again, it's up to you.
>>>>>
>>>>> The tracker suggests that it is an error in 2.5.93, but
>>>>> cmake/modules/BuildRocksDB.cmake suggests that it is only fixed in
>>>>> 2.6.2...??
>>>>
>>>> 2.6.2 is not yet released. so if you want to stick with tcmalloc, you
>>>> need to build from the latest (or recent enough) source. actually, i
>>>> am not sure what you question is..
>>>
>>> The question: Do you know if 2.6.2 is the first gperf-tools-version >
>>> 2.5.93 where this bug from the tracker is solved?
>>
>> https://github.com/ceph/ceph/pull/17788/commits/ff8d6a730bb774271b9eea6af338e2d3ce3c48a5
>
> fascinating reading...
> Shows that managing version can be a bitch.
>
>>>>>>> The workaround this (for the time being) is to call cmake with
>>>>>>> -DALLOCATOR=libc as is suggested.
>>>>>
>>>>> I'll use this as workaround for the time being, since that is least
>>>>> invasive. And when that works out, try and make a PR for the allocator
>>>>> selector in CmakeLists.txt, and/or wait until gperf-tools 2.6.2 is
>>>>> available on FreeBSD.
>>>>
>>>> what is issue you want to address?
>>>
>>> The issue I want to address is that I'm not able to build the intree
>>> rocksdb ATM. Because tcmalloc is not => 2.6.2, and tcmalloc is required
>>> because gperf-tools is detected.
>>
>> the error message offered two solutions. i think, what you need is
>> just to choose one of them.
>
> Right, so I'm going with libc for the moment...
>
> Thanx for your patient explanations,
i did send a mail with title of 'fix for "crash in rocksdb LRUCache
destructor with tcmalloc"' to this mailing list. if you have had read
it, it'd save both of us some time.
> --WjW
>
>
--
Regards
Kefu Chai
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Understanding some of the Cmake logics
2017-10-10 15:35 ` kefu chai
@ 2017-10-10 15:41 ` Willem Jan Withagen
0 siblings, 0 replies; 10+ messages in thread
From: Willem Jan Withagen @ 2017-10-10 15:41 UTC (permalink / raw)
To: kefu chai; +Cc: ceph-devel
On 10-10-2017 17:35, kefu chai wrote:
> On Tue, Oct 10, 2017 at 11:27 PM, Willem Jan Withagen <wjw@digiware.nl> wrote:
>> On 10-10-2017 17:19, kefu chai wrote:
>>> On Tue, Oct 10, 2017 at 11:13 PM, Willem Jan Withagen <wjw@digiware.nl> wrote:
>>>> On 10-10-2017 16:58, kefu chai wrote:
>>>>> On Tue, Oct 10, 2017 at 9:54 PM, Willem Jan Withagen <wjw@digiware.nl> wrote:
>>>>>> On 10-10-2017 15:31, kefu chai wrote:
>>>>>>> On Tue, Oct 10, 2017 at 8:06 PM, Willem Jan Withagen <wjw@digiware.nl> wrote:
>>>>>>>> On 10-10-2017 13:14, kefu chai wrote:
>>>>>>>>> + ceph-devel
>>>>>>>>>
>>>>>>>>> On Tue, Oct 10, 2017 at 4:49 AM, Willem Jan Withagen <wjw@digiware.nl> wrote:
>>>>>>>>>> Hi Kefu,
>>>>>>>>>>
>>>>>>>>>> Sort of wondering what in the current code below is the relation between
>>>>>>>>>> gpertools and tcmalloc?
>>>>>>>>>
>>>>>>>>> could you be more specific?
>>>>>>>>
>>>>>>>> While looking at the code of CMakeLists.txt:286, in the case that no
>>>>>>>> ALLOCATOR is defined.
>>>>>>>> There the current flow of code seems to suggest that gperf-tools is
>>>>>>>> always linked to tcmalloc. And once gperftools is found, the ALLOCATOR
>>>>>>>> is set to tcmalloc. This then breaks rocksdb building.
>>>>>>>> `set(HAVE_LIBTCMALLOC ${GPERFTOOLS_FOUND})`
>>>>>>>> like:
>>>>>>>> ```
>>>>>>>> CMake Error at cmake/modules/BuildRocksDB.cmake:63 (message):
>>>>>>>> Incompatible tcmalloc v2.6.1 and rocksdb v5.8.0, please install
>>>>>>>> gperf-tools
>>>>>>>> 2.5 (not 2.5.93) or >= 2.6.2, or switch to another allocator using
>>>>>>>> 'cmake -DALLOCATOR=libc'.
>>>>>>>> Call Stack (most recent call first):
>>>>>>>> src/CMakeLists.txt:825 (build_rocksdb)
>>>>>>>> ```
>>>>>>>>
>>>>>>>>>> Reason for the question is that once the gperf packages is installed,
>>>>>>>>>> as a bonus the code below declares the allocator to be tcmalloc.
>>>>>>>>>> But gperf under FreeBSD does not have tcmalloc.
>>>>>>>>>
>>>>>>>>> please note gperf here is not GNU perf [1] but google-perftools. and
>>>>>>>>> why do you claim the gperf under FreeBSD does not have tcmalloc? could
>>>>>>>>> you elaborate this a little bit?
>>>>>>>>
>>>>>>>> My bad, I mixed gperf and gperf-tools. We are talking about gperf-tools.
>>>>>>>> And yes, gperftools brings tcmalloc but not in the version mix that is
>>>>>>>> asked for. gperf-tools was at 2.5 one time, but has upgraded to 2.6.1_1
>>>>>>>> on FreeBSD[1]. Getting other versions will require manual work to get it
>>>>>>>> to work.
>>>>>>>>
>>>>>>>> But the question is:
>>>>>>>> Do I really want my allocator to be tcmalloc when the native
>>>>>>>> libc allocator is jemalloc?
>>>>>>>
>>>>>>> i think it's up to you.
>>>>>>
>>>>>> Oke, it is what the code now forces...
>>>>>> And I've tried avoiding tcmalloc uptil now, and would like to do so for
>>>>>> the time being..
>>>>>>
>>>>>>>> So is the test in CMakeLists.txt:286-324 what we want on FreeBSD?
>>>>>>>
>>>>>>> see http://tracker.ceph.com/issues/21422. so again, it's up to you.
>>>>>>
>>>>>> The tracker suggests that it is an error in 2.5.93, but
>>>>>> cmake/modules/BuildRocksDB.cmake suggests that it is only fixed in
>>>>>> 2.6.2...??
>>>>>
>>>>> 2.6.2 is not yet released. so if you want to stick with tcmalloc, you
>>>>> need to build from the latest (or recent enough) source. actually, i
>>>>> am not sure what you question is..
>>>>
>>>> The question: Do you know if 2.6.2 is the first gperf-tools-version >
>>>> 2.5.93 where this bug from the tracker is solved?
>>>
>>> https://github.com/ceph/ceph/pull/17788/commits/ff8d6a730bb774271b9eea6af338e2d3ce3c48a5
>>
>> fascinating reading...
>> Shows that managing version can be a bitch.
>>
>>>>>>>> The workaround this (for the time being) is to call cmake with
>>>>>>>> -DALLOCATOR=libc as is suggested.
>>>>>>
>>>>>> I'll use this as workaround for the time being, since that is least
>>>>>> invasive. And when that works out, try and make a PR for the allocator
>>>>>> selector in CmakeLists.txt, and/or wait until gperf-tools 2.6.2 is
>>>>>> available on FreeBSD.
>>>>>
>>>>> what is issue you want to address?
>>>>
>>>> The issue I want to address is that I'm not able to build the intree
>>>> rocksdb ATM. Because tcmalloc is not => 2.6.2, and tcmalloc is required
>>>> because gperf-tools is detected.
>>>
>>> the error message offered two solutions. i think, what you need is
>>> just to choose one of them.
>>
>> Right, so I'm going with libc for the moment...
>>
>> Thanx for your patient explanations,
>
> i did send a mail with title of 'fix for "crash in rocksdb LRUCache
> destructor with tcmalloc"' to this mailing list. if you have had read
> it, it'd save both of us some time.
I was/am about a zillion mails behind on Ceph-lists due to too much work
at my paying dayjob. And then still I might have forgotten about it
because it speaks about fc25/26 and tcmalloc. Where I only just today
found out that gperf-tools does use tcmalloc.
So I'm sorry for your time, but for me I feel a bit more educated.
--WjW
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2017-10-10 15:42 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <b06780c7-13ce-1274-735f-806a1f4f8062@digiware.nl>
2017-10-10 11:14 ` Understanding some of the Cmake logics kefu chai
2017-10-10 12:06 ` Willem Jan Withagen
2017-10-10 13:31 ` kefu chai
2017-10-10 13:54 ` Willem Jan Withagen
2017-10-10 14:58 ` kefu chai
2017-10-10 15:13 ` Willem Jan Withagen
2017-10-10 15:19 ` kefu chai
2017-10-10 15:27 ` Willem Jan Withagen
2017-10-10 15:35 ` kefu chai
2017-10-10 15:41 ` Willem Jan Withagen
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.