* Re: Understanding some of the Cmake logics [not found] <b06780c7-13ce-1274-735f-806a1f4f8062@digiware.nl> @ 2017-10-10 11:14 ` kefu chai 2017-10-10 12:06 ` Willem Jan Withagen 0 siblings, 1 reply; 10+ messages in thread From: kefu chai @ 2017-10-10 11:14 UTC (permalink / raw) To: Willem Jan Withagen, ceph-devel + ceph-devel On Tue, Oct 10, 2017 at 4:49 AM, Willem Jan Withagen <wjw@digiware.nl> wrote: > Hi Kefu, > > Sort of wondering what in the current code below is the relation between > gpertools and tcmalloc? could you be more specific? > > Reason for the question is that once the gperf packages is installed, > as a bonus the code below declares the allocator to be tcmalloc. > But gperf under FreeBSD does not have tcmalloc. please note gperf here is not GNU perf [1] but google-perftools. and why do you claim the gperf under FreeBSD does not have tcmalloc? could you elaborate this a little bit? -- [1] http://portsmon.freebsd.org/portoverview.py?category=devel&portname=gperf [2] http://portsmon.freebsd.org/portoverview.py?category=devel&portname=google-perftools > > So I guess I need to fix the code below the to prevent this implicit > relation > under FreeBSD. > > And the problem that arises from this all lies in compiling rocksdb, > because > under FreeBSD that also needs to be build with libc as allocator. > The introduction of the new rocksdb building makes this clear. > > CMake Error at cmake/modules/BuildRocksDB.cmake:63 (message): > Incompatible tcmalloc v2.6.1 and rocksdb v5.8.0, please install > gperf-tools > 2.5 (not 2.5.93) or >= 2.6.2, or switch to another allocator using 'cmake > -DALLOCATOR=libc'. > Call Stack (most recent call first): > src/CMakeLists.txt:825 (build_rocksdb) > > > --WjW > > > a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 286) > if(ALLOCATOR) > ca9946d2b5a (Bassam Tabbara 2016-10-10 14:37:58 -0700 287) > if(${ALLOCATOR} MATCHES "tcmalloc(_minimal)?") > 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 288) > find_package(gperftools REQUIRED) > 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 289) > set(HAVE_LIBTCMALLOC ON) > a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 290) > elseif(${ALLOCATOR} STREQUAL "jemalloc") > a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 291) > find_package(JeMalloc REQUIRED) > 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 292) > set(HAVE_LIBJEMALLOC ${JEMALLOC_FOUND}) > 83b74f52501 (Matt Benjamin 2016-12-21 23:36:37 -0500 293) > set(HAVE_JEMALLOC 1) > a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 294) endif() > a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 295) > else(ALLOCATOR) > 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 296) > find_package(gperftools) > 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 297) > set(HAVE_LIBTCMALLOC ${GPERFTOOLS_FOUND}) > 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 298) if(NOT > GPERFTOOLS_FOUND) > 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 299) > find_package(JeMalloc) > 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 300) > set(HAVE_LIBJEMALLOC ${JEMALLOC_FOUND}) > 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 301) > endif(NOT GPERFTOOLS_FOUND) > 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 302) > if(GPERFTOOLS_FOUND) > 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 303) > set(ALLOCATOR tcmalloc) > 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 304) > elseif(JEMALLOC_FOUND) > 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 305) > set(ALLOCATOR jemalloc) > 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 306) else() > ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 307) > if(NOT FREEBSD) > ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 308) # > FreeBSD already has jemalloc as its default allocator > ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 309) > message(WARNING "tcmalloc and jemalloc not found, falling back to libc") > ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 310) endif() > a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 311) > set(ALLOCATOR "libc") > 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 312) > endif(GPERFTOOLS_FOUND) > a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 313) > endif(ALLOCATOR) > -- Regards Kefu Chai ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Understanding some of the Cmake logics 2017-10-10 11:14 ` Understanding some of the Cmake logics kefu chai @ 2017-10-10 12:06 ` Willem Jan Withagen 2017-10-10 13:31 ` kefu chai 0 siblings, 1 reply; 10+ messages in thread From: Willem Jan Withagen @ 2017-10-10 12:06 UTC (permalink / raw) To: kefu chai, ceph-devel On 10-10-2017 13:14, kefu chai wrote: > + ceph-devel > > On Tue, Oct 10, 2017 at 4:49 AM, Willem Jan Withagen <wjw@digiware.nl> wrote: >> Hi Kefu, >> >> Sort of wondering what in the current code below is the relation between >> gpertools and tcmalloc? > > could you be more specific? While looking at the code of CMakeLists.txt:286, in the case that no ALLOCATOR is defined. There the current flow of code seems to suggest that gperf-tools is always linked to tcmalloc. And once gperftools is found, the ALLOCATOR is set to tcmalloc. This then breaks rocksdb building. `set(HAVE_LIBTCMALLOC ${GPERFTOOLS_FOUND})` like: ``` CMake Error at cmake/modules/BuildRocksDB.cmake:63 (message): Incompatible tcmalloc v2.6.1 and rocksdb v5.8.0, please install gperf-tools 2.5 (not 2.5.93) or >= 2.6.2, or switch to another allocator using 'cmake -DALLOCATOR=libc'. Call Stack (most recent call first): src/CMakeLists.txt:825 (build_rocksdb) ``` >> Reason for the question is that once the gperf packages is installed, >> as a bonus the code below declares the allocator to be tcmalloc. >> But gperf under FreeBSD does not have tcmalloc. > > please note gperf here is not GNU perf [1] but google-perftools. and > why do you claim the gperf under FreeBSD does not have tcmalloc? could > you elaborate this a little bit? My bad, I mixed gperf and gperf-tools. We are talking about gperf-tools. And yes, gperftools brings tcmalloc but not in the version mix that is asked for. gperf-tools was at 2.5 one time, but has upgraded to 2.6.1_1 on FreeBSD[1]. Getting other versions will require manual work to get it to work. But the question is: Do I really want my allocator to be tcmalloc when the native libc allocator is jemalloc? So is the test in CMakeLists.txt:286-324 what we want on FreeBSD? The workaround this (for the time being) is to call cmake with -DALLOCATOR=libc as is suggested. --WjW [1] https://svnweb.freebsd.org/ports/head/devel/google-perftools/Makefile?revision=450353&view=markup > > -- > [1] http://portsmon.freebsd.org/portoverview.py?category=devel&portname=gperf > [2] http://portsmon.freebsd.org/portoverview.py?category=devel&portname=google-perftools > >> >> So I guess I need to fix the code below the to prevent this implicit >> relation >> under FreeBSD. >> >> And the problem that arises from this all lies in compiling rocksdb, >> because >> under FreeBSD that also needs to be build with libc as allocator. >> The introduction of the new rocksdb building makes this clear. >> >> CMake Error at cmake/modules/BuildRocksDB.cmake:63 (message): >> Incompatible tcmalloc v2.6.1 and rocksdb v5.8.0, please install >> gperf-tools >> 2.5 (not 2.5.93) or >= 2.6.2, or switch to another allocator using 'cmake >> -DALLOCATOR=libc'. >> Call Stack (most recent call first): >> src/CMakeLists.txt:825 (build_rocksdb) >> >> >> --WjW >> >> >> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 286) >> if(ALLOCATOR) >> ca9946d2b5a (Bassam Tabbara 2016-10-10 14:37:58 -0700 287) >> if(${ALLOCATOR} MATCHES "tcmalloc(_minimal)?") >> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 288) >> find_package(gperftools REQUIRED) >> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 289) >> set(HAVE_LIBTCMALLOC ON) >> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 290) >> elseif(${ALLOCATOR} STREQUAL "jemalloc") >> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 291) >> find_package(JeMalloc REQUIRED) >> 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 292) >> set(HAVE_LIBJEMALLOC ${JEMALLOC_FOUND}) >> 83b74f52501 (Matt Benjamin 2016-12-21 23:36:37 -0500 293) >> set(HAVE_JEMALLOC 1) >> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 294) endif() >> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 295) >> else(ALLOCATOR) >> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 296) >> find_package(gperftools) >> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 297) >> set(HAVE_LIBTCMALLOC ${GPERFTOOLS_FOUND}) >> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 298) if(NOT >> GPERFTOOLS_FOUND) >> 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 299) >> find_package(JeMalloc) >> 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 300) >> set(HAVE_LIBJEMALLOC ${JEMALLOC_FOUND}) >> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 301) >> endif(NOT GPERFTOOLS_FOUND) >> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 302) >> if(GPERFTOOLS_FOUND) >> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 303) >> set(ALLOCATOR tcmalloc) >> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 304) >> elseif(JEMALLOC_FOUND) >> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 305) >> set(ALLOCATOR jemalloc) >> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 306) else() >> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 307) >> if(NOT FREEBSD) >> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 308) # >> FreeBSD already has jemalloc as its default allocator >> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 309) >> message(WARNING "tcmalloc and jemalloc not found, falling back to libc") >> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 310) endif() >> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 311) >> set(ALLOCATOR "libc") >> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 312) >> endif(GPERFTOOLS_FOUND) >> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 313) >> endif(ALLOCATOR) >> > > > ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Understanding some of the Cmake logics 2017-10-10 12:06 ` Willem Jan Withagen @ 2017-10-10 13:31 ` kefu chai 2017-10-10 13:54 ` Willem Jan Withagen 0 siblings, 1 reply; 10+ messages in thread From: kefu chai @ 2017-10-10 13:31 UTC (permalink / raw) To: Willem Jan Withagen; +Cc: ceph-devel On Tue, Oct 10, 2017 at 8:06 PM, Willem Jan Withagen <wjw@digiware.nl> wrote: > On 10-10-2017 13:14, kefu chai wrote: >> + ceph-devel >> >> On Tue, Oct 10, 2017 at 4:49 AM, Willem Jan Withagen <wjw@digiware.nl> wrote: >>> Hi Kefu, >>> >>> Sort of wondering what in the current code below is the relation between >>> gpertools and tcmalloc? >> >> could you be more specific? > > While looking at the code of CMakeLists.txt:286, in the case that no > ALLOCATOR is defined. > There the current flow of code seems to suggest that gperf-tools is > always linked to tcmalloc. And once gperftools is found, the ALLOCATOR > is set to tcmalloc. This then breaks rocksdb building. > `set(HAVE_LIBTCMALLOC ${GPERFTOOLS_FOUND})` > like: > ``` > CMake Error at cmake/modules/BuildRocksDB.cmake:63 (message): > Incompatible tcmalloc v2.6.1 and rocksdb v5.8.0, please install > gperf-tools > 2.5 (not 2.5.93) or >= 2.6.2, or switch to another allocator using > 'cmake -DALLOCATOR=libc'. > Call Stack (most recent call first): > src/CMakeLists.txt:825 (build_rocksdb) > ``` > >>> Reason for the question is that once the gperf packages is installed, >>> as a bonus the code below declares the allocator to be tcmalloc. >>> But gperf under FreeBSD does not have tcmalloc. >> >> please note gperf here is not GNU perf [1] but google-perftools. and >> why do you claim the gperf under FreeBSD does not have tcmalloc? could >> you elaborate this a little bit? > > My bad, I mixed gperf and gperf-tools. We are talking about gperf-tools. > And yes, gperftools brings tcmalloc but not in the version mix that is > asked for. gperf-tools was at 2.5 one time, but has upgraded to 2.6.1_1 > on FreeBSD[1]. Getting other versions will require manual work to get it > to work. > > But the question is: > Do I really want my allocator to be tcmalloc when the native > libc allocator is jemalloc? i think it's up to you. > So is the test in CMakeLists.txt:286-324 what we want on FreeBSD? see http://tracker.ceph.com/issues/21422. so again, it's up to you. > > The workaround this (for the time being) is to call cmake with > -DALLOCATOR=libc as is suggested. > > --WjW > > [1] > https://svnweb.freebsd.org/ports/head/devel/google-perftools/Makefile?revision=450353&view=markup > >> >> -- >> [1] http://portsmon.freebsd.org/portoverview.py?category=devel&portname=gperf >> [2] http://portsmon.freebsd.org/portoverview.py?category=devel&portname=google-perftools >> >>> >>> So I guess I need to fix the code below the to prevent this implicit >>> relation >>> under FreeBSD. >>> >>> And the problem that arises from this all lies in compiling rocksdb, >>> because >>> under FreeBSD that also needs to be build with libc as allocator. >>> The introduction of the new rocksdb building makes this clear. >>> >>> CMake Error at cmake/modules/BuildRocksDB.cmake:63 (message): >>> Incompatible tcmalloc v2.6.1 and rocksdb v5.8.0, please install >>> gperf-tools >>> 2.5 (not 2.5.93) or >= 2.6.2, or switch to another allocator using 'cmake >>> -DALLOCATOR=libc'. >>> Call Stack (most recent call first): >>> src/CMakeLists.txt:825 (build_rocksdb) >>> >>> >>> --WjW >>> >>> >>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 286) >>> if(ALLOCATOR) >>> ca9946d2b5a (Bassam Tabbara 2016-10-10 14:37:58 -0700 287) >>> if(${ALLOCATOR} MATCHES "tcmalloc(_minimal)?") >>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 288) >>> find_package(gperftools REQUIRED) >>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 289) >>> set(HAVE_LIBTCMALLOC ON) >>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 290) >>> elseif(${ALLOCATOR} STREQUAL "jemalloc") >>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 291) >>> find_package(JeMalloc REQUIRED) >>> 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 292) >>> set(HAVE_LIBJEMALLOC ${JEMALLOC_FOUND}) >>> 83b74f52501 (Matt Benjamin 2016-12-21 23:36:37 -0500 293) >>> set(HAVE_JEMALLOC 1) >>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 294) endif() >>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 295) >>> else(ALLOCATOR) >>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 296) >>> find_package(gperftools) >>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 297) >>> set(HAVE_LIBTCMALLOC ${GPERFTOOLS_FOUND}) >>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 298) if(NOT >>> GPERFTOOLS_FOUND) >>> 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 299) >>> find_package(JeMalloc) >>> 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 300) >>> set(HAVE_LIBJEMALLOC ${JEMALLOC_FOUND}) >>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 301) >>> endif(NOT GPERFTOOLS_FOUND) >>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 302) >>> if(GPERFTOOLS_FOUND) >>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 303) >>> set(ALLOCATOR tcmalloc) >>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 304) >>> elseif(JEMALLOC_FOUND) >>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 305) >>> set(ALLOCATOR jemalloc) >>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 306) else() >>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 307) >>> if(NOT FREEBSD) >>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 308) # >>> FreeBSD already has jemalloc as its default allocator >>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 309) >>> message(WARNING "tcmalloc and jemalloc not found, falling back to libc") >>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 310) endif() >>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 311) >>> set(ALLOCATOR "libc") >>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 312) >>> endif(GPERFTOOLS_FOUND) >>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 313) >>> endif(ALLOCATOR) >>> >> >> >> > -- Regards Kefu Chai ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Understanding some of the Cmake logics 2017-10-10 13:31 ` kefu chai @ 2017-10-10 13:54 ` Willem Jan Withagen 2017-10-10 14:58 ` kefu chai 0 siblings, 1 reply; 10+ messages in thread From: Willem Jan Withagen @ 2017-10-10 13:54 UTC (permalink / raw) To: kefu chai; +Cc: ceph-devel On 10-10-2017 15:31, kefu chai wrote: > On Tue, Oct 10, 2017 at 8:06 PM, Willem Jan Withagen <wjw@digiware.nl> wrote: >> On 10-10-2017 13:14, kefu chai wrote: >>> + ceph-devel >>> >>> On Tue, Oct 10, 2017 at 4:49 AM, Willem Jan Withagen <wjw@digiware.nl> wrote: >>>> Hi Kefu, >>>> >>>> Sort of wondering what in the current code below is the relation between >>>> gpertools and tcmalloc? >>> >>> could you be more specific? >> >> While looking at the code of CMakeLists.txt:286, in the case that no >> ALLOCATOR is defined. >> There the current flow of code seems to suggest that gperf-tools is >> always linked to tcmalloc. And once gperftools is found, the ALLOCATOR >> is set to tcmalloc. This then breaks rocksdb building. >> `set(HAVE_LIBTCMALLOC ${GPERFTOOLS_FOUND})` >> like: >> ``` >> CMake Error at cmake/modules/BuildRocksDB.cmake:63 (message): >> Incompatible tcmalloc v2.6.1 and rocksdb v5.8.0, please install >> gperf-tools >> 2.5 (not 2.5.93) or >= 2.6.2, or switch to another allocator using >> 'cmake -DALLOCATOR=libc'. >> Call Stack (most recent call first): >> src/CMakeLists.txt:825 (build_rocksdb) >> ``` >> >>>> Reason for the question is that once the gperf packages is installed, >>>> as a bonus the code below declares the allocator to be tcmalloc. >>>> But gperf under FreeBSD does not have tcmalloc. >>> >>> please note gperf here is not GNU perf [1] but google-perftools. and >>> why do you claim the gperf under FreeBSD does not have tcmalloc? could >>> you elaborate this a little bit? >> >> My bad, I mixed gperf and gperf-tools. We are talking about gperf-tools. >> And yes, gperftools brings tcmalloc but not in the version mix that is >> asked for. gperf-tools was at 2.5 one time, but has upgraded to 2.6.1_1 >> on FreeBSD[1]. Getting other versions will require manual work to get it >> to work. >> >> But the question is: >> Do I really want my allocator to be tcmalloc when the native >> libc allocator is jemalloc? > > i think it's up to you. Oke, it is what the code now forces... And I've tried avoiding tcmalloc uptil now, and would like to do so for the time being.. >> So is the test in CMakeLists.txt:286-324 what we want on FreeBSD? > > see http://tracker.ceph.com/issues/21422. so again, it's up to you. The tracker suggests that it is an error in 2.5.93, but cmake/modules/BuildRocksDB.cmake suggests that it is only fixed in 2.6.2...?? >> The workaround this (for the time being) is to call cmake with >> -DALLOCATOR=libc as is suggested. I'll use this as workaround for the time being, since that is least invasive. And when that works out, try and make a PR for the allocator selector in CmakeLists.txt, and/or wait until gperf-tools 2.6.2 is available on FreeBSD. --WjW >> [1] >> https://svnweb.freebsd.org/ports/head/devel/google-perftools/Makefile?revision=450353&view=markup >> >>> >>> -- >>> [1] http://portsmon.freebsd.org/portoverview.py?category=devel&portname=gperf >>> [2] http://portsmon.freebsd.org/portoverview.py?category=devel&portname=google-perftools >>> >>>> >>>> So I guess I need to fix the code below the to prevent this implicit >>>> relation >>>> under FreeBSD. >>>> >>>> And the problem that arises from this all lies in compiling rocksdb, >>>> because >>>> under FreeBSD that also needs to be build with libc as allocator. >>>> The introduction of the new rocksdb building makes this clear. >>>> >>>> CMake Error at cmake/modules/BuildRocksDB.cmake:63 (message): >>>> Incompatible tcmalloc v2.6.1 and rocksdb v5.8.0, please install >>>> gperf-tools >>>> 2.5 (not 2.5.93) or >= 2.6.2, or switch to another allocator using 'cmake >>>> -DALLOCATOR=libc'. >>>> Call Stack (most recent call first): >>>> src/CMakeLists.txt:825 (build_rocksdb) >>>> >>>> >>>> --WjW >>>> >>>> >>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 286) >>>> if(ALLOCATOR) >>>> ca9946d2b5a (Bassam Tabbara 2016-10-10 14:37:58 -0700 287) >>>> if(${ALLOCATOR} MATCHES "tcmalloc(_minimal)?") >>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 288) >>>> find_package(gperftools REQUIRED) >>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 289) >>>> set(HAVE_LIBTCMALLOC ON) >>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 290) >>>> elseif(${ALLOCATOR} STREQUAL "jemalloc") >>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 291) >>>> find_package(JeMalloc REQUIRED) >>>> 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 292) >>>> set(HAVE_LIBJEMALLOC ${JEMALLOC_FOUND}) >>>> 83b74f52501 (Matt Benjamin 2016-12-21 23:36:37 -0500 293) >>>> set(HAVE_JEMALLOC 1) >>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 294) endif() >>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 295) >>>> else(ALLOCATOR) >>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 296) >>>> find_package(gperftools) >>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 297) >>>> set(HAVE_LIBTCMALLOC ${GPERFTOOLS_FOUND}) >>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 298) if(NOT >>>> GPERFTOOLS_FOUND) >>>> 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 299) >>>> find_package(JeMalloc) >>>> 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 300) >>>> set(HAVE_LIBJEMALLOC ${JEMALLOC_FOUND}) >>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 301) >>>> endif(NOT GPERFTOOLS_FOUND) >>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 302) >>>> if(GPERFTOOLS_FOUND) >>>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 303) >>>> set(ALLOCATOR tcmalloc) >>>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 304) >>>> elseif(JEMALLOC_FOUND) >>>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 305) >>>> set(ALLOCATOR jemalloc) >>>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 306) else() >>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 307) >>>> if(NOT FREEBSD) >>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 308) # >>>> FreeBSD already has jemalloc as its default allocator >>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 309) >>>> message(WARNING "tcmalloc and jemalloc not found, falling back to libc") >>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 310) endif() >>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 311) >>>> set(ALLOCATOR "libc") >>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 312) >>>> endif(GPERFTOOLS_FOUND) >>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 313) >>>> endif(ALLOCATOR) >>>> >>> >>> >>> >> > > > ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Understanding some of the Cmake logics 2017-10-10 13:54 ` Willem Jan Withagen @ 2017-10-10 14:58 ` kefu chai 2017-10-10 15:13 ` Willem Jan Withagen 0 siblings, 1 reply; 10+ messages in thread From: kefu chai @ 2017-10-10 14:58 UTC (permalink / raw) To: Willem Jan Withagen; +Cc: ceph-devel On Tue, Oct 10, 2017 at 9:54 PM, Willem Jan Withagen <wjw@digiware.nl> wrote: > On 10-10-2017 15:31, kefu chai wrote: >> On Tue, Oct 10, 2017 at 8:06 PM, Willem Jan Withagen <wjw@digiware.nl> wrote: >>> On 10-10-2017 13:14, kefu chai wrote: >>>> + ceph-devel >>>> >>>> On Tue, Oct 10, 2017 at 4:49 AM, Willem Jan Withagen <wjw@digiware.nl> wrote: >>>>> Hi Kefu, >>>>> >>>>> Sort of wondering what in the current code below is the relation between >>>>> gpertools and tcmalloc? >>>> >>>> could you be more specific? >>> >>> While looking at the code of CMakeLists.txt:286, in the case that no >>> ALLOCATOR is defined. >>> There the current flow of code seems to suggest that gperf-tools is >>> always linked to tcmalloc. And once gperftools is found, the ALLOCATOR >>> is set to tcmalloc. This then breaks rocksdb building. >>> `set(HAVE_LIBTCMALLOC ${GPERFTOOLS_FOUND})` >>> like: >>> ``` >>> CMake Error at cmake/modules/BuildRocksDB.cmake:63 (message): >>> Incompatible tcmalloc v2.6.1 and rocksdb v5.8.0, please install >>> gperf-tools >>> 2.5 (not 2.5.93) or >= 2.6.2, or switch to another allocator using >>> 'cmake -DALLOCATOR=libc'. >>> Call Stack (most recent call first): >>> src/CMakeLists.txt:825 (build_rocksdb) >>> ``` >>> >>>>> Reason for the question is that once the gperf packages is installed, >>>>> as a bonus the code below declares the allocator to be tcmalloc. >>>>> But gperf under FreeBSD does not have tcmalloc. >>>> >>>> please note gperf here is not GNU perf [1] but google-perftools. and >>>> why do you claim the gperf under FreeBSD does not have tcmalloc? could >>>> you elaborate this a little bit? >>> >>> My bad, I mixed gperf and gperf-tools. We are talking about gperf-tools. >>> And yes, gperftools brings tcmalloc but not in the version mix that is >>> asked for. gperf-tools was at 2.5 one time, but has upgraded to 2.6.1_1 >>> on FreeBSD[1]. Getting other versions will require manual work to get it >>> to work. >>> >>> But the question is: >>> Do I really want my allocator to be tcmalloc when the native >>> libc allocator is jemalloc? >> >> i think it's up to you. > > Oke, it is what the code now forces... > And I've tried avoiding tcmalloc uptil now, and would like to do so for > the time being.. > >>> So is the test in CMakeLists.txt:286-324 what we want on FreeBSD? >> >> see http://tracker.ceph.com/issues/21422. so again, it's up to you. > > The tracker suggests that it is an error in 2.5.93, but > cmake/modules/BuildRocksDB.cmake suggests that it is only fixed in > 2.6.2...?? 2.6.2 is not yet released. so if you want to stick with tcmalloc, you need to build from the latest (or recent enough) source. actually, i am not sure what you question is.. > >>> The workaround this (for the time being) is to call cmake with >>> -DALLOCATOR=libc as is suggested. > > I'll use this as workaround for the time being, since that is least > invasive. And when that works out, try and make a PR for the allocator > selector in CmakeLists.txt, and/or wait until gperf-tools 2.6.2 is > available on FreeBSD. what is issue you want to address? > > --WjW > >>> [1] >>> https://svnweb.freebsd.org/ports/head/devel/google-perftools/Makefile?revision=450353&view=markup >>> >>>> >>>> -- >>>> [1] http://portsmon.freebsd.org/portoverview.py?category=devel&portname=gperf >>>> [2] http://portsmon.freebsd.org/portoverview.py?category=devel&portname=google-perftools >>>> >>>>> >>>>> So I guess I need to fix the code below the to prevent this implicit >>>>> relation >>>>> under FreeBSD. >>>>> >>>>> And the problem that arises from this all lies in compiling rocksdb, >>>>> because >>>>> under FreeBSD that also needs to be build with libc as allocator. >>>>> The introduction of the new rocksdb building makes this clear. >>>>> >>>>> CMake Error at cmake/modules/BuildRocksDB.cmake:63 (message): >>>>> Incompatible tcmalloc v2.6.1 and rocksdb v5.8.0, please install >>>>> gperf-tools >>>>> 2.5 (not 2.5.93) or >= 2.6.2, or switch to another allocator using 'cmake >>>>> -DALLOCATOR=libc'. >>>>> Call Stack (most recent call first): >>>>> src/CMakeLists.txt:825 (build_rocksdb) >>>>> >>>>> >>>>> --WjW >>>>> >>>>> >>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 286) >>>>> if(ALLOCATOR) >>>>> ca9946d2b5a (Bassam Tabbara 2016-10-10 14:37:58 -0700 287) >>>>> if(${ALLOCATOR} MATCHES "tcmalloc(_minimal)?") >>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 288) >>>>> find_package(gperftools REQUIRED) >>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 289) >>>>> set(HAVE_LIBTCMALLOC ON) >>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 290) >>>>> elseif(${ALLOCATOR} STREQUAL "jemalloc") >>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 291) >>>>> find_package(JeMalloc REQUIRED) >>>>> 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 292) >>>>> set(HAVE_LIBJEMALLOC ${JEMALLOC_FOUND}) >>>>> 83b74f52501 (Matt Benjamin 2016-12-21 23:36:37 -0500 293) >>>>> set(HAVE_JEMALLOC 1) >>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 294) endif() >>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 295) >>>>> else(ALLOCATOR) >>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 296) >>>>> find_package(gperftools) >>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 297) >>>>> set(HAVE_LIBTCMALLOC ${GPERFTOOLS_FOUND}) >>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 298) if(NOT >>>>> GPERFTOOLS_FOUND) >>>>> 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 299) >>>>> find_package(JeMalloc) >>>>> 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 300) >>>>> set(HAVE_LIBJEMALLOC ${JEMALLOC_FOUND}) >>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 301) >>>>> endif(NOT GPERFTOOLS_FOUND) >>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 302) >>>>> if(GPERFTOOLS_FOUND) >>>>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 303) >>>>> set(ALLOCATOR tcmalloc) >>>>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 304) >>>>> elseif(JEMALLOC_FOUND) >>>>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 305) >>>>> set(ALLOCATOR jemalloc) >>>>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 306) else() >>>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 307) >>>>> if(NOT FREEBSD) >>>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 308) # >>>>> FreeBSD already has jemalloc as its default allocator >>>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 309) >>>>> message(WARNING "tcmalloc and jemalloc not found, falling back to libc") >>>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 310) endif() >>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 311) >>>>> set(ALLOCATOR "libc") >>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 312) >>>>> endif(GPERFTOOLS_FOUND) >>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 313) >>>>> endif(ALLOCATOR) >>>>> >>>> >>>> >>>> >>> >> >> >> > -- Regards Kefu Chai ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Understanding some of the Cmake logics 2017-10-10 14:58 ` kefu chai @ 2017-10-10 15:13 ` Willem Jan Withagen 2017-10-10 15:19 ` kefu chai 0 siblings, 1 reply; 10+ messages in thread From: Willem Jan Withagen @ 2017-10-10 15:13 UTC (permalink / raw) To: kefu chai; +Cc: ceph-devel On 10-10-2017 16:58, kefu chai wrote: > On Tue, Oct 10, 2017 at 9:54 PM, Willem Jan Withagen <wjw@digiware.nl> wrote: >> On 10-10-2017 15:31, kefu chai wrote: >>> On Tue, Oct 10, 2017 at 8:06 PM, Willem Jan Withagen <wjw@digiware.nl> wrote: >>>> On 10-10-2017 13:14, kefu chai wrote: >>>>> + ceph-devel >>>>> >>>>> On Tue, Oct 10, 2017 at 4:49 AM, Willem Jan Withagen <wjw@digiware.nl> wrote: >>>>>> Hi Kefu, >>>>>> >>>>>> Sort of wondering what in the current code below is the relation between >>>>>> gpertools and tcmalloc? >>>>> >>>>> could you be more specific? >>>> >>>> While looking at the code of CMakeLists.txt:286, in the case that no >>>> ALLOCATOR is defined. >>>> There the current flow of code seems to suggest that gperf-tools is >>>> always linked to tcmalloc. And once gperftools is found, the ALLOCATOR >>>> is set to tcmalloc. This then breaks rocksdb building. >>>> `set(HAVE_LIBTCMALLOC ${GPERFTOOLS_FOUND})` >>>> like: >>>> ``` >>>> CMake Error at cmake/modules/BuildRocksDB.cmake:63 (message): >>>> Incompatible tcmalloc v2.6.1 and rocksdb v5.8.0, please install >>>> gperf-tools >>>> 2.5 (not 2.5.93) or >= 2.6.2, or switch to another allocator using >>>> 'cmake -DALLOCATOR=libc'. >>>> Call Stack (most recent call first): >>>> src/CMakeLists.txt:825 (build_rocksdb) >>>> ``` >>>> >>>>>> Reason for the question is that once the gperf packages is installed, >>>>>> as a bonus the code below declares the allocator to be tcmalloc. >>>>>> But gperf under FreeBSD does not have tcmalloc. >>>>> >>>>> please note gperf here is not GNU perf [1] but google-perftools. and >>>>> why do you claim the gperf under FreeBSD does not have tcmalloc? could >>>>> you elaborate this a little bit? >>>> >>>> My bad, I mixed gperf and gperf-tools. We are talking about gperf-tools. >>>> And yes, gperftools brings tcmalloc but not in the version mix that is >>>> asked for. gperf-tools was at 2.5 one time, but has upgraded to 2.6.1_1 >>>> on FreeBSD[1]. Getting other versions will require manual work to get it >>>> to work. >>>> >>>> But the question is: >>>> Do I really want my allocator to be tcmalloc when the native >>>> libc allocator is jemalloc? >>> >>> i think it's up to you. >> >> Oke, it is what the code now forces... >> And I've tried avoiding tcmalloc uptil now, and would like to do so for >> the time being.. >> >>>> So is the test in CMakeLists.txt:286-324 what we want on FreeBSD? >>> >>> see http://tracker.ceph.com/issues/21422. so again, it's up to you. >> >> The tracker suggests that it is an error in 2.5.93, but >> cmake/modules/BuildRocksDB.cmake suggests that it is only fixed in >> 2.6.2...?? > > 2.6.2 is not yet released. so if you want to stick with tcmalloc, you > need to build from the latest (or recent enough) source. actually, i > am not sure what you question is.. The question: Do you know if 2.6.2 is the first gperf-tools-version > 2.5.93 where this bug from the tracker is solved? >> >>>> The workaround this (for the time being) is to call cmake with >>>> -DALLOCATOR=libc as is suggested. >> >> I'll use this as workaround for the time being, since that is least >> invasive. And when that works out, try and make a PR for the allocator >> selector in CmakeLists.txt, and/or wait until gperf-tools 2.6.2 is >> available on FreeBSD. > > what is issue you want to address? The issue I want to address is that I'm not able to build the intree rocksdb ATM. Because tcmalloc is not => 2.6.2, and tcmalloc is required because gperf-tools is detected. Hardcoding the allocator during cmake-ing for now solves this catch-22. --WjW > >> >> --WjW >> >>>> [1] >>>> https://svnweb.freebsd.org/ports/head/devel/google-perftools/Makefile?revision=450353&view=markup >>>> >>>>> >>>>> -- >>>>> [1] http://portsmon.freebsd.org/portoverview.py?category=devel&portname=gperf >>>>> [2] http://portsmon.freebsd.org/portoverview.py?category=devel&portname=google-perftools >>>>> >>>>>> >>>>>> So I guess I need to fix the code below the to prevent this implicit >>>>>> relation >>>>>> under FreeBSD. >>>>>> >>>>>> And the problem that arises from this all lies in compiling rocksdb, >>>>>> because >>>>>> under FreeBSD that also needs to be build with libc as allocator. >>>>>> The introduction of the new rocksdb building makes this clear. >>>>>> >>>>>> CMake Error at cmake/modules/BuildRocksDB.cmake:63 (message): >>>>>> Incompatible tcmalloc v2.6.1 and rocksdb v5.8.0, please install >>>>>> gperf-tools >>>>>> 2.5 (not 2.5.93) or >= 2.6.2, or switch to another allocator using 'cmake >>>>>> -DALLOCATOR=libc'. >>>>>> Call Stack (most recent call first): >>>>>> src/CMakeLists.txt:825 (build_rocksdb) >>>>>> >>>>>> >>>>>> --WjW >>>>>> >>>>>> >>>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 286) >>>>>> if(ALLOCATOR) >>>>>> ca9946d2b5a (Bassam Tabbara 2016-10-10 14:37:58 -0700 287) >>>>>> if(${ALLOCATOR} MATCHES "tcmalloc(_minimal)?") >>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 288) >>>>>> find_package(gperftools REQUIRED) >>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 289) >>>>>> set(HAVE_LIBTCMALLOC ON) >>>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 290) >>>>>> elseif(${ALLOCATOR} STREQUAL "jemalloc") >>>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 291) >>>>>> find_package(JeMalloc REQUIRED) >>>>>> 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 292) >>>>>> set(HAVE_LIBJEMALLOC ${JEMALLOC_FOUND}) >>>>>> 83b74f52501 (Matt Benjamin 2016-12-21 23:36:37 -0500 293) >>>>>> set(HAVE_JEMALLOC 1) >>>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 294) endif() >>>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 295) >>>>>> else(ALLOCATOR) >>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 296) >>>>>> find_package(gperftools) >>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 297) >>>>>> set(HAVE_LIBTCMALLOC ${GPERFTOOLS_FOUND}) >>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 298) if(NOT >>>>>> GPERFTOOLS_FOUND) >>>>>> 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 299) >>>>>> find_package(JeMalloc) >>>>>> 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 300) >>>>>> set(HAVE_LIBJEMALLOC ${JEMALLOC_FOUND}) >>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 301) >>>>>> endif(NOT GPERFTOOLS_FOUND) >>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 302) >>>>>> if(GPERFTOOLS_FOUND) >>>>>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 303) >>>>>> set(ALLOCATOR tcmalloc) >>>>>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 304) >>>>>> elseif(JEMALLOC_FOUND) >>>>>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 305) >>>>>> set(ALLOCATOR jemalloc) >>>>>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 306) else() >>>>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 307) >>>>>> if(NOT FREEBSD) >>>>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 308) # >>>>>> FreeBSD already has jemalloc as its default allocator >>>>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 309) >>>>>> message(WARNING "tcmalloc and jemalloc not found, falling back to libc") >>>>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 310) endif() >>>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 311) >>>>>> set(ALLOCATOR "libc") >>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 312) >>>>>> endif(GPERFTOOLS_FOUND) >>>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 313) >>>>>> endif(ALLOCATOR) >>>>>> >>>>> >>>>> >>>>> >>>> >>> >>> >>> >> > > > ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Understanding some of the Cmake logics 2017-10-10 15:13 ` Willem Jan Withagen @ 2017-10-10 15:19 ` kefu chai 2017-10-10 15:27 ` Willem Jan Withagen 0 siblings, 1 reply; 10+ messages in thread From: kefu chai @ 2017-10-10 15:19 UTC (permalink / raw) To: Willem Jan Withagen; +Cc: ceph-devel On Tue, Oct 10, 2017 at 11:13 PM, Willem Jan Withagen <wjw@digiware.nl> wrote: > On 10-10-2017 16:58, kefu chai wrote: >> On Tue, Oct 10, 2017 at 9:54 PM, Willem Jan Withagen <wjw@digiware.nl> wrote: >>> On 10-10-2017 15:31, kefu chai wrote: >>>> On Tue, Oct 10, 2017 at 8:06 PM, Willem Jan Withagen <wjw@digiware.nl> wrote: >>>>> On 10-10-2017 13:14, kefu chai wrote: >>>>>> + ceph-devel >>>>>> >>>>>> On Tue, Oct 10, 2017 at 4:49 AM, Willem Jan Withagen <wjw@digiware.nl> wrote: >>>>>>> Hi Kefu, >>>>>>> >>>>>>> Sort of wondering what in the current code below is the relation between >>>>>>> gpertools and tcmalloc? >>>>>> >>>>>> could you be more specific? >>>>> >>>>> While looking at the code of CMakeLists.txt:286, in the case that no >>>>> ALLOCATOR is defined. >>>>> There the current flow of code seems to suggest that gperf-tools is >>>>> always linked to tcmalloc. And once gperftools is found, the ALLOCATOR >>>>> is set to tcmalloc. This then breaks rocksdb building. >>>>> `set(HAVE_LIBTCMALLOC ${GPERFTOOLS_FOUND})` >>>>> like: >>>>> ``` >>>>> CMake Error at cmake/modules/BuildRocksDB.cmake:63 (message): >>>>> Incompatible tcmalloc v2.6.1 and rocksdb v5.8.0, please install >>>>> gperf-tools >>>>> 2.5 (not 2.5.93) or >= 2.6.2, or switch to another allocator using >>>>> 'cmake -DALLOCATOR=libc'. >>>>> Call Stack (most recent call first): >>>>> src/CMakeLists.txt:825 (build_rocksdb) >>>>> ``` >>>>> >>>>>>> Reason for the question is that once the gperf packages is installed, >>>>>>> as a bonus the code below declares the allocator to be tcmalloc. >>>>>>> But gperf under FreeBSD does not have tcmalloc. >>>>>> >>>>>> please note gperf here is not GNU perf [1] but google-perftools. and >>>>>> why do you claim the gperf under FreeBSD does not have tcmalloc? could >>>>>> you elaborate this a little bit? >>>>> >>>>> My bad, I mixed gperf and gperf-tools. We are talking about gperf-tools. >>>>> And yes, gperftools brings tcmalloc but not in the version mix that is >>>>> asked for. gperf-tools was at 2.5 one time, but has upgraded to 2.6.1_1 >>>>> on FreeBSD[1]. Getting other versions will require manual work to get it >>>>> to work. >>>>> >>>>> But the question is: >>>>> Do I really want my allocator to be tcmalloc when the native >>>>> libc allocator is jemalloc? >>>> >>>> i think it's up to you. >>> >>> Oke, it is what the code now forces... >>> And I've tried avoiding tcmalloc uptil now, and would like to do so for >>> the time being.. >>> >>>>> So is the test in CMakeLists.txt:286-324 what we want on FreeBSD? >>>> >>>> see http://tracker.ceph.com/issues/21422. so again, it's up to you. >>> >>> The tracker suggests that it is an error in 2.5.93, but >>> cmake/modules/BuildRocksDB.cmake suggests that it is only fixed in >>> 2.6.2...?? >> >> 2.6.2 is not yet released. so if you want to stick with tcmalloc, you >> need to build from the latest (or recent enough) source. actually, i >> am not sure what you question is.. > > The question: Do you know if 2.6.2 is the first gperf-tools-version > > 2.5.93 where this bug from the tracker is solved? https://github.com/ceph/ceph/pull/17788/commits/ff8d6a730bb774271b9eea6af338e2d3ce3c48a5 > >>> >>>>> The workaround this (for the time being) is to call cmake with >>>>> -DALLOCATOR=libc as is suggested. >>> >>> I'll use this as workaround for the time being, since that is least >>> invasive. And when that works out, try and make a PR for the allocator >>> selector in CmakeLists.txt, and/or wait until gperf-tools 2.6.2 is >>> available on FreeBSD. >> >> what is issue you want to address? > > The issue I want to address is that I'm not able to build the intree > rocksdb ATM. Because tcmalloc is not => 2.6.2, and tcmalloc is required > because gperf-tools is detected. the error message offered two solutions. i think, what you need is just to choose one of them. > > Hardcoding the allocator during cmake-ing for now solves this catch-22. > > --WjW > >> >>> >>> --WjW >>> >>>>> [1] >>>>> https://svnweb.freebsd.org/ports/head/devel/google-perftools/Makefile?revision=450353&view=markup >>>>> >>>>>> >>>>>> -- >>>>>> [1] http://portsmon.freebsd.org/portoverview.py?category=devel&portname=gperf >>>>>> [2] http://portsmon.freebsd.org/portoverview.py?category=devel&portname=google-perftools >>>>>> >>>>>>> >>>>>>> So I guess I need to fix the code below the to prevent this implicit >>>>>>> relation >>>>>>> under FreeBSD. >>>>>>> >>>>>>> And the problem that arises from this all lies in compiling rocksdb, >>>>>>> because >>>>>>> under FreeBSD that also needs to be build with libc as allocator. >>>>>>> The introduction of the new rocksdb building makes this clear. >>>>>>> >>>>>>> CMake Error at cmake/modules/BuildRocksDB.cmake:63 (message): >>>>>>> Incompatible tcmalloc v2.6.1 and rocksdb v5.8.0, please install >>>>>>> gperf-tools >>>>>>> 2.5 (not 2.5.93) or >= 2.6.2, or switch to another allocator using 'cmake >>>>>>> -DALLOCATOR=libc'. >>>>>>> Call Stack (most recent call first): >>>>>>> src/CMakeLists.txt:825 (build_rocksdb) >>>>>>> >>>>>>> >>>>>>> --WjW >>>>>>> >>>>>>> >>>>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 286) >>>>>>> if(ALLOCATOR) >>>>>>> ca9946d2b5a (Bassam Tabbara 2016-10-10 14:37:58 -0700 287) >>>>>>> if(${ALLOCATOR} MATCHES "tcmalloc(_minimal)?") >>>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 288) >>>>>>> find_package(gperftools REQUIRED) >>>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 289) >>>>>>> set(HAVE_LIBTCMALLOC ON) >>>>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 290) >>>>>>> elseif(${ALLOCATOR} STREQUAL "jemalloc") >>>>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 291) >>>>>>> find_package(JeMalloc REQUIRED) >>>>>>> 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 292) >>>>>>> set(HAVE_LIBJEMALLOC ${JEMALLOC_FOUND}) >>>>>>> 83b74f52501 (Matt Benjamin 2016-12-21 23:36:37 -0500 293) >>>>>>> set(HAVE_JEMALLOC 1) >>>>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 294) endif() >>>>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 295) >>>>>>> else(ALLOCATOR) >>>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 296) >>>>>>> find_package(gperftools) >>>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 297) >>>>>>> set(HAVE_LIBTCMALLOC ${GPERFTOOLS_FOUND}) >>>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 298) if(NOT >>>>>>> GPERFTOOLS_FOUND) >>>>>>> 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 299) >>>>>>> find_package(JeMalloc) >>>>>>> 0ef09f32d46 (Ali Maredia 2016-05-02 18:21:32 -0400 300) >>>>>>> set(HAVE_LIBJEMALLOC ${JEMALLOC_FOUND}) >>>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 301) >>>>>>> endif(NOT GPERFTOOLS_FOUND) >>>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 302) >>>>>>> if(GPERFTOOLS_FOUND) >>>>>>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 303) >>>>>>> set(ALLOCATOR tcmalloc) >>>>>>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 304) >>>>>>> elseif(JEMALLOC_FOUND) >>>>>>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 305) >>>>>>> set(ALLOCATOR jemalloc) >>>>>>> 48a1f8eff5b (Kefu Chai 2016-06-08 11:34:44 +0800 306) else() >>>>>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 307) >>>>>>> if(NOT FREEBSD) >>>>>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 308) # >>>>>>> FreeBSD already has jemalloc as its default allocator >>>>>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 309) >>>>>>> message(WARNING "tcmalloc and jemalloc not found, falling back to libc") >>>>>>> ce3b71acd20 (Willem Jan Withagen 2017-02-12 15:03:56 +0100 310) endif() >>>>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 311) >>>>>>> set(ALLOCATOR "libc") >>>>>>> 6c12edbf9d2 (Kefu Chai 2016-08-11 00:12:05 +0800 312) >>>>>>> endif(GPERFTOOLS_FOUND) >>>>>>> a614c3b9c1e (Ali Maredia 2015-12-30 21:06:08 -0500 313) >>>>>>> endif(ALLOCATOR) >>>>>>> >>>>>> >>>>>> >>>>>> >>>>> >>>> >>>> >>>> >>> >> >> >> > -- Regards Kefu Chai ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Understanding some of the Cmake logics 2017-10-10 15:19 ` kefu chai @ 2017-10-10 15:27 ` Willem Jan Withagen 2017-10-10 15:35 ` kefu chai 0 siblings, 1 reply; 10+ messages in thread From: Willem Jan Withagen @ 2017-10-10 15:27 UTC (permalink / raw) To: kefu chai; +Cc: ceph-devel On 10-10-2017 17:19, kefu chai wrote: > On Tue, Oct 10, 2017 at 11:13 PM, Willem Jan Withagen <wjw@digiware.nl> wrote: >> On 10-10-2017 16:58, kefu chai wrote: >>> On Tue, Oct 10, 2017 at 9:54 PM, Willem Jan Withagen <wjw@digiware.nl> wrote: >>>> On 10-10-2017 15:31, kefu chai wrote: >>>>> On Tue, Oct 10, 2017 at 8:06 PM, Willem Jan Withagen <wjw@digiware.nl> wrote: >>>>>> On 10-10-2017 13:14, kefu chai wrote: >>>>>>> + ceph-devel >>>>>>> >>>>>>> On Tue, Oct 10, 2017 at 4:49 AM, Willem Jan Withagen <wjw@digiware.nl> wrote: >>>>>>>> Hi Kefu, >>>>>>>> >>>>>>>> Sort of wondering what in the current code below is the relation between >>>>>>>> gpertools and tcmalloc? >>>>>>> >>>>>>> could you be more specific? >>>>>> >>>>>> While looking at the code of CMakeLists.txt:286, in the case that no >>>>>> ALLOCATOR is defined. >>>>>> There the current flow of code seems to suggest that gperf-tools is >>>>>> always linked to tcmalloc. And once gperftools is found, the ALLOCATOR >>>>>> is set to tcmalloc. This then breaks rocksdb building. >>>>>> `set(HAVE_LIBTCMALLOC ${GPERFTOOLS_FOUND})` >>>>>> like: >>>>>> ``` >>>>>> CMake Error at cmake/modules/BuildRocksDB.cmake:63 (message): >>>>>> Incompatible tcmalloc v2.6.1 and rocksdb v5.8.0, please install >>>>>> gperf-tools >>>>>> 2.5 (not 2.5.93) or >= 2.6.2, or switch to another allocator using >>>>>> 'cmake -DALLOCATOR=libc'. >>>>>> Call Stack (most recent call first): >>>>>> src/CMakeLists.txt:825 (build_rocksdb) >>>>>> ``` >>>>>> >>>>>>>> Reason for the question is that once the gperf packages is installed, >>>>>>>> as a bonus the code below declares the allocator to be tcmalloc. >>>>>>>> But gperf under FreeBSD does not have tcmalloc. >>>>>>> >>>>>>> please note gperf here is not GNU perf [1] but google-perftools. and >>>>>>> why do you claim the gperf under FreeBSD does not have tcmalloc? could >>>>>>> you elaborate this a little bit? >>>>>> >>>>>> My bad, I mixed gperf and gperf-tools. We are talking about gperf-tools. >>>>>> And yes, gperftools brings tcmalloc but not in the version mix that is >>>>>> asked for. gperf-tools was at 2.5 one time, but has upgraded to 2.6.1_1 >>>>>> on FreeBSD[1]. Getting other versions will require manual work to get it >>>>>> to work. >>>>>> >>>>>> But the question is: >>>>>> Do I really want my allocator to be tcmalloc when the native >>>>>> libc allocator is jemalloc? >>>>> >>>>> i think it's up to you. >>>> >>>> Oke, it is what the code now forces... >>>> And I've tried avoiding tcmalloc uptil now, and would like to do so for >>>> the time being.. >>>> >>>>>> So is the test in CMakeLists.txt:286-324 what we want on FreeBSD? >>>>> >>>>> see http://tracker.ceph.com/issues/21422. so again, it's up to you. >>>> >>>> The tracker suggests that it is an error in 2.5.93, but >>>> cmake/modules/BuildRocksDB.cmake suggests that it is only fixed in >>>> 2.6.2...?? >>> >>> 2.6.2 is not yet released. so if you want to stick with tcmalloc, you >>> need to build from the latest (or recent enough) source. actually, i >>> am not sure what you question is.. >> >> The question: Do you know if 2.6.2 is the first gperf-tools-version > >> 2.5.93 where this bug from the tracker is solved? > > https://github.com/ceph/ceph/pull/17788/commits/ff8d6a730bb774271b9eea6af338e2d3ce3c48a5 fascinating reading... Shows that managing version can be a bitch. >>>>>> The workaround this (for the time being) is to call cmake with >>>>>> -DALLOCATOR=libc as is suggested. >>>> >>>> I'll use this as workaround for the time being, since that is least >>>> invasive. And when that works out, try and make a PR for the allocator >>>> selector in CmakeLists.txt, and/or wait until gperf-tools 2.6.2 is >>>> available on FreeBSD. >>> >>> what is issue you want to address? >> >> The issue I want to address is that I'm not able to build the intree >> rocksdb ATM. Because tcmalloc is not => 2.6.2, and tcmalloc is required >> because gperf-tools is detected. > > the error message offered two solutions. i think, what you need is > just to choose one of them. Right, so I'm going with libc for the moment... Thanx for your patient explanations, --WjW ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Understanding some of the Cmake logics 2017-10-10 15:27 ` Willem Jan Withagen @ 2017-10-10 15:35 ` kefu chai 2017-10-10 15:41 ` Willem Jan Withagen 0 siblings, 1 reply; 10+ messages in thread From: kefu chai @ 2017-10-10 15:35 UTC (permalink / raw) To: Willem Jan Withagen; +Cc: ceph-devel On Tue, Oct 10, 2017 at 11:27 PM, Willem Jan Withagen <wjw@digiware.nl> wrote: > On 10-10-2017 17:19, kefu chai wrote: >> On Tue, Oct 10, 2017 at 11:13 PM, Willem Jan Withagen <wjw@digiware.nl> wrote: >>> On 10-10-2017 16:58, kefu chai wrote: >>>> On Tue, Oct 10, 2017 at 9:54 PM, Willem Jan Withagen <wjw@digiware.nl> wrote: >>>>> On 10-10-2017 15:31, kefu chai wrote: >>>>>> On Tue, Oct 10, 2017 at 8:06 PM, Willem Jan Withagen <wjw@digiware.nl> wrote: >>>>>>> On 10-10-2017 13:14, kefu chai wrote: >>>>>>>> + ceph-devel >>>>>>>> >>>>>>>> On Tue, Oct 10, 2017 at 4:49 AM, Willem Jan Withagen <wjw@digiware.nl> wrote: >>>>>>>>> Hi Kefu, >>>>>>>>> >>>>>>>>> Sort of wondering what in the current code below is the relation between >>>>>>>>> gpertools and tcmalloc? >>>>>>>> >>>>>>>> could you be more specific? >>>>>>> >>>>>>> While looking at the code of CMakeLists.txt:286, in the case that no >>>>>>> ALLOCATOR is defined. >>>>>>> There the current flow of code seems to suggest that gperf-tools is >>>>>>> always linked to tcmalloc. And once gperftools is found, the ALLOCATOR >>>>>>> is set to tcmalloc. This then breaks rocksdb building. >>>>>>> `set(HAVE_LIBTCMALLOC ${GPERFTOOLS_FOUND})` >>>>>>> like: >>>>>>> ``` >>>>>>> CMake Error at cmake/modules/BuildRocksDB.cmake:63 (message): >>>>>>> Incompatible tcmalloc v2.6.1 and rocksdb v5.8.0, please install >>>>>>> gperf-tools >>>>>>> 2.5 (not 2.5.93) or >= 2.6.2, or switch to another allocator using >>>>>>> 'cmake -DALLOCATOR=libc'. >>>>>>> Call Stack (most recent call first): >>>>>>> src/CMakeLists.txt:825 (build_rocksdb) >>>>>>> ``` >>>>>>> >>>>>>>>> Reason for the question is that once the gperf packages is installed, >>>>>>>>> as a bonus the code below declares the allocator to be tcmalloc. >>>>>>>>> But gperf under FreeBSD does not have tcmalloc. >>>>>>>> >>>>>>>> please note gperf here is not GNU perf [1] but google-perftools. and >>>>>>>> why do you claim the gperf under FreeBSD does not have tcmalloc? could >>>>>>>> you elaborate this a little bit? >>>>>>> >>>>>>> My bad, I mixed gperf and gperf-tools. We are talking about gperf-tools. >>>>>>> And yes, gperftools brings tcmalloc but not in the version mix that is >>>>>>> asked for. gperf-tools was at 2.5 one time, but has upgraded to 2.6.1_1 >>>>>>> on FreeBSD[1]. Getting other versions will require manual work to get it >>>>>>> to work. >>>>>>> >>>>>>> But the question is: >>>>>>> Do I really want my allocator to be tcmalloc when the native >>>>>>> libc allocator is jemalloc? >>>>>> >>>>>> i think it's up to you. >>>>> >>>>> Oke, it is what the code now forces... >>>>> And I've tried avoiding tcmalloc uptil now, and would like to do so for >>>>> the time being.. >>>>> >>>>>>> So is the test in CMakeLists.txt:286-324 what we want on FreeBSD? >>>>>> >>>>>> see http://tracker.ceph.com/issues/21422. so again, it's up to you. >>>>> >>>>> The tracker suggests that it is an error in 2.5.93, but >>>>> cmake/modules/BuildRocksDB.cmake suggests that it is only fixed in >>>>> 2.6.2...?? >>>> >>>> 2.6.2 is not yet released. so if you want to stick with tcmalloc, you >>>> need to build from the latest (or recent enough) source. actually, i >>>> am not sure what you question is.. >>> >>> The question: Do you know if 2.6.2 is the first gperf-tools-version > >>> 2.5.93 where this bug from the tracker is solved? >> >> https://github.com/ceph/ceph/pull/17788/commits/ff8d6a730bb774271b9eea6af338e2d3ce3c48a5 > > fascinating reading... > Shows that managing version can be a bitch. > >>>>>>> The workaround this (for the time being) is to call cmake with >>>>>>> -DALLOCATOR=libc as is suggested. >>>>> >>>>> I'll use this as workaround for the time being, since that is least >>>>> invasive. And when that works out, try and make a PR for the allocator >>>>> selector in CmakeLists.txt, and/or wait until gperf-tools 2.6.2 is >>>>> available on FreeBSD. >>>> >>>> what is issue you want to address? >>> >>> The issue I want to address is that I'm not able to build the intree >>> rocksdb ATM. Because tcmalloc is not => 2.6.2, and tcmalloc is required >>> because gperf-tools is detected. >> >> the error message offered two solutions. i think, what you need is >> just to choose one of them. > > Right, so I'm going with libc for the moment... > > Thanx for your patient explanations, i did send a mail with title of 'fix for "crash in rocksdb LRUCache destructor with tcmalloc"' to this mailing list. if you have had read it, it'd save both of us some time. > --WjW > > -- Regards Kefu Chai ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Understanding some of the Cmake logics 2017-10-10 15:35 ` kefu chai @ 2017-10-10 15:41 ` Willem Jan Withagen 0 siblings, 0 replies; 10+ messages in thread From: Willem Jan Withagen @ 2017-10-10 15:41 UTC (permalink / raw) To: kefu chai; +Cc: ceph-devel On 10-10-2017 17:35, kefu chai wrote: > On Tue, Oct 10, 2017 at 11:27 PM, Willem Jan Withagen <wjw@digiware.nl> wrote: >> On 10-10-2017 17:19, kefu chai wrote: >>> On Tue, Oct 10, 2017 at 11:13 PM, Willem Jan Withagen <wjw@digiware.nl> wrote: >>>> On 10-10-2017 16:58, kefu chai wrote: >>>>> On Tue, Oct 10, 2017 at 9:54 PM, Willem Jan Withagen <wjw@digiware.nl> wrote: >>>>>> On 10-10-2017 15:31, kefu chai wrote: >>>>>>> On Tue, Oct 10, 2017 at 8:06 PM, Willem Jan Withagen <wjw@digiware.nl> wrote: >>>>>>>> On 10-10-2017 13:14, kefu chai wrote: >>>>>>>>> + ceph-devel >>>>>>>>> >>>>>>>>> On Tue, Oct 10, 2017 at 4:49 AM, Willem Jan Withagen <wjw@digiware.nl> wrote: >>>>>>>>>> Hi Kefu, >>>>>>>>>> >>>>>>>>>> Sort of wondering what in the current code below is the relation between >>>>>>>>>> gpertools and tcmalloc? >>>>>>>>> >>>>>>>>> could you be more specific? >>>>>>>> >>>>>>>> While looking at the code of CMakeLists.txt:286, in the case that no >>>>>>>> ALLOCATOR is defined. >>>>>>>> There the current flow of code seems to suggest that gperf-tools is >>>>>>>> always linked to tcmalloc. And once gperftools is found, the ALLOCATOR >>>>>>>> is set to tcmalloc. This then breaks rocksdb building. >>>>>>>> `set(HAVE_LIBTCMALLOC ${GPERFTOOLS_FOUND})` >>>>>>>> like: >>>>>>>> ``` >>>>>>>> CMake Error at cmake/modules/BuildRocksDB.cmake:63 (message): >>>>>>>> Incompatible tcmalloc v2.6.1 and rocksdb v5.8.0, please install >>>>>>>> gperf-tools >>>>>>>> 2.5 (not 2.5.93) or >= 2.6.2, or switch to another allocator using >>>>>>>> 'cmake -DALLOCATOR=libc'. >>>>>>>> Call Stack (most recent call first): >>>>>>>> src/CMakeLists.txt:825 (build_rocksdb) >>>>>>>> ``` >>>>>>>> >>>>>>>>>> Reason for the question is that once the gperf packages is installed, >>>>>>>>>> as a bonus the code below declares the allocator to be tcmalloc. >>>>>>>>>> But gperf under FreeBSD does not have tcmalloc. >>>>>>>>> >>>>>>>>> please note gperf here is not GNU perf [1] but google-perftools. and >>>>>>>>> why do you claim the gperf under FreeBSD does not have tcmalloc? could >>>>>>>>> you elaborate this a little bit? >>>>>>>> >>>>>>>> My bad, I mixed gperf and gperf-tools. We are talking about gperf-tools. >>>>>>>> And yes, gperftools brings tcmalloc but not in the version mix that is >>>>>>>> asked for. gperf-tools was at 2.5 one time, but has upgraded to 2.6.1_1 >>>>>>>> on FreeBSD[1]. Getting other versions will require manual work to get it >>>>>>>> to work. >>>>>>>> >>>>>>>> But the question is: >>>>>>>> Do I really want my allocator to be tcmalloc when the native >>>>>>>> libc allocator is jemalloc? >>>>>>> >>>>>>> i think it's up to you. >>>>>> >>>>>> Oke, it is what the code now forces... >>>>>> And I've tried avoiding tcmalloc uptil now, and would like to do so for >>>>>> the time being.. >>>>>> >>>>>>>> So is the test in CMakeLists.txt:286-324 what we want on FreeBSD? >>>>>>> >>>>>>> see http://tracker.ceph.com/issues/21422. so again, it's up to you. >>>>>> >>>>>> The tracker suggests that it is an error in 2.5.93, but >>>>>> cmake/modules/BuildRocksDB.cmake suggests that it is only fixed in >>>>>> 2.6.2...?? >>>>> >>>>> 2.6.2 is not yet released. so if you want to stick with tcmalloc, you >>>>> need to build from the latest (or recent enough) source. actually, i >>>>> am not sure what you question is.. >>>> >>>> The question: Do you know if 2.6.2 is the first gperf-tools-version > >>>> 2.5.93 where this bug from the tracker is solved? >>> >>> https://github.com/ceph/ceph/pull/17788/commits/ff8d6a730bb774271b9eea6af338e2d3ce3c48a5 >> >> fascinating reading... >> Shows that managing version can be a bitch. >> >>>>>>>> The workaround this (for the time being) is to call cmake with >>>>>>>> -DALLOCATOR=libc as is suggested. >>>>>> >>>>>> I'll use this as workaround for the time being, since that is least >>>>>> invasive. And when that works out, try and make a PR for the allocator >>>>>> selector in CmakeLists.txt, and/or wait until gperf-tools 2.6.2 is >>>>>> available on FreeBSD. >>>>> >>>>> what is issue you want to address? >>>> >>>> The issue I want to address is that I'm not able to build the intree >>>> rocksdb ATM. Because tcmalloc is not => 2.6.2, and tcmalloc is required >>>> because gperf-tools is detected. >>> >>> the error message offered two solutions. i think, what you need is >>> just to choose one of them. >> >> Right, so I'm going with libc for the moment... >> >> Thanx for your patient explanations, > > i did send a mail with title of 'fix for "crash in rocksdb LRUCache > destructor with tcmalloc"' to this mailing list. if you have had read > it, it'd save both of us some time. I was/am about a zillion mails behind on Ceph-lists due to too much work at my paying dayjob. And then still I might have forgotten about it because it speaks about fc25/26 and tcmalloc. Where I only just today found out that gperf-tools does use tcmalloc. So I'm sorry for your time, but for me I feel a bit more educated. --WjW ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2017-10-10 15:42 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <b06780c7-13ce-1274-735f-806a1f4f8062@digiware.nl>
2017-10-10 11:14 ` Understanding some of the Cmake logics kefu chai
2017-10-10 12:06 ` Willem Jan Withagen
2017-10-10 13:31 ` kefu chai
2017-10-10 13:54 ` Willem Jan Withagen
2017-10-10 14:58 ` kefu chai
2017-10-10 15:13 ` Willem Jan Withagen
2017-10-10 15:19 ` kefu chai
2017-10-10 15:27 ` Willem Jan Withagen
2017-10-10 15:35 ` kefu chai
2017-10-10 15:41 ` Willem Jan Withagen
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.