All of lore.kernel.org
 help / color / mirror / Atom feed
* [RFC PATCH] cmake: use Ninja backend instead of Make
@ 2018-01-16 13:25 Ross Burton
  2018-01-16 13:29 ` Alexander Kanavin
  2018-01-17 16:18 ` Otavio Salvador
  0 siblings, 2 replies; 8+ messages in thread
From: Ross Burton @ 2018-01-16 13:25 UTC (permalink / raw)
  To: openembedded-core

The Ninja backend is faster than Make for large projects (takes three minutes
off webkitgtk:do_compile for me), so use it by default.

Signed-off-by: Ross Burton <ross.burton@intel.com>
---
 meta/classes/cmake.bbclass | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/meta/classes/cmake.bbclass b/meta/classes/cmake.bbclass
index 10c4edcf1a5..ab29b1f1c23 100644
--- a/meta/classes/cmake.bbclass
+++ b/meta/classes/cmake.bbclass
@@ -1,7 +1,7 @@
 # Path to the CMake file to process.
 OECMAKE_SOURCEPATH ??= "${S}"
 
-DEPENDS_prepend = "cmake-native "
+DEPENDS_prepend = "cmake-native ninja-native "
 B = "${WORKDIR}/build"
 
 # We need to unset CCACHE otherwise cmake gets too confused
@@ -120,7 +120,7 @@ cmake_do_configure() {
 		oecmake_sitefile=
 	fi
 
-	cmake \
+	cmake -GNinja \
 	  $oecmake_sitefile \
 	  ${OECMAKE_SOURCEPATH} \
 	  -DCMAKE_INSTALL_PREFIX:PATH=${prefix} \
@@ -141,7 +141,7 @@ cmake_do_configure() {
 	  -Wno-dev
 }
 
-do_compile[progress] = "percent"
+do_compile[progress] = "outof:^\[(\d+)/(\d+)\]\s+"
 cmake_do_compile()  {
 	bbnote VERBOSE=1 cmake --build '${B}' --target ${OECMAKE_TARGET_COMPILE} -- ${EXTRA_OECMAKE_BUILD}
 	VERBOSE=1 cmake --build '${B}' --target ${OECMAKE_TARGET_COMPILE} -- ${EXTRA_OECMAKE_BUILD}
-- 
2.11.0



^ permalink raw reply related	[flat|nested] 8+ messages in thread

* Re: [RFC PATCH] cmake: use Ninja backend instead of Make
  2018-01-16 13:25 [RFC PATCH] cmake: use Ninja backend instead of Make Ross Burton
@ 2018-01-16 13:29 ` Alexander Kanavin
  2018-01-17 16:18 ` Otavio Salvador
  1 sibling, 0 replies; 8+ messages in thread
From: Alexander Kanavin @ 2018-01-16 13:29 UTC (permalink / raw)
  To: Ross Burton, openembedded-core

On 01/16/2018 03:25 PM, Ross Burton wrote:
> The Ninja backend is faster than Make for large projects (takes three minutes
> off webkitgtk:do_compile for me), so use it by default.

You do know why Ninja was invented in the first place, right? :)

http://neugierig.org/software/chromium/notes/2011/02/ninja.html

Alex


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [RFC PATCH] cmake: use Ninja backend instead of Make
  2018-01-16 13:25 [RFC PATCH] cmake: use Ninja backend instead of Make Ross Burton
  2018-01-16 13:29 ` Alexander Kanavin
@ 2018-01-17 16:18 ` Otavio Salvador
  2018-01-17 20:57   ` Randy MacLeod
  1 sibling, 1 reply; 8+ messages in thread
From: Otavio Salvador @ 2018-01-17 16:18 UTC (permalink / raw)
  To: Ross Burton; +Cc: Patches and discussions about the oe-core layer

On Tue, Jan 16, 2018 at 11:25 AM, Ross Burton <ross.burton@intel.com> wrote:
> The Ninja backend is faster than Make for large projects (takes three minutes
> off webkitgtk:do_compile for me), so use it by default.
>
> Signed-off-by: Ross Burton <ross.burton@intel.com>

I tried to test this but it did not apply in master. Could you rebase it?

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [RFC PATCH] cmake: use Ninja backend instead of Make
  2018-01-17 16:18 ` Otavio Salvador
@ 2018-01-17 20:57   ` Randy MacLeod
  2018-01-17 21:43     ` Burton, Ross
  0 siblings, 1 reply; 8+ messages in thread
From: Randy MacLeod @ 2018-01-17 20:57 UTC (permalink / raw)
  To: Otavio Salvador, Ross Burton
  Cc: Patches and discussions about the oe-core layer

On 2018-01-17 11:18 AM, Otavio Salvador wrote:
> On Tue, Jan 16, 2018 at 11:25 AM, Ross Burton <ross.burton@intel.com> wrote:
>> The Ninja backend is faster than Make for large projects (takes three minutes
>> off webkitgtk:do_compile for me), so use it by default.
>>
>> Signed-off-by: Ross Burton <ross.burton@intel.com>
> 
> I tried to test this but it did not apply in master. Could you rebase it?
> 

On my 64+64 core system that is shared, it reduced a webkit compile
from ~ 17 minutes to ~12 minutes for just the compile stage. YMMV.

$ cat webkit-before-compile.log
3.29user 0.50system 17:39.98elapsed 0%CPU
   (0avgtext+0avgdata 28532maxresident)k
0inputs+176outputs (0major+16335minor)pagefaults 0swaps


$ cat webkit-after-compile.log
2.70user 0.47system 12:09.24elapsed 0%CPU
    (0avgtext+0avgdata 28292maxresident)k
0inputs+160outputs (0major+14634minor)pagefaults 0swaps

I'm going to try to build world (oe-core + meta-oe + ...).


Octavio, it's a two line change! :)
This applied to master for me:

commit 3914c66f6e7123a64ab73e2b6707452bbfd10d02
Author: Ross Burton <ross.burton@intel.com>
Date:   Wed Jan 17 13:09:40 2018 -0500

     cmake: use Ninja backend instead of Make

     The Ninja backend is faster than Make for large projects
     (it takes three minutes off webkitgtk:do_compile for example)
     so use it by default.

     Acked-by: Randy MacLeod <Randy.MacLeod@windriver.com>
     Signed-off-by: Ross Burton <ross.burton@intel.com>

diff --git a/meta/classes/cmake.bbclass b/meta/classes/cmake.bbclass
index a4079dab0d..c46117acde 100644
--- a/meta/classes/cmake.bbclass
+++ b/meta/classes/cmake.bbclass
@@ -1,7 +1,7 @@
  # Path to the CMake file to process.
  OECMAKE_SOURCEPATH ??= "${S}"

-DEPENDS_prepend = "cmake-native "
+DEPENDS_prepend = "cmake-native ninja-native "
  B = "${WORKDIR}/build"

  # We need to unset CCACHE otherwise cmake gets too confused
@@ -117,7 +117,7 @@ cmake_do_configure() {
  		oecmake_sitefile=
  	fi

-	cmake \
+	cmake -GNinja \
  	  $oecmake_sitefile \
  	  ${OECMAKE_SOURCEPATH} \
  	  -DCMAKE_INSTALL_PREFIX:PATH=${prefix} \

-- 
# Randy MacLeod.  WR Linux
# Wind River an Intel Company


^ permalink raw reply related	[flat|nested] 8+ messages in thread

* Re: [RFC PATCH] cmake: use Ninja backend instead of Make
  2018-01-17 20:57   ` Randy MacLeod
@ 2018-01-17 21:43     ` Burton, Ross
  2018-01-18  0:13       ` Otavio Salvador
  2018-01-18  2:59       ` Randy MacLeod
  0 siblings, 2 replies; 8+ messages in thread
From: Burton, Ross @ 2018-01-17 21:43 UTC (permalink / raw)
  To: Randy MacLeod
  Cc: Otavio Salvador, Patches and discussions about the oe-core layer

[-- Attachment #1: Type: text/plain, Size: 1293 bytes --]

On 17 January 2018 at 20:57, Randy MacLeod <randy.macleod@windriver.com>
wrote:

> On 2018-01-17 11:18 AM, Otavio Salvador wrote:
>
>> On Tue, Jan 16, 2018 at 11:25 AM, Ross Burton <ross.burton@intel.com>
>> wrote:
>>
>>> The Ninja backend is faster than Make for large projects (takes three
>>> minutes
>>> off webkitgtk:do_compile for me), so use it by default.
>>>
>>> Signed-off-by: Ross Burton <ross.burton@intel.com>
>>>
>>
>> I tried to test this but it did not apply in master. Could you rebase it?
>>
>>
> On my 64+64 core system that is shared, it reduced a webkit compile
> from ~ 17 minutes to ~12 minutes for just the compile stage. YMMV.
>
> $ cat webkit-before-compile.log
> 3.29user 0.50system 17:39.98elapsed 0%CPU
>   (0avgtext+0avgdata 28532maxresident)k
> 0inputs+176outputs (0major+16335minor)pagefaults 0swaps
>
>
> $ cat webkit-after-compile.log
> 2.70user 0.47system 12:09.24elapsed 0%CPU
>    (0avgtext+0avgdata 28292maxresident)k
> 0inputs+160outputs (0major+14634minor)pagefaults 0swaps
>
> I'm going to try to build world (oe-core + meta-oe + ...).
>

Good win :)

There appears to be a small number of upstreams that break with ninja, so
I'll rework this patch so that the backend is an option but default to
Ninja.

Ross

[-- Attachment #2: Type: text/html, Size: 2072 bytes --]

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [RFC PATCH] cmake: use Ninja backend instead of Make
  2018-01-17 21:43     ` Burton, Ross
@ 2018-01-18  0:13       ` Otavio Salvador
  2018-01-18  2:59       ` Randy MacLeod
  1 sibling, 0 replies; 8+ messages in thread
From: Otavio Salvador @ 2018-01-18  0:13 UTC (permalink / raw)
  To: Burton, Ross; +Cc: Patches and discussions about the oe-core layer

On Wed, Jan 17, 2018 at 7:43 PM, Burton, Ross <ross.burton@intel.com> wrote:
> There appears to be a small number of upstreams that break with ninja, so
> I'll rework this patch so that the backend is an option but default to
> Ninja.

When doing so, please rebase on top of my patches and send it as
official patch (no RFC). Besides test, it is very welcome.

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [RFC PATCH] cmake: use Ninja backend instead of Make
  2018-01-17 21:43     ` Burton, Ross
  2018-01-18  0:13       ` Otavio Salvador
@ 2018-01-18  2:59       ` Randy MacLeod
  2018-01-18  3:04         ` Randy MacLeod
  1 sibling, 1 reply; 8+ messages in thread
From: Randy MacLeod @ 2018-01-18  2:59 UTC (permalink / raw)
  To: Burton, Ross
  Cc: Otavio Salvador, Patches and discussions about the oe-core layer

On 2018-01-17 04:43 PM, Burton, Ross wrote:
> There appears to be a small number of upstreams that break with ninja, 
> so I'll rework this patch so that the backend is an option but default 
> to Ninja.

The only failure that involved a cmake based package was piglit
for my qemux86-64 world build.

It didn't install:
   make: Entering directory 
'/.../tmp-glibc/work/core2-64-wrs-linux/piglit/1.0+gitrAUTOINC+c8f4fd9eeb-r0/build'
make: *** No rule to make target 'install/strip'.  Stop.

so that might not be related to this change. I haven't manually done
a world build on master recently.

As expected using ninja for the back-end of cmake just makes the
builds faster.

-- 
# Randy MacLeod.  WR Linux
# Wind River an Intel Company


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [RFC PATCH] cmake: use Ninja backend instead of Make
  2018-01-18  2:59       ` Randy MacLeod
@ 2018-01-18  3:04         ` Randy MacLeod
  0 siblings, 0 replies; 8+ messages in thread
From: Randy MacLeod @ 2018-01-18  3:04 UTC (permalink / raw)
  To: Burton, Ross
  Cc: Otavio Salvador, Patches and discussions about the oe-core layer

On 2018-01-17 09:59 PM, Randy MacLeod wrote:
> On 2018-01-17 04:43 PM, Burton, Ross wrote:
>> There appears to be a small number of upstreams that break with ninja, 
>> so I'll rework this patch so that the backend is an option but default 
>> to Ninja.
> 
> The only failure that involved a cmake based package was piglit
> for my qemux86-64 world build.
> 
> It didn't install:
>    make: Entering directory 
> '/.../tmp-glibc/work/core2-64-wrs-linux/piglit/1.0+gitrAUTOINC+c8f4fd9eeb-r0/build' 
> 
> make: *** No rule to make target 'install/strip'.  Stop.
> 
> so that might not be related to this change. I haven't manually done
> a world build on master recently.

Apparently already fixed by Ross in master-next:

commit 9fec5ef920bd63fe3cc2e623add0c7aead95ecae
Author: Ross Burton <ross.burton@intel.com>
Date:   Tue Jan 16 13:25:16 2018 +0000

     piglit: override install target name instead of do_install

../Randy

> 
> As expected using ninja for the back-end of cmake just makes the
> builds faster.
> 


-- 
# Randy MacLeod.  WR Linux
# Wind River an Intel Company


^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2018-01-18  3:05 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-01-16 13:25 [RFC PATCH] cmake: use Ninja backend instead of Make Ross Burton
2018-01-16 13:29 ` Alexander Kanavin
2018-01-17 16:18 ` Otavio Salvador
2018-01-17 20:57   ` Randy MacLeod
2018-01-17 21:43     ` Burton, Ross
2018-01-18  0:13       ` Otavio Salvador
2018-01-18  2:59       ` Randy MacLeod
2018-01-18  3:04         ` Randy MacLeod

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.