All of lore.kernel.org
 help / color / mirror / Atom feed
* Performance measurement: Building openembedded-core with and without overclocking on C-i7 2600K
@ 2011-10-17 20:44 Ulf Samuelsson
  2011-10-17 22:13 ` Khem Raj
  2011-10-18  6:28 ` Anders Darander
  0 siblings, 2 replies; 13+ messages in thread
From: Ulf Samuelsson @ 2011-10-17 20:44 UTC (permalink / raw)
  To: openembedded-devel

If anyone is interested...

Same machine in two different configurations.

.........................................................#1...........................#2
P8Z68V Pro
Core-i7 2600K @                             3.4 GHz..................4.4 GHz
8 GB @ ...........................................1333 
MHz...............1640 MHz
1TB WD Black 7200 RPM
Ubuntu 11-10

Almost all source code already in source directory

Building meta-toolchain is 13% faster with overclocking.


#1
build:    meta-toolchain
START                                   10:46:23        55:52
FINISH                                  11:42:15
build:    core-image-minimal
START                                   11:42:15        17:34
FINISH                                  11:59:49
build:    core-image-sato
START                                   13:40:47        01:19
FINISH                                  13:42:06
Build fails, since one package could not be downloaded.
build:    meta-toolchain-sdk
START                                   13:42:06        00:38
FINISH                                  13:42:44
build:    adt-installer
START                                   13:42:44        00:03
FINISH                                  13:42:51
build:    meta-ide-support
START                                   13:42:51        01:13
FINISH                                  13:44:04



#2
build:    meta-toolchain
START                                   20:18:51        49:19    (vs 55:52)
FINISH                                  21:08:10
build:    core-image-minimal
START                                   21:08:10        16:38    (vs 17:34)
FINISH                                  21:24:48
build:    core-image-sato
START                                   21:24:48        15:38    (vs 
01:19 - I think I have a problem in the first build)
FINISH                                  21:40:26
build:    meta-toolchain-sdk
START                                   21:40:26        00:59    (vs 00:38)
FINISH                                  21:41:25
build:    adt-installer
START                                   21:41:25        00:03    (vs 00:03)
FINISH                                  21:41:28
build:    meta-ide-support
START                                   21:41:28        01:03    (vs 01:13)
FINISH                                  21:42:31



-- 
Best Regards
Ulf Samuelsson




^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: Performance measurement: Building openembedded-core with and without overclocking on C-i7 2600K
  2011-10-17 20:44 Performance measurement: Building openembedded-core with and without overclocking on C-i7 2600K Ulf Samuelsson
@ 2011-10-17 22:13 ` Khem Raj
  2011-10-18  6:03   ` Ulf Samuelsson
  2011-10-18  6:28 ` Anders Darander
  1 sibling, 1 reply; 13+ messages in thread
From: Khem Raj @ 2011-10-17 22:13 UTC (permalink / raw)
  To: openembedded-devel

On 10/17/2011 1:44 PM, Ulf Samuelsson wrote:
> If anyone is interested...
>

thanks for numbers. Is there statistics on 64bit host Vs. 32bit build 
host available too ?

> Same machine in two different configurations.
>
> .........................................................#1...........................#2
>
> P8Z68V Pro
> Core-i7 2600K @ 3.4 GHz..................4.4 GHz
> 8 GB @ ...........................................1333
> MHz...............1640 MHz
> 1TB WD Black 7200 RPM
> Ubuntu 11-10
>
> Almost all source code already in source directory
>
> Building meta-toolchain is 13% faster with overclocking.
>
>
> #1
> build: meta-toolchain
> START 10:46:23 55:52
> FINISH 11:42:15
> build: core-image-minimal
> START 11:42:15 17:34
> FINISH 11:59:49
> build: core-image-sato
> START 13:40:47 01:19
> FINISH 13:42:06
> Build fails, since one package could not be downloaded.
> build: meta-toolchain-sdk
> START 13:42:06 00:38
> FINISH 13:42:44
> build: adt-installer
> START 13:42:44 00:03
> FINISH 13:42:51
> build: meta-ide-support
> START 13:42:51 01:13
> FINISH 13:44:04
>
>
>
> #2
> build: meta-toolchain
> START 20:18:51 49:19 (vs 55:52)
> FINISH 21:08:10
> build: core-image-minimal
> START 21:08:10 16:38 (vs 17:34)
> FINISH 21:24:48
> build: core-image-sato
> START 21:24:48 15:38 (vs 01:19 - I think I have a problem in the first
> build)
> FINISH 21:40:26
> build: meta-toolchain-sdk
> START 21:40:26 00:59 (vs 00:38)
> FINISH 21:41:25
> build: adt-installer
> START 21:41:25 00:03 (vs 00:03)
> FINISH 21:41:28
> build: meta-ide-support
> START 21:41:28 01:03 (vs 01:13)
> FINISH 21:42:31
>
>
>




^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: Performance measurement: Building openembedded-core with and without overclocking on C-i7 2600K
  2011-10-17 22:13 ` Khem Raj
@ 2011-10-18  6:03   ` Ulf Samuelsson
  2011-10-18 14:44     ` Khem Raj
                       ` (2 more replies)
  0 siblings, 3 replies; 13+ messages in thread
From: Ulf Samuelsson @ 2011-10-18  6:03 UTC (permalink / raw)
  To: openembedded-devel

2011-10-18 00:13, Khem Raj skrev:
> On 10/17/2011 1:44 PM, Ulf Samuelsson wrote:
>> If anyone is interested...
>>
>
> thanks for numbers. Is there statistics on 64bit host Vs. 32bit build 
> host available too ?

We discussed that a few days ago.

My comparision between 2 similar machines indicate that
32 bit 11.04 with Psyco-Dist is 20% faster than 64 bit with 11.04

The machine running 64 bit 11.04 was tested with 32 bit 11.04
using an external disk (connected through e-SATA) and results
were very close to the numbers for the other 32 bit machine

I didn't save those numbers but I think that is good enough
to draw the conclusion that adding JIT for 64 bit (PyPy?) will reduce build
time significantly.

It would be interesting to know the effect of faster disks.
I run SATA-II disks at the moment, and upgrading to SATA-III
might improve things.

Since you have multiple tasks, it might be worthwhile having several 
disks to
the disk heads spending most of their time moving around.
Have a lot of computer parts including a chassi mounting 6+ disks.
* /
* /home
* <OE_DIR>/source
* <OE_DIR>/build/tmp


Also what happens if it it built on a fast SSD, or even a RAMdisk?

Building the stuff below adds 25 GB + a 2 GB source directory.
With the Z68 chipset, you can get up to 32 GB.
It would be interesting to know how well the linux caching system
performs vs having <OE_DIR>/build/tmp etc. on a ramdisk.

The 8 GB DIMMs are not really available without ECC, but
Corsair is about to release a 32 GB kit which will set you
back $1000 so I wont do that test for the forseeable future.
Maybe someone else which will have such a machine anyway
would be interested enough to do a test.

One issue with this is that even 32 GB may not be enough.
With the current build using 27 GB, there is not much headroom to grow.

If using a ramdisk, proves to be significantly faster,
an alternative would be smaller ramdisk with some background task moving
packages to a real disk after the build of that package
has completed.



>> Same machine in two different configurations.
>>
>> .........................................................#1...........................#2 
>>
>>
>> P8Z68V Pro
>> Core-i7 2600K @ 3.4 GHz..................4.4 GHz
>> 8 GB @ ...........................................1333
>> MHz...............1640 MHz
>> 1TB WD Black 7200 RPM
>> Ubuntu 11-10
>>
>> Almost all source code already in source directory
>>
>> Building meta-toolchain is 13% faster with overclocking.
>>
>>
>> #1
>> build: meta-toolchain
>> START 10:46:23 55:52
>> FINISH 11:42:15
>> build: core-image-minimal
>> START 11:42:15 17:34
>> FINISH 11:59:49
>> build: core-image-sato
>> START 13:40:47 01:19
>> FINISH 13:42:06
>> Build fails, since one package could not be downloaded.
>> build: meta-toolchain-sdk
>> START 13:42:06 00:38
>> FINISH 13:42:44
>> build: adt-installer
>> START 13:42:44 00:03
>> FINISH 13:42:51
>> build: meta-ide-support
>> START 13:42:51 01:13
>> FINISH 13:44:04
>>
>>
>>
>> #2
>> build: meta-toolchain
>> START 20:18:51 49:19 (vs 55:52)
>> FINISH 21:08:10
>> build: core-image-minimal
>> START 21:08:10 16:38 (vs 17:34)
>> FINISH 21:24:48
>> build: core-image-sato
>> START 21:24:48 15:38 (vs 01:19 - I think I have a problem in the first
>> build)
>> FINISH 21:40:26
>> build: meta-toolchain-sdk
>> START 21:40:26 00:59 (vs 00:38)
>> FINISH 21:41:25
>> build: adt-installer
>> START 21:41:25 00:03 (vs 00:03)
>> FINISH 21:41:28
>> build: meta-ide-support
>> START 21:41:28 01:03 (vs 01:13)
>> FINISH 21:42:31
>>
>>
>>
>
>
> _______________________________________________
> Openembedded-devel mailing list
> Openembedded-devel@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel


-- 
Best Regards
Ulf Samuelsson




^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: Performance measurement: Building openembedded-core with and without overclocking on C-i7 2600K
  2011-10-17 20:44 Performance measurement: Building openembedded-core with and without overclocking on C-i7 2600K Ulf Samuelsson
  2011-10-17 22:13 ` Khem Raj
@ 2011-10-18  6:28 ` Anders Darander
  2011-10-18 12:05   ` Ulf Samuelsson
  1 sibling, 1 reply; 13+ messages in thread
From: Anders Darander @ 2011-10-18  6:28 UTC (permalink / raw)
  To: openembedded-devel

* Ulf Samuelsson <ulf_samuelsson@telia.com> [111017 22:45]:
> If anyone is interested...

Build stats are always interesting... ;)

> Same machine in two different configurations.

> .........................................................#1...........................#2
> P8Z68V Pro
> Core-i7 2600K @                             3.4 GHz..................4.4 GHz
> 8 GB @ ...........................................1333 
> MHz...............1640 MHz
> 1TB WD Black 7200 RPM
> Ubuntu 11-10

> Almost all source code already in source directory

This would severely restrict the usefulness of the comparison. I'd guess
that this means that in one case the source will be downloaded, while
it'll already be downloaded in the second test. Could you redo the
statistics with all sources downloaded? 

> Building meta-toolchain is 13% faster with overclocking.

Was the overclocking test run after the non-overclocking?

Cheers,
Anders

-- 
Anders Darander
ChargeStorm AB / eStorm AB



^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: Performance measurement: Building openembedded-core with and without overclocking on C-i7 2600K
  2011-10-18  6:28 ` Anders Darander
@ 2011-10-18 12:05   ` Ulf Samuelsson
  2011-10-18 12:19     ` Anders Darander
  0 siblings, 1 reply; 13+ messages in thread
From: Ulf Samuelsson @ 2011-10-18 12:05 UTC (permalink / raw)
  To: openembedded-devel

2011-10-18 08:28, Anders Darander skrev:
> * Ulf Samuelsson<ulf_samuelsson@telia.com>  [111017 22:45]:
>> If anyone is interested...
> Build stats are always interesting... ;)
>
>> Same machine in two different configurations.
>> .........................................................#1...........................#2
>> P8Z68V Pro
>> Core-i7 2600K @                             3.4 GHz..................4.4 GHz
>> 8 GB @ ...........................................1333
>> MHz...............1640 MHz
>> 1TB WD Black 7200 RPM
>> Ubuntu 11-10
>> Almost all source code already in source directory
> This would severely restrict the usefulness of the comparison. I'd guess
> that this means that in one case the source will be downloaded, while
> it'll already be downloaded in the second test. Could you redo the
> statistics with all sources downloaded?
I had already built it once before running the first test then updated
and I think one additional package was downloaded over a 100 MBps line
so it affects a little, but not too much.

I think the conclusion:
     Increasing the CPU speed by 30% and the memory frequency by 20%
     gives 10-15% faster build time is OK.



>> Building meta-toolchain is 13% faster with overclocking.
> Was the overclocking test run after the non-overclocking?
>
> Cheers,
> Anders
>


-- 
Best Regards
Ulf Samuelsson




^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: Performance measurement: Building openembedded-core with and without overclocking on C-i7 2600K
  2011-10-18 12:05   ` Ulf Samuelsson
@ 2011-10-18 12:19     ` Anders Darander
  2011-10-18 13:40       ` Ulf Samuelsson
  0 siblings, 1 reply; 13+ messages in thread
From: Anders Darander @ 2011-10-18 12:19 UTC (permalink / raw)
  To: openembedded-devel

* Ulf Samuelsson <ulf_samuelsson@telia.com> [111018 14:06]:
> 2011-10-18 08:28, Anders Darander skrev:
> > * Ulf Samuelsson<ulf_samuelsson@telia.com>  [111017 22:45]:
> >> If anyone is interested...
> > Build stats are always interesting... ;)

> >> Same machine in two different configurations.
> >> .........................................................#1...........................#2
> >> P8Z68V Pro
> >> Core-i7 2600K @                             3.4 GHz..................4.4 GHz
> >> 8 GB @ ...........................................1333
> >> MHz...............1640 MHz
> >> 1TB WD Black 7200 RPM
> >> Ubuntu 11-10
> >> Almost all source code already in source directory
> > This would severely restrict the usefulness of the comparison. I'd guess
> > that this means that in one case the source will be downloaded, while
> > it'll already be downloaded in the second test. Could you redo the
> > statistics with all sources downloaded?
> I had already built it once before running the first test then updated
> and I think one additional package was downloaded over a 100 MBps line
> so it affects a little, but not too much.

Ok, just curious. Not least as you had one image fail in the first
configuration, which you attributed to a failed download. That remark
made me wonder...

Cheers,
Anders

> I think the conclusion:
>      Increasing the CPU speed by 30% and the memory frequency by 20%
>      gives 10-15% faster build time is OK.

> >> Building meta-toolchain is 13% faster with overclocking.
> > Was the overclocking test run after the non-overclocking?


-- 
Anders Darander
ChargeStorm AB / eStorm AB



^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: Performance measurement: Building openembedded-core with and without overclocking on C-i7 2600K
  2011-10-18 12:19     ` Anders Darander
@ 2011-10-18 13:40       ` Ulf Samuelsson
  0 siblings, 0 replies; 13+ messages in thread
From: Ulf Samuelsson @ 2011-10-18 13:40 UTC (permalink / raw)
  To: openembedded-devel

2011-10-18 14:19, Anders Darander skrev:
> * Ulf Samuelsson<ulf_samuelsson@telia.com>  [111018 14:06]:
>> 2011-10-18 08:28, Anders Darander skrev:
>>> * Ulf Samuelsson<ulf_samuelsson@telia.com>   [111017 22:45]:
>>>> If anyone is interested...
>>> Build stats are always interesting... ;)
>>>> Same machine in two different configurations.
>>>> .........................................................#1...........................#2
>>>> P8Z68V Pro
>>>> Core-i7 2600K @                             3.4 GHz..................4.4 GHz
>>>> 8 GB @ ...........................................1333
>>>> MHz...............1640 MHz
>>>> 1TB WD Black 7200 RPM
>>>> Ubuntu 11-10
>>>> Almost all source code already in source directory
>>> This would severely restrict the usefulness of the comparison. I'd guess
>>> that this means that in one case the source will be downloaded, while
>>> it'll already be downloaded in the second test. Could you redo the
>>> statistics with all sources downloaded?
>> I had already built it once before running the first test then updated
>> and I think one additional package was downloaded over a 100 MBps line
>> so it affects a little, but not too much.
> Ok, just curious. Not least as you had one image fail in the first
> configuration, which you attributed to a failed download. That remark
> made me wonder...

OK, To be 100% correct:
I built meta-toolchain first so all packages for that build should be there.
This is much longer than anything else, so this is a good comparision.

Then the build #1 was started and measruements was taken.
This failed for core-image-sato due to one missing package, which
After build #1 was completed, I downloaded the missing package
and started build #2.

>
> Cheers,
> Anders
>
>> I think the conclusion:
>>       Increasing the CPU speed by 30% and the memory frequency by 20%
>>       gives 10-15% faster build time is OK.
>>>> Building meta-toolchain is 13% faster with overclocking.
>>> Was the overclocking test run after the non-overclocking?
>


-- 
Best Regards
Ulf Samuelsson




^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: Performance measurement: Building openembedded-core with and without overclocking on C-i7 2600K
  2011-10-18  6:03   ` Ulf Samuelsson
@ 2011-10-18 14:44     ` Khem Raj
  2011-10-18 14:47     ` Phil Blundell
  2011-10-23 17:50     ` Roman Khimov
  2 siblings, 0 replies; 13+ messages in thread
From: Khem Raj @ 2011-10-18 14:44 UTC (permalink / raw)
  To: openembedded-devel

On 10/17/2011 11:03 PM, Ulf Samuelsson wrote:
> My comparision between 2 similar machines indicate that
> 32 bit 11.04 with Psyco-Dist is 20% faster than 64 bit with 11.04
>

thats why I wanted to know if they were exactly same machine. Better to 
reinstall may be. I dont rule out that 32bit cant be faster but we need 
to measure it

> The machine running 64 bit 11.04 was tested with 32 bit 11.04
> using an external disk (connected through e-SATA) and results
> were very close to the numbers for the other 32 bit machine
>


> I didn't save those numbers but I think that is good enough
> to draw the conclusion that adding JIT for 64 bit (PyPy?) will reduce build
> time significantly.

Thats what I am interested in. If pypy would make it faster than
we could recommend it.




^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: Performance measurement: Building openembedded-core with and without overclocking on C-i7 2600K
  2011-10-18  6:03   ` Ulf Samuelsson
  2011-10-18 14:44     ` Khem Raj
@ 2011-10-18 14:47     ` Phil Blundell
  2011-10-18 22:34       ` Ulf Samuelsson
  2011-10-23 17:50     ` Roman Khimov
  2 siblings, 1 reply; 13+ messages in thread
From: Phil Blundell @ 2011-10-18 14:47 UTC (permalink / raw)
  To: openembedded-devel

On Tue, 2011-10-18 at 08:03 +0200, Ulf Samuelsson wrote:
> My comparision between 2 similar machines indicate that
> 32 bit 11.04 with Psyco-Dist is 20% faster than 64 bit with 11.04
> 
> The machine running 64 bit 11.04 was tested with 32 bit 11.04
> using an external disk (connected through e-SATA) and results
> were very close to the numbers for the other 32 bit machine
> 
> I didn't save those numbers but I think that is good enough
> to draw the conclusion that adding JIT for 64 bit (PyPy?) will reduce build
> time significantly.

You seem to have changed two things at once there (or perhaps just made
a bit too much of a leap in your conclusion).  Did you test 32-bit
without the JIT?

p.





^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: Performance measurement: Building openembedded-core with and without overclocking on C-i7 2600K
  2011-10-18 14:47     ` Phil Blundell
@ 2011-10-18 22:34       ` Ulf Samuelsson
  0 siblings, 0 replies; 13+ messages in thread
From: Ulf Samuelsson @ 2011-10-18 22:34 UTC (permalink / raw)
  To: openembedded-devel

2011-10-18 16:47, Phil Blundell skrev:
> On Tue, 2011-10-18 at 08:03 +0200, Ulf Samuelsson wrote:
>> My comparision between 2 similar machines indicate that
>> 32 bit 11.04 with Psyco-Dist is 20% faster than 64 bit with 11.04
>>
>> The machine running 64 bit 11.04 was tested with 32 bit 11.04
>> using an external disk (connected through e-SATA) and results
>> were very close to the numbers for the other 32 bit machine
>>
>> I didn't save those numbers but I think that is good enough
>> to draw the conclusion that adding JIT for 64 bit (PyPy?) will reduce build
>> time significantly.
> You seem to have changed two things at once there (or perhaps just made
> a bit too much of a leap in your conclusion).  Did you test 32-bit
> without the JIT?

The word indicate is not soo strong as you indicate ;-)
Just saying that if it looks like a duck, walks like a duck
it is probably a duck.

It takes too long to reinstall for that to be an alternative for me.
More data from other people could be an alternative.


>
> p.
>
>
>
> _______________________________________________
> Openembedded-devel mailing list
> Openembedded-devel@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel


-- 
Best Regards
Ulf Samuelsson




^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: Performance measurement: Building openembedded-core with and without overclocking on C-i7 2600K
  2011-10-18  6:03   ` Ulf Samuelsson
  2011-10-18 14:44     ` Khem Raj
  2011-10-18 14:47     ` Phil Blundell
@ 2011-10-23 17:50     ` Roman Khimov
  2011-10-23 21:50       ` Ulf Samuelsson
  2011-10-29 14:23       ` Raffaele Recalcati
  2 siblings, 2 replies; 13+ messages in thread
From: Roman Khimov @ 2011-10-23 17:50 UTC (permalink / raw)
  To: openembedded-devel

[-- Attachment #1: Type: Text/Plain, Size: 2277 bytes --]

В сообщении от 18 октября 2011 10:03:25 автор Ulf Samuelsson написал:
> Also what happens if it it built on a fast SSD, or even a RAMdisk?

Our main build server has two Xeons X5670 with 96 GB of RAM. It's running 24×7 
cyclicly doing clean isolated builds of all our OE projects (currently five). 
"Isolated" means that the the actual build is done inside a container (LXC) 
with no network available at all, that is a requirement for all our builds, to 
be able to take one tarball of sources, one tarball of OE tree and build the 
project somewhere in a nuclear bunker.

The system is configured in such way that container is located on tmpfs (with 
size of 50 GB). The most complex build takes about hour and a half on this 
system occupying about 45 GB of space in a ramdisk.

So today I tried to get some RAM vs. Linux cache statistics and switched this 
mount point over to newly created 60 GB LVM partition with ext4 on RAID0 array 
consisting of two SAS 15K drives.

The system made builds for three projects is this configuration and I see no 
difference at all, usual 1-2 minutes deviation. Granted, the system has quite 
powerful disks (RAID array gives about 380 MB/sec on hdparm) and things might 
be a little different on plain SATA drives, but frankly I'd expected to see 
the difference anyway since there are lots of small files involved in a build.

Maybe I should try to further degrade the disk system by creating some 
encrypted volume inside LVM, but still from what I see Linux caching and 
buffering works good enough, just give it as much RAM as you can.

But then also what you'll get from RAM or disk or even CPU upgrade depends on 
what type of build you have. Upgrading developers build servers from pair of 
4-core Opterons (don't remember exact model) with 8 GB of RAM to pair of Xeons 
E5620 with 24 GB of RAM with comparable disks gave about 20-30% of build time 
reduction for one project and 50% for another. But that builds are not 
isolated and use icecc cluster with all build servers available to the 
cluster, maybe that helps also in our situation.

-- 
 http://roman.khimov.ru
mailto: roman@khimov.ru
gpg --keyserver hkp://subkeys.pgp.net --recv-keys 0xE5E055C3

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: Performance measurement: Building openembedded-core with and without overclocking on C-i7 2600K
  2011-10-23 17:50     ` Roman Khimov
@ 2011-10-23 21:50       ` Ulf Samuelsson
  2011-10-29 14:23       ` Raffaele Recalcati
  1 sibling, 0 replies; 13+ messages in thread
From: Ulf Samuelsson @ 2011-10-23 21:50 UTC (permalink / raw)
  To: openembedded-devel

2011-10-23 19:50, Roman Khimov skrev:
> ? ????????? ?? 18 ??????? 2011 10:03:25 ????? Ulf Samuelsson ???????:
>> Also what happens if it it built on a fast SSD, or even a RAMdisk?
> Our main build server has two Xeons X5670 with 96 GB of RAM. It's running 24×7
> cyclicly doing clean isolated builds of all our OE projects (currently five).
> "Isolated" means that the the actual build is done inside a container (LXC)
> with no network available at all, that is a requirement for all our builds, to
> be able to take one tarball of sources, one tarball of OE tree and build the
> project somewhere in a nuclear bunker.
>
> The system is configured in such way that container is located on tmpfs (with
> size of 50 GB). The most complex build takes about hour and a half on this
> system occupying about 45 GB of space in a ramdisk.
>
> So today I tried to get some RAM vs. Linux cache statistics and switched this
> mount point over to newly created 60 GB LVM partition with ext4 on RAID0 array
> consisting of two SAS 15K drives.
>
> The system made builds for three projects is this configuration and I see no
> difference at all, usual 1-2 minutes deviation. Granted, the system has quite
> powerful disks (RAID array gives about 380 MB/sec on hdparm) and things might
> be a little different on plain SATA drives, but frankly I'd expected to see
> the difference anyway since there are lots of small files involved in a build.
>
> Maybe I should try to further degrade the disk system by creating some
> encrypted volume inside LVM, but still from what I see Linux caching and
> buffering works good enough, just give it as much RAM as you can.
>
> But then also what you'll get from RAM or disk or even CPU upgrade depends on
> what type of build you have. Upgrading developers build servers from pair of
> 4-core Opterons (don't remember exact model) with 8 GB of RAM to pair of Xeons
> E5620 with 24 GB of RAM with comparable disks gave about 20-30% of build time
> reduction for one project and 50% for another. But that builds are not
> isolated and use icecc cluster with all build servers available to the
> cluster, maybe that helps also in our situation.
>

Thanks,

I tried installing 32 bit Ubuntu 11.10 on a HDD and got results,
which are very close to the results of 64 bit ubuntu 11.10, but when I tried
hdparm the 32 bit system had 50 MB/s and the 64 bit system
had 100-110 MB/s so they are not comparable.

Also tried putting tmp/sysroots on a 3 GB tmpfs, and that only
made the build a couple of minutes faster on a 1 hour build.



>
>
> _______________________________________________
> Openembedded-devel mailing list
> Openembedded-devel@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel


-- 
Best Regards
Ulf Samuelsson



^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: Performance measurement: Building openembedded-core with and without overclocking on C-i7 2600K
  2011-10-23 17:50     ` Roman Khimov
  2011-10-23 21:50       ` Ulf Samuelsson
@ 2011-10-29 14:23       ` Raffaele Recalcati
  1 sibling, 0 replies; 13+ messages in thread
From: Raffaele Recalcati @ 2011-10-29 14:23 UTC (permalink / raw)
  To: openembedded-devel

Hi,

> Our main build server has two Xeons X5670 with 96 GB of RAM. It's running 24×7
> cyclicly doing clean isolated builds of all our OE projects (currently five).
> "Isolated" means that the the actual build is done inside a container (LXC)
> with no network available at all, that is a requirement for all our builds, to
> be able to take one tarball of sources, one tarball of OE tree and build the
> project somewhere in a nuclear bunker.

So, LXC sounds a better way than qemu for performances, isn't it?
Portability issues (next years operating systems) moving around the OE setup?

> The system is configured in such way that container is located on tmpfs (with
> size of 50 GB). The most complex build takes about hour and a half on this
> system occupying about 45 GB of space in a ramdisk.

I'm now building on xeon 5530 8 core (HP Z800) with 24GB DDR3.
20GB for tmpfs where I put only the tmp directory.
Recipes dir is on ssd 480MB/S writing speed.

> So today I tried to get some RAM vs. Linux cache statistics and switched this
> mount point over to newly created 60 GB LVM partition with ext4 on RAID0 array
> consisting of two SAS 15K drives.
>
> The system made builds for three projects is this configuration and I see no
> difference at all, usual 1-2 minutes deviation. Granted, the system has quite
> powerful disks (RAID array gives about 380 MB/sec on hdparm) and things might
> be a little different on plain SATA drives, but frankly I'd expected to see
> the difference anyway since there are lots of small files involved in a build.

You say you have no big differences between all tmpfs and SAS one?
I have instead SAS 15k RAID1, but I don't remember now their speed.
Moving from a complete SAS 15K setup to tmp dir in RAM I've instead
reduced from 4h to 1.5h for my image.

> Maybe I should try to further degrade the disk system by creating some
> encrypted volume inside LVM, but still from what I see Linux caching and
> buffering works good enough, just give it as much RAM as you can.
>
> But then also what you'll get from RAM or disk or even CPU upgrade depends on
> what type of build you have. Upgrading developers build servers from pair of
> 4-core Opterons (don't remember exact model) with 8 GB of RAM to pair of Xeons
> E5620 with 24 GB of RAM with comparable disks gave about 20-30% of build time
> reduction for one project and 50% for another. But that builds are not
> isolated and use icecc cluster with all build servers available to the
> cluster, maybe that helps also in our situation.

In my opinion the best setup is the one that I can move from a box to
another one without problem,
to be safe in case a box stops working.
If it stops I go to another one, I rsync download from the backup
server and restart.

Redundancy of cpu power should a different approach, but you "nuclear
bunker" contraints doesn't include this possibility.
I was in the past thinking to get a 15k€ building system with 2
computational nodes and a fast storage, with a standard debian on it.
A "small" blade center.
A developer ssh into it, he can do meld through nfs mount, start it's
own tagged qemu and compile his image.
If one core crash, the other one works anyway.

But now we are instead moving from 8 cores to 16 cores (IBM I don't
remember the model, something about 7000k€ off full price w/o
discount) with 64GB ram in order to be able to compile at least the
compilation at the same time.

Anyway it is difficult to convince developers to use central machine,
instead of using their own awful pc.
It's an hard job, but it's needed to speed up the stupid compilation job.

To complete the redundancy idea I'll have soon the 16cores with arch
linux and qemu through virtual manager.
If I have 16cores pb I'll get the used qemu image to the 8cores and I go on.

Bye,
Raffaele

-- 
www.opensurf.it



^ permalink raw reply	[flat|nested] 13+ messages in thread

end of thread, other threads:[~2011-10-29 14:29 UTC | newest]

Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-10-17 20:44 Performance measurement: Building openembedded-core with and without overclocking on C-i7 2600K Ulf Samuelsson
2011-10-17 22:13 ` Khem Raj
2011-10-18  6:03   ` Ulf Samuelsson
2011-10-18 14:44     ` Khem Raj
2011-10-18 14:47     ` Phil Blundell
2011-10-18 22:34       ` Ulf Samuelsson
2011-10-23 17:50     ` Roman Khimov
2011-10-23 21:50       ` Ulf Samuelsson
2011-10-29 14:23       ` Raffaele Recalcati
2011-10-18  6:28 ` Anders Darander
2011-10-18 12:05   ` Ulf Samuelsson
2011-10-18 12:19     ` Anders Darander
2011-10-18 13:40       ` Ulf Samuelsson

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.