Re: World builds HW Was: [PATCH 04/12] gnome-icon-theme: remove the recipe

[Martin Jansa <martin.jansa@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 4227 bytes --]

On Fri, Sep 18, 2015 at 11:38:30AM -0600, Gary Thomas wrote:
> On 2015-09-18 11:30, Martin Jansa wrote:
> > On Fri, Sep 18, 2015 at 12:24:00PM -0400, Randy MacLeod wrote:
> >> On 2015-09-07 11:14 AM, Martin Jansa wrote:
> >>> Full jenkins build test takes over 3 days (depending on sstate reuse,
> >>> one MACHINE can take more than 24 hours), so if I add them to
> >>> master-next after it's merged to oe-core it will take 3-6 days to
> >>> build-test them (6 days if they are sent just after current queue is
> >>> sent for build).
> >>
> >> Hi Martin,
> >
> > Hi Randy,
> >
> >> Replying privately but you can add the list back in your
> >> reply if you like.
> >
> > OK, adding both MLs
> >
> >> I never build all of meta-oe but a day seems like a long time.
> >> On a 3-4 year old 24 core system, with 64 GB RAM and
> >> 2 large disks, I build oe-core + a whitelist of meta-oe + misc layers in
> >> 5 hours using 18.3 of the 24 cores on average.
> >> What are the specs of your build machine?
> >
> > It's one of VMs running on box donated by HP to webos-ports project. It
> > has access to 8 E5-2630L 0 @ 2.00GHz cores (from IIRC 24) and slow IO
> > which we try to compensate a bit by using tmpfs for TMPDIR, VM has 80G
> > RAM, so I'm using 72G for tmpfs.
> >
> > Tom: please fill-in the gaps, I don't know any details about the real HW
> > it's running on.
> >
> > The same machine is also running some webos-ports builds sometimes.
> >
> > BUT be aware that my world build is much bigger than what you're
> > probably building and changes in oe-core almost always ensure very low
> > sstate reuse.
> >
> > e.g. last build with only small oe-core upgrade since previous build:
> > NOTE: Tasks Summary: Attempted 33283 tasks of which 5252 didn't need to be rerun and 33 failed.
> > NOTE: Build completion summary:
> > NOTE:   do_populate_sysroot: 21% sstate reuse (485 setscene, 1750 scratch)
> > NOTE:   do_package_qa: 12% sstate reuse (262 setscene, 1760 scratch)
> > NOTE:   do_package: 4% sstate reuse (76 setscene, 1750 scratch)
> > NOTE:   do_packagedata: 13% sstate reuse (281 setscene, 1750 scratch)
> > NOTE:   do_package_write_ipk: 12% sstate reuse (262 setscene, 1756 scratch)
> > NOTE:   do_populate_lic: 22% sstate reuse (521 setscene, 1806 scratch)
> 
> (slightly off topic): How did you get this report?

inherit buildstats-summary

> > real	1485m40.070s
> > user	4166m49.125s
> > sys	2825m50.068s
> >
> > BTW: you can see all this information in the world logs I'm sending to MLs
> > (but not all the builds make it to ML report).
> >
> >> A decent build box these days has 32+ cores, 64+ GB RAM  and
> >> and SSD or RAID disk filesystem. The costs of such a system
> >> are significant but worth it given the number of people's time
> >> involved in meta-oe.
> >
> > That's true, but I have to do with what I have available :).
> >
> >> Also, it would be nice if OE had a few such machines each
> >> working on builds for a MACHINE to reduce your turn-around
> >> to more like a day or so. :)
> >
> > Yes, it would be nice, I'm trying to get Tom King to set another VM like
> > this in another location (another HP donated server), hopefully he will
> > finish it soon.
> >
> > It's all hosted together with OE infrastructure, bandwidth is limited, but
> > all I need for these builds is to fetch sources and upload the logs, there
> > are no package feeds or sstate from these builds usable outside, but now it
> > rsync the downloaded sources to sources.openembedded.org
> >
> > If you know about some idling build servers, I would like to give them
> > something to do :).
> >
> > Regards,
> >
> >
> >
> 
> -- 
> ------------------------------------------------------------
> Gary Thomas                 |  Consulting for the
> MLB Associates              |    Embedded world
> ------------------------------------------------------------
> -- 
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 188 bytes --]