* [Buildroot] 2019.02 performance
@ 2019-03-11 21:59 Trent Piepho
2019-03-12 8:05 ` Thomas Petazzoni
0 siblings, 1 reply; 4+ messages in thread
From: Trent Piepho @ 2019-03-11 21:59 UTC (permalink / raw)
To: buildroot
With 2018.02, I had to disable the package size and related hooks as
they were too slow: over half the build time. As I'm sure some
remember, there was a thread about it and performance was
improved. Here's the results with 2019.02. I'm using a real config
for a product we have and testing on a reasonably modern native Linux
workstation with a decent NVMe SSD and a four core processor. Using
ccache, which had already built the software.
Total build time: real 22:53, CPU 39:04.
Breakdown by build step (host/stage/target/images mean install same),
across all packages:
configure 666.223
build 381.480
host 122.191
extract 57.489
target 57.039
stage 28.356
patch 9.711
postimage 9.008
finalize 6.396
images 0.063
download 0.022
The single threaded configure and cmake scripts remain the killer,
through total of all install steps is about 2/3 of the build step time.
Breakdown for the global instrumentation hooks, across all packages and
build steps:
check_host_rpath 64.345
step_pkg_size 32.992
check_bin_arch 26.286
step_check_build_dir 8.365
step_time 7.895
pkg_size is much more reasonable now. In total, these are about 10.2%
of the total build time.
Next are benchmarks from a cloud VM, which is considerably slower.
Total build time: Real 53:58, CPU 55:22
configure 1021.933
build 936.449
host 206.171
extract 106.642
target 98.028
stage 53.747
finalize 40.840
download 25.091
postimage 22.876
patch 19.153
images 0.119
The GIH breakdown:
check_host_rpath 117.981
step_pkg_size 49.881
check_bin_arch 46.035
step_time 12.573
step_check_build_dir 9.307
About 7.2% of total build time.
^ permalink raw reply [flat|nested] 4+ messages in thread
* [Buildroot] 2019.02 performance
2019-03-11 21:59 [Buildroot] 2019.02 performance Trent Piepho
@ 2019-03-12 8:05 ` Thomas Petazzoni
2019-03-12 9:57 ` Arnout Vandecappelle
2019-03-12 20:30 ` Trent Piepho
0 siblings, 2 replies; 4+ messages in thread
From: Thomas Petazzoni @ 2019-03-12 8:05 UTC (permalink / raw)
To: buildroot
Hello,
First of all, thanks a *lot* for sharing this kind of analysis. This is
very useful.
On Mon, 11 Mar 2019 21:59:03 +0000
Trent Piepho <tpiepho@impinj.com> wrote:
> With 2018.02, I had to disable the package size and related hooks as
> they were too slow: over half the build time. As I'm sure some
> remember, there was a thread about it and performance was
> improved. Here's the results with 2019.02. I'm using a real config
> for a product we have and testing on a reasonably modern native Linux
> workstation with a decent NVMe SSD and a four core processor. Using
> ccache, which had already built the software.
A hot ccache will drastically decrease the time of the "build" step of
each package, and make the "configure" and "install" steps look
comparatively very long. A build without ccache support or with a cold
cache will show a much larger "build" time I believe.
> Breakdown by build step (host/stage/target/images mean install same),
> across all packages:
>
> configure 666.223
> build 381.480
> host 122.191
> extract 57.489
> target 57.039
> stage 28.356
> patch 9.711
> postimage 9.008
> finalize 6.396
> images 0.063
> download 0.022
>
>
> The single threaded configure and cmake scripts remain the killer,
> through total of all install steps is about 2/3 of the build step time.
>
> Breakdown for the global instrumentation hooks, across all packages and
> build steps:
>
> check_host_rpath 64.345
> step_pkg_size 32.992
> check_bin_arch 26.286
> step_check_build_dir 8.365
> step_time 7.895
>
> pkg_size is much more reasonable now. In total, these are about 10.2%
> of the total build time.
10% just for the instrumentation is still a lot. Our of 22 minutes of
your build time, it's still ~2 minutes. The clear dominating factor
here is check_host_rpath. If I'm reading the check-host-rpath
correctly, we are checking all files in $(HOST_DIR) at the end of the
build of every package. Couldn't this be optimized by using the list of
files installed by a package, like we do for check-bin-arch ?
I think that back when check-host-rpath we introduced, we did not had
the list of files installed by host packages. But now that we do,
perhaps we should use it ?
For the other ones, I don't immediately see any easy optimization
route, except perhaps writing some dedicated C programs instead of
using the slower shell scripts forking like crazy.
Best regards,
Thomas Petazzoni
--
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
^ permalink raw reply [flat|nested] 4+ messages in thread
* [Buildroot] 2019.02 performance
2019-03-12 8:05 ` Thomas Petazzoni
@ 2019-03-12 9:57 ` Arnout Vandecappelle
2019-03-12 20:30 ` Trent Piepho
1 sibling, 0 replies; 4+ messages in thread
From: Arnout Vandecappelle @ 2019-03-12 9:57 UTC (permalink / raw)
To: buildroot
On 12/03/2019 09:05, Thomas Petazzoni wrote:
> For the other ones, I don't immediately see any easy optimization
> route, except perhaps writing some dedicated C programs instead of
> using the slower shell scripts forking like crazy.
Assuming check-bin-arch becomes the dominant one when we use the pkg-file-list
for check_host_rpath, I've taking a look at how crazy this script is forking.
The script itself is executed once per package, which doesn't sound excessive.
It then forks a single sed to get the files for the package, and for each
non-ignored file it does a readelf, a sed and a head. This doesn't sound
excessive either...
So the only thing I can think of, is to create a small C program that reads the
first 4 bytes of a file and checks if it is .ELF, to avoid the readelf+sed+head.
Doesn't sound like it would make that much of a difference...
We could also use the .files-list*.txt which contains only the files belonging
to the package itself, which saves the outer sed. But that would only make a
difference if there's a huge number of files in your output.
Regards,
Arnout
^ permalink raw reply [flat|nested] 4+ messages in thread
* [Buildroot] 2019.02 performance
2019-03-12 8:05 ` Thomas Petazzoni
2019-03-12 9:57 ` Arnout Vandecappelle
@ 2019-03-12 20:30 ` Trent Piepho
1 sibling, 0 replies; 4+ messages in thread
From: Trent Piepho @ 2019-03-12 20:30 UTC (permalink / raw)
To: buildroot
On Tue, 2019-03-12 at 09:05 +0100, Thomas Petazzoni wrote:
>
> A hot ccache will drastically decrease the time of the "build" step of
> each package, and make the "configure" and "install" steps look
> comparatively very long. A build without ccache support or with a cold
> cache will show a much larger "build" time I believe.
Yes, it will be larger. I don't have a benchmark handy with that info.
Perhaps I'll do one overnight. But what I see here, is that nearly
every build we do, both manually and as part of CI, has a very high
ccache hit rate. I was keeping track of the stats from CI for a while
on that.
> >
> > pkg_size is much more reasonable now. In total, these are about 10.2%
> > of the total build time.
>
> 10% just for the instrumentation is still a lot. Our of 22 minutes of
> your build time, it's still ~2 minutes. The clear dominating factor
This is just the build image time (from scratch + ccache). There is
also a documentation build, and steps like legal info, package build
time graphs, and dependency graphs are quite long. The dependency
graphs are clearly the biggest low hanging fruit once this step is
considered.
Though check host rpath could be sped up! One thing that might skew
these results is our host packages to target packages ratio might be
higher than typical. If I took out the host based python software and
host based image crypto signing stuff there'd be a lot fewer host ELF
files.
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2019-03-12 20:30 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-03-11 21:59 [Buildroot] 2019.02 performance Trent Piepho
2019-03-12 8:05 ` Thomas Petazzoni
2019-03-12 9:57 ` Arnout Vandecappelle
2019-03-12 20:30 ` Trent Piepho
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.