All of lore.kernel.org
 help / color / mirror / Atom feed
From: Daniele Buono <dbuono@linux.vnet.ibm.com>
To: qemu-devel@nongnu.org
Cc: "Thomas Huth" <thuth@redhat.com>,
	"Daniel P . Berrangé" <berrange@redhat.com>,
	"Philippe Mathieu-Daudé" <philmd@redhat.com>,
	"Wainer dos Santos Moschetta" <wainersm@redhat.com>,
	"Paolo Bonzini" <pbonzini@redhat.com>,
	"Alex Bennée" <alex.bennee@linaro.org>,
	"Daniele Buono" <dbuono@linux.vnet.ibm.com>
Subject: [PATCH v3 1/2] gitlab-ci.yml: Allow custom # of parallel linkers
Date: Wed,  3 Mar 2021 22:09:47 -0500	[thread overview]
Message-ID: <20210304030948.9367-2-dbuono@linux.vnet.ibm.com> (raw)
In-Reply-To: <20210304030948.9367-1-dbuono@linux.vnet.ibm.com>

Define a new variable LD_JOBS, that can be used to select
the maximum number of linking jobs to be executed in parallel.
If the variable is not defined, maintain the default given by
make -j

Currently, make parallelism at build time is based on the number
of cpus available.

This doesn't work well with LTO at linking, because with LTO the
linker has to load in memory all the intermediate object files
for optimization.
The end result is that, if the gitlab runner happens to run two
linking processes at the same time, the job will fail with an
out-of-memory error,

This patch leverages the ability to maintain high parallelism at
compile time, but limit the number of linkers executed in parallel.

Signed-off-by: Daniele Buono <dbuono@linux.vnet.ibm.com>
---
 .gitlab-ci.yml | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 8b6d495288..814f51873f 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -27,6 +27,10 @@ include:
       else
         ../configure --enable-werror $CONFIGURE_ARGS ;
       fi || { cat config.log meson-logs/meson-log.txt && exit 1; }
+    - if test -n "$LD_JOBS";
+      then
+        meson configure . -Dbackend_max_links="$LD_JOBS" ;
+      fi || exit 1;
     - make -j"$JOBS"
     - if test -n "$MAKE_CHECK_ARGS";
       then
-- 
2.30.0



  reply	other threads:[~2021-03-04  3:12 UTC|newest]

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-03-04  3:09 [PATCH v3 0/2] gitlab-ci.yml: Add jobs to test CFI Daniele Buono
2021-03-04  3:09 ` Daniele Buono [this message]
2021-03-04 10:38   ` [PATCH v3 1/2] gitlab-ci.yml: Allow custom # of parallel linkers Daniel P. Berrangé
2021-03-04  3:09 ` [PATCH v3 2/2] gitlab-ci.yml: Add jobs to test CFI flags Daniele Buono
2021-03-04 10:39   ` Daniel P. Berrangé
2021-03-04 11:21     ` Thomas Huth
2021-03-04 11:58       ` Daniel P. Berrangé
2021-03-04 19:56 ` [PATCH v3 0/2] gitlab-ci.yml: Add jobs to test CFI Alex Bennée

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210304030948.9367-2-dbuono@linux.vnet.ibm.com \
    --to=dbuono@linux.vnet.ibm.com \
    --cc=alex.bennee@linaro.org \
    --cc=berrange@redhat.com \
    --cc=pbonzini@redhat.com \
    --cc=philmd@redhat.com \
    --cc=qemu-devel@nongnu.org \
    --cc=thuth@redhat.com \
    --cc=wainersm@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.