qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
From: Stefan Hajnoczi <stefanha@redhat.com>
To: Thomas Huth <thuth@redhat.com>
Cc: "Daniel P . Berrangé" <berrange@redhat.com>,
	qemu-devel@nongnu.org,
	"Wainer dos Santos Moschetta" <wainersm@redhat.com>,
	"Philippe Mathieu-Daudé" <f4bug@amsat.org>,
	"Willian Rampazzo" <willianr@redhat.com>,
	"Alex Bennée" <alex.bennee@linaro.org>
Subject: Re: [RFC PATCH 0/5] Use ccache in the gitlab-CI
Date: Wed, 14 Apr 2021 11:31:52 +0100	[thread overview]
Message-ID: <YHbEmCF1Lem2g3Up@stefanha-x1.localdomain> (raw)
In-Reply-To: <20210414081907.871437-1-thuth@redhat.com>

[-- Attachment #1: Type: text/plain, Size: 2221 bytes --]

On Wed, Apr 14, 2021 at 10:19:02AM +0200, Thomas Huth wrote:
> Here are some patches to enable ccache in the gitlab-CI. Using the
> compiler cache can speed up some of the jobs in the CI by avoiding
> to recompile the same sources again and again. With this patch series,
> some jobs indeed get quite a bit faster. For example "build-system-fedora"
> goes down from 29 minutes (https://gitlab.com/thuth/qemu/-/jobs/1168149051)
> to 12 minutes (https://gitlab.com/thuth/qemu/-/jobs/1175479324) in the
> best case.
> However, and that's why I've marked this series still as RFC, there
> are some jobs which refuse to improve much, for example the *-cfi-* jobs
> are still running for a long time, see e.g.:
> 
>  https://gitlab.com/thuth/qemu/-/jobs/1168149079 (50 minutes on master)
>  https://gitlab.com/thuth/qemu/-/jobs/1175479342 (43 minutes with ccache)
> 
> Thus the overall run time did not improve very much yet.
> 
> Additionally, the jobs are sometimes running even slower, e.g. if the
> cache has not been populated yet or if there are a lot of cache misses,
> and I've seen some jobs hitting the 60 min time limit because of this.
> And I've even seen two or three jobs failing intermittendly with a
> crashing test, e.g.:
> 
>  https://gitlab.com/thuth/qemu/-/jobs/1175479336
> 
> Not sure whether that's related to some bad caching of ccache, or
> whether it's something that also occurs on the master branch recently?
> Has anybody else experienced such a problem?

The GitLab CI cache implementation does not appear to be designed for
ccache because the files are saved in a cache.zip file and then
extracted again each time a job runs.

That is very inefficient for something like a 1 GiB ccache directory.

More details:
https://docs.gitlab.com/ee/ci/caching/#availability-of-the-cache

The GitLab CI cache feature in its current form just doesn't look
suitable for ccache.

I wonder if we can implement ccache differently on dedicated runners.
The ccache directory would really be shared (mounted within job
containers) without the additional cache.zip step to avoid wasting CPU
cycles compressing/decompressing and RAM with duplicated page cache.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

      parent reply	other threads:[~2021-04-14 10:34 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-04-14  8:19 Thomas Huth
2021-04-14  8:19 ` [RFC PATCH 1/5] tests/docker/dockerfiles: Add ccache to containers where it was missing Thomas Huth
2021-04-14  8:19 ` [RFC PATCH 2/5] tests/docker/dockerfiles/ubuntu2004: Add missing symlink for cc Thomas Huth
2021-04-14 10:16   ` Philippe Mathieu-Daudé
2021-04-14  8:19 ` [RFC PATCH 3/5] gitlab-ci.d/crossbuilds: Limit the amount of targets in the cross-win64 build Thomas Huth
2021-04-14  8:19 ` [RFC PATCH 4/5] gitlab-ci.yml: Compile with ccache enabled Thomas Huth
2021-04-14  8:19 ` [RFC PATCH 5/5] gitlab-ci.d/crossbuilds.yml: Enable compilation with ccache Thomas Huth
2021-04-14 10:29   ` Philippe Mathieu-Daudé
2021-04-14 10:35     ` Philippe Mathieu-Daudé
2021-04-14 10:31 ` Stefan Hajnoczi [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=YHbEmCF1Lem2g3Up@stefanha-x1.localdomain \
    --to=stefanha@redhat.com \
    --cc=alex.bennee@linaro.org \
    --cc=berrange@redhat.com \
    --cc=f4bug@amsat.org \
    --cc=qemu-devel@nongnu.org \
    --cc=thuth@redhat.com \
    --cc=wainersm@redhat.com \
    --cc=willianr@redhat.com \
    --subject='Re: [RFC PATCH 0/5] Use ccache in the gitlab-CI' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).