All of lore.kernel.org
 help / color / mirror / Atom feed
From: Thomas Huth <thuth@redhat.com>
To: "Philippe Mathieu-Daudé" <f4bug@amsat.org>,
	qemu-devel@nongnu.org, "Alex Bennée" <alex.bennee@linaro.org>
Cc: "Daniel P . Berrange" <berrange@redhat.com>,
	Kyle Evans <kevans@freebsd.org>,
	Wainer dos Santos Moschetta <wainersm@redhat.com>,
	Willian Rampazzo <willianr@redhat.com>,
	Stefan Hajnoczi <stefanha@redhat.com>,
	Warner Losh <imp@bsdimp.com>
Subject: Re: [PATCH v3 4/6] gitlab-ci: Add ccache in $PATH and display statistics
Date: Fri, 21 May 2021 12:48:21 +0200	[thread overview]
Message-ID: <396e75d2-fe22-3054-fbee-d6de02339b2a@redhat.com> (raw)
In-Reply-To: <20c001a0-60ed-df08-b67d-884022e74d04@amsat.org>

On 20/05/2021 13.27, Philippe Mathieu-Daudé wrote:
> +Stefan/Daniel
> 
> On 5/20/21 10:02 AM, Thomas Huth wrote:
>> On 19/05/2021 20.45, Philippe Mathieu-Daudé wrote:
>>> If a runner has ccache installed, use it and display statistics
>>> at the end of the build.
>>>
>>> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
>>> ---
>>>    .gitlab-ci.d/buildtest-template.yml | 5 +++++
>>>    1 file changed, 5 insertions(+)
>>>
>>> diff --git a/.gitlab-ci.d/buildtest-template.yml
>>> b/.gitlab-ci.d/buildtest-template.yml
>>> index f284d7a0eec..a625c697d3b 100644
>>> --- a/.gitlab-ci.d/buildtest-template.yml
>>> +++ b/.gitlab-ci.d/buildtest-template.yml
>>> @@ -6,13 +6,18 @@
>>>          then
>>>            JOBS=$(sysctl -n hw.ncpu)
>>>            MAKE=gmake
>>> +        PATH=/usr/local/libexec/ccache:$PATH
>>>            ;
>>>          else
>>>            JOBS=$(expr $(nproc) + 1)
>>>            MAKE=make
>>> +        PATH=/usr/lib/ccache:/usr/lib64/ccache:$PATH
>>
>> That does not make sense for the shared runners yet. We first need
>> something to enable the caching there - see my series "Use ccache in the
>> gitlab-CI" from April (which is currently stalled unfortunately).
> 
> TL;DR: I don't think we should restrict our templates to shared runners.

I'm certainly not voting for restricting ourselves to only use shared 
runners here - but my concern is that this actually *slows* down the shared 
runners even more! (sorry, I should have elaborated on that in my previous 
mail already)

When I was experimenting with ccache in the shared runners, I saw that the 
jobs are running even slower with ccache enabled as long as the cache is not 
populated yet. You only get a speedup afterwards. So if you add this now 
without also adding the possibility to store the cache persistently, the 
shared runners will try to populate the cache each time just to throw away 
the results afterwards again. Thus all the shared runners only get slower 
without any real benefit here.

Thus we either need to get ccache working properly for the shared runners 
first, or you have to think of a different way of enabling ccache for the 
non-shared runners, so that it does not affect the shared runners negatively.

  Thomas



  reply	other threads:[~2021-05-21 10:49 UTC|newest]

Thread overview: 34+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-05-19 18:45 [PATCH v3 0/6] gitlab-ci: Allow using FreeBSD runners Philippe Mathieu-Daudé
2021-05-19 18:45 ` [PATCH v3 1/6] gitlab-ci: Extract &environment_variables template Philippe Mathieu-Daudé
2021-05-19 19:12   ` Willian Rampazzo
2021-05-20 18:00   ` Wainer dos Santos Moschetta
2021-05-19 18:45 ` [PATCH v3 2/6] gitlab-ci: Adapt JOBS variable for FreeBSD runners Philippe Mathieu-Daudé
2021-05-19 19:11   ` Willian Rampazzo
2021-05-20  5:50   ` Thomas Huth
2021-05-20 18:18   ` Wainer dos Santos Moschetta
2021-05-20 20:02     ` Warner Losh
2021-05-19 18:45 ` [PATCH v3 3/6] gitlab-ci: Run GNU make via the $MAKE variable Philippe Mathieu-Daudé
2021-05-19 19:10   ` Willian Rampazzo
2021-05-20  7:56   ` Thomas Huth
2021-05-20 18:26   ` Wainer dos Santos Moschetta
2021-05-19 18:45 ` [PATCH v3 4/6] gitlab-ci: Add ccache in $PATH and display statistics Philippe Mathieu-Daudé
2021-05-19 19:14   ` Willian Rampazzo
2021-05-20  8:02   ` Thomas Huth
2021-05-20 11:27     ` Philippe Mathieu-Daudé
2021-05-21 10:48       ` Thomas Huth [this message]
2021-05-21 10:50         ` Daniel P. Berrangé
2021-05-21 11:02           ` Thomas Huth
2021-05-21 11:53             ` Daniel P. Berrangé
2021-05-21 12:27               ` Philippe Mathieu-Daudé
2021-05-21 12:52                 ` Daniel P. Berrangé
2021-05-21 14:21                   ` Philippe Mathieu-Daudé
2021-05-25  5:49                     ` Philippe Mathieu-Daudé
2021-05-24 13:51                   ` Stefan Hajnoczi
2021-05-25  5:50                     ` Philippe Mathieu-Daudé
2021-05-21 12:36             ` Willian Rampazzo
2021-05-21 13:02               ` Thomas Huth
2021-05-19 18:45 ` [PATCH v3 5/6] gitlab-ci: Simplify before/after script for Avocado based jobs Philippe Mathieu-Daudé
2021-05-19 19:42   ` Willian Rampazzo
2021-05-20  8:04   ` Thomas Huth
2021-05-19 18:45 ` [PATCH v3 6/6] gitlab-ci: Add FreeBSD jobs Philippe Mathieu-Daudé
2021-05-19 19:43   ` Willian Rampazzo

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=396e75d2-fe22-3054-fbee-d6de02339b2a@redhat.com \
    --to=thuth@redhat.com \
    --cc=alex.bennee@linaro.org \
    --cc=berrange@redhat.com \
    --cc=f4bug@amsat.org \
    --cc=imp@bsdimp.com \
    --cc=kevans@freebsd.org \
    --cc=qemu-devel@nongnu.org \
    --cc=stefanha@redhat.com \
    --cc=wainersm@redhat.com \
    --cc=willianr@redhat.com \
    --subject='Re: [PATCH v3 4/6] gitlab-ci: Add ccache in $PATH and display statistics' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.