From: Thomas Huth <email@example.com> To: "Daniel P. Berrangé" <firstname.lastname@example.org> Cc: "Kyle Evans" <email@example.com>, "Philippe Mathieu-Daudé" <firstname.lastname@example.org>, "Wainer dos Santos Moschetta" <email@example.com>, firstname.lastname@example.org, "Willian Rampazzo" <email@example.com>, "Stefan Hajnoczi" <firstname.lastname@example.org>, "Alex Bennée" <email@example.com>, "Warner Losh" <firstname.lastname@example.org> Subject: Re: [PATCH v3 4/6] gitlab-ci: Add ccache in $PATH and display statistics Date: Fri, 21 May 2021 13:02:51 +0200 [thread overview] Message-ID: <email@example.com> (raw) In-Reply-To: <YKeQkqviHbxqcC6Y@redhat.com> On 21/05/2021 12.50, Daniel P. Berrangé wrote: > On Fri, May 21, 2021 at 12:48:21PM +0200, Thomas Huth wrote: >> On 20/05/2021 13.27, Philippe Mathieu-Daudé wrote: >>> +Stefan/Daniel >>> >>> On 5/20/21 10:02 AM, Thomas Huth wrote: >>>> On 19/05/2021 20.45, Philippe Mathieu-Daudé wrote: >>>>> If a runner has ccache installed, use it and display statistics >>>>> at the end of the build. >>>>> >>>>> Signed-off-by: Philippe Mathieu-Daudé <firstname.lastname@example.org> >>>>> --- >>>>> .gitlab-ci.d/buildtest-template.yml | 5 +++++ >>>>> 1 file changed, 5 insertions(+) >>>>> >>>>> diff --git a/.gitlab-ci.d/buildtest-template.yml >>>>> b/.gitlab-ci.d/buildtest-template.yml >>>>> index f284d7a0eec..a625c697d3b 100644 >>>>> --- a/.gitlab-ci.d/buildtest-template.yml >>>>> +++ b/.gitlab-ci.d/buildtest-template.yml >>>>> @@ -6,13 +6,18 @@ >>>>> then >>>>> JOBS=$(sysctl -n hw.ncpu) >>>>> MAKE=gmake >>>>> + PATH=/usr/local/libexec/ccache:$PATH >>>>> ; >>>>> else >>>>> JOBS=$(expr $(nproc) + 1) >>>>> MAKE=make >>>>> + PATH=/usr/lib/ccache:/usr/lib64/ccache:$PATH >>>> >>>> That does not make sense for the shared runners yet. We first need >>>> something to enable the caching there - see my series "Use ccache in the >>>> gitlab-CI" from April (which is currently stalled unfortunately). >>> >>> TL;DR: I don't think we should restrict our templates to shared runners. >> >> I'm certainly not voting for restricting ourselves to only use shared >> runners here - but my concern is that this actually *slows* down the shared >> runners even more! (sorry, I should have elaborated on that in my previous >> mail already) >> >> When I was experimenting with ccache in the shared runners, I saw that the >> jobs are running even slower with ccache enabled as long as the cache is not >> populated yet. You only get a speedup afterwards. So if you add this now >> without also adding the possibility to store the cache persistently, the >> shared runners will try to populate the cache each time just to throw away >> the results afterwards again. Thus all the shared runners only get slower >> without any real benefit here. >> >> Thus we either need to get ccache working properly for the shared runners >> first, or you have to think of a different way of enabling ccache for the >> non-shared runners, so that it does not affect the shared runners >> negatively. > > Is there anything functional holding up your previous full cccache support > series from last month ? Or is it just lack of reviews ? It's basically the problems mentioned in the cover letter and Stefan's comment here: https://lists.gnu.org/archive/html/qemu-devel/2021-04/msg02219.html The series needs more love and more testing, if it's feasible with the gitlab-CI architecture at all to use ccache in a good way ... if we don't get a real speedup by the patches, it's certainly not worth the effort to integrate this... If someone wants to have a try to improve the patch series, your help is certainly welcome - since at least I personally lack the time and motivation to improve this any further. Thomas
next prev parent reply other threads:[~2021-05-21 11:05 UTC|newest] Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-05-19 18:45 [PATCH v3 0/6] gitlab-ci: Allow using FreeBSD runners Philippe Mathieu-Daudé 2021-05-19 18:45 ` [PATCH v3 1/6] gitlab-ci: Extract &environment_variables template Philippe Mathieu-Daudé 2021-05-19 19:12 ` Willian Rampazzo 2021-05-20 18:00 ` Wainer dos Santos Moschetta 2021-05-19 18:45 ` [PATCH v3 2/6] gitlab-ci: Adapt JOBS variable for FreeBSD runners Philippe Mathieu-Daudé 2021-05-19 19:11 ` Willian Rampazzo 2021-05-20 5:50 ` Thomas Huth 2021-05-20 18:18 ` Wainer dos Santos Moschetta 2021-05-20 20:02 ` Warner Losh 2021-05-19 18:45 ` [PATCH v3 3/6] gitlab-ci: Run GNU make via the $MAKE variable Philippe Mathieu-Daudé 2021-05-19 19:10 ` Willian Rampazzo 2021-05-20 7:56 ` Thomas Huth 2021-05-20 18:26 ` Wainer dos Santos Moschetta 2021-05-19 18:45 ` [PATCH v3 4/6] gitlab-ci: Add ccache in $PATH and display statistics Philippe Mathieu-Daudé 2021-05-19 19:14 ` Willian Rampazzo 2021-05-20 8:02 ` Thomas Huth 2021-05-20 11:27 ` Philippe Mathieu-Daudé 2021-05-21 10:48 ` Thomas Huth 2021-05-21 10:50 ` Daniel P. Berrangé 2021-05-21 11:02 ` Thomas Huth [this message] 2021-05-21 11:53 ` Daniel P. Berrangé 2021-05-21 12:27 ` Philippe Mathieu-Daudé 2021-05-21 12:52 ` Daniel P. Berrangé 2021-05-21 14:21 ` Philippe Mathieu-Daudé 2021-05-25 5:49 ` Philippe Mathieu-Daudé 2021-05-24 13:51 ` Stefan Hajnoczi 2021-05-25 5:50 ` Philippe Mathieu-Daudé 2021-05-21 12:36 ` Willian Rampazzo 2021-05-21 13:02 ` Thomas Huth 2021-05-19 18:45 ` [PATCH v3 5/6] gitlab-ci: Simplify before/after script for Avocado based jobs Philippe Mathieu-Daudé 2021-05-19 19:42 ` Willian Rampazzo 2021-05-20 8:04 ` Thomas Huth 2021-05-19 18:45 ` [PATCH v3 6/6] gitlab-ci: Add FreeBSD jobs Philippe Mathieu-Daudé 2021-05-19 19:43 ` Willian Rampazzo
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --subject='Re: [PATCH v3 4/6] gitlab-ci: Add ccache in $PATH and display statistics' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.