From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: ** X-Spam-Status: No, score=2.5 required=3.0 tests=DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 02859C3F2CE for ; Sun, 1 Mar 2020 05:46:24 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CA57921D56 for ; Sun, 1 Mar 2020 05:46:23 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="s4JEFEt0" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CA57921D56 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 6DD776E427; Sun, 1 Mar 2020 05:46:15 +0000 (UTC) Received: from mail-pl1-x62d.google.com (mail-pl1-x62d.google.com [IPv6:2607:f8b0:4864:20::62d]) by gabe.freedesktop.org (Postfix) with ESMTPS id 7C6986E420; Sun, 1 Mar 2020 05:46:13 +0000 (UTC) Received: by mail-pl1-x62d.google.com with SMTP id g6so2890502plt.2; Sat, 29 Feb 2020 21:46:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=BMFva8+eOa5vh3fnHy/a9L8r9X1WjtrkhhfpiOzrz5c=; b=s4JEFEt05s6mpcwTrB3V/VPGd65ZgRxP7oWGdoMf+o1IznVj/+DarhDC9YPj2ZhUrc A54Td41jsBwVNJ+vsOY/8s9H5D/Wys38MGEhrU5A3WNNzDSMGKKKxhbyyIYgA8OC7jjR Og7LalY/Wv7jFjQEi5iVR+9uysUtyYesUnrGSPoA9zdXNnR7Ci0efXRx8YpMfwQ74NjG R5VQiQU271N7l4dhga3PQPlTixjm62NiKM7h2Ye76aPJUGVIW8dSwYqlKD/FbpkEudgd 1Dezu+GyAw1dkYWpf81u9A4zgxXikonV2AvKpuDjOWzkOddbwNXb8Rf7bpaLJ08J5uFC N5tA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=BMFva8+eOa5vh3fnHy/a9L8r9X1WjtrkhhfpiOzrz5c=; b=bm26AUOz5AcYXzHT94Xo6RBEuNoDzt8SvT6zUxl7jtywtPudWVehHVM3SZKiH+Qsip NfCnag2NvD4sJz74vryWqqooul4q+qmK75AzXHXQD4UICG+TRAuI3g8uOhXsVE45Qm2e 6fJKSEi3RDlP/PIr5eJg5HkwwgBpxIcmZFM8LD6vKS2xNiaM5rAXFvEn8bEZ883B3ItP YWVIUWa0pqbTVuu3CH2HsT7XgrfmlHaDXQFzbW63s8KGRKeoX6ywKPwc7jVAzrTDJzR6 YtJ+2D508bL6DDKA4OjqeQn89v9B4vm0zLFtadZUWR6hiAi7KooIgDnt6Ls7v1h0mGjB DfoA== X-Gm-Message-State: APjAAAW6Hy5BGH0HxShZKht48lMkuiGGF4RZRZqF+QdU7NvHfq2BOx/8 WDXgOetqLQYvPFOlp8jpj2KMAInI3l9EhJY/BPs= X-Google-Smtp-Source: APXvYqxDzPK1Bfz4mrHH3OesFTKhwvaemtm2OM0JFb07CjQ328g4kuHTaUK+kPzJ6RiqsRsjRARYH7P8aUrBu3C3PMI= X-Received: by 2002:a17:902:7898:: with SMTP id q24mr11203695pll.164.1583041572643; Sat, 29 Feb 2020 21:46:12 -0800 (PST) MIME-Version: 1.0 References: <6d2ec570f957b4504fb70e0b1f0632712a99dc0c.camel@collabora.com> <59f4ea1f13a9a9d37f7801b93061b4ae7dd595e2.camel@gmail.com> <5551426acf99f73d3ce8234c14c176c1c7a1fe44.camel@ndufresne.ca> In-Reply-To: <5551426acf99f73d3ce8234c14c176c1c7a1fe44.camel@ndufresne.ca> From: =?UTF-8?B?TWFyZWsgT2zFocOhaw==?= Date: Sun, 1 Mar 2020 00:46:00 -0500 Message-ID: Subject: Re: [Mesa-dev] [Intel-gfx] gitlab.fd.o financial situation and impact on services To: Nicolas Dufresne X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Erik Faye-Lund , =?UTF-8?Q?Timur_Krist=C3=B3f?= , Daniel Vetter , intel-gfx , "X.Org development" , dri-devel , wayland , "X.Org Foundation Board" , Xorg Members List , amd-gfx list , Jason Ekstrand , Mesa Dev , Discussion of the development of and with GStreamer Content-Type: multipart/mixed; boundary="===============1068757829==" Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" --===============1068757829== Content-Type: multipart/alternative; boundary="0000000000009a29a5059fc49643" --0000000000009a29a5059fc49643 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable For Mesa, we could run CI only when Marge pushes, so that it's a strictly pre-merge CI. Marek On Sat., Feb. 29, 2020, 17:20 Nicolas Dufresne, wrote: > Le samedi 29 f=C3=A9vrier 2020 =C3=A0 15:54 -0600, Jason Ekstrand a =C3= =A9crit : > > On Sat, Feb 29, 2020 at 3:47 PM Timur Krist=C3=B3f > wrote: > > > On Sat, 2020-02-29 at 14:46 -0500, Nicolas Dufresne wrote: > > > > > 1. I think we should completely disable running the CI on MRs whi= ch > > > > > are > > > > > marked WIP. Speaking from personal experience, I usually make a l= ot > > > > > of > > > > > changes to my MRs before they are merged, so it is a waste of CI > > > > > resources. > > > > > > > > In the mean time, you can help by taking the habit to use: > > > > > > > > git push -o ci.skip > > > > > > Thanks for the advice, I wasn't aware such an option exists. Does thi= s > > > also work on the mesa gitlab or is this a GStreamer only thing? > > > > Mesa is already set up so that it only runs on MRs and branches named > > ci-* (or maybe it's ci/*; I can't remember). > > > > > How hard would it be to make this the default? > > > > I strongly suggest looking at how Mesa does it and doing that in > > GStreamer if you can. It seems to work pretty well in Mesa. > > You are right, they added CI_MERGE_REQUEST_SOURCE_BRANCH_NAME in 11.6 > (we started our CI a while ago). But there is even better now, ou can > do: > > only: > refs: > - merge_requests > > Thanks for the hint, I'll suggest that. I've lookup some of the backend > of mesa, I think it's really nice, though there is a lot of concept > that won't work in a multi-repo CI. Again, I need to refresh on what > was moved from the enterprise to the community version in this regard, > > > > > --Jason > > > > > > > > That's a much more difficult goal then it looks like. Let each > > > > projects > > > > manage their CI graph and content, as each case is unique. Running > > > > more > > > > tests, or building more code isn't the main issue as the CPU time i= s > > > > mostly sponsored. The data transfers between the cloud of gitlab an= d > > > > the runners (which are external), along to sending OS image to Lava > > > > labs is what is likely the most expensive. > > > > > > > > As it was already mention in the thread, what we are missing now, a= nd > > > > being worked on, is per group/project statistics that give us the > > > > hotspot so we can better target the optimization work. > > > > > > Yes, would be nice to know what the hotspot is, indeed. > > > > > > As far as I understand, the problem is not CI itself, but the bandwid= th > > > needed by the build artifacts, right? Would it be possible to not hos= t > > > the build artifacts on the gitlab, but rather only the place where th= e > > > build actually happened? Or at least, only transfer the build artifac= ts > > > on-demand? > > > > > > I'm not exactly familiar with how the system works, so sorry if this = is > > > a silly question. > > > > > > _______________________________________________ > > > mesa-dev mailing list > > > mesa-dev@lists.freedesktop.org > > > https://lists.freedesktop.org/mailman/listinfo/mesa-dev > > _______________________________________________ > mesa-dev mailing list > mesa-dev@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/mesa-dev > --0000000000009a29a5059fc49643 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
For Mesa, we could run CI only when Marge pushes, so that= it's a strictly pre-merge CI.

Marek

On Sat., Feb. 29, 2020, 17:20 Nicolas Dufresne, <nicolas@ndufresne.ca> wrote:
Le samedi 29 f=C3=A9vrier 2020 =C3=A0 15:= 54 -0600, Jason Ekstrand a =C3=A9crit :
> On Sat, Feb 29, 2020 at 3:47 PM Timur Krist=C3=B3f <timur.kris= tof@gmail.com> wrote:
> > On Sat, 2020-02-29 at 14:46 -0500, Nicolas Dufresne wrote:
> > > > 1. I think we should completely disable running the CI = on MRs which
> > > > are
> > > > marked WIP. Speaking from personal experience, I usuall= y make a lot
> > > > of
> > > > changes to my MRs before they are merged, so it is a wa= ste of CI
> > > > resources.
> > >
> > > In the mean time, you can help by taking the habit to use: > > >
> > >=C2=A0 =C2=A0git push -o ci.skip
> >
> > Thanks for the advice, I wasn't aware such an option exists. = Does this
> > also work on the mesa gitlab or is this a GStreamer only thing? >
> Mesa is already set up so that it only runs on MRs and branches named<= br> > ci-* (or maybe it's ci/*; I can't remember).
>
> > How hard would it be to make this the default?
>
> I strongly suggest looking at how Mesa does it and doing that in
> GStreamer if you can.=C2=A0 It seems to work pretty well in Mesa.

You are right, they added CI_MERGE_REQUEST_SOURCE_BRANCH_NAME in 11.6
(we started our CI a while ago). But there is even better now, ou can
do:

=C2=A0 only:
=C2=A0 =C2=A0 refs:
=C2=A0 =C2=A0 =C2=A0 - merge_requests

Thanks for the hint, I'll suggest that. I've lookup some of the bac= kend
of mesa, I think it's really nice, though there is a lot of concept
that won't work in a multi-repo CI. Again, I need to refresh on what was moved from the enterprise to the community version in this regard,

>
> --Jason
>
>
> > > That's a much more difficult goal then it looks like. Le= t each
> > > projects
> > > manage their CI graph and content, as each case is unique. R= unning
> > > more
> > > tests, or building more code isn't the main issue as the= CPU time is
> > > mostly sponsored. The data transfers between the cloud of gi= tlab and
> > > the runners (which are external), along to sending OS image = to Lava
> > > labs is what is likely the most expensive.
> > >
> > > As it was already mention in the thread, what we are missing= now, and
> > > being worked on, is per group/project statistics that give u= s the
> > > hotspot so we can better target the optimization work.
> >
> > Yes, would be nice to know what the hotspot is, indeed.
> >
> > As far as I understand, the problem is not CI itself, but the ban= dwidth
> > needed by the build artifacts, right? Would it be possible to not= host
> > the build artifacts on the gitlab, but rather only the place wher= e the
> > build actually happened? Or at least, only transfer the build art= ifacts
> > on-demand?
> >
> > I'm not exactly familiar with how the system works, so sorry = if this is
> > a silly question.
> >
> > _______________________________________________
> > mesa-dev mailing list
> > mesa-dev@lists.freedesktop.org
> > https://lists.freedeskto= p.org/mailman/listinfo/mesa-dev

_______________________________________________
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mail= man/listinfo/mesa-dev
--0000000000009a29a5059fc49643-- --===============1068757829== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel --===============1068757829==--