All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Gmane Admin" <gley-yocto@m.gmane-mx.org>
To: yocto@lists.yoctoproject.org
Cc: yocto-gJxINTE1zeQw8ufyyF7U+SST3g8Odh+X-XMD5yJDbdMReXY1tMh2IBg@public.gmane.org,
	yocto-gJxINTE1zeQw8ufyyF7U+SST3g8Odh+X@public.gmane.org,
	Ferry Toth <fntoth-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Subject: Re: bitbake controlling memory use
Date: Sun, 11 Apr 2021 18:08:27 +0200	[thread overview]
Message-ID: <14207c27-7f5e-9a13-d8e8-28dbb0e7ea7b@gmail.com> (raw)
In-Reply-To: <CANNYZj_K-zpP-iTQMh5YA58dhBV88wLP-PouXS5KrHjybfsy-g@mail.gmail.com>

Op 11-04-2021 om 17:55 schreef Alexander Kanavin:
> On Sun, 11 Apr 2021 at 17:49, Gmane Admin <gley-yocto@m.gmane-mx.org 
> <mailto:gley-yocto@m.gmane-mx.org>> wrote:
> 
>     Yes, and make project doesn't care, because make is called with -j
>     16 so
>     that is what it does.
> 
>     So here's my pitch: bitbake can stop processes spawned by make, because
>     it knows that it started make on 4 recipies, each with -j 16. The
>     individual makes don't know about each other.
> 
> 
> And neither they should. They can simply abstain from spawning new 
> compilers if used RAM is, say, at 90% total. Then bitbake does not have 
> to get involved in babysitting those makes.
> 
> Alex
Bitbake does a lot of babysitting anyway :-) And is pretty good at it too.

To me, fixing make et al. is more work and less effective then adding a 
feature to bitbake. The only way to know how much memory the compiler 
will use for each spawned compiler is to let it run. And then it's too late.

This memory issue is all over our eco system and nobody cares (kernel, 
make etc.) The only thing moving is systemd's oom killer will arrive and 
start killing processes. So that will just stop our builds from completing.

Yeah, I prefer a babysitter over a child murderer :-)

Ferry


  reply	other threads:[~2021-04-11 16:08 UTC|newest]

Thread overview: 53+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-04-11 15:23 bitbake controlling memory use Gmane Admin
2021-04-11 15:27 ` [yocto] " Alexander Kanavin
2021-04-11 15:49   ` Gmane Admin
2021-04-11 15:55     ` [yocto] " Alexander Kanavin
2021-04-11 16:08       ` Gmane Admin [this message]
2021-04-11 16:19         ` Alexander Kanavin
2021-04-14  1:14           ` Randy MacLeod
2021-04-14  3:14             ` Khem Raj
2021-04-14  4:59             ` Richard Purdie
2021-04-17 22:17               ` Gmane Admin
2021-04-18  9:59                 ` [yocto] " Richard Purdie
2021-04-18 19:34                   ` Gmane Admin
     [not found]                   ` <16770AD5E94DEAE5.1089@lists.yoctoproject.org>
2022-12-24 18:58                     ` Ferry Toth
2022-12-26  2:11                       ` [yocto] " Randy MacLeod
2022-12-27 16:56                         ` Ferry Toth
2021-06-05 13:35               ` Gmane Admin
2021-06-08 19:08                 ` [yocto] " Trevor Gamblin
2021-06-10  7:09                   ` Gmane Admin
     [not found]               ` <1685B30F5819A655.13459@lists.yoctoproject.org>
2021-06-07 19:27                 ` Gmane Admin
2021-06-08 19:12                   ` [yocto] " Trevor Gamblin
2021-06-10  9:22                   ` Ferry Toth
2021-06-10 19:06                     ` [yocto] " Trevor Gamblin
2021-06-10 20:35                       ` Ferry Toth
2021-06-12 16:31                         ` Ferry Toth
2021-06-13  0:38                           ` Randy MacLeod
2021-06-13 10:58                             ` Ferry Toth
2023-01-03 14:15                             ` Ferry Toth
2023-01-03 14:18                               ` [yocto] " Alexander Kanavin
2023-01-03 14:29                                 ` Ferry Toth
2023-01-03 14:36                                   ` Quentin Schulz
2023-01-03 14:41                                     ` Ferry Toth
2023-01-03 15:03                                       ` Janne Kiiskila
2023-01-03 15:12                                         ` Ferry Toth
2023-01-03 15:24                                           ` Alexander Kanavin
     [not found]                                           ` <1736D5DCC66BC3DE.4716@lists.yoctoproject.org>
2023-01-03 15:37                                             ` Alexander Kanavin
2023-01-03 15:58                                               ` Ferry Toth
2023-01-03 17:33                                                 ` Alexander Kanavin
2023-01-08 22:13                                                   ` Ferry Toth
2023-01-09 10:39                                                     ` Alexander Kanavin
2023-01-09 10:43                                                       ` Richard Purdie
2023-01-09 10:54                                                         ` Ferry Toth
2023-01-19 19:59                                                           ` Ferry Toth
2023-01-09 10:49                                                       ` Ferry Toth
2023-01-10  1:57                                                         ` Randy MacLeod
2023-01-10  9:24                                                           ` Ferry Toth
2023-01-14 22:33                                                             ` Ferry Toth
2023-08-28 12:56                                                               ` Martin Hundebøll
2023-01-03 15:44                                   ` Martin Jansa
2021-04-12  2:25 ` Chen Qi
2021-04-12 18:45   ` Gmane Admin
2021-04-12  8:13 ` [yocto] " Robert Berger
2021-04-12 18:47   ` Gmane Admin
     [not found] ` <1b153bce-a66a-45ee-a5c6-963ea6fb1c82.949ef384-8293-46b8-903f-40a477c056ae.12d167de-8268-4441-bd03-b78a3e04713e@emailsignatures365.codetwo.com>
     [not found]   ` <1b153bce-a66a-45ee-a5c6-963ea6fb1c82.0d2bd5fa-15cc-4b27-b94e-83614f9e5b38.a8aa18ca-4a64-4525-851c-c6d34f355f1f@emailsignatures365.codetwo.com>
2021-04-12  8:28     ` [yocto] " Mike Looijmans

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=14207c27-7f5e-9a13-d8e8-28dbb0e7ea7b@gmail.com \
    --to=gley-yocto@m.gmane-mx.org \
    --cc=fntoth-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org \
    --cc=yocto-gJxINTE1zeQw8ufyyF7U+SST3g8Odh+X-XMD5yJDbdMReXY1tMh2IBg@public.gmane.org \
    --cc=yocto-gJxINTE1zeQw8ufyyF7U+SST3g8Odh+X@public.gmane.org \
    --cc=yocto@lists.yoctoproject.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.