Re: [OE-core] [PATCH 1/1] bitbake.conf: Set ZSTD_THREADS to half of cpu number

[Alexander Kanavin <alex.kanavin@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 1682 bytes --]

But does this mean we need to globally restrict this and make things twice
slower for everyone? I'm not sure.


Alex

On Tue, 9 Nov 2021 at 09:12, Robert Yang <liezhi.yang@windriver.com> wrote:

> The original value is very easy to cause do_packge error when cpu number is
> larger, for example, 128 cores and 512G mem:
>
> error: create archive failed: cpio: write failed - Cannot allocate memory"
>
> Set the ZSTD_THREADS to half of the CPU number can avoid the error in my
> testing.
>
> Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
> ---
>  meta/conf/bitbake.conf | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> index 71c1e52ad6..46ebf5113f 100644
> --- a/meta/conf/bitbake.conf
> +++ b/meta/conf/bitbake.conf
> @@ -833,7 +833,7 @@ XZ_DEFAULTS ?= "--memlimit=${XZ_MEMLIMIT}
> --threads=${XZ_THREADS}"
>  XZ_DEFAULTS[vardepsexclude] += "XZ_MEMLIMIT XZ_THREADS"
>
>  # Default parallelism for zstd
> -ZSTD_THREADS ?= "${@oe.utils.cpu_count(at_least=2)}"
> +ZSTD_THREADS ?= "${@int(oe.utils.cpu_count(at_least=4)/2)}"
>  ZSTD_THREADS[vardepvalue] = "1"
>
>  # Limit the number of threads that OpenMP libraries will use. Otherwise
> they
> --
> 2.17.1
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#157998):
> https://lists.openembedded.org/g/openembedded-core/message/157998
> Mute This Topic: https://lists.openembedded.org/mt/86926962/1686489
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> alex.kanavin@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

[-- Attachment #2: Type: text/html, Size: 2728 bytes --]