[PATCH] genboardscfg: limit to 240 jobs

[PATCH] genboardscfg: limit to 240 jobs

[Andre Przywara]

When genboardscfg.py is run on machines with 255 or more cores, the
process will consume more than 1024 file descriptors, which is a common
standard ulimit for user processes. As a consequence it will fail with a
lenghty Python trace, with the almost hidden message:
OSError: [Errno 24] Too many open files

It's somewhat questionable whether that level of parallelity is actually
useful for genboardscfg, so we limit the *default* number of jobs to the
safe number of 240, to avoid the problem.
If a user persists, she can still force a higher number via the -j
parameter - hopefully having raised the ulimit accordingly beforehand.

Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
 tools/genboardscfg.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/genboardscfg.py b/tools/genboardscfg.py
index 4ee7aa1f89..07bf681d1d 100755
--- a/tools/genboardscfg.py
+++ b/tools/genboardscfg.py
@@ -430,7 +430,7 @@ def main():
     # Add options here
     parser.add_option('-f', '--force', action="store_true", default=False,
                       help='regenerate the output even if it is new')
-    parser.add_option('-j', '--jobs', type='int', default=cpu_count,
+    parser.add_option('-j', '--jobs', type='int', default=min(cpu_count, 240),
                       help='the number of jobs to run simultaneously')
     parser.add_option('-o', '--output', default=OUTPUT_FILE,
                       help='output file [default=%s]' % OUTPUT_FILE)
-- 
2.25.1

Re: [PATCH] genboardscfg: limit to 240 jobs

[Heinrich Schuchardt]

On 1/11/22 16:34, Andre Przywara wrote:
> When genboardscfg.py is run on machines with 255 or more cores, the
> process will consume more than 1024 file descriptors, which is a common
> standard ulimit for user processes. As a consequence it will fail with a
> lenghty Python trace, with the almost hidden message:
> OSError: [Errno 24] Too many open files
>
> It's somewhat questionable whether that level of parallelity is actually
> useful for genboardscfg, so we limit the *default* number of jobs to the
> safe number of 240, to avoid the problem.
> If a user persists, she can still force a higher number via the -j

Nits:
%s/persists/insists/
%s/she/they/ if you don't specifically mean a female user.

Otherwise

Reviewed-by: Heinrich Schuchardt <xypron.glpk@gmx.de>

> parameter - hopefully having raised the ulimit accordingly beforehand.
>
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
>   tools/genboardscfg.py | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/tools/genboardscfg.py b/tools/genboardscfg.py
> index 4ee7aa1f89..07bf681d1d 100755
> --- a/tools/genboardscfg.py
> +++ b/tools/genboardscfg.py
> @@ -430,7 +430,7 @@ def main():
>       # Add options here
>       parser.add_option('-f', '--force', action="store_true", default=False,
>                         help='regenerate the output even if it is new')
> -    parser.add_option('-j', '--jobs', type='int', default=cpu_count,
> +    parser.add_option('-j', '--jobs', type='int', default=min(cpu_count, 240),
>                         help='the number of jobs to run simultaneously')
>       parser.add_option('-o', '--output', default=OUTPUT_FILE,
>                         help='output file [default=%s]' % OUTPUT_FILE)

Re: [PATCH] genboardscfg: limit to 240 jobs

[Simon Glass]

On Tue, 11 Jan 2022 at 08:35, Andre Przywara <andre.przywara@arm.com> wrote:
>
> When genboardscfg.py is run on machines with 255 or more cores, the
> process will consume more than 1024 file descriptors, which is a common
> standard ulimit for user processes. As a consequence it will fail with a
> lenghty Python trace, with the almost hidden message:
> OSError: [Errno 24] Too many open files
>
> It's somewhat questionable whether that level of parallelity is actually
> useful for genboardscfg, so we limit the *default* number of jobs to the
> safe number of 240, to avoid the problem.
> If a user persists, she can still force a higher number via the -j
> parameter - hopefully having raised the ulimit accordingly beforehand.
>
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
>  tools/genboardscfg.py | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

Reviewed-by: Simon Glass <sjg@chromium.org>

Please send such machine. Thank you.

Re: [PATCH] genboardscfg: limit to 240 jobs

[Simon Glass]

On Tue, 11 Jan 2022 at 08:35, Andre Przywara <andre.przywara@arm.com> wrote:
>
> When genboardscfg.py is run on machines with 255 or more cores, the
> process will consume more than 1024 file descriptors, which is a common
> standard ulimit for user processes. As a consequence it will fail with a
> lenghty Python trace, with the almost hidden message:
> OSError: [Errno 24] Too many open files
>
> It's somewhat questionable whether that level of parallelity is actually
> useful for genboardscfg, so we limit the *default* number of jobs to the
> safe number of 240, to avoid the problem.
> If a user persists, she can still force a higher number via the -j
> parameter - hopefully having raised the ulimit accordingly beforehand.
>
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
>  tools/genboardscfg.py | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

Reviewed-by: Simon Glass <sjg@chromium.org>

Please send such machine. Thank you.

Applied to u-boot-dm, thanks!