[Buildroot] [PATCH] support/testing/run-tests: fix --testcases option

[Buildroot] [PATCH] support/testing/run-tests: fix --testcases option

[Arnout Vandecappelle (Essensium/Mind)]

The --testcases option of run-tests says how many test cases to build in
parallel. It automatically derives a jlevel from it by dividing the
number of cores + 1 by the number of parallel testcases. However, this
will typically result in a fractional number. Make doesn't like
fractional numbers as argument to -j.

Convert the number to integer (rounding up).

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
 support/testing/run-tests | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/support/testing/run-tests b/support/testing/run-tests
index 022209b414..f9c0bebc48 100755
--- a/support/testing/run-tests
+++ b/support/testing/run-tests
@@ -87,7 +87,7 @@ def main():
             return 1
         # same default BR2_JLEVEL as package/Makefile.in
         br2_jlevel = 1 + multiprocessing.cpu_count()
-        each_testcase = br2_jlevel / args.testcases
+        each_testcase = int((br2_jlevel + args.testcases - 1) / args.testcases)
         if each_testcase < 1:
             each_testcase = 1
         BRConfigTest.jlevel = each_testcase
-- 
2.35.3

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH] support/testing/run-tests: fix --testcases option

[Yann E. MORIN]

Arnout, All,

On 2022-07-27 17:02 +0200, Arnout Vandecappelle (Essensium/Mind) spake thusly:
> The --testcases option of run-tests says how many test cases to build in
> parallel. It automatically derives a jlevel from it by dividing the
> number of cores + 1 by the number of parallel testcases. However, this
> will typically result in a fractional number. Make doesn't like
> fractional numbers as argument to -j.
> 
> Convert the number to integer (rounding up).
> 
> Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
> ---
>  support/testing/run-tests | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/support/testing/run-tests b/support/testing/run-tests
> index 022209b414..f9c0bebc48 100755
> --- a/support/testing/run-tests
> +++ b/support/testing/run-tests
> @@ -87,7 +87,7 @@ def main():
>              return 1
>          # same default BR2_JLEVEL as package/Makefile.in
>          br2_jlevel = 1 + multiprocessing.cpu_count()
> -        each_testcase = br2_jlevel / args.testcases
> +        each_testcase = int((br2_jlevel + args.testcases - 1) / args.testcases)

* br2_jlevel is an int, as multiprocessing.cpu_count() is an int, so it
  will be always >=2  (cpu_count() raises an error if it can't determine
  the number of CPU, so it will always return at least 1);

* args.testcases is an int, and is checked to be >=1

So, with the following, we're guaranteed to have an int that is >=1:

    int((br2_jlevel + args.testcases) / args.testcases)

Because br2_jlevel + args.testcases is guaranteed to always be bigger
than or equal to args.testcases.

>          if each_testcase < 1:
>              each_testcase = 1

So that test can be dropped now.

I've amended your change with the above. Please yell if my maths are
wrong... ;-)

Regards,
Yann E. MORIN.

>          BRConfigTest.jlevel = each_testcase
> -- 
> 2.35.3
> 
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH] support/testing/run-tests: fix --testcases option

[Arnout Vandecappelle]

On 27/07/2022 18:20, Yann E. MORIN wrote:
> Arnout, All,
> 
> On 2022-07-27 17:02 +0200, Arnout Vandecappelle (Essensium/Mind) spake thusly:
>> The --testcases option of run-tests says how many test cases to build in
>> parallel. It automatically derives a jlevel from it by dividing the
>> number of cores + 1 by the number of parallel testcases. However, this
>> will typically result in a fractional number. Make doesn't like
>> fractional numbers as argument to -j.
>>
>> Convert the number to integer (rounding up).
>>
>> Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
>> ---
>>   support/testing/run-tests | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/support/testing/run-tests b/support/testing/run-tests
>> index 022209b414..f9c0bebc48 100755
>> --- a/support/testing/run-tests
>> +++ b/support/testing/run-tests
>> @@ -87,7 +87,7 @@ def main():
>>               return 1
>>           # same default BR2_JLEVEL as package/Makefile.in
>>           br2_jlevel = 1 + multiprocessing.cpu_count()
>> -        each_testcase = br2_jlevel / args.testcases
>> +        each_testcase = int((br2_jlevel + args.testcases - 1) / args.testcases)
> 
> * br2_jlevel is an int, as multiprocessing.cpu_count() is an int, so it
>    will be always >=2  (cpu_count() raises an error if it can't determine
>    the number of CPU, so it will always return at least 1);
> 
> * args.testcases is an int, and is checked to be >=1
> 
> So, with the following, we're guaranteed to have an int that is >=1:
> 
>      int((br2_jlevel + args.testcases) / args.testcases)

  That way you're adding one if it is actually divisible. E.g.
testcases = 2, cpu_count == 3
  => br_jlevel = 4
  => each_testcase = (4 + 2) / 2 = 3
  => total cpu use = 3 * 2 = 6

  In practice though, num_cpus is almost always a multiple of 2, so the 
likelihood of it being divisible is very small. Also, in practice, it's probably 
better to have a total cpu use number that is higher than the actual number of 
cpus, since a lot of time time is spent on serial execution. So don't bother 
changing it again.

> Because br2_jlevel + args.testcases is guaranteed to always be bigger
> than or equal to args.testcases.
> 
>>           if each_testcase < 1:
>>               each_testcase = 1

  Yes, this is true even with the proper rounding, because br2_jlevel > 1 
(assuming you have at least one CPU).

  Regards,
  Arnout

> 
> So that test can be dropped now.
> 
> I've amended your change with the above. Please yell if my maths are
> wrong... ;-)
> 
> Regards,
> Yann E. MORIN.
> 
>>           BRConfigTest.jlevel = each_testcase
>> -- 
>> 2.35.3
>>
>> _______________________________________________
>> buildroot mailing list
>> buildroot@buildroot.org
>> https://lists.buildroot.org/mailman/listinfo/buildroot
> 
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH] support/testing/run-tests: fix --testcases option

[Peter Korsgaard]

>>>>> "Arnout" == Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> writes:

 > The --testcases option of run-tests says how many test cases to build in
 > parallel. It automatically derives a jlevel from it by dividing the
 > number of cores + 1 by the number of parallel testcases. However, this
 > will typically result in a fractional number. Make doesn't like
 > fractional numbers as argument to -j.

 > Convert the number to integer (rounding up).

 > Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>

Committed to 2022.05.x and 2022.02.x, thanks.

-- 
Bye, Peter Korsgaard
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot