[PATCH mptcp-next v2] selftests: mptcp: increase timeout to 20 minutes

[PATCH mptcp-next v2] selftests: mptcp: increase timeout to 20 minutes

[Matthieu Baerts]

With the increase number of tests, one CI instance, using a debug kernel
config and not recent hardware, takes around 10 minutes to execute the
slowest MPTCP test: mptcp_join.sh.

Even if most CIs don't take that long to execute these tests --
typically max 10 minutes to run all selftests -- it will helpi some of
them if the timeout is increased.

The timeout could be disabled but it is always good to have an extra
safeguard, just in case.

Please note that on slow public CIs with kernel debug settings, it has
been observed it can easily take up to 45 minutes to execute all tests
in this very slow environment with other jobs running in parallel.
The slowest test, mptcp_join.sh takes ~30 minutes in this case.

In such environments, the selftests timeout set in the 'settings' file
is disabled because this environment is known as being exceptionnaly
slow. It has been decided not to take such exceptional environments into
account and set the timeout to 20min.

Signed-off-by: Matthieu Baerts <matthieu.baerts@tessares.net>
---

Notes:
    On my CI at Tessares, another machine is now used with a different host
    kernel. mptcp_join.sh selftest is now regularly taking more than 10
    minutes when testing with a debug kernel.

    v2:
     - from 1h to 20 min (Mat) and adapt the commit message to explain why
       this don't cover all cases that have been observed.

 tools/testing/selftests/net/mptcp/settings | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/testing/selftests/net/mptcp/settings b/tools/testing/selftests/net/mptcp/settings
index a62d2fa1275c..79b65bdf05db 100644
--- a/tools/testing/selftests/net/mptcp/settings
+++ b/tools/testing/selftests/net/mptcp/settings
@@ -1 +1 @@
-timeout=600
+timeout=1200
-- 
2.34.1

Re: [PATCH mptcp-next v2] selftests: mptcp: increase timeout to 20 minutes

[Matthieu Baerts]

Hi Mat,

On 08/02/2022 10:10, Matthieu Baerts wrote:
> With the increase number of tests, one CI instance, using a debug kernel
> config and not recent hardware, takes around 10 minutes to execute the
> slowest MPTCP test: mptcp_join.sh.
> 
> Even if most CIs don't take that long to execute these tests --
> typically max 10 minutes to run all selftests -- it will helpi some of
> them if the timeout is increased.
> 
> The timeout could be disabled but it is always good to have an extra
> safeguard, just in case.
> 
> Please note that on slow public CIs with kernel debug settings, it has
> been observed it can easily take up to 45 minutes to execute all tests
> in this very slow environment with other jobs running in parallel.
> The slowest test, mptcp_join.sh takes ~30 minutes in this case.
> 
> In such environments, the selftests timeout set in the 'settings' file
> is disabled because this environment is known as being exceptionnaly
> slow. It has been decided not to take such exceptional environments into
> account and set the timeout to 20min.
> 
> Signed-off-by: Matthieu Baerts <matthieu.baerts@tessares.net>
> ---
> 
> Notes:
>     On my CI at Tessares, another machine is now used with a different host
>     kernel. mptcp_join.sh selftest is now regularly taking more than 10
>     minutes when testing with a debug kernel.
> 
>     v2:
>      - from 1h to 20 min (Mat) and adapt the commit message to explain why
>        this don't cover all cases that have been observed.

I hope you don't mind if I apply this patch now so my CI stops
complaining about all modifications of the export branch. I can fix it
later if needed.

- 87dac6279023: selftests: mptcp: increase timeout to 20 minutes
- ec54f23ec607: tg:msg: fix typo
- Results: a206ca2f7274..7b9f56739735

Builds and tests are now in progress:

https://cirrus-ci.com/github/multipath-tcp/mptcp_net-next/export/20220208T091856
https://github.com/multipath-tcp/mptcp_net-next/actions/workflows/build-validation.yml?query=branch:export

Cheers,
Matt
-- 
Tessares | Belgium | Hybrid Access Solutions
www.tessares.net

Re: [PATCH mptcp-next v2] selftests: mptcp: increase timeout to 20 minutes

[Mat Martineau]

On Tue, 8 Feb 2022, Matthieu Baerts wrote:

> Hi Mat,
>
> On 08/02/2022 10:10, Matthieu Baerts wrote:
>> With the increase number of tests, one CI instance, using a debug kernel
>> config and not recent hardware, takes around 10 minutes to execute the
>> slowest MPTCP test: mptcp_join.sh.
>>
>> Even if most CIs don't take that long to execute these tests --
>> typically max 10 minutes to run all selftests -- it will helpi some of
>> them if the timeout is increased.
>>
>> The timeout could be disabled but it is always good to have an extra
>> safeguard, just in case.
>>
>> Please note that on slow public CIs with kernel debug settings, it has
>> been observed it can easily take up to 45 minutes to execute all tests
>> in this very slow environment with other jobs running in parallel.
>> The slowest test, mptcp_join.sh takes ~30 minutes in this case.
>>
>> In such environments, the selftests timeout set in the 'settings' file
>> is disabled because this environment is known as being exceptionnaly
>> slow. It has been decided not to take such exceptional environments into
>> account and set the timeout to 20min.
>>
>> Signed-off-by: Matthieu Baerts <matthieu.baerts@tessares.net>
>> ---
>>
>> Notes:
>>     On my CI at Tessares, another machine is now used with a different host
>>     kernel. mptcp_join.sh selftest is now regularly taking more than 10
>>     minutes when testing with a debug kernel.
>>
>>     v2:
>>      - from 1h to 20 min (Mat) and adapt the commit message to explain why
>>        this don't cover all cases that have been observed.
>
> I hope you don't mind if I apply this patch now so my CI stops
> complaining about all modifications of the export branch. I can fix it
> later if needed.

No problem!

>
> - 87dac6279023: selftests: mptcp: increase timeout to 20 minutes
> - ec54f23ec607: tg:msg: fix typo
> - Results: a206ca2f7274..7b9f56739735
>
> Builds and tests are now in progress:
>
> https://cirrus-ci.com/github/multipath-tcp/mptcp_net-next/export/20220208T091856
> https://github.com/multipath-tcp/mptcp_net-next/actions/workflows/build-validation.yml?query=branch:export
>

--
Mat Martineau
Intel