[PATCH mptcp-net] selftests: mptcp: userspace pm: uniform verify events

[PATCH mptcp-net] selftests: mptcp: userspace pm: uniform verify events

[Matthieu Baerts]

Simply adding a "sleep" before checking something is usually not a good
idea because the time that has been picked can not be enough or too
much. The best is to wait for events with a timeout.

In this selftest, 'sleep 0.5' is used more than 40 times. It is always
used before calling a 'verify_*' function except for this
verify_listener_events which has been added later.

At the end, using all these 'sleep 0.5' seems to work: the slow CIs
don't complain so far. Also because it doesn't take too much time, we
can just add two more 'sleep 0.5' to uniform what is done before calling
a 'verify_*' function. For the same reasons, we can also delay a bigger
refactoring to replace all these 'sleep 0.5' by functions waiting for
events instead of waiting for a fix time and hope for the best.

Fixes: 6c73008aa301 ("selftests: mptcp: listener test for userspace PM")
Suggested-by: Paolo Abeni <pabeni@redhat.com>
Signed-off-by: Matthieu Baerts <matthieu.baerts@tessares.net>
---
 tools/testing/selftests/net/mptcp/userspace_pm.sh | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/tools/testing/selftests/net/mptcp/userspace_pm.sh b/tools/testing/selftests/net/mptcp/userspace_pm.sh
index 48e52f995a98..b1eb7bce599d 100755
--- a/tools/testing/selftests/net/mptcp/userspace_pm.sh
+++ b/tools/testing/selftests/net/mptcp/userspace_pm.sh
@@ -913,6 +913,7 @@ test_listener()
 		$client4_port > /dev/null 2>&1 &
 	local listener_pid=$!
 
+	sleep 0.5
 	verify_listener_events $client_evts $LISTENER_CREATED $AF_INET 10.0.2.2 $client4_port
 
 	# ADD_ADDR from client to server machine reusing the subflow port
@@ -928,6 +929,7 @@ test_listener()
 	# Delete the listener from the client ns, if one was created
 	kill_wait $listener_pid
 
+	sleep 0.5
 	verify_listener_events $client_evts $LISTENER_CLOSED $AF_INET 10.0.2.2 $client4_port
 }
 

---
base-commit: 1281a7eb9aa62b79cb9ad4d221ae31659ed892d0
change-id: 20230328-mptcp-selftests-userspace-sleep-verify-listen-4fb855bc054d

Best regards,
-- 
Matthieu Baerts <matthieu.baerts@tessares.net>

Re: selftests: mptcp: userspace pm: uniform verify events: Tests Results

[MPTCP CI]

Hi Matthieu,

Thank you for your modifications, that's great!

Our CI did some validations and here is its report:

- KVM Validation: normal (except selftest_mptcp_join):
  - Success! ✅:
  - Task: https://cirrus-ci.com/task/5618456821760000
  - Summary: https://api.cirrus-ci.com/v1/artifact/task/5618456821760000/summary/summary.txt

- KVM Validation: debug (except selftest_mptcp_join):
  - Success! ✅:
  - Task: https://cirrus-ci.com/task/4518945193984000
  - Summary: https://api.cirrus-ci.com/v1/artifact/task/4518945193984000/summary/summary.txt

- KVM Validation: normal (only selftest_mptcp_join):
  - Success! ✅:
  - Task: https://cirrus-ci.com/task/6744356728602624
  - Summary: https://api.cirrus-ci.com/v1/artifact/task/6744356728602624/summary/summary.txt

- KVM Validation: debug (only selftest_mptcp_join):
  - Success! ✅:
  - Task: https://cirrus-ci.com/task/5644845100826624
  - Summary: https://api.cirrus-ci.com/v1/artifact/task/5644845100826624/summary/summary.txt

Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/84382bc475d1


If there are some issues, you can reproduce them using the same environment as
the one used by the CI thanks to a docker image, e.g.:

    $ cd [kernel source code]
    $ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
        --pull always mptcp/mptcp-upstream-virtme-docker:latest \
        auto-debug

For more details:

    https://github.com/multipath-tcp/mptcp-upstream-virtme-docker


Please note that despite all the efforts that have been already done to have a
stable tests suite when executed on a public CI like here, it is possible some
reported issues are not due to your modifications. Still, do not hesitate to
help us improve that ;-)

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (Tessares)

Re: [PATCH mptcp-net] selftests: mptcp: userspace pm: uniform verify events

[Paolo Abeni]

On Tue, 2023-03-28 at 19:05 +0200, Matthieu Baerts wrote:
> Simply adding a "sleep" before checking something is usually not a good
> idea because the time that has been picked can not be enough or too
> much. The best is to wait for events with a timeout.
> 
> In this selftest, 'sleep 0.5' is used more than 40 times. It is always
> used before calling a 'verify_*' function except for this
> verify_listener_events which has been added later.
> 
> At the end, using all these 'sleep 0.5' seems to work: the slow CIs
> don't complain so far. Also because it doesn't take too much time, we
> can just add two more 'sleep 0.5' to uniform what is done before calling
> a 'verify_*' function. For the same reasons, we can also delay a bigger
> refactoring to replace all these 'sleep 0.5' by functions waiting for
> events instead of waiting for a fix time and hope for the best.
> 
> Fixes: 6c73008aa301 ("selftests: mptcp: listener test for userspace PM")
> Suggested-by: Paolo Abeni <pabeni@redhat.com>
> Signed-off-by: Matthieu Baerts <matthieu.baerts@tessares.net>
> ---
>  tools/testing/selftests/net/mptcp/userspace_pm.sh | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/tools/testing/selftests/net/mptcp/userspace_pm.sh b/tools/testing/selftests/net/mptcp/userspace_pm.sh
> index 48e52f995a98..b1eb7bce599d 100755
> --- a/tools/testing/selftests/net/mptcp/userspace_pm.sh
> +++ b/tools/testing/selftests/net/mptcp/userspace_pm.sh
> @@ -913,6 +913,7 @@ test_listener()
>  		$client4_port > /dev/null 2>&1 &
>  	local listener_pid=$!
>  
> +	sleep 0.5
>  	verify_listener_events $client_evts $LISTENER_CREATED $AF_INET 10.0.2.2 $client4_port
>  
>  	# ADD_ADDR from client to server machine reusing the subflow port
> @@ -928,6 +929,7 @@ test_listener()
>  	# Delete the listener from the client ns, if one was created
>  	kill_wait $listener_pid
>  
> +	sleep 0.5
>  	verify_listener_events $client_evts $LISTENER_CLOSED $AF_INET 10.0.2.2 $client4_port
>  }
>  
> 
> ---
> base-commit: 1281a7eb9aa62b79cb9ad4d221ae31659ed892d0
> change-id: 20230328-mptcp-selftests-userspace-sleep-verify-listen-4fb855bc054d
> 
> Best regards,

LGTM!

(no need for a forma acked-by tag imho here due to the already present
tag)

/P

Re: [PATCH mptcp-net] selftests: mptcp: userspace pm: uniform verify events

[Matthieu Baerts]

Hi Paolo,

On 04/04/2023 15:59, Paolo Abeni wrote:
> On Tue, 2023-03-28 at 19:05 +0200, Matthieu Baerts wrote:
>> Simply adding a "sleep" before checking something is usually not a good
>> idea because the time that has been picked can not be enough or too
>> much. The best is to wait for events with a timeout.
>>
>> In this selftest, 'sleep 0.5' is used more than 40 times. It is always
>> used before calling a 'verify_*' function except for this
>> verify_listener_events which has been added later.
>>
>> At the end, using all these 'sleep 0.5' seems to work: the slow CIs
>> don't complain so far. Also because it doesn't take too much time, we
>> can just add two more 'sleep 0.5' to uniform what is done before calling
>> a 'verify_*' function. For the same reasons, we can also delay a bigger
>> refactoring to replace all these 'sleep 0.5' by functions waiting for
>> events instead of waiting for a fix time and hope for the best.

(...)

> LGTM!
> 
> (no need for a forma acked-by tag imho here due to the already present
> tag)

Thank you for this review!

Now in our tree (fix for -net):

New patches for t/upstream-net and t/upstream:
- 8ce0e99c6ffe: selftests: mptcp: userspace pm: uniform verify events
- Results: 522e529cc873..43c38b337813 (export-net)
- Results: 8fbf51a0d18f..16abaebe8405 (export)

Tests are now in progress:

https://cirrus-ci.com/github/multipath-tcp/mptcp_net-next/export-net/20230405T134336
https://cirrus-ci.com/github/multipath-tcp/mptcp_net-next/export/20230405T134336

Cheers,
Matt
-- 
Tessares | Belgium | Hybrid Access Solutions
www.tessares.net