All of lore.kernel.org
 help / color / mirror / Atom feed
From: Matthieu Baerts <matthieu.baerts@tessares.net>
To: Mat Martineau <mathew.j.martineau@linux.intel.com>,
	Paolo Abeni <pabeni@redhat.com>
Cc: mptcp@lists.linux.dev
Subject: Re: [RFC PATCH v2] selftests: mptcp: more stable simult_flows tests
Date: Mon, 18 Oct 2021 14:30:58 +0200	[thread overview]
Message-ID: <9ae482b2-90e1-8e4c-7b98-9de3072bd242@tessares.net> (raw)
In-Reply-To: <1fe3d619-370f-d5bb-6d6f-af6890f9fc43@tessares.net>

On 18/10/2021 12:24, Matthieu Baerts wrote:
> Hi Paolo, Mat,
> 
> On 16/10/2021 02:12, Mat Martineau wrote:
>> On Fri, 15 Oct 2021, Paolo Abeni wrote:
>>
>>> Currently the simult_flows.sh self-tests are not very
>>> stables, expecially when running on slow VM.
>>>
>>> The tests mesures runtime for transfers on multiple subflows
>>> and check that the time is nearby the theoretical maximum.
>>>
>>> The current test infra introduces a bit of jitter in test
>>> runtime, due to multiple explicit delays. Additionally the
>>> runtime is measured by the shell script wrapper. On slow
>>> VM, the script overhead is measurable and subject to relevant
>>> jitter.
>>>
>>> One solution to make the test more stable would be adding more
>>> slack to the expected time; that could possibly hide reall
>>> regressions. Instead move the measurement inside the command
>>> doint the transfer, and drop most unneeded sleep.
> 
> Thank you for looking at that!
> 
> It is indeed more stable: with a debug kernel and a "slow host", the
> test fails almost all the time. Now I got 3 failures in X attempts, not
> bad! (and I stopped after the last failure)

And X=18 here ;-)

And for a maybe better output for the tests results from my previous email:

https://paste.centos.org/view/d75b3836

Cheers,
Matt
-- 
Tessares | Belgium | Hybrid Access Solutions
www.tessares.net

  reply	other threads:[~2021-10-18 12:31 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-10-15 15:09 [RFC PATCH v2] selftests: mptcp: more stable simult_flows tests Paolo Abeni
2021-10-15 16:14 ` selftests: mptcp: more stable simult_flows tests: Tests Results MPTCP CI
2021-10-16  0:12 ` [RFC PATCH v2] selftests: mptcp: more stable simult_flows tests Mat Martineau
2021-10-18 10:24   ` Matthieu Baerts
2021-10-18 12:30     ` Matthieu Baerts [this message]
2021-10-18 14:09       ` Matthieu Baerts
2021-10-20 17:43   ` Paolo Abeni
2021-10-21  8:54     ` Matthieu Baerts
2021-10-21  9:10       ` Paolo Abeni
2021-10-21 10:05 ` Matthieu Baerts

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=9ae482b2-90e1-8e4c-7b98-9de3072bd242@tessares.net \
    --to=matthieu.baerts@tessares.net \
    --cc=mathew.j.martineau@linux.intel.com \
    --cc=mptcp@lists.linux.dev \
    --cc=pabeni@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.