All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Daniel P. Berrangé" <berrange@redhat.com>
To: qemu-devel@nongnu.org
Cc: "Paolo Bonzini" <pbonzini@redhat.com>,
	"Laurent Vivier" <lvivier@redhat.com>,
	"Juan Quintela" <quintela@redhat.com>,
	"Thomas Huth" <thuth@redhat.com>,
	"Cornelia Huck" <cohuck@redhat.com>,
	qemu-s390x@nongnu.org,
	"Dr. David Alan Gilbert" <dgilbert@redhat.com>,
	"Daniel P. Berrangé" <berrange@redhat.com>
Subject: [PATCH 1/5] tests: wait max 120 seconds for migration test status changes
Date: Tue, 28 Jun 2022 11:54:30 +0100	[thread overview]
Message-ID: <20220628105434.295905-2-berrange@redhat.com> (raw)
In-Reply-To: <20220628105434.295905-1-berrange@redhat.com>

Currently the wait_for_migration_fail and wait_for_migration_complete
functions will spin in an infinite loop checking query-migrate status
to detect a specific change/goal. This is fine when everything goes
to plan, but when the unusual happens, these will hang the test suite
forever.

Any normally executing migration test case normally takes < 1 second
for a state change, with exception of the autoconverge test which
takes about 5 seconds. Taking into account possibility of people
running tests inside TCG, allowing a factor of x20 slowdown gives
a reasonable worst case of 120 seconds. Anything taking longer than
this is a strong sign that the test has hung, or the test should be
rewritten to be faster.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 tests/qtest/migration-helpers.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/tests/qtest/migration-helpers.c b/tests/qtest/migration-helpers.c
index a6aa59e4e6..e81e831c85 100644
--- a/tests/qtest/migration-helpers.c
+++ b/tests/qtest/migration-helpers.c
@@ -15,6 +15,14 @@
 
 #include "migration-helpers.h"
 
+/*
+ * Number of seconds we wait when looking for migration
+ * status changes, to avoid test suite hanging forever
+ * when things go wrong. Needs to be higher enough to
+ * avoid false positives on loaded hosts.
+ */
+#define MIGRATION_STATUS_WAIT_TIMEOUT 120
+
 bool got_stop;
 
 static void check_stop_event(QTestState *who)
@@ -166,8 +174,11 @@ static bool check_migration_status(QTestState *who, const char *goal,
 void wait_for_migration_status(QTestState *who,
                                const char *goal, const char **ungoals)
 {
+    g_test_timer_start();
     while (!check_migration_status(who, goal, ungoals)) {
         usleep(1000);
+
+        g_assert(g_test_timer_elapsed() < MIGRATION_STATUS_WAIT_TIMEOUT);
     }
 }
 
@@ -178,6 +189,7 @@ void wait_for_migration_complete(QTestState *who)
 
 void wait_for_migration_fail(QTestState *from, bool allow_active)
 {
+    g_test_timer_start();
     QDict *rsp_return;
     char *status;
     bool failed;
@@ -193,6 +205,8 @@ void wait_for_migration_fail(QTestState *from, bool allow_active)
         g_assert(result);
         failed = !strcmp(status, "failed");
         g_free(status);
+
+        g_assert(g_test_timer_elapsed() < MIGRATION_STATUS_WAIT_TIMEOUT);
     } while (!failed);
 
     /* Is the machine currently running? */
-- 
2.36.1



  reply	other threads:[~2022-06-28 10:57 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-06-28 10:54 [PATCH 0/5] tests: improve reliability of migration test Daniel P. Berrangé
2022-06-28 10:54 ` Daniel P. Berrangé [this message]
2022-06-28 12:47   ` [PATCH 1/5] tests: wait max 120 seconds for migration test status changes Laurent Vivier
2022-06-28 12:49   ` Thomas Huth
2022-06-28 10:54 ` [PATCH 2/5] tests: wait for migration completion before looking for STOP event Daniel P. Berrangé
2022-06-28 12:47   ` Laurent Vivier
2022-06-28 14:08   ` Dr. David Alan Gilbert
2022-06-28 14:10     ` Daniel P. Berrangé
2022-06-28 10:54 ` [PATCH 3/5] tests: increase migration test converge downtime to 30 seconds Daniel P. Berrangé
2022-06-28 12:47   ` Laurent Vivier
2022-06-28 10:54 ` [PATCH 4/5] tests: use consistent bandwidth/downtime limits in migration tests Daniel P. Berrangé
2022-06-28 14:16   ` Dr. David Alan Gilbert
2022-06-28 10:54 ` [RFC PATCH 5/5] tests: stop skipping migration test on s390x/ppc64 Daniel P. Berrangé
2022-06-28 13:18   ` Thomas Huth
2022-07-05  8:06   ` Thomas Huth
2022-07-05  8:09     ` Daniel P. Berrangé
2022-07-05  8:38       ` Dr. David Alan Gilbert
2022-06-28 13:19 ` [PATCH 0/5] tests: improve reliability of migration test Thomas Huth

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20220628105434.295905-2-berrange@redhat.com \
    --to=berrange@redhat.com \
    --cc=cohuck@redhat.com \
    --cc=dgilbert@redhat.com \
    --cc=lvivier@redhat.com \
    --cc=pbonzini@redhat.com \
    --cc=qemu-devel@nongnu.org \
    --cc=qemu-s390x@nongnu.org \
    --cc=quintela@redhat.com \
    --cc=thuth@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.