Re: [PATCH v3 9/9] tests/qtest: massively speed up migration-test

From: Thomas Huth <thuth@redhat.com>
To: "Daniel P. Berrangé" <berrange@redhat.com>, qemu-devel@nongnu.org
Cc: Laurent Vivier <lvivier@redhat.com>, Peter Xu <peterx@redhat.com>,
	Juan Quintela <quintela@redhat.com>,
	Leonardo Bras <leobras@redhat.com>,
	Paolo Bonzini <pbonzini@redhat.com>
Subject: Re: [PATCH v3 9/9] tests/qtest: massively speed up migration-test
Date: Thu, 1 Jun 2023 12:04:33 +0200	[thread overview]
Message-ID: <ce154c2c-6059-5669-1aff-a2af73a707d6@redhat.com> (raw)
In-Reply-To: <20230531132400.1129576-10-berrange@redhat.com>

On 31/05/2023 15.24, Daniel P. Berrangé wrote:
> The migration test cases that actually exercise live migration want to
> ensure there is a minimum of two iterations of pre-copy, in order to
> exercise the dirty tracking code.
> 
> Historically we've queried the migration status, looking for the
> 'dirty-sync-count' value to increment to track iterations. This was
> not entirely reliable because often all the data would get transferred
> quickly enough that the migration would finish before we wanted it
> to. So we massively dropped the bandwidth and max downtime to
> guarantee non-convergance. This had the unfortunate side effect

convergence

> that every migration took at least 30 seconds to run (100 MB of
> dirty pages / 3 MB/sec).
> 
> This optimization takes a different approach to ensuring that a
> mimimum of two iterations. Rather than waiting for dirty-sync-count

minimum

> to increment, directly look for an indication that the source VM
> has dirtied RAM that has already been transferred.
> 
> On the source VM a magic marker is written just after the 3 MB
> offset. The destination VM is now montiored to detect when the

monitored

...
> @@ -445,6 +459,91 @@ static void migrate_ensure_converge(QTestState *who)
>       migrate_set_parameter_int(who, "downtime-limit", 30 * 1000);
>   }
>   
> +/*
> + * Our goal is to ensure that we run a single full migration
> + * iteration, and also dirty memory, ensuring that at least
> + * one further iteration is required.
> + *
> + * We can't directly synchronize with the start of a migration
> + * so we have to apply some tricks monitoring memory that is
> + * transferred.
> + *
> + * Initially we set the migration bandwidth to an insanely
> + * low value, with tiny max downtime too. This basically
> + * guarantees migration will never complete.
> + *
> + * This will result in a test that is unacceptably slow though,
> + * so we can't let the entire migration pass run at this speed.
> + * Our intent is to let it run just long enough that we can
> + * prove data prior to the marker has been transferred *AND*
> + * also prove this transferred data is dirty again.
> + *
> + * Before migration starts, we write a 64-bit magic marker
> + * into a fixed location in the src VM RAM.
> + *
> + * Then watch dst memory until the marker appears. This is
> + * proof that start_address -> MAGIC_OFFSET_BASE has been
> + * transferred.
> + *
> + * Finally we go back to the source and read a byte just
> + * before the marker untill we see it flip in value. This

until

It's indeed much faster now, thank you very much for tackling this!

Tested-by: Thomas Huth <thuth@redhat.com>