From: Olaf Hering <olaf@aepfle.de>
To: xen-devel@lists.xenproject.org
Cc: Olaf Hering <olaf@aepfle.de>, Ian Jackson <iwj@xenproject.org>,
Wei Liu <wl@xen.org>, Anthony PERARD <anthony.perard@citrix.com>,
Juergen Gross <jgross@suse.com>
Subject: [PATCH v20210713 27/31] tools: add --min_remaining to libxl_domain_suspend
Date: Tue, 13 Jul 2021 20:06:01 +0200 [thread overview]
Message-ID: <20210713180605.12096-28-olaf@aepfle.de> (raw)
In-Reply-To: <20210713180605.12096-1-olaf@aepfle.de>
The decision to stop+move a domU to the new host must be based on two factors:
- the available network bandwidth for the migration stream
- the maximum time a workload within a domU can be savely suspended
Both values define how many dirty pages a workload may produce prior the
final stop+move.
The default value of 50 pages is much too low with todays network bandwidths.
On an idle 1GiB link these 200K will be transferred within ~2ms.
Give the admin a knob to adjust the point when the final stop+move will
be done, so he can base this decision on his own needs.
This patch adjusts xl(1) and the libxl API.
External users check LIBXL_HAVE_DOMAIN_SUSPEND_PROPS for the availibility
of the new .min_remaining property.
Signed-off-by: Olaf Hering <olaf@aepfle.de>
---
docs/man/xl.1.pod.in | 8 ++++++++
tools/include/libxl.h | 1 +
tools/libs/light/libxl_dom_save.c | 2 +-
tools/libs/light/libxl_domain.c | 1 +
tools/libs/light/libxl_internal.h | 1 +
tools/xl/xl_cmdtable.c | 23 ++++++++++++-----------
tools/xl/xl_migrate.c | 9 ++++++++-
7 files changed, 32 insertions(+), 13 deletions(-)
diff --git a/docs/man/xl.1.pod.in b/docs/man/xl.1.pod.in
index 9b104720d3..a40f1d4ba0 100644
--- a/docs/man/xl.1.pod.in
+++ b/docs/man/xl.1.pod.in
@@ -500,6 +500,14 @@ possible to use this option for a 'localhost' migration.
Number of copy iterations before final suspend+move (default: 5)
+=item B<--min_remaing> I<pages>
+
+Number of remaining dirty pages. If the number of dirty pages drops that
+low, the guest is suspended and the domU will finally be moved to I<host>.
+
+This allows the host admin to control for how long the domU will likely
+be suspended during transit.
+
=back
=item B<remus> [I<OPTIONS>] I<domain-id> I<host>
diff --git a/tools/include/libxl.h b/tools/include/libxl.h
index bf77da0524..28d70b1078 100644
--- a/tools/include/libxl.h
+++ b/tools/include/libxl.h
@@ -1715,6 +1715,7 @@ static inline int libxl_retrieve_domain_configuration_0x041200(
typedef struct {
uint32_t flags; /* LIBXL_SUSPEND_* */
uint32_t max_iters;
+ uint32_t min_remaining;
} libxl_domain_suspend_props;
#define LIBXL_SUSPEND_DEBUG 1
#define LIBXL_SUSPEND_LIVE 2
diff --git a/tools/libs/light/libxl_dom_save.c b/tools/libs/light/libxl_dom_save.c
index 106c08a199..14e197c581 100644
--- a/tools/libs/light/libxl_dom_save.c
+++ b/tools/libs/light/libxl_dom_save.c
@@ -381,7 +381,7 @@ static int libxl__domain_save_precopy_policy(struct precopy_stats stats, void *u
LOGD(DEBUG, shs->domid, "iteration %u dirty_count %ld total_written %lu",
stats.iteration, stats.dirty_count, stats.total_written);
- if (stats.dirty_count >= 0 && stats.dirty_count < LIBXL_XGS_POLICY_TARGET_DIRTY_COUNT)
+ if (stats.dirty_count >= 0 && stats.dirty_count < dss->min_remaining)
goto stop_copy;
if (stats.iteration >= dss->max_iters)
goto stop_copy;
diff --git a/tools/libs/light/libxl_domain.c b/tools/libs/light/libxl_domain.c
index 9f98cd7f2b..06ca7a7df6 100644
--- a/tools/libs/light/libxl_domain.c
+++ b/tools/libs/light/libxl_domain.c
@@ -528,6 +528,7 @@ int libxl_domain_suspend(libxl_ctx *ctx, uint32_t domid, int fd,
dss->fd = fd;
dss->type = type;
dss->max_iters = props->max_iters ?: LIBXL_XGS_POLICY_MAX_ITERATIONS;
+ dss->min_remaining = props->min_remaining ?: LIBXL_XGS_POLICY_TARGET_DIRTY_COUNT;
dss->live = props->flags & LIBXL_SUSPEND_LIVE;
dss->debug = props->flags & LIBXL_SUSPEND_DEBUG;
dss->checkpointed_stream = LIBXL_CHECKPOINTED_STREAM_NONE;
diff --git a/tools/libs/light/libxl_internal.h b/tools/libs/light/libxl_internal.h
index cfb83d80d7..c503fe156c 100644
--- a/tools/libs/light/libxl_internal.h
+++ b/tools/libs/light/libxl_internal.h
@@ -3649,6 +3649,7 @@ struct libxl__domain_save_state {
int debug;
int checkpointed_stream;
uint32_t max_iters;
+ uint32_t min_remaining;
const libxl_domain_remus_info *remus;
/* private */
int rc;
diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c
index 678b236d76..e10b580261 100644
--- a/tools/xl/xl_cmdtable.c
+++ b/tools/xl/xl_cmdtable.c
@@ -165,17 +165,18 @@ const struct cmd_spec cmd_table[] = {
&main_migrate, 0, 1,
"Migrate a domain to another host",
"[options] <Domain> <host>",
- "-h Print this help.\n"
- "-C <config> Send <config> instead of config file from creation.\n"
- "-s <sshcommand> Use <sshcommand> instead of ssh. String will be passed\n"
- " to sh. If empty, run <host> instead of ssh <host> xl\n"
- " migrate-receive [-d -e]\n"
- "-e Do not wait in the background (on <host>) for the death\n"
- " of the domain.\n"
- "--debug Enable verification mode.\n"
- "-p Do not unpause domain after migrating it.\n"
- "-D Preserve the domain id\n"
- "--max_iters N Number of copy iterations before final stop+move"
+ "-h Print this help.\n"
+ "-C <config> Send <config> instead of config file from creation.\n"
+ "-s <sshcommand> Use <sshcommand> instead of ssh. String will be passed\n"
+ " to sh. If empty, run <host> instead of ssh <host> xl\n"
+ " migrate-receive [-d -e]\n"
+ "-e Do not wait in the background (on <host>) for the death\n"
+ " of the domain.\n"
+ "--debug Enable verification mode.\n"
+ "-p Do not unpause domain after migrating it.\n"
+ "-D Preserve the domain id\n"
+ "--max_iters N Number of copy iterations before final stop+move\n"
+ "--min_remaining N Number of remaining dirty pages before final stop+move"
},
{ "restore",
&main_restore, 0, 1,
diff --git a/tools/xl/xl_migrate.c b/tools/xl/xl_migrate.c
index af117d4d56..14feb2b7ec 100644
--- a/tools/xl/xl_migrate.c
+++ b/tools/xl/xl_migrate.c
@@ -179,6 +179,7 @@ static void migrate_do_preamble(int send_fd, int recv_fd, pid_t child,
static void migrate_domain(uint32_t domid, int preserve_domid,
const char *rune, int debug,
uint32_t max_iters,
+ uint32_t min_remaining,
const char *override_config_file)
{
pid_t child = -1;
@@ -191,6 +192,7 @@ static void migrate_domain(uint32_t domid, int preserve_domid,
libxl_domain_suspend_props props = {
.flags = LIBXL_SUSPEND_LIVE,
.max_iters = max_iters,
+ .min_remaining = min_remaining,
};
save_domain_core_begin(domid, preserve_domid, override_config_file,
@@ -545,9 +547,11 @@ int main_migrate(int argc, char **argv)
int opt, daemonize = 1, monitor = 1, debug = 0, pause_after_migration = 0;
int preserve_domid = 0;
uint32_t max_iters = 0;
+ uint32_t min_remaining = 0;
static struct option opts[] = {
{"debug", 0, 0, 0x100},
{"max_iters", 1, 0, 0x101},
+ {"min_remaining", 1, 0, 0x102},
{"live", 0, 0, 0x200},
COMMON_LONG_OPTS
};
@@ -578,6 +582,9 @@ int main_migrate(int argc, char **argv)
case 0x101: /* --max_iters */
max_iters = atoi(optarg);
break;
+ case 0x102: /* --min_remaining */
+ min_remaining = atoi(optarg);
+ break;
case 0x200: /* --live */
/* ignored for compatibility with xm */
break;
@@ -613,7 +620,7 @@ int main_migrate(int argc, char **argv)
}
migrate_domain(domid, preserve_domid, rune, debug,
- max_iters, config_filename);
+ max_iters, min_remaining, config_filename);
return EXIT_SUCCESS;
}
next prev parent reply other threads:[~2021-07-13 18:17 UTC|newest]
Thread overview: 32+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-07-13 18:05 [PATCH v20210713 00/31] leftover from 2020 Olaf Hering
2021-07-13 18:05 ` [PATCH v20210713 01/31] tools: fix make rpmball Olaf Hering
2021-07-13 18:05 ` [PATCH v20210713 02/31] hotplug/Linux: fix starting of xenstored with restarting systemd Olaf Hering
2021-07-13 18:05 ` [PATCH v20210713 03/31] tools: add API to work with sevaral bits at once Olaf Hering
2021-07-13 18:05 ` [PATCH v20210713 04/31] xl: fix description of migrate --debug Olaf Hering
2021-07-13 18:05 ` [PATCH v20210713 05/31] tools: add readv_exact to libxenctrl Olaf Hering
2021-07-13 18:05 ` [PATCH v20210713 06/31] tools: show migration transfer rate in send_dirty_pages Olaf Hering
2021-07-13 18:05 ` [PATCH v20210713 07/31] tools: save: preallocate mfns array Olaf Hering
2021-07-13 18:05 ` [PATCH v20210713 08/31] tools: save: preallocate types array Olaf Hering
2021-07-13 18:05 ` [PATCH v20210713 09/31] tools: save: preallocate errors array Olaf Hering
2021-07-13 18:05 ` [PATCH v20210713 10/31] tools: save: preallocate iov array Olaf Hering
2021-07-13 18:05 ` [PATCH v20210713 11/31] tools: save: preallocate rec_pfns array Olaf Hering
2021-07-13 18:05 ` [PATCH v20210713 12/31] tools: save: preallocate guest_data array Olaf Hering
2021-07-13 18:05 ` [PATCH v20210713 13/31] tools: save: preallocate local_pages array Olaf Hering
2021-07-13 18:05 ` [PATCH v20210713 14/31] tools: restore: preallocate pfns array Olaf Hering
2021-07-13 18:05 ` [PATCH v20210713 15/31] tools: restore: preallocate types array Olaf Hering
2021-07-13 18:05 ` [PATCH v20210713 16/31] tools: restore: preallocate mfns array Olaf Hering
2021-07-13 18:05 ` [PATCH v20210713 17/31] tools: restore: preallocate map_errs array Olaf Hering
2021-07-13 18:05 ` [PATCH v20210713 18/31] tools: restore: preallocate populate_pfns pfns array Olaf Hering
2021-07-13 18:05 ` [PATCH v20210713 19/31] tools: restore: preallocate populate_pfns mfns array Olaf Hering
2021-07-13 18:05 ` [PATCH v20210713 20/31] tools: restore: split record processing Olaf Hering
2021-07-13 18:05 ` [PATCH v20210713 21/31] tools: restore: split handle_page_data Olaf Hering
2021-07-13 18:05 ` [PATCH v20210713 22/31] tools: restore: write data directly into guest Olaf Hering
2021-07-13 18:05 ` [PATCH v20210713 23/31] tools: recognize LIBXL_API_VERSION for 4.16 Olaf Hering
2021-07-13 18:05 ` [PATCH v20210713 24/31] tools: adjust libxl_domain_suspend to receive a struct props Olaf Hering
2021-07-13 18:05 ` [PATCH v20210713 25/31] tools: add callback to libxl for precopy_policy and precopy_stats Olaf Hering
2021-07-13 18:06 ` [PATCH v20210713 26/31] tools: add --max_iters to libxl_domain_suspend Olaf Hering
2021-07-13 18:06 ` Olaf Hering [this message]
2021-07-13 18:06 ` [PATCH v20210713 28/31] tools: add --abort_if_busy " Olaf Hering
2021-07-13 18:06 ` [PATCH v20210713 29/31] tools: add API for expandable bitmaps Olaf Hering
2021-07-13 18:06 ` [PATCH v20210713 30/31] tools: use xg_sr_bitmap for populated_pfns Olaf Hering
2021-07-13 18:06 ` [PATCH v20210713 31/31] tools: use superpages during restore of HVM guest Olaf Hering
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210713180605.12096-28-olaf@aepfle.de \
--to=olaf@aepfle.de \
--cc=anthony.perard@citrix.com \
--cc=iwj@xenproject.org \
--cc=jgross@suse.com \
--cc=wl@xen.org \
--cc=xen-devel@lists.xenproject.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).