* [Patch v2 1/6] migration/postcopy: reduce memset when it is zero page and matches_target_page_size
2019-11-07 12:39 [Patch v2 0/6] migration/postcopy: enable compress during postcopy Wei Yang
@ 2019-11-07 12:39 ` Wei Yang
2020-01-09 9:39 ` Juan Quintela
2019-11-07 12:39 ` [Patch v2 2/6] migration/postcopy: wait for decompress thread in precopy Wei Yang
` (5 subsequent siblings)
6 siblings, 1 reply; 18+ messages in thread
From: Wei Yang @ 2019-11-07 12:39 UTC (permalink / raw)
To: quintela, dgilbert; +Cc: qemu-devel, Wei Yang
In this case, page_buffer content would not be used.
Skip this to save some time.
Signed-off-by: Wei Yang <richardw.yang@linux.intel.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
migration/ram.c | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/migration/ram.c b/migration/ram.c
index 99a98b2da4..7938a643d9 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -4091,7 +4091,13 @@ static int ram_load_postcopy(QEMUFile *f)
switch (flags & ~RAM_SAVE_FLAG_CONTINUE) {
case RAM_SAVE_FLAG_ZERO:
ch = qemu_get_byte(f);
- memset(page_buffer, ch, TARGET_PAGE_SIZE);
+ /*
+ * Can skip to set page_buffer when
+ * this is a zero page and (block->page_size == TARGET_PAGE_SIZE).
+ */
+ if (ch || !matches_target_page_size) {
+ memset(page_buffer, ch, TARGET_PAGE_SIZE);
+ }
if (ch) {
all_zero = false;
}
--
2.17.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [Patch v2 2/6] migration/postcopy: wait for decompress thread in precopy
2019-11-07 12:39 [Patch v2 0/6] migration/postcopy: enable compress during postcopy Wei Yang
2019-11-07 12:39 ` [Patch v2 1/6] migration/postcopy: reduce memset when it is zero page and matches_target_page_size Wei Yang
@ 2019-11-07 12:39 ` Wei Yang
2020-01-09 9:40 ` Juan Quintela
2019-11-07 12:39 ` [Patch v2 3/6] migration/postcopy: count target page number to decide the place_needed Wei Yang
` (4 subsequent siblings)
6 siblings, 1 reply; 18+ messages in thread
From: Wei Yang @ 2019-11-07 12:39 UTC (permalink / raw)
To: quintela, dgilbert; +Cc: qemu-devel, Wei Yang
Compress is not supported with postcopy, it is safe to wait for
decompress thread just in precopy.
This is a preparation for later patch.
Signed-off-by: Wei Yang <richardw.yang@linux.intel.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
migration/ram.c | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/migration/ram.c b/migration/ram.c
index 7938a643d9..f59e3fe197 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -4375,6 +4375,7 @@ static int ram_load_precopy(QEMUFile *f)
}
}
+ ret |= wait_for_decompress_done();
return ret;
}
@@ -4406,8 +4407,6 @@ static int ram_load(QEMUFile *f, void *opaque, int version_id)
} else {
ret = ram_load_precopy(f);
}
-
- ret |= wait_for_decompress_done();
}
trace_ram_load_complete(ret, seq_iter);
--
2.17.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [Patch v2 3/6] migration/postcopy: count target page number to decide the place_needed
2019-11-07 12:39 [Patch v2 0/6] migration/postcopy: enable compress during postcopy Wei Yang
2019-11-07 12:39 ` [Patch v2 1/6] migration/postcopy: reduce memset when it is zero page and matches_target_page_size Wei Yang
2019-11-07 12:39 ` [Patch v2 2/6] migration/postcopy: wait for decompress thread in precopy Wei Yang
@ 2019-11-07 12:39 ` Wei Yang
2020-01-09 9:41 ` Juan Quintela
2019-11-07 12:39 ` [Patch v2 4/6] migration/postcopy: set all_zero to true on the first target page Wei Yang
` (3 subsequent siblings)
6 siblings, 1 reply; 18+ messages in thread
From: Wei Yang @ 2019-11-07 12:39 UTC (permalink / raw)
To: quintela, dgilbert; +Cc: qemu-devel, Wei Yang
In postcopy, it requires to place whole host page instead of target
page.
Currently, it relies on the page offset to decide whether this is the
last target page. We also can count the target page number during the
iteration. When the number of target page equals
(host page size / target page size), this means it is the last target
page in the host page.
This is a preparation for non-ordered target page transmission.
Signed-off-by: Wei Yang <richardw.yang@linux.intel.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
migration/ram.c | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/migration/ram.c b/migration/ram.c
index f59e3fe197..5c05376d8d 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -4017,6 +4017,7 @@ static int ram_load_postcopy(QEMUFile *f)
void *postcopy_host_page = mis->postcopy_tmp_page;
void *last_host = NULL;
bool all_zero = false;
+ int target_pages = 0;
while (!ret && !(flags & RAM_SAVE_FLAG_EOS)) {
ram_addr_t addr;
@@ -4051,6 +4052,7 @@ static int ram_load_postcopy(QEMUFile *f)
ret = -EINVAL;
break;
}
+ target_pages++;
matches_target_page_size = block->page_size == TARGET_PAGE_SIZE;
/*
* Postcopy requires that we place whole host pages atomically;
@@ -4082,8 +4084,10 @@ static int ram_load_postcopy(QEMUFile *f)
* If it's the last part of a host page then we place the host
* page
*/
- place_needed = (((uintptr_t)host + TARGET_PAGE_SIZE) &
- (block->page_size - 1)) == 0;
+ if (target_pages == (block->page_size / TARGET_PAGE_SIZE)) {
+ place_needed = true;
+ target_pages = 0;
+ }
place_source = postcopy_host_page;
}
last_host = host;
--
2.17.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
* Re: [Patch v2 3/6] migration/postcopy: count target page number to decide the place_needed
2019-11-07 12:39 ` [Patch v2 3/6] migration/postcopy: count target page number to decide the place_needed Wei Yang
@ 2020-01-09 9:41 ` Juan Quintela
0 siblings, 0 replies; 18+ messages in thread
From: Juan Quintela @ 2020-01-09 9:41 UTC (permalink / raw)
To: Wei Yang; +Cc: dgilbert, qemu-devel
Wei Yang <richardw.yang@linux.intel.com> wrote:
> In postcopy, it requires to place whole host page instead of target
> page.
>
> Currently, it relies on the page offset to decide whether this is the
> last target page. We also can count the target page number during the
> iteration. When the number of target page equals
> (host page size / target page size), this means it is the last target
> page in the host page.
>
> This is a preparation for non-ordered target page transmission.
>
> Signed-off-by: Wei Yang <richardw.yang@linux.intel.com>
> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
^ permalink raw reply [flat|nested] 18+ messages in thread
* [Patch v2 4/6] migration/postcopy: set all_zero to true on the first target page
2019-11-07 12:39 [Patch v2 0/6] migration/postcopy: enable compress during postcopy Wei Yang
` (2 preceding siblings ...)
2019-11-07 12:39 ` [Patch v2 3/6] migration/postcopy: count target page number to decide the place_needed Wei Yang
@ 2019-11-07 12:39 ` Wei Yang
2020-01-09 9:41 ` Juan Quintela
2019-11-07 12:39 ` [Patch v2 5/6] migration/postcopy: enable random order target page arrival Wei Yang
` (2 subsequent siblings)
6 siblings, 1 reply; 18+ messages in thread
From: Wei Yang @ 2019-11-07 12:39 UTC (permalink / raw)
To: quintela, dgilbert; +Cc: qemu-devel, Wei Yang
For the first target page, all_zero is set to true for this round check.
After target_pages introduced, we could leverage this variable instead
of checking the address offset.
Signed-off-by: Wei Yang <richardw.yang@linux.intel.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
migration/ram.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/migration/ram.c b/migration/ram.c
index 5c05376d8d..b5759793a9 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -4067,7 +4067,7 @@ static int ram_load_postcopy(QEMUFile *f)
page_buffer = postcopy_host_page +
((uintptr_t)host & (block->page_size - 1));
/* If all TP are zero then we can optimise the place */
- if (!((uintptr_t)host & (block->page_size - 1))) {
+ if (target_pages == 1) {
all_zero = true;
} else {
/* not the 1st TP within the HP */
--
2.17.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [Patch v2 5/6] migration/postcopy: enable random order target page arrival
2019-11-07 12:39 [Patch v2 0/6] migration/postcopy: enable compress during postcopy Wei Yang
` (3 preceding siblings ...)
2019-11-07 12:39 ` [Patch v2 4/6] migration/postcopy: set all_zero to true on the first target page Wei Yang
@ 2019-11-07 12:39 ` Wei Yang
2019-11-07 14:30 ` Dr. David Alan Gilbert
2020-01-09 9:42 ` Juan Quintela
2019-11-07 12:39 ` [Patch v2 6/6] migration/postcopy: enable compress during postcopy Wei Yang
2019-12-16 2:35 ` [Patch v2 0/6] " Wei Yang
6 siblings, 2 replies; 18+ messages in thread
From: Wei Yang @ 2019-11-07 12:39 UTC (permalink / raw)
To: quintela, dgilbert; +Cc: qemu-devel, Wei Yang
After using number of target page received to track one host page, we
could have the capability to handle random order target page arrival in
one host page.
This is a preparation for enabling compress during postcopy.
Signed-off-by: Wei Yang <richardw.yang@linux.intel.com>
---
v2:
* use uintptr_t to calculate place_dest
* check target pages belongs to the same host page
---
migration/ram.c | 18 ++++++++++--------
1 file changed, 10 insertions(+), 8 deletions(-)
diff --git a/migration/ram.c b/migration/ram.c
index b5759793a9..666ad69284 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -4015,7 +4015,7 @@ static int ram_load_postcopy(QEMUFile *f)
MigrationIncomingState *mis = migration_incoming_get_current();
/* Temporary page that is later 'placed' */
void *postcopy_host_page = mis->postcopy_tmp_page;
- void *last_host = NULL;
+ void *this_host = NULL;
bool all_zero = false;
int target_pages = 0;
@@ -4062,24 +4062,26 @@ static int ram_load_postcopy(QEMUFile *f)
* that's moved into place later.
* The migration protocol uses, possibly smaller, target-pages
* however the source ensures it always sends all the components
- * of a host page in order.
+ * of a host page in one chunk.
*/
page_buffer = postcopy_host_page +
((uintptr_t)host & (block->page_size - 1));
/* If all TP are zero then we can optimise the place */
if (target_pages == 1) {
all_zero = true;
+ this_host = (void *)QEMU_ALIGN_DOWN((uintptr_t)host,
+ block->page_size);
} else {
/* not the 1st TP within the HP */
- if (host != (last_host + TARGET_PAGE_SIZE)) {
- error_report("Non-sequential target page %p/%p",
- host, last_host);
+ if (QEMU_ALIGN_DOWN((uintptr_t)host, block->page_size) !=
+ (uintptr_t)this_host) {
+ error_report("Non-same host page %p/%p",
+ host, this_host);
ret = -EINVAL;
break;
}
}
-
/*
* If it's the last part of a host page then we place the host
* page
@@ -4090,7 +4092,6 @@ static int ram_load_postcopy(QEMUFile *f)
}
place_source = postcopy_host_page;
}
- last_host = host;
switch (flags & ~RAM_SAVE_FLAG_CONTINUE) {
case RAM_SAVE_FLAG_ZERO:
@@ -4143,7 +4144,8 @@ static int ram_load_postcopy(QEMUFile *f)
if (!ret && place_needed) {
/* This gets called at the last target page in the host page */
- void *place_dest = host + TARGET_PAGE_SIZE - block->page_size;
+ void *place_dest = (void *)QEMU_ALIGN_DOWN((uintptr_t)host,
+ block->page_size);
if (all_zero) {
ret = postcopy_place_page_zero(mis, place_dest,
--
2.17.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
* Re: [Patch v2 5/6] migration/postcopy: enable random order target page arrival
2019-11-07 12:39 ` [Patch v2 5/6] migration/postcopy: enable random order target page arrival Wei Yang
@ 2019-11-07 14:30 ` Dr. David Alan Gilbert
2019-11-08 0:40 ` Wei Yang
2020-01-09 9:42 ` Juan Quintela
1 sibling, 1 reply; 18+ messages in thread
From: Dr. David Alan Gilbert @ 2019-11-07 14:30 UTC (permalink / raw)
To: Wei Yang; +Cc: qemu-devel, quintela
* Wei Yang (richardw.yang@linux.intel.com) wrote:
> After using number of target page received to track one host page, we
> could have the capability to handle random order target page arrival in
> one host page.
>
> This is a preparation for enabling compress during postcopy.
>
> Signed-off-by: Wei Yang <richardw.yang@linux.intel.com>
Yep, that's better
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
> ---
> v2:
> * use uintptr_t to calculate place_dest
> * check target pages belongs to the same host page
> ---
> migration/ram.c | 18 ++++++++++--------
> 1 file changed, 10 insertions(+), 8 deletions(-)
>
> diff --git a/migration/ram.c b/migration/ram.c
> index b5759793a9..666ad69284 100644
> --- a/migration/ram.c
> +++ b/migration/ram.c
> @@ -4015,7 +4015,7 @@ static int ram_load_postcopy(QEMUFile *f)
> MigrationIncomingState *mis = migration_incoming_get_current();
> /* Temporary page that is later 'placed' */
> void *postcopy_host_page = mis->postcopy_tmp_page;
> - void *last_host = NULL;
> + void *this_host = NULL;
> bool all_zero = false;
> int target_pages = 0;
>
> @@ -4062,24 +4062,26 @@ static int ram_load_postcopy(QEMUFile *f)
> * that's moved into place later.
> * The migration protocol uses, possibly smaller, target-pages
> * however the source ensures it always sends all the components
> - * of a host page in order.
> + * of a host page in one chunk.
> */
> page_buffer = postcopy_host_page +
> ((uintptr_t)host & (block->page_size - 1));
> /* If all TP are zero then we can optimise the place */
> if (target_pages == 1) {
> all_zero = true;
> + this_host = (void *)QEMU_ALIGN_DOWN((uintptr_t)host,
> + block->page_size);
> } else {
> /* not the 1st TP within the HP */
> - if (host != (last_host + TARGET_PAGE_SIZE)) {
> - error_report("Non-sequential target page %p/%p",
> - host, last_host);
> + if (QEMU_ALIGN_DOWN((uintptr_t)host, block->page_size) !=
> + (uintptr_t)this_host) {
> + error_report("Non-same host page %p/%p",
> + host, this_host);
> ret = -EINVAL;
> break;
> }
> }
>
> -
> /*
> * If it's the last part of a host page then we place the host
> * page
> @@ -4090,7 +4092,6 @@ static int ram_load_postcopy(QEMUFile *f)
> }
> place_source = postcopy_host_page;
> }
> - last_host = host;
>
> switch (flags & ~RAM_SAVE_FLAG_CONTINUE) {
> case RAM_SAVE_FLAG_ZERO:
> @@ -4143,7 +4144,8 @@ static int ram_load_postcopy(QEMUFile *f)
>
> if (!ret && place_needed) {
> /* This gets called at the last target page in the host page */
> - void *place_dest = host + TARGET_PAGE_SIZE - block->page_size;
> + void *place_dest = (void *)QEMU_ALIGN_DOWN((uintptr_t)host,
> + block->page_size);
>
> if (all_zero) {
> ret = postcopy_place_page_zero(mis, place_dest,
> --
> 2.17.1
>
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Patch v2 5/6] migration/postcopy: enable random order target page arrival
2019-11-07 14:30 ` Dr. David Alan Gilbert
@ 2019-11-08 0:40 ` Wei Yang
0 siblings, 0 replies; 18+ messages in thread
From: Wei Yang @ 2019-11-08 0:40 UTC (permalink / raw)
To: Dr. David Alan Gilbert; +Cc: qemu-devel, Wei Yang, quintela
On Thu, Nov 07, 2019 at 02:30:25PM +0000, Dr. David Alan Gilbert wrote:
>* Wei Yang (richardw.yang@linux.intel.com) wrote:
>> After using number of target page received to track one host page, we
>> could have the capability to handle random order target page arrival in
>> one host page.
>>
>> This is a preparation for enabling compress during postcopy.
>>
>> Signed-off-by: Wei Yang <richardw.yang@linux.intel.com>
>
>Yep, that's better
>
>Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
>
Thanks for your suggestion :-)
--
Wei Yang
Help you, Help me
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Patch v2 5/6] migration/postcopy: enable random order target page arrival
2019-11-07 12:39 ` [Patch v2 5/6] migration/postcopy: enable random order target page arrival Wei Yang
2019-11-07 14:30 ` Dr. David Alan Gilbert
@ 2020-01-09 9:42 ` Juan Quintela
1 sibling, 0 replies; 18+ messages in thread
From: Juan Quintela @ 2020-01-09 9:42 UTC (permalink / raw)
To: Wei Yang; +Cc: dgilbert, qemu-devel
Wei Yang <richardw.yang@linux.intel.com> wrote:
> After using number of target page received to track one host page, we
> could have the capability to handle random order target page arrival in
> one host page.
>
> This is a preparation for enabling compress during postcopy.
>
> Signed-off-by: Wei Yang <richardw.yang@linux.intel.com>
>
> ---
> v2:
> * use uintptr_t to calculate place_dest
> * check target pages belongs to the same host page
Reviewed-by: Juan Quintela <quintela@redhat.com>
^ permalink raw reply [flat|nested] 18+ messages in thread
* [Patch v2 6/6] migration/postcopy: enable compress during postcopy
2019-11-07 12:39 [Patch v2 0/6] migration/postcopy: enable compress during postcopy Wei Yang
` (4 preceding siblings ...)
2019-11-07 12:39 ` [Patch v2 5/6] migration/postcopy: enable random order target page arrival Wei Yang
@ 2019-11-07 12:39 ` Wei Yang
2020-01-09 9:44 ` Juan Quintela
2019-12-16 2:35 ` [Patch v2 0/6] " Wei Yang
6 siblings, 1 reply; 18+ messages in thread
From: Wei Yang @ 2019-11-07 12:39 UTC (permalink / raw)
To: quintela, dgilbert; +Cc: qemu-devel, Wei Yang
postcopy requires to place a whole host page, while migration thread
migrate memory in target page size. This makes postcopy need to collect
all target pages in one host page before placing via userfaultfd.
To enable compress during postcopy, there are two problems to solve:
1. Random order for target page arrival
2. Target pages in one host page arrives without interrupt by target
page from other host page
The first one is handled by previous cleanup patch.
This patch handles the second one by:
1. Flush compress thread for each host page
2. Wait for decompress thread for before placing host page
Signed-off-by: Wei Yang <richardw.yang@linux.intel.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
migration/migration.c | 11 -----------
migration/ram.c | 28 +++++++++++++++++++++++++++-
2 files changed, 27 insertions(+), 12 deletions(-)
diff --git a/migration/migration.c b/migration/migration.c
index 354ad072fa..3c926a3ae3 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -1005,17 +1005,6 @@ static bool migrate_caps_check(bool *cap_list,
#endif
if (cap_list[MIGRATION_CAPABILITY_POSTCOPY_RAM]) {
- if (cap_list[MIGRATION_CAPABILITY_COMPRESS]) {
- /* The decompression threads asynchronously write into RAM
- * rather than use the atomic copies needed to avoid
- * userfaulting. It should be possible to fix the decompression
- * threads for compatibility in future.
- */
- error_setg(errp, "Postcopy is not currently compatible "
- "with compression");
- return false;
- }
-
/* This check is reasonably expensive, so only when it's being
* set the first time, also it's only the destination that needs
* special support.
diff --git a/migration/ram.c b/migration/ram.c
index 666ad69284..0d53786311 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -3449,6 +3449,14 @@ static int ram_save_iterate(QEMUFile *f, void *opaque)
rs->target_page_count += pages;
+ /*
+ * During postcopy, it is necessary to make sure one whole host
+ * page is sent in one chunk.
+ */
+ if (migrate_postcopy_ram()) {
+ flush_compressed_data(rs);
+ }
+
/*
* we want to check in the 1st loop, just in case it was the 1st
* time and we had to sync the dirty bitmap.
@@ -4026,6 +4034,7 @@ static int ram_load_postcopy(QEMUFile *f)
void *place_source = NULL;
RAMBlock *block = NULL;
uint8_t ch;
+ int len;
addr = qemu_get_be64(f);
@@ -4043,7 +4052,8 @@ static int ram_load_postcopy(QEMUFile *f)
trace_ram_load_postcopy_loop((uint64_t)addr, flags);
place_needed = false;
- if (flags & (RAM_SAVE_FLAG_ZERO | RAM_SAVE_FLAG_PAGE)) {
+ if (flags & (RAM_SAVE_FLAG_ZERO | RAM_SAVE_FLAG_PAGE |
+ RAM_SAVE_FLAG_COMPRESS_PAGE)) {
block = ram_block_from_stream(f, flags);
host = host_from_ram_block_offset(block, addr);
@@ -4126,6 +4136,17 @@ static int ram_load_postcopy(QEMUFile *f)
TARGET_PAGE_SIZE);
}
break;
+ case RAM_SAVE_FLAG_COMPRESS_PAGE:
+ all_zero = false;
+ len = qemu_get_be32(f);
+ if (len < 0 || len > compressBound(TARGET_PAGE_SIZE)) {
+ error_report("Invalid compressed data length: %d", len);
+ ret = -EINVAL;
+ break;
+ }
+ decompress_data_with_multi_threads(f, page_buffer, len);
+ break;
+
case RAM_SAVE_FLAG_EOS:
/* normal exit */
multifd_recv_sync_main();
@@ -4137,6 +4158,11 @@ static int ram_load_postcopy(QEMUFile *f)
break;
}
+ /* Got the whole host page, wait for decompress before placing. */
+ if (place_needed) {
+ ret |= wait_for_decompress_done();
+ }
+
/* Detect for any possible file errors */
if (!ret && qemu_file_get_error(f)) {
ret = qemu_file_get_error(f);
--
2.17.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
* Re: [Patch v2 6/6] migration/postcopy: enable compress during postcopy
2019-11-07 12:39 ` [Patch v2 6/6] migration/postcopy: enable compress during postcopy Wei Yang
@ 2020-01-09 9:44 ` Juan Quintela
0 siblings, 0 replies; 18+ messages in thread
From: Juan Quintela @ 2020-01-09 9:44 UTC (permalink / raw)
To: Wei Yang; +Cc: dgilbert, qemu-devel
Wei Yang <richardw.yang@linux.intel.com> wrote:
> postcopy requires to place a whole host page, while migration thread
> migrate memory in target page size. This makes postcopy need to collect
> all target pages in one host page before placing via userfaultfd.
>
> To enable compress during postcopy, there are two problems to solve:
>
> 1. Random order for target page arrival
> 2. Target pages in one host page arrives without interrupt by target
> page from other host page
>
> The first one is handled by previous cleanup patch.
>
> This patch handles the second one by:
>
> 1. Flush compress thread for each host page
> 2. Wait for decompress thread for before placing host page
>
> Signed-off-by: Wei Yang <richardw.yang@linux.intel.com>
> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Patch v2 0/6] migration/postcopy: enable compress during postcopy
2019-11-07 12:39 [Patch v2 0/6] migration/postcopy: enable compress during postcopy Wei Yang
` (5 preceding siblings ...)
2019-11-07 12:39 ` [Patch v2 6/6] migration/postcopy: enable compress during postcopy Wei Yang
@ 2019-12-16 2:35 ` Wei Yang
2019-12-18 19:55 ` Dr. David Alan Gilbert
6 siblings, 1 reply; 18+ messages in thread
From: Wei Yang @ 2019-12-16 2:35 UTC (permalink / raw)
To: Wei Yang; +Cc: qemu-devel, dgilbert, quintela
Would this one be picked up in this version?
On Thu, Nov 07, 2019 at 08:39:01PM +0800, Wei Yang wrote:
>This patch set tries enable compress during postcopy.
>
>postcopy requires to place a whole host page, while migration thread migrate
>memory in target page size. This makes postcopy need to collect all target
>pages in one host page before placing via userfaultfd.
>
>To enable compress during postcopy, there are two problems to solve:
>
> 1. Random order for target page arrival
> 2. Target pages in one host page arrives without interrupt by target
> page from other host page
>
>The first one is handled by counting the number of target pages arrived
>instead of the last target page arrived.
>
>The second one is handled by:
>
> 1. Flush compress thread for each host page
> 2. Wait for decompress thread for before placing host page
>
>With the combination of these two changes, compress is enabled during
>postcopy.
>
>---
>v2:
> * use uintptr_t to calculate place_dest
> * check target pages belongs to the same host page
>
>Wei Yang (6):
> migration/postcopy: reduce memset when it is zero page and
> matches_target_page_size
> migration/postcopy: wait for decompress thread in precopy
> migration/postcopy: count target page number to decide the
> place_needed
> migration/postcopy: set all_zero to true on the first target page
> migration/postcopy: enable random order target page arrival
> migration/postcopy: enable compress during postcopy
>
> migration/migration.c | 11 -------
> migration/ram.c | 67 +++++++++++++++++++++++++++++++++----------
> 2 files changed, 52 insertions(+), 26 deletions(-)
>
>--
>2.17.1
--
Wei Yang
Help you, Help me
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Patch v2 0/6] migration/postcopy: enable compress during postcopy
2019-12-16 2:35 ` [Patch v2 0/6] " Wei Yang
@ 2019-12-18 19:55 ` Dr. David Alan Gilbert
2020-01-06 1:29 ` Wei Yang
0 siblings, 1 reply; 18+ messages in thread
From: Dr. David Alan Gilbert @ 2019-12-18 19:55 UTC (permalink / raw)
To: Wei Yang; +Cc: qemu-devel, quintela
* Wei Yang (richardw.yang@linux.intel.com) wrote:
> Would this one be picked up in this version?
I think that one is on Juan's list for the pull he's going to do soon.
Dave
> On Thu, Nov 07, 2019 at 08:39:01PM +0800, Wei Yang wrote:
> >This patch set tries enable compress during postcopy.
> >
> >postcopy requires to place a whole host page, while migration thread migrate
> >memory in target page size. This makes postcopy need to collect all target
> >pages in one host page before placing via userfaultfd.
> >
> >To enable compress during postcopy, there are two problems to solve:
> >
> > 1. Random order for target page arrival
> > 2. Target pages in one host page arrives without interrupt by target
> > page from other host page
> >
> >The first one is handled by counting the number of target pages arrived
> >instead of the last target page arrived.
> >
> >The second one is handled by:
> >
> > 1. Flush compress thread for each host page
> > 2. Wait for decompress thread for before placing host page
> >
> >With the combination of these two changes, compress is enabled during
> >postcopy.
> >
> >---
> >v2:
> > * use uintptr_t to calculate place_dest
> > * check target pages belongs to the same host page
> >
> >Wei Yang (6):
> > migration/postcopy: reduce memset when it is zero page and
> > matches_target_page_size
> > migration/postcopy: wait for decompress thread in precopy
> > migration/postcopy: count target page number to decide the
> > place_needed
> > migration/postcopy: set all_zero to true on the first target page
> > migration/postcopy: enable random order target page arrival
> > migration/postcopy: enable compress during postcopy
> >
> > migration/migration.c | 11 -------
> > migration/ram.c | 67 +++++++++++++++++++++++++++++++++----------
> > 2 files changed, 52 insertions(+), 26 deletions(-)
> >
> >--
> >2.17.1
>
> --
> Wei Yang
> Help you, Help me
>
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Patch v2 0/6] migration/postcopy: enable compress during postcopy
2019-12-18 19:55 ` Dr. David Alan Gilbert
@ 2020-01-06 1:29 ` Wei Yang
0 siblings, 0 replies; 18+ messages in thread
From: Wei Yang @ 2020-01-06 1:29 UTC (permalink / raw)
To: Dr. David Alan Gilbert; +Cc: qemu-devel, Wei Yang, quintela
On Wed, Dec 18, 2019 at 07:55:38PM +0000, Dr. David Alan Gilbert wrote:
>* Wei Yang (richardw.yang@linux.intel.com) wrote:
>> Would this one be picked up in this version?
>
>I think that one is on Juan's list for the pull he's going to do soon.
>
>Dave
>
Happy New Year to all~
May I ask the plan for this patch set?
--
Wei Yang
Help you, Help me
^ permalink raw reply [flat|nested] 18+ messages in thread