* [Qemu-devel] [PATCH 0/3] migration/postcopy: unsentmap is not necessary @ 2019-08-19 6:18 Wei Yang 2019-08-19 6:18 ` [Qemu-devel] [PATCH 1/3] migration/postcopy: not necessary to do discard when canonicalizing bitmap Wei Yang ` (4 more replies) 0 siblings, 5 replies; 12+ messages in thread From: Wei Yang @ 2019-08-19 6:18 UTC (permalink / raw) To: qemu-devel; +Cc: pbonzini, Wei Yang, dgilbert, quintela Three patches to cleanup postcopy: [1]: split canonicalize bitmap and discard page [2]: remove unsentmap since it is not necessary [3]: cleanup the get_queued_page_not_dirty Wei Yang (3): migration/postcopy: not necessary to do discard when canonicalizing bitmap migration/postcopy: unsentmap is not necessary for postcopy migration: remove sent parameter in get_queued_page_not_dirty include/exec/ram_addr.h | 6 --- migration/ram.c | 94 +++++++---------------------------------- migration/trace-events | 2 +- 3 files changed, 16 insertions(+), 86 deletions(-) -- 2.17.1 ^ permalink raw reply [flat|nested] 12+ messages in thread
* [Qemu-devel] [PATCH 1/3] migration/postcopy: not necessary to do discard when canonicalizing bitmap 2019-08-19 6:18 [Qemu-devel] [PATCH 0/3] migration/postcopy: unsentmap is not necessary Wei Yang @ 2019-08-19 6:18 ` Wei Yang 2019-09-24 10:02 ` Dr. David Alan Gilbert 2019-08-19 6:18 ` [Qemu-devel] [PATCH 2/3] migration/postcopy: unsentmap is not necessary for postcopy Wei Yang ` (3 subsequent siblings) 4 siblings, 1 reply; 12+ messages in thread From: Wei Yang @ 2019-08-19 6:18 UTC (permalink / raw) To: qemu-devel; +Cc: pbonzini, Wei Yang, dgilbert, quintela All pages, either partially sent or partially dirty, will be discarded in postcopy_send_discard_bm_ram(), since we update the unsentmap to be unsentmap = unsentmap | dirty in ram_postcopy_send_discard_bitmap(). This is not necessary to do discard when canonicalizing bitmap. And by doing so, we separate the page discard into two individual steps: * canonicalize bitmap * discard page Signed-off-by: Wei Yang <richardw.yang@linux.intel.com> --- migration/ram.c | 14 +------------- 1 file changed, 1 insertion(+), 13 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 35552c090b..075ddc468c 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -2928,7 +2928,7 @@ static int postcopy_each_ram_send_discard(MigrationState *ms) } /** - * postcopy_chunk_hostpages_pass: canocalize bitmap in hostpages + * postcopy_chunk_hostpages_pass: canonicalize bitmap in hostpages * * Helper for postcopy_chunk_hostpages; it's called twice to * canonicalize the two bitmaps, that are similar, but one is @@ -2991,18 +2991,6 @@ static void postcopy_chunk_hostpages_pass(MigrationState *ms, bool unsent_pass, host_ratio); run_start = QEMU_ALIGN_UP(run_start, host_ratio); - /* Tell the destination to discard this page */ - if (unsent_pass || !test_bit(fixup_start_addr, unsentmap)) { - /* For the unsent_pass we: - * discard partially sent pages - * For the !unsent_pass (dirty) we: - * discard partially dirty pages that were sent - * (any partially sent pages were already discarded - * by the previous unsent_pass) - */ - postcopy_discard_send_range(ms, fixup_start_addr, host_ratio); - } - /* Clean up the bitmap */ for (page = fixup_start_addr; page < fixup_start_addr + host_ratio; page++) { -- 2.17.1 ^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [PATCH 1/3] migration/postcopy: not necessary to do discard when canonicalizing bitmap 2019-08-19 6:18 ` [Qemu-devel] [PATCH 1/3] migration/postcopy: not necessary to do discard when canonicalizing bitmap Wei Yang @ 2019-09-24 10:02 ` Dr. David Alan Gilbert 2019-09-25 0:25 ` Wei Yang 0 siblings, 1 reply; 12+ messages in thread From: Dr. David Alan Gilbert @ 2019-09-24 10:02 UTC (permalink / raw) To: Wei Yang; +Cc: pbonzini, qemu-devel, quintela * Wei Yang (richardw.yang@linux.intel.com) wrote: > All pages, either partially sent or partially dirty, will be discarded in > postcopy_send_discard_bm_ram(), since we update the unsentmap to be > unsentmap = unsentmap | dirty in ram_postcopy_send_discard_bitmap(). > > This is not necessary to do discard when canonicalizing bitmap. And by > doing so, we separate the page discard into two individual steps: > > * canonicalize bitmap > * discard page > > Signed-off-by: Wei Yang <richardw.yang@linux.intel.com> Yes, I think when I originally wrote it, the set of pages that was discarded was different; I think it was actually the set of !unsent & dirty - i.e. only pages that had been sent and then redirtied; it later got reworked to include unsent pages as well - so this lot can be simplified. Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> > --- > migration/ram.c | 14 +------------- > 1 file changed, 1 insertion(+), 13 deletions(-) > > diff --git a/migration/ram.c b/migration/ram.c > index 35552c090b..075ddc468c 100644 > --- a/migration/ram.c > +++ b/migration/ram.c > @@ -2928,7 +2928,7 @@ static int postcopy_each_ram_send_discard(MigrationState *ms) > } > > /** > - * postcopy_chunk_hostpages_pass: canocalize bitmap in hostpages > + * postcopy_chunk_hostpages_pass: canonicalize bitmap in hostpages > * > * Helper for postcopy_chunk_hostpages; it's called twice to > * canonicalize the two bitmaps, that are similar, but one is > @@ -2991,18 +2991,6 @@ static void postcopy_chunk_hostpages_pass(MigrationState *ms, bool unsent_pass, > host_ratio); > run_start = QEMU_ALIGN_UP(run_start, host_ratio); > > - /* Tell the destination to discard this page */ > - if (unsent_pass || !test_bit(fixup_start_addr, unsentmap)) { > - /* For the unsent_pass we: > - * discard partially sent pages > - * For the !unsent_pass (dirty) we: > - * discard partially dirty pages that were sent > - * (any partially sent pages were already discarded > - * by the previous unsent_pass) > - */ > - postcopy_discard_send_range(ms, fixup_start_addr, host_ratio); > - } > - > /* Clean up the bitmap */ > for (page = fixup_start_addr; > page < fixup_start_addr + host_ratio; page++) { > -- > 2.17.1 > -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH 1/3] migration/postcopy: not necessary to do discard when canonicalizing bitmap 2019-09-24 10:02 ` Dr. David Alan Gilbert @ 2019-09-25 0:25 ` Wei Yang 0 siblings, 0 replies; 12+ messages in thread From: Wei Yang @ 2019-09-25 0:25 UTC (permalink / raw) To: Dr. David Alan Gilbert; +Cc: pbonzini, quintela, Wei Yang, qemu-devel On Tue, Sep 24, 2019 at 11:02:08AM +0100, Dr. David Alan Gilbert wrote: >* Wei Yang (richardw.yang@linux.intel.com) wrote: >> All pages, either partially sent or partially dirty, will be discarded in >> postcopy_send_discard_bm_ram(), since we update the unsentmap to be >> unsentmap = unsentmap | dirty in ram_postcopy_send_discard_bitmap(). >> >> This is not necessary to do discard when canonicalizing bitmap. And by >> doing so, we separate the page discard into two individual steps: >> >> * canonicalize bitmap >> * discard page >> >> Signed-off-by: Wei Yang <richardw.yang@linux.intel.com> > >Yes, I think when I originally wrote it, the set of pages that was >discarded was different; I think it was actually the set of >!unsent & dirty - i.e. only pages that had been sent and then redirtied; >it later got reworked to include unsent pages as well - so this lot can >be simplified. > > Thanks for your time :-) > >Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> > >> --- >> migration/ram.c | 14 +------------- >> 1 file changed, 1 insertion(+), 13 deletions(-) >> >> diff --git a/migration/ram.c b/migration/ram.c >> index 35552c090b..075ddc468c 100644 >> --- a/migration/ram.c >> +++ b/migration/ram.c >> @@ -2928,7 +2928,7 @@ static int postcopy_each_ram_send_discard(MigrationState *ms) >> } >> >> /** >> - * postcopy_chunk_hostpages_pass: canocalize bitmap in hostpages >> + * postcopy_chunk_hostpages_pass: canonicalize bitmap in hostpages >> * >> * Helper for postcopy_chunk_hostpages; it's called twice to >> * canonicalize the two bitmaps, that are similar, but one is >> @@ -2991,18 +2991,6 @@ static void postcopy_chunk_hostpages_pass(MigrationState *ms, bool unsent_pass, >> host_ratio); >> run_start = QEMU_ALIGN_UP(run_start, host_ratio); >> >> - /* Tell the destination to discard this page */ >> - if (unsent_pass || !test_bit(fixup_start_addr, unsentmap)) { >> - /* For the unsent_pass we: >> - * discard partially sent pages >> - * For the !unsent_pass (dirty) we: >> - * discard partially dirty pages that were sent >> - * (any partially sent pages were already discarded >> - * by the previous unsent_pass) >> - */ >> - postcopy_discard_send_range(ms, fixup_start_addr, host_ratio); >> - } >> - >> /* Clean up the bitmap */ >> for (page = fixup_start_addr; >> page < fixup_start_addr + host_ratio; page++) { >> -- >> 2.17.1 >> >-- >Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK -- Wei Yang Help you, Help me ^ permalink raw reply [flat|nested] 12+ messages in thread
* [Qemu-devel] [PATCH 2/3] migration/postcopy: unsentmap is not necessary for postcopy 2019-08-19 6:18 [Qemu-devel] [PATCH 0/3] migration/postcopy: unsentmap is not necessary Wei Yang 2019-08-19 6:18 ` [Qemu-devel] [PATCH 1/3] migration/postcopy: not necessary to do discard when canonicalizing bitmap Wei Yang @ 2019-08-19 6:18 ` Wei Yang 2019-09-24 10:18 ` Dr. David Alan Gilbert 2019-08-19 6:18 ` [Qemu-devel] [PATCH 3/3] migration: remove sent parameter in get_queued_page_not_dirty Wei Yang ` (2 subsequent siblings) 4 siblings, 1 reply; 12+ messages in thread From: Wei Yang @ 2019-08-19 6:18 UTC (permalink / raw) To: qemu-devel; +Cc: pbonzini, Wei Yang, dgilbert, quintela Commit f3f491fcd6dd594ba695 ('Postcopy: Maintain unsentmap') introduced unsentmap to track not yet sent pages. This is not necessary since: * unsentmap is a sub-set of bmap before postcopy start * unsentmap is the summation of bmap and unsentmap after canonicalizing This patch just removes it. Signed-off-by: Wei Yang <richardw.yang@linux.intel.com> --- include/exec/ram_addr.h | 6 ---- migration/ram.c | 80 ++++++++--------------------------------- 2 files changed, 14 insertions(+), 72 deletions(-) diff --git a/include/exec/ram_addr.h b/include/exec/ram_addr.h index a327a80cfe..e96e621de5 100644 --- a/include/exec/ram_addr.h +++ b/include/exec/ram_addr.h @@ -44,12 +44,6 @@ struct RAMBlock { size_t page_size; /* dirty bitmap used during migration */ unsigned long *bmap; - /* bitmap of pages that haven't been sent even once - * only maintained and used in postcopy at the moment - * where it's used to send the dirtymap at the start - * of the postcopy phase - */ - unsigned long *unsentmap; /* bitmap of already received pages in postcopy */ unsigned long *receivedmap; diff --git a/migration/ram.c b/migration/ram.c index 075ddc468c..066eb4755f 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -2348,7 +2348,7 @@ static bool get_queued_page(RAMState *rs, PageSearchStatus *pss) dirty = test_bit(page, block->bmap); if (!dirty) { trace_get_queued_page_not_dirty(block->idstr, (uint64_t)offset, - page, test_bit(page, block->unsentmap)); + page, test_bit(page, block->bmap)); } else { trace_get_queued_page(block->idstr, (uint64_t)offset, page); } @@ -2619,10 +2619,6 @@ static int ram_save_host_page(RAMState *rs, PageSearchStatus *pss, } pages += tmppages; - if (pss->block->unsentmap) { - clear_bit(pss->page, pss->block->unsentmap); - } - pss->page++; } while ((pss->page & (pagesize_bits - 1)) && offset_in_ramblock(pss->block, pss->page << TARGET_PAGE_BITS)); @@ -2776,8 +2772,6 @@ static void ram_save_cleanup(void *opaque) block->clear_bmap = NULL; g_free(block->bmap); block->bmap = NULL; - g_free(block->unsentmap); - block->unsentmap = NULL; } xbzrle_cleanup(); @@ -2857,8 +2851,6 @@ void ram_postcopy_migrated_memory_release(MigrationState *ms) * Returns zero on success * * Callback from postcopy_each_ram_send_discard for each RAMBlock - * Note: At this point the 'unsentmap' is the processed bitmap combined - * with the dirtymap; so a '1' means it's either dirty or unsent. * * @ms: current migration state * @block: RAMBlock to discard @@ -2867,17 +2859,17 @@ static int postcopy_send_discard_bm_ram(MigrationState *ms, RAMBlock *block) { unsigned long end = block->used_length >> TARGET_PAGE_BITS; unsigned long current; - unsigned long *unsentmap = block->unsentmap; + unsigned long *bitmap = block->bmap; for (current = 0; current < end; ) { - unsigned long one = find_next_bit(unsentmap, end, current); + unsigned long one = find_next_bit(bitmap, end, current); unsigned long zero, discard_length; if (one >= end) { break; } - zero = find_next_zero_bit(unsentmap, end, one + 1); + zero = find_next_zero_bit(bitmap, end, one + 1); if (zero >= end) { discard_length = end - one; @@ -2938,16 +2930,12 @@ static int postcopy_each_ram_send_discard(MigrationState *ms) * clean, not a mix. This function canonicalizes the bitmaps. * * @ms: current migration state - * @unsent_pass: if true we need to canonicalize partially unsent host pages - * otherwise we need to canonicalize partially dirty host pages * @block: block that contains the page we want to canonicalize */ -static void postcopy_chunk_hostpages_pass(MigrationState *ms, bool unsent_pass, - RAMBlock *block) +static void postcopy_chunk_hostpages_pass(MigrationState *ms, RAMBlock *block) { RAMState *rs = ram_state; unsigned long *bitmap = block->bmap; - unsigned long *unsentmap = block->unsentmap; unsigned int host_ratio = block->page_size / TARGET_PAGE_SIZE; unsigned long pages = block->used_length >> TARGET_PAGE_BITS; unsigned long run_start; @@ -2957,13 +2945,8 @@ static void postcopy_chunk_hostpages_pass(MigrationState *ms, bool unsent_pass, return; } - if (unsent_pass) { - /* Find a sent page */ - run_start = find_next_zero_bit(unsentmap, pages, 0); - } else { - /* Find a dirty page */ - run_start = find_next_bit(bitmap, pages, 0); - } + /* Find a dirty page */ + run_start = find_next_bit(bitmap, pages, 0); while (run_start < pages) { @@ -2973,11 +2956,7 @@ static void postcopy_chunk_hostpages_pass(MigrationState *ms, bool unsent_pass, */ if (QEMU_IS_ALIGNED(run_start, host_ratio)) { /* Find the end of this run */ - if (unsent_pass) { - run_start = find_next_bit(unsentmap, pages, run_start + 1); - } else { - run_start = find_next_zero_bit(bitmap, pages, run_start + 1); - } + run_start = find_next_zero_bit(bitmap, pages, run_start + 1); /* * If the end isn't at the start of a host page, then the * run doesn't finish at the end of a host page @@ -2994,9 +2973,6 @@ static void postcopy_chunk_hostpages_pass(MigrationState *ms, bool unsent_pass, /* Clean up the bitmap */ for (page = fixup_start_addr; page < fixup_start_addr + host_ratio; page++) { - /* All pages in this host page are now not sent */ - set_bit(page, unsentmap); - /* * Remark them as dirty, updating the count for any pages * that weren't previously dirty. @@ -3005,13 +2981,8 @@ static void postcopy_chunk_hostpages_pass(MigrationState *ms, bool unsent_pass, } } - if (unsent_pass) { - /* Find the next sent page for the next iteration */ - run_start = find_next_zero_bit(unsentmap, pages, run_start); - } else { - /* Find the next dirty page for the next iteration */ - run_start = find_next_bit(bitmap, pages, run_start); - } + /* Find the next dirty page for the next iteration */ + run_start = find_next_bit(bitmap, pages, run_start); } } @@ -3033,13 +3004,10 @@ static int postcopy_chunk_hostpages(MigrationState *ms, RAMBlock *block) { postcopy_discard_send_init(ms, block->idstr); - /* First pass: Discard all partially sent host pages */ - postcopy_chunk_hostpages_pass(ms, true, block); /* - * Second pass: Ensure that all partially dirty host pages are made - * fully dirty. + * Ensure that all partially dirty host pages are made fully dirty. */ - postcopy_chunk_hostpages_pass(ms, false, block); + postcopy_chunk_hostpages_pass(ms, block); postcopy_discard_send_finish(ms); return 0; @@ -3077,19 +3045,6 @@ int ram_postcopy_send_discard_bitmap(MigrationState *ms) rs->last_page = 0; RAMBLOCK_FOREACH_NOT_IGNORED(block) { - unsigned long pages = block->used_length >> TARGET_PAGE_BITS; - unsigned long *bitmap = block->bmap; - unsigned long *unsentmap = block->unsentmap; - - if (!unsentmap) { - /* We don't have a safe way to resize the sentmap, so - * if the bitmap was resized it will be NULL at this - * point. - */ - error_report("migration ram resized during precopy phase"); - rcu_read_unlock(); - return -EINVAL; - } /* Deal with TPS != HPS and huge pages */ ret = postcopy_chunk_hostpages(ms, block); if (ret) { @@ -3097,12 +3052,9 @@ int ram_postcopy_send_discard_bitmap(MigrationState *ms) return ret; } - /* - * Update the unsentmap to be unsentmap = unsentmap | dirty - */ - bitmap_or(unsentmap, unsentmap, bitmap, pages); #ifdef DEBUG_POSTCOPY - ram_debug_dump_bitmap(unsentmap, true, pages); + ram_debug_dump_bitmap(block->bmap, true, + block->used_length >> TARGET_PAGE_BITS); #endif } trace_ram_postcopy_send_discard_bitmap(); @@ -3270,10 +3222,6 @@ static void ram_list_init_bitmaps(void) bitmap_set(block->bmap, 0, pages); block->clear_bmap_shift = shift; block->clear_bmap = bitmap_new(clear_bmap_size(pages, shift)); - if (migrate_postcopy_ram()) { - block->unsentmap = bitmap_new(pages); - bitmap_set(block->unsentmap, 0, pages); - } } } } -- 2.17.1 ^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [PATCH 2/3] migration/postcopy: unsentmap is not necessary for postcopy 2019-08-19 6:18 ` [Qemu-devel] [PATCH 2/3] migration/postcopy: unsentmap is not necessary for postcopy Wei Yang @ 2019-09-24 10:18 ` Dr. David Alan Gilbert 0 siblings, 0 replies; 12+ messages in thread From: Dr. David Alan Gilbert @ 2019-09-24 10:18 UTC (permalink / raw) To: Wei Yang; +Cc: pbonzini, qemu-devel, quintela * Wei Yang (richardw.yang@linux.intel.com) wrote: > Commit f3f491fcd6dd594ba695 ('Postcopy: Maintain unsentmap') introduced > unsentmap to track not yet sent pages. > > This is not necessary since: > > * unsentmap is a sub-set of bmap before postcopy start > * unsentmap is the summation of bmap and unsentmap after canonicalizing > > This patch just removes it. > > Signed-off-by: Wei Yang <richardw.yang@linux.intel.com> Yes, that's simpler Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> > --- > include/exec/ram_addr.h | 6 ---- > migration/ram.c | 80 ++++++++--------------------------------- > 2 files changed, 14 insertions(+), 72 deletions(-) > > diff --git a/include/exec/ram_addr.h b/include/exec/ram_addr.h > index a327a80cfe..e96e621de5 100644 > --- a/include/exec/ram_addr.h > +++ b/include/exec/ram_addr.h > @@ -44,12 +44,6 @@ struct RAMBlock { > size_t page_size; > /* dirty bitmap used during migration */ > unsigned long *bmap; > - /* bitmap of pages that haven't been sent even once > - * only maintained and used in postcopy at the moment > - * where it's used to send the dirtymap at the start > - * of the postcopy phase > - */ > - unsigned long *unsentmap; > /* bitmap of already received pages in postcopy */ > unsigned long *receivedmap; > > diff --git a/migration/ram.c b/migration/ram.c > index 075ddc468c..066eb4755f 100644 > --- a/migration/ram.c > +++ b/migration/ram.c > @@ -2348,7 +2348,7 @@ static bool get_queued_page(RAMState *rs, PageSearchStatus *pss) > dirty = test_bit(page, block->bmap); > if (!dirty) { > trace_get_queued_page_not_dirty(block->idstr, (uint64_t)offset, > - page, test_bit(page, block->unsentmap)); > + page, test_bit(page, block->bmap)); We could actually lose that last trace parameter. Dave > } else { > trace_get_queued_page(block->idstr, (uint64_t)offset, page); > } > @@ -2619,10 +2619,6 @@ static int ram_save_host_page(RAMState *rs, PageSearchStatus *pss, > } > > pages += tmppages; > - if (pss->block->unsentmap) { > - clear_bit(pss->page, pss->block->unsentmap); > - } > - > pss->page++; > } while ((pss->page & (pagesize_bits - 1)) && > offset_in_ramblock(pss->block, pss->page << TARGET_PAGE_BITS)); > @@ -2776,8 +2772,6 @@ static void ram_save_cleanup(void *opaque) > block->clear_bmap = NULL; > g_free(block->bmap); > block->bmap = NULL; > - g_free(block->unsentmap); > - block->unsentmap = NULL; > } > > xbzrle_cleanup(); > @@ -2857,8 +2851,6 @@ void ram_postcopy_migrated_memory_release(MigrationState *ms) > * Returns zero on success > * > * Callback from postcopy_each_ram_send_discard for each RAMBlock > - * Note: At this point the 'unsentmap' is the processed bitmap combined > - * with the dirtymap; so a '1' means it's either dirty or unsent. > * > * @ms: current migration state > * @block: RAMBlock to discard > @@ -2867,17 +2859,17 @@ static int postcopy_send_discard_bm_ram(MigrationState *ms, RAMBlock *block) > { > unsigned long end = block->used_length >> TARGET_PAGE_BITS; > unsigned long current; > - unsigned long *unsentmap = block->unsentmap; > + unsigned long *bitmap = block->bmap; > > for (current = 0; current < end; ) { > - unsigned long one = find_next_bit(unsentmap, end, current); > + unsigned long one = find_next_bit(bitmap, end, current); > unsigned long zero, discard_length; > > if (one >= end) { > break; > } > > - zero = find_next_zero_bit(unsentmap, end, one + 1); > + zero = find_next_zero_bit(bitmap, end, one + 1); > > if (zero >= end) { > discard_length = end - one; > @@ -2938,16 +2930,12 @@ static int postcopy_each_ram_send_discard(MigrationState *ms) > * clean, not a mix. This function canonicalizes the bitmaps. > * > * @ms: current migration state > - * @unsent_pass: if true we need to canonicalize partially unsent host pages > - * otherwise we need to canonicalize partially dirty host pages > * @block: block that contains the page we want to canonicalize > */ > -static void postcopy_chunk_hostpages_pass(MigrationState *ms, bool unsent_pass, > - RAMBlock *block) > +static void postcopy_chunk_hostpages_pass(MigrationState *ms, RAMBlock *block) > { > RAMState *rs = ram_state; > unsigned long *bitmap = block->bmap; > - unsigned long *unsentmap = block->unsentmap; > unsigned int host_ratio = block->page_size / TARGET_PAGE_SIZE; > unsigned long pages = block->used_length >> TARGET_PAGE_BITS; > unsigned long run_start; > @@ -2957,13 +2945,8 @@ static void postcopy_chunk_hostpages_pass(MigrationState *ms, bool unsent_pass, > return; > } > > - if (unsent_pass) { > - /* Find a sent page */ > - run_start = find_next_zero_bit(unsentmap, pages, 0); > - } else { > - /* Find a dirty page */ > - run_start = find_next_bit(bitmap, pages, 0); > - } > + /* Find a dirty page */ > + run_start = find_next_bit(bitmap, pages, 0); > > while (run_start < pages) { > > @@ -2973,11 +2956,7 @@ static void postcopy_chunk_hostpages_pass(MigrationState *ms, bool unsent_pass, > */ > if (QEMU_IS_ALIGNED(run_start, host_ratio)) { > /* Find the end of this run */ > - if (unsent_pass) { > - run_start = find_next_bit(unsentmap, pages, run_start + 1); > - } else { > - run_start = find_next_zero_bit(bitmap, pages, run_start + 1); > - } > + run_start = find_next_zero_bit(bitmap, pages, run_start + 1); > /* > * If the end isn't at the start of a host page, then the > * run doesn't finish at the end of a host page > @@ -2994,9 +2973,6 @@ static void postcopy_chunk_hostpages_pass(MigrationState *ms, bool unsent_pass, > /* Clean up the bitmap */ > for (page = fixup_start_addr; > page < fixup_start_addr + host_ratio; page++) { > - /* All pages in this host page are now not sent */ > - set_bit(page, unsentmap); > - > /* > * Remark them as dirty, updating the count for any pages > * that weren't previously dirty. > @@ -3005,13 +2981,8 @@ static void postcopy_chunk_hostpages_pass(MigrationState *ms, bool unsent_pass, > } > } > > - if (unsent_pass) { > - /* Find the next sent page for the next iteration */ > - run_start = find_next_zero_bit(unsentmap, pages, run_start); > - } else { > - /* Find the next dirty page for the next iteration */ > - run_start = find_next_bit(bitmap, pages, run_start); > - } > + /* Find the next dirty page for the next iteration */ > + run_start = find_next_bit(bitmap, pages, run_start); > } > } > > @@ -3033,13 +3004,10 @@ static int postcopy_chunk_hostpages(MigrationState *ms, RAMBlock *block) > { > postcopy_discard_send_init(ms, block->idstr); > > - /* First pass: Discard all partially sent host pages */ > - postcopy_chunk_hostpages_pass(ms, true, block); > /* > - * Second pass: Ensure that all partially dirty host pages are made > - * fully dirty. > + * Ensure that all partially dirty host pages are made fully dirty. > */ > - postcopy_chunk_hostpages_pass(ms, false, block); > + postcopy_chunk_hostpages_pass(ms, block); > > postcopy_discard_send_finish(ms); > return 0; > @@ -3077,19 +3045,6 @@ int ram_postcopy_send_discard_bitmap(MigrationState *ms) > rs->last_page = 0; > > RAMBLOCK_FOREACH_NOT_IGNORED(block) { > - unsigned long pages = block->used_length >> TARGET_PAGE_BITS; > - unsigned long *bitmap = block->bmap; > - unsigned long *unsentmap = block->unsentmap; > - > - if (!unsentmap) { > - /* We don't have a safe way to resize the sentmap, so > - * if the bitmap was resized it will be NULL at this > - * point. > - */ > - error_report("migration ram resized during precopy phase"); > - rcu_read_unlock(); > - return -EINVAL; > - } > /* Deal with TPS != HPS and huge pages */ > ret = postcopy_chunk_hostpages(ms, block); > if (ret) { > @@ -3097,12 +3052,9 @@ int ram_postcopy_send_discard_bitmap(MigrationState *ms) > return ret; > } > > - /* > - * Update the unsentmap to be unsentmap = unsentmap | dirty > - */ > - bitmap_or(unsentmap, unsentmap, bitmap, pages); > #ifdef DEBUG_POSTCOPY > - ram_debug_dump_bitmap(unsentmap, true, pages); > + ram_debug_dump_bitmap(block->bmap, true, > + block->used_length >> TARGET_PAGE_BITS); > #endif > } > trace_ram_postcopy_send_discard_bitmap(); > @@ -3270,10 +3222,6 @@ static void ram_list_init_bitmaps(void) > bitmap_set(block->bmap, 0, pages); > block->clear_bmap_shift = shift; > block->clear_bmap = bitmap_new(clear_bmap_size(pages, shift)); > - if (migrate_postcopy_ram()) { > - block->unsentmap = bitmap_new(pages); > - bitmap_set(block->unsentmap, 0, pages); > - } > } > } > } > -- > 2.17.1 > -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK ^ permalink raw reply [flat|nested] 12+ messages in thread
* [Qemu-devel] [PATCH 3/3] migration: remove sent parameter in get_queued_page_not_dirty 2019-08-19 6:18 [Qemu-devel] [PATCH 0/3] migration/postcopy: unsentmap is not necessary Wei Yang 2019-08-19 6:18 ` [Qemu-devel] [PATCH 1/3] migration/postcopy: not necessary to do discard when canonicalizing bitmap Wei Yang 2019-08-19 6:18 ` [Qemu-devel] [PATCH 2/3] migration/postcopy: unsentmap is not necessary for postcopy Wei Yang @ 2019-08-19 6:18 ` Wei Yang 2019-09-24 10:19 ` Dr. David Alan Gilbert 2019-09-16 6:06 ` [Qemu-devel] [PATCH 0/3] migration/postcopy: unsentmap is not necessary Wei Yang 2019-09-25 9:52 ` [Qemu-devel] " Dr. David Alan Gilbert 4 siblings, 1 reply; 12+ messages in thread From: Wei Yang @ 2019-08-19 6:18 UTC (permalink / raw) To: qemu-devel; +Cc: pbonzini, Wei Yang, dgilbert, quintela This is a cleanup for previous removal of unsentmap. The sent parameter is not necessary now. Signed-off-by: Wei Yang <richardw.yang@linux.intel.com> --- migration/ram.c | 2 +- migration/trace-events | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index 066eb4755f..a6d3d09ebd 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -2348,7 +2348,7 @@ static bool get_queued_page(RAMState *rs, PageSearchStatus *pss) dirty = test_bit(page, block->bmap); if (!dirty) { trace_get_queued_page_not_dirty(block->idstr, (uint64_t)offset, - page, test_bit(page, block->bmap)); + page); } else { trace_get_queued_page(block->idstr, (uint64_t)offset, page); } diff --git a/migration/trace-events b/migration/trace-events index 00ffcd5930..858d415d56 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -76,7 +76,7 @@ qemu_file_fclose(void) "" # ram.c get_queued_page(const char *block_name, uint64_t tmp_offset, unsigned long page_abs) "%s/0x%" PRIx64 " page_abs=0x%lx" -get_queued_page_not_dirty(const char *block_name, uint64_t tmp_offset, unsigned long page_abs, int sent) "%s/0x%" PRIx64 " page_abs=0x%lx (sent=%d)" +get_queued_page_not_dirty(const char *block_name, uint64_t tmp_offset, unsigned long page_abs) "%s/0x%" PRIx64 " page_abs=0x%lx" migration_bitmap_sync_start(void) "" migration_bitmap_sync_end(uint64_t dirty_pages) "dirty_pages %" PRIu64 migration_bitmap_clear_dirty(char *str, uint64_t start, uint64_t size, unsigned long page) "rb %s start 0x%"PRIx64" size 0x%"PRIx64" page 0x%lx" -- 2.17.1 ^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [PATCH 3/3] migration: remove sent parameter in get_queued_page_not_dirty 2019-08-19 6:18 ` [Qemu-devel] [PATCH 3/3] migration: remove sent parameter in get_queued_page_not_dirty Wei Yang @ 2019-09-24 10:19 ` Dr. David Alan Gilbert 0 siblings, 0 replies; 12+ messages in thread From: Dr. David Alan Gilbert @ 2019-09-24 10:19 UTC (permalink / raw) To: Wei Yang; +Cc: pbonzini, qemu-devel, quintela * Wei Yang (richardw.yang@linux.intel.com) wrote: > This is a cleanup for previous removal of unsentmap. > > The sent parameter is not necessary now. > > Signed-off-by: Wei Yang <richardw.yang@linux.intel.com> OK, you did... Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> > --- > migration/ram.c | 2 +- > migration/trace-events | 2 +- > 2 files changed, 2 insertions(+), 2 deletions(-) > > diff --git a/migration/ram.c b/migration/ram.c > index 066eb4755f..a6d3d09ebd 100644 > --- a/migration/ram.c > +++ b/migration/ram.c > @@ -2348,7 +2348,7 @@ static bool get_queued_page(RAMState *rs, PageSearchStatus *pss) > dirty = test_bit(page, block->bmap); > if (!dirty) { > trace_get_queued_page_not_dirty(block->idstr, (uint64_t)offset, > - page, test_bit(page, block->bmap)); > + page); > } else { > trace_get_queued_page(block->idstr, (uint64_t)offset, page); > } > diff --git a/migration/trace-events b/migration/trace-events > index 00ffcd5930..858d415d56 100644 > --- a/migration/trace-events > +++ b/migration/trace-events > @@ -76,7 +76,7 @@ qemu_file_fclose(void) "" > > # ram.c > get_queued_page(const char *block_name, uint64_t tmp_offset, unsigned long page_abs) "%s/0x%" PRIx64 " page_abs=0x%lx" > -get_queued_page_not_dirty(const char *block_name, uint64_t tmp_offset, unsigned long page_abs, int sent) "%s/0x%" PRIx64 " page_abs=0x%lx (sent=%d)" > +get_queued_page_not_dirty(const char *block_name, uint64_t tmp_offset, unsigned long page_abs) "%s/0x%" PRIx64 " page_abs=0x%lx" > migration_bitmap_sync_start(void) "" > migration_bitmap_sync_end(uint64_t dirty_pages) "dirty_pages %" PRIu64 > migration_bitmap_clear_dirty(char *str, uint64_t start, uint64_t size, unsigned long page) "rb %s start 0x%"PRIx64" size 0x%"PRIx64" page 0x%lx" > -- > 2.17.1 > -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [Qemu-devel] [PATCH 0/3] migration/postcopy: unsentmap is not necessary 2019-08-19 6:18 [Qemu-devel] [PATCH 0/3] migration/postcopy: unsentmap is not necessary Wei Yang ` (2 preceding siblings ...) 2019-08-19 6:18 ` [Qemu-devel] [PATCH 3/3] migration: remove sent parameter in get_queued_page_not_dirty Wei Yang @ 2019-09-16 6:06 ` Wei Yang 2019-09-20 18:53 ` Dr. David Alan Gilbert 2019-09-25 9:52 ` [Qemu-devel] " Dr. David Alan Gilbert 4 siblings, 1 reply; 12+ messages in thread From: Wei Yang @ 2019-09-16 6:06 UTC (permalink / raw) To: Wei Yang; +Cc: pbonzini, qemu-devel, dgilbert, quintela Hi, Dave and Juan Would you mind taking a look at this one? On Mon, Aug 19, 2019 at 02:18:40PM +0800, Wei Yang wrote: >Three patches to cleanup postcopy: > >[1]: split canonicalize bitmap and discard page >[2]: remove unsentmap since it is not necessary >[3]: cleanup the get_queued_page_not_dirty > >Wei Yang (3): > migration/postcopy: not necessary to do discard when canonicalizing > bitmap > migration/postcopy: unsentmap is not necessary for postcopy > migration: remove sent parameter in get_queued_page_not_dirty > > include/exec/ram_addr.h | 6 --- > migration/ram.c | 94 +++++++---------------------------------- > migration/trace-events | 2 +- > 3 files changed, 16 insertions(+), 86 deletions(-) > >-- >2.17.1 -- Wei Yang Help you, Help me ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH 0/3] migration/postcopy: unsentmap is not necessary 2019-09-16 6:06 ` [Qemu-devel] [PATCH 0/3] migration/postcopy: unsentmap is not necessary Wei Yang @ 2019-09-20 18:53 ` Dr. David Alan Gilbert 2019-09-23 5:56 ` Wei Yang 0 siblings, 1 reply; 12+ messages in thread From: Dr. David Alan Gilbert @ 2019-09-20 18:53 UTC (permalink / raw) To: Wei Yang; +Cc: pbonzini, qemu-devel, quintela * Wei Yang (richardw.yang@linux.intel.com) wrote: > Hi, Dave and Juan > > Would you mind taking a look at this one? Yes, I'll have a look, I'm just fighting through a bunch of other stuff. Dave > On Mon, Aug 19, 2019 at 02:18:40PM +0800, Wei Yang wrote: > >Three patches to cleanup postcopy: > > > >[1]: split canonicalize bitmap and discard page > >[2]: remove unsentmap since it is not necessary > >[3]: cleanup the get_queued_page_not_dirty > > > >Wei Yang (3): > > migration/postcopy: not necessary to do discard when canonicalizing > > bitmap > > migration/postcopy: unsentmap is not necessary for postcopy > > migration: remove sent parameter in get_queued_page_not_dirty > > > > include/exec/ram_addr.h | 6 --- > > migration/ram.c | 94 +++++++---------------------------------- > > migration/trace-events | 2 +- > > 3 files changed, 16 insertions(+), 86 deletions(-) > > > >-- > >2.17.1 > > -- > Wei Yang > Help you, Help me -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH 0/3] migration/postcopy: unsentmap is not necessary 2019-09-20 18:53 ` Dr. David Alan Gilbert @ 2019-09-23 5:56 ` Wei Yang 0 siblings, 0 replies; 12+ messages in thread From: Wei Yang @ 2019-09-23 5:56 UTC (permalink / raw) To: Dr. David Alan Gilbert; +Cc: pbonzini, quintela, Wei Yang, qemu-devel On Fri, Sep 20, 2019 at 07:53:51PM +0100, Dr. David Alan Gilbert wrote: >* Wei Yang (richardw.yang@linux.intel.com) wrote: >> Hi, Dave and Juan >> >> Would you mind taking a look at this one? > >Yes, I'll have a look, I'm just fighting through a bunch of other stuff. > Yep, thanks ~ Me too :-) Have a good day >Dave > >> On Mon, Aug 19, 2019 at 02:18:40PM +0800, Wei Yang wrote: >> >Three patches to cleanup postcopy: >> > >> >[1]: split canonicalize bitmap and discard page >> >[2]: remove unsentmap since it is not necessary >> >[3]: cleanup the get_queued_page_not_dirty >> > >> >Wei Yang (3): >> > migration/postcopy: not necessary to do discard when canonicalizing >> > bitmap >> > migration/postcopy: unsentmap is not necessary for postcopy >> > migration: remove sent parameter in get_queued_page_not_dirty >> > >> > include/exec/ram_addr.h | 6 --- >> > migration/ram.c | 94 +++++++---------------------------------- >> > migration/trace-events | 2 +- >> > 3 files changed, 16 insertions(+), 86 deletions(-) >> > >> >-- >> >2.17.1 >> >> -- >> Wei Yang >> Help you, Help me >-- >Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK -- Wei Yang Help you, Help me ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [Qemu-devel] [PATCH 0/3] migration/postcopy: unsentmap is not necessary 2019-08-19 6:18 [Qemu-devel] [PATCH 0/3] migration/postcopy: unsentmap is not necessary Wei Yang ` (3 preceding siblings ...) 2019-09-16 6:06 ` [Qemu-devel] [PATCH 0/3] migration/postcopy: unsentmap is not necessary Wei Yang @ 2019-09-25 9:52 ` Dr. David Alan Gilbert 4 siblings, 0 replies; 12+ messages in thread From: Dr. David Alan Gilbert @ 2019-09-25 9:52 UTC (permalink / raw) To: Wei Yang; +Cc: pbonzini, qemu-devel, quintela * Wei Yang (richardw.yang@linux.intel.com) wrote: > Three patches to cleanup postcopy: > > [1]: split canonicalize bitmap and discard page > [2]: remove unsentmap since it is not necessary > [3]: cleanup the get_queued_page_not_dirty Queued > > Wei Yang (3): > migration/postcopy: not necessary to do discard when canonicalizing > bitmap > migration/postcopy: unsentmap is not necessary for postcopy > migration: remove sent parameter in get_queued_page_not_dirty > > include/exec/ram_addr.h | 6 --- > migration/ram.c | 94 +++++++---------------------------------- > migration/trace-events | 2 +- > 3 files changed, 16 insertions(+), 86 deletions(-) > > -- > 2.17.1 > > -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK ^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2019-09-25 9:53 UTC | newest] Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-08-19 6:18 [Qemu-devel] [PATCH 0/3] migration/postcopy: unsentmap is not necessary Wei Yang 2019-08-19 6:18 ` [Qemu-devel] [PATCH 1/3] migration/postcopy: not necessary to do discard when canonicalizing bitmap Wei Yang 2019-09-24 10:02 ` Dr. David Alan Gilbert 2019-09-25 0:25 ` Wei Yang 2019-08-19 6:18 ` [Qemu-devel] [PATCH 2/3] migration/postcopy: unsentmap is not necessary for postcopy Wei Yang 2019-09-24 10:18 ` Dr. David Alan Gilbert 2019-08-19 6:18 ` [Qemu-devel] [PATCH 3/3] migration: remove sent parameter in get_queued_page_not_dirty Wei Yang 2019-09-24 10:19 ` Dr. David Alan Gilbert 2019-09-16 6:06 ` [Qemu-devel] [PATCH 0/3] migration/postcopy: unsentmap is not necessary Wei Yang 2019-09-20 18:53 ` Dr. David Alan Gilbert 2019-09-23 5:56 ` Wei Yang 2019-09-25 9:52 ` [Qemu-devel] " Dr. David Alan Gilbert
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).