[Qemu-devel] [PATCH 0/3] migration/postcopy: unsentmap is not necessary

[Qemu-devel] [PATCH 0/3] migration/postcopy: unsentmap is not necessary

[Wei Yang]

Three patches to cleanup postcopy:

[1]: split canonicalize bitmap and discard page
[2]: remove unsentmap since it is not necessary
[3]: cleanup the get_queued_page_not_dirty

Wei Yang (3):
  migration/postcopy: not necessary to do discard when canonicalizing
    bitmap
  migration/postcopy: unsentmap is not necessary for postcopy
  migration: remove sent parameter in get_queued_page_not_dirty

 include/exec/ram_addr.h |  6 ---
 migration/ram.c         | 94 +++++++----------------------------------
 migration/trace-events  |  2 +-
 3 files changed, 16 insertions(+), 86 deletions(-)

-- 
2.17.1

[Qemu-devel] [PATCH 1/3] migration/postcopy: not necessary to do discard when canonicalizing bitmap

[Wei Yang]

All pages, either partially sent or partially dirty, will be discarded in
postcopy_send_discard_bm_ram(), since we update the unsentmap to be
unsentmap = unsentmap | dirty in ram_postcopy_send_discard_bitmap().

This is not necessary to do discard when canonicalizing bitmap. And by
doing so, we separate the page discard into two individual steps:

  * canonicalize bitmap
  * discard page

Signed-off-by: Wei Yang <richardw.yang@linux.intel.com>
---
 migration/ram.c | 14 +-------------
 1 file changed, 1 insertion(+), 13 deletions(-)

diff --git a/migration/ram.c b/migration/ram.c
index 35552c090b..075ddc468c 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -2928,7 +2928,7 @@ static int postcopy_each_ram_send_discard(MigrationState *ms)
 }
 
 /**
- * postcopy_chunk_hostpages_pass: canocalize bitmap in hostpages
+ * postcopy_chunk_hostpages_pass: canonicalize bitmap in hostpages
  *
  * Helper for postcopy_chunk_hostpages; it's called twice to
  * canonicalize the two bitmaps, that are similar, but one is
@@ -2991,18 +2991,6 @@ static void postcopy_chunk_hostpages_pass(MigrationState *ms, bool unsent_pass,
                                                              host_ratio);
             run_start = QEMU_ALIGN_UP(run_start, host_ratio);
 
-            /* Tell the destination to discard this page */
-            if (unsent_pass || !test_bit(fixup_start_addr, unsentmap)) {
-                /* For the unsent_pass we:
-                 *     discard partially sent pages
-                 * For the !unsent_pass (dirty) we:
-                 *     discard partially dirty pages that were sent
-                 *     (any partially sent pages were already discarded
-                 *     by the previous unsent_pass)
-                 */
-                postcopy_discard_send_range(ms, fixup_start_addr, host_ratio);
-            }
-
             /* Clean up the bitmap */
             for (page = fixup_start_addr;
                  page < fixup_start_addr + host_ratio; page++) {
-- 
2.17.1

[Qemu-devel] [PATCH 2/3] migration/postcopy: unsentmap is not necessary for postcopy

[Wei Yang]

Commit f3f491fcd6dd594ba695 ('Postcopy: Maintain unsentmap') introduced
unsentmap to track not yet sent pages.

This is not necessary since:

    * unsentmap is a sub-set of bmap before postcopy start
    * unsentmap is the summation of bmap and unsentmap after canonicalizing

This patch just removes it.

Signed-off-by: Wei Yang <richardw.yang@linux.intel.com>
---
 include/exec/ram_addr.h |  6 ----
 migration/ram.c         | 80 ++++++++---------------------------------
 2 files changed, 14 insertions(+), 72 deletions(-)

diff --git a/include/exec/ram_addr.h b/include/exec/ram_addr.h
index a327a80cfe..e96e621de5 100644
--- a/include/exec/ram_addr.h
+++ b/include/exec/ram_addr.h
@@ -44,12 +44,6 @@ struct RAMBlock {
     size_t page_size;
     /* dirty bitmap used during migration */
     unsigned long *bmap;
-    /* bitmap of pages that haven't been sent even once
-     * only maintained and used in postcopy at the moment
-     * where it's used to send the dirtymap at the start
-     * of the postcopy phase
-     */
-    unsigned long *unsentmap;
     /* bitmap of already received pages in postcopy */
     unsigned long *receivedmap;
 
diff --git a/migration/ram.c b/migration/ram.c
index 075ddc468c..066eb4755f 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -2348,7 +2348,7 @@ static bool get_queued_page(RAMState *rs, PageSearchStatus *pss)
             dirty = test_bit(page, block->bmap);
             if (!dirty) {
                 trace_get_queued_page_not_dirty(block->idstr, (uint64_t)offset,
-                       page, test_bit(page, block->unsentmap));
+                       page, test_bit(page, block->bmap));
             } else {
                 trace_get_queued_page(block->idstr, (uint64_t)offset, page);
             }
@@ -2619,10 +2619,6 @@ static int ram_save_host_page(RAMState *rs, PageSearchStatus *pss,
         }
 
         pages += tmppages;
-        if (pss->block->unsentmap) {
-            clear_bit(pss->page, pss->block->unsentmap);
-        }
-
         pss->page++;
     } while ((pss->page & (pagesize_bits - 1)) &&
              offset_in_ramblock(pss->block, pss->page << TARGET_PAGE_BITS));
@@ -2776,8 +2772,6 @@ static void ram_save_cleanup(void *opaque)
         block->clear_bmap = NULL;
         g_free(block->bmap);
         block->bmap = NULL;
-        g_free(block->unsentmap);
-        block->unsentmap = NULL;
     }
 
     xbzrle_cleanup();
@@ -2857,8 +2851,6 @@ void ram_postcopy_migrated_memory_release(MigrationState *ms)
  * Returns zero on success
  *
  * Callback from postcopy_each_ram_send_discard for each RAMBlock
- * Note: At this point the 'unsentmap' is the processed bitmap combined
- *       with the dirtymap; so a '1' means it's either dirty or unsent.
  *
  * @ms: current migration state
  * @block: RAMBlock to discard
@@ -2867,17 +2859,17 @@ static int postcopy_send_discard_bm_ram(MigrationState *ms, RAMBlock *block)
 {
     unsigned long end = block->used_length >> TARGET_PAGE_BITS;
     unsigned long current;
-    unsigned long *unsentmap = block->unsentmap;
+    unsigned long *bitmap = block->bmap;
 
     for (current = 0; current < end; ) {
-        unsigned long one = find_next_bit(unsentmap, end, current);
+        unsigned long one = find_next_bit(bitmap, end, current);
         unsigned long zero, discard_length;
 
         if (one >= end) {
             break;
         }
 
-        zero = find_next_zero_bit(unsentmap, end, one + 1);
+        zero = find_next_zero_bit(bitmap, end, one + 1);
 
         if (zero >= end) {
             discard_length = end - one;
@@ -2938,16 +2930,12 @@ static int postcopy_each_ram_send_discard(MigrationState *ms)
  * clean, not a mix.  This function canonicalizes the bitmaps.
  *
  * @ms: current migration state
- * @unsent_pass: if true we need to canonicalize partially unsent host pages
- *               otherwise we need to canonicalize partially dirty host pages
  * @block: block that contains the page we want to canonicalize
  */
-static void postcopy_chunk_hostpages_pass(MigrationState *ms, bool unsent_pass,
-                                          RAMBlock *block)
+static void postcopy_chunk_hostpages_pass(MigrationState *ms, RAMBlock *block)
 {
     RAMState *rs = ram_state;
     unsigned long *bitmap = block->bmap;
-    unsigned long *unsentmap = block->unsentmap;
     unsigned int host_ratio = block->page_size / TARGET_PAGE_SIZE;
     unsigned long pages = block->used_length >> TARGET_PAGE_BITS;
     unsigned long run_start;
@@ -2957,13 +2945,8 @@ static void postcopy_chunk_hostpages_pass(MigrationState *ms, bool unsent_pass,
         return;
     }
 
-    if (unsent_pass) {
-        /* Find a sent page */
-        run_start = find_next_zero_bit(unsentmap, pages, 0);
-    } else {
-        /* Find a dirty page */
-        run_start = find_next_bit(bitmap, pages, 0);
-    }
+    /* Find a dirty page */
+    run_start = find_next_bit(bitmap, pages, 0);
 
     while (run_start < pages) {
 
@@ -2973,11 +2956,7 @@ static void postcopy_chunk_hostpages_pass(MigrationState *ms, bool unsent_pass,
          */
         if (QEMU_IS_ALIGNED(run_start, host_ratio)) {
             /* Find the end of this run */
-            if (unsent_pass) {
-                run_start = find_next_bit(unsentmap, pages, run_start + 1);
-            } else {
-                run_start = find_next_zero_bit(bitmap, pages, run_start + 1);
-            }
+            run_start = find_next_zero_bit(bitmap, pages, run_start + 1);
             /*
              * If the end isn't at the start of a host page, then the
              * run doesn't finish at the end of a host page
@@ -2994,9 +2973,6 @@ static void postcopy_chunk_hostpages_pass(MigrationState *ms, bool unsent_pass,
             /* Clean up the bitmap */
             for (page = fixup_start_addr;
                  page < fixup_start_addr + host_ratio; page++) {
-                /* All pages in this host page are now not sent */
-                set_bit(page, unsentmap);
-
                 /*
                  * Remark them as dirty, updating the count for any pages
                  * that weren't previously dirty.
@@ -3005,13 +2981,8 @@ static void postcopy_chunk_hostpages_pass(MigrationState *ms, bool unsent_pass,
             }
         }
 
-        if (unsent_pass) {
-            /* Find the next sent page for the next iteration */
-            run_start = find_next_zero_bit(unsentmap, pages, run_start);
-        } else {
-            /* Find the next dirty page for the next iteration */
-            run_start = find_next_bit(bitmap, pages, run_start);
-        }
+        /* Find the next dirty page for the next iteration */
+        run_start = find_next_bit(bitmap, pages, run_start);
     }
 }
 
@@ -3033,13 +3004,10 @@ static int postcopy_chunk_hostpages(MigrationState *ms, RAMBlock *block)
 {
     postcopy_discard_send_init(ms, block->idstr);
 
-    /* First pass: Discard all partially sent host pages */
-    postcopy_chunk_hostpages_pass(ms, true, block);
     /*
-     * Second pass: Ensure that all partially dirty host pages are made
-     * fully dirty.
+     * Ensure that all partially dirty host pages are made fully dirty.
      */
-    postcopy_chunk_hostpages_pass(ms, false, block);
+    postcopy_chunk_hostpages_pass(ms, block);
 
     postcopy_discard_send_finish(ms);
     return 0;
@@ -3077,19 +3045,6 @@ int ram_postcopy_send_discard_bitmap(MigrationState *ms)
     rs->last_page = 0;
 
     RAMBLOCK_FOREACH_NOT_IGNORED(block) {
-        unsigned long pages = block->used_length >> TARGET_PAGE_BITS;
-        unsigned long *bitmap = block->bmap;
-        unsigned long *unsentmap = block->unsentmap;
-
-        if (!unsentmap) {
-            /* We don't have a safe way to resize the sentmap, so
-             * if the bitmap was resized it will be NULL at this
-             * point.
-             */
-            error_report("migration ram resized during precopy phase");
-            rcu_read_unlock();
-            return -EINVAL;
-        }
         /* Deal with TPS != HPS and huge pages */
         ret = postcopy_chunk_hostpages(ms, block);
         if (ret) {
@@ -3097,12 +3052,9 @@ int ram_postcopy_send_discard_bitmap(MigrationState *ms)
             return ret;
         }
 
-        /*
-         * Update the unsentmap to be unsentmap = unsentmap | dirty
-         */
-        bitmap_or(unsentmap, unsentmap, bitmap, pages);
 #ifdef DEBUG_POSTCOPY
-        ram_debug_dump_bitmap(unsentmap, true, pages);
+        ram_debug_dump_bitmap(block->bmap, true,
+                              block->used_length >> TARGET_PAGE_BITS);
 #endif
     }
     trace_ram_postcopy_send_discard_bitmap();
@@ -3270,10 +3222,6 @@ static void ram_list_init_bitmaps(void)
             bitmap_set(block->bmap, 0, pages);
             block->clear_bmap_shift = shift;
             block->clear_bmap = bitmap_new(clear_bmap_size(pages, shift));
-            if (migrate_postcopy_ram()) {
-                block->unsentmap = bitmap_new(pages);
-                bitmap_set(block->unsentmap, 0, pages);
-            }
         }
     }
 }
-- 
2.17.1

[Qemu-devel] [PATCH 3/3] migration: remove sent parameter in get_queued_page_not_dirty

[Wei Yang]

This is a cleanup for previous removal of unsentmap.

The sent parameter is not necessary now.

Signed-off-by: Wei Yang <richardw.yang@linux.intel.com>
---
 migration/ram.c        | 2 +-
 migration/trace-events | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/migration/ram.c b/migration/ram.c
index 066eb4755f..a6d3d09ebd 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -2348,7 +2348,7 @@ static bool get_queued_page(RAMState *rs, PageSearchStatus *pss)
             dirty = test_bit(page, block->bmap);
             if (!dirty) {
                 trace_get_queued_page_not_dirty(block->idstr, (uint64_t)offset,
-                       page, test_bit(page, block->bmap));
+                                                page);
             } else {
                 trace_get_queued_page(block->idstr, (uint64_t)offset, page);
             }
diff --git a/migration/trace-events b/migration/trace-events
index 00ffcd5930..858d415d56 100644
--- a/migration/trace-events
+++ b/migration/trace-events
@@ -76,7 +76,7 @@ qemu_file_fclose(void) ""
 
 # ram.c
 get_queued_page(const char *block_name, uint64_t tmp_offset, unsigned long page_abs) "%s/0x%" PRIx64 " page_abs=0x%lx"
-get_queued_page_not_dirty(const char *block_name, uint64_t tmp_offset, unsigned long page_abs, int sent) "%s/0x%" PRIx64 " page_abs=0x%lx (sent=%d)"
+get_queued_page_not_dirty(const char *block_name, uint64_t tmp_offset, unsigned long page_abs) "%s/0x%" PRIx64 " page_abs=0x%lx"
 migration_bitmap_sync_start(void) ""
 migration_bitmap_sync_end(uint64_t dirty_pages) "dirty_pages %" PRIu64
 migration_bitmap_clear_dirty(char *str, uint64_t start, uint64_t size, unsigned long page) "rb %s start 0x%"PRIx64" size 0x%"PRIx64" page 0x%lx"
-- 
2.17.1

Re: [Qemu-devel] [PATCH 0/3] migration/postcopy: unsentmap is not necessary

[Wei Yang]

Hi, Dave and Juan

Would you mind taking a look at this one?

On Mon, Aug 19, 2019 at 02:18:40PM +0800, Wei Yang wrote:
>Three patches to cleanup postcopy:
>
>[1]: split canonicalize bitmap and discard page
>[2]: remove unsentmap since it is not necessary
>[3]: cleanup the get_queued_page_not_dirty
>
>Wei Yang (3):
>  migration/postcopy: not necessary to do discard when canonicalizing
>    bitmap
>  migration/postcopy: unsentmap is not necessary for postcopy
>  migration: remove sent parameter in get_queued_page_not_dirty
>
> include/exec/ram_addr.h |  6 ---
> migration/ram.c         | 94 +++++++----------------------------------
> migration/trace-events  |  2 +-
> 3 files changed, 16 insertions(+), 86 deletions(-)
>
>-- 
>2.17.1

-- 
Wei Yang
Help you, Help me

Re: [PATCH 0/3] migration/postcopy: unsentmap is not necessary

[Dr. David Alan Gilbert]

* Wei Yang (richardw.yang@linux.intel.com) wrote:
> Hi, Dave and Juan
> 
> Would you mind taking a look at this one?

Yes, I'll have a look, I'm just fighting through a bunch of other stuff.

Dave

> On Mon, Aug 19, 2019 at 02:18:40PM +0800, Wei Yang wrote:
> >Three patches to cleanup postcopy:
> >
> >[1]: split canonicalize bitmap and discard page
> >[2]: remove unsentmap since it is not necessary
> >[3]: cleanup the get_queued_page_not_dirty
> >
> >Wei Yang (3):
> >  migration/postcopy: not necessary to do discard when canonicalizing
> >    bitmap
> >  migration/postcopy: unsentmap is not necessary for postcopy
> >  migration: remove sent parameter in get_queued_page_not_dirty
> >
> > include/exec/ram_addr.h |  6 ---
> > migration/ram.c         | 94 +++++++----------------------------------
> > migration/trace-events  |  2 +-
> > 3 files changed, 16 insertions(+), 86 deletions(-)
> >
> >-- 
> >2.17.1
> 
> -- 
> Wei Yang
> Help you, Help me
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [PATCH 0/3] migration/postcopy: unsentmap is not necessary

[Wei Yang]

On Fri, Sep 20, 2019 at 07:53:51PM +0100, Dr. David Alan Gilbert wrote:
>* Wei Yang (richardw.yang@linux.intel.com) wrote:
>> Hi, Dave and Juan
>> 
>> Would you mind taking a look at this one?
>
>Yes, I'll have a look, I'm just fighting through a bunch of other stuff.
>

Yep, thanks ~ Me too :-)

Have a good day

>Dave
>
>> On Mon, Aug 19, 2019 at 02:18:40PM +0800, Wei Yang wrote:
>> >Three patches to cleanup postcopy:
>> >
>> >[1]: split canonicalize bitmap and discard page
>> >[2]: remove unsentmap since it is not necessary
>> >[3]: cleanup the get_queued_page_not_dirty
>> >
>> >Wei Yang (3):
>> >  migration/postcopy: not necessary to do discard when canonicalizing
>> >    bitmap
>> >  migration/postcopy: unsentmap is not necessary for postcopy
>> >  migration: remove sent parameter in get_queued_page_not_dirty
>> >
>> > include/exec/ram_addr.h |  6 ---
>> > migration/ram.c         | 94 +++++++----------------------------------
>> > migration/trace-events  |  2 +-
>> > 3 files changed, 16 insertions(+), 86 deletions(-)
>> >
>> >-- 
>> >2.17.1
>> 
>> -- 
>> Wei Yang
>> Help you, Help me
>--
>Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

-- 
Wei Yang
Help you, Help me

Re: [PATCH 1/3] migration/postcopy: not necessary to do discard when canonicalizing bitmap

[Dr. David Alan Gilbert]

* Wei Yang (richardw.yang@linux.intel.com) wrote:
> All pages, either partially sent or partially dirty, will be discarded in
> postcopy_send_discard_bm_ram(), since we update the unsentmap to be
> unsentmap = unsentmap | dirty in ram_postcopy_send_discard_bitmap().
> 
> This is not necessary to do discard when canonicalizing bitmap. And by
> doing so, we separate the page discard into two individual steps:
> 
>   * canonicalize bitmap
>   * discard page
> 
> Signed-off-by: Wei Yang <richardw.yang@linux.intel.com>

Yes, I think when I originally wrote it, the set of pages that was
discarded was different; I think it was actually the set of 
!unsent & dirty - i.e. only pages that had been sent and then redirtied;
it later got reworked to include unsent pages as well - so this lot can
be simplified.



Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

> ---
>  migration/ram.c | 14 +-------------
>  1 file changed, 1 insertion(+), 13 deletions(-)
> 
> diff --git a/migration/ram.c b/migration/ram.c
> index 35552c090b..075ddc468c 100644
> --- a/migration/ram.c
> +++ b/migration/ram.c
> @@ -2928,7 +2928,7 @@ static int postcopy_each_ram_send_discard(MigrationState *ms)
>  }
>  
>  /**
> - * postcopy_chunk_hostpages_pass: canocalize bitmap in hostpages
> + * postcopy_chunk_hostpages_pass: canonicalize bitmap in hostpages
>   *
>   * Helper for postcopy_chunk_hostpages; it's called twice to
>   * canonicalize the two bitmaps, that are similar, but one is
> @@ -2991,18 +2991,6 @@ static void postcopy_chunk_hostpages_pass(MigrationState *ms, bool unsent_pass,
>                                                               host_ratio);
>              run_start = QEMU_ALIGN_UP(run_start, host_ratio);
>  
> -            /* Tell the destination to discard this page */
> -            if (unsent_pass || !test_bit(fixup_start_addr, unsentmap)) {
> -                /* For the unsent_pass we:
> -                 *     discard partially sent pages
> -                 * For the !unsent_pass (dirty) we:
> -                 *     discard partially dirty pages that were sent
> -                 *     (any partially sent pages were already discarded
> -                 *     by the previous unsent_pass)
> -                 */
> -                postcopy_discard_send_range(ms, fixup_start_addr, host_ratio);
> -            }
> -
>              /* Clean up the bitmap */
>              for (page = fixup_start_addr;
>                   page < fixup_start_addr + host_ratio; page++) {
> -- 
> 2.17.1
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [PATCH 2/3] migration/postcopy: unsentmap is not necessary for postcopy

[Dr. David Alan Gilbert]

* Wei Yang (richardw.yang@linux.intel.com) wrote:
> Commit f3f491fcd6dd594ba695 ('Postcopy: Maintain unsentmap') introduced
> unsentmap to track not yet sent pages.
> 
> This is not necessary since:
> 
>     * unsentmap is a sub-set of bmap before postcopy start
>     * unsentmap is the summation of bmap and unsentmap after canonicalizing
> 
> This patch just removes it.
> 
> Signed-off-by: Wei Yang <richardw.yang@linux.intel.com>

Yes, that's simpler


Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

> ---
>  include/exec/ram_addr.h |  6 ----
>  migration/ram.c         | 80 ++++++++---------------------------------
>  2 files changed, 14 insertions(+), 72 deletions(-)
> 
> diff --git a/include/exec/ram_addr.h b/include/exec/ram_addr.h
> index a327a80cfe..e96e621de5 100644
> --- a/include/exec/ram_addr.h
> +++ b/include/exec/ram_addr.h
> @@ -44,12 +44,6 @@ struct RAMBlock {
>      size_t page_size;
>      /* dirty bitmap used during migration */
>      unsigned long *bmap;
> -    /* bitmap of pages that haven't been sent even once
> -     * only maintained and used in postcopy at the moment
> -     * where it's used to send the dirtymap at the start
> -     * of the postcopy phase
> -     */
> -    unsigned long *unsentmap;
>      /* bitmap of already received pages in postcopy */
>      unsigned long *receivedmap;
>  
> diff --git a/migration/ram.c b/migration/ram.c
> index 075ddc468c..066eb4755f 100644
> --- a/migration/ram.c
> +++ b/migration/ram.c
> @@ -2348,7 +2348,7 @@ static bool get_queued_page(RAMState *rs, PageSearchStatus *pss)
>              dirty = test_bit(page, block->bmap);
>              if (!dirty) {
>                  trace_get_queued_page_not_dirty(block->idstr, (uint64_t)offset,
> -                       page, test_bit(page, block->unsentmap));
> +                       page, test_bit(page, block->bmap));

We could actually lose that last trace parameter.

Dave

>              } else {
>                  trace_get_queued_page(block->idstr, (uint64_t)offset, page);
>              }
> @@ -2619,10 +2619,6 @@ static int ram_save_host_page(RAMState *rs, PageSearchStatus *pss,
>          }
>  
>          pages += tmppages;
> -        if (pss->block->unsentmap) {
> -            clear_bit(pss->page, pss->block->unsentmap);
> -        }
> -
>          pss->page++;
>      } while ((pss->page & (pagesize_bits - 1)) &&
>               offset_in_ramblock(pss->block, pss->page << TARGET_PAGE_BITS));
> @@ -2776,8 +2772,6 @@ static void ram_save_cleanup(void *opaque)
>          block->clear_bmap = NULL;
>          g_free(block->bmap);
>          block->bmap = NULL;
> -        g_free(block->unsentmap);
> -        block->unsentmap = NULL;
>      }
>  
>      xbzrle_cleanup();
> @@ -2857,8 +2851,6 @@ void ram_postcopy_migrated_memory_release(MigrationState *ms)
>   * Returns zero on success
>   *
>   * Callback from postcopy_each_ram_send_discard for each RAMBlock
> - * Note: At this point the 'unsentmap' is the processed bitmap combined
> - *       with the dirtymap; so a '1' means it's either dirty or unsent.
>   *
>   * @ms: current migration state
>   * @block: RAMBlock to discard
> @@ -2867,17 +2859,17 @@ static int postcopy_send_discard_bm_ram(MigrationState *ms, RAMBlock *block)
>  {
>      unsigned long end = block->used_length >> TARGET_PAGE_BITS;
>      unsigned long current;
> -    unsigned long *unsentmap = block->unsentmap;
> +    unsigned long *bitmap = block->bmap;
>  
>      for (current = 0; current < end; ) {
> -        unsigned long one = find_next_bit(unsentmap, end, current);
> +        unsigned long one = find_next_bit(bitmap, end, current);
>          unsigned long zero, discard_length;
>  
>          if (one >= end) {
>              break;
>          }
>  
> -        zero = find_next_zero_bit(unsentmap, end, one + 1);
> +        zero = find_next_zero_bit(bitmap, end, one + 1);
>  
>          if (zero >= end) {
>              discard_length = end - one;
> @@ -2938,16 +2930,12 @@ static int postcopy_each_ram_send_discard(MigrationState *ms)
>   * clean, not a mix.  This function canonicalizes the bitmaps.
>   *
>   * @ms: current migration state
> - * @unsent_pass: if true we need to canonicalize partially unsent host pages
> - *               otherwise we need to canonicalize partially dirty host pages
>   * @block: block that contains the page we want to canonicalize
>   */
> -static void postcopy_chunk_hostpages_pass(MigrationState *ms, bool unsent_pass,
> -                                          RAMBlock *block)
> +static void postcopy_chunk_hostpages_pass(MigrationState *ms, RAMBlock *block)
>  {
>      RAMState *rs = ram_state;
>      unsigned long *bitmap = block->bmap;
> -    unsigned long *unsentmap = block->unsentmap;
>      unsigned int host_ratio = block->page_size / TARGET_PAGE_SIZE;
>      unsigned long pages = block->used_length >> TARGET_PAGE_BITS;
>      unsigned long run_start;
> @@ -2957,13 +2945,8 @@ static void postcopy_chunk_hostpages_pass(MigrationState *ms, bool unsent_pass,
>          return;
>      }
>  
> -    if (unsent_pass) {
> -        /* Find a sent page */
> -        run_start = find_next_zero_bit(unsentmap, pages, 0);
> -    } else {
> -        /* Find a dirty page */
> -        run_start = find_next_bit(bitmap, pages, 0);
> -    }
> +    /* Find a dirty page */
> +    run_start = find_next_bit(bitmap, pages, 0);
>  
>      while (run_start < pages) {
>  
> @@ -2973,11 +2956,7 @@ static void postcopy_chunk_hostpages_pass(MigrationState *ms, bool unsent_pass,
>           */
>          if (QEMU_IS_ALIGNED(run_start, host_ratio)) {
>              /* Find the end of this run */
> -            if (unsent_pass) {
> -                run_start = find_next_bit(unsentmap, pages, run_start + 1);
> -            } else {
> -                run_start = find_next_zero_bit(bitmap, pages, run_start + 1);
> -            }
> +            run_start = find_next_zero_bit(bitmap, pages, run_start + 1);
>              /*
>               * If the end isn't at the start of a host page, then the
>               * run doesn't finish at the end of a host page
> @@ -2994,9 +2973,6 @@ static void postcopy_chunk_hostpages_pass(MigrationState *ms, bool unsent_pass,
>              /* Clean up the bitmap */
>              for (page = fixup_start_addr;
>                   page < fixup_start_addr + host_ratio; page++) {
> -                /* All pages in this host page are now not sent */
> -                set_bit(page, unsentmap);
> -
>                  /*
>                   * Remark them as dirty, updating the count for any pages
>                   * that weren't previously dirty.
> @@ -3005,13 +2981,8 @@ static void postcopy_chunk_hostpages_pass(MigrationState *ms, bool unsent_pass,
>              }
>          }
>  
> -        if (unsent_pass) {
> -            /* Find the next sent page for the next iteration */
> -            run_start = find_next_zero_bit(unsentmap, pages, run_start);
> -        } else {
> -            /* Find the next dirty page for the next iteration */
> -            run_start = find_next_bit(bitmap, pages, run_start);
> -        }
> +        /* Find the next dirty page for the next iteration */
> +        run_start = find_next_bit(bitmap, pages, run_start);
>      }
>  }
>  
> @@ -3033,13 +3004,10 @@ static int postcopy_chunk_hostpages(MigrationState *ms, RAMBlock *block)
>  {
>      postcopy_discard_send_init(ms, block->idstr);
>  
> -    /* First pass: Discard all partially sent host pages */
> -    postcopy_chunk_hostpages_pass(ms, true, block);
>      /*
> -     * Second pass: Ensure that all partially dirty host pages are made
> -     * fully dirty.
> +     * Ensure that all partially dirty host pages are made fully dirty.
>       */
> -    postcopy_chunk_hostpages_pass(ms, false, block);
> +    postcopy_chunk_hostpages_pass(ms, block);
>  
>      postcopy_discard_send_finish(ms);
>      return 0;
> @@ -3077,19 +3045,6 @@ int ram_postcopy_send_discard_bitmap(MigrationState *ms)
>      rs->last_page = 0;
>  
>      RAMBLOCK_FOREACH_NOT_IGNORED(block) {
> -        unsigned long pages = block->used_length >> TARGET_PAGE_BITS;
> -        unsigned long *bitmap = block->bmap;
> -        unsigned long *unsentmap = block->unsentmap;
> -
> -        if (!unsentmap) {
> -            /* We don't have a safe way to resize the sentmap, so
> -             * if the bitmap was resized it will be NULL at this
> -             * point.
> -             */
> -            error_report("migration ram resized during precopy phase");
> -            rcu_read_unlock();
> -            return -EINVAL;
> -        }
>          /* Deal with TPS != HPS and huge pages */
>          ret = postcopy_chunk_hostpages(ms, block);
>          if (ret) {
> @@ -3097,12 +3052,9 @@ int ram_postcopy_send_discard_bitmap(MigrationState *ms)
>              return ret;
>          }
>  
> -        /*
> -         * Update the unsentmap to be unsentmap = unsentmap | dirty
> -         */
> -        bitmap_or(unsentmap, unsentmap, bitmap, pages);
>  #ifdef DEBUG_POSTCOPY
> -        ram_debug_dump_bitmap(unsentmap, true, pages);
> +        ram_debug_dump_bitmap(block->bmap, true,
> +                              block->used_length >> TARGET_PAGE_BITS);
>  #endif
>      }
>      trace_ram_postcopy_send_discard_bitmap();
> @@ -3270,10 +3222,6 @@ static void ram_list_init_bitmaps(void)
>              bitmap_set(block->bmap, 0, pages);
>              block->clear_bmap_shift = shift;
>              block->clear_bmap = bitmap_new(clear_bmap_size(pages, shift));
> -            if (migrate_postcopy_ram()) {
> -                block->unsentmap = bitmap_new(pages);
> -                bitmap_set(block->unsentmap, 0, pages);
> -            }
>          }
>      }
>  }
> -- 
> 2.17.1
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [PATCH 3/3] migration: remove sent parameter in get_queued_page_not_dirty

[Dr. David Alan Gilbert]

* Wei Yang (richardw.yang@linux.intel.com) wrote:
> This is a cleanup for previous removal of unsentmap.
> 
> The sent parameter is not necessary now.
>
> Signed-off-by: Wei Yang <richardw.yang@linux.intel.com>

OK, you did...


Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

> ---
>  migration/ram.c        | 2 +-
>  migration/trace-events | 2 +-
>  2 files changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/migration/ram.c b/migration/ram.c
> index 066eb4755f..a6d3d09ebd 100644
> --- a/migration/ram.c
> +++ b/migration/ram.c
> @@ -2348,7 +2348,7 @@ static bool get_queued_page(RAMState *rs, PageSearchStatus *pss)
>              dirty = test_bit(page, block->bmap);
>              if (!dirty) {
>                  trace_get_queued_page_not_dirty(block->idstr, (uint64_t)offset,
> -                       page, test_bit(page, block->bmap));
> +                                                page);
>              } else {
>                  trace_get_queued_page(block->idstr, (uint64_t)offset, page);
>              }
> diff --git a/migration/trace-events b/migration/trace-events
> index 00ffcd5930..858d415d56 100644
> --- a/migration/trace-events
> +++ b/migration/trace-events
> @@ -76,7 +76,7 @@ qemu_file_fclose(void) ""
>  
>  # ram.c
>  get_queued_page(const char *block_name, uint64_t tmp_offset, unsigned long page_abs) "%s/0x%" PRIx64 " page_abs=0x%lx"
> -get_queued_page_not_dirty(const char *block_name, uint64_t tmp_offset, unsigned long page_abs, int sent) "%s/0x%" PRIx64 " page_abs=0x%lx (sent=%d)"
> +get_queued_page_not_dirty(const char *block_name, uint64_t tmp_offset, unsigned long page_abs) "%s/0x%" PRIx64 " page_abs=0x%lx"
>  migration_bitmap_sync_start(void) ""
>  migration_bitmap_sync_end(uint64_t dirty_pages) "dirty_pages %" PRIu64
>  migration_bitmap_clear_dirty(char *str, uint64_t start, uint64_t size, unsigned long page) "rb %s start 0x%"PRIx64" size 0x%"PRIx64" page 0x%lx"
> -- 
> 2.17.1
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [PATCH 1/3] migration/postcopy: not necessary to do discard when canonicalizing bitmap

[Wei Yang]

On Tue, Sep 24, 2019 at 11:02:08AM +0100, Dr. David Alan Gilbert wrote:
>* Wei Yang (richardw.yang@linux.intel.com) wrote:
>> All pages, either partially sent or partially dirty, will be discarded in
>> postcopy_send_discard_bm_ram(), since we update the unsentmap to be
>> unsentmap = unsentmap | dirty in ram_postcopy_send_discard_bitmap().
>> 
>> This is not necessary to do discard when canonicalizing bitmap. And by
>> doing so, we separate the page discard into two individual steps:
>> 
>>   * canonicalize bitmap
>>   * discard page
>> 
>> Signed-off-by: Wei Yang <richardw.yang@linux.intel.com>
>
>Yes, I think when I originally wrote it, the set of pages that was
>discarded was different; I think it was actually the set of 
>!unsent & dirty - i.e. only pages that had been sent and then redirtied;
>it later got reworked to include unsent pages as well - so this lot can
>be simplified.
>
>

Thanks for your time :-)

>
>Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
>
>> ---
>>  migration/ram.c | 14 +-------------
>>  1 file changed, 1 insertion(+), 13 deletions(-)
>> 
>> diff --git a/migration/ram.c b/migration/ram.c
>> index 35552c090b..075ddc468c 100644
>> --- a/migration/ram.c
>> +++ b/migration/ram.c
>> @@ -2928,7 +2928,7 @@ static int postcopy_each_ram_send_discard(MigrationState *ms)
>>  }
>>  
>>  /**
>> - * postcopy_chunk_hostpages_pass: canocalize bitmap in hostpages
>> + * postcopy_chunk_hostpages_pass: canonicalize bitmap in hostpages
>>   *
>>   * Helper for postcopy_chunk_hostpages; it's called twice to
>>   * canonicalize the two bitmaps, that are similar, but one is
>> @@ -2991,18 +2991,6 @@ static void postcopy_chunk_hostpages_pass(MigrationState *ms, bool unsent_pass,
>>                                                               host_ratio);
>>              run_start = QEMU_ALIGN_UP(run_start, host_ratio);
>>  
>> -            /* Tell the destination to discard this page */
>> -            if (unsent_pass || !test_bit(fixup_start_addr, unsentmap)) {
>> -                /* For the unsent_pass we:
>> -                 *     discard partially sent pages
>> -                 * For the !unsent_pass (dirty) we:
>> -                 *     discard partially dirty pages that were sent
>> -                 *     (any partially sent pages were already discarded
>> -                 *     by the previous unsent_pass)
>> -                 */
>> -                postcopy_discard_send_range(ms, fixup_start_addr, host_ratio);
>> -            }
>> -
>>              /* Clean up the bitmap */
>>              for (page = fixup_start_addr;
>>                   page < fixup_start_addr + host_ratio; page++) {
>> -- 
>> 2.17.1
>> 
>--
>Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

-- 
Wei Yang
Help you, Help me

Re: [Qemu-devel] [PATCH 0/3] migration/postcopy: unsentmap is not necessary

[Dr. David Alan Gilbert]

* Wei Yang (richardw.yang@linux.intel.com) wrote:
> Three patches to cleanup postcopy:
> 
> [1]: split canonicalize bitmap and discard page
> [2]: remove unsentmap since it is not necessary
> [3]: cleanup the get_queued_page_not_dirty

Queued

> 
> Wei Yang (3):
>   migration/postcopy: not necessary to do discard when canonicalizing
>     bitmap
>   migration/postcopy: unsentmap is not necessary for postcopy
>   migration: remove sent parameter in get_queued_page_not_dirty
> 
>  include/exec/ram_addr.h |  6 ---
>  migration/ram.c         | 94 +++++++----------------------------------
>  migration/trace-events  |  2 +-
>  3 files changed, 16 insertions(+), 86 deletions(-)
> 
> -- 
> 2.17.1
> 
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK