[PATCH 0/2] not use multifd during postcopy

[PATCH 0/2] not use multifd during postcopy

[Wei Yang]

We don't support multifd during postcopy, but user still could enable
both multifd and postcopy. This leads to migration failure.

Patch 1 does proper cleanup, otherwise we may have data corruption.
Patch 2 does the main job.

BTW, current multifd synchronization method needs a cleanup. Will send another
patch set.

Wei Yang (2):
  migration/multifd: clean pages after filling packet
  migration/multifd: not use multifd during postcopy

 migration/ram.c | 15 ++++++++++-----
 1 file changed, 10 insertions(+), 5 deletions(-)

-- 
2.17.1

[PATCH 1/2] migration/multifd: clean pages after filling packet

[Wei Yang]

This is a preparation for the next patch:

    not use multifd during postcopy.

Without enabling postcopy, everything looks good. While after enabling
postcopy, migration may fail even not use multifd during postcopy. The
reason is the pages is not properly cleared and *old* target page will
continue to be transferred.

After clean pages, migration succeeds.

Signed-off-by: Wei Yang <richardw.yang@linux.intel.com>
---
 migration/ram.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/migration/ram.c b/migration/ram.c
index 80dd2d55f9..7087bb73ed 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -929,10 +929,10 @@ static int multifd_send_pages(RAMState *rs)
         }
         qemu_mutex_unlock(&p->mutex);
     }
-    p->pages->used = 0;
+    assert(!p->pages->used);
+    assert(!p->pages->block);
 
     p->packet_num = multifd_send_state->packet_num++;
-    p->pages->block = NULL;
     multifd_send_state->pages = p->pages;
     p->pages = pages;
     transferred = ((uint64_t) pages->used) * TARGET_PAGE_SIZE + p->packet_len;
@@ -1114,6 +1114,8 @@ static void *multifd_send_thread(void *opaque)
             p->flags = 0;
             p->num_packets++;
             p->num_pages += used;
+            p->pages->used = 0;
+            p->pages->block = NULL;
             qemu_mutex_unlock(&p->mutex);
 
             trace_multifd_send(p->id, packet_num, used, flags,
-- 
2.17.1

[PATCH 2/2] migration/multifd: not use multifd during postcopy

[Wei Yang]

We don't support multifd during postcopy, but user still could enable
both multifd and postcopy. This leads to migration failure.

Skip multifd during postcopy.

Signed-off-by: Wei Yang <richardw.yang@linux.intel.com>
---
 migration/ram.c | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/migration/ram.c b/migration/ram.c
index 7087bb73ed..5876054195 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -2547,10 +2547,13 @@ static int ram_save_target_page(RAMState *rs, PageSearchStatus *pss,
     }
 
     /*
-     * do not use multifd for compression as the first page in the new
-     * block should be posted out before sending the compressed page
+     * Do not use multifd for:
+     * 1. Compression as the first page in the new block should be posted out
+     *    before sending the compressed page
+     * 2. In postcopy as one whole host page should be placed
      */
-    if (!save_page_use_compression(rs) && migrate_use_multifd()) {
+    if (!save_page_use_compression(rs) && migrate_use_multifd()
+        && !migration_in_postcopy()) {
         return ram_save_multifd_page(rs, block, offset);
     }
 
-- 
2.17.1

Re: [PATCH 0/2] not use multifd during postcopy

[Wei Yang]

Ping for comments.

On Sat, Oct 26, 2019 at 07:19:58AM +0800, Wei Yang wrote:
>We don't support multifd during postcopy, but user still could enable
>both multifd and postcopy. This leads to migration failure.
>
>Patch 1 does proper cleanup, otherwise we may have data corruption.
>Patch 2 does the main job.
>
>BTW, current multifd synchronization method needs a cleanup. Will send another
>patch set.
>
>Wei Yang (2):
>  migration/multifd: clean pages after filling packet
>  migration/multifd: not use multifd during postcopy
>
> migration/ram.c | 15 ++++++++++-----
> 1 file changed, 10 insertions(+), 5 deletions(-)
>
>-- 
>2.17.1

-- 
Wei Yang
Help you, Help me

Re: [PATCH 1/2] migration/multifd: clean pages after filling packet

[Juan Quintela]

Wei Yang <richardw.yang@linux.intel.com> wrote:
> This is a preparation for the next patch:
>
>     not use multifd during postcopy.
>
> Without enabling postcopy, everything looks good. While after enabling
> postcopy, migration may fail even not use multifd during postcopy. The
> reason is the pages is not properly cleared and *old* target page will
> continue to be transferred.
>
> After clean pages, migration succeeds.
>
> Signed-off-by: Wei Yang <richardw.yang@linux.intel.com>

Reviewed-by: Juan Quintela <quintela@redhat.com>

I don't like asserts, but I understand why you put them there.

Re: [PATCH 2/2] migration/multifd: not use multifd during postcopy

[Juan Quintela]

Wei Yang <richardw.yang@linux.intel.com> wrote:
> We don't support multifd during postcopy, but user still could enable
> both multifd and postcopy. This leads to migration failure.
>
> Skip multifd during postcopy.
>
> Signed-off-by: Wei Yang <richardw.yang@linux.intel.com>

Reviewed-by: Juan Quintela <quintela@redhat.com>

I was working in a different implementation, but I agree with the idea.

My patch series try to decide during negotiation if multifd + everything
else is setup or not.

Thanks, Juan.

Re: [PATCH 2/2] migration/multifd: not use multifd during postcopy

[Wei Yang]

On Tue, Nov 19, 2019 at 11:55:52AM +0100, Juan Quintela wrote:
>Wei Yang <richardw.yang@linux.intel.com> wrote:
>> We don't support multifd during postcopy, but user still could enable
>> both multifd and postcopy. This leads to migration failure.
>>
>> Skip multifd during postcopy.
>>
>> Signed-off-by: Wei Yang <richardw.yang@linux.intel.com>
>
>Reviewed-by: Juan Quintela <quintela@redhat.com>
>
>I was working in a different implementation, but I agree with the idea.
>
>My patch series try to decide during negotiation if multifd + everything
>else is setup or not.
>

Look forward your approach :-)

>Thanks, Juan.

-- 
Wei Yang
Help you, Help me

Re: [PATCH 0/2] not use multifd during postcopy

[Wei Yang]

Would this one be picked up this time?

On Sat, Oct 26, 2019 at 07:19:58AM +0800, Wei Yang wrote:
>We don't support multifd during postcopy, but user still could enable
>both multifd and postcopy. This leads to migration failure.
>
>Patch 1 does proper cleanup, otherwise we may have data corruption.
>Patch 2 does the main job.
>
>BTW, current multifd synchronization method needs a cleanup. Will send another
>patch set.
>
>Wei Yang (2):
>  migration/multifd: clean pages after filling packet
>  migration/multifd: not use multifd during postcopy
>
> migration/ram.c | 15 ++++++++++-----
> 1 file changed, 10 insertions(+), 5 deletions(-)
>
>-- 
>2.17.1

-- 
Wei Yang
Help you, Help me

Re: [PATCH 0/2] not use multifd during postcopy

[Wei Yang]

On Mon, Dec 16, 2019 at 10:35:39AM +0800, Wei Yang wrote:
>Would this one be picked up this time?

Happy new year to all.

Can I ask the plan for this patch set?

>
>On Sat, Oct 26, 2019 at 07:19:58AM +0800, Wei Yang wrote:
>>We don't support multifd during postcopy, but user still could enable
>>both multifd and postcopy. This leads to migration failure.
>>
>>Patch 1 does proper cleanup, otherwise we may have data corruption.
>>Patch 2 does the main job.
>>
>>BTW, current multifd synchronization method needs a cleanup. Will send another
>>patch set.
>>
>>Wei Yang (2):
>>  migration/multifd: clean pages after filling packet
>>  migration/multifd: not use multifd during postcopy
>>
>> migration/ram.c | 15 ++++++++++-----
>> 1 file changed, 10 insertions(+), 5 deletions(-)
>>
>>-- 
>>2.17.1
>
>-- 
>Wei Yang
>Help you, Help me

-- 
Wei Yang
Help you, Help me

Re: [PATCH 0/2] not use multifd during postcopy

[Juan Quintela]

Wei Yang <richardw.yang@linux.intel.com> wrote:
> On Mon, Dec 16, 2019 at 10:35:39AM +0800, Wei Yang wrote:
>>Would this one be picked up this time?
>
> Happy new year to all.
>
> Can I ask the plan for this patch set?

queued

>
>>
>>On Sat, Oct 26, 2019 at 07:19:58AM +0800, Wei Yang wrote:
>>>We don't support multifd during postcopy, but user still could enable
>>>both multifd and postcopy. This leads to migration failure.
>>>
>>>Patch 1 does proper cleanup, otherwise we may have data corruption.
>>>Patch 2 does the main job.
>>>
>>>BTW, current multifd synchronization method needs a cleanup. Will send another
>>>patch set.
>>>
>>>Wei Yang (2):
>>>  migration/multifd: clean pages after filling packet
>>>  migration/multifd: not use multifd during postcopy
>>>
>>> migration/ram.c | 15 ++++++++++-----
>>> 1 file changed, 10 insertions(+), 5 deletions(-)
>>>
>>>-- 
>>>2.17.1
>>
>>-- 
>>Wei Yang
>>Help you, Help me

Re: [PATCH 0/2] not use multifd during postcopy

[Wei Yang]

On Thu, Jan 09, 2020 at 10:50:25AM +0100, Juan Quintela wrote:
>Wei Yang <richardw.yang@linux.intel.com> wrote:
>> On Mon, Dec 16, 2019 at 10:35:39AM +0800, Wei Yang wrote:
>>>Would this one be picked up this time?
>>
>> Happy new year to all.
>>
>> Can I ask the plan for this patch set?
>
>queued
>

Thanks :-)

-- 
Wei Yang
Help you, Help me