All of lore.kernel.org
 help / color / mirror / Atom feed
From: Juan Quintela <quintela@redhat.com>
To: qemu-devel@nongnu.org
Cc: dgilbert@redhat.com, lvivier@redhat.com, peterx@redhat.com
Subject: Re: [Qemu-devel] [PATCH v12 00/21] Multifd
Date: Wed, 25 Apr 2018 13:44:51 +0200	[thread overview]
Message-ID: <87sh7jy0gc.fsf@secure.mitica> (raw)
In-Reply-To: <20180425112723.1111-1-quintela@redhat.com> (Juan Quintela's message of "Wed, 25 Apr 2018 13:27:02 +0200")

Juan Quintela <quintela@redhat.com> wrote:
> Hi
>
>
> [v12]
>
> Big news, it is not RFC anymore, it works reliabely for me.
>
> Changes:
> - Locknig changed completely (several times)
> - We now send  all pages through the channels.  In a 2GB guest with 1 disk and a network card, the amount of data send for RAM was 80KB.
> - This is not optimized yet, but it shouws clear improvements over precopy.  testing over localhost networking I can guet:
>   - 2 VCPUs guest
>   - 2GB RAM
>   - runn stress --vm 4 --vm 500GB (i.e. dirtying 2GB or RAM each second)
>
>   - Total time: precopy ~50seconds, multifd  around 11seconds
>   - Bandwidth usage is around 273MB/s vs 71MB/s on the same hardware
>
> This is very preleminary testing, will send more numbers when I got them.  But looks promissing.
>
> Things that will be improved later:
> - Initial synchronization is too slow (around 1s)
> - We synchronize all threads after each RAM section, we can move to only
>   synchronize them after we have done a bitmap syncrhronization
> - We can improve bitmap walking (but that is independent of multifd)

I forgot to put there that on the last 4 patches, I have not been able
to split them in a way that:
- is logical for review
- works for multifd tests in all versions

So, I ended trynig to get the "logical" viewe, and it works after the
last patch.  Why is that?
- Before I am able to transmit data, I need to be able to
  end/synchronize the different channels
- To finish channels in case of error, I just close the channels
  But I can't opet then yet.

I have to think if I can come with a simpler way to split it, but you
can also consider that the  last 3-4 patches are a single one.

Later, Juan.

  parent reply	other threads:[~2018-04-25 11:45 UTC|newest]

Thread overview: 60+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-04-25 11:27 [Qemu-devel] [PATCH v12 00/21] Multifd Juan Quintela
2018-04-25 11:27 ` [Qemu-devel] [PATCH v12 01/21] migration: Set error state in case of error Juan Quintela
2018-05-02 15:53   ` Dr. David Alan Gilbert
2018-05-09  8:15     ` Juan Quintela
2018-04-25 11:27 ` [Qemu-devel] [PATCH v12 02/21] migration: Introduce multifd_recv_new_channel() Juan Quintela
2018-04-25 11:27 ` [Qemu-devel] [PATCH v12 03/21] migration: terminate_* can be called for other threads Juan Quintela
2018-04-25 11:27 ` [Qemu-devel] [PATCH v12 04/21] migration: Be sure all recv channels are created Juan Quintela
2018-04-25 11:27 ` [Qemu-devel] [PATCH v12 05/21] migration: Export functions to create send channels Juan Quintela
2018-04-26  7:28   ` Peter Xu
2018-05-09  8:05     ` Juan Quintela
2018-04-25 11:27 ` [Qemu-devel] [PATCH v12 06/21] migration: Create multifd channels Juan Quintela
2018-04-25 11:27 ` [Qemu-devel] [PATCH v12 07/21] migration: Delay start of migration main routines Juan Quintela
2018-04-25 11:27 ` [Qemu-devel] [PATCH v12 08/21] migration: Transmit initial package through the multifd channels Juan Quintela
2018-05-02 17:19   ` Dr. David Alan Gilbert
2018-05-09  8:34     ` Juan Quintela
2018-04-25 11:27 ` [Qemu-devel] [PATCH v12 09/21] migration: Define MultifdRecvParams sooner Juan Quintela
2018-05-02 17:32   ` Dr. David Alan Gilbert
2018-04-25 11:27 ` [Qemu-devel] [PATCH v12 10/21] migration: Create multipage support Juan Quintela
2018-04-26  7:15   ` Peter Xu
2018-05-09 10:52     ` Juan Quintela
2018-05-02 17:52   ` Dr. David Alan Gilbert
2018-05-09 10:53     ` Juan Quintela
2018-04-25 11:27 ` [Qemu-devel] [PATCH v12 11/21] migration: Create multifd packet Juan Quintela
2018-05-02 18:04   ` Dr. David Alan Gilbert
2018-05-09 11:09     ` Juan Quintela
2018-05-09 11:12       ` Dr. David Alan Gilbert
2018-05-09 19:46         ` Juan Quintela
2018-05-11 16:36           ` Dr. David Alan Gilbert
2018-04-25 11:27 ` [Qemu-devel] [PATCH v12 12/21] migration: Add multifd traces for start/end thread Juan Quintela
2018-05-02 18:35   ` Dr. David Alan Gilbert
2018-05-09 11:11     ` Juan Quintela
2018-04-25 11:27 ` [Qemu-devel] [PATCH v12 13/21] migration: Calculate transferred ram correctly Juan Quintela
2018-05-02 18:59   ` Dr. David Alan Gilbert
2018-05-09 11:14     ` Juan Quintela
2018-05-09 19:46     ` Juan Quintela
2018-04-25 11:27 ` [Qemu-devel] [PATCH v12 14/21] migration: Multifd channels always wait on the sem Juan Quintela
2018-05-03  9:36   ` Dr. David Alan Gilbert
2018-05-23 10:59     ` Juan Quintela
2018-04-25 11:27 ` [Qemu-devel] [PATCH v12 15/21] migration: Add block where to send/receive packets Juan Quintela
2018-05-03 10:03   ` Dr. David Alan Gilbert
2018-04-25 11:27 ` [Qemu-devel] [PATCH v12 16/21] migration: Synchronize multifd threads with main thread Juan Quintela
2018-05-03 10:44   ` Dr. David Alan Gilbert
2018-05-09 19:45     ` Juan Quintela
2018-05-11 16:32       ` Dr. David Alan Gilbert
2018-04-25 11:27 ` [Qemu-devel] [PATCH v12 17/21] migration: Create ram_multifd_page Juan Quintela
2018-04-26  7:43   ` Peter Xu
2018-04-26  8:18   ` Peter Xu
2018-05-03 11:30     ` Dr. David Alan Gilbert
2018-05-23 11:13     ` Juan Quintela
2018-04-25 11:27 ` [Qemu-devel] [PATCH v12 18/21] migration: Start sending messages Juan Quintela
2018-05-03 14:55   ` Dr. David Alan Gilbert
2018-05-23 10:51     ` Juan Quintela
2018-04-25 11:27 ` [Qemu-devel] [PATCH v12 19/21] migration: Wait for blocking IO Juan Quintela
2018-05-03 15:04   ` Dr. David Alan Gilbert
2018-04-25 11:27 ` [Qemu-devel] [PATCH v12 20/21] migration: Remove not needed semaphore and quit Juan Quintela
2018-04-25 11:27 ` [Qemu-devel] [PATCH v12 21/21] migration: Stop sending whole pages through main channel Juan Quintela
2018-05-03 15:24   ` Dr. David Alan Gilbert
2018-04-25 11:44 ` Juan Quintela [this message]
2018-05-03 15:32   ` [Qemu-devel] [PATCH v12 00/21] Multifd Dr. David Alan Gilbert
2018-04-26  8:28 ` Peter Xu

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87sh7jy0gc.fsf@secure.mitica \
    --to=quintela@redhat.com \
    --cc=dgilbert@redhat.com \
    --cc=lvivier@redhat.com \
    --cc=peterx@redhat.com \
    --cc=qemu-devel@nongnu.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.