From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:46662) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1X4ATm-00006g-1M for qemu-devel@nongnu.org; Mon, 07 Jul 2014 11:05:25 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1X4ATe-00051h-I8 for qemu-devel@nongnu.org; Mon, 07 Jul 2014 11:05:17 -0400 Received: from mx1.redhat.com ([209.132.183.28]:44226) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1X4ATe-00051a-9F for qemu-devel@nongnu.org; Mon, 07 Jul 2014 11:05:10 -0400 Date: Mon, 7 Jul 2014 17:04:59 +0200 From: "Dr. David Alan Gilbert" Message-ID: <20140707150459.GD3443@work-vm> References: <1404495717-4239-1-git-send-email-dgilbert@redhat.com> <1404495717-4239-16-git-send-email-dgilbert@redhat.com> <53B7D2B8.7000002@redhat.com> <20140707143539.GB3443@work-vm> <53BAB466.8060609@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <53BAB466.8060609@redhat.com> Subject: Re: [Qemu-devel] [PATCH 15/46] Rework loadvm path for subloops List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Paolo Bonzini Cc: aarcange@redhat.com, yamahata@private.email.ne.jp, quintela@redhat.com, qemu-devel@nongnu.org, lilei@linux.vnet.ibm.com * Paolo Bonzini (pbonzini@redhat.com) wrote: > Il 07/07/2014 16:35, Dr. David Alan Gilbert ha scritto: > >* Paolo Bonzini (pbonzini@redhat.com) wrote: > >>Il 04/07/2014 19:41, Dr. David Alan Gilbert (git) ha scritto: > >>>From: "Dr. David Alan Gilbert" > >>> > >>>Postcopy needs to have two migration streams loading concurrently; > >>>one from memory (with the device state) and the other from the fd > >>>with the memory transactions. > >> > >>Can you explain this? > >> > >>I would have though the order is > >> > >> precopy RAM and everything > >> prepare postcopy RAM ("sent && dirty" bitmap) > >> finish precopy non-RAM > >> finish devices > >> postcopy RAM > >> > >>Why do you need to have all the packaging stuff and a separate memory-based > >>migration stream for devices? I'm sure I'm missing something. :) > > > >The thing you're missing is the details of 'finish devices'. > >The device emulation may access guest memory as part of loading it's > >state, so you can't successfully complete 'finish devices' without > >having the 'postcopy RAM' available to provide pages. > > I see. Can you document the flow (preferrably as a reply to this email > _and_ in docs/ when you send v2 of the code :))? Yep, will do; I need to go through and check through it before I write the detail reply. Dave > > From my cursory read of the code it is something like this on the source: > > finish precopy non-RAM > start RAM postcopy > for each device > pack up data > send it to destination > > and on the destination: > > while source sends packet > pick up packet atomically > pass the packet to device loader > (while the loader works, userfaultfd does background magic) > > But something is missing still, either some kind of ack is needed between > device data sends or userfaultfd needs to be able to process device data > packets. > > Paolo > > >Thus you need to be able to start up 'postcopy RAM' before 'finish devices' > >has completed, and you can't do that if 'finish devices' is still stuffing > >data down the fd. > > > >Now, if hypothetically you had: > > 1) A migration format that let you separate out device state so that you > >could load all the state of the device off the fd without calling the device > >IO code. > > 2) All devices were good and didn't touch guest memory while loading their > >state. > > > >then you could avoid this complexity. However, if you look at how Stefan's > >BER code tried to do 1 (which I don't do in my way of doing it), it was by > >using the same trick of stuffing the device data into a dummy memory file > >to find out the size of the data. And I'm not convinced (2) will happen > >this century. > > > >>Paolo > > > >Dave > >-- > >Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK > > > -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK